Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Grasping

Understand

Object segmentation for robotic grasping under dynamic conditions often faces challenges such as occlusion, low light conditions, motion blur and object size variance.

• To address these challenges, we propose a Deep Learning network that fuses two types of visual signals, event-based data and RGB frame data.
• The proposed Bimodal SegNet network has two distinct encoders, one for each signal input and a spatial pyramidal pooling with atrous convolutions.
• Encoders capture rich contextual information by pooling the concatenated features at different resolutions while the decoder obtains sharp object boundaries.