Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection

Understand

We propose a new method for fusing a LIDAR point cloud and camera-captured images in the deep convolutional neural network (CNN).

• The proposed method constructs a new layer called non-homogeneous pooling layer to transform features between bird view map and front view map.
• The sparse LIDAR point cloud is used to construct the mapping between the two maps.
• The pooling layer allows efficient fusion of the bird view and front view features at any stage of the network.