Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception

In this project, we achieved accurate indoor 3D object detection with a deep learning based method. To increase detection accuracy, we utilized both 2D RGB image and 3D point cloud in our segmentation algorithm. We first used Mask R-CNN to get 2D object bounding boxes and segmentation masks from RGB images, then located objects in 3D point clouds based on 2D detection output. Refinement of 3D object bounding boxes was achieved by point cloud filtering, bounding box orientation adjustment, object prior knowledge reference. This study can help service robots to move in cluttered indoor environments.