Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception

Accurate 3D object detection enables service robots to have 3D scene perception in cluttered indoor environments, but it’s usually a challenging task. In this paper, we proposed a two-stage 3D object detection algorithm which is to fuse multiple views of 3D object point clouds in the first stage and to eliminate unreasonable and intersection detections in the second stage. For each view, 3D object bounding box estimation has four steps: (1) 2D object semantic segmentation and 3D object point clouds reconstruction; (2) using Locally Convex Connected Patches (LCCP) method to segment the object from the background; (3) calculating the main object orientation with Manhattan Frame estimation method; (4) constructing 3D object bounding box. An object database is created and refined as more multi-view point clouds of the same object are fused. Incorrect and intersecting objects are removed from the object database based on prior knowledge. Experiments performed on both SceneNN dataset and a real indoor environment show the high accuracy and stability of our proposed method.

Recommended citation: Wang, L.; Li, R.; Sun, J.; Liu, X.; Zhao, L.; Seah, H.S.; Quah, C.K.; Tandianus, B. Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception. Sensors 2019, 19, 4092. https://doi.org/10.3390/s19194092
Download Paper