Perception System based on Cooperative Fusion of Lidar and Cameras

Abstract

This paper proposes a novel sensor fusion method capable of detection and tracking of road users under nominal as well as in border cases of system operation. The proposed method is based on a sensor-agnostic Bayesian late fusion framework, augmented with an optional exchange of detector activation information between sensors, referred to as cooperative feedback. Experimental evaluation confirms that we obtain competitive detection and tracking performance in normal operation. The main benefit of the proposed method is in cases of sensor failure where, due to the probabilistic modeling, we observed significant improvements of both detection and tracking accuracy over the state of the art.