Joint probabilistic data fusion for pedestrian detection in multimodal images

Abstract

Pedestrian detection systems are one of the emerging technologies due to their wide range of applications. However, due to the dynamic environments, the data obtained from a single visual sensor for these operations is not sufficient to cover all the environmental conditions, such as varying natural light, weather conditions, etc. Therefore, the use of multiple heterogeneous visual sensors for such systems is indispensable. However, this introduces another challenge of fusing diverse data from multiple visual sensors needing precise geometric alignment. This paper focuses on late fusion based on colour and thermal images using a probabilistic approach conditioned on the environmental variables affecting the sensor data, using off-the-shelf pedestrian detectors. This eliminates the need to train detectors on large annotated dataset and provides the way to deal with the situation when one of the sensor starts degrading or is unavailable. The results in this paper outperform state-of-the-art fusion methods by improving the overall accuracy of the system and providing adaptability when one sensor is unavailable.