Physically interpretable machine learning for robust façade bio-colonisation detection: Comparative insights from hyperspectral and RGB imaging under hard visual interference

Abstract

Reliable detection of bio-colonisation on heritage building façades is essential for preventive conservation but remains challenging due to complex visual interferences, such as shadows, moisture stains, and pollution. While RGB-based approaches and deep learning are increasingly adopted, their reliance on colour features often limits robustness in variable illumination conditions. This study proposes a physically interpretable machine learning framework combining Hyperspectral Imaging (HSI) and eXtreme Gradient Boosting (XGBoost) for automated bio-colonisation mapping. Using a medieval castle in Belgium as a case study, a 3D spectral façade pointcloud model is constructed, and RGB, HSI, and fused RGB–HSI approaches are systematically compared. Results show that RGB models attain a high F1-score on clean surfaces (F1 ≈ 97%), yet suffer from a high pixel-level False Positive Rate (FPR ≈ 73%) on visually unclean areas, where colour-based features overlap with mineral backgrounds. In contrast, HSI models maintain robust discrimination (FPR ≈ 2.2%) by exploiting the chlorophyll-related red-edge spectral signature. A spectral efficiency analysis identifies a critical performance leap at the 4-band multispectral (MSI) level, where the FPR drops from 8.8% to 2.6%, while full 31-band HSI provides the ultimate refinement for high-precision diagnostics. Furthermore, incorporating hard negatives during training proves decisive, reducing the fusion model’s FPR from 60% to 1.9%. Sensitivity analysis reveals that exhaustive annotation is unnecessary; a small, representative dataset (e.g., ≈20,000 pixels) ensures stable detection and minimises manual effort. The proposed framework enables quantitative, spectrally grounded condition assessment for heritage façades.