Cross-Modality Learning in Ophthalmology: Is There a Need for Increasing Variety in Data?

Abstract

The primary focus of our work extends beyond merely enhancing state-of-the-art predictive performance in cross-modal classification tasks. We aim to demonstrate, through AI, the critical necessity of maintaining the current industrial investment in multi-modalities that are complex, costly, and cumbersome in day-to-day clinical usage. To this end, we first analyzed the prediction accuracy gap between single and multi-modalities models. We then assessed whether the increased complexity of multi-modal predictors demands larger datasets compared to their single-modal counterparts. Finally, we explored whether leveraging multi-modal inputs can compensate for poor-quality images while still outperforming uni-modal approaches.