Enhancing automatic landmark localization in X-ray images using combined segmentation and regression models: application to lower limb alignment assessment

Abstract

Manual landmark detection in lower limb medical imaging is time-consuming and error-prone. Recently, authors have proposed automatic landmark detection methods based on image segmentation, coordinate regression, or a combination of both to aid clinicians. While the latter approach shows promising results, detailed optimization of its design choices, including the integration strategy and hyperparameter tuning, remains unexplored. This study investigates the optimal approach to combining image segmentation and coordinate regression, focusing on selecting suitable network architectures and optimizing their configurations. We contrasted two methods for training the network: end-to-end training and cascading the subnetworks, and assessed the optimal architecture for each strategy. For landmark segmentation, we compared U-Net and Swin-UNETR models, and for coordinate regression, we assessed VGG-16, ResNet-50, and Swin-B. Performance was evaluated in detecting eight landmarks in each leg of a complete lower-limb X-ray and in examining their influence on the clinical task of measuring lower-limb malalignment. Swin-UNETR slightly outperformed U-Net, with a lower Euclidean distance error (1.74 mm [0.98 mm] versus 1.75 mm [1.06 mm]) and significantly fewer false positives (160 vs. 502). For coordinate regression, VGG-16 in the end-to-end configuration achieved the highest accuracy (2.44 mm [2.12 mm]) and proved optimal for lower limb alignment assessment, with 97.88% of estimations falling within of the hip–knee–ankle angle ground truth. Combining Swin-UNETR with VGG-16 within an end-to-end framework yields the most accurate and robust performance for automated lower-limb landmark detection and alignment assessment.