Selection of image classifiers for noisy images through metalearning

Abstract

Machine Learning (ML) models are being deployed in numerous applications, ranging from autonomous vehicles to robotics. However, the performance of these ML models is highly dependent on situations within the application, such as noisy inputs or distorted images. Dealing with these different situations is complex, as the 'no free lunch' theorem states there is no single ML model that is able to achieve high performances in all situations. Selecting the model that is most suitable for a given context would therefore be beneficial. Although metalearning-based Algorithm Selection (AS) has proven to be an effective way of selecting the most suitable ML model for complete datasets, limited research has been done regarding dealing with dynamic data samples. This is especially important for noisy data samples, as noise highly affects the performance of ML models, such as image classifiers. Employing metalearning-based ML for selecting the best image classifier to classify the given noisy images has not been done before. Therefore, this work presents a methodology for carrying out this selection through metalearning. Based on the learned relationship between classification performance and the context of the noisy image, our AS strategy is able to select the most optimal image classifier for a specific situation. We compare different metamodels that carry out this selection, while we also compare the use of regression-based and classification-based selection. In our results, we show that there is a clear benefit in exploiting the robustness of different classifiers on the CIFAR-100 dataset via AS, as the per-class and overall classification accuracies are increased. We also compare our metalearing-based selection with random selection of image classifiers to show the significance of our AS methodology. Hence, this research enables future works to select the most suitable image classifying AS given the image context and noise distribution parameters.