The hawk eye scan: Halyomorpha halys detection relying on aerial tele photos and neural networks

Abstract

This paper faces the challenge of monitoring the Brown Marmorated Stink Bug (H.halys) ( Halyomorpha halys) ) within orchards, utilizing drones, and computer vision. H.halys is an invasive species originating from East Asia, that is extremely polyphagous and poses a significant threat to various crops. Our first contribution is a drone navigation protocol, which ensures risk-free drone flights in cluttered orchard environments, preserving image quality and avoiding obstacles. We then create a pioneering H.halys dataset consisting of aerial telephotos captured in the field autonomously by the drone. The dataset allows the development and evaluation for the first time of multiple ML models for H.halys detection in the field. We trained YOLOv5, YOLOv8, RETINANET, and FASTER-RCNN models using different learning methodologies, exploiting different percentages of images without the bug, and using different slicing procedures for the images. The Medium YOLOv5 model trained with all images containing a bug detects the largest number of H.halys on the testing set and overall performs the best, while RETINANET and FASTER-RCNN provide the best trade-off between precision and recall. Models vary in their ability to handle occluded H.halys and bug-free images, which are common since the presence of the bug cannot be predicted before capturing a photo. These results show promising potential for automating H.halys monitoring, despite the image complexity and the early dataset stage. Our work marks a significant step towards enhancing smart agriculture practices due to the simplicity of the data acquisition process and the off-the-shelf hardware selection.