Evaluation Metrics For Food Intake Activity Recognition Using Segment-wise IoU

Abstract

AI-assisted food intake monitoring systems have drawn considerable attention from researchers. To date, various approaches have been proposed to objectively and unobtrusively detect food intake activities by utilizing novel sensors and machine learning techniques. In the development of automated food intake monitoring systems, one crucial step is to evaluate the generated results from machine learning models. In this study, we illustrate the challenge arising from the inefficiency of traditional sliding-window-based evaluation in translating results into clinical indices (i.e. number of bites). Additionally, existing evaluation metrics only focus on detection performance (count the occurrence of eating gestures); however, the segmentation performance (temporal boundary of eating gesture) is missed, which is also a clinically meaningful index. Apart from the discussion of existing evaluation methods in food intake monitoring, we introduce the segment-wise evaluation scheme using the Intersection Over Union (IoU) as threshold to assess performance. This method facilitates the evaluation of both the detection and segmentation performance of eating activities. Two public food intake datasets are used in our case study to illustrate that the segment-wise method can yield more detailed information and a more comprehensive evaluation when compared to existing metrics. The proposed evaluation scheme has the potential to be applied to other human activity recognition (HAR) cases.