Intake Gesture Detection With IMU Sensor in Free-Living Environments: The Effects of Measuring Two-Hand Intake and Down-Sampling

Abstract

Food intake monitoring plays an important role in personal dietary systems. Numerous approaches have been proposed to automatically detect eating gestures using various sensors and machine learning. However, existing eating gesture detection approaches mainly focus on meal sessions. Such a task is still challenging in free-living environments due to longer monitoring duration and more non-feeding activities. This paper proposes a wearable Inertial Measurement Unit (IMU) based method to detect eating and drinking gestures in free-living environments. Two important factors that impede intake gesture detection in free-living environments are addressed: 1) how to handle IMU data from two hands, and 2) what is the impact of downsampling sensor data on performance. To integrate two-hand data, we propose a solution that combines hand mirroring and temporal concatenation techniques. The multi-stage temporal convolutional network (MS-TCN) is applied to effectively recognise intake gestures. A dataset contains 12 subjects with 67.5 h data is collected for validation. Moreover, IMU data with different sampling frequencies are processed to test performance. Validated by Leave-One-Subject-Out (LOSO) method, our approach (with 16 Hz sampling frequency) achieves a segmental F1-score of 0.826 and 0.893 for recognizing eating and drinking gestures, respectively. Results show that the proposed solution outperforms existing two-hand data combination approaches. Moreover, in our case, a higher sampling frequency does not always mean better performance.