Touchspeaker, a multi-sensor context-aware application for mobile devices: from application to implementation

Tapping with your finger on any place on your mobile device is a promising candidate for enhanced interaction between users and their mobile device. So far the touchscreen and the accelerometer are commonly used to infer finger tap events. However, the touchscreen consumes a significant amount of power and is not always accessible (i.e., when the device is used as a running assistant). The accelerometer can be power efficient but can’t differentiate well between a variety of contexts and positions. To address these limitations, we present TouchSpeaker, a novel technique for finger tap detection on mobile devices using the built-in speakers as primary sensors. We show that a combination of the speakers with other built-in sensors can distinguish between 9 different tap events with an accuracy of 98.3%, outperforming the state of the art. In addition, a robust version is implemented resulting in a false positive rate below 1%. For power constrained devices, we propose a configuration consisting of only the speakers and the accelerometer, achieving an accuracy of 95.2%.