JawSense: Recognizing Unvoiced Sound using a Low-cost Ear-worn System

This paper explores a new wearable system, called JawSense, that enables a novel form of human-computer interaction based on unvoiced jaw movement tracking. JawSense allows its user to interact with computing machine just by moving their jaw. We study the neurological and anatomical structure of the human cheek and jaw to design JawSense so that jaw movement can be reliably captured under the strong impact of noises from human artifacts. In particular, JawSense senses the muscle deformation and vibration caused by unvoiced speaking to decode the unvoiced phonemes spoken by the user. We model the relationship between jaw movements and phonemes to develop a classification algorithm to recognize nine phonemes. Through a prototyping implementation and evaluation with six subjects, we show that JawSense can be used as a form of hands-free and privacy-preserving human-computer interaction with 92% phoneme classification rate.