ASSETS 2015 Doctoral Consortium Project Summary: Reducing Viseme Confusion In Speech-Reading

Benjamin M. Gorman

Proceedings of the 17th international ACM SIGACCESS conference on Computers & accessibility


Speech-reading is an invaluable technique for people with hearing impairments or those listening in adverse listening conditions (e.g., in a noisy restaurant, near children playing loudly). However, speech-reading is often difficult because identical mouth shapes (visemes) can produce several speech sounds (phonemes); there is a one-to-many mapping from visemes to phonemes. This decreases comprehension, causing confusion and frustration during conversation. In this doctoral consortium, I will present PhonemeViz as a solution to this problem. PhonemeViz is a phoneme visualisation technique that allows a speech-reader to disambiguate confusing viseme-to-phoneme mappings by visualising phoneme information using movement within their peripheral vision. Early results suggest that PhonemeViz can enable speech-readers to overcome the barriers they face when speaking with other people.