Expressive interaction design using facial muscles as controllers

French, Fiona, Terry, Cassandra, Huq, Saif, Furieri, Isaac, Jarzembinski, Mateusz, Pauliukenas, Simas, Morrison, Neil and Shepherd, Karen (2022) Expressive interaction design using facial muscles as controllers. Multimodal Technologies and Interaction, 6 (9). ISSN 2414-4088

Preview

Text mti-06-00078-v2.pdf - Accepted Version Available under License Creative Commons Attribution No Derivatives 4.0. Download (4MB) | Preview

Abstract / Description

Here we describe a proof-of-concept case study focusing on the design and development of a novel computer interface that uses facial muscles to control interactivity within a virtual environment. We have developed a system comprised of skin-mounted electrodes that detect underlying muscle activity through electromyography. The signals from the electrodes are filtered and smoothed, then used as input data to an application that displays a virtual environment with a 3D animated avatar. The user’s expressions control the facial movements of the avatar, thus conveying user emotions through real-time animation of a representative face in a virtual scenario. To achieve this, we collaborated with our Public and Patient Involvement focus group to discuss concepts and design appropriate interactions, while simultaneously developing a prototype system. Programmers and 3D artists worked together to create a system whereby individual user facial muscles are connected to 3D animated models of the same muscle features represented in an avatar, providing the user with an option to receive visual and numerical feedback on the extent of their muscle control. Using the prototype system, people can communicate facial expressions virtually with each other, without the need for a camera. This research is part of an on-going project to develop a facial muscle rehabilitation system that can be optimized to help patients with conditions such as hypomimia.

Item Type:	Article
Uncontrolled Keywords:	interaction design; inclusive design; biometrics; facial expression recognition; rehabilitation; assistive technology; virtual reality; participatory design; multimodal interfaces; human-agent interaction
Subjects:	000 Computer science, information & general works 600 Technology 600 Technology > 610 Medicine & health
Department:	School of Computing and Digital Media
Depositing User:	Fiona French
Date Deposited:	14 Sep 2022 08:28
Last Modified:	14 Sep 2022 08:28
URI:	https://repository.londonmet.ac.uk/id/eprint/7914

Downloads

Actions (login required)

View Item