Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model

Merijn Eskes (Corresponding Author), Alfonsus Jacobus Maria Balm, Maarten J.A. van Alphen, Ludi E. Smeele, Ian Stavness, Ferdinand van der Heijden

Research output: Contribution to journalArticleAcademicpeer-review

3 Citations (Scopus)
28 Downloads (Pure)

Abstract

Purpose: Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model. Methods: Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles (act all) , selecting the three muscles showing highest muscle activity bilaterally (act 3) —this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance—and activating the muscles considered most relevant per instruction (act rel) , bilaterally. The model’s lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients (ρ). Results: The correlation coefficient between simulations and measurements with act rel resulted in a median ρ of 0.77. act 3 had a median ρ of 0.78, whereas with act all the median ρ decreased to 0.45. Conclusion: We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median ρ of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient’s own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy.

Original languageEnglish
Pages (from-to)47-59
Number of pages13
JournalInternational journal of computer assisted radiology and surgery
Volume13
Issue number1
Early online date31 Aug 2017
DOIs
Publication statusPublished - 1 Jan 2018

Fingerprint

Facial Expression
Muscle
Muscles
Lip
Chemical activation
Volunteers
Anatomy
Lip Neoplasms
Facial Muscles
Mouth Neoplasms
Photodynamic therapy
Photochemotherapy
Surface measurement
Radiotherapy
Tongue
Surgery
Virtual reality

Keywords

  • UT-Hybrid-D
  • Forward modelling
  • Functional inoperability
  • Head and neck cancer
  • Lips
  • Surface electromyography
  • Biomechanical modelling

Cite this

@article{9c46757aefff4ea4a605b6530c90358c,
title = "Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model",
abstract = "Purpose: Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model. Methods: Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles (act all) , selecting the three muscles showing highest muscle activity bilaterally (act 3) —this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance—and activating the muscles considered most relevant per instruction (act rel) , bilaterally. The model’s lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients (ρ). Results: The correlation coefficient between simulations and measurements with act rel resulted in a median ρ of 0.77. act 3 had a median ρ of 0.78, whereas with act all the median ρ decreased to 0.45. Conclusion: We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median ρ of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient’s own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy.",
keywords = "UT-Hybrid-D, Forward modelling, Functional inoperability, Head and neck cancer, Lips, Surface electromyography, Biomechanical modelling",
author = "Merijn Eskes and Balm, {Alfonsus Jacobus Maria} and van Alphen, {Maarten J.A.} and Smeele, {Ludi E.} and Ian Stavness and {van der Heijden}, Ferdinand",
note = "Springer deal",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/s11548-017-1659-5",
language = "English",
volume = "13",
pages = "47--59",
journal = "International journal of computer assisted radiology and surgery",
issn = "1861-6410",
publisher = "Springer",
number = "1",

}

Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model. / Eskes, Merijn (Corresponding Author); Balm, Alfonsus Jacobus Maria; van Alphen, Maarten J.A.; Smeele, Ludi E.; Stavness, Ian; van der Heijden, Ferdinand.

In: International journal of computer assisted radiology and surgery, Vol. 13, No. 1, 01.01.2018, p. 47-59.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model

AU - Eskes, Merijn

AU - Balm, Alfonsus Jacobus Maria

AU - van Alphen, Maarten J.A.

AU - Smeele, Ludi E.

AU - Stavness, Ian

AU - van der Heijden, Ferdinand

N1 - Springer deal

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Purpose: Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model. Methods: Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles (act all) , selecting the three muscles showing highest muscle activity bilaterally (act 3) —this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance—and activating the muscles considered most relevant per instruction (act rel) , bilaterally. The model’s lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients (ρ). Results: The correlation coefficient between simulations and measurements with act rel resulted in a median ρ of 0.77. act 3 had a median ρ of 0.78, whereas with act all the median ρ decreased to 0.45. Conclusion: We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median ρ of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient’s own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy.

AB - Purpose: Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model. Methods: Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles (act all) , selecting the three muscles showing highest muscle activity bilaterally (act 3) —this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance—and activating the muscles considered most relevant per instruction (act rel) , bilaterally. The model’s lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients (ρ). Results: The correlation coefficient between simulations and measurements with act rel resulted in a median ρ of 0.77. act 3 had a median ρ of 0.78, whereas with act all the median ρ decreased to 0.45. Conclusion: We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median ρ of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient’s own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy.

KW - UT-Hybrid-D

KW - Forward modelling

KW - Functional inoperability

KW - Head and neck cancer

KW - Lips

KW - Surface electromyography

KW - Biomechanical modelling

UR - http://www.scopus.com/inward/record.url?scp=85028765698&partnerID=8YFLogxK

U2 - 10.1007/s11548-017-1659-5

DO - 10.1007/s11548-017-1659-5

M3 - Article

VL - 13

SP - 47

EP - 59

JO - International journal of computer assisted radiology and surgery

JF - International journal of computer assisted radiology and surgery

SN - 1861-6410

IS - 1

ER -