Beyond the Beat: Modelling Intentions in a Virtual Conductor

Mark ter Maat, Rob M. Ebbers, Dennis Reidsma, Antinus Nijholt

  • 6 Citations

Abstract

We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.
Original languageUndefined
Title of host publicationINTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment
Place of PublicationBrussels, Belgium
PublisherICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)
Pages12
Number of pages10
ISBN (Print)978-963-9799-13-4
StatePublished - Jan 2008

Publication series

NameACM Digital Library
PublisherICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)
Number1

Fingerprint

Interaction
Conducting
Rehearsal
Module
Gesture
Audio analysis
Conversational analysis
Musicians
Musical score

Keywords

  • EWI-12717
  • IR-62293
  • METIS-250974

Cite this

ter Maat, M., Ebbers, R. M., Reidsma, D., & Nijholt, A. (2008). Beyond the Beat: Modelling Intentions in a Virtual Conductor. In INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment (pp. 12). (ACM Digital Library; No. 1). Brussels, Belgium: ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).

ter Maat, Mark; Ebbers, Rob M.; Reidsma, Dennis; Nijholt, Antinus / Beyond the Beat: Modelling Intentions in a Virtual Conductor.

INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment. Brussels, Belgium : ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2008. p. 12 (ACM Digital Library; No. 1).

Research output: Scientific - peer-reviewConference contribution

@inbook{75a99b6d6234462ea328ebef36f2c777,
title = "Beyond the Beat: Modelling Intentions in a Virtual Conductor",
abstract = "We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.",
keywords = "EWI-12717, IR-62293, METIS-250974",
author = "{ter Maat}, Mark and Ebbers, {Rob M.} and Dennis Reidsma and Antinus Nijholt",
year = "2008",
month = "1",
isbn = "978-963-9799-13-4",
series = "ACM Digital Library",
publisher = "ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)",
number = "1",
pages = "12",
booktitle = "INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment",

}

ter Maat, M, Ebbers, RM, Reidsma, D & Nijholt, A 2008, Beyond the Beat: Modelling Intentions in a Virtual Conductor. in INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment. ACM Digital Library, no. 1, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, Belgium, pp. 12.

Beyond the Beat: Modelling Intentions in a Virtual Conductor. / ter Maat, Mark; Ebbers, Rob M.; Reidsma, Dennis; Nijholt, Antinus.

INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment. Brussels, Belgium : ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2008. p. 12 (ACM Digital Library; No. 1).

Research output: Scientific - peer-reviewConference contribution

TY - CHAP

T1 - Beyond the Beat: Modelling Intentions in a Virtual Conductor

AU - ter Maat,Mark

AU - Ebbers,Rob M.

AU - Reidsma,Dennis

AU - Nijholt,Antinus

PY - 2008/1

Y1 - 2008/1

N2 - We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.

AB - We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.

KW - EWI-12717

KW - IR-62293

KW - METIS-250974

M3 - Conference contribution

SN - 978-963-9799-13-4

T3 - ACM Digital Library

SP - 12

BT - INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment

PB - ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)

ER -

ter Maat M, Ebbers RM, Reidsma D, Nijholt A. Beyond the Beat: Modelling Intentions in a Virtual Conductor. In INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment. Brussels, Belgium: ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering). 2008. p. 12. (ACM Digital Library; 1).