Playful vs. serious instruction giving in a 3D game environment

Roan Boer Rookhuiszen, Mariet Theune

Research output: Contribution to journalArticleAcademicpeer-review

6 Downloads (Pure)

Abstract

In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.
Original languageUndefined
Article number10.1016/j.entcom.2009.09.006
Pages (from-to)95-104
Number of pages10
JournalEntertainment computing
Volume1
Issue number2
DOIs
Publication statusPublished - Dec 2009

Keywords

  • IR-68746
  • METIS-264445
  • EWI-16930

Cite this

@article{b3c4765f4e2440af9eefb8eba8f30b2b,
title = "Playful vs. serious instruction giving in a 3D game environment",
abstract = "In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.",
keywords = "IR-68746, METIS-264445, EWI-16930",
author = "{Boer Rookhuiszen}, Roan and Mariet Theune",
note = "10.1016/j.entcom.2009.09.006",
year = "2009",
month = "12",
doi = "10.1016/j.entcom.2009.09.006",
language = "Undefined",
volume = "1",
pages = "95--104",
journal = "Entertainment computing",
issn = "1875-9521",
publisher = "Elsevier",
number = "2",

}

Playful vs. serious instruction giving in a 3D game environment. / Boer Rookhuiszen, Roan; Theune, Mariet.

In: Entertainment computing, Vol. 1, No. 2, 10.1016/j.entcom.2009.09.006, 12.2009, p. 95-104.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Playful vs. serious instruction giving in a 3D game environment

AU - Boer Rookhuiszen, Roan

AU - Theune, Mariet

N1 - 10.1016/j.entcom.2009.09.006

PY - 2009/12

Y1 - 2009/12

N2 - In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.

AB - In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.

KW - IR-68746

KW - METIS-264445

KW - EWI-16930

U2 - 10.1016/j.entcom.2009.09.006

DO - 10.1016/j.entcom.2009.09.006

M3 - Article

VL - 1

SP - 95

EP - 104

JO - Entertainment computing

JF - Entertainment computing

SN - 1875-9521

IS - 2

M1 - 10.1016/j.entcom.2009.09.006

ER -