Playful vs. serious instruction giving in a 3D game environment

Roan Boer Rookhuiszen, Mariet Theune

    Research output: Contribution to journalArticleAcademicpeer-review

    7 Downloads (Pure)

    Abstract

    In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.
    Original languageUndefined
    Article number10.1016/j.entcom.2009.09.006
    Pages (from-to)95-104
    Number of pages10
    JournalEntertainment computing
    Volume1
    Issue number2
    DOIs
    Publication statusPublished - Dec 2009

    Keywords

    • IR-68746
    • METIS-264445
    • EWI-16930

    Cite this

    @article{b3c4765f4e2440af9eefb8eba8f30b2b,
    title = "Playful vs. serious instruction giving in a 3D game environment",
    abstract = "In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.",
    keywords = "IR-68746, METIS-264445, EWI-16930",
    author = "{Boer Rookhuiszen}, Roan and Mariet Theune",
    note = "10.1016/j.entcom.2009.09.006",
    year = "2009",
    month = "12",
    doi = "10.1016/j.entcom.2009.09.006",
    language = "Undefined",
    volume = "1",
    pages = "95--104",
    journal = "Entertainment computing",
    issn = "1875-9521",
    publisher = "Elsevier",
    number = "2",

    }

    Playful vs. serious instruction giving in a 3D game environment. / Boer Rookhuiszen, Roan; Theune, Mariet.

    In: Entertainment computing, Vol. 1, No. 2, 10.1016/j.entcom.2009.09.006, 12.2009, p. 95-104.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - Playful vs. serious instruction giving in a 3D game environment

    AU - Boer Rookhuiszen, Roan

    AU - Theune, Mariet

    N1 - 10.1016/j.entcom.2009.09.006

    PY - 2009/12

    Y1 - 2009/12

    N2 - In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.

    AB - In this paper we introduce two NLG systems that we developed for the GIVE challenge, which was aimed at the evaluation of natural language generation (NLG) systems. The Challenge involved automatically generating instructions for users to carry out a task in a 3D game environment. One of our systems focused on generating optimally helpful ‘serious’ instructions while the other focused on entertainment, providing more playful instructions. We used the data gathered in the Challenge – both subjective user ratings and objective task performance data – to compare the efficiency and entertainment value of both systems. We found a clear difference in efficiency, but were unable to prove that one system was more entertaining than the other. This could be explained by the fact that the set-up and evaluation methods of the GIVE Challenge were not aimed at measuring entertainment. Based on our experiences, we give some suggestions for the set-up of future installments of the Challenge.

    KW - IR-68746

    KW - METIS-264445

    KW - EWI-16930

    U2 - 10.1016/j.entcom.2009.09.006

    DO - 10.1016/j.entcom.2009.09.006

    M3 - Article

    VL - 1

    SP - 95

    EP - 104

    JO - Entertainment computing

    JF - Entertainment computing

    SN - 1875-9521

    IS - 2

    M1 - 10.1016/j.entcom.2009.09.006

    ER -