In this chapter we describe three experiments investigating multimodal information presentation in the context of a medical QA system. In Experiment 1, we wanted to know how non-experts design (multimodal) answers to medical questions, distinguishing between what questions and how questions. In Experiment 2, we concentrated on how people evaluate multimodal (text+picture) answer presentations on their informativeness and attractiveness. In Experiment 3, we evaluated two versions of an automatic picture selection method, and compared answer presentations with automatically selected pictures to answer presentations with manually selected pictures.
|Title of host publication||Interactive Multi-modal Question-Answering|
|Editors||Antal van den Bosch, Gosse Bouma|
|Place of Publication||Berlin|
|Number of pages||27|
|Publication status||Published - May 2011|
|Name||Theory and Applications of Natural Language Processing|