Illustrating answers: an evaluation of automatically retrieved illustrations of answers to medical questions

W.E. Bosma, Mariet Theune, C.M.J. van Hooijdonk, E. Krahmer, F. Maes

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    43 Downloads (Pure)


    In this paper we discuss and evaluate a method for automatic text illustration, applied to answers to medical questions. Our method for selecting illustrations is based on the idea that similarities between the answers and picture-related text (the picture’s caption or the section/paragraph that includes the picture) can be used as evidence that the picture would be appropriate to illustrate the answer.In a user study, participants rated answer presentations consisting of a textual component and a picture. The textual component was a manually written reference answer; the picture was automatically retrieved by measuring the similarity between the text and either the picture’s caption or its section. The caption-based selection method resulted in more attractive presentations than the section-based method; the caption-based method was also more consistent in selecting informative pictures and showed a greater correlation between user-rated informativeness and the confidence of relevance of the system.When compared to manually selected pictures, we found that automatically selected pictures were rated similarly to decorative pictures, but worse than informative pictures.
    Original languageEnglish
    Title of host publicationAISB 2008 Convention Communication, Interaction and Social Intelligence, 1st-4th April 2008, University of Aberdeen
    Subtitle of host publicationVol. 10: Proceedings of the AISB 2008 Symposium on Multimodal Output Generation (MOG 2008)
    EditorsMariët Theune, Ielka van der Sluis, Yulia Bachvarova, Elisabeth Andre
    PublisherThe Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB)
    Number of pages8
    ISBN (Print)978-1-902956-69-5
    Publication statusPublished - Apr 2008
    EventAISB Symposium on Multimodal Output Generation, MOG 2008 - Aberdeen, Scotland, United Kingdom
    Duration: 3 Apr 20084 Apr 2008


    ConferenceAISB Symposium on Multimodal Output Generation, MOG 2008
    Abbreviated titleMOG
    Country/TerritoryUnited Kingdom
    CityAberdeen, Scotland


    Dive into the research topics of 'Illustrating answers: an evaluation of automatically retrieved illustrations of answers to medical questions'. Together they form a unique fingerprint.

    Cite this