The effect of inefficient designs on the stability of individual preferences

Marieke Geertruida Maria Weernink; Janine Astrid van Til; CGM Oudshoorn

Abstract

Introduction In conjoint analysis, efficient experimental designs often require the respondent to answer many choice tasks. To reduce respondent burden, less efficient designs are used. We aim to investigate the effects of using inefficient designs on the stability of individual preferences. Methods Two convenience samples completed a best‐worst scaling case 2 experiment exploring preferences for treatment of localized prostate cancer (PC) or breast cancer (BC). Treatments were described using 5 attributes and 2 levels in both cases. An efficient design (balanced and orthogonal) was used to construct 16 profiles/questions. The sequence of profiles ensured that after 6 and 12 questions level‐balance was obtained (no orthogonality). After completion of 6, 12 and 16 questions, each respondent received real‐time feedback on the importance of attributes (graph) and a preferred treatment recommendation (words). Feedback was based on bestminus‐worst scores. Results In total, 68 women and 23 men completed the experiment. 82% of respondents received the same treatment recommendations after completion of 6, 12 or 16 questions (85% BC, 78% PC). In addition, for 12% (7% BC, 17% PC) only the treatment recommendation after six questions differed from the latter two. For the BC‐case the mean‐difference in importance score per attribute between 6 and 12 questions was 7.5% (3.1%) and between 12 and 16 questions 3.9% (2.0%), P<0.001. For PC, the differences were 6.9% (3.4%) and 3.5% (1.5%) respectively, P<0.001. Although only two respondents had the same rank order of attributes based on importance, 81% of respondents never changed their most important attribute (82%‐BC, 78%‐PC), and 45% never altered their second‐most important attribute (47%‐BC, 39%‐PC). Conclusions For these cases, individual preferences are quite stable after six questions. However, adding more questions results in higher stability of attribute importance. More research is needed to examine whether inefficient designs can be used in value clarification exercises.

Original language	English
Pages	11-11
Number of pages	1
Publication status	Published - 4 Nov 2017
Event	7th meeting of the International Academy of Health Preference Research 2017: “ The Econometrics of Preference Heterogeneity ” - GTG Glasgow, Glasgow, United Kingdom Duration: 3 Nov 2017 → 4 Nov 2017 Conference number: 7 http://iahpr.org/wordpress/wp-content/uploads/2017/11/program171112b.pdf

Conference

Conference	7th meeting of the International Academy of Health Preference Research 2017
Abbreviated title	IAHPR
Country	United Kingdom
City	Glasgow
Period	3/11/17 → 4/11/17
Internet address	http://iahpr.org/wordpress/wp-content/uploads/2017/11/program171112b.pdf

Access to Document

program171112bFinal published version, 3.21 MB

Fingerprint Dive into the research topics of 'The effect of inefficient designs on the stability of individual preferences'. Together they form a unique fingerprint.

Cite this

Weernink, M. G. M., van Til, J. A., & Oudshoorn, CGM. (2017). The effect of inefficient designs on the stability of individual preferences. 11-11. Abstract from 7th meeting of the International Academy of Health Preference Research 2017, Glasgow, United Kingdom.

@conference{11a1eed3c9484b4792484e5572059f7d,

title = "The effect of inefficient designs on the stability of individual preferences",

abstract = "Introduction In conjoint analysis, efficient experimental designs often require the respondent to answer many choice tasks. To reduce respondent burden, less efficient designs are used. We aim to investigate the effects of using inefficient designs on the stability of individual preferences. Methods Two convenience samples completed a best‐worst scaling case 2 experiment exploring preferences for treatment of localized prostate cancer (PC) or breast cancer (BC). Treatments were described using 5 attributes and 2 levels in both cases. An efficient design (balanced and orthogonal) was used to construct 16 profiles/questions. The sequence of profiles ensured that after 6 and 12 questions level‐balance was obtained (no orthogonality). After completion of 6, 12 and 16 questions, each respondent received real‐time feedback on the importance of attributes (graph) and a preferred treatment recommendation (words). Feedback was based on bestminus‐worst scores. Results In total, 68 women and 23 men completed the experiment. 82{\%} of respondents received the same treatment recommendations after completion of 6, 12 or 16 questions (85{\%} BC, 78{\%} PC). In addition, for 12{\%} (7{\%} BC, 17{\%} PC) only the treatment recommendation after six questions differed from the latter two. For the BC‐case the mean‐difference in importance score per attribute between 6 and 12 questions was 7.5{\%} (3.1{\%}) and between 12 and 16 questions 3.9{\%} (2.0{\%}), P<0.001. For PC, the differences were 6.9{\%} (3.4{\%}) and 3.5{\%} (1.5{\%}) respectively, P<0.001. Although only two respondents had the same rank order of attributes based on importance, 81{\%} of respondents never changed their most important attribute (82{\%}‐BC, 78{\%}‐PC), and 45{\%} never altered their second‐most important attribute (47{\%}‐BC, 39{\%}‐PC). Conclusions For these cases, individual preferences are quite stable after six questions. However, adding more questions results in higher stability of attribute importance. More research is needed to examine whether inefficient designs can be used in value clarification exercises.",

author = "Weernink, {Marieke Geertruida Maria} and {van Til}, {Janine Astrid} and CGM Oudshoorn",

year = "2017",

month = "11",

day = "4",

language = "English",

pages = "11--11",

note = "7th meeting of the International Academy of Health Preference Research 2017 : “ The Econometrics of Preference Heterogeneity ”, IAHPR ; Conference date: 03-11-2017 Through 04-11-2017",

url = "http://iahpr.org/wordpress/wp-content/uploads/2017/11/program171112b.pdf",

}

TY - CONF

T1 - The effect of inefficient designs on the stability of individual preferences

AU - Weernink, Marieke Geertruida Maria

AU - van Til, Janine Astrid

AU - Oudshoorn, CGM

PY - 2017/11/4

Y1 - 2017/11/4

N2 - Introduction In conjoint analysis, efficient experimental designs often require the respondent to answer many choice tasks. To reduce respondent burden, less efficient designs are used. We aim to investigate the effects of using inefficient designs on the stability of individual preferences. Methods Two convenience samples completed a best‐worst scaling case 2 experiment exploring preferences for treatment of localized prostate cancer (PC) or breast cancer (BC). Treatments were described using 5 attributes and 2 levels in both cases. An efficient design (balanced and orthogonal) was used to construct 16 profiles/questions. The sequence of profiles ensured that after 6 and 12 questions level‐balance was obtained (no orthogonality). After completion of 6, 12 and 16 questions, each respondent received real‐time feedback on the importance of attributes (graph) and a preferred treatment recommendation (words). Feedback was based on bestminus‐worst scores. Results In total, 68 women and 23 men completed the experiment. 82% of respondents received the same treatment recommendations after completion of 6, 12 or 16 questions (85% BC, 78% PC). In addition, for 12% (7% BC, 17% PC) only the treatment recommendation after six questions differed from the latter two. For the BC‐case the mean‐difference in importance score per attribute between 6 and 12 questions was 7.5% (3.1%) and between 12 and 16 questions 3.9% (2.0%), P<0.001. For PC, the differences were 6.9% (3.4%) and 3.5% (1.5%) respectively, P<0.001. Although only two respondents had the same rank order of attributes based on importance, 81% of respondents never changed their most important attribute (82%‐BC, 78%‐PC), and 45% never altered their second‐most important attribute (47%‐BC, 39%‐PC). Conclusions For these cases, individual preferences are quite stable after six questions. However, adding more questions results in higher stability of attribute importance. More research is needed to examine whether inefficient designs can be used in value clarification exercises.

AB - Introduction In conjoint analysis, efficient experimental designs often require the respondent to answer many choice tasks. To reduce respondent burden, less efficient designs are used. We aim to investigate the effects of using inefficient designs on the stability of individual preferences. Methods Two convenience samples completed a best‐worst scaling case 2 experiment exploring preferences for treatment of localized prostate cancer (PC) or breast cancer (BC). Treatments were described using 5 attributes and 2 levels in both cases. An efficient design (balanced and orthogonal) was used to construct 16 profiles/questions. The sequence of profiles ensured that after 6 and 12 questions level‐balance was obtained (no orthogonality). After completion of 6, 12 and 16 questions, each respondent received real‐time feedback on the importance of attributes (graph) and a preferred treatment recommendation (words). Feedback was based on bestminus‐worst scores. Results In total, 68 women and 23 men completed the experiment. 82% of respondents received the same treatment recommendations after completion of 6, 12 or 16 questions (85% BC, 78% PC). In addition, for 12% (7% BC, 17% PC) only the treatment recommendation after six questions differed from the latter two. For the BC‐case the mean‐difference in importance score per attribute between 6 and 12 questions was 7.5% (3.1%) and between 12 and 16 questions 3.9% (2.0%), P<0.001. For PC, the differences were 6.9% (3.4%) and 3.5% (1.5%) respectively, P<0.001. Although only two respondents had the same rank order of attributes based on importance, 81% of respondents never changed their most important attribute (82%‐BC, 78%‐PC), and 45% never altered their second‐most important attribute (47%‐BC, 39%‐PC). Conclusions For these cases, individual preferences are quite stable after six questions. However, adding more questions results in higher stability of attribute importance. More research is needed to examine whether inefficient designs can be used in value clarification exercises.

M3 - Abstract

SP - 11

EP - 11

ER -