Framing of attribute’s levels: influence on the valuation and interpretation of outcomes from a Best-Worst Scaling experiment

Purpose: A Best-Worst Scaling case 2 (BWS) was conducted to elicit treatment preferences for symptom control, side-effects, and process characteristics of various treatments prescribed in Parkinson’s Disease. Since risks are often perceived as difficult to interpret, especially among subgroups such as elderly, it was chosen to decrease this cognitive load by operationalization of the levels of symptoms and side-effects as: seldom to never, sometimes or often suffering from. The disadvantage of this strategy was that the exact definition patients had in mind when reading ‘seldom to never, sometimes or often suffering from’ was unknown. Our objective was to study whether this operationalization influenced the interpretation and valuation of outcomes of our BWS experiment.

Methods: A post-questionnaire was distributed among 30 patients who participated in the BWS-study. Patients were asked to state whether they suffer seldom to never, sometimes or often from tremors. Subsequently, they were asked to specify the experienced duration for this amount of suffering in hour and minutes per day. This was followed by an assessment of the perceived duration for the two other (not experienced) suffering states of tremors. These questions were then repeated for the remaining 5 attributes: posture and balance problems, slowness of movement, dizziness, fatigue, and dyskinesia. In analysis, patients were grouped based on their experienced suffering state per attribute, medians were estimated and a perceived duration range for each attribute for each suffering state was estimated. Beside, an actual duration range for each attribute was estimated based on the data of the experienced states. Based on these results, subgroup analyses of our original BWS data were performed to study whether the experienced amount of suffering from an attribute influenced attribute importance.

Results: The perceived range of an attribute’s duration patients have in mind is smaller than the actual duration range. The perception of burden is most underestimated by patients that seldom to never suffer from an attribute with a factor ranging from 1.5 - 8 for the different attributes. This underestimation was also present for patients who sometimes or often suffered from an attribute, but was much smaller: ranging from 1.1 - 1.6 (and did not occur for dizziness). BWS subgroup analysis showed that patients who seldom to never suffer from an attribute assigned a lower importance to that attribute compared to patients who often suffer from it. These differences ranged from 2% for posture and balance problems to 11% for the suffering of dyskinesia (between seldom to never and often suffer from).

Conclusions: Te chosen operationalization strategy resulted in different interpretations of the subjective concepts and influenced BWS results. Patients who only have minor complaints from a symptom or side-effect, seem to underestimate the actual burden of having major complaints. As a result, patients assigned a lower attribute importance to attributes from which they only seldom to never suffer from and a higher attribute importance to attributes from which they often suffer from. In conclusion, attribute importance might have been different when patient were presented with objective information about the duration of the attribute.