Pre-testing with cognitive interviews highlights unanticipated decision making in a DCE

Image for Pre-testing with cognitive interviews highlights unanticipated decision making in a DCE
Publication date: 
January, 2016
Presentation type: 
Poster presentation
Related staff: 
Dr Alison Pearce (former staff)
Dr Aileen Timmons (former staff)
Prof Linda Sharp (former staff)

Background: Cognitive interviewing uses theories of cognitive psychology and survey methodology to understand how respondents perceive and interpret survey questions. There have been few examples of cognitive interviewing in developing discrete choice experiments (DCEs). Our aim was to use cognitive interviewing to explore two aspects of decision making: a cheap-talk script to reduce hypothetical bias, and the reality and complexity of opt-out options vs forced choices.

Methods: We conducted ‘think aloud’ cognitive interviews with 17 prostate and colorectal cancer survivors completing a rigorously developed DCE of preferences for cancer follow-up services. The DCE contained 8 choice sets each with 8 attributes, including cost. The survey also included: a cheap-talk script, both forced choice and opt-out choice sets, and debriefing questions for comprehension, engagement and attribute non-attendance (ANA). An iterative analysis process was completed by two researchers using a framework approach, assessing decision making within and across participants.

Results: Preliminary analysis suggests the cheap-talk script has little impact on individuals’ hypothetical bias, primarily because the cost attribute was often ignored. The realism of the opt-out was affected by respondents’ prior experiences of cancer follow-up. It emerged that many debriefing question responses, particularly ANA, were contradictory to the decision making within choice sets.

Conclusion: Our study tested aspects of decision making that have not been explored using cognitive interviewing, and demonstrates the insight gained from pre-testing. In cancer survivors considering follow-up options, the cheap-talk script did not have the anticipated effect, costs were ignored, and responses to debriefing questions may be misleading.

Note: This poster was awarded 'Best Poster Presentation' at the HESG Winter 2016 meeting.

Published abstract: 
Pearce AM, Watson V, Gallagher P, Timmons A, Sharp L
Alison Pearce
Conference/meeting title: 
Health Economics Study Group Meeting Winter 2016
Event date: 
6 Jan 2016 to 8 Jan 2016
Manchester, UK

Building 6800
Cork Airport Business Park
Kinsale Road, Cork T12 CDF7
Email Contact us here
Tel: +353 (0) 21 4318014
Fax: +353 (0) 21 4318016

NCRI directions

Go to top