Exploring the Utility of Nonfunctional Distractors

Abstract

Functional distractors (the incorrect options in a multiple-choice question) should draw attention from those test-takers who lack sufficient ability or knowledge to respond correctly. Unfortunately, previous research on distractors has demonstrated the unsettling reality that this rarely occurs in practice leading to recommendations for creating items with fewer incorrect alternatives. The purpose of the present study was to explore if these nonfunctional distractors (NFDs) may still yield value in detecting unusual examinee behavior. Using empirical data from a high-stakes licensure examination, examinees who selected an excessive number of NFDs were flagged and analyzed with respect to their response times and overall performance. Results indicated that these flagged examinees were also of extremely low ability, selected NFDs consistently across item sequence, and were homogenous in their pacing strategies - spending a similar amount of time when choosing a nonfunctional or functional distractor. Implications for relevant policy decisions, mitigation strategies, operational applications, and test security considerations are discussed.