Konold et al. (1993b)

Subjects were asked to select from among four possible sequences the "most likely" to result from flipping a coin five times. Contrary to the results of Kahnemanand Tversky (1972), the majority of subjects (72%) correctly answered that the sequences are equally likely to occur. This result suggests, as does performance on similar NAEP items, that most secondary school and college-age students view successive outcomes of a random process as independent. However, in a follow-up question, subjects were also asked to select the "least likely"result.Only half the subjects who had answered correctly responded again that the sequences were equally likely; the others selected one of the sequences as least likely. This result was replicated in a second study in which 20 subjects were inter- viewed as they solved the same problems. One account of these logically inconsistent responses is that subjects reason about the two questions from different perspectives. When asked to select the most likely outcome, some believe they are being asked to predict what actually will happen, and give the answer "equally likely" to indicate that all of the sequences are possible. This reasoning has been described by Konold (1989) as an "outcome approach"to uncertainty. This prediction scheme does not fit questions worded in terms of the least likely result, and thus some subjects select an incompatible answer based on "representativeness"(Kahneman& Tversky, 1972). These results suggest that the percentage of secondary school students who understand the concept of independenceis much lower than the latest NAEP results would lead us to believe and, more generally, point to the difficulty of assessing conceptual understanding with multiple-choice items.