ConMap

Traditional problem-based exams are not reliable tools for diagnosing students' knowledge and guiding pedagogical intervention; new tools grounded in cognitive science and educational research are needed. If one wishes to assess a students' knowledge in detail, rather than merely summarize items on which the student did or did not succeed, one needs a model of what a knowledge state is, instruments for probing a student's state of knowledge, and an understanding of the mechanism by which instruments probe the state. An effective diagnostic assessment must describe a student with reference to some suitably detailed model of physics knowing, learning, and application.

Our ConMap ("Conceptual Mapping") project has two interdependent objectives. One is developing a detailed, quantitative model of physics conceptual knowledge and its application to problem-solving and analysis. The other is creating practical tools to enhance teaching through frequent "formative assessment" and monitoring of learning. We have been investigating the potential of a set of simple, computer-administered term-association tasks for probing the concepts and interconnections within a physics student's knowledge store. By eliciting spontaneous associations between physics terms in a variety of ways and contexts, we hope to build up a map of the concepts within a topic area that a student has access to and of the web of associations providing structure to those concepts. Since experts and novices are known to structure their knowledge in qualitatively different ways - experts hierarchically around key principles with rich interlinking, novices chronologically with sparse interlinking - we hope to detect signatures of expertise in the patterns of associations elicited, and perhaps observe the onset of those signatures as learning occurs.

This project began with Ian Beatty's dissertation work in the 1990's. Since then, it has been pursued through the dissertation work of graduate students Jenny Chang and Edgardo Ortiz.

Preliminary work has established that the approach is "interesting and promising": We have found internal consistency within the data and partial correlations with other measures of knowledge, suggesting that the ConMap probes are in fact measuring something real, reproducible, and relevant. We have also experimented with quantitative dynamical models "explaining" some aspects of the observed data.

The original project is described in Ian's dissertation (2000) and, partially, in a paper published in 2002 the American Journal of Physics. Later work has not been written up (yet).