Research On Model-based Diagnosis About Learner Behavior

Both in professional and every-day life people have to interact with and reason about a large number of systems. Computer simulations can be used to construct interactive environments by means of which people can develop knowledge about the behaviour of these systems. The steady increase in computing power has in fact given simulation a solid position within the area of educational systems. However, several studies have shown that simulations are only effective when proper guidance is provided. Automating certain tutoring and training functions in order to provide such guidance requires the simulation model to be articulate. Two further requirements follow from this. Firstly, a specific simulation model should manifest all the behavioral features of the "real"system as far as those are relevant to the educational goals. Secondly, a simulation model should contain the appropriate handles, by means of which these features are indexed, to enable a knowledgeable communication with the learner about the model contents. Qualitative simulators provide a basis for generating articulate simulation models. Given such models, a remaining challenge concerns the automated handling of guidance by the educational system. Providing guidance means that the learning environment should be able to adapt the interaction to the situation at hand, both with respect to the specific subject matter that is considered and to the individual learner it is interacting with. Guidance may take on many forms, such as providing explanations, presenting counter examples, suggesting assignments, and the like. Whatever the specific form, individual guidance requires knowledge about the learner and hence involves assessment of his or her knowledge. More specifically, the educational system has to diagnose the learner's problem solving behavior. The subject matter we are dealing with in this article concerns "behavior analysis", i.e., learners should acquire problem solving skills such as predicting or explaining the behavior of systems using qualitative terms. Hence, the learner's problem solving behavior consists of a set of inferences about the behavior of these systems. The way the student interacts with the learning environment reflects this problem solving behavior. The educational system therefore has to monitor this interaction and diagnose deviations, with respect to some norm, in terms of problem solving errors made by the learner. In this article we propose a model-based, approach to diagnose learner behavior. Following , this choice is based on the fact that model-based diagnosis is an extensively studied and well-understood field of research, and hence we can reuse existing techniques and representations. Model-based diagnosis claims to be generic and domain independent. It does not require explicit fault models or bug catalogues for its operation, but instead reasons from a representation of the correct behavior of the system to be diagnosed. More specific, in model-based diagnosis the behavior of a device is compared to the behavior predicted by a model of that device. In the case of diagnosing learner behavior we propose to automatically generate this model using a qualitative simulator. On the basis of the output of the simulator a subject matter model can be constructed that acts as a normative model of the learner behavior and as such can be used for model based diagnosis. Deviations in the problem solving behavior of the learner, i.e., incorrect predictions or explanations concerning the behavior of the system that is the subject of the teaching activity, are then diagnosed as inferences made by the qualitative simulator that the learner cannot have applied correctly given the observations. The same diagnostic component (e.g., GDE) can be applied to different subject matter models, as long as they comply with the modeling principlesrequired by the diagnostic algorithms. One of the main requirements is that the system under diagnosis can be modeled as a set of connected components, for which the behavior can be defined individually. These components should model the smallest entities that can be individually repaired;diagnosis is only useful down to the level of possible repairs. In the context of learning, this means that these components should represent the smallest units of problem solving knowledge that are still relevant in an educational setting (e.g., that can be individually explained). The constraints that the component connection paradigm puts on the subject matter models are not only a requirement for diagnosis, but are also useful for other educational functions: a clear separation of the knowledge in terms of indexed elements that make sense in education is a prerequisite for automated structured explanation, generation of questions, or subject matter sequencing. Only with such a modular structure, different components can be freely combined, selected, and sequenced. Following the above argumentation, we propose a framework for the construction of interactive learning environments based on qualitative models. These models are used as simulations of the subject matter and provide the basis for the construction of advanced educational functions to support the interaction with the learner. The procedures implementing this approach should be domain independent, i.e., they should not depend on the specific system that is being simulated by the qualitative simulator, and they should do their work fully automatic. In the research presented we focus on the function "diagnosis of learner behavior"because it is central to any form of individualized interaction. The following topics must be addressed. First, it should be shown that the qualitative simulator represents knowledge that fits the way teachers and learners communicate about system behavior. The qualitative simulator should produce a description of the system behavior that is useful in an...