Models of Learning

A fundamental question in assessing learning environments is: how do we measure the amount of learning, or more specifically, what pre- and post-test scores indicate the same learning for everyone? Assume that “low” students score 40 (percent) on the pre-test while “top” students score 70, what post-test scores would indicate equal learning?  One possibility would be if both groups moved upwards by the same amount, say 10 % – e.g. to 50 and 80 respectively.  Another option would be learning the same fraction of what they didn’t know on the pretest.  Say the “low” group moved from 40 to 80, so they learned 2/3 of what they didn’t know on the pretest; then the corresponding top score would be 93 (rather than the impossible score of 110%).

Empirically it turns out that within many different classes the later model (keying on the fraction of initially unknown material that is learned (i.e. known) on the post-test; this fraction is the normalized gain mentioned above.  The normalized gain seems to be a natural measure of learning because it is much more dependent on the quality of instruction than the initial knowledge level of the students (R. R. Hake)

PLB08 Mathematical learning models that depend on prior knowledge and instructional strategies, Pritchard, D. E., Lee, Y-J, and Bao, L., Phys. Rev. ST Phys. Educ. Res. 4, 010109 (2008).

This paper gives a possible explanation for why the normalized gain is so often constant: this is the result predicted for pure “memory learning”, where the probability of a student’s learning each piece of new knowledge presented by the teacher is independent of their existing knowledge.  We also show that a simple “constructive learning” model where learning is linearly dependent on how muh you already know gives dramatically higher normalized gains for the top students, and lower for the initially weaker students.

Bayesian Knowledge Tracing

PBS13  Adapting Bayesian Knowledge Tracing to a Massive Open Online Course in edX Zachary A. Pardos, Yoav Bergner, Daniel T. Seaton, David E. Pritchard   Proceedings of 6th International Conference on Educational Data Mining, Memphis 2013

Bayesian knowledge tracing assumes that knowledge comes in discrete binary ‘knowledge components’ and uses Bayesian mathematics to infer the current probability of knowing each one.  Our attempt to apply it to our MOOC data (and in the process discover the knowledge elements) proved mostly futile owing to the lack of previously defined knowledge components and an inability to account for different paths through the course materials.