Online Education generates an immense amount of information about student interactions with the online learning environment. RELATE typically starts with time-stamped logs of every click that the student makes (open problem #7, give this wrong answer, consult pp 273-275 of the e-text, consult problem 3 on last week’s homework, give correct answer); these contain orders of magnitude more information than a typical gradebook. RELATE approaches these in the spirit of experimental physics (the PI’s background): examine the data from multiple perspectives (data mining), make or use existing mathematical models to quantify various aspects of the data, conduct purposeful experiments, and use the results to improve learning.
The first step in extracting understanding from the gigabytes of data (1 gigabyte is ~ 300k pages) in the log of a typical complete course is to “mine” the data, examining the time spent on various resources, the patterns of resource usage, how students spend their time. This shows several surprises: that most students have over 30% of their homework done two nights before it is due, that students vastly prefer video help when doing homework, but revert to the textbook on (open book) exams, and that copying homework is the best predictor of final exam grades. This last has many implications, discussed in a later section. Here are some examples of data mining:
SBC14 Who Does What in a Massive Open Online Course? Daniel T. Seaton, Yoav Bergner, Isaac Chuang, Piotr Mitros, and David E. Pritchard Communications, ACM Volume 57 April 2014 pp 58-65.
The vast majority of the participants spent less than 1 hour exploring this course; while almost 2/3 of those attempting more than 5% of the homework received a certificate. The certificate earners averaged about 100 hours, spent mostly on video lectures and homework, with the most references to the discussion forum. On the exams students spent most time on the e-textbook and made most references to the homework.
SKB14 Analyzing the Impact of Course Structure on eText Use in Blended Introductory Physics Courses Daniel T. Seaton, Gerd Kortemeyer, Yoav Bergner, Saif Rayyan, and David E. Pritchard, , American Journal of Physics 82, 1186-1197 (2014)
e-text use is roughly doubled in reformed courses with weekly or biweekly quizzes, relative to traditional courses with several midterms, each preceded by “cramming”; supplemental e-texts were used ~10 times less.
TSC11 When Students Can Choose Easy, Medium, Or Hard Homework Problems Raluca E. Teodorescu, Daniel T. Seaton, Caroline N. Cardamone, Saif Rayyan, Jonathan E. Abbott, Analia Barrantes, Andrew Pawl, and David E. Pritchard Physics Education Research Conference 2011 Volume 1413, Pages 81-84
Poster on time of completion