Michigan State University

Gathering and Timely use of Feedback from Individualized

On-Line Work.

Matthew Hall

Joyce Parker

Behrouz Minaei-Bigdoli

Guy Albertelli

Gerd Kortemeyer

Edwin Kashy

Matthew Hall

Joyce Parker

Behrouz Minaei-Bigdoli

Guy Albertelli

Gerd Kortemeyer

Edwin Kashy

Michigan State University

Alfred P. Sloan Foundation

Michigan State University

Andrew W. Mellon Foundation

National Superconducting Cyclotron Laboratory

National Science Foundation

Acknowledgements

Michigan State University

How can we keep instructors informed about how their students are doing?

How can an instructor tell where their students are going wrong?

How can an instructor use this feedback to provide a better learning experience for their students?

Questions

Michigan State University

Physics 183

Class timetraditional lecture

demonstrations

lecture exercises

quizzes

Resources for studentsphysics learning center

office hours

First year Calculus based Physics

Michigan State University

First Year Calculus based Physics Course

Assessmentindividualized on-line homework with feedback

supplemental problems, and mid-term and final exams

Evaluationmid-term tests and final exam

Michigan State University

Course Management System

LON-CAPA

Individualized problemshighly randomized

Multiple response typesnumerical, classification, ranking, click-on-image, etc

Immediate feedbackmultiple attempts

Michigan State University

Class Statistics

Students (S = 400)

Total number of problems (P = 250)

Multiple attempts are allowed (A <= 15)

Problems can have many responses (R = 1..10)

Volume of data is significant– S x A x R for a given problem– S x P x A x R for all problems in the course

Michigan State University

Variety of data

Each student receives their own version of a problem– Different statements– Different pictures– Different answers

Michigan State University

Cutting through the complexity and volume of data

• General statistics of class performance on a single problem

• Feedback on responses to complex conceptual problems

• Feedback to highlight specific student errors

• Evaluation of resources for exams

• Evaluation of overall student performance

Michigan State University

General Statistics of Class Performance on a Single

Problem

For a given problem• number of students attempting

• number of attempts

• number of correct attempts

• number of attempts per correct answer

Michigan State University

Number of attempts per correct answer

Michigan State University

A Complex Conceptual Problem

Classify Statements14 statements grouped into 6

concepts

One statement is presented from each concept

Each statement must be classified as one of 5 options

All classifications must be correct

Michigan State University

Feedback on a Complex Conceptual Problem

Michigan State University

Final weekFollowing weekBefore discussion

Correct Answers by Concept

Michigan State University

Feedback to Highlight Student Errors

Vector addition problem

Classify statements based on comparison of two quantities

14 statements, 7 of which are the reverse of the other statements, grouped into 5 concepts

Michigan State University

Feedback to Highlight Student Errors

3. Time to row across for K is .. for C

4. Time to row across for C is .. for K

Michigan State University

Evaluation of Resources used on Exams

• Degree of difficultyDIFF = 1 – (earned points / possible points)

• Degree of discriminationClassify students by performance on other exam questionsUse same ratio as aboveDISC = (ratio for good students) – (ratio for bad students)-1 to +1 +1 Only good students earned points 0 everyone did the same -1 Only poor students earned points

Michigan State University

Mid-term Exam Statistics

Problem Exam 1 DIFF

Exam 1 DISC

Exam 2 DIFF

Exam 2 DISC

123456789

1011

0.200.160.400.440.320.000.230.210.360.400.25

0.400.310.400.570.380.000.330.240.630.590.31

0.700.130.190.410.520.180.700.570.550.87

0.240.200.310.570.110.260.360.350.580.14

Michigan State University

Evaluating Class Performance

• Cumulative homework scores

• Cumulative exam scores

Michigan State University

3-D grade plot

Michigan State University

Evaluating Class Performance

• Correlation is moderate (r=0.43)

• Most students do well on homework• Students think doing well on homework means

success in the class

• The instructor implemented a “grade extrapolator” to allow students to interactively pose their future performance and view the resulting grade

Michigan State University

• Presented tools for feedback

• The feedback reflects the complexity of the problems

• More feedback tools needed– different problem types– different presentations of student answers

Conclusion

Michigan State University

Extra Slide - Atwood Problem Analysis, Massless Pulley