Dr. Cath ELLIS
University of Huddersfield, United Kingdom
Using Grading Analytics to Improve Student Learning
Presented by:
Using Grademark Analytics to Improve Student Learning
Dr Cath Ellis
University of Huddersfield
Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimising, learning and the environment in which it occurs.
(LAK cited in Ferguson)
Implicit within this definition are the assumptions that learning analytics make use of pre-existing, machine-readable data, that its techniques can be used to handle large data sets of data that would not be practicable to deal with manually. (Ferguson)
Business intelligence
Learning Analytics
Assessment analytics
Social Network Analytics
Discourse analytics
Nudge analytics
Academic Analytics
Content Analytics
Educational Data Mining
Accountability
Transparency
Improvement
Improvement
Informed decision making
Reduced attrition
Good degree results
Operationalisation
Staff resistance
Online learning only
Impenetrable
Depth
• Granularity
Breadth
• Large data sets
Assessment Analytics
Degree attainment
Progression
Module
Individual assessment
Learning outcome
Errors/strengths
Persistence Improvement
Assessment analytics: Blind spot
Analytics instead of assessment
• Constructivist learning theory
Data reliability
• Inter- and intra-rater reliability
• Validity of assessment design
Data availability
• Paper-based marking
• Not enough granularity
Electronic
Assessment
Management
E
A
M
Institution
Evaluation Retention Recruitment
Academic staff
Expertise Adds value Informs practice
Students
Meaningful Investment Guides behaviour
Assessment analytics
Marking
• Final mark
• Assessment criteria
• Common comments/errors
Feedback
• Final mark
• Assessment criteria
• Map Learning Outcomes
Originality checking
• Unoriginal text sources
Automatic marking
• MCQ
• Short answer/free text
Certification
• Discriminate between levels of achievement and students
• Selection for further study and employment
• License to practice
Student Learning
• Motivating students, steering their approach
• Inform teaching strategies
Quality Assurance
• Evidence for stakeholders
• Judge standards
Lifelong Learning
• Encourage skills development
• Self-evaluation and self-regulation
Why assess? Assessment of
learning Assessment for
learning
Bloxham and Boyd 2007
Motivating Students
Skills development
Discriminate between levels
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Criterion 5
Criterion 4
Criterion 3
Criterion 2
Criterion 1
% Students
Ru
bri
c C
rite
ria
ICCT Summative One Rubric Results 2010-11
Fail
3rd
2.2
2.1
1st
Inform teaching strategies
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Criterion 1 09-10
Criterion 1 10-11
Criterion 2 09-10
Criterion 2 10-11
Criterion 3 09-10
Criterion 3 10-11
Criterion 4 09-10
Criterion 4 10-11
Criterion 5 09-10
Criterion 5 10-11
% Students
Ru
bri
c C
rite
ria
2009-10/2010-11 Rubric Result Comparison
1st
2.1
2.2
3rd
fail
Motivating students
License to practice
Evidence for stakeholders
Judge standards
Self-evaluation
Granularity
References
• Bloxham, Sue, and Pete Boyd. 2007. Developing Assessment in Higher Education: A Practical Guide. 1st ed. Open University Press.
• Ferguson, R. 2012. The State of Learning Analytics in 2012: A Review and Future Challenges. Technical Report. UK: Knowledge Media Institute, The Open University.