Learning and analytics – where do the two meet?

Simon Knight @sjgknightImage from http://xkcd.com/903/ licensed under a Creative Commons Attribution-NonCommercial 2.5 License.

Interpretive flexibility• The basic question is not

what can we measure? The basic question is what does a good education look like? (Gardner Campbell)

• Do we value what we can measure, or measure what we really value? (Guy Claxton, BBC Radio 4 Education Debate, Nov. 2012) 84

Introduction to Epistemology, Pedagogy & Assessment

The Triad: Bounding the middle space

Foreground relationships between:• epistemology (the nature of knowledge)• assessment (of learnt knowledge?)• pedagogy (the nature of learning)

Key questions• What does it mean to

know?• How do we decide

(assess) if someone knows or not?

• How do we get people to come to know (to learn)?

Bounding the middle space1. LA ‘buy in’ to ways of

thinking about epistemology, assessment and pedagogy

2. For theoretical, practical, ethical reasons we should engage in these debates

3. These considerations have practical implications – the middle ground

Why does epistemology matter?“…assessment is one area where notions of truth, accuracy and fairness have a very practical purchase in everyday life”

(Williams, 1998, p. 221).

Why does epistemology matter?“…assessment is one area where notions of truth, accuracy and fairness have a very practical purchase in everyday life”

(Williams, 1998, p. 221).

• LA ‘buy in’ to particular ways of thinking about these issues, & are embedded in systems, but they might be flexible enough to move beyond the current impasse

Learning Analytics

• Data mining• Digital trace –

including linguistic data

• Deployed for pedagogic purposes

• Big and rich

Epistemology & Assessmentan example

Danish exams with internet access

http://news.bbc.co.uk/1/hi/education/8341589.stm

Danish exams with internet access

• Allows testing of problem-solving and analysis - sifting information

• "if you allow communication, discussions, searches and so on, you eliminate cheating because it's not cheating any more. That is the way we should think."

Epistemological assumptions

Danish exams with internet access

• Potential for auto-grading – LA role, a new model?http://www.timeshighereducation.co.uk/416090.article

A role for LA?

Epistemic assumptions

- It isn’t “knowledge” to recall facts- Knowledge lies in the use

Assessment…Formative or summative?What is being assessed?

Pedagogic assumptions

• Assessment is used in teaching (but doesn’t drive it)• Pedagogy should involve knowledge practices – not

assessment practices• Discourse is fundamental

Technological determinism?

• Technology is situated• Interpretive flexibility

The other side of the coin

Multiple perspectives

• One side: Direct technological affordances• The other: The practices around technologies– Educator practices– Wider policy/accountabilitycontext– Student practices

Educator Practices: LA potential

• E.g. at LAK14: Wise, Piety & Hickey

Policy: LA potential

• Accountability systems, educator support, investment in AfL, setting priorities (e.g. Denmark)

Student Practices: LA potential

• Analytics give unprecedented (?) access to “What students do” - & assumptions around this

• Potential shift, from standardised assessments – need for new (psychometric) models

LA in Structured Knowledge Building

• Strong CSCL tradition to ‘make explicit’ in structured environments (Knowledge Forum, Belvedere, etc.)

Natural Language Processing based LA

• Automated essay feedback• Dialogue scaffolds and automated tutors• Social functions – “if you’re interested in x,

you should talk to …”• Tutor support “student a might need some

help on…”

Hot Topics

• Analytics for student ‘dispositions’• ‘Personalisation’ through community building – Social networks– Discourse support– Community knowledge

• Educator support, visualising data

Conclusions1. Context of LA as assessment, and

pedagogic aid, in context of policy considerations – Danish example

2. Educator practices matter3. Hope to improve student

practices4. Use of data for particular

conversations/assessment/ AfL, is key

5. Design implications – foreground particular facets of data & activity

Thank you

@sjgknightsjgknight@gmail.comhttp://people.kmi.open.ac.uk/knight/

Our papers in this area:• http://oro.open.ac.uk/39226/ Epistemology, assessment, pedagogy:

where learning meets analytics in the middle space (2014)• http://events.kmi.open.ac.uk/icls-analytics/ ICLS workshop on

analytics for learning and becoming in practice (2014)

• Discourse, computation and context – sociocultural DCLA revisited (2013) http://oro.open.ac.uk/36640/

• Tracking epistemic beliefs and sensemaking in collaborative information retrieval (2013) http://oro.open.ac.uk/36553/

Acknowledgements

Thanks to my co-authors and supervisors Simon Buckingham Shum and Karen Littleton for their work on this and our workshop papers.

Thanks to Cindy Kerawalla and anonymous reviewers for their helpful suggestions on the earlier (LAK13) version of this paper.

Images – mostly from Wellcome Images http://wellcomeimages.org/ under CC licence