LEARNING ANALYTICS & THE LEARNING SCIENCES Zachary Stein, Ed.M, Ed.D (c) Lectica ®, Inc. Harvard...

LEARNING ANALYTICS&

THE LEARNING SCIENCES

Zachary Stein, Ed.M, Ed.D (c)Lectica®, Inc.

Harvard University

OVERVIEW Learning about learning

What is learning? What are the sciences of learning?

Analytics: definitions and approaches Narrowing in on learning analytics and its relatives Lessons from history: testing and technology in US

education

Putting the learning in learning analytics Lectical™ Assessments and the future of educational

assessment

© 2012 Lectica, Inc. all rights reserved

THE SCIENCES OF LEARNING

WHAT IS LEARNING?

There are over a dozen definitions currently in use by researchers and educators From learning as “changing connections between

neurons” to learning as “targeted behavior change” to learning as “the active construction of knowledge that is transferable, applicable, and reflectively held”

There is no one “right” definition, but we should set our terms so we know we are talking about the same thing


WHAT IS LEARNING? Individual researchers and innovators should

define what they mean A look at nearly a dozen papers on learning analytics

revealed no attempts at defining learning Yet taking pedagogical or institutional actions

based on learning analytics data assumes a theory of learning, which is often implicit and inarticulate How do you know what to do to promote learning if you

don’t know how learning works? We should look to the learning sciences for help…


WHAT ARE THE SCIENCES OF LEARNING?

They go way back and they are diverse and complex Philosophy, biology, psychology, neuroscience, etc.

They involves a wide variety of models, methods, and motives The emerging interdisciplinary field of Mind, Brain, and

Education (MBE) exemplifies the new science of learning Here I will look at one of the central orienting

generalization in this field, stemming from the cognitive developmental tradition


LEARNING AS A VIRTUOUS CYCLE

An idea that goes back to the first biologically oriented psychologist

A generalization covering learning processes at multiple levels, from neural-networks, to cognitive development, organizational and group learning (dynamic steering), and scientific investigation.

A process that explains the stage-like development of individual learners through learning sequences of increasing complexity


THE CIRCULAR REACTION: LEARNING AT ITS CORE

Baldwin, Piaget, and a baby in a highchair

think you got it... try it... get feedback.... try again (hopefully

at a higher level)....

Virtuous cycles always include tests


VIRTUOUS CYCLES OF LEARNING

Can occur at multiple levels in educational institutions: Students Teachers Administrators Researchers

Consider the example of a classroom A process we refer to as instructional dynamic

steering™ Note the role of embedded assessment…



LEARNING ANALYTICS REDUX

Let’s think through learning analytics with learning defined as a multi-level virtuous cycle…


ANALYTICS: DEFINITIONS AND APPROACHES

DATA-DRIVEN DECISION MAKING In a wide variety of contexts modern organizations

have become “centers of calculation and measurement” Scientific management Dynamic steering

In educational contexts this has mostly meant the application of business models and an increase in testing Learning analytics steps in as information technology opens

new opportunities for data generation and analysis (e.g., so-called “Big Data”)


LESSONS FROM THE HISTORY OF TESTING

Preferences for automation and economies of scale

Anti-theoretical orientation leaves us satisfied with weak, under-theorized measures

Measurement structures the system Preferences for objectivity increasingly

marginalize the judgment of those on the ground The informational needs of institutional decision-

makers override the learning needs of the students


THE VARIETIES OF LEARNING ANALYTICS

Learner-centric analytics Learning-centric analytics Classroom/cohort analytics Program evaluation analytics Institutional analytics

These can work at cross-purposes As in the history of testing

These can also work in tandem As is the case with Lectical Assessments


PUTTING THE LEARNING IN LEARNING ANALYTICS

CALL FOR RESEARCH BASED ASSESSMENTS National Research Council Committee on the

Foundations of Assessment (1999; 2001) Assessment design requires research

Tests should be built around a theory of learning, not just psychometrics and expert opinion

Goals and standards should be grounded in evidence about student capabilities

However, high-stakes tests continue to be built around psychometrics and expert opinion, with little input from the learning sciences


LECTICAL ASSESSMENTS R&D

Decide on key subject areas and topics in collaboration with educators and researchers

Conduct basic research in order to build empirically- grounded learning sequences that can be translated into low-inference scoring rubrics

Compile and “developmentally curate” related learning materials

The result: an integrated set of assessments and learning materials that support virtuous cycles of learning



MEASURING LEARNING

The Lectical Assessment System Based on Fischer’s Skill Theory (1980; 2006) A domain general, content independent, developmental

assessment system (Dawson, 2004; 2011) Excellent psychometric properties (Dawson-Tunik, Commons,

Wilson, & Fischer, 2005)

The common core metric behind all Lectcial Assessments And the tool used to build them


A LOOK AT THE LDMA

One of 7 existing Lectical assessments for use in higher-education Currently being used as part of large-scale leadership

education program for a major Canadian city. Like all Lectical assessments, the LDMA asks

students to construct short essays, which are then coded using empirically grounded rubrics based on research into a set of relevant learning sequences.

Feedback fosters learning for students, teachers, decision-makers, and researchers


STUDENT LEARN

Lectical Assessments support student learning by… requiring them to operate on concepts and apply skills encouraging reflection on personal performance (see

growth over time) providing rich, targeted, real-time feedback based on

test results, including suggestions for what comes next delivering targeted (and vetted) learning activities








TEACHERS LEARN

Lectical Assessments support teacher learning by… encouraging deep engagement with student performances

(teachers read and score tests, not computers) providing insight into student learning (through online student

and teacher reports) offering specific information, in real time, about what

individual students have learned and what they are most likely to benefit from learning next

providing learning resources that show how to target instruction to individual students (learning resources)



DECISION-MAKERS LEARN

Lectical assessments support decision-maker learning by providing…

complex multi-focal learning profiles, at the cohort, classroom, and organization (in real time)

multiple, repeated measures that reveal trends and trajectories (growth models)



RESEARCHERS LEARN

DiscoTest support educational research by... contributing to basic research about learning during

their construction generating large relational databases of detailed

individual learning profiles trends and trajectories contribute to our understanding of

learning in general

generating overviews of learning at the level of the cohort, classroom, and organization

having data standardized to a common core metric



SUMMING UP

The emerging field of learning analytics can benefit from taking the learning sciences seriously Otherwise, we run the risk of repeating the mistakes

made during the first wave of data-driven decision making in schools

Recent advances in educational assessments technology can be used to catalyze multi-level learning in education intuitions Leveraging true measures of learning to help students,

teachers, administrators and researchers

QUESTIONS?

FOR MORE INFORMATION

www.lectica.org [email protected]

Date post:	02-Jan-2016
Category:	Documents
Upload:	cathleen-ross
View:	213 times
Download:	0 times

LEARNING ANALYTICS & THE LEARNING SCIENCES Zachary Stein, Ed.M, Ed.D (c) Lectica ®, Inc. Harvard...

Documents