Adaptive Learning – The Next GenerationJune 8, 2011

2

• Integrating the design, building, monitoring, and improvement of learning environments; individualize learning experiences using our scale; and, ultimately, drive greater student career success.

• Former CLO for K12, Inc. – structured use of technology, cognitive science, on-line and off-line materials for 1,700 teachers, 55k students

• Former Publisher and General Manager for DK Multimedia, Inc.• Management consultant with McKinsey & Company • Education:

- Ph.D. in Electrical Engineering and Computer Science from MIT- M.D. from Harvard Medical School- M.A. in Electrical Engineering and Computer Science from MIT- M.A. in Mathematics from Oxford University- B.S. in Electrical Engineering and B.S. with Honors in Mathematics from the

University of Washington

Bror SaxbergChief Learning Officer, Kaplan, Inc.

3

• Kaplan University• Kaplan Higher Ed Campuses • Kaplan Legal Education• Kaplan Professional Education• Nursing

• KTPA• Kaplan Tutoring • Kaplan Bar Review • Kaplan Publishing• Kaplan Continuing Education

• Kaplan Higher Ed – Europe• Kaplan Professional – Europe• Kaplan Int’l Colleges• Global Pathways/English

Language• Kaplan Higher Ed – Asia• Kaplan Professional – Asia• Kaplan Higher Ed – Australia• Kaplan Professional – Australia• In Country Pathways – China

• Kaplan Compliance Solutions• Kaplan EduNeering• Kaplan IT Learning• Kaplan Latin America• Education Connection• Kidum• Colloquy

(U.S.)(U.S.)

Kaplan education spans domains and geography

4

Student’s wants and needs for learning align

PersonalizedPersonalized Effective & Efficient

Effective & Efficient InnovativeInnovative Lifelong

LearningLifelong Learning

Student FirstStudent First

An expert view: “Your fastest path to goals that matter”

Student’s view of what the best educator would be like:

• Targeted• Diagnostic• Adaptive• Flexible• Career-long

• Targeted• Diagnostic• Adaptive• Flexible• Career-long

• Targeted• Efficient

• Targeted• Efficient

• Diagnostic• Adaptive• Engaging• Flexible

• Diagnostic• Adaptive• Engaging• Flexible

• Flexible• Career-long

• Flexible• Career-long

5

Today…Today, students are treated as if they are the same.

The programs focus on graduation criteria.

To be successful, students’ fluencies should match evolving expert work

There is a disconnect that needs to be addressed.

6

2020…

Cognitive task analysis can objectively identify the evolving skills students need for their next stage

The Kaplan Way to Design and Deliver, then Measure and Evaluate will drive better outcomes

Cognitive Task Analysis

Designand

Deliver

Measureand

Evaluate

7

2020…We can then focus on unique strengths and challenges for each student to reach success.

Cognitive Task Analysis

Adaptive learning lets us personalize the educational experience, matching pace, progress and motivation

8

We have to do more than “normal” development

“Normal” course development process

• Look at existing materials for inspiration

• Ask individual faculty to make “best guess” changes

• Possibly pilot for user acceptance

• Build a new version with incremental changes

• Distribute immediately

Augments to drive excellence

• Explicitly connect with learning science to drive what to do

• Tie explicitly to research on what defines experts’ success

• Use history of previous learners to alter current instruction

• Personalize instruction based on student skills and needs

• Collect data on what works by testing improvements

9

Instructional design drives effectiveness

Learning Events

(hidden - inside students’ minds)

StudentPerformance

(observable -indicates

knowledge)

Instructional Events

(in the learning environment)

Knowledge

• Explicit: Information, Explanation, Examples, Demos

• Implicit: Practice tasks/activities (prompts and response)

• Diagnosis and feedback

• Explicit/Declarative/Conceptual/What• Implicit/Procedural/How• Knowledge Components

(Procedures + Facts, Concepts, Principles, Processes)

• Response accuracy/errors• Response fluency/speed• Number of trials• Amount of assistance (hints)• Reasoning

Motivation

• Orientation/Inoculation• Monitoring • Diagnosis and treatment:

Persuasion, Modeling, Dissonance

• Value beliefs• Self-efficacy beliefs• Attribution beliefs• (Emotions)

• Behavior related to• Starting • Persisting• Mental Effort

• Self-reported beliefs

Metacognition• Structure• Guidance

• Planning, Monitoring• Selecting, Connecting

• Amount of guidance required/requested

Instructional Design is the design of external conditions to support the internal conditions necessary for learning. Robert Gagne, 1965

Koedinger, K.R., Corbett, A.T., and Perfetti, C. (2010). The Knowledge-Learning-Instruction (KLI) Framework: Toward Bridging the Science-Practice Chasm to Enhance Robust Student Learning (Draft manuscript from the Pittsburgh Science of Learning Center)

10

Evidence-Driven Instructional Design

Guidance (for motivation and metacognition)

The evidence about learning points to a sequence of activities that optimizes learning.

10

Design goes one way. . ..

. . .delivery the other.

11

Best education will require investments up front Examples:

Cognitive Task analysis (CTA)

CTA for program design: $20,000 (internal resources)

CTA for course design: $5,000

[NOT necessarily for every program, every course!]

Interactive Media

Java and Flash-rich training: $5,000+ per hour of rich instruction

Complex simulations: $25,000+ per hour of simulation

[NOT necessarily for every hour of instruction!]

Platforms

Adaptive E-Learning Platform development: $10-20M (5 years)

Content rebuilds: Could be multiples of that!

12

Spending at scale: other industries are used to it

12.7%11.2%

7.6%

4.3% 4.1% 4.1%

2.3% 2.1% 1.9% 1.6% 1.5%

0%

2%

4%

6%

8%

10%

12%

14%

R&D to Sales Ratio by Industry (2004)

Avg. = 4.2%

Source: Booze Allen Hamilton Global Innovation 1000

Education organizations typically do not even report R&D

13

Concept Course Project Goals

• Develop a “concept course”, a prototype to showcase elements of Kaplan’s next generation learning environments

• Personalization

• Evidence-based instructional and multimedia design

• Open-source platform

• Mobile delivery (iPad)

13

14

Personalization: triple loop adaptation

Context forExamples, Practice,Tests

MotivationGuidance

Student’s Motivational State

Student’s Performance

Amount and Type of Examples, Practice,

Feedback

14

3.

2.

1.

Student’s Field of Study

• Personalizing content based on diagnosis of knowledge gaps improves learning outcomes and reduces the time to learn.• We can double the impact of adaptive learning by adding motivational guidance for students who need it.• This is the first scalable system that adapts to both student knowledge and motivation.

15

Rules EngineRules

Engine

StudentModel

StudentModel

Current and pastPerformance& Motivation

data

Example AExample BExample C

Activity AActivity BActivity C

Feedback AFeedback BFeedback C

CROSS-OBJECTMotivational Guidance

Content

PreviewInformation Example B Activity A AssessmentFeedback C

Prepare Practice Test

Motivational Guidance

For each learning outcome

Dynamically constructed learning objects and guidance

15

16

Learner interface, guidance, rules

17

Motivation theory: Beliefs drive behavior and performance

Sources: Bandura; Eccles & Wigfield; Pintrich & Schunk; Clark; Dweck

Self-Efficacy

Eff

ort

HighModerateLow

Will

Low High

SkillHigh

Low

17

18

Motivation data

Value• Intrinsic• Utility• Strengths

Self-Efficacy• Success• Distractions• Difficulty

18

Confidence

Course Overview

Prepare: After first example

Practice: After low performance

Attribution

• Controllable

• Uncontrollable

Time spent on each learning outcome

Indicator of mental effort

19

Rules and guidance for different patterns of skill and will

Perceived Value Type of Guidance Given

Low Explain and model the value (benefits, risks)

Performance Self-Efficacy Attribution Type of Guidance Given

High All NA Proceed to Test

Low High Controllable Reduce confidence, increase effort

Low High Uncontrollable Reattribute to effort or different strategy

Low Low Controllable Boost confidence

Low Low Uncontrollable Reattribute to effort or strategy and boost confidence

Low Med Controllable Motivation OK. Recommend more practice

Low Med Uncontrollable Reattribute to effort or different strategy

Performance Outcome Session Time Type of Guidance Given

Low Low Focus on spending more time

Low High Focus on using time more productively – provide training in study skills

Course overview

Test

19

Practice

20

Content: Why Creativity?

IBM poll of 1,500 CEOs identified creativity as the number one leadership competency needed to fuel business growth. On a global scale, creativity holds the potential for solving society’s largest problems. We need faster ways to develop this capability in the workforce.

Video footage from leading innovators of our time from Techonomy conference

20

21

Content: Differentiators of individual and team creativity

Notes1. Sources for “task analysis”:• Empirical investigations of highly creative people/teams• Research comparing people/teams with high creativity versus average creativity• Studies of performance on creative measures• Successful programs that teach creative behavior• Videos collected during 2010 Techonomy conference

2. Graphic modified from Amabile, 1998.3. Course will focus on Cognitive Skills and Motivation.4. * Fluid intelligence (Gf) is the ability to reason and to solve new

problems independently of previously acquired knowledge. It is the ability to analyze novel problems, and identify patterns and relationships that underpin these problems. It is correlated with working memory capacity and inductive reasoning.

Creativity

DomainExpertiseDomain

Expertise

FluidIntelligence*

FluidIntelligence*

MotivationMotivation

Cognitive Skills•Problem Definition•Divergence•Convergence•Execution

Cognitive Skills•Problem Definition•Divergence•Convergence•Execution

Individual

Processes and Culture that foster creativity (e.g., doesn’t punish risk-taking)

Organizational Structure promotes networking

Team members with diverse domain expertise

Autonomy in decision making, yet goal oriented

Tea m

21

22

Content: 7 Units - procedural learning outcomesUnit Learning Outcomes

What is Creativity?

• Define creativity, identify differentiators of creativity and describe a process to apply them

• Identify the myths and truths about becoming more creative

Problem Definition

• Define a problem as a gap

• Identify causes of the gap

Divergence

• Brainstorm solutions

• Identify and challenge assumptions

• Seek analogies and associations

Convergence• Narrow the field of possible solutions using critical thinking skills

• Recognize real-world constraints

Execution• Turn ideas into executable plans

• Develop prototypes

Motivation • Apply techniques to remain motivated during the creative process

Group Creativity

• Apply individual creativity techniques to working in groups

• As a leader, create an environment conducive to creativityGreen = Prototype

22

23

DEMO

23

24

Useful References on Learning and ID:• Why Students Don’t Like School, Daniel Willingham – highly readable! ;-) • Talent is Overrated, Geoffrey Colvin – highly readable! ;-) • E-Learning and the Science of Instruction, Clark and Mayer, 2nd ed.• “First Principles of Learning,” Merrill, D., in Reigeluth, C. M. & Carr, A. (Eds.), Instructional Design

Theories and Models III, 2009.• How People Learn, John Bransford et al, eds.• “Design factors for educationally effective animations and simulations,” Plass, J.L., Homer, B.D., Hayward,

E.O., J Comput High Educ (2009) 21:31–61• “The Implications of Research on Expertise for Curriculum and Pedagogy”, David Feldon, Education

Psychology Review (2007) 19:91–110• “Cognitive Task Analysis,” Clark, R.E., Feldon, D., van Merrienboer, J., Yates, K., and Early, S.. in Spector,

J.M., Merrill, M.D., van Merrienboer, J. J. G., & Driscoll, M. P. (Eds.), Handbook of research on educational communciatinos and technology (3rd ed., 2007) Lawrence Erlbaum Associates