Date post: | 13-Jun-2015 |
Category: |
Education |
Upload: | joseph-jay-williams |
View: | 118 times |
Download: | 1 times |
1
Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning,
Financial Success, and Cognitive & Learning Sciences Research
Joseph Jay WilliamsLytics Lab, Graduate School of Education
Office of the Vice Provost for Online Learning, [email protected]
www.josephjaywilliams.com
[Originally from Trinidad & Tobago – apologies for difficulty in following accent.]
2
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
3
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
4
I. Motivation
• Beliefs about Intelligence• To what extent do you agree?
How smart you are can be significantly changed by your effort and hard work.You have a certain amount of intelligence, and what you do doesn’t really change that much.
• Growth Mindset vs. Fixed Mindset• Dweck, 2008; Yeager & Walton, 2012• Williams, Paunesku, Haley & Sohl-Dickstein, 2013
5
MOOClet 1: Exercise (Khan Academy)
1. Number of Problems Practiced
2. Accuracy
[TEXT SHOWN HERE – Randomize]
6
Embed motivational messages
Practice-as-usual
Remember, the more you practice the smarter you become!
Growth Mindset Message
7
Growth Mindset Message helpful?
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Practice-as-usual
Jascha Sohl-Dickstein
8
Results: More motivated?
• Growth Mindset Message > Practice-as-Usual
• reduction in dropout rate• increase in Number of problems practiced • increase in Number Correct
• Accuracy: Problems correct/Problems attempted
• Accuracy improvement
9
Interpreting results
• Obvious?• Skeptical? Anything’s statistically
significant with large N…• Second experiment (run simultaneously)
10
Practice-as-usualGrowth Mindset Message
Positive Message
Different motivational messages
Some of these problems are hard. Do your best!Remember, the more you practice the smarter you become!
11
Does any positive message work?
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Positive Message• "Some of these problems are
hard. Just do your best."• "This might be a tough problem,
but we know you can do it.”
• Practice-as-usual
12
Effects of Positive Messages?
• Positive Messages not better than Practice-as-Usual!
• Growth Mindset > Positive Message• How intuitive is this result?• Teachers?• Ongoing: After incorrect answer• www.assistments.org Neil Heffernan• Gates Grant for Modeling Research
Rapid Experimentation: tiny.cc/mechanicalturk
13
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
14
Constraint Satisfaction
Practical ProductResearch
Financial Value
“MOOClets” or Modular Online Resources
• E.g. MOOClet: Math exercise, 5 minute interactive video, 30 minute lesson, Study Strategy coach, Quiz/Assessment
• www.josephjaywilliams.com/mooclets • MOOClets Group Tue 12-1 PST (remote)• tiny.cc/joinmooclets mailing list & updates• Williams, J.J. & Williams, B. A. (2013). Randomized
Experiments as a Methodological and Conceptual Tool for Improving Design of Online Learning. Data Driven Education, NIPS Workshop. tiny.cc/experimentsonlinelearning
15
16
Integrate Diverse Expertise & Innovations
17
Where is relevant research, best practice, technology, funding, market? tiny.cc/mapofspace, tiny.cc/stanfordmap
MOOCs (Coursera, EdX, Udacity)
www.KhanAcademy.org www.Assistments.org
K-12 Ed-Tech space (Bay Area, Boston...)(www.edsurge.com)
(www.edudemic.com )Higher-Ed Technology/LMS
E-Learning & Training forCorporate, State, Federal, Army
Psychological ScientistsComputer Scientists
Education
Corporate Training
Funding Agencies:IES, Dept. of Education, Dept. of Labor,
NIH, NSFPractical Reports:
What Works ClearinghouseIES Practice Guides
University Teaching & Learning CentersDepartment of Education reports
Think Tanks:SRI International
WestEd
Pittsburgh Science of Learning Center
Learning Sciences
Mooc Research Initiative
Open Learning Initiative
Distance LearningAthabasca, Western Governor’s University
Economics of Education
Habit & Behavior Change
Medical Education
Centers/Scholarship of Teaching & Learning
Assessment
PearsonETS
Open Education ResourcesCreative Commons
Foundations:Gates, Spencer, Hewlett, Kresge
Venture CapitalPrivate VCs
New SchoolsImpact Investing
Accelerators:Imagine K-12,
HCI, Design Thinking
Workforce Development
18
Relevant Resources
• www.josephjaywilliams.com/home/useful-software
• www.josephjaywilliams.com/education• www.josephjaywilliams.com/online-education-exp
eriments• tiny.cc/mapofspace• Applying Cognitive Science to Online
Learning. Paper presented at NIPS Workshop on Data Driven Education (tiny.cc/cognitivescienceinmoocs)
• Improving Outcome of Psychosocial Treatments by Enhancing Memory and Learning. Perspectives in Psychological Science.
19
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
Messages in Coursera?
20
Week 6 QuizWhether or not you’re correct on the first try, each attempt at a tough problem can make you better at statistics.
21
Motivational Introductions
Practice-as-usualHave fun in the course!
Anne Friedlander Growth MindsetText version of Mindset MessageMindset Message – Another InstructorPositive EncouragementBelonging
22
Instructor Guide to foster Growth Mindset• tiny.cc/idstrategymindset
Where do you think you will apply this?
Do you have any questions or want clarification?
Click to see any of the Case Studies below
Would you like to?
23
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
24
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
II. Learning through Questions/Assessments
• Discovery of patterns(Williams & Lombrozo, 2010, Cognitive Science)
• Use of prior knowledge(Williams & Lombrozo, 2013, Cognitive Psychology)
• Children’s causal learning (Walker, Williams, Lombrozo, Gopnik, 2012)
• Overgeneralize at expense of specific facts
(Williams et al, 2013, Journal of Experimental Psychology: General)
• Contradictions & belief revision (Williams, et al 2013; 2014; Cog Sci, CHI WiP)
• Comparison of examples (Edwards, Williams, Lombrozo, Gentner, 2013)
GLORP DRENTOnline/Blended
Educational Resources
26
Adding questions/activities
• tiny.cc/augmentedvideo• tiny.cc/augmentedexercise
27
III. General Learning & Problem-Solving Skills
Learning/Self-Questioning Strategies
• Specific Prompt VS.• General Strategy of Self-Questioning?• Teach strategies
(McNamara, 2004, Discourse Processes; Palinscar & Brown, 1984, Cognition and instruction; Williams & Lombrozo, 2010, Cognitive Science)
• Habit & Behavior Change • Theory & Technology: Embed & Remind• Methodology: “longitudinal”, large N
28e.g. Weinstein & Mayer, 1986 Rothman, Baldwin, & Hertel, 2004
29
Embedded “On-Request” Strategy Instruction
Clickable link.
[Click here to learn about the “What? Why? How?” strategy]
30
Self-questioning strategy: What? Why? How?
31
Cues – Reflect & Remind
32
Preliminary Results
N = ~ 800 per condition
• Prompted Problem-solving better than Practice-as-usual.
• Increased Accuracy on intervention and post-intervention problems.
33
Learning Assistant
• tiny.cc/learningassistant
34
Learning Assistant – e.g. 1
35
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
36
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
37
IV. Performance Support Tools & Checklists
38
Coach/Support Learning Behaviorstiny.cc/kalearningcoach
39
A Lesson: Concept + Digital Tool to apply Concept in Future
40
Tool for Difficult Conversations
41
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
42
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
43
Acknowledgements
• Tania Lombrozo & Tom Griffiths• John Mitchell & Candace Thille• Jascha Sohl-Dickstein• Jace Kohlmeier & Khan Academy• Sam Maldonado• Lytics Lab (lytics.stanford.edu)• VPOL (Vice Provost for Online
Learning, online.stanford.edu)