1
2
3
James Gee Randi Engle
Jim Pellegrino Susan Goldman John Bransford
Paul Pintrich
4
Situative Theorists at the IRL
5
James Greeno Etienne Wenger John Seely Brown
Allan Collins Jean Lave Roy Pea
IRL’s Seven Principles of Learning
1. Learning is fundamentally social. 2. Knowledge is integrated in the life of
communities. 3. Learning is an act of membership. 4. Knowing depends on engagement in practice. 5. Engagement is inseparable from empowerment. 6. “Failure to learn” is the normal result of
exclusion from participation. 7. We already have a society of lifelong learners.
6
9
11
Joyce Moore
Greeno, J. G., Moore, J. L., & Smith, D. R. (1993). Transfer of situated learning. In D. K. Detterman, & R. Sternberg, (1993). Transfer on trial: Intelligence, cognition, and instruction (pp. 99-167) . Norwood, NJ: AblexPublishing.
PLATO
LOGO
2003-2005
2005-2009
Project NML
2008-2011
Quest Atlantis
2005-2009
GenScope1995-2005
Exploring the “Situative Synthesis”• Motivation
– Hickey, D. T. (2003) Engaged participation vs. marginal non-participation: A stridently sociocultural model of achievement motivation. Elementary School Journal, 103 (4), 401-429
– Filsecker, M., Hickey, D. T. (2014). A multilevel analysis of the effects of external rewards on elementary students’ motivation, engagement, & learning. Computers & Education, 75, 136-148
• Program Evaluation– Hickey, D. T, & Zuiker, S. (2003). A new perspective for evaluating innovative science
learning environments. Science Education, 87, (3) 539-563 • Assessment
– Hickey, D. T., & Zuiker, S. J. (2012). Multi-level assessment for discourse, understanding, and achievement in innovative learning contexts. The Journal of the Learning Sciences, 22, (4) 1-65
– Hickey, D. T. (2015). A situative response to the conundrum of formative assessment. Assessment in Education: Principles, Policies, and Practices, 22 (2), 202-223.
• Networked Online Learning– Hickey, D. T., & Rehak, A. (2013). Wikifolios and participatory assessment for
engagement, understanding, and achievement in online courses. Journal of Educational Media and Hypermedia, 22 (4), 229-263.
• Digital Badges and eCredentials– Casilli, C, & Hickey, D. T. (2016). Transcending conventional credentialing and assessment
paradigms with information-rich digital badges. The Information Society, 32 (2), 117-129.17
Online Learning Research Contexts
2009-2012 in Sakai @IU 2012-present in CourseBuilder BOOC
2013-present in Canvas @ IU IUHS 2013-present in Canvas @ IU GSE
Use public contexts give meaning to knowledge tools
Reward productive disciplinary engagement
Grade artifacts through local reflections
Let individuals assess their understanding privately
Measure aggregated achievement discreetly 19
The BOOC TeamThomas Smith Garrett PoortingaTara Kelly
Xinyi ShenRetno Hendryanti21Karthik Bangera
1. Use “public” contexts to give meaning to knowledge tools
• Problematize knowledge from learners’ perspective– Publically open up the discipline
• Learners define (and redefine) a personally-meaningful context– Can be disciplinary problem, project,
investigation, etc. – Embodies their experience, interests,
& aspirations
RogersHall
22
23
24
25
26
28
New Feature #1Personalized Instructor Videos
31
New Feature #2“Drag & Rank”
Interaction
32
New Feature #3Streamlined Participation
Analytics
33
Public Engagement: Average Words/Wikifolo
2013 2014
34
0
500
1000
1500
2000
2500
3000
3500
1 2 3 4 5 6 7 8 9 10 11
Unit
For Credit(12)
OpenCompleters(10)
Open Drop(54)
0
500
1000
1500
2000
2500
3000
3500
1 2 3 4 5 6 7 8 9 10 11
For Credit
OpenCompleters
OpenDroppers
2. Encourage & reward productivelocal disciplinary engagement
• DE involves declarative knowledge and cultural practices
• PDE connects knowledge to practices, makes connections, finds resources, etc.
• Support PDE so it can be rewarded– Comment directly & locally on wikifolios– Work in the open and encourage “lurking” – No grades or mandatory posts
35
Randi Engle
Problematize Content- From learner’s perspective- Open up issues that experts see as closed
Give Students Authority- Avoid “known answer” questions- Position students as stakeholders
Establish Disciplinary Accountability- Defend positions - Be responsive and respectful
Provide Relevant Resources- Include time and information- Provide easy contextualized access
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Average Comments per Wikifolio
2013 2014
53
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9 10 11
Com
men
ts
Unit
For Credit
OpenCompleters
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9 10 11
Com
men
ts p
er W
ikifo
lio
Unit
For Credit(12)
OpenCompleters(10)
• Disciplinarity0 = non-disciplinary1 = re: education2 = re: assessment3 = re: chapter
• Contextualization0 = not referenced1 = context referenced
54
0
0.5
1
1.5
2
2.5
3
Enrolled(4, 244)
OpenCompleters
(4, 127)
Open Droppers(4, 71)
Disciplinarity and Contextualization of Representative Sample of Comments
Disciplinarity Contextualization
Disciplinary Engagement in 2013
Disciplinary Engagement in 2014
55
00.5
11.5
22.5
3
Disciplinarity Contextuality
• Disciplinarity0 = non-disciplinary1 = re: education2 = re: assessment3 = re: chapter
• Contextualization0 = not referenced1 = context referenced
Grade artifacts through local reflections
• Evaluate reflections on prior engagement.– Contextual engagement (suitability of context
for learning concepts)– Collaborative engagement (what you learned
from others)– Consequential engagement (consequences of
new knowledge)
• Public context but local interactions56
59
60
2015 Results: Individual Local Engagement with Instructor
61
0
100
200
300
400
500
600
700
1 2 3 4 5 6&7 8 9 10 11 12 13 14 15
Averge Words per Reflection
4. Let individuals assessunderstanding privately
• Offer ungraded open-ended items and performance tasks
• Cover the big ideas in the assignment
• Use to re-engage rather than remediate.– Items themselves have little formative
value beyond the correct answer62
PamelaMoss
Use this self-assessment to check your understanding of the big ideas in this assignment. You should complete these items from memory. If you are unable to answer more than one item you are not prepared for the unit exam. If you did poorly, you should go back and re-engage with your classmates, re-read the text, and discuss these ideas more with your peers. Start with the highly promoted work on the weekly public feedback.
5. Measure achievement discreetly
• Standards-oriented tests have narrow uses– Measure gains using pretests, compare instruction,
document improvement– Primary student function is motivating prior
engagement and catching slackers
• Refine tests (but don’t teach to them)– Use the item analysis routines in your LMS– Aim for about 85%, normal distribution, 1-2 @100%
• Protect test security– Withhold item-level feedback
• Randomly selected timed items (~1 min)– Include difficult “best answer” items– Don’t put searchable terms in item stem
Me!
Disciplinary Achievement 2014 & 2015
65
0
10
20
30
40
50
60
70
80
90
100
Practices Principles Policies Final
For Credit Open Completers
0
10
20
30
40
50
60
70
80
90
100
AssessmentPractices
AssessmentPrinciples
AssessmentPolicies
ComprehensiveFinal
All Students For Credit (8)
Self-Paced Course for 2016New Tools for Finding Peers
66
Self-Paced Course for
2016
New Tools for Archiving Wikifolios
67
69
Connie Yowell
Mimi Itow
DIGITAL BADGES CONTAIN INFOClaims & evidence; links to more
● Contain specific claims of competence
● Contain evidence supporting claims
● Links to more evidence including student work
● This is different from conventional credentials
● Can link to unlimited additional information
● It matters what claims and evidence are included
● Process is transformative and/or disruptive
OPEN BADGES CIRCULATEInfo circulates in value networks● Credibility of claims and
validity of evidence gets “crowdsourced”
● Earners can find opportunities and promote achievements
● Institutions can recognize broader accomplishments
● Employers can communicate needs and find skills
● Amplifies meaning of claims and evidence
● Amplifies deliberation over nature of claims and evidence
• Studied 2012 DML content awardees
• 600 badge content proposals
• 29 content developers supported
• 3 platforms supported
RebeccaItow
CathyTran
KaterinaSchenke
JamesWillis
Nate Otto
ChristineChow
JoshuaQuick
Cindy Cogswell
Accredited Context
Accredited Badges
None
Other
School or University
After School
Teacher PD
Informal & Other
Adult & Career
MuseumExtra-
Curricular
Common Core
Local Standards
Partners for 21CS
None Identified
K-12
Middle School
Secondary
College Students
Educators/ Other Vocational
/ Adult
AccreditationSetting
StandardsEarners
29 Diverse Approaches to Supporting, Assessing, & Recognizing Learning
• Seven “associationist” efforts– Self-paced mastery of specific competencies (CBE &
gamification), individual recognition• Twelve “constructivist” efforts
– Cohorted & self-paced, problem-based, performance & portfolio assessment, individual recognition.
• Five “situative” efforts– Cohorted, networked learning, peer & group
recognition• Five “hybrid” efforts
– All three types of practices
76
Capturing “Practical Wisdom”
• Aristotle’s Phronesis– Contextual and value-
laden
• Bent Flyvbjerg (2001)– Phronetic narratives
• Rich Halverson (2004)– Artifact-based
phronetic narratives
Categories of Badge Functions
Recognizing Learning
Assessing Learning
Motivating Learning
Studying Learning
Emergence of Badge Design Principles
Draft Initial Principles
Formalize General Principles
Bookmark Research
DPD PROJECTFINDINGS ACROSS BADGE EFFORTS
• Documented apparent badge and ecosystem status– Badges system in 2014 (none, different, or
intended)– Ecosystem in 2014 (none, different, or
intended)– Ecosystem in 2015 (none/paused,
existing, or thriving)
Badges appear to work better… In some places than others
Badges appear to work better… with sociocultural and hybrid learning.
How did badges work…where learning was competency-based?• Seven projects were CBE
– Self-paced, rejected “seat time”
– Mostly for course credit– Summative assessment (of
specific competencies)– Extrinsic motivation (i.e.,
credit, points, etc.• Competency badges
– Measurable and measured competencies
How did badges work…where learning was competency-based?
• SA&FS was suspended• All three Gates Project Mastery
awardees were all suspended– Pathways to Global Competency (Asia Society)– Level Up (Colorado) – Youth Film Maker (Philadelphia)
• Manufacturing Institute– SkillsUSA badge suspended– PLTW badge is seldom earned
• YALSA badges are seldom earned• Buzzmath badges are not open but
thriving
• Equating evidence with conventional criteria
• Determining who can authorize credit
• Maintaining a common definition of proficiency
• Building a sustainable model• Equitable education• Barriers to efficiency
– Technical– Financial barriers– Logistical barriers
Challenges that Rand Corp Evaluation Uncovered for Gates Project Mastery Projects
How did open badges work…where learning was participatory?
• Five project emphasized engaged participation– Cohorted course-based contexts– Crowdsourced recognition (e.g,
communal badges)– Transformative assessments– Social motivation (e.g., team
goals)
• Participation Badges – For assuming roles and identities
How did open badges work…where learning was participatory?
• MOUSE network mentors high school tech support– Thriving, many successful grads
• S2R network for high school media producers– Thriving, many successful grads
• PBS Student Reporting Labs for high school media– Thriving, many successful grads
• Cooper-Hewitt/Smithsonian Design Prep helps high schoolers develop design portfolios– Stated out more competency-based badges but thriving with
role-based participatory badges
• Design for America for undergraduate social design and engineering projects– Implemented Digital Loft badges– Program thriving but badges appear paused
How did open badge work…where learning was constructivist?
• Twelve projects emphasized more constructivist practices– Inquiry-oriented and project-based
contexts– Informal recognition– Formative assessment (e.g., open-
ended, rubrics, etc.)– Intrinsic motivation
• Completion badges– Assessable and assessed
• Assessment was a big challenge– More modest assessments and scaled
back assessment in thriving projects
How did open badges work…where practices were hybrid?
• Five projects were hybrids • Generally were broader
learning contexts– Associationist, constructivist,
and sociocultural activities– Mix of recognition and
assessment practices– Various forms of motivation
• Competency, Completion, and/or Participation Badges
• Assessment again a challenge– Thriving projects scaled back or
automated assessment
Six Arguments About Motivating Networked Learning with Open Badges
1. The information in badges is inherently meaningful2. Open information is even more meaningful3. Open credentials are particularly disruptive and
transformative 4. The intrinsic/extrinsic dichotomy is overstated5. The negative consequences of extrinsic rewards is
overstated6. Situative models of engagement are ideal for motivating
learning in digital networks.
90
Hickey, D. T., & Schenke, K. (in preparation). Thinking about motivation when learning is accomplished, assessed, and recognized in digital networks. In K. A. Renninger & S. E Hidi, The Cambridge Handbook of Motivation and Learning. Cambridge, MA: Cambridge University Press
91
92
93
Current Status of ePortfolios & Badges
Integration
Integration w/employer search
LRN in development
Integration
Open Badge Factory default
Tutors issue Credley Badges
Badges in development
“Interested” 94
+
+
+
95
Exploring the “Situative Synthesis”• Motivation
– Hickey, D. T. (2003) Engaged participation vs. marginal non-participation: A stridently sociocultural model of achievement motivation. Elementary School Journal, 103 (4), 401-429
– Filsecker, M., Hickey, D. T. (2014). A multilevel analysis of the effects of external rewards on elementary students’ motivation, engagement, & learning. Computers & Education, 75, 136-148
• Program Evaluation– Hickey, D. T, & Zuiker, S. (2003). A new perspective for evaluating innovative science
learning environments. Science Education, 87, (3) 539-563 • Assessment
– Hickey, D. T., & Zuiker, S. J. (2012). Multi-level assessment for discourse, understanding, and achievement in innovative learning contexts. The Journal of the Learning Sciences, 22, (4) 1-65
– Hickey, D. T. (2015). A situative response to the conundrum of formative assessment. Assessment in Education: Principles, Policies, and Practices, 22 (2), 202-223.
• Networked Online Learning– Hickey, D. T., & Rehak, A. (2013). Wikifolios and participatory assessment for
engagement, understanding, and achievement in online courses. Journal of Educational Media and Hypermedia, 22 (4), 229-263.
• Digital Badges and eCredentials– Casilli, C, & Hickey, D. T. (2016). Transcending conventional credentialing and assessment
paradigms with information-rich digital badges. The Information Society, 32 (2), 117-129.96