Date post: | 13-Apr-2017 |
Category: |
Education |
Upload: | greg-bybee |
View: | 264 times |
Download: | 1 times |
Sharin’ Ain’t “Social”The Science of Social Learning
Greg BybeeVP, Learning Solutions, NovoEd
March 17, 2016
Introduction
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
I’m an education technologist
● McKinsey & Company Consultant● Coursera, Product Manager● Renren, Education Business Development● NewSchools Venture Fund, EdTech Fellow● NovoEd, Learning Products (2013-present)
At NovoEd, I lead our learning solutions team
● Learning experience design (and ISD)● Learning solution strategy consulting● Course and program operations and facilitation● Customer success and support
I have a diverse background in training
● MA-Ed & MBA @ Stanford● Chair, Student Achievement @ Leadership High● Instructor @ 辰熙中英文学校 (China)● School Building @ ILAE in Ethiopia● Education Market Consultant and Volunteer
Hi, I’m Greg
Nice to meet you, too!
Introduction
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
Group Activity!!
● Form groups of 4-8.
● Answer the question: “what makes for an effective adult learning experience”○ Focus not on outcomes of the
learning, but the definition of the learning process
○ You’ll have 8 minutes.
● Add your ideas as text here:http://padlet.com/novoed/ls16
Weren’t you supposed to tell us that?
Source: Learning Solutions Session, March 17, 2016
Social collaborative
- Metacognition, teaching- Multiple perspectives- Peer-to-peer adaptivity- Felt accountability
Construction and Inquiry
- Problem-Based- Project-Based- Role plays, scenarios- Case method
Engaging
- Intrinsically Motivated- Relevant, authentic
My list(Okay admit I took more than 8 minutes...)
Experiential
+
Coherent
Self-directed, autonomous
Mastery, goal attainment, progression
Applied, relevant, authentic, purposeful
Engaging, fun, captivating
Inquiry-based, discovered
Constructed, created, explored
Scaffolded, workshopped
Interactive, multimodal, multisensory
“Knowing, being, doing”
Accountable, measured, transparent
Practiced, reinforced, rehearsed
Leveled, ZPD, personalized, adaptive
Chunked, cognitive load, working memory
Coherent, sequenced, schematicCurricular Design
- Sequencing, chunking- Scaffolded, coherent
Malcolm Knowles’ Theory of Andragogy
Assumptions1. Self-concept2. Experience reservoir3. Readiness to learn4. Orientation to learning5. Motivation to learn
Therefore, learning must be:1. Self-directed2. Experiential3. Relevant and authentic4. Problem-based
Sources: Knowles, M. (1984). The Adult Learner: A Neglected Species. & Kearsley, G. (2010). Andragogy: The theory Into practice database.
Source: Pappas, C. (2013). “The Adult Learning Theory.” eLearning Industry. Online.
Source: Pappas, C. (2013). “The Adult Learning Theory.” eLearning Industry. Online.
But this training is costly...
Airfare – Average of $800 per person
Hotel – 3 nights @ $200 per night $600
Catered Meals – 3 days @ $150 per day = $450
Labor – $500 per day (assuming $125k/yr) = $1,500
Transportation – $200 per person across 3 days
Group Activity – $300 per person
Room Rental of $2500 per day x 3 = $7500
Total Investment before a facilitator = $103,750
Facilitator = $15,000 per day x 3 = $45,000
---------------------------------------------------------------------------
Total Cost Per Person: $5,800
Source: Altman, Ian (2014) How Much Does Sales Training Cost? Forbes. Online.
...and it cannot scale across the organization
Companies spend 78% more on Executives than Emerging Leaders (and 360% more per Leader)
Source: Bersin by Deloitte (2014) Leadership Development Factbook 2014: Benchmarks and Trends in U.S. Leadership Development.
Online learning offered a solution, but didn’t deliver
My Background
Principles of Andragogy
Shameless Plug
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
Solutions for learning experience design and implementation
Online platform for experiential learning
NovoEd develops talent online.
We power the best talent development programs
Principles of NovoEd all learning
Intuitive, User Friendly
Experience Orientation
Pervasive and Social Learning
Data-Driven, Evidence-Based
These drive 20x higher engagement C
ompl
etio
n R
ate
2%
44%
Traditional Social
Stanford Professor Chuck Eesley conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.
50%
Figure 1. Completion Rate By Social Model in MOOCs
Learners are more engaged when they learn together. 68
NPS
50-50
0
95%+ User Satisfaction
60+ Net Promoter Score
Source: Survey of NovoEd learners (January 2016)
My Background
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
What is social learning?
The use of peer-to-peer (social) interaction as a pedagogical technique to drive learning.
Examples:
● 1:1 tutoring● Reciprocal Teaching● Think-Pair-share● Jigsaw ● Group work● Collaborative brainstorming
Source: I made this up last night after talking to another speaker.
Social learning is more than (just) sharing.
Communities visible to themselves with real identity
Peer-to-peer feedback, coaching, and mentorship
Collaboration in teams on relevant projects
Discussions with forums, and messaging
User generated content sharing with discovery
Why is social learning valuable?
Theories
● Relatedness motivation (Deci & Ryan)
● Felt accountability (Sutton & Rao)
● Social context of learning (Bandura)
● Cooperative learning (Dewey, Ross, et al)
● Social constructivism (Piaget)
● Joint inquiry (Dewey)
● Shared reflection of ELM (Kolb)
● Diversity of Learning Styles (Kolb)
● Mastery learning (Bloom)
● Community of Inquiry (Peirce, Dewey, Garrison, et al)
● Social development theory (Vygotsky)
● Truly adaptive learning (Bybee)
But you don’t have to subscribe to a single theory to leverage the techniques.
Source: I compiled this list last night - send me any I missed!
The benefits of social learning
Communities visible to themselves
Peer-to-peer feedback and coaching
Collaboration in teams on projects
Discussions with forums, messaging
User generated content sharing
● Social obligation, transparency, “felt accountability”● Self-regulating and adapting● Build networks, the new “content management system”
● Metacognition in providing feedback or mentorship● Personalized and adaptive; “network of 1:1 tutors”● More relevant guidance, “learning at speed of trust”
● Inquiry-driven, constructivist, PBL● Practice, rehearsal, interactivity● Self-directed and autonomous, right in ZPD
● Share diversity of skills, experiences, and perspectives● Crowd sourced, rapid response time● Relevant and authentic dialog drives connectedness
● Provides multiple representations & perspectives● Application focused, authentic, and relevant● Accountability and social proof
My Background
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Team Learning
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability
Team Size
Team Selection
Team Heterogeneity
Scaffolding Teamwork
Context and Methodology
Chuck EesleyAssistant Professor,Stanford University
● Teaching online since 2012● Morgenthaler Faculty Fellow ● 2010 Best Dissertation Award,
Academy of Management● Ph.D., MIT; BS, Duke University
Technology EntrepreneurshipChuck Eesley, Stanford UniversitySeptember 16, - November 17, 2013
● Eight-week course offered on NovoEd
● Team-based, experiential pedagogy
● 8 assignments, each a part of a team-based project to find and evaluate a startup idea
● Taught on NovoEd 11+ times and at Stanford since 2004
View the course at https://novoed.com/venture17
Multivariate regressions, descriptive statistics, and t-tests of difference in means
● n = 26, 248 students● Students self-selected into
experimental groups● Dependent Variables:
engagement and satisfaction measures
● Independent variables: whether the student participated in a team, if they worked individually, and if they had a mentor
● Control Variables: demographics, engagement level, and others
For more information, please contact Professor Eesley at [email protected].
Researcher Context Methodology
16x higher completion with peers and feedback
Com
plet
ion
Rat
e
21%
2%
44%
293
Individual with Discussions
Teams Teams with Feedback
Chuck Eesley, Stanford Professor, conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.
50%
Figure 2. Completion Rate By Social ModelRESULTS:
5x more sign-ins by individuals in teams A
vera
ge S
ign-
Ins
Per
Cou
rse
39
8
44
293
Individual with Discussions
Teams Teams with Feedback
Chuck Eesley, Stanford Professor, conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.
50
Figure 3. Sign-Ins Per User, By CohortRESULTS:
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability
Team Size
Team Selection
Team Heterogeneity
Scaffolding Teamwork
Rao & Sutton (2013) suggest felt accountability drives intrinsic motivation and engagement
Sutton and Rao discuss NovoEd and the value of felt accountability
in Scaling Up Excellence
"[NovoEd] built in numerous clever and easy-to-use social features to create a peer-powered network
that would link, organize, evaluate, and mentor students."
Huggy Rao Robert Sutton
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)
8 public MOOCs with self-formed teams on NovoEd from 2014-15
● Stanford GSB - Scaling Up● Stanford - Tech Entrepreneurship● Stanford - Creativity: Music● IDEO/ACUMEN - Design Kit ● Princeton- Global History Lab● Princeton - Making Gov't Work● Maastricht - Project-Based
Learning
3866 teams total
Important Note: Learners self-selected into various team sizes
Context and Methodology
Andrew LinfordInstructional Programs,NovoEd
● Coro Fellow (2013-14)● Instructor, Ministry of Education,
Singapore (2011-13)● BA, Stanford University
Retrospective cohort data analysis and descriptive statistics.
● n1 = 10, 315 studentsn2 = 3,866 teams
● Dependent Variables: percentage of assignments completed
● Independent variables: team size
● Control Variables: none
● Analysis done Sept 2015
For more information, please contact Andrew at [email protected]
Researcher Context Methodology
Note: Team size one is off chart (1734). Average exclude team size of 1. Including that, average is 2.7.
Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students
Learners prefer smaller teams, average is 4
Figure 4. Team Size PreferencesRESULTS:
Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students
Empirically, the optimal team size is 7.
Figure 5. Team Size vs. Assignment Completion
Per
cent
of A
ssig
nmen
ts C
ompl
eted
RESULTS:
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams (Stanford)
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)
3 private offerings of Presidio Institute’s Introduction to Cross Sector Leadership: Building Teams
420 learners total
Important Note: In two of the courses, learners self-selected into teams (220 learners). In the other course (200 learners), learners were assigned teams by the teaching team based on meeting availability times.
Context and Methodology
Retrospective cohort data analysis and descriptive statistics.
● n1 = 420 learners
● Dependent Variables:
○ Learners engaging in course-wide discussions
○ Learners chatting with team in private work space
○ Learners messaging other learners
○ Learners commenting on assignment submissions
● Control Variables: course size
● Analysis done Feb 2016
For more information, please contact Andrew at [email protected]
Researcher Context Methodology
Andrew LinfordManager, Support and Technical OpsNovoEd
Alison GoldManager of Leadership EducationCourse InstructorPresidio Institute
AndrewAlison
Autonomous Team Formation Increases Engagement
RESULTS:
Source: NovoEd and Presidio Institute Analysis (2016). 3 Courses, 420 learners
Per
cent
Lea
rner
s E
ngag
ing
in S
ocia
l Act
ivity
Per
cent
Lea
rner
s E
ngag
ing
in S
ocia
l Act
ivity
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams (Stanford)
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)
● Focused on organic teams formed by the students and analyze preferences of students when selecting a team.
● Analyzed the differences between randomly-assigned and self-selected teams.
● Analyzed signals to discern which activities had the greatest correlation with course completion.
● n1 = 11 courses● n2 = 24,000 active learners
For more information, please contact Milad at [email protected].
Context and Methodology
Milad EftekharData Science Intern, NovoEdCo-authored with Amin Saberi and Farnaz Ronaghi
● Former data researcher at Microsoft and Stanford
● Ph.D., University of Toronto● BS & MS, Computer
Engineering, Sharif University of Technology
● Analysis conducted on 11 public MOOCs with teams offered summer 2014.
● Courses were four to eight weeks long focusing on various business topics.
● Enrollments ranged from 200 to more than 25,000.
● Demographic data comes from self-completed student profiles. Data includes age, education, gender, and location.
Working Paper: “Team Formation Dynamics: A Study Using Online Learning Data.”
Researcher Context Methodology
Learners preferred teams with similarly aged members.
Figure 6. Homogeneity by age in team selection
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
RESULTS:
Figure 7. Age is a homophilic preference.
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Age homophily resulted in more successful teams.
Cum
ulat
ive
Den
sity
Average Age Distance
RESULTS:
Figure 8. Distance is a homophilic preference, particularly longitude due to timezones.
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Individuals prefer to join teams in similar time zones.
Cum
ulat
ive
Den
sity
Average Distance (km)
RESULTS:
Figure 9. Education Level is a homogenous preference
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Teams preferred and worked best with similar education levels
Cum
ulat
ive
Den
sity
Average Education Distance
RESULTS:
Figure 10. Skill diversity is a heterogenic preference
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Successful teams have diverse skill sets.
Cum
ulat
ive
Den
sity
Skill Entropy
RESULTS:
Note: Courses tended to 2/3 male, 1/3 female.
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Learners prefer gender mix, though 2/3 identify as male
RESULTS: Figure 11. Gender mix by course
Results are consistent across all courses.
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.
Team preferences are rational.
Characteristic Most Effective
SimilarAge
SimilarLocation
SimilarEducation Level
DiverseGender
DiverseSkill Set
Preference
Similar
Similar
Similar
Diverse
Diverse
RESULTS: Table 1. Team characteristic preferences and effectiveness
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams (Stanford)
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)
How did you run your teams?
Did any group...
● Use an ice breaker?
● Assign roles (timekeeper, note taker)?
● Divide into smaller groups?
● Google or use external sources?
● Students randomly assigned to two cohorts / groups.
● Group 1 was given team guidelines during week 2, with a suggested agenda and roles for the experience. Group 2 had no instructions.
● The post course survey then asked how well teams functioned and final projects were assessed.
● n1 = 97 studentsn2 = 103 students
For more information, please contact Drew at [email protected]
Drew RemikerInstructional ProgramsNovoEd
Alison GoldManager of Leadership EducationCourse InstructorPresidio Institute
Context and Methodology
● Course: Introduction to Cross Sector Leadership: Building Teams by the Presidio Institute
● Offered as private beta to members from United Way, Points of Light, Americorps, Kresge Foundation, ProInspire, WYMAN, and others
● Providing structure for team meetings and roles will result in a higher self-reported quality of teamwork and completion of final team assignment compared to organic team development.
Researchers Context Methodology
Alison Drew
Source: Presidio Institute (2015) Introduction to Cross Sector Leadership: Building Teams, Team Scaffolding Study. “Guidelines for First Team Meeting”
Example of teamwork scaffolding
EXHIBIT:
Source: Presidio Institute (2015) Introduction to Cross Sector Leadership: Building Teams, Team Scaffolding Study.
Collaboration scaffolding improved completion and teamwork quality
Characteristic Group 1 Group 2
Quality of teamwork 4.03 / 5.0 3.79 / 5.0
Completed final assignments
8 / 10 7 / 9
Figure 12. Team scaffolding results
Marginal improvement in self-reported teamwork quality and final assignment completion
RESULTS:
My Background
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Notifications
Course Champions
6 public MOOCs offerings before and after new notifications offering was released.
● Stanford GSB - Scaling Up Without Screwing Up
● Stanford - Tech Entrepreneurship Part 2
● Agder - Success Unleashed
41,657 teams total
Important Note: Almost nothing changed within the courses when offered a second time with the notifications feature available, although were slightly smaller. Learners self-selected teams in all courses.
Context and Methodology
Andrew LinfordManager, Support and Technical OpsNovoEd
● Coro Fellow (2013-14)● Instructor, Ministry of Education,
Singapore (2011-13)● BA, Stanford University
Retrospective cohort data analysis and descriptive statistics.
● n1 = 41,657 learners
● Dependent Variables: number of learners
● Independent variables: selection of different social indicators
● Control Variables: course size
● Analysis done March 2016
For more information, please contact Andrew at [email protected]
Researcher Context Methodology
Notifications
● Personal alerts about ongoing social activity (including responses in discussion forums), assignment reminders, and more
● Daily Course Digests with assignment reminders and an overview of course activity
● Easy reminders about and access to teaching team communication
Huge Impact on Submissions; Positive Impact on Other Social Activities
RESULTS:
Source: NovoEd Analysis (2016). 6 Free Public MOOCs, 41,657 learners
Figure 13. Social Activities Pre/Post Notifications
Agenda
My Background
Principles of Andragogy
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Notifications
Course Champions
● Program: Philanthropy University
● September - December, 2015
● Course champions chosen from active learners in the middle of the program (beginning of November)
● 7 courses, 200,291 unique learners
Context and Methodology
Lisa BrefiniPhilanthropy University TACourse Operations SpecialistNovoEd
Andrew LinfordManager, Support and Technical OpsNovoEd
Retrospective learner and course champion data analysis.
● 7 courses, each with 5-12 course champions
● Comparison - course (and course champions) social activity before and after course champions are selected
● Analysis performed December 2015
For more information, please contact Andrew at [email protected]
Researchers Context Methodology
Lisa Andrew
Course Champions at work
Course Champions increased course social activity
RESULTS:
Source: NovoEd (2015). Philanthropy University Courses from 2015.
Course Champions Selected
Figure 14. Weekly Discussion Activity
My Background
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda
● That experience must be engaging, driven by inquiry, social, and coherent.
● Adults require self-discovery and (extra) practice to overcome preexisting schema
● Knowledge acquisition < skill development < behavior change. Increasingly experiential.
● Online learning is no different. Scale the best of offline learning, not the worst (lectures).
● Social techniques include discussions, collaboration, sharing, feedback, community.
● Peer learning enables personalization, metacognition, and motivation.
● Teams and group work is the most critical.
CLOSING THOUGHTS
Learning must be experiential.
Learn Together.
http://novoed.com
Thank you! Please feel free to contact me!
Greg BybeeVice President, LearningNovoEd
@gregbybeehttp://linkedin.com/in/gregbybee
Email me for slides or the research briefs. I’d also love to help you implement in your org.