Adaptive Navigation Support and Open Social Learner Modeling for PAL

Peter BrusilovskySchool of Information Sciences,

University of Pittsburgh

Key Goals

• Leverage large volume of data left by past learners to improve learning process

• Better Interface– Open Social Learner Modeling Interface for

visual learning analytics and content access• Better Personalization Algorithms

– Enhancing personalized learning algorithms for personalized guidance and content recommendation

Three Directions of Project Work

• Exploring Open Social Learner Modeling interface– Diverse learning content– Multiple domains

• Enhancing personalized learning algorithms for personalized guidance – Pro-active content recommendation– Remedial content recommendation and guidance

• Develop architectural support and authoring tools for Open Social Learner Modeling– OSLM as a reusable component– Content and course authoring tools

Open Social Learner Modeling - I

Interactive Demo YouTube Demo

Open Social Learner Modeling - II

Open Social Learner Modeling - III

Accessing OSLM ADL Demo

• You can enter our at:• http://adapt2.sis.pitt.edu/kt• ADL Usernames:

–    adl01, adl02…, adl10–   Passwords are the same as the usernames

• You will see links for courses in Java, SQL and Python programming

• Note that most adlxx users are "empty", which means that they do not have activity yet. User adl01 has some activity.. 

OSLM Experience

• Developed full-semester courses with smart learning content for 3 domains, all accessible online, used in several universities– Java Programming– Introduction to Databases with SQL– Python Programming

• Evaluated Mastery Grids Interface in many classroom studies– Value of open learner model– Value of social comparison

Term / Course Research set up / comments Active Learner

s

Question

Attempts

Annotated Examples Viewed

Animated Examples Viewed

IS0017 Fall 2013 Mastery Grids and Portal 38 3832 747 -

IS0017 Spring 2014

Social enabled (preliminary version of MG)

41 2707 551 -

IS0017 Fall 2014 mixed (random assign toSocial and non-social

groups)

65 4563 1936 670

IS0017 Spring 2015

mixed (random assign toSocial and non-social

groups)

36 2146 947 281

IS0017 Fall 2015 mixed (random assign toSocial and non-social

groups)

58 7109 2689 1165

ASU Fall 2014 Social features enabled Recommendations enabled

100 9285 4186 -

ASU Fall 2015 Different recommendation algorithms (2 groups)

74 4364 1175 505

CS 401 Fall 2015 mixed (random assign toSocial and non-social

groups)

68 2715 1267 606

WSSU Fall 2013 2 groups, Control / Social 22 876 340 -

WSSU Fall 2014 2 groups, Control / Social 20 1837 618 112

National Sun Yat-Sen Univ. Taiwan Spring 2015

26 1550 889 420

JAVA Courses

Term / Course Research set up / comments Active Learne

rs

Question

Attempts

Annotated Examples Viewed

IS 1022 Fall 2013 Mastery Grids and Portal 15 530 212

IS 1022 Fall 2014 Social features enabled 33 1194 793

IS 1022 Spring 2015

Social features enabled 18 224 277

IS 1022 Fall 2015 Social features enabled 32 526 787

IS 2710 Fall 2013 Mastery Grids and Portal 44 510 213

IS 2710 Fall 2014 2 groups, Control / Social 97 6819 2876

IS 2710 Spring 2015

Social features enabled 33 3616 1506

IS 2710 Fall 2015 2 groups, Control / Social 56 3486 1531

SQL Courses

Term / Course Research set up / comments Active Learne

rs

Question

Attempts

Annotated Examples Viewed

Animated Examples Viewed

Parsons

AALTO Universty Fall 2015

Social and non-social groups 490 7909 6187 4545 9158

IS0012 Fall 2015 Social features enabled 19 1301 586 548 1628

Python Courses

OSLM – Some Findings

• OLM/OSLM significantly improve learning engagement, problem performance, learning gain

• Social comparison in OSLM further increase learner engagement

• OSLM helps students to work more efficiently

• OSLM preserves mastery orientation

OSLM Increases EngagementVariable

OSM OSLMU

Mean Mean Sessions 3.93 6.26 685.500*Topics coverage 19.0% 56.4% 567.500**Total attempts to problems 25.86 97.62 548.500**Correct attempts to problems 14.62 60.28 548.000**Distinct problems attempted 7.71 23.51 549.000**Distinct problems attempted correctly 7.52 23.11 545.000**Distinct examples viewed 18.19 38.55 611.500**Views to example lines 91.60 209.40 609.000**MG loads 5.05 9.83 618.500**MG clicks on topic cells 24.17 61.36 638.500**MG click on content cells 46.17 119.19 577.500**MG difficulty feedback answers 6.83 14.68 599.500**Total time in the system 5145.34 9276.58 667.000**Time in problems 911.86 2727.38 582.000**Time in MG (navigation) 2260.10 4085.31 625.000**

Aggregate: The Architecture behind MG• Extension of our original architecture

ADAPT2• Allows transparent connection of

independent smart learning content that is interactively delivered by smart content servers

• Supports extensive tracking of learner activities, learner record storage, learner modeling, group modeling, social comparison

• Supports multi-domain course authoring, content brokering and concept brokering

Animated Examples in MG

Smart Content and Content Authoring• We created large volume of reusable smart

content– With activity tracing, content brokering, authoring

• Pittsburgh team smart content– Interactive examples: Java, SQL, Python– Java exercises– SQL problems– Python exercises – YouTube video sections

• Helsinki team smart content– Animated examples (Java and Python)– Parsons problems (Python)

Full Support for Instructors

• Create course for any domain as sequence of any topics

• Connect smart learning content of several kinds from multiple content servers

• Create own content if existing content is not sufficient

• Create groups and subgroups, assign to classes

• Observe class/group work with MG

Open Content, Open Source• All developed content could be reused right from our

content server or by installing own content server• All the sources are available in GitHub

– The Mastery Grids Interface, back-end Aggregate and documentation can be found here.

– User model services can be found in here.– QuizJET Interface, Authoring Tool, Content Brokering and

documentations can be found here.– QuizPET Interface, Authoring Tool, Content Brokering and

documentations can be found here.– Parson Problem Authoring Tool can be found here.– Annotated Examples Interface, Authoring Tool, Content Brokering and

documentations can be found here.– Animated Examples Authoring Tool can be found here.– Videos User Interface, Authoring Tool, Content Brokering and

documentations can be found here

Algorithms

• Student Modeling– Several data-driven student modeling approaches– Most notable is FAST, an extension of BKT that

can use additional data New work: multi-content social student modeling

• Recommendation– Several algorithms for proactive content

recommendation and remedial example recommendation

– Performed several studies demonstrating the value of recommendation

Further Information

• Project Home page– Explanations, demos, videos, flyers– http://adapt2.sis.pitt.edu/wiki/ or http://bit.ly/1Ty5KOr

• GitHub– Sources, installation, system documentation– https://github.com/PAWSLabUniversityOfPittsburgh– https://github.com/acos-server/

• Publications, conference presentations– Interface, algorithms, studies, evaluation data– Available from the project home page