MOOCs and Quality Issues - QAQE Steering Group
Quality Assurance and Quality Enhancement in E-learning Special Interest Group (QE-SIG)
Helen Barefoot University of Hertfordshire
Jon RosewellThe Open University
Session Overview
•Definition of MOOCs
•Online learning
•The Quality Code
MOOCs and Quality Issues
3
So what is a MOOC?•Massive•Open•Online•Course
MOOC background information•Term coined in 2008•Connectivism and Connective Knowledge▫25 tuition paying students▫2300 other students from the general public taking
course online and free of charge
What are the key challenges associated with assuring the quality of MOOCs?
MOOCs and Quality Issues
8
Why bother with quality?•Students – know what they are committing to•Employers – recognition of content and skills •Authors – personal reputation, 'glow' of success • Institutions – brand reputation •Funders – philanthropic, venture caps, governments
•Quality agencies – on behalf of above
MOOCs and Quality Issues
9
TensionsDelivery: face-to-face . distanceAccreditation: credit . nonePrice: cost . freeEntry: selective . openScale: personal . massiveSupport: intensive . not supportedPedagogy: constructivism . transmissionTeacher: star . anonymous
Massive• Issue is not large size, but scale independence
•Aspects:▫Financial▫Technical▫Pedagogical
Learning design at scale• Individual learning•Small group collaboration• Large communities
Open•Different meanings:▫Open = free = gratis▫Open = free = libre▫Open entry▫Open = transparency
Course•What distinguishes MOOC from OER or learning
object?▫Size?▫Goal, learning outcomes?▫Measure of completion / achievement / attainment?▫Structure
As narrative (xMOOC)? Negotiated experience (cMOOC)?
▫Time?
Quality and learners“What are MOOCs actually aiming at? “Can the quality of MOOCs be assessed in the same way as any defined university course with traditional degree awarding processes? “Or do we have to take into account a different type of objective with MOOC learners? Are the learners mostly interested in only small sequences of learning, tailored to their own individual purpose, and then sign off and move to other MOOCs because their own learning objective was fulfilled?”Ulf-Daniel Ehlers, Ebba Ossiannilsson, Alastair Creelmanhttp://mooc.efquel.org/
Quality Assurance Agency “Our job is to safeguard quality and standards in UK universities and colleges, so that students have the best possible learning experience” Quality Code for Higher Education sets out the expectations all providers of UK higher education are required to meet. It gives all higher education providers a shared starting point for setting, describing and assuring the academic standards of their higher education awards and programmes and the quality of the learning opportunities they provide.
QAA event• Quality Code“Factors which apply to all learning opportunities regardless of location, mode of delivery, academic subject; MOOCs are no exception to that”
Anthony McClaran, QAA event (July, 2013)
The Quality Code
The Quality CodePart A: Setting and maintaining threshold academic standardsPart B: Assuring and enhancing academic qualityPart C: Information about higher education provision
The Quality CodePart A: Setting and maintaining threshold academic standards
Part B: Assuring and enhancing academic quality
• A1: The national level• A2: The subject level and qualification
level• A3: The programme level• A4: Approval and review• A5: Externality• A6: Assessment of achievement of
learning outcomes
• B1: Programme design and approval• B2: Admissions• B3: Learning and Teaching• B4: Enabling student development and
achievement• B5: Student engagement• B6: Assessment of students and the
recognition of prior learning• B7: External examining• B8: Programme monitoring and review• B9: Academic appeals and student
complaints• B10: Managing HE provision with others• B11: Research degrees
Part C: Information about higher education provision
Considering MOOCs through the lens of the Quality Code
B1: Programme design and approval• Procedures for design and approval for
MOOC courses?• Are external partners involved in the
design and approval?• Assessing the effectiveness of the
course and processes for monitoring and reviewing the course?
Course approval
B3; Learning and Teaching• Constructive alignment• Levelness• Teaching methods▫ Videos▫ Embedded questions ▫ Tutorials▫ Discussion forums▫ Wikis▫ Assessments
Considering MOOCs through the lens of the Quality Code
•B5; Student engagement• “improving the motivation of students to engage in
learning and to learn independently▫data
• the participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience”.
Considering MOOCs through the lens of the Quality Code
Source: edX – Presented by Lori Breslow – Enhancement Themes conference, June 2013
“the participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience”
• B6; Assessment of students and the recognition of prior learning
Considering MOOCs through the lens of the Quality Code
Moz
illa
badg
e in
fras
truc
ture
http
://o
penb
adge
s.or
g
http://www.dmlcompetition.net/
A badge is a validated indicator of accomplishment, skill, quality or
interest that can be earned in any environment
http://scouts.org.uk/
Back to basics…•Are students learning?▫Self assessment▫Quizzes ▫Peer assessment▫Analytics▫Survey evaluation
Monitoring and evaluation
Completion rates•Reported completion may be very low (1-10%)•Does that matter?▫With very large starting numbers, there are still many
learners completing▫Maybe learners achieve personal goals even if they
don’t complete▫Can MOOCs encourage access to HE if >90% have an
experience which is a ‘failure’?
ProvenanceReputation
Brand
ProvenanceReputation
Brand
creation use
user recommendationpeer review
MOOCMOOC
Quality points
checking
Quality Dimensions
ContentAccuracyCurrencyRelevance
Pedagogic EffectivenessLearning objectivesPrerequisitesLearning designLearning stylesAssessment
Reusability & opennessFormat & interoperabilityLocalisationDiscoverability: metadataDigital preservationAccessibility
Ease of useClarityVisual attractiveness, engagingClear navigationFunctional!
Benchmarks / indicators(NB: an unashamed plug!)•Conventional HE benchmarks not be appropriate •E-learning benchmarks a better starting point▫E-xcellence NEXT
http://e-xcellencelabel.eadtu.eu/
•MOOC benchmarks better!▫OpenupEd / E-xcellence MOOC (to come…)
http://openuped.eu/
OpenupEd www.openuped.eu • ‘Reflecting European values such as equity, equality
and diversity’•Courses should show features:▫Openness to learners▫Digital openness▫Learner-centred approach▫Media supported interaction▫Recognition options▫Quality focus▫Spectrum of diversity
UK’s first credit rated MOOC• Level 4• 20 credit points• Taught as MOOC for
undergraduate students and as open access
• 1000 students• 200 students going for credit• Assessed by a
• 10-minute podcast presentation (40%)
• 1500-word critical blog (60%)
And the latest…• University of California, Irvine, • “Tackle serious scientific issues,
related to events in the show”▫ Physics- "science behind decay“▫ Public health department- "What
can we learn from past epidemics?“
▫ Maths - "post-calculus maths can be used to model population and epidemic dynamics"
• TV series - audience of 10 million • Student numbers for online
course expected to be in hundreds of thousands