Karen Swan, Scott Day, Leonard Bogle & Traci van Prooyen
University of Illinois at Springfield
Exploring MOOC Pedagogies
American Council on Education’s (ACE) College Credit Recommendation Service isevaluating MOOCs for content and approving them for specific college credits if students pass ACE CREDIT tests ($150).ACE CREDIT evaluates MOOCs for their content; we are reviewing MOOCs for their pedagogical approaches.
CONTEXT
We developed an instrument to characterize the pedagogical approaches taken in MOOCs --
modeled on a similar instrument proposed by Tom Reeves (1996) for describing the pedagogical approaches found in computer-based instruction --“Pedagogical dimensions are concerned with those aspects of design and implementation . . . that directly affect learning.”
AMP (Assessing MOOC Pedagogy) tool
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
AMP TOOL
METHODSBy Fall 2013, ACE evaluated and approved 5 Coursera, 1 EdX, and 7 Udacity MOOCs -- College Algebra, BioElectricity, Genetics, Pre-Calculus, and Single Variable Calculus, Circuits, Introduction to Artificial Intelligence, Introduction to Computer Science, Introduction to Physics, Introduction to Statistics, Introduction to Parallel Programming, 3-D Modeling and HTML 5 Game Development
We reviewed the same 13 MOOCs and four other Coursera MOOCs that covered non-STEM courses.
METHODSWe iteratively revised the AMP tool as we tested its efficacy on the approved MOOCs, first by reducing the scale of each dimension from ten to five, and then by establishing specific criteria for each dimension, to improve inter-rater reliability. We reviewed all the approved MOOCs using the revised instrument.
Four researchers reviewed each MOOCs. Inter-rater reliability was >80% on all the reviews before meeting to achieve consensus on the ratings.
The approved COURSERA courses followed a format that resembled the traditional lecture/text/testing routine of traditional university courses, including cohorts and weekly hard deadlines.
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupport
ed) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupport
ed) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
college algebra
bioelectricity
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupport
ed) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupport
ed) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
genetics & evolution
pre-calculuspedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupport
ed) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupport
ed) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
single variable calculus pedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
college algebra bioelectricity genetics & evolution
pre-calculus single variable calculus
circuitspedagogical
category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more structure)
approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
The Coursera courses approved for ACE credit, as well as EdX’s Circuits, tended to be objectivist, teacher-centered, convergent, highly structured, more abstract than concrete, with minimal feedback. The courses tended to fall somewhere in the middle between supporting and not supporting cooperative learning, accommodating and not accommodating individual differences, artificial and authentic activities/assessment, and passive and active user roles.
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)
cooperative learning (unsupported) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
college algebra bioelectricity genetics & evolutionpre-calculus single variable calculus circuits
UDACITY courses followed a format that resembled programmed learning, including self-pacing and embedded quizzes.
interactive 3-d graphicspedagogical
category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)
cooperative learning (unsupported) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
html 5 graphicspedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
pedagogical category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative
learning (unsupported) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupported) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
introduction to artificial intelligence
introduction to computer sciencepedagogical
category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more structure)
approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupported) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupported) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
introduction to parallel processingpedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
introduction to physicspedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
introduction to statisticspedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to
content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5
(frequent, constructive
)cooperative
learning(unsupport
ed) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupport
ed) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
Udacity courses tended to be neither objectivist nor constructivist, slightly less teacher-centered and convergent than Coursera courses, highly structured, halfway between abstract and concrete, with immediate, clear, and constructive feedback. They also tended not to support cooperative learning, but because of their self-directed approach were quite accommodating of individual differences. Udacity also made an effort to develop authentic activities, and perhaps because of the large number of CS courses, supported a more generative user role than the ACE-approved courses from Coursera and EdX.
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)
focus of activities (convergent) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)
cooperative learning (unsupported) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
interactive 3-d graphics HTML 5 graphics intro to artificial intelligence intro to computer science intro to parallel processing
intro to physics intro to statisics
EDX COURSERA UDACITYepistemology 1.0 1.0 2.4role of teacher 1.0 1.4 2.0
focus of activities 1.0 1.0 1.9structure 5.0 5.0 4.9
approach to content 4.0 3.6 3.0feedback 3.0 2.0 4.3
cooperative learning 2.0 2.8 2.1accommodation of individual
differences 2.0
2.6 3.0activities/assessment 1.0 2.6 3.3
user role 2.0 3.0 3.1average ratings across platforms
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
Coursera EdX Udacity
comparisons of ACE approved courses across platforms
art & inquirypedagogical
category ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
comic books & graphic novels
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
jazz improvisation
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
the music of the Beatles
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
art & inquiry comic books & graphic novelsjazz improvisation the music of the Beatles
non-stem
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
art & inquiry comic books & graphic novelsjazz improvisation
non-stem
STEM BEATLES NON-STEMepistemology 1.0 1.0 4.7role of teacher 1.4 1.0 3.0
focus of activities 1.0 1.0 4.3structure 5.0 4.0 3.7
approach to content 3.6 2.0 2.3feedback 2.0 2.0 3.7
cooperative learning 2.8 2.0 3.0accommodation of individual
differences 2.6 2.0 2.7activities/assessment 2.6 1.0 4.7
user role 3.0 2.0 4.3average ratings across domains
pedagogical category
ratings
epistemology (objectivist) 1 2 3 4 5 (constructivist)
role of teacher (teacher-centered) 1 2 3 4 5 (student-
centered)focus of activities (convergent
) 1 2 3 4 5 (divergent)
structure (less structure) 1 2 3 4 5 (more
structure) approach to content (concrete) 1 2 3 4 5 (abstract)
feedback (infrequent, unclear) 1 2 3 4 5 (frequent,
constructive)cooperative learning (unsupporte
d) 1 2 3 4 5 (integral)
accommodation of individual
differences(unsupporte
d) 1 2 3 4 5 (multifaceted)
activities/assessment (artificial) 1 2 3 4 5 (authentic)
user role (passive) 1 2 3 4 5 (generative)
STEM Beatles non-STEM
(objectivist) 1 2 3 4 5 (constructivist)
(teacher-centered) 1 2 3 4 5 (student-
centered)(convergent
) 1 2 3 4 5 (divergent)
(less structure) 1 2 3 4 5 (more
structure) (concrete) 1 2 3 4 5 (abstract)
(infrequent, unclear) 1 2 3 4 5
(frequent, constructiv
e)(unsupporte
d) 1 2 3 4 5 (integral)
(unsupported) 1 2 3 4 5 (multifacet
ed)(artificial) 1 2 3 4 5 (authentic)
(passive) 1 2 3 4 5 (generative)
two metaphors for learning (Sfard, 1998)
acquisition
participation
CONCLUSIONS• AMP tool can effectively characterize
MOOC pedagogies• At least two distinct pedagogical
approaches can be distinguished• Future research will explore these
distinctions, as well as MOOCs offered on different platforms & differently organized
• Future research will try to link pedagogical to learning outcomes