Adventures in Emerging Media: An Updated Look at

Engagement Methods in Online Courses

Emily K. Johnson, Reuben Rogak, Rudy McDaniel, Robb Lindgren,

and Jon Friskics

University of Central FloridaTexts & Technology Ph.D. Program

ekj@knights.ucf.edu

• Online • Survey Course• Funded by a two-year grant• Elective• School of Visual Arts and Design• Fall 2010, Fall 2011, and Fall

2012

Introduction

• Continuation of research on student preferences in online course design

• Novel aspects of a unique, online course taught at UCF in the Fall semester of 2011 entitled Adventures in Emerging Media (AEM) were analyzed

Introduction

Unique course features that were studied:• An unfolding narrative released to

students throughout the semester• Student choice in which course

modules they wished to complete each week

• Assignments that were tied directly to students’ potential careers and job interviews

• A game-based achievement system (added in Year 2)

Background

Prior research suggests that these elements increase student motivation and learning:

•Relevant Tasks •Student Agency•Student Collaboration •Narrative Elements

Unique Course Features

• Assignmentstied directly to students’ potentialcareers and job interviews

Unique Course Features

• Unfolding narrative – “dream job”

Unique Course Features

• Choice in weekly learning modules

Unique Course Features

• Game-based Achievement System

Badges

“A 'badge' is a symbol or indicator of an accomplishment, skill, quality or interest. From the Boy and Girl Scouts…to the more recently popular geo-location game, Foursquare, badges have been successfully used to set goals, motivate behaviors, represent achievements and communicate success in many contexts.”

-The Mozilla Foundation and Peer 2 Peer University. “Open Badges for Lifelong Learning” (2012). Retrieved from: https://wiki.mozilla.org/images/5/59/OpenBadges-Working-Paper_012312.pdf

Achievements

Achievements, or earned tokens of accomplishment, often encourage players to spend more time within digital systems (esp. videogames) and to alter their playing habits in order to unlock particular types of challenges (e.g., find every coin in a given area or unlock a particular puzzle within a certain amount of time).

Rationale for Achievements

Motivate students to engage in behavior linked to success in online courses such as:• Completing assignments early• Answering a peer’s question• Offering constructive feedback to

peers• Completing more than one module per

week (only one per week required)

--Constituted a small percent of students’ participation grades.

Achievement Design

Two Types of Achievements

• Visible

• Hidden

5 Visible AchievementsAwarded for:• Posting a required introduction on

the course discussion forum• Successfully completing the first

three weeks of the course• Successfully completing two-thirds

of the modules• Successfully completing all required

modules• Turning in the Week Twelve

Milestone for the Final Project

6 Hidden AchievementsAwarded for:• Being the first person to post a project

each week• Answering a peer’s course-related

question on the discussion forum• Completing three modules for one week• Completing four modules for one week• Watching at least some of each video

contained in each module for a given week

• Turning in a project with exceptional detail or technical skill

Achievements Page

Awarding Achievements – Instructor View

Design

• Non-experimental• Descriptive• Mixed methods• Likert-scale options yielding

quantitative data • Open-ended questions added a

qualitative approach• Focus group after semester ended

Survey Participants

• 138 undergraduate students at UCF enrolled in Adventures in Emerging Media in Fall 2011

• 84 undergraduate students at UCF enrolled in a total of four other, “traditional” online courses in the sciences and humanities completed a modified survey

Sample questions for both groups:This course has affected the way I think about

my career goals.Strongly Disagree 1 2 3 4 5 6 7 Strongly

Agree

I feel that I benefited from interactions and collaborations with other students in this course.Strongly Disagree 1 2 3 4 5 6 7 Strongly

Agree

In the last 8 weeks of this course, how often have you done the following?

Commented on another student's workVery Often – Often – Sometimes – Never

AEM Specific Question ExampleWhat aspects of this course caused you to put more effort into these assignments, as compared to a “regular” online course? (Choose all that apply)

• The ability to choose my learning modules each week.

• The ability to choose one learning module for grading, AND go back to take additional modules with "no pressure."

• A story that unfolds in pieces over the period of the course (the "dream job" scenario).

• The relevancy of assignments designed to help me get a job after graduation.

• Having my work recognized in some way (like on the "Achieve" page).

• The ability to share my work with peers.• The ability to critique the work of my peers.• The ability to have my work critiqued by peers.

Design

• Responses of students from the AEM course and the four other courses were compared to analyze student perception and behavior between the two types of courses.

Achievements

Variable 1 Variable 2Pearson

Correlation (r)

Significance (p)

Comfort with Technology (1 to 7)

Motivated to Receive Achievements (1 to 7) .228 .010*

Number of Gaming Hours per Week

Motivated to Receive Achievements (1 to 7) .172 .057

• Pre-Survey: Badge system motivating particularly for those who were comfortable with technology and games

0

1

2

3

4

5

6

7

Felt Achievement System Had Positive Impact on Course

7 = Strongly Agree

Males FemalesMales Females0

1

2

3

4

5

6

7

Motivated by Other Students Receiving Achievements

7 = Strongly Agree

Males Females

Achievements

Variable 1 Variable 2 Pearson Correlation (r) Significance (p)

Felt Achievement System Was Positive (1 to 7)

Discussed Ideas Outside of Class (1 to 7) .175 .040*

Worked Harder To Receive Achievements (1 to 7)

Commented On Other Students’ Work (1 to 7) .217 .010*

Worked Harder To Receive Achievements (1 to 7)

Amount Of Time Spent Collaborating With Other Students

.242 .004**

Seeing Others Get Achievements Was Motivating (1 to 7)

Total Number Hours Spend On Course Per Week .158 .064

Achievements

• Post-Survey: Positive feelings about achievement system were correlated with other positive feelings in the course

Student Attitudes

• AEM Favorite Aspect: Course Design

• Other Positive Responses: Choice in learning modules Ability to take additional modules

without being tested on them Relevancy of assignments

Commented on peers’ work

AEM

Commented on peers’ work

Other Online Courses

Results: Likert Statements

• No notable differences

  AEM Other

Challenging Course 5.25 4.87

Learned Interesting Things 5.52 6.45

Meaningful Assignments 4.95 6.24

Benefitted from Student Collaboration

4.45 5.52

Tried to do well 5.93 6.42

Proud of work 5.70 5.37

Affected career goals 4.62 5.10

Recommend to a friend 5.05 6.31

Interest in similar course design 5.07 6.00

Results

Aspects you would like to see in other courses:• Choice of learning modules each week:

79%• The ability to take additional modules

without being tested on them: 64%• Relevancy of assignments: 55%• Have work critiqued by peers: 53%• Share work with peers: 49%• Ability to critique the work of peers:

44%• Have work recognized: 38%• Unfolding story: 28%

Results

Aspects that caused you to put forth more effort:• Choice of learning modules each week:

54%• Relevancy of assignments: 39%• Peers critiquing your work: 41%• Sharing work with peers: 37%• Ability to take additional modules

without being tested on them: 36%• Ability to critique the work of peers: 32%• Having my work recognized: 30%• Unfolding story: 14%

Discussion• AEM students- higher frequency of

collaboration

• Research suggests that collaboration is important

• Worthwhile to incorporate aspects of this course lending themselves to collaboration into other online courses.

Discussion• Little difference was seen in Likert-

scale averages between AEM & other courses

• This could be due to the nature of education at this level

• Course selection• Intrinsically motivated• Understand importance of

assignments

Discussion• Interesting gender patterns in

achievement opinions – Are badges more motivating to females? Was the design similar to social games, which females tend to prefer?

• AEM students indicated a desire to see many of the unique features implemented in future online courses

• Some factors to consider when designing a course

Discussion: Future Research• Similar course in different fields of

study• Determine most effective features• Investigate effects on skill mastery

Intended Improvements

• Improving narrative coherence; including student input

• Updating digital creation guidelines

• New module creation and improved test creation system

• Updating the look-and-feel of the site

ReferencesBeldarrain, Y. (2006). Distance education trends: Integrating new

technologies to foster student interaction and collaboration. Distance Education, 27(2), 139-153.

Caine, R. N., & Caine, G. (1991). Making Connections: Teaching and the human brain. Alexandria, VA: Association for

Supervision and Curriculum Development.Cameron, J. a. P., W. D. (2002). Rewards and intrinsic motivation:

Resolving the controversy. Westport, Connecticut: Bergin & Garvey.

Carr, D. (1986). Narrative and the real world: An argument for continuity. History and Theory, 25(2), 117-131. Deci, E. L. & Ryan, R. M. (1990). Intrinsic motivation and self-determination in human behavior. New York: Plenum.Keller, J., & Suzuki, K. (2004). Learner motivation and e-learning design: A multinationally validated process. Journal of Educational Media, 29(3), 229-239.

ReferencesLauritzen, C. a. J. M. (1997). Integrating learning through story: The narrative curriculum. Albany, N.Y: Delmar Publishers.Lindgren, R. & McDaniel, R. (in press). Transforming online learning through narrative and student agency. Educational Technology & Society.Liu, N. F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279-290.McCombs, B. L., Whisler, Jo Sue. (1997). The learner-centered

classroom and school: Strategies for increasing student motivation and achievement. San Francisco, CA: Jossey- Bass.Palloff, R., & Pratt, K. (2000). Making the transition: Helping teachers to teach online. EDUCAUSE: Thinking it through. Nashville, Tennessee.

 

ReferencesPasse, J. (1996). When students choose content: A guide to increasing motivation, autonomy, and achievement. Thousand Oaks, CA: Corwin Press.Patall, E. A., Cooper, H., & Wynn, S. R. (2010). The effectiveness and relative importance of choice in the classroom. Journal of Educational Psychology, 102(4), 896. Radovan, M. (2011). The relation between distance students' motivation, their use of learning strategies, and academic success. The Turkish Online Journal of Educational Technology, 10(1), 216-222. Varma, S., McCandliss, B. D., & Schwartz, D. L. (2008). Scientific and pragmatic challenges for bridging education and neuroscience. Educational Researcher, 37(3), 140-152.  

ekj@knights.ucf.edu