versus reading for pre-class preparation in a "flipped" classroom. Journal on Excellence in College Teaching~ 26(1), 5-28.

Qualitative Analysis of Animation Versus Reading for Pre-Class Preparation

in a "Flipped" Classroom

Adam M. Persky University of North Carolina at Chapel Hill

The "flipped" Glassroom model, including such approaches as team-based learning (TBL), stresses pre-class preparation. For three years in a pharmacokinetics course within a pharmacy curriculum, students had the choice of using reading material or a fully animated module to prepare for class. Qualitative methods were used to analyze students' preferences. Two thirds of the students preferred reading over the multimedia modules, with easier notation and self-pacing as the primary reasons. Students who preferred reading over the modules scored ap­proximately one-half grade higher on their quizzes. Despite the theoretical advantages offered by the visual presentation of information, students' preference for reading was due to several factors, primarily self-pacing.

Introduction

The "flipped" classroom model has been popularized in the recent years, but it dates back almost 20 years with blended learning environ­ments and team-based learning. The team-based learning (TBL) format has increased in popularity because its design focuses on pre-class preparation, holding students accountable for that preparation (that is, the reacilness assurance process), and using class time to focus on the application of concepts. Transitioning class time from content delivery to promoting higher-order thinking skills places greater responsibility on students for obtaining foundational information independently. Encouraging students to be self~directed learners can be facilitated by the use of technology.

5

b 1ournai on r,xceuence zn Lottege 1eacnzng

While using technology in lieu of lecture or other traditional methods has shown relatively small effect sizes in improving the acquisition of content knowledge"(Means, Toyama, Murphy, Bakia, & Jones, 2010; Tamim, 2011), it is unclear if students prefer technological material over more traditional material when preparing for class. This consideration is important, because there may be an underlying assumption that the current student generation (the millennials) are digital natives who prefer using technology over more traditional methods. This assumption can lead faculty to develop technology (such as animations and videos) in lieu of less resource-intensive readings that are equally effective for pre­class preparation. It may be important to discuss the best practices for technology in delivering information or facilitating learning within context of the situational factors (for instance, course structure, the purpose of the self-directed learning). For example, asking students to be more self-di­rected in preparing prior to class using "flipped" classrooms or TBL may require different attributes than an entire course that is self-paced (such as online learning) or if the self-paced material is for post-class activities (such as supplemental material).

The use of technology can offer some potential benefits to learners compared to traditional static methods (for example, reading), especial­ly by introducing the ability to animate concepts, build opportunities to interact with the material, and provide embedded practices offering immediate feedback. Because of the potential benefits and the increasing ease of developing educational technologies, there has been an increase in this mode of delivering information. We developed a fully animated, narrated multimedia module to offload content so that class time can be used for application of those concepts (Bell & Schraff, 2008). This module was based predominantly on an instructor-developed book available elec­tronically (an e-book) (see Figures la and lb and Table 1). In a randomized, between-subjects design pilot study comparing this module to lecture, we found no difference in learning gains between formats, and both groups rated the experience highly (Bell & Schraff, 2008; Persky, 2008) (see Table 2). Students agreed that both the lecture and module were easy to follow, that the information was presented in an engaging manner, that each format contained sufficient information, and that completing each format was worth the students' time. With the exception of being easy to follow, each of these attributes was rated more highly in the lecture group than in the module. Students also ranked their preference for learning (Persky, 2008; see Table 2). Reading was ranked last by both cohorts (56% for the lecture-group and 66% for the module group). In open-ended comments asking about advantages of the module over static print texts, students

rirurriuuun v1:;;. l'\t::uuzng JUT 1-re-Ltass 1-reparatzon

Figure la Visual Examples From the Module Depicting Protein Binding

<P {

·,_:·_] , 1:c1';1 Cl

Notes. (1) tools (e.g., equation helper, glossary); (2) navigation; (3) closed captioning; (4) slider bar for rewind/ fast forward; (5) main screen

I

®

noted the 24/ 7 access and immediate feedback as the primary advantag­es. Students also commented that the ability to replay information was helpful.

This pilot study had several limitations. First, it did not directly compare the multimedia module to reading material; this may be the more appropriate comparison, because the multimedia module was designed to help students prepare for class, not necessarily to replace the instructor. Second, because the pilot randomly assigned students to a format and asked about their prospective preferences for pre-class preparation, it does not represent actual student choice in an authentic class setting. Finally, the pilot was conducted on a single subset of the module (the piloted module) versus on the entire, completed module.

This report retrospectively analyzes students' preference for pre-class

/j 1ournat on 1::,xceuence zn Louege 1eacmng

Figure lb Visual Examples From the Book Depicting Protein Binding

(j)

@

Albumin,.

'-« 1-ac:id-glyeoprctein"

lipoproteinsi:

650

440

200-3400n

35-50a 0.5-7_5., "'

0.4-1.0a 0.01-0.022o:t "'

variable"'

Drugs·also ·bimHo·tissue· components·(e_g., ·proteins,· DNA;bon_e. · fat}·and thL1s,·a·portion·of-the·dru:g·in·the·tissue·is·bound·while·the·remarnder·of- © the-drug ·is·tmbound. ·Unfortunately, '\issue· binding·canno\'1:1as1ly· be· measured;:!irectly, ·bt.rt·binding ·to-plasma· protein-can-be· measured-easily ______ One-very·irnportant·reason·to ·foous·on· unbound ·drug·levels·is·the·faot· that-only unbound· dl'tjg :rs· able·to'1nteract·with· receptors, ·therefore,· unbound·drug- cormentrationsiiiotate·pharmaoologic·effeot. ·The'!)rocess· of·protein·binding· is·a· dynamic processwhere·drug·bound·to·protein·is·in· equilibrium·with·unbound ·drug. -This· is·an· important·concept{Q· underst_and, ·especially-when-<liscussing · hepatic·clearance.iJ

' ~·,_·L..., a __, --... -... ~.

• • Figure·3-5:£quiNbrium-exisis·between· drug·bound-to·proteins·in1he·biood, · b unbound·drug·in·=d·out·of·the·ceJlwid·drug·bound·to-tissve11

1

Below-is·an·equation·to·describe·how·tissue· bindlng·and·plasma·protein· binding·determine·the·11otume·of1:l1StributioniJ

Notes. (1) example table; (2) example graphic; (3) side-bar; (4) main text

preparation (reading versus the multimedia module) during an authentic classroom experience and summarizes factors related to their preference. A secondary objective is to examine students' preferences in relation to individual quiz performance in a TBL course. Based on the pilot study, we hypothesize that students would preferentially select the multimedia module over reading for pre-class preparation. Also based on pilot data, the secondary hypothesis is that there will be no difference in scores on the individual readiness assurance test (iRAT) between methods of prepa­ration. Results of this study could help inform instructors of reasons why students choose a specific format for information delivery for pre-class preparation.

t: QI

:E' 0 Oil = -~

E " QI

>-< .,, ll

-~ <

rl .s <!) --" ..0 -~ "' .. ~~ ~ ~ ;e " ~ 1' = 0

"' ·5 6 0 u

"' "' .. ~ ~ g ;i 0 ,..c "" ..= "' a oo rl ::;: . . . .

'"ljJ~ljJ

] EE t: .......... " --.... c ~ " " <!)

""' ~Cf)

-~ " "' - 'O ~ 15 Co :;:

.;:; 0

"" 0 " 0

" "'" ~ "'

J

.nTLUrtiULUfL Ub. L\.l::UULrLg JUT rrt:-LlU/jlj 1-rqJaranon

< --... z

~ rl ~

rl .±! >::J"i .-. -±!.­......; t--.. II)

rl "' . . "' .. 0 .-I

~~bb e II) o ~ ci ).I.;

..... .-I b.() ~ .. = {JC..?·~

"'~ § ..9:l ~ ;J "' en C..?

]

1 g 1 .i::'~ ~ ~ ·~ 0 ~ '.Jj

~-~ ~~

~ :~ 'O .8 "' 00 N

] "' "' "' E " ~ ..... 0

0 >< ·.i: ..... 0 ci;I ti 1-i s ~ 2::·2 °' < "'

"' rl

§ t3 ~ --... "' 2 fJ<

{J ~

0

11

l

"' -~ E "' rl rl

.8 "' "'

"en

~ "§

"' N

.8 °' rl

"'

i .8

~ 'O

"' - = "' 0 E . ., ·.i: u "' <!) ~ en

"" rl

00 C'i

gj 0

t3 ~ --... "' i

{J ~ o,_. 11 < sf.'1 " .. z~

"' ~ ·§ "' 21 .8 g;

~ § "§ J:2 .8 "' "'

] 0.

§ .8

.§ -'O f-<

t;j~ E ·~

'.Jj 1-1

"' <!) ~ 0.

~ = 'O 0

~ ..9:l 'O <'il ]~ Ei > ~<

• •

en ::;; >-< § 2 {I ~ ~ •

N ·g,

1 "' & "' "' ~ en

13

l 'O <!)

~ :E gj '1j ~ .;!3

<!) .. u ..0 <!) .. s - <!) ~ ,8 ~ • •

~ Cf)

~ ~

:@

2 ..o en ~~ "' >-< > = < 0

•

~ ., u

~

] ~ "' "'

"

§~ ~ >-< 7il = - 0 ·;:a U'.l ,,_ <~

Table 1 (continued) Comparison of Reading Material to Animated Leaming Object

Format

Reading Material

• Electronically (PDF) on LMS • Built in index and table of

contents • Original graphics and

animations • Custom equation

nomogram • Side bars emphasize

important concepts • Also available printed

Animated Module

• Designed on Shareable Content Object Reference Model (SCORM) standards

• Web-access • English language audio

narration • Closed captioning

synchronized text • Compliance with University

accessibility guidelines • Original graphics and

animations • Customized tools: Equation

Helper, Glossary, Scientific Calculator

• U s_er preference settings tool

Notes. "Run time is the amount of time if animations were place end to end. bSeat time consists of completing practices and quizzes and is estimated by -2 times the run time.

'Based on 138 to 200 words per minute reading time (Carver, 1992) LMS = Learning management system (i.e., Blackboard or Sakai)

Table 2 Summary of Pilot Data Comparing the Online Module

to Lecture in a Randomized Study (Bell & Schraff, 2008; Persky, 2008)

Lecture (n = 63) Module (n = 53) Lecture vs. Module

Pre-Test 4.0 (2.0) 3.0 (1.0) p > 0.05

Post-Test 6.0 (2.0) 5.0 (2.0) p > 0.05

Pre vs. Post p < 0.05 p < 0.05

Presentation Method Students' Order of Preference(%)

#1 #2 #3

Lecture (n = 63)

Reading(%) 5% 39% 56%

Lecture(%) 95% 5% 0%

Module(%) 3% 57% 40%

Module (n = 53)

Reading(%) 18% 16% 66%

Lecture(%) 58% 28% 14%

Module(%) 24% 56% 20%

>--< a

~ 0

"' ~ § re

a §" " "' "' g ~ >-' n>•

" !

§ ~ §

~ I " • ;: ~ '--~

' " ' ( ;: " " ' " " ~ ;: ~

' '

l

lL 1oumat on r,xceuence zn couege ieacnzng

Methods

Pharmacokinetics is a 3-credit course within the doctor of pharmacy program at the University of North Carolina Eshelman School of Phar­macy. The course has utilized TBL for three years (Persky, 2012). Prior to each of the five TBL modules in the course, students were given learning objectives for the instructor-developed study materials: a book and a fully animated, narrated multimedia module that included practices and quizzes (see Table 1). Problem sets, practice quizzes, and old examinations were made available to all students on a learning management system for self-assessment purposes only. A total of five iRATs were administered each year, and these assessments were similar year to year. Each iRAT contained 10 to 15 questions, totaling 60 questions for the semester. At the end of each semester, students were asked to indicate their preference for pre-class preparation using a bipolar-scale with each forma,t as anchors. Students also were asked to cotntnent on why they preferred one format over another in an open-response question.

Open-ended comments were initially coded by hand to derive initial "open" codes. The "open" codes were reviewed, and the coding scheme was revised to account for format (the book or multimedia module) and positive or negative items. Student comments were then coded using NVivo (version 10, QSR International, Burlington, MA). Themes were generated from the final codes.

Data were analyzed for normality and equal variance. Based on the results, a Kruskal-Wallis test.( an analysis of variance [ANOVA] on ranks) was used to compare the median iRAT scores across preferences (Statistica 10, StatSoft, Inc., Tulsa, OK). This study was deemed exempted for review by the University's institutional review board.

Results

Three hundred sixty-four students completed the survey (a response rate of 79%). Students' age upon admission to the program was 19-51 years old, with an average age of 23. Approximately 70% of students had a previous degree; the remaining 30% were equivalent to junior- and senior-level undergraduates. In addition, average student scores on the pharmacy college admissions test (the standardized entrance examina­tion) was in the top 85%, and the average entrance grade point average was 3.5 (out of 4.0); both these metrics indicate well-prepared students.

The majority of students preferred using the reading material for pre-class preparation (see Table 3). Therefore, I reject the hypothesis that

N(%)

fl.nzmanon vs. Keaazng JOT 1_,re-Ltass 1-.1reparatzon

Table 3 Comparison of Preference for Pre-Class Preparation

and Individual Quiz Score

E-Book Module Preference No Preference Preference

244 (67%) 62 (17%) 56 (16%)

Median Score' 87' 83 83

Interquartile 12 15 12 Range

10

Notes. 'p = 0.05 Kruskal-Wallis test vs. No Preference and Module Preference; bquiz is a composite of 60 individual readiness questionnaire questions.

students would prefer the module to the reading. Two hundred seven­ty-eight comments were submitted, resulting in 20 codes categorized by format (reading or multimedia module) and direction of preference (positive or negative). From these codes, three themes emerged: locus of control, personal learning preference, and sense-making (see Table 4).

Locus of Control

The first emergent theme was locus of control, which accounted for 34% of the cotntnents. This theme appeared to be the dominant theme for students that preferred using the book over the module (see Table 5). The elements within this theme were based on students' ability to efficiently self-pace their learning, the ease of reviewing material during study or re-study, and the ability to access the format anywhere. The most common response within this theme dealt with the ability to self-pace learning and was the primary reason stated by those that preferred the book:

With the book, I can read the material at my own pace, high­light if needed, and take notes. The module can sometimes go faster than I'd prefer, and I would find myself rewinding and re-watching material from time to time.

The book is much faster to go through because I can read at my own pace .... I will say, that the modules were helpful-when I made the time to go tluough them-but honestly, it took me about a good hour to get tluough each module, and I started getting tired of going through them.

l

"" 1uurnui un .c:x.c;eumce in LOLLege 1eacnzng

Table 4 Frequency of Codes by Format (Positive/Negative)

Locus of Control

Self-pace or efficient Convenience

Review

Preview

Study Preference

Listen Read

Visuals

Interaction

Format Attention

Sense Making

Organization Content Understand

Book

106/l

65/1 12/0

27/0 2/0-

123/9

25/6

2(0

45/0

41/0 10/3

73/0

9/0 31/0 33/0

Module

5/85

1/70 0/1

4/14

0/0

81/45

16/11

28/4

30/0

0/21 7/9

23/20

4/1 17/19 2/0

The most-frequent response in this theme was the ability to review ma­terial, either during the initial study or at some later time--for example, when doing cases or problem sets. The ability to review was not among the top-five reasons for those that preferred the book to the module:

I really liked having both, but the e-book made it possible for quick review or reference, whereas the module takes longer to find a particular issue.

The final major area within this theme was the ability to access mate­rial anywhere. Ease of access (convenience) was not among the top-five reasons for either student preference, however:

I preferred thee-book because I could go along at my own speed. The module was a bit fast paced. I also preferred the printed version of the book so that I could take it with me anywhere and be able to read it.

L"lrurnuriun -ul::i. I'\.euurng JUT .rre;-LtUl::il::i 1-rt:purunun

~ .g ~ !> ~ 0

"' ~ E i

.,., I';

"' ... - = ~~

"" " ... "' = fl

~ £

~ ... LO

~

1!l "" " ~ ~ t; "' " 0 ,..., a ""u N

'" ~ ~ ~

::i " :: ~+ :: -""' " ~ ~ " ·~ "" > 0 ~ ::g ...1

"' 0 E 0 '" 0 8:::;;

1!l "" " - '" "' " t; "' f:> O N

a ""u "' ~ 1! u E-< ...1

::i ti~ ,_ + ~­:: "' ""' iii .-"! ~ 0 :::1 0 "' "' <fl "' ~ E O ~ 0 a "'

" I "' ,....., '<""""I l·r-.. I \0

~I~ I:::;; I~ ...J i..J C/) i..J

+ -- :±: § •Jj

J ~ 00 .,.,

;:J

2 ~

i ~

-~ "' ~

~ ::g:

~

:±: §

1 8 "'

:±:

I u ::i 0

~

~

~

:±:

j ~ ,..., "'

ill

+1+ - -1" E 0

"" I u

5[5 0 0

"' "'

"' ~

] II

~

~

2 l 00 ~ .: 0 :.a rJ';

"' 8 ~ ..s ~ II 0 u 0 ...1

~' "' I ~

l ~co

':'-:.§ 5] s "' s "' 0 .: u ~

~ II "Jj :::;; "Qj C/)

& Q;' ~ u -:±:, 5

. '" ~~ ~"' z lS.,

"o

w 1uumaL on r,xceuence zn Louege 1eacnzng

It [e-book] is more portable and convenient because I do not need my computer.

Personal Learning Preference

Personal learning or study preference was the most-commented-on theme, accounting for 45% of the student comments. Preferences included listening, reading, visualizing, interacting, and any comments referring to maintaining students' overall attention. This was the dominant theme for students that preferred the module over reading (see Table 5). The most common response within this theme related to interacting with the material, either the built-in interactions within the multimedia module . (thatis, practices and self-assessments) or the ability to highlight and take notes on the reading. Interaction was among the top-five most cited for those that preferred reading ( #3) and those that preferred the module ( #2):

I prefer the e-book (priI)ted out) because I can highlight im­portant points and take notes in the margin. This helped me to prepare for quizzes and to go back and review for exams.

I found working through the modules with pen, paper, and calculator in hand to be a slow but very clear way to learn the material. This was because as the material is presented and you can test yourself as you go (press pause and work the problem before the module does), and you then retest with the end-of-section quizzes. I found this helps prevent delusions of understanding that can happen when I just read but do not test. Follow this up with the self-assessments and homework problems and you will be set.

The second-most-frequent learning preference was for reading, which was not among the top-five reasons stated by those that preferred the book; but a preference against reading was among the top-five reasons stated by those that preferred the module (see Table 5):

Ilike to be able to read and then reread. The module is too much of a hassle to rewind and re-watch. I do not like studying from a computer screen, so I prefer the printed out textbook where I can highlight and scribble notes.

I find that I have a hard time reading thee-book because I of­ten don't feel like I understand the concepts, and reading it is boring to· me.

The use of visual information-graphics and animations in the multi­media module, and figures and graphs in the book-were third among

rinzmatzon vs. 1'\..eaazng JOT l-'Te-1,_,tass 1.,rq;ararzon u

the preference theme (see Figure 1). The use of visual information was the primary stated factor for those that preferred the module (see Table 5):

The book goes more in depth about concepts and has useful charts and examples that help in understanding. The module is a good supplementary resource, definitely not a replacement!

The module visualized the information and helped to link it together much better than the book could. Many people who complained about the class not having a lecture or being difficult need to use the modulei it makes it much easier to understand in my experience.

Sometimes the module takes too long explaining certain things and too fast explaining others, so it makes me better to read through the book on my own. Also, sometimes the multimedia can be distracting as well.

Auditory information was toward the bottom of the learning-preference theme. For those that preferred the module, hearing information ranked among the top-five reasons stated (see Table 5):

I liked having the module because I was able to read and listen at the same time. It was also easy to replay each short section and watch the visual aids. So it generally appealed to more than one learning sense.

I learn better from reading and stopping at concepts that are hard for me to understand as opposed to listening to a lecture.

The overall ability of the format to hold students' attention was an im­portant feature and related to other factors within this theme. The ability of the module to hold attention was among the top-five reasons for those that preferred the module:

The modules feel like a lecture to me, and it is easy for me to tune out someone else talking. If I have to read it to myself, I am actively engaging and more likely to pay attention to the material.

I like the interactive nature of the module. I tend to glaze over while trying to read the book. Module is more interactive com­pared to reading the book! It makes me stay awake to click to the next sections (book can put me to sleep because it's not active).

The final cited area was the format of the materials, that is, printed or on a computer. The printed format of the book was a top-five reason stated by those that preferred the book:

,0 1uurnut un .c:x:ceucru;e zn L-Uuege leucnzng

I don't like studying from computer screens. I prefer something that I can read and write on and make notes in the margin.

Sense Making

The final theme focused on whether the information made sense in terms of organization, content detail, and· the students' overall under­standing of the material. The amount of content detail was the biggest issue, with student comments related to insufficient detail, appropriate detail, too much detail, and different material being included in the two products. Students that preferred the book or module both had positive comments about the content as important (that is, among the top-five reasons stated):

I prefer thee-book because it is more thorough, but I do like the module for use after to review.

I liked using both equally~the book for quick cramming and skimming of main ideas, and the module to explain diffici.tlt concepts.

The organization of each format received numerous comments. Or-ganization was not among the top reasons stated for either preference:

It is easier for me to identify what I am supposed to learn from thee-book, which makes it easier for me to organize the material. The module is a great visual tool, but takes a long time to get through so I often end up just reading.

I liked having the color diagrams and organized format of the module. The module also organized my time into sections. Sometimes just looking at a whole chapter of a book overwhelms me, so having the module break it up into sections really helps.

If the module corresponded more closely with the book, I would have chosen the module. I would normally prefer an interactive learning tool, but the module: 1) was in different order than the book, 2) too fast at times and too slow at other times, and 3) some of the questions aren't set up very well.

The final area pertained to overall understanding of the material. Over-all understanding was not a top-five stated reason for either preference:

The e-book is easier to understand. I think the module over­complicat~s things.

The module seemed to be straightforward and well laid out. I

L"1nrrnunun V/5. 1\..eaazng ror J're-Ltass Yreparatton

also liked how it mostly condensed a lot of the material from the book and presented it in an easy to understand manner.

H

A secondary, more quantitative approach was used to examine whether student preferences were related to performance. Performance on the iRAT was significantly better for students who preferred the reading ma­terial (see Table 3). Therefore, we reject the null hypothesis that students' individual quiz performance was independent of the format preference.

The majority of students (52%) used both resources during the course, and 44% used only the reading material. The majority (75%) of students who preferred the module also used the book but the majority of students who preferred the book (56%) did not use the module. There was no dif­ference on exam performance among groups (p = 0.18).

In two of the three student cohorts, students were asked to approximate how much time per week a student should spend for an individual course (assuming a 3-credit course is standard). The median response was 3 hours (inter-quartile range = 2 hours; average = 3.5 hours); this equates to 1 hour of pre-class preparation for every hour in class for a 3-credit course.

Discussion

Various approaches can be used to offset the acquisition of content knowledge outside of class, but students' preference for pre-class prepara­tion can depend on accessibility and efficiency. In pilot data, an animated module prepared students as well as lecture, but lecture was more pre­ferred by students than the module. However, reading was less preferred than the module in that pilot project. This analysis revealed a different pattern of preference compared to pilot data, in that student preference for pre-class preparation was reading, not an animated module. Overall, it appeared that locus of control was the predominant theme in students' selection of pre-class material format, with the book receiving more posi­tive comments about the ability to self-pace. While student preference is an important factor, it may be more important to focus on student learning outcomes. Contrary to the pilot data, there was a clear difference in pref­erence for pre-class preparation that did seem to translate to learning as measured by iRAT performance. The preference and performance aspects could be explained by several factors.

Locus of Control

Locus of control or perceived control over a task is a central concept in intrinsic motivation (Pintrich & Schunk, 2002) and may drive student

JVl-H flUt Vfl £..A.Lt:Ht:flC.:t: LrL L.ULLt:gt: lt:Ucnzng

preference for pre-class material format. Both thee-book and module had elements of locus of control, because both could allow for self-pacing. As captured by student comments, the e-book was easier to skim, review, and reference, and it was more portable than the module. The module did have fast-forward, rewind, and pause capabilities and tended to work best with a computer, although it could be used on "smart" devices. These rewind and pause features are the same features found on any video.on the Internet; therefore, it is not necessarily an unfamiliarity with the controls that made students dislike the ability to control the animation. Even within those that preferred the module, self-pacing was not a top-five factor. It has been documented within educational technology that user control to self-pace is a key element to learning (Mayer & Chandler, 2001). The. module was narrated at a relatively fixed speed, which may place it at a disadvantage, giving an advantage toward reading, because students could more easily adjust the amount of time they spent on a topic. In this study, the positive ability to self-pace with reading and the difficulty to self-pace the module was the primary reason students preferred to read.

The overall length of the materials also could have been of iinportance. It is probably quicker to complete the reading than the module, and the book provided sufficient detail to prepare for the readiness-assurance pro­cess. The strategy to complete a task quickly to obtain the desired outcome could be viewed as a strategic learning strategy, where students intend to obtain high grades and organize their time and effort for greatest effect and efficiency (Papinczak, 2009). Based on Table 1, assuming that students read for comprehension at 138 words per minute (Carver, 1992), and each iRAT covered an average of 2.8 chapters, it would take approximately 1 hour to read for each iRAT; this does not account for any additional study time that may be necessary to prepare for the quizzes. For the multimedia module, each iRAT covered on average 1.8 sections, for a total 1.8 hours of animation per iRAT. However, when the embedded practices and quizzes are included (that is, seat time vs. run time), it could take approximately one hour longer to complete the module than the reading materials (1. 9 hours vs 2.8 hours). The inclusion of practices may be perceived as the module being longer to complete. Students reported that they should spend about 3 hours preparing for class for a 3-credit course, which is within the amount of time to complete the reading-especially if addi­tional time is needed for re-reading, studying, and completing practice quizzes. It should be noted, however, that this 3-hour time expectation includes pre-class preparation and does not include time to complete assignments and any other work associated with the class.

.tinzmarzon vs. 1'\.eaazng JOT 1~re-Ltass Yrepararzon Ll

. Personal Learning Preference

While locus of control may be a differentiating factor, the majority of student comments were based on personal learning or study preferenc­es; this was the major reason students preferred the module-positive comments for visuals, interaction, and narration. Based on the theories of cognitive load, there are some theoretical advantages to combining visual information (e.g., pictures, animation, video) with narration (Mayer, 2005; Pass & Kester, 2006); this advantage was the primary reason for developing the module. Cognitive load theory suggests that the processing of visual and verbal information is constrained by working-memory capacity. The use of technology can take advantage of the modality effects or du­al-coding theory. Dual-coding theory suggests that images and words are processed in separate, limited-capacity channels of short-term working memory (Mayer, 2005). However, the dual-coding theory may be negated by pacing, split attention, redundancy, density, or other issues that may tax cognitive load (Kirschner, Ayres, & Chandler, 2011). In this case, some students commented on the module being too fast (a pacing issue) or the mathematical nature of the material (material density), both of which may place additional stress on cognition. In one study (Scheiter, Gerjets, & Catrambone, 2006), the use of animations to visualize mathematical solution procedures was unhelpful in learning problem-solving skills. The authors discuss that frequent use of animations can lead to increases in learning time and a decrease in performance. Animations are transient and present dynamic information, thus making it difficult for the learner to sufficiently process information (Ainsworth & VanLabeke, 2004; Ayres & Paas, 2007).

Thus, keeping the important information in working memory while constructing an internal representation could increase the cognitive load (Sweller & Chandler, 1994) and be a partial reason why a novice or expert might process video differently (Argue! & Jamet, 2009; Betrancourt & Tver­sky, 2000; Hasler, Kersten, & Sweller, 2007). Because of its transient nature, a student has no control over the pace of presentation of disappearing information (Argue! & Jamet, 2009) and is forced to rewind the material. As soon as learning time is limited, reduced learning occurs (Paas & Kes­ter, 2006; Paas, Renk!, & Sweller, 2003). Thus, it could be hypothesized that reading, in this investigation, may place less pressure on working memory compared to the narrated animations within the modules. It is understood, however, that reading is a cognitively demanding task (Destefano & Lefevre, 2007; Kalyuga, Chandler, & Sweller, 1999; Moss, Schunn, Schneider, McNamara, & Vanlehn, 2011).

I

LL journal on bxcellence zn College leachzng

The preference for reading could be for multiple reasons. Perhaps stu­dents are more accustomed to traditional forms of instruction. The biggest area of student comments both within the book and module preference was the ability to interact with the material.The online material featured built-in practices and quizzes, while the reading interaction came in the form of writing notes and highlighting text. These interactions were ob­viously a function of having access to printed material (approximately two thirds of the students tended to print the book). Interestingly, a sub­set of students did not want to interact with computers any more than absolutely necessary. It is important to note that students' preference does not necessarily translate to student learning. For a number of years, it was believed that learning styles were an important determinant in learning. The theory was that if a student had a given learning style, then matching instruction with that style would result in greater learning. This approach is referred to as "meshing." A recent review of the literature suggests that the theory of meshing does not hold true, that is, learning preferences do not relate to ability to learn (Pashler, McDaniel, Rohrer, & Bjork, 2009). The idea of learning preferences stems from study-skills literature (Hodges, Simpson, & Stahl, 2011), and students who prefer one format may naturally choose that preference for studying, although it still may not be the optimal method.

vVithin the learning preference and locus of control themes, some students responded negatively to the multimedia materials, saying that it was sometimes difficult to watch the animations and listen to the nar­ration but this was not a primary (top-five) reason (see Table 5) stated by those that preferred reading. This difficulty may be a function of extra­neous cognitive load by having animation and narration simultaneously presented (Ayres & Paas, 2007) and has been discussed with respect to cognitive load. It also could be caused by difference in study or learning strategies. Students may use different learning strategies when working with video or animation compared to reading (Kombartzky, Ploetzner, Schlag, & Berthold, 2010). Kombartzky found that students with a study strategy learned more from animations than those who that did not have a study strategy (Kombartzky et al., 2010). Recently, researchers showed that students utilized less critical-thinking skills when a patient case was presented as a video instead of as a written case (Basu Roy & McMahon, 2012), which may emphasize differential learning between formats. It is unknown if the study skills used when reading versus when engaging with the multimedia module were different; this issue warrants further exploration. Students observed that it was easier to take notes on the read-

Anzmatwn vs. 1~eaazng 1or Fre-uass Freparatwn Lo

ing materials than the animations, and this observation may be a result of a greater familiarity with reading materials in academic environments.

Sense Making

The final theme of student comments, sense making, seemed the least influential of the three themes. Students will learn if material has meaning and makes sense (Ignelzi, 2000; Sousa, 2006). The level of detail between the two formats did vary from section to section, but ultimately the module was largely built off the book, although the organization was different. The students were provided with learning objectives and guidance on what chapters or sections of the material within the book and module related to each quiz. The organization may be a contributing factor if students could not see the direct relationship between the RAP and the pre-class material.

Another area related to sense making could be the readability of the text. The reading material was developed by the instructor and has a reading index of 12 to 14 (see Table 2). A previous study within pharmacy demonstrated that pharmacy student reading ability was equivalent to grade 16 (range 9 through 19), and the typical books and material used within pharmacy education are at a reading level of 18 to 19 (range 16 through 21) (Fuller, Harlen, Cisneros, & Merz, 2007)

As demonstrated by the Fuller et al. (2007) study, one reason students may not read prior to class is because of the complexity of the text. The material that was developed for this particular course should be within the reading skill level of the student body. One appealing attribute of an­imations or recorded lectures is the natural filtering that occurs when an instructor puts complex text or concepts into a more digestible form that allows students to make sense of complex topics; this was accomplished here with reading material and the animated module.

Summary and Limitations

Students' overall learning from animation has had equivocal results compared to other visual representations (Betrancourt & Tversky, 2000; Craene, Kirschner, Valcke, & De Westelinck, 2005; Lewalter, 2003; Mayer, Hegarty, Mayer, & Campbell, 2005; Tversky, Morrison, & Betrancourt, 2002), despite the overall effect size of animations being around 0.40 when compared to static pictures across a variety of age groups, content areas, and types of knowledge (procedural, declarative) (Hoffler & Leutner, 2011). It may be of importance that the original theories on multimedia instruction by Mayer and his colleagues were predominantly of short

l

~· 1uu1nut un LJd.:t::ut::ru.:t: zn 1_.uuege 1eacnzng

duration (< 3 minutes). In this study, larger chunks of learning content had to be processed over a longer period of time; thus, it is possible that the study tasks in our research were more demanding, and the theoreti­cal advantages were minimized. In addition, static images may be more beneficial to novice learners than animation (Kalyuga, 2008). One final reason for potential learning differences may have to do with the amount of invested mental effort. Mental effort is influenced by perception of the medium (Salomon, 1983, 1984). Viewing an animation or video may be perceived as "easy" and thus learners may decide to refrain from a deeper processing of the presented information. This could potentially be overcome with appropriate study strategies.

One advantage of this study is that it was conducted in a real-life, au- . thentic classroom setting. In controlled conditions where students were allocated into groups, performance was equal. However, under real-life situations, students preferred to adopt the most efficient rµeans for gath­ering information, potentially resulting in increased learning. While the real-life circumstances are a potential benefit, it also could be a limitation. This analysis of student comments was limited in several ways. Students self-reported their preference after all of the iRATs were completed; therefore, there was no direct linkage between the pre-class preparation method used for any individual quiz. Students also chose which format to use, which could add to sample bias. Finally, practice was not taken into account. The multimedia module featured embedded quizzes and practices, but there are no data to suggest that one group completed more practices or self-assessments prior to the iRAT. A final question that is raised is the generalizability of the results. This study was conducted in a professional pharmacy program, a group of students that may be more · comfortable or have been more successful, through reading. The case can be made, however, that given the average age of students upon admission (23 years old), they may better reflect upperclassmen in an undergradu­ate curriculum and within the Millennial generation. Regardless, future studies should investigate these issues in an undergraduate population.

Implications

While there is increasing interest in using technology to promote self-di­rected learning, it is important for instructors to study how technology impacts learning compared to other methods. This research should include an examination of instructional design principles and how these animated or technological tools are used by the students. Both formats examined here seemed sufficient to get students to a baseline level of knowledge and comprehension.

r Antmatwn vs. Keactzng for Fre-Uass Freparation 25

Regardless of format, this study indicates that efficiency is an important factor in pre-class preparation. Students will choose methods that most quickly get them prepared and those that are easy to navigate, take notes from, and self-pace. For reading materials, it is important to consider the overall language and tone used, word counts, use of pictures, graphs and figures, and ability to print or annotate. Similar characteristics apply for videos, but visuals should be used to emphasize the most important or complex topics, and students may need additional instruction on how to study from them. It also would be important to take into consideration costs (that is, time to develop, money to develop, money to maintain, specialized resources) relative to the learning gains.

References

·Ainsworth, S., & VanLabeke, N. (2004). Multiple forms of dynamic rep­resentation. Learning and Instruction, 14(3), 241-255.

Argue!, A. I., & Jamet, E. (2009). Using video and static pictures to im­prove learning of procedural contents. Computers in Human Beluzvior, 25(2), 354-359.

Ayres, P., & Paas, F. (2007). Making instructional animations more effective: A cognitive load approach. Applied Cognitive Psychology, 21 (6), 695-700.

Basu Roy, R., & McMahon, G. T. (2012). Video-based cases disrupt deep critical thinking in problem-based learning. Medical Education, 46(4), 426-435.

Bell, M., & Schraff, L. (2008). Creation and refinement of a sustainable multimedia process in a higher education environment. Journal of the Research Center for Educational Technology, 4(2), 83-95.

Betrancourt, M., & Tversky, B. (2000). Effect of computer animation on users' performance: A review. Le Travail Humain, 63(4), 311-329.

Carver, R. P. (1992). Reading rate: Theory, research, and practical impli­cations. Journal of Reading, 36(2), 84-95.

Craene, B. D., Kirschner, P. A., Valcke, M., & De Westelinck, K. (2005). Multimedia learning in social sciences: Limitations of external graphical representations. Computers in Human Behavior, 21(4), 555-573.

DeStefano, D., & LeFevre, J.-A. (2007). Cognitive load in hypertext reading: A review. Computers in Human Behavior, 23(3), 1616-1641.

Fuller, S., Horlen, C., Cisneros, R., & Merz, T. (2007). Pharmacy students' reading ability and the readability of required reading materials. Amer­ican Journal of Pluzrmaceutical Education, 71(6), 111.

Hasler, B. A. S., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional animation. Applied Cognitive Psychology, 21 ( 6), 713-729.

~v 1uurnui un r,xceuence zn Louege 1eacn1ng

Hodges, R., Simpson, M. L., & Stahl, N. A. (2011). Teaching study strategies in developmental education: Readings on theory, research, and best practice. Boston, MA~ Bedford/ St. Martin's.

Hoffler, T. N., & Leutner, D. (2011). The role of spatial ability in learning from instructional animations: Evidence for an ability-as-compensator hypothesis. Computers in Human Behavior, 27(1), 209-216.

Ignelzi, M. (2000). Meaning making in the learning and teaching process. New Directions for Teaching and Learning, 82, 5-14.

Kalyuga, S. (2008). Relative effectiveness of animated and static diagrams: An effect of learner prior knowledge. Computers in Human Behavior, 24(3), 852-861.

Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351-371.

Kirschner, P., Ayres, P., & Chandler, P. (2011). Contemporary cognitive load theory research: The good, the bad and the ugly. Computers in Human Behavior, 27(1), 99-105.

Kombartzky, U., Ploetzner, R., Schlag, S., & Berthold, M. (201b). Develop­ing and evaluating a strategy for learning from animations. Learning and Instruction, 20, 424-433.

Lewalter, D. (2003). Cognitive strategies for learning from static and dye narnic visuals. Learning and Instniction, 13(2), 177-189.

Mayer, R. E. (2005). The Cambridge handbook of multimedia learning. Cam­bridge, UK; New York, NY: Cambridge University Press.

Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multime­dia messages? Journal of Educational Psychology, 93(2), 390-397.

Mayer, R. E., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promote active learning: Annotated illustrations versus narrated animations in multimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256-265.

Means, B., Toyama, Y., Murphy, R., Bakia, M., &Jones, K. (2010). Evaluation of evidence based practices in online learning: A meta-analysis and review of online learning studies. Washington, DC: U.S. Department of Education.

Moss, J., Schunn, C. D., Schneider, W., McNamara, D. S., & Vanlehn, K. (2011). The neural correlates of strategic reading comprehension: Cog­nitive control and discourse comprehension. Neuroimage, 58(2), 675-686.

Paas, F., & Kester, L. (2006). Learner and information characteristics in the design of powerful learning environments. Applied Cognitive Psychology, 20(3), 281-285.

Paas, F., Renk!, A., & Sweller, J. (2003). Cognitive load theory and instruc­tional design: Recent developments. Educational Psychologist, 38(1), 1-4.

Animation vs. Reading for Pre-Class Preparation 27

Papinczak, T. (2009). Are deep strategic learners better suited to PBL? A preliminary study. Advances in Health Sciences Education: Theory and Practice, 14(3), 337-353.

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2009). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105-119.

Persky, A. M. (2008). Multi-faceted approach to improve learning in phar­macokinetics. American Journal of Pharmaceutical Education, 72(2), 36.

Persky, A. M. (2012). The impact of team-based learning on a foundational pharmacokinetics course. American Journal of Pharmaceutical Education, 76(2), 31.

Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education: Theory, research, and applications (2nd ed.). Upper Saddle River, NJ: Merrill.

Salomon, G. (1983). The differential investment of mental effort in learning from different sou~ces. Educational Psychologist, 18(1), 42-50.

Salomon, G. (1984). Television is "easy" and print is "tough": The differen­tial investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology, 76(4), 647-658.

Scheiter, K., Gerjets, P., & Catrambone, R. (2006). Making the abstract concrete: Visualizing mathematical solution procedures. Computers in Human Behavior, 22(1), 9-25.

Sousa, D. A. (2006). How the brain learns (3rd ed.). Thousand Oaks, CA: Corwin Press.

Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instniction, 12(3), 185-233.

Tamim, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4-28.

Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human Computer Studies, 57(4), 247-262.

b

·---- -·- --~----H ............ H~ '--lJHC?)C 1.t::Ut:rung

Acknowledgments

The author would like to thank Catherine McAllister for her assistance and input during the preparation of this manuscript; the professional program (PharmD) students for their open-mindedness, feedback, and positive attitudes; and all the members of the TL Interactive team within the Information Technology Services (ITS) Teaching and Learning division, who, in part, helped develop the module.

Adam M. Persky is at the University a/North Carolina Eshelman School of Pharmacy, where he is currently a Clinical Associate Professor. He teaches Physiology and Pharma­cokinetics and has received several of the pharmacy school's teaching awards, including Best Overall Instructor. He has published articles on teaching and learning and is on the editorial boards of the American Journal of Pharmaceutical Education and College Teaching.

Brewer,-P. E., & Brewer, E. C. (2015). Pedagogical per­spectives for the online education skeptic. Journal on Excellence in College Teaching, 26(1), 29-52.

Pedagogical Perspectives for the Online Education Skeptic

Pam Estes Brewer Mercer University

Edward C. Brewer Appalachian State University

While online programs continue to grow at an astounding rate in higher education, many faculty remain skeptical of the efficacy of online models. This article provides an overview of some significant benefits of online education while recognizing some common concerns. An examination of the current liter­ature and the authors' own online experiences enable them to identify significant benefits in online education and suggest five foundational concepts for best practice. First and foremost, the goal of the online or hybrid classroom should not be to repli­cate the face-toface classroom but to offer an effective learning experience.

The debate continues. Online programs grow at an astounding rate, but skepticism as to their efficacy and a fear of losing the advantages of the face-to-face classroom persist. Online enrollment growth continues to outpace overall enrollment growth. In fact, a look at the data from a 2011 Sloan report shows that online growth has outpaced total growth since 2003 from at least 3.6 times (2008) to as much as 30.4 times (2005) (Allen & Seaman, 2011).

In a 2006-2007 study sponsored by the U.S. Department of Education, researchers reported that 66% of 2-year and 4-year Title IV degree-grant­ing postsecondary institutions offered online, hybrid, or other distance education courses. Of these, 61% offered fully online courses, and 35% offered hybrid online courses (Parsad & Lewis, 2008). In addition, 7 4% of public institutions believe that online education is critical for their long-

29