Practical Team-Based Learning from Planning to Implementation
Heather P. Whitley, PharmD, BCPS, CDE,a Edward Bell, PharmD, BCPS,b Marty Eng, PharmD, CGP,BCPP, CDP,c David G. Fuentes, PharmD, BCPP, CGP,d Kristen L. Helms, PharmD,a Erik D. Maki,PharmD,b Deepti Vyas, PharmDe
a Auburn University, Harrison School of Pharmacy, Auburn, Alabamab Drake University College of Pharmacy and Health Sciences, Des Moines, Iowac Cedarville University, Cedarville, Ohiod Pacific University School of Pharmacy, Hillsboro, Oregone Thomas J. Long School of Pharmacy and Health Sciences, University of the Pacific, Stockton, California
Submitted December 17, 2014; accepted April 29, 2015; published December 25, 2015.
Team-based learning (TBL) helps instructors develop an active teaching approach for the classroomthrough group work. The TBL infrastructure engages students in the learning process through theReadiness Assessment Process, problem-solving through team discussions, and peer feedback to ensureaccountability. This manuscript describes the benefits and barriers of TBL, and the tools necessary fordeveloping, implementing, and critically evaluating the technique within coursework in a user-friendlymethod. Specifically, the manuscript describes the processes underpinning effective TBL development,preparation, implementation, assessment, and evaluation, as well as practical techniques and advice fromauthors’ classroom experiences. The paper also highlights published articles in the area of TBL ineducation, with a focus on pharmacy education.
INTRODUCTIONTeam based learning is an instructional strategy
designed to support the development of learning inteams1 and is characterized by a 3-phase approach (Fig-ure 1). In the first phase (preparation), learners obtainnew information prior to class and then are held account-able for learned information in class with a brief assess-ment based on assigned material reviewed prior to classcalled the Readiness Assessment Process (RAP) – thesecond phase of TBL. The RAP includes a brief assess-ment of individual’s grasp on content followed by teamengagement of the same assessment. This sequenceseeks to leverage collaborative learning to assure thereadiness of the student cohort to engage problem solv-ing in the next phase (application exercises). During thethird phase (application), students share informationthey acquired while in teams to solve real-world prob-lems, apply learned information on an examination and,finally, provide peer feedback. Throughout this strategy,the instructor serves more as a facilitator than a contentexpert.
Teamwork permeates today’s health care institutions.The Accreditation Council for Pharmacy Education(ACPE), the American College of Clinical Pharmacy, theInstitute of Medicine, and the World Health Organizationpromote interprofessional collaborations in practice andeducation.2-4 Team-based care encourages academiciansto adopt educational styles that foster collaboration to solveproblems. While enhancing student education, TBL bene-fits health outcomes and interprofessional collaborations,in addition to improving faculty satisfaction and reducingfaculty workload.5
From a curricular perspective, the 2016 ACPE Ac-creditation Standards stress that curricula must includestrategies that actively engage learners and emphasize thatlearners should be given opportunities to apply founda-tional knowledge in a patient-care context.2 While thereare several ways to apply such knowledge in the patientcare context, TBL provides a simple active-learning strat-egy that can be employed on a regular basis within a largeclassroom context with minimal faculty resources. TheTBL model directly connects with the first, third, andfourth domains and competency areas in the CAPE Edu-cational Outcomes.6
All 4 domains from the CAPE Outcomes have beenadopted in full as the first 4 standards in the ACPE’s2016 Standards. The first ACPE domain, Foundational
Corresponding Author: Heather P. Whitley, MontgomeryFamily Medicine Residency Program, Baptist Health System,4371 Narrow Lane Road No. 100, Montgomery, AL 36116. Tel:334-280-7084. Fax: 334-613-3685. E-mail: [email protected]
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
Knowledge, is developed in phase 1 with overlap inphase 2 as readiness is assessed and feedback is givento students through a mini-lecture. Competency in foun-dational knowledge is significant to build on for ACPEdomain 3, Approach to Practice and Patient Care. ForACPE domain 3, student problem-solving is developedin phases 2 and 3 of the TBL strategy. In phase 2, lesscomplex applications may be used in the RAP, whereas,more complex problems are used in phase 3.
Through team efforts, students can enhance their abil-ities to think collaboratively, consider alternative view-points, educate others, communicate effectively to ensuretheir perspectives are considered, and advocate for patientsas they solve problems relevant to patient care and phar-macy practice. There is also opportunity to connect TBLto the fourth domain, Professionalism, by assessing in-dividual and team development of skills in the areas ofself-awareness, leadership, innovative thinking, andprofessionalism. This may be accomplished throughprofile assessments and team assessments that can pro-vide information useful in tracking student leadershipdevelopment and development of cognitive and affec-tive domains. Lastly, curricular content and experiencesmore loosely connect TBL to the second domain, Essen-tials for Practice and Care.
TBL is integrated into skill-set development in socialand health sciences within undergraduate and graduatecurricula, and across various topics. It has been exten-sively used in medical school education since 2001, andmedical educators are an integral part of, and leaderswithin, the TBL community. The success of TBL in edu-cating and training medical students is demonstrated inthe literature.7,8 According to Michaelsen, the pioneer inthis teaching strategy, TBL requires faculty members to
do the following: make their reading and preclass prepa-ration clear to students; use a mechanism to test students’individual understanding of content within the assignedreadings; use a mechanism to test students’ performancewithin a team that is consistent with, or similar to, thatused to ascertain individual understanding; engage stu-dents in discussion after the individual-based and team-based assessments to identify areas where knowledge isstrong and where it is weak.
When implemented well, this process can provideinsight to help instructors better target mini-lectures tofurther student learning. Team-based learning can beimplemented into any didactic course with a criticalnumber of students (around 10 at minimum).9 To ourknowledge, TBL has not been implemented in a purelyexperiential environment.
DEVELOPMENT, PREPARATION, ANDIMPLEMENTATION OF TBL
Forming teams is an integral part of the TBLprocess.Ideally, teams should be composed of 5 to 7 students withdiverse backgrounds to encourage varied perspectives.1
Team member diversification may include differing aca-demic achievement, abilities, and cultural background.To limit homogenous team formation that may occurthrough group self-selection, instructors should strategi-cally assign students to teams. Multiple methods exist forensuring heterogeneity, including having students answeryes/no to questions about specific characteristics (eg, ed-ucational background, grade point average, ethnicity) orcollecting relevant student information (such as learningstyle) before class, and devising groups accordingly.Once formed, teams should be permanent throughouta course or semester.
Figure 1. Team-based learning process.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
Learning objectives and phases of TBLWriting learning objectives for a TBL session fol-
low the same general principles as any other classroomteaching methodology. Instructors develop generallearning objectives for the entire course and specificobjectives for major units or course blocks. A unit, orblock, is approximately 6-10 hours of coursework.9 Apopular strategy for writing learning objectives for TBLis with the “backwards” course design, which uses ter-minal behavior objectives (ie, what the student should beable to do at the completion of the course) as the corner-stone for course development. If constructed optimally,the need for rote memorization of irrelevant details iseliminated.
Learning objectives also assist students in focusingon assigned readings during the preparation phase. Whilenot unique to TBL, preclass assigned activities are foun-dational to student success in all phases of TBL. Thepurpose is to provide essential background information,as opposed to a comprehensive in-depth review of thesubject. Reading assignmentsmay include book chapters,instructor-prepared handouts, prerecorded lectures,PowerPoint slides, or specific website reviews.10 Someauthors suggest a 30-page maximum for each block orunit of reading.11,12
The RAP holds students accountable for complet-ing the prereading assignment. At the beginning of eachmodule, students complete a brief individual quiz (usu-ally multiple-choice) called an individual readiness as-surance test (iRAT). In general, iRAT questions arewritten at the knowledge and comprehension level ofBloom’s Taxonomy to test students’ overall understand-ing of key concepts rather than factual knowledge. Aftercompleting the iRAT, the instructor collects all answersheets and distributes an identical quiz to be completedin teams—the tRAT. Teams discuss each question andarrive at an agreed upon answer. If an incorrect answer isinitially selected, discussion continues until the correctresponse is identified. Awarded points decrease in valueuntil the correct answer is finally chosen. This process isdynamic and, if the test is sufficiently challenging, gen-erates much discussion within the team. Instructors mayneed to clarify why correct answers are right and whyincorrect answers are wrong. Michaelsen contends thata short lecture may be used in this final step of the RAP;however, this lecture should be focused on what thestudents do not know, instead of reiterating the knowl-edge they already gained during the preparation phase.Afterward, students have the option to appeal missedquestions. This usually occurs if a team or individualbelieves that an item was ambiguous. The appeal shouldbe written and supported with sufficient evidence. Only
the team or individual drafting the appeal, if granted,receives credit for the challenged question. The RAPshould be used judiciously in the classroom to motivatestudents to arrive prepared. Frequent use may promotea “reward” system that favors memorizing details toacquire points and not necessarily the application ofcontent. A general rule is to use approximately 6-9 RATswithin a typical 15-week semester.
The next phase is the application phase. During thisphase, students work in small groups to use informationlearned in the preparation phase to solve complex ther-apeutic problems called application exercises (AEs)(Figure 2). Generally, AEs progress in difficulty andachieve higher levels of Bloom’s Taxonomy as studentsgain mastery of the topic. The exercises are carefullycrafted to help students gain an in-depth comprehensionof the material and also allow opportunities for teamdiscussion. To optimize learning, Michaelsen recom-mends the exercise process provide individual account-ability, team togetherness, interteam and intrateamdiscussion, immediate instructor feedback, and a rewardfor performance. Additionally, each AE should be rele-vant and applicable and ultimately lead to choosinga specific answer. Each teamworks on the same problemand reports answers simultaneously. Programs may uselarge answer-choice cards revealed by a designated teammember to answer an AE question.13 Dissention withinteams provides a starting point for discussion and oppor-tunities for input from multiple students.
Discussion serves to expand the students’ criticalthinking, as all facets of the solution are delineated anddiffering opinions offer alternative ways of thinkingabout the same problem. Discussion within teams allowsstudents to build support for their answers with evidenceand also challenges their own thought process throughpeer-to-peer debate. The instructor circulates in theclassroom to facilitate discussion and provide clarifica-tion during the AE. This not only gives students a sensethat an expert opinion is close by, but it also helps stu-dents stay on task. At the end of the discussion facilita-tion period, the instructor attempts to determine theclass’s level of understanding. During this time, the in-structor can reinforce basic concepts and tenets thatemerged during the RAP and application phase. Thisalso allows the instructor to fill in gaps in student learn-ing and highlight concepts that did not emerge during theapplication phase. Application exercises are not typi-cally graded but used for formative feedback. However,Michaelsen asserts that some reward system should be inplace to provide an incentive for group performance.This could be in the form of a small percentage of indi-vidual students’ overall grade.12,14
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
Equipment Needed for TBLIdentifying what resources are needed entails consid-
ering activities that will occur before, during, and afterclass. It is best done after completing the “backwards de-sign” process.9 Keeping TBL sessions organized and ontaskwhilemaintaining security of coursematerial is essen-tial. Learning management systems (LMS), such as Sakai(Apereo Foundation, Ann Arbor, MI) or Blackboard(Blackboard, Inc, Washington, DC), provide a secure andtraceable portal for student access and allow instructors tocontrol distribution. Alternatively, nondigital methodsmay be employed, the most economic method being paperfolders. Consider the direct cost of the system in addition tothe faculty workload involved with each method. Regard-less of the method selected, it is important to safeguardconfidential documents and educate students about whichmaterials may be used outside of class.
The iRAT portion of the RAP can be completed us-ing pen and paper, amachine-readable form (eg, Scantron[Scantron Corp., Eagan, MN]); electronically, using anaudience response system (eg, “clickers”); or an onlinesystem (eg, team-based testing system). Paper testing re-quires the least resources but is the most time consumingmethod for grading. Machine-readable forms and audi-ence response systems can expedite the grading processbut come with an additional cost for the forms and readeror devices and software.
The tRAT is more complex, as it requires immedi-ate feedback and multiple responses until a correct an-swer is identified. This may be accomplished usingImmediate Feedback Assessment Technique (IF-AT)
Forms (Epstein Educational Enterprises, Cincinnati,OH). These scratch-off answer forms have an opaquecoating similar to a lottery ticket, which allows teamsto immediately determine if their selected answer choiceis correct. Students can subsequently scratch off otherboxes in search of the correct answer. The IF-AT formsprovide opportunities for partial credit on the tRATdepending on the number of unsuccessful scratches. Stu-dents only receive full credit if they find the correctanswer on the first attempt. These forms are availablein lengths of 10, 25, and 50 questions, with either 4 or5 question/answer choice options. Each form has a tear-away number at the bottom that corresponds to a specifickey. The manufacturer of these forms – Epstein Educa-tional Enterprises (Cincinnati, OH) – also developeda free test bank program called “IF-AT Testmaker,”which keys items to the specific test form.
Anelectronic alternative to the IF-ATForm is anopensource program called Team Based Testing (TBT).15,16
Students individually sign on to individual Internet-readycomputers using this system to complete the tRAT. Eachgroup must choose a team leader, who then submits thetRAT answers. The system provides immediate feedbackon whether their choice was correct and dithers, or graysout, incorrectly selected answers for subsequent attempts.The instructor may download results in a spreadsheet for-mat and upload the information into an LMS system forconfidential student sharing.
Many of the same resources can be used for both theRAP and application exercises. To simultaneously dem-onstrate a specific choice, groups can respond by holding
Figure 2. Application exercise process.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
up a colored card for classmates to see. Similarly, for shortanswer or graphic responses, flip charts or white boardsmay be an option; however, if not built into the classroom,logistical issuesmay arise. Using nondigitalmethodsmayunfortunately hinder groups’ decision making if teamsobserve each other during the process. Using an electronicmethod (eg, ARS) where responses are blinded may helpavoid this issue. The technological learning curve in-volved in assembling and executing ARS sessions canbe an initial hindrance. Lastly, depending on the system,it can be challenging to link teams with their selectedanswers when students are not forthcoming. There areshared spaces in an LMS that can be used in this setting,including discussion boards, file exchange pages, blogs,and Wikis.
Traditionally, student peer evaluations are eitherhand-written or typed and submitted electronically. Inlarge classrooms, collating, reviewing, and grading canbe a daunting task. To date, most LMS’s do not have toolsto expedite this process. “iPeer” is an open-source pro-gram (Centre for Teaching, Learning and Technology,University of British Columbia, Vancouver, BC, Canada)designed for the TBL peer evaluation process. The pro-gram, which is written in PHP (Hypertext Preprocessor),must be installed on a local server and maintained byeach institution’s information technology staff. There-fore, start-up may be difficult because of the potentiallyunfamiliar software language and need for onsite instal-lation. Despite initial barriers, once the program is func-tional, it streamlines the peer-review process andmay beworth the effort.
Faculty training and workload requirementTeam-based learning may increase faculty workload
compared to delivering the same course material via tra-ditional lectures. Facultymembersmust identify or preparepreclassroom readings for students, with matching learningobjectives,17 which are a roadmap to faculty-identified keymaterial. This may include careful construction of supple-mental materials such as notes, PowerPoint presentations,or other resources (eg, video clips) to foster effective stu-dent learning.Once the reading and supplementalmaterialsare devised, appropriate RATquestions andAEs should bedeveloped, which may take several class sessions. Ensur-ing tRAT answers are coded correctly takes time. If usingnonelectronic means, preparation of materials includingphotocopying, placingRATs andAEs into individual teamfolders, and administering the iRATs and tRATs can betime consuming. Therefore, developing a new unit can betime consuming.
Faculty training should be specific to an institution’sneeds, aligned to TBL, and provide desired outcomes.
New faculty members should become acquainted withTBL through reading material and experiencing theTBL process before teaching. One method is to devisea nonthreatening TBL unit using a recent journal articlefor faculty training. Faculty members can submit unitobjectives, RAT items, and AEs prior to the trainingsession. The facilitator may then discuss which objec-tives, items, and AEs are appropriate. During the TBLtraining session, faculty members can complete theiRAT, tRAT, and AEs.
Team-based learning encompasses individual andgroup work so student participation and classroom timemanagement can be challenging. Some instructors sug-gest “time flags,” or other devices to alert the instructorwhen group discussion has ended. Instructors can thenmore easily gauge when most groups are ready to moveon to simultaneous reporting. Being flexible and manag-ing timemay allow planning for extra time to review topicconcepts and overflow of AE-stimulated class discussion,which can promote student learning.
FACILITATION TECHNIQUESThe primary role of the instructor in TBL is to
guide and encourage student learning by moving awayfrom lecturing and promoting student-led discussions.Through this method, instructors encourage criticalthinking, foster self-directed learning, monitor groupprocesses and progress, promote teamwork, encouragedebate and uncertainty, and create an environment con-ducive to learning.18 Facilitators in TBL are often con-tent experts who provide suggestions for resources,advice on how to learn, and encouragement and feedbackon content and process.
In TBL, facilitators must avoid providing too muchdirection, which may negatively affect group dynamicsand interfere with the development of skills related tojudgment, innovation, perspective, questioning, reflec-tion, and feedback.19 However, if facilitators providetoo little direction, teams may drift away from contentor process. The practice of facilitation requires instructorsto know when to intervene and when to allow groups tocome to their own conclusions.
Advocates of TBL argue that most experienced ed-ucators already have many of the skills and competenciesrequired to effectively implement TBL.20 The major dif-ference is that instead of focusing on how and what toteach, instructors focus on how to enhance student learn-ing. Essential facilitator skills include: creating a climatefor student-centered learning; responding to individualstudent needs; and guiding learners through their owndiscovery by asking open-ended questions as they engagein reflective dialogue and critical thinking.20
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
Facilitators create an effective learning climate bymaintaining an open mind and promoting inclusiveness.Clear and specific expectations for participation and be-havior are necessary for maintaining trust, cooperation,support, respect, and accountability. Instructors whodominate the discussion or lecture instead of encourag-ing dialogue can stifle group development and studentlearning.21 In guiding learners and encouraging dialogueand debate, facilitators can ask questions, summarizewhat has been said, or make observations in order topromote reflection.
Processes for iRAT/tRATFacilitation does not generally occur during the
iRAT; there is no discussion or facilitation while theiRAT is being completed. Students discuss content duringthe tRAT; however, the instructor does not facilitate dis-cussion at this point. Content-based questions may ariseafter the iRAT and tRAT are completed. Facilitation dur-ing this part of the TBL process will set the tone for theinstructor’s ability to command the class.
If the validity of the question is challenged, the fa-cilitator decides whether or not the challenge is legiti-mate. Rather than defending an answer, facilitators canacknowledge the student’s concern and reminds the stu-dent of the appeal process.22 This minimizes class disrup-tion, maintains the schedule, and allows for a morethorough review of the concern. At the end of the testingand appeal phase, facilitators may open the floor to stu-dent questions about the content or respond to items iden-tified in the individual test as challenging. A briefdiscussion or mini-lecture may be employed to clarifymajor issues, or offer evidence of personal expertise orcredibility.20
The timing of the RAT is determined prior to the firstclass. Some facilitators may require all students to com-plete the iRAT before the tRAT is started, which is whatMichaelsen recommends.23 Other instructors may givea 5-minute warning when half of the class has completedthe iRAT to help keep TBL activities on schedule. Thisstrategy onlyworks if iRATcompletion can be effectivelymonitored. Teams that complete the tRAT should be quietand remain in the classroom until the entire class is fin-ished. However, after a reasonable period of time, if allteams have not completed the tRAT, discussion can occuramong all teams.
All processes should be communicated upfront tostudents to ensure understanding of expectations, whichmay include that resources may not be used during theiRAT and tRAT and that answer choices on an IF-AT (ifused), may not be altered; doing so will result in the ques-tion being marked as incorrect. This includes situations
where students report accidentally scratching off an in-correct answer choice or report being pressured intoselecting an incorrect answer. If the policies are wellestablished up front, the facilitator may only need to re-mind students of the policy.
Processes for team application exercisesThe facilitator is responsible for monitoring group
progress and intervening when obstacles interfere witha team’s technical, informational, or motivational needs24
(Figure 2). Instructors canmonitor team progress by walk-ing around the classroom and observing students’ partici-pation and interactions.20 If students ask questions aboutthe application, the facilitator should respond by askingopen-ended questions that engage students in critical think-ing. A facilitator might respond to a question by asking,“Howwould anyone else answer that?” or “Where did youlook to try to find the answer?” or counting to 10 beforeaddressing students’ questions.20 If a question posed by anindividual team needs to be answered to move the AE for-ward, the facilitator may decide to address the entire class.However, a specific question arising from a team’s discus-sion may be addressed only with that team.
As team members collaborate and develop norms,roles, and expectations, a sense of accountability mayemerge. Preferably, team members will engage a non-participatory student rather than relying on the instruc-tor to prompt that individual to contribute. Groupground rules set by teams are beneficial.25 Such rulescould include no texting, surfing the Internet, studyingfor another class, or loafing while AEs are being com-pleted. However, if individual teams do not devise suchrules, the instructor may need to intervene. Althoughnot preferred, another strategy to engage a disinterestedstudent is for the instructor to ask that individual for ananswer to the AE. This may have consequences, how-ever, such as forcing the student to become more with-drawn and even resentful toward the instructor.
Another tactic that may raise team accountability isto allow the nonparticipatory student to select a teammember to share the spotlight. Team members may re-alize that this peer is responsible for the instructor’s ad-ditional attention and require better engagement in thefuture. Anonymous peer evaluations within each teamare also useful for maintaining individual accountability.The instructor should provide instruction at the start of thesemester about giving helpful, constructive feedback, inan effort to avoid students giving unsupportive, negativecomments.
Other strategies for enhancing student participationwithin teams include: arranging seating, if possible, ina circle or semi-circle so team members are facing each
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
other; using eye contact and other nonverbal communi-cation (nodding or smiling) to open a communicationchannel and select a student for a turn to speak; movingtoward the speaker to indicate interest in the presentationand that others should be listening as well; emphasizingthe value of student contributions or responses by para-phrasing or making connections between their contribu-tions and the discussion at large; using a token system(teammembers receive a certain number of tokens, whichare turned in each time the student contributes—the goalis to turn in all tokens by the end of the session); limitingfacilitator involvement in the discussion and encouragingstudents to develop their own ideas, questions, and re-sponses; drawing quiet students into the discussion withnonthreatening questions that do not require a detailed orcorrect response and encouraging other teams to expandon those ideas; and calling on different students in thegroup to increase attentiveness, participation, and ac-countability.26
At times, teamsmay struggle reaching an agreement.However, techniques may be employed to facilitate teamconsensus, such as providing a summaryof teamprogress 24
or asking students to contemplate another solution or re-think their position. Another strategy involves having theteam identify criteria they will use to make a decision andthen evaluate ideas against the criteria.27 This “goaround” technique requires each teammember to explaintheir assessment approach, which allows similar and con-flicting perspectives to be revealed.28 Using a majorityvote is not representative of consensus and should beavoided, if possible. Teams should make every effortto achieve consensus through discussion. Facilitatorneutrality is essential when it comes to students achiev-ing consensus and making critical decisions. Effectiveteam process may be sabotaged when members perceivethat the facilitator agrees or disagrees with a particularresponse.
One responsibility of the facilitator is to maintaina safe environment for dialogue and exploration. Con-flict among team members may be beneficial to a livelyand in-depth discussion about the AE. However, unre-solved conflict may produce a negative atmosphere andlimit effective teamwork. Ideally, team members areable to negotiate the conflict without instructor interven-tion, which ultimately helps to build team cohesivenessand provides the students with valuable lessons in con-flict resolution. Facilitators should allow students to en-gage in conflict resolution on their own withoutdisrupting others’ learning. When facilitator interven-tion is required, facilitators should discourage criticismand personal attacks, and instead refocus the discussiontoward concepts. This may involve carefully phrasing
responses to unconstructive comments and establishingground rules that promote and honor diversity.
Processes for class discussionThe most important skills of facilitators emerge dur-
ing discussion and sequential reporting of AE answers.Because this aspect allows teams to compare answers, it isessential for facilitators to focus classroom conversationthrough thoughtful questions, help students appropriatelysummarize their key points, support critical reflection,and provide positive feedback for participation.20 Whenstudents are reluctant to share individual or team ideas,one can ask what others can add to the idea. If a team doesnot engage in discussion, the facilitator may physicallymove closer, shifting focus to that team.29 Students canalso standwhen they are speaking or asking a question. Toprompt discussion, facilitators can inquire about accept-able, alternative options, or the rationale used to eliminateincorrect answers.29 Facilitatorsmay inquire about differ-ing opinions within teams, which gives students who didnot necessarily agree with their team’s answer the oppor-tunity to express their views. Alternatively, one can askteams to identify changes in the AE that would alter ananswer from incorrect to correct.29 This may lead to dif-fering group answers during simultaneous reporting,which can generate class discussion.
Throughout the discussion while teams are sharing,the facilitator should avoid verbal or nonverbal cues thatindicate agreement or disagreement as value judgmentsmay interfere with discussion. The facilitator shouldexercise caution and resist offering expert opinion be-fore all teams have had a chance to share and sufficientdiscussion has taken place. After a given question hasbeen thoroughly discussed, the facilitator should pro-vide an explanation of the correct answer or answers.This is the facilitator’s opportunity to critique the dif-ferent options and explain content that students may nothave fully understood.
ASSESSMENT AND EVALUATION OF TBLAssessment of programs relies heavily on the goals
of the individual topic, courses, and curriculum. Becausethe extent to which TBL is integrated into a curriculumcan vary greatly, schools must determine where this typeof learning may best meet the needs of students and fac-ulty members. Students may be introduced to this methodof learning early in a program so that TBL can be utilizedthroughout all courses. Alternatively, specific topics orclasses may be identified for a TBL experience. It is im-portant to consider the workload requirements of othercourses in order to assess the feasibility of students com-pleting the out-of-class learning.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
Smaller elective courses may be more realistic forsome schools and can still provide students with the op-portunity to gain self-learning and teamwork skills com-monly acquired through TBL. Alternatively, using TBLfor larger class sizes or for core courses provides moreopportunity for students to workwith a variety of studentsand concepts. Regardless, TBL can be successfully ac-complished in a variety of course sizes as long as there isadequate preparation and planning.
Preparation and Assessment of Faculty MembersFaculty development must be incorporated into the
initial planning long before TBL is implemented17 to laythe groundwork and foster open discussion of concernsand solutions. Invitation of TBL experts to faculty work-shops may increase the comfort level and confidence offaculty members and reduce their resistance to change.These efforts require a significant investment of re-sources and support by the college, dean, and other keyadministrators. The dean and department chair in partic-ular should be prepared to support faculty members inareas such as travel or resource purchases, workloadreallocation, mentoring, recognition of innovations inteaching, and budget support for scholarship of teachingTBL. Less obvious is the need for their support duringdeclines in student and/or peer evaluations of teachingperformance. Any pedagogy that induces discomfort instudents may result in a decrease in satisfaction amongsome students. Evaluators must be able to differentiatestudent feedback on the pedagogy from feedback spe-cific to a faculty member. Faculty members may requireseveral years to improve, respond to peer feedback, andadapt teaching materials to TBL.
Continuous quality improvement is another essen-tial aspect in the faculty planning process (Figure 3).Consider developing faculty “teaching circles” thatmeetperiodically throughout each semester. These circles canshare their experiences, successes, strategies for im-provement, and new resources to maintain consistencyof course modules. The circles agree upon and providepeer-review for the overall course syllabus, modulestructure, teaching objectives, grading schemes, RAT,AEs, and examination questions. Lastly, the teachingcircle faculty members, in addition to the departmentchair, provide formative feedback to the instructors fa-cilitating TBL.
If TBL is team-taught, administrative encourage-ment for faculty members to support the process canmaintain consistency. The faculty members most resis-tant to adopting a TBLmodel may be those who believethey have successfully mastered more traditional ap-proaches. “Faculty champions” who are enthusiastic
and willing to invest in development may assist withbringing more resistant faculty members on board.Overall, encouraging faculty members to promote a student-centered learning philosophy will assist in the accep-tance of TBL and reinforce benefits to student learning.9
Assessing TBL faculty members and instructors can bechallenging. The majority of faculty efforts are spent inthe planning process rather than in face-to-face timewith students. This requires a special means of assessingthe effectiveness of faculty members and places of em-phasis on nonstudent evaluations. Students’ individualperceptions of TBL may affect their ability to objec-tively assess a specific component of the course. Addi-tionally, much of the impact of TBL, such as developingproblem-solving skills, may be best assessed in laterstages of the curriculum. This should not prevent solic-itation and utilization of student feedback. However,student input should be measured in the context of otherforms of faculty assessment. Student evaluation of fac-ulty members should include the faculty member’spreparation, appropriateness, and confidence as wellas usefulness, clarity, and effectiveness of objectives,RAT, and AEs.
Self-assessment paired with peer assessments maybe the most appropriate way to measure a faculty mem-ber’s performance in TBL. These should focus on majorTBL components, such as RAT items (linked to lower-order Bloom’s Taxonomy levels) and examination items(linked to higher-order Bloom’s Taxonomy levels), bothof which should be paired to module objectives. Peerreview of the AE should focus on accuracy and realismin addition to effectiveness of an embedded trigger todrive students to consider specific concepts. Having bothcontent experts and those with less topic familiarity as-sess all components of the module provides a more fullevaluation. Combining student, peer, and self-evaluationsallows for a critical assessment of the relevance and clar-ity from the point of view of the expert as well as thestudent.
Student Peer AssessmentTeam-based learning involves extensive group work
for completion of the tRAT and AEs. Thus, this active-learning method relies heavily on student interactions inteam settings. Unfortunately, “social loafing” may de-velop when individuals are graded equally based on thegroup’s completed work. This phenomenon of unequalparticipation by individual groupmembers inhibits effec-tive learning in the team environment.30,31 As describedby Michaelsen, one of the 4 essential principles of TBLis that students are accountable for personal preparationand contributing to team work.13
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
In traditional classes, students who attend class unpre-pared put their own learning in jeopardy. In TBL, they alsofail to positively contribute to their team’s performance, po-tentially affecting group dynamics and other students’ suc-cess.Theneed for accountabilityon thepart of each student isa strength of TBL as a student-centered pedagogy. The chal-lenge created for faculty members is to address the potentialimpact of social loafing on individual and team learning.
Team-based learning requires a system of evaluatingstudent performance to assess the dynamics of individualinteractions within teams. Teammembers working togetherduring a course are in the position to provide feedbackon thecontributions and performance of their peers. While a num-ber of systems have been used for student peer evalua-tions,9,31 scores should comprise a significant portion of anindividual student’s grade.9 The peer-evaluation score canbe a stand-alone component of the final course grade orused as a multiplier to “weight” team grades for eachstudent.31 For example, a student receiving a peer-evaluationof 80% would receive 80% of the total points awardedfor the team’s graded work.
Providing feedback to peers can be a difficult task forstudents. To improve the success of the process, one canfirst educate students about the purpose and significance ofpeer evaluations. Health care professionals increasinglywork in teams in various practice settings, so it is appro-priate that pharmacy students learn in team environments.The feedback received from students regarding their teaminteractions can be valuable in their growth as profes-sionals. The more the peer-evaluation system is accepted,the more effectively students can incorporate the feedbackthey receive into changing their behaviors. Use of peer-evaluation tools, in which students remain anonymousand/or faculty advisors summarize feedback prior to com-municating results to students,may improve both the quan-tity and quality of feedback.32 Adequate training must beprovided to students so they can understand the mechanicsof the peer-evaluation tool to ensure accurate, effective,and efficient feedback.
When constructing the peer-evaluation instrument,it is important to allow students to provide quantitativeand qualitative feedback.33 Quantitative portions of peer
Figure 3. Team-based learning continued quality improvement and assessment.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
evaluations may be problematic because they allow stu-dents to assign similar grades to all of their teammates. Forexample, students may be asked to rate fellow students ona scale from1 to 10on a variety of characteristics valued bythe program (eg, preparation, responsibility, promptness,interpersonal skills, respect for others). Students may bereluctant to assign low scores or provide negative com-ments to peers, despite the confidentiality of the process.Some may have concerns for the potential of breech ofconfidentiality and subsequent retaliation from their stu-dent colleagues if authorship of negative, despite construc-tive, feedback is discovered. These concerns can lead toprovision of only positive comments and allocation of highscores. To preserve the integrity and effectiveness of thepeer-evaluation process in these situations, the quantitativesection of the instrument can include a forced ranking sys-tem. Total points can be determined for each item (eg, 42points for a 5-member team), with students being asked todivide the points among all team members.
Alternatively, students may be asked to assign top-to-bottom scores for each item so students rank their 4 teammembers byassigningone student 6points, one4points, one2points, andone0points. Students canprovide comments tojustify the rankings to develop a deeper understanding ofhow peers might improve their contributions to the team.Requiring quantitative ranking for peer evaluations withjustification challenges students to approach the process us-ing greater critical-thinking and assessment techniques andmayhelppreventgrade inflation.Regardlessof theapproachtaken to assess individual performance in the team setting,peer evaluation is a critical component of TBL because itincreases student accountability, supports higher-level andmore effective educational activities, and ultimately leads toimproved learning (Figure 3).
Improving Student Perceptions of TBLA school transitioning to TBL from other teaching
formats must be prepared for a significant culture shift.9
Although pharmacy education is adopting more active-teaching modalities, many students expect to attend classand listen passively to the instructor. Faculty membersshould anticipate resistance from students, given the ac-tive engagement and accountability required in TBL. Toease this transition, it may help to prepare students andfaculty members by facilitating discussion through stu-dent class representatives, liaisons to curriculum commit-tees, and focus groups.
Clarifying the expectations and rationale for TBL isessential in allaying students’ fears of this learning ap-proach. Course coordinators, deans, and department chairsshould discuss the new teaching format with student advi-sory groups prior to implementation. These informational
meetings ideally should be accomplished prior to initiatinga TBL course. Long-term benefits gained by using TBL ina professional curriculum can be explained and examplesof successful TBL programs in other health-related fieldscan be shared. To improve buy-in, one can consider in-corporating practice opportunities for students to engagein TBL efforts using nonpharmacy content (eg, RATsbased on the school’s student handbook). Low-stakes prac-tice activities can generate enthusiasm for and understand-ing of TBL as an effective learning technique.
Expectations regarding assessments of individualknowledge (eg, RATs, exams, peer assessments) shouldbe explained during the initial information session. In-volving students in deciding the relative weighting foreach grade component is “. . . the most effective way toalleviate student concerns about grades.”9 On the first dayof class, instructors should present all components of thegrading scheme and provide a range limit on each com-ponent. After some discussion, student teams or teamrepresentatives should reach a “. . . mutually acceptableset of weights for each of the grade components.”9 Thesestandards then are applied for all teams within a givencourse for the entire semester.
A period of adaptation and learning is expected forstudents and faculty members, which should be commu-nicated to students. Regular feedback structures to en-hance course development should be established. Thiscan be achieved through various methods, such as usingstudent focus groups or liaising with student advisorygroups. Midpoint student evaluations of the course and/or of instructors can be incorporated to assure studentstheir concerns are noted. Regardless of the methods, allfeedback approaches should facilitate dialogue betweenstudents and faculty members. Establishing structures forfeedback ahead of the course may help avoid emotionalreactions from students.
Assessment of the Process, Course, and ProgramThe impact of TBL should be assessed at multiple
levels and include analysis of student knowledge andproblem-solving skills. Schools implementing a newTBL component may compare and contrast student per-formance with the prior curricular model. Basic knowl-edge acquisition and higher order pharmacotherapy skillscan be compared through examination-item analyses ofsimilar content and question types. Further, by mappinga curriculum or component back to curricular objectives,administrators are able to determine the success or failureof the new program to meet these objectives.
Schools that use a standardized approach to studentevaluations (eg, tools for student ratings available throughthe IDEA center34 can compare changes in student
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
responses to achieving relevant objectives, assessing theperformance of the facultymembers, and assessing over-all course quality over time. Qualitative feedback fromclinical preceptors can be particularly useful. Preceptorsoften are better able to assess a student’s knowledgedeficits and problem-solving skills based on their levelof interactions. They also may offer valuable insight onthe general skills of a group of students participating inthe TBL program comparedwith those educatedwith theprevious curriculum. This anecdotal evidence can bequantified through a structured survey or end-of-pro-gram evaluation. Finally, though not always the ultimategoal, student success on standardized licensure exami-nations [eg, the North American Pharmacist LicensureExamination (NAPLEX)] can provide data regardingknowledge acquisition.
The transferability of TBL to other programs re-quires an assessment of the program’s resources and,depending on the level of integration, the acceptabilityof this approach for facultymembers and students, as wellas any staff engagement necessary to support TBL activ-ities. Using Bolman and Deal’s 4 leadership frames(structural, political, human resources, and symbolicframes), institutions can systematically approach an as-sessment of the feasibility of TBL.35
From a structural perspective, the current curricularmodel may either help or hinder application of TBL.Moreover, it might be more easily integrated into cur-ricula that offer extended hours of teaching in one course(usually 3 or more hours per teaching session), to moreeasily incorporate all TBL activities in one session andhave adequate time for meaningful discussion betweenstudents and faculty members. From a political perspec-tive, it might be useful for administrators seeking toconvert their curriculum to TBL to engage faculty mem-bers and identify champions of the idea and where re-sistance to TBL exists through open conversations aboutthe pros and cons of TBL.
Discussions about whether to start TBL in specificcourses or have a complete curricular revision using TBLapproaches should be done strategically and should beinclusive and open to faculty concerns. Strategic and in-tentional faculty development and student support ses-sions would also be needed to help faculty membersand students transition through this curricular change.36
Onemethod could be a presentation about TBL’s benefitsand challenges, as well as a defined statement from theprogram about how TBL will shape the roles and identi-ties of faculty members and students within the programso faculty can readily be seen as facilitators of learning,and students are prepared to engage collaborativelyin their own development. Consistent use of readiness
assessments can demonstrate the value the program pla-ces on proactive student-centered learning. The omissionof lecture-heavy sessions in TBL also communicates howmuch the program values meaningful dialogue betweenstudents and faculty members. Program administratorsand faculty members wishing to adopt this must considertheir available resources, the philosophy and feelings ofother faculty members and students, and the implicationsTBL might have on expectations of students and facultymembers.
CONCLUSIONTeam-based learning is an effective teaching strategy
for faculty members wishing to increase student engage-ment, active learning, and discussion in the classroom. Ithas been evaluated in various curricular models in phar-macy education and other disciplines. Through RAP, stu-dents better understand instructors’ expectations, arrive toclass prepared for discussion, and become active partici-pants in their learning within the framework of a team.Using TBL allows students to apply foundational knowl-edge to practice-based scenarios per the 2016 ACPE Ac-creditation Standards2 and directly integrates CAPEEducational Outcomes into the curriculum, as well.6 In-vestment is required in the areas of faculty developmentefforts about how to effectively use TBL, manage studentexpectations with a new teaching strategy, and providemethods for facilitating team dynamics to ensure properimplementation. Team-based learning activities should beregularly and effectively assessed to document progressionand verify that desired end points are achieved. To mean-ingfully measure effectiveness of any curricular change,faculty members, student, and course assessment metricsmust be considered in shaping future iterations of TBL.
ACKNOWLEDGMENTSThe authors wish to thank the following colleagues
for their collaborations: Linda Farho, Brenda Gleason,Micheline A. Goldwire, Sarah E. Grady, June F. Johnson,Katie McClendon, Marianne McCollum, and SuzanneRabi.
REFERENCES1. Michaelsen LK, Knight AB, Fink LD. Team-Based Learning: ATransformative Use of Small Groups in College Teaching. Sterling,VA: Stylus Publishing; 2004.2. Accreditation Standards and Key Elements for the ProfessionalProgram in Pharmacy Leading to the Doctor of Pharmacy Degree.Standards 2016. Released February 2, 2015. https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf. Accessed March 17, 20153. Page RL, Hume AL, Trujillo JM, et al. Interprofessionaleducation: principles and application. A framework for clinicalpharmacy. Pharmacotherapy. 2009;29(3):145e-164e.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
4. World Health Organization. Framework for action oninterprofessional education and collaborative practice. Geneva,Switzerland. Department of Human Resources for Health. 2010.http://www.who.int/hrh/resources/framework_action/en/5. Considine J, Currey J, Payne R, Williamson S. Participantevaluation of team-based learning using one-off teams in a hospitalsetting. Australas Emerg Nurs J. 2014. 17(2):68-76.6. Center for Advancement of Pharmacy Education. Educationaloutcomes 2013. http://www.aacp.org/documents/CAPEoutcomes071213.pdf. Accessed March 19, 2015.7. Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. Theimpact of team-based learning on medical students’ academicperformance. Acad Med. 2010;85(11):1739-1745.8. Parmelee D, Michaelsen LK. Team-based learning: it’s here and itworks! Acad Med. 2010;85(11):1658-1659.9. Michealsen LK, Parmalee DX, McMahon KK, Levine RE, eds.Team-based learning for health professions education: A guide tousing small groups for improving learning. Sterling, VA: StylusPublishing; 2008.10. Michaelsen LK, Sweet M, Parmelee DX. Team-based learning:Small group learning’s next big step. San Francisco, CA: Jossey-Bass,2008 No. 116. Print.11. Grady SE. Team-based learning in pharmacotherapeutics. AmJ Pharm Educ. 2011;75(7):Article 136.12. Letassy NA, Fugate SE, Medina MS, Stroup JS, Britton ML.Using team-based learning in an endocrine module taught across twocampuses. Am J Pharm Educ. 2008;72(5):Article 103.13. Michaelsen LK, Sweet M. Team-based learning. New Directionsfor Teaching and Learning. 2011;128:41-51. doi: 10.1002/tl.46714. Conway SE, Johnson JL, Ripley TL. Integration of team-basedlearning strategies into a cardiovascular module. Am J Pharm Educ.2010;74(2):Article 35.15. Robinson DH, Schraw G. Recent Innovations in EducationalTechnology that Facilitate Student Learning. Greenwich, CT:Information Age Publishing; 2008.16. Robinson D, Walker J. Technological alternatives to paper-basedcomponents of team-based learning. New Directions for Teachingand Learning. 2008;116:79-85.17. Sibley J, Parmelee DX. Knowledge is no longer enough:enhancing professional education with team-based learning. NewDirections for Teaching and Learning 2008;116:41-53.18. Wilkerson, L. Tutors and small groups in problem-basedlearning: lessons from the literature. New Directions for Teachingand Learning. 1996;68:23-32.19. Raelin JA. The role of facilitation in praxis. OrganizationalDynamics. 2008;35(1):83-95.
20. Lane DR. Teaching skills for facilitating team-based learning.New Directions for Teaching and Learning. 2008;116:55-68.21. Jaques D. ABC of learning and teaching in medicine: teachingsmall groups. BMJ. 2003;326:492-494.22. Michaelsen L, Sweet M. Team-based learning. Thriving inAcademe. 2008;25(6):5-8.23. Michaelsen LK, Sweet M. The essential elements of team-basedlearning. New Directions for Teaching and Learning 2008;116:7-27.24. Bailey ML, Luetkehans L. Ten great tips for facilitating virtuallearning teams. In: Distance Learning ’98. Proceedings of the AnnualConference on Distance Teaching and Learning (14th, Madison, WI,August 5-7, 1998).25. Druskat VW, Wollf SB. Building the emotional intelligence ofgroups. Harv Bus Rev. 2001;79(3):80-90, 164.26. Centre for Teaching Excellence. Facilitating effectivediscussions. https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/alternatives-lecturing/discussions/facilitating-effective-discussions. Accessed: January 31, 2012.27. Results Through Training. Tips for team facilitators. http://www.rttworks.com/%7Erttworks/www.rttworks.com/images/downloads/TipsFacil.HTML. Accessed January 31, 2012.28. Weiss J, Hughes J. Want collaboration? Accept – and activelymanage – conflict. Harv Bus Rev. 2005;83(3):92-101, 149.29. Pelley JW, McMahon KK. Chapter 8: Facilitator Skills. In:Michaelsen LK, Parmelee DX, McMahon KK, Levine RE, eds.Team-Based Learning for Health Professions Education. Sterling,VA: Stylus Publishing; 2008:99-102.30. Karau SJ, Williams KD. Social loafing: A meta-analytic reviewand theoretical integration. J Person and Social Psych. 1993;65(4):681-706.31. Michaelsen LK, Knight AB, Fink LD. Team-Based Learning:A Transformative Use of Small Groups. Westport, CT: PraegerPublishers; 2002.32. Basheti IA, Ryan G, Woulfe J, Bartimote-Aufflick K.Anonymous peer assessment of medication management reviews. AmJ Pharm Educ. 2010;(74)5:Article 7733. O’Brien CE, Franks AM, Stowe CD. Multiple rubric-basedassessments of student case presentations. Am J Pharm Educ.2008;72(3):Article 58.34. The IDEA Center. http://www.theideacenter.org/. AccessedFebruary 8, 2012.35. Bolman LG, Deal TE. Reframing Organizations: Artistry,Choice and Leadership. 3rd ed. Hoboken, NJ: John Wiley & Sons;2007.36. Kotter JP. Leading change: why transformation efforts fail.Harvard Bus Rev. Harvard Business School;1995.
American Journal of Pharmaceutical Education 2015; 79 (10) Article 149.
