Video Games Take Testingto the Next LeveiResearcliers see promise in game-likeassessments that measure compiex skills.

BY ROBERT ROTHMANFrom Harvard Education Letter

Young people playing Halo orWorld oí Warcraft might not real-ize it, but they are working on theprototypes for a future generationof student tests.

"A video game is nothing but aseries of tests," says James PaulGee, the Mary Lou Fulton Professorof Literacy Studies at Arizona StateUniversity and the author of WhatVideo Games Have to Teach UsAbout Learning and Literacy. Gameplayers, he notes, are continuallyusing their knowledge to solveproblems. They need to know,for example, how much energythey need to jump over chasms,which tool to use to open doors,and which weapons to use againstparticular foes. And, Gee says, "Atthe end, there's a 'supertest'; ifyou pass, you can take it to a newlevel."

Based on these principles. Geeand others are currently develop-ing new models of assessment thatimmerse students in virtual worldsto measure abilities that are diffi-cult, if not impossible, to captureon pencil-and-paper tests, suchas the ability to solve problemsand conduct scientific inquiry. Insome cases, these assessments arealso learning experiences, becausestudents receive instantaneousfeedback, as players do in actualvideo games.

In addition, developers say, theassessments document the choic-es students make with each clickof the game device, giving teach-ers and administrators a wealthof data on student abilities. Andunlike performance assessments,which are also designed to capturecomplex skills like problem solv-ing, they are more practical andless expensive to administer.

In many cases, these new as-

Robert Rothman is a senior fellow at the Alliance for Excellent Educationand a frequent contributor to the Harvard Education Letter. Condensed, withpermission, from the Harvard Education Letter, 26 (December 2010), 1-3. Formore information, please visit www.edletter.org.

www.eddigest.com

Video Games Take Testing to the Next Level

sessments complement existingstate tests for accountability. How-ever, game-based assessmentscan also be used formatively tohelp classroom teachers monitorstudent progress and adjust theirinstruction according to studentneeds.

Developers find game-basedassessments appealing becausethey can provide information onstudent competencies in problemsolving and effective communica-tion—skills developers believe arebecoming increasingly important.Conventional tests might tell whata student already knows but telllittle about what he or she can dowith that knowledge.

Yet it is precisely this advantagethat makes the new assessments adifficult sell. Even though the U.S.Department of Education is provid-ing millions of dollars to developcomputer-based assessments—wbich may or may tiot incorporategaming aspects—many parentscontinue to equate learning withwhat's measured on standardizedtests, says Daniel Schwartz, aprofessor of education at StanfordUniversity. To build acceptanceof game-based assessments, re-searchers and policy makers mustmake the case for assessing abroader set of student abilities.

"We need to come up with newmodels so that people have abetter discourse," says Schwartz."Learning doesn't mean multiplechoice."

Measuring Complex AbilitiesThe increasing popularity of

video and computer games maycause concern among parents, whofear their children are spendingtoo much time on them. However,educators and researchers increas-ingly recognize these games' po-tential as motivating and engaginglearning tools. In a 2005 HarvardEducation Letter article. Gee notedthat games "teach children notonly how to play but how to learn,and to keep on learning."

With that idea in mind, research-ers have been designing learningenvironments using video games.But tbe developments in assess-ment are potentially groundbreak-ing as well. Game-based assess-ments make it possible to measurecomplex abilities because they al-low assessors to observe students'activities in ways not possiblewith even the most sophisticatedpaper-and-pencil tests, says ChrisDede, the Timothy E. Wirtb Profes-sor in Learning Technologies atthe Harvard University GraduateSchool of Education.

"Virtual worlds present com-plicated situations," he says, "andstudents are asked to respond tothose complicated situations. Andon the back end, you get rich infor-mation about student responses."

For example, in one assessmentDede and his colleagues devel-oped, science students investigatethe depletion of a kelp forest in thefictional Kamtigua Bay in Alaska.

March 2011

THE EDUCATION DIGEST

Through the use of ati avatar,students take on the persona of ascientist; they walk around the en-vironment and make observatiotis,interact with other characters, anduse tools like readings for watertemperature and salinity to takemeasurements. They can chooseto change the environment—for example, shut down a powerplant—and measure the effects ofthese changes on kelp growth. Andstudetits not only record their deci-sions, they also indicate why theychose to do what they did.

With these observations, theassessment is able to measurehow well the students formedhypotheses, the quality of their ob-servations, and the validity of theinferences they made from theseobservations—and provide scoreson all of these skills, Dede notes. Allof these skills are scientific inquiryabilities that organizations such asthe National Research Council havesaid are critical, yet are difficult orimpossible to capture on conven-tional student tests, he says, andthus game-based assessmentscan complement existing tests,which measure student knowledgeand basic skills effectively. Geeagrees that the information suchassessments provide offers a morenuanced and complete picture ofstudent abilities than conventionaltests do. "It's difficult to know whatto do with a single score that tellswhat a student did on Tuesdayat 4:00 [when the Student took a

test]," he says. "Games give scoreson a number of variables. They'renot scoring whether a studentsucceeded or failed. They showhow the student performed, andin some cases how innovative asolution is."

Learning, Assessment,or Both?

Schwartz also believes thatgame-based assessments can mea-sure learning itself. Just as gamesprovide feedback to players toshow them how they can. get outof a situation in which they arestuck, an assessment can enablestudents to learn while taking itand measure whether they actu-ally do learn.

To illustrate this idea, Schwartzdescribes a game-based assess-ment he and colleagues developedthat asks students to move frombooth to booth in a carnival. Atthe "boss booths," the bosses poseproblems to students, such as ask-ing them to design a sealed tankin which both plants and animalscan survive, that students mustsolve in order to move to the nextbooth. If the students fail to solvethe problem correctly, the assess-ment will indicate why the solutionwas ineffective; the students canthen use resources provided by thegame/assessment or even othercharacters in the game to learnabout a better solution. In thatway, according to Schwartz, tbeassessment can measure whether

www.eddigest.com

Video Games Take Testing to the Next Level

they were effective in seeking ad-ditional information,

"A more dynamic image of alearner is an important thing," hesays. "What we think learning is,is driven by standardized tests:learning is a state. These things aremoving. It's better to measure theslope than the position,"

Gee agrees that assessmentsthat are learning experiences arealso valuable teaching tools: "An as-sessment is not useful to a learnerif it's not teaching him something,"

But Dede, who has helped de-sign game-based assessments tomeasure state and national stan-dards, disagrees with the idea ofdeveloping assessments that mix

helping students learn with mea-suring what they know, Dede main-tains that fostering students' learn-ing as part of an assessment con-founds the measurement of whatthey already know and can do. Hebelieves learning experiences thatprovide students feedback as theirskills and knowledge grow are im-portant and useful, but that formaldiagnostic and summative assess-ments should focus on measure-ment, so that they are accurate.

Meeting ChallengesOne challenge in designing game-

based assessments is coming upwith situations that are engagingbut not too familiar to students.

MASTER OF EDUCATIONIN INSTRUCTIONAL TECHNOLOGY

STRENGTHEN YOUR TEACHINGAND THEIR LEARNING WITH TECHNOLOGY.Prepare to take a leadership role in administrationand multimedia development. Enroll in the Master ofEducation |MEdl in instructional technology programfrom University of Maryland University College (UMUCl.

• Classes available fully online

• No GRE or GMAT required

» Financial aid and an interest-freemonthly payment plan available

UMUC Enrott now. S00-888-UMUCucnuc.edu/edUCate

March 2011

THE EDUCATION DIGEST

While they want to create situa-tions that are realistic and fun sothat students will want to engagewith them, developers must ensurethat some students do not havean advantage because they knowmore than others about the situa-tion. "Every student knows a littleabout the ocean and platits, butfew students know about kelp for-ests," Dede says. "It's important topick a subject matter that providesa level playing field."

Another challenge is managingthe voluminous data that the as-sessments yield. Students makemany decisions during the courseof these assessments, and each de-cision is registered on the comput-er, Schwartz notes. "The density ofinformation is a big deal," he says.To help deal with that problem,Schwartz is exploring whether itis possible to determine students'problem-solving abilities from afew key decision points: 'What pat-terns sort out the kids who did wellfrom the kids who didn't?"

Another challenge is ensuringassessments are technically soundand that they provide accuratemeasures of what they are in-tended to measure. "A challenge isbuilding a virtual environment soevery action has a clearly definedpurpose and every student's actioncan be interpreted as exhibitingknowledge, skills, and attitudes,"says Dede. 'Which task [indicates]the ability to hypothesize, or makeinferences?"

This challenge is particularlyacute because of psychometricconcerns over reliability—a mea-sure of whether students wouldget a similar score if they tookthe same test at a different time.These concerns tend to favormore traditional measures likemultiple-choice tests, which aremore likely to be administeredin exactly the same way to everystudent, rather than a complexperformance that might yield adifferent response from differentstudents, depending on the deci-sions made along the way. As aresult, measures that truly get atcomplex abilities have been hardto build, suggests Schwartz, addingthat the emphasis on reliability"may cut down on our freedom toinnovate."

Despite these challenges, re-searchers say game-based assess-ments are feasible and predictthey will become more prevalentin schools in the next few years.Parents of students who do wellon conventional tests are look-ing for more information on howtheir children are learning, saysSchwartz. "I hear from people inSilicon Valley. 'Do you have teststhat measure other things we careabout, like are students preparedto learn? Do they have deep under-standing?' There is a market forthis kind of assessment."

"This is undoubtedly wherewe'll go in the future," predictsGee. •

www.eddigest.com

Copyright of Education Digest is the property of Prakken Publications and its content may not be copied or

emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.

However, users may print, download, or email articles for individual use.