8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 1/22
Software for Managing Complex Learning: Examples from an Educational Psychology CourseAuthor(s): Daniel L. Schwartz, Sean Brophy, Xiaodong Lin, John D. BransfordSource: Educational Technology Research and Development, Vol. 47, No. 2 (1999), pp. 39-59Published by: SpringerStable URL: http://www.jstor.org/stable/30221079
Accessed: 12/11/2010 11:22
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=springer.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].
Springer is collaborating with JSTOR to digitize, preserve and extend access to Educational Technology
Research and Development.
http://www.jstor.org
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 2/22
S o f t w a r e f o r Manag ing
C o m p l e x L e a r n i n g : Exampl e s f r o m a n
Educational PsychologyC o u r s e
DaonelL Schwartz
Sear Bropny
Xicodong Lin
John D Bransford
Inquiry-basednstructionincluding roblem-,
project-,ndcase-based ethodsoften ncorpo-ratecomplexetsoflearning ctivities.The
numerousctivitites un theriskof becomingdisconnectedn themindsoflearnersnd
teachers.STAR.Legacysasoftwarehell hat
canhelpdesigners rganizeearning ctivities
intoaninquiry ycle hat seasyto under-
standandpedagogicallyound.Toensure hat
classroomeachersanadapt
heinquiry
ctivi-
tiesaccordingo their ocalresourcesnd
needs,STAR.Legacyasbuiltupon our types
ofdesignprinciples:earnerentered,nowl-
edgecentered,ssessmententered,ndcom-
munitycentered.Wedescribeowa
STAR.Legacyonstructedoran educational
psychologyoursehelped reserviceeachers
designand earnabouteffectivenquiry-basedinstruction.
Z New developmentsin learningtheory sug-
gest that many teachers-the present authors
included--can improve student learning by
changing their teachingpractices(e.g., Cogni-tion and Technology Group at Vanderbilt
[CTGV], 996).Ascollegeteachers,we often find
that our predominantmethod of teachingis to
assignchapterreadingsandthen togive lectures
and demonstrations of points we think are
important(see also, Nunn, 1996). We assess
learningby askingstudents to answermultiple-choice questions, give presentations,or write
essays that paraphraseand elaborateon what
they have learned. These methods of teachingand assessment "work" n the sense that most
studentscandemonstrate hattheyhave learned
something. Nevertheless, the quality of their
learning is often less than satisfying. Reading
assignmentsandfollow-up ecturescanproduceevidenceof learning hat ookssuccessfulat first
glancebut missesmanyelementsof understand-
ing when analyzedin moredetail (Bransford&
Schwartz, npress;Schwartz&Bransford, 998).
Students, orexample,often failto use spontane-
ously what they have learnedin a new setting
despitethe fact thatit is highly relevant.White-
head(1929)referred o thefailure oapplylearn-
ing as the "inert knowledge" problem. A
number of studies show that traditional
approachesto instructionoften produce inert
knowledge (e.g., Bereiter& Scardamalia,1985;Bransford,Franks,Vye,&Sherwood,1989;Gick
&Holyoak,1983;Perfetto,Bransford,&Franks,
1983).
ETR&D,ol.47. No.2, 1999,pp. 39-59 ISSN 042-1629 39
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 3/22
40 ETR&D.ol.47 No. 2
Instructional nnovations such as problem-
based, case-based and project-based earninghave been designed to combatthe inertknowl-
edge problem (for precise distinctionsamong
these approachessee Barronet al., 1998;Wil-liams, 1992). Instead of simply assigning fact-
based readingsor providing lectures,students
begin their inquiry with challengingproblems,and they learn information relevant to those
challenges as the need arises. Instructional
approaches that are organized around cases,
problems and projects have been used for a
number of years in professional schools for
trainingin medicine (e.g., Barrows,1985),busi-
ness (e.g., Gragg, 1940), law (e.g., Williams,
1992), and educational administration (e.g.,
Bridges&Hallinger,1995).Theseapproaches o
instructionare also being used with increasing
frequencyin K-12education(e.g.,Barron t al.,
1998;CTGV, 1992; 1997;Krajcik,Blumenfeld,
Marx,Bass,&Fredricks,1998;Penner,Lehrer,&
Schauble, 1998). Williams (1992) provides an
excellent review of problem-basedand case-
based learning. Data on the effectiveness of
these approachesfor student learningare dis-
cussed in Barronet al. (1998;see also, CTGV,
1997; Hmelo, 1998;Michael,Klee,Bransford,&
Warren,1993;Vyeet al., 1998).
There are risks associated with the use of
case-based, problem-based and project-based
learning.A majorrisk is thatengagementcan be
mistakenfor learning.Forexample,when com-
pleting a hands-on activitysuch as building a
modelrocket,
studentsmay
be active and
enthused, yet assessments of the systematic
understandingmay yield disappointingresults
(examplesare discussed in Barronet al., 1998).
Another risk comes from the assumption that
these are constructivist activities that requireteachers to eliminate traditionalactivities such
as assigning fact-basedreadings or providinglectures.Assumptionssuch as these fail to dif-
ferentiateconstructivismas a theoryof knowing
from theories of pedagogy. Constructivist heo-ries assume that people always use their prior
knowledge to constructnew knowledgeeven if
they are sitting through a lecture (e.g., Cobb,
1994).Lecturesare often not the bestway tohelpnovices learn because their knowledge is not
sufficiently differentiated to understanda lec-
ture at a deep level. However, if students are
given opportunitiesto develop well-differenti-
atedknowledge,lecturescanbe a powerfulwayto help students organizetheirknowledge and
experiences Schwartz&Bransford, 998).Thereare times for lectures and readings, but theyneed to occur when students are prepared to
appreciate he insightsthattheycontain.
Inour experiences,case-,problem-and proj-ect-based learning are most effective when
teachers,students and other interestedpartiesform learning ommunities,where there is indi-
vidual accountabilityyet people collaborate in
order to achieve important objectives, and
where there is access to expertisethatoften lies
outside the classroom community (e.g.,Bransfordet al., in press). Frequentopportuni-ties for formatively assessing individual and
group progress are also importantfor helpingstudentsachieve(e.g.,Barron t al., 1998;Vye et
al., 1998).
In this article we describea software shell,
STAR.Legacy,hat s designedtoguide attemptsto
helpstudents learn fromcase-,
problem-,and
project-based earning. STAR.Legacy upportstheintegrationof fourtypesof learningenviron-
ments that we believe are especially importantforenhancing earning(CTGV,npress):
1. Learner-centerednvironments hat focus on
knowledge,skillsand attitudes thatstudents
bringto thelearningsituation
2. Knowledge-centered environments that
focus on knowledgethat is organizedaround
coreconceptsorbig ideas thatsupportsubse-quent learning in the disciplines (e.g., see
Brown &Campione,1994;CTGV, n press)
3. Assessment-centered nvironments hathelpstudents' thinkingto becomevisible so that
both they and their teachers can assess and
revisetheirunderstanding
4. Community-centerednvironments hatcap-
italize on local settings to create a sense of
collaboration-both among students and
with othermembersof thecommunity
Integrating hese fourtypes of environments
requires what we call flexiblyadaptive nstruc-
tional design (Schwartz, Lin, Brophy, &
Bransford, n press). As teachers use problem-,
project-,or case-basedmaterials,they need to
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 4/22
SOFTWAREORCOMPLEX EARNING 41
tailorlearningactivities to the uniquequalitiesof their settings while maintaininga focus on
coredisciplinaryknowledge.Toadapta curricu-
lum (knowledge-centered) to their setting,
teachersneed to work with thepriorknowledge,skills, and cultural resources that a specific
group of students brings to a situation. To
accomplishthis, students need opportunities o
bring their knowledge and beliefs to bear on
school subjects(learner-centered),nd teachers
need frequent opportunitiesfor assessing stu-
dent progress toward knowledge standards
(assessment-centered).And, because commu-
nity is a powerful and variable property of
learning settings, teachers should be given
opportunities to maximize communityresourcesto motivate and enablecollaboration
and achievement community-centered).
STAR.Legacy promotes flexibly adaptiveinstructionaldesign in two primary ways. It
helps teachers adapt complex curricula by
includinga model of inquirythat draws atten-
tiontoeachof thelearningenvironmentswithin
a single software shell and that provides a
framework for making pedagogically sound
modifications. tsupports lexibility y includinga suite of softwaretools thataresimple to learn
and use,and thatmakeiteasy to modifya given
STAR.Legacy.
Figure 1 provides an overview of the
STAR.Legacynquiry model. This figure is a
screen shot of the software'sprimaryinterface.
Theinterfaceorganizesstudentactivity ntotyp-ical phases of inquiry and helps make their
learningprocessesvisibleto themselves and the
teacher.By clickingon thedifferent consof the
interface, the software branches to the corre-
sponding "pages"of resources and activities.
Thesepages provide opportunities orstudents
to complete progressively complex challenges(theChallengemountains n Figure1),to gener-ate their own ideas on how to meet the
Figure1 D STAR.LegacyHomePage"
Look Ahead"&Reflectack
Reflect Back
The Challenges
1ps
Help
Go Public
Test YourMettle
a?aorn;nZb
GenerateIdeas
MultiplePerspectives
Research& Revise
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 5/22
42 ETR&D,ol.47, No. 2
challenges (GenerateIdeas), to compare their
ideas to others and reflect on the differences
(Multiple Perspectives),as well as to develop,
assess,andrevise theirunderstanding Research
& Revise,TestYourMettle)beforetheypresenttheir final solutions to each challenge(GoPub-
lic). Toconvey these multipleopportunitiesfor
learning,assessing, and makingmodifications,
we use thenameSTAR,anacronym or software
technologyforactionand reflection.
In addition to the explicit inquiry cycle,which guides decisions about adaptation,
STAR.Legacyncludessimplesoftware tools for
modifyingthe resourcesavailablen
eachphaseof inquiry. Typically, teacherswork with and
modifya STAR.Legacyhat has been filledwith
activities and resources designed for inquiryinto a specifictopicsuch as ecosystemsor elec-
trical circuits (see Border Blues.Legacyin
Schwartz,Lin,et al., in press;and DC.Legacyn
Schwartz,Biswas,et al., in press). STAR.Legacymakes it easy for teachers o addnew resources,
including texts; patches to other programsor
websites; and video clips of themselves, theircolleagues and their previous students. For
example,video clips of colleaguescommentingon variouschallengesmightoccurin the Multi-
ple Perspectives part of the inquiry cycle. This
allows teachers to introducetheirstudents to a
larger learning community; for example, to
other professorson campus (or teachersin a
school) who have expertisethat is relevant to a
particular ourse.STAR.Legacylsoenablesstu-
dents tocontribute o thelearningenvironment.Afterdeveloping sufficientunderstanding,stu-dents canleavea legacyfornextyear'sstudents
by adding theirown insights or lessons (hence
the name Legacy).This can be very motivatingforstudents,and ithelpscurricula volve so that
theybestfitlocalneeds.
In the following sections we describe
STAR.Legacyn more detail. Wedescribe t in a
context where it was used it in an attempt to
improveour own practicesascollege professors.We developed a STAR.Legacy alled Learning
By Doing Legacy (LBD.Legacy).LBD.Legacywas designed to informeducationalpsychologystudents about problem- and project-based
learning.Our objectivewas to help preserviceteacherslearn to design and adapt this type of
instruction o the needs of theirstudents and the
resources of their community.One measure of
whether we achieved these objectives comes
from the legacies the students created to teach
next year's students. Later in the article, wedescribe how students'legaciesnaturally ncor-
porated many of the important pedagogical
principlesof STAR.Legacy s well as resources
from the local community. Unfortunately,becauseLBD.Legacywas new to us and the stu-
dents, we were not preparedto experimentally
comparestudent learningand behaviorin this
context with learningin more traditionalcon-
texts.Nevertheless,ourexperiencesand theleg-
acies the students created provided informaldata thatarequitevaluableandsuggest research
issues that are worthyof subsequentinvestiga-tion. In the remainderof thearticle,we describe
these observations in the process of detailingfouraspectsof STAR.Legacy:
1. Theexplicitnessof theinquirymodel
2. Thecomponentsof a single learningcycle
3. Themultiplelearningcyclesthathelp people
progressivelydeepentheirunderstanding4. The importanceof reflectingon the overall
learning process and creating legacies for
otherpeople to use
We conclude with a discussion of the soft-
ware tools that makeSTAR.Legacylexible and
theprospectsfor furtherdevelopmentof flexibly
adaptiveinstructionaldesign.
MAKING HENQUIRYODEL XPLICIT
Every STAR.Legacy, including LBD.Legacy,uses the basic interfaceshown in Figure1. The
interfaceis meant to providean explicitmodel
forstructuringnquiry nproblem-,project-,and
case-basedlearning.Themodel includes a num-
ber of componentsthat we describe more fullybelow. Forexample,there is the Look-Ahead&
Reflect Backcomponentthatstudentscompleteat the start and finish of any STAR.Legacy.Thereare themultiple Challengemountains that
students are encouraged to "climb" one after
another to progressively deepen theirexpertise.
And, there is the basic learning cycle that stu-
dents complete as they engage in inquiry for
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 6/22
SOFTWAREORCOMPLEX EARNING 43
each one of the multiple challenges of a
STAR.Legacy.
Thesecomponentsofferone possibleformal-
ization of insights gatheredthroughcollabora-
tions with teachers, trainers, curriculum
designers,and researchers.Thespecificcompo-nents of STAR.Legacywere chosenbecausetheyhave repeatedly appeared as important, yetoften implicit, components of learning and
instruction---componentssuch as continually
makingformativeassessments of studentprog-ress, and continually lluminatingways forstu-
dents to situate and think about a topic.
AlthoughSTAR.Legacyries to formalizethese
components, t is notintendedtoreplaceon-the-
spot, expertdecisions.Rather,t is meanttoaug-ment local expertiseby helping teachers(and
students)develop an understandingof learningevents. Teachersandstudents,forexample,maychoose to jump around the learning cycle
dependingon their assessments of their current
learningneeds.Tohelp peopleunderstand hese
options, context-sensitive "Tips" provide the
rationale for a particularcomponent of the
inquirymodel,alongwithsuggestionsforusingthespecificactivities ncludedwith thatcompo-nent. Figure2, forexample,shows theexplana-
tion thatappearswhen an individual clicks onthe Tips apple on the Look-Ahead & Reflect
Back page of LBD.Legacy. Hopefully, such
explanations help people understand the pointof an instructional design feature, and this
understandingwill, in turn, help them adaptinstruction o theirown ends.
The explicit inquiry model of STAR.Legacyhas been in response to our observationthat
learnings enhancedwhenteachersandlearners
can"see wherethey are" n acomplexsequenceof inquiry. This became apparent during the
implementation of an integrated model of
instruction and assessment called SMART,an
acronymfor "Scientificand MathematicalAre-
nas for Refining Thinking" Barron t al., 1998;
CTGV,1997).Using the SMARTmodel, class-
rooms progress from problem-basedlearningthat develops a solid knowledge foundation to
Figure O ASampleTipor he LookAhead-Reflect ackPage
Look Ahead
&Reflec
Reflect i.
Begin Here
Look Ahead &Reflect Back
IRs
The Look Ahead component includes a task that helps students
develop a concrete model of the sorts of things they will belearning about. The Look Ahead serves a numberof purposes:
* Learners preview where they are going in their learning andthe sorts of things they will come to understand.
* Students and teachers can pre-assess the things theyalready know about the domain and what they need to learnmore about.
* It can be motivating for students as it develops both theircuriosity and their aspirations about what they will know.
In this Look Ahead, students are asked to write what theythink is aood and bad about an actual hands-on pDrolect that
ps
IP
ook
Conmas Hand*OnProjet
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 7/22
44 ETR&D,ol.47, No. 2
more open-ended project-based learning.Withinthe model, therearemanyopportunitiesforstudentsto generate heir own ideas,consult
knowledge resources, share thoughts, and
assess and revise theirunderstanding.Bymov-
ing through cycles of learningand revision in
the context of related problem and project
challenges,students progressivelydeepen their
understanding(Barronet al., 1998;Vye et al.,
1998).
We knew from prior implementationsof
SMARTthat students (and teachers)often felt
lost in the details of thespecificactivities Barron
etal.,
1995).They
did not know howparticularactivitiesfittogetheror how theactivitieswould
contribute o their overallunderstandingof the
problemor theirabilityto completetheproject.Students and teacherswere at risk of viewingthe curriculumas a "humongous compilation"of learningactivitiesthat were not particularlyconnectedor dedicated to any larger goal. To
avoid this, we started to post "SMARTMaps"thatstudentsandteacherscould consultto mon-
itorlearning (Vyeet al., 1998).Theseprecursorsof the STAR.Legacyvisualizationshowed how
different learning activities joined to make a
sequence that supported the ultimate goal of
completing the final project. The explicitSMARTMaps helped students. For example,because the activityallotteda time forrevision,
students learned that revision is a naturalcom-
ponentof learning,rather han a punishment or
not learning(Schwartz,Lin,et al., inpress).
One of the goals of presenting an explicitinquirymodel is to help teachers and students
transfer inquiry practices from one topic to
anotherbecausethey can see similaritiesacross
the activities.Forexample, duringthepast yearwe have had the opportunityto introduce the
STAR.Legacy rameworkto a numberof K-12
inservice teacherswho had been using various
non-Legacycurriculumunits,such as ourprob-lem-based JasperAdventure and Scientists in
Actionseries(fordescriptions, ee CTGV,1997).We showed themhow these units looked when
placed in the STAR.Legacyframework. The
response was extremelypositive. Teachersfelt
thattheycouldinstantly"see" heimplicit earn-
ing cyclescommon to much of their instruction.
Moreover,the teachers found it easier to talk
with teampartnerswho teachother disciplinesbecause they could see that their instructional
techniquesactuallyshareda common structure
of inquiry.
TRAVERSINGHE ASICSTAR.LEGACYYCLE
In this section we traverse the elements of the
STAR.Legacy ycle. We focus primarilyon the
LBD.Legacythat was designed to help pre-service teachers understand the strengths,weaknesses,and differenthow-tos of
problem-,project,and case-basedapproaches to instruc-
tion. The LBD.Legacybegins with a preview in
Look Ahead (&ReflectBack).Afterwards,stu-
dents meet theirfirstinquiry challengeby click-
ing on the smallest mountain. To completethis
challenge, they complete each of the compo-nentsof the learningcycle.Whendone with the
first challenge, students click on the second
mountain to meet their second inquiry
challenge. They again completethe
learningcycle,usinga new set of resources ailoredto the
second challenge. After completing the same
process for the third challenge, students con-
clude LBD.Legacy y revisitingthe LookAhead& Reflect Back. In the following sections, we
trace hissequenceanddescribethose classroom
observationsthat seem most promising for fur-
therresearch.
LookAhead
The Look Ahead & Reflect Backis represented
by thebinoculars n the upper-leftcorner of the
maininterface(Figure1).Studentscomplete the
Look Ahead before they meet any of the
challenges.Clickingon thebinocular conbringsstudents to a screen that is designed to helpthem preview the knowledge domain and
develop learninggoals.Nearlyall models of learningand instruction
emphasize the importanceof goal setting (e.g.,Newell &Simon,1972).Often,goals forlearningarepresentedasspecificobjectives,primarily or
the teacher'seyes. For educationalpsychologythesemightinclude,"studentswill list three rea-
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 8/22
SOFTWAREORCOMPLEX EARNING 45
sons that problem-based learning improves
transfer,""studentswill compareand contrast
summative and formativeassessment,"and so
forth. Many attempts to list specific objectives
fallshortof the ideal becausetheyarefrequently
perceived as a discrete list of statements that
often seem vagueand unrelated.Ourpreferenceis to help teachersand studentsdevelop a more
coherentand concrete model of the topic that
theywill learnabout and the sortsof thingstheywill be able to accomplish by completingtheir
Legacy ourney.
The LookAhead& ReflectBack orLBD.Leg-
acy beginswith an audio narratorwho
explainsthat one of the benefits of expertise is the
increasedability to notice significantevents in
the environment.As an example, the students
are shown an eight-second,digitizedvideo clipof a single play in a footballgame. Afterwards,
they watch protocolsof two women who were
askedto recountwhat theyhad seen in theplay.The firstwoman,anovice,states,"I'm o embar-
rassed.I think... Ithinkthequarterbackanthe
football." The second, a relativeexpert,states,"OK, he defensive backsjumpedoffsideduringa blitz. Thequarterback andedthe ball toone of
the running backs who broke two tacklesand
was on his way to a touchdown."Thestudents
then get a chance to watch the football play
again to assess their own expertise.After this
example,the narrator xplainsthatLBD.Legacywill increasetheir ability to notice and under-
stand importantaspectsof a type of instruction
calledlearningbydoing,
n which students com-
plete problems, projects,and hands-on activi-
ties.To get a sense of theirown expertise n this
domain, the narratorsuggests that the students
evaluate theprovided exampleof a prototypicalhands-on projectthat involves makinga com-
pass frommagnets.The studentsdo not spendtheir time studying this particular project in
LBD.Legacy.Thecompass projectsimplyoffers
a brief and relatively familiarexample of an
activitythatfalls
under the rubricof
learningbydoing. The students' task is to notice what is
good about thecompass projectand what could
be improvedand how.
Currently,we see five potential benefits to
the Look Ahead. (a) It helps students to situate
theupcominglessonsandlearninggoals in their
priorknowledge, interests,and communitiesof
practice. b) Itprovideslearnersan opportunityto see their destinationin a domain of knowl-
edgeandto see the sortsof thingstheywill come
to understand. (c) It helps develop a shareddomain model that facilitates discourse and
communitywithin the classroom.(d) Itoffersa
learnerandteacherpre-assessment.As learners,
students can identify what they need to learn
moreabout.Forteachers, hisactivitycanoccur
ataclassroom evel so thatteachers anappraisetheirclass'sinitialdomainknowledge.This can
help teachersanticipate earningneeds, as well
as help them design and tailor their use of
STAR.Legacyand other classroom resources.Forexample,with the LBD.LegacyLookAhead,we were able to see that few of the education
students saw anything wrong with an activitythatprovidedno contextforlearningaboutcom-
passes, and no information hatexplainedwhy
compasses work, or why they are useful. This
indicated mportantgapsin thestudents'under-
standingof how to transformlearningby doinginto doingwith understanding.e) Finally, the
LookAhead can serve as a benchmarkorreflec-tion and self-assessment. In particular,after
completingthe learning cycles, students return
to ReflectBackon what theyhave learned com-
paredto their firsttryat the LookAhead. As we
documentlater,this can help students see how
muchtheyhavelearned.
TheChallenges
Aftercompletingthe LookAhead, students are
prepared to work on the STAR.Legacy
challenges. To access the first challenge, stu-
dents click on the smallest mountain. In
LBD.Legacy, hallenge1asksstudents togener-ate and explain reasonswhy they might want
people to learn by completing projects. To
anchortheirinquiry nto thischallenge,the stu-
dents watch a clip of a movie that shows a high-schoolhistoryteacherwho is givingastultifying
lectureon the chronologyof the Depressionera
(fora discussion of anchors, ee Bransford, her-
wood, Hasselbring,Kinzer,& Williams, 1990;
CTGV, 1994). Afterwards, a "narrator"in
LBD.Legacy uggests that the students use this
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 9/22
46 ETR&D, ol. 47, No. 2
example of instruction as a foil to help them
thinkaboutwhy projects an be a useful alterna-
tive. The remainderof the learningcycle pre-
pares the students to complete this challenge.
Aftercompletingthelearningcycleandpresent-ing their final answer to Challenge 1, the stu-
dentsrepeattheprocesswithChallenges2and 3
(describedbelow).
GenerateIdeas
After students understand a challenge, they
progresstoGenerate deas wherethey explicitlymaketheirfirst
attemptat
generatingssuesand
answersrelevant o thechallenge.Typically, he
Generate deaspageincludesa text that reminds
students of the challenge (e.g., justify project-based learning)and that instructs students to
record their initial ideas. Figure 3 shows the
ideas that one groupof education studentsgen-eratedforChallenge1 afterwatchingthe video
clip of the excessively passive classroom.The
students' thoughts were entered into an elec-
tronic notebook where they could be subse-
quentlyrevisedandimprovedupon.
In acomputer-rich lassroom,one way touse
thenotebookfeature s tohave eachstudentgen-erate ideas into a separatenotebook. Alterna-
tively, for the LBD.Legacy, he challenge was
watched as a whole class, and students gener-atedtheir nitial deasonpaper.Afterwards, tu-
dents offered their ideas in the whole-class
context, and the teachercombined those ideas
into the collectivenotebookshown in the figure.The notebook served as a focalpoint for further
discussionof thegenerated deas.
Thereareseveral reasons thatGenerateIdeas
is an explicit component of the Legacy cycle.One is that tencouragesstudents to shareideas;
everyone has an opportunityto hearwhat oth-
ers think.For the teacher, his complementsthe
Look Ahead by providing a more specificassessment of what the students understand
about the topic. For example, Figure 3 reveals
that students primarilythink about a project'sbenefits n termsof motivation.For hestudents,
Figure 0 TheClass'sNotebookEntriesor heGenerateIdeaPhaseofChallenge1
GenerateIdeas
Projectsnsurekidsworkat own level. This sgoodbecausetheycanunderstand omparedto a lectureover theirhead.
Professorwon'thave to lecture. Kids'llpayattention.
Canaccomplishmore n 45minutesof projectthan listening o a lecture.
SWhyPeojct
Help
Notebook
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 10/22
SOFTWAREORCOMPLEX EARNING 47
it provides an opportunityto learnwhat other
students think.Toofrequently, tudentshaveno
idea about theirpeers' knowledge.As a conse-
quence,itbecomesdifficultfor studentstobuild
community by takingadvantageof knowledgedistributedthroughouta classroom.
A second reason to have students generateideas is to make thinking explicit rather than
allowing it to remaintacitand vague.Theactof
specifically recordingtheir views abouta topic
helps students discover what they think and
know. This awareness is facilitated when stu-
dents contrast their ideas with other ideas.
Appropriatecontrasts, much like setting two
wines side by side or seeing two ideas juxta-
posed, can help students notice importantdis-
tinctions they might otherwise overlook or
dismiss (Bransfordet al., 1989; Bransford&
Nitsch, 1978;Gibson & Gibson, 1957;Schwartz
& Bransford,1998).Forexample,most instruc-
torsin the behavioral cienceshaveexperiencedthe frustration f students'statingthataparticu-lar point is just "commonsense,"when in fact,
the students would never haveappliedthe com-
mon sense on their own. When students areasked first to generate deas, theybecomemore
appreciativeof the contrastbetweentheir nitial
observationsandensuingobservations hattheyhadoriginallyoverlooked.Thisappreciation an
help themgraspwhat is new about ideasrather
thanmerely assimilate them into old ideas and
gloss overwhat is new and important.
MultiplePerspectives
Following the initial generationof ideas comes
Multiple Perspectives.There are many situa-
tions where multipleperspectivesarea natural
componentof learning(e.g.,a conferencepanel,
cooperativejigsaw groups,differentvoices in a
novel). In STAR.Legacy,his important eature
of learning is made explicit.ForChallenge1 of
LBD.Legacy, tudents hear the ideas generatedby four educationalexperts who also saw the
video of the Depression-era lecture. These
experts introducestudents to vocabularyand
perspectivesthat are quite different from their
own and thatcharacterize xpert approaches o
the topic. By focusing attention on contrasts
between what the studentsgeneratedand what
the experts generated, students are helped to
grasp the significanceof new informationand
understand ts relevanceforhelpingthemthink
differently Bransford&Schwartz, n press).
Recall, orexample,thatthestudentsprimar-
ily generated deasabout themotivationalvalue
of projects see Figure3).Contrast his with the
observationsgeneratedby the fourexpertswho
comprise the Multiple Perspectives for
Challenge1. Oneexpertpointsout that he origi-
nally thought motivation was the only signifi-cant issue,but then as he investigatedthe issue
of learninghe found out that the way students
are expected to learn in this formatoften has
profoundeffectson theirsubsequentabilitiesto
use that nformation.Anotherexpertnoticesthat
the students are all individually seated in the
video segmentand thatthere is none of the nat-
uralsocial interaction hatoften facilitates earn-
ing. A thirdexpertpointsout that it is importantto remember hatthere s a time forlecturesand
that they can be very effective-the trick is to
prepare people to be told. The fourth expert
explains that the tests that usually accompany
lecture-stylenstructionoftenfindno parallel n
the realworld.Consequently tudentsspend an
inordinateamountof time learninga skill (e.g.,
preparing for multiple-choice tests) that
becomes obsolete immediately upon leavingschool.
Aftertheeducationstudentslistenedto thesefour perspectives, the class was silent. The
instructorparenthetically
commented, "It is
amazing to find out what you never even
thoughtabout,"Thewhole classnodded vigor-
ously in agreement.A useful line of research
might formally document whether producingideasabouta topicbeforehearingdifferentper-
spectiveson thattopic increases earners' ntel-
lectualengagementwith the ideasat hand.
In addition to complementingthe Generate
Ideas phase, the Multiple Perspectives phase
serves a numberof purposes.First, he perspec-tives provide guidance into the topics that the
students should explore to learn about the
domain. The perspectives do not give awaysolutions; nsteadtheydirect the students to rel-
evant domains of inquiry.In the LBD.Legacy,these domains are cognition,social interaction,
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 11/22
48ETR&D.
ol. 47. No 2
and assessment.Second, the MultiplePerspec-tives feature ndicatesthatagivensituationusu-
ally has multiplevantagepointsand that this is
acceptable.This is differentfrommuchinstruc-
tion that provides only one model for how tothink about the material. Learning multiple
entrypointsintoa given topicincreases he flex-
ibilityof futureproblemsolving (Spiro&Jehng,1990).Third,there is greatadded value in hav-
ing expertscommenton thetypesof demonstra-
tion tapes that are currently used in manyclassrooms. Experts,for example, often notice
what is missing n the videotape and why that
absence s theoretically ignificant.
One of theexcitingpotentialsof MultiplePer-
spectives is thatit cancombineand makeavail-
able distributed expertise among people who
cannot easily be brought together.This has a
profound impact on the instructor'sability to
adapt and tailor instructionby capitalizingon
local resources.And, it servesas a way to create
an intellectualcommunity among people who
are too busy to meet formally.A good instance
of these benefitsmay be had by jumpingahead
for a moment to Challenge 2 of the second
inquirycycle.InChallenge2 ofLBD.Legacy,tu-dentsareasked todevelop a set of design princi-
ples for project-based earningand to organizethoseprinciples ntoa helpfulvisualization.
To help students make headway on this
challenge,we recruited ocal facultyand master
teachers for the Multiple Perspectives. Each
expert received a video cassette containinga
short segment that served as the anchor for
Challenge 2. The tape shows a local TV news
reportin which children launch model rockets
as partof a schoolproject.The teachersand fac-
ulty were asked to watch the videotapeat their
leisure.A weekor so later, heywerevideotapedas they gave short commentarieson what theyhad noticed relevant to design principles for
learning. Complementing their brief commen-
taries,theywere asked to suggesta readingthat
would help students who wantedto learnmore.
Students also watched the videotapeas partof
theirdesign challenge,and they generatedtheir
ideas in the notebook prior to hearing the
experts'perspectives.
Recruiting ocalexpertsto makeup the Mul-
tiple Perspectiveson the news broadcasthelped
to build the local intellectualcommunity.When
students saw one professor,for example, they
stated,"Hey,we've readstuffby him.So, that's
who he is. I'veseen him around."Severalteams
ofstudentstookadvantageof learningaboutthescholarly resources in their neighborhood.As
we describe later, they decided to interview
these facultyas a componentof the legacy theyleft for next year's students. This approach to
MultiplePerspectivesalso helps to build com-
munity among the experts themselves. The
experts were curious to see what the other
experts, theircolleagues, had to say. Although
they knew one another through college func-
tions, they rarely had a chance to hear oneanother'sexpertise.Like hestudents,afterhear-
ing their colleagues' insights, they often com-
mented, "I hadn't even thought of that."One
worthwhile question for future research is
whetherthis asynchronousgatheringof peoplehas a meaningfulimpact on the local commu-
nity.A secondquestionis whetherteachersfind
it sufficientlycompellingthattheywill add their
own expertsto adapta Legacyto their local cir-
cumstances.
Research&Revise
InResearch&Revise,studentsmaytakepartin
many different activities including collabora-
tions, consulting resources,listening to just-in-
tinr. lectures,completingskill-building essons,
workingwith legaciesleftby students frompre-vious years, and conducting simulations and
hands-on experiments. This component of
STAR.Legacyis very inclusive and supportsmost instructional designs. For example, for
Challenge 1 of LBD.Legacy, he resources are
primarily extual.Thestudents readarticlessug-
gestedby experts,and theyuse theirtextbookas
a resource orcompletingsimple tasks thathelpthem understand the potentialbenefits of proj-
ect-based learning. For Challenge 2, however,the students analyze video cases, practiceinstructionaltechniqueswith one another,and
evaluate Websites that offer project-based es-
sons. In eithercase, the key criterionfor includ-
ing a resource s thatthe instructionalmaterials
should help students reach the goals of explor-
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 12/22
SOFTWAREORCOMPLEX EARNING 49
ing a challenge and of revising the ideas that
theyoriginallygenerated.
We have found it useful to organize the
resources according to the Multiple Perspec-
tives. Figure4, forexample,shows the resource
topics available for Challenge 2. Each of the
images is keyed to one of the six expertswho
commented on Challenge2. By clickingon an
image, studentsmove to a page with resources
specificallyassociatedwith the ideas generated
by the relevantexpert.This organizationhelpsstudentsand teachersfollow thoseperspectivesthatseem particularly uited to theirneeds;we
do not assume that students and teacherswill
necessarilyuse all the resources availablein a
STAR.Legacy.
TestYourMettle
When students thinkthatthey have developedtheir understandingof the original challenge,
they are asked to complete Test Your Mettle
beforetheycan Go Publicwith their solutionto
the challenge.The "test"can takea number of
differentforms, includingmultiple-choice ests
with feedback,rubrics for evaluating products
they plan to make public, and "neartransfer"
problems.Test YourMettle s meant as a forma-tive instructionalevent, not a final exam. It is a
chancefor students to bump againstthe world
to see if theirknowledgeis up to the task.If it is
not, they should return o Research&Revise to
improvetheirunderstanding.
For the first cycle of LBD.Legacy, he Test
YourMettle asked the students to evaluate the
reasons they had generatedfor using project-based
learning.Eachstudent
preparedfive rea-
sons. The students met in small groups and
selected their best reasons from among their
group. They were told that the reasons from
each group would be compiled and the class
would vote on the top ten reasons.These rea-
sons would then be posted on the WorldWide
Web through Go Public, and they would be
evaluatedby studentsatStanfordUniversity.As
the studentsmet in theirgroups, they began to
Figure4 F Research&RevisePage forChallenge 2
Research
MeetingStandards
Doingwith
UnderNanding
Social Nornms
Culture
CoguitionAssessment
PreparedneforColIaamating
Tips
Help
Notebook
Legacies
Task
Reminder
Home-SchoolConmnecions
Learningn Context
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 13/22
50 ETR&D, ol. 47, No. 2
question what made a good reason, and theywanted examples. Test Your Mettle providedthis information.It stated, for example, that a
good reason might explain the conditions
needed to achieve a benefit fromproject-basedlearning.Forexample, they received a sampleframethat stated, "Projects an provide excel-
lent opportunitiesfor assessing student under-
standingBECAUSE... But,this canonly occur
IF... ."Eventually, hedifferentgroups sponta-
neously beganto ask or demandthatthey get a
chance to rewrite their reasons. The students
returned to Research & Revise to help them
develop an improvedset of five reasons.
Test YourMettleshould makethinkingsuffi-
ciently visible so thata teacher,knowledgeable
peer, or even the student,can identifythe need
for further learning. The preceding examplerelied on thecollegestudentstoassesstheirown
work relative to general standardsand to one
another. This had substantial effects on their
evaluationsof the qualityof their own learningand performance.An importantresearchques-tion is how to
supportless
sophisticatedstu-
dents so they can effectivelylearnto self-assess
(foran approachthat uses contrastingcases,see
Lin&Bielaczyc,1998).This is an important kill
to develop. Ideally, people should be able to
make estimations of their own understandingand make needed course correctionsbeforeheytake a big exam or turn in a big projectat the
office.
Test YourMettleshould serve as a powerful
formative assessment event that guides andmotivatesstudents to revise and improvetheir
work.Inbothmath and science,we have found
that formativeassessmentcoupledwithrevision
opportunities significantly increases achieve-
ment among secondary school students (e.g.,Barron t al., 1995;CTGV,1997;Vyeet al.,1998).
Although it seems obvious that formative
assessmentcoupledwith revisionshould lead to
betterperformance,t is strikinghow few class-
roomsactually encourageformativeassessmentand revision, especially formative self-assess-
ments undertakenby students. Most classroom
scienceprojects, orexample,areonly evaluated
by the teacher at the end (Towler& Broadfoot,
1992).Students never have the chanceto "alphatest"andimprovetheirprojects Gardner,1991).
Go Public
AftercompletingTestYourMettle,students are
preparedto Go Publicwith theirthinkingand
makeavailable to others theirbest solutions tothe originalchallenge.Go Public for Challenge
1, for example, requiredstudents to post their
solutions on the Web forevaluationby an out-
side audience.Thereareseveralreasons that we
ask students to Go Publicwith theirknowledge.One is that public presentationsof knowledgeadd a highstakescomponentthatmotivates stu-
dents to do well. Forexample,we suspect that
asking VanderbiltUniversity students to post
their"reasons orproject-basedearning"on theWebforevaluationby Stanford tudents had an
appreciableeffecton theirwillingness to revise
forChallenge1.Anotherreason forgoing publicis to make student thinkingvisible so otherstu-
dents and teachers can appreciatehigh qualityelements of understanding.Considerthecase of
thedesign-principlevisualizations thatstudents
produced forChallenge2. Manystudents were
annoyed by the unusual nature of the task. Yet,
when students publicly presented their solu-tions in GoPublic,the students in theclass were
uniformly mpressedbythecreativityanddepthrevealed in the visualizations-visualizations
that ranged from the jigsaw puzzle shown in
Figure5 to a three-dimensionalmobile to a Jeo-
pardyTM-typeamewithcolumnsof index cards
that had principleson one side and reasonson
the other. The opportunity to see each other's
ideas is importantif for no other reason than
thatstudentsoftendo not have a chance to learn
or appreciatethe more complex ideas of their
peers.
There are many different ways that one
might encouragestudents to Go Public. In our
work,we haveexploredaskingstudentsto pub-lish to the Internet, o presentwithin theirclass-
rooms, to meet with a panel of outside experts,or simply to write in a collectivenotebook. One
approach hat we find especially excitingcan be
done in the context of open-ended challengeswhere there is little concern about copying of
answers. Studentspublish to the STAR.Legacyitself and help adapt it for future generations.An example of this approachmay be found in
Challenge3 of LBD.Legacy.Thischallengeasks
students to writea college-style essay thatpres-
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 14/22
SOFTWAREORCOMPLEX EARNING 51
Figure5 O- A Design-PrincipleVisualizationMade Public nResponseto Challenge 2
low
ilk
, J d 3 \ ~ : e ;0 , 4 6 t e
q~t~h0(tYwA
entsagood andbadversion of a projectanduses
the class readings to explain the differences.
These essays have been added to LBD.Legacy,and thereforehave been made public.Students
from the class can take home a CD-ROM hat
includes theseessays,andstudentswho takethe
class in subsequent years will be able to read
these legaciesto help guide their own work.We
say more aboutleavinglegaciesbelow.
MULTIPLEEARNINGYCLESORPROGRESSIVEEEPENING
As illustrated n Figure1,STAR.Legacyncour-
agesmultiplechallenges,representedas increas-ingly tall mountains. (STAR.Legacy urrently
supportsup to fivechallenges.)Thesechallenges
provide opportunities for students progres-
sively to deepen their knowledge of the topic
being explored. The early challengescan pre-
pare students for more ambitious later
challenges.Elsewhere,we have found that it is
often advisable for students to begin with more
circumscribedproblemsbeforethey takeon the
complexity and open-endedness of projects(Barron t al., 1998).Forexample,in one study,
6th-gradestudents in one of two conditions
designed business plans for a booth at their
school's fun fair. For this project,students in
each condition received the same directive to
show how theirplanssatisfied variousfinancial
and logisticconstraints. n the Problem-to-Proj-ect condition, students firstcompleted a prob-lem-basedactivityby workingon a video-based
adventurecalled "TheBig Splash" romtheJas-
perAdventureseries.In theProject-Onlyondi-
tion,students did notcompleteany preparatory
activity prior to designing theirfun fairbooth.The differenceswere definitive.Students n the
Problem-to-Projectondition developed plansthat mathematically valuated the feasibilityof
differentboothplans in termsof cost and likelyincome.Incontrast, tudentsin theProject-Only
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 15/22
52 ETR&D, ol. 47, No. 2
condition spent most of their timedebatingthe
frillsthat would make thefun boothappealing.For LBD.Legacy,we used three challenges
that were intended to help the students first
noticevaluablepsychologicalaspectsof project-based learning,then considerhow to design for
those aspects, and finally actually develop les-
sons accordingto principlesof psychologyand
instructionaldesign. Challenge1asked students
to create reasons why project-based earning
may be useful. Challenge 2 asked students to
create and organize design principles for
implementing problem-and project-based ur-
ricula. Challenge 3 asked students to design
good and bad projectsand to explain why theywere good and bad in light of their previous
readingsand activities. For each challenge,the
students moved throughthe usual sequenceof
generating deas,hearingdifferentperspectives,
consulting resources that included scholarlyarticlesas well asactivities,assessingthequalityof theirwork,and thengoing public.
Thismay seem like a lot of timeto spend on
one idea, namely, learning by doing. However,
along theway, the studentsprogressivelydeep-ened their understandingof importantthemes
in educational psychology. Unlike traditional
classes, the themes of cognition,social interac-
tion,andassessmentwerenottaughtasself-con-
tained topics, presented in modular chapters.Instead, LBD.Legacy introduced importantfields and scholars n the service of helpingstu-
dents understand an important method of
instruction.
REFLECTINGACKON LEARNINGNDLEAVINGEGACIES
Reflect Back
We noted earlierthatSTAR tands for software
technology for action and reflection.One waythat we encouragereflectioncomes throughthe
ReflectBackthatoccurs aftercompletingall the
challenges. Students revisit the original LookAhead activity and compare their new,
informedresponses to theiroriginalwork. This
gives the students a chance to see how much
they have learned and to further extend their
knowledge about the domain and their own
learning.
To demonstrate that Reflect Backhelps stu-
dents appreciate their own learning,we ran a
small experiment with an LBD.Legacyclass.
Half (11)of thecollege sophomoresin one class
were randomly assigned to the experimental,
self-comparisoncondition.At the beginning of
instructionwith LBD.Legacy,hey completeda
Look-Ahead hatrequiredthem to evaluate the
compass-buildingprojectdescribedearlier.The
other half of the students completed a filler
activity.Two monthslater,aftercompletingthe
three challenges of the LBD.Legacy,all 22 stu-
dents evaluated thecompass-buildingprojectas
partof ReflectBack. At this time, the self-com-
parison students were given their original
responses. An hour later, masked as part of a
differentactivity,the studentswere askedto rate
how muchthey felttheyhad learned inthe class
on a 7-point scale with 7 being the most. The
self-comparisonstudents ratedtheirlearningat
5.3 whereas the control students rated their
learningat 4.1, a reliabledifference; (20) = 1.9,
SE= .63,p < .05,one-tailed.Ofcourse,we cannot
provethat the self-assessmentstudents'evalua-
tions were more accurate.Even so, besides the
obvious implicationfor how to increasecourse
ratings, he results indicatethattheopportunityto contrastpresent and past knowledge helpsstudents appreciate how much they have
learned.
Studentsrarelyhave a chance to appreciatehow muchthey have learned.Eventhough stu-
dentsusuallyreceivegrades, gradesare indirect
measures of knowledge growth and do notdepend on the student's recognizing-specific
knowledge gains. The lack of opportunities to
reflect on knowledge growth may be problem-
atic,especiallyforlower-achieving tudents.It is
importantfor students to recognizewhen theyhave been successful learners. Reflecting on
growthis especially mportantafterlearningsit-
uations that seemed confusing and perhaps
frustrating.We want students to develop a "tol-
erance for ambiguity" (Kuhn, 1962) and
"healthy courage spans" (Wertime,1979).See-
ing their perseverancerewarded with knowl-
edge gains is important to this end (Dweck,
1989). The Reflect Back feature encourages
observationsabout one's learning.
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 16/22
SOFTWAREORCOMPLEX EARNING 53
LeavingLegacies
STAR.Legacycapitalizeson technologicaldevel-
opmentsforeasily pressingCD-ROMs. tudents
can receive a CD of a STAR.Legacy fter it has
been adapted and modifiedby the teacherand
by the class. This CD provides an excellent
reviewof the course content. It can also include
the students'solutionsto the challengesas well
as the legacies theyand theteachershave left for
the next cohortof students.The students cansee
that they have createda usefulproductfor oth-
ers. We believe this can be very motivatingfor
thestudents as theyrealizethattheirinsightsare
valuable.
Not only do the legacies serve the students
who create them, they also benefitsubsequentstudents who use the modified Legacy. For
example,we arestudyingthebenefitsof havingmiddle-school students leave legacies that
describelearningstories.Thestudentsnarratea
personal event or realizationthat they believe
may help the next cohort of students do a good
job of learning.We predict that hearinga stu-
dent from a priorclass explain somethinglike,"thechallengewas initially frustrating utstick-
ing to it was worthit,"will havepositiveeffects.
Another way the legacies can help subse-
quent students is that they can provide exam-
ples of what a good challenge response looks
like. For the LBD.Legacy,we have alreadydescribed how the students left examples of
theiressays for futuregenerationsof users.As
anotherexample,
wevideotaped
a few students
who described the design-principlevisualiza-
tions they createdforChallenge2. Byincludingthese as legacies, next generationstudents can
get an idea of the creativity that is possiblewithinChallenge2 (e.g., Figure5).
Legaciescan also add new domain content.
This, of course, is part of the idea of flexibly
adaptive instructional design-instructionalresourcescanbe supplementedwith new mate-
rials thatcome fromteachers, tudents,and thelocal community.The first students who used
LBD.Legacy eft materialsthat now constitute
importantresources.We describethese legaciesnext because they bear on the question of
whetherLBD.Legacyan facilitate eachers'abil-
ities to design instruction. nprevious years,the
instructors f educationalpsychologyhadasked
students to teach the class fora given topic.The
number of "stand-uplectures"was surprising.The students who worked with LBD.Legacy
behavedquite differently.ForChallenge2 of LBD.Legacy,tudentshad
to develop a visual representationof design
principles. The Multiple Perspectives for this
challengeincludedsix expertswho eachrecom-
mended relevant readings. Consequently,Research& Reviseincludedsix sets of readingson six topics(seeFigure4). Requiringall the stu-
dents to read all the articleswould have taken
too long. Therefore,small groups of students
were asked to become expertsin a topic.Theirtask was to develop a firm understandingof
their readings and then to create 30 min of
instruction hatwould help the rest of the class
learnwhattheyfelt were theimportantpointsof
the material.They were told they could teach
any way they wanted,but that theirinstruction
eventually had to be "programmed"nto their
respectiveResearch&Revisesubsection.
Two aspects of the students' work suggestthat furtherdevelopmentand researchon flexi-
bly adaptive instructionaldesign is warranted.
The first is the students' capitalizationof local
resources.The second is their developmentof
instructionthat tacitlyincluded importantele-
mentsof theSTAR.Legacyycle.
Capitalizingn LocalResources
The students took advantageof local resourcesin two ways. (a) They spontaneously tracked
down the local experts shown in the Multiple
Perspectives.Somestudentssimplytalkedto an
expertto deepen their own understandingand
gather more readings. Other students inter-
viewed andvideotapedtheexpertsforinclusion
in theirinstruction.In one case, they even bor-
rowed the raw footage discussed in one of the
expert'sarticles,and used it to createa noticing
activitywhere other students tried to find exam-ples of the theoreticalpointsmade in the article.
Ineach of thesecases,one can see how the Mul-
tiple PerspectivesaspectofSTAR.Legacy elpedto create ntellectual ommunity.(b)Theyfound
local experts who did not appear in Multiple
Perspectives.In one case, for example, the stu-
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 17/22
54 ETR&D,ol.47, No. 2
dents createda lesson that comparedthe case-
based and clinicalmethodsof legal instruction.
To do this,they interviewedlaw professorsand
students,as well as lawyerswho reflectedback
on the relationshipbetween their legal educa-tion and theirpractice.In tandem,the two uses
of localresourcesprovideaniceexampleof how
STAR.Legacy supports the development of
instructionthatadaptsaccordingto opportuni-tieswithin the localcommunity.
IncorporatingheInquiryModel
The students'legacies naturally ended to incor-porateelementsof the learning cycle in flexible
ways. Previously,we found that many peoplewho have seen short demonstrations of
STAR.Legacyhave tended to worryaboutpro-ceduraldetails.Theyasked how Test YourMet-
tle differs froma Challenge,wondered whether
students could GenerateIdeas in Research &
Revise,and questionedwhether students could
go backwards in the cycle. This worried us
because we do not want to imply that learningevents can only occurin a given sequence.We
want people to think about these important
learningevents and to use themflexiblyin theirown instruction.Generating deas, forexample,is fundamentalto all learningand it should cer-
tainly"beallowed"in Research&Revise.Fortu-
nately, afterhaving seen the students' legacies,we believe that some of people's confusion
stems from their short exposure to the
STAR.Legacyframework and the novelty of
flexiblyadaptiveinstructionaldesign.
The students who usedLBD.Legacy daptedits inquiry model for their own legacies. The
majority of the students adapted the inquirymodel in a similarway. First,they beganwith a
challenge.Forexample, Figure6 shows the leg-
acy that one group of students developed to
teach about home-school communication. In
their legacy, they included four family profilesin written form (one is shown in the figure).
Figure6 O TheLegacy One Groupof StudentsLeftas a ResourceforLearningAboutHome-SchoolConnections
Research
What you should do.
Ms. Jones Mrs. Smith
Mr.Rodriguez Mr.Chang
InstructionsThomasFamilyLupton FamilyLione FamilyChavez Family
0Tips
Help
Notebook
You rerandmotherand-grandfthrioneYouWae
hadustody.fyourwo,.,
grandchilidren.sinetheir
parents :ied oyears g .You aveowedetlakehbestcarepossibleotfyourgrandchildrenndprpoydehebet .educetiohaa'e6iblebleo
ur.sources-ofincomea-dre:sociaalsecurity enefits
etirement.pay,
ndocciaalndija aideijobsniordercm.ak~urehe:chil&den eveytling'heyneed. Youareactivelyinvolved
extracrnicularectigitUies.ou,-
spendIsailytihmeadh ightandhelpithhomework nd
projects.ouwo.uld[enioyt
.keepregularconitactithheteacherandeas nolve aspossible.
Go Back
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 18/22
SOFTWAREORCOMPLEX EARNING 55
They also included four videotapedinterviews
with teachers. The challenge was to decide
which familyshould go with which teacherand
to explain why. A second commoncomponent
of the students' legacies was the instruction othe class membersthat they explicitlygeneratetheirsolutionsto thechallenge.So,in thecase of
the family-teacherconnection,the class gener-ated its solution to the challengeof matching
parents and teachers. Third, the students
includedtheview of anexpertorseveralexpertswho explainedtheiranswers and reasoning.In
the family-teachercase, the expert viewpointwas simply a written statementthat had been
craftedby the students. In the case of a legacy
featuring legal education, the students video-
taped an expert who wrote an articleon legaleducation.In eithercase, the studentsused the
experts as a combinationof MultiplePerspec-tives and Research& Revise.Thisis becausethe
expertspointedout what the students hadover-
looked,buttheyalso deliveredtheprimarycon-
tentabout what is important o know aboutthe
domain. Finally, as the fourth component of
theirlegacies,the studentsincludeda TestYour
Mettle that allowed the class to extend the
knowledge to a new context and to self-assess
whether they had learned the intended con-
cepts.Forexample,one groupof studentsasked
the class to apply its knowledgeto a previouslyunseenvideoof aclassroom.Theclasshad to see
if itcouldnoticethetheoreticallymportant on-
ceptsatplay in thisclassroomsetting.
All told, the students naturally ncorporatedthe ideas of a challenge,a periodof generation,the delivery of expertresourcesfromthe com-
munity,andtheopportunity o test understand-
ing. Some of the students seemed to be aware
that they were explicitly adapting the Legacy
cycle, whereas otherssimply thoughtthey had
comeup with a neatway to teach.Ineithercase,
the resultsare encouragingbecausethey show
that the STAR.Legacyframework provided
enough of a design theory that the studentscould create original instruction and could
adapt STAR.Legacy o their own instructional
goals.A worthwhile line of research,n addition
to formalizing the preceding observations,
mightbe to follow teachers nto theirclassrooms
to determinewhether, when, and how theycon-
tinue to adapt or create instructiononce theyleavetheluxuryof theuniversitysetting.
SOFTWAREOOLSHATROMOTEFLEXIBLYDAPTIVE
INSTRUCTIONALESIGN
Our surveys of professionaldesigners indicate
thatacommoncriterion or agood designmodel
is whetherit helps designersmakedesign deci-
sions as efficiently and smoothly as possible.
STAR.Legacy, owever, requiresmanyinstruc-
tionaldecisionsthatdependon an understand-
ing of learning theory (and on havingknowledge of the domain under instruction).
We try to scaffold the applicationof learning
theory by including the explicit inquiry cycleand the Tips described earlier. We are also
exploring a more inductive scaffold where
designersreceivea templateLegacythat houses
severalexamplesof prior designs foreachcom-
ponent. The designers can choose one of the
examples as a model to guide theirdesign for
thatcomponent,andtheycanslowly adaptitbyswapping in theirown content (Bell,1998).At
this point, however, we are primarily ryingto
maximizecreativityin design rather than pre-
maturely constraining people's designs. Our
experienceswith the LBD.Legacy uggest that
over a more protractedperiod of time peoplebecome comfortable with the STAR.Legacyframework.Additionally,in work with profes-sional instructionaldesigners(Schwartz,Lin,et
al., in press), we have noticed how differentcomponents,such as TestYourMettle, ncreas-
ingly help designers think explicitlyabout the
contentthey expectstudentsto know.Similarly,we have found that when designers begin to
sequence multiplechallengesratherthan use a
single one, they begin to consider what ideas
theyexpectthe learners o generalize.
We assume that an understandingof learn-
ing goals and events is an excellentway to help
designers to adapt to and capitalize on localneedsandresources.To thisend,we havedevel-
oped the explicitinquirycycle. Wealso assume
that thesimplicityof authoring n STAR.Legacy
encourages flexibility. STAR.Legacydoes not
include many authoringtools. Those tools that
are included areprimarily orhelpingdesigners
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 19/22
56 ETR&D,ol.47, No. 2
accesscontentthatis developedandstoredout-
side of the STAR.Legacyshell. For example,
LBD.Legacy ists articles that students should
consult; it launchesvideo and audio as in the
case of MultiplePerspectives; t launchessuchapplicationsasa WebBrowserand asimulation;
and it suggests activitiesthat can be completedin theclassroom.Inthis regard,STAR.Legacys
more of an organizational"launchpad"than a
placeto program argeamountsof originalcon-
tent.
Ourgoal is toprovidesimple ways forpeopleto authorand adaptSTAR.Legacyn pedagogi-
cally sound fashion. Consequently, we have
kept the authoring capabilities quite simple.Thereis a single tool pallet thatpeople use to
include content for any of the componentsof
STAR.Legacy.We have also designed the tools
toencouragereflectionon learninggoals.Thisis
important because we found that when we
introducedprototypesof STAR.Legacy, eopletended to organize differentlearningmaterials
according to types of media. For example, in
Research& Revise,they organizedthe learningresourcesaccordingto useful Internet ocations,
suggestions for hands-on activities, textual
resources,video segments, simulations,and soforth. This is a fine organizationalscheme fora
merchandisecatalog, but ideally, like instruc-
tional materials hould be organizedat theped-
agogical level (Murray,1998)or by the concept
they serve, rather than by the media they use.
Consequently,we modifiedthe tools to encour-
age designers to "justify" he learningfunction
for a particular nstructionalactivity. Figure7
shows an example of the main programming
tool thatencouragesthisjustification.Users add a learningevent to STAR.Legacy
by clicking on a design tool from the palletshown at therightof Figure7.The toolopens the
dialogboxshown at the upperleft of the figure.This box asks the designerto define various fea-
tures of the learningevent, including physical
appearance,a descriptiveaccount of its learning
Figure7 Z TheProgrammingTools orAddinga LearningEvent o STAR.Legacy
4 File Edit Tools Uiew Objects TeHt Tips 00 00 00Q.
Action Name: Parent-Teachernteraction
Re Action:O Movie @ Go to card
O Sound 0 OpenURL
a Launch O Launch application
( Clear ] info Help
Details: Card Name -SShow Parent-Teacher Interaction
Name
(Description CardName FSelect Card EditCard Notebook
SDmageone
* Youwill try toconnectamiliesand eachersonthebasisofimportant ropertieshatyounotice n each.
* Youwill learn that here areseveral distinct eacher
styleswhen t comes o makingonnections iththehome. Paret-Tea lnter.ctiori* Youwill also learn thatfor parents hereareseveraldistincttypesofexpectationsor being nvolved n their GO Back
child's chooling.
.................
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 20/22
SOFTWAREORCOMPLEX EARNING 57
function,and its action.The actiondefines what
type of interactiveevent occurswhen a student
or teacherclicks on the interface.These events
include playing a movie or sound, opening a
browsertoa specific ocation,showingtext,sim-ulations,and so forth.Inthe figure,the selected
actionopens a new page (shownin Figure6).
Toencourageattention o instructional oals,
designersareexpectedto provideeachlearningevent with a captionand a learningdescription.Thecaptionis meantto tell studentswhat topicor issue is covered by that particular earningevent. Duringinstruction, he descriptionof the
learningevent appearswhen the user rolls the
mouse over the caption. The description is
expected to preview how the activityrelatesto
accomplishingthe challenge.Thisprovides the
dual functionsofencouragingdesignersto think
about meeting instructionalgoals and helpinglearners recognize the intent and content of a
learningevent as it relates to the largergoal of
meeting a challenge.The descriptivebox at the
lower leftof Figure7 beginsto meetthesegoals,
although it does not directlymap the learning
activitybackto the challengeof creatingdesign
principles orproject-basedearning.
CONCLUSIONS
Ourdiscussionwas foundedon a basicassump-tion:to optimize the effectivenessof an instruc-
tionaldesign, it is important o make thedesignconformto importantprinciplesof learningand
assessment, and to fit the requirements, kills,and resources of teachers, learners,and their
community. We, among others, are workingtoward a learning theory for instructional
design that can help teachersadapt or designinstruction ortheirclassrooms. n acoursesuch
as educational psychology, the theories of
Piaget,Bandura,and Vygotskyprovideimport-
ant psychological insights into learning.How-
ever, those theoriesdo not easily translate nto
instructionalpractices.We believe a more "userfriendly" theorycan help people know enough
about the process of learning to make sound
instructional decisions and adaptations ofinstructional materials. We also believe that a
technologicalenvironment hat makesit easy to
implementthis theoryis usefulin thisregard.
To support improved pedagogy as we
researchearning heory,we havecreateda mul-
timedia, instructional design environment
called STAR.Legacy.STAR.Legacyoffers one
exampleof how to organizeand facilitate earn-
ing in "challenge-based"nvironmentssuch as
case-, problem- and project-based learning.
STAR.Legacy upports the design of environ-
ments that are simultaneouslylearner,knowl-
edge, assessment and community centered
(elaboration fthispointis available nCTGV,n
press).
An especially important feature of
STAR.Legacys that it is designed to be flexibly
adaptive.Instructionaldesignwould beeasier f
we could assume that we are aimingat a fixed
target. Assumingstudent and classroomhomo-
geneity, however, is analogous to the common
assumptionthatall thepeople from a particular
foreigncultureare the same. It is easy, forexam-
ple, to read about theJapaneseeducationalsys-temand assumethattheyhave only one way of
doing things. Anthropologists,however, note
that culture does not impose homogeneity;instead, it provides ways for organizing the
diversityinherentwithin its population(Sato&
McLaughlin,1992).Ratherthan assuming that
diversity s theexception,STAR.Legacys based
on the assumption that settings, teachers,and
learnershave importantvariability n terms of
local practices and standards, learningresources,styles, and prior knowledge. Conse-
quently, we want teachersto be able to adaptinstructional materials flexibly to fit their cir-
cumstances.
Our discussionof STAR.Legacyocusedpri-
marilyon our uses of it to improveourpracticesascollegeteachers whichoften includeteaching
preservice teachers). The data gathered are
informal for the most part, but informative
nonetheless. In particular,people have consis-
tentlyreactedextremelypositivelyto theabilityof STAR.Legacyto help them visualize the
inquiry process. We also found that students
(many of whom were preserviceteachers)and
designerswere able to reflecton learningandto
break ree fromprevious ways of doing things.
Ourultimategoal is to use STAR.Legacy s a
platformfor further research-not so much to
study STAR.Legacy ut to inquireinto general
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 21/22
58 ETR&D, ol. 47, No. 2
questions relevant to the design of complexinstruction.For example, what are the advan-
tages of beginninga unit with a clearoverview
(includingapretestthathelpsstudents visualize
the kinds of environmentsin which they musteventually use theirknowledge)versus simply
jumping into a challenge?To what extent do
opportunitiesfor studentsto firstgeneratetheir
own ideas help them better appreciate the
insightsofferedby theexpertsfeatured n Multi-
ple Perspectives?What is the effect of seeingone's own teacher as a videotaped expert?To
what extent mightcase-,problem-,and project-based lessons benefit from a summarizinglec-
ture designed to help students organize thediverse experiences that were part of their
inquiry (see Schwartz & Bransford,1998)?Our
simple study on how to increase students'
appreciation of their learning shows how
STAR.Legacyreadily supports research into
these questions and more. Our plan is to con-
tinueto collaboratewith teachersanddesigners,and to submitourquestionstoempirical est. OW
DanielL.Schwartz,eanBrophy, iaodong in,and
JohnD. Bransfordrewith heLearningechnologyCenter tVanderbiltniversity. aniel chwartz
[email protected] supportedby grant#R305F60090
fromtheDepartment f Education.Theauthors
thankthe educationalpsychologystudentsfortheir
contributions o thispaperand Amy Ryce or her
editorial alents.
REFERENCES
Barron,B.J.,Schwartz, D.L., Vye, N.J., Moore, A.,
Petrosino,A.,Zech, L.,Bransford, .D.,&Cognitionand TechnologyGroupat Vanderbilt.1998).Doingwithunderstanding:Lessons romresearch n prob-lem-and project-basedearning.JournalftheLearn-
ingSciences, , 271-311
Barron,B.J.,Vye, N.J., Zech, L., Schwartz,D.,Bransford, .D.,Goldman,S.R.,Pellegrino, .,Morris,
J., Garrison,S., & Kantor,R. (1995).Creatingcon-
texts for community-basedproblem solving: TheJasper hallengeseries.InC.N.Hedley,P.Atonacci,& M. Rabinowitz(Eds.),Thinking nd literacy:Themindat work.Hillsdale,NJ:Erlbaum.
Barrows,H.S.(1985).How odesign problem-basedur-
riculumforthepreclinicalears.New York:Springer.Bell, B. (1998).Investigateand decide learningenvi-
ronments:Specializing task models for authoring
tooldesign.JournalftheLearningciences, ,65-107.
Bereiter,C., &Scardamalia,M. (1985).Cognitivecop-
ingstrategiesand theproblemof "inert"knowledge.In S.F. Chipman, J.W.Segal, & R. Glaser (Eds.),
Thinkingndlearningkills:Current esearchndopen
questionsVol.2, pp.65-80).Hillsdale,NJ:Erlbaum.
Bransford,J.D., Franks,J.J.,Vye, N.J., & Sherwood,
R.D. 1989).New approaches o instruction:Becausewisdom can'tbe told. InS. Vosniadou &A. Ortony(Eds.),Similarityndanalogical easoning. ew York:
CambridgeUniversityPress.
Bransford,J.D., & Nitsch, K.E. (1978). Coming to
understandthings we could not previously under-
stand. InJ.F.Kavanagh& W.Strange(Eds.),Speechandlanguagen thelaboratory,chool ndclinic.Cam-
bridge,MA:MITPress.
Bransford,.D.,
&Schwartz,D.L. inpress).Rethinkingtransfer: A simple proposal with multiple
implications.To appear in A. Iran-Nejad& P.D.Pearson(Eds.),Review f Researchn Education 24,xxx-xxx. Washington DC: American Educational
ResearchAssociation.
Bransford,J.D., Sherwood, R.D., Hasselbring,T.S.,
Kinzer,C.K., & Williams, S.M. (1990). Anchored
instruction:Whywe need it andhow technologycan
help.InD.Nix & R.Spiro Eds.),Cognition,ducation,and multi-media:Exploringdeas n high technology.Hillsdale,NJ:Erlbaum.
Bransford, .D.,Zech, L.,Schwartz,D.L., Barron,B.J.,Vye, N., & Cognition and Technology Group atVanderbilt. (in press). Design environments thatinvite and sustainmathematicalthinking.Toappearin P. Cobb (Ed.), Symbolizing,ommunicating,nd
mathematizing: erspectivesn discourse, ools, andinstructionalesign.Mahwah,NJ:Erlbaum.
Bridges, E.M., & Hallinger, P. (1995). Implementingproblem asedearningn leadershipevelopment.ni-
versity of Oregon:ERICClearinghouseof Educa-
tionalManagement.Brown,A.L.,&Campione,J.C. 1994).Guided discov-
ery in acommunityof
learners.n K.
McGilly Ed.),Classroomessons:Integrating ognitivetheoryand
classroom ractice pp. 229-272). Cambridge,MA:
MITPress
Cobb,P.(1994).Wheres themind?Constructivistndsocioculturalperspectiveson mathematicaldevel-
opment.Educationalesearcher,3, 13-20.
Cognition and Technology Group at Vanderbilt.
(1992).TheJasper eries as an exampleof anchored
instruction: Theory, program description, and
assessment data. Educational sychologist, 7, 291-315.
Cognition and Technology Group at Vanderbilt.(1994).Generativelearningand anchored instruc-
tion:Design,research ndimplementationssues. In
B.P.M.Creemersand G.J.Reezigt(Eds.),Newdirec-
tions in educational esearch:Contributionsrom an
internationalerspective. roningen: CO.
Cognition and Technology Group at Vanderbilt.
(1996).Lookingat technologyin context:A frame-
8/8/2019 Schwartz et al
http://slidepdf.com/reader/full/schwartz-et-al 22/22
SOFTWAREORCOMPLEX EARNING 59
work for understanding echnologyand education
research.In D.C. Berliner& R.C.Calfee(Eds.),The
handbookfeducationalpsychologypp.807-840).NY:
Simon& SchusterMacMillan.
Cognition and Technology Group at Vanderbilt.
(1997). The Jasperproject:Lessons n curriculum,instruction, ssessment,ndprofessionalevelopment.Mahwah,NJ:Erlbaum.
Cognition and TechnologyGroup at Vanderbilt. in
press).Adventures nanchored nstruction:Lessons
frombeyondtheivorytower.Toappear n R.Glaser
(Ed.),Advances n instructionalsychologyVol. 5),Mahwah,NJ:Erlbaum.
Dweck, C.S. (1989).Motivation.In A. Lesgold& R.
Glaser(Eds.),Foundationsfor a psychologyf educa-
tion.Hillsdale,NJ:Erlbaum.
Gardner,H. (1991).Assessment n context:The alter-
native to standardizedtesting. In B. Gifford& C.O'Connor Eds.),Future ssessments:hanging iews
ofaptitude, chievement,ndinstruction(pp. 77-120).Boston:Kluwer.
Gibson,J.J.,&Gibson,E.J. 1957).Perceptual earning:Differentiation r enrichment.Psychological eview,
62,32-51.
Gick, M.L.,& Holyoak,K.J. 1983).Schema nduction
and analogical ransfer.CognitivePsychology,5,1-
38.
Gragg,C.L.(1940,October19).Wisdomcan'tbe told.
Harvard lumniBulletin.
Hmelo,C.E. 1998).Problem-basedearning:Effects nthe earlyacquisitionof cognitiveskillsin medicine.
JournalftheLearningciences, ,173-208.
Krajcik,.,Blumenfeld,P.C.,Marx,R.W.,Bass,K.M.,&
Fredricks, .(1998).Inquiry n project-basedcience
classrooms: Initialattemptsby middle school stu-
dents.JournalftheLearningciences, ,313-350.
Kuhn,T.S.(1962).The tructure fscientficrevolutions.
Chicago:UniversityofChicagoPress.
Lin,X.D.,&Bielaczyc,K.(1998).Supporting etacogni-tiveactivitiesnlearning bout omplexubject omains.
Manuscriptubmitted orpublication.
Michael, A.L.,Klee, T., Bransford, .D.,&Warren,S.(1993).The transition rom heory otherapy:Testof
two instructional methods. AppliedCognitivePsy-
chology, , 139-154.
Murray,T.(1998).Authoringknowledge-based utors:Tools for content, instructionalstrategy, student
model, and interfacedesign.Journal f theLearningSciences, ,5-64.
Newell, A.,&Simon,H.(1972).Human roblemolving.
EnglewoodCliffs,NJ:Prentice-Hall.
Nunn, C.E. (1996).Discussionin college classrooms:
Triangulating bservationand surveyresults.Jour-nalofHigherEducation,7,'23-26.
Penner,D.E.,Lehrer,R.,& Schauble,L. (1998).From
physical models to biomechanics:Adesign-basedmodelingapproach.Journalf theLearning ciences,
7,429-449.
Perfetto,G.A., Bransford,J.D.,& Franks,J.J.(1983).Constraintson accessin a problemsolving context.
MemoryndCognition,1,24-31.
Sato, N.E.,& McLaughlin,M W.(1992).Contextmat-ters:Teaching nJapanandin the UnitedStates.PhiDeltaKappan,3,359-366.
Schwartz,D.L.,Biswas,G., Bransford, .D.,Bhuva,B.,Balac,T., & Brophy,S. (in press).Computertoolsthat link assessment and instruction:Investigating
what makeselectricityhardto learn.Toappear nS.LajoieEd.),Computersscognitiveools:Thenextgen-eration.Mahwah,NJ:Erlbaum.
Schwartz,D.L.,&Bransford, .D. 1998).Atimefortell-
ing.Cognition&Instruction,6,475-522.
Schwartz,D.L.,Lin,X.D.,Brophy,S.,&Bransford, .D.(in press). Towards the development of flexiblyadaptive instructionaldesign. To appear in C.
Reigeluth Ed.),Instructionaldesign heories ndmod-
els,VolumeI.Mahwah,NJ:Erlbaum.
Spiro, R.J.,& Jehng,J.C.(1990).Cognitiveflexibilityand hypertext:Theoryand
technology
orthe non-linear and multidimensional raversalof complexsubjectmatter. n D. NixandR.J. piro Eds.),Cogni-tion,education,ndmultimedia:Exploringdeasnhigh
technology. illsdale,NJ:Erlbaum.
Towler,L.,& Broadfoot,P. (1992).Self-assessmentntheprimary chool. Educationaleview, 4,137-151.
Vye, N.J.,Schwartz,D.L.,Bransford, .D.,Barron,B.,Zech, L.,and Cognitionand Technology Groupat
Vanderbilt. 1998).SMART nvironments hatsup-portmonitoring, eflection, ndrevision.InHacker,
Dunlosky,& Graesser Eds.),Metacognitionn educa-
tionaltheory
ndpractice.Mahwah,NJ:
Erlbaum.
Wertime,R.(1979).Students'problemsand "courage
spans."InJ.Lockhead ndJ.Clements Eds.),Cogni-tive process nstruction.Philadelphia:The FranklinInstitutePress.
Whitehead,A.N. (1929).Theaimsof education.NewYork:MacMillan.
Williams,S.E. (1992).Puttingcase-basedinstruction
intocontext:Examples romlegaland medicaledu-
cation.JournalftheLearningciences,,367-427.