Educational Evaluation and Policy AnalysisSuimmer 2003, Vol. 25, No. 2, pp. 119-142

Resources, Instruction, and Research

David K. CohenStephen W. Raudenbush

Deborah Loewenberg BallUniversity of Michigan

Many researchers who study the relations benveen school resources and student achievement haveworkedfrom a cautsal model, wvhich typically is implicit. In this model, some resouirce orset of resourcesis the causal variable and student achievement is the ozutcome. In afewv recent, more nu(anced versions,resource effects depend on intervening influences on their use. We argue for a model in wvhich the keycautsal agents are situated in instruction; achievement is their outtcome. Conventional resources canenable or constrain the causal agents in instnrction, thus moderating their impact on student achieve-ment. Becautse these causal agents interact in wvays thzat are unlikely to be sorted out by multivatiateanalysis of natutralistic data, experimental trials of distinctive instnrctional systems are more likely tooffer solid evidence on instnrctional effects.

Keywords: expetiments, instnrctional effects, research and policy, school effects

For most of the history of U.S. public schools,conventional educational resources were seenas the key to making schools work. Educators,parents, and policymakers acted as though theyassumed that money, curriculum materials, facil-ities, and their regulation, caused learning. Manystill seem to assume that, as they write about the"effects" of class size or expenditures on learn-ing. The phrasing implies that resources carry"capacity." Regulation has been thought to workby steering resources and thus capacity, withinand among educational organizations; the ideais that ability grouping or segregation influenceachievement by influencing access to resources.These assumptions made school improvement

seem straightforward: allocate more resources orregulate schools' allocation of them.

Access to schooling does affect outcomes. Stu-dents learn algebra in classrooms, not on the street.High school students who study in academicallymore demanding curricula learn more than stu-dents in less demanding curricula, even when stu-dents' earlier achievement is taken into account.Disadvantaged students' achievement 'May falloff when they do not attend school in the summer(Alexander, Entwisle, & Olson, 2001). But sev-eral decades of research suggest that access itselfdoes not cause learning. Researchers report thatschools and teachers with the same resources dodifferent things, with different results for learning.

Earlier versions of this study were published in Boruch, R. & Mosteller, F. (Eds.), Evidence Matters: Randomized Trials in Edu-cation Research. Brookings Institution, 2002 and in the Center for Teaching Policy's working paper series (http://depts.washington.edu). Seattle: University of Washington. We thank Simona Goldin for extraordinary research assistance, and Jere Brophy,Anthony Bryk, Jeremy Finn, Fred Goffree, Henry Levin, Richard Mumane, Will Oonk, Annemarie Palincsar, Jeremy Roschelle,and Alan Ruby for helpful comments. Grants from The Pew Charitable Trusts and the Carnegie Corporation of New York toMichigan State University and The University of Michigan, from The Office of Educational Research and Improvement (U.S.Department of Education) to The Consortium For Policy Research in Education (CPRE) and The Center For the Study of Teach-ing and Policy, and The Atlantic Philanthropies, to CPRE, supported various elements of the research. None of these people oragencies are responsible for the ideas in this article.

119

Cohen, Raudenbush and Loivenberg Ball

The differences depend on the use of resources;access creates opportunities for resource use, butresources are only used by those who work ininstruction.

Some would say that is obvious, but if it is ob-vious in principle, it has not been so in practice.Consider the almost exclusive focus on resourceprovision, in 150 years of education policy; onlyin the last few decades has there been any atten-tion at all to use. Consider as well research on theeffects of resources, in which use also has receivedlittle attention. In most cases, the form of analysisseems to assume an unmediated relationship be-tween resources and learning. If the importance ofresource use is obvious, it remains to be under-stood. Hence we reconsider the role of resourcesin instruction. We discuss research, which illumi-nates the nature of resource use. We sketch a theo-retical view of instruction that focuses on resourcesand their use, and discuss evidence on class size toillustrate these ideas. We then consider the impli-cations of our ideas for research on the effects ofresources.

Relating Resources to Outcomes

Educational resources, conventionally con-ceived, refer to money or the things that moneybuys, including books, buildings, libraries, teach-ers' formal qualifications, and more. There is agreat deal of data on such resources, partly becauseit is required for official reporting, but those re-quirements were based on the view that the mea-sures of resources were valid measures of educa-tional quality. The underlying assumption wasthat learning depended on such resources. Yet fourdecades of research on the effects of resourcesraised basic questions about that assumption.They began with Project Talent, with Equality ofEducational Opportunity Survey (Coleman, et al.,1996), and with Inequality by a research group ledby Christopher Jencks (Jencks, et al., 1972). Tonearly everyone's surprise, conventional resourceswere weakly related to student performance. Dif-ferences among school libraries, teacher experi-ence and education, expenditures, science labs,and other resources had weak or no associationswith differences among school average studentachievement. Despite large differences in averageachievement among schools, and especially trou-bling differences between schools that enrolledthe children of affluent and poor parents, differ-ences in the educational resources that most peo-

120

ple thought significant were weakly related to dif-ferences in student performance among schools.The most powerful predictors of school-to-schooldifferences in average student performance wereschool-average parents' educational and socialbackgrounds in contrast to, which resources hadtrivial effects. Schools with more conventionalresources did not have substantially higher per-formance, once students' social and economicbackground was taken into account.

This was often taken to mean that schools didnot "make a difference," an idea that some con-servatives embraced to attack liberal social policy,and that some liberals rejected to defend it. But theresearch was limited: it asked not whether schoolsmade a difference, but whether some seemed tofoster more learning than others, given knowledgeof school-average student social and educationalbackground and resources. Researchers found thatdifferences in schools' aggregate achievementwere not much related to differences in theiraggregate resources.

Most studies since then, prominently includingthe meta-analyses of Eric Hanushek, supportedColeman and Jencks (-lanushek, 1981, 1989). Butsome recently revived claims for conventional re-sources that Coleman and Jencks had reported tobe ineffective. Larry Hedges and his colleagues re-analyzed scores of studies using a different ap-proach to meta-analysis than Hanushek, and foundthat money made a modest difference to studentscores (Hedges, Laine, & Greenwald, 1994). TheTennessee class size experiment showed that somestudents' learning benefited from dramatic classsize reductions (Finn & Achilles, 1990; Mosteller,1995). These reports diverge from the research ofColeman and Jencks, but it has not been clear whataccounts for the divergence. Researchers who an-alyzed the STAR data from Tennessee disagreeabout why class size made a difference, and noconvincing theoretical frame has been offered(Blatchford, Moriarty, Edmonds, & Martin, 1992).

Coleman and Jencks' research was a watershed.Debate about schools previously had focused onresource access and allocation, not results, partlybecause the latter often were tacitly assumed to beimplied in the former. Coleman and Jencks' workcalled that connection into question, and it soonwas difficult to take conventional resources asmeasures of educational quality, or to assume thatadding resources would reliably affect studentperformance. There was much consternation and

accusations, but in the wake of many hard words,several streams of new, more detailed, and oftencreative work began to illuminate the issues thatColeman and Jencks had broken open.

New Views of Instruction

In response to that shift, and often in deliberateopposition to the work of Coleman and Jencks,several researchers tried to decode educationalquality, to discern what made instruction work.One group sought to figure out whether someteaching was more effective and, if so, why. Theyprobed instructional processes and the resourcesused therein. Though they did not say that theywere studying resources, their work offers clues tohow resources are related to school outcomes. Inone summary of the evidence, more effectiveteachers were significantly different from that oftheir peers, at least as judged by students' gains onstandardized tests. More effective teachers plannedcarefully, used appropriate materials, made thegoals clear to students, maintained a brisk pace,checked student work regularly, and taught mate-rial again if students had trouble. They used classtime well and had coherent strategies for instruc-tion. They believed that their students could learnand that they had a large responsibility to help.These teachers deployed resources that helped stu-dents to learn, but the qualities that we just sum-marized were not resources that could be capturedwell in measures of teachers' formal qualifica-tions, or their schools' expenditures (Cooley &Leinhardt, 1978; Brophy & Good, 1986).

Other researchers brought a similar perspectiveto studies of schools. They sought to distinguishmore and less effective schools, and to identifywhat caused the difference. To do so they probedconnections between schools' collective charac-teristics and student performance. Faculty in un-usually effective schools appeared to share a visionof the purposes of instruction (Edmonds, 1984;Rutter et al., 1979; Rosenholtz, 1985). Theyagreed that schools' purpose was to promote stu-dent learning, that it was their responsibility tohelp students to learn, and that all students had realcapacity to learn. Teachers in such schools hadstronger commitment to students' academic suc-cess, and their principals helped to create and sus-tain these beliefs and practices (Edmonds, 1984;1979). A sophisticated study in this line, which fo-cused especially on Catholic high schools, foundthat teachers in more effective schools were more

Resoutrces, Instruction, and Research

likely to have a shared commitment to their stu-dents' academic success, to have strong collegialrelations, and to believe they were obliged to helpstudents learn (Bryk, Lee, & Holland, 1993). Alarge study of "restructuring" schools reachedsimilar conclusions (Newmann & Wehlage 1995).An extensive program of research on teachers'academic community in high schools reportedstrong relationships between teachers' communityand sense of collective responsibility for students'work on the one hand and students' academic per-formance on the other (McLaughlin & Talbert,2001). These characteristics of schools and de-partments could be seen as personal and socialresources-human and social capital-that weremobilized in some schools but not others.

A third line of inquiry probed teachers' andstudents' interactions over specific content, andoffered finer clues to the role resources play in in-struction. Researchers tried to map the domainsthat lay between such gross influences as the timethat teachers and students spent on the one hand,and what students learned on the other. Theyreported that time alone was not consequential(Cooley & Leinhardt, 1978). Only when the na-ture of academic tasks was taken into accountwere effects on learning observed. Teachers'task definition and students' task enactment werethe key influences, and students' performance ofinstructional tasks mediated between teachers'task setting and students' learning. One couldsee this work as an effort to track the paths bywhich several resources-curriculum materialsand teachers' knowledge chief among them-were used in instructional actions that affectedlearning (Leinhardt, Zigmond, & Cooley, 1981).Two other researchers identified the practicesthat distinguished more and less effective readers,and taught them to teachers who in turn taughtthem to students (Palincsar & Brown, 1984).When effectively taught, the practices improvedstudents' study in reading, and their readingachievement (Englert, et. al., 1991).

A different line of work showed that learners'resources could be as crucial as teachers', bydemonstrating that learners' attributions about in-telligence and learning play a key role in class-room work and learning (Dweck, 1986, 1988).Children who saw intelligence as fixed tended toavoid intellectual challenges that might publiclyreveal wrong answers, but children who thoughtthat intelligence was influenced by effort sought

Cohen, Rautdenbush and Lowvenberg Ball

out and used those challenges. Children could betaught to change their attributions; when thosewho saw intelligence as fixed were taught that itcould be influenced by effort, they increased effortand made better use of teachers' feedback. Theylearned how to study better, in part, by thinkingdifferently about the resources they brought toinstruction.

These studies show that some scholars' interestmoved from conventional resources, like money,teacher qualifications, and facilities, to particularinstructional practices and organizational arrange-ments, and the actions, knowledge, and culturethat they entail. If practice-embedded knowledgeand action affect learning, then teachers' and stu-dents' knowledge and actions also are resources.These personal resources mediate between theconventional resources that schools and schoolsystems deploy on the one hand, and learning ac-complishment on the other. Many researcherstreat teaching as though it directly provoked learn-ing, but in the work summarized here, effectiveteaching encouraged and closely supported whatstudents did in instruction, and students' workhelped them to learn, or noL Teaching is portrayedas activities that enable students to use materials,tasks, and other resources more or less well. Muchinstruction that researchers had associated withindividual teachers' work also turned out to havecollective features; it was shaped by teachers'work together, by leadership, and by the organiza-tions and cultures in which students and teachersworked.

The effects of resources depend on both accessand use: students and teachers cannot use re-sources they don't have, but the resources they dohave are not self-acting. Simply collecting a stockof conventional resources cannot create educa-tional quality, for quality does not arise simplyfrom such attributes. If resource effects depend ontheir use, then modeling the effects requires a the-ory of instruction, for that is where most resourcesare used. Understanding instruction poses a seri-ous challenge to causal inference about the effectsof resources, but we begin by sketching some ele-ments of a theory.

Resources and Instruction

Instruction consists of interactions among teach-ers and students,around content, in environments.The interactions occur in distance learning, smallgroups in classrooms, informal groups, tutorials,

122

and large lectures. "Interaction" refers to no par-ticular form of discourse but to teachers' and stu-dents' connected work, extending through, days,weeks, and months. Instruction evolves as tasksdevelop and lead to others, as students' engage-ment and understanding waxes and wanes, and or-ganization changes (Lampert, 2001). Instruction isa stream, not an event, and it flows in and drawson environments-including other teachers andstudents, school leaders, parents, professions, localdistricts, state agencies, and test and text publish-ers. This view of instruction has roots in early 19thcentury ideas, yet many researchers and practi-tioners still refer to teaching as though it wassomething done by teachers to learners.

To illustrate, we sketched a hypothetical 2nd-grade mathematics class. The school districtrecently made mathematics a priority for improve-ment, and adopted a new text series. Every teacherreceived a complete set of the materials, and, tosupport the initiative, the district mathernatics co-ordinator organized ten professional developmentsessions for teachers. Several elementary schoolprincipals devoted meetings to math instruction. Adistrict committee also developed a map, indexingthe goals and benchmarks of the new texts to thestate tests, to help teachers connect them. Yet theschool board is divided on the prograrn and re-cently cut the math coordinator position from fullto three-quarters time, to fund computer support atthe middle school.

To continue from the day before, the teacheroffers a conventional subtraction problem:

72- 28

But her focus is less conventional. She asks stu-dents to do more than calculate the answer. She in-structs them to copy the problem and to use base-ten blocks to model the numbers and processesprecisely, as well as to figure out and justify theanswer.' She walks around, observes students'work, and sees that some are having trouble usingthe base-ten blocks. One boy is trying to count out72 using only small cubes. Another is not using theblocks, but is meticulously drawing 72 hash markson his paper. And though several have efficientlymodeled 72 with seven rods and two cubes, someare also modeling 28 while others are trying to take28 away from 72.

Resources, Instnrction, and Research

The teacher opens discussion by asking stu-dents for the answer to the problem. She gets fourdifferent answers: 46, 100, 56, and 44. Recordingeach answer on the board, she says, "Let's figureout which one is right." Danya's hand goes up: "Ican show it with my blocks. It should be 100.""Nooo," call out several children. "Let's see whatDanya does," replies the teacher, knowing thatDanya made an error common for students at thislevel-adding instead of subtracting. But theteacher thinks that working through and reasoningmathematically about the solution in public, andexposing the mix-up, will be useful for Danya'sand her classmates' learning.

Danya goes to the overhead and carefully laysout seven rods and two little cubes and then tworods and eight little cubes. "Notice how Danyahas represented 72," the teacher points out, step-ping in to review an important concept. "She usesas many rods as she can then the rest with cubes.Can someone explain how what she has donewith the blocks matches how we wvrite 72?" Sev-eral children explain: "The seven rods go with the7, for seven tens, or 70. And the two little cubesgo with the 2, for two ones. So it is seventy plustwo," asserts Guina.

Danya then pushes all the blocks together: shecounts the eight and two little cubes as ten, andtrades them for a rod. As she starts to count the tenrods, 'Ten, twenty, thirty . . ." she realizes hererror. "I was adding," she announces, ruefully."The answer is not 100." "Good work, Danya,"says the teacher, crossing 100 off the list of pro-posed answers. "Being able to figure out when ananswer is not right, and why, is important in math-ematics. Would someone else like to try to showwhich is the right solution?"

"It's 44," Katie announces confidently, and "Ican show it." At the overhead, she correctly rep-resents 72 with seven rods and two cubes, and thentrades in one of the rods for ten more cubes, re-sulting in six rods and twelve cubes. She quicklyremoves two rods and eight cubes. "See? It's44." Several children nod. "What do you think,Danya?" asks the teacher. Danya nods. "Mmm-hmm. I agree."

"Ruben, can you show with numbers whatKatie did with the blocks?" asks the teacher call-ing on a boy who has been sitting slumped over inhis chair. "I didn't do it," he says softly. "My dadtold me that only babies use blocks to do math.""Blocks are not for babies," replies the teacher.

"Using the blocks shows that you can explainwhat we are doing. Who wants to show it with thisproblem? Then we will try another."

James comes up to the board and writes theconventionial subtraction procedure:

6 12

'2- 28

44

He explains that the 12 is the twelve cubes thatKatie had after she traded in one of the rods, andthat then she had only six rods left, not seven.Then he explains that he took away eight cubesfrom the twelve, leaving four, and two rods fromthe six, leaving four. "So that shows 44, like Katieshowed, and this is how we write it."

"Nice job, Jarnes," comments the teacher. "Sonow we see that 56 is not the right answer, and not46 either. Does anyone know how someone wouldget one of these answers?" "By forgetting to re-group," calls out Lucy. "By forgetting to cross outthe tens," shouts Leon. "Good, okay, so these aremistakes that children make sometimes," says theteacher. "Let's try a couple more." She puts twonew problems on the board, strategically selectedby the curriculum developers to focus the studentson the decision of whether or not a problem re-quires regrouping:

51- 19

59- 11

"I want you to work alone on this for a few min-utes, and this time and this time, I would like to seeeveryone showing it with the blocks and also inwriting, okay?" While the children start copyingthe problem onto their notepads, she stoops downbeside Ruben to help him get started. She knowsthat his father is upset with the new math programand is worried that this is affecting Ruben's work.She works with him to set up the first number. Hesits, immobile. Then he slowly gets five littlecubes and one little cube, separate, and lays themout next to each other in two groups:

The teacher asks him what number the blocksshow. "Fifty-one," he begins, and then says: "No,

Cohen, Raudenbush and Lovenberg Ball

I guess this is only six." "Terrific. So do you seehow we can show 51 with the fewest blocks?"asks the teacher. Ruben pauses, and begins slowlyto pull out some of the rods, and counts, "Ten,twenty, thirty, forty, fifty," he murmurs. Pullingover one little cube, he continues, "Fifty-one.""That's it, you got it now" says the teacher."What's next?" "I need to trade in!" he exclaims.The teacher asks Ruben to do it. He carefully takesone of the rods and trades it in for ten little cubes."Can you record what you did now?" directs histeacher. He crosses off the 5 and writes 4, andcrosses off the 1 and writes 11. The teacher tellshim to finish the problem.

The teacher sees the principal at the door andbeckons her in. She asks how Ruben is doing, andreports that his father complained that his son wasnot getting enough math skills, that they spendtoo much time working with blocks and toys andnot enough time doing mathematics. The teachersuggests meeting to explain the work. "Perhapswe should meet with more parents, since he is notthe only one," the principal says.

In this example, what we casually call teachingis not what teachers do, say, or think, though thatis what many researchers have studied and manyinnovators have tried to change. Teaching is whatteachers do, say, and think with learners, concern-

ing content, in particular organizations and otherenvironments, in time. Teaching is a collection ofpractices, including pedagogy, learning, instruc-tional design, and managing organization.2 Thereare more practitioners than teachers, more prac-tices than pedagogy, and the environments ofteaching and learning are implicated in the inter-actions. These ideas are roughly depicted inFigure 1.

Resources are used as teachers design lessons,set tasks, interpret students' work, and managetime and activity. To do so teachers and learnersmust operate in several domains: they must holdand use knowledge, coordinate instruction, mo-bilize incentives for performance, and manageenvironments. The domains are not always dis-tinct in practice, but it is more convenient to treatthem separately for analysis.

Knowledge Use

The effects of resources depend partly onknowledge. The best materials are of little use ifteachers cannot turn them to advantage in fram-ing tasks, or if students cannot use them to engagethe tasks. Ample school budgets can have no con-structive effect on learning if they are not usedto hire good teachers and enable them to workeffectively. Observers would report that such

FIGURE 1. Instruction as interaction.

124

Resources, Instniction, and Research

schools had rich resources, but that the potentialto affect instruction was unrealized (Odden &Busch, 1998).

Knowledge counts in several ways. Teacherswho know a subject well, and know how to makeit accessible to learners, will be more likely tomake good use of a mathematics text, to use it toframe tasks productively and use students' workwell, than teachers who don't know the subject,or know it but not how to open it to learners. Ourteacher used knowledge of subtraction and mod-eling to focus students' work on key topics. Sheknew that having students use the blocks merelyto obtain an answer was not sufficient, and so em-phasized correspondence among the numbers,their representation in the base-ten blocks, the op-eration of subtraction, and the symbolic form ofthe problem. She also made error analysis, a sta-ple of good mathematical work, part of instruc-tion, rather than treating errors as shameful. Theteacher attended to Ruben's resistance, and usedher knowledge to open the content and help himuse the task. She used her mathematical insight toexploit both the arithmetic and the representa-tional dimensions of the problem. Had she seenthis as only a matter either of getting answers orof using manipulatives, the lesson would have un-folded differently. Yet there is no way that teach-ers and students can know enough to always useresources optimally, for that would require omni-science. Nor can those who design materials orother resources know enough to fashion thembest for all uses, for that would require perfectforesight.

Similarly, students who have learned to reflecton their ideas, listen carefully, and express them-selves clearly are likely to make better use of ma-terials, teachers, and other students' work. Theyalso are likely to make it easier for other studentsand teachers to use their work. How students andteachers organize their interactions also shapes ac-cess to resources: if they work in classrooms thatsupport the respectful expression, explanation, andscrutiny of ideas, they are likely to generate moreusable material for instruction, and have more re-sources to use, than classrooms in which teachersdo most of the talking, students work in isolation,and errors are shamed.

This domain of practice includes an extra-ordinary array: knowledge of academic subjects,practices of learning and teaching in which suchknowledge is managed and the organization and

management of instruction. In our example, theclassroom was structured so that children coulddiscuss their work. The teacher managed in partby listening attentively and helping students to at-tend to each others' ideas. Students' achievementthus depends in part on how deftly teachers probeand understand their work; the strengths or dis-advantages they "bring" are partly a matter of whatteachers can see and hear, and how skillfully theyrespond. In another class, with a teacher who coulduse mathematics less well, Ruben might havebeen seen as "unmotivated," offered an incentiveto engage, and might not have gotten the help heneeded to make progress. What reformers term in-structional "capacity" is not a fixed attribute ofteachers, students, or materials, but a variable fea-ture of interaction among them.

As things presently stand, teacher quality isless well predicted by formal qualifications thanby more direct indicators of teachers' knowledge,which probably also proxies for their ability tomake pedagogically fruitful use of materials andstudents' work (Ferguson, 1991). The currentweak validity of formal qualifications is an arti-fact of existing professional education, in whichintending teachers are not well educated in con-formance with sound standards of academic per-formance. Instead they are sketchily educated inconformance with very general standards that areweakly related to teaching performance and aca-demic learning. If teachers were better educatedin conformance with academic performance stan-dards, formal certification would be more tightlyrelated to performance, and qualifications wouldbe better proxies for teaching proficiency.

Coordinate Instruction

The use of resources also depends on coordina-tion in instruction. One dimension of coordinationconcerns teachers' and students' work on content.The teacher in our example worked on subtractionproblems, but if students were fiddling with theirpencils, passing notes, or drawing, they would beless likely to learn from the tasks she posed. Ifseveral students were absent the day before, theywould be less likely to know what was being dis-cussed, and to learn from the problems. Even ifeveryone worked on the problems, and knew whatto do, the teacher might addressed them algorith-mically while the curriculum supported the devel-opment of mathematical ideas. If she addressed thework as'the text did but not probed students' ideas,

Cohen, Raudenbusht and Lovenberg Ball

she would not have known how they understood it.Instruction occurs in time, which opens up anotherdimension of coordination. How do these subtrac-tion problems connect with each student's worktomorrow, and what is to happen in two weeks?Learning depends on students and teachers makingbits of lesson work develop and connect, yet thereare always absences, memory lapses, and inatten-tion to contend with (Lampert, 2001).

Since instruction consists of more or less com-plex interactions among teachers, learners, andcontent, there are many opportunities for uncoor-dination. Other dimensions of coordination con-cern pacing across time, relations among class-rooms within grades, among successive grades,and between work in schools and external guid-ance for instruction, as from academic standardsand assessments. Coordinating instruction thusdepends on making connections among teachers'and students' ideas, among students' ideas, amongboth over time, and between both and elements inthe environment. These things depend on teach-ers' knowledge of content, on how it is repre-sented, on learners' understanding, on agents inthe environment, and on the will to make fruitfulconnections. Coordinating instruction in thesesenses also depends on social resources that buildtrust, support the collection and analysis of evi-dence about teaching and leaming, and enablecommunication about the evidence.

Each dimension presents potential sub-prob-lems. If students and teachers do not focus on thesame task, learning is likely to suffer. If students'work is not paced to maintain cognitive demand,students may be overwhelmed or bored, and leam-ing will suffer. If steps are not taken to coordinatein these and other ways-including how work isorganized in periods, days,-and years, and studentmobility within and among schools-instructionis likely to be less effective. If our teacher merelyassigned problems and collected and graded pa-pers, as has been typical, the lesson would havebeen far different. Instead she coordinated in manyways, including her trouble-shooting work withRuben, her use of base-ten blocks, and increasingdifficulty from the first problem to the second twoproblems. Using resources effectively depends onsuch coordination.

Mobilize Incentives

It takes effort to teach and learn, and that oftencreates friction within learners and among them.

Incentives are required to mobilize effort to over-come that friction, and creating such incentives isa third instructional domain. Teachers have incen-tives to exert themselves and press for ambitiouswork, for their professional success depends onlearners' success-or on explaining why learnerscould not succeed. Learners also have reasons towork hard, for it can satisfy their curiosity andwish to learn, enhance their sense of competence,and enable them to meet teachers' and parents'hopes. But teachers and learners also have incen-tives to do less ambitious work, for friction andeffort increase as learners encounter more dif-ficulty, as do uncertainty, risk of failure, andchances to disappoint themselves and others. Thesubtraction problems our teacher used are of thislatter sort, for the representation and explanationcan be challenging for second graders. Teacherswho frame such work are more likely to encounterlearners' resistance, frustration, and failure, evenif greater success beckons. Learners and teacherswho doless ambitious work reduce these problemsand increase the chance of some success. Teachersand students face a dilemma that is stilched intothe work: should they aim low, accepting modestresults in return for some success, or aim high, risk-ing resistance and failure in hope of more impres-sive accomplishments for learners and teachers?To teach and learn is to manage that conflict.

In our example, the teacher posed a challeng-ing task when she asked her students to representa conventional subtraction problem using con-crete materials, and to make a careful correspon-dence between their work with the blocks and thesteps of the procedure. Her introduction of thetask with a review problem allowed students toget started and build confidence and to be clearabout what she wanted them to do. Her invitationto resolve discrepancies in the proposed answers,using the blocks and explanation, and to do so asa performance in front of the class, focused thestudents on precision and meaning, and engagedthem in a challenge. Mobilizing incentives to learnand teach is not simply a matter of "motivating"students and teachers, but also of using knowledgeand skill to situate incentives to work hard in spe-cific academic tasks, and using performance in thetasks to motivate engaged work. A basketballcoach who was keen for his players to win, andgood at cheering them on, but knew little about of-fensive playmaking, would be as unlikely to pro-duce winning results as a history teacher who was

126

Resouirces, Instnrction, and Research

keen for her students to learn, but knew little aboutthe historical record, or how to set suitably en-gaging and challenging academic tasks.

Manage Environments

Instruction is situated in what often are depictedas external influences, including other teachers,school leaders, parents, district policies, state re-quirements, and more. But if these things do insome sense exist outside instruction, they alsoappear within it. A fourth domain of instructionis managing such elements of the environment.When teachers and students deal with problemsof coordination, resource use, and incentives,they do so in and with environments. Teachersand students are more likely to exert themselvesif schools are linked to institutions of higher ed-ucation or firms that offer strong incentives forambitious performance, for students and teachersimport the incentives. Teachers and studentswhose principals urge ambitious work will bemore likely to do it, while equally able colleaguesin schools whose principals prefer less ambitiousperformance will be less likely. Again, those work-ing inside instruction import external guidance(Bishop & Mane, 1998; Bishop, 1998, Skrla &Scheurich, 2001). Teachers and students workwith the classroom manifestations of such influ-ences; though they have little leverage at theirsource, they can notice or ignore them, capitalizeon them or leave them unused.

Coordination also is less difficult in environ-ments that offer coherent guidance for instruction.As the teacher in our example worked on mathe-matical concepts and skills, she also dealt withparents' views of the new math curriculum. Shelearned that Ruben and perhaps others were doingpoorly in part because their parents disparaged thework. She also knew of many signals about in-struction, including upper-grade teachers' expec-tations, her principal's exhortations to make sureall students develop basic skills, the district's in-vestment in the new curriculum that focused onconcepts, and state tests that rewarded speed andaccuracy. The United States has,had distinctivelyincoherent guidance for instruction, which makesit more difficult to coordinate within instruction.Standards-based reform has sought to order theconfusion, but it has not reduced the prolifera-tion of guidance in many states, and may have in-creased it, as new guidance that calls for coherence

has been laid on many earlier layers of less coher-ent guidance.

Teachers and students shape environments bywhat they notice and how they respond, but envi-ronments shape attention and response. If schoolleaders place a high priority on improving disad-vantaged students' work, teachers are more likelyto engage that task. If leaders go further, by offer-ing teachers opportunities to learn how to improve,it is more likely that teachers will constructivelydeal with student disadvantage. How educatorsmanage environments is influenced both by theclarity and authority of priorities, and by teachers'and learners' attention, will, and knowledge. Themore knowledgeable and skillful teachers are, themore likely they will make productive use of sig-nals from the environment, but the more inchoatethe environments, the more difficult it is for eventhe best teachers to make such use of them.

Many researchers treat environments and prac-tice as separate; they view teachers and learnersas technical workers inside practice, and environ-ments as outside influences. Researchers andeducators often portray economic and social dif-ferences among families as external causes ofdifferences in student performance, yet those dif-ferences only count as they become active insideinstruction, as learners import elements of the en-vironment and teachers interpret them. Studentsand teachers are delegates from environments be-yond the technical and professional world, yet theyare theikey agents in that world. The designers andpublishers of materials also frame content to man-age the environments of instruction, as when textsintended for sale in Southern states fail to mentionevolution. Teaching and learning are not simplyinternal technical work that external environmentsinfluence, for teachers and learners work, insideinstruction, with and on elements of what is con-ventionally thought to lie beyond practice.

Resources Reconsidered

Our analysis distinguished among types ofresources, and offered a view of causality. Con-ventional resources include teachers' formalqualifications, books, facilities, class size, andtime. Personal resources include practitioners'will, skill, and knowledge. Environmental andsocial resources include state guidance for in-struction, academic norms, professional leader-ship, and family support. Each type counts.Students in classes of 35 probably have less

127

access to teachers' time and expertise than thosein classes of 15. Students with outmoded textsprobably have access to less substantial contentthan those with up-to-date books. Students in lessdeveloped nations, with uneducated teachersand few books have access to fewer resourcesthan those in industrialized nations with better-educated teachers and more books.

Yet conventional resources only count as theyenter instruction, and that happens only as they arenoticed and used. Some sorts of resources seemmore immediately usable than others: books andother materials are close to the center of instruc-tion, but class size is not as close to the center, norare school facilities.3 Their effects all depend onteachers' and students' personal resources: theirknowledge, skill, and will, but some sorts of re-sources are easier to use than others. Studentsoften learn from materials on their own, with nomediation by teachers. Environmental and socialresources like leadership are most likely to influ-ence learning by influencing what more immedi-ate resources teachers and students notice and use.Leaders can support or deflect common academicpriorities: the principal in our example engagedthe school with mathematics by making it a focusof conversation and attending to parents' views. Ifteachers' and students' use of resources is centralto instruction, the chief means by which actors intheir environment can influence use is by focusingtheir attention and improving their capabilities asusers.

If we are roughly correct, it is quite unlikely thatnaturalistic research could, by itself, yield valid in-ferences about the use of conventional resources.The pathways are too intricate, and the ways inwhich teachers and learners capitalize on theavailable resources, or compensate for their lack,are extremely complex. We argue, a bit further on,that the interactive nature of classroom life is likelyto obscure causal relationships among resources,users, and outcomes. At the same time, studies ofhow conventional resources are used can lead toimportant hypotheses about the effects those re-sources might have under varied approaches to in-struction. We discuss the evidence on class size inthe next section, to illustrate this point. Such hypo-theses would be an excellent contribution, butonly if they could be tested systematically. Weconsider below how such hypotheses could begenerated and tested concerning resources andinstruction.

Class Size

The evidence on class size reduction (CSR) of-fers an opportunity to apply and develop the ideassketched above. It is a distinctive type of resource,not as close to the center of instruction as materi-als, but not as remote as many environmental in-fluences. Hence it is more usable than the lattersort of resource, and less usable than the formersort. Research on class size has accumulated fordecades, including small experiments, studies ofnaturally occurring variation, and two recent largestudies: the experiment in Tennessee (STAR), andthe evaluation of the initial years of Wisconsin'sCSR program (SAGE). Despite differences intheir designs, STAR and SAGE had comparableeffects, which are consistent with those that GeneGlass and his colleagues published in their earliermeta-analyses (Glass et al., 1982). Finn andAchilles (1999) summarized the STAR results:"On average, students in small classes evidencedsuperior academic performance to those in otherconditions . .. The effects were always attributableto the difference between the average performanceof small classes and that of other class types ...The benefits were substantially greater for minor-ity students or students attending iiner-cityschools in each year of the study ... [and] wasalso statistically significant for all school subjectsin every subsequent year.. ." (Finn & Achilles,1999, pp. 98-99).

These findings are relatively uncontested, butdifferences ". . . arise over the implications ...One interpretation, shared by Achilles, Bain,Finn, Mosteller, and others, is that the STAR find-ings confirm the intuition of most teachers: Chil-dren perform significantly better in classes withfewer students ... class size reduction is expen-sive but .. . less expensive than ... allowing stu-dents to fall behind in school." Others questionthe size but not the direction of the effects, andhold ". . . that the costs of the class size reductionsfar outweigh the small achievement gains" (Ritter& Boruch, 1999).4

Our interest is in the dynamics of class size ef-fects; we want to use the research to illuminate re-source use, and that requires comparisons amongtypes of use. Yet most of the research focused onaverage effects, not on the distribution of effectsor sources of variation in that distribution. A fewstudies report on the distribution of effects, re-porting that their size and direction varies. AlanKrueger reported that two-thirds of the ". . . small-

Resources, Instruction, and Research

class effects are positive, while one-third are neg-ative.... Thus, some schools are more adept attranslating smaller classes into student achieve-ment. . ." (Krueger, 1999, p. 526), Eric Hanushekfound fewer positive effects: in 79 STAR schoolswhich had classes for each experimental condition(small, regular, regular with aide), small classesoutperformed regular classes in 40 schools. In theremaining 39, students in small classes made gainsthat were equal to or less than those of students inregular sized classes, hence there was not equallyeffective use of smaller classes.5

What distinguished classrooms in which therewere positive effects from those in which therewere not? The answers could illuminate resourceuse, and might suggest ways to enhance the effectsof class size reduction or other resources. WhenGene Glass and his colleagues undertook theirstudies, they found substantial positive effectson students' academic performance, and discussedpossible causal mechanisms. They wrote that:"Class size has no magical, unmediated effect onstudent achievement. Instead, it influences whatthe teacher does, his or her manner with the stu-dents, and what the students themselves do orare allowed to do. These differences in class-room process in turn influence outcome measureslike student achievement, student attitudes, andteacher morale. It is essential to study and under-stand this full sequence of events. A class size re-duction provides an opportunityfor improvementsin classroom processes. Teachers can take ad-vantage of this opportunity in different ways andto different degrees. [emphasis added]" (Glass etal., 1982, p. 67)6

Several researchers investigated how teachersor students "..... take advantage of this opportu-nity," with some useful results. Yet none seemedto frame the inquiries with theories that pointedto likely explanations, and several seem to haveexpected spontaneous change. Hence few re-searchers were in a position to test hypothesesabout how this resource was used. But twogroups did use STAR to investigate the dynam-ics of teachers' and students' response. Finn andAchilles did a Grade 4 follow-up of STAR class-rooms, collecting test scores and behavioral data,7

and concluded that students' behavior changedmuch more than teachers'. ". . . The key to thebenefits of small classes is increased student en-gagement in leaming ... every student is on thefiring line. It is difficult or impossible to with-

draw from teaching-learning interactions ..

(Finn and Achilles, 1999, p. 103.)The importance of group size seems undeni-

able, but if it is ". . . difficult or impossible" forstudents to hide in smaller classes, how to ex-plain the small classes which did not outperformlarge ones? The answer seems to turn on stu-dents' and/or teachers' use of the resource, eitherbecause students who did not gain as much fromsmall classes were more advantaged and thusless sensitive to the resource change, or becauseteachers made it more difficult for some studentsto improve their use of this resource, or both.8 Ineither event, if differential use seems critical,published studies offer no evidence with whichto pursue the point.

Were there evidence, our theory suggests atleast three hypotheses about students' and teach-ers' resource use. Students could make better useof themselves and materials with CSR, capitaliz-ing on fewer distractions to attend to their ownwork more effectively, or to spend more time onit, or both. Or they could make better use of teach-ers, using the teachers' greater availability to gainmore attention than in large classes. Or teacherscould press students to do either or both of thesethings. In the former case, students would use thechanged social situation to improve their opportu-nities to learn, and they would be the key causalagent. In the second, the effects would depend onstudents trying to use their teachers, and on teach-ers responding constructively, so causality wouldbe joint. In the third, teachers' initiative woulddrive either or both of the first two mechanisms.'Either the second or the third entail some changein teaching, whether in the allocation of time, pres-sure brought to bear on students, or both.'0 Themechanisms are not mutually exclusive, but dif-ferent. Stating the hypotheses, which we wouldhave been less likely to do without our theory, alsoshows that if class size reduction changes oppor-tunities to teach and learn, they are not irresistible;students and teachers must use the opportunities,and that requires will and knowledge.

Carolyn Evertson and John Folger also usedSTAR to probe the dynamics of use, and their re-port could bear out all three hypotheses. Theystudied math and reading lessons in 52 2nd-gradeclassrooms, and wrote that ". . . in mathematics,students in small classes initiated more contactswith the teacher for purposes of clarification, giv-ing answers to questions that were open to thewhole class and contacting the teacher privately

129

Cohen, Raudenbusla and Lowvenberg Ball

for help [hypothesis 2]. In reading, more studentswere on task, fewer students were off task, andstudents spent less time waiting for the next as-signment" [hypotheses 1 and perhaps 3,]. Theyalso wrote that ". . . there are predictable differ-ences in class processes that follow simply fromthe numbers: students are more visible; each stu-dent is more likely to get a turn more often dur-ing class lessons; students don't have to wait aslong for help; student initiate more contacts withteachers." (Evertson & Folger, 1989, pp. 9-10).

These actuarial differences are clear in thenumbers, but nothing follows "simply." As Glasset al wrote, they are opportunities; CSR becameeffective only as it was used by teachers and stu-dents. In another report on STAR, Evertson andRandoph tried to probe change in teaching in re-sponse to CSR. To their surprise, it changed littleor not at all: ". . . instructional formats observedin these [2nd-and 3rd-grade STAR] classroomsshowed little variation . .. tried-and-true methodsof reading and niathematics instruction [pre-vailed]." (Evertson & Randoph, 1990, p. 98.) But"change in teaching" might mean several things.It could mean that students' or teachers' the allo-cation of time changed; it could mean that therewere more student initiated requests for help thatteachers responded to; or it could mean that therewas more teacher pressure on some students towork; or it could mean some combination ofthese. None of these changes entailed new peda-gogy or content. The study suggests that higherscores were due to the use of added time and so-cial space. If so, teachers and students turned toinstructional mechanisms that were easy to use,not to unfamiliar practices that would have beenmore difficult.

Finn and Achilles also concluded, in an analy-sis of 4th-grade classrooms, that ". . . teachers donot, defacto, alter their primary teaching strate-gies. Small classes are academically superior notbecause they encourage new approaches to in-struction, but because teachers can engage in more(perhaps enouigh) of the basic strategies they havebeen using all along. More profound changes occurin students' participation in learning, includingstudents who would be unwilling to participate ifthey were part of a larger class ... When class sizesare reduced, the pressure is increased for each stu-dent to participate in learning, and every studentbecomes more salient to the teacher. As a result,there is more instructional contact, and studentlearning behaviors are improved." Poor and mi-

nority students benefit most because ". . . dis-engagement is found more commonly amongminority or low-income students ... " (Finn &Achilles, 1998, p. 103)"

This feature of the response to STAR was alsofound in Wisconsin's SAGE program. This non-experimental program began in 1996-97, andgrew rapidly. Researchers evaluated the initialtwo years, comparing the performance of studentsin classes of 15 or fewer and a single teacher andin classes of 30 students and two teachers, withthat of comparison students in classes of 30 stu-dents and one teacher. Researchers reported testscore gains of approximately the same size as thosein STAR, in both of the first two groups when com-pared with the third (Molnar, et. al., 1999). Theyalso reported dynamics that parallel those inSTAR. SAGE teachers in both treatment condi-tions reported that they spent less time on dis-cipline and class management, more time withindividual students, more time covering the samecontent as in previous years, but little other changein instruction (Molnar, et al., 1999; Zahoric,1999). Apart from ". . . slight increase in hands-onactivities .. . the dominant mode of teaching re-mains direct instruction. Teachers continue tostructure, manage, and pace all activities. Theteacher gives information, asks questions, praisescorrect responses, and controls interactions withstudents in other ways. The students are largelypassive in that their role is to listen and to followthe teacher's directions." (Zahoric, 1999, p. 52.)

On this view, CSR offers students and teachersmore access to each other. On average, both takeit, and scores improve. Yet this is not the end ofthe explanatory line, for time and attention are notthe only influences on resource use. Our theoryholds that curricula and environments may influ-ence instruction, and Evertson and Randolph notethat STAR occurred in a state with a vigorousbasic skills program. Tennessee Basic Skills First(TBSF) included a system of tests that focused onthe TBSF curriculum: 'The state designed objec-tives and an assessment system for the basic skillscurriculum (language arts and math). Local schooldistricts are required to either use the state's sys-tem or design their own which meets state guide-lines ... Skills are measured by frequent objectivetests. The demands of the program ... encourageadherence and few deviations ... While class sizewas manipulated in this study, outcome measureswere not. Student achievement was still measuredon standardized tests tightly tied to this basic-skills

Resources, Instruction, and Researcht

oriented approach to reading and math." (Evertsonand Randolph, 1990, p. 101.)12

This suggests anotherhypothesis about resourceuse in STAR: pervasive and potent state pressurefor basic skills inhibited change in content andpedagogy. Content was prescribed, and there wereincentives to conform. Evertson and Randolphwrote that the basic skills system permeated class-rooms. "Where this type of recall-oriented perfor-mance is closely linked to what will be tested,there may be no need or encouragement for teach-ers to change ... to more complex or multitasksettings." (Evertson & Randolph, 1990, p. 101). Itwould be easy to conclude that TBSF the reasonthat pedagogy changed little in STAR. Evertsonand Randolph used it to explain why professionaleducation for teaching smaller classes, which theyoffered teachers, had no discemable effect. Yet,if TBSF probably influenced teaching, giving somuch weight to that environmental influencewould be justified only if there were other studiesthat reported substantial change in teaching in re-sponse to class size reduction, absent programslike TBSF, and several other studies reveal little orno change in pedagogical approaches, even whenprograms like TBSF did not operate (Betts, et al.,1999; Rice, 1999). No studies report substantialchange in pedagogy owing to class size reduction,so explanations for stability and change must takeaccount of more than TBSF.

Our frame leads us to several hypotheses aboutwhy teaching might change slowly or not at allwith class size reduction. Each derives from ouranalysis of the domains in which teachers mustwork. One is that many teachers lacked the skilland knowledge to make more complex changes,and another is that they lacked the incentives. Weexpect that both played a role in teachers' and stu-dents' use of the added resource. Teachers mustweigh incentives to push students and themselvesto produce more learning and thus more profes-sional success, against disincentives for such pusharising from greater effort, difficulty, problems,failure, and student resistance.13 Substantial classsize reduction offers most teachers and many stu-dents ways to manage this problem at a rather lowcost: teachers can increase the likelihood of suc-cess for students and themselves with little or noincrease in instructional effort, simply by allocat-ing the same amount of instruction, for the samecontent, to many fewer students. Smaller classesalso make it easier for teachers to coordinate in-

struction, and to use knowledge they already knowhow to use, to better advantage; if so, they wouldbe likely to actually reduce the effort teachers hadto make to achieve satisfying results. Studentswould gain an increased probability of success,satisfying their own wishes to learn, their teachers'and parents' desires, or both, simply by takingmore advantage of their teachers' availability, re-duced distraction, or both.

These ideas fit the published evidence. It wouldbe unwise to conclude that teachers and stu-dents in STAR were only responding to changesin group size and time, or that the changes wereeither "simple" or nearly unavoidable.,Even ifthere had been no pressure for basic skills, wewould expect no different distribution of teachers'instructional responses. Modest change in that dis-tribution would occur if there were strong en-vironmental pressures for different content andpedagogy, and more change would occur if, in ad-dition, teachers had substantial help in leaminghow to use the CSR resource differently, in waysthat made it possible for them to solve problems ofincentives, coordination, and resource use. If re-duced class size prompts change in instruction, itwill do so within the parameters suggested by ouraccount of work in the four central domains ofteaching and learning.'4

Class size reduction changes the resourcesavailable, but the effects on teaching and learningdepend on how teachers, students, and others intheir vicinity use the resource. We expect classesin which no improvement occurred to have teach-ers or students who either saw no promise of im-provement in the reduction, or were unwilling orunable to take advantage of it. That would holdfor other resources that teachers bring to instruc-tion, including their content knowledge. Considerthe finding that teachers with higher test scores,or who know more about a subject, have studentswith higher scores in that subject (Coleman, et al.,1966; Jencks, et al., 1972; Ferguson, 1991). If weview teachers as people who stand and deliver,those who know more have more to deliver. Butthe very words are deceptive, because knowledgeis not self-enacting: if students benefit from teach-ers' content knowledge, it is either because stu-dents are able to use the knowledge without benefitof teachers' instructional skill, or because teach-ers with more content knowledge can, on aver-age, put that knowledge to better use in teaching.The two sorts of knowledge differ. Many teachers

m

Cohen, Raudenbush and Loivenberg Ball

who know mathematics can barely use it to teach,as many university students know. Knowledge ofa subject is a necessary condition of such use, forteachers cannot use knowledge they do not have,but it is not sufficient.

More generally, any mix of personal and envi-ronmental resources opens some possibilities forand contains some limits on the use of any con-ventional resource. Given any particular set ofpersonal and environmental resources, researchmight show that added conventional resourcesappeared to independently affect learners' accom-plishments, other things being equal. But that ap-parent independent effect actually would expressan interaction among personal, environmental,and conventional resources. The instructional ef-fects of conventional resources depend on their us-ability, their use by the agents of instruction, andthe environments in which they work. When addedconventional resources appear to directly affectlearning, it is because they are usable, becauseteachers and students know how to use them, andbecause environments enabled or did not impedetheir use. The effects of conventional resource in-crements should be taken to imply these relation-ships. Moreover, these effects express an averageover many differences in use. The potential effectof something like the Tennessee class size exper-iment could be greater than the average, perhapsmuch greater, if teachers and students who madeweak or modest use of it were taught to do better.Experiments, which estimate only the average ef-fects of resources and present no evidence on theiruse, severely limit what can be learned.

If these ideas are correct, then when added re-sources lie outside the range of teachers' and stu-dents' knowledge, norms, and incentives, theywill have no discemable effect.' 5 A hypotheticallegislature might mandate that teachers use inno-vative content standards to engage learners inmore creative and demanding work. The legisla-ture might even provide money to write and dis-seminate the standards and support discussion ofthem. Yet research on the effects of such a policywould probably show that the new resources hadno average positive effects on students' learning,for the policy would have required most teachersand students to work well beyond their skills,knowledge, and will, without providing opportu-nities and incentives for them to learn much more.

When research fails to find effects for particu-lar conventional resources, it should not be seen as

132

evidence that such resources are ineffective untilseveral other explanations are ruled out. One isthat teachers and learners did not know how to usethe resource; ruling that out would require researchon the effects of teaching them to use it. Anotherpossibility is that the change was not enough toenable significantly better use, given extant prac-tices, knowledge, and norms. The evidence onclass size suggests that only large reductionspermit changes in teaching and learning, giventeachers' and students' extant knowledge and mo-tivation.' 6 A third possibility is that some conven-tional resources are not salient to the learning inquestion. Science laboratories might bear on sci-ence learning, but it would be surprising if theywere linked to reading. Building two gymnasiums,three pairs of bathrooms, and a larger playgroundfor every school could have many good effects,but they seem unlikely to be academic effects.

Instructional Interaction and Research

Our theoretical frame makes interaction be-tween teachers and students over content centralto instruction, and portrays teachers and studentsas interdependent actors: teachers' effectivenessdepends partly on how well they can use students'ideas and initiatives, and students' effectivenessdepends partly on how well they can use the taskstheir teachers set, the comments their teachersmake, etc.. . How teachers and students use re-sources like class size depends on their work to-gether. None of this is news to those who observeinstruction, but the interdependence of which wewrite poses a major challenge for research on re-source effects. Teachers and learners are thinkingbeings, and they use each other and other resourcesbased on judgments about which resources to use,how, with whom, and to what end. They base thesejudgments on what they know and believe aboutthemselves, one another, and the content. Someteachers judge with great care and seek evidencewith which they might revise, but others judgequickly and with little care. In either event, teach-ers calibrate instruction to their view of students'capabilities, and their own capabilities to teach.Schools formalize such calibration in abilitygroups, grade retention or promotion, and relatedpractices, which allocate resources within classesand schools, and even, within the same student,across subjects. Students also make judgmentsabout instruction and calibrate their use of re-sources to their estimates of teachers' and parents'

---

Resoutrces, Instrnction, and Researci

expectations, and their own preferences. In ourexample, Ruben navigated between his father'sdisdain for the math program and his teacher's in-tervention. Ruben was less prepared for class, andless interested, since his father told him that thework was not worth doing. Ruben's teacher had tobelieve he could do the work, convince him that itwas worth doing, and help him to use what heknew to do it.

One cannot imagine instruction without suchcalibration-even computer-based instruction in-cludes it. But if teachers adjust instruction withinand among students on the basis of their estimatesof students' capabilities, there will be significantdifferences in the resources that teachers use, orthe ways they use them, with individual studentsamong subjects, among students within classes,and among classes within schools. If so, how canresearchers identify, observe, and measure the re-sources that are used in instruction? If teachers ad-just the tasks they assign and the materials they use,correct estimates of resource allocation and ef-fects would depend on valid evidence of use. Thatwould depend on teachers knowing and articulat-ing what they did, and having the time and incli-nation to do so, or on researchers' valid obser-vation of teachers' reports and practices, or both.Such evidence would not be easy to define or col-lect, especially since teachers often adjust theirown knowledge, skill, and will as they apply them.How can teachers be aware of such things? If theyare, how can researchers learn about them? Lack-ing valid and reliable evidence on such matters,how could we make valid inferences about the ef-fects of resources on learning? Nonexperimentalstudies of resource effects on student outcomesthat fail to take account of how teachers adjust in-struction in light of theirjudgments about studentswill likely misestimate and confound the resourcesused, those merely present, and their effects.

Recent research on "dynamic treatment re-gimes" in medicine and psychotherapy illumi-nates this matter (Robins, Greenland, & Hu, 1999).In such regimes, the treatment is calibrated to thecurrent status of the patient, on the basis of an as-sessment of the patient's condition. The regimeconsists of one set of rules for assessing those tobe treated and another set for assigning interven-tions to them. One can arrive at strong causal in-ferences about the effects of any such regime, ifthose treated are randomly assigned to alternativeregimes. Weaker causal inferences can be based

on quasi-experimental comparisons of regimes.However, within a given treatment regime, whichis where nonexperimental research often operates,it appears impossible to make a meaningful causalinference about the association between treatmentand response. This is because in the continuing ad-justment process, treatments are as much a con-sequence of the patient's current condition as thecause of a subsequent condition.'7 Research inmedicine and psychotherapy shows that a regres-sion of responses on treatments, controlling initialstatus, will not give a reasonable estimate of atreatment effect."8 It also suggests that the effectsof interactive treatment regimes can only be accu-rately evaluated if: (a) there are different regimesthat (b) consist of well-explicated rules for assign-ing treatments, given particular statuses, and (c) theregimes vary across patients treated.

Education is not medicine, and few educationalinterventions come close to the precision of manyin medicine. But if teachers calibrate instruction totheir views of student ability, one could make ac-curate causal inference about instructional effectsonly by reconceiving and redesigning instructionas a regime, or system, and comparing it with dif-ferent systems: "Regime" refers not to authoritar-ian prescriptions, but to systematic approaches toinstruction, in which the desired outcomes arespecified and observed, and in which the intendedoutcomes are rationally related to consistent meth-ods of producing those outcomes. The key featureof such systems of instruction is not the detail ofspecification but consistency between instructionalends and means, and across instruction amonglearners and among teachers.

Conventional resources are not a system of in-struction, for they cause nothing. They are used ornot used in particular systems of instruction. Re-source effects depend both on their availabilityand on their use within those systems. The centralfocus in research on resources therefore should bethe instruction in which resources are used-andhow they are used, and to what effect-not the re-sources alone. One key reason for this is that re-sources and their effects are likely to vary amongthe instructional systems in which they are used.A text that focused almost entirely on phonemicawareness in an instructional system that was de-signed for whole-language instruction probablywould be used differently, and have different ef-fects, than it would in an instructional system thatwas designed for phonemic awareness.

Cohen, Rauidenbusht and Lowvenberg Ball

This line of reasoning has both theoretical ap-peal and several vexing aspects. The continuingadjustment of resources within instruction callsinto question the interpretation of a vast body ofcorrelational research on relations between dis-crete instructional behavior and student outcomes,including many of the studies that we discussedearlier (Brophy, 1988). If research on dynamictreatment regimes may not require randomizedexperiments, randomization is optimal for causalinference. That suggests more caution about causalinference from nonexperimental evidence, a nar-rower role for survey research than has recentlybeen the case in education, and a larger role for ex-perimental and quasi-experimental research. Butif such studies offer a better grip on causality, theyare more difficult to design, instrument, and carryout, and more costly.

New Designs for Research

We have discussed two significantly differ-ent perspectives on resources. In the inherited,dominant perspective, conventional resources aretreated as if they were active agents of instruction,and the key problem is to identify and then deploythe resource mix most likely to improve learning.In a more recent and still developing perspective,teachers and students, and features of their envi-ronments are the active agents in instruction,and the key problem is to identify and mobilizethe knowledge, practices, and incentives that willenable them to best use themselves and otherresources. One perspective is grounded in estab-lished habits of thought and politics, while theother is grounded in studies that probe how school-ing works.

There has been movement between these views.In the last decade or two some policymakers beganto revise views of resources, partly in response toresearch on instruction. Standards-based reform ispremised on the view that schools' learning goalsshould be clarified, and resources used to achievethose goals. Standards and accountability are seenas important because they would influence re-source use. Officials in some districts and stateshave encouraged schools to focus on improv-ing the use of resources (Odden & Busch, 1998).There has been growing interest in more directmeasures of teaching quality and improvingteachers' knowledge through professional devel-opment, rather than relying on course titles and de-grees. There is growing attention to resource use,and the conditions that influence it.

Much action and debate nonetheless focus onclass size, teachers' qualifications, facilities andequipment, and budgets, for these have long beenthe indicators of school quality. They are plainlyvisible, and it is easier to observe and quantifydollars and class size than teachers' knowledge ofmathematics or their skill in using students' work.It is easier to manipulate dollars and class size thanknowledge and skill, and they are much easier toassociate with the taxes citizens pay. Policy-makers' and school managers' actions often arecontested, and require justification to tax payersand voters that generate demands for evidencepoliticians turned to research for help. Specialistsin education, economics, politics and sociologyhave increasingly occupied themselves with theeffects of schooling, and they attend chiefly to theresources that play a part in policy and argument.Data are relatively easy to come by, have face va-lidity, and interest policymakers. Policymakersand researchers can most easily deal with the re-sources that are least directly related to students'learning, while policymakers and managers canleast easily deal with the resources that are mostdirectly related to learning. Improved researchcould help to bring the two closer together.

The Frame

The overarching research question cannot be"Do resources matter"? No deliberate attempt tolearn or teach is conceivable in the absence of con-ventional resources, and there is ample evidencethat teaching is causally related to learning. Theoverarching question must be: "What resourcesmatter, how, and under what circumstances?"

One key circumstance is the desired result. Thequestion can only be answered once an educa-tional goal, and a strategy to achieve it, have beenadopted and spelled out. Thus a better question is:"What do educators need to do a particular job?"Putting it that way helps to make clear that the an-swer would depend strongly on what "the job"was, that is, what is to be taught and learned. Con-ventional resources do not follow from definingan instructional goal, for the instruction requiredto achieve any given goal might be done withsomewhat less skilled teachers, somewhat largerclasses, or a somewhat smaller budget for materi-als and equipment.

Hence the first question should be: "What in-structional approach, aimed at what insbtuctionalgoals, is sufficient to insure that students achieve

134

Resouirces, Instnrction, and Research

those goals?" A second question follows: "Whatresources are required to implement this instruc-tional approach"? The research question shouldnot be the one that most researchers concernedwith school effects have asked, namely, "How dothe available resources affect learning?" Sinceresources can enable and constrain particular in-structional aims and methods, the second ques-tion often would be followed by a third: "Is itpossible to achieve the same or similar resultswith a different mix of resources?" Educationalpolicy and practice inevitably involve negotiationamong goals, instructional means, and resources,and research should weigh the consequences ofvaries resource constraints within instructionalapproaches. It is, however, illogical to conceiveof resources as the "cause" and learning as the out-come. Systems of instruction are the cause, andresources are facilitators or inhibitors of teachingand learning.

Most researchers have placed conventional re-sources at the center of inquiry, and tried to iden-tify how each affects performance, or what thebest mix is. We propose instead to place teachingand learning at the center of inquiry, and to designresearch that helps to identify the resources thatbest support particular goals. It may not seemnovel to write that the effects of conventionalresources depend on how they are used, but thechange would be a revolution of sorts, for it as-sumes that resources are means, and can onlywork in relation to instructional ends. To acceptthat is to bring a kind of theory of relativity to thestudy of resource effects, for one can only con-ceive the effect of resources in relation to a spec-ified aim and a strategy to achieve it. Building anew lab may be essential to one approach to sci-ence instruction but irrelevant to another. Classsize probably is salient to literacy instruction if itentails frequent, high-quality feedback on studentwriting and serious class discussion of the writ-ing, but that approach also requires literate, moti-vated teachers. Class size might be less importantto other educational aims. Research on resourceswould be more fruitful if it was grounded in con-jectures about such relationships and evidence onthe conjectures.

Active and Passive Research Programs

Programs of research on instructional resourcesshould focus on well defined systems. One exam-ple might be a program carefully designed to im-

prove reading in the primary grades, which linkscurriculum and teaching of phonemic awareness,text recognition, and comprehension, to specificassessments in those areas. Such systems of in-struction would have several critical features. Oneis outcome measures that would require studentsto present the academic performances that the in-struction is designed to help them leam. Anotheris the optimal features of the treatment that is in-tended to produce the outcomes, including moreor less elaborated versions of the academic tasksthat were central to the regime, and optimal ver-sions of the instructional media needed to enact thetasks. A third feature would be optimal descrip-tions of the teaching that is intended to help stu-dents use the tasks and materials to produce thedesired performances, including descriptions ofhow teachers would be expected to deal with stu-dents' responses to the tasks.

Such systems would require much more con-sistency in instruction than has been common inthe United States. For without such consistencyit would be impossible, within a system, either tovalidly estimate its effects or to systematically varysome resource constraints while holding otherelements constant. There are very different waysto achieve consistency: instruction could be rela-tively tightly scripted at one extreme, while atanother, communities of practice could be builtaround agreed-upon elements of instruction, usingintensive communication among teachers aboutexamples of students' and teachers' work to de-velop and learn shared criteria of quality andmethods of instruction. In the former case, consis-tency would be created by teachers closely follow-ing detailed directions, while in the latter it wouldbe created by developing professional knowledgeand norms around a skeleton of objectives andtasks. Instruction within regimes could be consis-tent in either case, but the means to achieve con-sistency, and quite likely the content of instructionitself, would vary. Combinations of the two meth-ods and others also could achieve the requiredconsistency.

Though any such system would contain articu-late rules that regulated or characterized instruc-tion, there could be enormous variability in therange of instructional behavior that are governedby such rules. Teaching and leaming school sub-jects are ill-structured domains, and even in themost constrained regimes, rules could not coveranything like the entire range of instruction. A

135

Cohen, Raudenbush and Loivenberg Ball

great deal must be left to teachers and students todeal with on the spot, and, in devising regimes,those who would change instruction would have todecide on the features of instruction to which theywould attend, and those that they would ignore.Some systems might focus on a very constraineddomain, like word recognition or multiplication,while others might focus on broader domains likereading comprehension or place value.

Research on these systems would address tworather different sorts of questions. A first line ofwork should probe the effects they have for stu-dents on its central outcomes, when resources areplentiful. A second line could test the effects ofsuch regimes under various resource constraints,which also could allow various modifications ofthe regime that enable its enactment under differ-ent conditions."9 Pursuing the two lines of workfor any regime would yield evidence about its ef-fects under a variety of resource conditions, in-cluding those that might be optimal. Pursuingboth lines of research for regimes that share out-comes, wholly or in part, would yield evidenceabout their robustness, generalizability, and costeffectiveness. As each was tested and modified,the research program would reveal the resourcesneeded, as well as the ways in which they must becoordinated to produce effects, given the regime.This active research agenda does more than pas-sively discem the effects of extant resource con-figurations; it seeks valid causal inferences aboutspecific instructional designs.

This agenda would give priority to research ondesigned systems of instruction, and thus wouldrequire excellent programs of development, fieldtesting, and revision. A focus on regimes alsowould imply a high priority on experimental andquasi-experimental tests under varied resourcesconstraints. Our principle of relativity also meansthat there could be neither "regime-free" answersto questions about levels, combinations, and co-ordination of resources, nor "regime-free" studiesof their effects.

Given our analysis of mutual adjustment withininstruction, consistent regimes seem the most rea-sonable way to probe causal relationships betweenresources and learning. It would not be useful forresearchers to attempt to disassemble regimes intotheir components and do conventional research ontheir effects. The work that we propose would im-prove understanding of the complex relationshipswithin teaching and learning and open up oppor-

136

tunities for more coherent research on instructionand its effects.

Our approach also would tend to reclefine theopportunities for, and limits on, the sorts of pas-sive research that have become conventional inthe study of instructional effects. Observational orsurvey research typically use existing achieve-ment tests, which do not offer a student outcomemeasure that expresses the aims of a specific, well-conceived approach to instruction. A deliberatelydesigned regime will set clear instructional goals,and research on it would require outcome mea-sures that assess achievement of these goals.Broad-purpose achievement tests woulcl be usedwithin the agenda we propose, since it would beimportant to know how new regimes bear on moretraditionally defined success. But conventionalassessments are unlikely to capture the proximaloutcomes of a well-defined system of instruction.

Even if that problem was solved, passive in-quiry cannot yeild strong evidence on the effectsof instructional systems in best-case situations orunder resource constraints. For instruction is a sys-tem of interaction in which students and teacherscontinually mutually adjust, so it would be extra-ordinarily difficult either to uncover and delineatehow a given resource is used, or to distinguish welldefined regimes. Existing instructional arrange-ments and resource use emerged through histori-cal processes of negotiation and accommodation.To the extent that regimes occur naturally, theyhave developed in part to cope with the resourceconstraints of given settings. It would be difficultor impossible to answer the question "What re-sources are essential, given the regime," becauseof mutual adjustments around existing resources.Instructional practice within a given setting tendsto involve a mix of individualized adaptations, andin natural conditions there would be little chanceto hold the regime constant and vary resources. Ifonly survey data are available, it is essential tomeasure student background and school contextand statistically adjust for them in models that re-late instruction to outcomes, but that can tell us lit-tle about what would happen if instruction weredeliberately modified. Ethnographies and surveyscan reveal how teachers and students think and actwithin a setting, but they cannot reveal how thingswould change with new regimes and resources.Deliberately developed regimes would.

Passive inquiry would play several roles in theapproach that we sketched. Large-scale surveys

could roughly estimate the range of instructionalapproaches, and related resource availability,within regimes. Paired with ethnographies, sur-veys might enable researchers to discern the extentto which anything like coherent systems of in-struction occur "naturally," and, if they do, to iden-tify them. Carefully focused ethnography couldclarify the configuration and operational featuresof existing regimes. Ethnography also can be in-valuable to those who design instruction, for". . . designers can deepen their understandings ofand therefore broaden their abilities to describetheir own regimes when they closely observe andinterrogate expert teachers who implement them(because the teachers will raise questions aboutgaps in the current description, do some things inways that are different from and often better thanwhat the regime called for, and add things thatthe developers may want to adopt as useful elab-orations)" (Brophy, 1988, p. 15). Paired surveyand ethnographic research could illuminate whatstudents know and can do over a range of naturallyoccurring instruction and settings, and focus at-tention on where educational effort might be di-rected. Studies of school facilities and resourceallocation would be useful, mostly for how theyenable or constrain the implementation of well-defined instructional regimes.

Though we argue that instruction is so inter-active as to preclude treating resources as indi-vidual variables, we do not argue that instructionalsystems must remain black boxes. The design ofinstructional systems would require extensivelearning about how instructional systems work,both in their development and in practice, underoptimal and various sub-oprimal conditions.Micro-ethnographies would be helpful in bothareas, and could suggest ways to design instruc-tion for specific sub-groups. Research that clar-ified the internal dynamics of instructional systemwould be especially useful in comparing dynam-ics across resource variations within regimes.

The approach sketched here contains importantroles for active and passive research. One justifi-cation for the former is that it would give an ex-plicit definition to regimes and resources, thuscreating a basis for valid causal inference. Anotheris that it would create a useful context for surveyand ethnographic research, which currently floatlargely free of knowledge-building frameworks.Extant instruction reflects accommodations to cur-rently available resource levels, to views of stu-

Resources, Instnrction, and Research

dent background, and to prior achievement. Validcausal inference about the effects of instruction orresources is extremely elusive in such webs of mu-tual accommodation. Economists would describethis as a situation in which the causal variable ofinterest is "endogenous," that is, determined inpart by current levels of outcomes and other un-observable factors that lead educators to makechoices and compromises. That makes it very dif-ficult to separate effects of causal variables fromthose of a host of other factors, observed and not.Naturalistic survey and ethnographic research canhelp to advance understanding in several impor-tant areas, but they are not well suited to producingdefensible causal conclusions. Doing that requirescausal variables to be made "exogenous," i.e.,varied independent of confounding factors.

The best way to do that is through deliberate,well-defined interventions, to which schools orclassrooms are assigned randomiy. Such assign-ment of students to regimes may be feasible insome instances, but we anticipate that schools orclassrooms could more often be so assigned.20 In-terpreting the results of such experiments wouldrequire carefully designed research on the dynam-ics of instruction within regimes. Active and pas-sive research would be interdependent: withoutethnographic research on instructional dynamics,it would be difficult or'impossible to grasp the roleor importance of various influences on instruction,and so to interpret the experimental evidence. Theresults of passive research programs also couldhelp to generate ideas for regimes and resource al-location within them, as well as helping to expli-cate the dynamics of well-developed instructionalsystems. Active programs of research, in whichdeliberate interventions vary resources in relationto well-articulated regimes, are at the heart of ourproposal, but so are well-designed programs ofpassive research.

Some skeptics expect that administrators, teach-ers, or parents will flatly resist randomized assign-ment to alternative instructional approaches. Whilethere surely will be resistance to such studies insome instances, there may also be strong incen-tives to participate. Participation reveals a publiccommitment to school improvement, and will typ-ically bring new resources to participants. More-over, school districts are currently under pressureto adopt innovative approaches; tying these tosound research will often be appealing. We expectthose who support instructional innovation to

137

Cohen, Raudenbush and Loivenberg Ball

offer incentives to participate in serious tests of in-novations. In some cases, random assignment tonovel regimes will not be practical. In these cases,users will select the regimes, and sound quasi-experimental design will be essential. Researchersmust identify and control selection biases, andcausal inference will be more tentative (Cook &Campbell, 1979). But in both true and quasi-experiments, study of the dynamics of instructionwould be essential to illuminate the role and im-portance of various influences on instruction.

Effective instructional systems will only resultfrom systematic development, including researchthat makes it possible to specify the requiredresources, and their relation to specific aims. Inaddition to the benefits already discussed, the de-velopment of such systems would tend to supportmore precise and common professional conversa-tion. It also would make it possible to rigorouslyevaluate altermative regimes relative to commongoals, to evaluate claims about the effects of par-ticular levels or combinations of resources, withinregimes, and to enable evaluation of varied ver-sions, for which resource requirements differed.Such research is essential both to the more appliedtask of learning how to improve schooling, and tothe more basic task of defining educative resourcesand learning how they are used in instruction.

Conclusion

The research program that we have sketched isnot a design for all educational research, but forinquiries that focus on resource and instructionaleffects. We proposed a dramatic shift, from a re-search frame that gives priority to conventionalresources and asks how they affect leaming, toone that gives priority to coherent systems of in-struction and asks how resources are used withinthem. One key premise is that because resourcesbecome active when used in mutual instructionaladjustment, they are unlikely to have a fixed in-structional value. Their value is likely to depend onthe uses to which they are put, which in tum de-pends on the ends and means of instruction. Tounderstand the nature and effects of resources, re-searchers must focus on how instructional endsand means are defined, and on what resources arecrucial to them. Thus we have proposed designingcoherent instructional regimes, submitting thoseregimes to tests of their effectiveness, and assess-ing how resource constraints modify their effec-tiveness. Within this framework, a variety of pas-

sive forms of research, including surveys andethnographies, can play important roles.

Our proposal has complementary benefits andcosts. On the one hand, our picture of instructionas a system of interactive mutual adjustrnent com-plicates understanding of the dynamics of teachingand learning, and of the ways in which resourcesinfluence them. In such a system, the value ofresources is likely to depend on the ways they areused. That raises fundamental questions about howvalidly conventional research can tease out thecausal influence of particular resources, across agreat variety of schools and classrooms. That mayunsettle many researchers, but it seems inescapableif our account of instruction is roughly right. On theother hand, our account offers a theoretical framefor research on instructional and resource effectsthat builds on several decades of work, that opensup promising research agendas, and that createsopportunities to lodge active and passive researchwithin mutually reinforcing knowledge-buildingstructures.

Some might argue that these agendas would beinsufficient to illuminate policy makers' decisions.To know what resources are optimal for a givenapproach in mathematics tells us little about whatresources are needed in general. Small classesmight be needed to enact a given approach in lit-eracy, but teachers' subject matter preparation,rather than small classes, might be the crucial in-gredient in teaching an effective math curriculumto the same grade. Small classes taught by knowl-edgeable teachers may not be fiscally feasible.Varied studies would tend to send mixed signalsabout how many teachers to hire and what qual-ifications to require, but that is just what extantresearch has done. One chief task of a coherenteducational research program would be to makejust such trade-offs visible, based on sound empir-ical study. If our approach is correct, policymakerswould be well advised to adopt more complex ap-proaches to resource allocation, that capitalize onthe role of resource use.

Others might argue that developing such anagenda is infeasible because well-specified in-structional regimes could not be devised or be-cause experiments could not be done, or becausethe entire enterprise would be too costly. Yet thelast decade's work in reading at The National In-stitute for Child Health and Development, and insome whole-school reforms, show that carefullydesigned systems of instruction can be created,

--------

Resouirces, Instnrction, and Research

and that experimental research is possible (Cook,Hunt, & Murphy, 1999). The work that we pro-pose would be difficult, but if educators and re-searchers took the ideas seriously, a great deal thatseems difficult today could soon be feasible. Onereason for our confidence is the rising demand forsolid evidence on the effects of such interventions;there is likely to be a market for just such work.Another is the recent experience with reading re-search and some whole-school reform models.Still another is the success of several seeminglyimpossible experiments in health care, housing,and welfare. And another is the growth evidence-based medicine, which faced similar problems.

The sorts of research and instructional designthat we have sketched would take careful planningand clear priorities, for such work must be strate-gic. Researchers cannot investigate everything, oreven half of what they might wish to. We aim toinvestigate a few key issues well. Even a modestlydesigned program would require a broad and en-ergetic constituency that included public and non-public supporters and more capable managementthan educational research has had. It also wouldrequire a level of federal commnitment to scientificresearch management that was closer to health re-search than to education. The result could yeild astrearn of sound evidence on the resources requiredto attain a variety of educational aims, and informthought and debate about the aims of schoolingand levels of investment in education. In time itcould inform debate and decisions, and mighteven close out unfruitful arguments as well ashighlight new problems. But it could not prescribedecisions about resources, for those require inter-action among people and groups whose authorityto decide is civic rather than scientific, and whooften differ. Research on instructional resourcescould inform but not replace a broad discussionabout schools and their improvement.

Notes

' Base-ten blocks are wooden blocks designed tomodel place value: a small cube, a rod composed of tensmall cubes, a flat square composed of a hundred of thesmall cubes or ten of the rods, and a block made of tenof the flat squares, or a thousand little cubes. In thiscase, the teacher is using the little cubes as ones and therods as tens.

2 For convenience, we often refer in what follows to"instruction," in which we include this clump of prac-tices, rather than either teaching alone, or the moreclumsy "teaching and learning with materials."

3 We are indebted to Jere Brophy (personal com-munication, Nov 8, 2002), for this and several otherimportant points.

4 See Hanushek, (1999). Several analysts agree withhim that when the size of achievement gains attribut-able to class size reduction are compared with the costs,CSR is shown to be very expensive. See Slavin, (1990),and Levin, Glass, and Meister, (1984).

5 Hanushek, (1999) p. 157. Some differences be-tween the two analyses may be due to differences inthe analyses; Krueger's are based on a pooled within-school sample, but Hanushek's is not.

6 Given Glass et al's analysis, it is puzzling that Finnand Achilles write: ". ..dozens of earlier studies" didnot help to clarify ". . . the classroom processes that dis-tinguish small from large classes" (Finn & Achilles,1990 p. 102). Yet Glass et al. discuss nearly all of themechanisms taken up by Finn and Achilles and otherstudies. Slavin, (1990), reanalyzed Glass's data, and re-ported that much of the effect was due to tutorials, someof which had no academic content.

7 Teachers ". . . rated each pupil who had been inSTAR on the . .. Student Participation Questionnaire...[which assesses specific leaming behaviors . . .judgedby educators to be important . . . The instrument yieldsreliable, valid measures of the effort students allot tolearning, initiative taking in the classroom, and .. . dis-ruptive or inattentive-withdrawn behavior. Finn andAchilles, (1999 pp. 101-042).

8 Jeremy Finn (personal communication, 6/17/02),reports unpublished analyses showing that the size anddirection of the effects are inversely tied to student SES;the more advantaged students were, the smaller the pos-itive effects on achievement.

I Finn and Achilles, (1999) prefer this explanation.10 Al teachers are willing or able to do so; see

Blatchford, et. al (2002)." Blatchford, et. al (2002), report similar results in a

British study; students in small classes got more ofteachers' time, and teachers were more satisfied, butthere was no report of dramatic change in teaching.

12 The TBSF tests were used in the primary grades,but the CTBS and Stanford Achievement Test revealedsimilar results; Finn and Achilles (1990).

13,Ehrenberg, Brewer, Gamoran, and Willms (2001)argue that the institutionalized nature of schooling, inwhich pedagogy is decoupled from organization, ispart of the explanation, and technical constraints likerequired texts are another.

'4 Mitchell, Beach, and Badarak (1989) argue thatthe advantage of small classes may be the result of non-random assignment of very low achieving students tolarge classrooms (alternatively, the nonrandom assign-ment of higher achieving students to small classes).Their models of classroom processes are consistentwith the skewing of large class teaching to the weakest

139

Cohen, Raudenbush and Lovenberg Ball

students, which, in turn, could result in slower learningfor all. The presence of more low-achieving students inlarge classes than would have been expected by chancefrom the overall sample suggests possible migration ofmore able students to smaller classes; if so, this wouldbe a serious threat to the internal validity of the STARexperiment. We have found no way to check on thispossibility. Even if Mitchell et al. are correct, fromour perspective they only offer a different causalmechanism for explaining the use of small class size.For a very thoughtful discussion of the STAR exper-iment on this and other points, see Goldstein andBlatchford (1998).

'5This probably is what Eric Hanushek (1989,1996) means, when he argues that teachers mediate theeffects of resources.

16 Jere Brophy refers to these as "threshold effects"(personal communication, November2002), and notesthat they probably hold for range of resources.

17The causal effect of treatment A relative to treat-ment B is defined as the difference between the po-tential outcomes of a student under A or B. Causalinferences are thus meaningful only when a student ispotendally exposable to alternative treatments (Hol-land, 1986). If a regime were perfectly and thus strictlyenacted, only one possible "treatment" would be con-ceivable at any moment for a particular student: therewould be a treatment A but no treatmentB! This meansthat such a student would have only one potential out-come, so that no causal effect could be defined. In prac-tical terms, this means that, within a perfectly enforcedregime, it is not possible to find or even imagine a stu-dent who is similar to the student of interest but who re-ceives a different dose. It is quite feasible, however, todefine the causal effects of alternative regimes and todesign studies that can estimate those causal effects.

"I The renowned "pygmalion" experiments (Rosen-thal & Jacobson, 1968) in classrooms can be read as ascheme to change the instruction (read "treatment") thatteachers offered students by experimentally inflatingevidence on students' IQs (Raudenbush 1984).

19 Medical researchers distinguish between efficacyand effectiveness trials. Efficacy trials establish whethera treatment can have significant positive effects in care-fully controlled settings with plentiful resources; theyattempt to estimate a maximum potential effect. Effec-tiveness trials take efficacious treatments to the fieldwhere implementation is more challenging and re-sources more constrained. Our recommended approachto studies of educational improvement is similar, withthe added proviso that effectiveness trials deliberatelyvary key resources so that their effects can be rigorouslyevaluated.

20 For example, in evaluating whole-school reformefforts, one might envision assigning schools ratherthan students randomly to treatments. This could bedone ethically if many schools expressed interest in

140

adopting a reform but resources allowed iinplementa-tion in only a restricted number of sites at any one time.Intervenors could promise all schools the opportunityto participate, but the timing of participation would bedecided via lottery. A randomized "wait-list" controlgroup of schools would then be available. If instruc-tional systems were instead chosen by school or class-room, a strong effort to explicate the regime and asso-ciated resources would be in order, as would tentativecausal inference.

I References

Alexander, K., Entwisle, D., & Olson, L. (2001).Schools, achievement, and inequality: A seasonal per-spective. Educational Evaluation and Policy Analy-sis, 23(2), 71-191.

Betts, J. R., & Skolnik, J. L., (2001). The behavioral ef-fects of variation in class size: The case of mathteachers Educational Evaluation and Policy Analy-sis, 23(2), 193-215.

Bishop, J. (1998). .The effect of curriculum-based ex-ternal exit exam systems on student achievement.Consortium For Policy Research In Education.Philadelphia, PA: University of Pennsylvania.

Bishop, J., & Mane, F. (1998, November). The NewYork state reform strategy: Incentive effects of mini-mum competency exam graduation requirements.Paper presented at the National Invitational Confer-ence on "Education in Cities: What Works, and WhatDoesn't," Wisconsin.

Blatchford, P. Moriarty, V., Edmonds, S., & Martin, C.,(2002). Relationships between class size and teach-ing: A multimethod analysis of British infant schoolsAmerican Educational Research Journal, 39(1),101-132.

Brophy, J. (1988). Research on teacher effects: Uses andabuses. The Elementary School Jolrnal, 89, 3-22.

Brophy, J., & Good, T. (1986). Teacher behavior andstudent achievement. In M. C. Wittrock (Ed.), Hand-book of research on teaching, third ed., (pp. 328-375). MacMillian Publishing.

Bryk, A., Lee V., & Holland, P. (1993). Catholicschools and the common good. Cambridge, MA:Harvard University Press.

Coleman, J., Campbell, E., Hobsen, C., McPartland,J., Mood, A., Weinfeld, F., & York, R., (1966).Equality of educational opportunity survey. Wash-ington, DC: U.S. Government Printing Office.

Cook, T. D., & Campbell, D. (1979). Quasi-experi-mentation. New York: Rand McNally.

Cook, T. D., Hunt, H. D., & Murphy, R. F. (1999).Comer's school development program in Chicago:A theory-based evaluation. Unpublished paper,Chicago, IL: Northwestern University.

Cooley, W., & Leinhardt, (1978). Instructional di-mensions study. Pittsburgh, PA: Learning ResearchAnd Development Center, University of Pittsburgh.

Resources, Instruction, and Research

Dewey, J. (1905). The clhild and the curriculutm, andschool and society. Chicago, IL: University ofChicago.

Dweck, C. S. (1986). Motivational processes affectinglearning. American Psychologist, 5, 1179-1187.

Dweck, C. S. (1988). Children's thinking about traits:Implications for judgments of the self and others.Child Developmnent, 69(2), 391-403.

Edmonds, R. (1979). Effective schools for the urbanpoor. Educational Leadership, 37, 15-27.

Edmonds, R. (1984). School effects and teacher ef-fects. Social Policy, 15, 37-39.

Englert, C. S., Raphael, T. E., Anderson, L. M.,Anthony, H. M., & Stevens, D. D. (1991). Makingstrategies and self-talk visible: Writing instruction inregular and special education classrooms. AmericanEducational Research Journal, 28(2), 337-372.

Ehrenberg, R.,Brewer,D., Gamoran, A., &Willms, D.,(2001). Class size and student achievement. Psycho-logical Science In The Public Interest, 2(1), 1-30.

Evertson, C. M., & Folger, J. K. (1989, March).Small class, large class: What do teachers do dif-ferently? Paper presented at the annual meeting ofthe American Educational Research Association,San Francisco, CA.

Evertson, C. M., & Randoph. (1990). Teaching practicesand class size: A new look at an old issue. PeabodyJoumal Of Education 85-105.

Ferguson, R. F. (1991). Paying for public education:New evidence on how and why money matters.Harvard Journal on Legislation, 28, 465-498.

Finn, J. D., & Achilles, C. (1990). Answers and ques-tions about class size: A statewide experiment. Amer-ican Educational Research Journal, 27, 557-77.

Finn, J. D., & Achilles, C. M. (1999). Tennessee's classsize study: Findings, implications, misconceptions.Educational Evaluation and Policy Analysis, 21(2),97-109.

Folger, J., (1989). Lessons For Class Size Policy AndResearch, Peabody Journal Of Education, 67(1)123-132.

Goldstein, H., & Blatchford, P. (1998) Class size andeducadonal achievement: A review of methodologywith particular reference to study design. BritishEducational Research Journal, 24(3), 255-268.

Hanushek, E. A. (1981). Throwing money at schools.Journal of Policy Analysis and Management, 1,19-41.

Hanushek, E. A. (1989). The impact of differentialexpenditures on school performance. EducationalResearcher, 18(4), 45-65.

Hanushek, E. A., (1996). School resources and studentperformance. In G. Burtless (Ed.), Does money mat-ter2, (pp.) Washington, DC: Brookings Institution.

Hanushek, E. A., (1999) Some findings from an inde-pendent investigation of the Tennessee STAR Ex-

periment, and from other investigations of class sizeeffects. Educational Evaluation and PolicyAnalysis,21(2),143-163.

Hedges, L. V., Laine, R. D., & Greenwald, R. (1994).Does money matter? A meta-analysis of studies ofthe effects of differential school inputs on studentoutcomes (An exchange: Part I). Educational Re-searcher, 23(3), 5-14.

Holland, P. (1986). Statistics and causal inference.Journal of the American Statistical Association,81(396), 945-960.

Jencks, C., Smith, M., Acland, H., Bone, M., Cohen,D., Gintis, H., Heyns, B., & Michelson, S. (1972).Inequality. New York, Basic Books.

Krueger, A. (1999). Experimental effects of educationproduction functions, Quarterly Journal Of Eco-nomics, 114(2).

Lampert, M. M., (2001). Teaching problems and theproblems of teaching. New Haven, CT: Yale Uni-versity Press.

Leinhardt, G., Zigmond, N., & Cooley, W., Readinginstruction and its effects. American EducationalResearch Journal, 18(3), 343-361.

Levin, H., Glass, G., & Meister, G., (1984), Cost-effectiveness of educational interventions. Report#84-Al l, Stanford, CA: Stanford Center For Educa-tional Research.

McLaughlin, M. & Talbert, J. (1999). Professionalcommunity and the work of high school teaching.Chicago, IL: University of Chicago Press.

Mitchell, D., Beach, S., & Badarak, G. (1989). Mod-elling the relationship between -achievement andclass size: A re-analysis Of the Tennessee ProjectStar Data. Peabody Journal Of Education 34-74

Molnar, A., Smith, P. Zahoric, J., Parker, A., Hallbach,A., & Ehrle, K. (1999). Evaluating the SAGE Pro-gram: A pilot program in targeted pupil-teacher re-duction in Wisconsin. Educational Evaluation andPolicy Analysis, 21(2), 165-177.

Mosteller, F. (1995). The Tennessee study of class sizein the early school grades. Future of Children, 5(2),113-127.

Newmann, F. M., & Wehlage, G. G. (1995). Successfilschool restructuring: A report to the public and edu-cators. Center on Organization and Restructuring ofSchools, University of Wisconsin-Madison: U.S. De-partment of Education, OERI. RI 17Q00005-95.

Odden, A, & Busch, C., (1998). Financing schoolsfor high performance: Strategiesfor improving theuse of educational resources. San Fransisco, CA:Jossey-Bass.

Palinscar, A. M., (1986). The role of dialogue in pro-viding scaffolded instruction, Educational Psychol-ogist, 21, 73-98.

Palinscar, A. M., & Brown, A., (1984). Recipro-cal teaching of comprehension-fostering and

n

Cohen, Rautdenbush and Lovenberg Ball

comprehension-monitoring activities, Cognitionand Instruction, 1, 117-175.

Raudenbush, S. W. (1984). Magnitude of teacher ex-pectancy effects on pupil IQ as a function of thecredibility of expectancy induction: A synthesis offindings from 18 experiments. Journal of Educa-tional Psychology, 76(1), 85-97.

Rice, J. (1999). The impact of class size on instruc-tional strategies and the use of time in high schoolmathematics and science courses pp. 215-29.

Ritter, G. W., & Boruch, R. F. (1999). The politicaland institutional origins of a randomized controlledtrial on elementary school class size: Tennessee'sProject STAR. Educational Evaluation and PolicyAnalysis, 21(2), 111-126.

Robins, J. M., Greenland, S., & Hu, F. (1999). Estima-tion of the causal effect of a time-varying exposureon the marginal mean of a repeated binary outcome.Journal of the American Statistical Association,94(447), 687-701.

Rosenholz, S. J. (1985). Effective schools: Interpret-ing the evidence. American Journal of Education,93, 352-388.

Rosenthal, R., & Jacobson, L. (1968). Pygmalionin the classroom. New York: Holt, Rinehart andWinston.

Rubin, D. B. (1974). Estimating causal effects of treat-ments in randomized and nonrandomized studies.Journal of Edutcational Psychology, 66, 688-701.

Rutter, M., Maughan, B., Mortimore, P., Ousten, M., &Smith, A. (1979). Fifteen thousand hours: Secondaryschools and their effects on children. Cambridge,MA: Harvard University Press.

Skrla, L., Scheurich, J. J. (2001). Displacing deficitthinking in school district leadership. Education andUrban Society, 33(3), 235-262.

Slavin, R., (1990). Class size and student achievement:Is smaller better? Contemporary Education, 62(1),6-12.

Slavin, R., (1990). Class size and student achievement:Small effects of small classes. Educational Psychol-ogist, 24, 99-110.

Zahoric, J. (1999). Reduced class size leads to in-dividualized instruction. Educational Leadership,57(1), 50-53.

Authors

DAVID K. COHEN is John Dewey Professor ofEducation, Walter A. Annenberg Professor of PublicPolicy University of Michigan School of Education 610East University, 4109 SEB; DKCohen@umich.edu.His aeras of specialization are education policy, therelations between policy and practice, and schoolimprovement.

STEPHEN W. RAUDENBUSH is Professor ofEducation and Statistics, School of Education, andSenior Research Scientist, Institute For Social Re-search, University of Michigan 610 East University,4109 SEB; His areas of specialization are.

DEBORAH LOWENBERG BALL is ThurnauProfessor of Education, School of Education, Univer-sity of Michigan, 610 East University 4109 SEB;dball@umich.edu; Her areas of specialization areteacher education and mathematics educaton.

Manuscript Received June 20, 2002Revision Received January 20, 2003

Accepted June 4,2003

142