ALLYN & BACON/LONGMANwww.ablongman.com
EDUCATIONAL RESEARCH IN AN AGE OF ACCOUNTABILITY©2007
Robert E. Slavin
ISBN 0-205-43982-9
P R E F A C EThe pages of this Sample Chapter may have slight variations in final published form.
PR
EF
AC
E
Visit www.ablongman.com/replocator to contact your local Allyn & Bacon/Longman representative.
About the Author
Robert Slavin is currently Director of the Center for Data-Driven Reform in Edu-cation at Johns Hopkins University and Chairman of the Success for All Founda-tion. He received his B.A. in psychology from Reed College in 1972, and his Ph.D.
in social relations in 1975 from Johns Hopkins University. Dr. Slavin has authored or co-authored more than 200 articles and 20 books, including Educational Psychology: Theory into Practice (Allyn & Ba-con, 1986, 1988, 1991, 1994, 1997, 2000, 2003); Cooperative Learn-ing: Theory, Research, and Practice (Allyn & Bacon, 1990, 1995); Show Me the Evidence: Proven and Promising Programs for America’s Schools (Corwin, 1998); Effective Programs for Latino Students (Erl-baum, 2000); and One Million Children: Success for All (Corwin, 2001). He received the American Educational Research Association’s Raymond B. Cattell Early Career Award for Programmatic Research in 1986, the Palmer O. Johnson award for the best article in an AERA journal in 1988, the Charles A. Dana award in 1994, the James Bry-ant Conant Award from the Education Commission of the States in 1998, the Outstanding Leadership in Education Award from the Horace Mann League in 1999, the Distinguished Services Award
from the Council of Chief State School Officers in 2000, and the University of Pennsylvania–CPRE Award for Contributions to Educational Research in 2005.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
fm.indd vfm.indd v 8/25/2006 11:19:09 AM8/25/2006 11:19:09 AM
Brief Contents
PART I INTRODUCTIONChapter 1 Educational Research in an Age of Accountability 1
PART II QUANTITATIVE RESEARCH DESIGNSChapter 2 Randomized Experimental Designs 25
Chapter 3 Quasi-Experiments 51
Chapter 4 Time Series Designs 65
Chapter 5 Nonexperimental Quantitative Designs 82
Chapter 6 Survey Research 104
PART III QUALITATIVE AND ACTION RESEARCH DESIGNSChapter 7 Introduction to Qualitative Research 120
Chapter 8 Qualitative Designs 141
Chapter 9 Action Research 162
PART IV PLANNING AND IMPLEMENTING RESEARCHChapter 10 Measurement 173
Chapter 11 Threats to Internal and External Validity 200
Chapter 12 Planning and Implementing the Study 218
PART V DATA ANALYSISChapter 13 Basic Statistics 240
Chapter 14 Intermediate Statistics 271
PART VI READING AND REPORTING RESEARCHChapter 15 Writing Up the Study 294
vii
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
fm.indd viifm.indd vii 8/25/2006 11:19:11 AM8/25/2006 11:19:11 AM
Contents
ix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Features xixPreface xxi
PART I INTRODUCTION
Chapter 1Educational Research in an Age of Accountability 1
Evidence-Based Education 2The Role of Research in Evidence-Based Education 2
Education Policy and Reform 3Accountability 3 ■ Effectiveness 3
What Is Research? 5The Best Possible Answer to the Best Possible Question 7
Types of Research in Education 7Quantitative Research 7 ■ Qualitative Research 8 ■ Other Types of Research 8
Research Design 9The Logic of Research Design 9
Important Elements in Research 10Hypotheses 10 ■ The Gremlin 11 ■ Disproving the Null Hypothesis 14 ■ Theory 14 ■ Statistical Significance 15 ■ False Positive and False Negative Errors 17 ■ Internal Validity 20 ■ External Validity 21
Essentials of Research Design 22
Research Navigator 23Key Terms 23Activity 23
Exercises 23
Further Reading 24
PART II QUANTITATIVE RESEARCH DESIGNS
Chapter 2Randomized Experimental Designs 25
Experimental Comparisons in an Age of Accountability 26
Random Assignment 27Random Assignment of Individuals 28
Stratified Random Assignment 28
Randomized Experimental Comparisons 31Control Groups 32
Intent to Treat 33
Pretesting 34
Experiments with More Than Two Treatments 36Interactions in Factorial Experiments 37
fm.indd ixfm.indd ix 8/25/2006 11:19:11 AM8/25/2006 11:19:11 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Ordinal versus Disordinal Interactions 38
Factorial Designs with More Than Two Factors 41
Alternatives to Random Assignment of Individuals 43Random Assignment of Classes, Schools, and Teachers 44
Delayed Treatment Control Group Designs 45
Within-Teacher Random Assignment 45
Example of an Experiment 48
Research Navigator 49Key Terms 49Activity 49
Exercises 49
Further Reading 50
Chapter 3Quasi-Experiments 51
Conducting Quasi-Experiments 52
Minimizing Selection Bias in Quasi-Experiments 54
Making Comparisons 56Pre–Post Comparisons 57
Successive-Year Comparisons 58
Artificial Control Groups 59
What If Pretests Are Not Equal in Different Treatment Groups? 60
Research Navigator 63Key Terms 63Activity 63
Exercises 63
Further Reading 64
Chapter 4Time Series Designs 65
Single-Case Experiments 67Reversal (ABA) Designs 67
Multiple-Baseline Designs 69
Groups as Single Cases 71
Statistics in Single-Case Designs 73
Limitations of Single-Case Designs 74
Research Navigator 79Key Terms 79Activity 79
Exercises 79
Further Reading 80
Chapter 5Nonexperimental Quantitative Designs 82
Correlational Designs 83Correlation Coefficients 84
Correlational Designs Using Categorical Variables 87
Advantages of Correlational Designs 89
Causal–Comparative Designs 90
Large Correlation Matrices 90
x Contents
fm.indd xfm.indd x 8/25/2006 11:19:23 AM8/25/2006 11:19:23 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Problem of Causation in Correlational Research 91
Mutual Causation 91 ■ Self-Selection Bias 91 ■ Spurious Correlations 92
The Role of Theory in Correlational Research 93
Control Variables 95Multiple Regression 96 ■ Limitations of Control Variables 97
Example of a Correlational Study 99
Descriptive Research 99Survey Research 99
Assessment Research 100
Research Navigator 101Key Terms 101Activity 102
Exercises 102
Further Reading 103
Chapter 6Survey Research 104
Types of Surveys 105Face-to-Face Interviews 106
Telephone Interviews 106
Mail and E-Mail Surveys 106
Creating the Research Instrument 107Constructing Questionnaires 108
Open versus Closed Form 108 ■ Principles of Questionnaire Construction 109 ■ Sociometric Questionnaires 110
Constructing Interviews 111Creating an Interview Protocol 111 ■ Piloting the Interview Protocol 112 ■
Recording Interviews 112 ■ Preventing Bias in Interviews 113 ■ Coding Interview Responses 113
Sampling 114Random Samples 114
Cluster Samples 114 ■ Stratified Random Samples 114
Samples of Convenience 115
Missing Data 116
Research Navigator 118Key Terms 118Activity 118
Exercises 119
Further Reading 119
PART III QUALITATIVE AND ACTION RESEARCH DESIGNS
Chapter 7Introduction to Qualitative Research 120
Characteristics of Qualitative Research 121Qualitative research uses the natural setting as the direct source of data and the researcher as the key instrument 122
Qualitative research is descriptive 123
Qualitative research is concerned with process, rather than simply with outcomes or products 124
Qualitative research includes an inductive analysis of data 124
Meaning is subjective and of essential concern to the qualitative approach 124
Contents xi
fm.indd xifm.indd xi 8/25/2006 11:19:46 AM8/25/2006 11:19:46 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Qualitative researchers are aware of their subjective perspective 125
Steps in Qualitative Research 125Identify What Will Be Studied 126
Identify Who Will Be Studied 126
Collect Data 126
Analyze Data 127
Generate Hypotheses 128
Make Interpretations and Conclusions 129
Data Collection and Qualitative Data 129Collecting Data 129
Naturalistic Observation 130 ■ Open-Ended Interviews 131
Types of Data 132Field Notes 132 ■ Documents and Photographs 133 ■ Statistics 133
Triangulation 133
Qualitative versus Quantitative Research: The Wrong Question 135How They Differ 135
Generalizability versus Depth 135 ■ Hypothesis Testing versus Hypothesis Generation 135
When Are Qualitative Methods Useful? 137
Mixed Methods: Integrating Qualitative and Quantitative Methods 138Qualitative–Quantitative Approach 138
Quantitative–Qualitative Approach 138
Combined Approach 139
Research Navigator 139Key Terms 139Activity 139
Exercises 140
Further Reading 140
Chapter 8Qualitative Designs 141
Different Perspectives, Different Approaches 142
Traditional Approaches 142Ethnography 142
Forms of Ethnographic Data 144 ■ Steps in Ethnographic Research 144 ■ When Is Ethnography Appropriate? 147 ■ Limitations of Ethnography 147
Phenomenology 147Forms of Phenomenological Data 149 ■ Steps in Phenomenological Research 149 ■ When Is Phenomenology Appropriate? 150 ■ Limitations of Phenomenology 150
Case Studies 150Forms of Case Study Data 150 ■ Steps in Case Studies 151 ■ Example of a Qualitative Study 151 ■ When Are Case Studies Appropriate? 152 ■ Limitations of Case Studies 152
Historical Research 154Forms of Historical Data 154 ■ Steps in Historical Research 155 ■ When Is Historical Research Appropriate? 156 ■ Limitations of Historical Research 156
Content Analysis 156Forms of Content Analysis Data 156 ■ Steps in Content Analysis 156 ■ When Is Content Analysis Appropriate? 158 ■ Limitations of Content Analysis 159
Postmodern Research Designs 159Critical Theory 159
Feminist Approach 159
Research Navigator 160Key Terms 160Activity 160
xii Contents
fm.indd xiifm.indd xii 8/25/2006 11:19:59 AM8/25/2006 11:19:59 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises 160
Further Reading 161
Chapter 9Action Research 162
Understanding Action Research 163Action Research versus Traditional Research 164
Types of Action Research 164Practical Action Research 164 ■ Participatory Action Research 165 ■ Political Action Research 165
Conducting Action Research 166Steps in Action Research 167
When Is Action Research Appropriate? 169
Limitations of Action Research 171
Research Navigator 172Key Terms 172Activity 172
Exercises 172
Further Reading 172
PART IV PLANNING AND IMPLEMENTING RESEARCH
Chapter 10Measurement 173
Measures: Reliability and Validity 174
Reliability 174The Importance of Reliability 175 ■ Forming Scales 177 ■ Computing Reliability Coefficients 177 ■ When Is Scale Reliability Adequate? 178
Validity 178Face Validity 179 ■ Content Validity 179 ■ Predictive Validity 182 ■ Concurrent Validity 182 ■ Construct Validity 182
Types of Measures 183Achievement and Aptitude Tests 183
Standardized Tests 184 ■ Criterion-Referenced Tests 185 ■ Authentic Tests 186 ■ Researcher-Made, Content-Specific Tests 186 ■ Questionnaire Scales 188
Behavioral Observation 189
Constructing a Behavioral Observation System 190
Reliability and Bias in Behavioral Observation 191
When Is Reliability Adequate? 193 ■ Increasing Reliability of Behavioral Observations 193 ■ Writing the Observation Manual 194 ■ Reliability for Rating Scales 194
Determining Sample Size 194
When Is Sample Size Adequate? 195
Statistical Significance versus Practical and Theoretical Importance 197
Research Navigator 198Key Terms 198Activity 198
Exercises 199
Further Reading 199
Contents xiii
fm.indd xiiifm.indd xiii 8/25/2006 11:20:01 AM8/25/2006 11:20:01 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 11Threats to Internal and External Validity 200
Threats to Internal Validity 201History 201
Maturation (Passage of Time) 201
Testing Effects 202
Instrumentation Effects 202
Selection Bias 203
Statistical Regression 204
Mortality (Attrition) 205
Confounding Variables 206
Threats to External Validity 207Lack of Internal Validity 209
Nonrepresentativeness 209
Artificiality 211
Reactivity 213Hawthorne Effects 214 ■ John Henry Effects 214
Mistaken Causal Models 214
Research Navigator 216Key Terms 216Activity 216
Exercises 216
Further Reading 217
Chapter 12Planning and Implementing the Study 218
Planning Your Own Study 219
Choosing a Problem 219Criteria for a Good Research Topic 219
Other Considerations in Choosing a Topic 220
Reviewing the Literature 221Gathering Preliminary Information 221
Widely Focused Literature Search 221
Primary Resources for Information Gathering in Education 222
Choosing Search Terms 223
Summarizing Studies 223
Inclusion Criteria 224Germaneness 224 ■ Methodology 224 ■ Other Features 224
Writing the Review 224
Synthesis 224
Methodology 226
Significance 226
Writing Style 226
Meta-Analysis 226
Writing a Proposal 228Elements of a Research Proposal 229
Statement of the Problem 229 ■ Hypotheses 230 ■ Brief Literature Review 230 ■ Procedures 230 ■ Schedule 230
xiv Contents
fm.indd xivfm.indd xiv 8/25/2006 11:20:15 AM8/25/2006 11:20:15 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Roughing Out and Discussing the Proposal 231
Evaluating the Proposal 231
Planning Study Procedures 232Teacher’s Manual 232
Observer’s Manual and Interviewer’s Manual 233
Testing Instructions 233
Research Ethics: Consent, Confidentiality, and Human Subjects Review 234
Human Subjects Review and Informed Consent 234 ■ Confidentiality 234
Gaining Access to Schools and Implementing the Project 235
Implementing the Project 236Maintaining the Integrity of the Research 236
Report Back Your Findings 237
Research Navigator 237Key Terms 238Activity 238
Exercises 238
Further Reading 239
PART V DATA ANALYSIS
Chapter 13Basic Statistics 240
Descriptive Statistics 241Computing Statistics 241
Data Analysis with SPSS 241
Scales of Measurement 243Nominal Scale 243 ■ Ordinal Scale 243 ■ Interval Scale 243 ■ Ratio Scale 244
Measures of Central Tendency 244Mean 244 ■ Median 244 ■ Mode 245
Measures of Dispersion 247Range 247 ■ Standard Deviation and Variance 247
The Normal Curve 249
z-Scores 253Percentile Scores 253 ■ Normal Curve Equivalent 253
Skewed Distributions 254
Kurtosis 255
Standard Error of the Mean 256
Inferential Statistics 256The Null Hypothesis (H0) 257
Criteria for Rejecting the Null Hypothesis 259 ■ Type I and Type II Errors 259 ■ Rejecting the Null Hypothesis 259
One-Tailed and Two-Tailed Tests of Significance 261
t-Test for Comparisons of Two Independent Group Means 263
Computing t 263 ■ Homogeneity of Variances 266 ■ t-Test for Comparisons of Two Means from Matched Groups 267
Research Navigator 269Key Terms 269Activity 269
Exercises 270
Contents xv
fm.indd xvfm.indd xv 8/25/2006 11:20:28 AM8/25/2006 11:20:28 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 14Intermediate Statistics 271
Analysis of Variance (ANOVA) 272Comparison of Three Groups: 3 × 1 ANOVA 272
Two-Factor ANOVA 273
Analysis of Covariance (ANCOVA) 275Analysis of Variance versus Analysis of Covariance 278
Individual Comparisons 278
Effect Size 280
Correlation 281Correlations with Categorical Variables 282
Partial Correlation 284
Linear and Nonlinear Relationships 285
Nonparametric Statistics 286Chi-Square (χ2) 286
Statistics for Reliability 289KR 20 289
Coefficient Alpha 291
Research Navigator 291Key Terms 291Activity 291
Exercises 291
Further Reading 293
PART VI READING AND REPORTING RESEARCH
Chapter 15Writing Up the Study 294
Writing a Thesis or Dissertation 295Format and Style 295
Parts of Your Thesis 295Title Page 296 ■ Acknowledgments 296 ■ Table of Contents and Lists of Tables and Figures 296 ■ Chapter 1: Introduction 296 ■ Chapter 2: Review of the Literature 297 ■ Chapter 3: Methods 297 ■ Chapter 4: Results 299 ■ Chapter 5: Discussion 299 ■ References 300 ■ Appendices 300
Writing a Journal Article or Conference Paper 300Format and Style of Journal Articles and Conference Papers 300
Abstract 301 ■ Introduction 301 ■ Methods 301 ■ Results 301 ■ Discussion 301 ■ References 301 ■ Tables and Figures 301
Choosing a Journal 302
Conference Papers 303
Tips on Getting an Article Published 304
Research Navigator 304Activity 304
Further Reading 304
xvi Contents
fm.indd xvifm.indd xvi 8/25/2006 11:20:36 AM8/25/2006 11:20:36 AM
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
AppendixesAppendix 1 Percentage of Area Lying between the Mean and Successive Standard Deviation Units under the Normal Curve 306
Appendix 2 Critical Values of t 307
Appendix 3 Values of F at the 5% and 1% Significance Levels 308
Appendix 4 Abridged Table of Critical Values for Chi-Square 310
Appendix 5 Table of Random Numbers 311
Appendix 6 Answers to Exercises 312
Appendix 7 Example of an Experimental Article 314
Appendix 8 Example of a Correlational Article 336
Appendix 9 Example of a Qualitative Article 354
Glossary 383
References 389
Index 395
Contents xvii
fm.indd xviifm.indd xvii 8/25/2006 11:20:49 AM8/25/2006 11:20:49 AM
xix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Features
The Savvy Researcher
Randomization 34Evaluating Test Scores 53Avoiding Misleading Analysis 74Explaining Correlations 94Beware of Sampling Bias! 117Triangulation 134The Gremlin and Ethnographic Studies 148The Gremlin and Historical Research 157The Gremlin and Action Research 166Watch for Disadvantages in Measurement 181The Gremlin and Internal Validity 209Beware Artificiality 213Reading Critically 229Finding Significance 266Leave No Covariates Behind! 279
Research with Class
Using Evidence to Support Hunches 6Testing Effectiveness 42Testing a Concept 57Using a Multiple Baseline Design in Class 76Supporting Your Case 98Surveying Parents 107Identifying Needs 131Using Case Study to Inform 153Applying the Concepts 170Checking for Reliability 176Absence of Research 212Using Research for Decision Making 227
fm.indd xixfm.indd xix 8/25/2006 11:20:49 AM8/25/2006 11:20:49 AM
Preface
A fearsome creature stalks the pages of this book. His name is the Gremlin. The Gremlin represents the critic who punches holes in any research design and who finds something wrong in nearly every study. Every student of research in educa-tion fears the Gremlin, whether the Gremlin is a real person or merely a figment of the imagination.
The Gremlin can be tough, but also fair, and it’s possible to meet the Gremlin’s requirements. That’s what this book is about. Educational Research in an Age of Accountability is written to give students of research in education a clear explanation of the logic behind design, analysis, and writing of research, so you can both understand and critically evaluate research done by others and do research yourself that is meaningful to you, valuable to the field, and satisfying to the Gremlin. In this era of standards-based edu-cation and evidence-based practices, teachers are often asked to offer proof of effectiveness for the strategies used in the classroom. Possessing the basic knowledge and skills in educational research is a valuable tool in today’s education climate.
I have a few admissions to make. First, I love this stuff. I hope in writing this book I’ve communicated my enthusiasm for the subject, and I hope you’ll come to share my enthusiasm. To me, research in education is not merely an academic exercise, it’s a
way to improve the practice of education for children. Every educator knows the potential children possess and wants to maximize that potential. A researcher has the opportunity to make a lasting impact by bringing new knowledge to bear on issues that are important to children everywhere. It is endlessly exciting if you do it right. Savvy readers of research can enrich their understanding of effective educa-tion, and protect themselves from misleading claims.
I’ve done a lot of research, using a wide variety of research designs in a wide variety of settings involving children from pre-K to high school. For this reason, I’m going to tell you how research is really done, not how it should be done theoreti-cally. Not everyone who reads this book will become a researcher, but everyone will need to understand research and be able to read it critically. I hope you will learn not only how to satisfy the Gremlin but also how to be the Gremlin, to become a sharp-eyed critic of research able to recognize research that is valid and meaningful from research that is not.
xxi
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
fm.indd xxifm.indd xxi 8/25/2006 11:20:50 AM8/25/2006 11:20:50 AM
xxii Preface www.ablongman.com/slavinresearch1e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
How This Book Is OrganizedThis book is organized somewhat differently from other books on research methods. After an introduction (Part I) follow eight chapters on various research designs: five quantitative (Part II: Quantitative Research Designs), and three qualitative (Part III: Qualitative and Action Research Designs). These designs serve as the core of the book as they present the logic of experimental design, the strengths and limitations of each approach, and the kinds of research problems each is intended to address.
Part IV, Planning and Implementing Research, discusses the practicalities of research: evaluating internal and external validity, measurement, and then setting up a study of your own, which involves choosing a topic, doing literature reviews, and writing research proposals. This is the point at which you may likely be plan-ning your own research, either as an exercise or in preparation for the real thing.
Part V, Data Analysis, is a practical, easy-to-follow guide to basic and interme-diate statistics that focuses on the ideas behind the numbers, not only on compu-tational procedures. Finally, Part VI, Reading and Reporting Research, discusses reading research and writing research reports.
Features in This TextThroughout Educational Research in an Age of Accountability, special features draw the reader’s focus to issues of recurring importance in the field of educational research as well as to aid with review and understanding of key concepts in educa-tional research.
In a time when educators are held accountable for learning outcomes, the in-formal, nontechnical style of the book offers pre- and in-service educators the basic understanding of research needed to understand research findings and conduct studies of their own.
The Gremlin, representing a re-search critic, shows up throughout the book as a personification of the world of critical readers of research, including professors and editors. By learning how to satisfy the Gremlin, stu-dents also learn to ask themselves critical ques-tions as they design their own research and to intelligently respond to others’ research, while learning to become critical readers.
The Savvy Researcher features alert the reader to common errors and misleading uses of research in education, so that students will become critical consumers of research.
■
■
■
Measures: Reliability and Validity 181. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . .
The Savvy Researcher
Watch for Disadvantages in Measurement
W hen reading experimental research, the Gremlin is always on the lookout for measures that may give an advantage to the experimental group. For example, imagine a month-long study in which the ex-
perimental group completed elaborate projects relating to the cultures of South America, while the control group experienced a traditional survey course on world cultures. The posttest focused entirely on South America.
“Preposterous!” fumes the Gremlin. “Such a study will be seriously biased to-ward the experimental group, which presumably spent much more time on South America during the study month!”
Studies often report results of assessments used just to check to see that the treatments were implemented. For example, a reading comprehension program emphasizing teaching students to summarize might use a measure of summariza-tion skill (when the control group did not learn to summarize). There is nothing wrong with this if the implementation measure (summarization, in this case) is clearly labeled as such and a measure that is fair to both groups (e.g., a compre-hension measure) is the only outcome that matters. Yet all too often, researchers report so-called positive effects of treatments on outcomes that only assess expo-sure to particular content, not better teaching strategies.
The Gremlin says “Keep your eyes open for this common error!”
fm.indd xxiifm.indd xxii 8/25/2006 11:20:50 AM8/25/2006 11:20:50 AM
. . Preface xxiii
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Research with Class features contain appli-cations of the principles of research in each chapter to studies that could be done in a single class or school, answering questions practicing educators might ask. Each feature demon-strates how educators with limited resources can do research in their own settings. They also bring the principles of research down to earth by showing how they can be used to shed light on realistic questions that educators face.
■
Traditional Approaches 153. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Research with Class
Using Case Study to Inform
Joan Bukowski, a kindergarten teacher, read several articles indicating that sociodramatic play serves an important role in young children’s language de-
velopment (e.g., Cooper & Dever, 2001; Mellou, 1994). Although she had always had a dramatic play center in her classroom, she had not given much thought to how children really benefited from it.
Joan decided to conduct a case study of the dramatic play center in her class. She filled out a checklist of which children played in the dramatic play center. She conducted running-record observations of the children’s behavior in the dramatic play center, recording the different kinds of behaviors they engaged in while in the center. She interviewed the children about what they liked to do in the dramatic play center, what materials they used the most, and what additional materials they would like in the center.
Joan’s observations and interviews indicated that it was mostly the girls in her class who played in the dramatic play center. Based on these data, Joan added materials that she thought would attract the boys to the center, including Hoover, the class hamster, a big favorite with the boys. She continued her obser-
i d f d h h b i h l b l i h d i l
................................................................
The final chapter, Writing Up the Study, focuses on writing master’s theses and other research reports.
Actual research articles, annotated with notes that highlight critical issues, are included in the book appendixes to assist readers’ understanding and critical thinking skills.
Strong pedagogical aids help students retain and test their understanding of key concepts. The text is filled with both real examples and “thought experi-ments” intended to link key concepts of research to students’ own experiences. In addition, aids include margin glossary definitions and review exercises at the end of each chapter intended to help students think about and apply what they have learned in simulated research contexts. The statistics chapters (13 and 14) have specific answers, which are provided in the appendices section at the back of the book.
Throughout the text, cartoons drawn by my colleague James Bravo take the technical subject of research and shine a humorous light on it, while encouraging the reader to think about some key concepts of research methods and to serve as mnemonics for some of the vocabulary of research.
Special activities using Research Navigator are also included at the end of every chapter. This powerful research tool allows readers to in vestigate key con-cepts from the book using a collection of resources available online at www.researchnavigator.com, in-cluding EBSCO’s ContentSelect Academic Journal Database and the New York Times. Purchase of this book allows you free access to this exclusive pool of information and data. A personal code and access in-structions are included on the inside cover of this book.
■
■
■
■
■
238 Chapter 12 Planning and Implementing the Study www.ablongman.com/slavinresearch1e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
RESEARCH NAVIGATOR
best-evidence synthesis 228Family Educational Rights and Privacy Act 235
human subjects review 234meta-analysis 226
Key Terms
If you have access to Research Navigator, located on the MyLabSchool website (www.mylabschool.com) enter the following keywords to find articles related to the content of this chapter:
Literature reviewsMeta-analysisResearch ethicsHuman subjectsInstitutional review boardFamily Educational Rights and Privacy ActInformed consentConfidentiality
Activity
E X E R C I S E S
1. Which of the following would be a good topic for study by a graduate student? What are the positive and negative features of each?
2. In the Review of Educational Research, find and critique a review in an area that interests you.
fm.indd xxiiifm.indd xxiii 8/25/2006 11:20:52 AM8/25/2006 11:20:52 AM
xxiv Preface www.ablongman.com/slavinresearch1e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
USING RESEARCH NAVIGATORTM This edition of Educational Research in an Age of Accountability is designed to integrate the content of the book with the valuable research tool, Research NavigatorTM, a collection of research databases, instruction, and contemporary publications available to you online at www.researchnavigator.com.
At the end of every chapter a Research NavigatorTM section prompts the reader to use key terms in each chapter to expand on the concepts of the text and further explore the work being done throughout the field of education. To gain access to Research NavigatorTM, go to www.researchnavigator.com and log in using the passcode you’ll find on the inside front cover of your text. Research NavigatorTM learning aids include the following components:
EBSCO’s ContentSelect Academic Journal Database EBSCO’s ContentSelect Academic Journal Database contains scholarly, peer-reviewed journals. These published articles provide you with specialized knowledge and informa-tion about your research topic. Academic journal articles adhere to strict scientific guidelines for methodology and theoretical grounding. The information obtained in these individual articles is more scientific than information you would find in a popular magazine, newspaper article, or on a web page.
The New York Times Search by Subject Archive Newspapers are consid-ered periodicals because they are issued in regular installments (e.g., daily, weekly, monthly), and provide contemporary information. Information in periodicals may be useful, or even critical, for finding up-to-date material or information to support specific aspects of your topic. Research NavigatorTM gives you access to a one-year, “search by subject” archive of articles from one of the world’s leading newspapers—The New York Times.
“Best of the Web” Link Library Link Library, the third database included with Research NavigatorTM, is a collection of web links, organized by academic subject and key terms. Searching on your key terms will provide you a list of five to seven editorially reviewed websites that offer educationally relevant and reliable content. The web links in Link Library are monitored and updated each week, reducing your incidence of finding “dead” links.
In addition, Research NavigatorTM includes the following extensive online con-tent detailing the steps in the research process:
Starting the Research Process ■ Internet Research
Finding and Evaluating Sources ■ Using Your Library
Citing Sources ■ Starting to Write
For more information on how to use Research NavigatorTM go to www.ablongman.com/aboutrn.com.
■
■
■
fm.indd xxivfm.indd xxiv 8/25/2006 11:20:57 AM8/25/2006 11:20:57 AM
. . Preface xxv
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TEXT SUPPLEMENTSFor the Instructor
Instructor’s Manual with Test Items. Compiled by Elizabeth Dore of Radford University, the Instructor’s Manual contains chapter overviews, annotated lec-ture outlines, suggested readings and media, and a complete set of assessment items.
Computerized Test Bank. The printed Test Bank is also available electroni-cally through our computerized testing system, TestGen EQ. Instructors can use TestGen EQ to create exams in minutes by selecting from the existing database of questions, editing questions, or writing original questions. The test bank contains a variety of testing items including multiple choice, true/false, short-answer, two levels of essay questions (conceptual and reflective), and con-cept integration items that ask students to apply a combination of concepts and principles to a written teaching scenario. The Computerized Test Bank is available to adopters by contacting their local representative.
PowerPointTM Presentation. Created by Julia Harper of Azusa Pacific Univer-sity, robust PowerPointTM presentations create an active learning environment that will be like sitting in a lecture hall. Each chapter includes a voiceover discussion created by the text author for corresponding slides, as well as various assessments and in-depth examples. Available for download from the Instructor Resource Center site by contacting your local representative.
MyLabSchool. Discover where the class-room comes to life! From videoclips of teachers and students interacting to sample lessons, portfolio templates, and standards integration, Allyn and Bacon brings your students the tools they’ll need to succeed in the classroom, with content easily integrated into your courses. Delivered as an access protected website or within Course Compass, Allyn and Bacon’s course management system, this program gives your students powerful insights into how real class-rooms work and a rich array of tools that will support them on their journey from their first class to their first classroom. MyLabSchool also includes a direct connection to Research NavigatorTM.
For the Student
Companion Website (www.ablongman.com/slavinresearch1e). Developed by Emilie Johnson, Lindenwood University, this rich site is organized by chap-ter and provides a full complement of study and review resources that supple-ment the general text.
Student Activity Manual. Written by Lisa M. Abrams and Richard Mohn of Virginia Commonwealth University, the Student Activity Manual includes
■
■
■
■
■
■
fm.indd xxvfm.indd xxv 8/25/2006 11:20:58 AM8/25/2006 11:20:58 AM
xxvi Preface www.ablongman.com/slavinresearch1e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
brief chapter overviews with key concepts, outlines designed to facilitate note taking, group and individual activities aligned with the key concepts, web-based resources, and an answer key at the end of the manual.
AcknowledgmentsI would like to acknowledge the assistance of several people in writing this book, beginning with my colleague Bette Chambers, who helped with Chapters 7, 8, and 9 and reviewed other chapters, as did Anne Chamberlain and Nancy Madden. The talented James Bravo drew the cartoons, and Susan Davis worked tirelessly on inputting the manuscript. I also thank the writers of the supplements: Lisa M. Abrams and Richard Mohn of Virginia Commonwealth University (Student Activ-ity Manual), Emilie Johnson of Lindenwood University (Companion Website), Julia Harper of Azusa Pacific University (PowerPoint presentation), and Elizabeth Dore of Radford University (Instructor’s Manual with Test Items).
I also wish to thank my colleagues who served as reviewers. Reviewers’ com-ments provided invaluable information that helped us identify the needs in the market and shape the text to meet those needs. Reviewers include Robert DiGi-ulio, Johnson State College; Elizabeth Dore, Radford University; David Gilman, Indiana State University; Tracy Irani, University of Florida; Ernest Johnson, The University of Texas at Arlington; Dianna Newman, State University of New York at Albany; and Edyth Wheeler, Towson University.
Finally, I’d particularly like to acknowledge my Allyn and Bacon partners in this endeavor, beginning with senior editor Arnie Burvikovs, senior development editor Mary Kriener, and editorial assistant Erin Reilly. I am also grateful to market-ing manager Erica DeLuca, editorial-production administrator Annette Joseph, and photo editor Annie Pickert.
fm.indd xxvifm.indd xxvi 8/25/2006 11:21:02 AM8/25/2006 11:21:02 AM
