Date post: | 07-Apr-2018 |
Category: |
Documents |
Upload: | anonymous-sewu7e6 |
View: | 214 times |
Download: | 0 times |
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 1/218
A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES
TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH
SCHOOLS IN TEXAS
ARTHUR L. PETTERWAY
Submitted to the Graduate School
Prairie View A&M University in partial fulfillment of the requirements for the degree of
DOCTOR OF PHILOSOPHY
May, 2007
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 2/218
Dr. Will itsonis
Dr. Camille ibsonr. David Herrington
Fr. . Paul M
Dean, The Whitlowe R. Green
I
. Dou las HermondH O Aid- ie.
c k _
e of Education
A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES
TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH
SCHOOLS IN TEXAS
A Dissertation
by
ARTHUR L. PETTERWAY
Approved as to style and content by:
L. M. Paul Mehta, Chair
Dr. William Parker
Dean, Graduate School
May 2007
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 3/218
iii
ABSTRACT
A Mixed - Method Analysis of the Impact of High Stakes Testing on
English Language Learners in Major Urban High Schools in Texas
April, 2007
Arthur L. Petterway: B.A. – Dillard University
M.Ed., Prairie View A&M University
Dissertation Chair: Dr. M. Paul Mehta
Ample research has been conducted on the intrinsic validity of
standardized assessments, and on the factors affecting the assimilation
and integration of English language learners (ELLs). The reliability of
these assessments as a universal tool to measure student learning, and
as a basis for determining school performance needed closer
examination.
The purpose of this study was to determine the impact of high-
stakes testing on ELLs. This was shown in both the quantitative and
qualitative dimensions of the study. Data obtained from Texas Education
Agency (TEA) were used to determine whether there was a relationship
between the percentage of ELLs enrolled in a school and the percentage
of all students passing the 10th grade Texas Assessment of Knowledge
and Skills (TAKS) tests in the core areas of English Language Arts and
Mathematics given in 2003, 2004, 2005, and 2006. The qualitative
aspect of this study explored what certified English as a Second
Language( ESL) teachers, non-certified ESL teachers who teach ELLs,
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 4/218
iv
administrators, and district ESL personnel viewed as the impact that
high stakes standardized assessments had on ELLs, ESL curriculum,
and instruction in ESL classrooms.
This study determined the impact of high-stakes testing on ELLs
using the explanatory design of mixed method analysis. Data of 173
major urban high schools obtained from the Texas Education Agency
(TEA). It was determined through the Pearson correlation computations
using the Statistical Package for the Social Sciences (SPSS) that there
was a significant relationship between the percent of ELLs enrolled in a
school and the percent of all students passing the 10th Grade TAKS tests
in English Language Arts and Mathematics. In the qualitative portion of
the study, the views and opinions of district ESL personnel were
gathered. Principals, assistant principals, ESL and non-ESL teachers
took part in an on-line, open-ended questionnaire; one-on-one
interviews; and focus groups. The focus groups addressd the purposes
of statewide testing; its intended consequences; problems and changes
created by TAKS, and the recommendations to improve ESL curriculum
and instruction.
The results of the study affirmed the expected outcome that a
significant relationship existed between the percentage of ELLs enrolled
in a school and the percentage of all students passing the 10th grade
TAKS tests in both core areas of English Language Arts and
Mathematics. The regression analysis predicted that as the percentage of
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 5/218
v
ELLs in a school increased, the performance on the statewide, high-
stakes testing in terms of all students passing the 10th grade TAKS tests
decreased. Respondents of the study considered TAKS as a tool to gauge
knowledge in the different core areas. English language learners were
expected to have at least average scores on TAKS. There was a difference
in the expected and actual results; respondents observed dismal or
failing performance of ELLS in the actual results in TAKS. This was
evident by the high failure rate of ELLs in their respective schools. Higher
dropout rate and lower graduation rate of ELLs were problems
encountered due to TAKS. Respondents favored a different test for ELLs,
possibly given at a later date after ELLs had studied in the country for at
least several years. Respondents believed that interventions were needed
to help ELLs perform better. Both the school and the home, together with
the community, have to be involved in preparing ELLs for their present
and future roles in the American society.
Results of this study may provide valuable data to district and
school administrators to develop strategies that will improve the
performance of ELLs on the statewide, high-stakes testing and to develop
assessments that truly measure learning without the nullifying effect of
linguistic and cultural bias. The study may also help to enhance the
reliability of standardized assessments as a tool to determine
accountability for student performance.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 6/218
vi
DEDICATION
This dissertation is dedicated in humble gratitude to my Lord and
Savior, Jesus Christ, in whom I move, trust, and have my being.
Through HIS divine wisdom and purpose, HE gave me my parents:
Bob Stevenson Petterway
November 23, 1923-September 28, 1992
and
Myrtice Lee Petterway
February 10, 1927-February 15, 1959
They are now in Glory with HIM sharing this divine blessing that HE has
bestowed upon me.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 7/218
vii
ACKNOWLEDGEMENTS
I would like to acknowledge those without whom this work would
not have been possible. First and foremost I wish to acknowledge the
blessings bestowed upon me by my Lord and Savior Jesus Christ.
Among these blessings are the kind souls I mention here.
First, I would like to acknowledge my dissertation committee,
Dr. M. Paul Mehta, Dr. William A. Kritsonis, Dr. Douglas Hermond,
Dr. David Herrington, and Dr.Camille Gibson. They have been thorough,
fair, understanding, demanding, and, most of all, dedicated to academic
excellence in all phases of this work. I would especially like to thank
Dr. M. Paul Mehta for taking time, along with his duties as the Dean of
Education, and carrying the baton of being my committee chairman and
Dr. Robert Marshall, my former committee chairman (who would not let
Me fall off of my bicycle and taught me how to eat an elephant) for the
long hours they have spent and the endless patience they have shown as
they have guided me through this endeavor. I also wish to thank
Dr. Kritsonis for his passion for making sure that I get published before I
receive my PhD and taking on the duties of head cheerleader for this
project. Thanks goes to Dr. Hermond for serving as lead statistician and
for not allowing me to attempt to eat the whole pie and limiting me to a
small slice. Many thanks also to Dr. Herrington who guided me in tying
up several loose ends. I am also grateful to Dr. Gibson for taking time out
of her very busy schedule to offer her support and encouragement.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 8/218
viii
Additional thanks go to Dr. William Parker, Dean of Graduate Studies,
for believing in me and giving me the opportunity to prove myself
beginning with my pursuit of my Masters Degree.
I also gratefully acknowledge all of my professors and thank them
for the wisdom and knowledge they have so generously shared. I would
further like to thank all of the faculty and staff of Prairie View A&M
University who have contributed to my achievement in countless ways. I
also wish to acknowledge the unwavering encouragement of my student
cohort as we shared the joy and pain of this incredibly challenging
pursuit.
I would like to thank my principal, Mrs. Linda Llorente, for her
understanding and support as I pursued this dream. She has been
abundantly patient and understanding of the demands this work has
placed on my time and energy. I also wish to acknowledge the
contributions of my peers and colleagues at Austin High School.
Although it is virtually impossible to name all who have contributed to
the completion of this work, I feel that there are several who must be
thanked by name. I would like to thank Andy Lamboso and Rhodora
Maligad who helped with proofreading and typing along with Kathy Koch,
Betty Shaw, Debbie Kubiak, and Raul Asoy who helped with proofreading
and editing.
Finally, I wish to acknowledge the precious prayers offered by the
righteous to strengthen and uphold me through this challenging time,
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 9/218
ix
with a special thanks to my most avid prayer warrior, my aunt, Mrs. Sina
Gunnels, and her chicken with a snuff cup under its wing.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 10/218
ix
TABLE OF CONTENTS
Page
ABSTRACT ..................................................................................... iii
DEDICATION ................................................................................. vi
ACKNOWLEDGEMENT .................................................................. vii
TABLE OF CONTENTS ................................................................... ix
LIST OF TABLES .......................................................................... xiv
CHAPTER I. INTRODUCTION .......................................................... 1
Statement of the Problem....................................................... 3
Purpose of the Study ............................................................. 6
Research Questions ............................................................... 7
Quantitative .................................................................. 7
Null Hypothesis One ...................................................... 7
Null Hypothesis Two ...................................................... 7
Qualitative .................................................................... 7
Description of the Research Design ........................................ 8
Assumptions ......................................................................... 9
Limitations of the Study ........................................................ 9
Delimitations of the Study .................................................... 10
Definition of Terms ............................................................... 10
Significance of the Study ...................................................... 12
Organization of Study ........................................................... 13
CHAPTER II. REVIEW OF LITERATURE ......................................... 15
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 11/218
x
Page
No Child Left Behind (NCLB) ................................................. 15
Historical Perspective .................................................. 15
Description of the Key Factors ..................................... 18
Expectations for Parents .............................................. 20
Response to NCLB ....................................................... 21
Adequate Yearly Progress (AYP) ............................................. 22
Purpose and Support to NCLB ..................................... 22
Changes and Updates.................................................. 24
AYP and Limited English Proficient (LEP) Students ................ 30
Definition of English Language Learners (ELLs) ........... 30
Issues and Other Considerations of LEP ....................... 34
High Stakes/Statewide Testing ............................................. 43
Principles of Testing Programs ..................................... 44
Accountability in Testing ............................................. 50
Effects of High Stakes Testing on Student Motivation .... 52
Other Considerations of Assessment on Testing ............ 56
Related Studies .................................................................... 59
Summary ............................................................................. 61
CHAPTER III.METHODOLOGY. ...................................................... 63
Introduction .................................................................................. 63
Research Questions .............................................................. 64
Quantitative ................................................................ 64
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 12/218
xi
Page
Null Hypothesis One .................................................... 64
Null Hypotheses Two ................................................... 65
Qualitative .................................................................. 65
Research Methods ................................................................ 66
Research Design................................................................... 67
Quantitative ................................................................ 68
Qualitative .................................................................. 68
Pilot Study ........................................................................... 68
Population and Sample ......................................................... 70
Quantitative ................................................................ 70
Qualitative ................................................................. 71
Instrumentation ................................................................... 72
Instruments ................................................................ 72
Validity ....................................................................... 74
Reliability ................................................................... 74
Research Procedures ............................................................ 75
Quantitative ................................................................ 75
Qualitative ................................................................. 75
Data Collection and Recording .............................................. 76
Quantitative ................................................................ 76
Qualitative ................................................................. 76
Data Analysis ....................................................................... 78
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 13/218
xii
Page
Quantitative ................................................................ 78
Qualitative .................................................................. 79
Summary ............................................................................. 80
CHAPTER IV ANALYSIS OF DATA .................................................. 83
Findings .............................................................................. 85
Quantitative Research Question .................................... 85
Null Hypothesis One ..................................................... 86
Null Hypothesis Two ..................................................... 86
Qualitative Research Question ..................................... 100
Discussion ........................................................................ 141
Summary .......................................................................... 145
CHAPTER V SUMMARY, CONCLUSION, IMPLICATIONS AND
RECOMMENDATIONS ................................................................. 147
Summary .......................................................................... 147
Conclusions ...................................................................... 150
Implications ...................................................................... 151
Recommendations for Further Study .................................. 154
REFERENCES ............................................................................. 157
APPENDICES .............................................................................. 177
Appendix A IRB .................................................................. 178
Appendix B Consent Form .................................................. 183
Appendix C Interview Questions .......................................... 186
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 14/218
xiii
Page
Appendix D On-Line Questionnaire ..................................... 191
Appendix E Letter to Participants ........................................ 196
Appendix F Request for Extant Data from T.E.A... ................ 198
VITA ........................................................................................... 201
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 15/218
xiv
LIST OF TABLES
Table
4.1.1 Means and Standard Deviations of ELLs Enrolled in School
and All Students Passing the 2003 10th Grade TAKS for
English Language Arts and Mathematics .............................. 87
4.1.2 Means and Standard Deviations of ELLs Enrolled in School
and All Students Passing the 2004 10th Grade TAKS for
English Language Arts and Mathematics ............................... 87
4.1.3 Means and Standard Deviations of ELLs Enrolled in School
and All Students Passing the 2005 10th Grade TAKS for
English Language Arts and Mathematics ............................... 88
4.1.4 Means and Standard Deviations of ELLs Enrolled in School
and All Students Passing the 2006 10th Grade TAKS for
English Language Arts and Mathematics ............................... 88
4.1.1.1 Comparison of Results in 10 th Grade English Language
Arts TAKS ......................................................................... 89
4.1.1.2 Comparison of Results in 10 th Grade Mathematics TAKS...... 90
4.2.1 Pearson Correlation: 2003 10th Grade TAKS for English
Language Arts and Mathematics............................................ 91
4.2.2 Pearson Correlation: 2004 10th Grade TAKS for English
Language Arts and Mathematics............................................ 91
4.2.3 Pearson Correlation: 2005 10th Grade TAKS for English
Language Arts and Mathematics............................................ 92
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 16/218
xv
Table
4.2.4 Pearson Correlation: 2006 10th Grade TAKS for English
Language Arts and Mathematics............................................ 92
4.2.5 Coefficients for Percentage of All Students Passing the
2003 10th Grade English Language Arts TAKS ........................ 93
4.2.6 Coefficients for Percentage of All Students Passing the
2003 10th Grade Mathematics TAKS ...................................... 94
4.2.7 Coefficients for Percentage of All Students Passing the
2004 10th Grade English Language Arts TAKS ........................ 95
4.2.8 Coefficients for Percentage of All Students Passing the
2004 10th Grade Mathematics TAKS ...................................... 96
4.2.9 Coefficients for Percentage of All Students Passing the
2005 10th Grade English Language Arts TAKS ........................ 97
4.2.10 Coefficients for Percentage of All Students Passing the
2005 10th Grade Mathematics TAKS ...................................... 98
4.2.11 Coefficients for Percentage of All Students Passing the
2006 10th Grade English Language Arts TAKS ........................ 99
4.2.12 Coefficients for Percentage of All Students Passing the
2006 10th Grade Mathematics TAKS .................................... 100
4.3 Distribution of Respondents by Gender .................................. 102
4.4 Distribution of Respondents by Age ........................................ 102
4.5 Distribution of Respondents by Professional Position ............... 103
4.6 Distribution of Respondents by Highest Degree Earned ........... 103
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 17/218
xvi
Table
4.7 Distribution of Respondents by Years of Experience in
Education ............................................................................. 104
4.8 Distribution of Respondents by Certifications Held .................. 105
4.9 Why is TAKS Given as a Statewide Test to ELLs? .................... 107
4.10 What are the Anticipated Results of Statewide Testing
for ELLs? ............................................................................ 111
4.11 What are the Actual Results of Statewide Testing
for ELLs? ............................................................................ 115
4.12 What are the Intended Consequences of TAKS for ELLs? ....... 118
4.13 What Has Happened to ELLs Because of TAKS? .................. 122
4.14 What Problems Have Occurred for ELLs Due to TAKS? ......... 126
4.15 What Changes Have Occurred for ELLs Due to TAKS? .......... 129
4.16 What Recommendations are suggested for Improvement
of ELLs Performance on TAKS?............................................. 132
4.17 What are the Recommendations, with Greatest Value,
are offered for ELLs Success on TAKS? ................................. 136
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 18/218
1
CHAPTER I
INTRODUCTION
For years, the English language learners (ELLs) have been
subjected to educational systems that did not expect them to rise to the
same standards as their native English-speaking peers (Winograd, 2002).
Although that it can take several years to acquire the second language
skills needed to be successful in school (Collier, 1989), too often English
language learners born in the U.S. are still in English as a second
language (ESL) classes and far behind their grade level peers in the
content areas by the time they reach high school (Freeman & Freeman,
2002).
One factor that should be considered in this failure to reach grade
level requirements is that language may constitute an element of self-
identity. It is possible that minority groups are insistent on retaining
their ethnic language as their “first.” English proficiency then would be a
mere elective instead of an indispensable learning tool. If this is the case,
schools are being held accountable for the consequences of a socio-
cultural phenomenon that is beyond their limited powers to address.
Public schools are under close scrutiny. Since they are supported
by public funds, there is an increasing demand for accountability. The
No Child Left Behind Act (NCLB) now requires all students to be
accounted for in any state‟s assessment system, although that has not
always been the case (Abedi, 2004). School districts are now required to
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 19/218
2
clearly demonstrate they deserve, and effectively utilize public funding. In
itself, this is not a disturbing trend. Institutions that are wholly or partly
supported by public funds should be accountable. This is essentially a
consequence of democracy. A government that is created by, and for the
people, is so unlike an aristocracy that is not required to serve a
constituency beyond the guarantee of protection from marauders or
invading armies. The U.S. system of government empowers the state to
undertake measures that guarantee the common good. This goes beyond
the guarantee of physical safety, since the term “common good” has a
wider application, and implies a calculated sensitivity to every citizen‟s
pursuit of happiness. While education is not categorized as a
fundamental right, it is perceived as primary among a bundle of values
essential for every person‟s quest for self-fulfillment and happiness. This
explains why there is little argument about whether the government
should be involved in education at all, and whether this is an endeavor
better left to the private sector (Abedi, 2004).
The government‟s involvement in education opens a wide avenue
for the analysis and evaluation of results. In today‟s world, it is not
enough that public schools have adequate facilities, although this
constitutes one level of analysis. It is important that schools are safe and
teachers are qualified, although in the hierarchy of priorities considered
for evaluating schools, these outcomes are not standard. Schools are
judged principally based on the amount of learning that takes place in
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 20/218
3
their classrooms. As an internal act, the evidence of learning is analyzed
from scores students obtain on standardized assessments.
Institutions are now facing an ever-increasing demand for
accountability. There is pressure from every conceivable corner to make
public schools accountable to their stakeholders. This means that it is
not enough for students to learn in school; it is equally important that
learning should occur in ways that are measurable. If students are
unable to demonstrate what they have learned, it is presumed that no
learning took place at all. The time when public schools are allowed to
operate without proven success is over. It is appropriate to inquire about
the valid manifestations of success and learning, and how they may
actually be measured. Cultural construct renders school rankings flawed
to a certain extent since they become less accurate as a measure of the
faculty and administration‟s performance. Instead, they become
unintended indicators of the ethnicity of the students to which schools
cater (Abedi, 2004).
Statement of the Problem
High stakes assessment systems are meant to bring attention to
the needs of ELLs, who are most at risk of not reaching the educational
goals set for them (Anderson, 2004). But what results do statewide
accountability tests really produce for ELLs (Anderson, 2004)?
Assessment systems usually produce both positive and negative
consequences (Anderson, 2004). The positive and negative consequences
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 21/218
4
of assessments are what is called „washback‟ (Alderson & Wall, 1993), or
how the results of an assessment affect the stakeholders taking the test
(Anderson, 2004).
While quantifiable washback effects such as increased dropout
rates or increased referral to Special Education have been researched,
assessment washback is more complicated than numbers alone can tell
(Anderson, 2004). Students who qualify for Special Education may be
allowed to take alternative assessments in lieu of the state assessments
such as the Texas Assessment of Knowledge and Skills (TAKS). It is
interesting to note that while the numbers of African-American and
Hispanic students are over-represented in Special Education, about eight
to nine percent of ELLs are identified as receiving Special Education
services in the United States (D‟Emilio, 2003; Zehler, Fleischman,
Hopstock, Pendzick, & Stepherson, 2003). While these assessments are
not on grade level, schools are expected to demonstrate that, based on
students‟ scores on alternative assessments, improvement in academic
performance is taking place.
Data are needed that tell us more about the full range of intended
and unintended consequences occurring in schools today (Anderson,
2004). Since school rankings affect student and faculty morale, they
serve more as a force for the preservation of the status quo than a force
for improvement in student performance. A school that works hard to
ensure that learning occurs, and that its students progress academically,
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 22/218
5
but which has a large proportion of ELLs, will risk being ranked as
underperforming because the measure used to evaluate its performance
is blind to this important demographic reality.
One way to get at these data is by talking with the stakeholders at
the schools. Educators are the ones who deal directly with the impact of
high stakes assessments, but are overlooked in research. While teachers‟
opinions are often cited as anecdotal evidence that a problem exists,
their expert observations often go unrecorded in any systematic way
(Anderson, 2004).
Standardized assessments are a measure for holding schools
accountable for student learning. At the present time, schools in Texas
are ranked Exemplary, Recognized, Acceptable or Underperforming,
depending on the performance of their students in the Texas Assessment
of Knowledge Skills (TAKS). This produces a vicious cycle since
exemplary schools attract the best students who may leave
underperforming schools to seek what is perceived to be a higher quality
of instruction in higher ranked schools. These labels tend to have a self-
fulfilling effect, or at least they make it difficult for underperforming
schools to achieve higher performance scores on standardized tests,
since they face the additional burden of surmounting language barriers
and a history of low performance.
Related to this concern is the prevailing system of voluntary
segregation in most zones and districts. Some schools have either a
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 23/218
6
predominant population of White, Hispanic, or African-American
students. Each of these student groups is given the same tests, and yet
they have varying degrees of proficiency in the language in which the
assessments are given. It begs to be asked whether these assessments,
in fact, measure learning and whether they are linguistically and
culturally neutral. The implication is that these students will be able to
answer the test questions even if they do not have equal exposure to
cultural references that may frame some of the test questions.
This study is intended to explore what educators perceive as the
consequences of statewide assessment for ELLs and what they observe
as actually occurring (Anderson, 2004).
Purpose of the Study
The purpose of this study was to determine the impact of high-
stakes testing on ELLs. This was shown in both the quantitative and
qualitative dimensions of the study. Data obtained from TEA were used
to determine whether there is a relationship between the percentage of
English language learners enrolled in a school and the percentage of all
students passing the 10th Grade TAKS tests in the core areas of English
Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To
support the quantitative aspect, this study explored what certified ESL
teachers, non-certified ESL teachers who teach ELLs, administrators,
and district ESL personnel viewed as the impact that high stakes
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 24/218
7
standardized assessments have on ELLs, ESL curriculum and
instruction, and what they observed as actually occurring.
Research Questions
Quantitative
Is there a relationship between the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005, and 2006?
Hypotheses
H01: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
English Language Arts given in 2003, 2004, 2005, and 2006?
H02: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
Mathematics given in 2003, 2004, 2005, and 2006?
Qualitative
The major question addressed by this study was: What are the
anticipated and observed consequences of the statewide testing,
specifically TAKS, on ESL curriculum and instruction as viewed by
certified ESL teachers, non-certified ESL teachers who teach ELLs,
school administrators, and district ESL personnel?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 25/218
8
This major question was explored using the following probes:
1. Why is TAKS given as a statewide test?
2. What are the intended consequences of this statewide testing? (Or
what has happened because of TAKS?)
3. What problems have occurred related to or because of TAKS?
4. What changes were caused by this statewide testing?
5. What are your recommendations to improve this statewide testing?
6. What needs to be done for the ESL students to improve their
performance in general and specifically for this statewide test?
Description of the Research Design
The study analyzed the issues and challenges faced by ELLs and
the public schools that serve them. Quantitative data for this research
were gathered from the Texas Education Agency (TEA) regarding the
percentage of ELLs and the performance of 10th grade students from the
major urban high schools in Texas on the Texas Assessment of
Knowledge and Skills (TAKS) tests in English Language Arts and
Mathematics for 2003, 2004, 2005, and 2006. Qualitative data were
derived from one-on-one and focus group interviews and an on-line
questionnaire focusing on respondents‟ views and opinions about the
various ways that standardized assessments impact ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 26/218
9
Assumptions
Fraenkel and Wallen (2003) stated that an assumption is anything
taken for granted rather than tested or checked. This study is no
different and the following assumptions were made: (a) that the first
language of the ELLs is Spanish and they have varying degrees of fluency
in the English language; (b) that the ESL curriculum is appropriate for
the mastery of the TAKS test for the ELLs; (c) that the on-line open-ended
qualitative questionnaire will be completed by the respondents on time;
and (d) that the respondents in the focus groups will truthfully express
their views and opinions regarding issues or concerns brought to the
group.
Limitations of the Study
Limitations of the study included several factors: mainly the
qualitative questionnaire and the manner in which respondents gave
their responses. The questionnaire may have vague questions open to
more than one interpretation. The pilot study helped in streamlining the
questionnaire to remove or modify such vague issues or concerns.
Another limitation may have been the manner in which the respondents
answered the question. For one reason or another, they may not have
truthfully answered some of the questions. The respondents may or may
not have completed the questionnaire due to no ready access to a
computer or they just did not want to complete the questionnaire. These
non-respondents became part of the mortality factor involved in the
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 27/218
10
study. Responses to the open-ended questions became difficult to classify
under a certain category. This was facilitated through the Non-
Numerical, Unstructured Data, Indexing Searching & Theorizing Vivo-
“Nudist Alive” (NVivo) software system (Version 7.0) and by the focus
group interviews where the respondents helped determine the category of
such responses.
A factor that may have been encountered in the quantitative
dimension of the study was the lack of intended data for the study.
Diligent efforts were made to gather data from available sources.
Delimitations of the Study
The questions for the on-line qualitative questionnaire may have
been a delimitation of the study. The pilot study contributed to the
improvement of the qualitative tool. Another delimitation may have been
the choice of participants, especially in the focus groups. T he “snowball
technique” addressed this issue. Better interaction happened with added
„quality‟ members to the focus groups.
Qualitative data are available and the inclusion of the quantitative
aspect of the study provided a challenge and an opportunity to determine
if certain factors of the study have any impact on the ELLs.
Definition of Terms
Content Standards are broad descriptions of the knowledge, skills, and
understandings that schools should teach and students should acquire
in a particular subject area (McLaughlin & Shepard, 1995).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 28/218
11
English Language Learners (ELLs) is the preferred term to describe a
student whose native language is other than English (Chamot &
O‟Malley, 1994). These students require instructional modifications, and
eventually take the TAKS after two years of enrollment in the school
districts.
High Stakes Assessment is an assessment in which student promotion
(i.e., high school graduation) can be denied if the scores do not reflect
competence (NCBE, 1997).
Limited English Proficient (LEP) refers to a student with a language
background other than English, and whose proficiency in English is such
that the probability of academic success in an English-only classroom is
below that of an academically successful peer with an English-language
background (CCSSO, 1992).
No Child Left Behind Act (NCLB) of 2001 (PL – 107 – 110). It is the
reauthorization of the Elementary and Secondary Education Act (ESEA).
Opportunity-to-learn (OTL) Standard defines the level and availability of
programs, staff and other resources sufficient to enable all students to
meet challenging content and performance standards (McLaughlin &
Shepard, 1995).
Performance Standards are concrete examples and explicit definitions of
what students have to know and be able to do to demonstrate that such
students are proficient in the skills and knowledge framed by the content
standards (McLaughlin & Shepard, 1995).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 29/218
12
Standardized Assessments include the Texas Assessment of Knowledge
and Skills (TAKS) and the State and Locally-Developed Alternative
Assessment (SLDAA) for students who are exempted from the TAKS. A
standardized assessment is a measurement of what students know and
can do (McLaughlin & Shepard, 1995).
Standards-based Reform requires setting standards of performance in
academic subject areas as a means of improving the substance of school
curricula and increasing the motivation and effort of students, teachers,
and school systems and thereby improving student achievement
(McLaughlin & Shepard, 1995).
Washback shows how the results of an assessment affect the
stakeholders taking the test (Alderson & Wall, 1993).
Significance of the Study
Expected outcome of this study may possibly provide additional
valuable data for writers or researchers in regard to biases in
standardized assessments that may encourage school districts to develop
assessments that truly measure learning without the nullifying effect of
linguistic and cultural bias. Additionally, this study enhances the
reliability of standardized assessments as a tool in determining
accountability where the performance of English language learners is
concerned.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 30/218
13
Organization of the Study
Chapter I identifies the problem this study addresses: the impact
of high stakes assessments on the curriculum and instruction of English
language learners. It includes the hypotheses and research questions of
the present study. Included are the definitions of terms valuable to the
study.
Chapter II includes the review of literature about the essential
conditions and factors regarding the NCLB Act, the AYP implications for
concerned schools, high-stakes, statewide assessments and the
implications and challenges they present to the preparation and
education of ELLs. The information reveals the difficulties that English
language learners face when taking these high stakes assessments, the
possible positive and negative consequences and possible “ washback”
related to the assessments.
A mixed methods study is identified and expounded in Chapter III.
Quantitative data for this research were gathered from the Texas
Education Agency regarding the percentage of ELLs and the performance
of major urban high schools in Texas in the statewide test (Texas
Assessment of Knowledge and Skills) for 2003, 2004, 2005, and 2006.
Qualitative data were derived from an on-line, open-ended questionnaire
and interviews that focused on the respondents‟ views and opinions
about the varied ways standardized assessments impact English
language learners.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 31/218
14
Results of the study are presented in detail in Chapter IV.
Quantitative results include the available data collected from Texas
Education Agency. Results of computations employing the Statistical
Package for the Social Sciences (SPSS) statistical package, (Version 14.0)
are shown in tabular presentations and explanations regarding the
relationship among the variables are included. Qualitative results include
the participants‟ views and opinions on the impact of high stakes testing
on English language learners and the information collected from the on-
line, open-ended questionnaire, individual and focus group interviews.
Major findings of the study are discussed in Chapter V. Impact of
high stakes standardized assessments on English language learners are
also summarized. Other relevant factors that influenced this study are
presented, as well as recommendations for future research.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 32/218
15
CHAPTER II
REVIEW OF LITERATURE
Key issues and concerns about the No Child Left Behind Act (NCLB) of
2001 and the Adequate Yearly Progress (AYP) are major parts of the review
of related literature. Included are the principles and accountability involved
in high-stakes testing and the descriptions and accommodations given to
the ultimate beneficiary of the efforts exerted by the federal and state
policymakers, the school and district administrators – the learners,
specifically, the English language learners who strive to be better citizens of
this country. Short description of related studies on statewide testing and
English language learners (ELLs) are given to show their tie-in with this
study.
No Child Left Behind (NCLB)
Historical Perspective
The NCLB Act of 2001 (PL – 107 -110), is the reauthorization of the
Elementary and Secondary Education Act (ESEA). The ESEA was first
passed in 1965 with the goal of improving the U. S. educational system by
providing better education for students in poverty through an increase in
services to them. The ESEA provided federal funds for schools but did not
require accountability in the use of those funds. In 2003, the Center of
Educational Policy clarified why accountability was not part of ESEA in
1965: “At that time, the federal role in education was marginal, most state
education agencies had very limited authority and capabilities, and local
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 33/218
16
people were extremely wary that more federal aid would bring federal
control” (p.iv).
The National Assessment of Educational Progress (NAEP) was initiated
as a federal testing program at about the same time when ESEA came into
existence. NAEP was tasked to report how the nation‟s students were
performing on selected items at the three grade levels --- 4th, 8th and 12th.
Brennan (2004) reported that there were fears that the NAEP might become
a “high-stakes federal testing program” found in some European countries.
He explained that, “to help preclude that possibility, it was written into law
that NAEP could not report scores for individual students” (p.2). The NAEP
evolved through the 1980s and early 1990s from a reporting of item scores
to test scores and then, on a trial basis, to a reporting of scores that
addressed achievement levels (below basic, basic, proficient, and advanced).
It is currently used to confirm state NCLB testing results which, according
to Brennan, “is the de facto elevation of NAEP to a federally-mandated high-
stakes testing program” (p.9).
Through the NCLB Act, policymakers in Washington seek to raise
academic achievement in the nation by requiring schools to assess all
students on specified content areas and report their progress toward
proficiency. Focus of NCLB is on core academic subjects as defined in the
law: “The term „core academic subjects‟ means English, reading or language
arts, mathematics, science, foreign language, civics, and government,
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 34/218
17
economics, arts, history, and geography” (U.S. Department of Education,
2002).
The premise of NCLB is that our nation‟s schools are failing. Thus, the
purpose of NCLB is raising the achievement of all students and eliminating
the achievement gap among students differentiated by race, ethnicity,
poverty, disability, and English proficiency. Since this Act redefines the
federal role in education policy that has traditionally been a state
responsibility, it merits the attention of educators, parents and citizens.
Because the NCLB Act has an impact on the teaching and the learning of
the core content areas, including languages, language educators need to be
informed about it.
If a roomful of educators were asked which word or phrase best sums
up No Child Left Behind (NCLB), many would say accountability . Others
might propose student achievement, proficiency or raised expectations . But
perhaps the most accurate word to encapsulate the United States‟ most
ambitious federal education law – which proposes to close achievement gaps
and aims for 100% student proficiency by 2014 - is testing. Certainly, the
focus on holding schools accountable for student achievement on
standardized tests sets NCLB apart from previous versions of the law.
(Guilfoyle, 2006).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 35/218
18
Description of the Key Factors
There are four key elements in the NCLB Act (Rosenbusch, 2005):
(1) Accountability. States are required to establish a definition of
student proficiency in the core academic subjects of Reading/Language
Arts, Mathematics and Science through prescribed indicators and set a
timetable to bring all students in all subgroups up to the defined levels of
proficiency by 2013-2014. The school must report to parents their child‟s
progress in each targeted academic subject annually, and the state is
required to report the results of students‟ performance on the annual tests
for every public school to parents and the community. Schools that fail to
meet state-defined AYP toward their defined goals for two years are
identified as needing improvement. Schools that have not met AYP after four
years are subject to restructuring or reconstitution.
(2) Testing. States must develop and administer annual tests that
define the proficiency that all students are expected to reach in
Reading/Language Arts, Mathematics, and Science. States must include a
sample of students in fourth and eighth grades in a biennial NAEP in
Mathematics and Reading to verify state assessments.
NCLB requires that by School Year (SY) 2005-2006, each state must
measure every child‟s progress in Reading and Mathematics in each of
grades 3 through 8 and at least once during grades 10 through 12. In the
meantime, each state must meet the requirements of the previous law
reauthorizing ESEA (the Improving America‟s Schools act of 1994) for
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 36/218
19
assessments in Reading and Mathematics at three grade spans (3-5; 6-9;
and 10-12). By SY 2007-2008, states must have in place Science
assessments to be administered at least once during grades 3-5; grades 6-9;
and grades 10-12. States must ensure that districts administer a test of
English proficiency to measure oral language, Reading and Writing skills in
English to all limited English proficient students, as of SY 2002-2003.
Students may still undergo state assessments in other subject areas (i.e.,
History, Geography, and Writing skills), if and when the state requires it.
NCLB requires assessments only in the areas of Reading/Language Arts,
Mathematics, and Science.
(3) Teacher Quality. Public elementary and secondary school teachers
who teach core content areas are required to be “highly qualified,” which is
defined as having full state certification (may be attained through alternate
routes specified by the state), holding a bachelor‟s degree, and having
demonstrated subject matter competency as determined by the state under
NCLB guidelines. States are required to develop a plan by the end of 2005-
2006 to ensure that every teacher is highly qualified to teach in his or her
core content area.
(4) Scientifically-Based Research. The NCLB Act requires that all
educational decisions be informed by scientifically-based research as
defined in the legislation. The NCLB Act funds for Reading First Grants, for
example, are to be used for methods of reading instruction backed by
scientifically-based research.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 37/218
20
Expectations for Parents Due to NCLB (from Collegeboard.com)
(1) New standards for students will require that beginning 2005,
students in grades 3 through 8 must be tested in Mathematics and English
to ensure they are meeting state standards. Students in Grades 10 through
12 will be tested at least once. By 2007, states will begin testing students in
Science as well. Results of the yearly tests will be known to parents. NCLB
requires that school districts provide parents with an annual “report card”
that shows how well students in each school performed. The information is
broken down by race, ethnicity, gender, disability status, and other
categories so that parents will know how well each school is doing in
educating minority students or those with disabilities.
(2) By the end of SY 2005-2006, teachers must be “highly qualified” in
the subjects they teach. States will determine what skills teachers must
have to be “highly qualified”, but the requirements could include a degree in
the subject they teach or extra training. States must provide annual report
cards about teacher certifications, including the percentage of classrooms in
the state not taught by highly qualified teachers. Principals must also
maintain information about whether or not their school‟s teachers meet the
requirements.
(3) Each year, schools must increase the number of students who
achieve state standards. At the end of 12 years, all students should be able
to pass the tests. Schools that fail to achieve this progress will be targeted
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 38/218
21
for improvements that could include increased funding or staff and
curriculum changes.
(4) NCLB requires school districts to notify parents if the child‟s
school has been identified as needing improvement as a result of failing to
increase the number of students meeting state standards.
(5) About half of all public schools receive funding to help students
from low-income families. If such a school is targeted for improvement and
fails after two years, parents can choose to transfer their child to another
school or enroll in free tutoring. Parents have this choice for as long as the
school fails to adequately perform.
Response to NCLB (Rosenbusch, 2005)
NCLB has engendered controversy that is centered in part on the
increased role of the federal government in educational policy. A majority of
Americans believe that decisions about what is taught in public schools
should be made at the local level by the school board (61%), rather than at
the state level (22%) or the federal level (15%) (Rose & Gallup, 2003).
Results of a 2004 survey indicate that they disagree with “the major
strategies NCLB uses to determine whether a school is or is not in need of
improvement” (Rose & Gallup, 2004, p.2). For example, 83% of those
surveyed believe that testing only in English and Mathematics will not yield
a fair picture of the school, 73% say it is not possible to judge a student‟s
proficiency in English and Mathematics on a single test, and 81% are
concerned that basing decisions about school on students‟ performance in
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 39/218
22
English and Mathematics will mean less emphasis on art, music, history
and other subjects.
In the U.S. Department of Education, there is support for high
standards and high expectations for every child, but the NCLB focus on
standardized testing is resulting in a narrowing of the curriculum and a
“sorting of students” (Marshak, 2003, p.229) and “could halt the
development of truly significant improvements in teaching and learning”
(Lewis, 2002, p.179). The National Education Association supports the
NCLB Act in its goal but views it as an obstacle to improving public
education because of its focus on “punishment rather than assistance”, and
“mandates rather than support for effective programs” (National Education
Association, n.d.).
Adequate Yearly Progress (AYP)
Purpose and Support to NCLB
The No Child Left Behind Act of 2001 (NCLB; Public Law No. 107-110,
115 Stat. 1425, 2002), the most recent reauthorization of the Elementary
and Secondary Act of 1965, holds states using federal funds accountable for
student academic achievement. States are required to develop a set of high-
quality, yearly student assessments that include, at a minimum,
assessments in Reading/Language Arts, Mathematics and Science. Each
year, they must report student progress in terms of percentage of students
scoring at the “proficient” level or higher. This reporting is referred to as
adequate yearly progress (AYP). A state‟s definition of AYP should also
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 40/218
23
include high school graduation rates and an additional indicator for middle
schools to reach the “proficient” level or higher, which must be no more
than 12 years after the start date of the 2001 – 2002 school year, provided
that the first increase occurs within the first 2 years (Abedi, 2004).
AYP will be reported for schools, school districts, and the state for all
students. In addition, AYP must be reported for the following subgroup
categories of students: (a) economically disadvantaged students, (b)
students from major racial and ethnic groups, (c) students with disabilities,
and (d) students with limited English proficiency (LEP). According to the
educational statistics for 2000 – 2001 school year, the total number of
students labeled as LEP in the nation‟s public schools is more than 4.5
million or 9.6% of total enrollment; (National Center for Education Statistics
[NCES], 2002).
States are continuing to find new ways to calculate AYP under the
NCLB, in order to increase the number of schools and districts that meet
the student achievement targets set by law. Over the past few years, the
U.S. Department of Education (ED) has allowed states to make many
changes in the way they determine AYP, including the following: (1)
confidence intervals, which make allowances for natural fluctuations in test
scores and essentially bolster a school‟s or subgroup‟s percentage of
students scoring at proficient levels; (2) performance indices that allow
schools to get “partial credit” for the performance of students below the
proficient level; (3) retesting, which allows students to retake a different
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 41/218
24
version of the same test and permits schools to use a student‟s best score to
count toward AYP, and (4) increased minimum subgroup sizes, which mean
that in many schools, subgroups do not get counted for AYP purposes. The
changes have the effect of making it easier for the schools to make AYP,
early indications are that the number of schools not making AYP has leveled
off, despite predictions that this number would increase as proficiency
targets rose (Olson, 2005).
Changes and Updates
In NCLB‟s original conception, determining AYP for a subgroup of
students, a school, or a district was already fairly complicated. States had to
establish, for every year between 2003 and 2014, a set of ever-increasing
state targets in terms of the percentage of students scoring at the proficient
level or above on annual tests, with a final goal of 100% proficiency in 2014.
If at least 95% of the students in each subgroup are tested, and if all
students and subgroups meet the state proficiency targets, the school or
district makes AYP. The school has to meet targets for an additional
academic indicator, such as the graduation or attendance rate. The law has
a “safe harbor” provision: if a school or subgroup fails to meet the state
targets, it could still make AYP if it reduces the number of students who are
not proficient from the previous year by 10%, and meets its additional
academic indicator.
Some other state changes that have been approved are briefly
summarized below (Center on Education Policy, 2005):
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 42/218
25
Minimum subgroup size. To make AYP, schools and districts must
meet achievement targets for each significant subgroup of students enrolled,
such as African-American students, low-income students, or students with
disabilities. Higher minimum subgroup sizes mean that in many schools,
subgroups do not get counted for AYP purposes.
Thirteen states increased their minimum subgroup sizes in 2004; ten
more did so in 2005. The trend is away from a single minimum size and
toward larger subgroup sizes, different subgroup sizes for different
subgroups and/or purpose, and the use of formulas for determining
subgroup sizes. Georgia is one state that uses a formula approach. Its
subgroup size varies according to the size of the school; the minimum size is
either 40 students or 10% of a school‟s student population, whichever is
greater, with a cap of 75 students.
Participation averaging. NCLB requires 95% of the students in every
school and every subgroup within a school to take each subject test
required by the Act. If this test participation requirement is not met, the
school cannot make AYP even if its test scores meet state targets. In March
2004, the Department relaxed this requirement, allowing states to average
their participation rates over two or three years, so that a 94% participation
rate one year could be balanced by a 96% participation rate the following or
previous year. In 2005, six states changed their accountability plans to
incorporate this new policy, in addition to the 32 that did so last year.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 43/218
26
English language learners. Initially the U.S. Department of Education
(ED) required all English language learners to be tested with the same
grade-level tests as other students. In response to state and local criticism,
the Department revised its policy in February 2004 to allow states to exempt
immigrant students who are in their first year of enrollment in a U.S. school
for less than one year from taking the regular state English Language Arts
tests. These students still have to take an English language proficiency test
and a Mathematics test, but the results need not count toward AYP. When
calculating AYP for the subgroup of English language learners, states can
also count the progress of former English language learners for two years
after they reach English proficiency. Six more states adopted these changes
in 2005, in addition to the 36 states that did so in 2004.
Extra time is given for students with disabilities and English language
learners to graduate. In 2005, eight states received approval from ED to
count students with disabilities and/or English language learners as
graduating on time even if they need extra years of high school. Seven states
received permission to do this in 2004. For students with disabilities, their
individualized education plans would need to call for extra years of high
school beyond age 18. English language learners can be counted as
graduating on time if it takes five years, or as determined on a case-to-case
basis (Center on Education Policy, 2005).
Identifying districts for improvement. In 2005, ED approved
amendments requested by 13 states to identify a district as being in need of
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 44/218
27
improvement only when it does not make AYP in the same subject and
across all three grade spans (elementary, middle and high school) for two
consecutive years. In 2004, 18 states made this change. California
attempted to have ED accept a relatively lenient method that exempted
districts where low-income students reached a certain level on state tests.
ED rejected that method, and California settled on the grade span approach
instead (Davis & Sack, 2005).
Annual measurable objectives. Eleven states changed their annual
score targets in 2005; four states did so in 2004. For example, Florida was
allowed to change its schedule of annual measurable objectives so that
targets would increase in smaller increments annually, rather than in large
increments every three years (Olson, 2005); Virginia did so as well. Several
other states, including Alabama, Alaska, New Mexico, and North Carolina,
changed their annual targets because they were introducing new
assessments.
NCLB is a demanding law. The achievement goals are ambitious, and
the burden on states and districts of declaring schools in need of
improvement and then imposing sanctions on them is high. To try to meet
these demands, states have a strong incentive to keep the numbers of
schools and districts not making AYP as low as possible. Unable to change
the fundamental requirements written into the law, states are using
administrative methods to lessen the numbers of schools and districts not
making the AYP – confidence intervals, indexing, and other techniques.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 45/218
28
Education Secretary Margaret Spellings has been more flexible than
her predecessor in policies regarding students with disabilities, and in
granting special exemptions to some districts in the areas of school choice
and supplemental educational services (tutoring). Secretary Spellings has
decided to allow the Chicago school district to provide tutoring despite the
fact that the district has been identified for improvement (Gewertz, 2005).
This exemption was then extended to New York City, Los Angeles, Boston,
Memphis, Anchorage, and Dayton. This was a regulatory change.
Secretary Spellings went further with four districts in Virginia by
suspending a key element of the law itself, invoking a clause in NCLB that
allows the Secretary of Education to do so. Her action exempted these
districts from the law‟s requirement that they provide school choice before
tutoring (Olson, 2005). Secretary Spelling‟s letter to Virginia officials
indicates that this is a pilot program intended to raise the numbers of
students receiving supplemental educational services (Spellings, 2005). In
addition, districts in the five states most affected by Hurricane Katrina were
allowed to postpone, for one year, the consequences that follow when a
school is in need of improvement, such as tutoring, restructuring, and
corrective action (Olson & Davis, 2005).
ED‟s willingness to make adjustments based on state and local
experience is commendable. But on the downside, parents in many states
would now find it difficult to understand what it means when a school does
or does not make AYP, and what criteria were used to determine this
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 46/218
29
success or failure. For example, parents in Pennsylvania may see a report
card that indicates that their child‟s elementary school has made AYP, but
might wonder whether the school is improving or whether it simply made
AYP as the result of what might be seen as a new “loophole” in the law. The
parents probably would not understand that the school may have made AYP
through the use of a 95% confidence interval, safe harbor with a 75%
confidence interval, or the Pennsylvania Performance Index as a second safe
harbor. In other states, parents of English language learners, students with
disabilities, or other subgroups may not realize that raising the minimum
subgroup sizes means that their children no longer count for AYP purposes
at the school level. They might not realize that the use of confidence
intervals allows for considerable leeway in a subgroup‟s test scores not
available to larger groups of students, and that this is occurring despite the
assertion that improving achievement for subgroups is a major focus of the
law.
Other drawbacks to the increasing complexity may contribute in the
difficulty of discerning clear trends in the number of schools and districts
not making AYP, because the rules governing AYP keep changing every year.
Amid these changes, it is impossible to determine whether an increase in
the number of schools making AYP within a state is due to better teaching
and learning or NCLB rule changes. The constant rule changes, particularly
the use of large confidence intervals and ever-increasing minimum
subgroup sizes, may raise questions about whether the law is being watered
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 47/218
30
down so much that it shortchanges the very groups of disadvantaged
children that it aims to help. Public support may wither if the
implementation of the law is perceived as deceptive or confusing.
As states continue to learn from one another about the new types of
flexibility that ED is allowing, and as state achievement targets continue to
rise until 2014, changes in AYP policies are likely to occur at a more rapid
pace, at the expense of the public‟s ability to understand these changes.
More transparency is needed at both the state and federal levels. States
must fully and clearly explain their rationales for requesting changes to
accountability plans. Once changes are approved by ED, they should be
explained in such a way that the public understand how AYP is determined.
At the federal level, ED should more systematically and promptly
publicize its decisions about what types of changes to state accountability
plans are and are not acceptable, and why. The current process of granting
changes does not help state officials learn from other states‟ experiences,
nor does it help them understand how ED is interpreting the intent of the
law.
Adequate Yearly Progress (AYP) and Limited English Proficient (LEP)
Students
Definition of English Language Learners (ELLs) and LEP
Limited English Proficient (LEP) students are students who lack
sufficient English skills to participate in a regular education, all-English
speaking classroom. English Language Learner (ELL), according to Rivera
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 48/218
31
and Stansfield (1998), is a positive way to refer to any LEP student in
English.
NAEP does not provide a definition of the LEP population; instead it
presents criteria for the inclusion of LEP students. NAEP inclusion criteria
indicate that: A student who is identified on the Administration Schedule as
LEP and who is a native speaker of a language other than English should be
included in the NAEP assessment unless: (a) the student has received
Reading or Mathematics instruction primarily in English for less than 3
school years including the current year , and (b) the student cannot
demonstrate his or her knowledge of Reading or Mathematics in English
even with an accommodation permitted by NAEP (NCES, 2001).
Due to the importance of LEP subgroups in NCLB accountability and
reporting, NCLB provides an operational definition of LEP (NCLB, 2002).
According to this definition: T he term „limited English proficient‟, when
used with respect to an individual, means an individual (a) who is aged 3
through 21; (b) who is enrolled or preparing to enroll in an elementary
school or secondary school; (c) who was not born in the United States or
whose native language is a language other than English; who is a Native
American or Alaska Native, or native resident of the outlying areas; and who
comes from an environment where a language other than English has had a
significant impact on the individual‟s level of English language proficiency;
or who is migratory, whose native language is a language other than
English, and who comes from an environment where a language other than
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 49/218
32
English is dominant; and (d) whose difficulties in speaking, reading, writing,
or understanding the English language may be sufficient to deny the
individual the ability to meet the State‟s proficient level of achievement on
State assessments described in section 111(b)(3); the ability to successfully
achieve in classrooms where the language of instruction is English; or the
opportunity to participate fully in society.
The term “English language learner” (ELL) is a recent designation for
students whose first language is not English. This group includes students
who are just beginning to learn English as well as those who have already
developed considerable proficiency. The term reflects a positive focus on
what these students are accomplishing – mastering another language- and
is preferred by some researchers to the term “limited English proficient”
(LEP), the designation used in federal and state education legislation and
most national and state data collection efforts (August & Hakuta, 1997;
LaCelle-Peterson & Rivera, 1994).
The ELL population is highly diverse, and any attempt to describe the
group as a whole, as with any diverse group of people, is bound to result in
inaccurate generalizations. While this group of students share one
important feature - the need to increase their proficiency in English - they
differ in many other important respects. ELLs are a diverse cross-section of
the public school student population. The primary language, cultural
background, socio-economic status, family history, length of time in the
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 50/218
33
United States, mobility, prior school experiences, or educational goals of any
student in this group can distinguish him or her from any other ELLs.
ELLs represent a rapidly growing, culturally and linguistically diverse
student population in the United States. In 2000-2001, LEP students
comprised nearly 4.6 million public high school students. The majority were
Spanish speakers (79.0%), followed by Vietnamese (2.0%), Hmong (1.6%),
Cantonese (1.0%), and Korean (1.0%). Since the 1990-1991 school year, the
limited English proficient population has grown approximately 105%, while
the overall school population has increased by only 12%.
English learners matriculate in schools throughout the nation, but
most frequently in large urban school districts in the Sun Belt states, in
industrial states in the Northeast, and around the Great Lakes. This trend
is changing as immigrants move to more affordable suburban and rural
areas and to areas where language-minority families are relative newcomers,
such as the Midwest. More than half (56.1%) reside in four states alone:
California (32.9%), Texas (12.4%), Florida (5.6%) and New York (5.2%)
(Kindler, 2002). English learners represent one in four K – 12 students in
California schools (California Department of Education, 2000).
This population includes recent immigrants as well as children born
in the United States. In the 2000-2001 school year, more than 44% of all
LEP students were enrolled in Pre-K through Grade 3; about 35% were
enrolled in Grades 4 – 8; and only 19% were enrolled at the high school level
(Kindler, 2002). Many LEP students attend schools where most of their
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 51/218
34
peers live in poverty. There are numerous differences among English
learners; for example, Spanish-speaking families tend to have lower parental
educational attainment and family incomes than Asian-or Pacific-language
families (August & Hakuta, 1997).
Many criteria are used across the nation for identification of ELLs.
Among the most commonly used criteria are Home Language Survey results
and scores from English proficiency tests. There are reasons to believe that
the Home Language Survey results may not be valid because of parents‟
concern over equity in education for their children, parents‟ citizenship
issues, and communication problems (Abedi, 2004b). Similarly, there are
concerns about the validity of current English proficiency tests, such as the
Language Assessment Scales and other commonly used assessments
(Zehler, Hopstock, Fleischman & Greniuk, 1994). Criterion-related validity
coefficients, or the correlation between English proficiency tests and other
existing valid measure of English proficiency, are not strong, explaining less
than 5% of the common variance (Abedi, 2003). Finally, in terms of content
and construct validity, there is little evidence that the contents of the
existing English proficiency tests align sufficiently with commonly accepted
English language proficiency standards, such as standards by Teachers of
English to Speakers of Other Languages (Bailey & Butler, 2003).
Issues and Other Considerations of LEP
Disaggregated progress reports by subgroups mandated by the NCLB
legislation will monitor the nation‟s goal of having “no child left behind.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 52/218
35
However, there are major issues in this disaggregated reporting among
different subgroup categories (students who are economically
disadvantaged, students from major racial and ethnic groups, students with
disabilities, and LEP students). NCLB requirement for subgroup reporting
may give the impression that students in the subgroup categories start the
achievement race at about the same level and can progress with other
students at about the same rate. This might be an overly optimistic view of
the situation of less advantaged learners. By focusing this discussion on
the consequences for schools enrolling LEP students, we see how putting
into practice the policy may produce invalid assessment and unreliable
reporting while exacerbating the burdens of current educators. Following is
a discussion of some challenges in AYP measurement and reporting for LEP
students.
Results of research on the assessment of LEP students suggest a
strong confounding of language and performance. LEP students exhibit
substantially lower performance than non-LEP students in subject areas
high in language demand. Studies suggest that the large performance gap
between LEP and non-LEP may not be due mainly to lack of content
knowledge. LEP students may possess the content knowledge but may not
be at the level of English language proficiency necessary to understand the
linguistic structure of assessment tools. Strong confusion of language
factors and content-based knowledge makes assessment and accountability
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 53/218
36
complex for LEP students and, very likely, students in other targeted
groups.
Because of the strong effect of language factors on the instruction and
assessment of LEP students, they lag far behind native English speakers.
This leads to huge initial differences. LEP students start with substantially
lower baseline scores. More important, unless LEP students‟ English
language proficiency is improved to the level of native English speakers-
which is not an easy task- they will not be able to move at the same rate on
the Adequate Yearly Progress line as do native English speakers.
NCLB cannot have much of an effect on the initial performance
differences between LEP and non-LEP students. A more sensible question
here is whether or not NCLB can provide enough resources to schools with a
large number of LEP students to help them increase these students‟
language proficiency to a sufficient extent that they can progress with their
native English speaker peers in both instruction and assessment.
Inconsistency in LEP classification across and within states makes
AYP reporting for LEP students even more complex. If students are not
correctly identified as LEP, how can their AYP be reliably reported at a
subgroup level? Although NCLB attempts to resolve this issue by providing a
definition for this group, its criteria for classifying LEP students may face
the same problems as the existing classification system (Abedi, 2003;
Zehler, Hopstock, Fleishman & Greniuk, 1994).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 54/218
37
Inconsistency in the classification of LEP students may lead to more
heterogeneity in the LEP subgroup. With a more heterogeneous population,
larger numbers of students are needed to provide the statistically reliable
results required by NCLB. The population of LEP students in many districts
and states is sparse. In many states, there may not be enough students in a
district or school to satisfy even the minimum number of 25 students
suggested in the literature (Linn, Baker & Herman, 2002). Other researchers
have argued that even 25 students may not be enough to provide
statistically reliable results and have proposed a minimum group size of 100
students (Hill & DePascale, 2003). Considering a small number of LEP
students in many districts and states, the small group size for LEP reporting
would be another obstacle in regard to reliable AYP reporting.
The LEP subgroup suffers from yet another major problem related to
AYP reporting: The lack of stability of this group. In many states and
districts across the nation, LEP students‟ level of English proficiency is
reevaluated regularly, and if they reach a proficient level of English
proficiency, they move out of the LEP subgroup. While this helps the more
English-proficient students receive more appropriate instruction and
assessment, it results in the LEP subgroup continuing to be low-performing.
The students in this group will always be labeled as underachievers, and
schools with large number of LEP students will be stuck in the “need for
improvement” category.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 55/218
38
Some states with substantial numbers of LEP students have
expressed concern over this issue. They have proposed ideas and negotiated
with the federal government to ease the level of possible negative impact
that this situation may have on school, district, and state accountability.
For example, Indiana and Delaware will continue to include exited LEP
students in the LEP subgroup for 2 years after they have been determined to
be proficient in English. Georgia plans to include LEP students as long as
they still receive services through the English for Speakers of Other
Languages program, even if they have met exit criteria (Erpenbach, Forte-
Fast & Potts, 2003). In California, students re-designated as LEP will remain
in the LEP category until they reach the proficient or above level on the
California Standards Test in English-language arts for 3 consecutive years
(California Department of Education, 2003). However, the question of
whether this policy will provide a long-term solution to the problem of LEP
subgroup instability or serve only as a temporary relief remains
unanswered.
The measurement of the academic achievement of LEP students is
much more complex than what the NCLB legislation conceives. A fair
assessment of students in the four targeted subgroup categories requires
much more serious consideration than is outlined in the law. Despite
attempting to solve the age-old problem of heterogeneity among LEP
students, the NCLB seems to perpetuate it, thereby leaving more room for
children to be left behind.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 56/218
39
On the other hand, NCLB‟s attention to students in the four subgroup
categories in general and to the LEP population in particular is a step in the
right direction. Considering that Title III of NCLB requires assessment of
LEP students‟ English proficiency on an annual basis and providing support
to states to develop reliable and valid measures of students‟ proficiency is
promising. Any decisions concerning assessment for all subgroups,
particularly LEP students, must be informed by results of research and
experience in the education community.
Currently, several tests for measuring students‟ level of English
language proficiency exist. Some of these tests have been used for many
years by different states and districts. In spite of the existence of such tests,
states are developing new English language proficiency tests with funding
through NCLB‟s Enhanced Assessment Instruments. A reasonable
explanation for this might be that states did not find that the existing tests
provided reliable and valid measures of students‟ level of English language
proficiency as required by NCLB. If this is the reason for the development of
the new tests, then the test developers should be aware of problems in the
existing tests to avoid the same problems in the new tests.
For example, a careful review of some of the most commonly used
language proficiency tests concluded that the tests differ considerably in
types of tasks and specific item content and are based on different
theoretical emphases prevalent at the time of their development (Zehler,
Hopstock, Fleischman & Greniuk, 1994). This suggests that in the case of
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 57/218
40
some of the existing tests, the English language proficiency domain was not
operationally defined before the test development process. This and similar
studies and reviews should inform the development process of new tests.
For example, it is imperative this domain be operationally defined before any
effort in developing an English proficiency test. This definition should be
based on current developments in the areas of psycholinguistics,
developmental psychology, education, linguistics, and psychometrics.
Content standards for English for speakers of other languages should also
be considered (Bailey & Butler, 2003).
In analyzing data from the administration of existing language
proficiency tests, researchers have expressed concerns about the reliability
and validity of these tests, the adequacy of the scoring directions, and the
limited populations on which test norms are based. For example, analyses
of several large data sets from different locations across the nation have
shown validity problems in predicting LEP classification and lack of power
in identifying different levels of English language proficiency among the LEP
student population (Abedi, 2003; Abedi, Leon, & Mirocha, 2003). Those
involved in the development of new English language proficiency tests
should learn from such research and should conduct more analyses on the
wealth of data that exists in this area. To be considered valid and reliable
measures of English language proficiency, as outlined in the NCLB, new
tests must first go through a rigorous validation process. Otherwise, there
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 58/218
41
may not be a reasonable justification to spend the limited NCLB resources
on English language proficiency test development (Abedi, 2003).
As a final thought, assessment and accountability of LEP students
cannot be pursued in isolation of other important factors. An effective
education system for LEP students that may lead to a successful AYP
outcome should include at least three interactive components: (a)
classification, (b) instruction, and (c) assessment. A problem in any one of
these components may affect the other two. For example, a student
misclassified as LEP student may be assigned a different curriculum and
thus receives inappropriate instruction. Alternately, inappropriate
instruction may result in low performance that may in turn result in
misclassification. While each component has a unique role, they share
common ground - the effect of language factors or barriers. Unnecessary
linguistic complexity of assessment may threaten the validity and
equitability of assessment among LEP students. Complex linguistic
structure of instruction may negatively affect LEP students‟ ability to
understand classroom instruction, and invalid assessment of students‟ level
of English proficiency may result in misclassification. In a positive light,
valid assessment may provide diagnostic information that can inform
instruction and classification (Abedi, 2003).
An effective way to help LEP students reach proficiency in the AYP
model is to consider the broader picture using the interactive model. The
following are few critical needs:
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 59/218
42
1. Improve current LEP classification and assessment. There is a need to
establish a common definition of English language proficiency and
substantially improve the validity of LEP instruments. Among other
things, validity of LEP assessment can be enhanced by avoiding
cultural biases and reducing unnecessary linguistic complexity of
assessments.
2. Improve monitoring of progress. Schools need effective and valid data
collection methods that can be used to monitor LEP progress at every
stage of a student‟s education. Weaknesses must be quickly
addressed with appropriate instructional strategies.
3. Improve teacher quality. LEP students need teachers who are well
qualified in both language development and content, each of which
plays a crucial role in LEP student achievement. The federal
government can play a key role in this process by funding and
encouraging programs that improve teacher capacity in this dual role.
Teachers of LEP students should receive training in content delivery,
language sheltering, and the teaching of the academic language.
4. Consider redesignated LEP students as part of the LEP subgroup that
established the baseline score. State plans allowing redesignated
students to remain in the LEP subgroup for only a limited time are
temporary fixes. While new LEP students are added to the subgroup,
redesignated students should be retained for AYP reporting. This
“semicohort” approach to tracking LEP students allows the progress of
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 60/218
43
redesignated students to be counted toward subgroup AYP progress
(Abedi, 2003).
Based on the results of the research, policymakers, lawmakers, and
decision makers are urged to take appropriate action to correct the
inequities resulting from the NCLB in regard to the subgroups targeted by
the legislation, particularly the LEP student subgroup. What is encouraging
is that states, in collaboration with the federal government, are taking steps
to remedy some of these issues. The hope is that these continued efforts will
bring more fairness into the assessment of and accountability for LEP
students (Abedi, 2003).
High Stakes / Statewide Testing
The 2001 reauthorization of the Elementary and Secondary Education
Act (ESEA), also known as the No Child Left Behind (NCLB), carries testing
and accountability requirements that will substantially increase student
testing and hold all schools accountable for student performance. This
legislation marks a major departure from the federal government‟s
traditional role regarding elementary and secondary education. It requires
that states administer Reading and Mathematics tests annually in grades 3
– 8 and during one year in high school starting in 2005 – 2006. These
requirements will affect almost 25 million students each school year
(National Center for Education Statistics, 2002).
NCLB requires states to meet adequate yearly progress (AYP) goals to
ensure school accountability for student achievement on state tests. Schools
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 61/218
44
that fail to achieve AYP goals face demanding corrective actions, such as
replacement of school staff, implementation of new curriculum, extension of
the school day or academic year, parental choice options, and, finally,
complete reorganization.
Today‟s widespread implementation of standards-based reform and
the federal government‟s commitment to test-based accountability ensure
that testing will remain a central issue in education for the foreseeable
future. Test results can provide useful information about student progress
toward meeting curricular standards. But when policymakers insist on
linking test scores to high-stakes consequences for students and schools,
they often overlook lessons from the long history of research (Abrams &
Madaus, 2003).
Current emphasis on testing as a tool of education reform continues a
long tradition of using tests to change pedagogical priorities and practices.
In the United States, this use of testing dates back to 1845 in Boston, when
Horace Mann, then Secretary of the Massachusetts State Board of
Education, replaced the traditional oral examination with a standardized
written essay test. Internationally, high-stakes testing extends as far back
as the 15th century in Treviso, Italy, where teacher salaries were linked to
student examination performance (Madaus & O‟Dwyer, 1999).
Principles of Testing Programs
A 1988 examination of the effects of high-stakes testing programs on
teaching and learning in Europe and in the United States (Madaus, 1988)
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 62/218
45
identified seven principles that captured the intended and unintended
consequences of such programs. Current research confirms that these
principles still hold true for contemporary statewide testing efforts.
Principle 1: The power of tests to affect individuals, institutions,
curriculum, or instruction is a perceptual phenomenon. Tests produce large
effects if students, teachers, or administrators believe that the results are
important. Policymakers and the public generally do believe that test scores
provide a reliable, external, objective measure of school quality. They view
tests as symbols of order, control and attainment (Airasian, 1988).
Today‟s high-stakes testing movement relies on the symbolic
importance of test scores. Forty-eight states currently require schools to
provide the public with “report cards” (Edwards, 2003). Goldhaber and
Hannaway (2001) found that the stigma associated with a school receiving a
low grade on the state report card was a more powerful influence on Florida
teachers than were the school-level sanctions imposed for poor test results.
Principle 2: The more any quantitative social indicator is used for
social decision making, the more likely it will be to distort and corrupt the
social process it is intended to monitor. In other words, placing great
importance on state tests can have a major influence on what takes place in
the classrooms, often resulting in an emphasis on test preparation that can
compromise the credibility or accuracy of test scores as a measure of
student achievement.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 63/218
46
We can assess whether this principle still applies today by examining
the relationship between rising state test scores and scores on other
achievement tests. Both old and new studies of this relationship (Amrein &
Berliner, 2002; Haladyna, Nolen & Haas, 1991; Klein, Hamilton, McCaffrey
& Stecher, 2000; Linn, 1998) show that improvements in the state test
scores do not necessarily reflect general achievement gains.
We can find examples of this second principle in two recent surveys of
teachers‟ opinions. In one national study, roughly 40% of responding
teachers reported that they had found ways to raise state-mandated test
scores without, in their opinion, actually improving learning (Pedulla,
Abrams, Madaus, Russell, Ramos & Miao, 2003). Similarly, in a Texas
survey, 50% of the responding teachers did not agree that the rise in Texas
Assessment of Academic Skills (TAAS) scores “reflected increased learning
and high-quality teaching” (Hoffman, Assaf & Paris, 2001, p. 488).
Principle 3: If important decisions are based on test results, then
teachers will teach to the test. Curriculum standards and tests can focus
instruction and provide administrators, teachers, and students with clear
goals. A substantial body of past data and recent research confirms that as
the stakes increase, the curriculum narrows to reflect the content sampled
by the test (Jones et al., 1999; Madaus, 1991; McMillan, Myran, &
Workman, 1999; Pedulla et al., 2003; Stecher, Barron, Chun & Ross, 2000).
New York State, where the state department of education is requiring
schools to spend more time on the NCLB-tested areas of Reading and
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 64/218
47
Mathematics, provides an example on how such pressure encourages
schools to give greater attention to tested content and decrease emphasis on
non-tested content. According to one school principal, “the art, music, and
everything else are basically out the window… something has to go”
(Herszenhorn, 2003).
Principle 4: In every setting where a high-stakes test operates, the
examination content eventually defines the curriculum. Pressure and
sanctions associated with a state test often result in teachers using the
content of past tests to prepare students for the new test. Several studies
have documented that an overwhelming majority of teachers feel pressure to
improve student performance on the state test. For example, 88% of
teachers surveyed in Maryland and 98% in Kentucky believed that they were
under “undue pressure” to improve student performance (Koretz, Barron,
Mitchell & Keith, 1996a, 1996b). As an outgrowth of this pressure, the
amount of instructional time devoted to specific test preparation often
increased.
Studies have found that teachers are spending a sizable amount of
instructional time and using a variety of test-specific methods to prepare
students for their state tests (Herman & Golan, n.d.; Hoffman, Assaf, &
Paris, 2001). In North Carolina, 80% of elementary teachers surveyed “spent
more than 20% of their total instructional time practicing for the end-of-
grade tests” (Jones et al., 1999, p. 201). A national survey found that
teachers in high-stakes states were four times more likely than those in low-
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 65/218
48
stakes setting to report spending more than 30 hours a year on test
preparation activities, such as teaching or reviewing topics that would be on
the state test, providing students with items similar to those on the test,
and using commercial test-preparation materials from previous years for
practice (Pedulla et al., 2003).
Principle 5: Teachers pay attention to the form of the questions of
high-stakes tests (short-answer, essay, multiple-choice, and so on) and
adjust their instruction accordingly. A wide variety of research confirms that
test format does influence instruction in both positive and negative ways.
Studies in states that require students to formulate and provide
written responses to test questions show an increased emphasis on teaching
writing and higher-level thinking skills (Taylor, Shepard, Kinner &
Rosenthal, 2003). For example, in Kentucky, 80% of teachers surveyed
indicated that they had increased their instructional emphasis on problem
solving and writing as a result of the portfolio-based state test (Koretz,
Barron, Mitchell, & Keith, 1996a).
In several studies, teachers have reported decreases in the use of
more time-consuming instructional strategies and lengthy enrichment
activities (Pedulla et al., 2003). A study found that the format of the state
test may adversely affect the use of technology for instructional purposes:
One-third of teachers in high-stakes states said that they were less likely to
use computers to teach writing because students were required to construct
handwritten responses on the state test (Russell & Abrams).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 66/218
49
Principle 6: When test results are the sole or even partial arbiter of
future education or life choices, society treats test results as the major goal
of schooling rather than as a useful but fallible indicator of achievement.
Almost 100 years ago, a chief inspector of schools in England described this
principle in a way that resonates today: Whenever the outward standard of
reality (examination results) has established itself at the expense of the
inward, the ease with which worth (or what passes for such) can be
measured is ever tending to become itself the chief, if not sole, measure of
worth. And in proportion as we tend to value the results of education for
their measurableness, so we tend to undervalue and at last ignore those
results which are too intrinsically valuable to be measured (Holmes, 1911).
In the next five years, almost half of U.S. states will require students
to pass a state-mandated test as a requirement for graduation (Edwards,
2003). As a result, a passing score on the state test is the coin of the realm
for students, parents, teachers, and administrators. The social importance
placed on state test scores ensures that students‟ successful performance
on the state test is the ultimate goal for schools. Local press coverage on
school pass rates and anecdotal evidence that scores on the state test may
influence local real estate sales show the importance of test performance as
a surrogate for education quality.
Principle 7: A high-stakes test transfers control over the curriculum to
the agency that sets or controls the examination. State standards-based
reform efforts leave the details and development of testing programs to state
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 67/218
50
departments of education and whomever the department contracts with to
construct the test. This system shifts the responsibility for determining
curricular priorities and performance standards away from local school
administrators or classroom teachers and often results in a one-size – fits-all
curriculum and test.
Falmouth, Massachusetts, provides a recent noteworthy example of
how a high-stakes state test can override local control. Under the threat of
losing state funding and the licensure of the school principal and
superintendent, the Falmouth School Committee reversed a decision to
award diplomas to special-needs students who failed the Massachusetts
state examination, thus shattering the hopes of a student seeking
admittance to a nonacademic culinary degree program (Myers, 2003).
Accountability in Testing
No one denies the importance of accountability. The relationship
between test scores and accountability, however, is not as simple as some
people think. The seven principles formulated in 1988 have been acted out
in state after state in the past 15 years and clearly reveal the serious flaws
in the practice of using a single high-stakes measure to hold all students
and schools accountable.
Cut-off scores that place students in such performance categories as
needs improvement, basic, proficient, or advanced are arbitrary. The
subjective methods used to categorize students into performance categories
often lack validity (Horn, Ramos, Blumer & Madaus, 2000). Most
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 68/218
51
policymakers and the public do not understand the psychometric
underpinnings of the tests. Issues that might seem trivial to them, such as
the assumptions made when running computer programs that produce
scaled scores, and even basic decisions about rounding, have significant
consequences when categorizing students.
Like any measurement tool that produces a number, test scores are
fallible. Yet most state laws do not consider margin of error when
interpreting students‟ scores. Misguided executive decisions, poorly
conceived legislation, understaffing, unrealistic reporting deadlines, and
unreasonable progress goals can cause numerous errors in test scores
(Rhoades & Madaus, 2003).
One single test can only sample knowledge and cannot give a full
picture of what students know and can do. As an illustration, Harlow and
Jones‟s (2003) interviews with students showed that on the science portion
of the Third International Mathematics and Science Study (TIMSS), the
students had more knowledge about concepts than their written answers
had demonstrated for more than half of the test questions. Conversely, the
interviews suggested that for one-third of the items, students lacked a
sound understanding of the information assessed even though they had
given the correct response.
A fundamental principle in social science research is to always use at
least two methods when studying social science phenomena because relying
on only one method can produce misleading results. We need to enhance
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 69/218
52
state testing programs by including multiple measures of student
achievement. Measuring in a variety of ways does not mean giving students
multiple opportunities to take the same test, but rather incorporating other
methods of measurement or additional criteria, such as teacher judgments,
when making decisions about grade promotion and graduation (Harlow &
Jones, 2003).
As districts, schools, and teachers respond to federal and state-based
accountability policies, we must step back from a blind reliance on test
scores. We need to acknowledge that tests, although useful, are also fallible
indicators of achievement. We also need to recognize that when test scores
are linked to high-stakes consequences, they can weaken the learning
experiences of students, transform teaching into test preparation, and taint
the test itself so that it no longer measures what it was intended to measure
(Harlow & Jones, 2003).
Effects of High-Stakes Testing on Student Motivation and Learning
Current generation of policymakers did not invent high-stakes testing.
Tests of various sorts have determined which immigrants could enter the
United States at the turn of the 20th century, who could serve in the armed
forces, who was gifted, who needed special education, and who received
scholarships to college. But the NCLB Act of 2001 aims to make high-stakes
testing more pervasive than ever before, mandating annual testing of
students in grades 3 – 8 in Reading and Mathematics.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 70/218
53
Federal legislators who overwhelmingly passed this act into law
apparently assumed that high-stakes testing would improve student
motivation and raise student achievement. Because testing programs
similar to those required by NCLB already exist in many states, we can put
that assumption to the test.
Eighteen states currently use examinations to grant or withhold
diplomas: Alabama, Florida, Georgia, Indiana, Louisiana, Maryland,
Minnesota, Mississippi, Nevada, New Jersey, New Mexico, New York, North
Carolina, Ohio, South Carolina, Tennessee, Texas and Virginia. Most of
these states also attach to their state assessments a broad range of other
consequences for students, teachers, and schools. The experiences of these
states can help predict how the new nationwide program of high-stakes
testing will affect student achievement.
Unfortunately, the evidence shows that such tests actually decrease
student motivation and increase the proportion of students who leave school
early. Further, student achievement in the 18 high-stakes testing states has
not improved on a range of measures, such as the National Assessment of
Educational Progress (NAEP), despite higher scores on the state‟s own
assessments. (Amrein & Berliner, 2003).
High-stakes testing assumes that rewards and consequences attached
to rigorous tests will “motivate the unmotivated” to learn (Orfield &
Kornhaber, 2001). The “unmotivated” are usually identified as low socio-
economic students in urban schools, often African Americans and Latinos.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 71/218
54
Researchers have found that when rewards and sanctions are attached to
performance on tests, students become less intrinsically motivated to learn
and less likely to engage in critical thinking. In addition, they have found
that high-stakes testing cause teachers to take greater control of the
learning experiences of their students, denying their students opportunities
to direct their own learning. When the stakes get high, teachers no longer
encourage students to explore the concepts and subjects that interest them.
Attaching stakes to tests apparently obstruct students‟ path to becoming
lifelong, self-directed learners and alienates students from their own
learning experiences in school (Sheldon & Biddle, 1998).
Dropout rates are climbing throughout the United States and many
researchers hold high-stakes testing at least partly to blame (Rothstein,
2002). Some researchers found that dropout rates were 4 to 6 percent
higher in schools with high school graduation examinations. Another study
reported that students in the bottom quintile in states with high-stakes
testing were 25% more likely to drop out of high school than were their
peers in states without high-stakes testing (Jacob, 2001). Researchers in yet
another study found that failing these tests significantly increased the
likelihood that even students with better academic records would drop out
(FairTest & Massachusetts CARE, 2000).
More and more teenagers are exiting formal schooling early to earn a
General Educational Development (GED) credential (Murnane, Willett, &
Tyler, 2000). Although young people who have earned such alternative
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 72/218
55
degrees do not technically count in dropout statistics, many of them
undoubtedly left school because of their concerns about passing rigorous
graduation tests.
Students who repeat a grade are significantly more likely to drop out
of school (Goldschmidt & Wang, 1999). In states where promotion to the
next grade hinges on passing the state exams, high-stakes testing policies
contribute to higher dropout rates in the long run. Even before they actually
take the test, struggling students are more likely to be retained in grade if
they attend schools in high-stakes testing environments. By holding low-
achieving students back, schools ensure that these students have more of
the knowledge necessary to perform well on high-stakes testing the next
year and keep low-performing students‟ test scores out of the composite test
performance in the grades in which high-stakes testing matter.
In Texas, students from racial minorities and low socio-economic
backgrounds are being retained in Grade 9 at very high rates before taking
the Texas Assessment of Academic Skills (TAAS) in Grade 10. Many teachers
retain students if they doubt their potential to pass TAAS the following year.
McNeil (2000) estimated that half of all minority students enrolled in Texas
high schools are technically enrolled as freshmen. Although some of them
are 9th graders for the first time, thousands of others have been retained in
the 9th grade once or even twice. Other researchers (Haney, 2000, 2001;
Klein, Hamilton, McCaffey & Stecher, 2000; Yardley, 2000) have verified her
numbers. In 1998, one in every four African American and Latino 9th
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 73/218
56
graders in Texas was retained (Fisher, 2000). After these students are
retained, thousands of them drop out of school.
Common problems of high-stakes testing programs are quite likely to
affect the breadth and depth of student learning. If schools narrow the
curriculum they teach; make heavy use of drill activities tied to the state
test; cheat by over-identifying language minority and special education
students and then keeping these students from taking these tests; retain
poorly performing students in grade; and encourage those who are at least
likely to pass the state‟s test to drop out, then scores on state tests will
almost certainly go up. But have students really learned any more than they
did before high-stakes testing policies were instituted (Fisher, 2000)?
Other Considerations of Assessment and Testing
Although NCLB now requires all students to be accounted for in any
state‟s assessment system, this has not always been the case (Anderson,
2004). In the past, groups of students such as English language learners or
students in Special Education were systematically excluded from large scale
assessments (State, 1999), or their scores were not reported (Thurlow,
Neilson, Tellucksingh, & Ysseldyke, 2000).
In the 2002-2003 school year, the Texas Assessment of Knowledge
and Skills (TAKS) replaced the Texas Assessment of Academic Skills (TAAS)
as the primary statewide assessment program. TAKS is designed by
legislative mandate to be more comprehensive than its predecessors and
encompasses more of the state-mandated curriculum, the Texas Essential
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 74/218
57
Knowledge and Skills (TEKS), at more grade levels than TAAS did. The high
school level assessments, administered at Grades 9, 10 and 11, are
grounded in the high school TEKS curriculum. By law, students for whom
TAKS is the graduation testing requirement must pass exit level tests in four
content areas – English Language Arts, Mathematics, Science, and Social
Studies – in order to graduate from a Texas public high school (Technical
Digest, 2003-2004).
In Texas, there is evidence that the numbers of black and Hispanic
students in Special Education rose between 1994-1998 while the state
implemented its statewide testing program which excluded some Special
Education students from the reported scores (Haney, 2000). It is interesting
to note that while the numbers of African-American and Hispanic students
are over-represented in Special Education, about eight to nine percent of
English language learners are identified as receiving Special Education
services in the US (D‟Emilio, 2003, June; Zehler, Fleischman, Hopstock,
Pendzick, & Stepherson, 2003).
Labeling schools can have an impact on teacher and student morale
(Anderson, 2004). Certainly, poor test scores or poorly explained
assessment systems can result in decreased student motivation (Lane &
Stone, 2002). Teachers have also reported that the high-stakes nature of
some assessments can have a negative impact on student morale (Flores &
Clark, 2003). Although some teachers have reported that their English
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 75/218
58
language learners can reach the high standards set for them, they may need
more time than other students (Hood, 2003).
For English-language learners, the additional requirements of an exit
examination could increase dropout rates (Anderson, 2004). Hispanic
students, many of whom are English-language learners, have higher
dropout rates than the population as a whole (Barro & Kolstad, 1987;
Kaufman, Alt, & Chapman, 2001).
In another study, teachers reported that increasing emphasis on test
scores cause them to dislike their jobs (Hinde, 2003). In a study examining
the discussion and journal entries from teachers, Flores and Clark (2003)
found that teachers were not against accountability and viewed it as distinct
from statewide testing, but also thought that an over-emphasis on testing
resulted in unbalanced curriculum and inappropriate instructional
decisions. In order for teachers to make specific changes to instruction, the
assessments needs to be clear as to what skills are being assessed (Popham,
2003). Some teachers may react to low test scores of English language
learners by teaching to the test while others may ignore the impact of the
test scores all together (Alderson & Wall, 1993).
Testing cannot be divorced from socio-cultural, economic, and
psychological issues (Solano-Flores & Trumbull, 2003). ELLs, for instance,
may not score any differently in an assessment even when allowed to use a
dictionary (Albus, Thurlow, Liu & Bielinski, 2005). This is further
complicated by the theory forwarded by Wang and Koda (2005) that ELLs as
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 76/218
59
a group may have diverse styles in developing English Language proficiency.
Therefore modifications are adapted to teach ELLs the academic content of a
lesson, and at the same time support the acquisition of a new language
(Major, Fitzmaurice, Bunta, & Balasubramanian, 2005).
A study of washback from a test in Hong Kong demonstrated that
change in the assessment could change the ways in which teachers and
students interacted (Cheng, 1999). School Administrators and teachers, as
well as students, need to be motivated to change the way learning takes
place and also be invested in demonstrating achievement on the
assessments in order for washback to instruction to take place and be
successful (Lane & Stone, 2002).
Related Studies
In a study on “Intended and Unintended Consequences of Statewide
Testing for ESL Curriculum and Instruction”, Anderson (2004) examined
what positive or negative impact assessment systems have on the
curriculum and instruction of English language learners in one Midwestern
school district. The researcher used focus groups and interviews to obtain
views of educators on the consequences of statewide testing for ELLs.
Positive consequences that were identified included more teacher
collaboration, changes in curriculum and instruction, better alignment
between ESL and content area curricula and more focus on reading and
writing. Negative consequences included student and teacher frustration,
more teaching to the test occurring, and a narrowed curriculum. Educators
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 77/218
60
in the study also identified problems with the accountability system and
made recommendations for how it could be improved (Anderson, 2004).
Another study on “Inclusion of Students with Limited English
Proficiency in National Assessment of Educational Progress (NAEP):
Classification and Measurement Issues” conducted by Abedi (2004) reported
the major concerns over classification and measurements for students with
limited English proficiency (LEP). Issues included the poor operational
definition of English proficiency construct and validity concerns on the
existing language proficiency tests. The study discussed issues concerning
the classification of ELLs and elaborated on factors that impact decisions to
include ELLs in NAEP assessments. With funding through a competitive
bidding process authorized under the NCLB section on Enhanced
Assessment Instruments, there are national efforts underway to develop
English proficiency tests that can be used to provide valid measures of
students‟ level of English proficiency (Abedi, 2004).
Wall (2000) made a microethnographic case study entitled “A Case
Study of Secondary School Efforts Toward English Language Learner
Success in a Standards-Based Reform System.” This study was designed to
describe and interpret the site-based decision-making process of a
collaborative study group of high school educators as they focused on the
appropriate participation of ELLs in a district wide, standards-based, reform
initiative. The research question which guided the study was: From what
perspectives and with what outcomes does a collaborative group of site-
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 78/218
61
based, high school educators deliberate the participation of ELLs in a
standards-based reform system which mandates high stakes assessments?
Three themes emerged from the study: (a) personal discovery, (b) informed
action, and (c) instructional advocacy. These themes suggested phases of
sociolinguistic accommodation through which educators progress in their
reform-based deliberations regarding appropriate approaches to support
ELLs in a high-stakes assessment system (Wall, 2000).
This study on the impact of high-stakes testing on ELLs in major
urban high schools in Texas showed quantitatively how the percentage of
ELLs enrolled in a school affects the school‟s performance in the State‟s
assessment. Qualitatively, it gathered the input and feedback of educators
on the different concerns included in the study: (a) purpose of TAKS, (b)
changes caused by TAKS, (c) consequences of TAKS, (d) recommendations to
improve TAKS, and (e) needs of ELLs.
Summary
As stated in chapter I, the purpose of this study was to determine the
impact of high-stakes testing on ELLs. This was shown in both the
quantitative and qualitative dimensions of the study. Data obtained from
TEA were used to determine whether there is a relationship between the
percentage of English language learners enrolled in a school and the
percentage of all students passing the 10th grade TAKS tests in the core
areas of English Language Arts and Mathematics given in 2003, 2004, 2005
and 2006. To support the quantitative aspect, this study explored what
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 79/218
62
certified ESL teachers, non-certified ESL teachers who teach ELLs,
administrators, and district ESL personnel viewed as the impact that high
stakes standardized assessments have on ELLs, ESL curriculum and
instruction, and what they observed as actually occurring.
The mandates and key elements of the NCLB were geared towards
improving the achievement of students in the different public schools of the
United States. The measure of adherence was channeled through the AYP
that the different schools and districts of the different states monitor and
report. High-stakes testing became the measuring stick that gauged the
achievement of students in the different core subject areas. Issues and
concerns were centered on the ELLs regarding the different moves and
accommodations given to this special subgroup of learners. Feedback
regarding the issues and concerns of the different studies and researches
included both positive and negative dimensions.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 80/218
63
CHAPTER III
METHODOLOGY
Introduction
Standardized testing and assessments have become necessary
facets of American education. Consequently, accountability testing is
currently implemented in practically every state in the U.S. Since the
purpose of this increased level of accountability is to ensure that all
students are receiving a quality, standards-based education, it is
important to document the consequences of the system to ensure that
the intended reforms are taking place. One of the goals of the
accountability system should be to document any negative opposing
impact that could occur so that interventions can be developed so that
these consequences can be minimized.
The purpose of this study was to determine the impact of high-
stakes testing on ELLs. This is shown in both the quantitative and
qualitative dimensions of the study. Data obtained from TEA were used
to determine whether there is a relationship between the percentage of
English language learners enrolled in a school and the percentage of all
students passing the 10th grade TAKS tests in the core areas of English
Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To
support the quantitative aspect, this study explored what certified ESL
teachers, non-certified ESL teachers who teach ELLs, administrators,
and district ESL personnel viewed as the impact that high stakes
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 81/218
64
standardized assessments have on ELLs, ESL curriculum and
instruction, and what they observed as actually occurring.
The study also addressed concerns regarding the validity of
student evaluations, and the common inferences made about student
performance in these assessments. There is a need to know how public
schools that have diverse student attributes can be held accountable on
the basis of one uniform and universal standard. Since the standardized
assessments are given in English, schools with predominantly Hispanic
populations may already be at a disadvantage through no fault of their
own. What needs to be examined is whether standardized assessments
facts are free from linguistic and cultural bias as viewed by teachers of
ELLs.
Research Questions
Quantitative
Is there a relationship between the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005 and 2006?
Null Hypotheses
HO1: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
English Language Arts given in 2003, 2004, 2005 and 2006.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 82/218
65
HO2: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
Mathematics given in 2003, 2004, 2005 and 2006.
Qualitative
The major question answered by this study was: What are the
anticipated and observed consequences of statewide testing specifically,
TAKS, on ELLs, ESL curriculum and instruction as viewed by certified
ESL teachers, non-certified ESL teachers who teach ELLs, school
administrators, and district ESL personnel?
This major question was explored using the following probes:
1. Why is TAKS given as a statewide test?
2. What are the intended consequences of this statewide testing?
(Or what has happened because of TAKS?)
3. What problems have occurred related to or because of TAKS?
4. What changes were caused by this statewide testing?
5. What are your recommendations to improve this statewide
testing?
6. What needs to be done for the ELLs to improve their
performance in general and specifically for this statewide test?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 83/218
66
Research Methods
Both descriptive and comparative research techniques were
employed in the explanatory design of the mixed methods study.
Fraenkel and Wallen (2003) stated that Creswell describes the two types
of mixed methods.
1. In a triangulation design, the researcher simultaneously collects
both quantitative and qualitative data, compares results, and
then uses those findings to see whether they validate each other
(p. 443).
2. In an explanatory design, the researcher first collects and
analyzes quantitative data, and then obtains qualitative data to
follow up and refine the quantitative findings (p. 443).
For this study, the explanatory design was used. Quantitative data
for this research were gathered through TEA to determine if a
relationship existed between the percentage of English language learners
enrolled in a school and the percentage of all students passing the 10th
grade TAKS tests in the core areas of English Language Arts and
Mathematics given in 2003, 2004, 2005 and 2006. Qualitative data were
obtained through the on-line, open-ended questionnaire and individual
and focus group interviews about the varied ways in which standardized
assessments impacted ELLs.
For the qualitative research component, the study used the cross-
sectional, open-ended questionnaire. A cross-sectional, open-ended
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 84/218
67
questionnaire collects information from a sample that has been drawn
from a predetermined population. Furthermore, the information is
collected at just one point in time, although the time it takes to collect
the data may take anywhere from a day to a few weeks or more (Fraenkel
& Wallen, 2003).
The study also utilized descriptive research methods. Isaac and
Michael (1995, p. 46) describes this type of research as: “to describe
systematically a situation or area of interest factually or accurately.” A
survey study also falls under the classification of descriptive research.
Van Dalen (1979) lists the purpose of survey studies:
1. To collect detailed factual information that describes existing
phenomena.
2. To identify problems or justify current conditions and practices.
3. To make comparisons and evaluations.
4. To determine what others are doing with similar problems or
situations and benefit from their experience in making future
plans and decisions.
Research Design
Since the study utilized the explanatory design of the mixed
methods, the investigator first gathered quantitative data from Texas
Education Agency (TEA) regarding the major urban high schools in
Texas. TEA records personnel assisted in accessing and retrieving data
from the TEA website. Qualitative data were obtained through the on-
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 85/218
68
line, open-ended questionnaire and individual and focus group
interviews; views and opinions of the respondents were gathered and
collated to validate and support the quantitative data.
Quantitative Data
From the Texas Education Agency, the following data regarding the
urban high schools were gathered: the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005 and 2006.
Qualitative Data
Qualitative data were obtained using an on-line, open-ended
questionnaire given to principals, assistant principals, ESL district
personnel, ESL certified teachers and non-ESL certified teachers who
were purposively sampled for the study and through the individual and
focus group interviews using open-ended questions about the varied
ways in which standardized assessments impact ELLs.
Pilot Study
Two Houston Independent School District schools, not included in
the main study were selected for the pilot study. Quantitative data were
obtained regarding the schools’ percentage of English language learners
enrolled in a school and the percentage of all their 10th grade students
passing TAKS in the two core areas of English Language Arts and
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 86/218
69
Mathematics. This was for the four school years starting with the first
school year 2002 – 2003, when TAKS was administered.
During the pilot study the on-line questionnaire underwent pre-
testing with three basic considerations: (1) administered the pre-test
under conditions comparable to those anticipated in the final study; (2)
analyzed the results to assess the effectiveness of the trial questionnaire
to yield the information desired; and (3) made appropriate additions,
deletions, and modifications to the questionnaire (Isaac & Michael,
1995).
Qualitative data resulting from an on-line open-ended
questionnaire on the six different concerns listed below were tabulated
combining the results from the two schools. Results were categorized
using the NVivo software package but the categories were modified based
on the expert opinion of the respondents belonging to the focus groups.
The frequencies for the responses by the different respondents (teachers,
school administrators and district ESL personnel) pertaining to the
different categories were tallied and percentages were computed. Listing
of categories was based on the total frequencies; those categories
identified most by the respondents were listed first followed by those with
lower frequencies. The different concerns included the following: (1)
Purpose of TAKS; (2) Consequences of TAKS; (3) Problems Related to
TAKS; (4) Changes Caused by TAKS; (5) Recommendations to Improve
TAKS; and (6) Needs of ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 87/218
70
Results of the focus group and one-on-one interviews were
validated against the results of the on-line questionnaire and provided
explanation or support for the answers given. The categories for the
different responses were affirmed or modified by the focus groups.
Population and Samples
Quantitative Data
The TEA provided the data on the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005 and 2006. The researcher
used purposive sampling in selecting schools for this study. Purposive
sampling is based on the assumptions that the investigator wants to
discover, understand, and gain insight and therefore must select a
sample from which the most can be learned (Merriam, 1998).
While not a random sampling of high schools, the sample is not
intended to create results that can be generalized to all major urban high
schools in the U.S. A purposive sampling was used in order to provide a
representative sample of the major urban high schools in Texas in order
to gain in-depth insight into what impact might be occurring. The impact
that might emerge from this study might occur in other high schools, but
it is important to take into account the characteristics of the high
schools as well as the assessment system in the state in order to
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 88/218
71
extrapolate from the findings and make comparisons with other
situations (Patton, 1990).
Qualitative Data
The on-line, open-ended questionnaire was given to the principals,
assistant principals, certified ESL teachers and non-ESL certified
teachers handling ELLs of the selected schools and to the district
personnel:
Total
1) ESL Teachers 30
2) Non-Certified ESL Teachers 30
3) Principals 10
4) Assistant Principals 20
5) District ESL Personnel 8
Total
The different focus groups consisted of ESL certified and non-ESL
certified teachers handling ELLs. One-on-one interviews involved the
selected principals and the selected district ESL personnel. The same
schools and district personnel who answered the on-line questionnaire
were included in the focus groups and one-on-one interviews. Selection
of the participants in the focus group interviews utilized the snowballing
technique. Participants will identify others whose input or experience will
also be valuable to the study (Krathwohl, 1993).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 89/218
72
Since the researcher has the obligation to respect and protect the
rights and wishes of the research participants, the following actions were
done: (1) the researcher protected anonymity of the participants by using
computer-given codes for the responses; and (2) the researcher informed
the participants about the purpose of the survey.
The security of the raw data gathered through the records sections
of TEA and the selected schools, responses to the on-line questionnaire
and the transcripts of the interviews was assured in order to protect the
anonymity of the participants and to uphold the trustworthiness of the
study.
The above concerns regarding trustworthiness and confidentiality
of data or information were shared with the participants when the
researcher contacted them through e-mail, telephone, mail, or in person.
Instrumentation
Quantitative data were accessed and retrieved from the TEA
website regarding the major urban high schools in Texas. Data were
organized for computations utilizing the SPSS software package, Version
14.0.
The on-line, open-ended questionnaire provided one of the bases
for the qualitative data. The triangulation method included categorizing
the responses to the online, open-ended questionnaire into emergent
themes, interviewing the focus groups of teachers and assistant
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 90/218
73
principals and one-on-one interviews with the principals and district ESL
personnel.
According to Fraenkel and Wallen (2003) personal interview is
probably one of the most effective ways there is to enlist the cooperation
of respondents in a survey; rapport can be established, questions can be
clarified, unclear or incomplete answers can be followed up and so on.
Patton (1990) expounds that the purpose of interviews is to gain
access to those areas of the participants’ experiences or thought which
cannot be observed. Consequently, interviews will play a significant role
in data collection, a role which generally cannot be duplicated by other
means (Dexter, 1970).
Fraenkel and Wallen (2003) cite the following advantages of open-
ended questions in survey research (a) allows more freedom of response;
(b) easier to construct; and (c) permits follow-up by interviewer. But there
are also disadvantages: (a) responses tend to be inconsistent in length
and content across respondents; (b) both questions and responses may
be subject to misinterpretation; and (c) responses are harder to tabulate
and synthesize. However, these disadvantages can be minimized through
the use of the NVivo software package, expert help from the focus groups
in classifying categories, follow-up interviews with the focus groups and
one-on-one interviews.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 91/218
74
Validity and Reliability
For validity and reliability, the following expert opinions were
considered. “Validity, I mean truth: interpreted on the extent to which an
account accurately represents the social phenomena to which it refers”
(Hammersley, 1990, p. 57). “Reliability refers to the degree of consistency
with which instances are assigned to the same category by different
observers or by the same observer on different occasions” (Hammersley,
1990, p. 67). The triangulation method involving the analysis of the
qualitative data, collation of data from the on-line questionnaire and
interviews assured the validity and reliability of the survey questions and
the explanatory design of the mixed methods study.
For the quantitative dimension of the study, validity and reliability
measurements were derived from the TAKS report prepared by TEA.
Validity is a process of collecting evidence to support inferences made
from scoring results of an assessment. In the case of TAKS, test results
are used to make inferences about the students’ knowledge and
understanding of the Texas Essential Knowledge and Skills (TEKS). Test
reliability indicates the consistency of measurement. TAKS test
reliabilities are based on internal consistency measures, in particular on
the Kuder-Richardson Formula 20 (KR-20) for tests involving
dichotomously scored (multiple choice) items and on the stratified
coefficient alpha for tests involving a mixture of dichotomous and
polytomous (essay-prompt and short answer) items.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 92/218
75
In order to build trustworthiness in the qualitative aspect of the
study, four different criteria were considered to meet this need:
(1)credibility, which aims to produce findings that are believable and
convincing; (2) transferability, which attempts to apply findings in one
setting to other contextually similar settings; (3) dependability, which
addresses the question concerning which findings are consistent with
those of other similar investigations; and (4) confirmability, which
ensures that both the process and the product are auditable (Isaac &
Michael, 1995).
Research Procedures
Quantitative
After appropriate permissions for data gathering were obtained,
records personnel of TEA were contacted and arrangements made as to
process and assistance regarding acquisition of data for the study. The
dry-run or pilot study with the two HISD schools facilitated the above
process.
Qualitative
The questions in a survey, and the way they are asked, are of
crucial importance (Fraenkel & Wallen, 2003). The authors refer to Floyd
Fowler who points out that there are four practical standards that all
survey questions should meet:
1. Is this a question that can be asked exactly the way it is
written?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 93/218
76
2. Is this a question that will mean the same thing to everyone?
3. Is this a question that people can answer?
4. Is this a question that people will be willing to answer, given the
data collection procedures? (Fowler, 1984).
After the questionnaire was refined based on the suggestions of the
focus groups during the pilot study, the questionnaire was placed on-line
to respondents of the study. Prior to this, the researcher contacted the
respondents in person, by phone, by email or mail. Furthermore, the
researcher arranged dates with the different schools and districts for the
focus group and one-on-one interviews.
Data Collection and Recording
Quantitative
The data for the major urban high schools regarding the
percentage of ELLs enrolled in a school and the percentage of all Grade
10 students passing the TAKS tests in English Language Arts and
Mathematics during the four years when TAKS was administered,
starting school year 2002 – 2003 were obtained from the TEA website.
Qualitative
An on-line, open-ended questionnaire was answered by the
principals, assistant principals, ESL teachers, and non-certified teachers
handling ELLs of the selected major urban high schools in Texas. District
ESL personnel were also requested to answer the same questionnaire.
The focus groups offered expert opinions regarding the categories to use
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 94/218
77
in classifying the responses to the questionnaire. Further clarification
was requested from the principals and the district ESL personnel during
the one-on-one interviews.
Results of the questionnaire were placed in categories suggested by
the focus groups after initial classification was done through the NVivo
software system. Transcripts of the interviews and focus groups were
entered into the NVivo software system (version 7.0) and coded according
to themes that emerged in the data. NVivo provides a sophisticated way
of electronically organizing interview transcripts for analysis and
classification into themes and allowed the researcher to work with a large
amount of transcript data. The themes that emerged from the data were
compiled and compared between high schools. While NVivo was a
valuable sorting tool that allowed the researcher to code, sort, and recall
data in different ways, the researcher developed and created codes for
the responses gathered. The analysis was done by the researcher using
NVivo’s capabilities to sort out the complexities of the rich data from the
interviews and focus groups. A program such as NVivo can help the
researcher ensure that the qualitative data were well-organized
(Weitzman, 2000).
One of the strengths of collecting qualitative data is the richness of
the information that can be collected and which can capture a theme in a
more complete way than the researcher may be able to summarize. This
evidence directly from the data was used to show a clear connection
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 95/218
78
between the data and the identified themes (Marshall, 1990). The rich
description of the themes from the participants’ own words also aids in
verifying that the themes identified are those that the participants
actually voiced (Creswell, 1998; Krueger & Casey, 2000).
The researcher triangulated quantitative data analysis, qualitative
data analysis, and interviews in order to strengthen the credibility of the
survey study. By combining multiple observers, theories, methods, and
data sources, researchers can hope to overcome the intrinsic bias that
comes from single-method, single-observer and single-theory studies
(Denzin, 1970). With the mix of analyses, the author has better tools to
discuss the impact of statewide testing on ELLs, ESL curriculum and
instruction.
Data Analysis
Quantitative
Descriptive statistics and analyses were performed to test each
variable. After the data were examined and properly inputted, the next
step was to compute for Pearson r correlation coefficients using the SPSS
statistical package and test for statistical relationship at p < 0.05. For
other analyses, the predictor variable is the percentage of ELLs enrolled
in a school and the outcome variables were the percentages of all
students passing the Grade 10 TAKS tests in English Language Arts and
Mathematics. For each of the years under study, two separate Pearson r
correlations were computed; the percentage of ELLs enrolled in a school
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 96/218
79
was compared with the English Language Arts results to determine if
they have significant relationship and the other comparison was with the
Mathematics results. The SPSS computations showed three different
results in tabular form: (1) the means and the standard deviations of the
percentage of ELLs enrolled in a school and the percentages of all
students passing the TAKS tests in English Language Arts and
Mathematics during the four years under study; (2) Pearson r correlation
coefficients to determine if there was significant relationship between the
percentage of ELLs enrolled in a school and the percentages of all
students passing the English Language Arts and Mathematics TAKS
tests given in 2003, 2004, 2005 and 2006; and, (3) regression analysis
using the percentage of ELLs enrolled in a school as the predictor
variable to predict the percentage of students passing in the 10th grade
TAKS tests in the core areas of English Language Arts and Mathematics.
Qualitative
The information for the qualitative portion of the study included
the emergent themes shown as categories in the frequency distribution
table. The frequency distribution is a table in which all score units are
listed in one column and the number of individuals receiving each score
appears as frequencies in the second column (Isaac and Michael, 1995).
Frequencies were tallied and percentages were computed. Categories
with higher percentages were listed first followed by those with lower
percentages. An overview preceded each table giving the emergent
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 97/218
80
themes mostly cited by the respondents. Anecdotal records followed the
tables - these are the views and opinions of the respondents regarding
the different concerns included in the study.
Summary
In this study the researcher considered the aspects of procedural
consistency, neutrality of findings, and truth value to assure the study of
trustworthiness. “Valid inquiry in any sphere… must demonstrate its
truth value, provide the basis for applying it, and allow for external
judgments to be made about the consistency of its procedures and the
neutrality of its findings or decisions” (Erlandson, 1993).
Quantitative data that were sourced as aggregate data from the
TEA website included the percentage of English language learners
enrolled in a school and the percentage of all students passing the 10th
grade TAKS tests in the core areas of English Language Arts and
Mathematics given in 2003, 2004, 2005 and 2006.
Qualitative data were collated from the responses of selected
respondents to the on-line questionnaire regarding the anticipated and
observed consequences of the statewide testing, specifically TAKS, on
ESL curriculum and instruction as viewed by ESL teachers, school
administrators and district ESL personnel. Interviews were conducted
with the focus groups and one-on-one interviews involved the principals
and district ESL personnel.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 98/218
81
Presentation of data included: (a) the quantitative data analysis on
the correlation between the percentage of ELLs enrolled in a school and
the percentage of all students passing the 10th grade TAKS in English
Language Arts and Mathematics and the regression analysis using the
percentage of ELLs enrolled in a school as predictor variable and the
percentage of all students passing the 10th grade TAKS in ELA and
Mathematics as outcome variables; (b) qualitative data analysis
classifying responses to the on-line, open-ended questionnaire as
different emergent themes; and (c) anecdotal records from the interviews
with the different focus groups, principals, and district ESL personnel.
The relationship between the percentage of ELLs enrolled in a
school and the percentage of all students passing the 10th grade TAKS in
each of the core areas of English Language Arts and Mathematics was
determined using the SPSS program for Pearson r correlation. The
regression analysis resulted to linear regression equations predicted the
percentage of all students passing in 10th grade TAKS in English
Language Arts and Mathematics using the percentage of ELLs enrolled in
a school as predictor variable.
Emergent themes were categorized through the NVivo software
package and suggestions of the focus groups. The anecdotal records
expressed the views and opinions of the respondents regarding the
following and focused on the ELLs: (a) the purpose of the statewide, high
stakes TAKS; (b) intended consequences of TAKS; (c) problems related to
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 99/218
82
TAKS; (d) changes caused by TAKS; (e) recommendations to improve
performance in TAKS; and, (f) the needs of ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 100/218
83
CHAPTER IV
ANALYSIS OF DATA
The purpose of this study was to determine the impact of high-
stakes testing on English Language Learners (ELLs). This was shown in
both the quantitative and qualitative dimensions of the study. Both
quantitative and qualitative dimensions of the study provided the status
of high-stakes testing as it affected ELLs and how it influenced efforts in
schools to improve performance of students, particularly ELLs. Data
obtained from Texas Education Agency (TEA) were used to determine
whether there was a relationship between the percentage of ELLs
enrolled in a school and the percentage of all students passing the 10th
grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core
areas of English Language Arts and Mathematics given in 2003, 2004,
2005, and 2006. To support the qualitative aspect, this study explored
what certified English as a Second Language( ESL) teachers, non-
certified ESL teachers who teach ELLs, administrators, and district ESL
personnel viewed as the impact that high stakes standardized
assessments had on ELLs, ESL curriculum and instruction, and what
they observed as actually occurring.
With the No Child Left Behind (NCLB) Act of 2001, student
achievement has been placed in the forefront of quality education.
Aligned to this effort is statewide testing aimed at assessing students‟
performance and status of the school. In this chapter, the relationships
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 101/218
84
between the percentage of English language learners enrolled in a school
and the percentage of all students passing the 10th grade TAKS tests in
the core areas of English Language Arts and Mathematics given in 2003,
2004, 2005, and 2006 are shown. The results of the Statistical Package
for the Social Sciences (SPSS) computations show the: (1) means and
the standard deviations of the percentage of ELLs enrolled in a school
and the percentages of all students passing the 10th grade TAKS tests in
English Language Arts and Mathematics during the four years under
study; (2) Pearson r correlation coefficients between the percentage of
ELLs enrolled in a school and the percentages of all 10th grade students
passing the English Language Arts and Mathematics TAKS tests given in
2003, 2004, 2005, and 2006; and, (3) regression analysis using the
percentage of ELLs as the predictor variable.
Following this quantitative dimension of the study is the qualitative
information regarding TAKS and its effects on the ELLs - its intended
purpose and consequences, problems encountered related to this
statewide testing, changes that occurred due to TAKS and
recommendations to improve the performance of ELLs in general and
specifically on this high-stakes test. The tabulated results of the open-
ended, on-line questionnaire and the supporting explanation gathered
through the individual and focus group interviews comprised the
qualitative portion of this study. ESL district personnel, principals,
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 102/218
85
assistant principals, ESL-certified teachers and non-ESL certified
teachers handling ELLs provided the needed information.
Schools with ELLs are asked to account for their performance in
line with NCLB. School and district administrators may get feedback
both from the numerical results of high-stakes testing and from inputs of
teachers and other personnel responsible for handling ELLs. Since the
presence of ELLs in schools is a reality administrators have to face, it
may be beneficial to be aware of the views and opinions of concerned
personnel in the school system regarding interventions that may improve
the curriculum and/or instruction of ELLs.
Findings
For the quantitative portion of the study, data for the 173 urban
high schools were obtained from Texas Education Agency; however, for
the year 2005, data available were only for 155 high schools. The data
gathered were collated and the SPSS software package was utilized for
the needed computations. The following research question was the focus
of the quantitative dimension of the study:
Research Question - Quantitative
Is there a relationship between the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005, and 2006?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 103/218
86
Two null hypotheses were formulated to answer the above
question:
HO1: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
English Language Arts given in 2003, 2004, 2005, and 2006.
HO2: There is no statistically significant relationship between the
percentage of English language learners enrolled in a school and
the percentage of all students passing the 10th grade TAKS tests in
Mathematics given in 2003, 2004, 2005, and 2006.
The results are presented in the following order: (a) the descriptive
statistics (mean, standard deviation, number of cases); (b) the Pearson r
correlation coefficients; and, (c) regression analysis showing the linear
equations which can be used to predict the outcomes in the 10th grade
TAKS tests in English Language Arts and Mathematics.
Tables 4.1.1 to 4.1.4 show that the average percentage of ELLs in
the urban high schools of Texas during the four years under study
starting from 2003, ranged from 6.94% to 8.30 %; the average percentage
of ELLs enrolled in a school was 7.79%.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 104/218
87
Table 4.1.1
Means and Standard Deviations of ELLs Enrolled in School and All
Students Passing the 2003 10th Grade TAKS for English Language Arts
and Mathematics.
Mean Std. Deviation N
ELLs Enrolled
In School
08.31 08.98 165
All StudentsPassing ELA 62.87 18.11 151
All Students
PassingMathematics 61.85 20.84 151
Table 4.1.2
Means and Standard Deviations of ELLs Enrolled in School and All
Students Passing the 2004 10th Grade TAKS for English Language Arts
and Mathematics.
Mean Std. Deviation N
ELLs EnrolledIn School
08.30 09.64 147
All StudentsPassing ELA 68.28 16.24 130
All StudentsPassingMathematics 53.57 22.44 132
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 105/218
88
Table 4.1.3
Means and Standard Deviations of ELLs Enrolled in School and All
Students Passing the 2005 10th Grade TAKS for English Language Arts
and Mathematics.
Mean Std. Deviation N
ELLs Enrolled
In School
7.62 10.80 147
All StudentsPassing ELA 59.39 20.13 135
All Students
PassingMathematics 47.68 23.35 134
Table 4.1.4
Means and Standard Deviations of ELLs Enrolled in School and All
Students Passing the 2006 10th Grade TAKS for English Language Arts
and Mathematics.
Mean Std. Deviation N
ELLs EnrolledIn School
6.94 9.94 167
All StudentsPassing ELA 78.05 13.68 155
All StudentsPassingMathematics 50.13 22.40 152
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 106/218
89
Tables 4.1.1 through 4.1.4 reflect the descriptive statistics showing
the percent of all students passing in English Language Arts were:
62.87% in 2003, 68.28% in 2004, 59.39% in 2005 and 78.05% in 2006.
The descriptive statistics showing the results of all students passing in
Mathematics were: 61.85% in 2003, 53.57% in 2004, 47.68% in 2005
and 50.13% in 2006.
Based on the above tables, Tables 4.1.1.1 and 4.1.1.2 compare the
results of 10th grade TAKS in English Language Arts and Mathematics.
Table 4.1.1.1
Comparison of Results in 10th Grade English Language Arts TAKS
Year
PercentELLs
Enrolledin
School
Means-AllStudentsPassingCurrent
Year
Means-AllStudentsPassingPrevious
YearDifference
(+/-)
PercentIncrease
(Decrease)
2003 8.30 62.87 -- -- --
2004 8.29 68.28 62.87 +5.41 08.61
2005 7.62 59.39 68.28 -8.89 -13.02
2006 6.94 78.05 59.39 +18.66 031.42
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 107/218
90
Table 4.1.1.2
Comparison of Results in 10th Grade Mathematics TAKS
Year
Percent
ELLsEnrolled
inSchool
Means-All
StudentsPassingCurrent
Year
Means- All
StudentsPassingPrevious
YearDifference
(+/-)
PercentIncrease
(Decrease)
2003 8.30 61.85 -- -- --
2004 8.29 53.57 61.85 -8.28 -13.39
2005 7.62 47.68 53.57 -5.89 -10.99
2006 6.94 50.13 47.68 +2.45 05.14
Tables 4.2.1 to 4.2.4 show the obtained Pearson r correlation
coefficients for the correlations between percent of ELLs enrolled in a
school and the percent of all students passing in 10th grade TAKS test in
English Language Arts and Mathematics. The obtained Pearson r
correlation coefficients between the percentage of ELLs enrolled in a
school and the percentage of all students passing in 10th grade TAKS
were: -0.349 in 2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006
for English Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in
2005 and -0.356 in 2006 for Mathematics. All the computed Pearson r
correlation coefficients were significant at the 0.05 level, two-tailed. We
therefore reject the null hypotheses. All the obtained Pearson r
correlation coefficients were negative for both content areas; this inverse
relationship indicated that as the percentage of ELLs increased, the
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 108/218
91
percentage of all 10th grade students passing in each of the core areas of
English Language Arts and Mathematics decreased.
Table 4.2.1
Pearson Correlations: 2003 10th Grade TAKS for English Language Arts
and Mathematics.
Pearson Correlation
Percent of All
StudentsPassing ELA
Percent of All
StudentsPassing Math
Percent of ELLs Enrolled -.349(**) -.293(**)
Sig. (2-tailed) .00000 .00000
** Correlation is significant at the 0.05 level (2-tailed).
Table 4.2.2
Pearson Correlations: 2004 10th Grade TAKS for English Language Arts
and Mathematics.
Pearson Correlation
Percent of AllStudents
Passing ELA
Percent of AllStudents
Passing Math
Percent of ELLs Enrolled -.392(**) -.351(**)
Sig. (2-tailed) .00000 .00000
** Correlation is significant at the 0.05 level (2-tailed).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 109/218
92
Table 4.2.3
Pearson Correlations: 2005 10th Grade TAKS for English Language Arts
and Mathematics.
Pearson Correlation
Percent of AllStudents
Passing ELA
Percent of AllStudents
Passing Math
Percent of ELLs Enrolled -.297(**) -.382(**)
Sig. (2-tailed) .00000 .00000
** Correlation is significant at the 0.05 level (2-tailed).
Table 4.2.4
Pearson Correlations: 2006 10th Grade TAKS for English Language Arts
and Mathematics.
Pearson Correlation
Percent of AllStudents
Passing ELA
Percent of AllStudents
Passing Math
Percent of ELLs Enrolled -.398(**) -.356(**)
Sig. (2-tailed) .00000 .00000
** Correlation is significant at the 0.05 level (2-tailed).
A regression analysis for the data in 2003 is shown in Tables 4.2.5
and 4.2.6. The correlation coefficient between the percentage of ELLs
enrolled in a school and the percentage of all students passing in 10th
grade TAKS in English Language Arts for 2003 is -0.349. The Pearson r
between the percentage of ELLs enrolled in a school and the percentage
of all students passing in 10th grade TAKS in Mathematics is -0.293.
Both correlation coefficients are significant at p < 0.05, two-tailed.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 110/218
93
The percentage of all students passing in 10th grade English
Language Arts TAKS can be predicted using the linear regression
equation: Ŷ = 68.71 – 0.70X, where X is the percentage of ELLs enrolled
in a school. For example, using the percentage of ELLs data for 2003
(X = 8.30), the predicted value for percent of all students passing 10th
grade English Language Arts is equal to 68.71 – 0.70(8.30) or 62.90.
Actual result was 62.87. The percentage of all students passing in 10th
grade Mathematics TAKS can be predicted using the linear regression
equation: Ŷ = 67.49 – 0.68X, where X is the percentage of ELLs enrolled
in a school. A similar example for Mathematics gives a predicted value of
67.49 – 0.68(8.30) or 61.85. Actual result was 61.85.
Table 4.2.5
Coefficients for Percentage of All Students Passing the 2003 10th Grade
English Language Arts TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
T Sig.
B Std.Error
Beta
1 (Constant) 68.71 1.89 36.37 .000
Percent ELLs 0 -.70 0.16 -.349 0-4.55 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
English Language Arts TAKS
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 111/218
94
Table 4.2.6
Coefficients for Percentage of All Students Passing the 2003 10th Grade
Mathematics TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
T Sig.
B Std.Error
Beta
1 (Constant) 67.49 2.22 30.43 .000
Percent ELLs 0 -.68 0.18 -.293 -3.74 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
Mathematics TAKS
Results of the regression analysis for the data in 2004 are shown
in Tables 4.2.7 and 4.2.8. The correlation coefficient between the
percentage of ELLs enrolled in a school and the percentage of all
students passing in 10th grade TAKS in English Language Arts for 2004
is -0.392. The Pearson r between the percentage of ELLs enrolled in a
school and the percentage of all students passing in 10th grade TAKS in
Mathematics is -0.351. Both correlation coefficients are significant at
p < 0.05, two-tailed.
The percentage of all students passing in 10th grade English
Language Arts TAKS can be predicted using the linear regression
equation: Ŷ = 73.76 – 0.66X, where X is the percentage of ELLs enrolled
in a school. For example, using the percentage of ELLs data for 2004
(X = 8.29), the predicted value for percent of all students passing 10th
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 112/218
95
grade English Language Arts is equal to 73.76 – 0.66(8.29) or 68.29.
Actual result was 68.28. The percentage of all students passing in 10th
grade Mathematics TAKS can be predicted using the linear regression
equation: Ŷ = 60.34 – 0.82X, where X is the percentage of ELLs enrolled
in a school. A similar example for Mathematics gives a predicted value of
60.34 – 0.82(8.29) or 53.54. Actual result was 53.57.
Table 4.2.7
Coefficients for Percentage of All Students Passing the 2004 10th Grade
English Language Arts TAKS.
Model Unstandardized
Coefficients
Standardized
Coefficients
T Sig.
B Std.Error
Beta
1 (Constant) 73.76 1.74 42.46 .000
Percent ELLs 0 -.66 0.14 -.392 -4.83 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
English Language Arts
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 113/218
96
Table 4.2.8
Coefficients for Percentage of All Students Passing the 2004 10th Grade
Mathematics TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
T Sig.
B Std.Error
Beta
1 (Constant) 60.34 2.42 24.90 .000
Percent ELLs 0-.82 0.19 -.351 0-4.28 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
Mathematics TAKS
A regression analysis for the data in 2005 is shown in Tables 4.2.9
and 4.2.10. The correlation coefficient between the percentage of ELLs
enrolled in a school and the percentage of all students passing in 10th
grade TAKS in English Language Arts for 2005 is -0.297. The Pearson r
between the percentage of ELLs enrolled in a school and the percentage
of all students passing in 10th grade TAKS in Mathematics is -0.382.
Both correlation coefficients are significant at p < 0.05, two-tailed.
The percentage of all students passing in 10th grade English
Language Arts TAKS can be predicted using the linear regression
equation: Ŷ = 63.61 – 0.55X, where X is the percentage of ELLs enrolled
in a school. For example, using the percentage of ELLs data for 2005
(X = 7.62), the predicted value for percent of all students passing
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 114/218
97
10th grade English Language Arts is equal to 63.61 – 0.55(7.62) or 59.42.
Actual result was 59.39. The percentage of all students passing in 10th
grade Mathematics TAKS can be predicted using the linear regression
equation: Ŷ = 53.97 – 0.83X, where X is the percentage of ELLs enrolled
in a school. A similar example for Mathematics gives a predicted value of
53.97 – 0.83(7.62) or 47.65. Actual result was 47.68.
Table 4.2.9
Coefficients for Percentage of All Students Passing the 2005 10th Grade
English Language Arts TAKS.
Model Unstandardized
Coefficients
Standardized
Coefficients
t Sig.
B Std.Error
Beta
1 (Constant) 63.61 2.03 31.28 .000
Percent ELLs 0-.55 0.15 -.297 0-3.59 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
English Language Arts TAKS
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 115/218
98
Table 4.2.10
Coefficients for Percentage of All Students Passing the 2005 10th Grade
Mathematics TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
t Sig.
B Std.Error
Beta
1 (Constant) 53.97 2.29 23.54 .000
Percent ELLs 0-.82 0.17 -.382 -4.74 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
Mathematics TAKS
To show that the percentage of ELLs enrolled in a school affected
the percentage of all students passing the 10th grade TAKS tests in
English Language Arts and Mathematics, the regression analysis results
are shown in Table 4.2.11 and Table 4.2.12. The correlation coefficient
between the percentage of ELLs enrolled in a school and the percentage
of all students passing in 10th grade TAKS in ELA for 2006 is -0.398. The
Pearson r between the percentage of ELLs enrolled in a school and the
percentage of all students passing in 10th grade TAKS in Mathematics is
-0.356. Both correlation coefficients are significant at p < 0.05, two-
tailed.
The percentage of all students passing in 10th grade English
Language Arts TAKS can be predicted using the linear regression
equation: Ŷ = 81.85 – 0.55X, where X is the percentage of ELLs enrolled
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 116/218
99
in a school. For example, using the percentage of ELLs data for 2006
(X = 6.94), the predicted value for percent of all students passing 10th
grade ELA is equal to 81.85 – 0.55(6.94) or 78.03. Actual result was
78.05. The percentage of all students passing in 10th grade Mathematics
TAKS can be predicted using the linear regression equation:
Ŷ = 55.70 – 0.80X, where X is the percentage of ELLs enrolled in a
school. A similar example for Mathematics gives a predicted value of
55.70 – 0.80(6.94) or 50.15. Actual result was 50.13.
Table 4.2.11
Coefficients for Percentage of All Students Passing the 2006 10th Grade
English Language Arts TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
t Sig.
B Std.Error
Beta
1 (Constant) 81.85 1.24 66.28 .000
Percent ELLs 0-.55 0.10 -.398 -5.37 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
English Language Arts TAKS
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 117/218
100
Table 4.2.12
Coefficients for Percentage of All Students Passing the 2006 10th Grade
Mathematics TAKS.
Model UnstandardizedCoefficients
StandardizedCoefficients
t Sig.
B Std.Error
Beta
1 (Constant) 55.70 2.08 26.78 .000
Percent ELLs 0-.80 0.17 -.356 -4.67 .000
a. Predictor: Percentage of ELLs Enrolled in a School
b. Outcome variable: Percentage of All Students Passing in 10th Grade
Mathematics TAKS
Research Question – Qualitative
After the refinements were done during the pilot study, an open-
ended, on-line questionnaire was given to the principals, assistant
principals, ESL district personnel, ESL certified teachers and non-ESL
certified teachers who were purposively sampled from urban high schools
in Texas.
The major question answered by this study was: What are the
anticipated and observed consequences of the statewide testing
specifically, TAKS, on ESL curriculum and instruction as viewed by
certified ESL teachers, non-certified ESL teachers who teach ELLs,
school administrators, and district ESL personnel?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 118/218
101
This major question was explored using the following probes:
1. Why is TAKS given as a statewide test?
2. What are the intended consequences of this statewide testing? (Or
what has happened because of TAKS?)
3. What problems have occurred related to or because of TAKS?
4. What changes were caused by this statewide testing?
5. What are your recommendations to improve this statewide testing?
6. What needs to be done for the ESL students to improve their
performance in general and specifically for this statewide test?
Before the above questions for the qualitative study are answered,
the following tables give the demographic information regarding the
respondents.
Demographic Characteristics of the Respondents
Demographic information regarding the respondents included
gender, age, current position, highest degree earned, years of experience
in public education and certification(s) achieved.
The open-ended, on-line questionnaire for the qualitative aspect of
the study was intended for 98 respondents. Six principals responded,
together with 9 assistant principals, 6 ESL district personnel, 15 ESL
certified teachers and 19 non-ESL certified teachers – a total of 55
respondents, for a response rate of 56%.
Table 4.3 shows that 69% of the respondents were female and 31%
were male.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 119/218
102
Table 4.3
Distribution of Respondents by Gender.
Gender Frequency Percent (%)
Female 38 69
Male 17 31
N=55
The age of the respondents is shown in Table 4.4. With respect to
age, 31% of the respondents were between 41 – 50 years old and 51 – 60
years old. The group of 31 – 40 years old comprised 22% of the
respondents; 9% of the respondents were older than 60 years old and the
youngest between 21 – 30 years old was 7% of the group.
Table 4.4
Distribution of Respondents by Age.
Age Frequency Percent (%)
21-30 04 07
31-40 12 22
41-50 17 31
51-60 17 31
> 60 05 09
N=55
It is shown in Table 4.5 that the non-ESL certified teachers
comprised 35% of the respondents and 27% were ESL certified teachers.
The administrators accounted for the remaining 38%: 16% were
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 120/218
103
assistant principals, 11% were principals, and 11% were ESL district
personnel.
Table 4.5
Distribution of Respondents by Professional Position.
Current Position Frequency Percent (%)
Principal 06 11
Assistant Principal 09 16
ESL District Personnel 06 11
ESL Certified Teacher 15 27
Non-ESL Certified Teacher 19 35
N=55
Table 4.6 displays that 53% of the respondents have Master‟s
degrees, 40% have Bachelor‟s degrees and 7% have Doctorate degrees.
Table 4.6
Distribution of Respondents by Highest Degree Earned.
Highest Degree Frequency Percent (%)
Bachelor‟s Degree 22 40
Master‟s Degree 29 53
Doctorate Degree 04 07
N=55
Considering years of experience in public education, it is shown in
Table 4.7 that 40% of those who participated in the study had more than
20 years of public school experience. Thirty-one percent had 11 – 20
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 121/218
104
years of public school experience, 16% had 5 years or less, and 13% had
been in the public schools for 6 – 10 years.
Table 4.7
Distribution of Respondents by Years of Experience in Education.
Years of Experience Frequency Percent (%)
1 – 5 09 16
6-10 07 13
11-20 17 31
> 20 22 40
N=55
Table 4.8 displays the certifications of the ESL district personnel,
principals, assistant principals, ESL certified and non-ESL certified
teachers who handle ELLs that participated in this study. Some of these
administrators and teachers had more than one certification. Based on
total number of respondents, 47% had certification in English Language
Arts, 20% in Mid-Management, 18% in Mathematics, 16% in
Principalship, and 13% in Special Education.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 122/218
105
Table 4.8
Distribution of Respondents by Certification.
Certification Frequency
English Language Arts 26
Mid-Management 11
Mathematics 10
Principal 09
Special Education 07
Bilingual 02
Educational Diagnostician 02
Biology 02
Sociology 02
Supervisor 02
Reading 02
Librarian 02
Superintendent 02
History 02
Technology 01
Marketing
Counseling
1
1
Vocational Home Economics 01
Social Studies Composite 01
N=55
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 123/218
106
The findings regarding the qualitative portion of the study were
presented as follows: (1) the nine open-ended responses to the on-line
questionnaire; (2) frequency tables showing the emergent themes,
frequencies and percentages; and, (3) anecdotal views and opinions of
district ESL personnel, principals, assistant principals, ESL-certified
teachers and non-certified ESL teachers handling ELLs regarding the
issues or concerns.
Answers of some respondents belonged to more than one
emergent theme; the total number of answers may have exceeded the
total number of respondents. The percentage shown after the total
responses given for each emergent theme was computed based on total
number of respondents.
Why is Texas Assessment of Knowledge and Skills (TAKS) given as a
Statewide Test?
Why is TAKS given as a statewide test? Responses are shown in
Table 4.9. Of the 55 respondents, 40% viewed TAKS as a tool to gauge
knowledge in the core academic areas. Twenty-nine percent (29%) of the
respondents considered TAKS as a means to determine the school‟s
effectiveness and performance (Exemplary, Recognized, Academically
Acceptable, or Academically Unacceptable), while 16% regarded TAKS as
an instrument to appraise the effectiveness of the state curriculum. Only
7% indicated that this statewide test is mandated by law and is aligned
with NCLB.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 124/218
107
Table 4.9
Why is TAKS Given as a Statewide Test to ELLs?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach (15) NonTeach (19) Tot/%(55)
Gauge Skills/TEKS 3 3 3 6 7 22/40
School Accountability 0 2 6 3 5 0 16/29
Assess State Curri-Std 1 1 0 3 4 00 9/16
Mandated by NCLB 1 0 1 0 2 04/7
Com Schools/Districts 1 0 1 1 0 0003/5
Generate Revenue 1 0 1 0 1 0003/5
Test College Prep 0 0 1 1 0 00 2/4
Political Appeasement 0 0 0 1 0 00 1/2
No Response 0 0 0 0 1 00 1/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 125/218
108
One ESL district personnel (EDP-03) gave this opinion about TAKS:
“It is administered to measure the knowledge and skills of Texas
students,” and “it provides a comparative data table for critics.” An
assistant principal (AP-06) supported this view: “TAKS is to help gauge
students‟ basic skills” and “to make schools accountable for students‟
learning.”
Two high school principals (P-06 and P-04) referred to the TAKS as
an instrument of accountability: “…currently, TAKS is the State
Accountability System test in order to comply with NCLB.”
A more comprehensive response regarding the purpose of TAKS
was given by non-ESL certified teacher (non-ESL-14) who handles ELLs:
“At the state level, the TAKS is the state education agency‟s tool for
determining the accreditation status of schools and school
districts…With the enactment of the No Child Left Behind Act, the
TAKS is now being used also as a tool, among other indicators, for
determining the attainment or non-attainment of an adequate
yearly progress (AYP). It is expected that a school carrying
standards-based curriculum and intervention, especially one that
is predominantly attended by minority populations, would show
improving performance until the targeted year 2013 - 2014 when
all children have met standards, which is the essence of the phrase
„ No Child Left Behind.‟ The TAKS performance is disaggregated into
subgroups, so that again, there is a way of determining whether
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 126/218
109
achievement gaps are being bridged. Much of the accountability for
student improvement on knowledge and skills in the four academic
areas falls on schools. The other indicators of adequate yearly
progress are completion rate and attendance.”
In addition to being an instrument of accountability, respondents
suggested that assessment results provide data for comparative analysis.
According to one assistant principal (AP-09), TAKS is given “to
compare/contrast statewide similarities and differences” in achievement
levels “between schools” and across districts.
During one of the focus group sessions, teachers gave these
comments regarding TAKS:
“The test administration is the same for all students. We have to
follow the guide from the state. But, getting the kids ready for the
test is different. We don‟t do anything differently, but we should.
It‟s going to take more than what we are doing right now to ensure
these kids (ELLs) graduate with the same diploma, meeting the
same standards, as the general population.”
Knowing what TAKS is for, does not necessarily translate into
unanimity in terms of agreeing on what schools must do to respond to
the challenge that the test brings. Respondents‟ opinions however ,
indicate the sense that schools need to adapt, and do things differently if
they expect to achieve some level of parity between the TAKS
achievement scores of ELLs and their non-ELL counterparts.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 127/218
110
Anticipated Results of ELLs on English Language Arts and Mathematics
TAKS
Table 4.10 shows the responses to the following question: What
are the anticipated results on the statewide testing, specifically English
Language Arts and Mathematics portion of the Texas Assessment of
Knowledge and Skills (TAKS), on English language learners (ELLs)?
Almost half (49%) of those who responded expected ELLs to have at least
average scores in TAKS. However, 16% observed ELLs‟ performance to be
lower than non-ELLs, and 15% projected low results for ELLs because of
their lower English language skills.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 128/218
111
Table 4.10
What are the Anticipated Results of Statewide Testing for ELLs?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Average Scores 1 2 3 3 7 16/29
Everyone Pass/Meet AYP 2 1 3 3 2 0 11/20
Lower than Non-ELLs 1 2 0 3 3 009/16
Low: Language Barrier 1 1 1 3 2 08/15
Curriculum Improvement 2 0 0 0 2 0004/7
Dismal 0 0 1 0 1 0002/4
Cause Dropouts 1 0 0 1 0 00 2/4
Gap in Scores to Lessen 1 0 1 0 0 00 2/4
Team Effort Improvement 0 0 0 0 2 00 2/4
Political 0 0 0 1 0 00 1/2
Modified Diploma 0 0 0 1 0 00 1/2
Not Sure 0 0 0 1 0 00 1/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 129/218
112
One high school principal (P-04) anticipated that the: “ results of
statewide testing are that all students will be expected to meet minimum
state standards.” Another high school principal (P-02) expressed with
some sense of exasperation that:
“English language learners tend to score extremely low on the ELA
and Mathematics portions of the TAKS. The state and the federal
governments expect us to work miracles with these groups of
students. If a student is in the U.S. for a year or two, he/she needs
extra remediation to get a significant gain in TAKS scores. He may
not necessarily pass the TAKS, but he will make gains. Keep in
mind the NCLB ruling. Making any gains is not enough. The
students must perform at 70% or better on all four sections of
TAKS. At this point, all administrators in poverty-stricken
areas are in fear of losing their jobs due to the low TAKS scores.
We are indeed under extreme pressure!”
One principal (P-05) indicated that the TAKS “will have a negative
impact on the English language learners because of their difficulty in
reading and writing in English.” The same negative outlook was
expressed by an assistant principal (AP-03) who bluntly predicted that:
“ELL students will fare worse than the general population.”
A non-ESL teacher handling ELLs (Non-ESL-14) proposed a
measure to address the feared, and somewhat expected under-
achievement of ELL Learners:
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 130/218
113
“ Title I programs are instituted to address the intervention needs
of ELLs so that they perform well in statewide testing. The said
federal assistance is aimed at incrementally improving the
performance of disadvantaged groups, which include the ELLs and
the ethnic population they comprise. In my particular school,
intervention translates into identifying those who are at risk in this
group, breaking down their previous performances into specific
areas and conducting the needed mentoring.”
During a focus group interview, one member of the group
expressed this view regarding the anticipated results TAKS :
“I think that if we are going to require ELLs to be as successful on
TAKS as any native English speaker then those students need
specific language classes in school. ELLs need English classes
whose sole purpose is teaching them to understand and speak
English, with basic grammar skills included. In addition to a
language class, they do still need a literature-based class in which
they are learning to read and analyze text, as well as write both
creatively and analytically. Also, having tutoring available, either
built into the school day, or after school, during which ELLs
practice speaking and listening in English would do a world of
good. Teachers would not need to be the only tutors either, we
could tap into a college‟s community service group, or even take
advantage of peer tutoring.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 131/218
114
Actual Results of ELLs in English Language Arts and Mathematics TAKS
Table 4.11 summarizes the responses for the question: What are
the actual results of TAKS, specifically English Language Arts and
Mathematics, on ELLs? Responses were related to how the ELLs in the
schools actually performed on TAKS. Results were more negative than
positive.
Twenty-two percent (22%) observed dismal or failing performance, 18%
noticed performance of the ELLs to be lower than non-ELLs and 11%
mentioned that low results frustrate ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 132/218
115
Table 4.11
What are the Actual Results of Statewide Testing for ELLs?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Dismal/Failing 0 1 5 1 5 12/22
Lower than Non-ELLs 3 1 1 3 2 0 10/18
Average Results 0 2 2 2 3 009/16
No Response 0 0 0 4 4 08/15
Frustrated ELLs 3 0 0 2 1 0006/11
Better than Average 0 2 1 0 0 000 3/5
Don‟t Know 0 0 0 0 2 00 02/4
Gap Narrowed 0 0 0 0 2 00 02/4
Showed Areas to Improve 0 0 0 1 0 00 01/2
Political 0 0 0 1 0 00 01/2
Teacher Teach TAKS 0 0 0 1 0 00 01/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 133/218
116
Actual results in TAKS by ELLs are shared by an assistant
principal (AP-03): “ELL students do fare worse than the general
population.” The same observation was given by a non -ESL certified
teacher (Non- ESL- 03):
“Actual results show that ELLs perform poorer than native
speakers and naturally so for they are very much handicapped as
far as their understanding of the intricacies of the language is
concerned. Their scores result in schools being „below standard‟, so
to speak.”
A non-ESL certified teacher (Non-ESL-14) offered a contrasting
view by sharing the actual experience in their district:
“I do not have the exact statistical report as far as the
performances of ELLs are concerned, but my school district came
up with a report lauding the improvement across grade levels in
Mathematics and English Language Arts among ELLs, except in 9th
Grade Mathematics. The district superintendent also mentioned
about the narrowing of the achievement gap, and cited the
percentage points of improvement.”
Intended Consequences of TAKS for ELLs
Table 4.12 summarizes the responses to the question: What are
the intended consequences of this statewide testing as it relates to ELLs?
With respect to the intended consequences of TAKS, 18% of the
respondents expressed the understanding that statewide testing is
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 134/218
117
intended to eventually result in ELLs performing as well as the rest of the
students and 18% agreed that TAKS is a tool to assess ELLs. Eleven
percent (11%) of the respondents expressed confidence that ELLs can
improve academically and eventually join the mainstream, and 11%
demonstrated awareness that TAKS may improve the graduation rate.
Teachers (9% of the respondents) expressed their wish for an opportunity
to teach ELLs individually with the end in mind of improving their
performance in TAKS.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 135/218
118
Table 4.12
What are the Intended Consequences of TAKS for ELLs?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Same Level of Expectations 1 2 0 4 3 10/18
Assess ELLs 1 1 1 4 3 0 10/18
ELLs Improve-Mainstream 2 0 1 1 2 0006/11
Improve Graduation Rate 0 1 1 2 2 006/11
ELLs Learn English ASAP 0 0 2 2 2 00 06/11
School Accountability 1 1 1 1 1 0000 5/9
Individual Teaching of ELLs 0 0 0 2 3 00 005/9
Equal Learn-Opportunities 0 1 1 0 1 0 003/5
Improve Teacher Practices 0 0 1 1 1 00 003/5
Not Sure 1 0 1 0 1 0 03/5
No Response 0 0 0 0 1 00 1/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 136/218
119
An assistant principal (AP- 09) stated that statewide testing is
intended to ensure equality of learning opportunities for minority groups.
Another assistant principal (AP-03) saw the tests as eventually resulting
in the elevation of standards and the improvement of ELL students‟
English language skills. A number of non-ESL certified teachers held
similar views:
“With this statewide testing, ELLs can show how much they have
gotten into the „mainstream‟ of the American educational system.
In so doing, the government will get an „assurance‟ that graduates
of this educational system (or future adults of this society) are
competent and can function well in… this system” (Non-ESL-03).
“The ultimate objective of this educational reform is for everyone...
particularly minority populations, to be educationally competent so
that they become more and more socio-economically integrated
into the American mainstream” (Non-ESL-14).
While the respondents demonstrated competent awareness of the
intended consequences of the TAKS, a few were quick to point out the
challenges that testing poses. A principal (P-02) stated:
“I agree with the No Child Left Behind. We care about each and
every student, but you can‟t compare a 5 th generation student with
a student who has only experienced a month or two in the U.S.A.
and cannot speak the English language or may not have had
formal education in his/her home country. These students are an
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 137/218
120
extremely difficult challenge. This challenge is rarely seen in the
suburban schools.”
On the other hand, one non-ESL certified teacher (Non-ESL-09)
observed that the intended consequence of the TAKS:
“Is to let ELL students stay in high school for more than 4 years
because they can not graduate without passing the TAKS exit
level…it should serve as a challenge to ELL kids to strive more and
work harder because …we all agree that at some point,
Mathematics in itself is a language…so anybody who has this class
is learning a new language… ELLs should not use language as a
barrier to learning Mathematics.”
An English teacher of a focus group expressed this opinion
regarding how statewide assessment affects ELLs:
“Statewide assessments that are intended for non -ESL students
have detrimental effects on the ELL populations. Many are indeed
good students, but they get their results back and only see failure.
The educator also sympathizes because he/she sees firsthand how
far the student has come, but there is no state test for progress. It
becomes frustrating for all parties involved. Sometimes
administrators assume the teacher is not doing his/her job
because the scores come back so low.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 138/218
121
What Has Happened to TAKS, As It Relates to ELLs
What has happened because of TAKS, as it relates to ELLs?
Responses given to this question are shown in Table 4.13. Twenty-six
percent (26%) of the respondents reported that there is a high failure rate
among ELLs. Other respondents observed that ELLs experienced a
diminished self-esteem because of low scores (reported by 11% of the
respondents) and tests caused frustration and exasperation on the part
of ELLs (also seen by 11% of the group). There is also pressure on ELLs
and the schools they attend to improve performance as noted by 11% of
the respondents. Eleven percent (11%) of the respondents reported that
there is now an overwhelming emphasis on test performance in their
campuses.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 139/218
122
Table 4.13
What Has Happened to ELLs Because of TAKS?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Higher Failure Rate-ELLs 1 1 5 3 4 14/26
Dropout Rate Increased 2 1 1 1 2 0 7/13
ELLs Have Low Self-Esteem 0 1 1 1 3 006/11
Pressure on School/ELLs 0 1 1 1 3 06/11
Emphasis Test Performance 1 0 0 3 2 00 6/11
Frustration/Exasperation 1 1 0 2 2 00 6/11
Focus on Needs of ELLs 1 1 0 1 1 00 4/7
Teachers Teach to TAKS 0 0 0 2 2 00 4/7
Continue Testing Changes 0 0 1 0 2 00 03/5
ELLs Repeat TAKS Often 0 0 0 2 0 00 02/4
Acceptable Test Scores 0 1 0 1 0 00222/4
Get Rid of TAKS 0 0 1 1 0 00 2/4
No Response 0 0 0 1 1 00 2/4
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 140/218
123
District personnel (EDP-03) noted that “many of the ELL students
have dropped out, fallen behind and in some instances stopped caring.
Those who successfully passed the TAKS are concerned with their family
and friends, so they share the failure. An assistant principal (AP-02)
stated that TAKS has resulted in an “increase in dropout rate, low self
esteem.”
An ESL teacher (ESL-01) has observed that “many ELLs fear TAKS
and do not think they will ever be able to pass it… many drop out of
school altogether.” Another ESL teacher (ESL-05) noticed something
similar: “Every year ELLs either pass and exit the program, or fail… they
experience a sense of failure.” “The test has caused frustration and
exasperation on the part of many students because the stakes are too
high. They are only judged on this one test” (ESL – 10). Similarly, a non-
ESL teacher (Non- ESL- 03) observed that:
“Because of TAKS (and its results), ELLs have become the “trail
enders” (the ones at the end of the trail). Not performing well in
these tests and being “left behind” by the “better” students has
given them lower self-esteem and feelings of frustration. Failing the
test one too many times may result in feelings of frustration and,
possibly, rejection on their part.”
An ESL teacher (ESL-07) observed that “ELLs have been
penalized, a large portion must re-take TAKS multiple times in order to
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 141/218
124
pass, and schools have been held accountable for the failure of that
population of students.”
A more optimistic response was expressed by a non-ESL teacher
(Non- ESL-14) regarding what happened to ELLs because of TAKS:
“School administrators and educators have become more aware of
the standards and expectations that students must reach for them
to become competent human resource in the workplace and
society. Not only has there been a gap among ethnic populations,
with the ELLs at the disadvantaged end of the spectrum; there has
also been a gap between competencies of exiting high school
students and the actual skills and scholastic aptitude required in
postsecondary education. Falling on the shoulder of the academic
workforce, the TAKS has pushed these school stakeholders to
intensify their efforts of addressing the needs of ELLs. Student
expectations and academic objectives are being laid out more
clearly, and their assessments more meticulously scrutinized with
students, as part of instruction, in order for them to become more
aware and self-reflective in their learning. There is a culture of
concern for statistical performance, broken down furthermore into
numbers vis-à-vis chunks of knowledge and skills in a specific
subject area. Even ELLs have become familiar with the language of
testing.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 142/218
125
Problems of ELLs, Due to TAKS
What problems have occurred, if any, to ELLs related to or because
of TAKS? Responses are summarized in Table 4.14. Of the 55
respondents, 24 % mentioned the higher dropout rate as one of the
problems experienced by ELLs due to TAKS. At least one from each group
expressed this problem. Graduation rate decreased for ELLs due to this
statewide test - this was echoed by 22% of the respondents. Another
negative aspect is perceived to be the low esteem of ELLs as seen by 13%
of the group; 15% of the respondents realized the need to support ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 143/218
126
Table 4.14
What Problems Have Occurred for ELLs Due to TAKS?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Higher dropout rate 3 2 1 4 3 13/24
Low graduation rates 2 0 1 4 4 0 11/20
Support for ELLs 0 1 1 4 2 008/15
Low self-esteem 0 1 1 3 2 07/13
Instruction is test oriented 1 0 0 2 1 00004/7
Increased achievement gap 0 0 2 1 1 000 4/7
Ostracized due to low scores 0 0 1 0 2 00 03/5
Tutorials/staff development 0 1 0 1 1 00 03/5
True ability not measured 0 0 2 0 1 00 03/5
No problem 0 2 0 0 1 00 03/5
Limited reading ability 0 0 0 0 2 00 02/4
Rush ELLs to learn English 0 0 0 0 1 01001/2
No Answer 0 0 0 0 1 01001/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 144/218
127
Problems experienced by principals included the following: “Some
have dropped out; others have done well enough to finish high school but
fear college. Others have lost all hope and courage to continue” (P-03).
“The results of TAKS have caused the English language learners to feel
as if they are failures” (P-05).
A non-ESL teacher (Non-ESL-03) observed the same problem:
“ELLs … may just disengage themselves from any school activities and
studies, and may, ultimately, check out of school.” This was echoed by
an assistant principal (AP-07) who observed that ELLs “leave high school,
complete all credits but do not receive diploma due to failing statewide
exam.” Though challenged, a principal (P-02) remained optimistic:
“Some of our English language learners‟ hopes and dreams of
passing the state exams never become a realization, but we don‟t
give up on them. We keep them and continue to work with them
individually so that they may succeed.”
The ELLs‟ performance in TAKS was explained by a non -ESL
teacher (Non-ESL-14):
“ELLs enroll into their grade levels not quite equipped with the
prerequisite skills to perform” according to standards. “Even with
the administration of the TAKS in the native language of the ELLs,
there still exists an academic gap, because the testing language is
not essentially native but academic, an orientation with which
ELLs are not familiar. Only 4 years old, the TAKS as a federal tool
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 145/218
128
for adequate yearly progress is battling against a culture (focused
on) economic survival. The TAKS measures scholastic aptitude and
academic achievement - attributes that make up the profile for
college readiness, or postsecondary education.”
It was also observed that ELLs are more likely to need employment
to survive. ELLs predominantly come from the working class, and they
view attendance in school as a bureaucratic legal requirement rather
than as an opportunity to develop skills necessary for social mobility.
Changes Which Occurred as a Result of Statewide Testing, as They
Relate to ELLs
Table 4.15 displays the responses to the question: What changes
have occurred as a result of statewide testing as it relates to ELLs? Due
to the low performance of ELLs, 35% of those who responded to the
questionnaire saw the need to focus and intervene on behalf of ELLs.
Although 16% observed no changes because of TAKS, 15% realized that
there will be more pressure at home and school to help ELLs pass TAKS.
Thirteen percent (13%) of the respondents affirmed their commitment to
be accountable for the ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 146/218
129
Table 4.15
What Changes Have Occurred for ELLs Due to TAKS?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Focus/Intervention for ELLs 2 3 2 4 8 19/35
None 0 0 3 3 3 0 9/16
More pressure: school/home 1 1 1 3 2 008/15
Accountability for ELLs 1 2 2 1 1 07/13
Teacher development 0 1 1 2 1000 5/9
Uncertain 2 0 0 2 1 000 5/9
Emphasis: test performance 2 0 0 0 1 00 3/5
Positive pressure to perform 0 0 0 1 1 00 2/4
Instruction is test oriented 0 0 0 0 2 00 2/4
Fewer Exemptions 1 0 0 1 0 00 2/4
Need effective leadership 0 0 1 0 2 00 1/2
Not welcome: public schools 0 0 0 0 1 010 1/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 147/218
130
In response to what should be done regarding the expected
changes, a non-ESL teacher (Non-ESL-11) suggested that “the changes
that have occurred include more intervention programs in classrooms,
on campuses and in districts.” This move was also supported by another
non-ESL teacher (Non-ESL-14):
“ There is a growing number of ELLs being motivated to perform
according to standards. These are the ones who can actually see
beyond high school graduation upon passing the exit-level TAKS.
Opportunities for postsecondary education exist along with
standardized testing, and so the high school journey for these ELLs
seeing the long-term meaning of the TAKS gives them a greater
sense of direction. The TAKS has also created positive pressure
among this group of ELLs in that they have become aware of
specific academic competencies they are expected to attain,
thereby creating in them greater self-efficacy and confidence for
postsecondary education.”
An assistant principal (AP-06) observed that:
“ The changes that have occurred in some schools have been
teachers receiving more staff development in how to instruct
students whose first language is not English. More responsibility
has been placed on teachers in all classes to help strengthen the
students‟ ability to pass the test.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 148/218
131
A non-ESL teacher (Non- ESL- 09) also saw that “changes in
teaching strategies are affected…teachers should be able to teach using
English as a medium of instruction and yet accommodate the ELLs in
their classes.”
A principal (P-02) offered this strategy to cope with changes due to
TAKS:
“We are focused on each child. The teachers are aware of their
strengths and weaknesses. Teachers meet with administrators to talk
about the students and what strategies they will use to help the ELL
attain the level of academic achievement that the state and the school
require.”
Recommendations to Improve Performance of ELLs in Statewide Testing
Respondents gave their recommendations on how to improve the
overall performance of ELLs in TAKS; these are shown in Table 4.16.
Recommendations involved: (1) different/fairer test; and, (2) better
assistance from the school through the teachers and the curriculum.
Twenty percent (20%) of the respondents clamored for a different and fair
test, while 9% opted for a test to be given 5 to 7 years after ELLs started
schooling in the country. Eighteen percent (18%) recognized the need for
better prepared teachers; 15% requested for modifications in teaching;
and 11% proposed a paced curriculum for ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 149/218
132
Table 4.16
What Recommendations are suggested for Improvement of ELLs Performance on TAKS?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Different/fairer test 1 0 0 5 5 11/20
Better prepared teachers 1 3 0 4 2 0 10/18
Teaching modifications 2 2 1 1 2 008/15
Paced curriculum 2 0 1 0 3 06/11
Test ELLs later (5-7years) 0 0 1 2 2 000 5/9
Increase overall performance 0 0 1 2 2 000 5/9
Intense English programs 0 1 3 0 0 444 4/7
Bilingual teachers for ELLs 0 0 0 0 3 00 3/5
Monitor ELLs 0 1 1 1 0 00 3/5
Test preparation classes 1 0 0 0 1 00 2/4
Smaller classes-create PLCs 0 0 1 1 0 00 2/4
Involvement of parents 0 0 0 0 2 010 2/4
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 150/218
133
ESL district personnel (EDP-03) recommended: “State legislators
(individuals who formulate the TAKS test) need to speak with ELL
teachers as well as the ELL students” to develop a more appropriate test
which measures achievement.
Two ESL teachers (ESL-06 and ESL-07) offered the following
changes in the administration of the TAKS to ELLs:
“There should be varying levels of the TAKS test for ELLs. When
students fail the TAKS, it‟s not because they are incapable of
passing…but the test is not a very ELL-friendly exam. If TAKS is
„supposed‟ to be a basic skills test for non-ELLs, then a test should
be devised to test the basic skills that ELLs have acquired since
they have learned English. It is not fair that a student that has
been speaking English their entire life takes the same test as a
person that has only been speaking English 3 or more years in
school.” “Have an „alternate‟ state test for the students who enter
the country/school at such a late age, which they realistically
cannot be expected to have enough of a grasp of English to pass
the TAKS. This of course would be a small portion of the ELLs.”
A non-ESL teacher (Non-ESL-16) proposed “that ELLs be given at
least 5 years of residency before they are given a statewide test. Teachers
need to be certified to teach ELLs so that the “problem” can be addressed
properly.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 151/218
134
An assistant principal (AP-02) recommended this plan of action:
“allow them more time to learn the English language before
administering the TAKS Exit Level Test; schedule them in intense
English/Reading classes for 75% of the school day.”
An even more drastic move was suggested by a non-ESL teacher
(Non-ESL-17) who suggested that we “do away with TAKS as a state test”
altogether.
Another non-ESL teacher (Non- ESL-14) recommended the
following regarding TAKS:
“Curriculum and instruction must translate standards into
classroom experience that takes into account their socio-cultural
milieu, their prior knowledge and skills and their linguistic
repertoire. It would help if those who are academically lagging
behind, or enrolled at a level several years above their actual level,
are given remedial courses so that they do not necessarily have to
study material that is incoherent to them, but would bring them
up from where they are to where they are supposed to be. Smaller
classes or intervention groups would perhaps ensure that greater
attention is given to the individual ELL‟s need. Since intrinsic
motivation for academic study is undermined by the survival
mode, perhaps a culture of career orientation can be instituted
where professionals from the community could conduct career
activities in schools. This will indirectly motivate towards ELLs
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 152/218
135
seeing the significance of standardized testing. Increased
involvement of parents is a big factor in this intrinsic motivation.”
Recommendations by a focus group included the following: (1)
“Much, much smaller classes for one thing, but it never happens because
it always comes down to money”; (2) “Home and community push for
excellence. Adult business tutors brought in on a weekly basis”; and (3)
“I believe that technology, certain computer programs, may be beneficial
to the ELL population.”
Recommendations of Greatest Value for ELLs‟ Success on Statewide
Testing
From the recommendations given, the respondents identified what
will be of greatest value for ELLs‟ success on statewide testing. These
suggestions are given in Table 4.17. Intervention for ELLs was
recommended by 27% of the respondents, while 16% emphasized quality
instruction, and 13% suggested a modified test. A more intensive English
program was proposed by 9% of the group, while 9% also observed the
need for help from home or for ELLs to have meaningful tutoring in
school.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 153/218
136
Table 4.17
What are the Recommendations, with the Greatest Value, are offered for ELLs Success on TAKS?
Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)
Interventions 2 3 1 3 6 15/27
Quality Instruction 1 2 1 0 5 0 9/16
Modified Test 0 0 1 3 3 007/13
Intensive Programs for ELLs 1 0 2 2 0 0 5/9
Tutoring 0 0 0 2 30005/9
More ESL Teachers 0 0 0 2 2 0004/7
Practice Test 2 0 1 0 1 00 4/7
5 or More Years Residency 0 0 0 1 3 00 4/7
Bilingual/ESL Classes 0 1 2 0 0 00 3/5
None/Don‟t Know 0 1 1 0 1 00 3/5
Smaller Class 0 0 0 1 0 00 1/2
Modified Diploma 0 0 0 1 0 00 1/2
Effective School Leadership 0 0 1 0 0 00 1/2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 154/218
137
A principal (P-03) considered this scenario as beneficial to the
ELLs:
“Better trained ESL teachers, more training for the traditional
teachers and training for administration on an assets-based model
rather than deficit. Also setting up more parent and community
involvement. Parents do care and want the best for their children
but they need to feel comfortable coming to school and made to feel
welcomed.”
An assistant principal (AP-09) suggested that: “Bilingual/ESL
classes should be continuously offered. Certified Bilingual/ESL teachers
should be recruited to offer even more assistance to these students.” This
suggestion finds support from an ESL teacher (ESL-03) who stated that:
“ELLs must be taught the skills that will allow them to assimilate into a
new society at a rigorous level. The test should reflect their needs and
also be more culturally sensitive.”
A non-ESL teacher (Non-ESL-14) averred that: “The remedial
intervention program for ELLs whose academic skills fall below their
actual grade level would greatly help in an immediate manner.” Yet
another non-ESL teacher (Non-ESL-17) suggested that “ESL teachers
and regular teachers focus on teaching skills that would be most
beneficial to the students based on their future aspiration (work or
college after high school).”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 155/218
138
Regarding statewide testing, an assistant principal (AP-06)
proposed that:
“ The test should be written in the language the student speaks.
Are we testing for academic skills? Or, are we trying to hold some
schools more accountable than others when giving tests? Schools
that have a higher minority rate must have teachers who are more
specialized in special education or bilingual education.”
During one of the interviews, a principal stated a rather grim
scenario involving ELLs:
“Standards are going to be difficult for them to achieve due to
their lack of language and academic skills. It takes an ELL student
five to seven years to acquire language skills adequate for the rigor
of high school curriculum. These students eventually pass but not
before causing tremendous challenges on our respective campuses.
These students show up on our „dropout lists.‟ They have a huge
impact on all our student data. For the most part these students
come from dysfunctional homes. The father is in Mexico. Mother is
here trying to raise a house full of kids- often forced to move from
place to place due to financial difficulties. These students seldom
stay in one school very long. I think the family unit should move
here together and stay until the student graduates from high
school.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 156/218
139
Another principal offered a radical suggestion regarding what can
be done to handle ELLs regarding the issue of statewide testing:
“If I could, I would send them (ELLs) all away from this school.
They would all be placed in a school designed to introduce,
develop, and build their language and culture skills, they bring our
scores down. That shouldn‟t be any surprise. Their skills are weak.
Their language is weak. Many of these students develop discipline
and attendance problems. Our ISD provides a „new comers‟
program for ESL students. The campus is a separate campus-away
from the main high school. Students remain on the „new comers‟
campus for about six months. Then, when they are ready, the
students are transitioned onto the main campuses. They ruin our
state scores; and wreak havoc on AYP scores. We have a large
population of these kids. Our school data indicates that we have
19 percent ESL students. That does not, however, include the
monitored kids. Most ESL students eventually pass the state exam
but we will never get above „acceptable‟ status if we continue to try
to teach and educate these kids on our campuses, within the
regular population. The faculty and staff must work ten times
harder than schools with few ESL kids.”
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 157/218
140
A principal explained how a strategy worked in their campus to
help ELLs:
“ELL students who continue to be mon itored are placed in the
appropriate English Language Arts course and are served by
certified ESL instructors. As with all of our students, tutorial
assistance is provided one hour each week after school for the four
key academic content areas. Tutorials begin the second week of
school and continue throughout the year (generally increasing in
number prior to TAKS testing in the spring). Teachers closely
monitor student performance and parents are contacted at the first
sign of difficulty. Informal parent teacher conferences are held, and
the student is encouraged to take every advantage of the
assistance provided by the instructional staff. Students are held
accountable for their study time. Note that our students are
expected to study approximately one and a half hours per school
night. Students and parents sign a contract to this effect prior to
being accepted at the school. If informal parent conferences are not
successful in providing the assistance and motivation to improve
academic success, the Intervention Assistance Team is employed.”
ELLs enrolling in a school during anytime during the year is a
reality school administrators and teachers have to face in public schools.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 158/218
141
They have to be ready to map out a course of action for each ELL to
provide guidance and education during the stay of that student in their
school.
Discussion
Research Question - Quantitative
Is there a relationship between the percentage of English language
learners enrolled in a school and the percentage of all students passing
the 10th grade TAKS tests in the core areas of English Language Arts
and Mathematics given in 2003, 2004, 2005, and 2006?
The descriptive statistics showing the means of the 10th grade
TAKS for English Language Arts and Mathematics do not indicate
improvement in performance despite the decrease in the percent of ELLs
enrolled in a school. Percent ELLs was 8.30% in 2003, 8.29% in 2004,
7.62% in 2005 and 6.94% in 2006. However, it is in the computations for
Pearson r correlation that the significant relationship was determined.
Using the SPSS software package for the TEA data of the sampled
schools, the obtained Pearson r correlation coefficients were: -0.349 in
2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006 for English
Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in 2005, and
-0.356 in 2006 for Mathematics. All of the computed Pearson r
correlation coefficients were significant at the 0.05 level, two-tailed. All of
the obtained Pearson r correlation coefficients were negative for both
content areas; this inverse relationship indicated that as the percentage
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 159/218
142
of ELLs increased, the percentage of all 10th grade students passing in
each of the core areas of English Language Arts and Mathematics
decreased. A regression analysis using the percentage of ELLs enrolled in
a school as a predictor variable resulted to a linear regression equation
which can predict the percentage of all students passing in 10th grade
TAKS in English Language Arts and Mathematics.
Any school which has ELLs should be aware of the number of
these types of students. If there is a surge in the influx of ELLs sometime
during the year, the school should carefully plan on how to prepare them
for future testing. Both curriculum and instruction for ELLs may be put
under review for improvement to better address this growing population
of ELLs.
Research Question - Qualitative
What are the anticipated and observed consequences of the
statewide testing specifically, TAKS, on ESL curriculum and instruction
as viewed by certified ESL teachers, non-certified ESL teachers who
teach ELLs, school administrators, and district ESL personnel?
This major question was explored using the following probes:
1. Why is TAKS given as a statewide test?
Respondents gave the following answers: (1) TAKS is given as a tool
to gauge knowledge in the core areas or what is supposed to be taught;
(2) TAKS is considered as a means to determine the school‟s status
(Exemplary, Recognized, Academically Acceptable, or Academically
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 160/218
143
Unacceptable); (3) TAKS is regarded as an avenue to assess the state
curriculum or standards; and (4) this statewide test is mandated by law
and is aligned with NCLB. The reasons why TAKS is given have been
understood by the different school personnel.
2. What are the intended consequences of this statewide testing?
Respondents expressed the understanding that statewide testing is
intended to eventually result in ELLs performing as well as the rest of the
students. Respondents are confident that ELLs can improve academically
and eventually join the mainstream; they are also aware that TAKS is a
requirement for graduation. Teachers expressed their wish for an
opportunity to teach ELLs individually with the end in mind of improving
their performance in TAKS. School personnel affirmed their commitment
on improving the situation of ELLs in school.
3. What problems have occurred related to or because of TAKS?
Respondents identified the following as problems encountered by
ELLs due to TAKS: (1) higher dropout rate; (2) graduation rate decreased
for ELLs; and (3) low self-esteem of ELLs. These problems are rooted on
the difficulty ELLs face in passing the TAKS tests.
4. What changes were caused by this statewide testing?
Due to TAKS, schools experienced the negative reality that there is
a high failure rate among ELLs. Other unpleasant realities included low
self-esteem because of low scores and tests caused frustration and
exasperation on the part of ELLs. There is mounting pressure on the
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 161/218
144
school and ELLs and emphasis is placed on test performance. Changes
require action to improve the situation of the school, particularly those
aspects which are related to the ELLs. Low scores have to be improved
and ELLs need counseling on ways on how to have a positive view of
anything one meets in life.
5. What are your recommendations to improve this statewide testing?
Recommendations involved two main factors: (1) deferment of the
test, possibly a different but fair test; and, (2) better assistance from the
school through the teachers and the curriculum, modifications in
teaching and possibly a paced curriculum for ELLs. Since one
requirement of NCLB is testing and results indicate progress, there is
need for school personnel to continually monitor ELLs and devise
strategies to better help these students.
6. What needs to be done for the ESL students to improve their
performance in general and specifically for this statewide test?
The following recommendations were seen as more helpful to ELLs:
Specific interventions for ELLs, quality instruction, a more intensive
English program, help from home and meaningful tutoring in school.
Both the school and the home, together with the community have to be
partners in getting involved with ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 162/218
145
Summary
Since the researcher utilized the explanatory design of mixed
methods, both quantitative and qualitative aspects were considered in
this study.
From the data gathered from TEA and after utilizing the SPSS
software package for Pearson r correlation, the obtained results indicate
that there existed significant, negative relationships between the percent
of ELLs enrolled in a school and the percentage of 10th grade students
passing in the English Language Arts and Mathematics TAKS tests given
in 2003, 2004, 2005, and 2006. The null hypotheses were then rejected.
The regression analysis provided the linear regression equations to
predict the percentage of all students passing in 10th grade TAKS tests in
ELA and Mathematics using the percentage of ELLs enrolled in a school
as the predictor variable.
For the qualitative dimension of the study, the respondents were
mostly certified in English Language Arts, Mathematics, and Special
Education for teachers and some process of administrative certification
for ESL district personnel, principals and assistant principals.
As a statewide test, TAKS was seen as a tool to gauge knowledge
and skills in the core areas. According to the respondents, students who
took TAKS were expected to pass and have at least average scores.
Unfortunately, most of the respondents saw dismal or low results in their
respective schools, especially among ELLs. Consequently, focus had been
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 163/218
146
centered on this group of students and recommendations included
testing them at a later date by using a modified test, and having
interventions to improve the overall performance of ELLs.
Administrators and teachers realized the need to improve the
quality of instruction and provided interventions especially geared
towards improving the academic performance of ELLs. It is hoped that
both instruction and curriculum for ELLs were directed towards
improving the plight of ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 164/218
147
CHAPTER V
SUMMARY, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS
Summary
The purpose of this study was to determine the impact of high-
stakes testing on English Language Learners (ELLs). This was shown in
both the quantitative and qualitative dimensions of the study. Both
quantitative and qualitative dimensions of the study provided the status
of high-stakes testing as it affected ELLs and how it influenced efforts in
schools to improve performance of students, particularly ELLs. Data
obtained from Texas Education Agency (TEA) were used to determine
whether there was a relationship between the percentage of ELLs
enrolled in a school and the percentage of all students passing the 10th
grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core
areas of English Language Arts and Mathematics given in 2003, 2004,
2005, and 2006. To support the qualitative aspect, this study explored
what certified English as a Second Language( ESL) teachers, non-
certified ESL teachers who teach ELLs, administrators, and district ESL
personnel viewed as the impact that high stakes standardized
assessments had on ELLs, ESL curriculum and instruction.
Review of literature included the important consideration of the No
Child Left Behind Act (NCLB). NCLB holds states using federal funds
accountable for student academic achievement. States are required to
develop a set of high-quality, yearly student assessments that include, at
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 165/218
148
a minimum, assessments in Reading/Language Arts, Mathematics and
Science. NCLB requires states to report Adequate Yearly Progress (AYP)
for all students and for subgroups, including students with limited
English proficiency (Abedi, 2004). Phrases such as “student
achievement,” “proficiency,” “raised expectations” and “testing” are
implications of NCLB. Certainly, the focus on holding schools
accountable for student achievement on standardized tests sets NCLB
apart from previous versions of the law. (Guilfoyle, 2006).
The focus of the study is the ELLs. T he term “English language
learner” is a recent designation for students whose first language is not
English. This group includes students who are just beginning to learn
English as well as those who have already developed considerable
proficiency. The driving force behind including English language learners
in statewide accountability testing is the legislation requiring it. In order
to continue to receive Title I funds through NCLB, states must set high
standards for all students and implement accountability systems to
measure progress towards those standards. NCLB specifically states that
English language learners must be included in statewide accountability
testing, that their scores must be disaggregated so that it can be seen
how they are achieving as a subgroup, and that the assessment system
must accommodate their linguistic needs (“NCLB”, 2002).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 166/218
149
High-stakes testing -- using standardized scores to impose
consequences affecting teachers and students – has been embraced
widely in recent years as a way to hold educators and students
accountable for their performance. Experts say the movement is one of
the most significant shifts in U.S. education in decades (Whoriskey,
2006).
The goal of statewide accountability testing for English language
learners (or for all students for that matter) is to improve standards-
based practices. The intended “washback” of including English language
learners in standards-based assessment has been described as providing
“the leverage needed to raise expectations for English language learners,
and the emphasis on higher level skills should improve the quality of
teaching provided to them” (Lachat, 1999, p.60), “feedback that will allow
instructional leaders to improve instructional programs” (Lacelle-
Peterson & Rivera, 1994, p.64), and will ideally “…help students reach
the standards by (a) influencing what is taught and how it is taught (i.e.,
„washback‟ to instruction), (b) providing data to guide instructional
modifications, and (c) targeting resources to schools they are most
needed” (Rivera & Vincent, 1997, p.336). In addition, Mehrens (2002)
states that large-scale assessments have two major purposes: to drive
reform and to gauge if reform policies have had an impact on student
learning. These goals are especially important for English language
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 167/218
150
learners who often face socio-economic, cultural, and linguistic
challenges to academic achievement.
Demographic Data
Total respondents who answered the on-line questionnaire totaled
55 – 35% are non-ESL certified teachers and 27% are ESL-certified
teachers. The administrators accounted for the remaining 38%- 16% are
Assistant Principals, 11% are Principals and 11% are ESL District
Personnel.
Conclusions
The analysis of the quantitative data in Chapter IV led the researcher
to draw the following conclusions:
1. The descriptive statistics showing the means of the 10th grade
TAKS for ELA and Mathematics do not indicate improvement in
performance despite the decrease in the percent of ELLs enrolled in
a school.
2. All the obtained Pearson r correlation coefficients to determine
whether there is a relationship between the percentage of English
language learners enrolled in a school and the percentage of all
students passing the 10th grade TAKS tests in the core areas of
English Language Arts and Mathematics given in 2003, 2004,
2005, and 2006 were all significant at the 0.05 level, two-tailed.
Both null hypotheses were rejected.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 168/218
151
3. The negative Pearson r correlation coefficients implied that as the
percentage of ELLs in a school increased, performance on both
English Language Arts and Mathematics decreased.
4. The linear regression equations may be used to predict outcomes
in 10th grade TAKS tests in ELA and Mathematics using the
percentage of ELLs enrolled in a school as the predictor variable.
The analysis of the qualitative data in Chapter IV led the researcher to
draw the following conclusions:
1. Texas Assessment of Knowledge and Skills (TAKS) were perceived
by respondents as a tool to gauge knowledge in the core areas.
2. ELLs were expected to have at least average scores on TAKS.
3. There was a difference in the expected and actual results.
Respondents observed dismal or failing performance of ELLS in
actual results in TAKS. This was evident by the high failure rate of
ELLs in their respective schools.
4. Higher dropout rate and lower graduation rates of ELLs were
problems encountered due to TAKS.
5. Respondents favored a different test for ELLs.
6. Respondents believed that interventions were needed to help ELLs
perform better.
Implications
The research data gathered in the course of this study suggested
that while there was a common perception that ELLs performed poorly
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 169/218
152
on high stakes testing, there was no unanimity among professionals in
the field of education regarding the viability of options that might be
considered in addressing the low achievement level of ELLs. This was not
necessarily relevant, although it suggested that the appreciation of the
problem and its causes lent itself to biases and distortions depending on
the personal circumstance and perspective of those presenting these
options. It was clear from the study that schools needed to do things
differently, if they expected ELLs to perform better on standardized
assessments. The major implications of the study were as follows:
1. The performance of schools in high stakes testing was affected by
the size and proportion of ELLs taking the test. At the same time,
ELLs were not evenly distributed across campuses. The
performance of schools on standardized tests was influenced to a
degree by the voluntary segregation in many districts of ethnic
groups who speak English only as an adopted language. This was
a phenomenon that was beyond the power of school districts to
address, and that required wide coordination among various
government agencies to develop an appropriate policy response.
2. The extended deferment of standardized tests administered in
English to ELLS should be considered. The primary goal of these
tests was to measure learning, that might be more accurately
accomplished if the assessment was done in the language the
student was most proficient. There was the expectation that the
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 170/218
153
student will eventually be proficient in English as well. Since a
second language is acquired in degrees, it might be reasonable to
assume that ELLs would not readily have the same facility for
English as a native speaker. Administering the test in English
before the ELL student was ready for it would compromise the
stated goal of measuring learning as accurately as possible.
3. Learning is transmitted through communication. Due to the
unique linguistic characteristic of ELLs, unique strategies,
modifications, and instructions need to be used to maximize their
capacity to learn concepts and skills. It is futile to assume that
ELLs will learn the same way as native speakers of the English
language. It follows that education professionals need the
specialized training and support to be able to facilitate learning for
ELLs.
4. The Language Proficiency Assessment Committee (LPAC) in each
campus needs to take a more active role in monitoring the progress
of ELLs and devising specific plans to properly respond to the
requirements designed specifically for ELLs. They need to
undertake a regular evaluation of instruction and curriculum for
ELLs and communicate findings and recommendations to all
stakeholders – school administrators, teachers, parents and the
ELLs.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 171/218
154
5. Interventions to improve the situation of ELLs should include
specific action plans to devise a more intensive English program in
schools and a continued emphasis on quality instruction
employing strategies suggested by educational experts who have
extensively researched on such courses of action.
Recommendations for Further Study
Based on the results of the study, the researcher recommends the
following concerns for further study:
1. A study to determine what additional supports are needed to
ensure that English language learners will pass high-stakes tests.
2. A study to identify what data are needed to make fair high-stakes
decisions about English language learners (like subject grades,
samples of class work and recommendations of teachers and
counselors).
3. A study to determine the reasons why English language learners
scored lowest among student groups in the Texas Assessment of
Knowledge and Skills tests in the core areas of English Language
Arts and/or Mathematics.
4. A study to explore different approaches in school campuses
regarding handling of English language learners in terms of
instruction, curriculum and other pertinent or related aspects
(such as some sort of evaluation - academic, social, financial, etc.)
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 172/218
155
that may guide administrators and teachers to effectively handle
English Language learners.
5. A study to determine the performance of 10th grade English
language learners compared to non-English language learners and
non-classified students based on the different objectives of the
Texas Assessment of Knowledge and Skills tests in either or both
Mathematics and English Language Arts.
6. A study to determine the impact of high stakes testing on English
language learners as viewed by parents and students.
7. A study to explore different instruments to measure academic
performance of English language learners.
8. A study to determine if there is significant difference between
performance in the different core areas of English language
learners belonging to different language groups.
This study affirmed the expected outcome that a significant
relationship existed between the percentage of ELLs enrolled in a school
and the percentage of all students passing the 10th grade TAKS tests in
both core areas of English Language Arts and Mathematics. The
regression analysis predicted that as the percentage of ELLs in a school
increased, the performance in the statewide, high-stakes testing in terms
of all students passing the 10th grade TAKS tests decreased. The
respondents of the study considered the Texas Assessment of Knowledge
and Skills (TAKS) as a tool to gauge knowledge in the different core areas.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 173/218
156
English language learners were expected to have at least average scores
on TAKS. There was a difference in the expected and actual results;
respondents observed dismal or failing performance of ELLS in the actual
results in TAKS. This was evident by the high failure rate of ELLs in their
respective schools. Higher dropout rate and lower graduation rate of
ELLs were problems encountered due to TAKS. Respondents favored a
different test for ELLs, possibly given at a later date after ELLs had
studied in the country for at least several years. Respondents believed
that interventions were needed to help ELLs perform better. Both the
school and the home, together with the community have to be involved in
preparing ELLs to be better prepared for their present and future roles in
the American society.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 174/218
157
REFERENCES
Abedi, J. (2004). The No Child Left Behind Act and English Language
Learners: Assessment and Accountability Issues. Educational
Researcher, 33(1), 4-14.
Abedi, J. (2004). Will You Explain the Question? Principal Leadership
(High School Ed.), 4(7), 27-31.
Abedi, J., Hofstetter, C. & Lord, C. (2004). Assessment Accommodations
for English Language Learners: Implications for Policy-Based
Empirical Research.
Abedi, J. (2004). Inclusion of Students with Limited English Proficiency
in NAEP: Classification and Measurement Issues. CSE Report 629
CRESST/ University of California, Los Angeles.
Abedi, J. (2004). Will You Explain the Title? Principal Leadership (High
School Ed.), 4(7), 27-31.
Abedi, J., Leon, S., & Mirocha, J (2003). Impact of student language
background on content-based performance: Analyses of extant
data (CSE Tech. Rep. No. 603). Los Angeles: University of
California, National Center for Research on Evaluation, Standards,
and Student Testing.
Abrams, L.M., & Madaus, G.F. (2003). The Lessons of High-Stakes
Testing. Educational Leadership , 61(3), 31-35.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 175/218
158
Airasian, P. W. (1988). Symbolic validation: The case of state-mandated,
high-stakes testing. Educational Evaluation and Policy Archives ,
10(4), 301-313.
Albus, D., Thurlow, M. L., Liu, K. K., & Bielinski, J. (2005). Reading Test
Performance of English Language Learners Using an English
Dictionary. The Journal of Educational Research , 98(4), 245-254.
Albus, D., Thurlow, M. L., & Liu, K. K. (2002). 1999-2000 Participation
and Performance of English Language Learners Reported in Public
State Documents and Web Sites (LEP Projects Report 3).
Minneapolis, MN: University of Minnesota, National Center on
Educational Outcomes.
Alderson, J. C., & Wall, D. (1993). Does Washback Exist? Applied
Linquistics , 14(2), 115-129.
Amrein, A. L., & Berliner, D. C. (2003). The Effects of High-Stakes
Testing On Student Motivation and Learning. Educational
Leadership , 60(5), 32-38.
Anderson, M. E. (2001). Item Response Differences Among Spanish-
Speaking ESL Students and Native English-Speaking Students
on a Standardized Reading Test. Unpublished Manuscript.
Anderson, M. E. (2004). Intended and Unintended Consequences of
Statewide Testing for ESL Curriculum and Instruction. (UMI No.
3137152).
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 176/218
159
Anderson, M. E., Thurlow, M. L., Swierznin, B., Liu, K. K., & Bielinski, J.
(2000). Bilingual Accommodations for Limited English Proficient
Students on Statewide Reading Tests: Phase (No. 31 Minnesota
report). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Artiles, A. J., Rueda, R., Salazar, J. J, & Higareda, I. (2005). Within
Group Diversity in Minority Disproportionate Representation:
English Language Learners in Urban City Schools. Exceptional
Children , 71(3), 283-300.
August, D., & Hakuta, K. (1997). Improving Schooling for Minority
Children: A Research Agenda. Washington, D.C.: National
Academy Press.
Barro, S.M., & Kolstad, A. (1987). Who Drops Out of High School?
Findings from High School and Beyond. Washington, D.C.: Center
for Education Statistics, Office of Research and Improvement, U.S.
Department of Education.
Brennan, R. L. (2004). Revolutions and Evolutions in Current
Educational Testing. Center for Advanced Studies in Measurement
and Assessment: CASMA Research Report #6.
California Department of Education. (2000). California demographics
data. Retrieved February 24, 2003, from
http://www.cde.ca.gov/demographics/
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 177/218
160
Center on Educational Policy (2005). From the capital to the classroom:
Year 3 of the No Child Left Behind Act. Washington, DC: Author.
Chamot, A. U., & O’Mailley. J.M. (1994). The CALLA Handbook:
Implementing the Cognitive Academic Language Learning Approach .
Reading, MA: Addison-Wesley.
Cheng, L. (1999). Changing Assessment: Washback on Teacher
Perceptions and Actions. Teaching and Teacher Education , 15,
253-271.
Cheng, L. (2003). Looking at the Impact of a Public Examination Change
on Secondary Classroom Teaching: A Hong Kong Case Study.
Journal of Classroom Interaction , 38(1), 1-10.
Cizek, G.J. (2003). When Teachers Cheat. Education Digest , 68(6), 28-31.
Cobb, T. (2000). One Size Fits All? Francophone Learners and English
Vocabulary Tests. Canadian Language Review , 57(2), 295-324.
Collier, V.P. (1989). How Long? A Synthesis of Research on Academic
Achievement in a Second Language. TESOL Quarterly , 23(3), 509-
531.
Coltrane, B., (2002).English Language Learners and High-Stakes Tests:
An Overview of the Issues. (November, 2002). Center for Applied
Linguistics.
Council of Chief State School Officers. (1992). Recommendations for
Improving and Assessment and Monitoring of Students with
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 178/218
161
Limited English Proficiency. Washington, DC: Council of Chief
State School Officers.
Creswell, J.W. (1998). Qualitative Inquiry and Research Design:
Choosing Among Five Traditions. Thousand Oaks, CA: Sage.
Davis, M. R. & Sacks, J. L. (2005). California, U.S. Department of
Education strike deal on NCLB rules [Electronic version].
Education Week .
Datnow, A., Borman, G.D., Stringfield, S., & Overman L.T. (2003).
Comprehensive School Reform in Culturally and Linguistically
Diverse Contexts: Implementation and Outcomes from a Four-Year
Study. Educational Evaluation & Policy Analysis , 25(2), 143-170.
D’Emilio, T. E. (2003). LEP Student Participation in Special Education:
Over or Under-Representation? Paper Presented at the CCSSO
Conference on Large-Scale Assessment, San Antonio, Texas
Denzin, H. (1970). The research act: A Theoretical Introduction to
Sociological Methods . New York: McGraw-Hill.
Dexter, L.A. (1970). Elite and Specialized Interviewing . Evanston, IL:
Northwestern University.
Edwards, V. (Ed.). (2003, Jan. 9). Quality Counts 2003: If I can’t learn
from you… (Education Week Special Report), 12(17). Bethesda,
MD: Editorial Projects in Education.
Erlandson, D., Harris, E., Skipper, B., & Allen, S. (1993). Doing
Naturalistic Inquiry . Newbury Park: Sage.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 179/218
162
Erpenbach, W.J., Forte-Fast, E., & Potts, A. (2003). Statewide
educational accountability under NCLB (An Accountability
Systems and Reporting State Collaborative on Assessment and
Student Standards Paper). Washington, DC: Council of Chief
State School Officers.
Escamilla, K., Mahon, E., Riley-Bernal, H., & Rutledge, D. (2003). High-
Stakes Testing, Latinos, and English Language Learners: Lessons
from Colorado. Bilingual Research Journal , 27(1), 25-47.
FairTest (2000).
Flores, B.B., & Clark, E.R. (2003). Texas Voices Speak About High-
Stakes Testing: Preservice Teachers, Teachers, and Students,
Current Issues in Education (Vol.6).
Fowler, F. J. (1984). Survey Research Methods . Beverly Hills: Sage.
Fraenkel, J. & Wallen, N. (2003). How to Design and Evaluate Research
In Education (5th ed.). New York, NY: McGraw-Hill.
Freeman, Y.S., & Freeman, D., E. 2002). Closing the Achievement Gap:
How to Reach Limited-Formal-Schooling and Long-Term English
Learners. Portsmouth, NH: Heinemann.
Garcia, G. E., & Pearson, P. D. (1994). Assessment and diversity. In L.
Darling-Hammond (Ed.), Review of Research in Education (Vol. 20).
Washington, DC: American Educational Research Association.
Gewertz, C. (2005). Ed. Dept. allows Chicago to provide NCLB tutoring
[Electronic version]. Education Week.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 180/218
163
Goldhaber, D., & Hannaway, J. (2001). Accountability with a kicker:
Observations on the Florida A+ Plan. Paper presented at the
annual meeting of the Association of Public Policy and
Management, Washington, DC.
Goldschmidt, P. and Wang, J. (1999) "When Can Schools Affect Dropout
Behavior: A Longitudinal Multilevel Analysis," American
Educational Research Journal , vol. 36, no. 4, pp.715-738, Winter.
Guilfoyle, C. (2006). “NCLB: Is There Life Beyond Testing?” Educational
Leadership , vol. 64, no. 3, pp. 8-13, November 2006.
Haladyna, T., Nolen, S., & Haas, N. (1991) Raising standardized
achievement test scores and the origins of test score pollution.
Educational Researcher , 20(5). 2-7
Hammersley, M. (1992). What’s Wrong with Ethnography ? London:
Routledge.
Haney, W. (2000). The Myth of the Texas Miracle in Education, Education
Analysis Policy Archives (Vol.8).
Harlow, A., & Jones, A. (2003, July). Why students answer TIMSS
science test items the way they do. Paper presented at the annual
conference of the Australian Science Education Research
Association, Melbourne, Victoria, Australia.
Herman, J., & Golan, S. (n.d.). Effects of standardized testing on
teachers and learning (CSE Technical Report 334). Los Angeles:
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 181/218
164
National Center for Research on Evaluation, Standards, and
Student Testing.
Herszenhorn, D. (2003, July 23). Basic skills forcing cuts in art classes.
The New York Times , p. B1.
Heubert, J. (2000).High-Stakes Testing: Opportunities and Threats. In
Pines, M (ed.). The Continuing Challenge: Moving the Youth
Agenda Forward (Policy Issues Monograph 00-02, Sar Levitan
Center for Social Policy Studies. Baltimore, MD: Johns Hopkins
University Press.
Heubert, J.P. (2003). First, Do No Harm. Educational Leadership , 60(4),
26-30.
Heubert, J., & Hauser, R. (Eds.) (1999). High stakes testing for tracking,
promotion, and graduation . Washington, DC: National Academy
Press.
Hill, R.K., & DePascale, C. A. (2003). Reliability of no child left behind
accountability designs. Educational Measurement: Issues and
Practice , 22(3), 12-20.
Hinde, E.R. (2003). The Tyranny of the Test: Elementary T eacher’s
Conceptualizations of the Effects of State Standards and Mandated
Tests on Their Practice, Current Issues in Education . (Vol.6).
Hoffman, J. V., Assaf, L., & Paris, S. G. (2001). High Stakes Testing in
Reading: Today in Texas, Tomorrow? The Reading Teacher .
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 182/218
165
Holmes, E. (1911). What is and what might be: A study of education in
general and elementary in particular. London: Constable.
Hood, L. (2003). Immigrant Students, Urban High Schools: the Challenge
Continues. Revised 09/16/2003, from
http://www.carnegie.orge/immigrantstudents.pdf.
Horn, C. (2003). High-Stakes Testing and Students: Stopping or
Perpetuating a Cycle of Failure? Theory Into Practice , 42(1), 30-41.
Horn, C., Ramos, M., Blumer, I., & Madaus, G. (2000). Cut scores:
Results may vary. Chestnut Hill, MA: National Board on
Educational Testing and Public Policy, Boston College.
Isaac, S. & Michael, W. (1997). Handbook in Research and Evaluation
for Education and the Behavioral Sciences (3 rd ed.) San Diego, CA:
EdiTs/Educational and Industrial Testing Services.
Johnson, D. (2001). Performance Pentagon: Five Strategies to Help All
Students Make the Grade. NASSP Bulletin , 85, 40-55.
Jones, M., Jones, B., Hardin B., Chapman, L., Yarbough, T., & Davis, M.
(1999). The impact of high-stakes testing on teachers and
students in North Carolina. Phi Delta Kappan , 81(3), 199-203.
Kaufman, P., Alt, M. N., & Chapman, C. D. (2001). Drop-Out Rates in the
United States: 2000 ( No.NCES 2002-114). Washington, DC:
National Center on Educational Statistics.
Kinder, A. L. (2002). Survey of the states’ limited English proficient
students and available educational programs and services, 2000-
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 183/218
166
2001 summary report. Washington, DC: National Clearinghouse
for English Language Acquisition and Language instruction
Educational Programs.
Klein, S., Hamilton, L., McCaffrey, D., & Stecher, B. (2000). What do test
scores in Texas tell us? Education Policy Analysis Archives , 8(49).
[Online]. Available: http://epaa.asu.edu/epaa/v8n49
Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996a). Perceived effects
of the Kentucky Instructional results information system (KIRIS)
(MR-792-PCT/FF). Santa Monica, CA: Rand.
Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996b). Perceived
effects of the Maryland school performance assessment program
(CSE Technical Report 409). Los Angeles: National Center for
Research on Evaluation, Standards, and Student Testing.
Krathwohl, D. (1993). Methods of Educational and Social Science
Research: An Integrated Approach. New York: Longman.
Krueger, R.A., & Casey, M. A. (2000). Focus Groups: A Practical Guide for
Applied Research (3 rd ed.). Thousand Oaks, CA: Sage Publications.
Lacelle-Peterson, M., & Rivera, C. (1994). Is it Real for all Kids? A
Framework for Equitable Assessment Policies for English Language
Learners. Harvard Educational Review , 64(1), 55-75.
Lachat, M. A. (1999). What Policy Makers and School Administrators
Need to Know About Assessment Reform for English Language
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 184/218
167
Learners. Northeast and Islands Regional Educational Laboratory
at Brown University. Providence.
Lane, S., & Stone, C. A. (2002). Strategies for Examining the
Consequences of Assessment and Accountability Programs.
Educational Measurement: Issues into Practice, 21(1), 23-30.
Lewis, A. (2002). A Horse Called NCLB. Phi Delta Kappan , 84(3),
179-180.
Linn, R. (1998). Assessments and Accountability (CSE Technical Report
490 ). Boulder, CO: CRESST/University of Colorado at Boulder.
Linn, R. L. (2000). Assessment and Accountability. Educational
Researcher , 29(2), 14-16.
Linn, R. L., Baker, E. L., & Herman, J. L. (2002). Minimum group size
for measuring adequate yearly progress: The CRESST line. Los
Angeles: University of California, Center for the Study of
Evaluation/National Center for Research on Evaluation,
Standards, and Student Testing.
Liu, K. & Thurlow, M. (1999). Limited English Proficient Students’
Participation and Performance on Statewide Assessments:
Minnesota Basic Standards Reading and Math, 1996-1998.
National Center on Educational Outcomes.
Madaus, G. (1988). The influence of testing on the curriculum. In L.
Tanner (Ed.), Critical issues in curriculum (pp. 83-121). Chicago:
University of Chicago Press.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 185/218
168
Madaus, G. (1991, January). The effects of important tests on students:
Implications for a national examination or system of examinations.
Paper prepared for the American Educational Research Association
Invitational Conference on Accountability as a State Reform
Instrument, Washington, DC.
Madaus, G., & O’Dwyer, L. (1999). A Short History of Performance
Assessment. Lessons learned. Phi Delta Kappan , 80(9), 688-695.
Major, R.C., Fitzmaurice, S.M., Bunta, F., & Balasubramanian, C.
(2005). Testing the Effects of Regional, Ethnic, and International
Dialects of English on Listening Comprehension. Language
Learning 55 (1) 37-69.
Marshak, D. (2003). No Child Left Behind: A Foolish Race Into the
Past.Phi Delta Kappan , 85(3), 229-231.
Marshall, C. (1990). Goodness Criteria: Are They Objective or Judgment
Calls? In E. G. Guba (Ed.), The Paradigm Dialog (pp. 188-197).
Thousand Oaks, CA: Sage.
Maxwell, J. (2005). Qualitative Research Design. An Interactive Approach
(2 nd ed.) . Thousand Oaks, CA: Sage Publications.
McLaughlin, M.W., & Shepard, L.A. (1995). Improving Education
Through Standards-Based Reform. Stanford, CA: The National
Academy of Education.
McMillan, J., Myran, S., & Workman, D. (1999, April 19-23). The
Impact of Mandated Statewide Testing on Teachers’ Classroom
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 186/218
169
Assessment and Instructional Practices. Paper presented at the
annual meeting of the American Educational Research Association,
Montreal, Quebec, Canada.
McNeil, L.M. (2000). Contradictions of school reform: Educational costs of
standardized testing. New York: Routledge.
Mehrens, W. A. (1998). Consequences of assessment: what is the
evidence? Vice Presidential Address for Division D, annual meeting
of the American Educational Research Association, San Diego.
Menken, K. (2000).What Are the Critical Issues in Wide-Scale
Assessment of English Language Learners? Framing Effective
Practice, 23-27.
Merriam, S. B. (1998). Qualitative Research and Case Study Applications
in Education , San Francisco, CA: Jossey-Bass.
Miller & Fredericks (2000). Research on High-Stakes Tests and
Instruction. Retrieved October 9, 2005 from
http://vnweb.hwwilsonweb.com/hww/results/results_single_fullte
xt.jhtml.
Murnane, R. & Levy, F. (1996). Teaching the New Basic Skills . New York:
The Free Press.
Murnane, Richard J., John B. Willett, Yves Duhaldeborde, and John H.
Tyler. 2000. How important are the cognitive skills of teenagers in
predicting subsequent earnings? Journal of Policy Analysis and
Management 19 (4) 547-568.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 187/218
170
Myers, K. (2003, July 16). A dream denied: Aspiring chef rethinks her
future as Falmouth school board bows to state pressure on MCAS.
Cape Cod Times. Available: www.capecodonline.com.
National Center for Education Statistics. (2002). Public school student,
staff, and graduate counts by state: School year 2000-01 (NCES
Publication 2002-348). Washington, DC: Author.
National Clearinghouse for Bilingual Education. (1997). High-Stakes
Assessment: A Research Agenda for English Language Learners.
Retrieved June 21, 1999, from
http://www.ncbe/gwu.edu/ncbepubs/reports/highstakes.htm#Te
sting.
National Commission of Excellence in Education. (1983). A Nation at
Risk: The Imperative for Educational Reform: a Report to the
Nation and the Secretary of Education, United States Department
of Education. Washington, D.C.: The Commission: [Supt. of Docs.,
U.S. G.P.O. distributor].
No Child Left behind Act of 2001, 20 U.S.C. 6301 et.seq. (2002).
O'Byrne, B. (2001). Needed: A Compass to Navigate the Multilingual
English Classroom. Journal of Adolescent & Adult Literacy 44 (5)
440-9.
Olson, L. (2005). Defying predictions, state trends prove mixed on
schools making NCLB targets [Electronic version]. Education
Week.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 188/218
171
Orfield, G. & Kornhaber, M. L. (2001). Raising standards or raising
barriers? Washington, DC: The Century Foundation Press.
Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2 nd
ed.) . Thousand Oaks, CA: Sage Publications.
Pedulla, J., Abrams, L., Madaus, G., Russell, M., Ramos, M., & Miao, J
(2003). Perceived effects of state-mandated testing programs on
teaching and learning: Findings from a national survey of
teachers. Chestnut Hill, MA: National Board on Educational
Testing and Public Policy, Boston College.
Popham, J. W. (2003). The Seductive Allure of Data. Educational
Leadership, 60(5), 48-51.
Reyes, P. & Fletcher, C. (2003). Successful Migrant Students: The Case of
Mathematics. Journal of Curriculum and Supervision , 18(4), 306-
333.
Rex, L. A. & Nelson, M. C. (2004). How Teachers’ Professional Identities
Position High-Stakes Test Preparation in Their Classrooms.
Teachers College Record , 106(6), 1288-1331.
Rhoades, K., & Madaus, G. (2003). Errors in standardized tests: A
systemic problem. Chestnut Hill, MA: National Board on
Educational Testing and Public Policy, Boston College.
Rivera, C., & Stansfield, C. (1998). Leveling the Playing Field for English
Language Learners: Increasing the Participation in State and Local
Assessments Through Accommodations. In R. Brandt (Ed.),
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 189/218
172
Assessing Student Learning: New Rules, New Realities (pp. 65-92).
Arlington, VA: Educational Research Service.
Rivera, C., Stansfield, C., Scialdone, L., & Sharkey, M. (2000). An
analysis of state policies for the inclusion and accommodation of
ELLs in state assessment programs during 1998 – 1999 (Executive
Summary). Washington, DC: The George Washington University,
Center for Equity and Excellence in Education.
Rivera, C., & Vincent. (1997), High School Graduation Testing: Policies
and Practices in the Assessment of English Language Learners.
Educational Assessment , 4(4), 335-355.
Rose, L.C. & Gallup, A.M. (2003). The 35th Annual Phi Delta Kappa/
Gallop Poll of the Public’s Attitudes Toward the Public Schools. Phi
Delta Kappan, 85.1, 41-56.
Rosenbusch, M. H. (2005). The No Child Left Behind Act and Teaching
and Learning Languages in U.S. Schools. The Modern Language
Journal, 89(2), 250-261.
Russell, M., & Abrams, L. (in press). Instructional uses of computers for
writing: The impact of state testing programs. Teachers College
Record.
Sheldon K. and Biddle, B. (1998). Standards, accountability and school
reform: Perils and pitfalls. Teachers' College Record 100 (1): 164-
180.
Sirkin, R. M. (2006). Statistics for the Social Sciences. (3 rd ed.). Thousand
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 190/218
173
Oaks, CA: Sage Publications.
Solano-Flores, G., & Trumbull, E. (2003). Examining Language in
Context: The Need for New Research and Practice Paradigms in the
Testing of English Language Learners. Educational Researcher ,
32(2), 3-13.
Spellings, M. (2005). Decision Letter on Request to Amend Virginia
accountability plan. Retrieved November 5, 2005 from
http://www.ed.gov/print/admins/lead/account/letters/acva5.ht
ml.
Stake, R. (1999). The Goods on American Education. Phi Delta Kappan ,
668-672.
Stecher, B., Barron, S., Chun, T., & Ross, K. (2000). The effects of the
Washington state education reform on Standards and Student
Testing.
Sum, A. (1999). Literacy in the Labor Force. Washington, DC: National
Center for Education Statistics.
Taylor, G., Shepard, L., & Kinner, F. (2003). A Survey of T eachers’
Perspectives on High-Stakes Testing in Colorado: What Gets
Taught, What Gets Lost (CSE Technical Report No. 588). Santa
Cruz, CA and Los Angeles, CA: Center for Research on Evaluation,
Diversity, and Excellence and CRESST.
Technical Digest, 2003 – 2004
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 191/218
174
Texas Assessments of Knowledge and Skills (TAKS), (2005). Data
Resources and Research. Texas Education Agency (TEA) from
http://www.tea.state.tx.us/data.html.
Thurlow, M. L., Nelson, J. R., Teelucksingh, E., & Ysseldyke, J. E. (2000).
Where’s Waldo? A T hird Search for Students with Disabilities in
State Accountability Reports (Technical Report No.25).
Minneapolis, MN: National Center on Educational Outcomes.
Thompson, G. L. The Real Deal on Bilingual Education: Former
Language-Minority Students Discuss Effective and Ineffective
Instructional Practices. Educational Horizons , 78(20, 80-92.
U. S. Department of Education. (2002). Elementary and Secondary
Education. Title IX. General Provisions. Part A, Section 9101.
Definitions. [11] Core Academic Subjects. Washington, D.C.: U.S.
Dept. of Education.
U.S. Department of Education, Office of the Undersecretary, Planning
and Evaluation Service. (1999). Promising results, continuing
challenges: The final report of the national assessment of Title I.
Washington, DC: Author
U.S. Department of Education. (1995). The Improving America’s Schools
Act of 1994: Reauthorization of the Elementary and Secondary
Education Act. Washington, D.C.: U.S. Dept. of Education.
Van Dalen, D.B. (1979). Understanding Educational Research (4 th ed.).
New York: McGraw-Hill.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 192/218
175
Wall, M. E. (2004). A Case Study of Secondary School Efforts Toward
English Language Learner Success in a Standards-Based Reform
System.
Wang, M. & Koda, K. (2005). Commonalities and Differences in Word
Identification Skills among English Second Language Learners.
Language Learning , 55 (1) 71-98.
Weitzman, E.A. (2000). Software and Qualitative Research. In N.K.
Denizin & Y.S. Lincoln (Eds.), The Handbook of Qualitative
Research (2nd ed., pp. 803-820). Thousand Oaks, CA: Sage.
Whoriskey, P. (2006).Political backlash builds over high-stakes testing.
Public support wanes for tests seen as punitive . Washington Post
Company.
Winograd, M. (2002). Equity: A Prerequisite for Reform. Principal
Leadership , 2(8), 42-47.
Wright, W. E., (2006).A Catch-22 for Language Learners. Educational
Leadership . November 2006/Volume 64/Number 3.
Zehler, A. M., Fleischman, H. L., Hopstock, P. J., Pendzick, M. L., &
Stephenson, T. G. (2003). Descriptive Study of Services to LEP
Students and LEP Students with Disabilities. (No. 4 Special Topic
Report: Findings on Special Education LEP. students). Arlington,
Va.: Development Associates, Inc.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 193/218
176
Zehler, A. M., Hopstock, P. J., Fleischman, H. L., & Greniuk, C. (1994).
An Examination of Assessment of Limited English Proficient
Students. Special Issues Analysis Center,Task Order Report.
Arlington, VA: Development Associates.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 194/218
177
APPENDICES
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 195/218
178
APPENDIX A
IRB
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 196/218
179
PRAIRIE VIEW A&M UNIVERSITYThe Texas A&M University System
P. O. Box 4149
Prairie View, Texas 77446-4149
Office of the Vice President v. 936.857.4494Research and Development f. 936. 857.2255
March 21, 2006
TO: Mr. Arthur L. Petterway, Principal Investigator
Doctoral Student, Education Leadership and Counseling
Dr. Ben DeSpain, EDLC – Faculty Advisor
FROM: Marcia C. Shelton, Compliance Officer, Regulatory Research
Institutional Review Board
SUBJECT: IRB Protocol Review – Protocol Status
Title: A Mixed Mthods Analysis of the Impact of High Stakes Testing on English Language
Learners in Major Urban High Schools in Texas
Protocol Number: 200-103
Review Category: Full Review - (primary reviewer – L. Myers)
Approval Date: February 13, 2006
The approval determination was based on the following Code of Federal Regulations (CFR)
46.101(b) (2). Research involving the use of educational tests (cognitive, diagnostic,
aptitude, achievement), survey procedures, interview procedures or observation of public
behavior, unless:(i) information obtained is recorded in such a manner that human subjects can be
identified, directly or through identifiers linked to the subjects; and (ii) any disclosure of the human subjects' responses outside the research could reasonably place the subjects atrisk of criminal or civil liability or be damaging to the subjects' financial standing,
employability, or reputation.
Remarks:
The Institutional Review Board – Human Subjects in Research, Prairie View A&M University
has reviewed and approved the above referenced protocol. Your study has been approved for
one year – February 13, 2006- February 12, 2007. As the principal investigator of this study, you
assume the following responsibilities:
Renewal: Your protocol must be re-approved each year in order to continue the research. You
must also complete the proper renewal forms in order to continue the study after the initialapproval period.
Adverse events: Any adverse events or reactions must be reported to the IRB immediately.
Amendments: Any changes to the protocol, such as procedures, consent/assent forms, addition of
subjects, or study design must be reported to and approved by the IRB.
Completion: When the study is complete, you must notify the IRB office and complete the
required forms.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 197/218
180
Date: 14-Jan-2006 00:33:10 -0600
From: <[email protected]>
To: <[email protected]>
Subject: Exam Confirmation - Clinical Research Training
Registration Summary:
01/14/2006
Arthur L Petterway Ph.D
Prairie View A&M University
Educational Leadership & Counseling
P. O. Box 519
n/a
MSC
Prairie View TX 77446-0519United States
Tel: (713)924-1622 Fax: (713)924-1619
E-mail: [email protected]
This e-mail is to verify that you successfully completed the NIH Clinical Research
Training course.
You answered 20 out of a total of 25 questions for a final grade of 80%.
If you are an NIH principal investigator, you have fulfilled the Training and Education
Standard issued by the NIH for conducting clinical research within the intramural
research program.
Please print this e-mail and retain for your records.
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 198/218
181
CITI Course in The Protection of Human Research Subjects
Sunday, February 19, 2006
CITI Course Completion Recordfor Arthur Petterway
To whom it may concern:
On 1/15/2006 , Arthur Petterway (username=apetterw) completed all CITI Program requirements for the Basic CITI Course in The Protection of HumanResearch Subjects.
Learner Institution: Texas A&M University
Learner Group: Group 2.
Learner Group Description: Social and Behavioral Research Investigators and Key Personnel
Contact Information:
Gender: Male
Which course do you plan to take?: Social & Behavioral Investigator CourseOnly
Role in human subjects research: Principal Investigator
Mailing Address:
5300 N. Braeswood Blvd
#247
Houston
Texas
77096
USA
Email: [email protected]
Office Phone: (713)924-1622
Home Phone: (713)498-8667
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 199/218
182
The Required Modules for Group 2. are:Date
completed
Introduction 01/15/06
History and Ethical Principles - SBR 01/15/06
Defining Research with Human Subjects - SBR 01/15/06
The Regulations and The Social and Behavioral Sciences -SBR
01/15/06
Assessing Risk in Social and Behavioral Sciences - SBR 01/15/06
Informed Consent - SBR 01/15/06
Privacy and Confidentiality - SBR 01/15/06
Research with Prisoners - SBR 01/15/06
Research with Children - SBR 01/15/06
Research in Public Elementary and Secondary Schools - SBR 01/15/06
International Research - SBR 01/15/06
Internet Research - SBR 01/15/06
Conflicts of Interest in Research Involving Human Subjects 01/15/06
Texas A&M University 01/15/06
Additional optional modules completed: Datecompleted
For this Completion Report to be valid, the learner listed above must beaffiliated with a CITI participating institution. Falsified information andunauthorized use of the CITI course site is unethical, and may be consideredscientific misconduct by your institution.
Paul Braunschweiger Ph.D.Professor, University of MiamiDirector Office of Research Education
CITI Course Coordinator
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 200/218
183
APPENDIX B
CONSENT FORM
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 201/218
184
CONSENT FORMA MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES
TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGHSCHOOLS IN TEXAS
I have been asked to participate in a research study. This study isintended to explore what certified English as a Second Language (ESL)teachers, non-certified ESL teachers who teach English LanguageLearners (ELLs), campus administrators , and district ESL personnelview as the impact that high stakes assessments have on ELLs, ESL curriculum and instruction , and what they observe as occurring. I wasselected to be a possible participant because I am either a campus
administrator, a teacher handling ELLs or a district ESL coordinator of the selected major urban high schools in Texas. A total of 118 peoplehave been asked to participate in this study. The purpose of this study isto determine the views and opinions of campus administrators, teachers
handling ELLs, and district ESL coordinators regarding the impact of statewide testing, specifically Texas Assessment of Knowledge and Skills(TAKS), on the curriculum and instruction of this special group of students.
If I agree to be in this study, I will be asked to answer an on-linequestionnaire. I may also form part of the focus group which will havegroup interviews verifying and validating my views and opinionsregarding the open-ended questions. If I am a principal or a district ESL coordinator my interview will be one-on-one. The proceedings will betranscribed so that the researcher can go back to sections for better
understanding. I may volunteer another knowledgeable respondent inorder that the succeeding interviews will be more productive ormeaningful. This study will only take at least two interviews after the on-line questionnaire has been answered. The risks associated with thisstudy are almost non-existent. The benefits of participation are mycontributions to the betterment of the ESL curriculum, instruction andtesting.
I will receive no compensation for my participation in this research
study. A simple ‘thank you’ note will suffice.
This study is confidential since the data will be dealt with only by theresearcher and transcripts will be kept in a safe storage. The records of this study will be kept private. No identifiers linking me to the study willbe included in any sort of report that might be published.
Date _____________ Initial________________ Page 1 of 2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 202/218
185
Research records will be stored securely and only the researcher, ArthurL. Petterway, will have access to the records. For the transcripts, onlyArthur L. Petterway will also have access to the information contained
therein. My decision whether or not to participate will not affect my
current or future relations with Prairie View A & M University. If I decideto participate, I am free to refuse to answer any of the questions thatmay make me uncomfortable. I can withdraw at any time without myrelations with the University, job, benefits, etc., being affected. I cancontact Arthur L. Petterway at (713)748-8303 or [email protected] Dr. Robert Marshall, at (936)857-4127 [email protected] withany questions about this study.
This research study has been reviewed by the Institutional Review Board-Human Subjects in Research, Prairie View A & M University. Forresearch- related problems or questions regarding subjects’ rights, I can
contact the Institutional Review Board through Ms. Marcia C. Shelton,Research Compliance Officer, Anderson Hall Room 311, PO Box 4149,Prairie View, TX 77446, at 936.857.2541 and at [email protected] have read the above information. I have asked questions and have
received answers to my satisfaction. I have been given a copy of thisconsent document for my records. By signing this document, I consent toparticipate in the study.
Signature of the Subject: ______________________________ Date: __________
Signature of Investigator: ______________________________ Date: __________
Original – ResearcherCopy – Participant
Date _____________ Initial________________
Page 2 of 2
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 203/218
186
APPENDIX C
INTERVIEW QUESTIONS
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 204/218
187
INTERVIEW /FOCUS GROUP QUESTIONS
Please help us serve English Languages Learners (ELLs) more
effectively by taking a few moments to answer these interview
questions. Confidentiality will be maintained throughout this process, as
only I will have access to the data. Data and information will be kept in a
safe home vault for a period of seven years, after which time they will be
destroyed. Summarized data will be published in my dissertation. Your
participation is greatly appreciated, as you will be making a significant
contribution to the English as a Second Language (ESL) curriculum,
instruction and testing.
1. Describe your experience with English Language Learners.
2. In your estimation, how many English Language Learners
graduate each year from your school?
3.
Do you think this number has increased or decreased in recent
years? Why?
4. Describe your school’s support system for former English
Language Learners.
5. What is your understanding of the Texas Comprehensive
Assessment Plan as it relates to ELLs?
6. Should the English Language Learner be held to a separate
standard of promotion or to the same standards as the regular
population? Why?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 205/218
188
7. With respect to your school, how do the promotion standards
affect your English Language Learner population? What will it
take to facilitate the English Language Learners’ success in
meeting graduation standards?
8. What concerns need to be addressed before administrating TAKS
to the English Language Learner? Who should be responsible in
resolving these concerns?
9. Why should English Language Learners be held/or not held liable
or accountable to the same standards or requirements?
10. What do you think are the intended consequences of the statewide
accountability test (TAKS), specifically in terms of ESL curriculum
and instruction?
11. Has the ESL program implemented in schools failed/or served its
purpose or the state’s standardized assessment?
12.How would you rate the success of the ESL program in regards to
standardized assessment?
13. To what extent do the consequences of a statewide assessment
affect the ELL student, educator and district?
14. In what ways, if any, has the curriculum for ESL students
changed as a result of their participation in the assessment?
15.What is your opinion of the cause of the low performing scores of
ELL students?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 206/218
189
16.What do you need to do to adequately prepare these students for
success with standardized assessments and graduation
standards?
17. What are your recommendations for the future testing of the
English Language Learner?
18.Given that the criteria being used are high-stakes, what additional
supports are needed to ensure that ELLs will be able to meet
them?
19.How do alternative assessments (e.g., Spanish language exams)
compare to mainstream assessments?
20.When is the use of native language assessments appropriate?
21.How does the placement of accommodations impact comparability
with mainstream student performance?
22.Do wide-scale tests with the permitted accommodations fully
assess English Language L earners’ knowledge and abilities or does
the system need to be fully redesigned such that the needs of these
students are addressed in the development of the assessments?
23.Do you think other data collection methods, such as portfolios or
other performance assessments, would yield more accurate results
with regard to ELLs than traditional assessments?
24.What sorts of information is needed to make fair high-stakes
decisions about ELLs (e.g., grades, classroom performance, an
array of samples of student work, teacher recommendations)?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 207/218
190
25.What would be the most beneficial system(s) of accountability to
ensure that these students are making progress in what they know
and can do in important content areas?
26.What supports are necessary to aid states and districts in their
alignment of assessments, standards, curricula, and instruction?
TAKS = Texas Assessment of Knowledge and Skills
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 208/218
191
APPENDIX D
ON-LINE/HARD COPY QUESTIONNAIRE
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 209/218
192
A MIXED METHODS ANALYSIS OF THE IMPACT OF
HIGH STAKES TESTING ON ENGLISH LANGUAGE
LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN
TEXAS Please help us serve English Languages Learners(ELLs) more effectively by taking a few moments to fill out thisquestionnaire. The results will be returned to usautomatically via the web. Confidentiality will be maintainedthroughout this process, as only I will have access to thedata. A random numeric code will be electronically generatedand assigned to each qualitative questionnaire. Data andinformation will be kept in a safe home vault for a period ofseven years, after which time they will be destroyed.Summarized data will be published in my dissertation. Yourparticipation is greatly appreciated, as you will be making asignificant contribution to the English as a Second Language(ESL) curriculum, instruction and testing. 1.) GENDER
MALE FEMALE
2.) AGE 21-30 31-40 41-50 51-60 61+
3.) CURRENT POSITION Principal Assistant Principal
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 210/218
193
ESL Certified Teacher NON-ESL Certified Teacher (who teach ELLs) District ESL Personnel
4.) HIGHEST DEGREE EARNED Bachelors Masters Doctorate
5.) What certifications do you hold?
6.) YEARS OF EXPERIENCE IN PUBLIC EDUCATION 1-5
6-10 11-20 21+
7.) What are the anticipated results of the statewide testing,
specifically, English Language Arts (ELA) and Mathematics
portion of the Texas Assessment of Knowledge and Skills
(TAKS), on English Language Learners?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 211/218
194
8.) What are the actual results of the statewide testing,
specifically, ELA and Mathematics portion of the Texas
Assessment of Knowledge and Skills (TAKS), on English
Language Learners?
9.) Why is the Texas Assessment of Knowledge and Skills
(TAKS) given as a statewide test?
10.) What are the intended consequences of this statewide
testing as it relates to English Language Learners?
11.) What has happened because of TAKS as it relates to
English Language Learners?
12.) What problems have occurred, if any, to the English
Language Learners because of TAKS?
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 212/218
195
13.) What changes have occurred as a result of statewide testing
as it relates to English Language Learners?
14.) What general recommendations would you suggest to
improve the overall performance of English Language Learners
on statewide testing ?
15.) Based on your recommendations above, which will be of
greatest value for ELLs success on statewide testing?
Thank you for taking the time to fill out this questionnaire. If
you need to contact us - you can click on the followingemail [email protected] Submit Form Reset Form
-1
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 213/218
196
APPENDIX E
LETTER TO PARTICIPANTS
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 214/218
197
LETTER TO PARTICIPANTS
Arthur L. Petterway 05/12/20065300 N. Braeswood Blvd. # 247
Houston, Texas 77096
Dear Sir or Madam,I am presently in the Ph.D. in Education Leadership program at
Prairie View A & M University. I am currently conducting a dissertationresearch on high-stakes testing and English language learners. Mydissertation topic is “A Mixed Methods Analysis of the Impact of High-Stakes Testing on English Language Learners in Major Urban High
Schools in Texas”. I realize how busy you are in meeting the challenges of
your work but I hope that you will take time to complete an on-linequalitative questionnaire that I have prepared to gather research data.You can access the qualitative questionnaire at
The questionnaire will include a major question on what are theanticipated and observed consequences of the statewide testing,
specifically, Texas Assessments of Knowledge an Skills (TAKS), onEnglish as a Second Language (ESL) curriculum and instruction asviewed by certified ESL teachers, non-certified ESL teachers who teachEnglish Language Learners (ELLs), school administrators, and districtESL personnel.
Confidentiality will be maintained throughout this process, as only
I will have access to the data. A random numeric code will beelectronically generated and assigned to each qualitative questionnaire.Data and information will be kept in a safe home vault for a period of seven years, after which time they will be destroyed. Summarized data will be published in my dissertation. Your participation is greatlyappreciated, as you will be making a significant contribution to the ESL curriculum, instruction and testing.
Should you have any questions or comments, please feel free to
contact me at (713)748-8303 or [email protected]. You mayContact my dissertation chair, Dr. Robert Marshall, at (936)857-4127 or
[email protected]. Once again, I appreciate your attention to thismatter and look forward to your favorable response.
Sincerely,
Arthur L. PetterwayPh.D. student – Prairie View A & M University
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 215/218
198
APPENDIX F
REQUEST FOR EXTANT DATA FROM TEXAS EDUCATION AGENCY
(TEA)
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 216/218
199
Dear Mr. Rod Rowell,
As per our telephone conversation on July 3, 2006, I am submitting my request for extant data to TEA.
Requestor Name: Arthur L. Petterway
Company Name: PhD Doctoral Student at Prairie View A&M University
Address: 5300 N. Braeswood Blvd. #247
City/State/Zip: Houston, Texas 77096
Telephone: Office-(713)924-1600; Home-(713)748-8303; Cell-(832)693-2809Fax Number: (713)924-1619
Requestor Email Address: [email protected]
Brief Summary of Request: I am currently working on my dissertation, A Mixed Methods Analysis of the
Impact of High Stakes Testing on English Language Learners in Major Urban High schools in Texas , at
Prairie View A&M University and need your help. This is a Public Information Request to please provide me
with the 2002-2003; 2003-2004; 2004-2005; 2005-2006 TAKS Summary Report-(ALL) for 10th grade
English Language Arts and Mathematics of all of the high schools in the following school districts:
015907 SAN ANTONIO ISD
015910 NORTH EAST ISD
015915 NORTHSIDE ISD 057905 DALLAS ISD
071902 EL PASO ISD
071905 YSLETA ISD
101912 HOUSTON ISD
220901 ARLINGTON ISD
220905 FORT WORTH ISD 227901 AUSTIN ISD.
Your immediate attention to this matter will be greatly appreciated. If you have any questions, please don't hesitate tolet me know.
Sincerely,
Arthur L. Petterway Assistant Principal
HOUSTON INDEPENDENT SCHOOL DISTRICT
STEPHEN F. AUSTIN SENIOR HIGH SCHOOL
1700 DUMBLE HOUSTON,
TEXAS 77023-3195
PH: (713) 924-1600 FAX: (713) 924-1619
ARTHUR L.PETTERWAY
Assistant Principal
Home of the Mighty Mustangs
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 217/218
200
From: Woli, Urbe [mailto:[email protected]]
Sent: Tue 7/11/2006 3:23 PMTo: Petterway, Arthur L
Cc: PIR; Eaton, Jennifer; Woli, UrbeSubject: PIR # 6541 - Grade 10 Campus TAKS Summary Data Mr. Arthur Petterway, The 2003 – 2005 TAKS campus summary data you are requesting (see attached PDFdocument) are available for download at no cost from our website athttp://www.tea.state.tx.us/student.assessment/reporting/taksagg/index.html . The 2006TAKS campus summary data will be posted at the same location by the end of August,2006. Due to the large size of the downloadable data files, you will need to use SAS orSPSS to process the data files.
Please let us know if you prefer that we generate the data for you. To enable us provideaccurate data; please list all the statistics you need in the data. Contact me at (512) 463-
9536 to discuss your data needs.
Thank you. Urbe Woli Student Assessment Division Texas Education Agency Tel: (512) 463-9536
8/3/2019 Arthur L. Petterway's PhD Dissertation: A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON E…
http://slidepdf.com/reader/full/arthur-l-petterways-phd-dissertation-a-mixed-method-analysis-of-the 218/218
VITA
ARTHUR L. PETTERWAY5300 N. Braeswood Blvd, #247
Houston, Texas 77096
EDUCATIONAL HISTORY
Prairie View A&M University, Prairie View, Texas, Ph. D.in Education Leadership, Expected Graduation Date August, 2007
Prairie View A&M University, Prairie View, Texas, M.Ed.
in Education Administration, August, 1999