West Virginia General Summative Assessment 2018–2019 ......in grades 3–8 in ELA and mathematics....

West Virginia General Summative Assessment

2018–2019

Volume 4 Reliability and Validity

WVGSA 2018–2019 Technical Report: Volume 4

Reliability and Validity i West Virginia Department of Education

TABLE OF CONTENTS

1. INTRODUCTION AND OVERVIEW OF RELIABILITY AND VALIDITY EVIDENCE.................................................................................... 1

1.1 Reliability .............................................................................................................................2 1.2 Validity ................................................................................................................................3

2. PURPOSE OF THE WEST VIRGINIA GENERAL SUMMATIVE ASSESSMENT .................................................................................................. 5

3. RELIABILITY .................................................................................................. 6

3.1 Reliability for ELA and Mathematics ..................................................................................6 3.2 Reliability for Science..........................................................................................................7 3.3 Test Information Curves and Standard Error of Measurement for ELA and

Mathematics .........................................................................................................................8 3.4 Standard Error of Measurement for Science ......................................................................12 3.5 Reliability of Achievement Classification .........................................................................13

3.5.1 Classification Accuracy ............................................................................................14 3.5.2 Classification Consistency ........................................................................................15

3.6 Precision at Cut Scores ......................................................................................................17 3.7 ELA Writing Prompts Inter-Rater Reliability ....................................................................19

3.7.1 Automated Essay-Scoring Engine .............................................................................19

4. EVIDENCE OF CONTENT VALIDITY ........................................................ 30

4.1 Content Standards ..............................................................................................................30 4.2 Alignment of AIRCore Item Banks to the Content Standards and Benchmarks ...............32

5. EVIDENCE ON INTERNAL-EXTERNAL STRUCTURE ............................. 32

5.1 Correlations among Reporting Category Scores................................................................33 5.2 Confirmatory Factor Analysis for Spring 2018 ELA and Mathematics ............................35

5.2.1 Factor Analytic Methods...........................................................................................36 5.2.2 Results .......................................................................................................................39 5.2.3 Discussion .................................................................................................................42

5.3 Local Independence, Spring 2018......................................................................................43 5.4 Convergent and Discriminant Validity ..............................................................................45 5.5 Relationship of Test Scores to External Variables ............................................................51 5.6 Cluster Effects for Science.................................................................................................52

6. FAIRNESS IN CONTENT .............................................................................. 54

6.1 Statistical Fairness in Item Statistics..................................................................................54


Reliability and Validity ii West Virginia Department of Education

7. SUMMARY .................................................................................................... 54

8. REFERENCES ................................................................................................ 56

LIST OF TABLES

Table 1: Test Form and Administration Mode ............................................................................... 1 Table 2: Reliability Coefficients, ELA and Mathematics ............................................................... 7 Table 3: Marginal Reliability Coefficients for Science .................................................................. 7 Table 4: Classification Accuracy Index, ELA .............................................................................. 15 Table 5: Classification Accuracy Index, Mathematics ................................................................. 15 Table 6: Classification Accuracy Index, Science ......................................................................... 15 Table 7: Classification Consistency Index, ELA .......................................................................... 16 Table 8: Classification Consistency Index, Mathematics ............................................................. 17 Table 9: Classification Consistency Index, Science ..................................................................... 17 Table 10: Achievement Levels and Associated CSEM, ELA ...................................................... 17 Table 11: Achievement Levels and Associated CSEM, Mathematics ......................................... 18 Table 12: Achievement Levels and Associated Conditional Standard Error of Measurement,

Science ................................................................................................................................... 19 Table 13: Writing Rubrics ............................................................................................................ 21 Table 14: Rating Agreement Example .......................................................................................... 22 Table 15: Percent Agreement in Human-Scoring and Auto-Scoring ........................................... 24 Table 16: Quadratic-Weighted Kappa Coefficients ...................................................................... 27 Table 17: The First 500 Cases Percent Agreement in Human-Scoring and AutoScoring ............ 29 Table 18: Number of Items for Each Reporting Category, ELA .................................................. 31 Table 19: Number of Items for Each Reporting Category, Mathematics ..................................... 31 Table 20: Number of Items for Each Reporting Category, Science ............................................. 32 Table 21: Correlations among Reporting Categories, ELA .......................................................... 34 Table 22: Correlations among Reporting Categories, Mathematics ............................................. 34 Table 23: Correlations among Reporting Categories, Science ..................................................... 35 Table 24: Goodness-of-Fit Second-Order CFA, Spring 2018 ELA ............................................. 40 Table 25: Goodness-of-Fit Second-Order CFA, Spring 2018 Mathematics................................. 40 Table 26: Correlations Among ELA Factors, Spring 2018 .......................................................... 41 Table 27: Correlations Among Mathematics Factors, Spring 2018 ............................................. 42 Table 28: ELA Q3 Statistic, Spring 2018 ..................................................................................... 44 Table 29: Mathematics Q3 Statistic, Spring 2018 ........................................................................ 44 Table 30: Correlations across Subjects, Grade 3 .......................................................................... 46 Table 31: Correlations across Subjects, Grade 4 .......................................................................... 46 Table 32: Correlations across Subjects, Grade 5 .......................................................................... 47 Table 33: Correlations across Subjects, Grade 6 .......................................................................... 47 Table 34: Correlations across Subjects, Grade 7 .......................................................................... 48 Table 35: Correlations across Subjects, Grade 8 .......................................................................... 48 Table 36: Correlations across Spring 2019 ELA, Mathematics and Science Scores.................... 49 Table 37: Correlation between Summative and Interim Scores, ELA.......................................... 50 Table 38: Correlation between Summative and Interim Scores, Mathematics ............................. 50


Reliability and Validity iii West Virginia Department of Education

Table 39: Correlations between Spring 2017 and Spring 2018 Scores, ELA ............................... 51 Table 40: Correlations between Spring 2017 and Spring 2018 Scores, Mathematics .................. 51

LIST OF FIGURES

Figure 1: Sample Test Information Function .................................................................................. 8 Figure 2: Conditional Standard Errors of Measurement, ELA ....................................................... 9 Figure 3: Conditional Standard Errors of Measurement, Mathematics ........................................ 11 Figure 4: Conditional Standard Errors of Measurement, Science ................................................ 13 Figure 5: Second-Order Factor Model, ELA ................................................................................ 39 Figure 6: Cluster Variance Proportion for Science Operational Items in Elementary School ..... 53 Figure 7: Cluster Variance Proportion for Science Operational Items in Middle School ............ 53

LIST OF APPENDICES

Appendix A: Student Demographics and Reliability Coefficients Appendix B: Conditional Standard Error of Measurement Appendix C: Classification Accuracy and Consistency Index by Subgroups


Reliability and Validity 1 West Virginia Department of Education

1. INTRODUCTION AND OVERVIEW OF RELIABILITY AND VALIDITY EVIDENCE

The state of West Virginia implemented a new assessment program for operational use during the 2017–2018 school year. This new program, named the West Virginia General Summative Assessment (WVGSA), replaced the Smarter Balanced Assessment Consortium (SBAC) in English language arts (ELA) and mathematics and replaced the West Virginia Educational Standards Test in science. The WVGSA is delivered as online, adaptive assessments to students in grades 3–8 in ELA and mathematics. For science, the test is administered online for grades 5 and 8 using a linear-on-the-fly test (LOFT) design. Accommodated versions are available for each grade, including braille and a large-print data entry interface (DEI) form for ELA, mathematics, and science. Spanish language versions of the mathematics and science tests are also available. Table 1 shows the complete list of tests for the second year of operational test administration, spring 2019.

Table 1: Test Form and Administration Mode

Subject (language/format) Administration Mode Grade

ELA (English/adaptive) Online 3–8

ELA (English/adaptive-braille) Online 3–8

ELA (English/fixed-DEI) Paper 3–8

Mathematics (English/adaptive) Online 3–8

Mathematics (English/adaptive-braille) Online 3–8

Mathematics (Spanish/adaptive) Online 3–8

Mathematics (English/fixed-DEI) Paper 3–8

Mathematics (English/fixed-braille) Paper 3–8

Science (English/LOFT) Online 5 and 8

Science(Spanish/LOFT) Online 5 and 8

Science (English/fixed-DEI) Paper 5 and 8

Science (English/fixed-braille) Paper 5 and 8

Given the intended uses of these tests, both reliability evidence and validity evidence are necessary to support appropriate inferences of student academic achievement from the WVGSA scores. The analyses to support reliability and validity evidence that are reported in this volume were conducted on the basis of students’ completed test results, which were obtained through the online administration of English versions.

The purpose of this report is to provide empirical evidence that will support a validity argument for the uses of and inferences from the WVGSA. This volume addresses the following four topics:

• Reliability. The reliability of WVGSA adaptive test forms are estimated in the item response theory (IRT) framework using marginal reliability. The reliability estimates are presented by grade and subject, as well as by demographic subgroup. This discussion also



includes conditional standard errors of measurement (CSEM), the reliability of performance classifications, and interrater reliability of ELA writing scores provided by AIR’s AutoScoring Model.

• Content validity. This section presents evidence showing that test forms were constructed to measure the West Virginia College- and Career-Readiness Standards (WVCCRs) with a sufficient number of items targeting each area of the test blueprint.

• Internal structure validity. Evidence is provided regarding the internal relationships among the subscale scores to support their use and to justify the item response theory (IRT) measurement model. This type of evidence includes observed and disattenuated Pearson correlations among reporting categories. As explained in detail in Volume 1 of this technical report, for science, the IRT model is a multidimensional model, with an overall dimension representing proficiency in science and nuisance dimensions that take into account within-item local dependencies among scoring assertions. In this volume, evidence is provided with respect to the presence of item cluster effects.

• Relationship of test scores to external variables: Evidence of convergent and discriminant validity is provided using observed and disattenuated subscore correlations both within and across subjects. The correlations between interim and summative assessments, as well as the correlation between SBAC spring 2017 and WVGSA spring 2018 summative assessments in ELA and mathematics, are also presented.

• Test fairness. Fairness is statistically analyzed using differential item functioning (DIF) in tandem with content alignment reviews by specialists.

1.1 RELIABILITY

Reliability refers to consistency in test scores. Reliability can be defined as the degree to which individuals’ deviation scores remain relatively consistent over repeated administrations of the same test or alternate test forms (Cronbach & Algina, 1986). For example, if a person takes the same or parallel tests repeatedly, he or she should receive consistent results. The reliability coefficient refers to the ratio of true score variance to observed score variance:

ρXX′ =σT2

σX2

Another way to view reliability is to consider its relationship with the standard errors of measurement (SEM)—the smaller the standard error, the higher the precision of the test scores. For example, classical test theory assumes that an observed score (X) of an individual can be expressed as a true score (T) plus some error (E), 𝑋𝑋 = 𝑇𝑇 + 𝐸𝐸. The variance of 𝑋𝑋 can be shown to be the sum of two orthogonal variance components:

𝜎𝜎𝑋𝑋2 = 𝜎𝜎𝑇𝑇2 + 𝜎𝜎𝐸𝐸2

Returning to the definition of reliability as the ratio of true score variance to observed score variance, we can arrive at the following theorem:



ρXX′ =σT2

σX2=σx2 − σE2

σX2= 1 −

σE2

σX2

As the fraction of error variance to observed score variance tends to zero, the reliability then tends to 1. The classical test theory SEM, which assumes a homoscedastic error, is derived from the classical notion expressed above as 𝜎𝜎𝑋𝑋�1 − ρXX′ , where 𝜎𝜎𝑋𝑋 is the standard deviation of the scaled score, and ρXX′ is a reliability coefficient. Based on the definition of reliability, this formula can be derived as follows:

ρXX′ = 1 −σE2

σX2,

σE2

σX2= 1 − ρXX′ ,

σE2 = σX2(1 − ρXX′),

𝜎𝜎𝐸𝐸 = 𝜎𝜎𝑋𝑋�(1 − ρXX′).

In general, the standard error of measurement is relatively constant across samples, as the group dependent term, 𝜎𝜎𝑋𝑋, can be shown to cancel out:

𝜎𝜎𝐸𝐸 = 𝜎𝜎𝑋𝑋�(1 − ρXX′) = 𝜎𝜎𝑋𝑋�(1 − (1 −σE2

σX2)) = 𝜎𝜎𝑋𝑋�

σE2

σX2= 𝜎𝜎𝑋𝑋 ∙

𝜎𝜎𝐸𝐸𝜎𝜎𝑋𝑋

= 𝜎𝜎𝐸𝐸.

This shows that the standard error of measurement in the classical test theory is assumed to be a homoscedastic error, irrespective of the standard deviation of a group.

In contrast, the standard errors of measurement in IRT vary over the ability continuum. These heterogeneous errors are a function of a test information function that provides different information about examinees depending on their estimated abilities. Often, the test information function (TIF) is maximized over an important performance cut, such as the proficiency cut score.

Because the TIF indicates the amount of information provided by the test at different points along the ability scale, its inverse indicates the lack of information at different points along the ability scale. This lack of information is the uncertainty, or the measurement error, of the score at various score points. Conventionally, fixed-form tests are maximized near the middle of the score distribution, or near an important classification cut, and have less information at the tails of the score distribution. See Section 3.3 for the derivation of heterogeneous errors in IRT.

1.2 VALIDITY

Validity refers to the degree to which “evidence and theory support the interpretations of test scores entailed by proposed uses of tests” (American Educational Research Association [AERA], American Psychological Association [APA], and National Council on Measurement in Education [NCME], 2014). Messick (1989) defines validity as “an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and



appropriateness of inferences and actions based on test scores and other modes of assessment.” Both definitions emphasize evidence and theory to support inferences and interpretations of test scores. The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014) suggest five sources of validity evidence that can be used in evaluating a proposed interpretation of test scores. When validating test scores, these sources of evidence should be carefully considered.

The first source of evidence for validity is the relationship between the test content and the intended test construct (see Section 4). For test score inferences to support a validity claim, the items should be representative of the content domain, and the content domain should be relevant to the proposed interpretation of test scores. To determine content representativeness, diverse panels of content experts conduct alignment studies, in which experts review individual items and rate them based on how well they match the test specifications or cognitive skills required for a construct (see Volume 2 for details). Test scores can be used to support an intended validity claim when they contain minimal construct-irrelevant variance. For example, a mathematics item targeting a specific mathematics skill that also requires advanced reading proficiency and vocabulary has a high level of construct-irrelevant variance. Thus, the intended construct of measurement is confounded, which impedes the validity of the test scores. Statistical analyses, such as factor analysis or multi-dimensional scaling of relevance, are also used to evaluate content relevance. Results from factor analysis for the fixed form spring 2018 WVGSA are presented in Section 5.2 for ELA and mathematics. Factor analysis was not possible for spring 2019 due to the switch to computer adaptive testing. Evidence based on test content is a crucial component of validity because construct underrepresentation or irrelevancy can result in unfair advantages or disadvantages to one or more groups of examinees.

Technology-enhanced items should be examined to ensure that no construct-irrelevant variance is introduced. If some aspect of the technology impedes or advantages a student in his or her responses to items, this could affect item responses and inferences regarding abilities on the measured construct (see Volume 2). For ELA and mathematics, AIRCore makes use of the technology-enhanced items developed by AIR, and the items are delivered by the same engine used for delivery of the Smarter Balanced assessment. Hence, the WVGSA makes use of items that have the same technology-enhanced functionality as those found on these other assessments. The same engine is used to deliver the science assessment. Science clusters typically consist of multiple interactions; interactions have the same technology-enhanced functionality as the ELA and mathematics assessments.

The second source of validity evidence is based on “the fit between the construct and the detailed nature of performance or response actually engaged in by examinees” (AERA, APA, & NCME, 2014). This evidence is collected by surveying examinees about their performance strategies or responses to specific items. Because items are developed to measure specific constructs and intellectual processes, evidence that examinees have engaged in relevant performance strategies to correctly answer the items supports the validity of the test scores.

The third source of evidence for validity is based on internal structure: the degree to which the relationships among test items and test components relate to the construct on which the proposed test scores are interpreted. Differential item functioning, which determines whether specific items may function differently for subgroups of examinees, is one method for analyzing the internal



structure of tests (see Volume 1). Other possible analyses to examine internal structure are dimensionality assessment, goodness-of-model-fit to data, and reliability analysis (see Sections 3 and 5 for details).

A fourth source of evidence for validity is the relationship of test scores to external variables. The Standards (AERA, APA, & NCME, 2014) divides this source of evidence into three parts: convergent and discriminant evidence, test-criterion relationships, and validity generalization. Convergent evidence supports the relationship between the test and other measures intended to assess similar constructs. Conversely, discriminant evidence delineates the test from other measures intended to assess different constructs. To analyze both convergent and discriminant evidence, a multitrait-multimethod matrix can be used. Additionally, test-criterion relationships indicate how accurately test scores predict criterion performance. The degree of accuracy mainly depends on the test’s purpose, such as classification, diagnosis, or selection. Test-criterion evidence is also used to investigate predictions of favoring different groups. Due to construct underrepresentation or construct-irrelevant components, the relation of test scores to a relevant criterion may differ from one group to another. Furthermore, validity generalization is related to whether the evidence is situation-specific or can be generalized across different settings and times. For example, sampling errors or range restriction may need to be considered in order to determine whether the conclusions of a test can be assumed for the larger population.

Fifth, the intended and unintended consequences of test use should be included in the test-validation process. Determining the validity of the test should depend upon evidence directly related to the test; this process should not be influenced by external factors. For example, if an employer administers a test to determine hiring rates for different groups of people, an unequal distribution of skills related to the measurement construct does not necessarily imply a lack of validity for the test. However, if the unequal distribution of scores is in fact due to an unintended, confounding aspect of the test, this would interfere with the test’s validity. As described in Volume 1 and in this volume, test use should align with the intended purpose of the test.

Supporting a validity argument requires multiple sources of validity evidence. This enables one to evaluate if sufficient evidence has been presented to support the intended uses and interpretations of the test scores. Thus, determining the validity of a test first requires an explicit statement regarding the intended uses of the test scores, and, subsequently, evidence that the scores can be used to support these inferences.

2. PURPOSE OF THE WEST VIRGINIA GENERAL SUMMATIVE ASSESSMENT

The primary purpose of West Virginia’s K–12 assessment system is to yield accurate information on students’ achievement of West Virginia’s education standards. The West Virginia General Summative Assessment (WVGSA) supports instruction and student learning by measuring growth in student achievement. Assessments can be used as indicators to determine whether students in West Virginia are ready with the knowledge and skills that are essential for college education and careers.

West Virginia’s educational assessments also provide evidence for the requirements of state and federal accountability systems. Test scores can be employed to evaluate students’ learning progress



and to help teachers to improve their instruction, which in turn has a positive effect on students’ learning over time.

The tests are constructed to measure student proficiency on The Standards (AERA, APA, & NCME, 2014). The test was developed in adherence to the principles of universal design to ensure that all students have access to the test content. Volume 2, Test Development, describes in more detail the WVGSA standards and test blueprints. Additional evidence of content validity can also be found in Section 4. The WVGSA test scores are useful indicators for understanding individual students’ academic achievement of the West Virginia’s content standards and evaluating whether students are progressing in their performance over time. Additionally, both individual and aggregated scores can be used to for measuring reliability of the test. The reliability of the test scores can be found in Section 3.

The WVGSA is a criterion-referenced test that is designed to measure student performance for English language arts (ELA) and mathematics on the West Virginia College and Career Readiness Standards and for science on the Next Generation Content Standards and Objectives for Science in West Virginia Schools (WV NxGen Science Standards). As a comparison, norm-referenced tests are designed to compare or rank all students with one another. The WVGSA standards and test blueprints are discussed in Volume 2.

The scale score and relative strengths and weaknesses at the reporting category (domain) level were provided for each student to indicate student strengths and weaknesses in different content areas of the test, relative to the other areas and to the district and state. These scores serve as useful feedback that teachers can use to tailor their instruction, provided that they are viewed with the same caution that accompanies use of reporting category scores. Thus, to support their practical use across the state, we must examine the reliability coefficients for and the validity of these test scores.

3. RELIABILITY

3.1 RELIABILITY FOR ELA AND MATHEMATICS

WVGSA ELA and mathematics are adaptive testing administrations. Because there is no set form in adaptive testing, marginal reliability was computed for the scale scores, taking into account the varying measurement errors across ability range. Marginal reliability is a measure of the overall reliability of an assessment based on the average conditional SEM, estimated at different points on the ability scale, for all students.

Marginal reliability (𝜌𝜌�) is defined as

𝜌𝜌� = [𝜎𝜎2 − �∑ 𝐶𝐶𝐶𝐶𝐸𝐸𝐶𝐶𝑖𝑖2𝑁𝑁

𝑖𝑖=1𝑁𝑁 �]/𝜎𝜎2,

where N is the number of students; is the conditional SEM of the theta score for student i; and is the variance of the theta score. The higher the reliability coefficient, the greater the precision of the test. Table 2 presents the marginal reliability coefficients for all students. The reliability coefficients for all subjects and grades range from 0.87 to 0.92. Appendix A provides

iCSEM2σ



further breakdown, including reliability coefficients for demographic subgroups and reporting categories.

Table 2: Reliability Coefficients, ELA and Mathematics

Subject Grade Reliability Subject Grade Reliability

ELA

3 0.89

Mathematics

3 0.92

4 0.88 4 0.92

5 0.90 5 0.92

6 0.89 6 0.89

7 0.90 7 0.87

8 0.91 8 0.89

3.2 RELIABILITY FOR SCIENCE

Classical test-theory-based reliability indices, are not appropriate for science for two reasons. First, the science test is administered under a linear-on-the-fly test design. Potentially, each student gets a unique set of items, whereas classical test-theory-based reliability indices require that the same set of items is administered to a (large) group of students. Second, since IRT methods are used for calibration and scoring, the measurement error of ability estimates is not constant across the ability range, even for the same set of items. The reliability of science is computed in the same way as the marginal reliability defined in section 3.1. The marginal reliability of science for the overall sample is reported by grade in Table 3. The overall reliability ranges from 0.84 to 0.86. The reliability for students who received a complete test (18 items) is about the same as the overall reliability for both grades. Due to the new structure of the science test, AIR has also explored the relationships between reliability and other important factors, such as the effect of nuisance dimension (see Volume 1, Section 5.2.1). It was found that if the local dependencies among assertions pertaining to the same item are ignored, the marginal reliability increases to approximately 0.90. Ignoring local dependencies can be achieved either by computing the MLE ability estimates under the unidimensional Rasch model or by setting the variance parameters to zero for all item clusters when computing the MMLE ability estimates under the 1PL bifactor model (see Volume 1, Section 6.2.1).

Table 3: Marginal Reliability Coefficients for Science

Grade Sample Size Reliability

5 19,540 0.84

8 19,043 0.86



3.3 TEST INFORMATION CURVES AND STANDARD ERROR OF MEASUREMENT FOR ELA AND MATHEMATICS

Within the IRT framework, measurement error varies across the range of ability because of the test providing varied information across the range of ability as displayed by the test information function (TIF). The TIF describes the amount of information provided by the test at each score point along the ability continuum. The inverse of the TIF is characterized as the conditional measurement error at each score point. For instance, if the measurement error is large, less information is being provided by the assessment at the specific ability level.

Figure 1 displays a sample TIF with three vertical lines indicating the performance cuts. The graphic shows that this test information is maximized in the middle of the score distribution, meaning it provides the most precise scores in this range. The curve is lower at the tails, indicating that the test provides less information about examinees at the tails relative to the center.

Figure 1: Sample Test Information Function

0

1.5

3

4.5

6

7.5

9

10.5

12

13.5

15

-5 -4 -3 -2 -1 0 1 2 3 4 5

Info

rmat

ion

Theta

Information

Test Information level 2 level 3 level 4



Computing these TIFs is useful to evaluate where the test is maximally informative. In IRT, the TIF is based on the estimates of the item parameters in the test, and the formula used for the WVGSA is calculated as:

𝑇𝑇𝑇𝑇𝑇𝑇(𝜃𝜃𝑠𝑠) = � 𝐷𝐷2𝑎𝑎𝑖𝑖2 �∑ 𝑗𝑗2𝐸𝐸𝐸𝐸𝐸𝐸�∑ 𝐷𝐷𝑎𝑎𝑖𝑖(𝜃𝜃𝑠𝑠 − 𝑏𝑏𝑖𝑖𝑖𝑖)𝑗𝑗

𝑖𝑖=1 �𝑚𝑚𝑖𝑖𝑗𝑗=1

1 + ∑ 𝐸𝐸𝐸𝐸𝐸𝐸�∑ 𝐷𝐷𝑎𝑎𝑖𝑖(𝜃𝜃𝑠𝑠 − 𝑏𝑏𝑖𝑖𝑖𝑖)𝑗𝑗𝑖𝑖=1 �𝑚𝑚𝑖𝑖

𝑗𝑗=1

𝑁𝑁𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺

𝑖𝑖=1

− �∑ 𝑗𝑗𝐸𝐸𝐸𝐸𝐸𝐸�∑ 𝐷𝐷𝑎𝑎𝑖𝑖(𝜃𝜃𝑠𝑠 − 𝑏𝑏𝑖𝑖𝑖𝑖)𝑗𝑗

𝑖𝑖=1 �𝑚𝑚𝑖𝑖𝑗𝑗=1

1 + ∑ 𝐸𝐸𝐸𝐸𝐸𝐸�∑ 𝐷𝐷𝑎𝑎𝑖𝑖(𝜃𝜃𝑠𝑠 − 𝑏𝑏𝑖𝑖𝑖𝑖)𝑗𝑗𝑖𝑖=1 �𝑚𝑚𝑖𝑖

𝑗𝑗=1

�

2

� + � 𝐷𝐷2𝑎𝑎𝑖𝑖2 �𝑄𝑄𝑖𝑖𝑃𝑃𝑖𝑖�𝑃𝑃𝑖𝑖 − 𝑐𝑐𝑖𝑖1 − 𝑐𝑐𝑖𝑖

�2

�𝑁𝑁3𝐺𝐺𝑃𝑃

𝑖𝑖=1

,

where 𝑁𝑁𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺 is the number of items that are scored using the generalized partial credit model (GPCM) items; 𝑁𝑁3𝐺𝐺𝑃𝑃 is the number of items scored using 3PL or 2PL model; i indicates item i (𝑖𝑖 ∈{1,2, . . . ,𝑁𝑁}); 𝑚𝑚𝑖𝑖 is the maximum possible score of the item; s indicates student s; and 𝜃𝜃𝑠𝑠 is the ability of student s.

The standard error for estimated student ability (theta score) is the square root of the reciprocal of the TIF:

𝑠𝑠𝑠𝑠(𝜃𝜃) = 1

�𝑇𝑇𝑇𝑇𝑇𝑇(𝜃𝜃𝑖𝑖).

It is typically more useful to consider the inverse of the TIF rather than the TIF itself, as the standard errors are more useful for score interpretation. For this reason, standard error plots are presented in Figure 2 and Figure 3 respectively, instead of the TIFs for ELA and mathematics. The plots presented in this section are based on the scaled scores reported in spring 2019. Vertical lines represent the three achievement level cut scores.

Figure 2: Conditional Standard Errors of Measurement, ELA





Figure 3: Conditional Standard Errors of Measurement, Mathematics



The conditional standard error curves follow the typical expected trends with smallest values observed near the middle of the score scale. Desirably, the lowest standard errors are observed at the proficiency cut (the middle vertical line between Partially Meets Standard and Meets Standard score ranges) for most tests. Appendix B includes conditional standard errors of measurement at each scale score point and corresponding achievement levels.

3.4 STANDARD ERROR OF MEASUREMENT FOR SCIENCE

The computation method of conditional standard error for science has been described in the Section 6.2 of Volume 1. Figure 4 presents the conditional standard error curves for science. The lowest standard errors are observed near the proficiency cut for both grades, which is a desirable test property.



Figure 4: Conditional Standard Errors of Measurement, Science

3.5 RELIABILITY OF ACHIEVEMENT CLASSIFICATION

When student performance is reported in terms of achievement levels, a reliability of classifying students into a specific level can be computed in terms of the likelihood of accurate and consistent classification as specified in Standard 2.16 in The Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014).

The reliability of achievement classification can be examined in terms of the classification accuracy and classification consistency. Classification accuracy refers to the agreement between the classifications based on the form taken and the classifications that would be made based on the students’ true scores if hypothetically they could hypothetically be obtained. Classification consistency refers to the agreement between the classifications based on the form taken and the classifications that would be made based on an alternate, equivalently constructed test form.

In reality, the true ability is unknown, and students are not administered an alternate, equivalent form. Therefore, classification accuracy and consistency are estimated based on students’ item scores, the item parameters, and the assumed latent ability distribution as described in the following sections. The true score is an expected value of the test score with measurement error.

For student j, the student’s estimated ability is 𝜃𝜃�𝑗𝑗 with a standard error of measurement (SEM) of 𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�, and the estimated ability is distributed as 𝜃𝜃�𝑗𝑗~𝑁𝑁 �𝜃𝜃𝑗𝑗, 𝑠𝑠𝑠𝑠2�𝜃𝜃�𝑗𝑗��, assuming a normal distribution, where 𝜃𝜃𝑗𝑗 is the unknown true ability of student j. The probability of the true score at performance level 𝑙𝑙 (𝑙𝑙 = 1,⋯ , 𝐿𝐿) is estimated as



𝐸𝐸𝑗𝑗𝑗𝑗 = 𝐸𝐸(𝑐𝑐𝑃𝑃𝑗𝑗 ≤ 𝜃𝜃𝑖𝑖 < 𝑐𝑐𝑈𝑈𝑗𝑗) = 𝐸𝐸 � 𝑐𝑐𝑃𝑃𝑗𝑗 − 𝜃𝜃�𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

≤𝜃𝜃𝑗𝑗 − 𝜃𝜃�𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

< 𝑐𝑐𝑈𝑈𝑗𝑗 − 𝜃𝜃�𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

�

= 𝐸𝐸 �𝜃𝜃�𝑗𝑗 − 𝑐𝑐𝑈𝑈𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

<𝜃𝜃�𝑗𝑗 − 𝜃𝜃𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

≤ 𝜃𝜃�𝑗𝑗 − 𝑐𝑐𝑃𝑃𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

� = Φ�𝜃𝜃�𝑗𝑗 − 𝑐𝑐𝑃𝑃𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

� − Φ�𝜃𝜃�𝑗𝑗 − 𝑐𝑐𝑈𝑈𝑗𝑗𝑠𝑠𝑠𝑠�𝜃𝜃�𝑗𝑗�

�,

where 𝑐𝑐𝑃𝑃𝑗𝑗 and 𝑐𝑐𝑈𝑈𝑗𝑗 denote the score corresponding to the lower and upper limits of the performance level 𝑙𝑙, respectively.

Classification accuracy and consistency for all students as well as subgroups by achievement level is shown side by side for comparison in Appendix C.

3.5.1 Classification Accuracy

Using 𝐸𝐸𝑗𝑗𝑗𝑗, the expected number of students at level 𝑙𝑙, based on students from observed level k, can be expressed as

𝐸𝐸𝐴𝐴𝑖𝑖𝑗𝑗 = � 𝐸𝐸𝑗𝑗𝑗𝑗𝑝𝑝𝑗𝑗𝑗𝑗 𝜖𝜖 𝑖𝑖

,

where 𝐸𝐸𝑙𝑙𝑗𝑗 is the jth student’s performance level. The values of 𝐸𝐸𝐴𝐴𝑖𝑖𝑗𝑗 are the elements used to populate the matrix 𝑬𝑬𝑨𝑨, a 𝐿𝐿 × 𝐿𝐿 matrix of conditionally expected numbers of students to score within each performance level based on their true scores. The classification accuracy (CA) at level 𝑙𝑙 is estimated by

𝐶𝐶𝐴𝐴𝑗𝑗 =𝐸𝐸𝐴𝐴𝑖𝑖𝑗𝑗𝑁𝑁𝑖𝑖

,

where 𝑁𝑁𝑖𝑖 is the observed number of students scoring in performance level 𝑘𝑘.

The classification accuracy for the 𝐸𝐸th cut is estimated by forming square partitioned blocks of the matrix 𝑬𝑬𝑨𝑨 and taking the summation over all elements within the block as follows:

𝐶𝐶𝐴𝐴𝐶𝐶 = ��𝐸𝐸𝐴𝐴𝑖𝑖𝑗𝑗

𝑝𝑝

𝑗𝑗=1

𝑝𝑝

𝑖𝑖=1

+ � � 𝐸𝐸𝐴𝐴𝑖𝑖𝑗𝑗

𝑃𝑃

𝑗𝑗=𝑝𝑝+1

𝑃𝑃

𝑖𝑖=𝑝𝑝+1

� 𝑁𝑁� ,

where 𝑁𝑁 is the total number of students.

The overall classification accuracy is estimated from the diagonal elements of the matrix:

𝐶𝐶𝐴𝐴 =𝑡𝑡𝑡𝑡(𝑬𝑬𝑨𝑨)𝑁𝑁 .

Table 4 through Table 6 provide the overall classification accuracy and the classification accuracy for the individual cuts for ELA, mathematics and science. The overall classification accuracy of the test ranges from 77% to around 80% for ELA, from 80% to 81% for mathematics, and from 75% to 76% for science. The individual cut accuracy rates are high across all grades, forms, and subjects, with the minimum value being 89.60% for science grade 5. It denotes that more than 89%



of the time we can accurately differentiate students between adjacent achievement levels in the spring 2019 WVGSA.

Table 4: Classification Accuracy Index, ELA

Grade Overall Accuracy (%)

Cut Accuracy (%)

Cut 1 Cut 2 Cut 3

3 78.78 92.83 91.72 94.21

4 77.13 92.57 91.57 92.89

5 78.07 92.98 91.53 93.49

6 78.70 92.69 91.44 94.55

7 79.82 92.98 91.87 94.95

8 79.91 93.38 92.05 94.47

Table 5: Classification Accuracy Index, Mathematics

Grade Overall Accuracy (%)

Cut Accuracy (%)

Cut 1 Cut 2 Cut 3

3 81.03 94.37 92.59 94.06

4 80.13 94.13 92.06 93.89

5 80.55 92.99 92.67 94.85

6 80.83 92.24 92.90 95.65

7 79.95 92.13 92.58 95.17

8 80.74 92.74 93.07 94.76

Table 6: Classification Accuracy Index, Science

Grade Overall Accuracy (%) Cut Accuracy (%)

Cut 1 Cut 2 Cut 3

5 74.97 90.89 89.60 94.41

8 75.83 89.71 91.34 94.69

3.5.2 Classification Consistency

Assuming the test is administered twice independently to the same group of students, similarly to accuracy, a 𝐿𝐿 × 𝐿𝐿 matrix 𝑬𝑬𝑪𝑪 can be constructed. The element of 𝑬𝑬𝑪𝑪 is populated by



𝐸𝐸𝐺𝐺𝑖𝑖𝑗𝑗 = �𝐸𝐸𝑗𝑗𝑗𝑗𝐸𝐸𝑗𝑗𝑖𝑖

𝑁𝑁

𝑗𝑗=1

,

where 𝐸𝐸𝑗𝑗𝑗𝑗 is the probability of the true score at performance level 𝑙𝑙 in test one, and 𝐸𝐸𝑗𝑗𝑖𝑖 is the probability of the true score at performance level 𝑘𝑘 in test two for the 𝑗𝑗th student. The classification consistency index for the cuts (CCC) and overall classification consistency (CC) were estimated in a way similar to CAC and CA.

𝐶𝐶𝐶𝐶𝐶𝐶 = ��𝐸𝐸𝐺𝐺𝑖𝑖𝑗𝑗

𝑝𝑝

𝑗𝑗=1

𝑝𝑝

𝑖𝑖=1

+ � � 𝐸𝐸𝐺𝐺𝑖𝑖𝑗𝑗

𝑃𝑃

𝑗𝑗=𝑝𝑝+1

𝑃𝑃

𝑖𝑖=𝑝𝑝+1

� 𝑁𝑁� ,

and

𝐶𝐶𝐶𝐶 =𝑡𝑡𝑡𝑡(𝑬𝑬𝑪𝑪)𝑁𝑁 .

Table 7 through Table 9 provide the overall classification consistency and classification consistency for the individual cuts for ELA, mathematics, and science. The overall classification consistency of the test ranges from 68% to 72% for ELA, from 72% to 73% for mathematics, and from 66% to 67% for science. The individual cut consistency rates are high across all grades, forms, and subjects, with the minimum value being 85.44% for science grade 5. In all achievement levels, classification accuracy is slightly higher than classification consistency. Classification consistency rates can be lower than classification accuracy; the consistency is based on two tests with measurement errors, but the accuracy is based on one test with a measurement error and the true score. The accuracy and consistency rates for each achievement level are higher for the levels with smaller standard error.

Table 7: Classification Consistency Index, ELA

Grade Overall Consistency (%)

Cut Consistency (%)

Cut 1 Cut 2 Cut 3

3 70.32 89.80 88.34 91.81

4 68.30 89.44 88.08 89.97

5 69.59 90.04 88.05 90.82

6 70.16 89.67 87.88 92.31

7 71.56 89.96 88.54 92.83

8 71.81 90.61 88.77 92.21



Table 8: Classification Consistency Index, Mathematics

Grade Overall Consistency (%)

Cut Consistency (%)

Cut 1 Cut 2 Cut 3

3 73.47 92.06 89.56 91.61

4 72.42 91.71 88.83 91.36

5 72.99 90.16 89.66 92.73

6 73.15 88.99 89.93 93.84

7 71.93 88.69 89.49 93.19

8 73.38 89.66 90.18 92.58

Table 9: Classification Consistency Index, Science

Grade Overall Consistency (%) Cut Consistency (%)

Cut 1 Cut 2 Cut 3

5 65.59 87.27 85.44 92.11

8 66.80 85.68 87.82 92.55

3.6 PRECISION AT CUT SCORES

Table 10 through Table 12 present the mean CSEM at each achievement level by grade and subject. These tables also include achievement level cut scores and associated CSEM.

Table 10: Achievement Levels and Associated CSEM, ELA

Grade Achievement Level Mean CSEM Cut Score (Scale Score) CSEM at Cut Score

3

1 16.55 - -

2 11.45 550 12.16

3 10.53 586 10.93

4 10.78 616 10.25

4

1 21.30 - -

2 12.94 563 14.32

3 12.15 599 12.19

4 13.51 629 12.42

5

1 17.35 - -

2 12.51 588 12.98

3 12.61 622 12.29

4 13.79 655 13.09




6

1 21.06 - -

2 13.82 597 15.83

3 12.61 639 12.74

4 13.32 680 12.53

7

1 19.34 - -

2 13.20 602 14.23

3 12.64 644 12.65

4 13.45 685 12.86

8

1 17.49 - -

2 13.22 613 13.80

3 13.13 656 13.01

4 14.75 698 13.52

Table 11: Achievement Levels and Associated CSEM, Mathematics


3

1 11.99 - -

2 7.57 401 8.10

3 7.29 426 7.25

4 8.09 448 7.39

4

1 16.24 - -

2 9.68 422 10.58

3 9.06 456 9.23

4 9.68 478 9.02

5

1 17.25 - -

2 10.94 449 11.63

3 10.50 487 10.58

4 11.01 513 10.49

6

1 23.30 - -

2 13.75 474 15.19

3 11.94 518 12.43

4 11.66 550 11.55




7

1 29.49 - -

2 15.01 503 16.51

3 13.74 548 13.96

4 13.69 583 13.56

8

1 32.85 - -

2 17.21 529 19.01

3 15.22 587 15.71

4 15.13 617 14.89

Table 12: Achievement Levels and Associated Conditional Standard Error of Measurement, Science


5

1 6.69 - -

2 5.47 537 5.60

3 5.58 555 5.44

4 7.05 568 5.81

8

1 7.02 - -

2 5.39 837 5.56

3 5.37 855 5.29

4 6.36 867 5.53

3.7 ELA WRITING PROMPTS INTER-RATER RELIABILITY

The spring 2019 writing responses were scored with the AIR AutoScoring Model. The validity of this machine scoring system was assessed at the beginning of the testing window.

3.7.1 Automated Essay-Scoring Engine

The AIR AutoScoring Model, AIR’s automated essay-scoring engine, uses a statistical process to evaluate writing prompts. The AIR AutoScoring Model evaluates papers against the same rubric used by human raters, but a statistical process is used to analyze each paper and assign scores for each of the three dimensions. The engine uses the same process for scoring essays every time that a new prompt is submitted, regardless of whether the data is obtained from an operational assessment or an independent field test.

Statistical rubrics are effectively proxy measures. Although they can directly measure some aspects of writing conventions (e.g., use of passive voice, misspellings, run-on sentences), they do



not directly measure argument structure or content relevance. Hence, while statistical rubrics often prove useful for scoring essays and even for providing some diagnostic feedback in writing, they do not develop a sufficiently specific model of the correct semantic structure to score many propositional items. Furthermore, they cannot provide the explanatory or diagnostic information available from an explicit rubric. For example, the frequency of incorrect spellings may predict whether a response to a factual item is correct; high-performing students may also have better spelling skills. Spelling may prove useful in predicting the human score, but it is not the reason that the human scorer deducts points. Statistical rubrics are not about explanation or reason but rather about a prediction of how a human would score the response.

The AIR AutoScoring engine uses a statistical rubric with great success, as measured by the rater agreements observed relative to the human-to-human rater agreements. This technology is similar to all essay-scoring systems in the field. Although some systems replace the statistical process with a “neural network” algorithm, that algorithm functions like the statistical model. Not all descriptions of essay-scoring algorithms are as transparent as AIR’s, but any time that a training set is used for the machine to “learn a rubric,” the same technology is being used.

The engine is designed to employ a “training set”: a set of essays scored with maximally valid scores that are used to form the basis of the prediction model. The quality of the human-assigned scores is critical to the identification of a valid model and final performance of the essay-scoring engine. Moreover, an ideal training sample over-represents high- and low-scoring papers and is selected according to a scientific sampling design with known probabilities of selection.

The training process of the essay-scoring engine has two phases. The first phase requires oversampled, high- and low-scoring papers, leaving an equally weighted representative sample for the second phase. The first phase is used to identify concepts that are proportionately represented in high-scoring papers. Here, concepts are defined as words and their synonyms, as well as clusters of words used meaningfully in proximity.

The second phase takes a series of measures on each essay in the remaining training set. These measures include latent semantic analysis (LSA) based on the concepts identified in the first phase; other semantic measures indicate the coherence of concepts within and across paragraphs and a range of word-use and syntactic measures. LSA is similar to a data-reduction method identifying common concepts within the narrative and reducing the data to a configurable number of LSA dimensions.

For each trait in the rubric, the system estimates an appropriate statistical model in which these LSA and other syntactic characteristics described earlier serve as the independent variables, and the final, resolved score serves as the dependent variable in an ordered probit regression. This model, along with its final parameter estimates, is used to generate a predicted or “proxy” score. The probability of scoring in the pth category is compared to a random draw from the uniform distribution, and a final score point of 1 through 4 is determined from this comparison.

In addition to the training set, an independent random sample of responses is drawn for the cross-validation of the identified scoring rubric. As with the training set, student responses in the cross-validation study are hand scored, and the LSA and other syntactic characteristics of the papers are computed. Subsequently, a second machine score is generated by applying the model coefficients obtained from the ordered probit in the training set. This forms a predicted score for



the papers in the cross-validation set for each dimension in the rubric, which can then be used to evaluate the agreement rates between the human and autoscore engine.

When implementing the essay-scoring engine, we expect the computer-to-human agreement rates to be at least as high as the human-to-human agreement rates obtained from the double-scored process. If the engine yields scores with rater agreement rates that are at least as high as the human-to-human rater agreement rates, then the scoring engine can be deployed for operational scoring. If the computer-to-human agreement rates are not as high as the human-to-human rates, then adjustments to the scoring engine statistical model are necessary to find a scoring model that yields rater agreement rates that match the human-to-human rates.

Table 16 presents the weighted kappa for the training sample and the validation sample for the writing prompts adopted in the spring 2019 WVGSA. The validation sample generally has higher or similarly weighted kappa than the training sample.

To train AIR’s automated essay-scoring engine, a subset of papers was scientifically selected and scored by two human raters. To train AIR’s scoring engine, a subset of papers was selected using stratified random sampling and scored by two human raters. Essay responses to the AIRCore writing prompts were sent to the vendor Measurement Incorporated or Data Recognition Corporation for human scoring. Using anchor papers selected by content experts and finalized rubrics (Table 13), human raters were trained to score writing responses at the rangefinding meeting. Raters revisit anchor papers and rubrics, including a range of sample responses and scores, at rangefinding meetings to refamiliarize themselves with scoring.

At the rangefinding meeting, raters were assigned to groups. Training the raters began as the leader of each group read student responses out loud to raters. The raters independently referred to the anchors and rubrics, and they shared what they thought the score for the particular response should be. If the decision among raters was unanimous, they engaged in a brief discussion and moved to the next response. If the decision was not unanimous, the raters had a discussion, referring to the anchors and rubrics to reach a consensus.

Table 13: Writing Rubrics

Dimension Rubric Maximum Score Point

Conventions

The response demonstrates an adequate command of basic conventions. The response may include the following:

• Some minor errors in usage but no patterns of errors • Adequate use of punctuation, capitalization, sentence formation, and

spelling

2

Evidence and Elaboration

The response provides thorough and convincing support, citing evidence for the controlling idea or main idea that includes the effective use of sources, facts, and details. The response includes most of the following:

• Smoothly integrated, thorough, and relevant evidence, including precise references to sources

• Effective use of a variety of elaborative techniques (including but not limited to definitions, quotations, and examples) demonstrating an understanding of the topic and text

• Clear and effective expression of ideas using precise language

4



Dimension Rubric Maximum Score Point

• Academic and domain-specific vocabulary clearly appropriate for the audience and purpose

• Varied sentence structure demonstrating language facility

Purpose, Focus, and

Organization

The response is fully sustained and consistently focused within the purpose, audience, and task; it has a clear controlling idea and effective organizational structure creating coherence and completeness. The response includes most of the following:

• Strongly maintained controlling idea with little or no loosely related material

• Skillful use of a variety of transitional strategies to clarify the relationships between and among ideas

• Logical progression of ideas from beginning to end with a satisfying introduction and conclusion

• Appropriate style and objective tone established and maintained

4

Two trained raters score each writing item response sent for human scoring. When scores from reader 1 and reader 2 are not in adjacent agreement, the response is sent for resolution scoring by a team leader or scoring director. The final item score is based on the resolution score, when present, or on the initial read. Score discrepancies are resolved before being sent to AIR. Percent agreement rates are computed to ensure that the machine scores are comparable to the human scores. As seen in Table 14, the percentage of exact agreement (when two raters gave the same score), the percentage of adjacent ratings (when the difference between two raters was 1), and the percentage of non-adjacent ratings (when the difference was larger than 1) are all computed. In this example, the exact agreement was 2/4, or 50%, and the adjacent and non-adjacent percentages were 25% each.

Table 14: Rating Agreement Example

Response Rater 1 Rater 2 Agreement

1 2 3 Adjacent

2 1 1 Exact

3 2 2 Exact

4 2 0 Non-Adjacent

Likewise, inter-rater reliability monitors how often scorers are in exact agreement with each other and ensures that an acceptable agreement rate is maintained. The calculations for inter-rater reliability in this report are as follows:

• Percent Exact: total number of responses by scorer in which scores are equal divided by the number of responses that were scored twice

• Percent Adjacent: total number of responses by scorer in which scores are one score point apart divided by the number of responses that were scored twice

• Percent Non-Adjacent: total number of responses by scorer where scores are more than one score point apart divided by the number of responses that were scored twice, when applicable



Table 15 displays percent agreement in the training sample and validation sample. The total number of LSA dimensions and the sample size for the comparisons are also presented in Table 15. In the training sample, the percentage of exact agreement ranged from 62% to 79%. The percentage of adjacent rating ranged from 20% to 37%. The percentage of non-adjacent rating ranged from 0% to 4%. In the validation sample, the percentage of exact agreement ranged from 65% to 83%. The percentage of adjacent rating ranged from 17% to 35%. The non-adjacent rating percentages ranged from 0% to 2%. Table 15 shows that the essay-scoring engine produced comparable results with human scores.



Table 15: Percent Agreement in Human-Scoring and Auto-Scoring

Grade Item ID Dimension Human Scoring in Training Sample AIR Auto-Scoring Model in Validation Sample

% Exact % Adjacent % Non- Adjacent LSA % Exact % Adjacent % Non-

Adjacent N for

comparison

3

14161

Conventions 68.95 28.31 2.74 40 71.69 27.63 0.68 438

Evidence and Elaboration 67.04 31.38 1.58 10 67.49 31.60 0.90 443

Purpose, Focus, and Organization 64.58 34.03 1.39 100 67.82 31.71 0.46 432

14162

Conventions 68.37 30.07 1.56 50 73.05 24.94 2.00 449



4

14163

Conventions 64.18 34.07 1.76 50 69.67 29.67 0.66 455

Evidence and Elaboration 76.13 23.44 0.43 40 80.22 19.78 0 465

Purpose, Focus, and Organization 68.60 30.75 0.65 40 74.41 25.59 0 465

14164

Conventions 64.82 34.76 0.42 40 69.94 29.64 0.43 469


Purpose, Focus, and Organization 73.34 26.24 0.44 40 77.42 22.58 0 465

5

14165

Conventions 71.74 28.26 0 50 74.13 24.78 1.09 460

Evidence and Elaboration 72.11 27.02 0.87 40 72.55 27.45 0 459


14172

Conventions 73.13 26.88 0 10 75.21 23.96 0.83 480





Grade Item ID Dimension Human Scoring in Training Sample AIR Auto-Scoring Model in Validation Sample

% Exact % Adjacent % Non- Adjacent LSA % Exact % Adjacent % Non-

Adjacent N for

comparison

6

14166

Conventions 68.16 31.62 0.22 40 77.95 21.16 0.89 449



14167

Conventions 67.50 31.26 1.26 50 75.00 24.58 0.42 480


Purpose, Focus, and Organization 64.22 31.78 4 100 70.32 28.63 1.05 475

7

14168

Conventions 78.42 21.58 0 40 83.40 16.60 0 482

Evidence and Elaboration 73.92 26.08 0 50 81.36 18.64 0 483


14169

Conventions 64.66 34.91 0.43 50 76.50 23.06 0.43 464



8

14170

Conventions 76.28 22.86 0.86 100 77.99 22.01 0 468



14171

Conventions 76.50 23.28 0.22 40 80.09 19.47 0.44 452



* Essays that were given a condition code by AutoScoring Model or human readers were excluded



In addition to the percent agreement rates, the quadratic-weighted kappa values were computed for the training sample and the validation sample for the writing prompts adopted in the spring 2019 WVGSA.

Cohen’s kappa (Cohen, 1968) is an index of inter-rater agreement after accounting for the agreement that could be expected due to chance. This statistic can be computed as

𝐾𝐾 =𝑃𝑃𝑜𝑜 − 𝑃𝑃𝑐𝑐1 − 𝑃𝑃𝑐𝑐

,

where 𝑃𝑃𝑜𝑜 is the proportion of observed agreement, and 𝑃𝑃𝑐𝑐 indicates the proportion of agreement by chance. Cohen’s kappa treats all disagreement values with equal weight. Quadratic-weighted kappa coefficients (Cohen, 1968), however, allow unequal weights, which can be used as a measure of validity. Quadratic-weighted kappa coefficients were calculated using the formula below:

𝐾𝐾𝑤𝑤 =𝑃𝑃′𝑜𝑜 − 𝑃𝑃′𝑐𝑐1 − 𝑃𝑃′𝑐𝑐

,

where

𝑃𝑃′𝑜𝑜 =∑𝑤𝑤𝑖𝑖𝑗𝑗𝐸𝐸𝑜𝑜𝑖𝑖𝑗𝑗𝑤𝑤𝑚𝑚𝑚𝑚𝑚𝑚

,

𝑃𝑃′𝑐𝑐 =∑𝑤𝑤𝑖𝑖𝑗𝑗𝑝𝑝𝑐𝑐𝑖𝑖𝑗𝑗𝑤𝑤𝑚𝑚𝑚𝑚𝑚𝑚

,

where 𝐸𝐸𝑜𝑜𝑖𝑖𝑗𝑗 is the proportion of the judgments observed in the ijth cell, 𝐸𝐸𝑐𝑐𝑖𝑖𝑗𝑗 is the proportion in the ijth cell expected by chance, and 𝑤𝑤𝑖𝑖𝑗𝑗 is the disagreement weight.

Table 16 shows the quadratic-weighted kappa for the training sample and the validation sample. The kappa index ranges from 0 to 1, where values of 0 indicate no agreement, and values of 1 indicate perfect agreement. In the training sample, quadratic-weighted kappa coefficients for grades 3–8 operational writing prompts by dimension ranged from 0.52 to 0.81. In the validation sample, the range was from 0.56 to 0.76. The validation sample generally has higher or similar weighted kappa values than the training sample.



Table 16: Quadratic-Weighted Kappa Coefficients

Grade Item ID Dimension Quadratic-Weighted Kappa

Two Human Raters Human and Machine

3

14161

Convention 0.53 0.64

Elaboration 0.68 0.63

Purpose 0.65 0.65

14162



Purpose 0.62 0.59

4

14163



Purpose 0.54 0.58

14164



Purpose 0.62 0.64

5

14165



Purpose 0.64 0.61

14172



Purpose 0.60 0.58

6

14166



Purpose 0.61 0.60

14167



Purpose 0.64 0.70

7

14168



Purpose 0.59 0.67

14169



Purpose 0.58 0.56



Grade Item ID Dimension Quadratic-Weighted Kappa

Two Human Raters Human and Machine

8

14170



Purpose 0.74 0.70

14171



Purpose 0.81 0.75

The AIR AutoScoring Model can generate condition codes to indicate that the response provided by the student is considered invalid and therefore incorrect. All condition codes receive the lowest possible dimension score for purposes of ability estimation. The machine-generated condition codes, also referred to as rule-based condition codes, are as follows:

• NO_RESPONSE. No non-blank characters are detected in the response. • NOT_ENOUGH_DATA. Student response is less than the minimum number of words

configured in the rubric. • PROMPT_COPY_MATCH. Student response is copied from the passage or item prompt

(currently flagged when a 74% match is found, but this parameter is configurable). • DUPLICATE_TEXT. Student response is repeated text copied over and over (currently

flagged when a 70% match is found, but this parameter is configurable). • NONSPECIFIC. Essay scoring engine predicts the assignment of a condition code. Even

after training the system, there can be responses that do not fall into any of the pre-set categories. For those responses, the system will generate a condition code of NONSPECIFIC.

Based on tests administered using the AIRCore Interim and Summative test banks, a confidence index is produced for each dimension of the prompts used for the summative spring 2019 writing assessment. The essays that received the lowest 15 percent of confidence index values and any responses that receives a condition code of NONSPECIFIC were routed for human verification. In addition, as a part of AutoScoring Model quality control, the first 500 students’ responses for each writing prompt were verified by human readers.

The human verification process was conducted by the sequence described below:

• If the verification reader assigned a score that was the same as the machine-assigned score, the machine-assigned score was accepted to be the final dimension score.

• If the first verification reader did not assign the same score as the machine-assigned score, the essay was sent to the second verification reader. If the second reader’s score matched with either machine or the first reader’s score, the matching score was accepted to be the final score.



• If the second verification reader’s score did not match with the machine or first reader’s score, the essay was sent to the scoring supervisor for assigning the final score.

• If a verification reader assigned a condition code, the condition code was accepted to be the final score.

Table 17 provides the agreement rate and quadratic-weighted Kappa coefficients between the scores provided by the AIR AutoScoring model and the first human verification reader in a sample of students who submitted their essay responses during an early period of the testing window.

Table 17: The First 500 Cases Percent Agreement in Human-Scoring and AutoScoring

Grade Item ID Dimension

Human and AIR AutoScoring Model Agreement in the First 500 Cases

% Exact % Adjacent % Non- Adjacent Q W Kappa N

3

14161

Conventions 50.92 43.00 6.08 0.50 493

Evidence and Elaboration 50.10 48.28 1.62 0.38 493

Purpose, Focus, and Organization 55.78 43.00 1.22 0.49 493

14162

Conventions 42.94 50.40 6.66 0.40 496



4

14163

Conventions 63.86 34.34 1.80 0.53 498



14164

Conventions 59.32 40.28 0.40 0.55 499



5

14165

Conventions 69.28 29.72 1.00 0.62 498



14172

Conventions 64.40 34.20 1.40 0.57 500



6

14166

Conventions 70.28 28.12 1.60 0.54 498



14167

Conventions 75.56 24.24 0.20 0.69 499





Grade Item ID Dimension

Human and AIR AutoScoring Model Agreement in the First 500 Cases

% Exact % Adjacent % Non- Adjacent Q W Kappa N

7

14168

Conventions 75.56 24.24 0.20 0.64 499



14169

Conventions 78.26 21.74 0.00 0.71 497



8

14170

Conventions 82.32 17.48 0.20 0.62 492



14171

Conventions 79.32 20.08 0.60 0.67 498



4. EVIDENCE OF CONTENT VALIDITY

This section demonstrates that the knowledge and skills assessed by the WVGSA are representative of the content standards of the larger knowledge domain. We describe the content standards for the WVGSA and discuss the test development process, mapping WVGSA tests to The Standards. A complete description of the test development process can be found in Volume 2, Test Development.

4.1 CONTENT STANDARDS

The WVGSA was aligned to the ELA, mathematics, and science standards adopted by West Virginia in April 2017. The standards are available for review at the following URL: https://wvde.us/assessment/scaled-score-information/wvgsa-in-grades-3-8/. Blueprints were developed to ensure that the test and the items were aligned to the prioritized standards that they were intended to measure. A complete description of the blueprint and test construction process can be found in Volume 2 of the WVGSA technical reports.

Table 18 through Table 20 present the number of items in the item pool measuring each reporting category by grade for ELA, mathematics and science, respectively.

https://wvde.us/assessment/scaled-score-information/wvgsa-in-grades-3-8/



Table 18: Number of Items for Each Reporting Category, ELA

Grade Reporting Category Number of Items

3

Informational Text (IT) 111

Literary Text (LT) 101

Writing and Language (WL) 76

4

Informational Text 131

Literary Text 77

Writing and Language 84

5


Literary Text 76


6


Literary Text 94


7


Literary Text 103


8


Literary Text 51


Table 19: Number of Items for Each Reporting Category, Mathematics


3

Measurement, Data, and Geometry (MDG) 84

Numbers and Operations in Base Ten & Fractions (NBTF) 243

Operations and Algebraic Thinking (OAT) 150

4




5




6

Expressions and Equations (EE) 138

Geometry & Statistics and Probability (GSP) 54

Ratios and Proportional Relationships & Number System (RPNS) 192




7

Expressions and Equations (EE) 64

Geometry (G) 68

Ratios and Proportional Relationships & Number System (RPNS) 142

Statistics and Probability (SP) 63

8

Expressions and Equations & Number System (EENS) 147

Functions (F) 75

Geometry & Statistics and Probability (GSP) 153

Table 20: Number of Items for Each Reporting Category, Science

Grade Reporting Category Cluster Standalone

5

Earth and Space Science (ESS) 2 4

Life Science (LS) 2 4

Physics Science (PS) 2 4

8

Earth and Space Science (ESS) 2 4

Life Science (LS) 2 4

Physics Science (PS) 2 4

4.2 ALIGNMENT OF AIRCORE ITEM BANKS TO THE CONTENT STANDARDS AND BENCHMARKS

An independent alignment study is planned for spring 2019.

5. EVIDENCE ON INTERNAL-EXTERNAL STRUCTURE

In this section, the internal structure of the assessment is explored using the scores provided at the reporting category level. The relationship of the subscores is just one indicator of the test dimensionality.

In ELA, there are three reporting categories per grade: Reading Standards for Informational/Nonfiction Text, Reading Standards for Literature/Fiction, and Writing and Language Standards. In mathematics, reporting categories differ in each grade (see Table 15).

Scale scores and relative strengths and weaknesses based on each reporting category were provided to students. Evidence is needed to verify that scale scores and relative strengths and weaknesses for each reporting category provide both different and useful information for student achievement.

It may not be reasonable to expect that the reporting category scores are completely orthogonal—this would suggest that there are no relationships among reporting category scores and would make justification of a unidimensional IRT model difficult, though reporting these separate scores could



then easily be justified. On the contrary, if the reporting categories were perfectly correlated, a unidimensional model could be justified, but the reporting of separate scores could not.

One pathway to explore the internal structure of the test is via a second-order factor model, assuming a general mathematics construct (first factor) with reporting categories (second factor), and that the items load onto the reporting category they intend to measure. If the first-order factors are highly correlated and the model fits data well for the second-order model, this provides evidence of unidimensionality, as well as reporting subscores. A second-order factor model was fit to the spring 2018 fixed forms for ELA and mathematics.

The science assessment is modeled with the Rasch testlet model (Wang & Wilson, 2005). Unlike the models for ELA and mathematics, the IRT model for science is a high-dimensional model, incorporating a nuisance dimension for each item cluster, in addition to an overall dimension representing the overall proficiency in science. This approach is innovative and quite different from the traditional approach of ignoring local dependencies. Validity evidence on the internal structure will focus on the presence of cluster effects and how substantial they are.

Another pathway is to explore observed correlations between the subscores. However, as each reporting category is measured with a small number of items, the standard errors of the observed scores within each reporting category are typically larger than the standard error of the total test score. Disattenuating for measurement error could offer some insight into the theoretical true score correlations. Both observed correlations and disattenuated correlations are provided in the following section.

5.1 CORRELATIONS AMONG REPORTING CATEGORY SCORES

Table 21 and Table 22 present the observed and disattenuated correlation matrix of the reporting category raw scores for ELA and mathematics, respectively. Values in the lower triangle are observed correlations, and values in the upper triangle are disattenuated. Diagonals (highlighted in gray) are the reliability coefficient. In ELA, the observed correlations range from 0.51 and 0.64, and disattenuated correlations range from 0.70 to 0.84. In mathematics, the observed correlations among the reporting categories range from 0.57 to 0.78, and disattenuated correlations range from 0.79 to 0.97.

Table 23 presents the observed and disattenuated correlation matrix of the reporting category scale scores for science. The observed correlations range from 0.60 to 0.67, and disattenuated correlations range from 0.97 to 0.99.

In some instances, these correlations were lower than one might expect. However, as previously noted, the correlations were subject to a large amount of measurement error at the strand level, given the limited number of items from which the scores were derived. Consequently, over-interpretation of these correlations, as either high or low, should be made cautiously.



Table 21: Correlations among Reporting Categories, ELA

Grade Reporting Category Mean # of Items per Student

Cat1 Cat2 Cat3

3

Reading Informational Text (Cat1) 13.5 0.71* 0.70 0.72

Reading Literary Text (Cat2) 16.0 0.51 0.74* 0.71

Writing and Language (Cat3) 10.8 0.55 0.56 0.83*

4




5




6




7


Reading Literary Text (Cat2) 13.2 0.61 0.71 0.80

Writing and Language (Cat3) 10.1 0.61 0.61 0.81

8

Reading Informational Text (Cat1) 16.4 0.75 0.84 0.81



Table 22: Correlations among Reporting Categories, Mathematics

Grade Reporting Category Mean # of Items

per Student

Cat1 Cat2 Cat3 Cat4

3

Measurement, Data, and Geometry (Cat1) 9.4 0.76* 0.96 0.90 -

Numbers and Operations in Base Ten & Fractions (Cat2)

14.2 0.76 0.83* 0.93 -

Operations and Algebraic Thinking (Cat3) 10.4 0.71 0.77 0.82* -

4



16.1 0.71 0.85* 0.96 -




Grade Reporting Category Mean # of Items

per Student

Cat1 Cat2 Cat3 Cat4

5



16.1 0.77 0.85* 0.92 -


6

Expressions and Equations (Cat1) 11.5 0.77* 0.83 0.97 -

Geometry & Statistics and Probability (Cat2) 8.0 0.57 0.61* 0.86 -

Ratios and Proportional Relationships & Number System (Cat3) 14.5 0.77 0.61 0.82* -

7

Expressions and Equations (Cat1) 8.7 0.74* 0.79 0.85 0.79

Geometry (Cat2) 8.0 0.57 0.70* 0.87 0.81

Ratios and Proportional Relationships & Number System (Cat3) 9.0 0.65 0.65 0.79* 0.87

Statistics and Probability (Cat4) 8.3 0.57 0.57 0.65 0.71*

8

Expressions and Equations & Number System (Cat1) 11.4 0.80* 0.87 0.96 -

Functions (Cat2) 8.6 0.65 0.70* 0.86 -

Geometry & Statistics and Probability (Cat3) 14.0 0.77 0.65 0.81* -

Table 23: Correlations among Reporting Categories, Science

Grade Reporting Category ESS LS PS

5

ESS 0.62* 0.98 0.97

LS 0.63 0.67* 0.97

PS 0.60 0.63 0.62*

8

ESS 0.70* 0.97 0.99

LS 0.67 0.69* 0.98

PS 0.66 0.65 0.64* ESS: Earth and Space Science; LS: Life Science; PS: Physical Science

5.2 CONFIRMATORY FACTOR ANALYSIS FOR SPRING 2018 ELA AND MATHEMATICS

In the 2018-2019 school year, the WVGSA was administered as an adaptive test. Unlike the fixed-form tests administered in the 2017-2018 school year, the number of students who took each item was not always sufficient for conducting confirmatory factor analysis. Due to this restriction, the internal structural validity evidence supported by 2017-2018 WVGSA student data is summarized in this section. The 2017-2018 WVGSA and 2018-2019 WVGSA were constructed based on the



same content standards and very similar test blueprints. The internal structure of the two assessments is expected to be equivalent, with some degree of variability in model coefficients.

The WVGSA had test items designed to measure different standards and higher-level reporting categories. Test scores were reported as an overall performance measure. Additionally, scores on the various reporting categories were also provided as indices of strand-specific performance. The strand scores were reported in a fashion that aligned with the theoretical structure of the test derived from the test blueprint.

The results in this section are intended to provide evidence that the methods for reporting WVGSA strand scores align with the underlying structure of the test and provide evidence for the appropriateness of the selected IRT models. This section is based on a second-order confirmatory factor analysis, in which the first order factors load onto a common underlying factor. The first-order factors represent the dimensions of the test blueprint, and items load onto factors that they are intended to measure. The underlying structure of the ELA and mathematics tests was generally common across all grades, which is useful for comparing the results of our analyses across grades.

Although the test consisted of items targeting different standards, all items within a grade and subject were calibrated concurrently using the various IRT models described in this technical report. This implies the pivotal IRT assumption of local independence (Lord, 1980). Formally stated, this assumption posits that the probability of the outcome on item i depends only on the student’s ability and the characteristics of the item. Beyond that, the score of item i is independent of the outcome of all other items. From this assumption, the joint density (i.e., the likelihood) is viewed as the product of the individual densities. Thus, maximum likelihood estimation of person and item parameters in traditional IRT is derived based on this theory.

The measurement model and the score reporting method assume a single underlying factor, with separate factors representing each of the reporting categories. Consequently, it is important to collect validity evidence on the internal structure of the assessment to determine the rationality of conducting concurrent calibrations, as well as using these scoring and reporting methods.

5.2.1 Factor Analytic Methods

A series of confirmatory factor analyses (CFA) were conducted using the statistical program Mplus [version 7.31] (Muthén & Muthén, 2012) for each grade and subject assessment. Mplus is commonly used for collecting validity evidence on the internal structure of assessments. Weighted least squares means and variance adjusted (WLSMV) was employed as the estimation method because it is less sensitive to the size of the sample and the model and is also shown to perform well with categorical variables (Muthén, du Toit, & Spisic, 1997).

As previously stated, the method of reporting scores for West Virginia implies separate factors for each reporting category, connected by a single underlying factor. This model is subsequently referred to as the implied model. In factor analytic terms, this suggests that test items load onto separate first-order factors, with the first-order factors connected to a single underlying second-order factor. The use of the CFA in this section establishes some validity evidence for the degree to which the implied model is reasonable.



A chi-square difference test is often applied to assess model fit. However, it is sensitive to sample size, almost always rejecting the null hypothesis when the sample size is large. Therefore, instead of conducting a chi-square difference test, other goodness-of-fit indices were used to evaluate the implied model for the WVGSA.

If the internal structure of the test was strictly unidimensional, then the overall person ability measure, theta (𝜃𝜃), would be the single common factor, and the correlation matrix among test items would suggest no discernable pattern among factors. As such, there would be no empirical or logical basis to report scores for the separate performance categories. In factor analytic terms, a test structure that is strictly unidimensional implies a single-order factor model in which all test items load onto a single underlying factor. The development below expands the first-order model to a generalized second-order parameterization to show the relationship between the models.

The factor analysis models are based on the matrix 𝑺𝑺 of tetrachoric and polychoric sample correlations among the item scores (Olsson, 1979), and the matrix 𝑾𝑾 of asymptotic covariances among these sample correlations (Jöreskog, 1994) is employed as a weight matrix in a weighted least squares estimation approach (Browne, 1984; Muthén, 1984) to minimize the fit function:

𝑇𝑇𝑊𝑊𝑃𝑃𝑊𝑊 = vech(𝑺𝑺 − 𝚺𝚺�)′𝑾𝑾−𝟏𝟏vech(𝑺𝑺 − 𝚺𝚺�)

In the equation above, 𝚺𝚺� is the implied correlation matrix, given the estimated factor model, and the function vech vectorizes a symmetric matrix. That is, vech stacks each column of the matrix to form a vector. Note that the WLSMV approach (Muthén, du Toit, & Spisic, 1997) employs a weight matrix of asymptotic variances (i.e., the diagonal of the weight matrix) instead of the full asymptotic covariances.

We posit a first-order factor analysis in which all test items load onto a single common factor as the base model. The first-order model can be mathematically represented as

𝚺𝚺� = 𝚲𝚲𝚲𝚲𝚲𝚲′ + 𝚯𝚯,

where 𝚲𝚲 is the matrix of item factor loadings (with 𝚲𝚲′ representing its transpose), and 𝚯𝚯 is the uniqueness, or measurement error. The matrix 𝚲𝚲 is the correlation among the separate factors. For the base model, items are thought only to load onto a single underlying factor. Hence 𝚲𝚲 is a p x 1 vector, where p is the number of test items and 𝚲𝚲 is a scalar equal to 1. Therefore, it is possible to drop the matrix 𝚲𝚲 from the general notation. However, this notation is retained to more easily facilitate comparisons to the implied model, such that it can subsequently be viewed as a special case of the second-order factor analysis.

For the implied model, we posit a second-order factor analysis in which test items are coerced to load onto the reporting categories they are designed to target, and all reporting categories share a common underlying factor. The second-order factor analysis can be mathematically represented as:

𝚺𝚺� = 𝚲𝚲(𝚪𝚪𝚲𝚲𝚪𝚪′ + 𝚿𝚿)𝚲𝚲′ + 𝚯𝚯,

where Σ̂ is the implied correlation matrix among test items, 𝚲𝚲 is the p x k matrix of first-order factor loadings relating item scores to first-order factors, 𝚪𝚪is the k x 1 matrix of second-order factor loadings relating the first-order factors to the second-order factor with k denoting the number of



factors, 𝚲𝚲 is the correlation matrix of the second-order factors, and 𝚿𝚿 is the matrix of first-order factor residuals. All other notation is the same as the first-order model. Note that the second-order model expands the first-order model such that 𝚲𝚲 → 𝚪𝚪𝚲𝚲𝚪𝚪′ + 𝚿𝚿. As such, the first-order model is said to be nested within the second-order model.

There are three reporting categories for ELA and three to four categories for mathematics (see Table 18 and Table 19 for reporting category information). Therefore, the number of rows in 𝚪𝚪 (k) differs between subjects, but the general structure of the factor analysis is consistent across ELA and mathematics.

The second-order factor model can also be represented graphically, and a sample of the generalized approaches is provided in Figure 5. This figure is representative of the factor analyses performed for all grades and subjects, with the understanding that the number of items within each reporting category could vary across the grades.

The purpose of conducting a confirmatory factor analysis for the WVGSA was to provide evidence that each individual assessment in the WVGSA implied a second-order factor model: a single underlying second-order factor with the first-order factors defining each of the reporting categories.



Figure 5: Second-Order Factor Model, ELA

5.2.2 Results

Several goodness-of-fit statistics from each of the analyses are presented in Table 24 and Table 25. These tables present the summary results obtained from confirmatory factor analysis. Three goodness-of-fit indices were used to evaluate model fit of the item parameters to the way students responded to the items. The root mean square error of approximation (RMSEA) is referred to as a badness-of-fit index so that a value closer to 0 implies better fit and a value of 0 implies best fit. RMSEA below 0.05 is generally considered good fit, and RMSEA over 0.1 suggests poor fit (Browne & Cudeck, 1993). The Tucker–Lewis index (TLI) and the comparative fit index (CFI) are incremental goodness-of-fit indices. These indices compare the implied model to the baseline model in which no observed variables are correlated (i.e., there are no factors). Values greater than 0.9 are recognized as acceptable, and values over 0.95 are considered good fit (Hu & Bentler, 1999). As Hu and Bentler (1999) suggest, the selected cut-off values of the fit index should not be overgeneralized and should be interpreted with caution.



Based on the fit indices, the model showed good fit across content domains. For all tests, RMSEA was equal to or less than 0.05, and CFI and TLI were equal to or greater than 0.95 except for grade 4 Mathematics Form B, which had a TLI of 0.94.

Table 24: Goodness-of-Fit Second-Order CFA, Spring 2018 ELA

Grade Form df RMSEA CFI TLI Convergence

3 A* 943 0.04 0.96 0.96 Yes

B 986 0.04 0.95 0.95 Yes

4 A 986 0.03 0.97 0.97 Yes

B 986 0.03 0.96 0.96 Yes

5 A* 943 0.03 0.97 0.97 Yes

B* 943 0.03 0.96 0.96 Yes

6 A 986 0.03 0.97 0.97 Yes

B* 987 0.03 0.96 0.96 Yes

7 A 942 0.03 0.97 0.97 Yes

B 986 0.03 0.96 0.96 Yes

8 A* 987 0.03 0.98 0.98 Yes

B 986 0.04 0.97 0.97 Yes *For these tests, the second-order model was run by constraining the residual variance of a certain factor to zero due to non-significant negative residual variance.

Table 25: Goodness-of-Fit Second-Order CFA, Spring 2018 Mathematics

Grade Form df RMSEA CFI TLI Convergence

3 A* 817 0.03 0.98 0.97 Yes

B 816 0.03 0.98 0.98 Yes

4 A 816 0.03 0.97 0.97 Yes

B 816 0.05 0.95 0.94 Yes

5 A 816 0.03 0.97 0.97 Yes

B 816 0.03 0.97 0.97 Yes

6 A 816 0.03 0.97 0.97 Yes

B 816 0.02 0.98 0.98 Yes

7 A 815 0.02 0.99 0.99 Yes

B* 816 0.03 0.98 0.98 Yes

8 A 816 0.03 0.96 0.96 Yes

B 816 0.03 0.96 0.96 Yes *For these tests, the second-order model was run by constraining the residual variance of a certain factor to zero due to non-significant negative residual variance.

The second-order factor model converged for all tests. However, the residual variance for one factor fell slightly below the boundary of 0 for some of the tests (ELA grade 3 Form A, grade 5



Form A and B, grade 6 Form B, and grade 8 Form B; mathematics grade 3 Form A and grade 7 Form B) when using the Mplus software package. This negative residual variance may be related to the computational implementation of the optimization approach in Mplus, it may be a flag related to model misspecification, or it may be related to other causes (Van Driel, 1978; Chen, Bollen, Paxton, Curran & Kirby, 2001). The residual variance was constrained to 0 for these tests. This is equivalent to treating the parameter as fixed, which does not necessarily conform to our a priori hypothesis.

The estimated correlations between the reporting categories from the second-order factor model are presented in Table 26 and Table 27 for ELA and mathematics, respectively. In all cases, these correlations are very high. However, the results provide empirical evidence that there is some detectable dimensionality among reporting categories.

Table 26: Correlations Among ELA Factors, Spring 2018

Grade Reporting Category

Form A Form B

IT LT IT LT

3 LT 0.99 - 0.97 -

WL 0.76 0.75 0.78 0.76

4 LT 0.96 - 0.98 -

WL 0.73 0.76 0.80 0.80

5 LT 0.98 - 0.97 -

WL 0.78 0.79 0.78 0.75

6 LT 0.96 - 0.98 -

WL 0.78 0.80 0.84 0.82

7 LT 0.98 - 0.97 -

WL 0.78 0.77 0.79 0.77

8 LT 0.96 - 0.95 -

WL 0.77 0.80 0.82 0.80 IT: Reading Standards for Information/Nonfiction Text; LT: Reading Standards for Literature/Fiction Text; WL: Writing and Language Standards



Table 27: Correlations Among Mathematics Factors, Spring 2018


Form A Form B

MDG NBTF MDG NBTF

3 NBTF 1.00 - 0.98 -

OAT 0.97 0.97 0.95 0.96

4 NBTF 0.96 - 0.89 -

OAT 0.93 0.97 0.94 0.92

5 NBTF 0.94 - 0.97 -

OAT 0.86 0.86 0.94 0.92 MDG: Measurement, Data, and Geometry; NBTF: Numbers and Operations in Base Ten & Fractions; OAT: Operations and Algebraic Thinking


Form A Form B

EE GSP EE GSP

6 GSP 0.95 - 0.95 -

RPNS 0.96 0.95 0.96 0.95 EENS: Expressions and Equations & Number System; GSP: Geometry & Statistics and Probability; RPNS: Ratios and Proportional Relationships & Number System

Grade

Reporting Category

Form A Form B

EE G RPNS EE G RPNS

7

G 0.93 - - 0.94 - -

RPNS 0.98 0.93 - 0.97 0.97 -

SP 0.98 0.93 0.98 0.92 0.92 0.95 EE: Expressions and Equations; G: Geometry; RPNS: Ratios and Proportional Relationships & Number System; SP: Statistics and Probability


Form A Form B

EENS F EENS F

8 F 0.95 - 0.97 -

GSP 0.96 0.93 0.94 0.94 EENS: Expressions and Equations & Number System; F: Functions; GSP: Geometry & Statistics and Probability

5.2.3 Discussion

In all scenarios, the empirical results suggest that the implied model fits the data well. That is, these results indicate that reporting an overall score in addition to separate scores for the individual reporting categories is reasonable, as the inter-correlations among items suggest that there are detectable distinctions among reporting categories.

Clearly, the correlations among the separate factors are high, which is reasonable. This again provides support for the measurement model, given that the calibration of all items is performed concurrently. If the correlations among factors were very low, this could suggest that a different



IRT model would be needed (e.g., multidimensional IRT) or that the IRT calibration should be performed separately for items measuring different factors. The high correlations among the factors suggest that these alternative methods are unnecessary and that our current approach is preferable.

Overall, these results provide empirical evidence and justification for the use of our scoring and reporting methods. Additionally, the results provide justification for the IRT model employed currently.

5.3 LOCAL INDEPENDENCE, SPRING 2018

The validity of the application of IRT depends greatly on meeting the underlying assumptions of the models. One such assumption is local independence, which means that for a given proficiency estimate, the (marginal) likelihood is maximized, assuming that the probability of correct responses is the product of independent probabilities over all items (Chen & Thissen, 1997):

L(θ) = �� Pr(𝑧𝑧𝑖𝑖|θ)𝐼𝐼

𝑖𝑖=1

𝑓𝑓(θ)dθ

When local independence is not met, there are issues of multidimensionality that are unaccounted for in the modeling of the data (Bejar, 1980). In fact, Lord (1980) noted that “local independence follows automatically from unidimensionality” (as cited in Bejar, 1980, p.5). From a dimensionality perspective, there might be nuisance factors that are influencing relationships among certain items after accounting for the intended construct of interest. These nuisance factors can be influenced by several testing features, such as speediness, fatigue, item chaining, and item or response formats (Yen, 1993).

Yen’s Q3 statistic (Yen, 1984) was used to measure local independence, which was derived from the correlation between the performances of two items. Simply, the Q3 statistic is the correlation among IRT residuals and is computed using the following equations:

𝑑𝑑𝑖𝑖𝑗𝑗 = 𝑢𝑢𝑖𝑖𝑗𝑗 − 𝑇𝑇𝑗𝑗�𝜃𝜃�𝑖𝑖�.

where 𝑢𝑢𝑖𝑖𝑗𝑗 is the item score of the jth examinee for item i; 𝑇𝑇𝑖𝑖�𝜃𝜃�𝑗𝑗� is the estimated true score for item i of examinee j, which is defined as

𝑇𝑇𝑖𝑖�𝜃𝜃�𝑗𝑗� = �𝑦𝑦𝑖𝑖𝑗𝑗𝑃𝑃𝑖𝑖𝑗𝑗(𝑚𝑚

𝑗𝑗=1

𝜃𝜃�𝑗𝑗)

where 𝑦𝑦𝑖𝑖𝑗𝑗 is the weight for response category l; m is the number of response categories; and 𝑃𝑃𝑖𝑖𝑗𝑗(𝜃𝜃�𝑗𝑗) is the probability of response category l to item i by examinee j with the ability estimate 𝜃𝜃�𝑗𝑗.

The pairwise index of local dependence Q3 between item i and item i’ is

𝑄𝑄3𝑖𝑖𝑖𝑖′ = 𝑡𝑡(𝑑𝑑𝑖𝑖, 𝑑𝑑𝑖𝑖′),

where r refers to the Pearson product-moment correlation.



When there are n items, n(n-1)/2, Q3 statistics will be produced. The Q3 values are expected to be small. Table 28 and Table 29 present summaries of the distributions of Q3 statistics—minimum, 5th percentile, median, 95th percentile, and maximum values from each grade and subject. The results show that about 90% of the items between the 5th and 95th percentiles, for all grades and subjects, were smaller than a critical value of 0.2 for |𝑄𝑄3| (Chen & Thissen, 1997).

Table 28: ELA Q3 Statistic, Spring 2018

Grade Form Unconditional

Observed Correlation

Q3 Distribution Within Passage Q3**

Minimum 5th Percentile Median 95th

Percentile Maximum* Minimum Maximum

3 A 0.26 -0.21 -0.12 -0.01 0.03 0.87 -0.15 0.17

B 0.24 -0.22 -0.12 -0.01 0.03 0.88 -0.07 0.15

4 A 0.23 -0.20 -0.11 -0.02 0.02 0.60 -0.09 0.14

B 0.26 -0.16 -0.10 -0.02 0.02 0.61 -0.07 0.10

5 A 0.25 -0.19 -0.11 -0.01 0.03 0.57 -0.04 0.12

B 0.24 -0.18 -0.11 -0.01 0.03 0.58 -0.05 0.12

6 A 0.25 -0.21 -0.11 -0.02 0.02 0.53 -0.07 0.15

B 0.29 -0.17 -0.11 -0.02 0.02 0.57 -0.12 0.18

7 A 0.23 -0.24 -0.11 -0.01 0.02 0.63 -0.07 0.11

B 0.26 -0.22 -0.10 -0.02 0.02 0.66 -0.07 0.32

8 A 0.25 -0.23 -0.11 -0.02 0.02 0.79 -0.05 0.14

B 0.29 -0.17 -0.11 -0.02 0.02 0.79 -0.07 0.13 * Maximum Q3 values are from elaboration and organization dimensions of the writing prompt. ** Within Passage Q3 values are computed for each item pair within a passage.

Table 29: Mathematics Q3 Statistic, Spring 2018



Q3 Distribution

Minimum 5th Percentile Median 95th Percentile Maximum

3 A 0.39 -0.11 -0.06 -0.02 0.02 0.33

B 0.39 -0.13 -0.07 -0.02 0.02 0.21

4 A 0.40 -0.09 -0.06 -0.02 0.01 0.40

B 0.42 -0.13 -0.07 -0.02 0.02 0.71

5 A 0.38 -0.09 -0.06 -0.02 0.01 0.40

B 0.39 -0.13 -0.07 -0.02 0.02 0.71

6 A 0.38 -0.15 -0.06 -0.02 0.02 0.22

B 0.37 -0.15 -0.06 -0.02 0.02 0.22

7 A 0.37 -0.11 -0.06 -0.02 0.02 0.21

B 0.37 -0.12 -0.07 -0.02 0.02 0.31





Q3 Distribution

Minimum 5th Percentile Median 95th Percentile Maximum

8 A 0.35 -0.12 -0.07 -0.02 0.03 0.34

B 0.35 -0.13 -0.07 -0.02 0.02 0.51

5.4 CONVERGENT AND DISCRIMINANT VALIDITY

According to Standard 1.16 of The Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014), it is necessary to provide evidence of convergent and discriminant validity evidence. It is a part of validity evidence demonstrating that assessment scores are related as expected with criteria and other variables for all student groups. However, a second, independent test measuring the same constructs as ELA, mathematics, and science in West Virginia, which could easily permit for a cross-test set of correlations, was not available. Therefore, the correlations between subscores within and across ELA, mathematics, and science were examined alternatively. The a priori expectation is that subscores within the same subject (e.g., mathematics) will correlate more positively than subscore correlations across subjects (e.g., mathematics and ELA). These correlations are based on a small number of items; consequently, the observed score correlations will be smaller in magnitude as a result of the very large measurement error at the subscore level. For this reason, both the observed score and the disattenuated correlations are provided.

Observed and disattenuated subscore correlations were calculated both within and across subjects. The pattern is generally consistent with the a priori expectation that subscores within a test correlate higher than correlations between tests measuring a different construct with a few small notes on the writing dimensions.

Table 30 through Table 35 show the observed and disattenuated score correlations across ELA and mathematics subscores for grades 3–8 in which students took both subjects, including science for grades 5 and 8. Values in the lower triangle are observed correlations, and values in the upper triangle are disattenuated. Diagonals (highlighted in gray) are the reliability coefficient for ELA, mathematics, and science. The subscores are scale scores instead of raw scores, and the number of students used for the computation of the correlations are presented as well.



Table 30: Correlations across Subjects, Grade 3

Subject Number of Students Reporting Category

ELA Mathematics

Cat1 Cat2 Cat3 Cat1 Cat2 Cat3

ELA

Reading Informational Text (Cat1) 0.71* 0.70 0.72 0.67 0.68 0.67

18,560

Reading Literary Text (Cat2) 0.51 0.74* 0.71 0.65 0.66 0.65

Writing and Language (Cat3) 0.55 0.56 0.83* 0.77 0.77 0.76

Mathematics

Measurement, Data, and Geometry (Cat1) 0.49 0.49 0.61 0.76* 0.96 0.90

Numbers and Operations in Base Ten & Fractions (Cat2) 0.52 0.52 0.64 0.76 0.83* 0.93

Operations and Algebraic Thinking (Cat3) 0.51 0.51 0.63 0.71 0.77 0.82*

*Diagonal value represents the reliability coefficient of the reporting category. Observed correlations are below the diagonal, and disattenuated are above.



ELA Mathematics


ELA


19,205



Mathematics

Measurement, Data, and Geometry (Cat1) 0.49 0.53 0.56 0.72* 0.91 0.86

Numbers and Operations in Base Ten & Fractions (Cat2) 0.55 0.60 0.65 0.71 0.85* 0.96

Operations and Algebraic Thinking (Cat3) 0.53 0.58 0.62 0.64 0.78 0.77*






ELA Mathematics Science

Cat1 Cat2 Cat3 Cat1 Cat2 Cat3 Cat1 Cat2 Cat3

ELA

19,494

Reading Informational Text (Cat1) 0.73* 0.82 0.79 0.71 0.72 0.74 0.82 0.84 0.81

Reading Literary Text (Cat2) 0.60 0.74* 0.81 0.71 0.72 0.74 0.83 0.85 0.80

Writing and Language (Cat3) 0.60 0.63 0.81* 0.77 0.79 0.81 0.80 0.81 0.78

Mathematics

Measurement, Data, and Geometry (Cat1) 0.53 0.54 0.60 0.76* 0.96 0.90 0.84 0.81 0.83

Numbers and Operations in Base Ten & Fractions (Cat2) 0.57 0.57 0.65 0.77 0.85* 0.92 0.84 0.80 0.81

Operations and Algebraic Thinking (Cat3) 0.55 0.56 0.64 0.68 0.74 0.76* 0.83 0.81 0.82

Science

Earth and Space Science (Cat1) 0.55 0.56 0.57 0.58 0.60 0.56 0.62* 0.98 0.97

Life Science (Cat2) 0.59 0.60 0.60 0.58 0.60 0.58 0.63 0.67* 0.97

Physical Science (Cat3) 0.54 0.55 0.55 0.57 0.59 0.56 0.60 0.63 0.62*




ELA Mathematics


ELA


19,588



Mathematics

Expressions and Equations (Cat1) 0.55 0.53 0.63 0.77* 0.83 0.97

Geometry & Statistics and Probability (Cat2) 0.46 0.43 0.51 0.57 0.61* 0.86

Ratios and Proportional Relationships & Number System (Cat3) 0.58 0.55 0.66 0.77 0.61 0.82*






ELA Mathematics

Cat1 Cat2 Cat3 Cat1 Cat2 Cat3 Cat4

ELA

Reading Informational Text (Cat1) 0.75* 0.84 0.78 0.67 0.70 0.73 0.71

18,849

Reading Literary Text (Cat2) 0.61 0.71* 0.80 0.68 0.71 0.73 0.73

Writing and Language (Cat3) 0.61 0.61 0.81* 0.68 0.73 0.76 0.73

Mathematics

Expressions and Equations (Cat1) 0.50 0.49 0.53 0.74* 0.79 0.85 0.79

Geometry (Cat2) 0.51 0.50 0.55 0.57 0.70* 0.87 0.81

Ratios and Proportional Relationships & Number System (Cat3) 0.56 0.55 0.61 0.65 0.65 0.79* 0.87

Statistics and Probability (Cat4) 0.52 0.52 0.55 0.57 0.57 0.65 0.71*






ELA

18,952

Reading Informational Text (Cat1) 0.75* 0.80 0.79 0.74 0.70 0.75 0.80 0.80 0.80

Reading Literary Text (Cat2) 0.59 0.71* 0.74 0.68 0.65 0.68 0.78 0.78 0.77

Writing and Language (Cat3) 0.63 0.58 0.84* 0.78 0.71 0.77 0.77 0.76 0.77

Mathematics

Expressions and Equations & Number System (Cat1) 0.58 0.52 0.64 0.80* 0.84 0.94 0.82 0.81 0.82

Functions (Cat2) 0.51 0.46 0.54 0.63 0.70* 0.84 0.77 0.77 0.78

Geometry & Statistics and Probability (Cat3) 0.58 0.51 0.64 0.76 0.63 0.81* 0.83 0.83 0.83






Science

Earth and Space Science (Cat1) 0.58 0.55 0.59 0.61 0.54 0.62 0.70* 0.97 0.99

Life Science (Cat2) 0.57 0.55 0.58 0.60 0.53 0.61 0.67 0.69* 0.98

Physical Science (Cat3) 0.55 0.52 0.56 0.58 0.52 0.59 0.66 0.65 0.64*


Additionally, the correlation was computed among the overall scores for the three tested subjects: ELA, mathematics, and science. Correlations are presented in Table 36 and are relatively high, between 0.74 and 0.77.

Table 36: Correlations across Spring 2019 ELA, Mathematics and Science Scores

Grade N ELA with Mathematics Science with ELA Science with Mathematics

5 19,506 0.76 0.77 0.74

8 18,963 0.76 0.76 0.75



In fall 2018, optional ELA and mathematics interim assessments were administered. These tests were also online and adaptive. Examinees who took both optional interim assessments and the summative assessment in spring 2019 were identified for conducting the cross-test set of correlations. Table 37 and Table 38 present the correlations between summative spring 2019 and interim assessments from fall 2018. Values in the lower triangle are observed correlations, and values in the upper triangle are disattenuated. Diagonals (highlighted in gray) are the reliability coefficients derived from either the summative 2019 data or fall 2018 interim data. The means and standard deviations of scale score reported in the tables are based on data that includes examinees who took both the summative assessment and the interim assessment. Across all tests, observed correlations are medium to high, ranging from 0.58 to 0.74. Disattenuated correlations range from 0.65 to 0.86.

Table 37: Correlation between Summative and Interim Scores, ELA

Grade Test Scale Score Mean

Scale Score SD Summative Interim N

3 Summative 577.15 39.19 0.89 0.71

6,632 Interim 560.76 53.09 0.62 0.86

4 Summative 593.39 45.44 0.88 0.79

6,098 Interim 573.8 56.24 0.70 0.89

5 Summative 616.88 45.56 0.90 0.78

6,159 Interim 603.79 52.88 0.70 0.89

6 Summative 626.16 46.39 0.89 0.78

5,827 Interim 601.44 54.66 0.69 0.89

7 Summative 629.53 47.62 0.90 0.65

5,548 Interim 614.41 59.9 0.58 0.89

8 Summative 646.51 48.77 0.91 0.78

5,356 Interim 626.41 56.46 0.70 0.89

Table 38: Correlation between Summative and Interim Scores, Mathematics



3 Summative 426.72 32.83 0.92 0.82

7,718 Interim 408.57 35.69 0.74 0.88

4 Summative 451.97 41.06 0.92 0.84

7,584 Interim 435.55 45.78 0.74 0.84

5 Summative 477.47 47.23 0.92 0.81

8,005 Interim 459.87 54.94 0.71 0.83

6 Summative 492.41 55.74 0.89 0.84

7,206 Interim 473.52 62.39 0.72 0.82





7 Summative 523.27 64.05 0.87 0.86

7,427 Interim 512.28 63.57 0.74 0.85

8 Summative 561.85 75.94 0.89 0.84

7,965 Interim 538.51 80.06 0.73 0.84

5.5 RELATIONSHIP OF TEST SCORES TO EXTERNAL VARIABLES

The relationship of test scores to external variables, measuring the same or related constructs, is an important source of validity evidence. The WVGSA was first administered to students during the spring of 2018, replacing the Smarter Balanced Assessment Consortium (SBAC) assessments in ELA and mathematics. Ideally, we would correlate two different tests measuring a common construct administered within a similar time period. Here, we present correlations between two different tests measuring a common construct but measured one year apart. We expected the correlations to be high, suggesting that the WVGSA has a high relationship with an externally developed measure (the SBAC assessments), but the time gap between the two different assessments was also expected to cause correlations to be lower than if the two tests were measured within a similar testing window. Table 39 and Table 40 present correlations between SBAC scores from spring 2017 and WVGSA scores from spring 2018. Correlations are between 0.80 and 0.85, which is relatively high compared to industry standards.

Table 39: Correlations between Spring 2017 and Spring 2018 Scores, ELA

Grade: Spring 2017 → Spring 2018

N Spring 17 Marginal Reliability

Spring 18 Marginal Reliability Correlations Disattenuated

Correlations

3 → 4 16,832 0.93 0.89 0.80 0.88

4 → 5 16,961 0.92 0.90 0.80 0.88

5 → 6 16,183 0.93 0.91 0.82 0.89

6 → 7 16,411 0.92 0.90 0.82 0.90

7 → 8 16,512 0.92 0.91 0.82 0.90

Table 40: Correlations between Spring 2017 and Spring 2018 Scores, Mathematics

Grade: Spring 2017 → Spring 2018

N Spring 17 Marginal Reliability

Spring 18 Marginal Reliability Correlations Disattenuated

Correlations

3 → 4 16,634 0.94 0.89 0.83 0.91

4 → 5 16,768 0.94 0.90 0.82 0.89

5 → 6 16,001 0.93 0.88 0.82 0.91

6 → 7 16,232 0.93 0.89 0.85 0.93

7 → 8 16,342 0.92 0.89 0.82 0.91



The WVGSA for science was also first administered to grades 5 and 8 students during the spring of 2018, replacing the West Virginia Educational Standards Test. Students were in grade 4 and grade 7 in the previous year and did not take the science assessments. Therefore, correlations between spring 2017 and spring 2018 scores cannot be computed for science.

5.6 CLUSTER EFFECTS FOR SCIENCE

The science assessment is modeled with the Rasch testlet model (Wang & Wilson, 2005). Unlike the models for ELA and mathematics, the IRT model for science is a high-dimensional model, incorporating a nuisance dimension for each item cluster, in addition to an overall dimension representing the overall proficiency in science. Section 5.2 of Volume 1 presents a detailed description of the IRT model, which is illustrated using a directed graph in Figure 5 (Volume 1, Section 5.2). The psychometric approach for the science assessment is innovative and quite different from the traditional approach of ignoring local dependencies. The validity evidence on the internal structure presented in this section relates to the presence of cluster effects and how substantial they are.

Simulation studies conducted by Rijmen, Jiang and Turhan (2018) confirmed that both the item difficulty parameters and the cluster variances are recovered well for the Rasch testlet model under a variety of conditions. Cluster effects with a range of magnitudes were recovered well. The results obtained by Rijmen, Jiang and Turhan (2018) confirmed earlier findings reported in the literature (e.g., Bradlow, Wainer, & Wang, 1999) under conditions that were chosen to closely resemble the science assessment. For example, in one of the studies, the item location parameters and cluster variances used to simulate data were based on the results of a pilot study.

We examined the distribution of cluster variances obtained from 2018 IRT calibration. For elementary school, the estimated value of the cluster variances of all operational, scored items ranged from 0.11 to 4.46, with a median value of 0.53 and a mean value of 0.77. The median value is slightly smaller than the estimated variance parameter of the overall science dimension ( 60.0ˆ 2 =θσ ). For middle school, the estimated value of the cluster variances of all operational, scored items ranged from 0.06 to 5, with a median value of 0.56 and a mean value of 0.66. The median value is slightly larger than the estimated variance parameter of the overall science dimension ( 53.0ˆ 2 =θσ ). Figure 6 and Figure 7 present the histograms of the cluster variances expressed as the proportion of the total variance for all operational items for elementary and middle school, respectively. For both grade bands, a wide range of cluster variances is observed. These results indicate that, for both grades, cluster effects can be substantial and provide evidence for the appropriateness of a psychometric model that explicitly takes local dependencies among the assertions of an item cluster into account.



Figure 6: Cluster Variance Proportion for Science Operational Items in Elementary School

Figure 7: Cluster Variance Proportion for Science Operational Items in Middle School

0

2

4

6

8

10

12

0.1-0.2 0.2-0.3 0.3-0.4 0.4-0.5 0.5-0.6 0.6-0.7 0.7-0.8 0.8-0.9

0

5

10

15

20

25

0.1-0.2 0.2-0.3 0.3-0.4 0.4-0.5 0.5-0.6 0.6-0.7 0.7-0.8 0.8-0.9 0.9-1



6. FAIRNESS IN CONTENT

The principles of universal design of assessments provide guidelines for test design to minimize the impact of construct-irrelevant factors in assessing student achievement. Universal design removes barriers to provide access for the widest range of students possible. Seven principles of universal design are applied in the process of test development (Thompson, Johnstone, & Thurlow, 2002):

1. Inclusive assessment population

2. Precisely defined constructs

3. Accessible, non-biased items

4. Amenable to accommodations

5. Simple, clear, and intuitive instructions and procedures

6. Maximum readability and comprehensibility

7. Maximum legibility

Test development specialists have received extensive training on the principles of universal design and apply these principles in the development of all test materials. In the review process, adherence to the principles of universal design is verified by West Virginia educators and stakeholders.

6.1 STATISTICAL FAIRNESS IN ITEM STATISTICS

Due to the use of adaptive testing in spring 2019 ELA and mathematics, the number of West Virginia students who saw each item is relatively small. DIF analysis for the WVGSA for ELA and mathematics was not available due to the small sample size for each demographic group. However, DIF analysis was conducted with other states that field-tested the items for the initial item bank. A thorough content review was performed in those states. The details surrounding this review of items for bias is further described in Volume 1, Section 4.4, along with the DIF analysis process for WVGSA science.

7. SUMMARY

This report is intended to provide a collection of reliability and validity evidence to support appropriate inferences from the observed test scores. The overall results can be summarized as follows:

• Reliability. Various measures of reliability are provided at the aggregate and subgroup levels, showing that the reliability of all tests is in line with acceptable industry standards.

• Content validity. Evidence is provided to support the assertion that content coverage on each test was consistent with the test specifications of the blueprint across testing modes.



• Internal structural validity. Evidence is provided to support the selection of the measurement model, the tenability of model assumptions, and the reporting of an overall score and subscores at the reporting category levels.

• Relationship of test scores to external variables. Evidence of convergent and discriminant validity is provided to support the relationship between the test and other measures intended to assess similar constructs, as well as between the test and other measures intended to assess different constructs.



8. REFERENCES

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: Author.

Bejar, I. I. (1980). Biased assessment of program impact due to psychometric artifacts. Psychological Bulletin, 87(3), 513–524.

Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153–168.

Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3, 296–322.

Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.

Chen, F., Kenneth A. Bollen, P. Paxton, P. Curran, & J. Kirby. (2001). Improper solutions in structural equation models: Causes, consequences, and strategies. Sociological Methods & Research, 29, 468–508.

Chen, W. H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.

Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart, and Winston.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

Feldt, L. S., & Brennan, R. (1989). Reliability. In R. L. Linn (Ed.), Educational Measurement (3rd ed.) (pp. 105–146). New York: Macmillan.

Feldt, L. S., & Qualls, A. L. (1996). Bias in coefficient alpha arising from heterogeneity of test content. Applied Measurement in Education, 9, 277–286.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

Jöreskog, K. G. (1994). On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika, 59(3), 381–389.

Kuder, G. F., & Richardson, M. W. (1937). The theory of estimation of test reliability. Psychometrika, 2(3), 151–160.



Lee, G., & Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets. Applied Measurement in Education, 12, 237–255.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed.) (13–103). New York: Macmillan.

Muthén, B. O. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115–132.

Muthén, B. O., du Toit, S. H. C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Unpublished manuscript.

Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.

Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44, 443–460.

Qualls, A. L. (1995). Estimating the reliability of a test containing multiple item formats. Applied Measurement in Education, 8, 111–120.

Raju, N. S. (1977). A generalization of coefficient alpha. Psychometrika, 42, 549–565.

Rijmen, F., Jiang, T., & Turhan, A. (2018, April). An Item Response Theory Model for New Science Assessments. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY.

Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271–295.

Thompson, S. J., Johnstone, C. J., & Thurlow, M. L. (2002). Universal design applied to large scale assessments (Synthesis Report 44). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved October 2002 from http://education.umn.edu/NCEO/OnlinePubs/Synthesis44.html.

van Driel, Otto P. (1978). On various causes of improper solutions in maximum likelihood factor analysis. Psychometrika, 43, 225–243.

Wang, W. C., & Wilson, M. (2005). The Rasch testlet model. Applied Psychological Measurement, 29(2), 126–149.

Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.

Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187–213.

http://education.umn.edu/NCEO/OnlinePubs/Synthesis44.html



Yoon, B., & Young, M. J. (2000). Estimating the Reliability for Test Scores with Mixed Item Formats: Internal Consistency and Generalizability. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.

Appendix A

Student Demographics and Reliability Coefficients


Student Demographics and Reliability Coefficients A–1 West Virginia Department of Education

Table 1: Number of Students by Demographic Subgroup, ELA

Grade Overall Female Male African American

American Indian/ Native

Alaskan

Asian Hispanic Multi-Racial

Pacific Islander White Non-LEP LEP

3 18,559 9,060 9,499 714 8 110 380 750 11 16,575 18,402 157

4 19,215 9,479 9,736 783 22 122 381 770 7 17,130 19,051 164

5 19,555 9,482 10,073 732 19 129 411 753 12 17,499 19,407 148

6 19,602 9,486 10,116 790 11 137 389 745 9 17,519 19,480 122

7 18,838 9,124 9,714 825 22 120 371 610 12 16,878 18,731 107

8 19,054 9,095 9,959 835 16 137 385 529 8 17,144 18,954 100

*Note: Subgroup is not reported due to small sample size (sample size <30).

Table 2: Number of Students by Demographic Subgroup, Mathematics



Alaskan



3 18,560 9,058 9,502 714 8 111 377 751 11 16,577 18,407 153

4 19,205 9,473 9,732 783 22 122 377 769 7 17,125 19,046 159

5 19,557 9,482 10,075 734 19 130 400 754 12 17,508 19,420 137

6 19,588 9,479 10,109 794 11 137 379 747 9 17,509 19,475 113

7 18,849 9,129 9,720 825 22 120 366 611 12 16,893 18,746 103

8 19,056 9,088 9,968 836 16 137 376 529 8 17,154 18,965 91




Table 3: Reliability Coefficients by Demographic Subgroup, ELA



Alaskan



3 0.89 0.90 0.89 0.87 0.82 0.91 0.88 0.89 0.92 0.89 0.89 0.84

4 0.88 0.88 0.87 0.86 0.81 0.90 0.87 0.86 0.94 0.88 0.88 0.78

5 0.90 0.90 0.90 0.88 0.89 0.92 0.90 0.89 0.89 0.90 0.90 0.83

6 0.89 0.89 0.88 0.87 0.88 0.92 0.89 0.88 0.91 0.89 0.89 0.80

7 0.90 0.90 0.89 0.87 0.89 0.92 0.89 0.89 0.92 0.90 0.90 0.84

8 0.91 0.90 0.91 0.89 0.91 0.92 0.91 0.91 0.90 0.91 0.91 0.81


Table 4: Reliability Coefficients by Demographic Subgroup, Mathematics



Alaskan



3 0.92 0.92 0.93 0.90 0.95 0.94 0.92 0.93 0.95 0.92 0.92 0.91

4 0.92 0.91 0.92 0.90 0.86 0.93 0.91 0.91 0.94 0.92 0.92 0.89

5 0.92 0.91 0.92 0.90 0.91 0.94 0.91 0.91 0.83 0.92 0.92 0.90

6 0.89 0.90 0.89 0.85 0.92 0.94 0.88 0.88 0.90 0.89 0.89 0.83

7 0.87 0.87 0.87 0.79 0.87 0.94 0.84 0.85 0.92 0.87 0.87 0.79

8 0.89 0.89 0.89 0.85 0.87 0.94 0.88 0.88 0.90 0.89 0.89 0.82




Table 5: Scale Score Summary by Reporting Category, ELA Grade 3

Reporting Category Mean SD Min Max Reliability SEM

Reading Informational Text 566.36 65.10 420 750 0.71 31.55

Reading Literary Text 567.93 62.34 420 750 0.74 27.68

Writing and Language 577.16 45.39 420 750 0.83 18.02






























Table 11: Scale Score Summary by Reporting Category, Mathematics Grade 3


Measurement and Data and Geometry 419.70 41.13 308 548 0.76 18.54

Numbers and Operations—Base Ten and Fractions 426.70 35.93 308 548 0.83 13.54

Operations and Algebraic Thinking 421.17 44.34 308 548 0.82 17.02















Expressions and Equations 485.97 72.03 330 720 0.77 30.64

Geometry and Statistics and Probability 479.34 78.43 330 720 0.61 46.60

Ratios and Proportional Relationships and Number System 490.40 61.47 330 720 0.82 23.20



Expressions and Equations 497.68 97.01 348 750 0.74 44.51

Geometry 510.75 79.43 348 750 0.70 39.99

Ratios and Proportional Relationships and Number System 521.02 78.57 348 750 0.79 32.50

Statistics and Probability 517.18 92.15 348 750 0.71 47.14



Expressions and Equations and Number System 554.31 91.15 350 830 0.80 36.51

Functions 539.41 99.02 350 830 0.70 51.56

Geometry and Statistics and Probability 551.02 84.16 350 830 0.81 33.48



Table 17: Scale Score Summary by Reporting Category, Science Grade 5


Physical Science 549.55 16.51 500 600 0.62 10.06

Earth and Space Science 549.24 17.08 500 600 0.62 10.40

Life Science 549.17 17.67 500 600 0.67 9.96

Table 18: Scale Score Summary by Reporting Category, Science Grade 8


Physical Science 847.13 16.28 800 900 0.64 9.71

Earth and Space Science 847.25 18.76 800 900 0.70 10.18

Life Science 848.54 18.24 800 900 0.69 10.08

Appendix B

Conditional Standard Error of Measurement


Conditional Standard Error of Measurement B–1 West Virginia Department of Education

Table 1: CSEM at Each Scale Score, ELA Grade 3

ELA Grade 3

Scale Score

Achievement Level CSEM

420 1 61.13 423 1 62.17 425 1 62.5 427 1 60.92 428 1 52.13 429 1 50.54 432 1 55.76 433 1 54.42 434 1 54.77 435 1 57.12 438 1 55.49 439 1 51.76 440 1 47.29 442 1 52.11 443 1 45.77 444 1 47.94 446 1 45.55 448 1 54.86 450 1 44.09 452 1 47.27 453 1 39.82 454 1 42.33 455 1 38.98 456 1 38.42 457 1 41.4 458 1 38.1 459 1 40.09 460 1 37.31 461 1 34.06 462 1 36.69 463 1 42.19 464 1 32.93 465 1 34.92 466 1 31.45 467 1 34.72 468 1 33.17 469 1 33.27 470 1 34.72 471 1 33.45 472 1 31.88 473 1 34.42 474 1 31



ELA Grade 3

Scale Score


475 1 30.56 476 1 28.34 477 1 31.16 478 1 29.03 479 1 30.16 480 1 31.24 481 1 28.74 482 1 28.09 483 1 27.13 484 1 26.1 485 1 26.84 486 1 26.28 487 1 26.16 488 1 25.61 489 1 25.41 490 1 24.38 491 1 24.6 492 1 24 493 1 23.65 494 1 22.92 495 1 23.01 496 1 23.71 497 1 21.89 498 1 21.64 499 1 21.61 500 1 21.79 501 1 20.94 502 1 20.39 503 1 20.11 504 1 19.73 505 1 19.41 506 1 19.23 507 1 19.23 508 1 18.89 509 1 18.6 510 1 18.4 511 1 18.05 512 1 17.92 513 1 17.48 514 1 17.15 515 1 17.12 516 1 16.86 517 1 16.73 518 1 16.28



ELA Grade 3

Scale Score


519 1 16.3 520 1 16.16 521 1 15.75 522 1 15.67 523 1 15.61 524 1 15.3 525 1 15.15 526 1 14.96 527 1 14.88 528 1 14.63 529 1 14.5 530 1 14.25 531 1 14.3 532 1 14.11 533 1 13.72 534 1 13.77 535 1 13.53 536 1 13.52 537 1 13.23 538 1 13.32 539 1 13.04 540 1 13 541 1 12.95 542 1 12.74 543 1 12.79 544 1 12.63 545 1 12.69 546 1 12.52 547 1 12.3 548 1 12.37 549 1 12.26 550 2 12.16 551 2 12.06 552 2 12.12 553 2 12.1 554 2 11.91 555 2 11.93 556 2 11.75 557 2 11.8 558 2 11.7 559 2 11.67 560 2 11.59 561 2 11.66 562 2 11.53



ELA Grade 3

Scale Score


563 2 11.58 564 2 11.63 565 2 11.54 566 2 11.47 567 2 11.41 568 2 11.37 569 2 11.4 570 2 11.32 571 2 11.43 572 2 11.35 573 2 11.19 574 2 11.3 575 2 11.23 576 2 11.28 577 2 11.17 578 2 11.16 579 2 11.15 580 2 11.07 581 2 11.06 582 2 11.07 583 2 11.04 584 2 10.98 585 2 10.97 586 3 10.93 587 3 10.92 588 3 10.89 589 3 10.82 590 3 10.82 591 3 10.84 592 3 10.76 593 3 10.68 594 3 10.65 595 3 10.66 596 3 10.5 597 3 10.58 598 3 10.56 599 3 10.57 600 3 10.48 601 3 10.5 602 3 10.43 603 3 10.47 604 3 10.46 605 3 10.41 606 3 10.28



ELA Grade 3

Scale Score


607 3 10.27 608 3 10.25 609 3 10.3 610 3 10.25 611 3 10.36 612 3 10.14 613 3 10.16 614 3 10.19 615 3 10.16 616 4 10.25 617 4 10.29 618 4 10.15 619 4 10.21 620 4 10.27 621 4 10.4 622 4 10.29 623 4 10.4 624 4 10.3 625 4 10.34 626 4 10.32 627 4 10.4 628 4 10.42 629 4 10.55 630 4 10.48 631 4 10.54 632 4 10.55 633 4 10.62 634 4 10.62 635 4 10.64 636 4 10.76 637 4 10.81 638 4 10.82 639 4 10.86 640 4 10.91 641 4 11.05 642 4 11.04 643 4 11 644 4 11.12 645 4 11.18 646 4 11.22 647 4 11.43 648 4 11.37 649 4 11.41 650 4 11.43



ELA Grade 3

Scale Score


651 4 11.49 652 4 11.66 653 4 11.72 654 4 11.82 655 4 11.78 656 4 11.67 657 4 11.94 658 4 12.03 659 4 11.67 660 4 12.39 661 4 12.23 662 4 12.34 663 4 12.36 664 4 11.88 665 4 12.61 666 4 12.17 667 4 12.89 668 4 12.53 669 4 12.85 670 4 12.82 671 4 13 672 4 13.22 673 4 13.12 674 4 13.53 675 4 13.4 676 4 13.74 677 4 13.79 679 4 13.38 680 4 13.89 681 4 13.85 682 4 14.39 683 4 14.55 684 4 14.87 685 4 14.98 686 4 14.5 688 4 14.84 689 4 15.8 690 4 15.27 691 4 15.9 692 4 15.24 693 4 13.05 704 4 17.75 705 4 17.64 709 4 17.34



ELA Grade 3

Scale Score


713 4 20.94 724 4 24.59 725 4 24.9


ELA Grade 4

Scale Score


431 1 62.51 434 1 59.32 435 1 66.75 436 1 57.49 437 1 54.53 438 1 63.18 439 1 52.86 440 1 57.1 441 1 55.44 443 1 58.51 444 1 47.77 445 1 58.32 446 1 56.63 447 1 56.44 448 1 52.58 449 1 57.84 450 1 51.32 451 1 51.22 452 1 49.89 453 1 49.91 454 1 51.11 455 1 51.05 456 1 51.29 457 1 45.61 458 1 48.02 459 1 47.4 460 1 48.61 461 1 45.28 462 1 48.25 463 1 45.79 464 1 41.89 465 1 47.41 466 1 43.47



ELA Grade 4

Scale Score


467 1 44.1 468 1 41.37 469 1 43.36 470 1 42.73 471 1 40.83 472 1 40.53 473 1 43.42 474 1 42.76 475 1 40.2 476 1 39.51 477 1 40.03 478 1 38.58 479 1 39.26 480 1 38.77 481 1 38.03 482 1 35.48 483 1 37.04 484 1 36.83 485 1 35.57 486 1 33.44 487 1 34.09 488 1 34.66 489 1 34.29 490 1 34.42 491 1 32.89 492 1 32.22 493 1 31.45 494 1 31.92 495 1 31.61 496 1 31.14 497 1 31.62 498 1 31.45 499 1 29.76 500 1 29.1 501 1 29.66 502 1 28.81 503 1 28.53 504 1 28.29 505 1 27.53 506 1 27.45 507 1 27.75 508 1 26.47 509 1 26.48 510 1 26.4



ELA Grade 4

Scale Score


511 1 25.49 512 1 25.84 513 1 24.88 514 1 24.62 515 1 24.82 516 1 23.99 517 1 23.75 518 1 23.18 519 1 23.11 520 1 23.2 521 1 22.86 522 1 22.22 523 1 22.55 524 1 22 525 1 21.89 526 1 21.68 527 1 20.91 528 1 21.34 529 1 20.88 530 1 20.82 531 1 20.18 532 1 19.92 533 1 19.52 534 1 19.58 535 1 18.97 536 1 18.72 537 1 18.65 538 1 18.46 539 1 18.4 540 1 18.32 541 1 18.18 542 1 18.1 543 1 17.56 544 1 17.23 545 1 17.13 546 1 17.11 547 1 16.71 548 1 16.53 549 1 16.21 550 1 16.08 551 1 15.99 552 1 15.71 553 1 15.98 554 1 15.8



ELA Grade 4

Scale Score


555 1 15.62 556 1 15.16 557 1 15.35 558 1 15.13 559 1 14.87 560 1 14.96 561 1 14.74 562 1 14.48 563 2 14.32 564 2 14.14 565 2 14.01 566 2 14.06 567 2 13.89 568 2 13.81 569 2 13.65 570 2 13.76 571 2 13.43 572 2 13.44 573 2 13.56 574 2 13.25 575 2 13.16 576 2 12.97 577 2 13.08 578 2 12.99 579 2 12.9 580 2 12.9 581 2 12.88 582 2 12.75 583 2 12.76 584 2 12.76 585 2 12.57 586 2 12.48 587 2 12.33 588 2 12.59 589 2 12.53 590 2 12.42 591 2 12.34 592 2 12.34 593 2 12.32 594 2 12.34 595 2 12.21 596 2 12.2 597 2 12.24 598 2 12.18



ELA Grade 4

Scale Score


599 3 12.19 600 3 12.18 601 3 12.04 602 3 12.13 603 3 12.09 604 3 12.18 605 3 12.11 606 3 12.13 607 3 12.09 608 3 12.02 609 3 12.01 610 3 12.05 611 3 12.08 612 3 12.04 613 3 12.15 614 3 12.04 615 3 12.13 616 3 12.18 617 3 12.2 618 3 12.15 619 3 12.18 620 3 12.16 621 3 12.13 622 3 12.2 623 3 12.17 624 3 12.24 625 3 12.31 626 3 12.32 627 3 12.24 628 3 12.35 629 4 12.42 630 4 12.42 631 4 12.48 632 4 12.44 633 4 12.51 634 4 12.54 635 4 12.5 636 4 12.66 637 4 12.59 638 4 12.73 639 4 12.72 640 4 12.69 641 4 12.79 642 4 12.92



ELA Grade 4

Scale Score


643 4 12.96 644 4 12.93 645 4 13.07 646 4 13.12 647 4 13.05 648 4 13.14 649 4 13.24 650 4 13.26 651 4 13.29 652 4 13.42 653 4 13.5 654 4 13.56 655 4 13.6 656 4 13.76 657 4 13.77 658 4 13.79 659 4 13.78 660 4 13.91 661 4 13.87 662 4 14.15 663 4 14.21 664 4 14.24 665 4 14.28 666 4 14.3 667 4 14.42 668 4 14.43 669 4 14.6 670 4 14.6 671 4 14.7 672 4 14.64 673 4 14.84 674 4 14.93 675 4 15.04 676 4 15.07 677 4 15.04 678 4 15.16 679 4 15.44 680 4 15.24 681 4 15.11 682 4 15.55 683 4 15.58 684 4 15.48 685 4 15.65 686 4 15.62



ELA Grade 4

Scale Score


687 4 15.9 688 4 15.88 689 4 16.26 690 4 16.38 691 4 16.19 692 4 16.18 693 4 16.37 694 4 15.86 695 4 16.17 696 4 15.9 697 4 16.66 698 4 16.65 699 4 16.43 700 4 16.53 701 4 16.47 702 4 17.02 703 4 16.66 704 4 17.35 705 4 17.33 706 4 17.01 707 4 17.25 708 4 16.63 709 4 17.35 710 4 16.98 711 4 17.36 712 4 17.26 713 4 16.84 714 4 17.22 715 4 17.11 716 4 17.24 717 4 16.94 718 4 18.66 720 4 17.59 721 4 17.64 723 4 17.96 725 4 17.9 726 4 18.88 727 4 17.97 729 4 17.79 730 4 17.23 731 4 18.32 733 4 21.23 742 4 19.89 748 4 20.35



ELA Grade 4

Scale Score


782 4 32.05 787 4 28.35


ELA Grade 5

Scale Score


450 1 55.09 453 1 54.81 457 1 51.48 458 1 49.85 460 1 47.56 463 1 49.15 466 1 46.79 467 1 44.79 468 1 42.74 469 1 43.22 470 1 42.87 471 1 43.77 472 1 43.77 473 1 39.14 474 1 43.87 475 1 43.23 476 1 42.79 477 1 40.11 478 1 36.69 479 1 40.13 480 1 41.71 481 1 39.37 482 1 37.5 483 1 39.69 484 1 38.44 485 1 38.18 486 1 38.03 487 1 36.25 488 1 34.28 489 1 34.77 490 1 34.95 491 1 35.51 492 1 34.21 493 1 34.2



ELA Grade 5

Scale Score


494 1 32.38 495 1 32.17 496 1 33.32 497 1 33.75 498 1 32.51 499 1 32.25 500 1 31.09 501 1 31.07 502 1 30.72 503 1 30.89 504 1 30.67 505 1 28.38 506 1 29.58 507 1 28.82 508 1 28.04 509 1 27.97 510 1 28.07 511 1 26.83 512 1 28.51 513 1 26.44 514 1 26.13 515 1 25.86 516 1 25.74 517 1 24.65 518 1 25.21 519 1 24.32 520 1 24.75 521 1 23.97 522 1 24.66 523 1 23.95 524 1 23 525 1 23.6 526 1 22.08 527 1 22.48 528 1 21.8 529 1 22.31 530 1 22.13 531 1 21.64 532 1 21.4 533 1 20.95 534 1 20.7 535 1 20.71 536 1 20.84 537 1 20.2



ELA Grade 5

Scale Score


538 1 19.85 539 1 19.8 540 1 19.44 541 1 19.33 542 1 19.21 543 1 19.01 544 1 19 545 1 18.26 546 1 18.13 547 1 18.27 548 1 17.85 549 1 17.67 550 1 17.3 551 1 17.6 552 1 17.13 553 1 17.04 554 1 16.69 555 1 16.42 556 1 16.38 557 1 16.14 558 1 16.07 559 1 16.19 560 1 15.96 561 1 15.67 562 1 15.78 563 1 15.37 564 1 15.16 565 1 15.15 566 1 14.94 567 1 14.86 568 1 14.79 569 1 14.66 570 1 14.46 571 1 14.38 572 1 14.25 573 1 14.13 574 1 14.29 575 1 13.86 576 1 14.01 577 1 13.83 578 1 13.77 579 1 13.71 580 1 13.53 581 1 13.47



ELA Grade 5

Scale Score


582 1 13.56 583 1 13.45 584 1 13.29 585 1 13.26 586 1 13.3 587 1 13.12 588 2 12.98 589 2 13.01 590 2 12.9 591 2 12.81 592 2 12.91 593 2 12.88 594 2 12.77 595 2 12.67 596 2 12.66 597 2 12.67 598 2 12.72 599 2 12.59 600 2 12.51 601 2 12.52 602 2 12.47 603 2 12.56 604 2 12.53 605 2 12.47 606 2 12.45 607 2 12.44 608 2 12.34 609 2 12.36 610 2 12.33 611 2 12.31 612 2 12.24 613 2 12.29 614 2 12.23 615 2 12.47 616 2 12.35 617 2 12.37 618 2 12.26 619 2 12.36 620 2 12.27 621 2 12.39 622 3 12.29 623 3 12.35 624 3 12.35 625 3 12.43



ELA Grade 5

Scale Score


626 3 12.38 627 3 12.48 628 3 12.45 629 3 12.32 630 3 12.38 631 3 12.49 632 3 12.52 633 3 12.31 634 3 12.48 635 3 12.43 636 3 12.55 637 3 12.73 638 3 12.49 639 3 12.58 640 3 12.66 641 3 12.66 642 3 12.74 643 3 12.72 644 3 12.83 645 3 12.84 646 3 12.86 647 3 12.81 648 3 12.87 649 3 12.95 650 3 12.94 651 3 12.87 652 3 12.95 653 3 13 654 3 13 655 4 13.09 656 4 13.04 657 4 13.13 658 4 13.19 659 4 13.24 660 4 13.26 661 4 13.16 662 4 13.22 663 4 13.16 664 4 13.17 665 4 13.19 666 4 13.46 667 4 13.34 668 4 13.34 669 4 13.43



ELA Grade 5

Scale Score


670 4 13.48 671 4 13.35 672 4 13.52 673 4 13.44 674 4 13.45 675 4 13.55 676 4 13.64 677 4 13.57 678 4 13.57 679 4 13.74 680 4 13.69 681 4 13.77 682 4 13.68 683 4 13.96 684 4 13.79 685 4 13.93 686 4 13.78 687 4 13.84 688 4 14.03 689 4 14.15 690 4 14.21 691 4 14.15 692 4 14.13 693 4 14.24 694 4 14.29 695 4 14.46 696 4 14.23 697 4 14.35 698 4 14.33 699 4 14.68 700 4 14.75 701 4 14.78 702 4 14.53 703 4 14.8 704 4 14.96 705 4 14.95 706 4 14.9 707 4 14.87 708 4 14.97 709 4 15.04 710 4 15.02 711 4 14.83 712 4 15.29 713 4 15.48



ELA Grade 5

Scale Score


714 4 15.63 715 4 15.57 716 4 15.81 717 4 15.41 718 4 16.24 719 4 16.01 720 4 15.95 721 4 16.16 722 4 15.65 723 4 15.78 724 4 16.5 725 4 16.11 726 4 16.33 727 4 16.72 728 4 16.28 729 4 16.52 730 4 16.03 731 4 16.54 732 4 17 733 4 16.95 734 4 16.47 735 4 17.35 736 4 16.59 737 4 18.27 738 4 18.25 739 4 17.75 740 4 17.91 741 4 17.69 743 4 18.27 744 4 18.64 748 4 17.56 750 4 18.55 751 4 19.82 752 4 18.43 753 4 19.24 755 4 19.39 756 4 21.43 769 4 23.61 772 4 24.75 775 4 20.71 792 4 36.44 799 4 35.46




ELA Grade 6

Scale Score


464 1 56.24 465 1 58.09 466 1 52.49 468 1 48.7 469 1 51.47 470 1 53.18 471 1 49.41 472 1 52.75 473 1 47.11 474 1 48.43 475 1 49.02 476 1 47.37 477 1 47.36 478 1 49.21 479 1 49.94 481 1 51.29 482 1 51.09 484 1 44.31 485 1 41.86 486 1 47.91 487 1 43.96 488 1 41.84 489 1 46.02 490 1 43.25 491 1 43.68 492 1 42.15 493 1 39.63 494 1 41.39 495 1 39.89 496 1 37.91 497 1 37.92 498 1 39.77 499 1 39.65 500 1 39 501 1 38.69 502 1 36.37 503 1 35.75 504 1 37.18 505 1 35.41 506 1 35.29 507 1 34.46 508 1 33.82



ELA Grade 6

Scale Score


509 1 34.2 510 1 34.02 511 1 34.66 512 1 33.45 513 1 31.25 514 1 32.16 515 1 30.82 516 1 33.48 517 1 30.96 518 1 30.95 519 1 30.02 520 1 29.59 521 1 29.94 522 1 29.71 523 1 28.74 524 1 29.15 525 1 28.47 526 1 27.87 527 1 27.44 528 1 26.65 529 1 27.68 530 1 26.2 531 1 26.44 532 1 26.11 533 1 26.66 534 1 26.52 535 1 24.68 536 1 25.33 537 1 23.83 538 1 25.03 539 1 25.13 540 1 24.33 541 1 24.97 542 1 23.84 543 1 23.82 544 1 22.85 545 1 23.67 546 1 22.96 547 1 22.53 548 1 22.49 549 1 22.85 550 1 21.99 551 1 22.17 552 1 22.36



ELA Grade 6

Scale Score


553 1 22.13 554 1 22.05 555 1 21.51 556 1 21.17 557 1 21.28 558 1 21.33 559 1 20.93 560 1 20.79 561 1 20.63 562 1 20.76 563 1 20.32 564 1 20.25 565 1 20.14 566 1 20.07 567 1 19.7 568 1 19.91 569 1 19.49 570 1 19.48 571 1 19.19 572 1 19.08 573 1 18.71 574 1 18.6 575 1 18.6 576 1 18.47 577 1 18.27 578 1 18.11 579 1 18.45 580 1 17.79 581 1 17.91 582 1 17.78 583 1 17.6 584 1 17.22 585 1 17.39 586 1 17.25 587 1 17.01 588 1 16.94 589 1 16.81 590 1 16.62 591 1 16.43 592 1 16.39 593 1 16.22 594 1 16.21 595 1 16.04 596 1 15.86



ELA Grade 6

Scale Score


597 2 15.83 598 2 15.75 599 2 15.53 600 2 15.46 601 2 15.31 602 2 15.22 603 2 15.09 604 2 15.01 605 2 14.94 606 2 14.76 607 2 14.67 608 2 14.47 609 2 14.3 610 2 14.4 611 2 14.28 612 2 14.1 613 2 14.03 614 2 13.98 615 2 13.92 616 2 13.88 617 2 13.75 618 2 13.67 619 2 13.66 620 2 13.53 621 2 13.45 622 2 13.45 623 2 13.4 624 2 13.25 625 2 13.17 626 2 13.24 627 2 13.11 628 2 13.12 629 2 13.12 630 2 12.99 631 2 12.96 632 2 12.97 633 2 12.96 634 2 12.87 635 2 12.84 636 2 12.81 637 2 12.79 638 2 12.81 639 3 12.74 640 3 12.8



ELA Grade 6

Scale Score


641 3 12.61 642 3 12.58 643 3 12.6 644 3 12.68 645 3 12.63 646 3 12.56 647 3 12.6 648 3 12.63 649 3 12.67 650 3 12.58 651 3 12.51 652 3 12.68 653 3 12.58 654 3 12.51 655 3 12.56 656 3 12.6 657 3 12.57 658 3 12.57 659 3 12.63 660 3 12.68 661 3 12.62 662 3 12.62 663 3 12.56 664 3 12.58 665 3 12.53 666 3 12.62 667 3 12.66 668 3 12.66 669 3 12.57 670 3 12.72 671 3 12.61 672 3 12.51 673 3 12.59 674 3 12.45 675 3 12.58 676 3 12.63 677 3 12.6 678 3 12.57 679 3 12.59 680 4 12.53 681 4 12.59 682 4 12.61 683 4 12.64 684 4 12.7



ELA Grade 6

Scale Score


685 4 12.57 686 4 12.67 687 4 12.68 688 4 12.71 689 4 12.75 690 4 12.79 691 4 12.9 692 4 12.93 693 4 12.95 694 4 13 695 4 12.93 696 4 13.04 697 4 13.19 698 4 13.15 699 4 13.15 700 4 13.29 701 4 13.42 702 4 13.46 703 4 13.47 704 4 13.42 705 4 13.54 706 4 13.52 707 4 13.47 708 4 13.84 709 4 13.71 710 4 13.7 711 4 13.94 712 4 13.89 713 4 13.84 714 4 13.87 715 4 14.01 716 4 14.09 717 4 14.14 718 4 14.18 719 4 14.24 720 4 14.26 721 4 14.31 722 4 14.37 723 4 14.4 724 4 14.47 725 4 14.31 726 4 14.5 727 4 14.56 728 4 14.39



ELA Grade 6

Scale Score


729 4 14.84 730 4 14.93 731 4 14.6 732 4 14.57 733 4 14.64 734 4 14.9 735 4 14.9 736 4 15.01 737 4 14.98 738 4 14.81 739 4 14.18 740 4 14.84 741 4 15.05 742 4 14.49 743 4 15.38 744 4 14.92 745 4 15.23 746 4 15.09 747 4 15.35 748 4 15.6 749 4 15.62 750 4 15.74 752 4 16.26 754 4 16.04 756 4 15.64 758 4 17.03 759 4 17.02 760 4 16.57 761 4 17 762 4 16.2 763 4 17.65 764 4 18.33 765 4 17.17 769 4 18.99 770 4 17.87 771 4 17.86 772 4 18.53 773 4 18.11 780 4 19.4 792 4 24.58 798 4 23.99 819 4 32.71




ELA Grade 7

Scale Score


470 1 56.03 471 1 52.73 472 1 49.18 473 1 50.33 474 1 50.55 476 1 47.25 477 1 44.45 478 1 45.67 479 1 45.87 480 1 46.14 481 1 49.94 482 1 49.42 483 1 46.1 484 1 45.55 485 1 47.96 486 1 43.11 487 1 45.15 488 1 41.61 489 1 43.6 490 1 45.64 491 1 44.66 492 1 40.99 493 1 42.19 494 1 40.44 495 1 44.21 497 1 43.68 498 1 41.25 499 1 38.7 500 1 36.88 501 1 38.96 502 1 38.58 503 1 38.59 504 1 37.69 505 1 41.51 506 1 35.51 507 1 34.46 508 1 34.92 509 1 35.46 510 1 35.08 511 1 35.72 512 1 33.86 513 1 34.52



ELA Grade 7

Scale Score


514 1 33.54 515 1 33.12 516 1 30.51 517 1 30.17 518 1 31.37 519 1 30.78 520 1 31.86 521 1 30.71 522 1 29.85 523 1 30.48 524 1 28.63 525 1 29.03 526 1 28.12 527 1 28.15 528 1 27.43 529 1 28.18 530 1 27.53 531 1 27.15 532 1 27.76 533 1 27.15 534 1 26.21 535 1 26.08 536 1 26.15 537 1 25.24 538 1 25.66 539 1 25.04 540 1 24.89 541 1 24.25 542 1 24.35 543 1 23.65 544 1 23.62 545 1 23.74 546 1 23.15 547 1 22.97 548 1 22.84 549 1 22.78 550 1 22.65 551 1 22.23 552 1 21.47 553 1 21.56 554 1 21.15 555 1 21.5 556 1 20.67 557 1 20.88



ELA Grade 7

Scale Score


558 1 20.42 559 1 20.12 560 1 20.12 561 1 20.19 562 1 19.32 563 1 19.55 564 1 19.28 565 1 19.06 566 1 18.63 567 1 18.82 568 1 18.15 569 1 18.17 570 1 17.7 571 1 18.01 572 1 17.89 573 1 17.72 574 1 17.12 575 1 16.94 576 1 16.56 577 1 16.98 578 1 16.87 579 1 16.64 580 1 16.28 581 1 16.33 582 1 16.25 583 1 16.21 584 1 16.06 585 1 15.66 586 1 15.98 587 1 15.47 588 1 15.26 589 1 15.3 590 1 14.97 591 1 15.19 592 1 14.82 593 1 14.94 594 1 14.86 595 1 14.64 596 1 14.46 597 1 14.33 598 1 14.19 599 1 14.23 600 1 14.07 601 1 14.04



ELA Grade 7

Scale Score


602 2 14.23 603 2 13.97 604 2 13.82 605 2 13.72 606 2 14.02 607 2 13.8 608 2 13.57 609 2 13.65 610 2 13.69 611 2 13.6 612 2 13.59 613 2 13.51 614 2 13.32 615 2 13.38 616 2 13.43 617 2 13.35 618 2 13.36 619 2 13.31 620 2 13.21 621 2 13.19 622 2 13.21 623 2 13.16 624 2 13.18 625 2 13.06 626 2 13.01 627 2 13.08 628 2 13.01 629 2 12.97 630 2 13 631 2 12.93 632 2 12.77 633 2 12.94 634 2 12.92 635 2 12.84 636 2 12.75 637 2 12.78 638 2 12.81 639 2 12.82 640 2 12.79 641 2 12.72 642 2 12.71 643 2 12.64 644 3 12.65 645 3 12.61



ELA Grade 7

Scale Score


646 3 12.68 647 3 12.64 648 3 12.53 649 3 12.61 650 3 12.6 651 3 12.52 652 3 12.55 653 3 12.57 654 3 12.57 655 3 12.52 656 3 12.54 657 3 12.47 658 3 12.46 659 3 12.51 660 3 12.63 661 3 12.45 662 3 12.56 663 3 12.49 664 3 12.58 665 3 12.58 666 3 12.61 667 3 12.55 668 3 12.68 669 3 12.78 670 3 12.71 671 3 12.69 672 3 12.7 673 3 12.72 674 3 12.86 675 3 12.67 676 3 12.73 677 3 12.84 678 3 12.82 679 3 12.87 680 3 12.79 681 3 12.86 682 3 12.76 683 3 12.9 684 3 12.88 685 4 12.86 686 4 12.91 687 4 12.98 688 4 12.89 689 4 12.96



ELA Grade 7

Scale Score


690 4 13.04 691 4 12.97 692 4 12.99 693 4 13.03 694 4 13.05 695 4 13.08 696 4 13 697 4 13.17 698 4 13.11 699 4 13.08 700 4 13.09 701 4 13.3 702 4 13.24 703 4 13.23 704 4 13.43 705 4 13.41 706 4 13.29 707 4 13.41 708 4 13.26 709 4 13.57 710 4 13.34 711 4 13.5 712 4 13.61 713 4 13.45 714 4 13.41 715 4 13.69 716 4 13.73 717 4 13.76 718 4 13.72 719 4 13.62 720 4 13.73 721 4 13.76 722 4 13.78 723 4 14 724 4 14.17 725 4 13.89 726 4 13.83 727 4 14.24 728 4 14.01 729 4 14.34 730 4 13.99 731 4 13.98 732 4 14.48 733 4 14.33



ELA Grade 7

Scale Score


734 4 14.95 735 4 14.27 736 4 14.23 737 4 14.59 738 4 14.81 739 4 15.19 740 4 15.09 741 4 14.94 742 4 14.93 743 4 14.64 744 4 15.38 745 4 15.52 746 4 15.41 747 4 15.36 748 4 15.51 749 4 15.7 750 4 15.6 751 4 15.71 752 4 16.08 753 4 16.1 754 4 15.9 755 4 16.19 756 4 15.99 757 4 16.07 758 4 16.39 759 4 16.28 760 4 16.14 761 4 17.02 762 4 16.53 763 4 16.67 764 4 16.76 766 4 16.99 767 4 17.23 768 4 16.52 770 4 16.96 771 4 16.54 773 4 17.33 775 4 17.96 776 4 17.86 783 4 19.05 786 4 18.82 801 4 22.1




ELA Grade 8

Scale Score


480 1 63.99 482 1 48.98 484 1 49.88 485 1 40.43 486 1 48.3 487 1 37.48 488 1 41.54 489 1 45.6 490 1 45.32 491 1 43.92 492 1 42.7 493 1 45.93 494 1 44.27 495 1 43.18 496 1 42.63 499 1 38.77 500 1 39.18 501 1 37.29 502 1 44.45 503 1 41.09 504 1 43.94 505 1 38.63 506 1 36.84 507 1 35.69 508 1 38.28 509 1 34.74 510 1 33.8 511 1 33.92 512 1 33.76 513 1 32.99 514 1 31.59 515 1 33.75 516 1 33.75 517 1 32.1 518 1 31.38 519 1 32.19 520 1 30.28 521 1 30.16 522 1 29.09 523 1 30.13 524 1 30.76 525 1 29.09



ELA Grade 8

Scale Score


526 1 29.37 527 1 29.28 528 1 29.28 529 1 28.06 530 1 27.37 531 1 26.41 532 1 28.33 533 1 26.89 534 1 27.2 535 1 26.41 536 1 26.51 537 1 24.94 538 1 25.03 539 1 24.52 540 1 24.39 541 1 24.25 542 1 24.67 543 1 23.9 544 1 23.96 545 1 23.45 546 1 23.19 547 1 23.62 548 1 22.23 549 1 22 550 1 21.92 551 1 22.2 552 1 21.07 553 1 20.88 554 1 20.65 555 1 20.82 556 1 20.88 557 1 20.13 558 1 19.98 559 1 20.05 560 1 19.64 561 1 19.7 562 1 19.21 563 1 19.2 564 1 19.08 565 1 18.99 566 1 18.57 567 1 18.38 568 1 18.42 569 1 18.06



ELA Grade 8

Scale Score


570 1 17.73 571 1 17.73 572 1 17.47 573 1 17.27 574 1 17.39 575 1 17.35 576 1 17.07 577 1 16.89 578 1 17.04 579 1 16.78 580 1 16.45 581 1 16.38 582 1 16.57 583 1 16.09 584 1 15.94 585 1 16.09 586 1 15.65 587 1 15.5 588 1 15.67 589 1 15.4 590 1 15.26 591 1 15.22 592 1 15.19 593 1 15.03 594 1 15.05 595 1 15.03 596 1 14.83 597 1 14.71 598 1 14.64 599 1 14.66 600 1 14.5 601 1 14.42 602 1 14.45 603 1 14.26 604 1 14.13 605 1 14.14 606 1 14.1 607 1 13.99 608 1 13.99 609 1 13.87 610 1 13.82 611 1 13.68 612 1 13.95 613 2 13.8



ELA Grade 8

Scale Score


614 2 13.71 615 2 13.65 616 2 13.6 617 2 13.7 618 2 13.59 619 2 13.5 620 2 13.48 621 2 13.48 622 2 13.42 623 2 13.36 624 2 13.35 625 2 13.39 626 2 13.31 627 2 13.38 628 2 13.24 629 2 13.22 630 2 13.35 631 2 13.16 632 2 13.18 633 2 13.28 634 2 13.31 635 2 13.04 636 2 13.23 637 2 12.98 638 2 13.14 639 2 13.1 640 2 13.06 641 2 13.01 642 2 13.21 643 2 13.13 644 2 12.96 645 2 13.11 646 2 12.86 647 2 12.92 648 2 12.88 649 2 13.07 650 2 13.05 651 2 12.96 652 2 13.06 653 2 13.14 654 2 13.04 655 2 13.02 656 3 13.01 657 3 13.09



ELA Grade 8

Scale Score


658 3 13.12 659 3 12.98 660 3 13.06 661 3 12.97 662 3 13.06 663 3 12.92 664 3 12.94 665 3 13.05 666 3 13.02 667 3 13 668 3 13.08 669 3 13.09 670 3 13.03 671 3 13.02 672 3 13.07 673 3 12.94 674 3 13.06 675 3 13.07 676 3 13.08 677 3 13.01 678 3 13.08 679 3 13.18 680 3 13.13 681 3 13.2 682 3 13.16 683 3 13.23 684 3 13.17 685 3 13.13 686 3 13.17 687 3 13.26 688 3 13.34 689 3 13.29 690 3 13.28 691 3 13.35 692 3 13.36 693 3 13.45 694 3 13.41 695 3 13.39 696 3 13.48 697 3 13.44 698 4 13.52 699 4 13.48 700 4 13.6 701 4 13.66



ELA Grade 8

Scale Score


702 4 13.74 703 4 13.83 704 4 13.84 705 4 13.9 706 4 13.93 707 4 14.03 708 4 13.96 709 4 13.99 710 4 13.97 711 4 14.08 712 4 14.08 713 4 14.17 714 4 14.12 715 4 14.24 716 4 14.36 717 4 14.31 718 4 14.43 719 4 14.63 720 4 14.61 721 4 14.77 722 4 14.59 723 4 14.79 724 4 14.72 725 4 14.74 726 4 14.76 727 4 14.8 728 4 14.75 729 4 14.92 730 4 14.86 731 4 15.19 732 4 14.94 733 4 15.02 734 4 15.33 735 4 15.38 736 4 15.44 737 4 15.55 738 4 15.43 739 4 15.79 740 4 15.4 741 4 15.45 742 4 15.88 743 4 15.95 744 4 15.58 745 4 15.85



ELA Grade 8

Scale Score


746 4 15.96 747 4 15.66 748 4 16 749 4 15.83 750 4 16.97 751 4 16.29 752 4 16.26 753 4 16.72 754 4 17.04 755 4 16.69 756 4 17.06 757 4 17.07 758 4 16.78 759 4 17.01 760 4 18.23 761 4 17.65 762 4 18.14 763 4 17.77 764 4 18.09 765 4 17.88 766 4 18.92 767 4 18.04 768 4 17.83 769 4 19.1 770 4 18.63 771 4 19.67 772 4 18.97 773 4 19.15 774 4 20.01 775 4 19.29 776 4 20.02 777 4 19.47 779 4 21.33 780 4 20.77 781 4 21.48 782 4 20.94 783 4 21.1 784 4 22.18 785 4 22.2 786 4 22.04 787 4 22.74 790 4 23.55 791 4 22.79 792 4 24.09



ELA Grade 8

Scale Score


793 4 21.99 795 4 24.68 796 4 22.91 797 4 23.97 798 4 25.92 806 4 26.56 807 4 29.51 809 4 30.14 835 4 35.3 841 4 38.73 847 4 37.72 851 4 43.12

Table 7: CSEM at Each Scale Score, Mathematics Grade 3

Mathematics Grade 3

Scale Score


308 1 38.15 309 1 38.11 310 1 34.5 311 1 34.08 312 1 33.9 313 1 33.38 314 1 31.17 315 1 32.57 316 1 29.1 317 1 29.75 318 1 29.82 319 1 29.63 320 1 28.03 321 1 30 322 1 28.31 323 1 28.19 324 1 26.29 325 1 26.67 326 1 25.7 327 1 24.23 328 1 24.4 329 1 24.66 330 1 24.2 331 1 23.48



Mathematics Grade 3

Scale Score


332 1 23.24 333 1 22.41 334 1 22.35 335 1 22.36 336 1 22.33 337 1 20.94 338 1 20.81 339 1 20.32 340 1 19.8 341 1 20.78 342 1 18.99 343 1 18.41 344 1 18.66 345 1 18.6 346 1 17.58 347 1 17.52 348 1 16.87 349 1 16.79 350 1 16.36 351 1 16.43 352 1 15.65 353 1 15.67 354 1 15.53 355 1 15.55 356 1 14.36 357 1 14.44 358 1 14.18 359 1 14.29 360 1 13.65 361 1 13.37 362 1 13.2 363 1 12.98 364 1 12.88 365 1 12.62 366 1 12.33 367 1 12.23 368 1 12.07 369 1 11.82 370 1 11.54 371 1 11.59 372 1 11.4 373 1 11.04 374 1 10.89 375 1 10.83



Mathematics Grade 3

Scale Score


376 1 10.6 377 1 10.52 378 1 10.23 379 1 10.26 380 1 9.93 381 1 10.01 382 1 9.85 383 1 9.64 384 1 9.5 385 1 9.43 386 1 9.26 387 1 9.26 388 1 9.18 389 1 8.95 390 1 8.87 391 1 8.84 392 1 8.72 393 1 8.59 394 1 8.5 395 1 8.41 396 1 8.34 397 1 8.32 398 1 8.25 399 1 8.23 400 1 8.17 401 2 8.1 402 2 8.04 403 2 7.99 404 2 7.93 405 2 7.89 406 2 7.83 407 2 7.75 408 2 7.74 409 2 7.67 410 2 7.64 411 2 7.63 412 2 7.58 413 2 7.55 414 2 7.53 415 2 7.53 416 2 7.48 417 2 7.46 418 2 7.43 419 2 7.41



Mathematics Grade 3

Scale Score


420 2 7.37 421 2 7.33 422 2 7.33 423 2 7.31 424 2 7.3 425 2 7.31 426 3 7.25 427 3 7.26 428 3 7.26 429 3 7.25 430 3 7.24 431 3 7.24 432 3 7.24 433 3 7.26 434 3 7.24 435 3 7.28 436 3 7.26 437 3 7.28 438 3 7.24 439 3 7.3 440 3 7.28 441 3 7.3 442 3 7.32 443 3 7.34 444 3 7.36 445 3 7.36 446 3 7.38 447 3 7.46 448 4 7.39 449 4 7.42 450 4 7.47 451 4 7.46 452 4 7.54 453 4 7.54 454 4 7.53 455 4 7.57 456 4 7.61 457 4 7.57 458 4 7.64 459 4 7.68 460 4 7.68 461 4 7.7 462 4 7.76 463 4 7.77



Mathematics Grade 3

Scale Score


464 4 7.83 465 4 7.89 466 4 7.9 467 4 7.94 468 4 7.98 469 4 7.92 470 4 7.99 471 4 8.06 472 4 8.15 473 4 8.21 474 4 8.24 475 4 8.41 476 4 8.45 477 4 8.34 478 4 8.49 479 4 8.49 480 4 8.56 481 4 8.63 482 4 8.75 483 4 8.73 484 4 8.77 485 4 8.91 486 4 8.93 487 4 9.24 488 4 8.98 489 4 9.15 490 4 9.46 491 4 9.35 492 4 9.41 493 4 9.6 494 4 9.59 495 4 9.65 496 4 9.95 497 4 9.74 498 4 9.88 499 4 10.05 500 4 10.6 501 4 10.31 502 4 10.69 503 4 11.07 504 4 10.97 505 4 11.4 506 4 11.22 507 4 11.37



Mathematics Grade 3

Scale Score


508 4 11.26 509 4 11.88 510 4 11.72 511 4 13.52 512 4 12.31 513 4 12.84 514 4 12.81 515 4 12.77 516 4 17.17 517 4 12.93 518 4 14.41 519 4 13.19 520 4 13.64 521 4 19.02 522 4 14.28 523 4 15.53 525 4 14.21 526 4 15.71 527 4 15.58 528 4 16.1 529 4 16.2 530 4 17.3 531 4 19.05 532 4 17.93 534 4 19.58 535 4 19.35 537 4 18.9 539 4 19.66 541 4 20.89 542 4 23.57 544 4 21.9 548 4 25.01


Mathematics Grade 4

Scale Score


312 1 47.87 313 1 41.8 314 1 47.05 315 1 47.48



Mathematics Grade 4

Scale Score


316 1 36.98 317 1 43.42 318 1 41.63 320 1 38 321 1 37.46 322 1 42.44 323 1 38.14 324 1 38.34 325 1 35 326 1 35.93 327 1 35.18 328 1 34.91 329 1 35.09 330 1 33.72 331 1 32.89 332 1 32.47 333 1 27.85 334 1 32.1 335 1 31.28 336 1 31.28 337 1 31.49 338 1 28.4 339 1 30.48 340 1 28.82 341 1 28.4 342 1 28.09 343 1 28.43 344 1 26.8 345 1 25.98 346 1 26.8 347 1 26.67 348 1 24.93 349 1 25.54 350 1 25.55 351 1 24.41 352 1 24.43 353 1 24.14 354 1 23.32 355 1 23.5 356 1 22.24 357 1 22.83 358 1 21.7 359 1 21.63 360 1 21.36



Mathematics Grade 4

Scale Score


361 1 20.44 362 1 20.55 363 1 19.93 364 1 20.65 365 1 19.75 366 1 19.49 367 1 19.17 368 1 19.03 369 1 18.77 370 1 18.04 371 1 17.78 372 1 17.51 373 1 17.55 374 1 16.71 375 1 16.89 376 1 17.12 377 1 16.3 378 1 16.44 379 1 16.74 380 1 15.99 381 1 15.87 382 1 15.77 383 1 15.41 384 1 15 385 1 14.87 386 1 14.5 387 1 14.24 388 1 14.36 389 1 14.15 390 1 13.95 391 1 13.87 392 1 13.37 393 1 13.42 394 1 13.04 395 1 13.08 396 1 12.91 397 1 13.17 398 1 12.73 399 1 12.52 400 1 12.68 401 1 12.56 402 1 12.23 403 1 12.16 404 1 11.89



Mathematics Grade 4

Scale Score


405 1 12.1 406 1 12 407 1 11.77 408 1 11.55 409 1 11.49 410 1 11.57 411 1 11.42 412 1 11.35 413 1 11.1 414 1 11.12 415 1 11.03 416 1 10.98 417 1 10.87 418 1 10.9 419 1 10.63 420 1 10.69 421 1 10.58 422 2 10.58 423 2 10.46 424 2 10.38 425 2 10.33 426 2 10.22 427 2 10.22 428 2 10.1 429 2 10.13 430 2 10.04 431 2 9.98 432 2 9.97 433 2 9.9 434 2 9.77 435 2 9.79 436 2 9.74 437 2 9.71 438 2 9.7 439 2 9.65 440 2 9.64 441 2 9.51 442 2 9.53 443 2 9.47 444 2 9.48 445 2 9.49 446 2 9.4 447 2 9.44 448 2 9.36



Mathematics Grade 4

Scale Score


449 2 9.39 450 2 9.28 451 2 9.27 452 2 9.26 453 2 9.26 454 2 9.28 455 2 9.2 456 3 9.23 457 3 9.14 458 3 9.2 459 3 9.16 460 3 9.1 461 3 9.1 462 3 9.09 463 3 9.03 464 3 9.07 465 3 9.07 466 3 9.04 467 3 9.07 468 3 9.01 469 3 9.04 470 3 8.96 471 3 8.91 472 3 8.99 473 3 8.97 474 3 9.03 475 3 8.98 476 3 8.97 477 3 8.98 478 4 9.02 479 4 9.01 480 4 9.09 481 4 9.04 482 4 9.12 483 4 9.07 484 4 9.13 485 4 9.06 486 4 9.09 487 4 9.13 488 4 9.11 489 4 9.15 490 4 9.14 491 4 9.26 492 4 9.19



Mathematics Grade 4

Scale Score


493 4 9.29 494 4 9.3 495 4 9.34 496 4 9.28 497 4 9.36 498 4 9.41 499 4 9.47 500 4 9.44 501 4 9.44 502 4 9.44 503 4 9.45 504 4 9.53 505 4 9.52 506 4 9.62 507 4 9.63 508 4 9.61 509 4 9.76 510 4 9.69 511 4 9.91 512 4 9.83 513 4 9.85 514 4 9.87 515 4 10.1 516 4 10.02 517 4 10.13 518 4 10.15 519 4 10.41 520 4 10.2 521 4 10.32 522 4 10.44 523 4 10.73 524 4 10.5 525 4 10.6 526 4 10.74 527 4 10.87 528 4 10.74 529 4 11.07 530 4 10.96 531 4 10.88 532 4 11.16 533 4 11.66 534 4 11.56 535 4 10.82 536 4 11.45



Mathematics Grade 4

Scale Score


537 4 11.71 538 4 11.96 539 4 12.06 540 4 12.46 541 4 11.76 542 4 12.87 543 4 11.33 544 4 12.8 545 4 12.63 546 4 12.69 547 4 13.63 548 4 13.1 549 4 13.75 550 4 13.98 551 4 13.18 552 4 14.31 553 4 13.88 554 4 14.77 555 4 14.03 556 4 15.13 557 4 14.42 558 4 14.61 559 4 15.11 560 4 16.1 561 4 15.22 564 4 16.4 565 4 16.32 566 4 17.41 568 4 17.02 569 4 17 570 4 17.64 573 4 20.44 574 4 18.01 575 4 19.11 576 4 19.62 577 4 19.76 579 4 22.21 580 4 19.45 581 4 18.59 582 4 23.54 584 4 23.16 586 4 24.01 589 4 23.4 590 4 24.62



Mathematics Grade 4

Scale Score


591 4 24.15 592 4 23.51 595 4 26.65 605 4 27.81 608 4 33.03


Mathematics Grade 5

Scale Score


320 1 52.46 321 1 41.92 322 1 51.71 323 1 47.27 324 1 48.07 325 1 44.51 326 1 43.58 327 1 43.35 328 1 44.57 329 1 43.01 330 1 43.78 331 1 40.19 332 1 43.59 333 1 44 334 1 39.62 335 1 37.13 336 1 41.26 337 1 39.73 338 1 38.5 339 1 39.66 340 1 39.59 341 1 37.18 342 1 37.41 343 1 35.72 344 1 36.53 345 1 37.71 346 1 37.43 347 1 35.86 348 1 35.37 349 1 35 350 1 35.31



Mathematics Grade 5

Scale Score


351 1 34.12 352 1 32.94 353 1 33.65 354 1 32.1 355 1 32.39 356 1 31.46 357 1 29.86 358 1 29.88 359 1 30.23 360 1 29.69 361 1 28.91 362 1 29.34 363 1 28.8 364 1 28.59 365 1 27.76 366 1 27.7 367 1 27.74 368 1 25.67 369 1 26.41 370 1 25.88 371 1 26.39 372 1 25.05 373 1 24.92 374 1 24.39 375 1 23.53 376 1 24.6 377 1 23.99 378 1 22.44 379 1 22.8 380 1 22.59 381 1 22.21 382 1 22.36 383 1 21.38 384 1 20.96 385 1 21.28 386 1 20.73 387 1 20.07 388 1 20.02 389 1 20.39 390 1 20.49 391 1 19.3 392 1 18.78 393 1 19.01 394 1 18.71



Mathematics Grade 5

Scale Score


395 1 18.93 396 1 18.3 397 1 17.59 398 1 17.6 399 1 17.71 400 1 17.13 401 1 16.64 402 1 16.88 403 1 16.82 404 1 16.41 405 1 16.55 406 1 15.68 407 1 15.94 408 1 15.59 409 1 15.57 410 1 15.4 411 1 15 412 1 15.11 413 1 14.96 414 1 14.73 415 1 14.62 416 1 14.43 417 1 14.2 418 1 14.17 419 1 14.12 420 1 13.8 421 1 13.68 422 1 13.59 423 1 13.38 424 1 13.39 425 1 13.35 426 1 13.51 427 1 13.09 428 1 12.99 429 1 12.77 430 1 12.67 431 1 12.74 432 1 12.58 433 1 12.52 434 1 12.4 435 1 12.34 436 1 12.36 437 1 12.38 438 1 12.17



Mathematics Grade 5

Scale Score


439 1 11.99 440 1 12.03 441 1 12.02 442 1 11.89 443 1 11.86 444 1 11.74 445 1 11.76 446 1 11.74 447 1 11.69 448 1 11.63 449 2 11.63 450 2 11.5 451 2 11.47 452 2 11.48 453 2 11.38 454 2 11.24 455 2 11.28 456 2 11.3 457 2 11.19 458 2 11.15 459 2 11.15 460 2 11.1 461 2 11.08 462 2 11.05 463 2 10.92 464 2 10.92 465 2 10.9 466 2 10.91 467 2 10.89 468 2 10.78 469 2 10.81 470 2 10.8 471 2 10.78 472 2 10.75 473 2 10.76 474 2 10.73 475 2 10.7 476 2 10.62 477 2 10.68 478 2 10.71 479 2 10.74 480 2 10.69 481 2 10.63 482 2 10.69



Mathematics Grade 5

Scale Score


483 2 10.67 484 2 10.69 485 2 10.6 486 2 10.66 487 3 10.58 488 3 10.61 489 3 10.58 490 3 10.53 491 3 10.61 492 3 10.5 493 3 10.51 494 3 10.4 495 3 10.51 496 3 10.45 497 3 10.47 498 3 10.53 499 3 10.49 500 3 10.47 501 3 10.56 502 3 10.51 503 3 10.45 504 3 10.46 505 3 10.49 506 3 10.44 507 3 10.43 508 3 10.44 509 3 10.44 510 3 10.55 511 3 10.41 512 3 10.54 513 4 10.49 514 4 10.54 515 4 10.51 516 4 10.56 517 4 10.52 518 4 10.65 519 4 10.63 520 4 10.54 521 4 10.59 522 4 10.66 523 4 10.62 524 4 10.63 525 4 10.69 526 4 10.69



Mathematics Grade 5

Scale Score


527 4 10.73 528 4 10.74 529 4 10.71 530 4 10.66 531 4 10.71 532 4 10.66 533 4 10.68 534 4 10.66 535 4 10.84 536 4 10.62 537 4 10.8 538 4 10.76 539 4 10.79 540 4 10.66 541 4 10.79 542 4 10.75 543 4 10.75 544 4 10.71 545 4 10.83 546 4 10.69 547 4 10.73 548 4 10.75 549 4 10.72 550 4 10.87 551 4 10.76 552 4 11.03 553 4 10.75 554 4 10.87 555 4 10.91 556 4 10.99 557 4 10.86 558 4 11 559 4 11.14 560 4 11.04 561 4 11 562 4 11.28 563 4 10.94 564 4 11.3 565 4 11.16 566 4 11.35 567 4 11.77 568 4 11.58 569 4 11.32 570 4 11.75



Mathematics Grade 5

Scale Score


571 4 11.75 572 4 11.8 573 4 11.96 574 4 11.87 575 4 12.12 576 4 12.6 577 4 12.24 578 4 12.19 579 4 12.94 580 4 12.73 581 4 12.32 582 4 12.93 583 4 13.25 584 4 12.73 585 4 13.27 586 4 13.06 587 4 13.23 588 4 13.79 589 4 14.07 590 4 13.46 591 4 13.37 592 4 13.22 593 4 14.92 594 4 14.05 595 4 16.24 596 4 15.24 597 4 15.23 598 4 13.71 599 4 13.36 600 4 16.95 601 4 14.24 602 4 14.45 603 4 14.98 604 4 15.8 605 4 14.5 606 4 14.72 607 4 15.34 608 4 16.74 609 4 15.43 610 4 15.49 611 4 15.51 612 4 16.04 613 4 19.7 615 4 15.89



Mathematics Grade 5

Scale Score


616 4 16.48 617 4 16.68 618 4 16.83 620 4 22.65 621 4 18.51 622 4 22.75 627 4 19.01 628 4 18.8 629 4 18.87 632 4 19.48 635 4 21.69 638 4 23.89 645 4 25.59 646 4 27.98 647 4 29.4 654 4 29.9 658 4 28.21 659 4 36.75


Mathematics Grade 6

Scale Score


330 1 68.25 331 1 62.72 332 1 52.61 333 1 64.07 334 1 56.52 335 1 67.76 336 1 63.79 337 1 57.08 338 1 56.67 339 1 52.4 340 1 52.41 341 1 53.8 342 1 56.61 343 1 55.12 345 1 49.56 346 1 43.27 347 1 47.77 348 1 51.24



Mathematics Grade 6

Scale Score


349 1 54.23 350 1 46.97 352 1 48.06 353 1 45.01 354 1 44.4 355 1 42.44 356 1 47.93 357 1 42.79 358 1 45.82 359 1 42.52 360 1 44.21 361 1 42.85 362 1 41.88 363 1 41.4 364 1 39.35 365 1 42.79 366 1 37.64 367 1 43.17 368 1 38.9 369 1 37.17 370 1 38.53 371 1 36.37 372 1 37.79 373 1 35.11 374 1 38.08 375 1 34.49 376 1 35.97 377 1 35.81 378 1 35.44 379 1 32.59 380 1 34.55 381 1 32.84 382 1 32.25 383 1 32.14 384 1 32.04 385 1 31.44 386 1 30.32 387 1 30.66 388 1 30.48 389 1 31.56 390 1 29.79 391 1 29.7 392 1 29.54 393 1 30.08



Mathematics Grade 6

Scale Score


394 1 28.17 395 1 28.91 396 1 27.82 397 1 26.8 398 1 27.68 399 1 26.29 400 1 26.68 401 1 25.78 402 1 26.45 403 1 25.7 404 1 25.29 405 1 26.46 406 1 24.7 407 1 24.31 408 1 24.55 409 1 23.83 410 1 24.4 411 1 24.11 412 1 23.54 413 1 23.39 414 1 23.25 415 1 23.19 416 1 22.39 417 1 22.65 418 1 22.04 419 1 21.35 420 1 21.81 421 1 21.86 422 1 21.05 423 1 21.2 424 1 21.11 425 1 21.22 426 1 21.07 427 1 20.07 428 1 20.37 429 1 19.91 430 1 19.73 431 1 19.17 432 1 19.33 433 1 19.05 434 1 18.97 435 1 18.64 436 1 18.99 437 1 18.71



Mathematics Grade 6

Scale Score


438 1 18.43 439 1 18.58 440 1 18.14 441 1 18.19 442 1 18.01 443 1 17.69 444 1 17.89 445 1 17.85 446 1 17.61 447 1 17.49 448 1 17.09 449 1 17.2 450 1 16.86 451 1 16.85 452 1 16.76 453 1 16.63 454 1 16.71 455 1 16.6 456 1 16.42 457 1 16.53 458 1 16.3 459 1 16.22 460 1 16.26 461 1 16.23 462 1 16.01 463 1 15.78 464 1 15.97 465 1 15.77 466 1 15.89 467 1 15.63 468 1 15.77 469 1 15.5 470 1 15.44 471 1 15.37 472 1 15.29 473 1 15.2 474 2 15.19 475 2 15.13 476 2 15.13 477 2 14.97 478 2 14.87 479 2 14.83 480 2 14.81 481 2 14.67



Mathematics Grade 6

Scale Score


482 2 14.66 483 2 14.48 484 2 14.59 485 2 14.46 486 2 14.37 487 2 14.26 488 2 14.26 489 2 14.25 490 2 14.13 491 2 14.09 492 2 13.98 493 2 14.07 494 2 13.83 495 2 13.77 496 2 13.68 497 2 13.72 498 2 13.64 499 2 13.51 500 2 13.36 501 2 13.41 502 2 13.38 503 2 13.23 504 2 13.24 505 2 13.22 506 2 13.05 507 2 12.99 508 2 12.97 509 2 12.96 510 2 12.9 511 2 12.81 512 2 12.76 513 2 12.7 514 2 12.63 515 2 12.58 516 2 12.5 517 2 12.54 518 3 12.43 519 3 12.41 520 3 12.39 521 3 12.24 522 3 12.32 523 3 12.21 524 3 12.27 525 3 12.12



Mathematics Grade 6

Scale Score


526 3 12.06 527 3 12.17 528 3 11.98 529 3 12.03 530 3 12.04 531 3 11.95 532 3 11.88 533 3 11.92 534 3 11.85 535 3 11.89 536 3 11.77 537 3 11.76 538 3 11.75 539 3 11.72 540 3 11.72 541 3 11.69 542 3 11.62 543 3 11.61 544 3 11.52 545 3 11.65 546 3 11.5 547 3 11.56 548 3 11.49 549 3 11.57 550 4 11.55 551 4 11.44 552 4 11.53 553 4 11.49 554 4 11.48 555 4 11.52 556 4 11.42 557 4 11.5 558 4 11.4 559 4 11.43 560 4 11.44 561 4 11.39 562 4 11.43 563 4 11.38 564 4 11.28 565 4 11.44 566 4 11.41 567 4 11.35 568 4 11.38 569 4 11.32



Mathematics Grade 6

Scale Score


570 4 11.46 571 4 11.17 572 4 11.31 573 4 11.28 574 4 11.25 575 4 11.29 576 4 11.23 577 4 11.28 578 4 11.29 579 4 11.29 580 4 11.13 581 4 11.37 582 4 11.43 583 4 11.44 584 4 11.43 585 4 11.34 586 4 11.35 587 4 11.36 588 4 11.2 589 4 11.31 590 4 11.32 591 4 11.76 592 4 11.39 593 4 11.29 594 4 11.38 595 4 11.58 596 4 11.85 597 4 11.5 598 4 11.31 599 4 11.22 600 4 11.75 601 4 11.99 602 4 11.81 603 4 12.01 604 4 11.59 605 4 11.79 606 4 12.17 607 4 11.91 608 4 12.25 609 4 12.24 610 4 12.02 611 4 12.09 612 4 12.11 613 4 12.18



Mathematics Grade 6

Scale Score


614 4 12.45 615 4 12.58 616 4 11.89 617 4 12.51 618 4 12.75 619 4 12.82 620 4 12.85 621 4 12.19 622 4 12.82 623 4 12.67 624 4 12.94 625 4 13.32 626 4 13.06 627 4 13.53 628 4 13.99 629 4 13.38 630 4 14.11 631 4 14.1 632 4 13.68 633 4 14.9 634 4 14.81 635 4 15.31 636 4 14.85 637 4 15.1 638 4 14.27 639 4 15.33 640 4 15.51 641 4 16.45 643 4 17.35 644 4 17.18 645 4 16.54 646 4 16.44 647 4 17.79 648 4 17.73 649 4 17.25 650 4 18.54 651 4 21.08 652 4 17.16 654 4 18.39 656 4 18.55 658 4 19.41 660 4 20.91 667 4 19.71 669 4 22.29



Mathematics Grade 6

Scale Score


670 4 22.1 671 4 24.2 674 4 24 675 4 23.79 682 4 28.39 696 4 27.88 707 4 36.2 710 4 45.75 720 4 53.7


Mathematics Grade 7

Scale Score


348 1 75.99 349 1 76 350 1 76 351 1 76 352 1 76 353 1 76 354 1 76 355 1 76 356 1 76 357 1 75.07 358 1 75.27 359 1 75.01 360 1 76 361 1 74.01 362 1 73.89 363 1 72.94 364 1 68.75 365 1 71.12 366 1 62.07 367 1 70.69 368 1 67.33 369 1 67.28 370 1 70.55 371 1 65.37 372 1 65.5 373 1 65.07 374 1 68.01



Mathematics Grade 7

Scale Score


375 1 65.6 376 1 62.53 377 1 63.94 378 1 62.89 379 1 62.54 380 1 60.23 381 1 60.89 382 1 61.05 383 1 61.17 384 1 58.25 385 1 57.39 386 1 56.01 387 1 56.99 388 1 56.21 389 1 57.23 390 1 53.02 391 1 52.17 392 1 51.8 393 1 53.87 394 1 50.34 395 1 50.01 396 1 50.56 397 1 49.36 398 1 48.57 399 1 49.15 400 1 47.04 401 1 47.44 402 1 46.32 403 1 46.12 404 1 44.82 405 1 44.52 406 1 44.6 407 1 44.23 408 1 43.45 409 1 43.76 410 1 42.06 411 1 40.96 412 1 43.45 413 1 41.05 414 1 41.2 415 1 38.95 416 1 38.65 417 1 39.73 418 1 38.71



Mathematics Grade 7

Scale Score


419 1 38.1 420 1 37.4 421 1 37.33 422 1 35.95 423 1 36.12 424 1 33.57 425 1 35.67 426 1 34.84 427 1 33.81 428 1 33.14 429 1 33.62 430 1 33.66 431 1 32.18 432 1 32.72 433 1 31.7 434 1 31.26 435 1 31.24 436 1 30.64 437 1 30.59 438 1 29.79 439 1 29.65 440 1 28.89 441 1 28.87 442 1 28.42 443 1 28.39 444 1 28.12 445 1 27.48 446 1 27.03 447 1 26.49 448 1 26.62 449 1 26.84 450 1 25.78 451 1 24.9 452 1 25.34 453 1 24.99 454 1 24.32 455 1 24.63 456 1 24.07 457 1 23.44 458 1 23.36 459 1 23.52 460 1 22.81 461 1 22.78 462 1 22.64



Mathematics Grade 7

Scale Score


463 1 22.52 464 1 21.99 465 1 21.86 466 1 21.8 467 1 21.22 468 1 21.49 469 1 21.2 470 1 21.14 471 1 20.71 472 1 20.76 473 1 20.04 474 1 20.25 475 1 19.87 476 1 20.01 477 1 19.66 478 1 19.5 479 1 18.72 480 1 19.21 481 1 19.04 482 1 19.09 483 1 18.73 484 1 18.58 485 1 18.53 486 1 18.37 487 1 18.32 488 1 17.87 489 1 17.72 490 1 17.89 491 1 17.67 492 1 17.55 493 1 17.7 494 1 17.49 495 1 17.03 496 1 17.07 497 1 17.11 498 1 17.09 499 1 16.78 500 1 16.69 501 1 16.8 502 1 16.6 503 2 16.51 504 2 16.51 505 2 16.38 506 2 16.42



Mathematics Grade 7

Scale Score


507 2 16.34 508 2 16.22 509 2 16.03 510 2 15.96 511 2 16.07 512 2 15.82 513 2 15.83 514 2 15.59 515 2 15.61 516 2 15.47 517 2 15.45 518 2 15.39 519 2 15.45 520 2 15.18 521 2 15.17 522 2 15.17 523 2 15.14 524 2 15.12 525 2 15.04 526 2 14.83 527 2 14.69 528 2 14.8 529 2 14.56 530 2 14.62 531 2 14.54 532 2 14.52 533 2 14.48 534 2 14.47 535 2 14.23 536 2 14.28 537 2 14.35 538 2 14.08 539 2 14.23 540 2 14.14 541 2 14.24 542 2 14.09 543 2 14.11 544 2 14.06 545 2 14.08 546 2 14 547 2 14.02 548 3 13.96 549 3 13.86 550 3 13.88



Mathematics Grade 7

Scale Score


551 3 13.94 552 3 13.91 553 3 13.84 554 3 13.79 555 3 13.79 556 3 13.78 557 3 13.7 558 3 13.63 559 3 13.86 560 3 13.72 561 3 13.76 562 3 13.74 563 3 13.76 564 3 13.67 565 3 13.69 566 3 13.61 567 3 13.58 568 3 13.73 569 3 13.75 570 3 13.74 571 3 13.59 572 3 13.79 573 3 13.68 574 3 13.62 575 3 13.77 576 3 13.48 577 3 13.77 578 3 13.65 579 3 13.61 580 3 13.57 581 3 13.67 582 3 13.63 583 4 13.56 584 4 13.61 585 4 13.56 586 4 13.58 587 4 13.61 588 4 13.52 589 4 13.54 590 4 13.54 591 4 13.54 592 4 13.45 593 4 13.48 594 4 13.38



Mathematics Grade 7

Scale Score


595 4 13.27 596 4 13.41 597 4 13.36 598 4 13.25 599 4 13.33 600 4 13.33 601 4 13.25 602 4 13.3 603 4 13.37 604 4 13.29 605 4 13.36 606 4 13.26 607 4 13.36 608 4 13.12 609 4 13.34 610 4 13.43 611 4 13.44 612 4 13.37 613 4 13.43 614 4 13.36 615 4 13.25 616 4 13.39 617 4 13.28 618 4 13.28 619 4 13.17 620 4 13.34 621 4 13.13 622 4 13.43 623 4 13.24 624 4 13.37 625 4 13.38 626 4 13.46 627 4 13.6 628 4 13.54 629 4 13.31 630 4 13.63 631 4 13.8 632 4 13.84 633 4 13.89 634 4 13.74 635 4 14.04 636 4 13.49 637 4 13.78 638 4 13.88



Mathematics Grade 7

Scale Score


639 4 14.31 640 4 13.83 641 4 14.28 642 4 13.94 643 4 14.43 644 4 14.18 645 4 14.43 646 4 13.95 647 4 13.86 648 4 13.81 649 4 14.46 650 4 14.07 651 4 14.13 652 4 14.46 653 4 14.86 654 4 15.2 655 4 14.74 656 4 14.86 657 4 14.87 658 4 15.82 659 4 14.86 660 4 14.68 661 4 14.92 662 4 16.16 663 4 14.63 664 4 14.76 665 4 14.97 666 4 15.67 668 4 15.56 669 4 16.07 670 4 15.16 671 4 15.81 673 4 15.94 674 4 15.95 675 4 17.83 676 4 15.81 677 4 15.37 678 4 15.18 679 4 15.36 680 4 16.83 681 4 16.67 682 4 16.06 683 4 18.4 684 4 17.29



Mathematics Grade 7

Scale Score


685 4 17.6 686 4 15.95 687 4 16.71 688 4 17.69 689 4 14.46 690 4 11.95 691 4 15.76 693 4 19.48 694 4 23.85 695 4 18.26 696 4 14.09 697 4 16.68 699 4 8.71 700 4 14.82 702 4 17.84 703 4 10.55 704 4 11.67 705 4 23.97 706 4 21.44 707 4 20.57 708 4 19.33 710 4 21.36 712 4 24.33 715 4 21.49 721 4 24.72 722 4 25.3 742 4 31.55 750 4 34.67


Mathematics Grade 8

Scale Score


350 1 89.96 351 1 80.33 352 1 80.64 353 1 90.79 354 1 78.58 355 1 75.59 356 1 78.07 357 1 83.9



Mathematics Grade 8

Scale Score


358 1 77.59 359 1 74.91 360 1 77.29 361 1 70.88 362 1 82.64 363 1 71.42 364 1 70.88 365 1 68.67 366 1 73.28 367 1 71.75 368 1 71.73 369 1 74.1 370 1 67.75 371 1 68.64 372 1 69.23 373 1 68.63 374 1 61.92 375 1 65.65 376 1 65.96 377 1 66.71 378 1 60.81 379 1 66.23 380 1 62.57 381 1 59.27 382 1 63.19 383 1 64.06 384 1 62.77 385 1 57.97 386 1 59.44 387 1 58.92 388 1 55.48 389 1 55.08 390 1 53.51 391 1 58.73 392 1 57.67 393 1 56.03 394 1 54.01 395 1 59.12 396 1 52.69 397 1 55.53 398 1 56.31 399 1 53.59 400 1 51.35 401 1 49.42



Mathematics Grade 8

Scale Score


402 1 49.9 403 1 50.36 404 1 47.19 405 1 49.39 406 1 48.82 407 1 48.38 408 1 49.96 409 1 48.15 410 1 47.37 411 1 46.82 412 1 46.86 413 1 44.86 414 1 44.78 415 1 45.7 416 1 45.61 417 1 43.57 418 1 43.63 419 1 44.1 420 1 44.39 421 1 42.78 422 1 41.36 423 1 42 424 1 41.75 425 1 39.54 426 1 40.9 427 1 39.04 428 1 40.47 429 1 39.82 430 1 38.55 431 1 37.97 432 1 37.87 433 1 37.98 434 1 37.57 435 1 35.93 436 1 36.13 437 1 36.6 438 1 35.12 439 1 34.87 440 1 35.49 441 1 35.59 442 1 34.49 443 1 34.97 444 1 33.66 445 1 33.47



Mathematics Grade 8

Scale Score


446 1 33.58 447 1 33.57 448 1 32.56 449 1 33.46 450 1 32.48 451 1 32.76 452 1 32.27 453 1 32.02 454 1 31.74 455 1 30.44 456 1 30.56 457 1 30 458 1 30.75 459 1 30.17 460 1 30.21 461 1 29.76 462 1 28.82 463 1 28.53 464 1 28.31 465 1 28.6 466 1 28.28 467 1 28.29 468 1 27.3 469 1 28.02 470 1 27.76 471 1 27.32 472 1 27.15 473 1 26.92 474 1 26.48 475 1 26.71 476 1 26.69 477 1 25.63 478 1 25.61 479 1 25.64 480 1 25.28 481 1 25.26 482 1 25.03 483 1 24.72 484 1 24.57 485 1 23.87 486 1 24.53 487 1 23.69 488 1 23.69 489 1 23.48



Mathematics Grade 8

Scale Score


490 1 23.58 491 1 23.46 492 1 23.35 493 1 23.25 494 1 23 495 1 22.81 496 1 22.62 497 1 22.5 498 1 22.44 499 1 22.28 500 1 22.13 501 1 22.17 502 1 21.99 503 1 22 504 1 21.67 505 1 21.8 506 1 21.26 507 1 21.14 508 1 21.22 509 1 21.38 510 1 20.93 511 1 21.04 512 1 20.82 513 1 20.64 514 1 20.53 515 1 20.5 516 1 20.39 517 1 20.03 518 1 20.07 519 1 20.13 520 1 20.22 521 1 19.82 522 1 20.14 523 1 19.72 524 1 19.56 525 1 19.35 526 1 19.63 527 1 19.42 528 1 19.27 529 2 19.01 530 2 19.03 531 2 19.06 532 2 19.06 533 2 18.85



Mathematics Grade 8

Scale Score


534 2 18.97 535 2 18.83 536 2 18.76 537 2 18.53 538 2 18.53 539 2 18.4 540 2 18.38 541 2 18.26 542 2 18.15 543 2 18.13 544 2 18.15 545 2 17.91 546 2 17.99 547 2 17.82 548 2 17.77 549 2 17.65 550 2 17.62 551 2 17.64 552 2 17.55 553 2 17.57 554 2 17.34 555 2 17.27 556 2 17.34 557 2 17.2 558 2 17.16 559 2 16.98 560 2 16.94 561 2 16.98 562 2 16.96 563 2 16.92 564 2 16.73 565 2 16.7 566 2 16.51 567 2 16.64 568 2 16.58 569 2 16.48 570 2 16.47 571 2 16.47 572 2 16.29 573 2 16.25 574 2 16.21 575 2 16.26 576 2 16.08 577 2 15.95



Mathematics Grade 8

Scale Score


578 2 16.1 579 2 15.93 580 2 15.83 581 2 15.9 582 2 15.72 583 2 15.81 584 2 15.78 585 2 15.61 586 2 15.69 587 3 15.71 588 3 15.44 589 3 15.44 590 3 15.54 591 3 15.52 592 3 15.58 593 3 15.34 594 3 15.43 595 3 15.36 596 3 15.29 597 3 15.2 598 3 15.42 599 3 15.18 600 3 15.15 601 3 14.99 602 3 15.09 603 3 15.13 604 3 15.18 605 3 15.12 606 3 15.12 607 3 15.03 608 3 15.12 609 3 14.95 610 3 15.12 611 3 14.98 612 3 15.08 613 3 15.03 614 3 14.88 615 3 15 616 3 14.79 617 4 14.89 618 4 14.76 619 4 14.79 620 4 14.73 621 4 14.83



Mathematics Grade 8

Scale Score


622 4 14.8 623 4 14.79 624 4 14.74 625 4 14.82 626 4 14.75 627 4 14.79 628 4 14.6 629 4 14.6 630 4 14.54 631 4 14.57 632 4 14.54 633 4 14.37 634 4 14.52 635 4 14.57 636 4 14.56 637 4 14.49 638 4 14.42 639 4 14.39 640 4 14.39 641 4 14.38 642 4 14.38 643 4 14.44 644 4 14.38 645 4 14.54 646 4 14.37 647 4 14.49 648 4 14.71 649 4 14.27 650 4 14.5 651 4 14.51 652 4 14.25 653 4 14.37 654 4 14.7 655 4 14.55 656 4 14.62 657 4 14.42 658 4 14.68 659 4 14.57 660 4 14.63 661 4 14.63 662 4 14.86 663 4 14.89 664 4 14.79 665 4 14.85



Mathematics Grade 8

Scale Score


666 4 14.84 667 4 15.04 668 4 15.03 669 4 15.04 670 4 14.62 671 4 15.33 672 4 14.99 673 4 15.14 674 4 15.05 675 4 15.4 676 4 15.31 677 4 15.08 678 4 15.18 679 4 15.41 680 4 15.49 681 4 15.24 682 4 15.43 683 4 15.23 684 4 15.43 685 4 15.34 686 4 15.59 687 4 15.52 688 4 15.66 689 4 15.45 690 4 15.24 691 4 15.85 692 4 15.86 693 4 15.84 694 4 15.59 695 4 16.19 696 4 15.86 697 4 16.11 698 4 16.23 699 4 16.27 700 4 16.39 701 4 15.92 702 4 16.38 703 4 16.1 704 4 16.5 705 4 16.1 706 4 16.13 707 4 16.95 708 4 16.22 709 4 17.58



Mathematics Grade 8

Scale Score


710 4 16.58 711 4 17.02 712 4 17.01 713 4 16.79 714 4 16.98 715 4 16.46 716 4 16.69 717 4 17.22 718 4 17.47 719 4 17.13 720 4 18 721 4 17.16 722 4 17.04 723 4 16.92 724 4 17.87 725 4 18.32 726 4 17.45 727 4 19.95 728 4 17.13 729 4 17.61 730 4 17.52 731 4 16.95 732 4 18.21 733 4 18.55 734 4 18.87 735 4 18.41 736 4 18.5 737 4 19.32 738 4 18.56 739 4 18.83 741 4 19.18 742 4 18.59 743 4 19.83 744 4 18.24 745 4 21.24 746 4 19.28 747 4 18.1 748 4 18.33 749 4 17.87 750 4 18.5 751 4 19 752 4 22.03 754 4 27.95 755 4 20.28



Mathematics Grade 8

Scale Score


756 4 19.62 757 4 22.19 758 4 20.24 759 4 22.84 763 4 22.39 764 4 26.43 765 4 26.4 768 4 21.58 773 4 26.45 775 4 25.48 777 4 23.32 778 4 24.27 780 4 23.29 781 4 28.58 783 4 27.17 784 4 29.82 789 4 27.71 794 4 30.78 798 4 35.24 799 4 29.18 801 4 30.74 807 4 37.68 808 4 33.17 812 4 38.08 825 4 36.68 828 4 43.33 829 4 47.18 830 4 46.18

Table 13: CSEM at Each Scale Score, Science Grade 5

Science Grade 5

Scale Score


500 1 9.36 505 1 8.74 506 1 8.67 508 1 8.01 509 1 8.7 510 1 8.64 511 1 7.56 512 1 7.91



Science Grade 5

Scale Score


513 1 7.51 514 1 7.32 515 1 7.42 516 1 7.15 517 1 7.24 518 1 6.99 519 1 7.09 520 1 6.76 521 1 6.76 522 1 6.66 523 1 6.57 524 1 6.48 525 1 6.4 526 1 6.41 527 1 6.31 528 1 6.21 529 1 6.18 530 1 6.09 531 1 6.07 532 1 6.04 533 1 5.96 534 1 5.93 535 1 5.9 536 1 5.84 537 2 5.8 538 2 5.76 539 2 5.77 540 2 5.68 541 2 5.72 542 2 5.71 543 2 5.64 544 2 5.65 545 2 5.67 546 2 5.64 547 2 5.64 548 2 5.64 549 2 5.64 550 2 5.65 551 2 5.66 552 2 5.67 553 2 5.67 554 2 5.69 555 3 5.7 556 3 5.74



Science Grade 5

Scale Score


557 3 5.76 558 3 5.78 559 3 5.78 560 3 5.82 561 3 5.87 562 3 5.9 563 3 5.96 564 3 6.03 565 3 6.06 566 3 6.15 567 3 6.15 568 4 6.2 569 4 6.25 570 4 6.29 571 4 6.41 572 4 6.42 573 4 6.55 574 4 6.6 575 4 6.65 576 4 6.74 577 4 6.84 578 4 6.83 579 4 7.06 580 4 7 581 4 7.33 582 4 7.17 583 4 7.3 584 4 7.48 585 4 7.49 586 4 7.73 587 4 7.72 588 4 7.86 589 4 8.25 590 4 8.43 591 4 8.31 592 4 8.85 593 4 8.5 594 4 8.75 595 4 8.64 596 4 9.38 597 4 9.21 598 4 8.86 599 4 8.41 600 4 10.6



Table 14: CSEM at Each Scale Score, Science Grade 8

Science Grade 8

Scale Score


800 1 13.62 805 1 9.05 806 1 8.57 808 1 9.11 809 1 11.08 810 1 8.83 811 1 8.35 812 1 7.9 813 1 9.08 814 1 7.96 815 1 8.38 816 1 7.53 817 1 7.35 818 1 7.41 819 1 7.49 820 1 7.33 821 1 7.15 822 1 7.07 823 1 7.05 824 1 6.9 825 1 6.77 826 1 6.77 827 1 6.6 828 1 6.53 829 1 6.54 830 1 6.47 831 1 6.38 832 1 6.31 833 1 6.24 834 1 6.17 835 1 6.12 836 1 6.13 837 2 6.05 838 2 6.02 839 2 5.95 840 2 5.95 841 2 5.89 842 2 5.85 843 2 5.87 844 2 5.85 845 2 5.83



Science Grade 8

Scale Score


846 2 5.76 847 2 5.77 848 2 5.76 849 2 5.74 850 2 5.74 851 2 5.74 852 2 5.73 853 2 5.73 854 2 5.73 855 3 5.72 856 3 5.73 857 3 5.75 858 3 5.79 859 3 5.77 860 3 5.81 861 3 5.79 862 3 5.8 863 3 5.81 864 3 5.83 865 3 5.85 866 3 5.89 867 4 5.91 868 4 5.96 869 4 6.06 870 4 6.06 871 4 6.06 872 4 6.07 873 4 6.16 874 4 6.19 875 4 6.22 876 4 6.26 877 4 6.33 878 4 6.38 879 4 6.52 880 4 6.5 881 4 6.61 882 4 6.47 883 4 6.76 884 4 6.74 885 4 6.64 886 4 6.81 887 4 7.09 888 4 7.16 889 4 7.48



Science Grade 8

Scale Score


890 4 7.3 891 4 7.1 892 4 7.53 893 4 7.34 894 4 7.68 895 4 7.41 896 4 9.32 897 4 7.73 898 4 8.59 899 4 8.79 900 4 9.19

Appendix C

Classification Accuracy and Consistency Index by Subgroups


Classification Accuracy and Consistency Index by Subgroups C–1 West Virginia Department of Education

Table 1: Classification Accuracy by Demographic Subgroup, ELA

Group N Overall (%) By Cut (%) By Level (%) Cut 1 Cut 2 Cut 3 L1 L2 L3 L4 Grade 3

All Students 18,571 78.78 92.83 91.72 94.21 86.64 75.21 72.21 84.13 Male 9,506 79.10 92.31 91.98 94.80 87.37 74.96 72.08 83.79 Female 9,065 78.44 93.37 91.45 93.60 85.61 75.47 72.34 84.41 Multi-racial 750 78.62 91.75 92.66 94.19 85.35 75.26 73.10 83.08 American Indian / Alaskan Native 8 83.73 88.84 96.79 98.09 83.40 83.24 84.95 #N/A Asian 111 79.88 95.45 92.97 91.42 82.44 80.52 71.96 84.47 Hispanic 380 78.52 91.18 92.54 94.78 87.18 74.07 72.50 81.01 African American 716 79.25 91.56 92.02 95.65 88.23 75.03 71.38 79.08 White 16,584 78.76 92.94 91.64 94.16 86.61 75.22 72.19 84.32 Pacific Islander 11 85.08 99.33 92.44 93.31 98.40 83.66 81.29 86.25 LEP 157 81.78 90.88 93.51 97.37 89.56 77.66 72.60 70.11 Non-LEP 18,414 78.76 92.84 91.70 94.18 86.60 75.19 72.21 84.18

Grade 4 All Students 19,223 77.13 92.57 91.57 92.89 84.95 71.89 67.93 85.29 Male 9,740 77.27 91.89 91.75 93.53 85.51 71.87 67.82 84.57 Female 9,483 76.98 93.26 91.39 92.24 84.13 71.91 68.02 85.84 Multi-racial 770 76.54 91.50 90.85 94.09 84.92 71.71 68.58 84.50 American Indian / Alaskan Native 22 78.21 87.34 95.08 95.62 80.94 76.36 76.26 67.11 Asian 122 79.78 94.84 93.52 91.36 81.55 69.85 66.09 90.58 Hispanic 381 78.37 91.65 92.42 94.21 88.29 71.84 66.49 85.56 African American 783 77.58 90.25 91.81 95.42 84.70 71.84 67.81 85.62 White 17,138 77.09 92.73 91.56 92.70 84.90 71.91 67.94 85.22 Pacific Islander 7 80.15 99.61 87.37 93.10 97.79 60.47 51.47 99.95 LEP 164 79.89 88.48 93.93 97.39 87.77 73.00 67.98 76.08 Non-LEP 19,059 77.10 92.60 91.55 92.85 84.90 71.88 67.93 85.30

Grade 5 All Students 19,557 78.07 92.98 91.53 93.50 87.63 71.12 69.66 85.60 Male 10,074 78.84 92.60 91.97 94.21 88.59 71.38 69.61 85.23 Female 9,483 77.25 93.39 91.06 92.74 86.06 70.86 69.70 85.90



Group N Overall (%) By Cut (%) By Level (%) Cut 1 Cut 2 Cut 3 L1 L2 L3 L4

Multi-racial 753 77.92 93.21 90.70 93.95 88.88 72.00 68.30 83.32 American Indian / Alaskan Native 19 77.23 97.19 89.85 90.17 98.36 72.16 69.06 86.35 Asian 129 80.74 96.32 92.28 92.08 87.72 69.42 70.66 89.58 Hispanic 411 78.41 93.10 90.00 95.24 90.44 69.82 68.22 87.76 African American 732 78.50 90.17 92.73 95.55 87.63 70.85 69.78 82.09 White 17,501 78.03 93.06 91.54 93.36 87.50 71.14 69.72 85.65 Pacific Islander 12 77.83 88.31 96.11 93.36 75.05 79.36 73.40 92.88 LEP 148 84.81 92.60 93.40 98.77 93.74 71.17 70.39 84.87 Non-LEP 19,409 78.02 92.99 91.51 93.46 87.52 71.12 69.65 85.60


Grade 7 All Students 18,843 79.82 92.98 91.87 94.95 87.55 75.42 75.80 83.31 Male 9,718 80.45 92.25 92.53 95.66 88.32 75.34 75.30 82.42 Female 9,125 79.15 93.76 91.18 94.20 86.16 75.50 76.21 83.94 Multi-racial 610 79.54 92.35 91.81 95.36 86.88 76.09 74.11 82.66 American Indian / Alaskan Native 22 81.12 86.53 98.54 96.04 83.97 69.14 82.82 83.97 Asian 120 80.66 95.74 93.66 91.25 82.23 73.22 73.89 89.92 Hispanic 371 79.22 92.20 91.43 95.57 85.36 75.96 76.33 81.95 African American 826 81.22 91.25 92.69 97.27 89.11 75.49 75.68 80.10 White 16,882 79.77 93.09 91.83 94.84 87.54 75.40 75.86 83.28




Pacific Islander 12 79.13 97.86 90.94 90.33 94.81 73.37 72.54 86.34 LEP 107 80.87 88.22 93.46 99.17 87.63 73.68 73.26 99.22 Non-LEP 18,736 79.82 93.01 91.86 94.93 87.55 75.43 75.81 83.30


Table 2: Classification Consistency by Demographic Subgroup, ELA


All Students 18,571 70.32 89.79 88.34 91.81 79.80 65.63 62.37 76.72 Male 9,506 70.77 89.07 88.72 92.63 81.07 65.23 62.22 76.18 Female 9,065 69.84 90.55 87.94 90.96 78.00 66.05 62.51 77.17 Multi-racial 750 70.10 88.52 89.42 91.80 78.83 66.09 62.92 74.44 American Indian / Alaskan Native 8 73.25 82.23 93.97 96.48 75.46 70.44 80.12 7.81 Asian 111 72.40 93.47 90.02 88.56 79.15 61.97 62.94 83.49 Hispanic 380 70.07 87.62 89.34 92.73 80.11 64.77 62.64 72.49 African American 716 70.96 87.93 88.87 93.83 83.14 65.27 61.67 66.28 White 16,584 70.28 89.95 88.23 91.73 79.62 65.66 62.34 77.02







Grade 6 All Students 19,608 70.16 89.67 87.88 92.31 79.04 64.12 67.27 73.79




Male 10,119 70.87 88.19 88.81 93.57 80.84 63.80 66.74 72.75 Female 9,489 69.40 91.25 86.89 90.96 75.79 64.46 67.71 74.53 Multi-racial 745 70.17 89.37 86.86 93.62 79.23 64.33 68.13 72.06 American Indian / Alaskan Native 11 70.23 89.28 88.47 92.24 80.61 68.62 56.26 75.07 Asian 137 73.22 95.21 92.91 84.89 80.95 62.29 65.71 81.06 Hispanic 389 70.51 88.61 89.23 92.34 76.65 65.05 69.55 73.86 African American 790 71.47 87.36 88.83 95.00 82.42 64.10 64.91 70.99 White 17,525 70.07 89.77 87.81 92.19 78.86 64.09 67.29 73.77 Pacific Islander 9 62.31 90.99 89.69 81.49 78.25 63.91 51.32 60.24 LEP 122 73.41 82.08 91.78 99.30 83.00 63.08 57.96 82.21 Non-LEP 19,486 70.14 89.72 87.86 92.26 78.99 64.12 67.29 73.78


Grade 8 All Students 19,058 71.81 90.62 88.77 92.21 81.76 67.23 66.15 74.69 Male 9,961 73.15 89.49 89.75 93.72 84.12 66.99 65.78 73.58 Female 9,097 70.34 91.85 87.69 90.56 76.91 67.49 66.44 75.44 Multi-racial 530 71.85 89.47 89.81 92.34 79.45 69.09 66.52 74.34 American Indian / Alaskan Native 16 71.53 88.49 86.42 96.55 87.57 48.92 55.55 83.01 Asian 137 76.33 97.09 92.02 87.06 86.46 61.20 67.96 84.97




Hispanic 386 72.25 89.65 89.25 93.13 80.58 70.12 63.73 76.18 African American 835 72.62 87.22 89.90 95.28 82.39 65.93 67.32 70.93 White 17,146 71.73 90.79 88.64 92.08 81.80 67.20 66.13 74.53 Pacific Islander 8 62.52 87.20 91.06 84.08 47.67 66.02 58.43 69.79 LEP 101 75.77 84.22 92.62 98.72 84.51 68.29 63.72 48.42 Non-LEP 18,957 71.79 90.65 88.75 92.18 81.73 67.22 66.15 74.70

Table 3: Classification Accuracy by Demographic Subgroup, Mathematics


All Students 18,576 81.04 94.36 92.60 94.06 88.50 76.24 73.65 88.10 Male 9,511 81.32 94.73 92.64 93.95 88.94 76.45 73.49 88.44 Female 9,065 80.73 93.98 92.55 94.19 88.07 76.04 73.81 87.70 Multi-racial 751 80.88 93.88 92.23 94.76 88.96 75.78 72.68 88.63 American Indian / Alaskan Native 8 87.22 94.51 95.47 97.24 92.07 86.53 74.22 100.00 Asian 112 87.64 98.30 96.46 92.88 95.93 79.10 75.73 92.58 Hispanic 380 81.78 92.68 93.69 95.39 89.25 76.42 73.90 87.28 African American 716 80.63 92.09 92.48 96.05 88.57 75.00 73.86 84.48 White 16,587 80.99 94.49 92.56 93.93 88.43 76.31 73.67 88.08 Pacific Islander 11 81.62 94.43 96.98 90.21 82.34 76.50 71.56 88.58 LEP 158 82.73 92.75 94.26 95.71 92.90 75.56 73.07 87.92 Non-LEP 18,418 81.02 94.37 92.58 94.05 88.45 76.25 73.65 88.10

Grade 4 All Students 19,218 80.13 94.13 92.06 93.89 88.03 77.97 67.38 87.90 Male 9,740 80.74 94.24 92.47 93.98 88.63 77.72 67.53 88.44 Female 9,478 79.51 94.02 91.63 93.80 87.32 78.21 67.23 87.27 Multi-racial 769 80.38 92.94 92.32 95.07 87.35 78.27 68.58 87.58 American Indian / Alaskan Native 22 81.84 91.56 91.90 98.36 86.13 80.93 71.42 97.01 Asian 122 83.83 98.86 94.25 90.68 87.84 80.97 66.31 91.49




Hispanic 381 81.40 92.76 93.58 95.02 90.18 78.43 67.02 88.67 African American 784 80.68 91.95 92.83 95.85 88.26 77.51 67.35 87.54 White 17,133 80.04 94.28 91.96 93.74 87.98 77.96 67.35 87.85 Pacific Islander 7 78.79 98.72 85.99 94.05 95.52 #N/A 50.29 86.64 LEP 164 81.90 92.64 93.59 95.64 93.06 77.19 66.23 85.77 Non-LEP 19,054 80.12 94.14 92.05 93.88 87.94 77.98 67.38 87.90


Grade 6 All Students 19,602 80.83 92.24 92.90 95.65 88.93 75.76 70.34 86.88 Male 10,117 81.52 92.29 93.34 95.85 89.69 75.81 70.41 86.71 Female 9,485 80.10 92.19 92.43 95.44 87.95 75.70 70.28 87.04 Multi-racial 747 81.35 91.62 92.94 96.75 88.59 75.61 71.20 85.45 American Indian / Alaskan Native 11 81.13 97.95 90.48 92.66 98.66 70.56 72.00 96.74 Asian 137 85.74 96.00 94.88 94.85 84.99 77.46 73.90 95.07 Hispanic 388 80.01 91.00 93.18 95.78 87.30 75.54 69.17 82.45 African American 794 83.28 91.03 94.81 97.40 90.39 75.64 71.18 85.40 White 17,514 80.68 92.31 92.79 95.54 88.90 75.77 70.27 86.87 Pacific Islander 9 72.33 93.07 93.62 85.60 79.68 81.51 66.99 64.73 LEP 122 84.38 89.01 95.62 99.71 88.91 77.53 67.85 99.60




Non-LEP 19,480 80.81 92.26 92.88 95.63 88.93 75.75 70.35 86.86 Grade 7

All Students 18,856 79.95 92.13 92.58 95.17 88.30 74.04 68.85 86.60 Male 9,726 80.42 92.30 92.76 95.28 88.76 74.19 68.85 86.93 Female 9,130 79.46 91.95 92.39 95.05 87.77 73.89 68.86 86.24 Multi-racial 611 80.85 91.09 93.00 96.69 87.88 74.75 69.24 86.74 American Indian / Alaskan Native 22 83.84 94.70 93.42 95.64 92.51 70.09 68.92 90.24 Asian 120 86.81 97.99 94.72 94.08 96.39 75.89 70.16 93.15 Hispanic 370 79.03 90.36 93.26 95.33 88.58 73.90 68.36 77.09 African American 826 82.08 90.70 94.18 97.09 89.51 74.46 68.73 80.27 White 16,895 79.78 92.23 92.46 95.03 88.19 73.98 68.86 86.66 Pacific Islander 12 80.43 96.98 90.70 92.74 #N/A 84.76 57.17 99.35 LEP 107 83.77 90.26 95.08 98.35 91.23 73.14 63.49 86.97 Non-LEP 18,749 79.93 92.14 92.57 95.15 88.27 74.04 68.86 86.60

Grade 8 All Students 19,069 80.75 92.74 93.07 94.76 89.13 76.79 60.69 88.28 Male 9,976 82.01 92.68 93.76 95.42 90.08 76.94 60.63 89.35 Female 9,093 79.36 92.81 92.32 94.04 87.75 76.66 60.75 87.20 Multi-racial 529 81.05 92.94 92.84 95.11 90.42 78.08 60.55 85.71 American Indian / Alaskan Native 16 81.00 93.62 92.74 94.35 90.33 69.47 57.96 76.04 Asian 137 87.25 97.86 96.21 92.98 93.00 77.70 59.37 92.84 Hispanic 387 79.96 91.17 93.02 95.60 88.50 75.58 60.46 85.71 African American 836 81.96 90.15 94.91 96.77 89.56 77.27 59.12 84.06 White 17,156 80.65 92.86 92.97 94.65 89.06 76.76 60.76 88.39 Pacific Islander 8 76.65 85.83 95.08 95.42 79.50 69.80 #N/A 82.66 LEP 102 83.17 89.20 96.25 97.65 91.49 73.20 61.08 85.43 Non-LEP 18,967 80.74 92.76 93.06 94.75 89.11 76.81 60.69 88.28



Table 4: Classification Consistency by Demographic Subgroup, Mathematics


All Students 18,576 73.47 92.06 89.57 91.61 81.72 67.51 63.87 82.97 Male 9,511 73.89 92.54 89.66 91.45 82.40 67.54 63.85 83.52 Female 9,065 73.04 91.55 89.48 91.78 81.05 67.48 63.89 82.32 Multi-racial 751 73.34 91.40 89.10 92.61 82.37 67.74 62.25 83.05 American Indian / Alaskan Native 8 79.97 91.09 93.03 95.64 87.98 72.31 71.30 85.72 Asian 112 82.32 97.37 94.76 90.02 84.10 73.38 62.25 91.29 Hispanic 380 74.31 89.68 90.98 93.42 82.70 67.67 63.18 83.33 African American 716 73.03 88.87 89.52 94.43 84.13 65.83 63.48 76.67 White 16,587 73.41 92.24 89.51 91.42 81.48 67.55 63.98 82.94 Pacific Islander 11 74.42 92.41 94.86 87.02 66.83 72.65 56.46 85.53 LEP 158 75.66 89.78 91.65 94.02 85.60 68.48 63.04 81.64 Non-LEP 18,418 73.45 92.08 89.55 91.59 81.67 67.50 63.88 82.97


Grade 5 All Students 19,571 73.00 90.15 89.67 92.74 82.02 69.50 56.74 82.10 Male 10,082 73.88 90.48 90.18 92.79 83.45 69.00 56.75 83.15 Female 9,489 72.06 89.81 89.12 92.68 80.32 69.98 56.72 80.85




Multi-racial 754 72.16 88.04 89.91 93.80 81.00 67.71 56.31 81.49 American Indian / Alaskan Native 19 74.38 95.62 84.98 93.06 89.70 70.21 61.42 82.34 Asian 130 79.15 95.12 94.17 89.50 78.37 66.62 54.98 90.52 Hispanic 411 73.67 89.65 88.83 94.76 84.81 69.70 56.88 78.95 African American 734 75.12 88.04 91.64 95.08 85.54 69.55 57.16 76.97 White 17,511 72.89 90.30 89.57 92.57 81.77 69.59 56.74 82.12 Pacific Islander 12 58.81 88.25 83.09 86.76 61.02 61.62 58.68 48.40 LEP 149 78.13 87.09 94.18 96.59 87.87 66.17 57.51 80.85 Non-LEP 19,422 72.96 90.18 89.63 92.71 81.93 69.52 56.73 82.10


Grade 7 All Students 18,856 71.94 88.69 89.49 93.19 83.21 63.98 57.30 80.19 Male 9,726 72.53 88.92 89.71 93.33 84.05 63.81 57.32 80.74 Female 9,130 71.31 88.44 89.26 93.03 82.25 64.15 57.27 79.61 Multi-racial 611 72.87 87.08 90.15 95.10 83.55 64.30 55.75 81.88 American Indian / Alaskan Native 22 76.61 91.23 91.56 93.29 90.83 55.65 59.53 78.14 Asian 120 81.20 96.93 92.28 91.64 84.23 68.41 54.79 92.57 Hispanic 370 70.99 86.24 90.42 93.79 82.65 64.67 56.48 70.49 African American 826 74.43 86.48 91.59 95.83 86.82 62.60 55.20 68.64 White 16,895 71.73 88.85 89.33 92.99 82.92 64.00 57.44 80.19








Table 3: Classification Accuracy by Demographic Subgroup, Science


All Students 19,539 75 91 90 94 82 76 66 77 Male 10,060 75 91 90 94 83 76 66 78 Female 9,479 74 91 89 94 81 76 66 77 Multi-racial 751 76 90 91 95 82 77 67 77 American Indian / Alaskan Native 19 71 92 85 94 84 71 62 52 Asian 129 74 95 90 89 82 75 65 79 Hispanic 411 76 90 90 96 83 75 67 82 African American 725 76 86 93 98 83 76 64 71 White 17,492 75 91 89 94 82 76 66 77 Pacific Islander 12 72 94 83 95 83 69 70 73 LEP 146 79 85 95 98 87 73 64 67 Non-LEP 19,393 75 91 90 94 82 76 66 77

Grade 8 All Students 19,040 76 90 91 95 83 76 64 80 Male 9,960 76 89 92 95 83 76 64 81 Female 9,080 75 90 90 94 82 77 64 79 Multi-racial 529 76 89 92 95 84 76 63 81 American Indian / Alaskan Native 16 79 85 95 98 77 77 73 94 Asian 137 79 96 91 92 83 77 65 91 Hispanic 386 76 89 92 95 83 76 64 77 African American 834 77 86 94 97 85 75 63 80 White 17,130 76 90 91 95 82 76 64 80 Pacific Islander 8 71 91 85 95 76 68 74 71 LEP 100 77 83 96 98 83 74 61 63 Non-LEP 18,940 76 90 91 95 83 76 64 80



Table 4: Classification Consistency by Demographic Subgroup, Science


All Students 19,539 66 87 85 92 70 69 55 65 Male 10,060 66 87 86 92 72 69 55 67 Female 9,479 65 87 85 92 68 69 55 63 Multi-racial 751 66 86 87 93 71 70 56 61 American Indian / Alaskan Native 19 62 91 79 92 74 69 44 41 Asian 129 65 92 86 85 70 64 57 72 Hispanic 411 66 86 85 95 70 69 55 67 African American 725 68 81 90 97 74 68 52 57 White 17,492 65 88 85 92 70 69 55 65 Pacific Islander 12 62 91 77 93 73 62 50 74 LEP 146 71 80 93 98 79 67 47 60 Non-LEP 19,393 66 87 85 92 70 69 55 65

Grade 8 All Students 19,040 67 86 88 93 73 68 53 70 Male 9,960 68 85 89 93 75 67 54 72 Female 9,080 66 86 86 92 70 69 53 68 Multi-racial 529 67 85 89 93 76 67 53 68 American Indian / Alaskan Native 16 69 80 92 97 74 62 59 87 Asian 137 71 94 88 89 71 65 57 85 Hispanic 386 67 84 89 94 74 69 52 67 African American 834 69 81 91 96 77 66 52 65 White 17,130 67 86 88 92 73 68 54 70 Pacific Islander 8 59 87 80 92 69 60 49 63 LEP 100 69 77 94 98 77 65 44 48 Non-LEP 18,940 67 86 88 93 73 68 53 70

Date post:	03-Nov-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

West Virginia General Summative Assessment 2018–2019 ......in grades 3–8 in ELA and mathematics....

Documents