PARCC Bias and Sensitivity Review

August 2012

Coral Gables

1Confidential and Proprietary.

Points of Discussion

Purpose of Bias and Sensitivity Reviews

Identifying Bias and Stereotypes

Sensitivity Awareness and Detecting Passage Bias

Guiding Questions and Sample Items

2Confidential and Proprietary.

Purpose of Bias and Sensitivity Review

Review test materials for potential sources of bias and stereotypes.

Apply professional test development standards to ensure materials are fair and not insensitive or offensive.

3Confidential and Proprietary.

What Is Bias?

Language or content that prevents members of a group from demonstrating they possess the knowledge and skills being measured.

Language or content that advantages members of a group in demonstrating they possess the knowledge and skills being measured.

4Confidential and Proprietary.

Common Forms of Bias and Stereotypes

Regional and geographic bias Gender and age stereotypes Ethnic, cultural, and religious stereotypes Socioeconomic and occupational

stereotypes

5Confidential and Proprietary.

Guideline #1

Avoid Cognitive Sources of Construct-Irrelevant Variance

6Confidential and Proprietary.

Some Background

Construct = knowledge, skill, or other attribute (KSA) you are trying to testConstruct-relevant = related to KSA that you are trying to testConstruct-irrelevant = KSA that is not related to what you are trying to testVariance = differences in test scores

7Confidential and Proprietary.

Construct-irrelevant variance lowers validity.

Construct-irrelevant variance that has different

effects across groups lowers fairness.

Construct-irrelevant differences across groups

decrease validity & fairness.

Construct-relevant differences across groups are

valid and, therefore, fair.

Variance, Validity & Fairness

8Confidential and Proprietary.

Avoid Affective Sources of Construct-Irrelevant Variance

Guideline #2

9Confidential and Proprietary.

Sensitivity Awareness

Any reference or language in an item or passage that might cause a student to

have an emotional reaction during the test administration can prevent a

student from being able to accurately demonstrate ability.

Note: The emotional factor is not limited to negative emotions, but can also

include a “giggle factor” in items.

10Confidential and Proprietary.

Avoid Physical Sources of Construct-Irrelevant Variance

Guideline #3

11Confidential and Proprietary.

Visual stimuli in the middle of paragraphs

Decorative rather than informative illustrations

Fonts that are hard to read

Letters that look alike (e.g., O, Q) used as labels for dif-ferent things in the same item/stimulus

Samples of Barriers

12Confidential and Proprietary.

How Might Item Bias be Detected?

Judgmental Procedure• Bias and sensitivity review prior to field-testing

Statistical Procedure• DIF (Differential Item Functioning) analysis following field-

tests and operational administrations

13Confidential and Proprietary.

Guiding Questions for Passages

Does the passage disadvantage any population (gender, race, ethnicity, language, religion, socioeconomic status, disability or geographic region) for non-educationally relevant

reasons?

Does the passage contain controversial or emotionally charged subject matter that is not supported by the Common Core State Standards?

Is the passage potentially offensive, demeaning, insensitive, or negative toward any population?

Does the passage depict any population in a stereotypical manner?

14Confidential and Proprietary.

Sample Passage #1

Why You Get HungryNot hungry? Don’t worry…you will be. Just a little while from now, you'll begin to feel something just under your rib cage. It may seem like a little pressure or warmth. Some people think it feels like hollowness. …You’ve been getting hungry every few hours since the day you were born. Isn't it about time to understand what’s going on? Don’t blame an empty stomach. Early in the twentieth century, doctors noticed something interesting about patients who had their stomachs surgically removed. The patents still felt hunger the same old way. So doctors and scientists decided the feeling of hunger must come from somewhere else. It turned out to be a tiny area, about the size of a garbanzo bean, in the middle of your brain. It’s called the hypothalamus (hi-po-THAL-a-mus). It constantly monitors what’s going on in your bloodstream and sends reports to different parts of your brain. Blood’s too warm? Make ‘em sweat. Blood’s too cool? Make ‘em shiver. Not enough fuel in the blood? Make ‘em eat. How? By making ‘em hungry. It’s your brain creating that unmistakable feeling called hunger. Hunger is powerful, and it’s meant to be unpleasant. Once it starts, there’s really nothing you can do to relieve it except eat. And so you eat—usually until a second part of your hypothalamus tells your brain that the amount of fuel in your blood is just right …

Fairness Guideline 1, 2, or 3? 15Confidential and Proprietary.

Sample Passage #2

The Dog Who Wouldn’t BeMy third class that morning happened to be French. The teacher was a desiccated female whose spiritual home may have been Paris, but who had never actually been farther east than Winnipeg. She was affected, humorless, and a tyrant. None of us liked her. Yet I actually felt sorry for her when, in the midst of the declination of an irregular verb, Wol whumped moodily in through the second-story window and slid to an unsatisfactory halt upon the top of her hardwood desk. The exclamation with which she greeted him was given in very old Anglo-Saxon, without even a hint of a French accent. I had an interview with the principal after this incident, but he was a reasonable man and the upshot was that I escaped corporal punishment on the understanding that my owl would stay at home in future. I achieved this end, but only at the cost of giving the owls the free run of our house. Some ten minutes before I was due to leave for school I would invite Weeps and Wol into the kitchen, where they were allowed to finish off the bacon scraps left from our breakfast. Apparently Wol accepted this as a sufficient bribe, for he made no further attempt to follow me to school; and Weeps, who always accepted Wol’s lead gave me no more trouble either. …

Fairness Guideline 1, 2, or 3?16Confidential and Proprietary.

Sample Passage #3

Let’s discuss

Tiny Houses: a tiny trend that runs against the mainstreamIn 2008, Ben and Sarina Speed of Maine challenged themselves to build and live in the smallest house possible—one that would accommodate only their most fundamental needs along with the needs of their young children and two cats. Among the reasons for downsizing was their conviction that smaller houses are environmentally friendly because they consume less energy. Indeed, the couple reports that now the electric bill for their 640 square foot home, which is smaller than some garages, averages $20 per month.The Speeds have joined an increasing number of Americans committed to shrinking their residential footprints and purging their lives of excesses they have come to view as burdensome, indulgent, harmful to the environment, or all of the above. How small is small? Jay Shafer, the co-founder of the Small House Society, says designs can range in size from 65 square feet to 874 square feet. Even the high end of the range marks a substantial reduction from the average sized home. Of course, not everyone approves of this philosophy. Supporters and opponents of tiny homes—also called eco-homes, micro-homes, or mini-homes—tend to be resolute in their positions.That tiny houses use less energy and are therefore environmentally friendly is indisputable; the smaller the area, the less energy required for heating and cooling…

17Confidential and Proprietary.

Sample Passage #4

A Treasury of Western FolkloreUpon the 10th of May, 1869, the rival roads approached each other, and two lengths of rails were left for the day’s work. At 8 A.M., spectators began to arrive; at quarter to 9 A.M., the whistle of the Central Pacific Railroad is heard, and the first train arrives, bringing a large number of passengers. Then two additional trains arrive on the Union Pacific Railroad, from the East. At a quarter of 11 A.M., the Chinese workmen commenced leveling the bed of the road with picks and shovels, preparatory to placing the ties. At a quarter past eleven the Governor's (Governor Stanford’s) train arrived. The engine was gaily decorated with little flags and ribbons—the red, white, and blue. The last tie is put in place—eight feet long, eight inches wide, and six inches thick. It was made of California laurel, finely polished, and ornamented with a silver escutcheon, bearing the following inscription:

THE LAST TIE LAID ON THE PACIFIC RAILROAD, MAY 10, 1869. …In San Francisco, the wires were connected with the fire-alarm in the tower, where the heavy ring of the bell might spread the news immediately over the city, as quick as the event was completed. Waiting for some time in impatience, at last came this message from Promontory Point, at 2:27 P.M.

18Confidential and Proprietary.

Almost ready. Hats off, prayer is being offered.A silence for the prayer ensued; at 2:40 P.M., the bell tapped again, and the officer at

Promontory said:

We have got done praying, the spike is about to be presented. …

Sample Passage #4, Continued

Let’s discuss

Questions?

20Confidential and Proprietary.

How To Proceed

Read through a specific set of test materials, keeping in mind the guiding questions. Do not review these materials from the point of view

of a content expert, instead review them only from the perspective of fairness.

If possible sources of bias, stereotypes, or sensitivity are detected, note them in your comments, and then they will be discussed by the group.

The recorder will take complete and precise notes for the group.

21Confidential and Proprietary.

Botkin, B. A. . A Treasury of Western Folklore. New York: Crown Publishers, 1951

Haduch, Bill. Food Rules! New York: Puffin, 2001Mowat, Farley. The Dog Who Wouldn’t Be. Ontario: Little Brown

& Co, 1957

Passage Sources

22Confidential and Proprietary.