Presenting School Choice Information to Parents: An Evidence … · 2018-10-29 · Presenting...

Presenting School Choice Information to Parents: An Evidence-Based Guide

NCEE 2019-4003

Presenting School Choice Information to Parents: An Evidence-Based Guide 2

Choosing a school for a child can be a rewarding but difficult task for parents.1 A recent survey reported that parents often feel confused by the information available to them as part of the school choice process.2 This can be exacer-bated by limited time and resources to devote to the process—constraints that can particularly affect low-income and otherwise disadvantaged families.3 If confusion results in uninformed decision-making then parents might not select schools that best fit their children’s needs and the possible benefits of school choice could be undermined.

Understanding how information can influence parents’ school choice behavior is important now, more than ever. In many parts of the country, opportunities to choose a school are increasing. Nationwide, the number of charter schools, magnet schools, and private school voucher programs has risen in recent years, as have open enrollment policies that allow parents to choose from among traditional public schools.4 For example, over the most recent five years for which data are available, the number of magnet schools increased by almost 50 percent and the number of charter schools grew by more than a third (see Figure 1).

There is also new demand for making school information available to families. The federal Every Student Succeeds Act (ESSA) passed in 2015 requires states and districts to provide public report cards that include information on state, district, and school performance and progress in an “understandable” and “user-friendly” manner.5 These report cards are likely to build on the school choice directories (online, on paper, or both) that many districts, states, and other education organizations already offer to help families navigate their options.

INTRODUCTION

0

2000

2009-10 2010-11 2011-12 2012-13 2013-14 2014-15

4000

6000

8000

10000

2,2132,722

2,949

4,952 5,274 5,696

3,1513,254 3,285

6,079 6,465 6,747

Charter SchoolsMagnet Schools

Num

ber o

f Sch

ools

Source: National Center for Education Statistics, Common Core of Data, Public Elementary/Secondary School Universe Survey.

Figure 1. Number of Charter and Magnet Schools Has Grown Over Time


Currently, how information is provided in school directories or report cards varies in both substance and style.6 For example, some provide lengthy and detailed profiles for each school, while others opt for more concise and modest profiles. Some present information primarily with numbers, while others use graphs or icons such as color-coded letter grades. Research in fields outside of education indicates that the way information is presented affects deci-sion making.7 But little is currently known about how those results apply to school choice information and whether different strategies might help or hinder parents as they go through the choice process. The U.S. Department of Education’s guidance on ESSA report cards suggests there is more to be learned about best practices for displaying school information and that obtaining feedback from parents is essential.8

The Department, which also financially supports school choice programs through initiatives such as the Public Charter Schools Program and the Magnet Schools Assistance Program, seeks to improve the evidence on how to present information about schools to the public. This evidence-based guide explores how presentation decisions can affect parents’ understanding, ease of use, and satisfaction with the school information provided, and can affect how parents rank their school choices. The evidence that informs the guide comes from a rigorous experiment conducted in late 2016 for the Department’s Institute of Education Sciences.9

This guide is a resource for those considering how to present information to parents (“information providers”). The recommendations in the guide are “evidence-based” because they emerged from a randomized study that is described below. The intended audience for the guide includes, for example, education officials responsible for deciding which information about schools to release publicly, along with designers responsible for translating that information into carefully formatted materials. Some information providers work in public agencies, like school districts or state departments of education, but many others work in nonprofit organizations or other private companies. The purpose of this guide is to help information providers—whoever and wherever they are—incorporate evidence into the many decisions involved in presenting information about schools. Table 1 explains some of the terms used in this guide.

Table 1. Terms Used in this Guide

Information display The full display that parents see when they visit a school choice website. For the study, this included, for example, a map, glossary of terms, and 16 individual school profiles.

School profile The part of an information display containing information about a single school. The school profiles appear as rows in a table.

Measure A specific piece of information about a school. Measures were grouped into four different “categories of information” (distance from home, academics, safety, and resources). For example, the number of laptops or tablets per 100 students is a measure within the resources category of information.

Factor An aspect of the information displays that the research team varied such that different study participants saw different displays. The study examined five factors. For example, the “amount of information” factor varied how much information about each school was shown.

Strategy One way of implementing a particular factor. For example, the “amount of information” factor tested three strategies: lower amount, higher amount, and progressive disclosure (where parents initially viewed a lower amount but had the option to click a link that revealed more information).

Variation of an information display

Any specific version of an information display used in the study. The study tested 72 variations, because there were 72 possible combinations of strategies across the five factors.

Note: Terms in bold are defined in this table.


For the study, researchers worked with a firm specializing in presenting education data to create an online “information display” of individual “school profiles” for a hypothetical school district. Parents who participated in the study were each randomly assigned to review one variation of the information display—as if they were considering elementary school options for their youngest child—and then answered questions about the display and the schools it described. This type of experiment is commonly used for market research and by behavioral and social scientists to examine the effects of specific variations (e.g., color of a product, wording of instructions, etc.) in controlled environments. For this study, the research team compared responses to the questions for parents who saw different information displays to determine how the design of the displays affected key outcomes: parents’ understanding of the school information, their satisfaction with the display and its perceived ease of use, and the choices they made when ranking the schools.

Figure 2 illustrates each step of the study and is followed by a description of how the study worked and what specifically was tested. A more detailed discussion of the study and its methods appears in the technical appendix, Section B.1.

THE STUDY AND HOW IT WORKED

Figure 2. How the Study Worked

Study team recruits low-income parents

Study team randomly assigns parents to 1 of 72 displays of school information (”information displays”)

Study team compares outcomes across parents viewing dif ferent displays to see which display strategies are best

(A) Parents complete initial survey• Answer questions to confirm study eligibility

• Provide background information

(C) Parents complete outcomes survey• Rank top 3 preferred schools for their child

• Take test to demonstrate “understanding” of display

• Rate ease of use and satisfaction with display

(B) Parents view school information display including profiles for 16 hypothetical schools


Recruitment and Participants

Parents were recruited by a market research firm that maintains a database of potential survey respondents. Partici-pants received modest compensation from the firm if they were eligible for and participated in this particular study.

Those interested in participating began by completing an initial survey to determine if they met eligibility require-ments, which included living with at least one school-age child (aged 3 to 18) for whom they helped make education related decisions. To participate parents also had to be low-income (annual household income of $40,000 or less) because research indicates that these parents face particular challenges related to accessing and utilizing information about schools. Also, many of today’s school choice programs specifically target low-income families.10

In total, 3,500 parents from across all 50 states participated in the study and are represented, anonymously, in the study’s findings. In addition to household income, the initial survey gathered information about these parents’ educa-tional attainment, Internet use, and prior experiences with choosing schools.11 This enabled researchers to determine if particularly disadvantaged parents, those familiar with online information displays, or those with school choice experience tended to respond to school information displays differently from other groups.

Viewing the Information Displays

After completing the initial survey, parents viewed an information display with profiles of 16 schools from a hypothetical district. The schools were designed to differ from one another with respect to their distance from the parent’s hypothetical home and the infor-mation reported about school academics, safety, and resources (such as number of laptops). The research team generated this hypothet-ical information to create trade-offs between schools—for example, the school located closest to home also had relatively poor academic performance, low safety, and few resources compared to other schools. Parents were asked to imagine that they were selecting an elementary school for their youngest child12 as they examined the profiles and used them to compare schools, because the study’s information display included only elementary schools.

An important aspect of this study is that each parent was assigned, at random,13 to see one of 72 variations of the school information display (see next section). However, there were four features common to all information displays (as shown in Figure 3). They are:

1. a glossary that defines key terms such as “proficiency rate” that may be unfamiliar to some parents;

2. a map with a house symbol representing the parent’s hypothetical home and markers representing the location of each school;

3. 16 school profiles with information about each school’s distance from home, academics, safety, and resources which are information categories that parents report caring about14,15; and

4. a simple, standard data layout that made it possible to scan up or down and compare the 16 different schools on each information category.


Figure 3. Features Common to all Information Displays

The features noted above were adopted because they are consistent with beliefs about best design practices from the research literature16 and those recommended by the study’s design firm, a company with experience creating school information displays for a variety of state and district customers. (See the appendix for further discussion of why information providers might consider these features when developing their own materials.) The goal was to avoid any designs that are known to hinder understanding, satisfaction, and ease of use (outcomes discussed in more detail below) and instead focus on testing potentially promising strategies against one another.

Outcomes

After parents viewed the specific information display that they had been randomly assigned to see, they began the “outcomes survey.” (The school information display remained visible, so parents could refer to it as needed.) The study varied what parents saw in order to measure the effects of any one strategy on understandability, ease of use, satisfaction, and choices made by parents. These outcomes are important to consider when evaluating school information displays, according to both ESSA report card guidance and prior school choice research.17 The outcomes were defined and measured as follows:

Understanding (comprehension of the information provided). To assess parents’ understanding

of the school profiles, the survey asked six factual (right-or-wrong) questions that required

parents to refer to information in the school profiles. Parents could consult the profiles

while answering the questions. The researchers assigned points for correct answers and

calculated the percentage of total possible points that each parent earned. A higher

percentage reflects greater understanding.

Example: The survey asked parents which schools, from a list, are within one mile

of home and have at least 50 laptops or tablets per 100 students.


Ease of use (self-reported ability to use the display to learn about schools). To assess

parents’ attitudes about ease of use, six survey items asked parents how much they agreed

or disagreed with statements about the ease or difficulty of using the profiles. Responses

were combined into an ease-of-use index that captured the percentage of statements with

which parents agreed or strongly agreed. A higher percentage reflects greater ease of use.

Example: The survey asked parents whether they would need someone to help them in

order to use the school profiles effectively.

Satisfaction (extent to which parents reported liking the display). To assess parents’

satisfaction, four survey items asked parents how much they agreed or disagreed with

statements about their satisfaction with the profiles. Responses were combined into a

satisfaction index that captured the percentage of statements with which

parents agreed or strongly agreed. A higher percentage reflects greater satisfaction.

Example: The survey asked parents if they would like to use these information

displays to choose a school for their own children.

Choices (decisions based on the information in the display). To measure the effect of

information on the types of schools chosen, the survey included a school-ranking exercise.

After reviewing the profiles, parents completed a two-step process to rank their preferred

schools as if they were choosing a school for their youngest child. First, they selected their

top three schools from the 16 schools listed. Then, they ranked those three schools from

most to least preferred. By looking at the characteristics of the schools selected, researchers

calculated how much weight parents gave to certain categories of school information, such

as distance from home and academic performance, when making school selections.

In order to determine how variations of the information display and specific display strategies led to different outcomes, the study team compared outcomes for parents who saw any given strategy to outcomes for parents who were assigned to see a different strategy. Random assignment of parents to different information displays ensures that differences in average outcomes can be attributed to the differences in information display strategies. A more detailed descrip-tion of how outcomes were analyzed is in the technical appendix, Section B. This guide only reports on results with a high probability, given the data, that one strategy was better than the other(s).18


WHAT THE STUDY TESTEDCreating a school information display requires an information provider to make many more decisions than any single study could examine. The research team, with guidance from a diverse group of practitioners and scholars, chose to investigate a set of potentially consequential decisions where little is known about how design choices relate to the study’s outcomes of interest. Specifically, the research team varied five aspects of the information displays (called “factors”), testing two or three versions of each (called “strategies”), for a total of 72 variations. Figure 4 shows an example of an information display from the study and highlights the factors that were varied. Additional examples of all the strategies are presented later in the guide along with more detail concerning why the factors and strategies that were tested might affect outcomes of interest.

Figure 4. Features of the Information Display that Varied in the Study

Note: Each parent was assigned at random to view one variation of the information display, each of which represented a unique combination of five strategies tested in the study. For example, a parent who saw a display like the one shown above was assigned to see the following strategies: numbers plus graphs, district averages, official district-provided information plus parent ratings, progressive disclosure, and a default sort order based on the distance from home. This is one of the 72 variations tested in the study. 19


ORGANIZATION OF THE GUIDEThe five factors along with the specific information display strategies that were varied for each (see Figure 4) were chosen to allow the study to answer six questions. The first five questions, each of which focuses on a single factor, ask which strategies make school information displays most understandable, easy to use, and satisfying to parents. The sixth question, which looks across all five factors, asks how design decisions might affect the school choices parents make.

The next section of this guide—“Study Findings and Implications”— addresses each of the six questions, explaining why each question was important to answer and what the study found. That section is followed by a discussion— “Key Insights From the Study”—of the researchers’ interpretations of the results. The final section discusses some reasons why the results and implications should not be viewed as definitive. In particular, they come from a single study, conducted online, where parents were asked to consider 16 hypothetical elementary schools. Findings may be different in real-world settings with parents deciding where to enroll children in school, especially for parents who might have more (or fewer) than 16 schools to choose from or may be choosing middle or high schools.

Additional details about the study’s sample, methods, and findings appear in the accompanying technical appendix.


STUDY FINDINGS AND IMPLICATIONS

Question 1 Which FORMAT for displaying data on schools is best— numbers only, numbers plus icons, or numbers plus graphs?

Why and how format was tested

One of the most important issues information providers have to grapple with is how to present numerical data—such as how many students in a school tested as proficient on a state assessment. Not only does ESSA require that report cards include a variety of achievement performance and progress indicators (in an understandable and user-friendly way) but also parents report that such information, along with information about school safety, is what they care about most.20

There are multiple options for displaying numerical data, from presenting numbers and percentages on their own to adding visual elements such as graphs or charts to creating icons such as letter grades (A-F) or color-coded symbols. Research outside of education indicates that graphs and icons can help people organize and interpret information21 and indeed there are districts and third party information providers using these approaches.22 However, it is not actually known if, in the context of school information, alternative formats make numerical data easier to understand, are preferred by parents, and are worth the effort required to create them.

The study tested three format strategies:23,24

Numbers Only

Numbers + Graphs

Note: Examples here use the default distance sort, lower-information strategy, no district averages, and no parent ratings.

Numbers + Icons

As shown, these strategies were varied for academic and safety information (where parents may seek help interpreting information) but not for distance from home and school resources.


Question 1 Findings

Changing the format improved understanding by up to 3 percentage points and improved satisfaction by up to 1 percentage point.

• Numbers only was the most understandable format (i.e., parents were most likely to answer factual questions correctly).

• All three formats were similarly easy to use (i.e., parents reported all were similarly easy to use).

• Numbers plus graphs was the most satisfying format (i.e., parents reported liking it the most).

Format (Mean 0-100 scale)

Strategy Understanding

Ease of Use

Satisfaction

Numbers Only 75.0^ 86.0 90.0

Numbers + Icons 72.5 86.0 89.5

Numbers + Graphs 72.0 86.0 90.5^

Note: The caret (^) and bold blue text indicate strategies that have at least a 70 percent chance of producing the greatest understanding, ease of use, or satisfaction compared to the other strategies tested. N= 3,500. The understanding scale represents the percentage correct on a comprehension test. Other scales represent the average percentage of ease of use and satisfaction statements with which parents agreed or agreed strongly (after reverse-coding any negative statements).

Implications

Evidence suggests there may be a trade-off. If making the information display understandable is most important, then a numbers-only format may be best. If making it satisfying is most important, then adding graphs to accompany numbers may be the better option.


Question 2 Is it useful to show parents DISTRICT AVERAGES on key mea-sures of school performance for comparison purposes?

Why and how inclusion of district averages was tested

Research in fields outside of education suggests that comparative information (e.g., providing the average cost of an auto insurance policy alongside the sticker price for a given insurance plan) can make decision tasks easier for choosers and studies have documented that offering reference points (the average cost in the example above) does shape consumer choices.25 ESSA encourages this type of comparative information in state and district report cards, for example by requiring districts to provide information on how academic achievement in a particular school compares to the district and the state averages.

However, in the context of school choice, it is not known if providing parents with such averages helps them make sense of data shown for individual schools and whether or not the information is useful as they consider their options. On one hand, district averages might give context to information that is presented for an individual school. (For example, a parent may see that students score about the same on achievement tests in the school closest to their home when compared to the district average.) On the other hand, district averages might simply be seen as clutter, adding to the amount of information a parent has to consider, and undermining their ability to understand or use the information provided.

The study tested two strategies:

Not showing district averages

Showing the district average

As shown, district averages were presented for academic, safety, and school resources information but not for distance from home.

Note: Examples here use the default distance sort, lower-information strategy, numbers only format, and no parent ratings.


Question 2 Findings

Showing district averages reduced ease of use by 0.5 percentage points and reduced satisfaction by 1 percentage point.

• Both strategies were similarly understandable.

• No district average was the easiest to use.

• No district average was the most satisfying.

District Average (Mean 0-100 scale)


Ease of Use

Satisfaction

No district average 73.5 86.5^ 90.5^

District average shown 73.5 86.0 89.5

Note: The caret (^) and bold blue text indicate strategies that have at least a 70 percent chance of producing the greatest understanding, ease of use, or satisfaction compared to the other strategies tested. Understanding scale represents the percentage correct on compre-hension test. Other scales represent average percentage of ease of use and satisfaction statements with which parents agreed or agreed strongly (after reverse-coding any negative statements). N=3,500.

Implications

Evidence suggests that including the district averages would make the information harder to use and less satisfying to parents.


Question 3 Is it useful to include parents’ opinions of schools as an additional SOURCE OF INFORMATION for other parents?

Why and how source of information was tested

When choosing schools, parents rely on information from both official sources (such as district reports of students’ performance on state tests) and other parents (such as word-of-mouth reports on school quality from their neigh-bors).26 Some education research suggests that parents value and use opinions offered by other parents about schools more so than data that comes from school, district, or state reports.27

However, surveying parents—a main way districts or third party information providers might provide parent opinions—can be time-consuming and expensive. Currently it is unclear how information from parent surveys affects outcomes such as understanding of and satisfaction with information displays.

The study tested two strategies:

District Only: Showing only official data provided by school districts (e.g., test score proficiency rates and suspension rates)

District + Parent Ratings: Combining district-provided official data with results from a parent satisfaction survey on academics and safety

Note: Examples here use the default distance sort, lower-information strategy, numbers only format, and no district average. School rankings were the same for parent satisfaction measures as measures based on district data for the same category of information.


Question 3 Findings

Including parents’ opinions reduced understanding by 0.5 percentage points but increased satisfaction by 2.5 percentage points.

• Showing district-only data was the most understandable.

• Both strategies, including or omitting parent ratings, were similarly easy to use.

• District plus parent ratings was the most satisfying.

Source of Information (Mean 0-100 scale)


Ease of Use

Satisfaction

District Only 73.5^ 86.0 89.0

District + Parent Ratings 73.0 86.0 91.5^


Implications

Evidence suggests there is a trade-off. If making the information display understandable is most important, then omitting parent ratings may be best. If making it satisfying is most important, then adding parent ratings to accom-pany district provided information may be the better option.


Question 4 What AMOUNT OF INFORMATION about each school should be shown in a summary of the choices available to parents?

Why and how amount of information was tested

Research in fields outside of education suggests that limiting the amount of information shown can make an infor-mation display easier to understand.28 On the other hand, some studies show that displays that are too simple can be less satisfying because choosers may not feel there is enough information to make good decisions.29 Those providing school information wrestle with how much information parents want and need to make informed choices. For example, ESSA is requiring several pieces of information about each school be included in district provided report cards.30 However, a recent review of 14 online school information displays found some offer substantially more infor-mation than others when initially showing information for each school profiled.31

In the context of school choice, there is not much evidence on what is enough information so that parents learn about individual schools without presenting so much information that they become overwhelmed or confused. Online information displays offer an option to allow parents the ability to start with a lower-amount of information and then click on a link to reveal more detailed information. But whether or not such a strategy is actually helpful to parents is unknown.


Higher Amount: up to six measures per category (e.g., six measures related to school resources)

Progressive Disclosure: lower amount of information initially displayed, with the parent’s option of clicking to see the higher amount of information for any school.

Note: Examples here use the default distance sort, numbers-only format, no parent ratings, and no district average.

Lower Amount: one key measure per category of school information (e.g., one measure related to school resources)

The study tested three strategies:


Question 4 Findings

Varying the amount of information improved satisfaction by up to 1.5 percentage points

• The three strategies were similarly understandable.

• The three strategies were similarly easy to use.

• Showing a higher amount of information was the most satisfying.

Amount of Information (Mean 0-100 scale)


Ease of Use

Satisfaction

Lower Amount 73.5 86.5 89.5

Higher Amount 73.0 86.0 91.0^

Progressive Disclosure

73.5 86.0 89.5


Implications

Evidence suggests that the higher information display would be more satisfying to parents. Information providers should consider displaying more than a single row of information on each school.


Question 5 Does it matter in what ORDER the school profiles initially appear on the information display?

Why and how default sort order was tested

When designing school choice information, information providers must select an order in which to present schools, whether in a paper or online display. Information displays in current use, such as those in Washington, DC, New Orleans, and Detroit, each present the initial list of schools in a different order (distance from home, name, and performance, respectively).32

There is little evidence regarding whether initially sorting by distance versus, say, academic performance, affects parents’ ability to understand information.33 Moreover, it seems possible —but has not been demonstrated—that the order in which schools are presented may influence how school data are actually used. For example, sort order might affect the schools that parents read about and consider, since many parents may not read about every school presented in a guide and could be more likely to consider the first schools listed. Researchers working in fields other than education have found that ordering people’s options from the highest-quality option first to the lowest-quality option last tends to result in people choosing higher-quality options.34 This is consistent with the often-observed practice of “satisficing,” where people making a choice search only until they find a selection that meets their standards, rather than considering and comparing all of the available options.35

The study tested two defaults for how the list of schools was sorted (though in each case parents were able to change the initial sort order):

Sorting by distance

(continued)


Note: Examples here use the numbers-only format, lower information, no parent ratings, and no district average.

Sorting by academic quality

Question 5 Findings

Understanding and satisfaction were 1 percentage point higher when the initial sort order was by distance.

• Sorting by distance was most understandable to parents as a way to list schools by default.

• The two default sort orders were similarly easy to use.

• Sorting by distance was the most satisfying way to list schools by default.

Default Sort Order (Mean 0-100 scale)


Ease of Use

Satisfaction

By Distance 74.0^ 86.0 90.5^

By Academics 73.0 86.0 89.5


Implications

Evidence suggests that sorting by distance may be a better way to make information understandable and satisfying to parents.


Question 6 Can information be presented in ways that encourage parents to choose an academically higher-performing school?

Why and how encouragement to select academically higher-performing schools was tested

Unlike Questions 1 through 5, each of which considers the effects of varying one particular factor on understanding, ease of use, and satisfaction, Question 6 considers all five factors but focuses on a behavioral outcome—the choices that parents make, particularly whether a given strategy makes them more likely to choose schools with higher academic performance.

There is reason to believe the five factors this guide examines (format, default sort order, amount of information, and so on) might affect choice behavior. For example, it is well known that health insurance choices, food choice, and retail shopping are all influenced by the types of factors discussed.36 There is little evidence, however, to guide school choice information providers who may want parents to give extra consideration to academically higher performing schools.

As a reminder, after parents viewed the profiles for the 16 hypothetical schools, they were asked to select and rank their top three as if they were choosing a school for their youngest child. Then the research team looked at the types of schools that the parents preferred the most. These data were used to test whether different ways of presenting the information related to parents’ selecting schools that were higher-performing academically. The term “higher-performing” refers only to the proficiency rate at the school, defined for parents as “the percentage of students who scored Proficient or better on the 2016 state assessments (average across math and reading for all tested grades).”


Question 6 Findings

Parents chose academically higher-performing schools if the profiles were sorted by academic performance as the default, used icons (i.e., A-F color coded letter grades) to represent performance categories, or included lower amounts of information rather than higher amounts.37

Implications

If encouraging parents to select academically higher performing schools for their children is a goal, evidence suggests that the best ways to do so may include initially sorting schools by academic performance, representing academic performance with icons that clearly differentiate different categories, and using a simpler display with a lower amount of information.

• Default sort order: Parents could re-sort the list of schools by distance, academic performance, or school name, but the initial (default) sort order mattered.38 When the list of schools is initially sorted by academic performance instead of distance, parents choose schools with higher proficiency rates (by about five points, 67 percent versus 62 percent proficient).39

Note: N = 3,500

• Format: The format that added A to F icons to the numbers in the display led parents to select schools with better academic perfor-mance, a difference of more than two points between the display with icons and the display with only numbers.

• Amount of information: The simpler display, which had just one measure for each infor-mation category, led parents to pick schools with better academic performance. This was true regardless of whether the profile offered progressive disclosure (had an option to display more details). More specifically, the simpler display led parents to choose schools whose proficiency rate was three points higher on average than with the higher-information display.

Sort bydistance

Sort byacademics

62.7%67.3%

Differenceis 4.6%

Average academic performanceof parent’s top-ranked schools

by default sort order(Percentage Proficient)

Numbers-onlyformat

Graphs andnumbers

format

Icons andnumbers

format

63.8%66.2%65.0%


by format(Percentage Proficient)

Lowerinformation

Parent chooses(default is lower)

Higherinformation

66.0% 62.9%66.2%


by amount of information(Percentage Proficient)


KEY INSIGHTS FROM THE STUDY

The findings for the study’s six questions provide some guidance about information presentation. But looking across them suggests three broader insights that information providers can use in designing school information displays.

1. Decisions about how to display information may require designers to make trade-offs between understandability, ease of use, satisfaction, and effects on parents’ choices.

In the study, some design choices forced trade-offs in that they improved information displays for one outcome at the expense of another outcome. For example, the study showed that including parents’ ratings of schools and using graphs is more satisfying to parents, but both of these strategies make the information display harder to understand.

Easier to understand, but less satisfying

Harder to understand, but more satisfying

Another example of a trade-off is that it is possible to design information displays to encourage parents to select certain types of schools, such as those with higher academic performance, but doing so could unintentionally make the information harder to understand and use. For example, sorting schools by academic performance instead of distance leads parents to select higher-performing schools, but it also makes the profiles harder to understand and less satisfying. Information providers need to decide which outcomes are most important.

2. All of the different school information displays explored in this guide were understandable, easy to use, and satisfying to most parents.

Large majorities of parents were able to correctly answer factual questions (to demonstrate understanding) and agreed or agreed strongly with positive statements about the information displays (indicating ease and satisfaction), regardless of which variation they saw.40 For example, while including district averages led to lower ease of use and lower satisfaction, parents who saw district averages still agreed or strongly agreed with 86 percent of ease of use statements and 90 percent of satisfaction statements, on average.41

High rates of understanding, ease of use, and satisfaction suggest that the information display organization used as the backbone of this study was itself well designed. Specifically, it is possible to construct the best (and worst) infor-mation displays for a given outcome (e.g., understanding) by going through each of the five factors and selecting the strategy that improves each outcome the most (or the least). Even the worst set of strategies for information displays


had scores for each outcome that might appear reasonable to some information providers. For example, parents who saw the information display variation that performed worst in terms of understanding still got 71.1 percent of the points correct on the understanding task.

As discussed earlier in this guide, all of the displays tested in the study used a simple, standard layout with a glossary, a map, and four categories of information presented in such a way that it was easy to compare schools. The aspects of the displays that were varied were only meant to test potentially promising options against one another. Therefore, it may not be surprising that even the “worst” information displays were relatively understandable, easy to use, and satisfying.

It should be noted that although variations between individual displays were often subtle and none of the varia-tions led to what might be considered poor outcomes, information providers have to make several design decisions, and the effects can add up. To quantify this, one can identify the combination of strategies—considering all five factors together—that produce the best score for a given outcome and compare it to the combination of strategies that produce the worst score for that same outcome. This exercise, presented in detail in section C of the technical appendix, suggests that there was a difference in parent satisfaction of 5.2 points on the 0–100 scale (percentage of items with which parents agree or agree strongly) between the best and worst information display. These design deci-sions also had cumulative effects on the academic performance of the schools picked by parents in the study, an effect of almost 20 percentile points. That is the equivalent of pushing parents from selecting the 4th ranked school to the top-ranked school in the study’s hypothetical school district.

3. How information is displayed can lead parents to choose different types of schools.

Whether information providers consciously plan to influence choices or not, the ways that information is displayed may lead parents to choose schools based on certain categories of information, by making certain types of schools stand out. As discussed under question 6, initially sorting schools by academics, including icons, and displaying a lower amount of information all encouraged parents to select schools that had stronger academic performance.

An important insight from the study is that decisions about information displays can influence the schools that parents choose in many ways beyond those already discussed. Findings in the technical appendix (see Table C.2) show that some strategies tested as part of the study led parents to choose schools that were closer to home or had better measures of school safety or resources. Taken together the study findings suggest information providers may want to be aware of how specific design decisions may influence the choices parents make.

People seem to choose based on what is most prominently displayed.

One way that information can be prominently displayed is by listing it first. The study showed that sorting schools by an academic performance measure can lead parents to choose academically higher-performing schools, but the same principle can apply to any category of information. For example, setting the default sort order to distance led parents to choose schools that were more than half a mile closer to home, on average.

One possible explanation for these findings is that parents may take shortcuts when making decisions. They might look at what is most prominently displayed, settle on a choice, and then decide to stop looking further. This phenomenon has been demonstrated by researchers in other fields as well.42

Sort bydistance

Sort byacademics

1.7 miles

2.3 miles

Differenceis 0.6 miles

Average distance from home of parent’s top-ranked schools

by default sort order(Miles)


Another way that information can be prominently displayed is to have it take up more space on the page. For example, profiles that showed more details about school resources (higher information displays) caused parents to be more likely to select schools with more resources—5 more laptops per 100 students, on average—compared to parents that saw fewer details about school resources (lower information displays) (see technical appendix, Table C.2). It is possible this finding occurred because the profiles with more details devoted more space to resources compared to other catego-ries of information. Specifically, safety and resources each take up the same amount of space in the profile that shows fewer details (“lower amount”), but in the profile that has more details (“higher amount”), the description of resources takes up twice as much space as the description of safety.43

Formatting strategies can influence choices by highlighting or obscuring differences.

Certain formatting strategies can accentuate differences between schools while others may obscure or downplay them. An example from the study illustrates this point. In the display on the left of the figure (showing icons), the three schools appear very different in terms of safety, rated as A, B, or C, each with a different color. The three schools fall into distinct categories, each represented by different colors and letters. However, the display on the right (showing graphs) makes the three schools look very similar. The horizontal bar graphs are presented on a 0 to 100 scale, which makes the first two schools look nearly identical in terms of safety. Therefore, the same difference of 3 percentage points (between School 1 and School 2) can appear large or small depending entirely on

visual representation. In fact, the study found that parents who were shown icons for safety measures chose schools with better safety scores, compared to parents who were shown graphs of the same data (see Table C.2 in the tech-nical appendix).

A possible reason why icons made safety information more relevant to parents is that the underlying safety scores for the schools in the display were closely bunched together. The icon format used letter grades to label numbers that would have otherwise appeared similar on a 0-100 scale; this may have accentuated differences in the safety measures and led parents to emphasize safety in their school selections. This type of categorization effect has been demon-strated in other fields. 44

Categorical displays like icons can serve to either accentuate or mask differences, depending on how the categories are defined. For example, a district could give an “A” to every school with 80 to 100 percent of its students with no suspensions, reserving “B” and “C” ratings for lower percentages. Then all three schools in the example above would receive an “A” and parents might be less likely to let safety influence their choice of school.


LIMITATIONS TO KEEP IN MINDThis guide is intended to help information providers incorporate evidence from rigorous research into decisions about presenting information on schools to parents. However, even the best evidence cannot replace good judgment or knowledge of a local context. Nor can it resolve disagreements about which goals to prioritize when a design choice is shown to improve one outcome at the expense of another. For example, if a more detailed display improves satisfaction but reduces ease of use, this guide does not try to say which is more important.

The following limitations help to put the research in perspective.

1. Experimental conditions are the not the same as real-life conditions.

In a real-world setting, parents can take in official online school information along with other information on repu-tation and quality they get through friends and neighbors and direct contact with schools and their staff. In the study, the online information display was the only information parents had about schools. Also, the stakes are lower in the study because the choices are hypothetical, so parents may be less motivated to review and consider information than if they’re making an actual choice to enroll their children in a school. Nevertheless, the study aimed to ensure that parents participating in the study took the exercise seriously and the research team removed respondents who tried to speed through the survey (a proxy for not responding sincerely).

Graphical displays can also be designed to accentuate or mask differences. For example, the suspension data could be presented on an 80 to 100 scale instead of a 0 to 100 scale. The examples below show how the zoomed-in scale (80 to 100) makes the schools look different, whereas the otherwise-identical graph with a zoomed-out scale (0 to 100) make them look similar, even though the actual differences between the schools are the same in both displays.

In short, information providers using graphs or icons must choose whether to draw parents’ attention to certain differences between schools or to mask those differences. Visual formats can be used to accomplish either goal.

Zoomed-out Scale Obscures Differences

Zoomed-in Scale Draws Attention to Differences


2. The parents used in the study may differ from the parents of interest to some readers of the guide.

The study used an “opt-in” approach, meaning that the participants do not statistically represent a well-defined set of parents. It is possible that those who sign up to participate in surveys such as the one used in this study, and those who put forth effort on the survey, may differ in distinct ways from the national population of low-income parents involved in making school choices. For example, those participating in online survey panels may have more expe-rience in viewing online information displays than other parents due to their time spent completing surveys. Those who persist and attend to the instructions may have a keener interest in the topic. However, because there is no way to develop a list containing the national population of low-income parents of school age children who are shopping for schools, there is no way to compare the characteristics of the study sample to those in the target population. Also, the survey was conducted in English, so it does not represent the potentially large and important group of parents who have limited English language skills.

Despite these limitations, the set of study participants was geographically diverse, representing all 50 states (as shown in the technical appendix, Figure B.5), and nearly evenly divided among urban, suburban, and rural respondents (also in the technical appendix, Table B.6).

In addition, most of the findings presented in this guide remain consistent if the sample is restricted to just parents with the lowest income, the lowest education, non-intensive Internet use, or with prior experience with school choice.45 Specifically, of the 18 separate findings presented in the findings and implications section of this guide, 16 were consistent for parents with the lowest income, 13 were consistent for parents with the lowest education, 17 for the parents with fewer than 30 hours of weekly Internet use, and 15 for parents with prior experience with school choice. The full set of subsample findings is presented in the appendix, Tables C.3 through C.11.

3. Some of the findings described in this guide can have more than one interpretation.

The guide indicates that parents found the higher-information display more satisfying than the lower-information display (Question 4). This could reflect a preference for more detail over less detail, or it could reflect a preference for the specific measures that were revealed in the higher information display that were not included in the lower information display. For example, the presence of art studios and computer labs was included in the higher but not the lower information display. Similarly, the higher information display showed estimated commute time in addition to distance. Another example is the addition of parent ratings to the school profiles (Question 3). This led parents to find the information display less understandable but more satisfying. This could be caused by the fact that the infor-mation was derived from parent surveys or it could simply be another example of the effect of changing the total amount of information in the display.

Despite some of these limitations, information providers should consider the study’s lessons and think about running their own studies to test various ways of designing information displays and school profiles if they want to under-stand how school information can work best in their areas. Feedback from parents who participate in these studies could lead to better, more accessible information and possibly, to better school choice outcomes. Meanwhile, this guide demonstrates that information providers already have the potential to design school information displays in ways that help parents through the school choice process and ultimately benefit the education system.


ENDNOTES1 For simplicity, this guide uses the term “parents” to refer to anyone who is making or contributing to the decision to choose a school. In reality, other adults—and the students themselves (Condliffe, Boyd, and DeLuca, 2015; Valant, 2014)—often play an important role in choosing a school.2 Gross, DeArmond, and Denice (2015) describe the causes—and potential solutions—to parental confusion in the “high-choice” cities of Denver and New Orleans. Stein and Nagro (2015) examine one possible source of this confu-sion: the excessive complexity of the informational materials available to school-choosing families in many large districts. 3 Haxton and Neild (2012); Stein and Nagro (2015); and Stewart and Wolf (2014) have documented the challenges that school choice information can pose for families with low income, low education levels, or limited English.4 The National Center for Education Statistics has docu-mented the increase in both the number and enrollment of charter and magnet schools. See “Table 216.20. Number and enrollment of public elementary and secondary schools, by school level, type, and charter and magnet status: Selected years, 1990-91 through 2014-15” in the Digest of Education Statistics, online at https://nces.ed.gov/programs/digest/d16/tables/dt16_216.20.asp last accessed on December 7, 2017.5 U.S. Congress, “Every Student Succeeds Act of 2015.” Pub. Law 114-95 § 114 Stat. 1177 (2015-2016).6 Stein and Nagro (2015)7 See Hibbard et al. (2002) and Hildon, Allwood, and Black (2012) for evidence related to presentation of information about healthcare plans and providers.8 Section B.5 of “Every Student Succeeds Act State and Local Report Cards Non-Regulatory Guidance” January, 2017. Accessed at https://www2.ed.gov/policy/elsec/leg/essa/essastatereportcard.pdf, December 7, 2017.9 The study was conducted by Mathematica Policy Research.10 Harris and Larsen (2015); Haxton and Neild (2012); Stein and Nagro (2015); Stewart and Wolf (2014)11 This part of the study also introduced a special survey item called an instructional manipulation check (IMC). The IMC, which is described in more detail in the technical appendix, is a question that is easy to answer correctly if one reads the instruc-tions, but tempting to answer incorrectly if one does not. This is designed to check whether participants were complying with the study’s instructions and to encourage them to do so.12 The majority of parents in the study (70 percent) reported that their youngest child was elementary school aged or younger and, in turn, the hypothetical scenario was likely easy for these parents to imagine.13 By assigning parents at random to one of the displays, the researchers could be sure that the only difference, on average, between groups of parents is the information display that they saw. This makes it possible to conclude that any differ-ences in outcomes were caused by the information display strategy being studied.

14 This is not a comprehensive list of the categories of information that parents consider. Notably, the school profiles omitted student demographic data although research indicates that parents consider student demographics when choosing schools. Including demographic information would have complicated interpretation of the study’s results. 15 Glazerman and Dotter (2017); Harris and Larsen (2015); Hastings, Kane, and Staiger (2009); Schneider, Teske, and Marschall (2002); Stewart and Wolf (2014)16 Many of these principles are drawn from research on web design. See Hoekman (2010)17 Schneider and Buckley (2002); “Every Student Succeeds Act of 2015.” Pub. Law 114-95 § 114 Stat. 1177 (2015-2016).18 The study used Bayesian inference, described in the technical appendix, to calculate the probability that any given difference was greater than zero. As discussed in the appendix, the study only reports on results where that poste-rior probability was at least 70 percent.19 With five factors, each with two or three strategies, there are 72 possible combinations of strategies. Formally, the study design is a 3x2x2x3x2 factorial experiment. The technical appendix contains details about the study’s methodology.20 Harris and Larsen (2015); Dynarski et al. (2016); Glazerman and Dotter (2017)21 See Hibbard et al. (2002), Borgmeier and Westenhoefer (2009), Jones and Richardson (2007), and Jacobsen et al. (2014).22 Glazerman (2017) describes and provides links to 14 Internet sites that provide such information. Two exam-ples are the My School DC school finder at http://find.myschooldc.org/ and Great Schools Detroit at https://www.greatschools.org/michigan/detroit/schools/, both accessed September 20, 2017.23 The current study retained the numbers in all cases to ensure that all parents in the study had at least an exposure to the common information which was used in testing under-standing. 24 For each measure where letter grades were shown, grades were defined based on the relative ranks of the 16 schools in the display: ranks 1-4 received an “A,” ranks 5-8 received a “B,” ranks 9-15 received a “C,” and the school ranked 16th received an “F.” Parents did not receive an explanation of how the grades were defined; however, the value of the relevant measure was shown as a number next to each letter-grade icon, so it was possible to identify the cutoff value between grades by reading the information in the display.25 Kahneman (1992); Kahneman and Tversky (1979); Thaler (1985)26 Schneider et al. (1998); Schneider, Teske, and Marschall (2002); Bell (2009); Valant (2014)27 Valant (2014)28 Cronqvist and Thaler (2004); Peters et al. (2007)29 Bundorf and Szrek (2010); Chakraborty et al. (1994)

http://find.myschooldc.org/

http://find.myschooldc.org/


30 These pieces of information include “the number and percentage of students at each of three or more levels of achievement on each of the academic assessments in mathe-matics, reading/language arts, and science”, the number and percentage of English learners achieving English language proficiency, four-year adjusted cohort graduation rate, and whether and for what reason the school is identified for comprehensive or targeted support and improvement. Furthermore, this information must be presented separately for economically disadvantaged students, students from each major racial and ethnic group, children with disabilities, and English learners. See “Every Student Succeeds Act of 2015.” Pub. Law 114-95 § 114 Stat. 1177 (2015-2016).31 Glazerman (2017)32 Glazerman (2017)33 Sorting by name (alphabetical sort) is also common, although this strategy was not included in the study because it assumes that parents are already familiar with the schools and would want to look up specific schools about which they had prior knowledge.34 Hibbard et al. (2002); Hildon, Allwood, and Black (2012)35 March (1994); Simon (1955) 36 Akaah and Korngaonkar (1983); Jansen et al. (2000); Lewinsohn and Mano (1993); Srinivasan and Park (1997); Thaler and Sunstein (2009)37 Every factor that was tested affected the types of schools parents chose. This section highlights particular strategies that encouraged parents to choose academically higher-per-forming schools. The full set of findings summarizing how display strategies influenced the types of schools selected by parents is shown in the technical appendix, Section C.1, Table C.2, and discussed later in this guide.

38 It “mattered” if the probability that there was a positive effect was at least 70 percent. Only findings that met this threshold are discussed in this guide. The probability is calcu-lated as from the posterior distribution of a Bayesian model described in the technical appendix Section B.1.39 The vertical axes on all of the figures goes from 45 percent to 95 percent proficiency, which represents the lowest and highest proficiency rates of the schools used in the study (excluding one school that was created as an outlier on purpose to test readers’ attentiveness). 40 Descriptive statistics for these outcomes are presented in the technical appendix, Tables B.1 and B.3.41 This guide refers to predicted averages, which are based on the whole sample, not just parents assigned to one particular combination of factors.42 Browne et al. (2007) 43 Another explanation could be that parents found the new information (such as whether there is a music room, art studio, and so on) to be especially appealing.44 McKelvie (1978)45 The sample included parents in urban, suburban and rural areas. While distance or commute times to school could be interpreted differently in urban areas than in suburban or rural areas, the experiment had an insufficient number of respondents from urban areas to allow for separate analyses by urbanicity. Among parents in the experiment, living in an urban area was associated with more exposure to school choice and with being lower income (chi-square tests for independence are rejected at the 0.001 level for both asso-ciations, as shown in the technical appendix B). These two characteristics, prior choice exposure and being low income, are two aspects of urban areas that may affect how individ-uals experience school choice information, and the study examined these two groups of parents as subsamples.


REFERENCESAkaah, I.P., and P.K. Korgaonkar. “An Empirical Comparison of the Predictive Validity of Self-Explicated, Huber-Hybrid, Traditional Conjoint, and Hybrid Conjoint Models.” Journal of Marketing Research, vol. 20, no. 2, 1983, pp. 187–197.Bell, Courtney A. “All Choices Created Equal? The Role of Choice Sets in the Selection of Schools.” Peabody Journal of Education, vol. 84, no. 2, 2009, pp. 191–208. Borgmeier and Westenhoefer “Impact of different food label formats on healthiness evaluation and food choice of con-sumers: a randomized-controlled study.” BMC Public Health, vol. 9, 2009, pp. 1-12.Browne, Glenn J., Mitzi G. Pitts, and James C. Wetherbe. “Cognitive Stopping Rules for Terminating Information Search in Online Tasks.” MIS Quarterly vol. 31, no. 1, March 2007, pp. 89–104.Bundorf, M. Kate, and Helena Szrek. “Choice Set Size and Decision Making: The Case of Medicare Part D Prescription Drug Plans.” Medical Decision Making: An International Journal of the Society for Medical Decision Making, vol. 30, no. 5, 2010, pp. 582–593.Chakraborty, G., R. Ettenson, and G. Gaeth. “How Con-sumers Choose Health Insurance.” Journal of Health Care Marketing, vol. 14, no. 1, 1994, pp. 21–33. Condliffe, Barbara F., Melody L. Boyd, and Stefanie DeLuca. “Stuck in School: How Social Context Shapes School Choice for Inner-City Students.” Teachers College Record, vol. 117, no. 3, 2015, pp. 1–36.Cronqvist, Henrik, and Richard H. Thaler. “Design Choices in Privatized Social-Security Systems: Learning from the Swedish Experience.” American Economic Review, vol. 94, no. 2, 2004, pp. 424–428. Dynarski, Mark, Julian Betts, and Jill Feldman. “Applying to the DC Opportunity Scholarship Program: How Do Parents Rate Their Children’s Current Schools at the Time of Appli-cation and What Do They Want in New Schools?” NCEE Evaluation Brief. Washington, DC: U.S. Department of Education, April, 2016.Glazerman, Steve, and Dallas Dotter. “Market Signals: Evidence on the Determinants and Consequences of School Choice from a Citywide Lottery.” Educational Evaluation and Policy Analysis, April 2017. doi: 10.3102/0162373717702964Glazerman, Steven M. “Shopping for Schools: Mapping Choice Architecture in the Education Marketplace.” Mathematica Working Paper Series, No. 59. Princeton, NJ: Mathematica Policy Research, December 2017.Gross, Betheny, Michael DeArmond, and Patrick Denice. “Common Enrollment, Parents, and School Choice: Early Evidence from Denver and New Orleans.” Bothell, WA: Center on Reinventing Public Education, May 2015.Harris, Douglas, N., and Matthew Larsen. “What Schools Do Families Want (and Why)? School Demand and Informa-tion Before and After the New Orleans Post-Katrina School Reforms.” New Orleans, LA: Education Research Alliance for New Orleans, January 2015.

Hastings, Justine S., Thomas J. Kane, and Douglas O. Staiger. “Heterogeneous Preferences and the Efficacy of Public School Choice.” NBER Working Paper No. 2145. Cambridge, MA: National Bureau of Economic Research, 2009. Haxton, Clarissa L., and Ruth Curran Neild. “Parents’ ‘Hard’ Knowledge of Admission Criteria and Acceptance in Phil-adelphia’s High School Choice Process.” Journal of School Choice, vol. 6, no. 4, 2012, pp. 483–503. Hibbard, Judith H., Paul Slovic, Ellen Peters, and Melissa Finucane. “Strategies for Reporting Health Plan Performance Information to Consumers: Evidence from Controlled Studies.” Health Services Research, vol. 37, no. 2, April 2002, pp. 291–313.Hildon, Zoe, Dominique Allwood, and Nick Black. “Impact of Format and Content of Visual Display of Data on Comprehen-sion Choice and Preference: A Systematic Review.” International Journal on Healthcare, vol. 24, no. 1, 2012, pp. 55–64. Hoekman Jr., Robert. Designing the Moment: Web Interface Design Concepts in Action. Berkeley, CA: Pearson Education, 2010.Jacobsen, Rebecca, Jeffrey Snyder, and Andrew Saultz. “Infor-mation or Shaping Public Opinion? The Influence of School Accountability Data Format on Public Perceptions of School Quality.” American Journal of Education, vol. 121, no. 1, November 2014, pp. 1–27.Jansen, S.J., A.M. Stiggelbout, M.A. Nooij, and J. Kievit. “The Effect of Individually Assessed Preference Weights on the Relationship Between Holistic Utilities and Nonprefer-ence-Based Assessment.” Quality of Life Research, vol. 9, no. 5, 2000, pp. 541–557.Jones, Gary, and Miles Richardson. “An Objective Examination of Consumer Perception of Nutrition Information Based on Healthiness Ratings and Eye Movements.” Public Health Nutrition, vol. 10, no. 3, 2007, pp. 238-244.Kahneman, Daniel. “Reference Points, Anchors, Norms, and Mmixed Feelings.” Organizational Behavior and Human Decision Processes, vol. 52, no. 2, March 1992, pp. 296–312. Kahneman, Daniel, and Amos Tversky. “Prospect Theory: An Analysis of Decision Under Risk.” Econometrica, vol. 47, no. 2, March 1979, pp. 263–292.Lewinsohn, S., and H. Mano. “Multi‐Attribute Choice and Affect: The Influence of Naturally Occurring and Manip-ulated Moods on Choice Processes.” Journal of Behavioral Decision Making, vol. 6, no. 1, 1993, pp. 33–51.March, James G. Primer on Decision Making: How Decisions Happen. New York, NY: Simon and Schuster, 1994. McKelvie, S.J. “Graphic Rating Scales — How Many Catego-ries?” British Journal of Psychology, vol. 69, 1978, pp. 185–202.Peters, E., D. Dieckmann, A. Dixon, J.H. Hibbard, and C.K. Mertz. “Less Is More in Presenting Quality Information to Consumers.” Medical Care Research and Review, vol. 64, no. 2, April 2007, pp. 169–190. Schneider, Mark, and Jack Buckley. “What Do Parents Want from Schools? Evidence from the Internet.” Educational Eval-uation and Policy Analysis, vol. 24, 2002, pp. 133–144.


Schneider, Mark, Paul Teske, and Melissa Marschall. Choos-ing Schools: Consumer Choice and the Quality of American Schools. Princeton, NJ: Princeton University Press, 2002. Simon, Herbert A. “A Behavioral Model of Rational Choice.” Quarterly Journal of Economics, vol. 69, no. 1, February 1955, pp. 99–118. Srinivasan, V., and C.S. Park. “Surprising Robustness of the Self-Explicated Approach to Customer Preference Structure Measurement.” Journal of Marketing Research, vol. 34, no. 2, 1997, pp. 286–291.Stein, Mark L., and Sarah Negro. “The Readability and Complexity of District-Provided School-Choice Informa-tion.” Journal of Education for Students Placed at Risk, vol. 20, no. 3, August 2015, pp. 199–217.

Stewart, Thomas, and Patrick J. Wolf. The School Choice Journey: School Vouchers and the Empowerment of Urban Families. New York, NY: Palgrave MacMillan, 2014.Thaler, Richard. “Mental Accounting and Consumer Choice.” Marketing Science, vol. 4, no. 3, 1985, pp. 199–214.Thaler, Richard H., and Cass R. Sunstein. Nudge: Improving Decisions about Health, Wealth, and Happiness. New York: Penguin Group, 2009.Valant, Jon. “Better Data, Better Decisions: Informing School Choosers to Improve Education Markets.” Washington, DC: American Enterprise Institute, November 2014.

U.S. Department of EducationBetsy DeVosSecretary

Institute of Education SciencesMark SchneiderDirector

October 2018

The report was prepared by Steven Glazerman, Ira Nichols-Barrer, Jon Valant, and Alyson Burnett for the Institute of Education Sciences under Contract No. ED-IES-15-C-0048 with Mathematica Policy Research. The project officer is Meredith Bachman in the National Center for Education Evaluation and Regional Assistance.

The content of the publication does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be:

Glazerman, Steven, Ira Nichols-Barrer, Jon Valant, and Alyson Burnett (2018). Presenting School Choice Information to Parents: An Evidence-Based Guide (NCEE 2019-4003). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

This report is available on the IES website at http://ies.ed.gov/ncee.

Upon request, this report is available in alternate formats such as Braille, large print, audiotape, or computer diskette. For more information, please contact the Department’s Alternate Format Center at 202-260-9895 or 202-205-8113.

Acknowledgments

This guide, and the study on which it is based, was a collaborative effort that benefited from the contributions of many people. The study team received useful input from its technical working group: Sujata Bhat, Rebecca Jacobsen, Cait Lamberton, Ellen Peters, Aesha Rasheed, Peter Bergman, Judith Hibbard, and Paul Teske. Eric Johnson provided valuable guidance on consumer behavior research and the design of the experiment. Andrew Gelman answered questions about Stan software.

The online information displays were designed and implemented by Tembo, Inc., led by Meg Towle and Michael Moore. GfK provided the survey sample used in the study.

At Mathematica, Phil Gleason provided expert review at every stage of the design and report. Lisbeth Goble navigated the data collection through approvals, and Jesse Chandler was instrumental in designing the survey and planning experimental logistics. Mariel Finucane advised on statistical issues. The team received expert programming assistance in Stan from Daniel Kassler and quality assurance review by Shira Mitchell. Adrienne Jones provided additional research assistance and project management support. The guide was edited by Molly Cameron and Daryl Hall. Steven Malick and Kristin Hallgren read drafts of the guide. The docu-ments were produced by William Garrett and Jessica Coldren, with graphic design and layout by Laura Watson-Sarnoski.

U.S. Department of EducationInstitute of Education Sciences

550 12th Street, SW, Washington, DC 20202(202) 245-6940

https://ies.ed.gov/ncee/

Presenting School Choice Information to Parents: An Evidence-Based Guide

Date post:	25-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Presenting School Choice Information to Parents: An Evidence … · 2018-10-29 · Presenting...

Documents