Date post: | 26-Mar-2015 |
Category: |
Documents |
Upload: | amia-hayes |
View: | 215 times |
Download: | 3 times |
Moving to Opportunity TOWN HALL
Abt Associates, June 26, 2003
Background and Challenges: Judie Feins Findings: Robin Jacob Implications: Larry Orr
Abt Associates Inc.
2
WHY MTO?
Program Objective: Provide poor families living in high-poverty public or assisted housing with the opportunity to move to low-poverty neighborhoods with a Section 8 rent subsidy and counseling.
Research Objective: Measure the impact of moving to lower poverty neighborhoods on the outcomes of adults and children.
3
Precursors of MTO
1) Long history of research showing harmful effects of bad neighborhoods
2) Chicago’s Gautreaux Program: court-ordered mobility of poor, African-American families seemed to show positive effects of suburban life—especially on the children
3) Trend of increasingly concentrated poverty in American cities 1980-1990 suggested growing harm, unless a remedy could be found to reverse these neighborhood effects.
4
The Demonstration Program
The MTO program ran from 1994 to1998 in five cities: Baltimore, Boston, Chicago, L.A. and N.Y.
Families with children were recruited from public and assisted housing in concentrated-poverty neighborhoods. Many of the housing developments were very distressed and dangerous places.
Any resident who wanted the chance to get a mobile housing subsidy and move out could sign up, as long as the family met Section 8 eligibility rules.
Some 5,300 families volunteered; 4,608 were found eligible.
5
The Research Design
For research purposes, families were randomly assigned into one of three groups:
The experimental group received special Section 8 vouchers that could be used only in census tracts with poverty rates below 10 percent. Nonprofit counseling agencies in each city helped the experimental group families to locate and lease suitable housing in low-poverty areas.
The Section 8 group received regular Section 8 vouchers, which could be used anywhere they found a suitable unit with a willing landlord and rent below the program cap. These families did not receive any mobility counseling.
The control group received no vouchers but continued to be eligible for project-based assistance.
CBS News Correspondent John Roberts. From Ghetto To White Picket Fence
BALTIMORE6/5/2000
11/17/2000
The Los Angeles Times
Housing, Not School, Vouchers Are Best Remedy for Failing Schools - Jan 31, 1999, Larry Cuban
A Social Experiment in Pulling Up Stakes; Aid: Does neighborhood affect economic and school success? Five cities relocate poor families to find out.- Sep 23, 1997; pg. 1, Larry Gordon
A Fresh Start Housing: The Moving to Opportunity program will take families out of the projects to see if a new environment helps them succeed. Nov 8, 1994; pg. 1, Larry Gordon
Washington Post
" In Baltimore, Getting a Lease on Middle-Class Life. " May 10, 2000; pg. 1 Amy Goldstein.
New York Times
" Better Than a Voucher, a Ticket to Suburbia. " October 18, 2000. Richard Rothstein
Chicago Tribune• "Foes kill housing plan funds" - Dec 15, 1994; Laurie Abraham• "Where Should Poor Families Live?" - Jul 23, 1994; Lori Montgomery• "Hostility Toward Relocating the Poor is a Matter of Race" - Apr 27, 1994; Clarence Page
7
A Long and Challenging Effort
8
THE EARLY CHALLENGES
Abt Associates began working on the MTO design with HUD in September 1993. Our initial job:
Convincing HUD to shift to a 3-group design
Developing uniform (but flexible) procedures
Managing implementation (started July 1994)
Surviving a political fire-storm
Quietly growing the demonstration
Finishing intake and lease-up (March 1999).
9
AND THEN…
Database construction: We designed and built sophisticated relational database, which continues to evolve;
Sample tracking: We kept up with 4,608 families (19,000+ persons) by active and passive means, despite high mobility rates and the disruption of public housing revitalization and demolition.
10
CHALLENGES OF THE INTERIM EVALUATION
Research team: Abt partnered with a group of academics, the Urban Institute, and two data subcontractors to win the interim evaluation contract in July 2000.
Broad scope: HUD’s interest extended well beyond the usual housing and neighborhood issues to education, health, delinquency, employment and earnings, income and self-sufficiency.
Complex data collection: The study required that several kinds of data be gathered from almost 11,000 sample members in a compressed period.
11
Evaluation Team
ANALYSTS FROM:
Abt Associates (design, implementation, data collection, impact analysis)
Urban Institute (qualitative analysis)
National Bureau of Economic Research (impact analysis)
Georgetown University (collection and analysis of crime data)
12
Interim Evaluation Funders
• U.S. Department of Housing & Urban Development
• Grants via National Bureau of Economic Research National Institute for Child and Health Development (NICHD) National Science Foundation (NSF) National Institute of Mental Health (NIMH) Robert Wood Johnson Foundation Smith Richardson Foundation Russell Sage Foundation W.T. Grant Foundation Spencer Foundation MacArthur Foundation
• Grants via Georgetown University National Consortium on Violence Research (NSF) Brookings Institution
13
Interim Evaluation Data Sources
Pre-Survey Qualitative Data In-depth semi-structured interviews with adult heads of
household and children informed survey design.
Quantitative Data Structured surveys with adults, teens, and children
Achievement tests of teens and children
Measurement of adult blood pressure, child height and weight
Administrative data from state and local agencies (earnings, TANF, food stamps, arrest records)
14
Interim Evaluation Surveys
Full Sample (adults, youth, children) Fielded from January through June 2002.
Achieved survey response rates of 80% of adults, 77% of children, 76% of youth.
3-in-10 Sub-sample Random draw late in June from non-complete cases.
Focused field resources on a sub-set of hard-to-find cases; included travel to remote sites.
Increased response rates considerably and reduced the risk of non-response bias.
15
Interim Evaluation Response Rates
High Effective Response Rates Attained a weighted adult response rate of 90 percent.
Attained youth/child surveys and achievement test response rates of 86 to 90 percent.
Group Differences Difference in response rates between random assignment
groups was less than 1 percent
Very low risk of non-response bias between groups
16
Interim Evaluation Findings
What did we learn?
17
Estimation Methods
All estimates regression-adjusted with standard set of covariates, including (where available) pre-RA value of outcome
ITT = “intention to treat” – impact on entire treatment group, including those who did not lease up
TOT = “treatment on treated” – impact on those who leased up only
Tests of significance at .05 level
18
Mobility Outcomes
Poverty rates of current locations are substantially reduced (entire experimental and Section 8 groups)
Fraction minority population in experimental group locations is reduced, but more than half moved to areas 80%+ minority
Almost half of experimental lease-ups were in tracts with increasing poverty from 1990 to
2000
More than half of experimental group lease-ups moved again to somewhat higher
poverty tracts
19
Impacts on Neighborhood Outcomes
Significant positive impacts for both experimental and Section 8 groups on:
Feeling safe in the neighborhood (day and night)
Police coming when called
All measures of neighborhood quality
Significant reductions for both experimental and Section 8 groups in:
Witnessing drug activity in the neighborhood
See public drinking, groups hanging out
Crime victimization over last six months
20
Impacts on Housing Outcomes
Significant increases for both experimental and Section 8 groups in:
Most measures of housing quality
Utility payment problems
Prevalence of housing assistance receipt
No significant impacts on current total housing cost
21
A Focus on Educational Outcomes
The following factors were hypothesized to be mediators of educational outcomes:
Community-Level
Quality of Schools
Community Norm and Values
Social and Physical Environment
Student and Family-Level
Parent Attitudes and Behaviors
Student Attitudes and Behaviors
22
School Characteristics
Modest improvements for both groups
70% of Experimental lease-ups remained in the same large urban school district
C Mean E-C S8-C
Percent free lunch .720 -.133 -.058
Percent white
Percent limited English proficient
.090
.180
.100
-.063
.046
ns
Pupil-teacher ratio 14.6 ns ns
Percentile rank on State Assessments
.170 .091 .026
23
School Climate
No significant effects on school climate
C Mean E-C S8-C
There is a lot of cheating in tests & assignments
.416 ns ns
Discipline in school is fair .724 ns ns
Disruptions from other students inhibit learning
.640 ns ns
Child feels safe in school .775 ns ns
Teachers are interested in students .810 ns ns
24
Other mediators of educational outcomes
Community-Level Mediators Significant impacts on social and physical environment
Some evidence of impact on community norms and values but not on peer role models
Some evidence of positive impact on economic opportunities but not
on earnings or employment of sample adults.
Student- and Family-Level Mediators No significant effects on parental monitoring
No significant effects on parental involvement in school
No significant effects on student school-related behaviors
25
Impacts on Education Outcomes
No significant impact on student achievement
No significant impacts on grades, coursework, special ed placement, graduate rates or college attendance.
26
Other Outcomes: Health
Adult Physical and Mental Health
Physical health & substance use: largely insignificant impacts
Mental Health: significant E-C impacts
Obesity: significant E-C impacts
Youth Physical and Mental Health
Physical health: no significant impacts
Mental health: improvement for girls
27
Other Outcomes: Youth Behavior
Youth Delinquency & Risky Behavior
Behavior Problems Index: significant increase in self-reported behavior problems among boys
Delinquency: no significant impact
Arrests & Risky Behavior: substantial gender differences
Employment and School Attendance
No significant effects for boys, but an increase in full-time school attendance for girls in the experimental group and a reduction in full-time employment for girls in Section 8
28
Other outcomes: Income & Earnings
Receipt of Public Assistance
No significant impacts on current receipt of public assistance, either for full sample or subgroups by ethnicity and barriers to employment
Household Income & Poverty Status
No significant impacts on income, poverty, food security and self-sufficiency
29
Implications for Policy
30
Does this mean there are no impacts on these outcomes?
On many outcomes, only fairly large impacts can be detected with confidence—e.g., to be 80% sure of detecting impacts as significant:
Adult earnings would have to be increased by about 40% in the experimental group, 30% in the Section 8 group
TANF benefits would have to be reduced by 50%
Youth asthma attacks would have to be reduced by 67%
The fact that an impact estimate is not statistically significant does not mean there was no impact—it
means we don’t know if there was an impact.
31
Policy Implications - Context
The worst concentrations of urban poverty are usually HUD-subsidized.
In US, high-poverty neighborhood almost always means high-crime neighborhood.
There are legitimate concerns about “relocating the ghetto”. These might lead to restricted mobility programs.
32
Will mobility programs work?
Can voluntary restricted mobility programs move extremely low-income public housing tenants to middle-class neighborhoods?
Yes—at least, for 48%.
However, geographic restrictions come at some cost to lease-up (60% of Section 8 comparison group moved)
Those who move are likely to follow the “path of least resistance.” In MTO lease-ups were predominantly in minority neighborhoods, where poverty rates trending up.
33
Who benefits, and for how long?
Those who move enjoy substantially better housing, safer neighborhoods, less stress, better mental health, and lower obesity. There is some indication that girls behavior and motivation to succeed improves.
These benefits are still significant 4-7 years after the initial move.
Benefits to the rest of society are less clear. There is some indication of lower rates of criminal behavior among girls, but higher rates of crime and delinquency among boys. There are no reductions in welfare costs.
34
What do these results imply about choice of policies for improving the lives of low-income families?
Are the problems of low-income families environmental, or due to attributes of the families themselves?
These results imply that a change in environment can improve the family’s sense of well-being directly, and have some salutary effects on the behavior of youth.
But physical health, educational performance and attainment, employment, earnings, and welfare dependence do not appear to be sensitive to residential environment, at least within 4-7 years.
To improve these outcomes within that time frame requires policies designed to deal directly with these specific problems—such as educational improvements, employment and training, or welfare-to-work programs.
35
Dr. Larry Orr, Abt Associates
Dr. Judith Feins, Abt Associates
Dr. Jeffrey Kling, Princeton University
Dr. Jens Ludwig, Georgetown University
Dr. Robin Tepper Jacob, Abt Associates
Dr. Barbara Goodson, Abt Associates
Dr. Lisa Sanbonmatsu, NBERDr. Lawrence Katz, Harvard University
Dr. Jeffrey Liebman, Harvard Univsersity
Dr. Erik Beecroft, Abt Associates
Dr. Rhiannon Patterson, Abt Associates
Dr. Alvaro Cortes, Abt Associates
Ms. Carissa Climaco, Abt Associates
Ms. Debi McInnis, Abt Associates
Mr. Robert Teitel, Abt Associates
Dr. Susan Popkin, Urban Institute
Analysts: