Roca Strategic Business Plan

Roca Strategic Business Plan FY 2013 – FY 2017

Roca Strategic Business Plan FY 2013 to FY 2017

Page | i

The United States has a growing number of young people who are not only disconnected but are unable to engage in traditional programming. Roca understands that without intervention, our young people will end up repeatedly incarcerated. It is increasingly clear that the criminal justice system is not only expensive, but in many cases is a tremendous failure. Incarceration cannot be the only answer for these young adults. As a result, many states are looking for other options. Roca is an outcomes driven organization that serves extremely high‐risk young people in Chelsea, Revere, East Boston and Springfield, Massachusetts. Roca takes a unique approach to moving very high‐risk young people out of violence and poverty. Roca doesn’t wait for these young people to re‐engage in society on their own. Left to their own devices, most never will. Instead, we go out and find them on the streets. Roca seeks out the most difficult, challenging individuals for whom other programming has failed, and systematically works to establish positive, consistent relationships built on trust and respect. Roca focuses the delivery of its Intervention Model to address the needs of 17‐24 year old young men who demonstrate a strong propensity for violence, crime and/or adult incarceration, who are either: 1) aging out of the juvenile justice or juvenile probation systems; 2) connected with the adult justice system; or, 3) are high risk youth from the community being served1. Roca takes a unique approach to moving these very high risk young people out of violence and poverty. We tell them the truth, teach them to trust, and help them learn and practice the skills needed to transform their lives and get on the path to achieving economic independence. Roca has developed and operates an Intervention Model designed to help the most high‐risk young people break the destructive cycles of poverty, violence and perpetual incarceration. The Intervention Model pushes young people to identify, confront and overcome destructive behaviors and learn the skills needed to re‐engage and succeed in society, education, and the economy. Roca’s Intervention Model is one of the solutions to stopping the repeated cycle of early incarceration and recidivism for high‐risk young people. The Intervention Model is designed with three long term outcomes in mind: 1) to reduce incarcerations as tracked through violations and /or new convictions of crime; 2) to increase compliance with court ordered conditions; and 3) to demonstrate quatifiable movement towards economic independence through retained employment for each of the young men Roca serves. The Intervention Model is a cognitive‐restructuring and skills development intervention, enabling young people to transform their lives and move toward the outcomes of economic independence (successfully keeping a job) and living out of harm’s way (with a primary focus of staying out of prison). Building off of our success to date and pulling from evidence‐based practices in behavioral health, criminal justice and workforce development, the Intervention Model includes two years of intensive programming with two additional years of follow up for retention and sustainability. Based on a framework for change used in medical and mental health fields, the Intervention Model has four core components:

1. relentless outreach, on‐going and aggressive outreach and follow‐up designed to meet young people where they are and build trust;

2. transformational relationships, an intensive case management model; 3. stage‐based programming designed to increase young people ability to move toward economic independence through life

skills, educational and pre‐vocational, and employment programming; and, 4. work with engaged institutional partners, a partnership model with criminal justice, health, education, and other

institutions to increase systemic capacity for intervention with very high risk young people and provision of needed supports.

1 Note: Roca will continue to serve high risk young women and young parents through its site in Chelsea as a secondary target population.

Executive Summary


Page | ii

To the best of our knowledge, Roca’s Intervention Model for very high‐risk young people is the only full‐time, long‐term, behavioral change intervention delivered on the street, for this population, by a non‐mandating authority. Roca has created an observable impact on its targeted group of young men. In FY 2012, Roca served 409 very high‐risk young men through its intervention model. Despite the fact that Roca focuses its work on young men who are unable to participate or engage in traditional work or programming, of those eligible for services, Roca retained 78% and engaged them in the intervention model throughout the course of the year. Of those who completed the intensive component of the model in 2012, 90% have had no new arrests; 100% have had no new technical violations; 79% of program graduates were on track to retain employment for a minimum of 6 months; and 70% have demonstrated education gains. Roca understands the potential of our both our primary and secondary service populations to be both dangerous and expensive (i.e. social assistance, incarceration, teen parenthood, substance abuse/mental health issues, preventable hospitalization), and is committed to working with them to create positive impacts in their lives, reconnecting them to their communities and helping them become taxpayers and positive participants in society. The economic benefit of Roca’s work is dramatic. While we invest $24,500 in our Intervention Model for four years of programming with each young person, alternative costs to society are massive. It presently costs Massachusetts approximately $47,500 per year for person in prison, with average sentences of 4.75 years, resulting in a total average cost per person of $225,625, and this does not include any additional costs to the family or community that result from incarceration or any costs from subsequent incarcerations. The Journal of Criminology estimated the average lifetime expenses of social services and incarceration for a high‐risk individual is between $250,000 and $2,000,000.

The difference is clear. Roca’s work not only addresses the needs of high‐risk young people and the communities

in which they live in a more effective way than traditional approaches, but it saves money in the long run. Through the scaled growth of our Intervention Model, Roca will establish a nationally recognized, evidence‐based practice for young people to leave the streets and gangs, change their lives, and get out of violence and poverty. Roca is in an early‐stage growth for impact that meets the criteria of “organizations that have demonstrated a significant level of effectiveness and are increasing their capacity for scale readiness.” Roca is proposing to scale social impact through an increase in direct services and indirect influences through increases in:

1. the quantity of services and geography through expansion of services at our existing locations and through two additional program replications, growing from serving 710 to 1,755 young men per year2

2. the quality of services through expanding the organization’s delivery capacities and demonstrating the Intervention Model as an evidence based practice; and,

3. influencing public policy and practice through demonstrating alternative programming models, advocacy, the Roca engaged institution strategy, networks, and a knowledge development/sharing agenda.

2 Roca will also increase its services delivery to young mothers increasing from 200 young mothers being served in FY 13 to 388 young mothers

being served in FY 17.


Page | iii

The specifics of Roca’s growth plan are outlined in the table below:

Service Cap

acity A. Expand Roca’s impact on high risk young men from 710 young men in FY 13 to 1,755 young men through the expansion of

services out of the Chelsea and Springfield hubs and the replication of the Intervention Model in two new locations. (This includes significant growth for young men in retention and advancement.)

B. Develop and Implement an Alternative Community Corrections Pilot C. Apply the Intervention Model and Adjust Appropriately for Very High Risk Young Mothers, increasing services from 200 young

mothers in FY 13 to 338 young mothers in FY 17.

Program

Quality And

Evaluation

Systems

A. Expand Roca’s Impact on very high risk young men by ensuring fidelity to the model throughout replication and enhancing business development and partnerships.

B. Demonstrate the Intervention Model for Very High Risk Young Men as Evidence Based through the completion of model codification and documentation and conducting third party implementation, outcome and impact evaluations.

C. Apply the Intervention Model and adjust appropriately for use with very high risk young mothers.

Organ

izational

Strengthening A. Increase organizational staffing, leadership and management capacity for scale, impact, and sustainability, ensuring sufficient

capabilities to operate a minimum of four sites in multiple states.

B. Increase development capacity for sustainability, ensuring sufficient capabilities to operate a minimum of four sites in multiple states.

C. Create long term strategy to demonstrate increased impact of the Intervention Model.

Finan

ces

A. Develop, Launch and Complete 25th Anniversary Campaign. B. Explore and position for additional public funding sources/strategies (e.g. pay for success financing, alterative community

corrections financing, and government contracting.) C. Ensure Roca’s long term financial stability through resource development activities including growing the endowment to $5

million and operating reserves to 3‐4 months of operating cash.


Page | iv

Executive Summary ..................................................................................................................................................................................... i Table of Contents ...................................................................................................................................................................................... iv Introduction ............................................................................................................................................................................................... 1 Current Challenges and Demand for Services............................................................................................................................................ 1

Criminal Justice Need.......................................................................................................................................................................... 1 Services for Very High Risk Young People ........................................................................................................................................... 1

The Response: Roca's Intervention Model ............................................................................................................................................... 1

Intervention Model ............................................................................................................................................................................. 1 Applications of the Model .................................................................................................................................................................. 7 Evaluation to Date/Impact of the Model ............................................................................................................................................ 8

Social Impact and Rationale for New Approach ...................................................................................................................................... 12 Growth Plan ............................................................................................................................................................................................. 14

Milestones and Deliverables ............................................................................................................................................................. 18 Increasing Service Capacity – Program Expansion and Replication .................................................................................................. 31 Business Imperatives ........................................................................................................................................................................ 33 Advocacy for Growth and Expansion ................................................................................................................................................ 34

Organizational and Management ............................................................................................................................................................ 35

Mission, Vision and History .............................................................................................................................................................. 35 Theory of Change .............................................................................................................................................................................. 35 Organizational Structure ................................................................................................................................................................... 39 Leadership Management and Staffing .............................................................................................................................................. 39 Next Steps in Performance Based Management .............................................................................................................................. 41

Financial Projections ................................................................................................................................................................................ 43

Financial Operating Plan ................................................................................................................................................................... 43 Financial Projections and Implications ............................................................................................................................................. 43 Primary Public Funding Source (Pay For Success or Alternative Community Corrections Financing) ............................................. 45 Growth Capital and 25th Anniversary Campaign ............................................................................................................................... 45

Risks and Opportunities ........................................................................................................................................................................... 47 Appendices............................................................................................................................................................................................... 49

A. Detailed Description of the Model B. Research Informing the Model C. Organizational Charts D. Biographies of Management Team and Board of Directors/Board of Directors Listing E. Pay for Success Preliminary Financial Information F. FY 2012 Outcomes Report G. FY 2012 Springfield Outcome Highlights

H. Impact Evaluation Plan

Table of Contents


Page | 1

America’s young people are dying. They are dying quickly from guns and gangs and slowly from poverty and isolation. They are the young people and young parents in gangs, in and out of state systems, using drugs and alcohol, poor, hungry, often homeless and/or suffering from the trauma of war. There is currently a tremendous gap in services available for these very high‐risk young people. They should be the leaders of tomorrow. Instead, they are often disconnected from society, labeled very high‐risk youth, and fall between the cracks of the very systems designed to help and protect them. While there are a great number of services and programs serving young people, there are currently no intervention models available for these young people who are simply unable to show up for traditional programming. This group of young people is not even at the starting line to economic independence will be unable to succeed without an intervention designed to change behaviors. Without deliberate, systematic intervention, high‐risk young people will continue to end up in prison or involved in violence. At the same time, it has also become clear that the criminal justice system ‐ the network of institutions responsible for rehabilitating these young people and keeping the public safe ‐ is more expensive and less ineffective than alternative approaches. Relying on current corrections practices is bankrupting state governments, and simultaneously creating more dangerous criminals. It is now apparent that incarceration cannot be the only answer for these young adults. As a result, many states are looking for other options. Roca has developed and operates an Intervention Model designed to help the most high‐risk young people break the destructive cycles of poverty, violence and perpetual incarceration. The Intervention Model pushes young people to identify, confront and overcome destructive behaviors and learn the skills needed to re‐engage and succeed in society, education, and the economy. The Intervention Model pushes young people to identify, confront and overcome destructive behaviors and learn the skills needed to re‐engage and succeed in society, education, and the economy. To the best of our knowledge, Roca’s Intervention Model for very high‐risk young people is the only full‐time, long‐term, intervention based the combined principles of cognitive re‐structuring, skills development, motivational interviewing, and transitional employment delivered on the street, for this population, by a non‐mandating authority. Roca is now seeking to expand the impact of its Intervention model, enabling it to reach a larger number of young people from across the country. Specifially, Roca will expand its programmiing to better serve very high risk young men, aged 17‐24, who are either 1) aging out of the juvenile justice or juvenile probation systems with a strong propensity for reincarceration as an adult; 2) connected with the adult justice system; or 3) are high risk members of the community being served who have a strong propensity for incarceration as an adult.3 SERVICE CAPACITY

A. Expand Roca’s impact on high risk young men from 710 young men in FY 13 (535 in intensive and 175 in retention services) to 1,755 young men in FY 17 (1,100 in intensive and 655 in retention services) through the expansion of services out of the Chelsea and Springfield hubs and the replication of the Intervention Model in two new locations, one to being operations in FY 14 and the second to begin operations in FY 16.

B. Develop and Implement an Alternative Community Corrections Pilot.

C. Apply the Intervention Model and Adjust Appropriately for Very High Risk Young Mothers, increasing services from 200 young mothers in FY 13 (100 young mothers in healthy families and 100 high risk young mothers in intensive services) to 338 young mothers in FY 17 (100 young mothers in healthy families and 200 high risk young mothers in intensive services and 38 young mothers in retention services.)

3 Note: Roca will continue to serve high risk young women and young parents through its site in Chelsea.

Introduction


Page | 2

PROGRAM QUALITY AND EVALUATION SYSTEMS A. Expand Roca’s Impact on very high risk young men by ensuring fidelity to the model throughout replication and enhancing business

development and partnerships.

B. Demonstrate the Intervention Model for Very High Risk Young Men as Evidence Based through the completion of model codification and documentation and conducting third party implementation, outcome and impact evaluations.

C. Apply the Intervention Model and adjust appropriately for use with very high risk young mothers. ORGANIZATIONAL STRENGTHENING

A. Increase organizational staffing, leadership and management capacity for scale, impact, and sustainability, ensuring

sufficient capabilities to operate a minimum of four sites in multiple states.

B. Increase development capacity for sustainability, ensuring sufficient capabilities to operate a minimum of four sites in multiple states.

C. Create long term strategy to demonstrate increased impact of the Intervention Model.

FINANCES

A. Develop, launch and complete 25th anniversary campaign (including increased focus on individuals and corporations).

B. Explore and position Roca for additional public funding sources/strategies (e.g. pay for success financing, alternative community corrections financing, and government contracting.

C. Ensure Roca’s long term financial stability through resource development activities including growing endowment to $5 million and operating reserves to 3‐4 months of operating cash.


Page | 1

Criminal Justice Need US incarceration rates are currently the highest in the world‐ five times higher than average rates in most developed countries. Just as troubling, the United States has a growing number of young people who are disconnected, uneducated, impoverished and unable to succeed in traditional programming and employment. These disenfranchised young people have fueled the U.S.’ skyrocketing incarceration rate for years. They are also, by and large, both the perpetrators and the victims of the violence that occurs in our cities every day. Roca understands that without deliberate, systematic intervention, these high‐risk young people will continue to end up in prison or involved in violence. At the same time, it has also become clear that the criminal justice system‐ the network of institutions responsible for rehabilitating these young people and keeping the public safe‐ is too expensive and too ineffective. Current corrections practices are bankrupting state governments and simultaneously creating more dangerous criminals. It is now apparent that incarceration cannot be the only answer for these young adults. As a result, many states are looking for other options. Roca is poised to provide the criminal justice system an option for change, and success. Whereas traditional public policy has taken two basic approaches to increasing public safety (prevention and suppression), Roca offers a third approach: intervention. Prevention programs typically attempt to stop trouble before it starts, targeting in‐school and younger youth with academic tutoring, athletics and peer leadership programming. The suppression approach is defined by traditional law enforcement practices, such as policing and incarceration. Intervention programming fills the void traditional approaches leave by adressing the needs of high‐risk young people once they have already engaged in dangerous or criminal activity. Intervention programming moves young people toward behavioral change, helping them confront their destructive tendencies, and providing them a pathway toward economic self sufficiency and safe, healthy living. Services for Very High Risk Young People A multitude of excellent programs meet the needs of “at‐risk youth,” a broad category of young people falling within a vast continuum of preparedness for education, career, and safety. At the top of the spectrum are programs reaching high school graduates who need help in their efforts to enter college and find career‐track employment opportunities. A second tier of programs help young people stay in school and complete high school. A third tier help high school drop‐outs earn their GED and find employment. Finally, a fourth tier focuses on keeping the most at‐risk kids out of lock‐up, safe from street violence, and into programs for substance abuse and basic literacy. Young people in this fourth tier are most accurately described as youth in crisis or very high‐risk youth. Young people in this 4th tier are Roca’s focus. Roca serves very high‐risk, low income, young people, ages 16 – 24, from Chelsea/Revere East Boston and Springfield, MA. Many of these young people are drops‐outs, substance abusers, gang members, have extensive criminal records, and/or come from broken homes without the income to support basic living requirements. The young people that define our target population have not, and cannot succeed in other educational and employment programs because they are neither ready, nor willing.They are not ready developmentally, intellectually or emotionally. And the behavioral barriers to their personal success far exceed their ability to participate in structured programming that requires consitent, disciplened self‐management and motivation. More often than not, these youth are the ones who fall between the cracks of the very systems that were designed to help and protect them. The United States does not have a coherent policy to prevent very high risk young people from becoming disconnected from society or to help them successfully transition to adulthood. Yet, there is an overwhelming amount of research and analysis on this pressing

Unlike other programs that work with young people who are motivated to improve their lives and who have basic skills, Roca seeks out the most difficult and challenging young people who are unwilling or unable to attend traditional programming, work or school. Many of the

young people engaged in the first stages of Roca’s programming do not possess the most basic social, educational and employment skills associated with

financial self‐sufficiency and responsible, healthy living. According to the Urban Institute, in a report drafted for Strategic Grant Partners, Roca is serving a population

that is “almost unique” among the hundreds of programs assessed.

Current Challenges and Demand for Services


Page | 2

issue. Within the six year period between 1989 and 19954 alone, there were over 2,500 articles published on the subject of at‐risk youth and their families. Despite abundant discussion of young, at‐risk populations, the field still lacks a degree of coherence. There remains a dire need for outcomes‐focused programming, carefully designed to help at‐risk youth overcome the hardships and barriers to success they experience early in their lives. This sort of highly‐focused, outcomes and evidence‐driven programming is the key to increasing the proportion of individuals who at the age of 25 have a high school diploma, are employed with career advancement possibilities, and are not engaged in adverse, risky behavior. As states and municipalities begin to recognize this gap in services, many are looking a pay for success financing as a way to demonstrate effective programs while at the same time demonstrating cost savings. This financing mechanism also allows the state or municipality to mitigate their financial risk when working with new programming models. Roca’s Intervention Model was developed and implemented address this high risk, underserved population, while simultaneously serving to inform the United States’ policies for very high‐risk youth. When implemented correctly, Roca’s Intervention Model has enormous capacity to change public policy, reduce public costs (such as those associated with incarceration and public assistance) and increase public safety by reducing criminal and delinquent behavior. While there are many prevention programs that target teenagers for the purpose of stopping trouble before it starts, such as sports or after school academic programs, evidence has shown that they make little impact on mitigating violence and poverty among older teens and young adults. As for suppression and punishment, these tactics do little to address high rates of recidivism, and cost public the public billions of dollars on a yearly basis. It has also become clear, that long‐term and mandatory sentencing practices do more to increase offenders’ chances of reengaging in criminal behavior after release, than they do to deter further criminal behavior. The criminal justice system has also created tremendous waste by investing in programs that are not driven by evidence‐based best practices for rehabilitation and social reintegration. New research shows that eliminating mandatory sentencing, shortening terms for non‐violent offenses, and incorporating community‐based rehabilitation programs not only saves criminal justice systems money, but dramatically reduces rates of recidivism. In light of this recent research, Roca is poised to serve as one of the most viable institutional alternatives to traditional suppression and punishment practices. For years, Roca has challenged the traditional means of addressing violence and poverty by working with and for the young people that other institutions have either cast aside or dealt with indiscriminately. Intervention, as we have found, is the underemphasized middle ground between prevention and punishment‐ and it is proving to hold far more promise in truly unraveling the behavioral and economic cycles that have doomed so many young people to lives of poverty and violence. The common misconception that long‐term, cognitive‐restructuring programs don’t work or require far too much in the way of human and economic resources to be viable on a large scale, has done much to damage the possibility for progressive political and policy decisions. Roca serves as a clear counterweight to that argument and the Intervention Model should be considered by policy makers as a formal alternative to traditional sentencing and rehabilitation programs. The need for interventions for very high‐risk young people is growing each day, as demonstrated by the number of requests received by Roca for assistance; the number of requests Roca receives regarding program replication; and, Roca’s selection as the Commonwealth’s apparent provider for its social impact financing pilot project (pay for success.) Roca has received a significant amount of attention across Massachusetts and across the country for its Intervention Model for Very High Risk Young people. This has included inquiries ranging from information requests to full scale replication requests from organizations, public agencies and private foundations from across the country.5 The U.S. Department of Labor has provided Roca with a $2.4M grant to test the effectiveness of its Intervention Model, through its “Pathways out of Poverty” funding stream. Similarly, Roca has had conversations with both the U.S. Department of Justice and others to examine the long term outcomes and impacts of the Intervention Model. Roca continues to work with these agencies as well as private funders, to meet their needs around learning and information sharing with respect to its Intervention Model.

4 See Kemshall, H. (2008) ‘Risks, Rights and Justice: Understanding and Responding to Youth Risk’ Youth Justice. 8(1): 21‐37.Swadner, B.B. and Lubeck, S. (1995) ‘The Social Construction of Children and their Families “At Risk”: An Introduction’, in B. B. Swadner and S. Lubeck (eds) Children and Families ‘At Promise’: Deconstructing the Discourse of Risk. New York: State University of New York Press. Tait, G. (1995) ‘Shaping the “At Risk Youth”: Risk, Governmentality and the Finn Report’, Discourse 16(1): 123–34.

5 Currently, two cities, in addition to Springfield have approached Roca to conduct a full‐scale replication.


Page | 1

Intervention Model Target Population Roca’s services focus on 17‐24 year old men in crisis and on the streets, particularly those involved with our criminal justice system, because this group is responsible for much of the violence in our cities. Roca chooses to focus on this group because it is the group that has most often been underserved or neglected by public and private institutions, with the exception of the criminal justice system, and is the group responsible for the majority of violence in our cities. Roca’s target population has become more focused over the past year. Through this business plan, Roca will focus on 17‐24 year olds in crisis and on the streets, particularly those involved with our criminal justice system, because this group is responsible for much of the violence in our cities. Specifically, Roca is seeking to serve young men who demonstrate a strong propensity for violence, crime and/or adult incarceration, who are either:

1. aging out of the juvenile justice or juvenile probation systems; 2. connected with the adult justice system; or, 3. are high risk youth from the community being served.

Roca’s choice to work with this population is based on the risk principle. According to the Crime and Justice Institute, “The risk principle identifies who to target for the most intensive of services and programming. Specifically, limited resources should be directed to those at highest risk for involvement in the criminal justice system. Further, higher risk clients should receive the greatest dosage of treatment and intervention. This principle is of critical importance given the serious constraints of limited budgets and working with growing community supervision and prison and jail populations.”6 Roca knows that it can help these young men create the behavior changes critical to keeping them out of prison and engaged as members of our communities. Overview of the Intervention Model Development of the Intervention Model is Roca’s most important accomplishment. While conventional criminal justice and public policy has been designed to respond to violence and poverty in urban communities with a limited set of institutional tools, the Intervention Model serves as a progressive alternative that is both cost‐efficient and highly‐effective. Traditionally, federal and state governments have operated within a narrow framework, employing two basic strategies: prevention and suppression. Though these strategies remain fundamental components in the fight against poverty and violence, they exclude an equally important strategy: intervention. The Intervention Model effectively combines relationship‐building and targeted programming (life skills, education and employment) to support young people in developing the necessary skills to create positive behavioral changes. Once a consistent relationship is established, it becomes the foundation for cognitive‐behavioral change, helping the young person move through a long‐term, stage‐based plan for improving their lives, creating opportunities as they make educational, life‐skills and employment gains. Each youth worker is trained in various evidence‐based and clinical techniques for promoting behavioral change, specifically:

cognitive‐restructuring, an approach designed to help people identify and change dysfunctional thoughts that contribute to problem behaviors;

motivational interviewing, a client‐centered counseling style designed to help clients explore and resolve ambivalence;

stage‐based learning, which helps young people practice the academic and pre‐vocational skills needed to achieve social and economic independence; as well as,

6 Bechtel, Kristin and Barbara A. Pierce, MPPM. “An Overview of What Works in Correctional Interventions.” Crime and Justice Institute (January 2011).

The Response: Roca’s Intervention Model


Page | 2

transitional employment, a stage‐based approach for helping individuals gain and practice critical entry‐level employment skills while earning a subsidized wage.

The process starts with intensive outreach, follow‐up to engage and reengage young people over and over again, and building relationships that support change. Then Roca offers programming that meets young people where they are developmentally, operates basic and advanced transitional employment, creates jobs, and works with cities, criminal justice partners, and other organizations on deliberate intervention and support strategies. Each component of Roca’s Intervention Model is designed to drive toward measurable, positive outcomes. The following image illustrates Roca’s Intervention Model.

The Intervention Model is designed for young people who have dropped out of school, who are involved in drugs and gangs, have limited or no employment history, and/or are young parents. Young people must also live in a contextually determined radius that promotes participation in center‐based services. By seeking out these young people where they are, in the streets, Roca aims to directly intervene in their lives during the most crucial points. But the process of engaging these youth is not easy. Roca staff must seek a young person out over and over again, until the consistency of their interactions forms the basis for a relationship. Once a consistent relationship is established, it becomes the foundation for cognitive‐behavioral change, helping the young person move through a long‐term, stage‐based plan for improving their lives, creating opportunities as they make educational, life‐skills and employment gains. Each youth worker is trained in various evidence‐based and clinical techniques for promoting behavioral change, specifically cognitive‐restructuring, a problem‐focused approach designed to help people identify and change dysfunctional beliefs, thoughts and patterns of behavior that contribute to problems, motivational interviewing, which is an evidence‐based client‐centered counseling style for eliciting behavior change by helping clients to explore and resolve ambivalence, skill‐based learning, which helps young people practice the academic and pre‐vocational skills needed to achieve social and economic independence, as well as transitional employment which is a national best practice, and stage‐based approach for helping individuals gain and practice critical entry‐level employment skills while earning a subsidized wage. Through this work we help young people move out of violence and poverty over 2 years, with 2 additional years of supportive follow‐up for program graduates. The process starts with intensive outreach, follow‐up to reengage young people, and building relationships that support change. Then Roca offers programming that meets young people where they are developmentally, operates basic and advanced transitional employment, creates jobs, and works with cities, criminal justice partners, and other


Page | 3

organizations on deliberate intervention and support strategies. Each component of Roca’s Intervention Model is designed to drive toward measurable, positive outcomes. The table below illustrates each of the components of the Intervention Model and provides details regarding frequency and/or dosage for each element.

Program Component Service Category Activity/Dosage

Relentless Outreach & Follow‐Up

Daily outreach times N/A

Paired outreach N/A

Critical incident response plans

N/A

Transformational Relationships

Contacts

Phase 1‐Engagement‐2‐3 contacts weekly

Phase 2 Behavior Change‐2‐3 contacts weekly

Phase 3 Sustaining‐2wkly, 1wkly, 2monthly, 1 monthly

Engagement Programming

Weekly outings

Physical fitness‐volleyball, basketball, weight room, etc…

Arts/Music/Dance

Formal Field Trips

Stage Based Programming

Life Skills

Houses of Healing/Power Source‐10 sessions Street Smart‐SA/HIV Curriculum‐10 sessions Circles‐10‐12 Clinic Visits‐2 Pregnancy Prevention Workshops‐4 Parenting Group‐10 sessions (if a young parent)

Education/Pre‐Voc

Completion of Baseline Educational Assessment Minimum of 3 Follow‐Up Educational Assessments Increased weekly attendance and participation Minimum 2 informal prevocational Roca certification trainings Minimum 2 formal industry recognized certification trainings

Employment/Job Readiness Programming

Basic Transitional Employment Workforce Readiness Workshops‐10 Core Workshops Mock Interviews‐minimum of 3 interviews One on One Job Coaching‐Basic Job Search, Transportation Plans, Resumes, Child

Care Plans‐if needed Demonstrated Behavior Skill Assessments‐minimum of 40

Work with Engaged Institutions

Direct Service Activities

Court advocacy as appropriate Coordinated interventions with key partners (police, probation, school, etc..) Accessing Services‐health, mental health, benefits, housing etc… Cross Agency Programming (groups, events, workshops, etc…)

Formal and Informal systems change

N/A

The image below illustrates the timeline a participant goes through as they engage in the Intervention Model. Figure 3: Participant Flow through 24 Month Intervention Model


Page | 5

Implementation Model Goals and Objectives: Roca’s Intervention Model is designed to ensure that graduates will achieve the sustained outcomes of no incarcerations and retained employment (measured five years from enrollment. In order to achieve these outcomes, Roca has developed the following intermediate outcomes and measures.

2 Years from enrollment 3 Years from enrollment 4 Years from enrollment

Educational Gains

increase on assessments

attainment of industry recognized certification

Educational Gains

continued increases on assessments


Educational Gains

continued increases on assessments


Employment

30 hrs/wk

competitive employment placement

minimum of 30 days

Employment Retention

no gaps in employment longer than 30 days


no gaps in employment longer than 30 days

No New Arrests

self report w/YW verification

want to get to Gov. Report

No New Arrests



No New Arrests



No New Technical Violations









In addition, Roca has developed the following short‐term outcomes for the Intervention Model:

Outcome Performance Measures

Progress through the Stages of Change

Decreasing criminal/delinquent behavior

Decreasing street/gang involvement

Decreasing substance use/abuse Increasing employment engagement

Increasing educational engagement

Assessed as Work Ready (by 15‐18 mo from enrollment)

Completion of 4 domains of Workforce Readiness Criteria

Positive Behavioral Changes (indicated on Out of Harm’s Way Assessment)

Readiness for change Bond w/Roca Staffs Pro‐Social Support‐bond with other adults Pro‐Social Support/Behaviors –peers Future Orientation Substance Use General Delinquency Sexual Activity & Practices


Page | 6

Roca has also detailed out a list of quality indicators to demonstrate progress as young people move towards outcomes:

Programming Component Quality Indicators

Transformational Relationship

Min. 2x/wk actual contact with Youth Worker –after 2mo from enrollment

Relapse in Phase 1 Movement to Phase 2 (4‐6mo from enrollment)

Movement to Phase 3 (by 24mo from enrollment)


1x/wk attendance in any programming ‐‐‐2‐4mo from enrollment

2x/wk in life skills/education/pre‐voc/employment programming—4mo‐24mo from enrollment

Transitional Employment Status and Progress o Active (# of days in) o Bench (# of days in) o Re‐hire (# of days in)

Educational Status and Progress o Completion of educational baseline assessment o # of follow up educational assessments o Level of attendance o Level of participation o # of informal pre‐vocational trainings o # of informal pre‐vocational certificates

Life Skills o Completed Houses of Healing/Power Source Course o Completed # of Circles o Completed # of SA Groups/Mtgs. o # of Clinic Visits o Completed Parenting Group o Pregnancy Prevention

Workforce Readiness Criteria

# of items on Basic Needs Checklist completed o Picture ID, Social Security Card, Birth Certificate, Resident Alien/Green Card o Health Insurance o Stable Mailing Address o Appropriate Email Address o Appropriate Voice Mail Message o Transportation Plan o Benefits Enrollment o Child Care Plan o Resume

# of Workforce Readiness Workshops completed (pre/post tests)

Demonstrated Skill Assessments –attainment of consistent scores from crew supervisor, youth worker, pre‐vocational instructors

# of concrete benchmarks completed o Minimum of 2 industry recognized vocational trainings o Completion of a Mock Interview with Workforce Readiness Coach o Basic job search (on‐line) o Travel to a location ‐ Transportation Capacity o For those in Transitional Employment ‐ 60 consecutive workdays o For those not in Transitional Employment at Roca – Meet the programming attendance standard of 2x/wk for 2 months – has to be educational and/or life skills programming


Page | 7

Applications of the Model Roca currently utilizes its Intervention Model to serve very high‐risk young people, whom have been involved with various public, criminal justice and youth services institutions. We have vast experience serving young people that are court‐involved, on probation, parole or were previously incarcerated. After in depth review of our program outcomes, and review of recent field research, Roca has identified two targeted applications of the Intervention Model and corresponding areas of service where demand for effective intervention programming remains unmet. In its first application, the following two categories of young people can be served through the existing Intervention Model:

The first target group can be classified as that which Roca has traditionally served, but on a broader scale ‐ young people who are: heavily street/court/gang involved; have extensive involvement with the criminal justice system; are high school drop‐outs or are on the verge of dropping out; have little to no work history; and/or, are high‐risk young parents. These are the young people in gangs, using and selling drugs, and having children they can’t support. These young people will continue to be Roca’s primary focus. Our goal of moving these high‐risk young people toward economic independence and safe, healthy living will remain unchanged, as will the basic programmatic structure for achieving the desired outcomes.

Roca has chosen to break out a second target population as a newly identified subgroup based on feedback from state officials and experts in the field of social services. This group is defined by those young people aging out of juvenile justice, juvenile probation, and children’s services, who are identified as requiring further services to avert incarceration. According to state officials, approximately one‐third of youth that age out of DYS are arrested for another crime and convicted within a year; that number increases to 40% in the second year. Many of them end up in the adult corrections system. It is expected that a number of young people aging out of the other two systems are on the same trajectory. In our determination, this group should receive an almost identical Intervention Model program structure as the first subgroup. The challenge in service delivery for this population derives primarily from budgetary constraints. However, there is interest both in the Commonwealth of Massachusetts and nationally to work with this particular group of young people.

Roca also believes that there is a second application – using the Intervention Model as an alternative to community corrections and/or incarceration. We understand that there may be some operational differences and legislative changes required to make this a reality. However, we believe that the evidence‐based practices Roca has employed on a non‐mandated basis have vast potential for reducing recidivism and re‐incarceration among this group of young offenders. As criminal justice reform begins to gain momentum throughout the country, it is becoming clearer that diverting people to evidence‐based community corrections programming not only increases public safety through truly rehabilitative outcomes, but significantly reduces corrections budgets by curbing recidivism. There is preliminary interest from some criminal justice partners in this application, as well as legislative language that would permit community corrections funds to support a pilot. This group consists of young people (17+) that are actively involved in the criminal justice system (incarcerated, on probation, and/or on parole). Roca has extensive experience serving young people on probation, parole or with criminal records. Historically however, members of this population were engaged in Roca’s Intervention Model on a voluntary basis or as an informal component of their path to restitution. Roca now proposes to deliver the Intervention Model to this target population as a mandated, comprehensive probation program that can be stipulated through criminal sentencing.

For high‐risk young people in the criminal justice system, the proposed Alternative Community Corrections Model (ACCM) will provide intensified services during the first 12 months of engagement. This Model will serve young offenders, ages 18 to 24, who are actively involved in the justice system and deemed high‐risk to reoffend by validated risk and needs assessment standards. This ACCM could serve as alternative sentencing and/or an alternative community corrections model. The purpose of the ACCM is to give criminal justice entities the option of placing high‐risk offenders in the Pilot Program as as a means of achieving truly rehabilitative outcomes, thereby preventing recidivism and reoffense. In doing so, the criminal justice system will reduce incarceration and public safety expenses associated with this population. This intervention is designed to achieve several important outcomes for the population served, including:

A reduction in incarcerations as tracked through violations and/or convictions of new crime;


Page | 8

An increase in compliance with court‐ordered conditions; and,

A quantifiable movement towards economic independence through retained employment. It is expected that the ACCM meet all informational needs of participating courts and corrections entities. In accomplishing these specific outcomes, the ACCM will demonstrate a new, progressive model for reducing the long‐term public costs associated with this high‐risk population, specifically related to recidivism and incarceration. If the desired outcomes are achieved, the Commonwealth (and potentially other states across the nation) may choose to expand these mandated intervention services, thereby achieving systemic reform. The ACCM will create a new, mandated Intervention Model that differs from current community corrections services in its intensity, comprehensiveness and reliance on evidence‐based practices. The ACCM is an adaptation (higher dosage and mandated participation) of the Intervention Model that vastly improves upon current community corrections services, offering a far more comprehensive and holistic approach to criminal rehabilitation through expanded and integrated programming. The ACCM has been created for high‐risk offenders assessed by a validated screening instrument. The ACCM will combine: community service (both for restitution and as a means to increase skills and services within the community); restorative justice; intensive case management; life skills including cognitive‐restructuring activities; developmentally appropriate education and pre‐vocational programming; transitional employment; job placement and retention; and long‐term follow‐up with program graduates. Roca is seeking to implement the ACCM as an alternative to existing community corrections and incarceration in Massachusetts, using the success of this pilot to implement programming across the Commonwealth and nationally. Roca will also continue to work to develop a tightly held franchise model to share this intervention and the lessons it teaches on a wider basis. Evaluation to Date/Impact of the Model Roca Internal Outcomes – FY 12: Roca’s Intervention Model has shown significant results for very high‐risk young people who are out of school and are street, court and gang involved. In FY 2012, Roca served 409 very high‐risk young men through its intervention model. The following provides some demographics on the young men served:

22% were African American, 1% were Asian, 7% were Caucasian, 59% were Latino, 5% were multi‐racial, and the remaining 6% were of other ethnicities;

89% were street and gang involved;

88% had had prior arrests, 69% had prior convictions, 37% had prior adult incarcerations, 48% had prior adult supervisions, and 33% had prior juvenile supervisions;

80% were high school drop outs;

88% were not employed in the six months prior to enrollment; and

25% were young fathers. Despite the fact that Roca focuses its work on young men who are unable to participate or engage in traditional work or programming, of those eligible for services, Roca retained 78% and engaged them in the intervention model throughout the course of the year. Of those engaged in the intensive component of the model (the first two years) during any period throughout the year, the following provide some detailed performance statistics:

67% of participants in Phase II (the behavior change phase of programming) had intentional, face‐to‐face, conversations with their youth workers a minimum of 2 to 3 times per week;

80% were engaged in stage based programming;

67% were engaged in educational or prevocational programming with 52% demonstrating educational or prevocational gains;

121 participants attained at least one industry‐recognized pre‐vocational certification;

73% demonstrated no new arrests;


Page | 9

67% demonstrated no new technical violations;

51 unduplicated young people were placed in employment and 67% retained that employment through the end of the reporting period (note: Participants are not working towards employment until they have been engaged in the Model for an estimated 18‐21 months)

Of those who completed the intensive component of the model in 2012, 90% have had no new arrests; 100% have had no new technical violations; 79% of program graduates were on track to retain employment for a minimum of 6 months; and 70% have demonstrated education gains. Please see attached for a copy of Roca’s FY 12 Outcomes Report and a Springfield FY 12 Outcomes Highlights for additional details regarding our most recent outcomes. Similar results were demonstrated for participants engaged in the model in 2011. In FY11 (July 1, 2010‐June 30, 2011), Roca served a total of 763 young people through Transformational Relationships and programming to support development in life skills, employment and education. Of those young people, 469 were identified as meeting the very high‐risk criteria for Roca’s Intervention Model (heavily street/court/gang involved, extensive involvement with the criminal justice system, high school drop‐outs, little to no work history, and high‐risk young parents). In fiscal 2011, Roca successfully retained 86% of young people. Of those: 76% had no new arrests; 83% had no new pregnancies (under the age of 24); and, 93% were attending life skills, education and/or employment programming. These outcomes are also sustained for those completing two years of the intervention model. The 225 Roca graduates who received supportive services in FY11 also achieved high outcomes, surpassing all organizational goals: 90% sustained employment and/or remained in educational training; 86% had no new arrests; and 81% had no new pregnancies before the age of 24. Roca’s Evaluation and Performance Based Management: After a recent site visit to Roca where he examined Roca’s performance based management systems, Mario Morino author of Leap of Reason: Managing to Outcomes in an Era of Scarcity, noted: “Roca has gone far beyond simple data capture and reporting. They have developed many of the higher‐level performance management characteristics and skills we addressed in Leap of Reason. They clearly have visionary leadership, a strong and highly specific theory of change, solid definitions of outcomes and measurements, a highly talented and engaged staff, a data collection and reporting system that links those elements together. But perhaps most important, they have created and sustained an organizational culture that values and operationalizes performance management at nearly every level, addressing each area of programming, with clear buy‐in from all staff and constituents. What they achieved is quite advanced and very impressive. “ Since its inception in 1988, Roca has helped more than 17,000 young people make positive, profound changes in their lives, creating a nationally acclaimed model of transformational relationships as a vehicle for youth development, pioneering effective local, regional, and national relationships with government, state, religious, health, and community partners. Roca has spent nearly seven years looking at evidence‐based practices and programs in an effort to more effectively serve our communities highest risk young men who without an intervention face a reality of jail, violence, and poverty. Due to the lack of programs targeted specifically for this target population, Roca felt the need to create an effective intervention model that consistently drives this group of young men toward positive outcomes. Roca intentionally developed its model by:

1. Studying, adapting and implementing elements of evidenced based programs and approaches based on foundational

theories of behavior change; and, 2. Working to align the Intervention Model and Roca’s organizational practice with the Eight Evidenced Based Principles of

Effective Intervention identified for community corrections.

As a result of this learning, Roca now operates a clear, single‐service Intervention model that offers two years of intensive services and two years of follow up support. Roca has made significant progress with the design and coaching of the model and the development of a performance management system, detailed later in this narrative. Roca has also spent a significant amount of time over the last few years evaluating its efforts and outcomes in order to continuously improve the model. Roca’s Intervention Model is designed to ensure that graduates will achieve the sustained outcomes of no incarcerations and retained employment (measured five years from enrollment). Roca’s Intervention Model is aligned with the clinically proven


Page | 10

cognitive‐behavioral Stages of Change, which is managed and tracked through a customized, web‐based software system by Social Solutions called Efforts Toward Outcomes (ETO). Roca enters participant data into the system, including eligibility and enrollment data, individual needs and a service plan to meet those needs, all training, certification, employment and wage data, and all post‐employment services for one year after employment. This system tracks staff efforts, performance indicators, and participant outcomes, and allows us to customize individual and collective queries and reports. Roca utilizes a range of stage‐based outcomes focused on young people living out of harm’s way and moving towards economic self‐sufficiency. Youth Workers select appropriate outcomes to focus on with young people based on their individual issues and barriers to success. Behavioral change outcomes examine: substance abuse; educational engagement; employment engagement; healthy/unhealthy relationships; pregnancy prevention; court compliance; street/gang involvement, etc. Self‐sufficiency measures include: program attendance; program retention; school attendance; overall educational gains; academic skill gains; pre‐vocational skill achievements; progress in transitional employment programming; job placement; and job retention and advancement. ETO captures movement toward intended outcomes through tracking the development of the transformational relationship over time and through tracking the stages of change related to specific behavior change outcomes. ETO data provides management and staff the opportunity to enter and use individual and organizational level data to assess processes and progress toward outcomes. Roca has also aligned its staff development, supervision, and management through a comprehensive performance‐based management system to track individual, program level and organizational level performance indicators.

Level of Evidence of Effectiveness: While Roca began closely examining outcomes recently (in the past five years), we are showing tremendous results to date. We believe that we meet the criteria for “Demonstrated Effectiveness” and with our independent program evaluations underway, are hoping to move to towards “Proven Effectiveness.” Roca engaged in a systemic cycle of research, design, action, tracking data, and use of data for continuous improvement to deliver an intervention worthy of the young people we serve. As a result of this learning and commitment to performance management and continuous improvement, Roca has made significant progress with the design and coaching of the model and the development of a performance management system. In addition to its work internally, Roca worked extensively for several years with the Crime and Justice Institute and Abt Associates and is currently working with Chapin Hall at the University of Chicago on developing performance management reports to continuously improve data quality and accuracy. Roca is showing positive progress and is working to continue improving the data quality and completeness, particularly as it moves to conduct two independent program evaluations in Chelsea/Revere/East Boston and Springfield. Roca has also undergone several implementation and outcome evaluations conducted by the Crime and Justice Institute and Abt Associates. These evaluations are described in more detail below. While we believe that this work establishes Roca’s Intervention Model as having “Demonstrated Effectiveness,” we are eager to pursue rigorous evidence of the Intervention Model’s impact and move towards “Proven Effectiveness.” To do this, Roca has engaged with Chapin Hall at the University of Chicago to conduct an independent program evaluation that will examine the impact of the Intervention Model on very high risk young men aged 17‐24 who are likely to be reincarcerated in both Chelsea/Revere/East Boston and Springfield, MA. A brief description of the evaluation design is included below. Formal Evaluation of the Intervention Model: Few large‐scale, longitudinal evaluations have been completed on comprehensive approaches to intervening with the populations Roca serves, and those evaluations that have been conducted show mixed results. With its holistic approach to intervention work, Roca has the opportunity to demonstrate what it takes to move very high‐risk young people out of poverty and violence. Our agency is uniquely positioned to prove the efficacy of its model through rigorous evaluation, and, if necessary, apply targeted modifications based on the critical analysis. Roca’s sound theory of change, comprehensive data collection system, and, most importantly, its commitment to determine the organization’s impact provides an opportunity to prove the Intervention Model as a solution to poverty and violence. For the past four years, Roca has worked with the Crime and Justice Institute and Abt Associates to assess program implementation, preliminary outcomes and to help prepare for an impact evaluation. Among other work products, this preliminary work included developing: a strategy for examining the actuarial risk level of Roca participants (including the preparation of a risk assessment


Page | 11

tool); and, a measurement device designed to capture the aspects of youth worker‐participant interactions considered most critical to Roca’s model. Roca is now working with Chapin Hall at the University of Chicago (Chapin Hall) on a formal implementation evaluation to demonstrate our model as an evidence‐based intervention that consistently moves high‐risk young people toward positive outcomes. The Crime and Justice Institute and Abt Associates have also completed an Implementation Evaluation of both Chelsea and Springfield. These reports indicate that Roca is not only targeting very high risk young men but is retaining them in a way many other programs are unable to do. In the Chelsea report, they note that Roca is, in fact serving the highest risk population and that Roca is succeeding with this population. Looking at Intermediate outcomes, the evaluators found that of the study sample: 81% maintained a constructive relationship with at least one adult outside of Roca and 84% of the participants did not experience a new arrest during participation with Roca. For Roca, one of the most interesting results of the evaluation was process related. According to the evaluators, one of the most successful indicators of program success was meeting the standard of a minimum of two contacts per week. This is true for both engagement and long term participation. This finding, along with others, has helped Roca guide and move its work forward, coaching staff on the importance of meeting our contact standards for participants. Roca has been assessed and/or benchmarked by several outside researchers, including the Crime and Justice Institute and David Hunter. Overwhelmingly, these reports note that Roca cannot be benchmarked against other programs. The Crime and Justice Institute noted in their report, “Roca is a unique community‐based program serving a population of youth in a city facing a number of daunting problems‐‐‐high crime, high immigration, high unemployment, high teen pregnancy and high levels of school dropouts. It is difficult to find appropriate programming to benchmark Roca’s outcomes given the nature of the community in which the program operates (high crime, high school dropout, poverty), the intensity of the multiple problems the youth face in Roca, and the stage of readiness (or motivation) that these youth are in. Nonetheless, in this setting, Roca is finding success with programming at levels not found in similar programs that serve lower risk populations.” Planned, Rigorous Impact Evaluation: Roca is also working with Chapin Hall to evaluate the long term impact of the Intervention Model on the recidivism and employment rates of very high risk young men. This evaluation will be tied to Roca’s participation in the Commonwealth’s PFS initiative and will specifically examine the Model’s impact on the young men we serve who are: aging out of DYS; aging out of juvenile probation; or, on adult probation, who are determined to be at high risk for reincarceration. The centerpiece of this evaluation will be a Randomized Control Trial which will be “backstopped’ by a Difference in Differences approach.

Recidivism will be assessed using administrative data from the Massachusetts corrections system. Key outcomes will be new arraignments, convictions resulting from these new arraignments, and spells of incarceration. The evaluation will extend to allow assessment of outcomes at least 6 years after referral to Roca, and we anticipate that longer term assessments will be possible as well. While avoided days of incarceration will be the focus of the payment incentives in the pay for success contract ‐ since that is where most of the budgetary savings to the state accrue ‐ the evaluation will also allow an examination of the quantity of types of new offenses committed in both the treatment and control groups, critical components to a full evaluation of the social benefits from reduced criminal activity. To assess successful transitions to adulthood, we will be performing impact evaluations on employment rates, earnings levels, and educational enrollment and achievement. The employment and earnings data will be obtained from the Massachusetts state unemployment insurance records. Educational achievement/enrollment will come from three sources: the state's unified database for secondary education, the state's GED database, and the federal post‐secondary clearinghouse. A detailed plan for this evaluation is included as an attachment to this business plan.


Page | 12

Vision for Influencing Sector or System Change: Targeted investments in corrections reform, particularly in the realm of probation, community corrections and intensive supervision have already proven worthwhile in several other states. Michigan, Kansas, and Connecticut, among others, invested in evidence‐based reentry, probation programming and community corrections and reduced their respective prison populations while saving hundreds of millions of dollars. The up‐front returns on these investments were not only substantial in reducing re‐offense, but the states were able to reinvest the savings into further preventative measures and community‐based supervision programs. Roca will build off of its Intervention Model to serve offenders ages 17–24 and for young people aging out of DYS and juvenile probation. This Model will serve young offenders who are actively involved in the justice system and deemed high‐risk to reoffend by validated risk and needs assessment standards. The model will also continue to serve as a community based intervention for our highest risk young men. An investment in Roca’s intervention model for this population would direct resources to a preventative program that has proven to reduce corrections costs for the target population. This intervention is designed to achieve several important outcomes, each of which has significant associated cost savings, including:

• A reduction in incarcerations as tracked through violations and/or convictions of new crime; • An increase in compliance with court‐ordered conditions; and, • A quantifiable movement towards economic independence through retained employment.

It is expected that the Roca’s Intervention Model will meet all informational needs of participating courts and corrections entities. In accomplishing these specific outcomes, the Intervention Model will demonstrate a new, preventative model for reducing the long‐term public costs associated with these high‐risk populations, specifically related to recidivism and incarceration. If the desired outcomes are achieved, the Commonwealth and other states across the nation may choose to expand these mandated intervention services, thereby achieving systemic reform. Roca’s Intervention Model differs from current community corrections services in its intensity, comprehensiveness and reliance on evidence‐based practices. The Intervention Model vastly improves upon current community corrections services, offering a far more comprehensive and holistic approach to criminal rehabilitation through expanded and integrated programming. Roca is seeking to implement the Intervention Model as an alternative to current community corrections programming and incarceration and as a preventative strategy for young people aging out of DYS and juvenile probation and other very high risk young men. The success of this pilot is intended to drive the implementation of similar programming across the Commonwealth and nationally. Cost‐Benefit Proposition: While conventional criminal justice and public policy has been designed to respond to violence and poverty in urban communities with a limited set of institutional tools, the Intervention Model serves as a progressive alternative that is both cost‐efficient and highly effective. Last year alone, the Massachusetts Department of Corrections held 10,276 criminally sentenced individuals within its jurisdiction. The average cost to house an inmate in the MA DOC per year reached $45,917 in 2010. The length of stay per inmate, male or female, averaged over 4 years. To compound these expenses, the recidivism rate within a three year period post release was recorded at 43% between 2004 and 2007. While many states have already made substantive criminal justice reforms, producing vast savings and reducing crime, Massachusetts continues to invest in an inefficient and ineffective corrections system. On corrections alone, the state spent $1.2 billion in 2009‐ elevating incarceration expenditures above the combined budgets of the departments of higher education, public health and social services.

Social Impact and Rationale for New Approach


Page | 13

Similarly, the Journal of Quantitative Criminology estimated the average lifetime expenses of social services and incarceration for a high‐risk individual between $250,000 and $2,000,000. In The Monetary Value of Saving a High Risk Youth, Mark A. Cohen explains: “Based on a 2% discount rate, the typical career criminal causes $1.3‐1.5 million in external costs; a heavy drug user, $370,000 to $970,000; and a high school drop‐out, $243,000 to $388,000. Eliminating duplication between crimes committed by individuals who are both heavy drug users and career criminals results in an overall estimate of the monetary value of saving a high‐risk youth of $1.7 million to $2.3 million.” [1] In contrast, as noted earlier, Roca invests an average of $22,500 over a 3 to 4 year period – a fraction of what the state spends per prisoner. In other words, 4 full years of comprehensive Roca programming, including our Transitional Employment program, costs less than 1 year of incarceration for an inmate in Massachusetts. Added to these cost savings are additional savings in the administrative costs of the criminal justice system and savings associated with improved court compliance. But Roca’s model should not be viewed exclusively as an alternative to incarceration. In reality, the aggregate, long‐term economic value of the Intervention Model cannot be fully calculated without assessing comparative and combined costs of incarceration, welfare, public health, lost tax revenue, unemployment, law enforcement, substance abuse rehabilitation, etc. The greatest strength of the Intervention Model is that it moves young people toward sustainable financial independence, creating the foundation for a lifetime of productive economic activity.

[1] Cohen is an economist on the faculty of the Owen School of Management, Vanderbilt University. Cohen, Mark A. “The Monetary Value of

Saving a High‐Risk Youth.” Journal of Quantitative Criminology, Volume 14, No. 1, 1998.


Page | 14

Roca is now seeking to expand the impact of its Intervention model, enabling it to reach a larger number of young people from across the country. Specifially, Roca will expand its programmiing to better serve very high risk young men, aged 17‐24, who are either 1) aging out of the juvenile justice or juvenile probation systems with a strong propensity for reincarceration as an adult; 2) connected with the adult justice system; or 3) are high risk members of the community being served who have a strong propensity for incarceration as an adult. Roca is in an early‐stage growth for impact that meets the criteria of “organizations that have demonstrated a significant level of effectiveness and are increasing their capacity for scale readiness.”7 Roca is proposing to scale social impact by increasing both direct services (quantity of impact / geographical scale) and indirect influence (promotion of the Intervention Model). Roca’s growth plan includes an increase in direct services and indirect influences through:

1. the quantity of services and geography through expansion of services at our existing locations and through two additional program replications, growing from serving 710 to 1,755 young men per year8

2. the quality of services through expanding the organization’s delivery capacities and demonstrating the Intervention Model as an evidence based practice; and,

3. influencing public policy and practice through demonstrating alternative programming models, advocacy, the Roca engaged institution strategy, networks, and a knowledge development/sharing agenda.

In conducting this work, Roca will: 1) demonstrate the efficacy of its model; 2) create adequate resources to support this expansion as well as other work serving this very high‐risk population; 3) disseminate information about the Intervention Model and the efficacy of working with this target population; and, 4) create the political will locally, regionally and nationally to support services for this often neglected target population. Roca will also work to develop partners to help champion these efforts. Finally, Roca will learn more about its impact and make a determination regarding the appropriate methodology for long term scale and impact. Roca is proposing to scale social impact by increasing both direct services (quantity of impact / geographical scale) and indirect influence (promotion of the Intervention Model). Overall, Roca has two critical goals for its strategic growth: 1) Demonstrate the Intervention Model for Very High‐Risk Youth as evidence‐based and expand use of the model including replicating the Intervention Model for Very High‐Risk Youth; and, 2) Increase organizational capacity for scale, impact, and sustainability. Roca will begin this early stage growth work regionally and then expand nationally, utilizing the following strategies, each with its own specific milestones:

7 As defined by the Social Impact Exchange and found at http://www.socialimpactexchange.org/bpc_overview.cfm 8 Roca will also increase its services delivery to young mothers increasing from 200 young mothers being served in FY 13 to 388 young mothers

being served in FY 17.

Growth Plan


Page | 15

SERVICE CAPACITY9

A. Expand Roca’s impact on high risk young men. Roca will increase its services from 710 young men in FY 13 (535 in intensive and 175 in retention services) to 1,755 young men in FY 17 (1,100 in intensive and 655 in retention services young men through the expansion of services out of the Chelsea and Springfield hubs and the replication of the Intervention Model in two new locations. Roca will grow its programming for young men, expanding the number of young men served at its existing sites through expansion into contiguous communities. Roca will also expand the number of young men it serves through the startup of two new replication sites, one to being operations in FY 14 and the second to begin operations in FY 16. The following table illustrates the number of young people served at each proposed site. The Chelsea and Springfield are broken out between young men who will be part of the pay for success project and those who will not.

FY 2013 FY 2014 FY 2015 FY 2016 FY 2017

Primary Service Population – Very High Risk Young Men

Chelsea (Non‐Pay for Success) Intensive Services (Years 1 and 2) 235 150 150 202 297

Retention Services (Years 3 and 4) 130 132 172 120 126

Chelsea (Pay for Success Only) Intensive Services (Years 1 and 2) 100 225 225 173 78 Retention Services (Years 3 and 4) 0 0 69 175 202

Chelsea Total Intensive Services (Years 1 and 2) 335 375 375 375 375


Replication 1: Springfield (Non‐Pay for Success)

Intensive Services (Years 1 and 2) 100 50 50 66 151


Replication 1: Springfield (Pay for Success Only)

Intensive Services (Years 1 and 2) 100 175 175 159 74


Replication 1: Springfield Total Intensive Services (Years 1 and 2) 200 225 225 225 225


Replication Site 2 Intensive Services (Years 1 and 2) 0 100 200 250 300


Replication Site 3 Intensive Services (Years 1 and 2) 0 0 0 100 200


Total Young Men Served Intensive Services (Years 1 and 2) 535 700 800 950 1100


B. Develop and implement an Alternative Community Corrections Pilot.

Roca will work with the Commonwealth of Massachusetts, and other locations as appropriate, to develop and implement a pilot project demonstrating the Intervention Model as an alternative to incarceration or community corrections.

C. Apply the Intervention Model and Adjust Appropriately for Very High Risk Young Mothers, increasing services from 200 young mothers in FY 13 (100 young mothers in healthy families and 100 high risk young mothers in intensive services) to 338 young mothers in FY 17 (100 young mothers in healthy families and 200 high risk young mothers in intensive and 38 in retention services.) The following table illustrates the number of young people served out of the Chelsea site. This includes 100 young mothers per year who will be served through the Healthy Families Home Visiting Program. In addition Roca will growth its services for very high risk young mothers to serve 200 young mothers in intensive services and an additional 38 young mothers in retention and support services by FY 17.

9 For additional details regarding Service Expansion please see the Increasing Service Capacity – Program Expansion and Replication section of

this business plan, starting on page 31.


Page | 16

FY 2013 FY 2014 FY 2015 FY 2016 FY 2017

Secondary Service Population – Very High Risk Young Mothers

High Risk Young Mothers Intensive Services (Years 1 and 2) 100 100 100 150 200


Healthy Families Young Mothers Healthy Families – Home Visiting Clients 100 100 100 100 100

Total Young Mothers Served 200 223 238 237 338

PROGRAM QUALITY AND EVALUATION SYSTEMS10

D. Expand Roca’s Impact on very high risk young men. Roca will work to improve its program quality while working to: ensure fidelity to the program model throughout program expansions; and 2) enhance business development and partnerships to ensure an appropriate number of transitional employment and unsubsidized employment positions are available for participants.

E. Demonstrate Intervention Model for Very High Risk Young Men as Evidence Based and expand the use of the model.

1. Roca will work to complete model codification and documentation.

2. Roca will work with a third party evaluator to administer annual implementation and outcome evaluations addressing each operating site.

3. Roca will conduct a comprehensive impact evaluation of the Intervention Model as it serves very high risk young men, aged 17‐24 who are involved with the criminal justice system or have a high propensity to be involved with the adult criminal justice system.

F. Apply the Intervention Model and adjust appropriately for use with very high risk young mothers.

1. Finalize and pilot adjusted intervention model for high risk young mothers demonstrating consistent outcomes over time.

2. Codify and document the intervention model as it applies to high risk young mothers.

3. Within three years develop stable funding source for young mothers programming.

ORGANIZATIONAL STRENGTHENING11

D. Increase organizational staffing, leadership and management capacity for scale, impact, and sustainability, ensuring sufficient capabilities to operate a minimum of four sites in multiple states. 1. Management

Develop and coach leadership and management competencies including related performance based management and scorecard.

Increase capacity of fiscal, administrative, and HR systems and staffing for quality and growth.

Ensure appropriate staffing team to meet organizational goals.

2. Board of Directors

Increase capacity to lead organization through growth objectives.

Increase fundraising capacity

10 For additional details regarding Program Quality and Evaluation Systems please see the sections Demonstrating the Model as Evidence Based,

starting on page 8 and Next Steps in Performance Based Management starting on page 41. 11 For additional details regarding Organizational Strengthening please see the sections Organization and Management, starting on page 39.


Page | 17

E. Increase development capacity for sustainability, ensuring sufficient capabilities to operate a minimum of four sites in

multiple states. Roca will develop and implement a broad based communications strategy.

F. Create long term strategy to demonstrate increased impact of the Intervention Model. 1. Determine and put in place appropriate management, legal and board structures for replication

2. Plan for and start up second full scale program replication and document lessons learned for replication and information

sharing efforts.

3. Plan for and start up third full scale program replication and document lessons learned for replication and information

sharing efforts.

4. Develop and support advisory boards for replication sites.

5. Investigate and consider application of intervention model to other populations.

6. Create long term plan for growth and impact.

FINANCES12

D. Develop, launch and complete 25th anniversary campaign (including increased focus on individuals and corporations).

E. Explore and position Roca for additional public funding sources/strategies (e.g. pay for success financing, alternative community corrections financing, and government contracting

F. Ensure Roca’s long term financial stability through resource development activities including growing endowment to $5 million and operating reserves to 3‐4 months of operating cash.

12 For additional details regarding Finances please see the Financials section of this business plan, starting on page 43.


Page | 18

Milestones and Deliverables

Measurable Five‐Year Performance Goals: Based on Roca’s plans for early‐stage growth, which include program replication and expansion, program evaluation and developing a clear strategy for long term scale and impact, Roca has developed a clear implementation plan, illustrated below, that includes detailed timetables, milestones, and measurable three year performance goals for this plan. FY13 Goal FY 14 Goal FY 15 Goal FY 16 Goal FY 17 Goal

Service Capacity

Expand Roca’s impact on very high risk young men (in number and geography)

Young people served through TR's:

Total Young Men ‐ Primary Population 710 918 1,186 1,497 1,755

C/R/EB 465 507 616 670 683

Springfield 245 311 370 406 421 Operating Site 3 0 100 200 321 451 Operating Site 4 0 0 0 100 200

Young Men Served In Chelsea/Revere/East Boston Site

Young People In Primary Target Population: Serve 375 young men through the addition of new slots, expansion into new communities and the use of Pay for Success Financing.

335 very high risk young men served through intensive transformational relationships and stage based programming.





Provide services to 308 young people in Phase III transformational relationships (program graduates), providing ongoing retention and advancement support.

Provide retention and advancement services to 130 very high‐risk young people per year.





Expand transitional and advanced transitional employment opportunities for young people in transformational relationships.

Total 46‐58 slots serving 112‐136 young people

54‐64 slots serving 132‐153 young people




Basic Transitional Employment (ratio of 3 young people per slot)






Advanced Transitional Employment (ratio of 1.5 young people per slot)






Coming of Age Staff (ratio of 1 young person per slot)

5‐7 slots 5‐7 slots 5‐7 slots 5‐7 slots 5‐7 slots


Page | 19

FY13 Goal FY 14 Goal FY 15 Goal FY 16 Goal FY 17 Goal

Expand job placement services to ensure that a minimum of 75% of those that complete 2 year model are placed in unsubsidized employment.

150 young people will be placed in unsubsidized employment





Young People Served in Springfield Site








Provide retention and advancement services to 45 very high‐risk young men.





Expand transitional and advanced transitional employment opportunities for young people in transformational relationships in Springfield.



















1 slot 2 slots 3 slots 3 slots 3 slots

Expand job placement services to ensure that a minimum of 75% of those that complete 2 year model are placed in unsubsidized employment.






Young People Served In Third Operating Site (to be determined)










Page | 20


Expand transitional and advanced transitional employment opportunities for young people in transformational relationships in third location.
















N/A N/A 1 slot 2 slots

Expand job placement services to ensure that 75% of those that complete 2 year model are placed in unsubsidized employment.




Young People Served In Fourth Operating Site (to be determined)




Expand transitional and advanced transitional employment opportunities for young people in transformational relationships in third location.










N/A N/A


Page | 21


Develop and Implement an Alternative Community Corrections Pilot

Develop and implement an alternative community corrections pilot.

Pilot small alternative community corrections program in Chelsea and Springfield locations

Continue to advocate for larger alternative community corrections approach in Massachusetts.

Respond to any Commonwealth of MA RFP’s allowing Roca to serve as a formal alternative to incarceration.

Continue to operate and work to expand alternative community corrections program in Chelsea and Springfield locations

Continue to advocate for the expansion of alternative community corrections in MA.

Advocate for use of Roca’s model as an alternative to incarceration in replication sites.










Apply the Intervention Model and adjust appropriately for very high risk young mothers

Youth Mothers Served

Total Young Mothers Served in C/R/EB 710 918 1,186 1,497 1,755

Healthy Families 100 100 100 100 100

High Risk Young Mothers 100 145 175 223 275

Serve 100 young mothers through Healthy Family Home Visiting Program

100 young mothers provided with home visiting services.





Serve 200 very high risk young mothers through an adapted version of the intervention model

100 very high risk young mothers served through intensive transformational relationships and stage based programming.






Provide retention and advancement services to 45 very high‐risk young mothers.





Page | 22


Program Quality and Evaluation Systems

Expand Roca’s impact on very high risk young men (in number and geography)

Improve program quality by ensuring fidelity to the program throughout scaling process.

Complete research with other programs who have conducted large scale replications around development of fidelity protocols.

Develop and implement Roca cross‐site fidelity protocols.

Continue to refine and utilize Roca’s performance based management tools to ensure program quality and fidelity to the model across operating sites.

Administer all fidelity protocols in replication sites and continue to provide intensive training for all staff around the implementation of the intervention model.


Continue to administer all fidelity protocols in replication sites and continue to provide intensive training for all staff around the implementation of the intervention model.






Improve program quality by enhancing business development and partnerships (TEP/Job Creation)

Create business development strategy to provide for expanded business development across sites.

Ensure all business development work is aligned with Roca stage based programming and pre‐vocational training efforts.

Continue to refine and implement business development strategy to provide for expanded business development across sites.





Page | 23


Demonstrate Intervention Model for very high‐risk young men as evidence‐based and expand use of the model

Complete model codification and documentation Finalize and implement expanded training agenda associated with performance based management.

Completion of materials, curriculum, operation manuals, standardized supervision tool etc.

Build out remaining performance management reports

Develop measurable benchmarks for all other organizational components

Draft complete organizational scorecard and pilot early winter 2013 and use for FY’14 with a review at the end of the year.

Revisit organizational competencies and update competency framework and associated performance based management tools.

Update materials based on progress of evaluations.

Implement with continuous improvement recommendations.


Implement third party Implementation and Outcome Evaluation each year addressing each operating site.

Complete annual implementation and outcome evaluations.





Conduct a comprehensive impact evaluation of the Intervention Model as it serves very high risk young men, aged 17‐24 who are involved with the criminal justice system or have a high propensity to be involved with the adult criminal justice system.

Raise financing for impact evaluation.

Complete Year 1 of impact evaluation.

Complete year 2 of impact evaluation.



Apply the Intervention Model and adjust appropriately for very high risk young mothers

Finalize and pilot adjusted intervention model for high risk young mothers demonstrating consistent outcomes over time.

Complete theory of change for high risk young mothers. Pilot refined intervention model for high risk young mothers.

Continue to operate and refine intervention model for very high risk young mothers.





Page | 24


Codify and document the intervention model as it applies to high risk young mothers.

Document program model, curriculum, operation manuals, standardized supervision tool etc.

Create appropriate performance management reports

Develop measurable benchmarks program operations.

Draft programmatic scorecard.

Review and update all program materials and manuals.

Full implement all performance based management tools

Assess project against measurable benchmarks using programmatic scorecard. Make changes as appropriate.

Update materials based on progress of evaluations.



Within three years develop stable funding source for young mothers programming.

Develop and implement fundraising strategy to support expanded services for high risk young mothers and develop sustainable funding source for project.

Continue to implement fundraising strategy to support expanded services for high risk young mothers and develop sustainable funding source for project.

Continue to implement fundraising strategy to support expanded services for high risk young mothers and develop sustainable funding source for project.

Review and update fundraising strategy

Implement updated fundraising strategy.


Page | 25


Organizational Strengthening

Increase organizational capacity for scale, impact, and sustainability.

Increased Management Capacity

Develop and coach leadership and management competencies including related performance based management and scorecard.

Develop competency based staffing structure for all senior management. This will include all associated performance based management tools.

Assess all management based on defined competency requirements.

Create staff development/coaching plan.

Develop organization wide scorecard and subsidiary scorecards.

Utilize staff competency structure for all management hires.

Implement staff development and coaching plan and update as necessary.

Utilize organizational scorecards to support staff and leadership development.



Utilize organizational scorecards to support staff and leadership development







Increase capacity of fiscal, administrative, and HR systems and staffing for quality and growth.

Review fiscal administrative and HR structures and determine capacity building needs for growth.

Restructuring Development and Finance functions and change role of Chief Knowledge Officer to include all human resources activities.

Make all appropriate and necessary staffing changes.





Page | 26


Ensure appropriate staffing team to meet organizational goals.

Develop sustainable staffing structure for the organization’s senior management team.


Update emergency succession plan and create long term succession plan.




Update emergency succession plan and create long term succession plan.


Increased Development Capacity

Development and implementation of broad based communications strategy.

Work with communications consultant to develop and implement a broad based communications strategy supporting Roca’s work.

Run informational sessions to provide information about Roca to a minimum of 200 people per year.

Expand Roca’s digital media presence through the use of blogs, social media and online newsletters.

Complete work on Roca’s branding, including a refined mission and vision statement.

Build in‐house capacity for communications work.

Continue to implement strategy, update as necessary.





Page | 27


Increased Board of Directors Capacity

Increase capacity to lead organization through growth objectives.

Create and implement board development strategy.

Create training opportunities for Board leadership.

Expand board membership to address growth capacity needs.

Implement board development strategy and refine as necessary.




Increase fundraising capacity Continue to work with Board of Directors to increase fundraising capacity through expansion of Board Membership and strengthening of development committee.

Establish board give and get policy.

Develop and implement fundraising strategy for Board of Directors and Development Committee.

Implement fundraising strategy for Board of Directors and Development Committee and update as necessary.

Implement fundraising strategy for Board of Directors and Development Committee and update as necessary




Page | 28


Creation of Long Term Strategy for Increased Impact of the Intervention Model

Determine and put in place appropriate management, legal and board structures for replication

Develop planning board to assist with replication planning.

Complete analysis of management competencies and structuring of leadership team.

Work with legal counsel to determine appropriate legal and board structures for replication.

Continue to work with planning committee for replication sites.

Implement staffing and structural changes based on management competency and structuring work.

Create appropriate legal and board structures for new replication site.


Continue to monitor necessary changes in management and board capacity and address as necessary.

Continue to work with planning committee for replication site and develop new committee for third replication site.


Create appropriate legal and board structures for new replication site.



Plan for and start up second full scale program replication and document lessons learned for replication and information sharing efforts.

Finalize site selection and begin full year of planning and fundraising for second site operations.

Launch second full scale replication.

Continue to operate second full scale replication.



Plan for and start up third full scale program replication and document lessons learned for replication and information sharing efforts.

Begin vetting process for selection of third replication site.

Finalize site selection and begin full year of planning and fundraising for second site operations.

Launch third full scale replication.

Continue to operate third full scale replication.

Develop and support advisory boards for replication sites.

Set and meet annual goals for the Springfield Project. Set up Advisory Board for second replication site.

Set and meet annual goals for the Springfield Project and second replication site.

Set up Advisory Board for third replication site. Set and meet annual goals for the Springfield Project and second replication site.

Set and meet annual goals for the Springfield Project and second and third replication sites.

Set and meet annual goals for the Springfield Project and second and third replication sites.

Investigate and consider application of intervention model to other populations.

Investigate and consider application of intervention model to other populations.

Continue to investigate and consider application of intervention model to other populations.




Page | 29


Create long term plan for growth and impact Convene regular meetings of advisory committee for replication and scale (Board of Directors Strategic Planning Committee). Committee to provide recommendations regarding long term scale and impact. Provide support to planning and startup of second replication.

Finalize a strategic plan to scale impact and beginning planning and fundraising efforts against that plan. Provide support to planning and startup of second replication.

Begin to implement strategic plan for scale and impact. Provide support to planning and startup of third replication.

Continue to implement strategic plan for scale and impact. Provide support to planning and startup of third replication.

Continue to implement strategic plan for scale and impact.

Finances

Increase organizational capacity for scale, impact, and sustainability.

Develop, launch and complete 25th anniversary campaign (including increased focus on individuals and corporations).

Increase capacity of development team including hiring a second Development Associate and adding increased staff focus on individuals and corporations. Develop comprehensive campaign strategy. Run silent phase of $25 million anniversary campaign. Begin full scale implementation of $25 million anniversary campaign.

Continue to administer $25 million anniversary campaign.

Continue to administer and close out $25 million anniversary campaign.


Page | 30


Exploration and positioning for additional public funding sources/strategies (e.g. pay for success financing, alternative community corrections financing, and government contracting),

Increase the percentage covered in MA, with rates determined each year.

Ensure this source represents a predominant source of funding in any new site.

This may include using financing tools such as “pay for success” and “social impact bonds”

Implement Roca’s comprehensive strategy to establish a predominant source of public funding for the Intervention model and update as needed.




Ensure Roca’s long term financial stability through resource development activities including growing endowment to $5 million and operating reserves to 3‐4 months of operating cash.

Achieve FY13 goal of $13.9 million in revenue. Increase endowment by $200,000. Maintain 3‐4 of cash in operating reserve.






Page | 31

Increasing Service Capacity – Program Expansion and Replication Roca’s Intervention Model has shown significant promise that can help this very high‐risk group of young people make significant changes that will help them escape violence and poverty. While Roca has shown tremendous success in Chelsea, Revere and East Boston, to truly demonstrate its effectiveness this model must work in more than one location and for more than one group of young people. In order to gather this proof, Roca has developed plans to expand and replicate its services, evaluate those services and then plan for long‐term scale and impact. In order to begin this process, Roca began its replication work in Springfield, MA. This allowed Roca to begin in a city with whom the city had a partnership and at the same time was within reasonable travel distance from Chelsea. However, Roca plans for its future replications with the understanding that it will need to expand both geographically and into cities where new relationships must be developed. Roca has also opted to time this work in conjunction with its impact evaluations, allowing for the generation of independent data support the outcomes and effectiveness of the Intervention Model. This will provide the organization with a combined set of hands‐on, qualitative and quantitative data that will help in formulating Roca’s plan to effectively scale impact and further replicate. Program Expansion: Roca’s growth plan includes an increase in direct services and indirect influences through: an increase in quantity of services through expansion in Chelsea/Revere/East Boston; an increase in quantity of services and geography through program replication; and an increase in quality of services through expanding the organization’s delivery capacities. Specifically, in an effort to serve more street/court/gang involved youth, Roca has proposed to expand and focus the intervention Model to serve very high risk, disengaged young men involved with the criminal justice system. This will allow Roca to grow from serving 710 very high risk young men in FY 2013 to 1,755 young people by the close of FY 2017. Each of these young men will be between the ages of 17 and 24; be justice system involved or juvenile justice system involved with risk indicators predictive of adult criminal justice involvement; have no employment history; have no high school diploma or GED; and live in a contextually determined radius that promotes participation in center based services. Roca will achieve these increases through the implementation of two expansion strategies: 1) Expansion into new communities through our existing sites; and 2) replication of the program model in two additional sites.

Expansion into New Communities through Existing Sites: In the past several years Roca has focused its services on the highest risk young men in the communities we serve, which has reduced the number of men in each community who are appropriate for Roca services. However, with the addition of services in contiguous communities in MA, Roca’s services are projected to expand in each current facility. This will include adding services in Cambridge, Everett, Lynn, Malden, Medford, Somerville, Winthrop, Agawam, Chicopee, Holyoke, Ludlow, West Springfield and Westfield. Adding services in these continuous communities will allow Roca’s Chelsea site to increase from 465 young men in FY 13 to 683 young men in FY 17 (including 375 young men in intensive services and 308 young men in follow up services.) Similarly, Roca’s Springfield site will move from serving 245 young men in FY 13 to 421 young men in FY 17 (including 225 young men in intensive services and 196 young men in follow up services.)13 Roca is currently implementing a contiguous community strategy that will allow it to successfully expand into these communities. This work includes developing relationships with local law enforcement, political representatives and service providers. Much of this replication work will be financed through the pay for success contract Roca is currently negotiating with the Commonwealth. In addition, Roca has planned to finance a portion of this expansion and for continued operations in Springfield through its 25th Anniversary Fundraising Campaign, described in detail later in this business plan.

13 Please note: these numbers include projections based on the MA pay for success pilot beginning operations in March 2013.


Page | 32

Expansion through New Program Replication Sites: Chelsea and Springfield are both philanthropically underserved communities, as the bulk of the philanthropic resources within Massachusetts are targeted to services within the City of Boston. Perhaps more importantly, in addition to the communities being underserved, Roca’s target population of highly disengaged, criminally involved young men is philanthropically underserved across the country and the communities and neighborhoods where they reside are often our poorest and most philanthropically under‐resourced. Despite the lack of resources, many of these communities face problems of crime, poverty and youth disengagement far surpassing some of our larger, more highly resourced communities.

It is expected that it will take an estimated year of research to determine Roca’s next appropriate site. While several organizations have come to Roca to discuss the possibility of replication within their communities, Roca has established a deliberate process to select new sites.

Roca looks for municipalities with whom there is some natural connection to Roca and its staff and state agencies that operate in a way that will support money, with particular attention paid to how funds are managed. Specifically, Roca will select specific sites for replication based upon a set of defined selection criteria, described below.

1. Roca seeks to replicate its services in communities that have expressed an interest in having Roca come into their

community. In other words, Roca seeks to replicate its services in communities where it is invited to work.

2. Roca must be able to determine clear need for the program. To do this there must be a documented value proposition for the work and there must be an adequate number of young people in need of services to bring the program to scale (250‐300 young people) with the potential for additional growth in the state and/or by intervention model.

Ideally, this need can be documented through public data (i.e. data regarding rates of violence and crime among young men.) In addition to a critical mass of young people to be served, Roca also looks to ensure that there is an unmet need for services and/or gap in services that is local recognized. Additionally, Roca must be able to develop a clear mapping of institutions, programming and resources, as well as identified gaps within the local service delivery system.

3. Roca looks for four critical lead local partners in each community: 1) a strong criminal justice leader; 2) a strong

political leader; 3) an individual with demonstrated credibility with street/court/gang involved young people; and, 4) because we are targeting philanthropically underserved communities, a business partner who can help build credibility and generate funds for programming.

4. Using the same practices it currently implements for its engaged institutions work, Roca will work to ensure that it has

developed relationships with additional key criminal justice partners and other key service providers either through contracts and/or referrals to ensure adequate levels of services will be available for the young people we serve.

5. During the planning phase of any new replication, Roca will develop a high level planning board from the targeted

community who will assist Roca in navigating the community and generating an appropriate level of funding to support start up and operations. This committee will include lead partners, business leaders and other key supporters. This group will initially focus on politics and fund raising and will ultimately shift into an advisory committee for the replication site. Finally, Roca will work to develop relationships in each site with key service providers and other community partners to support ongoing operations. At least one member of each of these committees will also be on the Roca Board of Directors.

6. Any selected location must have “a place to be” or an appropriate physical location from which Roca can operate. The location must be approximately 5,000 square feet for the building with additional space for parking. Additionally, the space must be located in an accessible place for the majority of the target population and on public transportation routes with consideration for gang neutrality. To support this, Roca must have a clear capital campaign strategy and a place where it might operate prior to the completion of the capital work.

Finally, for the second site replication, to begin in FY 14, the Roca replication team must be able to travel to and from the site in one day.


Page | 33

Timing for Expansion: Roca has also learned that because of the complicated nature of the intervention model and the potential risks associated with it, that program replication cannot be rushed. Rather, time must be taken to both build relationships and develop adequate resources to ensure long‐term sustainability. For example, from replicating programming in Springfield, Roca has developed a critical understanding of the partnerships needed for success and the time it takes to effectively develop those partnerships. Developing these partnerships took a significant period of time. As such, as Roca plans to conduct work in new geographic areas, Roca sets aside a full year from the point a site has been selected prior to beginning program operations to work on building these relationships. Required Financial Resources for Replication: Because of Roca’s focus on philanthropically underserved and under‐resourced communities and populations, a standard has been developed for:

1) raising a minimum of $4 million over a two to three year period prior to expansion into any new city; 2) developing a comprehensive development strategy that both addresses community needs and ensures long‐term financial

sustainability; 3) creating a viable capital plan to address needs for physical building, van and equipment purchase; 4) developing a comprehensive plan for creating transitional employment and job placement opportunities; and, 5) working with a Board of business people in each community to help raise appropriate resources.

Roca has developed a fundraising plan to be used in helping to reach it overall programmatic goals and will work with its Development Committee of the Board of Directors, the Springfield Advisory Committee and Advisory Committees for nay new sites to develop the required resources. Roca also understands that we must learn more about the resources required to effective implement this work. As such, Roca will rely on our Scale and Impact committee for assistance in understanding and planning for these needs. Business Imperatives

The following business imperatives must be addressed in order to move forward with scaling:

Business Imperative Tasks

I. Ensure Fidelity to the Roca Intervention Model

a. Complete programming transition to an intensive two year model with two years of follow up. b. Complete curriculum, program operations manuals, staff training, coaching strategies, and

other staff development tools needed in order to ensure fidelity to the model. c. Complete performance based management tools, system and implementation to enable Roca

to drive to outcomes. d. Continue outcome and implementation evaluations as needed. e. Finalize plans and raise funds for impact evaluations and develop ongoing evaluation plans for

replication sites.

II. Determine Pathway to Sustainable Government Funding

a. Explore ways to utilize pay for success financing and/or social impact bonds. b. Complete and implement plan to leverage dollars for an alternative community corrections

model.

III. Further Develop Organizational Capacity to Implement Growth Strategy

a. Further develop and codify current multi‐source funding strategies to include: i. Increased individual and corporate support; and, ii. Sustainable business support to finance transitional employment.

b. Determine appropriate organizational and management structures to support growth plans while protecting core operations.

c. Develop capacity of organizational leadership to meet growth needs: i. Management ii. Board of Directors.


Page | 34

Business Imperative Tasks

IV. Develop and Launch Organizational Replication Strategy

a. Develop comprehensive site selection methodology and tools. i. Determine minimum entry requirements for any new replication site ii. Determine strategy to meet physical plant needs; iii. Funding support required; and, iv. Political support required/leadership in the community.

b. Develop launch process and year 1 start up support to be provided i. Time table for program start up and implementation ii. Coaching and staff training iii. Management requirements

c. Develop systems for ongoing site support d. Implement Replication Strategy

To conduct this work, Roca launched a strategic committee of the board, including non‐board members, to assist the organization in addressing this strategic challenge.

Advocacy for Growth and Expansion

In its role in working with very high risk young people, Roca is currently participating in an advocacy agenda to have legislation and funding available for criminal justice reform, including the promotion of an alternative community corrections model and leverage public dollars as a source of funding to support this work. Roca is participating in this work as part of the “Smart on Crime” campaign, currently being led by MassInc. Nationally, community corrections and criminal justice reform efforts are focusing on low and moderate risk populations. Unfortunately, addressing community corrections with these populations does not allow criminal justice systems to maximize their impact or cost savings. As part of its strategic growth strategy, Roca is seeking to shift this focus to address high risk populations, specifically high risk young people. Roca will work with its criminal justice partners to support a sub‐campaign around this issue, looking at work on a local, state and national scale. Roca will start to pull practitioners together (including government) to see what’s possible for 17‐24 year olds. Roca will develop a full advocacy plan, by Spring, 2013, detailing the scope of the work involved with this advocacy work. It is estimated that Roca will require staffing assistance (.5 FTE in and FY 13, 1 FTE thereafter) as well as some additional, outside support to help with state and federal advocacy and positioning.


Page | 35

Mission, Vision and History

Roca loves young people and relentlessly drives to get them out of violence and go to work. Roca is demonstrating a powerful solution to violence and poverty through its Intervention Model for very high risk young people. Serving young people in Chelsea/Revere/East Boston and Springfield, MA, Roca works with individuals that no one else wants to help. We focus all of our work on achieving long term outcomes for these young people. Roca is about relentless outreach and remarkable results. Roca sees a future in which young people leave the streets and gangs, take responsibility for their lives, and succeed at work. Young immigrant mothers will raise their children in safety and be recognized for their contributions to society. Our communities will have the ability to keep young people out of harm’s way and, in turn, thrive through their participation and leadership. We use concrete measures to track positive change toward mutually agreed upon life outcomes with highly disengaged individuals for whom other programming has not worked.

Roca is unique in the work that it does.

1. Roca seeks out the most difficult, challenging young people – the young people who are unwilling or unable to attend traditional programming, work or school.

2. Roca’s services are designed to work with young people who are not prepared to participate in traditional programming and all program components address issues of relapse, using failure as a tool to help young people learn.

3. Roca is an outcomes‐driven organization and has created an Intervention Model to change the lives of our young people and our communities.

4. Roca works with systems to change how they work with those young people. 5. Roca utilizes evidenced‐based practices combined with experience in working with this target population to effectively

intervene with very high‐risk young people (for more information, please see “Research and Evidence that Informs the Model” in the attachments to this business plan.

Founded in 1988, Roca is a youth development organization committed to serving the most disenfranchised and disengaged young people ages 14‐24 (street/court/gang involved; drop‐outs; young parents; and refugee and immigrants). Through intensive relationship building, Roca helps young people re‐engage in society ‐ moving them into educational, employment, and life skills programming. Launched from a single grant from the Teen Challenge Fund of Massachusetts, a passionate group of young people and adults started Roca to address teen pregnancy prevention and violence in Chelsea. Roca has helped more than 17,000 young people make positive, profound changes in their lives, creating a nationally acclaimed model of transformational relationships as a vehicle for youth development, pioneering effective local, regional, and national relationships with government, state, religious, health, and community partners. Most importantly we love and believe in young people. It is our enormous privilege to spend time with them and support them in their efforts to change. Over the next few years we hope to get better at what we do, expand our impact, and be worthy of serving the young people that need our attention.

Theory of Change

When young people are re‐engaged through positive and intensive relationships they can gain competencies in life skills, education and employment that move them toward living out of harm’s way and toward economic independence. The following image illustrates Roca’s Theory of Change and our current basis for performance based management.

Organizational Structure and Management

Roca Strategic Business Plan

FY 2013 to FY 2017 Page | 36


FY 2013 to FY 2017 Page | 37


FY 2013 to FY 2017 Page | 38

Based on this theory of change, the following image illustrates the logic model that underlies Roca’s Intervention Model.

Inputs Strategies/Activities Outputs Performance Measures Intermediate Outcomes

Long Term Outcomes

Personnel

Funding

Community partners

Referral sources

Facility/physical

Resources

Advisors

Evaluation Database

Performance Management System

Staff Training/ Development & Supervision

Outreach


Engagement Activities

Stage Based Life Skills

Health Services

Stage Based Education

Stage Based Pre‐Voc

Stage Based Job Readiness

Transitional Employment

Job Development

Engaged Institutions

# in TRs

# who move to a P2 TR

# attending programming and/or engagement activities

# attending Life Skills Programming

# Attending Educational Programming

# Attending Pre‐Voc/Job Readiness Programming

# with Educational

Baseline Assessment

# Completing Pre‐Voc Certifications

# in Transitional Employment

# with Completed

Workforce Readiness Criteria

# who attended a

clinic visit

# placed in unsubsidized employment

% Retained in Transformational Relationships (n=all TRs *14)

% Meeting frequency dosage‐min of 2 actual contacts/wk after 2 months from enrollment (n=all TRs enrolled 2 months or more)

Has Service Plan within 6 months of enrollment (n=all TRs enrolled 6 months or more)

% Meeting programming attendance standard‐min 1x/wk to any programming/activities after 2 months from enrollment (n=all TRs enrolled 2 months or more)

% Attending Stage Based Programming (Life Skills, Education, Pre‐Voc, Job Readiness) min. 2x/wk after 4 months from enrollment(n=all TRs enrolled 4 months or more)

% Demonstrating gains on Educational Assessments (n=participants with baseline assessment and at least 1 follow up)

% Meeting Workforce Readiness Criteria‐completed checklist of all 4 domains for WFR by 18 months from enrollment (n=all TRs enrolled at 18 months)

% who attended a minimum of 1 clinic visit @ Clinic within 12 months from enrollment (n=all TRs enrolled at 12 months)


–minimum of 3mo

Academic Gains—

Increased GED/ESL testing scores;

on track for grade promotion

No New Arrests and No New Technical Violations—

Min of 6mo

Sustained Employment Retention & Advancement

Sustained Academic Gains

No New Incarcerations

14 All TRs refers to all participants engaged in the model with the exclusion of those determined as ineligible within 60 days of enrollment and those dismissed for the following

reasons: 1. Moved out of geographic location, 2. Long‐term incarceration‐‐ only if participant had open case with no disposition at enrollment and no new arrests while enrolled


Page | 39

Organizational Structure

Current Organizational Structure: Roca, a 501(c)(3) corporation, has extensive experience in receiving, managing and expending large federal, state and private foundation grants. Currently, Roca manages an $8.6 million annual operating budget, with no debt, and has consistently maintained clean audits on an annual basis. Roca maintains all accounting, contract management, purchasing and recordkeeping functions in accordance with generally accepted accounting procedures. Roca is committed to being a high‐performing and adaptive organization that uses real‐time data for continuous improvement and evaluating our impact. Roca’s intervention model is based on the implementation of a combination of evidence‐based practices. Additionally, Roca has implemented individual evidence‐based practices as part of its work, including: Transitional Employment, the Harbor Area Healthy Families Program, Power Source, StreetSmart, and an adaptation of the CAS Carerra Teen Pregnancy Prevention Curriculum. Roca has created a nationally‐acclaimed model of Transformational Relationships as a vehicle for youth development, and pioneering effective local, regional, and national relationships with government, state, religious, health, and community partners. Roca’s Intervention Model has been touted as a national best practice by many governments, advocates and services providers, and sought after nationally and internationally. Required Changes to Organizational Structure: Roca’s current organizational structure was designed to operate a single site model. To address that, Roca is now working to develop a structure that will allow for management across sites while still providing sufficient management at each site and ensuring fidelity to Roca’s Intervention Model. Roca will also complete its implementation of the Balridge Framework for management and will utilize the organizational profile it creates to inform any required changes to the organizational structure. Roca also understands that to be a truly effective organization, one must have deliberate goals for learning how to operate as a business and that those learning objectives must grow and change over time. This is particularly true as the world of nonprofits continues to change. Roca will develop a clear set of learning objectives each year to address growth. Among other topics, this will include addressing issues of expansion into other cities and states, maintaining fidelity and integrity to the model, setting appropriate goals for services inside and outside of the core model, and positioning for a variety of funding opportunities. Leadership Management and Staffing

Current Leadership/Management: Roca has a strong team of diverse leadership including: Molly Baldwin, Executive Director; Tom McKenna, Managing Director for Massachusetts; Anisha Chablani, Chief Knowlege Officer; Jacob Sauerborn, Chief Fiscal Officer; Lili‐An Elkins, Chief Development and Strategy Officer; Dana Betts, Direcotr of Programming; and, Jake Jacobs, Director of Business Development and Major Gifts. Roca has also recently hired a Criminal Justice Programs Director who will be responsible for the MA pay for success initiative as well as working towards Roca’s operating a formal alternative to incarceration in Massachusetts. This team has a century of combined experience management, non‐profit finance, youth development, social work, workforce development, high risk youth work, corrections and community organizing. This leadership team has led Roca through dramatic growth over the past five years, helped launch Roca’s first program replication, and helped pilot transitonal employment in two additional sites across the Commonwealth. Additionally, key members of this leadership team also have extensive experience outside of Roca in developing and expanding social service and transitional employment programs.

Roca is also fortunate to have a strong Board of Directors that bring business, community, non‐profit, education, evaluation, and food services expertise to the organization and promotes volunteerism among young people, parents, and community members through its programming. Roca’s Board of Directors provides a strong degree of support to the project. Roca’ s Board of Directors includes community representatives as well as former program participants and focuses on meeting the needs of this very high‐risk population within the communities it serves. The Board of Directors includes leaders from a variety of institutions, including: Choate, Hall & Stewart, LLP; Catamount Management Corporation; delaCruz Communications, LLC; Endicott College; Feinberg & Kamholtz; Fidelity Institutional Products; Fidelity Institutional Wealth Management; Fitzgerald Company, Inc.; Global Inclusion;


Page | 40

Greenberg Traurig, LLP; Harbour Food Service; MassInc; The Mentor Network; North Shore Community College; Palmer Paving Corporation; and, Work’N Gear. Brief biographies for each of the members of Roca’s senior leadership team and Board of Directors are included as an attchment to this document. Additionally, it should also be noted that Roca’s Board of Directors has a strategic planning committee of board and non‐board members that are focused on the development of this model and creating the capacity for scale. This is a newly formed group with diverse expertise in the areas needed for this effort. This growing group currently includes:

Kevin Burke, Former Secretary of the MA Executive Office of Public Safety and Security and Visiting Professor at Endicott College;

Steve Butz, Executive Chairman and Founder, Social Solutions;

Edward Conley, Lieutenant, Chelsea Police Department;

Edward Dolan, Commissioner, MA Department of Youth Services;

Sandra Edwards, Vice President of Institutional Advancement, North Shore Community College;

Brian Elliott, Sergeant, Springfield Police Department;

Lenard W. Engels, Esq. , Managing Associate for Policy, Crime and Justice Institute;

David Guarino, Partner, Melwood Global;

Brian Hatleberg, City Councilor, City of Chelsea, Independent Financial Advisor;

Dwight Robson (Roca Board Member), Chief Public Strategy and Marketing Officer, The Mentor Network; and,

Christian Scorzoni, Cosgrove, Eisenberg & Kiley, PC.

Proposed Changes to Leadership/Management Structure: Roca has worked to assess its capacity to manage expansion and has begun to better understand its needs and build its capacity to sustain this work. This includes hiring a Managing Director for MA, who provides oversight to all MA sites, and a Criminal Justice Services Director, who will oversee the MA pay for success project. This expansion of staffing has already done a great deal to help move the organization and ensure that all senior management are in their appropriate roles. Roca has also begun the process of developing a sustainable senior management team for growth. Roca has completed an in‐depth assessment of the competencies needed to work as line staff in any Roca facility and developed an appropriate professional development/HR system to ensure these competencies are in place. Roca has recently received support to assess the competencies needed for the senior and middle management teams as the organization expands. This work, expected to be completed in FY13, will include the development of competencies, training tools to build competencies, assessment and tracking tools to determine progress of existing staff, new job descriptions and an organizational scorecard that will allow Roca to assess progress in all areas of the organization, including staff development. Roca understands the need for both succession palnning and talent development to ensure programmatic success. Roca has an emergency succession plan in place, which will be updated in the Spring of 2013, and is working on succession plans for all senior and middle management team members, to be completed and implemented by FY 14. Roca is currently facing some challenges in matching our existing management needs to the pace at which the organization is growing. As such, Roca will need to develop management/leadership capacity within both its middle and senior management and in the development of an organizational replication team. The replication team will be trained to provide coaching and assistance around not only laying the groundwork for any new replication site but also for providing operational training and coaching as any site gets off the ground. In addition, Roca is working with a management coach and a board member to assess current organizational management needs and to plan for the future. As part of this work, Roca will also work to create a plan for talent development and staff coaching. The illustration below details the leadership structure we believe Roca will need ot move to as it expands its operations. This table, along with Roca’s full orgnaizational structure both now and as it project to appear in FY 17 are included in the organizaitonal charts attached to this business plan.


Page | 41

ORGANIZATION WIDE

Executive Director (includes Government Affairs) Chief Finance Officer

Chief Planning, Strategy and Development Officer Chief Operating Officer – Program Quality/Program Design and Administration

Chief Knowledge Officer/HR Director Director of Business Development and Major Gifts

IT Director/Evaluation Coordinator Staff Development/Community Coordinator

Development Manager

MASSACHUSETTS (STATE 1) STATE /REGION 2 STATE/REGION 3

Managing Director for MA

Criminal Justice Director

Finance Director/Comptroller

Director of Development

Data Specialist

CHELSEA SPRINGFIELD REPLICATION SITE 1

TBD REPLICATION SITE 2

TBD REPLICATION SITE 3

Director of Programs o TEP Coordinator o Education/Pre‐Voc

Coordinator o TR Coordinator

Director for Western MA

Assistant Director o TEP Coordinator o Education/Pre‐Voc


Site Director o Director of Development o Data Specialist o Finance

Director/Comptroller o TEP Coordinator o Education/Pre‐Voc


Site Director o Director of Development o Data Specialist o Finance

Director/Comptroller o TEP Coordinator o Education/Pre‐Voc


Next Steps in Performance Based Management After a recent site visit to Roca where he examined Roca’s performance based management systems, Mario Morino author of Leap of Reason: Managing to Outcomes in an Era of Scarcity, noted:

Roca has gone far beyond simple data capture and reporting. They have developed many of the higher‐level performance management characteristics and skills we addressed in Leap of Reason. They clearly have visionary leadership, a strong and highly specific theory of change, solid definitions of outcomes and measurements, a highly talented and engaged staff, a data collection and reporting system that links those elements together. But perhaps most important, they have created and sustained an organizational culture that values and operationalizes performance management at nearly every level, addressing each area of programming, with clear buy‐in from all staff and constituents. What they achieved is quite advanced and very impressive.

Since the inception of ETO in 2006, Roca has moved from data collection to simple analysis to performance based management. As is typical with a new data system, it took a few years before Roca was comfortable enough with the quality of the data in ETO to begin using it in decision making. Starting in 2009, as part of the evaluation work, CJI worked with Roca on the establishment of performance measures. These measures were focused on counts of clients and activities and moved to intervention outputs and outcomes. After multiple iterations, reports were designed. Roca used the measures and report templates to develop multi‐level reports and scorecards. By 2010, Roca had developed a full set of automated reports for managers, supervisors and staff to monitor their own performance and the organization’s overall performance. Since 2012, Roca has continued to work to enhance and expand these tools in partnership with its new independent evaluator, Chapin Hall.

Client level reports are reviewed each week so that adjustments can be made as necessary to youth worker, program instructor, and work crew supervisor engagement approaches and overall efforts. Each month, managers and supervisors look at progress toward outputs and outcomes (e.g., participant retention, employment placements) and examine if key functions are on track, such as


Page | 42

assessments and work plans. Performance measures are reviewed by staff at all levels every quarter and outcomes are assessed in depth annually. The most immediate task in Roca’s work with respect to performance based management is the completion of the transition of the Intervention model to two years of intensive programming and with years of less intensive follow‐up for retention and sustainability. To achieve this short term goal, Roca must: 1) ensure all participants who have been engaged in the Intervention Model for more than two years are provided with completion plans, detailing an exit strategy and plan for working with them to obtain employment and/or other appropriate services; 2) completion of the refinement and expansion of follow‐up/retention programming components; and, 3) codification of the transition process and protocols that moves participants from Roca’s intensive services phase to its follow up services. These tasks are all targeted for completion by the end of FY 2013. Roca is also working to finalize the tools included in its performance based management system and to pilot those tools in the spring of 2013. These tools include: Roca’s Operations and Implementation Manuals for the intervention model; curriculum manuals; standardized supervision tools; work planning protocols; supervision protocols; coverage protocols; supervisor and program manager guides; an ETO user manual; a fiscal manual; and, an administrative manual. Roca will also finish refining some of its performance management reports, including an update to: the central intake tool; the quarterly assessment; the out of harm’s way assessment; and, transition tools between program phases. Roca will pilot all of these tools in the spring of 2013 and fall of 2014, with revisions and updates made during FY 2014. Roca is also developing measurable benchmarks for all other organizational components and will draft a complete organizational scorecard to be piloted in FY 2013. Once these tasks have been completed and implemented, they will be reviewed on an annual basis, in light of progress on other program evaluation activities, and will be refined to ensure continuous programmatic quality improvement.


Page | 43

Financial Operating Plan Financial Projections and Implications: Roca is seeking a total of $86,368,828 over five years for program operations and expansion, as detailed in this business plan. This includes a $25,000,000, 25th Anniversary fundraising campaign. This will include resources to support $77,081,183 in activities during those fiscal years, along with approximately $9,287,644.80 in funding for sustainability and future year needs. The following tables illustrate a summary of organizational revenues and expenses for each fiscal year. Please note these revenues include targeted revenues for growth and expansion included as part of Roca’s 25th Anniversary Fundraising Campaign.

FY 2013 ‐ 2017 Operating Revenues FY13 FY14 FY15 FY16 FY17

Pay for Success PFS Start Up Capital (Foundation Grants) $4,400,000 ‐ ‐ ‐ ‐ Quarterly PFS Payments (Government) 430,960 2,801,240 $3,632,497 $4,631,725 $3,964,373 PFS Quarterly Success Payments (Government) ‐ 104,533 299,830 399,263 426,208 PFS Interest Payments 38,884 144,291 107,573 139,988 160,190

Work Contracts/Earned Revenues 381,436 1,776,217 2,089,789 2,466,253 2,802,439 Government Grants/Contracts (Non‐PFS) 3,366,038 4,800,000 4,500,000 4,500,000 4,550,000 Foundation Grants (Non‐PFS) 3,370,000 4,250,000 4,500,000 4,400,000 4,300,000 Individuals 523,369 750,000 850,000 1,000,000 1,250,000 Corporations 350,000 500,000 600,000 700,000 850,000 Events 736,734 600,000 700,000 800,000 800,000

In‐Kind Donations 275,000 300,000 325,000 350,000 375,000

Total Revenues $13,872,421 $16,026,281 $17,604,689 $19,387,228 $19,478,210

Expense Summary FY13 FY14 FY15 FY16 FY17

FT Adult Staff 4,311,172 6,162,565 7,162,665 8,472,922 9,609,539

PT Adult Staff 88,736 88,736 88,736 88,736 88,736

Youth Staff 1,337,401 1,725,080 1,840,239 2,480,173 2,866,142

Contracted Services 589,029 708,535 724,356 854,457 854,457

Program Operations 1,919,790 2,581,341 2,074,205 3,494,846 3,774,870

Capacity Building 765,299 697,200 674,451 674,451 202,500

Total Direct Costs 9,011,427 11,963,456 13,687,325 16,065,586 17,417,243

A&G 1,374,217 1,750,041 1,845,755 1,927,873 2,038,260

Total Costs 10,385,644 13,713,498 15,533,081 17,993,459 19,455,502

Surplus (Deficit) 3,486,777 2,312,783 2,071,608 1,390,769 22,707

Note: If all benchmarks are met, the balance owed on PFS payments due after FY 2017 will total approximately $5,360,813.

Financial Projections


Page | 44

The tables below include Roca’s projected balance sheets for the same time period. This financial plan will allow Roca to achieve its goal of maintaining 3 months of operating cash in its operating reserve while at the same time growing its endowment to $5,000,000.

FY 12 FY 13 FY14 FY 15 FY16 FY17

Assets

Cash 1,583,508 2,363,667 2,692,967 2,775,124 2,950,000 780,788

Accounts Receivable 2,532,802 4,077,066 4,680,549 4,500,000 4,483,893 6,610,813

Prepaid Assets 77,146 950,000 1,000,000 1,000,000 1,100,000 1,150,000

Roca Future Fund (Quasi‐Endowment) 4,208,288 4,500,000 4,650,000 5,000,000 5,000,000 5,000,000

Roca Investments 1,357,027 1,500,000 1,700,000 1,800,000 2,000,000 2,225,000

Fixed Assets 924,534 839,534 1,869,534 3,789,534 4,749,534 4,624,534

Total Assets 10,683,305 14,230,267 16,593,050 18,864,658 20,283,427 20,391,135

Liabilities and Net Assets

Liabilities:

Accounts Payable 119,747 130,000 150,000 250,000 275,000 310,000

Accrued Liabilities 270,068 320,000 350,000 450,000 450,000 500,000

Total Liabilities 389,815 450,000 500,000 700,000 725,000 810,000

Net Assets

Unrestricted:

Operating 249,640 589,640 510,000 545,000 678,853 695,000

Book Value Fixed Assets 924,534 839,534 1,869,534 3,789,534 4,749,534 4,624,534

Board Reserved 740,959 500,000 500,000 500,000 600,000 600,000

Quasi ‐ Endowment 4,245,945 4,500,000 4,650,000 4,750,000 5,000,000 5,000,000

Operating Reserve 1,500,000 2,163,676 3,314,095 3,753,828 4,873,228 5,350,263

Total Unrestricted 7,661,078 8,592,850 10,843,629 13,338,362 15,901,615 16,269,797

Unreleased Temp Restricted ‐ Capital 2,000,000 2,000,000

Unreleased Temp Restricted ‐ Operations 2,632,412 5,187,417 3,249,421 2,826,296 3,656,812 3,311,338

Total Net Assets 10,293,490 13,780,267 16,093,050 18,164,658 19,558,427 19,581,135

Total Liabilities & Net Assets 10,683,305 14,230,267 16,593,050 18,864,658 20,283,427 20,391,135

Change in Net Assets 3,486,777 2,312,783 2,071,608 1,393,770 22,708

Months of Operating Reserves: 2.5 2.9 2.9 3.25 3


Page | 45

Primary Public Funding Source (Pay For Success or Alternative Community Corrections Financing) While Roca’s mixed funding strategy has been successful for Roca to date, it is not sustainable nor will it readily adapt to an organization with multiple sites. AS SUCH, ROCA BELIEVES THAT EACH SITE MUST HAVE SOME SOURCE OF SUSTAINABLE PUBLIC FUNDING PAYING FOR A SIGNIFICANT PORTION OF ITS WORK. In each additional location, Roca will seek public funding support before beginning operations and will work to utilize a pay for success financing model to generate this support. For its Massachusetts sites, serving both current and expansion locations, this funding will come from pay for success financing from the Department of Administration and Finance. Roca is currently negotiating with the Commonwealth of Massachusetts as part of its proposed Social Innovation Financing Program. Roca has been selected as the Commonwealth apparent successful bidder for this project which will serve approximately 900 young people over a six year period, allowing Roca to expand into each of the contiguous communities around its Chelsea and Springfield facilities. The Commonwealth has allocated $25 million towards this purpose. A significant portion of this funding will pay for our contiguous community expansions. Participation in this project requires Roca to raise $4,400,000 in up front capital that will support cash flow and program operations over time. Additional cash flow from FY 2013 to 2017 is illustrated in the table below. Additionally, if Roca achieves all of its projected outcomes for the project, the Commonwealth will owe Roca an additional $5,360,813 after FY 2017. A detailed cash flow projection for this project is included as an attachment to this business plan.

MA PFS Cash Flow Projections FY 13 to FY 17

FY 13 FY 14 FY 15 FY 16 FY 17

Up Front ‐ Quarterly PFS Payments 430,960.00 2,801,240 3,632,497 4,631,725 3,964,393

PFS Quarterly Success Payments 104,533 299,830 399,263 426,208

PFS Interest Payments 38,884.00 144,291 107,573 139,988 160,190

Total Raised 469,844.00 3,050,063.97 4,039,900.13 5,170,975.71 4,550,791

Roca believes that in each of its operating sites, there must be a mix of funding support that includes a primary public funding source that is a natural match to our mission and beneficiaries. The PFS contract illustrates Roca’s most significant progress to date in generating such a public funding source and Roca will pursue similar contracts for each of its replication sites. The groundwork involved in investigating some of these opportunities in other locations has already begun. For example: in Baltimore, MD, Roca has begun conversations with Baltimore Safe and Sound, one of the leading partners in each of the city’s social compacts, and is working to begin conversations with the local team working on pay for success contracting; in New York, Roca submitted a joint RFI response with Children’s Aid Society regarding using Roca’s model, through a pay for success contracting model, to reduce recidivism (note, this team is now preparing to submit an additional response to the City of New York); and, one of Roca’s partners, Third Sector Capital Partners, is working in three other states on pay for success and has been working to generate an interest in Roca’s model in those locations. In addition to these opportunities, Roca is also working to market its Intervention Model as an alternative to community corrections or incarceration. We believe that this will be an additional venue for public funding and is directly in line with Roca’s Intervention Model and its impacts on the incarceration rates of very high risk young men. Growth Capital and 25th Anniversary Campaign In an effort to support the work outlined in this strategic plan, including: Roca’s expansion into contiguous communities; Roca’s Direct Replication into two additional sites; and, the development of sustainability funding for Roca’s existing programming, Roca will celebrate its 25th Anniversary by operating a $25 Million, 25th Anniversary Fundraising Campaign. The table below illustrates the fundraising targets and timing goals for the 25th Anniversary Fundraising Campaign.


Page | 46

25th Anniversary Campaign FY 2013 FY 2014 FY 2015 Totals

Chelsea Campaign Sustaining Funds ‐ 750,000 750,000 1,500,000

Start Up Capital for Pay for Success 4,400,000 ‐ ‐ 4,400,000 Growth Capital for Development of 3 New Sites ‐ 4,000,000 4,000,000 8,000,000

Capacity Building Dollars ‐ 3,000,000 2,000,000 5,000,000

Sustaining Funds for Springfield 750,000 500,000 500,000 1,750,000

Young Mothers Validation and Expansion 250,000 500,000 850,000 1,600,000

Capital Costs for three new sites ‐ 1,500,000 1,500,000 3,000,000

$5,400,000 $10,250,000 $9,600,000 $25,250,000

To support the work surrounding this 25th Anniversary Campaign, Roca will be supported by the Development Committee of its Board of Directors and its Springfield Advisory Committee. The fundraising targets outlined above include raising $4.4 million in capital to support start up and pay for success cash flow issues in Massachusetts. The financial model for this project will allow these funds to be leveraged over time, ultimately resulting in a cash balance for the project at the end of the PFS project of approximately $6,560,745. This includes a surplus of $2,160,745 beyond the initial fundraising capital. Based on the type of funds raised initially, all remaining funds will be reinvested into future Roca operations. Fundraising targets also include a $4,000,000 operating support and $1,500,000 capital raise for each replication site. As Roca finalizes the selection of cities for program replication, it will create business advisory committees responsible for helping raise necessary replication startup funding that would be considered matching funds to this grant. Roca will also seek out pay for success financing and/or financing around alternative community corrections work as part of this fundraising work. Finally, this campaign includes: $3,000,000 in capacity building dollars, including funds to pay for a rigorous impact evaluation; $1,500,000 in Chelsea sustaining funds; $1,750,000 in Springfield Sustaining Funds; and $1,600,000 in funds to support the validation and expansion of the use of Roca’s Intervention Model for very high risk young mothers.


Page | 47

The current times pose both significant challenges and opportunities to achieving the proposed plan to scale impact. The leading challenges are:

Raising sufficient revenues to meet the needs for program operations and expansion ‐ o The economic status of the country and the world dramatically and negatively impacts employment, government,

businesses and funding, and o There is no one predominant government funding source for Roca’s work. Therefore, the organization must continue to

build and maintain the capacity to raise funds from diverse sources and adapt to economy. At the same time, Roca must develop a comprehensive plan to advocate for a predominant public funding source. Roca must also include a contingency plan that will address how the organization will respond if we are unable to develop a predominant government funding source;

Creating Sufficient Employment Opportunities and Business Development;

Developing sustainable funding in philanthropically underserved areas;

Addressing Roca’s inability to determine most appropriate way to determine long term scale and impact; and,

Combating the real and perceived danger of the very high‐risk, street/court/gang involved young people with whom Roca works.

Governmental entities across the country are at varying levels of readiness and interest in developing an Alternative to Corrections Model. This disparity in both political will and legal readiness will make it challenging to successfully move a national campaign around providing alternatives to corrections for very high risk young men.

The key opportunities are:

The ability to leverage Roca’s growing success in demonstrating outcomes and helping young people change their lives;

The leadership and passion of the emerging Roca Board of Directors and talented Roca staff team;

Roca has a strong, emerging board with capacity for high level support of this plan and the ability to fundraise;

The failures and costs of large systems, i.e. prisons, which create openings for effective alternatives such as Roca; and,

The increased interest in and attention on highly disengaged young adults.

There is an increased level of interest across the county in performance based contracting with a significant among of both public and philanthropic interest in pay for success models, including the use of social impact bonds.

In line with the interest in performance based financing, there is also an increasing interest by both government and private philanthropic sources in supporting high performing, outcomes driven organizations that help people make real changes in their lives while, at the same time, saving public resources.

Addressing these realities, Roca has assessed the following risks and developed the associated risk mitigation plans:

Risk Risk Mitigation

Raising sufficient revenues to meet the needs for program operations and expansion.

The current state of the economy places a significant challenge in raising sufficient revenues to meet organizational needs for current operations and proposed expansions. To address this, Roca has restructured its development staffing team to include staff focused on particular areas of fundraising (including a Development Director.) Roca is also working to increase the fundraising capacity of its Board of Directors. Roca is also seeking to continue to increase its private, corporate and individual support. Roca has developed a multi‐source strategy for funding (described in detail. in the “fundraising section” of this plan.) Roca currently operates in a fiscally sound manner, without debt, and has consistently come in under budget when looking at final expenditures for the year. Roca will continue this practice as it moves forward, and will commit to only expanding services as appropriate resources become available. Roca is also working on comprehensive plans to access pay for success and/or social impact bond

Risks and Opportunities


Page | 48

Risk Risk Mitigation

financing constructs that will allow the organization to develop a predominant public funding source. This plan will include a comprehensive advocacy strategy around developing a predominant source of government funding. It will also include contingency planning in the event a predominant source of public funding cannot be identified. This includes addressing Roca’s ability to continue to reach its scale goals without this funding source and the impact that this would have on the timing of the goals and objectives outlined in this strategic business plan.

Creating Sufficient Employment Opportunities and Business Development

Because transitional and other employment opportunities are not always readily available within the community, Roca has determined that it must not only look for these opportunities but that It must also work to create these opportunities for the young people served. As such, Roca continues to develop partnerships with local businesses around job creation and business development. Roca is currently seeking to develop a green cleaning/property maintenance business that will serve both Chelsea/Revere/East Boston and Springfield, and continues to look for other business development opportunities. To provide additional support to this work, Roca applied for and received funding from the Corporation for National and Community Service for a VISTA position focused on business development and the creation of employment opportunities. Roca has also sought out funding to generate business development opportunities from the United Way and other local and regional foundations. Additionally, to support this work, Roca is in the process of hiring a Director of Business Development and Transitional employment and Roca works with a business advisory committee that helps the organization with ongoing support, guidance and business development assistance.

Developing sustainable funding in philanthropically underserved areas.

Roca understands that not only does it need to develop sufficient funding to support program operations in each of its sites, but that it must also develop sustainable funding sources in each community, particularly those that are philanthropically underserved. As such, will ensure that there will be a minimum of $2 million raised in any community before Roca begins operations. At the same time, Roca will work to develop partnerships with local leaderships, including a committee of business leaders committed to helping raise necessary revenues.

Addressing Roca’s inability to determine most appropriate way to determine long term scale and impact

In an effort to ensure that Roca determines the most appropriate way to scale up and share its learning and impact, several activities are currently in process. 1) Roca is implementing its first program replication, allowing the organization to learn from its own experience in determining the best mechanisms to replicate and share information. 2) Roca is currently undergoing two rigorous program evaluations; allowing us to learn how to most effectively replicate services. 3) Finally, Roca has developed a scale and impact advisory committee who works with Roca to assess the results of the program replication and evaluations and to determine the best mechanisms to share learning and impact. This committee will also make recommendation regarding future replication ventures.

Combating the real and perceived danger of the very high‐risk, street/court/gang involved young people with whom Roca works.

To address both the real and perceived danger of working with this population, Roca will rely on three critical strategies: 1) Roca will seek to hire personnel with experience working with this population and will continuously develop organizational talent and train them in the techniques critical to working with this population, such as motivational interviewing and cognitive behavioral therapy; 2) Roca will work with a strong local board in each location, that includes business people who will work with Roca to address perceived issues of danger; and, 3) Roca will develop a marketing strategy to address issues of both perceived and real danger and the consequences of the violence often perpetrated by this population of very‐high risk youth and the impact of that violence.


Page | 49

Appendices


Page | 50

Roca has developed and operates the Intervention Model designed to help the most high‐risk young people break the destructive cycles of poverty, violence and perpetual incarceration. The Intervention Model pushes young people to identify, confront and overcome destructive behaviors and learn the skills needed to re‐engage and succeed in society, education, and the economy. To the best of our knowledge, Roca’s Intervention Model for very high‐risk young people is the only full‐time, long‐term, intervention based the combined principles of cognitive re‐structuring, skills development, motivational interviewing, and transitional employment delivered on the street, for this population, by a non‐mandating authority. Roca’s work is based on a simple, but powerful theory: when young people are re‐engaged through positive and intensive relationships they can gain competencies in life skills, education and employment and move toward economic independence and living out of harm’s way. The validity of that theory is being proven by the success of Roca’s Model. Roca helps young people learn and practice the critical academic and employment skills they need, as well as evidence‐based methods for engaging young people in cognitive‐restructuring activities.

Roca’s Intervention Model is a cognitive‐restructuring intervention, enabling young people to move toward the outcomes of economic independence (successfully keeping a job) and living out of harm’s way (with a primary focus of staying out of prison). Roca’s Intervention Model connects very high‐risk youth to each other and adults by building positive and intensive relationships. Using these relationships and targeted programming (life skills, education and employment) Roca will help youth develop the necessary skills to reduce violence and create positive behavioral changes. The project will start with intensive outreach and follow‐up, offer programming, operate basic and advanced transitional employment, create jobs, and work with the City, several criminal justice partners, and other organizations. Using its Intervention Model, Roca works with very high‐risk young people to create behavior change and bring them over the starting line so that they can effectively engage in existing forms of programming, education and jobs and move towards economic independence. The decision to explicitly work with the highest risk populations establishes Roca’s model as distinctive. There are very few agencies in the U.S. with transitional employment programs serving a similar population, and none of those programs are grounded in relationships. The Relentless Outreach and Transformational Relationships components of our Intervention Model set Roca apart and highlight the agency as a leader and innovator in working with hard to reach young people. Roca recognizes that no amount of programming can force a young person to change ‐ the desire, will and follow‐through must be initiated by the young person him/herself. Through relationships, Roca enables young people to see the change that needs to happen in their lives and then assists them to actually make that change. Intervention Model Overview Roca has developed an operates an intervetnion model designed to help very high‐risk young people, specficially those who are street, court or gang‐involved, in moving out of violence and poverty. Our Intervention Model, created to help young people make positive behavioral changes, was designed for young people who are out of school and work and engaged in anti‐social, negative and often criminal activities.

Figure 1: Targeted Young People

Appendix A: Detailed Description of the Program Model


Page | 51

The Intervention Model is decidedly Roca’s most important accomplishment. While conventional criminal justice and public policy has often been designed to respond to violence and poverty in urban communities with a fairly limited set of institutional tools, the Intervention Model serves as a progressive alternative that is both cost‐efficient and highly effective. Specifically regarding crime, criminal justice and criminal rehabilitation, federal and state governments have operated within a narrow framework, employing two basic strategies: prevention and suppression. Though these strategies will forever remain fundamental components in the fight against poverty and violence, they exclude an equally important strategy: intervention. By developing a multidimensional intervention program for the most high‐risk young people, grounded in evidence‐based and proven practices, Roca has proven that the cycle of poverty and violence that continues to plague our cities can indeed be interrupted. Roca chooses to focus on 17‐ 24 year olds in crisis, because it is the group that has most often been neglected by public and private institutions, and is the group responsible for the majority of violence in our cities. The Intervention Model is designed for young people who have already dropped out of school, who are involved in drugs and gangs, have criminal records, are on probation, or are young parents. By seeking out these young people where they are, in the streets, we aim to directly intervene in their lives during the most crucial points; not before the trouble has started, and not after the fact. By creating consistent, positive relationships as a foundation for cognitive‐behavioral change, we help young people move along a long‐term, stage‐based plan for improving their lives, offering opportunities to make educational, life‐skills and employment gains. Roca’s Intervention Model is based on the five Stages of Change, (a framework commonly used in medical and mental health fields that outlines the stages an individual typically moves through to improve and then sustain life changes). The chart below outlines the stages of change as they apply to Roca’s young people.

Stage of Change Definition Youth Worker’s Role (or other adult working with young people)

Example: Decreasing Street/Gang Involvement

PRE‐CONTEMPLATION

NO: Young person is not thinking about or has explicitly rejected change.

Increase awareness of need for change, personalize information on risks and benefits.

Participant actively engaged in street/gang behavior and unwilling to see it as harmful.

RELAPSE: W

E ALL DO IT

CONTEMPLATION MAYBE: Young person is thinking about change, and perhaps seeks out some program, may respond to some positive suggestions from staff.

Motivate, encourage making specific plans.

Participant beginning to think about the potential harm of street/gang involvement.

PLANNING A GOOD IDEA: Young person and staff talk about what it would take to make change happen and what the young person wants in the future.

Assist in developing concrete action.

Participant discussing difficulties of street/gang involvement with youth worker and ways to shift behaviors/activities.

ACTION ACT DIFFERENTLY: Implementation of specific action plans. The young person begins to take positive steps toward improving his or her life through practice (trial and error) in the context of a plan that staff and the young person have discussed in detail.

Assist with feedback, problem‐solving, social support, and reinforcement.

Participant actively attempting to engage in more positive activities (education, employment, life skills, etc.).


Page | 52

Stage of Change Definition Youth Worker’s Role (or other adult working with young people)

Example: Decreasing Street/Gang Involvement

SUSTAINING MORE THAN 51%: Through ongoing staff support during difficult times and new cooperative efforts, the young person is achieving concrete improvement in his or her life, moving demonstrably toward achieving self‐sustaining lifestyle and living out of harm’s way. This does not mean everything is perfect. “If you get to 51% you’ve made change.”

Assist in coping, reminders, finding alternatives, avoiding slips/relapses (as applies).

Participant maintaining engagement in positive activities and making progress on other goals.

The very high‐risk young people that Roca serves are between the ages of 14‐ 24, and are street, court, drug or gang‐involved, disengaged, in danger of dropping out, have dropped out, or are young parents. The young people that define our target population have not, and cannot succeed in other educational and employment programs because they are neither ready, nor willing.They are not ready developmentally, intellectually or emotionally. And the behavioral barriers to their personal success far exceed their ability to participate in structured programming that requires consitent, disciplened self‐management and motivation. The majority of programs for youth and young adults that exist today are not designed to work with this hard‐to‐serve population; rather they are designed to work with individuals who are in the “action” phase. Few programs are capable or even willing to reach and retain this group of young people who often begin in the “pre‐contemplation” or “contemplation” phases. Roca, on the other hand, has become distinctive for its success in doing what other youth development organizations can’t. We are successful because we refuse to lose young people due to fear, distrust, lack of motivation, social and academic disengagement, repeated harmful and problem behaviors, or even illegal behaviors. Most programs would dismiss these high‐risk young people at these difficult tipping points. But our model was built with the understanding that young people will relapse multiple times. We understand that they will fail to turn up for programming, and revert to delinquent and self‐destructive behaviors‐ but Roca stays with them! (For more information on the stages of change which forms one of the bases for this programming, please see page 21 of this plan.) Roca does not wait for these young people to reengage on their own. Left to their own devices, most never do. We go out and find them ‐ where they are at ‐ in the streets. We tell them the truth, teach them to trust, and help them learn and practice the skills they need to transform their lives in order to get on the path to economic independence and safe, healthy living. Roca’s intervention Model has shown significant, early, positive results for very high‐risk young people who are out of school and are street, court and gang involved. Services in both Chelsea/Revere/East Boston and Springfield focus on this core group of young people. We are currently evaluating this model to determine its true impact on these groups. At the same time, we believe this Intervention Model can also be extremely effective for those in‐school youth who are at risk of dropping out and for very high‐risk young parents. Roca is now testing and refining the model for work with these groups. Components of Roca’s Intervention Model Roca’s Intervention Model connects very high‐risk youth to each other and adults through intensive relationships and uses targeted programming (life skills, education and employment) to support young people to develop the necessary skills to reduce violence and create positive behavioral changes. The project starts with intensive outreach and follow‐up, offers programming, operates basic and advanced transitional employment, creates jobs, and works with the City, several criminal justice partners, and other organizations. The critical success factors to the Intervention Model include: a profound belief in the ability of young people to develop the capacity to change their lives; relentlessness in our outreach, follow up and commitment to show up with consistency; love for young people and a commitment to truth, trust, and transformation; consistent implementation of the model; strong management and leadership capacity (paid and volunteer); and effective institutional partnerships committed to the same outcomes.


Page | 53

Figure 1: Participant Pathway Through Intervention Model

The Intervention Model has four components (each of which is described in detail in the narrative and in the illustration on the previous page): 1) relentless outreach and follow up; 2) transformational relationships; 3) stage based programming; and, 4) work with engaged institutions. Focused on helping young people make behavior change, the chart below demonstrates the course of behavior change and attainment of life, education and employment skills that each of Roca’s participants is guided through.

Relentless Outreach and Follow Up: We cannot wait for high‐risk young people to show up at our doors, because they never will. Youth workers build relationships by finding young people where they are at, learning where they hang out, knocking on their doors, and continuously circling the neighborhood in the infamous Roca vans. It is the relentlessness of a youth worker who shows up day after day, no matter what, that awakens hope in a young person. As trust builds and the visits to Roca increase, the youth worker strategically develops a relationship that is a commitment between the youth and youth worker: they are now “in it” together.

But what defines relentless outreach and follow up in the most fundamental sense, is the consistency of a youth worker’s efforts in connecting and reconnecting with a young person. Because high‐risk young people have had little or no experience with adults who impose consistent expectations of healthy behavior, they are prone to frequent disengagement and rejection of constructive relationships. Therefore, a youth worker must relentlessly reconnect with a young person who periodically rejects them and refuses to engage in programming. This pattern is crucial to the process and must happen continuously throughout the relationship. While a young person may show up at Roca three times in one week, they may just as likely refuse to come back the following week. It is the youth worker’s responsibility to track down that young person and reengage them.

Transformational Relationships (Intensive Case Management): The underlying theory behind the Intervention Model is that relationships change people– that positive change comes about within the context of mutuality, shared experience, and a sense of responsibility, not only to oneself, but to another. Roca engages young people in relationships for the purpose of change. These relationships are called transformational relationships. They are not relationships based upon institutionalized power dynamics; Roca is not a mandated intervention. They are relationships in which a significant amount


Page | 54

of time (up to one year) is allotted to simply building trust. These intensive relationships are used to consistently and frequently engage young people in a variety of skill building opportunities over a period of up to three years. These relationships are designed to motivate young people to participate and decide for themselves that they want to change and take concrete actions to change. They are used to support young people when they have setbacks or relapses, to support young people when their families are unable or unwilling, and to protect young people from harm when they are not protecting themselves. At root, transformational relationships are developed to challenge young people when they are not challenging themselves.

Roca knows that when a young person is re‐engaged through positive and intensive relationships, he/she can go on to gain competencies in life skills, education and employment. Therefore, at the core of our High‐Risk Youth Intervention Model is the Transformational Relationship (TR). Each youth worker carries a caseload of 25 young people and connects with each young person no less than three times each week. Youth workers are available 24 hours a day and are often the one adult in a young person’s life that is there when they go to court or enter lock up; who visits them during incarceration and picks them up when they are out. Their relationship is not a friendship – it is more profound‐ effectively intentional, and mutually respectful. Youth workers are trained in motivational interviewing and cognitive behavioral strategies preparing them to use their self effectively to guide and support young people.

Stage‐Based Programming (Life Skills, Educational, Prevocational, and Employment Programming): The third part of Roca’s Model, Stage‐Based Programming, engages young people where they are in the stages of change cognitively and behaviorally.15 By applying this evidence‐based framework in the program delivery, young people move along a pathway of education, pre‐vocational training, life skills, transitional employment and unsubsidized employment opportunities. For example, one young man who was in pre‐contemplation about education—who was very vocal in his hatred of all things school‐related—occasionally was persuaded by his youth worker to drop‐in on open tutoring. After about a year, this young man decided he was ready for a pre‐GED class, and now plans to take and pass his GED next year. Another young person was completely negative about school, but enjoyed the drop‐in carpentry class at Roca. He was eventually able to make the transition from a drop‐in class, to certificate training, to transitional employment. By matching programming to a young person’s stage of change, instead of placing all young people into a one‐size‐fits‐all curriculum, Roca is driving this population, who will not move in a straight trajectory, toward positive outcomes.

An example of Roca’s stage‐based programming is our transitional employment model. It takes 18 – 24 months for young people to put in 60 days of work in a row. Only after completing these consecutive work days, can a program participant begin to work on lifelong employment and opportunities. For our young people, it is not a linear path to success‐ they get started, get angry, don’t like being told what to do, get high, get arrested, lose their work slot, many times over and finally they succeed. Through Roca’s basic transitional employment program, the Key Project, the organization operates three supervised work crews that engage in partially‐subsidized work projects contracted trough municipal public works departments and private companies. The work is simple and designed to teach people to show up, follow instruction, and develop basic work skills such as cleaning, painting and maintenance. This program is built especially for our high‐risk young people who have challenges getting and keeping jobs. It takes into account the stages of change, relapse phases, and the need for multiple opportunities to succeed.

Work with Engaged Institutional Partners: The fourth part of the High‐risk Youth Intervention Model is the Engaged Institution Strategy. The institutions that are in a young person’s life—schools, local government, agencies, and organizations—are just as influential to the needs and growth of a young person as Roca. In recognizing this, Roca sought to create partnerships with these institutions, open the lines of communication, and benefit from each other’s expertise. Roca’s work has resulted in initiatives such as: area‐specific gang intervention practices with the Chelsea Police, shared youth work practices with the Department of Social Services,and an integrated partnership and on‐site programming with the Chelsea Public Schools. Roca is committed to shifting the trajectories of not only our young people, but of all the institutions involved in a young person’s life. Through dialogue and action, Roca continues to promote and produce

15 Roca has taken Prochaska’s stages of change and adapted them for a very high‐risk youth development model. Backed by evidence from multiple disciplines (e.g. substance abuse, medicine, harm reduction, etc.), promoting young people through the stages of change allows them to develop the intrinsic skills, desires, and capacities to successfully move towards safety and self‐sufficiency. Additional details regarding stage based programming can be found later in this proposal under “Research and Evidence that Informs the Model.”


Page | 55

alternative, restorative policies in our result in a systemic change in how our communities address the needs of high‐risk youth. Our engaged institutions work has resulted in a statewide task force on transitional employment, a shift in area gang intervention practices, and an unprecedented emerging partnership with the Hampden County Sheriff’s Department in Springfield where Roca is beginning its first replication.

In addition to the systemic formal and informal policy and practice changes that we work toward, Roca’s Engaged Institutions strategy promotes transformational relationships between organizations and systems. This can be seen through evolving and developing relationships that include the cities of Chelsea, Revere, Boston, and Springfield: Hamden County Sheriff’s Department in Springfield, MA; Chelsea, Revere, and Springfield Police; Chelsea Public Schools; The Federal District Court, Chelsea District Court and Department of Probation; Massachusetts General Hospital / Roca health clinic; and, more than 70 area employers.

In a time when our criminal justice systems are in crisis, its failures and costs have tremendous implications for our communities. Roca’s engaged institutions strategy, combined with the use of Roca’s Intervention Model, can provide one resource to assist in this crisis and provide a potential alternative to traditional incarceration and/or suppression programs.


Page | 56

Intervention is an important component in the criminal and juvenile justice continuum. Intervention strategies aim to help individuals who are already involved in harmful behaviors. Research has shown that working with individuals who are already involved in harmful behaviors is challenging but can produce positive outcomes. Although current research does not provide all the answers, it can guide our work so that strategies can be more effective. Additionally, we must learn from the best practices of others while individualizing strategies for our own communities and for each individual. Roca believes that intervention work must have the following four guiding principles to be successful: 1) involvement of organizations that care about young people and believe they can change; 2) targeted and based on data; 3) individualized, intentional and focused on outcomes; and 4) long term. Conducting appropriate intervention work for this target population requires Roca to employ a unique strategy‐ one that combines hard‐nosed, committed street work with lessons learned from groundbreaking academic research. The Intervention Model is the product of over two decades of passionate work on the streets with young people. We have integrated the knowledge we have gained from our direct, hands‐on experience with evidence‐based practices from cognitive‐behavioral therapy, motivational interviewing, transitional employment, and promising/best practices from restorative justice and re‐entry.16 Each of these practices are described below:

Cognitive‐Behavioral Therapy/Treatment (CBT): Noted as an evidence‐based practice by the US Department of Justice, Cognitive‐Behavioral Therapy/Treatment (CBT) is a problem‐focused approach designed to help people identify and change the dysfunctional beliefs, thoughts, and patterns of behavior that contribute to their problems. It is the most evidence‐based form of psychotherapy. CBT has been successfully applied across settings (e.g., schools, support groups, prisons, treatment agencies, community‐based organizations, churches) and across ages and roles (e.g., students, parents, teachers). Problem behaviors that have been particularly amenable to change using CBT have been 1) violence and criminality, 2) substance use and abuse, 3) teen pregnancy and risky sexual behaviors, and 4) school failure.

Motivational Interviewing: Motivational interviewing, included in SAMHSA’s National Registry of Evidence‐based Programs and Practices, is a directive, client‐centered counseling style for eliciting behavior change by helping clients to explore and resolve ambivalence. Compared with nondirective counseling, it is more focused and goal‐directed. The examination and resolution of ambivalence is its central purpose, and the counselor is intentionally directive in pursuing this goal. The concept of motivational interviewing evolved from experience in the treatment of problem drinkers, and was first described by Miller (1983) in an article published in Behavioral Psychotherapy. These fundamental concepts and approaches were later elaborated by Miller and Rollnick (1991) in a more detailed description of clinical procedures.

Transitional Employment: Transitional employment is an evidence‐based model and, cited by the National Transitional Jobs Network and MDRC17, Roca’s transitional employment is a national model for working with high‐risk young people. Transitional Employment offers a critical solution for helping employers to fill jobs with job‐ready, entry‐level employees. Transitional employment programs are designed to address the needs of those with the greatest barriers to employment ‐‐ individuals who cannot succeed in traditional workforce development strategies and/or employment. Complimenting a range of excellent training programs, higher level employment programs and sector training, transitional employment will increase the number of people who are successfully able to enter the workforce.

16 Research regarding these linkages was conducted by the Crime and Justice Institute in their 2006 report, “Interventions for High‐Risk Youth: Applying Evidence‐Based Theory and Practice to the Work of Roca.”

17 D. Bloom, S. Rich, C. Redcross, E. Jacobs, J. Yahner, and N. Pindus. Alternative Welfare‐to‐Work strategies for the Hard‐to‐Employ: Testing Transitional Jobs and Pre‐employment Services in Philadelphia. MDRC, October 2009 and C. Redcross, D. Bloom, G. Azurdia, J. Zweig, and N. Pindus. Transitional Jobs for Ex‐Prisoners: Implementation, Two‐Year Impacts, and Costs of the Center for Employment Opportunities (CEO) Prisoner Reentry Program MDRC, August 2009.

Appendix B: Research Informing the Model


Page | 57

Restorative Justice: Restorative justice is a promising/best practice from the justice system that emphasizes repairing the harm caused or revealed by criminal behavior. Practices and programs reflecting restorative purposes respond to crime by: 1) identifying and taking steps to repair harm, 2) involving all stakeholders, and 3) transforming the traditional relationship between communities and government in responding to crime. Restorative justice seeks to involve the entire community in rehabilitating offenders and holding them accountable for their behavior. By bringing together victims, offenders, families, and other key stakeholders in a variety of settings, restorative justice helps offenders understand the implications of their actions and provides an opportunity for them to become reconnected to the community. Some of the most common programs typically associated with restorative justice include mediation and conflict resolution programs, restitution and community service, family group conferences, victim‐offender mediation, and victim impact panels. If implemented properly, these programs have either proven to be effective or are promising in lowering recidivism, reducing offending, and improving victim satisfaction. In addition to these evidence‐based and promising practices, Roca’s Intervention Model is based upon several critical adapted

frameworks for service delivery, each of which is outlined below.

The Stages of Change: Every young person we serve has a different story, different destructive behaviors, and in turn, different needs. We meet them where they are, physically and developmentally, we help them identify which of their behaviors are destructive, and through persistent, relentless relationship building, we help guide them through the stages of behavior change. Each must, in his or her own way, move through these five stages in order to change the particular behaviors that serve as barriers to success:

1) Pre‐contemplation: not intending to take action, resistent, unmotivated. This is where most Roca participants are

pyschologically “at” when we reach out to them, typically unwilling to acknowledge their behaviors as destructive in their lives;

2) Contemplation: beginning to think about change, becoming more aware of the pros of changing, but acutely aware of the cons. In this stage, Roca staff have more and more conversations with young people, help them weigh the pros and cons, provide useful information, and enagage them in programming that helps create a context for the emotional and intellectual process that positive behavioral change brings about;

3) Preperation or Planning: intending to take action, and thinking about ways to make change happen. When young people reach this stage they begin to trust a youth worker’s intentions, begin to envision themselves doing something different in their lives, and envision themselves benifitting from our programming;

4) Action: making specific, overt modifications in their lifestyles. In reality, this is the stage young people get to practice new behaviors over and over again with intensive support from Youth Workers, helping them process the new feelings and thoughts, challenges and success of the change;

5) Sustaining: At this stage, the young person is “doing” the new behavior and is less tempted to relapse as a result of increasing confidence that they can change their lives.

Getting young people to the stage of “sustaining” positive behavior change is Roca’s primary goal. By sustaining change, young people can continue towards more conventional educational and employment goals. In Building a Learning Agenda Around Disconncted Youth, a report commissioned in 2009 by the Bill and Melinda Gates foundation, Roca is cited for its “multi‐faceted and long‐lasting” approach. The report goes on to discuss Roca’s work, and praise it for creating an effective means to reengage the most disconnected young people: “ Roca’s work is based on the premise of strong relationships. They have developed powerful strategies for connecting and building trust with at risk youth.” Roca has had such success, because the implementation of our Intervention Model, much like the clinical framework for change that guides it, is structured to reflect and support the complex processes neccesary for young people to change. For additioanl information regarding the stages of changes, please see the illustration on page 15 of this business plan. As change can be extemely difficult, young people often relapse and return to negative and destructive behavior. Unlike other programs, Roca turns this setback into opportunity. Relapse becomes a tool that allows the youth worker to re‐engage the young person and bring them back again and again. This allows a young person to grow, have the hard conversations, and move forward. A young person who relapses into substance abuse or other negative behaviors is not going to show up again willingly; the youth worker must find and engage the young person however many times is necessary‐ 100, 500‐ whatever it takes, until the young person is ready to trust and show up. This is what it takes to re‐engage young people and keep them from slipping back into harm’s


Page | 58

way. This is how we keep kids alive. Roca does not take for granted the difficulty and level of energy necessary to help very high‐risk young people, and the implementaion of our Intervention Model reflects this.

Resiliency Theory: The risk factors that increase a child’s likelihood of delinquency, dropping out of school, and poor academic performance are well known. However, what was poorly understood until the last decade were the factors that allowed some children to flourish despite a multitude of risk factors. Resiliency Theory identifies the protective factors that allow a youth to recover from trauma and overcome a negative environment; the theory was originally developed to identify factors that are predictive of academic success for children.18 19 Resiliency Theory posits that the development of resiliency factors, rather than simply the mitigation of risk factors, will improve outcomes for youth. The central tenant of the model is that a caring, supportive relationship with at least one non‐parent adult is an essential component of success. Resiliency theory provides the underpinning of many mentoring programs, and provides theoretical support for Roca’s transformational relationships, peacemaking circles, life skills education, and engaged institutions. Activities that encourage pro‐social bonding and skill development increase the likelihood that youth will be able to overcome the negative impact of their environment.

Eight Guiding Principles for Reducing Risk and Recidivism: Finally, much of Roca’s Intervention model is based on the Eight Guiding Principles for Reducing Risk and Recidivism. In 2002, the National Institute of Corrections and the Crime and Justice Institute entered into a cooperative agreement to develop and implement an integrated model of evidence‐based practice in community corrections. The model includes three elements: Collaboration, Organizational Development, and Evidence‐Based Principles.20 These eight steps include: 1) assessing actuarial need; 2) enhancing intrinsic motivation of those served; 3) targeting interventions based on participant need; 4) skill train with directed practice; 5) increase positive reinforcement; 6) engage ongoing support in the community; 7) measure relevant practices; and 8) provide measurement feedback. The principles represent the most current research in the field as to what works in reducing recidivism among offenders in the community. Many of the principles are relevant to the high‐risk population with which Roca works.

18 Henderson, N. (nd) The Resiliency Quiz. Available online at http://www.resiliency.com/htm/resiliencyquiz.htm.

19 Henderson, N and Milstein, M (1996) Resiliency in Schools: Making it Happen for Students and Educators. Thousand Oaks, CA: Corvin Press.

20 Crime and Justice Institute (2004) Implementing Evidence‐Based Practice in Community Corrections: The Principles of Effective Intervention. Boston, MA: Author. Available online at: http://crjustice.org/cji/evidencebased.pdf.


Page | 59

Eight Guiding Principles for Reducing Risk and Recidivism and How They Apply to Roca

Roca’s programming is designed to first engage highly disconnected and disengaged youth and then involve them in programming based on the stages of change that allows them 1) to participate in developmentally appropriate programming; and 2) relapse over and over again until they are ready to take action and sustain changes. This chart is a comparison of Roca’s emerging work with youth and young adults to the 8 Evidence‐Based Principles for Effective Interventions as described in the report “From Incarceration to Community” By the Crime and Justice Institute, June 2004, which are considered effective interventions for community corrections:

8 Evidence‐Based Principles

Overview of Principle Roca Practices as of 2/16/12

1. Assess Actuarial Risk/Need

Sound assessment that identifies dynamic and static risk factors is the cornerstone of effective supervision. If risks and needs are not properly identified and prioritized, appropriate interventions and services cannot be delivered.

CJI (Crime and Justice Institute) completed a report examining where Roca’s tools align with risk and needs items on eight commonly used, validated risk and needs assessment instruments including: the Level of Service Inventory‐Revised (LSI‐R), the Offender Screening Tool (OST) and the Field Re‐assessment Offender Screening Tool (FROST)2, the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), the Juvenile Assessment Generic (JAG), the Wisconsin, the Youth Level of Service Case Management Inventory (YLS‐CMI), the Violence Risk Appraisal Guide (VRAG), and the Colorado Juvenile Risk Assessment (CJRA). The majority of actuarial assessment tools examine a variety of static and dynamic risk factors. All of these instruments capture items related to criminal history, as this static indicator is strongly correlated with recidivism. Roca is consistently collecting 41% of the data contained in these actuarial tools according to CJI, however, as a result of this report, Roca modified its Central Intake Form to include critical missing items

- Roca’s Central Intake Form was revised to identify and assess risk level for eligibility of participants. Additional questions were added to capture static risk factors including age of first arrest, # of prior arrests, prior convictions, prior incarcerations, prior supervision, prior charges, and other dynamic risk factors including: substance use, antisocial peers, mental health, employment history, educational history, and public assistance involvement.

- In FY’11, Roca worked with CJI to clearly define what high risk means for Roca and how that definition is or isn’t consistent with the

field. It was determined that it meant high risk for continued participation in violence, criminal behavior and incarceration. In order for young people to be enrolled in Roca’s Model, he must have felony and/or a specific set of misdemeanor charges, no work history, and little to no education history. In FY’11, Roca separated its outcomes between the overall group of young people served as compared to the group determined at high risk meeting these criteria. This process of determining who are high risk young people was a result of: reviewing data and determining for whom the program was most appropriate; a decision to stop enrolling lower risk participants; and an evolving commitment to develop and focus the Roca intervention model to help high risk young adults make changes and succeed.

- Given the intensive nature of the model, it became increasingly clear through data that higher risk young people are more

appropriate for the model. By matching assessment and programming, the organization is better able to appropriately address needs. In the implementation evaluation by CJI for FY’11, it was determined that the more risk characteristics a participant has,


Page | 60



the more likely he or she is to progress through the intervention model. - The Roca Springfield Project focused on a group of high risk young offenders. All the young people in the program have felony

charges, no work and little to no educational history. Roca Springfield participant risk factors in FY’11 were: 100% felony arrest; 100% gang and/or systems involved; 99% dropped out of school and had no GED; and 96% with no job and poor to no job retention.

- Over the past year, in Chelsea, Revere, and East Boston, Roca increasingly began to serve high risk young people with the same

criteria as Springfield. In response to this, Roca has begun work in transitioning to serving a primary and secondary target population. Use of these population categories will begin formally in FY’13.

o Primary Target Population: In October, 2011, Roca participated in its theory of change process and determined the primary population for the organization moving forward will be young men ages 17 or 18 (depending on when they are charged as an adult for a crime) with felonies and/or specific misdemeanors, no work history and little to no educational history. This primary population will be the focus of Roca services at all replication sites.

o Secondary Target Population: In Chelsea, Revere, and East Boston Roca will continue to work with a secondary target

population that will include very high risk young women and young mothers. Roca is currently working on a comprehensive definition of this secondary target population.

Next Steps 1. Roca is looking into how to administer or work with partners who could administer an actuarial risk/need tool (i.e. DYS, DOC or

Probation). There are pros and cons to each of these approaches that will be reviewed and addressed over the next year. 2. Roca is implementing the revised Central Intake Form and will examine the possibility of weighting the tool to score level of risk for

the program

2. Enhance Intrinsic Motivation

Using motivational interviewing techniques, as opposed to direct persuasion or nondirective counseling, can help build intrinsic motivation in offenders. This is instrumental in initiating and maintaining behavior change.

- Roca trains all staff in Motivational Interviewing (MI). The training includes: 6 hours of overall training on the MI annually. 1.5 hour monthly MI Reinforcement training; and 1hour weekly supervision that includes a supervision tool with targeted questions to ensure the capacity of staff to use MI.

- Over the past year, Roca has also developed a series of topic specific trainings using MI including: Managing through the Holidays;

Sexual Health & Pregnancy Prevention; Living Clean and Sober; Emotional Health and Well‐being; Positive Mental Health; Healthy Relationships; Pro‐Social Lifestyle; Healing & Recovery from Trauma; Knowing and Exercising Your Rights; Workforce Readiness; Retention and Advancement; Reengaging with education‐Out of School Youth; Staying in school‐In School Youth; and Conversations about Weapons.

- Roca staff use MI in their intentional 2 – 3 contacts per week with young people to help them develop intrinsic motivation to help

them change their behaviors and move towards outcomes.


Page | 61



Next Steps 1. Finalize MI coaching modules. 2. Train another Roca staff member as MI Trainer. 3. Review evidence‐based MI, compare to Roca use of MI, and determine next steps if appropriate.

3. Target Interventions

This principle incorporates five related subordinate principles—the principles of risk, criminogenic need responsivity, dosage, and treatment. In general, the principle states that supervision and treatment should target higher‐risk offenders, focus on needs related to criminal behavior, be responsive to the offender’s unique issues, be delivered in the correct dosage, and be specified in the court’s sentence.

Risk/Need - See 1. Assess actuarial risk/need information. Roca clarified and narrowed target population to meet higher risk group, moving

from serving an at risk population to a high risk offender population. - Roca defined program outcomes and activities to drive toward those outcomes for target population that are grounded in the

evidenced based practices and principles and informed by effective interventions to address criminogenic needs including: antisocial behavior, antisocial personality, criminal thinking, criminal associates, employment and education, substance abuse, family and recreation.

- Please note: Roca will continue to serve a secondary population of young mothers who will be worked through a different targeted

intervention focused on relevant outcomes for young mothers)

Responsivity - See 1. Assess actuarial risk/need information. Roca clarified and narrowed target population to meet higher risk group, moving

from serving an at risk population to a high risk offender population. - Roca incorporated evidenced based practices and foundational theories of behavior change into core program model and

developed stage based programming to appropriately match participants to the right programming. Given the lack of specific data about the effect of evidence based practices with 18 – 24 year old young people in the community not under mandated criminal justice system custody and care, Roca selected components of evidence‐based practices and adjusted them for the specific population and setting (Hunt & Pierce, 2011).

- Using the Transtheoretical Model’s stages of behavior change (Prochaska & DiClemente, 1984), Roca developed and implemented

life skills, educational, pre‐vocational, and transitional employment programming based on the stages of change and allowing for multiple relapses in behavior. Staging all aspects of the intervention to the readiness of the young person is central to the Roca approach. Youth workers are trained to identify where the young person is in terms of readiness for change – not just once or at the beginning of the intervention but continuously throughout. This is done through the consistent youth worker participant interactions to observe language, attitudes, participation and other behaviors. This assessment of readiness informs the intensity (e.g., drop‐in versus a 10‐session class) of life skills, education and employment programming in which young people are engaged.

- CJI reported that at the core of Roca’s work is responsivity to meet young people where they are and remove barriers (CJI, 2012)


Page | 62



- When young people move from Phase I to Phase II in the model they work with their youth worker to develop a an individualized goal plan. The youth worker then supports them to participate in programming to make progress on their goals.

Dosage - Roca established contact and programming dosage standards to test and evaluate for accuracy and effectiveness. - Upon review of literature for evidence based criminal justice and substance abuse programming and review of Roca youth

outcomes, Roca revised overall program timeline from 3 years to 2 years and defined very clear program benchmarks throughout in order to coordinate and structure services more effectively.

- As reported by CJI (December 2011), Roca’s long held assumption that a minimum of two contacts each week through outreach

matters is correct – this frequency of contact is significantly associated with participant progression through the model and retention in the model. Youth worker contact with participants twice weekly moves participants into the behavior change‐focused phase of the intervention four months faster than those participants without that contact frequency.

atment Roca trains and supervises staff to work strategically and in a timely way with participants through the stages of change framework,

using motivational interviewing techniques and cognitive behavioral theory based approaches as appropriate - Young people needing additional mental health and/or substance abuse treatment are referred to certified providers and given

intensive support and follow‐up by youth workers to participate. Additionally, Roca works with substance abuse partners to raise additional funds to provide on‐site counseling to young people. Other

- Roca implemented its first replication site in Springfield focused on high risk young men ages 18 – 24. CJI conducted an evaluation study of the Springfield site to look at process and implementation of the replication. . This report is still in process of final edits.

- CJI conducted an implementation evaluation of the Roca programming in Chelsea, Revere and East Boston to look at fidelity, model

development and outcomes. This report is still in process of final edits. Next Steps

3. Roca is in the process of completing the transition of the following components of programming to separate the primary and secondary target populations including: a. Primary Target Population: Work for this population will include a clear identification of high risk young men appropriate for

the Roca Intervention Model; and, a determination of a process for all new referrals and/or identification of young people on the street meet this criteria. All new replication sites will focus on the target population.

b. Secondary Target Population: Roca will work to develop a clear identification of secondary target population in


Page | 63



Chelsea/Revere/East Boston of young mothers and very high risk young women. As part of defining the secondary target population, Roca will host a theory of change session focused only on young mothers for Chelsea, Revere, and East Boston. Roca has piloted a highly successful programming time block for young mothers and their babies; this will be expanded this spring. In future years, Roca will only serve secondary population in Chelsea, Revere and East Boston.

4. Roca has begun separating programming times and activities by target populations; this process will be completed by the start of

FY’13. 5. Roca has done extensive work on curriculum, protocols, and flow of programming for high risk young men. 6. Roca is in the process of finalizing an operations manual for the model including programming blocks, programming flow, safety

and emergency procedures.

1. Roca will continue to use data and will work with external evaluators to further develop the intervention model as an evidence‐based practice itself.

2. Roca is in the process of developing and will implement fidelity monitoring tools. 3. Roca will be examining the data and the research to continue to work on the dosage and frequency of certain program components

for the Alternative Sentencing/Community Corrections Program Model that the organization is developing. 4. Roca is improving and/or completing the following: a) program processes and procedures; b) staff training and development; c) life

skills, education and pre‐vocational curricula for stage‐based programming; and d) operations manual.

4. Skill Train with Directed Practice

Cognitive behavioral treatment methods have been shown to be effective at changing behavior and reducing recidivism. Examining thinking processes, role playing, and positive reinforcement are key components of this type of treatment.

- Roca staff members are trained in the concepts and techniques of Cognitive Behavioral Treatment. Roca designed and implemented staff training and coaching to increase skill and capacity to deliver model with fidelity, develop on‐going knowledge and skills to deliver cognitive behavioral based interventions, and increase effectiveness in moving young people to outcomes

- Roca designed and implemented the program model to create and support corrective experiences and opportunities for

restructuring distorted thinking in order to work on behavior change. All components of the model work together to keep young people engaged over time in a process of change in order to change behaviors and attain positive outcomes.

- Please note: Roca is clear that it has not implemented Cognitive Behavior Treatment (CBT) with fidelity. Additionally, we are clear

that if CBT is not implemented with fidelity that it is not effective. However, given the lack of age and setting specific CBT programs, Roca has begun to implement some components and techniques of CBT. Roca is clear that this needs further development.

- Roca designed and implemented the performance management system to reinforce practice, monitor implementation, assess

progress, study and continuously improve the model, and evaluate effectiveness and impact.


Page | 64



Next Steps 1. Roca will review evidence based Cognitive Behavioral Programs to determine if any of these specific programs could be

incorporated into the Roca Intervention Model and/or be used for specific purposes such as the Alternative Community Corrections Pilot. These models include: Thinking for a Change; Aggression Replacement Training; and Reasoning and Rehabilitation.

2. Roca will engage in a further review and study of Acceptance and Commitment Therapy (ACT) and the possible incorporation of the

practices into the model. 3. Roca will continue to use outcomes data to refine the model. Roca will prepare for an impact evaluation of its Intervention Model

in FY’14.

5. Increase Positive Reinforcement

When learning new skills and behaviors, people respond better to positive rather than negative reinforcement. Research suggests a ratio of four positive reinforcements for each negative. It is also important to note that even when applied sporadically, positive reinforcements can be effective, unlike negative reinforcements.

- Roca’s overall environment as a place based model is very positive, welcoming and nurturing. An underlying assumption of the model and the organization is that young people can make positive changes in their lives.

- Roca’s core values are Truth, Trust, and Transformation. The organization seeks to implement and make transparent those values

in all aspects of the work. - Roca staff deliberately does on‐going work to create a culture of hope and possibility for and with young people. - The programming includes weekly and monthly behavioral skill assessments to reinforce positive behaviors and identify areas for

improvement with participants. - The programming includes formal celebrations and completion recognition ceremonies. - Staff members receive positive reinforcement for successful work.

Next Steps

1. Roca will study evidence based practices in these areas in further detail as applied and integrated into criminal justice programming.

2. Roca will work with the evaluation team to document and further develop and systemize the efforts at positive reinforcement. 3. Expand and systemize staff recognition.

6. Engage Ongoing Support in Natural

Realigning offenders with prosocial support systems in their

- Roca is a placed based model that serves participants in the geographically defined area for this purpose. Roca staff deliberately work with young people to establish pro‐social relationships.which also foster additional prosocial skills,


Page | 65



Communities communities is critical for sustained behavior change. Attitudes and behaviors are strongly reinforced in one’s daily living environment. Without prosocial reinforcement in this setting, the chance of long term positive effects are diminished.

involvement in prosocial activities. - Roca hires staff and works with numerous volunteers and partners who live and work in the community to ensure that the

relationships and connections formed with participants can be supported in an on‐going way outside of straight programming time. - Roca uses practices including restorative justice peacemaking circles, community service, and work with engaged institutional

partners to increase the networks and practices of support available to participants in the community. - In addition to two years of intensive support and intervention, the model has a two year follow‐up period with less intensive

support to provide long term reinforcement. Next Steps

1. Roca will work with evaluation team to learn more about this work and adjust appropriately. 2. Roca will increase the use of restorative justice peacemaking circles.

7. Measure Relevant Processes

Measuring outcomes is crucial; it is what evidence‐based practices are based on. Offender as well as staff performance must be measured.

- Roca has a clear theory of change that was re‐visited and refined in November 2011. - Roca has clear logic model with activities, outputs, and short/intermediate/long‐term outcomes. - Roca has developed a detailed timeline with key benchmarks for programming and assessment that is being implemented in FY’12. - Roca has designed, implemented and refined a rigorous performance management and evaluation system to monitor progress of

participants, assess effectiveness of staff and programming, and inform continuous improvement. - Roca implemented a structured Staff Supervision Tool tracked in its performance management system to assess staff performance

and to link staff efforts toward the progress of participants. - Roca has and continues to work with external evaluators on implementation and outcomes evaluation to assess effectiveness.

Roca worked with CJI over a five year period. - Since Roca’s first theory of change and initial performance management system implementation, the organization uses data each

year to understand what is taking place day to day, learn about its programming, improve the work with staff, and improve the overall quality of programming. Over the past five years, Roca reviewed, refined, changed and updated the model each year. Based on these efforts, Roca finalized its intervention model and began the transition to full implementation of the refined model this fiscal year.

- Roca has engaged Chapin Hall for the next phase of its implementation and outcomes evaluation and to prepare itself for and then

implement an impact evaluation targeted for FY’14.


Page | 66



Next Steps 1. Roca will work with Chapin Hall to increase capacity for reporting outcomes and indicators for individual participants and cohorts

over time as well as annually. 2. Roca will continue to use “real time”, quarterly and annual data to monitor programming and make improvements as appropriate. 3. Roca will finalize a programmatic and organizational scorecard. 4. Roca will seek an outside review of the model as it aligns or doesn’t with evidence‐based correctional practices.

8. Provide Measurement Feedback

Providing feedback increases accountability and has been associated with enhanced motivation for change.

- Roca implements a regular practice of using data daily, weekly, monthly, quarterly, and annually with staff and partners to assess participant progress and organizational capacity.

- Roca has increased its capacity for reporting to ensure the availability of data for immediate feedback and improvement purposes

each year. - Roca uses data with participants to assess and demonstrate progress toward outcomes in order to support motivation for change. - Roca is currently finalizing the performance management system to connect staff supervision and training to annual review process

Next Steps

1. Roca will document and implement organizational feedback processes with participants, staff, managers, board, and other invested parties to establish consistent practices.

2. Roca will finalize the performance management system to connect staff supervision and training to annual review process. 3. Roca will develop a programmatic and organizational scorecard for review and continuous improvement.

Appendix C: Organizational Charts


FY 2013 to FY 2017 Page | 70

The following includes brief bios for Roca’s Board Leadership leading the strategic planning initiative and the current members of Roca’s senior management team.

DWIGHT ROBSON, BOARD CHAIRMAN is the Vice President for Public Strategy for The Mentor Network. Dwight leads the work of the Public Strategy Group, which is responsible for developing and implementing The Network’s agenda with regard to Marketing, Communications and Government and Community Affairs. Prior to joining The Network in 2003, Dwight held senior policy and management positions in Massachusetts state government, most recently as an assistant state treasurer. He also has extensive political experience after holding several key positions in high‐profile campaigns for Governor and the U.S. Senate and formerly served on the board of The American Network of Community Options and Resources (ANCOR). Dwight received a B.A. in political science from the University of Massachusetts‐Lowell.

MARK HAGGERTY, BOARD TREASURER. Mark Haggerty is president of Fidelity Capital Markets (FCM), the institutional trading arm of Fidelity Investments providing trading, products and services to a wide array of clients and Fidelity businesses. Fidelity Investments is the largest mutual fund company in the United States, the No. 1 provider of workplace retirement savings plans, a leading online brokerage firm and one of the largest providers of custody and clearing services to financial professionals. Mr. Haggerty joined FCM in 1996 in the risk management and strategic planning group, where he focused on market risk management, and new products development. Subsequently, Mr. Haggerty took on management responsibilities for a number of trading desks, including securities lending, portfolio trading, and structured products. In 2003, he spearheaded FCM's entry into the Prime Brokerage business and added this unit to his management responsibility until he was named executive vice president of FCM in June 2005. Mr. Haggerty assumed his current role in 2006. Prior to joining Fidelity Investments, Mr. Haggerty worked for KPMG Peat Marwick in Philadelphia and New York as a consultant, providing risk management consulting services to derivatives trading desks. Mr. Haggerty received a bachelor of science degree in economics, summa cum laude, with concentrations in finance and accounting from The Wharton School of the University of Pennsylvania. Mr. Haggerty previously served as a member of the Board for the Boston Equities Exchange and on the advisory board of Redibook LLC. Mark is FINRA Series 4, 7, 9, 10, 24 & 63 licensed.

STEPHEN KANEB. Steve Kaneb has years of experience in various fields. Currently, he is Vice President of Real Estate at Catamount Management Corporation and HP Hood LLC. He is also Founder and Chief Operating Officer of Verdant Pastures, a 93 acre mixed‐use development in Epping, New Hampshire emphasizing community, economic and environmental stewardship. Steve earned his Bachelor of Science degree in Mechanical Engineering from Worcester Polytechnic Institute and is a Registered Professional Engineer. His development experience includes retail, office, warehouse, industrial and residential. Throughout his career, Steve has maintained strong philanthropic and community involvement. He currently serves Farmsteads of New England (Director); International Medical Equipment Collaborative (Chairman); New Hampshire Institute of Politics Advisory Board; Roca (Board of Directors’ Vice President and member of Executive Committee); Roman Catholic Diocese of Manchester (Finance Council Member); The Catholic University of America (Trustee); and, the Town of South Hampton Conservation Commission (Chairman). He also serves at the Rockingham County Jail as a Eucharistic Minister. Steve resides with his wife, Andrea, and their five children in South Hampton, New Hampshire.

GREG TORRES, HONORARY BOARD MEMBER. Greg Torres is currently President of MassINC. Greg Torres joined MassINC in June of 2007. As President, he is responsible to the Board of Directors for setting policy, leading fund raising activities, and guiding program operations. Greg also serves as publisher of CommonWealth magazine, now a leader in investigative journalism. Greg began his career working with juvenile delinquents in the early 1970s in Boston. As the de‐institutionalization movement proceeded, Greg was instrumental in developing community‐based programs for adolescents with his work at the Massachusetts Committee on Criminal Justice. As Assistant Secretary for Criminal Justice under Governor Michael Dukakis, he led reform efforts in the adult correctional system as well. From 1984‐1992 Greg served as Chief of Staff to the Massachusetts’ Senate Committee on Ways and Means.In 1992 Greg joined the MENTOR Network as Senior Vice President, having served as a founding board member in 1980. Assuming the role of President and CEO from 1996‐2005, Greg led the growth of MENTOR from a regional company providing services primarily to children into a national organization serving people of all ages in a wide variety of settings, now operating in thirty‐seven states. Having retired as CEO, Greg continues to chair the board of the

Appendix D: Biographies of Leadership Team and Board of Directors Listing


FY 2013 to FY 2017 Page | 71

MENTOR Network. Greg is a graduate of the John F. Kennedy School of Government at Harvard University (MPA 1982) and of St. Vincent’s College in Latrobe PA (BA 1971). Greg chaired the board of Roca, a Chelsea‐based organization from 2006‐2009. He lives in Winchester with his wife Betsy Pattullo. They have two sons, Jess and Gabe, and two grandchildren, Jack and Lydia.

KELLY KALICK. Kelly Kalick serves as the Treasurer of the Roca Inc. Board and has served in this capacity since March 2010. Prior to serving on the Roca Board she served as a member of the Roca finance committee. Ms Kalick is President of Harbour Food Service Equipment, a family owned and operated food service equipment and supply company located in Chelsea, Ma. It is through Harbour that Ms. Kalick first became involved with Roca when Harbour was asked to assist in the expansion of the Roca culinary dept. In the past Ms. Kalick has been involved in fundraising efforts in both Boston and Cambridge. Ms. Kalick lives in Boston.

KEVIN BURKE. Kevin Burke is currently a visiting professor at Endicott College. Kevin is a former District Attorney and Former Secretary of The MA Executive Office of Public Safety and Security. As Massachusetts Secretary of Public Safety and Security, Kevin led 15 departments and agencies including the state homeland security division, the state police, the department of corrections, and the National Guard. As the district attorney for Essex County, he was a leader in developing legislation and legal policy for Massachusetts and for the nation. He was the author of the Massachusetts Victim’s Bill of Rights, and he founded the state’s first child sexual abuse unit, which became a national model. He also served in the Massachusetts House of Representatives.

ANTHONY DIPAOLO. Anthony DiPaolo is President and CEO of Work'N Gear. Anthony DiPaolo is a seasoned entrepreneur and executive with over 30 years of leadership experience in retail, manufacturing, product development, product sourcing and domestic and international sales. Mr. DiPaolo has also committed his work ethic and leadership to various volunteer and philanthropic activities in his community. As a graduate of The George Washington University, Anthony Dipaolo began his career with Mitsubishi as a Sales Representative in their footwear division responsible for $20MM in sales and related product development. He traveled extensively throughout Europe and the Far East brokering deals between Dow Chemical and the Romanian government, and trade agreements with other Eastern Bloc countries. He later became the Vice President of Sales for Shoe Visions, a fashion footwear brand sold through major retailers in the United States including Kinney, Thom McAn, and JC Penney. Upon selling Herman Survivors, Anthony DiPaolo purchased Work ‘N Gear out of bankruptcy, a 45‐store chain specializing in the sale of workwear apparel and footwear. He assumed the role of CEO and President of Work ’N Gear and raised $40MM in venture capital investments. Since 2002, Mr. DiPaolo has built a new infrastructure, a strong management team, and redesigned and repositioned the brand in the marketplace. In addition to his successes in the business world, Anthony has made important contributions to his community and beyond. While at Work ‘N Gear, he developed the “Boot Trade” program. Each year, before the cold of winter hits the northern states, this program provides thousands of pairs of gently worn boots to homeless people in 12 states. Anthony has always been committed to mentoring children. He has been an active volunteer and Coach of Youth Baseball in his current hometown of Cohasset, MA and was formerly the President of Greenwich Junior Babe Ruth Baseball in Greenwich, CT. He currently sits on the Board of Directors of Roca, an outreach program for at‐risk youths. Anthony DiPaolo and his wife, Gloria, are proud parents of their two college‐aged boys who are currently attending the University of South Carolina. Anthony is an active outdoorsman. In his free time, he enjoys sport fishing on his boat with his boys, his friends and his wife, playing a friendly round of golf, and skiing.

Roca Senior Leadership Team

Molly Baldwin is the Founder/Executive Director of Roca. She has been a tireless advocate, mentor and community convener reaching out to the highest‐risk young people from our area’s most dangerous urban communities, and bringing together the major institutions, corporations, and agencies that affect these young people’s lives. Over the past 23 years, Molly’s efforts have helped over 25,000 young people make profound and positive changes in their lives.

Molly launched Roca from a single grant from the Teen Challenge Fund of Massachusetts in 1988 to address teen pregnancy and violence in Chelsea. Today, Roca serves young people from Chelsea, Revere, East Boston, Charlestown, Winthrop and Springfield and operates its nationally recognized Intervention Model to promote behavioral change with our highest‐risk youth. Pulling from a variety of evidence based practices in criminal justice, youth development and workforce development, Roca continues


FY 2013 to FY 2017 Page | 72

to boldly help young people grow, learn and become agents of change in their own lives and in the community, moving them out of violence and poverty and keeping them out of prison. A graduate of UMass, Amherst, Molly began her professional life as a street youth worker in Revere, relentlessly showing up for troubled youth, at their homes, on the street, all the time, until they came to trust a single adult who would not fail them. These transformational relationships are still the bedrock of Roca’s work, as a committed team of youth workers similarly and intensively reach out to over 700 disengaged and disenfranchised young people each year. Currently completing her Master’s Degree in Education from Lesley University, Ms. Baldwin is the recipient of numerous regional and national awards including: The Boston Ten Point Coalition Award, The Fraternidad Guatamaleca de Massachusetts Recognition Award, The Instituto de Salud Latina Award, and the American Medical Association National Congress on Adolescent Health Award for Excellence in Intervention to name a few.

Lili Elkins has served as Roca’s Chief Strategy and Development Officer for the past three years and prior to that served as a consultant to Roca for five years. Prior to coming to Roca, Ms. Elkins worked as a full‐time consultant working with non‐profit and governmental organizations, providing assistance with strategic planning, business planning, program design and development, operations, financial management, resource allocations, performance measurement, data collection, resource development and other programmatic related areas.

In addition to her work at Roca, Ms. Elkins is also an Associate Professor at Columbia University’s School of International and Public Affairs, where she taught Government and Nonprofit Finance (Ms. Elkins is currently on leave from this position.) Ms. Elkins has also taught for the Columbia Business School’s Institute for Nonprofit Management, the University of Pennsylvania, the Universidad Externado de Colombia , the Commonwealth of Pennsylvania, the Garden State Area Health Education Center, the US Department of Health and Human Services, and the Ford Motor Company International Fellowship Program, among others. Ms. Elkins served as the Executive Vice President and Chief Operating Officer for the Transitional Work Corporation for a period of approximately two years. As a founding leader of the organization, Ms. Elkins helped to start up and operate one of the nation’s largest transitional employment programs in the country, serving more than 2,200 long term public assistance recipients each year. Prior to her work with TWC, Ms. Elkins was the Deputy Director for the Philadelphia AIDS Consortium, a regional planning council and fiscal agent responsible for more than six million dollars in HIV funding. Ms. Elkins has also worked at the United Nations as an educational program developer and at the Port Authority of New York and New Jersey, focusing on information technology and data security. Ms. Elkins has over 20 years of fundraising, program design and program management experience working with non‐profits. Ms. Elkins, a member of the New Jersey Bar, has her JD from Rutgers University, an MA in public policy and administration from Columbia University, and a BA from Barnard College, Columbia University.

Jacob Sauerborn is the Chief Financial Officer of Roca. Jacob is a skilled manager of finance with demonstrated ability to effect positive organizational change, and to strengthen financial and structural positions. Jacob brings a valuable blend of skills in budgeting, information systems, accounting & finance and human resources to the Roca team, as well as experience in reporting to Boards of Directors. Jacob has over 30 years of experience in the financial field.

Thomas McKenna is the Managing Director for Massachusetts and will oversee all aspects of program operation in Chelsea, east Boston, Revere and Springfield, MA. Prior to working for Roca, Tom was at the Brooklyn Democracy Academy High School, where he was founding principal of the transfer high school serving 200 over‐age/under‐credited and at‐risk high school students. He was previously Assistant Principal at the Island Academy in East Elmhurst, N.Y., where he supervised a staff of 93 educators and support staff in the largest correctional education short‐term facility in North America (Rikers Island) servicing approximately 5,000 students per year. Previously, McKenna served as college instructor, a new teacher mentor, literacy coach and English language teacher at a variety of New York City schools. At Roca, he will oversee and supervise approximately a staff of 50 in Chelsea and Springfield, which include assistant directors, youth workers, programming and operations for the sites.


FY 2013 to FY 2017 Page | 73

Anisha Chablani is the Chief Knowledge Officer of Roca. Anisha began her career at Roca as a street worker in 2000, and began running programming in 2002. As the Chief Knowledge Officer, Anisha’s overall responsibility is for staff coaching, ensuring program fidelity and continuous improvement, especially leading the effective driving towards outcomes and the utilization of program evaluation system. Anisha has over 15 years of direct service experience working with very high risk young adults and has over 8 years of program and organizational management experience. Ms. Chablani has a Bachelor of Arts Degree from Boston College and is currently completing her Master’s Degree in Education from Lesley University.

Dana Betts is Roca’s Director of Programming, responsible for all programming operating out of Roca’s Chelsea site. Dana has worked at Roca since 2005, starting her work as a youth worker and growing in responsibilities over that time. Before joining Roca she worked as a DSS family unification monitor and DYS tracker for Spectrum Health Systems in Chelsea, MA. In this role, Ms. Betts worked with DSS involved adolescents and families in establishing healthy familial relationships. Prior to her work in Chelsea, Dana was a residential teacher at Boston Higashi High School in Randolph. She earned a Bachelor of Arts degree from the College of the Holy Cross, majoring in Sociology and Latin American Studies. Dana is currently working towards a Master’s Degree in Education from Lesley University.

David (Jake) Jacobs began as Roca’s Director of Business Development and Transitional Employment in 2012. Jake now serves as the organization’s Director of Business Development and Major Gifts. Jake Jacobs is a creative Senior Sales Executive with over 30 year of experience and sustained success through individual sales activity and programs driven through direct and indirect sales teams ‐ with a focus on reworking existing business models, creating new sales organizations and building strategic partnerships that allow for mutual growth and evaluating all possibilities to gain greater success.

Yusef Id‐Deen currently serves as Roca Springfield’s Assistant Director. In this role, Mr. Id‐Deen is responsible for providing oversight for all program operations, facilities, and community relations at Roca’s Springfield location. Prior to this, Yusef served as supervisor for the Girls’ Club Family Center in Springfield. Mr. Id‐Deen has over five years of experience working with very high risk young people, including those with criminal records. Mr. Id‐Deen also worked for the Hampden County Sheriff’s office, providing individual counseling and casework through both the Western Massachusetts Correctional Alcohol Center and the Community Safety Center. Mr. Id‐Deen holds an Associate’s Degree from Springfield Technical Community College, a Bachelor’s Degree in Sociology from Westfield State College and a Master’s in Social Work, focusing on mental health and substance abuse, from the University of Connecticut School of Social Work.


FY 2013 to FY 2017 Page | 74

BOARD OF DIRECTORS LISTING October 17, 2012

Name/Title Affiliation/Address Representation

Dwight Robson, Board Chariman, Strategic Committee

Chief Public Strategy & Marketing Officer, The Mentor Network 313 Congress St Fl 5th, Boston, MA 02210 W: 617‐790‐4800 [email protected]

Business Leader

Jan Callahan, Vice Chairperson Development Committee

Palmer Paving Corporation 25 Blanchard Street, Palmer, MA 01069 W: 413‐283‐8354

Business Leader, Community Leader

Mark Haggerty, Treasurer, Finance Committee

President, Fidelity Capital Markets Services 200 Seaport Blvd, Z2H, Boston, MA 02210 W: 617‐563‐1729 C: 617‐678‐9588 [email protected]

Business Leader

Deborah Wayne, Clerk, Finance Committee

Owner, Deborah Wayne and Assoc.380 Broadway, Chelsea, MA 02150 W: 617‐884‐1222, 781‐321‐3222 [email protected]

Business Leader,Community Leader, Civic Leader, Chelsea Resident

Greg Torres , Honorary Board Member

President, MassINC18 Tremont Street, Boston, MA 02108 W: 617‐224‐1703 C: 617‐285‐5097

Business Leader

Molly Baldwin, Board Member, Strategic Committee, Business Development Committee

Executive Director, Roca 101 Park Street Chelsea, MA 02150

Civic Leader, Community Leader, Chelsea Resident

Kevin Burke, Board Member Strategic Committee

Former Secretary of the Executive Office of Public Safety and Security Visiting Professor, Endicott College Email West Classroom Building Rm #501 Endicott College, 376 Hale Street, Beverly, MA 01915 [email protected] (978) 232‐2380

Civic Leader

David Canter, Board Member EVP, Practice Management & Consulting Fidelity Institutional Wealth Services 200 Seaport Boulevard, Z2A, Boston, MA 02210 W: (617) 563‐7409 [email protected]

Business Leader

Jacqui Conrad, Board Member, Development Committee

Founder, delaCruz Communications, LLC 6 Cabot place, Suite 6, Stoughton, MA 02072 W: (781)341‐3800 [email protected]

Business Leader

Dallas Cruz, Board Member Associate, Choate, Hall & Stewart LLP Two International Place, Boston , MA 02110 W: (617)248‐4013 [email protected]

Business Leader, Program Alumni

Anthony DiPaolo, Board Member, Development Committee

President, CEO, Work'N Gear2300 Crown Colony Drive, Suite 300, Quincy, MA 02169 W: 781‐746‐0125 [email protected]

Business Leader


FY 2013 to FY 2017 Page | 75

Name/Title Affiliation/Address Representation

Sandra B. Edwards, Board Member, Strategic Committee

Vice President of Institutional AdvancementNorth Shore Community College 1 Ferncroft Road, Danvers, MA 01923 W: 978‐762‐4000 x5497 [email protected]

Business Leader

Matthew Feinberg, Board Member

Attorney at Law Feinberg and Kamholtz 125 Summer Street, 6th Floor, Boston, MA 02110 W: 617‐431‐5368

Business Leader

Brian Fitzgerald, Board Member, Strategic Planning Committee

President & CEO, Fitzgerald Company, Inc. 88 Elm Street PO BOX 268, West Springfield, MA 01090 W: (413) 747‐ 4100 [email protected]

Community Leader, Business Leader

Paul Francisco, Board Member Vice President, Global InclusionState Street One Lincoln Street, Boston, MA 02111 617‐664‐6966

Civic Leader, Business Leader

Gary Greenberg, Board Member, Development Committee

Co‐Managing Shareholder, Greenberg Traurig, LLPOne International Place, Boston, MA 02110 W: 617.310.6000 [email protected]

Business Leader

Kelly Kalick, Board Member, Development Committee

Harbour Food Services229 Marginal St, Chelsea, MA 02150 617‐884‐3900 x222 [email protected]

Business Leader Community Leader

Stephen J. Kaneb, Board Member, Development Committee, Finance Committee

Catamount Management Corporation6 Kimball Lane, Lynnfield, MA 01940 W: 617 660 7403 [email protected]

Business Leader

Eric Rodriguez, Board Member 37 East View Drive , Milford MA 01757 (508) 377‐8866 [email protected]

Business Leader, Program Alumni

Ramiro Torres, Board Member, Development Committee

JAM'N 94.5, WJMN FM Boston10 Cabot Road Suite #302, Medford, MA 02155 W: 781‐663‐2500 [email protected] [email protected]

Business Leader

ADDITIONAL COMMITTEE MEMBERS

Name/Title Affiliation/Address CDC Representation

Jay Ash, Business Committee City Manager, City of Chelsea300 Broadway, Chelsea, MA 02150 617‐466‐4100 [email protected]

Civic Leader

Shuvam Bhaumik , Audit Committee Member

Wealth ManagementMorgan Stanley Smith Barney LLC 53 state st. ‐ 39th fl. ‐ exchange place | boston, ma. 02109 b: 617.589.3215 | f: 617.227.9070 [email protected]

Community LeaderBusiness Leader Chelsea Resident


FY 2013 to FY 2017 Page | 76

Name/Title Affiliation/Address CDC Representation

Ed Dolan, Strategic Committee Deputy Commissioner, Department of Youth Services Tower Point 27 Wormwood Street, Suite 400, Boston, MA 02210 [email protected] (617) 960‐3303

Civic Leader

Len Engels, Esq., Strategic Committee

Managing Associate for Policy at Crime and Justice Institute, Community Resources for Justice 355 Boylston Street, Boston, MA 02116 617‐482‐2520 x129 [email protected]

Civic Leader

Peter Nordblom, Strategic Committee, Business Committee

President, Nordblom Company15 Third Avenue, Burlington, MA 01803 [email protected] (781) 238‐4804

Business Leader

Dennis McGurk, Business Committee

283 Concord St, Gloucester, MA 01930(978) 281‐5021 [email protected]

Business Leader

Cindy Montoya, Development Committee

Diversity Manager, Stop N’ ShopStaffing & Diversity Dept. Ahold USA (617) 770‐8615

Business LeaderCivic Leader Chelsea Resident

Stephen Stoute, Finance Committee

47 Orange Street, Apt #3Chelsea, MA 02150 (857) 251‐4959 [email protected]

Roca Alumni, Community Leader Chelsea Resident


FY 2013 to FY 2017 Page | 77

Roca Pay for Success Summary Data Sheet

Overall State Budget Information

Total State Budget Allocated to Trust $ 25,000,000

Required for SPV/Intermediary $ 1,500,000

Required for YOU $ 1,000,000 Balance Potentially Available to Roca $ 22,500,000

Participants Served

Total Unduplicated Participants 894

Overall Roca Budget

Total Budget $ 22,002,482

Program Budget $ 21,477,482

4 Year Cost Per Participant $ 24,024

Evaluation Cost $ 525,000

Roca Capacity (Constrained by Budget)

TR1/2 Slots at any given time ‐ Chelsea hub 225

TR1/2 Slots at any given time ‐ Springfield hub 175

Incarcerations Avoided

Estimated Incarcerations Avoided 412

Risk Sharing Information

% of Risk ‐ Incarceration Payments Only 15.00%

Total Roca Risk ‐ Incarceration & Employment Payments 0.00%

Up Front Payments

% of Payment for Incarceration Avoided 85%

Quarterly Payment per TR1/2 $ 2,155 Quarterly Payment per TR3 $ 1,340

Proposed Success Payments

Incarcerations Avoided $ 53,500

Qtr. Payment Employment Under $3K/Quarter $ ‐

Qtr. Payment Employment Over $3K/Quarter $ ‐ Note: Proposed payment structure likely to change to include a small payment for engagement in employment.

State Funding

Total State Funding Required if all Deliverables Are Met $ 22,051,441

% of Payments from Incarcerations Avoided 100.00%

Total Payments from Incarcerations Avoided $ 22,051,441

% of Payments from Employment 0.00%

Total Payments from Employment $ ‐ Note: Proposed payment structure likely to change to include a small payment for engagement in employment.

Appendix E: Pay for Success Preliminary Financial Information


FY 2013 to FY 2017 Page | 78

Start Up Capital

Minimum Required to Meet Cash Flow Needs $ 2,500,000

Ending Cash Balance Based on Raise of Minimum Start Up $ 3,496,284

Surplus at End of Grant $ 996,284

Risk to Roca Core Financing Based on Minimum Capital Raise

0% of Outcomes Achieved $ 796,128

50% of Projected Outcomes Achieved $ 398,064

75% of Projected Outcomes Achieved $ 199,032

Fundraising to Cover 100% of Roca Risk

Maximum Required to Cover 100% of Roca Risk $ 4,400,000

Ending Cash Balance Based on Raise of Maximum Start Up $ 6,560,745

Surplus at End of Grant $ 2,160,745

Note: All Capitalization numbers are preliminary and are subject to change based on final payment structures

Intermediary/SPV Financials

Initial Grant Funding Required $ 11,000,000

Initial Loan Funding Required $ 11,000,000

Final SPV Funds Available to be Allocated Across Parties TBD

Incarceration Avoided Payment Needed to Include Intermediary/YOU $ 57,005


FY 2013 to FY 2017 Page | 87

Appendix G: FY 12 Outcomes Report Appendix F: FY 12 Outcomes Report


FY 2013 to FY 2017 Page | 88

Roca’s Performance Benchmark And Outcomes Report

Fiscal Year 2012 July 2011 – June 2012


FY 2013 to FY 2017 Page | 89

Organizational Overview ................................................................................................................................. 1

Effort to Become High Performing ........................................................................................................................... 1

FY12 Implementation Changes ................................................................................................................................. 2

Organizational Participant Enrollment ............................................................................................................ 4

Enrollment and Eligibility .......................................................................................................................................... 4

FY12 Enrollments ...................................................................................................................................................... 5

Attrition Reasons ...................................................................................................................................................... 5

Demographics ........................................................................................................................................................... 5

Primary Target Population Performance Benchmarks ..................................................................................... 6

Total Served .............................................................................................................................................................. 6

Attrition Reasons ......................................................................................................................................... 7

Demographics .............................................................................................................................................. 7

Risk Factors .................................................................................................................................................. 8

Relentless Outreach ................................................................................................................................................. 8

Transformational Relationships ............................................................................................................................... 9

Phase 1 Contact Standards .......................................................................................................................... 9

Phase 2 Contact Standards .......................................................................................................................... 10

Stage Based Programming........................................................................................................................................ 10

Programming Engagement .......................................................................................................................... 10

Programming Standards .............................................................................................................................. 11

Educational and Prevocational Programming ............................................................................................. 11

Work Force Readiness .................................................................................................................... 12

Transitional Employment ............................................................................................................... 13

Job Development and Placement .................................................................................................. 15

Reducing Arrests and Technical Violations .............................................................................................................. 15

Phase 3 Intermediate Outcomes ..................................................................................................................... 16

Lessons Learned and Next Steps ...................................................................................................................... 17

Table of Contents


FY 2013 to FY 2017 Page | 90

Efforts to Become High Performing

Roca’s journey to becoming a high performing, effective non‐profit started many years ago when it began asking a critical question—Are we helping young people change their lives and how do we know? After realizing that, in spite of great dedication and hard work, Roca was not helping young people change behaviors to improve their lives to a significant degree; Roca took stock and rethought what it was doing. Roca understood that just creating a place for young people to belong or be engaged in activities wasn’t good enough. Rather, Roca realized that these young people deserved and needed the organization to get better at its mission to move them out of harm’s way and toward economic independence. Roca had to learn how to help them change and learn new behaviors.

In order to do this, Roca had to:

Understand how proven theories, lessons, and skills could be applied and practiced with a group of young people most give up on;

Study, adapt and implement elements of evidenced based programs and approaches based on foundational theories of behavior change;

Research and study evidence‐based practices and programs in an effort to more effectively serve our communities highest risk young men who without an intervention face a reality of jail, violence, and poverty;

Work to align the Intervention Model and Roca’s organizational practice with the Eight Evidenced Based Principles of Effective Intervention21 identified for community corrections;

Build on its own direct experience of what worked and what did not to refine and develop new approaches to improve the likelihood of impacting the lives of our young people;

Deepen its capacity to examine the ways in which it collected and used data regularly to continuously inform its strategic and operational goals; and,

Be accountable to its young people and with full faith and transparency, figure out how to help them change their lives or shut down.

Roca engaged in a systemic cycle of research, design, action, tracking data, and use of data for continuous improvement to deliver an intervention worthy of the young people we serve. As a result of this learning and commitment to performance management and continuous improvement, Roca has made significant progress with the design and coaching of the model and the development of a performance management system.

Since 2005, when Roca went through its first Theory of Change Process, the following areas have been refined and clarified every year resulting in significant impacts on Roca’s learning and evaluation agenda.

Clarity of the Model Moved from a multi‐service youth development model to a single service intervention model designed to

intentionally move a targeted group of very high risk young people to outcomes. Developed and operationalized a model based on the Transtheoretical Model/Stages of Change, Cognitive Behavioral

Theory and Motivational Interviewing to create a framework for an entire experience, with multiple opportunities for young people to work on long term behavior change.

Started looking at and using evidence based principles from the criminal justice system to help young people learn how to change their behavior.

21 Crime and Justice Institute (2004). Implementing evidence‐based practice in community corrections: the principles of effective intervention.

Boston, MA: Author. Available online at: http://crjustice.org/cji/evidencebased.pdf.

Organizational Overview


FY 2013 to FY 2017 Page | 91

Performance Based Management Built a comprehensive system to track data and to use the data to inform the on‐going continuous improvement of:

the model, staff coaching and training, programmatic operations, and data collection and quality assurance practices. Developed an extensive training and supervision agenda to support and enhance the ongoing skills, knowledge, and

capacity of staff to deliver the model as intended.

Stage Based Programming Developed capacity to deliver programming that meets young people where they are in their readiness to engage

and learn. Roca’s Stage‐Based Programming was designed to engage young people where they are in the stages of change cognitively and behaviorally. The Stages of Change include: Pre‐contemplation, Contemplation, Planning, Action, and Sustaining. The Stages of Change Process also recognizes Relapse as a fundamental part of the process of behavior change. Roca incorporated this fundamental principle of the Transtheoretical Model of Behavior Changes (Prochaska and DiClemente, 1984) into its programming – a “stage‐matched” approach. This means that the activities or approaches are designed to match where each young adult is in the progression in order move them progressively through the stages of change toward successful outcomes.

Trained Youth Workers to identify where the young person is in terms of readiness for change – not just once or at the beginning of the intervention but continuously throughout. This is done through the consistent youth worker participant interactions to observe language, attitudes, participation and other behaviors. This assessment of readiness informs the intensity (e.g., drop‐in versus a 10‐session class) of life skills, education and employment programming in which young people are engaged.

Worked rigorously on building and evolving its workforce readiness and pre‐vocational programming to create meaningful curriculum, skill building, and pathways that lead to real employment opportunities for the target population.

FY12 Implementation Changes

Model Timeline

In July 2011, Roca changed the length of the Intervention Model to a 2 year intensive model with 2 years of follow up. This change was made as a result of research on effective models in corrections and reviewing our data and discovering that young people who were in the model too long were beginning to do worse. We had to move them out of the model earlier for them to maintain their momentum for long term behavior change. Through further examination of the participant data, we studied the participants who were doing the best with employment and reduction in criminal involvement outcomes. On average, these participants had spent 24‐28 months in Roca’s Intervention Model. Based on the data, we reduced the intensive portion of the model from 3 years to 2 years to increase the success rate of our young people.

Supervision Tools

In July 2011, Roca implemented new standardized supervision tools in Efforts to Outcomes. The new tools significantly increased our capacity to monitor the fidelity of implementation of the transformational relationships and to identify staff coaching and training needs

Primary Target Population

In November 2011, following our 3rd Theory of Change Process, Roca refined and narrowed the primary target population for the Intervention Model to focus on serving:

Young Men;

17‐24 years old;

Justice System Involved or Juvenile Justice Systems involved w/risk indicators predictive of adult criminal justice system involvement;

No employment history; and,

Live in contextually determined radius that promotes participation in center based services.


FY 2013 to FY 2017 Page | 92

Roca’s choice to work with this primary population was based in part on the risk principle. According to the Crime and Justice Institute, “The risk principle identifies who to target for the most intensive of services and programming. Specifically, limited resources should be directed to those at highest risk for involvement in the criminal justice system. Further, higher risk clients should receive the greatest dosage of treatment and intervention. This principle is of critical importance given the serious constraints of limited budgets and working with growing community supervision and prison and jail populations.”22 Additionally, in examining and learning about our work through the first replication in Springfield, which only serves young men, it became even clearer that focusing on this population increased the capacity to implement the Intervention Model and drive toward outcomes. Roca knows that it can help these young men create the behavior changes critical to keeping them out of prison and engaged as members of our communities.

This change in target population required a re‐structure of the Roca Chelsea Site to align with newly defined target population and accommodate its secondary population of young mothers. The change also initiated a process of completing young people out of Roca who had been engaged in Roca prior to the new model and who did not meet the target population. Roca also clarified its enrollment, intake, and eligibility practices to ensure that the young people to be served in the model meet the appropriate risk level. Finally, Roca clarified its benchmarks and outcomes for the target population. As indicated in the Participant Timeline below, Roca has become very clear on how to move its young people through the model in order to support behavior change, skill acquisition, and the attainment of long term outcomes.

Secondary Target Population

In FY12 Roca also committed to continued service of a secondary population of high risk young mothers. While traditional services for young mothers exist within our communities, there are few, if any, programs designed to target very high risk young mothers who are involved in high risk behaviors and refuse voluntary services. Many young mothers are: street involved; gang involved; abusing drugs/alcohol; involved in domestic violence situations; involved in the juvenile justice system; involved with the foster care system; have dropped out of school; are unemployed; have been victims of abuse and neglect; struggle with trauma/mental health problems; have unstable living conditions; are isolated immigrants/refugees;

22 Bechtel, Kristin and Barbara A. Pierce, MPPM. “An Overview of What Works in Correctional Interventions.” Crime and Justice Institute (January 2011).


FY 2013 to FY 2017 Page | 93

are engaged in risky sexual behavior; and, are at high risk of secondary/tertiary pregnancy. This is sad for the young women themselves and often tragic for their children.

In Chelsea, Revere, and East Boston, Roca works with 200 young mothers a year. Roca operates the Harbor Area Healthy Families Program for 100 first time pregnant and parenting teens. Roca also works with a higher risk group of young mothers combining its Intervention Model with parenting education. Healthy Families, based on the national model of Healthy Families America, is a statewide program that is funded and administered by the Children's Trust Fund (CTF). The goals of the program are to: Reduce child abuse and neglect by supporting nurturing parenting; Achieve optimal health, growth, and development in infancy and early childhood; Promote educational attainment, job readiness, and life skills; Reduce repeat teen pregnancies; and Promote optimal parental health and wellness. Funded by the Massachusetts Pregnant and Parenting Teen Initiative (MPPTI), the Mass Department of Children and Families Young Parent Support Program, and several private foundation, Roca serves an additional 100 young mothers with a combination of its high risk intervention model, parenting education, and nursing support. Using effective engagement efforts, Roca successfully identifies, recruits, and retains very high risk young mothers for up to two years in the programming.

Page | 93


Page | 4

Enrollment and Eligibility Process

Young people are initially connected to Roca through various means. Youth Workers identify young people as potential participants through their intensive outreach and street work. Many young people are referred through partners including: local Police Departments, District and Superior Probation Departments, Courts, the Department of Youth Services, schools, health clinics, and other community based organizations.

When a young person is identified as a potential Roca participant, the young person is assigned to a Youth Worker who begins making contacts with the young person and gathering information to assess eligibility. From the first day of enrollment, Youth Workers have 60 days to determine the young person’s eligibility for Roca’s Intervention Model. Eligibility requirements are based on meeting the target population criteria. Additionally, Roca seeks to serve young people who are not able to engage in other programs, who are resistant to engagement and change, and who are in need of an intervention strategy.

Young people are considered ineligible for the following reasons:

they are not in need of an intensive intervention;

they do not meet the target population criteria;

they cannot participate in a cognitive behavioral model;

they are incarcerated and cannot be accessed for contacts/services;23

they moved out of the geographic area and cannot be accessed for contacts/services; and/or

they are unable to be contacted/located.24

If it is determined that a young person is ineligible with in the first 60 days, they are removed from the Youth Worker’s caseload. Young people can only be removed from a caseload by a senior administrator. Therefore, a removal request has to be submitted with the ineligibility reason and it must be approved by the supervisor in order for the dismissal to take place. If a participant remains on a caseload after 60 days from enrollment, Roca considers itself accountable for driving to outcomes with the young person.

Given the change in Roca’s primary target population, eligibility criteria were aligned with the newly defined primary target population criteria beginning in November 2011. Any young person who did not meet the new criteria was considered ineligible for the Model. Additionally, Roca worked to identify participants who had been enrolled for a number of years (prior to the clear articulation of desired outcomes, benchmarks and timelines) and participants who no longer met the target population criteria. For this group of participants, completion plans were developed in order to set achievable goals and dismiss these participants over a

23 These are young people are typically referred while incarcerated who do not have release dates or are serving a long term sentence. They are eligible for services when they are released. 24 These are young people for whom we are unable to obtain correct addresses, telephone numbers or other identifying information that we are able to use to locate the young person

Highlights

In FY12, participants included 4 categories of young people:

Young men who met Roca’s newly defined primary target population criteria

Young mothers

Young women who met Roca’s previous high risk target population definition

Young men who met Roca’s previous high risk target population definition

In FY13, Roca will only be working with:

Young men who met Roca’s newly defined primary target population criteria

Young Mothers

Given Roca’s focus on demonstrating impact with the newly defined primary target population, the later part of this report is focused on studying the benchmarks and progress for young men who met Roca’s newly defined primary target population criteria.

P | 4

P | 4

Organizational Participant Enrollment


Page | 5

designated period of time. Some young people were able to be completed from the intervention as a result of attaining outcomes; however, a majority were dismissed as unsuccessful and were provided with support and referrals to other services as appropriate.

FY12 Enrollments

In FY12, Roca enrolled a total of 1120 young people. This includes both young people already enrolled at the start of the fiscal year, as well as those who enrolled during FY12.

888 young people were enrolled in Roca Chelsea

232 young people were enrolled in Roca Springfield

The chart below displays the participants’ enrollment status on June 30, 2012, the last day of the fiscal year. In FY12 Roca actively worked to dismiss young people who no longer fit the newly defined primary and secondary target populations throughout the year. Those that were active remained engaged in Roca programs while those who had been completed, dismissed, or determined as ineligible were no longer engaged in Roca.

Participant Attrition Reasons (Ineligible and Dismissed)

Participant Demographics

Highlights

Of the 1120 young people enrolled:

631 remain actively engaged

202 were dismissed

194 were assessed as ineligible

93 completed goals and/or benchmarks related to Pre‐FY12 programming and follow up benchmarks

(but no longer being continued in the model or tracked for outcomes)

Program Retention

Roca’s overall retention rate is 65% of all participants enrolled (this includes those who were assessed as ineligible for the model within the first 60 days of enrollment).

Of those who were considered eligible for the model and services at Roca, Roca’s retention rate is 78%.

P | 5

P | 5


Page | 6

Roca Roca

Gender City

Male 62% Chelsea 46%

Female 38% East Boston/Boston 17%

Age Range Revere 9%

16 Years Old and Younger 14% Springfield 21%

17 ‐ 18 Years Old 25% Other 7%

19 ‐ 21 Years Old 38% Race

22 ‐ 24 Years Old 19% African American 15%

25 Years Old and Older 1% Asian 1%

Unknown 2% Caucasian 8%

Hispanic 63%

Bi/Multi‐Racial 4%

Other 10%

As Roca is committed to demonstrating impact with its newly defined primary target

population, the following information expands on Roca’s work with young men, ages 17‐24 who are street/gang involved and/or who have been involved with the criminal justice system. This data represents Roca’s first effort at ONLY benchmarking this target population of young men to the performance benchmarks and outcomes

In order to provide a clearer picture of how the changes made to the model in FY12 have impacted delivery and to begin looking at how to benchmark annual progress of participants through the model, this year’s data breaks out participants who were enrolled prior to the start of FY12 and those enrolled during FY12 at each of the sites, Chelsea and Springfield.

As mentioned earlier, those enrolled prior to the start of FY12 had originally been on a 3year timeline for intensive services. Given the change in the timeline to a 2 year intensive at the beginning of FY12, participants were worked to the new timeline benchmarks despite having started under a different initial model.

Additionally, this year, new enrollments were particularly high. The change to the 2 year timeline and the refinement of the target population also required a process of completing young people out of Roca who had been engaged in Roca for more than 3 years and who did not meet the newly defined target population. As a result, Roca was able to accommodate new enrollments for the newly defined primary target population.

In FY12, Roca served a total of 409 young men who met these target population criteria.

Primary Target Population Performance Benchmarks


Page | 7

(Please Note: this number does not include those young men who were enrolled but determined ineligible for not meeting target population risk level within the first 60 days of enrollment)

Of the 409 young men served:

................................................................................................................................................................ 178 had been enrolled prior to the start of FY12

................................................................................................................................................................ 231 were new enrollments in FY12

Participant Attrition Reasons

As indicated, the primary reason for dismissal is “Incarceration for an Extended Period of Time”. Often the young men Roca serves have pending trials and or suspended sentences when they are initially enrolled.

Highlights

Roca Chelsea served 273 young men

......................................................29 had been enrolled prior to the start of FY12

......................................................44 were new enrollments in FY12

Roca Springfield served 136 young men

......................................................9 had been enrolled prior to the start of FY12

......................................................7 were new enrollments in FY12

Of the 409 young men:

298 remain actively engaged

111 were dismissed

Of the young men in Roca’s primary

target population who were

P | 7

P | 6

Worth Knowing

84% _Percent of New Enrollments Retained.

P | 7

P | 7


Page | 8

If a young person is convicted and/or has a suspended sentence imposed for a violation that exceeds 18‐24months, Roca removes them from the caseload as the participant is no longer accessible for the intervention. However, these young men are still considered eligible and may be re‐enrolled when they return to the community.

Participant Demographics

Roca Chelse

a

Roca Springfield

Roca

Gender

Male 100% 100% 100%

Female N/A N/A N/A

Age Range

17 – 18 Years Old 32% 18% 27%

19 – 21 Years Old 47% 46% 46%

22 – 24 Years Old 22% 37% 27%

City

Chelsea 53% N/A 36%

East Boston/Boston 22% N/A 15%

Revere 10% N/A 7%

Springfield N/A 100% 33%

Other 14% 0% 10%

Race

African American 11% 46% 22%

Asians 2% 0% 1%

Caucasians 9% 2% 7%

Hispanics 68% 40% 59%

Bi/Multi‐Racial 6% 4% 5%

Other 4% 8% 6%

Risk Factors at Enrollment

Highlights

Cities Served

Roca Chelsea Race

Roca Springfield Race

P | 8

P | 7


Page | 9

Launched in July 2010, the Springfield Replication had a very focused criteria for enrollment specifically, young men who were involved in the criminal justice system.

Roca Chelse

a

Roca Springfield

Roca

Risky/Anti‐Social Behavior

Street/Gang Involved 86% 96% 89%

Criminal Justice Systems Involvement

Prior Arrests 83% 96% 88%

Prior Convictions 59% 88% 69%

Prior Adult Incarcerations 28% 56% 37%

Prior Adult Supervisions 44% 55% 48%

Prior Juvenile Supervisions 34% 32% 33%

Education Status

High School Drop Out – Has GED 5% 10% 7%

High School Drop Out – Needs GED 69% 81% 73%

Employment History

No Employment – Past 6 Months 84% 95% 88%

Unemployed 84% 96% 88%

Young Parent

Young Father 16% 42% 25%

Given the focus of the primary target population and enrollment criteria for Chelsea, we expect to see similar demographic patterns for Chelsea in the coming year as new participants must meet a certain level of criminal justice system involvement.

Relentless Outreach

The young men Roca targets for its Intervention Model do not easily show up and are often unwilling to engage. They are young men who are still actively involved in criminal and street behaviors and as such are often evasive and try to make themselves hard to find. Relentless outreach is Roca’s strategy to find them, engage them, and to stay in it with them for the long run. In the very beginning, when a youth worker is trying to build a relationship with a young person, outreach efforts far exceed actual contacts. Youth workers go to their homes, talk with parents, friends, associates, go to known hot spots, parks, and corners all in the effort of finding the young person and demonstrating the consistency of their effort and commitment to engage with him.

Highlights

As seen in the enrollment demographics, Springfield enrollments had higher rates of involvement in the criminal justice system.

Criminal Justice Systems Involvement

P | 9

P | 8


Page | 10

Once the relationship has begun to develop, outreach is still the critical tool for youth workers to build trust, reliability and consistency. Young People need to see that the youth workers are coming to find them when they don’t show up and are going to look for them when something is going on in the street in order to make sure they are safe. Outreach is not something that happens in the beginning of the relationship, but all throughout their engagement with Roca.

The most critical time for outreach is actually further into the relationship and after the young person has been engaged in his process of change. It is in the moments of relapse, when the young person blows up, gives up, and doesn’t want anything to do with Roca anymore that outreach becomes the most primary and crucial strategy employed by youth workers. Youth Workers have to do whatever it takes to find our young people and help them realize that even in the hardest of moments, we are not going anywhere.


Roca understands relationships as the foundational component to supporting young people’s change. The development of the transformational relationship component and the emphasis on how Roca staff members engage in building trust aligns with the research on the importance of relationships related to behavior change. The young people in Roca’s target population have formed norms and values that have mostly been influenced through unhealthy and often harmful experiences in and of relationships. The young adults have had years of negative reinforcements. Roca attempts to counter the years of negative influences, to build positive norm and value influencing relationships, and to engage young people in the process of transformation through the strategic and intentional implementation of unremitting outreach and transformational relationships.

Because high‐risk young people have had little or no experience with adults who are both non‐judgmental and have high expectations of healthy behavior, they are prone to frequent disengagement and rejection of constructive relationships. Therefore, a youth worker must ceaselessly reconnect with a young person who periodically rejects him or her and refuses to engage. This pattern is crucial to the process and must happen continuously all throughout the relationship. While a young person may show up at Roca three times in one week, he or she may just as likely refuse to come back the following week. It is the youth worker’s responsibility to track down that young person and reengage. Because the relationships is the critical tool used for change, the youth worker’s need to effectively know how to use himself or herself to engage young people and build trust is essential.

Through evaluation studies conducted on Roca’s Model, the single most predictive indicator of retention and attainment of long term outcomes is young people receiving 2‐3 contacts each week. Youth Workers are expected to make 2‐3 actual contacts each week with all 25 of the young people on their caseloads. These contacts are intentional interactions with the participant by the youth worker to build and deepen relationships and to push, encourage, challenge, and support behavior change. Actual Contacts do not include contacts with any third parties, i.e. parents, girlfriends, probation officers, etc… Additionally, actual contact percentages do not include efforts made by the youth worker that did not result in a face to face or a verbal conversation with the participant.

Of the 409 participants served in the Primary Population:

Worth Knowing

P | 10

P | 10


Page | 11

PE = Pre‐FY12 Enrollments NE = New FY12 Enrollments AE = All Enrollments


241 are Phase 1 Participants



Phase 1 Contact Standards

Due to the very nature the target population recruited, new Phase 1 participants are expected to be harder to contact. Initial engagement in the Phase 1 requires a great deal of effort on the part of the youth worker which doesn’t always result in an actual contact with the young person. This year, with the high volume of brand new enrollments in the Phase 1, youth workers faced additional challenges making 2‐3 actual contacts each week with new participants. Quite often, when a young person was initially enrolled on a youth workers caseload, there were missing

or incorrect addresses and phone numbers. Roca relied heavily on its referring partners to help gather information that could be used by youth workers to track these newly enrolled young people down.

While new enrollments in FY12 were managed more efficiently to the new program timelines, there were challenges related to moving those previously enrolled to a Phase 2 in a timely manner. The goal is to move Phase 1 participants to a Phase 2 between 4‐6months. For participants previously enrolled, while some had not yet hit their 6 moth benchmark, others were not moved to a Phase 2 as a result of not managing the length enrolled and administrative actions required to move to a Phase 2. Compliance reports were created at the end of FY12 to track and identify potential management issues related to the movement of young people through the model.

Phase 2 Contact Standards

With the establishment of a relationship, the percentage of young people who youth workers are able to make actual contacts with increases. Phase 2 participants were more likely to meet the standard of being seen 2‐3 times each week.

New enrollments had higher rates of contacts than those enrolled prior to FY12


Highlights

Roca categorizes where a young person is in the model by his Phase status.

Phase 1: Engagement

This is the time when the youth worker is diligently working to establish a relationship that is strong enough to support young people through a process of change.

Phase 2: Behavior Change During this phase, youth workers are intentionally working with young people on moving through the stages of change related to risky behaviors that are most definitely barriers to long term positive outcomes including: criminal involvement, substance abuse, unhealthy relationships, and other anti‐social behaviors.

Phase 3: Sustaining

By this phase, young people have attained certain benchmarks and outcomes and are being supported through less intensive services on sustaining and advancing those outcomes.

P | 11

P | 9


Page | 12

Roca’s stage based programming approach aligns well with what the literature suggests is the most effective practice in creating opportunities for individuals to learn and practice new behaviors. When young people are attached to programming that “meets them where they are” in their readiness, willingness, and ability to engage, and that allows for them to practice and experience their own challenges and successes, the level of participation and potential success increases. By applying this evidence‐based framework in the program delivery, young people move along a pathway of education, pre‐vocational training, life skills, transitional employment and unsubsidized employment opportunities.

Programming Engagement

Getting this group of young people into programming, while challenging, is critical to supporting young people in their change process and their skill attainment. Youth workers try to drag young people into programming early on in their enrollment in the model to expose young people to the learning possibilities and to help them begin the practice of showing up.

Percentage Engaged in Stage Based Programming


Highlights

Phase 1 and 2 Contact Standards Comparison

67% of all Phase 2 Participants were seen 2‐3 times weekly

80% of all participants were engaged in stage based programming

Programming Engagement

PE = Pre‐FY12

Enrollments

NE = New FY12 Enrollments

P | 12

P | 10


Page | 13

Programming Standards

Ideally, regular attendance to programming builds as young people become more engaged in their own change and learning processes. Roca is studying the patterns of participation for Phase 1 and Phase 2 participants to understand both what it takes to get young people into programming and what the expected level of attendance should be going forward for this target population. As we can already see from the data below, new enrollments are attending at a higher rate than those who were enrolled previously to FY12. With the clarity of the model and the new timeline initiated in FY12, youth workers were more intentional about getting new enrollments into programming at a faster pace earlier in their enrollment.

Education and Pre‐Vocational Programming

Our educational programming helps young people re‐engage and advance in education, and either re‐enter school or obtain their GED. Roca offers instruction on basic literacy, pre‐GED, GED, Microsoft applications, writing, and tutoring/homework help. Roca helps young people whose first language is not English to improve proficiency in literacy and communication skills through multi‐level ESL classes.

Pre‐Vocational Training provides young people with the basic job skills and certifications needed to enter the workforce and sustain employment. Young people participate in on‐site training courses to earn other informal/formal industry certifications including: OSHA, CPR, Heartsaver AED, Lead Safe Removal, Carpet Care, Floor Care, ServSafe, forklift operation, green cleaning, housekeeping and more. We also offer apprenticeships, such an apprenticeship with the Carpenters' Union. Soft skills are developed in resume and interview training, job readiness and career exploration.


Worth Knowing

P | 13

P | 13

Highlights

More than half of all the Phase 2 participants attended programming at least 1‐2 times weekly

67% of all participants were engaged in educational and pre‐vocational programming

52% of those engaged in educational and prevocational programming made gains

P | 13

P | 11


Page | 14

Industry Recognized Certification Breakdown

Roca Chelse

a

Roca Springfield

Roca

GED 4 5 9

Basic Cleaning 0 24 24

Carpet Care 0 14 14

CPR 17 10 27

First Aid 0 21 21

Floor Care 0 13 13

Fork Lift Training 45 0 45

Green Cleaning 0 7 7

Lead Safe Removal 16 0 16

OSHA 10 ‐ Construction 16 0 16

OSHA 10 ‐ General Industry 16 0 16

OSHA 30 4 0 4

OSHA Asbestos 16 0 16

OSHA Blood Bourne Pathogen 13 0 13

OSHA Hazard Communication 0 20 20

Restroom Sanitation 0 18 18

ServSafe 2 0 2

Slip and Fall Prevention 0 14 14

Wood Floor Care 0 9 9


Highlights

121 young people attained at least one industry recognized certification

148 young men were engaged in Workforce Readiness Workshops

102 participants had a minimum of 6 Demonstrated Skills and Behaviors Assessments

P | 14

P | 12


Page | 15

Highlights

Roca served 172 unduplicated participants in Transitional Employment.

161 were served in Basic Transitional Employment

31 were served in Advance Transitional Employment

P | 15

P | 13

To improve the quality and impact of our pre‐vocational training, Roca developed four new 30‐hour pre‐vocational curriculums in construction, hospitality, retail and culinary arts. These courses will be launched in FY13 , and require full attendance. We customized our hospitality training based on Marriott's employment guidelines, and Whole Foods assisted with the design of our culinary arts curriculum. These new courses will be delivered over 10 week periods and will help young people prepare to enter transitional employment, build a resume and increase their employability

Work Force Readiness

Getting this group of young men ready for work is not as easy as just placing them in a job. In order to be successful at work, there are steps that have to happen, documents that have to be obtained, skills that have to be learned, and behaviors that have to be practiced and mastered. In an effort to more appropriately assess young people’s workforce readiness, Roca developed a Workforce Readiness Criteria that includes four domains young people have to complete to be considered workforce ready.

Domain 1 is based on a checklist of basic needs that must be completed including: Photo ID, Social Security Card, Birth Certificates, Resumes, Transportation Plans and Childcare Plans.

Domain 2 requires participants to complete 12 workforce readiness workshops on topics that include: Sexual Harassment, Workplace Communications, Mock Interviews, Understanding CORI, and Conflict Management.

Domain 3 covers demonstrated skills and behaviors that are observed and assessed by crew supervisors, youth workers, and vocational instructors. Skills and behaviors being assessed include: being on time, appropriate language, team work, following directions, positive attitude, and personal hygiene. Participants need a minimum of 6 Assessments and to consistently receive a rating of always or often to be considered for job placement.

Domain 4 tracks concrete benchmarks of achievement including: successfully completing transitional employment, meeting program attendance standards and completing a minimum of 2 industry recognized vocational certifications

Transitional Employment

Roca operates basic and advanced transitional employment slots designed to support the development and practice of workforce behaviors necessary for job placement and retention.

The flow chart below illustrates the progression of a young person through transitional employment and job placement services at Roca.


Page | 16

P | 16

P | 13

Basic Transitional Employment

Basic Transitional Employment was designed to help young people learn and practice how to work by working. This includes the development of basic soft skills necessary for successfully retaining employment, including: showing up on time, completing an entire work day, following directions, workplace communications, etc… In the process of learning how to go to work, young people may lose their transitional work slot multiple times. Each time a young person loses their job, they must undergo a rehire process that moves them toward positive behavioral change. Based on the new timeline of the model, young people can be in basic transitional employment for a maximum of 18 months from the day they first begin on a work slot.

Successful completion of basic transitional employment for participants is defined by reaching 60 days of consecutive employment without losing their work slot or missing work without cause. Once participants meet this successful completion benchmark, they will either be referred to Advanced Transitional Employment or to job placement. This referral is based on their progression not only through the work experience but also through other parts of the model.

Transitional Employment Flow Chart

Unsubsidized Job

Job Retention

ATEP Slot Work Readi

Ter

Ter

Orientation

BTEP Bench

BTEP Active Slot

Work Readi

Re‐Hire

Meet

BTEP = Basic Transitional Employment

ATEP = Advanced Transitional Employment

WR = Work Readiness


Page | 17

The following benchmark information is based on the 161 young men who were served in Basic Transitional Employment in FY12 only (these benchmarks do not include placement numbers for those young people who completed basic transitional employment prior to FY12 who were being served in advanced transitional employment or job placement only in FY12).

Advanced Transitional Employment

Advanced transitional employment was designed to support young people who have completed basic transitional employment but need more time to build overall workforce readiness skills before being placed in unsubsidized employment.

Successful completion of advanced transitional employment for participants is defined by continuous retention of their advanced transitional employment slot and the completion of all workforce readiness criteria.

The following benchmark information is based on the 31 young men who were served in Advanced Transitional Employment in FY12 only (these benchmarks do not include placement numbers for those young people who completed advanced transitional employment prior to FY12 who were being job placement only in FY12).

Dismissed from Roca Model

Removed from Youth Worker caseloads and 57%

Could Not Complete

Met maximum allowed length enrolled in 43%

4Unsuccessfull

Placed in Advanced Transitional Employment 64%

Placed in Unsubsidized Employment 24%

Pending Placement 12%

2Successfully Compl

Active Slot:

Currently filling a slot on a work crew 31%

Bench Slot

Eligible to fill a slot on a work crew 21%

Re‐Hire Slot

Working towards becoming eligible for 48%

9Actively Enrolle

161

Parti

Highlights

Basic Transitional Employment

Of those who successfully completed Basic Transitional Employment:

86% have retained employment

Advanced Transitional Employment

Of those who successfully completed Advanced Transitional Employment:

75% have retained employment

P | 14


Page | 18

Dismissed from Roca Model

Removed from Youth Worker caseloads and 25%

Could Not Complete

Met maximum allowed length enrolled in 75%

8 Unsuccessfull

31

Parti

Placed in Unsubsidized Employment 80%

Pending Placement 20%

1Successfully


Page | 19

Job Development and Placement

Given the changes in the model this year, the following placement and retention benchmarks for this report are only for the primary target population young men who were placed in unsubsidized employment in FY12.

There a total of 63 job placements made. Of those job placements, 51 were unduplicated participants.

................................................................................................................................................................ 67% were still employed at the end of the reporting period

................................................................................................................................................................ 33% are working towards replacement

Reducing Arrests and Technical Violations

As one of the long term outcomes for participants is no incarcerations, Roca tracks performance indicators for arrests and technical violations for throughout their engagement in the model. While it is expected that young people may have new arrests and technical violations while in the model, Roca hopes to see a reduction over time in both areas for individual participants as they move through the model. The impact and benefits of the new model rolled out in FY12 can also be seen in the positive progress of new enrollments who have less arrests and violations than those enrolled prior to FY12


Highlights

Overall Job Placement

67% of young people placed in unsubsidized employment in FY12 were still working at the end of the fiscal year.

Reducing New Arrests and Technical Violations

P | 15


Page | 20



Page | 21

In FY12, 39 primary population participants graduated to a Phase 3. Phase 3 is the 2 year less intensive follow up phase of Roca’s Intervention Model. Young people are graduated to a Phase 3 when they have completed the intensive 2 years of the model and they have meet the intermediate outcomes of: Unsubsidized Employment Retention (min of 3 months); Educational Gains; No New Arrests (min 6 mo); and, No Technical Violations (min of 6 mo).

Young people are tracked in a Phase 3 for 2 years. They are provided with less intensive support through contacts with a TR3 Youth Worker, are able to continue participating in educational and pre‐vocational programming, and are assisted with job development and advancement support.

By definition, those who graduated to Phase 3 came from the population already in progress at the start of FY2012, as they had to have completed 2 years of programming. All of the Phase 3 participants were served in the Roca Chelsea Site as Roca Springfield Replication just completed 2 years and will not have TR3 graduates until FY13.

Of those who graduated to a Phase 3 in FY12:

P | 21

P | 16

Phase 3 Intermediate Outcomes


FY 2013 to FY 2017 Page | 22

It has been an intensive year of transition as a result of implementing changes to the model, target population, and program operations. These changes have provided clarity and structure in driving young people toward outcomes, in coaching and holding staff accountable to performance indicators and standards, and in developing improved operations, systems, and measurements to improve our capacity to deliver a high quality, replicable model of intervention with this target population.

As evidenced in the data, overall, new enrollments in FY12 are doing better than those enrolled prior to FY12. These new enrollments have been managed more efficiently to the timeline and benchmarks of the model. Also, from the beginning of their engagement, they have been able to participate in programming that is now strategically designed to move our young people through the model, incrementally working on behavior change and building their capacity to participate in more structured opportunities for skill development directly linked to the attainment of short, intermediate, and long term outcomes.

Becoming worthy enough to serve this group of young men and to help them make real changes in their behaviors is intricately connected to our on‐going engagement in a process of continuous quality improvement. By working systemically and strategically to improve our programming and organizational structures we have been able to increase our ability to have a meaningful impact and to be of value to those we serve. Completing participants who do not meet the target population, improving the ways in which we measure ourselves, making adjustments to track performance, using data to manage performance, and focusing on staff training and development has allowed Roca to get to more clear on the outcomes we care about with the group that is most in need of this level of intervention.

With all of the changes implemented in FY12, Roca is now poised for the next level of work to fully align itself with Eight Evidenced Based Principles of Effective Intervention and to move toward demonstrating the impact of its model through more rigorous evaluation efforts. To do this, Roca will continue to look at critical look at current research and will specifically be looking at: a deeper understanding and potential implementation of risk/needs assessment; specific Cognitive Behavioral Programs that may enhance the model, and; research and consideration of sanctions as informed by the current thinking in evidenced based practices. Roca will also continue to use its data to ensure fidelity and to improve programmatic and participant benchmark targets.

Roca’s 3 main Implementation Goals include:

Increasing contact standards to 80%;

Increasing programming standards to 80%; and,

Improving overall job placement and retention rates.

FY13 Targets and Benchmarks:

Roca Chelsea will serve 470 (350 TR1 & TR2; 120 TR3) high risk young men Actual Starting point as of 7/1/12‐ Roca Chelsea will serve 420 (300 TR1 & TR2; 120 TR3)

Roca Springfield will serve 180 (150 TR1 & TR2; 30 TR3) high risk young men

Actual Starting point as of 7/1/12‐ Roca Springfield will serve 130 (100 TR1 & TR2; 30 TR3)

80% will be seen a minimum of 2‐3x a week by their youth worker 80% will meet programming standard of 2x per week in any programming 80% will be actively engaged in alternative education and workforce readiness programming

Lessons Learned and Next Steps


FY 2013 to FY 2017 Page | 23

40% will be engaged in Transitional Employment Programming

Of those engaged: 80% will complete workforce readiness criteria (within 18 months from enrollment) 80% will attain a minimum of 2 industry recognized vocational certifications (within18 months from enrollment) 80% will demonstrate academic gains (within 24 months from enrollment) 80% will be placed in unsubsidized employment (within 24 months from enrollment)

80% of those placed will retain employment for a minimum of 3 months


FY 2013 to FY 2017 Page | 24

Appendix G: FY 12 Springfield Outcomes Highlights


FY 2013 to FY 2017 Page | 25

Roca Springfield FY12 Performance Benchmark Report Participants Served and Retention In FY12, Roca Springfield served 136 participants: 49 or 36% had been enrolled prior to the start of FY12 87 or 64% were new enrollments in FY12 Participant Enrollment Status: 99 or 73% remained active at the end of the fiscal year 37 or 27% were dismissed for the following reasons

15 or 42% were dismissed as “Incarcerated for extended time period” 21 or 28% were dismissed as “Moved out of geographic service area” 5 or 7% were dismissed as “Unable to contact/locate” 11 or 15% were dismissed as “Unable to move to Phase 2” 10 or 14% were dismissed as “Unable to move to Phase 3”

Roca Springfield was able to retain 73% of the participants served throughout FY12. Participant Demographics

Participant Risk Factors

Total % Total %

Risky/Anti‐Social Behavior Education Status

Street/Gang Involved 130 96% High School Drop Out ‐ Has GED 13 10%

Criminal Justice Systems Involvement High School Drop Out ‐ Needs GED 110 81%

Prior Arrests 131 96% Employment History

Prior Convictions 120 88% No Employment ‐ Past 6 Months 129 95%

Prior Adult Incarcerations 76 56% Unemployed 131 96%

Prior Adult Supervisions 75 55% Young Parent

Prior Juvenile Supervisions 44 32% Young Father 57 42%

62 or 46% are African American

3 or 2% are Caucasian 55 or 40% are Hispanic 5 or 4% are Bi/Multi‐Racial

11 or 8% are of other races

136 or 100% reside in Springfield

24 or 18% are 17‐18 Yrs Old 62 or 46% are 19‐21 Yrs Old 50 or 37% are 22‐24 Yrs Old

0 or 0% are Female

136 or 100% are Male


FY 2013 to FY 2017 Page | 26

Transformational Relationships Of the 136 participants served in Roca Springfield 73 or 84% are Phase 1 participants 14 or 16% are Phase 2 participants Contact Standards 56% were seen 2‐3 times weekly

49% of Phase 1 participants were seen 2‐3 times weekly 76% of Phase 2 participants were seen 2‐3 times weekly

Stage Based Programming Programming Engagement 102 or 75% were engaged in Stage Based Programming

91 or 67% were engaged in Life Skills/Engagement programming 88 or 65% were engaged in Alternative Education and/or Prevocational programming 85 or 63% were engaged in Employment programming

Percentage Engaged in Stage Based Programming




FY 2013 to FY 2017 Page | 27


Programming Standards 35% attended programming at least once a week

27% of Phase 1 Participants attended programming at least once a week 51% of Phase 2 Participants attended programming at least once a week

46% attended programming at least twice a month 36% of Phase 1 Participants attended programming at least twice a month 65% of Phase 2 Participants attended programming at least twice a month

Alternative Education and Prevocational Programming 88 or 65% were engaged in Alternative Education and/or Prevocational Programming

45 or 51% attained at least 1 industry recognized certificate 29 or 33% attained at least 2 industry recognized certificates

Industry Recognized Certification Breakdown

Total

GED 5

Basic Cleaning 24

Carpet Care 14

CPR 10

First Aid 21

Floor Care 13

Green Cleaning 7

OSHA Hazard Communication 20

Restroom Sanitation 18

Slip and Fall Prevention 14

Wood Floor Care 9



FY 2013 to FY 2017 Page | 28

Workforce Readiness

Domain 1: Basic Needs Checklist

52 or 38% have started a basic needs checklist 1 or 2% completed 1‐3 basic needs 14 or 27% completed 4‐7 basic needs 37 or 71% completed 8‐11 basic needs 6 or 12% completed all basic needs

Domain 2: Workforce Readiness Workshops

59 or 43% were engaged in workforce readiness workshops 17 or 29% completed 1‐3 workforce readiness workshops 13 or 22% completed 4‐7 workforce readiness workshops 29 or 49% completed 8‐11 workforce readiness workshops 11 or 19% completed all workforce readiness workshops

Domain 3: Demonstrated Workforce Behaviors and Skills

35 or 26% have completed a minimum of 6 workforce demonstrated skills and behaviors assessment 4 or 11% demonstrates 1‐3 workforce behaviors/skills 8 or 23% demonstrates 4‐7 workforce behaviors/skills 15 or 43% demonstrates 8 ‐13 workforce behaviors/skills 6 or 17% demonstrates all workforce behaviors/skills

Transitional Employment Programming Basic Transitional Employment (BTEP) 66 participants were engaged in Basic Transitional Employment

42 or 64% are actively enrolled in BTEP 12 or 29% are on an Active slot 10 or 24% are on a Bench slot 20 or 48% are in the Rehire process

10 or 15% have successfully completed BTEP 7 or 70% were placed in ATEP 1 or 10% were placed in unsubsidized employment 2 or 10% are pending placement in unsubsidized employment

14 or 21% could not complete BTEP 9 or 64% were dismissed from Roca’s model 5 or 36% did not retain 60 consecutive working days within the allotted period of 18 months in

transitional employment Advanced Transitional Employment (ATEP) 9 participants were engaged in Advanced Transitional Employment


FY 2013 to FY 2017 Page | 29

4 or 44% are actively enrolled in ATEP 3 or 33% have successfully completed ATEP

3 or 100% were placed in unsubsidized employment Job Development and Placement There were a total of 10 job placements made. Of those job placements, 9 were unduplicated participants. 7 or 78% were still working at the end of the reporting period 2 or 22% are working towards replacement

Reducing Arrests and Technical Violations 103 or 76% had No New Arrests 43 or 86% had No New Technical Violations




FY 2013 to FY 2017 Page | 30

Appendix H: Impact Evaluation Plan

A Strategic Plan to Integrate Continuous Quality Improvement with an Impact Evaluation Of Roca’s Intervention Model for Very High Risk Young Men Prepared for Roca, Inc. The Center for State Child Welfare Data Chapin Hall, University of Chicago

Introduction 2 Context 2

Continuous Quality Improvement/Performance‐based Management 2

Replication 3

Massachusetts’ Pay for Success Initiative 3 Evaluation Objectives 3 CQI Framework 4 Basic Program Data 4

Enrollment Data 5

Length of Service 10

Service Intensity and Outcomes 12 Impact Evaluation 14

Outcomes 15

Data Sources 16

Sample Selection 16

Statistical Methodology 16 Implementing the Evaluation 20

Design, Data Assessment, And Instrumentation 20

Fieldwork And Formative Evaluation 20

Summative Evaluation 21 Pay‐for‐Success – Evaluation Challenges 21 Safeguards For Target Population 22

2

Introduction

About one year ago, the leadership at Roca approached senior staff at the Center about the possibility of helping Roca get the most out of its commitment to performance-based management/continuous quality improvement processes.25 The specific request was for help with the in-house knowledge base. Roughly four years ago, Roca invested in a case management system that is now producing useful data about the life experiences of the served youth and the role services play in their lives. Roca wanted help turning that data into a more actionable form. In the course of events, Roca also asked for a strategic evaluation plan that would help it become a high performance, knowledge-driven organization.26

Roca’s main objective is to move from a program with a promising evidence base to a program recognized for its demonstrated impact. This strategic plan lays out an evaluation strategy that will help the organization meet that objective. The plan starts with a brief characterization of Roca’s strategic context. Specifically, three drivers are shaping Roca’s orientation to the future. The first is operational in that Roca has taken a number of steps to improve its performance-based management/continuous quality improvement process (PBM/CQI). The other two drivers are more situational. Committed to serving a high need, high-risk population, Roca hopes to replicate its program in other high need areas. In so doing, Roca sees the process of replication as an opportunity to refine and test its approach to the work it does. Finally, Roca is on the cusp of signing a Pay-for-Success (PFS) contract with the Commonwealth of Massachusetts. Because the PFS contract carries evaluation requirements, PFS represents a unique opportunity to study the Roca program and its impact on the lives of youth at risk of returning to prison.

The plan then turns to the evaluation strategy itself. After a brief description of the objectives, the discussion lays out the PBM/CQI process. In the larger organizational context, Roca needs a continuous approach to building an ever-stronger organization. Formal evaluations are an important element of Roca’s broader strategy but only if the benefits of doing formal evaluations can be integrated into an operationally viable approach to on-going learning. To place Roca’s CQI capacity in context, the discussion includes a presentation of the data already in use by Roca’s staff.

The CQI discussion is followed by a description of the impact evaluation, which includes a description of the outcomes, data sources, sample procedure, and statistical models for assessing Roca’s impact. The evaluation section closes with a brief description of the Pay-for-Success initiatives, its challenges, and a description of how client rights will be protected.

Context

Roca’s leadership is fully committed to a rigorous evaluation of its program including, where appropriate and feasible, random assignment of youth into treatment and control groups. Contingencies affecting Roca’s evaluation strategy are:

Continuous Quality Improvement/Performance-based Management

Roca understands the need for rigorous evaluation. At the same, over the long run, successful programs have to stand on their own, released from the operational constraints of random assignment, but still able to learn in on-going dynamic way. To this end, Roca is building a CQI model that integrates a carefully constructed logic

25 We will use the terms performance-based management and continuous quality improvement interchangeably. Both terms are meant to convey Roca’s interest in learning through doing. 26 As part of its current, on-going management, staff at Roca have undertaken steps to further focus the agency’s program. That work is reflected in other parts of the application. The strategic plan is more focused on capacity and intent. The data presented in later sections are illustrative of efforts by Roca to be more strategic around the deliberate use of data as part of its management practice. That capacity will be brought to bear on the evaluation of the intervention model for very high-risk men.

3

model and routine collection of program data with vigorous program management through feedback mechanisms. The CQI approach relies on the client data collected by Roca that has been retrofit into a highly structured longitudinal data file that links program inputs with client outcomes. The result is a highly unique ‘dose-response’ dataset that links the timing of program interventions with the service trajectory of the clients. The dataset is compatible with a wide range of statistical models for testing the impact of Roca services. By combining the precision of rigorous statistical models with on-going CQI feedback, the CQI model being developed by Roca meets the dual test of rigor and practical utility.

Replication

If and when the opportunity comes along, Roca intends to expand its services within Massachusetts and to other localities. Each such replication represents an opportunity to test the program’s integrity across a range of social, economic, and political contexts. However, until sites have been selected, it is difficult to describe an evaluation plan in detail, except to say that replication sites will be judged in part on their ability to support a basic impact evaluation. Among the criteria, access to administrative data for purposes of drawing comparisons between Roca and a business-as-usual counterfactual is essential.

Massachusetts’ Pay for Success Initiative

Over the course of the next several years, Roca’s interest in testing its impact will, in all probability, be tied to the Pay-for-Success (PFS) initiative coming out of the Massachusetts Executive Office of Administration and Finance. PFS matches innovative interventions with fiscal incentives to promote positive outcomes within the targeted population. Because of the way financing and outcomes are tied together, random assignment is the preferred approach to the PFS evaluation. As such, PFS provides an ideal opportunity to integrate Roca’s CQI model with a random assignment impact evaluation.

Evaluation Objectives

Program evaluation generally involves two main tasks: implementation evaluation and the impact evaluation. The first task addresses the question: did the program as implemented adhere to the design of the program developers? The assessment of implementation generally focuses on three broad concerns: the process of care, the quality of care, and service capacity. Process and quality standards are derived from applicable regulatory standards and any manualized statement regarding how a practice model (e.g., an evidence-based intervention) is to be implemented. Capacity refers to whether staffing, training, and physical plant requirements are part of the practice model. For example, based on our knowledge of Roca’s work with young men, the Roca model prescribes an expected number of client contacts given the treatment stage. The evaluation of implementation would gather the data needed to determine whether this process of care requirement is being met.

In the innovative financing context, implementation assessment is important because the cost of delivering the bundle of community-based services has to be compared to the savings associated with the averted utilization of high-end services. The implementation analysis will articulate what the intervention, as delivered, looks like so that program leaders and state officials understand what replication entails.

The second task has to do with the impact evaluation. Impact addresses the straightforward question: did the intervention work? Is educational achievement in the treatment group greater than in the control group? Are re-incarceration rates lower in the treatment group? Because effects are likely to differ between sub-groups, the impact evaluation has to determine whether some study participants benefitted more than others. It is also desirable to ascertain the extent to which model fidelity affected outcomes. This is a dose/response question: was success more common among the participants that received a full dose of the services?

4

CQI Framework

A commitment to Continuous Quality Improvement (CQI) is at the heart of Roca’s evaluation strategy. The CQI process is a deliberate, feedback-driven process that links the organization’s clinical model with process of care requirements, quality of care requirements, and capacity requirements with mission critical person-level outcomes. Organizational learning happens as the alignment of intention, execution, and impact comes into view and stakeholders ask whether their expectations are being met. Feedback is the process that matches intention with what is known about implementation and impact, reveals whether an implementation/impact gap exists, and informs whether anything should be done to adjust basic strategies going forward.

A simplified map of the CQI process is provided in Figure 1. Briefly, the process begins at the top of the cycle when stakeholders describe their target outcomes together with a rationale for how and why their intervention will change outcomes for transition age youth at risk of incarceration (phase 1). Much of this work has already been done; the CQI process will refine the intervention going forward.

Building the connection between outcomes, the theory of change, and the intervention is detailed, interactive work that involves various stakeholders. Through that interaction, the program staff determines the rationale for why the intervention and impact are connected. It is a process that ends with a schematic that describes the parts of the intervention - the process of care, the expected quality standards, and the organizational/workforce capacity needed to implement the plan - and their connection to the outcomes (see Figure 1).

Figure 1: The CQI Learning Process

Define problem/outcome Develop theory of change

Design intervention

Implement intevention

Measure outcomesMonitor implementation

Provide feedback

Adjust intervention as needed

Process of CareInvestments

Quality of CareInvestments

Investments inCapacity

1

2

3

4

Program staff then set out to implement their intervention (phase 2). Simultaneously, the monitoring process begins. By virtue of the work done during the design phase, the evaluator’s work of designing a system for measuring implementation fidelity (i.e., the process evaluation) and outcomes (i.e., the impact evaluation) will have been completed for the most part. Because phase 3 involves feedback, the design phase will have to identify both proximal and distal outcomes, with the proximal outcomes serving as a road map that points to whether the intervention will produce results in the longer run. To support this phase, the evaluation plan has to include near-real time data collection and a mechanism for meeting with/reporting back to Roca on an interim basis so that adjustments to the intervention can be contemplated as needed. The adjustment phase invokes the original design and theory of change by asking whether the original assumptions are producing the intended results (phase 4). Learning takes place when the reality of what is happening is assessed against the expected outcome and stakeholders interpret the extent to which the two are aligned.

5

Basic Program Data

The CQI process is contingent on the availability of data. In this section, we examine data from Roca’s ETO database. Presenting the data serves the following purposes. First, the data illustrate how well developed the data reporting capacity already is.27 Regardless of the approach, the best CQI processes and evaluation studies employ data that connect the type and level of service received with the outcomes gained on a person-by-person basis. It is in this way that refinements to the intervention are possible. Because Roca’s intervention is well specified, the data collected show the ‘effort given’ on a client-by-client basis, a feature that reduces the need for expensive field data collection. Second, data on hand can be used to place Roca’s service population in context. Bearing in mind that the Roca’s target population is now focused more so on young men, we examine basic demographic data and we present data that address three important clinical processes: time enrolled in the program, number of programming events, and termination reason. Enrollment measures how long youth are engaged; programming indicates the service level, and termination reasons are indicative of how services ended. With regard to programming, Roca’s service model is prescriptive with respect to weekly service levels; outcomes focus on keeping youth out of jail, successful passage from one program stage to another; and positive exits. Again, these data illustrate Roca’s capacity to engage in a meaningful CQI process and impact evaluation.

Enrollment Data

The enrollment data by year are presented in Tables 1 through 4. The ETO data covers enrolled youth from 2009 forward, although data for the most recent years are considered more complete. In Table 1, the data show how many youth are counted as having reached the Transformational Relationship stage, which is regarded as central to the Roca service model. These data are categorized by spell number, which refers to whether the clients are being served for the first time. As the data suggest, most youth are being served for the first time, although with the passage of time, the fraction of youth in a second or third service spell has increased. This is an artifact of time; as the Roca program matures, more of its former clients have had the opportunity to return.

27 It may be helpful to provide some additional details on the data set as it exists now. The file to be used as part of the CQI process (and the impact evaluation) has a discrete-time or person period structure. Each client’s service record is divided into weekly episodes; during each weekly episode, service contacts (frequency and type) are recorded. The weekly structure parallels the timescale used to define Roca’s intervention. Thus we have the level of precision needed to track transitions through the various stages and judge the extent to which contact with youth influence how the trajectory unfolds. Because workers serve more than one youth, we have incorporated the nested structure into the design of the database. This means that to the extent that outcomes vary as a function of the worker, worker effects can be extracted from the data. Also, to the extent the geographic clustering affects outcomes (e.g., youth from some neighborhoods within the larger service region have higher than average re-incarceration rates because of the social context in which they live), the spatial clustering can also be taken into account. Importantly, because the file contains a range of control variables – age, gender, race/ethnicity, prior criminal history, etc. – the analysis of outcomes can be adjusted to account for client level differences. Finally, as the file is fully longitudinal, Roca’s past performance can be compared with its present performance to determine whether innovative changes to the program improves Roca’s performance relative to its own past performance. Although self-to-self comparisons are not a critical text of Roca’s impact relative to business-as-usual, the self-to-self comparisons can be used to substantiate the benefits of in-house service innovations, which is the objective of the CQI process.

6

Table 1. Number of Clients in Roca Transformational Relationships at the Start of Each Year

Spell Number Spell Number

Year Spell

1 Spell

2 Spell

3 Total Spell

1 Spell

2 Spell

3 Total

2009 459 16 0 475 97% 3% 0% 100%

2010 461 26 2 489 94% 5% 0% 100%

2011 399 27 2 428 93% 6% 0% 100%

2012 455 65 2 522 87% 12% 0% 100%

Year-by-year enrollment data by age at enrollment are presented in Table 2. Insofar as the data maintained by Roca cover youth enrolled prior to 2009, the data are broken down into a pre-2009 category and single years from 2009 to 2011, the last year for which a full year of data are available.28 In general, enrollment grew by about 14 percent, from 324 in 2009 to 368 in 2011. During this time span, the population enrolled shifted to slightly older group. The data in Figure 2 suggest that 19 and 20 year olds, in particular, made up a larger fraction of the total than in prior years. Of course, changes in the caseload of this type, given that age and the propensity to offend are correlated, have to be taken into account when judging whether Roca’s interventions are successful.

Table 2. Client Population in Service at Beginning of 2009 and Newly Enrolled in 2009, 2010, and 2011

Year of Enrollment Year of Enrollment

Program Start Age

P

2 2 2T

P

2 2 2T

15 or Less

9 2 3 31 2

91

81

16 6 3 4 3 1 1 1 1 1 1

28 The database used to prepare the tables and figures was created in the spring of 2012. It has been subsequently refreshed with data through 8/31/2012.

7

17 7 4 3 3

1 1 1 1 1 1

18 6 6 5 5

2 1 2 1 1 1

19 5 5 3 5

1 1 1 1 1 1

20 4 3 2 4

18

19

1 1

21 3 2 2 3

17 7 9

18

22 2 1 2 2 9 5 5 9 7 6

23 & above

3 2 3 51

6 71 1

9

All ages

4 3 3 31 1 1 1 1 1

8

Figure 2. Age at Enrollment by Year.

The expansion into Springfield, Massachusetts in 2009 was an important change for Roca. As of 2010, about 27 percent of the youth served were from Springfield. Enrollment growth in Springfield and Chelsea between 2010 and 2011 was comparable, with an average increase across both sites of 15.7 percent overall, relative to prior years.

Table 3. Location of Service for Clients in Service at Beginning of 2009 and Newly Enrolled in 2009, 2010, 2011.


Program Location

Pre‐

2009

2009

2010

2011

Total

Pre‐

2009

2009

2010

2011

Total

Chelsea 475 324 233 269 130

1100

%100

% 73% 73% 88%

Springfield 85 99 184 0% 0% 27% 27% 12%

All locations 475 324 318 368 148

5100

%100

%100

%100

%100

%

9

Roca serves a diverse population. That said, most youth are Hispanic and African American. Overall, the proportion of the enrolled population that is Hispanic has been shrinking whereas the proportion of African Americans in on the rise. This reflects the addition of Springfield as a service hub.

10

Table 4. Race/Ethnicity of Clients in Service at Beginning of 2009 and Newly Enrolled in 2009, 2010, 2011


Client Ethnicity

Pre‐ 2009

2009

2010

2011

Total

Pre‐2009

2009

2010

2011

Total

White 44 27 21 38 130 9% 8% 7% 10% 9%

African American 30 28 58 69 185 6% 9% 18% 19% 12%

Hispanic 347 213 196 199 955 73% 66% 62% 54% 64%

Bi/Multi Racial 28 17 17 8 70 6% 5% 5% 2% 5%

Other/Missing 26 39 26 54 145 5% 12% 8% 15% 10%

All clients 475 324 318 368 148

5100

%100

%100

%100

%100

%

Length of Service

Roca has a very specific service model. The prescriptive nature of the model is guided by theory as it relates to the challenges young people face when making the transition to adulthood, the barriers that young people have to overcome, and a sense for how the therapeutic process unfolds over the life course. Roca’s intervention brings these issues to bear on the service design in a very explicit way relative to outcomes and service intensities. Specifically, the outcomes have to do with jobs, education, and avoidance of prison; the process of care involves carefully planned requirements for contacts with youth as they move through the helping process.

Time spent enrolled in the Roca program is presented in Table 5.29 Displayed is the median duration of the enrollment period. The median is an indicator of how much time elapsed between enrollment and the end of services. The data suggest a modest increase in time spent enrolled. The data also suggest that time spent enrolled is higher for females, African Americans, and youth between the ages of 19 and 20. In each of these categories, time enrolled is rising over time.

29 Among the recent service model changes, Roca reduced the expected service duration from 4 to 3 years, largely because of their concern over whether the incremental benefit achieved in the last year justified the expectation of a fourth year. This is an example of the kind of reflective thinking Roca wants to undertake as part of the strategic evaluation plan.

11

Table 5. Median Lengths of Enrollment (Months) in Roca Programs for Entry Cohorts in Different Year

Demographic

Year of Enrollment

20

2010

20

All Youth 10 11 11

Gender

Female 9.

14 ‐

Male 11 9 8

Ethnicity

White 7 5 ‐

African American 8.

10.5 6

Hispanic 11 12 ‐

Bi/Multi Racial 11 ‐

Age at Program Start

16 17 15 ‐

17 7 11 ‐

18 11 18 ‐

19 12 16.5 ‐

20 9 12 9

21 5 12 7

22 10 5 3

- Median not yet observed.

An unadjusted survival plot of the underlying data shows the same basic consistency (Figure 2). The slope of the survival curve is fairly constant, which suggests a fairly constant rate of service termination. Termination rates do slow after an extended period, but one would expect that among youth who remain enrolled, there comes a time when the likelihood of ending services drops but for successful completion of the intervention.

Figure 2. Time Enrolled in Roca Services

12

Service Intensity and Outcomes

As Roca engages in its PBM/CQI process, the issue of how long youth are served relative to programmatic intent will become increasingly important. Again, the issue is one of dose and response. Roca’s intervention model presumes movement through various phases, which it aligns with service intensity and clinical progress. It is a model that places expectations on how long youth remain engaged on the one hand and why youth leave, on the other.

The salience of service duration, intensity, and outcomes is revealed in Table 6. Of the 324 youth enrolled in 2009, the enrollment year with the longest follow-up time, 52 are still receiving services and another 38 graduated. A small handful (7) was incarcerated according to the ETO database.30 Data from the ETO source files also suggest that 37 youth had not yet moved far enough along to earn transition the next phase in the sequence, which is an indication that the client is not making the hoped for progress. Of course, whether the program is meeting the needs of those youth is an equally important question to answer.

Table 6. Program End Reason for Youth in Service at Beginning of 2009 and Newly Enrolled in 2009, 2010, 2011

Program End Reason 2009 2010 2011 Total

Graduated 38 9 1 48

Ineligible 164 133 90 387

Incarcerated 7 7 3 17

Not in target population 20 25 26 71

Unable to move to Phase 2 26 21 8 55

Unable to move to Phase 3 11 8 0 19

30 One reason why the Roca data needs to be linked to state-maintained sources has to do with the post-Roca information on incarceration, employment, and education. It is possible that youth will be incarcerated but that their incarceration is not recorded in the Roca database.

13

Transfer to another worker 6 0 0 6

Still in service 52 115 240 407

All ages 324 318 368 1010

Whether youth move to the next phase of the service model is affected by the level of service delivered (or received). The Roca model spells out a specific number of programming contacts each week. Using enrollment and termination data, we determined the number of weekly service intervals provided by Roca. For example, in 2009, the enrolled youth had so far been involved in 18,032 person weeks during which time programming events would have been initiated. From that year, about 80 percent of the person-week intervals had no programming events. The data also show that by 2011, there were fewer person-weeks without a programming event.

14

Table 7. Number of Week Intervals with Different Number of Weekly Programming Events for Each Year of Enrollment

Year

Enrolled

Total Clients

Enrolled

Number of Weekly Programming

Events Total # of

Week Interva

l

# With 2 +

Programming 0 1 2 3

4

2009 324

14,648

1,

74 344

18,032 1,546

2010 318

9,895

2,

94 538

14,221 2,283

2011 351

3,873

1,

48 264

6,107 1,221

Total 993

28,4

6

4, 2, 1,1

38,360 5,050

2009 324 81

%10

4% 2% 3

100% 9%

2010 318 70

%14

7% 4% 6

100% 16%

2011 351 63

%17

8% 4% 8

100% 20%

Total 993 74

%13

6% 3% 5

100% 13%

Note: Only clients in their first continuous spell of service are included. Clients who return to service are not included. Only week intervals with a full 7 days are included; which means the last week interval of any client with fewer than 7 days are excluded. There were 368 clients enrolled in 2011; the 17 clients with spell duration of less than one week are excluded.

Impact Evaluation

The CQI process brings structure to the ways in which an organization improves its own performance. That said, without a suitable counterfactual, the CQI process cannot, except in rare circumstances, answer the

15

impact question: Do the services provided deliver better outcomes for clients than business as usual or some other service alternative.

To overcome this problem, Roca is engaged in a two-stage negotiation that will yield the data needed conduct an impact evaluation. The first stage involves gaining access to statewide administrative data; the second stage involves the Massachusetts’ Pay-for-Success initiative. Because both efforts rely heavily on access to administrative data, the strategies are interrelated.

Outcomes

Impact evaluation starts with program outcomes. In Table 1, we list the identified outcomes and possible measures. In each case, we are generally interested in two dimensions: time to and likelihood of a given outcome. For example, the interventions could influence whether something happens (likelihood) or how long it takes to happen (time to). The measurement approach used has to detect these effects separately.

For the first three outcomes – arraignments, convictions, and incarcerations – two types of measures are proposed. The first – the percent of clients with or without the event during six-month intervals – will capture differences in the probability of the event in that interval. In this way, clients with multiple events during the program (say, an arraignment in the first six months and a second two years later) will be incorporated into the measure. The second – the percent of clients with no event during three years – distinguishes those participants who never have a negative event from those who have one or more negative events.

Length of incarceration, employment, and educational enrollment are outcomes where the duration, rather than the likelihood, would be measured. Fewer days of incarceration are better than more days incarcerated. More weeks employed are better than fewer weeks employed. In these cases, the median number of days is the proposed measure. Survival analysis generally, and discrete time models specifically, are especially useful tools for these measures.

Table 1: Possible Measures for Roca Evaluation

Outcome Measure

(from point of referral/assignment unless otherwise noted)

Arraignments Percent of clients with arraignment at six months intervals

Percent of clients with no arraignments within 3 years

Convictions Percent of clients with conviction at six months intervals

Percent of clients with no conviction within 3 years

Incarcerations Percent of clients with incarcerations at six months intervals

Percent of clients with no incarcerations within 3 years

Length of Incarcerations Median duration of incarcerations

Earnings Earnings during program

Earnings in six month intervals following program

Employment status Percent of clients not working at program start that enters the labor force within three years.

16

Median duration of employment

Educational attainment Percent of clients without GED at start earning GED within three years

Percent of clients with GED at start completing a community college or college degree

Educational enrollment Percent of clients not enrolled in school at the start who enroll in school within three years

Median duration of continuous enrollment in education program

Data Sources

Incarceration will be assessed using administrative data from the Massachusetts corrections system. Key outcomes will be new arraignments, convictions resulting from these new arraignments, and spells of incarceration. The evaluation will extend to allow assessment of outcomes at least 3 years after referral to Roca, and we anticipate that longer-term assessments will be possible as well. Although avoided days of incarceration will be the focus of the payment incentives in the pay for success contract -- since that is where most of the budgetary savings to the state accrue -- the evaluation will also allow an examination of the quantity of types of new offenses committed in both the treatment and control groups, critical components to a full evaluation of the social benefits from reduced criminal activity.

To assess successful transitions to adulthood, we will be performing impact evaluations on employment rates, earnings levels, and educational enrollment and achievement. The employment and earnings data will be obtained from the Massachusetts state UI records. Educational achievement and enrollment will come from three sources: the state's unified database for secondary education, which will allow analysis of high school completion; the state's GED database, which will allow analysis of GED attainment rates, and the federal post-secondary clearinghouse, which will allow us to measure enrollment in post-secondary educational programs such as community college and vocational institutions.

Sample Selection

The sample selection scheme for the impact evaluation is relatively straightforward. Eligible youth will randomly selected from rosters provided to an intermediary. Because Roca plans to serve a subset of the at-risk population, the roster will segregate the target population first. Random selection will occur after the target group has been stratified. In advance, appropriate power tests will be conducted so that an ample sample size will be targeted.

Statistical Methodology

Although not the only opportunity to conduct a randomized field study, Massachusetts’ Pay-for-Success (PFS) initiative offers an important opportunity because of the way finance and outcomes are integrated. With that in mind, the approach to the impact study accommodates two possibilities – one linked to a randomized study; the other better suited, perhaps, to Roca’s plan to expand services.

Eligible youth being released into communities served by Roca will be randomized into a group that is referred to Roca and a group that is not referred to Roca. The comparison of outcomes between those referred to Roca and those not referred to Roca will provide one assessment of the impact of Roca's services.

17

Roca plans to expand its services into new communities and to provide more service in communities in which it is already operating. By comparing changes in outcomes over time for youth in the target population in these expansion areas to changes in outcomes in other locations within the state where Roca is not operating, we offer a second approach to assessing the impact of Roca's services.

Because the impact assessment in the pay-for-success initiative will be used not only to judge the effectiveness of the program but also as the basis for performance-related payments, it is critical that the evaluation methodology be both rigorous and robust.

Rigor. Patterns of recidivism among young adults are affected by a diverse range of external factors—from changes in local policing and corrections policies to fluctuations in the state economy. A key challenge in this program evaluation is separating the effects due to such external factors from the effect of Roca’s intervention.

To address this, the centerpiece of the evaluation will be a Randomized Control Trial (RCT). Generally considered the “gold standard” for rigorous program evaluation, the key strength of an RCT approach is that, on average, there are no systematic differences between individuals in the treated and control groups. With this setup, any difference between the two groups is due to the effect of the intervention and not to external factors.

Robust. In practice, even the best-planned RCTs face a host of implementation issues, from failed randomizations to individuals not complying with assigned treatments. Since the PFS payments depend directly on the estimated treatment effect, the final estimate must be robust to such complications in the RCT.

In particular, some members of the control group may end up being referred to Roca through other channels. The state and Roca have agreed that there will be no denial of service for anyone who is referred to Roca, and Roca will not be aware of the identities of people in the control group.

In order to prevent Roca from being penalized for serving individuals in the control group, the RCT impact estimate will be scaled up by the difference between the share of treatment group individuals served by Roca and the share of control group members served by Roca. However, this approach will only succeed if a significantly higher fraction of treatment group individuals than control group individuals end up being served by Roca.

To ensure that the evaluation approach is robust to the possibility that similar fractions of individuals in the treatment and control group end up being served by Roca, we propose to create a backstop to the RCT—a Difference-in-Differences (DD) analysis that provides a secondary estimate of the treatment effect. This analysis consists of two basic steps. First, we compare the outcomes in the trial period to historical data from the same location. Second, we correct this estimate for time-specific effects using data from locations without program access.

Once we obtain both of the RCT and DD estimates, we take a weighted average of the two to yield one overall treatment effect estimate. In general, we believe that the results from an RCT are more reliable than comparable results from a DD analysis. If the RCT faces few complications, we therefore want to put nearly all of the weight on the RCT and very little on the DD analysis. However, if, for example, the noncompliance rate in the RCT is high, then we want to put more weight on the DD analysis. In the end, this weighted estimate combines the rigor of the RCT with the robustness of using two separate approaches.

To demonstrate this approach, we first consider a much simplified version in which we observe individuals in two cities, A and B, at time T = 0 and T = 1. In city A, no one receives the treatment in time T = 0. In time T = 1, half of the individuals in city A are randomly assigned to receive the treatment, which is an encouragement to participate in Roca’s program, with the remaining half in the control group. Importantly, individuals in city A do not necessarily comply with their randomly assigned treatment status—some individuals encouraged to

18

participate in Roca do not do so (known as never-takers in the econometric literature); while some individuals that did not receive explicit encouragement to participate in Roca do so anyway (known as always-takers in the econometric literature).

In city B, no one receives the treatment in either time T = 0 or T = 1. In other words, no one in city B can participate in Roca’s program.

RCT Estimate. The simplest estimate of the effect of the intervention is the Intent-to-Treat (ITT) estimate, which is the difference between the average outcomes for those individuals in city A randomly assigned to the

treatment group, , and those randomly assigned to the control group, .

However, since some individuals do not comply with the assigned treatment, we have to adjust the ITT

estimate to determine the impact of Roca participation. We define to be the estimated proportion of

individuals in the treatment group that participate in the program, and to be the same for the control group. We can then obtain the classic Instrumental Variable (IV) estimate, which provides a rigorous estimate of the treatment effect for those who are induced to participate in the program by the intervention:

Standard errors for this estimate can be obtained using either the Delta Method or a Two-Stage Least Squares procedure, discussed in more detail below.

DD Estimate. We perform a similar analysis as for the RCT estimate, but rather than use a single difference, we estimate the ITT effect using a Difference-in-Differences approach:

where is the average outcome for all individuals in city A at time T = 1, etc. As with the RCT estimate, we have to

correct this ITT estimate for the fact that not all individuals complied with their assigned treatment. If we let be the estimated proportion of individuals in city A that participate in Roca’s program, then we can obtain a

similar estimate to the Instrumental Variable estimate defined above:31

Combining the RCT and DD Estimates. The overall treatment effect estimate is a weighted average of the two

separate estimates, and . As discussed above, if the RCT is carried out exactly as intended, virtually all

of the weight should be on . If the randomized experiment instead faced serious challenges, then virtually

all of the weight should be on . We accomplish this goal by using a precision‐weighted average of the two estimates, where

31 While is similar to an Instrumental Variable estimate in many ways, it is not technically an IV estimate. See the

technical appendix for more discussion.

19

Then the weight on the RCT is:

And the final estimate is:

with standard error:

This weighting scheme ensures that the more precise estimate is given greater weight in the final calculations.

Extensions For The Pay-For-Success Evaluation. In practice, we do not observe a simple two-city, two-time setup. Instead, we observe multiple cities over multiple time periods as well as detailed information about individuals in the study population, such as age, education, and type of prior offense.

Fortunately, extending the above model to this new setting is relatively straightforward. First, we can extend

the estimate from the RCT, , via Two-Stage Least Squares (TSLS) with mixed effects:

where Xik are individual-level covariates, k = 1, …, K; and are random effects by group Gi; Wi is an indicator for actively receiving the treatment (i.e. participating in Roca’s program); and Zi is an indicator for

being randomly assigned to treatment. With this setup, the TSLS estimate of the treatment effect is , with corresponding standard error.

We can obtain similar estimates for the DD estimator (following Bertrand, Duflo, and Mullainathan [2004] and Hansen [2006]):

20

where and are time random effects; and are time-city random effects; and is an indicator for whether individual i is in an area with access to the program. As with the RCT IV estimate, the relevant DD

estimate is , with corresponding standard error obtained via TSLS.

As above, the overall weighted estimator is the precision-weighted average of the two estimates:

In the end, we believe that this estimate will ensure that the program evaluation yields an estimate of treatment effects that is both rigorous and robust.

Implementing the Evaluation

With respect to implementing the evaluation, the model of choice is a collaborative one, with an emphasis on transparency. In high stakes evaluation, trust in the process is critical; transparency at every step ensures engagement of all stakeholders throughout the process.

The process of building the evaluation follows a well-established pattern. The first phase involves design, data assessment, and instrumentation. Field data collection takes place in the second phase. The formative evaluation, which is the use of preliminary evaluation data to shape how the program unfolds, also takes place during the second phase. The summative evaluation, which addresses the impact and implementation questions, brings the project to a close in the third phase. Details of the proposed work in each phase are outlined below.

Design, Data Assessment, and Instrumentation

The principal objective during the first phase is evaluation design: operationalization of the independent variables, the dependent variables (i.e., outcomes), selection of data sources, and development of the counterfactual (i.e., random assignment vs. other approach).

Data assessment involves an audit of the administrative/electronic data held by state agencies and the provider to determine the extent to which independent and dependent variables are captured in those sources. Data from these sources may also prove useful for determining whether random assignment is possible and whether these data are stable enough to be used as part of the counterfactual, either as a standalone approach or in combination with other methods, as part of a multiple indicator approach.

To the extent there are gaps in the electronic data, data collection instruments and a data collection schedule will have to be built into the design. Data collection for the implementation analysis will also be sorted out during Phase I. Although data collection is instrument dependent, web-based applications for client enrollment, data collection, and formative evaluation are an efficient solution.

Fieldwork and Formative Evaluation

The fieldwork phase is operational: the goal is to carry out the data collection as specified in the design.

The formative evaluation involves bringing interim data to the stakeholders so that program operators, state agencies, and other stakeholders (e.g., investors in the case of PFS) have a near-to-real time understanding of how the project is unfolding. Although common in evaluation designs, the formative evaluation is particularly important in PFS context because of the way outcomes, risk, and financing are interconnected. In a worst-case scenario, if projected outcomes fall below certain levels, the stakeholders may want to revisit expectations and plans for moving forward. The formative evaluation would provide the inputs for these discussions, should they be necessary.

21

Summative Evaluation

The evaluation concludes with a report that assesses whether the program objectives were met. Although the evaluator is responsible for ensuring that the highest scientific standards are observed during the data analysis and summative interpretation, the form and substance of the report are stakeholder driven. Of course, the specific details of what the report will cover are tied to all of the issues that have been previously discussed.

Pay-for-Success – Evaluation Challenges

Social innovation financing is predicated on the idea that social service delivery systems are inefficient in their allocation of services relative to need. As a consequence, outcomes for clients are sub-optimal, which contributes to higher than necessary long-run costs to government. Social innovation financing addresses the underlying dynamics by turning fiscal incentives and (social) investment opportunities on their head.

Among their other features, the fiscal models on which social innovation financing rests typically involve the redistribution of risk. Specifically, in exchange for the opportunity to shift the service model toward prevention, providers and their investors accept greater financial risk than they would otherwise face. For example, under many if not all-financing schemes in current use, providers are reimbursed for the services they deliver. If the services provided do not improve outcomes and more services are needed, providers are not penalized per se, there are few if any incentives to change the mix of services offered, and the risks fall squarely on the public sector. Social innovation financing shifts purchasing decisions from services to outcomes. Providers are given the flexibility to adjust their clinical (i.e., programmatic) approach and are then judged on whether their new service array achieves the desired outcomes. The failure to improve outcomes rests more squarely with the provider, as do the financial risks. Investors may also share in these risks, depending on the risk-sharing arrangements.

In contrast to so-called clinical trials in which the end goal is a summative assessment of whether the intervention worked, evaluation in the context of social innovation financing requires far greater focus on both formative and summative evaluation modes. The formative evaluation component is critical because of the in situ operating model. Traditional clinical trials are funded to test whether an intervention works, so there is no direct link between the funding and the outcome of the trial. In contrast, social innovation financing involves a direct connection between funds available and the outcome; real-time feedback (i.e., formative evaluation molded around a CQI framework) is much more important as a risk management strategy. Program operators cannot wait until the project is over to learn whether it worked. In this context, the evaluation design has to assume a more active, formative role in how the model unfolds. Creating this infrastructure and then integrating it with program operations is a key design challenge for everyone involved.

The Massachusetts social innovation financing model sets forth an ambitious agenda with respect to the life course outcomes of the target population. Appropriately, the program model operates over an extended period of time, with the expectation that during the window of opportunity, youth will realize a brighter future with the help of the offered services. Two issues in this context are particularly important. First, many of the life course outcomes will take place after the formal evaluation window. As such, the evaluation design has to account for robust measurement of observed changes, on the one hand, and robust projections of what is likely to happen after the evaluation window, on the other. Second, it is difficult to manage all of the exogenous factors (i.e., period effects) that could influence whether youth outcomes change during the program window and beyond. For example, changes in local, state, or national economies could cause the labor market to shrink or grow, as the case may be. Criminal behavior, employment, and schooling are all affected by contextual factors that intervention developers and program administrators cannot reasonably control. Isolating a treatment effect in these dynamic contexts requires a deft methodological touch that invites a mixed method approach.

22

A related concern has to do with scale. Smaller, tightly designed experimental designs may offer the best evidence of impact but it is unclear from studies of this type whether the same effects are possible at scale. This is a classic problem in studies of employment where it is often possible to show treatment effects on employment outcomes but it is much more difficult to show changes in employment rates when the programs are taken to scale. If social innovation financing, beyond the early trials, is expected to deliver system level benefits (i.e., to bend the cost curve of government), it is not certain that results from smaller scale studies can be extrapolated into system effects. Backend simulation models may help solve this problem, but the use of system dynamic or agent-based simulations adds to the design considerations at the project’s front-end.

Finally, for the most part, systemic inefficiencies are often treated as fixed effects. That is, poor outcomes at the individual level are deemed to be the result of a poor match between the services offered and the needs of individuals. Less thought is given to whether the mismatch is a variable problem, i.e., more likely in some parts of the state than others. If the need/service mismatch is a variable problem - larger in some parts of Massachusetts than others - the evaluation design will be much more reliant on mixed effects models. This is not an evaluation issue as much as it is a program design problem. If effects vary by place, then investments and expectations of return on those investments should vary as well. The evaluation will have to be carefully calibrated in the event that place-based effects are present.

Safeguards For Target Population

Risks facing the target population generally fall into two categories: the risk of disclosure of confidential information and the risk of adverse treatment effects. With respect to the former, the evaluator will be responsible for designing an evaluation and data collection plan that safeguards the rights of participants as human subjects. Review of the project by the relevant IRB(s) will insure that standard practice is observed. In addition, the Center has a long track record of protecting confidential information so that risks to subjects associated with confidential information will be minimal. We use secure data transfer protocols, store data on secure servers, and de-identify data once the need for identifying data has been eliminated. Where necessary, we analyze confidential data on stand-alone computers (i.e., computers without a network connection). Our reports are reviewed internally (and externally by state agencies) to ensure that confidential information is not released. We also take steps to prevent deductive disclosure.

Adverse treatment effects are harder to judge and manage. IRB requirements mandate that evaluators have a process for notifying the proper authority when research subjects pose a threat of harm to themselves or others. Protocols require that treatment subjects who wish to opt out of the study may do so at any time. Participation in the study is voluntary, a point that is made clear during the consent process. Participants may withhold any information at their discretion. Consent for access to electronic data may, in some cases, be required. Adverse events - release of confidential information, harm to a participant - must be reported to the IRB.

Finally, the formative evaluation phase offers a second opportunity to assess how the project is going. If in the judgment of the stakeholders, there is an adverse impact on participants in some general or aggregate sense, the formative evaluation includes a process whereby suspension of the study is considered.

23

Date post:	14-Feb-2017
Category:	Documents
Upload:	doantruc
View:	225 times
Download:	0 times

Roca Strategic Business Plan

Documents