C O R P O R A T I O N The Appalachia Partnership Initiative’s Investments in Education, Workforce Development, and the Community Analysis of the First Stage, 2014–2016 Gabriella C. Gonzalez, Shelly Culbertson, Nupur Nanda • The Appalachia Partnership Initiative (API) sponsored K–12 STEM education programs that incorporated hands-on and project-based instructional models that aimed to improve skills acquisition. API K–12 science, technology, engineering, and mathematics (STEM) education programs served 11 percent of the region’s 680,000 school-age children. • Workforce programs engaged with industry leaders to support employability of participants, but few focused specifically on transitioning workers. • The API programs were heavily represented in Pennsylvania and West Virginia, but lightly represented in Ohio. • Supportive government, industry, and education partners were key to sustainability. • Programs with varied funding sources viewed themselves as more financially sustainable. • There is a vibrant tristate regional STEM-supportive ecosystem. The API served as a key partner in this system and started to engage in policy advocacy. • Universities served as key resources for API programs, and many funders contributed to the API programs, but their efforts were not coordinated. • The API can further its vision for the region through ongoing efforts to broaden regional awareness of STEM career opportunities, strengthening STEM teacher training and development, and facilitating collaboration across networks. • To support the longer-term financial sustainability of API programs, the API could further leverage existing networks of STEM education and workforce development funders; these networks offer opportunities to engage long-lasting sources of support within the region. Key findings T he tristate Appalachia region of southwestern Pennsylvania, northern West Virginia, and eastern Ohio is anticipated to face shortages of workers with science, technology, engineering, and math (STEM) skills for the energy and advanced manufacturing industries over the coming decade. In 2014, the Appalachia Partnership Initiative (API) launched in order to invest in K–12 STEM education and workforce development programs and initiatives that could help address these shortages by improving skills and opportunities for the region’s population. is report assesses the API’s progress toward its vision, covering the first years of the API: October 2014 through July 2016. Interim assessments will be conducted annually from 2016 to 2019. A final sum- mative evaluation, forthcoming in 2020, will analyze the effect of API efforts through time and report on short-term outcomes.
Transcript
C O R P O R A T I O N
The Appalachia Partnership Initiative’s Investments in Education, Workforce Development, and the CommunityAnalysis of the First Stage, 2014–2016
Gabriella C. Gonzalez, Shelly Culbertson, Nupur Nanda
• The Appalachia Partnership Initiative (API) sponsored K–12 STEM education programs that incorporated hands-on and project-based instructional models that aimed to improve skills acquisition. API K–12 science, technology, engineering, and mathematics (STEM) education programs served 11 percent of the region’s 680,000 school-age children.
• Workforce programs engaged with industry leaders to support employability of participants, but few focused specifically on transitioning workers.
• The API programs were heavily represented in Pennsylvania and West Virginia, but lightly represented in Ohio.
• Supportive government, industry, and education partners were key to sustainability.
• Programs with varied funding sources viewed themselves as more financially sustainable.
• There is a vibrant tristate regional STEM-supportive ecosystem. The API served as a key partner in this system and started to engage in policy advocacy.
• Universities served as key resources for API programs, and many funders contributed to the API programs, but their efforts were not coordinated.
• The API can further its vision for the region through ongoing efforts to broaden regional awareness of STEM career opportunities, strengthening STEM teacher training and development, and facilitating collaboration across networks.
• To support the longer-term financial sustainability of API programs, the API could further leverage existing networks of STEM education and workforce development funders; these networks offer opportunities to engage long-lasting sources of support within the region.
Key findings
The tristate Appalachia region of southwestern Pennsylvania, northern West Virginia, and eastern Ohio is anticipated to face shortages of workers with science, technology, engineering, and math (STEM) skills for the energy and advanced manufacturing industries over the coming decade. In 2014, the Appalachia
Partnership Initiative (API) launched in order to invest in K–12 STEM education and workforce development programs and initiatives that could help address these shortages by improving skills and opportunities for the region’s population.
This report assesses the API’s progress toward its vision, covering the first years of the API: October 2014 through July 2016. Interim assessments will be conducted annually from 2016 to 2019. A final sum-mative evaluation, forthcoming in 2020, will analyze the effect of API efforts through time and report on short-term outcomes.
1. THE NEED FOR STEM EDUCATION AND WORKFORCE DEVELOPMENT INVESTMENTS IN APPALACHIAThe combination of horizontal drilling and hydraulic fractur-ing to tap natural gas has resulted in the Utica and Marcellus shales becoming major sources of natural gas supply within the United States. This is predicted to bring significant, long-term economic benefits in employment to the tristate Appalachia region encompassing parts of Ohio, West Virginia, and Penn-sylvania.1 The U.S. Energy Information Administration (EIA) estimated that the Marcellus and Utica shales contributed to 85 percent of the increase in natural gas production between January 2012 and January 2015 within the United States.2 According to a 2013 estimate, the number of jobs supported by the shale gas industry is projected to increase by 1.8 million between 2012 and 2025, contributing approximately $1.6 tril-lion to U.S. government revenue within the same time period.3 In 2015 and 2016, there were lower-than-anticipated oil and gas prices and consequently a reduction in the immediate produc-tion and extraction of natural gas and oil in the tristate region.
Yet, the need for workers is expected to resume over the medium and longer term once oil and gas prices recover, particularly in middle-skilled jobs. Middle-skilled positions typically require a high school education, with some additional postsecondary training or occupational certification.4 Employers have reported difficulty in filling middle-skilled jobs that require STEM skills.5 The growing need for a STEM-skilled workforce is likely to further intensify given impending retirements of large numbers of older workers;6 nationally, nearly 25 percent of workers employed in extraction and production occupations are over age 55 and nearing retirement.7
There is also demand for improvement in STEM curricula in the region to prepare students for future job opportunities in the energy sector. A 2014 study found that 40 percent of surveyed parents, educators, and business leaders in the tristate Appalachia region reported that they believe the local school systems need to do a better job of preparing students for future jobs.8 The study also found a lack of awareness—especially in the rural areas examined—about how STEM education can prepare students for lucrative job opportunities in the future. Importantly, the study found that rural regions “represent one of the greatest, yet underexploited, opportunities for STEM education to impact workforce development.”9
To support the development of a skilled workforce that meets the needs of the energy and advanced manufacturing industries, the API launched in 2014. The initiative was led by the Social Investment Team of Chevron North America’s Appalachian Mountain Business Unit. Chevron committed to investing $20 million to support K–12 STEM education and STEM workforce development programs to educate and train local adult workers, and to catalyze community engagement and policy advocacy on issues related to STEM education and workforce development in 27 counties in the tristate region (see Figure 1).
The API consists of an expanding network of partner organizations from the business, foundation, nonprofit, research, and education sectors. In addition to the Chevron Corporation, founding members (referred to as API leaders) include the Allegheny Conference on Community Development and the Claude Worthington Benedum Foundation. Recent new members are the Grable Foundation (2016) and Catalyst Connection (2017). Membership is
Employers have reported difficulty in filling middle-skilled jobs that require STEM skills. . . . To support the development of a skilled workforce that meets the needs of the energy and advanced manufacturing industries, the API launched in 2014.
2
Figure 1. The 27 Counties Composing the API Region
Monroe
Harrison
Carroll
Columbiana
Mahoning
Tuscarawas
Stark
Jefferson
Hancock
Indiana
Lawrence
Beaver
Mercer
Butler
Armstrong
Allegheny
Brooke
Belmont
Wetzel
Marshall
Monongalia
Westmoreland
Greene
Washington
Fayette
Marion
Ohio
Pennsylvania
Maryland
West Virginia
Ohio
RAND RR2017-1
3
expected to expand over time. The RAND Corporation serves as the external research and analysis lead for the API.
THE APPALACHIA PARTNERSHIP INITIATIVE’S VISION FOR THE REGIONThe API’s vision is to promote “a sustainable regional energy and manufacturing education and employment ecosystem that supports the region’s broader economic development.” From March through July 2015, API leaders developed a strategic framework to meet this vision for its programming with RAND facilitation (see Figure 2). The API strategic framework is a logic model10 that maps the types of programs the API expects to fund, desired outputs and outcomes of
programming, and a communitywide next-generation vision illustrated on the far right of Figure 2. We also developed accompanying indicators for outputs and outcomes of the strategic framework to measure the API’s progress.11 API leaders aim to achieve this vision by supporting programs and activities in three categories:
• K–12 STEM-related educational opportunities that willexpand and maintain the pipeline of local talent qualifiedfor jobs in the energy and advanced manufacturing sectors
• workforce development programs that aim to build thecapacity of local workers so they are prepared for and haveaccess to jobs in the energy and advanced manufacturingsectors
• community-catalyst efforts that build integrative rela-tionships among the region’s corporate, nonprofit, govern-ment, and community leaders.
• More workers preparedto enter energy andmanufacturing careerpathways
Key issues and assumptions: Skills mismatch - Aging workforce – Employability – Lack of awareness of STEM-related careers – Underemployment (veterans, rural areas)
• Understanding
• Performance
• Knowledgeable teachers
• Careerconnections
• Understanding
• Hiring & retention
• Linked training& industry
• Careerconnections
• Funding &commitments
• Policyprioritization
• Education andawareness
• Engagement
Inputs Program Activities
Outputs (direct results from program) Short-term
(current)
Outcomes
Mid-term (3–5 years)
Long-term (5–10 years)
Next-Generation (10–30 years)
Improved STEM K−12 and postsecondary education outcomes in the API region
Increased employment in energy and manufacturing in the API region, filling job and skills demand
Active and sustainably funded workforce policy community
Staff
Time
Money
Materials
Equipment & Facilities
Technology
Advocacy
Networking
In-kind Contributions & Expertise
A sustainable regional energy and manufacturing education and employment ecosystem that supports broader economic development
Directly Measurable API Results
Outcomes - Impact
Directly Measurable API Results The API as One Contributing Actor
4
As of December 2016, the API sponsored the following K–12 STEM programs in the API footprint:12
• The Carnegie Science Center managed three programswith an emphasis on informal museum-based learning: theStudent Energy Summit, Grand Slam Science on the Road,and a stationary and mobile Fabrication Laboratory (Fab Lab).
• Intermediate Unit 1 also managed a mobile Fab Lab.• The Children’s Museum of Pittsburgh operated Explore
Making and Building the West Virginia Network,initiatives with an emphasis on informal, museum-basedlearning.
• Education Alliance developed an educational model forSTEM Network Schools in West Virginia.
• Project Lead the Way provided professional development innew curricula focused on project-, problem-, and activity-
based STEM instructional approaches in schools in Pennsylvania, Ohio, and West Virginia.
Two grantees developed or operated programs that focus on both K–12 STEM and workforce development: Catalyst Connection launched seven different programs focused on engaging students with employment in manufacturing, and Central Greene School District developed a commercial driving course, Natural Resources.
Workforce development programs included the following: • ShaleNET is a partnership among employers and four
technical colleges in Pennsylvania, Ohio, and WestVirginia.
• Southwest Training Services operated employment servicesfor coal workers recently laid off or dislocated.
Figure 3. Time Line of Initiation of Funding for API Grantees and Programs
RAND RR2017-3
Energy Innovation Center (est. 2014)Energy Safety Passport
2013 2015
STEM Workforce Development Workforce Development / STEM
Education Alliance (est. 1983)
ShaleNET (est. 2010)Navarro College, Stark State College,
Central Green School District (est. 2010)Natural Resources
Westmoreland County Community College, Pennsylvania College of Technology
Carnegie Science Center (est. 1939)Student Energy Summit, Grand Slam Science
Southwest Training Services (est. 2000)
Joint Workforce Development/STEM to STEM (est. 2015) Catalyst Connections
(est. 2002)Adventures in Technology/Manufacturing Innovation Challenge, Middle School Student Video Contest, Manufacturing Career Exploration Program in partnership with Girls Scouts, Pathways to Apprenticeship, Professional Development on Project-Based Learning
API Announcement (October 21, 2014)
Project Lead the Way (est. 1997)
Children’s Museum of Pittsburgh (est. 1972)Explore Making, Building the West Virginia Maker Network
2014 2016
5
• The Energy Innovation Center managed the Energy SafetyPassport program, a safety training program for energyemployees.
Figure 3 shows a time line of when the API started funding programs and the grantees that manage the programs.
REPORT OBJECTIVES AND APPROACHRAND is conducting a multiyear assessment of the API’s prog-ress toward its vision. Interim assessments will be conducted annually from 2016 to 2019. This report summarizes findings from our first interim assessment and covers the first years of the API: October 2014 through July 2016. It establishes a baseline against which subsequent analyses can be com-pared. Because this is a first snapshot, this descriptive portrait describes activities and documents outputs as specified in Fig-ure 2, but does not aim to assess outcomes. A final summative evaluation, forthcoming in 2020, will analyze the effect of API efforts through time and report on short-term outcomes.
This baseline assessment asked four questions:13 • Strategic alignment: Were the API’s programs and activi-
ties aligned with the API’s vision and strategy, as docu-mented in the API Strategic Framework?
• Geographic scope and beneficiaries: What was the geo-graphic scope of API programs, and which beneficiaries didthe programs reach?
• Sustainability: How sustainable were the API programs?• Community catalyst: What steps did API leaders take
to catalyze a community of stakeholders to work towardsimilar goals?
To answer these questions, we relied on four sources of data:14 • Interviews and site visits. Between June and August
2016, we interviewed API program administrators, visitedprogram sites, and reviewed program documentation.Interview questions focused on program activities andmission, beneficiaries, facilitators and barriers to imple-mentation, sustainability, funding or in-kind support, andreported effectiveness to date.15
• Quantitative indicators. Between August and October 2016, program administrators submitted to us quantitative indicators, such as numbers of participants in the pro-grams, based on a data collection template that we sent to each program. Program administrators were asked to pro-vide as much data as feasible, covering the period between when the API started funding their program to July 2016. All K–12 STEM education programs provided us with the data requested. About one-half of the workforce develop-ment programs provided us with the data requested.16
• Survey of API leaders. In November 2016, API lead-ers provided written responses to a questionnaire that we administered. These questions focused specifically on the role each API leader took through July 2016 to serve as a community catalyst, the nature of their community rela-tionships, and connections.17
• Review of literature. To inform our understanding of the extent to which programs were incorporating evidence-based practices in their programming, we conducted a literature review of select topics, including museum-based informal education; activity-, problem-, and project-based instructional approaches; sector-based career pathways; promising models in workforce development programs; network analysis in nonprofit settings; and related policies and activities in K–12 STEM education and workforce development.
We analyzed the interview notes and API leader question-naire responses by organizing responses according to the four research questions and aggregating similarities and differ-ences. We compared programs’ activities and outputs with those outlined in the strategic logic model. A Pittsburgh-based technology and design firm, Informatics Studios, used program quantitative indicators to create an infographic map to pictori-ally describe the geographic scope and key output indicators.18 Using data from the data-collection templates, interviews, and API leader questionnaire, we produced network-analysis dia-grams to analyze relationships among API programs, funders, and regional partners.19 We created network diagrams of these connections to assess overall network composition, pinpoint opportunities for the API to leverage existing links between programs and organizations, and highlight opportunities to leverage networks to advance API goals.
This first assessment had three important limitations: • Any overall judgment of API effectiveness in achieving
its vision would be premature. Therefore, this early-stage
assessment measures progress toward the vision and offers suggestions for future decisionmaking regarding program-ming, funding, or API direction.
• This report evaluates the API portfolio, not individual pro-grams. It focuses on how programs collectively promote the API’s goals and where gaps exist.
• This analysis incorporates data from multiple sources, some of which were self-reported or incomplete because of the nascence of the API programs and their data-collection efforts. Future reports will include more comprehensive data.
STRATEGIC ALIGNMENTThe strength of a portfolio of programs depends, in part, on the extent to which programs’ activities and goals match those articulated in a logic model.20 To determine this, we analyzed API programs’ activities and goals and documented how aligned those were with the activities and outputs illustrated in the API Strategic Framework in Figure 2. This first-year description of alignment can provide API leaders with informa-tion on potential redundancies or gaps in programming, which could be used to make decisions about where to pursue further programming. The next sections provide summaries about the findings are related to skills acquisition (for both K–12 STEM and workforce development); professional development; skills alignment; and industry engagement, networks, and awareness (for both K–12 STEM and workforce development).
Programs Incorporated Hands-on and Project-Based Instructional Models That Aimed to Improve Skills AcquisitionAPI’s central goal was to fund innovative, hands-on, and project-based learning experiences in formal and informal education settings as a way to improve students’ acquisition of STEM skills and to improve their connection to careers in the 21st-century STEM labor market. The API strategically funded programs that were either pilot efforts or that used promising instructional models meant to improve participants’ skills. Although these programs and their instructional methods had not yet been rigorously tested to determine their effectiveness, API investments allowed for these approaches to be tried out; eventually, if proven to be successful, the API would encourage that these approaches be scaled up.
6
Table 1 lists instructional models employed by K–12 STEM and workforce development programs. Some of these programs, such as the ones at the Carnegie Science Center and the Children’s Museum of Pittsburgh, exposed children to STEM concepts and helped them develop STEM skills in informal settings, such as through maker spaces and fabrication laboratories. Other programs employed hands-on, practice-based learning approaches to improve students’ and workers’ acquisition of STEM skills in traditional classroom-based settings, such as Project Lead the Way, ShaleNET, and the Natural Resources course at Central Greene School District.
API-Funded Teacher Professional Development Efforts Were for Current Teachers, Not for Pre-Service TeachersAll of the API’s K–12 educational programs offered professional development to train teachers in the instructional practice and hands-on learning techniques used in the API programs, such as project-based learning or particular technologies used in the FabLabs or MakerSpaces. For example, Project Lead the Way offered extensive training before and during the school year to teachers on teaching methods for the Project Lead the Way engineering and computer science curriculum. At the time of this study, 961 teachers in API-sponsored K–12 programs
Table 1.
API Grantee Skills Acquisition Model
K–12 STEM Education Programs
Carnegie Science Center: Chevron Center for STEM Education and Career Development
Hands-on learning experience for students to create their own projects (Student Energy Summit)
Hands-on, multimedia experience integrating lessons in physics with baseball (Grand Slam Science, On the Road)
Hands-on STEM learning experience for students to digitally fabricate own products (FabLab)
Children’s Museum of PittsburghHands-on learning experience for students to create and make their own crafts and
projects to take home (Explore Making)Student crafts and projects in maker spaces (Building the West Virginia Network)
Education Alliance Strategic planning process to include industry (STEM Network Schools)
Project Lead the Way Project-, problem-, and activity-based instructional approaches
K–12 STEM Education/Workforce Development Programs
Natural Resources at Central Greene School District
Hands-on technical training, courses that cover the scientific background for the industry, commercial driver’s license training, safety training and certificates.
Catalyst Connection
Student-made videos about “what makes manufacturing cool,” partnered with a manufacturing company (Middle School Student Video Contest)
Student research-based projects partnered with a manufacturing company (Adventures in Technology)
Project-based learning training for adultsProject-based learning training for teachers (Professional Development Related
Activities)Supporting manufacturing companies in the development of internship and
apprenticeship opportunities
Workforce Development Programs
ShaleNETStackable credentialing (with multiple entry and exit points, where individuals may
build degrees after obtaining technical certificates or credentials) and career pathways
Southwest Training Services, Inc. Work-based learning
Energy Innovation Center’s Energy Safety Passport
Hands-on learning of a cross-company safety curriculum
7
received professional development training.
However, the professional devel-opment offered was in-service training, for those already working as teachers, as opposed to pre-service training, which would take place while potential teachers are gaining their certification. Program administra-tors reported that attrition among the program-specific trained teachers was a common problem: When teachers depart, they would take their new skills with them.
Few Workforce Programs Focused Specifically on Transitioning WorkersA core population of interest for API leaders was transitioning workers, such as veterans or dislocated workers. Transitioning workers hold skill sets that could transfer to energy-and manufacturing-sector jobs. The API therefore set out to fund programs that supported the transferability of skills of transitioning workers or provided reskilling so that prior competencies and skills could be incorporated in current employment. We found that two of the five workforce development grantees explicitly focused on transitioning workers: ShaleNET and Southwest Training Services. ShaleNET colleges recruited and tracked veterans to enter their stackable credential coursework, and Southwest Training Services program specifically focused on displaced coal miners. At the time of this study, program administrators documented 46 veterans and displaced coal miners who had participated in these programs.21 The API can now determine whether to sponsor more programs that focus on transitioning workers or whether this is sufficient for the time being.
Programs Engaged with Industry Leaders to Support Employability of Participants All of the workforce development programs reported they had engaged industry leaders (over 100 companies in total). Fifteen companies provided in-kind expertise and time, 16 collaborated on curriculum, and three partnered with indus-try in job-placement efforts, through career counseling, job databases, or exposure to companies with possible job openings.
Industry supported workforce develop-ment programs by providing techni-cal expertise in curriculum design to meet on-the-job demands, equipment to ensure that students practiced their skills using the most up-to-date mate-rials, and workplace experiences for students, whether through internships, visits to companies, or guest lecture-ships.
Administrators of K–12 programs reported offering various ways to link secondary students with industry. These included:
• Helping secondary students make connections with local industry for jobs that require a secondary degree through career guidance, guest speakers, and mentoring field trips.
• Collaborating with companies to provide materials, equipment, and input on curriculum.
• Training high school guidance counselors on the availabil-ity of STEM career options.
Programs Included Strategies to Improve Awareness, but Efforts Were Not Linked to an Overarching StrategyAs explained earlier in this report, research has demonstrated there is little awareness about STEM career pathways in the region. Therefore, the API Strategic Framework lists improved awareness as a key output of both K–12 education and work-force development programs. At this early stage, our analy-sis was not able to measure improved awareness of program participants. Instead, we reviewed efforts by the programs that aimed to improve awareness. We found that both K–12 education and workforce programs sought to improve aware-ness of STEM careers through their approaches to teaching, training, and public activities.22 Program administrators noted that the experiential learning methods used in classroom- and museum-based K–12 STEM programs were intended to bring about a “cultural shift” or “mindset change” in how students viewed STEM education and careers; it was considered one way to excite students and motivate them to pursue STEM-
961teachers receiving
professional development in K−12
STEM programs
46veterans and transitioning
workers served
15companies
contributing in-kind expertise and time
16companies collaborating
on workforce development programs’
curriculum
companies participating in
workforce programs
100
8
related careers. Workforce program administrators aimed to improve awareness through instruction and training techniques and by establishing connections between students and energy and manufacturing employers. Two indicators reveal pro-grams’ early efforts to increase awareness around STEM career opportunities: Between October 2014 and July 2016, K–12 and workforce development programs received eight public awards—casting attention on their programs—and hosted 65,323 attendees at the programs’ public events.
While interviewees lauded that there seemed to be more attention to improving STEM skills or awareness across the region, interviewees perceived that programs’ activities were isolated from one another and unconnected to a broader API systemwide strategic view of how to change student interest, engagement, or awareness of STEM career pathways. Moreover, there was no direct measure of awareness about STEM careers and whether awareness has increased across the region.
GEOGRAPHIC SCOPE AND BENEFICIARIESOur analysis in this section documents where API programs were located across the 27 counties listed in Figure 1, the number of people the programs reached (scope and scale), and the percentage of all potential beneficiaries who were reached by API-funded pro-grams. Yet it is important to note that the API recognized that its programs alone could not feasibly reach all individuals across the entire 27-county footprint. Thus, the findings reported in this first snapshot of scope and scale can inform API leaders on where to pursue further programming: Should the API focus on increased breadth of its programming or pinpoint specific regions where there might be gaps in programming?
API Programs Were Heavily Represented in Pennsylvania and West Virginia, and Lightly Represented in OhioAs shown in Figures 4 and 5, API programs were available in all 27 API counties.23 Although there appeared to be geographic parity, a closer look revealed that most API-funded programs were available in southwestern Pennsyl-vania, with a concentration of programs in Allegheny County, where there was also the largest number of public schools, school-age children (age 3–17), and working age adults (age 18–64). All API workforce development programs were available in counties in northern West Virginia and in Allegheny County. There were few API-sponsored programs avail-able in Ohio counties (two workforce development programs and two K–12 STEM programs).
This distribution of programs presents an opportunity for API leaders to consider how to balance investments across the three states. Programs are currently concentrated both in Chev-ron’s locations across the region and in more-populated coun-ties. Given the API’s stated emphasis on rural communities, going forward, API leaders could consider to either continue to focus efforts in more-populous counties or in rural counties with fewer programs in place that might have the greatest need. To make these types of decisions, API programs will need to be able to collect data on the number of beneficiaries served by each program in each county (to date, not all programs are able to collect that information).
API Programs Reached Targeted BeneficiariesAt the time of this study, K–12 STEM education programs served 11 percent (75,316) of the region’s 680,000 school-age children in informal museum-based settings or in formal education settings in 122 schools (of which 87 were in rural communities). Fifty-five percent of those served fell under each program’s definition of low income. API programs were thus reaching a proportion of the populations that API leaders wanted to focus attention on: schoolchildren
65,323people who attended
public events associated with the API
public awards received
8
122schools
87schools in rural
communities
5postsecondary institutions supported by the API that serve rural communities
75,316 students
9
in rural communities or those in lower-income families. The workforce development programs that provided us with data reported 383 participants, and 77 instructors received occu-pational-related professional development. Five postsecondary institutions supported by the API were based in rural counties.
SUSTAINABILITYPromoting sustainable programs that continue into the future without API funding was a key goal of API leaders. Therefore, we analyzed which factors facilitated or inhibited API pro-grams’ sustainability—the extent to which the benefits of an activity are likely to continue into the future and in particular after donor funding has been withdrawn27—and programs’ ability to achieve desired goals.
Supportive Government, Industry, and Education Partners Were Key to SustainabilityPrograms administrators identified three factors central to sup-porting their ability to meet program goals and future sustain-ability:28
• Government support: For some programs, federal, state, and local governments provided supportive policy contexts or apportioned funds for K–12 STEM and workforce development, which in turn provided more secure financ-ing and ability to develop capacity of staff. For example, ShaleNET launched with grants from the U.S. Depart-ment of Labor’s Education and Training Administration, totaling $19.9 million. At the state level, the West Virginia governor’s STEM Council, the Pennsylvania governor’s STEM Planning Committee, and the Ohio State Depart-ment of Education considered policies in support of K–12 STEM initiatives.
• Industry support: Industry leaders engaged with and provided in-kind or financial support to API programs,
including staff time and expertise, equipment, or access to work sites.
• Education support: Program administrators emphasized the “enthusiasm” and “commitment” of teachers, school administrators, and trainers in participating programs because of a perceived need for new STEM approaches. This, in turn, facilitated programs being able to meet expected goals.
Multiple Challenges Could Impede Programs’ SustainabilityProgram administrators also identified challenges to meeting current and future goals:
• Perceptions about STEM careers: Program administra-tors reported that youth, parents, guidance counselors, and adult workers continued to have misperceptions or lacked awareness about opportunities in STEM careers, required skills of STEM jobs and tasks, and potential salaries.
• Weak links between education and STEM careers: Program administrators pointed to a lack of clear pathways between K–12 education and middle-skill STEM jobs; it would be important for students, parents, school leaders, teachers, and guidance counselors to understand which courses in middle or high school were in demand in the workplace. While Ohio developed such career pathways, Pennsylvania and West Virginia had not.
• Teacher qualifications and turnover: When teachers or instructors in particular programs left, programs needed to train anew. Administrators of K–12 education programs lamented that pre-service teacher colleges do not educate their teachers-in-training with needed skills in project-based instruction or particular subject-matter expertise. The workforce development program administrators reported competing for quality instructors with the private sector.
• Inconsistent leadership support: Program administrators believed that leadership support at the school or district level for STEM K–12 education programs was uneven. They speculated that this lack of support might stem from competing priorities, the need to adhere to state testing requirements that leave little flexibility in curricula and scheduling, constrained budgets, and lack of understand-ing of new STEM learning approaches, project-based instructional methods, or informal learning models.
lower-incomeparticipants
55% 383
workforce development program participants
receiving API scholarship assistance
10
• Industry fluctuations: Program administrators that pro-vided certifications in the manufacturing or energy sectorsfound it difficult to attract participants between 2015 and2016 because of decreased employment in those sectors incomparison with the initial shale gas boom between 2011and 2015. Employment demand was expected to increaseagain, but program administrators feared that their pro-grams that are dependent on enrollment to survive couldfold before the economy rebounds.
Programs with Varied Funding Sources Viewed Themselves as More Financially Sustainable Most API programs received financing from multiple sources, including foundations and corporations (such as API partners), tuition and fees, fundraisers, federal and state government funding, and in-kind contributions (e.g., expertise, equip-ment, site access to companies). Program administrators who reported that they had a financially sustainable model to
Figure 4. Geographic Distribution of K–12 STEM Programs, by County in API Region
Number of school-age children in households (ages 3–17) Number of public schools
Number of API K–12 STEM Education Programs0 200,000
8Monroe
7Harrison
8Carroll
38Columbiana
72Mahoning
31Tuscarawas
140Stark
43Jefferson
11Hancock
20Indiana
25Lawrence
49Beaver
40Mercer
31Butler
10Armstrong
251Allegheny
10Brooke
23Belmont
6Wetzel
14Marshall
19Monongalia
86Westmoreland
13Greene
60Washington
42Fayette
21Marion
14
2
2
2
2
2
2
2
2
2
2
2
2
2
3
2
4
2
2
2
4
3
3
4
4
4
3
3
Ohio
Pennsylvania
Maryland
West Virginia
Ohio
ools in Ohio is from Great Schools (undated); number of public schools in Pennsylvania is from Pennsylvania Department of Education (undated); number of public schools in West Virginia is from WV State Department of Education (undated).RAND RR2017-4
SOURCES: Number of school age children is from the American Community Survey 2009–2014 five-year estimates; numberof public schools in Ohio is from Great Schools; number of public schools in Pennsylvania is from Pennsylvania Department ofEducation; number of public schools in West Virginia is from West Virginia State Department of Education.26
11
Figure 5. Geographic Distribution of Workforce Development Programs, by County in API Region
Size of working-agepopulations (ages 18–64) Number of Workforce Development Programs
0 800,000
Pennsylvania
Maryland
West Virginia
Ohio
SOURCE: Size of working age population is from American Community Survey 2009–2014 �ve-year estimates.RAND RR2017-5
Monroe
Harrison
Carroll
Columbiana
Mahoning
Tuscarawas
Stark
Jefferson
Hancock
Indiana
Lawrence
Beaver
Mercer
Butler
Armstrong
Allegheny
Brooke
Belmont
Wetzel
Marshall
Monongalia
Westmoreland
Greene
Washington
Fayette
Marion
Ohio
2
2
2
2
2
2
8
3
34
2
2
2
2
8
2
2
2
2
2
22
2
8
8
8
8
12
continue their efforts typically were those with a diverse set of funding sources. In contrast, program administrators with heavy reliance on a single source of funding expressed concern over risks to the continuity of their programs. They noted that their budgets were often unpredictable because of changing or conflicting national and state priorities. Several programs described the benefit of having one or two respected funders: The name recognition of the funders made it easier to obtain other external funding.
COMMUNITY CATALYSTRecognizing that the set of API programs, on their own, would not be sufficiently at scale to achieve the API’s next-generation vision, API leaders aimed to mobilize a community of educa-tion, workforce, private-sector, foundation, government, and nonprofit leaders to work toward similar goals. As illustrated in the API Strategic Framework in Figure 2, API leaders sought to leverage financial and in-kind contributions, engage in policy advocacy, mobilize community involvement, and develop net-
SOURCE: Size of working-age population is from American Community Survey 2009–2014 five-year estimates.
works for change. The analysis provided in this section assesses the extent to which API leaders have catalyzed a community to date. We found several trends, gaps, and opportunities for the API to further act as a community catalyst. API leaders can use the findings from this analysis to consider how they can most effectively manage their individual and collective relationships to achieve the community catalyst goals.29
There Is a Vibrant Regional STEM-Supportive Ecosystem At the time of this study, other initiatives related to API goals were underway in all three states, with variation in partnerships at the local, state, and interstate levels. In Pennsylvania, Pittsburgh-based organizations had been active in developing new K–12 STEM education models, such as through Pittsburgh’s Remake Learning Network and Pittsburgh’s nationally leading role in “the Maker Movement.” Pittsburgh and Youngstown, Ohio, had both received federally designed “Maker City” status. West Virginia had a number of K–12 STEM initiatives, yet some interviewees noted that, in general, West Virginia lagged behind other states in regional and national partnerships, a view that, as a state, they were “not as plugged in as other states.” Ohio initiatives included the Ohio STEM Hub Network, Ohio Career Pathways, the “Choose Ohio First” scholarship, and the federally designated Manufacturing Hub status of Youngstown, Ohio. Despite the rich policy environment in Ohio, few API programs have been engaged with state or local government entities in Ohio, unlike the API program engagement with government in Pennsylvania and West Virginia. All three states had federally funded Manufacturing Extension Partnerships and Workforce Investment Boards, along with membership in the Appalachian Regional Commission, a federally funded regional economic development organization.
The API Served as a Key Partner in the Regional STEM-Supportive Ecosystem and Started to Engage in Policy AdvocacyAPI leaders and program administrators held leadership positions in other related STEM education and workforce development initiatives. In particular, the Chevron Corporation, Benedum Foundation, and Allegheny Conference on Community Development cosponsored the Tri-State Shale Summit of October 2015. There, the governors of the three
states signed the Tri-State Regional Cooperation Agreement, which committed the states to “optimize added economic value to the Tri-State Region in a safe and environmentally responsible manner.” There has not been tristate coordination of K–12 STEM education activities similar to the tristate cooperation agreement for workforce development.
As another example, multiple API programs (such as Carnegie Science Center, the Children’s Museum of Pittsburgh, and the Intermediate Unit 1 Fab Lab) have been involved in efforts such as defining and developing the “STEM Ecosys-tem Community of Practice”30 and “the Maker Movement,” 31 which are national collaborative efforts, both at a regional level.
The API Initiated Networks Across Programs and Among Industry and Higher Education Partners K–12 STEM Programs Were More Connected with One Another Than Were Workforce Development ProgramsFigure 6 shows the relationships among API leaders and programs. K–12 STEM programs and Catalyst Connection (which conducted both K–12 and workforce development programs) reported consistent interaction and cooperation with one another: they knew of one another, met face-to-face, and/or shared promising practices. (It is important to note that lines between shapes signal that the two entities have a connection, as reported by the API program administrator; the lines do not signify the depth or length of that relationship.) In contrast, workforce development programs reported fewer connections with one another. While the four ShaleNet colleges collabo-rated regularly (in quarterly meetings), no other workforce development program worked in conjunction with others of its kind. Consequently, API leaders may wish to bring the work-force development program administrators together to share information or plan informal or formal collaborative activities. Nurturing the spontaneous formation of “ecosystems” would be one way to promote long-term sustainability of API programs.
Programs’ Networks Were Largely Isolated from One AnotherFigure 7 shows the relationships between API programs and other entities, such as businesses, government organizations, or civic associations.32 As the diagram shows, API programs
13
Figure 6. API Programs’ Connections to One Another
RAND RR2017-6
= API leadership
= K–12 API programs
= API workforce development programs
NaturalResources Course atGreeneCounty
SouthwestTrainingServices
EnergyInnovation
Center
WestmorelandCounty
CommunityCollege
PennsylvaniaCollege ofTechnology
StarkState
CollegePierpont
Communityand Technical
College
ProjectLead the Way
Children’sMuseum ofPittsburgh
Chevron
EducationAlliance
CatalystConnection
IntermediateUnit 1
CarnegieScienceCenter
Benedum
AlleghenyConference
14
are individually connected with many organizations, yet their networks were largely isolated from one another. Having better information exchanges and coordination among these networks could improve overall collaborative success. Some benefits of such deliberate coordination could include shared curriculum development, improved economies of scale, and collaborative activities toward common goals. The API could serve as a cen-tral hub for this type of cooperation.
Universities Served as Key Resources in These NetworksFigure 8 shows collaborations among API programs and higher-education institutions in the region, including public and private universities, as well as community or technical colleges. It shows that the largest number of higher-education
institutions that collaborated with API programs were located in Pennsylvania, followed by West Virginia and then Ohio. Carnegie Mellon University and the University of Pittsburgh were clearly central to the network, having had a significant number of collaborations with API programs. API leaders might reference this diagram in considering which relationships to create, develop, or strengthen with higher-education institu-tions, for example; to draw on for resources, knowledge, and convening capability; or to advise on curriculum development, offering courses for particular skills needed by industry, and teacher training.
Many Funders Contributed to API Programs, but Their Efforts Were Not CoordinatedFigure 9 shows the overall funding network for API programs, including the additional 26 funders outside of the API. (Since
Figure 7. API Programs’ Connections to External Partners
RAND RR2017-7
= K–12 API programs
= API workforce development programs
= non-API organizations
Southwest Training Services
Department of Labor WorkforceInvestment Boards
Energy Innovation Center
Stark State CollegeWestmoreland County
Community College
Pierpont Communityand Technical College
Natural Resources Courseat Greene County
Pennsylvania Collegeof Technology
ProjectLead the WayChildren’s
Museum ofPittsburgh
EducationAlliance
CatalystConnection
IntermediateUnit 1
Carnegie ScienceCenter
Remake LearningNetwork
Fab Foundation
LRNG
15
the completion of data collection for this report, the Grable Foundation joined API leadership. Future diagrams will include their contributions.) Similar to the relationships among API programs and other entities, this diagram shows a disconnected network of funders. Each API program had its own separate network of multiple funders. With few exceptions, apart from API leaders, other funders in this network rarely funded more than one API program, although they might fund programs outside of the API. Going forward, API leaders may consider how to strategically broaden, coordinate, and leverage the API funding base. Knowing where new networks may form, map-ping potential academic ties to those networks and identify-ing related philanthropic benefactors could support financial sustainability of API programs for years to come.
LOOKING AHEADThe API has set an ambitious, next-generation vision to improve the region’s energy and manufacturing education and employment ecosystem in support of broader economic devel-opment, both through investing in particular programs and through catalyzing a community of likeminded stakeholders to work toward these goals. This analysis reported on the vision, programs, beneficiaries, sustainability, and networks of the API during its first two years. However, as this is just the beginning of the API, this report did not aim to assess how effective the API has been or will be in reaching its long-term vision to sup-port the region’s economic development, and indeed, it would be too early to tell.
At this early stage in the API, we have found that the programs’ activities and goals were fairly well aligned with the API’s vision and were geographically reaching beneficiaries across the 27-county API footprint. Documenting these API strategies, advocacy, and community connections, as well as
Figure 8. API Programs’ Connections with Institutions of Higher Education
RAND RR2017-8
= K–12 API programs
= API workforce development programs
= higher-education institutions in Pennsylvania
= institutions in West Virginia
= Tri-State University Energy Alliance, which includes the University of Pittsburgh, Carnegie Mellon University, West Virginia University and Case Western Reserve University
Southwest Training Services
Marshall University
Point Part University
Community College of Allegheny County
Community College of Beaver County
Robert Morris University
Stark State CollegeWestmoreland County
Community College
Butler CountyCommunity College
WV NorthernCommunity College
Sinclair Community College
Pierpont Communityand Technical College
Natural Resources Courseat Greene County
Pennsylvania Collegeof Technology
ProjectLead the Way
Children’s Museumof Pittsburgh
Education Alliance
Wheeling Jesuit University
Waynesburg University
Penn State University
Carnegie Mellon University
University of Pittsburgh
TriState University Energy Alliance
Duquesne University
California University of PA
Washington & Jefferson
Bethany College
West Liberty University
CatalystConnection
IntermediateUnit 1
Carnegie Science Center
Case Western Reserve University
West Virginia University
Kent State University
University of Akron
Bucknell University
16
API programs’ services and network, provided a basis for distin-guishing successes in the API’s portfolio and spotting emerging gaps and opportunities for new directions. Such information will help API leaders, as well as other STEM education and workforce development stakeholders, determine where future resources and initiatives can best be deployed.
RECOMMENDATIONSLooking ahead, we offer the following recommendations orga-nized by the four research questions that this study considered.
Strategic Alignment• Continue to gauge participants’ perceptions and
awareness of STEM careers and API programs. Withthe collection of data through field-participant surveys ofteachers, students, and other program participants, the APIcould track the progress of its programs in meetings theirgoals with respect to changing awareness and perceptionsabout STEM education and careers.
• Map pathways between K–12 education and middle-skill jobs. Career pathways are models that show whichskills, degrees, and certifications can lead to specificSTEM-related careers and middle-skilled jobs. However,there are no regional career pathways that map the highschool coursework, badges, or skills that a student should
Figure 9. API Programs’ Leveraged Funding
RAND RR2017-9
= API leadership
= K–12 API programs
= API workforce development programs
= funders of each program, other than API funders
Southwest Training Services
Department of Labor WorkforceInvestment Boards
BelleJar Foundation
Pirates Charities
Dewey H. & Irene Moomaw Foundation
The Buhl Foundation
Stark State College
Westmoreland CountyCommunity College
Community Foundationfor the Ohio Valley
Association of Science &Technology Centers
Remake LearningNetwork
Pierpont Communityand Technical College
Natural Resources Courseat Greene County
Pennsylvania Collegeof Technology
ProjectLead the Way
Children’s Museumof Pittsburgh
Education Alliance
Benedum
Schenk Foundation
Chevron
The Pittsburgh Foundation
Noble Energy
Roberts Foundation
WV Departmentof Agriculture
Dominion Gas
WV Culture Grant
Kennametal Inc.
Latrobe Partners in Education Foundation
Catalyst Connection
IntermediateUnit 1
Carnegie Science Center
Energy InnovationCenter
Southwestern Energy
Tri-County Oil & Gas
The Ross Foundation
Walmart
Covestro
The Heinz Endowments
Sprout Fund
Alcoa Foundation
Haman Foundation
17
obtain for entry into postsecondary education, training programs, or middle-skill entry-level positions. These could be useful to support awareness and understanding among youth, parents, high school guidance counselors, and human resource professionals.
Beneficiaries and Geographic Scope• Fill programming gaps. Although the API has funded
multiple new experimental instructional models in K–12STEM education and workforce development, certainprogramming gaps, with respect to the goals articulated inthe API strategic logic model, merit attention. API leaderscould consider funding efforts for (1) a coordinated top-
down strategy to promote stakeholder awareness of STEM careers; (2) pre-service teacher training; (3) workforce development programs that support adult workers who are most in need of employment, particularly dislocated and transitioning workers; and (4) efforts that galvanize the stakeholder community to support local, state, regional, and federal policies that promote K–12 STEM education and workforce development.
Sustainability• Strengthen and expand networks of relationships with
regional funders. Programs relied on financial supportfrom 26 different funders (in addition to the API). The API
could serve as a strategic facilitator between these funders and its programs to more effectively finance initiatives.33
• Determine priority challenges for API to focus on solv-ing. API has sponsored multiple individual programs that address particular local needs and use innovative models to support skills acquisition and awareness. Yet, a regional analysis of the most significant challenges facing K–12 STEM education and workforce development is missing. The API is now well positioned to develop such a strategic high-level vision, which would be useful for policymakers as well as regional funders. For example,
− K–12 STEM program administrators reported that
attrition of teachers and school leaders undermine
their effectiveness. New teachers and instruc-
tors must be retrained, since they lack pre-service
professional development for the activity-based or
problem-based models. Here, a priority may be to
support university programs that might offer pre-
service teacher training on these types of instruc-
tional practices.34
− Programs encountered challenges to their sustain-
ability at the policy level. API leaders could consider
which of these challenges they might be able to
influence through funding or policy engagement.
An analysis of challenges, as well as prioritization of
the main problems to solve, could inform what the
API funds or where to direct community catalyst
efforts in the future.
Community Catalyst• Sponsor tristate meetings for K–12 STEM education.
During the annual Tri-State Shale Summit meetings, the API could convene a meeting for policymakers, philan-thropic foundations, and education stakeholders to pro-mote and coordinate K–12 STEM education policies, data sharing, and outcomes. Enhancing information exchanges and coordination among the K–12 STEM networks could foster collaboration; enhance connections among pro-grams, community leaders, and policymakers; and support the development of a coherent vision for K–12 STEM education in the region.
• Focus on deepening connections among workforce development programs. Our network analyses revealed less collaboration among workforce development programs than among their K–12 STEM counterparts. While API programs individually had many connections, their net-works were largely isolated from one another, limiting pros-pects for sharing promising practices. Better information exchanges and coordination among the networks could improve individual and collaborative success. For example, as a start, the API could convene a workshop among its workforce development programs to share lessons learned and discuss possible synergies.
• Craft a broad strategy for policy engagement to support tristate education and workforce development. Our analyses found that, to date, API leaders and programs have engaged networks of community leaders and funding sources. Some program administrators reported individu-ally engaging with policymakers at the school district, city, county, state, or federal level about their program’s needs. Now that the API has established a track record during its beginning years, it is well positioned to develop and advocate for strategic policy goals at the state, city, or district level for K–12 STEM education and workforce development.
18
ACKNOWLEDGMENTSWe are grateful for the generous support of the Appalachia Partnership Initiative (API) partners, including Chevron Corporation, the Claude Worthington Benedum Foundation, the Allegheny Conference on Community Development, and the Grable Foundation. In particular, we thank the following individuals from these organizations who have contributed their leadership to these efforts, as well as inputs to the evalu-ation plans: Judith Dunbar, Mary Murrin, Trip Oliver, Lau-rie Serwinski, and Lee Ann Wainwright at Chevron; James Denova at the Claude Worthington Benedum Foundation; Linda Topoleski and Laura Fisher at the Allegheny Conference for Community Development; and D’Ann Swanson and Gregg Behr at the Grable Foundation. We also thank representatives from the following API-funded science, technology, engineer-ing, and mathematics (STEM) and workforce development programs for their help in collecting program data through
their participation in interviews and by sharing program docu-mentation: the Children’s Museum of Pittsburgh; Project Lead the Way; the Intermediate Unit 1 Fab Lab; Carnegie Science Center’s Fab Lab and Grand Slam Science; ShaleNET; Catalyst Connections; Green County; and Southwest Training Services.
Finally, we would like to thank RAND colleagues who contributed to this report. Catherine Augustine provided a quality-assurance review for the RAND Education unit, reading early drafts of this report and sharing important commentary. Harold Greene conducted the network diagram analysis. Serafina Lanna contributed to the literature review. Katherine Giglio and Maria McCollister helped with editing, formatting, and communication of results. We would also like to thank our objective peer reviewers, Peter Morrison and Joie Acosta, for their insights and suggested changes, which greatly improved the report. Any errors within are the respon-sibility of the authors alone.
19
NOTES1 For discussion of economic and employment history and current key indicators and outcomes in the tristate Appalachia region, see Center for Regional Economic Competitiveness and West Virginia University, Appalachia Then and Now: Examining Changes in the Region Since 1965, Washington, D.C.: Appalachian Regional Commission, February 2015. As of December 23, 2015: http://www.arc.gov/assets/research_reports/AppalachiaThenAndNowCompiledReports.pdf; Gabriella C. Gonzalez, Kyle Siler-Evans, Gerald Paul Hunter, and Matthew D. Baird, Wages, Employment, and STEM Education in Ohio, Pennsylvania, and West Virginia: Report No. 1 (2016), Santa Monica, Calif.: RAND Corporation, RR-1357-CHC, 2016. As of August 9, 2017: https://www.rand.org/pubs/research_reports/RR1357.html; Gabriella C. Gonzalez, Kyle Siler-Evans, Gerald Paul Hunter, and Nicholas Broten, Wages, Employment, and STEM Education in Ohio, Pennsylvania, and West Virginia: Report No. 2 (2017), Santa Monica, Calif.: RAND Corporation, RR-1863-CHC, 2016. As of August 9, 2017: https://www.rand.org/pubs/research_reports/RR1863.html; and Kelvin Pollard and Linda A. Jacobsen, “The Appala-chian Region: A Data Overview from the 2006–2010 American Com-munity Survey,” Population Reference Bureau, February 2012.
2 U.S. Energy Information Administration, “Marcellus, Utica Provide 85% of U.S. Shale Gas Production Growth Since Start of 2012,” July 28, 2015. As of March 4, 2016: https://www.eia.gov/todayinenergy/detail.cfm?id=22252
3 IHS, America’s New Future: The Unconventional Oil and Gas Revolu-tion and the U.S. Economy, Vol. 3: A Manufacturing Renaissance—Main Report, Engelwood, Colo., September 2013. As of March 4, 2016: http://www.energyxxi.org/sites/default/files/pdf/Americas_New_Energy_Future_Phase3.pdf
4 Michael E. Porter, David S. Gee, and Gregory J. Pope, America’s Unconventional Energy Opportunity: A Win-Win Plan for the Economy, the Environment, and a Lower-Carbon, Cleaner-Energy Future, Cambridge, Mass.: Harvard Business School and Boston Consulting Group, June 2015. As of March 4, 2016: http://www.hbs.edu/competitiveness/Documents/america-unconventional-energy-opportunity.pdf
5 Burning Glass Technologies, the Council for Adult and Experiential Learning, Allegheny Conference on Community Development, Inflec-tion Point: Supply, Demand, and the Future of Work in the Pittsburgh Region, 2016. As of November 18, 2016: http://www.alleghenyconference.org/wp-content/uploads/2016/08/InflectionPoint.pdf; Accenture, Burning Glass Technologies, and Harvard Business School, Bridge the Gap: Rebuilding America’s Middle Skills, Cambridge, Mass.: U.S. Competitiveness Project, Harvard Business School, November 2014. As of March 4, 2016: http://www.hbs.edu/competitiveness/Documents/bridge-the-gap.pdf; and Gabriella C. Gonzalez, Sean Robson, Andrea Phillips, Gerald Hunter, and David S. Ortiz, Energy-Sector Workforce Development in West Virginia: Aligning Community College Education and Training with Needed Skills, Santa Monica, Calif.: RAND Corporation, RR-812-NETL, 2015. As of March 4, 2016: http://www.rand.org/pubs/research_reports/RR812.html
6 Iryna Lendel, Iryna, Andrew R. Thomas, Bryan Townley, Thomas Murphy, and Ken Kalynchuk, Economics of Utica Shale in Ohio: Workforce Analysis, Cleveland, Ohio: Center for Economic Develop-ment and Energy Policy Center, Cleveland State University, Paper 1330, September 2015. As of March 4, 2016: http://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?article=2330&context=urban_facpub; Center for Energy Workforce Development, “2014 Strategic Goals,” Washington, D.C., 2013. As of March 4, 2016: http://www.cewd.org/mem_resources/2014-CEWD-Goals.docx; Anthony P. Carnevale, Nicole Smith, and Michelle Melton, STEM: Science, Tech-nology, Engineering, Mathematics, Washington, D.C.: Georgetown University Center on Education and the Workforce, October 2011. As of March 4, 2016: https://cew.georgetown.edu/report/stem
7 Porter, Gee, and Pope, 2015.
8 Campos Research Strategy, Work to Do: The Role of STEM Educa-tion in Improving the Tri-State Region’s Workforce, Pittsburgh, Pa.: Carnegie Science Center, 2014. As of March 4, 2016: http://www.carnegiesciencecenter.org/csc_content/stemcenter/pdf/Work_to_Do_The_Role_of_STEM_Education_in_Improving_the_Tri-State_Regions_Workforce.pdf
9 Campos Research Strategy, 2014, p. 4.
10 For more information about the use of logic models in program evaluation, see Gretchen Jordan, “A Theory-Based Logic Model for Innovation Policy and Evaluation,” Research Evaluation, Vol. 19, No. 4, 2010, pp. 263–274; John A. McLaughlin and Gretchen B. Jordan, “Logic Models: A Tool for Telling Your Program’s Perfor-mance Story,” Evaluation and Program Planning, Vol. 22, No. 1, 1999, pp. 65–72; W.K. Kellogg Foundation, Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide, Battle Creek, Mich., 2004. As of March 4, 2016: https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide
11 For more information about how the strategic logic model and study questions were developed, see Appendix B in the accompany-ing Technical Appendix (Shelly Culbertson, Gabriella C. Gonzalez, and Nupur Nanda, Launching the Appalachia Partnership Initiative: Investing in K–12 STEM Education and Workforce Development in Energy and Advanced Manufacturing: Technical Appendix, Santa Monica, Calif.: RAND Corporation, RR-2017/1-CHC, 2017. As of September 11, 2017: https://www.rand.org/pubs/research_reports/RR2017z1.html ).
12 Details of all programs participating in the API from API inception in 2013 to 2016 can be found in Appendix A of the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
13 For more information about how these questions were developed, see Appendix B in the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
14 More information about the data sources can be found in Appendix B in the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
15 A copy of the interview questionnaire is available in Appendix G in the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
16 The quantitative indicators were developed in July 2015 in delibera-tion with API leaders; each indicator linked directly to a component of the strategic logic model. At this point in time, program admin-istrators were able to report on the direct services provided by their programs and number of participants (“outputs” in the strategic logic model). A sample data collection template is available in Appendix H of the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
17 The API leader questionnaire is available in Appendix I in the accompanying Technical Appendix.
18 The static map is included in the accompanying Technical Appen-dix (Culbertson, Gonzalez, and Nanda, 2017), and online interactive maps will be updated annually to correspond with RAND’s analysis and are available on the API website: http://www.rand.org/appalachiapartnership.html
19 Interactive maps with specific information about the API and the programs in which it is investing can be found on http://appalachiapartnership.com. For more information about the methodology employed, see Appendix B in the accompanying Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
20 John H. Reed and Gretchen Jordan, “Using Systems Theory and Logic Models to Define Integrated Outcomes and Performance Mea-sures in Multi-Program Settings.” Research Evaluation, Vol. 16, No. 3, 2007, pp. 169–181.
21 This number may be higher; only two workforce development pro-grams counted veterans or displaced coalminers. Service to Oppor-tunity, an API program that matched veterans with middle-skill jobs in the oil and gas sector, was not included in the our study because it had stopped operations in order to modify its design and format. While it was operating, 722 veterans registered for its job matching services. We will include this program in subsequent analyses.
22 For more information on the data and methods used in the analysis and detailed findings, of API alignment, see Appendix C in the Tech-nical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
23 For more information on the data and methods used in this analy-sis, and detailed findings, on beneficiaries and geographic reach of API programs, see Appendix D in the Technical Appendix.
24 Great Schools, “Ohio School District,” undated. As of July 26, 2017: http://www.greatschools.org/schools/districts/Ohio/OH
25 Pennsylvania Department of Education, “Educational Names and Addresses,” undated. As of July 26, 2017: http://www.edna.ed.state.pa.us/Screens/wfSearchEntity.aspx
26 West Virginia Department of Education, “WV School Directory,” undated. As of July 26, 2017: https://wvde.state.wv.us/ed_directory/
27 Organization for Economic Cooperation and Development (OECD), DAC Criteria for Evaluating Development Assistance, 2017. As of March 24, 2016: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm
28 For more information on the data and methods used in the analysis and detailed findings of program sustainability, see Appendix E in the Technical Appendix (Culbertson, Gonzalez, and Nanda, 2017).
29 Peter DeLeon and Danielle M. Varda, “Toward a Theory of Col-laborative Policy Networks: Identifying Structural Tendencies,” Policy Studies Journal, Vol. 37, No. 1, February 2009, pp. 59–74; Danielle M. Varda, Anita Chandra, Stefanie A. Stern, and Nicole Lurie, “Core Dimensions of Connectivity in Public Health Collaboratives,” Journal of Public Health Management and Practice, Vol. 14, No.5, September 2008, pp. E1–E7; Laurie E. Paarlberg, and Danielle M. Varda, “Com-munity Carrying Capacity A Network Perspective,” Nonprofit and Voluntary Sector Quarterly, Vol. 38, No. 4, August 2009, pp. 597–613; Danielle M. Varda, “A Network Perspective on State-Society Synergy to Increase Community-level Social Capital,” Nonprofit and Voluntary Sector Quarterly, Vol. 4, No. 5, August 2010, pp. 896–923; and Jessica H. Retrum, Carrie L. Chapman, and Danielle M. Varda, “Implications of Network Structure on Public Health Collabora-tives,” Health Education & Behavior, Vol. 40, No. 1 suppl., October 2013, pp. 13S–23S.
30 For more information, see STEM Ecosystems, “Community of Practice,” undated. As of July 27, 2017: http://stemecosystems.org/first-community-of-practice. Fifty-four STEM Learning Ecosystems across the United States make up the Community of Practice. In the tristate region, there are two STEM Learning Ecosystems: in Pitts-burgh, Pennsylvania, and in Clairsville, Ohio.
31 The nationwide Maker Movement gained traction through the launch of MAKE Magazine in 2005 by Maker Media in San Fran-cisco. The first Maker Faire hosted by Maker Media was in 2006 in the Bay Area. Since then, numerous Maker Faires have been orga-nized throughout the world, including (1) Flagship Maker Faires (“faires curated and produced by the Maker Media team”), (2) Fea-tured Maker Faires (“larger-scale regional events”), (3) Mini Maker Faires (“community events”), and (4) School Maker Faires (“K–12 Faires [closed to the general public]).” Maker Faires display success-ful do-it-yourself projects and innovations in any field (e.g., science, art, engineering, performance). For more information, see Maker Faire, “A Bit of History,” 2016. As of December 14, 2016: http://makerfaire.com/makerfairehistory; and Maker Faire, “The Maker Movement,” 2016. As of December 14, 2016: http://makerfaire.com/maker-movement/
32 The ShaleNet colleges’ connections are not shown in this figure. At the time of this study, three of the four colleges had not provided these data to us.
33 In October 2016 (after the time of this study), Chevron Corpora-tion, Benedum Foundation, and Grable Foundation convened a meeting with corporate funding community members with interests related to education and the workforce; this was the first time that such a group had met. The meeting facilitated an initial exchange about interests and led to discussions about possibilities for collabora-tion. This nascent initiative might lead to coordinated funding and leadership action going forward and could be used as a blueprint for engaging other potential funders in the region.
34 It is important to note that, collectively, the API K–12 STEM programs trained more than 700 teachers at the time of this study. Therefore, although individual programs might experience attrition, it is unknown whether those teachers have applied their knowledge of applied, project-based instruction to other locations or to support other students. This movement, therefore, might have been positive for the entire region.
22
The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest.
RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. R® is a registered trademark.
The research was conducted within two units of the RAND Corporation: RAND Education and RAND Justice, Infrastructure, and Environment. The Technical Appendix associated with this report is available at https://www.rand.org/pubs/research_reports/RR2017z1.html, documents the study’s methodology and data and expands on the findings summarized in this report.
This research was sponsored by the Appalachia Partnership Initiative (API). This report should interest two audiences: (1) regional education, business, and community leaders concerned with science, technology, engineering, and mathemat-ics (STEM) education and the career readiness of workers in the energy and advanced manufacturing sectors; and (2) policy analysts interested in how network analyses can help to advance regional innovation.
The Appalachia Partnership InitiativeChevron North America Exploration and Production (CNAEP) Appalachian Moun-tain Business Unit’s Social Investment Team was established to meet several goals in the Pennsylvania, West Virginia, and Ohio region. Goals include strengthening science, technology, engineering, and mathematics (STEM) education in middle and high schools and improving pathways for high school graduates and adult learners to careers in the oil and gas industries and in advanced manufacturing. As part of these efforts, in 2014, Chevron’s Social Investment Team launched the Appalachia Partnership Initiative (API), a partnership of businesses, nonprofit organizations, and education institutions in the region. As of April 2017, the API consisted of representatives from Chevron, the Claude Worthington Benedum Foundation, the Grable Foundation, the Allegheny Conference for Community Development, and Catalyst Connections.
The RAND Corporation has served continuously as the independent research and analysis lead for the API. RAND is undertaking annual assessments from 2016 through 2020 to track the progress that API-sponsored programs are making in supporting the API’s goals over time.
About the AuthorsGabriella C. Gonzalez is a senior sociologist at RAND. Her research focuses on career and technical education, work-force development, school-to-work transitions, and program evaluation. She is the RAND lead of the API.Shelly Culbertson is a senior policy researcher at RAND. Her research focuses on education, workforce development, innovation policy, and international development.Nupur Nanda is a research assistant at RAND.
Limited Print and Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of our research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html.
For more information on this publication, visit www.rand.org/t/RR2017.
