Tennessee STEM Innovation Network: Final Evaluation Report · Tennessee Consortium on Research,...

CARLA C. JOHNSON

AUGUST 2014

Tennessee STEM Innovation Network:Final Evaluation Report

Tennessee Consortium on Research, Evaluation, & Development

Peabody #44 | 230 Appleton Place | Nashville, Tennessee 37203 Phone 615-322-5538 | Fax 615-322-6018

www.tnconsortium.org

The Tennessee Consortium on Research, Evaluation, and Development is a research and policy center at Peabody College of Vanderbilt University’s Peabody College. The Consortium is funded through the State of Tennessee’s Race to the Top grant from the United States Department of Education (grant #S395A100032).

Please visit www.tnconsortium.org to learn more about our program of research and recent publications.

This study was supported by the Tennessee Consortium on Research, Evaluation, and Development (the Consortium) at Vanderbilt University’s Peabody College, which is funded by the State of Tennessee’s Race to the Top grant from the United States Department of Education (grant #S395A100032). Any errors remain the sole responsibility of the author. The views expressed in this paper do not necessarily reflect those of sponsoring agencies or individuals acknowledged.

Direct all correspondence to Carla C. Johnson, Purdue University, 100 N. University Street, West Lafayette, Indiana 47907 ([email protected]).

Tennessee STEM Innovation Network Evaluation Annual Report|i

TABLE OF CONTENTS Contents ........................................................................................................................................................... i Figures ............................................................................................................................................................ vi Tables ............................................................................................................................................................. vii Executive Summary .................................................................................................................................... ix SECTION ONE Introduction ................................................................................................................................................... 1 Background ...................................................................................................................................................... 1 SECTION TWO Research Methods ........................................................................................................................................ 3 Research Questions ......................................................................................................................................... 3 Conceptual Framework .................................................................................................................................. 4 Data Collection and Analysis ........................................................................................................................ 5

STEM Community Awareness Survey (TSIN/Hub Impact Survey) ........................................ 5 Response Rate – STEM Community Awareness Survey (SCAS) ............................................ 7 Analysis of STEM Community Awareness Survey ................................................................. .7

Teacher Effectiveness Data ............................................................................................................. 8 Classroom Observation Instrument ........................................................................................... 8 Response Rate - Teacher Observation Data ............................................................................ 10 Analysis of Teacher Observation Data ................................................................................... 10

Focus Group Interviews and Individual Interviews .................................................................. 11 Analysis of STEM School Curriculum and Site Visits …………………………………...11 Limitations…………………………………………………………………………….... 14

SECTION THREE Findings Overall for TSIN Investment – Research Question 3 ................................................... 15 Sustainability of the TSIN ............................................................................................................................ 15 SECTION FOUR Findings Overall for TSIN Investment – Research Question 4 ................................................... 17 Connectivity of Network ............................................................................................................................. 17 SECTION FIVE Findings Overall for TSIN Investment – Research Question 6 …………………………….19 Classroom Observation Findings……………………………………………………………….. 19 Design Lesson .................................................................................................................................. 19 Implementation of Lesson ............................................................................................................. 21 Classroom Culture ........................................................................................................................... 23 Mathematics/Science Content Domain ....................................................................................... 25

Tennessee STEM Innovation Network Evaluation Annual Report|ii

SECTION SIX Findings Overall for TSIN – Research Question 8 .......................................................................... 28 Student Opportunities in STEM ................................................................................................................ 28 SECTION SEVEN Findings Overall for TSIN – Research Question 9 ………………………………………….30 STEM Community Awareness Survey (SCAS)………………………………………………….. 30

Change in Views of STEM Awareness and Support……………………………………..30 SECTION EIGHT Narratives for TSIN Regional STEM Innovation Hubs and STEM Platform Schools …...32 Chattanooga: Southeast Tennessee STEM Innovation Hub (SETN) ……………………………33 Hub and School Setting………………………………………………………………… 34 Research Question 1……………………………………………………………………. 34 Research Question 2……………………………………………………………………. 35 Research Question 5……………………………………………………………………. 36 Research Question 7……………………………………………………………………. 37

Project-based Learning with Integrated Content (A1) …………………………………..38 Connections to STEM Programs (A2)………………………………………………. 39 Integration of Technology/Virtual Learning (A3)……………………………………...40 Authentic Assessment (A4) ………………………………………………………...40 Professional Development (A5) ……………………………………………………...40 Outreach, Support, and Focus on Underserved Students (A6) ................................................. 41 STEM Work-based Learning Experiences (B1)………………………………………41 Participation in STEM Network (B2)……………………………………………….42 STEM Plan (B3) …………………………………………………………………42 Alignment of Career Pathways with Post-secondary (C1) ………………………………..42 Post-secondary Credit Completion (C2) ……………………………………………….43

Research Question 9 …………………………………………………………………….44 Concluding Observations………………………………………………………………..49

Cookeville: Upper Cumberland Rural STEM Initiative ………………………………………….51 Hub and School Setting …………………………………………………………………51 Research Question 1……………………………………………………………………. 52 Research Question 2 …………………………………………………………………….53 Research Question 5 …………………………………………………………………….54 Research Question 7 …………………………………………………………………….55

PSES: Project-based Learning with Integrated Content (A1)……………………………56 PSES: Connections to STEM Programs (A2) ………………………………………..56 PSES: Integration of Technology/Virtual Learning (A3)………………………………57 PSES: Authentic Assessment (A4)…………………………………………………. 57 PSES: Professional Development (A5)……………………………………………….58 PSES: Outreach, Support, and Focus on Underserved Students (A6) .................................... 58 PSES: STEM Work-based Learning Experiences (B1) ……………………………….58 PSES: Participation in STEM Network (B2) ………………………………………..59 PSES: STEM Program Plan (B3) ………………………………………………….59 PSES: Alignment of Career Pathways with Post-secondary (C1) ………………………..59 PSES: Post-secondary Credit Completion (C2) ………………………………………..60 PSMS: Project-based Learning with Integrated Content (A1) …………………………...60

Tennessee STEM Innovation Network Evaluation Annual Report|iii

PSMS: Connections to STEM Programs (A2)………………………………………..61 PSMS: Integration of Technology/Virtual Learning (A3) ……………………………...61 PSMS: Authentic Assessment (A4) …………………………………………………62 PSMS: Professional Development (A5) ………………………………………………62 PSMS: Outreach, Support, and Focus on Underserved Students (A6) .................................... 62 PSMS: STEM Work-based Learning Experiences (B1) ………………………………63 PSMS: Participation in STEM Network (B2) ………………………………………63 PSMS: STEM Program Plan (B3) …………………………………………………63 PSMS: Alignment of Career Pathways with Post-secondary (C1) ………………………..64 PSMS: Post-secondary Credit Completion (C2) ………………………………………..64

Research Question 9 …………………………………………………………………….64 Concluding Observations………………………………………………………………..70

Johnson City: ETSU Northeast STEM Innovation Hub ………………………………………...72 Hub and School Setting …………………………………………………………………72 Research Question 1……………………………………………………………………. 73 Research Question 2 …………………………………………………………………….74 Research Question 5 …………………………………………………………………….75 Research Question 7 …………………………………………………………………….76

Project-based Learning with Integrated Content (A1) …………………………………..76 Connection to STEM Programs (A2) ………………………………………………..77 Integration of Technology/Virtual Learning (A3)……………………………………...78 Authentic Assessment (A4) ………………………………………………………78 Professional Development (A5) ……………………………………………………78 Outreach, Support, and Focus on Underserved Students (A6) ................................................. 79 STEM Work-based Learning Experiences (B1)………………………………………79 Participation in STEM Network (B2) ………………………………………………80 STEM Program Plan (B3)…………………………………………………………80 Alignment of Career Pathways with Post-secondary (C1) ………………………………..81 Post-secondary Credit Completion (C2) ……………………………………………….81

Research Question 9……………………………………………………………………. 81 Concluding Observations ............................................................................................................... 87

Knoxville: STEMSpark East Tennessee STEM Innovation Hub ......................................................... 89 Hub and School Setting .................................................................................................................. 89 Research Question 1 ....................................................................................................................... 90 Research Question 2 ....................................................................................................................... 91 Research Question 5 ....................................................................................................................... 92 Research Question 7 ....................................................................................................................... 92

Project-based Learning with Integrated Content (A1) .............................................................. 93 Connections to STEM Programs (A2) ................................................................................... 94 Integration of Technology/Virtual Learning (A3) .................................................................. 95 Authentic Assessment (A4) ................................................................................................... 95 Professional Development (A5) ............................................................................................... 96 Outreach, Support, and Focus on Underserved Students (A6) ................................................. 96 STEM Work-based Learning Experiences (B1) .................................................................... 97 Participation in STEM Network (B2) .................................................................................. 97 STEM Program Plan (B3) .................................................................................................... 98 Alignment of Career Pathways with Post-secondary (C1) ........................................................ 98 Post-secondary Credit Completion (C2) ................................................................................... 99

Tennessee STEM Innovation Network Evaluation Annual Report|iv

Research Question 9 ....................................................................................................................... 99 Concluding Observations ............................................................................................................. 104

Memphis: West Tennessee STEM Hub (WTSH) .................................................................................. 106 Hub and School Setting ................................................................................................................ 106 Research Question 1 ..................................................................................................................... 107 Research Question 2 ..................................................................................................................... 107 Research Question 5 ..................................................................................................................... 108 Research Question 7 ..................................................................................................................... 108

Project-based Learning with Integrated Content (A1) ............................................................ 109 Connections to STEM Program (A2) .................................................................................. 109 Integration of Technology/Virtual Learning (A3) ................................................................ 110 Authentic Assessment (A4) ................................................................................................. 110 Professional Development (A5) ............................................................................................. 110 Outreach, Support, and Focus on Underserved Students (A6) ............................................... 110 STEM Work-based Learning Experiences (B1) .................................................................. 110 Participation in STEM Network (B2) ................................................................................ 111 STEM Plan (B3) ................................................................................................................ 111 Alignment of Career Pathways with Post-secondary (C1) ...................................................... 111

Post-secondary Credit Completion (C2) .................................................................................. 111 Research Question 9 ..................................................................................................................... 111 Concluding Observations ............................................................................................................. 115

Nashville: Middle Tennessee STEM Innovation Hub (MTSI) ............................................................ 117 Hub and School Setting ................................................................................................................ 117 Research Question 1 ..................................................................................................................... 119 Research Question 2 ..................................................................................................................... 119 Research Question 5 ..................................................................................................................... 120 Research Question 7 ..................................................................................................................... 121

Project-based Learning with Integrated Content (A1) ............................................................ 121 Connections to STEM Program (A2) .................................................................................. 122 Integration of Technology/Virtual Learning (A3) ................................................................ 122 Authentic Assessment (A4) ................................................................................................. 123 Professional Development (A5)……………………………………………………..123 Outreach, Support, and Focus on Underserved Students (A6) ............................................... 123 STEM Work-based Learning Experiences (B1) .................................................................. 124 Participation in STEM Network (B2) ................................................................................ 124 STEM Plan (B3) ................................................................................................................ 125 Alignment of Career Pathways with Post-secondary (C1) ...................................................... 125

Post-secondary Credit Completion (C2) .................................................................................. 125 Research Question 9 ..................................................................................................................... 126 Concluding Observations ............................................................................................................. 130

SECTION NINE Concluding Observations ...................................................................................................................... 132 Sustainability of TSIN ................................................................................................................................ 132 Connectivity of Network ........................................................................................................................... 133 Growth in Teacher Effectiveness ............................................................................................................. 133 Growth in Student STEM Opportunities ............................................................................................... 134 Growth in STEM Stakeholder Awareness .............................................................................................. 134

Tennessee STEM Innovation Network Evaluation Annual Report|v

TSIN Innovations ....................................................................................................................................... 134 Summary ....................................................................................................................................................... 136 References ........................................................................................................................... 137 Appendix ............................................................................................................................. 138 Appendix A .................................................................................................................................................. 139 Appendix B .................................................................................................................................................. 140 Appendix C .................................................................................................................................................. 141 Appendix D .................................................................................................................................................. 142

Tennessee STEM Innovation Network Evaluation Annual Report|vi

LIST OF FIGURES Figure 1. Tennessee STEM Innovation Network (TSIN) Innovations ............................................... 18 Figure 2. Design of Lesson Average Score Over Time for Round One Programs ............................ 20 Figure 3. Design of Lesson Average Score Over Time by Program Classification for Round One

Programs ........................................................................................................................................ 21 Figure 4. Implementation of Lesson Average Score Over Time for Round One Programs ............ 22 Figure 5. Implementation of Lesson Average Score Over Time by Type for Round One

Programs ........................................................................................................................................ 23 Figure 6. Classroom Culture Domain Average Score Over Time for Round One Programs .......... 24 Figure 7. Classroom Culture Domain Average Score Over Time by Program Classification for

Round One Programs ................................................................................................................. 25 Figure 8. Mathematics/Science Content Domain Average Score Over Time for Round One

Programs ........................................................................................................................................ 26 Figure 9. Mathematics/Science Content Domain Average Score Over Time by Program

Classification for Round One Programs .................................................................................. 27

Tennessee STEM Innovation Network Evaluation Annual Report|vii

LIST OF TABLES Table ES1. LSC Overall Rating .................................................................................................................. xiii Table ES2. Classroom Observation Findings – all THEC STEM PD Programs ............................. xiv Table ES3. SCAS Overall and Construct Comparison Baseline (Fall 2012) to End of Year Two (Spring 2014) …….……………………………………………………………………… . xv Table 1. TSIN Hubs and Platform Schools(s) Included in Evaluation ................................................. 2 Table 2. SCAS Constructs and Items .......................................................................................................... 6 Table 3. Demographic Frequencies and Percentages: Baseline and Follow-up SCAS Survey

Samples…………………………………………………………………………………….8 Table 4. LSC Domains ................................................................................................................................... 9 Table 5. LSC Overall Rating .......................................................................................................................... 9 Table 6. STEM Attribute Implementation Rubric Category 1: Integrated STEM Curriculum

Aligned with State, National, International and Industry Standards (with Subcategory (A1-A6) and Associated Items) …………………………………………………………….......12

Table 7. STEM Attribute Implementation Rubric Category 2: Ongoing Community and Industry Engagement (with Subcategory (B1-B3) and Associated Items) ….……............………..13 Table 8. STEM Attribute Implementation Rubric Category 3: Connections with Post-secondary

Education (with Subcategory (C1-C2) and Associated Items)…………………...….……...13 Table 9. Percentage Agreement by Stakeholder Group Increasing Student STEM Opportunities in Tennessee .............................................................................................................. 28 Table 10. Percentage Agreement by Regional STEM Innovation Hub Increasing Student STEM Opportunities in Tennessee ................................................................................... 29 Table 11. SCAS Overall and Construct Comparison from Baseline (Fall 2012) to End of Year Two

(Spring 2014) ...……………..………………………………………………………..……31 Table 12. STEM School Chattanooga STEM Attributes Ratings ......................................................... 38 Table 13. K-12 Educators Areas of Growth in Agreement on SCAS for SETN Hub ...................... 45 Table 14. K-12 Educators Areas of Decline in Agreement on SCAS for SETN Hub……………46 Table 15. Business/Community Stakeholder Areas of Growth in Agreement on SCAS at Baseline

for SETN Hub .................................................................................................................................... 46 Table 16. Business/Community Stakeholder Areas of Decline in Agreement on SCAS at Baseline

for SETN Hub .................................................................................................................................... 47 Table 17. Higher Education Faculty Areas of Growth in Agreement on SCAS for SETN Hub …48 Table 18. Higher Education Faculty Areas of Decline in Agreement on SCAS for SETN Hub…49 Table 19. Prescott South Elementary School STEM Attributes Ratings …………………………55 Table 20. Prescott South Middle School STEM Attributes Ratings ……………………………...60 Table 21. K-12 Educators Areas of Growth in Agreement on SCAS for UCRI Hub……………66 Table 22. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for UCRI

Hub ……………………………………………………………………………………….68 Table 23. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for UCRI

Hub …….. ……………….………………………………………………………………68

Tennessee STEM Innovation Network Evaluation Annual Report|viii

Table 24. Higher Education Faculty Areas of Growth in Agreement on SCAS for UCRI Hub ….69 Table 25. Higher Education Faculty Areas of Decline in Agreement on SCAS for UCRI Hub…70 Table 26. Innovation Academy STEM Attributes Ratings ……………………………………….76 Table 27. K-12 Educators Areas of Growth in Agreement on SCAS for ETSU Hub……………83 Table 28. K-12 Educators Areas of Decline in Agreement on SCAS for ETSU Hub ……………84 Table 29. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for ETSU

Hub……………… ………………………………………………………………………84 Table 30. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for ETSU

Hub ……………………………………………………………………………………... 85 Table 31. Higher Education Faculty Areas of Growth in Agreement on SCAS for ETSU Hub .... 86 Table 32. Higher Education Faculty Areas of Decline in Agreement on SCAS for ETSU Hub ..... 87 Table 33. L&N STEM Academy STEM Attributes Ratings .................................................................. 93 Table 34. K-12 Educators Areas of Growth in Agreement on SCAS for STEMSpark Hub ......... 100 Table 35. K-12 Educators Areas of Decline in Agreement on SCAS for STEMSpark Hub .......... 101 Table 36. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for

STEMSpark Hub ………………………………………………………………………. 102 Table 37. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for

STEMSpark Hub………………………………………………………………………... 103 Table 38. Higher Education Faculty Areas of Growth in Agreement on SCAS for STEMSpark

Hub …………………………………………………………………………………….. 103 Table 39. Higher Education Faculty Areas of Decline in Agreement on SCAS for STEMSpark

Hub ..................................................................................................................................................... 104 Table 40. Southwind STEM High School STEM Attributes Ratings ................................................. 109 Table 41. K-12 Educators Areas of Growth in Agreement on SCAS for WTSC Hub ................... 112 Table 42. K-12 Educators Areas of Decline in Agreement on SCAS for WTSC Hub .................... 113 Table 43. Business/Community Stakeholder Areas of Growth in Agreement on SCA for WTSC

Hub ..................................................................................................................................................... 113 Table 44. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for WTSC

Hub ..................................................................................................................................................... 113 Table 45. Higher Education Faculty Areas of Growth in Agreement on SCAS for WTSC Hub .. 114 Table 46. Higher Education Faculty Areas of Decline in Agreement on SCAS for WTSC Hub .. 115 Table 47. Stratford STEM Magnet High School STEM Attributes Ratings ..................................... 121 Table 48. K-12 Educators Areas of Growth in Agreement on SCAS for MTSI Hub ..................... 127 Table 49. K-12 Educators Areas of Decline in Agreement on SCAS for MTSI Hub ..................... 128 Table 50. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for MTSI

Hub ..................................................................................................................................................... 128 Table 51. Higher Education Faculty Areas of Growth in Agreement on SCAS for MTSI Hub ... 129 Table 52. Higher Education Faculty Areas of Decline in Agreement on SCAS for MTSI Hub .... 130

burns

Cross-Out

Tennessee STEM Innovation Network Evaluation Annual Report|ix

EXECUTIVE SUMMARY The Tennessee STEM Innovation Network (TSIN) was established through Executive Order of the Governor of Tennessee in 2011. The TSIN is a public-private collaboration of the Tennessee Department of Education and the Battelle Memorial Institute. The purpose of the TSIN is to “promote and expand the teaching and learning of science, technology, engineering, and mathematics (STEM) education in K-12 public schools across Tennessee.” The TSIN is comprised of six regional STEM innovation hubs and seven STEM platform schools. The work of the TSIN is structured to support the implementation of the Tennessee STEM Strategic Plan, which includes four broad goals and supporting strategies designed to move Tennessee forward as a “national leader in STEM talent production and economic prosperity” (p. 3). The research questions, which guided the evaluation of the TSIN, include:

1. Does the TSIN model create strong, sustained partnerships within and across regional Hubs? (structural)

2. What motivates stakeholder engagement in TSIN? (structural) 3. Is the TSIN model sustainable? How do regional partnerships contribute to

sustainability? (structural) 4. How are the regional partnerships, including pockets of innovation in hubs and

schools, connected across the state? (structural) 5. What are the truly innovative approaches within the TSIN? What are the factors that

contribute to and/or inhibit these innovations? (structural) 6. How does STEM teaching and learning change or improve as a result of the TSIN?

(outcomes) 7. What does the curriculum within TSIN platform schools consist of, and what are

teacher and student roles within the curriculum? (outcomes) 8. Are there more STEM opportunities for students and more students involved in

STEM in TN? (outcomes) 9. Is community awareness and support for STEM increased as a result of the TSIN?

(outcomes) This final evaluation report will focus on the complete analysis of data collected for the two-year evaluation of the TSIN (August 2012 to May 2014).

Tennessee STEM Innovation Network Evaluation Annual Report|x

CONCEPTUAL FRAMEWORK The complexities of educational reform have been discussed throughout the literature as hundreds of initiatives are funded and implemented each year in K-12 systems across the U.S. Reformers continue to struggle with reforming the system – often one school or district at a time - seeking better ways of understanding the influences that facilitate or restrain educational change. Michael Fullan’s (2006) change theory will serve as the lens for examining the implementation of the TSIN and associated regional STEM reform efforts and assessing the ability of efforts to produce desired results. The first premise of change theory is collective motivation for reform. Fullan (2006) argued that motivation and engagement of stakeholders are key precursors for achieving desired change. The second premise of change theory is capacity building, described as “any strategy that increases the collective effectiveness of a group to raise the bar and close the gap of student learning,” and “the more one invests in capacity building, the more one has the right to expect greater performance” (Fullan 2006, p. 9). One of the primary foci for regional STEM Hubs is to build capacity within regions, and the role of the TSIN is to build an infrastructure to connect the associated regional Hubs. Learning in context is Fullan’s (2006) third premise, which focuses on acquiring new knowledge and strategies within the local context. The TSIN approach of implementing regionally focused Hubs aligns well with the necessary local context of reform. Premise four addresses changing “the larger context” to ensure and sustain change, which is essentially building infrastructure within the state and collective motivation to stay the course. Within the larger context it is essential to be proactive in addressing distractors – or variables within the system that favor the status quo (Fullan, 2006). The TSIN was designed to create the infrastructure, but also to generate STEM awareness and advocate for STEM reform on the state level. Premise five is described as “a bias for reflective action” including a shared vision and ownership of reform. Again, the planned structure for the TSIN was designed to enable the network to generate a shared vision and ownership of the transformational effort. The sixth premise of change theory is “tri-level engagement,” an essential component to reform, including the school and community, the school district, and the state as key components (Fullan, 2006). Tri-level engagement is “pursuing strategies that promote mutual interaction and influence within and across the three levels” but not necessarily alignment. As Fullan (2006) argued, alignment is “a static unachievable goal” (p. 11). Persistence and flexibility comprise premise seven. Fullan (2006) argued that because the previous “six premises are complex to manage and must be cultivated over time, including bumpy cycles, a strong resolve is necessary to stay the course” (p. 11). Most of the published research on STEM reform is based upon small-scale reform. The evaluation of the implementation of the TSIN provides an opportunity to examine a much larger scale effort to collectively transform STEM education at a state level with regionally focused investments. Study Methods This evaluation used both qualitative and quantitative data to determine the impact of the TSIN. Data collection included teacher classroom observations (video-recorded), the STEM Community Awareness Survey, focus group interviews, individual interviews, and analysis of STEM school curriculum.

Tennessee STEM Innovation Network Evaluation Annual Report|xi

Classroom Observat ions The TSIN evaluation utilized data collected for the Tennessee Higher Education Commission (THEC) STEM Teacher professional development programs (also funded through Race to the Top). Each participating teacher was required to submit three recordings of their teaching: one prior to participation in the THEC STEM PD program, one midway through the program, and the final video at the end of the program. Each video was scored using the Local Systemic Change (LSC) Classroom Observation Protocol, which was developed by Horizon Research for use with the National Science Foundation’s (NSF) funded State Systemic Initiatives (SSI) as a measure of reform-based instructional practices in science and mathematics. The instrument examines design of lesson, implementation of lesson, culture of instruction, and content knowledge delivered. STEM Community Awareness Survey The STEM Community Awareness Survey (SCAS) was created using the network strategic plan goals as a guiding framework for development and used to determine TSIN and Hub Impact. TSIN Strategic Plan goals include (1) increase student interest, participation, and achievement in STEM; (2) expand student access to effective STEM teachers and leaders; (3) ensure a well-prepared, ready-made STEM workforce for the state by reducing the STEM talent and skills gap; and (4) build community awareness and support for STEM. Participants in the SCAS included K-12 educators, higher education faculty, and business/community stakeholders from across the state. Focus Groups and Interv iews Focus group interviews and Individual Hub Director/School Principal/Teacher interviews were conducted at each of the site visits. All interviews were audio recorded and transcribed for qualitative analysis. Focus group interviews were conducted with individuals who had been involved with activities of the Hub, but were not members of the leadership team of the Hub. In some cases, Hub Directors included members of their Governing Board/Advisory Council despite specific directions not to include these individuals. Focus group sessions were scheduled for a one-hour time slot to provide time for informed consent collection, directions, and approximately 45 minutes of conversation guided by focus group interview protocol questions. Analys is o f STEM School Curri culum The instrument used in this evaluation for conducting the Analysis of STEM School Curriculum was the STEM Attributes Implementation Rubric, (SAIR) which was developed for use in North Carolina by the Friday Institute at North Carolina State University. Permission was granted from the Friday Institute to use this metric in the evaluation of the TSIN. The SAIR consists of three main categories, 11 subcategories, and 41 items in total included in the 11 subcategories. Each subcategory is an individual rubric that is scored.

Tennessee STEM Innovation Network Evaluation Annual Report|xii

KEY FINDINGS Overall Findings Sustainabi l i ty o f Partnerships and TSIN The potential for sustainability of most aspects of the TSIN is very good. An intensive commitment was made from the start by TSIN to establish a sustainable infrastructure across the state that could be maintained following the short two-year period of start-up funding. Sustainability for the TSIN would require not only support to continue with the TSIN administration overall, but also regional success in establishing buy-in and revenue for the STEM Innovation Hubs (and schools). Four of the six TSIN Regional STEM Innovation Hubs have achieved concrete sustainability more than one year beyond the scope of the grant. These Hubs include the West Tennessee STEM Collaboratory Hub (WTSC), the Upper Cumberland Rural STEM Initiative Hub (UCRSI), the East Tennessee State University Hub (ETSU), and the Southeast Tennessee STEM Hub (SETN). Though some have downsized Hub staffing, the core work of each Hub will continue through additional external resources and partnerships. The remaining two TSIN Regional STEM Innovation Hubs were both housed within large K-12 school districts, which created a dilemma for each at the end of funding since the mission of the Hubs is on supporting K-12 education regionally. In regards to STEM Platform Schools, all six schools will be sustained beyond the current year (Innovation Academy, L&N STEM Academy, Prescott Elementary, Prescott Middle, STEM School Chattanooga HS, Stratford HS,). The infusion of resources through Race to the Top helped each of these schools to establish infrastructure to jump start activities. Most have forged extensive partnerships with community business/industry and higher education that will support continued STEM programming experiences. Innovation Academy, located in the ETSU Hub, is the only school that will not continue operation in the format supported by the TSIN funding. The school model will change to a school within a school approach rather than the original independent school design. Overall, most aspects of the TSIN will be sustained at the regional level within Regional STEM Innovation Hubs and STEM Platform Schools. Overwhelmingly stakeholders interviewed, including Hub leadership and partners, have voiced the need for continuing state-level connectivity of Regional STEM Innovation Hub and STEM Platform School efforts. Connect iv i ty The TSIN Model is established upon the premise of connectivity and the network has established a seamless organizational model that allows for real-time sharing and learning from each other. The format of the TSIN provides multiple pathways for dissemination both within regional Hubs and across the state. Monthly network conference calls are conducted where regional sites can add agenda items and provide updates on resources, opportunities, challenges and lessons learned. The TSIN collective group acts as a sounding board for innovation and problem solving group to address issues. The TSIN leadership also has developed Action Teams to address various key

Tennessee STEM Innovation Network Evaluation Annual Report|xiii

components of the network including Advocacy and Partnerships, Communications and Marketing, and Resources. The TSIN collective group is also engaged in STEM nationally as a member of STEMx and as a participant in the annual U.S. News and World Report STEM Solutions Conference each year (most recently as an invited panel to share experiences). Innovations emerging from Hubs and Schools have been extensively, frequently and broadly disseminated across the network and the state at large. Teacher Effec t iveness Overall, the THEC STEM PD programs significantly improved in all four domains (design, implementation, culture, and content) from baseline to end of program. Design of lesson includes the planning, organization, resources, attention to equity, level of collaboration, flow of lesson, assessments, and sense making that take place during the delivery of lesson. Implementation of lessons consists of the level of investigative mathematics/science included, quality of management of classroom, pace of lesson, modifications made, questioning strategies, and formative assessments included in the delivery of the lesson. Classroom culture refers to the amount of active participation of all students and level of collaborative learning, including having students explore their own ideas, questions, conjectures, and propositions or to challenge the ideas of others. Finally, the mathematics/science content knowledge domain focuses on the accuracy of content knowledge delivered by the teacher, as well as the alignment of content to appropriate grade and student levels of understanding. Each item within each domain ranges is scored on a scale of 0 to 5, with 0 being used when there is no evidence of a component within a domain, and a score of 5 awarded when a component is used “to a great extent”. Each domain has multiple questions that are scored individually, and an overall rating (i.e., mean score) for each domain is generated (see Table ES1).

Table ES1. LSC Overall Rating

Score Title

0-1.9 Ineffective Instruction

2-2.9 Elements of Effective Instruction

3-3.9 Beginning of Effective Instruction

4-4.9 Accomplished, Effective Instruction

5 Exemplary Instruction

Tennessee STEM Innovation Network Evaluation Annual Report|xiv

Table ES2. Classroom Observation Findings – all THEC STEM PD Programs

Domain Baseline Rating End Rating End Classification

Design 2.39 2.88 Elements of Effective Instruction

Implementation 2.61 3.30 Beginning of Effective Instruction

Classroom Culture 2.84 3.48 Beginning of Effective Instruction

Content Knowledge 2.90 3.50 Beginning of Effective Instruction

Student Opportunit i es Research Question Eight of this two-year evaluation of the TSIN was focused on determining if the TSIN investment resulted in increased opportunities in Tennessee for K-12 students in STEM education. An examination of data from baseline (fall 2012) and at six month intervals (spring 2013, fall 2013, spring 2014) revealed growth in stakeholder perception of K-12 STEM student opportunities in Tennessee (Table 9). Specifically, for the overall population surveyed, the percentage agreement raised from 58.1% in fall 2012 to 70.7% in spring 2014. This growth was also reflected in all three-stakeholder groups (Business/Community Stakeholders, Higher Education Faculty, and K-12 Educators). Business/Community Stakeholder perception of K-12 student STEM opportunities increased from 61.8% (fall 2012) to 77.8% (spring 2014). Similarly, Higher Education Faculty agreement scores improved from 70.1% (fall 2012) to 74.1% (spring 2014). Finally, K-12 Educator agreement increased from 54.2% (fall 2012) to 60.1% (spring 2014). Findings indicated that STEM Community Stakeholders in each region also reported increased agreement with the presence of more K-12 STEM education opportunities for students in each region (Table 10). All TSIN Regional STEM Innovation Hubs had at least 64% agreement by end of the second year, indicating most participants perceived more K-12 STEM student opportunities in the region. STEM Community Awareness Survey Stakeholder opinions (Business/Community Stakeholders, Higher Education Faculty, and K-12 Educators) regarding the quality of STEM education in Tennessee improved in all four constructs (Industry Engagement, Awareness and Resources, Preparation of Students, and Careers and Workforce) across the evaluation. Participants had a moderately favorable attitude toward STEM awareness and support on the entire survey, averaging “Agree” Overall at the end of the evaluation (Baseline M = 2.73; End M = 2.98). Additionally, participants improved their reported perceptions in the following survey subsections to a level of “Agree” by end of evaluation: Industry Engagement (Baseline M = 2.73; End M = 3.04), Awareness and Resources (Baseline M = 2.91; End M = 3.16), and Careers and Workforce (Baseline M = 2.69; End M = 2.91). However, the survey subsection (construct) Preparation of Students had the lowest reported average scores, which improved but were not at an “Agree” level at the end of the evaluation (Baseline M = 2.44; Follow-Up M = 2.66). The TSIN made substantial progress in transforming Tennessee Stakeholder STEM Awareness in the areas of Industry Engagement, Awareness and Resources, Careers and Workforce, though continued effort

Tennessee STEM Innovation Network Evaluation Annual Report|xv

is needed in the Preparation of Students K-20. Table ES2 shows results for the overall survey and each subsection, along with average net change from baseline to follow-up.

Table ES3. SCAS Overall and Construct Comparison Baseline (2012) to End of Year Two (Spring 2014)

Survey Construct (Total Points Possible)

Baseline M(SD)

Spring 2013 M(SD)

Fall 2013 M(SD)

Spring 2014 M(SD)

Average Net Change

Overall (156 points)

106.61 (13.22) 107.12 (13.46) 105.44 (15.07) 116.30 (17.25) +9.69

Industry Engagement (IE) (32 points)

21.83 (4.21) 22.14 (4.17) 21.91 (4.83) 24.28 (4.74) +2.45

Awareness and Resources (AR) (52 points)

37.85 (4.97) 37.89 (4.79) 37.62 (5.52) 41.09 (6.77) +3.24

Preparation of Students (PR) (24 points) 14.65 (3.32) 14.73 (3.37) 14.16 (3.57) 15.96 (4.05) +1.31

Careers and Workforce (CW) (48 points) 32.31 (4.24) 32.46 (4.22) 31.73 (4.62) 34.88 (5.08) +2.57

Individual TSIN Hub and STEM Platform School Narratives In addition to the overall statewide analysis of teacher effectiveness and SCAS data, individual Hub and School narratives were compiled to examine in closer detail regional STEM awareness, motivations for STEM stakeholder engagement, regional innovations, STEM school curriculum characteristics and STEM school and regional Hub progress. SUMMARY Overall, the two-year evaluation of the TSIN revealed positive growth, as well as improvements in various areas for each of the Regional STEM Innovation Hubs and the STEM Platform Schools. Motivations for stakeholder engagement in the TSIN were revealed, along with growing awareness regarding STEM for K-12, higher education, and business/community stakeholders. Several innovations emerged out of the work of TSIN Regional STEM Innovation Hubs and STEM Platform Schools that can be disseminated across the state to help support more K-12 schools to engage in STEM education reform.

Tennessee STEM Innovation Network Evaluation Annual Report|1

I. INTRODUCTION BACKGROUND The Tennessee STEM Innovation Network (TSIN) was established through executive order of the Governor of Tennessee in 2011. The TSIN is a public-private collaboration of the Tennessee Department of Education and the Battelle Memorial Institute. The purpose of TSIN is to, “promote and expand the teaching and learning of science, technology, engineering, and mathematics (STEM) education in K-12 public schools across Tennessee.” The TSIN is comprised of nine regional STEM innovation hubs and ten STEM platform schools. The work of the TSIN is structured to support the implementation of the Tennessee STEM Strategic Plan, which includes four broad goals and supporting strategies designed to move Tennessee forward as a “national leader in STEM talent production and economic prosperity” (p. 3). The four broad goals and supporting strategies of the TN STEM Strategic Plan include the following:

1. Increase student interest, participation, and achievement in STEM. a. Establish regional STEM innovation Hubs. b. Launch STEM platform schools. c. Ensure all students have access to rigorous STEM courses. d. Identify, develop, and share STEM curriculum tools.

2. Expand student access to effective STEM teachers and leaders. a. Increase effective STEM teachers and leaders. b. Replicate proven STEM teacher training programs. c. Use STEM schools as learning labs. d. Boost STEM teacher supply. e. Enhance STEM teacher capacity and reach. f. Increase quality STEM professional development. g. Support school leaders.

3. Reduce the state’s STEM talent and skills gap. a. Increase accelerated STEM learning opportunities. b. Increase partnerships between business and education. c. Increase STEM postsecondary degree production.

4. Build community awareness and support for STEM. a. Build communication tools, develop messages, and identify delivery channels. b. Conduct media outreach. c. Identify and showcase STEM public events. d. Increase STEM stakeholder engagement.


This final evaluation report will focus on the complete analysis of data collected for the two-year evaluation of the TSIN (August 2012 to May 2014). This will include two years of site visit data (four visits) for the first five Hubs (Chattanooga, Cookeville, Johnson City, Knoxville, and Nashville) and the STEM platform school associated with each Hub (see Table 1). The Memphis Hub and Southwind STEM Academy did not come on board until the second year of this evaluation. Therefore, data for this sixth Hub and platform school will only include two visits (baseline and final).

Table 1. TSIN Hubs and Platform School(s) Included in Evaluation

Regional STEM Innovation Hub STEM Platform School(s)

Chattanooga: Southeast TN STEM Innovation Hub

STEM School Chattanooga

Cookeville: Upper Cumberland Rural STEM Initiative

Prescott South Elementary School Prescott South Middle School

Johnson City: ETSU Northeast STEM Innovation Hub

Innovation Academy of Northeast TN

Knoxville: STEMspark East TN Innovation Hub

L&N STEM Academy

Memphis: West TN STEM Hub Southwind STEM Academy

Nashville: Middle TN STEM Innovation Hub

Stratford STEM Magnet High School Hattie Cotton STEM Magnet School


II. RESEARCH METHODS RESEARCH QUESTIONS Nine research questions, listed below, guided this evaluation. Five of the questions are structural in nature, and four are outcomes based. All are aligned with the goals of the TN STEM Strategic Plan:

1. Does the TSIN model create strong, sustained partnerships within and across regional Hubs? (structural)

2. What motivates stakeholder engagement in TSIN? (structural) 3. Is the TSIN model sustainable? How do regional partnerships contribute to

sustainability? (structural) 4. How are the regional partnerships, including pockets of innovation in hubs and

schools, connected across the state? (structural) 5. What are the truly innovative approaches within the TSIN? What are the factors that

contribute to and/or inhibit these innovations? (structural) 6. How does STEM teaching and learning change or improve as a result of the TSIN?

(outcomes) 7. What does the curriculum within TSIN platform schools consist of and what are

teacher and student roles within the curriculum? (outcomes) 8. Are there more STEM opportunities for students and more students involved in

STEM in TN? (outcomes) 9. Is community awareness and support for STEM increased as a result of the TSIN?

(outcomes) This final report will focus on providing evidence of progress related to research questions 1-9. In this report, research questions 3, 4, 6, 8, 9 will be addressed at the TSIN (network) level overall in regard to findings of the first year of the evaluation. Individual regional Hub level narratives will also be included that will focus on evidence responding to research questions 1, 2, 5, 7, and 9. The following research activities (with the exception of those marked with *) will be used to gather evidence of progress and guide this report:

• TSIN and Hub Impact Surveys (e.g., STEM Community Awareness Surveys). • Student STEM Survey* (replaced with individual Hub narratives). • Focus Groups and Interviews (conducted during Hub/school site visits). • Student Achievement*. • Teacher Effectiveness (using THEC STEM PD program data). • Tracking System for HS and HE Graduates* (removed per TDOE). • Analysis of STEM School Curriculum (within STEM platform schools). • Impact – Tracking and Identifying Innovations from TSIN • Site Visits (of Hub and STEM platform schools)


CONCEPTUAL FRAMEWORK The complexities of educational reform have been discussed throughout the literature as hundreds of initiatives are funded and implemented each year in K-12 systems across the U.S. Reformers continue to struggle with reforming the system – often one school or district at a time - seeking better ways of understanding the influences that facilitate or restrain educational change. Michael Fullan’s (2006) change theory will serve as the lens for examining the implementation of the TSIN and associated regional STEM reform efforts and assessing the ability of efforts to produce desired results. The first premise of change theory is collective motivation for reform. Fullan (2006) argued that motivation and engagement of stakeholders are key precursors for achieving desired change. The second premise of change theory is capacity building, described as “any strategy that increases the collective effectiveness of a group to raise the bar and close the gap of student learning,” and “the more one invests in capacity building, the more one has the right to expect greater performance” (Fullan, 2006, p. 9). One of the primary foci for regional STEM Hubs is to build capacity within regions, and the role of the TSIN is to build an infrastructure to connect the associated regional Hubs. Learning in context is Fullan’s (2006) third premise, which focuses on acquiring new knowledge and strategies within the local context. The TSIN approach of implementing regionally focused Hubs aligns well with the necessary local context of reform. Premise four addresses changing “the larger context” to ensure and sustain change, which is essentially building infrastructure within the state and collective motivation to stay the course. Within the larger context it is essential to be proactive in addressing distractors – or variables within the system that favor the status quo (Fullan, 2006). The TSIN was designed to create the infrastructure as well as to generate STEM awareness and advocate for supporting STEM reform on the state level. Premise five is described as having “a bias for reflective action,” including a shared vision and ownership of reform. Again, the planned structure for the TSIN was designed to enable the network to generate a shared vision and ownership of the transformational effort. The sixth premise of change theory is “tri-level engagement,” an essential component of reform including the school and community, the school district, and the state as key players (Fullan, 2006). Tri-level engagement is “pursuing strategies that promote mutual interaction and influence within and across the three levels” but not necessarily alignment, which Fullan (2006) argued is “a static unachievable goal” (p. 11). Persistence and flexibility comprise premise seven. Fullan (2006) argued that, because the previous “six premises are complex to manage and must be cultivated over time, including bumpy cycles, a strong resolve is necessary to stay the course” (p. 11). Most of the published research on STEM reform has been based upon small-scale reform efforts. The evaluation of the implementation of the TSIN provides an opportunity to examine a much larger scale effort to collectively transform STEM education at a state level with regionally focused investments.


DATA COLLECTION AND ANALYSIS The evaluation of TSIN included a variety of qualitative and quantitative data. The data collection and analysis activities for this report included the STEM Community Awareness Survey (SCAS), teacher classroom observations in digital recording format, and site visit data including focus groups, interviews, and the STEM Attributes Rubric that was completed for each STEM Hub Platform School. Each of these is described in more detail below. STEM Community Awareness Survey (TSIN/Hub Impact Survey) The STEM Community Awareness Survey (SCAS) was created using the network strategic plan goals as a guiding framework for development and was used to determine TSIN and Hub Impact. Data from this survey were used to answer Research Questions Eight and Nine. TSIN Strategic Plan goals include the following: (1) increase student interest, participation, and achievement in STEM; (2) expand student access to effective STEM teachers and leaders; (3) ensure a well-prepared, ready-made STEM workforce for the state by reducing the STEM talent and skills gap; and (4) build community awareness and support for STEM. The instrument developed to assess STEM awareness and support aligns primarily with goal four, but also gleans some insight from other goals as well. This emphasis is purposeful, as this survey is the main data-gathering tool aligned with goal four, whereas other data sources inform progress on the other goals. Extensive piloting has been done with the SCAS and resulted in a total of 39 Likert-type items all on a 1-4 point scale (Strongly Disagree, Disagree, Agree, Strongly Agree). The SCAS measures four main item themes: Industry Engagement in STEM Education (IE) (8 items), STEM Awareness and Resources (AR) (13 items), Regional STEM Careers & Workforce (CW) (12 items), and Preparation of Students for Success in College & Careers (PR) (6 items). Table 2 has an overview of the constructs and items included in the SCAS.


Table 2. SCAS Constructs and Items

Regional STEM Careers & Workforce

Preparation of Students for Success

STEM Awareness and Resources

Industry Engagement in STEM

Signing bonuses/incentives

K-12 schools prepare students to be

successful in post-secondary study

K-12 schools in this region understand the importance

of STEM education

It is important for area businesses to have STEM

partnerships with local schools

Increase in STEM positions

Students are knowledgeable

regarding STEM careers

Parents in this region understand the importance

of STEM education

I have participated in community funded STEM

education programs in K-12

Increase in diversity in STEM positions

K-12 schools effectively teach

STEM knowledge and skills

More work needs to be completed to spread awareness of STEM

education

I have participated in guest speaking focused on STEM

education in K-12

Ability to fill STEM positions

State testing assesses STEM knowledge and

skills

STEM skills are integral to student success today

There are opportunities for K-12 students to complete internships/co-ops in the

region

Importance of recruiting for STEM

locally

K-12 schools prepare critical thinkers and

problem solvers

Increasing the STEM talent pool is necessary for

economic vitality

There are organizations interested in providing

STEM opportunities for K-12 students in the region

Students are informed of STEM careers

Community partners are engaged in making STEM learning more relevant through real-

world connections

Students with post-secondary education are more likely to secure a career in a STEM field

There has been an increase in K-12 STEM

opportunities for students in this region in the last year

K-12 schools teach STEM knowledge and skills aligned with jobs

Local organizations recruit STEM talent

online

Colleges/universities offer scholarships for students

to pursue STEM degree in my region

I have worked closely in partnership with

business/community, higher education, or K-12 in my

current role

All K-12 students should have access to

STEM education

Information related to STEM opportunities

in the region is available online

There are STEM education websites with activities for K-12 teachers and students

in my region

STEM online tools are available to this

region

Information on regional STEM career

opportunities is available online


Response Rate – STEM Community Awareness Survey (SCAS) A total of 8,296 participants chose to respond to at least part of the survey. Data from these participants were considered usable for analysis if they completed at least one full section from the survey. There were 5,526 (64.2 percent) participants who completed the survey across the baseline in Fall 2012 (n = 2,446, 46.1 percent), and follow-up survey administrations in Spring 2013 (n = 1,349, 25.4 percent), Fall 2013 (n = 545, 10.3 percent), and Spring 2014 (n = 986, 18.2 percent). Although there were more participants at the time of the baseline distribution, samples from the baseline and follow-up data collections were similar in terms of types of participants. In all instances, K-12 teachers were the largest group completing the survey followed by higher education faculty and then business community members. Approximately two thirds of the samples were comprised of females, and about half self-identified as being from a rural location, followed by suburban, and then urban. Finally, STEM hub region representation was similar from baseline to follow-up survey distribution (Table 3). Analys is o f STEM Community Awareness Survey (SCAS) Data A longitudinal trend study was conducted for this evaluation survey research. The study is longitudinal because we collected data at more than one time point (baseline and multiple follow-ups). It is a trend study because the general population is being sampled with a new sample used at each point. Trend studies are more convenient to conduct, but they are limited because participants are not tracked over time, making repeated measures inferential statistical analyses impossible. Descriptive analyses of net change are, however, able to be identified. Any analyses assessing for differences in overall survey or subscales by groups (professional role or STEM hub region) will employ One-Way ANOVAs at each time point (baseline and follow-ups). Again, since our sample is from the same population but is not comprised of the exact same individuals, we are unable to conduct inferential analyses for the change in baseline to follow-up. Instead, we will examine descriptive statistics (means and standard deviations) and look for practical differences between baseline and follow-up survey distributions.


Table 3. Demographic Frequencies and Percentages: Baseline (2012) and Follow-up SCAS Survey Samples

Demographic Variables

Fall 2012* 38.9% (n = 2,552)

Spring 2013 24.7% (n = 1,620)

Fall 2013 11.9% (n = 781)

Spring 2014 24.5% (n = 1,601)

Pro

fess

iona

l R

ole

K-12 Teacher 49.9% (n = 1,273) 40.9% (n = 663) 32.1% (n = 251) 27.0% (n = 432)

Higher Education Faculty

28.8% (n = 735)

56.5% (n = 916) 58.2% (n = 455) 53.1% (n = 848)

Business Community 21.3% (n = 544) 2.6% (n = 41) 9.7% (n = 75) 19.9% (n = 321)

Gen

der

Female 67.4% (n = 1,720) 70.0% (n = 1,134) 57.9% (n = 452) 64.8% (n = 1,037)

Male 32.6% (n = 832) 30.0% (n = 486) 42.1% (n=329) 35.2% (n = 564)

Age

18-25 years old 1.6% (n = 41) 0.9% (n = 15) 0.6% (n = 4) 0.8% (n = 14)

26-35 years old 16.0% (n = 408) 17.5% (n = 284) 11.9% (n = 93) 11.9% (n = 191)

+35 years old 82.4% (n = 2,103) 81.6% (n = 1,321) 87.6% (n = 684) 87.2% (n = 1,396)

Geo

grap

hic

Loc

atio

n

Rural 53.4% (n = 1,362) 50.4% (n = 817) 45.5% (n = 355) 51.6% (n = 826)

Urban 14.6% (n = 373) 15.5% (n = 251) 19.8% (n =155) 14.5% (n = 232)

Suburban 31.0% (n = 817) 32.5% (n = 552) 33.0% (n = 271) 32.7% (n = 543)

STE

M H

ub

Knoxville 17.4% (n = 443) 17.0% (n = 275) 11.3% (n = 88) 13.5% (n = 216)

Chattanooga 10.8% (n = 276) 14.3% (n = 231) 9.9% (n = 77) 7.2% (n = 115)

Cookeville 17.6% (n = 448) 15.4% (n = 249) 20.7% (n = 162) 12.3% (n = 197)

Nashville 36.2% (n = 924) 34.1% (n = 552) 21.0% (n = 164) 26.9% (n = 430)

ETSU/Tri-Cities 18.1% (n = 461) 19.3% (n = 313) 14.9% (n = 116) 14.6% (n = 234)

Memphis NA NA 22.3% (n = 174) 25.5% (n = 409)

*Baseline Sample Note. All percentages may not add up to 100% for each demographic variable due to missing data for some participants Teacher Effectiveness Data Teacher observations were conducted for use in determining potential increased use of STEM pedagogical skills and STEM content knowledge for THEC STEM PD programs situated in Tennessee. Each participating teacher in all funded STEM PD programs was asked to submit three digital recordings of an appropriate STEM lesson. The first recording was to be conducted prior to beginning participation in the THEC STEM PD program. The second was to occur at the midpoint of participation (August 2012/2013), and the final recording was to be completed and submitted by December 2012/2013. Classroom Observat ion Instrument The Local Systemic Change (LSC) Classroom Observation Protocol is an observation tool used to assess the degree of instructional reform in math and science. The LSC protocol was developed by Horizon Research for use with the National Science Foundation’s (NSF) funded State Systemic Initiatives (SSI) as a measure of reform-based instructional practices. The LSC Classroom


Observation Protocol is being used as the measure of growth in teacher pedagogical skill use and is one measure of teacher content knowledge for the THEC STEM PD program. The LSC tool is valid for use in this evaluation based on the research-based foundation and widescale implementation of the LSC protocol in many empirical studies. Using the LSC, teacher instruction is observed and given ratings on 32 items included in four domains (see Table 4).

Table 4. LSC Domains

Domain Number of Items

Design of Lesson 10

Implementation of Lesson 7

Classroom Culture 6

Math/Science Content 9 The Design of Lesson domain focuses on the structure of the observed lesson and investigates a variety lesson considerations, such as the sequencing of instructional activities, roles of students and teachers, resources available, eliciting of prior knowledge, time provided for sense making, attention to diversity, and collaborative learning. The Implementation of Lesson domain examines the use of investigative STEM strategies employed by the teacher, as well as the pace of the lesson, attention to student understanding, questioning strategies, and both formative and summative assessments. The Classroom Culture domain assesses a teacher’s ability to create and facilitate a classroom environment that supports active participation, respect for ideas, effective collaboration, and inquiry into student ideas, questions, and real-world connections. The Mathematics/Science Content domain examines teacher understanding of content, as well as appropriateness of the level of content included in the lesson, the level of student engagement with content, and interdisciplinary and real-world connections presented by the teacher. Each item within each domain range is scored on a scale of 0 to 5, with 0 assigned when there is no evidence of a component within a domain, and a score of 5 assigned when a component is used “to a great extent”. Each domain has multiple questions that are scored individually, and an overall rating (i.e., mean score) for each domain is generated (see Table 5).

Table 5. LSC Overall Rating

Score Title

0-1.9 Ineffective Instruction

2-2.9 Elements of Effective Instruction

3-3.9 Beginning of Effective Instruction

4-4.9 Accomplished, Effective Instruction

5 Exemplary Instruction An overall score of 0 to 1.9 is characterized with a rating of Ineffective Instruction. The LSC protocol describes this category as a classroom where there is “little or no evidence of student thinking or


engagement with important ideas of mathematics/science. Instruction is highly unlikely to enhance students’ understanding of the discipline or to develop their capacity to successfully do mathematics or science.” With this rating, the delivered lesson is characterized as either passive learning or activity for activity’s sake. Passive learning is when students are passive recipients of information from the teacher or textbook. Activity for activity’s sake happens when a hands-on activity is employed with no clear purpose and does not lead to student conceptual development of STEM. An overall score of 2-2.9 receives the rating of Elements of Effective Instruction. The LSC protocol describes this category as a classroom where “instruction contains some elements of effective practice but there are serious problems in the design, implementation, content, and/or appropriateness for many students in the class.” Examples include inappropriate content and/or level of content, lack of ability to address student difficulties, and lack of opportunities for inquiry and investigation of student ideas and problem solving. An overall score of 3-3.9 is classified as Beginning Stages of Effective Instruction. The LSC protocol describes this category as a classroom where “instruction is purposeful and characterized by quite a few elements of effective practice.” In this classroom, students are engaged in meaningful work at times, but there are still a few weaknesses with the delivery of the lesson. An overall score of 4-4.9 is characterized as Accomplished, Effective Instruction. The LSC protocol describes this category as a classroom that is “purposeful and engaging for most students.” Students are engaged in meaningful work, including investigations, and the lesson is well designed and implemented. Some limitations in ability to adapt content and/or pedagogy still exist, and ability to respond to student needs is also limited. Instruction is “quite likely” to enhance student ability to do STEM. An overall score of 5 is Exemplary Instruction. The LSC protocol describes this category as a classroom where, “purposeful instruction [is occurring] and all students are highly engaged most or all of the time with meaningful work.” The lesson is “artfully implemented”; the teacher is flexible and responds to student needs and interests; and instruction is highly likely to enhance student understandings of the discipline and to develop their capacity to do STEM. Response Rate - Teacher Observat ion Data Collectively, for Round One and Two, 667 teachers were observed at least once. Of those 667 teachers, 236 teachers (35.4 percent) completed two full observations, which were then scored, and 270 teachers (40.5 percent) completed three full observations that were subjected to LSC scoring. These 270 teachers will serve as the sample for this report, as they participated in the entire PD program and provide the most accurate measure of change over time. Of the 270 teachers with three observations, a majority of participants are White (93.7 percent, n = 253). Further, participants from PD Rounds were unevenly distributed with a majority coming from Round Two (67.4 percent, n = 182) in comparison to Round One (32.6 percent, n = 88). Analys is o f Teacher Observat ion Data A team of evaluators rated individual teacher videos and these ratings were analyzed quantitatively. All videos were viewed and scored by two independent raters using the LSC Classroom Observation Protocol in four domains - design of lesson, implementation of lesson, mathematics/science content


knowledge, and classroom culture - were rated and an additional overall rating was assigned. These measures are used to determine improvement in teacher pedagogical skills and content knowledge as demonstrated through actual teacher practice. Total scores for each domain were computed. Domain sections were comprised of varying numbers of items (see Table 4). Individual item ratings ranged from 1-5 with 1 being lowest and 5 being highest (see Table 5). In addition to the domain rating, an overall rating was also assigned to each teacher for each lesson. To assess teacher growth in specific classroom practices over time and by program classification (high school chemistry, elementary science, primary math, elementary math, middle grades math) a 3-Within, 7-Between Repeated Measures ANOVAs with post-hoc investigation for differences at each observation time and between program classification groups was conducted. Finally, growth examinations between all observation time points using 3-Within Repeated Measures ANOVAs with post-hoc investigation for each specific program’s STEM Teacher Quality results are conducted. Since sample sizes for individual programs are small, one-tailed tests were run to increase the sensitivity for finding statistically significant differences over time. Focus Groups and Individual Interviews Focus group interviews and Individual Hub Director/School Principal/Teacher interviews were conducted at each of the site visits. All interviews were audio recorded and transcribed for qualitative analysis. Focus group interviews were conducted with individuals who had been involved with activities of the Hub but were not members of the leadership team of the Hub. In some cases, Hub Directors included members of their Governing Board/Advisory Council despite specific directions to not include these individuals. The focus group sessions were scheduled for one-hour time slots in order to provide time for informed consent collection, directions, and approximately 45 minutes of conversation guided by focus group interview protocol questions (see Appendix). The individual interviews were conducted during site visits and were scheduled in advance. With each site visit the Hub Director, STEM Platform School principal, and one to three teachers were interviewed using an interview protocol (see Appendix). Questions focused on their views of STEM, activities for students, planning/PD activities for teachers, school curriculum, interactions between the Hub/STEM School, and plans for sustainability. Interviews ranged between 20 and 60 minutes in duration. Analysis of STEM School Curriculum and Site Visits The instrument used in this evaluation for conducting the Analysis of STEM School Curriculum was the STEM Attributes Implementation Rubric (SAIR), which was developed for use in North Carolina by the Friday Institute at NC State University. Permission was granted from the Friday Institute to use this metric in the evaluation of the TSIN. The SAIR consists of three main categories and 11 subcategories with 41 items included within the 11 subcategories. Each subcategory is an individual rubric that is scored. Tables 6-8 provide a description of the three categories, as well as their subcategories and associated items. The full STEM Attributes Implementation Rubric is included in Appendix D.


Table 6. STEM Attribute Implementation Rubric Category 1: Integrated STEM Curriculum

Aligned with State, National, International and Industry Standards (with Subcategory (A1-A6) and Associated Items)

Project-based learning

with integrated content across STEM subjects

(A1)

Connections to effective in

and out-of-school STEM

programs (A2)

Integration of

technology and virtual

learning (A3)

Authentic assessment

and exhibition of STEM skills

(A4)

Professional development on integrated

STEM, partnerships,

post-secondary

connections (A5)

Outreach, support, and

focus on underserved students (A6)

Frequency of PBL STEM

Network connections

Common technology access and

mastery

Authentic assessment use

Individualized professional development

Culture of trust

Integration/multiple subjects in PBL

Research & development

Integrated technology –

multiple modes and frequency

Teachers develop

assessments collaboratively

Job-embedded professional development for teachers

Recognize under-

represented students

(policies and practices)

Teacher professional learning communities

for planning PBL

Students and STEM

Professionals

Technology resources for

teachers

Culture of innovation

Student specific professional development

Inspire under-represented

students through

programs in and out of school in

STEM fields

Frequency of STEM Integration Technology

support Celebrate

STEM work

Frequency of professional development

School physical space


Table 7. STEM Attribute Implementation Rubric Category 2: Ongoing Community and Industry Engagement (with Subcategory (B1-B3) and Associated Items)

STEM work-based learning experiences (B1)

Participation in STEM Network (B2)

STEM Program Plan (B3)

Learning directly connected to industries

Trusted collaboration in STEM Network

STEM program plan

Students work in teams Communication tools Communicate STEM program plan

Teachers interact with STEM industries Stakeholders and funding Program data

STEM professionals and lesson planning

Resource allocation

Table 8. STEM Attribute Implementation Rubric Category 3: Connections with Post-secondary Education (with Subcategory (C1-C2) and Associated Items)

Alignment of student’s career pathways with post-secondary STEM program(s) (C1)

Credit completion at community colleges, and colleges/universities (C2)

Vertical planning Credit completion availability

Counselor and student relationships Student enrollment

Counselors and teachers communicate Comprehensive advising

Information sharing On each of the 41 items included in the SAIR, the STEM Platform School could receive a rating of Early (starting), Developing, Advanced (Prepared), or Target. The rubric provides clear information on activities/characteristics that must be presented or observed during the evaluation visit in order to receive the various ratings for each item. For this evaluation, the STEM Attributes Implementation Rubric was used as a qualitative measure to track progress of STEM Platform Schools in moving toward full implementation of the STEM approach. Four site visits were conducted at each of the STEM Platform Schools within each of the first five Hubs (Chattanooga, Cookeville, ETSU, Knoxville, and Nashville). The first site visits were conducted in fall 2012, spring 2013, fall 2013, and spring 2014. Two site visits were conducted at the sixth Hub (Memphis) in fall 2013 and spring 2014. During each site visit, several classrooms were visited and field notes were taken. Additionally, interviews were conducted with school leadership and teachers. Following the site visit, all data were analyzed, and the STEM Attributes Implementation Rubric was completed based upon data available for the evaluation.


Limitations All quantitative research is subject to limitations from methodological threats to internal and external validity (Onwuegbuzie, 2000). Internal validity focuses on the research design and asks if it is appropriate to attribute the differences found in the dependent variable to the independent variable and nothing else. External validity addresses a study’s ability to generalize findings from one study to and across populations, settings, and times. For this evaluation study, two major methodological limitations to validity are acknowledged: (1) teacher participation in data collection, and (2) nature of the content knowledge tests. Teacher participation in data collection is a potential external validity limitation in this evaluation study. Out of 667 participating teachers in the THEC STEM PD programs, the response rate for one classroom observation was 82.1 percent (n = 623). Only 40.4 percent (n = 269) of participating teachers responded with three full classroom observations. Furthermore, because some participants did not participate in the data collection process, findings of this evaluation are vulnerable to non-response error. Non-response error refers to the possibility that a significant number of THEC STEM PD teachers may choose to not respond, and that these non-respondents are significantly different from THEC participants who responded. Thus, the results may become non-generalizable to the larger program sample. Any time a response rate is under 60-70 percent, non-response needs to be examined further. In this evaluation, participant demographics (e.g., program content, program grade level focus, gender, and ethnicity) for those responding to data collection procedures are similar to that of the overall participant group. As such, we concluded that there does not appear to be any systematic non-response issues, making this a less significant concern than if specific subgroups of individuals chose not to participate.


III. FINDINGS OVERALL FOR TSIN INVESTMENT – RESEARCH QUESTION 3: IS THE TSIN MODEL SUSTAINABLE? HOW DO REGIONAL PARTNERSHIPS CONTRIBUTE TO SUSTAINABILITY? __________ SUSTAINABILITY OF TSIN The Tennessee STEM Innovation Network (TSIN) was funded through Race to the Top funding received by the state of Tennessee in 2011 as a unique public-private collaboration between the Tennessee Department of Education (TDOE) and Battelle Memorial Institute (Battelle) designed to “promote and expand the teaching and learning of science, technology, engineering and mathematics (STEM) education in K-12 public schools across Tennessee.” One of the primary structural research questions in this evaluation was focused on the sustainability of the TSIN, including how the regional partnerships contribute to sustainability. Allocated funding supported the establishment of the TSIN office (staffed by director Wes Hall, and other supporting staff) housed in Nashville, as well as six Regional STEM Innovation Hubs situated across the state in strategic cities/towns (Chattanooga, Cookeville, Knoxville, Johnson City, Memphis, and Nashville). The regional TSIN Hubs were established as a network for statewide connectivity to be used as a pipeline for dissemination of STEM innovations including resources, curriculum, opportunities, and awareness/advocacy materials. Situated within each of the Regional STEM Innovation Hubs was a STEM Platform School (or schools) that was designed to be a site for development and dissemination of innovative approaches to STEM, supported by the Regional STEM Innovation Hubs. An intensive commitment was made from the start by the TSIN to establish a sustainable infrastructure across the state that could be maintained following the short two-year period of start-up funding. Previous educational reforms efforts indicate that sustainability of an entity of this type requires a combination of two key components. First, substantial buy-in of regional stakeholders with the means to provide support for the initiative in the long-term is key to sustainability. Second, secured funding for activities through potential avenues including partners, foundations, fee-for-service models, and/or other funding agencies is necessary. Sustainability for the TSIN would require not only support to continue with the TSIN administration overall, but also regional success in establishing buy-in and revenue for the STEM Innovation Hubs (and schools). Four of the six TSIN Regional STEM Innovation Hubs have achieved concrete sustainability more than one year beyond the scope of the grant. These Hubs include the West Tennessee STEM Collaboratory Hub (WTSC), the Upper Cumberland Rural STEM


Initiative Hub (UCRSI), the East Tennessee State University Hub (ETSU), and the Southeast Tennessee STEM Hub (SETN). Interestingly, three of the four are housed within higher education institutions with existing infrastructure for STEM education that will be leveraged to continue Hub work indefinitely. The fourth is housed within a non-profit organization whose mission to support public education has identified STEM as an area of focus long term. Though some of the Hubs have downsized staffing, the core work of each Hub will continue through additional external resources and partnerships. The remaining two TSIN Regional STEM Innovation Hubs were both housed within large K-12 school districts, which created a dilemma for each at the end of funding since the mission of the Hubs is to support K-12 education regionally. The MTSI Hub, located in Metropolitan Nashville Public Schools (MNPS) was successful in gaining a commitment from MNPS to support a 120-day extension of activities to enable time to further explore sustainability scenarios with Belmont University and other potential partners. The STEMSpark Hub, located within the Knox County School System suspended activities in June 2014 as the district is working on next steps for the Hub with limited support that includes a part-time position to lead the efforts. The Hub was not successful in securing partner buy-in or funding to continue work. In regard to STEM Platform Schools, five of the six schools will be sustained beyond this year (Stratford HS, Prescott Elementary, Prescott Middle, L&N Academy, STEM School Chattanooga HS). The infusion of resources through Race to the Top helped each of these schools to establish infrastructure to jump start activities. Most have secured extensive partnerships with community business/industry and higher education that will support continued STEM programming experiences. Innovation Academy, located in the ETSU Hub, is the only school that will not continue operation in the format established originally by the TSIN funding. District administration decided to move the school into an existing larger middle school where Innovation Academy will continue to operate through a school within a school model in the 2014-15 academic year. Overall, most aspects of the TSIN will be sustained at the regional level within Regional STEM Innovation Hubs and STEM Platform Schools. Additionally, at the state TSIN level, an application for no cost extension of Race to the Top funding has provided TSIN staff the opportunity to aggressively seek out key partners and targeted funding to sustain the TSIN. Overwhelmingly stakeholders interviewed, including Hub leadership and partners have voiced the need for continuing to have state-level connectivity of Regional STEM Innovation Hub and STEM Platform School efforts. This will continue to allow others to learn from the TSIN investments. Many K-12 schools have already benefited from visiting these sites and attending professional development sessions to learn how to implement STEM. The TSIN is a national model for establishing strong, locally-driven infrastructure for supporting educational reform.


IV. FINDINGS OVERALL FOR TSIN INVESTMENT – RESEARCH QUESTION 4: HOW ARE THE REGIONAL PARTNERSHIPS, INCLUDING POCKETS OF INNOVATION IN HUBS AND SCHOOLS, CONNECTED ACROSS THE STATE? CONNECTIVITY OF NETWORK The TSIN Model is established upon the premise of connectivity, and the network has established a seamless organizational model that allows for real-time sharing and learning between Hubs. As depicted in Figure 1 there have been many new innovations that have emerged from each of the six TSIN Regional STEM Innovation Hubs. The format of the TSIN provides multiple pathways for dissemination both within regional Hubs and across the state. For example, Hubs and Schools are encouraged to visit other partners in the network for professional growth. Many have visited at least two other sites to date. Monthly network conference calls are conducted where regional sites can add agenda items and provide updates on resources, opportunities, challenges and lessons learned. The TSIN collective group acts as a sounding board for innovation and problem solving group to address issues. The TSIN leadership also has developed Action Teams to address various key components of the network including Advocacy and Partnerships, Communications and Marketing, and Resources. Leadership of the TSIN (state and regional) are engaged in continuous professional growth and are actively involved in learning from others as well as broadening capacity statewide through delivery of training sessions to enable others to implement STEM. The TSIN collective group is also engaged in STEM nationally as a member of STEMx as a participant in the annual U.S. News and World Report STEM Solutions Conference each year (most recently as an invited panel to share experiences). Innovations emerging from Hubs and Schools have been extensively, frequently, and broadly disseminated across the network and the state at large. The connectivity that has been afforded by the TSIN has already begun to pay dividends indirectly through improved teaching and learning of STEM in Tennessee.


Figure 1. Tennessee STEM Innovation Network (TSIN) Innovations

TSIN Innovations

SETN Hub

STEM School Chattanooga HS STEM PSAs

STEM Teaching Fellows Technology Integration

Integrated STEM curriculum Innovate Education Channel

UCRSI Hub

Prescott South ES and MS STEM Mobile

Rural STEM Ed Conference STEM Coach

Out-of-School Programs Early STEM Learning

ETSU Hub

Innovation Academy MS Just-in-Time PD

Co-Teaching Model Technology Integration

Integrated STEM Curriculum Community Service

STEMSpark Hub

L&N STEM Academy Technology Integration

STEM Internships STEM Arc

Hub Branding

WTSC Hub

Southwind STEM HS STEM Ambassadors

Collaborative Leadership

MTSI Hub

Stratford STEM Magnet HS Student STEM Expo

STEM Instructional Designers


V. FINDINGS OVERALL FOR TSIN INVESTMENT – RESEARCH QUESTION 6: HOW DOES STEM TEACHING AND LEARNING CHANGE OR IMPROVE AS A RESULT OF THE TSIN? CLASSROOM OBSERVATION FINDINGS The Local Systemic Change Classroom Observation Protocol (LSC) was used to examine teacher effectiveness through classroom observations in four key areas: design of lesson, implementation of lesson, culture of instruction, and content knowledge delivered. Analysis of these videos revealed significant improvement in all four areas as indicated by findings presented below. Design of Lesson Domain An analysis of data for the THEC STEM PD programs (29 total programs) indicated there was significant growth in the Design of Lesson construct, which encompasses the extent of planning, organization, resources, equity, collaboration, flow, assessments, and sense making that takes place in the lesson delivery. At baseline, the mean score average (2.39) was rated a Level 2: Elements of Effective Instruction (M = 23.87, SD = 5.24), which increased to 2.67 (M = 26.67, SD = 4.84) at the second observation point midway through the professional development program, and increased further to (average score of 2.88) at the final observation (M = 28.82, SD = 5.35), F(2) = 39.59, p < .000. The effect size is considered large (p

2=.132), with 13.2% of the variance in Design of Lesson scores accounted for by time of the observation. Figure 2 shows the statistically significant overall increase in average Design of Lesson scores over time.


Figure 2. Design of Lesson Domain Average Score Over Time for THEC STEM PD Programs

Average scores could have an overall range of 10-50, since there are 10 items on a 5-point scale in this sub-section. Statistically significant increases were noted between all observation points.

State level findings did vary by type of program (e.g., mathematics, science, or grade range), meaning there was a statistically significant difference in Design of Lesson between program classifications, F(6) = 4.31, p < .000. The effect size is considered medium (ηp

2 = .117), with 11.7 percent of the variance in design of lesson score accounted for by type of program. The only significant differences in program type were between Middle School Math and several other program classifications. Middle School Math scores were significantly lower than Elementary Science (p < .000), Elementary Math (p < .05), High School Math (p < .01), and Middle School Math/Science (p < .05). The average design of lesson score across time ranged from 1.85 (High School Math/Science) to 3.20 (Elementary Math). These scores are equivalent to a Level 1: Ineffective Instruction and Level 3: Beginning stages of Effective Instruction respectively. There was also a statistically significant interaction between program classification and time of observation for Design of Lesson, F(12) = 2.50, p < .001. This means that the group overall improved over time. The effect size is considered medium (ηp

2 = .071), with 7.1 percent of the variance in design of lesson score accounted for by the interaction between observation time and program classification. Figure 3 shows that all program classifications increased in design score from baseline to mid-program observations and again increased from mid- to end-of-program observations.

p < .000

p < .000

p < .000


Figure 3. Design of Lesson Domain Average Score over Time by Program Classification for THEC STEM PD Programs

Average scores could have an overall range of 10 -50, since there are 10 items on a 5-point scale in this sub-section. The only significant differences noted over time were between Middle School Math programs, which were significantly lower compared to Elementary Science (p < .000), Elementary Math (p < .05), High School Math (p < .01), and Middle School Math/Science (p < .05).

Implementation of Lesson Domain Regardless of program classification, teachers involved in Tennessee’s Race to the Top STEM PD schools significantly improved their Implementation of Lesson scores from their average baseline rating of 2.61 or a Level 2: Elements of Effective Instruction (M = 18.30, SD = 4.12), to an average rating of 2.92 (M = 20.41, SD = 4.26) at the second observation, recorded at the mid-point of the professional development program. Scores finally rose to an average rating of 3.30 or a Level 3 at the end-point observation (M = 23.11, SD = 4.51), F(2) = 37.91, p < .000. The implementation of lesson construct considers the level of investigative mathematics/science in the lesson, quality of classroom management strategies, pace of the lesson, ability to modify instruction based upon student understanding, teacher questioning strategies, and formative assessments. The effect size is considered large (p

2 = .226) with 22.6 percent of the variance in Implementation of Lesson scores accounted for by time of the observation. Figure 4 shows the statistically significant increase in average Implementation of Lesson scores over time.


Figure 4. Implementation of Lesson Domain

Average Score over Time for THEC STEM PD Programs

Average scores could have an overall range of 7-35, since there are seven items on a 5-point scale for this sub-section. Statistically significant increases were noted between all observation points.

State level findings did not vary by type of program (e.g., mathematics, science, or grade range) in terms of Implementation of Lesson, meaning there was not a statistically significant difference in implementation of lesson between program classifications, F(6) = 1.94, p > .05. The effect size is considered small (ηp

2 = .057), with 5.7 percent of the variance in implementation of lesson score accounted for by type of program. The average implementation of lesson score across time ranged from 2.39 (High School Science) to 4.00 (High School Math/Science), scores that are equivalent to a Level 2: Elements of Effective Instruction and Level 4: Accomplished, Effective Instruction respectively. There was also not a statistically significant interaction between program classification and time of observation for design of lesson, F(12) = 1.17, p > .05. The effect size is again considered small (ηp

2 = .035), with 3.5 percent of the variance in implementation of lesson scores accounted for by the interaction between observation time and program classification. Figure 5 shows that all program classifications increased in implementation score from baseline to mid-program observations and again increased from mid- to end-of-program observations, and that none of the increases were significantly different by program over time.

p < .000

p < .000

p < .000


Figure 5. Implementation of Lesson Domain

Average Score over Time by Type for THEC STEM PD Programs

Average scores could have an overall range of 7-35, since there are seven items on a 5-point scale for this sub-section. No significant differences were revealed for programs across time by group.

Classroom Culture Domain The THEC STEM PD participants also significantly improved their overall Classroom Culture scores from baseline average rating of 2.84 or a Level 2: Elements of Effective Instruction (M = 17.03, SD = 4.06), to an average rating of 3.20 (M = 19.19, SD =3.89) on the second observation recorded at the mid-point of the professional development program. This rating increased to an average rating of 3.48 or a Level 3: Beginning Stages of Effective Instruction (M = 20.90, SD = 4.10) at the end-point observation, F(2) = 51.85, p < .000. The effect size is considered large (p

2 = .149), with 14.9 percent of the variance in Classroom Culture scores accounted for by time of the observation. Figure 6 shows the statistically significant increase in average Classroom Culture scores over time. Classroom Culture refers to the amount of active participation of all students and level of collaborative learning, including allowing students to explore their own ideas, questions, conjectures, and propositions or to challenge the ideas of others.


Figure 6. Classroom Culture Domain Average Score Over Time for THEC STEM PD Programs

Average scores could have an overall range of 6-30 since there are six items on a 5-point scale for this sub-section. Statistically significant increases were noted between all observation points.

There was a statistically significant difference in classroom culture between program classifications, F(6) = 6.52, p < .000. The effect size is considered large (ηp

2 = .150), with 15.0 percent of the variance in classroom culture score accounted for by type of program. The only significant differences in program type were between High School Science programs, and two other program classifications. High School Science scores were significantly lower than High School Math (p < .01) and Middle School Math/Science (p < .05) programs. The average classroom culture score across time ranged from 2.42 (High School Math/Science) to 4.00 (High School Math/Science), which are equivalent to a Level 2: Elements of Effective Instruction and Level 4: Accomplished, Effective Instruction respectively. A statistically significant interaction exists between program classification and time of observation existed for classroom culture, F(12) = 2.29, p > .05. The effect size is considered small (p

2 = .058), with 5.8 percent of the variance in Classroom Culture scores accounted for by the interaction of time of the observation and program classification. Figure 7 shows that all program classifications increased in classroom culture score from baseline to mid-observations and again increased from mid- to end-of-program observations.

p < .000

p < .000

p < .000


Figure 7. Classroom Culture Domain Average Score Over

Time by Program Classification for THEC STEM PD Programs

Average scores could have an overall range of 6-30 since there are six items on a 5-point scale for this sub-section. The only significant differences noted over time were between High School Chemistry programs, which were significantly lower compared to High School Math (p < .01) and Middle School Math/Science (p < .05) programs.

Mathematics/Science Content Domain THEC STEM PD participants significantly improved their Mathematics/Science Content scores from a baseline score of 2.90, Level 2: Elements of Effective Instruction (M = 26.09, SD = 5.04), to an average rating of 3.22 (M = 29.00, SD = 4.99) or Level 3: Beginning Stages of Effective Instruction at the second observation point, mid-way through the professional development program. By the end of the program, participants experienced further growth, with an average score of 3.50 overall (M = 31.52, SD = 5.46), F(2) = 56.25, p < .000. The effect size is considered large (p

2

= .159), with 15.9 percent of the variance in Mathematics/Science Content scores accounted for by time of the observation. Figure 8 shows the statistically significant increase in average Mathematics/Science Content scores over time.


Figure 8. Mathematics/Science Content Domain Average Score Over Time for THEC STEM PD Programs

Average scores could have an overall range of 9 – 45 since there are 9 items on a 5-point scale for this sub-section. Statistically significant increases were noted between all time points.

There is a statistically significant difference in mathematics/science content between program classifications, F(6) = 5.33, p < .000. The effect size is considered medium (ηp

2 = .126), with 12.6 percent of the variance in classroom culture score accounted for by type of program. The only significant differences in program type were between High School Math programs and two other programs classifications. High School Math programs scored, significantly higher than both Elementary Math and High School Chemistry programs (p < .05). The average classroom culture score across time ranged from 2.61 (High School Math/Science) to 4.00 (High School Math/Science), which are equivalent to a Level 2: Elements of Effective Instruction and Level 4: Accomplished, Effective Instruction respectively. There is not a statistically significant interaction between program classification and time of observation for mathematics/science content, F(12) = 1.35, p > .05. The effect size is considered small (p

2 = .035), with 3.5 percent of the variance in mathematics/science content scores accounted for by the interaction of time of the observation and program classification. Figure 9 shows that all program classifications increased in classroom culture scores from baseline to mid-observations and again increased from mid- to end-of-program observations.

p < .000

p < .000

p < .000


Figure 9. Mathematics/Science Content Average Domain Score

Over Time by Program Classification for THEC STEM PD Programs

Average scores could have an overall range of 9-45, since there are nine items on a 5-point scale for this sub-section. The only significant differences in program type noted over time were between High School Math programs, which were significantly higher compared to Elementary Math and High School Chemistry programs (p < .05).


VI. FINDINGS OVERALL FOR TSIN INVESTMENT – RESEARCH QUESTION 8: IS COMMUNITY AWARENESS AND SUPPORT FOR STEM INCREASED AS A RESULT OF THE TSIN? STUDENT OPPORTUNITIES IN STEM Research Question Eight of this two-year evaluation of the TSIN was focused on determining if the TSIN investment resulted in increased opportunities in Tennessee for K-12 students in STEM education. One item on the SCAS that was implemented to stakeholders across the state (e.g. 10,000 teachers proportionately distributed across the state, 4,000 faculty in higher education institutions, and all regional Chambers of Commerce members in Tennessee) was “Overall, there has been an increase in K-12 STEM education opportunities for students in the region in the last year”. An examination of these data from baseline (fall 2012) and at six month intervals (spring 2013, fall 2013, spring 2014) revealed growth in stakeholder perception of K-12 STEM student opportunities in Tennessee (Table 9). Specifically, for the overall population surveyed, the percentage agreement raised from 58.1% in fall 2012 to 70.7% in spring 2014. This growth was reflected in all three stakeholder groups (Business/Community Stakeholders, Higher Education Faculty, and K-12 Educators). Business/Community Stakeholder perception of K-12 student STEM opportunities increased from 61.8% (fall 2012) to 77.8% (spring 2014). Similarly, Higher Education Faculty agreement improved from 70.1% (fall 2012) to 74.1% (spring 2014). Finally, K-12 Educator agreement increased from 54.2% (fall 2012) to 60.1% (spring 2014). Interestingly, Higher Education faculty agreement shifted the least, indicating that this group may have been most aware of these opportunities at baseline and, through efforts of the TSIN Regional STEM Innovation Hubs, more Business/Community Stakeholders and K-12 Educators gained awareness of the opportunities for students in STEM. This increase in awareness was likely was coupled with an actual increase in opportunities as well.

Table 9. Percentage Agreement by Stakeholder Group: Increasing Student STEM Opportunities in Tennessee

Stakeholder Group Fall 2012* Spring 2013 Fall 2013 Spring 2014

Overall 58.1% (n=1,396) 63.8% (n=849) 60.3% (n=481) 70.7% (n=1,449)

K-12 Educators 54.2% (n=904) 60.0% (n=608) 52.4% (n=155) 60.1% (n=264)

Higher Education Faculty 70.1% (n=324) 75.6% (n=208) 59.0% (n=276) 74.1% (n=635) Business/Community

Stakeholders 61.8% (n=168) 78.6% (n=33) 69.4% (n=50) 77.8% (n=550)

*Baseline Sample


A further analysis of the SCAS data related to agreement with the statement “Overall, there has been an increase in K-12 STEM education opportunities for students in the region in the last year” was also conducted for each Regional STEM Innovation Hub collectively. Findings indicated that STEM Community Stakeholders in each region also reported increased agreement with the presence of more K-12 STEM education opportunities for students in each region (Table 10). The UCRSI Hub (Cookeville) experienced the largest gains across the two years (16.8%), while STEMSpark (Knoxville) produced the smallest gains (3.6%). However, all TSIN Regional STEM Innovation Hubs had at least 64% agreement by end of the second year, indicating most participants perceived more K-12 STEM student opportunities in the region. In fact, the UCRSI Hub, which demonstrated the most growth in agreement, also had the strongest agreement percentage (84.3%), followed by SETN (Chattanooga) with 79.7%, and ETSU (Johnson City) with 78.5%. Overall, perceptions of available STEM opportunities in Tennessee during the two-year period of this evaluation increased considerably.

Table 10. Percentage Agreement by Regional STEM Innovation Hub: Increasing Student STEM Opportunities in Tennessee

Hub (Location) Fall 2012* Spring 2013 Fall 2013 Spring 2014

SETN (Chattanooga) 62.9% (n=146) 67.2% (n=78) 53.7% (n=44) 79.7% (n=51)

UCRSI (Cookeville) 56.1% (n=162) 70.6% (n=101) 59.3% (n=96) 84.3% (n=86)

STEMSpark (Knoxville) 71.2% (n=190) 68.6% (n=107) 69.0% (n=60) 74.8% (n=89)

ETSU (Johnson City) 69.0% (n=198) 70.9% (n=124) 65.5% (n=78) 78.5% (n=102)

WTSC (Memphis) N/A N/A 52.6% (n=102) 64.0% (n=135)

MTSI (Nashville) 58.3% (n=387) 65.1% (n=207) 50.9% (n=89) 74.3% (n=165)

*Baseline Sample


VII. FINDINGS OVERALL FOR TSIN – RESEARCH QUESTION 9: IS COMMUNITY AWARENESS AND SUPPORT FOR STEM INCREASED AS A RESULT OF THE TSIN? STEM COMMUNITY AWARENESS SURVEY (SCAS) The STEM Community Awareness Survey (SCAS) was administered with a sample of K-12 teachers, higher education faculty, and members of the local business community in fall 2012 (baseline), and at three follow-up points: spring 2013, fall 2013, and spring 2014. Participants were invited by either direct email (K-12 and Higher Education participants) or by forwarded email messages from regional Chambers of Commerce (Business participants). The SCAS survey was delivered online via Survey Monkey. K-12 teachers and higher education faculty members were selected from provided and developed databases, and business community participants were recruited by regional chambers of commerce through distribution of the request to their entire membership email listserv. A total of 8,296 participants chose to respond to at least part of the survey. Data from these participants were considered usable for analysis if they completed at least one full section from the survey. There were 5,526 (64.2 percent) participants who completed the survey across the baseline in Fall 2012 (n = 2,446, 46.1 percent), and follow-up survey administrations in Spring 2013 (n = 1,349, 25.4 percent), Fall 2013 (n = 545, 10.3 percent), and Spring 2014 (n = 986, 18.2 percent). Although there were more participants at the time of the baseline distribution, samples from the baseline and follow-up data collections were similar in terms of types of participants. In all instances, K-12 teachers were the largest group completing the survey followed by higher education faculty and then business community members. Approximately two thirds of the samples were comprised of females, and about half self-identified as being from a rural location, followed by suburban, and then urban. Finally, STEM hub region representation was similar from baseline to follow-up survey distribution (Table 2). Change in Views of STEM Awareness and Support First, the overall state level change overall in views of STEM awareness and support for all stakeholders combined will be examined. Opinions regarding STEM education improved in all four constructs (Industry Engagement, Awareness and Resources, Preparation of Students, and Careers and Workforce) across the evaluation. Participants had a moderately favorable attitude toward STEM awareness and support on the entire survey, averaging “Agree” Overall at the end of the evaluation (Baseline M = 2.73; End M = 2.98). Additionally, participants improved their reported perceptions in the following survey subsections to a level of “Agree” by end of evaluation: Industry


Engagement (Baseline M = 2.73; End M = 3.04), Awareness and Resources (Baseline M = 2.91; End M = 3.16), and Careers and Workforce (Baseline M = 2.69; End M = 2.91). However, the survey subsection Preparation of Students had the lowest reported average scores, which improved but were not at an “Agree” level at the end of the evaluation (Baseline M = 2.44; Follow-Up M = 2.66). The TSIN made substantial progress in transforming Tennessee Stakeholder STEM Awareness in the areas of Industry Engagement, Awareness and Resources, and Careers and Workforce, though continued effort is needed in the Preparation of Students K-20. Table 11 shows results for the overall survey and each subsection, along with average net change from baseline to follow-up.

Table 11. SCAS Overall and Construct Comparison Baseline (Fall 2012) to End of Year Two (Spring 2014)

Survey Construct (points possible)

Fall 2012* M(SD)

Spring 2013 M(SD)

Fall 2013 M(SD)

Spring 2014 M(SD)

Average Net

Change Overall

(156 points) 106.61 (13.22) 107.12 (13.46) 105.44 (15.07) 116.30 (17.25) +9.69

Industry Engagement (IE) (32 points) 21.83 (4.21) 22.14 (4.17) 21.91 (4.83) 24.28 (4.74) +2.45

Awareness and Resources (AR) (52 points) 37.85 (4.97) 37.89 (4.79) 37.62 (5.52) 41.09 (6.77) +3.24

Preparation of Students (PR) (24 points) 14.65 (3.32) 14.73 (3.37) 14.16 (3.57) 15.96 (4.05) +1.31

Careers and Workforce (CW) (48 points) 32.31 (4.24) 32.46 (4.22) 31.73 (4.62) 34.88 (5.08) +2.57

*Baseline Sample


VIII. NARRATIVES FOR TSIN REGIONAL STEM INNOVATION HUBS AND STEM PLATFORM SCHOOLS Chattanooga: Southeast Tennessee STEM Innovation Hub (SETN) ......................... 35 Cookeville: Upper Cumberland Rural STEM Initiative ................................................ 59 Johnson City: ETSU Northeast STEM Innovation Hub .............................................. 86 Knoxville: STEMSpark East Tennessee STEM Innovation Hub ............................. 108 Memphis: West Tennessee STEM Hub (WTSH) ......................................................... 130 Nashville: Middle Tennessee STEM Innovation Hub (MTSI) .................................. 146


Hub Narrative Chattanooga: Southeast Tennessee STEM Innovation Hub (SETN) STEM School Chattanooga

HUB AND SCHOOL SETTING The Southeast Tennessee STEM Innovation Hub (SETN) is administered by the Public Education Foundation (PEF) in Chattanooga, Tennessee. The Managing Director of SETN during this evaluation was Tracey Carisch and the Director of Learning was Keri Randolph. The SETN Hub leadership structure includes a regional STEM Council that provides vision and insight on the overall direction of the STEM effort. Some organizations that are formal partners and have participants on the regional STEM Council include Chattanooga Chamber of Commerce, Cleveland City Chamber of Commerce, Chattanooga Regional Manufacturers Association, Chattanooga Technology Council, Enterprise Center, UNUM, Volkswagen, Wacker, Comcast, Electric Power Board, Girls Inc., Chattanooga Nature Center, Hunter Museum of American Art, Creative Discovery Center, Allied Arts, University of Tennessee at Chattanooga, Chattanooga State Community College, Rhea County Department of Education, Bradley County Schools, and the Hamilton County PTA Board. The goals of the SETN Hub were to:

1. Establish a regional STEM Hub that supports the platform school and maximizes regional impact by connecting and mobilizing existing and emerging STEM assets to improve STEM achievement across the educational pipeline.

2. Establish a STEM platform school to serve as an educational demonstration site for innovative practices in STEM teaching and learning.

3. Increase the capacity of the SETN STEM Hub to import and export innovations in STEM teaching and learning via active engagement with the central TSIN statewide infrastructure.

Four areas of strength were identified for the SETN Hub across the evaluation. First, the SETN Hub invested extensive energy into the building of regional STEM awareness and advocating for STEM education. This was apparent through the website (stemhub.wpengine.com) and through the public service announcements (PSAs) that were developed and are being delivered online and through the local cable provider Comcast. A second area of strength for the SETN Hub was in building capacity regionally for the implementation of STEM through the STEM Teaching Fellows. In this program, district level/building level STEM instructional leaders were identified and selected to attend a yearlong professional development program to build their STEM leadership and pedagogical content knowledge. The third area of rewarding investment for the SETN Hub was the marketing and financial support to the STEM Platform School (STEM School Chattanooga). This resulted in additional resources and visibility for the school. The final area of strength was the level and variety of differentiated PBLs that were implemented at STEM School Chattanooga across grades 9-11. The plans for juniors included having student teams choose a real-world problem based


upon their interest from a developed list of nearly 15 problems. Student teams work with industry and educational partners to develop potential solutions for the problem. The STEM School Chattanooga (SSC) is a grade 9-12 high school situated on the campus of Chattanooga State Community College in Chattanooga, Tennessee. This magnet high school recruits students from across the Hamilton County School District through a lottery process for the 75 freshman seats each year. The school began with a freshman class in 2012-2013 academic year and will add a class each year until a full complement of grades 9-12 is achieved. The mission of the school is to “develop and share a new paradigm for world-class education using technology as a gateway to cultivate students’ inquisitive nature, exercise innovation, think critically, and collaborate to become leaders who are self-sufficient learners with the same passion as Chattanooga’s Renaissance.” The first year of the school six teachers were hired, one each for science, social studies, mathematics, English, art, and technology. The students at SSC benefit from a flexible scheduling model, where students flow freely from room to room with no bells in an open-air classroom format. A fab lab was being established for students during the second year of the project that would be ready for students to utilize in fall 2014. Through the partnership with Chattanooga State there were growing numbers of dual credit opportunities for students at SSC in the second year of the evaluation. The curriculum model for the school is STEAM2, meaning the focus is science, technology, engineering, the arts, mathematics, and medicine. Students are engaged in project and problem-based learning (PBL) experiences with dedicated time in the regular school day for working on PBLs in teams with teachers serving as facilitators. All PBLs are purposefully integrated back into the academic content area classes. STEM School Chattanooga is a STEM integrated school, where all core content areas integrate STEM skills and content across disciplines through PBL. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. The SETN Hub was established under the fiscal leadership of the Public Education Foundation (PEF) residing in Chattanooga, Tennessee. STEM is one of the key areas of focus for PEF and the SETN Hub was positioned from the start to form a successful network through leveraging existing partnerships established through PEF. Many of the partners of the SETN Hub were already engaged with PEF. However, several new partnerships were cultivated during the first two years of the SETN Hub’s work. Beyond the establishment of a governing board for the SETN Hub, partnerships were targeted and aligned to the Business Partnership Strategy established by the SETN Hub. This strategy included focus areas of curriculum, STEM events, STEM materials, student internships, and teacher internships. As a result of this approach, many stakeholders who were interviewed shared that they did not have a full understanding of the entire operations or vision of the STEM Hub. This could be due to the centralized leadership structure, since the two directors conducted most of the work of the SETN Hub and there were no working groups established with stakeholders. However, there was intensive buy-in from partners and desire to move the region forward in the areas of STEM talent development, workforce development overall, post-secondary study, and economic development.


At the end of funding the SETN Hub had developed and was in the process of implementing a transition plan which included downsizing of the Hub staff (two directors to one) and scaling down operations to support programs that demonstrated the most potential for impact and revenue generation in the future. Partners will continue to be engaged through the work of the SETN Hub in the targeted activities beyond the funding of the grant. The SETN Hub transition is a very good example of implementation of a successful sustainability plan. The STEM School Chattanooga, the platform STEM school situated within the SETN Hub, has also established strong partnerships with Chattanooga State Community College (where the school is housed), along with multiple community partners who continue to work with teachers to provide unique educational experiences each year. Due to the fact that STEM School Chattanooga is a school within the Hamilton County School District and because of the extensive community partner engagement, the infrastructure has been established to sustain the partnerships far beyond the duration of the Race to the Top funding. In fact, many of the innovations within the school have been scaled up and are being implemented across the district. Plans are also being implemented to expand the school within the Chattanooga State Community College campus to include grades 9-12 as conceptualized. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question Two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). The Public Education Foundation (PEF) based in Chattanooga is the home of the SETN Hub. The SETN Hub is under the leadership of Managing Director Tracey Carisch and Director of Learning Keri Randolph. The SETN Hub has broad-based membership, including stakeholders from K-12 (Hamilton County and others), Higher Education (University of Tennessee – Chattanooga, Chattanooga State) and Business/Community (Chamber of Commerce, Comcast, TVA, Volkswagen, and other regional businesses). Through the series of four site visits (fall 2012/2013 and spring 2013/2014), four focus group sessions were conducted. These included a discussion of motivations of stakeholders for being involved in the work of the SETN Hub. The first focus group was well attended, and the second focus group interview included only two participants. The third focus group was conducted by phone due to scheduling conflicts during the third site visit. There were many participants in attendance for the fourth and final focus group (conducted on site) sessions. The primary motivations for stakeholder engagement in the regional STEM effort were consistent across the two-year evaluation. They are listed below in order of prominence were:

• Desire to develop future STEM talent for the region. • Commitment to workforce development and economic development

for the region and seeing STEM as a vehicle to move things forward. • Desire to grow STEM awareness for parents and the community. • Desire to be a part of a collective STEM reform effort. • Desire to grow excitement for students in STEM.


• Desire to improve level of college and career readiness in the areas of STEM. As the evaluation progressed from year one into year two, stakeholders placed less emphasis on items concerned with process and more on outcomes based motivations (mainly the top two bullets in the list). All other motivations (bullets 3-6) were considered to be inputs to the system with future talent and economic development identified as primary drivers of motivation for participation. There was considerable discussion of the progress for Chattanooga and moving away from patterns of previous decades when talented students would leave the area due to lack of options in the region. Stakeholders believed that Chattanooga had turned the corner on this and that the STEM effort would act as a catalyst to further Chattanooga’s efforts to become more innovative and a destination of choice for business. RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question Five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. The innovative approaches for the SETN Hub and STEM School Chattanooga (SSC) during the TSIN evaluation included the following:

1. STEM Teaching Fellows: This program led by Keri Randolph, Director of Learning for the SETN Hub, is focused on building STEM instructional leadership capacity across the region. There were 24 teachers included in the first year of STEM Teaching Fellows, and 38 in year two, representing 12 regional school districts. The STEM Teaching Fellows are provided with professional development (summer days and monthly meetings) during which they build their STEM pedagogical content knowledge expertise and build skills in forging local STEM partnerships for their district.

2. STEM public service announcements: The SETN Hub and STEM School Chattanooga have developed a series of commercials/short video clips online to support growing community awareness regarding STEM and STEM pedagogy. The videos feature administrators, teachers, and students from the STEM School Chattanooga in segments that teach the public about STEM and offer specific strategies for creating effective STEM learning environments.

3. Mastery approach: Rather than assigning a grade for mathematics to students and moving students forward as a group, SSC utilizes a technology platform that enables self-paced progress for students to complete modules. Teachers provide workshop time where they provide extra support for students struggling to master specific


concepts. This approach ensures mastery and moves to eliminate gaps in mathematics learning.

4. Technology Integration: 1:1 iPad technology at STEM School Chattanooga – All students at SSC have iPads, which they utilize throughout the school day for all aspects of their work.

5. Housing of STEM School Chattanooga on Chattanooga State Community College campus: This approach provides high school students with experience on a post-secondary campus physically, as well as through coursework (chemistry labs), and dual credit opportunities. The proximity facilitates a variety of ways of partnering on projects and delivery of coursework, as well as other experiences for students.

6. STEM PBL curriculum with each module/unit tied to a community partner: One of the PBLs partnered with Volkswagen, for example, and student teams were taken to the facility to present their concepts to teams of STEM professionals. This real-world connection moves beyond paper and simulated learning experiences to real-world settings with opportunities for students to gain exposure to careers, and for STEM organizations to learn more about the future talent pipeline. It also has provided multiple opportunities for students to engage in learning within industry and alternative settings. Further, as students progress through grades 9-12 there is more student choice included in the selection and completion of PBLs planned (beginning in 11th grade).

7. Innovate Education Channel (launched June 2013): A Hub-administered channel on Vimeo and Comcast designed to provide online video-based resources for educators, parents, and students to grow understanding and expertise in STEM. This online portal will provide teachers in the region an opportunity to showcase and disseminate their work in STEM.

8. Transition plan for sustainability: While not entirely innovative, the methodical approach to streamlining the work of the SETN Hub while maintaining strong infrastructure within the hosting organization (PEF) is an innovative approach for the TSIN. The SETN Hub has developed a streamlined, more targeted focus for efforts based on the activities that were the most impactful during the two years of Race to the Top funding. These will be continued into the post-funding years.

RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? The STEM School Chattanooga is a 1:1 iPad school, where the focus of technology is to create, not just consume. The STEM School Chattanooga has an integrated STEM curriculum approach. The first visit (baseline) for STEM School Chattanooga took place December 12-13, 2012. The second site visit was conducted May 7-8, 2013. The third visit October 10-11, 2013, and the final site visit took place March 20-21, 2014. Table 12 provides an overview of the ratings for STEM School Chattanooga for the four site visits conducted across the two-year evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.


Table 12. STEM School Chattanooga

STEM Attributes Ratings

STEM Attributes Fall 2012 Spring 2013 Fall 2013 Spring

2014

A1) Project-based learning with integrated content across STEM subjects. Target Target Target Target

A2) Connections to effective in and out-of-school STEM programs. Developing Advanced Advanced Target

A3) Integration of technology and virtual learning. Target Target Target Target

A4) Authentic assessment and exhibition of STEM skills. Advanced Advanced Advanced Advanced

A5) Professional development of integrated STEM curriculum, community/industry partnerships and postsecondary education connections.

Early Developing Developing Advanced

A6) Outreach, support and focus on underserved, especially females, minorities, and economically disadvantaged.

Advanced Advanced Advanced Advanced

B1) STEM work-based learning experiences, to increase interest and abilities in fields requiring STEM skills for each student and teacher.

Developing Advanced Target Target

B2) Participation in STEM Network. Advanced Advanced Advanced Target

B3) STEM Program Plan. Advanced Advanced Advanced Target

C1) Alignment of student’s career pathways with post-secondary STEM program(s). Early Developing Developing Advanced

C2) Credit completion at community colleges, colleges, and/or universities. Early Early Developing Advanced

Project-based Learning with Integrated Content (A1) During the first site visit (fall 2012) students at STEM School Chattanooga (SSC) were making team presentations of their final product (iMovie) associated with the problem-based learning unit focused on game design. In this PBL unit, students were presenting the concept for a game rather than an actual game design. For instance, in art class students presented their developed business cards for their game design company. Across the year, students at STEM School Chattanooga experienced six PBL units. The unit completed before the evaluation visit was focused on creating a prototype of an energy efficient home, and these prototypes were on display. Each day at SSC students have Tech Time during which they work with the technology teacher in the large common area. During this time the other content teachers have common planning time when they work in their STEM professional learning community to develop future units and discuss needed modifications and interventions. A key focus at SSC is on developing student collaboration, critical thinking, and innovation. This is reflected in the focus on PBL, as students have 45 minutes each


day to work on their PBL in teams in addition to time in core content classes, which also have connections to the PBL unit through content area academic standards delivered in the disciplinary curriculum. During the second site visit (May 2013) students were not actively working on a PBL unit. However, teachers were soliciting student input on their experiences with the five PBL units from the year to learn how to improve them for the next year. One of the most powerful things observed at SSC is the level of independent student learning. With the exception of one class, students were actively driving their own learning. This may be attributable to the mindset of constant integration of STEM through PBL at this school. At the third site visit (fall 2013) students were working on a PBL unit that required them to take a piece of artwork and recreate it innovatively using technology. There was clear evidence of student choice and voice within this PBL. The physical science class visited during this evaluation was conducted in a laboratory on the campus of Chattanooga State. The opportunity to have frequent exposure to post-secondary learning environments is an incredible experience for students at STEM School Chattanooga. The final site visit took place in April 2014. The ninth grade students were engaged in the PBL unit to designing an energy efficient house. The tenth grade students were working on a PBL simulation of a nuclear attack and developing a play that would animate their response to a potential nuclear attack. Plans are in place for increased student flexibility next year (2014-15) in which students in the eleventh grade will have a list of multiple PBL’s, each involving a community partner and both problem and project-based activities. In this new model, teams will self-select their group PBL units. SSC received the highest rating of “Target” for the A1 attribute at both the baseline (December 2012) and all follow up visits. This reflects the observation of project/problem-based learning used regularly, along with devoted time for common planning and development of a STEM professional learning community for teachers. The PBLs were observed as integrated across most core subject areas. Additionally, SSC created their units of instruction, which incorporated higher level thinking skills. At SSC, multiple spaces are available for collaboration for teachers, students, and external partners. Connections to STEM Programs (A2) SSC maintained a connection with the SETN Hub from the start of the school two years ago and has benefited from the marketing and fundraising work that the Hub has conducted. As a result, multiple partnerships have been formed to enhance the delivery of PBLs in real-world contexts and to provide field experiences (e.g., Volkswagen, Hunter Museum of Art). SSC is also fortunate to have Dr. Tony Donen at the helm. Across the first year of the school (and this evaluation) Dr. Donen worked to move teachers forward in accessing research and best practices and using data for school improvement. At baseline, SSC received a rating of “Developing” for A2, mostly due to the fact that the first three PBL units did not have several direct ties to external partners or multiple opportunities for students to participate in STEM learning environments outside of school. However, this is an area that improved across the first year, and in May 2013 (second site visit) SSC received an “Advanced”


rating in this area, as new partnerships were added and more purposeful use of data and research was observed for school improvement. STEM School Chattanooga continued to make extensive progress in this area across the second year of the school and achieved a rating of “Target” by the final site visit, as it was clear the school had engaged multiple partners in authentic learning experiences for students. Integration of Technology/Virtual Learning (A3) SSC was founded on a 1:1 technology platform, which utilizes iPads for every facet of learning, including self-paced mathematics classes and the creation of new products and innovations through the PBL units. The Algebra I classes had no physical textbook; rather, students accessed the Houghton Mifflin online book for the course, which was accompanied by instructional videos – creating a self-paced learning paradigm for mathematics. The focus in mathematics was on mastery, and each student worked independently through the materials and completed assessments as they were ready. However, students could not move forward until they mastered the materials and received the mastery score on the assessment. Each day, a common mathematics instructional time allowed for “workshop,” where the math teacher could pull students needing remediation on various areas. Edmodo was the platform of choice for SSC and was described as a “one-stop shop” for student assignments and calendars. Evernote was used for creation of folders to submit and organize assignments. Gaggle was used as the student email platform. Each classroom was equipped with Promethean boards on wheels on which students could “air-play” wirelessly to project their work from the iPad. Technological assistance was available on demand. It was determined that SSC was on “Target” in regard to the integration of technology attribute A3 at both baseline (December 2012) and all follow up site visits. Authentic Assessment (A4) From the beginning, SSC was designed to utilize multiple indicators of student success. These forms of assessing student progress ranged from traditional assessments to portfolios and project deliverables (e.g., iMovies, prototypes, reports, presentations). The mastery approach for teaching mathematics was innovative and student centered. On a regular basis, teachers examined data related to student progress and made instructional decisions accordingly. Many examples of student work were exhibited onsite, including PBL prototypes, artwork, 3-D printer products, and so forth. However, SSC had online exhibits of student work only occasionally during the first year. There was a strong culture of support for innovation among teachers, partners, students, and parents. SSC received a rating of “Advanced” in the Authentic Assessment (A4) attribute at both baseline and all follow up site visits. Professional Development (A5) Professional development opportunities for teachers at SSC were limited during the first year. This was likely due to the fact that the school had no funding allocated for professional development out of their state grant monies. Dr. Donen, the principal of SSC provided three days of PD the first summer prior to the school opening. This whole-group PD focused on planning their first PBL unit, as well as two days of support learning how to use the iPad, iBooks, and other technology. Topics


Donen focused on included how to establish a flipped classroom, grading, and collaboration. The focus on specific STEM content for specific types of learners was limited in these three days. The SETN Hub did not provide much in the way of PD support to SSC outside of their regionally focused work. Some individual teachers applied for district or other funding to attend conferences and other PD sessions. There were no individual PD plans for teachers established in the first year and the number of PD contact hours was minimal. Therefore, SSC received a rating of “Early” at baseline. This improved to “Developing” at the second site visit (May 2013) due to the increased focus by Dr. Donen on supporting teachers with resources on specific STEM content for specific types of learners. The second year of the school operation opened up a few more opportunities for teachers since two teachers joined the STEM Fellows program through the Hub, and there was more support through the Hub for teachers within the school, and others attended national professional development conferences. Therefore, the rating for STEM School Chattanooga progressed to “Advanced” at the final site visit. Outreach, Support, and Focus on Underserved (A6) SSC received a rating of “Advanced” for the A6 attribute at baseline (fall 2012) and at all follow up site visits across the two years,. This attribute is primarily focused on programs and policies that directly address engaging students from under-represented groups in STEM. SSC was established as a STEM magnet school within the Hamilton County School District and recruited students by lottery. As a result, the student population was diverse and consisted of students from a variety of ethnic/racial and socio-economic backgrounds. The environment of SSC included a strong culture of trust, inquiry and creativity for students, teachers, administrators, and partners, emphasizing the inclusion of all students in the learning process. There were policies and practices (including the regional lottery) that supported equity and access for all students. Further, through a partnership with Chattanooga State Community College and other partners, students learned about STEM career opportunities. Across the first year of SSC there were still only a few in-school programs that were designed to inspire under-represented and struggling students (e.g., engineering/robot project). However, this did improve somewhat in the second year of the school due to the growing number of partnerships and opportunities for students (e.g. Volkswagen). STEM Work-based Learning Experiences (B1) SSC experienced some growth from baseline to second visit (spring 2013) in this area. At the first site visit, SSC received a rating of “Developing” in regard to STEM Work-based learning experiences. Several PBLs were connected to current work in STEM industries and careers (e.g., viruses, climate change, game design, robotics, and energy efficiency). As the school year progressed, the frequency of these types of experiences increased. Teachers at SSC had limited interactions with other STEM professionals at the start of the year, however interactions increased by spring 2013 resulting in occasional collaborations with STEM professionals (e.g., Volkswagen, Hunter Museum). There were two components of B1 that did not change across the first year. Unfortunately, very few STEM teachers at SSC participated in customized applied learning experiences to increase their STEM pedagogical content knowledge. On a positive note, students at SSC were engaged on a daily


basis to work together and learn in teams to frame problems and test solutions through PBL. SSC moved up to a rating of “Advanced” for the second site visit (May 2013), due to progress in two of the five areas of this attribute. In the second year of SSC, more focus was devoted to this aspect of the STEM Attributes and SSC moved up at the third visit to a rating of “Target”, which was sustained at the final visit as well. There were noticeably more in-school learning experiences directly tied to current work in STEM industries and careers. Students worked on a daily basis in teams to solve problems, and teachers collaborated frequently with STEM professionals to develop new learning experiences for students. Participation in STEM Network (B2) SSC maintained a trusted collaboration with the SETN across the first year of this evaluation. The Hub provided exceptional opportunities for marketing and fundraising for the STEM Platform School, as well as connections to potential partners. Communication tools such as social media platforms (e.g., Twitter), newsletters, and websites were used by SSC to communicate externally on a frequent basis. The SETN Hub and leadership of SSC and Hamilton County School District meet at least semi-annually to discuss sustainability of the school. The rating for SSC at baseline and second site visit (May 2013) was “Advanced”. This relationship continued and strengthened across the second year, as the Hub was more active in supporting the school. Further, thanks to efforts by Dr. Donen, the SSC grew their own network of partners extensively in year two, bringing on targeted partners to engage with students on various PBL units (existing and planned for 11th grade). The rating for SSC on this construct both in fall 2013 and spring 2014 was “Target”. STEM Plan (B3) The leadership team of SSC is involved in building the STEM curriculum and STEM Program Plan one year at a time as the school grows by a grade level each year. During the first year of the evaluation, SSC was comprised of only freshman students. Due to the need to build the curriculum as the school year progressed, SSC was involved frequently with communication of an evolving program plan to potential partners and stakeholders across the year. However, SSC had access to on-demand and up-to-date student data on STEM performance to guide developmental work and necessary decision-making. SSC had access to discretionary funding to support the implementation of their planned STEM curriculum. SSC received a rating of “Advanced” at both baseline and second visit (May 2013) in this area. In year two, SSC did improve their access to data and used it to drive instruction. Teachers and partners worked collaboratively to build the plan for year two and three at SSC. The rating for SSC on this construct was raised to “Target” by the end of the second year (final site visit). Alignment of Career Pathways with Post-secondary (C1) During the first year of SSC, teachers did not vertically plan within and across grade levels and schools for two reasons. First, the team of six teachers was building the curriculum as they were teaching it for their freshman class. The planning for the sophomore curriculum did not take place until late in the school year. Second, the nature of the school as a STEM magnet made the scale of feeder schools so large that it would be difficult to plan vertically with middle schools. In the future,


however, SSC could work with the district to align with the middle school curriculum since it should be consistent across the district. SSC students had access to a career counselor who was onsite a few days a week during the first year if they sought her out. If not, it was estimated that students met with her at least quarterly to discuss potential post-secondary and career pathways. Meetings between teachers and the counselors were infrequent, estimated to occur annually. However, STEM teachers shared information with students occasionally regarding STEM careers as the year progressed. SSC received a rating of “Early” at baseline (December 2012) and “Developing” at site visit two (May 2013) for attribute C1 due to increased teacher involvement in informal STEM career counseling. In year two, SSC experienced growth in this area as well. Students had more access to career counselors and teachers also purposefully integrated STEM careers within the context of each PBL and beyond. The partnership with Chattanooga State was also a vehicle used in the second year to expose students frequently to post-secondary options and experiences. At the end of year two, SSC received a rating of “Advanced” in this area. Post-secondary Credit Completion (C2) There were no formal course offerings for dual credit available to the freshman class at SSC. As a result, fewer than 10 percent of students were enrolled in credit completion activities. However, career counselors and STEM teachers/administrators understand the importance of access to these opportunities and there are plans to offer dual credit coursework choices to students in their junior/senior year in partnership with Chattanooga State Community College. SSC received a rating of “Early” for the C2 attribute at both baseline (December 2012) and second site visit (May 2013). In year two, there was much progress in this area for SSC. In the fall there were plans in the works for dual-credit opportunities for students, and teachers were gaining a better understanding of the options for students. By the end of year two, plans were established for students to complete credit opportunities, and enrollment in this area for the following year had begun (e.g. Biology). SSC received a rating of “Advanced” on this construct at the final site visit/end of year two.


RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN? An examination of the SCAS findings related specifically to the SETN Hub was conducted by completing an analysis of the respondents who reported living in one of the counties included in the service area (e.g., Bradley, Grundy, Hamilton, Loudon, Marion, McMinn, Meigs, Monroe, Polk, Rhea, and Sequatchie). There were no Business/Community respondents for the second (spring 2013) administration of the SCAS. The number of participants who completed the survey at each administration from the SETN region included: 276 for fall 2012, 231 for spring 2013, 77 for fall 2013, and 115 for spring 2014. Overall, there was considerable growth in agreement regarding regional progress in STEM education for the SETN Hub. There were five main areas in which all three stakeholder groups experienced growth in agreement across the two-year evaluation.

• Stakeholders in the SETN Hub recognized the importance of Business/Community partnerships with K-12 education, as all stakeholder groups increased their agreement with this construct across the two-year evaluation (Business/Community 94% to 100%, Higher Education, 92% to 98%, and K-12 95% to 100%).

• All stakeholder groups increased in agreement regarding the availability of opportunities for K-12 students in STEM in the region (Business/Community 83% to 100%, Higher Education, 68% to 80%, and K-12 64% to 80%) from baseline to end of year two.

• The third area of agreement was related to K-12 schools understanding the importance of STEM education, as the Business/Community group increased from 86% to 100%, Higher Education increased from 63% to 83%, and K-12 improved from 81% to 85%).

• SETN Stakeholders reported growth in parental awareness of the importance of STEM education across the two-year evaluation (Business/Community 63% to 100%, Higher Education, 25% to 53%, and K-12 33% to 50%).

• The perception of availability of online resources related to STEM opportunities in the region was a final area of agreement across all stakeholder groups from baseline to end of year two (Business/Community 79% to 83%, Higher Education, 58% to 79%, and K-12 76% to 83%).

SETN Hub regional stakeholder groups had one common area of decline in agreement from baseline to end of the evaluation. This was in the area of volunteering and/or serving as a guest speaker in K-12 schools (Business/Community 78% to 67%, Higher Education, 94% to 73%, and K-12 77% to 61%). Both the K-12 and Business/Community stakeholder groups reported decreased agreement with the ability of K-12 schools in the region to effectively teach STEM knowledge and skills (Business/Community 42% to 33%, and K-12 58% to 41%). Tables 13-18 provide an overview of areas of growth in agreement and decline in agreement for each of the stakeholder groups (e.g., K-12, Business/Community, and Higher Education).


Table 13. K-12 Educators Areas of Growth in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

I believe it is important for area businesses to be involved in STEM partnerships with K-12 schools in my region.

95% 97% 93% 100%

I have had business/community funded STEM education programs or events in my school or school district.

46% 49% 52% 73%

I have had community/business volunteers for STEM programs or events in my school or school district.

48% 51% 48% 69%

There are opportunities for K-12 students to complete internships or co-ops in the region.

62% 56% 52% 73%

There are organizations interested in providing STEM education opportunities for K-12 students in this region.

71% 74% 52% 84%

Overall, there has been an increase in K-12 STEM opportunities for students in the region in the last year.

64% 66% 44% 80%

My school district understands the importance of STEM education. 81% 83% 76% 85%

Parents in this region understand the importance of STEM. 33% 39% 38% 50% There are colleges and/or universities and/or community colleges that offer scholarships for students to pursue STEM degrees in my region.

78% 75% 86% 86%

Information on regional STEM opportunities is available online. 76% 76% 67% 83% Local organizations recruit STEM talent online. 46% 48% 38% 50% Information related to STEM opportunities in my region is available online.

67% 67% 57% 80%

There are other STEM online tools available to this region. 69% 63% 62% 78% Students in this region are knowledgeable about the STEM careers that will be in high demand when they graduate.

44% 47% 39% 65%

K-12 public schools in this region effectively teach students STEM knowledge and skills.

57% 55% 39% 65%

The K-12 schools in this region prepare students who are critical thinkers and problem solvers.

62% 64% 57% 68%

There are businesses and industries that provide signing bonuses and/or incentives for workers choosing a STEM career in the region.

36% 41% 35% 65%

Organizations have experienced an increase in the number of STEM positions available in the last year in this region.

55% 59% 47% 65%

Organizations have been able to fill all STEM-related positions within the last year in this region.

32% 36% 24% 59%

Organizations have experienced an increase in the number of women and minorities in STEM positions in the last year in this region.

32% 40% 41% 59%

Organizations have been able to fill STEM-related positions with local STEM talent.

30% 34% 18% 62%

All K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be available in the region.

47% 61% 47% 74%

Career-oriented education is for all students. 78% 75% 71% 88%

Preparing students for careers in STEM is a top priority for schools in the region.

46% 51% 35% 65%


Table 14. K-12 Educators Areas of Decline in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2013

I have had community and/or business guest speakers in my school or school district. 77% 68% 63% 61%

I have worked closely with community/business organization members in my role as an educator. 43% 41% 41% 29%

The K-12 public schools in this region effectively teach students STEM knowledge and skills. 58% 55% 39% 41%

The state standardized tests used in this region’s K-12 schools adequately assess STEM knowledge and skills. 42% 37% 22% 28%

Table 15. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

I believe it is important for area businesses to be involved in STEM partnership(s) with K-12 schools in my region.

94% No participants 93% 100%

There are opportunities for K-12 students in our region to complete internships or co-ops in this region.




I have worked closely with community/business organization members in my role as an educator.


My organization understands the importance of STEM education. 71% No participants 100% 100%

The K-12 schools in this region understand the importance of STEM education. 86% No participants 71% 100%

Parents in this region understand the importance of STEM education. 63% No participants 43% 100%

Students with postsecondary education are more likely to secure a career in a STEM field. 90% No participants 86% 100%

There are colleges and/or universities that offer scholarships for students to pursue STEM degrees in my region.


Information related to STEM opportunities in my region is available online. 79% No participants 67% 83%

K-12 schools in this region prepare students who are critical thinkers and problem solvers. 58% No participants 57% 67%


Table 16. Business/Community Stakeholder

Areas of Decline in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

My organization has funded STEM education programs or events in K-12 schools in our region. 50% No participants 83% 33%

Employees from my organization have served as guest speakers for local K-12 schools. 78% No participants 100% 67%

My organization is interested in providing STEM education opportunities for K-12 students in this region.


Local organizations recruit STEM talent online. 74% No participants 38% 67%

There are other STEM online tools available to this region. 74% No participants 62% 67%

K-12 public schools in this region effectively teach students STEM knowledge and skills. 42% No participants 39% 33%

Community partners are engaged in making K-12 STEM education more relevant through providing real-world connections in this region.


Organizations have experienced an increase in the number of STEM positions available in the last year in the region.


Organizations have experienced an increase in the number of women and minorities in STEM positions in the last.


It is important for businesses in this region to be able to recruit skilled workers locally. 88% No participants 100% 75%

All K-12 students should have access to STEM education. 77% No participants 100% 50%

Preparing students for careers in STEM is a top priority for schools in the region. 80% No participants 29% 75%


Table 17. Higher Education Faculty Areas of Growth in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

I believe it is important for area businesses to be involved in STEM partnership(s) with K-12 schools in my region. 92% 96% 91% 98%

There are organizations interested in providing STEM education opportunities for K-12 students in the region. 78% 80% 68% 84%

Overall, there has been an increase in K-12 STEM education opportunities for students in the region in the last year. 68% 67% 57% 80%

I have worked closely with K-12 STEM teachers or administrators as a representative of my institution. 41% 44% 41% 59%

The K-12 schools in this region understand the importance of STEM education. 63% 80% 69% 83%

Parents in this region understand the importance of STEM education. 25% 40% 39% 53%

There are STEM education websites available for this region that include activities for K-12 teachers and students. 50% 80% 61% 78%

Local organizations recruit STEM talent online. 38% 40% 50% 52% Information related to STEM opportunities in my region is available online. 58% 76% 58% 79%

There are other STEM online tools available to this region. 55% 62% 69% 72% Students in this region are prepared by K-12 schools to be successful in postsecondary education. 38% 74% 55% 62%

Students in this region are knowledgeable about the STEM careers that will be in high demand when they graduate. 24% 45% 30% 57%



The K-12 schools in this region prepare students who are critical thinkers and problem solvers. 19% 61% 43% 57%


29% 39% 31% 58%

Organizations have experienced an increase in the number of STEM positions available in the last year in the region. 53% 80% 43% 60%


29% 40% 26% 52%

Organizations have been able to fill all STEM-related positions within the last year in this region. 29% 0% 20% 50%

Organizations have been able to fill STEM-related positions with local STEM talent. 29% 0% 20% 48%

K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be regionally available. 57% 40% 37% 62%


Preparing students for careers in STEM is a top priority for schools in this region. 31% 51% 29% 58%


Table 18. Higher Education Faculty Areas of Decline in Agreement on SCAS for SETN Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

My institution has provided volunteers for STEM education programs or events in K-12 schools in our region. 94% 80% 66% 73%

Information on regional STEM career opportunities is available online. 88% 80% 69% 81%

CONCLUDING OBSERVATIONS - Southeast Tennessee STEM Innovation Hub (SETN) and STEM School Chattanooga The Southeast Tennessee STEM Innovation Hub (SETN) is the only Hub in the TSIN led by a community organization, making it unique in its approach in the state. Being situated within an intermediary organization (not within higher education or K-12) provided some flexibility that enabled a concerted effort toward growing regional STEM awareness and advocacy. Across the two years of the evaluation of TSIN, SETN produced a series of STEM awareness videos and public service announcements (PSAs) that were disseminated widely across the region. SETN also focused on building capacity through the STEM Teaching Fellows program. The TSIN investment in SETN has resulted in the establishment of a STEM focus in the Chattanooga region and a sustained Hub presence through the Public Education Foundation, which has committed to supporting a scaled back staff that will continue to seek funding to conduct targeted STEM work. The STEM School Chattanooga (SSC) opened its doors in fall 2012 and two years later has made substantial progress to ensure the sustainability of this Hamilton County School. Space has been secured within the current site at Chattanooga State Community College to house the 9-12 school and construction is taking place to establish a state-of-the-art “Fab Lab” on site for students. In addition to a PBL-driven curriculum for students in 9th and 10th grades, SSC will further expand the STEM curriculum to provide student teams choice in the PBL units they explore beginning in 11th grade – each tied to a STEM industry partner for real-world, authentic learning. SSC is the best-in-class example within the TSIN of integrated STEM PBL learning at the high school level. This is key because high schools across the country are grappling with determining how to establish this approach within traditional schools. Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through their series of regional STEM innovation networks and STEM platform schools. Collective motivation is apparent within the SETN Hub. Stakeholders shared their motivations for being engaged in the work of the Hub, including a desire to grow STEM awareness, share STEM resources, contribute to workforce and economic development, and an overall desire to be a part of something larger that will grow student excitement about STEM careers. The SETN had a great need for a focus on capacity building and will need to continue this focus in the region beyond the Race to the Top Funding period, since stakeholder responses to the SCAS revealed a lack of confidence in the K-12 schools’ ability to effectively prepare students in STEM. The SETN STEM


Teaching Fellows was one of the innovative approaches that emerged from the Hub designed to address these capacity issues, and this program is one that SETN plans to continue beyond the initial grant funding period. STEM School Chattanooga faculty should also take the opportunity to become more of a resource to teachers in the region through providing professional learning opportunities for others in addition to the numerous building tours they provide annually. The SETN Hub has conducted extensive work in the area of creating new knowledge and strategies within the local context to develop collective understandings of STEM. For example, the STEM PSAs and other advocacy work, and the Education Channel for students, parents, and educators across the region has begun to pay early dividends, as stakeholders who completed the SCAS indicated growing agreement that parents in the region understand the importance of STEM. These pieces feed into changing the larger context in the region regarding the understanding of STEM and awareness of regional STEM needs tied to workforce and economic development. The SETN Hub has an extensive leadership team that meets quarterly and provides broad-based input into the direction of the work, creating a shared vision and ownership of the STEM reform effort that is truly viewed as an economic development strategy. However, moving forward, SETN should consider how to engage stakeholders in doing the work and creating the plans. Currently the group functions as more of a sounding board for input. SETN has achieved tri-level engagement, in which leadership from the STEM School Chattanooga and other regional districts, as well as multiple business/community/higher education stakeholders are engaged in the work of the SETN Hub. Overall, the TSIN investment in Southeast Tennessee’s STEM Hub and STEM platform school has resulted in the establishment of infrastructure that will provide support for the region (particularly students and teachers) in STEM for many years to come and should result in an enhanced STEM talent pipeline for the Chattanooga area.


Hub Narrative Cookeville: Upper Cumberland Rural STEM Initiative Prescott South Elementary School Prescott South Middle School

HUB AND SCHOOL SETTING The Upper Cumberland Rural STEM Initiative (UCRSI) is housed in the Millard Oakley STEM Center at Tennessee Tech University under the guidance of Dr. Sally Pardue. The UCRSI has a rural focus, and activities of the Hub have been designed to leverage the strong outreach foundation established by the Millard Oakley STEM Center over the past several years. There are 19 regional school district partners in the UCRSI, as well as Business/Community partners (e.g., Averitt Express, Highlands Initiative, USDA, and ORAU), and Higher Education partners, with Tennessee Tech being the main institution. The goals of the UCRSI are to:

1. Support the teachers of PK-8 learners with STEM resources to enhance classroom instruction.

2. Enact a sustainable design of STEM platform school and hub network that links the students and teachers with regional and national STEM assets.

3. Connect PK-8 students with their local community STEM experiences and stories. The objectives of UCRSI include to:

1. Establish a STEM platform school with the administrative and teacher support to implement uniquely crafted place-based thematic STEM modules and to use inquiry science throughout the school.

2. Partner with regional local education agencies to implement proven STEM instructional enhancements in a scalable fashion in other elementary schools.

3. Collaborate with regional STEM assets to provide human resources, financial funds, and/or material goods to the platform school and to the Hub’s network of elementary schools.

4. Enhance access to highly engaging STEM learning environments with a mobile STEM learning unit for use throughout the region.

5. Form STEM workgroups to develop STEM modules: place-based and problem-based learning units that are cross-correlated to science, mathematics, English-language arts, and social studies standards.

6. Form professional learning collectives for the UCRSI stakeholders. 7. Create quality video segments for virtual field trips as supplements and extensions

for the STEM modules. 8. Delivering informal, out of school learning opportunities for at-risk learners. 9. Engaging parents and families of elementary and middle school STEM learners in

community-based awareness events.


The website for UCRSI, located at www.ucrsi.org, highlights upcoming STEM events in the region, educator resources, and information on mini-grants, the STEMmobile, and other items to promote STEM in the region. Hub partners are not listed or recognized on the website. Users can create an account to access virtual space for collaboration on projects and so forth. The UCRSI Hub had four main areas of strength identified across the two-year evaluation. First, the support provided to the STEM Platform Schools (Prescott South Elementary and Middle School) to develop STEM curriculum modules was a key investment, though at the end of the second year these modules were not fully in use. Teachers at Prescott had begun to use the Legacy Cycle to plan and implement their own PBL units within their classrooms in addition to the modules, however. The second area of strength for the UCRSI Hub was in delivery of professional development in STEM for the region, and the conceptualization and implementation of the STEM Mobile. These assets will likely have major impact on both raising the level of STEM teacher expertise and providing high-quality STEM learning experiences for children in rural areas. The third area of strength for the UCRSI Hub was the partnership with Biz Foundry to provide Coding Camps for students across the region. The fourth area of strength for the UCRSI Hub is the leadership in rural STEM education (e.g. conference), since, within the TSIN, this Hub maintains the most focus on rural education. The Prescott Elementary STEM Platform School (grades K-5) and the Prescott Middle STEM Platform School (grades 6-8) are located in the Putnam County School District on the same campus in adjacent buildings in Cookeville, Tennessee, and both schools are supported by UCRSI. These two schools are housed in new facilities that are technology rich and supportive of collaborative and virtual learning. A STEM Coach was hired (Laura Luna), who is housed full time on site at Prescott South to support teachers in their implementation of STEM. The Prescott South schools are beginning STEM-enhanced schools, as they have included some STEM units and some individual STEM discipline content modules (e.g., science, mathematics, and engineering) into their teaching of core classes. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. The UCRSI Hub was established within Tennessee Technological University (TTU) and the existing Millard Oakley STEM Center (MO STEM), under the leadership of Dr. Sally Pardue, providing substantial established infrastructure and partnerships that were leveraged to ramp up work of the UCRSI Hub very quickly after selection for inclusion in the TSIN. MO STEM had many funded professional development and K-12 student programs underway that were folded into the work of the UCRSI Hub. The focus of the work of this Hub was to further establish the ability to reach out into the corners of the region through the provided funding and to build key components that would be sustained by TTU beyond the initial TSIN resources. The first component was the development of a vibrant website that would serve as a connectivity hub for teachers across the region – providing information and access to resources, as well as opportunities for professional development and networking. The second component was the design and development of the STEM Mobile – a mobile STEM laboratory designed to travel throughout the region to bring authentic STEM experiences to children in all schools. These two


key pieces, coupled with the long and strong reputation, partnerships, and work of the MO STEM Center have positioned the UCRSI Hub to be sustained long beyond the reach of the initial funding for the TSIN. The STEM Platform Schools in the UCRSI Hub are located within the Putnam County School District. The Prescott South Elementary and Middle Schools are housed within new, state-of-the-art facilities and both building leaders have strong commitments to the STEM focus. The UCRSI Hub has provided some support for the school in the form of professional development opportunities focused primarily on the Legacy Cycle and curriculum development. It is anticipated that the UCRSI Hub will continue to partner with the Prescott schools, along with other targeted community partners, to ensure these schools will continue to have sustained partnerships to continue the authentic STEM approach for the next several years. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question Two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). Through the series of four site visits (fall 2012, 2013 and spring 2013, 2014) four focus group sessions were conducted, which included a discussion of motivations of stakeholders for being involved in the work of the UCRSI Hub. Participants included stakeholders from K-12 (Putnam County and other districts), Higher Education (Tennessee Tech), and the Business/Community group (Chamber of Commerce, other regional business representatives). Primary motivations for stakeholder engagement in the regional STEM effort included:

• Affiliation with partnering school districts and/or employed at Prescott South Elementary and/or Middle schools.

• Desire to grow regional STEM awareness for parents and the community. • Interest in increasing exposure of students regionally to potential STEM careers in

the region. • Desire to be a part of a collective STEM regional reform effort. • Involvement in the STEM Mobile project (e.g., other K-12 district and higher

education stakeholders). • Commitment to workforce development and economic development for the region

and seeing STEM as a vehicle to move things forward. • Desire to integrate the teaching of critical thinking, problem solving in K-12 further. • Desire to develop future STEM talent for the region. • Desire to partner to provide integrated, real-world, STEM field experiences. • Desire to partner to secure support for funded STEM education programs. • Desire to improve the teaching and learning of STEM in K-12.


RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question Five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. The innovative approaches for the UCRSI Hub and Prescott South STEM Platform Schools included the following:

1. The STEM Mobile: A resource for the 18 school district partners in the UCRI region, this STEM Lab on wheels was conceptualized by TTU and realized through Hub and sponsor funding. The UCRSI Hub will manage the scheduling of the STEM Mobile, which will begin service fall 2013. The STEM Mobile will spend one week at each school it visits. Professional development will be provided for teachers prior to the use of the STEM Mobile.

2. UCRSI Rural STEM Education Conference: This conference provides a great opportunity for rural STEM teachers in the UCRSI region and beyond to learn about innovative approaches for teaching STEM in rural settings.

3. STEM Coach on-site at Prescott South: Having an on-site champion and facilitator for implementing STEM, particularly at schools transforming to STEM rather than opening new, is a great strategy. This is particularly true when the STEM teachers have little time to prepare for the STEM implementation. The STEM Coach can find and secure resources and engage in co-teaching with STEM teachers to help them build their expertise. During the second year there was some focus on growing teacher capacity, as the STEM Coach would not likely be supported beyond the TSIN grant funding.

4. Virtual learning studio: Due to the decreased funding for school field trips, virtual field trips and learning are economically sound options for bringing the real world to students.

5. Extensive out-of-school programs for students and parents: The extensive number of afterschool programs provides a great resource for parents and students who may not know how to gain more exposure to STEM careers and content for their children. It also doubles as a way to grow community awareness of STEM.

6. Meaningful, early STEM learning experiences: Prescott South is a great model for the state and beyond of engaging children as early as kindergarten in authentic, problem-based learning experiences.


RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? The Prescott South Elementary School and Prescott South Middle Schools are located in the UCRSI Hub, and both schools have a STEM-enhanced curriculum. The first visit (baseline) for the Prescott South schools took place January 17-18, 2013. The second site visit was conducted May 6-7, 2013. The third site visit was January 27-28, 2014, and the final site visit was completed April 7-8, 2014. Table 19 provides an overview of the ratings for Prescott South Elementary School, and Table 18 outlines the ratings for Prescott South Middle School for each site visit conducted for the two-year evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.

Table 19. Prescott South Elementary School

STEM Attributes Ratings

STEM Attributes

Fall 2012

Spring 2013

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects.

Advanced Advanced Advanced Target

A2) Connections to effective in and out-of-school STEM programs.

Developing Developing Developing Target

A3) Integration of technology and virtual learning.


A4) Authentic assessment and exhibition of STEM skills.



Developing Developing Developing Advanced



B1) STEM work-based learning experiences, to increase interest and abilities in fields requiring STEM skills, for each student and teacher.

Developing Developing Advanced Advanced

B2) Participation in STEM Network. Advanced Advanced Advanced Advanced B3) STEM Program Plan. Advanced Advanced Advanced Advanced C1) Alignment of students’ career pathways with post-secondary STEM program(s).

N/A N/A N/A N/A

C2) Credit completion at community colleges, colleges, and/or universities.

N/A N/A N/A N/A


PSES: Project-based Learning with Integrated Content (A1) Prescott South Elementary School (PSES) had limited implementation of project-based learning units during the first year. One module on population and water quality was implemented in fall 2012. This PBL unit involved grade 5 in a five-day investigation. Teachers at PSES were involved in working with Dr. Sally Pardue (UCRI Hub Director and TTU Millard Oakley STEM Center Director) on planning four modules for PK-2, and grades 5-6 focused on energy (1), water (2), and food/body/health (3) that will be implemented in the 2013-2014 year. Teachers received quarterly common planning time to work on these units with Dr. Pardue. The curriculum at PSES, outside of the one STEM unit for the 2012-2013 year, consisted of Delta and FOSS modules for science, which implemented inquiry-based learning for students. Over 76 percent of teachers at PSES made explicit efforts to integrate STEM into their core subjects. However, this was on an infrequent basis. For example, a one-day STEM-connected lesson was planned where all teachers would connect their content areas to STEM. There was also an engineering week planned, where teachers integrated engineering activities into their content areas. At PSES there were multiple facilities and spaces for collaboration (face-to-face and virtual). PSES received a rating of “Advanced” on the A1 attribute. In year two, PSES made a concerted effort for growth in this area, which resulted in observable progress at each of the two site visits (fall 2013 and spring 2014). Specifically, in fall 2013 project/problem based learning was not observed in many classroom visited. However, there was intensive time being invested in planning of collaborative PBL units to be implemented in the spring. The third site visit rating remained at “Advanced”. At the spring 2014 site visit, PBL was observed in all classes visited, beginning at kindergarten. This PBL unit was focused on having students develop a prototype of a new piece of playground equipment that could be installed at the school. This unit required students to learn about shapes, collaborate with others, and communicate ideas to their group and beyond. The culminating event included the design and production of a 3-D model of their favorite design that was presented to the building principal for approval. The rating for PSES at the final site visit was “Target”. PSES: Connections to STEM Programs (A2) PSES frequently engages with the UCRSI Hub, and there are partnerships with TTU, the Chamber of Commerce, Cummins Filtration, and others to enhance student learning. Direct experiences with experts in STEM fields are available for students at several points across the school year. Teachers and administrators at PSES occasionally access and share research and best practice related to STEM and use this data for program improvement. The STEM Coach, Laura Luna, invests a significant amount of time in building up a best practice resource base for teachers at PSES. The rating for PSES on attribute A2 at baseline (January 2013) and the second visit (May 2013) was “Developing”. In the fall of year two, PSES had begun to identify areas for improvement in this construct, however the site visit did not reveal enough data to constitute movement in the rating, which remained at “Developing.” However, across the school year more substantial connections were made with external partners (e.g. sound tile PBL for cafeteria) and a STEM career day was hosted. Both of


these were meaningful, real-world ways to engage students in working with STEM partners both in and out of school. The rating for PSES at the final site visit was “Target”, reflecting these improvements, as well as a stronger integration of best practice in the design of learning experiences for students. PSES: Integration of Technology/Virtual Learning (A3) PSES has a technology-rich learning environment with Promethean boards available in all classrooms, as well as access to iPad carts for each grade level for use with students. These common technology resources are connected to the curriculum and are being accessed by more than 86 percent of students. Some of the technology applications that are in use at PSES for teachers and students include Base Camp, Active-Votes, Active-Expressions, and Compass Learning. Teachers at PSES have on-demand access to digital instructional resources for STEM, and teachers receive regular resources notifications and updates from the STEM Coach, Laura Luna. There is also very good tech support available to the school with on-site support staff. PSES received a rating of “Advanced” on the A3 attribute at both baseline (January 2013) and the second site visit (May 2013). Year two marked a period of growth for PSES in the area of technology integration as well. In year one there were some delays in ordering and utilizing student technology (e.g. iPads), resulting in less implementation than planned. Teachers were provided more training on technology integration and offered more support from the STEM Coach regarding ways to utilize tools. At the fall site visit, teachers were still getting up to speed and those who were using technology during this visit had students using tools in a more traditional manner (e.g. typing, watching videos). The fall 2013 rating remained at “Advanced”. However, at the final site visit, clear progress was demonstrated. At this visit, students were using online simulations in science class and, in early grades, students were exposed to 3-D printing. The final rating for this construct in spring 2014 was “Target” for PSES. PSES: Authentic Assessment (A4) At PSES, under the leadership of Catherine Jones (Principal), a new grading system was implemented in the 2012-2013 school year focused on a standards-based approach in which each student is assigned a rating of either a level 1, 2, or 3. Students receive this rating for each standard that is taught in elementary school. Multiple forms of assessment are used. In addition to traditional assessments, students are rated using this new grading system (rubric) to track their progress toward mastery. Teachers at PSES have common grade-level planning time each day, and at the end of each nine weeks they discuss student assessment data. There are multiple on-site displays of student work. However, there are no online displays of student work/projects. Innovation is frequently honored at PSES. The rating for PSES on attribute A4 for baseline (January 2012) and the second site visit (May 2013) was “Advanced”. In year two of the evaluation, PSES teachers continued to utilize authentic assessment in an “Advanced” manner. However, student work was not celebrated through online or out-of-school exhibits and/or forums. Further, there was no scheduled time occurring monthly for teachers to collaborate to analyze student performance data and to develop multiple measures of student success with benchmarks and performance-based assessments. Therefore, the rating remained at “Advanced” at the fall 2013 and spring 2014 site visits.


PSES: Professional Development (A5) In terms of professional development (PD), PSES teachers participate in large group sessions available at the school. These have ranged from one-day trainings on iPads to Foss and Delta Science modules. Training was also provided on Engineering Is Elementary curriculum and Ag in the Classroom. Some teachers attended the NSTA STEM Forum and other individual TTU STEM PD. There is a job-embedded approach to PD at PSES, and the resources occasionally focus on specific STEM content (e.g., engineering) for specific types of student-learners. The teachers at PSES participated in 30 or more hours of PD as an average in the first year of the evaluation. The rating PSES received on the A5 attribute was “Developing” at both baseline (January 2013) and the second site visit (May 2013). In year two, professional development sessions continued much in the same fashion as year one, as evidenced by data collected at the fall 2013 site visit. There was little opportunity for teachers to develop individualized PD plans. The rating at the third site visit (fall 2013) was “Developing”. PSES administration recognized the need for improvement in this area and provided an opportunity for several teachers to attend the National Science Teachers Association (NSTA) meeting in spring 2014, where they were able to individualize their PD at this conference. Therefore, the rating for spring 2014 for PSES was “Advanced”. PSES: Outreach, Support, and Focus on Underserved Students (A6) The PSES environment incorporated a strong culture of trust, inquiry, and creativity, as expressed by the multiple administrators and teachers who were individually interviewed during the two site visits. PSES had in place policies and practices that support equity and access for all students. There was an extensive amount of STEM-focused student programs available to students at PSES. For example, the STEAM Academy (summer), field trips for multiple grades to the Millard Oakley STEM Center for STEM activities, STEM in the Library, Math Night, Tuesday Treasures, Full STEAM Ahead, CSI STEM Night, star gazing, and so forth. In these programs, PSES students, parents, and teachers had multiple opportunities to engage with STEM professionals in discussions and learn more about STEM careers. PSES received a rating of “Advanced” on the A6 attribute at both baseline (January 2013) and the second site visit (May 2013). The third site visit (fall 2013) did not reveal significant growth in this area and, as a result, the rating remained at “Advanced”. In the spring, the number of in and out-of-school programs made available to students at PSES focused on STEM had increased from fall (e.g. virtual field trips, Innovate Camp, video game design after school program, NOVA Making Stuff family event). Therefore, the rating for PSES reached “Target” at the final site visit conducted in spring 2014. PSES: STEM Work-based Learning Experiences (B1) PSES had several in-school learning opportunities across the 2012-2013 school year that were connected to current work in STEM industries. For example, the virtual field trip to the Denver Museum and to TTU to the Millard Oakley Center, as well as the PBL and other STEM focused lessons. Students had frequent opportunities to work and learn in teams, such as the Groundhog Day lesson in which students came up with ways to block the light to keep the groundhog from seeing its shadow. Unfortunately, few PSES teachers had the opportunity to participate in


customized, applied learning experiences to increase their STEM pedagogical content knowledge this year (2012-2013). However, teachers at PSES occasionally collaborated with STEM professionals, and this activity was growing near the end of the school year (2012-2013). The PSES rating for attribute B1 was “Developing” at both baseline (January 2013) and the second site visit (May 2013). PSES improved in this construct in year two of the evaluation. The increased focus on virtual field trip opportunities (e.g. Stupendous Sharks) and new partnerships with STEM professionals (e.g. Sound Levels Investigation) provided frequent opportunities for children. The rating for PSES for the fall 2013 visit was “Advanced” and this rating was retained at the final site visit in spring 2014. PSES: Participation in STEM Network (B2) PSES has worked closely with the UCRSI Hub, and across the 2012-2013 school year established a reciprocal and trusted collaboration with the regional STEM network. The UCRSI Hub has provided professional development on STEM curriculum development and other topics, connected partners to PSES, and provided field trips for students on multiple occasions. PSES uses communication tools frequently (e.g., websites and newsletters) and share events and achievements of the school externally. PSES leadership participates in the UCRSI Hub Advisory Board, which meets semi-annually to examine sustainability options. PSES received a rating of “Advanced” on the B2 attribute at both baseline (January 2013) and the second site visit (May 2013). The strong relationship between the PSES and the UCRSI Hub continued across the second year of the evaluation. However, communication tools continued to be used only on a frequent basis and external communication did not happen “regularly.” The rating for this construct in year two remained at “Advanced” for PSES. PSES: STEM Program Plan (B3) PSES does not have an articulated STEM Program Plan. The focus of the work at PSES has been on development of STEM modules and defining the STEM Program Plan as they move forward, with some refining along the way. However, PSES has been able to generate momentum for the STEM work through the communication of the vision for STEM at PSES with stakeholders and teachers. Further, student data are available for teachers to access, and are considered in curriculum planning and in PSES decision-making. Finally, funds from the STEM school grant have included discretionary funds that can be leveraged to advance the implementation of the STEM approach at PSES. The rating for the B3 attribute for both baseline (January 2013) and the second site visit (May 2013) was “Advanced.” In year two of the evaluation, PSES began efforts to integrate the use of PBL in more classrooms on a more regular basis. However, a fully articulated STEM Program Plan was not developed and in place. Therefore, the rating for this construct remained at “Advanced” for both the fall 2013 and spring 2014 site visits. PSES: Alignment of Career Pathways with Post-secondary (C1) Does not apply.


PSES: Post-secondary Credit Completion (C2) Does not apply.

Table 20. Prescott South Middle School STEM Attributes Ratings

STEM Attributes Fall 2012

Spring 2013

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects. Advanced Advanced Advanced Advanced

A2) Connections to effective in and out-of-school STEM programs. Developing Advanced Advanced Target

A3) Integration of technology and virtual learning. Advanced Advanced Advanced Target A4) Authentic assessment and exhibition of STEM skills. Developing Developing Developing Advanced


Developing Developing Developing Advanced





B2) Participation in STEM Network. Advanced Advanced Advanced Advanced B3) STEM Program Plan. Developing Developing Advanced Advanced C1) Alignment of students’ career pathways with post-secondary STEM program(s). N/A N/A N/A N/A

C2) Credit completion at community colleges, colleges, and/or universities. N/A N/A N/A N/A

PSMS: Project-based Learning with Integrated Content (A1) Prescott South Middle School (PSMS) had limited implementation of project-based learning (PBL) units during the first year, with only one module on population and water quality developed that was implemented in fall 2012. This PBL unit involved grade 6-8 in a five-day investigation. Teachers at PSMS were involved in working with Dr. Sally Pardue (UCRSI Hub Director and TTU Millard Oakley STEM Center Director) on planning four modules for grade 6 focused on (1) energy, (2) water, and (3) food/body/health that will be implemented in the 2013-2014 year. Teachers received quarterly common planning time to work on these units with Dr. Pardue. The curriculum at PSMS, outside of the one STEM unit for the 2012-2013 year, consisted of Delta and FOSS modules for science, which implemented inquiry-based learning for students. The middle school students were also involved in working with MP Components to design a method to remove the Otter Boxes from the iPads. This type of real-world collaboration with a STEM organization is exactly the type of problem-based learning that should be taking place at STEM Platform Schools.


Over 76 percent of teachers at PSMS made explicit efforts to integrate STEM into their core subjects. However, this was on an infrequent basis. For example, a one-day STEM connected lesson was planned where all teachers would connect their content areas to STEM. There was also an engineering week planned where teachers integrated engineering activities into their content areas. At PSMS there were multiple facilities and spaces for collaboration (face-to-face and virtual). PSMS received a rating of “Advanced” on the A1 attribute at both baseline (January 2013) and the second site visit (May 2013). In year two of the evaluation, PSMS retained their rating of “Advanced” across the year (fall 2013 and spring 2014). Teachers had at least quarterly planning time, but did not implement PBL frequently across multiple subjects. Further, classrooms that were observed did not include PBL experiences that focused on the promotion of higher-level thinking. PSMS: Connections to STEM Programs (A2) PSMS frequently engages with the UCRI Hub, and there are partnerships with TTU, Chamber of Commerce, Cummins Filtration, and others to enhance the learning of students. Direct experiences with experts in STEM fields are available for students at several points across the school year. The partnership with MP Components and the “Otter Box Project” is a great example of this. PSMS had other field trips and partnerships that enabled more experiences for students with STEM professionals. Teachers and administrators at PSMS access and share research and best practice related to STEM and use this data for program improvement. The STEM Coach, Laura Luna, invests a significant amount of time in building up a best practice resource base for teachers at PSMS. The rating for PSMS on attribute A2 at baseline (January 2013) was “Developing” but improved to “Advanced” at the second visit (May 2013). In year two of the TSIN evaluation, PSMS continued engaging with STEM program partners outside of the school for support of various projects (e.g. Sound Levels Investigations, Biz Foundry 3D printer training). The fall site visit revealed frequent but not continuous engagement with STEM partners and direct experiences with STEM professionals happened several times across the fall. The rating for the third site visit was “Advanced”, which improved to “Target” at the final site visit in spring 2014. This was due to the continual increase in frequency of opportunities and engagement of more STEM professionals in the school. PSMS: Integration of Technology/Virtual Learning (A3) PSMS has a technology-rich learning environment with Promethean boards available in all classrooms, as well as access to iPad carts for each grade level for use with students. These common technology resources are connected to the curriculum and are being accessed by more than 86 percent of students. Teachers at PSMS have on-demand access to digital instructional resources for STEM and teachers receive regular resources notifications and updates from the STEM Coach, Laura Luna. There is also very good tech support available to the school with on-site support staff. PSMS received a rating of “Advanced” on the A3 attribute at both baseline (January 2013) and the second site visit (May 2013).


Year two marked a period of growth for PSMS in the area of technology integration as well. In year one there were some delays in ordering and utilizing student technology (e.g. iPads), which resulted in less implementation than was planned. Teachers were provided more training on technology integration and offered more support from the STEM Coach regarding ways to utilize tools. At the fall site visit, teachers were still getting up to speed and those who were using technology during the site visit had students using tools in a more traditional manner (e.g. typing, watching videos). The fall 2013 rating remained at “Advanced”. However, at the final site visit, clear progress was demonstrated, as students were using online simulations in science class, and students were exposed to 3-D printing. The final rating for this construct in spring 2014 was “Target” for PSMS. PSMS: Authentic Assessment (A4) PSMS utilized a traditional grading format overall and approximately 51-75 percent of teachers at PSMS use multiple indicators of student success regularly (including PBL projects). Teachers at PSMS have quarterly collaboration time when they examine student data to inform instruction. High-quality student work is displayed on-site at PSMS but online exhibits are not in place yet, and it is not clear if there are plans to do so. The PSMS leadership does honor and encourage innovation for both faculty and students. The rating for PSMS on attribute A4 at baseline (January 2013) and the second site visit (May 2013) was “Developing”. This rating stayed the same in fall 2013 at the third site visit, as there was only occasional focus on student exhibits. PSMS made some progress between the third and fourth site visits, raising the rating on this construct to “Advanced”. More focus was devoted to celebration of student work on-site, online, and in other forums. PSMS: Professional Development (A5) In terms of professional development (PD), PSMS teachers participated in large group sessions available at the school. These have ranged from one-day trainings on iPads to Foss and Delta Science modules. Some teachers attended the NSTA STEM Forum and other individual TTU STEM PD. There is a job-embedded approach to PD at PSMS, and the resources occasionally focus on specific STEM content (e.g., engineering) for specific types of student-learners. The teachers at PSMS participated in 30 or more hours of PD as an average in the first year of the evaluation. The rating PSES received on the A5 attribute was “Developing” for both baseline (January 2013) and the second site visit (May 2013). In year two, professional development sessions continued, though there was little opportunity for teachers to develop individualized PD plans. The rating at the third site visit (fall 2013) was “Developing”. PSMS administration recognized the need for improvement in this area and provided an opportunity for several teachers to attend the National Science Teachers Association (NSTA) meeting in spring 2014, where they were able to individualize their PD at this conference. Therefore, the rating for spring 2014 for PSMS was “Advanced”. PSMS: Outreach, Support, and Focus on Underserved Students (A6) The PSMS environment incorporated a strong culture of trust, inquiry, and creativity, as expressed by the multiple administrators and teachers who were individually interviewed during the two site visits. PSMS had in place policies and practices that support equity and access for all students. An extensive number of STEM-focused student programs were available to students at PSMS. For


example, the STEAM Academy (summer), field trips for multiple grades to the Millard Oakley STEM Center for STEM activities, STEM in the Library, Math Night, Full STEAM Ahead, CSI STEM Night, star gazing, and so forth. In these programs, PSMS students, parents and teachers had multiple opportunities to engage with STEM professionals in discussions and learn more about STEM careers. PSMS received a rating of “Advanced” on the A6 attribute at both baseline (January 2013) and the second site visit (May 2013). PSMS continued the second year with providing opportunities for students to gain experience with STEM through in and out-of-school activities. The fall 2013 site visit findings indicated PSMS continued efforts at an “Advanced” level on this construct. PSMS made extensive progress across the year with multiple virtual field trip opportunities, Kite Festival, and other after-school programs for parents and students. The final rating for PSMS in this area was “Target”. PSMS: STEM Work-based Learning Experiences (B1) PSMS had several in-school learning opportunities across the 2012-2013 school year that were connected to current work in STEM industries. For example, the field trip to the TTU Millard Oakley Center, as well as the PBL and other STEM-focused lessons. Unfortunately, very few PSMS teachers had the opportunity to participate in customized, applied learning experiences to increase their STEM pedagogical content knowledge this year (2012-2013). However, teachers at PSMS occasionally collaborated with STEM professionals, and this activity was growing near the end of the school year (2012-2013). The PSES rating for attribute B1 was “Developing” at both baseline (January 2013) and the second site visit (May 2013). Year two of the evaluation revealed a few more teachers engaged in STEM workplace learning experiences. The rating for PSMS at the third and fourth site visits remained at “Advanced”, representing the connections with STEM professionals and students ability to frequently work and learn in teams at PSMS. However, more focus on customized learning experiences for teachers in STEM is an area that did not improve significantly. PSMS: Participation in STEM Network (B2) PSMS has worked closely with the UCRSI Hub, and across the 2012-2013 school year established a reciprocal and trusted collaboration with the regional STEM network. The UCRSI Hub has provided professional development on STEM curriculum development and other areas, connected partners to PSES, and provided field trips for students on multiple occasions. PSMS uses communication tools frequently (e.g., websites and newsletters) to share events and achievements of the school externally. PSMS leadership participates in the UCRSI Hub Advisory Board, which meets semi-annually to examine sustainability options. PSMS received a rating of “Advanced” on the B2 attribute at both baseline (January 2013) and the second site visit (May 2013). PSMS maintained the relationships with the UCRSI Hub and other STEM partners across the second year of the TSIN evaluation. The rating for PSMS for the fall 2013 and spring 2014 site visits was “Advanced”. PSMS: STEM Program Plan (B3) PSMS does not have an articulated STEM Program Plan. The focus of the work at PSMS has been on developing STEM modules and defining the STEM Program Plan as they move forward with some refining along the way. However, PSMS has been able to generate momentum for the STEM work through communicating the vision for STEM at PSMS with stakeholders and teachers.


Further, student data are available for teachers to access and are considered in curriculum planning and in PSMS decision-making. Finally, funds from the STEM school grant included discretionary funds that can be leveraged to advance the implementation of the STEM approach at PSMS. The rating for the B3 attribute for both baseline (January 2013) and the second site visit (May 2013) was “Advanced”. In year two of the TSIN evaluation, PSMS maintained the rating of “Advanced” in this construct. PSMS constantly communicated the STEM focus of the school and used data on STEM performance to inform instructional decisions. PSMS: Alignment of Career Pathways with Post-secondary (C1) Does not apply. PSMS: Post-secondary Credit Completion (C2) Does not apply. RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN? An examination of the SCAS findings related specifically to the UCRSI Hub was conducted by completing an analysis of the respondents who reported living in one of the counties included in the service area (e.g., Bledsoe, Cannon, Clay, Cumberland, DeKalb, Fentress, Jackson, Lebanon, Macon, Overton, Pickett, Putnam, Smith, Sumner, Trousdale, Van Buren, Warren, and White). The number of participants who completed the survey at each administration from the UCRSI region included: 448 for fall 2012, 249 for spring 2013, 162 for fall 2013, and 197 for spring 2014. Overall, there was considerable growth in agreement regarding regional progress in STEM education for the UCRSI Hub. There were seven main areas where all three stakeholder groups experienced growth in agreement across the two-year evaluation.

• All stakeholder groups increased in agreement regarding the availability of opportunities for K-12 students in STEM in the region (Business/Community 60% to 95%, Higher Education, 78% to 84%, and K-12 51% to 70%) from baseline to end of year two.

• Stakeholders in the UCRSI Hub recognized the importance of Business/Community partnerships with K-12 education, as all stakeholder groups increased their agreement that they had worked closely with partners across the two-year evaluation (Business/Community 50% to 82%, Higher Education, 48% to 67%, and K-12 38% to 45%).

• UCRSI stakeholders reported K-12 schools understand the importance of STEM education, as the Business/Community group increased from 78% to 94%, Higher Education increased from 82% to 89%, and K-12 improved from 78% to 83%).

• UCRSI Stakeholders reported growth in parental awareness of the importance of STEM education across the two-year evaluation (Business/Community 56% to 91%, Higher Education, 49% to 73%, and K-12 26% to 51%).

• All stakeholder groups reported increased agreement regarding the construct focused on K-12 schools in the region effectively teaching students STEM knowledge and skills


(Business/Community 45% to 51%, Higher Education, 46% to 60%, and K-12 54% to 68%).

• UCRSI stakeholders reported improved agreement that organizations have been able to fill STEM-related positions with local STEM talent (Business/Community 37% to 41%, Higher Education, 24% to 52%, and K-12 24% to 71%).

• All stakeholders reported increased agreement that preparing students for careers in STEM is a top priority for schools in the region (Business/Community 47% to 50%, Higher Education, 42% to 55%, and K-12 43% to 61%).

The UCRSI Hub regional stakeholders (Higher Education and Business/Community Stakeholders) experienced decline in agreement in a few areas. The K-12 Educator group did not experience decline in agreement in any area. The Business/Community group for UCRSI declined only in one area related to the engagement of community partners in making K-12 STEM Education more relevant through real-world connections (43% to 34%). There was decline in several areas for the Higher Education group, however only two items dropped below 50% agreement. The first item was the same one mentioned for the Business/Community group previously (real-world connections) which decreased from 57% to 42%. The second item dropped from 63% to 49% related to recruitment of STEM talent online. Tables 21-25 provide an overview of areas of growth in agreement and decline in agreement for each of the stakeholder groups.


Table 21. K-12 Educators

Areas of Growth in Agreement on SCAS for UCRSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

I have had business/community funded STEM education programs or events in my school or school district. 43% 57% 44% 63%

I have had community/business volunteers for STEM education programs or event in my school/school district. 37% 47% 56% 68%

I have had community/business guest speakers in my school or school district. 68% 71% 65% 80%

There are opportunities for K-12 students in our region to complete internships or co-ops in this region. 53% 59% 62% 70%

There are organizations interested in providing STEM education opportunities for K-12 students in this region. 60% 66% 68% 75%

Overall, there has been an increase in K-12 STEM opportunities for students in the region in the last year. 51% 60% 53% 70%





STEM skills are integral to student success today. 90% 94% 93% 94% Increasing the STEM talent pool is necessary for economic vitality. 92% 94% 93% 94%

Students with post-secondary education are more likely to secure a career in a STEM field. 90% 90% 90% 97%

There are colleges and/or universities and/or community colleges that offer scholarships for students to pursue STEM degrees in my region.

78% 84% 79% 83%

Local organizations recruit STEM talent online. 39% 36% 39% 54%

Information related to STEM opportunities in my region is available online. 56% 57% 57% 63%

Students in this region are prepared by K-12 schools to be successful in post-secondary study. 71% 75% 66% 88%




Community partners in this region are engaged in making K-12 STEM education more relevant through providing real-world connections.

40% 44% 47% 51%


21% 27% 22% 61%


Table 21. K-12 Educators Areas of Growth in Agreement on SCAS for UCRSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014



Organizations have experienced an increase in the number of women and minorities in STEM positions in the last year. 28% 31% 26% 75%

It is important for businesses in this region to be able to recruit skilled workers locally. 94% 90% 91% 96%

K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be available in the region.

49% 57% 44% 71%

All K-12 students should have access to STEM education. 89% 90% 70% 96%


Preparing students for careers in STEM is a top priority for schools in the region. 43% 46% 39% 61%


Table 22. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for UCRSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

My organization has funded STEM education programs or events in K-12 schools in our region. 62% 50% 43% 89%

My organization has provided volunteers for STEM education programs or event in K-12 schools in our region. 60% 67% 40% 86%

Employees from my organization have served as guest speakers for local K-12 schools. 70% 50% 60% 86%



Organizations are interested in providing STEM education opportunities for K-12 students in this region. 68% 40% 35% 91%

I have worked closely with K-12 schools in my role within my organization. 50% 33% 30% 82%

My organization understands the importance of STEM education. 88% 67% 79% 98%



K-12 public schools in this region effectively teach students STEM knowledge and skills. 45% 20% 53% 51%

K-12 schools in this region prepare students who are critical thinkers and problem solvers. 40% 0% 50% 44%


All students should receive information about careers that are expected to be in demand in this region when they graduate from K-12 schools and postsecondary institutions.

96% 60% 83% 100%

All K-12 schools in this region teach STEM skills and knowledge appropriate for jobs that will be available in the region. 48% 0% 44% 53%

Career-oriented education is for all students. 85% 80% 65% 88% Preparing students for careers in STEM is a top priority for schools in the region. 47% 33% 45% 50%

Stakeholders within community/business organizations have STEM skills and knowledge that could be an asset to K-12 schools in this region. 92% 83% 90% 97%

Table 23. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for UCRSI Hub

SCAS Item

Fall 2012

Spring 2013

Fall 2013

Spring 2014

Community partners are engaged in making K-12 STEM education more relevant through providing real-world connections in this region. 43% 20% N/A 34%


Table 24. Higher Education Faculty Areas of Growth in Agreement on SCAS for UCRSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014


There are opportunities for K-12 students to complete internships or co-ops in the region. 60% 62% 64% 82%






Students with postsecondary education are more likely to secure a career in a STEM field. 91% 90% 87% 97%






There are businesses and industries that provide signing bonuses and/or incentives for workers choosing a STEM career in the region. 31% 47% 31% 47%





All students should receive information about careers that are expected to be in demand in the region when they graduate from K-12 schools and post-secondary institutions.

87% 89% 88% 99%

K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be available in the region. 37% 50% 39% 61%

All K-12 students should have access to STEM education. 87% 89% 82% 99% Preparing students for careers in STEM is a top priority for schools in the region. 42% 45% 43% 55%

Students with post-secondary education are more likely to secure a career in a STEM field. 95% 88% 87% 97%



Table 25. Higher Education Faculty Areas of Decline in Agreement on SCAS for UCRSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

My institution has funded STEM education programs or events in K-12 schools in our region. 88% 98% 52% 79%

My institution has provided volunteers for STEM education programs or event in K-12 schools in our region. 93% 98% 56% 79%



There are colleges and/or universities that offer scholarships for students to pursue STEM degrees in my region. 90% 87% 83% 85%

There are STEM education websites available for this region that include activities for teachers and students. 87% 78% 76% 77%




There are other STEM online tools available to this region. 72% 82% 62% 65%

Community partners in this region are engaged in making K-12 STEM education more relevant through providing real-world connections. 57% 59% 53% 42%

Stakeholders within community/business organizations have STEM skills and knowledge that could be an asset to K-12 schools in this region.

87% 84% 82% 85%

CONCLUDING OBSERVATIONS - Upper Cumberland Rural STEM Initiative, Prescott South Elementary School, Prescott South Middle School The Upper Cumberland Rural STEM Initiative Hub (UCRSI) leveraged existing supports through TTU’s Millard Oakley STEM Center and funding through the TSIN to develop an infrastructure that will support STEM reform in this region for many years to come. The primary vehicle for sustained support will be the STEM Mobile that was conceptualized and constructed with support from UCRSI partners. The STEM Mobile will make the rounds delivering an authentic STEM learning environment to schools across the region each year. This, coupled with numerous opportunities through the Millard Oakley STEM Center for teacher professional development and student STEM learning, have strengthened the foundation of STEM in the Upper Cumberland Region. The two STEM platform schools within the UCRSI (Prescott South Elementary and Middle Schools) have made substantial progress across the two years of support as well. UCRSI has provided assistance to teachers, along with the STEM Coach Laura Luna, to transform the existing


schools to a new STEM approach. This was completed in small but powerful steps in year one that began with infusing more effective science teaching into the school through new curriculum (e.g. FOSS, Delta Science), and addition of engineering-focused components (e.g. Engineering is Elementary). In year two, Prescott South teachers received support from leadership to begin integrating PBLs, including some authentic challenges for the school that became teachable moments (Otter Box iPad case problem, sound issues in cafeteria). By the end of year two, Prescott appeared to be on the road to more fully implementing STEM attributes in addition to their extensive after-school and out-of-school STEM experiences for students. Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through its series of regional STEM innovation networks and STEM platform schools. Collective motivation for stakeholders within the UCRSI Hub for being involved in STEM reform was driven by a commitment to workforce and economic development to sustain the region. Stakeholders realized that a focus on STEM would attract new industry to the region that would further sustain the quality of life through economic vitality. There was also a growing desire reported on the SCAS to grow opportunities for students in STEM. Overall, regional STEM stakeholders on the SCAS were very positive in their responses and reported growing agreement with STEM efforts making an impact on the region in as early as year one, with increased agreement in year two as well. . Capacity building was a key area of focus for the UCRSI Hub. Not only was there focus on getting the two STEM platform schools up and running, but there was also a great amount of energy invested in pushing out STEM innovations to the region. The most innovative strategy emerging for the UCRSI Hub was the conceptualization and implementation of the STEM Mobile. A second piece was the extensive portfolio of STEM professional growth opportunities available through the Hub and TTU, including teacher and student Code Camps. In regard to creating new knowledge and strategies within the local context, the key activity for the UCRSI Hub is leading the STEM Rural Schools Conference each year, where K-12 and other stakeholders from across the region gain understandings of strategies for implementing STEM in a rural context. The UCRSI Hub has an extensive leadership team, including administrators from the STEM platform schools, that meets quarterly and provides broad-based input on the direction of the work, creating a shared vision and ownership of the STEM reform effort. Additionally, there is tri-level engagement, where leadership from the STEM platform schools and other regional districts, as well as multiple business/community/higher education stakeholders are engaged in the work of the UCRSI Hub. The ability of the UCRSI Hub to persist and be flexible to changes was evidenced in the rollout of the STEM Mobile. The work of the UCRSI Hub is aligned nicely with the TTU Millard Oakley STEM Center and the likelihood of sustaining efforts over the long term is very good.


Hub Narrative Johnson City: ETSU Northeast STEM Innovation Hub

HUB AND SCHOOL SETTING The ETSU Northeast Tennessee STEM Innovation Hub (ETSU Hub) is housed at East Tennessee State University (ETSU), under the leadership of Dr. Jack Rhoton. Business/Community partners who are members of the ETSU Hub include Eastman Chemical, Domtar Paper Mill, Wellmont Health System, Nuclear Fuel Services, Inc., Mountain States Health Alliance, Niswonger Foundation, LMR Plastics, Walmart Foundation, Snap-on, Aerojet, Pal’s, and ALO Industries. A 26-member advisory council comprised of stakeholders from industry, K-12, community college, and ETSU provides guidance for the work of the ETSU Hub. The mission of the ETSU Hub is to “interconnect K-12 schools, higher education institutions, businesses, foundations/non-profits, and community organizations to design, develop, and demonstrate innovative, sustainable and transferable STEM learning experiences. These STEM collaborations seek to engage students, teachers, develop a skilled workforce, and increase STEM literacy throughout the region.” The goals of the ETSU Hub are to:

1. Assist in the design, launch, and sustainment of the Kingsport/Sullivan County Platform School (Innovation Academy).

2. Amplify and accelerate the reach and impact of the STEM best practices, including people, tools, and networks.

3. Document effects of new learning experiences on short-term and long-term student learning, student motivation, and college and career trajectories and performance, with the intent to publish new findings in the education literature.

4. Coordinate and support efforts to seek funding from federal and private foundations to support STEM education models that prepare students for college and career.

5. Strengthen instructional practices in K-12 STEM Hub classrooms through professional development for teachers.

6. Develop a region-wide asset-mapping project, which will identify STEM resources available in Northeast Tennessee and support efforts to capitalize on these assets.

7. Provide a communication network to enhance sharing of expertise and strengthening existing collaboration while building new partnerships.

8. Disseminate STEM innovation and best practices with the TSIN and with other STEM Hubs in the state.

The essential activities of the ETSU Hub are to:

1. Train region’s STEM educators. 2. Disseminate STEM education innovation. 3. Serve as a voice of STEM in the community. 4. Cultivate partnerships.


The website for the ETSU Hub is www.netstemhub.com. This website includes information regarding programming for elementary, middle, high school students, and teachers, as well as post-secondary study. In addition, partnering businesses and foundations are recognized, and regional STEM assets available for educators are listed. There were three areas of strength identified for the ETSU Hub across this two-year evaluation. First and foremost, the intensive and extensive support to the STEM Platform School, Innovation Academy (IA), was nothing short of exceptional. The support included real-time, weekly professional development for teachers on STEM content and pedagogy, on-site STEM expertise for co-teaching and guest speaking, and connecting IA to potential partners, just to name a few. The exposure that students at IA received on a frequent basis to STEM professionals during the 2012-2013 and 2013-14 school year was extensive. A second area of strength for the ETSU Hub was the effort to grow STEM awareness in the region through a series of meetings with local stakeholders and efforts to leverage extensive partnerships to create a collective voice. This also included marketing the IA. The third area was serving as the go-to provider for an extensive portfolio of STEM professional development and K-12 student programs for the region through partnerships such as a long-standing relationship with Eastman Chemical (e.g., Mathletes and upcoming Scienceletes). To accomplish the work of the ETSU Hub, ETSU benefitted from work completed over the past decade in STEM areas, including an extensive network of established partnerships and infrastructure for dissemination across the region. Innovation Academy of Northeast Tennessee is a middle school located in Kingsport, TN, that is co-governed and administered by the Kingsport City School District and the Sullivan County School District. Teachers who were hired at the school in the two years of this evaluation were a mix of faculty from both districts. IA opened in fall 2012 with a sixth and seventh grade. In fall 2013, the school added the eighth grade to their facility. There were approximately 75 students per grade level in this small school setting, with close to 50 percent of the population coming from each partnering district. IA was a STEM-integrated school where the STEM integration took place across all content areas through PBL modules that integrate STEM skills and content seamlessly. In the second year of the evaluation, Kingsport City School District decided to withdraw from the administration of IA and Sullivan County School District decided to close the existing IA and integrate the students and some of the faculty into an existing larger middle school. It is unclear how much of the identity of IA will remain in the coming school year 2014-15. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. Stakeholders leveraged over a decade of engagement efforts and partnerships in East Tennessee to build the ETSU Hub, led by Dr. Jack Rhoton at ETSU. The synergies within the ETSU Hub resulted in the establishment of the platform school, IA, which was co-administered by Kingsport City Schools and Sullivan County School District. For this site, it is necessary to consider the two components, ETSU Hub and IA, as individual entities since the ETSU Hub established the necessary infrastructure that will enable partnerships to be sustained and to continue to provide STEM education support to the region for years to come. Most of the progress can be attributed to housing the STEM Hub within an institution of higher education


(ETSU) and also within a successful existing STEM education center (Center of Excellence in Mathematics and Science Education), and to leveraging previous and current funded projects (e.g. Mathletes, THEC STEM professional development, federal grants) to support teachers, schools, and students. On the other hand, IA had administrative challenges from the beginning due to the co-administrative structure. The school did not have an individual identity (IRN number) and normal operational procedures (grading, attendance, staff, reporting, access to funds) were cumbersome at best. Further, competing visions and agendas for the two school districts often distracted from the outstanding work taking place at the school. As a result, one school district pulled out near the end of the second year (Kingsport) and the other district decided to close the school and merge students into a larger middle school with only some of the IA staff included and without the IA principal’s leadership. Moreover, it is unclear how and if partners will continue to be engaged in IA and sustainability of the school partnerships with others is highly unlikely. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question Two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). The ETSU Hub began with many stakeholders at the table, leveraging the work of Dr. Rhoton and his outreach in the region over the past decade. In conversations with participants in focus group sessions, it was clear that many partners were engaged in the work of the Hub and Innovation Academy. Through conversations with many of those partners (e.g., Domtar Paper, Wellmont Health System, ETSU faculty, and Eastman Chemical Company) and other various stakeholders within the K-12 partner districts (Kingsport and Sullivan County), motivations for engagement with the Hub and Innovation Academy were revealed. Primary motivations for stakeholder engagement in the regional STEM effort included the following:

• Desire to learn more about STEM. • Desire to increase student participation in science and mathematics K-20. • Desire to grow the STEM talent pipeline to meet local STEM career needs. • Desire to provide early exposure for students to potential STEM careers in the

region. • Desire to be a part of a collective STEM reform effort. • Opportunity to identify and leverage available STEM resources in the region. • Excitement about Innovation Academy and desire to spread/replicate innovations to

other schools. • Desire to improve the regional educational system to better attract new employees

from outside of the area to move there for positions. • Desire to provide a better education and future for East TN youth.


RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question Five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. Several innovative approaches were identified for the ETSU Hub and Innovation Academy (IA) across the TSIN evaluation, which included the following:

1. Integrated STEM Curriculum: The curriculum at Innovation Academy was fully integrated within the core subjects (language arts, science, social studies, mathematics) and arranged around a series of problem-based learning (PBLs) units, which are tied to 10 of the top 25 innovations that have most impacted society. The curriculum was developed by a team of teachers from the school, in partnership with ETSU and other community agency partners.

2. Just-in-Time Professional Development: Professional development at Innovation Academy was designed and delivered “just in time,” where ETSU faculty met with teachers weekly to deliver support for content and strategies that are planned for the following week.

3. Co-Teaching Model: Partners of the school (ETSU, Domtar, Eastman, etc.) were involved in co-delivery of PBL instruction with teachers onsite regularly. This approach gave students critical access to STEM professional onsite, as well as provided on-the-job STEM professional development for teachers who grew their STEM content knowledge and ability to apply content to the real world.

4. Community Service Focus: Students at Innovation Academy were involved in community service, including making presentations to the city council and creating gifts for the elderly in local assisted living settings. This community service approach helped to instill the values of giving back to the community, which is particularly important in rural areas.

5. Integration of Technology: Innovation Academy is a 1:1 iPad technology school. They received the distinguished designation of Apple Distinguished Program. The access to technology has been a critical piece of the transformation of IA and other successful STEM schools nationally. One-to-one access is necessary for creating fully integrated STEM learning environments, providing immediate access to information, modes of communicating with others, and new content.


RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? Student-centered instruction was observed as a daily routine at Innovation Academy during both site visits across the 2012-2013 academic year. Innovation Academy (IA) was an STEM middle school with an integrated STEM curriculum approach. The first visit (baseline) for IA took place November 29-30, 2012. The second site visit was conducted May 2-3, 2013. The third site visit was completed October 23-24, 2013, and the final site visit was March 25-26, 2014. Table 26 provides an overview of the ratings for Innovation Academy for the four site visits conducted as part of this two-year evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.

Table 26. Innovation Academy STEM Attributes Ratings

STEM Attributes

Fall 2012

Spring 2013

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects. Target Target Target Target

A2) Connections to effective in and out-of-school STEM programs. Advanced Advanced Target Target

A3) Integration of technology and virtual learning. Target Target Target Target A4) Authentic assessment and exhibition of STEM skills. Developing Advanced Advanced Target A5) Professional development of integrated STEM curriculum, community/Community partnerships and postsecondary education connections.


A6) Outreach, support and focus on the underserved, especially females, minorities, and economically disadvantaged.

Developing Advanced Advanced Target


Advanced Target Target Target

B2) Participation in STEM Network. Advanced Target Target Target B3) STEM Program Plan. Advanced Advanced Advanced Target C1) Alignment of students’ career pathways with post-secondary STEM program(s). N/A N/A N/A N/A

C2) Credit completion at community colleges, colleges, and/or universities N/A N/A N/A N/A

Project-based Learning with Integrated Content (A1) During the first site visit (November 2012) students at Innovation Academy were immersed in a PBL unit focused on beginning a service-learning project tied to working with the elderly in assisted living. Additionally, some students were finishing presentations in social studies of their iMovies that had been developed for the previous project (Medicines, Vaccines, and Antibiotics). This project was focused on an outbreak of deadly disease forcing people in the U.S. to relocate to another country. Students were learning how to create a Glogster in language arts class. In science class some


students were receiving remediation in preparation for the TCAP assessment, while others worked on the service-learning project in stations. The second site visit (May 2013) revealed continued and more intensive focus on PBL. Instruction these two days was working outside on their Eco-Friendly PBL. A variety of partners (Domtar, ETSU) were actively engaged in instruction this visit and were co-teaching with the teachers in stations outside. Some of the activities included students incorporating mathematics to calculate speed and volume of water flow in the stream, using science to examine soil core samples and conduct dissolved oxygen testing, as well as counting macro invertebrate in the stream. Students used their iPads to collect data and display/communicate their findings. The second year of Innovation Academy was also rich with PBL experiences for students. The fall site visit revealed students working to build raised beds outside the school with community partners and also working with a landscape engineer to design the landscape surrounding the school. At the spring site visit, each grade level was working on its own PBL (for example, sixth grade were working on a PBL unit focused on helping refugees). PBLs were the guiding anchor for the entire curriculum at Innovation Academy. It was clear from observing students in action that this approach had permeated their entire learning experience. Innovation Academy received the highest rating of “Target” for the A1 attribute at both the baseline and all follow up visits. This reflects the observation of project/problem-based learning used regularly at Innovation Academy, integrated across all core subjects. Teachers at IA had daily common planning time, which well exceeded the required quarterly planning for the target rating. Additionally, IA teachers shared and created their units of instruction, which incorporated higher level thinking skills. At IA, multiple spaces were available for collaboration for teachers, students, and external partners. Connections to STEM Programs (A2) It was clear from both the baseline (November 2012) and all subsequent follow up visits that IA was tightly connected to the ETSU Hub and associated network and maintained multiple community, post-secondary, and business/community partnerships. In addition to Domtar, ETSU, and Eastman, IA targeted partnerships with individuals in the community, including a landscape architect who worked with the teachers to develop and implement their first PBL of the school year for sixth grade focused on learning about this career and developing a proposal for the landscape design in front of the school building. IA staff worked with ETSU regularly to access and share research on best practices and incorporate this knowledge into the plans for instruction for students and program improvement. Year two of Innovation Academy resulted in more partnerships and connections for authentic learning that included experiences for students outside of school. In addition, the faculty at IA was able to access more research and devote more time to integration of student data for program improvement. IA received a rating of “Advanced” for the A2 attribute at both baseline and the second visit. The second year of the evaluation, IA moved from an “Advanced” rating in fall 2013 to a “Target” rating at the final site visit (spring 2014).


Integration of Technology/Virtual Learning (A3) IA was founded on a 1:1 technology platform using the iPads as the main instructional tool for students. As a result, IA had achieved, with only a few exceptions, a paper-free learning environment for students. The immediate and constant access to information for students created a technology and information rich setting for learning. In addition to 100 percent of students having access to iPads, IA also has Apple TV and display devices in every classroom, enabling students to “air play” their work and create a collaborative learning community where discourse between students and with teachers was fluid. District-provided and teacher supported technological assistance was available on demand. It was determined that IA was on “Target” in regard to the integration of technology attribute A3 at both baseline (November 2012) and also for all the follow up visits. In fact, IA was presented the prestigious designation of Apple Distinguished Program in spring 2014 for their innovative use of technology. Authentic Assessment (A4) At baseline (fall 2012) IA teachers were being encouraged to use multiple indicators of student success. Teachers examined data related to student progress quarterly and made instructional decisions accordingly. Many examples of student work were exhibited onsite. There was a strong culture of support for innovation among teachers, partners, students, and parents. IA received a rating of “Developing” in the Authentic Assessment (A4) attribute at baseline (November 2012). The second visit to IA revealed progress in the A4 attribute. Greater percentages (51-75 percent) of teachers were actively using multiple indicators of student success (e.g., performance, project-based, and portfolio assessments). Teachers were collaborating at least monthly to examine data on student achievement and using results to inform their teaching. At the beginning of the year, teachers were more focused on developing the curriculum they would teach the third and fourth quarters, which left less time to focus on student data. The evaluation visit took place in May 2013 and provided a better reflection of work in regards to A4. There was also an increase in onsite and online exhibits of student work (newsletters). Students were developing videos, blogs, and other products of their work as additional forms of assessment in addition to traditional formats (e.g., Cooters and iExcel mathematics assessment). The rating for IA at the second visit (May 2013) was “Advanced” in the area of Authentic Assessment. This rating remained the same at the fall 2013 visit, as IA teachers continued effectively using authentic assessment to drive their instruction. Between fall 2013 and spring 2014, IA students, teachers, and administration devoted more effort toward celebrating high-quality student work both on-site, online, and in other outside forums and competitions. The rating for the spring 2014 final visit for Authentic Assessment for IA was “Target”. Professional Development (A5) In terms of professional development (PD), IA received a rating of “Advanced” at both baseline (November 2012) and second visit (May 2013). This was in part due to their innovative approach to PD with the real-time, job-embedded, weekly support provided by ETSU faculty. The PD for IA teachers focused specifically on STEM content and strategies for teaching the content to middle grades learners. The teachers at IA participated in 30 or more hours of PD across the first year of the school’s operation, with most focused on community/industry partnerships and connections to post-secondary education.


The one aspect of attribute A5 that IA lacked was individual teacher professional development plans. Most of the PD that teachers participated in was group focused rather than individualized for teachers. This was an area that IA began to work on in fall 2013, and some progress was noted at the third site visit. Many of the teachers participated in individualized professional development experiences during summer breaks and also attended national conferences and presented their work in the area of STEM integration. Therefore, the rating at the final site visit improved to “Target” based upon the extensive emphasis on professional growth through individualized learning and sharing and presenting to others their own STEM integration efforts. Outreach, Support, and Focus on Underserved Students (A6) IA received a rating of “Developing” for the A6 attribute at baseline (November 2012), which is primarily focused on programs and policies that directly address engaging students from under-represented groups in STEM. The focus of IA was to provide high-quality learning experiences in STEM for students from socio-economically diverse backgrounds in eastern Tennessee. The environment of IA included a strong culture of trust, inquiry, and creativity for students, teachers, administrators, and partners, emphasizing the inclusion of all students in the learning process. However, at baseline there were few programs (in and out of school) that specifically focused on building excitement and introducing STEM careers to students with diverse backgrounds. Some progress in this area was observed on the second visit to IA (May 2013). There were more experiences for students to engage with STEM professionals, both during and outside of the school day. IA received a “Target” rating for this attribute at the second site visit. During the second year of the TSIN evaluation, IA continued to foster a strong culture of trust, inquiry and creativity for students in a highly rigorous learning environment. In the second year there were even more opportunities for students to engage in learning outside of the school with STEM professionals. The rating of “Target” was maintained at the fall 2013 and spring 2014 site visits for IA. STEM Work-based Learning Experiences (B1) Due to the exceptional level of partner engagement at IA, the rating for B1 at baseline and second visit were both at the “Target” level. At IA, the series of STEM PBLs were frequently tied to current challenges in STEM industries and careers. Students worked in cooperative learning teams on a daily basis in their PBLs, focusing on solving problems with clearly defined expectations. Teachers from IA participated in customized PD and applied learning experiences with partners from ETSU in both onsite, real-time PD and also STEM PD opportunities at ETSU funded through the state. Additionally, IA teachers planned the IA curriculum collaboratively with STEM professionals to provide the real-world lens for teaching academic content standards through PBL. IA continued to excel in this area across the second year of the TSIN evaluation and was touted as the model of innovation for authentic partner engagement. Many new partnerships were cultivated in the second year. The rating for IA for the third and final site visits (fall 2013, spring 2014) was also “Target.”


Participation in STEM Network (B2) The ETSU Hub provided extensive support and partnerships for IA. The baseline visit (November 2012) focus group was comprised of partners ranging from county commissioners to science coordinators from other districts, Eastman Chemical Company employees, and ETSU faculty. The Governing Board of IA provided administrative leadership for the school focused on personnel issues, funding, and so forth. The curricular and partnership aspects of IA were led by ETSU with reciprocal and trusted collaboration with teachers and partners. The ETSU Hub worked to promote IA, and the school, in turn, promoted the efforts and programs of the Hub. The Governing Board of the school met quarterly to make decisions and provide governance for the school. At the time of the first visit, only one meeting had taken place. Communication tools for IA were in the process of being developed at baseline. A website for the school was in place but other communication tools were not used regularly at that point. The rating for B2 for IA at baseline was “Advanced”. The second visit to IA provided more examples of engagement between IA and the ETSU Hub. Teachers from IA had worked with ETSU to plan PD to share their curriculum and strategies with schools across the region for that summer (2013). Many teachers from IA attended the ETSU Spring Conference. IA worked to further build out its website and developed a newsletter (Drill and Driver) for the public. In addition, teachers and students presented at city council meetings and engaged in service learning in the community, which added further prominence to the work of IA and the ETSU Hub. The rating for B2 at the second visit (May 2013) was determined to be “Target”. In the second year of the TSIN evaluation, the ETSU Hub further established supports for IA and multiple paths for IA to engage in the STEM Network. Further, IA was actively engaged in the larger TSIN network as well, disseminating best practices from IA with others across the network. Sharing of curriculum, partnership models, and uses of technology were frequently documented across year two. The rating for IA at the third site visit (fall 2013) and the final site visit (spring 2014) were also “Target.” STEM Program Plan (B3) IA leaders decided to craft their curriculum around the theme of “innovations” rather than purchasing commercial curriculum for their school. This required leveraging the rich partnership base established by the ETSU Hub to design problems/projects that were driven from real STEM fields that were aligned with real-world problems. As a result, some curriculum was planned in the weeks prior to IA opening, and the rest was developed as the school year went on. Therefore, at baseline, the school did not have a final STEM Program Plan in place, outside of the curriculum design principles and overall vision of the school. In the process of going out into the community and seeking to establish more partnerships for the school, IA staff frequently communicated the vision of the school and generated more awareness and buy-in for the work of the ETSU Hub and the school. As the year moved forward, more detail was provided to the STEM Program Plan. A benefit of building the curriculum as the 2012-2013 academic year progressed was that teachers and administrators could assess student performance on the developed PBLs and determine future programmatic and instructional decisions based upon this data. One of the assets of IA was access to the grant funding, which provided some discretionary funding to advance the implementation of


the work. IA received a rating of “Advanced” on attribute B3 at both baseline (November 2012) and the second visit (May 2013). IA realized growth in this area across year two of the evaluation. At the fall 2013 visit, IA was beginning to engage stakeholders in development of the STEM Plan for expanding IA to include grades 9-12. The work had not commenced yet at the fall visit, therefore, the rating remained the same as in spring 2013 (Advanced). However, the planning group made extensive progress between fall and spring visits and had a fully developed STEM Plan for grades 6-12 at that time. IA received a rating of “Target” for the final visit in spring 2014. Alignment of Career Pathways with Post-secondary (C1) Does not apply. Post-secondary Credit Completion (C2) Does not apply. RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN? An examination of the SCAS findings related specifically to the ETSU Hub was conducted by conducting an analysis of the respondents who reported living in one of the counties included in the service area (Carter, Cocke, Green, Hamblen, Hancock, Hawkins, Johnson, Sullivan, Unicoi, and Washington). The number of participants who completed the survey at each administration from the ETSU region included: 461 for fall 2012, 313 for spring 2013, 116 for fall 2013, and 234 for spring 2014. Overall, all three stakeholder groups (K-12 Educators, Business/Community Stakeholders, and Higher Education Faculty) increased their agreement positively in most areas across the two-year evaluation. There were two main areas that all three stakeholder groups experienced growth in agreement in across the two years of the evaluation.

• All three ETSU Hub stakeholder groups reported growth in engagement with K-12 students and teachers across the two years of the evaluation, as Business/Community reported 59% agreement at baseline, which increased rose to 79% at end of the second year. Similarly, Higher Education faculty reported 25% growth (24% to 59%) and K-12 teachers reported a 9% increase (40% to 49%) in agreement.

• A second area of growth in agreement across the two-year evaluation for all three groups was the importance of providing career-oriented education for all students, and Higher Education reported 9% growth (80% to 89%), Business/Community reported 3% growth (87% to 90%) and K-12 recorded 9% growth (84% to 93%).

There were a few areas of decline across the two years for individual stakeholder groups. The K-12 Educator group agreement decreased (87% to 77%) in regards to their district understanding the


importance of STEM education and also reported that schools in the region also do not understand the importance of STEM education (83% to 77%). A final area of decline in agreement for K-12 was regarding K-12 school preparation of students to be critical thinkers and problem solvers (58% to 50%). This was also an area of decline for the Business/Community Stakeholder group (51% to 46%). Interestingly, the Business/Community group decreased in agreement with their expertise as an asset in the area of STEM skills and knowledge for K-12 schools (85% to 79%). Finally, Business/Community Stakeholders reported decreased agreement in the areas of use of online tools (61% to 54%), as well as to promote STEM job opportunities (60% to 56%) and recruit talent online (59% to 54%). Higher Education Faculty declined in agreement in the online area as well in the following areas: use of online tools (80% to 63%), online information regarding STEM job opportunities (81% to 67%), and recruiting talent online (66% to 62%). Higher Education also reported decreased agreement with the Business/Community Stakeholder expertise being an asset in the area of STEM skills and knowledge for K-12 schools (89% to 81%). Finally, reported engagement in K-12 schools as guest speakers (91% to 83%), volunteers (85% to 76%), and funded STEM education programs (82% to 74%) decreased across the two years for Higher Education Faculty. Tables 27-32 provide an overview of areas of growth in agreement and decline in agreement for each of the stakeholder groups.


Table 27. K-12 Educators Areas of Growth in Agreement on SCAS for ETSU Hub

SCAS Item

Fall 2012

Spring 2013

Fall 2013

Spring 2014

Community/business volunteers for STEM education programs in my school or school district. 54% 59% 49% 63%

Parents in the region understand the importance of STEM education. 32% 38% 33% 61%


STEM skills are integral to student success today. 92% 94% 85% 97% Increasing the STEM talent pool is necessary for economic vitality. 92% 93% 85% 100% Students in this region are prepared by K-12 schools to be successful in post-secondary study. 72% 77% 63% 81%

Colleges and/or universities offer scholarships for students to pursue STEM degrees in my region. 83% 77% 70% 90%

There are STEM education websites available for this region that include activities for teachers and students. 77% 76% 67% 84%



Students in this region are prepared by K-12 schools to be successful in postsecondary study. 72% 76% 82% 81%



36% 34% 29% 63%

Organizations have experienced an increase in number of STEM positions available in the last year in this region. 37% 41% 29% 75%

Organizations have been able to fill all STEM-related positions within the last year in the region. 30% 44% 43% 67%



All K-12 students should have access to STEM education. 89% 93% 91% 100% Preparing students for careers in STEM is a top priority for schools in this region. 44% 57% 52% 54%



Table 28. K-12 Educators Areas of Decline in Agreement on SCAS for ETSU Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

My district understands the importance of STEM education. 87% 84% 85% 77%

The schools in this region understand the importance of STEM education. 83% 81% 85% 77%


Table 29. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for ETSU Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014



My organization has provided volunteers for STEM education programs or events in K-12 schools in our region. 65% 38% 50% 80%




I have worked closely with K-12 STEM teachers or administrators as a representative of my organization. 59% 38% 50% 79%

My organization understands the importance of STEM education. 92% 86% 85% 98%


Parents in the region understand the importance of STEM education. 62% 43% 70% 90%



43% 14% 44% 48%



Table 30. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for ETSU Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014




There are other STEM online tools available to this region. 61% 86% 52% 54%

K-12 schools in this region prepare students who are critical thinkers and problem solvers. 51% 57% 63% 46%


90% 100% 100% 86%

All K-12 students should have access to STEM education. 90% 100% 91% 86% Stakeholders within community/business organizations have STEM skills and knowledge that could be an asset to K-12 schools in this region.

85% 100% 81% 79%


Table 31. Higher Education Faculty Areas of Growth in Agreement on SCAS for ETSU Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

There are opportunities for K-12 students to complete internships or co-ops in this region. 71% 73% 55% 86%


I have worked closely with K-12 teachers and administrators. 34% 36% 40% 59%


Parents in this region understand the importance of STEM education. 37% 39% 40% 67% Students in this region are prepared by K-12 schools to be successful in post-secondary study. 44% 34% 64% 56%






Organizations have experienced an increase in the number of STEM positions available in the last year in this region. 41% 64% 38% 63%

Organizations have experienced an increase in the number of women and minorities in STEM positions in the last year in this region. 39% 45% 36% 53%


Organizations have been able to fill STEM-related positions within the last year in this region. 33% 30% 38% 44%



Preparing students for careers in STEM is a top priority for schools in this region. 36% 33% 48% 50%


Table 32. Higher Education Faculty Areas of Decline in Agreement on SCAS for ETSU Hub

SCAS Item

Fall 2012

Spring 2013

Fall 2013

Spring 2014


Employees from my institution have served as volunteers for local K-12 schools. 85% 65% 59% 76%

Employees from my institution have served as guest speakers for local K-12 schools. 91% 95% 69% 83%

There are colleges and/or universities and/or community colleges that offer scholarships for students to pursue STEM degrees in my region.

85% 81% 77% 78%



Local organizations recruit STEM talent online. 66% 64% 52% 62% Information related to STEM opportunities in my region is available online. 81% 77%

56% 67%

There are other STEM online tools available to this region. 80% 78% 60% 63% Community partners in this region are engaged in making K-12 STEM education more relevant through providing real-world connections.

65% 59% 59% 59%

Stakeholders within community/business organizations have STEM skills and knowledge that could be an asset to K-12 schools in the region.

89% 91% 75% 81%

CONCLUDING OBSERVATIONS - ETSU Northeast STEM Innovation Hub The TSIN’s investment in the ETSU Hub and Innovation Academy (IA) platform STEM school produced the most innovative model of partner engagement within a STEM school in the state. The level of involvement in the day-to-day operations of the school required extensive time, commitment, and stamina. The ETSU Hub achieved this by partnering with IA teachers to support development and implementation of curriculum. Students and teachers at IA had frequent interactions with STEM professionals through co-teaching, guest speakers, field trips, and professional development opportunities. Beyond the ETSU Hub’s work with IA, a key component of the Hub’s work was to provide leadership for the region in growing STEM awareness through extensive STEM professional development opportunities for K-12 teachers. This is something that ETSU has been engaged with for many years and, notably, ETSU has administered some of the best professional development programs in the state. The ETSU Hub will be sustained as a part of the TSIN beyond Race to the Top funding since it is anchored in more than a decade of STEM education work at ETSU and continues to have a robust partnership structure to support K-12 STEM in the region.


Innovation Academy was the only TSIN middle school that achieved a fully integrated STEM PBL curriculum. IA received national recognition by Apple as a Distinguished Program in 2014 for their seamless integration of technology into daily instruction, one of only two TSIN schools to receive this designation. As the TSIN funding ended for IA, the existing school was closed and moved to a larger middle school in district. It is unclear how many staff will be retained. The building principal was not moved with the school. Unfortunately this school of great promise and accomplishment will not be sustained in current format beyond TSIN funding. Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through its series of regional STEM innovation networks and STEM platform schools. The ETSU Hub stakeholders’ collective motivation for being involved with the work of the Hub was focused on desire to improve K-12 educational opportunities for students in STEM to better prepare them for post-secondary and careers in STEM fields. The capacity building efforts of the ETSU Hub were extensive, and stakeholders who completed the SCAS reported growing agreement regarding the ability of K-12 schools in the region to prepare students for STEM positions across the evaluation. All stakeholder groups agreed there was more engagement in K-12 schools by business/community/higher education by the end of the evaluation. Further, there was more agreement across all groups that students in the ETSU region should have career-oriented education. The ETSU Hub has remained an active leader in creating new knowledge and strategies in STEM for the region. The ETSU Hub continued to sponsor an annual STEM Conference each spring at ETSU to showcase STEM innovations across the region. Further, the ETSU Hub offered many professional development programs in STEM the two years of the evaluation by leveraging funding from the Tennessee Higher Education Commission (THEC) and other sources. There has been much growth in interest from other K-12 schools in implementing STEM programs and approaches in the ETSU Hub region. IA staff may have contributed to this increased interest, as they provided professional development for teachers from around the region who wanted to learn more about their work in STEM each year. This creation of new knowledge and changing the larger regional context to see STEM as more accessible was another form of capacity building. The ETSU Hub has an extensive leadership team, including leadership from the STEM platform school, which meets quarterly and provides broad-based input into the direction of the work, creating a shared vision and ownership of the STEM reform effort. Additionally, there is tri-level engagement, where leadership from the STEM platform school and other regional districts, as well as multiple business/community/higher education stakeholders are engaged in the work of the ETSU Hub.


Hub Narrative Knoxville: STEMSpark East Tennessee STEM Innovation Hub L&N STEM Academy

HUB AND SCHOOL SETTING The STEMspark East Tennessee STEM Innovation Hub (STEMspark) is housed in the Knox County School District and led by Hub Director Marilyn Roddy. Business/Community partners in STEMspark include TVA, Innovation Valley, Inc., Youth Action Council, Great Schools Partnership, and Tennessee PTA. Higher education partners include the University of Tennessee, Knoxville, Pellissippi State Community College, Maryville College, Oak Ridge Associated Universities, Johnson University, Roane State Community College, and Walters State Community College. The STEMSpark Hub began as the KARST Hub and was led by Becky Ashe, whom also served as the Principal of L&N Academy. In the second year (and first year of this evaluation) Marilyn Roddy was selected as the Hub director. Roddy moved forward with renaming the Hub to STEMSpark and began working on marketing and branding the Hub. There were two site visits conducted (fall 2012 and spring 2013) for the STEMSpark Hub. K-12 partnering districts include Alcoa City Schools, Anderson County Schools, Blount County Schools, Campbell County Schools, Claiborne County Schools, Clayton Bradley STEM Academy, Clinton City Schools, Episcopal Schools of Knoxville, Grainger County Schools, Jefferson County Schools, Lenoir City Schools, Loudon City Schools, Maryville City Schools, Oak Ridge Schools, Oneida Special School District, Roane County Schools, Scott County Schools, Sevier County Schools, Tate’s School, Union County Schools, and Webb School of Knoxville. The goal of STEMspark is to leverage all available resources to amplify opportunities for all students. The activities of the STEMspark include:

1. Providing professional development for K-12 teachers in the 13-county region. 2. Offering K-12 students in the region STEM learning opportunities. 3. Making available STEM learning and career preparation activities for families and

students. 4. Offering curriculum and lesson plans showing implementation of STEM education

best practices and design thinking. 5. Establishing strategies and venues for sharing innovation with the region.

The website for STEMspark is www.stemspark.com. The website provides resources for partners, educators, and students/parents are available. Upcoming STEM events are profiled, and potential sources of support for STEM activities are linked. The majority of STEMSpark activities for the first year of this evaluation were focused on marketing of the Hub and building partnerships with K-12 school districts. A strategy was being developed in May 2013 to situate STEMSpark to become a regional provider of professional development in partnership with Pellissippi State Community


College. The strength of the STEMSpark Hub is the growing momentum in STEM within K-12 districts in the region. Many innovations are bubbling up, and STEMSpark is beginning to play the role as a conduit to connect the many stakeholders with interest in STEM. In the second year the TSIN evaluation, STEMSpark invested considerable time in revising the purpose of the Hub with support from external consultants, resulting in a new strategic plan. Marketing of the STEMSpark brand was a focus and Hub leadership filed for trademark protection. The main thrust of the work of STEMSpark in year two continued to focus on connectivity and resources. Some of the primary activities included delivery of teacher professional development (e.g. grant writing, Biology in a Box, Middle School Science Teacher Learning Academy, mini-grants) and communication of STEM opportunities regionally through a monthly newsletter. STEMSpark forged partnerships with 19 LEA’s in their region. L&N STEM Academy is the platform school associated with STEMspark. L&N operates as a magnet high school housing grades 9-12, which are selected through an open-lottery process. The curriculum of L&N Academy is centered on design principles, and all students take a series of year-long courses focused on STEM integration (STEM I-IV) across their high school trajectory. Within each of these STEM courses, students are engaged in project and problem-based learning and work in teams to solve real-world problems through inquiry-based instruction. L&N is a 1:1 iPad technology school and received Apple’s Distinguished School designation for the 2012-2013 academic year. All students at L&N will participate in an individually tailored STEM internship during their senior year. There are opportunities for advanced placement and dual credit coursework for all students. The L&N STEM Academy is a STEM-enhanced school where students are immersed in STEM through a series of STEM coursework that does not explicitly connect back to content area coursework. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. The STEMSpark Hub is situated within the Knox County School District under the leadership of Marilyn Roddy, Director, who assumed the position at the beginning of the second year of funding for the school (first year of evaluation). STEMSpark made progress in establishing a regional brand, and regional stakeholders were provided with information on available resources through various forms of communication. The main focus of partnership establishment was on K-12 school districts and higher education partners, and STEMSpark made progress in this area with 19 LEA partners and multiple community colleges (e.g. Pelissippi State) and the University of Tennessee in year two of the evaluation. STEMSpark did not establish strong business/corporation partnerships or secure external funding necessary for sustainability. Further, STEMSpark did not effectively leverage the innovative approaches from L&N STEM Academy (the STEM platform school within STEMSpark) as a resource and only very minor connections between the Hub and the STEM School were evidenced, since STEMSpark provided little to no support to L &N STEM Academy. The STEMSpark Hub model, with leadership situated within a Tennessee K-12 school district, presents a serious challenge to sustaining a Hub with the primary mission of serving the needs of K-12 and higher education across the region. It is clear that a local school district cannot commit local funding to a regional STEM Hub. Moreover, without success in


generating business/community buy-in for STEMSpark during the period of TSIN funding, the STEMSpark Hub had very little chance in being sustained. At the end of the TSIN funding, the STEMSpark Director (Roddy) departed. The school district is now working on a transition plan for the leadership of the Hub. One aspect of the STEMSpark work that will be sustained is the L&N STEM Academy, the STEM platform school situated within Knox County Schools. This award-winning school has forged many partnerships with business/community and higher education partners evidenced by the placement of all senior students within individualized, STEM-industry internships. Each of these partnerships was established through the work of the Dean of STEM within the school. L&N STEM Academy is a vibrant example of partnerships with mutually beneficial rewards that is situated to be sustained indefinitely. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question Two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). A stakeholder group was assembled for the four focus group sessions made up of participants from K-12 districts (e.g., Knox and Blount), Higher Education (e.g., Pellissippi Community College and University of Tennessee), and Business/Community (e.g., Girl Scouts and Oak Ridge Laboratory). A component of each focus group session was to have participants respond regarding their motivation for involvement in the STEMSpark Hub work. Motivations for stakeholder engagement in the regional STEM effort included the following:

• Desire to learn about and benefit from potential resources available for teaching of STEM

• Desire to leverage STEM as a tool for workforce and economic development. • Desire to grow STEM awareness for parents in the region. • Interest in being a part of a collective STEM reform effort. • Desire to address the K-12 STEM skills gap to better prepare students for post-

secondary study. • Desire to grow the relevance of K-12 STEM through connecting STEM

careers/industry partners to learning experiences • Desire to improve regional educational system to better compete for new industry in

the region.


RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question Five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. The several innovative approaches from STEMSpark and L&N STEM Academy across the TSIN evaluation, included the following:

1. Technology Integration: L&N Academy is a 1:1 iPad technology school. They have received recognition as an Apple Distinguished School (2012-2013) for their seamless integration of technology. This integration of technology is vital to the implementation of STEM real-world learning experiences that are the main component of STEM integration.

2. Series of STEM Coursework: The L&N Academy STEM Arc is a series of four yearlong STEM courses focused on engaging students in project and problem-based learning in a scaffolded approach with each unit tied to STEM partners. This model has been replicated in other states within high schools already and will continue to provide an innovative approach for integration of STEM within a traditional high school format.

3. Individualized STEM internships: All seniors at L&N STEM Academy are provided support to match their interests and experiences with an individualized STEM internship in business/industry. The extent of the partnerships established to support this is phenomenal. This is a model to learn from and replicate.

4. Hub Branding: This is an innovative approach that might lead to recognition of TSIN Hub efforts outside of Tennessee if structured correctly. Unfortunately, STEMSpark specifically may not benefit from this given the uncertain future of this Hub.

RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? L&N STEM Academy is a highly engaging, student-centered STEM-focused high school with a curriculum that is STEM enhanced. The first visit (baseline) for L&N Academy took place October 9-10, 2012. The second site visit was conducted April 30, 2013, and May 1, 2013. The third site visit took place on October 21-22, 2013, and the final site visit was May 8-9, 2014. Table 33 provides an overview of the ratings for L&N for the four site visits conducted for this two-year evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.


Table 33. L&N STEM Academy STEM Attributes Ratings

STEM Attributes Fall 2012

Spring 2013

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects. Advanced Advanced Advanced Advanced

A2) Connections to effective in and out-of-school STEM programs. Advanced Target Target Target

A3) Integration of technology and virtual learning. Advanced Target Target Target A4) Authentic assessment and exhibition of STEM skills. Developing Advanced Advanced Advanced


Developing Advanced Advanced Advanced





B2) Participation in STEM Network. Developing Early Advanced Advanced B3) STEM Program Plan. Advanced Advanced Advanced Target C1) Alignment of students’ career pathways with post-secondary STEM program(s). Early Early Developing Advanced

C2) Credit completion at community colleges, colleges, and/or universities. Developing Developing Advanced Advanced

Project-based Learning with Integrated Content (A1) The L&N STEM Academy provides students with an opportunity to participate in project and problem-based learning regularly through a series of STEM courses (I-IV), where students are enrolled in one, yearlong course each year in addition to their regular content coursework (e.g., Algebra, Physics, History, and English). The STEM course approach ensures a dedicated time for focus on STEM each day. The STEM courses are co-taught by two teachers with varying content area expertise. The curriculum is delivered in a spiraled approach, in which students work in teams to engage in problem-based scenarios involving community STEM partners, various online resources, and real-world constructs necessary for coming up with potential solutions or products. L&N has a 1:1 iPad environment, which facilitates student work on PBLs. The STEM course PBLs are not purposefully connected back into the content classes. However, some effort has been invested on an individual teacher basis to integrate STEM coursework into content coursework. At L&N, STEM course teachers receive common planning time every other day in addition to a 90-minute planning session once a month. Further, all teachers at L&N participate in professional learning communities once a week. The learning approach at L&N focuses on student direction, student independence, application of principles that are learned, and the ability to professionally demonstrate learning.


Multiple facilities and spaces are available for face-to-face and virtual collaboration among students and teachers (including virtual collaboration). L&N Academy received a rating of “Advanced” on the A1 attribute at both baseline (October 2012) and second visit (April 2013). In year two of the evaluation, L&N STEM Academy continued to provide daily opportunities for students to engage in PBL within the STEM course series. Business/industry partners were fully engaged in working with student teams and students continued to develop innovative solutions to real-world problems. However, PBL projects were not integrated back into the content area coursework. Such integration would broaden student opportunities to learn traditional content within this type of pedagogical environment. Leadership indicated that this will be the next step for the school and they are actively seeking resources to support the intensive work that will be required to make this happen. The rating for L&N remained at “Advanced” for this construct at both the fall 2013 and spring 2014 site visit. Connections to STEM Programs (A2) Connections to STEM Programs is an attribute in which L&N experienced growth across the first year of this evaluation (2012-2013). L&N has continuously engaged in the STEM network (both STEMSpark and TSIN) since its inception. In fact, teachers and administrators from L&N have provided leadership for both, as they have shared their expertise through professional development sessions, school visits, and coaching support for others in STEM. L&N has actively worked to build their partnership base, mostly independent of the STEMSpark Hub. The rich partnership base of L&N has produced exceptional learning opportunities for students at L&N through the STEM courses and beyond (e.g., guest speakers, guest judges of PBL products, field experiences, internships, field trips, and competitions). Students at L&N have frequent, direct experiences with STEM professionals and STEM learning environments (e.g., UT) available across the school year. The use of research and best practices became more frequent across the 2012-2013 year for L&N. The administrative team of Principal and Asst. Principals/Deans has developed a supportive system for providing access to research, best practice, and data for teachers to use in individual course and school improvement. L&N received a rating of “Advanced” at baseline (October 2012), which moved up to “Target” at the second site visit (April 2013) on attribute A2 due to progress in this area. L&N STEM Academy continued to be deeply engaged in the work of the TSIN overall during the second year of this evaluation. Many members of the administration and staff worked with TSIN to share the innovative approaches that were in use at the school. Many of the other STEM schools in the network also visited L&N (e.g. Innovation Academy, STEM School Chattanooga) to learn more about how to enhance their programming. Faculty at L&N continued to regularly access research and best practice through national and local conferences and book studies. L&N also continued to infuse more and more direct experiences for students with STEM professionals. The rating for L&N remained at “Target” for the fall 2013 and spring 2014 site visits.


Integration of Technology/Virtual Learning (A3) The L&N STEM Academy has a 1:1 iPad environment for students, which provides a learning platform that enables access to information for consumption and creation necessary for the STEM and PBL focused school. Technology tools are integrated seamlessly into teaching and learning as students use their iPads for creating presentations, research, note taking, and accessing online portals for assignments and communicating with others. Each classroom at L&N has SmartBoard technology, which teachers use to facilitate the learning process. Additionally, students frequently “air play” their work to share and engage in discourse with others. The chemistry course at L&N has a flipped classroom approach in which students watch videos of the lesson presented by the teacher prior to coming to class. Class time, therefore, is focused more on dialog between teacher and students and time is allocated to work on problems and labs in a student-centered format. At baseline (October 2012), L&N received a rating of “Advanced”, which moved up to “Target” at the second site visit (April 2013). This was due to progress in the frequency of availability of digital STEM instructional resources for teachers, which improved, and the access to tech support, which increased due to the growing expertise of teachers onsite. Year two of the evaluation revealed that technology integration continued to be a main thrust of the daily operations at L&N STEM Academy. Students were actively using technology to create solutions rather than just consume information. In the STEM III class, student teams conceptualized a new business and developed virtual marketing materials, including webpages, and presented these to a panel of STEM professionals. The rating for L&N at the fall 2013 and spring 2014 site visits remained at “Target”. Authentic Assessment (A4) The focus on authentic assessments and exhibition of STEM skills at L&N was another area that improved across the first year of this evaluation. At baseline (fall 2012), only a portion (51-75 percent) of teachers at L&N used multiple indicators, including performance, project-based, and portfolio assessments. Additionally, opportunities to collaborate and examine student data on multiple measures of success happened only quarterly. By spring 2013, the majority of teachers at L&N used authentic assessments and collaborated monthly to examine student performance results to inform instruction. It was clear at the baseline visit that L&N STEM Academy frequently honored student and faculty innovation. This progressed to a consistent basis, and some incentives for innovation were observed at the second site visit (spring 2013). However, celebration of high-quality student work through on-site and online exhibits occurred only on an occasional basis. L&N received a rating of “Developing” at baseline, which moved up to “Advanced” at the second site visit due to progress on attribute A4. L&N STEM Academy continued to immerse students daily in authentic learning environments, which included many forms of authentic formative assessments. The rating for L&N remained at


“Advanced” across year two due to the only occasional emphasis on celebrating high-quality student work in on-site and on-line exhibits. Professional Development (A5) Professional development (PD) at L&N Academy during the first year of this evaluation was mostly focused on large group delivery of a variety of strategies aligned with STEM and technology integration delivered through a job-embedded approach. The STEMSpark Hub did not provide specific PD support for L&N academy or any sessions appropriate for high school STEM teachers. There was little focus in the L&N-provided PD on actual STEM content for specific learners. However, L&N did support teachers to attend PD outside of the school, as well as to attend national conferences (e.g., National Science Teachers Conference). Teachers at L&N participated in 30 or more hours of PD across the 2012-2013 year. L&N received a rating of “Developing” on the PD attribute at baseline (October 2012), which improved to an “Advanced” rating at the second site visit (April 2013) due to an increase in emphasis on job-embedded PD. Professional development opportunities for teachers at L&N improved with a few more options for teachers to attend conferences in year two. However, most of the sessions hosted at the school were focused on the needs of all instead of individualized approaches. STEM professional development frequently focused on specific STEM learners. The rating for year two site visits (fall 2013 and spring 2014) remained at “Advanced” for L&N. Outreach, Support, and Focus on Underserved Students (A6) The L&N STEM Academy was established as a STEM magnet school within the Knox County School District and recruited students by lottery to attend. As a result, the student population was diverse and consisted of students from a variety of ethnic/racial, and socio-economic backgrounds, which was reflective of the overall composition of the district. The environment of L&N included a culture of trust, inquiry, and creativity for students, teachers, administrators, and partners, emphasizing the inclusion of all students in the learning process. There were policies and practices (including the regional lottery) that supported equity and access for all students. L&N made some strides to improve on this attribute across the first year of the evaluation. However, at baseline only one program was in place to specifically target under-represented student excitement in STEM careers (Engineers banquet). The number of programs grew by the end of the school year to two or more in-school programs. The baseline rating of “Developing” rose to “Advanced” at the second site visit in April 2013. At the fall 2013 site visit, L&N’s rating remained at “Advanced” as there were several in and out of school programs available to students to engage with STEM professionals. This rating improved to “Target” by the final site visit, as the emphasis in this area had resulted in individualized STEM internships for each and every student at L&N their senior year. This coupled with the multiple in and out-of-school opportunities that had continued to grow across the year resulted in a rating of “Target” for this construct at the spring 2014 visit.


STEM Work-based Learning Experiences (B1) L&N experienced some growth from baseline to second visit (spring 2013) in this area. At the first site visit (October 2012), L&N received a rating of “Developing” in regard to STEM Work-based Learning Experiences. At baseline, several L&N STEM course PBLs were connected to current work in STEM industries and careers (e.g., engineering and health care). As the school year progressed, the frequency of these connections increased. L&N teachers occasionally collaborated with other STEM professionals to design learning environments to empower students to think critically and address real-world problems. However, the Dean of STEM interacted with STEM stakeholders frequently in this area. Two components of B1 did not change across the first year at L&N. Unfortunately, few teachers participated in customized applied learning experiences to increase their STEM pedagogical content knowledge. On a positive note, students at L&N were engaged on a daily basis to work together and learn in teams to frame problems and test solutions through PBL. The rating for L&N on the B1 attribute increased to a rating of “Advanced” for the second site visit (April 2013) due to progress in frequently connecting PBLs to STEM industries and careers. L&N continued the frequent focus within the STEM course curriculum on STEM industries and careers in fall 2013. Students worked in teams on a daily basis to frame problems and test solutions with clearly defined individual and team expectations. The rating for L&N at the third site visit (fall 2013) remained at “Advanced.” Across the year, teachers at L&N began to collaborate more frequently with STEM professionals on various in and out of school projects and activities for students. L&N teachers also had more opportunities to participate in applied STEM learning experiences of their own. The rating for L&N STEM Academy at the final site visit was “Target” due to the progress established across the second year of this evaluation. Participation in STEM Network (B2) L&N’s participation in the STEMSpark Hub decreased across the first year of this evaluation (2012-2013). Several factors contributed to this, including the Hub leadership shift from Becky Ashe (who is also principal of L&N) to Marilyn Roddy in early fall 2012. As a result, the Hub work was more focused outward on building up partnerships and relationships in the region. STEMSpark did not provide any specific support to L&N during the 2012-2013 year. Interviews with school and Hub leadership also revealed little collaboration across these two entities. It seemed the Hub did not choose to focus on leveraging the intellectual capital at L&N for dissemination across the region. However, L&N independently worked with several districts, as well as the larger TSIN network to provide resources and share best practices. Therefore, the overall rating for this attribute decreased across the 2012-2013 year, as the baseline rating (October 2012) was “Developing”. It slipped to “Early” at the spring site visit (April 2013) as relationships between the STEMSpark Hub and L&N appeared to dissipate further. Communication tools used across the first year remained unchanged, and newsletters, social media, webinars, and other external communications for L&N were used occasionally. There was also little focus on assembling a team of stakeholders to focus on sustainability issues in year one, as this was rarely a topic of discussion for L&N leadership.


L&N struggled in the first year to make real connections with the STEMSpark Hub. As a result, L&N began to make their own connections with other K-12 schools and the other Hubs and schools within the TSIN. New partnerships and opportunities for delivering and receiving professional learning experiences increased in year two. For example, L&N hosted multiple TSIN schools to come and visit and learn from their staff. Clearly, the TSIN valued the connection with L&N and a trusted collaboration was further established. The rating for L&N at both the fall 2013 and spring 2014 site visits was “Advanced” in this area. STEM Program Plan (B3) The STEM Program Plan in place at L&N Academy is called the STEM Arc. The STEM Plan articulates a clearly spiraling curriculum for the school and associated objectives and outcomes. The L&N leadership works frequently to communicate the STEM school vision to the community stakeholders. On-demand, up-to-date data on student progress is readily available. The STEM School grant includes discretionary funding that can be used to advance the implementation of the STEM focus. L&N Academy was rated “Advanced” on the B3 attribute at both baseline (December 2012) and the second site visit (May 2013). L&N had a clear STEM curriculum plan that was developed across year one of the evaluation. At the fall 2013 site visit, leadership began to discuss expanding the plan – however no clear progress was completed at that time. The rating for fall 2013 remained at “Advanced.” Across year two, this plan was expanded to also consider other aspects of the school including sustainability, professional development, student opportunities, in a more purposeful manner. The rating for L&N at the final site visit (spring 2014) was “Target” in the STEM Program Plan area. Alignment of Career Pathways with Post-secondary (C1) L&N was in their second year of operation during the first year of this evaluation (2012-2013). There was no effort invested during this year on vertical planning with feeder elementary and middle schools, likely due to the large number of schools from which L&N are drawn. Additionally, L&N did not have a career counselor on staff, limiting career and post-secondary connections for students and creating challenges for students and teachers in planning pathways to post-secondary education. However, due to the work of the Dean of STEM, some information about post-secondary STEM programs and STEM careers was made available. The rating for L&N on this attribute was “Early” for both baseline (fall 2012) and the second site visit (April 2013). In fall 2013, leadership at L&N had begun working with community college partners to articulate a pathway that would be implemented the following year for students to complete an associate’s degree in a technology field while completing their high school degree. The rating for L&N rose to “Developing” at this third site visit as acknowledgement of this planning. By the spring 2014 final visit, the fully articulated plan was in place. Further, counselors, teachers, and STEM professionals were actively working with students early on to provide guidance regarding future pathways. The rating for L&N at the final visit was “Advanced”.


Post-secondary Credit Completion (C2) At L&N there were some opportunities for credit completion with post-secondary institutions, including courses in Latin, computer science, etc. The number of opportunities increased from baseline (fall 2012) to the second site visit (spring 2013). However, only 10-15 percent of students were enrolled in these courses. Teacher awareness of these opportunities grew across the school year. The rating for L&N on the C2 attribute was “Developing” at both baseline (fall 2012) and second site visit (spring 2013). In the second year of the evaluation, L&N had provided many more opportunities for students to complete dual credit coursework, including English and Mathematics. Also, a four-year pathway for computer science and a couple of other majors were being established with Pellissippi State for the following school year. The rating for L&N in this construct for both fall 2013 and spring 2014 was “Advanced”. RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN? An examination of the SCAS findings related specifically to the STEMSpark Hub was conducted by completing an analysis of the respondents who reported living in one of the counties included in the service area (e.g., Anderson, Blount, Campbell, Claiborne, Grainger, Jefferson, Knox, Morgan, Roane, Scott, Sevier, and Union). The number of participants who completed the survey at each administration from the STEMSpark region included: 443 for fall 2012, 275 for spring 2013, 88 for fall 2013, and 216 for spring 2014. Overall, all stakeholder groups experienced growth in agreement on most constructs, though each group also experienced some decline in agreement as well. There was only one area of increased agreement across the two-year evaluation that was common for all three stakeholder groups (Business/Community, Higher Education, K-12). This area was regarding the construct focused on K-12 schools in the region effectively teaching students STEM knowledge and skills (Business/Community 26% to 51%, Higher Education, 26% to 52%, and K-12 67% to 79%). Interestingly, there were no common areas of decline across the three stakeholder groups. Findings in this area demonstrate that STEMSpark efforts may not have influenced overall agreement or disagreement of stakeholders in the region during the two years of this evaluation. Stakeholder groups remained independent in their opinions and experiences overall. Tables 34-39 provide an overview of areas of growth in agreement and decline in agreement for each of the stakeholder groups.


Table 34. K-12 Educators Areas of Growth in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014









78% 74% 72% 88%


Table 35. K-12 Educators Areas of Decline in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall 2012

Spring 2013

Fall 2012

Spring 2014

I believe it is important for area businesses to be involved in STEM partnerships with K-12 schools. 95% 92% 95% 90%

I have had business/community funded STEM education programs or events in my school or district. 58% 60% 58% 32%

I have had community/business volunteers for STEM education programs or events in my school or district. 56% 58% 56% 47%

I have had community/business guest speakers in my school or district. 75% 72% 75% 61%







More work needs to be completed to spread awareness of STEM education. 94% 92% 94% 90% STEM skills are integral to student success today. 92% 90% 92% 87% Students with post-secondary education are more likely to secure a career in a STEM field. 92% 88% 92% 87%

There are colleges and/or universities and/or community colleges that offer scholarships for students to pursue STEM degrees in my region. 82% 78% 82% 77%




There are other STEM online tools available to this region. 74% 68% 74% 67% Students in this region are prepared by K-12 schools to be successful in post-secondary study. 76% 70% 76% 66%





Community partners in this region are engaged in making K-12 STEM education more relevant through providing real-world connections. 52% 48% 33% 34%



Increasing the STEM talent pool is necessary for economic vitality. 92% 96% 96% 83%


Table 36. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014


My organization has provided volunteers for STEM education programs or event in K-12 schools in our region. 73% 72% 75% 92%



My organization is interested in providing STEM education opportunities for K-12 students in this region. 81% 67% 75% 97%

I have worked closely with K-12 schools in my role as a Business/community stakeholder. 67% 50% 50% 90%








All K-12 schools in this region teach STEM skills and knowledge appropriate for jobs that will be available in the region. 26% 43% 67% 51%

All K-12 students should have access to STEM education. 95% 93% 100% 100% Preparing students for careers in STEM is a top priority for schools in this region. 67% 36% 44% 72%


Table 37. Business/Community Stakeholder Areas of Decline in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall

2012 Spring 2013

Fall 2013

Spring 2014

There are businesses and industries that provide signing bonuses and/or incentives for workers choosing a STEM career in this region.

45% 31% Too few participants

40%

Organizations have experienced an increase in the number of STEM positions available in the last year in this region.


60%



31%

Table 38. Higher Education Faculty Areas of Growth in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014



I have worked closely with community/business organization members in my role as a faculty member. 21% 24% 36% 61%




Organizations have experienced an increase in the number of STEM positions available in the last year in this region. 48% 50% 42% 55%



Organizations have been able to fill STEM-related positions with local talent. 25% 32% 13% 34%

It is important for business in this region to be able to recruit workers locally. 92% 92% 84% 99%

All K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be available in the region. 26% 41% 47% 52%




All K-12 students should have access to STEM education. 87% 89% 93% 94% Career-oriented education is for all students. 66% 74% 74% 79%


Table 39. Higher Education Faculty Areas of Decline in Agreement on SCAS for STEMSpark Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014


Employees from my institution have served as volunteers for local K-12 schools. 93% 66% 74% 78%

Employees from my institution have served as guest speakers for local K-12 schools. 93% 88% 80% 81%


My institution understands the importance of STEM education. 98% 95% 86% 87% There are STEM education websites available for this region that include activities for K-12 teachers and students. 81% 91% 75% 72%


Local organizations recruit STEM talent online. 63% 68% 53% 59% There are other STEM online tools available to this region. 80% 76% 65% 72% All students should receive information about careers that are expected to be in demand in the region when they graduate from K-12 schools and post-secondary institutions.

98% 95% 90% 94%

CONCLUDING OBSERVATIONS - STEMSpark East Tennessee STEM Innovation Hub and L&N STEM Academy The TSIN investment in the STEMSpark Hub resulted in increased awareness of STEM in the greater Knoxville region. STEMSpark offered several STEM professional development sessions for teachers and experiences for K-12 students in STEM. The STEMSpark website served as a conduit to connect partnering school districts in the region to potential resources and opportunities for their schools. However, the STEMSpark Hub, though housed in the Knox County School System, struggled to make authentic connections with the L&N STEM Academy (STEM Platform School), which might have amplified the potential impact of the STEMSpark Hub both in the short and long term. The focus of the TSIN Hubs was to provide support for STEM education across the region. Since the STEMSpark Hub was administered by a local school district, it was vital to gather outside sources of funding through potential partners and agencies for sustainability. STEMSpark was not able to do this since the majority of partners were educational institutions, and collective buy-in from business/industry was not achieved. These alignment issues resulted in an unsure future for STEMSpark at the end of TSIN funding. L&N STEM Academy was very successful in the first three years of operation (2011-14). L&N faculty disseminated innovations from the school across the region and beyond so that others may learn how to implement STEM approaches as well. Other schools have adopted the STEM Arc that comprises the STEM series of coursework at L&N. All seniors at L&N participate in individualized STEM internships. L&N is one of two TSIN schools that were recognized by Apple for their


innovative use of student technology, receiving the Distinguished School Award in 2013. L&N STEM Academy is a best-in-class model of an enhanced STEM school approach for the state of Tennessee. Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through their series of regional STEM innovation networks and STEM platform schools. The collective motivations for STEMSpark stakeholders differed a bit from the other regional STEM Hubs in that their main motivation for being involved with the Hub was the ability to connect to a mechanism that would disseminate the growing amount of work going on regionally in STEM. Focus group sessions (made up mostly of K-12 stakeholders) revealed that many schools regionally had implemented STEM. Across the two years of the evaluation business/community stakeholder engagement with STEMSpark was minimal, however higher education and K-12 partnerships were strong. In regard to capacity building, STEMSpark invested considerably in teacher professional development work. This was the only area all three stakeholder groups agreed improved across the two years. There were no other areas of common increase or decrease in agreement, indicating the STEMSpark efforts had little to no influence on other aspects of STEM Awareness or increasing STEM opportunities in their region. Therefore, it is difficult to determine what impact STEMSpark has had on changing the larger context in their region. STEMSpark has a shared leadership platform with an advisory council. However, there did not seem to be a shared vision, and it is unclear if all stakeholder groups are represented and actively engaged. STEMSpark failed to achieve tri-level engagement, as the school and business stakeholders were not included regularly in the work of the hub. The persistence and flexibility of STEMSpark was limited by the small core leadership format and the lack of inclusion of school staff and regional stakeholders in meaningful work of the Hub beyond professional development.


Hub Narrative Memphis: West Tennessee STEM Collaboratory Hub Southwind STEM High School

HUB AND SCHOOL SETTING The West Tennessee STEM Hub (WTSC) is housed at the University of Memphis and led by Dr. Stephanie Ivey (Project Director/PI), Dr. Alfred Hall (Hub Director), Dr. Peter Bridson (Partnership Coordinator), Dr. Lee Allen (Instructional Technology Director), Kimberly Grantham (Communications Director, and Dr. Cathy Wilson (Sustainability Director). The goals of the WTSC Hub are to:

1. To support successful operation of the STEM Platform School (Southwind High School) and facilitate dissemination of best practices across West Tenneseee.

2. To assemble and facilitate utilization of curricula aligned with state and Common Core State Standards that effectively engage West Tennessee K-12 students in STEM learning.

3. To create a sustainable culture of sharing and collaboration in the local, regional, and state STEM community through the establishment of productive partnerships among K-12 and IHE STEM faculty, business/industry representatives, and community organizations.

The organizational structure of the WTSC Hub includes a Governing Board, Steering Committee, and four Teams (Professional Development/Career and Technical Education, Instructional Technology Team, Communications Team, and Sustainability Team) to guide the work of the Hub. The evaluation of the WTSC Hub and Southwind STEM Platform School was only conducted across one year (2013-14) due to the delay in funding and launch of STEM efforts in West Tennessee. The primary activities of the WTSC Hub included the following:

• Master Teacher Corps (Teacher Professional Development) • K-20 STEM Education Conference • STEM Challenge Summit for MS/HS Students (STEM Herpetarium Challenge) • STEM Ambassadors (Mentoring and Tutoring for K-12) • STEM Speakers Bureau • STEM Spotlights (Individual and Industry) • STEM Module Development • Communication of Resources, Opportunities to Region

An area of strength for the WTSC Hub is the shared leadership structure including multiple working groups. The WTSC Hub also has extensive programming, including resources and educational opportunities/support for teachers and students.


Southwind High School is the platform STEM school (school within a school) for the WTCSH. The STEM approach includes offering students courses that include Intro to Engineering, and Robotics in addition to traditional coursework. In year one (2013-14) 125 ninth graders were the first students to begin study in the STEM program at Southwind. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. The WTSC Hub is well positioned to sustain partnerships established through the TSIN funding (Race to the Top). The WTSC Hub is located at the University of Memphis led by faculty who have a long track record in securing funding for and building STEM programs. The WTSC Hub have many other funded programs that complement the TSIN programming. This Hub has the strongest leadership model of the existing TSIN Hubs. Leadership is comprised of several actively working teams, as well as a cohort of fully engaged partners collaborating in strategic planning and implementation of programming. The WTSC Hub has a fully articulated sustainability plan that was collaboratively developed. It is unclear if Southwind High School’s STEM Program will be maintained long-term. This is only the end of the first year, and at this point the STEM coursework consists of just one engineering class (additional offerings will be added each year). However, the connection to the Hub is minimal, and Shelby County Schools, along with other neighboring districts, are going through a process of de-consolidation (just consolidated one year ago). The building principal also just retired. These factors combined make the prospects for sustainability doubtful. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). There are multiple higher education, business/industry, and K-12 partners who are members of the WTSC Hub. The primary strength of this Hub is the intensive involvement of multiple partners in the planning, development, and implementation of Hub activities. Therefore, input from each of the stakeholder groups was needed to determine the motivations for involvement. Research Question two explored the motivations that partners of TSIN Regional STEM Innovation Hubs have for being involved. Through the qualitative analysis of focus group data from the first two site visits (fall 2012, spring 2013) some insight was gained in regards to initial motivations for engagement with the STEM Hub. Some of the motivations for stakeholder engagement in the regional STEM effort included:

• Interest in increasing exposure of students to potential STEM careers in the region. • Desire to be a part of a collective STEM reform effort.


• Desire to improve K-12 education in the region. • Desire to develop future STEM talent for the region. • Desire to better prepare students for success in postsecondary education.

RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question Five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. The innovative approaches coming out of the work of the WTSC Hub and Southwind STEM High School during the 2013-2014 school year included: 1. STEM Ambassadors: The infusion of undergraduate students within local high school

settings as mentors and tutors to help ensure success of students is a powerful approach. 2. Collaborative Leadership Structure: The organizational structure of this Hub is more likely

to result in sustained and significant impact than other approaches within the TSIN where most have a Hub Director (or two) and no other shared responsibility, commitment, or others who are engaged in the “work.”

RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? Southwind STEM High School is a STEM focused high school with a curriculum that is STEM enhanced. The first visit (baseline) for SSMHS took place December 6h and 7th, 2013. The second site visit was conducted May 9th and 10th, 2014. Table 40 provides an overview of the ratings for SSMHS for the first year of the evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.


Table 40. Southwind STEM Magnet High School: STEM Attributes Ratings

STEM Attributes

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects Early Early A2) Connections to effective in and out-of-school STEM programs Early Developing

A3) Integration of technology and virtual learning Developing Developing A4) Authentic assessment and exhibition of STEM skills Early Early A5) Professional development of integrated STEM curriculum, community/industry partnerships and postsecondary education connections Early Early

A6) Outreach, support and focus on underserved, especially females, minorities, and economically disadvantaged Early Developing

B1) STEM work-based learning experiences, to increase interest and abilities in fields requiring STEM skills, for each student and teacher Early Early

B2) Participation in STEM Network Early Developing B3) STEM Program Plan Early Developing C1) Alignment of students’ career pathways with post-secondary STEM program(s) Early Early

C2) Credit completion at community colleges, colleges, and/or universities Early Early

Project-based Learning with Integrated Content (A1) The use of project-based learning (PBL) was evidenced only in one class during the fall 2013 site visit. In this class students were exploring buoyancy and the teacher challenged students to create a boat out of foil (an activity that has been used in science for many years) that would hold the most pennies without sinking. In all other classes observed, students were engaged in traditional instruction ranging from taking notes to completing worksheets. In interviews with teachers it was revealed that PBL was not a focus and that very few had received any training in the pedagogy. There were no attempts to integrating STEM across content areas as well. The rating for the fall site visit (2013) was “Early”. The spring 2014 site visit revealed very little change. In the engineering class students were making paper airplanes and throwing them across the room (this was the assignment) without much guidance from the teacher who remained seated at his desk. Other content classes were again completing worksheets (fill-in-the-blank) and no integration was observed. The rating for spring 2014 was also “Early”. Connections to STEM Programs (A2) Connections to STEM programs (in and out of school) were not evidenced during the fall site visit (2013) and the rating for Southwind was “Early”. At the spring visit (2014) there was more evidence of teachers engaging with the STEM Hub and beginning to secure a few partnerships with STEM industry. The rating for Southwind at the final site visit was “Developing.”


Integration of Technology/Virtual Learning (A3) Southwind STEM program teachers had access to technology resources including laptop computers and online materials for instruction. Students were observed delivering presentations (in Intro to Engineering) with their back to the class and reading from the slides indicating that many had not mastered the skills necessary for utilizing technology to communicate information. The rating for Southwind for this construct was “Developing” at both the fall 2013 and spring 2014 site visits. Authentic Assessment (A4) Across the two visits to SSHS it was unclear if teachers in the STEM Program utilized authentic assessments. The fall site visit (2013) included observations of students delivering presentations, which is an example of authentic assessment. However, in all other classes students were observed taking notes and completing worksheets. The fall rating for SSHS was “Early.” During the spring 2014 site visit, similar types of assessment were observed. One difference was there were no student presentations and in one class students were allowed to use their notes to complete the end of course exam. The rating for SSHS remained the same for the final site visit at “Early.” Professional Development (A5) Teachers at SSHS shared that they were not provided with any professional development on STEM up front, or across year one of the school opening. One teacher sought out PD through the STEM Hub (STEM Master Teacher Training) and received STEM training. Others interviewed had not received any support. The teacher who did attend the STEM Master Teacher training shared that she would like to engage (and have others in the school) in some serious PD on STEM. This was not a focus for the staff of the STEM program. In fact, it was not clear that any PD was provided for teachers at the school during year one. The rating for this attribute at both fall 2013 and spring 2014 visits was “Early”. Outreach, Support, and Focus on Underserved Students (A6) At the beginning of the first year of the SSHS there was a core group of teachers who were starting to get to know each other and develop trust. However, there were not any in-school programs focused on engaging students in real-world STEM experiences. The rating for SSHS at the fall site visit (2013) was “Early”. The spring visit (2014) revealed some efforts had been made in this area to open up more opportunities for students (e.g. STEM Hub Herpetarium Challenge) and the rating was improved to “Developing.” STEM Work-based Learning Experiences (B1) At the beginning of the school year, teachers at SSHS were trying to figure out what it meant to be a teacher at a STEM school. Some teachers devoted their time independently to researching and planning experiences for students in STEM. Students occasionally worked in teams to solve problems. However, teacher connections with STEM professionals were rare. The rating for the fall 2013 site visit and spring 2014 visit was “Early.”


Participation in STEM Network (B2) The SSHS leadership maintained some collaboration with the STEM Hub through the occasional exchange of resources and participation in programming at the beginning of the school year. There was little involvement beyond this level, which resulted in a rating of “Early” for the fall 2013 site visit. However, the level of engagement in STEM Hub opportunities increased in spring 2014, including the development of some communication tools for SSHS and development of a marketing strategy. The rating for spring 2014 improved to “Developing.” STEM Plan (B3) SSHS had no formal STEM Plan in place during the first year of operation beyond a list of potential coursework for students to complete. The rating for SSHS at the fall 2013 site visit was “Early.” There was some progress in this area, as discretionary funding was utilized mid-year to advance the communication of STEM activities with the community. The rating rose to “Developing” at the spring 2014 site visit. Alignment of Career Pathways with Post-secondary (C1) Due to the fact that this was the first year of operation and only 9th grade students were represented at the school, there was little focus on career pathways and post-secondary programs at SSHS. Vertical planning and counseling were not in place. The rating for SSHS in this area was “Early.” Post-secondary Credit Completion (C2) There were no formal plans articulated regarding credit completion opportunities for students in the STEM program at SSHS. Less than 10% were enrolled (zero). The rating for this attribute was “Early” at both the fall 2013 and spring 2014 site visits. RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN An examination of the SCAS findings related specifically to the WTSC Hub was conducted by completing an analysis of the respondents who reported living in one of the counties included in the service area (e.g. Benton, Carroll, Chester, Crockett, Decatur, Dyer, Fayette, Gibson, Hardeman, Hardin, Haywood, Henderson, Henry, Lake, Lauderdale, Madison, McNairy, Obion, Shelby, Tipton, and Weakley). The number of participants who completed the survey at each administration from the West TN STEM Hub region included: 174 for fall 2013, and 409 for spring 2014. Overall, all three stakeholder groups (K-12 Educators, Business/Community Stakeholders, and Higher Education Faculty) increased their agreement positively in most areas across the two-year evaluation. There were three main areas in which all three stakeholder groups experienced growth in agreement in across the two years of the evaluation.

• One area of growth in agreement across the two-year evaluation for all three groups was parental understanding of the importance of STEM Education. Higher Education reported


an increase from 40% agreement to 43%, Business/Community reported improved agreement from 33% to 82%, and K-12 recorded growth from 33% to 37% in agreement.

• The second area for WTSC where all stakeholders increased in agreement was the increase in number of STEM positions in the region over the last year. Business/Community agreement increased from 73% to 88%, Higher Education increased from 70% to 74% agreement, and K-12 teachers’ agreement increased from 54% to 57%.

• The final area of agreement for all three groups was that organizations have been able to fill all STEM-related positions within the last year in the region. Business/Community agreement increased from 33% to 34%, Higher Education increased from 38% to 53% agreement, and K-12 teachers’ agreement increased from 33% to 51%.

The Business/Community Stakeholder, Higher Education Faculty, and K-12 Educator groups had a few areas of decline, but none of these were common across the three groups.

Table 41. K-12 Educators Areas of Growth in Agreement on SCAS for WTSC Hub

SCAS Item Fall 2013

Spring 2014

I have had business/community funded STEM education programs or events in my school or school district.

46% 51%

I have had community/business volunteers for STEM education programs or event in my school or school district.

47% 50%

My organization is interested in providing STEM education opportunities for K-12 students in this region.

58% 66%


51% 56%

Parents in this region understand the importance of STEM education. 33% 37%

There are STEM education websites available for this region that include activities for K-12 teachers and students.

75% 84%

Local organizations recruit STEM talent online. 44% 54%

Information related to STEM opportunities in my region is available online. 56% 69%

There are other STEM online tools available to this region. 66% 70% Students in this region are knowledgeable about the STEM careers that will be in high demand when they graduate.

50% 78%

K-12 public schools in this region effectively teach students STEM knowledge and skills.

53% 68%


39% 56%

Organizations have experienced an increase in the number of STEM positions available in the last year in the region.

39% 56%

Organizations have been able to fill all STEM-related positions within the last year in this region.

33% 51%

Organizations have experienced an increase in the number of women and minorities in STEM positions in the last year.

35% 52%

Organizations have been able to fill STEM-related positions with local STEM talent. 33% 55%

Preparing students for careers in STEM is a top priority for schools in the region. 52% 63%


Table 42. K-12 Educators Areas of Decline in Agreement on SCAS for WTSC Hub

SCAS Item Fall 2013

Spring 2014

More work needs to be completed to spread awareness of STEM education. 95% 90%

All students should receive information about careers that are expected to be in demand in the region when they graduate from K-12 schools and post-secondary institutions. 100% 87%

Table 43. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for WTSC

SCAS Item Fall 2013

Spring 2014

My organization has funded STEM education programs or events in local schools or districts. 58% 83%

Employees from my organization have been volunteers for STEM education programs or events in local schools. 46% 86%

Employees from my organization have been guest speakers in local schools. 66% 94% There are opportunities for K-12 students to complete internships or co-ops in the region. 56% 97%

My organization is interested in providing STEM education opportunities for K-12 students in this region. 100% 94%

Overall, there has been an increase in K-12 STEM education opportunities for students in the region in the last year. 51% 97%

My organization understands the importance of STEM education. 71% 100% I have worked closely with K-12 schools through my role as a business/community stakeholder. 37% 77%

Parents understand the importance of STEM education. 33% 82% Organizations have experienced an increase in the number of STEM positions available in the last year in this region. 39% 96%

Organizations have been able to fill all STEM-related positions within the last year in this region. 33% 34%

Table 44. Business/Community Stakeholder

Areas of Decline in Agreement on SCAS for WTSC

SCAS Item Fall 2013

Spring 2014

Information on regional STEM career opportunities is available online. 56% 41%

The state standardized tests used in this region’s K-12 schools adequately assess STEM knowledge and skills. 57% 16%



Table 45. Higher Education Faculty: Areas of Growth in Agreement on SCAS for WTSC Hub

SCAS Item Fall 2013

Spring 2014

There are opportunities for K-12 students in our region to complete internships or co-ops in this region. 55% 64%

I have worked closely with community/business organization members in my role as a faculty member. 40% 46%

Parents in this region understand the importance of STEM education. 40% 43%

There are STEM education websites available for this region that include activities for K-12 teachers and students. 69% 81%

Information related to STEM opportunities in my region is available online. 56% 68%

There are other STEM online tools available to this region. 60% 68% Students in this region are prepared by K-12 schools to be successful in post-secondary study. 64% 70%

Students in this region are knowledgeable about the STEM careers that will be in high demand when they graduate. 45% 51%

The K-12 public schools in this region effectively teach students STEM knowledge and skills. 55% 61%

The K-12 schools in this region prepare students who are critical thinkers and problem solvers. 52% 56%

There are businesses and industries that provide signing bonuses and/or incentives for workers choosing a STEM career in the region. 31% 48%

Organizations have experienced an increase in the number of STEM positions available in the last year in the region. 38% 53%

Organizations have been able to fill all STEM-related positions within the last year in this region. 38% 53%

Organizations have experienced an increase in the number of women and minorities in STEM positions in the last year in this region. 30% 45%


All K-12 schools in this region teach the STEM skills and knowledge appropriated for jobs that will be available in the region. 44% 61%

Career-oriented education is for all students. 74% 83%

Preparing students for careers in STEM is a top priority for schools in this region. 45% 57%


Table 46. Higher Education Faculty Areas of Decline in Agreement on SCAS for WTSC

SCAS Item Fall 2013

Spring 2014

My institution has provided guest speakers for K-12 schools in our region. 72% 72%

More work needs to be completed to spread awareness of STEM education. 94% 91% The state standardized tests used in this region’s K-12 schools adequately assess STEM knowledge and skills. 52% 46%

All students should receive information about careers that are expected to be in demand in the region when they graduate from K-12 schools and post-secondary institutions. 95% 89%

The K-12 schools in this region understand the importance of STEM education. 81% 76%

My organization understands the importance of STEM education. 90% 81%

CONCLUDING OBSERVATIONS - Memphis: West Tennessee STEM Collaboratory Hub and Southwind STEM High School The TSIN investment in the WTSC Hub resulted in an effective, multi-faceted, model for building STEM capacity in the greater Memphis region. The WTSC Hub, administered by the University of Memphis, has extensive regional stakeholder buy-in and partnerships that are included within a shared leadership organizational plan that is unlike other Hubs in the TSIN. In addition to the six directors devoted to this work, there are four working Teams that have oversight for various components of the operation of the Hub. The activities of the WTSC Hub include focus on teacher development, curriculum development, annual conferences, engaging STEM professionals in K-12, growing knowledge of STEM careers, growing awareness of STEM, mentoring, tutoring, and STEM challenges for students. The WTSC Hub is loosely connected to the STEM platform school (Southwind High School) through the Master Teacher Corps and other targeted supports as needed. The WTSC Hub has a best-in-class organizational structure for a regional STEM Hub housed at a university in Tennessee. The frequent, deep, and meaningful engagement of partners in the administration of the Hub is markedly different from the usual advisory role of partners represented in other TSIN Hubs and is a model with great promise in terms of sustainability. Southwind High School’s STEM program (school-within-a-school) opened in fall 2013 with 9th grade students. The STEM focus for year one included an Intro to Engineering course for all students, with the other classes in the school being delivered in a traditional format with low rigor (e.g. using notes on an end of level exam, fill-in-the-blank worksheets). Core teachers had not received any professional development on STEM content or pedagogy through the school. It is unclear what the future holds for Southwind STEM, as there have been some administrative shifts and teacher mobility. The school could benefit from a much stronger relationship with the WTSC Hub in the future to help move the level of STEM within the school beyond “Early” or “Developing.”


Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through their series of regional STEM innovation networks and STEM platform schools. WTSC Hub stakeholder collective motivations for being engaged in the STEM reform initiative ranged from a desire to improve K-12 schools and preparation for postsecondary success, and the desire to spark interest in K-12 students in STEM careers and post-secondary study. Capacity building was a major focus of the WTSC Hub and the SCAS revealed that all three stakeholder groups reported growth in parental understanding of the importance of STEM education, and increase in STEM positions in the region. A final area of increased agreement for all three stakeholder groups was the improved ability for regional STEM industry to fill vacancies. There were no areas of decline that were common across all three stakeholder groups, indicating that overall there has been progress in the WTSC Hub region in many key areas without any common areas of concern for all. WTSC made considerable progress in generating new knowledge and strategies within the local context, as well as changing the larger context. For example, the annual K-20 STEM Education Conference and the Master Teacher Corps have both contributed to achieving these tasks. WTSC is the best TSIN example of tri-level engagement, which WTSC has representation from K-12, community/business, higher education, and TSIN in decision making and implementation of activities for the Hub. WTSC has excellent engagement from higher education, business/industry and K-12 stakeholder groups. In summary, WTSC has established excellent infrastructure to support the sustainability of STEM support for the region for many years to come.


Hub Narrative Nashville: Middle Tennessee STEM Innovation Hub (MTSI) Stratford STEM Magnet High School

HUB AND SCHOOL SETTING The Middle Tennessee STEM Innovation Hub (MTSI) is housed in the Metro Nashville Public School District and led by Dr. Vicki Metzgar, Hub Director. The partners in the MTSI Hub include K-12 (e.g., Metro Nashville Public Schools, Robertson County, Cheatham County), Higher Education (e.g., Vanderbilt University, Middle Tennessee State University, Lipscomb University, Nashville State Community College), and Business/Community partners (e.g., Adventure Science Center, Deloitte Services LP, Tennessee Business Roundtable). The goals of the MTSI Hub were to:

1. Support the STEM Platform School and other schools in the region by collecting and distributing data on the existing STEM assets and STEM education gaps in the region.

2. Implement rigorous teacher professional development programs, curriculum designs and implementation reflecting the latest research on effective STEM pedagogy, applying research-rich STEM education and project experiences.

3. Conduct annual events and engagement opportunities for community and partners to build additional partnerships, grow opportunities, and sustain the work of the hub.

The activities of the MTSI Hub included the following:

1. Gather data of partner school districts’ current student achievement status with respect to STEM subjects.

2. Gather data related to teacher effectiveness in each partner school district with respect to STEM subjects.

3. Gather data from the TDOE related to STEM teacher production and effectiveness of STEM teaching graduates at colleges and universities in the Middle Tennessee region.

4. Provide teacher externships for training in real-world STEM job situations. 5. Identify, develop, and share STEM curricular tools with all districts across the

region. 6. Provide for sharing of teacher professional development for in-service teachers in all

counties of middle Tennessee. 7. Train a cadre of teacher leaders from participating K-12 districts via professional

development contracts with higher education. 8. Identify and share best practices from STEM platform schools and other Hub region

schools for replication. 9. Conduct Business STEMposium. 10. Conduct student STEM Expo.


11. Continually cultivate business partners and relationships with STEM community partners.

12. Develop a suite of “plug and play” opportunities for businesses to participate in that provide meaningful support for schools.

13. Identify and pursue grant opportunities for collaborative proposals. 14. Develop meaningful partnerships between business and STEM community

organizations through a structured engagement process. The primary focus of the work of the MTSI Hub has been funding the delivery of professional development programs through a variety of higher education partners. In year two there were thirty programs provided through partners of the MTSI Hub. Some examples of funded MTSI Hub professional development opportunities included:

• NASA workshop through Dyer Observatory • Inquiry Science workshop through Vanderbilt Center for Science Outreach • Robotics teacher training and student camps through Lipscomb University • Mathematics workshop through Austin Peay State University • Aspirnauts program through Vanderbilt University • GIS training through Vanderbilt University • Engineering is Elementary through Harpeth Hall • Stream Studies through Cumberland University

An area of strength for the MTSI Hub is their extensive partnerships with higher education institutions in the region and the large scale of their Hub coverage area, including many K-12 partnering districts. There was limited engagement of the business/community stakeholder group in the MTSI Hub, outside of informal education centers, non-profits (e.g. PENCIL Foundation, BioTN Foundation) and two primary business partnerships (Nissan North America and Aegis). The website for the MTSI hub is www.midtnstem.com and the site provides information on upcoming events and professional development in the region. The MTSI Hub also focused considerable energy in providing student STEM opportunities, including the annual STEM Expo. Growing regional awareness is also a thrust of the work of MTSI Hub, which includes the annual STEM-Posium. There is no information about the MTSI Hub goals or objectives on the website. The vision, however, is emphasized, and states: “The Middle Tennessee STEM Innovation Hub will promote student interest and improve student achievement in STEM in order to create graduates from secondary and post-secondary institutions who are ready for STEM college work and STEM careers.” Stratford STEM Magnet High School is a 9-12 high school located within the Metro Nashville Public School District. In fall 2011, Stratford converted to a STEM curricular approach. Stratford is a large urban high school arranged in academies where students may choose to enroll in The Academy of Science and Engineering or The Academy of National Safety and Securities Technologies. Within the Academy of Science and Engineering, students may choose from the Interdisciplinary Science, Engineering, or Biotechnology pathway. The Academy of National Safety and Securities Technologies has two choices of focus for students: National Security Technology or


Computer Simulation and Game Programming pathways. Students at Stratford take their academic core classes (e.g., math, science, English, social studies) and then also complete their academy classes (e.g., interdisciplinary science I, forensic science). Stratford STEM Magnet High School is a STEM-enhanced model, as the majority of STEM and PBL focus takes place within the academy coursework and is not integrated back into the core content classes explicitly or purposefully. RESEARCH QUESTION 1: Does the TSIN model create strong, sustained partnerships within and across regional Hubs? Research question one of this evaluation focused on the sustainability of partnerships established within each Hub and STEM School. The MTSI Hub is administered by the Metropolitan Nashville Public Schools (MNPS). The MTSI Hub formed extensive partnerships with multiple higher education institutions and K-12 school districts, as well as some community partners in informal education and foundations. There were few business partners for a Hub of this size in an urban city and the Hub Director (Vicki Metzgar) shared that this was the most difficult area of her work. Strong buy-in from this group is critical for sustainability. At the end of the funding period, MNPS committed to sustaining funding for 120 days for the Hub Director and a half-time administrative assistant. Negotiations were taking place for Belmont University to house the Hub, though it was not clear if they would administer the Hub moving forward. It does seem that there is collective will in the region – particularly from higher education and K-12 - for this Hub and the existing partnerships to be sustained. It is unclear whether this will happen, however, as the short-term bridge funding will only buy some time to apply for general funding to operate the Hub for the long-term. If Belmont University takes on administration of the Hub, the likelihood of sustaining the partnerships is very good. Much like other Hubs situated in K-12 districts, there are competing agendas at work since it is very difficult to justify the allocation of local funds for resources that will benefit other school districts in a regional scale. MNPS is also home to the Stratford STEM Magnet High School, which has developed their own strong partnerships with STEM industry through the various STEM-focused Academies that comprise the STEM focus at the school. Extensive work has been completed to cultivate these partnerships and fold partners into a role within the school to advise programming within each Academy. The leadership of MNPS is committed to Stratford long-term and the infrastructure for sustainability of partnerships with Stratford is very strong. RESEARCH QUESTION 2: What motivates stakeholder engagement in TSIN? Research Question two of this TSIN evaluation focused on uncovering the motivations of regional stakeholders for being involved in the TSIN Regional STEM Innovation Hub(s). There are multiple higher education and K-12 partners who are members of the MTSI Hub. Most of the higher education partners have been involved in delivery of professional development opportunities for teachers in the region, funded through mini-grants from the MTSI Hub. For example, Cumberland University (Biology Stream Studies and Chemistry on a Shoestring), Vanderbilt Center for Science Outreach (Hands on Science), Austin Peay University (Integrated Math and Science), and Middle


Tennessee State University (MTeach). K-12 school and center partners include Harpeth Hall who has delivered Engineering is Elementary workshops. The MTSI Hub also includes partners from Nissan and Texas Instruments to name a few. Research Question two explored the motivations that partners of TSIN Regional STEM Innovation Hubs have for being involved. Through the qualitative analysis of focus group data from the four site visits (fall 2012, 2013 spring 2013, 2014) some insight was gained in regards to motivations for engagement with the STEM Hub. Motivations for stakeholder engagement in the regional STEM effort included:

• Involvement with Stratford STEM Magnet High School or other STEM program in K-12 partner district.

• Opportunities for recruiting future workers for local STEM small business. • Interest in increasing exposure of students to potential STEM careers in the region

(specifically engineering – Nissan). • Desire to be a part of a collective STEM reform effort. • Desire to improve Metro Nashville Schools. • Desire to develop future STEM talent for the region.

RESEARCH QUESTION 5: What are the truly innovative approaches within the TSIN? What are the factors that contribute to and/or inhibit these innovations? Research Question five of this TSIN evaluation focused on identification of the truly innovative approaches that come out of the TSIN Regional STEM Innovation Hubs and/or the TSIN STEM Platform Schools. Specifically, these innovations must be new and creative approaches to building regional STEM capacity and/or building STEM programs (e.g., schools or student programs). Therefore, using approaches that may have been used in a widespread manner elsewhere would not meet the criteria for STEM innovation. Instead, this question focuses on relatively new approaches that are in need of dissemination and promotion. Innovative approaches identified within the MTSI Hub and Stratford STEM Magnet High School included:

1. Student STEM Expo: This event provided students from across the MTSI Hub region an opportunity to present their STEM research projects in a forum where innovation and creativity are celebrated. The event took place at partnering Belmont University and was well attended.

2. STEM Instructional Designers (SID) on-site at Stratford that support development of curriculum and delivery of professional development. The level of intensive, real-time support delivered to teachers at Stratford STEM Magnet High School (SSMHS) is exceptional. The SIDs work intensively to engage external business/community partners in the academy coursework, internships for students and externships for teachers.


RESEARCH QUESTION 7: What does the curriculum within TSIN platform schools consist of, and what are teacher and student roles within the curriculum? Stratford STEM Magnet High School (SSMHS) is a STEM academy focused high school with a curriculum that is STEM enhanced. The first visit (baseline) for SSMHS took place December 6h and 7th, 2012. The second site visit was conducted May 9th and 10th, 2013. The third site visit took place December 2-3, 2013, and the final site visit was conducted February 24-25 due to scheduling conflicts later in the spring associated with state assessments. Table 47 provides an overview of the ratings for SSMHS for the four site visits conducted for the two-year evaluation. Specific evidence for each rating will be discussed in more detail within each attribute.

Table 47. Stratford STEM Magnet High School STEM Attributes Ratings

STEM Attributes

Fall 2012

Spring 2013

Fall 2013

Spring 2014

A1) Project-based learning with integrated content across STEM subjects Developing Developing Advanced Advanced

A2) Connections to effective in and out-of-school STEM programs Developing Advanced Advanced Advanced

A3) Integration of technology and virtual learning Early Developing Developing Developing A4) Authentic assessment and exhibition of STEM skills Early Developing Advanced Advanced

A5) Professional development of integrated STEM curriculum, community/industry partnerships and postsecondary education connections

Developing Developing Advanced Advanced

A6) Outreach, support and focus on underserved, especially females, minorities, and economically disadvantaged


B1) STEM work-based learning experiences, to increase interest and abilities in fields requiring STEM skills, for each student and teacher

Developing Advanced Advanced Advanced

B2) Participation in STEM Network Advanced Advanced Advanced Advanced B3) STEM Program Plan Advanced Advanced Advanced Advanced C1) Alignment of students’ career pathways with post-secondary STEM program(s) Early Developing Developing Developing

C2) Credit completion at community colleges, colleges, and/or universities Early Developing Developing Developing

Project-based Learning with Integrated Content (A1) Project-based learning (PBL) is used at SSMHS within the pathway coursework each nine weeks. The PBLs are not connected to the core content classes (e.g., Chemistry, English, Algebra) in any formal way for integration of content. There has been some limited planning time for teachers who have focused on integrating teaching and learning across grades/content areas within their professional learning community work. However, only 25 percent or fewer of teachers at SSMHS


have made explicit efforts to integrate STEM across core subjects, requiring students to independently synthesize knowledge across disciplines. SSMHS has multiple facilities and spaces for face-to-face and virtual collaboration to take place among teachers and students. SSMHS received a rating of “Developing” on the A1 attribute at baseline (December 2012) and at the second site visit (May 2013). The second year of the evaluation SSMHS continued to emphasize PBL within the academy specialized classes. A new instructional component at SSMHS was the integration of STEM themes across each nine weeks (Patterns, Cause and Effect, Systems, and Change). Some of the content area coursework reflected PBL tied to these themes on a frequent basis. Teachers had quarterly planning time (at least) to develop PBL units that were taught within individual content classes tied to the theme. This progress was reflected in the “Advanced” rating for the fall 2013 and spring 2014 site visits. Connections to STEM Programs (A2) At SSMHS, the academies frequently engage in the MTSI Hub through work with partners identified through Hub relations, and also through professional development sessions that have been made available through mini-grants to partnering post-secondary institutions. SSMHS academies have integrated direct experiences several times a year with STEM professionals and STEM learning environments both during and outside of school. The SSMHS administration access research and best practices (through STEM Instructional Coaches) related to STEM goals. Stratford STEM Magnet High School received a rating of “Developing” at baseline (December 2012), which improved to “Advanced” on this attribute at the second site visit in May 2013. SSMHS continued to be connected frequently with the STEM Hub. Further, each academy had an extensive array of partnerships that were involved in an advisory capacity in the programming for each. The rating for SSMHS in Connections to STEM Programs for year two site visits (fall 2013, spring 2014) remained at “Advanced.” Integration of Technology/Virtual Learning (A3) Technology access at SSMHS is provided through laptop carts that are available in the academies. There is not a 1:1 technology environment at this large, urban, high school. However, common technology resources and curriculum are available for teachers and students and up to 50 percent of students have mastered common technology applications. For example, students delivered a presentation using Prezi in the criminal justice course observed and students used desktop computers in the game design course to create a sprite (character in game). Technology is used regularly to support teaching and learning at SSMHS. Teachers have access to digital instructional resources for STEM. Tech support at SSMHS is provided frequently. At baseline, SSMHS received a rating of “Early” on this attribute (December 2012). Due to concerted efforts to increase teacher access to resources and frequency of use of technology by students, the rating raised to “Developing” by the second site visit in May 2013. Technology integration remained at the “Developing” stage for year two site visits (fall 2013 and spring 2014) as SSMHS was not a 1:1 environment and technology was only used in about 50% of the classrooms during any given site visit. Students did not appear to be fluent with using online


portals (e.g. Chemistry class) for completing assignments. The use of digital instructional materials was frequent. Authentic Assessment (A4) SSMHS was very focused on tracking student progress on the ACT as well as End of Course exams and progress to graduation. However, the emphasis on using multiple alternative formats of assessing student progress (e.g., performance, project-based, portfolio) was limited and time for teachers to share authentic assessments or to create these was not provided more than a couple of times across the first year of this evaluation. Student work was celebrated occasionally through on-site and online exhibits. As the year progressed, SSMHS placed more emphasis on encouraging faculty innovation. SSMHS was rated as “Early” at the baseline visit in December 2012. This rating increased to “Developing” at the second site visit (May, 2013) due to increased efforts to encourage faculty innovation and use of authentic assessments in conjunction with traditional assessment formats. SSMHS invested considerable energy in the second year into the use of data from multiple assessments (formative and summative) to gauge student progress and make necessary modifications. The rating for SSMHS in this area was raised to Advanced during the fall 2013 site visit and remained steady for the final visit (spring 2014). Students, teachers, and administrators celebrated student work frequently through on-site, local and state forums (e.g. STEM Expo, Science Bowl). Professional Development (A5) The on-site STEM Instructional Designers that were funded through a STEM School Magnet grant provided weekly professional development for teachers at SSMHS. The majority of these sessions where large groups focused on a variety of things including technology use, classroom management, differentiated instruction, literacy, TCAP analysis, and strategies for implementing STEM. The PD was delivered in a job-embedded approach. However, only occasionally did the PD focus on specific STEM content for specific types of student-learners. At the baseline site visit, teachers had only had nine hours or less of PD (December 2012), but this was scaled up in spring and by the second site visit the amount of PD had increased to nearly 30 hours or more on average for teachers at SSMHS. The baseline rating (December 2012) and second visit rating (May 2013) for SSMHS on attribute A5 was “Developing.” Year two for SSMHS included more opportunities for teachers to participate in summer externships, individualized professional growth experiences focused on STEM. Other areas of PD included sessions on using data, PBL, etc. There was more teacher choice in focus for PD. As a result the rating for SSMHS in year two rose to “Advanced” for the fall 2013 and spring 2014 site visits. Outreach, Support, and Focus on Underserved Students (A6) At SSMHS, a culture of trust, inquiry, and creativity existed and was most prominent in the academy courses. Core content classes were still delivered through very traditional formats (e.g., lecture, note taking, working mathematical problems in large groups). Stratford had policies and practices in place that supported equity and access for all students. This school was not only a neighborhood school


but also a school of choice, opening up access to all students in the MNPS district. There were several in and out of school programs for students focused on growing excitement about STEM for under-represented students and supporting them to be successful at SSMHS. The rating for SSMHS at baseline (December 2012) and the second site visit (May 2013) was “Advanced” on the A6 attribute. This area continued to be a focus for SSMHS and students had multiple opportunities to engage with STEM professionals and learn about STEM careers. Due to the focus of the Academy model, most of the academy specialized coursework was delivered either by a STEM professional or co-taught with a STEM professional. The STEM Expo and FIRST Robotics are good examples of some of the opportunities for students. The rating for SSMHS remained at “Advanced” across year two (fall 2013 and spring 2014 site visits). STEM Work-based Learning Experiences (B1) The academy curriculum at SSMHS included multiple connections to current work in STEM industries and careers. The connections in core coursework were much less frequent and obvious during the first year of this evaluation. However, due to the academy model and the scheduling in which students attended their academy courses each day, students were provided with opportunities to work in teams to frame problems and test solutions on a daily basis. Partnerships with Vanderbilt University and others provided high-quality opportunities for teachers to participate in STEM externships and 25 percent to 50 percent of teachers had at least one customized applied-learning experience in order to increase their STEM pedagogical content knowledge during the 2012-2013 calendar years. Additionally, teachers in the academies and those who teach core coursework occasionally collaborate with other STEM professional to develop curriculum that empowers students to address real-world challenges. SSMHS received a rating of “Advanced” on the B1 attribute at both baseline (December 2012) and the second site visit (May 2013). Again, SSMHS provided daily opportunities for students to gain STEM work-based learning experiences through the Academy coursework (e.g. criminal justice, forensic science, engineering, biotechnology). Further, teachers were provided job-embedded experiences (academy teachers and some content area as well) with STEM professionals and all teachers were offered options to complete STEM externships. The rating for SSMHS in year two remained at “Advanced” in this area (fall 2013 and spring 2014 site visits). Participation in STEM Network (B2) SSMHS leadership maintained a reciprocal and trusted collaboration with the MTSI Hub and collaborated within the network to exchange resources and to share best practice. Across the 2012-2013 year, SSMHS communication tools improved and the frequency of communication with external constituents increased as well. The SSMHS leadership participates with the MTSI Hub to meet and seek additional future funding for the STEM effort in Middle Tennessee and at Stratford specifically. The SSMHS received a rating of “Advanced” on the B2 attribute at both baseline (December 2013) and during the second site visit (May 2013).


In year two, SSMHS continued engaging with MTSI to support instruction within the school, opportunities for students (e.g. STEM Expo) and professional development for teachers. The rating for SSMHS in year two continued at “Advanced” in this area (fall 2013 and spring 2014 site visits). STEM Plan (B3) In year one no formal STEM Program Plan had been drafted for SSMHS. However, in year two the school developed STEM Magnet Standards to guide instruction at SSMHS. The curriculum for the academies has been clearly articulated and overall school STEM activities contribute to the implementation of the academies. Further, the SSMHS leadership works frequently to communicate the STEM school vision to the community stakeholders. On-demand, up-to-date on student progress toward ACT and graduation are available for teachers at SSMHS. Funding provided through the STEM School grant, as well as a Magnet School grant include discretionary funding that can be used to advance the implementation of the STEM focus. SSMHS was rated “Advanced” on the B3 attribute at both baseline (December 2012) and the second site visit (May 2013). In year two, with the development of the STEM Magnet Standards and continued emphasis of the STEM vision to all stakeholders, the rating for this construct was “Advanced” at both fall 2013 and spring 2014 site visits. SSMHS also developed a video documentary that is now on their website that explains to the public about the STEM approach at the school. Alignment of Career Pathways with Post-secondary (C1) Teachers at SSMHS do not plan vertically with the feeder middle schools in the district. Students at SSMHS do have career counselors that work closely with students to plan and track their pathways to careers and post-secondary education. Teachers are not involved in this process for the most part. However, academy teachers and some core teachers do share information with students about STEM careers and pathways frequently. The rating for SSMHS at baseline (December 2012) was “Early” which improved to “Developing” by the second site visit (May 2013) due to efforts to get teachers more actively involved in the career counseling process. In the second year of the evaluation, SSMHS continued to support students with career counseling at least quarterly. However, teachers and administrators have not identified a plan for vertical articulation with feeder schools. The rating for SSMHS in this area remained at “Developing” for the fall 2013 and spring 2014 site visits. Post-secondary Credit Completion (C2) At baseline (December 2012), SSMHS had no formal course offerings for credit completion in place. However, by the second site visit (May 2013), SSMHS has partnered with Nashville State Community College to offer a computer applications course that had an enrollment of 20 students. Both teachers and career counselors understand the importance of credit completion and SSMHS has plans to explore the options for students in the coming year. The baseline rating for SSMHS was “Early” and the second site visit (May 2013) rating improved to “Developing” on the C2 attribute. In year two there were plans for adding on additional dual credit coursework. However, the rating for this attribute remained at “Developing” for fall 2013 and spring 2014 site visits.


RESEARCH QUESTION 9: Is community awareness and support for STEM increased as a result of the TSIN? An examination of the SCAS findings related specifically to the MTSI Hub was conducted by completing an analysis of the respondents who reported living in one of the counties included in the service area (e.g., Bedford, Cheatham, Coffee, Davidson, Dickson, Franklin, Giles, Hickman, Houston, Humphreys, Lawrence, Lewis, Lincoln, Marshall, Maury, Moore, Perry, Robertson, Rutherford, Stewart, Wayne, Williamson, Wilson). The number of participants who completed the survey at each administration from the MTSI region included: 924 for fall 2012, 552 for spring 2013, 164 for fall 2013, and 430 for spring 2014. Overall, all three stakeholder groups (K-12 Educators, Business/Community Stakeholders, and Higher Education Faculty) increased their agreement positively in most areas across the two-year evaluation. There were four main areas that all three stakeholder groups experienced growth in agreement in across the two years of the evaluation.

• All three MTSI Hub stakeholder groups reported growth in K-12 schools in the region understanding the importance of STEM education. Business/Community reported 86% agreement at baseline, which increased to 97% at the end of the second year. Similarly, Higher Education faculty reported 73% agreement at baseline to 86%, and K-12 teachers reported 72% agreement at baseline which rose to 76%) in agreement at end of the two years.

• A second area of growth in agreement across the two-year evaluation for all three groups was parental understanding of the importance of STEM Education. Higher Education reported 26% growth (43% to 69%), Business/Community reported 22% growth (70% to 92%) and K-12 recorded 18% growth (31% to 49%).

• The third area for the MTSI Hub where all stakeholders increased in agreement was the increase of K-12 STEM opportunities for students in the region. Business/Community agreement raised from 73% to 88%, Higher Education increased from 70% to 74% agreement, and K-12 teachers’ agreement increased from 54% to 57%.

• The final area of increased agreement for all groups was that preparing students for careers in STEM is a top priority for the region. Business/Community agreement raised from 48% to 51%, Higher Education increased from 38% to 52% agreement, and K-12 teachers’ agreement increased from 46% to 54%.

The Business/Community Stakeholder group did not experience decline in agreement on any items of the survey. Higher Education and K-12 had a few areas of decline, but none of these were common across the two groups.


Table 48. K-12 Educators Areas of Growth in Agreement on SCAS for MTSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

I have had business/community funded STEM education programs or events in my school or school district. 54% 59% 46% 62%

I have had community/business volunteers for STEM education programs or event in my school or school district. 53% 60% 51% 58%






Local organizations recruit STEM talent online. 39% 44% 34% 61% Information related to STEM opportunities in my region is available online. 58% 65% 54% 72%

There are other STEM online tools available to this region. 61% 69% 55% 74% Students in this region are knowledgeable about the STEM careers that will be in high demand when they graduate. 43% 46% 49% 56%

There are businesses that provide signing bonuses or incentives for workers choosing a STEM career in the region. 28% 29% 19% 53%





76% 79% 62% 82%

All students should receive information about careers that are expected to be in demand in the region when they graduate from K-12 schools and post-secondary institutions.

94% 92% 96% 97%




75% 79% 62% 82%



Table 49. K-12 Educators Areas of Decline in Agreement on SCAS for MTSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014



My organization is interested in providing STEM education opportunities for K-12 students in this region. 66% 69% 63% 61%

Table 50. Business/Community Stakeholder Areas of Growth in Agreement on SCAS for MTSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014

There are opportunities for K-12 students in our region to complete internships or co-ops in this region. 68% 75% No

participants 90%

My organization has funded STEM education program or events in local schools or districts. 61% 53% No

participants 77%

Employees from my organization have volunteered for STEM education programs in local schools or districts. 63% 53% No

participants 79%

Employees from my organization have been guest speakers in local schools or districts. 73% 57% No

participants 85%

There are opportunities for K-12 students to complete internships or co-ops in the region. 79% 100% No

participants 90%

There are organizations interested in providing STEM education opportunities for K-12 students in this region. 76% 67% No

participants 88%

Overall, there has been an increase in K-12 STEM opportunities for students in the region in the last year. 73% 59% No

participants 88%

I have worked closely with K-12 schools through my role as a business/community stakeholder. 53% 33% No

participants 76%

The K-12 schools in this region understand the importance of STEM education. 86% 89% No

participants 97%

Parents in this region understand the importance of STEM education. 70% 60% No

participants 92%


92% 100% No participants 96%

Preparing students for careers in STEM is a top priority for schools in the region. 48% 40% No

participants 51%


Table 51. Higher Education Faculty Areas of Growth in Agreement on SCAS for MTSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014



Overall, there has been an increase in K-12 STEM opportunities for students in the region in the last year 70% 76% 54% 74%

I have worked closely with community/business organization members in my role as a faculty member. 33% 35% 38% 59%



More work needs to be completed to spread awareness of STEM education. 92% 90% 93% 95%













92% 91% 95% 96%

All K-12 schools in this region teach the STEM skills and knowledge appropriate for jobs that will be available in the region. 32% 48% 52% 56%




Table 52. Higher Education Faculty Areas of Decline in Agreement on SCAS for MTSI Hub

SCAS Item Fall 2012

Spring 2013

Fall 2013

Spring 2014


My institution has provided volunteers for STEM education programs or event in K-12 schools in our region. 94% 97% 63% 70%

My organization understands the importance of STEM education. 95% 97% 77% 87% There are organizations interested in providing STEM education opportunities for K-12 students in this region. 87% 74% 68% 75%

There are colleges and/or universities and/or community colleges that offer scholarships for students to pursue STEM degrees in my region. 87% 95% 75% 80%



CONCLUDING OBSERVATIONS - Middle Tennessee STEM Innovation Hub (MTSI) and Stratford STEM Magnet High School The TSIN investment in the MTSI Hub and STEM platform school resulted in positive outcomes for both entities. The MTSI Hub is administered by the Metro Nashville Public Schools (MNPS) under the leadership of Dr. Vicki Metzgar. The MTSI Hub developed a diversified portfolio of teacher professional development and student STEM opportunities across the two years of the TSIN evaluation. A key to this success was leveraging the numerous higher education institution partners across the region to develop and deliver STEM PD for teachers. This shared organizational model enabled funding to be used more efficiently to offer an array of sessions for a variety of teachers. The STEM Expo achieved great success as a forum for celebrating student STEM innovation annually. The MTSI Hub also supported Stratford STEM Magnet High School (SSMS) in connecting regional partners to the school and providing some resources to support activities there. At the end of the evaluation, MNPS had agreed to support the MTSI Hub into the year following the TSIN funding. However, MTSI is also working with other partners (e.g. Belmont) to create a long-term solution to supporting the sustainability of the Hub. SSMS made great progress in the first three years of the school. The impact on student progress and performance on college-ready assessments is considerable (as reported to the evaluator). There are vibrant, STEM-career focused STEM Academy choices in place for students. The SSMS has cultivated strong partnerships with STEM industry to support most of the pathways. SSMS holds significant promise to be a best-in-class model for TSIN. This can be achieved by more focus on infusing the pathway STEM content, PBL, and 21st Century Skills across the core coursework (e.g. mathematics, social studies).


Fullan’s (2006) change theory explicates the necessary conditions for achieving reform. This lens is being used to examine the progress of the TSIN in achieving STEM educational reform in Tennessee through their series of regional STEM innovation networks and STEM platform schools. MTSI Hub stakeholder collective motivations for being engaged in the STEM reform initiative ranged from a desire to improve MNPS, to the desire to spark interest in K-12 students in STEM careers and post-secondary study. Capacity building was a major focus of the MTSI Hub and the SCAS revealed all three stakeholder groups reported growth in K-12 school understanding of the importance of STEM education, increased parental understanding of the importance of STEM education, and increase in STEM opportunities for students in the region. A final area of increased agreement for all three stakeholder groups was that the top priority for K-12 schools in the region was to prepare students for careers in STEM. There were no areas of decline that were common across all three stakeholder groups, indicating that overall there has been progress in the MTSI Hub region in many key areas without any common areas of concern for all. In regards to the role of MTSI in generating new knowledge and strategies within the local context, as well as changing the larger context, there was considerable effort focused on this through the STEM-Posiums each year, where regional stakeholders came together to discuss critical issues in STEM for the region. The MTSI Hub also focused on tri-level engagement and has representation from K-12, community/business, higher education, and TSIN in decision making for the Hub. MTSI has excellent engagement from higher education and K-12 stakeholder groups. MTSI did not achieve the desired level of business/industry engagement with the Hub, and, although there are key anchor partners, an urban region of this size should have many more partners at the table. This may be a key area to target in the coming year as the MTSI Hub develops future sustainability infrastructure beyond the MNPS commitment to extend the work one year.


IX. CONCLUDING OBSERVATIONS The Race to the Top investment into the Tennessee STEM Innovation Network (TSIN) resulted in the establishment of six TSIN Regional STEM Innovation Hubs and seven TSIN STEM Platform Schools. The two-year evaluation of the TSIN investment indicated a positive impact for TSIN across the state of Tennessee in building STEM teacher quality, growing STEM stakeholder awareness, and forming partnerships and STEM focused schools that have produced many innovative strategies and approaches that will move the state of Tennessee and the nation forward. SUSTAINABILITY OF TSIN Overall, most aspects of the TSIN will be sustained at the regional level within Regional STEM Innovation Hubs and STEM Platform Schools. The WTSC, UCRSI, SETN, and ETSU Hubs have fully functioning sustainability plans that have begun implementation. The MTSI Hub has a commitment from MNPS to fund a 120-day extension to provide time to secure partners for sustainability. The STEMSpark Hub has ceased operation and the future of this Hub is under consideration by the Knox County School System. Five STEM Platform Schools have clear sustainability plans and long-term school district commitments, driven by strategic planning locally, including L&N STEM Academy, STEM School Chattanooga, Stratford STEM Magnet High School, Prescott South STEM Elementary School, and Prescott South STEM Middle School. Southwind STEM High School has had considerable teacher and administrator turnover, as well as district consolidations taking place, though it appears that the school will continue to function in the 2014-15 school year. Leadership within Sullivan County Schools moved to close Innovation Academy and to merge students and a few existing staff into an existing middle school in district due to budgetary and administrative decisions. This school will not continue to exist as it was designed and implemented through the TSIN investment. However, Innovation Academy will be a school within a school at another middle school in the same district. At the state TSIN level, an application for no cost extension of Race to the Top funding has provided TSIN staff the opportunity to aggressively seek out key partners and targeted funding to sustain the TSIN. Overwhelmingly the stakeholders interviewed, including Hub leadership and partners, voiced the need for continuing to have state-level connectivity of Regional STEM Innovation Hub and STEM Platform School efforts. This ongoing connectivity will continue to allow others to learn from the TSIN investments. Many K-12 schools have already benefited from visiting these sites and attending professional development sessions to learn how to implement STEM. The TSIN is a national model for establishing strong, locally driven infrastructure for supporting educational reform.


CONNECTIVITY OF NETWORK It is clear that the TSIN has achieved an intricate and efficient model of connectivity across the state of Tennessee during the two years of this evaluation. The format of the TSIN provides multiple pathways for dissemination both within regional Hubs and across the state. Monthly network conference calls are conducted where regional sites can add agenda items and provide updates on resources, opportunities, challenges and lessons learned. The TSIN collective group acts as a sounding board for innovation and problem solving group to address issues. The TSIN leadership also has developed Action Teams to address various key components of the network including Advocacy and Partnerships, Communications and Marketing, and Resources. Leadership of the TSIN (state and regional) are engaged in continuous professional growth and are actively involved in learning from others as well as broadening capacity statewide through delivery of training sessions to enable others to implement STEM. The TSIN collective group is also engaged in STEM nationally, as a member of STEMx and as a participant at the annual U.S. News and World Report STEM Solutions Conference each year (most recently as an invited panel to share experiences). Innovations emerging from Hubs and Schools have been extensively, frequently, and broadly disseminated across the network and the state at large. The connectivity that has been afforded by the TSIN has already begun to pay dividends indirectly through improved teaching and learning of STEM in Tennessee. GROWTH IN TEACHER EFFECTIVENESS The TSIN evaluation utilized data from the 29 THEC STEM Professional Development (PD) programs to determine the impact of Tennessee teacher participation in STEM professional development on growth in STEM teacher quality. THEC STEM PD included over 600 teachers from across the state from various grade bands (elementary, middle, secondary) and content areas (science and mathematics). The span of the THEC STEM PD reached across the state and into each of the six TSIN Regional STEM Innovation Hubs. An analysis of the data using the LSC protocol found significant growth in STEM pedagogical skills, as observed in participant-submitted digital recordings of their instruction. The ability of teachers to design effective STEM lessons increased from 2.39 to 2.88 on the 5-point scale. Teacher implementation of effective STEM instruction also increased significantly from 2.61 to 3.30. Additionally, participants were able to transform their learning environments and create classroom culture, which supports investigative STEM education (2.84 to 3.48). Finally, THEC STEM participating teachers increased their use of accurate STEM content knowledge from a baseline rating of 2.90 (prior to participation in the program) to 3.50 at the end of the STEM professional development program. The THEC STEM PD program (funded through Race to the Top) produced over 600 teachers who delivered more effective STEM instructional practices, reaching thousands of students across Tennessee.


GROWTH IN STUDENT STEM OPPORTUNITIES The TSIN investment into Regional STEM Innovation Hubs and associated other STEM activities across the state during the two-year evaluation period resulted in more perceived opportunities for students in STEM. Specifically, for the overall population surveyed, the percentage agreement raised from 58.1% in fall 2012 to 70.7% in spring 2014. This growth was also reflected in all three-stakeholder groups (Business/Community Stakeholders, Higher Education Faculty, and K-12 Educators). Business/Community Stakeholder perception of K-12 student STEM opportunities rose from 61.8% (fall 2012) to 77.8% (spring 2014). Similarly, Higher Education Faculty agreement improved from 70.1% (fall 2012) to 74.1% (spring 2014). Finally, K-12 Educator agreement increased from 54.2% (fall 2012) to 60.1% (spring 2014). The findings in this area were also reflected in the six TSIN Regional STEM Innovation Hubs where each Hub had at least 64% agreement by end of the second year, indicating most participants perceived more K-12 STEM student opportunities in the region. GROWTH IN STEM STAKEHOLDER AWARENESS The STEM Community Awareness Survey (SCAS) was administered at six-month intervals across the two-year evaluation, beginning in fall 2012 and ending in spring 2014. The analysis of data revealed growth in STEM awareness for participants on the constructs of Industry Engagement in K-12 STEM, Community STEM Awareness and STEM Resources, Preparation of K-16 Students in STEM, and Awareness of STEM Careers and Workforce Needs. Participants had a moderately favorable attitude toward STEM awareness and support on the entire survey, averaging “Agree” Overall at the end of the evaluation (Baseline M = 2.73; End M = 2.98). Additionally, participants improved their reported perceptions in the following survey subsections to a level of “Agree” by end of evaluation: Industry Engagement (Baseline M = 2.73; End M = 3.04), Awareness and Resources (Baseline M = 2.91; End M = 3.16), and Careers and Workforce (Baseline M = 2.69; End M = 2.91). However, the survey subsection (construct) Preparation of Students had the lowest reported average scores, which improved but were not at an “Agree” level at the end of the evaluation (Baseline M = 2.44; Follow-Up M = 2.66). The TSIN made substantial progress in transforming Tennessee Stakeholder STEM Awareness in the areas of Industry Engagement, Awareness and Resources, Careers and Workforce, though continued effort is needed in the Preparation of Students K-20. Table 8 shows results for the overall survey and each subsection, along with average net change from baseline to end. A further examination of data for each individual TSIN Regional STEM Innovation Hub revealed specific areas of growth and identified areas of needed focus in the future for the six Hubs (SETN, UCRI, ETSU, STEMSpark, WTSC, and MTSI). The TSIN investment has produced increased Community Stakeholder STEM Awareness across the state of Tennessee. TSIN INNOVATIONS The TSIN investment in Regional STEM Innovation Hubs and STEM Platform Schools has resulted in the emergence of eight key “best-in-class” TSIN innovations that are worthy of disseminating on a national scale and have the potential to move STEM reform forward in a variety of settings and with diverse populations.


The first area of innovation is the STEM School Engagement Model in East Tennessee, which consists of the partnership between the ETSU Hub and Innovation Academy (IA). As discussed in the ETSU Hub Narrative, the level of partner engagement in the educational process at IA was outstanding. STEM professionals from a variety of organizations, worked alongside STEM teachers to develop and implement (co-teaching) innovative STEM integrated curriculum and real-world learning experiences – including a focus on service learning for middle school students. This STEM-rich learning environment is one to which other schools and Hubs should aspire. In addition, the real-time, on-the-job professional development support provided by ETSU STEM faculty and STEM professionals is an approach that will support the successful implementation of integrated STEM education. The second area of innovation is the STEM Arc High School Curriculum Series at L&N Academy in Knoxville, TN within the STEMSpark Hub. Implementing STEM curriculum within a traditional high school is difficult at best, with all the requirements for graduation, athletics, and advanced placement/dual-credit coursework. The L&N STEM Arc curriculum, with the series of STEM coursework delivered outside of the content coursework may be a solution for moving many more high schools toward implementing high-quality STEM preparation for students infused with STEM professionals and individualized, student-driven, real-world internships for all seniors at L&N STEM Academy. The third area of innovation is the SETN Hub’s Model for STEM Advocacy and Awareness. The SETN Hub’s combination of pushing out media based STEM awareness and advocacy materials (e.g. PSAs, Innovate Education Channel) through a partnership with local cable providers, coupled with grass roots efforts to build K-12 capacity for implementation through the STEM Teaching Fellows, has potential to be a model for regional transformation in STEM. This two-pronged approach to generating capacity to implement and sustain STEM has shown considerable promise in the first year of the evaluation. The fourth area of innovation is the Rural Model for STEM Outreach being implemented by the UCRSI Hub. The key to this regional work is the STEM Mobile, which will work to bring STEM to schools in rural areas. The STEM Mobile is a state-of-the-art STEM laboratory on wheels that will travel on a schedule through UCRSI partnering counties to visit schools for extended opportunities for students to experience STEM. This great strategy leverages limited resources in a mobile format so that thousands of students and teachers can benefit from the investment. The majority of the STEM Mobile expense has been covered with grant and donor funding, making sustainability of this resource attainable with commitments from partners in the UCRSI Hub. Further, the UCRSI Hub also sponsors an annual Rural STEM Education Conference designed to grow STEM teacher capacity in the region. A fifth area of innovation is the Early STEM Learning Experiences being developed and implemented at Prescott South Elementary School in the UCRSI Hub. In kindergarten, students are being engaged in problem-based and project-based learning experiences with ties to the real world. The PSES model for harnessing the creativity and curiosity of children is one that can be leveraged to scale up early STEM learning across the state. These in-school opportunities are coupled with extensive out-of-school programs at Prescott South that bring the community and real work further into the lives of students at a very early age.


A sixth area of innovation is the WTSC’s Hub Model of Shared Leadership that engages Hub partners far beyond a typical advisory capacity. WTSC has all hands on deck to not only conceptualize, problem-solve, and design solutions, but also to create a forum where partners have rolled up their sleeves to be a part of doing the work of the WTSC Hub. WTSC has established great capacity (e.g. STEM Ambassadors, working groups) through bringing partners into the fold as compared to other Hubs who conduct the work with Hub staff and a very few volunteers. This model should be explored as a means to sustaining the work TSIN has started in the state. A seventh of innovation for the TSIN is the MTSI Hub’s STEM Expo. The annual event is designed to showcase K-12 student STEM innovation in a competitive environment. The STEM Expo complements other more targeted STEM opportunities like FIRST Robotics since it is open-ended and encompasses the vast diversity of STEM careers. This event should be leveraged and scaled up to make it open to all students in Tennessee. An eighth and final area of innovation for the TSIN is the 1:1 Technology Integration for promoting authentic STEM learning at Innovation Academy, L&N Academy, and STEM School Chattanooga. These three schools have achieved complete infusion where students are creating new solutions, not just using technology to consume information, raising the level of technology use to a daunting level. The Apple national designations of two of these schools (IA and L&N) speaks directly to the level of innovation in implementation within the TSIN. SUMMARY This final evaluation report for the TSIN focused on the 2012-2014 school year data and has addressed Research Questions 1-9. Overall, this two-year evaluation has revealed areas of innovation and considerable growth in the STEM arena for the state of Tennessee. The TSIN Regional STEM Innovation Hubs and STEM Platform Schools have had identifiable impact and some innovative models have emerged from the research and development format of the network and STEM platform schools. At the conclusion of this evaluation, coinciding with the end of Race to the Top Funding, a clear success story is evident in the fact that many of the Hubs and schools that were enabled by start-up funding from the TSIN will continue to exist and prosper indefinitely through established, strong partnerships and commitments from organizations and stakeholders. Next steps for TSIN will be to leverage the emergent innovations and disseminate to other Tennesseans for the benefit of all. There is much that can be learned from those entities that were and were not sustained. The state of Tennessee will reap the rewards of the STEM investment for many years to come.


REFERENCES Fullan, M. (2006). Change theory: A force for school improvement [Centre for Strategic Education Seminar

Series Paper No. 157]. Retrieved from http://www.michaelfullan.ca/articles/. Onwuegbuzie, Anthony J. (2000, November). Expanding the framework of internal and external

validity in quantitative research. Paper presented at the annual meeting of the Association for the Advancement of Educational Research, Ponte Vedra, FL.

Appendix: Tennessee STEM Innovation Network Evaluation Annual Report|138

APPENDICES Appendix A – Focus Group Interview Questions ………………………………111 Appendix B – Principal Interview Questions ………………………………….....112 Appendix C – Teacher Interview Questions…………………………………..…113 Appendix D – STEM Attributes Rubric ……………………………………….114


APPENDIX A FOCUS GROUP INTERVIEW QUESTIONS

1. Tell me how you have been involved in the activities of the Hub this past year.

2. How have you been involved in supporting K-12 education prior to working with the Hub and/or school? 3. What are your motivations for being involved in the STEM Hub?

4. What are some of the challenges your region experiences related to education?

5. What are some of the regional STEM activities or opportunities for students

that you are aware of?

6. How would you describe parent engagement in K-12 overall for your region?

7. What are the best ways to prepare high school students to be successful in STEM careers and college? 8. What are some of the ways you believe industry and business should be involved

in supporting K-12 education?

9. What are the strengths of your region in regards to STEM?

10. How successful do you believe the Southeast TN STEM initiative has been so far?

11. What is one area that you would like to see the regional Hub focus on this year?

12. Any other comments to share?


APPENDIX B PRINCIPAL INTERVIEW QUESTIONS

1. Describe the STEM School for me? What is the vision for the school?

2. How were students selected to attend the STEM School?

3. Describe the STEM Curriculum.

4. How were teachers selected for the STEM School?

5. How are teaching and learning different at this school?

6. What types of professional development have teachers been provided? Future plans?

7. Describe the guidance support, specifically career guidance students receive here.

8. What types of planning time do teachers have?

9. How are partners engaged in the work of the school?

10. What are the dual credit opportunities for students here? What percent of students will participate?

11. Describe the access to technology students have here.

12. What have been some of the challenges of getting the school up and running?

13. What types of experiences do students have outside of the school in STEM?

14. What types of assessments do you use to determine student progress?

15. How is student work exhibited?

16. How are students exposed to STEM careers outside of guidance?

17. What ways does the school communicate with the public?

18. Is there a developed STEM program plan (beyond 9th)?

19. How are the funds from the grant being invested at the school?


APPENDIX C TEACHER INTERVIEW QUESTIONS

1. Tell me about your background and what you did prior to coming to this school?

2. What is your role here at this school and why did you choose to teach here?

3. What is your definition of STEM?

4. What do you believe is the most effective way to teach science and/or mathematics?

5. Describe the current collaboration opportunities for teachers at this school.

6. What do you think will best prepare your students to be successful in the future in STEM?

7. What is your vision for the school over the next couple of years?

8. How is teaching and learning different here than at other schools?

9. How do you use technology on a daily basis in your classroom?

10. Tell me about the challenges you or other teachers experience implementing the STEM approach here at this school.

11. How would you describe overall community awareness of STEM?

12. What is the most important accomplishment for the school thus far?

13. Do you feel like you are supported to by administration at this school to implement STEM?

14. What types of support would you like to have to improve your ability to teach STEM?

15. Describe how you use assessment results to refine your practice? How do you assess student learning?

16. Describe the professional development opportunities you receive at this school.

17. What are some of the experiences students have outside of school here in STEM?

18. How are partners engaged in the work of the school?


APPENDIX D

STEM Program Implementation Rubric

Pilot - December 2011

Page 2 contains 11 “STEM Attributes,” or characteristics and strategies of effective STEM programs - a framework created by the North Carolina Department of Public Instruction. The remainder of the document is a pilot STEM program implementation rubric based on these 11 attributes. The rubric was created by The Friday Institute, with backing from The Golden LEAF Foundation, to support schools and districts to build their STEM programs.

HOW TO USE THIS RUBRIC: The rubric acts as a framework for building STEM programs and is based on the North Carolina Department of Public Instruction’s list of 11 STEM-program attributes. It outlines a four-stage implementation continuum for key elements of each of the 11 attributes, with the Target level of implementation at the far right. With your leadership team, discuss each page of the rubric and highlight the cells that best describe your school or program. Where you see space in the table, make notes about your school or program. What does your STEM school or program look like? What is/are 1-3 action steps that your school or program might take to advance in these areas? These notes can act as a reminder or future reference point in future planning meetings.

These evaluation instruments were identified, modified, or developed through support provided by the Golden LEAF Foundation. The Friday Institute grants you permission to use these instruments for educational, non-commercial purposes only. You may use an instrument as is, or modify it to suit your needs, but in either case you must credit its original source. By using this instrument you agree to allow the Friday Institute to use the data collected for additional validity and reliability analysis. The Friday Institute will take appropriate measures to maintain the confidentiality of all data. For information about additional permissions, please contact Dr. Jeni Corn, Director of Evaluation Programs at The Friday Institute, at [email protected].


STEM Attributes **

Reference STEM Implementation Rubric

Early

Developing

Advanced

Target

Integrated Science, Technology, Engineering and Mathematics (STEM) curriculum, aligned with state, national, international and industry standards

A1) Project-‐based learning with integrated content across STEM subjects A2) Connections to effective in and out-‐of-‐school STEM programs A3) Integration of technology and virtual learning A4) Authentic assessment and exhibition of STEM skills A5) Professional development on integrated STEM curriculum, community/industry partnerships and postsecondary education connections

A6) Outreach, support and focus on underserved, especially females, minorities, and economic disadvantaged

On-‐going community and industry engagement B1) A communicated STEM plan is adopted across education, communities and businesses

B2) STEM work-‐based learning experiences, to increase interest and abilities in fields requiring STEM skills, for each student and teacher

B3) Business and community partnerships for mentorship, internship and other STEM opportunities that extend the classroom walls

Connections with postsecondary education C1) Alignment of student’s career pathway with post-‐secondary STEM program(s)

C2) Credit completion at community colleges, colleges and/or universities

North Carolina Department of Public Instruction STEM Education Schools and Programs STEM Attribute Implementation Rubric

STEM Attributes describe a quality STEM education school or program. For each attribute, there are criteria to describe an Early, Developing, Prepared, or Targeted school or program. These criteria will help schools learn the steps needed to become a prepared or targeted quality program. STEM Attributes are based on local, state and national research and public feedback from 125 practitioners, educators, and business leaders.

2


(A1) Curriculum: Project-based learning (PBL) with integrated content across subjects

Early (Starting)

Developing

Advanced (Prepared)

Target

Description of Your School/ Program

Action Steps

Fr

eque

ncy

of P

BL

Project-based learning is used rarely in 1-2 subject(s)/grade

level(s), providing few learning experiences that have high

potential for student engagement (e.g., using

technology tools, participating in issues- or community-based

activities, and completing capstone projects that address

real-world problems)

Project-based learning is used occasionally in more than 2 STEM

subjects/grade levels, providing some learning experiences have

high potential for student engagement (e.g., using

technology tools, participating in issues- or community-based

activities, and completing capstone projects that address

real-world problems)

Project-based learning is used

frequently in all STEM subjects at all grade levels so that many

learning experiences have high potential for student engagement

(e.g., using technology tools, participating in issues- or

community-based activities, and completing capstone projects that

address real-world problems)

Project-based learning is used regularly across multiple subjects

at all grade levels, so that a majority of learning experiences have high potential for student

engagement (e.g., using technology tools, participating in

issues- or community-based activities, and completing

capstone projects that address real-world problems)

Mul

ti-

subj

ect

PLC

s

No common planning time

focuses on integrating teaching and learning across

grades/content areas

Annual common planning time focuses on integrating teaching

and learning across grades/content areas

Biannual common planning time focuses on integrating teaching


Quarterly common planning time focuses on integrating teaching


ST

EM

PLC

s

Teachers occasionally share lessons and activities through infrequent, common planning

and professional learning community meetings

In their professional learning communities teachers

occasionally share lessons and activities that promote higher-

level thinking

In their professional learning communities teachers frequently

share and co-create new or improved activities that promote

higher-level thinking

In their professional learning communities teachers regularly

share and co-create new or improved activities that promote

higher-level thinking

Freq

uenc

y of

ST

EM

Inte

grat

ion

Up to 25% of teachers make

explicit efforts to integrate STEM across core subjects,

requiring students to synthesize knowledge across disciplines

26-50% of teachers make explicit efforts to integrate STEM across core subjects, requiring students to synthesize knowledge across

disciplines

51-75% of teachers make explicit efforts to integrate STEM across core subjects, requiring students to synthesize knowledge across

disciplines

Over 76% of teachers make

explicit efforts to integrate STEM across core subjects, requiring

students to synthesize knowledge across disciplines

Phys

ical

Computer labs or classrooms are transformed into

collaborative spaces and project work areas when necessary

At least one space is available specifically for student

collaboration and project work

At least 2 facilities and spaces are available specifically for

student collaboration and project work

Multiple facilities and spaces are available for face-to-face and virtual collaboration among

students and teachers, including small group learning areas,

project rooms, inquiry studios, and exhibition spaces

Spac

e

3


(A2) Curriculum: Connections to effective in- and out-of-school programs

Early (Starting)

Developing

Advanced (Prepared)

Target Description of

Program

Action Steps

ST

EM N

etw

ork

Program has limited engagement with a STEM network, participating in occasional cross-sector

partnerships and collaborations

Program has occasional engagement with a STEM network and is seeking to establish a few partnerships with other schools, communities, post-secondary

institutions, and business/industry

Program frequently engages in a STEM network, maintaining

several partnerships with other schools, communities, post- secondary institutions and

business/industry experts and resources

The program continuously engages in a STEM network, maintaining

multiple partnerships and establishing new ones that

connect schools to communities, post-

secondary institutions and STEM business/industry experts and resources

R

esea

rch

&

Dev

elop

men

t Program leaders and

participants do not access and share research and best practices related to their

program goals

Program leaders and participants

occasionally access and share research and best practices related

to their program goals, and occasionally use this data for

program improvement

Program leaders and

participants frequently access and share research and best

practices related to their program goals, and use this

data for program improvement

Program faculty/staff regularly access and

share research and best practices related to their program goals, and use

this data for program improvement

St

uden

ts a

nd S

TEM

Leaders are creating plans to

provide opportunities for students to meet STEM

professionals and to participate in STEM learning environments outside school (e.g. field trips,

clubs, competitions, study trips, internships, and

summer/afterschool/weekend programs taught by STEM teachers and/or industry

professionals)

Direct experiences with STEM professionals and STEM learning

environments both during and outside school are available to

students 1-2 times throughout the year (e.g. field trips, clubs, competitions, study trips,

internships, and summer/afterschool/weekend

programs taught by STEM teachers and/or industry professionals)

Direct experiences with STEM

professionals and STEM learning environments both

during and outside school are available to students several

times throughout the year (e.g. field trips, clubs, competitions,

study trips, internships, and summer/afterschool/weekend

programs taught by STEM teachers and/or industry

professionals)

Direct experiences with STEM professionals and

STEM learning environments both during

and outside school are available to students

continuously throughout the year (e.g., field trips,

clubs, competitions, study trips, internships, and

summer/afterschool/week end programs taught by STEM teachers and/or industry professionals)

Prof

essi

onal

s

Your School/

4


(A3) Curriculum: Integration of technology and virtual learning

Early (Starting)

Developing

Advanced (Prepared)

Target Notes on Your

Program

Action Steps

Com

mon

Te

chno

logy

Common technology resources linked to standards and

curriculum have been identified

Common technology resources

linked to standards and curriculum are available for teachers and

students; up to 50% of students have mastered common technology

applications

Common technology resources

linked to standards and curriculum are being used by

most teachers and students; 51- 85% of students have mastered

common technology applications

Common technology resources linked to

standards and curriculum are being used by all

teachers and students; more than 86% of students

have mastered common technology applications

In

tegr

ated

Tec

hnol

ogy

A few virtual, computer-based, mobile, and other technology tools are used infrequently to support teaching and learning

Virtual, computer-based, mobile, and other technology tools are used

occasionally to support teaching and learning through activities such

as web-based lessons, projects requiring students to use computer

applications and other online learning activities

Virtual, computer-based, mobile, and other technology tools are used frequently to

support teaching and learning through activities such as web-

based lessons, projects requiring students to use

computer applications and other online learning activities

Virtual, computer-based, mobile, and other

technology tools are integrated seamlessly into

teaching and learning, including web-based

lessons on standards- based content, projects

requiring students to use computer applications, online communication between and among

teachers and students, etc.

Te

ch R

esou

rces

fo

r Tea

cher

s

Teachers have occasional access to digital instructional

resources for STEM

Teachers have frequent access to digital instructional resources for

STEM

Teachers have on-demand access to digital instructional

resources for STEM throughout the entire school, and teachers

receive occasional STEM resource notifications and

updates

Teachers have on- demand access to digital instructional resources for

STEM in various instructional settings (e.g.

school, home, community) and teachers

receive regular STEM resource notifications and

updates

Te

ch

Teachers and administrators rarely have access to tech

support for both maintenance and consulting

Teachers and administrators

occasionally have access to tech support for both maintenance and

consulting

Teachers, administrators and

students have frequent access to tech-support for both

maintenance and consulting

Teachers, administrators and students have on-

demand access to tech- support for both

maintenance and consulting

Supp

ort

School/

5


(A4) Curriculum: Authentic assessments and exhibition of STEM skills

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

Aut

hent

ic

Ass

essm

ents

Teachers are being encouraged and supported to use multiple indicators of student success,

including performance, project- based and portfolio

assessments

As many as 50% of teachers use multiple indicators of student

success, including performance, project-based and portfolio

assessments

51-75% of teachers use multiple

indicators of student success, including performance, project-

based and portfolio assessments

All teachers and students are immersed in a student-

centered learning environment that supports

the use of multiple indicators of success, such

as performance, project- based and portfolio

assessments

Te

ache

rs C

olla

bora

tivel

y D

evel

op A

sses

smen

ts

Teachers do not share assessment strategies (e.g. formative, benchmark and

summative assessments or performance-based

assessments)

A couple times a year teachers share assessment strategies (e.g.

formative, benchmark and summative assessments or

performance-based assessments); they occasionally co-create

assessments

Teachers collaborate quarterly to discuss strategies for

analyzing student performance and for using results to inform

instruction, and to develop multiple measures of student

success (e.g. formative, benchmark, summative, and

performance-based assessments)

Teachers collaborate at least monthly to discuss strategies for analyzing

student performance and for using results to inform

instruction, and to develop multiple

measures of student success (e.g. formative, benchmark, summative, and performance-based

assessments)

C

ultu

re o

f In

nova

tion

Students, teachers and administrators rarely celebrate

high-quality student work in STEM

Students, teachers and

administrators celebrate high- quality student work in STEM with

occasional on-site and online exhibits

Students, teachers and

administrators celebrate high- quality student work in STEM

with frequent on-site and online exhibits

Students, teachers and administrators celebrate high-quality student work

in STEM through ongoing student exhibits on- site, online and/or in state

and national forums

C

eleb

rate

Program leadership

occasionally honors and encourages innovation among

students

Program leadership frequently honors and encourages innovation

among students

Program leadership and

program participants frequently honor and encourage

innovation among both faculty and students

Program culture consistently honors,

encourages and incentivizes innovation

among faculty, students, parents, and others

STEM

Wor

k

Your School/

6


(A5) Curriculum: Professional development on integrated STEM curriculum, community/industry partnerships and connections with post-secondary education

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

Indi

vidu

aliz

ed

PD

Teachers participate in large group professional development sessions to acquire basic STEM

skills

Teachers participate in large group professional development sessions

focusing on building capacity to integrate STEM effectively into

content areas, with follow-up that facilitates implementation

Individual teachers have unique STEM professional development

goals and are able to tailor as much as 50% of their

professional development activities to meet their individual

needs

Individual teachers have unique STEM professional

development goals and are able to tailor over 75%

of their professional development activities to

meet their individual needs

Job-

em

bedd

ed

PD

Job-embedded approach to professional development, with opportunities for practice and

reflection, is rarely used

Job-embedded approach to professional development, with opportunities for practice and

reflection, is occasionally used

Job-embedded approach to professional development, with opportunities for practice and reflection, is frequently used

Job embedded approach

to professional development, with

opportunities for practice and reflection, is regularly

used

Spec

ific

to

Stud

ent-

Lear

ners

Professional development

resources lack specificity and focus on standardized, scripted

teaching strategies


resources occasionally focus on specific STEM content for specific

types of student-learners


resources frequently focus on specific STEM content for specific types of student-

learners

Professional development resources regularly focus on specific STEM content

for specific types of student-learners

Fr

eque

ncy

Teachers participate in less than

9 hours per year of STEM professional development, which

addresses content, community/industry partnerships

and connections with post- secondary education

Teachers participate in 9-18 hours per year of STEM professional development, which addresses

content, community/industry partnerships and connections with

post-secondary education

Teachers participate in 19-29

hours per year of STEM professional development, which addresses content,

community/industry partnerships and connections with post-secondary education

Teachers participate in 30 or more hours per year of

STEM professional development, which addresses content, community/industry

partnerships and connections with post- secondary education

of P

D

Your School/

7


(A6) Curriculum: Outreach, support and focus on underserved students, especially females, minorities and economically disadvantaged students

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

C

ultu

re o

f Tru

st

A few program leaders have articulated what the culture of

trust, inquiry and creativity looks like, emphasizing the inclusion of all students and adults in this

culture

A core group of program participants

maintain a culture of trust, inquiry and creativity, emphasizing the

inclusion of all students and adults in this culture

A culture of trust, inquiry and creativity exists throughout a majority of participants in the

program, emphasizing the inclusion of all students and

adults in this culture

A strong culture of trust, inquiry and creativity exists

between and among participating students,

teachers and administrators,

emphasizing the inclusion of all students and adults

in this culture

Rec

ogni

ze U

nder

- R

epre

sent

ed

Stud

ents

No policies and practices that support equity and access for all

students

Policies and practices that support equity and access for all students

identify under-represented or struggling students; they engage as

much as 50% of those students

Policies and practices that support equity and access for

all students identify under- represented or struggling

students; they engage 51-75% of those students

Policies and practices that support equity and access for all students

identify and engage over 75% of under-

represented or struggling students

In

spire

Und

er-R

epre

sent

ed

1 in-school programs inspires under-represented and

struggling students to be excited about STEM subjects and introduces the students to

careers in STEM fields

2 or more in-school programs inspire under-represented and

struggling students to be excited about STEM subjects and introduce

the students to careers in STEM fields

2 or more in-school programs

and 1-2 out-of-school programs inspire under-represented and

struggling students to be excited about STEM subjects and introduce the students to careers in STEM fields (e.g. direct experiences with real STEM professionals through summer bridge programs and

field trips facilitated by community youth development

organizations)

Multiple in-school and out-of-school programs

inspire under-represented and struggling students to be excited about STEM subjects and introduce

the students to careers in STEM fields (e.g. direct experiences with real STEM professionals

through summer bridge programs and field trips facilitated by community

youth development organizations)

Stud

ents

Your School/

8


(B1) Community: Work-based learning experiences to increase interest and abilities in fields requiring STEM skills for each student and teacher

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

Lear

ning

Dire

ctly

C

onne

cted

to

Indu

strie

s

Program leaders are

researching and planning in- school learning opportunities

that directly connect to current work in STEM industries and

careers

1-2 in-school learning opportunities per year are directly connected to

current work in STEM industries and careers

Several in-school learning opportunities are directly

connected to current work in STEM industries and careers

In-school learning opportunities are frequently directly

connected to current work in STEM industries and

careers

Stud

ents

W

ork

in T

eam

s

Students rarely work and learn in teams to frame problems and

test solutions

Students occasionally work and

learn in teams to frame problems and test solutions, with clearly

defined individual and team expectations

Students frequently work and

learn in teams to frame problems and test solutions, with clearly defined individual

and team expectations

On a daily basis students work and learn in teams to frame problems and

test solutions, with clearly defined individual and

team expectations

Te

ache

rs In

tera

ct

with

STE

M In

dust

ries

Very few STEM teachers participate in customized,

applied learning experiences in order to increase their STEM

content knowledge and develop their pedagogy of inquiry and

problem-solving

As many as 25% of STEM teachers participate in at least 1 customized, applied learning experience in order

to increase their STEM content knowledge and develop their

pedagogy of inquiry and problem- solving (e.g. teacher fellowships, externships, team-teaching with STEM industry partners, etc.)

As much as 50% of STEM

teachers participate in at least 1 customized, applied learning experience in order to increase their STEM content knowledge and develop their pedagogy of

inquiry and problem-solving (e.g. teacher fellowships,

externships, team-teaching with STEM industry partners,

etc.)

All STEM teachers participate in customized,

applied learning opportunities in order to

increase their STEM content knowledge and

develop their pedagogy of inquiry and problem- solving (e.g. teacher

fellowships, externships, team-teaching with STEM

industry partners, etc.)

ST

EM P

rofe

ssio

nals

&

Teachers rarely interact with other STEM professionals in business, industry and higher

education

Teachers occasionally have limited interactions with other STEM

professionals in business, industry and higher education

Teachers occasionally

collaborate with other STEM professionals in business,

industry and higher education, developing together new learning environments to

empower students to think critically and address real-world

problems

Teachers frequently collaborate with other STEM professionals in business, industry and

higher education, developing together new learning environments to

empower students to think critically and address real-world

problems

Less

on P

lann

ing

Your School/

9


(B2) Community: Business and community partnerships for mentorships, internships and other opportunities extend the classroom walls

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

Trus

ted

Col

labo

ratio

n in

ST

EM N

etw

ork

Program leadership maintains some collaboration within a STEM network through the

occasional exchange of resources

Program leadership maintains collaboration within a STEM network through the exchange of resources and the sharing of best practices

and lessons learned

Unified program leadership maintains reciprocal and trusted

collaboration within a STEM network through the exchange of resources and the sharing of

best practices and lessons learned

Unified leadership maintains reciprocal and

trusted collaboration within a STEM network through transparent interactions

and decisions, open communication, exchange

of resources, sharing of best practices and lessons learned, and reinforcement of shared visions and goals

C

omm

unic

atio

n To

ols

Communication tools, such as social media platforms,

newsletters, webinars, and meetings are used infrequently

to communicate externally

Communication tools, such as social media platforms, newsletters,

webinars, and meetings are used occasionally to communicate

externally

Communication tools, such as social media platforms,

newsletters, webinars, and meetings are used frequently to

communicate externally

Communication tools, such as social media

platforms, newsletters, webinars, and meetings

are used regularly to communicate externally

St

akeh

olde

rs &

A team of stakeholders rarely assembles to discuss STEM

education problems or to create long-term funding streams

A team of stakeholders assembles roughly every 2-3 years to discuss

STEM education problems, including long-term funding; these

individuals include the district leadership team, local business

partners, and other STEM industry professionals

A team of stakeholders

assembles annually to continue building long-term funding streams; these individuals

include the district leadership team, local business partners,

and other STEM industry professionals

A team of stakeholders assembles semi-annually

to maintain long-term funding streams; these individuals include the

district leadership team, local business partners,

and other STEM industry professionals

Fund

ing

Your School/

10


(B3) Community: “STEM Attributes” are evident in leadership’s, teachers’ and counselors’ plans of work and are communicated to community-based organizations

Early (Starting)

Developing

Advanced (Prepared)

Target

Description of Your School/ Program

Action Steps

ST

EM P

rogr

am P

lan

The leadership team creates a basic STEM program plan in which 1-3 STEM Attributes

are evident

The leadership team creates a detailed STEM program plan grounded in research and in

which 3-7 STEM Attributes are evident

The leadership team, which

includes at least one student, creates a detailed STEM program plan grounded in research, aligned with district strategic plans focused on student achievement in STEM

and demonstrates evidence of 7-10 STEM Attributes

The leadership team, which includes multiple students, leads stakeholders in a collaborative

decision-making process to create a STEM program plan grounded in

research, aligned with district strategic plans and demonstrating

evidence of 10 or more STEM Attributes

Com

mun

icat

e ST

EM

Prog

ram

Pla

n

The leadership team’s

minimal communication of a STEM program plan and

other activities with teachers and key stakeholders

maintains limited participation and buy-in

The leadership team’s

occasional communication of a STEM program plan and other activities with teachers and key

stakeholders develops some participation and buy-in

The leadership team’s frequent communication of the STEM

program plan and other activities with teachers and key

stakeholders secures increased participation and buy-in and bolsters sustainability of the

initiative

The leadership team’s constant communication of the STEM

program plan and other activities with teachers and key

stakeholders secures maximum participation and buy-in and bolsters sustainability of the

initiative

Pr

ogra

m D

ata Student data on STEM

performance is available annually to administrators and teachers and is rarely used to inform instructional and programmatic decision-

making

Student data on STEM

performance is available annually to administrators and teachers and is used yearly to

inform instructional and programmatic decision-making

Student data on STEM performance is available quarterly

to administrators and teachers and is used to inform instructional

and programmatic decision- making and to support continuous improvement throughout the year

On-demand, up-to-date student data on STEM performance is available to administrators and teachers and is used to inform instructional and programmatic

decision-making

R

esou

rce

Limited discretionary funds

are allocated for implementation of STEM

strategies

Discretionary funds and other resources are allocated to advance implementation of

some STEM strategies outlined in the program plan

Discretionary funds and other resources are allocated to

advance implementation of most of the STEM strategies outlined in

the program plan

Discretionary funds and other resources are allocated to

advance implementation of all the STEM strategies outlined in the

program plan

Allo

catio

n

11


(C1) Connections: Alignment with students’ career pathways to post-secondary programs

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

Vert

ical

Pl

anni

ng

Teachers do not vertically plan

within and across (between elementary, middle and high)

schools

Teachers vertically plan within and

across (between elementary, middle and high) schools every 2-3 years

Teachers vertically plan within

and across (between elementary, middle and high)

schools annually

Teachers vertically plan across grade levels and

between schools (elementary, middle, and high) schools biannually

C

ouns

elor

& S

tude

nt

Rel

atio

nshi

ps

Career counselors and students

have brief and limited interactions

Career counselors and students communicate virtually or face-to- face at least quarterly about the

students’ future plans and how they connect to their academic activities

Career counselors and students have developed one- on-one relationships, meeting face-to-face at least quarterly to discuss, plan and track the connections and alignment of students’ pathways to careers and post-secondary education

Career counselors and students have developed one-on-one relationships and use both face-to-face

and virtual communication

frequently, including at least quarterly face-to- face meetings, to plan, discuss and track the

connections and alignment of students’

pathways to careers and post-secondary education

Cou

nsel

ors

&

Teac

hers

C

omm

unic

ate

Career counselors and teachers

do not meet to discuss the alignment of students’ pathways to post-secondary careers and

education

Career counselors and teachers meet annually to discuss the

alignment of students’ pathways to post-secondary careers and

education

Career counselors and

teachers meet semi-annually to discuss the alignment of

students’ pathways to post- secondary careers and

education

Career counselors and teachers meet quarterly to discuss the alignment of students’ pathways to post-secondary careers

and education

In

form

atio

n

Information about post-

secondary STEM programs and STEM career topics is rarely accessed and shared with

counselors

Information about post-secondary STEM programs and STEM career topics is occasionally accessed and

shared with counselors

Information about post-

secondary STEM programs and STEM career topics is frequently accessed and

shared with both teachers and counselors

Information about post- secondary STEM

programs and STEM career topics is regularly

accessed and shared with both teachers and

counselors

Shar

ing

Your School/

12


(C2) Connections: Availability of credit completion with post-secondary institutions, including community colleges, colleges and/or universities

Early (Starting)

Developing

Advanced (Prepared)


Program

Action Steps

C

redi

t Com

plet

ion

Ava

ilabi

lity

STEM program/school includes no formal course offerings for which credit completion would be available, but occasionally supports students to enroll in

courses offered by post- secondary institutions

STEM program/school includes a few course offerings for which credit

completion would be available based upon a limited agreement and

relationship with a post-secondary institution

STEM program/school includes

multiple course offerings for which credit completion is

available based upon developing agreements and relationships with 1-2 post-

secondary institutions; offerings were thoughtfully selected based upon the school’s

resource needs and the student population’s needs

STEM program/school includes a wide variety of course offerings for which

credit completion is available based upon

strong agreements and relationships with 2-3 post-

secondary institutions; offerings were thoughtfully selected based upon the school’s resource needs

and the student population’s needs

St

uden

t En

rollm

ent

Less than 10% of target students are enrolled in any

credit completion opportunities

10-50% of target students are enrolled in any credit completion

opportunities

51-75% of target students are enrolled in credit completion

opportunities

Over 76% of target students are enrolled in

credit completion opportunities

C

ompr

ehen

sive

Some career counselors

understand the credit completion and post-secondary enrollment process and rarely

advise students on this opportunity

Career counselors understand the credit completion and post-

secondary enrollment process and occasionally advise students on this

opportunity

Both career counselors and

STEM teachers understand the credit completion and post-

secondary enrollment process and occasionally advise

students on this opportunity

All career counselors and teachers thoroughly

understand the credit completion and post- secondary enrollment process and regularly

advise students on this opportunity

Adv

isin

g

Your School/

13


References

Friday Institute for Educational Innovation (2008). North Carolina Learning Technology Initiative (NCLTI) framework for planning. Raleigh, NC: Author. Available from http://www.fi.ncsu.edu/assets/research_papers/nc-11-learning-technology-initiative-planning/nclti-planning-framework-.doc.

Ready, Set Go (2011). Statewide STEM Strategy. Raleigh, NC: Author.

Rowley, J. (2010). STEM Education Quality Rubrics. University of Dayton, Ohio

Texas High School Project T-STEM Initiative (2010). Texas Science Technology Engineering and Mathematics Academies Design Blueprint, Rubric, and Glossary. Available from: http://ntstem.tamu.edu/Academies/blueprint.pdf

Wake County Public Schools STEM Schools Collaborative Network (2011). STEM School Readiness Self-Assessment V 1.0. Raleigh, NC: Author.

These evaluation instruments were identified, modified, or developed through support provided by the Friday Institute. The Friday Institute grants you permission to use these instruments for educational, non-commercial purposes only. You may use an instrument as is, or modify it to suit your needs, but in either case you must credit its original source. By using this instrument you agree to allow the Friday Institute to use the data collected for additional validity and reliability analysis. The Friday Institute will take appropriate measures to maintain the confidentiality of all data. For information about additional permissions, please contact Dr. Jeni Corn, Director of Evaluation Programs at The Friday Institute, at [email protected].

Date post:	25-Aug-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Tennessee STEM Innovation Network: Final Evaluation Report · Tennessee Consortium on Research,...

Documents