Date post: | 12-Jan-2016 |
Category: |
Documents |
Upload: | august-hensley |
View: | 222 times |
Download: | 1 times |
NOVA Evaluation ReportNOVA Evaluation Report1996-20021996-2002
NOVA Evaluation Report1996-2003
Higher Education Faces Higher Education Faces Significant ChallengesSignificant Challenges
Nation at Risk (1983)Benchmarks for Science Literacy, Project
2061 (American Assoc. for the Advancement of Science 1993)
Shaping the Future (NSF 1996) NASA Strategic Plan, Implementation
Plan for Education (1999 – 2003)No Child Left Behind (2002)
Higher Education Faces Higher Education Faces Significant ChallengesSignificant Challenges
Higher education faculty are attempting to improve the effectiveness of undergraduate science and math courses.
In the past the process has been slow and the results limited.
NOVA Professional Development NOVA Professional Development for Higher Education STEM for Higher Education STEM
FacultyFacultyNASA Opportunities for Visionary Academics
(NOVA)A professional development program assisting faculty in developing innovative STEM curricula.
NOVA ObjectivesNOVA Objectives
Disseminate NASA’s pre-service education model.
Promote development of the model.Sustain the change process.Increase collaboration.Stimulate and conduct research.
Phases of the NOVA ProgramPhases of the NOVA Program
Phase I—Professional Development Workshop and Proposal Development
Phase II—Research and disseminationPhase III—NASA Field Center Program,
online course enhancementsLeadership Development ConferenceResearch and evaluation, including on-
site assessment
NOVA EvaluationNOVA Evaluation
Evaluation of NOVA 1996-2003 included extensive and diverse data.
The NOVA Evaluation Report is organized around ten central points or question.
NOVA Evaluation Instrumentation Timeline
1
1.NOVA
Documents
2
2.Pre NOVA Workshop
3
3.During NOVA
Workshop
4
4.Proposal Planning
5
5.Implementation Phase: During the year after submission
6
6.Sustaining
Phase:1 year after
proposal submission
7
7.Institutionalization Phase: 2 Years after
proposal submission
Workshop Agendas
Brochures
Workshop Books
NOVA Mission and Work Plans
NOVA Reports
NOVA Products
Background Survey
Pre-Status Survey
Pre-Effacacy Survey
EDCATS
Pre-barriers Survey
Action Plan
NOVA Proposal
Interim-Status Survey
Site Visit:Faculty Surveys
Course MaterialsFaculty Interviews
Post-EfficacyClass Observations
Administrator InterviewsSummary results
Site Visit:Student Surveys
Student focus groups
Perceptions of teachingAttitudes
toward science Teachingefficacy
Nature of Science and/or Mathematics
Final Participant
Report
Post-Post Status Survey
Post-Post Efficacy
Student Surveys
includes barriers,
internal and external impact
Dissemination Artifacts
Long-Term Status Survey
Student and Graduate Surveys
Dissemination Artifacts
NOVA Proposal Review Ranking
1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students
240 institutions attended workshops from February 1996 to February 2003.
NOVA Workshop Institutional Participants
0
10
20
30
40
50
Workshop 43 42 45 27 47 17 19
1996 1997 1998 1999 2000 2001 2002
1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students
240 institutions attended workshops from February 1996 to February 2003.
In 44 states, Washington D.C., Puerto Rico, and the U.S. Virgin Islands.
1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students
240 institutions attended workshops from February 1996 to February 2003.
In 44 states, Washington D.C., Puerto Rico, and the U.S. Virgin Islands.
308 multidisciplinary NOVA faculty funded from 88 institutions
Fig. 4. NOVA Proposals Submitted and Funded
Nonfunded42%(64)
Funded58%(88)
Funded
Nonfunded
1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students
240 institutions attended workshops from February 1996 to February 2003.
In 44 states, Washington D.C., Puerto Rico, and the U.S. Virgin Islands.
308 multidisciplinary NOVA faculty funded from 88 institutions.
154 courses; over 42,000 students, increasing by 12,000 annually.
Undergraduate Students Enrolled In Or Graduated From NOVA
STEM Courses
0
10000
20000
30000
40000
50000
Students 525 1600 4550 14400 22000 31500 42500
1996 1997 1998 1999 2000 2001 2002
1. Characteristics of NOVA Institutions, 1. Characteristics of NOVA Institutions, Faculty, and StudentsFaculty, and Students
240 institutions attended workshops from February 1996 to February 2003.
In 44 states, Washington D.C., Puerto Rico, and the U.S. Virgin Islands.
308 multidisciplinary NOVA faculty funded from 88 institutions.
154 courses; over 42,000 students, increasing by 12,000 annually.
Institutions range from Doctoral/Research I to Associate of Arts Community Colleges.
NOVA Institution Classification by Percent - Funded
R-I15%
R-II11%
MA-I40%
MA-II6%
B-I9%
B-II17%
AA2%
2. Dissemination of the NOVA Model2. Dissemination of the NOVA Model
23 Phase I Workshops.Over 150 journal articles, reports, and
conference presentations.Research-supported best practices
shared within and between institutions.Over 100 NOVA-related national,
regional, and local workshops.
A Sample of National ConferencesA Sample of National Conferences
AAPT AERA AIAA ASEE AETS ATE ACS AAAS NCTM NARST
SCST QEM International Conference on
Mathematics/Science Education & Technology
12th International Conference on College Teaching and Learning
Project Kaleidoscope Conference
2. Dissemination of the NOVA Model2. Dissemination of the NOVA Model
Non-NOVA faculty in over half of NOVA institutions implemented NOVA-type change.
Clone and NOVA-like spin-offs.Administrators reported faculty
incorporating inquiry-based activities into non-NOVA courses.
Springboard to other grant writing.A third of non-funded Phase I teams made
changes based on NOVA model.
2. Dissemination of the NOVA Model2. Dissemination of the NOVA Model
Evolution over 8 years from innovative institution-based courses to a national dissemination and collaboration network.
Personnel, data, and materials from NASA Enterprises incorporated in a wide population of higher education institutions and students.
3. Congruency of the NOVA Model with 3. Congruency of the NOVA Model with Major Reports on Teacher PreparationMajor Reports on Teacher Preparation
Eight major elements aimed at changing how science and math are taught in K-12 classrooms by influencing “how” content is taught at the college level.
3. Congruency of the NOVA Model with 3. Congruency of the NOVA Model with Major Reports on Teacher PreparationMajor Reports on Teacher Preparation
1. Continuous professional development. 2. Sustained collaboration among
interdisciplinary teams of faculty and administrators.
3. Courses based on national standards for science, math, and technology.
4. Courses utilize content based on NASA Strategic Enterprises.
3. Congruency of the NOVA Model with 3. Congruency of the NOVA Model with Major Reports on Teacher PreparationMajor Reports on Teacher Preparation
5. Program improvement at NOVA institutions through ongoing research.
6. Instructional strategies that are inquiry based and center on student interaction.
7. Extensive use of technology.
8. Collaborations among institutions in the NOVA network.
4. Participant Reactions to the NOVA 4. Participant Reactions to the NOVA Professional Development ModelProfessional Development Model
PositiveUseful in developing innovative
pedagogy.Helped develop skills in technology and
curriculum innovation.Increased confidence and expectations.
Reactions to the NOVA program:
4. Participant Reactions to the NOVA 4. Participant Reactions to the NOVA Professional Development ModelProfessional Development Model
Introduce innovative teaching and/or STEM courses.
State education requirements, content standards.
Help validate already innovative faculty.Recruit other faculty into a network of
innovation.
Reasons for participating in NOVA:
4. Participant Reactions to the NOVA 4. Participant Reactions to the NOVA Professional Development ModelProfessional Development Model
Team approach.Commitment to the project.Belief in the model.Use of NASA resources.Monetary support.
Important factors facilitating change:
5. Impact of the NOVA Model on the 5. Impact of the NOVA Model on the Collaborative Work and Organizational Collaborative Work and Organizational
Climate of STEM FacultiesClimate of STEM Faculties
Increase in type and expertise of research-supported teaching practices
Other funding or grant opportunitiesNew department programsProfessional disseminationBreadth of expertise, experienceFaculty initiative, resolution of problemsCampus climate focused on student learning
6. Effect of the NOVA Model on Higher 6. Effect of the NOVA Model on Higher Education CoursesEducation Courses
Chemistry, physics, physical science
16%
Earth and space science
21%
Methods5%
Engineering4%
Biology, life science, life in
space, environmental
science24%
Nature of science, inquiry
science, integrated
science10%
Mathematics, Science, and Technology
20%
6. Effect of the NOVA Model on Higher 6. Effect of the NOVA Model on Higher Education CoursesEducation Courses
Majority of courses at the freshman or sophomore entry level.
NOVA Phase I Course Levels by Percent
200-Level18%
400-Level7%300-Level
13%
500-Level1%
100-Level61%
7. Impact of the NASA Mission, Data and 7. Impact of the NASA Mission, Data and Information, and Fundamental Questions on Information, and Fundamental Questions on
the NOVA Coursesthe NOVA Courses
Multiple connections to NASA Strategic Enterprises.
Used by NOVA students and carried into their K-12 classrooms.
Shared between colleagues.Disseminated on institutional web sites.Presented at national conferences.
Percent of NOVA Courses Making Connections with each NASA
Enterprise
ESE37%
HEDSE19%
SSE25% BPRE
1%
ATE11%
05101520253035
Number of Institutions
5 4 3 2 1
Number of NASA Strategic Enterprises
Multiple NASA Entrprise Connections
7. Impact of the NASA Mission, Data and 7. Impact of the NASA Mission, Data and Information, and Fundamental Questions on Information, and Fundamental Questions on
the NOVA Coursesthe NOVA Courses
Multiple connections to NASA Strategic Enterprises.
Used by NOVA students and carried into their K-12 classrooms.
Shared between colleagues.Disseminated on institutional web sites.Presented at national conferences.
8. Aspects of NOVA Professional 8. Aspects of NOVA Professional Development Model that are Effective in Development Model that are Effective in
Creating and Sustaining ChangeCreating and Sustaining Change
Continuous faculty interaction.Administrative support.Sense of common purpose and trust.NOVA structural supports made possible
and promoted change processes.Faculty action (practitioner) researchDevelopment and sustaining of teams - a
key factor in accomplishing goals.
9. Impact of the NOVA Model on Classrooms 9. Impact of the NOVA Model on Classrooms and Studentsand Students
Faculty action research found significant improvement in student content knowledge, attitudes, and inquiry skills.
Students had a higher efficacy to teach K-12 science.
9. Impact of the NOVA Model on Classrooms 9. Impact of the NOVA Model on Classrooms and Studentsand Students
Decrease in traditional classroom lecture and memorization approaches
Greater use of an inquiry approach in teaching
Emphasis on interactive, student-centered methods
Use of NOVA courses as models for developing additional courses and for meeting state higher education guidelines
9. Impact of the NOVA Model on Classrooms 9. Impact of the NOVA Model on Classrooms and Studentsand Students
Undergraduate education majors’ science teaching efficacy, content mastery, and teacher performance positively affected.
State standardized achievement test scores in science higher for students of NOVA courses.
Studies on impact of NOVA courses on students:
Students experienced science and math as a process rather than being told about content.
Integration of lecture and lab promoted experiential, inquiry learning.
Increase in students’ STEM content knowledge.
Female and minority interest in NOVA courses increased.
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students
Higher end-of-course growth and understanding than students in similar “non-NOVA” courses.
Two national studies on student understanding of the nature of science and mathematics:
VNOS-I
05
101520253035
ExperimentalStudents
ComparisonStudents
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students
Two national studies on student understanding of the nature of science and mathematics:
Higher end-of-course growth and understanding than students in similar “non-NOVA” courses.
Increased interest and lowered anxiety about this content in their professional and personal lives.
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students
Results related to students’ ability to succeed in science and math-related tasks and careers.
Two national studies on student understanding of the nature of science and mathematics:
Higher end-of-course growth and understanding than students in similar “non-NOVA” courses.
Increased interest and lowered anxiety about this content in their professional and personal lives.
NOVA’s impact on higher education—
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students
NOVA’s impact on higher education—
Important to professionals and our educated citizenry in a democratic society.
10. Impact of the NOVA Model on Science and 10. Impact of the NOVA Model on Science and Mathematics Literacy of Higher Education StudentsMathematics Literacy of Higher Education Students