Educational Practices and Outcomes Assessment of SAGE 2YC

Connect to the webinar: http://serc.adobeconnect.com/sage2yc/

Please use phone or skype for audio

Educational Practices and Outcomes Assessment of SAGE 2YC

November, 2016Heather Macdonald, College of William & Mary

Debra Bragg and Heather McCambly, Bragg & AssociatesPamela Eddy, College of William & Mary

Ellen Iverson, SERC

Webinar Goals and Overview

• Review SAGE 2YC project, goals, and theory of change• Introduce plan for collecting data that will allow us to measure

project outcomes – Educational practices inventory– Outcomes assessment, common outcomes measures

• Q & A about the plan

SAGE 2YC Project TeamEvaluation & Research Project Leaders SERC Staff

Debra Bragg Pamela Eddy

Heather McCambly Ellen Iverson

Heather Macdonald Eric Baer

Jan Hodder Norlene Emerson

Carol Ormand John McDaris

SAGE 2YC Teams and Their Work

• SAGE 2YC Teams: 24 faculty Change Agents in 10 regions+ Administrators+ Institutional Research

• Action plans developed at June 2016 workshop considering three project strands– Individual – Geoscience program/department– Regional workshop

Project strands• Supporting academic success

of all students• Broadening participation• Facilitating students’

professional pathways

SAGE 2YC Goals

Faculty learn from research

Faculty make changes in their

own practice

Faculty share what they are learning with the geoscience education

community

• Build a national network of 2YC geoscience faculty and administrators led by 2YC faculty leaders, called Change Agents, who catalyze change at multiple levels

• Implement high-impact, educational practices that: •Support the academic success of all students•Broaden participation•Facilitate professional pathways into the geosciences

• Investigate professional development models that support the cycle of innovation to improve geoscience education

Cycle of Innovation

Theory of Change Logic Model

Outcomes of Educational Practice Changes to 2YC Geoscience Education in:•Access•Participation•Course Success•Pathway Progression

+

EducationalPractices

SAGE 2YC Leadership Team

Support

+

Change Agents

Change Agent Action Plans

SAGE 2YC Professional

Development

Cycle of Innovation

Cycle of Innovation

Educational Practices Inventory

• Research-based practices related to the project goals– Academic success of all students– Broadening participation in the geosciences– Facilitating students’ professional pathways

• Two inventories of faculty practices:– What individual faculty do – What faculty do collectively, in the context of their department or

program

Outcomes Assessment Goals

• Assess implementation of educational practices• Apply outcomes assessment to your geoscience program(s)

and course(s)• Link outcomes assessment to the three strands of SAGE 2YC:–Academic success–Broadening participation–Pathways progression – transfer and career

Participation

Geoscience Course EnrollmentBroadening Participation in Geoscience Courses

Course Success Outcomes

Success in Geoscience CoursesGeoscience Course Success by Student Sub-group

Pathways Progression Outcomes

Enrollment in Multiple Geoscience CoursesMajor in Geoscience Program

Snapshot of Outcomes Assessment Measures

Approach

• Use Outcomes Assessment Worksheets, with instructions provided by the SAGE 2YC evaluators, Debra Bragg & Heather McCambly

• Partner with institutional research (IR)

• Engage college administration to help secure data, if needed

• Work collaboratively with Debra Bragg and the leadership team (led by Heather Macdonald) on your assessment plan

• Start the outcomes assessment in Spring 2017 and continue to the end of the grant

Outcome Measure: Geoscience Course Enrollment

Question: What is the enrollment in geoscience courses, including courses receiving an “educational change”, and how does course enrollment change from Spring 2017 to the end of the grant?

Measure: Term-by-term enrollment at census date in geoscience courses, including geoscience courses identified as having received an “educational change”, from Spring 2017 to the end of the grant.

Method 1. List geoscience courses

offered by your college2. For each term, indicate

whether course is offered and whether the course has received an “educational change”

3. For each geoscience course offered, count the total number of students enrolled in each course at census date.

Outcome Measure: Broadening Participation in Geoscience Courses

Question: What is the enrollment of underrepresented student groups in geoscience courses, and does geoscience course enrollment by underrepresented student groups change from Winter/Spring 2017 to the end of the grant?

Measure: Term-by-term geoscience course enrollment by student sub-group (e.g., gender, race, ethnicity).

MethodFor each course and by term offered, count students affiliated within sub-groups by gender, race, ethnicity, and other demographic variables (according to IPEDS definitions) at census date.

Outcomes Measure: Success in Geoscience Courses

Question: How many students and what percentage of students in geoscience courses succeed, and does course success change from Winter/Spring 2017 to the end of the grant?

Measure: Term-by-term geoscience course completion with a grade of “C” or higher.

Method1. For each course and by

term, count the number of students who completed and passed the course with a grade of “C” or higher.

2. Compute percentage of students deemed successful (#1) by the total enrollment in same geoscience course.

Outcome Measure: Geoscience Course Success by Student Sub-Group

Question: By sub-group, how many students succeed in geoscience courses, and does the percentage of course enrollee success change from Winter/Spring 2017 to the end of the grant?

Measure: Term-by-term geoscience course completion with a grade of “C” or higher by student sub-group.

Method1. For course and by term,

count the number of students within sub-groups who completed and passed the geoscience course with a grade of “C” or higher.

2. Divide #1 by the total count of students within sub-groups who enrolled in the same geoscience course.

Outcomes Measure: Pathways Progression

Question: How many students take more than one geoscience course, how many students major in geoscience, and do these numbers change from Winter/Spring 2017 to the end of the grant?

Method: • Student enrollment in more than

one geoscience course. • Students with declared major or

concentration in geoscience.

1. By course and by term, count the number of students who enroll at census date who have enrolled in more than one geoscience course.

2. By course and by term, count the number of students who enroll at census data who are declared geoscience majors.

Method

Outcomes Measure: Pathways Progression by Student Sub-Group

Question: How many students by sub-group who take more than one geoscience course and major in geoscience? How do these numbers change from Winter/Spring 2017 to the end of the grant?

Method: • Student sub-group enrollment in

more than one geoscience course. • Student sub-group with declared

major in geoscience.

1. By course and by term, count the number of students by sub-group who enroll at census date who have enrolled in more than one geoscience course.

2. By course and by term, count the number of students by sub-group who enroll at census data who are declared geoscience majors.

Method

Outcomes Measures• Participation and Access

– Term-by-term enrollment at census date in geoscience courses – Term-by-term geoscience course enrollment by student sub-group

• Course Success and Access– Term-by-term geoscience course completion with a grade of “C” or higher– Term-by-term geoscience course completion with a grade of “C” or higher by student sub-

group

• Pathways Progression and Access– Student enrollment in more than one geoscience course– Student sub-group enrollment in more than one geoscience course– Students with declared major or concentration in geoscience– Student sub-group with declared major in geoscience

Timeline for Outcomes Assessment

• November 2016 – this webinar• Mid-December 2016 – Outcomes Assessment Guide and Worksheets shared

with Change Agents• January 2017 – telephone conference calls to review outcomes assessment

plan with Change Agents• Educational practices inventories

– Individual: December 2016 - January 2017– Department: Early winter, 2017

• Winter/Spring 2017 – Change Agents launch outcomes assessment process• June 2017 – Change Agent Annual Workshop in Tacoma, WA

Outcomes Assessment ResourcesPrimary Sources: – Change Agent Action Plans– SAGE 2YC Proposal and Logic Model

Other Sources:– Gates Foundation/Institute for Higher Education & Policy’s

scan of 20+ improvement initiatives– Alliance for Quality Career Pathways Framework– The Voluntary Framework for Accountability– Pathways to Results Equity & Outcomes Templates

Questions?

Contacts:• Outcomes Assessment: Debra Bragg, Bragg.Associates.Inc@gmail.com• Inventories: Pamela Eddy, peddy@wm.edu • Overall project: Heather Macdonald, rhmacd@wm.edu

Thank you!

Webinar evaluation form: http://serc.carleton.edu/165105