+

STARS Evaluation Assistant Webinar 1September 19, 2014

Evaluation Projects

+Part 1: What Evaluation Assistants Do

+What is Evaluation?

+How do we tell our story?

+EA Role

Team Lead for Local Evaluation:

1. Activity Reports (1 per activity)

Story Teller of Your STARS Computing Corps

2. Narrative Reports (April 15)

Applied Researcher

3. Evaluation Project and poster at Celebration 2015

Submit Activity Reports Online

ongoing

Participate in meetings and webinars

monthly

EA Toolkit Support Resources

What you do How you do it

Demonstration

+Recap of EA Deliverables

Activity Reports

Consent Forms & Registration

Fall Synopsis of Activity Reports by Thanksgiving

EA Project Milestones: Logic Model (Oct 3) Timeline (Oct 24)

Activity Reports

Narrative Report (April 15)

Evaluation Poster (May 1)

FALL 2014 Spring 2015

+Part 2:Understanding EvaluationEvaluation Projects

+ Evaluation is….

Evaluation is the systematic assessment of the worth or merit of some object (Trochim)

Applied social science research

A way of viewing the project, its participants, it effect

Accounting for CONTEXT and MEANING

Determining the VALUE of something

Measurements are the currency

+ Understanding Evaluation

1.Systematically define what data will be collected, how, when & by whom

2.Process the data to determine what it means within specified contexts

3.Tell constituents about findings in appealing & appropriate ways

4.Disseminate broader findings to share within disciplines

+Evaluative Thinking

Reflective practice of systematically using data to inform decisions

Ongoing process

Approach to making decisions, what actions to take, to become/remain effective at the organizational (or individual) goals

It is LEARNING and APPLYING learning

+ Evaluation and Academic Research

Evaluation Academic Research

Applies research methods to answer questions about a program’s efficacy & value

Considers context (stakeholders, climate, etc.)

Makes a judgment call on WORTH, and defines value structure (i.e. value to whom?)

Applies research methods to answer a research question

Knowledge gained in scientific discipline

Does not apply value structure; values are seen as bias

+ In summary, evaluation…

Uses research methods

Applies methods to social, political and economic realities

Helps to interpret academic research within larger socio-economic/geo-political realities

Answers the “so what?” of observed outcomes

Research = Knowledge Discovery

Evaluation = Extracting Meaning from Knowledge

Context Inputs Outputs Outcomes/ImpactGoals

Computing enrollment & graduation is in a state of decline nationally

K12 students lack appropriate background to pursue computing

Computing pipeline lacks diversity

NSF Funding for 50 Alliance members

K12 Partnerships

STARS Leadership Corp

Faculty Associates

Community & Industry Partners

K16 Outreach

Celebration

Demonstration Projects

Social media

Online Repository & Toolkit

Dissemination

Evaluation

K16: Develop, test, deploy academic interventions; Pedagogical reform

Increased post-secondary enrollment & graduation

Increased scholarship & research

Increased dissemination of best practices & national adoption of Alliance Community of Practice

Increased BPC partnerships & advocacy

K16 Recruitment

Graduate School Bridging

Post-secondary Student Retention

Scholarship Advancement

Alliance Sustainability

Dissemination

Scaling

Formative assessment: SLC and K16 Outreach, Partnerships, Tools, Celebration

Community of Practice & National Resource: Lessons Learned, Best Practices, Social Network, Resources & Evaluation Repository/Toolkit, Research, Summative Evaluation

STARS Alliance Logic Model

Context Inputs Outcomes/ImpactGoals

CHALLENGES

PROBLEM TO SOLVE

ACTIVITIES

RESOURCES

Short & Long Term Accomplishments

E.G.

Tools/relationships built, developed

Numbers served

Attitudes

Actions

What do you hope to accomplish and how will you know?

Logic Model Framework for Computing Corps

Due Oct. 3

Overview of EA Projects Evaluation project that the EA manages

Can do alone or manage a team of Corps students at your school

A distinctive measurement of the impact of a Corp project at your school Example- measure impact of high school outreach program on high

school student interest in computing Example- assess mentoring program at your school

+EA Projects

Are NOT a description of what the Corp is doing or has done

Are a summation of data collected, observed, documented about the impact of what the Corp has done at a school

Components of EA Projects

Determine which Corp program to assess at your school

Align project to the program goals E.g. if your Corp conducts high school outreach, what are the goals? Determine which goal(s) to measure and what data to collect Complete a project description and timeline (see

www.starsalliance.org EA Toolkit for samples)

Assess the program and report findings via Celebration Poster Session

Get Support Corps Project Teams

Share ideas and tools within your STARS Computing Corp Ask prior EAs at your school for input, ideas, suggestions

Visit the EA Toolkit Sample IRBs, sample outreach surveys and more

Discuss with EA community Online or via monthly meetings

+

Q&AEvaluation Projects