TGYS Orientation DayGet ready for evaluation!

ROADMAP

• TGYS Goals• Reasons for Evaluation• Evaluation Plans

• Design• Measurement• Data Collection• Timelines

• Annual Results

TGYS GOALS

REDUCE YOUTH CRIME AND VIOLENCE

PREVENT CHILD ABUSE AND NEGLECT

• Improve school performance

• Increase adult bonding• Increase self-esteem• Increase life skills• Decrease bullying• Decrease ATOD use• Decrease delinquency• Decrease recidivism• Improve quality of early

care and educational programs

• Achieve developmental milestones

• Increase positive parenting skills

TGYS Programs

and Activities

REASONS FOR EVALUATION

I. Your Participants

II. Reporting

III.

Funding

REASON I

What’s working well?

What isn’t?

Program refinement

REASON I

GPA

School Engagemen

t

REASON II Package and streamline reporting

Mean NStd.

Deviation

Desired

Direction of

Change?

Sig.Effect

Size

Perceived relevance of school work - PRE 3.42 26 0.98

Yes 0.26 0.21

Perceived relevance of school work - POST 3.63 26 0.86

Enjoyment of school - PRE 3.26 25 0.47Yes 0.20 0.29

Enjoyment of school - POST 3.40 25 0.53

Effort in school - PRE 3.53 25 0.61Yes 0.17 0.24

Effort in school - POST 3.68 25 0.63

Educational aspirations - PRE 2.92 25 1.34Yes 0.04* 0.42

Educational aspirations - POST 3.48 25 1.22

School Bonding Total - PRE 3.36 25 0.53Yes 0.04* 0.39

School Bonding Total - POST 3.57 25 0.58

REASON III Competitive edge for funding

INDIVIDUAL EVALUATION PLANS

Step 1: Choose an evaluation

design

Step 2: Decide on the

best measurement

tool

Step 3: Design a data collection and management protocol

EVALUATION DESIGN

Pre-Test

Your Program

Post-Test

EVALUATION DESIGN

Pre-Test Your Program Post-Test

Pre-Test Post-Test

Group One: Program Participants

Group Two: Waiting-list Participants

MEASUREMENT

Anticipated

Outcomes

Validated,

Reliable, &

Practical Tools

MEASUREMENT

Outcomes: Corresponding ScaleLife Skills: Resilience

Adult Bonding: Perceived Social Support

School Performance: Direct School Records

School Performance: School Engagement

Self-Efficacy: Self Efficacy

Bullying: Bullying, Fighting, & Victimization

Alcohol, Tobacco, and Other Drug Use: Use & Attitudes

Delinquency: Attitudes toward Delinquency

Recidivism: Colorado Criminal Contacts/Re-offenses

Quality of Early Care and Education: Teaching Strategies GOLD

Developmental Milestones: Social Competence

Parenting Skills: Parenting Practices Interview

DATA COLLECTION

It’s “FUN”damental!

Electronic Surveys Paper Surveys Data Management

DATA COLLECTIONELECTRONIC SURVEYS

Obtain a Link

Complete Submit

Example survey link:Alternatives 4 Youth_COSchBond.

DATA COLLECTIONELECTRONIC SURVEYS

DATA COLLECTIONPAPER SURVEYS

Step 1: Ordering

Step 2: Preparation

Step 3: Submission

Double-check your order

Note the return address

Note the ID format

DATA COLLECTIONPAPER SURVEYS

Step 1: Ordering

Step 2: Preparation

Step 3: Submission

• Survey condition

• Administrative Section

• Instructions

• Confidentiality

DATA COLLECTIONPAPER SURVEYS

Useful Data

Condition • Using originals

• Stapling or hole punching

• Using pencils

DATA COLLECTIONPAPER SURVEYS

• Fill in pre, post

or follow-up test

• Fill out Org ID and Local ID from top to bottom

• Date

DATA COLLECTION

ALL SURVEYS

Useful Data

Condition

Admin.Section

DATA COLLECTION

ALL SURVEYS

Useful Data

Condition

Admin.Section

Instructions

• Read thoroughly

• Pick best answer

• Answering questions

• Birthdate

DATA COLLECTION

ALL SURVEYS

Useful Data

Condition

Admin.Section

Instructions

Confidentiality

• Testing space

• Survey handling

Step 1: Ordering

Step 2: Preparation

Step 3: Submission

Julie MaertensColorado State

UniversityCampus Delivery 1876Fort Collins, CO 80523

DATA COLLECTIONPAPER SURVEYS

Assign unique IDs

Keep track of each ID

Keep your log confidential

DATA MANAGEMENT

Pre-test Post-test

1001 1001

1003 1005

2002 2002

2003 2003

2014 2112 Sample Size = 3

DATA MANAGEMENT

DATA MANAGEMENT

Deadline Task Person in Charge

August 16 Order pre-test surveys (expect 2-week turn-a-round)

Mark V.

September 3 Organize surveys: count, fill out admin section

Susan C.

September 22 Administer surveys to youth

Laura F.

September 29 Check surveys: bubbles filled-in, admin complete

Susan C.

December 1 Mail pre-test surveys to CSU (2 weeks prior to Dec 15 deadline)

Mark V.

DATA MANAGEMENT

DATA MANAGEMENT

Pretest DeadlineDecember 15, 2011

Posttest DeadlineJune 30, 2012

INDIVIDUAL EVALUATION PLANS

Evaluation Design

Measurement Tool

Data Collection Protocol & Timeline

RESULTS

RESULTS

RESULTS

Mean NStd.

Deviation

Desired

Direction of

Change?

Sig.Effect

Size

Perceived relevance of school work - PRE 3.42 26 0.98

Yes 0.26 0.21

Perceived relevance of school work - POST 3.63 26 0.86

Enjoyment of school - PRE 3.26 25 0.47Yes 0.20 0.29

Enjoyment of school - POST 3.40 25 0.53

Effort in school - PRE 3.53 25 0.61Yes 0.17 0.24

Effort in school - POST 3.68 25 0.63

Educational aspirations - PRE 2.92 25 1.34Yes 0.04* 0.42

Educational aspirations - POST 3.48 25 1.22

School Bonding Total - PRE 3.36 25 0.53Yes 0.04* 0.39

School Bonding Total - POST 3.57 25 0.58

THANK YOU!

Also:jmtgys@lamar.colostate.edu

CSU Evaluation Office(970)491-3830 main(877)724-4111 fax

Questions?