Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | jayson-felix-short |
View: | 215 times |
Download: | 0 times |
Improving Program Quality in Central Texas
Agenda Overview Accomplishments Improvements Future Opportunities Break
Assessors
Methods Trainers
Community Action Framework for Positive Youth Development
You Are Here!
(Gambone et al., 2004)
4
Overview
STEP 1
Decide to build
system
STEP 2a
Self-assessmen
t
STEP 2b
External assessmen
t
STEP 3
Plan for improveme
nt
STEP 4
Carry out plan
SAESystem
Accountability Environment
PLCProfessional
Learning Community
POSPoint
Of Service
Quality Matters presentation
PQA BasicsPQA IntermediateAnchoring
Planning with Data
A la carte Methods Workshops
TAQuality Coaching
STEP 5
Measure change
Program Self-Assessment
External Assessment
Observation-Reflection
Safe Environment Healthy food and drinks
Program space and furniture Emergency procedures Psychological and emotional safety
Physically safe environment
Reframing conflictEncouragementSkill building
Active engagementSession flow
Welcoming atmosphere
What is Assessed?
Partner with adults
Lead and mentorBe in small groups
Experience belongingInteraction
Make choices
Plan
ReflectEngagement
Healthy food and drinks Program space and furniture Emergency procedures
Psychological and emotional safety
Physically safe environment
Reframing conflictEncouragementSkill building
Active engagementSession flow
Welcoming atmosphere
Supportive Environment
Alignment of Professional Development Opportunities
Aligned Professional Development Courses
Cycle of Improvement
Assess program
Assess Program
More Professional Development
Attend Professional Development
Start Here
Over the Span of a
Year
Highlights from Palm Beach County, FL: Prime Time Initiative Goal was to develop a comprehensive,
county-wide system of supports and resources for OST programs
Overall Reported StrengthsTeamwork
Support
Opportunities for networking, program enhancements and training
Prime Time (continued) Specific Professional Development Strengths
Refined offerings to meet shared learning needs
Connect training to program improvement plans
Variety of modules
Quality of workshop leadership
Tools and ideas for program improvement*Real changes
Better communication
Better activities
More youth involvement
Impetus to make time for change
Quality Improvement System
QIS Process•Baseline Assessment•Quality Advisor Assignment•Letter of Recommendation for Improvement•Training•Self-assessment*•Program Improvement Plan*•Implementation •Reassessment
Prime Time (continued) Pilot Quality Improvement System
40 baseline programs; 37 follow-up
Concerns about managing baseline assessments and how assessment would be used
How were assessment handled better? Unclear. Just says “they learned to do it better”
Internal/external assessment value became clearer to providers as it was used for quality improvement
Satisfaction increased with use and understanding
Prime Time Lessons: It Takes Time!
To collect baseline data and develop a quality improvement plan
To form meaningful relationships with people in the improvement process/resource providers
It takes initiative, time and energy for staff to reflect on work and implement new practices
Also, the culture of expectations are as diverse as the participants and affect the process
Prime Time Recommendations:
Ensure sufficient communication about the process, especially for new participants
Consider presenting score reports in person for new participants (by a knowledgeable party)
Bring agency/site directors together to discuss strategies to best support the improvement process (talk to each other about what helps)
Strengthen rapport between sites and improvement facilitators (advisors/coaches)
Accomplishments
Accomplishments
Educating Leaders – Quality Matters Building Local Capacity
27 trained self assessors 19 trained external assessors6 methods trainers
Improving Program Quality3 methods trainings22 attended Planning with Data
Comparison ScoresNational Average and Austin External ScoresNational Average: N= 735 offerings in 180 organizations Austin External Assessments: N=12 offerings
1
2
3
4
5
I. SafeEnvironment
II. SupportiveEnvironment
III. InteractionOpportunities
IV. EngagedLearning
Comparison ScoresAustin External and Self Assessment ScoresAustin External Assessment: N=12offeringsAustin Self Assessment: N=18 assessments
1
2
3
4
5
I. SafeEnvironment
II. SupportiveEnvironment
III. InteractionOpportunities
IV. EngagedLearning
Additional Data Notes
Engagement was significantly related to both supportive environment and interaction, but the relationship was strongest with interaction
Thus, programs that were stronger in interaction were more likely than any others to be strong in engagement
Key factors for high engagement Opportunity to develop a sense of belonging Opportunities to act as group facilitators* Opportunity to partner with adults Support youth with encouragement Support youth with building new skills Support active engagement* Provide welcoming atmosphere
* Explains 56% of variance in engagement
Improvements
Score Improvement Suggestions
Our goal is for each person to measure similar actions the same way each time we go to a location, regardless of where it is (reliability) - and for this action to be measured correctly (validity)
More Suggestions Provide supporting evidence Scores can only be: 1, 3, 5 or X Small groups (III-M)
(2) If no small groups are used, then “1” not NR (X)
(3) Quality of small groups Conflict/Supportive Environment (II-K)
If not observed ask the follow-up questions
Evidence: no conflict or no procedure for conflict
What else can be improved?
Table discussions Report out
FutureOpportunities
Future Opportunities
Quality Coach Quality Advisor Youth PQA Trainers Quality Matters Presentation More Methods Trainings &
Assessments
Quality Coach
Prepares youth program professional to help youth works improve their program or the way they work with youth.
Supervisors or Managers
Network ConsultantsSeptember 2009
*CYPQ Quality Coaching Endorsement
Quality Advisors
Prepares individuals to be network consultants to work with programs and staff throughout the quality improvement process.
October/November 2009
Youth PQA Trainers
Prepares participants to train others in Youth PQA related workshops including:
Youth PQA Basics
Youth PQA Intermediate
Planning with DataJune/July 2009
*Participants must have extensive experience collecting PQA data, participated in Basics, Intermediate and Planning with Data trainings.
ExternalAssessor
Methods Trainings & Assessments
12 - month calendar Increase in cost per training
$20 for programs involved in assessment process
$25 for anyone interested in the modules
More baseline assessments coming in April/May
Quality Matters Presentation
Present recent findings from the YPQI-Setting Change Study, highlight the data and success of local QIS and facilitate a discussion about embedding the improvement work.
Decision-makers, stakeholders, funders and programs.
New Cycle of Improvement
Assess program
Assess Program
More Professional Development
Attend Professional Development
MethodsTrainers
MethodsTrainers
SelfAssessor
ExternalAssessor
SelfAssessor
ExternalAssessor
Over the Span of a
YearQualityCoach
QualityCoach
Start Here
Quality Advisor
Youth PQATrainer
Professional Development
Professional Development and Quality Assessment 2009
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
External Assessments Baselines continue
Planningw/data
Professional Development
Outcomes and Indicators, Youth Service Mapping -
Compact Process Begins
Annual ExternalAssessments
Annual - Self Assessments
Quality CoachingWorkshop
Youth PQATraining
Of Trainers
Quality Advising
Workshop1
Quality Matters
Presentation
Quality Matters
Presentation
Professional Development
QI Overview
ExternalAssessor
SelfAssessor
What is next? short- term
March 31st Ask, Listen and Encourage Module
Reframing Conflict April / May – External Assessments Method Schedule of Modules
Break
Score Improvement Suggestions
Our goal is for each person to measure similar actions the same way each time we go to a location, regardless of where it is (reliability) - and for this action to be measured correctly (validity)
More Suggestions Provide supporting evidence Scores can only be: 1, 3, 5 or X Small groups (III-M)
(2) If no small groups are used, then “1” not NR (X)
(3) Quality of small groups Conflict/Supportive Environment (II-K)
If not observed ask the follow-up questions
Evidence: no conflict or no procedure for conflict
Example: Supportive Environment(II-J. Staff Support)
“Teacher asked many open-ended questions: ‘What’s happening in this poem?’
‘What does it mean to rearrange the earth?’
‘What are some of the other things these poems have in common?’
‘How do you feel about this?’
‘Anything you didn’t like or a weird word you would change?’”
Example: Supportive Environment (II-G. Session Flow)
“Teacher thoroughly explains writing exercise twice. A student comes late. Teacher has a student explain it to the late student and then follows up with additional directions.”