February 2011

Camp Fire USA

Quality Improvement Field Test Report

Leah Wallace, Samantha Sugar & Amanda Sutter

David P. Weikart Center for Youth Program Quality

2

Table of Contents

Introduction and Background ............................................................................................................. 3

Findings and Recommendations ......................................................................................................... 5

Fidelity and Participation.................................................................................................................... 7

Program Quality- Observational Program Self Assessment ................................................................... 9

Program Quality- Organizational Program Self Assessment ................................................................ 12

Survey Responses ............................................................................................................................. 15

Appendix A: Data Table Camp Fire PQA - Form A .......................................................................... 18

Appendix B: Data Table Camp Fire PQA - Form B .......................................................................... 21

Appendix C: Open-ended Survey Responses ...................................................................................... 24

3

ASSESSConduct Program Self-

Assessment and External Assessment with the Youth PQA.

PLANCreate an improvement

plan based on data.

IMPROVECarry out improvement plan to improve point-

of-service quality.

The Youth Program Quality Intervention (YPQI)

Introduction and Background

The Camp Fire USA Quality Improvement Field Test began in November 2009 when the Camp Fire USA National

Office commissioned the David P. Weikart Center for Youth Program Quality (Weikart Center) to help build and

implement a quality improvement system for Camp Fire Councils across the United States using the Youth

Program Quality Intervention as a model. The Camp Fire USA National Office is committed to assuring all Camp

Fire USA programs meet system-wide standards and as a result have sponsored the research and development of the

Camp Fire USA Program Standards and the Camp Fire USA Program Quality Assessment. Use of these standards

and tools annually will help to systematize assessment and reporting across the diverse array of Camp Fire

programs, while also supporting improvement processes that can benefit all young people in these programs.

The Youth Program Quality Intervention (YPQI) is the Weikart Center’s experimentally validated assessment-driven

continuous improvement process. The YPQI is designed to improve the quality of afterschool services by: (a)

building managers’ continuous quality improvement skills; (b) increasing the quality of instructional practices

delivered in afterschool programs; and ultimately, (c) increasing youths’ engagement with program content and

their skill-building opportunities.

The Youth Program Quality Intervention (YPQI)

The YPQI model defines instructional quality as a set of professional practices that, in combination, increase

participating youths’ access to positive developmental experiences. The components of instructional quality emerge

directly from developmental science (Eccles & Gootman, 2002; Gambone, Klem, & Connel, 2002) and the ongoing

research around the Youth Program Quality Assessment (Youth PQA; High/Scope Educational Research Foundation,

2005; Smith & Hohmann, 2005; Blazevski, Smith, Devaney & Sugar, 2008), a standardized observational measure

of instructional practice. The Youth PQA, from which the Camp Fire PQA is derived, is composed of four domains

that comprise 18 scales (summarized in Figure 1.2) and 60 observable items. Higher scores on these observational

items, particularly in the domains of interaction and engagement, are associated with higher levels of youth self-

reports of engagement, while very low levels of quality are associated with youth disinterest. In addition, programs

with high quality instruction provide youth with opportunities to practice emerging social and emotional skills (e.g.,

efficacy, communication, empathy, problem solving) that support success in adolescence and early adulthood.

The Youth Program Quality Intervention follows the Assess-Plan-Improve sequence depicted in Figure 1.1 to help

program staff improve the quality of instruction that they provide for youth. While the Youth Program Quality

Intervention is designed to produce changes in both policies and organizational settings, the ultimate goal is to

improve quality in the instructional setting—the places where youth program experiences occur. This approach to

quality improvement and workforce development is currently being implemented in several thousand agency,

school, and community-based settings in over 20 states.

4

The Camp Fire PQA

The Weikart Center partnered with Camp Fire USA to create a custom version of the standardized assessment tool

called the Camp Fire Program Quality Assessment (PQA), which is aligned with the Camp Fire USA National

Program Standards. The Camp Fire PQA developed for the field test was composed of two forms: (1) Form A,

which has 18 scales and 61 items and (2) Camp Fire PQA Form B, which are composed of 19 scales and 90 items.

Form A consists of a set of observable standards for best practices for staff instruction in youth development

programs. Form B assesses the quality of organizational supports for the youth programs assessed in Form A.

Access

Figure 1.2 The Camp Fire PQA

Pyramid of Program Quality

5

Findings and Recommendations

The findings below are described in detail throughout the remainder of the report.

Findings Summary

Participation was high. Forty Camp Fire USA Councils out of 62 councils that ordered

box sets completed program self assessments using the Camp Fire PQA Form A and/or

Form B.

The intervention was light. Councils were not required to submit improvement plans or

participate in Youth Work Methods trainings, elements that are often a part of more

intensive interventions and support the improvement process.

Utilization of available supports was low. Telecommunication provided access to

trainings and supports when live applications were not feasible. Some Council staff

attended live trainings, but the majority of support was delivered via online trainings or

webinars. Additional ongoing supports such as the help desk and phone/email support

were not fully utilized by Councils.

Most council staff found the process helpful. The majority of surveyed staff found the

project worth their time and effort. Many said that it led to an improvement in staff skills

and program quality.

Some Councils struggled with staffing and time. Some Councils reported that they felt

pressed for time and struggled with staffing limitations when completing the observations,

assessments, and improvement planning.

6

Recommendations

For investments to improve the front-line workforce and volunteers:

Quality assessment data is most importantly considered at the level of the individual staff member or

program for which it was collected. However, based on the information obtained during the CFPQA

program self assessments across all councils, three of the lowest scoring areas could benefit from

increased professional development opportunities:

Culture: Camp Fire USA values the cultures of their participating youth. Providing

intentional opportunities for youth to discuss and share their family culture can affirm youth,

expand world views and encourage respect

Leadership and communication skill building: All youth have the potential to be leaders and

to use their words and ideas to shape their reality. Providing youth with opportunities to

lead, to mentor other youth, and to make presentations can make the youth program a

context in which leadership and communication skills can emerge

Planning and reflection: The skills of making plans for the future and learning from the past

can help youth succeed in school and in life. These skills are tied into what brain scientists

call executive functions, and play an important role in directing attention to tasks and

decision making that connects to understanding consequences.

For investments in management skills:

A Camp Fire USA director’s ability to enact continuous quality improvement is an important factor

in the successful delivery of high quality programming. Establishing the best timeline for the YPQI

within the annual CFUSA cycle will require input from Councils. Offering multiple or flexible entry

points to the cycle may assist Councils in meeting requirements. Together, Weikart Center and

CFUSA could offer advice for customization to meet the needs of individual councils. An emphasis

on low-stakes accountability and supports in the form of professional development and coaching will

help encourage strong buy-in from Councils. Through CFUSA building a cadre of local or regional

trainers that can hold workshops and trainings, Council staff will have increased support for their

improvement process.

For expansion of the Quality Improvement System:

Increase capacity to provide supports and coaching. CFUSA could build human capacity at the

national and regional levels in the form of a cadre of endorsed CFPQA external assessors and

Youth Work Methods trainers, who could serve as coaches and mentors. These endorsed

assessors, trainers and consultants could be used to formally support Councils through the

CFUSA’s full Assess-Plan-Improve sequence and processes.

Identify councils that are passionate about the work and use them as mentors and

ambassadors. CFUSA could identify a small cohort of ―early adopter‖ Councils and invite them

to pilot a deeper intervention more closely aligned with the full Youth Program Quality

Intervention (includes the addition of live trainings and external assessments.)

Support regional Continuous Quality Improvement Work That Involves Camp Fire Councils.

CFUSA could work with the Weikart Center to identify opportunities for Councils to connect

with local or regional initiatives. CFUSA could encourage Councils to take full advantage of

available training and technical assistance.

Add a Youth Outcome Metric. CFUSA could integrate Search Institute’s Development Assets

Profile (DAP) into the CFUSA quality improvement process. This could leverage the analysis

and reporting infrastructure that the Weikart Center is developing via its Quality Assessment and

Asset Building project through the Ready by 21 partnership.

Encourage use of Technical Assistance time. Technical Assistance time was underutilized by

Councils in the field test. Weikart Center staff are available to support the quality improvement

process.

7

Fidelity and Participation

In a recent randomized field trial, funded by the William T. Grant Foundation, the Youth Program Quality

Intervention model produced positive and sustained effects on both managers’ continuous improvement practices

and the quality of instruction delivered by individual staff. Notably, these effects were strongest in sites that

implemented all elements of the model (Smith et al., in preparation).

Table 1.1 describes elements of the Camp Fire quality improvement process and compares its core elements to the

elements of the Youth Program Quality Intervention. Fidelity to the Youth Program Quality Intervention is somewhat

limited, with the Camp Fire process concentrating on program self assessment.

Table 1.1 - Alignment between Youth Program Quality Intervention & Camp Fire USA Quality Improvement

Field Test

Element YPQI CFUSA Notes on Training and Action

External assessment at

baseline (Youth PQA)

Program self assessment at

Baseline (Youth PQA)

Councils completed assessment of 123 programs

in 40 councils during the spring and summer of

2010.

Improvement Planning

Planning with Data webinars in June and

September 2010 used program self assessment

data. Program improvement plans were not

collected.

Youth Work Methods

trainings (High/Scope Active

Participatory Approach

aligned to Youth PQA)

TA Coaching for site

managers (focused on

continuous improvement

practices, managers receive

support in the YPQI process)

Up to 2 hours of phone and email technical

assistance coaching from Weikart Center staff

was available for each Council.

Quality Coaching for staff

(focused on instruction,

managers receive coaching

workshop and support front-

line staff through strengths-

based feedback using the

Youth PQA)

8

Participation in the Quality Improvement Process

Table 1.2 shows the level of participation in the various elements of the Camp Fire USA Quality Improvement Field

Test. Sixty-two councils volunteered to be a part of the field test and received the Camp Fire PQA Box Set, a

collection of materials and access to online training and webinars. A total of 40 councils, or 61% of the councils

that volunteered to participate, submitted completed program self assessments by September 2010.

Table 1.2 Participation levels in Camp Fire Quality Improvement Field Test

PQA Element

Total # of

Councils

Participating

Details

Live PQA Training at the GROW

Conference 32 46 people in attendance

Online PQA Basics Training 14 51 people participating in the short Online

PQA Intro Course

Webinar Attendance n/a 171 total participants at 7 webinars

Phone and Email technical

Assistance from the Weikart Center 37

Total TA Time: 4 hrs 50 minutes (~8

min/council)

Completion of CF PQA Form A 40 123 Council Programs Assessed

Completion of CF PQA Form B 30 40 Council Programs Assessed

Average # of Staff involved in

program self assessment 4.64 staff Self-reported from survey

Average # of staff hours required for

program self assessment

11.17 staff

hours Self-reported from survey

9

Program Quality- Observational Program Self Assessment

Overall Quality- Camp Fire PQA Form A

Program self assessment is the cornerstone of the YPQA quality improvement process because it gives programs the

opportunity to evaluate their practices and consider areas of improvement. The scores are not as important as the

process itself. Most crucial is for programs to consider best practices and make changes; the scores simply provide

information to better understand their own practice.

Figure 2.1 shows the overall quality scores for the 4 domains in the Camp Fire PQA Form A. Consistent with

youth program assessments conducted using the original Youth PQA, Camp Fire Councils scored highest in the

Safe Environment domain and lowest in the Interaction and Engagement domains. This indicates that Councils

have a strong foundation in creating safe and supportive environments for youth, while having less consistency in

creating opportunities for youth to develop a sense of belonging and practice higher order cognitive skills including

planning, choice and reflection. Overall, scores for each domain are quite high, as is typical for self assessments

during a pilot year. Scores are likely to be lower in year 2 of the intervention due to increased familiarity and

comfort with the tool, which commonly causes assessors to have a more critical eye when doing program self

assessment. See Appendix A for the full details of the Form A data.

Figure 2.1. Overall Observational Quality Scores (Camp Fire PQA Form A)

1

1.5

2

2.5

3

3.5

4

4.5

5

I. Safe Environment II. Supportive Environment

III. Interaction IV. Engagement

Cam

p F

ire

PQ

A S

core

DomainCampFire Councils Form A (N=40)

10

Low-Scoring Items- Camp Fire PQA Form A

During the program self assessment process, 10 items scored a ―1‖ on the Camp Fire PQA Form A in 20% or more

of the observed offerings. Scoring a ―1‖ on the Youth PQA means that particular practice was not observed at all

during the offering, indicating that the developmental experience or staff practice was not present or only available

to a small percentage of participants. For example, for item ―II-F Ambient Item: Staff are sensitive to youth’s

culture‖ this staff practice received a score of a ―1‖ in 36% of the Youth PQA forms. Therefore, self assessment

showed that this practice was not present for 36% of young people observed.

Table 2.1 Low-Scoring Items in Observational Assessment

% of Offerings Scoring a "1"

Item N=123

IIF Ambient Item: Staff are sensitive to youths' culture. 36%

IIM3 Youth identify with program offering 35%

IIF1 Opportunities to share family culture 34%

IVP1 Plans for projects and activities 32%

IVP2 Planning strategies 28%

IVP Ambient Item: Staff guide youth in deliberate planning or goal-

setting.

27%

IIF2 Staff respect youth's culture 26%

IIG Staff are prepared for the activity. 25%

IVR1 Youth reflect on what they are doing 23%

IIG2 Materials and supplies ready 22%

11

High Scoring Items- Prevalence of Quality Instructional Practices

Table 2.2 presents high-scoring items from the Form A program self assessment. These are Camp Fire PQA items

where at least 80% of offerings scored a ―5,‖ indicating that the practice is present at a high level.1 For example,

92% of self assessments reported that staff smile, use friendly gestures and make eye contact the majority of the

time. The table below represents programming elements in which Camp Fire councils can be said to consistently

excel.

All of the high scoring items are concentrated in the Supportive Environment domain. This indicates that youth

program staff is able to consistently provide emotional and material support to youth and lay the foundation for

cognitive and social development.

Table 2.2 High Scoring Items in Observational Assessment- Camp Fire PQA Form A

% of offerings scoring a "5"

Item N=123

IIE4 Staff smile, use friendly gestures, make eye contact 92%

IIE5 Ambient Item: Staff provides welcoming experience for youth. 92%

IIE3 Staff tone of voice and language 90%

IIG3 Enough materials and supplies for all youth 83%

IIE1 Emotional Climate is Positive 82%

IIK1 Staff actively involved with youth 82%

III1 Youth engage with materials or ideas 81%

1 Note that this does not include items in the Safe Environment domain as they do not directly address staff instructional

practices.

12

Program Quality- Organizational Program Self Assessment

Overall Quality- Camp Fire PQA Form B

Figure 2.2 summarizes overall quality scores measured by the Camp Fire PQA Form B, which is structured as an

organizational interview and designed to assess the quality of organizational policies and practices in place to

support youth programs. Councils scored lowest in the Youth Centered Policies and Practices domain, which

measures the involvement of youth and families in creating program structures. They scored the highest in the

Access domain, which measures the program’s accessibility to all youth. See Appendix B for the full details of the

Form A data.

Figure 2.2: Overall Organizational Quality Scores- Camp Fire PQA Form B

1

1.5

2

2.5

3

3.5

4

4.5

5

V. Youth Centered Policies & Practices

VI. High Expectations for Youth & Staff

VII. Access

Cam

p F

ire

PQ

A F

orm

B S

core

CampFire Councils (N=30)

13

Low-Scoring Organizational Items- Camp Fire PQA Form B

During the program self assessment process, 10 areas of program structure scored a ―1‖ on the Camp Fire PQA

Form B in 30% or more of the programs assessed. Scoring a ―1‖ on the Youth PQA means that particular program

structure is not in place. For example, for the item VD4 ―Youth and families are involved in staff training and

evaluation‖, self assessments reported that 89% of programs do not have a policy or regular practice to include

youth/families in training and evaluation. It is noteworthy that the three items with the highest percentage of

programs scoring a ―1‖ are related to youth and family involvement in organizational decision making.

Table 2.3 Low Scoring Item- Camp Fire PQA Form B

% of Council Programs Scoring a "1"

Item N=51

VD4 Youth and families are involved in staff training and evaluation 89%

VI03 Key stakeholders are involved in decision making 71%

VC1 Youth and staff share decisions about physical environment 54%

VD7 Youth and families are involved in governing bodies. 47%

VIN6 Short-term staff participate in self-evaluation 39%

VIJ4 Stakeholders and/or the general public give feedback 36%

VIF1 New staff have preservice orientation 32%

VIG1 Young people identify with organization 32%

VIJ3 Feedback from past participants is collected 31%

VIJ6 Programs are offered based on data 30%

14

High Scoring Organizational Items- Prevalence of Quality Program Structures

Table 2.3 presents organizational items in the Camp Fire PQA in which at least 80% of assessed Council programs

scored a ―5,‖ meaning that the program organizational practice is in place at a high level. For example, for the item

―Staff identifies and reduces risk factors,‖ 100% of self assessments found this practice to be a regular occurrence.

The highest scoring items are concentrated in the following two scales: (1) Program Polices Enhance Health and

Safety and (2) Staff Qualification Support a Positive Youth Development Focus. In addition, all assessed Councils’

programs reported that they consistently identify and reduce risk factors including those Councils that provide

transportation for youth reporting that they have transportation safety polices in place.

Table 2.3 High Scoring Organizational Items- Camp Fire PQA Form B

% of Programs Scoring a "5"

Item N=56

VE4 Staff identifies and reduces risk factors 100%

VE5 Transportation safety 100%

VE2 Adequate staff coverage in case of emergencies 98%

VE3 Crisis and risk management plan in place 98%

VE1 Consistent with Accreditation Requirements 93%

VA3 Program Director: youth development job experience 89%

VA1 Administrator: youth development experience 86%

15

Survey Responses

Participant Satisfaction and Staff-Reported Impacts

In December 2010, 79 staff from 62 councils were invited to participate in a survey about their participation in the

Camp Fire USA Quality Improvement Field Test. Of the Council staff invited, 47 staff (59%) completed the survey.

Complete responses to open-ended questions are listed in Appendix C, starting on page 18.

Response to the field test was overwhelmingly positive, with large majorities responding that the process was worth

their time and effort, it helped the council staff understand quality, it led to improvements in staff training, and it

improved the overall quality of youth programming. Fewer Councils reported that the process was helpful in a

fundraising or marketing capacity.

Figure 3.1 Percent of Councils Answering “Agree” or “Strongly Agree” to Questions about the Quality

Improvement Field Test’s Value

When asked to give open-ended response to questions about what changes they have noticed in both staff and youth

in their programs, the response was again overwhelmingly positive, with many councils reporting strong youth

engagement. One respondent noted, ―Students have become more invested in the program, because of the

leadership opportunities presented to them.‖ Another stated that there is more trust in youth and better relationships

with families, ―They trust the staff more. The families are more engaged and responsive to the youth workers when

phone calls are made home (from our after school program).‖

Respondents also reported positive changes in staff, specifically, that staff were more aware of standards of quality

and that the Camp Fire PQA had provided a common language for discussions of quality. One said, ―I have seen

the staff change their own definition of quality programming and focus on those areas where we have found

weaknesses. The language that the staff uses to define program has shifted to a more PQA focused language and

the culture of improvement has developed.‖

0

10

20

30

40

50

60

70

80

90

100

It has helped our staff understand

quality

It has helped up improve the

quality of youth programs

It has been useful in

discussions with funders

It has helped us describe our programs in marketing materials

It has led to changes or

improvements in staff training

It has been worth our time

and effort

Per

cen

t in

Agr

eem

ent

N=4

0

16

In addition to the fact that Councils overwhelmingly felt that the CFPQA had a positive effect on their

programming, 30% of Councils were able to market their participation in the process to encourage funders or

potential program members. Considering the challenges of marketing quality improvement work, this is a

significant number of councils.

Feedback and Suggestions for Future Quality Improvement Work

When asked what additional supports they would benefit from or what changes they would like to have made to the

quality improvement process, respondents repeatedly emphasized the pressures of staffing and time. One

respondent said,

―I think that forms A and B were very well thought out and helpful from a youth development approach and

showed our strengths and weaknesses and where improvement was needed… I think that the amount of

time required was unrealistic for me and the staff. For us it was also a little unrealistic to think that staff and

volunteers would grasp the concept in a short period of time and get reliable results. There's definitely a

learning curve. I do think that continuous program improvement is a must if we're committed to our

mission...‖

When reporting the number of staff and staff hours involved in the project, there was wide variation. Table 3.1

provides both the number of staff and number of staff hours for the program self assessment and improvement

process. It is possible that some Councils may have been less efficient in their use of time and resources – and this

is not uncommon for the first round of implementation. With the learning experience of the field test and with

additional coaching, implementation time can almost certainly be reduced for many Councils, or better,

implementation can be intensified within the same overall time commitment.

Table 3.1 Range in Staff and Total Staff Hours Required

Element Low # High #

# of Staff Involved in Self Assessment 2 15

Total # of Staff Hours Required to Complete Self

Assessment

2 15

# of Staff Involved in Improvement Planning 2 29

Total # of Staff Hours Required for Improvement

Planning

2 60

Feedback on Supports, Training and Coaching

Flexible phone and email technical assistance time was underutilized by Councils, with only 5 hours used out of 130

available hours. The majority of feedback indicated that Councils were highly satisfied with the technical

assistance. One Council reported, ―We felt very supported in this. The Weikart Center was always very responsive

when we had questions.‖ However, one Council reported a negative experience with the technical assistance,

describing missed phone calls and saying, ―I wish that my administrative support/TA contact would have been

more responsive.‖ This negative experience and the limited use of the technical assistance time could indicate that a

more a formal coaching and support process, driven more by the needs of the Councils would be more helpful. As

sites continue in this process and it becomes institutionalized through annual assessments, each site will increasingly

build capacity to complete their self assessments. Not only will staff be more able to deal with the technical elements

of filling out and reporting on the CFPQA, but they will build instructional skills that will really sustain the

continuous improvement process.

Many Councils asked for more tangible support in the form of live training, coaching and support with the creation

of the improvement plans. Several survey respondents echoed this request, ―I would love to see more regional

trainings, but I do not see that as a barrier.‖

17

There is a lot of opportunity in the fact that Councils have a generally positive perception of external assessment.

Most Councils, or 49% of those that participated are in support of having an external assessor visit or are at least

interested in learning more information about the process. This suggests that Camp Fire USA should consider

engaging a small number of committed Councils in the external assessment process accompanied by strong

coaching supports.

Table 3.2 Support for External Assessment

Is your council interested in considering an external assessor for the

coming year?

% Response

(N=40)

No

41%

We would want to know more about this before committing

49% Yes

The recommendations in this report reflect the desire for greater supports and Council’s concerns regarding time

and staffing.

18

Appendix A: Data Table Camp Fire PQA - Form A

Data Table for Camp Fire USA PQA

All Camp Fire

Offerings N=134

Cares Offerings

n=34

Camp Offerings

n=50

Contributes Offerings

n=13

Connects Offerings

n=21

I Safe Environment 4.77 4.87 4.73 4.85 4.68

A. Health and Safety Practices 4.76 4.97 4.51 5.00 4.90

1. Health and safety 4.58 5.00 4.04 5.00 4.81

2. Sanitation 4.80 4.88 4.60 5.00 5.00

3. Ventilation and lighting 4.97 5.00 4.92 5.00 5.00

4. Temperature 4.64 5.00 4.36 5.00 4.81

B. Appropriate emergency & safety procedures 4.61 4.71 4.66 4.71 4.34

1. Emergency procedures 4.02 4.29 4.04 4.08 3.48

2. Fire extinguisher 4.86 4.88 4.90 4.67 4.79

3. First aid kit 4.80 4.76 4.92 5.00 4.43

4. Appropriate emergency & safety procedures 4.90 5.00 5.00 5.00 4.27

5. Youth are supervised 4.67 4.76 4.60 5.00 4.81 C. Program Space and Furniture accommodate the activities 4.93 5.00 4.88 4.85 4.95

1. Sufficient space 4.87 5.00 4.84 4.38 5.00

2. Space can be modified 4.93 5.00 4.84 5.00 5.00

3. Suitable space 4.97 5.00 5.00 5.00 4.81

4. Furniture 4.93 5.00 4.83 5.00 5.00

D. Healthy food and drinks are provided 4.79 4.78 4.88 4.85 4.51

1.Drinking water 5.00 5.00 5.00 5.00 5.00

2. Adequate food and drinks 4.84 4.75 4.88 5.00 4.78

3. Healthy food and drinks 4.46 4.63 4.76 4.11 3.59

4. Food preparation is safe and sanitary 4.67 4.74 4.64 5.00 4.33

II Supportive Environment 4.28 4.30 4.22 4.59 4.19

E. Staff provide a welcoming atmosphere 4.66 4.67 4.56 4.96 4.67

1. Emotional Climate 4.62 4.45 4.68 5.00 4.52

2. Staff greet youth and family 4.38 4.45 4.16 4.85 4.43

3. Staff tone of voice and language 4.79 4.88 4.68 5.00 4.81

4. Staff smile, use friendly gestures, make eye contact 4.84 4.88 4.72 5.00 4.90

Overall: Staff provides welcoming experience for youth. 4.83 4.76 4.83 5.00 4.79

F. Staff show sensitivity to youths' family and culture 3.89 4.10 3.76 3.88 3.81

1. Opportunities to share family culture 2.95 3.36 2.60 2.67 3.38

2. Staff respects youth's culture 3.50 4.06 3.20 3.31 3.29

3. Staff and youth show respect and inclusion 4.67 4.53 4.72 4.85 4.62

4. There are no biased comments 4.50 4.41 4.49 4.54 4.52

Overall: Staff are sensitive to youths' culture. 4.26 4.41 4.13 4.38 4.17

G. Staff are prepared for activities. 4.37 4.45 4.33 4.79 4.16

1. Start on time 4.02 4.12 4.04 4.69 3.57

2. Materials and supplies ready 4.55 4.47 4.57 5.00 4.50

19

Data Table for Camp Fire USA PQA

All Camp Fire

Offerings N=134

Cares Offerings

n=34

Camp Offerings

n=50

Contributes Offerings

n=13

Connects Offerings

n=21

3. Enough materials and supplies for all youth 4.61 4.76 4.48 5.00 4.58

Overall: Staff are prepared for the activity. 4.49 4.53 4.40 5.00 4.37

H. Program activities are appropriately challenging 4.32 4.37 4.36 4.38 4.05

1. Appropriate amount of time 4.39 4.29 4.47 4.54 4.14

2. Activities are appropriately challenging 4.27 4.24 4.20 4.69 4.33

3. Balance high- and low-energy activities 4.24 4.59 4.28 3.92 3.67

Overall: Staff appear to have provided appropriate challenge. 4.31 4.41 4.32 4.69 3.94

I. Activities support active engagement 4.36 4.38 4.23 4.54 4.40

1. Youth engage with materials or ideas 4.59 4.59 4.58 4.85 4.33

2. Tangible products or performances 4.27 4.41 4.12 4.38 4.24

3. Youth talk about what they are doing 4.33 4.41 4.04 4.54 4.62

4. Balance concrete and abstract 4.23 4.12 4.17 4.38 4.37

Overall: Staff provide active learning experiences for youth. 4.41 4.47 4.09 5.00 4.53

J. Staff support youth in building new skills. 4.28 4.15 4.29 4.92 4.00

1. Youth encouraged to try new skills 4.12 3.91 4.22 4.85 3.67

2. Mistakes allowed 4.39 4.39 4.33 5.00 4.10

Overall: Staff support youth in building new skills. 4.28 4.09 4.38 4.85 3.89

K. Staff support youth with encouragement. 4.06 4.04 3.83 4.69 4.13

1. Staff actively involved with youth 4.60 4.41 4.59 5.00 4.60

2. Staff use specific, non-evaluative language 3.89 3.88 3.69 4.69 3.90

3. Open-ended questions 3.73 3.82 3.36 4.38 3.87

Overall: All staff are engaged with youth all the time. 4.13 3.94 4.00 5.00 4.29

L. Staff encourage youth to resolve conflicts 4.25 4.06 4.36 4.59 4.13

1. Approach calmly 4.38 4.25 4.34 4.82 4.60

2. Staff seek input from youth 4.16 4.13 4.15 4.64 3.57

3. Relationship between actions and consequences 4.20 3.78 4.31 4.82 4.25

4. Staff follow up with parent/guardian 4.32 4.30 4.54 4.00 4.00

Overall: Staff handle youths' feelings and conflicts supportively. 4.31 4.12 4.47 5.00 3.80

III. Interaction 4.04 3.99 4.00 4.19 4.04 M. Youth have opportunities to develop a sense of belonging. 4.16 4.10 4.30 4.23 3.87

1. Get to know each other 4.17 4.12 4.27 4.17 4.14

2. Inclusive relationships 4.39 4.24 4.44 4.69 4.30

3. Youth identify with program offering 4.30 4.29 4.36 4.38 4.14

4. Publicly acknowledge achievements 3.84 3.73 4.11 3.62 3.54

Overall: Staff make an effort to make all youth feel a part 4.26 4.15 4.34 4.69 3.82

N. Youth have opportunities to collaborate 4.06 4.00 3.91 4.33 4.21

1.Work towards shared goals 4.24 4.12 4.08 4.38 4.52

2. Work cooperatively 4.26 4.29 4.12 4.54 4.30

3. Interdependent tasks 3.79 3.59 3.65 4.08 4.10

Overall: Staff provide collaborative experiences. 4.08 3.94 3.92 4.38 4.30

20

Data Table for Camp Fire USA PQA

All Camp Fire

Offerings N=134

Cares Offerings

n=34

Camp Offerings

n=50

Contributes Offerings

n=13

Connects Offerings

n=21

O. Youth have opportunities to act as group leaders. 3.89 3.85 3.68 4.00 4.24

1. Group conversation 4.20 4.12 4.11 4.23 4.46

2. Opportunities to lead a group 3.48 3.61 3.16 3.77 3.60

Overall: Staff provide youth with opportunities to lead 3.89 4.00 3.62 4.23 4.06

IV. Engagement 3.29 3.20 3.20 3.71 3.22 P. Youth have opportunities to set goals and make plans. 2.94 2.76 2.93 3.54 2.80

1. Plans for projects and activities 2.90 2.76 2.96 3.00 2.79

2. Planning strategies 3.02 2.87 2.88 4.08 2.80

Overall: Staff guide youth in deliberate planning 3.04 2.87 2.91 4.08 2.89

Q. Youth have opportunities to make choices 3.56 3.53 3.66 3.21 3.47

1. Share age-appropriate control 3.69 3.47 3.69 3.77 3.70

2. Open-ended choices 3.45 3.53 3.65 2.54 3.40

3. Take activity in a new direction 3.55 3.59 3.67 3.31 3.21

Overall: Youth have a choice in how they spend their time. 3.64 3.53 3.63 3.46 3.82

R. Youth have opportunities to reflect. 3.39 3.29 3.05 4.38 3.47

1. Youth reflect on what they are doing 3.25 3.24 2.83 4.08 3.50

2. Staff support or guide reflection 3.35 3.24 3.09 4.38 3.30

3. Youth give feedback on the activities 3.59 3.41 3.30 4.69 3.60

Overall: Youth have opportunities to look back 3.39 3.29 3.05 4.54 3.33

TOTAL 4.10 4.09 4.02 4.34 4.07

21

Appendix B: Data Table Camp Fire PQA - Form B

Data Table for Camp Fire USA PQA Form B All Camp Fire

Councils

N=30

V. Youth Centered Policies and Practices 3.84

A. Staff qualifications support a positive youth development focus. 4.33

1. Administrator: youth development experience 4.58

2. Program Director: youth development training/ education 4.09

3. Program Director: youth development job experience 4.78

4. Staff: relevant education/training 3.82

5. Staff: job experience 4.33

6. Staff: reflect demographics of program participants 4.40

B. Program offerings tap youth interests and build multiple skills. 4.13

1. Influence format or content of program offerings 3.39

2. Program offerings: programmatic focus 4.37

3. Organization: programmatic focus 4.63

C. Youth have an influence on the setting and activities in the organization. 2.85

1. Shared decisions about physical environment 2.29

2. Shared decisions about program offerings 3.12

3. Youth lead sessions 3.24

D. Youth and family members have an influence on the structure and policy of the organization. 3.02

1. Staff respond to family questions 4.36

2. Staff create opportunities for families to participate 3.98

3. Youth and family members participate in program quality reviews 3.27

4. Youth and families involved in staff training and evaluation 1.13

5. Youth and families involved in recruitment 3.03

6. Youth and families involved in community outreach 3.07

7. Youth and families involved in governing bodies. 2.35

E. Program policies and procedures exist to enhance the health and safety of all participants 4.85

1. Consistent with Accreditation Requirements 4.79

2. Adequate staff coverage in case of emergencies 4.87

3. Crisis and risk management plan in place 4.86

4. Staff identifies and reduces risk factors 5.00

5. Transportation safety 5.00

6. Smoking and drug/alcohol use not allowed 4.82

7. Background check 4.69

8. References and experience for all youth workers 5.00

9. All staff are trained in reporting requirements for child abuse 4.82

10. Policies are in place to guide staff interactions with youth 5.00

11. Emergency information for each participant is on file 5.00

12. There are current, signed agreements with providers 4.28

VI. High Expectations for Youth and Staff 3.99

F. Organization promotes staff development. 4.08

22

Data Table for Camp Fire USA PQA Form B All Camp Fire

Councils

N=30

1. New staff pre-service orientation 4.28

2. Professional development within organization 4.25

3. Staff skills and training 4.37

4. Adult education principles used when training staff 3.71

5. Support for staff to attend courses, conferences, workshops 3.81

6. Professional development outside the organization 3.63

7. Staff meet at start of program cycle 4.18

8. Staff meet during program cycle 4.36

G. Organization promotes supportive social norms. 4.28

1. Young people identify with organization 4.32

2. Expectations for personal and social interaction 4.44

3. Encouraging youth to share personal concerns 4.52

4. Staff pursue knowledge on issues and cultures of the youth 3.85

H. Organization promotes high expectations for young people. 4.07

1. High expectations for young people 3.93

2. Acknowledge achievements 4.20

I. Organization is committed to ongoing program improvement. 4.07

1. Assesses youth outcomes 4.03

2. Staff evaluations 3.97

3. Assesses program quality 4.12

4. Program improvement based on assessment 4.17

J. This program is an integral part of the council's program service's plan. 3.47

1. Feedback from young people in program 4.31

2. Feedback from parents/guardians 4.02

3. Feedback from past participants 3.17

4. Feedback from stakeholders and/or the general public 2.84

5. Demographic information 3.23

6. Programs offered based on data 3.21

7. Primary audience for each program based on data 3.31

K. There are intentional strategies that support an overall pattern of growth in services to youth 3.17

1. Intentional strategy that supports growth in services 3.59

2. Intentional strategy to increase participants 2.74

L. For each program, there is a program business plan which includes … 4.30

1. Camp Fire USA mission statement, and core values 4.44

2. Program activities relate to desired outcomes 4.32

3. Written outcomes for youth 3.96

4. Plan supports the needs of the target population. 4.52

M. There is a supervision plan in force that includes a ratio of youth to adults to maximize safety 4.46

1. Ratio of adults to youth 4.76

2. Staff circulate 4.62

3. Minimum number of adult staff 4.00

N. Program administrators assess job performance and satisfaction among staff 3.70

1. Program staff are supported in their work. 4.61

23

Data Table for Camp Fire USA PQA Form B All Camp Fire

Councils

N=30

2. Team work 4.32

3. Staff demonstrate a sense of purpose 4.68

4. Staff receive continuous feed back 3.54

5. Year-round staff participate in self-evaluation 2.66

6. Short-term staff participate in self-evaluation 2.48

7. Supervisors receive training 3.59

O. The program administration provides for sound leadership and management 4.21

1. Strategies to support sustainability 3.79

2. Welcoming atmosphere for staff 4.78

3. Key stakeholders are involved in decision making 3.72

4. Compliance with standards and licensing. 4.53

P. Each program is evaluated for consistency with the goals and business plan. 4.11

1. Goals for sound financial management 4.55

2. Population and participation projections 3.92

3. Results of youth development outcomes 3.86

VII. Access 4.23

Q. Staff availability and longevity with the organization supports youth-staff relationships 4.59

1. Staff with the program the entire program cycle 4.49

2. Staff have returned from previous program cycle 4.69

R Barriers to participation are addressed 3.91

1. Availability of eligible youth 4.00

2. Distance, transportation, neighborhood safety barriers 3.34

3. Cost barriers 4.38

S. Organization communicates with families, other organizations, and schools. 4.43

1. Mechanisms for communication 4.26

2. Communication with other organizations and schools 4.56

T. Parents are informed of the program's purpose, schedules… 4.00

1. Structured opportunities for families to be involved 4.08

2. Formal communication with families 3.99

3. Family members involved in decision-making 3.37

4. Schedules are completed and announced in advance 4.60

TOTAL 4.02

24

Appendix C: Open-ended Survey Responses

34. As a result of participating in this program quality improvement process,

what changes, if any, have you seen for your staff? (i.e. in turnover, morale,

quality of instruction, etc.) (N=28)

Consistency of measures and jargon across varied programs and insight via assessments and observation.

This assessment has not been useful to us because it is so black and white. Most individual sessions do not fit the format. For

example- We observed a Teens in Action meeting. It was a planning meeting only with one girl in charge. Therefore,

leadership was not distributed, they didn't have any hands-on activities, there was no room for expression of beliefs, etc, etc, etc.

That doesn't mean that the program isn’t well balanced over a whole year. You'd have observed programs over and over and

over in order to get a true picture.

At first there were groans and some volunteers and my other staff member felt it was going to create more work. Then morale

was a little better with the understanding that our programs are improving.

They are more direct on the activities that they do. Becoming more involved and modeling for the youth.

Our turnover rate and morale seem to be better. Thanks to our instruction improvements, our training was more to the point

and covered new materials. The staff seem to be better equipped to handle most situations.

We were already using YPQA prior to the CF National pilot. However, we used the pilot as opportunity to expand the types of

programs with which we use the tool. I have seen increased discussion of issues of quality and best practice across our program

models due to use of the tool, and subsequent changes in policies and practices to support quality.

We've done a review with them on the need for it and how it truly helps us assess our performance to know where we are strong

and where we need to improve.

We have not been using the tool long enough to determine changes. We have done assessments, created improvement plans

which we plan on implementing this Spring.

Better program quality--more prepared, organized.

It has helped us to identify what qualities we want in our staff during hiring processes and in volunteer recruitment.

n/a

This process hasn't resulted in any changes in staff by itself. The process used with other in-services and trainings has resulted

in a minor change in staff. Some staff that were a part of the process are no longer with us due to budget cuts and downsizing

and volunteers that were a part of the process are no longer with us due to school schedules etc so there's not really a way to

measure what effect the process has had. We only have 2 part-time direct service staff at the present time and only one some

indirect experience with the process.

We haven't really seen any changes for staff however, this has been a tool to generate discussion and it has been a resource for

our program quality specialist.

Staff are excited that their questions and concerns regarding program quality are being addressed. They seek structured advice

on how to improve their programming and the toolkit gives them just that.

It has allowed the entire team to work on ways to improve our skills in various areas.

I have seen the staff change their own definition of quality programming and focus on those areas where we have found

weaknesses. The language that the staff uses to define program has shifted to a more PQA focused language and the culture of

improvement has developed.

Staff are trying new things and focusing more on youth led programming and initiatives. They also understand what

expectations we have of them.

We have been able to train staff better on the issues we have found not working. Having the pyramid with the different levels

allowed us to see where we were according to National standards and as a program. We obviously want to be able to deliver a

program using the tip top but need to obtain more skills. This is my goal as program administrator.

We increased the training of staff that works directly with the children.

Greater consideration and attention to child & youth contribution to the program and program activities. Also, a more

purposeful interest in family needs and family/youth-wellness--as it impacts program involvement.

25

The morale has always been good, but the turnover rate has not improved due to the hiring of college students

Understanding the focus of quality in programming, tailoring a position to focus on quality, observing the ripple effect down to

leaders, specific direction of quality best practices

Being more aware of the quality that the Council expects and the National Headquarters. It was certainly a learning tool for

new employees in regards to Camp Fire USA as a whole. It also helped all staff to focus on quality and the Board was certainly

pleased with the effort put in by the staff.

Understanding the why's of doing things a certain way. (Risk management, youth planning, etc.) Particularly in the staff that

helped with the assessment.

I am the only paid staff; our volunteer staff did not have much to do with this process and only were involved in being assessed

by our assessment team. Findings were discussed with our board of directors who seemed interested in discussing ways to

improve our program quality, but no changes in training, or staff morale have occurred

They understand their roles better and how to show up as a youth worker. Quality of instruction has increased immensely

among those that attend methods trainings.

We already used the YPQA so in terms of our direct delivery staff, this had little impact. The use of an outside partner to assess

our programming has helped us improve quality at the management level.

We have not completed the full process, and I don't feel that I can respond to most of these questions. I have every expectation

that this will be a useful process and I look forward to continuing work on it this winter and on a go-forward basis.

35. As a result of participating in this program quality improvement process,

what changes, if any, have you seen for the youth with whom you work?

(Outcomes, motivation, attachment to program, etc.) (N=23)

I feel that our internal assessments produce much more detailed feedback and are able to show improvements or "failures" in

much smaller increments.

I have not seen very many changes yet.

They seem to be more interested and focus on what is being offered because they have had more input in what they are doing.

We haven't done outcomes yet so we don't really know but attachment to program has never been a problem. Our traditional

clubs have grown this year and hopefully it is in part due to new training.

Too early to tell yet for the CF pilot.

Same as above.

More excitement with program.

Not anything tangible as of yet as we are moving into this process.

n/a

We conduct a 5 day a week afterschool program with youth who have been attending for over 5 years. Yearly program quality

improvement has been an ongoing process to some degree, so this process in itself hasn't resulted in major changes for the

youth. This process along with our internal plan and outcome measurements required from our funders has resulted in

programming that is more intentional; outcome based and better meets the needs of the youth we serve.

In our council we already have an approach to quality that includes implementing National Afterschool Association and

American Camping Association standards into each of our programs. CFPQA was just a piece. It is difficult to gage results

based solely on CFPQA impact.

Youth are excited to attend programming and maintain a high level of engagement in program activities.

Students have become more invested in the program, because of the leadership opportunities presented to them.

One of the greatest areas of improvement is the level of involvement of the youth in programming and in other decisions. Staff

are much more engaged in supporting this role in the youth.

Youth seem more engaged and more motivated to participate consistently.

26

We have involved the children more in decision making and this really adds investment by them.

More direct involvement from youth in program activity ideas. More opportunities for leadership among youth (Jr. Counselors)

Greater interest in program participation.

We use our outcomes to measure the changes in our youth but have not done ones yet this year.

There were areas of improvement in quality which were different from our first year of our pilot assessment.

They appear to be more involved with the activities and appreciate the program overall. Not to say they did not enjoy prior, but

they respect the organization and happy to be a part of it.

I do not directly work with the youth except at special events We have ALWAYS had great amounts of success with our youth

they are always motivated, they are always excited about the program and can't wait to do anything we ask of them, there are

not any ways that they could be better

They trust the staff more. The families are more engaged and responsive to the youth workers when phone calls are made home

(from our after school program).

We were able to share best practices across one site because partners were able to observe other sites.

36. Overall, what would you change, if anything, about the quality improvement

process that you participated in? (N=28)

Not due

I would like more and quicker access to national results or averages - both CF and other. Second it remains important to

support the importance and utility of internal assessments (vs. external). I understand the strength of external assessments and

have benefited from it, however I do not think this is an option (or a sustainable option) for many, if not most councils.

The ratings need to cover a large scale- like 1 to 10. There is no room here to show small improvements or regressions.

Nothing

The training on conducting the assessments was wonderful and thorough. However, I didn't feel that there was adequate

training on creating the improvement plan. In fact, I didn't realize the process was completed. I was waiting on more info for

what I thought would be "Phase 2" of the project. In fact, I thought that message was communicated at the National Leadership

Summit in November after findings from Phase 1 (assessment) were reported to the group.

Try and make it less time consuming. It took a very long time to observe and even longer to score.

We needed more help in identifying our problems and correcting them.

We are considering expanding the process to include a wider range of stakeholders - youth, parents, alumni, board members

etc...beyond site supervisors and program directors.

To combine it with the one we currently do instead of duplicating efforts.

It's hard to say - it's difficult to measure some of our outdoor programs with this tool, which is challenging. Once we've

implemented the changes made in the improvement plan I'll have a better idea.

The assessment takes too much time.

Having more support from council board to help make this process happen. Because of everything, this has had to take a back

seat whether or not I would choose to make it a first or secondary priority.

More specific ways to improve areas where we are weak.

For me, the 6 hours of training wasn't as useful as the written materials, i.e. the training guide and forms A and B so I would

have preferred to have the option to choose the method of training. I understand the concept of a team approach but the reality

is that direct service staff doesn’t have the time to do the observations-there's no one left to deliver the program and supervise

the youth. Using volunteers has its problems i.e. background and objectivity. Time was a critical issue for me. There just

wasn't time for staff to get trained, do observations, do the scoring and participate in the improvement plan. Our staff are part-

time and budget restrictions prohibit overtime. I can see our direct service staff and volunteers being a part of the process, but

not to the extent that was recommended for this. I would have liked to pick and choose the method of training and if I had a

problem or question, have a contact person to e mail. Time was the biggest problem-this definitely took away from my daily

27

administrative and program duties.

I would change the time-line. The majority of our programs are held during the school year and we were unable to finish the

process by the due date.

Timing of the assessments

I would love to see more regional trainings, but I do not see that as a barrier.

I think it is an excellent and necessary process.

The phone conference calls were VERY HARD to hear and understand. For us the webinars did not work on our computer so

we had to listen on the phone. We could not take advantage of the power points and being able to comment.

How results are tabulated.

Going forward we might consider a specific workshop among assessors to discuss definitions of ideas/concepts/ ratings in the

assessment tool to help insure reliability--or insure program-area specific experts are the ones administering the assessment: i.e.

early childhood person assesses child care program, school age expert assesses after school program, camping expert assesses

camp.

The only thing I would change would be the time of the year.

Provide training modules for all areas of the Camp Fire PQA. Those areas that the Youth Work Methods don't cover.

Nothing of major change. All staffs plate stays full and with the cut backs we all carry a full load.

Some questions did not fit with the setting. In particular activities that are outdoors rather than in a building. The amount of

time required for each observation.

I think it was quite frustrating at times, but it wasn't anything to do with the assessment itself, it was more internal, we have

very few volunteers who could spare more of their time than they already dedicate to our council to do this, and I had a hard

time getting across to some people how important it could be for our use in the future, unfortunately it was looked at as a

waste of time and resources by many, but I am optimistic that attitudes will change with time and experience with the process I

do believe it will get better with time, it will be easier next time

The most frustrating part of this is managing a YPQA in our 21st CCLC programs and the CFPQA in the rest of the programs.

I wish we could just do one, rather than having to do both.

I wish that my administrative support/ TA contact would have been more responsive. I set up a phone conference with her to

have some questions answered regarding Form B. I was five minutes late to the conference (I apologize) and missed her call. I

called her back and left voicemails for her at two extensions and sent her an email (within 10 minutes of the original phone

conference date) and never received a response. That was disappointing.

37. What kind of supports would you like to have provided the next time you go

through a program improvement process? (N=21)

The assessment is too time consuming and I had help from National!! Also- I've listed only the STAFF that participated in this

survey because that was all that was asked. We've also had volunteers participating. I'm not sure where that fits into your

questionnaire.

None

See above.

If there were any ideas on how to shorten the time in scoring but still receive the same results.

More suggestions from the program improvement organizers.

Easier to use digital tools that include room for anecdotes. More training materials to use with staff. More models for how to

engage diverse stakeholders in the process. Funding for the additional staff time required to do this process well.

Getting advice from others that have completed the process was valuable.

See above. Also, more staff input. This process has come down to 1-2 staff making it happen and not a system wide process

where different managers do observations and bring things to the table. Even going over the observation data, only the

observers participated in the meeting. As many were part time staff or volunteers, priority was not to "pay" for their time to

28

come in and sit down and talk about what was observed. Also, from National, more impetus on the upper management to

create an atmosphere of YPQA in the councils versus 1-2 staff who try to implement it. This could have been better to make

board members and CEO's responsible for this.

Just a person to contact if there are questions.

The resources such as the webinar and manual were sufficient.

How best can our council integrate data captured from previous years into national's concentration on the YPQA?

I was given the support that I needed during this process.

We have made a commitment to the improvement process. For us that means we are able to change programs, training, and

anything else related to process that is identified in our improvement plan to get moving toward the quality target that we have

set.

A more detailed webinar about scoring the YPQA.

You provided good support when we asked.

More time

We appreciated our support for the Weikert Center--very helpful. Just having gone through the process will inform our future

assessment work.

Training opportunities closer to local councils.

We felt very supported in this. The Weikart Center was always very responsive when we had questions.

Probably more staff and of course revenue and time.

I think that the online training was great. It was a really useful tool in training not only myself and my staff but our partners.

38. Do you have any other comments about your experience? (N=21)

Took too much time from the staff

Good partnership. Great direction for organization. Look forward to further supports and efforts for increased validity of

internal assessors.

No

I (and my council) enjoyed the process and learned very much. The assessment tools were useful. I hope to receive more tools

to assist with the planning and implementation components.

Not at this time.

I wish it was done in the summer, in July, so we would not be working on so many things at once.

Again - this was duplication and quite time consuming, would like to combine it with the one we do or just do one.

Due dates should not be at the same time as other required reports to national. Too many things at one time in May/June and

August/September.

no

I think some good can come out of this if I can get people on board to talk about this in the first place. It has been a slow

process, but sometimes that is ok. With the strategic plan being put into place, I have been a little frustrated that YPQI is not

being discussed as a major element.

I think that forms A and B were very well thought out and helpful from a youth development approach and showed our

strengths and weaknesses and where improvement was needed. All the written materials were very helpful. I think that the

amount of time required was unrealistic for myself and the staff. For us it was also a little unrealistic to think that staff and

volunteers would grasp the concept in a short period of time get reliable results. There's definitely a learning curve. I do think

that continuous program improvement is a must if we're committed to our mission, youth development and stake holders.

We have a long history of incorporating quality standards into our programs so the CFPQA didn't introduce any new concepts

for our council. We also offer a wide variety of programs - from Before and After School, to Day Camp and Resident Camp, so

it was challenging to make the tool applicable to all of our programs. Overall we enjoyed the process and would be interested

in participating in similar processes in the future.

29

YPQA is a great toolkit that works! Youth and staff alike benefit from it. It also serves invaluable in catching everyone up to

speed when discussing program quality.

This experience allowed everyone involved to really take a look at the way we run quality programs. And not just discuss it, but

take steps to improve those areas.

While I think this experience ultimately had good intentions, there were some unnecessary aspects to the process.

No--we look forward to continued use of this tool and the ultimate positive impact it will have in our programs and services.

We had the benefit of a previous pilot program in our City which gave us prior training and experience with the YPQA. This is

why we did not attend the CFYPQA basics, and planning with data.

No, other than we at the Gulf Wind Council do two evaluations a year involving parents, and the youth. We need the data for

grants and funders. The experience just helped with more data.

Positive

Given the demands of your job, do you feel the process of creating and implementing a program improvement plan was a good

use of your time? It would be if I had any to spare As the only staff person in a small, but busy council who only works 20

hours/week for more than half the year, it was next to impossible for me to actually create or implement anything,

unfortunately we are old and grouchy and resistant to change, which wasn't helpful either. I see this as a great opportunity to

find ways to improve our council and help us move forward, unfortunately, I am the only staff person and it took a lot of my

time, I was not able to create or implement an effective improvement plan by myself. It is something that would be really great

for us if we had a few more staff Using an External Assessor would include an outside person who would conduct the

assessment. Additional training and expenses are involved with external assessors. Is your council interested in considering an

external assessor for the coming year? I believe an external assessment would be useful for us, but that would probably be

viewed as an unnecessary expense, and I feel the results would not be used effectively

I love this tool! It's my favorite, by far!!!