of 48
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
1/48
National 4-H Science
Leadership Academy
Year Two
Regional 4-H Science Academies
Evaluation ReportJune 15, 2012
Mary E. Arnold, Ph.D.
Project Evaluator
Oregon State University
With assistance from
Courtney Archibeque, MpH
Brooke Nott, M.S.
Graduate Research Assistants
Oregon State University
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
2/48
i
Table of Contents
Evaluators Statement .. ii
Acknowledgements .. iii
Executive Summary .. iv
Year Two Regional 4-H Science Academy Overview .. 1
Knowledge and Skills for 4-H Science: Impact of the Regional Academies .. 8
Curriculum Focus .. 8
Evaluation Focus .. 10
Professional Development Focus ..... 11
Fund Development Focus .. 12
Summary .. 14
Confidence and Intention to Teach Others ... 15Curriculum .. 15
Evaluation .. 18
Professional Development ..... 21
Fund Development .. 23
Readiness to Facilitate 4-H Science .. 26
Summary .. 27
Science Liaisons Estimated Reach Beyond Regional Academies ..... 28
Estimates of Knowledge and Skill Dissemination . 28
Additional Resources Garnered for 4-H Science .... 29
Narrative Questions Content Analysis ... 33
Plans for Next Three Months .. 33
Additional Support Needs .. 35
Post-Academy Training Opportunity Ideas.. 36
Appendix 1: Feedback from New England on Effectiveness of Virtual Academy ..... 39
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
3/48
ii
Evaluators Statement
This document serves as the final evaluation report for the National 4-H Science Academy: Year
Two Regional Academies program sponsored by National 4-H Council with funding from the
Noyce Foundation. The academies were held in five locations across four regions of the country
between January and April, 2012. All academy participants were invited to participate in theprogram evaluation.
All data for the evaluation were entered by participants directly into an on-line data collection
system. Access to the system was provided by the evaluator to the participants for data entry,
but only the evaluator and her research assistants had access to the actual dataset. The
integrity and accuracy of the raw data rests with the individual participants. The integrity and
accuracy of the analysis and interpretation rests solely with me as the project evaluator. To this
end, I certify that the analysis and results presented in this document are complete and
accurate insofar as the data entered by the participants were as well. Any questions or
concerns about this report should be addressed to me.
Mary E. Arnold, Ph.D.
Project Evaluator, Oregon State University
June 15, 2012
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
4/48
iii
Acknowledgements
This program evaluation could not have taken place without the dedicated help and support of many
individuals.
First and foremost, I would like to thank National 4-H Council and the Noyce Foundation for theopportunity to conduct the Year Two Regional Academy evaluation. It was exciting to witness the
movement from the national to the regional level, and to document the important work that took place
as a result of the academies.
I would like to express my thanks and gratitude to Janet Golden, Eddie Locklear, Jo Turner, and Maila
Oliveria at National 4-H Council for their support and help with various aspects of the evaluation.
Thank you, also, to the lead planners at each of the five academies, for keeping me apprised of the plans
for your academies, and more than anything for your timely assistance with getting participant contact
information to me.
A very special thank you to my co-authors and graduate assistants: Ms. Courtney Archibeque and Ms.
Brooke Dolenc Nott. Your focused assistance with the data analysis and report preparation could not be
replaced. Thank you both especially for your cheerful willingness to concentrate your work time on the
report so we could meet the expected deadline.
I would like to thank each and every academy participant who contributed data. The sincerity with
which you approached the evaluation was evident in the data and information you provided. Without
your help, there would be nothing to report. So thank you for your willingness to help make this
possible.
Finally, thank you to the Noyce Foundation for the generous support of the National 4-H Science
Leadership Academy. The funding provided by the foundation made this important program possible. Asa result, 4-H programs across the country are more prepared to develop and sustain programs for youth
in science, technology, engineering, and math.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
5/48
iv
Executive Summary
Year Two Regional 4-H Science Academy Overview
Five regional 4-H Science Academies were held during the second year of the National4-H Science Leadership Academy project:
o Northeast (Mid-Atlantic), State College, PA, January 24-26 (73 participants)o North Central-St. Louis, MO, February 7-10 (228 participants)o West-Davis, CA, March 26-28 (100 participants)o Southern- Huntsville, AL, April 9-12 (78 participants)o Northeast (New England), Virtual Academy, April 10-11 (36 participants)
515 invitations to participate in the academy evaluation were sent, with 427 returned,of an overall return rate of 82.9%:
Demographic data from the regional academies indicate that the primary audience forthe academy (county 4-H agents) was reached.
Academy attendance overall appears to represent the size and involvement of 4-HScience programs for each region.
Participants report a fairly even exposure to the academy topics, with slightly moreattending sessions on curriculum and professional development than the other areas.
This, however, matches the results of the national academy evaluation and follow-up in
which participants expressed the most interest in further training in the areas of
curriculum and professional development, especially in the area of scientific inquiry.
Evaluation and fund development participation was also fairly strong, along withpartnerships, which is a topic area added to the regional academies.
The lowest exposure was in the area of marketing, but this area was not explicitly statedin the academy request for applications (RFA).
An interesting finding is that over three-quarters of the academy participants werefemale. This may reflect the gender balance of the 4-H professional workforce,
especially at the county 4-H agent level. But it could also have implications for potential
capacity for engaging girls in 4-H Science.
Prior awareness and use of tools and resources for 4-H science revealed mixed results,with curriculum and webinars showing the greatest awareness and use. Other resources
revealed low level of awareness and use, particularly fund development and urbanscience resources. Since participants should have learned more about these resources at
the regional academy, it will be important to monitor levels of use of the resources in
future evaluations.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
6/48
v
Knowledge and Skills for 4-H Science: Impact of the Regional
A positive result in the area of program evaluation is in using existing tools to evaluate4-H Science programs. This result dovetails nicely with the results of the national science
academy evaluation that showed a distinct need for ready-to-go instruments to
evaluate 4-H Science programs
A significant result appears to be in the area of teaching science inquiry to facilitators of4-H Science programs. This item shows the greatest change in reported level of ability,
with 74% reporting moderate or high levels at the end of the academy. This result
dovetails nicely with the results from previous academy evaluations that showed a
marked need for further training in science inquiry.
The emphasis on inquiry was evident in the results in the professional development andcurriculum areas. This builds nicely on the results of past academy evaluations that
revealed a need for more training on inquiry. It appears that the regional academies met
this request well, and served to further the infusion of science inquiry as an important
aspect of 4-H Science.
There appears to be continued concern in Fund Development (especially with thetoolkit) and Evaluation (especially in providing leadership for evaluation). Learning to
utilize the fund development toolkit and providing leadership for evaluation were two
explicitly stated outcomes for the academies, yet they appear to lag behind the rest of
the outcomes.
Confidence and Intention to Teach Others
The results indicate a positive personal and organizational readiness to facilitate 4-HScience
There is a general emphasis on furthering 4-H Science through curriculum andprofessional development.
There is less ability to facilitate 4-H Science in the fund development and evaluationareas than in curriculum and professional development.
Science Liaisons Estimated Reach Beyond Regional Academies
It is interesting to note the greater reach planned to staff in the areas of infusing scienceand evaluation science programs. This may be an early indicator of a move away from
emphasizing inquiry toward other topics, and could indicate that inquiry as a basis for
4-H Sciences is beginning to be understood and utilized more easily.
The lower levels of dissemination of information to staff related to the science plan ofaction, providing leadership for evaluation, and using the fund development toolkit is
consistent with other findings in this report, and again perhaps indicative of the
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
7/48
vi
specialist audience that needs to be reached about these topics. Clearly, not all staff will
benefit from this information and targeted dissemination efforts should be considered.
Virtual Academy
The Northeast-New England regional academy was held virtually. The evaluation of thisformat received mixed results, with several suggestions for improvement. Because the
Year Three academies are going to be virtual, the feedback and critique from the New
England Academy is presented in Appendix One of this report.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
8/48
1
Year Two: Regional 4-H Science Academies Overview
Five regional 4-H Science Academies were held during the second year of the National 4-H
Science Leadership Academy project:
Northeast (Mid-Atlantic), State College, PA, January 24-26.North Central-St. Louis, MO, February 7-10
West-Davis, CA, March 26-28
Southern- Huntsville, AL, April 9-12
Northeast (New England), Virtual Academy, April 10-11
Academy planners submitted a proposal in response to the Regional 4-H Science Academy RFA
issued by National 4-H Council. The RFA outlined specific content, goals, and outcomes for the
regional academies. Each region submitted a proposal that sufficiently met the requirements
for the regional academies and received funding to plan and host the academy in their region.
The stated goal of the 4-H Regional Science Academies was to:
Build the capacity of state and local 4-H faculty, staff, and volunteers to offer high quality,
sustainable 4-H Science programs by helping them:
Design, implement, evaluate and sustain 4-H Science programs Develop strategies to secure funds and other resources to support 4-H
Science at the state and local levels
Assist Land-grant Universities and counties with developing and refiningtheir 4-H Science Plans of Action
Identify additional training and resources needed to support the 4-H ScienceMission Mandate
Furthermore, specific content was to be provided in order to meet the outcomes for the
academies in the following areas:
Fund Development
1. Every LGU will have a written vision and case for supporting 4-H Science financially.
2. LGUs will update their 4-H Science Plans of Action to include a revenue plan that articulates
the role of government grants, fees for service (if applicable) and philanthropic support as well
as a multi-year fund development plan which includes specific fundraising goals to achieve that
revenue plan.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
9/48
2
3. Every LGU will have a plan for identifying, recruiting and engaging volunteer champions who
are passionate about 4-H and active in a variety of fund development activities in support of the
sustainable expansion of 4-H Science programming within that state.
Curriculum1. Participants will increase their skills to train others on curriculum components and program
quality.
2. Participants will be prepared to develop new, and revise existing curricula, incorporating
identified criteria and standards.
3. Participants will be able to train others to identify how science is infused through mission
mandate areas in 4-H.
Evaluation
1. Participants will learn to utilize existing tools and identify appropriate evaluation methods.
2. Participants will increase their knowledge to determine when and how to do evaluation that
meets their needs.
3. Participants will increase their knowledge and skills to provide leadership to and build
capacity for evaluation at Land-grant Universities and at the county level.
Professional Development
1. Participants will be able to utilize a train-the-trainer model to train staff and volunteers to
facilitate 4-H Science programs.
2. Participants will be able to train staff and volunteers to recruit, develop, and retain
volunteers to facilitate 4-H Science programs.
3. Participants will be able to train staff and volunteers to use 4-H Science professional
development tools and resources.
4. Participants will be able to transfer knowledge and skills learned across all three mission
mandates.
Partnership Development
1. Regional academy planning teams were encouraged to offer training in partnership
development.
The evaluator worked directly with the lead contact person for each academy to secure contact
information (names and e-mail addresses) for all academy participants. Participants were
contacted via e-mail and invited to take part in an on-line post-academy evaluation. Invitations
were sent immediately following the completion of each academy, with up to three follow-up
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
10/48
3
reminders to non-respondents over the course of 2-3 weeks. A total of 515 invitations were
sent, with 427 returned of an overall return rate of 82.9%:
Participant response rates for all the academies were sufficiently strong. Table 1.0 shows the
number of respondents and response rates by academy.
Table 1.0 Response Rates by Academy Invitations Respondents Response Rate
Northeast (Mid-Atlantic) 73 62 84.9%
North Central 228 185 81.1%
South 78 63 80.7%
West 100 87 87.0%
Northeast (New-England) 36 30 83.3%
Total 515 427 82.9%
Participants at the regional academies were overwhelmingly female (76.8%). Sixty five
respondents indicated they had attended the National 4-H Science Leadership Academy at theNational 4-H Conference Center in December, 2010; 351 respondents reported they did not
attend the national academy. Those who had attended the national academy were evenly
distributed according to the focus track they had attended at the national level:
17 attended the curriculum focus track 17 attended the evaluation focus track 15 attended the fund development track 16 attended the professional development track
By far, the majority of participants in the regional academies were county 4-H agents (221
reported this). Additional participant roles in 4-H are presented in Figure 1.0.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
11/48
4
Figure 1.0 Participant Roles in the 4-H Program
* Note: Respondents could indicate more than one role; thirty-five (9%) indicated more than one role.
Respondents were provided the option of indicating their role was something other than those
listed. Table 1.1 provides a summary of the other roles identified by respondents.
Table 1.1 Other Roles Indicated
4-H Program Coordinator 26
Regional Agent or Specialist 17
State Level Educator 12
Community Worker/Educator 6CYFAR/Military 4
Student 2
County Director 1
Retired 1
As stated earlier, regional academies were designed to provide professional development
training for 4-H Science in: Curriculum, Evaluation, Fund Development, and Professional
Development. Content related to Partnerships and Marketing was strongly encouraged. Most
respondents reported experiencing content in the areas of Professional Development and
Curriculum at the academies. This is consistent with the content emphasis of the academies
that emphasized science inquiry in the curriculum and professional development areas. Figure
1.1 shows the number of respondents who reported participating in learning opportunities by
content area.
255
33 31 24
221
3617
4 3
63
0
50
100
150
200
250
Volunteer Partner
(funder,
program
supporter)
State 4-H
Liaison
State 4-H
Science
Specialist
State 4-H
Specialist
(not
science
specific)
County 4-H
Educator
(agent role)
County 4-H
Educator
(non-agent
role)
State 4-H
Program
Leader
State 4-H
Foundation
Staff
State 4-H
Foundation
Director
Missing
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
12/48
5
Figure 1.1 Number of Participants in Academy Learning Areas
* Note: Respondents could indicate participating in more than one area
One last demographic question asked participants about their pre-academy awareness and use
of tools and resources to support 4-H Science (see Figure 1.2).
Approximately 250 respondents (58%) were aware of resources and tools related tocurriculum and to the 4-H Science webinars. But considerably fewer had actually used
these tools: Webinars- 32%; curriculum website- 44%; and curriculum pieces -47%.
Slightly less (226; 53%) were aware of the state or LGUs science plan of action.
For professional development, 130 (30%) reported awareness of the resources, and 85(20%) reported using them.
Just 142 (33%) reported being aware of the fund development resources, and only 37(9%) reported using the resources.
Only 92 (22%) reported being aware of the web resource for science in urbancommunities, with just 32 (7%) reporting having used these resources.
204
111101
111
164
60 62
0
50
100
150
200
250
Curriculum Evaluation Fund
Development
Partnerships Professional
Development
Marketing Attended all
tracks
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
13/48
6
Figure 1.2 Frequencies for Awareness and Use of 4-H Sciences Resources- Pre- Academy
Summary
Demographic data from the regional academies indicate that the primary audience forthe academy (county 4-H agents) was reached.
Academy attendance overall appears to represent the size and involvement of 4-HScience programs for each region.
Participants report a fairly even exposure to the academy topics, with slightly moreattending sessions on curriculum and professional development than the other areas.
This, however, matches the results of the national academy evaluation and follow-up inwhich participants expressed the most interest in further training in the areas of
curriculum and professional development, especially in the area of scientific inquiry.
Evaluation and fund development participation was also fairly strong, along withpartnerships, which is a topic area added to the regional academies.
The lowest exposure was in the area of marketing, but this area was not explicitly statedin the academy request for applications (RFA).
An interesting finding is that over three-quarters of the academy participants werefemale. This may reflect the gender balance of the 4-H professional workforce,
especially at the county 4-H agent level. But it could also have implications for potentialcapacity for engaging girls in 4-H Science.
Prior awareness and use of tools and resources for 4-H science revealed mixed results,with curriculum and webinars showing the greatest awareness and use. Other resources
revealed low level of awareness and use, particularly fund development and urban
science resources. Since participants should have learned more about these resources at
254
142
252 255
92
130
226
137
37
187201
32
85
158
0
50
100
150
200
250
300
National 4-H
Science
Webinars
Fund
Development
Toolkit
4-H Science
Curriculum
Website
National 4-H
Science
CurriculumPieces
4-H Science in
Urban
CommunitiesWebsite
4-H Science
Professional
DevelopmentToolkit
State or LGU 4-
H Science Plan
of Action
Aware Of
Used
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
14/48
7
the regional academy, it will be important to monitor levels of use of the resources in
future evaluations.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
15/48
8
Part Two
Knowledge and Skills for 4-H Science: Impact of the Regional Academy
The second part of the of the regional 4-H Science academy evaluation focused on the
knowledge and skills that participants reported leaving the academies with. Respondents were
asked to rate how much the academy contributed to their knowledge or skill related to
important learning items contained in the academy on a nominal scale of: (1) none; (2) low; (3)
moderate; and (4) high. For the purposes of this evaluation, we were interested in the number
of participants that reported that the academy contributed a moderate or high level to the
development of skill or knowledge about 4-H Science.
However, we knew that the backgrounds of those attending the academy in regards to 4-H
Science would vary considerably, so we also asked respondents to rate their level of knowledge
or skill related to important learning items prior to the academyon the same nominal scale.
For these questions, we were interested in the number of participants that reported none or
a low level of skill or knowledge prior to the academy.
In order to best understand the impact of the academies on personal skill and knowledge, both
of these questions should be considered in tandem. The figures below show the results for both
questions by content area.
Curriculum
Just under half of the respondents reported none or low levels of skill and knowledge related to
4-H Science curriculum before the academy (see Figure 2.0). This indicates that the majority of
respondents had considerable experience in curriculum before attending the academy.
However, the majority of respondents reported that the academy had contributed a
moderate or high amount to their skill and knowledge related to 4-H Science Curriculum (se
Figure 2.1).
This was particularly true in:
Why inquiry is critical to 4-H Science Program quality for 4-H Science Curriculum components for 4-H Science
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
16/48
9
Figure 2.0 Percent of Participants Report None or Low Before the Academy
Figure 2.1 Percent Reporting Academy Learning was Moderate or High
13.4 9.5
24.3 23.211.6 15.6 12.1
35.731.7
32.3 30.9
30.930.4
25.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.080.0
90.0
100.0
Curriculum
components for
4-H science
Program quality
for 4-H science
programs
Develop new
science
curriculum
Revise
curriculum
incorporatingscience
curriculum and
standards
Identify high
quality PYD
sciencecurriculum
Science across
all 4-H mission
mandate areas
Why science
inquiry is critical
in 4-H
Low
None
41.3 36.1 38.6 38.0 35.4 34.823.0
41.0 46.429.8 31.4
43.5 39.8 63.1
0.0
10.0
20.0
30.0
40.0
50.0
60.070.0
80.0
90.0
100.0
Curriculum
components
for 4-H science
Program
quality for 4-H
science
programs
Develop new
science
curriculum
Revise
curriculum
incorporating
sciencecurriculum and
standards
Identify high
quality PYD
science
curriculum
Science across
all 4-H mission
mandate areas
Why science
inquiry is
critical in 4-H
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
17/48
10
Evaluation
About 60% of the respondents reported none or low levels of skill and knowledge related to 4-H
Science program evaluation before the academy (see Figure 2.2), indicating that most of the
respondents did not have strong skill for program evaluation coming into the academy. About
the same percentage reported that they learned a moderate or high amount related to 4-HScience evaluation, meaning at least 40% of respondents reported learning nothing or only
low levels of skill and knowledge related to evaluation (see Figure 2.3). Changes in skill and
knowledge for evaluation are difficult to interpret from these results, and may be most likely
due to the reality that it takes considerable time to develop good skills in program evaluation.
The most positive result is in the area of using existing tools to evaluate 4-H Scienceprograms. This result dovetails nicely with the results of the national science academy
evaluation that showed a distinct need for ready-to-go instruments to evaluate 4-H
Science programs
The least positive result was in providing leadership for 4-H Science evaluation. This islikely reflective of the academy audience; while 4-H agents may engage in evaluation
efforts, they are not typically the ones who provide leadership for such efforts.
Figure 2.2 Percent of Participants Report None or Low Before the Academy
22.6 19.1 20.5 20.8 21.9
39.8 42.6 40.3 38.3 37.8
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.080.0
90.0
100.0
Use exisiting tools for
evaluating 4-H sci.programs
Identify methods for
evaluating 4-H scienceprograms
How to evaluate a 4-H
science program
When to evaluate a 4-
H science program
Provide leadership for
4-H Science evaluation
Low
None
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
18/48
11
Figure 2.3 Percent Reporting Academy Learning was Moderate or High
Professional Development
Professional development appears to be an area where many respondents had considerable
experience prior to the academy, with 50% or less reporting none or low levels of expertise (see
Figure 2.3). This was particularly the case in the area of using train the trainer models with staff
and volunteers, and for forming successful youth-adult partnerships. The ratings for post-
academy levels of knowledge and skill rose considerably with over 60% reporting moderate or
high levels for each skill (see Figure 2.4).
The most significant results appear to be in the area of teaching science inquiry tofacilitators of 4-H Science programs. This item shows the greatest change in reported
level of ability, with 74% reporting moderate or high levels at the end of the academy.
This result dovetails nicely with the results from previous academy evaluations that
showed a marked need for further training in science inquiry.
42.5 42.8 44.3 41.3 36.8
19.5 15.7 14.4 13.015.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.080.0
90.0
100.0
Use exisiting tools for
evaluating 4-H sci.
programs
Identify methods for
evaluating 4-H
science programs
How to evaluate a 4-
H science program
When to evaluate a
4-H science program
Provide leadership
for 4-H Science
evaluation
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
19/48
12
Figure 2.4 Percent of Participants Report None or Low Before the Academy
Figure 2.5 Percent Reporting Academy Learning was Moderate or High
Fund Development
Fund development is the area in which respondents reported the least amount of prior
expertise (see Figure 2.6). Over 50% of respondents reported no or low levels of knowledge and
skill for every item, and this is particularly true in using the fund development toolkit, where
81% reported no or low levels. At the end of the academy at least 60% of respondents reported
moderate to high levels of knowledge and skill for each item, with the exception of using the
fund development toolkit; just 45% of respondents reported moderate or high levels for this
item (see Figure 2.7).
14.4 15.3 14.4 15.5 18.7 10.3
23.4 23.732.2 35.6
33.3
31.4
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Train the trainer
model for
volunteers
Train the trainer
model for staff
Train staff to
recruit develop
and retain
volunteers
Train volunteers to
recruit develop
and reatain
volunteers
Teach scientific
inquiry to
facilitators of 4-H
sci. programs
Form YAPs to
support 4-H sci
programs
Low
None
41.9 37.5 42.7 39.8 31.444.1
29.530.6 22.9 23.6 45.8
29.2
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Train the trainer
model for
volunteers
Train the trainer
model for staff
Train staff to
recruit develop
and retain
volunteers
Train volunteers
to recruit develop
and reatain
volunteers
Teach scientific
inquiry to
facilitators of 4-H
sci. programs
Form YAPs to
support 4-H sci
programs
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
20/48
13
These results suggest that the fund development area is one in which most 4-H agentsdo not have a great deal of experience. However, the reported increase in knowledge
and skills in fund development as a result of the academy is fairly strong, which
indicates that the academy served an important role in skill and knowledge
development in the fund development area. The lack of skill to understand and use the fund development toolkit remains a
concern. However, it is not entirely clear how much the use of the toolkit was
emphasized during the academies. The low rating could be simple a reflection of the
toolkit not being emphasized, or they could be indicative of confusion about how to use
the toolkit, or it could be a reflection of the fact that most 4-H agents are not focused on
fund development. Given the resources that have gone into developing the toolkit, it
will be important to keep monitoring its use by 4-H Science programs.
Figure 2.6 Percent of Participants Report None or Low Before the Academy
31.1 30.620.6
55.7
27.016.5
25.4 28.6 22.1
26.4 28.538.5
25.8
39.0
36.4
40.2 36.439.2
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Develop
state or LGU
Plan of
Action
Develop
county Plans
of Action
Why invest in
4-H science -
for donors
Fund
development
toolkit use
Volunteer
champions
for 4-H
science
Partnershipss
for 4-H
science
Identify
prospective
donors
Support fund
development
for 4-H
science
Work with
volunteers to
support fund
development
Low
None
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
21/48
14
Figure 2.7 Percent Reporting Academy Learning was Moderate or High
Summary
Overall, these results indicated that the learning at the academy was strong and positivefor all of the content areas.
The emphasis on inquiry was evident in the results in the professional development andcurriculum areas. This builds nicely on the results of past academy evaluations that
revealed a need for more training on inquiry. It appears that the regional academies met
this request well, and served to further the infusion of science inquiry as an important
aspect of 4-H Science.
There appears to be continued concern in Fund Development (especially with thetoolkit) and Evaluation (especially in providing leadership for evaluation). Learning to
utilize the fund development toolkit and providing leadership for evaluation were two
explicitly stated outcomes for the academies, yet they appear to lag behind the rest of
the outcomes. It may be that there is a need to identify and match these two aspects of
the academy to the right audience. Who is it at each LGU that has the capacity and right
position description to further these outcomes? Given the specialized nature of these
outcomes, these results may be a function of missing the audience who can best use the
toolkit and/or provide leadership for evaluation. Future offerings in these areas should
be targeted at those who have the capacity to achieve them.
42.234.1
40.434.4
42.7 46.8 44.7 40.7 39.9
28.429.9
33.1
9.9
18.8
33.3
19.4 21.7 24.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Develop
state or LGU
Plan of
Action
Develop
county Plans
of Action
Why invest in
4-H science -
for donors
Fund
development
toolkit use
Volunteer
champions
for 4-H
science
Partnershipss
for 4-H
science
Identify
prospective
donors
Support fund
development
for 4-H
science
Work with
volunteers to
support fund
development
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
22/48
15
Part Three
Confidence and Intention to Teach Others
One of the goals of the National 4-H Science Leadership Academy is to disseminate knowledge
and skills related to 4-H Science programming to professionals and volunteers throughout the
National 4-H system. This goal is reflected in the structure of the academy for Year Two, in that
resources were moved from the national to the regional level with the intention of reaching
more people through focused regional academies. As such, it is an additional goal that
participants in the regional academies will develop skills and confidence to continue to build
4-H Science programs locally by providing professional development opportunities for other
4-H professionals, volunteers, and partners at the local level. To this end, a section of the
regional academy evaluation focused on participants confidence to use information they
learned and confidence to teach the information to others. In addition, participants were asked
about their intention to teach others.
Participants were asked to rate their level of confidence to use each item and to teach it to
others. They were also asked to rate the likelihood that they would teach each items to others
as part of their 4-H Science program. The figures below show the percentage of respondents
who rated each item moderate or high.
Curriculum
Respondents reported fairly high levels of confidence related to each of the curriculum items
(see Figure 3.0). The greatest level of confidence was in understanding that inquiry is critical
component of 4-H Science (90%). The lowest was in developing curriculum that incorporates
science criteria and standards (69%). This is consistent with previous evaluation reports that
indicate states are not planning to develop their own science curriculum, but look for
curriculum that is developed by others with expertise in 4-H Science.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
23/48
16
Figure 3.0 Percent of Participants Report Moderate or High Confidence to use Item
(N = 309)
Respondents also reported moderate to high levels of confidence to teach curriculum items to
others (see Figure 3.1). The highest level was in teaching why science inquiry is critical to 4-H
Science (91%); the lowest was teaching others how to develop curriculum that incorporates
science criteria and standards. The reported levels of confidence to teach others mirrors the
respondents own levels of confidence in each item and supports the emerging evidence that
science inquiry as a foundational part of 4-H Science is taking hold, while independent efforts
to develop 4-H Science curriculum is a lesser priority for many LGUs.
37.2 41.4 40.4 43.0 42.145.3
32.5
47.646.6
29.032.6
45.0 38.4 58.4
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Curriculum
components
for 4-H science
Program
quality for 4-H
science
programs
Develop
curriculum that
incorporates
science criteria
and standards
Revise
curriculum that
incorporates
science
curriculum and
standards
Identify high
quality PYD
science
curriculum
How science is
infused
through 4-H
mission
mandate areas
Why science
inquiry is a
critical
component of
4-H science
curriculum
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
24/48
17
Figure 3.1 Percent of Participants Report Moderate or High Confidence to Teach Others
A similar picture emerges for respondents plans to teach others about 4-H Science curriculum
(see Figure 3.2). Most respondents intend to teach others about science inquiry for 4-H Science
(90%), followed by program quality (85%) and curriculum components for 4-H Science (83%).
Only 61% have intentions to teach about developing curriculum for 4-H Science.
Figure 3.2 Percent of Participants Report Moderate or High Intention to Teach Others
42.2 45.1 41.8 42.8 44.8 45.1 40.8
40.541.8
24.328.6
36.9 35.5 49.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Curriculum
components
for 4-H science
Program
quality for 4-H
scienceprograms
Develop
curriculum that
incorporatesscience criteria
and standards
Revise
curriculum that
incorporatesscience
curriculum and
standards
Identify high
quality PYD
sciencecurriculum
How science is
infused
through 4-Hmission
mandate areas
Why science
inquiry is a
criticalcomponent of
4-H science
curriculum
High
Moderate
37.5 39.3 34.3 37.343.0 42.0
34.1
45.8 46.0
27.031.0
36.8 39.7 56.0
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Curriculum
components for
4-H science
Program quality
for 4-H science
programs
Develop
curriculum that
incorporates
science criteria
and standards
Revise
curriculum that
incorporates
science
curriculum and
standards
Identify high
quality PYD
science
curriculum
How science is
infused through
4-H mission
mandate areas
Why science
inquiry is a
critical
component of
4-H science
curriculum
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
25/48
18
In addition to determining intentionsforteaching content to others, it is important to also
determine intentions not to teach content to others. This provides a picture of potential
problems in reaching the information dissemination goals of the regional academies. Figure 3.3
shows the percentage of respondents who indicated they did not plan to teach content to
others. These ratings all fell below 15%, which is low, but still indicative that some respondentsdid not have plans to teach others, which could mildly impact the success of the academys goal
to disseminate knowledge and skill beyond those who attended the academy.
Figure 3.3 Percent of Participants Report No Intention to Teach Others
Evaluation
Respondents reported lower levels of confidence related to each of the program evaluation
items (see Figure 3.4). The greatest level of confidence was in understanding how to use
existing tools for evaluating 4-H Science programs (70%) and the lowest was providing
leadership for evaluation (61%).
9.311.9 12.2
9.06.5
8.0
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
Use train-the-trainer
model to train 4-H
science volunteers
Use train-the-trainer
model to train staff
in 4-H Science
Train staff to recruit
develop and retain
volunteers
Train volunteers to
recruit develop and
retain volunteers
Teach scientific
inquiry to 4-H
science facilitators
Form YAPs to
support 4-H Science
programs
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
26/48
19
Figure 3.4 Percent of Participants Report Moderate or High Confidence to use Item
Respondents reported even less confidence to teach others about evaluation (see Figure 3.5).
on the high side, 61% indicated confidence to teach others about using evaluation tools, while
on the low side, only 55% felt confident to teach others how to identify appropriate methods
for evaluating 4-H Science programs.
Figure 3.5 Percent of Participants Report Moderate or High Confidence to Teach Others
44.451.8 51.7 49.8 42.8
25.5 14.2 12.9 16.318.8
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
How to use exisiting
tools for evaluating 4-
H science programs
How to identify
appropriate methods
for evaluating science
programs
Determining how to
evaluate a 4-H science
program
When to evaluate a 4-
H science program
Provide leadership for
evaluation of 4-H
science programs
High
Moderate
42.1 43.1 47.8 47.1 39.3
18.9 11.410.8 12.8
15.8
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.090.0
100.0
How to use exisiting
tools for evaluating 4-H science programs
How to identify
appropriate methodsfor evaluating science
programs
Determining how to
evaluate a 4-H scienceprogram
When to evaluate a 4-
H science program
Provide leadership for
evaluation of 4-Hscience programs
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
27/48
20
Despite the lower levels of reported confidence related to evaluation and teaching to others,
over 50% reported a moderate or high intention to teach these concepts to others (see Figure
3.6).
Figure 3.6 Percent of Participants Report Moderate or High Intention to Teach Others
More respondents reported that they did not plan to teach others in the area of program
evaluation. As Figure 3.7 shows, the percentage ranged from 10 % for how to use existing
tools to 17.6% for providing leadership for program evaluation. These results could reflect the
fact that program evaluation is a complex topic that takes time to master, as well as that most
county agents are not typically involved in teaching evaluation to others. The bright note in
these results is that teaching others to use existing tools for 4-H Science programs appears to
be the topic that will be taught the most to others. This supports the findings of previous
evaluations that indicated having access to ready tools for evaluation should be a priority.
39.1 43.2 43.2 40.1 36.6
24.8 16.3 16.0 17.0 19.0
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
How to use exisiting
tools for evaluating 4-
H science programs
How to identify
appropriate methods
for evaluating science
programs
Determining how to
evaluate a 4-H
science program
When to evaluate a
4-H science program
Provide leadership
for evaluation of 4-H
science programs
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
28/48
21
Figure 3.7 Percent of Participants Report No Intention to Teach Others
Professional Development
Over 70% of respondents reported moderate or high confidence levels to use and teach items
related to professional development (see Figures 3.8 and 3.9). Intentions to teach others
dropped off slightly, especially around training staff and volunteers to identify, recruit and
retain volunteers for 4-H Science. Even so, over 65% reported moderate or high intentions to
teach others about all professional development items (see Figure 3.10).
Figure 3.8 Percent of Participants Report Moderate or High Confidence to use Item
10.1
15.3 15.0 15.617.6
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
How to use exisiting tools
for evaluating 4-H science
programs
How to identify
appropriate methods for
evaluating science
programs
Determining how to
evaluate a 4-H science
program
When to evaluate a 4-H
science program
Provide leadership for
evaluation of 4-H science
programs
41.538.4
51.0 50.843.1 44.6
38.137.0
25.3 22.6 38.8 38.6
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Use train-the-
trainer model to
train 4-H science
volunteers
Use train-the-
trainer model to
train staff in 4-H
Science
Train staff to
recruit develop
and retain
volunteers
Train volunteers
to recruit develop
and retain
volunteers
Teach scientific
inquiry to 4-H
science
facilitators
Form YAPs to
support 4-H
Science programs
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
29/48
22
Figure 3.9 Percent of Participants Report Moderate or High Confidence to Teach Others
Figure 3.10 Percent of Participants Report Moderate or High Intention to Teach Others
An analysis of the respondents who reported no intention to teach others about curriculum
topics revealed relatively low percentages ranging from 6.5% for teaching scientific inquiry to
12.2% for training staff to recruit volunteers. These results are similar to those found for the
curriculum area and are indicative of the general emphasis placed on furthering 4-H Sciencethrough curriculum and professional development training.
43.9 39.851.9 51.0 46.3 44.6
36.1 34.7 21.8 20.1 32.8 33.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Use train-the-
trainer model to
train 4-H science
volunteers
Use train-the-
trainer model to
train staff in 4-H
Science
Train staff to
recruit develop
and retain
volunteers
Train volunteers
to recruit develop
and retain
volunteers
Teach scientific
inquiry to 4-H
science
facilitators
Form YAPs to
support 4-H
Science programs
High
Moderate
35.1 31.539.5 43.2 43.3 39.4
39.537.4 27.6
22.236.1
36.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Use train-the-
trainer model to
train 4-H science
volunteers
Use train-the-
trainer model to
train staff in 4-H
Science
Train staff to
recruit develop
and retain
volunteers
Train volunteers
to recruit develop
and retain
volunteers
Teach scientific
inquiry to 4-H
science
facilitators
Form YAPs to
support 4-H
Science programs
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
30/48
23
Figure 3.11 Percent of Participants Report No Intention to Teach Others
Fund Development
Things are a bit less positive in the fund development area, with respondents reporting
considerably less confidence to use some of the fund development content (see Figure 3.12).
This is particularly true for confidence to use the fund development toolkit, with only 48%
reporting moderate or high confidence.
Figure 3.12 Percent of Participants Report Moderate or High Confidence to use Item
(N=316)
9.311.9 12.2
9.06.5
8.0
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
Use train-the-trainer
model to train 4-H
science volunteers
Use train-the-trainer
model to train staff
in 4-H Science
Train staff to recruit
develop and retain
volunteers
Train volunteers to
recruit develop and
retain volunteers
Teach scientific
inquiry to 4-H
science facilitators
Form YAPs to
support 4-H Science
programs
47.8 46.4 52.836.3 45.0
51.9 51.3 46.3 48.4
25.9 28.1 19.6
11.313.1
29.613.9 15.7 16.5
0.010.020.030.040.050.060.070.080.090.0
100.0
Develop a
state or LGU
Plan ofAction
Develop
county Plans
of Action
Make case
for investing
in 4-Hscience for
donors
Utilize fund
development
toolkit
Identify
recruit,
engagevolunteer
champions
for 4-H
science
Develop
partnerships
for 4-Hscience
Identify
prospective
donors
Work with
others to
support funddevelopment
for 4-H
science
Work with
volunteers to
support funddevelopment
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
31/48
24
Confidence to teach the fund development to others drops off even more (see Figure 3.13), for
each of the items. The one that remains strongest is developing partnerships for 4-H Science
where 78% of respondents indicated moderate or high confidence to teach to others. A similar
pattern emerges for intention to teach others (see Figure 3.14).
Figure 3.13 Percent of Participants Report Moderate or High Confidence to Teach Others
44.2 45.5 46.835.7
46.052.6 47.0 43.5 45.9
21.826.9
16.3
7.5
9.7
25.0
11.5 13.5 12.7
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Develop a
state or LGU
Plan of
Action
Develop
county Plans
of Action
Make case
for investing
in 4-H
science for
donors
Utilize fund
development
toolkit
Identify
recruit,
engage
volunteer
champions
for 4-H
science
Develop
partnerships
for 4-H
science
Identify
prospective
donors
Work with
others to
support fund
development
for 4-H
science
Work with
volunteers to
support fund
development
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
32/48
25
Figure 3.14 Percent of Participants Report Moderate or High Intention to Teach Others
The percentage of respondents reporting that they have no intention to teach others,
however, is highest for the fund development area content (see Figure 3.15). This is
particularly true with teaching others how to utilize the toolkit (22.8%), developing the state or
LGU plan of action (15.1%) and working with others to support fun development for 4-H Science
(15%). On a bright note, only 7.1 percent indicated no plans to teach about developing
partnerships for 4-H Science.
35.1 35.741.4
31.437.8
46.841.6 40.8 43.0
27.233.8 24.2
13.2
19.1
31.2
18.8 19.3 18.5
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Develop a
state or LGU
Plan ofAction
Develop
county Plans
of Action
Make case
for investing
in 4-Hscience for
donors
Utilize fund
development
toolkit
Identify
recruit,
engagevolunteer
champions
for 4-H
science
Develop
partnerships
for 4-Hscience
Identify
prospective
donors
Work with
others to
support funddevelopment
for 4-H
science
Work with
volunteers to
support funddevelopment
High
Moderate
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
33/48
26
Figure 3.15 Percent of Participants Report No Intention to Teach Others
Readiness to Facilitate 4-H Science at Participating Land Grant Universities
One last question in this section related to dissemination of knowledge asked participants
about their personal and organizational readiness to facilitate 4-H Science in their state.
Participants were asked to rate their level of agreement with several statements related to
personal and organizational readiness to facilitate 4-H Science programs at their LGU (N = 330).
92.4% agreed or strongly agreed that the team attending the regional academy fromtheir LGU is ready to facilitate 4-H Science programs
91.8% agreed or strongly agreed that there is a positive environment at their LGU for 4-HScience
92.5% agreed or strongly agreed that they have the skills needed to developpartnerships for 4-H Science
Figure 3.16 shows the percentage of respondents who agreed or strongly agreed with each
item.
15.111.7 11.4
22.8
14.8
7.1
13.015.0
12.6
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
Develop a
state or LGU
Plan of Action
Develop
county Plans
of Action
Make case for
investing in 4-
H science for
donors
Utilize fund
development
toolkit
Identify
recruit,
engage
volunteer
champions for
4-H science
Develop
partnerships
for 4-H
science
Identify
prospective
donors
Work with
others to
support fund
development
for 4-H
science
Work with
volunteers to
support fund
development
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
34/48
27
Figure 3.16 Percentage of Respondent Agreement for Personal and Organizational Readiness
Summary
The results indicate a positive personal and organizational readiness to facilitate 4-HScience
There is a general emphasis on furthering 4-H Science through curriculum andprofessional development.
There is less ability to facilitate 4-H Science in the fund development and evaluationareas than in curriculum and professional development.
61.854.9 59.8 54.9
45.6 51.559.8
29.733.5 32.6 32.0 46.2
26.421.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Skills to
develop
partnerships for
4-H Science
Ready to
facilitate 4-H
Science
Group
attending
academy is
ready to
facilitate 4-H
Science
LGU is
organizationally
ready to
support 4-H
Science
Positive
environment at
LGU for 4-H
Science
Adequate
support at LGU
for 4-H Science
Prepared to
form youth-
adult
partnerships to
support 4-H
Science
Strongly Agree
Agree
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
35/48
28
Part Four
Science Liaisons: Estimated Reach Beyond Regional Academies
Estimates of Knowledge and Skill Dissemination
Thirty-three participants in the regional science academies indicated they are the 4-H Science
Liaison for their state or LGU. Twenty-three of the 33 (70.0%) responded to questions asked of
the liaisons only. These questions asked liaisons to estimate the total number of people that
they plan to reach for 4-H Science using the tools and resources provided as part of the
academy. Liaisons were asked to report this information for their whole state or LGU, not just
for them personally.
High and low parameter estimates were calculated to create a range of estimated reach for
disseminating content area information to each audience. The Table 4.0 presents the estimated
ranges for reaching others with content. The estimates are organized by audience (staff,
volunteers, and partners). Cells highlighted in GREEN indicate the areas where the most reach is
planned; cells highlighted in RED indicate the areas where the least reach is planned.
Table 4.0 Liaison Estimates of Knowledge and Skill Dissemination
Topic Staff Volunteers Partners
Developing State or LGUs science Plan of Action 233-545 383-650 22-146
Science infused throughout three 4-H mission
mandates
628-1055 1664-2151 67-213
Evaluating 4-H Science Programs 628-1050 1426-1985 55-177
Teaching scientific inquiry to facilitators of 4-H
Science568-985 2046-2221 73-235
Making the case for investing in 4-H Science 222-520 945-1360 85-271
Providing leadership evaluating 4-H science programs 171-430 712-1060 19-113
Using the train-the-trainer model to train others 426-760 1086-1585 567-713
Developing partnerships for 4-H Science 407-755 1077-1545 551-729
Developing curriculum that incorporates inquiry,
science criteria and standards316-640 767-1125 28-116
Revising curriculum to incorporate inquiry, criteria
and standards 496-870 1248-1815 551-677
Utilizing the fund development toolkit 220-485 820-1135 27-105
Engaging champions for 4-H Science fund
development259-525 800-1205 209-410
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
36/48
29
Staff
The greatest planned reach for staff is in the areas of 1) teaching how science is infusedthroughout the mission mandate areas; and 2) evaluating science programs.
The least amount of planned reach is in the areas of 1) developing the science plan of action; 2)providing leadership for evaluating 4-H Science programs; and 3 utilizing the fund development
toolkit.
Volunteers
The greatest planned reach for volunteers is in the areas of 1) infusing science throughout thethree mission mandate areas; and 2) teaching scientific inquiry to facilitators of 4-H Science
programs.
The least amount of planned reach for volunteers is in developing the state of LGU science planof action.
Partners
The areas with the greatest planned reach for partners are: 1) using the train-the-trainer modelto train others; 2) developing partnerships for 4-H Science; and 3) revising curriculum to
incorporate inquiry, criteria and standards.
Summary
While it is important to note that the figures presented in Table 4.0 are only estimates and only
from twenty-three state Science Liaisons, they do paint an interesting and consistent picture of
the potential dissemination reach of skills and knowledge from the regional academies.
It is interesting to note the greater reach planned to staff in the areas of infusing scienceand evaluation science programs. This may be an early indicator of a move away from
emphasizing inquiry toward other topics, and could indicate that inquiry as a basis for 4-
H Sciences is beginning to be understood and utilized more easily.
The lower levels of dissemination of information to staff related to the science plan ofaction, providing leadership for evaluation, and using the fund development toolkit is
consistent with earlier findings in this report, and again perhaps indicative of the
specialist audience that needs to be reached about these topics. Clearly, not all staff will
benefit from this information and targeted dissemination efforts should be considered.
Additional Resources Garnered for 4-H Science
Liaisons were also asked about new partnerships, gifts, grants, and other donations that have
been leveraged to facilitate 4-H Science in their state. Liaisons reported the following funding
secured for 4-H Science:
Gifts: $159,000.00 (6 liaisons reporting)
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
37/48
30
Grants: $2,007,900 (14 liaisons reporting) Donations: $102,000 (2 liaisons reporting) In-kind estimates: $135,390 (5 liaisons reporting)
In addition to financial resources, Table 4.1 lists the new partnerships developed for 4-HScience by LGUs.
Table 4.1 New Partnerships formed by LGUs to Facilitate 4-H Science
Partnerships
Robotics support through National 4-H Council We are working with libraries and a state level contact for libraries - trying to increase 4-H
STEM programs in libraries and connect to literacy.
We've applied for Smith-Lever funding for a project to develop materials for a 4-H summerSTEM/literacy library program.
We are increasing efforts to connect with campus partners in departments and researchcenters.
We have secured funding for our statewide STEM Program Work Team to hold a springprofessional development/STEM Plan development retreat.
OJJDP 4-H Tech Wizards Walmart - HealthJam and Summer Nutrition Programs JCP RoboticsUSB Biotech ADM Grant to Support Think Green Environmental Program
Received grants from United Soybean Board for biotech, TecXite NSF subgrant forengineering, Gear-tech-21 NSF subgrants for robotics and Geospatial, and grants from MFA
and JC Penney for robotics.
Several corporate partners: Best Buy, 3M, CASE Int. Corporate Partnerships include Lowe's & Walmart. Institutional partnerships includeOklahoma State University and University of Oklahoma. Working to establish a relationship
with other higher education institutions.
- LGU departments (i.e. Physics, Education, Engineering, etc.) - 4-H Foundation partnering tofind funding for 4-H Science programs - LGU Engineering Alumni to partner with county
educators - Association with other state regional universities.
Tennessee Geographic Information Council is support of a GIS 4-H program. Foundation support from within the Texas 4-H Program and from outside foundation
sources have been very generous in the 4-H Science area.
We have been using our extension funds, resources we obtained from other mini-grants, andcollaborating with LGUs in the state for science resources (staff training and resource
materials and facilities) Partnership with a public utility to support the NYSD Industry partners to support robotics
program Grants to support science programs
California Afterschool Network - Professional development and 4-H curriculumdissemination. California Science Teachers' Association - 4-H workshops at their conferences
Grants from National 4-H Council FairPoint Communications - funds for equipment and volunteer training; UVM Extension
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
38/48
31
AmeriCorps & VISTA support funding; State 4-H Foundation equipment grants;
The University has created a new position that will focus entirely on 4-H Science. They havealso shifted and identified 4-H Science as a focus for all Extension staff.
JCPenney Foundation (4-H Robotics Grant) ESRI -GPS grant Funding from private source fordevelopment of junior master gardener program
Finally, Liaisons were asked to describe other developments in their 4-H Science program that
are direct results of the national and or regional 4-H science academy programs. These
developments are presented in Table 4.2
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
39/48
32
Table 4.2 Developments in 4-H Science as a Result of the National and Regional Science
Academy Programs.
Our staff are making the "subtle shifts" to inquiry based learning. Strategic planning at the state level Primarily work on our state STEM plan as a result of the National Academy. As a result of the
Regional Academy we will be doing more revisions to our state plan and holding a Program
Work Team retreat and a State Academy for county Educators and volunteers.
Simply organization and unifying state objectives and delivery methods. Also build networksand validate our work. Very beneficial!
Extension administration support of funds for staff professional development in 4-H Science(60 staff attended regional academy)
Determined to train all staff in the inquiry process in the coming year. To figure out how toincorporate the inquiry process into curriculum and teaching lessons for state wide specialists.
Hosted NC regional 4-H science academy with approximately 55 Missouri delegates. Networking with other LGU's and picking up on some of the resources has been a direct result
of these academy efforts.
Marketing logos, templates, - curriculum templates, new curricula related to science/STEM -Shared programs, curricula, etc. - Developed network of educators, support each other
Discussion about a 4-H STEM Specialist and this person's involvement on a campus-wideoutreach coordinators' committee.
As a result of participating in the National 4-H Science Academy, we have developed interestin increasing youth participation in science and agricultural science related careers. We have
also submitted a proposal to USDA/NIFA to build our extension staff capability to organize
youth in schools and communities with limited resources to develop interest in science. We
are utilizing the resources provided to us during the 4-H Science Academy to conduct effective
assessment of our science program
State-wide awareness of 4-H Science Mission Mandate Discussion on how to better organize 4-H programs and develop 4-H Project info sheets. more marketing to promote and recognize science efforts Development of the Science Rich 4-H Project Handbooks Resources leveraged in new state 4-H Science resource library and 17 new state 4-H Science
learning kits, new 4-H Science webpage, new state 4-H Science volunteer recruitment survey,
greater Extension investment in teacher & staff training, development of state 4-H Science
Post-programming technical assistance & support to extend 4-H Science learning. Increase in trainings offered to staff and volunteers around 4-H science. Have completed 4-H
Science training with at least 30 staff and over 150 volunteers.
Bigger focus on evaluating programs and efforts. More use of Train the Trainer withvolunteers Larger awareness and efforts with fundraising
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
40/48
33
Part Five
Narrative Question Content Analysis
Participants were asked three narrative questions: 1) list three things they planned to do
related to 4-H Science in the next three months; 2) list the main thing related to 4-H science
that they need additional support with the most; and 3) list three ideas/needs for post-
academy training opportunities. A content analysis of the 1663 responses to these questions
was conducted using the qualitative analysis software MAXQDA. For each question, comments
were categorized into common themes. The categories for each question, as well as the
percent in which each category was mentioned among the comments, are listed below (see
Tables 5.0 5.2). Some comments contained two or more themes and so one comment could
have several categories attached to it. The most frequently mentioned themes for each
question are described in slightly more detail with examples of typical comments under these
themes.
Three Plans to Implement in the Next Three Months
After analyzing the 889 comments, 19 different themes were decided upon that represented
the responses related to plans for the next three months (see Table 5.0). The most common
responses related to the following themes:
Planning, leading, developing, or assisting a science program (16.3% of responses)This included work on urban science programs, geospatial programs, science Saturdays, science spin
clubs, science camps, filmmaking, after-school programming, Youth Science Research Exhibition,
food science, garden projects, and science programs in general.
Recruiting, training, or informing volunteers and staff on science (14.4% of responses)This included teaching regional, state, and local volunteers, sharing with staff what was learned at
academy, training volunteers and staff to incorporate science in programs and lessons, recruiting
volunteers to facilitate science programs, training afterschool staff and agents, encouraging state
teams to develop trainings for staff and volunteers, designing a volunteer training, and leading
workshops.
Inquiry based learning (12.7% of responses)This included learning more about inquiry based learning, training others on inquiry based learning,
adding inquiry based learning to existing programs, using the inquiry method with all 4-H projects,
offering support with the inquiry based approach, providing hands-on inquiry based opportunities,
revising trainings to meet inquiry based standards, and adjusting lesson plans to include an inquiry
based approach.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
41/48
34
Involving more science in existing programs (7.9% of responses)This included involving more science in camps, conferences, retreats, current projects (i.e. animals,
clothing, foods, shooting sports), workshops, judging events, afterschool programs, fairs, recognizing
science in all projects, and supporting volunteers in bringing out the science in their project.
Curriculum (7.4% of responses)This included developing 4-H science curriculum, using curriculum ideas to teach leaders, enhancing
curriculum, researching curricula, reviewing curricula to assure science model, adding inquiry based
approach to curriculum, informing others of curriculum to use, accessing websites for curriculum,
using the national curriculum development template, conducting webinars on 4-H Science
curriculum, and borrowing other states curriculum.
Table 5.0 Plans to Implement in the Next Three Months
PercentMentioned
Planning, leading, developing, or assisting science programs 16.3%
Recruiting, training, informing volunteers and staff on science 14.4%
Inquiry based learning 12.7%
Involving more science in existing programs 7.9%
Curriculum 7.4%
Securing partnerships 6.3%
Fund development (i.e. use fund development toolkit, identify opportunities, seek donors and
grants)
6.2%
State or county science plan of action (i.e. review, evaluate, or develop plan) 6.0%Evaluation of science programs (i.e. use evaluation tools, plan an evaluation, improve evaluation) 5.5%
Robotics (i.e. facilitate a robotics club, offer robotics workshops) 4.6%
Using webinars, on-line resources, and national website for information on science 2.6%
Networking with colleagues 2.2%
Obtaining and using science resources 1.8%
Promotion of science (i.e. make 4-H science more visible, include science in monthly newsletter,
promote science to youth)
1.6%
Obtaining more information on what was learned at academy 0.7%
Academy did not provide what was needed (i.e. no new information, too much lecture) 0.5%
Recruiting and training youth 0.4%Train others in Positive Youth Development 0.2%
Other (i.e. set goals, write articles for volunteers, survey needs in community, set dates for
meetings, write program descriptions)
2.7%
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
42/48
35
Response Themes for Additional Support Needs
After analyzing the 268 comments, 18 different themes were decided upon that represented
the responses related to additional support needed (see Table 5.1). The most common themes
are listed below. It is interesting to note that Fund Development and Evaluation are the areas
in which most additional support is requested. This is a marked change from the Year Oneevaluation and follow-up that indicated the greatest need was in the area of scientific inquiry,
and perhaps yet another indicator that the knowledge and skill needs for facilitating 4-H
Science are shifting.
Themes:
More funds and fund development (19.7% of responses)This included funds for programs, staff, curriculum, and supplies, funding on county and state levels,
funding opportunities and training, and finding donors.
Evaluation support and tools (11.2% of responses)This included evaluations to show impact to partners and donors, how to use existing evaluation
tools, ready-to-use standard evaluations, activities that include evaluations, webinars on evaluation,
and understanding evaluation at different levels of program development and delivery.
More volunteers, staff, and manpower (9.7% of responses)This included finding, recruiting, maintaining, and training volunteers, more staff to run programs,
promoting volunteer opportunities, staff with evaluation experience, and volunteers with
commitment.
Materials, resources, and equipment (9.1% of responses)This included new equipment and computers, access to science materials and 4-H science resources,
easy and quick science resources, storage space, promotional materials, volunteer training resources,
tool-kits, one-page science activities, and curriculum.
Training related to science inquiry and hands-on science activities (7.9% of responses)This included more in-depth understanding of inquiry learning, inquiry based curriculum examples,
hands-on learning programs to share, tools to help volunteer and staff use inquiry-based learning,
applying inquiry based learning to areas that do not appearto be science related, teaching
traditional leaders to incorporate inquiry into existing programs, and hands-on demonstrations.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
43/48
36
Table 5.1 Additional Support Needed
Percent
Mentioned
More funds and fund development 19.7%
Evaluation support and tools 11.2%
More volunteers, staff, and manpower 9.7%
Materials, resources, and equipment 9.1%
Training related to science inquiry and hands-on science 7.9%
Obtaining and developing curriculum and content (i.e. easy to use curriculum, how to use curriculum) 6.4%
Training volunteers and staff (i.e. state training for volunteers, training for staff to identify science in
programs)
5.5%
Time 4.8%
Institutional support, organization, and planning 4.2%
Utilizing science in already existing programs 3.3%
More information, workshops, or conferences on science 3.0%
Partnerships (i.e. establishing science partners, partnering with colleges and universities, adult-youth
partnerships)
2.7%
Promotion of science and programs 1.8%
Creating a plan of action 1.5%
Support of curriculum revision 1.5%
Access to materials at academy 0.9%
Applying what was learned at academy 0.9%
Other (i.e. update state website, knowledge on subject matter, consistency in implementation,
robotics)
5.8%
Response Themes for Three Ideas or Needs for Post-Academy Training Opportunities
After analyzing the 506 comments, 24 different themes were decided upon that represented
the responses related to additional training opportunities (see Table 5.2). The most common
responses related to the following themes:
Recruiting and training of volunteers and staff (10.5% of responses)This included recruiting more committed volunteers, training volunteers, staff, and state leaders in
science, training on science needs, state and local training, face-to-face trainings, and training thatincludes a shadowing component.
Curriculum (9.8% of responses)This included training on national 4-H curriculum, curriculum that applies to different ages, easy-to-
use curriculum suggestions, adapting curriculum to inquiry-based, developing curriculum, hands-on
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
44/48
37
training using curriculum, how to use curriculum, online trainings for curriculum, and a place to
receive feedback on curriculum.
Sharing capabilities for resources, information, and success stories (9.4% of responses)This included resource site with links to other science 4-H web sites, blogs or eNewsletters that help
with sharing best practices, online community of academy attendees, regional and statewide sharing
of ideas, weekly tips on science learning, sharing of states marketing tools, games, and skills, website
listing key resources covered, contacts from academy, and shared materials.
Fund development (8.3% of responses)This included finding potential donors, funding support from NSF and/or USDA, funding sources and
opportunities, follow-up on fund toolkit, examples of a 4-H Science programs utilizing multiple types
of funding mechanisms, and training staff on fundraising.
Evaluation (7.9% of responses)This included how to use existing tools for evaluation, designing impact evaluations, evaluation
resources and strategies, how and when to evaluate programs, simple evaluations to go with
activities, and practical evaluation help.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
45/48
38
Table 5.2 Three Ideas or Needs for Post-Academy Training Opportunities
Percent
Mentioned
Recruiting and training of volunteers and staff 10.5%
Curriculum 9.8%
Sharing capabilities for resources, information, and success stories 9.4%
Fund development 8.3%
Evaluation 7.9%
Inquiry-based learning (i.e. hands on inquiry training, mentor in inquiry training, integrating inquiry
into programs)
7.2%
Partnerships (i.e. developing and recruiting) 7.0%
Promoting and marketing science and information learned 4.5%
Specific science programs (i.e. Robotics and Gear teach 21) 4.3%
Webinars on various topics (i.e. curriculum, volunteers, funding) 4.0%
Informal science, integrating science into existing projects 3.8%
Hands-on science training 2.8%
Developing and implementing state/county plans 2.1%
Grant writing and grant opportunities 1.9%
Specific topics (i.e. PYD, citizenship and science, youth leadership, team building, learning styles) 1.9%
Another academy 1.7%
Train-the-trainer 1.5%
Technology (i.e. use of ScienceHub in my4-H.org, social media training) 1.3%
Resources (i.e. obtaining, training, and implementing) 0.9%
Follow-up on progress made by programs after the academy 0.9%
Time 0.6%
Professional development 0.6%
Science connected with other mandates 0.6%
Other (i.e. tour science museums, more handouts, individual follow-up, national science
requirements)
6.6%
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
46/48
39
Appendix 1: Feedback from New England on Effectiveness of Virtual
Academy
Table 13.0 Northeast New England additional questions
Frequency %How effective was the
virtual format of the
academy?
Not effective at all
Somewhat effective
Effective
Very effective
Missing
5
13
4
1
7
16.7
43.3
13.3
3.3
23.3
Do you think virtual
training is as effective as
face to face training?
Yes
No
Missing
7
15
8
23.3
50.0
26.7
Which type of training do
you prefer?
Virtual
Face to face, on site
No Preference
Missing
2
16
5
7
6.7
53.3
16.7
23.3
Why do you prefer virtual
training?
Reduced time away from office
Reduced Travel costs
More effective use of my time
Easier to focus on content that is presented
2
2
2
2
100.0
100.0
100.0
100.0
Why do you prefer face to
face training?More effective use of my time
Easier to focus on content that is presented
I learn more this way
Provides network opportunities
*Percentage was calculated as: the number who
responded/the number who said face to face
training was preferred
3
13
10
10
18.75*
81.25*
62.5*
62.5*
Open Ended Questions:
Comments about the Virtual Training format:
1. Test the technology FIRST. It was so frustrating to have presenters unfamiliar with the formatthey were using. There was no forethought about how to present to multiple locations. 2.
There was the ability to do hands on pieces- however this was not done. I think if the planning
team had spent more time thinking though ways to make this day more interactive the training
would have been much better.
8/22/2019 Year Two National 4-H Science Academy Final Report[1]
47/48
40
At times the slide(s) on the screen were too small to read! I think we could just have easily taken part in this training from our own desks. It was difficult to
sit for long periods listening, and reading - incorporating more time to interact with the website
tools would have been helpful. As a region and state we rarely have time for discussion and
planning.
It could have been more effective if it was better organized. When we broke up into groups thedirections were consistently vague and hard to follow.
It was awful, totally boring and in my opinion misrepresented. Lapsed periods of engagement due to impersonal nature of this method of delivery. Too many
technical difficulties leading to distraction.
Not what I expected - way, way too much sitting and getting lectured to. Don't need someoneto read a handout to me.
Some of the presentations were helpful; others had technical problems that limitedeffectiveness. e.g. some slides had too much content, making them hard to read. Time was
wasted, even in effective presentations, when presenters had trouble loading their power points
or websites. URLs were not clearly displayed. Small group activities lacked sufficient direction
and structure.
The "hybrid" model of listening to a presenter and then spending time working with ordiscussing the information in our room was very effective. The slow pace of a webinar and the
slow conversation make is somewhat painful to sit through, but the savings in travel time is
worth it. Also, presenters could present less information and give people more time to explore
what they want to/need to know about that info.
The power points got tiresome after a while and the day could have been better organized withonsite personnel beforehand.
The technology was difficult, but in lieu of long distance travel, I thought it was OK. I think thehost could have had us do more experiential activities in between webinars
Too long Need active activities/break outs. Several things would have led well to engage in role-play like
approac