Staff Flu CampaignEvaluation Tool
Why evaluate your campaign?Evaluation is a vital element of your flu campaign. Well planned evaluation helps identify what worked and what didn’t work so well. This will help you identify good practice to share, and also inform planning for a stronger campaign next year. Sometimes, one small change can have a big impact and evaluation can help identify that change and the size of the impact.Evaluation of your local flu campaign should aim to: • Gather information from your team - this demonstrates
you value their opinion, and can provide a valuable insight • Identify key stakeholders• Gather information from key stakeholders - this should
include patients/service users and can be enlightening• I dentify and highlight good work that has been carried out
by your team(s) • Identify interventions that seem to have been successful
and also consider if they can be improved • Identify interventions that were less successful and
consider if they can be improved• Identify awareness raising resources, and consider if
they have delivered the messages as planned to the target group
• Identify opportunities to strengthen the campaign going forward
• Inform planning • Estimate resources needed for a strong campaign
next season
The information may be gathered in a variety of ways. A campaign assessment checklist may help when collecting this information (appendix 1).The information you gather as part of your evaluation can be useful in planning your next campaign and also to inform business cases if required.
Reviewing and evaluating your health and social care worker flu campaignThe aim of this guide is to help you reflect and evaluate this year’s health and/or social care worker influenza vaccination campaign, to identify and address any areas that may require further work and also to identify areas of success. This reflection can help in planning future campaigns.
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu CampaignEvaluation Tool
Getting startedConsider:• What you want to measure• What/when/how are you going to measure• Who you are going to ask to contribute• How you will share your findings
Some suggestions of potential outputs/outcomes would be:• Vaccine uptake by clinical area, directorate and professional
group• Number of and percentage of front line
staff protected vs unprotected • Vaccine uptake in areas with a
peer vaccinator vs without one
Some questions that may help you identify valuable detail behind the figures might include: • Have we done anything different this year? • If yes: • What did we do this year that was different to previous
years? • Did this appear to have any effect? • If yes, what effects have been identified (positive and
negative)? • If no; why not? • What feedback have you had from your staff? • What are the main points arising? • What feedback have you had from service users/
patients? • What are the main points arising?
A campaign assessment checklist may help when collecting this information (appendix 1). Examples of staff questionnaires to help gather information about staff opinion are included in appendix 2 and appendix 3. How you gather this information is also a consideration. You could use a traditional questionnaire (appendix 3), hold focus groups, or be more innovative and creative with gathering information (appendix 4). Displaying a staff opinion dartboard (appendix 4) in areas where staff congregate such as the canteen, staff room or staff changing facilities is another way to collect staff opinion.
What to do with the information you collectThe information collected should be analysed and reflected on. It will become the building blocks for the following year’s campaign and to develop areas that may require investment.Separate your findings into individual departments, directorates, staff groups and/or sites as this will help identify differences and may direct you to areas of good practice as well as help highlight areas where more support may be needed next season.
It can be helpful to use this information to encourage competition between teams, staff groups, and departments within your organisation and also to benchmark against other similar organisations. You can learn from those with higher vaccine uptake rates, identifying and sharing good practice. Another way to sort your findings is into themes, as some issues may be organisation wide.
Using your findingsUse the information you have gathered in this evaluation to inform your end of flu season report. Remember to include positive and negative, qualitative and quantitative data and reflect on how this can influence your recommendations on planning a stronger campaign.
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
Initial Questions
Flu vaccine uptake % in all staff
Flu vaccine uptake % in front line* staff
Are any staff groups lower than others? Do we know why?
Is the policy to vaccinate all staff or front line staff only
How was the number of front line staff calculated?
Date planning the campaign started
Do you have a documented flu plan?
Does the organisation have an aspirational vaccine uptake target? If so - what is it?
Was the planning robust and complete?
How was the flu vaccine promoted to staff?Email, social media etc
Were communications colleagues included in sharing messages?
Comments...
Appendix 1. A campaign assessment tool
* A definition of front line staff can be found in The Green Book (Immunisation against infectious disease) Chapter 19 influenza.https://www.gov.uk/government/uploads/ system/uploads/attachment_data/file/456568/2904394_Green_Book_Chapter_19_v10_0.pdf
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
Who was involved in planning the campaign?
Staff-side representation
Occupational health
Communications
Volunteers within the organisation
Estates and facilities
Board-level representation
Human resources
Infection control
Peer vaccinators
IT
Other
Comments...
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
How are challenges and successes identified?
Via staff?
Through the flu team?
At the Board?
Using evaluation forms?
Using social media polls?
Other
Comments...
Tackling the challenges
What challenges have been identified?
What are we doing, as a team, about these challenges?
Comments...
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
Acknowledging and sharing successes
What successes were identified?
How were successes celebrated and/or shared?
Comments...
Incentives
Was there an incentive scheme?
If yes - describe
If yes - how was impact measured and how is this being evaluated?
Would you recommend an incentive scheme for next year? (describe)
Comments...
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
How was vaccination provided for staff?
Arranged clinics in OH department
Arranged OH clinics in other venues
Drop in OH clinics
Roving clinics
Peer vaccinators
On request
Through line managers and at key gatherings (such as mandatory training)
Other (describe)
Comments...
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
2015/16 2016/17 Action 2017/18 campaign
If you have peer vaccinators...
Who are the peer vaccinators?
How is impact measured and did they have an impact?
Did staff know who the peer vaccinators were?
If not, how could they be identified next year?
Comments...
In Summary...
What went well?
What didn’t go so well?
What lessons were learnt?
What will be done differently next season?
Comments...
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
Appendix 2. Staff Flu Campaign Template Questionnaire
1. Did you have a flu vaccination this year? Yes/No
2. If yes, why? (tick all that apply). If no - go to question 4.
To protect myself from getting flu
To protect against spreading flu
To protect my family from getting flu
To protect my patients from getting flu
I get the vaccination every year - I don’t know why
I felt compelled
Other (please describe)
3. Where did you receive your flu vaccination? (tick one only)
By a peer vaccinator
In department/office/ward
In occupational health department by appointment
In occupational health department at a ‘drop in’ clinic
In a staff area (canteen etc)
At my GP surgery
At a community pharmacy
Other (please describe)
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
4. If you answered NO to question 1, why didn’t you have a flu vaccine this year? (tick all that apply)
I wasn’t offered a flu vaccination
I didn’t know I was entitled to it
I didn’t know how to get it
I was offered it but the time/date was inconvenient
I think I might be allergic to one of its components
I didn’t want to experience side effects from the vaccine
I don’t believe the vaccine is effective
I don’t believe the vaccine is safe
Other (please describe)
5. How likely are you to have a flu vaccination next year? (tick one only)
I will definitely have it
I will probably have it
I might have it
I probably won’t have it
I definitely won’t have it
Undecided
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
6. How did you hear about the flu vaccination this year?
Colleague in my department/team
Occupational Health Department
News item on the intranet (NHS Wales)
Information on the internet including social media such as facebook, twitter etc
I didn’t – this questionnaire is the first I’ve heard of it
Other (please describe)
7. What would you do differently to increase flu vaccine uptake among your colleagues/staff?
Comment
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
Appendix 3. Brief staff questionnaire
1. Have you had or are you going to have your flu vaccination? Yes No Not sure
2. Please tell us why you made this decision?
3. If you are not going to have your vaccination, or are not sure, what would encourage you to have it?
NHS staff are key to patient safety and we would like to ask you about the flu vaccination
www.beatflu.org f T© Public Health Wales NHS Trust 2017 / Iechyd Cyhoeddus Cymru 2017 This resource has been modified from an original resource developed by NHS flu fighter, a leading workforce campaign delivered by NHS Employers
Staff Flu Campaign
Vaccine availability
Flu campaign communications
Clinic location
Peer vaccinator knowledge
Missed the mark
Met expectations
Exceeded expectations
PHW
030
04.
17
Appendix 4. Staff opinion dartboard tool
For a quick and visual way for staff to give feedback you could use the dartboard tool.
On a piece of flipchart paper create the diagram below and display it in areas with high staff footfall like the canteen. The heading can be changed to gather information to meet your evaluation aims.
Using a pen or small sticky dots, staff can place a mark within each of the four areas of the dart board. The closer the mark is to the centre, the better they rate that aspect of the campaign.
It might be an interesting exercise to place several of these dartboards in different departments to gather a range of information. This feedback can be used to help tailor next season’s campaign.