Post on 18-Apr-2018
transcript
Evaluation of the facilitator development programmeJanuary 2013
we help develop the skills of social workers
Table of Contents
Executive Summary 2
1. Introduction 5
2. Approach 5
2.1. Aim 5
2.2. Methods 5
2.3. Response 5
2.4. Analysis of responses 6
3. Key findings 7
3.1. Expectations of the course 7
3.2. Impact on skills 8
3.3. Impact on personal practice 12
3.4. Impact on confidence to support NQSWs 17
3.5. Impact on confidence to support colleagues 21
3.6. Range of techniques 23
3.7. Organisational culture 26
3.8. Overall impression of the course 27
3.9. Improvements to the programme 28
4. Conclusions 29
Evaluation of the Facilitator Development Programme, 2013
Published by Skills for Care, West Gate, 6 Grace Street, Leeds LS1 2RP www.skillsforcare.org.uk © Skills for Care 2013 Reference no. R050 Copies of this work may be made for non-commercial distribution to aid social care workforce development. Any other copying requires the permission of Skills for Care. Skills for Care is the employer-led strategic body for workforce development in social care for adults in England. It is part of the sector skills council, Skills for Care and Development. This work was researched and compiled by Liz Burtney of Skills for Care Bibliographic reference data for Harvard-style author/date referencing system: Short reference: Skills for Care [or SfC] 2013 Long reference: Skills for Care, Evaluation of the Facilitator Development Programme, 2013, www.skillsforcare.org.uk
2
Executive Summary
Introduction
Following the successful delivery of the Facilitator Development Programme to support NQSWs
in 2010/11, Inter-Logics was asked to repeat the course in 2011/12. During 2011/12 six courses
were delivered in total, each lasting four days. The courses were offered in London (2 courses),
Birmingham, Leeds, Manchester and Truro between October 2011 and March 2012. While 90
participants started the programme, 75 completed the workshops and assessment were
involved in the Facilitator Development Programme (FDP), which offered a combination of
related theory with practical facilitation skills and methods. This summary outlines the key
findings from the evaluation.
Methods
There were two main elements to the research: qualitative telephone interviews and three
waves of questionnaires; pre FDP, immediately post FDP and 4-6 months after completion of
the course. In total 12 participants were interviewed (hereafter referred to as interviewees) and
147 responses to the three waves of the survey were gathered (Wave 1: 51 responses; Wave 2:
51 responses; Wave 3: 47 responses). The responses to the three waves of the survey were
not matched.
Findings
Interviewees pointed out that they were already a highly skilled workforce as reflected in their
current job role, predominantly that of learning and development. Indeed some described
themselves as ‘specialists’ and therefore making it difficult for learning and development of any
kind to have a significant impact. The same is true for respondents to the questionnaires
(hereafter referred to as respondents) who where a highly skilled workforce.
Nonetheless, many interviewees felt that the training had reinforced existing skills and
knowledge and provided an opportunity to reflect and consolidate. For example, there was a
reported increase in skills perceived by respondents pre and post FDP, a finding substantiated
by the qualitative work.
“It’s made me think more about listening to people, valuing their ability to sort
problems out … It has had a real impact – it was good to focus on listening and
questioning."
Impact of the FDP
When asked directly, the majority of respondents to wave 2 and 3 of the survey indicated that
the programme had a direct impact on their understanding of the methods used in action
learning, their range of tools used to support others and facilitation skills with small groups.
Around a third felt the programme had impacted on their skills to deliver action learning
processes, ability to help others identify learning needs, facilitation skills with larger groups and
critical thinking skills. This reflected the expectations set out by respondents at the outset of the
course.
3
Impact on skills
Findings from the survey indicated the course had a significant impact on the following skills:
Critical thinking
Facilitation of both smaller and larger groups
Critical evaluation of own development.
The qualitative study reflected these findings and in addition found that the FDP had also made
an impact on skills associated with broader learning and development. Interviewees felt they
now had more techniques on which to draw depending on the situation, and felt they had the
skills to change from one technique to another to reflect both the context and the audience.
Impact on individual practice
From the qualitative study, interviewees described how the FDP had enabled them to use the
skills and techniques from the course and integrate them into everyday practice beyond action
learning groups per se. Examples given included the change of approach to facilitating and
chairing meetings as skills in devolving responsibility for the direction of discussions and
meetings to those present rather than feeling the need to direct. Others described how the FDP
had enabled them to improve relationships with supervisees through more constructive
feedback based on improved listening skills and the use of specific techniques to provide
support. A number of comments around the use of techniques to improve reflective practice
were made and interviewees were keen to point out the value of taking time to think about
workload planning, approaches to challenges and time pressures.
Supporting NQSWs
Respondents to the quantitative questionnaire were asked about past and current experience in
supervising NQSWs. The FDP had a significant impact on levels of confidence across a number
of areas pre and post course including:
Support the development of a community that shares practice
Help NQSWs develop skills for reflecting on their practice
Help NQSWs to understand and develop emotional resilience
Develop NQSWs critical thinking skills
Help NQSWs identify their own leaning needs
Help NQSWs develop better listening and questioning skills.
The FDP appeared to have a significant impact on the confidence of respondents across all of
the areas listed above. There was no significant impact on confidence levels associated with
facilitating groups of NQSWs. However, it is worth noting that for some there was a slight dip
between wave 2 (immediately following the survey) and wave 3 (4-6 month follow up).
Supporting colleagues
All respondents to the quantitative surveys were asked to reflect on their confidence levels
before and after the FDP across a the same areas and there was a significant improvement in:
Supporting the development of a community that shares practice
Helping others develop skills for reflecting on their practice
Helping others identify their own leaning needs
4
Helping others develop better listening and questioning skills.
Again the sample is small for this study so findings should be treated with caution but will
provide indicative results and trends for the direction of confidence levels among participants.
Findings from the qualitative study would support this improvement in confidence across
motivation and willingness to use action learning and other techniques, to be flexible and
respond to different situations, to help work with groups more effectively and to allow groups to
come to their own solution.
Use of techniques
Interviewees were starting to try out new techniques with supervisees and colleagues. The
majority of interviewees also felt that the programme had a significant impact on their
confidence levels associated with both understanding of the action learning approach, and the
range of techniques explored though the programme.
Conclusions
Findings from the survey and the small-scale qualitative study indicate the FDP has made an
impact on a number of key areas including perceived skills in facilitation, critical thinking and
evaluation. In addition, respondents report significant increases in confidence levels associate
with supporting NQSWs and colleagues. The course also prompted changes in both individual
practice, and in some instances, organisational practices.
5
1. Introduction
Following the successful delivery of the Facilitator Development Programme (FDP) to support
NQSWs in 2010/11, Inter-Logics was asked to repeat the course in 2011/12 but on a reduced
scale reflecting available budgets. Each course was conducted over four days between October
2011 and March 2012 in six separate venues1 across England with a total of 75 participants
completing the workshops and assessment2. The course offered a combination of related theory
with practical facilitation skills and methods. More information on the detail of the course can be
found in Boydell and Abbott (2012)3. This report forms the basis of the evaluation of the work.
2. Approach
2.1. Aim
The aim of the evaluation was to determine the impact of the facilitator development programme
on participants’ knowledge, skills and subsequent practice.
2.2. Methods
Participants were approached at the outset of the learning and introduced to the evaluation of
the work. There were two methods used in the evalaution: online survey and depth telephone
interviews.
Quantitative surveys
Participants were asked to complete a questionnaire at three separate points:
1. Prior to the programme commencing
2. Immediately upon finishing the programme
3. 4-6 months after programme completion.
Respondents had the option to complete the survey either online via surveymonkey, submit an
emailed copy or request a paper-based version of the questionnaire which was subsequently
transferred into survey monkey. In the Phase 3 questionnaire, participants were asked to
indicate their willingness to take part in a qualitative interview to complement the quantitative
work and hear more about the impact of the course back in the workplace.
Qualitative interviews
As indicated above, respondents were asked if they would be willing to take part in subsequent
qualitative work. From the positive responses, a sample was drawn and approached initially by
email to confirm interest and establish a suitable time for interview. Interviews lasted on average
30 mins. The interviews were recorded, typed up and sent back to respondents for further
reflection and confirmation of accuracy.
2.3. Response
Quantitative surveys
In total, 147 (out of a possible 225) responses were received across the three cohorts:
1 Two in London, one in Birmingham, Leeds, Manchester and Truro.
2 In total, 90 started the programme, 79 completed the workshops, of which 75 completed the assessment.
3 Boydell T, Abbott C (2012) Training and development programme for action learning facilitators. Sheffield: Inter-
logics.
6
Wave 1: 51 respondents
Wave 2: 51 respondent
Wave 3: 45 respondents.
A profile of the sample can be found in appendix 1 but Table 1 below gives an indication of the
spread of responses across the six venues.
Table 1: Responses by cohort
Wave 1 Wave 2 Wave 3
London (1st cohort) 19% 15% 14%
Birmingham 25% 21% 25%
Leeds 6% 10% 20%
Manchester 28% 6% 14%
London (2nd cohort) 8% 21% 14%
Truro 13% 27% 14%
Base (n=) 47 48 44
Qualitative research
In total 12 participants agreed to take part in the qualitative telephone interviews. These
responses were spread across the six programmes and were conducted between 6-8 months
after the full and final completion of the programme.
2.4. Analysis of responses
Quantitative surveys
The responses from the survey were initially analysed in survey monkey and further testing
conducted in SPSS. Both parametric and non-parametric tests were used to identify areas of
significance between the three waves, depending the distribution of the responses. Where
findings are reported as significant, we are 95% confident that the reported difference between
the waves of the survey responses are not linked to chance.
Qualitative interviews
Using thematic analysis, the key points from the interviews were drawn out and are presented
below according to the main themes identified.
7
3. Key findings
3.1. Expectations of the course
Wave 1: Prior to commencing the programme, participants were asked through an open-ended
question about their expectations of the programme and what they would hope to be better at
following the FDP. The most popular responses included:
A working knowledge of the methods employed in action learning
A deeper understanding of action learning
An improvement in facilitation skills with both small and larger groups
Better skills to help support other people.
Table 2 below indicates the reflection of the responses from waves 2 and 3 in terms of what the
programme actually delivered that translated into greater skills and understanding.
Respondents were asked to choose up to three areas where they perceived the programme
had made a difference.
Table 2: Impact of FDP on perceived skills
Wave 2 Wave 3
Greater understanding of the methods used in action learning 79% 81%
Broader range of tools to use when supporting others 66% 86%
Improved facilitation skills with small groups 60% 62%
Improved planning skills to deliver active learning processes 40% 40%
Greater skills to help others identify learning needs 38% 29%
Improved facilitation skills with large groups 34% 36%
Improved critical thinking skills 34% 31%
Enhanced skills to provide constructive feedback to others 30% 29%
Other learning outcomes 8% 2%
It would appear that the course was indeed successful in improving understanding of action
learning and associated techniques, facilitation skills, albeit it to a greater extent skills
associated with facilitating smaller groups, and tools to help support others. Table 3 sets out the
intention and reality of changes to practice when back at the workplace indicated by open-
ended responses to the survey.
Table 3: Intended and actual changes to workplace practice following FDP
Wave 2 (intended) Wave 3 (actual)
Planned/actual changes in questioning and
listening skills
13% 9%
Planned/actual change in support and
supervision
6% 20%
Planned/actual change in facilitation approach 1% 14%
Planned/actual set up of action learning set 24% 18%
Planned/use of action learning techniques 41% 28%
Planned/promotion of action learning among
colleagues
16% 10%
8
3.2. Impact on skills
Respondents to the surveys and interviewees involved in qualitative study (hereafter referred to
as respondents and interviewees respectively) were asked to reflect on their skills in a number
of areas that were covered as part of the Facilitator Development Programme (FDP). It is worth
noting that respondents were already highly skilled professionals, with a background in learning
and development (e.g. practice educator, learning and development manager). Again the
majority of interviewees came from a learning and development background, involved in either
developing learning in the organisation or practice education. As a result, this was a difficult
cohort to develop in terms of skills given their background in this work they could be described
as ‘specialist’ in this area. This was in part reflected in the comments about the programme.
Indeed one interviewee did feel that the course didn’t have a great deal of impact on current
skills but reflected that this was an area of their specialism so would be difficult to have a big
impact. Nonetheless, many interviewees felt that the programme had reinforced existing skills
and knowledge and provided an opportunity to reflect and consolidate.
“There was a sense that the skills are already there…but sometimes the
confidence to use them or to work slightly differently isn’t there so the course was
good for that.”
“It did give the opportunity to reflect on skills and build confidence in using the
skills I already have.”
“…not sure it’s made a difference to how I facilitate but it reinforces learning that I
have already.”
One interviewee spoke of how she had previously been involved in action learning but this
course gave her a good opportunity to update her skills in this area.
“Having been on a course a few years back, this course seemed to introduce a lot
more techniques, flexibility in approach and a more relaxed introduction to setting
up action learning.”
Questioning and listening skills
Respondents were asked to reflect on their questioning and listening skills, before the course,
immediately after and again 4-6 months after the course finished. From Chart 1 below it is clear
that self-perception of skills around asking questions appeared to dip upon completion of the
programme although it did appear to increase at the follow up stage. While the shift was not
statistically significant, the pattern is interesting to note. It would appear that respondents felt
less confident after the course, than before, indicating some critical appraisal of their skills in
this area as clarified through the qualitative study.
For listening skills, there was a slight increase in self-perception of skills following the FDP,
although this was not statistically significant.
9
Chart 1: Questioning skills (where 1=no skills in questioning up to 10=very high level
questioning skills4)
The findings from the qualitative study would indicate that questioning and listening skills were
indeed areas where interviewees felt they had existing skills but the FDP had a marked impact
on their perception and subsequent ability. In particular, interviewees reflected on how the
programme had given a structure around which to frame questions and reminded them of the
importance of listening to others’ perspectives.
“It’s made me think more about listening to people, valuing their ability to sort
problems out … It has had a real impact – it was good to focus on listening and
questioning."
“I’ve done coaching and mentoring questions but I think what I find most
challenging is to think about how to ask questions in the right way rather than
give advice or answers to problems. Looking at ’Constellations’ and ‘Story
Telling’ methods helps me not to present my solution.”
“It really hones your questioning and listening skills because you really have to
listen to people and the problem. You really have to think long and hard about
the kinds of questions you think need to be asked …that was the best bit of the
training.”
Facilitation skills
When asked about facilitating groups, respondents reported a significant improvement in skills
for facilitating both small and larger groups. Chart 2 and chart 3 below show the shift between
Wave 1 of the questionnaire before the FDP and again after the course.
4 For this and following questions respondents were given a scale of 1-10 to indicate their relative position on the
scale
10
Chart 2 Skills in facilitating small groups (where 1=no skills up to 10=very high level
skills)
Chart 3 Skills in facilitating larger groups (where 1=no skills up to 10=very high level
skills)
The impact of the course on facilitation skills was also noted in the qualitative research. A
number of interviewees commented on the change in approach to facilitation and supervision
with NQSWs.
“Definitely skills around facilitating groups and approaches to supervision have
been improved since attendance at the training.”
“The trainers did work that made me think of how to take a more relaxed
approach to working with a group.”
11
Critical thinking and reflective practice
Respondents were asked to think about their skills in both critical thinking and reflective
practice. Findings indicate that there was a significant shift in critical thinking skills that were
perceived to be improved following the programme as indicated in chart 4 below.
Chart 4 Skills in critical thinking (where 1=no skills up to 10=very high level skills)
While not significant, there was a shift in respondents perceived skills associated with reflective
practice with fewer rating their skills around 5 or 6 and more likely to rate as 9 or 10.
These findings were supported by the qualitative study. While only mentioned by a small
number of interviewees, there was discussion of the impact the course had on reflective
practice and critical thinking.
I have the skills now to work with them [NQSWs] to reflect on practice and think
about critical analysis
Critical evaluation of own development
There was a significant shift in skills associated with critical evaluation of one’s own
development noted by the respondents to the quantitative survey. Respondents were more
likely to rate their skills in this area as higher following the course. Chart 5 below sets out
responses across the three waves of the survey.
12
Chart 5: Skills in critical evaluation of own development (where 1=no skills up to 10=very
high level skills)
Summary
In summary, the FDP course would appear to have had a lasting impact on self perceived skills,
particularly in consolidating existing learning, improving listening and questioning skills,
reflective practice, facilitation and critical thinking skills.
3.3. Impact on personal practice
When asked about the impact of the course on their practice, interviewees were able to
describe changes in the use of techniques and approaches in the organisation. One interviewee
in the qualitative study did feel that the learning had made no impact on practice; this was linked
to the broad experience already possessed and the collaborative approach already taken in
existing practice. All of the other interviewees were able to describe in some detail the impact
the learning had made on both personal and organisational practice.
Use of approach with others
In addition to employing action learning, interviewees in the qualitative study shared how they
had also introduced some of the approaches and skills developed as a result of the programme
into existing learning and development courses and meetings. For example, some described
how they had taken a new approach to chairing with a definite move to devolve responsibility of
the meeting onto all participants.
“This has come as a direct result of the training….about empowering a group
and helping them find their own path through the learning but being there for
them if they need support…I’m not there to facilitate the group but will support
them.”
One interviewee felt the improved listening skills had led to a better relationship on which to
base constructive feedback and critical analysis with supervisees. Others described the use of
specific techniques, e.g. “Learning from Success” into existing packages.
13
“Have been able to apply the techniques to team meetings, individual
supervision and mentoring and personal reflection.”
Another interviewee described her experience of the “Constellation” approach.
“It really worked because they were a visual group of people and it was a really
good basis for the discussion that we had afterwards.”
Others described how they felt able to take a step back and make changes to their
practice.
“So if something doesn't work or the skills mix or management mix isn’t right for
the approach I had first thought of, then I will change and reflect this back to the
group.”
Approach to work
Interviewees in the qualitative study described a number of ways in which the learning had
impacted on approaches to personal aspects of work. These included attention to workload
planning and prioritisation. One interviewee felt the learning had ‘given permission’ to think
about workloads and how best to organise on a day-to-day basis.
“I’m more effective about how my work is set out and planned and how people
can be more involved.”
“The course served as a really good reminder of the importance of planning and
reflection. Sometimes this is really difficult when you’re under pressure at work,
but taking the time out to think about how you are going to approach something
or what worked well on reflection is really important.”
In particular interviewees were able to set this within the context of reflective practice. The
course seemed to give a number of interviewees to take a step back, resist fire fighting and
think about the bigger picture and how to approach things in a different way.
“I think it means that you have to take a step back as sometimes we can get
bogged down with day to day things and by taking a step back and doing it in a
different way can be really helpful.”
Another described how the course had made them think about how they structure meetings and
supervision sessions. Finally, the impact on facilitating meetings was noted and the subsequent
positive impact on outcomes from meetings.
Implementing action learning groups
When presented with a question to reflect on their skills when setting clear ground rules for an
action learning group there was a clear and significant improvement in perceived ability to do
this noted in wave 2 and 3 compared to wave 1 (pre FDP). Immediately after the course
14
respondents were more confident in their own ability to set clear ground rules but this did dip off
at wave 3.
Chart 6: “I find it difficult to set clear ground rules for action learning groups” (where
1=strongly disagree through to 10=strongly agree)
Findings from the qualitiatve work perhaps throw some light on this finding as a number of
interviewees did comment that while they believed in the potential of the approach, they had
some difficulty identifying an opportunity to use action learning in their workplace. Indeed this
was a frustration expressed by a few of those interviewed. Having completed the learning, there
was a lack of opportunity to put the learning into practice. However, the majority of interviewees
had either introduced or planned to introduce action learning into their approach to work. Those
who hadn’t managed to do so at the time of interview had plans to as soon as possible.
“I am convinced of the potential of the approach both in terms of building
internal capacity but improving how people work together and learn. I hope to
get going with the approach once things have settled down internally.”
Others had been successful in identifying an opportunity to implement action learning in their
organisation with different groups within the organisation, for example, trainers, NQSWs.
“I have used the learning in train the trainers….we used the action learning set
technique and got a group to form questions for the person who had the
problem….that was useful.”
One participant reported back that they had positive responses from colleagues.
“It’s early days but people seem to value the approach and seem to be getting
something they haven’t had from any other route.”
15
Some participants had formed a post programme action learning set of their own to enable them
to keep involved in action learning, refresh knowledge of techniques and to support others in the
group use the learning.
“The aim is to keep us going with the [action learning] approach and we can try
out different methods and techniques in order to see how they work and how we
can improve.”
The value of this follow on support was echoed by other interviewees as a way of keeping in
touch with colleagues, and ensuring the programme remains “live and fresh”.
Cascading knowledge
From the qualitative study it was clear that a small number of interviewees had also begun to
cascade their knowledge of action learning to colleagues and supervisees for example, through
team meetings. This included discussions about action learning itself as well as some of the
techniques that can be used.
“I have been able to share the learning…our next team meeting is all about
leading a discussion in an action learning set way … so yes, actively sharing
the learning with others.”
Other changes in behaviour
The quantitative study specifically asked about other behaviours as reflected in the statements
below:
“I find it hard not to give direct advice”
“I am able to support participants to give each other feedback on issues”
“I am good at modelling what is expected as an action learning participant” (significant
improvement)
“I often forget to celebrate supervisees’ positive work”
“I enable participants to reflect on work issue and act on outcomes”
“Empathising with NQSWs is not my strong point”
“I am good a facilitating a supportive but challenging environment for development”
While no further significant changes were noted between the three waves of the survey for the
statements above, there was a trend for improvement for two of the statements. Respondents
indicated that the FDP had improved their ability to empathise with newly qualified social
workers (NQSWs) and their ability to support colleagues give each other positive feedback on
issues (see charts 7 & 8).
16
Chart 7: “Empathising with NQSWs is not my strong point” (where 1=strongly disagree
through to 10=strongly agree)
Chart 8: “I am able to support participants to give each other feedback on issues”
(where 1=strongly disagree through to 10=strongly agree)
Summary
From the qualitative study, interviewees described how the FDP had enabled them to use the
skills and techniques from the course and integrate them into everyday practice beyond action
learning groups per se. Examples given included the change of approach to facilitating and
chairing meetings as skills in devolving responsibility for the direction of discussions and
meetings to those present rather than feeling the need to direct. Others described how the FDP
had enabled them to improve relationships with supervisees through more constructive
feedback based on improved listening skills and the use of specific techniques to support. A
number of comments around the use of techniques to improve reflective practice were made
and interviewees were keen to point out the value of taking time to think about workload
planning, approaches to challenges and time pressures. Finally, a number of interviewees had
been active in cascading the knowledge gained through the FDP.
17
Other behaviour changes were queried through the surveys and a significant improvement
noted between wave 1 and 2 in particular for respondents ability to set clear ground rules for
action learning groups. There was also a trend (although not significant) for improvements in
empathising with NQSWs and supporting colleagues give each other feedback on issues. No
significant differences were noted for the other five statements which focused on giving direct
advice, modelling expectations of an action learning participant, celebrating positive work,
encouragement of self reflection and facilitating a challenging environment.
3.4. Impact on confidence to support NQSWs
Across the waves, respondents were filtered according to whether or not they currently
supervise or have in the recent past supervised NQSWs. Those who had no experience were
filtered to another part of the questionnaires. The numbers are small so findings should be
treated with caution but there were some significant differences between the three waves
(Wave 1: 39 respondents; Wave 2: 38 respondents; Wave 3: 29 respondents).
Those with current or recent experience were asked to reflect on their confidence levels when
supporting NQSWs across a number of aspects including confidence to:
Support the development of a community that shares practice
Help NQSWs develop skills for reflecting on their practice
Help NQSWs to understand and develop emotional resilience
Develop NQSWs critical thinking skills
Help NQSWs identify their own leaning needs
Help NQSWs develop better listening and questioning skills
The FDP appeared to have a significant impact on the confidence of respondents across all of
the areas listed above and illustrated in the charts below. It did not appear to have a significant
impact on the confidence of facilitating groups of NQSWs.
Chart 9: Confidence to support a community that shares practice (where 1 = not at all
confident up to 10 = very confident)
18
It is worth pointing out that while confidence has improved since wave 1, there appears to be a
slight dip after the initial post-programme period. This might be linked to the boost that learning
and development of this nature might bring, or the lack of subsequent experience of putting the
skills and techniques into practice.
Chart 10: Confidence to help NQSWs develop reflective practice skills (where 1 = not at
all confident up to 10 = very confident)
Chart 11: Confidence to help NQSWs understand and develop emotional resilience
(where 1 = not at all confident up to 10 = very confident)
As noted above, there does appear to be a dip in confidence with wave 3 respondents for both
these areas although some minor increases in those stating ‘very confident’ noted.
19
Chart 12: Confidence to help NQSWs develop critical thinking skills (where 1 = not at all
up to 10 = very confident)
Chart 13: Confidence to help NQSWs identify their own learning needs (where 1 = not at
all confident up to 10 = very confident)
As above, there appears to be some shift in confidence levels between wave 2 and 3. However,
with these two areas, there appears to be a move to being ‘very confident’. The same applies to
chart 14 below.
20
Chart 14: Confidence to help NQSWs develop better listening and questioning skills
(where 1 = not at all confident up to 10 = very confident)
This confidence and willingness to support NQSWs was reflected in the qualitative study. There
was a sense from a number of interviewees that attendance on the course had encouraged
them to work with supervisees and colleagues to encourage reflective practice. Others felt that
students they worked with would have a new understanding of working together and how you
can share the power in working relationships, challenge assumptions and discuss things more
openly.
“The students themselves have gone away recognising that they can reflect on
what they've done as they can discuss things more openly.”
In one instance impact on organisational practices was noted.
“…the head of service got to hear what NQSWs felt about the support they get
using World Café approach and there has been organisational change to reflect
this. Just the drawings were so powerful and the message was really taken up.”
Interviewees recognised the importance of building skills and capacity in others particularly in
the current economic climate.
“I have plans to work with this group to encourage reflective practice, working
together and importantly to build skills with social workers rather than to just
solve problems for them. This is much more sustainable and better for the
individual and organisation in the long term.”
In summary, respondents involved in NQSW supervision report significant changes in
confidence levels particularly around developing communities to share practice, reflection,
development of emotional resilience, questioning, listening and critical thinking skills in NQSWs,
helping NQSWs to identify learning needs. This increase in confidence was supported through
the qualitative study.
21
3.5. Impact on confidence to support colleagues
All respondents to the quantitative surveys were asked to reflect on their confidence levels
before and after the FDP across a the same areas and there was significant improvement in
confidence levels in:
Supporting the development of a community that shares practice
Helping others develop skills for reflecting on their practice
Helping others identify their own leaning needs
Helping others develop better listening and questioning skills.
However, there were no significant differences noted for the following:
Helping others to understand and develop emotional resilience
Developing others critical thinking skills.
Again the sample is small for this study so findings should be treated with caution but will
provide indicative results and trends for the direction of confidence levels among participants.
Chart 15: Confidence to support the development of a community that shares practice
(where 1 = not at all confident up to 10 = very confident)
As with NQSWs there appears to be a dip after reported in wave 3 with regard to confidence
levels in this area.
22
Chart 16: Confidence in helping others develop skills for reflecting on their practice
(where 1 = not at all confident up to 10 = very confident)
Chart 17: Confidence to help others identify their own learning needs
(where 1 = not at all confident up to 10 = very confident)
23
Chart 18: Confidence to help others develop better listening and questioning skills
(where 1 = not at all confident up to 10 = very confident)
There would appear to be some indication that more support is needed at least for some
respondents to boost confidence again between wave 2 and 3.
In summary, respondents reported improvements in confidence to support work colleagues,
significantly around supporting community to share practice, helping others develop reflection
skills, listening and questioning skills and to help others identify their own learning needs.
3.6. Range of techniques
From the quantitative study, it would appear that knowledge of techniques used in action
learning sets improved across the board following attendance at the FDP. For example, the
table below sets out the changes in knowledge associated with complexity mapping. Full details
of all techniques included can be found in appendix 2.
Table 3: Change in knowledge for the “Complexity Mapping” technique (where 1 is no
knowledge through to 10 which is a lot of knowledge of the technique)
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10
Wave 1 55% 16% 16% 12 0%
Wave 2 2% 6% 21% 55% 13%
Wave 3 5% 2% 31% 36% 16%
24
However, while knowledge had improved, respondents did not appear to have been using the
techniques to any great extent. The most popular techniques were “Complexity Mapping” and
“Learning from Success” and 10% and 9% of respondents respectively report having used the
technique at the follow up questionnaire (4-6 months after completion of programme). This
appears on the surface to contradict findings above which indicates the use of AL techniques by
respondents. However, findings from the qualitative study would indicate that respondents are
using the techniques but it is more the AL approach that has proven useful, for example,
questioning and listening skills, facilitation skills, general approach to supervision rather than the
specific techniques themselves which has had the biggest impact.
Through the qualitative interviews it was clear that course participants had been able to work
with the techniques explored on the course to varying degrees of success. There was
recognition that one of the skills taken away from the course was the ability to think about a
situation and choose the right method for that situation to maximise the outcomes.
“It’s good to know that a few techniques can be used in different places and the
ones I’ve used are easy to adapt for different situations.”
There had been some use of the “World Café” technique, “Complexity Mapping”, “Five Steps”,
“Gossip” method, “Constellation” method, “Storytelling” and “Learning from Success”.
Five Steps: Experience of this technique was described by one participant as useful in setting
out that there wasn't a right or wrong answer to problems. However, one of those interviewed
did express reservations about the method. In particular, the difficulty in managing the
technique so that it didn’t descend into a more aggressive questioning or advice giving session.
Complexity mapping: There was limited use of this approach and mixed experiences. One
respondent described how it was a useful and familiar approach and the one interviewee who
described using it felt it had limited applicability in their particular situation.
“This is really good as people are more familiar with this kind of approach so it’s
a good introduction and a good way of sorting out the themes that we need to
address”
Learning from Success: This technique had been used by a number of those interviewed and
the view was that this was a really positive approach to use which worked well for both the
individual telling the story but also in terms of extending learning to colleagues.
“There was a lot of learning from the session and there was learning for the
candidate but also the others who were taking part in the session.”
Constellation method: There was limited experience of this approach described however, where
it had been used, respondents were positive.
25
“I find this method useful and have used it the most. I’m most comfortable with
this method”
Story Telling: Again this had limited use and one respondent did describe some
difficulties with this approach.
“I struggle with this…sitting someone in the corner and hearing their story. I
struggle to make sure that the story teller is OK just by looking at their body
language”
Gossip method: Only one participant described using this approach and was
challenged initially.
“I was quite anxious for the first five minutes … this technique takes quite a bit
of management and through training learnt how to develop good and supportive
environment, how to let people out of the technique if they need to stop.”
Confidence to use new techniques
The qualitative study was able to probe interviewees on their views on confidence levels
associated with the techniques explored through the FDP. The majority of interviewees felt that
the programme had impacted on their levels of confidence, and in particular this was linked to
greater knowledge and understanding of the action learning approach associated theory.
“I have more confidence and motivation to go ahead and use action learning.”
In addition, there was an increased confidence and willingness to use the tools that had been
described and used on the course. In addition, interviewees reported increased confidence in
thinking about using different methods to different situations and learners.
“I have the confidence to now think about what to use and when to use different
methods.”
“….[I have] confidence to think about how you might use different techniques
and confidence to introduce into new situations.”
Regardless of previous knowledge, some felt that the programme had given them confidence to
apply techniques and facilitation skills where previously they would have struggled.
“I am now confident that I have got the skills in my toolkit to be able to facilitate a
conversation more effectively rather than brushing over things.”
“I have gained confidence in doing small group work and that has had an impact on the
people I work with.”
26
This in turn has had an impact on practice (discussed in more detail below) but some described
the confidence in themselves now to devolve responsibility of group meetings over to
participants rather than carry the burden.
“Have the confidence to devolve the power from one person and spreads the
responsibility of the meeting and outcomes among all participants.”
“I struggle to hold back and not share my personal experience and opinion but I
need them to come up with own solutions and thinking for themselves. This is
about asking the right questions.”
However, a minority did point out that while confidence had improved, the opportunities to
introduce and use new found skills and techniques were sometimes lacking and this was
problematic.
“One problem has been the lack of opportunity to use the techniques learned at
the training.”
In summary, interviewees were starting to try out new techniques with supervisees and
colleagues. The majority of interviewees also felt that the course had a significant impact on
their confidence levels associated with either understanding of the action learning approach,
and the range of techniques explored though the course.
3.7. Organisational culture
Following attendance at the course interviewees were able to articulate some of the benefits to
the approach in terms of having potential to change the culture within organisations to be more
reflective and supportive of staff. Others felt that the approach was a positive move as it allowed
organisations to learn from within rather than rely on outside ‘experts’ coming into the
organisation. Benefits were viewed both in terms of cost and relevance of learning but also
empowering the organisation and individuals. Finally, the flexibility of the approach was viewed
as a real strength as it could reflect different situations and individual needs.
“The approach helps with changing team culture and encourages more
discussion back at the workplace and greater support from within the team”
However, those interviewed recognised there had to be organisational support for implementing
the approach in order for it to make a real difference to the culture and in order to maximise the
benefits of the learning. There also needed to be buy in from colleagues.
“People need to have commitment to the process otherwise they won’t stay or
contribute if the they there isn’t any value.”
In addition, some struggled to identify opportunities to use the learning from the course and this
was seen as a real barrier to moving forward.
27
3.8. Overall impression of the course
All of those interviewed and open-ended comments taken from the wave 3 questionnaires were
positive about the approach and the programme undertaken describing it as ‘inspiring’,
‘motivational’ and ‘building confidence’.
“It was very good – very informative. I was a bit sceptical to start with but the
course was worth it’s weight in gold and that’s why I decided to take it further.”
It was a cost effective, flexible and good approach to learning and
development.”
“I found the course really stimulating and felt I learned a lot”
“I felt the training was very useful and it was clear to see how I could implement
it within my work with NQSWs and the social care workforce more widely”
Many described the new ideas and approaches to working as the learning while others felt it
had given some insight into the different ways in which people can learn and how to respond to
this. Indeed learning from each other.
“The course is really focused on people learning from each other and to really
understand how powerful that can be is a new thing for me”
Many felt their learning took place in a supported environment and they reported having good
fun, building relationships with others in their cohort and developed understanding of other ways
of working.
“Personally I felt it was really useful to meet people who have had similar
experiences. Not necessarily the same role or same difficulty but similar enough
to make a really good connection and to learn from each other.”
Interviewees were appreciative of the tension between time available and the scope of material
presented but most felt the pace of the course was about right and the supporting materials
useful back in the workplace.
Only one interviewee expressed some difficulties with the programme. They felt that while the
course had been a positive experience, it was more of an introduction to the approach and more
exposure and training was needed before the approach could be fully utilised within the
organisation5. In particular, there was a need for more experience in using different techniques,
more guidance needed in setting up a group, how to explain ground-rules, manage difficult
situations etc. While intrigued by the approach, there was not enough depth or practice.
“You can’t have a go once [using the technique] which is effectively what we
did, and be expected to be experts. That’s not enough depth or practice.”
5 Subsequently the programme has been altered to reflect this perspective
28
Overall there was praise for the course and for the trainers who delivered the
sessions.
“Really enjoyed meeting the trainers and felt their approach was refreshing and
enjoyable.”
“I found them [the facilitators] very supportive outside of the course as well so
they delivered above and beyond the actual course.”
“It felt that we were getting people who really knew their stuff and they were
really interesting characters. It felt like we were getting top drawer input and
that’s good for credibility of the training.”
3.9. Improvements to the programme
From the qualitative study one common theme to arise in response to this question was the
need for some sort of follow up. Whether this is a follow up training day, ongoing contact with
trainers or others within the cohort, the course definitely left participants wanting more6.
“It would be really useful to have a follow up day to keep you in touch with the
learning especially if you’d had less opportunity to use this back at work.”
Indeed a small number of those interviewed were keen to proceed onto the Level 5 course to
deepen their understanding and experience of the approach.
One interviewee felt that the guidance could give more detail on how to go about setting up an
action learning set or how to use the techniques. More of a ‘how to’ document to take you
through the process would have been useful especially for techniques that perhaps were more
complex or hadn’t been used as much on the course.
Finally a small number of participants felt the course could have been longer.
“Maybe longer days would have been beneficial so starting earlier and making
more use of the time the participants had together.”
Although one person felt it could have been condensed, perhaps reflecting previous experience
and styles of learning, a view that the trainers are considering.
“If I was being really critical I think we could have covered the material in 3 days
rather than 4. However we used different approaches and ways to get to know
each other as part of that and it was useful and I’ve used it again.”
6 Self supported managed action learning sets were encouraged in all locations except Truro however only in London
and Birmingham were these taken up.
29
4. Conclusions
The FDP course would appear to have had a lasting impact on self perceived skills, particularly
in consolidating existing learning, improving listening and questioning skills, reflective practice,
facilitation and critical thinking skills.
From the qualitative study, interviewees described how the FDP had enabled them to use the
skills and techniques from the course and integrate them into everyday practice beyond action
learning groups per se. Examples given included the change of approach to facilitating and
chairing meetings as skills in devolving responsibility for the direction of discussions and
meetings to those present rather than feeling the need to direct. Others described how the FDP
had enabled them to improve relationships with supervisees through more constructive
feedback based on improved listening skills and the use of specific techniques to support. A
number of comments around the use of techniques to improve reflective practice were made
and interviewees were keen to point out the value of taking time to think about workload
planning, approaches to challenges and time pressures. Finally, a number of interviewees had
been active in cascading the knowledge gained through the FDP.
Other behaviour changes were queried through the surveys and a significant improvement
noted between wave 1 and 2 in particular for respondents ability to set clear ground rules for
action learning groups. There was also a trend (although not significant) for improvements in
empathising with NQSWs and supporting colleagues give each other feedback on issues. No
significant differences were noted for the other five statements which focused on giving direct
advice, modelling expectations of an action learning participant, celebrating positive work,
encouragement of self reflection and facilitating a challenging environment.
Respondents involved in NQSW supervision report significant changes in confidence levels
particularly around developing communities to share practice, reflection, development of
emotional resilience, questioning, listening and critical thinking skills in NQSWs, helping
NQSWs to identify learning needs. This increase in confidence was supported through the
qualitative study.
Additionally, respondents reported improvements in confidence to support work colleagues,
significantly around supporting community to share practice, helping others develop reflection
skills, listening and questioning skills and to help others identify their own learning needs.
Findings from the qualitative study would indicate that participants had been able to work with
techniques explored through the programme. Some techniques appeared to be more popular,
e.g. “Complexity Mapping” but all agreed that the range of techniques was useful and offered a
flexible approach to different settings, situations and groups.
The findings would suggest that the course was well received, delivered to a high standard and
had an impact on skills, confidence and to some extent practice.
30
Appendix 1: Profile of sample
Wave 1 respondents gave details for the following variables:
Employment sector
o 83% worked in the statutory sector
o 10% worked in the voluntary sector
o 6% worked in the private sector
Organisational footprint
o 44% worked in adult services
o 19% worked in children’s services
o 37% worked in a role that covered both adult and children’s services
Job role (respondents were allowed to tick more than one box hence totals do not add
up to 100%)
o 29% were workforce development officers or managers
o 20% worked as a senior social worker
o 20% had a role as a practice educator
o 14% had a team leader or manager role
o 14% had a learning and development role
o 14% were social workers
o 10% worked as an independent practice educator
o 10% were advanced practitioners.
This is the only wave this level of detail was collected for as there were concerns over attrition
by repeating a number of questions for respondents with knowledge that the variables were not
going to be used in the analysis of data.
31
Appendix 2: Knowledge of different action learning techniques
Action learning Column1 Column2 Column3 Column4 Column5 Column6
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 29% 39% 10% 18% 2% 2%
Wave 2 0% 7% 26% 57% 11% 0%
Wave 3 4% 2% 16% 60% 11% 5%
Learning process
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 0% 22% 24% 48% 4% 2%
Wave 2 2% 2% 19% 60% 17% 0%
Wave 3 2% 4% 18% 37% 35% 2%
Complexity mapping
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 55% 16% 16% 12% 0% 2%
Wave 2 2% 6% 21% 55% 13% 2%
Wave 3 5% 2% 31% 36% 16% 10%
Five questions
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 76% 12% 10% 0% 0% 0%
Wave 2 4% 11% 36% 38% 9%
Wave 3 4% 7% 39% 35% 7% 7%
Gossip
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 85% 10% 6% 0% 0% 0%
Wave 2 4% 12% 25% 42% 14% 2%
Wave 3 7% 9% 35% 32% 11% 5%
Constellations
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 86% 8% 6% 0% 0% 0%
Wave 2 4% 8% 26% 51% 10% 0%
Wave 3 7% 7% 46% 23% 12% 5%
Story telling
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 70% 8% 18% 2% 0% 0%
Wave 2 4% 8% 21% 55% 12% 0%
Wave 3 4% 7% 42% 30% 9% 7%
Learning from success
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 76% 2% 17% 2% 0% 2%
32
Wave 2 4% 2% 24% 55% 13% 2%
Wave 3 5% 4% 35% 30% 16% 9%
Open space technology
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 81% 8% 12% 0% 0% 0%
Wave 2 2% 6% 35% 43% 13% 0%
Wave 3 9% 7% 44% 23% 14% 2%
World café
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 79% 6% 6% 4% 2% 4%
Wave 2 0% 4% 27% 52% 17% 0%
Wave 3 7% 5% 42% 23% 16% 7%
Talking stick
1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used
Wave 1 82% 10% 6% 2% 0% 0%
Wave 2 12% 4% 14% 49% 12% 6%
Wave 3 12% 14% 45% 14% 12% 2%