Coaching & Mentoring Stuart Burns, Archbishops’ Council August 2015 The Experiences of Ministry survey reports from 2011 and 2013 reflect the changing cultures of ministry development across the Church of England. Taken alongside wider learning (for example research in the Church of Wales, reflections from providers of ministerial development) it is of note that there is an increasing interest in and use of coaching approaches to ministry and professional development. On further investigation it is also clear that the term ‘coaching’ covers a wide variety of both interpretation and practice, and although the language is becoming more commonplace, the meaning and understanding of the terminology and practice varies. The Church of England CMD Panel recognised that a reflection that sought to bring clarity to this development would be beneficial. This paper is a brief contribution to the emerging coaching climate in the Church of England and Church in Wales, and will draw from three dioceses who have sought to integrate coaching into their ministry development (Leicester, Liverpool and Bangor), a 3 rd sector organisation who have been incorporating this approach, as well as providers of ministry training (CPAS, 3DCoaching). 1 Generally, coaching is requested or sought as a ministry development or learning intervention through a recognised presenting need. For example this can be an individual seeking help around clarity in vocation, someone wanting to talk through an issue or problem they face at work or in ministry, or a question of transition to a new role or job. These may have been initiated as a response to an organisational offer of coaching provision, or through an organisational directive (e.g. Ministerial Development Review follow up). Often during the first coaching conversation ‘clients’ will be unaware of the range of developmental or learning interventions on offer, or of the different approaches available. In all cases clarity of definition is crucial, and misunderstandings can occur when there is a mismatch in language and expectations. Tensions can also occur when a ministry development intervention is requested by a 3 rd party client. For example, a CMD officer being approached by an Archdeacon to ‘Do some coaching with X’. This is an area open to misunderstandings, (to say nothing of legislation) and potential disappointment for both coach, coachee, and the request initiator (client). For such requests clarity of both process and meaning would alleviate much of uncertainty. 1 Following Western, in this paper, I will use the term coaching to embrace the multiple and overlapping practices of coaching and mentoring, as coaching has become the dominant term. Mentoring will be named separately when a distinction is necessary. ‘Coachee’ will refer to the recipient of coaching and the term ‘client’ will refer to an organization or their representative sponsoring the coaching work.
Transcript
Coaching &
Mentoring
Stuart Burns, Archbishops’ Council August 2015 The Experiences of Ministry survey reports from 2011 and 2013 reflect the changing cultures of
ministry development across the Church of England. Taken alongside wider learning (for example
research in the Church of Wales, reflections from providers of ministerial development) it is of note
that there is an increasing interest in and use of coaching approaches to ministry and professional
development. On further investigation it is also clear that the term ‘coaching’ covers a wide variety
of both interpretation and practice, and although the language is becoming more commonplace, the
meaning and understanding of the terminology and practice varies. The Church of England CMD
Panel recognised that a reflection that sought to bring clarity to this development would be
beneficial. This paper is a brief contribution to the emerging coaching climate in the Church of
England and Church in Wales, and will draw from three dioceses who have sought to integrate
coaching into their ministry development (Leicester, Liverpool and Bangor), a 3rd sector organisation
who have been incorporating this approach, as well as providers of ministry training (CPAS,
3DCoaching).1
Generally, coaching is requested or sought as a ministry development or learning intervention
through a recognised presenting need. For example this can be an individual seeking help around
clarity in vocation, someone wanting to talk through an issue or problem they face at work or in
ministry, or a question of transition to a new role or job. These may have been initiated as a
response to an organisational offer of coaching provision, or through an organisational directive (e.g.
Ministerial Development Review follow up). Often during the first coaching conversation ‘clients’ will
be unaware of the range of developmental or learning interventions on offer, or of the different
approaches available. In all cases clarity of definition is crucial, and misunderstandings can occur
when there is a mismatch in language and expectations.
Tensions can also occur when a ministry development intervention is requested by a 3rd party client.
For example, a CMD officer being approached by an Archdeacon to ‘Do some coaching with X’. This
is an area open to misunderstandings, (to say nothing of legislation) and potential disappointment
for both coach, coachee, and the request initiator (client). For such requests clarity of both process
and meaning would alleviate much of uncertainty.
1 Following Western, in this paper, I will use the term coaching to embrace the multiple and overlapping practices of coaching and mentoring, as coaching has become the dominant term. Mentoring will be named separately when a distinction is necessary. ‘Coachee’ will refer to the recipient of coaching and the term ‘client’ will refer to an organization or their representative sponsoring the coaching work.
Through conversations with coaching deliverers and practitioners three key questions have been
identified that will form the basis for this brief paper.
1. What do we mean when we talk about coaching or mentoring?
2. How can you integrate a coaching or mentoring approach to a wider organisational culture?
3. Is it worth it?
1. What do we mean when we talk about Coaching or Mentoring?
This is perhaps the primary question. The narratives and assumed definitions that underlie a
conversation about a coaching or mentoring intervention can seriously skew expectations
and the enactment of the intervention. It is clear that confusion exists around definitions
both of and within terms in common use in coaching literature, and coaching parlance.
Particular dissonance occurs in use and understanding of the terms ‘mentoring’ and
‘coaching’. It is of note that both CPAS and 3DCoaching noted that their first response when
approached for coaching training or provision by a potential client was to seek clarity on the
model of coaching requested and the assumptions behind the request. It is also noteworthy
that 3D and CPAS themselves operate with different, yet equally valid, coaching approaches.
The following definitions are indicative of the coaching and mentoring narratives and
influences that abound within the field.
The International Coaching Federation2 (ICF) defines coaching as
Partnering with clients in a thought-provoking and creative process that inspires
them to maximize their personal and professional potential. 3
The influential work of John Whitmore, that is still aligned with the ICF principles of coaching
emphasises ‘Performance Coaching’, and is best known for the GROW model of coaching.
This more performance orientated approach is defines coaching as
Facilitating an individual through the process of achieving a specific development
of competence and capacity in relation to their professional role.4
3D Coaching follow the ICF competencies when describing coaching, and define coaching
simply as
The ancient art of conversation. Conversation with an edge,5 and as
‘A conversation where someone feels heard and moves forward’.
2 http://coachfederation.org/index.cfm 3 Taking this definition into their work 3DCoaching speak of Coaching as ‘conversation with an edge’ and use the ICF principles as core to their approach 4 CPAS ‘Differences between roles’ handout 5 http://www.3dcoaching.com/ & http://www.3dcoaching.com/discover/3d-coaching-principles
These definitions are clearly on a continuum and serve to illustrate the broad understanding
of coaching particularly within the developmental field. When you further add mentoring
into the definition mix, the field becomes harder to understand as the personal relationship
between coach/mentor and coachee becomes difficult to differentiate. Bill Hull highlights
the time bound specific nature of coaching in contrast to the more open ended relational
aspect of mentoring often found in discipleship and Christian community.6 His consideration
of coaching and mentoring draws from the relationship between Paul and Timothy, and
notes both the long-term mentoring relationship, and the immediate time bound ‘coaching’
specific elements.
One potential means of differentiating between coaching and mentoring is to recognise the
long term characteristic of a mentoring relationship as in general mentoring focuses on
longer-term development or progress within an organization whereas coaching is more
‘presenting issue’ led.7
As an example, the mentoring specific definition found within the work of CPAS defines
mentoring as
A dynamic, intentional voluntary relationship of trust in which one person (the
mentor) enables another (the mentee) to maximise the grace of God within their
lives, and develop their potential in the service of God’s kingdom purposes.8
Simon Western, in ‘Coaching and Mentoring: A Critical Reflection’ notes that
both mentoring and coaching in the modern context selectively draw on a range of
the same narratives to describe the activity. However, it seems that coaching and
mentoring are essentially similar in nature (Garvey et al., 2009: 27). Mentoring has a
longer tradition than coaching but both activities share many of the same practices,
applications and values. In the end it comes down to a choice of terminology and
the meanings associated with that terminology. … Coaching has become the
dominant term (Garvey, 2010: 351–2).9
Western also notes that
Whilst coaching has become the real ‘buzz word’ and has grown hugely, mentoring
is perhaps an unsung hero in the field of development. Its potential is huge, and with
careful planning and a lot of support, can be impressively effective. (Western
2012:53)
As Western states clearly the terms ‘coaching’ and ‘mentoring’ are not easily distinguishable,
and neither are coaching approaches and models always the most easy to surface and
discuss. How the terms are used is a matter of choice and consistency. How the approaches
are initiated is a matter of choice and resource. It is often the case that initial conversations
6 Hull, The complete book of discipleship’ NavPress 2006 p209 7 The CMI also note that ‘A further distinction between coaching and mentoring is that coaching is usually a
line management function, whereas mentoring is almost always out of the line’. 8 CPAS ‘Differences between roles’ handout (unpublished) 9 Western Introduction 201:2 Box 1
about ministry development through either coaching or mentoring proceed in shorthand
form and consequently skew expectations for the future learning and development
intervention. There needs to be clear conversations in both the procurement and reviewing
of coaching and mentoring interventions. Once the terms in use have been clearly surfaced,
the request for an ‘intervention’ in response to the presenting needs can be more clearly
examined.
Questions for Reflection
To begin to answer our initial question, ‘What do we mean when we talk about coaching or
mentoring?’ It is necessary to ask of ourselves:-
What do I mean, when I talk about coaching or mentoring?
What are my working assumptions about coaching or mentoring?
What are my organisations working assumptions about coaching or mentoring?
What does my conversation partner / organisation mean when they talk about
coaching or mentoring?
Appendix 1 offers a potential framework for an initial coaching conversation.
2. How can you integrate a coaching approach to a wider organisational culture?
There is no one clear way to integrate coaching into a wider organisation – but there are questions
that can help illuminate the possibilities and potential constraints.
There are three scenarios that illustrate the way in which coaching may be integrated (or not) into a
diocese or organisation.
A) Through individual requests.
Evidence would suggest that within the Church of England the majority of commercial
coaching interventions are either self-referrals from an individual with a presenting need, or
are a diocesan initiated intervention. Of these vocational coaching is a common request,
either as preparation for a specific job interview, or to explore a potential role change.
Diocesan referrals to coaching providers cover both of these aspects and providers are
recommended by word of mouth, or through a diocesan officer.
In this scenario coaching is an addition to a diocesan ministry development provision, and is
accessed by those with the wherewithal to search out the opportunity, or to be provoked
into acting for themselves. Coaching and Mentoring are in these instances separate from the
overall learning culture of a diocese and are regarded as something of an addendum for
those who are minded to learn in this way.
As Coaching / Mentoring become more popular and the benefits more obviously seen, there
is a more stated desire to integrate the individual coaching provision into a wider
organisational culture. Indeed this short paper is a response to the increase in requests for
guidance on ways to integrate coaching to an existing provision.
B) Through an intentional training programme
It is also common for diocese either to invite external providers of training to bring expertise
and experience into the diocese to develop capacity, or for diocese to outsource their
provision to an external body. Usually there is an individual with a positive experience of
being coached who wants to bring the provision into the organisation, and who acts as a
‘champion’ for the proposal. They are also, more often than not, aware of potential funding
to leverage this provision. These intentional training programmes are occasionally required
MDR (through Senior Staff policy) or are targeted invites from a Ministry Development
department.
For intentional training programmes delegates are invited to participate in a number of
ways, either by targeted invitation due to role, or self-selection once the benefit of coaching
and the quality of training is established in a diocese. There are some accounts of training
being ‘required’ by a senior staff initiative and attendance mandatory.
In all instances as a wider number of individuals become familiar with coaching or mentoring
approaches, and bring back to, or cascade through the diocese a level of expectation and
engagement with the principles they have learnt.
C) Through wider culture change in an organisation
Coaching or Mentoring is not an answer to all an organisation’s development needs. Neither
will introducing coaching make an organisation a cutting edge learning institution overnight.
Initiatives, such as A and B, when they disperse through an organisation, can adapt and
change a culture in only a limited way. Without a clear purpose and wider ownership they
will not on their own provide organisational development. They will impact on a few, seep
into some wider team practices, and increase capacity of those who may well have found a
way of doing that anyway.
An understanding of your current organisational learning culture is a pre-requisite for
implementing any new initiative. Until the current organisational environment is recognised
and named shifting the culture to a more intentional coaching approach will be more
difficult. Whatever traction a new initiative initially engages with will over time dissolve into
the dominant existing approach. While there is an argument that, as salt and light, a
coaching approach can begin to effect change whether there is a designated ‘strategy’ or
not, change by osmosis will encounter resistance and be slower to gain traction and progress
will be harder to evaluate.
Crucial to a wider culture change is an understanding of the public and perceived ethos of
any senior management team. For example does the senior staff team model expertise and
direct intervention in leadership? Is it a community of learners where there is value in 'not
knowing'? Is it open to risk or risk averse? How is a vision both communicated and
encouraged? Do the perceived ethos of leadership of a senior team, and the wider
organization’s perception of that leadership converge at any point or is there dissonance in
accounts?
The ethos and assumptions of organisational learning has a high degree of influence. If for
example, the sharing of learning is predominantly through an ‘expert’ mode, whereby
learning is seen as a commodity to be passed on by those with more qualifications,
experience, or position, then the implementation of any new initiative will be the preserve
of those who also have become ‘experts’ in any field. On the other hand, if learning is more
democratic and dialogically based in which context is in itself an actor in the learning
process, then the subject of the learning is shared more widely, is more likely to integrate
into team practices, and is formative in more than an intellectual sphere.
Learning approaches can also be disjointed through an organisation. Management levels can
embody a style and ethos of learning at odds with training departments. Training
departments can have no understanding of the overall vision of management levels, and a
silo mentality can easily develop couched in ‘Training development’ v ‘Management problem
solving’ terms.
The question that arises is thus; ‘What would a good learning culture look like to this
organisation as a whole?’ The secondary questions that follows allows the coaching
approach to be integrated; ‘Where would coaching fit into this? What would it take to get
there?’
Questions such as these are illuminated by considering the current reality – so, for example,
investigating who agrees the development approach you take as an organisation, and
knowing who contracts the content and style of any training are important pieces of
information. Similarly being aware of who reviews the outcomes, (if indeed they are
reviewed), and what those who review these outcomes or initiatives measure as they review
are key pieces of information.
Similarly being aware of organisational constraints, habits or modes of being that will hinder
any learning development are important. If your organisational gatherings are highly ‘Expert’
based it is unlikely that a ground up coaching approach will seep throughout the whole
learning culture of your organisation. It will, like water, flow around the big institutional
rocks, causing occasional turbulence depending upon seasonal influences!
Questions for Reflection
The decision to develop coaching provision within a diocese can be taken by diocesan Ministry
Officers, rather than through any Senior Staff request. Where coaching uptake is requested through
a Senior Staff, it is often mandated through a Ministry Officer to implement. An effect of this is that a
‘mid’ level of implementation is attempting to influence both a senior or Bishops staff, and a wider
diocesan culture in Parishes and beyond. To use general terms, there are implementation
consequences of both ‘top down’ and ‘middle led’ initiatives that both release and constrain the
current provision or system of ministry development.
In the scenarios scoped above there are questions that arise which illuminate the way coaching is
integrated within an organisation.
1. How does coaching relate to your Ministerial Development Review / Line Management / Working Agreement practice? Are coaching or mentoring interventions required because of the absence of good line management?
2. Who is able to access coaching? Paid employees, clergy, licensed ministers, all?
3. When are people able to access coaching? How open is the process, and how clear are the expectations? Who invites, who covers the cost, and who reviews? At what stages of ministry is coaching available? New role, end of curacy, specific project, MDR outcomes? Who knows this information, and how available is it?
4. Who doesn’t access coaching in your diocese? Why? What does this say about your organisational learning and development approach? Who fears coaching? Who in your organisation gets ‘exposed’ in a coaching culture?
5. As you seek to integrate coaching throughout your organisation are you using Intentional Interventions or relying upon ‘Guerrilla Coaching’ to subvert culture? Are you introducing a coaching culture in a non-coaching environment?
6. Who provides, equips and sustains the coaches that you use? Are they internally developed, or externally sourced? Who coaches your coaches? Where do your coaches learn? How do they reflect upon and improve their practice? Are they supervised transparently?
Illustrative Accounts
A 3rd Sector Development Agency
A 3rd sector development agency, with a central staff of around 60 people sought to
develop coaching throughout their organisation. Working in a largely relational culture,
they sought culturally relevant developmental opportunities to enable ‘better
conversations’ that could be resourced from a limited Learning & Development budget.
Over a 3 year period they engaged an external provider to deliver coaching training. The
external provider was selected for quality of delivery and similarity and understanding of
ethos. They ran two cohorts, the first (self-selecting cohort) comprising people largely
passionate or interested about coaching.
After the initial training individuals were invited to self-select for further training, and for
their willingness to use the skills learnt within the organisation. This involved being willing
to offer up to 10 hours per year to coach colleagues, and to attend further two courses
along with a quarterly peer review.
To review their coaching provision and the impact this has had on the organisation they
draw from a pool of qualitative research including; targeted reflections from all who have
attended training immediately after training and approximately 6 months post course,
collated anecdotal evidence gathered every quarter using examples from triplet reflection
groups, line management reports, appraisals and individual development plans.
Overall they consider themselves to be an organisation that has a coaching culture, and
that this is a positive way to develop staff capabilities within their existing ethos,
framework and budget.
June 2015
Diocese of Leicester
The diocesan adoption of a strategic vision (Shaped by God) in 2005 required investment in
leadership across the diocese as new methods of working was envisaged. As a result of
delivering Leadership and Missional development programmes in line with this vision, along
with an increasing awareness of the role of leadership teams (lay and ordained) the Diocese
of Leicester began to develop the coaching skills and qualifications of members of its
Mission & Ministry department initially through qualified providers known to the Diocese,
and then through more structured and intentional training.
Through on-going development with dual- role clergy it was recognised that individuals
transitioning into new roles benefitted from intentional transition coaching from someone
outside of the direct line management arrangement. Initially offered to those moving role
within the diocese, the benefit of transitional coaching to those entering any new position
was recognised, and encouraged. The success of transitional coaching led to requests for
more vocational, and project specific coaching.
To further support the demand for coaches, and to increase the capacity and skill of the
existing Ministry department, external providers were brought in to run coaching
workshops and training. The training was ultimately offered to all within the diocese,
including Lay Pioneers, Youth Workers and diocesan department staff. This training
invitation was nearly always as a result of an MDR, or vocational coaching conversation. On
completion of the initial training programme, individuals were given the option of further
training (to an accredited International Coaching Federation level), of occasional coaching
development opportunities, or of no further training. They were also offered the
opportunity to formally use their new coaching skills through some of the transition and
vocational coaching that was routinely offered.
Although an initially need driven initiative coaching requests and provision dramatically
increased over a 3-5 year period. Several Mission & Ministry staff are now engaged in ICF
qualification, coaching provision is regularly reviewed, and standards (for example for
Missional Coaching) are aligned. Recent diocesan initiatives (e.g Pioneer Development
Workers / Fresh Expressions of Church) have coaching principles at their heart. A core
cohort of internal coaches’ meet for ongoing training and supervision, and regular
supervision of more occasional coaches is in place.
Senior staff have recognised the benefit of a coaching approach, and some have
themselves been on relevant training. At present coaching occurs most clearly outside of
the line management / hierarchical structures of the diocese, but is understood to be an
integral part of ministry development, for lay and clergy alike.
Mission & Ministry, Leicester May 2015
Diocese of Liverpool
In Liverpool the department of Lifelong Learning has begun to explore the impact coaching
could have through emerging needs identified in Curate / Training Incumbent supervision
training. There were two key drivers. The first was to challenge the ‘Expert’ training model,
and instil a more ‘community of practice’ learning approach. The second was, at its
simplest, to facilitate richer conversations in a range of pastoral and ministerial settings.
Initially the Lifelong Learning team spent time deciding the approach they wanted to take.
The time spent clarifying terminology and teasing out the benefits and challenges of both
coaching and mentoring was foundational to their subsequent development across the
diocese. Having highlighted the critical working relationships, and explored various
options, the coaching model was pursued. There was no intent to privilege one model
against others but a sense that the core skills developed in coaching were deemed to be
most transferrable across the organisation.
The coaching approach resonated with the emerging ministry development needs, and in
particular the need to facilitate ministers to make sense of new working environments
especially at transitional times in ministry was key. The coaching disciplines of working with
the presenting issue from the coachee, of having responsibility for any outcomes in the
room remaining with the coachee, and having a clear contracting process are all visible and
identifiable indicators of a change in practice and mind-set across the diocese.
Coaching has not been the only developmental approach the department has undertaken.
However, it is clearly an important and formative tool in the overall vision of the
department and the diocese and one in which both clergy and lay people are being trained.
Through intentional training provision the Lifelong Learning department has begun to
encourage a coaching approach throughout the diocese, and has targeted training
opportunities. These have included ministers in transition, Area Deans, the diocesan
vocations team and MDR reviewers. The diocese aspires to a ‘Lifelong Learning culture’ and
is itself in a period of transition, with a relatively new and still forming Senior Staff team.
The challenge is to continue to engage coaching as an important servant to the emerging
diocesan vision, the support and training of clergy and a learning culture which enables
discipleship, pastoral ministry and missional to flourish.
Lifelong Learning, Liverpool June 2015.
Diocese of Bangor
The implementation of coaching within the Diocese of Bangor was fully incorporated across
the diocese through a strategic decision. This was an Epsicopally led initiative that was
agreed by Bishop’s Council and implemented through an intentional process. Coaching was
not brought in as a separate initiative, but as part of an overall learning approach.
Coaching Training was delivered in partnership with an external provider and delivered to
intentionally diverse groups. The first course participants were a mixture of Bishop’s
Council, Senior Staff, Curates, and Clergy from Mission and Ministry Areas. The second
cohort also included Children, Youth and Family Ministry Enablers.
The development of the coaching approach has incorporated the introduction of transition
coaching for anyone beginning a new role within the diocese, and a current initiative
includes the training and capacitating of Action Learning Sets.
In this instance an organisational policy has been introduced through engagement across
all areas of the organisation, and has added to emerging diocesan strategy and vision.
Indeed the time taken to surface assumptions and expectations in the contracting of
external providers was itself a fruitful learning experience for the Diocese. This was not a
‘quick fix’ approach, but clearly a well researched intentional and fully explored
partnership.
Coaching is now becoming embedded within the organisation and is embodied in the
learning approach of the presiding Bishop. It has made an observable impact upon the
ministry of the diocese, and has become a mainstay of ministry development and support.
Learning Church principles are impacting the development of all ministry and mission areas,
and coaching is clearly a valued approach that brings impact and development in many
spheres.
Diocese of Bangor, June 2015
3. Is it worth it? Can you measure effectiveness & impact?
Evaluating the benefit of coaching is difficult and only limited studies have been competed in what is
still an emerging field.10 It is to be noted that a recent report from drawing from the 2011 and 2013
Experiences of Ministry surveys has highlighted the positive role of Professional Development across
the Church of England, and further insights will emerge from the 2015 survey.11
Often within an organisation coaching provision will be one intervention amongst many, both formal
and informal. Both coaching and mentoring goals can be individual before they are institutional, and
thus harder to quantify. Traditional analysis of ‘Return on Investment’ (ROI) is difficult to correlate
particularly in a vocational relational setting, and the long term relationship (particularly of
mentoring) can make even mid-term conclusions hard to perceive.
There is clear evidence throughout the literature that it would not be cost-effective (or indeed desirable) to measure every possible intervention in the workplace; particularly where there are significant costs associated with in-depth evaluation. It is therefore important to make clear judgments on what should be measured.12
However, just because evaluation of coaching is difficult doesn’t mean it isn’t worthwhile, nor that it
shouldn’t be rigorous. Measurement of training effectiveness has been largely dependent upon
Kirkpatrick’s model since its introduction in 1954. This (evolving) model focusses on Return of
Training Investment.
Kirkpatrick’s Model essentially measures results (or return on investments) through performance
improvement, behaviour change, work load capacity and immediate evaluation of training input.
Kirkpatrick evaluates;
1. Reaction of student - what they thought and felt about the training
2. Learning - the resulting increase in knowledge or capability
3. Behaviour - extent of behaviour and capability improvement and
implementation/application
4. Results - the effects on the business or environment resulting from the trainee's
performance
10 Coaching and mentoring are increasingly been used as methods of learning and development however they are also some of the interventions that are the least likely to be evaluated (Industrial Society, 1998 and Leedham, 2005). This position does not appear to have changed as in the CIPD (2008) survey only 8% of L&D respondents evaluate the effectiveness of coaching through a formal evaluation process. Whilst some exceptions exist such as Paige (2002) it remains evident that there is little empirical evidence in this area. Mavin, Lee & Robson p4 HEFCE 2010 11 See http://www.ministrydevelopment.org.uk/professionaldevelopment 12 Mavin, Lee & Robson note that there is a working recommendation that only 5-10% of training interventions should be evaluated at an in-depth level 2010 p4.
Kirkpatrick recommended all these measures for a full and meaningful evaluation of learning in
organizations, although their application broadly increases in complexity, and usually cost, through
the levels from level 1-4.13
However there has been a recognition that the original Kirkpatrick model has been used to focus too
heavily upon financial return, and as a result a new model based on partnership and organisational
agreement has been developed. This model focusses upon Return on Expectation (ROE)
The move towards ‘Return on Expectations’ (ROE) over ‘Return on Investment’ (ROI) as a means of
assessing impact and value of a training intervention does provide some opportunity for a more
insightful and realistic conversation around the benefits and costs of a coaching culture in an
organisation together with a process that encourages clear contracting at the outset.
Return on expectations (ROE) requires stakeholders to evaluate the training outcomes in
terms of what they expect to be able to do after undertaking the intervention. The rationale
behind this concept is that the success of the intervention is assessed by looking at the extent
to which these expectations have been met; thus relying on carefully constructed and
specified expectations to reduce ambiguity.14
The key questions underpinning a ROE approach are: 15
What were the original expectations of the organisational stakeholders for the learning or
training? Have those expectations since changed?
What changes have occurred as a result of the learning processes?
To what extent have stakeholder expectations been met?
13 For a summary see www.kirkpatrickpartners.com, as well as more independent summaries such as http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm. A good summary of evaluation approaches can be found at www.infed.org/biblio/b-eval.htm - Smith, Mark K. (2001, 2006) ‘Evaluation’ in the encyclopaedia of informal education. 14 Mavin, Lee & Robson 2010 p7-9 15 Anderson (2009b) as above
Once critical behaviours are identified, learning content can be crafted and addressed.
Who can you craft the coaching intervention with? External courses or internal provision?
6. Monitor and adjust Learning creates a strong foundation for success, but it does not create ROE in and of itself. Kirkpatrick levels 3&4 (Behaviour and Results) need to be monitored and adjusted for progress and alignment with overall organisational mission (The end is the beginning).
What are your feedback loops telling you? What are your coaches noticing? How do your feedback loops complement your current provision and future planning?
16 www.kirkpatrickpartners.com/OurPhilosophy/ReturnOnExpectations drawing from the Kirkpatrick 4 level Evaluation of Learning. (See http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel
A further useful framework is the CIRO model of evaluation originally introduced by Warr,
Bird and Racham.17 The CIRO model focuses on four areas shown below.18 Perhaps a key
strength of this model is that it takes into account the resources and context of the
organisation (within the input measure) and the organisational objectives (within the
context measure).
CIRO Framework
Accompanying indicative question Suggested Coaching / Mentoring training application
Context What needs to be addressed? What are the ultimate objectives? What are the intermediate objectives? What are the immediate objectives?
What is the main thing? What are the ultimate, intermediate and immediate objectives of the coaching training intervention? What other factors do we need to be aware of?
Input What is likely to bring about the changes?
What training or coaching resources do we have, and how can they best be deployed to meet the main need? Who can craft this intervention with you? External courses or internal provision?
Reaction How did the learners react to the training?
What are the participants saying about the new skill they have learnt?
Outcome What are immediate, intermediate and ultimate outcomes? What happened at the learner, workplace, team, organisational level as a result of the training?
What is actually happening as a result of the training event? How is coaching actually utilised and grounded within the organisation? What is the difference you can see? What’s the next step?
Both ROE and CORI offer a framework for evaluating both impact and effectiveness of a coaching or mentoring intervention. Both rely upon knowing the context and ‘mission’ of the
17 Warr, P., Bird, M. & Rackham, N. Evaluation of management training. London: Gower Press 1970. 18 Mavin, Lee & Robson p10. The main difference from Kirkpatrick’s model in that CIRO focuses on measurements taken before and after the training has been carried out. Four levels cited in Elliott et al., 2009.
wider organization before embarking on any provision. Both ROE and CORI have something to offer in the occasions when coaching / mentoring is requested by a third party, and thus when verifiable data for outcome is needed. Third party contracting needs clarity of purpose, a clear line of accountability, preferred outcomes, and an agreed and transparent ending. CORI in particular (with the ultimate, intermediate, immediate categories) provides a means of marking progress, and of delivering relevant feedback to the 3rd party initiator, as well as concrete stages of progress to the individual coachee.
Whatever evaluation framework is used the question remains – are the coaching approaches and training initiatives provided delivering individual learning and organisational insight in a verifiable way?
Coaching is not a panacea to cover all training needs, or ministry deficiencies. It is a highly effective approach to ministry development that when used with foresight and appropriate contracting and can be an important factor in individual and organisational development. It is also worth noting that the conclusions from the UK National Skills Task Force in 200019
reinforced the crucial role of managers in developing learning:
”the capability and commitment of managers and key workers throughout an organisation’s hierarchy are the most important factors in determining the provision and effectiveness of workplace learning (both formal and informal)”.
Reflective Questions
1. Why are you asking questions of evaluation? Who wants to know, and why?
2. What does your organisation / diocesan overall learning culture look like? What does a good coaching culture look like to you?
3. What are your own feedback loops telling you?
4. What are your coaches telling you that they are noticing?
5. How do your feedback loops complement insights from others in the system?
6. Is there alignment in your Ministerial review / Line Management / Working Agreement / Appointments practice?
7. How do you know when is ‘enough’?
8. How do you decide when to ‘refer on’ to another strategy or category of
provider?
9. How transparent and effective are your ‘ending’ conversations?
19 National Skills Task Force, 2000:37
Appendix 1: What container is most appropriate to the intervention?
An image that is useful is to consider the ‘Intervention container’ within the developmental process.
This ‘container’ could contain the intervention of a coach, mentor, spiritual director, consultant,
therapist, or pastoral care provider (and there are many others).
The diagram below illustrates the process and possibilities of getting the container right, for any
