Co-Existing Problems (CEP)

Skills Development

CEP Skills Development

Requirement for services and clinicians to be CEP capable. NZ guidelines already exist. (Te Ariari o te Oranga )

Questions raised about:

How do we do it?

What have we tried already?

How could we do it better?

How do we know when we have got there?

Improve the skills of clinicians from a wide variety of disciplines in assessing and treating clients with co-existing disorders

Support clinicians in attaining skills and to gain confidence in using them.

Achieve a cultural shift in MH and Addiction services nationwide

What do we need to do?

Skills Development versus Competencies

Competencies are related to the individual clinician

Skills - are about the work we do

Development and Attainment of skills- knowledge and practical application

Skills measurement - quantifying/ qualifying skills development eg. Using Workplace Based Assessments (WBAs)

Does

Shows How

Knows How

Knows

Miller GE 1990. The assessment of clinical skills/competence/performance. Academic Medicine, 65 Supp 563-567

Miller’s Pyramid

Competencies

Currently already exist.

Linked with professional bodies – DANA,

DAPAANZ, RANZCP, Te Pou, ?Peer support

Whilst these competencies are worded differently,

they tend to say the same things!

When demonstrating skills attainment, clinicians

should automatically demonstrate the

competencies required to implement the skills.

Skills Set

Can be quantified dependent on experience and training of clinician

Makes CEP a practical proposition as clinicians can identify which skills to work towards

Skills can be mapped against competencies e.g, assessment skills include communication, collaboration and cultural competencies

Aligns with Te Ariari o te Oranga approach

Examples of skills

Undertake a comprehensive assessment

Complete a risk assessment

Implement a brief intervention

Screen for mental illness/substance use disorder

Implement a relapse prevention strategy

SKILLS

Foundation

Capable

Enhanced

Brief Interventions Knowledge of brief interventions Able to apply simple brief intervention strategies

Application of brief

interventions

Demonstrates skills in utilising brief interventions in a variety of settings

Comprehensive

mental health,

substance use and

gambling

assessment

Knowledge of DSM-IV/ICD-10 for MH, SUDs (inc. Subs induced) and PG Assessment of recent and lifetime mental health symptoms/problems (patterns of use and tx) Assessment of recent and lifetime substance use Assessment of recent and lifetime gambling behaviour (patterns of use and tx)

Assessment of mental

health, substance use and

gambling history in the

context of psychological and

physical functioning,

symptomatology and

withdrawal history

Demonstrates knowledge and application of DSM-IV/ICD-10 criteria for mental health, substance use disorders (including substance-induced states) and Pathological Gambling

Monitoring and

testing of

substances, alcohol

and medications

Knowledge of common substance testing procedures and laboratory investigations Knowledge of common blood/urine tests

Application of substance

testing procedures and

laboratory investigations.

Comprehensive knowledge

of methods for substance

testing

Demonstrates knowledge and interpretation of common substance testing procedures and laboratory investigations

• Copyright © 2006 The Royal College of Physicians and Surgeons

of Canada. http://rcpsc.medical.org/canmeds. Reproduced and

adapted with permission.

Clinical

Expert

Skills mapped against competency. Comprehensive Ax

Health Advocate. (Competency)

Fosters positive attitudes to counter stigma towards

people with substance use, mental health and gambling

disorders. Demonstrate an understanding of issues

pertaining to coexisting disorders in diverse groups

including cultural, ethnic, indigenous and disadvantaged

groups across all stages of life. (Role Competency)

Does the assessor advocate on behalf of the client when

the family/whanau is present?

Does the assessor counter prejudicial attitudes in a

multidisciplinary team setting?

Does the assessor display a non-judgemental attitude in

all settings? (Indicative questions)

Skills Development

Self directed skills development

eg.:DVD’s

online learning

Knowledge acquisition:

eg.: training events

workshops

Mentoring/Supervision

eg. Supervision

work based assessments

•How do we know when someone has the skills? •Why do we need to know if skills have been attained? •What will this mean for 1.MH Clinicians? 2.Addiction clinicians? •Do we need more than competencies?

Break for Questions

Miller’s Pyramid and Assessment

Does

Shows How

Knows How

Knows

Observed interviews with feedback

Written Exams eg MCQ

Complex written exams

Workplace based assessment

Assessment

Miller GE 1990. The assessment of clinical skills/competence/performance. Academic Medicine, 65 Supp 563-567

Workplace Based Assessments to measure acquisition of skills

• ACE: Assessment of Clinical Expertise

• CBD: Case Based Discussion

• MSF: Multi Source Feedback

• MDT/Case Conference

• Logbook

• Client Satisfaction Questionnaire

Piloting the Skills

Skills were developed at 3 levels recognising the variety of professional disciplines and experience that exist

Foundation, Capable, Enhanced

Baskets of skills could be chosen by clinicians dependent on the work environment

Why do we need a Pilot?

• Does the framework work?

• Works to win the confidence of other stakeholders

• Maps the resources required

• Provides baseline data

• Enables adaptation and alteration

Outcomes and the future

• Can this skills development approach be used for core mental health and addiction skills?

• How are we going to continue evaluating?

• Do we need funding or can we implement from existing resource?

• Does it work???

Evidence of effectiveness –

Selected literature supporting CEP

skills development pilot

Competencies/Skills:

• International competencies (Graham &White 2011; www.ccsa.ca 2010)

• Challenges of linking competencies to clinical care (Jones et al., 2011) thus the move to a skills focus.

• Practical implications of implementing competencies (Mulder et al., 2010)

Work Based Assessments:

• Significant correlation between scores from the CEX and exam scores (Searle 2008)

• WBAs as an essential element of British psychiatrists training

• ‘Need for good planning and understanding around the WBAs prior to initiation ‘(Menon, Winston and Sullivan 2012)

• Highlights validity measuring complexities and points to the value of utility (Holsgrove 2010)

Workforce Development: • Workforce development typically is of individuals ,

through education and training , in a knowledge transfer structure (Allsop &Helfgott 2002, Roche 2009)

• Training as a stand-alone event, results in limited or non-sustained change (Arthur et al. 1998, Baer et al. 2004, Bennett et al. 2007,

Roche 2002)

• Requires systems wide approach that supports the transfer of skills and knowledge Eg appropriate supervisory, peer and organisational support post training (Roche 2002, Cromwell 2004, Lim & Morris 2006, Taylor 2000)

Benefits and Barriers

Effective for enhancing CEP capability

Assessor and clinician benefits

Best clinical practice

Use alongside other approaches & strategies

Sustained change

• Identifying enhanced practitioners

• Understanding the process and value

• Resourcing

• Service support

• Clarity about its future

Evaluation Objectives

• Validity

• Inter-rater reliability

• Generalisability

• Feasibility

• Utility

Acknowledgements

Pilot project assessors and clinicians

Clients/tangata whaiora

Services who supported the pilot

Co-existing Disorders Team, CCDHB

Participants

Wellington Assessors:

counsellor (2),

social worker (2),

nurse (3),

psychiatrist(1)

Christchurch Assessors: nurse (7)

Wellington Clinicians (n=11): – DHB (7); NGO (4)

– Addiction (4); Mental health (7)

Wellington Clinicians:

At initiation

n=16

At completion

n=11

Professional discipline Support worker 2 2

Cultural worker 1 1

Counsellor 6 3

Social worker 3 2

Nurse 3 2

Psychiatrist 1 1

Self reported/perceived level of

competence at initiation

Foundation 8 7

Capable 6 3

Capable/Enhanced 2 2

Study Design

Written survey – Qualitative and quantitative responses

Training package

Skills Framework guiding document

CEP skills framework

Each of the work based assessments

The pilot overall

Focus groups – Separate assessor and clinician focus groups

Barriers and benefits

Value and utility of the framework and WBAs

Assessor/clinician relationships

The pilot project overall

Key Findings Skills Framework

From its original form, requires further development

Clinicians would like feedback on their level of capability

Clinicians will straddle a number of skills across the range of foundation, capable and enhanced

Services could link the framework and WBAs to employment and professional development opportunities/plans

Key Findings Work Based Assessment

Benefits clinician and the assessor

CBDs and ACEs most time consuming but beneficial

Formative and summative assessment

Approach to structured supervision

Clarify the assessor/supervisor role

A compulsory element to completion

Responding to client feedback

Reflective practice

Encourages sustained behaviour change and practice

Pilot shows applicability and benefits to clinical practice:

– Addiction and mental health clinicians

– Across disciplines

– NGOs and DHBs

– Assessors and clinicians alike

– Across service and disciplines

A willingness to up skill and develop CEP capability

Resourcing intensity cannot be underestimated

Key Findings overall

Conclusion

The skills framework and associated WBAs have value and utility as a tool to enhance CEP capability

They encourage implementation of best clinical practice and supervision

Would be best serviced to complement, not supersede formal knowledge building and existing forms of professional development

Addressing training from a whole systems perspective, within the context of a robust workforce development plan is required to support

effective practice.

References Allsop, S. J., & Helfgott, S. (2002). Whither the drug specialist? The workforce development needs of drug specialist staff and agencies. Drug and Alcohol review,

21, 215-222.

Arthur, W., Jr., Bennett, W., Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human

Performance, 11, 57–101.

Baer, J.S., Rosengren, D.R., Dunn, C., Wells, E.A., Ogle, R., & Hartzler, B. (2004). An evaluation of workshop training in motivational interviewing for addiction and

mental health clinicians. Drug and Alcohol Dependence, 73, 99-106.

Bennett, G. A., Moore, J., Vaughan, T., Rouse, L., Gibbins, J.A., Thomas, P., James, K., & Gower, P. (2007). Strengthening Motivational Interviewing skills following

initial training: A randomised trial of workplace-based reflective practice. Addictive Behaviors, 32, 2963-2975.

Bhugra, D., Malik, A. & Brown N. (2007) “Workplace-based Assessments in psychiatry” published College Seminar Series of the Royal College of Psychs., UK.

CCSA. Competencies for Canada’s Substance Abuse Workforce. www.ccsa.ca/eng/priiorities/workforce/competencies/pages/default.aspx. Retrieved on 16/7/2012.

Cromwell, S. E., & Kolb, J. A. (2004). An examination of work-environment support factors affecting transfer of supervisory skills training to the workplace. Human

Resource Development Quarterly, 15(4), 449-471.

Holsgrove, G. Reliability issues in the assessment of small cohorts. General Medical Council Guidance Paper. 2010

Friedman, Mark (2005),Trying Hard is Not Good Enough:How to Produce Measurable Improvements for Customers and Communities. Trafford Publ.Canada p81.

Graham, H. & White, R. (2011) Comorbidity Competencies: Skills Indicators by support recovery from comorbidity in Tasmania, University of Tasmania: Australia

Jones, M.D., Rosenburg,A., Gilhooly, J. and Carraccio C. Competencies, outcomes and controversy-Linking professional activities to competencies to improves

resident education and practice. Academic Medicine. 2011; vol 86. 2:1-5

Lim, D. H., & Morris, M. L. (2006). Influence of trainee characteristics, instructional satisfaction, and organizational climate on perceived learning and training

transfer. Human Resource Development Quarterly, 17, 85-115.

Menon, S., Winston,M. & Sullivan G. Workplace-based assessment: attitudes and perceptions among consultant trainers and comparison with those of trainees.

The Psychiatrist Online Jan. 2012. 36:16-24;

Miller GE (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65 Supp 563-5672

Mulder, H. ten Cate, O., Daalder, R. & Berkverns, J. Building a competency-based workplace curriculum around entrustable professional activities: The case of

physician assistant training. Med. Teach 2010;32(10):e453-9.

Roche, A. M. (2009). New horizons in AOD workforce development. Drugs: education, prevention and policy. 16 (3), 193 - 204.

Roche, A. M. (2002). Workforce Development Issues in the AOD Field: A Briefing Paper for the Inter-Governmental Committee on Drugs. Unpublished Report.

National Centre for Education and Training on Addiction.

Searle, G. Is Cex good for psychiatry? An evaluation of workplace-based assessment. Psychiatric Bulletin. 2008; 32:271-273

Taylor, M. C. (2000). Partners in the transfer of learning: A qualitative study of workplace literacy programs. Proceedings of the 41st Annual Adult Education

Research Conference (pp. 459-461). Vancouver: University of British Columbia.

Walters, S. T., Matson, S. A., Baer, J.S., & Ziedonis, D. M. (2005). Effectiveness of workshop training for psychosocial addiction treatments: A systematic review.

Journal of Substance Abuse Treatment, 29, 283-293.

The Royal college of physicians and surgeons of Canada. 2006. http://rcpsc.medical.org/canmeds

“Culture change does not happen in complex

organisations through one or two training

sessions... it happens with little events that

happen thousands of times in the life of an

organisation”

(Friedman, 2005:81)