Toolkit Grus.ppt

8/22/2019 Toolkit Grus.ppt

1/29

The Competency AssessmentToolkit for Professional

PsychologyCatherine L. Grus, Ph.D.,

Associate Executive Director forProfessional Education and Training

Education DirectorateAmerican Psychological Association


2/29

THANK YOU

Many thanks to

Nadya Fouad, University of Wisconsin

Nadine Kaslow, Emory University Benchmarks WorkgroupsAssessment Toolkit Workgroup


3/29

Overview

Considerations in the assessment ofcompetence

Development of the Toolkit Putting the toolkit into practice


4/29

Competence

the habitual and judicious use ofcommunication, knowledge, technical

skills, clinical reasoning, emotions, values,and reflection in daily practice for thebenefit of the individual and community

being served.From Epstein and Hundert (2002)


5/29

Competence

is developmental depending on stage ofprofessional functioning

context dependent executed according to ethical principles,

guidelines, standards, and values of the

profession and requires public verification


6/29

Competencies

Observable

Measurable Containable

Practical

Derived by experts And flexible

are elements of competence that are .


7/29

Assessment of Competence -Considerations

Use of both formative and summativeevaluation methods

Authentic evaluations

Real world tasks, meaningful activities

[High] Fidelity: Degree to which

the assessment represents

actual performance


8/29

Employ a range of perspectives (multi-

informant) and methods Challenge of evaluating attitudes and

values

Psychometric issues Feasibility/costs Lifelong self-assessment


9/29

What methods are commonlyused to assess students/

trainees?

In class examinations and course grades

Faculty/supervisor completes rating scale Letters of recommendation EPPP, state jurisprudence exams


10/29

Developing Faculty/SupervisorCompetencies

Training in difficult conversations How to critique constructively

Develop awareness of potential dual roles inevaluation and how that could influence results

Benefits of having other faculty evaluate studentsperformance

Explicit obligations to make evaluations accurate Grade inflation Letters of recommendation to market vs. evaluate

Knowledge of a range of assessment tools,access to resources toolkits


11/29

Competency Assessment forToolkit for Professional

Psychology

Charge from APA Board of Educational Affairs:

Develop a Toolkit for professional psychology Purpose: Promote broader implementation of

competence assessment and provide informationabout application of assessment methods to the

assessment of competence

Coordinated with Benchmarks Work Group


12/29

Benchmarks Core Competencies

Foundational Competencies: Professionalism Reflective practice

Scientific knowledge andmethods Relationships Individual and cultural

diversity

Ethical and legalstandards and policy Interdisciplinary systems

Functional Competencies: Assessment Intervention

Consultation Research and evaluation Supervision Teaching Administration Advocacy


13/29

How are the BenchmarksOrganized?

Core Foundational and FunctionalCompetencies

Essential Component: what are thecritical elements of/whatknowledge/skills/attitudes that make upthis competency?

Behavioral Anchor: what would it looklike if you saw it (essential component)?


14/29

Components of Toolkit

Background and IntroductionAssessment Method Fact Sheets

Grid of Assessment Methods andCompetencies/Essential Components Bestused for

F t Sh t P tf li R i


15/29

Fact Sheet: Portfolio Review

DescriptionA portfolio is a collection of products, gathered by the person being assessed

Application

Competency Applies to1

PredominantUse of2

Developmental Level3

Professionalism E F,S 2,3,4

Reflective Practice E F,S 1,2,3,4

Implementation

Deciding on form (e.g., web-based or hard copy)Psychometric PropertiesReliability has not been well established due to the variable content included

in a portfolioStrengths

Provides a low cost assessment strategyChallengesRequires intense commitment of time and labor intensive for the assessors

and the person being assessedFuture directions

Create and implement training modules for assessors and people beingassessed in the portfolio assessment strategy


16/29

Annualreview

Casereviews

Process/outcomedata

RatingForms

Consumersurveys

OSCE

Portfolioreviews

RatingsPerformance

Recordreviews

Self-assessment

Simulations/rolep

lays

Stand.patientinte

rview

Stand.oralexams

Writtenexams

360Evaluations

Intervention Knowledge

interventions

2 3 3 2 1 2 1 1

Intervention

planning

2 1 2 1 2 2 3 1 2 2 1

Skills 2 2 1 3 2 1 2 1 2 1 1 2

Implementation 1 1 1 1 1 1 2 1 2 1 2 2 2 3 1

Progress

evaluation

3 1 2 2 2 3 2 2 1 2 2

Grid of Possible Assessment Methods(Toolkit)


17/29

Assessment Methods in Toolkit

360 evaluation Portfolio OSCE Structured Written & Oral

Exams

Case Presentation Simulation/Role Play Competence Evaluation

Rating Form Self-Assessment

Ratings of live orrecorded performance

Standardized Client

Interview Client/Patient

Process/OutcomeMeasure

Consumer SatisfactionSurvey

End of RotationPerformance Review


18/29

Putting it all Together

Benchmarks

Toolkit

Using the Toolkit andBenchmarks to Guide the

Assessment of Student Learning

Outcomes


19/29

Identify expectedstudent learningoutcomes using

competency model

(e.g., Benchmarks)

Selectassessmentmethod(s)

Expected level ofcompetence attainment

and training plan

developed with trainee

Formativeassessment

Student demonstratesproblems with

competencies acquisition

Student demonstratesemerging competencies

Training plancontinues

Remediation plandeveloped

Summative evaluation to assessreadiness for next level of training


20/29


competency model

(e.g., Benchmarks)


21/29

Program Outcome: Students will demonstrate thefollowing intervention (functional) competencies priorto going on internship.

1. Knowledge of scientific, theoretical, empirical andcontextual bases of intervention, including theory,research, and practice

2. Formulates and conceptualizes cases and plan

interventions utilizing at least one consistenttheoretical orientation3. Clinical skills4. Implements evidence-based interventions that take

into account empirical support, clinical judgment,

and client diversity (e.g., client characteristics,values, and context)5. Evaluate treatment progress and modify treatment

planning as indicated, utilizing established outcomemeasures


22/29


competency model

(e.g., Benchmarks)



23/29

Annualreview

Casereviews

P

rocess/outcomedata

R

atingForms

C

onsumersurveys

O

SCE

P

ortfolioreviews

R

atingsPerformance

R

ecordreviews

S

elf-assessment

S

imulations/roleplays

S

tand.patientinterview

S

tand.oralexams

W

rittenexams

3

60Evaluations

Intervention Knowledge

interventions

2 3 3 2 1 2 1 1

Intervention

planning

2 1 2 1 2 2 3 1 2 2 1

Skills 2 2 1 3 2 1 2 1 2 1 1 2

Implementation 1 1 1 1 1 1 2 1 2 1 2 2 2 3 1

Progress

evaluation

3 1 2 2 2 3 2 2 1 2 2

Grid of Possible Assessment Methods(Toolkit)


24/29


competency model

(e.g., Benchmarks)



and training plan



25/29

Developmental Level

A. Knowledge of Interventions

Readiness for Practicum Readiness for InternshipEssential Component:

Basic knowledge of scientific, theoretical,

and contextual bases of intervention and

basic knowledge of the value of evidence-

based practice and its role in scientific

psychology

Behavioral anchor:

Articulates the relationship of EBP to thescience of psychology

Identifies basic strengths and weaknesses of

intervention approaches for different problems

and populations

Essential Component:

Knowledge of scientific, theoretical,

empirical and contextual bases of

intervention, including theory, research,

and practice

Behavioral Anchor:

Demonstrates knowledge of interventionsand explanations for their use based on EBP

Demonstrates the ability to select

interventions for different problems and

populations related to the practice setting

Investigates existing literature related to

problems and client issues

Writes a statement of ones own theoretical

perspective regarding intervention strategies


26/29


competency model

(e.g., Benchmarks)



and training plan


Formativeassessment







27/29

Remediation Plan

Competenc

y Domain/

Essential

Component

s

Problem

Behaviors

Expectations

for Acceptable

Performance

Trainees

Responsibilitie

s/

Actions

Supervisors/

Faculty

Responsibilities

/

Actions

Timeframe

for

Acceptable

Performance

Assessmen

t

Methods

Dates of

Evaluatio

n

Consequence

s

for

Unsuccessful

Remediation


28/29


competency model

(e.g., Benchmarks)



and training plandeveloped with trainee

Formativeassessment






Summative evaluation to assessreadiness for next level of training


29/29

For more information:

http://www.apa.org/ed/graduate/competency.html

Coming soon:

Kaslow, N. J., Grus, C. L., Campbell, L. C., Fouad,N. A., Hatcher, R. L., & Rodolfa, E. R. (in press)

Competency Assessment Toolkit forProfessional Psychology. Training andEducation in Professional Psychology.
http://www.apa.org/ed/graduate/competency.htmlhttp://www.apa.org/ed/graduate/competency.html

Date post:	08-Aug-2018
Category:	Documents
Upload:	mohamed-bekheet
View:	212 times
Download:	0 times

Toolkit Grus.ppt

Documents