How To Interpret CAP How To Interpret CAP Proficiency Testing EvaluationsProficiency Testing Evaluations

Elaine Staley, MT(ASCP)Elaine Staley, MT(ASCP)Compliance OfficerCompliance Officer

St. Luke’s Hospital – Mayo Clinic St. Luke’s Hospital – Mayo Clinic JacksonvilleJacksonville

ObjectivesObjectives

Identify performance issues by Identify performance issues by evaluating data provided on the PT evaluating data provided on the PT evaluation and Participant Summaryevaluation and Participant SummaryList investigational techniques for List investigational techniques for responding to Proficiency Testing responding to Proficiency Testing Exception Summary (PTES)Exception Summary (PTES)Assemble a quarterly review of PT Assemble a quarterly review of PT scores by using the All Analyte scores by using the All Analyte ScorecardScorecard

e-Lab Solutionse-Lab Solutions

The best way to view CAP evaluations is The best way to view CAP evaluations is to use e-Lab Solutionsto use e-Lab Solutions Request access from your institution’s Request access from your institution’s

administrator administrator

CAP Evaluations 101CAP Evaluations 101

Printed EvaluationPrinted Evaluation Exception CodesException Codes

Interactive EvaluationInteractive Evaluation

Participant SummaryParticipant Summary

CAP Evaluation CAP Evaluation What Should You Review?What Should You Review?

Evaluate each analyte and specimen for Evaluate each analyte and specimen for negative/positive bias, trends or shiftsnegative/positive bias, trends or shifts

Evaluate ungraded challengesEvaluate ungraded challengesIf an evaluation criteria is not listed in the If an evaluation criteria is not listed in the

Participant Summary you can use information Participant Summary you can use information provided by manufacturer or use Rhoads provided by manufacturer or use Rhoads http://www.dgrhoads.com/db2004/ae2004.phphttp://www.dgrhoads.com/db2004/ae2004.php

Education Challenges Code 26Education Challenges Code 26

POC.03275 Phase IIPOC.03275 Phase II N/A YES NON/A YES NOIs there evidence of evaluation and, if indicated, Is there evidence of evaluation and, if indicated, corrective action in response to "unacceptable" corrective action in response to "unacceptable" results on the proficiency testing reports and results results on the proficiency testing reports and results of the alternative performance assessment system?of the alternative performance assessment system?

NOTE: The evaluation must document the specific NOTE: The evaluation must document the specific reason(s) reason(s) for the "unacceptable" result(s) and actions taken to for the "unacceptable" result(s) and actions taken to reduce the reduce the likelihood of recurrence. This must be done within likelihood of recurrence. This must be done within one month one month after the POCT program receives its evaluation. In after the POCT program receives its evaluation. In addition, addition, each ungraded challenge, each educational each ungraded challenge, each educational challenge, and challenge, and each episode of nonparticipation must be reviewed each episode of nonparticipation must be reviewed and and corrective action instituted as appropriate.corrective action instituted as appropriate.

Review Results to Prevent FailureReview Results to Prevent Failure

First InstrumentFirst Instrument

First Instrument

-1.5

-1

-0.5

0

0.5

1

1.5

SO-01 SO-02 SO-03 SO-04 SO-05

Difference

Upper Limit

Lower Limit

Second InstrumentSecond InstrumentSecond Instrument

-1.5

-1

-0.5

0

0.5

1

1.5

SO-01 SO-02 SO-03 SO-04 SO-05

Difference

Upper Limit

Lower Limit

Evaluation ReviewEvaluation ReviewIs There Something Wrong?Is There Something Wrong?

Reviewing Ungraded/Educational Reviewing Ungraded/Educational ChallengesChallenges

HFC-B Neutrophils

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

HFC-03 HFC-04

Result

Target

Low er Limt

Upper Limit

Reviewing Ungraded/Educational Reviewing Ungraded/Educational Challenges ContinuedChallenges Continued

HFC-B Lymphocytes

0.0

10.0

20.0

30.0

40.0

50.0

60.0

HFC-03 HFC-04

Result

Target

Low er Limt

Upper Limit

Reviewing Ungraded/Educational Reviewing Ungraded/Educational Challenges ContinuedChallenges Continued

HFC-B Monocytes

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

HFC-03 HFC-04

Result

Target

Lower Limt

Upper Limit

Proficiency Testing Exception Proficiency Testing Exception Summary (PTES) Summary (PTES)

Unsatisfactory PT performance Unsatisfactory PT performance Failure to attain the minimum satisfactory score for an analyte, test, Failure to attain the minimum satisfactory score for an analyte, test,

subspecialty, or specialty for a testing event. subspecialty, or specialty for a testing event. Unsuccessful PT performance (2 of 3 testing events)Unsuccessful PT performance (2 of 3 testing events)

Failure to attain the minimum satisfactory score for an analyte, test, Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for two consecutive or two of three subspecialty, or specialty for two consecutive or two of three consecutive testing events. consecutive testing events.

Critical PT performance (3 of 4 testing events)Critical PT performance (3 of 4 testing events) Failure to attain the minimum satisfactory score for an analyte, test, Failure to attain the minimum satisfactory score for an analyte, test,

subspecialty, or specialty for three consecutive or three of four subspecialty, or specialty for three consecutive or three of four consecutive testing events. A laboratory must immediately cease consecutive testing events. A laboratory must immediately cease testing for that analyte or the discipline. testing for that analyte or the discipline.

PTES ResponsePTES Response

College of American Pathologists How to Respond College of American Pathologists How to Respond to Unsatisfactory Proficiency Testing Eventsto Unsatisfactory Proficiency Testing Events

ExampleExample

Reasons for FailureReasons for Failure

Review Images On-LineReview Images On-Line

Use Images for TrainingUse Images for Training

ExampleExample

All Analyte Score Card All Analyte Score Card What is it and How Can it Help?What is it and How Can it Help?

The all analyte score card can be filtered The all analyte score card can be filtered various waysvarious ways

Results can be downloaded into PDF or Results can be downloaded into PDF or Excel formatExcel format

Provides a cumulative summary for quality Provides a cumulative summary for quality teamteam

Example of All Analyte Score CardExample of All Analyte Score Card

CAP Accepted PT ProvidersCAP Accepted PT Providers

AccuTest (Healthmetrx Digital PT)AccuTest (Healthmetrx Digital PT)American Association of Bioanalysts (AAB) American Association of Bioanalysts (AAB) American Proficiency Institute (API)American Proficiency Institute (API)American Society of Histocompatibility and American Society of Histocompatibility and Immunogenetics (ASHI)Immunogenetics (ASHI)College of American Pathologists (CAP)College of American Pathologists (CAP)Control Laboratory PT Program Control Laboratory PT Program South-Eastern Organ Procurement Foundation South-Eastern Organ Procurement Foundation (SEOPF)(SEOPF)Wisconsin State Laboratory of Hygiene Wisconsin State Laboratory of Hygiene Proficiency Testing Program (WSLH)Proficiency Testing Program (WSLH)

CAP LAP AudioconferenceCAP LAP Audioconference

POCT and CAP AccreditationPOCT and CAP Accreditation Selwyn J. Baptist, MD, FCAP Selwyn J. Baptist, MD, FCAP

September 20, 2006September 20, 2006

1:00 – 2:00 pm 1:00 – 2:00 pm

Free enrollment to institutions accredited by CAPFree enrollment to institutions accredited by CAP

AACC Southeast POC ChapterAACC Southeast POC Chapter

Point-of-Care-Testing: Evidenced Based Point-of-Care-Testing: Evidenced Based Practices and Regulatory IssuesPractices and Regulatory Issues

November 2-4, 2006November 2-4, 2006

Francis Marion HotelFrancis Marion HotelCharleston, South Carolina Charleston, South Carolina

http://www.pointofcare.net/AACC_Southeast_POC_Meeting.htmhttp://www.pointofcare.net/AACC_Southeast_POC_Meeting.htm

Central Florida POC NetworkCentral Florida POC Network

6th Annual Health First Point-of-Care Conference6th Annual Health First Point-of-Care Conferenceand Exhibitionand Exhibition

Friday, November 17th, 2006

Cape Canaveral Hospital inCocoa Beach, FL

http://www.pointofcare.net/Central_Florida/November2006Meeting.htm

ReferencesReferences

GP27-A Using Proficiency Testing (PT) to GP27-A Using Proficiency Testing (PT) to Improve the Clinical LaboratoryImprove the Clinical LaboratoryCollege of American Pathologists How to College of American Pathologists How to Respond to Unsatisfactory Proficiency Respond to Unsatisfactory Proficiency Testing EventsTesting EventsWhat to do With PT Providers – 2004 LAP What to do With PT Providers – 2004 LAP Audioconference – Gerald Hoeltge, MDAudioconference – Gerald Hoeltge, MDCAP Laboratory Accreditation Program CAP Laboratory Accreditation Program ManualManual

Thank YouThank You

Contact InformationContact InformationElaine Staley, MT(ASCP)Elaine Staley, MT(ASCP)

E-mail – E-mail – staley.elaine@mayo.edustaley.elaine@mayo.edu