PROFICIENCY TESTING MANUALProficiency Testing Manual College of American Pathologists 4 General...

transcript

PROFICIENCY TESTING MANUAL

1 General Customer Information .................. 3–10• OrderConfirmation ...............................................3• RegulatoryReporting ............................................3• Binders/Glossaries ...............................................4• KitDeliveryandStorage .......................................4• CAPIdentificationNumber .....................................4• ReplacementSpecimens ....................................... 4

• TestingInstructionsandCompletionTime ..................5• SecondInstrumentReporting ..................................5• CorrectionstoResultsandCMSReporting

Instructions .........................................................6• Fax-BackResponsePrograms ................................7• Evaluations .........................................................7• CorrectionstoResults ...........................................7• CustomerSupport ................................................7• ProgramCertificates .............................................8• LimitationsofPT ..................................................9• LaboratoryAccidents ............................................9• LimitationsofPTLetter ....................................... 10

2 Result Form Data Entry ..............................11–13• Overview .......................................................... 11

• PreprintedMethodSummaryPage ........................ 11

• ExceptionCodes ................................................ 12

• MasterLists ...................................................... 12

• ReportingBoxesandDecimalPrecision .................. 13

• “LessThan”or“GreaterThan”Values ..................... 13

3 e-LAB Solutions Suite (ELSS) .................. 14–16• GettingStarted .................................................. 14

• OnlineDataSubmission ...................................... 15

• OnlineReports .................................................. 16

• PerformanceAnalyticsDashboard ......................... 16

Table of ContentsProficiency Testing (PT) Manual

4 How to Interpret Your Evaluation ............. 17–20• GeneralGuidelinesforEvaluation .......................... 17

• SelectionofaTargetValue ........................................... 17

• CalculationofSummaryStatistics .......................... 18

• ComparativeStatistics ......................................... 20

5 Continuing Education (CE) ....................... 21–22• ContinuingMedicalEducation(CME)Category1 ...... 21

• CEforNonphysicianLaboratoryPersonnel .............. 21

• CEforCytotechnologists...................................... 22

6 Discipline-Specific Reporting Information ................................................. 23–25

• HowtoCompletetheResultForm .......................... 23

7 Evaluations and Participant Summaries (PS) .............................................................. 26–29

• Overview ..................................................... 26

• ReviewingYourEvaluation .................................. 26

• PS .................................................................. 28

• HowtoPerformaSelf-Evaluation .......................... 29

8 Laboratory Legislation and Centers for Medicare & Medicaid Services (CMS) Reporting .................................................. 30–37• ProvisionofResultstoCMSandStateAgencies ....... 33

• UseofReasonCodesforNonevaluatedSpecimens .. 35

• CMSPerformanceSummary ............................... 36

• CMSPerformanceSummaryData ......................... 36

InterlaboratoryComparisonProgramCycle

1 Catalog Delivered

September:Thecatalog issenttoyourlaboratory.

2 Orders Completed

September–December: Yourlaboratoryplacesitsorder.

3 Orders Processed

September–December: Orderquantitiesarereserved.

4 Order Confirmations Sent

September–December:Confirmationreportsaresenttoyourlaboratoryafteryourorderisprocessed.

5 Kits Mailed

Kitsarepreparedandsentfromthemanufacturertoyourlaboratory.

6 Results Submitted

CompletedresultformsarereturnedtotheCAPwheredataaresummarized.

7 Evaluation Criteria Reviewed

Thescientificresourcecommitteereviewsresultsandtheimpactofevaluationcriteria.

8 Reports Available

Reportsaresenttoandmadeavailableonlineforyourlaboratory,regulatoryagency,and/orconsultants.

9 Certificate of Participation Provided

Atthebeginningoftheprogramyear,yourlaboratoryreceivesacertificateofparticipation.

Proficiency Testing Manual College of American Pathologists 3

General Customer Information

General Customer InformationOrder ConfirmationAfteryourlaboratory’sorderisreceived,anorderconfirmationissentthatcontainsthefollowinginformation:

• Shippingaddress

• Billingaddress

• Telephoneandfaxnumber

• Listofprogramsordered

• Listofagenciesand/orconsultantstowhomyouhaverequestedcopiesofyourevaluationbesent

Reviewtheconfirmationdocumentcarefully.Ifyouhavechanges,returnthedocumentwithintwoweeksofreceiptto:

Mail: CustomerDataManagement CollegeofAmericanPathologists 325WaukeganRoad Northfield,Illinois60093-2750

Fax: 847-832-8168(countrycode:001)

Regulatory ReportingTheCAPwillautomaticallyforwardresultsforanalytesregulatedforPTtotheCMSforlaboratoriesthathaveprovidedaClinicalLaboratoryImprovementAmendments(CLIA)identificationnumber.

Tomakechangestoyourlaboratory’sanalytereportingselections(ARS),contacttheCAPat800-323-4040option1oraccessyourreportonlineatcap.org.

Documentationwillberequestedandmaybefaxedto847-832-8168.Anexplanationofregulatoryreportingandcurrentlaboratorylegislationisincludedinsection8.

Binders/GlossariesToprovideyouwithaneasywaybywhichtostoreyourSurveysprogrammaterials,foreverynineeligibleprograms,onethree-ringbinderwithtabsandcustomlabelswillbesenttoyouatnocharge.Thereisnoneedtoorderthebindersastheywillbeshippedautomaticallyuponplacementofyourorder.Additionalbindersareavailableforpurchasethroughtheonlinestore.

Ifyouareenrolledinhematologyand/orclinicalmicroscopySurveyscontainingphotographsformorphologicidentification,youwillbeabletoaccessanonlineglossaryoftermsforyourgeneraluse.Ifyouneedahardboundcopy,calltheCAP.

Kit Delivery and StorageRefertotheshippingscheduleandadviseyourreceivingdepartmentaccordingly.Ashippingscheduleisprovidedtoyourlaboratorywiththeorderforms,oryoucanobtainacustomizedshippingcalendarbyloggingintoe-LABSolutionsSuite(ELSS)atcap.organdclickingonMy PT Shipping Calendar.

ThereceivingdepartmentofyourhospitalorlaboratoryshouldbeadvisedhowtohandletheCAPkits.Insistonprompttransfertothelaboratory.Kitsreceivedbythehospitalbutnotdeliveredtothelaboratoryarenoteligibleforfreereplacement.Unlessotherwisespecified,storethespecimensintherefrigerator.

CAP Identification NumberEachSurveyparticipantreceivesaCAPidentificationnumberthatisprintedonallresultforms.Thisnumberwillalsoappearoneachevaluationreceivedbyyourlaboratory.ItishelpfultohavethisnumberavailablewhencontactingtheCAP.

Replacement Specimens Kitsmaycontainspecimensforanalysis.Checkthecontentsagainstthekitinstructionsuponreceipt.Ifthekitisincompleteorcontainsbrokenorunlabeledspecimens,contacttheCAPassoonaspossibleforafreereplacement.

Intheeventthatareplacementspecimenisrequired,retainyouroriginalresultformwhileawaitingthearrivalofthereplacementspecimens.Thereplacementspecimenswillbesentinthesamemannerasyouroriginalspecimens.Whenyoureceivethereplacementspecimens,itisimperativethatyoucompletetestingassoonaspossibletocomplywiththeoriginalduedate.Duetoregulatoryrestrictions,aduedateextensionmaynotalwaysbepossible.Occasionally,itmaynotbepossibleforthemanufacturertoreplaceyourspecimen(s)orkit.Inthiscase,filltheexceptioncode33bubbleontheresultform.Anotewillappearonyourevaluationreportandyouwillnotbepenalized.

BecausePTmaterialsmustbeprocuredinadvanceofshipment,onoccasion,additionalinventoryisavailableforanominalfee.AdditionalmaterialmaybepurchasedafterthecloseoftheformalPTevent.

TopurchasethesematerialscontacttheCAPat800-323-4040option1.Allliteratureassociatedwiththisproductincludingthesummarydatawillbeprovided.This option does not replace routine PT.

Thesematerialsmaybeusedforbutnotlimitedto:

• Competencyassessment

• Instrumenttroubleshooting

• Training

• Education

• Research

PT Referral InformationPerCLIA,aspublishedbytheUnitedStatesFederal Register,PTspecimensmustbetestedwiththelaboratory’sregularworkload,usingroutinemethodsandtestingthePTspecimensthesamenumberoftimesitroutinelytestspatientspecimens.

WhenhandlingPTspecimens,laboratoriesmustnotcommunicateresultsnorshareorreferspecimensfortestsnotonthelaboratory’smenu.Ifreferralfortestingisroutinelyperformedforpatientspecimens,thepracticecannotbefollowedforPTspecimens.ReferralisconsideredtobemovementofthespecimenfromalaboratorywithaCLIAidentificationnumbertoanotherlaboratorythathasadifferentCLIAidentificationnumber.LaboratoriesmustensurethatpersonneldonotshareresultsorreferPTspecimensforanyreflexortestingoutsidetheirCLIAidentificationnumber.

Results Due Date TheSurveysprogramisusedforcertificationofcertainlaboratories.Sincepromptnessisconsideredindeterminingcertification,wecannotacceptlateentries.Results must be received at the CAP no later than midnight, Central Time by the due date on the result form.Resultformsreceivedbeyondthedatenotedwillnotbeevaluated.Participantswillreceiveanevaluationindicatingthattheresultswerereceivedpasttheevaluationcut-offdatealongwithaPSthatcanbeusedforself-evaluation.

Second Instrument ReportingIn2014,theCMSdirectedallPTprovidersthatlaboratoriessubjecttoCLIAregulationsarenotpermittedtotestPTspecimensonmultipleinstrumentsunlessthatishowthelaboratoryroutinelytestspatientspecimens.

Recently,theCMScommunicatedtoapprovedPTprovidersthatthis directive applies to all analytes,includingthosenotlistedinSubpartIoftheCLIAregulations,aswellasanalytes/methodscategorizedaswaivedtests(suchaswholebloodglucosemeters).

Becausethesanctionsaresevereiftheregulationsarenotfollowed,theCAPPTprogramnolongerincludestheoptionforsecondinstrument.Tomeetyourlaboratory’sneeds,theCAPnowoffersitsQualityCrossCheckprogram.SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogortheRegulatoryInformationpageviacap.orgforfurtherdetails.

Corrections to Results• Correctionstoresultscanbemadeanytimepriortotheduedatelistedontheresult

form;reviewyourlaboratorydatacarefullypriortosubmission.Forresultsapprovedonline,correctionsmustalsobemadeonline.Faxedormailedcorrectionswillnotbeaccepted.

• Toviewthestatusofyourresults,gotocap.organdclickonSubmit/EnterPTResults.

• Foranytestingthatyoudonotroutinelyperforminyourlaboratory,leaveallreportingareasforthattestblank,includingmethodinformation.ExceptionsarenotedintheCriticalReportingInformationsectionofthekitinstructionsorontheresultform. Note:Apenaltywillnotbeappliedforblankresponsesinthecaseofeducationalchallenges,challengesnotformallygraded,ortheproperuseofexceptioncodes.

CMS Reporting InstructionsForanyregulatedanalytesthatyourlaboratorydoesnotreportormayhavediscontinued,youmustupdateyourreportingsettingsviatheonlineCMSAnalyteReportingSelectionstoolviaELSS.Additionalregulatoryinformationcanbefoundviacap.orgontheProficiencyTestingpageunderLaboratoryImprovement.Ifyouhaveanyquestions,callthecustomercontactcenterat800-323-4040option1.

Fax-Back Response ProgramsTheCAPoffersimmediate(fax-back)responsesforthesubmissionofthelaboratoryformsforthefollowingeducationalanatomicpathologyprograms:

• InterlaboratoryProgramsinGynecologicandNongynecologicCytopathology(PAP/NGC)

• Fine-NeedleAspirationGlassSlideProgram(FNAG)

Individualresultformscanbesubmittedonline(preferredmethod)orfaxedtotheCAP.CME/CEcertificatesmaybeself-claimedviacap.orgbyclickingontheLearningtab.

EvaluationsEvaluationsarepostedonlineandmailedapproximately2–5weeksaftertheshipdateofthekit.Thistimeisneededforprocessingdata,establishingevaluationcriteria,andpreparingtheparticipantsummary(PS).

Customer SupportRegularHours:Monday-Friday,7:00am –5:30pm CT

ExtendedHours:Monday-Friday9:00pm –5:00amCT(emailonly)

Contact us to:

• Updatecontactinformation(telephonenumber,emailaddress,faxnumber,oraddress)

• Receiveadditionalsupportinginformationordocumentsfororderingorimportingpurposes

• AskaCAPexpertquestionsaboutaspecificCAPprogram

Phone: 001-847-832-7000option1

Email: Gotocap.organdselectCONTACT & SUPPORTatthebottomofthehomepage.

Sendanemailtocontactcenter@cap.org

Laboratory Accident Hotline:

AccidentalexposureswhileprocessingspecimensfromtheCAP,includingcutsandcontaminationofmucousmembranesornon-intactskin,shouldbereportedtotheCAPhotlineforevaluationandquestionsregardingprophylaxis.Pleaseprovidetheidentificationnumberoftherelevantspecimen.

Phone:

001-847-832-7000option1 DuringContactCenterhours

001-847-470-2812 Allotherhours

IncountrieswheretheCAPhasadesignateddistributor,directlycontactyourCAPdistributorifyouhavequestionsrelatedtopricingoranyotherinquirynotlistedabove.

YourlaboratorywillbenotifiedofkitshipmentviaCAPTRAKerSMandemail.Ifyoudonotreceiveyourkitwithin7–10calendardaysfollowingtheshipdate,orifyoureceiveanincompleteordamagedshipment,callthecustomercontactcenter.Requestsforreplacementsafterthe10thcalendardaycannotbeaccepted.

Fordocumentacknowledgment,includetheappropriateemailaddressonthefirstpageofyourorderforminthesectiontitled“PTShippingContact.”

Tochoosenottoparticipateinthisprogram,participantscancontacttheCAPat800-323-4040option1.

Program CertificatesAfterthecompletionoftheprogramyear,participatinglaboratorieswillreceiveacertificaterecognizingeachinstitution’sparticipationintheCAPPTprogramanditscommitmenttopatientcare.CertificatesaresignedbytheCAPpresidentandaresuitableforframing.

Deficiency ResponseTheCAPdoesnotrequirethatyousubmitdocumentationforallPTdeficiencies.However,itisrecommendedthatsuchdocumentationberetainedinyourlaboratory.TheCAPLaboratoryAccreditationProgramissuesaseparatereport,the“ProficiencyTestingExceptionSummary,”thataddressesdeficienciesforCAP-accreditedlaboratories.Instructionsforresponsewillbeincludedwiththereport.

Handle With CautionPTspecimensmustbehandledwithcaution.Eachshipmentincludesabiohazardwarningstatementexplainingproperhandling.

Laboratory AccidentsIncidentsofpersonnelexposuretoinfectiousspecimens,throughneedlesticks,contaminationofthemucousmembranesthroughsplashesoraerosolization,orcutsfromcontainers,shouldbereportedimmediatelytotheCAP.

24-hourhotline:800-443-3244

Havethefollowinginformationavailable:

• CAPnumber

• Phonenumber

• Nameofinstitution/city/state

• Nameofpersonaffected,ifotherthancaller

• Dateandtimeofincident

• Whereandhowaffected

• Surveyandspecimennumber

• Nameandtelephonenumberoflaboratorydirector

Thisinformationwillberelayedtoapathologistmemberoftheappropriateresourcecommitteewhowillcontacttheparticipant’slaboratorydirectororhospitalemployeehealthservicesphysicianwithinstructionsconcerningprophylaxis.

Limitations of PTDuetothemanufacturednatureofthespecimensandthelogisticsofshipping,PTdoesnotalwayscorrelatewiththemannerinwhichfresh,clinicalspecimensarehandled.Aletteraddressingthesedifferencesisincludedforgeneralusebyyourlaboratory.

Limitations of PT Letter

The College of American Pathologists (CAP) Surveys program is the largest externalquality assessment program in theworld.As such, it provides an unparalleled selectionofchallengesandoffersthelargestdatabaseinexistenceforinterlaboratorycomparison.TheCAPhasaccumulatedsignificantexperienceinmanagingthistypeofprogramandisknowledgeableinitsusesandlimitations.

Performance onCAPSurveys is not to be taken as the sole indicator of a laboratory’sabilities.AproficiencytestingSurveyisbutoneofanumberofprogramsthatlaboratoriesshould employ to assess, manage, and improve quality. In addition to Surveys, propermethodvalidation,qualitycontroltesting,periodiccalibrationandinstrumentmaintenance,employeecompetencytesting,andlaboratoryinspectionandaccreditationprovideimportanttoolsformeasuringlaboratoryperformanceandensuringquality.

TheSurveysprogram,althoughoutstanding,isnotaperfectmeasuringdevice.Anumberof factors limit this tool’s ability to measure laboratory accuracy. Specific limitationsinclude requisite use of matrix materials that may impact test systems differently thanpatientspecimens;theappropriatenessofgroupingresponsesaccordingtomethodology,instrumentation, and test platforms; varying size of comparison groups with attendantvariabilityofstatisticalparameters;regulatedlimitationsinsamplingoflaboratories’testingsystems;difficultiesinquantitationattheextremesofanalyteconcentration;andunsuitabilityofcertainfederally-mandatedevaluationlimits.

Thus,acertainnumberof responses thataregradedasunacceptable inSurveyswill infactbeacceptable,andacertainnumberofresponsesgradedasacceptablewillinfactbeunacceptable.Although unsuccessful or unsatisfactorySurveys performancemay reflectproblemswithinalaboratory,itdoesnotconstituteproofofinadequateperformanceoraninabilitytomeetpatientneeds.

Sincerely,

RaoufE.Nakhleh,MD,FCAP

Chair,CouncilonScientificAffairs

Limitations of PT Letter

The College of American Pathologists (CAP) Surveys program is the largest externalquality assessment program in the world. As such, it provides an unparalleled selectionof challenges and offers the largest database in existence for interlaboratory comparison.The CAP has accumulated significant experience in managing this type of program and isknowledgeableinitsusesandlimitations.

Performance on CAP Surveys is not to be taken as the sole indicator of a laboratory’sabilities.Aproficiency testing Survey is but one of a number of programs that laboratoriesshould employ to assess, manage, and improve quality. In addition to Surveys, propermethod validation, quality control testing, periodic calibration and instrument maintenance,employee competency testing, and laboratory inspection and accreditation provide importanttoolsformeasuringlaboratoryperformanceandensuringquality.

The Surveys program, although outstanding, is not a perfect measuring device. A numberof factors limit this tool’s ability to measure laboratory accuracy. Specific limitationsinclude requisite use of matrix materials that may impact test systems differently thanpatient specimens; the appropriateness of grouping responses according to methodology,instrumentation, and test platforms; varying size of comparison groups with attendantvariability of statistical parameters; regulated limitations in sampling of laboratories’testingsystems; difficulties in quantitation at the extremes of analyte concentration; and unsuitabilityofcertainfederally-mandatedevaluationlimits.

Thus, a certain number of responses that are graded as unacceptable in Surveys will infact be acceptable, and a certain number of responses graded as acceptable will in fact beunacceptable. Although unsuccessful or unsatisfactory Surveys performance may reflectproblems within a laboratory, it does not constitute proof of inadequate performance or aninabilitytomeetpatientneeds.

Sincerely,

RaoufE.Nakhleh,MD,FCAP

Chair,CouncilonScientific Affairs

2018 Proficiency Testing Manual College of American Pathologists 9

Result Form Data Entry

Result Form Data EntryOverviewTheresultformisadocumentonwhichyourecordyourmethodsofanalysisandPTdata.Thecompletedresultformmustbereturnedonline(preferred)viaELSSorbyfaxtotheCAPbytheduedatelistedontheform.

RetainacopyofthecompletedresultformforyourrecordsbeforesubmittingittotheCAP.

PertheCentersforMedicare&MedicaidServices(CMS),changestosubmitteddatacannotbemadeaftertheduedateontheresultform.Reviewallinformationonthepreprintedmethodsummarypageandallentriesmadeontheresultformforaccuracypriortosubmission.Onceyouhaveenrolled/registered,logintoELSSviacap.org,andclickonSubmit/EnterPTResultstoviewthestatusofyoursubmitteddata.

Preprinted Method Summary PageOnceyourlaboratoryinitiallyprovidesamasterlistcodeforamethod,instrument,and/orreagent,theCAPwillretainthisinformation,nolongerrequiringyoutoreportthemrepetitivelythroughouttheyear.Carefullycheckeachmasterlistselectiontoensurethecorrectcodesarelisted.Tochangeacode,enteritintheappropriateboxesontheresultform.

Exception CodesIfyourlaboratoryneedstoreportananalyticalproblemforanentiretestorindividualspecimenswithinatest,leavetheresultareablankandfilltheappropriatetwo-digitexceptioncodebubbleontheresultform.

Exception Code Reason

11 Unabletoanalyze(documentationtobeprovidedbylaboratory).

22 Resultisoutsidemethod/instrumentreportablerange.

33 SpecimendeterminedtobeunsatisfactoryaftercontactingtheCAP.

Itisthelaboratory’sresponsibilitytodocumenttheappropriateuseofexceptioncodesifrequestedduringalaboratoryinspection.Refertothekitinstructionsformoreinformation.

Master ListsChoosetheappropriatemethodorinstrumentprovidedonthekitinstructionsorresultform.

NotifytheCAPandthemanufacturerifyourmethodorinstrumentisnotlisted.Ifthemanufacturerofyourmethodorinstrumentisnotlistedonthekitinstructionsorresultform,enteritinthe“UseofOther”sectionattheendoftheresultform.

Identificationmasterlistsareprovidedformicrobiology,bloodcellidentification,urinesediment,clinicalmicroscopy,andprovider-performedmicroscopy.Selectacodefromtheappropriatemasterlistandenteritontheresultform.

Forbloodcellidentification,urinesediment,clinicalmicroscopy,andprovider-performedmicroscopy,allpossibleidentificationsareincludedonthemasterlists.Do not usethecode010,“Other,Specify.”Theuseofthiscodewillbeevaluatedasanunacceptableresponse.

HTML SectionTheCAPhasimplementedamorecustomer-friendlyonlineinterfaceforenteringresultsviaELSS.TheenhancementprovidesonlineresultformsinanHTMLformat,replacingthePDFtechnology,butretainsthedesign.Theformhelps:

1. Reducecommonclericalerrors

2. Increasedataaccuracy

3. Approveonlinedataefficiently

4. Searchandnavigatewithease

What is different about the form?

Itprovidesvalidationforrequiredinformation.Forexample,ifavalue/resultispopulatedforanyspecimenforasingletest,thenavalue/resultmustbepopulatedformanufacturer,method,instrument,orreagent.ItalsopreventsincompletedatafrombeingsubmittedtotheCAP.Forexample,ifyou:

• Donotenteraunitofmeasureintheonlineresultform,apop-upmessagewillappearandaskyoutoprovidetherequiredinformation.

• Enteravalue/resultinanincorrectformat(eg,###.#insteadof###.)ontheonlineresultform,apop-upmessagewillappearandaskyoutorevisetheinformation.

Inaddition,theform:

• Allowsnavigationbytest(forlaboratoriesthatdonotreporteverytestinaPTevent)

• Providesanapprovalbuttononthetopofeachresultformpage

• Aimstolessenbrowsercompatibilityissues.

‘Less Than’ or ‘Greater Than’ ValuesDonotattempttoadd“lessthan”or“greaterthan”tothevalueyousubmitunlessthisoptionisprovidedontheresultform.Wheretheoptiontoreport“greaterthan”or“lessthan”isgiven,youmustselectthebubbleunderneaththeappropriatereportingboxtoindicateyourresponseisa“lessthan”or“greaterthan”value.Allotherresultswillbeconsidered“equalto”values.

Wherenooptiontoreport“greaterthan”or“lessthan”isgiven,refertotheExceptionCodessectionortheGeneralReportingInformationsectionofthekitinstructions.

e-LAB Solutions Suite (ELSS)

e-LAB Solutions Suite (ELSS)Getting StartedELSSistheCAP’sonlineportaltomanageyourlaboratoryimprovementprograms.Inordertotakefulladvantageofthefunctionalitiesavailableonline,userscanselectivelyenablelaboratoriesandpersonnelby:

Creating an Online AccountInordertoaccessELSS,youmustcreateapersonalWebaccountbygoingtocap.organdclickingonLogin.UnderNewUsers,clickonCREATEANACCOUNT;thisonlyneedstobedoneonceperuser.

Logging InOnceyouhaveestablishedanonlineaccount,youwillbepromptedtoenteryouruserIDandpassword.Thismustbedoneeverytimeyouvisitcap.org.Bothmembersandnonmembersutilizethesameloginfunctionality.

Enroll/RegisterWhenyourlaboratoryfirstplacesanorder,thelaboratorydirectorwillreceiveanenroll/registeremailwithaCAPNumberandPIN#.Thelaboratorydirectormustdesignatewhowillbethelaboratory’ssiteadministrator.Yourlaboratoryonlyhastoenroll/registeronce.Theenrollment/registrationprocessestablishesthelaboratory’sonlineaccesstoCAPprogramsthroughELSSanddefinesthesiteadministrator’sresponsibilities.

Requesting AccessIndividualizedaccountsforlaboratoryusersallowflexibilityindeterminingaccesslevels.Becauseusersareindependententitiesfromthelaboratory,userscanbeaffiliatedwithmultiplelaboratoriesandtheirsecuritycanbeadministeredinadifferentmannerateachsite.

Torequestaccesstoalaboratory,gotocap.org,clickonAccesse-LABSolutionsSuite,andlogin.ClickonRequestPermissionfromYourSiteAdmin,andenteryourlaboratory’sCAP#.

IndividualprivilegescanbeaccessedbyclickingonView/UpdateMyLabPermissions.

e-LAB Solutions Suite (ELSS)

Online Data SubmissionOnlineresultformsmakedataentryfasterandmoreaccurate.YourlaboratorywillminimizePTfailuresondataentrythroughtheuseofanenhancedHTML-basedresultform.Youcanalsoverifyreceiptofdataandviewfaxedormaileddata.

AutomatedPTdatareportingisavailablethroughe-LABSolutionsConnect.TheCAPhaspartneredwithDataInnovationsInstrumentManager™middlewaresolutiontohelpfacilitatetheconnectionandtransmission.Withe-LABSolutionsConnect,theCAP’squantitativePTresultsareautomaticallytransmittedfromyourlaboratory’sinstrumentsorlaboratoryinformationsystemtoELSS.

Ifnecessary,faxedresultsmustbesentto866-FAX-2CAP(866-329-2227).

Online ReportsWhentheresultformduedatehaspast,dataareprocessedandgradedaccordingtostringentgovernmentalandcommitteecriteria.LaboratoryresultsandgradinginterpretationsaredisplayedinanindividualizedreportaccessiblebyclickingonEvaluationReportsunderProficiencyTesting/QualityManagement.Usingthefilteroptions,selecttheprogramandreport.

Theonlineevaluationallowstheusertoeasilynavigatethedataanalytebyanalyte.Also,usersbenefitbytheinclusionofdetailedimageshyperlinkedonthereport.LaboratorymanagerscanaccesstheAllAnalyteScorecard,allowingthemtocustomizethescorecarddatafortheirlaboratoryasameansofidentifyingdeficienciesortrendsinperformance.

Otherancillaryreportsthataccompanytheevaluationareavailableonline,aswell,forreviewandtodownloadforfuturereference,includingthePS,finalcritiques,andannualsummaries,whichcontainusefuldataandeducationthatcanbereviewedandaccessedbyalluserswithappropriatesecurity.

Performance Analytics DashboardOurPerformanceAnalyticsDashboardgivesthecomprehensivedataviewyouneedtoproactivelyfocusenergyonareasthatneedimmediateattentionwhilefilteringoutdistractions.Updateddaily,thisWeb-basedreportingsolutionforyourCAPPTandaccreditationperformancereducesthestressofmanagingtoday’ssinglelaboratoryoranexpansivenetwork’sperformancewithonlyafewclicks.

How to Interpret Your Evaluation

How to Interpret Your EvaluationThischapterincludesgeneralinformationregardingevaluations.Asectionexplaininghowtointerpretyourindividualevaluationwillbeincludedwitheachmailing.

General Guidelines for EvaluationOnFebruary28,1992,theSecretaryofHealthandHumanServices(HHS)publishedthefinalrulesimplementingtheCLIAof1988.TheseregulationsestablishedevaluationcriterialimitsformanyoftheanalytesincludedintheCAPSurveysprograms.ThetargetvaluesaredeterminedbythescientificresourcecommitteesoftheCAP.ForthoseanalytesnotincludedinthePTportionofCLIA,thetargetvaluesandevaluationcriteriaaredeterminedsolelybythescientificresourcecommittees.

Selection of a Target ValueTominimizetheeffectofmethoddifferencesandtoallowcomparisonofallmethods,participants’resultsarecombinedintocomparablemethod/instrumentgroupscalledpeergroups.Itisimportantforparticipantstoprovidecompleteinformationregardingthemethodorinstrumentusedinordertobecombinedintheappropriatepeergroup.

Formostanalytes,thepeergroupmeanisdesignatedasthetargetvalueforevaluation.Thepeergroupmeanisthepreferredtargetifnosingletargetvalueexiststhatcanprovideanaccuracy-basedtargetthatistraceabletothe“truevalue”asdeterminedbyadefinitiveorreferencemethodanalysis.Thepeergroupmustconsistofgreaterthannineresultsafteroutlierexclusion,andthevariabilityofthepeergroupdatamustnotbetoogreat.

Ifpeergroupdataarenotavailableoraretoovariable,methodgroupstatisticsmaybeused.Themethodgroupmustalsoconsistofgreaterthannineresultsanddemonstrateacceptablevariabilitybeforeitisusedasthetargetgroup.

Ifpeerandmethodgroupstatisticsarenotavailable,acomparativemethodgroupmaybedesignatedasthetargetmean.ThecomparativemethodisnotthemethodrecommendedbytheCAP;however,itisestablishedasahistoricallyreliablemethodandisusedforevaluatingresultsfrommethodsthathaveaninsufficientnumberofparticipantstogenerateaseparatepeergroupand/ormethodgroupstatistics.Ifnocomparativemethodexistsfortheanalyte,resultswillnotbeevaluated.

Forsomeanalytes,asingletargetvalueisusedinwhichconsistentresultsare

demonstratedacrossallpeergroups.

Calculation of Summary Statistics

Peer Group Results Resultsaregroupedaccordingtothemethodusedforanalysisandscreenedforoutliers.Variousstatisticsarecalculatedfromtheremainingdatathatsummarizethepeergroup’sresponses.Thesesummarystatisticsmayincludethefollowing:

• themean(theaverageofthereportedresults)

• thestandarddeviation(ameasureofthevariabilityoftheparticipantresults,oftenabbreviatedasSD)

• thecoefficientofvariation(CV)

• themedian(themiddlevalueinanorderedlistofthenon-outlierresults)

• thelowvalue(thelowestvaluereported)

• thehighvalue(thehighestvaluereported)

• thefinalcountofreportedresultsthatwerenotexcludedasoutliers.

Outlier Detection TechniqueOutlierexclusionisnecessarybecausealargeseriesofresultsfrequentlywillincludesomeaberrantvalues.Thesemayarisefrominstrumentmalfunction,technicalerrors,specimenmix-ups,misplaceddecimals,incorrectunitsofmeasure,ordataentryerrors.Ifanyresultsareexcluded,theoutlierprocessisrepeatedusingtheremainingvalues.Thesummarystatisticsthatappearonyourreportsdonotreflectresultsthatwereconsideredtobeoutliersduringeitheroutlierpass.

Quantitative Procedures/RoundingAllquantitativeresponsesareevaluatedbasedonarangeofacceptability.Thisrangeisdeterminedusingatargetvalueandalimit.Thelimitwillbeeitherafixedinterval(eg,±5mg/dL),apercentageofthemean(eg,±25%),anSD(eg,±3SD),oravariablerange(eg,±6mg/dLor10%,whicheverisgreater).ThePSincludedwithyourevaluationsthecriteriausedtoevaluateyourperformance.Thefollowingsectionsprovidesspecificexamplesofhowtocalculatetherangeofacceptabilitydependinguponthecriteriaused.

Benefit-of-the-doubtroundingisusedtodeterminetherangeofacceptableresults.Theupperlimitofacceptabilityisobtainedbyroundinguptothenextreportableresult,whilethelowerlimitisdeterminedbytruncating.

Fixed Range ExampleYourlaboratoryreportsasodiumresultof138mmol/L.Thepeergroupmeanis139.5mmol/L.

Theevaluationlimitforsodiumis±4mmol/L.Theacceptablerangeisdeterminedbytheformula139.5mmol/L±4mmol/L,whichis135to144mmol/L;therefore,yourreportedresultof138mmol/Liswithinthecalculatedacceptablerangeof135to144mmol/Lwhenusingbenefit-of-the-doubtrounding.

Percentage of the Mean ExampleYourlaboratoryreportsanalbuminresultof3.1mg/dL.Thepeergroupmeanis3.39mg/dL.Theevaluationlimitforalbuminis±10%.Tenpercentof3.39mg/dLis0.34mg/dL.Theacceptablerangeisdeterminedbytheformula3.39mg/dL±0.34mg/dL,whichis3.0to3.8mg/dLwhenusingbenefit-of-the-doubtrounding;therefore,yourreportedresultof3.1mg/dLiswithinthecalculatedacceptablerangeof3.0to3.8mg/dL.

Standard Deviation ExampleYourlaboratoryreportsathyroidstimulatinghormone(TSH)resultof16.4µU/mL.Thepeergroupstatisticsareasfollows:mean=15.7µU/mL,SD=1.5,andCV=9.6.TheevaluationlimitforTSHis±3SD.3x1.5=4.5.Theacceptablerangeisdeterminedusingtheformula15.7µU/mL±4.5µU/mL,whichis11.2to20.2µU/mLwhenusingbenefit-of-the-doubtrounding;therefore,yourresultof16.4µU/mLiswithintheacceptablerangeof11.2to20.2µU/mL.

Variable Range ExampleYourlaboratoryreportsatotalbilirubinresultof4.5mg/dL.Thepeergroupmeanis4.68mg/dL.Theevaluationlimitfortotalbilirubinis±0.4mg/dLor20%,whicheverisgreater.Twentypercentof4.68is0.936.Sincethepercentagelimitof0.94isgreaterthantheintervallimitof0.4,thepercentagelimitisappliedtothetargetvalue.Theacceptablerangeisdeterminedusingtheformula:4.68±0.936,whichis3.7to5.7mg/dLwhenusingbenefit-of-the-doubtrounding;therefore,yourresultof4.5mg/dLiswithintheacceptablerangeof3.7to5.7mg/dL.

ThePSincludedwithyourevaluationwilllistthecriteriausedtoevaluateyourperformance.Todeterminetheacceptablerange,abenefit-of-the-doubtroundingprocedureisutilizedwhengrading.

Calculation of the Standard Deviation Index (SDI)Theevaluationlistsyourresults,thestatisticsforyourpeergroup,andyournormalizedresultsasanSDI.ThisvalueisobtainedbysubtractingthegroupmeanfromyourresultandthendividingbytheSD.

TheSDIisexpressedintermsofthenumberofSDsfromthemean,withanarithmeticsignindicatingthedirectionofthedifference.ThecalculationoftheSDInormalizesyourresultand,therefore,allowsforacomparisonofresultsfromspecimensofdifferentconcentrationsofananalyte.

WhenacomparativemethodhasbeendesignatedforSurveysanalysis,asecondsetofstatisticsislistedcomparingyourresultswiththoseobtainedusingthecomparativemethod.ThefiguresshownarethemeanandSDforthecomparativemethodandyourresultasanSDIusingthesestatistics.Itispossibleforyourresulttobedefinedas“good”performanceinyourmethodgroupandyetproduceacomparativeSDIgreaterthantwo.ThiswilloccurifyourmethodhasalargeanalyticbiasoralargeSD.ItispossibletoreceiveacomparativeSDIlowerthanthemethodgroupSDI,althoughthisrarelyoccurs.Inpractice,mostparticipantsreceivesimilarmethodgroupandcomparativeSDIs.

Comparative Statistics

Quantitative ProceduresYourevaluationcontainsplotsoftherelativedistanceofyourreportedresultsasapercentageofallowabledeviationfromthetargetvalue.Thenumericdigitindicatesthenumberofresultsataplotlocation.Thealloweddeviationmaybecalculatedasfollows:

Ifyourresultisgreaterthanthetargetmean:

Percentageof AcceptableDeviation =100x

yourresult-targetupperlimit-targetmean

Ifyourresultislessthanthetargetmean:

Percentageof AcceptableDeviation =100x

yourresult-targettargetmean-lowerlimit-

Qualitative ProceduresForqualitativeresponses,consensusagreementofrefereeorparticipatinglaboratoriesisusedforevaluation.Generally,80%agreementisrequired.

Continuing Education (CE)

Continuing Education (CE) Continuing Medical Education (CME) Category 1TheCAPisaccreditedbytheAccreditationCouncilforContinuingMedicalEducation(ACCME)toprovidecontinuingmedicaleducationforphysicians.

TheCAPdesignatestheseeducationalactivitiesforamaximumofthestatednumberofAMAPRACategory1Credits™.Physiciansshouldonlyclaimcreditscommensuratewiththeextentoftheirparticipationintheactivity.

TheAmericanMedicalAssociationhasdeterminedthatphysiciansnotlicensedintheUSwhoparticipateintheseCMEactivitiesareeligibleforAMAPRACategory1Credit™.

SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogforavailableCMEprograms.

CE for Nonphysician Laboratory PersonnelThisactivityisacceptabletomeettheCErequirementsfortheAmericanSocietyforClinicalPathology(ASCP)BoardofRegistryCertificationMaintenanceProgram.ThisactivityisapprovedforcontinuingeducationcreditinthestatesofCaliforniaandFlorida.

AllnonphysicianlaboratoryprofessionalsinyourlaboratorycanearnindividualCEcreditsbycompletingtherelatededucationreadingandonlinelearningassessmentquestionsviacap.org.

Surveys CE ProgramsDiscipline MaximumCECredits

Chemistry

The number of credits are specific to the program mailing.

Go to cap.org for up-to-date activity listings.

Coagulation

Hematology

Histology(HistoQIP)

Immunology

Microbiology

TherapeuticDrugMonitoring/Endocrinology

Toxicology

TransfusionMedicine

ReproductiveMedicine

Continuing Education (CE)

Learning Cycle InformationEacheducationactivityprovidesinformationoncommontechnicalandnontechnicalissuesencounteredinalllaboratorysettings.ToreceiveCEcredit,youmustcompletetheeducationreadingprovidedinyourPSandanswertheonlinelearningassessmentquestions.Mosteducationactivitiesareavailablefor12monthsandmustbecompletedwithinthattimeframe.CEcreditwillbeappliedtowardtheyearinwhichtheactivityiscompleted.DetailedinformationonhowtoaccesstheonlinecomponentswillbeincludedineachPS.TheCAPisanapprovedproviderofaccreditedCME.InordertoremainaCMEprovidertheCAPmustabidebyallACCMEandAMArulesforaccreditedCME.ThismeansthattheCAPcannotopenacoursetoallowcompletionofthecourseaftertheexpirationdate.

CE for CytotechnologistsCytotechnologistsmayapplythecreditsfromthegynecologicandnongynecologic(FNA,FNAG,NGC,andTICP)programstowardtherequirededucationalactivitiesfortheAmericanSocietyforCytopathology(ASC)ContinuingEducationCreditProgram.

Online Virtual Microscopy Education ProgramsTheCAPoffersonlineeducationprogramsthatuseDigitalScopetechnologytopresentwholeslideimagesfromactualglassslidesforavarietyofsitesandspecimentypes.Thistechnologysimulatesamicroscope,allowingyoutoscantheimageandusemultiplemagnificationstoviewthematerial.Fromtheimagesandclinicalinformationprovided,youselectadiagnosis,answerlearningassessmentquestions,andreceiveimmediatefeedbackonline.

SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogforavailableonlinevirtualmicroscopyeducationprograms.

Discipline-Specific Reporting Information

Discipline-Specific Reporting InformationHematologyToreportyourbloodcellidentification,selectthebestidentificationcodefromtheHematologyBloodCellIdentificationMasterListprovidedinthekitinstructions.ForfurtherdescriptionoftheHematologyBloodCellIdentificationMasterListchoices,refertotheBloodCellIdentificationsectionofthecurrentCAP Hematology and Clinical Microscopy Glossary,whichcanbeaccessedatcap.org.

Ifresultsarereportedforbothbloodcellidentificationandautodifferentials,thebloodcellidentificationwillbereportedtoCMS.

TheHematologyBloodCellIdentificationMasterListchoice,“Immaturecellorabnormalcell,wouldreferforidentification,”mustbereservedforcellsyourarelyencounterandareunabletospecificallyidentify.GradingofthisresponsewillfollowtheguidelinessetforthintheJuly26,1993,Federal RegisterNotice.

CoagulationForplasma-basedcoagulationtesting[prothrombintime(PT),activatedpartialthromboplastin(APTT),fibrinogen],aninstrumentandreagentcodearerequiredforproperevaluation.ParticipantsenrolledinwholebloodtestingforPTneedonlyindicateaninstrument(ifrequested)andtheirresults.ForallPTmodules,reportingofinternationalnormalizedratio(INR)resultsisoptional.Plasma-basedandwholebloodINRareevaluated.

UrinalysisThereareseparateurinalysisandspecificgravitymethodandinstrumentmasterlistsinthekitinstructions.Toensureanaccuratepeergroupevaluationofyourresults,itiscriticaltoprovideaccuratemethodandinstrumentinformation.

Aspecificlistofreportingoptionsisprovidedforeachurinalysisprocedure.Itisnotfeasibletoprovidealistofreportingchoicesspecificforeverypossibledipstickbeingmarketedtolaboratories.Subsequently,theresultrangeslistedmaynotexactlycorrelatewiththerangesusedwithyourinstrument/dipstick.Inthesefewcases,choosetherangethatmostcloselymatchesyourintendedresult.

Toreporturinesediment,clinicalmicroscopy,orprovider-performedmicroscopy,selectthebestidentificationcodefromtheUrineSedimentMasterListprovided.ForfurtherdescriptionoftheHematologyBloodCellIdentificationMasterListchoices,refertotheBloodCellIdentificationsectionofthecurrentCAP Hematology and Clinical Microscopy Glossary,whichcanbeaccessedatcap.org.

Urinalysis Dipstick TestsForqualitativeproceduresinurinalysis,evaluationisbasedonparticipantconsensusbymethodandinstrument.Foreachanalyte,aminimumoftwo,butnotmorethanfour,responseswillbegivenapassingscore.Analyteresultsgraded“good”performancemusthave80%participantconsensus.Eightypercentparticipantconsensuscanbedeterminedbygroupingthemodewiththenextoneortwomostfrequentresponses.Thisgroupwillbegiven“good”performance.“Acceptable”performancewillbegiventoadditionalresponsesuntilaminimumof90%ofparticipantresultsaregivenapassingscore.Inthecaseofanegativespecimen,negativeresponsesmustconstitute90%participantconsensus.Specimenswithresultsforoneormoremethodsdistributedoverbothnegativeandpositiveresponse(nonconsensus)willnotbeevaluated.Specimensforwhichthereisgreaterthan90%ofparticipantresponsesdistributedovermorethanfourresponseswillbegradedasnonconsensus.

MicrobiologyWhereappropriate,aclinicaldiagnosis,age,andsourcearelistedtosimulateatrueclinicalsituationandtoallowlaboratorypersonneltoselectappropriatemediaormethodsforprocessingthesespecimens.However,asthepathogenicbacteriapresentinanyofthesespecimensmaybeisolatedfrommultiplesourcesofthebody,allparticipantsshouldattemptidentificationoftheorganismspresentinallthesespecimens.

PertheFederal Register,aSurveymustgradealaboratory’sabilitytodistinguishbetweenapathogenandacontaminant.Culturechallengeswillbedesignatedintheinstructionstobehandledas“identifyprincipalpathogen”or“identifyallorganisms”challenges.Participantsmustreportinthismannerevenwhenthisdiffersfromtheirlaboratory’sroutinepractice.Forexample,aurinespecimencontainsKlebsiella pneumoniae andStaphylococcus epidermidis.Iftheinstructionsindicateto“Identifyallorganisms,”bothorganismsshouldbereported.Iftheinstructionsindicateto“identifyprincipalpathogen,”onlytheKlebsiella pneumoniaeshouldbereported.IftheStaphylococcus epidermidisisreported,itwouldbepenalized.

Specimenresultswillbeevaluatedif80%ormoreoftheparticipantlaboratoriesagreeontheidentificationofthetestorganism(s)togenusortogenusandspecies.Intheabsenceofparticipantconsensus,refereelaboratorieswillbeused.

TheCLIAregulationsstatethatalaboratorymustperformaminimumoffivespecimensineachtestingeventforthesubspecialtyofbacteriology.Thefivechallengescanincludeacombinationofthefollowingspecimens:

• Bacterialantigendetection

• Bacterialidentification(culture)

• Gramstain

• Antimicrobialsusceptibility

Proceduresassayedwithwaivedmethodologieswillnotcounttowardthefive-challengeminimum.ThelaboratoryisresponsibleformaintainingthefivespecimenspertestingeventforitsremainingnonwaivedtestsinthesubspecialtywhenatestiswaivedbytheFoodandDrugAdministrationmidyear.

Antimicrobial Susceptibility TestingParticipantswillbeaskedtoperformsusceptibilitytestsusingtheantimicrobialagentsandtechniquesinroutineuseintheirindividuallaboratories.Thelaboratoriesshouldreportonlyantibioticsappropriatefortherapyofinfectionsatthesiteindicatedinthepatienthistory.SeecurrentClinicalandLaboratoryStandardsInstitute(CLSI)documentsM2,M7,M100,orotherappropriatedocumentsforguidance.

Interpretationresults(susceptible,intermediate,andresistant)willbepenalizedfor:

• Agentsthatarenotclinicallyappropriateforthesiteofinfection(meningitis,pneumonia,urinarytract,etc)

• UseofmethodsCLSIadvisesagainst

• Useofmethodsthatthemanufacturerrecommendsagainstusing,duetopoorperformance

Selectivereportingforthepresumedsiteofinfectionhelpsimproveclinicalrelevance,encouragesappropriatetherapy,andhelpstominimizeselectionofresistance.

Evaluations and Participant Summaries (PS)

Evaluations and Participant Summaries (PS) OverviewShortlyafteryousubmityourPTresultstotheCAP,anevaluationofyoursubmittedresultswillbeavailableonlineormailedbacktoyou.Yourevaluationcanbeusedasaqualityassurancetooltoassesshowyouperformedcomparedtootherparticipants.Itisimportantthatyoureviewandunderstandtheinformationpresentedinyourevaluation:

• Demographicinformation:Providesinformationaboutyourlaboratory,includingthenameofyourinstitution,yourCAPidentificationnumber,andanyagenciesorconsultantsyouhavedesignatedtoreceivecopiesofyourevaluation.

• Resultarea:Containsallresultsreportedforaparticularmailingandstatisticaldatausedforevaluationpurposes.AdetaileddescriptionofevaluationdataspecificforeachdisciplineispresentedineachPS.

• CMSperformancesummaryreport:IncludesinformationoncurrentandcumulativeperformanceforregulatedanalytestobesenttotheCMS.

Reviewing Your Evaluation TotrulyrealizethebenefitofPT,itisimportantthatyoutake

thetimetocarefullyreviewyourevaluation.Youcangainvaluableinsightintoyourlaboratory’soverallprocessesbyfollowingtheseeasystepsinreviewingthisreport.

1. Reviewthedemographicinformationontheevaluation.Ifanyinformationisincorrectorhaschanged,contacttheCAPat800-323-4040option1.

2. Compareinformationonyourevaluationwithresultsonyourphotocopyorprintedcopyoftheresultform.IfanyofyourdatawasenteredbytheCAPincorrectly,contactusimmediately.CorrectionsduetodataentryerrorsmadebytheCAPmustberequestedwithinfourweeksafterthefirstevaluationwasmailed.

3. Lookforanyunacceptableresults.Common,easilycorrectedreasonsforunacceptableresultsinclude:

• Incorrectorincompletemethod/instrumentdata

• Clericalerror

• Decimalpointplacement

• Specimenhandlingerror

Remember,whateverthecause,CLIAstatesthatallPTdeficienciesmustbedocumentedandcorrectiveactiontakentoresolvethedeficiency.

4. Occasionally,incorrectentryofsubmitteddataoccurs.Ifthisisduetoyourtranscriptionerrororfailuretocompletetheresultformappropriately,yourentrycannotbereevaluated.IftheerrorismadebytheCAP,contacttheCAPat800-323-4040option1forfurtherassistance.

5. Althoughtheresultsmaynotbeformallyevaluated,youcancompareyourresultswiththedataprovidedinthePSreport.Youcanusethe“allmethodmean”ormedian,low,andhighvaluestocompareyourresultsforaself-assessmentofyourperformance.

6. Forquantitativedata,justknowingthatyouare“withinlimits”doesnottellyouifyouareexperiencingaslowlydevelopingbiasthatmayresultinfuturefailures.Thekeytooptimaluseofyourevaluationdataistolookatthecolumnwherestandarddeviationindexes(SDI)arereported.Ifyounoteanyofthefollowingtendencies,itmaybeadvantageoustoexamineyourlaboratoryprocessesfurther:

• TheaverageSDIismorethan±1.5:thismayindicateasignificantsystematicerror.Reviewcalibrationdataandtechnique.Reviewexpirationdatesofcalibratorsandreagents.

• OneofyourSDIsisgreaterthan±3ortotalSDIisgreaterthan4(oneSDIis-2andoneis+2.5foratotalof4.5):thismayindicateasignificantrandomerror.Reviewyourproceduretodeterminewhereanyunwantedimprecisionmaybeoccurring.

7. Whentheevaluationhasanonevaluationcodelisted,refertoyourPSforvaluableinformation.

8. VerifythatallregulatedanalytesforwhichyoureportedresultsareincludedontheCMSPerformanceSummaryReport.

9. MakesurethelaboratorydirectorreviewsandsignsallPTevaluations.

PS Inadditiontoyourevaluation,eachlaboratoryreceivesaPSforthatmailingthatlistsresultsfromallparticipantsforeachanalytegroupedbythemethodology.Thisreportprovidesvaluableinformationtotheparticipantintheformofcomparativedataandeducationactivities.

Program UpdateThissectionoftheparticipantsummarycontainsinformationaboutevaluationcriteriainuseforthatmailing.Italsohighlightsimportantmethod,manufacturer,andspecimeninformationthatpertainstothatmailing.

Quantitative DataTheparticipantsummaryprovidesthestatisticaldataneededtoreviewyourPTresults.Thereportliststhemean,SD,andCVforpeergroupsconsistingof10ormorelaboratories.

Qualitative DataQualitativedataevaluationisbasedonconsensusofparticipantand/orrefereeresponses.ThePSliststheparticipantresponsesalongwiththepercentagereportingthatresponse.Whereavailable,refereedataisalsoincluded.Thispracticeprovideshigher-quality,evaluatedchallengestoourparticipants.

How to Perform a Self-EvaluationAsmentionedpreviously,occasionallyaPTchallengecannotbeevaluatedforavarietyofreasons:

• Lackofparticipantconsensus

• Insufficientdata(<10responsesforagivenmethod)

• Perceivedcompatibilityissues

InordertocomplywiththequalityassuranceaspectofPTasoutlinedinCLIA,youmusthavesomemechanismtoevaluateyourPTresults.HereareafewexamplesofhowthedatapresentedinthePScanassistwiththistask.

Quantitative ResultsIfyouperformatestandtherearefewerthannineotherlaboratoriesreportingresultsforthattest,yourresultwillnotbeevaluated.Youcandeterminehowwellyouperformedcomparedtoallparticipantswhoreportedresultsbyusingthe“allinstrumentmethod”datapresentedinthePS(ifprovided).Forexample,youperformhemoglobinanalysisusingtheCoulterLH500.Thereareaninsufficientnumberofresultstoformapeergroup(<10);therefore,yourresultsarenotgraded.NotethatinthePSthereisanall-instrumentmean,standarddeviation,andcoefficientofvariation,whichcanbeusedasareferencevalue.ByapplyingthepublishedCMSevaluationlimits(±7%)tothismean,youcandeterminehowwellyouperformedcomparedtothisreferencevalue.Forexample:

Your result: 13.8 g/dL

All Instrument Mean: 13.77 g/dL

Range of Acceptability: 12.8–14.8 g/dL

Inthisexample,yourresultwouldbeconsideredwithinrangewhencomparedtotheall-instrumentmean.Documentthisself-assessmentonyourevaluation.Whenyouperformthisself-assessment,anyunacceptableresultshouldbedocumentedandinvestigatedandcorrectiveactionshouldbetakenaswouldbedoneforformallyevaluatedresults.Thissametechniquecanbeusedwhenonlyamedian,low,andhighvaluearereportedforananalyte.

Qualitative Results Ifaqualitativeresultisnotevaluatedduetolackofrefereeorparticipantconsensus,youcanstillevaluatehowwellyourlaboratory’sresultagreedwiththecorrectresponsebyusingthedatainthePS.Forexample,oneoftheGramstainchallengescouldnotbegradedduetolackofparticipantconsensus(77%reportedgram-negative,23%reportedgram-positive).ThePSindicatesthattheorganismwasPseudomonas aeruginosa,agram-negativerod/bacilli.Compareyourresultwiththecorrectresult.Investigateanddocumentanycorrectiveactiontaken.Reviewtheeducationalcritiqueaccompanyingtheresultforhelpfulsuggestionsonlaboratorytechnique.

Laboratory Legislation and CMS Reporting

Laboratory Legislation and CMS ReportingInFebruary28,1992,theSecretaryofHHSpublishedthefinalrulesimplementingCLIA.TheCLIAregulationsreplacedtheMedicare,Medicaid,andCLIA’67standardswithasinglesetofrequirementsthatapplytoalmostalllaboratoriestestinghumanspecimens.Standardsforlaboratorypersonnel,qualitycontrol,andqualityassurancearebasedontestcomplexityandriskfactors.TheregulationsalsoestablishapplicationproceduresandfeesforCLIAcertification,aswellasenforcementproceduresandsanctionsapplicablewhenlaboratoriesfailtomeetstandards.

CLIAappliestoalllaboratories,physicianoffices,orotherentitiesperformingtestingonhumanspecimensforthepurposeofprovidinginformationforthediagnosis,prevention,ortreatmentofdiseaseorimpairmentofhumanbeings.Laboratoriesthatconducttestingforthepurposeofassessingthehealthofindividuals(eg,testingforinsurancepurposes)arealsosubjecttoCLIA.ThefollowinglistscertainlaboratoriesthatarenotsubjecttoCLIA:

• Laboratoriesconductingonlyforensictesting

• Researchlaboratoriesthatdonotreportpatientresults

• ComponentsorfunctionsoflaboratoriescertifiedbytheSubstanceAbuseandMentalHealthServicesAdministration

• LaboratorieslocatedinastateinwhichthelicensureprogramisapprovedbytheCMSasmeetingCLIAstandards

• InternationallaboratoriesnotperformingtestsonUnitedStatescitizens

UnderCLIA,allclinicallaboratories,regardlessoflocation,size,ortype,mustmeetstandardsbasedonthecomplexityoftheteststheyperform.Threelevelsoftestingcomplexityaredefinedintheregulations:waived,provider-performedmicroscopy,andnonwaived.Laboratoriesperformingprovider-performedmicroscopyornonwaivedtestingmustmeetrequirementsforPT,patienttestmanagement,qualitycontrol,qualityassurance,andpersonnel.Thesespecificrequirementsdonotapplytotestsinthewaivedcategory.

TheuniformPTprogramregulationsmandatethatyourlaboratorymustenrollinaPTprogramapprovedbytheDepartmentofHealthandHumanServices,theCMSparentregulatoryagency,foreachofthespecialtiesandsubspecialtiesforwhichitseekscertification.AllanalytesthatareregulatedforPTappearinboldtypeintheSurveysandAnatomicPathologyEducationProgramscatalog.

YourlaboratorymustnotifyCMSoftheapprovedprogram(s)inwhichyouparticipateandauthorizethePTprogramtoreleasealldatatotheCMS.YourlaboratorymustparticipateinanapprovedPTprogramforoneyearbeforedesignatingadifferentprogram.CMSmustbenotifiedbeforeyourlaboratorychangesPTprograms.ForteststhatarenotsubjecttoPTintheseregulations,yourlaboratorymuststillestablishtheaccuracyandreliabilityofitstestproceduresatleasttwiceayear.

PTspecimensmustbetestedwiththeregularpatientworkloadbypersonnelwhoroutinelyperformtesting.Yourlaboratory’sroutinetestingmethodsmustbeused.TheindividualtestingthespecimensandthelaboratorydirectormustattesttotheroutineintegrationofspecimensusingaformprovidedbythePTprogram.LaboratoriesthatperformtestsonPTspecimensmustnotengageininterlaboratorycommunicationspertainingtotheresultsofPTspecimen(s)untilafterthedatebywhichthelaboratorymustreportPTresultstotheprogramforthetestingeventinwhichthespecimensweresent.LaboratorieswithmultipletestingsitesorseparatelocationsmustnotparticipateincommunicationsordiscussionsconcerningPTspecimenresultsuntilafterthedatebywhichthelaboratorymustreportPTresultstotheprogram.Yourlaboratorymustalsomaintainacopyofallrecords,includingtheformusedtorecordthePTresults(includingtheattestationsignatures),foraminimumoftwoyears.

YourlaboratorymustsuccessfullyparticipateinaPTprogramapprovedbyCMS.“Unsuccessfulproficiencytestingperformance”isa“conditionlevel”deficiencyandmayresultinlaboratorysanctionssuchassuspensionoftheCLIAcertificateandMedicarepaymentsforthespecialty,subspecialty,andanalyteinvolved.Failuretoachieveasatisfactoryoveralltestingeventperformancefortwoconsecutivetestingeventsortwooutofthreeconsecutivetestingeventsisconsideredunsuccessfulperformance.

Proceduresassayedwithwaivedmethodologieswillnotcounttowardthefive-challengeminimum.

Failuretoattainanoveralltestingeventscoreofatleast80%isunsatisfactoryperformanceforanalytesinallspecialtiesandsubspecialtiesexceptABOgroup,D(Rh)typing,andcompatibilitytestingforwhich100%isrequired.

FailuretoreturnPTresultsforatestingeventisunsatisfactoryperformanceandwillresultinascoreof“0.”Foranyunsatisfactorytestingeventforreasonsotherthanfailuretoparticipate,yourlaboratorymustundertakeappropriatetrainingandemploythetechnicalassistancenecessarytocorrecttheproblem.Allremedialactionmustbedocumentedandsuchdocumentationkeptfortwoyearsatyourlaboratoryforpossiblereferencebyinspectionandaccreditationteams.Aspartoftheseregulations,criteriahavebeenestablishedbywhichaPTprovider’sprogrammaybeevaluatedforapprovalbyHHS.TheCAPhasmadeeveryefforttoensurethattheSurveysprogramhasmettherequirementssetforthbytheFebruary28,1992,FinalRule.

Special Considerations for the Regulatory Requirements for the Specialty of ImmunohematologyThespecialtyofImmunohematologycomprisesfoursubspecialtiesasfollows:

• ABO/Rh

• Unexpectedantibodydetection

• Compatibilitytesting

• Antibodyidentification

A100%scoreisrequiredtoachievesatisfactoryperformanceforABO/Rhandcompatibilitytesting.

Forunexpectedantibodydetectionandantibodyidentification,an80%scoreisrequired.Theconsensuspercentagerequiredtoestablishagradedchallengeissetat95%participantor100%refereeconsensusforABO/Rhandcompatibilitytestingand95%refereeorparticipantconsensusforunexpectedantibodyorantibodyidentificationchallenges.

Special Considerations for the Regulatory Requirements for the Specialty of MicrobiologyTheCLIAregulationsstatethatalaboratorymusttestaminimumoffivespecimensineachtestingeventforthesubspecialtiesofbacteriology,mycobacteriology,mycology,parasitology,andvirology.Withineachofthesesubspecialties,varioustypesoftestingarerequired.

Forbacteriology,thefivechallengesmayincludeacombinationofthefollowingspecimens:

• Bacterialantigendetection

• Bacterialidentification

• Gramstain

• Antimicrobialsusceptibility

Chlamydia trachomatisresultsareaggregatedunderthesubspecialtyofbacteriology.

Formycobacteriology,thefivechallengesmayincludeacombinationofthefollowingspecimens:

• Acid-faststain

• Mycobacterialidentification

• Antimycobacterialsusceptibility

Formycology,fiveidentificationsarerequired.

Forparasitology,fiveidentificationsarerequiredandmayconsistoftestingbyfecalsuspension,Giemsa-stainedbloodsmear,antigendetection,and/orPVAslide.

Forvirology,thefivechallengesmayincludeacombinationofthefollowingspecimens:

• Viralantigendetection

• Viralidentification(culture)

Note: In order to meet the regulatory requirements for microbiology subspecialties, carefully follow the kit instructions included with each Surveys mailing.

Regulatory Requirements for the Specialty of Cytology: Gynecologic ExaminationsForcytology,theexaminationconsistsoftenslidesfromfourdiagnosticcategories,includingUnsatisfactory,NegativeorBenign,LSILandHSILorcarcinoma.Tobesuccessfulincytology,ascoreof90%mustbeachieved.Detailedinstructionswillbeprovidedwiththetestmaterials.

Provision of Results to CMS and State AgenciesToassistincomplyingwiththerequirementthatyourlaboratoryresultsbereleasedtoHHS,theCAPwilltransferdatatotheCMSifyouhaveprovidedaCLIAidentificationnumber.Wewillforwardpaperorelectroniccopiesofyourresultstoyourstatedepartmentofhealth,actingonbehalfoftheCMS,ifyouauthorizeustodoso.Toauthorizereleaseofresultstostateagencies,indicateyourrequestontheorderconfirmationreportsentafteryourSurveysorderisprocessed,orsendalettertotheCAP.YourSurveysevaluationwillthenreflectthenameoftheagencytowhichtheinformationwasprovided.AnyquestionsregardingrequirementsforforwardingyourPTresultsmaybeansweredviayourstatedepartmentofhealthoroneoftheCMSregionaloffices.

Consultwithyourstatedepartmentofhealthforadditionallawsorregulationsconcerninginspection,accreditation,andPTrequirementsthatmayaffectthelicensingofyourlaboratoryanditspersonnel.

CopiesoftheFebruary28,1992,CLIAregulationscanbeobtainedbycontactingtheGovernmentPrintingOfficebytelephoneat202-512-1800orbyfaxat202-512-2250andrequestingCFRTitle42Parts400-429and430-End.Youmayalsoaccessdocumentsonlineathttps://www.gpo.gov/fdsys/pkg/FR-1992-02-28/pdf/FR-1992-02-28.pdf

IfyouhaveanyquestionsregardingtheautomatictransferofresultstoCMSoryourperformancesummaries,contacttheCAPat800-323-4040option1.

Use of Reason Codes for Nonevaluated SpecimensSomeindividualresultsarenotevaluatedforcertainlaboratoriesforavarietyofreasons.Areasoncodeexplainingthespecificcircumstancewillappearonindividualevaluation,alongwithabriefexplanationofwhatthatcodemeans.Belowisaledgerofpossiblereasoncodesthatcanbeassigned.

Reason Code Description or Explanation

11 Unabletoanalyze(documentationtobeprovidedbylaboratory).

20 Noappropriatetarget/responsecouldbegraded.

21 Specimenproblem.

22 Resultisoutsidethemethod/instrumentreportablerange.

24 Incorrectresponseduetofailuretoprovideavalidresponsecode.

25 Inappropriateuseofantimicrobial.

26 Educationalchallenge.

27,31 Lackofparticipantorrefereeconsensus.

28Responsequalifiedwitha“greaterthan”or“lessthan”sign;orunabletoquantitate.

29 Inappropriateresponse.

30 Scientificcommitteedecision.

33 SpecimendeterminedtobeunsatisfactoryaftercontactingtheCAP.

35 Testingnotperformedonthisspecimentype.

40 Resultsforthiskitwerenotreceived.

41 Resultsforthiskitwerereceivedpasttheduedate.

42 Nocreditassignedduetoabsenceofresponse.

43 Theorderforthiskitwascanceled;resultsnotevaluated.

44Thisdrugisnotincludedinourtestmenu.Useofthiscodecountsasacorrectresponse.

46 Quantitationnotappropriate.

55 Exemptiongrantedduetoanaturaldisaster.

77 Improperuseofexceptioncodeforthismailing.

88Labdoesnotperformtestsfromthissourceordoesnotperformthistestonpatients.

91Therewereaninsufficientnumberofcontributingchallengestoestablishacompositegrade.

92Compositegradecouldnotbeestablishedduetotheuseofmultiplenon-gradedreasoncodesfortheindividualchallenges.

It is the laboratory’s responsibility to document the appropriate use of these exception codes should this be requested during a laboratory inspection.

CMS Performance Summary

CMS Performance Summary Data1. a. Kit ID:definesthecustomer’suniquekitnumberforeachmailing.

b. Kit Mailed:liststhedatetheSurveysmailingwasshipped. c. Original Evaluation:liststhedatetheevaluationwasoriginallygenerated.

2. RegulatedAnalyte:listsallregulatedtestsincludedinthespecialty/subspecialtyasdefinedbytheCLIAregulationsforthemodulesinwhichyouareenrolled.

3. SummaryofYourPreviousResponses:liststhetotalnumberofspecimensyouhavetestedandthenumberofacceptableresponsesforprevioustestingevents.

4. CAPNumberandKit:definestheCAPnumberandkitsequence.

5. TestEvent:identifiestheproductfulfillmentgroupandshipment.

6. SummaryofYourCurrentResponses:liststhetotalnumberofspecimensyouhavetestedandthenumberofacceptableresponsesforthattestingevent.

7. Current Event Performance Interpretation:indicateseithersatisfactory(≥80%)orunsatisfactory(<80%)performanceforeachanalyteandtheoverallperformanceforthespecialty/subspecialty.ForABOgroup,D(Rho)type,andcompatibilitytesting,ascoreof100%isrequired.Whenresultshavenotbeenreceivedfromyourlaboratoryforashipment,thisareawillbeblank.

• Ascoremaynotappear(fieldisblank)duetothefollowingreasons:

• LabhasindicatedtotheCAPthattheregulatedanalyteshouldnotbereportedtoagencies

• Challengeswerenotgraded,usingreasoncodesthatarenotreportedonthescorecard

• ThemethodreportedfortheanalyteiswaivedbytheCMS

• Noresultswerereported

8. Cumulative Performance Interpretation: indicatessuccessful(≥80%)orunsuccessful(<80%)performanceforeachanalyteandforthespecialty/subspecialty.ForABOgroup,D(Rho)type,andcompatibilitytesting,ascoreof100%isrequired.A<1>symboldenotesthatyourperformanceissuccessful;however,becauseyouhadlessthan80%onthepreviousmailing,youarestillatrisktobeunsuccessfulforthenextmailing.A<2>denotesyouarecurrentlysuccessfulbutatriskforthenexttwomailingsasyouwereunsatisfactoryforthismailing.Thesecodesareapplicabletoboththeanalyteandtheoverallspecialty/subspecialtyscores.A<3>denotescurrentlyunsuccessfulperformance.A<4>denotesthatscorecardperformanceispendingafutureevaluationormaynotbeapplicableduetodiscontinuedtestingortheuseofawaivedmethod.

PROFICIENCY TESTING MANUALProficiency Testing Manual College of American Pathologists 4 General...

Documents