8/12/2019 EQAS Interpretation
1/69
Many lab professionals think Qualityin a Medical Laboratory is .
Accurate
Timely
Reliable
Reproducible
RESULTS
8/12/2019 EQAS Interpretation
2/69
INSTITUTE OF BIOCHEMISTRY WELCOMES YOU ALL TO
THE CME
ON
QUALITY ASSURANCE IN CLINICAL
LABORATORY
UP, CLOSE & PERSONAL
8/12/2019 EQAS Interpretation
3/69
8 rights make Quality in a MedicalLaboratory is .
Choosing the Righttest
Collecting RightSpecimen
After Rightpatient preparation
Testing by the Rightmethod
Reporting the Rightbased on
RightReference intervals at the
Righttime and at the
RightPrice
8/12/2019 EQAS Interpretation
4/69
Tissue
Material
Suppliers
Equipment
Process or
Techniques
Environment
Personnel
Patient or Client
(sample sources)
Physical Damage
or Contamination
Microbial
Contamination
Result:
Accuracy
Precision
Reliability
EfficiencyConfidentiality
Information
MrX
MsY
Where can you go wrong in a lab ?
8/12/2019 EQAS Interpretation
5/69
Quality is .
A subjective term - for which each person has his / her owndefinition
Technically Quality can have two meanings
A product or service that fulfills the defined and expected
requirement
stated and implied needs
A service or product free from defects & deficiencies
8/12/2019 EQAS Interpretation
6/69
QUALITY CONTROL
QUALITY ASSURANCE
QUALITY SYSTEM
QUALITY
MANAGEMENT
Stages of Quality - Hierarchy
8/12/2019 EQAS Interpretation
7/69
WHO definition of QA & QC is .
QA - Includes Internal QC, External QA, pre-analyticphase, test standardization, post-analytic phase,management, and organization (WHO, 1992)
QC - Internal quality control (IQC)set of proceduresfor continuously assessing laboratory work and theemergent results; immediate effect, should actuallycontrol release of results (WHO, 1981)
8/12/2019 EQAS Interpretation
8/69
Quality Assurance & QualityControl - difference is .
QA is for correction & prevention of errors or defectsin the entire lab
QC is detection of errors and defects in testing process
8/12/2019 EQAS Interpretation
9/69
Purpose of Internal & External QC
Internal QC
For CONTINUOUS & IMMEDIATE (DAILY)monitoringof the laboratory work and the emergent results in order todecide whether the results are reliable enough to be releasedto physicians(WHO, 1981)
Measures Precision& Repeatabilityof the systems &methods in use in the lab.
Illusion of short term accuracy of the lab results.
Do not detect the accuracy or trueness of patient results over alonger term
8/12/2019 EQAS Interpretation
10/69
Purpose of Internal & External QC
External QC
For PERIODIC AND RETROSPECTIVE monitoring of lab
results by an independent external agency to indicate to thelaboratory and its staff the accuracy or bias in their systems &methods
Lab can know its shortcomings and change their Internal QualityAssurance procedures.
8/12/2019 EQAS Interpretation
11/69
Why EQAS is necessary?Serves as an educational tool and help to monitor & improve the
performance of the lab
Measures the accuracy or bias of its results and stability ofmethodsOver a longer period of time in terms of years
Mandatory requirement for applicant & accredited labs
Non participation or repeated failures in an EQAS or PT
programme may result in temporary suspension or cancellation ofaccreditation for those non EQAS tests
Gives the laboratories, both the management & technical staff,
added confidence in their patient test results
8/12/2019 EQAS Interpretation
12/69
WHAT DOES IT IDENTIFY ?
Identifies systematic kit & reagent problems, water qualityproblems, analyte calibration stability and status, equipmentperformance
Indicator of where to direct improvement efforts
Identifies training needs of lab personnel
Benchmarks the labs performance against others
Early Warnings System for Problems
8/12/2019 EQAS Interpretation
13/69
HOW SHOULD IT BE USED ?
Should only be used for motivating staff & not topunish them
Inaccurate lab results are not due to technicians
But due to failure of lab systems
8/12/2019 EQAS Interpretation
14/69
HOW SHOULD IT BE USED ?
EQAS / PT samples should be treated exactly as the patientsamples Only then the correct situation in the lab can be foundfixing the problem becomes easy
Never
Run the calibration on the day of reporting EQAS sample if it isnot a scheduled /required calibration
Repeat the EQAS samples where as the patient samples are testedonly once and give the mean of multiple runs
Ask a specific analyst run the EQAS / PT sample
8/12/2019 EQAS Interpretation
15/69
REMEMBEREQAS.
SUPPLEMENTS Internal Quality Control
NEVER a SUBSTITUTE for Internal QC
Both measure 2 different aspects of quality
8/12/2019 EQAS Interpretation
16/69
INTERPRETATION OF EQAS REPORTS
1.VISVariance Index Score
2. SDIStandard Deviation Index
3. Z-ScoreClassical & Robust
8/12/2019 EQAS Interpretation
17/69
Variance Index Score
First proposed by the United Kingdom National Quality ControlScheme (UKNEQUAS)
CCV (Chosen Co-efficient of Variation) & DV (Designated Value)
used to calculate VIS
CCV is just the Allowable Limit of Error for an analyte (TEa) - Sumof both imprecision and bias
Set & recommended by WHO after studying the performance ofmany Indian labs
8/12/2019 EQAS Interpretation
18/69
CCV of some common Analytes
Glucose 7.5 Sodium 2.3
Urea 10 Potassium 5.0Creatinine 10 Chloride 6.0
CK 7.3 AST 12.5
T.Bilirubin 19.2 ALT 17.3
T.Protein 7.5 ALP 15.5
Albumin 7.5 Amylase 15.5
Calcium 6.0 LDH 15.5
Uric acid 7.7 Phosphorus 7.8Cholesterol 7.5 Bicarbonate 9.0
TGL 14 HDL- C 7.6
HDL 7.6 Iron 15
8/12/2019 EQAS Interpretation
19/69
Calculation of VIS
Designated Value [DV] = 120 mg %
Participant's result = 95 mg%
% Variation [%V] = Participant's Result - Designated value--------------------------------------- X 100
Designated value
120-95 X 100 = 25 X 100120 120
= 20.8VarianceIndex = %V X 100 = 20.8 X 100 = 277
CCV 7.5VIS = 277
8/12/2019 EQAS Interpretation
20/69
How to read the EQAS results ?
Check the VIS & OMVIS values for each parameter every month
Check if your value is close to DV
Closer it is lower will be your VIS & better is your labs accuracy
Remember If your VIS is < 50 it is regarded and given as zero score
Even if >400, it is still given as 400 only
Calculation of VIS
8/12/2019 EQAS Interpretation
21/69
Interpretation of VIS
VIS Performance
< 100 Very good
100 -150 good
150 -200 satisfactory room for improvement
> 200 Not acceptable
If VIS of >200 on two or more occasions for the same analyte,them check your standardization procedures & calibration
Indicates an accuracy problem (systematic error / bias )
8/12/2019 EQAS Interpretation
22/69
Interpretation of VIS
Check the monthly OMVIS.(Overall Mean VIS) cumulativeperformance over a period
OMVIS Performance
< 100 Very good - your result are very close to DV
150-200 Need to take care of those parameters for whichthe reported values are very different from the DV for that particularmethod
> 250 You are probably reporting many wrong results &you should take urgent steps to locate the problem and correct them
8/12/2019 EQAS Interpretation
23/69
Calculation of VIS Method MeanThe Method Mean' - Mean obtained from results of allparticipating labs following the same method including results ofoutliers
The Designated Valueis the value obtained after excludingresults, from labs with same method, which are > 3SD of MethodMean and recalculating the mean after eliminating the outliers - Mean
of inliers only
The 'Reference Mean' - Mean obtained at the organizing lab afterexposing the QC samples to ambient temperature (25-35 C) for a
period of 7- days (transport time) and analysing them on five differentdays
The reference mean is shown against the method by which it wasanalysed in the organizing lab
8/12/2019 EQAS Interpretation
24/69
Mean value obtained atthe Reference Lab (CMC)after exposing 5 vials toambient Temp. for 9days and analyzing themon 5 diff. days
8/12/2019 EQAS Interpretation
25/69
Mean value obtained at theReference Lab (CMC) afterexposing 5 vials to ambientTemp. for 9 days and analyzing
them on 5 diff. days
8/12/2019 EQAS Interpretation
26/69
Mean value of results fromof all participating labs withsame method
8/12/2019 EQAS Interpretation
27/69
Value given (DV) is the mean of allparticipating labs for that method afterexcluding results from labs outside 3SDof the Mean Value (of the participatinglabs with the same method)
8/12/2019 EQAS Interpretation
28/69
STANDARD DEVIATION INDEX
Another Statistical toolassigned to the lab bythe EQAS / PT provider on the performance ofthe lab for each analyte in a EQAS cycle
A measure of relativeinaccuracy / relativebias
Wh t i l G i
8/12/2019 EQAS Interpretation
29/69
What is normal or Gaussiandistribution ?
Out of 100 results
68.2% of values fall
within
1SD
95.5% of values fall
within 2SD
99.7% of values fall
within 3SD
8/12/2019 EQAS Interpretation
30/69
IS KNOWING SDI USEFUL ?
Yes - A measure of the result around a mean among agroup of values
Since 95 % of all results in a normal population fallwithin 2 SDs of the mean, +2 SD is considered anacceptable laboratory value
Expressed in the units being measured
8/12/2019 EQAS Interpretation
31/69
IS KNOWING SDI USEFUL ?
The number of Standard Deviations that your labs mean differsfrom the Peer Group Mean
. Difference is converted into SD units (SDI)
The SDI indicates how large / small the difference is betweenyour result and target value
Simply said
Difference between your result and group mean in
terms of the number of standard deviations from the
overall mean
8/12/2019 EQAS Interpretation
32/69
IS KNOWING SDI USEFUL ?
Actual magnitude of the difference in the units of thetest may look too small or too large
To figure the actual size of this inaccuracy / bias inconcentration units, you need to multiply by the actualSDI by of the group SD.
For e.g if the group mean is 102 mg/dL for TGL and
group SD is 5 mg/dL and your SDI score is 1.2Actual quantitative difference is 1.2 x 5 = 6 mg/dL
8/12/2019 EQAS Interpretation
33/69
IS KNOWING SDI USEFUL ?
Any SDI of 2.0 or greaterin a EQAS cycle for anyanalyte deserves special concernindicates some for ofsystematic error
Any test whose average SDI is 1.0 or greater deserves
some special attention because your method shows asystematic difference from the group.
Likely to lead to unacceptable results in future
SDI up to 1.0your performance is satisfactoryyourresult is with 1 SD of the groupyou are with in the68.7 % of labs result whose values are close to mean
8/12/2019 EQAS Interpretation
34/69
IS KNOWING SDI USEFUL ?
if you observe SDIs such as -0.4, -0.2,-0.5 - 0.5, - 0.5 and -1.0 (all negative)for an analyte in successive cycles,your method is generally running onthe low side and is negatively biased,
on average, by +0.6 SDI
You are reporting precise pateintvalues but lower than the true valueby 0.6 SD ()
So better
calibrate your instrument and analyte
or requires instrument maintenance
8/12/2019 EQAS Interpretation
35/69
Z-Score
Classical Z Scoresame as SDICan be used for internal quality control also
8/12/2019 EQAS Interpretation
36/69
Z-Score
Robust Z scorestatistic is used when the distribution of resultsof participating labs is not Gaussian (not normally distributed)and there are varied results / outliers
Both accuracy and precision (repeatability as well as
reproducibility) are assessed in terms of robust Z score - bothwithin and between labs Z score (ZB & ZW)
The participant labs are asked to analyse the same sample
TWICEand submit both results to the EQAS provider
8/12/2019 EQAS Interpretation
37/69
Z-Score
8/12/2019 EQAS Interpretation
38/69
Z-ScoreRobust Z score =
Normalised Labs result- Median result of all labs )
Normalized IQR ( Inter Quartile Range)
It is calculated based on the median value (central value) and theinterquartilerangeAll results from participating labs are arranged in an ascending manner (lowestto highest) the central value is taken as the median
The 25th and 75th percentile values are calculated The interquartile range is thedifference between the 75th & 25th percentile
The 25th quartile (Q1) is the value below which a quarter of the results lie.Similarly, the 75th quartile (Q3) is the value above which a quarter of the resultslie.
IQR = Q3 - Q1
Normalized inter quartile range (NIQR): NIQR = IQR x 0.7413 (a
constant)
8/12/2019 EQAS Interpretation
39/69
Z-Score
The between laboratory Z-score (ZB) =(S - median ) / (IQR x 0.7413)S = (A + B)/[square root of (2)] = standardised sum ofthe two results for a laboratory (where A and B areresults of two samples of the same test).
Within laboratory Z-score (ZW) =
(D- median / (IQR X 0.7413)D = (A - B)/[square root of (2)] = standardized
difference between the two results for a laboratory Whiletesting two specimens in an EQAS / PT programme bya lab performance both ZW and ZB should beconsidered simultaneously for assessing the performance
8/12/2019 EQAS Interpretation
40/69
Z-Score
Interpretation of Robust Z-scores:Z score less than 2 - SatisfactoryZ score 2 but less than 3 - Questionable perfromanceZ score more than 3Unsatisfactory
Both ZB & ZW should be < 2
Using only one Z-score may misleading
ZW < 2 ZB > 2 = Higher bias i.e low reproducibility
ZB < 2 ZW > 2 = low precision (i.e., low repeatability)
For assessing laboratory's technical competency, bothZW and ZB value should be low at the same time.
8/12/2019 EQAS Interpretation
41/69
EXERCISES ON INTERPRETATIONOF EQAS REPORTS
Dr.V.K.Ramadesikan
8/12/2019 EQAS Interpretation
42/69
Why Analysis of EQAS reports is important ?
True benefit of EQAS /PT proficiency testing, lies incarefully & critically evaluating your results and report
Proper analysis of EQAS testing results can revealproblems even before failure in EQAS or even an
adverse patient resultPotential problems can be recognised by recognisingpatterns from graphs
Review both the present cycle results as well as
performance from previous cycle for the same analyteGraphs of Z score or SDI or VIS or % deviation fortrends
Otherwise trends will be missed
8/12/2019 EQAS Interpretation
43/69
How to recognise problems ?
Results may consistently be different from thetarget peer group mean - All results on one sideof the mean (may be close to mean or at variabledistances from mean)SystematicError
Majority of results are close to the target valuebut some show larger deviation s on one side orboth side of meanRandomError
Reasons and hence corrective action differs
8/12/2019 EQAS Interpretation
44/69
How to recognise problems ?Random error is an error / mistake / inaccuracy that has no set
pattern. Its occurrence cannot be predicted
Results on an average are close to target mean. Few results showlarge deviations on either side of target
Detected by sudden, undue % deviation /SDI /Z Score> 3.5
Indicates labs imprecision / poor reproducibility
Easily identifiedvalues are far beyond the usuale.g SDI from1.5 to 4.06.0
8/12/2019 EQAS Interpretation
45/69
How to recognise problems ?
Systematic errorset pattern of error / mistake. Its occurrencecan be explained and corrected
Constant difference between the participant labs value and
group mean All results lie on one side of mean
Indicates labs bias or inaccuracy but good precision
A progressive increase in deviation on the same side shift orstabilizes after a gradual increase trends in
8/12/2019 EQAS Interpretation
46/69
Sources of Random Errors
Random errors are blunders
Transcription errorsresult not correctly transcribed from theinstrument to workbook or PT sheet or from the Workbook to thesystem
Misplaced decimal points e.g serum potassium 58.2 instead of 5.82
Result was entered in the wrong instrument or method group on the
result form of PT provider.
Lab might have changed the method or instrument recently but notupdated with PT provider
8/12/2019 EQAS Interpretation
47/69
Sources of Random Errors
Mislabeling errors, interchanging results of PT specimens,misplacing specimens in analyser rack, inappropriate reagents andstandards
Result of some other analyte entered in the PT form of the provider
in the system
Wrong units. Result in one unit in the instrument but not convertedto unit of EQAS provider e.g ug /mL instead of ng/L , mg/dLinstead of g/L
Result found on evaluation report from PT provider not matchingwith the result on the report form mistake of PT provider
Calculation errors (conversion from one unit to another)
8/12/2019 EQAS Interpretation
48/69
Sources of Systematic ErrorsRecalibration if not done earlieresp if a new lot of reagent has
been used or if the open vial stability of the reagent is doubtful
Instrument maintenancemajor part needs servicing orreplacement (optics, alignment, incubation temperature failure,Internal QC values having a bias
Recent instrument malfunction, instrument failure soon after PTspecimen testing
Assay settings (sample volume, reagent volume, delay time
incubation time no of readings to be taken etc. )
improper reconstitution of QC materials, water quality, notfollowing manufacturers / PT providers instructions whilereconstituting, storing or handling reagents / QC materials
What type of Error is indicated
8/12/2019 EQAS Interpretation
49/69
What type of Error is indicatedin greenand red?
8/12/2019 EQAS Interpretation
50/69
Systematic Error
SAMPLE EQAS MONTHLY REPORT
8/12/2019 EQAS Interpretation
51/69
SAMPLE EQAS MONTHLY REPORT
Your result
SAMPLE EQAS MONTHLY REPORT
8/12/2019 EQAS Interpretation
52/69
SAMPLE EQAS MONTHLY REPORT
SAMPLE EQAS END OF CYCLE REPORT
8/12/2019 EQAS Interpretation
53/69
SAMPLE EQAS END OF CYCLE REPORT
SAMPLE EQAS END OF CYCLE REPORT
8/12/2019 EQAS Interpretation
54/69
SAMPLE EQAS END OF CYCLE REPORT
8/12/2019 EQAS Interpretation
55/69
SAMPLE CAP EQAS REPORT
8/12/2019 EQAS Interpretation
56/69
SAMPLE CAP EQAS REPORT
SAMPLE CAP EQAS REPORT
8/12/2019 EQAS Interpretation
57/69
SAMPLE CAP EQAS REPORT
SAMPLE CAP EQAS REPORT
8/12/2019 EQAS Interpretation
58/69
SAMPLE CAP EQAS REPORT
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
59/69
SAMPLE EQAS REPORT
Transcription errorMisplacing of specimens or interchanging resultsof PT specimens
U 1 31.3 909.51 U3
U 2 13.1 28.31 U1
U 3 1031.9 14.8 U2
8/12/2019 EQAS Interpretation
60/69
Result - 226 Mean445 SDI 4.34
ALT226 U/L
Mean445
SDI
4.34
8/12/2019 EQAS Interpretation
61/69
Analytical errorCalculation error
PT was repeated with 1 in 2 dilution as results were
high but the result was not multiplied by 2
Diluted result was sent
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
62/69
SAMPLE EQAS REPORT
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
63/69
SAMPLE EQAS REPORT
Erratic ResultsVery poor precision and accuracy
ProblemInternal Controls wide SD due to variousreasons
Instrument not maintained & calibrated reagentsproblems deterioration
Analyte not calibrated
Procedural problems
SAMPLE CAP EQAS REPORT
8/12/2019 EQAS Interpretation
64/69
SAMPLE CAP EQAS REPORT
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
65/69
SAMPLE EQAS REPORT
1. Systemic error may have seeped in the system
anytime after the last EQAS sample reportinglook at daily controls for any shift in values
2. May be a random errorKeep a watch on dailycontrols and follow during next EQAS
3. Wrong sample was tested
4. Wrong preanalytical stagereconstitution,storage etclook at values of other analytes
Are they with in acceptable limits of SDI and in the
same direction ?
SAMPLE CAP EQAS REPORT
8/12/2019 EQAS Interpretation
66/69
SAMPLE CAP EQAS REPORT
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
67/69
SAMPLE EQAS REPORT
1. Systematic errorone showing a positive bias
and other a negative bias
2. Needs
Recalibration for analyte
Instrument maintenance
Internal QC Value Mean needs to be reset
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
68/69
SAMPLE EQAS REPORT
SAMPLE EQAS REPORT
8/12/2019 EQAS Interpretation
69/69
SAMPLE EQAS REPORT
1. Fairly a good performance in terms of accuracy
and precision
2. Values on either side of mean No Bias
3. Values within =/- 0.5 SD