| Date post: | 02-Jan-2016 |
| Category: |
Documents |
| Upload: | polly-warren |
| View: | 217 times |
| Download: | 0 times |
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
On the Role of Proficiency Testing Schemes in
Laboratory Accreditation
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
5.9 Assuring the quality of test and calibration results
The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following:a) regular use of certified reference materials and/or internal quality control using secondary reference materials;b) participation in interlaboratory comparison or proficiency-testing programmes;c) replicate tests or calibrations using the same or different methods;d) retesting or recalibration of retained items;e) correlation of results for different characteristics of an item.NOTE The selected methods should be appropriate for the type and volume of the work undertaken.
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
EXTERNAL QUALITY CONTROL: PROFICIENCY TESTING
Inter-laboratory comparison
Target value and standard deviation based upon results of participants
Proficiency testing
Organizer decides on target value and target standard deviation
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
The prime purpose is to allow participating laboratories to become aware of unsuspected errors in their work and to take remedial action.
The participant can make three comparisons of performance:
- with an externally determined standard of accuracy- with that of peer laboratories- with its own past performance
PROFICIENCY TESTING, WHY:
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
PLEASE NOTE:
Interlaboratory comparisons may be designed for purposes other than proficiency testing:
- validation of methods- characterization of candidate reference materials- self-assessment of a laboratory in a performance test
A precondition for selection is that the scheme is operated according to ISO/IEC Guide 43 and/or ILAC Guide G-13.
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Why mandatory PT activities ?
1) ILAC requires a policy of the Accreditation Bodies related to the participation of accredited laboratories in PT.
2) ILAC makes recommendations
3) A National Accreditation Body may transfer this recommendation into its own policy, e.g.:
…one PT activity prior to gaining accreditation and one activity relating to each major sub-area of major disciplines of a laboratory’s scope of accreditation at least every four years; where applicable and where practical….
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ?
Sources: - National accreditation bodies - Peer laboratories already participating - PT-organizers in the own country - Internet e.g. www.eptis.bam.de -
http://www.ianz.govt.nz/aplac/documents/web_docs/APLACPT003.pdf
- Accreditation & Quality Assurance journal
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
• PT-testing is not about ‘passing’ or ‘failing’ a test.
• PT-testing is about taking part and learning from the results
• One bad result does not make a laboratory bad. The challenge is to prevent that it repeats
• One good result does not make a laboratory supreme. The challenge is to repeat this performance
• Goal: consistent satisfactory performance
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Auditors : KEEP IN MIND
• PT participation done under “daily conditions” ?
• Performance over sevral years with same PT organizer
• PT operated in accordance to ISO/IEC Guide 43-1
• The scheme documentation and statistical protocol available ?
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ?
What IF NOT ?
- Analysis of blind samples- Analysis of spiked samples- Analysis of multiple sub-samples- Exchange of samples with other laboratories- Use of more than one analytical method
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
QC/QA and method validation ?…. PT IS OFTEN A WRONG CHOICE Auditor: Check the method validation !!
For assessment of sources of error ?….PT CAN BE A WRONG CHOICEAuditor: Check how and who did the PT measurement
Because their clients say so ?….BUT WILL THEIR CLIENTS JUDGE ON BASIS OF 1 TEST ONLY ?
Auditor: Check which client asked for this
WRONG OBJECTIVES
To get an impartial and independent view of the laboratory’s performance ?…. PT DOES NOT INDICATE THE REAL (DAY-BY-DAY) PERFORMANCE
Auditor: Explain the PT report
Because they have no suitable calibrants ?…. POSSIBLY, BUT CREATING A NEW RM BY PT IS A TIME-CONSUMING PROCESS…
Auditor: What did they do so far ?
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
As double-blinded study
A B C D E
Aroclor 1242 0 62 % 0 0 143 %benzo(k)fluoranthene 0 55 % 119 % 0 55 %chrysene 0 59 % 73 % 0 49 %arsenic 84 % 55 % 99 % 57 % 61 %cadmium 10 % 12 % 98 % 20 % 25 %lead 27 % 15 % 88 % 16 % 34 %
As PT test
A B C D E
Aroclor 1242 120 % 128 % 102 % 102 % 111 %benzo(k)fluoranthene 210 % 0 0 0 210 %chrysene 139 % 94 % 77 % 72 %158 %arsenic 138 % 106 % 91 % 103 % 112 %cadmium 138 % 100 % 89 % 104 %102 %lead 132 % 91 % 99 % 112 % 115 %
Ref. R.Forman, Pittcon 2002
Proficiency testing : insight in the REAL performance ?
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
RELEVANT DOCUMENTATION
• Planned action
• List of PT’s in which is participated
• SOP’s for selection participation,administration, evaluation and follow-up
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
VALIDITY OF RESULTS OF PT
The period of validity of the result obtained by a laboratory in a single round of a PT scheme is limited to the time that the laboratory performed the test !!
If a laboratory achieves a satisfactory result in a single round, the result shall not be used to support a claim that the laboratory obtained reliable data on any other occasion !!!
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Is the PT-scheme in which they participated relevant in view of the scope of accreditation ?
Check:• The normal IQC and results thereof: frequency, control charts, criteria, evaluation, remedial/corrective actions• Type of material in daily measurement vs. IQC material vs. PT material• Frequency of measuring routine samples of the type supplied as PT• Measurand • Measurand value in comparison with typical ‘daily’ range• Deadline for reporting (turn-around time in comparison with routine work)• Who analyzed it (senior staff member vs. technician ?)• (Non) availability of IQC material -what else for routine control ?• (Non) availability of suitable calibrants - how method validation ?• Number of participants• Expected number of different techniques and/or number of techniques, similar as applied in the laboratory• Sample dimensions comparable to daily work conditions ? • How may the majority have measured it: best measurement capability or routine?• Frequency of this type of PT testing• Results of previous PT’s on same material
CHECKLIST FOR AUDITORS
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
ISO/IEC 17025 and PROFICIENCY TESTING
DON’T FORGET:
If the PT score is poor (|z| > 2 or |z|>3) :
- Assess how the target uncertainty was determined; compare this with e.g. the associated value predicted by the Horwitz equation Sometime target uncertainties are unrealistic small, set by metrologists under best measurement conditions, not taking instead of “routine” conditions
- Check your own measurement uncertainty
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
STUDY THE OVERALL PERFORMANCE OF ALL PARTICIPANTS:
Unsatisfactory where the majority is satisfactory ?Unsatisfactory where a significant number performed poorly ?
Both : indications of problems regarding methodology… ?then re-validate
criteria for satisfactory performance incorrectly set ? then check performance of analysts….
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
IS IT JUSTIFIABLE TO EXCLUDE OUTLIERS ?
ISO-34 Clause 5.15.1
“…The reference material producer should never rely entirely on a statistical analysis of the characterization data when assessing the property values of interest. Outliers should not be excluded on purely statistical evidence until they have been thoroughly investigated and, where possible, the reasons for the discrepancies identified. Alternatively, the use of robust statistics may be appropriate in some cases….”
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
DO WE NEED STATISTICS IN PT ?
Yes:- For “objective” demonstration of performance- For “objective” comparison of laboratories/
techniques/methods- If PT material has to be upgraded to RM
No:
- If participants use results for finding deficiencies- If fast turn-around times are necessary
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Bad news:Many things in nature are not “normally” distributed.Good news:Much of what is not “normally” distributed in biology would be “normally” distributed if you took the logarithm of each data item.
Bad news:Much of what is in books about statistics has to do with “normally” distributed data. Good news:Robust statistics provide useful information even if applied to not “normally” distributed data.
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
WHY DO ACCREDITED LABS PERFORM NOT BETTER THAN NON-ACCREDITED LABS?
BECAUSE THESE COMPARISONS BETWEEN LAB-PERFORMANCES ARE BIASED !!!
1. Quality conscious laboratories join PT as part of their quality improvement strategy; other, less conscious laboratories are less likely to participate.
2. Non-accredited laboratories that perform badly may more likely to drop participation in PT than accredited labs, who need it to maintain their accreditation.
So:
there is an inclusion of only “good” non-accredited laboratories with both “good” and “bad” accredited laboratories !!
3. Accredited laboratories may even have no motivation to perform ‘super’ good;results are fit for the (clients) purpose4. Accredited laboratories may have much more pressure on routine performance (large throughput) than non-accredited laboratories
ISO/IEC 17025 and PROFICIENCY TESTING
Dr. Peter Bode, Associate ProfessorDelft University of Technology, Interfaculty Reactor InstituteMekelweg 152629JB Delft, The NetherlandsTel. 31-15-2783530 fax. 31-15-2783906 e-mail [email protected]
Thank you for listening
![arXiv:1301.3342v2 [cs.LG] 8 Mar 2013Barnes-Hut-SNE Laurens van der Maaten Pattern Recognition and Bioinformatics Group, Delft University of Technology Mekelweg 4, 2628 CD Delft, The](https://static.documents.pub/doc/80x56/5e7c26c56314682cea03cbf0/arxiv13013342v2-cslg-8-mar-2013-barnes-hut-sne-laurens-van-der-maaten-pattern.jpg)
