SWGDRUG Approach to SWGDRUG Approach to ValidationValidation

Scott R. OultonScott R. OultonSWGDRUG Secretariat and SWGDRUG Secretariat and

Associate Laboratory Director Associate Laboratory Director for the Drug Enforcement Administration (DEA)for the Drug Enforcement Administration (DEA)

Southwest LaboratorySouthwest LaboratoryVista, CaliforniaVista, California

SWGDRUGSWGDRUG

Scientific Working Scientific Working Group for the Analysis Group for the Analysis

of Seized Drugsof Seized Drugs

Mission of SWGDRUGMission of SWGDRUG

The mission of SWGDRUG The mission of SWGDRUG is to recommend minimum is to recommend minimum standards for the forensic standards for the forensic examination of seized examination of seized drugs and to seek their drugs and to seek their international acceptance.international acceptance.

SWGDRUG ObjectivesSWGDRUG Objectives Specifying requirements for practitioners’ Specifying requirements for practitioners’

knowledge, skills and abilities, knowledge, skills and abilities, Promoting professional development,Promoting professional development, Providing a means of information exchange Providing a means of information exchange

within the forensic science community, within the forensic science community, Promoting ethical standards of Promoting ethical standards of

practitioners,practitioners, Providing minimum standards for Providing minimum standards for

examinations and reporting,examinations and reporting, Establishing quality assurance Establishing quality assurance

requirements,requirements, Considering relevant international Considering relevant international

standards, andstandards, and Seeking international acceptance of Seeking international acceptance of

SWGDRUG recommendationsSWGDRUG recommendations

2005 SWGDRUG Process2005 SWGDRUG Process The SWGDRUG process is an The SWGDRUG process is an

international forensic science international forensic science community endeavorcommunity endeavor

The role of the core committee is to The role of the core committee is to vote to accept or reject vote to accept or reject subcommittee recommendationssubcommittee recommendations

In order for a proposal to become an In order for a proposal to become an official recommendation, 3/4s of the official recommendation, 3/4s of the full core committee must be present. full core committee must be present. 2/3s of those present must vote in 2/3s of those present must vote in the affirmative (YES) for a proposal the affirmative (YES) for a proposal to become a recommendationto become a recommendation

Website www.swgdrug.orgWebsite www.swgdrug.org

WebsiteWebsite Future features:Future features:

Search EngineSearch Engine Questionnaire FormQuestionnaire Form Membership LoginMembership Login

SWGDRUG Core CommitteeSWGDRUG Core Committee

• DEA – Nelson SantosDEA – Nelson Santos• Secretariat – Scott Oulton Secretariat – Scott Oulton (non-voting)(non-voting)• FBI - Eileen WaningerFBI - Eileen Waninger• ASCLD – Garth GlassburgASCLD – Garth Glassburg• NIST - Susan BallouNIST - Susan Ballou• ASTM and NEAFS- Jack ASTM and NEAFS- Jack MarioMario• Educator – Dr. Chris TindallEducator – Dr. Chris Tindall• Educator – Dr. Suzanne BellEducator – Dr. Suzanne Bell

SWGDRUG Core CommitteeSWGDRUG Core Committee

• CAC & NWAFS - Jerry CAC & NWAFS - Jerry MassettiMassetti• MAFS - Richard PaulasMAFS - Richard Paulas• MAAFS - Linda JacksonMAAFS - Linda Jackson• SAFS - Dr. Conrad RobersonSAFS - Dr. Conrad Roberson• SWAFS - Gary ChasteenSWAFS - Gary Chasteen• South Africa - Tshepo SholeSouth Africa - Tshepo Shole• Connecticut DPS – Dr. Connecticut DPS – Dr. Robert PowersRobert Powers

SWGDRUG Core CommitteeSWGDRUG Core Committee

• Canada - Richard LaingCanada - Richard Laing• Japan - Dr. Kishi TohruJapan - Dr. Kishi Tohru• United Kingdom - Dr. Sylvia United Kingdom - Dr. Sylvia BurnsBurns• Australia - Catherine QuinnAustralia - Catherine Quinn• Germany - Dr. Udo ZerellGermany - Dr. Udo Zerell• ENFSI - Dr. Erkki SippolaENFSI - Dr. Erkki Sippola• UNODC - Dr. Iphigenia UNODC - Dr. Iphigenia NaidisNaidis

SWGDRUG Continued SuccessSWGDRUG Continued Success All forensic drug analysts must stay All forensic drug analysts must stay

involved in the process by contributing involved in the process by contributing comments, criticisms and suggestionscomments, criticisms and suggestions

If you agree with the process, let us If you agree with the process, let us knowknow

If you don’t agree with what you are If you don’t agree with what you are reading in the publication sources, let reading in the publication sources, let us knowus know

Write or E-mail any core committee Write or E-mail any core committee membermember

ValidationValidation Definition and Purpose of ValidationDefinition and Purpose of Validation

Validation is the confirmation by Validation is the confirmation by examination and the provision of objective examination and the provision of objective evidence that the particular requirements evidence that the particular requirements for a specific intended use or application for a specific intended use or application have been fulfilled have been fulfilled [ISO 9000:2005(E)][ISO 9000:2005(E)]

Studies to determine method Studies to determine method performance must be carried out by performance must be carried out by qualified/competent analysts using qualified/competent analysts using appropriate calibrated equipmentappropriate calibrated equipment

ValidationValidation Why is validation necessary?Why is validation necessary?

We make measurements every day that We make measurements every day that affect people’s lives (fines, affect people’s lives (fines, incarceration, etc.)incarceration, etc.)

Decisions are made by our results. Decisions are made by our results. There can be a high cost if we are wrongThere can be a high cost if we are wrong

Our customers expect to be able to trust Our customers expect to be able to trust the resultsthe results

Methods must be sufficiently reliable Methods must be sufficiently reliable that decisions can be made with that decisions can be made with confidence.confidence.

ValidationValidation When to validate a method?When to validate a method?

New method to address particular New method to address particular problemproblem

Revise existing method for improvements Revise existing method for improvements or new problemor new problem

QA suggests method is changing with QA suggests method is changing with timetime

Use of another laboratory’s method or Use of another laboratory’s method or different instrumentdifferent instrument

Demonstrate equivalence between two Demonstrate equivalence between two different methodsdifferent methods

ValidationValidation Who decides how and to what Who decides how and to what

degree of validation is required degree of validation is required for method validation?for method validation? The Laboratory is responsible for The Laboratory is responsible for

ensuring methods are validatedensuring methods are validated Clients InputClients Input

Laboratory and client should Laboratory and client should agree to method usedagree to method used Usually done retrospectivelyUsually done retrospectively

SWGDRUG’s Approach to SWGDRUG’s Approach to ValidationValidation

See PART IV BSee PART IV B

QUALITY ASSURANCE/VALIDATION QUALITY ASSURANCE/VALIDATION OF ANALYTICAL METHODSOF ANALYTICAL METHODS

Analytical SchemeAnalytical Scheme Laboratory ResponsibilityLaboratory Responsibility Operational EnvironmentOperational Environment DocumentationDocumentation Validation PlanValidation Plan

SWGDRUGSWGDRUGAnalytical SchemeAnalytical Scheme

An analytical scheme shall be An analytical scheme shall be comprised of validated comprised of validated methods appropriate for the methods appropriate for the analyteanalyte

Identify analyte of interest, Identify analyte of interest, preclude false-positives and preclude false-positives and minimize false-negativesminimize false-negatives

For quantification the method For quantification the method should reliably determine the should reliably determine the amount of analyte presentamount of analyte present

SWGDRUGSWGDRUGAnalytical SchemeAnalytical Scheme

Verification – shall be Verification – shall be conducted on methods from conducted on methods from non-routine, published non-routine, published literature, another literature, another laboratory’s protocolslaboratory’s protocols demonstrate that a demonstrate that a

representative set of reference representative set of reference materials has been carried materials has been carried through the process and through the process and yielded the expected resultsyielded the expected results

SWGDRUG SWGDRUG Validating MethodsValidating Methods

Laboratory responsibility to Laboratory responsibility to determine whether SOPs determine whether SOPs have been validated, verified have been validated, verified or require further or require further validation/verificationvalidation/verification

All methods shall be All methods shall be validated/verified to validated/verified to demonstrate performance in demonstrate performance in normal operating normal operating environmentsenvironments

SWGDRUG SWGDRUG Validating MethodsValidating Methods

The entire validation/verification The entire validation/verification process shall be documented. process shall be documented. Documentation shall include, but Documentation shall include, but is not limited to the following:is not limited to the following: Personnel involvedPersonnel involved DatesDates Observations from the processObservations from the process Analytical dataAnalytical data Conclusions and/or recommendationsConclusions and/or recommendations Authorization approval signatureAuthorization approval signature

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Purpose/scope – introductory statement Purpose/scope – introductory statement (what, purpose and results required)(what, purpose and results required)

Performance specification – list of specific Performance specification – list of specific objectivesobjectives

Process review – after completion, revisit Process review – after completion, revisit objectives to ensure they were objectives to ensure they were satisfactorily metsatisfactorily met

Analytical method – state the method to be Analytical method – state the method to be validated (each step shall demonstrate validated (each step shall demonstrate satisfactory performance)satisfactory performance)

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Reference materials – appropriate Reference materials – appropriate materials used for qualitative and materials used for qualitative and quantitative procedures (traceability quantitative procedures (traceability required)required)

Selectivity – assess capability of Selectivity – assess capability of method to identify/quantify the analytemethod to identify/quantify the analyte

Matrix effects – assess impact of Matrix effects – assess impact of interfering componentsinterfering components

Recovery – Determine for quantitative Recovery – Determine for quantitative analysisanalysis

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Precision (Repeatability/Reproducibility) – Precision (Repeatability/Reproducibility) – determine:determine: Acceptable limitsAcceptable limits Qualitative/Quantitative – run 10 timesQualitative/Quantitative – run 10 times Non-routing Methods – may differNon-routing Methods – may differ

Trueness – quantitative analysis, assessed by:Trueness – quantitative analysis, assessed by: Replicate measurements at different Replicate measurements at different

concentrationsconcentrations Performance of standard addition methodPerformance of standard addition method Comparison to proficiency test resultsComparison to proficiency test results Comparison with different validated methodsComparison with different validated methods

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Range – determine concentration or Range – determine concentration or amount limits for which the method amount limits for which the method is applicableis applicable

Limit of detection (LOD) – lowest Limit of detection (LOD) – lowest amount of analyte to be detected and amount of analyte to be detected and identified (not necessarily identified (not necessarily quantitatively accurate)quantitatively accurate)

Limit of quantitation (LOQ) – lowest Limit of quantitation (LOQ) – lowest concentration that has acceptable concentration that has acceptable level of uncertaintylevel of uncertainty

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Linearity – quantitative methodsLinearity – quantitative methods Determine mathematical relationship Determine mathematical relationship

(calibration curve) between (calibration curve) between concentration and responseconcentration and response

LOQ forms lower end of working rangeLOQ forms lower end of working range Determine acceptable variation from Determine acceptable variation from

calibration curvecalibration curve Determine upper limits of working Determine upper limits of working

rangerange Robustness – alter method Robustness – alter method

parameters and determine any parameters and determine any changes to accuracychanges to accuracy

SWGDRUG GeneralSWGDRUG GeneralValidation PlanValidation Plan

Ruggedness – assess the Ruggedness – assess the factors external to the methodfactors external to the method

Uncertainty – assess Uncertainty – assess contribution of random and contribution of random and systematic errorssystematic errors

Quality control – establish Quality control – establish acceptance criteria prior to acceptance criteria prior to implementation of the methodimplementation of the method

Validation TypeValidation Type Qualitative analysis - the parameters Qualitative analysis - the parameters

that need to be checked are that need to be checked are selectivity, limit of detection and selectivity, limit of detection and reproducibility.reproducibility.

Quantitative analysis – the Quantitative analysis – the parameters that need to be checked parameters that need to be checked are selectivity, matrix effects, are selectivity, matrix effects, recovery, precision, trueness, LOQ, recovery, precision, trueness, LOQ, linearity, robustness, ruggedness, linearity, robustness, ruggedness, reproducibility and uncertaintyreproducibility and uncertainty

SWGDRUG Supplemental SWGDRUG Supplemental DocumentsDocuments

Adopted by the Core Committee in Adopted by the Core Committee in August 2005August 2005 The supplemental documents are not The supplemental documents are not

SWGDRUG recommendations SWGDRUG recommendations Intended to be a resource for those Intended to be a resource for those

responsible for implementing SWGDRUG responsible for implementing SWGDRUG recommendationsrecommendations

Not inclusive and SWGDRUG recognizes that Not inclusive and SWGDRUG recognizes that there are many ways of implementing the there are many ways of implementing the recommendationsrecommendations

Living documents and as such, SWGDRUG Living documents and as such, SWGDRUG invites comments. Send your comments to invites comments. Send your comments to swgdrug@hotmail.com swgdrug@hotmail.com

SWGDRUG SWGDRUG Validation of Validation of

Analytical MethodsAnalytical Methods Supplemental Document SD-2 for Part IVB entitled Supplemental Document SD-2 for Part IVB entitled

“Quality Assurance/Validation of Analytical “Quality Assurance/Validation of Analytical Methods” was Methods” was finalized for publication in February finalized for publication in February 20062006

Designed to assist laboratories develop a general Designed to assist laboratories develop a general validation validation planplan which meet their individual which meet their individual requirementsrequirements

Intended to be used in conjunction with SWGDRUG Intended to be used in conjunction with SWGDRUG Recommendations, Part IVB, Section 2Recommendations, Part IVB, Section 2

The document is organized into two sections:The document is organized into two sections: Section I provides guidance on the issues to Section I provides guidance on the issues to

consider when using various analytical techniquesconsider when using various analytical techniques Section II is an example of a completed validation Section II is an example of a completed validation

planplan

SWGDRUG Properties SWGDRUG Properties for Validation Processfor Validation Process Section 1 of the document is Section 1 of the document is

organized into Categories of organized into Categories of Analytical Techniques (IR, MS, Analytical Techniques (IR, MS, HPLC, etc.) from Table 1 – HPLC, etc.) from Table 1 – Categories A, B and CCategories A, B and C

For each instrument or method, For each instrument or method, specific properties are detailed specific properties are detailed that could have an effect on how that could have an effect on how the validation process is the validation process is formulated and executedformulated and executed

SWGDRUG Properties SWGDRUG Properties for Validation Processfor Validation Process

Section II demonstrates a purpose-Section II demonstrates a purpose-defined validation plan for a defined validation plan for a particular method.particular method.

The aim is to show how a complete The aim is to show how a complete validation plan may appearvalidation plan may appear

The example should not be directly The example should not be directly applied to methodology used by any applied to methodology used by any laboratory without first considering laboratory without first considering the specific purpose of a method and the specific purpose of a method and its relevant operational environmentits relevant operational environment

SWGDRUG Properties SWGDRUG Properties for Validation Processfor Validation Process

Supplemental Supplemental Document SD-Document SD-22

SWGDRUG Future PlansSWGDRUG Future Plans

SWGDRUG intends on SWGDRUG intends on adding additional adding additional supplemental supplemental documents to help documents to help illustrate validationsillustrate validations Actual validated methodActual validated method

Example qualitative and Example qualitative and quantitative validationsquantitative validations

Uncertainty examplesUncertainty examples

US Validation PerspectiveUS Validation Perspective Disclaimer: This information is solely Disclaimer: This information is solely

based upon my personal experiences in based upon my personal experiences in different laboratories in the USdifferent laboratories in the US

Validation requirements generally Validation requirements generally thought of only for quantitative analysisthought of only for quantitative analysis

Qualitative methods exist, but in a Qualitative methods exist, but in a general mannergeneral manner

Through influence of ISO, laboratory’s Through influence of ISO, laboratory’s are developing more specific qualitative are developing more specific qualitative methodsmethods

US Validation PerspectiveUS Validation Perspective Smaller labs have simple Smaller labs have simple

qualitative methods, but only qualitative methods, but only looking for a few specific looking for a few specific drugsdrugs

We previously relied upon We previously relied upon the analyst’s training and the analyst’s training and experienceexperience

Analyst discretion on which Analyst discretion on which instrument and/or method to instrument and/or method to utilizeutilize

US Validation PerspectiveUS Validation Perspective

Changing method Changing method parameter acceptable parameter acceptable to achieve to achieve identification (e.g., GC identification (e.g., GC oven temp.)oven temp.)

Fear of developing Fear of developing technicians as technicians as opposed to scientistsopposed to scientists

US Validation PerspectiveUS Validation Perspective DEA Analytical DEA Analytical

Sufficiency DocumentSufficiency Document Originally thought sufficient Originally thought sufficient

for qualitative analysisfor qualitative analysis However, no specific However, no specific

qualitative validation qualitative validation determining selectivity, determining selectivity, limit of detection and limit of detection and reproducibility at a reproducibility at a minimumminimum

US Validation PerspectiveUS Validation Perspective DEA recently accredited in ISO DEA recently accredited in ISO

1702517025 Developed general guidelines for Developed general guidelines for

qualitative testingqualitative testing Although we were accredited, we Although we were accredited, we

are currently deciding what are currently deciding what constitutes a qualitative method constitutes a qualitative method validationvalidation

QuestionsQuestions Make methods instrument and drug Make methods instrument and drug

specific?specific? Make methods only instrument Make methods only instrument

specific?specific?

US Validation PerspectiveUS Validation Perspective Problem with quantitative Problem with quantitative

methods, we have numerous methods, we have numerous ones for the same analyteones for the same analyte In upwards of 5-10 different In upwards of 5-10 different

methods varying minimal methods varying minimal parametersparameters

Considering standardizing Considering standardizing methodology and limit number methodology and limit number of choicesof choices

Questions ??Questions ??

Scott R. OultonScott R. Oulton

E-mail: E-mail: scott.r.oulton@usdoj.govscott.r.oulton@usdoj.gov

Phone: (760) 597-7906Phone: (760) 597-7906

A copy of this A copy of this presentation, can be presentation, can be downloaded at:downloaded at:

www.swgdrug.org/www.swgdrug.org/events.htmevents.htm