Screen Design and Assay Technology Special Interest Group (SIG)

February 6th, 2017

Cell-Based In Vitro Assay Automation: Balancing Technology & Data Reproducibility/Predictability

Adam Weinglass, Mary Jo Wildey Merck & Co., Inc., Kenilworth, NJ, USA

Challenge

• Supporting a given GPCR program with structurally distinct classes of molecules classes at different stages of optimization.

• Ensuring In vitro support for a target executed within Merck and at a CRO.

• Across companies, scientists’ experience & access to instrumentation and automation varies.

• Driving Chemistry SAR requires reproducible data between operators and sites.

• Alignment of protocols & technologies critical-requires communication.

• All data is uploaded to the same database where common tools are used for analysis.

Mission High quality, reproducible & comparable data across sites every week!

Agonism of Gq-coupled GPCR monitored using IP1 accumulation assays in Tier 1 & 2 of ROP

GPR40 Agonist

• Homogeneous Time-Resolved Fluorescence (HTRF) based assay to measure the accumulation of D-myo-inositol monophosphate (IP1) – a metabolite of IP3.

• LiCl in the assay buffer inhibits phosphatases which degrade IPs & leads to IP1 accumulation.

• Competition – intracellular accumulation of IP1 inhibits the FRET signal. Assay Procedure

Day 1: • Seed cells in 384 well plates in growth medium. • Incubate cells overnight at 37ºC/5% CO2 Day 2: • Remove growth media from cells. • Add Stimulation buffer containing LiCl. • Treat with compound (Echo transfer) for 1 hour @

37ºC/5% CO2. • Lyse cells and add FRET reagents. • Read on the Envision.

GPCR1/HEK293 IP1 Accumulation assay performing acceptably at CRO using statistical criteria (MSR Analysis)

Potency

Mean Ratio; Minimum Significant Ratio

• Classical Z’>0.4 does not guarantee reproducible EC50/IC50-need a ‘global reliability index’ of a dose response screen.

• Minimum Significant Ratio (MSR) defines the

‘smallest potency ratio between any pair of compounds that is statistically significant.

[SD is the standard deviation of the replicate EC50 results from an N=2].

SDMSR *2210=

[Eastwood et al. (2008), JBS] [Andy Lau (MRL Statistics)]

Issue: Chemistry concerned about EC50 variability observed in a new structural series of cpds in GPCR1/HEK293 Assay @ CRO:

Variability not captured in GPCR1 Tracking Metrics

• Performance formally evaluated periodically by MSR analysis. • Ref. EC50, Z’ & Signal/Baseline tracked in every assay. • Tracking metrics not ‘designed’ around distinct structural series-

relevant to ‘well-behaved’ series.

MSR Analysis

Tracking Metrics

Investigation 1: Evaluate variability of structurally diverse series within MRL using GPCR1/HEK293 assay

Dump/Pat N=1 vs. N=2 (separate days)

1

10

100

1000

10000

1 10 100 1000 10000

Run

1

Run 2

0.1

1

10

100

1000

10000

10 100 1000 10000

Rat

io

Geometric Mean

Potency Ratio vs Geometric Mean Potency

Ratio

MR

RLs

LsA

1

10

100

1000

10000

100000

1 10 100 1000 10000 100000

Run

1

Run 2

0

50

100

150

200

250

300

0 50 100 150 200 250 300

Run

1

Run 2

MR 0.5, MSR 19

MR 1.2, MSR 5.81

Potency Efficacy

KW

CRO

Variability within Merck. Variability between sites.

Dump/Pat @ KW, single operator Dump/Pat:

KW vs CRO

Problem Statement Both Intra- & Inter-site metrics for assays struggling to meet criteria for SAR

support with Dump/Pat

• Structural series identified that perform inconsistently in GPCR1 (and GPCR2) IP1 assays; variability in EC50 observed when using the “Dump/Pat” method to remove growth media:

• Variability between companies (MRK vs CRO), departments (Biology vs Assay Operations) & biologists (scientist 1 vs. scientist 2).

• How can we modify these primary assays to generate reproducible results regardless of who or where the assay is run?

– ‘Simplify’ the assay protocol to eliminate inconsistencies

• Based on the high plasma protein binding of chemical matter, considered the following steps to improve consistency.

– Run in serum, so residual amounts from Dump/Pat no longer an issue.

– Find a means to consistently remove residual serum (i.e. move away from Dump/Pat) that can be applied to all orthogonal assays across sites.

Methods for Removing Growth Media • Dump/Pat

– Widely used “traditional” technique for adherent cell based assays – Manually invert plate, shake/flick plate several times over a waste reservoir, blot

the plate on paper towels several times – Highly inconsistent between operators as well as with the same operator within

a day or across days • Cell Suspension

– Eliminate the cell adherent process (no overnight incubation); same day assay – Complete removal of serum in the assay (Wash cells in assay buffer) – Aligns with the protocol used for cAMP assays

• Centrifugal Washer (BlueWasher) – Uses centrifugation to remove buffer from assay plates, designed to be used as

an automated “dump/pat’ paradigm – No contact with cell monolayer – Variety of programming options which can be used with or without a dispensing

protocol • Plate Washer/Dispenser (BioTek, GNF, PlateMate etc.)

– Automated method to aspirate from and dispense into an assay plate – Complicated programming, risk of losing cells, known residual volume

remaining in the wells, variable between cell types, very finicky

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

0.1

1

10

100

0.1 10 1000

Rat

io

Geometric Mean

Potency Ratio vs Geometric Mean Potency Ratio

MR

RLs

LsA

RefLine

0

50

100

150

200

250

300

350

0 50 100 150 200 250 300 350

Run

1

Run 2

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

MR 1.1, MSR 5.14

Potency Efficacy

MR 7.23, MSR 45.9

KW

Dum

p/P

at

KW Suspension

Unacceptable variability. Cpds more potent in suspension mode & capture ‘false negatives’

Dump/Pat vs. Cell Suspension Suspension @ KW, same operator, separate days:

Solid Correlation

Methods for Removing Growth Media Suspension cells: Challenging assay performance using GPCR1/HEK293 cells

and a wrinkle/opportunity with structurally distinct series

Washing by Centrifugation

10 Reproducible/homogenous; Low %CV; No interference with cell layer.

0.1

1

10

0.1 10 1000

Rat

io

Geometric Mean

Potency Ratio vs Geometric Mean Potency Ratio

MR

RLs

LsA

RefLine

MR 0.84, MSR 2.7

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

0

50

100

150

200

250

300

350

0 50 100 150 200 250 300 350

Run

1

Run 2

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

Potency Efficacy KW Bluewasher

KW

Dum

p/P

at

MR 12.76, MSR 42.79

Dump/Pat vs. Bluewasher Bluewasher @ KW, same operator, separate days:

Strong Correlation

Acceptable variability. Cmpds are more potent in

Bluewasher paradigm & capture ‘false negatives’

Methods for Removing Growth Media Centrifugal washing: Acceptable assay performance using GPCR1/HEK293

cells and the same wrinkle/opportunity

Comparison of Growth Media Methods for GPCR1/HEK293 Cells: Structurally distinct series is sensitive to media removal paradigm - centrifugal

washer provides acceptable assay statistics Dump/Pat vs. Suspension

Dump/Pat vs. Bluewasher

Suspension vs. Bluewasher

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Run

1

Run 2

Dum

p/Pa

t

Dum

p/Pa

t

Susp

ensi

on

Suspension Bluewasher Bluewasher

MR 12.76, MSR 42.79 MR 1.2, MSR 5.88 MR 7.23, MSR 45.9

Across several assays/programs, centrifugal washing provides a reliable & consistent means to remove growth media within the context of a statistically robust assay

GPCR1/HEK293 IP1 Assay GPCR1/HEK293 IP1 Assay

13

Which assay reality is the right one?

GPCR1/HEK293 Dump/Pat • Benchmark to in vivo efficacy. • Reliable for certain classes.

GPCR1/HEK293 Centrifugal Washer • Closer to serum-free ‘intrinsic’ potency. • Improved consistency in the data. • Decreases the rate of “false” negatives.

MSR=42.8 on test set with distinct structural class

GPCR1 assay format changed/clarified SAR Relatively inactive historic cpds suddenly aligned with modified media removal

paradigm

In vitro profile aligns better with translatable assays

O

O NOH

O

L-001842278hGPR40 IP1 EC50 224 nMhGPR120 IP1 EC50 424 nM

L-002485812 (Racemic)hGPR40 EC50 103 nMhGPR120 EC50 318 nM

O

O NOH

O

L-002190770 GPR120 LID Compoundh GPR40 IP1 EC50 3440 nM Really??hGPR120 IP1 EC50 105 nM

O

O NOH

O

L-002312918hGPR40 IP1 EC50 324 nMhGPR120 IP1 EC50 121 nM

O

O NOH

O F

O

O NOH

O

L-002190770 GPR120 LID CompoundhGPR40 IP1 EC50 154 nM

Bluewasher; Cpd A (Racemic) hGPCR1 EC50 103 nM

Bluewasher; Cpd C

hGPCR1 EC50 324 nM

Bluewasher; Cpd B

hGPCR1 EC50 224 nM

Dump/Pat Historic Data; Cpd X hGPCR1 EC50 3440 nM-??

Bluewasher; Cpd X hGPCR1 EC50 154 nM

Biology/Pharmacology Team→Happy

Updated Project Status

Program Team→Comfortable

Automation Team→Not so Happy

Methods for Removing Growth Media • Dump/Pat

– The widely used “traditional” technique for adherent cell based assays – Manually invert plate, shake/flick plate several times over a waste reservoir, blot the plate

on paper towels several times – Highly inconsistent between operators as well as with the same operator within a day or

across days

• Cell Suspension – Eliminate the cell adherent process (no overnight incubation); same day assay – Complete removal of serum in the assay (Wash cells in assay buffer) – Aligns with the protocol used for cAMP assays

• Centrifugal Washer (BlueWasher) – Uses centrifugation to remove buffer from assay plates, designed to be used as an

automated “dump/pat’ paradigm – No contact with cell monolayer – Variety of programming options which can be used with or without a dispensing protocol

• Plate Washer/Dispenser (BioTek, GNF, PlateMate etc.) – Automated method to aspirate from and dispense into an assay plate – Aspirate/dispense heights and positions must be programmed and periodically checked

to minimize risk of losing cells – Residual volume remaining in the wells – Variable between cell types

Plate Washers • Several vendors and models of plate washers available for walk-up use or robotic

integration – Automated method to aspirate from and dispense into an assay plate – Aspirate/dispense heights and positions must be programmed and periodically checked

to minimize risk of losing cells – Biotek EL406 is currently the favored online and off-line plate washer

• In an effort to standardize, minimize error, and improve efficiency of the GPCR1 and GPCR2 IPOne assays, the manual method was successfully transferred to an integrated robotic system

Walk-Up vs. Integrated Automation

Walk-Up Protocol Integrated Automation Protocol

Manual movement of assay plates between instruments

Hands free automation of assay steps

Imprecise plate processing times Considerable reduction of biologist’s time in assay execution

Less accurate incubation times of assay plates with compound/agonist

Software-controlled plate pace timings and plate order processing

Increased chance of human error such as plates being processed out of order or being dropped

Accurate incubation timings throughout assay execution

Limited number of plates possible per assay run with numerous compound plates being screened against multiple cell lines

Capacity for increased number of compounds across multiple cell lines which can be processed in parallel

GPCR1 and GPCR2 IP1 HighRes Automation Flow

Read EnVision

615/665nM

Media Exchange/Stimulation

Buffer Addition Biotek

Washer/peripump dispenser

10ul change tips

Compound and controls addition

Echo 555 50nl

IP One Standard

Curve Addition

Agilent Bravo 10ul, columns

1&24

Detection Addition

Multidrop Combi 10ul

Incubation Cytomat C-24

Incubator 1 hour @37deg. C

Incubation Cytomat C-24

Incubator 1 hour @RT

Prep time: ~30 minutes

Automated/Unattended time: ~5 hours

Walk-up vs. Integrated Automation Assay Validation

GPR120 IPONE Human cells: manual vs HiRes(2) Run 1 & Run 2

0.1

1

10

10 100 1000 10000

Rat

io

Geometric Mean

Potency Ratio vs Geometric Mean Potency

Ratio

MR

RLs

LsA

Ref Line

Statistical Analysis of "Bland-Altman" PlotEquivalence Test Results Reproducibility Test Results

N 18MR 1.30 MSR (Within-Run) 1.81RLs 1.12 LsA 0.72

1.50 2.34Sig Diff Between SD 0.0908

Runs Test, p = 0.0017

GPR120 IPONE MOUSE cells: manual vs HiRes(2) Manual & HiRes

0.1

1

10

10 100 1000 10000

Rat

io

Geometric Mean

Potency Ratio vs Geometric Mean Potency

Ratio

MR

RLs

LsA

Ref Line

Statistical Analysis of "Bland-Altman" PlotEquivalence Test Results Reproducibility Test Results

N 18MR 0.98 MSR (Within-Run) 1.87RLs 0.84 LsA 0.53

1.15 1.83Sig Diff Between SD 0.0957

Runs Test, p = 0.8205

•MR represents the average potency ratio across the 2 runs

•MSR represents the largest potency ratio that can be considered a random change within a run of the assay

MR acceptance criteria 1+/- 0.5 MSR acceptance criteria 3+/- 1.0

GPCR2 GPCR2

Correlation Across Multiple Cell Lines with Integrated Automation and the Biotek Washer

Mouse Cell Line

R2= 0.977

Manual vs. HiRes with 45-degree Line

10

100

1000

10000

10 100 1000 10000

Run

1

Run 2

R2=0.976

Manual vs. HiRes with 45-degree Line

10

100

1000

10000

10 100 1000 10000

Run

1

Run 2

Human Cell Line

Walk-Up and Automated Assay show good correlation across both cell lines

Root Cause Discussion Observations

• In-house biology team – Expert in vitro biologists – Most will prefer Dump/Pat – Competent with user-friendly (robust) walk-up automation – Relies on automation staff for fine tuning and QC on systems

like plate washers

• External biology team – Expert in vitro biologists – Prefers Dump/Pat – Competent with user-friendly (robust) walk-up automation – Do not have access to equivalent automation staff for fine

tuning and QC on systems like plate washers

• Automation team – Experts in walk-up and integrated automation – Understand the need to monitor and adjust devices such as

plate washers for plate type, lot changes, cell line types

• Assumption that all methods were equivalent all the time so it was OK to substitute dump/pat for the plate washer

• Underestimated the shortcomings of dump/pat and the impact of residual media

• Couldn’t predict the chemistry sensitivity to the residual serum

• We had the perfect storm…… ….and a new solution…. . . . that blurs the skillset lines of assayists and automation

Root Cause Discussion Observations

Building Fit-for-Purpose In vitro Assays: Points for Discussion & Lessons Learned

• How do we design in vitro cell-based functional assays to improve accuracy/translatability within the context of infrastructure? • Understand how broader team are using the data (e.g. ranking cpds

vs. interpreting in vivo efficacy data/making human dose projections). • Ensure ‘identical’ conditions are used for orthogonal assays that may

ultimately be compared (e.g. cross-species evaluation).

• Be appreciative of the physiochemical properties of chemical matter being evaluated (e.g. PPB, sticking to plastics).

• Try to project! Evaluate the skillset of all the different groups that may be involved in developing/running assays during program lifetime. • If assays are likely to be externalized to CRO’s or transfer

between Departments, assess whether all groups have the same equipment infrastructure-do they need to?