41
Good Measure: Driving Site Performance with Metrics-Based Communication April 2, 2015
Good Measure:Driving Site Performance with Metrics-Based Communication
April 2, 2015
About DrugDev
DrugDev is a technology company that provides cloud-based solutionswhich enable sponsors, CROs and investigators to do more trials together.
• Founded as a network of global investigators actively seeking new trials• Committed to driving technology standards to transform clinical operations• Acquired CFS Clinical in 2013 and TrialNetworks in 2014• Implemented a Innovation Lab as an incubator for novel clinical systems• Truly global solutions with offices in King of Prussia, London and Boston
9 of the top 10 pharmaceutical companies and 4 of the top 5 CROs trust DrugDev technology to improve their global clinical operations
2
What Does “Do More Trials” Mean?
• The pharmaceutical industry has no shortage of promising drug candidates
• It has become too costly, time-consuming and risky to test them all
• This inefficient drug development process is unsustainable and must change
Sponsors and CROs are eager to disrupt thecurrent process using modern technology thatmakes life easy for investigators and sites.
Our goal is to help you do more trials and bringmore drugs to patients who need them, faster.
3
Proven Technology Solutions
Find and activate ideal global investigators
Ensure accurate and timely site payments
Optimize clinops with a unified study platform
Match and master all site and investigator data
4
About Your Presenters
Bryan FarrowDirector of Trial Site ExperienceDrugDev TrialNetworks
Beth HarperPresidentClinical Performance Partners
Beth has been passionately pursuing solutions for enhancing site performance, optimizing enrollment and fostering improved sponsor-site relationships for over 30 years.
Bryan draws on his interests in logic, statistics, psychology and communication to help sponsors tap the total potential of the sites with which they partner.
5
Learning Objectives
• Explain why metrics are so powerful, and identify which ones matter most to sites
• Develop a site performance report template• Recognize, and reinforce the behaviors of, your top-
performing sites• Energize your lagging sites through transparent reporting of
their performance metrics• Carry out a root cause analysis of site performance issues• Lead a productive, supportive, and fact-based discussion with
sites to maximize their performance
6
Some Guiding Principles
• “If you don't know where you’re going,you'll end up someplace else.” – Yogi Berra
• “You can’t change what you don’t measure.”– (Incorrectly) attributed to statistician W. Edwards Deming.
7
Part One: Generating the Will
“How is my site doing…”– “…compared to my site’s goal?”– “…compared to other sites?”– “…compared to study goals and
targets?”
• Your sites should be asking these questions at least once a week.
• You should be telling them the answers every week.
• If they aren’t meeting the goal, what can we do about it?
8
Part Two: Showing the Way
“If they aren’t meeting the goal, what can we do about it?”(The Million Dollar Question, literally)
9
Generating the Will
Imagine your manager knocks on your office door one morning and hands you this spreadsheet…
Bryan’s Trial Management Report
Metric Value Metric Value DifferenceNumber of trials Bryan is managing
4 Number of trials the average CTM is managing
4 Equivalent
Number of Bryan’s trials on or under budget
3 Number of trials the average CTM has on or under budget
2 +50%
Number of Bryan’s trials on schedule
1 Number of trials the average CTM has on schedule
2 -50%
10
Generating the Will
• How would you react? With pride? Annoyance? Motivation? All are perfectly sensible.
• Chances are, you won’t feel complacence or indifference.
• “Let’s face it: any photo you’re not in, isn’t that interesting.” – Comedian Jim Gaffigan.
• A corollary: Any facts about me, especially metrics, are irresistible.
• Hence the rise of the Nike Fit and other “me”-centric apps.
• So how does this apply to enrolling patients and managing data for a clinical trial?
11
Generating the Will
Break your goals down and give them to the sites. Then…
Regularly (once a week to one a month, depending on pace of activity)
Always
Tell each site how they are progressing toward this goal.
Allow each site to find out for themselves.
Tell the sites how their performance compares to that of other sites in the trial.
Allow each site to find out for themselves.
Publicly congratulate sites for achieving milestones on the way to their goal.
Treat the site for what they are: a group of dedicated, highly-trained and passionate partners in bringing therapies to market.
12
Generating the Will
• Regular site performance reporting (enrollment and data management “biofeedback”) is essential to improvement
• You can provide that reporting with tools currently on your desktop– Access to IWRS and EDC reports– Excel– An email application
• For the greatest impact, you’ll need a secure study platform that empowers site members to find performance metrics (and other trial resources) on their own.
13
Generating the Will
• We’ll start with an initiative you can put in place this week.
• Let’s suppose you want to recognize those sites that have…– Been activated for six or more months– Achieved a “randomized per month” rate that is 20% or
greater above the rate for their country as a whole• Let’s further suppose that you want to incentivize those sites
that have…– Been activated for six or more months– Achieved a “randomized per month” rate that is 20% or
more below the rate for their country as a whole
14
Generating the Will
• A few basic formulas in Excel, once applied to data generated by your IWRS, can instantaneously tell you:– The randomization rate for each country– The randomization rate for each site– The variance between each site’s rate, their country’s
rate and the trial rate• Now how do you get that information into the hands of
your sites?• Obviously, you cannot disclose the recruitment activity of
Site A to Site B• But you can tell Site A and Site B exactly how they are
performing relative to trial averages.
15
Generating the Will
• That’s where mail merge, available with most email applications, comes into play.
• With Excel’s help, you’ve created a spreadsheet with data like this:
16
Generating the Will
Upload that data into your email application’s mail merge function and prepare an email like this…
“…In the <months> months since your site was activated, you’ve screened <screened> patients and randomized <randomized>. Your randomization rate of <site_rate> patients per month is <rate_difference> than that of the trial-wide rate, which currently stands at <trial_rate> patients per month…”
Or…
17
Generating the Will
“…. As of <date>, your site has <queries> queries greater than 14 days old still unresolved. Your rate of <site-query-rate> queries per subject is <query-rate-difference> that of the trial wide rate, which stands at <trial-query-rate> queries per subject…”
18
Generating the Will
• There is, at least practically, no limit to the number of variables you could include in your message.
• However, the usual rules of effective email communication remain in force. – Begin with a clear call to action. – Keep it short. – Clearly state how and to whom recipients
should respond.
19
Generating the Will
Designate one to two hours each week to this task, and you’ve effectively “spoken” to every site in your trial about their site.
20
Generating the Will
• Individuals consume information in different ways.
• Some prefer a narrative to data.• Hence the great utility of an online
platform, as opposed to just emails.
21
Let’s briefly tour some web-based tactics.
22
Understanding and Intervening on Site Performance Problems
23
If Sites Aren’t Performing (or Responding to) the Performance Metrics / Communication…
We need to understand
WHY!
Can You Guess…
How many potential root causes there are to low enrollment?
1. 152. 253. 504. 1005. 150
Because There Are ~150 Root Causes…• Contributing to low
enrollment – (let alone other site
performance problems)• It’s tempting to just start
treating the symptoms but first we need to understand the causes– And there are always
multiple factors at play
26
Diagnostic and Troubleshooting Framework
Site Is Not Enrolling
Site Can’t Find Patients
Patient is not aware of the trial
There is no study awareness
program in place
Site did not develop a
recruitment plan to build study
awareness
Site did not know how to do this
Sponsor did not request this
There are no recruitment
funds available to raise
awareness for the study
The funds have been exhausted
No funds were ever requested
There are no recruitment
materials available
Site did not create any
Sponsor did not provide any
Materials developed by not approved by IRB
Site is participating in
competing trials
Site isn’t actively looking
Site has lost interest in the
trial
The 5-Why’s Model
WHY?
WHY?WHY?
WHY? WHY?
How Might Relationships Affect the Problems/Solutions?
Site Not Enrolling
Site lost interest
Another more appealing
project
PI never really interested
Feasibility assessment
process never explored PI
interest
Feasibility assessment process relies on fax survey
only – no direct site communication
Feasibility process allows for direct site communication but “wrong” person at sponsor is communicating with “wrong”
person at site
Other project pays more
Site rarely hears from
sponsor
Sponsor SOPs require site visit only
after patient is enrolled
Site contact transactional +
tedious
CRA not well trained in effective
site communication
Training available but CRA doesn’t take
advantage
Sponsor doesn’t appreciate value of
strategic site communications and
site relationship management so
doesn’t provide CRA Training
CRA too busy to invest in site relationship
Site isn’t complying with the
protocol?
Site doesn’t understand design or rationale
Site hasn’t been
trained
CRA or site is Too
Busy
Site hasn’t understood the
training
Training effective but
site didn’t understand
rationale
Rationale not
presented
Rationale presented but sites
didn’t buy-in
Sites disagreed with rationale
Sites didn’t understand
protocol development
process
Sites weren’t included in
protocol development
process
Training approach wasn’t effective
Site wasn’t paying attention
Site doesn’t care or has lost Interest
How would your solution(s) vary depending on the root
cause?
The Same Methodology Can Be Applied to Any Type of Performance Problem
Site Isn’t Improving After Sharing the Performance Metrics
Site behavior
didn’t change
Site (in general) wasn’t aware of
their metrics
Site didn’t check email
Too busyLost in spam and
no one ever followed upSite didn’t log on
to the portal Didn’t know it existed
Lost access info / password
New (key) staff member at site
who wasn’t included in the
distribution lists or educated about
the metrics
Sponsor / CRO was never notified about staff changeSite is not
motivated by metrics /
comparative or competitive data
This type of data doesn’t appeal to them or drive their
motivations for conducting research
The mode of communication wasn’t effective
The key influencer at the site (e.g., PI or Director) prefers face-to-face discussion
on performance
Fact-based conversations can be foreign to many site facing personnel so coaching in effective conversations is often needed!
Speaking of Effective Conversations…
If you don’t enroll any more by next week we’ll have to
shut your site down!
How come I’m just now hearing about our enrollment issues?
(Hint: This type of conversation doesn’t qualify
as being productive!)
Coaching for Productive Recruitment Conversations
The same concepts apply to any type of performance related conversation!
33
34
35
36
Re-Cap
• It’s tempting to just start doing something to intervene on site performance issues
• Taking the time to do a thorough root cause analysis can help focus your efforts and ensure you see an impact
• Finding out the best way to present and finesse the information ensures that your message doesn’t get lost
37
Key Points
• Measure, not only for yourself, but for every member of every site.
• Numbers tell a powerful story, but not the whole story.
• Data sets the stage for a mutually-informed, in-person discussion: one about solving performance issues.
38
So Tell Us… Did We Meet Our Objectives?
• Explain why metrics are so powerful, and identify which ones matter most to sites
• Develop a site performance report template• Recognize, and reinforce the behaviors of, your top-performing
sites• Energize your lagging sites through transparent reporting of their
performance metrics• Carry out a root cause analysis of site performance issues• Lead a productive, supportive, and fact-based discussion with
sites to maximize their performance
39
Please Feel Free to Contact Us!
Beth [email protected]
Bryan [email protected]
@engageyoursites
We’d like to follow up with you in six months to see if you were able to implement any of these communication tactics and learn about your results. Survey coming in the Fall!
40
Thank You!
