Clinical KPIs and Partnership Optimization - cbinet.com1).pdf · The median time from subject visit...

© Copyright 2017 TriRadial Solutions

Key Performance Indicators (KPIs) and Sponsor/CRO Partnership Optimization

February 23, 2017CBI Clinical Trial

Budgeting & Forecasting

Geoff GarabedianTriRadial Solutions


Agenda

2

Ø Outsourcing Trends and Partnership Dynamics

Ø Lessons Learned –Sponsor and CRO Perspectives

Ø Metrics to Help Optimize Partnership Collaboration


Outsourcing is increasing, with more focus on partnerships, but progress is slow

3

Bene

fits

PreferredProviders

StrategicPartners

AlliancesandJointVentures

DegreeofIntimacy

Tactical/Transactional

1

2

3

4

Accordingtoarecentsurvey…• 90%ofsponsors

plantoincreasetheiruseofCROs

• 98%indicateCROoversightisatopapriority

• 34%feeltheyachievedtheirplannedCROgoals

• Lessthan20%believetheirrelationshipsareproductive

Surveysource:ComprehendSystems,Inc.CROOversightBenchmarkReport,2016

reality

target


A trust deficit in most partnerships limits potential speed, quality and efficiency

4

3KeyFactors:

Lackofaccesstoreal-timeinsightstoinvestigateandresolveissuesquickly

Ø DatatransparencymostoftencitedasakeyrequirementforimprovingCROoversightandcollaboration

1

Metricsnotalignedwithcommongoalsacrossthepartnership(s),aretooreactive,anddonotpromoteinnovation

3

UnnecessaryoverlapandsharedresponsibilitiesbetweenSponsorsandCROs2

CurrentState TargetState


Effective CRO governance is based on a common vision and driven by metrics

MissionVisionValues

Program&Partnership

Goals

Structureand

ResourcesProcesses

TechnologyandData

Metrics

ContinuousImprovement

5

Ensureeachcomponentmaximizessuccessandreturnsforboth parties

© Copyright 2017 TriRadial Solutions 6

Lessons Learned


Sponsor Perspectives

• Most sponsors do not have enough visibility to issues as they arise, let alone a way to predict when issues are looming

• The governance plan, including issue escalation and quality agreements, is developed in a silo or not agreed to upfront

• CROs often more focused on “the relationship” and follow-on work• Penalty clauses alone do not typically drive better performance;

incentives hold more promise and can drive more long-term value• Overlapping roles & responsibilities cause inefficiencies and confusion• Metrics are sometimes too narrowly focused or confining• Site performance not necessarily a good indicator of CRO performance/

quality• Many companies do not engage sites regarding metrics or feedback

7


CRO Perspectives

• Sponsors create unnecessary oversight layers and want visibility to day-to-day details that do not necessarily impact trial success

• Little opportunity to get involved in design activities• Sponsors talk about strategic partnerships, but actions demonstrate

more tactical outsourcing of specific activities (but at ”preferred pricing”)• Overlapping roles & responsibilities cause inefficiencies and confusion• Sometimes not a “unified voice” to the customer• Often little or no incentive to being more efficient or innovative• Vision and strategy not always communicated throughout the

partnership; employees are not motivated to go above and beyond in support of the strategy

8


Development of KPIs


SourceDataSystems

SourceDataSystems

Just because you can measure something, doesn’t mean you should

10

Metrics (and Other Information

Requirements)

Questions (for Assessing Goal Performance)

Information

Goals (and Other Performance Expectations)

AssessedBy ...

AnsweredBy ...

Comprising ... Provide DataPoints for ...

To Answer ...

To AssessPerformanceRelated to ...

InternalandExternal

Stakeholders

Takeasystematicapproachtodefiningmetricsthatalignwithbusinessandpartnershipgoals

SourceDataSystems

Thereshouldbeoneversionofthetruth,visibleacrossthepartnership


Considerations for Metrics Development

• Develop measures along multiple dimensions

• Measure process performance, not people performance

• Align metrics with the quality management system

• Don’t optimize separately for each partner

11

• Leverage benchmarks and historical data to set thresholds

• Incorporate the site perspective to help evaluate CRO performance

• Use 360o survey tool across the partnership to assess ongoing collaboration/value

• Develop role-based dashboards that allow management by exception


Balance the types of metrics being used

12

Metric Type Description Example Metric

Leading Indicator

Provides information that you can act on immediately to get the trial/ process back on track.

The proportion of sites activated versus expected would be a leading indicator for whether subject enrollment is likely to be on track.

Lagging Indicator

Provides information that you can use for future trials or for baselining for process improvement efforts.

The time taken from “last subject last visit” to database lock.

Cycle-time Measures the time taken to complete a task.

The median time from subject visit to data entry into an electronic data capture system.

Timeliness Measures whether a particular milestone has been met.

The number of days between planned and actual dates of the first site activated.

Efficiency Measures the amount of resource required to complete a task or set of tasks versus that expected.

The difference between the actual final total contract value and the initial baseline contract value for a CRO running a clinical trial.

Quality Measures how well an output from a process meets the requirements of the customer of that process.

The proportion of expedited safety reports that are received by regulatory authorities within the required timelines gives an indication of the quality of the pharmacovigilance reporting process.


StudyStart-Up StudyConduct StudyCloseout

A balanced set of measures is critical for long-term success

13

Ø Initial KPIs will depend on data available

Ø Incorporate satisfaction surveys and other feedback mechanisms

Ø Actively review and discuss measures jointly during governance meetings

Time

Cost

Quality

Compliance

Planning/ Performance

Partnership Health


Dimension Metric Definition

Time

AwardtoExecutedContract Cycletimefromprojectofficiallyawarded tofullyexecutedWorkOrder(calendardays)

DataSystemsAvailable %DatabasemanagementandDataCaptureSystemsgo-livewithintheagreedtimelines

SiteActivationTime–contractsandessential docs

Cycle-timepersitefrom initialcontact withInvestigatorto signedcontractandallessentialdocumentsreceived&IRBapproval

MajorMilestonesAchieved %keystudymilestonesmetorexceededvs.baseline(e.g.,ProtocolApproved,eCRFApproved,FPI, LPI,LPLV,DBL,FinalCSR)

COTurnaroundTime DaysfromCOnotification toCOexecution

%DBLDelays PercentDBLsthatdonotoccur ontimeduetosponsororCROconstraintsvs.totaldelays

QueryResponsetime Averageresponsetimetoqueryresolution

IssueClosureTime Averagetimetocloseopenissues

Example Metrics and Definitions



Cost

%BudgetVariance(Plannedvs.Actual)

Costvariancefromoriginalworkorderforstudyor portfolioofstudies.Calculatedbycategory(labor,pass-throughcostsandinvestigators)andfortotalstudycosts

FinancialPlans/ToolsInPlace %oftrialswithallpartnershipfinancialtoolsinplacebyFPI(ifrequired)

ChangeOrderImpact %IncreaseinbudgetduetoChangeOrders

TimelinessofPayments Average timefrominvoicedatetopayment

OversightFTERatio MeasureofsponsorandCROemployeesresponsibleforstudydelivery/managementtoindicatelevelofoversight(andredundancy)

LaborUnitCosts(e.g.,pervisit,persite,permonth, perunitofdatacleaned)

Costofrelevantstudycomponents dividedbyapplicablevariable(e.g.,monitoringbudgetand#ofenrolledsubjects).Canalsoincludevariancestoplan

15




Quality

%ofTrialQualityManagementPlansinPlacePriortoFPI

%oftrialswithsignedQualityManagementPlansinPlacePriortoFPI

PatientRecruitment %ofsitesrecruitingmorethat50%ofplannedsiteenrollment

PatientRetention %patientsretained:patientsrandomized/dosedineachstudywhoarenotlosttofollowupanddonotwithdrawconsent

DocumentQuality Qualityofdocumentation, assessedbysponsor

CRAQuality Qualityofmonitoringstaff, assessedbysite

#ofCriticalSiteAudit/InspectionObservations

Totalnumberofsiteauditand/orinspectionfindingsthatarecritical

# ofDatabaseUnlocks #ofunplanned databaseunlocks(i.e.,DBunlockstosolvedataissuesdetectedafterlock,requiringre-transferandre-analysis)

16




Compliance

ProjectManagementPlans inPlace

%oftrialswithPMPsignedandin placepriortoFPI(ormutuallyagreeddate)

SAEReportingCompliance %ofreportedSAEswithinregulatorytimelinesvsthetotalnumberofreportedSAEs

SUSARReportingCompliance %ofreportedSUSARswithinregulatorytimelines vsthetotalnumberofreportedSUSARs

ProtocolDeviations/SiteCompliancetoProtocol

%ofenrolledsubjectswithprotocolcritical/majordeviations/issuesdefinedaspervariouscategories(e.g., eligibility/entrycriteria,IP,ICF,sourcedoccriteria,SAE)

PartnershipTrainingCompliance

%compliance(trainingcompletedvs.required)forpartnership-specificprocessesandSOPs

17




Planning/Performance

SiteActivation – drugsupply %ofsitesreadytoenrollwithdrugreceivedatsite

TimelineShifts #oftimestimelinesshiftedperstudy

Plannedvs.ActualSitesOpen(readytoenroll)

%oftrialsmeetingorexceedingplannedvs.actualsitesopen(readytoenrollsubjects)

EnrollmentPlanAccuracy(plan vs.actual)

%oftrialsmeetingorexceedingplanned vs.actualenrollmenttargets(randomization)

Non-EnrollingSites % ofsiteswhohavenotenrolledasubject(siteinitiationvisitdate>100days)vs.totalnumberofsitesinitiated

ProtocolAmendment Rate Totalnumberofprotocolamendmentsdividedbynumberofactiveprotocolsduringthesameperiod

18




PartnershipHealth

Appointment ofStafftoKeyRoles

Keystaffinplaceasagreedpercontractandtimelines

StaffTurnoverRate %ofsponsororCROstaffdedicatedtoaPartnershipstudyatthebeginningofthequarterbutnolongerworkingonthesamestudyattheendofthequarter

Innovation NumberofCRO-generated ideasthathaveapositiveimpactonthedesign,qualityorperformanceofatrial

360PartnershipSatisfactionSurvey

Periodic(e.g.,quarterly) scoresfromstandardsurveytoindicateemployeeexperienceandengagementinPartnership (opentoallsponsorandCROpersonnelinvolvedinpartnership)

ExternalInvestigator/SiteSatisfactionSurvey

Scoresfromsurveyofstudysite personnel(e.g.,investigator,nurses,studycoordinators,etc.)

19



Detailed Indicator Definitions

20

Eachmetricshouldhaveadetaileddefinitionthatgoesbeyondadescriptionandformula

Metric#:32a-1Name:MeanStudyStart-upTime

KPICategory:CycleTime

Sub-category:ClinicalStudyCycleTime

Description:Daysfromprotocolapprovaltofirstsubjectrandomized,forallsitesinstudy

UnitofMeasure:CalendardaysFormula/Definition:Median#calendardaysfromCSPtoFSI

Calc.Frequency:EndofstudyDataSource:CTMS

ReportingLevel(s):Study,Portfolio

Target:90Days

Leading/Lagging:LaggingRelatedKPIs:

BusinessGoalLinkage:

AdditionalAnalysisDrill-down:

Comments:


Summary

Ø It’s not all about metrics – Spend time on up-front collaboration to develop a shared vision, define common goals and select appropriate KPIs to measure those goals

Ø Take a holistic view – Agree on a governance plan that proactively manages the relationship and provides effective oversight across several areas (cost, quality, performance, issue escalation, etc.)

Ø Find a Balance – Identify a mix of activity-based and outcomes-based measures, along multiple dimensions, that align with goals

Ø Simplify – Focus on a manageable set of KPIs that incent the right behaviors

Ø Visualize – Develop a set of role-specific dashboards that help improve performance through data-driven decision making

21


Geoff GarabedianM: 201.218.8827

[email protected]

Date post:	11-Aug-2019
Category:	Documents
Upload:	phungdan
View:	214 times
Download:	0 times

Clinical KPIs and Partnership Optimization - cbinet.com1).pdf · The median time from subject visit...

Documents