Causal Leverage AnalysisLance Shea

PartnerBakerHostetlerWashington, DC

Substantiating Claims With Patient Input Evidence

Data, Information & Evidence• Although “data,” “information,” and “evidence” are often used as if

they were interchangeable terms, they are not. • Data are best understood as raw measurements of some thing or

process. By themselves they are meaningless; only when we add critical context about what is being measured and how do they become information.

• That information can then be analyzed and combined to yield evidence, which in turn, can be used to guide decision making.

• Robert M. Califf and Rachael Sherman, What We Mean When We Talk About Data, FDA Voice (Dec. 10, 2015), available at http://blogs.fda.gov/fdavoice/index.php/2015/12/what-we-mean-when-we-talk-about-data/ (last accessed Jan. 9, 2017).

Evidence• Substantial Evidence• “What evidentiary support should a firm have for its communications

that are consistent with the FDA-required labeling [for a product that already has been approved by the FDA]?”• “[G]ounded in fact and science”• “[P]resented with appropriate context”• “[S]cientifically appropriate”• “[S]tatistically sound”• “[A]ccurately characterized”

FDA, Guidance for Industry – Medical Product Communications That Are Consistent With the FDA-Required Labeling – Questions and Answers 6 (Draft Guidance 2017).

Evidence

• Investigator measured data (endpoints)

• Patient input data

• Patient perspectives

• Patient perspective “information”• Qualitative PPI

• Quantitative PPI

• Patient reported outcomes (endpoints)

Challenges Generally

“It can be proven that most claimed research findings are false.”

Ioannidis, Why Most Published Research Findings Are False, PLoS Medicine 2:8, e124 (2005).

Challenges Generally

“Tens of billions of dollars of public and private money are invested globally in trials every year . . . Many of these resource[s] are wasted, often because . . . insufficient account is taken of existing evidence when choosing questions to address . . . .”

Treweek, et al., Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform, Trials 16:261, 2 (2015).

Challenges Generally• [V]alid causal inference from nonrandomized studies about

treatment effects depends on many factors other than confounding. These include:• whether the causal question motivating the study is clearly specified, • whether the design matches that question and avoids design biases, • whether the analysis matches the design,• the appropriateness and quality of the data, • the fit of adjustment models, and • the potential for model searching to find spurious patterns in vast

data streams. Goodman, et al., Using Design Thinking to Differentiate Useful From Misleading Evidence in Observational Research, JAMA 317:7, 705 (2017) (formatting supplied).

Challenges Generally

“Some of these issues are recognizable and remediable, whereas others may defy solely analytic solutions, leading to the rejection of some nonrandomized studies to guide treatment.”

Id.

Challenges Generally• “Traditional risk adjustment in observational studies is

vulnerable to differences in unmeasured or unknown prognostic factors between groups (residual confounding) that leave the results open to bias. . . .

• If RCTs cannot be conducted, it will remain impossible to determine whether adjusted estimates are accurate or misleading.”

Agoritsas, et al., Adjusted Analyses in Studies Addressing Therapy and Harm Users’ Guides to the Medical Literature, JAMA 317:7, 748-759, 758 (2017) (formatting supplied).

Challenges Specifically

• “[P]harma companies . . . need a more effective data analytics practice . . .

• However, pharma companies must be careful to avoid falling for the myth of big data . . ..

https://www.strategyand.pwc.com/trend/2017-life-sciences-trends.

Challenges Specifically

• [I]ndividual patient preferences may vary . . .

• A patient may not assign the same values to various risks and benefits . . .

FDA, Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling – Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders 7 (2016).

Challenges Specifically

• A torrent of data

• Subjective, variable and often uncontrolled

Causal Leverage Analysis

• Science-driven analysis of product performance evidence for lifecycle defense and expansion.

• CLA = legal + technical

CLA – Nexus of Law and Science

• Law• Circumstantial evidence

• Relevance

• Probativeness

• Materiality

• Proximate causation

• But-for cause

• Legal cause

• Science• Association

• Reliability

• Validity

• Fit

• Causal inference

• Counterfactual

• Causal inference factors

Why “Causal Leverage” Analysis

“[C]ausal inference is . . . embedded in public health practice . . . regulatory processes . . . [products liability] legal proceedings . . . [and] principles of evidence-based medicine . . . .”

Why “Causal Leverage” Analysis

"The history of public health and of its quantitative disciplines, epidemiology and biostatistics, can be seen as one long discourse on disease causation, the ultimate targets of which are to find and to mitigate reversible causes."

T. Glass, S. Goodman, et al., Causal Inference in Public Health, Annu. Rev. Public Health 34:34:61-75, 62 (2013).

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Categorize Important

Information

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

CLA – Assessing Weight of Evidence

Research Question

Association?

Casual Inference?

Causal Inference Factors• Experiment• Dose timing• Subject selection• Effect strength• Dose-response• Biological mechanism• Consistency• Specificity• Analogy

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Evidence• Substantial Evidence• “What evidentiary support should a firm have for its

communications that are consistent with the FDA-required labeling [for a product that already has been approved by the FDA]?”• “[G]ounded in fact and science”• “[P]resented with appropriate context”• “[S]cientifically appropriate”• “[S]tatistically sound”• “[A]ccurately characterized”

Causal Leverage AnalysisFrame or

Understand the Research Question

Synthesize Prior Knowledge to

Collect Important Information

Categorize Important

Information

Assess Reliability of Important Information

Assess Aggregate Evidence of

Association or Causal Association

Appendix

Definitions

Patient Information• “Patient input” includes a wide range of information and

perspectives including anecdotal comments . . . patient opinions . . . patient responses to qualitative ad hoc surveys, and quantitative measurements of patient-reported outcomes.

• “Patient perspectives” refer to a type of patient input, and includes information relating to patients’ experiences with a disease or condition and its management. . . . “patient preference information” [is] . . . one specific type of patient perspective.

Patient Information

[P]atient preference information . . . is defined as: qualitative or quantitative assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions.

Patient Information• [Q]ualitative PPI may be useful in identifying which

outcomes, endpoints or other attributes are valued most by patients and which factors affect patients’ perspectives on risk and benefit.

• Quantitative PPI can provide estimates of how much different outcomes, endpoints or other attributes are valued by patients, and the tradeoffs that patients state or demonstrate they are willing to make among them.

Patient Information

A patient-reported outcome (PRO) is any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.