Date post: | 01-Apr-2015 |
Category: |
Documents |
Upload: | kendrick-perham |
View: | 213 times |
Download: | 1 times |
David C. Chang, PhD, MPH, MBADirector of Outcomes ResearchUCSD Department of Surgery
Introduction to Outcomes Research Methods and Data Resources
Surgery and public health
Problem in surgical clinical research
•Unregulated
•FDA regulation applies only to “devices” (whether a real device, or a molecular device in the form of a drug)
•Procedural medicine are not regulated
• Many reasons: complexity, difficulty in standardizing, difficulty of enforcement (“surgeons know best” attitude)
•Self-regulation
Erroneous literature
RCTs often too late
“Tipping Point”
EVAR-1, DREAM OVER
Social responsibility
•It is our responsibility in academic medicine, to shoulder the responsibility that, in other fields of medicine, has been assumed by the FDA
•To ensure that only good treatment modalities are applied to patients
Biggest barrier to good research?
•Not having a correctly constructed hypothesis
•Incorrect design
•Don’t know how to get data
•Fear of statistics
Typical questions
•Components
• What/why/when/how• Verb• Condition
•“Why is the sky blue?”
•“What is the typical presentation of appendicitis?”
•Open-ended
Open-ended questions
•Descriptive analysis
•Observational study = no comparison = no statistical test
•Only one denominator
• May have more than one numerator, generating more than one ratio
• All ratios are calculated with the same denominator
43%
57%
Descriptive statistics
P value not applicable to compare different parts of the same population
Value and pitfall
•To explore the unknown
• When you know nothing, the first step is to explore and document the numbers
•Risk of over-generalizing
45%
55%
43%
57%
Inferential statistics
P value applicable for comparing parts of two populations
What is a hypothesis?
•Question ≠ hypothesis
•Questions: usually open-ended
•Hypothesis: usually is closed-ended, asking for a yes/no answer
• Statistical testing can only give yes/no answers
The process – study design
Study design phase Data preparation Analysis phase
Question development Select database Univariate
Define population Link database Bivariate
Define subset Select data elements Multivariable
Define outcome Generate new data elements Sensitivity
Define primary comparison Subset analysis
Define covariates
Steps in constructing a hypothesis
•Specify the outcomes (O in PICO)
• Common oversight: Often focus on the P, but vague about O (a typical question, “What is the outcome (?) of xyz patients?”)
•Specify the comparisons (C in PICO)
• Not done in open-ended questions
•Specify covariates (control variables, adjustment)
Hypothesis statement
•y = b1X1 + b2X2 + b3X3
•Death = age + race + gender + insurance…
Inclusion/exclusion criteria
•Just like a clinical trials (“eligibility criteria”)
•Diagnosis and/or procedure codes?
•Common mistake
45%
55%
43%
57%
Comparison
Outcome
•Mortality?
• Rare
•Complications
•Length of stay
•Charges
•Be judicious
Covariates / independent variables
•Patient demographcis
•Patient comorbidity
•Surgeon volume
•Hospital volume
•Hospital type (teaching vs non-teaching)
•Area (rural vs urban)
Hierarchy of influence on surgical outcomes
Technique and Management
Patient
Surgeon
Hospital
Region
Nation
Outcomes research
Clinical trials
The process – data preparation
Study design phase Data preparation Analysis phase
Question development Select database Univariate
Define population Link database Bivariate
Define subset Select data elements Multivariable
Define outcome Generate new data elements Sensitivity
Define primary comparison Subset analysis
Define covariates
Overview of public and semi-public databases
Multi-specialty
•Administrative Databases
• Nationwide Inpatient Sample (NIS)
• Medicare, Medicaid• California OSHPD
•Clinical Databases
• National Surgical Quality Improvement Program (NSQIP)
Specialty-specific
•Trauma
• National Trauma Databank (NTDB) •O
ncology• Surveillance, Epidemiology, and
End Results (SEER)• National Cancer Databank (NCDB)
•Transplant
• United Network for Organ Sharing (UNOS)
Administrative databases
Advantages
•Large patient numbers
•Less selection bias
•Can be linked to other databases containing other non-medical information
Disadvantages
•Limited clinical course information
•Limited surgical procedure information
NSQIP/non-NSQIP in-hospital mortality
Select data elements
Generate new data elements
•Most time consuming step of outcomes analysis
•Not every component of your research question is readily available in the database
• For example, comorbidity• Charlson Index, Elixhauser Index
•Some common concepts actually undefined
• Readmission?
What is a “re-admission”?
•Not all “admissions” are “re-admissions”
•30-day?
•Elective?
•Transfers?
•Diagnosis-specific?
•Preventable?
The process – analysis
Study design phase Data preparation Analysis phase
Question development Select database Univariate
Define population Link database Bivariate
Define subset Select data elements Multivariable
Define outcome Generate new data elements Sensitivity
Define primary comparison Subset analysis
Define covariates
Hypothesis statement
•y = b1X1 + b2X2 + b3X3
•Death = age + race + gender + insurance…
Table 1: Descriptive analysis
Table 2: Bi-variate analysis(unadjusted comparison)
Table 3: Multivariable analysis(adjusted analysis)
Analysis for Table 1
43%
57%
Analysis for Table 1
P value not applicable to compare different parts of the same population
Analysis for Table 1
•% for categorical data
•Mean/median/SD for continuous data
•For exploratory studies, descriptive studies, case series, etc., this would be the end of the process
•Reminder, avoid overgeneralizing
Analysis for Table 2
Analysis for Table 2
•Think about data types…
• Continuous data• Categorical data• (Ordinal data)
Analysis for Table 2
•Two questions to think about when picking a stats test…
• What is my outcome/dependent variable? What is my independent/input variable?
• What type of data do I have for each?• 4 possible combinations:
• 2 variables• 2 data types
X = inputY = outcomeCat.
Cat.
Cont.
Cont.
T-test
Rank sum
ROC2
Correlation
Analysis for Table 2
Analysis for Table 3
X = inputY = outcomeCat.
Cat.
Cont.
Cont.
Logistic regression
Linear regression
T-test
Rank sum
ROC2
Correlation
Analysis for table 3
Subset analysis
•Consistency of findings
•Generalizability
Generalizability
“This is not research anymore”
“That guy”