20
Meta-regression with DisMod-MR: how robust is the model? June 18, 2013 Hannah M Peterson Post-Bachelor Fellow
| Date post: | 27-Jan-2015 |
| Category: |
Technology |
| Upload: | institute-for-health-metrics-and-evaluation-university-of-washington |
| View: | 119 times |
| Download: | 2 times |
Meta-regression with DisMod-MR: how robust is the model?
June 18, 2013
Hannah M Peterson
Post-Bachelor Fellow
2
Global Burden of Disease Study 2010
3
YLDs• Measures morbidity
• Requires age-specific prevalenceo For 291 outcomes
o For 2 sexes
o For 187 countries
o For 3 years
4
Is negative-binomial distribution the best choice?
DisMod-MR
5
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
6
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
7
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
8
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
9
Potential experimental frameworks
• Data collectiono Ideal
o Impractical
• Simulationo Impossible to know true data distribution
• Out-of-sample cross validationo Do not have to choose distribution
10
Out-of-sample cross validation
11
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
12
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
13
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
14
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
• Use fit to calculate statistics for test data
15
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
• Use fit to calculate statistics for test data
• For each distribution
• For 1000 test-train splits
• For each disease data set
16
Comparing distributions
How to determine the best distribution?
17
Metrics of evaluation
• Biaso Measures the average difference between observation and estimate
• Median absolute error (MAE)o Measure of overall magnitude of error
• Percent coverage (PC)o Percent of time estimate uncertainty interval contains observation
18
Results
Percent of wins (%)
Distribution Bias MAE PC Total
Normal 22.1 20.6 34.6 25.7
Lognormal 29.7 13.0 36.5 26.4
Binomial 26.3 48.3 1.9 25.5
Negative-binomial 21.9 18.1 27.1 22.4
19
Conclusions
• Choice of distribution doesn’t greatly influence results
• Best overall performance: lognormal distribution
o Contingent on method to adjust data whose value is 0
• Further investigate when each distribution performs best
o Dependent on number of covariates, priors, amount of data?
