Post on 16-Jul-2015
transcript
Yes - 2 studies No - 6 studies
Study 1 Study 3Study 2 Study 4
Study 5
Study 6
Study 7
Study 8
6 studies indicate ‘no’ so should we conclude there’s no abnormal cytokine profile in autism?
Yes No
Study 1 (n = 200; clinical diagnosis)
Study 3 (n = 10; self-report)
Study 2 (n = 100; clinical diagnosis
Study 4 (n = 8; self-report)
Study 5 (n = 13; self-report)
Study 6 (n = 5; self-report)Study 7 (n = 15; self-report)
Study 8 (n = 17; self-report)
Big differences in study quality but are the 2 ‘yes’ studies worth more than then 6 ‘no’ studies?
Meta-analysis is an objective and transparent technique to synthesise data from a number of related studies.
It’s very easy for others to ‘game’ a meta-analysis to get the outcome they want - watch out for this.
9 Circles of scientific hell
‘Sins’ that can influence the data in your
meta-analysis
Sins that are often
overlooked in meta-analysis
1. Have a good research question
•Is there a debate in the literature? •Perhaps a research question is settled but you want to look
at a moderator
2. Pilot your search terms•Too broad and you’ll be swamped, too narrow and you’ll
miss papers •Use relevant databases (Pubmed + Embase will have you
covered) •Also a good ‘feasibility’ check
3. Document everything!•Can someone reading your paper recreate your analysis? •This makes your analysis transparent
3. Extract the data•Can help having an ‘data extraction’ form where you enter
important study details •Gold standard is having 2 people do this and a third
adjudicating any disagreement
There’s a few software packages you can use;
• Comprehensive meta-analysis (recommended)• R packages (tricky but more flexibility with figures)• An excel spreadsheet that comes with Cumming (2014)
You can extract almost any data to create a common effect size
• P-values and sample size• Means and SDs• Correlation coefficients (‘easiest’ meta-analysis)• Still not enough info? Contact the author!
•Most authors oblige (it’s a citation!)•Not likely they’ll have data if older than 10 years
The software/package will calculate common effect sizes (even if you’re extracting different types of data) and then calculate a summary effect size
Forest plot
sub-summary effect size (i.e., what the
overall impact of one cytokine?
Overall effect size (i.e., what’s the summary of
ALL studies?)
Publication bias?•Are there ‘missing’ studies? •A scatterplot of standard error against individual effect size •Large studies tend to have small SE (near top) •There should be an even spread (especially near the bottom)
Should be about 4 more studies here
What happens if there’s bias?•You can impute the missing studies and re-analyse •If your overall conclusions don’t change with the inclusion of the
studies you’re in the clear
Imputed studies
Here you can get some clues as to which factors are driving a result (i.e., is this due to one cytokine?)
Other common moderator analyses
• Gender - is this only found in one gender?• Age - is this stronger/weaker in older people?• Study quality - what’s the effect of ‘bad’ studies?• Different types of measures • Clinical groups - e.g., Bipolar vs. schizophrenia