How To Read A Medical Paper: Part 3, Assessing the Statistical Analysis

How To Read A Paper 1

How To Read A Paper IIIStatistics – again…

Dr Luke KaneMay 2014


References

• Presentation based on T Greenhalgh’s series “How to read a paper” from the BMJ

• Greenhalgh, T. (1997) “How to read a paper” British Medical Journal. Web, accessed April-May 2014 at <http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper>


What to Look For When Appraising the Statistics

• Checklist of questions to help you appraise the statistical validity of a paper

• Have the authors set the scene?– Have they determined whether their groups are

comparable? – And if necessary, adjusted for baseline differences?

• What sort of data have they got and have they used appropriate statistical tests?

• Have they used unusual tests? Why?• Is the data analysed according to the original protocol?


Have the Authors: “Set the scene”?

• I.e. have they laid out clearly the characteristics of the groups?– For example, ratio of M:F, socioeconomic status,

age etc– If there are differences then this can pose a

challenge when interpreting the results• Adjustments can be made to allow for these differences


Remember This?


Have They Used the Appropriate Statistical Tests?

• Categorical and continous variables• Parametric and non-parametric tests– Parametric assumes the data has a particular form

of distribution– Non-parametric make no assumption about the

form – Parametric tests are more powerful than non-

parametric tests– Always use a parametric test if you can!


Little Summary of Stats So Far…

The field of statistics exists because it is usually impossible to collect data from all individuals of interest (population). Our only solution is to collect data from a subset (sample) of the individuals of interest, but our real desire is to know the “truth” about the population. Quantities such as means, standard deviations and proportions are all important values and are called “parameters” when we are talking about a population. Since we usually cannot get data from the whole population, we cannot know the values of the parameters for that population. We can, however, calculate estimates of these quantities for our sample. When they are calculated from sample data, these quantities are called “statistics.” A statistic estimates a parameter.


Parametric & Non-Parametric

• Parametric statistical procedures rely on assumptions about the shape of the distribution (i.e. assume a normal distribution) and about the parameters (i.e. means and standard deviations) of the assumed distribution.

• Nonparametric statistical procedures rely on no or few assumptions about the shape or parameters of the population distribution from which the sample was drawn. – Hoskin T (2012)


Parametric & Nonparametric

• This helps you to decide which test to use…• Remember all of the parametric procedures

rely on an assumption of approximate normality


Analysis Type Example Parametric Procedure

Nonparametric Procedure

Compare means between two distinct/independent groups

Is mean BP for patients assigned to placebo different from the mean for patients in the treatment group?

Two-sample t-test Wilcoxon rank- sum test

Compare two quantitative measurements taken from the same individual

Was there a significant change in BP between start and the 6m follow-up measurement in treatment group?

Paired t-test Wilcoxon signed- rank test

Compare means between three or more distinct/independent groups

If our experiment had 3 groups (placebo, new drug #1, new drug #2), see if BP differs among the three groups?

Analysis of Variance(ANOVA)

Kruskal-Wallis Test

Estimate the degree of association between two quantitative variables

Is systolic blood pressure associated with patient’s age?

Pearson coefficient of correlation

Spearman’s rank correlation


Example

• Length of stay in hospital

• Compare M/F• Describe the

distribution• F: Right skew

– Mean 60– Median 30

• M: Normal– Mean 30– Median 30


Example Continued

• Assumption of normality not reasonable• Therefore cannot use a parametric test• BUT can usea Wilcoxon Rank Sum Test

Analysis Type Example Parametric Procedure

Nonparametric Procedure

Compare means between two distinct/independent groups

Is mean BP for patients assigned to placebo different from the mean for patients in the treatment group?

Two-sample t-test

Wilcoxon rank- sum test


Example Continued

• If we do a Wilcoxon Rank Sum Test– P-value is 0.63– Not less than 0.05 (which is the value that we typically compare

against)– Therefore not statistically significant– There is no difference in length of stay between genders

• If you assume the data is normally distributed and do a Two Sample T test – P-value is 0.04– Therefore statistically significant– There is a statistically significant difference between length of stay

for male and females


So why can’t we just use nonparametric tests?

• Nonparametric tests are good as they make fewer assumptions about the distribution of measurements in population

• BUT– They are less statistically powerful• They need a larger sample size

– Their results are often less easy to interpet


Small Sample Sizes

• More likely to have variable data• I.e. not normally distributed• So often need to use a non-parametric test


Transformation

• Gives the graph a normal shape– Mathematical transformation– Such as using log, square root (√x), reciprocal (1/x)

• Some data cannot be transformed


Obscure Tests

• Obscure means unclear, uncertain, hazy, vague• There are 1000s of statistical tests– There are about 12 important ones– Most of the rest should only be used in certain, specific

situations• If you are reading a paper with a standard set of data

which was collected in a standard way but the test they use is very obscure then you need to know WHY!?

• They should explain why they chose it and describe it with references


Is the Data Analysed According to the Original Protocol?

• Often trials are stopped before they are finished if the new drug is doing well– This is to make sure that negative results don’t

come up later– This is very bad science– You should try to work out if they have done this

but it is very hard!– You can look at a registry to compare what they

said they would do and what they did do


Paired Tests

• If you measure something twice on each subject• Each measurement if paired with another

measurement– E.g. blood pressure lying and standing– Height of a tree before and after fertiliser– IQ of younger and older brothers

• You would assume that the paired values are closer together than random values– Unless you allow for this then the results will be

significantly biased


Summary

• Setting the scene– Group characteristics

• Think about the data– Continuous vs categorical– Distribution

• Parametric or non-parametric tests• Appropriate tests• Obscure tests• Transformation• Paired Tests• Protocol


References• Bowers, D. (2008) Medical Statistics from Scratch: An Introduction for

Health Professionals. USA: Wiley-Interscience.• Grant, A. (2014) “Epidemiology for tropical doctors”. Lecture (S6) from

the Diploma of Tropical Medicine & Hygiene, London School of Hygiene & Tropical Medicine.

• Greenhalgh, T. (1997) “How to read a paper” British Medical Journal. Web, accessed April-May 2014 at <http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper>

• Hoskin, T (2012) Parametric and non-parametric: Demystifying the Terms. Retrieved from http://www.mayo.edu/mayo-edu-docs/center-for-translational-science-activities-documents/berd-5-6.pdf

• http://www.gla.ac.uk/sums/users/jdbmcdonald/PrePost_TTest/instruct1.html

http://www.mayo.edu/mayo-edu-docs/center-for-translational-science-activities-documents/berd-5-6.pdf




Date post:	18-Jun-2015
Category:	Health & Medicine
Upload:	drlukekane
View:	177 times
Download:	0 times

How To Read A Medical Paper: Part 3, Assessing the Statistical Analysis

Health & Medicine