DR TAMSIN NEWLOVE-DELGADOACADEMIC REGISTRAR IN PUBLIC HEALTH
STUDY DESIGNS:case control, cohort and
qualitative
Aims and objectives
To outline and revise: Aetiology and causation Case control study design, advantages and disadvantages The odds ratio Cohort study design, advantages and disadvantages Relative risk
To summarize some key points about qualitative study design: Use Methods Quality in qualitative studies
Relevant Paper 3 Syllabus
3.1.10. Knows the benefits and weaknesses of different quantitative study designs to address different clinical questions: Case-control Cohort
3.6 Critically appraises cohort and case control studies
Relevant Paper 3 Syllabus
3.4. Qualitative Methods Knows when to apply qualitative research methodologies Additional approaches to sampling in qualitative studies Different approaches to data gathering in qualitative
studies The role of qualitative methodologies in instrument (i.e.
screening, diagnostic, outcome measure) development Methods for validating qualitative data Methods for minimising bias Methods of analyzing data Data saturation
3.6 – Critically appraises qualitative research
Plan of afternoon
1pm-2.30pm – Case control and cohort studies: including coffee break and exam questions
2.30pm – 3.30pm approx – Qualitative studies
The Hierarchy of evidence
RCT: not always the answer1
UnnecessaryImpractical/UnethicalInappropriate
Prognosis Diagnosis Quality issues And more
Study designs2
Objective Common design
Prevalence Cross-sectional
Incidence Cohort
Cause Cohort, case-control, cross-sectional (in order of reliability)
Prognosis Cohort
Treatment effect Controlled trial
Issues of how, why etc as opposed to what or how much
Qualitative design
Investigating aetiology
Epidemiological studies of aetiology are usually observational not experimental
An observed association may be due to: True cause Reverse causation Chance (random error) Bias (systematic error) Confounding
Happy with these concepts?
Investigating aetiology
Questions of causation
The Bradford-Hill criteria (J Roy Soc Med 1965:58:295-300)
1. Strength of the association.2. Consistency of findings.3. Specificity of the association.4. Temporal sequence of association.5. Biological gradient.6. Biological plausibility.7. Coherence.8. Experiment.
Can you think of examples where this doesn’t work?
The Case Control Study: Design
The case control study: design3
Advantages
Efficient for studies of rare diseases and diseases with long latent periods
Cheap, simple, quick (in comparison to cohorts)
Can examine multiple exposures – generate hypotheses
Sometimes the only practical option (e.g. where long latent period between exposure and disease)
Disadvantages
There are many!Can study only one outcomeNotorious for being prone to bias:
Sampling/selection bias – selection of cases and controls
Observation and recall bias
Not good for rare exposuresThe temporal sequence between exposure
and disease may be difficult to determine.As with all studies, confounding
Selecting cases
Need a clear case definition and sourceIncident or prevalent cases? Cases selected for a study should be
representative of all cases of the disease in the population.
Should be a random sample of all patients with the disease
This is difficult!: many cases not diagnosed or misdiagnosed
A hospital sample in some diseases may be very different from a community sample
Selecting controls
Controls are used to estimate the prevalence of exposure in the population which gave rise to the cases.
The ideal control group would comprise a random sample from the general population that gave rise to the cases.
Controls should meet all the criteria for cases, apart from having the disease itself; but they should have the potential to develop it
Recruiting more than one control per case may improve the statistical power of the study (up to 4 controls per case)
Selecting controls
Convenience sampleMatched sampleUsing two or more control groupsUsing population base sample e.g. from
registers
Selecting controls: matching
Matching – Some studies are matched to select cases/controls who are as similar as possible e.g on age, ethnicity etc Difference between cases and controls therefore cannot be a
result of differences in the matching variables – for example, to take age into account
Can be useful in small samples – as we might not have sufficient subjects to adjust for several variables at once.
Difficult/complicated to match on too many factors.... In a large study with many variables it is easier to take an unmatched control group and adjust in the analysis for the variables on which we would have matched, using ordinary regression methods.
Important not to match on basis of risk factor of interest / too many factors – ‘overmatching’ may make the controls unrepresentative and underestimate the true difference
Matching means effect on disease of matched variables cannot be studied
Example of selection bias
• Bias introduced through poor selection of controls
• Case control studies of NSAIDS (exposure) in colorectal cancer3
Case control studies in psychiatry
Suicide a popular subject……. Barraclough, B., Bunch, J., Nelson, B., et al
(1974) A hundred cases of suicide: clinical aspects. British Journal of Psychiatry, 125, 355-373.
More modern examples: Fuller Torrey E, Rawlings R, Yolken RH. The
antecedents ofpsychoses: a case-control study of selected risk factors.
Schizophr Res2000; 46: 17–23.
Case control studies and the odds ratio
Estimates the strength of association between an exposure and an outcome
Does not calculate relative risk as retrospective
Does not give incidence/prevalence – unless all cases in a population are included
The odds ratio is a measure of the odds of exposure in the cases, compared to the odds of exposure in the control group.
OR: 2 by 2 table
Cases Controls Total
Exposed a b A + b
Unexposed c d C + d
Total A + c B +d
OR = (a/c)/(b/d)
Imaginary worked example – Cats and schizophrenia
Schizophrenia Controls
Owned cat as child 80 100
Did not own cat as child
20 300
Total 100 400
Imaginary example– are cats associated with schizophrenia?
Odds of exposure in the cases: 80/20 = 4Odds of exposure in the controls: 100/300 = 0.33Odds ratio: 4/0.33 = 12.12So……the odds of having had a cat as a child in the
group with schizophrenia were over 12 times the odds of having had a cat as a child in the control group –
Or those with schizophrenia were over 12 times more likely to have had a cat as a child….
Why might we get this result?
Feline bias
Selection bias Cases recruited through a charity that runs ‘pet
experiences’ for people with mental illness Controls were a hospital sample recruited from an
allergy clinic Both of these would spuriously increase estimate of
effect
Recall bias Are those with schizophrenia more likely to
remember/report having had a cat – particularly if aware of hypothesis in question
Cohort studies
Cohort study design
Usually prospective; but can be retrospective
A prospective cohort
Prospective and retrospective cohort
Cohort studies may be prospective or retrospective, but both types define the cohort on the basis of exposure, not outcome.
Prospective cohort studies – participants are identified and followed up over time until the outcome of interest has occurred, or the time limit for the study has been reached. A temporal relationship between exposure and outcome can be established.
Retrospective cohort studies – exposure and outcome have already occurred at the start of the study. Pre-existing data, such as medical notes, can be used to assess any causal links, so lengthy follow-up is not required.
Advantages
Can investigate risk factors impossible to study in controlled trials - e.g. smoking or asbestos
Describe incidence and natural historyMultiple outcomes can be measured for any one
exposure.Exposure is measured before the onset of disease
(in prospective cohort studies).Good for measuring rare exposures, for example
among different occupations.Demonstrate direction of causality.Can calculate relative risk
Disadvantages
Expensive, time consumingLoss to follow up can introduce biasNeed a large sample size – especially for less
common outcomesNot good for rare outcomes or long latency
periodsNeed to maintain consistency of follow up
over timeSystematic misclassification of exposure or
outcome status – information bias
Sources of bias in cohort studies
Differential misclassification: can lead to an over- or underestimate of the effect between exposure and outcome.
Losses to follow up : degree to which losses to follow up are related to either exposure or outcome can lead to serious bias in the measurement of effect of exposure and outcome.2
Cohort studies in psychiatry: example
Andreasson et al: cannabis consumption and development of schizophrenia in a cohort of 45,570 Swedish conscripts4.
Relative risk in cohort studies
Analysis Riskexp = a / (a+c) (divide by total exposed)
Riskunexp = b / (b+d) (divide by total unexposed)
Estimate relative risk = Riskexp / Riskunexp
Indicates increased/decreased risk of disease assoc with exp: RR = 1 – risk is same in exposed and unexposed groups RR > 1 – risk is greater in exposed group RR < 1 – reduction in risk in exposed group Exposed to factor:
Yes No Total
Disease of interest:
Yes a b a+b
No c d c+d
Total a+c b+d N = a+b+c+d
Example: relative risk from Swedish conscript cohort study4
Cannabis exposure
None Low Medium High
Schizophrenia
197 18 10 21
No schizophrenia
41083 2818 692 731
Total 41280 2836 702 752
Coffee and exam questions
Qualitative studies
Answers questions such as: What is X & how does X vary in diff circumstances & why? Not ‘how big is X or how many X’s are there?
Concerned with meanings people attach to their experience of social world & how make sense of world
Uses of qualitative research
Preliminary to quantitative research Helps ensure validity of data obtained E.g. interviews to inform a survey
To validate quantitative research or provide a diff perspective on same social phenomena
Used independently to uncover social processes or access areas of social life not amenable to quantitative research
Address the 'gap' between evidence-based approaches based on the findings of randomised control trials and the practice of clinical decision-making in individual cases
Qualitative vs. quantitative
Quantitative Qualitative
Type of reasoning Deduction Induction
Objectivity Subjectivity
Causation Meaning
Type of question Pre-specified Open-ended
Outcome orientated Process orientated
Type of analysis Numerical estimation Narrative description
Qualitative research methods: data gathering
InterviewsFocus groupsObservational/ethnographic
Interviews
StructuredSemi-structuredDepth interviews
Consider: recordingReflexivity
Note on Reflexivity5
The researcher is not a neutral/mechanical tool The researcher is not doing an experiment in which she/he
sets the agenda The person/people the researcher talks to are not inanimate
objects, they also have agency and may try to set the agenda themselves
All social research, especially qualitative social research, hinges on social relationships:
They are affected by interpersonal dynamics and The researcher AND researched 'co-produce' social
encounters. Reflexivity is reflecting, or thinking critically, carefully,
honestly and openly, about the research experience and process.
Focus groups
Strengths Help to identify group norms/cultural values Group processes can help people to explore and clarify
their views in ways that may be less easily accessible in interview
Can encourage participation from those reluctant to be interviewed
Can encourage contributions from people who feel have nothing to say
Weaknesses Not easy option - data generated can be complex Potential issues with confidentiality, or with ‘sensitive’
topics
Observational/ethnographic
Instead of asking questions about behaviour – the researcher systematically watches people and events to observe everyday behaviours and relationships
Aspires to be ‘naturalistic’ in that people are studied in situ with as little interference by the researcher as is feasible and ethical
Covert or overtParticipant or non-participant
Observational/ethnographic
Choice of setting is purposive Consider characteristics of researcher, group and setting
Male, female Young, old Naïve or experienced Accepted by group but don’t ‘go native’!
Ethical issues Covert research roles must be justified
Recording observational data Relies on researcher acting as research instrument and documenting
the world they observe Good memory Clear and detailed recording Jotted notes Sift, decode and make sense of data to make meaningful
Sampling in qualitative research
Often a smaller sample size Rich in detail Phenomenon only needs to appear once Not describing incidence/prevalence or statistical
significance
Quantitative research uses probability samplingQualitative research uses non-probability sampling
not representative samples findings cannot be generalised to the whole study
population from which the sample was taken. the people in the study population do not each have an
equal chance of being selected.
Sampling in qualitative research
Purposive sampling individual participants are selected deliberately for their specific
characteristics that are of importance to the study Quota based sampling
A quota is a defined number that must be included in a sample :ensures that a certain number of subjects from different subgroups with specific characteristics appear in the sample, so that all these characteristics are represented.
Snowball sampling Useful for hard-to-reach groups/populations Start with one or two contacts, ask for
referrals/recommendations etc
Theoretical sampling : sampling related to previously developed hypotheses or theories
Analysing qualitative data (1)
Data preparation Nature and scale of qualitative data Transcription Notes made during observation have to be turned into
detailed descriptive accounts Relationship between data and analysis
Transcripts provide descriptive record – not explanations Analytical process begins during data collection as data
already gathered are analysed and feed into/shape ongoing data collection
QUANT QUAL
CollectAnalyse
CollectAnalyse
Analysing qualitative data (2)
Goal To develop analytic categories to describe & explain social
phenomena May be derived inductively (from the data) or deductively
(predefined themes drawn from schedule & research Qs) 3 broad approaches
Thematic analysis Grounded theory Framework approach
Initial steps Manage & make sense of data Reading & re-reading to identify initial set of themes Coding, label themes, use their language Organising, grouping & refining of themes/categories
Software packages CAQDAS
Thematic analysis
Simplest formMost commonly used Groups data into themesCan simply describe &/or identify
relationships between themes Often includes themes that are anticipated
(literature review) or those that emerge from the data
Grounded theory
Glaser & Strauss coined term Process of coding & identifying categories as they ‘emerge’
from data Iterative process
Modified grounded theory (see Charmaz 2007)Theoretical samplingSaturation - CodingConstant comparison Use of memos
Framework approach
Developed by National Centre for Social ResearchMore deductiveSuited to applied or policy researchStarts from aims & objectives already set for study More structured topic guideAnalytical process similar to thematic but more explicit
& more strongly informed by prior knowledge5 stages:
Familiarisation Identifying a thematic framework Indexing Charting Mapping and interpretation
Ensuring quality
Improving validity (‘closeness to truth’) Triangulation: use of three or more different research
methods in combination; principally used as a check of validity
Respondent validation Checking of transcripts/tapes by independent analyst Reflexivity Attention to negative cases (‘deviant case analysis’ where
researcher's explanatory scheme appears weak or is contradicted by the evidence)
Clear exposition of methods of data collection & analysis
Critical appraisal checklists – CASP
IN SUMMARY2
Objective Common design
Prevalence Cross-sectional
Incidence Cohort
Cause Cohort, case-control, cross-sectional (in order of reliability)
Prognosis Cohort
Treatment effect Controlled trial
Issues of how, why etc as opposed to what or how much
Qualitative design
References
1. Greenhalgh T. How to Read a Paper2. Mann CJ. Observational Research methods:
Research design 2: cohort, cross-sectional and case-control studies. Emerg Med 2003 20: 54-60
3. Schulz KF and Grimes DA. Case Control Studies: research in reverse. The Lancet doi:10.1016/S0140-6736(02)07605-5
4. Andreasson S., Engstrom A., Allebeck P., Rydberg U. Cannabis and schizophrenia. A longitudinal study of Swedish conscripts (1987) Lancet, 2 (8574), pp. 1483-1485+1486.
5. Research Consortium on Educational Outcomes and Poverty. http://manual.recoup.educ.cam.ac.uk/wiki/index.php/Main_Page