Post on 17-Aug-2019
transcript
Relationship between diet and ankylosing spondylitis:
a systematic review
Research protocol
November 2015
Juliane Hinz,1,2
Hadeel M. Abbood1
Dr. Tatiana V. Macfarlane1
Dr. Ejaz Pathan3
Prof. G. J. Macfarlane4
Katy Gordon5
1 Aberdeen Dental School and Hospital, University of Aberdeen, AB25 2ZD
2 University of Bremen, Bibliothekstraße 1, 28359 Bremen
3 Aberdeen Royal Infirmary, Foresterhill, Aberdeen, AB25 2ZN
4 Epidemiology Group, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD
5 Pain Concern, 62-66 Newcraighall Road, Edinburgh, EH15 3HS
2
Table of contents
Introduction .............................................................................................................................. 3
Rationale: Why it is important ............................................................................................... 3
Aim ............................................................................................................................................ 3
Objectives.................................................................................................................................. 3
Participants ............................................................................................................................... 3
Outcomes .................................................................................................................................. 4
Method ...................................................................................................................................... 4
SEARCH STRATEGY ................................................................................................................................................... 4 INCLUSION AND EXCLUSION CRITERIA .......................................................................................................................... 4 DATA EXTRACTION .................................................................................................................................................. 4 QUALITY ASSESSMENT .............................................................................................................................................. 4 DISSEMINATION ...................................................................................................................................................... 5
Milestones ................................................................................................................................. 5
Funding ..................................................................................................................................... 5
References:................................................................................................................................ 6
Table of appendixes
Appendix 1 Map of searching terms in MEDLINE and Embase .............................................. 7
Appendix 2 Flow diagram of study selection process ............................................................... 8
Appendix 3 Data extraction form .............................................................................................. 9
Appendix 4 Methodology Checklist 1: Systematic Reviews and Meta-analyses .................... 10
Appendix 5 Methodology Checklist 2: Controlled Trials ........................................................ 12
Appendix 6 Methodology Checklist 3: Cohort studies ............................................................ 14 Appendix 7 Methodology Checklist 4: Case-Control studies.................................................. 18
3
Introduction
Ankylosing spondylitis (AS) is a chronic inflammatory rheumatic disease that can affect the
axial skeleton. AS is a prototype of a group of overlapping inflammatory diseases named
spondyloarthropathies (Khan 2009). It leads to new bone formation, such as ankyloses of the
vertebral column (Baraliakos et al. 2008). Symptoms include inflammatory back pain,
asymmetrical peripheral inflammation in several peripheral joints, enthesitis, and specific
organ involvement such as uveitis (Braun et al. 2007). Over a longer time-period, the disease
slowly effects in progressive limitation of spinal mobility and limited chest expansion (Khan
2009). If the disease is not treated, the inflammation of the spinal joints will slowly destroy
the cartilage and fibrous tissue of the surrounding structures as well as the ligaments and
replace them with bone (Weisman 2011).
One of the causes for AS is genetic as there is a strong association with the genetic marker
HLA-B27. However the biological relationship between the gene and AS have not yet been
satisfactory explained (Khan 2009).
Diet has significant effects on health in general and especially on vulnerability to chronic
diseases. Diet is related to obesity, heart diseases, hypertension, diabetes, osteoporosis,
cancer and oral diseases. For example a high intake of carbohydrates can cause periodontitis.
Periodontitis (PD) is a form of periodontal disease, which is defined as any disorder of the
tissue surrounding and supporting the teeth (Pihlstrom et al. 2005). Mostly, this disorder is
characterized by a gingival inflammation caused by pathogenic bacteria on the dental plaque
(Pihlstrom et al. 2005).
Rationale: Why it is important
Patients with AS reported worsening of disease activity with certain foods, but only few
studies showed an association between AS and diet (Sundström et al. 2011).
Aim
To assess the relationship between diet and AS.
Objectives
The systematic review has the following objectives:
a) To investigate whether there is any evidence that patients with AS report different diet
style to those without.
b) To find out whether diet may affect the severity of AS.
c) To investigate whether persons with particular diets are less likely to develop AS.
d) To investigate if dietary interventions improve symptoms of AS.
Participants
The studies will be restricted to human studies. There will be no restriction to gender,
socioeconomic class or ethnicity.
People under 16 years will be excluded.
4
Outcomes
Outcome measurements will be values of pain, severity of AS symptoms and the presence or
absence of AS.
Method
Search strategy
MEDLINE and Embase databases will be used for the electronic search for studies up to
2015.
The search strategy terms will be grouped into two categories. Group I terms will be related
to Ankylosing spondylitis (ankylosing spondylitis, Spondyloarthropath$, Spondylitis,
Spondylarthropath$, spondyloarthritis) and group II will include terms associated with diet
(nutrition, food, diet, Food habits, nutritional status, Vitamins, Antioxidants, fatty acids,
cholesterol, carbohydrates, dietary protein, calcium, fish oils, fruit, vegetables,
micronutrients). Intra-group terms will be combined with the Boolean commands ‘OR’, and
Inter-group terms will be combined with ‘AND’.
Two journals (Annals of the Rheumatic Diseases and Annual Review of Nutrition) will be
searched manually from 2010 to 2015.
Inclusion and exclusion criteria
All observational studies on humans will be included (Cross-sectional, Cohort, Case-control
and Case series studies). Randomized controlled trials will be included.
We will include studies in any language.
Data extraction
After completing the database search and manual search, two independent reviewers will
screen the title and the abstract of each study following the inclusion criteria. If disagreement
occurs between the two reviewers, a third reviewer will be consulted. Data extraction will be
done by two independent reviewers. They will extract the required data from each study.
These information include: study type, population description (sample size, mean age,
gender, method of recruitment), participation rate, definition of AS, type of dietary intake
assessment, values that describe the relationship between AS and diet (see appendix 3). The
data will be used to create tables. The tables will contain the key question of the study, type
of ankylosing spondylitis diagnosis, diet assessment values. The measurement of the
association between diet and ankylosing spondylitis will be quoted as reported by the authors:
OR, RR, p-value, confidence interval and other analysis that can measure the association.
It is planned to conduct a meta-analysis to pool the results, dependent on how big the
differences between the studies will be.
Quality assessment
Scottish Intercollegiate Guidelines Network (SIGN) methodology checklists1 will be
performed to assess the quality of individual studies (see appendix 4 to 7). The four
assessment tools will be used for systematic reviews, randomized controlled trials, cohort
1 Scottish Intercollegiate Guidelines Network (SIGN), http://www.sign.ac.uk/methodology/checklists.html
5
studies and case control studies. Two reviewers will conduct the quality assessment, if
disagreement occurs, a third reviewer will be consulted.
Dissemination
Study protocol will be registered with PROSPERO http://www.crd.york.ac.uk/PROSPERO/
We aim to publish this review in a peer reviewed journal.
We will follow Prisma guidelines for reporting of systematic reviews http://www.prisma-
statement.org/ .
Milestones
Protocol finalised - November 2015
Search strategy - November 2015
Study selection – November 2015
Data extraction – November 2015 – December 2015
Quality assessment – December 2015 – January 2016
Summary of results – January 2016
Writing final report – January 2016
Circulation of report for comments – February 2016
Submission for publication - March 2016
Funding
This work was supported by Erasmus, the National Ankylosing Spondylitis Society (NASS)
and the University of Aberdeen.
6
References:
Baraliakos, X., Listing, J., von der Recke, A., Braun, J. 2009, “The Natural Course of
Radiographic Progression in Ankylosing Spondylitis -- Evidence for Major Individual
Variations in a Large Proportion of Patients”, Journal of Rheumatology, vol. 36, no. 5, p.
997.
Braun, J. & Sieper, J. 2007, “Ankylosing Spondylitis”, Lancet, vol. 369, no. 9570, pp. 1379-
1390.
Khan, M. 2009, “Ankylosing Spondylitis” in Oxford University Press, pp. 1-21
Pihlstrom, B. L., Michalowicz, B. S., Johnson, N. W. 2005, “Periodontal diseases”, Lancet,
vol. 366, no. 9499, pp. 1809–1820.
Roberts, K. & Flaherty, SJ. 2010 “Review of dietary assessment methods in public health.” In
Oxford: National Obesity Observational 2010, p. 3.
Sundström, B., Wållberg-Jonsson, S. & Johansson, G. 2011 “On diet in ankylosing
spondylitis”, Clinical Rheumatology, vol. 30, no. 1, pp. 71-76.
Weisman, M. 2011 “Ankylosing Spondylitis” in Oxford University Press, pp. 3-6.
7
Appendix 1 Map of searching terms in MEDLINE and Embase
AND
All the groups results will be combined with AND
Group I
terms regarding ankylosing spondylitis
Group II
terms regarding diet
8
Appendix 2 Flow diagram of study selection process
Records that will be identified through
database searching (n=?)
Scr
een
ing
Incl
ud
ed
Eli
gib
ilit
y
Iden
tifi
cati
on
Additional records that will be identified through hand-searching
journals (n = ?)
Records screening (n = ?)
Records after screening (n = ?)
Excluding records due to language or validity (n
= ?)
Full-text articles assessment for eligibility
(n = ?)
Full-text articles excluded (n = ?)
- No AS - No diet - Neither AS nor diet - Not looking at
relationship between AS and diet
Studies that will be included in qualitative synthesis
(n =?)
Removal of Duplication (n = ?)
9
Appendix 3 Data extraction form
ID: Researcher:
Title: First author’s surename:
Year of publication Country of study:
Type of study
Participants Participation rate Number: Male Female Age range:
AS diagnosis criteria
Diet assessment type
Research Question:
☐ Is there any evidence that patients with AS report different diet style to those without?
☐ May diet affect the severity of AS?
☐ Are persons with particular diets less likely to develop AS?
☐ May dietary interventions improve symptoms of AS?
Type of AS diagnosis Diet Outcome P Value Effect
measure
10
Appendix 4
S I G N Methodology Checklist 1: Systematic Reviews and Meta-analyses
SIGN gratefully acknowledges the permission received from the authors of the AMSTAR tool to base this
checklist on their work: Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C,. et al.
Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.
BMC Medical Research Methodology 2007, 7:10 doi:10.1186/1471-2288-7-10. Available from
http://www.biomedcentral.com/1471-2288/7/10 [cited 10 Sep 2012]
Study identification (Include author, title, year of publication, journal title, pages)
Guideline topic: Key Question No:
Before completing this checklist, consider:
Is the paper relevant to key question? Analyse using PICO (Patient or Population
Intervention Comparison Outcome). IF NO reject. IF YES complete the checklist.
Checklist completed by:
Section 1: Internal validity
In a well conducted systematic review: Does this study do it?
1.1 The research question is clearly defined and the
inclusion/ exclusion criteria must be listed in the
paper.
Yes □
If no reject
No □
1.2 A comprehensive literature search is carried out.
Yes □
Not applicable
□
If no reject
No □
1.3 At least two people should have selected studies.
Yes □
No □
Can’t say □
1.4 At least two people should have extracted data. Yes □ No □
Can’t say □
1.5 The status of publication was not used as an
inclusion criterion. Yes □ No □
1.6 The excluded studies are listed. Yes □
No □
11
1.7 The relevant characteristics of the included
studies are provided.
Yes □
No □
1.8 The scientific quality of the included studies was
assessed and reported. Yes □ No □
1.9 Was the scientific quality of the included studies
used appropriately? Yes □ No □
1.10 Appropriate methods are used to combine the
individual study findings. Yes □
Can’t say □
No □
Not applicable
□
1.11 The likelihood of publication bias was assessed
appropriately.
Yes □
Not applicable
□
No □
1.12 Conflicts of interest are declared.
Yes □ No □
SECTION 2: OVERALL ASSESSMENT OF THE STUDY
2.1 What is your overall assessment of the
methodological quality of this review? High quality (++) □
Acceptable (+) □
Low quality (-)□
Unacceptable – reject 0 □
2.2 Are the results of this study directly applicable to
the patient group targeted by this guideline? Yes □ No □
2.3 Notes:
12
Appendix 5
S I G N Methodology Checklist 2: Controlled Trials
Study identification (Include author, title, year of publication, journal title, pages)
Guideline topic: Key Question No: Reviewer:
Before completing this checklist, consider:
1. Is the paper a randomised controlled trial or a controlled clinical trial? If in doubt, check the study design algorithm available from SIGN and make sure you have the correct checklist. If it is a controlled clinical trial questions 1.2, 1.3, and 1.4 are not relevant, and the study cannot be rated higher than 1+
2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist.
Reason for rejection: 1. Paper not relevant to key question 2. Other reason (please specify):
Section 1: Internal validity
In a well conducted RCT study… Does this study do it?
1.1 The study addresses an appropriate and clearly focused question.
Yes
Can’t say
No
1.2 The assignment of subjects to treatment groups is randomised. Yes
Can’t say
No
1.3 An adequate concealment method is used.
Yes
Can’t say
No
1.4 The design keeps subjects and investigators ‘blind’ about treatment allocation.
Yes
Can’t say
No
1.5 The treatment and control groups are similar at the start of the trial. Yes
Can’t say □
No
1.6 The only difference between groups is the treatment under investigation.
Yes
Can’t say
No
1.7 All relevant outcomes are measured in a standard, valid and reliable way.
Yes
Can’t say
No
1.8 What percentage of the individuals or clusters recruited into each treatment arm of the study dropped out before the study was completed?
1.9 All the subjects are analysed in the groups to which they were randomly allocated (often referred to as intention to treat analysis).
Yes
Can’t say
No
Does not apply
13
1.10 Where the study is carried out at more than one site, results are
comparable for all sites.
Yes
Can’t say
No
Does not apply
SECTION 2: OVERALL ASSESSMENT OF THE STUDY
2.1 How well was the study done to minimise bias?
Code as follows:
High quality (++)
Acceptable (+)
Low quality (-)
Unacceptable – reject 0
2.2 Taking into account clinical considerations, your evaluation of the methodology used, and the statistical power of the study, are you certain that the overall effect is due to the study intervention?
2.3 Are the results of this study directly applicable to the patient group targeted by this guideline?
2.4 Notes. Summarise the authors’ conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question and mention any areas of uncertainty raised above.
14
Appendix 6
S I G N Methodology Checklist 3: Cohort studies
Study identification (Include author, title, year of publication, journal title, pages)
Guideline topic: Key Question No: Reviewer:
Before completing this checklist, consider:
1. Is the paper really a cohort study? If in doubt, check the study design algorithm available from SIGN and make sure you have the correct checklist.
2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison
Outcome). IF NO REJECT (give reason below). IF YES complete the checklist..
Reason for rejection: 1. Paper not relevant to key question □ 2. Other reason □ (please specify):
Please note that a retrospective study (ie a database or chart study) cannot be rated higher than +.
Section 1: Internal validity
In a well conducted cohort study: Does this study do it?
1.1 The study addresses an appropriate and clearly focused question.i Yes □
Can’t say □
No □
SELECTION OF SUBJECTS
1.2 The two groups being studied are selected from source populations that are
comparable in all respects other than the factor under investigation.ii Yes □
Can’t say □
No □
Does not apply □
1.3 The study indicates how many of the people asked to take part did so, in each of the
groups being studied.iii
Yes □
No □
Does not apply □
1.4 The likelihood that some eligible subjects might have the outcome at the time of
enrolment is assessed and taken into account in the analysis.iv Yes □
Can’t say □
No □
Does not apply □
1.5 What percentage of individuals or clusters recruited into each arm of the study
dropped out before the study was completed.v
1.6 Comparison is made between full participants and those lost to follow up, by exposure
status.vi Yes □
Can’t say □
No □
Does not apply □
ASSESSMENT
1.7 The outcomes are clearly defined.vii Yes □
Can’t say □
No □
15
1.8 The assessment of outcome is made blind to exposure status. If the study is
retrospective this may not be applicable.viii Yes □
Can’t say □
No □
Does not apply □
1.9 Where blinding was not possible, there is some recognition that knowledge of
exposure status could have influenced the assessment of outcome.ix Yes □
Can’t say □
No □
□
1.10 The method of assessment of exposure is reliable.x Yes □
Can’t say □
No □
1.11 Evidence from other sources is used to demonstrate that the method of outcome
assessment is valid and reliable.xi Yes □
Can’t say □
No □
Does not apply□
1.12 Exposure level or prognostic factor is assessed more than once.xii Yes □
Can’t say □
No □
Does not apply □
CONFOUNDING
1.13 The main potential confounders are identified and taken into account in the design
and analysis.xiii Yes □
Can’t say □
No □
STATISTICAL ANALYSIS
1.14 Have confidence intervals been provided?xiv Yes □ No □
SECTION 2: OVERALL ASSESSMENT OF THE STUDY
2.1 How well was the study done to minimise the risk of bias or confounding?xv
High quality (++) □
Acceptable (+) □
Unacceptable – reject 0
2.2 Taking into account clinical considerations, your evaluation of the methodology used, and the statistical power of the study, do you think there is clear evidence of an association between exposure and outcome?
Yes
Can’t say
No
2.3 Are the results of this study directly applicable to the patient group targeted in this guideline?
Yes □ No □
2.4 Notes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question and mention any areas of uncertainty raised above.
i Unless a clear and well defined question is specified in the report of the review, it will be difficult to assess how well it has met its objectives or how relevant it is to the question you are trying to answer on the basis of the conclusions.
ii This relates to selection bias.* It is important that the two groups selected for comparison are as similar as possible in all characteristics except for their exposure status, or the presence of specific prognostic factors or prognostic markers relevant to the study in question.
16
iii This relates to selection bias.* The participation rate is defined as the number of study participants divided by the number of eligible subjects, and should be calculated separately for each branch of the study. A large difference in participation rate between the two arms of the study indicates that a significant degree of selection bias* may be present, and the study results should be treated with considerable caution.
iv If some of the eligible subjects, particularly those in the unexposed group, already have the outcome at the start of the trial the final result will be subject to performance bias.* A well conducted study will attempt to estimate the likelihood of this occurring, and take it into account in the analysis through the use of sensitivity studies or other methods.
v This question relates to the risk of attrition bias.*The number of patients that drop out of a study should give concern if the number is very high. Conventionally, a 20% drop out rate is regarded as acceptable, but in observational studies conducted over a lengthy period of time a higher drop out rate is to be expected. A decision on whether to downgrade or reject a study because of a high drop out rate is a matter of judgement based on the reasons why people dropped out, and whether drop out rates were comparable in the exposed and unexposed groups. Reporting of efforts to follow up participants that dropped out may be regarded as an indicator of a well conducted study.
vi For valid study results, it is essential that the study participants are truly representative of the source population. It is always possible that participants who dropped out of the study will differ in some significant way from those who remained part of the study throughout. A well conducted study will attempt to identify any such differences between full and partial participants in both the exposed and unexposed groups. This relates to the risk of attrition bias.* Any unexplained differences should lead to the study results being treated with caution.
vii This relates to the risk of detection bias.* Once enrolled in the study, participants should be followed until specified end points or outcomes are reached. In a study of the effect of exercise on the death rates from heart disease in middle aged men, for example, participants might be followed up until death, or until reaching a predefined age. If outcomes and the criteria used for measuring them are not clearly defined, the study should be rejected.
viii This relates to the risk of detection bias.* If the assessor is blinded to which participants received the exposure, and which did not, the prospects of unbiased results are significantly increased. Studies in which this is done should be rated more highly than those where it is not done, or not done adequately.
ix This relates to the risk of detection bias.* Blinding is not possible in many cohort studies. In order to asses the extent of any bias that may be present, it may be helpful to compare process measures used on the participant groups - e.g. frequency of observations, who carried out the observations, the degree of detail and completeness of observations. If these process measures are comparable between the groups, the results may be regarded with more confidence.
x This relates to the risk of detection bias.* A well conducted study should indicate how the degree of exposure or presence of prognostic factors or markers was assessed. Whatever measures are used must be sufficient to establish clearly that participants have or have not received the exposure under investigation and the extent of such exposure, or that they do or do not possess a particular prognostic marker or factor. Clearly described, reliable measures should increase the confidence in the quality of the study
xi This relates to the risk of detection bias.* The primary outcome measures used should be clearly stated in the study. If the outcome measures are not stated, or the study bases its main conclusions on secondary outcomes, the study should be rejected. Where outcome measures require any degree of subjectivity, some evidence should be provided that the measures used are reliable and have been validated prior to their use in the study.
xii This relates to the risk of detection bias.* Confidence in data quality should be increased if exposure level is measured more than once in the course of the study. Independent assessment by more than one investigator is preferable.
xiii Confounding is the distortion of a link between exposure and outcome by another factor that is associated with both exposure and outcome. The possible presence of confounding factors is one of the principal reasons why observational studies are not more highly rated as a source of evidence. The report of the study should indicate which potential confounders have been considered, and how they have been assessed or allowed for in the analysis. Clinical judgement should be applied to consider whether all likely confounders have been considered. If the measures used to address confounding are considered inadequate, the study should be downgraded or rejected, depending on how
17
serious the risk of confounding is considered to be. A study that does not address the possibility of confounding should be rejected.
xiv Confidence limits are the preferred method for indicating the precision of statistical results, and can be used to differentiate between an inconclusive study and a study that shows no effect. Studies that report a single value with no assessment of precision should be treated with extreme caution.
xv Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of
criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most
criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (0): Either most criteria not met, or significant flaws relating to key aspects of study design.
Conclusions likely to change in the light of further studies.
18
Appendix 7
S I G N Methodology Checklist 4: Case-control studies
Study identification (Include author, title, year of publication, journal title, pages)
Guideline topic: Key Question No: Reviewer:
Before completing this checklist, consider:
1. Is the paper really a case-control study? If in doubt, check the study design algorithm available from SIGN and
make sure you have the correct checklist.
2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist.
Reason for rejection: Reason for rejection: 1. Paper not relevant to key question □ 2. Other reason □ (please
specify):
Section 1: Internal validity
In an well conducted case control study: Does this study do it?
1.1 The study addresses an appropriate and clearly focused question.xv Yes
Can’t say
No
SELECTION OF SUBJECTS
1.2 The cases and controls are taken from comparable populations.xv Yes
Can’t say
No
1.3 The same exclusion criteria are used for both cases and controls.xv Yes
Can’t say
No
1.4 What percentage of each group (cases and controls) participated in the study?xv Cases:
Controls:
1.5 Comparison is made between participants and non-participants to establish their
similarities or differences.xv Yes
Can’t say
No
1.6 Cases are clearly defined and differentiated from controls.xv Yes
Can’t say
No
1.7 It is clearly established that controls are non-cases.xv Yes
Can’t say
No
ASSESSMENT
19
1.8 Measures will have been taken to prevent knowledge of primary exposure influencing
case ascertainment.xv
Yes
Can’t say
No
Does not apply
1.9 Exposure status is measured in a standard, valid and reliable way.xv Yes
Can’t say
No
CONFOUNDING
1.10 The main potential confounders are identified and taken into account in the design
and analysis.xv Yes
Can’t say
No
STATISTICAL ANALYSIS
1.11 Confidence intervals are provided.xv Yes
No
SECTION 2: OVERALL ASSESSMENT OF THE STUDY
2.1 How well was the study done to minimise the risk of bias or confounding? xv High quality (++) □
Acceptable (+) □
Unacceptable – reject 0 □
2.2 Taking into account clinical considerations, your evaluation of the
methodology used, and the statistical power of the study, do you think there is
clear evidence of an association between exposure and outcome?
Yes
Can’t say
No
2.3 Are the results of this study directly applicable to the patient group targeted by this guideline?
Yes No
2.4 Notes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question and mention any areas of uncertainty raised above..
xv Unless a clear and well defined question is specified in the report of the review, it will be difficult to assess how well it has met its objectives or how relevant it is to the question you are trying to answer on the basis of the conclusions.
xv Study participants may be selected from the target population (all individuals to which the results of the study could be applied), the source population (a defined subset of the target population from which participants are selected), or from a pool of eligible subjects (a clearly defined and counted group selected from the source population. If the study does not include clear definitions of the source population it should be rejected.
xv All selection and exclusion criteria should be applied equally to cases and controls. Failure to do so may introduce a significant degree of bias into the results of the study.
xv Differences between the eligible population and the participants are important, as they may influence the validity of the study. A participation rate can be calculated by dividing the number of study participants by the number of eligible subjects. It is more useful if calculated separately for cases and controls. If the participation rate is low, or there is a
20
large difference between the two groups, the study results may well be invalid due to differences between participants and non-participants. In these circumstances, the study should be downgraded, and rejected if the differences are very large.
xv Even if participation rates are comparable and acceptable, it is still possible that the participants selected to act as cases or controls may differ from other members of the source population in some significant way. A well conducted case-control study will look at samples of the non-participants among the source population to ensure that the participants are a truly representative sample.
xv The method of selection of cases is of critical importance to the validity of the study. Investigators have to be certain that cases are truly cases, but must balance this with the need to ensure that the cases admitted into the study are representative of the eligible population. The issues involved in case selection are complex, and should ideally be evaluated by someone with a good understanding of the design of case-control studies. If the study does not comment on how cases were selected, it is probably safest to reject it as a source of evidence.
xv Just as it is important to be sure that cases are true cases, it is important to be sure that controls do not have the outcome under investigation. Control subjects should be chosen so that information on exposure status can be obtained or assessed in a similar way to that used for the selection of cases. If the methods of control selection are not described, the study should be rejected. If different methods of selection are used for cases and controls the study should be evaluated by someone with a good understanding of the design of case-control studies.
xv If there is a possibility that case ascertainment can be influenced by knowledge of exposure status, assessment of any association is likely to be biased. A well conducted study should take this into account in the design of the study.
xv The primary outcome measures used should be clearly stated in the study. If the outcome measures are not stated, or the study bases its main conclusions on secondary outcomes, the study should be rejected. Where outcome measures require any degree of subjectivity, some evidence should be provided that the measures used are reliable and have been validated prior to their use in the study.
xv Confounding is the distortion of a link between exposure and outcome by another factor that is associated with both
exposure and outcome. The possible presence of confounding factors is one of the principal reasons why observational studies are not more highly rated as a source of evidence. The study should indicate which potential confounders have been considered, and how they have been allowed for in the analysis. Clinical judgement should be applied to consider whether all likely confounders have been considered. If the measures used to address confounding are considered inadequate, the study should be downgraded or rejected. A study that does not address the possibility of confounding should be rejected.
xv Confidence limits are the preferred method for indicating the precision of statistical results, and can be used to differentiate between an inconclusive study and a study that shows no effect. Studies that report a single value with no assessment of precision should be treated with extreme caution.
xv Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (0): Either most criteria not met, or significant flaws relating to key aspects of study design. Conclusions likely to change in the light of further studies.