Post on 16-Jul-2015
transcript
COHG Editors’ & Clinical Advisors’
Meeting
23 October 2013
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
Agenda
COHG Editors & Clinical Advisors Meeting 23 October 2013
1. Welcome2. Role of Editor 3. Prioritisation of topics 4. Title registration – discussion of proposed titles 5. Introduction to MECIR 6. Screening of reviews/ Tips for good practice 7. Worked example – flossing review:
a. Abstract b. Risk of Biasc. Outcomes d. Summary of findings e. Consistency across review
The role of a Cochrane OHG
Editor
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
Co-ordinating Editors
Managing Editor/Deputy Managing Editor
Trials Search Co-ordinator (Feedback Editor)
Consumer Co-ordinator
Editors
Clinical Advisors
Referees
Research/Editorial Support Co-ordinator
Administrative Assistant
COHG’s Editorial team
To support review teams:
805 registered-COHG authors
COHG’s membership
To explore the role of Editors
Minimum requirements
Input into editorial process
Role of Contact Editor
Open discussion
Aim of session
Minimum requirements:
Completed a review prior becoming an editor
Attend at least two Cochrane/COHG events a year (e.g. editorial meetings, review proposal meetings, regional meetings, colloquia, training days…)
Input into editorial process
Act as Contact Editor
Editors
Selecting and shaping the Cochrane Reviews prepared by COHG
new review proposals
priorities / emerging areas of research
Providing comments on review drafts
Editorial process input
Allocated reviews within specialty;
To provide ad-hoc methodological and/or content advice to review teams throughout entire editorial process;
Judge whether the review is ready for peer review and help identify appropriate referees;
Look over submitted comments; outline expected revisions; advise authors on response to referee comments;
Examine revised reviews and confirm whether review should go forward for signing off/publication.
Act as Contact Editor
What aspects of the Editor’s role could be made easier?
List of assigned reviews?
Improved checklist form?
More/less time for comments?
Sufficient support from Editorial base?
Unmet training needs?
Ways to keep up-to-date with developments in Cochrane or COHG?
Discussion
Prioritisation of Cochrane OHG Reviews
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
Base-line salaries (NIHR)
Global Alliance (donations from dental bodies) British Orthodontic Society, UK Royal College of Surgeons of Edinburgh, UK New York University College of Dentistry, USA British Society of Paediatric Dentistry, UK National Center for Dental Hygiene Research &
Practice, USA Canadian Dental Hygienists Association, Canada British Society of Periodontology, UK British Society of Oral Surgeons, UK Mayo Clinic, USA
Funding providers
Importance of our reviews asking relevant questions
Important reviews are kept up-to-date
Two international prioritisation exercises so far:
Orthodontics and paediatric dentistry
Prioritisation of reviews
1. Invitation to all COHG paediatric authors;
2. Invitation to international expert panel;
3. Two teleconferences;
4. Final list of reviews
Paediatric prioritisation exercise process
Paul Ashley, UK
Joel Berg, USA
Michael Casas, Canada
Urshla Chaudhry (BSPD), UK
Ben Cole, UK
Michael Cranfield, UK
Ivana Čukovid-Bagid, Croatia
Peter Day, UK
Chris Deery, UK
Monty Duggal, UK
Mike Harrison, UK
Marie-Therese Hosey, UK
Milton Houpt, USA
Norbert Krämer, Germany
Jessica Lee, USA
Nick Lygidakis, Greece
David Manton, Australia
Jeanette Mooney, UK
Linda Rosenberg, USA
Svante Twetman, Denmark
Jaap Veerkamp, Netherlands
Richard Welbury, Scotland
Expert panel members: paediatric dentistry
Prevention
Fluoride varnishes, gels, mouthrinses, toothpastes (8)
Sealants (2)
Results of paediatric prioritisation exercise
Treatment
Different methods for treating and managing decay in primary and permanent dentition (6)
Pulp treatment (1)
Preoperative analgesics (1)
Results of paediatric prioritisation exercise
Additional topics (with reviews)
Primary school-based behavioural interventions for preventing caries
Hypnosis for children undergoing dental treatment
Sedation of children undergoing dental treatment
Results of paediatric prioritisation exercise
Additional topics (no reviews)
Complex interventions for preventing caries
Diagnosis of caries
Management of early caries lesions in children
Results of paediatric prioritisation exercise
Paediatric experts wanted to collaborate over guidelines
Not what we are here for, but we thought of an idea to help:
Guideline repository
Expert panel discussion: paediatric guidelines
The purpose of the repository is to:
help identify priority review topics that could inform guideline development;
identify areas of duplication/overlap, where evidence tables could be shared between Guideline Development Groups (GDGs);
increase stakeholder involvement in guideline development by widening dissemination.
Guideline repository
We already liaise with NICE and SIGN to ensure we are prioritising the reviews they need
Fluoride varnish review
Initial review published in 2002 (9 trials)
Update published July 2013 (22 trials)
Guideline implementation
Prioritisation exercises need to:
include all stakeholders
cover whole of oral health
Prioritisation of reviews
Introduction to MECIR
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
Methodological Expectations of CochraneIntervention Reviews
Methodological standards for the conduct of new Cochrane Intervention Reviews December 2012
Standards for the reporting of new Cochrane Intervention Reviews December 2012
Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS) February 2013
MECIR
Standards for conduct
Standards for conduct
Standards for reporting
Standards for reporting
Reporting of PLS (PLEACS)
Reporting of PLS (PLEACS)
Aim
To highlight some of the key features of a Cochrane Review and apply MECIR standards where appropriate:
Abstract Risk of Bias Outcomes Summary of Findings Consistency across the review
Sambunjak D, Nickerson JW, Poklepović T, Johnson TM, Imai P, Tugwell P, Worthington HV. Flossing for the management of periodontal diseases and dental caries in adults. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD008829. DOI: 10.1002/14651858.CD008829.pub2.
Worksheet
You’ve been provided with: A set of MECIR standards for the reporting of
abstracts
An abstract for the flossing review
Read through the abstract and considerwhether the reporting of the results and conclusion in the abstract complies with the MECIR standards.
Abstract
Do the conclusions follow on from the results?
What changes, if any, would you make?
Abstract
Risk of Bias
Chapter 8 of the Cochrane Handbook
It’s about validity of findings
It’s not about ‘quality’
A main component of the Summary of Findings
Bias Authors’
judgement
Support for judgement
Random sequence generation
(selection bias)
Allocation concealment
(selection bias)
Blinding of participants and
personnel (performance bias)
Blinding of outcome assessment
(detection bias)
Incomplete outcome data
(attrition bias)
Selective reporting
(reporting bias)
Other bias
Example RoB table
Children were sequentially allocated. The first child who came for examination to the first group, second child to the second group
Random sequence generation:HIGH - systematic method, not true randomisation
Further examples
The study was conducted in a single blind fashion
One of the blinding domains, but we don’t know which one: UNCLEAR – who is blinded?
Further examples
An envelope was opened, in which it indicated whether the patient should receive the bio-adhesive gel or not
Allocation concealment:UNCLEAR – is it opaque, sealed and sequentially numbered, and were they centrally/remotely assigned???
Further examples
22 participants had amalgam restorations whilst 23 had composite restorations. 8/47 teeth had restoration failures in the amalgam group compared to 6/52 in the composite group.
Selective reporting:HIGH - reported at tooth level rather than at patient level, ignoring the paired nature of the data and clustering of teeth in the mouth
Further examples
In a trial of drug X versus placebo to treat oral mucositis, 13/35 were withdrawn from the drug X group, 7 of these due to lack of efficacy. 25/34 were withdrawn from the placebo group, 17 due to lack of efficacy. An intention-to-treat (ITT) analysis was carried out including those remaining.
Incomplete outcome data:HIGH – only 9 of the 34 in placebo group analysed
Further examples
Summarising RoB
Include ‘all outcomes that are likely to be meaningful to clinicians, patients (consumers), the general public, administrators and policy makers….’
may not have considered/evaluated all in trial
unlikely to be reported in same publication
useful to acknowledge even if not recorded/ reported
Which outcomes?
What are the main outcomes of interest for decision making?
Importance of outcomes from clinicians, consumer advisory groups, guidelines, scoping search COMET initiative
≤3 primary outcomes (beneficial and adverse)
Info for SoF table – key information ‘There should be no more than seven main outcomes, which
should generally not include surrogate or interim outcomes.’
Directly relevant to review question, not ‘what the review found’
Prioritisation
be well-defined measuring a single concept or where an outcome is complex, this should be clearly specified
specify the timing of measurement clinically important period of measurement multiple observations introduce unit of analysis issues
specify the nature of measurement subjective or objective outcomes, validated scale of
measurement, consistent with other reviews in same clinical area?
be directly measured wherever possible, avoiding surrogate outcome measures
have relevance for future updates of the review
The outcomes of a review should evaluate both the beneficial andnegative effects of the interventions.
Outcomes should…
"Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“
The main outcome measures for children and carers were long-term freedom from the main symptom of dental decay - pain.
Primary outcome measures
freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling
time until filling or crown needs to be replaced or requires further intervention
proportion of filled or crowned teeth retained until appropriate age of shedding
Problem illustration
Single or multiple outcomes
"Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“
Other measures of success recorded and analysed
absence of clinical or radiographic evidence of secondary caries
other clinical signs of pathology (fracture of tooth or filling, wear of crown, inflammation of gingival (gum) tissue)
patient satisfaction
costs to patient and provider
adverse events
Problem illustration
Different outcomes (surrogate?)
Single or multiple outcomes
Any adverse events or those likely to be related to the intervention?
Primary outcome measures :
Pain from dental decay (after the procedure) at time points x, y and z
Treatment success at x year (defined as freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling)
Time until filling or crown needs to be replaced or requires further intervention
Proportion of filled or crowned teeth retained until appropriate age of shedding
Other measures of success recorded and analysed:
Absence of clinical evidence of secondary caries (add definition how clinical evidence of secondary caries are to be measured). Where studies do not report clinical outcomes for secondary caries, radiographic evidence will be reported if available.*
Inflammation of gingival (gum) tissue
Peri-operative pain - Pain (as rated by children) during the procedure (adverse event)
Bleeding ( adverse events)
Patient satisfaction
Costs to patient and provider*This means that only one of these will be reported if both have data are available in the studies. If only radiographic is available, then this will be included as a surrogate for the real outcome of interest.
Possible solutions
Primary outcomesThe main outcome of interest was success or failure of the replacement or repair restoration and associated tooth as assessed by clinical examination.
The primary outcome measures were therefore the clinical acceptability or unacceptability of each restoration, defined by the USPHS criteria, Ryge criteria or modifications of these scales, and assessed by clinical examination.
It was anticipated that this would be recorded as success or failure of the restoration, and/or that further repair or replacement of the restoration was necessary.
Other outcome measures indicating the failure of a replacement or repaired amalgam restoration included the following (occurring in relation to the repaired/replaced amalgam): placement of an additional restoration e.g. crown or inlay; root filling; clinical symptoms e.g. pain, swelling, diagnosis of pulpitis, abscess
formation, and extraction of the tooth.
Studies should have determined success or failure according to the same criteria used in the decision to replace or repair the restoration.
Comparable review
Secondary outcomesIn addition to the main outcome measure, any outcomes reported perioperatively (e.g. pain/discomfort) and postoperatively (i.e. within 48 hours; e.g. pain/discomfort) were to be recorded. Where any other outcomes were presented e.g. related to patient experience or aesthetics these were also to be recorded.
Timing of outcome assessmentThe decision on which outcome period to use for the review was to be based on the most commonly reported period(s) of assessment amongst studies meeting the inclusion criteria.
Outcome data from all periods of follow-up were to be included, but where the period of follow-up differed between studies, this was to be categorised as medium-term (less than 5 years) or long-term (5 years and above).
Time-to-event (survival data) was to be collected and analysed where available.
Comparable review
Remember!
Reporting of outcomes should rarely determine the eligibility of studies for a review.
Studies shouldn’t be excluded because they provide no 'usable' data. Even in absence of any information for a review outcome there will still be an entry into the SoFtable.
Look for ‘expected’ key outcomes in the clinical area are routinely measured
Study eligibility
Summary of Findings
Based on work presented by Nancy Santesso, Quebec 2013
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
Developed PICO
Located studies
Abstracted data from studies
Assessed the risk of bias in the studies
Synthesised the data for each outcome
Assessed heterogeneity and publication bias
Assess the evidence and draw conclusions
Summary of Findings
How do we:
interpret results and draw conclusions?
GRADE
present results to reader/users?
Summary of Findings tables
Summary of Findings
Two main concepts when interpreting results:
magnitude of effect
confidence in that effect
Summary of Findings
Summary of Findings
Summary of Findings
Summary of Findings
Determinants of quality of evidence/confidence in effect
Consider 5 factors to evaluate the quality of the evidence:
Risk of bias
Inconsistency (or heterogeneity)
Indirectness (PICO and applicability)
Imprecision(number of events and confidence intervals)
Publication biasSee also Cochrane Handbook, Chapter 12
Quality of Evidence varies from:
HIGH
MODERATE
LOW
VERY LOW
Confidence
Pulling it together
Quality Interpretation
HighWe are very confident that the true effect lies close to that of the
estimate of the effect.
ModerateWe are moderately confident in the effect estimate:
The true effect is likely to be close to the estimate of the effect,
but there is a possibility that it is substantially different.
LowOur confidence in the effect estimate is limited:
The true effect may be substantially different from the estimate
of the effect.
Very lowWe have very little confidence in the effect estimate:
The true effect is likely to be substantially different from the
estimate of effect.
Pulling it together
Systematic review and clinical question information: Participants, interventions and comparisons
Outcomes
Present important outcomes:
Select from primary outcomes identified a priori in protocol or those important to making recommendations
Choose patient-important outcomes
Include outcomes, whether you have results or not
Describe the outcomes (e.g. Provide scale range and indicate a high score)
Outcomes
What happens to people with, and without, intervention?
Results: Absolute effects
Results: Relative effects
Only some studies contributed information about an outcome
Results: Number of participants/studies
Results: Quality of the evidence
Results: Comments
Clarification
Judgements
Transparency
Results: Footnotes
Considerations when making conclusions Do I believe the results from these studies?
Risk of bias
Are the results consistent across studies? Inconsistency
Are these all of the studies? Publication bias
Is this effect size precise? Imprecision
How do these results apply? Applicability, directness
Conclusions
You are provided with a SoF table for the flossing review.
Two forest plots showing the results for gingivitis and plaque at 1 month.
Examine the SoF and the Forest Plots for discussion
Flossing review: SoF
You are provided with:
MECIR standards that relate to the conclusions from the main text of the review
the conclusions from the flossing review
First consider the information presented inthe conclusion alone, and then compare it to that in the abstract and summary of findings
Internal consistency
• What changes, if any, do you think are necessary in the conclusion?
• How does the information in the conclusion compare to that in the abstract and summary of findings?
• Are there any additional changes you would make and why?
Internal consistency
What aspects of the Editor’s role could be made easier?
List of assigned reviews?
Improved checklist form?
More/less time for comments?
Sufficient support from Editorial base?
Unmet training needs?
Ways to keep up-to-date with developments in Cochrane or COHG?
Discussion