Core Outcome Measures in Effectiveness Trialswww.comet-initiative.org

Dr Aoife WatersMRC North West Hub for Trials Methodology Research

University of Liverpool, UK

• Background– What is the problem? – Why do we need a more scientific approach to

outcomes?

• Core Outcome Sets (COS)

• The COMET Initiative

Outline

Why are outcomes important?

• Interventions are compared in RCTs by measuring differences in patient outcomes between the groups

• Selection of appropriate outcomes is therefore crucial in order to assess the effectiveness of an intervention

• “Clinical trials are only as credible as their outcomes” (Tugwell 1993)

What is the problem?• Several tens of thousands of research studies are underway and 500+ are

published every week

• Identifying effectiveness of interventions can be challenging because studies in the same clinical area or of comparable interventions describing findings in different ways

• It’s also difficult for trials to affect policy and healthcare decision making unless they are designed in ways that reveal meaningful answers for patients

• Measurement and reporting of outcomes needs to be tidied up if research is to achieve its aim of helping practitioners and patients to improve health care and health

Cosmetic Outcomes Systematic Review Aspects of cosmesis assessed

Potter et al. Assessment of Cosmesis After Breast Reconstruction Surgery: a Systematic Review. Ann Surg Oncol (2011) 18:813–823

Problems with outcomes in six important Cochrane reviews

5 most accessed in 2009*:• Preventing childhood obesity • Treating childhood obesity• Promoting activity in children• Preventing falls in the elderly• Preventing Type 2 DM

Most cited in 2009*:• Nicotine replacement therapy

*Tovey D. Impact of Cochrane Reviews [editorial]. The Cochrane Library 2010 (7 July)

“We sought data for rate of falls, number of people falling, and number of people sustaining a fracture. However, few studies provided fracture data.”

(Preventing falls in the elderly)

“No study reported relevant data on diabetes and cardiovascular related morbidity, mortality and quality of life..” (Preventing Type 2 DM)

“The studies … varied greatly in intervention design, outcome measurements and methodological quality.” (Preventing childhood obesity)

“Appropriate short- and long-term outcomes need to be defined for children and youth at various weight levels, rather than using conventional or adult-oriented outcomes.” (Treating childhood obesity)

“The studies were heterogeneous in terms of study design, quality, target population, theoretical underpinning, and outcome measures, making it impossible to combine study findings using statistical methods..”

(Promoting activity in children)

“Definitions of abstinence varied considerably ... In five studies it was unclear exactly how abstinence was defined.” (Nicotine replacement therapy)

Outcome reporting bias-the selection of a subset of the original recorded outcomes in a study,

selected on the basis of the results, for inclusion in publication

• ORB suspected in at least one trial in 34% of 283 reviews (Kirkham et al, BMJ 2010)

•42 significant meta-analyses– 8 (19%) would not have remained significant– 11 (26%) would have overestimated the treatment effect by > 20%

• Outcomes that are statistically significant are more likely to be fully reported OR 2.2 to 4.7 (Dwan et al, PLoS ONE 2008)

Core outcome sets– an agreed standardised set of outcomes that should be

measured and reported, as a minimum, in all clinical trials in specific areas of health or health care

•Disease/condition specific•All treatment types or a particular intervention •Should consider both benefits and harms•The minimum – expect others to be collected•Focus of effectiveness trials•Involve stakeholders in their development•Relevant within routine clinical practice

Advantages of COS•Increases consistency across trials

• Maximise potential for trials to contribute to meta- analyses of these key outcomes

•Reduction in selective reporting

• Much more likely to measure appropriate outcomes due to early involvement of stakeholders

•More likely to reach a conclusion about an interventions effectiveness more quickly

The COMET (Core Outcome Measures in Effectiveness Trials) Initiative

• Brings together people interested in the development of COS

• Facilitate and promote development and application of COS

• Liverpool, 2010; Bristol, 2011- Trialists, systematic reviewers, health service users, clinical

teams, journal editors, trial funders, policy makers, regulators

• ‘What’ to measure– Guidance on methods for developing core outcome sets,

including patient involvement and funding applications– Providing a reporting standard for COS

• ‘How’ to measure (validity, reliability, feasibility)– PROMIS– COSMIN– TREAT-NMD

The COMET (Core Outcome Measures in Effectiveness Trials) Initiative

COMET website and database • COMET website makes it easier to find existing research,

helping• funders who may wish to fund work in this area avoid

duplication• researchers who want to know what developments

are underway

www.comet-initiative.org/studies/search

• Work is ongoing to identify, collate and maintain relevant resources in an online searchable database

• 130 completed projects in various areas of health

• COS development is planned or ongoing in 25 clinical areas, with a further 32 in discussion

www.comet-initiative.org/studies/search

COMET website and database

Search results

• In the months since the launch of the COMET website and database (August 2011)– 1335 searches have been undertaken– 3874 individuals visited (6679 visits, 27791 page views)– 91 countries visiting the site

Impact

• Scope• Identifying existing knowledge• Stakeholder involvement• Consensus methods• Achieving global consensus• Implementation • Regular review, feedback, updating

Development of core outcome sets: issues to consider (submitted)

• SR and survey – Identify COS (and other relevant work) – Describe methods used for COS development– Survey COS developers

• Cochrane CRG survey • Funding applications• COMET III• PPI meeting

What next?

Stakeholder buy-in: NIHR HTA

‘Where established Core Outcomes exist they should be included amongst the list of outcomes unless there is good reason to do otherwise. Please see The COMET

Initiative website at www.comet-initiative.org to identify whether Core Outcomes have been

established.'

Professor Hywel Williams, Chair of the NIHR HTA Commissioning Board: ‘Patients and professionals making decisions about health care need access to reliable evidence. The new COMET database will help researchers across the NIHR family and beyond when choosing the outcomes to include in the studies that will establish this evidence base'.

• Currently little consistency– Unacceptable waste of data– Need to make things better

• It is vital to collect important outcomes in all trials– Especially outcomes important to patients – There should be a more scientific approach to outcomes

• Growing activity in development of core outcomes and support for COMET

• Improving the quality of evidence to support informed healthcare decision and inform policy

In conclusion

www.comet-initiative.orgElizabeth Gargon

e.gargon@liv.ac.uk Twitter: @COMETinitiative

AcknowledgmentsCOMET Management Group: Doug Altman, Jane Blazeby, Elizabeth

Gargon, Mike Clarke, Paula WilliamsonFunding: MRC HTMR Network, now MRCCollaborators: Peter Tugwell, Maarten Boers, Caroline Terwee, Holger

Schunemann, Michael Rose, Sunita Vohra, Roberto D’Amico, Lorenzo Moja