Evidence-Based Websites
Douglas Badenoch
Minervation Ltd
About me
• Strathclyde University– Lecturer, 1990-95
• Oxford CEBM– Programme Manager,
1996-2002
• Minervation Ltd– Founding Director,
2002-
About Minervation
Evidence-based health care consultancy
• Information science• Usability testing• Design, hosting and
development
Our Work
Content• J&J, NLH
Evaluation• CRUK, NHS Evidence
Design, hosting and development
• CEBM, EQUATOR Network, ScottishDental
EBW could mean…
1. Websites supporting EBHC
2. Websites about EBHC
3. Embedding evidence in websites
4. Evidence about (the effectiveness of) websites
This session mostly about…
1. Websites supporting EBHC
2. Websites about EBHC
3. Embedding evidence in websites
4. Evidence about (the effectiveness of) websites
“We can have more impact on health by doing what we already know than by any device or technology likely to become available in the next decade.”Muir Gray, 2005
We know what they should look like
1 There’s good evidence that we are not getting the evidence to where it’s needed, but that when we do, it has a positive impact upon practice.– Accessible, user-friendly– Full text, evidence-based summaries
• Clinicians have about 5 minutes to look for evidence– About half their questions go unanswered– They perform better when they have access to
evidence-based summaries
Websites about EBHC
4. The “maverick” traditions of EBHC innovation show no signs of abating
• Reporting guidelines
• Clinical research online
• Uncertainties
www.equator-network.org
Worldwide initiative to bring together all that we know about how to report the results of research.
• By study type• By clinical specialty• By publication type• By document structure
www.equator-network.org
Audience:• Researchers• Editors• RG developers
Current developments:• Implementing a database of RGs• Linking RGs to the underlying
evidence base• Supporting the implementation of
RGs“Take me to your
Reporting Guidelines”
We’re starting to do more about…
3. Embedding evidence-based methods into content production
• User involvement in requirements, comprehensive literature searching, critical appraisal and transparent evidence synthesis
• Review and updating protocols
www.scottishdental.org
Set up in 2003
Relaunched 2007
Taxonomy-based
User centred design
UK Prostate Link www.prostate-link.org.uk
Quality-assessed information about prostate cancer
• Regular updates and alerts
• Popular with users, not with providers!
Instruments
• No lack of them (over 270 in our 2005 review)!– Searched health and information science
databases– Very few instruments assessed all of the
elements that mattered to our users– Of those that did, none had been validated– “Levels of evidence” impractical– Evidence of problems with policing
Implementation
The assessment method had to:
• Address what matters to end-users
• Encompass validated, evidence-based methods of evaluating information
• Be valid and transparent
• Produce a quantitative score for use in search ranking
Implementation
Usability Questions: 1. Is the site accessible without a
login? 2. Does the site conform to web
accessibility standards? 3. Is the site design clear and
transparent? 4. Is the site design consistent
from one page to another? 5. Can users find what they need
on the site? 6. Is the format of information
clear and appropriate for the audience?
Reliability Questions: 7. Is it clear who has developed
the web site and what their objectives are?
8. Does the site report a robust quality control procedure?
9. Is the page content checked by an expert?
10. Is the page updated regularly?
11. Does the page cite relevant sources where appropriate?
Validation
Blind, independent comparison of two trained assessors:
• Information specialists
• Ranked 42 websites• Measured the rank
order correlation• P < 0.0001
Prostate Cancer Support Federation
http://prostatecancerfederation.org.uk/
A small user-led charity which provides unique information on a minimal budget.
Usability Questions: Score out of 3
1. Is the site accessible without a login? 3
2. Does the site conform to web Accessibility standards? 2
3. Is the site design clear and transparent? 3
4. Is the site design consistent from one page to another? 2
5. Can users find what they need on the site? 1
6. Is the format of information clear and appropriate for the audience? 2
Reliability Questions:
1. Is it clear who has developed the web site and what their objectives are? 3
2. Does the site report a robust quality control procedure? 0
3. Is the page content checked by an expert? 0
4. Is the page updated regularly? 1
5. Does the page cite relevant sources where appropriate? 2
Total 19/33
Embedding quality
• Greater public awareness– e.g Bad Science
• Various quality initiatives must be joined up
• Role of the IS must be central to quality
• Will people start looking at the About Us pages?
From prevalence to incidence
• Having identified the existing best available evidence, the challenge is to communicate it effectively and keep it up to date.
We have a real problem with..
4. Working out whether they work or not– Standards of proof are lower than normal NHS
interventions– EBHC websites (type 1) don’t even use the
traditional methods well– User modelling is only now taking centre stage in
evaluating website performance– Political imperatives will override all of that
anyway