Cancer In The News: Bias And Quality In Media Reporting Of ... · Cancer In The News: Bias And...

CancerInTheNews:BiasAndQualityInMediaReportingOfCancerResearch.

AmandaAmberg,DarrenN.Saunders.

SchoolofMedicalSciences,UniversityofNewSouthWales,Sydney,NSW2052,Australia.

Correspondence:

Assoc.Prof.DarrenSaundersSchoolofMedicalSciencesUniversityofNewSouthWalesSydney,AustraliaPhone:+61293852494Email:[email protected]

Keywords:cancer,research,communication,bias,gender,media

.CC-BY-NC-ND 4.0 International licenseavailable under awas not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprint (whichthis version posted September 19, 2018. ; https://doi.org/10.1101/388488doi: bioRxiv preprint

https://doi.org/10.1101/388488

http://creativecommons.org/licenses/by-nc-nd/4.0/

Amberg&Saunders–Biasincancerresearchreporting

2

Abstract

Cancer research in the news is often associatedwith sensationalising and inaccurate reporting,

giving rise to false hopes and expectations. The role of study selection for cancer-related news

stories is an important but less commonly acknowledged issue, as the outcomes of primary

research are generally less reliable than those of meta-analyses and systematic reviews. Few

studieshaveinvestigatedthequalityofresearchthatmakesthenewsandnopreviousanalysesof

theproportionsofprimaryandsecondaryresearchinthenewshavebeenfoundintheliterature.

Themainaimofthisstudywastoinvestigatethenatureandqualityofcancerresearchcoveredin

online news reports by four major news sources from USA, UK and Australia. We measured

significantvariationinreportingquality,andobservedbiasesinmanyaspectsofcancerresearch

reporting,includingthetypesofstudyselectedforcoverage,andinthespectrumofcancertypes,

genderofscientists,andgeographicalsourceofresearchrepresented.Wediscusstheimplications

ofthesefindingforguidingaccurate,contextualreportingofcancerresearch,which iscritical in

helping thepublicunderstandcomplex scienceandappreciate theoutcomesofpublicly funded

research,avoidunderminingtrustinscience,andassistinformeddecision-making.



https://doi.org/10.1101/388488



3

Introduction

Canceristhesecondleadingcauseofdeathglobally,accountingfor8.8milliondeathsin2015[1].

ThetotalannualeconomiccostofcancerwasestimatedatapproximatelyUS$1.16trillionin2010

[2]anditisestimatedthat30-50%ofcancerdeathscouldbepreventedbymodifyingriskfactors

including tobacco exposure, alcohol consumption, obesity, exercise and infection [3]. Cancer is

complexandchallengingtostudy,andnewsreportingoncancerresearchissusceptibletohype,

contradictionandmisinformation.Clearlycommunicatingtheoutcomesandcontextofresearchis

key to helping non-specialists understand complex science, and assisting patients and families

make informeddecisions aboutmodifying risk and treatment selection. Poor reporting practice

may have serious consequences for public and scientific communities alike, including the

generationoffalseorunmetexpectations,potentiallyfuellingdisappointmentandalossoftrust

inscience[4,5].

Few studies have focused on quantifying the types and quality of scientific research that gain

attentioninthenews,whichisarguablyasimportantasaccuratetranslationofresearchpaperto

newsstory.Evenanewsreportthatperfectlydescribesthefindingsofastudyisoflittlevalueto

thepublicifthefindingsthemselvesarebasedonweakevidence.Further,thereliabilityandlong-

termimpactofprimaryresearchisrarelyknownatthetimeofpublication.Incancerresearchand

otherfields,ithasbeendemonstratedthatinitiallyreportedeffectsizestendtobenotablylarger

thanthosepublished insubsequentmeta-analysesandthatastrikingproportionofpublications

will be refutedby follow-up studies [6-8]. ThisphenomenonwashighlightedbySchoenfeldand

Ioannidis in theirmeta-analysisondietary riskandpreventionof cancer [9].News reportingon

cancerresearchhaspreviouslybeenassociatedwithpooraccuracy,sensationalisingheadlinesand

presentation of conflicting information [10]. A preference towards reporting novel primary

researchstorieswithlowreplicationlikelihoodoftenresultintherefutation(orfailedreplication)



https://doi.org/10.1101/388488



4

of a research finding getting significantly less attention than the initial finding itself [11],

reinforcingan‘asymmetryofbullshit’[12].

Previousanalyseshaveobserved that thedistributionofnews storiesby canceranatomical site

mirrorsincidenceratesmorecloselythanmortalityratesbutthatcertaincancertypeswereover-

orunder-represented.Thesedistortions,potentiallydrivenbypersonalisationbias(e.g.,celebrity

profiles)were also reflected in riskperceptionanddiscrepancies in funding for cancer research

[13-15]. Others have also documented variation in quality, topic coverage and style of cancer

researchreportinginprintmedia[16-18].Previousresearchinvestigatingthequalityoftranslation

from research papers and press releases to news stories highlights widespread problems with

inadequatereferencinganddistortedreporting[19-24].Whilethetabloidor ‘popularpress’was

themainculprit,thesameissuesalsoexistinthemoreprestigiousbroadsheetnewsoutlets[20,

25].Itmaybetemptingforresearchers,journalists,philanthropicbodiesandresearchinstitutions

to sensationalise scientific findings in their pursuit of funding, readers or publicity. A common

consequence of poor quality reporting is a hype cycle characterised by false expectations and

subsequentdisappointment[4].Hypemaybegeneratedbyjournalists,institutionalpressreleases,

orthescientiststhemselvesandcanthenbeamplifiedthroughthemediacycle[21,26].

Primary research more easily lends itself to ‘breakthrough’ headlines since, by definition, it

presents original data. Quality and reliability are not intrinsic features of meta-analyses and

systematic reviews but depend on appropriate systematic methods [27, 28]. Nevertheless, the

natureandpurposeoftheseformsofsecondaryresearch–collating,comparingandre-analysing

a set of primary studies to reduce uncertainty – render them less susceptible to error than

individual primary studies [7]. Based on this assumption, secondary research is an important

source of science news for the general public, yet there are indications of a reporting bias

favouring primary studies [29]. In academic publishing, peer review and the accumulation of



https://doi.org/10.1101/388488



5

primaryresearchpapersfollowedbymeta-analysesandreviewarticlesaredesignedtohelpfilter

inaccuracies of individual study outcomes but there is no equivalent formalised system in the

newsmediabeyondstandardeditorialoversight.Analysisof734front-pagestoriesaboutmedical

researchinmajornewspapersfoundthatjustoverhalfwerebasedonpaperspublishedinpeer-

reviewedjournals,aminorityofthesebeingsystematicreviews[29].Thisemphasisestheneedfor

bettercharacterisationofthetypesandqualityofcancerresearchstudiesthatgainattention in

thenews.

Hence,thisstudyaimedtoanalyseselectionbiasandqualityofcancerresearchpublishedinfour

majornewssourcesfromtheUK,USAandAustraliaoverasix-monthperiodin2017.



https://doi.org/10.1101/388488



6

Methods

Newsreportcollection

Twentynewsreportsweresampledfromeachoftheonlineversions(publishedbetweenMarch

andSeptember2017)ofTheGuardian (UKedition),TheNewYorkTimes (NYTimes),TheSydney

Morning Herald (SMH) and the Australian Broadcasting Corporation (ABC), generating a total

datasetof80reports.Thefollowingsearchtermswereusedwithinthesearchfunctiononeach

source’s website: ‘cancer study’, ‘cancer research’, ‘cancer science’, ‘targeted therapy’, ‘cancer

screening’, ‘cancer screening study’, ‘tumour research’, ‘tumor research’, ‘cancer treatment’,

‘cancergenetics’and‘cancerscientists’.Onlyoriginalreportswereincludedinthesample,reports

re-publishedfromothernewssourceswereexcludedtoavoidoverlapinthedata.Thereportshad

to discuss a study investigating a cancer-related issue including epidemiology, carcinogens,

screening, diagnostics, therapies, basic biology, risk or prevention. General reports which dealt

withcancer-relatedtopicsbutwhichdidnotbasethediscussiononaspecificstudywereexcluded.

Whenseveralstudieswerediscussedandnoonecentralstudycouldbedistinguished,thereport

was excluded. Details of individual reports (including Pubmed ID (PMID), quality scores and

hyperlinkstoreports)arecontainedinSupplementaryTable1.

Classificationandscoring

Where possible, the study discussed in each media report was traced and classified as basic

research,clinicalresearch,epidemiologicalresearch,meta-analysisorsystematicreviewaccording

to the classification of Röhrig et al. [30]. Original research sources cited in news reports were

classified as either published paper, conference, report, press release, pre-print article, funding

source/researcherorunknown.News reportqualitywas scoredaccording toamatrixbasedon

elevencriteriaadaptedfrompreviousstudies(Table1)[19,21].Australiancancer incidenceand

mortalitystatisticswereobtainedfromtheAustralianInstituteofHealthandWelfare[31].Gender



https://doi.org/10.1101/388488



7

ofseniorauthors(i.e.namesappearinginfirstandlastpositiononauthorlist)andquotedexperts

in each report was also quantified using pronouns quoted in reports or on homepage. Source

nationalitywasclassifiedaccordingtoprimaryacademicaffiliationofcorrespondingauthors.

Dataanalysis

Chi-square or Fischer’s exact tests were used to compare the categorical variables (primary or

secondary study), assuming statistical significance if P<0.05. Quality scores were aggregated

according to Table 1 andmean scores calculated. Statistical significance of differences inmean

scores between the news sources was tested using one-way ANOVA, assuming Gaussian

distribution,formultiplecomparisonswithaTukeytest.Whenanalysingnationalbias,allstudies

that had been conducted as international collaborations were included in the ‘international’

category.

Table1.NewsReportQualityScoringCriteria.

Criterion Scoring Notes

Peerreviewedresearchsource 0,3(no,yes) Peerreviewwasassignedaheavierweightingthanotherbinarycriteriatoreflectimportance.

Conflictsofinterestorfundingsourceidentified 0,1(no,yes) Conflictsofinterestorfundingsourceshadtobementionedinthenewsreporttomeetthiscriterion.

Independentexpert(s)quoted 0,1(no,yes) Independentexpertsmustnotbeaffiliatedwiththepaper,publishingjournal,researchinstituteorfundingbody.

Linktoresearchsource 0,1(no,yes) Linksmustleaddirectlytotheresearchsource(publishedpaper,conferenceabstract,etcetera).Linkstojournalhomepagereceived0forthiscriterion.

Traceableresearchsource 0,1(no,yes) Enoughinformationprovidedtoallowtracingthesourcewithin5min.

Studylimitationsidentified 0,1(no,yes) Requiredmentioningalimitationofthestudy’smethod,evidence,conclusionorimplications.Generalstatementsaboutwhatthestudydidnotaimtoinvestigatewerenotsufficienttofulfilthiscriterion.

Placedstudyinbroaderresearchcontext 0,1(no,yes) Thereportshouldrefertorelatedknowledgeortheoriesgeneratedbyresearchersunaffiliatedwiththestudyinfocus.

Absoluterisksorbenefitsquantified 0,1(no,yes) Risksandbenefitspresentedbyastudyshouldbedescribedinabsolutenumbers.Percentagesortotalincidencedidnotfulfilthisrequirement.Thiscriterionwasnotapplicabletosomereports.Forthese,thetotalscoreswereadjustedasaproportionofthemaximum:(assignedscore÷16)×17.

Misleadingheadline 0,1(yes,no) Thisincludedsensationalising,incorrectorotherwisemisleadingheadlines.

Emphasismaintained 0-3 Themainaims,outcomesandimplicationsaspresentedinthestudyshouldbemaintainedintheheadlineandbodyofthenewsreport.Thescoringrangewasasfollows:emphasismaintainedinneitherheadlinenorbody(0),ineitherheadlineormostofbody(1),inbothheadlineandmostofbody(2),inbothheadlineandallofbody(3).

Avoidedovergeneralisation 0-3 Overgeneralisationcouldrefertosamplepopulations,thetargetsofatreatmentorotheraspectsofthestudydependingonitsclassificationandtopic.Scoreswereallocatedasfollows:bothheadlineandbodyovergeneralising(0),eitherheadlineorbodyovergeneralising(1),headlineandbodymostlyavoidedovergeneralisation(2),headlineandallofbodyavoidedovergeneralisation(3).



https://doi.org/10.1101/388488



8

Results

Contentbias

The long-term implications of primary research findings are rarely known at the time of

publication. Indeed, while basic research is absolutely essential to scientific progress, there is

evidencethataremarkableproportionofpublishedresultswillberefutedbyfurtherinvestigation

or that subsequent meta-analyses will report notably smaller effect sizes, in cancer as well as

otherresearchareas[6-8].SchoenfeldandIoannidis[9]publishedameta-analysishighlightingthis

phenomenon in research on cancer risk and prevention. Similar trends have been observed in

basicmedicalresearch,whereonlyasmallfractionofthemostencouragingearlyfindingsendup

inclinicaluse[32].Thisbecomesimportantwhenscientificresearchpapersgainattentioninthe

mainstream press, the predominant source of science news for the public. However, there is

limitedliteratureoncontentbiasinsciencenewsreporting.Qualityandstylehavebeenshownto

varyacrossnewsoutlets[16,17],buteventhelargestnewspaperswiththebestoverallstandards

tend to cover more studies with poorer methodology and observational studies over RCTs or

systematicreviews[33-35].

Toinvestigatethedistributionofdifferentstudytypesincancerresearchreporting,wequantified

thenumberofprimaryandsecondarystudiesandtheirsubtypes inasampleof80newsstories

from4differentoutlets.Ofthesamplesreports,92.5%(74/80)werebasedonprimaryresearch

studies (Fig. 1a).When studieswere further classifiedby subtype, epidemiological studieswere

themostprevalentoverall,accountingfor38.75%ofreports(31/80),followedbyclinicalandbasic

researchat28.75%(23/80)and23.75%(19/80)respectively(Fig.1b).Secondarystudiesconsisted

of four systematic reviews and twometa-analyses (Fig. 1b).One studydidnot fit in anyof the

categoriesandwasthereforeclassifiedas‘other’.



https://doi.org/10.1101/388488



9

Figure 1. Study type bias represented in online news reports about cancer: A. Frequency ofprimary and secondary research represented in online news reports about cancer research. B.Distributionofresearchsubtypesrepresentedinnewsreports(n=80).

Wenextclassifiedandquantifiedthesourceof researchstudiescited inoursampleof80news

reports. Most news reports (68.75% or 55/80) were based on peer-reviewed papers (Fig. 2),

sometimesaccompaniedbyapress release.Researchpublishedoutsideof traditional academic

journals (e.g. reports issued by government agencies)were the basis of 12.5% (10/80) of news

items, and 10% (8/80) of reports were based on conference presentations (five of these were

publishedconcurrentlywithamajorannualcancerconference).Twonewsreportswerebasedon

institutional press releases without an associated published scientific paper, and two others

referredonlytotheresearcherorfundingsource.Thesourceofthreenewsreportscouldnotbe

traced(Fig.2).

Epidemiologica

l

Clinica

lBas

ic

System

atic r

eview

Meta-an

alysis

Other0

10

20

30

40

Num

ber o

f rep

orts

Primary

Secondary

0

20

40

60

80

Num

ber o

f rep

orts

Amberg - Figure 1.CC-BY-NC-ND 4.0 International licenseavailable under a

was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (whichthis version posted September 19, 2018. ; https://doi.org/10.1101/388488doi: bioRxiv preprint

https://doi.org/10.1101/388488



10

Figure2.Distributionofresearchtypesformingthebasisofonlinenewsreportsaboutcancerresearch(n=80).Cancertypesrepresentedinreporting

We examined the distribution of cancer types (defined by anatomical site) represented in our

cohortof80newsreports.Themostfrequentcategoryobservedwasnon-specific(i.e.notrelated

toaspecificcancertype),representing18/80(22.5%)reportsandpossiblyreflectingastrongbias

towards more basic research on disease mechanisms and risk. Among cancer types explicitly

identifiedinnewsreports,breast(15%),melanoma(11.3%),lung(8.8%)andblood(8.8%)cancers

werethemostfrequentlyreported(Figure3a).Reportsspecificallymentioninggastric,testicular,

brainandpancreascancerweretheleastfrequentlyobserved,witheachonlybeingrepresented

inasinglereport.Manycancertypeswerenotrepresentedinnewsreportsatallduringsample

period.Whenanalysedinthecontextofrelativeratesofincidenceandmortalityofspecificcancer

types,weobservedastrongcorrelationbetweenreportingandincidenceofspecificcancertypes

(R2 = 0.594, p=0.0013) but not with mortality (Figure 3b,c). Research on cervical cancer was

reported more frequently than would be expected relative to incidence, while prostate and

Published

pap

er

Confere

nce

Report

Press

relea

se

Funding so

urce/r

esea

rcher

Unknown

0

20

40

60

Num

ber

of n

ews

repo

rts

Amberg - Figure 2 .CC-BY-NC-ND 4.0 International licenseavailable under awas not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made


https://doi.org/10.1101/388488



11

colorectal cancer were under-represented in news reports (Figure 3b). Relative to mortality,

cervical,melanomaandbreastcancerwereover-represented,whilelung,pancreas,andcolorectal

cancerwereunder-represented(Figure3c).

Fig3:Analysisofbiasincancertypesreported.A.Distributionofcancertypesinresearchstudiescoveredinnewsreports,withAustralianincidenceandmortalityrates;Reportingfrequency(asaproportionoftotal)comparedwithrelativeincidence(B)andmortality(C).

Amberg - Figure 3

A

B C

non-spec

ificbrea

st

melanoma

lungblood

prosta

tece

rvix

colorec

tal liver

uterus

ovaria

n

gastri

c

testic

ularbrai

n

pancre

as0

5

10

15

20

25

0

5000

10000

15000

20000

frequ

ency

Incidence / mortality (total)

incidencemortality

0 5 10 150

5

10

15

Incidence (%)

repo

rting

(%)

prostatecervix

colorectal

0 5 10 15 200

5

10

15

20

Mortality (%)

repo

rting

(%)

breast

melanoma

lungcervix

colorectal

pancreas



https://doi.org/10.1101/388488



12

Reportingquality

News reporting of research is of most value to readers if it accurately conveys the outcomes,

contextandimplicationsoftheresearch.Particularlyinthecontextofseriousdiseaseslikecancer,

accurate reporting is critical for informing decisions on modifying risk, choice of intervention,

understandingprognosis,etc.The levelofconsensusbetweennewsarticlesandrelatedoriginal

researchpapersasamarkerofaccuracyhasbeenthefocusofextensiveresearch.Almostthree

decadesago,Singer [19]proposedananalyticalmodel for scoringaccuracywithelevencriteria,

including issues of incorrect statements, misleading headlines, sensationalisation and

overgeneralisation,andthesehavebecomebenchmarksforevaluatingqualityofmediareportson

research.ModifiedversionsofSinger’smethodhaverevealedpoorcitationpracticeandfrequent

misleading reporting in the news [22-24]. Although the tabloid press is the biggest culprit in

misinterpreting science news, studies have demonstrated that a lack of quality also exists in

broadsheetnewspapers[20,25].Today’sonlineenvironment isthoughttoresult inmanymajor

newsoutletsutilisingthesamesourcesofinformation,potentiallyresultinginanamplifiedspread

of poor quality reporting [36]. Research institutes and funding bodies seeking publicity and

philanthropic supportmay exploit this space as press releases play amajor role in shaping the

contentofmanynewsarticles[21].

WemeasuredreportingqualityinourcohortusingascoringmatrixmodifiedfromSinger[19]and

Taylor[21]forcomparisonbetweenvariousnewsoutlets(Table1).TheNYTimeshadthehighest

averagequality score (12.9),while the lowest average scoreswere seen in theAustraliannews

sources (9.5 and 8.8 forABC and SMH respectively), with reports in The Guardian averaging a

qualityscoreof11.2 (Fig.4).Reports intheNYTimesalsodisplayedthemostconsistentquality

scores, although there was variability in quality observed in all news sources. Where online

readership statistics were available (i.e. for the 20 studies published by ABC), we observed no



https://doi.org/10.1101/388488



13

relationship between reporting quality and readership (not shown). Similarly, we observed no

obviousrelationshipbetweenstudytypeandreportingquality,butthelownumberofsecondary

studiesreportedlimitedthisanalysis.

Figure4.Qualityscoresofnewsreportsoncancerresearch.Eachpointrepresentsthescoreofanindividualnewsreport.Barsindicatethemean+SEMforeachnewsoutlet(n=20foreach).

Nationalbiasinreportingofcancerresearch

While research performed in theUSA dominates scientific output in terms of papers published

[37],reportingonresearchinalocalcontexthasimportantimplicationsforbothconsumersand

scientists alike. Different risk factors may have proportionally different significance for various

audiencesanditiscriticalthatscientistsareabletoreachthemostrelevantaudiencesontopics

of local importance.Forexample,understandingtheroleofUVexposureinmelanomarisk isan

important consideration in Australia. Further, it is important for both scientists and consumers

aliketohaveoutcomesofpubliclyfundedresearchcommunicatedtotaxpayers.

Guardian

NYTimes

SMHABC

0

5

10

15

20

Qua

lity

scor

e

P=0.0373

P<0.0001

P=0.0012

Amberg - Figure 4



https://doi.org/10.1101/388488



14

To analyse national bias in cancer research reporting, we analysed the country of origin of

researchcited ineachnews report -determinedby theprimaryaffiliationof thecorresponding

author of the research publication (where available). At least half of the reports in each news

sourcewerebasedonresearchfromthesamecountryinwhichthenewsorganisationwasbased

(Fig.5).TheGuardian(UKedition)hadthemostdiversenationaloriginofresearchcited,withonly

50%ofreportsbasedonresearchperformedprimarilyintheUK.Incontrast,70%ofreportsinthe

NYTimeswerebasedonresearchstudiesperformedprimarilyintheUSand72.5%ofAustralian

newsreportswerebasedonAustralianresearch(65%and80%bySMHandABC,respectively)(Fig.

5).Viewedfromtheoppositeperspective,thegreatmajority(29/30studies,or97%)ofAustralian

researchstudiesrepresentedinourcohortwereonlyreportedinAustraliannewsoutlets.Inother

words,only1/30Australianstudiesreceivedinternationalcoverage(Fig5).

Figure 5. National bias in reporting of cancer research. Sankey chart showing relationshipsbetweencountryofresearchorigin(left)tocountryinwhichreportingnewsorganisationisbased(right), with bar sizes representing proportional representation. The UK is represented by TheGuardian,USAbyNYTimesandAustraliabySMHandABC.

Amberg - Figure 5



https://doi.org/10.1101/388488



15

Genderbiasinreportingofcancerresearch

Female scientists face a suite of documented biases [38-41], and a number of studies have

established that women are under-represented in news media [42, 43]. More specifically, the

systematicunder-recognitionof femalescientists–knownastheMatildaeffect [44]–hasbeen

demonstrated in science communication,where publications frommale authors are associated

with greater perceived scientific quality [45]. Across our entire cohort of 80 news reports, we

observedasignificantgenderbiasamongseniorauthors,with60%(67/112)of researchstudies

reported havingmale senior authors (Fig 6a).We also observed a significant bias towardmale

expertsbeingquotedinnewsreports,with68%(100/148)ofquotedexpertsbeingmale.Asimilar

trend was observed in individual news outlets, with the exception of the ABC - where equal

representationofmaleandfemaleseniorauthorswasobservedinthestudiesformingthebasisof

news reports (Fig 6b). The bias towards quoting male experts in online reports about cancer

researchwasconsistentacrossindividualnewsoutlets(Fig6c).



https://doi.org/10.1101/388488



16

Figure6.Genderbiasinseniorauthorsandquoted experts in news reports.Distribution of senior authors and expertsacross the entire cohort (n=80), anddistributionofauthors(B)andexperts(C)inindividualnewsoutlets(n=20each).

authors experts0

50

100

Num

ber

overall

male

female

ABCSMH

NYT

Guardian

0

5

10

15

20

25

author

Num

ber

male

female

ABCSMH

NYT

Guardian

0

10

20

30

40

expert

Num

ber

male

female

Amberg - Figure 6

A

B

C

authors experts0

50

100

Num

ber

overall

male

female

ABCSMH

NYT

Guardian

0

5

10

15

20

25

author

Num

ber

male

female

ABCSMH

NYT

Guardian

0

10

20

30

40

expert

Num

ber

male

female

Amberg - Figure 6

A

B

C

authors experts0

50

100

Num

ber

overall

male

female

ABCSMH

NYT

Guardian

0

5

10

15

20

25

author

Num

ber

male

female

ABCSMH

NYT

Guardian

0

10

20

30

40

expertN

umbe

rmale

female

Amberg - Figure 6

A

B

C



https://doi.org/10.1101/388488



17

Discussion

Clear communication of research outcomes in context is important in helping non-specialists

understandtheoftencomplexandchallengingcontentsofscientificpublications.Poorreporting

may hinder informeddecision-making aboutmodifiable risks and treatment selection, generate

falseorunmetexpectations,andunderminetrustinscience[4,5].Inordertobetterunderstand

factors influencing reporting of cancer research we analysed the distribution of study types,

researchsources,reportingquality,genderbias,andnationalbiasinonlinenewsreportsbyfour

majornewsoutletsinUSA,UKandAustraliaoverasix-monthperiodin2017.

Our analysis demonstrated that primary research studies predominate over secondary studies.

Thismayreflectasimilartrendinthepublicationfrequencyofthesestudytypes–bytheirnature,

systematic reviews and meta-analyses are less abundant than primary studies – or the way

institutespromotetheirresearch.Further,evidenceofaskewtowardslowerqualitystudiesinthe

news [33] indicates thatpublicationbiasmightbeanunderlying factor innewspapers favouring

researchwithpoorermethodology.Similartrendshavebeenobservedinotherpublications,with

observational studies on average receiving more attention than both systematic reviews and

randomised controlled trials (RCTs) [34, 35]. This is problematic as the features thus attracting

media attention are often a key driver of hype, confusion and false expectations. Hence, our

observedbiastowardsreportingprimarystudiesmayincreasetheriskofgeneratinghypeabout

cancerresearchvianewsreporting.

Whensearchingfornovelstories,journalistsarelikelytofavourprimaryresearchfindingsdueto

their novelty and frequently larger effect sizes [7] but basic research articles are also themost

susceptible to sensationalisation [36]. While experienced scientists and many journalists likely

know to view these papers as potentially useful pieces in a much greater puzzle, the general

populationmay not have the experience or specialist knowledge to interpret individual reports



https://doi.org/10.1101/388488



18

criticallyinabroadercontext.Despiteblamecommonlybeingattributedtojournalistsandpress

releases,statementsintheresearcharticlesthemselvesareoftenexaggeratedandmaygenerate

hype.Forexample,manyobservationalstudies inhigh impact journalscontainadviceonclinical

practicewithoutmentioningtheneedforconfirmationbyRCTs[26]orfailtodiscusspopulation

biases,smallsamplessizes,ordifficultiesintranslatinganimalstudiestohumans.

The lowoccurrenceof secondary studies reported inour sampleofonline reports is consistent

withpreviousfindingsshowingthatsystematicreviewsrepresentedasmallproportionofmedical

research news [29]. This trend is likely reflected in the frequency of these studies published in

journals, although we could not find any literature that directly investigated the publication

frequency of different study types in medical research. We found a majority of online news

articlesreportingonpeer-reviewedpapers,howeverthismaybepartlyexplainedbyourexclusion

of more general news articles that did not report on a specified study. Previous studies have

highlightedinconsistentqualityandaccuracyofsciencenewsreportingpracticesatmultiplelevels,

rangingfrominstitutionalpressreleasestonewspieces[16,17,21],andfoundthatstudytypes

withpoorermethodologygainmoremedia coverage than researchbasedon strongerevidence

[33-35].Ouranalysisofreportingqualityandstudytypedistributioninonlinenewsisconsistent

withpreviousevidenceofpoorqualityreportingbybroadsheetnewssources[19,20,23]anda

bias towardsprimary research [6,9, 46,47]. Thispredominanceofprimary studies in thenews

increases the likelihood that the general public is left with a distorted perception of cancer

researchandaninaccurateviewofscientificprogress[36].Thisispartlyduetotheincreasedrisk

ofprematurestatementsaboutyetun-replicatedresearchoutcomesandthegreaterlikelihoodof

consequent contradiction or refutation. For example, cancer drug attrition rates are

characteristically high, with only 5% of preclinical anticancer drugs reaching clinical use, and



https://doi.org/10.1101/388488



19

reproducibility of high-impact studies in this area is low [46]. Even therapies tested in phase II

clinicaltrialsrarelysucceedinreachingclinicalpractice[47].

Reportingqualityscoresvariedwithinandacrossthenewssources.Meanqualityscoresindicated

similaritybetweenTheGuardianandNYTimesbutweresignificantlylowerintheAustraliannews

sources.Pointingtoareasforpotentialfutureimprovement,SMHandABCoftenfailedtoconsult

anindependentexpert,providealinktothestudybeingdiscussed,maintainemphasis,andavoid

overgeneralisation.While all sources regularly failed to mention limitations of the study being

reported, Australian news more often omitted such statements. It should also be noted that

availabilityof resources indifferentmediaoutletsmayhavea significant influenceon reporting

qualitybutwewerenotabletomeasurethiseffect.Theonlinenewsmedia’sselectionofresearch

sourceshaspreviously beenevaluatedby testingwhetherAltmetric scores correlate to citation

indicesor journal impact factors,withconflicting results [48,49].Authorshaveargued that this

showsthatthebestresearcharticlesarenotnecessarilytheonesthatmakeittothenews.While

thismaybetrue,methodsbasedonAltmetricscoreswillnotcapturethequantityofnewsarticles

that discuss research without directly referencing a study. Furthermore, other studies have

indicatedthatalternativemetricsmaygenerateinconsistentresultsdependingondatacollection

methods[50],andthatcitationindexisnotnecessarilythebestmarkerofstudyquality[51,52].

Wedidnotobserveany relationshipbetweenquality and readership in reportsby theABC but

unfortunately couldnotexpand thisanalysisasaudiencedatawasnotavailable forothernews

outlets.Thelownumberofsecondarystudiesmadethecohortinthisstudyunderpoweredforan

analysiscomparingtypeandquality.

Previousanalysisof cancer research storieson theBBCwebsite from1998-2006 foundaheavy

focus on breast cancer, followed by lung and prostate cancers [53]. Almost a quarter of news

reports in our cohort did not refer to a specific cancer type, possibly reflecting the strong bias



https://doi.org/10.1101/388488



20

towards reporting of basic research on disease mechanisms and risk. However, an important

underlyingfactorinthistrendmayalsobetheemergingtrendtodefinecancertypebymolecular

ratherthananatomicalclassifiers[54,55].Weobservedacorrelationbetweentherepresentation

of various cancer types (classified by anatomical site) and the relative incidence rates of those

cancertypes(Figure3b),butnorelationshipbetweenreportingfrequencyandrelativemortality

rates (Figure3c).Cervical,melanomaandbreast cancerwereover-represented relative to their

respectivemortality rates,while lung, pancreas, and colorectal cancerwereunder-represented.

Continued public and media interest on the effectiveness of the HPV vaccine Gardasil, and

changes in screening practices may be a relevant consideration in the over-representation of

reports focussing on cervical cancer. Over or under representation of different cancer types in

research reporting can skew public awareness of risk factors and may also drive inequities in

publicandphilanthropicfundingofresearchdirectedasspecificcancertypes.

Weobservedastrikingnationalbias,wherenewsoutletsweremorelikelytoreportonresearch

performedinthesamecountrytheywerebased.Ifthedistributionofstudiesreportedinallnews

outletsmirroredglobal researchoutput, thevastmajorityof reportswouldbebasedonstudies

performed in the US, as they dominate in terms of papers published [37]. However, a perfect

reflectionofglobaloutputisnotnecessarilydesirable.Itmaybedifficultforlocalscientiststoget

media attention outside their own country, and institutional press offices may have stronger

contact networkswith local reporters. Reporting research of relevance to distinct geographical

areas (e.g. epidemiological investigations on specific populations) may be very important in

informingthepublicwithregardstolocalriskfactorsandoutcomesofpubliclyfundedresearch.

Hence,thepredominanceofepidemiologicalstudiesinnewsreportsisonepossiblecontributorto

theobservednationalbias..Conversely,prioritisingreportingonlocalresearchmeansthepublic



https://doi.org/10.1101/388488



21

maynotgetaccesstoimportantinformationfrombroadersources,althoughthemodernonline

environmentputsglobalnewswithinreachofthemajorityofpeople.

TheMatilda effect, which describes the systematic under-recognition of female scientists, has

beenextendedtosciencecommunication,wheregreaterperceivedscientificqualityisassociated

with publications frommale authors [44, 45]. Further, under-representationofwomen in news

mediamoregenerallyiswellestablished[42,43].Weobservedastrikinggenderbiasinbothstudy

selection and reporting in our cohort,with themajority of reports based on studieswithmale

senior authors and quoting male experts. This bias has potential to compromise high-quality

coverageofresearchby limitingdiversityofopinion,and likelyservestoreinforcingstereotypes

and further entrench gender inequity among researchers by providing public visibility and

recognitionpredominantlytomalescientists.Thesuiteofbiasesfacedbyfemalescientistsiswell

documented [38-41], and the gender bias in news reports likely reflects an underlying

predominance ofmen among the ranks of senior scientists. The possibility of other underlying

biases can’t be excluded, including differences in the availability and/or willingness of male

expertstospeaktojournalists.Regardless,ourdatahighlightsaneedforjournalists,scientistsand

institutes to significantly improve efforts to ensure equal representation of male and female

scientistsinnewsreportsoncancerresearch.

Limitations

As cancer research output is not expected to fluctuate notably throughout the year, it was

assumedthatasix-monthsampleperiodlimitedto20reportsfromeachofthefournewsoutlets

would provide a representative cohort to analyse. An exception was conference-based news

reports,whichpeakedat the timeofamajorcancerconference in June (ASCOannualmeeting,

https://am.asco.org/).InallnewsoutletsapartfromTheGuardian,the20reportscomprisedthe

majority of relevant reports within the selected time frame and should thus be considered



https://doi.org/10.1101/388488



22

representative samples of each source. Although the size of our cohort limits a comprehensive

representationofnationalreportingtrends,thechosensourcesareallmajornewsoutletsintheir

respectivecountriesandso likelyprovidea reasonable indicationofbroader trends. Itwasalso

assumedthatallstudiescouldbeclassifiedaccordingtothemodeloutlinedbyRöhrigetal.[56].

Secondarystudiesweretooraretoallowanalysisofanypotentialrelationshipbetweenstudytype

andreportingquality,whichshouldbeinvestigatedinlargerdatasets.

Recommendationsandconclusions

Thequalityindicatorsmeasuredinourcohortprovideusefulguidelinesforjournaliststoconsider

in providing the most informative and accurate reporting of research. These are particularly

relevant tominimising the potential for hyperbole, providing an objective account of research

outcomes and implications, and in assisting readers to critically assess the report. For example,

reports in the Australian news outlets (SMH and ABC) often failed to consult an independent

expert, provide a link to the research study, and avoid overgeneralisation. While all sources

regularly failedtomention limitationsof thestudybeingdiscussed,Australiannewsmoreoften

omittedsuchstatements.Theselimitationsmayreflectlimitedtimeand/orresourcesofreporters

or a trend towards having fewer specialist reporters in Australianmedia, butwe are unable to

quantify these in our dataset.Our data highlight the importance of consideringwhich research

types are selected for coverage. Acknowledging the importance of journalists providing

independent and novel context, interpretation and insight to individual stories, a better

representationofthecurrentstateincancerresearchwouldbeachievedbyattemptingtocovera

more balanced proportion of primary and secondary studies, from national and international

sources.Scientistsandjournalistsshouldalsotakecaretomentionthelimitationsofnovelideasin

researchandrefrainfrompresentingfindingsascertain.



https://doi.org/10.1101/388488



23

Conversely, when communicating with the news media, scientists should be conscious of the

possiblediscrepancybetween impact inthescientificcommunityandamongthegeneralpublic.

Limitations and uncertainties should always be highlighted.Where possible, readers should be

made aware ofwhat type of study is being reported on,whether it is peer-reviewed and how

strong the supporting evidence is. These are important considerations, as even prestigious

newspapersdonotnecessarilygiveafairrepresentationofcurrentprogressincancerresearch.

Asfarasweknow,thisisthefirstcombinedanalysisofstudytypedistribution,reportingquality,

and other biases in cancer research reporting. These data highlight the presence of significant

biasesandprovideabasisforimprovingtheselectionofstudiesbeingselectedformediacoverage,

and theway those studies are reported. Future analyses should build on the findings reported

herebyincorporatingthelong-termoutcomesandimpactofthestudiesthatappearinthenews

media.Itwouldalsobeusefultoevaluatetowhatextentcorrectionsfollowinthenewsafterone

of thesestudieshavebeen refutedoradeclared ‘breakthroughdrug’ fails to reach themarket.

Further, analyses of relationships between readership, study type and reporting quality would

offerinsightintohowdemand-driventhesebiasesmaybe.

Accurate,contextualreportingofcancerresearch is imperative inhelpingthepublicunderstand

complexandchallengingscienceandappreciatetheoutcomesofpubliclyfundedresearch,avoid

underminingtrustinscience,andassistinformeddecision-making.



https://doi.org/10.1101/388488



24

Acknowledgements

TheauthorswishtothankDrAdamDunnandDrJonathonWebbforthoughtfulandconstructive

discussionandcommentsondraftmansucripts.



https://doi.org/10.1101/388488



25

References

1. Ferlay,J.,etal.,Cancerincidenceandmortalityworldwide:sources,methodsandmajorpatternsinGLOBOCAN2012.IntJCancer,2015.136(5):p.E359-86.

2. Stewart,B.W.,etal.,Worldcancerreport2014.2014,Lyon,FranceandGeneva,Switzerland:InternationalAgencyforResearchonCancer,WHOPress.xiv,630pages.

3. Collaborators,G.B.D.R.F.,Global,regional,andnationalcomparativeriskassessmentof79behavioural,environmentalandoccupational,andmetabolicrisksorclustersofrisks,1990-2015:asystematicanalysisfortheGlobalBurdenofDiseaseStudy2015.Lancet,2016.388(10053):p.1659-1724.

4. Borup,M.,etal.,Thesociologyofexpectationsinscienceandtechnology.TechnologyAnalysis&StrategicManagement,2006.18(3-4):p.285-298.

5. Caulfield,T.,Biotechnologyandthepopularpress:hypeandthesellingofscience.TrendsBiotechnol,2004.22(7):p.337-9.

6. Gonon,F.,etal.,Whymostbiomedicalfindingsechoedbynewspapersturnouttobefalse:thecaseofattentiondeficithyperactivitydisorder.PLoSONE,2012.7(9):p.e44275.

7. Ioannidis,J.A.andO.A.Panagiotou,Comparisonofeffectsizesassociatedwithbiomarkersreportedinhighlycitedindividualarticlesandinsubsequentmeta-analyses.JAMA,2011.305(21):p.2200-2210.

8. Michiels,S.,S.Koscielny,andC.Hill,Predictionofcanceroutcomewithmicroarrays:amultiplerandomvalidationstrategy.TheLancet,2005.365(9458):p.488-492.

9. Schoenfeld,J.D.andJ.P.Ioannidis,Iseverythingweeatassociatedwithcancer?Asystematiccookbookreview.TheAmericanJournalofClinicalNutrition,2013.97(1):p.127-134.

10. Aggarwal,A.,R.Batura,andR.Sullivan,Themediaandcancer:educationorentertainment?AnethnographicstudyofEuropeancancerjournalists.Ecancermedicalscience,2014.8:p.423.

11. Dumas-Mallet,E.,etal.,Poorreplicationvalidityofbiomedicalassociationstudiesreportedbynewspapers.PLOSONE,2017.12(2):p.e0172650.

12. Earp,B.D.,Theunbearableasymmetryofbullshit.HealthWatchNewsletter,2016.101:p.4-5.

13. Cohen,E.L.,etal.,Cancercoverageingeneral-audienceandBlacknewspapers.HealthCommun,2008.23(5):p.427-35.

14. Jensen,J.D.,etal.,Publicestimatesofcancerfrequency:cancerincidenceperceptionsmirrordistortedmediadepictions.JHealthCommun,2014.19(5):p.609-24.

15. Jensen,J.D.,etal.,Makingsenseofcancernewscoveragetrends:acomparisonofthreecomprehensivecontentanalyses.JHealthCommun,2010.15(2):p.136-51.

16. Entwistle,V.andM.Hancock-Beaulieu,HealthandmedicalcoverageintheUKnationalpress.PublicUnderstandingofScience,1992.1(4):p.367-382.

17. Lee,C.-J.,etal.,ComparinglocalTVnewswithnationalTVnewsincancercoverage:anexploratorycontentanalysis.JournalofHealthCommunication,2014.19(12):p.1330-1342.

18. Clarke,J.N.andM.M.Everest,Cancerinthemassprintmedia:Fear,uncertaintyandthemedicalmodel.SocialScience&Medicine,2006.62(10):p.2591-2600.

19. Singer,E.,Aquestionofaccuracy:howjournalistsandscientistsreportresearchonhazards.JournalofCommunication,1990.40(4):p.102-116.

20. Wilson,A.,etal.,Mediareportingofhealthinterventions:signsofimprovement,butmajorproblemspersist.PLoSONE,2009.4(3):p.e4831.



https://doi.org/10.1101/388488



26

21. Taylor,J.W.,etal.,Whenmedicalnewscomesfrompressreleases—acasestudyofpancreaticcancerandprocessedmeat.PloSone,2015.10(6):p.e0127848.

22. Brechman,J.,C.-j.Lee,andJ.N.Cappella,Lostintranslation?ScienceCommunication,2009.30(4):p.453-474.

23. Brechman,J.M.,C.-j.Lee,andJ.N.Cappella,Distortinggeneticresearchaboutcancer:frombenchsciencetopressreleasetopublishednews.JournalofCommunication,2011.61(3):p.496-513.

24. Moyer,A.,etal.,Accuracyofhealthresearchreportedinthepopularpress:breastcancerandmammography.HealthCommunication,1995.7(2):p.147-161.

25. Wilson,A.,etal.,Doesitmatterwhowritesmedicalnewsstories?PLoSMedicine,2010.7(9):p.e1000323.

26. Prasad,V.,etal.,Observationalstudiesoftenmakeclinicalpracticerecommendations:anempiricalevaluationofauthors'attitudes.JournalofClinicalEpidemiology,2013.66(4):p.361-366.

27. Jüni,P.,etal.,Thehazardsofscoringthequalityofclinicaltrialsformeta-analysis.JAMA,1999.282(11):p.1054-1060.

28. Stang,A.,CriticalevaluationoftheNewcastle-Ottawascalefortheassessmentofthequalityofnonrandomizedstudiesinmeta-analyses.EuropeanJournalofEpidemiology,2010.25(9):p.603-605.

29. Lai,W.Y.Y.andT.Lane,Characteristicsofmedicalresearchnewsreportedonfrontpagesofnewspapers.PLoSOne,2009.4(7):p.e6103.

30. Rohrig,B.,etal.,Typesofstudyinmedicalresearch:part3ofaseriesonevaluationofscientificpublications.DtschArzteblInt,2009.106(15):p.262-8.

31. Welfare,A.I.o.H.a.,CancerinAustralia.2017.32. Contopoulos-Ioannidis,D.G.,E.E.Ntzani,andJ.P.Ioannidis,Translationofhighlypromising

basicscienceresearchintoclinicalapplications.TheAmericanJournalofMedicine,2003.114(6):p.477-484.

33. Selvaraj,S.,D.S.Borkar,andV.Prasad,Mediacoverageofmedicaljournals:dothebestarticlesmakethenews?PLoSONE,2014.9(1):p.e85355.

34. Wang,M.T.M.,etal.,Mediacoverage,journalpressreleasesandeditorialsassociatedwithrandomizedandobservationalstudiesinhigh-impactmedicaljournals:acohortstudy.PLoSONE,2015.10(12):p.e0145294.

35. Haneef,R.,etal.,Factorsassociatedwithonlinemediaattentiontoresearch:acohortstudyofarticlesevaluatingcancertreatments.ResearchIntegrityandPeerReview,2017.2(1):p.9.

36. Granado,A.,Slavestojournals,serfstotheweb:TheuseoftheinternetinnewsgatheringamongEuropeansciencejournalists.Journalism,2011.12(7):p.794-813.

37. Wagner,C.S.andK.Jonkers,Opencountrieshavestrongscience.Nature,2017.550:p.32-33.

38. Putnam,K.E.,The5BiasesPushingWomenOutofSTEM.HarvardBusinessReview,2015.93(6):p.22-22.

39. Shen,H.,Inequalityquantified:Mindthegendergap.Nature,2013.495(7439):p.22-4.40. Helmer,M.,etal.,Genderbiasinscholarlypeerreview.Elife,2017.6.41. Lariviere,V.,etal.,Bibliometrics:globalgenderdisparitiesinscience.Nature,2013.

504(7479):p.211-3.42. Jia,S.,etal.,WomenAreSeenMorethanHeardinOnlineNewspapers.PlosOne,2016.

11(2).



https://doi.org/10.1101/388488



27

43. Ross,K.andC.Carter,Womenandnews:Alongandwindingroad.MediaCulture&Society,2011.33(8):p.1148-1165.

44. Rossiter,M.W.,TheMatthew-MatildaEffectinScience.SocialStudiesofScience,1993.23(2):p.325-341.

45. Knobloch-Westerwick,S.,C.J.Glynn,andM.Huge,TheMatildaEffectinScienceCommunication:AnExperimentonGenderBiasinPublicationQualityPerceptionsandCollaborationInterest.ScienceCommunication,2013.35(5):p.603-625.

46. Ocana,A.,etal.,Preclinicaldevelopmentofmolecular-targetedagentsforcancer.NatRevClinOncol,2011.8(4):p.200-209.

47. Arrowsmith,J.,Trialwatch:PhaseIIfailures:2008–2010.NatRevDrugDiscov,2011.10(5):p.328-329.

48. Chapa,J.,Z.Haq,andA.S.Cifu,Comparativeanalysisofthefactorsassociatedwithcitationandmediacoverageofclinicalresearch.Scientometrics,2017.112(3):p.1271-1283.

49. O'Connor,E.M.,etal.,Newsworthinessvsscientificimpact:arethemosthighlycitedurologypapersthemostwidelydisseminatedinthemedia?BJUInternational,2017.

50. Zahedi,Z.,R.Costas,andP.Wouters,Howwelldevelopedarealtmetrics?Across-disciplinaryanalysisofthepresenceof‘alternativemetrics’inscientificpublications.Scientometrics,2014.101(2):p.1491-1513.

51. Ioannidis,J.A.,Contradictedandinitiallystrongereffectsinhighlycitedclinicalresearch.JAMA,2005.294(2):p.218-228.

52. Berghmans,T.,etal.,Citationindexesdonotreflectmethodologicalqualityinlungcancerrandomisedtrials.AnnOncol,2003.14.

53. Lewison,G.,etal.,Howdothemediareportcancerresearch?AstudyoftheUK'sBBCwebsite.BrJCancer,2008.99(4):p.569-76.

54. Heim,D.,etal.,Cancerbeyondorganandtissuespecificity:next-generation-sequencinggenemutationdatarevealcomplexgeneticsimilaritiesacrossmajorcancers.IntJCancer,2014.135(10):p.2362-9.

55. Hoadley,K.A.,etal.,Cell-of-OriginPatternsDominatetheMolecularClassificationof10,000Tumorsfrom33TypesofCancer.Cell,2018.173(2):p.291-304e6.

56. Röhrig,B.,etal.,Typesofstudyinmedicalresearch.DtschArzteblInternational,2009.106(15):p.262-268.



https://doi.org/10.1101/388488


Date post:	22-Sep-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Cancer In The News: Bias And Quality In Media Reporting Of ... · Cancer In The News: Bias And...

Documents