ENHanCE Position Paper #1 – Quantitative and Qualitative Approaches to the Prioritisation of Diseases

Introduction

The prioritisation of diseases is an important process for international, national and regional agencies

charged with disease surveillance and implementation of disease management and control, given

limited resources. It is also important for funders of research; so that they can identify the most

important areas for investment. However, the application of risk analysis as a prioritisation tool often

involves evaluating a large number of hazards where it is not feasible to conduct an in-depth

assessment of all pathogens. Ranking of communicable diseases and zoonoses into prioritisation lists

can be undertaken using several different approaches: qualitative (Valenciano et al., 2001), semi-

quantitative (Krause et al., 2007, McKenzie et al., 2007, Cardoen S. et al. 2009) and quantitative

(Fosse et al., 2008). In qualitative studies, estimation of parameters and risks is done using words

(relatively high, low…), whereas in quantitative work, numbers or probabilities are used. Semi-

quantitative studies lie somewhere in the middle, using a mix of qualitative terms and/or signs and

numbers. Qualitative and semi-quantitative approaches are criticised due to their potential

subjectivity and the large amount of resources they use, respectively. However, most quantitative

methods require some input of expert opinion, giving them a degree of subjectivity too.

The objective of this position paper was to illustrate and review existing methodologies for

prioritisation of diseases in order that attendees to the ENHanCE project agency meeting would be

able to identify the approaches most useful to them for future planning. Our aim within the ENHanCE

project itself is to build a repeatable and transparent future approach to prioritisation of diseases

using a simple but robust algorithm. The results presented were obtained from a literature and web

search using keywords such as prioritisation, animal disease, zoonoses and human disease.

As work within the Discontools project, http://www.discontools.eu/home/index (IFAH, Europe 2009)

had already scanned worldwide for existing methodologies of prioritisation, this review starts by

presenting its work and has extracted its table of comparison of the methodologies for the summary

and extended it by adding eight more articles and reports.

Whilst reading the position paper, the agencies were asked to think about the questions below.

Which approach to disease prioritisation is most useful to your agency: qualitative, semi-

quantitative and quantitative?

Is examination of endemic or exotic pathogens or perhaps both most useful to you?

Are you most interested in pathogens at high risk (of change) or which have a high impact

(on society)?

Would independent prioritisation by us, use of another prioritisation tool, or aggregation of

other prioritisation tools thus creating a generic tool be most useful to you?

Do you prefer the use of weighting or scoring by expert opinion or objective measures only?

Following engagement with stakeholders, we feel within the ENHanCE project that when

undertaking prioritisation of diseases we should:

Start prioritising diseases using a large list and qualitative approach, thereafter tailoring our

approach further using semi-quantitative and quantitative methods on smaller lists of

diseases.

Include both endemic and exotic diseases within the lists of diseases to be prioritised.

Dependent upon whom the main stakeholders are, prioritise on pathogens at high risk of

change or which have a high impact on society.

Try to undertaken independent prioritisation of diseases within the ENHanCE project, rather

than using other prioritisation tools, or an aggregation of other prioritisation tools.

Use a combination of initially objective measures followed by expert opinion within formal

risk assessment.

ENHanCE Position Paper #1

1. The Discontools project

Existing methodologies for prioritisation of diseases were reviewed within the Discontools project,

with four articles and four projects examined. Out of the eight methods described, three (CVOs,

Veterinary Surveillance Prioritisation Project, McKenzie) followed a traditional risk assessment

approach (including risk identification) quite closely, with the criteria used within each relating to the

different steps of hazard identification, probability of the hazard occurring (release and exposure

assessments) and consequences assessment (with the scoring of impacts). The other five methods

used multi-criteria decision tools, which were also based on risk. All assigned scores using a semi-

quantitative approach, a full quantitative approach being mentioned but not explained for ‘priority

setting of emerging zoonoses’. Table 1 is a comparison of the eight prioritisation methodologies

described within Discontools. Table 2 illustrates a further eight methodologies added for the

ENHanCE project. Table 3 summarises the outputs of these disease prioritisation exercises. The

Discontools project has developed a prioritisation model for 45 diseases, based on 28 criteria split

into six modules: disease knowledge, impact on wider society, impact on public health, impact on

trade, animal welfare and control tools. Within the model, each criterion is scored and the modules

are thereafter weighted.

2. Qualitative approaches

In addition to some studies reviewed by Discontools, we have identified three further qualitative

approaches to disease prioritisation.

Dufour and others (2006)

a. Criteria for inclusion of diseases

A qualitative approach was used to examine the OIE list of diseases

(http://www.oie.int/Eng/maladies/en_classification.htm). Animal diseases likely to be influenced by

climate (either through an increase of incidence and/or of geographical extension) were identified,

and examination of their mode of transmission for risks associated with arthropod vectors, molluscs,

wild vertebrates, and other risks (for instance climate related changes in human behaviour) was

made, omitting those which could not be classified into one of these categories.

b. Ranking method

Risks and possible evolution factors dependent upon global warming were then identified, and

qualitatively assessed from null (the evolution of the epidemiological situation has nothing to do with

climate warming), to high (the evolution of the epidemiological situation depending upon global

warming is very probable), using the AFSSA qualitative risk evaluation matrix. All the diseases

assessed as “null” were further eliminated.

For the 20 remaining diseases, the probability of occurrence combined with the impact of the disease

was assessed qualitatively using the same qualifiers as previously, under 3 headings:

- health consequences for the animal (mean severity)

- public health consequences for zoonoses (prevalence and mean gravity)

- collective economic consequences in animals

The conclusion of the work was that six diseases should be considered as a priority amongst which

five were vector-borne.

Eger et al. (2009)

a. Criteria for inclusion of diseases

The inclusion of the foci of international organisations (top ten of their priorities), scientific journals,

national statistics, and experiences was the base for a pre-selection of human diseases. An additional

discriminator was to not take into account diseases for which management programmes existed or

were planned, bearing in mind the scope of health insurance activities. As a result, a list of 14

diseases was compiled.

b. Ranking method

A catalogue of criteria was developed and summarised in 6 dimensions: political criteria,

economic/legal criteria, structural/management criteria, medical criteria, social criteria, and

integrated care experience. A detailed literature review was then conducted, completed using grey

literature and expert interviews. Results for criteria were presented in qualitative terms, for example

average, clear, low, high, rising, existing.

Finally, the scientific board and additional experts were invited to a workshop to discuss the process,

evaluate the outcomes and conduct a prioritisation of diseases according to their expertise and from

the perspective of a national social security agency. As valuation of the goals and criteria seemed

necessary to make the discussion and process more manageable, four top-priority goals derived from

the criteria were identified: Incidence/effect, Should be/As is-ratio, Long term perspective,

Complexity/feasibility.

An evaluation and weighting of the 14 diseases was then conducted through a discussion process,

which aimed at unanimous consensus of all experts. Diseases were classified into three levels of

prioritisation after agreement had been reached on their weighting: high/strong, average, low/weak.

InVS (2006)

a. Criteria for inclusion of diseases

Experts in veterinary and human public health together chose inclusion criteria for diseases, using

them to build a list of non food-borne zoonoses. 37 zoonoses were considered in the first-step of the

process, based on lists A and B of the OIE. Three types of risk were considered, not taking into

account political criteria and social perceptions:

- human public health: incidence, prevalence, mortality, lethality…

- animal health: existence of a surveillance system, existence of methods and

means to control the disease

- risk related to the international context/environment and economical

impact of diseases

b. Ranking method

Experts filled in individually a questionnaire per disease and the results were thereafter discussed by

all. The qualitative assessment was transformed into scores in the final step, and 11 priority diseases

and 9 important diseases were identified.

Conclusion to qualitative approaches

While Dufour et al. (2006) and Eger et al. (2009) based all their prioritisation on using qualitative

qualifiers, InVS instead chose to use a numerical analysis at the end of their process. The qualitative

approach enabled the selection of a few diseases from amongst a large list with a relatively simple

and quick method which was easy to communicate to decision makers but quite subjective.

However, difficulty in the choice of the right qualifier was also raised by Dufour and others (2006),

but this was solved by assessing each disease comparatively to the others, and not in an absolute

manner.

1. Semi-quantitative approaches

Cardoen et al. (2009)

a. Criteria for inclusion of diseases

An exhaustive list of food and water-borne zoonoses was established based upon literature review

and on the opinion of a working group of scientific experts.

b. Ranking method

An evidence based semi-quantitative methodology was developed. Scores were given according to 5

criteria related to public health (severity and occurrence in humans), animal health (severity of

disease coupled with economic consequences and occurrence in animals), and food (occurrence in

food). The impact of each zoonotic agent on the five criteria was scored on a scale from 0 to 4. For

each zoonosis an individual score per criterion together with its standard error, which reflected

expert heterogeneities, was calculated. The average total scores were calculated using a clustered

bootstrap. For each zoonotic agent a total score (from 0 to 20 points) was calculated as the sum of

the bootstrapped average scores per criterion. Independently, the relative importance of each of the

five criteria was weighted by seven food-chain risk managers. The zoonotic agents were ranked

based on the overall weighted scores and were grouped into four statistically different levels of

importance, taking into account the combined input from risk assessors and risk managers.

Table 1. Comparison of the eight prioritisation methodologies reviewed by the DISCONTOOLS project.

Author(s) Dufour et al. Eger et al. InVS Cardoen et al. Perkins et al. Dixit & Anand Fosse et al.

Country France Austria France Belgium Australia Rajasthan France

Year 2004 2006 2006 2008? 2007 2004 2008

Organisation Afssa Viennese health Insurance+ medical university

InVS Federal egency for the safety of food chain

ACIAR (government)

Government of Rajasthan

ENVN

Priorisation objective

Surveillance Planning for the Austrian health system

Planning actions for InVS

Risk manager recommendations

Assist Indonesia for control and prevention

Surveillance Surveillance

Number of diseases

45 14 37 51 agents 13 ? 35 agents

Number of criteria

6 6 >16 5 7 3 6

Scoring system

No No No 4 tiered 5 tiered - Yes

Weighting applied

- Yes ? Yes Yes - Yes

Methodology of collecting opinion

Bibliography and expert opinion

Literature + workshop with experts

Workshop with experts

Working group Literature and national statistics

Data from directorate of health

Literature

Number of participants

11 ? ? 35 - 2 1

Type of participants

Academia/research, government stakeholders

All stakeholders Government stakeholders, academia/research

Scientific institutions or universities

- government research

Type of approach

Qualitative Qualitative Qualitative Semi-quantitative

Semi-quantitative

Semi-quantitative

Quantitative

Table 2. The eight prioritisation methodologies added for the ENHanCE project

Diseases or pathogens

RIV

M (

2006

)

McK

enzi

e et

al. (

2007

)

FAO

/OIE

(20

04)

Doh

erty

(20

00)

WH

O (

2006

)

Duf

our

et a

l.

(200

6)

Eger

et

al.

(200

9)

InV

S (2

001)

Card

oen

et a

l.

(200

9)

Perk

ins

et a

l.

(200

7)

Dix

it &

Ana

nd

(200

4)

Foss

e et

al.

(200

7)

Acute flaccid para lys is IAdenovirus IAfrican Horse s ickness I IAlveolar hydatid (echinococcus multilocularis ) I IAnthrax I I IASF IBatrachochytrium dendrobatidis IBluetongue IBrucella dephini, Brucella mani IBrucel los is (B. melitensis ) I I I I IBSE agent/ Creutzfeld Jacob disease I I ICampylobacter spp I I I ICBPP* ICetacean morbillivirus IChickenpox ICholera I IClamydia psittaci IClostridium botulinum I I ICrimean Congo Hemorrhagic fever ICryptosporidium parvum I ICSF ICyclosporias is ICysticercos isdiphteria I IDuck plague IDuck vi rus hepati tis IE. Coli STEC IE. Coli VTEC I IEpizootic haematopoietic necros is Ifasciola hepatica IFMD* IFoodborne diseases IFurunculos is IGenita l clamydia IGiardia intestinalis I IGonorrhea Igroup B streptococcal disease in neonates IHantavirus pulmonary syndrome Ihepati tis A I IHepati tis B I Ihepati tis C I IHerpes vi rus IHIV/AIDS I IHPAI I I IInfluenza I I IInvas ive group A streptococcal IInvas ive meningococcal disease I IInvas ive pneumococcal disease IJapanese encephal i tis I I ILeprosy I ILeptospira interrogans australis ILeptospiros is I I I IListeria monocytogenes I I ILyme disease IMalaria I Imeas les I IMumps IMycobacterium spp I I I INew world screwworm INewcastle disease INipah vi rus INorovirus IOld world screwworm IPacheco's disease IParamyxoviruses - bats IPasteurella multocida serogroup A IPasteurel los is Ipertuss is I IPlague I I Ipol iomyel i tis I IPPR IPsittacine circovirus IPs i ttacine pox IPs i ttacos is IQfever I IRabies I I I I I

Diseases or pathogens (cont.)

RIV

M (

2006

)

McK

enzi

e et

al. (

2007

)

FAO

/OIE

(20

04)

Doh

erty

(20

00)

WH

O (

2006

)

Duf

our

et a

l.

(200

6)

Eger

et

al.

(200

9)

InV

S (2

001)

Card

oen

et a

l.

(200

9)

Perk

ins

et a

l.

(200

7)

Dix

it &

Ana

nd

(200

4)

Foss

e et

al.

(200

7)

Rinderpest - Stomati tis/enetri ti s IRoss River Fever IRotavirus I Irubel la I IRVF* I ISalmonella spp I I I I I ISheep pox*/goat pox* IShigel los is I ISindbis vi rus IStaphylococcus aureus ISteptococcus spp ISyphi l i s I ITetanus I IToxoplasmosis (Toxoplasma gondii ) I I I ITularémia ITyphoid I IUni locular hydatid (echinococcus granulosus ) I IVenezuelan Equine Encephalomyel i ti s IVira l haemorrhagic septicemia IVisceral leishmanios is IWest Ni le Virus I I I IYel low fever I IYersinia enterolitica I I

* diseases for which trend analys is and predictions wi l l be emphas ized

Table 3. A summary of the outputs of previous disease prioritisation exercises.

Perkins et al. (2007)

a. Criteria for inclusion of diseases

An initial list of candidate diseases was compiled through a combination of literature review, web

searching and through contacts with a variety of individuals who had been involved in animal health

activities in Indonesia over a number of years. The criteria for inclusion in this initial list included:

- zoonotic disease

- measurable impact on both livestock and humans

- either evidence or suspicion that the disease was present in Indonesia

From a list of 13 diseases, six were identified as high priority. These did not include diseases for

which insufficient data or information was available to estimate whether the disease was present or

to attempt to assess their impact, or diseases assessed as being unlikely to have any adverse impact

on livestock health or production.

b. Ranking method

Three impact assessment methods were used to rank the list of diseases:

- modified ILRI scoring system (see Box 1), with two different results: socio-

economic impacts that aimed to assess the impact on livestock, and an

adjusted zoonotic score that aimed to assess the impact of diseases on

human health

- DALY: Disability Adjusted Life-Year

- Economic losses

The overall rank was derived from the arithmetic average of the ranks of the four component

assessments and assumed equal weighting of these four methods.

Box 1. The ILRI approach (Described in Perry et al. (2002))

Specific criteria are developed for measuring impacts and each criterion is scored on a scale of 0-5 with 5

representing the most severe type of impact. Scores are then combined to produce a single composite index.

The criterion specific scores (7) are then combined using weighting to produce an overall socio-economic

impact score. This can be used as an un-adjusted estimate of the relative importance of the different diseases

for smallholders.

For the zoonotic score, public health impacts are assessed based on the incidence of disease in livestock, the

human population at risk and the severity of the disease in affected individuals (two criteria). In some cases

scores can be adjusted using weighting.

Dixit & Anand (2004)

a. Criteria for inclusion of diseases and method

Data from 1998-2002 were obtained from the Directorate of Health, Govt. of Rajasthan and analysed

year by year, comparing incidence scenario and DALYs.

Conclusion to semi-quantitative approaches

Many of the semi-quantitative approaches used are hardly comparable. Perkins (2007) and Dixit &

Anand (2004) used similar approaches and settings. Cardoen et al. (2009), however, followed a multi-

criteria decision model with a semi-quantitative approach. They suggested that this circumvents the

problems usually encountered in quantitative methods, such as a lack of data and in qualitative

methods such as subjectivity and unreliability (Cox et al., 2005). Their approach is novel, because it

includes the effect of the policy priorities of risk managers within the final ranking, rather than only

the scientific expertise of risk evaluators (scientific experts).

2. Quantitative approaches

Fosse et al. (2008)

a. Criteria for inclusion of pathogens

Based on a large literature review, biological hazards potentially transmitted to pork consumers were

sorted according to analytical, geographical and historical criteria, in order to identify currently

established biological hazards for European consumers of pork.

b. Ranking method

The hierarchy of hazards could be calculated according to risk scores (see Box 2), considered as cross

functions of the incidence of human cases attributed to pork consumption and the calculated

severity scores of these cases. A ratio for non-control of hazards during and after meat inspection

(the mean incidence of human cases attributable to pork consumption divided by the mean

prevalence of hazards on pork carcasses) was calculated, and comparison between non-control ratio

and risk scores was made to identify the hazards for which new meat control methods should be

considered a priority.

Box 2. The calculation of the risk scores

Risk score: Rλ=Ipork*Sλ where:

Pcar: mean rate of prevalence on pork carcasses

H: rate of hospitalisation: number of hospitalised people among sick people

L: lethality of human clinical cases due to biological hazards: number of deceased people among sick people

Sλ: clinical severity of symptoms induced in humans=H+λL, with λ term to strengthen the epidemiological

weighting of hazards which may be lethal.

Ipork: mean incidence rate=I*PAP

PAP: pork attributable proportion= npork/ntotal with npork and ntotal for one given hazard the number of human

cases due to pork consumption and the total number of human cases due to food consumption

Conclusion to quantitative approaches

This single study used purely a quantitative approach. Interestingly, the results were very similar in

terms of the ranking of the pathogens to that of Cardoen et al. (2009), which used a semi-

quantitative methodology. Other attempts at quantitative analyses may perhaps not have been

made because such analyses are very data rich.

References

Cardoen, S., Van Huffel, X. et al. (2009). Foodborne Pathogens and Disease, 6(9): pp1083-1096.

Cox, L.A. & Babayer, D. (2005). Risk Analysis, 25(3): pp651-662.

Defra (2006). Approaches to the prioritisation of resources: a brief review of selected public sector organisations in the UK

and abroad. (www.defra.gov.uk/foodfarm/farmanimal/diseases/vetsurveillance/documents/prioritisation-resources.pdf).

Discontools (2009). Approaches to the prioritization of diseases: a worldwide review of existing methodologies for health

priority settings.

(www.discontools.eu/documents/1207_Draft%20Review%20of%20existing%20methodologies%20for%20priority%20settin

gs.pdf).

Dixit, A.K. & Anand P.K. (2004). Retrospective analysis and prospective prioritisation of disease burden in Rajasthan. Indian

Council of Medical Research. (www.icmr.nic.in/annual/2004-05/dmrc/ar45_4b.pdf).

Dufour, B., Moutou, F., Hattenberger, A.M. & Rodhain, F. (2008). Rev. sci. tech. Off. int. Epiz., 27(2): pp541-550.

Eger, K., Gleichweit, S., Rieder, A., Stein, K.V. (2009). International Journal of Integrated Care, 9: e91.

Fosse, J., Seegers, H. & Magras, C. (2008). Veterinary Research,39:01.

InVS: Definition des priorités dans le domaine des zoonoses non alimentaires BEH Juillet 2006 n°27-28

Miller, G.E. (1990). Academic Medicine, 65(9suppl): ppS63-67.

Perkins, N., Patrick, I., Patel, M. & Fenwick, S. (2007). Assessment of zoonotic diseases in Indonesia. Australian Centre for

International Agricultural research. (http://www.aciar.gov.au/publication/FR2008-01).

Valenciano M. et al. Definition des priorités dans le domaine des zoonoses non alimentaires 2000-2001. Rapport de l’InVS, AFSSA, ENVN, La direction générale de la santé, le centre hospitalier universitaire Cochin et la cellule interregionale d’epidemiologie EST. (http://www.invs.sante.fr/publications/2002/def_priorite_zoonoses/index.html).

Disease prioritisation – Position paper #1

Questions

Which approach to disease prioritisation is most useful to your agency: qualitative/semi- quantitative/quantitative?

Is examination of endemic or exotic pathogens, or both, more useful to you?

Are you most interested in pathogens at high risk (of change) or those which have a high impact (on society)?

Would independent prioritisation by us, use of another prioritisation tools or aggregation of other prioritisation tools to create a generic tool be most useful to you?

Do you prefer the use of weighting or scoring by expert opinion or objective measures only?

Answers

All approaches; different for specific questions - 22%

Endemic and exotic - 88% High impact on society - 55% Independent prioritization by ENHanCE - 58%

Objective measures - 38%

Semi- quantitative - 22% Exotic - 12% High risk of change and high impact on society - 33%

Don’t mind - 14% Both useful when using different prioritization tools - 24%

Start with large list & qualitative approach, then tailor further using semi-quantitative & quantitative with smaller lists - 22%

High risk of change - 12%

Don’t mind – would prefer a choice of several, comparing diseases from different perspectives - 14%

Weighting/scoring by an expert - 38%

Qualitative - 12%

Independent assessment by more than one group can be helpful, but only if consensus is reached - 14%

Qualitative & semi- quantitative - 11%

Depends upon the question - 11%