Avai lable onl ine at www.sc iencedirect .com

journal homepage: www.elsevier .com/ locate / jva l

Articles

Challenges in Research and Health Technology Assessment ofRare Disease Technologies: Report of the ISPOR Rare DiseaseSpecial Interest GroupSandra Nestler-Parr, PhD, MPhil, MSc1, Daria Korchagina, PhD, MSc2,*, Mondher Toumi, MD, MSc, PhD3,Chris L. Pashos, PhD4, Christopher Blanchette, PhD, MBA5, Elizabeth Molsen, RN6,Thomas Morel, MSc, MA7, Steven Simoens, PhD, MSc, MA7, Zoltán Kaló, PhD8,9,Ruediger Gatermann, MA, MBA10, William Redekop, PhD11

1Rare Access Ltd., Leamington Spa, England, UK; 2Mental Health and Public Health Unit (Inserm U669), University of Paris-Sud,Paris, France; 3Public Health and Chronic Disease Laboratory, Aix-Marseille University, Marseille, France; 4Global Outcomes andEpidemiology Research, Takeda Pharmaceuticals International, Inc., Cambridge, MA, USA; 5College of Health and Human Services,University of North Carolina at Charlotte, Charlotte, NC, USA; 6Scientific & Health Policy Initiatives, ISPOR, Lawrenceville, NJ, USA;7KU Leuven Department of Pharmaceutical and Pharmacological Sciences, Leuven, Belgium; 8Department of Health Policy and HealthEconomics, Eötvös Loránd University (ELTE), Budapest, Hungary; 9Syreon Research Institute, Budapest, Hungary; 10Healthcare Policyand External Affairs Europe, CSL Behring, Biotherapies for Life, Marburg, Germany; 11Health Technology Assessment, ErasmusUniversity, Rotterdam, The Netherlands

A B S T R A C T

Background: Successful development of new treatments for rarediseases (RDs) and their sustainable patient access require overcom-ing a series of challenges related to research and health technologyassessment (HTA). These impediments, which may be unique to RDsor also apply to common diseases but are particularly pertinent inRDs, are diverse and interrelated. Objective: To develop for the firsttime a catalog of primary impediments to RD research and HTA, andto describe the cause and effect of individual challenges. Methods:Challenges were identified by an international 22-person expertworking group and qualitative outreach to colleagues with relevantexpertise. A broad range of stakeholder perspectives is represented.Draft results were presented at annual European and North AmericanInternational Society for Pharmacoeconomics and Outcomes Research(ISPOR) congresses, and written comments were received by the 385-strong ISPOR Rare Disease Review Group from two rounds of review.Findings were refined and confirmed via targeted literature search.

Results: Research-related challenges linked to the low prevalence ofRDs were categorized into those pertaining to disease recognition anddiagnosis, evaluation of treatment effect, and patient recruitment forclinical research. HTA-related challenges were classified into issuesrelating to the lack of a tailored HTA method for RD treatments anduncertainty for HTA agencies and health care payers. Conclusions: Iden-tifying and highlighting diverse, but interrelated, key challenges in RDresearch and HTA is an essential first step toward developing implement-able and sustainable solutions. A collaborative multistakeholder effort isrequired to enable faster and less costly development of safe, efficacious,and appropriate new RD therapies that offer value for money.Keywords: cost-effectiveness, health policy, health technologyassessment, orphan designation, rare diseases.

Copyright & 2018, International Society for Pharmacoeconomics andOutcomes Research (ISPOR). Published by Elsevier Inc.

Background to the Rare Disease Working Group

In 2013, two working groups were established under the auspices ofthe International Society for Pharmacoeconomics and OutcomesResearch (ISPOR) Rare Disease Special Interest Group. The first work-ing group undertook a review of rare disease [RD] terms anddefinitions, motivated by recognition of the lack of a universal

definition of rare diseases or health technologies for their treatmentand the existing diversity of definitions applied to rare diseases. Theoutput of this research was the article “Rare Disease Terminology andDefinitions—A Systematic Global Review: Report of the ISPOR RareDisease Special Interest Group,” published in 2015 in Value in Health [1].

The second working group, Challenges in Research and HealthTechnology Assessment of Rare Disease Technologies, identified and

1098-3015$36.00 – see front matter Copyright & 2018, International Society for Pharmacoeconomics and Outcomes Research (ISPOR).

Published by Elsevier Inc.

https://doi.org/10.1016/j.jval.2018.03.004

The authors represent the ISPOR Rare Disease Special Interest Group’s Challenges in Research and Health Technology Assessment ofRare Disease Technologies Working Group.The authors declare that they followed the ISPOR Code of Ethics publication guidelines. Theauthors declare that they have no competing interests.

*Address correspondence to: Daria Korchagina, University of Paris-Sud, 97 Boulevard de Port Royal, 75679 Paris, France.E-mail: dko@creativ-ceutical.com

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0

reviewed challenges faced by those engaged in research and healthtechnology assessment (HTA) in RDs and their treatments. The goalof the working group was to evaluate these challenges and dissem-inate the findings via publication and presentations. An outline wasinitially developed by working group members representing differentstakeholder perspectives from Europe and the United States, andsequential drafts were reviewed and modified during monthly tele-conferences among the coauthors. Further feedback was obtainedduring work-to-date presentations given at annual ISPOR Europeancongresses in Dublin, Amsterdam, Milan, and Glasgow, and annualISPOR international meetings in Montreal and Boston. The finalversion of this article represents the outcome of these discussions,conference feedback, and written comments received by members ofthe ISPOR Rare Disease Special Interest Group from two rounds ofreview.

Introduction

Approximately 60 million people in the United States and Euro-pean Union are affected by an RD [2]. Although RD research andclinical development of technologies for the treatment of RDs arerapidly expanding areas, there is still no universally acceptedterminology or definition as to what constitutes an RD. Typically,RDs are characterized by low frequency, where frequency isexpressed in terms of prevalence or incidence within a specificcountry or geographical region. A global review of RD terminologyfound that 58% of definitions included a prevalence thresholdwith an average global threshold of 40 cases/100,000 people [1].

According to the European Organisation for Rare Diseases(EURORDIS), an alliance of more than 700 RD patient organiza-tions in 65 countries, more than 6000 distinct RDs exist, of whichapproximately 80% are of genetic origin [3]. On average, five newRDs are described every week in the medical literature [3]. RDsrepresent a broad assortment of disorders and constellations ofclinical signs and symptoms but the vast majority of RDs affectchildren and are chronic and life threatening [4]. No cure existsfor the substantial majority of RDs, and only a few RD treatmentswith proven efficacy are currently available [5].

Consequently, countries throughout the world have recog-nized the need to enact laws and regulations to provide incen-tives for the development of new and innovative technologies forthe treatment of RDs [6–10]. Advancements in molecular genetics,understanding of disease pathogenesis, and medical technologyhave led to enhanced identification of RDs and pathways forimproving RD diagnosis, prognosis, and treatment, as well asmore accurate subclassification of common diseases into collec-tions of RDs with distinct phenotypes [11–13].

However, the development of new RD therapies faces significantobstacles with respect to research and HTA. These challenges hadnot previously been evaluated comprehensively and, consequently,this ISPOR Rare Disease working group developed a multistakeholdercatalog of the principal difficulties faced in RD research, duringevidence generation for HTA, and HTA of RD treatments. Althoughidentification of the obstacles is an important first step towardproviding efficient and practical solutions, the working group antici-pates the generation of another detailed report providing recom-mendations to address the challenges identified here. Challengesrelated to pricing of health technologies for RDs, their adoption, andpatient access were not within the remit of this project.

Methods

Impediments to RD research and HTA were identified by aworking group comprising 22 members with relevant expertise,as well as through qualitative outreach to colleagues specializing

in RDs in contract research, the life sciences industry, andacademia. The preliminary list of challenges underwent threerounds of review by the working group for comprehensiveness,refinement, and merger of duplicates. The challenges identifiedwere analyzed for interrelationships and classified into catego-ries. Findings were underpinned by a targeted literature search.Written comments were received by the 385-person ISPOR RareDisease Review Group from two rounds of review and furtherverified when draft reports were presented at ISPOR annualinternational congresses in North America and Europe. A broadrange of stakeholder perspectives from researchers, clinicians,industry, regulatory and HTA agencies, patients, payers, andmarket access specialists are represented in this report.

Results

The following sections describe the cause and effect of individualchallenges and their relevance in RDs. Results are grouped intochallenges relating to RD research and HTA of RD treatments,respectively, and subcategorized. It should be noted that,although the identified impediments are diverse, they are inter-related. Furthermore, some of the identified issues are unique toRDs, whereas others also apply to common diseases but areespecially relevant or burdensome in RDs.

Challenges in Research

Research-related challenges linked to the low prevalence of RDswere grouped into three categories, as illustrated in Figure 1.

Disease recognition and diagnosisSeveral interrelated challenges pertain to the recognition anddiagnosis of RDs, all of which impinge on the quality of epide-miologic and clinical studies and complicate the characterizationof unmet patient needs, potential efficacy, safety, effectiveness,and value of treatments for RDs.

Lack of familiarity with RDs. Insufficient awareness and knowl-edge of RDs can increase the likelihood of misdiagnosis anddelayed accurate diagnosis [14–21]. Patients unfamiliar withpertinent signs and symptoms may not seek medical advicewhen appropriate. Similarly, clinicians may fail to recognize thedisease [14,15] or may incorrectly attribute symptoms to commondiseases with which they are more familiar. This is reflected inthe average delay of 7.6 years in the United States and 5.6 years inthe United Kingdom before a patient with an RD receives thecorrect diagnosis [22]. In a survey-based outcome study ofsymptomatic patients with α-1 antitrypsin deficiency, an under-recognized rare genetic condition that increases the risk of lungemphysema and liver disease, the average diagnostic delay was8.3 ± 6.9 years after onset of symptoms [23].

Disease heterogeneity. Incomplete understanding of a diseaseand its etiology may severely limit the comparability of findingsfrom epidemiologic and clinical studies. Heterogeneity in patho-genesis, symptom presentation, natural history, disease severity,and progression can greatly impede efforts to characterize an RDin clinical research and to identify it in routine clinical practice,often resulting in misdiagnosis and an underestimation of truedisease frequency.

The heterogeneous clinical presentation of many RDs ham-pers identification of affected patients, as seen across the broadspectrum of phenotypes in Gaucher’s disease, ranging from lethaldisease in neonates to asymptomatic older adults [24]. It is notuncommon for patients with RDs, such as Behçet’s disease orlate-onset Pompe disease, to exhibit a long initial asymptomaticphase during which their condition may not be identified [25,26].In many RDs, no genotype–phenotype correlations have yet been

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0494

established [24]. In addition, RD patients may not seek medicaladvice until symptoms become burdensome. These factors fur-ther complicate efforts to fully understand the presentation,etiology, and natural history of an RD, thereby making it partic-ularly difficult to provide individual patients with prognosticinformation. Natural history and, consequently, prognosis mayalso vary greatly; for example, patients with acute myeloidleukemia, which is an RD known to have wide heterogeneity,have been shown to have a 5-year survival that can vary between16% and 65% based on phenotypic characteristics that can beused for prognostication [27].

Increased knowledge of an RD will elucidate subtype com-monalities and differences. RDs that may have initially appearedunrelated have subsequently been found to have commoncausality, as occurred in several genetic metabolic disorderswithin the lysosomal storage disease family [28]. Mucopolysac-charidoses (MPS), a subgroup of this disease family, are caused bythe absence or malfunctioning of lysosomal enzymes. SeveralMPS disorders, termed “syndromes,” appeared unrelated, asinitial presentation and subsequent progression were found tovary significantly. Three of the seven MPS syndromes identifiedto date were described and named before the discovery of thelysosome enabled an accurate characterization of the underlying,common disease mechanism [29,30]. The similarities in thegenetic origin of different MPS types have motivated substantialinvestment over the past decade into development of innovativetreatments for several syndromes within this disease family [30].

In contrast, other diseases that initially seemed relatedbecause of similar clinical presentations were later found to havedifferent causations. The disease, initially termed “weisses Blut”(German for white blood) by the German pathologist RudolphVirchow in the 19th century who found abnormally high levelsof white blood cells in his patients, is now known as “Leukämie”(Greek “leukos” for white and “aima” for blood) [31]. Today,leukemia is no longer seen as a single disease but recognized asa family of distinct diseases of different pathogenesis, whichrequire markedly different therapeutic strategies [32].

Difficulties in establishing specific and sensitive diagnostic criteria.Diagnosis of an RD may be straightforward in patients with

pathognomic clinical features. However, owing to the heteroge-neity of many RDs and the difficulty with correct interpretation ofcomplex investigational testing algorithms, diagnostic certaintycan be problematic. Establishment of relevant and specific diag-nostic criteria, including diagnostic tests, may be hindered by thelack of (1) a sufficiently large patient cohort from which criteriacan be reliably characterized, (2) consensus on diagnostic criteriadue to heterogeneous disease presentation, and (3) diagnosticcriteria that are applicable to all cases across the range of diseaseheterogeneity. For example, establishing the diagnosis of cysticfibrosis is straightforward in most cases, but it is more complex ina small proportion of patients. The “sweat test” to assess the chloridecontent of sweat is a standard screening test in infants but is oftenunreliable in patients with a less severe clinical phenotype who maypresent later in life [33]. Diagnosis through genotyping may also beproblematic in milder cases, such as in the absence of a homozygousΔF508 gene abnormality and when uncharacterized abnormal genesare present [33].

Misdiagnosis. The foregoing problems commonly result inmisdiagnoses that can lead to inappropriate treatment. Suchtreatment may not only produce side effects, but can even masksymptoms of the underlying condition, thereby further delayingthe correct diagnosis and the initiation of appropriate treatment.In α-1 antitrypsin deficiency, for example, patients are frequentlymisdiagnosed as suffering from asthma and inappropriatelytreated with inhaled asthma therapies. It may take several yearsof inadequate response to these treatments and multiple con-sultations with different specialists before the correct diagnosis ismade [23,34]. In Wilson’s disease, a rare genetic disease of coppermetabolism, the heterogeneity of initial symptoms and the broadage range at onset prompt presentation to a diverse range ofspecialists, which is a contributing factor in the diagnostic delayand frequent initial misdiagnosis of the condition [35]. The mostcommon cause of death among patients affected with Wilson’sdisease, which is always fatal if left untreated, is delayeddiagnosis [36].

Geographic variation. The prevalence of an RD may varybetween regions and countries, ranging from clustering in somegeographic areas to wide dispersal in others [25,37,38]. A

Fig. 1 – Challenges in RD research and in HTA of RD treatments.

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0 495

prevalence study of Behçet’s disease, a chronic, multisystemvasculitis also referred to as “Silk Road disease,” revealed asignificant geographic variation globally. The highest prevalencewas reported in Turkey, at 421 cases/100,000 people, followed byother countries along the ancient trading route between the FarEast and the Mediterranean Sea, and the lowest in the UnitedKingdom, at 0.6 cases/100,000 people [25]. Other examples of RDsthat exhibit significant geographic variation in prevalence includeFinnish disease heritage, a group of rare hereditary diseases thatare overrepresented in Finland [39], and α-1 antitrypsin defi-ciency, in which the prevalence of the Z genetic variant that isassociated with severe deficiency is considered to reflect themigration patterns of Viking descendants [40].

When the true geographic variation in disease frequency isunknown, prevalence data from epidemiologic studies performedin a particular region may be wrongly extrapolated to a widergeography in the event of prevalence heterogeneity. The exis-tence of single-site or single-region disease cohorts may leadresearchers to suspect the existence of regional variation indisease frequency [41]. Conversely, geographic differences indisease definition and diagnostic criteria may result in over-estimates of regional prevalence variation. In Behçet’s disease,for example, a higher percentage of cases in low-prevalencecountries has been seen in patients whose ancestry is traceableto high-prevalence areas [25]. However, the disease frequency inthese isolated high-prevalence pockets is not generalizable.

Although robust epidemiologic studies might establish the degreeof phenotypic variation in disease frequency and thereby help toascertain disease etiology, they may not provide sufficient data toallow the distinction of genetic from environmental factors, espe-cially when conducted in populations that rarely migrate.

Evaluation of treatment effectThe reliable assessment of the effectiveness of medical inter-ventions for the treatment of conditions across the entirespectrum of disease severity and prevalence presents manychallenges. However, some of these challenges are of particularrelevance in RDs as a direct consequence of their rarity andvariable clinical phenotype, as detailed previously.

Heterogeneity of prognosis and treatment effect. The combinationof disease rarity and heterogeneity reduces the certainty withwhich an RD technology’s efficacy, safety, and effectiveness canbe determined, as treatment effect may vary significantly in asingle RD [34–36]. This variability also hampers interventionalstudies as it requires increased sample sizes for suitably poweredclinical trials [42,43].

Genetic and biological markers are potentially useful tools toidentify patient subgroups that are most likely to respond to aparticular treatment [44], to develop targeted treatment approaches,and to predict treatment responses and prognosis [45]. For example,several technologies were developed for cystic fibrosis that targetspecific mutations in the cystic fibrosis transmembrane conduc-tance regulator [CFTR] gene [46–48]. However, the utilization ofbiological and clinical markers for patient recruitment into clinicaltrials will have conflicting effects on heterogeneity and rarity;although subgroups may be more homogeneous, they will de factobe smaller in size. This may substantially prolong patient recruit-ment in studies of treatment efficacy and safety.

Selection bias. Clinical trials may be biased toward the inclu-sion of patients with more severe disease and more severesymptoms, whereas patients with a milder disease course mayeither not be identified as having the RD in question or may notmeet the inclusion criteria. Selection bias may also impedegeneralizability of the trial results to a broader patient populationin other ways. For example, clinical trials of novel treatments formultiple myeloma often exclude elderly patients even though theincidence of myeloma is higher in the elderly [49]. Consequently,

treatment efficacy demonstrated in controlled clinical trials maydiffer from the effectiveness observed in routine clinical practice(although this may not always be the case [50]).

Clinical trials tend to be conducted in centers with expertisein specific RDs, which may lead to selection bias toward thosepatients more able to withstand travel to the study center [51–53].

Uncertainties related to validated trial outcomes. The developmentand validation of disease-specific, sensitive, robust, and relevantclinical, patient-reported or observer-reported outcomes (PROs;ObsROs) is more challenging in RDs compared to commondiseases owing to the rarity and heterogeneity of RDs [54,55]. Arecent report suggests solutions for common challenges in PROand ObsRO assessments in RD clinical trials [56]. It has also beenproposed that biomarkers, currently utilized as diagnostic, pre-dictive, or pharmacodynamic tools and surrogate outcomes, maybe validated for use as primary endpoints in clinical trials tofacilitate the development of novel therapeutics for RDs currentlywithout effective therapies [57].

Patient recruitment for clinical researchDisease rarity poses a hurdle for patient recruitment to any formof research [58,59]. In turn, small study populations complicatethe accrual of robust evidence on which to base treatmentdecisions and HTA evaluations.

Geographic limitations in patient recruitment. Geographic varia-tions in existing clinical expertise [53,60] and disease prevalenceinfluence the ability to recognize and study a RD across geog-raphies [61]. This may result in geographical variability in boththe recruitment of sufficient patient numbers for clinical researchand the existence of clinical expert centers with the requiredresearch capability. Expert centers, such as the European Refer-ence Networks (ERNs), and patient advocacy groups may repre-sent useful sources for patient recruitment for RD research. In2017, 24 ERNs were launched with the aim of improving diagnosisand treatment of rare or low-prevalence complex diseases orconditions [62]. These virtual networks of RD expert centers,medical specialists, and health care providers share knowledge,best-practice, and some resources for research and the jointdelivery of highly specialized health care across EU borders [63].

Insufficiently specific coding systems. Common diagnostic codingsystems, such as ICD-9 (International Classification of Diseases,version 9) and ICD-10, may not be sufficiently specific or sensitivefor certain RDs [64,65]. A 2015 audit of 6,954 clinical entities listedby Orphanet revealed that only 355 entities had a unique andspecific code in ICD-10, and only 162 could be specifically mappedto a set of ICD-10 codes [65]. Patients with an RD may therefore behard to identify from electronic medical records or other (e.g.,administrative) databases. A more rigorous and resource-con-suming search for patients with a particular RD may be neededby using, for example, uncoded free-text fields. Text mining withartificial intelligence or machine learning is, for the most part, inearly stages of development for validation of diagnosis, andtogether with issues of disease rarity and heterogeneity, compli-cates epidemiologic research into many RDs.

Orphanet suggested a classification system that endorsesrepresentation of 5,400 RDs in the foundation layer of ICD-11[65], which is 10 times more than in ICD-10 coding. ICD-11 is dueto be launched by the World Health Organization (WHO) in 2018,and the adoption of the proposed, more specific, classification ofRDs is likely to expedite patient identification and clinicalresearch in RDs.

Ethical and legal hurdles. The use of traditional randomized,controlled trial designs may, in some instances, be deemedinappropriate for interventional studies in RDs; for example,randomization and inclusion of control arms may be unethical[58,59,66,67]. Achieving a sufficient sample size in prospectivestudies is also hampered by ethical and legal hurdles [59]. The

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0496

evidence base typically required in the development, HTA, andlife-cycle management of RD treatments is often at odds with theavailable sample size for RD research and should therefore bebalanced with the need to ensure scientific credibility andcompliance with ethical trial design [68,69]. Although not yetextensively employed, alternative trial designs in RD interven-tional study, such as n-of-1 trials, randomized placebo-phasedesign, enriched enrollment, randomized withdrawal design, andadaptive designs, may be more appropriate to generate reliableevidence in RDs than conventional studies [70,71].

Challenges in Health Technology Assessment

Although regulatory approval processes make concessionsbecause of the rarity of certain diseases, reimbursement policyfrequently does not follow suit. The European Medicines Agency(EMA) commented in 2013 that a number of new medicinesauthorized by the EMA are either not reimbursed by nationalhealth systems or are not used as expected because they do notmeet the requirements of HTA bodies [72].

The level of scientific and clinical evidence that is deemedsufficient by regulators to grant a novel health technology amarketing authorization may not fulfill the requirements of HTAagencies for issuing a positive reimbursement recommendation[73]. In addition, differences exist in decision-making criteriabetween different HTA bodies, often resulting in geographicdifferences in reimbursement status and patient access to RDtreatments [6,74].

HTA agencies are primarily concerned with the satisfactorydemonstration of a treatment’s value for money, and by the highper patient costs of many RD treatments. Manufacturers gener-ally explain these high costs with the necessity to recoverinvestments made into the development of new RD interventionsfrom small patient populations. Whether such high costs arejustified continues to be debated, but there is no doubt that theevaluation of a treatment’s value poses multiple methodologicalchallenges that are particularly pertinent in RDs. The identifiedchallenges relating to the HTA of RD technologies were groupedinto two categories, as illustrated in Figure 1.

No tailored HTA method for RD treatmentsAppraisal of health technologies for reimbursement increasinglyincludes evaluation of both clinical and economic evidence.Standard HTA methods require robust information on compara-ble efficacy, effectiveness, and associated costs of new healthcare interventions, including data on morbidity, mortality, qual-ity of life (QoL), and health care utilization. Owing to thechallenges of research in RDs and their treatments, not all thesetypes of data may be available at the time of the HTA. HTAmethods that are based primarily on cost-effectiveness analysesgenerally lack sufficient flexibility to allow a comprehensiveevaluation that takes account of the complications associatedwith generating a robust evidence base early in a health inter-vention’s life cycle. However, this issue is particularly pertinent in RDtreatments. Even if a reliable estimation of a RD treatment’s cost-effectiveness can be obtained, its incremental cost-effectiveness ratio(ICER) is frequently well in excess of the “accepted” levels, and thetreatment would not be reimbursable according to conventionalcost-effectiveness criteria [6]. Whereas some argue that this is nota reason for adopting tailored appraisal methods for RD technologiesthat grant a premium for rarity [75], others endorse a more holisticappraisal of the value of RD treatments [76,77].

Lack of sufficient and robust clinical data. Owing to the research-related challenges described previously, the available clinicalevidence is often limited in RDs. A review of the design of pivotalclinical trials for 64 orphan medicinal products with EMA market-ing authorization revealed multiple methodological shortcomings.

Nonrandomized trials constituted more than 35% of all reviewedtrials, while a control arm was absent in more than 30%, and QoLmeasures were available in only 27% of trials [78].

Patient registries for RDs are increasingly utilized to gatherdata on the natural history of a disease, identify patient sub-groups, recruit patients for clinical research, and facilitate long-term patient follow-up and real-world data generation [79–83].Although data from patient registries have, in principle, thepotential to address some of the data gaps, their use in standardHTAs still suffers from shortcomings, such as inadequate orinconsistent data collection [13].

No established standard of care. Standard HTA methods arebased on a comparative analysis of the treatment under consid-eration versus the best available treatment alternative. In theabsence thereof, an established standard of best supportive careis used as the comparator. Supportive care may differ acrossgeographies, and its effectiveness and cost may not be wellstudied. Therefore, an established standard of care in the man-agement of RDs is often lacking. In addition, routine clinicalpractice in RD management may differ from that recognized byhealth authorities; many existing molecules have been used “off-label” in clinical practice to treat RDs where no effective licensedalternatives exist [84–88].

Insufficient knowledge of the natural history of the disease. Theshortage of reliable information on the clinical, humanistic, andeconomic burden of RDs poses a challenge for accurate assess-ment of the value and impact of a new RD technology. Thefrequently progressive and degenerative nature of RDs, pairedwith a poor understanding of the disease’s natural history, isproblematic for HTA modeling and projection of long-term treat-ment outcomes and associated costs [89].

Lack of validated instruments to assess efficacy and effectiveness endpoints. The absence of robust long-term outcomes data, such asmortality, and validated QoL instruments commonly impedes thereliable estimation of a RD technology’s treatment benefit andoutcomes such as the incremental quality-adjusted life years(QALYs), a key metric in cost-effectiveness analysis for manyHTA agencies [90]. The development and validation of new,disease-specific and sensitive end points for small RD patientpopulations can be very time consuming and costly. Approachesfor adaptation of existing outcomes for RD populations have beensuggested [56], and various disease-specific QoL measures havebeen developed, such as the Huntington Quality of Life Instru-ment [91] and the Quality of Life-Primary Ciliary Dyskinesiainstrument [92]. The use of biomarkers as primary end pointsin RD trials has also been proposed [57].

Application of ICER thresholds. When HTA agencies assess RDtechnologies based on the criterion of cost-effectiveness alone,the resulting ICER is likely to exceed the explicit or implicitthreshold value in many jurisdictions [6]. Appraisals drivenprimarily by cost-effectiveness offer only a partial evaluation ofa RD technology’s entire range of benefits to patients, the healthcare system, and society. Reasons for this include that suchappraisals tend not to account for social value judgments, forexample, the nature of the disease, its rarity and severity, and theavailability of alternative treatment options [93–95].

With RDs, it is therefore particularly important to identify andweigh all factors of the health technology that provide value andincorporate them into HTA methods, such as by using multi-criteria decision analysis (MCDA) methodology [96,97]. However,in the absence of MCDA methods specific for RD HTA it is unclearhow value elements of a technology, which cannot be captured ina cost-effectiveness evaluation, can be integrated into existingdecision-making frameworks to inform price and reimbursementdecisions. Although some simulations exist [98,99], it is notknown how the incorporation of broader value elements into adecision-making framework would affect the decisions.

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0 497

HTA agencies are taking steps to develop more tailoredprocesses for the appraisal of RD technologies, in consultationwith patients, clinicians, manufacturers, payers, and policy-makers. Some HTA bodies apply specific criteria to increasethe cost-effectiveness thresholds and thereby facilitate reim-bursement of RD technologies. QALY weighting based on diseaseprevalence was proposed as a way to adapt the standard cost-effectiveness approach for RD interventions [89,100]. In Sweden,the cost-effectiveness threshold is adaptable based on diseaseseverity [74]. The Scottish Medicines Consortium (SMC) mayaccept a higher than usual ICER for RD treatments if the newmedicine meets certain “decision-modifiers,” such as evidence ofsubstantial improvement in life expectancy, QoL, or the absenceof alternative therapeutic options of proven benefit [101]. TheNational Institute for Health and Care Excellence (NICE) estab-lished the Highly Specialized Technologies (HST) program for theevaluation of innovative technologies for ultra-rare conditions inEngland. NICE-HST takes much broader criteria into considera-tion than NICE’s standard appraisal process and applies signifi-cantly higher ICER threshold ranges, dependent on theincremental QALY gain that the new intervention delivers [102].Germany’s Gemeinsamer Bundesausschuss (G-BA) appraisesorphan drugs through a simplified evaluation process thatassumes a default additional clinical benefit by virtue of orphandesignation, provided their annual sales do not exceed €50million during the first 12 months of commercialization [103].

Uncertainty for HTA authoritiesAs a consequence of the research-related hurdles described inthe first part of this report, health technology assessors faceuncertainty when (1) translating clinical efficacy data from trialsinto estimates of clinical effectiveness in a real-world setting; (2)evaluating the overall added-value of the new health technologyand the extent to which it addresses current unmet medicalneeds; and (3) quantifying health care costs, utilization, andpossible savings over the life time of the disease or patient, inthe context of their respective health care setting. Low diseaseprevalence was found to be associated with a less robustevidence base in HTA submissions for RD interventions [104].This may leave HTA agencies and health care payers short of theevidence required to make informed and defendable decisions onthe reimbursement of an intervention at a given cost. The level ofuncertainty in relation to the available evidence base for an RDtechnology is therefore likely to negatively correlate with itsreimbursed price and reimbursement status [105].

Budget constraints, paired with the uncertainties related toeffectiveness, value-for-money, and budget impact of RD tech-nologies, increase HTA authorities’ hesitancy in issuing positivereimbursement decisions. Although the impact of RD treatmentsis relatively minor in terms of proportional health care, and evenpharmaceutical, spending [106–109], both total and proportionalexpenditures on RD treatments are continually rising as a resultof the steady growth in the number of RD treatments that aredeveloped and commercialized [110–112].

Concerns over long-term affordability of RD treatments andthe limitations of evidence generation in many RDs throughconventional clinical trials have compelled policymakers to con-sider alternative, or complementary, methods for the evaluationof clinical efficacy and cost-effectiveness of new interventions.Such policies usually grant controlled and/or restricted patientaccess to a new intervention with the expectation that real-worldevidence will be generated for the treatment in a clinical setting.These data are intended to close data gaps and enable continu-ous evaluation of a treatment’s real-world effectiveness, cost-effectiveness, and value. Managed access agreements in Englandor temporary reimbursement under the G-BA’s orphan drug

framework that impose requirements for real-world evidencecollection are examples of such programs [103,113]. Althoughthese have methodological challenges and can be resourceintensive, they offer a pragmatic solution in the absence of arobust evidence base available at the time of an HTA.

Conclusion

The difficulties faced by different stakeholder groups in RDclinical research and HTA of RD treatments significantly impactthe development and the evaluation of these technologies.Research-related challenges, linked predominantly to the lowprevalence of RDs, and hurdles related to the HTA of RDtechnologies result in uncertainty for decision makers in healthcare. This may consequently impact reimbursement, adoption,and equity of patient access to RD treatments.

The interrelated nature of these challenges requires a collec-tive approach by clinical researchers, industry, HTA agencies,patients, and policymakers toward developing implementableand sustainable solutions. A collaborative effort across stake-holder groups is required to reduce the time and cost for thedevelopment of safe and effective new therapies for RDs thataddress unmet patient needs, provide value for money, andfacilitate equitable patient access. Adopting current best practiceand developing new approaches are necessary to overcome thesechallenges and advance RD treatment options and patient accessand improve health outcomes. Many organizations, includingISPOR, EURORDIS, the National Organization for Rare Diseases(NORD), the European Network for HTA (EUnetHTA), and the RareDiseases Clinical Research Network, have developed proposals toaddress many of the issues highlighted in this report, and whichwill provide guidance for future RD research.

Acknowledgments

The individual contributions by Matthew Magestro, Lorenzo Man-tovani, Lesley G. Shane, and Vlad Zah are gratefully acknowledged.The authors express special thanks to Ségolène Aymé, EmeritusDirector of Research, Founder of Orphanet, Chair of the TopicAdvisory Group on Rare Diseases at WHO, Member of the EuropeanCommission Expert Group on Rare Diseases, Project Leader ofSupport IRDiRC, and Editor-in-Chief, Orphanet Journal of Rare Diseases.

The authors also thanks the members of the Challenges inAssessment and Appraisal of Rare Disease Diagnostics & TreatmentsWorking Group, who contributed to the development of this articleand have participated as reviewers: Joe Biskupiak, Jacqueline Bow-man-Busato, Dyfrig Hughes, Mohit Jain, Katarzyna Kolasa, Zhimei(Jamae) Liu, Phil Ruff, Peter Sun, and Art Zbrozek. Finally, the authorsthank the following ISPOR members for valuable feedback on draftsof this report: Alexander Artyomenko, Caroline Conti, Lincy Lal, JadeMarshall, Gabriel Tremblay, Anup Raj Upreti.

All comments, many of which were substantive and con-structive, were considered. ISPOR member comments contributeto the high-quality, consensus nature articles that characterizeISPOR Special Interest Group and Task Force publications. Thesteady and capable support of ISPOR and its staff is appreciated.

The authors have no other financial relationships to disclose.

R E F E R E N C E S

[1] Richter T, Nestler-Parr S, Babela R, et al. Rare disease terminology anddefinitions—A systematic global review: Report of the ISPOR RareDisease Special Interest Group. Value Health 2015;18:906–14.

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0498

[2] Wilson C. Patient network and advocacy groups. In: Hernberg-Ståhl E,Reljanovic M, eds. Orphan Drugs: Understanding the Rare DiseaseMarket and Its Dynamics. Cambridge, UK: Woodhead Publishing, 2013.

[3] European Organisation for Rare Diseases (EURORDIS). Position paper:Why research on rare diseases? 2010. Available from: http://www.eurordis.org/sites/default/files/publications/why_rare_disease_research.pdf. [Accessed February 20, 2017].

[4] European Organisation for Rare Disease (EURORDIS). Rare diseases:understanding this public health priority. 2005. Available from: http://www.eurordis.org/IMG/pdf/princeps_document-EN.pdf. [Accessed April27, 2017].

[5] Luzzatto L, Hollak CE, Cox TM, et al. Rare diseases and effectivetreatments: are we delivering? Lancet 2015;385:750–2.

[6] Gammie T, Lu CY, Babar ZU-D. Access to orphan drugs: Acomprehensive review of legislations, regulations and policies in 35countries. PLoS ONE 2015;10:e0140002.

[7] Grienenberger A. Understanding orphan drug regulations: an EU andU.S. comparative analysis. J Biolaw Bus 2004;7:58–61.

[8] Haffner ME. The Food and Drug Administration’s Office of OrphanProducts Development: incentives, grants, and special designationsspeed therapies for orphan diseases. Retina 2005;25(Suppl S90).

[9] Wilson C. Policies and research funding. In: Hernberg-Ståhl E,Reljanovic M, eds. Orphan Drugs: Understanding the Rare DiseaseMarket and Its Dynamics. Cambridge, UK: Woodhead Publishing, 2013.

[10] Liu BC, He L, He G, et al. A cross-national comparative study of orphan drugpolicies in the United States, the European Union, and Japan: towards amade-in-China orphan drug policy. J Public Health Policy 420;31:407–20.

[11] Loughnot D. Potential interactions of the Orphan Drug Act andpharmacogenomics: a flood of orphan drugs and abuses? Am J LawMed 2005;31:365–80.

[12] Cote A, Keating B. What is wrong with orphan drug policies? ValueHealth. 2012;15:1185–91.

[13] Simoens S. Pricing and reimbursement of orphan drugs: the need formore transparency. Orphanet J Rare Dis 2011;6:42.

[14] Engel P, Bagal S, Broback M, et al. Physician and patient perceptionsregarding physician training in rare diseases: the need for strongereducational initiatives for physicians. J Rare Dis 2013;1:1–15.

[15] Kostopoulou O, Delaney BC, Munro CW. Diagnostic difficulty and errorin primary care: a systematic review. Fam Pract 2008;25:400–13.

[16] European Organisation for Rare Disease (EURORDIS). The voice of12,000 patients: experiences and expectations of rare disease patientson diagnosis and care in Europe. 2009. Available from: http://www.eurordis.org/IMG/pdf/voice_12000_patients/EURORDISCARE_FULLBOOKr.pdf. [Accessed February 9, 2017].

[17] Limb L, Nutt S, Sen A. Experiences of rare diseases: an insight frompatients and families. 2010. Available from: https://www.raredisease.org.uk/media/1594/rduk-family-report.pdf. [Accessed February 9, 2017].

[18] Gahl WA. The battlefield of rare diseases: where uncommon insightsare common. Sci Transl Med 2012;4:154ed7-54ed7.

[19] Gahl WA, Markello TC, Toro C, et al. The NIH Undiagnosed DiseasesProgram: insights into rare diseases. Genet Med 2012;14:51–9.

[20] Gahl WA, Tifft CJ. The NIH Undiagnosed Diseases Program: lessonslearned. JAMA 2011;305:1904–5.

[21] European Organisation for Rare Disease (EURORDIS). Survey of thedelay in diagnosis for 8 rare diseases in Europe (‘EURORDISCARE 2’).2007. Available from: http://www.eurordis.org/sites/default/files/publications/Fact_Sheet_Eurordiscare2.pdf. [Accessed February 9, 2017].

[22] Miller DH, Weinshenker BG, Filippi M, et al. Differential diagnosis ofsuspected multiple sclerosis: a consensus approach. Mult Scler2008;14:1157–74.

[23] Campos MA, Wanner A, Zhang G, et al. Trends in the diagnosis ofsymptomatic patients with alpha-1 antitrypsin deficiency between1968 and 2003. Chest 2005;128:1179–86.

[24] Sidransky E. Gaucher disease: complexity in a “simple” disorder. MolGenet Metab 2004;83:6–15.

[25] Leonardo NM, McNeil J. Behçet’s disease: is there geographicalvariation? A review far from the Silk Road. Int J Rheumatol 2015;2015:7.

[26] Bertoldo F, Zappini F, Brigo M, et al. Prevalence of asymptomaticvertebral fractures in late-onset Pompe disease. J Clin Endocrinol Metab2015;100:401–6.

[27] Leunis A, Redekop WK, van Montfort KAGM, et al. The development andvalidation of a decision-analytic model representing the full diseasecourse of acute myeloid leukemia. Pharmacoeconomics 2013;31:605–21.

[28] Winchester B, Vellodi A, Young E. The molecular basis of lysosomalstorage diseases and their treatment. Biochem Soc Trans 2000;28:150–4.

[29] Brands M. Enzyme-replacement therapy in mucopolysaccharidoseswith a specific focus on MPS VI. Erasmus University Rotterdam, 2013.

[30] Noh H, Lee JI. Current and potential therapeutic strategies formucopolysaccharidoses. J Clin Pharm Ther 2014;39:215–24.

[31] Mukherjee S. The Emperor of All Maladies: A Biography of Cancer. NewYork, NY: Scribner, 2010.

[32] Patlak M. Targeting leukemia: from bench to bedside. FASEB J 2002;16:273e-273e.

[33] De Boeck K, Vermeulen F, Dupont L. The diagnosis of cystic fibrosis.Presse Med 2017;46:e97–108.

[34] Stoller JK, Sandhaus RA, Turino G, et al. Delay in diagnosis of alpha-1antitrypsin deficiency: a continuing problem. Chest 2005;128:1989–94.

[35] Prashanth LK, Taly AB, Sinha S, et al. Wilson’s disease: diagnosticerrors and clinical implications. J Neurol Neurosurg Psychiatry2004;75:907–9.

[36] Walshe JM. Cause of death in Wilson disease. Mov Disord 2007;22:2216–20.

[37] Yazıcı H, Biyikli M, van der Linden S, et al. The ‘zero patient’ design tocompare the prevalences of rare diseases. Rheumatology 2001;40:121–2.

[38] Alpsoy E, Akman-Karakas A, Uzun S. Geographic variations inepidemiology of two autoimmune bullous diseases: pemphigus andbullous pemphigoid. Arch Dermatol Res 2015;307:291–8.

[39] Norio R. Finnish Disease Heritage I: characteristics, causes, background.Hum Genet 2003;112:441–56.

[40] Cox DW, Woo SL, Mansfield T. DNA restriction fragments associatedwith alpha 1-antitrypsin indicate a single origin for deficiency allele PIZ. Nature 1985;316:79–81.

[41] de la Paz MP, Groft SC. Rare Diseases Epidemiology. Dordrecht: SpringerScienceþBusiness Media 2010.

[42] Wittes J. Sample size calculations for randomized controlled trials.Epidemiol Rev 2002;24:39–53.

[43] Noordzij M, Tripepi G, Dekker FW, et al. Sample size calculations: basicprinciples and common pitfalls. Nephrol Dial Transplant 2010;25:1388–93.

[44] Bell SC, Wood ME. Biomarkers: Their role in CFTR modulator therapiesfrom early development to the clinic. Am J Respir Crit Care Med 2018.http://dx.doi.org/10.1164/rccm.201801-0177ED.

[45] Gülbakan B, Özgül RK, Yüzbas-ıoglu A, et al. Discovery of biomarkers inrare diseases: innovative approaches by predictive and personalizedmedicine. EPMA J 2016;7:24.

[46] Accurso FJ, Rowe SM, Clancy JP, et al. Effect of VX-770 in persons withcystic fibrosis and the G551D-CFTR mutation. N Engl J Med2010;363:1991–2003.

[47] Ratjen F, Hug C, Marigowda G, et al. Efficacy and safety of lumacaftorand ivacaftor in patients aged 6–11 years with cystic fibrosishomozygous for F508del-CFTR: a randomised, placebo-controlledphase 3 trial. Lancet Respir Med 2017;5:557–67.

[48] Rowe SM, Daines C, Ringshausen FC, et al. Tezacaftor-ivacaftor inresidual-function heterozygotes with cystic fibrosis. N Engl J Med2017;377:2024–35.

[49] Blommestein HM, Verelst SG, de Groot S, et al. A cost-effectivenessanalysis of real-world treatment for elderly patients with multiplemyeloma using a full disease model. Eur J Haematol 2016;96:198–208.

[50] Chari A, Romanus D, Luptakova K, et al. Duration of therapy (DOT) andtime to next therapy (TTNT) of bortezomib, carfilzomib and ixazomibcombinations with lenalidomide/dexamethasone (VRd, KRd, IRd) inpatients (pts) with relapsed/refractory multiple myeloma (RRMM): clinicalpractice in the United States vs clinical trial experience. Blood 2017;130:1818.

[51] Wastfelt M, Fadeel B, Henter JI. A journey of hope: lessons learnedfrom studies on rare diseases and orphan drugs. J Intern Med 2006;260:1–10.

[52] Evangelista T, Hedley V, Atalaia A, et al. The context for the thematicgrouping of rare diseases to facilitate the establishment of EuropeanReference Networks. Orphanet J Rare Dis 2016;11:17.

[53] Rare Disease UK. Centres of excellence for rare diseases. 2013. Availablefrom: https://www.raredisease.org.uk/media/1601/centres-of-excellence.pdf. [Accessed February 14, 2017].

[54] Basch E, Bennett AV. Patient-reported outcomes in clinical trials of rarediseases. J Gen Intern Med 2014;29:801–3.

[55] International Rare Diseases Research Consortium (IRDiRC). Patient-centered outcome measures initiatives in the field of rare diseases. 2016.Available from: http://www.irdirc.org/wp-content/uploads/2017/12/PCOM_Post-Workshop_Report_Final.pdf. [Accessed February 20, 2017].

[56] Benjamin K, Vernon MK, Patrick DL, et al. Patient-reported outcomeand observer-reported outcome assessment in rare disease clinicaltrials: An ISPOR COA Emerging Good Practices Task Force report. ValueHealth 2017;20:838–55.

[57] Kakkis ED, O’Donovan M, Cox G, et al. Recommendations for thedevelopment of rare disease drugs using the accelerated approvalpathway and for qualifying biomarkers as primary endpoints.Orphanet J Rare Dis 2015;10:16.

[58] Augustine EF, Adams HR, Mink JW. Clinical trials in rare disease:challenges and opportunities. J Child Neurol 2013;28:1142–50.

[59] Griggs RC, Batshaw M, Dunkle M, et al. Clinical research for raredisease: opportunities, challenges, and solutions. Mol Genet Metab2009;96:20–6.

[60] European Union Committee of Experts on Rare Diseases (EUCERD).EUCERD workshop report. National centres of expertise for rarediseases & European collaboration between centres of expertise. 2011.Available from: http://www.eucerd.eu/?post_type=document&p=1231.[Accessed February 14, 2017].

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0 499

[61] Shafie AA, Chaiyakunapruk N, Supian A, et al. State of rare diseasemanagement in Southeast Asia. Orphanet J Rare Dis 2016;11:107.

[62] European Reference Network. Working for patients with rare, low-prevalence and complex diseases. 2017. Available from: https://ec.europa.eu/health/sites/health/files/ern/docs/2017_brochure_en.pdf.[Accessed February 18, 2018].

[63] Héon-Klin V. European Reference networks for rare diseases: what isthe conceptual framework? Orphanet J Rare Dis 2017;12:137.

[64] Nigwekar SU, Solid CA, Ankers E, et al. Quantifying a rare disease inadministrative data: the example of calciphylaxis. J Gen Intern Med2014;29:724–31.

[65] Ayme S, Bellet B, Rath A. Rare diseases in ICD11: making rare diseasesvisible in health information systems through appropriate coding.Orphanet J Rare Dis 2015;10:35.

[66] de Melo-Martín I, Sondhi D, Crystal RG. When ethics constrains clinicalresearch: trial design of control arms in “greater than minimal risk”pediatric trials. Hum Gene Ther 2011;22:1121–7.

[67] de Groot S, Rijnsburger AJ, Versteegh MM, et al. Which factors maydetermine the necessary and feasible type of effectiveness evidence? Amixed methods approach to develop an instrument to help coveragedecision-makers. BMJ Open 2015;5:e007241. http://dx.doi.org/10.1136/bmjopen-2014-007241.

[68] Schipper I, Weyzig F. Ethics for drug testing in low and middle incomecountries: considerations for European market authorisation.Amsterdam: SOMO Centre for Research on Multinational Corporationsand Wemos Foundation, 2008.

[69] Weyzig F, Schipper I. SOMO briefing paper on ethics in clinical trials #1:examples of unethical trials. Amsterdam: SOMO Centre for Research onMultinational Corporations and Wemos Foundation, 2008.

[70] Abrahamyan L, Feldman BM, Tomlinson G, et al. Alternative designs forclinical trials in rare diseases. Am J Med Genet C: Semin Med Genet2016;172:313–31.

[71] Bolignano D, Pisano A. Good-quality research in rare diseases: trialsand tribulations. Pediatr Nephrol 2016;31:2017–23.

[72] European Medicines Agency. EMA-HTA workshop: bringing togetherstakeholders for early dialogue inmedicines development. 2014. Availablefrom: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2014/05/WC500166228.pdf. [Accessed February 15, 2017].

[73] Tafuri G, Pagnini M, Moseley J, et al. How aligned are the perspectivesof EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice. Br J Clin Pharmacol 2016;82:965–73.

[74] Tordrup D, Tzouma V, Kanavos P. Orphan drug considerations inhealth technology assessment in eight European countries. Int J PublicHealth 2014;1:86–97.

[75] McCabe C, Tsuchiya A, Claxton K, et al. Assessing the economicchallenges posed by orphan drugs: a comment on Drummond et al. IntJ Technol Assess Health Care 2007;23:397–401; author reply 1–4.

[76] Drummond MF. Challenges in the economic evaluation of orphandrugs. Eurohealth 2008;14:16–7.

[77] Annemans L, Aymé S, Le Cam Y, et al. Recommendations from theEuropean Working Group for Value Assessment and Funding Processesin Rare Diseases (ORPH-VAL). Orphanet J Rare Dis 2017;12:50.

[78] Picavet E, Cassiman D, Hollak CE, et al. Clinical evidence for orphanmedicinal products: a cause for concern? Orphanet J Rare Dis 2013;8:164.

[79] European Organisation for Rare Diseases (EURORDIS). 2013 EURORDISpolicy fact sheet - rare disease patient registries. 2013. Available from:http://www.eurordis.org/sites/default/files/publications/Factsheet_registries.pdf[Accessed February 16, 2017].

[80] Rare Diseases Task Force. Patient registries in the field of rare diseases:overview of the issues surrounding the establishment, management,governance and financing of academic registries. 2011. Available from: http://www.eucerd.eu/?post_type=document&p=1218. [Accessed February 16, 2017].

[81] Gliklich RE, Dreyer NA, Leavy MB. Registries for Evaluating PatientOutcomes: A User’s Guide. Washington, DC: U.S. Government PrintingOffice, 2014.

[82] Woodward L, Johnson S, Walle JV, et al. An innovative andcollaborative partnership between patients with rare disease andindustry-supported registries: the Global aHUS Registry. Orphanet JRare Dis 2016;11:154.

[83] Rubinstein YR, Groft SC, Bartek R, et al. Creating a global rare diseasepatient registry linked to a rare diseases biorepository database: RareDisease-HUB (RD-HUB). Contemp Clin Trials 2010;31:394–404.

[84] Wittich CM, Burkle CM, Lanier WL. Ten common questions (and theiranswers) About Off-label Drug Use. Mayo Clin Proc 2012;87:982–90.

[85] Dooms M, Cassiman D, Simoens S. Off-label use of orphan medicinalproducts: a Belgian qualitative study. Orphanet J Rare Dis 2016;11:144.

[86] Dresser R, Frader J. Off-Label prescribing: A call for heightened professionaland Government oversight. J Law Med Ethics 2009;37: 476–396.

[87] Caro JS, L; Kolominsky PL, Hamed A. Is off-label drug use enhancing orlimiting access for patients with rare diseases? In: ISPOR 19th AnnualEuropean Congress. Vienna, Austria, 2016.

[88] Camp R, West R. Understanding off-label use and the informationneeds of patients: a pilot Eurordis survey in rare diseases. 2012.

Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Presentation/2012/11/WC500135170.pdf. [AccessedFebruary 17, 2017].

[89] Simoens S, Picavet E, Dooms M, et al. Cost-effectiveness assessment oforphan drugs: a scientific and political conundrum. Appl Health EconHealth Policy 2013;11:1–3.

[90] Clarke JT. Is the current approach to reviewing new drugs condemningthe victims of rare diseases to death? A call for a national orphan drugreview policy. CMAJ Can Med Assoc J 2006;174:189–90.

[91] Clay E, De Nicola A, Dorey J, et al. Validation of the first quality-of-lifemeasurement for patients with Huntington’s disease: the HuntingtonQuality of Life Instrument. Int Clin Psychopharmacol 2012;27:208–14.

[92] Behan L, Leigh MW, Dell SD, et al. Validation of a health-relatedquality of life instrument for primary ciliary dyskinesia (QOL-PCD).Thorax 2017;72:832–9.

[93] Nicod E, Kanavos P. Developing an evidence-based methodologicalframework to systematically compare HTA coverage decisions: amixed methods study. Health Policy 2016;120:35–45.

[94] Paulden M, Stafinski T, Menon D, et al. Value-based reimbursementdecisions for orphan drugs: a scoping review and decision framework.Pharmacoeconomics 2015;33:255–69.

[95] Gutierrez L, Patris J, Hutchings A, et al. Principles for consistent valueassessment and sustainable funding of orphan drugs in Europe.Orphanet J Rare Dis 2015;10:53.

[96] Thokala P, Devlin N, Marsh K, et al. Multiple criteria decision analysisfor health care decision making—an introduction: Report 1 of theISPOR MCDA Emerging Good Practices Task Force. Value Health2016;19:1–13.

[97] Marsh K, Ijzerman M, Thokala P, et al. Multiple criteria decisionanalysis for health care decision making—emerging good practices:Report 2 of the ISPOR MCDA Emerging Good Practices Task Force.Value Health 2016;19:125–37.

[98] Kolasa K, Zwolinski KM, Kalo Z, et al. Potential impact of theimplementation of multiple-criteria decision analysis (MCDA) on thePolish pricing and reimbursement process of orphan drugs. Orphanet JRare Dis 2016;11:23.

[99] Schey C, Krabbe PFM, Postma MJ, et al. Multi-criteria decision analysis(MCDA): testing a proposed MCDA framework for orphan drugs.Orphanet J Rare Dis 2017;12:10.

[100] Hughes DA, Tunnage B, Yeo ST. Drugs for exceptionally rare diseases:do they deserve special status for funding? QJM 2005;98:829–36.

[101] Scottish Medicines Consortium (SMC). SMCmodifiers used in appraisingnew medicines. Available from: http://solidarites-sante.gouv.fr/IMG/pdf/rapport_d_activite_2015_.pdf. [Accessed February 18, 2018].

[102] National Institute for Health and Care Excellence (NICE). Interimprocess and methods of the highly specialised technologies programmeupdated to reflect 2017 changes. 2017. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-guidance/NICE-highly-specialised-technologies-guidance/HST-interim-methods-process-guide-may-17.pdf. [Accessed February 18, 2018].

[103] Bouslouk M. G-BA benefit assessment of new orphan drugs in Germany:the first five years. Expert Opin Orphan Drugs 2016;4:453–5.

[104] Richter T, Janoudi G, Amegatse W, et al. Characteristics of drugs forultra-rare diseases versus drugs for other rare diseases in HTAsubmissions made to the CADTH CDR. Orphanet J Rare Dis 2018;13:15.

[105] Nicod E. Why do health technology assessment coveragerecommendations for the same drugs differ across settings? Applyinga mixed methods framework to systematically compare orphan drugdecisions in four European countries. Eur J Health Econ2017;18:715–30.

[106] Schlander M, Adarkwah CC, Gandjour A. Budget impact analysis ofdrugs for ultra-orphan non-oncological diseases in Europe. Expert RevPharmacoecon Outcomes Res 2015;15:171–9.

[107] Logviss K, Krievins D, Purvina S. Impact of orphan drugs on Latvianbudget. Orphanet J Rare Dis 2016;11:59.

[108] Hutchings A, Schey C, Dutton R, et al. Estimating the budget impact oforphan drugs in Sweden and France 2013–2020. Orphanet J Rare Dis2014;9:22.

[109] Schey C, Milanova T, Hutchings A. Estimating the budget impact oforphan medicines in Europe: 2010–2020. Orphanet J Rare Dis 2011;6:62.

[110] Comité économique des produits de santé (CEPS). Rapport d’activité2015. 2016. Available from: http://solidarites-sante.gouv.fr/IMG/pdf/rapport_d_activite_2015_.pdf. [Accessed February 17, 2017].

[111] EvaluatePharma. Orphan Drug Report 2013. Available from: http://info.evaluategroup.com/rs/evaluatepharmaltd/images/EP_OrphanDrugReport2013.pdf. [Accessed April 27, 2017].

[112] Kanters TA, Steenhoek A, Hakkaart L. Orphan drugs expenditure in theNetherlands in the period 2006–2012. Orphanet J Rare Dis 2014;9:154.

[113] National Institute for Health and Care Excellence (NICE). Managed AccessAgreement: Ataluren for treating nonsense mutation Duchenne musculardystrophy (nmDMD). Available from: https://www.nice.org.uk/guidance/hst3/resources/managed-access-agreement-july-2016-pdf-2553024061.[Accessed February 18, 2018].

V A L U E I N H E A L T H 2 1 ( 2 0 1 8 ) 4 9 3 – 5 0 0500