1

EMEA Performance Indicators Pre-Authorisation

Bo AronssonEMEA

EMEA-EFPIA Info Day 2009

2

Contents

• Analysis Period• EMEA questionnaires• Results• Summary• Work in progress (Predictors of outcome)• Conclusions

3

Analysis Period

• EMEA Data Set: All applications with outcome between 1 January 2003 and 31 December 2008– Period of questionnaires follows annual reporting

to Management Board– Source:

• Questionnaires to (co-)rapporteurs • Scientific Memory Database

• EFPIA Data Set: October 06-October 08

4

EMEA QuestionnairesEMEA Questionnaires• Two versions have been used in 2003-2008

• “Old” version, implemented in 2000

– 10-point Scale (0 dissatisfied to 10 satisfied)

• “New” version implemented in 2007– Keep some of the same domains from “old” questionnaire

– Includes new domains (e.g., Scientific Advice)

– 5-point Likert Scale (1 agree to 5 disagree)

– Note: validation ongoing

• Questionnaires administered after day 80• Average scores between Rapporteur/Co-rapporteur, per product

• Exclusion of duplicates

5

EMEA Questionnaires Compared

5Yes-RMP/PVP

1Yes-Communication

4Yes-Scientific Advice

-3YesSPC, PL, Labelling

-3Q NC CStudy Reports

-3Q NC CSummary

-2NC COverview

11Q NC C3Q NC CEvidence-Data/Design3Q NC C 3Q NC CDossier Presentation

No.V2007No.V2000Item

Q= quality, NC=non-clinical, C=clinical, CPh=clinical pharmacology; CE=clinical efficacy, CS=clinical safety, PVP=pharmacovigilance plan.

6

Data Set 2003-2008 (N=209)• Allows to explore 2 domains and Parts of

Dossier– Presentation of the Dossier for Q, NC and C – Evidence (Data/Studies) included in the dossier for

Q, NC and C

209443643333221No. Questionnaires ("new" +

"old")

74636186928970Compliance (%)281705950363630No. Outcomes

31301No. "new" quest.

178143543333221No. "old" quest.

Total200820072006200520042003Year

7

Product Characteristics (N=209)

40.6785Scientific Advice

75.12157PositiveOutcome

13.8729Other

7.1815V

11.0023N

23.9250L

17.7037J

7.1815C

5.2611B

13.8829AATC

26.7956Orphan StatusPercentFrequency

8

Questionnaire Results

•All scores converted to 10-point Scale (0 dissatisfied to 10 satisfied)

9

Presentation of Dossier (N=209)

Quality Non-clinical Clinical

02

46

810

Sco

re

10

Evidence by Module (N=209)

Quality Non-clinical Clinical

02

46

810

Sco

re

11

Presentation B

y Time (N

=209)

0.001.00

2.003.004.00

5.006.007.008.00

9.0010.00

1-Jan-03

1-May-03

29-Aug-03

27-Dec-03

25-Apr-04

23-Aug-04

21-Dec-04

20-Apr-05

18-Aug-05

16-Dec-05

15-Apr-06

13-Aug-06

11-Dec-06

10-Apr-07

8-Aug-07

6-Dec-07

4-Apr-08

2-Aug-08

30-Nov-08

Outcom

e

Score

Quality

Non-clinical

Clinical

Poly. (Q

uality)P

oly. (Non-clinical)

Poly. (C

linical)

12

Evidence by M

odule by Time (N

=209)

0 1 2 3 4 5 6 7 8 9 10

1-Jan-03

1-May-03

29-Aug-03

27-Dec-03

25-Apr-04

23-Aug-04

21-Dec-04

20-Apr-05

18-Aug-05

16-Dec-05

15-Apr-06

13-Aug-06

11-Dec-06

10-Apr-07

8-Aug-07

6-Dec-07

4-Apr-08

2-Aug-08

30-Nov-08

Outcom

e

Score

Quality

Non-clinical

Clinical

Poly. (Q

uality)P

oly. (Non-clinical)

Poly. (C

linical)

13

Evidence by Orphan (N=209)

No Orphan

24

68

10

Non-clinical

No Orphan

02

46

810

Quality

Sco

re

No Orphan

24

68

10

Clinical

14

Is the Score Associated with Outcome and Clock-stop?

15

Success by Outcome Year (N=209)

IsPos 0 1

PERCENT

0

10

20

30

40

50

60

70

80

90

100

YearFinal2003 2004 2005 2006 2007 2008

16

Clinical Evidence versus Outcome (N=209)

Neg. Pos.

24

68

10

Data Clinical

Sco

re

17

Average Score and Clock-stop (N=209)

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

10.00

0 200 400 600 800

Clock-stop (Days)

Ave

rage

Sco

re(a

ll Pr

esen

tatio

n/Ev

iden

ce fo

r Q, N

, C)

18

Summary• Majority satisfaction

– No new time trends– Orphan status associated with lower satisfaction with

Evidence for all modules (and Presentation, data not shown)

• Satisfaction with Clinical Evidence associated with outcome and clock-stop

• Need to improve compliance with questionnaire• Future

– Further validate new questionnaire and explore new domains (work in progress)

– Predictors of Outcome (work in progress)

19

Predictors for Outcome - SA

Work in progress

20

Proportion of Proportion of MAAsMAAs that received SA (by that received SA (by outcome year)outcome year)

38%

47%

56%

0%

10%

20%

30%

40%

50%

60%

2006 (n=50) 2007 (n=59) 2008 (n=70)

SA given

21

Distribution of Scientific Advice over eligibility Distribution of Scientific Advice over eligibility --20082008

Eligibility with/without SA 2008

6 5

16

11

1

1

3

18

5 40

0

5

10

15

20

25

30

35

Biotech MandatoryIndication

Orphan New ActiveSubstance

Signif icantInnovation/Patients

interest

WHO Generic

No SA

SA

22

Did the Company follow SA?

• 31 “new” questionnaires in study period– 16 with SA given– 6/16 (35%) show poor

compliance according to Rapporteurs (score <5)

• Is SA or compliance to SA related to outcome?

Compliance

PoorGood

23

Size of company, success rate and Compliance with SA

36%29%50%56151+***

63%34%72%5321-150**

84%46%89%83Top 20 largest*

Compliance with SA

Proportion with SA

Success rate

Number applications

Pharmasize

*Top 20 largest (n=83) defined as being among the 20 largest companies **21-150 (n=53) defined as being among the 21 – 150 largest companies ***151+ (n=56) defined as not being among the 150 largest companies based on Total revenues 2005 according to Scrips Pharmaceutical League Tables 2006.

Regnstroem et al., (in manuscript)

24

Predictors of Outcome

Data on ranking was only available for 148 applications (work in progress)Stepwise logistic regression. Compliance, retrospectively assigned in Regnstroem et al. (in manuscript)

78.3111.1759.593SA & Compliant vs. (No SA or Not Compliant)

0.7440.1220.301Major Objection on RCT (No vs Yes)

1.9011.161.485Clinical Evidence (0-10)

3.361.0791.904Company Size (1: 151+; 2: 21-150; 3: Top 20 largest)

95% WaldConfidence Limits

Point EstimateEffect

Odds Ratio Estimates

25

Conclusions

• Most important factors associated with outcome – Compliance with Scientific Advice– Company Size– Rapporteurs’ satisfaction with Clinical

Evidence submitted– Major Objections on the Lack or

Randomised Controlled Trials

26

Acknowledgments

• Analysis– Francesco Pignatti– Franz Koenig– Jan Regnstroem

• Data Management– Esther Cozar Calvente– Nadia Kresse– Monica Simeoni