VALIDATION OF IN VITRO-IN VIVO CORRELA TION MODELS

David Young,! James A. Dowell,! Deborah A. Piscitelli,! and John Devane2

!IVIVR Cooperative Working Group Pharmacokinetics-Biopharmaceutics lab School of Pharmacy University of Maryland 100 Penn Street Room 540 Baltimore, Maryland 21201

2IVIVR Cooperative Working Group Elan Corporation Athlone Ireland

I. INTRODUCTION

7

A valid model can be defined as one that is a) well-grounded on principles or evi­dence, b) able to withstand criticism or objection, and c) capable of serving the purpose of the model. In the area of mathematical modelling, there are two criteria in the validation process: internal criteria and external criteria (1). The internal criteria establish the valid­ity of a model within the conditions of the model itself (e.g., the model's consistency and algorithmic integrity). The external criteria, on the other hand, are based on the model's purpose, theory, and data, considering those conditions that are not internal properties of the model.

The FDA has appropriately realized that, although in vitro-in vivo correlation (IVIVC) models can be developed, investigators need to evaluate the validity of the mod­els (2). The draft guidance on the development, evaluation and application of IVIVC mod­els states that the "validation is a broad term encompassing experimental and statistical techniques used during development and evaluation of a correlation which aid in deciding whether the correlation is both predictive and of good quality" (2). Although the emphasis in the Draft Guidance is on the predictive performance of the models (a part of the exter­nal criteria), the other components of the validation process cannot be ignored given the definition of validation within the guidance. In order to thoroughly understand the valida­tion of an IVIVC model, this chapter will briefly describe 1) other components of the vali­dation process that one should consider, 2) the methods that are discussed in the FDA Draft Guidance, and 3) alternate approaches to the validation ofIVIVC models. The inten-

In Vitro-in Vivo Correlations, edited by Young et a/. Plenum Press, New York, 1997 87

88 D. YoungetaL

tion of this chapter is not to be an exhaustive review of the issues associated with IVIVC model validation (e.g., experimental design, number and type of formulations). Instead, the purpose of this chapter is to discuss some approaches that can be used in the validation of Level A IVIVC models.

II. INTERNAL CRITERIA FOR VALIDATION

The internal criteria evaluate the appropriateness of the internal properties of the model (e.g., model consistency, algorithm) (1). A model's consistency refers to the suit­ability of the model and mathematical structure. The model should be mathematically logical and have no mathematical or conceptual misgivings. A very simple example ofthis is the ability to correctly set up the set of first order differential equations that describe a two compartment pharmacokinetic model with first order elimination. If an error is made setting up these equations, the results from this model would not be reliable. Another con­sideration regarding model structure, especially for complicated compartmental pharma­cokinetic models, is that a unique solution for the equations must exist, or the results obtained would be meaningless.

The algorithm or method by which we obtain estimates of the model parameters must also be evaluated. The algorithm must be numerically and statistically appropriate given the model and the type of data. Usually established software is used for these analy­sis and the integrity of the algorithm has already been tested. Newer methodologies and software, however, must be validated and cross-validated with proven methods. Errors due to the implementation of a less than robust algorithm, soft termination criteria, or large round off errors are examples of a solution method or software that does not meet internal criteria.

III. EXTERNAL CRITERIA FOR VALIDATION

There are four external criteria for model validation: theoretical, heuristic, empirical, and pragmatic validity (1). The theoretical validity must be continually applied to a model at all levels of development. It is a criteria which states that the model must be consistent with accepted theories or previously validated models. Parameter values, model identities and model structure should not conflict with the proven and validated information avail­able. Also, any expressions and relationships derived from the model must be able to exist with the proven science. For instance, a drug is modeled by a linear two compartment model, yet is known to be eliminated by means of a saturable enzymatic conversion. A linear model would probably be inappropriate in this situation.

The potential value of a model must also be considered as a practical question be­fore a model is even developed. A model may be used to identify important relationships, explain an observation, to test a possible hypothesis, or discover an alternative hypothesis. The failure of a model to pass tests of heuristic validity indicates a model is not useful. Heuristic validity is a qualitative question asked at the onset of modeling and when a model is established. It questions the potential value of the developed model.

Empirical validity involves looking at the overall goodness-of-fit of a model. There are many proven tests available to determine empirical validity of a model, all of which examine the data and quality of the fit. The majority of these methods examine the residu­als in some manner.

Validation of in Vitrtr-in Vivo Correlation Models 89

The last part of the external criteria is an evaluation of the pragmatic validity. The pragmatic validity determines to what extent the model satisfies the purpose of the model. This purpose often includes either a desire to describe the biological system or to predict what occurs under new conditions. For example, if the purpose of the model is to predict the in vivo response from the in vitro dissolution, then the predicted in vivo response (based on the in vitro data from a given formulation) must be statistically compared to the observed in vivo response when the formulation is administered to human volunteers. Re­gardless of the specific purpose, pragmatic validity often requires some evaluation of the model's ability to represent the actual data. Most tests for this include an evaluation of a models ability to predict the observed data.

IV. FDA DRAFT GUIDANCE ON INTERNALIEXTERNAL PREDICTABILITY

Since the purpose of the IVIVC model is to predict the in vivo response from an in vitro dissolution profile, the evaluation of these models in the FDA Draft Guidance has emphasized their predictive performance. Two approaches to estimate this performance are internal and external predictability.

Internal predictability evaluates how well the IVIVC model describes the data used to develop the model. This approach is recommended for all IVIVC analysis. The Draft Guidance states that the internal predictability can be assessed using the method of cross­validation, as long as three or more formulations are used in the model development. Cross-validation is widely used in regression analysis to validate models but, in the con­text of IVIVC, the term cross-validation is slightly different. The procedure of cross-vali­dation in IVIVC is:

1. leave the data from one formulation out and develop the model with the remain­ing formulations

2. predict the in vivo response of the formulation left out from the in vitro dissolu­tion of the formulation and the IVIVC model

3. calculate the prediction error by comparing the observed in vivo response to the predicted response

4. repeat steps 1-3 for each formulation

Other measures of the internal predictability which are not discussed in the draft guidance are some of the measures used in evaluating the goodness-of-fit of the model (i.e., empirical validation). These measures are the weighted residual sum of squares and the weighted residual plots. These measures should be evaluated when all the formulations are used in developing the model and when comparing alternate models.

External predictability is the second method of evaluating the IVIVC model. This approach is a more comprehensive evaluation of the ability for IVIVC model to predict and should be used when a) internal predictability is not conclusive, b) only two formula­tions with different release rates are available to develop the IVIVC model, c) the correla­tion is being developed for a drug with narrow therapeutic index, and d) whenever one desires a more comprehensive validation of the model. This approach determines how well the IVIVC model describes data which are not used in the model development. The ideal situation is that the formulations not used in the model development should be dif­ferent enough in release rate to appropriately evaluate the predictability of the model. The procedure of calculating the external predictability is:

90 D. Young et al.

1) develop the IVIVC model with a given set of formulations 2) predict the in vivo response of a formulation not used in the model development

based on the in vitro dissolution of that formulation and the IVIVC model 3) calculate the prediction error by comparing the observed in vivo response to the

predicted response

V. PREDICTABILITY METRICS

Given that the validity of the IVIVC model depends on the internal and/or external predictability, their are two other features of the validation that are important: predict­ability metrics and statistical criteria for defining "good" predictability or a valid model. In the previous description of predictability, the metric used to evaluate the predictability has been described simply as the prediction error. However, the most appropriate metrics to measure bias, precision, or both in this application are still under investigation. Some of the prediction metrics being evaluated are prediction error, absolute value of the predic­tion error, squared prediction error, and root mean squared prediction error.

VI. STATISTICAL CRITERIA FOR A VALID MODEL

The statistical approaches to determine the validity of an IVIVC model is another area of active research. There are a number of approaches that one can use to determine how well an IVIVC model predicts in vivo response for both internal or external predict­ability.

The first statistical approach is to evaluate the accuracy or bias and the precision of the predictions by statistically comparing the mean prediction error (bias) and root mean squared error to zero (e.g., HO: Mean Prediction Error = 0, Ha: mean prediction error not equal to zero). A similar approach has been more fully described by Sheiner and Beal in 1977(3).

A second approach statistically determines if the IVIVC model predicts better than a naive model. One naive model that has been discussed by the FDA (and presented in the Draft Guidance)is the naive averaged model. This model actually represents the mean in vivo response at each time for the formulations used in the IVIVC model. This mean in vivo response is then assumed to have either the in vitro dissolution of the formulation left out in cross-validation or the in vitro dissolution of the formulation not used in external validation. The naive averaged in vivo response at each in vitro value can then be com­pared to the observed in vivo response for the formulation left out or not used in the model development. The prediction error can then be calculated. The predictability of the naive averaged model can then be compared statistically to the IVIVC model.

To evaluate the relative accuracy (e.g., prediction error) and/or precision (e.g., abso­lute prediction error) of the IVIVC model with the accuracy and precision of a naive model, the null hypothesis for this approach would be defined as: the prediction metrics for the IVIVC model and the naive model are statistically equivalent at a p<O.OS. The al­ternative hypothesis would be that the prediction metrics for the IVIVC model is statisti­cally less than the naive model. The IVIVC model is then considered valid only when both the accuracy and precision are significantly smaller in the IVIVC model. This approach

Validation of in Vitro-in Vivo Correlation Models 91

can also be modified to compare the IVIVC model to another IVIVC model (e.g., linear model) instead of the naive model.

A third approach uses the principles of resampling statistics to determine the prob­ability that the predictability from the IVIVC model may occur just "by chance" (4). There are a number of ways to approach this problem and one is presented below:

1) calculate the change in the in vivo response at each time point for each formula­tion used in developing the model

2) randomly choose from any of the formulations the first in vivo response value 3) randomly choose the change in the in vivo (calculated from step 1) between the

first time point and second time point for any formulation and add it to the value obtained in step 2.

4) repeat the random choice at each time point and add it to the in vivo response calculated at the previous time point. The final curve represent an in vivo re­sponse that may occur "by chance".

5) the "by chance" in vivo response is then assumed to have either the in vitro dis­solution of the formulation left out in cross-validation or the in vitro dissolution of the formulation not used in external validation. For cross-validation, the "by chance" model will be repeated for each formulation left out.

6) the ability of the IVIVC model to predict the formulation not used in developing the IVIVC model can then be compared to the predictability of the "by chance" model.

7) this is repeated 100-1000 times and one counts how many times the IVIVC model prediction metrics is less than the "by chance" metrics. If the IVIVC model predictability is less than or equal to the "by chance" model more than 95% of the time, than the IVIVC model predictability could not have occurred by chance and the model is valid.

VII. CONCLUSION

Although the methods used to develop an IVIVC model are critical, the validation of the IVIVC model is equally important. This chapter has attempted to describe some of the principles of IVIVC model validation. Additional research is presently underway to evalu­ate the different approaches to validation, to determine which metric is most important, and to investigate the use of more complex resampling statistics to develop and validate IVIVC models.

REFERENCES

I. "The Validation of Models of Metabolic and Endocrine Systems", Chapter 9, The Mathematical Modeling of Metabolic and Endocrine Systems: Model Formulation, Identification, and Validation. E.Carson, C. CobelIi, L. Finkelstein, 1983, John Wiley & Sons, Inc., Toronto, Canada

2. Guidance for Industry: Extended Release Solid Oral Dosage Forms. Development, Evaluation and Applica­tion of In Vitro/In Vivo Correlations. Center for Drug Evaluation and Research (CDER), July I, 1996, FDA Draft Guidelines or Chapter 25 of this publication.

3. Scientific Commentary: Some Suggestions for Measuring Predictive Performance. Sheiner, L.B., Heal, S. Journal of Pharmacokinetics and Biopharmaceutics, Vol. 9, No.4, 1981.

4. Resampling Stats: User s Guide by Julian Simon, 9/1995, Resampling Stats, Inc., Arlington, Virginia.