+ All Categories
Home > Documents > Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert...

Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert...

Date post: 18-Sep-2018
Category:
Upload: vuthuan
View: 214 times
Download: 0 times
Share this document with a friend
20
A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014 www.leadscope.com 1 [email protected] Leadscope® Genetox Expert Alerts White paper A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014
Transcript
Page 1: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 1 [email protected]

Leadscope® Genetox Expert Alerts

White paper

A new ICH M7 compliant expert alert system to predict the

mutagenic potential of impurities

March 2014

Page 2: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 2 [email protected]

A new ICH M7 compliant expert alert system to predict the

mutagenic potential of impurities

Abstract The current International Conference on Harmonisation (ICH) M7 guideline on drug impurities states

that two distinct in silico methodologies can be used to qualify certain drug impurities as not mutagenic.

This paper outlines the development and use of a new expert rule-based system to predict the results of

a bacterial mutagenesis assay. In the development of this system, an initial library of mutagenicity

structural alerts was identified from the literature. This process included consolidating the same or

similar alerts cited in multiple publications. Information on plausible mechanisms was collected

alongside the structural definitions. Factors that deactivate the alerts were also identified from the

literature and through an analysis of the corresponding data using the Leadscope data mining software.

Over 200 distinct alerts were identified and these alerts were further validated against a reference

database of over 7,000 chemicals with known bacterial mutagenesis results. Only validated alerts with a

sufficiently strong association with positive expert-reviewed calls from Salmonella and E. coli strains

were included. A prediction of the bacterial mutagenesis assay can be made using these validated alerts;

however, this is only possible when the compound is within the applicability domain of the alert system.

In addition, a confidence score based upon information collected for each alert is provided alongside the

positive or negative call. This paper outlines the expert alerts system and presents validation results,

both as a standalone system and in combination with statistical-based approaches and available

experimental data.

Introduction Impurities are generated as part of the pharmaceutical manufacturing process. They result from using

different reagents, catalysts and solvents in the synthesis of the drug substance and are also attributable

to subsequent degradation. The International Committee on Harmonisation (ICH) has recently issued a

draft guidance (currently at Step 2) that covers the qualification of mutagenic impurities [1]. The

purpose of the guidance is to ensure that any impurities present in the drug substance pose a low-level

of risk of causing cancer. As such, the guideline’s focus is on identifying DNA reactive substances, usually

detected using the bacterial reverse mutation assay (Ames) [2]. In the absence of carcinogenicity or

Ames data for the actual or potential impurities, a computational structure-activity analysis can be

performed to help understand whether a substance can be classified as not mutagenic. Where this

negative classification is not possible, further in vitro or in vivo tests are required to support a negative

classification or the impurity should be controlled below the limits established in the guideline.

To perform the computational structure-activity analysis, the guideline states that two complementary

in silico methodologies should be used in the assessment. One should be expert rule-based and the

second should utilize a statistical-based methodology. The methods used in the assessment should be

Page 3: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 3 [email protected]

compliant with the OECD validation principles [3] and should be reviewed using expert analysis,

especially in cases where the results are conflicting. If the results from the two methodologies indicate

no predicted mutagenic potential, the impurity can be classified as having no mutagenic concern.

An expert rule-based system is one of the methodologies required to support an ICH M7 compliant

computational analysis. In this type of system, knowledge concerning different structural features that

are associated with mutagenicity is encoded as rules in the system. If these structure-activity rules map

onto the compound being evaluated, then it may indicate the compound is mutagenic. A number of

current systems make use of this rule-based methodology including ToxTree[4], HazardExpert [5], DEREK

[6,7], and Oncologic [8]. ToxTree is a rule-based system that incorporates a module for mutagenicity

which includes the Benigni-Bossa rule-base. HazardExpert contains a mutagenicity module which makes

use of a series of toxicophores from the literature. DEREK is an alerts-based system that incorporates

mutagenicity rules and generates several levels of prediction including “Certain”, “Probably”,

“Plausible”, “Equivocal”, “Doubted”, “Improbable”, “Impossible”, “Open”, “Contradicted”, and “Nothing

to report”. At this time, there is no domain of applicability analysis and hence no negative predictions.

Oncologic is an expert system that makes predictions based on a series of user decisions making it

difficult to predict batches of chemicals.

The approaches discussed were not developed to specifically address the criteria of the ICH M7

guideline. One of the main disadvantages of all current approaches is their lack of an applicability

domain assessment (one of the OECD validation principles). This is a particularly important part of a M7

submission, since the lack of an alert should never be used to state that a compound has no mutagenic

concern. This would imply that all possible structural alerts have been identified and that chemistry is

static. Aryl boronic acid is an example where a new structural alert has been recently identified [9].

In addition, these approaches are not generally quantitatively assessed using a large database of Ames

results. This is important since the guideline explicitly states the necessity for the alerts system to be

predictive of the bacterial mutation assay. This also supports transparency in the prediction results by

providing the data used to assess and justify the structural alerts.

Decisions compliant with ICH M7 guidance require a clear positive or negative call. While less precise

predictions may be helpful when using these tools in early discovery for prioritization, they are

ambiguous for regulatory needs. Prediction results such as “Probable” or “Plausible” need to be

translated into a positive/negative call. This level of ambiguity permits inconsistent application of these

tools.

Finally, the tools described do not address the complex relationships between regulatory acceptable

statistical model results, alert results, available experimental data and the necessity for expert opinion

support tools that are essential when supporting the ICH M7 guideline.

Page 4: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 4 [email protected]

The following paper outlines a new expert rule-based system designed to address the needs of

regulators and pharmaceutical scientists who must implement the ICH M7 guidance. The alert

knowledge base is built using a quantitative assessment of publicly-cited alerts using a large database of

Ames results. For each alert, deactivating factors are enumerated through an analysis of the available

Ames data and the literature. The alerts are organized hierarchically, to precisely define the relationship

between any alerts with any specific classes of concern. This rule base is annotated with plausible

mechanisms collected from the literature. This linkage between the data and the biological

interpretation of the alerts is critical to support a transparent assessment of the results. This paper

outlines the methods used to build the alerts knowledge base and the methodology used to make

predictions. Validation results are presented and a discussion of its use to support the ICH M7 guidance

is provided.

Methods

Reference set An extensive high quality genetic toxicity database containing the results of the bacterial mutagenesis

assay along with chemical structures has been used to support the development of this rule-based

expert system. This database, referred to as a reference set, supports the evaluation of the expert rules.

The use of the term reference set is to contrast this database from a training set used in QSAR model

development. This set will not be used to ‘build’ the alerts system, rather it will be used (1) to assess the

alerts cited in the literature (in combination with an expert judgment), (2) to support applicability

domain assessment and (3) to generate scores that are reflective of the confidence in the alert.

Two high quality databases were used, each comprised of data from numerous sources: the training sets

used to build QSAR models at the US FDA with Research Collaboration Agreement partners (RCA-QSAR)

[10,11] and the Leadscope SAR 2014 database [12,13].

The RCA-QSAR database is comprised of 3,979 chemicals, with data on overall Salmonella strains as well

as 1,198 chemicals with a composite TA102 / E. coli strain call. These datasets contain non-proprietary

data harvested from FDA approval packages and the published literature.

The Leadscope SAR 2014 Genetox Database is an extensive collection of genetic toxicology studies

(including bacterial mutagenesis, chromosome aberration, mammalian mutagenesis, and in vivo

micronucleus) and contains 6,805 chemicals with graded bacterial mutagenesis calls. Information was

collected from the electronic source or manually harvested to create the data set. Specifically, the

following sources were used: (1) the US Food and Drug Administration (FDA) Centre for Food Additives

and Applied Nutrition (CFSAN) Food Additive Resource Management system (FARM) and Priority based

Assessment of Food Additives (PAFA) [14]; (2) the US FDA’s Center for Drug Evaluation and Research

(CDER) Pharmacology Reviews based on the new drug approval (NDA) documents [15]; (3) the Chemical

Page 5: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 5 [email protected]

Carcinogenicity Research Information System (CCRIS) [16]; (4) the National Toxicology Programs (NTP)

genetic toxicology database [17]; (5) the Tokyo-Eiken database [18]; (6) and other publications.

The two databases were combined for the purposes of developing a comprehensive reference set for

use in supporting the development and application of the expert rule-based system. The process of

combining the database was performed in two steps. Firstly, the chemical structures were merged

through using a chemical registration system to assign a unique identifier to each chemical and merging

entries on the basis of this identifier. Next the graded endpoints for Salmonella and E. coli were

combined from the different sources, resulting in a database of over 7.000 chemicals each with a

positive/negative overall bacterial mutation call. The distribution of positive and negative data is

summarized in Figure 1. The reference set also covers a diverse collection of compounds since they have

been derived from many different sources, including pharmaceuticals, pesticides, industrial chemicals

and food additives. This diversity is illustrated by clustering the reference set using the Leadscope

software, using a cut-off distance of 0.5 [19-24]. A summary of 4 out of the 2,269 clusters is presented in

Figure 2. There were 1,220 clusters with two or more examples, and 1,049 singletons (clusters with one

example).

Figure 1: Summary of the database endpoint content

Figure 2: Clustering of the reference sets by structural similarity

Page 6: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 6 [email protected]

Alert compilation A bacterial mutagenesis structural alert is based on a molecular substructure defining a reactive center,

as illustrated in Figure 3. In this figure, an aziridine substructure is shown which had been cited in

multiple publications as a structural alert for mutagenicity ("aromatic and aliphatic aziridinyl" [25];

"aziridine" [26]; "oxiranes and aziridines" [27]; "SA_7: epoxides and aziridines" [28,29]). For any

mutagenicity structural alert, there should be a relationship between this reactive center identified

within the molecule and its ability to either directly or indirectly (through one or more metabolic steps)

interact with DNA. For example, in [29] aziridines have been described as “… extremely reactive

alkylating agents that may react by ring-opening reactions … activity of these compounds depends on

their ability to act as DNA cross-linking agents, via nucleophilic ring-opening of the aziridine moiety by

N7 positions of purines.”

Figure 3: Aziridine example alert

A number of publications have published summaries of proposed alerts [25-29,31-34]. The first step in

the process of developing an expert alert-based system is to encode these published alerts as

substructural definitions. This involves defining the one or more substructures that define the alerts.

Many publications also include a description of the mechanistic basis for the proposed alerts and this

rational is captured alongside the structural definitions.

Next, the alerts are consolidated wherever possible; however, this process is challenging since not all

publications define the same alert in the same way. This is illustrated in Figure 4 where four papers cite

the alert aziridine in different ways. The Ashby 1988 paper has substitutions on both carbons and any

substitutions on the nitrogen is ambiguous [25], the Kazius 2005 paper places no restrictions on any

attachments [26], the Bailey 2005 and Benigni 2008 & 2011 papers’ definitions include both

epoxides/oxiranes in the same definition [27-29]; however, no restrictions on any of the atoms are

presented in Bailey but in the Benigni papers the nitrogen should have one attachment to any atom. The

process of consolidating the alerts must take into account the different definitions and ways the alert

have been defined. It was performed on a case-by-case basis. The plausible mechanisms are also helpful

in refining the alerts’ definitions.

Page 7: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 7 [email protected]

Figure 4: Example of alerts definitions from different papers

Once these alerts have been encoded and consolidated, they are organized hierarchically, as illustrated

in Figure 5. In this example, the alert 28: nitrosamine is a parent alert, with three more specific child

alerts related to it. There are several reasons for doing this. Firstly, it helps in establishing a mechanistic

explanation, particularly where any child alert is lacking or has limited mechanistic information, as it

may be inherited from the parent alert. Secondly, when the expert alerts are used to make prediction, a

score is calculated reflecting the precision of the alert. If more than one alert matches the target

compounds being assessed, then the most precisely defined alert (i.e. the alert in that is closest to any

terminal node in the hierarchy) is used to generate this score. Finally, providing a summary of the

alert(s) that fire is useful; however, a list of many related alerts is not helpful. This hierarchy can be used

to select the most precise and relevant alert to present.

Figure 5: Hierarchical organization of alerts

Page 8: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 8 [email protected]

In addition to the primary alert, it is also important to define any factors that would deactivate the alerts

as a result of electronic or steric effects or by blocking an important metabolic step. For example, for the

alert primary aromatic amine, an acidic group in the para position “…prevents proton abstraction from

NH2 group of anilines … by the ferric peroxo intermediate of CYP1A2.” [30]. Unfortunately, there is

limited information in the literature on precise factors that deactivate the alert, therefore, an exercise of

data mining the reference set was undertaken to better understand these factors. This process used the

informatics tools available in the Leadscope software to identify and quantitatively assess deactivating

factors [19-24]. This process used the 27,000 pre-defined structural features in Leadscope and

generated new chemical scaffolds associated with negative bacterial mutagenicity Any deactivating

fragments identified were quantitatively evaluated using the reference set. Figure 6 illustrates three

example deactivating fragments for the alert aromatic nitro.

Figure 6: Examples of deactivating fragments

The literature also describes sub-classes of alerts that represent cohorts of concern or highly active

subclasses. These are essentially another alert; however, they can be linked with the other alerts

through the hierarchical relationship as described earlier. For example the benzene, 1-amino(NH2)-, 4-

aryl- is identified as a specific sub-class of the primary aromatic amines since “these chemical features

make the nitrenium ion (DNA-reactive intermediate) more stable due to the electron donating property

of the benzene ring, and thus more reactive with DNA” [35]. There are also limited examples in the

literature so a data mining exercise of the reference set was undertaken to identify these classes.

The complex relationships between the alerts, and the deactivating fragments and/or highly active

subclasses (all organized in a hierarchy) as well the accompanying non-structural information such as the

source of the alerts and the mechanistic rationale for the alert is encoded as an XML document [36]. The

alert can also be defined as one or more “Rules” to accommodate examples such as polycyclic aromatic

hydrocarbons, which require the definition of a number of unique cyclic systems to fully define all

possible planar systems for this alert. This XML document is linked to the structural definitions for the

Page 9: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 9 [email protected]

alerts, deactivating fragments, and the highly active subclasses, which are defined in the SD file format

[37]. The information contained in the XML file as well as the structural definitions, provides the source

of the information used by the expert alerts system. Figure 7 provides an example of an XML alert

record for an alert.

Figure 7: Example of an XML record to represent an alert

Alert Assessment The purpose of the alert assessment process is to establish which alerts should be used to make

predictions. These “active” alerts should have clear evidence that they would be predictive of a positive

outcome in the bacterial mutagenesis assay, as required by the ICH M7 guideline. To make this

assessment, alerts are initially assigned as active where there are more than five examples of

compounds in the reference set that match the alert and when greater than 70% of those examples are

positive. However, it is not enough to rely solely on the number of examples for each alert. A second

assessment is performed by removing those compounds that also contain other alerts. Where

compounds containing only the alert in question also shows good correlation, then there is more

convincing evidence for the alert to be assigned as active. This assessment is illustrated in Figure 8 with

two examples. There is good supporting data to include the aziridine in the list of active alerts. Another

example of a potential alert is furans, a proposed alert for mutagenicity from the paper [30]. There is

some evidence of an association in the reference set with 143 examples since almost 66% were positive

(compared to 47% active in the entire reference set). However, if compounds that contain an additional

alert such an aromatic nitro, etc. are removed, there are only 33 remaining examples from the reference

set with 24% positive (lower than the average for the reference set). Hence, based upon this data there

is not enough evidence to suggest furan should be included as an active alert.

Page 10: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 10 [email protected]

Alert

Number of

examples (all) Precision (all)

Number of examples only

contain the alert (isolated) Precision (isolated)

24: aziridine 33 1 19 1

85: furans 143 0.6573 33 0.2424

Figure 8: Alert assessment

Alerts are also classified as “inactive” for the bacterial mutagenesis assay. These alerts show little

correlation with the data. In these cases a cut-off of five examples with less than 50% positive, which

represents the approximate percentage of active/inactive in the entire reference set, is used.

There is a final category which is assigned to the remaining alerts – “indeterminate”. Alerts where there

is insufficient number of examples to classify the alert as either active or inactive would fit into this

category. Alternatively, there may be a sufficient number of examples, but, the number of positive

examples is between the active and inactive cut-off (50% - 70% positives). One explanation is that there

may be insufficient information that can be gleaned from the data concerning deactivating factors or

highly active subclasses. Another group of indeterminate alerts exist when there is additional

information to question the alert. For example, it has been reported that acid halides show positive

Ames data only in the presence of and as a consequence of the use of DMSO [38]. Hence, this additional

knowledge could be used to assign them as indeterminate. Although, indeterminate alerts are not be

used to make any positive calls, they may be used in subsequent expert review. Since the alerts are

hierarchically organized, it is possible to have a mixture of active, inactive and indeterminate alerts at

various levels of the hierarchy which illustrates another benefit of this hierarchical organization.

Classification and scoring To use alerts to make predictions to support a regulatory decision, i.e. whether a test compounds should

be classified as positive or negative, it is necessary to devise a series of rules for classification. In

addition, it is important to understand whether the compound is in the applicability domain and hence

whether any prediction could be made at all.

A positive call is made for compounds within the applicability domain where an active alert matches the

test compound and there are no deactivating fragments in the specified relationship to the primary

alert. In Figure 9, the two chemicals presented would be predicted as positive. Chemical A matches a

Page 11: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 11 [email protected]

terminal hydrazine (highlighted in red) and chemical B matched both an aromatic nitro as well as an

aziridine.

Figure 9: Examples of positive results

Where an active alert matches, yet a deactivating fragment is also present in the portion of the structure

as the active alert, then a negative call is made. A negative call is also made where no alert is identified

and the test compound is within the applicability domain. In Figure 10, both chemicals C and D are

predicted negative. In chemical C, an aromatic primary amine is present (shown in gray); however, there

is a corresponding deactivating fragment in the 2-position. In chemical D, no alerts are identified. A

prediction for both chemicals can be made because they are in the applicability domain of the alerts

based on their distances to at least one compound in the reference set.

Figure 10: Examples of negative results

Page 12: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 12 [email protected]

It is essential to only make a negative call for compounds that do not represent novel classes of

chemistry where no historical data has been collected. This assessment of the applicability domain is

made using a structural similarity cut-off to the reference set based on the Leadscope structural

fingerprint and a Tanimoto distance [19-24]. This ensures that negative predictions are not extrapolated

to novel areas of chemistry where there is no Ames data. A call of indeterminate occurs when only an

alert marked as indeterminate is identified yet the compound is still in the domain of applicability.

Figure 11 illustrates a not-in-domain result as well as an indeterminate result.

Figure 11: Examples of out-of-domain and indeterminate results

To accompany the positive or negative classification, a score is generated further indicate the

confidence in the prediction. This score reflects the ratio of positive/negative example compounds from

the reference set and is defined as precision. When multiple alerts match the test compound, the most

precise or highest score is selected. When a negative classification is made, the score is reflective of the

background positive/negative ratio for all reference set compounds with no active alerts. In Figure 9,

chemical A matches a terminal hydrazine which is 94% positive based upon the examples in the

database. Compound B in Figure 9 matched two alerts and the alert with the highest precision value is

selected, in this case aziridine. In Figure 10 compound C and D both have a precision score of 0.1567

which is the positive/negative ratio for all compounds that do not contain an alert in the reference set.

Consensus rules for ICH M7 The ICH M7 guidance states that two in silico methodologies need to be used in the assessment of

impurities; however, if data is available it can also be used. Thus a complex series of possibilities exist for

generating an overall final call. Generally, the availability of data will take precedence over a prediction.

If one or more prediction method indicates a positive, in the absence of data related the predicted

endpoint, the overall call is positive.

Page 13: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 13 [email protected]

Results The Leadscope Ggenetox Expert Alerts (described in this paper) have been implemented as part of the

Leadscope Model Applier v1.8 (alongside the existing statistical-based QSAR models). To assess the

performance of this alert system, two data sets were used: (1) the reference set (as described in

Reference set section) and (2) the Hansen data set. The Hansen set includes data described in the

Hansen et al. publication [39] where the full set includes 6,512 chemicals. However, 2,680 were in the

RCA-QSAR training set so were removed. A number of other chemicals were also removed based on

stereochemical considerations or their inability to be modelled leaving 3734 compounds. Figure 12

shows the performance results for the reference set. A positive/negative prediction was made for 6,949

of the 7,112 total number of chemicals (97.9% coverage). There were 5,868 correct predictions or 84.1%

concordance (2,749 true positive plus 3,119 true negative), with 528 false positives and 553 false

negatives. The values for sensitivity, specificity, negative predictivity as well as positive predictivity are

also presented. Figure 13 shows the corresponding performance statistics for the Hansen data set.

Concordance 84.1%

Sensitivity 83.3%

Specificity 85.5%

Negative Predicitivity 84.3%

Positive Predicitivity 83.9%

Coverage 97.9%

Figure 12: Performance statistics (reference set)

Concordance 79.6%

Sensitivity 87.0%

Specificity 69.3%

Negative Predicitivity 79.5%

Positive Predicitivity 79.6%

Coverage 98.6%

Figure 13: Performance statistics (Hansen)

In Figure 14, the performance of the alert system is compared to the performance of the M7 consensus

call for the Hansen data set. This analysis also takes into account the availability of data as well as the

Page 14: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 14 [email protected]

results from two statistical-based QSAR models for Salmonella and E. coli [10,11]. This M7 consensus call

results in improved performance, particularly for sensitivity and negative predictivity.

Hansen (alerts) Hansen (M7 consensus)

Concordance 79.6% 79.4%

Sensitivity 87.0% 94.8%

Specificity 69.3% 58.4%

Negative Predicitivity 79.5% 89.1%

Positive Predicitivity 79.6% 75.7%

Coverage 98.6% 99.7%

Figure 14: Performance statistics (M7 consensus for the Hansen set)

Discussion ICH M7 guidance outlines for regulators and pharmaceutical sponsors the process for qualifying a drug

impurity as having no mutagenicity concern. As part of this process a computational analysis can be

performed to qualify certain classes of impurities as negative. The guideline also states that any in silico

systems should adhere to the OECD validation principles where models should have: “1) a defined

endpoint, 2) an unambiguous algorithm, 3) a defined domain of applicability, 4) appropriate measures of

goodness-of-fit, robustness and predictivity, 5) a mechanistic interpretation, if possible.” The new expert

alert-based system described here is built to specifically make predictions for the bacterial mutation

endpoint following the ICH M7 guidance. The algorithm has been outlined in detail in this publication

including the methodology used to assess the applicability domain. The results from both an internal

and external validation have been presented which summarizes the system’s goodness-of-fit, robustness

and predictivity. Finally, when any alert matches a target test compound, a summary of the literature-

derived mechanistic basis for each matched alert is provided.

One of the guiding principles in the development of this system was transparency of the predictions.

This is implemented by linking the alerts to a quantitative assessment based on an extensive collection

of mutagenicity data. This not only provides a level of confidence in the results but also supports expert

judgment that may accompany the results. The alerts are defined such that it is possible to immediately

answer the questions – which literature sources did the matched alerts come from, how is the structure

of the alert defined, what is their mechanistic justification, what data supports the use of the alert (see

Figure 15). The system will also highlight the presence of any indeterminate alert and hence could be

used as part of any additional expert opinion.

Page 15: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 15 [email protected]

Figure 15: Explanation of alert firing

The system also addresses how results from both an expert alert-based system in combination with

statistical-based QSAR models as well as any available experimental data can be used to generate an

overall call to use in the assessment of the ICH M7 compliant results. The ICH M7 guidance includes

language stating that when an API has been tested and the results were negative, but an impurity was

predicted positive and both the API and the impurity share a positive alerting fragment, then it is

possible to argue that the impurity could be classified as negative. By highlighting the components of

both the statistical-based model as well as the expert-alerts based model that trigger the positive

results, it is possible to readily make this expert opinion, as illustrated in Figure 16.

Page 16: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 16 [email protected]

Figure 16: Looking at the weight of evidence

Conclusion A new expert alert-based system has been described to directly support the ICH M7 guideline for drug

impurities. This paper has described the process of developing the alert knowledge base as well as the

reference set used to quantitatively assess the alert rules. These alerts are based on well-defined

mutagenicity structural alerts from the literature. They have been assessed, alongside deactivating

factors as well as active subclasses (which represent possible cohorts of concern). The paper has also

described how predictions are made based on the presence of an alert with no defined deactivating

factors as well as determining whether the target test compound is similar enough to known classes of

chemicals to be predicted - that it is not trying to extrapolate to areas of chemistry the system has never

seen. The good validation results as well as its adherence with the ICH M7 guidance and OECD validation

principles allow this system to be used in the assessment of impurities with confidence.

Acknowledgements We would like to thank Naomi L. Kruhlak, Lidiya Stavitskaya, and Barbara L. Minnier from the US FDA

who, through the RCA agreement have provided Leadscope with access to the RCA-QSAR data set as

well as input into the system. We would also like to thank Errol Zeiger and Ron Snyder for their expert

advice as well as the beta testers from the pharmaceutical industry and consultants to the

pharmaceutical industry.

Page 17: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 17 [email protected]

References [1] Assessment and control of DNA reactive (mutagenic) impurities in pharmaceuticals to limit potential

carcinogenix risk (M7)

(http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM34

7725.pdf accessed: March 5th, 2014)

[2] OECD guideline for testing of chemicals: Bacterial Reverse Mutation Test 21st July 1997

(http://www.oecd-ilibrary.org/environment/test-no-471-bacterial-reverse-mutation-

test_9789264071247-en accessed March 5th 2014)

[3] OECD (2007). Guidance document on the validation of (Quantitative) structure activity relationships

[(Q)SAR] models.

(http://ihcp.jrc.ec.europa.eu/our_labs/predictive_toxicology/background/oecd-principles accessed

March 5th 2014)

[4] Benigni R, Bossa C, Jeliazkova N, Netzeva T & Worth A (2008b). The Benigni / Bossa rulebase for

mutagenicity and carcinogenicity – a module of Toxtree. EUR 23241 EN.

http://ecb.jrc.ec.europa.eu/qsar/publications/

[5] Smithing MP & Darvas F (1992). Hazardexpert: an expert system for predicting chemical toxicity. In:

Food Safety Assessment (JW Finlay, SF Robinson and DJ Armstrong). Washington, American Chemical

Society: 191-200.

[6] Ridings JE, Barratt MD, Cary R, Earnshaw CG, Eggington CE, Ellis MK, Judson PN, Langowski JJ,

Marchant CA, Payne MP, Watson WP & Yih TD (1996). Computer prediction of possible toxic action from

chemical structure: an update on the DEREK system. Toxicology 106(1-3), 267-279.

[7] Sanderson DM & Earnshaw CG (1991). Computer prediction of possible toxic action from chemical

structure; the DEREK system. Human and Experimental Toxicology 10, 261-273.

[8] Woo Y-T & Lai DY (2005). OncoLogic: A mechanism-based expert system for predicting the

carcinogenic potential of chemicals. In Predictive Toxicology (C Helma, ed), 385-413

[9] Michael R. O’Donovan,Christine D. Mee, Simon Fenner, Andrew Teasdale, David H. Phillips Boronic

acids—A novel class of bacterial Boronic acids—A novel class of bacterial Mutagen, Mutation

Research/Genetic Toxicology and Environmental Mutagenesis, Volume 724, Issues 1–2, 18 September

2011, Pages 1–6

Page 18: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 18 [email protected]

[10] L. Stavitskaya, B. L. Minnier, R. D. Benz, N. L. Kruhlak, FDA Center for Drug Evaluation and Research,

SOT 2013 “ Development of Improved Salmonella Mutagenicity QSAR Models Using Structural

Fingerprints of Known Toxicophores”

[11] L. Stavitskaya, B. L. Minnier, R. D. Benz, N. L. Kruhlak, FDA Center for Drug Evaluation and Research, “Development of Improved QSAR Models for Predicting A-T Base Pair Mutations “,GTA 2013b poster [12] Leadscope SAR Genetox Database 2013 User Guide

[13] Yang C, Hasselgren CH, Boyer S, Arvidson K, Aveston S, Dierkes P, Benigni R, Benz RD, Contrera J,

Kruhlak NL, Matthews EJ, Han X, Jaworska J, Kemper RA, Rathman JF and Richard AM. Understanding

Genetic Toxicity Through Data Mining: The Process of Building Knowledge by Integrating Multiple

Genetic Toxicity Databases, Toxicology Mechanisms and Methods, 2008, 18:2, 277 – 295

[14] Food and Drug Administration CFSAN EAFUS. 2007. U.S. FDA Center for Food Safety and Applied

Nutrition, Everything Added to Food in the United States.

(http://www.fda.gov/food/ingredientspackaginglabeling/foodadditivesingredients/ucm115326.htm

Accessed March 9, 2014)

[15] FDA CDER FOI site: http://www.accessdata.fda.gov/scripts/cder/drugsatfda. Accessed March 9,

2014

[16] CCRIS: http://toxnet.nlm.nih.gov/cgi-bin/sis/htmlgen?CCRIS Accessed March 9, 2014

[17] R W Tennant The genetic toxicity database of the National Toxicology Program: evaluation of the

relationships between genetic toxicity and carcinogenicity. Environ Health Perspect. 1991, 96: 47–51.

[18] Tokyo-Eiken. 2007. Tokyo Metropolitan Institute of Public Health, Mutagenicity of food additives.

[19] Roberts, G.; Myatt, G.J.; Johnson, W.P.; Cross, K.P.; Blower, P.E.; Leadscope: Software for Exploring

Large Sets of Screening Data. J. Chem. Inf. Comput. Sci., 2000, 40, 1302-1314.

[20] Johnson, D.E.; Blower, P.E.; Myatt, G.J.; Wolfgang, G.; Chem-tox informatics: Data mining using a

medicinal chemistry building block approach. Current Opin. Drug Disc. Develop., 2001, 4(1), 92-101.

[21] Cross, K.P.; Myatt, G.; Yang, C.; Fligner, M.F.;Verducci, J.S.; Blower, Jr., P.E. Finding Discriminating

Structural Features by Reassembling Common Building Blocks, J. Med. Chem., 2003, 46, 4770-4775.

[22] Blower, P.; Cross, K.; Fligner, M.; Myatt, G.; Verducci, J.; and Yang, C. Systematic analysis of large

screening sets, Curr. Drug Disc. Technol., 2004, 37-47.

[23] Yang, C.; Cross, K.; Myatt, G.; Blower, P.; Rathman, J. Building Predictive Models For PTP1B

Inhibitors Based On Discriminating Structural Features By Reassembling Medicinal Chemistry Building

Blocks, J. Med. Chem., 2004, 47, 5984 – 5994.

Page 19: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 19 [email protected]

[24] Blower, P.E.; Cross, K.C.; Eichler, G.S.; Myatt, G.J.; Weinstein, J.N.; and Yang, C. Comparison of

Methods for Sequential Screening of Large Compound Sets, Comb. Chem. High Throughput Screen.,

2006; 9:115-22

[25] Ashby J, Tennant RW. Chemical structure, Salmonella mutagenicity and extent of carcinogenicity as

indicators of genotoxic carcinogenesis among 222 chemicals tested by the U.S.NCI/NTP Mutat. Res.

1988, 204:17-115

[26] Bailey AB, Chanderbhan N, Collazo-Braier N, Cheeseman MA, Twaroski ML. The use of structure-

activity relationship analysis in the food contact notification program. Regulat Pharmacol Toxicol 2005;

42:225-235.

[27] Benigni R, Bossa C, Jeliazkova N, Worth A. The Benigni / Bossa Rulebase for Mutagenicity and

Carcinogenicity - A Module of Toxtree - EUR 23241 EN 2008

[28] Benigni R & Bossa C, Mechanisms of Chemical Carcinogenicity and Mutagenicity: A Review with

Implications for Predictive Toxicology, Chem Rev. 2011 Apr 13;111(4):2507-36

[29] Ciaravino V, Plattner J, Chanda S, An Assessment of the Genetic Toxicology of Novel Boron-

Containing Therapeutic Agents, Environmental and Molecular Mutagenesis 54:338-346 (2013)

[30] Enoch SJ & Cronin MTD, A review of electrophilic reaction chemistry involved in covalent DNA

binding, Critical Reviews in Toxicology, 2010, 40, 728-748

[31] S. Enoch, M.T.D. Cronin Development of new structural alerts suitable for chemical category

formation for assigning covalent and non-covalent mechanisms relevant to DNA binding , Mutation

Research 743 (2012) 10-19

[32] Kazius J, McGuire R, Bursi R. Derivation and Validation of Toxicophores for Mutagenicity Prediction.

J. Med. Chem., 2005, 48: 312-320

[33] OECD Report No.120 Report of the Expert Consultation on Scientific and Regulatory Evaluation of

Organic Chemistry Mechanism-Based Structural Alerts for the Identification of DNA Binding Chemicals

Part 1 and Part 2 (2010)

[34] Shamovsky I et. al. J Am Chem Soc. 2011 Oct 12;133(40):16168-85

[35] Galloway S.M. et al. Regulatory Toxicology and Pharmacology 66 (2013) 326–335

[36] Harold ER, Means WS (2009), XML in a Nutshell 3rd edition, O'Reilly Media

[37] Dalby, A, Nourse, JG, Hounshell, WD, Gushurst, AKI, Grier DL, Leland BA, Laufer, J. (1992).

"Description of several chemical structure file formats used by computer programs developed at

Molecular Design Limited". Journal of Chemical Information and Modeling 32 (3): 244.

Page 20: Leadscope® Genetox Expert Alerts White paper A new ICH … · A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014  2

A new ICH M7 compliant expert alert system to predict the mutagenic potential of impurities March 2014

www.leadscope.com 20 [email protected]

[38] Amberg A, Braun K, Czich A, Kauffmann HM, Spirkl HP, Stammberger I, Troschau G, Positive

genotoxicity results due to the formation of genotoxic reaction-products of the solvent DMSO with

carboxylic/sulfonic acid halides. Society of Toxicology 2008 poster

[39] Hansen K. et al., J. Chem. Inf. Model. 2009, 49:2077-2081.


Recommended