Convergence and Technologies

Network Propagation Predicts Drug Synergyin CancersHongyang Li, Tingyang Li, Daniel Quang, and Yuanfang Guan

Abstract

Combination therapies are commonly used to treatpatients with complex diseases that respond poorly tosingle-agent therapies. In vitro high-throughput drug screen-ing is a standard method for preclinical prioritization ofsynergistic drug combinations, but it can be impractical forlarge drug sets. Computational methods are thus beingactively explored; however, most published methods werebuilt on a limited size of cancer cell lines or drugs, and itremains a challenge to predict synergism at a large scalewhere the diversity within the data escalates the difficulty ofprediction. Here, we present a state-of-the-field synergyprediction algorithm, which ranked first in all subchallengesin the AstraZeneca-Sanger Drug Combination PredictionDREAM Challenge. The model was built and evaluatedusing the largest drug combination screening dataset at thetime of the competition, consisting of approximately 11,500experimentally tested synergy scores of 118 drugs in 85cancer cell lines. We developed a novel feature extractionstrategy by integrating the cross-cell and cross-drug infor-

mation with a novel network propagation method and thenassembled the information in monotherapy and simulatedmolecular data to predict drug synergy. This represents asignificant conceptual advancement of synergy prediction,using extracted features in the form of simulated posttreat-ment molecular profiles when only the pretreatment molec-ular profile is available. Our cross-tissue synergism predic-tion algorithm achieves promising accuracy comparablewith the correlation between experimental replicates andcan be applied to other cancer cell lines and drugs to guidetherapeutic choices.

Significance: This study presents a novel networkpropagation–based method that predicts anticancer drugsynergy to the accuracy of experimental replicates, whichestablishes a state-of-the-field method as benchmarkedby the pharmacogenomics research community involvingmodels generated by 160 teams. Cancer Res; 78(18); 5446–57.�2018 AACR.

IntroductionNumerous meta-analysis studies have shown that multi-

agent therapies achieve better efficacy compared with single-agent therapies when treating complex diseases such as cancersand cardiovascular diseases (1–4). The "triple drug cocktail"for AIDS, introduced in 1995, is a prime example (5); com-pared with monotherapy treatments, it showed dramaticallyimproved efficacy in patients and became one of the earliestsuccessful combination therapies in history (6, 7). Multi-agenttherapies involving targeted and nontargeted drugs have nowbecome a standard practice in cancer treatments (8). Thetargeted drugs are usually designed to interact with driveroncogenes in cancers. For example, Trastuzumab binds to theHER2 and interferes the proliferation of HER2-positive breastcancer cells (9). However, human cancers involve numerousgenomic/epigenomic aberrations and complex interactionsbetween the compartments. Single-agent regimens could fail

when downstream factors or parallel pathways in the cancercells are activated to compensate for the drug effect (10, 11).Drug combinations can achieve better outcomes than single-agent therapy through many mechanisms. First, targeting mul-tiple oncogenic pathways or mutations simultaneously canreduce the chance of developing drug resistance (12–14).Second, lower toxicity and better efficacy can be achieved atthe same time through synergism. There are three types of drugcombinatorial effects: additive effect, synergistic effect, andantagonistic effect. Additive effect is estimated by assumingno interaction exists between the drugs. Synergism means thatthe combined effect is larger than the additive effect (15).Antagonism is the opposite of synergism.

The properties of FDA-approved drugs are not understoodwell enough to infer the complex pharmacokinetic interactionsbetween drugs. Therefore, efforts have been made to find syn-ergistic drug combinations via both experimental and compu-tational methods. The standard way is through in vitro or in vivodrug screening experiments. However, technical and economicbarriers make this process expensive and slow. Drug screeningexperiments are easy to operate on a small scale but becomehighly labor intensive as the number of drugs grows. Apart fromthe technical limitations, the economic issue between drugcompanies and sectors is also a big impediment to the develop-ment of drug combinations (16).

In view of these concerns, in silico methods have beenproposed to find candidate drug combinations for furtherexperimental tests (17–21). Most of the existing methods aim

Department of Computational Medicine and Bioinformatics, University ofMichigan, Ann Arbor, Michigan.

Note: Supplementary data for this article are available at Cancer ResearchOnline (http://cancerres.aacrjournals.org/).

Corresponding Author: Yuanfang Guan, University of Michigan-Ann Arbor, AnnArbor, MI 48109. Phone: 734-764-0018; E-mail: gyuanfan@umich.edu

doi: 10.1158/0008-5472.CAN-18-0740

�2018 American Association for Cancer Research.

CancerResearch

Cancer Res; 78(18) September 15, 20185446

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

at predicting the synergistic effects of two drugs, as the tripledrug combinatorial effect is technically harder to predict andlacks experimental data for model evaluation. Many datasetshave been utilized for feature extraction, such as drug chemicalstructure, drug target information, side effects, gene expres-sion data, and monotherapy data. Despite the highly diversecomputational approaches, the main bottleneck in drug com-bination prediction remains the lack of experimental data fordrug combinations (22). The Cancer Cell Line Encyclopedia(CCLE; ref. 23) and Genomics of Drug Sensitivity in Cancer(GDSC; ref. 24) are two of the most significant databases foranticancer drug discovery (25). They both provide explicitbaseline information, such as cell line genetic profiles. Thoselarge-scale baseline datasets can be used to build models, butthe prediction performance is often evaluated on a smalldataset. Those "gold standard" drug combinations usuallycome from the FDA Orange Book, published literatures, orclinical trials. The limited drug screening dataset could poten-tially affect the robustness of the models. There also exists adifficulty in the performance comparison among the estab-lished methods, due to the inconsistency in the scoring metricsand evaluation datasets.

The mechanisms of synergistic effects are not universalamong different drugs or cancers. It is unknown whether syn-ergistic effects of drugs can be predicted accurately for cancercell lines from different tissues and drugs that have distincttargets. In addition, the relationship between different featuresand the contributions of different data to the prediction accu-racy remains unclear. Here, we present a biological networkpropagation–based algorithm for large-scale drug synergy pre-diction. This model integrates the indirect drug targeting effectsby interrogating the gene–gene network structure, and is opti-mized using a large drug combination screening dataset, pro-vided by the AstraZeneca-Sanger Drug Combination PredictionDREAM Challenge (22, 26). The DREAM Challenge data rep-resent the largest drug screening dataset at the time of theDREAM challenge, consisting of approximately 11,500 experi-mentally assessed continuous synergy scores across 85 cancercell lines and 118 targeted drugs. Furthermore, the DREAMChallenge provides a platform for evaluating all submissionsin a uniform and consistent fashion in order to crowdsourcethe best solutions for synergy prediction. Among the sub-missions from 160 competing teams, our method achieved thebest performance in all subchallenges. Our model also high-lights the predictive power of monotherapy data and baselinemolecular data, and demonstrates that using a combined featuredataset will improve the prediction performance comparedwith single predictors. Moreover, our model is able to transferinformation among cell lines and drugs, and achieves highperformance comparable with the theoretical limit, i.e., thereproducibility of experimental replicates.

Materials and MethodsDrug screening data acquisition

In this research, we integrated the monotherapy data, drugtarget information, baseline molecular features, and the gene–gene interaction network to predict drug synergy (Fig. 1A–C). Atotal of approximately 11,500 synergy scores were experi-mentally assessed and provided by the DREAM consortium.The synergistic effects were measured through a combinatorial

in vitro drug screening of 85 cancer cell lines and 118 anony-mous chemical compounds. The cancer cell lines come frombreast (n ¼ 34), lung (n ¼ 22), urinary tract (n ¼ 14),gastrointestinal tract (n ¼ 12), male genital system (n ¼ 2),and lymphoma (n ¼ 1). All these cancer cell lines were authen-ticated using the short-tandem repeat assay at AstraZeneca cellbanking. Each cell bank was confirmed to be mycoplasma-free, and these cells were cultured for up to 20 passages. Ineach drug screening experiment, 5 nontrivial doses of eachdrug were tested. The observed drug responses were summa-rized in a 6 � 6 matrix. The efficacy of the drugs was measuredby the percentage of the remaining cells after drug treatmentcompared with the untreated cells. The synergy scores werecalculated with Combenefit v1.31 (27) using the monotherapyand drug combination response data. Additive effects werecalculated using the Loewe model (28, 29). Positive synergyscores indicate synergism, and negative synergy scores indicateantagonism. The absolute values represent the strength of thesynergistic/antagonistic effect. The synergy scores can be sum-marized in a sparse 3D tensor (Drug A � Drug B � Cell line;Fig. 2A; Supplementary Fig. S1), where approximately 2% ofthe elements are filled with known synergy scores. The mono-therapy data of all drugs in the training and testing sets areavailable. The drug combination screening data are partitionedinto the "Training set," "Leaderboard set," and "Final evalua-tion set" (Table 1). In this research report, we used both thetraining set and the leaderboard set for model optimization.The final evaluation set was kept confidential during the modeldevelopment stage and was used for evaluating and comparingthe performance of participants.

The participants were asked to predict synergy scores in twoscenarios, which correspond to two subchallenges (Fig. 1B):1. Subchallenge 1 (SC1) asks to predict the synergy scores of

drug–drug combinations that have been previouslytested in other cell lines.

2. Subchallenge 2 (SC2) asks to predict the synergy score ofdrug–drug combinations without prior experimental testresults. The two subchallenges shared the same trainingsamples and feature datasets, but evaluated the algorithmson different testing samples (Table 1).

Drug effects and synergy scoresThe drug effect (cell viability) is calculated by fitting the

following equation to an observed dose–response curve:

E að Þ ¼ 100þ E¥ � 100

1þ IC50

a

� �H;

where EðaÞ is the observed cell viability at concentration a. TheIC50, H, and E¥ represent the concentration where half of thecells are killed, the slope of the curve, and the maximum kill,respectively. Similarly, the combination effect Eða; bÞis the cellviability in the 2D space of a drug pair. This effect can bedecomposed using the following equation:

E a; bð Þ ¼ A a; bð Þ þ S a; bð Þwhere Aða; bÞ is the additive effect, and Sða; bÞ is the synergisticeffect, based on the definition using the Loewe model (28, 29).These effect values are obtained by computing the integral of the2D curve in logarithmic concentration space.

Predict Drug Synergy with Network Propagation

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5447

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

Baseline molecular data, pharmacogenetic data, and genenetwork acquisition

Apart from the drug screening data, the DREAM consortiumalso provided the drug target information and physico-chemical properties of all 118 drugs, as well as 4 sets of baselinemolecular profiles of all 85 cancer cell lines: (i) GDSC gene

expression data from Affymetrix Human Genome U219 arrayplates (E-MTAB-3610), (ii) GDSC gene mutation data fromwhole-exome sequencing with Illumina HiSeq 2000 AgilentSureSelect (EGAS00001000978), (iii) GDSC copy-number var-iation (CNV) data from Affymetrix SNP6.0 microarrays(EGAS00001000978), (iv) DNA methylation data from the

Figure 1.

The overall algorithm design and network propagation strategy. A, Four feature sets were used to predict drug synergy, including monotherapy data,molecular data, gene interaction network, and drug target information. B, The aims of SC1 and SC2. In SC1, participants were asked to predict drugsynergy for drug pairs with combinational training data (blue cells), whereas in SC2, the combinational training data for target drug pairs wereunavailable C, The pipeline of simulating posttreatment molecular features. The original molecular features were first filtered to exclude nontargetgenes. Then, a gene–gene interaction network was used to simulate the values of both drug target genes and other genes connected to them.

Table 1. The splitting of drug combination screening data in subchallenges 1 and 2

Training þ leaderboard Final evaluation setNumber of drug combinations Number of synergy scores Number of drug combinations Number of synergy scores

SC1 539 6,598 167 1,089SC2 370 3,826

NOTE: "Drug combinations" refers to unique drug–drug pairs. Synergy scores are calculated for each drug screen experiment that involves an interesteddrug–drug pair and an interested cancer cell line.

Li et al.

Cancer Res; 78(18) September 15, 2018 Cancer Research5448

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

Bellvitge Biomedical Research Institute (raw methylation datafrom GEO: GSE68379). We also used two external datasets:(i) CCLE gene expression data and (ii) CCLE CNV data, whichare both publically available at https://portals.broadinstitute.org/ccle. We utilized a published mouse gene network (30),which illustrates the probabilistic functional linkages among20,581 protein-coding genes. This network is available athttp://fntm.princeton.edu. The full dataset from DREAM canbe downloaded at https://openinnovation.astrazeneca.com/data-library.html. The detailed description of the generationmethod is available at https://www.synapse.org/DrugCombinationChallenge. In the following content, we will refer to thesynergy scores as gold standards or target values of the model,and the other datasets as feature datasets or predictors.

The rationale of algorithm designWe integrated the various datasets by building individual

models and then ensembling the prediction of each model toobtain a final prediction. The synergy scores predicted by differentmodels were combined by a weighted average:

pFinal ¼Pk

i¼1 piwiPki¼1 wi

;

where pi is the prediction made based on the ith model, wiis theweight of that model, and k is the total number of models.The left side of the equation is the final prediction. Here, pi andpFinal are both vectors representing a list of predicted synergyscores, whereas wi is a positive scalar.

We developed three models for making predictions: the globalsynergy model (GSM), the local synergy model (LSM), and thesingle drug model (SDM; Fig. 2B). GSM uses a single training setand makes predictions for all testing samples at once. LSMconstructs the training set for each unknown synergy score in thetesting dataset and makes predictions separately. The trainingdataset of LSM is a subset of the drug pairs in GSM, including onlythe pairs when either drug in the testing drug pair appears. SDMis similar to LSM, except that the training dataset is generatedfor each drug rather than drug combination and the predictedsynergy scores are represented by the average of the two pre-dicted values (Supplementary Fig. S2).

Each of the models was developed using the random forestalgorithm implemented by MATLAB TreeBagger (NumTrees ¼300, keeping the other parameters to default values). The analysiswas implemented using Perl v5.10.1, Python 3.5.4, and R 3.4.1.The scripts submitted to DREAM are available at https://www.synapse.org/#!Synapse:syn5665972. In the original model, weused 3 sets of features for SC1 and 4 sets of features for SC2. In thisarticle, we only implement and discuss the two most significantfeature sets (see Discussion).

Cross-validation setupDuring the development stage, we used 5-fold cross-

validation to test the prediction performance of our model.First, the Training þ Leaderboard data were randomly parti-tioned into five equally sized sets. Then we sequentially usedone of the five sets as the testing set and the rest four sets asthe training dataset. This process was repeated 5 times. Theprediction performance was evaluated using the scoring metric.The final performance of a model was represented by theaverage of the 5 sets � 5 repeats ¼ 25 scores.

After the model was finalized, all the training data (Training þLeaderboard) were fed into the model, and the final evaluationdataset was used to evaluate the prediction performance of themodel. The models we used for SC1 and SC2 were different andoptimized separately (Supplementary Table S1).

The scoring metrics for continuous and binary predictionsFor each drug combination, SC1 requires continuous predic-

tions of the synergy scores in multiple cell lines, whereas SC2

Figure 2.

A visualization of training samples and the pipeline of the GSM and LSM. A, Theknown synergy scores are visualized as a 3D matrix, where the three axes aredrug A, drug B, and cell line. Each blue dot represents an experimentalmeasurement of the combination effect of drugs A and B in a cancer cell line.This matrix is not symmetric as it should be because the order of drug Aand B is kept as it was in the raw dataset. B, In the global model, all the trainingdata are fed into the random forest classifier, and the predictions are made atonce. In the LSM, the training data are first split into mutually nonexclusivesubsets. In the example described in the figure, the interested unknownscore is the synergy score for drug Blue and drug Red in cell line Gray. Thesubset of training data should contain all the experiments that involve eitherdrug Blue, drug Red, or both. The model is then trained on the subset andgives predictions for the synergy scores of drug Blueþ drug Redþ cell line Gray.In the last step, the predictions are collected together.

Predict Drug Synergy with Network Propagation

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5449

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

requires binary predictions indicating whether two drugs aresynergistic or not. In SC1, we used the original predicted values.In SC2, we centered all the predicted synergy scores to zero andlabeled the positive scores with "1" (predicted synergy) and thenegative scores with "0" (predicted nonsynergy).

As the sample size for each drug combinations varies (i.e.,some drug combinations were tested in more cell lines thanothers), applying Pearson correlation directly on all synergyscores will potentially give more weights to drug combinationsthat have more experiments. Therefore, to give equal weights toall drug combinations, a weighted average of Pearson correla-tion was used as the scoring metric in SC1, which is calculatedusing the following formula.

rw ¼PN

i¼1

ffiffiffiffiffiffiffiffiffiffiffiffiffini � 1

priP N

i¼1

ffiffiffiffiffiffiffiffiffiffiffiffiffini � 1

p ;

where N is the total number of tested drug combinations, ni isthe number of cell lines a specific drug combination was testedon, and ri is the conventional Pearson correlation between thepredicted and observed synergy scores for a drug combinationacross multiple cell lines. This is the primary metric used in SC1in the challenge. Similarly, the tie-breaking metric is also theweighted average of Pearson correlation, but evaluated on thesubset of drug pairs with the best synergy score larger than orequal 20.

We also computed the area under the receiver operatingcharacteristic curve (ROC-AUC) to evaluate the performance ofthe binary predictions. We labeled the observed synergy scores�20 as "1" (observed synergy) and scores <20 as "0" (observednonsynergy). This threshold is also used by the DREAM con-sortium. For the binary predictions in SC2, the DREAM con-sortium chose a sequential three-way ANOVA as the primaryscoring metric to measure the significance of the binary sep-aration.

s ¼ �a� log10 pð Þ:

Here, a is the sign of the effect size. And p is the P value of theANOVA test. In addition to the weighted Pearson correlation(WPC) and three-way ANOVA, ROC-AUC was also consideredto assess the prediction performance.

Predicting drug synergy withmonotherapy drug–response dataTwo types of data were provided for each monotherapy exper-

iment: (i) The dose–response data, which include the drugresponses of 5 nonzero doses. (ii) The data quantification andquality check information. We applied data augmentation togenerate two sets of training features and three sets of testingfeatures. Eventually, three lists of predictions were obtained. Thefinal prediction was the weighted average of predictions usingthree models (see details in Supplementary File).

Simulating posttreatment molecular data with networkpropagation for feature extraction

As a form of feature extraction, we simulated the posttreat-ment molecular features (3D feature matrix) by modifyingthe baseline molecular features (2D feature matrix) accordingto the drug target information (2D feature matrix) and apreviously published gene network (30). These simulated fea-tures act as a set of extracted informative features that areunique to each (Drug A � Drug B � Cell line) triplet. Genes

that are not the targets of any drugs in the training dataset wereremoved to speed up computation. Then, the drug target geneswere mapped to the gene network and assigned a probability ofinteraction with other genes. After this step, we modifieddifferent molecular features of genes using the following strat-egy (Fig. 1C). The drug effects on nontarget genes were scaledaccording to the interaction probability between the nontargetand target genes. For gene mutation features: vt ¼ 1;vnt ¼ v0 �maxfp1; p2:::pkg. For CNV, gene expression and genemethylation features: vt ¼ 0; vnt ¼ v0 � ð1�maxfp1; p2:::pkgÞ.Here, v0 is the original value of the molecular feature,vt is thesimulated value for the target genes, vnt is the simulated valuefor the nontarget genes, and pk is the interaction probabilitybetween two genes.

ResultsMonotherapy has intrinsic correlation with synergism

When we examined the monotherapy features, we foundthat they were intrinsically correlated with drug synergisticeffects. The raw monotherapy data included the drug responses(5 dosages/drug) and quality control information of the experi-ments. To construct a more complete feature space, we per-formed feature extraction and derived 135 predictors based onthe raw monotherapy data (see Supplementary File). Figure 3Ashows the Pearson correlations between each derived mono-therapy feature and the target values (i.e., the synergy scores).Approximately 55% (74/135) of the features are significantlycorrelated with the synergy scores (FDR < 0.05; see Supple-mentary Table S2 for the correlations). Most of the monother-apy features, including all top 10 features (i.e., features with10 highest absolute correlation coefficients), are negativelycorrelated with the synergy scores. Among the top ten features,nine are exclusively derived from the dose–response curve, andsix are in the form of maxfEa; k; Eb; kg, where Ea; k is the efficacyof drug aat dosage k. The correlation analysis shows that mono-therapy data have a small but significant correlation with thesynergy scores. The information in different types of mono-therapy data has varied correlations with the synergy scores,which implies varied predictive powers.

Novel network propagation strategy integrates the keyinformation of the gene–gene interaction network

Because all the original molecular features (gene expression,CNV, methylation, and mutation) are cell line–specific, thesefeatures are identical for samples from the same cell line buttreated with different drugs or drug pairs. Clearly, different drugsinteract with different proteins or DNAs, interfering with thecorresponding functional pathways. Therefore, the posttreatmentgenomic information should vary dramatically, depending onboth the drug effects and the cell line–specific baseline molecularfeatures. To leverage the prior knowledge of the gene–gene net-work and drug target genes, we developed a novel feature extrac-tion strategy that simulates the drug-specific molecular featuresfor all the cell lines. Compared with the Original model withoutsimulation, the Propagated model with simulated data achievedhigher prediction correlations in terms of both the primary andtie-breaking evaluation metrics used in the challenge (Fig. 3B;see Materials and Methods about the scoring metrics).

For CNV, expression, and methylation, we assume thedrug target genes are silenced so that the values (vt) become

Li et al.

Cancer Res; 78(18) September 15, 2018 Cancer Research5450

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

zero. The nontarget genes are also affected, proportional tothe probabilities (p) of their connections to the target genes.For each nontarget gene, we assume that it is affected mostlyby the strongest connected drug target gene. Therefore, thesimulated molecular feature of a nontarget gene (vnt) becomesv0 � ð1�maxfp1; p2; . . .; pkgÞ, where v0 is the original value ofthe molecular feature, p1; p2; . . .; pk are the connection strengthsto target gene 1, gene 2, . . ., gene k. For genetic mutation, thevalues are binary: mutation v0 ¼ 1 and nonmutation v0 ¼ 0. Weassume that a mutated gene is not affected by the drug, and itsvalue remains 1. For a nontarget gene, the value is adjusted,proportional to the connection probabilities of drug targetgenes v0 �maxfp1; p2; . . .; pkg.

Molecular and monotherapy features complement eachother in predicting drug synergy

We extracted the information in the molecular data withnetwork propagation and found that it complemented themono-

therapy data when predicting drug synergy. We simulated theposttreatment gene expression, methylation, mutation, and CNVprofiles by leveraging the interactions between the direct andindirect drug targets. In order to reduce model complexity andremove irrelevant information, we performed an aggressive pre-filtering. First, we used a local model and utilized only a subset oftraining samples each time. Second, we kept only the target genesof the drugs in the training data subset as predictors (Fig. 1C).Finally, the six molecular datasets were filtered down to approx-imately 900 molecular features (see Materials and Methods).The posttreatment molecular profiles were then generated byincorporating the gene–gene interaction and the drug targetinformation into the baseline molecular features. This approachis innovative because it fuses three 2D feature spaces into a 3Dfeature space and avoids feature redundancy, which is especiallybeneficial for random forest (see Discussion). In SC1, the perfor-mance of LSM using molecular features is approximately 0.32,which is 0.04 lower than GSM using monotherapy features

Figure 3.

Prediction performance of models using different features. A, The Pearson correlation between the constructed monotherapy features and the goldstandard (observed synergy scores). The features with P value < 0.5 are shown in red. B, The WPC between predictions and observations usingsimulated posttreatment molecular features through network propagation (Propagated) or not (Original). The Propagated model achieved highercorrelations in terms of both the primary and tie-breaking metrics used in the challenge. C, The cross-validation results of SC1 and SC2. Ourfinal combined model is the ensemble of these three models and has the best performance. D, The ternary plot of using different weights for differentmodels and the corresponding three-way ANOVA results in SC2. The directions of the three axes are indicated by the short lines aligned along the borderof the ternary heatmap.

Predict Drug Synergy with Network Propagation

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5451

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

(�0.36), but still significantly better than random (i.e., �0).To further improve the prediction accuracy, we assembled theresults of LSM and GSM through weighted averaging. A gridsearch strategy was performed to optimize the weights assignedto each model. For SC1, as the weight of the molecular featurechanges from 0 to 1 (Supplementary Fig. S3), the predictionperformance first increases then decreases and reaches its maxi-mum at pCombined ¼ 0:3� pLSM þ 0:7� pGSM. As the predictedvalues are on the same scale for different feature sets, this resultindicates that monotherapy data are more informative than theposttreatment molecular features. Meanwhile, the combinedmodel that integrates both features performed better than anyindividual feature set (Fig. 3C).

Molecular andmonotherapy features are both informative andnonredundant for drugs without prior experiments (SC2). Wekept both GSM and LSM, and tested SDM using molecularfeatures (Fig. 3D). The ternary plot shows that, in consistencewith SC1, monotherapy data are still a key predictor, as theperformance drops as the weight of monotherapy featuresdecreases. The superior prediction power of monotherapy datacan also be shownby comparing the individualmodels in parallel(Fig. 4A and B). The performance of SDM (8.18) is marginally

better than the GSM (6.81). And the performance of GSM(11.34) is the best among the three models. Similarly, we assem-bled these three models via weighted averaging (Fig. 3D). Thehighest performance (three-way ANOVA ¼ 16.87) was achievedat pCombined ¼ 0:4� pSDM þ 0� pLSM þ 0:6 � pGSM (Supplemen-tary Table S1). The secondbest performance (three-wayANOVA¼16.78) involves contributions from both SDM (weight ¼ 0.25)and LSM (weight ¼ 0.10). Again, we find that monotherapyfeatures are more informative than molecular features.

Monotherapy provides the essential information about thedirect treatment effect on a cancer cell line, which is the basis ofdrug synergism. For example, if a drug does not work on aspecific cell line at all, it is unlikely to synergize with any otherdrugs. Therefore, the monotherapy-based GSM contains thebasic probability of a drug to synergize and has a largercontribution. In addition, the single drug response is alsopredictable from the observed and simulated genomic features.So even if the monotherapy results are absent for some drug–cell line pairs, our genomics-based model can still make pre-dictions. In fact, the genomics-based model further considersthe gene–gene interactions, which are reflected by the observedand simulated genomic features through network propagation.

Figure 4.

The comprehensive evaluation of the performance using different features and metrics. A, The cross-validation results of three individual models andthe combined model in SC2, evaluated via three-way ANOVA. B, The performance decreases when the sample size is reduced. Here, 100% representsthat all the available training samples are used, whereas 75% (50%) means that we randomly subsampled 75% (50%) of all the samples in thetraining dataset. C, The performance comparison of the models in SC1 and SC2 using ROC-AUC. Our final combined model is the ensemble of thesethree models and has the best performance. D, The overall distribution of the predicted and the observed synergy scores in SC1.

Li et al.

Cancer Res; 78(18) September 15, 2018 Cancer Research5452

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

For example, when two drugs synergize by targeting a specificfunctional pathway, genes within this pathway are simulta-neously affected by both drugs. Through our network propa-gation algorithm, multiple genomic features under the treat-ment of both drugs are simulated. In this way, the priorknowledge of gene–gene interactions is integrated to improvethe prediction of synergism.

Drug synergy information is transferable among cell lines anddrugs

By integrating the cross-cell and cross-drug information, wefind that the synergy information is transferable among cell linesand drugs, and can be strengthened by within-cell and within-drug information. We used two uniform scoring metrics (seeMaterials and Methods), which are the WPC and ROC-AUC,to compare the prediction performance of our model in SC1and SC2. The results of WPC showed that the models in SC1 allperformed better than the correspondingmodels in SC2 (Fig. 3C).This is expected because in SC1 the training data contain priorexperimental results of the drug combinations in the testingdataset, whereas SC2 does not. The WPC assesses the accuracyof continuous predictions based on linear regression. We testedthe ability of the model on making binary (synergy/nonsynergy)predictions using ROC-AUC (Fig. 4C). In both SC1 and SC2, thecombined model performed better than any one of the singlemodels. The rank of the performance of GSM and LSM is reversedin SC1 compared with WPC results. This is because our model isoptimized under the WPC metric. Even though the predictionperformance of the combinedmodel in SC2 (WPC¼0.38, AUC¼0.68) is less significant than SC1 (WPC ¼ 0.41, AUC ¼ 0.75), itstill proves that the drug synergy information is transferablebetween different cell lines and drugs, and that the drug synergycan be predicted without any prior experiments of the interesteddrug combination. Meanwhile, the drug tests of the interestedcombinations can improve the prediction accuracy.

Under the benefit of this DREAM challenge, we gained accessto a large-scale drug combination screening dataset. To testthe generalizability of our model and to investigate the influ-ence of the size and diversity of the dataset on the predictionperformance, we randomly subsampled 50% and 75% of thetraining dataset and assessed their prediction accuracy used5-fold cross-validation (Fig. 4B). The prediction performanceis improved by approximately 0.02 and 0.01 as dataset goesfrom 50% to 75% and 75% to 100%, respectively. This resultshows that the performance of our model improves as the sizeof the training data increases. It also indicates that this modelis robust and has the potential to be generalized to includemore cell lines and drugs, while maintaining or improving theprediction performance.

High generalization performance of our model in thelarge-scale unseen dataset

Our model retains high performance in predicting drugsynergy in the held-out final testing dataset provided by theDREAM competition (Table 1; Fig. 4D). As cross-validationonly utilizes 80% of the whole training dataset for training, theperformance of the model on the final evaluation set, whichuses 100% of the training data, reaches even higher scores thanthe cross-validation results. For SC1, the final testing datasetconsists of 1,089 unknown synergy scores. The WPC betweenthe predicted and observed synergy scores is 0.47 (Fig. 4D),

which is 0.07 higher than the cross-validation performance.This score is comparable with the correlation between exper-imental replicates (r ¼ 0.56) as recognized as the upper boundin the DREAM challenge (22). The ROC-AUC is 0.76, which isalso higher than the cross-validation results (0.75; Fig. 4C).This result indicates that our model is robust and is not over-fitted to the training dataset. The area under of precision–recallcurves (AUPRC) were also calculated to demonstrate the goodperformance of our model on the unbalanced data (Supple-mentary Fig. S4). The final evaluation dataset of SC2 consistsof 3,826 unknown synergy scores involving 370 unique com-binations of 104 drugs. None of the 370 drug combinationsoverlaps with the combinations in the training dataset, but allthe drugs are from the training dataset. Figure 5A and B showsthe prediction performance of our model on the final testingdataset. The predicted values are generally well correlated withthe observed values, except for the extreme values on two tailsof the observed dataset. The three-way ANOVA score is 54.16,which is significantly improved compared with the cross-validation result (score ¼ 16.87). This again demonstrates therobustness of our model.

The prediction performance varies dramatically among celllines and drugs

The prediction performances are highly dependent on thecharacteristics of the cell line and drug(s). The drug screeningdata include 85 cancer cell lines and 118 drugs, which cover awide spectrum of synergistic/antagonistic effects caused bydistinct mechanisms. The diversity of the underlying biologicalmechanism could result in various predictableness of differentdrugs and cell lines. We investigated the prediction perfor-mances for each drug combination, individual drug, and indi-vidual cell line (Fig. 5C). The performance is represented by thePearson correlation between all the observed and the predictedsynergy scores of the interested subject (drug, drug combina-tion, or cell line). Figure 5C shows that the performance variedgreatly (see the performance for all cell lines, drug combina-tions, and drugs in Supplementary Tables S3, S4, and S5,respectively). This result is consistent with the other teams inthe challenge (22), where a subset of approximately 20% drugcombinations were poorly predicted across all methods. It onthe one hand shows the possibility of predicting synergy withextraordinary accuracy for a subset of drug and cell lines, andon the other hand indicates that a subset of drugs might not bepromising candidates for drug combination in view of thedifficulty in predicting their synergistic effects.

We further calculated the prediction correlation across testdrug pairs for each cell line, and the tissue-specific perfor-mances of SC1 and SC2 are shown in Fig. 6A and Supplemen-tary Fig. S5, respectively (Supplementary Tables S3 and S6).Our model achieved relatively high correlation in digestivesystem (median, 0.689) and lung (median, 0.583). And foreach drug, the weighted average correlation was calculated, andthe top 20 predictable drugs are shown in Fig. 6B. Specifically,drugs including ALK_2, MTOR_5, and PTK2 are relatively easyto predict, with correlation values above 0.75 (SupplementaryTables S7, S8, and S9). The drug identity is anonymized andmasked by the target gene, due to confidentiality agreements.Moreover, we investigated the performance of predicting thebest drug pairs on the held-out test set using our model. Thepredicted ranks of the true best pairs were calculated. We found

Predict Drug Synergy with Network Propagation

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5453

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

that for 30% of the cell lines, we correctly predicted the bestsynergistic drug pairs (rank 1 in Fig. 6C), and for more than73% of the cell lines, the predicted ranks of the best pairs wereamong top 5 (ranks 1 to 5 in Fig. 6C).

We summarized the best synergized drug pairs for the 85 testcell lines (Supplementary Table S10). Then, we categorized these85 pairs based on the tissues where the test cell line originated(Supplementary Fig. S6). We find that drugs are more likely tosynergize in lung, achieving a median synergy score of 91.81among 22 test cell lines. For the urogenital system, the score isrelatively low, with a median value of 70.22 among 14 cell lines.Furthermore, the relationship between the prediction perfor-mance and the number of drug targets was investigated. For eachdrug, we calculated the WPC from our predictions and thenumber of genes it directly targets (Supplementary Fig. S7A). The

selected drug categories are shown in different colors. We did notfind any significant correlation between the prediction perfor-mance and the number of target genes (Pearsonproduct–momentcorrelation ¼ –0.058; P value ¼ 0.553). Moreover, we categoriz-ed the drugs based on their target—whether it is a protein orDNA (including carboplatin, CarboTaxol, cisplatin, FOLFIRI,FOLFOX, gemcitabine, oxaliplatin, and topotecan). The overalldistribution of prediction correlations for these two categories isshown in Supplementary Fig. S7B. We did not find any significantdifferences between these two types of drugs.

DiscussionIn silico synergistic drug combination prediction has drawn

extensive attention in the past decades, yet so far there are

Figure 5.

The diversity of predictionperformances. A, The overalldistribution of the predicted and theobserved synergy scores in SC2.B, The ROC curves and AUCs ofmodels using the monotherapy,molecular, or combined features. Theperformance was evaluated on thefinal testing set in SC1. C, Thedistribution of the predictionperformance for individual drugs,unique drug combinations, andindividual cell lines. The x-axisrepresents the value of the WPC.The y-axis represents the frequencyof the corresponding Pearsoncorrelation.

Li et al.

Cancer Res; 78(18) September 15, 2018 Cancer Research5454

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

no computational tools that have received wide acceptance orcome into routine drug development process. One reason isthat the available gold-standard dataset, which is essential formodel training and validation, is limited in size and format(binary labels of synergy or nonsynergy). In this research, wehad access to a large drug combination screening dataset thatincluded approximately 11,500 continuous synergy scores andbuilt a novel synergism prediction algorithm. The predictors arederived from the monotherapy data, baseline molecular data(including gene expression, methylation, CNV, and mutation),the gene–gene interaction network, and the drug target infor-mation. We simulated the posttreatment features by integratingthe prior pharmacokinetic knowledge of drugs and biological

knowledge of gene–gene interactions into the molecular pro-files in order to generate an informative set of features. Bycomparing the performances of single models, we find thatmonotherapy data are more predictive of drug synergy than thesimulated molecular profile, and the combined model per-forms better than any individual models. In addition, we findthat the predictability varies greatly among cell lines and drugs.This explains why studies using small training sets are alsocapable of making accurate predictions. The result provides alist of candidate drugs that is promising for drug selection dueto their relatively high predictableness and another list of drugsthat requires further exploration and is currently not suitablefor in silico prediction.

Figure 6.

The different aspects of model validation on the external test dataset. A, The tissue-specific distribution of the prediction correlations for all drug pairs.Note that blood and skin are excluded from the figure because they only have 1 and 2 cell lines, respectively. B, The top 20 easiest drugs to predictsynergy. For most drugs, their names are masked by their primary target genes, which were used in the AZ dataset. We first calculated the WPC ofevery drug pair across test cell lines. Then for each drug, the average correlation of all drug pairs containing it was calculated as its correlation. C, Theprediction rankings of the best synergized drug pairs in all cell lines. For each cell line, the prediction synergy scores were ranked and compared with thedrug pairs with the highest observed scores. The prediction rankings were summarized and are shown in the pie chart. For more than 73% test cell lines, ourmodel generated useful information indicating the potential best drug pair. And for 30% cell lines, the best synergized pairs were correctly predicted.

Predict Drug Synergy with Network Propagation

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5455

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

We chose random forest as the primary classifier because it isa nonlinear model that can account for the high-level interac-tions between features. These characteristics make it a goodchoice for solving intrinsically nonlinear biological problems.The network propagation–based feature simulation is a keycontributor to the novelty of the model. If a na€�ve approach isused instead, where different features are aligned alongsidewith each other, the gene–gene interaction and drug targetinformation will be identical for samples that are treated withthe same drugs. The same is true for samples tested on the samecell line. As a result, the features that have identical valuesacross multiple samples will then become redundant and dilutethe informative features, reducing the learning efficiency. Incontrast, our simulation method avoids the above issues andretains the useful information.

The robustness and accuracy of our model have been vali-dated on a large external test set covering more than a total of4,916 combinations of diverse drug pairs across 85 cancer celllines. The overall performance is close to the theoretical upperlimit. Specifically, we successfully predicted the best synergisticdrug pairs in 30% test cell lines and provided useful rankinginformation (rank � 5) of the best pairs in more than 73% testcell lines (Fig. 6C). With monotherapy results and genomicdata, our model can be used in predicting drug synergism for abroad range of cell lines and drug types in the future. Theembedded ideas of gene–gene network propagation and trans-fer learning across cell lines and drugs establish a new way toaddress the computational prediction of drug synergism. It isgeneralizable for other cell lines or new drugs.

We examined the predictive power of multiple features in thisresearch, whereasmore features can be added to themodel. In oursubmission to theDREAMchallenge, in addition tomonotherapyfeatures and simulated molecular features, we added one addi-tional feature in SC1 and two additional features in SC2 to furtherimprove the performance (see Supplementary File). The differ-ence between the performance of the submitted full model andthe simplifiedmodel ismarginal (�0.02; Supplementary Fig. S8).

Thus, the two additional features are capable of improving theoverall prediction performance but are weaker predictors com-pared with the two key feature sets. The size of the drug screeningdataset grows quickly in recent years. A bigger drug combinationscreening dataset (�22,000 samples) was published after theDREAM challenge (31). New methods built on this dataset haveshown promising future for this research field (32). Our modelalso has the potential to be trained on even larger datasets andreach higher performance. In sum, our model shows the possi-bility of making cross-cancer large-scale predictions of drug syn-ergy and can potentially facilitate the experimental design of drugcombinations screening.

Disclosure of Potential Conflicts of InterestNo potential conflicts of interest were disclosed.

Authors' ContributionsConception and design: H. Li, Y. GuanDevelopment of methodology: H. Li, Y. GuanAnalysis and interpretation of data (e.g., statistical analysis, biostatistics,computational analysis): T. Li, Y. GuanWriting, review, and/or revision of the manuscript: H. Li, T. Li, D. Quang,Y. GuanStudy supervision: Y. Guan

AcknowledgmentsThis work has been supported by NSF 1452656, Michigan Institute for Data

Science (MIDAS), and George M. O'Brien Kidney Research Core Centers.Y. Guan is supported by these grants. We thank Hongjiu Zhang for technicaland editorial assistance. We thank Shuai Hu for discussion about therapy-resistant and drug treatment in cancers.

The costs of publication of this article were defrayed in part by thepayment of page charges. This article must therefore be hereby markedadvertisement in accordance with 18 U.S.C. Section 1734 solely to indicatethis fact.

Received March 9, 2018; revised June 27, 2018; accepted July 23, 2018;published first July 27, 2018.

References1. Łopuch S, Kawalec P, Wi�sniewska N. Effectiveness of targeted therapy as

monotherapy or combined therapy in patients with relapsed or refractorymultiple myeloma: a systematic review and meta-analysis. Hematology2014;20:1–10.

2. Gu L, Liu H, Fan L, Lv Y, Cui Z, Luo Y, et al. Treatment outcomes oftranscatheter arterial chemoembolization combined with local ablativetherapy versus monotherapy in hepatocellular carcinoma: ameta-analysis.J Cancer Res Clin Oncol 2013;140:199–210.

3. Zhang X, Zhang X-J, Zhang T-Y, Yu F-F, Wei X, Li Y-S, et al. Effect andsafety of dual anti-human epidermal growth factor receptor 2 therapycompared to monotherapy in patients with human epidermal growthfactor receptor 2-positive breast cancer: a systematic review. BMCCancer 2014;14:625.

4. Escobar C, Barrios V. Combined therapy in the treatment of hypertension.Fundam Clin Pharmacol 2010;24:3–8.

5. Food and Drug Administration. Attacking AIDS with a "cocktail" therapy:drug combo sends deaths plummeting news. July 1, 1999 [cited 2018 Jan24]. Available from: https://aidsinfo.nih.gov/news/493/attacking-aids-with-a-cocktail-therapy–drug-combo-sends-deaths-plummeting

6. Hammer SM, Saag MS, Schechter M, Montaner JSG, Schooley RT, JacobsenDM, et al. Treatment for adult HIV infection: 2006 recommendations ofthe International AIDS Society–USApanel. TopHIVMed 2006;14:827–43.

7. Henkel J. Attacking AIDS with a "cocktail" therapy? FDA Consum 1999;33:12–7.

8. Tallarida RJ. Quantitative methods for assessing drug synergism. GenesCancer 2011;2:1003–8.

9. Schramm A, De Gregorio N, Widschwendter P, Fink V, Huober J. Targetedtherapies in HER2-positive breast cancer - a systematic review. Breast Care2015;10:173–8.

10. Rosell R, Karachaliou N, Morales-Espinosa D, Costa C, Molina MA,Sansano I, et al. Adaptive resistance to targeted therapies in cancer. TranslLung Cancer Res 2013;2:152–9.

11. Neel DS, Bivona TG. Resistance is futile: overcoming resistance to targetedtherapies in lung adenocarcinoma. NPJ Precis Oncol 2017;1.

12. Komarova NL, Richard Boland C. Cancer: calculated treatment. Nature2013;499:291–2.

13. Flaherty KT, Infante JR, Daud A, Gonzalez R, Kefford RF, Sosman J, et al.Combined BRAF and MEK inhibition in melanoma with BRAF V600mutations. N Engl J Med 2012;367:1694–703.

14. Hu C-MJ, Zhang L. Nanoparticle-based combination therapy towardovercoming drug resistance in cancer. Biochem Pharmacol 2012;83:1104–11.

15. Sucher NJ. Searching for synergy in silico, in vitro and in vivo. Synergy2014;1:30–43.

16. Humphrey RW, Brockway-Lunardi LM, Bonk DT, Dohoney KM,Doroshow JH, et al. Opportunities and challenges in the developmentof experimental drug combinations for cancer. JNCI J Nat Cancer Inst2011;103:1222–6.

Li et al.

Cancer Res; 78(18) September 15, 2018 Cancer Research5456

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from

17. Gayvert KM, Aly O, Platt J, Bosenberg MW, Stern DF, Elemento O. Acomputational approach for identifying synergistic drug combinations.PLoS Comput Biol 2017;13:e1005308.

18. Zhao X-M, Iskar M, Zeller G, Kuhn M, van Noort V, Bork P. Prediction ofdrug combinations by integrating molecular and pharmacological data.PLoS Comput Biol 2011;7:e1002323.

19. Flobak Å, Baudot A, Remy E, Thommesen L, Thieffry D, Kuiper M, et al.Discovery of drug synergies in gastric cancer cells predicted by logicalmodeling. PLoS Comput Biol 2015;11:e1004426.

20. Huang L, Li F, Sheng J, Xia X, Ma J, ZhanM, et al. DrugComboRanker: drugcombination discovery based on target network analysis. Bioinformatics2014;30:i228–36.

21. Iwata H, Sawada R, Mizutani S, Kotera M, Yamanishi Y. Large-scaleprediction of beneficial drug combinations using drug efficacy andtarget profiles. J Chem Inf Model 2015;55:2705–16.

22. MendenMP,WangD,GuanY,MasonM, Szalai B, BulusuKC, et al. A cancerpharmacogenomic screen powering crowd-sourced advancement of drugcombination prediction. BioRxiv February 13, 2018. Available from:http://dx.doi.org/10.1101/200451

23. Barretina J, Caponigro G, Stransky N, Venkatesan K, Margolin AA, Kim S,et al. The cancer cell line encyclopedia enables predictive modelling ofanticancer drug sensitivity. Nature 2012;483:603–7.

24. Yang W, Soares J, Greninger P, Edelman EJ, Lightfoot H, Forbes S, et al.Genomics of drug sensitivity in cancer (GDSC): a resource for thera-

peutic biomarker discovery in cancer cells. Nucleic Acids Res 2013;41:D955–61.

25. Azuaje F. Computational models for predicting drug responses in cancerresearch. Brief Bioinform 2017;18:820–9.

26. Wang D, Menden M, Yu T. Data description of AstraZeneca-Sanger drugcombination prediction DREAM challenge [cited 2017 Oct 15]. Availablefrom: https://www.synapse.org/#!Synapse:syn4231880/wiki/235651

27. Di Veroli GY, Fornari C, Wang D, Mollard S, Bramhall JL, Richards FM,et al. Combenefit: an interactive platform for the analysis and visual-ization of drug combinations. Bioinformatics 2016;32:2866–8.

28. Fitzgerald JB, Schoeberl B, Nielsen UB, Sorger PK. Systems biology andcombination therapy in the quest for clinical efficacy. Nat Chem Biol2006;2:458–66.

29. Geary N. Understanding synergy. Am J Physiol Endocrinol Metab 2013;304:E237–53.

30. Guan Y, Myers CL, Lu R, Lemischka IR, Bult CJ, Troyanskaya OG. Agenomewide functional network for the laboratory mouse. PLoS ComputBiol 2008;4:e1000165.

31. O'Neil J, Benita Y, Feldman I, Chenard M, Roberts B, Liu Y, et al. Anunbiased oncology compound screen to identify novel combinationstrategies. Mol Cancer Ther 2016;15:1155–62.

32. Preuer K, Lewis RPI, Hochreiter S, Bender A, Bulusu KC, Klambauer G.DeepSynergy: predicting anti-cancer drug synergy with Deep Learning.Bioinformatics 2017;34:1538–46.

www.aacrjournals.org Cancer Res; 78(18) September 15, 2018 5457

Predict Drug Synergy with Network Propagation

Cancer Research. by guest on September 1, 2020. Copyright 2018 American Association forhttps://bloodcancerdiscov.aacrjournals.orgDownloaded from