1
S TABILITY INDICATORS IN ( BIOLOGICAL ) NETWORK INFERENCE Giuseppe Jurman 1 , Michele Filosi 1,2 , Roberto Visintainer 1 , Samantha Riccadonna 3 , Cesare Furlanello 1 1 Fondazione Bruno Kessler, Trento and 2 University of Trento, Italy and 3 Fondazione Edmund Mach San Michele all’Adige, Italy We propose how to quantify inference variability with respect to data perturbation, and, in particular, data subsampling. We introduce a set of four indicators allowing the researcher to quantitatively evaluate the reliability of the inferred/non-inferred links. For a given ratio of removed data and for a give number of resampling, we quantitatively assess the mutual distances among all inferred networks and their distances to the network generated by the whole dataset. The rationale is that, the smaller the average distance, the stabler the network. We also provide a ranked list of the stablest links and nodes, where the rank is induced by the variability of the link weight and the node degree across the gen- erated networks, the less variable being the top ranked. As a network distance we employ the HIM distance, which represents a good compromise between local (link-based) and global (structure- based) measure of network comparison. As a first testbed in a controlled situation the four indicators are computed on a synthetic dataset for different instances of a correlation network with different measures, highlighting the impact of a False Discovery Ratio filter on the network re- construction method. Finally, we show the use of the stability measures in comparing the relevance networks inferred on a miRNA microarray dataset with paired tissues extracted from a cohort of 241 hepatocellular carcinoma patients. S TABILITY INDICATORS p D: s ALG NODES={x 1 D ,...,x p D } LINK WEIGHTS= w 11 D ,...,w 1p D w p1 D ,...,w pp D w hk D ... ... ... ... p D: s p D i : n p D 1 : n p D r : n ... ... ALG ALG ALG n<s r≤ s n ) ) Stability of the entire network I 1 (n, r )= {HIM(N D , N D i ): i = 1, ... , r } I 1 (n, r )= {HIM(N D , N D i ): i = 1, ... , r } I 1 (n, r )= {HIM(N D , N D i ): i = 1, ... , r } Distances between the network constructed on the whole dataset and the networks inferred from the different subsampling replicates. I 2 (n, r )= {HIM(N D i , N D j ): i , j = 1, ... r , i = j } I 2 (n, r )= {HIM(N D i , N D j ): i , j = 1, ... r , i = j } I 2 (n, r )= {HIM(N D i , N D j ): i , j = 1, ... r , i = j } Mutual distances among the networks inferred from the different subsampling replicates. Stability (reliability) of single nodes and links I 3 (n, r )= {a D i hk } I 3 (n, r )= {a D i hk } I 3 (n, r )= {a D i hk } for i = 1, ... , r and k , h = 1, ... , p I 4 (n, r )= {∂ (x D i h )} I 4 (n, r )= {∂ (x D i h )} I 4 (n, r )= {∂ (x D i h )} for i = 1, ... , r and h = 1, ... , p and ∂ the degree function Variability of node degree and link weight of the networks inferred from the different subsampling replicates R ESAMPLING S CHEMA : I LOO (leave-one-out stability): n = s - 1,r = 1 I 20 × k -fold 20 × k -fold 20 × k -fold cross validation for k = 2, 4, 10 (k 2, k 4 and k 10) -→ n = s(k -1) k and r = 20k . HIM N ETWORK D ISTANCE HIM Metric Product Metric of { Hamming - edit distance - focus only on local presence/absence of matching links. Ipsen-Mikhailov - spectral distance - evaluate global structure of topologies. (A) (B) H IM 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 P(A,B) 0.6 P(A,F) 0.59 P(A,E) 0.52 I II III IV HIM(G , H)= 1 √ 2 H(G , H) 2 + IM(G , H) 2 , H(G , H)= 1 N (N -1) 1≤i =j ≤N |A (1) ij - A (2) ij | , IM(G , H)= γ (G , H)= ∞ 0 [ρ G (ω , γ ) - ρ H (ω , γ )] 2 dω . Representation of the HIM distance in the Ipsen-Mikhailov and Hamming distance space between networks A versus B, F and E, where F is the fully connected network and E is the empty one. P (G , H) represents the distance between two networks G and H whose coordinates are x = H(G , H) x = H(G , H) x = H(G , H) and y = IM(G , H) y = IM(G , H) y = IM(G , H) and the norm of P is √ 2 times the HIM distance HIM(G , H). I N -nodes network as N -atoms system connected by identical elastic strings, ¨ x i + N j =1 A ij (x i - x j )= 0 for i = 0, ··· , N - 1, I The vibrational frequencies ω i satisfy λ i = ω 2 i , I ρ(ω )= K N -1 i =1 γ (ω - ω k ) 2 + γ 2 , spectral density as sum of Lorentz distributions, I γ is the common width, half-width at half-maximum (HWHM), equal to half the interquartile range, γ (E , F )= 1, I K is the normalization constant solution of ∞ 0 ρ(ω )dω = 1. 0 1 2 3 4 5 6 7 G 0 2 4 6 8 0.00 0.05 0.10 0.15 0.20 0.25 0.30 ρ G ( γ , ω ) 2 0 4 6 8 spec(L G ) REFERENCES Baralla et al. Inferring Gene Networks: Dream or Nightmare? Annals of the New York Academy of Science, 2009. Budhu et al. Identification of Metastasis-Related MicroRNAs in Hepatocellular Carcinoma Hepatology, 2008. Faith et al. Large-Scale Mapping and Validation of Escherichia coli Transcriptional Regulation from a Compendium of Expression Profiles PLoS Biology, 2007. Gillis and Pavlidis The role of indirect connections in gene networks in predicting function Bioinformatics, 2011. Ipsen and Mikhailov Evolutionary reconstruction of networks Physical Review E, 2002. Jurman et al. Stability Indicators in Network Reconstruction arXiv, 2012. Jurman et al. A glocal distance for network comparison arXiv, 2012. Meyer et al. Verification of systems biology research in the age of collaborative competition Nature Biotechnology, 2011. Miller et al. Identifying Biological Network Structure, Predicting Network Behavior, and Classifying Network State With High Dimensional Model Representation (HDMR) PLoS ONE, 2012. Prill et al. Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges PLoS ONE, 2010. Reshef et al. Detecting novel associations in large datasets Science, 2011 Volinia et al. Reprogramming of miRNA networks in cancer and leukemia Genome Research, 2010. FDR EFFECT ON CORRELATION NETWORKS To assess the different level of stability in a correlation network inferred by a set of synthetic high-throughput signals when the inference is computed with or without False Discovery Rate control. S YNTHETIC B ENCHMARK . Corr(f i , f j ) ≈ 0.9 0.7 0.4 f20 f19 f18 f17 f16 f15 f14 f13 f12 f11 f10 f9 f8 f7 f6 f5 f4 f3 f2 f1 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 I WGCNA [Langfelder et al., 2008] I MIC [Reshef et al., 2011] I WGCNA with FDR correction Adj = {a hk } where a hk = |Corr(x h , x k )| if|F z (h, k )|≥ 1 0 otherwise I1 I2 0.00 0.02 0.04 0.06 0.00 0.05 0.10 k10 k2 k4 LOO ● ● ● ● k10 k2 k4 LOO k10 k2 k4 LOO k10 k2 k4 LOO k10 k2 k4 LOO MINE WGCNA WGCNAFDR1e-2 WGCNAFDR5e-3 WGCNAFDR1e-4 ● MI RNA NETWORK ON A H EPATOCELLULAR C ARCINOMA DATASET I 482 tissue samples from 241 patients [Budhu et al., 2008, Volinia et al., 2010]. I For each patients, a sample from cancerous hepatic tissue and a sample from surrounding non-cancerous hepatic tissue. I Ohio State University CCC MicroRNA Microarray 2.0: 11520 probes, 250 non-redundant human and 200 mouse miRNA. I After preprocessing, the dataset HCC of 240+240 paired samples described by 210 human miRNA is analyzed (210 ♂+ 30 ♀). I HCC is partitioned into four subsets combining the sex and disease status phenotypes(MT, FT, MnT, FnT). I NFERENCE A LGORITHMS C OMPARISON I I1 captures the robustness of the algorithms to subsampling. I I2 tends to express the homogeneity of the dataset. I The bigger the sample-size for reconstruction the stabler the result. I1 I2 0.00 0.02 0.04 0.06 0.08 0.0 0.2 0.4 0.6 k10 k2 k4 LOO ● ● ● ● k10 k2 k4 LOO k10 k2 k4 LOO k10 k2 k4 LOO ARACNE CLR TOM WGCNA ● MT FT MnT FnT let.7a.1.prec let.7a.2.precNo1 let.7a.2.precNo2 let.7a.3.prec let.7b.prec let.7c.prec let.7d.prec let.7d.v1.prec let.7d.v2.precNo2 let.7e.prec let.7f.1.precNo2 let.7f.2.prec2 let.7g.precNo1 let.7iNo1 let.7iNo2 007.2.precNo2 007.3.precNo1 009.3No1 010a.precNo1 010b.precNo1 016a.chr13 016b.chr3 017.precNo2 020.prec 021.prec.17No1 023a.prec 023b.prec 024.1.precNo1 024.1.precNo2 024.2.prec 025.prec 026a.precNo1 026b.prec 027a.prec 027b.prec 029a.2No1 029a.2No2 029c.prec 030a.precNo1 030b.precNo1 030c.prec 030d.precNo2 031.prec 032.precNo1 032.precNo2 034precNo1 092.prec.13.092.1No1 092.prec.13.092.1No2 092.prec.X.092.2 093.prec.7.1.093.1 095.prec.4 096.prec.7No2 099b.prec.19No2 099.prec.21 100.1.2.prec 100No1 101.1.2.precNo2 102.prec.1 103.2.prec 103.prec.5.103.1 105.prec.X.1.105.1 106aNo1 106bNo1 106.prec.X 107No1 107.prec.10 123.precNo1 123.precNo2 124a.1.prec1 124a.2.prec 125a.precNo1 125b.1 126No1 126No1 126No2 126No2 128a.precNo2 129.2No1 129.precNo1 1.2No1 1.2No2 130a.precNo2 130bNo1 130bNo2 132.precNo2 133bNo2 135.2.prec 135a.1No1 135a.2No1 38.2.prec 142.prec 145.prec 148aNo1 148bNo1 148bNo2 148.prec 149.prec 150.prec 152.precNo2 155.prec 15aNo1 16.1No1 16.2No1 181a.precNo2 181b.1No1 181b.2No1 181b.2No2 181b.precNo1 181c.precNo1 184.precNo2 185.precNo2 192.2.3No1 192No1 193.precNo1 194.1No1 194.2No1 194.precNo1 195.prec 196a.1No1 196a.2No1 196bNo2 197.prec 199a.1.prec 199b.precNo2 206.precNo1 206.precNo2 213.precNo1 214.prec 215.precNo1 215.precNo2 216.precNo1 219.1No1 219.1No2 219.2No2 21No1 221.prec 222.precNo1 222.precNo2 223.prec 224.prec 26a.1No1 26a.1No2 26a.2No1 296No1 299No1 29b.1No1 29b.2.102prec7.1.7.2 302bNo2 30c.1No1 30c.2No1 30eNo1 320No2 321No1 321No2 323No2 324.5pNo1 324No2 325No1 326No1 326No2 328No1 335No2 338No1 338No2 339No2 340No2 342No1 345No2 346No1 34aNo1 34bNo2 368No1 371No1 373No1 373No1 373No2 373No2 3p21.v1.v2.AntiS5P 3p21.v1.v2.sense5P 3p21.v3.v4.sense13P 3p21.v3.v4.sense35P 3p21.v3.v4.sense45P let.7a.1.prec let.7a.2.precNo1 let.7a.2.precNo2 let.7a.3.prec let.7b.prec let.7c.prec let.7d.prec let.7d.v1.prec let.7d.v2.precNo2 let.7e.prec let.7f.1.precNo2 let.7f.2.prec2 let.7g.precNo1 let.7iNo1 let.7iNo2 007.2.precNo2 007.3.precNo1 009.3No1 010a.precNo1 010b.precNo1 016a.chr13 016b.chr3 017.precNo2 020.prec 021.prec.17No1 023a.prec 3b.prec 024.1.precNo1 024.1.precNo2 024.2.prec 025.prec 026a.precNo1 026b.prec 027a.prec 027b.prec 029a.2No1 029a.2No2 029c.prec 030a.precNo1 030a.precNo2 030b.precNo1 030c.prec 030d.precNo2 031.prec 032.precNo1 032.precNo2 034precNo1 092.prec.13.092.1No1 092.prec.13.092.1No2 092.prec.X.092.2 .prec.7.1.093.1 095.prec.4 096.prec.7No2 099b.prec.19No2 099.prec.21 100.1.2.prec 100No1 101.1.2.precNo1 101.1.2.precNo2 102.prec.1 103.2.prec 103.prec.5.103.1 105.prec.X.1.105.1 106aNo1 106bNo1 106.prec.X 107No1 107.prec.10 122a.prec 123.precNo1 123.precNo2 124a.1.prec1 124a.2.prec 124a.3.prec 125a.precNo1 125b.1 125b.2.precNo2 126No1 126No1 126No2 126No2 127.prec 128a.precNo1 128a.precNo2 128b.precNo1 129.2No1 129.precNo1 129.precNo2 1.2No1 1.2No2 130a.precNo2 130bNo1 130bNo2 132.precNo2 133a.1 133bNo2 135.2.prec 135a.1No1 135a.2No1 136.precNo2 138.2.prec 140No2 142.prec 145.prec 146.prec 148aNo1 148bNo1 148bNo2 148.prec 149.prec 150.prec 152.precNo1 152.precNo2 155.prec 15aNo1 16.1No1 16.2No1 181a.precNo2 181b.1No1 181b.2No1 181b.2No2 181b.precNo1 181c.precNo1 184.precNo2 185.precNo2 191.prec 192.2.3No1 192No1 193.precNo1 193.precNo2 194.1No1 194.2No1 194.precNo1 195.prec 196a.1No1 196a.2No1 196bNo2 197.prec 198.prec 199a.1.prec 199b.precNo2 205.prec 206.precNo1 206.precNo2 210.prec 212.precNo1 212.precNo2 213.precNo1 214.prec 215.precNo1 215.precNo2 216.precNo1 218.2.precNo2 219.1No1 219.1No2 219.2No2 21No1 221.prec 222.precNo1 222.precNo2 223.prec 224.prec 26a.1No1 26a.1No2 26a.2No1 296No1 299No1 29b.1No1 29b.2.102prec7.1.7.2 301No2 302bNo2 30c.1No1 30c.2No1 30eNo1 320No1 320No2 321No1 321No2 323No2 324.5pNo1 324.5pNo2 324No2 325No1 326No1 326No2 328No1 331No2 335No2 338No1 338No2 339No2 340No2 342No1 342No2 345No2 346No1 34aNo1 34bNo2 34cNo2 368No1 371No1 373No1 373No1 373No2 373No2 3p21.v1.v2.AntiS5P 3p21.v1.v2.sense5P 3p21.v3.v4.sense13P 3p21.v3.v4.sense35P 3p21.v3.v4.sense45P let.7a.2.precNo1 let.7a.3.prec let.7c.prec let.7d.prec let.7d.v1.prec let.7d.v2.precNo2 let.7e.prec let.7f.2.prec2 let.7iNo1 let.7iNo2 007.2.precNo2 009.3No1 016a.chr13 016b.chr3 017.precNo2 020.prec 021.prec.17No1 023a.prec 023b.prec 024.1.precNo2 024.2.prec 025.prec 026a.precNo1 026b.prec 027a.prec 027b.prec 029a.2No1 029a.2No2 030a.precNo1 030b.precNo1 030c.prec 030d.precNo2 031.prec 032.precNo1 032.precNo2 092.prec.13.092.1No2 092.prec.X.092.2 093.prec.7.1.093.1 095.prec.4 101.1.2.precNo1 101.1.2.precNo2 102.prec.1 105.prec.X.1.105.1 107No1 123.precNo2 124a.1.prec1 124a.2.prec 126No2 129.2No1 129.precNo1 1.2No1 1.2No2 130a.precNo2 130bNo1 130bNo2 132.precNo2 133a.1 133bNo2 135a.1No1 135a.2No1 38.2.prec 148aNo1 148bNo1 148bNo2 149.prec 152.precNo1 152.precNo2 16.2No1 181a.precNo2 181b.1No1 181b.2No1 181b.2No2 181b.precNo1 181c.precNo1 185.precNo2 191.prec 192.2.3No1 192No1 193.precNo1 194.2No1 196a.1No1 196a.2No1 196bNo2 198.prec 199a.1.prec 199b.precNo2 206.precNo1 206.precNo2 210.prec 213.precNo1 214.prec 215.precNo1 215.precNo2 216.precNo1 219.1No1 219.2No2 21No1 222.precNo1 26a.1No1 26a.1No2 26a.2No1 296No1 299No1 29b.1No1 302bNo2 30c.1No1 30c.2No1 320No2 321No1 321No2 323No2 324.5pNo1 325No1 326No1 326No2 328No1 331No2 338No1 339No2 340No2 342No2 345No2 346No1 34bNo2 368No1 371No1 373No1 373No1 373No2 373No2 3p21.v1.v2.AntiS5P 3p21.v1.v2.sense5P 3p21.v3.v4.sense13P 3p21.v3.v4.sense35P let.7a.1.prec let.7a.2.precNo1 let.7a.2.precNo2 let.7a.3.prec let.7b.prec let.7c.prec let.7d.prec let.7d.v1.prec let.7d.v2.precNo2 let.7e.prec let.7f.1.precNo2 let.7f.2.prec2 let.7g.precNo1 let.7iNo1 let.7iNo2 007.2.precNo2 007.3.precNo1 009.3No1 010a.precNo1 016a.chr13 016b.chr3 017.precNo2 020.prec 021.prec.17No1 023a.prec 3b.prec 024.1.precNo1 024.1.precNo2 024.2.prec 025.prec 026a.precNo1 026b.prec 027b.prec 029a.2No1 029a.2No2 029c.prec 030a.precNo1 030a.precNo2 030b.precNo1 030c.prec 030d.precNo2 031.prec 032.precNo1 032.precNo2 034precNo1 092.prec.13.092.1No1 092.prec.13.092.1No2 092.prec.X.092.2 prec.7.1.093.1 095.prec.4 096.prec.7No2 099b.prec.19No2 099.prec.21 100.1.2.prec 100No1 101.1.2.precNo1 101.1.2.precNo2 102.prec.1 103.prec.5.103.1 105.prec.X.1.105.1 106aNo1 106bNo1 106.prec.X 107No1 107.prec.10 123.precNo1 123.precNo2 124a.1.prec1 124a.2.prec 125a.precNo1 125b.1 125b.2.precNo2 126No1 126No1 126No2 128a.precNo1 128a.precNo2 128b.precNo1 129.2No1 129.precNo2 1.2No1 1.2No2 130a.precNo2 130bNo2 132.precNo2 133a.1 133bNo2 135.2.prec 135a.2No1 136.precNo2 138.2.prec 140No2 145.prec 146.prec 148bNo2 148.prec 149.prec 150.prec 152.precNo1 152.precNo2 155.prec 15aNo1 16.1No1 16.2No1 181a.precNo2 181b.1No1 181b.2No1 181b.2No2 181b.precNo1 181c.precNo1 184.precNo2 185.precNo2 191.prec 193.precNo1 194.1No1 194.2No1 194.precNo1 196a.1No1 196a.2No1 196bNo2 197.prec 198.prec 199b.precNo2 205.prec 206.precNo1 206.precNo2 212.precNo1 213.precNo1 214.prec 215.precNo1 215.precNo2 216.precNo1 218.2.precNo2 219.1No1 219.1No2 219.2No2 21No1 221.prec 222.precNo1 222.precNo2 223.prec 224.prec 26a.1No1 26a.1No2 26a.2No1 296No1 299No1 29b.1No1 29b.2.102prec7.1.7.2 301No2 302bNo2 30c.1No1 30c.2No1 320No2 321No1 321No2 323No2 324.5pNo1 324.5pNo2 324No2 325No1 326No1 326No2 328No1 331No2 335No2 338No1 338No2 339No2 340No2 342No1 342No2 345No2 346No1 34aNo1 34bNo2 34cNo2 368No1 371No1 373No1 373No1 373No2 373No2 3p21.v1.v2.AntiS5P 3p21.v1.v2.sense5P 3p21.v3.v4.sense35P 3p21.v3.v4.sense45P B IOLOGICAL R ESULTS MnT FT FnT HIM 0.0412 0.0858 0.0235 MT 0.1265 0.0618 MnT 0.0684 FT -0.05 0.00 0.05 -0.005 0.005 Coordinate 1 Coordinate 2 MT MnT FT FnT Statistics on I4 id hsa-mir idx1 hsa-mir idx2 MT MnT FT FnT (a) 321No1 321No2 1 1 9 2 (b) 016b.chr3 16.2No1 3 12 15 309 - (c) 021.prec.17No1 21No1 27 5 2 921 (d) 219.1No1 321No2 2 6 1903 314 (e) 326No1 342No2 132 1017 3 - (f) 192.2.3No1 215.precNo1 4 300 4 3340 (a) is top ranking in all four cases as expected (hsa-mir 321No1 and hsa-mir 321No2 denote essentially the same miRNA) (b) and (c) as (a), but with less stability in the FnT network due to noise. (d) has different stability between the male and the female (link probably associated to sex rather than HCC). (e) is very stable for FT, while is not even picked up as a link by CLR in the FnT network. (f) is a very well known cancer associated link, as confirmed by high stability in MT and FT. I1 I2 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.00 0.02 0.04 0.06 0.08 0.10 0.12 k10 k2 k4 LOO k10 k2 k4 LOO k10 k2 k4 LOO k10 k2 k4 LOO FT FnT MT MnT I1 vs. I2 plot for CLR inferred networks in the 4 subgroups. I M much stabler than F I MT stability similar to MnT I F LOO worse than M 4/10 Fold I FnT much worse than FT Authors acknowledge funding by the European Union FP7 Project HiperDART NetSci 2013 - International School and Conference on Network Science http://mpba.fbk.eu {jurman, filosi, visintainer, furlan}@fbk.eu, [email protected]
