Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | calvin-simon |
View: | 218 times |
Download: | 3 times |
Revealing the internal structures of gene
expression data sets
Matthias E. Futschik
Institute for Theoretical Biology
Humboldt-University, Berlin, Germany
Hvar sommer school, 2004
OverviewIt is a two way road: Top-down vs bottom-
up approachesGood to see: Visualisation
PCA and Multi-dimensional scalingGuilt by association: Clustering
Hard clustering vs soft clusteringGattica becomes alive: Classification
Approaches of modelling in molecular biology
Bottom-upapproach
Top-down approach
System-wide measurements
Set of measurements of single component
Underlying molecular mechanism
Network of interactions of single components
Visualisation
• Important tool for detection of patterns remains visualization of results.
• Examples are MA-plots, dendrograms, Venn-diagrams or projection derived fro multi-dimensional scaling.
• Do not underestimate the ability of the human eye
Principal component analysisPCA: • linear projection of data onto majorprincipal components defined by the eigenvectors of the covariance matrix.• PCA is also used for reducing the dimensionality of the data.• Criterion to be minimised: square of the distance between the original and projected data. This is fulfilled by the Karhuven-Loeve transformation
Px Px
1( )( )
1
ti i
i
x xn
C
P is composed by eigenvectors of the covariance matrix
Example: Leukemia data sets by Golub et al.: Classification of ALL and AML
Sammon`s mapping:• Non-linear multi-dimensional scaling such as Sammon's mapping aim to optimally conserve the distances in an higher dimensional space in the 2/3-dimensional space.• Mathematically: Minimalisation of error function E by steepest descent method:
Multi-linear scaling
Example: DLBCL prognosis – cured vs featal cases
2( )1
Nij ij
Ni j ijiji j
D dE
DD
Clustering: Birds of a feather flock together
• Clustering of genes – Co-expression indicates co-regulation:
functional annotation– Clustering of time series
• Clustering of array: – finding new subclasses in sample-space
• Two-way clustering:– Parallel clustering of samples and genes
Clustering methodsUnsupervised classification of genes and/or samples. Movtivation: Co-expression indicates co-regualtion
General division into hierarchical and partitional clustering
Hierachical clustering • can be divisive or agglomerative producing nested clusters. • Results are usually visualised by tree structures dendrogram. • Clustering depends on the linkage procedure used: single, complete, average, Ward,... • A related family of methods are based on graph-theoretical approach (e.g. CLICK).
Profilíng of breast cancer by Perou et al
Example for hierarchical clustering
A Alizadeh et al, Nature. 2000Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling
Clustering methods II
Partitional clustering • divides data into a (pre-)chosen number of classes.• Examples: k-means, SOMs, fuzzy c-means, simulated annealing, model-based clustering, HMMs,...• Setting the number of clusters is problematic
Cluster validity: • Most cluster algorithms always detect clusters, even in random data. • Cluster validation approaches address the number of existing clusters. • Approaches are based on objective functions, figures of merits, resampling, adding noise ....
Hard clustering vs. soft clusteringHard clustering:
• Based on classical set theory
• Assigns a gene to exactly one cluster
• No differentiation how well gene is represented by cluster centroid
• Examples: hierachical clustering, k-means, SOMs, ...
Soft clustering:
• Can assign a gene to several cluster
• Differentiate grade of representation (cluster membership)
• Example: Fuzzy c-means, HMMs, ...
K-means clustering
1
1
0 1
1
0
ij
kk N
hc ij iji
N
ijj
i j
M U R j
N i
2( ) i ji j
E d x c
Partitional clustering is frequently based on the optimisation of a given objective function. If the data is given as a set of N dimensional vectors, a common objective function is the square error function:
where d is the distance metric and cj is the centre of clusters.
• Partitional clustering splits the data in k partitions with a given integer k. • Partition can represented by a partition matrix U that contains the membership values μij of each object i for each cluster j.• For clustering methods, which is based on classical set theory, clusters are mutually exclusive. This leads to the so called hard partitioning of the data.
Hard partions are defined as
k is the number of clusters and N is the number of data objects.
K-means algorithm
• Initiation: Choose k random vectors as cluster centres cj ;
• Partitioning: Assign xi to if for all k with k ;
• Calculation of cluster centres cj based on the partition derived in
step 2: The cluster centre cj is defined as the mean value of all
vectors within the cluster; • Calculation of the square error function E; • If the chosen stop criterion is met, stop; otherwise continue with
step 2.
For the distance metric D, the Euclidean distance is generally chosen.
Hard clustering is sensitive to noise
Example data set:
Yeast cell cylce data by Cho et al.
Standard procedure is pre-filteringof genes based on variation due to noise sensitvity of hard clustering.However, no obvious threshold exists!(Heyer et al.: ca. 4000 genes, Tavazoe et al.: 3000genes, Tamayo et al.: 823 genes)
=> Risk of essential losing information
=> Need of noise robust clustering method
Standard deviation of expression
Soft clustering is more noise robust
Hard clustering always detects clusters, even in random data
Soft clustering differentiates clusterstrength and, thus, can avoid detection of 'random' clusters
Genes with high membership valuescluster together inspite of added noise
Differentiation in cluster membership allows profiling of cluster cores● A gene can be assigned to several clusters● Each gene is assigned to a cluster with a membership value between 0
and 1● The membership values of a gene add up to one● Genes with lower membership values are not well represented by the
cluster centroid● Expression of genes with high membership values are close to cluster
centroid
=> Clusters have internal structures
Membership value > 0.5 Membership value > 0.7
Hard clustering
Varitation in cluster parameter reveals cluster stability
Variation of fuzzification parameter m determines 'hardness' of clustering:m → 1: Fuzzy c-means clustering becomes equivalent to k-means m → ∞: All genes are equivally assigned
to all clusters.
By variation of m clusters can be distinguished by their stability.
Weak cluster lose their core
Strong clusters maintain their core forincreasing m
m=1.1 m=1.3
Periodic and aperiodic clusters
Periodic clusters of yeast cell cycle:
Aperiodic clusters:
=> Aperiodic clusters were generally weaker than periodic clusters
Global clustering structure
c-means clustering allows definitionof overlap of clusters i.e. how manygenes are shared by two clusters. This enables to define a similarity measure between clusters. Global clustering structures can be visualised by graphs i.e. edges representing overlap.
Increasing number of clusters
Non-linear 2D-projection by Sammon's Mapping
=> Sub-clustering reveals sub-structures
M. Futschik and B. Carlisle, Noise robust, soft clustering of gene expression data (in preparation)
Classifaction of microarray data Many diseases involve (unknown) complex interaction of
multiple genes, thus ``single gene approach´´ is limited genome-wide approaches may reveal this interactions
To detect this patterns, supervised learníng techniques from pattern recognition, statistics and artificial intelligence can be applied.
The medical applications of these “arrays of hope” are various and include the identification of markers for classification, diagnosis, disease outcome prediction, therapeutic responsiveness and target identification.
Gattica – the Art of Classifaction
In contrast to clustering approaches, algorithms for supervised classification are based on labelled data.
Labels assign data objects to a predefined set of classes.
Frequently the class distributions are not known, so the learning of the classifiers is inductive.
Task for classification methods is the correct assignment of new examples based on a set of examples of known classes.
Generalisation
Class 1
Class 2
?
?
?
Challanges in classification of microarray data
• Microarray data inherit large experimental and biological variances
• experimental bias + tissue hetrogenity• cross-hybridisation• ‘bad design’: confounding effects
• Microarray data are sparse • high-dimensionality of gene (feature) space• low number of samples/arrays• Curse of dimensionality
• Microarray data are highly redundant•Many genes are co-expressed, thus their expression is strongly
correlated.
Classification I: Models
• K-nearest neighbour– Simple and quick method
• Decision Trees– Easy to follow the classification process
• Bayesian classifier– Inclusion of prior knowledge possible
• Neural Networks– No model assumed
• Support Vector Machines– Based on statistical learning theory; today`s state of art.
Criteria for classification
Accuracy: how closely are the results to the true values
Precision: how variable are the results compared to the true value
Sensitivity: how many true posítive are detected
Specificity: how many of the selected genes are true positives.
1-SensitivityS
pec
ific
ity
ROC curve
Getting a good team:
Feature Extraction - gene selection Selection of genes based on :• Parametric tests e.g. t-test• Non-parametric tests eg. Wilcoxon-Rank test But the 11 best players do not necessary form the best team!
Selection of groups of genes which act as good: Sensitivity measure
Genetic Algorithms
Decision tree
SVD
Bayesian Classifiers
( ) ( ) ( ) P C P C Px x x
The most fundamental classifier in statistical pattern recognition is the Bayes classifier which is directly derived from the Bayes theorem.
Suppose a vector x belongs to one of k classes. The probability P(C,x) of observing x belonging to class C is
P(x|C): conditional probability for x given class C is observed
P(C): prior probability for class C
Similarly, the joint probability P(C,x) can be expressed by
P(C|x): conditional probability for C given object x is observed P(x): is the prior probability of observing x.
( ) ( ) ( ) P C P C P Cx x
Bayesian Classifiers
( ) ( ) ( ) ( ) P C P C P C Px x x
( ) ( )( )
( )
P C P CP C
P
xx
x
( ) ( ) j kP C P Cx x
Since equations 1 and 2 describe the same probability P(C,x), we can derive
for all classes Cj with . This rule constitutes the Bayes classifier.
This is the famous Bayes theorem, which can be applied for classification as follows. We assigned x to class Cj if
Decision trees
• Stepwise classification into two classes• Comprehensible ( in contrast to black box approaches)• Overfitting can occur easily• Complex (non-linear) interactions of genes may not be reflected in tree structure
Aritificial neural networks
1
1
i ii
w x by
e
•ANN were originally inspired by the functioning of biological neurons.
•Two major components: neurons and connections between them.
•A neuron receives inputs xi and determines the output y based on an
activation function. An example of a non-linear activation function is the sigmoid function
• Multi-layer perceptrons are hierachically structured and trained by backpropagation
Support vector machines
( )
( )
( )( )( )
( )
( )( )
(.)( )
( )
( )
( )( )
( )
( )
( )( )
( )
Feature spaceInput space
( ) ( ) dK x y x y
•SVMs are based on statistical learning theory and belong to the class of kernel based methods.
•The basic concept of SVMs is the transformation of input vectors into a highly dimensional feature space where a linear separation may be possible between the positive and negative class members
Example study: Tumour/Normal classification
Motivation: Colon cancer should be detected as early as possible to avoid invasive treatment.
Data: Study by Alon et al. based on expression profiling of 60 samples with Affymetrix GeneChips containing over 6000 genes.
Method: Adaptive neural networks
M.Futschik et al, AI in medicine, 2003
Network structure can be translated in
linguistic rules