• Tronche (1997) tested 50 predicted HNF1 TFBS using an in vitro binding test and found that 96% of the predicted sites were bound!
• Stormo and Fields (1998) found in detailed biochemical studies that the best weight matrices produce scores highly correlated with in vitro binding energy
PSSM SCORE
BIN
DIN
G
EN
ER
GY
3 Ackn.: R. Bruskiewich and F. Brinkman, MBB with material from Wyeth W. Wasserman and Shannan Ho Sui
…The Bad…
• Fickett (1995) found that a profile for the myoD TF made predictions at a rate of 1 per ~500bp of human DNA sequence
– This corresponds to an average of 20 sites / gene (assuming 10,000 bp as average human gene size)
4 Ackn.: R. Bruskiewich and F. Brinkman, MBB with material from Wyeth W. Wasserman and Shannan Ho Sui
31/03/2014
3
…and the Ugly! Human Cardiac a-Actin gene analyzed
with a set of profiles (each line represents a TFBS prediction)
Futility Conjecture: TFBS
predictions, as a collective group,
are almost always wrong!
True binding sites are defined by
properties not incorporated into
the PSSM profile scores
Red boxes are protein coding exons -
TFBS predictions excluded in this analysis
5 Ackn.: R. Bruskiewich and F. Brinkman, MBB with material from Wyeth W. Wasserman and Shannan Ho Sui
PSSMs - Conclusions
6 Ackn.: R. Bruskiewich and F. Brinkman, MBB with material from Wyeth W. Wasserman and Shannan Ho Sui
• PSSMs accurately reflect in vitro binding properties of DNA binding proteins
• Suitable binding sites occur at a rate far too frequent to reflect in vivo function
• Bioinformatics methods that use PSSMs for binding site studies must incorporate additional information to enhance specificity
• Unfiltered predictions are too noisy for most applications • Note: Organisms with short regulatory sequences are less problematic
(e.g. yeast and bacteria)
Example of a bioinformatics challenge that needs more biological information to make predictions!
31/03/2014
4
Phylogenetic footprinting
Phylogenetic footprinting relies upon two major concepts:
1. The function and DNA binding preferences of transcription factors are well-conserved between diverse species.
2. Important non-coding DNA sequences that are essential for regulating gene expression will show differential selective pressure. A slower rate of change occurs in TFBS than in other, less critical, parts of the non-coding genome.
How to improve? PhysBinder: an integrative tool based on random forest and flexible inclusion of
biophysical properties improves prediction of transcription factor binding sites.
The noncontacted spacer region in the DNA-binding site for the E2 protein encoded by the HPV type 16 genome
– role of nucleotides INSIDE of the TF binding motif
Zhang Y et al. PNAS 2004;101:8337-8341
E2 is a homodimer where each monomer inserts an alpha-helix in the major groove of an "ACCG" sequence (direct readout), holding the DNA in an arc and leaving the spacer NNNN uncontacted by the E2 protein. This spacer is responsible for the indirect readout effect since it is not bound by E2 but its flexibility (if AT-rich) allows for the DNA to be bent "around" E2. The higher the bendability of this region the higher the affinity and the strength of the interaction.
31/03/2014
8
Genomic Regions Flanking E-Box Binding Sites Influence DNA Binding Specificity of bHLH Transcription Factors through DNA Shape – role of nucleotides located OUTSIDE of the TF binding motif
model the binding specificity from a collection of aligned sequences known to bind the TF in vitro or in vivo
treat DNA as a uniform static structure that is independent of the nucleotide sequence
the PWM method [Stormo] takes into account only the nucleotide frequency at each position of the TFBS and assumes independence between those positions (additivity).
Recently, it was shown that for most TFs, dependencies exist between nucleotide positions in their binding sites [Hu; ChIP-Seq].
• Structure-based methods:
Use information from available crystal structures of TF–DNA complexes [e.g. Angarica]
Some are valuable for comparative modelling and seem promising for TFBS prediction
None of the structure-based methods have offered substantial improvement over the PWM method yet!
Modelling transcription factor binding specificity (II)
The PhysBinder Approach
• Sequence-based method
Uses the random forest (RF) algorithm with features that cover:
Nucleotide positional dependencies -> NPD model
Nucleotide sequence-dependent structural characteristics -> structural model
Combines the NPD model and the structural model and tries to integrate the PWM score in the combined model
The goal is to find the features combination(s) that maximize(s) the classification accuracy for each transcription factor binding model individually
31/03/2014
10
Random Forest algorithm
http://www.ualberta.ca/~drr3/random-forest.html
• Methodology
Ensemble of unpruned classification or regression trees (CARTs) by bootstrapping samples of the training data and using random feature selection in the tree induction process.
Disadvantage: embedded feature selection procedure cannot handle large numbers of irrelevant features: comprehensive filter feature selection and wrapper-based feature selection before the final
model is trained
Structural features – Pearson correlation analysis
Red: no correlation, Yellow: slight correlation, White: high correlation The structural characteristics are correlated to some extent:
the feature selection procedures and the RF algorithm decide which features are most relevant for each TF.
31/03/2014
11
Building the Model – (A) Input
ChIP-Seq Peak Regions (bed),…
Extract Genomic Sequences (fasta)
Align sequences using MEME: Extend 50 bp upstream and downstream
w.r.t. the start of the found motif.
Use 100 to build Model Use remainder for validation
Positive sequences (P) Negative sequences (N)
Randomly select 1000 background sequences with a length of 100 bp from the genome of interest.
(B) Calculation of the structural and NPD profiles
Each nucleotide sequence, from either class, is converted into multiple series of
values; each series provides values for a specific DNA structural characteristic
at all positions of the TFBS and its context (structural model), or simply consists
of one base or two base parts of the sequence (NPD).
31/03/2014
12
(C) Filtering out relevant features
• Basic selection of relevant features (i.e. positions) is made by statistical
comparison of distributions of values for positive and negative sequences
with mild thresholds:
• In order not to exclude too many features
• To permit detection of their interactions later on
(D) Wrapper-based feature selection Further selection is performed through cross-validation performance evaluation
with the RF algorithm. Per characteristic, redundant features are removed by
sequential backwards elimination (SBE). Several models with one characteristic
might be merged through best incremental ranked subset (BIRS). The final NPD
model and final structural model can be merged into one integrative model.
31/03/2014
13
The resulting model can be used by RF to
predict the likelihood that a nucleotide
sequence is a TFBS, after converting the
sequence into series of the features
contained in the model.
(E) TFBS prediction
based on their prediction scores:
for both the structural method and the NPD method this is the RF confidence score, which is assigned to each sequence and indicates the certainty with which this sequence is predicted to belong to either the positive or the negative class.
for PWMs, we used the matrix similarity score.
visualized by ROC curves and precision-recall curves.
Each ROC and precision-recall curve shown is derived from a threshold-based average of 20 curves. Data for each of these 20 curves were obtained by training the model with a randomly taken subset of 80% of the data and testing that trained model on the remaining 20%.
Principle component analysis was performed on the full models to select a top five feature set for each TF (default parameters).
Evaluation of performance of the classification models
• Both the structural model and the NPD model include features at positions that precede the actual TFBS.
• The background genomic sequence in which SP1 binding sites are embedded is very similar to the consensus sequence of such sites.
• A PWM would thus predict many TFBSs, whereas the NPD model and structural model can look beyond position-independent nucleotide frequencies, each in its own way.
31/03/2014
17
Biological relevance of the top 5 selected features (PCA)
3 out of 5 top features are dinucleotide features. The dinucleotides together build the pattern 5′-CGTG-3′, known as the hypoxia-response element (HRE). Most important determining factor of HIF1 binding ; fully conserved in every HIF1 binding site.
the majority deals with the DNA conformation and the tendency to the A/B-DNA conformation. A shift to a non-standard B-DNA conformation can drastically alter the binding capacity of P53 and is likely responsible for the specific binding to the wide variety of P53 motifs
‘SP1’ distorts the B-structure of the DNA toward a more A-DNA oriented structure . Two global features of the SP1 model confirm the importance of DNA conformational features. The two CC-dint features are an indication of the cytosine enrichment in the canonical SP1 recognition element (CCCGCC).
‘STAT1’ shows a strong preference for sequences containing two palindromic half-sites (TTC…GAA), leading to a dyad symmetry. The inclusion of the dinucleotide features for AA, TT, GA and TC, together TTTC…GAAA, is the most specific variant of all STAT1 binding motifs (Ehret et al.)
‘TBP’ is one of the most well known DNA benders and it was shown that the unbound TATA box is already pre-bent. When looking at the top five features, four out of five top features contain properties about DNA bending, confirming the tendency of TBP to bend the DNA.
1. Inherent structural properties of DNA are involved in specific recognition by TFs to an extent that depends on each TF
2. A purely structural model performs often worse than the NPD model describing both base readout and a big portion of shape readout. – The relative importance of the more simple NPD characteristic consequently cannot be
ignored when analyzing TFBS binding patterns in the eukaryotic models.
3. Structural properties contain information other than the nucleotide sequence and can be used to further improve classification accuracy.
4. The PWM score that merely represents base readout in its most simple form, is sometimes complementary to the model combining the structural model and NPD model.
5. Most importantly, we present an integrative approach that can easily combine two or three different approaches to establish the best possible prediction of TFBSs.
PhysBinder - Implementation
PhysBinder is a novel online tool that is based on a flexible and extensible algorithm for the prediction of TFBSs.
Broos S, Soete A, Hooghe B, Moran R, van Roy F, De Bleser P. PhysBinder: improving the prediction of transcription factor binding sites by flexible inclusion of biophysical properties. Nucleic Acids Res. 2013 Apr 24. [Epub ahead of print] PubMed PMID: 23620286.
Hooghe B, Broos S, van Roy F, De Bleser P. A flexible integrative approach based on random forest improves prediction of transcription factor binding sites. Nucleic Acids Res. 2012 Aug;40(14):e106. doi: 10.1093/nar/gks283. Epub 2012 Apr 5. PubMed PMID: 22492513; PubMed Central PMCID: PMC3413102
ZEB Transcription Factor Switching in Melanoma Cells Results in Mitf Loss, and is Associated with Melanoma Progression: Is Mitf a target gene for Zeb1?
Downregulation of MITF promoter activity in the B16 melanoma cell line upon overexpression of Zeb1. An empty vector was used as a control (CTRL).
Identification of highly conserved ZEB1 binding sites in the MITF 5’- region by using Physbinder.
31/03/2014
20
Input
1
Input
2
31/03/2014
21
Input
Genomic regions can be fetched from a variety of species and associated genome assemblies.
Input
1
2
You can enter multiple locations at once, one per line…
31/03/2014
22
Input
3
Input
31/03/2014
23
Input - Summary
1
2
3
Sequences can be uploaded by one of the following means: 1. pasting a set of FASTA-formatted
sequences in the input field 2. indicating genomic regions in the ‘Fetch
genomic regions’ text field. 3. uploading a file with FASTA-formatted
sequences
Select a treshold Precalculated thresholds ("Max. Precision", "Average", "Max. F-Measure") are calculated on an external control set. Alternatively, one can enter a custom threshold. Scores and thresholds range from 1 to 1000. This score is calculated by the Random Forest algorithm and indicates the confidence the algorithm has in the result. We calculated thresholds to decide which results are valid in three ways: 1. The Max. Precision threshold guarantees a minimal number of false positive
predictions, while assuring that the positive predictions are of top quality. 2. The Max. F-Measure threshold tries to balance precision and recall (% of
identified true positives). It is a weighted average of the precision and recall, where an F-Measure score reaches its best value at 1 and worst value at 0.
3. The Average threshold is the average between precision and the F-Measure threshold. This threshold is a good starting point if you have no idea about the most suitable threshold.
Users that want to use their own custom score should take a look at the ROC curves on the models page to get an idea of what to expect from each score.
31/03/2014
24
Select a treshold
Just select one of the three precalculated thresholds ("Max. Precision", "Average", "Max. F-Measure") or set a custom threshold (enter a number between 1 and 1000). The default threshold is "Average."
Select a transcription factor binding site model Currently we offer more than 60 different vertebrate TFBS models. The number of models will continue to grow in the future. All models are build from available ENCODE ChIP-Seq experiments and other sources (e.g. http://www.ncbi.nlm.nih.gov/geo/) . It is possible to filter models by species name or by model type or to use a custom search term. Model type: 1. DE (Direct Evidence): models built from experimental data that
clearly contain a consensus motif that has been reported in literature.
2. PAF (Possibly Associated Factor): models built from ChIP-Seq data that clearly contain a sensible motif that has NOT been associated with the transcription factor yet. These are often factors associated with the transcription factor. It is also possible that the model represents the actual transcription factor with a consensus sequence that is not yet known.
You can select models in the models window. We use color codes to indicate the type of model: green means DE; yellow stands for PAF; human is blue and mouse is grey.
Filter for species name. Currently we have human and mouse models.
Select a transcription factor binding site model
31/03/2014
26
Select a transcription factor binding site model
Filtering for evidence type. Choose "DE" or "PAF".
Select a transcription factor binding site model
Enter a search term in the input field to search for a certain transcription factor. It is possible to search for aliases of the transcription factor. Just make sure you select the "Include Aliases in Search Terms" checkbox.
If you click on the grey triangle at the bottom of a model icon, all aliases will appear.
31/03/2014
27
Optional arguments Additional options that need an extra explanation. - Email address: It is possible to provide an email address but this is not required. If an email address is provided, an email will be sent when the calculations are finished. If the calculations result in an error, you will get an error report. We will not use your email address for any purpose other than informing you about your calculations. - Use as filter: For performance reasons, it is possible to pre-filter the sequences using a short PWM with mild thresholds in order to get a maximum recall. This will really increase the speed! In order to limit the load on our servers, we decided to enable this option by default. Unless you have a specific reason why not to use this filter step, it is best to keep it turned on.
Output - Summary By default, the summary section is hidden. When the user clicks on the green arrow, a table with some statistics about the results is shown. The summary section on the results page gives an indication of the number of hits that exceed the chosen threshold for each model. The results can be sorted according to the model or according to the input sequence.
By default, the summary table is ordered by model and lists the number of hits per sequence. You can also order the table by sequence ID. Each sequence link is clickable.
31/03/2014
28
Output – Change thresholds Thresholds can be changed by entering a new value in the "Change thresholds" field. You can enter any value between 1 and 1000 in this field. You can also type "avg", "ppv", or "f1" in this field, to respectively get the average threshold, the max precision or the max f-measure threshold for each model.
Click on the green arrow to the right of the "Set Threshold line", enter any value you like in the threshold field.
Output – Detailed results
The results are visualized in this section. Per sequence, hits that exceeds the threshold are indicated with a colored bar. The bar is shaded from a light color (low score) to a dark color (high score). An arrow indicates the orientation of the binding site (forward or reverse). More information is displayed by clicking on the arrow. Binding sites of models can be dynamically shown or hidden by clicking on the corresponding checkboxes. Nucleotides in a gray colored font were not scanned due to model limits. Repeats from RepeatMasker and Tandem Repeats Finder are shown in lower case; non-repeating sequence is in upper case (As used in UCSC Genome Browser for downloadable genome data).
Output – Detailed results Show/hide hits of a model by clicking on the checkboxes above each sequence. Toggling this checkbox will only show/hide the hits on this sequence. If no hits are found for a particular model, no checkbox for this model is shown.
Toggling checkboxes on the side of the screen will affect all sequences.
Output – Working with genomic regions.
It is possible to map the different sequences to a human or mouse reference genome. This can be done by clicking on the "blat" button below each sequence. If the sequences were fetched from UCSC on the input page, this is not necessary because the sequence location is known already. If the sequence location is known, either from the input page or by blatting the sequence, some extra options are available: 1. The sequence can be visualized in the UCSC Genome Browser. In
order to do this, click on the "Map to UCSC" button. 2. A BED file with the genomic regions is available for download. Just
click on the "Get BED file" button. 3. ENCODE data available for the region can be integrated into the
results. Click on the "show ENCODE TFBS" button to fetch all ENCODE regions for this sequence.
31/03/2014
30
If the sequence originates from the human or mouse genome, blat can be used to look for the genomic region.
Select the correct reference genome for your sequence. If your sequences are human, select hg19. If the sequences come from mouse, select mm10. Press the Blat button to continue. It can take a while to process the results.
Output – Working with genomic regions.
Output – Working with genomic regions.
When blat is executed, some other options become available. Select the blat result with the highest overlap from the drop-down list (indicated in %). It is now possible to map your sequences to UCSC, to download a BED file and to integrate ENCODE data.
Clicking the "map to UCSC" button will visualize all TFBS hits in the UCSC Genome Browser.
31/03/2014
31
Output – Working with genomic regions.
Binding sites are visualized as a custom track in the genome browser. The original query sequence is shown as a black bar.
Downloading the results is done by clicking the "get BED file" button.
Output – Working with ENCODE data
We integrated the TFBS ChIP-seq sets from human in the PhysBinder web tool. ENCODE data can be integrated by clicking on the "show ENCODE TFBS" button. ENCODE data will be shown, along with the PhysBinder predictions, as grey bars. This way the genomic context of the PhysBinder predictions become immediately clear. The different ENCODE tracks can be toggled on or off using the ENCODE checkboxes
31/03/2014
32
Output – Working with ENCODE data
By clicking on the show ENCODE TFBS button, overlapping TFBS ChIP-seq sets are indicated on the sequence as grey bars.
When no ENCODE tracks are found in the sequence, a message is displayed below the sequence.
Output – Working with ENCODE data
If ENCODE tracks for this genomic region are found, A list of checkboxes will appear next to the sequence. The ENCODE tracks are visualized as simple grey bars below the sequence. In regions with many ENCODE tracks, the grey bars will be stacked and the area becomes darker.
31/03/2014
33
Output – Working with ENCODE data
The different ENCODE tracks can be switched on or off by clicking the corresponding checkboxes. You can switch on or off multiple tracks by clicking the "select all" or "deselect all" option below the checkboxes. Do not forget to press the update button!
Output – Create publication graphics
For each sequence in the results section we offer a download of the FASTA-file and feature-color-file with the binding sites. Both files can be used in Jalview to create custom visualizations of the results.
Click on the download link for the fasta and feature color file to save them locally. This way you can create graphics in Jalview (http://www.jalview.org/ ).
Example Kyo et al. identified a core promoter of 181 bp responsible for the transcriptional activity of the TERT gene, encoding the catalytic subunit of telomerase. This 181-bp region, consisting of the 5’-UTR and the upstream promoter region, contains two E-boxes bound by MYC in vivo. Between these E-boxes, Kyo et al. discovered and validated five GC-boxes that are bound by SP1.
Kyo S et al. Nucl. Acids Res. 2000;28:669-677
Sequences of the hTERT core promoter and consensus motifs for factor
binding sites.
All predicted TFBS match the experimentally determined locations reported by Kyo et al.
31/03/2014
35
Example – Visualization of the models in the UCSC Genome Browser
Q.: I need a genome-wide list of genes possibly targeted by transcription factor X, transcription factor Y,…
A1.: Use PhysBinder-CLI* (recommended but depending on the availability of good ChIP-Seq data to build a model)
“Identification of human and mouse target genes for GATA1, GATA2 and, if possible, TAL1” => We make models on request*. Just point us to a good ChIP-Seq data set.
A2.: Use PWM-based methods (faster, but mind the futility theorem)
• Using PhysBinder explore the vicinity of the TAL1 (hg19) TSS and promoter for GATA1 and GATA2 transcription factor binding sites.
• Map the results to the UCSC genome browser and add tracks for transcription factor binding data and histone marks that are often found near active regulatory elements.
Exercise - Solution
31/03/2014
37
You can download the FASTA file and feature color file of each result. These files can be used to make publication graphics in Jalview.
Output – Create publication graphics
This is how the fasta file looks like
31/03/2014
38
This is how the feature color file looks like
In Jalview, click file > Input Alignment > from File.
Select the FASTA file and click Open.
31/03/2014
39
By default the sequence will be visualized on one line. You can change this behaviour by clicking “Wrap” in the “Format” menu.
Sometimes features will be shown in between the sequence. Turn this on or off with the “Show Annotations” option in menu “View”.
31/03/2014
40
Load the predicted binding sites by going to menu file and clicking the “Load Features / Annotations” option.
31/03/2014
41
Select the fc-file and click open.
The binding sites will be shown on top of the sequences
31/03/2014
42
Change the settings of the different features by going to the “Feature Settings…” option in the “View” menu.
Here you can change the names and colors of the different binding sites or show/hide sites.
31/03/2014
43
Create an image for use in publications or other purposes by going to File > Export Image > PNG or other format.
