Dr. André MatagneCentre for Protein Engineering
Protein Factory & Robotein® facilitiesUNIVERSITY OF LIEGE
Joint initiative on the quality of recombinant proteins and peptides
Quality control of purified proteins to improve research data reproducibility
Dr. André MatagneCentre for Protein Engineering
Protein Factory & Robotein® facilitiesUNIVERSITY OF LIEGE
Joint initiative on the quality of recombinant proteins and peptides
Quality control of purified proteins to improve research data reproducibility
Clemens Fraikin(Jasco Benelux B.V.)
Dr. André MatagneCentre for Protein Engineering
Protein Factory & Robotein® facilitiesUNIVERSITY OF LIEGE
Joint initiative on the quality of recombinant proteins and peptides
Quality control of purified proteins to improve research data reproducibility
Dr. André MatagneCentre for Protein Engineering
Protein Factory & Robotein® facilitiesUNIVERSITY OF LIEGE
Joint initiative on the quality of recombinant proteins and peptides
Quality control of purified proteins to improve research data reproducibility
Dr. André MatagneCentre for Protein Engineering
Protein Factory & Robotein® facilitiesUNIVERSITY OF LIEGE
Joint initiative on the quality of recombinant proteins and peptides
Quality control of purified proteins to improve research data reproducibility
QC team of ARBRE and P4EUNick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)
Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)
Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)
Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)
Blanca Lopez-Mendez – University of Copenhagen, Denmark (ARBRE-MOBIEU)
Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)
André Matagne – University of Liège, Belgium (ARBRE-MOBIEU)
Annabel Parret - EMBL Hamburg, Germany (P4EU)
Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)
Kim Remans - EMBL Heibelberg, Germany (P4EU)
Stephan Uebel - MPI for Biochemistry Munich, Germany (ARBRE-MOBIEU)
• A lot of time and resources is spent on poor quality samples
• The best experiments in the world will turn rubbish into expensive rubbish
Our analysis of the situation (as core facilities) in different European institutions
• A lot of time and resources is spent on poor quality samples
Improving the quality of the samples is essential to enhance the quality (i.e. accuracy, reproducibility) of the results we produce
• The best experiments in the world will turn rubbish into expensive rubbish
Our analysis of the situation (as core facilities) in different European institutions
• A lot of time and resources is spent on poor quality samples
“It is the way we have prepared samples in the lab for ten years….”“But some experiments have worked with this sample…”“SEC? DLS? MS?... My boss thinks it is a waste of time…”“I’ll do the experiment anyway - it may work…”“I don’t know how to do it…”“I don’t have time…”
Our analysis of the situation (as core facilities) in different European institutions
Freedman LP, Cockburn IM, Simcoe TS (2015) The Economics of Reproducibility in Preclinical Research. PLoS Biol 13(6): e1002165.(using 2012 data) 459 citations on April 19th, 2021 (source: Scopus)
(56.4109)
Tremendous economic impact
Freedman LP, Cockburn IM, Simcoe TS (2015) The Economics of Reproducibility in Preclinical Research. PLoS Biol 13(6): e1002165.(using 2012 data) 459 citations on April 19th, 2021 (source: Scopus)
(56.4109)
$10.4 billion
worth of
research!!!
Tremendous economic impact
Tremendous economic impact
70% of 1576 researchers had been unable to reproduce another’s experiment and over half had even been unable to reproduce theirownBaker, M. (2016) 1500 scientists lift the lid on reproducibility, Nature 533, 452-544
Tremendous economic impact
70% of 1576 researchers had been unable to reproduce another’s experiment and over half had even been unable to reproduce theirownBaker, M. (2016) 1500 scientists lift the lid on reproducibility, Nature 533, 452-544
Free, T. (2020) The ABCs of reproducibility; effectingattitudes, behaviors and change (2020), BioTechniques, 69, 359-361
[5] Knudtson, K. et al. (2019) Survey on scientific shared resourcerigor and reproducibility, J. Biomol. Tech, 30, 36-44.[6] Baker, M. (2016) 1500 scientists lift the lid on reproducibility, Nature 533, 452-544
(…)
Tremendous economic impact
70% of 1576 researchers had been unable to reproduce another’s experiment and over half had even been unable to reproduce theirownBaker, M. (2016) 1500 scientists lift the lid on reproducibility, Nature 533, 452-544
Free, T. (2020) The ABCs of reproducibility; effectingattitudes, behaviors and change (2020), BioTechniques, 69, 359-361
5] Knudtson, K. et al. (2019) Survey on scientific shared resourcerigor and reproducibility, J. Biomol. Tech, 30, 36-44.[6] Baker, M. (2016) 1500 scientists lift the lid on reproducibility, Nature 533, 452-544
359-361
(…)
✓ Improve experimental design/data analysis!
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
✓ Raise the educative and basic question: howmuch can I trust my protein? What should I know?
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
✓ Raise the educative and basic question: howmuch can I trust my protein? What should I know?
✓ Check that we are working with:
the correct protein
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
✓ Raise the educative and basic question: howmuch can I trust my protein? What should I know?
✓ Check that we are working with:
the correct protein
that it is pure/homogeneous
it has right fold/activity
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
✓ Raise the educative and basic question: howmuch can I trust my protein? What should I know?
✓ Check that we are working with…
the correct protein
t
✓ Report the results of each control in publications
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
✓ Improve experimental design/data analysis!
✓ Improve QC and characterization of the‘protein’ samples (biological reagents)
✓ Raise the educative and basic question: howmuch can I trust my protein? What should I know?
✓ Check that we are working with…
the correct protein
that it is pure/homogeneous
it has right fold/activity
✓ Report the results of each control in publications
✓ Encourage more extensive use of supplementary data
Are we wasting 50% of life science and pre-clinical research buget? What can we do?
Protein identity and production parameters➢ protein name and complete sequence, by providing a
NCBI or UniProt accession number, cloning strategy and the source of the DNA (species)
➢ expression vector and host strain, including the tags and cleavage sites used, with the full amino acid sequence of the final protein
➢ Expression and purification protocol, namely the detailed description of all the production steps
Our proposal:
minimal information to provide in publications
Protein identity and production parameters➢ protein name and complete sequence, by providing a
NCBI or UniProt accession number, cloning strategy and the source of the DNA (species)
➢ expression vector and host strain, including the tags and cleavage sites used, with the full amino acid sequence of the final protein
➢ Expression and purification protocol, namely the detailed description of all the production steps
Protein concentration (A = log I0/I = e280·C·l)Specifying the method used for quantification and themolar extinction coefficient at 280 nm, if applicable
Our proposal:
minimal information to provide in publications
Protein identity and production parameters➢ protein name and complete sequence, by providing a
NCBI or UniProt accession number, cloning strategy and the source of the DNA (species)
➢ expression vector and host strain, including the tags and cleavage sites used, with the full amino acid sequence of the final protein
➢ Expression and purification protocol, namely the detailed description of all the production steps
Protein concentration (A = log I0/I = e280·C·l)Specifying the method used for quantification and themolar extinction coefficient at 280 nm, if applicable
Storage conditionsi.e. final buffer composition (pH, buffers, salts and additives), storage temperature and, when applicable, freezing or lyophilization conditions
Our proposal:
minimal information to provide in publications
PurityChecked by SDS-PAGE, Capillary Electrophoresis (CE) orReverse Phase Liquid Chromatography(RPLC)
Our proposal:
minimal quality control tests to be performed
Homogeneity (size distribution)checked preferably by Size Exclusion Chromatography (SEC) and/or Dynamic Light Scattering (DLS), or by SEC in combination with Multi Angle Light Scattering (SEC-MALS), Field Flow Fractionation (FFF) or FFF-MALS, or analytical ultracentrifugation (AUC)
PurityChecked by SDS-PAGE, Capillary Electrophoresis (CE) orReverse Phase Liquid Chromatography(RPLC)
Our proposal:
minimal quality control tests to be performed
Homogeneity (size distribution)checked preferably by Size Exclusion Chromatography (SEC) and/or Dynamic Light Scattering (DLS), or by SEC in combination with Multi Angle Light Scattering (SEC-MALS), Field Flow Fractionation (FFF) or FFF-MALS, or analytical ultracentrifugation (AUC)
Identity and integritychecked preferably by intact protein mass (MS), peptide mass fingerprint (MS), or Edman sequencing (correct sequence? no proteolysis? no undesired modifications? expected PTMs?)
PurityChecked by SDS-PAGE, Capillary Electrophoresis (CE) orReverse Phase Liquid Chromatography(RPLC)
Our proposal:
minimal quality control tests to be performed
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Our proposal:
extended quality control tests
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Conformational stability/folding stateChecked by Circular Dichroism (CD), Fourier Transform Infra Red (FTIR), NMR, Differential Scanning Calorimetry (DSC); “spectral signatures” and “thermal denaturation signatures” (very useful for lot-to-lot consistency)
Our proposal:
extended quality control tests
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Conformational stability/folding stateChecked by Circular Dichroism (CD), Fourier Transform Infra Red (FTIR), NMR, Differential Scanning Calorimetry (DSC); “spectral signatures” and “thermal denaturation signatures” (very useful for lot-to-lot consistency)
Homogeneity (charge distribution, conformational species)Checked by analytical Ion Exchange Chromatography (IEX), analytical Hydrophobic Interaction Chromatography (HIC), Isoelectric Focusing (IEF)
Our proposal:
extended quality control tests
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Conformational stability/folding stateChecked by Circular Dichroism (CD), Fourier Transform Infra Red (FTIR), NMR, Differential Scanning Calorimetry (DSC); “spectral signatures” and “thermal denaturation signatures” (very useful for lot-to-lot consistency)
Homogeneity (charge distribution, conformational species)Checked by analytical Ion Exchange Chromatography (IEX), analytical Hydrophobic Interaction Chromatography (HIC), Isoelectric Focusing (IEF)
Protein competent fraction, i.e. the relative amount of active protein Measured as specific activity, by active-site titration or other suitable methods (to measure biological function)
Our proposal:
extended quality control tests
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Conformational stability/folding stateChecked by Circular Dichroism (CD), Fourier Transform Infra Red (FTIR), NMR, Differential Scanning Calorimetry (DSC); “spectral signatures” and “thermal denaturation signatures” (very useful for lot-to-lot consistency)
Homogeneity (charge distribution, conformational species)Checked by analytical Ion Exchange Chromatography (IEX), analytical Hydrophobic Interaction Chromatography (HIC), Isoelectric Focusing (IEF)
Protein competent fraction, i.e. the relative amount of active protein Measured as specific activity, by active-site titration or other suitable methods (to measure biological function)
Optimization of storage conditions (buffer, pH, etc.)Minimize aggregation, improve solubility and stability (also activity!)
Our proposal:
extended quality control tests
UV absorbance spectrum between 340 nm and 240 nm
to check nucleic acid content, absence of aggregates and measure [P]Mandatory if protein binds nucleic acids
Conformational stability/folding stateChecked by Circular Dichroism (CD), Fourier Transform Infra Red (FTIR), NMR, Differential Scanning Calorimetry (DSC); “spectral signatures” and “thermal denaturation signatures” (very useful for lot-to-lot consistency)
Homogeneity (charge distribution, conformational species)Checked by analytical Ion Exchange Chromatography (IEX), analytical Hydrophobic Interaction Chromatography (HIC), Isoelectric Focusing (IEF)
Protein competent fraction, i.e. the relative amount of active protein Measured as specific activity, by active-site titration or other suitable methods (to measure biological function)
Optimization of storage conditions (buffer, pH, etc.)Minimize aggregation, improve solubility and stability (also activity!)
Batch-to-batch consistency Mandatory if more than one batch is to be usedUse some of the methods listed for minimal/extended QC
Our proposal:
extended quality control tests
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• Mass spectrometry
• Size exclusion chromatography
• Dynamic light scattering
Raynal et al. (2014) Microbial Cell Factories 13, 180
Minimum quality control guideline(recommended tests)
The minimum quality control relies on five(first-line) methods only:
• SDS-PAGE
• UV-Visible spectrophotometry
• MS
• SEC-MALS
• Dynamic light scattering
=>=>=> cost is less than 200 µg of protein
~10 µg (coomassie blue staining)
(~400 µg)
~60 µg (Mr 105, i.e. 0.6 nmole)
~100 µg
(~5 µg)
Minimum quality control guideline(recommended tests)
31 institutions
43 laboratories
188 samplesGives a 90% confidence level in our analysis, with a 6% margin of error
Collection of experiment datafor statistical purposes
6%20%
26% 21%
53%
74%
69%
31%
Survey of protein QC procedures
1/3 samples tested for purity, homogenityand identity/integrity failed at least one ofthem
Correlation between QC results and satisfactory results obtained in downstream applications
Out of these 130 samples, among those that passed the three criteria (purity, more than 90% assessed by SDS-PAGE, aggegation state by DLS or SEC, identity by intact mass or peptide mass fingerprint), 74% yielded satisfactory results, 20% suceeded only partially and 6% failed in downstream applications
In contrast, most (ca. 80%) of the samples that failed one or more of the minimal QC tests didnot give satisfactory results in downstream applications
6%20%
26% 21%
53%
74%
69%
31%
Survey of protein QC procedures
1/3 samples tested for purity, homogenityand identity/integrity failed at least one ofthem
Success in downstream applications (130 samples)
Guidelines with minimal data that needs to be known before using a protein
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
✓ encourage the whole scientific community (researchers, editors and funding agencies alike) to implement this QC methodology
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
✓ encourage the whole scientific community (researchers, editors and funding agencies alike) to implement this QC methodology
✓ raise awareness amongst colleagues and collaborators
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
✓ encourage the whole scientific community (researchers, editors and funding agencies alike) to implement this QC methodology
✓ raise awareness amongst colleagues and collaborators
✓ reverse the trend for less and less information available in mainsteam journals on theproduction and purification of laboratory reagents
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
✓ encourage the whole scientific community (researchers, editors and funding agencies alike) to implement this QC methodology
✓ raise awareness amongst colleagues and collaborators
✓ reverse the trend for less and less information available in mainsteam journals on theproduction and purification of laboratory reagents
✓ lobby editors to encourage/oblige a more extensive use of supplementary data and also the report of the results of each check-point in research papers
Identity HomogeneityPurity Integrity
So what is our proposal?
Guidelines with minimal data that needs to be known before using a protein
✓ our guidelines are the result from the collective expertise of ca. 150 laboratories
✓ lay the groundwork for the standardization of procedures and reproducibility of data in any laboratory that produces recombinant proteins
✓ encourage the whole scientific community (researchers, editors and funding agencies alike) to implement this QC methodology
✓ raise awareness amongst colleagues and collaborators
✓ reverse the trend for less and less information available in mainsteam journals on theproduction and purification of laboratory reagents
✓ lobby editors to encourage/oblige a more extensive use of supplementary data and also the report of the results of each check-point in research papers
✓ improve reproducibility between labs and increase confidence in published data
Identity HomogeneityPurity Integrity
So what is our proposal?
The QC team of ARBRE MOBIEU and P4EU
Nick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)
Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)
Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)
Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)
Blanca Lopez-Mendez - University of Copenhagen, Denmark (ARBRE-MOBIEU)
Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)
André Matagne - University of Liège, Belgium (ARBRE-MOBIEU)
Annabel Parret - EMBL Hamburg, Germany (P4EU)
Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)
Kim Remans - EMBL Heibelberg, Germany (P4EU)
Stephan Uebel - Max-Planck Institute for Biochemistry, Munich, Germany (ARBRE-MOBIEU)
On behalf of…
The QC team of ARBRE MOBIEU and P4EU
Nick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)
Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)
Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)
Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)
Blanca Lopez-Mendez - University of Copenhagen, Denmark (ARBRE-MOBIEU)
Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)
André Matagne - University of Liège, Belgium (ARBRE-MOBIEU)
Annabel Parret - EMBL Hamburg, Germany (P4EU)
Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)
Kim Remans - EMBL Heibelberg, Germany (P4EU)
Stephan Uebel - Max-Planck Institute for Biochemistry, Munich, Germany (ARBRE-MOBIEU)
On behalf of…
list of recommended tests (QC Guidelines)
The QC team of ARBRE MOBIEU and P4EU
Nick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)
Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)
Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)
Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)
Blanca Lopez-Mendez - University of Copenhagen, Denmark (ARBRE-MOBIEU)
Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)
André Matagne - University of Liège, Belgium (ARBRE-MOBIEU)
Annabel Parret - EMBL Hamburg, Germany (P4EU)
Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)
Kim Remans - EMBL Heibelberg, Germany (P4EU)
Stephan Uebel - Max-Planck Institute for Biochemistry, Munich, Germany (ARBRE-MOBIEU)
On behalf of…
list of recommended tests (QC Guidelines)
Titel der Präsentation (Editierbar im Folienmaster), Ort, 20.04.2021 (Editierbar im Folienmaster)Sabine Suppmann, Biochemistry Core Facility 9th P4EU meeting, Nov30
Network of more than 100 members from > 40 protein facilities, mainly but not exclusively in Europe
▪ Started in 2010 (Hüseyin Besir, EMBL)
▪ Share expertise and information
▪ Exchange materials & protocols
▪ New tools & technologies
▪ Establish standards for P related work
▪ Benchmarking and SOPs
▪ Enable collaboration
▪ Access to external facilities
▪ Training of staff/usersfrom Hüseyin Besir, CTLS 2014
(2015)
A large communityhas already joined
More than 170
resource labs/infrastructures/facilities
from 30 different european countries
working in very different contexts ,
some more "research topic oriented"
and others
more "technology oriented"
http://arbre-mobieu.eu/