Olexandr Isayev, Ph.D.University of North Carolina at Chapel Hill
http://olexandrisayev.com
Mastering Computational Chemistry with
Deep Learning
@olexandr
ANI-1: An extensible DL potential with DFT accuracy at force field computational cost
Chem. Sci., 2017, 8, 3192-3203
DOI: 10.1039/C6SC05720A
(http://arxiv.org/abs/1610.08935)
Joint work with Justin S. Smith and Adrian Roitberg
University of Florida
POSTER & Fast Forward Talk:ANI-1: Solving quantum mechanics
with deep learning on GPUs
By Justin Smith
ANAKIN-MEAccurate NeurAl networK engINe for Molecular Energies
+ =
We want to train a padawan network to become a DFT jedi master
Why ANI-1 ???
AniThe force is strong!
Quantum Mechanics 101
Time-independent Schrödinger equation
F(r) = E E
Acc
ura
cy
Force fields
Semi-empirical QM
DFT & HF CCSD(T)
1 103 105 107 109
Time
Accessible molecular systems
Acc
ura
cy
Force fields
Semi-empirical QM
DFT & HF CCSD(T)
ANI-1 Potential
1 103 105 107 109
Time
Accessible molecular systems
Rel. error in total energy of ~6 x 10-4 % vs. DFT Accuracy ~1 kcal/molSpeedup of 105-106
Molecular Mechanics / Force Fields
Protein - Ligand Docking
MMFF94
PM7
Kanal, Hutchison, Keith Submitted Slide credit: G. Hutchison, University of Pittsburg
Molecular Conformers
Design Principles
Create a “Force Field” in the sense of a mapping from coordinates R Energy
(Forces) with no a-priori functional form
• Accurate and reproducible
• Fast
• Input consisting only of things that the Schrödinger equation needs. (i.e. atomic
numbers and positions, plus charge and spin)
• Forces as true gradients of the energy
• Extensible in atomic elements
• Extensible to molecules of very different sizes
• Self-learning
How does ANI-1 work?
Molecular representation (MR)• Transformation from coordinates to a deep learning friendly input
vector
• Accomplished through heavy modifications of Behler and Parrinello symmetry functions[1] or atomic environment vector (AEV or Ԧ𝐺𝑖
𝑋)
• Ԧ𝐺𝑖𝑋 provides atoms local chemical environment to a cutoff radius
• Mods provide recognizable features in MR
• Mods provide better atomic number differentiation
𝑞1Ԧ𝑞
NNP (O)
NNP (H)
𝐸1𝑂 𝐸1
𝐻 𝐸2𝐻
𝑞2 𝑞3
Atomic
Energies
𝐸𝑇Total
Energy
Ԧ𝐺2𝐻Ԧ𝐺1
𝐻Ԧ𝐺1𝑂
+ +
Each color
represents a
distinct NNP
1) J. Behler and M. Parrinello, Phys. Rev. Lett., 2007, 98, 146401.
High-dimensional neural network potential (HDNNP)[1]
• Utilizes AEVs by computing one for each atom
• Total energy takes on a sum of atomic contributions
• Allows training to datasets with many molecules of different size (diverse)
• One NNP per atomic number
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
H2O
Molecular Representation
R = 5 A
What do you need?
• ANI requires TONS of data
• Currently we run ~20M DFT data points. To be released soon
• Molecules with 1 to 8 atoms from GDB database
• Train network on the data
• Validate on separate data
• Test on ‘known sizes’ (Molecules with <= # max heavy atoms per molecule in training set)
• Interpolation
• Test on ‘unknown sizes’ (Molecules larger than any in the training set)
• Extrapolation
• Best network architecture: 768 – 128 – 128 – 64 – 1 (122,944 weights + 321 biases)
• AEV cutoff – Radial SFs: 4.6Å; Angular SFs: 3.1Å
• AEV setup – 32 radial functions; 8x8 angular functions (768 elements)
• Included atomic numbers: H, C, N, O, S, F
• Trained and tested on in-house C++/CUDA program (NeuroChem)
• Trained on batches of 1024 molecules from ANI-1 dataset
• Approximate training time: ~2000 epochs or ~48 hours
• Early stopping with learning rate annealing
• % of ANI-1 dataset utilization: Training: 80% Validation: 10% Test 10%
• Final fitness (RMSE) – Training set: 1.299 kcal/mol
Validation set: 1.348 kcal/mol
Test set: 1.359 kcal/mol
Training the ANI-1 potential
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
• Determine agreement of ANI-1 total potential energy to DFT (ωB97x/6-31g(d))
• 131 Randomly selected molecules with 10 heavy atoms
• Generated ~62 conformations for each of them
• Total of ~8200 structures/energies (300 kcal/mol energy range for each molecule)
ANI-1 test case 1
Total energy correlationANI-1 vs. DFT
(131 molecules with 10 heavy atoms, 8200 total molecules + conformations) [units: kcal/mol]
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
73 total structures10 Heavy atoms25 Total atomsRMSE: 1.2 kcal/mol (0.048 kcal/mol/atom)DFT time: 1143.11sANI time: 0.0032s
357000x speedup!
Relative Energy correlation (30kcal/mol)
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
• ANI-1 potential’s smoothness and goodness of fit to DFT potential surface scans
• Molecules considered are relatively large molecules
(53, 31, and 44 atoms)
• 4 scans included: (bond stretch, angle bend, and two dihedral scans)
ANI-1 test case 2
ANI-1 potential unrelaxed scans
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
ANI-1 potential unrelaxed scans
J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203
Simulating a box of water on ANI-1.1(Chads Hopkins) From 50ps MD run @ 300K
ANI-1.1 theoretical OH vibrational spectra
Self-diffusion coefficient
Exp. IR Absorbance
Method x10^-05 cm^2/s
Experiment 2.5
ANI-1.1 3.2
TIP3P 5.9
TIP4P 3.3
Diels- Alder Reaction
C
DB
A
The Big PictureAn automated and self consistent data generation framework
ANI network agent
IRC Pool GDB Pool
CVMD/MC Sampler
Online database Pool
CV Structure Sampler
Structure Pools
CV Conformer Search
Determine bad structures
Compute normal mode coordinates
Carry out restrained NMS
Compute Cluster
Database of molecular properties
(i.e. energies)
Retrain networks
Computations with QM
• Universal NN potential for small organic molecules
• Accuracy of high quality DFT calculations
• Extremely fast evaluation: <0.001 s/molecule on 1 GPU
• Up 106 speedup in comparison to DFT
• Can do molecular dynamics, reactions and break bonds!
• Stay tuned!
Summary