Screening billions of candidates for solidlithium-ion conductors: A transfer learningapproach for small dataCite as: J. Chem. Phys. 150, 214701 (2019); https://doi.org/10.1063/1.5093220Submitted: 18 February 2019 . Accepted: 10 May 2019 . Published Online: 03 June 2019
Ekin D. Cubuk, Austin D. Sendek , and Evan J. Reed
ARTICLES YOU MAY BE INTERESTED IN
Unsupervised machine learning in atomistic simulations, between predictions andunderstandingThe Journal of Chemical Physics 150, 150901 (2019); https://doi.org/10.1063/1.5091842
Ultrastable and polyamorphic states of vapor-deposited 2-methyltetrahydrofuranThe Journal of Chemical Physics 150, 214502 (2019); https://doi.org/10.1063/1.5091796
A Bayesian approach for identification of ice Ih, ice Ic, high density, and low density liquidwater with a torsional order parameterThe Journal of Chemical Physics 150, 214504 (2019); https://doi.org/10.1063/1.5096556
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
Screening billions of candidates for solidlithium-ion conductors: A transfer learningapproach for small data
Cite as: J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220Submitted: 18 February 2019 • Accepted: 10 May 2019 •Published Online: 3 June 2019
Ekin D. Cubuk,1,2 Austin D. Sendek,2,3 and Evan J. Reed2,a)
AFFILIATIONS1Google Brain, Mountain View, California 94043, USA2Department of Materials Science and Engineering, Stanford University, Stanford, California 94305, USA3Department of Applied Physics, Stanford University, Stanford, California 94305, USA
ABSTRACTMachine learning (ML) methods have the potential to revolutionize materials design, due to their ability to screen materials efficiently. Unlikeother popular applications such as image recognition or language processing, large volumes of data are not available for materials designapplications. Here, we first show that a standard learning approach using generic descriptors does not work for small data, unless it is guidedby insights from physical equations. We then propose a novel method for transferring such physical insights onto more generic descriptors,allowing us to screen billions of unknown compositions for Li-ion conductivity, a scale which was previously unfeasible. This is accomplishedby using the accurate model trained with physical insights to create a large database, on which we train a new ML model using the genericdescriptors. Unlike previous applications of ML, this approach allows us to screen materials which have not necessarily been tested before (i.e.,not on ICSD or Materials Project). Our method can be applied to any materials design application where a small amount of data is available,combined with high details of physical understanding.
Published under license by AIP Publishing. https://doi.org/10.1063/1.5093220
Due to its ability to extract insights from existing experimentaland simulation data, machine learning (ML) is a promising tool forscientists. In materials science, in particular, interest in the applica-tion of ML to materials design is rapidly growing due to the exis-tence of rich materials datasets produced over the last century thatcan inform new design principles and guide screening efforts. Theseapproaches are especially appealing when considering that the spaceof candidate materials is an effectively infinite materials candidatespace to choose from, and it is intractable to screen large spaceswith experimental or first principles methods. One shortcoming ofstate-of-the-art ML methods is that they require large datasets withmore than 104–106 examples. This is prohibitive for materials designapplications, since most datasets for specific material properties havearound 101–103 examples. For example, Saad et al. have trained aML model on 44 materials to predict the melting temperature of sub-octet compounds,1 and Seko et al. trained a model on lattice thermal
conductivity using 101 materials.2 Ghiringhelli et al. trained a modelon 82 materials to predict the energy difference between zinc blendeand rock salt phases, and they emphasized the importance of thedescriptor choice.3 Lee et al. used a dataset of 270 materials to train amodel on bandgaps but used results from density functional theory(DFT) calculations as descriptors to improve their results.4 Simi-larly, Seko et al. found that using DFT calculations on bulk modulusand cohesive energy as descriptors improved their ML model onmelting temperature significantly, where their training set had 248materials.5
ML models have also been applied to predict oxide6 and Li7
conductivity. More recently, a logistic regression model has beentrained on Li-ion conductivity measurements available in the exper-imental literature, totaling 40 materials.8 We call this dataset exp-dataset in the rest of the manuscript. In that work, the descrip-tors were created based on previous physics-based proposals for the
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-1
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
estimation of Li-ion conductivity. Despite the small size of the train-ing set, the physics-based descriptors made it possible to train anML model that generalized. The descriptors used structural infor-mation about the materials: the average number of lithium neigh-bors for each lithium, the average sublattice bond ionicity, the aver-age anion-anion coordination number in the anion framework, theshortest lithium-anion distance, and the average shortest lithium-lithium distance. We will refer to this model as structural ML (sML).This model was found to exhibit an F1 score 3.5x better than ran-dom guessing for identifying Li-ion conductors with a conductivitygreater than 1 × 10−4 S × cm−1, which allowed for the discovery ofmany new promising solid electrolytes by screening approximately12 000 known lithium containing materials in the Materials Project(MP) database.9
While screening known materials for promising properties isextremely useful, a long standing dream of computational materi-als science is the ability to screen all possible materials that 92 stableelements will allow, most of which have never been synthesized andcharacterized. A key roadblock is that physically motivated descrip-tors requiring experimental measurements or first-principles calcu-lations such as crystal structure, bulk modulus, bandgap, and densityhave been reported to be necessary for making accurate predictionsabout material properties in many cases. Therefore, it is frequentlyonly possible to screen materials whose crystal structure and otherproperties are already known, and/or DFT calculations have alreadybeen applied to them to infer the descriptors. This is the reasonthat the sML could only screen the candidates that are available inMaterials Project (MP),9 since the sML model can only be used onmaterials with known structures.
An alternative is to use generic descriptors that do not requireDFT calculations or structural information. One example is ele-mental descriptors, which represent a material only by its chemicalcomposition with functions of the properties of the individual ele-ments that make up the material.10 Elemental descriptors can beused to screen novel material compositions, where the structure isnot known, since they do not require any information other than theelements and stoichiometry. However, elemental models are usuallynot based on any physical models of the target property and aretherefore expected to require significantly larger amounts of train-ing data than available, if they can be trained to produce a predictivemodel at all.
Here, we quantify the importance of using descriptors that areguided by physical laws for training on small data by training anML model only using only elemental descriptors on the same datasetthat the sML was trained on. We call this the elemental ML (eML)model. The elemental descriptors we consider are the atomic num-ber, group, period, electronegativity, electron affinity, boiling tem-perature, melting temperature, density, and ionization energy of theelements. To represent a material described by a particular chemi-cal formula, we use the weighted average of the elemental features aswell as the standard deviation and the maximum value among theelements present in the material. Finally, we also use the L1, L2, andL3 norm of fractions of each element in the composition. These giveus a total of 30 elemental descriptors.
When we use all of these descriptors to train a linear supportvector machine (SVM)11,12 using the leave-one-out cross-validationwith 40 data points (29 in the low conductivity class below 1 × 10−4
S × cm−1 and 11 in the high conductivity class), we reach a training
accuracy of 97.5% and a validation accuracy of 72.5%. This model isseverely overfit since its predictive power on the training set does notgeneralize to that on the validation set; this is expected since there are31 parameters to optimize using 40 samples.
Next, we try to find subsets of the 30 descriptors with better val-idation error. To do this, we train with every subset of the 30 featuresusing leave-one-out cross-validation. Figure 1 shows the training seterror and validation set error for the subset of a certain size withthe best validation error. We also plot the validation error of thesML, whose descriptors are physically motivated and use structuralinformation. We note that the eML reaches a better validation errorthan the sML with 6 or more descriptors. The best structural model(sML) has 5 descriptors with a validation accuracy of 90.0%, whereasthe best elemental (eML) model has 7 descriptors with a validationaccuracy of 97.5%. A naïve interpretation of these results impliesthat eML is better than sML, although we will find that the oppo-site is true upon more careful analysis. We note that the validationaccuracy on such small datasets may not generalize well.13 Figure 1also shows the accuracy expected for random guessing, providinga baseline comparison. Random guessing accuracy is calculated byassuming that we randomly pick 11 of the 40 samples to be in thehigh-conductivity class.
As a first step to more rigorous testing of the predictive powerof the eML, we compare the predictions of the eML and sML on allLi containing materials in the MP. We call the 12 716 such materialsthe MP-dataset. sML predicts 1392 of these materials to be good Li-ion conductors. eML only predicts 176 of these materials to be goodconductors, which gives us a 13% agreement. Given this poor agree-ment, it is not possible for sML and eML to both be more accuratethan 90%. This suggests that the 97% validation accuracy of eML inFig. 1 is overly optimistic.
FIG. 1. On the exp-dataset, the elemental model outperforms the structural model.Best average leave-one-out training and validation errors as a function of numberof descriptors for the eML are shown in blue and red, respectively. The validationerror of the sML is shown in green. Triangles represent the prediction (test) accu-racy on the DFT-dataset. sML gets an F1 score of 0.5 and a precision of 100%,while eML gets an F1 score of 0 and a precision of 0%. This plot indicates thatthe physical features of the sML lead to superior predictive (test) power, despitethe superior leave-one-out validation performance of the eML model with genericfeatures.
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-2
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
To more rigorously determine whether the sML or eML holdsmore predictive power, we develop a test dataset by randomly pick-ing 21 of the 12 716 lithium containing materials in the MP databaseand simulate their Li-ion conductivity using DFT based molecu-lar dynamics (MD) at 900 K. Note that the number of materi-als chosen for these simulations was limited due to computationalconstraints. For more details about these simulations, see Ref. 14.Since these materials are randomly chosen from the distributionof known materials, they constitute the fairest possible test set toevaluate the two ML models. We call this dataset of 21 materi-als and their DFT-determined conductivities DFT-dataset. We findthat the sML predicts the conductivity of the materials in the DFT-dataset with 90.5% accuracy (19 out of 21), whereas eML achievesa 52.4% accuracy (11 out of 21). Further evaluation shows thatthe sML is about 3 times better than random guessing for pick-ing good Li-ion conductors.14 The latter metric is the most rele-vant one for researchers engaged in identifying, synthesizing, andcharacterizing new solid ion conductors with potential for batteryapplications.
Despite the superior validation performance of eML on theexp-dataset, we see that the sML generalizes well to a test set(DFT-dataset), whereas the eML does not. The discrepancy can beexplained by the features used in the two models: the sML’s fea-tures are picked carefully from physically inspired models reportedover several decades in the literature8 and are expected to inter-polate between training data points and perhaps even extrapolateinto unfitted regions, while the eML’s features are generic and notnecessarily related to the physics of Li-ion conductivity. This resultstands in contrast with recent results in deep learning research,where deep models with generic simple descriptors have been per-forming best on tasks from natural language processing (NLP) toimage recognition.15 However, systems in materials science are quitedifferent from the tasks where deep learning with generic descrip-tors has excelled, due to the difference of the volume of trainingdata involved. It is not surprising that a generic model trained ona small dataset cannot generalize well.13 Finding the correct modelthat maps from generic descriptors to a specific property empiri-cally requires more than 104
−106 training set examples, which canbe available in NLP and vision applications but rarely in materialsscience.
The importance of the descriptor selection from physical con-siderations has been observed for a diverse set of materials scienceapplications;1–5,16–26 however, it is not always possible to find the rel-evant physical descriptors for the desired application. Furthermore,even if physical descriptors have been identified, they are not alwayseasily accessible. As an example, consider the ML models that usethe results of DFT calculations as their descriptor input.4,5 TheseML models require that the structure of the candidate material isknown, and that a DFT calculation of that structure is available.These constraints make it impossible to screen a large set of novelcandidate materials, since DFT calculations are computationallyexpensive. In addition, having to know the structure of the materialforces one to screen already synthesized and characterized materialswhose structure has been reported in the literature [e.g., and con-tained in MP or the Inorganic Crystal Structure Database (ICSD)27],which precludes the original goal of trying to find a novel, previouslyunknown material. This constraint applies to the sML as well, since
structural parameters are used as descriptors. For this reason, ithas been used to screen Li containing materials whose structureis already determined via DFT or experiments, which limited thenumber of screened candidates to 12 716.
To overcome this trade-off between ML model accuracy andpotential for screening previously uncharacterized materials, wepropose a transfer learning28 approach. We first train an accurateML model using physically inspired descriptors. For the currentapplication of designing fast Li-ion conductors, this accurate modelis the sML, trained on the exp-dataset. Then, we use the sML to makepredictions on the MP-dataset. Next, we use the predictions on the12 716 materials as labels to train a new model using generic, ele-mental descriptors. Since this dataset is much larger, we find thatit is possible to train a good generic model. We call this elementalmodel trained on the outputs of the structural model the elemental-structural ML model (esML). The esML’s descriptors require onlycomposition information, so it can be used to screen novel materialswithout structural or other experimental or DFT based information.Thus, once an accurate esML is trained, it can be used to screen allpossible combinations of elements efficiently.
Our esML was trained on the same descriptors as eML detailedabove, including the brute force feature selection procedure. Wefound that a small subset of elemental descriptors, including thestandard deviation of the elemental electron affinity, boiling tem-perature, and melting temperature as well as the largest period num-ber and the largest minimum ionization energy of the elements,maximized the predictive capability. Using these 5 descriptors, wewere able to develop an esML that reproduces model predictionsof the sML with 93% 10-fold cross-validation accuracy as well as92% hold-out test set accuracy. This result supports our view thattraining ML models using generic descriptors require larger trainingsets.
To further test the validity of esML, we calculate its accuracyon the exp-dataset. The esML was not directly trained on the exp-dataset, but on the MP-dataset which has a different distribution ofmaterial compositions. Despite this difference in the datasets, wefind that the esML achieves an 87.5% accuracy on predicting theexperimentally measured conductivities, approaching the 90% val-idation accuracy for the sML for this dataset which is as good as onecould expect. Next, we apply the esML on the DFT-dataset. We findthat esML predicts the conductivity of these materials with 86.4%accuracy, approaching the 90.9% accuracy of the sML on this datasetand significantly better than the 54.5% accuracy of the eML. Over-all, these tests show that esML is almost as accurate as sML, whilerequiring only compositional information (Table I).
Next, we screen all ternary and quaternary material composi-tions for Li-ion conductivity with the esML. Because the model isextremely fast to evaluate, much faster than DFT calculations, weare able to screen 20 × 109 compounds, with 1% increments in com-position for each element. Without screening for any other criteria,we find that 60% of all compositions are predicted to be good Li-ionconductors by the esML. This is likely to be too large given that onlyapproximately 10% of materials in the MP are predicted to be goodLi-ion conductors by both the sML and esML. Furthermore, approx-imately 9% of the randomly selected materials in MP were foundto be good conductors by DFT MD. This discrepancy may resultin part from the fact that most of the 20 × 109 candidates are not
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-3
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
TABLE I. The accuracy achieved by sML, eML, and esML on the three differentdatasets. The exp-dataset contains experimental measurements, the MP-dataset arepredictions of the sML model for Li-containing MP structures, and the DFT-datasetare calculations of ionic conductivity for 21 randomly chosen materials employed asa test set here.
thermodynamically stable compositions and the esML is not trainedto give good predictions on unstable compositions since it was onlytrained on stable compositions in the MP.
To identify materials that are more likely to be chemically sta-ble, we seek to filter the 20 × 109 screened materials by enforcingthat the weighted sums of the common oxidation states of the ele-ments add up to zero (see Table II for the allowed oxidation states).We then find that only about 10% of the compositions are stable.Among these compositions that are likely to be stable, approximately7% are predicted to be good Li-ion conductors by the esML, withconductivity greater than 1 × 10−4 S × cm−1.
Next, we screen for cost and weight requirements for batteryapplications. For cost, we follow the US Department of Energygoal of $10/m229 or less, assuming a 10 thickness. We approximatethe cost using the per-mass costs of the raw elements as listed inWikipedia.8 Because transportation and other critical applicationsrequire batteries with high energy per unit volume or mass, werestrict our screening to the first four rows of the periodic table. Toincrease the chance that the Li-ion conductor is easy to synthesize,is stable, electronically insulating, and has a large electrochemicalwindow,14,30 we focus our screening to oxides. Oxides are tradition-ally thought to have lower Li-ion conductivity on average but largerwindows of electrochemical stability. For this reason, we use theesML to find outlier oxides with good Li ion conductivity. Finally,we verify that candidate compositions are more likely to be thermo-dynamically stable against metallic lithium so they may be used insolid-state batteries with lithium metal anodes, by excluding compo-sitions that include transition metals. Material stoichiometries with
TABLE II. Allowed oxidation states during screening.
low index ratios that satisfy all the screening criteria are listed inTable III.
As mentioned above, most oxides whose Li conductivity hasbeen measured exhibit poor conductivity (see Table I in Ref. 8),and oxides have not been proposed as good conductors in pre-vious material screening applications of ML (Table III in Ref. 8).The Li conductivity of LiPO3 has been studied previously, wherefast Li-ion migration was observed along preferential pathways inthe glassy matrix.31 Stable phases of MgO2 under high pressurehave been discovered recently,32 but it is not clear if they can belithiated.
So far, we have used elemental descriptors as the genericdescriptors that are application agnostic to construct the esML.Next, we take this idea a step further and get rid of all descrip-tors. Instead of using aspects of elements that we think might beimportant (such as electron affinity, ionization potential, and melt-ing point) followed by descriptor down-selection, we attempt tolearn a predetermined number of elemental descriptors from datathat produce the best model. Since the elemental descriptors arechosen arbitrarily, they may not be the best choice for elucidatingthe relationship between composition and the material property ofinterest. Instead, we learn a useful featurization of elements, specif-ically for the material property of interest, by training the elementaldescriptor vectors as well as the rest of the neural network simulta-neously. Thus, we train the network as well as the atomic features onthe prediction of ionic conductivity. Similar to the word embeddingapproach in NLP called word2vec,33 we embed atoms in an arbitrarydescriptor space, which we call atom2vec. In this formalism, eachmaterial is represented by the atomic vectors of the elements thatmake up the material. Material stoichiometry is given by a reduc-tion applied to the atomic vectors such that the material representa-tion has the same number of dimensions as each atomic vector (seeFig. 2 for a schematic of the neural network architecture). In thiswork, we have tried combining the following reduction methods:weighted average, standard deviation, and maximum/minimum ofeach dimension. We found that for this application, weighted aver-age by itself performs as well as the combination of all reductionmethods. We used just the weighted average for results below, forsimplicity.
Before the training process begins, the number of atomic vec-tors or features to be determined is chosen. Each atomic vector, ofdimensionality Datoms, is randomly initialized. Then, we apply super-vised ML using a neural network, where the optimal atomic vectorsthat describe each element is learned from data by stochastic gradi-ent descent, just as the other parameters of the network (the weights
TABLE III. Materials that satisfy all of the screening criteria: stability, high Liconductivity, low cost and weight, and a large window of electrochemical stability.
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
FIG. 2. Schematic for the conductivity classification of a single material using atom2vec-esML. The material is presented to the algorithm as a vector of compositions. Forexample, LiPO3 would be represented by 0.2 on the third row, 0.2 on the fifteenth row, and 0.6 on the eighth row. Thus, Natoms is the number of elements in the periodic table,but we use 94 as we only consider the elements with atomic number up to 94. This composition vector is replicated along the column dimension and multiplied elementwisewith the embedding matrix. Each row of the embedding matrix is the atomic vector for the corresponding element. This elementwise multiplication gives us the embeddedmaterial, where most rows are populated with zeros since most materials have only a few elements. For example, the embedded material of LiPO3 would only have nonzeroelements on the third, fifteenth, and eighth rows. Then, we apply the reduction methods as described in the text and concatenate the results. For example, if the reductionmethod is averaging, then we take the average of each column of the embedded material matrix. The concatenated vector has the length NR × Datoms (see Fig. 3 for theselection of Datoms), where NR is the number of reduction methods employed. At this stage, the material is embedded into a fixed dimension (regardless of the number ofelements present in the material), and we apply a standard 2-layer fully-connected neural network to classify it as a good or a bad conductor (with Nh nodes in the hiddenlayer). We found that the results are not too sensitive to the number of hidden nodes and used 20 in this paper. The final layer employs a standard softmax layer withcross-entropy loss.
and the biases). This way, instead of arbitrarily choosing elemen-tal properties such as electron affinity or ionization energy, we letthe data decide which aspects of an element are important for thetask at hand. This procedure also potentially allows for the reuse ofthe learned descriptors for a different task. For example, the atomicvectors can be learned using a material property which has plentyof data available and then used for machine learning another mate-rial property which has limited data. This application has not beenexplored further in this work. We note that a similar approach hasrecently used unsupervised learning to construct atomic vectors,34
whereas we used supervised learning to optimize the atomic vectorsfor the application of interest.
In Fig. 3, we show the training and test set accuracy of the esMLtrained on learned atomic vectors, which we call atom2vec-esML, asa function of Datoms. We show that it is possible to learn the output ofsML with high accuracy, without any manual selection of elementaldescriptors. Interestingly, the validation accuracy of atom2vec-esMLreaches a plateau after 3 features, which implies that the atom2vecrepresentation is more compact than the elemental descriptor set.This is not surprising since the elemental descriptors are constrainedto be correlated with each other, whereas atom2vec representation isnot. For this reason, we choose Datoms to be 4 for the results pre-sented in this paper. Finally, as a sanity check, we confirm that allcompositions listed in Table III are predicted to be good conductorsby atom2vec-esML.
To reveal the machine-learned periodic table of Li ion con-ductivity that is implied by these learned features, we look at themagnitude of the dot product between each atomic vector and thedirection perpendicular to the classification hyperplane of theatom2vec-esML. This dot product tells us how much each atomicvector is contributing to the material’s predicted conductivity. We
calculate this dot product 1000 sets of learned features obtainedfrom different sets of random initial conditions for better statis-tics. In Fig. 4, we show the distribution of these dot products forselected atoms. Positive values correspond to higher contributionto conductivity. We see that Li has the highest contribution over-all. Anions with the largest contribution are S and P atoms, whereasN, O, and F atoms have the smallest contribution to conductivity.This agrees with the conventional wisdom that oxides and fluoridesare empirically found to be the slowest Li ion conductors.
FIG. 3. The accuracy for predicting the output of the sML model without using anystructural or physically inspired descriptors, as a function of the dimensionality ofthe embedding space of atoms. Training and validation accuracies are shown inblue and red, respectively.
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-5
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
FIG. 4. Contribution of each atomic vec-tor to Li-ion conductivity. We calculatethe dot product between the atomicvectors and the conductivity hyperplane1000 times using random initial condi-tions and show the distribution. Positivevalues indicate atoms that are expectedto increase Li-ion conductivity whenincluded in a crystal, while negative val-ues indicate those atoms expected todecrease Li-ion conductivity.
We have presented two new methods for transfer machinelearning in materials science. These methods allow us to train ontasks with available datasets in order to make predictions about othertasks for which data may not be available. Using these methods, wehave screened 20 × 109 materials for the first time and proposed ahandful of promising candidates for Li ion conductors in solid elec-trolytes. These transfer learning methods have great potential to beapplied to other materials applications of interest, where physicallymotivated descriptors are important but data are not abundant withthose descriptors.
The authors acknowledge funding from the TomKat Center forSustainable Energy at Stanford University and the Toyota ResearchInstitute Accelerated Materials Design and Discovery program.
REFERENCES1Y. Saad, D. Gao, T. Ngo, S. Bobbitt, J. R. Chelikowsky, and W. Andreoni, “Datamining for materials: Computational experiments with a b compounds,” Phys.Rev. B 85, 104104 (2012).2A. Seko, A. Togo, H. Hayashi, K. Tsuda, L. Chaput, and I. Tanaka, “Predictionof low-thermal-conductivity compounds with first-principles anharmonic lattice-dynamics calculations and bayesian optimization,” Phys. Rev. Lett. 115, 205901(2015).3L. M. Ghiringhelli, J. Vybiral, S. V. Levchenko, C. Draxl, and M. Scheffler, “Bigdata of materials science: Critical role of the descriptor,” Phys. Rev. Lett. 114,105503 (2015).4J. Lee, A. Seko, K. Shitara, K. Nakayama, and I. Tanaka, “Prediction model ofband gap for inorganic compounds by combination of density functional the-ory calculations and machine learning techniques,” Phys. Rev. B 93, 115104(2016).5A. Seko, T. Maekawa, K. Tsuda, and I. Tanaka, “Machine learning with system-atic density-functional theory calculations: Application to melting temperaturesof single-and binary-component solids,” Phys. Rev. B 89, 054303 (2014).6L. Xu, L. Wencong, P. Chunrong, S. Qiang, and G. Jin, “Two semi-empiricalapproaches for the prediction of oxide ionic conductivities in abo 3 perovskites,”Comput. Mater. Sci. 46, 860–868 (2009).7K. Fujimura, A. Seko, Y. Koyama, A. Kuwabara, I. Kishida, K. Shitara,C. A. Fisher, H. Moriwake, and I. Tanaka, “Accelerated materials design of lithiumsuperionic conductors based on first-principles calculations and machine learningalgorithms,” Adv. Energy Mater. 3, 980–985 (2013).8A. D. Sendek, Q. Yang, E. D. Cubuk, K.-A. N. Duerloo, Y. Cui, and E. J. Reed,“Holistic computational structure screening of more than 12 000 candidates forsolid lithium-ion conductor materials,” Energy Environ. Sci. 10, 306–320 (2017).
9A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia,D. Gunter, D. Skinner, G. Ceder, and K. A. Persson, “The materials Project:A materials genome approach to accelerating materials innovation,” APL Mater.1, 011002 (2013).10L. Ward, A. Agrawal, A. Choudhary, and C. Wolverton, “A general-purposemachine learning framework for predicting properties of inorganic materials,”npj Comput. Mater. 2, 16028 (2016).11C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn. 20, 273–297(1995).12C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,”ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011).13L. G. Valiant, “A theory of the learnable,” Commun. ACM 27, 1134–1142(1984).14A. D. Sendek, E. D. Cubuk, E. R. Antoniuk, G. Cheon, Y. Cui, and E. J. Reed,“Machine learning-assisted discovery of solid Li-ion conducting materials,”Chem. Mater. 31, 342–352 (2019).15Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444(2015).16J. Behler and M. Parrinello, “Generalized neural-network representationof high-dimensional potential-energy surfaces,” Phys. Rev. Lett. 98, 146401(2007).17Y. He, E. D. Cubuk, M. D. Allendorf, and E. J. Reed, “Metallic metal–organicframeworks predicted by the combination of machine learning methods andab initio calculations,” J. Phys. Chem. Lett. 9, 4562–4569 (2018).18P. Z. Hanakata, E. D. Cubuk, D. K. Campbell, and H. S. Park, “Acceleratedsearch and design of stretchable Graphene Kirigami using machine learning,”Phys. Rev. Lett. 121, 255304 (2018).19E. D. Cubuk, B. D. Malone, B. Onat, A. Waterland, and E. Kaxiras, “Represen-tations in neural network based empirical potentials,” J. Chem. Phys. 147, 024104(2017).20B. Onat, E. D. Cubuk, B. D. Malone, and E. Kaxiras, “Implanted neural networkpotentials: Application to Li-Si alloys,” Phys. Rev. B 97, 094106 (2018).21L. Bassman, P. Rajak, R. K. Kalia, A. Nakano, F. Sha, J. Sun, D. J. Singh, M. Aykol,P. Huck, K. Persson, and P. Vashishta, “Active learning for accelerated design oflayered materials,” npj Comput. Mater. 4, 74 (2018).22J. Hoffmann, Y. Bar-Sinai, L. Lee, J. Andrejevic, S. Mishra, S. M. Rubinstein, andC. H. Rycroft, “Machine learning in a data-limited regime: Augmenting exper-iments with synthetic data uncovers order in crumpled sheets,” Sci. Adv. 5(4),eaau6792 (2019).23G. Cheon, E. D. Cubuk, E. R. Antoniuk, L. Blumberg, J. E. Goldberger, andE. J. Reed, “Revealing the spectrum of unknown layered materials with superhu-man predictive abilities,” J. Phys. Chem. Lett. 9, 6967–6972 (2018).24L. Ward, R. Liu, A. Krishna, V. I. Hegde, A. Agrawal, A. Choudhary, andC. Wolverton, “Including crystal structure attributes in machine learning mod-els of formation energies via Voronoi tessellations,” Phys. Rev. B 96, 024104(2017).
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-6
The Journalof Chemical Physics ARTICLE scitation.org/journal/jcp
25F. Ren, L. Ward, T. Williams, K. J. Laws, C. Wolverton, J. Hattrick-Simpers, andA. Mehta, “Accelerated discovery of metallic glasses through iteration of machinelearning and high-throughput experiments,” Sci. Adv. 4, eaaq1566 (2018).26N. Artrith, B. Hiller, and J. Behler, “Neural network potentials for metals andoxides–First applications to copper clusters at zinc oxide,” Phys. Status Solidi B250, 1191–1203 (2013).27A. Belsky, M. Hellenbrandt, V. L. Karen, and P. Luksch, “New developmentsin the inorganic crystal structure database (ICSD): Accessibility in support ofmaterials research and design,” Acta Crystallogr., Sect. B: Struct. Sci. 58, 364–369(2002).28S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. DataEng. 22, 1345–1359 (2010).29B. D. McCloskey, “Attainable gravimetric and volumetric energy density ofLi–S and Li ion battery cells with solid separator-protected Li metal anodes,”J. Phys. Chem. Lett. 6, 4581–4588 (2015).
30J. Xie, A. D. Sendek, E. D. Cubuk, X. Zhang, Z. Lu, Y. Gong, T. Wu, F. Shi,W. Liu, E. J. Reed, and Y. Cui, “Atomic layer deposition of stable LiAlF4 lithiumion conductive interfacial layer for stable cathode cycling,” ACS Nano 11, 7019–7027 (2017).31M. Vogel, “Complex lithium ion dynamics in simulated LiPo3 glass studied bymeans of multitime correlation functions,” Phys. Rev. B 68, 184301 (2003).32Q. Zhu, A. R. Oganov, and A. O. Lyakhov, “Novel stable compounds in themg–o system under high pressure,” Phys. Chem. Chem. Phys. 15, 7696–7700(2013).33T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributedrepresentations of words and phrases and their compositionality,” in NIPS’13 Pro-ceedings of the 26th International Conference on Neural Information ProcessingSystems, Vol. 2, pp. 3111–3119 (2013).34Q. Zhou, P. Tang, S. Liu, J. Pan, Q. Yan, and S.-C. Zhang, “Learning atoms formaterials discovery,” Proc. Natl. Acad. Sci. U. S. A. 115, 201801181 (2018).
J. Chem. Phys. 150, 214701 (2019); doi: 10.1063/1.5093220 150, 214701-7
