Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles

Aik Rui Tan, Shingo Urata, Samuel Goldman, Johannes C. B. Dietschreit & Rafael Gómez-Bombarelli 

npj Computational Materials 9: 225 (2023)

Editorial Summary

Neural network potentials: Who has the best performance of uncertainty quantification

Over the last decade, neural networks (NN) have increasingly been deployed to study complex materials systems. NN interatomic potentials (NNIPs) have been widely used for applications in reactive processes, protein design, solids, solid-liquid interfaces, coarse-graining, and more. Nevertheless, NNIPs remain susceptible to making poor predictions in extrapolative regimes. To maximize NNIP robustness and avoid distribution shift, the training data should contain representative samples from the same ensemble that the simulation will visit. However, since high-quality ab initio calculations are computationally expensive, quantifying model uncertainty and active learning are the key to training robust NNIPs. In this work, Aik Rui Tan et al. from the department of Materials Science and Engineering, Massachusetts Institute of Technology, examined multiple uncertainty quantification (UQ) schemes for improving the robustness of NNIPs through active learning. In particular, the authors used mean-variance estimation (MVE), deep evidential regression, and Gaussian mixture models (GMM) to evaluate their performance in ranking uncertainties using multiple metrics across three different data sets. In general, ensemble-based methods consistently perform well in terms of uncertainty rankings outside of the training data domain and provide the most robust NNIPs. MVE has been shown to perform well mostly in identifying data points within the training domain that corresponds to high errors. Deep evidential regression, offers less accurate epistemic uncertainty prediction, while GMM is more accurate and lightweight than deep evidential regression and MVE. The lack of a one-size-fits-all solution, the high cost of the more robust ensemble-based method, and the disappointing performance of elsewhere-promising evidential approaches confirm that UQ in NNIPs is an ongoing challenge for method development in AI for science.