npj Computational Materials

Multi-reward reinforcement learning based development of inter-atomic potential models for silica

Aditya Koneru, Henry Chan, Sukriti Manna, Troy D. Loeffler, Debdas Dhabal, Andressa A. Bertolazzo, Valeria Molinero & Subramanian K. R. S. Sankaranarayanan

npj Computational Materials 9: 125 (2023)

doi.org/10.1038/s41524-023-01074-9

Published online: 18 July 2023
Abstract| Full Text | PDF OPEN

Abstract: Silica is an abundant and technologically attractive material. Due to the structural complexities of silica polymorphs coupled with subtle differences in Si–O bonding characteristics, the development of accurate models to predict the structure, energetics and properties of silica polymorphs remain challenging. Current models for silica range from computationally efficient Buckingham formalisms (BKS, CHIK, Soules) to reactive (ReaxFF) and more recent machine-learned potentials that are flexible but computationally costly. Here, we introduce an improved formalism and parameterization of BKS model via a multireward reinforcement learning (RL) using an experimental training dataset. Our model concurrently captures the structure, energetics, density, equation of state, and elastic constants of quartz (equilibrium) as well as 20 other metastable silica polymorphs. We also assess its ability in capturing amorphous properties and highlight the limitations of the BKS-type functional forms in simultaneously capturing crystal and amorphous properties. We demonstrate ways to improve model flexibility and introduce a flexible formalism, machine-learned ML-BKS, that outperforms existing empirical models and is on-par with the recently developed 50 to 100 times more expensive Gaussian approximation potential (GAP) in capturing the experimental structure and properties of silica polymorphs and amorphous silica.

摘要: 二氧化硅是一种含量丰富且具有技术吸引力的材料。由于二氧化硅多晶体的结构复杂性以及硅-氧键特性的细微差异，开发准确的模型来预测二氧化硅多晶体的结构、能量和性质仍然具有挑战性。目前的二氧化硅模型范围从计算效率高的Buckingham形式（BKS、CHIK、Soules）到反应力场（ReaxFF），和最近的虽灵活但计算成本高的机器学习势。在这里，我们通过使用一个实验训练数据集，介绍了一种通过多重奖励强化学习（RL），改进BKS模型的形式和参数化。我们的模型同时捕获了石英（平衡态）以及其他20种亚稳态二氧化硅多晶体的结构、能量、密度、状态方程和弹性常数。我们还评估了它捕获非晶态性质的能力，并强调了BKS型函数形式在同时捕获晶体和非晶态性质方面的局限性。我们展示了提高模型灵活性的方法，并引入了一种灵活的形式，既机器学习的ML-BKS，它优于现有的经验模型，在捕捉二氧化硅多晶体和非晶态的实验结构和性质方面与最近开发的计算成本高出50到100倍的高斯逼近势（GAP）相媲美。

Editorial Summary

Revealing the Silica Interatomic Potential Model: The Magic of Reinforcement Learning

Silica is a material widely used in various fields owning to its eco-friendliness and polymorphs. However, due to its structural complexities of polymorphs coupled with subtle differences in Si–O bonding characteristics, it is crucial to develop models that can accurately predict its structure and properties. To address this challenge, this study proposes an approach based on multi-reward reinforcement learning (RL) to enhance the silica atomic potential model. A team led by Valeria Molinero and Subramanian K. R. S. Sankaranarayanan from the Department of Mechanical and Industrial Engineering at the University of Illinois, the Center for Nanoscale Materials at Argonne National Laboratory, and the Department of Chemistry at the University of Utah, USA, introduce a workflow based on multi-reward reinforcement learning, significantly improving the accuracy of predicting the structure and properties of silica polymorphs by reparameterizing the van Beest, Kramer, and van Santen (BKS) model. They have incorporated a hierarchical reward system to eliminate biases related to weight selection during the search process. This enhancement allows the model to simultaneously capture the structure, energetics, density, equation of state, and elastic constants of both equilibrium quartz and 20 other metastable silica polymorphs. The results reveals that the ML-BKS model effectively addresses the trade-off between polymorphs and amorphous properties, enhancing the model's flexibility. The precise predictions of structural features have significantly enhanced the overall performance of the model. Importantly, when it comes to predicting the structure and properties of both polymorphs and amorphous silica, the ML-BKS model exhibits efficiency comparable to the Gaussian approximation potential (GAP) model, but with faster computational speed. This study represents a significant improvement in the silica atomic potential model, providing robust support for the applications of silica in various fields.

二氧化硅原子间势模型揭秘：强化学习的魔力

二氧化硅是一种广泛应用于众多领域的材料，具有生态友好性和多晶现象。然而，由于其复杂的多晶体结构和硅-氧键的细微差异，建立能够准确预测其结构和性质的模型至关重要。为此，该研究开发了一种基于多重奖励强化学习（RL）的方法，改进了二氧化硅原子势模型。来自美国伊利诺伊大学机械与工业工程系、阿贡国家实验室纳米材料中心和犹他大学化学系的Valeria Molinero和 Subramanian K. R. S. Sankaranarayanan研究团队，提出了一种基于多重奖励强化学习的工作流程，通过重新参数化van Beest、Kramer和van Santen（BKS）模型的系数和形式，显著提高了预报二氧化硅多晶体结构和性质的准确性。他们引入了分层奖励系统，消除了搜索过程的权重选择偏差，从而使模型能够同时捕获了平衡态石英和其他20种亚稳态二氧化硅多晶体的结构、能量、密度、状态方程和弹性常数。研究表明，ML-BKS模型有效解决了多晶性质和和非晶性质之间权衡的问题，提高了模型的灵活性。ML-BKS模型对结构特征的精确预测，极大改善了该模型整体性能。值得一提的是，在预报二氧化硅多晶体和非晶态的结构和性质方面，ML-BKS模型与高斯近似势（GAP）模型相当，但计算效率更高。这项研究改进了二氧化硅原子势模型，为二氧化硅的应用提供了有力的支撑。