Predicting Materials Properties with Little Data Using Shotgun Transfer Learning

Hironao Yamada; Chang Liu; Stephen Wu; Yukinori Koyama; Shenghong Ju; Junichiro Shiomi; Junko Morikawa; Ryo Yoshida

doi:10.1021/acscentsci.9b00804

Predicting Materials Properties with Little Data Using Shotgun Transfer Learning

ACS Cent Sci. 2019 Oct 23;5(10):1717-1730. doi: 10.1021/acscentsci.9b00804. Epub 2019 Sep 30.

Authors

Hironao Yamada¹, Chang Liu^{1

2}, Stephen Wu^{1

3}, Yukinori Koyama², Shenghong Ju⁴, Junichiro Shiomi^{2

4}, Junko Morikawa^{2

5}, Ryo Yoshida^{1

2

3}

Affiliations

¹ The Institute of Statistical Mathematics, Research Organization of Information and Systems, Tachikawa, Tokyo 190-8562, Japan.
² National Institute for Materials Science, Tsukuba, Ibaraki 305-0047, Japan.
³ The Graduate University for Advanced Studies, Tachikawa, Tokyo 190-8562, Japan.
⁴ The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan.
⁵ Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8550, Japan.

Abstract

There is a growing demand for the use of machine learning (ML) to derive fast-to-evaluate surrogate models of materials properties. In recent years, a broad array of materials property databases have emerged as part of a digital transformation of materials science. However, recent technological advances in ML are not fully exploited because of the insufficient volume and diversity of materials data. An ML framework called "transfer learning" has considerable potential to overcome the problem of limited amounts of materials data. Transfer learning relies on the concept that various property types, such as physical, chemical, electronic, thermodynamic, and mechanical properties, are physically interrelated. For a given target property to be predicted from a limited supply of training data, models of related proxy properties are pretrained using sufficient data; these models capture common features relevant to the target task. Repurposing of such machine-acquired features on the target task yields outstanding prediction performance even with exceedingly small data sets, as if highly experienced human experts can make rational inferences even for considerably less experienced tasks. In this study, to facilitate widespread use of transfer learning, we develop a pretrained model library called XenonPy.MDL. In this first release, the library comprises more than 140 000 pretrained models for various properties of small molecules, polymers, and inorganic crystalline materials. Along with these pretrained models, we describe some outstanding successes of transfer learning in different scenarios such as building models with only dozens of materials data, increasing the ability of extrapolative prediction through a strategic model transfer, and so on. Remarkably, transfer learning has autonomously identified rather nontrivial transferability across different properties transcending the different disciplines of materials science; for example, our analysis has revealed underlying bridges between small molecules and polymers and between organic and inorganic chemistry.