Despite high hopes that artificial intelligence will transform materials discovery, Google DeepMind researcher Ekin Dogus Cubuk said the technology may be better at optimizing known materials.
Non-AI approaches to materials discovery, such as combinatorial materials science, excel at making predictions from known data — but venture outside of that data and accuracy drops off. AI may run into those same challenges, Cubuk said.
“In machine learning…the farther you get from the training set distribution, the worse your predictions are,” he said, speaking on Catalyst with Shayle Kann. That’s one reason AI models haven’t predicted any commercially deployable materials yet, he said.
DeepMind is one of several players trying to discover new materials that could be used in things like better battery chemistries, powerful carbon-capture sorbents, and room-temperature superconductors. And researchers have had some success so far; Microsoft announced in January that it discovered a new solid state electrolyte candidate with promise for better batteries, after just two weeks of using AI to comb through options.
But paradigm-shifting breakthroughs that have real commercial potential may still depend heavily on human extrapolation and luck. For example, game-changing materials like silicon transistors and tungsten filaments came from random experimentation and educated guesswork.
“Humans have found ways of discovering things that were beyond their theories,” Cubuk said; and some, like the discovery of superconductors, have resulted in “paradigm shifts,” in how entire industries operate.
“AI hasn’t really done this yet,” he added. “Even today’s best AI models seem to be really good at kind of doing the textbook stuff…but then when you think about being more creative and trying to shift the paradigm, it’s been more difficult.”
Discoveries aside, he’s much more optimistic about the ability of AI to speed up the optimization of materials. He offered the example of the discovery of high-temperature superconductors. Part of that discovery involved replacing the lanthanum in lanthanum-barium-copper-oxide (LBCO) superconductors with the similar element yttrium, making it more useful. He believes AI could help speed up that sort of refinement process.
“So I think what computational machine learning can give us here is — even if they can’t do the paradigm shift and go from cuprate to a completely different superconductor — they can at least help us do this optimization, the exploitation part: to go from LBCO to YBCO faster,” Cubuk added.
The data shortage
One hurdle that materials science researchers like Cubuk are grappling with, though, is the fact that AI models need data — and lots of it.
AI researchers train large language models (LLMs) like ChatGPT on billions of data points from the internet. But material scientists often have a tiny fraction of that to work with. The data on the physical properties of a given material type may involve just a few thousand data points, said Cubuk.
“We don’t even have a good, large data set where you can train or validate your synthesis predictions on,” he added.
To generate more data — on things like electrical conductivity and formation energy of various materials — scientists must run experiments, which require time and investment. One workaround is to use computer models to simulate hundreds of thousands or millions of theoretical materials.
“We can simulate data,” Cubuk explained. “And the simulations come from our physical approximations of quantum mechanics, so they tend to be somewhat informative.”
Still, training models on that simulated data may not be as accurate or useful as training on experimental data. So the question, Cubuk said, is “how many experimental data points [are] worth how many computational data points?”
It’s one reason why there are new efforts to generate experimental data, like the Toyota Research Institute’s Synthesis Advanced Research Challenge, focused on advancing solid-state theory.
“They’re trying to create a lot of experimental synthesis data so that you can kind of like bootstrap and start using machine learning and computation,” Cubuk said.
The good news, though, is that LLMs may offer lessons on how to use this combined approach of using both simulated data and experimental data to train models. The internet data that LLMs are trained on is “not very high quality,” involving sentences without good labels or organization. But pre-training using that messy data can be “quite effective,” if you then fine-tune the model with specific tasks.
Cubuk suggested mirroring that approach of pre-training models on simulated data, and then fine-tuning them on experimental data from labs.
“If we end up having these hundreds of millions of points from computation and then a few points — like 100,000 points — from experiments, maybe then we can get some good results,” he said. That’s one reason Cubuk doubts models will ever fully replace lab work. He sees AI-models as complementing the hands-on work done by scientists.
That’s the approach that the team at DeepMind took with its AI-based tool GNoME, or Graph Networks for Materials Exploration. It predicted 2.2 million new, theoretical materials, including 380,000 that were stable at zero degrees Kelvin, making them promising candidates for trying to synthesize in the lab.
“Simulations and machine learning [have] been making progress, not yet in putting materials in devices [and] products, but at least in making useful predictions,” Cubuk said.
Editor’s note: This story was updated on September 23 to make one correction and one clarification: 1) Most reputable sources attribute the discovery of the high temperature superconductor YBCO at the University of Houston in 1987 to Dr. Paul Chu, not Karl Mueller and Georg Bednorz as previously reported. Mueller and Bednorz discovered the high-temperature superconductor LBCO in 1986, winning a joint Nobel Prize for their work in 1987; and 2) Google DeepMind discovered 2.2 million theoretical materials, 380,000 of which that were stable at zero degrees Kelvin, making them promising candidates for synthesizing in the lab.
