AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Cool vaccines in rural Kenya: solar solution has been awarded by UNIn May 2026, Tabitha Awuor Amollo is spending some weeks as a guest scientist at HZB, analysing perovskite thin films at BESSY II. The Kenyan physicist from Egerton University, Nairobi, was recently recognised for her achievements in research and teaching. For the development of a solar-powered refrigeration system for use in rural health centres, she has been awarded the 2026 Organization for Women in Science for the Developing World (OWSD)-Elsevier Foundation Award. An interview on exceptional projects and daily struggles of a scientist. Questions were asked by Antonia Rötger.
- BESSY II: How intrinsic oxygen shortens the lifespan of solid-state batteriesAlthough solid-state batteries (SSBs) demonstrate high performance and are intrinsically safe, their capacity currently declines rapidly. A team from the TU Wien, Humboldt-University Berlin and HZB has now analysed a TiS₂|Li₃YCl₆ solid-state half-cell in operando at BESSY II using a special sample environment that allows for non-destructive investigation under real operating conditions. Data obtained by combination of soft and hard X-ray photoelectron spectroscopy (XPS and HAXPES) revealed a new degradation mechanism that had not previously been identified in solid-state batteries. They have gained some surprising insights, particularly regarding the harmful role played by intrinsic oxygen. This study provides valuable information for improving design and handling of such batteries.
- Spintronics at BESSY II: Real-time analysis of magnetic bilayer systemsSpintronic devices enable data processing with significantly lower energy consumption. They are based on the interaction between ferromagnetic and antiferromagnetic layers. Now, a team from Freie Universität Berlin, HZB and Uppsala University has succeeded in tracking, for each layer separately, how the magnetic order changes after a short laser pulse has excited the system. They were also able to identify the main cause of the loss of antiferromagnetic order in the oxide layer: the excitation is transported from the hot electrons in the ferromagnetic metal to the spins in the antiferromagnet.