AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- New Helmholtz Young Investigator Group at HZB on perovskite solar cellsSilvia Mariotti is returning to HZB as head of the new Helmholtz Young Investigator Group ‘Perovskite-based multi-junction solar cells’. The perovskite expert, who was previously based at Okinawa University in Japan, aims to advance the development of multi-junction solar cells made from different perovskite layers.
- Hydrogen storage in MXene: It all depends on diffusion processesTwo-dimensional (2D) materials such as MXene are of great interest for hydrogen storage. An expert from HZB has investigated the diffusion of hydrogen in MXene using density functional theory. This modelling provides valuable insights into the key diffusion mechanisms and hydrogen's interaction with Ti₃C₂ MXene, offering a solid foundation for further experimental research.
- HZB and National University Kyiv-Mohyla Academy start cooperation in Energy and ClimateHelmholtz-Zentrum Berlin für Materialien und Energie GmbH (HZB) and the National University of "Kyiv-Mohyla Academy" (NaUKMA) have signed a Memorandum of Understanding (MoU). The MoU serves as the starting point for collaborative research, academic exchange, and capacity-building between the two institutions. Actions will be taken to establish the Joint Research and Policy Laboratory at NaUKMA in Kyiv. The aim of the future laboratory is to jointly develop research and policy analysis, focusing on the energy and climate dimensions of Ukraine’s EU integration.