AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Catalysis research at HZB gets new facilityAs part of the CatLab project, HZB has acquired a unique facility for measuring the catalytic performance of thin-film catalysts. Built by ILS in Adlershof, it has now been delivered. The facility consists of a total of eight chemical reactors in which catalytic systems can be tested. At over €2.5 million, this is the largest single investment in the CatLab project.
- Protein crystallography at BESSY II: faster, better and more and more automaticMany diseases are linked to malfunctions of proteins in the organism. The three-dimensional architecture of these molecules is often highly complex, but it can provide valuable insights into biological processes and the development of drugs. X-ray diffraction at the MX beamlines of BESSY II can be used to decipher the 3D structure of proteins. To date, more than 5000 structures have been solved at the three MX beamlines. Here, we present a review and an outlook with Manfred Weiss, head of the research group for macromolecular crystallography.
- Humboldt-Fellow at HZB-Institute for Solar Fuels: Alexander R. UhlAlexander R. Uhl, UBC Okanagan School of Engineering in Kelowna, Canada, aims to develop with Roel van de Krol from the HZB Institute for Solar Fuels an efficient and inexpensive photoelectrolyser for producing hydrogen using sunlight. His stay is being funded by the Alexander von Humboldt Foundation.