AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- HZB and National University Kyiv-Mohyla Academy start cooperation in Energy and ClimateHelmholtz-Zentrum Berlin für Materialien und Energie GmbH (HZB) and the National University of "Kyiv-Mohyla Academy" (NaUKMA) have signed a Memorandum of Understanding (MoU). The MoU serves as the starting point for collaborative research, academic exchange, and capacity-building between the two institutions. Actions will be taken to establish the Joint Research and Policy Laboratory at NaUKMA in Kyiv. The aim of the future laboratory is to jointly develop research and policy analysis, focusing on the energy and climate dimensions of Ukraine’s EU integration.
- Long Night of Science 2025Welcome!
- MAX IV and BESSY II initiate new collaboration to advance materials scienceSwedish national synchrotron laboratory MAX IV and Helmholtz-Zentrum Berlin (HZB) with BESSY II light source jointly announce the signing of a 5-year Cooperation Agreement. The new agreement establishes a framework to strengthen cooperation for operational and technological development in the highlighted fields of accelerator research and development, beamlines and optics, endstations and sample environments as well as digitalisation and data science.