AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Environmental Chemistry at BESSY II: Radicals in waterwaysHow do radicals form in aqueous solutions when exposed to UV light? This question is important for health research and environmental protection, for example with regard to the overfertilisation of water bodies by intensive agriculture. A team at BESSY II has now developed a new method of investigating hydroxyl radicals in solution. By using a clever trick, the scientists gained surprising insights into the reaction pathway.
- Theory meets practice – We’re heading back to HTW Berlin!The HZB’s BIPV consultancy office (BAIP) is once again coordinating and delivering the lecture series “Building-Integrated Photovoltaics”.
- AI-driven Catalyst Discovery: €30 million funding for German consortiumSix partners from research and industry, including Helmholtz-Zentrum Berlin (HZB), the Fritz-Haber-Institute of the Max Planck Society (FHI), BASF, Dunia Innovations, Siemens Energy, and the Technical University Berlin are launching a joint project to accelerate the catalyst discovery. The German Federal Ministry for Science, Technology and Space (BMFTR) is providing €30 million in funding for ASCEND (Accelerated Solutions for Catalysis using Emerging Nanotechnology and Digital Innovation). The research initiative targets the defossilisation of energy-intensive industries while safeguarding industrial competitiveness, with a focus on the chemical sector. The five-year project will start on 1st April 2026.