AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Porous Radical Organic framework improves lithium-sulphur batteriesA team led by Prof. Yan Lu, HZB, and Prof. Arne Thomas, Technical University of Berlin, has developed a material that enhances the capacity and stability of lithium-sulphur batteries. The material is based on polymers that form a framework with open pores (known as radical-cationic covalent organic frameworks or COFs). Catalytically accelerated reactions take place in these pores, firmly trapping polysulphides, which would shorten the battery life. Some of the experimental analyses were conducted at the BAMline at BESSY II.
- Metallic nanocatalysts: what really happens during catalysisUsing a combination of spectromicroscopy at BESSY II and microscopic analyses at DESY's NanoLab, a team has gained new insights into the chemical behaviour of nanocatalysts during catalysis. The nanoparticles consisted of a platinum core with a rhodium shell. This configuration allows a better understanding of structural changes in, for example, rhodium-platinum catalysts for emission control. The results show that under typical catalytic conditions, some of the rhodium in the shell can diffuse into the interior of the nanoparticles. However, most of it remains on the surface and oxidises. This process is strongly dependent on the surface orientation of the nanoparticle facets.
- KlarText Prize for Hanna TrzesniowskiThe chemist has been awarded the prestigious KlarText Prize for Science Communication by the Klaus Tschira Foundation.