AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- BESSY II is back in operation after maintenance shutdownOn 7 September 2026, BESSY II was set into operation after a four-week shutdown. BESSY II is scheduled to resume full user operation on 22 September.
- 3D magnetic field experiment at BESSY II takes spintronics a step further(Fe0.63Ni0.3Pd0.07)3P or FNPP is a magnetic material that exhibits complex magnetic structures even at room temperature. This makes the material of interest for spintronics, a field that could enable data processing with significantly lower energy consumption. One potential application is novel magnetic memory devices. However, generating and modifying the desired structures in a controlled manner remains a challenge to date. A new study led by HZB has now taken a step forward in this regard. They demonstrated at the worldwide unique VEKMAG-Station at BESSY II, that a tiny external B-field in the plane of the magnetic patterns is sufficient to change them.
- Thin nickel films boost green hydrogen productionUltra-thin nickel oxide coatings can significantly improve the performance of anion exchange membrane (AEM) water electrolysers, a promising technology for producing green hydrogen. This was demonstrated by a team led by HZB scientist Dr. Michelle Browne. The results show that a nickel oxide film only 10 nanometres thick outperformed conventional nickel oxide powder electrodes while using less than 1% of the catalyst material. The study is published in the Journal of Materials Chemistry A,