AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Sodium-ion batteries: New storage mechanism for cathode materialsLi-ion and Na-ion batteries operate through a process called intercalation, where ions are stored and exchanged between two chemically different electrodes. In contrast, co-intercalation, a process in which both ions and solvent molecules are stored simultaneously, has traditionally been considered undesirable due to its tendency to cause rapid battery failure. Against this traditional view, an international research team led by Philipp Adelhelm has now demonstrated that co-intercalation can be a reversible and fast process for cathode materials in Na-ion batteries. The approach of jointly storing ions and solvents in cathode materials provides a new handle for the designing batteries with high efficiency and fast charging capabilities. The results are published in Nature Materials.
- 10 million euros in funding for UNITE – Startup Factory Berlin-BrandenburgUNITE – Startup Factory Berlin-Brandenburg has been recognised by the Federal Ministry for Economic Affairs and Energy as one of ten nationwide flagship projects for science-based start-ups. UNITE is to be established as a central transfer platform for technology-driven spin-offs from science and industry in the capital region. The Helmholtz Centre Berlin will also benefit from this.
- New Helmholtz Young Investigator Group at HZB on perovskite solar cellsSilvia Mariotti starts building up the new Helmholtz Young Investigator Group ‘Perovskite-based multi-junction solar cells’. The perovskite expert, who was previously based at Okinawa University in Japan, aims to advance the development of multi-junction solar cells made from different perovskite layers.