AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- TH Wildau and Helmholtz Zentrum Berlin signed comprehensive cooperationOn 21 May 2025, the Technical University of Applied Sciences Wildau (TH Wildau) and the Helmholtz-Zentrum Berlin für Materialien und Energie (HZB), signed a comprehensive cooperation agreement. The aim is to further promote networking and cooperation, particularly in basic research, to increase the scientific excellence of both partners and to develop competence networks in research, teaching and the training of young scientists.
- Green hydrogen: MXene boosts the effectiveness of catalystsMXenes are adept at hosting catalytically active particles. This property can be exploited to create more potent catalyst materials that significantly accelerate and enhance the oxygen evolution reaction, which is one of the bottlenecks in the production of green hydrogen via electrolysis using solar or wind power. A detailed study by an international team led by HZB chemist Michelle Browne shows the potential of these new materials for future large-scale applications. The study is published in Advanced Functional Materials.
- Joint Berlin Data & AI Center plannedData-driven research is crucial for tackling societal challenges- whether in health, materials, or climate research. In a collaboration that is so far unique, Berlin University Alliance (BUA), the Max Delbrück Center, and the Helmholtz-Zentrum Berlin, together with the Zuse Institute Berlin, aim to establish a powerful Data and AI Center in the German capital.