AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- HZB congratulates Peter Hegemann on winning the Nobel PrizeOur warmest congratulations go to Peter Hegemann, Professor of Neuroscience and Experimental Biophysics at Humboldt University in Berlin. Together with Prof. Georg Nagel (Julius Maximilian University of Würzburg) and Prof. Karl Deisseroth (Stanford University, USA), he has been awarded the 2026 Nobel Prize in Physiology or Medicine. We are delighted for him, his team, Humboldt University and the entire Berlin scientific community.
- Science in Berlin: collaborative, diverse, innovative, internationalScience is a key social and economic factor for Berlin. It creates jobs and adds value, attracts talent from all over the world, and enriches city life. To ensure that Berlin remains one of the world’s leading centres of science in the future, science must feature even more prominently on the political agenda. This is what BR50 stands for: over 50 non-university research institutions, including the HZB, are joining forces in this network. Following the election in Berlin, BR50 is now addressing politicians with a position paper, accompanied by the campaign “𝗪𝗶𝗿 𝗳𝗼𝗿𝘀𝗰𝗵𝗲𝗻 𝗳ü𝗿 𝗕𝗲𝗿𝗹𝗶𝗻.”
- Does African Architecture exist?The world’s first Pan-African Architecture Biennale took place this September in Nairobi, Kenya. People from the worlds of architecture, art, academia and politics from 54 African countries and the diaspora had travelled to Nairobi to exchange ideas on traditional and contemporary building methods, materials, experiences and projects. Samira Aden co-organised this biennial as lead curator. We spoke to her about it.