AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Berlin Science Award goes to Philipp AdelhelmBattery researcher Prof. Dr. Philipp Adelhelm has been awarded the 2024 Berlin Science Award. He is a professor at the Institute of Chemistry at Humboldt University in Berlin (HU) and heads a joint research group at HU and the Helmholtz Zentrum Berlin (HZB). The materials scientist and electrochemist is investigating sustainable batteries, which play a key role in the success of the energy transition. He is one of the leading international experts in the field of sodium-ion batteries.
- Scrolls from Buddhist shrine virtually unrolled at BESSY IIThe Mongolian collection of the Ethnological Museum of the National Museums in Berlin contains a unique Gungervaa shrine. Among the objects found inside were three tiny scrolls, wrapped in silk. Using 3D X-ray tomography, a team at HZB was able to create a digital copy of one of the scrolls. With a mathematical method the scroll could be virtually unrolled to reveal the scripture on the strip. This method is also used in battery research.
- Long-term test shows: Efficiency of perovskite cells varies with the seasonScientists at HZB run a long-term experiment on the roof of a building at the Adlershof campus. They expose a wide variety of solar cells to the weather conditions, recording their performance over a period of years. These include perovskite solar cells, a new photovoltaic material offering high efficiency and low manufacturing costs. Dr Carolin Ulbrich and Dr Mark Khenkin evaluated four years of data and presented their findings in Advanced Energy Materials. This is the longest series of measurements on perovskite cells in outdoor use to date. The scientists found that standard perovskite solar cells perform very well during the summer months, even over several years, but decline in efficiency during the darker months.