AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Ultrafast demagnetisation - Thinner iron, stronger effectThe thinner an iron film, the more easily it can be demagnetised by a laser pulse. Researchers at the University of Duisburg-Essen, working with Helmholtz-Zentrum Berlin and other partners, have discovered why: the key lies in the magnetic order at the material's interfaces. In the long term, these findings could contribute to the development of new technologies for magnetic data storage. The results have been published in the scientific journal Advanced Science.
- Blind spot for the arrangement of atomsArtificial intelligence is increasingly being used to predict the properties of new materials. A research team from Jena and the United States now shows in a study published in the journal Nature Computational Science that when language models are used for this purpose, they reliably identify which types of atoms a material contains. However, they make insufficient use of information about how those atoms are arranged in space. Neither larger models nor additional training data resolve this weakness. For material properties that depend strongly on structure, AI models specifically designed to process spatial relationships are therefore the more reliable choice. (DOI: 10.1038/s43588-026-01045-2)
- Patterns of polar vortices in lead-free superlattices on siliconA team at HZB has, for the first time, reported polar vortices in (BaTiO3/SrTiO3)10 superlattices on silicon. These results expand the range of materials and substrates on which polar topological textures can occur and open up new options for incorporating complex ferroelectric states into silicon-based technologies.