AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Long-term test shows: Efficiency of perovskite cells varies with the seasonScientists at HZB run a long-term experiment on the roof of a building at the Adlershof campus. They expose a wide variety of solar cells to the weather conditions, recording their performance over a period of years. These include perovskite solar cells, a new photovoltaic material offering high efficiency and low manufacturing costs. Dr Carolin Ulbrich and Dr Mark Khenkin evaluated four years of data and presented their findings in Advanced Energy Materials. This is the longest series of measurements on perovskite cells in outdoor use to date. The scientists found that standard perovskite solar cells perform very well during the summer months, even over several years, but decline in efficiency during the darker months.
- Sodium-ion batteries: New storage mechanism for cathode materialsLi-ion and Na-ion batteries operate through a process called intercalation, where ions are stored and exchanged between two chemically different electrodes. In contrast, co-intercalation, a process in which both ions and solvent molecules are stored simultaneously, has traditionally been considered undesirable due to its tendency to cause rapid battery failure. Against this traditional view, an international research team led by Philipp Adelhelm has now demonstrated that co-intercalation can be a reversible and fast process for cathode materials in Na-ion batteries. The approach of jointly storing ions and solvents in cathode materials provides a new handle for designing batteries with high efficiency and fast charging capabilities. The results are published in Nature Materials.
- 10 million euros in funding for UNITE – Startup Factory Berlin-BrandenburgUNITE – Startup Factory Berlin-Brandenburg has been recognised by the Federal Ministry for Economic Affairs and Energy as one of ten nationwide flagship projects for science-based start-ups. UNITE is to be established as a central transfer platform for technology-driven spin-offs from science and industry in the capital region. The Helmholtz Centre Berlin will also benefit from this.