AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- BESSY II: New sample environment allows glimpse into thermocatalytic processesA novel measurement cell allows, for the first time, soft and hard X-ray investigations under high pressures of up to 20 bar and temperatures of up to 400°C. This provides new insights into thermocatalytic processes, such as the Fischer-Tropsch synthesis for producing synthetic fuels. The development of the measurement cell is considered a significant achievement within the Care-O-Sene project.
- Precision interface chemistry pushes perovskite solar cells beyond 26% efficiencyAn international research collaboration has developed a new molecular strategy for controlling one of the most critical interfaces in perovskite solar cells. The resulting solar cells reached a power conversion efficiency of 26.19% in the n i p architecture, together with strong operational stability under prolonged illumination and elevated temperature. The results have been published in the Journal of the American Chemical Society.
- Perovskite triple-junction solar cells: Even more efficient with GO/SAM bilayersPerovskite semiconductors efficiently convert sunlight into electrical energy; they are also inexpensive and extremely lightweight. A team at HZB has developed a triple-junction solar cell comprising different perovskite semiconductors, with a novel bilayer of graphene oxide (GO) and a self-assembled monolayer (SAM) as the hole conductor. This bilayer significantly increases both efficiency and long-term stability. The efficiency of the novel perovskite triple-junction solar cell is 27.3% and shows hardly any decline even after more than 770 hours of operation. The study has been published in the renowned journal Joule.