AI in Chemistry: Study Highlights Strengths and Weaknesses
Computing power in the chemistry lab: Kevin Jablonka (left) and his team at HIPOLE Jena. Photo: Renzo Paulus
How well does artificial intelligence perform compared to human experts? A research team at HIPOLE Jena set out to answer this question in the field of chemistry. Using a newly developed evaluation method called “ChemBench,” the researchers compared the performance of modern language models such as GPT-4 with that of experienced chemists.
The study has recently been published in the journal Nature Chemistry (DOI 10.1038/s41557-025-01815-x).
More than 2,700 chemistry tasks from research and education were tested—ranging from fundamental knowledge to complex problems. In areas such as reaction prediction or the analysis of large datasets, AI models often excelled with high efficiency. However, a critical weakness became apparent: the models also produced confident answers even when they were factually incorrect. Human chemists, by contrast, were more cautious and questioned their own assessments.
“Our study shows that AI can be a valuable tool—but it is no substitute for human expertise,” says Dr. Kevin M. Jablonka, lead author of the study. The findings offer important insights for the responsible use of AI in chemical research and education.
HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena) is an institute of HZB in cooperation with Friedrich Schiller University Jena (FSU Jena).
- Five Berlin-based research institutions join forces in data-driven materials researchResearch data is regarded as key to materials research in the age of artificial intelligence (AI). Five Berlin-based research institutions have now signed a Memorandum of Understanding (MoU) to establish long-term collaboration in the fields of research data, data infrastructures and AI.
- Green hydrogen with PEC electrolysers: New insights into transport processesOne method of storing solar energy is to use PEC electrolysers to produce hydrogen. However, scaling up this technology remains challenging. Now, a team at the HZB Institute for Solar Fuels has used 2D fluorescence imaging and particle velocimetry to observe the movement of ions and dissolved gases within the electrolyte during electrolysis. These new insights may prove useful in the development of larger PEC electrolysers.
- New contact material boosts the efficiency of perovskite solar cellsA newly developed material for the electron contact improves the efficiency of single perovskite solar cells and perovskite/silicon tandem solar cells. The new material is based on a carborane molecule. It offers several advantages over the standard material C60, as shown by the study led by Steve Albrecht’s team. The new material has since been patented and is already commercially available.