AI agents deliver results – but do they reason scientifically?
Corral makes visible how AI agents arrive at their results: the benchmark breaks down agent runs into hypotheses, tests, evidence, judgments and corrections. This makes it possible to identify both productive forms of scientific reasoning and problematic patterns, such as when evidence is not taken into account. © HIPOLE Jena / Corral
A research team co-led by Kevin Maik Jablonka from the Helmholtz Institute for Polymers in Energy Applications Jena (HIPOLE Jena) and N. M. Anoop Krishnan from the Indian Institute of Technology Delhi has developed Corral, a new benchmark for AI agents in science. The preprint “AI scientists produce results without reasoning scientifically” has been published on arXiv (https://doi.org/10.48550/arXiv.2604.18805). The analysis shows that current systems can execute scientific workflows and deliver results; however, they often do not follow the basic principles of scientific testing and reasoning.
Artificial intelligence is expected not only to write texts or analyse data, but also to plan scientific experiments, analyse results and generate new knowledge. But when can an AI system truly be said to be doing science? Is it enough for the final result to be correct – or must the path to that result also meet scientific standards? This question is addressed in a new preprint by Jablonka’s team.
With Corral, the researchers developed a benchmark that evaluates AI-based scientific agents not only by their results, but also by how they arrive at them. To do this, the team analysed more than 25,000 agent runs across eight scientific domains – ranging from molecular simulations and materials data analysis to spectroscopic structure elucidation and hypothesis-driven chemical tests. The evaluation examined not only whether a task was solved, but also whether the systems take evidence into account, generate and test hypotheses, and revise their assumptions when confronted with contradictory results.
“We need to be clearer about what kind of scientific reasoning we expect from such AI systems,” says Jablonka. “When it comes to epistemic rigor, better training procedures may help. But in areas where we need reliable guarantees about the reasoning process, we will probably need different systems – for example, systems with symbolic and formally verifiable components.”
HIPOLE Jena is an institute of the Helmholtz-Zentrum Berlin für Materialien und Energie (HZB), operated in cooperation with Friedrich Schiller University Jena and the Center for Energy and Environmental Chemistry Jena (CEEC). Regarding the preprint, Helmholtz AI has already published a detailed background article on the work: https://www.helmholtz.ai/detail/do-ai-scientists-actually-do-science-new-benchmark-probes-the-reasoning-behind-the-results-featuring-dr-kevin-maik-jablonka-helmholtz-ai-associate/
- Hatice Mutlu appointed to the Helmholtz Institute HIPOLE in JenaProf. Dr Hatice Mutlu is the new professor of sustainable polymer chemistry at HIPOLE Jena, the Helmholtz Institute for Polymers in Energy Applications. On 17 August 2026, she visited the HZB campus in Berlin-Adlershof including the BESSY II lightsource and discussed future collaboration with researchers and the board of directors.
- Special prize by Jugend Forscht awarded to former school internShaoxuan Wang has won a special prize in the Jugend Forscht competition for her project ‘Smart Capsules – intelligent microcapsules for glucose-dependent insulin release’. She designed the project as a “special achievement” for her A-levels. To carry out the experimental work, Shaoxuan was able to spend a further two weeks in the laboratory at the HZB Institute for Electrochemical Energy Storage, where she had completed a school internship two years earlier.
- Five Berlin-based research institutions join forces in data-driven materials researchResearch data is regarded as key to materials research in the age of artificial intelligence (AI). Five Berlin-based research institutions have now signed a Memorandum of Understanding (MoU) to establish long-term collaboration in the fields of research data, data infrastructures and AI.