Blind spot for the arrangement of atoms

Artificial intelligence is increasingly being used to predict the properties of new materials. A research team from Jena and the United States now shows in a study published in the journal Nature Computational Science that when language models are used for this purpose, they reliably identify which types of atoms a material contains. However, they make insufficient use of information about how those atoms are arranged in space. Neither larger models nor additional training data resolve this weakness. For material properties that depend strongly on structure, AI models specifically designed to process spatial relationships are therefore the more reliable choice. (DOI: 10.1038/s43588-026-01045-2)

To predict the properties of a material, an AI model needs to know which elements it contains and how its atoms are arranged in space. ‘This arrangement is often crucial in chemistry,’ says Dr Kevin M. Jablonka, head of the Carl Zeiss Foundation junior research group ‘Polymers in Energy Applications’ at the Friedrich Schiller University Jena and HIPOLE Jena. ‘Diamond and graphite are made of the same carbon atoms. Yet diamond is transparent and extremely hard, whereas graphite is soft and conducts electricity. The difference lies in the arrangement of the atoms.’ Until now, it has been difficult to determine whether language models genuinely capture such geometric relationships or primarily rely on chemical composition.

To separate the two clearly, the team developed its own test datasets in which the positions and chemical identities of the atoms could be varied independently. The researchers also specified how strongly an artificially calculated, energy-like target value depended on each type of information. ‘This allowed us to distinguish clearly what the models had actually learned,’ explains first author Nawaf Alampara of Friedrich Schiller University Jena. Making this distinction is difficult with real materials: particular combinations of elements tend to form particular structures. A model may therefore arrive at a correct prediction without adequately capturing the spatial structure.

The same pattern emerged across all test datasets: the language models reliably learned which types of atoms were present. However, the more strongly the target value depended on their positions, the larger the prediction errors became. The researchers call this drop in performance the ‘Coordinate-Category Cliff’—a discrepancy between the processing of categorical and spatial information. Analyses of the models’ internal attention signals also showed that they made little use of the coordinates.

The team then examined whether the finding also held for real materials. Using MatText, an open-source test system developed specifically for the study, the researchers represented crystal structures as text in nine different ways. Additional geometric information often offered no advantage when predicting formation energies and mechanical properties: simpler representations without explicit coordinates sometimes performed just as well or even better.

Scaling did not resolve the weakness either. Neither training datasets containing up to two million structures nor language models with up to 70 billion parameters improved the predictions clearly and reliably. By contrast, specialised geometric and graph-based neural networks achieved lower errors on comparable tasks. ‘This is therefore not a question of AI or no AI,’ Jablonka emphasises. ‘The crucial point is that the tool must match the problem.’ Where spatial structures are decisive, models developed specifically for this type of information currently remain the more reliable choice.

The findings extend beyond materials research. They show that accurate predictions alone do not prove that an AI model has captured the relationships that are actually decisive. ‘In the natural sciences in particular, we therefore need to examine not only how accurately a model predicts, but also which information its results are based on,’ says Jablonka. The study was conducted in collaboration between Friedrich Schiller University Jena, HIPOLE Jena and Intel Labs in the United States.

The open-source test system MAtText  is available under https://lamalab-org.github.io/MatText/

About HIPOLE Jena
The Helmholtz Institute for Polymers in Energy Applications (HIPOLE Jena) was founded in 2020 as a joint initiative of Helmholtz-Zentrum Berlin (HZB) and Friedrich Schiller University Jena (FSU Jena). It develops innovative polymer materials for sustainable energy technologies offering high efficiency, scalability and mechanical flexibility. Through its close collaboration with the Center for Energy and Environmental Chemistry Jena (CEEC Jena), HIPOLE Jena combines cutting-edge materials development with high-precision characterisation. Its research ranges from polymer-based batteries and photovoltaics to functional self-healing materials and sustainable chemistry, with the aim of opening up new pathways towards a climate-neutral energy future.

About the Carl Zeiss Foundation
The Carl Zeiss Foundation aims to create freedom for scientific breakthroughs. As a partner of excellent science, it supports basic and applied research as well as teaching in the STEM fields of mathematics, computer science, natural sciences and technology. Founded in 1889 by the physicist and mathematician Ernst Abbe, the Carl Zeiss Foundation is one of Germany’s oldest and largest private foundations supporting science. It is the sole owner of Carl Zeiss AG and SCHOTT AG. Its projects are financed from the dividends distributed by the two foundation companies.