New software based on Artificial Intelligence helps to interpret complex data
Experimental data is often not only highly dimensional, but also noisy and full of artefacts. This makes it difficult to interpret the data. Now a team at HZB has designed software that uses self-learning neural networks to compress the data in a smart way and reconstruct a low-noise version in the next step. This enables to recognise correlations that would otherwise not be discernible. The software has now been successfully used in photon diagnostics at the FLASH free electron laser at DESY. But it is suitable for very different applications in science.
More is not always better, but sometimes a problem. With highly complex data, which have many dimensions due to their numerous parameters, correlations are often no longer recognisable. Especially since experimentally obtained data are additionally disturbed and noisy due to influences that cannot be controlled.
Helping humans to interpret the data
Now, new software based on artificial intelligence methods can help: It is a special class of neural networks (NN) that experts call "disentangled variational autoencoder network (β-VAE)". Put simply, the first NN takes care of compressing the data, while the second NN subsequently reconstructs the data. "In the process, the two NNs are trained so that the compressed form can be interpreted by humans," explains Dr Gregor Hartmann. The physicist and data scientist supervises the Joint Lab on Artificial Intelligence Methods at HZB, which is run by HZB together with the University of Kassel.
Extracting core principles without prior knowledge
Google Deepmind had already proposed to use β-VAEs in 2017. Many experts assumed that the application in the real world would be challenging, as non-linear components are difficult to disentangle. "After several years of learning how the NNs learn, it finally worked," says Hartmann. β-VAEs are able to extract the underlying core principle from data without prior knowledge.
Photon energy of FLASH determined
In the study now published, the group used the software to determine the photon energy of FLASH from single-shot photoelectron spectra. "We succeeded in extracting this information from noisy electron time-of-flight data, and much better than with conventional analysis methods," says Hartmann. Even data with detector-specific artefacts can be cleaned up this way.
A powerful tool for different problems
"The method is really good when it comes to impaired data," Hartmann emphasises. The programme is even able to reconstruct tiny signals that were not visible in the raw data. Such networks can help uncover unexpected physical effects or correlations in large experimental data sets. "AI-based intelligent data compression is a very powerful tool, not only in photon science," says Hartmann.
Now plug and play
In total, Hartmann and his team spent three years developing the software. "But now, it is more or less plug and play. We hope that soon many colleagues will come with their data and we can support them."
- BESSY II: Phosphorous chains – a 1D material with 1D electronic propertiesFor the first time, a team at BESSY II has succeeded in demonstrating the one-dimensional electronic properties of a material through a highly refined experimental process. The samples consisted of short chains of phosphorus atoms that self-organise at specific angles on a silver substrate. Through sophisticated analysis, the team was able to disentangle the contributions of these differently aligned chains. This revealed that the electronic properties of each chain are indeed one-dimensional. Calculations predict an exciting phase transition to be expected as soon as these chains are more closely packed. While material consisting of individual chains with longer distances is semiconducting, a very dense chain structure would be metallic.
- Did marine life in the palaeocene use a compass?Some ancient marine organisms produced mysterious magnetic particles of unusually large size, which can now be found as fossils in marine sediments. An international team has succeeded in mapping the magnetic domains on one of such ‘giant magnetofossils’ using a sophisticated method at the Diamond X-ray source. Their analysis shows that these particles could have allowed these organisms to sense tiny variations in both the direction and intensity of the Earth’s magnetic field, enabling them to geolocate themselves and navigate across the ocean. The method offers a powerful tool for magnetically testing whether putative biological iron oxide particles in Mars samples have a biogenic origin.
- What vibrating molecules might reveal about cell biologyInfrared vibrational spectroscopy at BESSY II can be used to create high-resolution maps of molecules inside live cells and cell organelles in native aqueous environment, according to a new study by a team from HZB and Humboldt University in Berlin. Nano-IR spectroscopy with s-SNOM at the IRIS beamline is now suitable for examining tiny biological samples in liquid medium in the nanometre range and generating infrared images of molecular vibrations with nanometre resolution. It is even possible to obtain 3D information. To test the method, the team grew fibroblasts on a highly transparent SiC membrane and examined them in vivo. This method will provide new insights into cell biology.