ESS Demonstrates Full Data Processing Pipeline in Operation

The European Spallation Source (ESS) has successfully demonstrated the functionality of its full data processing pipeline. The online demonstration, conducted by the Data Management and Scientific Computing (DMSC) Centre for the DMSC Science and Technology Advisory Panel (STAP), showcased the end-to-end process from proposal submission to analyzed results.

The demonstration covered data pipelines for instruments including BIFROST, DREAM, ESTIA, LOKI, and NMX, utilizing simulated or, where possible, real data from other neutron sources. The workflow included submitting proposals via the ESS user portal, accessing and analyzing data, and ensuring the availability of generated results. The objective was to validate the readiness of ESS's data systems for the facility's upcoming neutron science operations.

To further test the pipeline's functionality on actual hardware, a light tomography demonstration was performed at the YMIR test stand in Lund. YMIR served as a substitute for the ODIN instrument, confirming the data processing pipeline's compatibility with neutron instrument hardware. This step marks a significant milestone in the development of ESS's data infrastructure.

Thomas H. Rod, Head of DMSC, described the demonstration as a success, though areas for improvement were identified. "It was incredibly rewarding to see all the components we have developed function as expected," Rod stated, emphasizing the immense value of the demonstrations for development efforts. This milestone signifies ESS's commitment to operational excellence and ensures the infrastructure is prepared to meet the demands of a world-class neutron facility.