Powering discovery: how the Janet Network connects the world’s most data-hungry science

CERN, the European Organisation for Nuclear Research, is the world’s largest particle physics laboratory. It operates the Large Hadron Collider (LHC), which scientists are using to investigate the fundamental structure of the universe and the basic constituents of matter.

Essential to these experiments is the transfer of large amounts of data between institutions and researchers collaborating around the world. This amounts to in excess of two thousand petabytes (PBs) of data across the networks each year.

A globally distributed computing infrastructure called the Worldwide LHC Computing Grid (WLCG) enables CERN to collaborate with over 170 universities and research institutes worldwide across its range of projects.

To ensure resilience, even in the event of a catastrophe, Tier 1 institutions have custodial responsibility for the data, which is stored on discs and tapes in huge quantities - around 1 exaByte (EB) on disk and 2 EB of tape storage.

The UK contribution to the WLCG is GridPP, a community of particle physicists and computer scientists based in the United Kingdom and at CERN, drawing on expertise from 18 UK institutions.

Jisc and the Janet Network have a relationship with particle physics that goes back for many decades, and Jisc is a participating member of the International LHC Network coordinating group that meets twice yearly. This involves all peer National Research and Education Networks (NRENs), including GÉANT, the collaboration of European NRENs. GÉANT directly connects with the Janet Network in London, from where up to 400Gbps of data can flow to and from institutions throughout the UK.

We also join weekly GridPP operational meetings and support in numerous other ways, including monitoring performance and participating in exploring new technologies that improve data transfer.

The primary link between CERN and the UK is the Tier 1 centre at RAL on the Harwell Campus in Oxfordshire, which is funded and managed by the Science and Technology Facilities Council (STFC). The site has 400Gbps resilient IP connectivity to Janet and a dedicated 200Gbps private link to CERN.

Meeting the highest data demands of any scientific project, anywhere in the world

The LHC collides proton beams at the highest energies that have ever been created on Earth, producing by far the most data from any scientific project in the world.

Among its experiments are ATLAS and CMS, operating at the energy frontier and famous for their role in the 2012 discovery of the Higgs Boson particle; LHCb (Large Hadron Collider beauty), which focuses on the subtle differences between matter and antimatter by studying the decay of heavy particles containing the "beauty" quark; and ALICE, dedicated to heavy Ion physics.

So far, only around 4% of the universe is understood in detail, with dark matter expected to account for roughly 25% of all matter. The search for candidates for dark matter is a key strand of the experiments. The resulting data, stored at CERN, is distributed to hundreds of institutions around the world. It is then reconstructed into products that international teams of collaborating physicists can analyse before sharing results.

This is achieved through a highly sophisticated and performant distributed computing system. In fact, the data demands of particle physics effectively created the model for distributed computing on an enormous scale - long before cloud computing became common.

Reliability, bandwidth and low packet loss are crucial to enable high-rate data movement. Maintaining strong connectivity and computing capability is also essential to preserve the UK’s competitiveness on a global stage, where it has traditionally ‘punched above its weight’.

The networking infrastructure is not peripheral to the science - it’s part of it

Peter Clarke, professor of physics at the University of Edinburgh, is deputy project lead of GridPP, for which he is responsible at the project management level for networking. He says:

“Modern particle physics is as much a computing and data challenge as it is a physics challenge. The detectors themselves are extraordinary pieces of engineering, but extracting meaningful science from the data depends on this enormous distributed infrastructure sitting behind everything. Reliable, high-bandwidth network infrastructure is not peripheral to the science - it is part of the science.”

Preparing to meet the challenges of the future

Every new generation of experiments at CERN produces more data and requires more computing power. Staying ahead by consistently upgrading infrastructure, improving software efficiency and increasing network capacity is essential.

We participate in regular data challenges, which involve global collaboration to push vast amounts of data through the network to test its current capability and how it will stand up to future demands. The Particle Physics Data Challenge in 2024 transferred approximately 120 petabytes of data worldwide in just two weeks - still only a quarter of the demand expected by 2030, when the High-Luminosity phase begins. The next global test will take place in February 2027 with a target of 50% of that demand.

The experiences of large-scale data handling from CERN are being used by Jisc and others to inform other large data transfer projects, including the Dune Neutrino experiment in Chicago which transfers data over ESNet to the UK, the Square Kilometre Array radio telescope project, which has giant telescopes in Australia and South Africa and is headquartered at Jodrell Bank, Manchester, and the Vera Rubin telescope in Chile, for which the UK is processing 25% of the data.

“Jisc provides an extremely high-quality service that we can depend on. Most physicists probably never think about it day to day, which is the sign that it is working properly. In most places people only go to their network teams if they think something is wrong. In GridPP, we make a point of telling Jisc what a good job they are doing".

—Professor Peter Clarke, University of Edinburgh and GridPP

Further information

If you’re new to Jisc and would like to speak to one of our experts regarding a Janet Network connection, make a customer enquiry.

Alternatively, if you’re an existing member or customer, please contact your relationship manager.