National Grid Service
The goal of the NGS is to support innovation and collaboration in academic research across a wide range of disciplines.
Version 2: September 2008 (read Version 1)
What is a Grid?
When browsing the web, you type in a URL or click on a hyperlink and the page appears in your browser. Beneath this simple action, however, a complicated chain of events occurs to retrieve and load the elements within the page – images, text, animations, etc. – from remote sources spread over the planet.
A grid shares with the web the ability to access remote, distributed resources while hiding the underlying complexities. Whereas the web retrieves information, however, a grid also retrieves and integrates services providing computational power, data storage capacity and instruments. These resources are connected into a virtual organisation, which is really a social structure of people who have agreed to share resources in some manner for a period of time. A grid thus enables people to work collaboratively on large-scale scientific, engineering and compute- and data-intensive challenges.
The software that enables these resources to work together with little or no user intervention is called ‘middleware’ because it lies between the network connecting the resources and the user. It is analogous to the protocols a web browser launches when retrieving all the different elements of a web page.
Most grids use middleware based on the Globus toolkit, which is open source software developed by the Globus Alliance. The Storage Resource Broker (SRB), developed by the San Diego Supercomputer Center, is a commonly used tool for storing and accessing data over a grid. However, grid middleware is still under development and it could be some years before a complete set of standards and protocols is defined.
The UK NGS
The National Grid Service (NGS), funded by the Joint Information Systems Committee (JISC) and the Engineering and Physical Sciences Research Council (EPSRC), was created in October 2003 and entered full production in September 2004. It is led and coordinated through the Rutherford Appleton Laboratory (RAL) in collaboration with the University of Manchester, the University of Oxford and the White Rose Grid at Leeds.
The NGS’s resources are provided by the four founding members plus the national high performance computing facility (HPCx) and a growing number of partner and affiliate sites. The High End Computing Terascale Resource (HECToR), the UK’s new high performance computing facility, is due to join shortly. The four core sites currently provide more than 190TB of data storage and 580 dual-core processor nodes for computation, with Manchester and RAL focusing on data-intensive applications.
The standardisation of interfaces, based on the Globus toolkit and SRB, allows additional sites to join and the Universities of Cardiff, Lancaster, Queen’s University Belfast, Glasgow and Westminster are contributing a variety of additional compute resources. The University of Westminster also provides a portal to allow NGS users to manage their use of the service through a graphical interface.
Standard interfaces also enable the NGS to interoperate with other grids both nationally and internationally. Within the UK, these include GridPP (the particle physics grid), campus grids at a growing number of universities and regional grids such as the White Rose Grid. Internationally, the NGS provides a national ‘gateway’ to other e-infrastructures, such as the US TeraGrid. It also helps to ensure international interoperability by participating in activities related to the European Enabling Grids for E-sciencE (EGEE) project and sitting on international standards bodies such as the Open Grid Forum (formerly the Global Grid Forum).
How to Use the NGS
Before applying for an account on the NGS, you need a digital certificate from the UK Certification Authority. You can then apply for an NGS account online where you can also find help and advice.
Users who know which resources they want to access can submit a job directly to their chosen site or to their chosen NGS resource. Others can access the most appropriate available resources for a task by means of resource brokers or web-based portals. Portals have the advantage of allowing users to build workflows that, as well as defining procedures for executing a job, also capture metadata about how the job was done, thus facilitating later repetition.
The NGS is working with JISC to simplify access by making it compatible and interoperable with the new single sign-on to web-based services that JISC is introducing for the UK academic community. From early 2009, users with a UK Access Management Federation login will be able to access NGS services via a web portal. This is one of many services JISC will be rolling out over the next few years to take advantage of single sign-on for universities via the federation.
Future plans for the NGS aim to provide integrated, coherent access to the full range of the UK’s computation and database research facilities, together with a range of sophisticated services to support novel collaborative and cross-resource activities.
Who’s using the NGS
The goal of the NGS is to support innovation and collaboration in academic research across a wide range of disciplines. The following are some examples of NGS use:
- The NGS, in conjunction with other national grids such as the US TeraGrid, is helping clinicians plan surgery with greater accuracy by providing them with real-time visualisations of blood flow through a patient’s brain. The visualisations are of computer simulations of blood flowing through a computer model of the brain derived from magnetic resonance imaging (MRI) scans. The surgeon is able to steer the simulation by moving the model around to get a better look at areas of interest
- The NGS is hosting the database of Computer Crystal Structures for a major Research Council-funded project. It contains the computational predictions of the crystal structures adopted by an organic molecule. Users have easy access to data files via a data portal and the Crystal Navigator web interface
- A real, but anonymous, model of the UK population based on 2001 census data, is being run on the NGS to develop national demographic simulations and to predict future population trends. Social scientists and policy planners can use the model to help answer specific questions about, for example, future demand for services in a particular area under different scenarios
- RealityGrid a major UK e-science project, simulated and visualised complex condensed matter structures, such as oil moving through porous rock. Some of the simulations used supercomputers on two continents – HPCx in the UK, accessed via the NGS, and supercomputers on the US TeraGrid, which were linked in a transatlantic grid
The NGS is also developing the grid interfaces to large-scale facilities, such as the ISIS Neutron Source and the Diamond Light Source, both located at RAL in Oxfordshire. The aim is to define the middleware that will enable researchers to transfer results directly to compute resources for post-processing, analysis and visualisation.
NGS helps plan radiotherapy treatment
‘Radiotherapy is a critically important tool in the treatment of cancer. To get the most effective treatment possible, we need to simulate (model) the interactions between the radiation and the tumour. A radiotherapy treatment consists of approximately 100 trillion such interactions, and as a consequence, the full simulation requires an excessive amount of computer time.
Previously, we had to run the simulations on a PC over several days. However, access to the National Grid Service has reduced calculation times by a factor of about 10: answers are now available in a matter of hours.’
Dr Steven Weston, Institute of Medical Physics and Engineering, Yorkshire Centre for Clinical Oncology
This briefing paper was compiled and edited by Dr Andrew Richards, Director of the National Grid Service, and Judy Redfearn from the JISC Communications & Marketing team.
Further Information
Introduction to grids
Middleware
Grid news