Why use external data?
Information used to support business intelligence and contribute to strategic planning can be derived from data both generated and held within organisations and from data that is generated and hosted by external data collectors and organisations.
Although there are many sources of external data of value and relevance to organisations, discussion here is confined to eight bodies, six of whose chief role is to provide educational services and data to government and organisations themselves.
These are the:
- Higher Education Statistics Agency (HESA)
- Universities and Colleges Admissions Service (UCAS)
- Higher Education Funding Council for England (HEFCE)
- Universities and Colleges Information Systems Association (UCISA)
- Higher Education Academy (HEA)
- Learning Records Service (LRS) (formerly Managing Information Across Partners (MIAP))
- Administrative Data Liaison Service (ADLS), a research council funded service that supports research into administrative data
- Office of National Statistics (ONS), the provider of statistics to government, also holds educational data
The ability to ‘add new information sources (internal or external)’ has been identified by the project as a ‘required attribute’ for any business intelligence systems.
This enables organisations to draw data and information from across internal applications (such as HR, finance and student record systems) as well as in combination with external data sources as required.
External data relevant to all the subject categories identified through our survey is available to a greater or lesser degree. Although most of this external data available is statistical, some qualitative data (such as the National Student Survey) is available too.
Currently although reuse of external data across the sector remains patchy, data providers are working to help HE and FE organisations to apply it more effectively in their planning processes and interest in the ability to ‘benchmark’ organisational performance against that of peers continues to grow.
The benefits of using external data
Organisations that use external data effectively have the potential to place themselves ahead of the game in terms of strategic planning and competitiveness within the sector. Benefits include:
- External data providers make available high quality information and data for reuse by organisations to support strategic planning
- The quality of data held is assured
- Large quantities of data are freely available to organisations from providers’ websites
- Bespoke services are provided when more detailed data is required
- Regular publications are provided in hard copy form by some providers
- High level data on peer organisations enables comparisons to be made
- Time series and historical data enables comparisons over time
- Training in the use of data is offered by some providers
- Ongoing discussion between providers aims to provide a rounded service
- Data providers are working proactively to enhance the usability of their data
- Allows an organisation to benchmark specific aspects of its own performance against that of peer and/or rival organisations.
The challenges associated with using external data
There are still challenges in delivering and using external data for optimum results, both for organisations and data providers. These challenges include:
- Working with statistics is still seen as a burden rather than a benefit by some managers
- Some managers still see working with statistics as a function just for the IT department
- Without experience it can be difficult to frame the right question to ask external providers
- It can be expensive to acquire data from external data providers
- It can be difficult to translate statistics into meaningful information accurately
- Providers need to supply more guidance and case studies on reuse to the sector
- A lack of data join up (about the same data) between external providers can lead to inefficiency and inaccurate outcomes
- It can be difficult to join up externally with internally held data to draw accurate conclusions
- It is difficult to obtain data at a sufficient level of detail for making useful comparisons with competitors
- Using open source datasets (such as those available from data.gov.uk or via other Open API’s (application programming interfaces)) can lead to additional complications. The University of Sheffield, for example, ran into the following issues in relation to data provided from data.gov.uk:
- Variations in the level of granularity of data within/between datasets making comparison difficult
- Inconsistent/missing metadata
- Sporadic mix of topics covered.
The experience of the University of Sheffield with using open data from data.gov.uk exemplifies some of the common problems experienced when working with big data.
These are sometimes described as the ‘3 Vs’: Volume, Velocity and Variety. ‘Volume’, predictably refers to the problems caused by the sheer size of the data involved and the specific tools and techniques required to analyse it.
‘Velocity’ is a recognition of the speed of update or change to the data which can often be considerable; and ‘Variety’ the vast differences in data and description often present in large datasets, which can make analysis extremely challenging – as Sheffield discovered.
Available sources of external data
Our benchmarking guide contains a wealth of detailed information on the main sources of external data available for organisations to draw upon, broken down by both its source and the type of data available. The guide, developed in association with HESA, also contains further information on the pros and cons of benchmarking an organisation’s performance against its peers and provides a practical, step-by-step approach to developing an organisation’s capability in this area.