JISC

The aim of this study was to identify measures and mechanisms for assessing the fairness and effectiveness of the application and selection process in admissions to higher education. It was intended that these measures could be used as part of the evaluation of the planned programme of technical demonstrator projects on the use of technology to support the admissions process.

Executive Summary

Fairness in admissions is an aim of all higher education institutions (HEIs), but is a complex concept which does not lend itself easily to technological mechanisms. Users of the admissions process: applicants and their schools and advisors, generally consider that the process is fair and efficient, and have few suggestions to offer to improve these.

There is however a mismatch between what HEIs consider to be fair, and what students and schools consider to be fair, especially around the use of contextual data provided during the applications process.

HEIs monitor admissions activity in an attempt to measure fairness, efficiency and effectiveness and examples of good practice in these areas were identified. There are few tools available to help them do this, but our analysis and modelling work suggests four broad areas, where further work would assist HEIs in the design and delivery of future systems that would address fairness issues:

Provision of statistical tools for use in the review of performance data, particularly at the time of end of cycle reviews

It is suggested that statistical analysis over several years’ data would enable admissions managers and systems designers to revise policies and re-design processes on the basis of robust conclusions about performance against widening participation targets, applications and intake trends, and actual outcomes.

Measurement of the effectiveness of widening participation activities in relation to outcomes

HEIs are committed to widening participation activities, in order to stimulate applications from specific groups of learners. However, it is not yet clear that many HEIs are able to measure the effectiveness of this activity in terms of long term outcomes rather than first year intake.

Technology-supported decision-making

It appears practical to create electronic tools to support decision-making, particularly for courses in the UCAS system, for which electronic interfaces with UCAS already exist. These tools could be integrated with UCAS systems, linking with existing data exchange mechanisms. While it is likely that there would be efficiency and effectiveness gains here, safeguards would be necessary, so that automatic processing rules do not mistakenly make decisions against those applicants for whom holistic assessment is essential.

Policy and process changes as a result of monitoring

Our research has shown a gap between the extensive monitoring of applications, offers and intake, and activity to change policies and processes as a result of this monitoring. This may be partly explained by an emphasis in HEIs on longer term change, rather than short term change. However, we believe that there may be potential for greater impact on policies and processes than admissions systems currently exhibit. Greater availability of contextual data in the future is likely to help with this agenda, but it is important that the provision of data is surrounded by good quality management and statistical tools, so that admissions staff can interpret the data and revise their policies and processes in a well-informed environment.

Points made in the report which refer to the UCAS scheme will be discussed with UCAS.

Download the full report below. This report is available electronically only.