By enabling data sharing between internal IT systems and between one organisation and another, APIs facilitate not only the dissemination of knowledge but also the design of innovative online services for students and staff.
By removing the need to repeat laborious data entry tasks, they save staff time and increase organisational efficiency.
This report explains these and other key benefits of APIs, demonstrates these benefits with case studies and contextualised examples, and suggests some steps to start using APIs both as a technology and as a business change methodology. The aim throughout is to highlight the key opportunities and challenges facing UK further and higher education organisations in achieving good practice for implementing APIs, and to provide real world case studies and actionable advice as to how to use APIs to further typical institutional goals. The writing is non-technical and is aimed at senior university and college managers.
Continue reading Section 1 for a quick overview, Section 2 for case studies to refer to, Section 3 for contextualised benefits, and Section 4 for action points. Look out for the persuading your organisation notes for quick tips.
What is an API?
An Application Programming Interface (API) allows one piece of software to make use of the functionality of, or data available to, another. It provides a consistent, programmatic method for accessing a resource; it is simply a structured way of exposing functionality.
APIs are essential tools for high quality system architectures, and the concept of APIs can be applied for numerous advantages.
An API is a machine-to-machine interface rather than a user interface. It allows developers to access the functionality of other software modules through well-defined data structures...
Marieke Guy, Jisc Good APIs report
What can an API do?
APIs make it easy to efficiently share data and processes. So in the context of higher education and research institutions - where the focus of work and stated mission is akin to "the creation, dissemination and curation of knowledge" (the stated mission of the University of Edinburgh, for example) - APIs are an ideal tool for increasing awareness of work done and services offered, in an efficient manner.
In order to remain competitive a university must maintain a strong brand, and this brand is primarily predicated on presentation of informational assets. For these assets to be effective they must be easily shared both within the institution as well as via the wider social web, and an API is the ideal way of achieving this (illustrated in Figure 1).
Read section 2 for more information.
What are the benefits?
Interoperability is a key requirement for anyone wishing to benefit from the social nature of the web as a driver of growth and investment; this includes universities, colleges, research groups and businesses. An institution that offers an information infrastructure on which others can build is held in high regard in any scholarly community; APIs provide the means to do this.
Successful implementation of APIs across a digital estate results in greater flexibility and improved ability to efficiently present outputs as useful and re-usable artefacts, creating an opportunity to enhance an institution's reputation or brand within and beyond the higher education community; in effect, a good API can be used to create an advertisement for the finest examples of an institution's wares. But it is more than just an advertisement - the API enables use and re-use; it is a tool by which we can disseminate knowledge.
Read section 3 for more information.
OK, how do we do it?
The technical solutions for implementing good APIs have long been available, and there are many mature examples of tried and tested API implementations to learn from. What is required in addition to this is the correct approach to implementing an appropriate solution, investment in the right people and skills, and the understanding that the successful use of APIs is fast becoming a fundamental best practice for successful and efficient business in the modern knowledge economy.
Read section 4 for more information.
Figure 1. How to jump the information gap - utilise APIs to avoid costly repetition of laborious tasks, and realise the benefits of efficiently sharing data with consumers both locally and across the world
There are many great examples of APIs in everyday use, both within and outwith higher education and research. A lot can be learned by examining a few of these.
University of Lincoln phone directory
Lincoln developed a staff directory that took advantage of APIs to create a client-facing corporate service based on existing data. On account of their work over recent years to develop a large pool of API-exposed datasets, they were able to prototype a new staff directory in less than 48 hours, using APIs to pull data from disparate sources within the institution.
Rather than requiring a time-consuming and costly approval process to build a system from the ground up and manually populate it with data, they were able to quickly combine existing infrastructure and present a search across a number of datasets. The data was also exposed as Linked Data and was contributed to data.ac.uk, further increasing exposure to and usability by external developers.
Several existing APIs were used to pull information from a variety of internal sources such as local LDAP data, EPrints data and staff profiles from local WordPress instances. This allowed aggregation of various sorts of data such as student demographics, room location and usage, course and bibliographic data. It is now all easily available via a central resource that students, staff, and others can use as supporting data for new projects; in fact, just within the scope of writing this report, we were able to create an interactive visualisation by querying the phone directory API. (see Figure 2)
University of British Columbia
The University of British Columbia (UBC) implemented APIs to improve their Student Information Services (SIS), to cope with their administrative reporting needs and improve data security.
Following an incident where a UBC laptop containing sensitive data was left on a train in Vancouver, senior management called for better control over the the transfer of data. As a result, the Enterprise Architecture group pushed to overhaul the existing business processes and IT systems to utilise APIs, advocating and providing support for improved data management practices that could reduce the risk of data loss.
After detailed business process analysis, UBC was able to use APIs to streamline their systems and automate many regular reporting tasks. In combination with access rights management, this allowed for improved control and higher efficiency for data sharing activities.
As an example, requests for lists of students graduating in November - which previously took as long as a week due to reliance on generation, sharing and combination of over a thousand separate reports - could be securely queried and delivered the same day, with significantly reduced overheads and less opportunity for manual error.
The university is now working to integrate with external partners; for example, all UBC students get a monthly transit pass for local transport, which requires sharing enrolment data with the Transit Authority and transport companies; direct access can be now be offered securely and efficiently via well managed APIs, saving administration costs on both sides.
Jisc Discovery Programme examples
Jisc have funded numerous projects that have produced APIs. Its recent Discovery Programme set out to "create a metadata ecology to support better access to vital collections data in libraries, archives and museums and facilitate new services for UK education and research". Many of the projects under the programme completed in Summer 2012, and most have published APIs with examples and documentation, for the express purpose of exposing large collections of data for use and re-use within and beyond their communities. Short descriptions and links to some of the project blogs and/or final reports are now available:
- ServiceCORE is an Open University project that "aims to develop a new nation-wide aggregation service that will improve the discovery of research publications stored across British Open Access repositories"
- Bricolage at the University of Bristol provides APIs and demonstrations such as building a browser-based mapping tool for their Geology collection
- The Pelagios project at Open University is developing online resources that refer to places in the Ancient World. They provide discovery and visualisation examples
- The Open Biblio 2 project hosted by Cambridge University Library demonstrates the value of open access to bibliographic metadata, and exposes numerous collections via JSON APIs.
As with many web startups, Mendeley, which provides bibliographic and networking services for researchers, have made APIs a fundamental part of their business service. Their API was released with exceptional fanfare, including a competition with considerable prizes for developers to suggest new applications and features. One of the judges, a publisher from the technical industry, said of the competition:
I always tell developers to work on stuff that matters. It's time to stretch beyond the consumer Internet and what better place to focus than on furthering the cutting edges of science?
Tim O'Reilly, Founder, O'Reilly Media - as quoted on a Mendeley binary battle blog post
Mendeley's API provides access to Authors, Papers, Publications and Tags as core resource types. This list may be similar to that of the core resources of some universities or colleges. It may therefore be possible for a university or college to expose similar APIs for use by their staff and students (and beyond) to do new and interesting things that benefit themselves, their peers, their institute, or the community as a whole.
Eventbrite is one of the most popular online events, bookings and facilities management apps. Their API demonstrates some of the advantages of a programmatic approach to organising information, in that it enables additional applications to integrate bookings data to further enhance functionality. Many universities could benefit by considering how to organise and expose their information in a similar fashion.
Exposing information about the level of engagement with events such as public lectures via an API could provide a useful and re-usable resource when developing other systems. For example a combination of room bookings data, lecture/event attendee data, and institutional marketing information, could be used to create a reporting system that informs the institution of new business opportunities, in the form of delivering desired courses or research programmes.
Justifying investment requires clear insight into how APIs will be implemented and what value they bring to an organisation's core mission.
Here are some examples that contextualise the key benefits of efficient information accessibility and usability.
Use and re-use of institutional output on a global scale
Geographical location used to be a defining factor for any institution; but now, staff and funding readily cross international boundaries, and research impact (on the basis of citation count - see Figure 3) can be demonstrated to increase as international collaboration increases. A platform that binds research efforts to core infrastructure, thereby helping to distinguish an institution, becomes increasingly valuable in the global competition for funding.
A key advantage of the automated nature of an API is the ability to deal with large numbers of transactions without the need for comparable increase in administrative costs. However at present many universities work with data in ways that hinder efficiency, by using stop-gap measures that attempt to digitalise paper processes rather than utilising digital systems to their full advantage. Examples include:
- dumping data from databases then manually editing
- formatting data as read only files, such as PDF
- transferring data via email attachment
- manually updating multiple versions of data in spreadsheet form
- using multiple sources to manually build monthly reports
While these methods may solve the immediate problem of getting the data to where it is needed, they rarely address deeper issues. The underlying data is very often not exposed in a useful format, and as demand increases, so does the process cost. Adoption of APIs is a solution to this problem.
Automate laborious tasks, reduce costs
In the 1920s the League of Nations came together to address an issue brought about by the growth of the rail network: with the interface between nations lacking a standard identity mechanism, processing traveller documentation was becoming increasingly laborious. The solution was found in 1921 when the League of Nations successfully agreed the modern passport system, which remains in use today.
The process for passing information across the borders of finance, HR, learning resources, library, departmental and other systems is often a labour-intensive chore that soaks up resources. Like nation-states in the 1900s, universities must address the issue of how data can easily but carefully cross the borders of their own systems.
The role of an organising body (such as Jisc) is vital to encourage cost efficiency by demonstrating common methods for passing data across system borders. The introduction of APIs, like that of a standard passport, enables us to agree what information can be shared and - more importantly - the process by which we can reliably share it.
When designing APIs to facilitate the flow of data between systems, the cost savings may at first appear difficult to assess - an API will often replace a series of manual steps whose individual simplicity obfuscates the overall intensity of resource consumption.
Typical steps that may occur every time data must cross a departmental or institutional border are represented in Figure 5; the spent labour resources of producing a single data product are of course dependent on the process used - but nevertheless are very high, very repetitive, and very prone to error. In comparison, Figure 6 demonstrates the same activity utilising a well-architected API - the initial labour costs are higher due to the need to develop the API, but subsequent re-use is far more efficient.
Figure 5. A manual data sharing workflow; low unit cost, high overall cost
Figure 6. An API-driven data sharing workflow; higher initial cost, lower overall institutional and sector cost, lower risk of manual error
Attract and retain students and staff with improved digital reputation
Whilst the global nature of communications technology has advanced academic research and learning, it has also driven a huge change in the core services that institutions must provide as part of their digital infrastructure which, in turn, has changed how institutional brands are perceived. As illustrated in Figure 7, a 2010 study into the First Year Experience in Australian Universities: Findings from 1994 to 2009 from the University of Melbourne shows that Australian students spend more time researching on the web when applying to university; in addition, research into the economic value of a brand shows that Americans are more likely to choose a brand they already know, whilst 50% of customers will try new products from brands they already trust; and thirdly, recent Jisc reports show that British students who continue and complete their courses do more web-based study than those that drop out.
Figure 7. Statistics extracted from a selection of Jisc, University of Melbourne, and Hodgson Meyers studies
In light of this, modern institutions require a platform for learning and research that is both effective and clearly differentiated from competitors. This means developing a brand that is recognisable, and being able to deliver a high quality of service to customers. Research by Upton and Staats entitled Radically Simple IT, for the Harvard Business Review, points to the fact that a modular approach to systems architecture (which is achieved via APIs) is vital to shift IT from being a simple provider for existing operations to a platform for new services and new business.
With a technology platform that is able to leverage data and media assets, the information systems of the entire university come to bear on brand recognition across a multitude of channels. The advantages of APIs in this context are two-fold: they encourage innovation by allowing staff, students and external collaborators to easily and securely access and manipulate data in innovative ways; and they help to prevent reputational damage by introducing good practice to data management and thus providing opportunities to reduce risk.
There are key issues to consider when implementing APIs. in addition to approaching the problem with an appropriate frame of mind, some investigation into how similar problems are being solved by peers must be tempered with careful attention to the specifics of the problem within a particular context. Fortunately, there is a wealth of information available to assist with this process.
Re-use tried and tested methods
One of the great advantages of the wider web industry leading the way when it comes to APIs is that there are more than enough examples not only to learn from, but also to copy. There are several different types of APIs, but one dominant method is "Resource Oriented Architecture" - also known by developers as "Representational State Transfer" or REST, as presented by Roy Fielding in his PhD dissertation.
The critical aspect of this from a managerial perspective is understanding data resources - such as research outputs like theses or experimental data, or simply author or location information - and their potential uses. Consider which types of resources are important to your organisation, then investigate how similar resources are exposed via APIs in our examples in section 2, or by big players in the social web industry such as those listed in Table 1.
Given that an API can be subject to the common attacks seen on the web, security is a valid consideration. However, an API is also capable of being secured in exactly the same way as normal web pages - so there are plenty of well-known security measures that can be put in place. The detail of these is beyond the scope of this report, but the categories to consider are:
Secure communication. Transport Layer Security (TLS), as seen on secure "https" or "padlocked" websites, provides for encrypted communication between the consumer and provider.
Authorisation. This is about determining what the authenticated consumer of your API is allowed to do. It is possible to limit the number of requests a consumer can make, or provide read-only access to particular consumers, for example.
API implementations of relevance to further and higher education functions
Administrative (Finance, HR, Admissions, Facilities etc)
- PayPal handles payment transactions. They moved from a SOAP to their more modern REST API, while still maintaining security and functionality.
- Xero provides an API-integrated accounts management system.
- Salesforce is a Client (or Contact) Relationship Management System. A core resource type for Salesforce is a "contact", as defined in their REST API
- The Twilio API defines how calls and text messages can be accessed. With universities looking to implement SMS messaging to all their students and staff, this API's clearly defined resource types may be of interest.
Teaching and learning
YouTube uses many basic resources that are very similar to most course management systems, such as video, channel, playlist, events, comments, subscriptions & inbox.
Research publication systems (EPrints, DSpace, Fedora, Equella) use a common API called SWORD. It is based on the Atom Publishing Protocol (used by blogging platforms like WordPress and Blogger). SWORD is an example of what can be achieved in cost savings for an entire sector if common ways or working are agreed.
The cost of repeatedly performing administrative tasks can mount quickly despite low unit cost, whereas API maintenance cost is generally low after initial development, despite the required investment in skilled employees or outsourcing to skilled providers. As Jisc's DevCSI project relates, the ability to recognise the business needs and opportunities for integrating APIs remains the job of local expert trained staff; a disengaged developer may not understand the use case or value proposition for why a contacts database could benefit from communication with the course management system (so a student can text their lecturer about study questions, for example).
Invest in skilled people
Some vendors may be less likely to offer standards-based APIs by default, as it may be fiscally beneficial to "lock in" customers. This approach is opposed to the idea of having multiple systems each with clear logic for the service they provide and then coupling those systems together. A perfect example of this is the Virtual Learning Environment which has grown to be dozens of systems all bundled into one system. However, vendors will accede to customer demand, so make it clear that a standards-based API is a non-negotiable requirement.
Although the potential exists for institutions to make significant savings via API implementations, careful consideration must be taken to ensure business processes are properly replicated - and changed, where necessary. This requires a team that is oriented and skilled towards development activities; a locally maintained team will provide better cost per delivery and may have better specific domain knowledge, or alternatively an outsourced team can provide development resource scalability - although usually at a higher unit cost.
Any team taking on this sort of development work must be capable of interacting with people and processes across the institution, and of forging agreements across departmental borders, so that data can be shared. This is a technology driven business integration task. As APIs help us solve border problems, it is important to include stakeholders from roles with a perspective on long term strategy as well as individuals with an understanding of specific resource-oriented requirements, from each department involved in the data sharing activity.
Start with a pilot project
There is a perfectly valid school of thought that APIs require a strong centralised IT strategy, and indeed if this centralised strategy can be achieved then it will prove valuable. However, the culture of universities and colleges are often collegiate in their organisation. Where possible, choose a pilot project with one simple clear goal to help two departments share their functions or datasets easily together. From there, it will be possible to identify further uses for the data and to demonstrate further previously inconceivable efficiency gains, adding fuel to the fire of interest for the next API development project. Key business processes may operate in an entirely different way once automated, so mitigate any risks arising from these changes by keeping each change small and simple. Consider for each change the requirements and impact of security, demand, authentication, authorisation, and accountability.
Further and higher education organisations will naturally continue to evolve, and must eventually interoperate with and contribute to the various streams of data a person (such as a student) has about themselves. A personalised API via which a student accesses all their academic information is yet to be created, and there is bound to be a niche problem somewhere within or between the layers illustrated in Figure 8 that people at your institution are desperate to have a solution for. Identify the problem, and start from there.
Jisc has funded several API implementation projects for which budget information could be made available, along with helpful hints and assistance from the people involved. Review the aforementioned Jisc Discovery projects and contact people that have experience in a relevant area, or seek suggestions from the people involved in producing this report, or from technical people within your own institution.
In all projects, be sure to publicly document your progress as widely as possible; this will ensure maximum engagement and user buy-in, even where the project is having difficulty. Being honest and open about any difficulties will encourage people to assist in solving them rather than becoming disillusioned with the enterprise. Wider discussion of project successes will provide valuable evidence for others considering future development projects.
Commissioned by Jisc on behalf of Higher Education Funding Council for England.
Our thanks to Andy McGregor and our beta readers: Joss Winn (Lincoln University), Paul Walk (UKOLN), Richard Jones (Cottage Labs), Stuart Brown (Open University), John Norman (Cambridge University), Max Whitney (New York University)
Please cite this report as:
Jisc (2012): The Advantages of APIs.