In addition to the OCU BI maturity model we have also created an implementation model outlining the various stages of implementation that an organisation is likely to move through when progressing its BI capacity.
It provides a conceptual model to help understand and compartmentalise the steps required to realise a successful BI system implementation. Whilst still helping to enable you to understand your current position, its primary value is as a ‘journey planner’, designed to help you predict and prepare for some of the stages you will need to pass through on your way.
The model outlined below describes the different stages of BI implementation, as observed in interviews with HE and FE managers, interviews and discussions with vendors, published research and case studies, and experience in other sectors.
No part of this model is intended to be prescriptive; the model has no force or authority apart from its usefulness to individual projects and practitioners.
In order to further flesh out each stage, we have also constructed six composite case studies which describe some of the various routes available to achieving successful BI (see section on routes to implementation). These composite case studies have been structured around the six stages of implementation described within this model.
Each of the eleven Jisc BI projects also considered their current position at the outset of their work and charted the progress they made through the prism of this model. An observation common to many of the projects was that it is often more instructive to apply this model to specific areas of an organisation (a department, function, system, dataset or reporting requirement etc) rather than attempt to define a single overall statement of which stage of implementation best describes the organisation as a whole.
It is inevitable that practice will vary considerably from area to area and it is often far more useful to narrow the area of investigation to uncover such differences and compare pockets of practice than it is to seek to shoehorn the entire organisation into a single instance. The following section has further reflections on the experience of these projects in using the implementation model.
Characteristics of each stage
Each stage in the journey to the successful implementation of mature BI systems is described briefly below. There is a risk that these descriptions may verge on caricature, but each feature described has been observed or reported at least once in the course of our research.
In particular, the order of stages 2 and 3 varies: sometimes (often in successful BI projects) improvements to corporate information (stage 2) are at least begun before a BI Project (stage 3); however there are cases where the BI Project (stage 3) precedes and facilitates any Improvements to Corporate Information (stage 2).
Stage 2 (Coherent Information: centrally reliable, locally responsible) is subdivided into several sub-stages. The sub-stages are useful because most successful BI projects need to address each sub-stage, but they often do them in a different order and at different times during the project. They express discrete, separable elements of the stage of BI maturity in which information is brought under control and given appropriate attention.
Planning or reporting meetings may spend up to 2/3 of their time wrangling about data quality and disputing specific data items. Gathering coordinated data (e.g. the cost of a new course, the income generated per faculty member, the profile of students who do not complete their courses) is a difficult manual process. In this stage the governance structures for information management, risk management and KPIs may not yet be established.
1. Traditional information sources: fragmented and mistrusted
Planning or reporting meetings may spend up to 2/3 of their time wrangling about data quality and disputing specific data items. Gathering coordinated data (e.g. the cost of a new course, the income generated per faculty member, the profile of students who do not complete their courses) is a difficult manual process.
In this stage the governance structures for information management, risk management and KPIs may not yet be established.
2. Coherent information: centrally reliable, locally responsible
a. Build governance structures
There do not seem to be any cases where an HE or FE organisation has achieved stages 2 or 3 without high level support for improved information management, data quality, strategic thinking about risks and KPIs, and evidence based decision-making. This support should be from the Vice-Chancellor, a Pro-Vice Chancellor, the Principal, and/or a Deputy Principal. Involvement of academic and support staff at all levels is required as well, but without this leadership and support from the top, it is unlikely that any BI system will have enough impetus to succeed.
b. Replace systems, if necessary
Some central systems may not be fit for purpose; they may be old, overly bespoke, or unsupported. The next step is to identify systems which are not for purpose (as stand-alone) systems and to replace them. A central BI system will find it difficult to succeed if it is trying to gather data from ill-functioning sources.
c. Make local staff responsible for the data they supply
HE and FE organisations gather data from a wide variety of internal (and external) sources. The initial data must be correct. There is an unhelpful history of blaming the (computer) systems for data problems.
Each individual person, and each team, putting data into any organisational system must be personally and specifically responsible for the data quality. Nobody should be able to complain about the quality or accuracy of data for which they were responsible, and any complaint about data input by another must be a direct claim that that other person or team has not done their duty.
This personal responsibility must include support staff (finance, HR, administrators, estates) and academic staff.
Some organisations have found data workshops to be helpful. In these, Management Information staff explain why data are needed (to report to external authorities like HEFCE, and for internal planning), what different data mean, and how data can be collected accurately. Staff can discuss issues with reporting, data, and systems. An informed, educated and cooperative community produces better data.
d. Support data entry with validation
In parallel with 2c, systems should be checked to make sure that they promote good data quality. Where possible, entries should be selected from pick lists; entries like dates should be taken from calendars or have their format checked by the system. Data should be entered once only, if possible.
Examine data and see what the most common errors are: then build aids into the system to prevent or catch these errors.
e. Require that the central systems be used
The final step is to require that all levels of the organisation use the data from the central systems. Of course this requires that the central systems be available to all parts of the organisation and that the data needed locally be obtainable from the central system. For these reasons, this step may not be possible until after a BI system is introduced.
However, it is important to realise that local ad hoc systems are not just ways of consuming staff time. These local spreadsheets or little databases prevent parts of the organisation from being committed to supporting the central data systems. The local collections take up the local attention and effort, and leave the central systems with second-class support and as the targets of blame.
3. BI project: selecting an approach and a vendor
- Select a system that replaces some or all of your existing systems and provides a dashboard linked to the remaining systems (‘single central system’ approach)
- Build a data warehouse with a system that displays information from it (‘data warehousing’ approach)
- Build your own BI system (‘BI as IT project’ approach)
- Select a system that can see all your existing data and display them in a dashboard (‘varied vendor’ approach)
It is also important to decide on the level of services you will need from your system vendor or from external consultants. You can select a system which your own staff can configure, link to data sources, and modify as your needs change. You can also select a system where essentially any change is made by an external expert.
Bespoke and home-made systems are possible, and are used by some organisations. There are also a number of open-source and commercial systems offering different approaches to BI.
While there is a range of prices, and levels of support, cost should not be a reason for avoiding a BI system. BI systems can be introduced for about the cost of one full time staff member, and a good BI system will almost always save the equivalent of one full time staff member in time needed to collect data, correlate different systems, prepare reports, and answer strategic questions from senior management.
4. Initial BI system
When a BI system is selected and installed, most successful projects limit the types of data covered in the initial phase. Sometimes this is financial and personnel data, other times student data are the priority. The important aspect is to get a limited system up and working, and to obtain visible benefits from it.
Some projects go for a BI system that will link to all of their data sources from day one. This may be riskier and more expensive, but it can be a successful strategy.
In either case, make sure the initial system is configured to suit your organisation’s style and needs, and make sure that some benefits are realised and are visible to the whole community.
5. Growing BI coverage and involvement
No BI system should be seen as something you install and forget about. Once a BI system is available, other parts of the organisation (service and support departments, academic departments, research teams) will find ways that they could use the BI system to save time, improve data gathering and interpretation, and improve services.
Provision should be made for the BI system to evolve and grow. Eventually it will probably be providing data to senior management, to all other levels of organisational staff, to students, to prospective students and to the public.
6. Reliable predictions and forecasting
HE and FE organisations are constantly changing, in response to society’s changes, to government initiatives, and to the results of their own research.
The ultimate value of a BI system is that it allows an organisation to understand its past, its present situation, and to predict the effects of likely futures. A mature BI system can show the likely financial and human results of closing a course, opening a new course, increasing the proportion of part-time students, refurbishing a building, and so on.
The UK higher education sector is becoming increasingly aware of the potential value of using ‘analytics’ data for modelling and forecasting and a successful, mature BI implementation should represent an important component part of this capability.
What is ‘analytics’?
Analytics is the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data.
Adam Cooper, 2012
Analytics is notable in that it is a headline grabbing trend in many domains, but has also been around for a long time under various other labels. One consequence of that longevity is that there is a bewildering array of tools available that can support an analytics process in some way.
Some of the Jisc BI projects also actively engaged in this area, in particular the Open University and the University of Bedfordshire, both of whom looked to utilise activity data generated as a by-product of learner interaction with different technologies to help identify at an early stage those learners who may be at an increased risk of failing in their studies.