A variety of system implementation issues were faced within the eleven projects, leading to a number of strategies being employed to overcome them. This section explores some of the key findings to emerge, and the strategies employed, in this area.
1. A pilot is a useful way of confirming that a system is functioning as users expect while limiting the exposure of an untried system
A very common strategy adopted by many of the projects was to avoid an organisation wide ‘big bang’ approach by piloting the system first in a subunit of the university, usually within an individual department, school or faculty. This approach had the advantages of both managing expectations and keeping the feedback focused and manageable while at the same time having the system used by ‘real’ users.
One of the principal advantages of a pilot in advance of a general launch is that it can be more easily established whether or not the solution is in fact solving the problem that the users are experiencing as anticipated. It also provides an executive sponsor and/or champion with an opportunity to show their worth if the system requires changes in business practices while at the same time limiting the risk to the business. Often there are two facets for the pilot to address:
- Have the project staff understood the business problem and addressed it correctly?
- Does the solution they have developed present the required information in an accessible, user-friendly format?
A successful pilot should help provide further impetus to the project, hopefully leading to a heightened level of expectation in the institution as a whole that the system will bring the hoped for benefits. This is, of course, all to the positive, but may well lead to its own issues of how to manage this increased expectation. The more successful the pilot, the more you can expect to have to answer the “when can I have one of those?” type questions and the greater the need to have a realistic and achievable plan up your sleeve.
2. A pilot can highlight unforeseen complications that can be addressed or managed in a controlled manner
For some projects such as that at the University of Glasgow, the piloting of their application raised some challenging questions. The project experienced some tension between formalised coding structures for research activity such as the Joint Academic Coding System (JACS) codes and the perceptions of the researchers and how they viewed their own work, which was at times much less structured.
The pilot highlighted that these tensions were more complex than had been thought and that resolving them would not be a simple task. An ongoing question for the application would therefore be how to manage users’ own personal lexicons while at the same time maintaining a formal, universal coding structure and reconciling the two.
3. Like all other projects, BI developments have to balance time and resources against ever-increasing user expectation
Most projects will experience some level of constraint in their deliverables due to both time and developer resource limitations. In a project, where the budget is set at the outset, it is often the case that there is a fine balance between the fundable resource and the demands of the users (and project sponsors) for solutions to set deadlines. Projects that considered this issue very early on in their planning generally fared better than those who had to confront this issue at a later, and often less convenient, time.
4. The aphorism ‘garbage in, garbage out’ is particularly applicable to BI systems
Throughout the funded projects, there were many examples of organisations discovering that their BI project illuminated shortcomings in data capture and quality. Often these shortcomings had been known about or suspected for some time but the inclusion of the data in BI outputs resulted in the organisations having to make significant improvements in underlying data capture processes before the business intelligence system could be demonstrated to be robust.
For example, the Open University found that their data sources were difficult to locate and had a variety of different owners each with their own priorities; whereas the University of Central Lancashire had a similar issue but in addition discovered that the format of the data they required was inconsistent and needed a detailed analysis before it could be used. There were relatively few instances of data owners being uncooperative but where these did occur the disruption to the project was significant.
5. Projects that require or produce large datasets need to consider how these will be managed and stored
An issue for both the University of Bedfordshire and the Open University was the potentially very large volume of the data that they were collecting and processing. In both cases the project had to consider how to manage large volumes of data and how this would impact on their hardware. Bedfordshire actively considered a cloud solution and may still use that technology as a way of keeping their historical datasets online.
6. Organisational and local measures may not always be the same. BI projects need to consider the degree to which they will accommodate differing interpretations and analyses of their data
Another issue for the University of Bedfordshire was the need to augment the standard ‘definitive’ set of data with facilities that allowed users to explore their data in a variety of ways according to their own priorities. To achieve this the project had to support a much freer use of parameters and graphical outputs than they had originally anticipated.
7. Open source and open data can be very useful but do not underestimate the time and skill required to make good use of them
Both the University of Manchester and the University of Sheffield opted for solutions at least partially based on open source and open data concepts. Sheffield came to the conclusion that there were severe limitations in using generic tools to manipulate open data. Manchester felt that development in open source took longer than using some of the proprietary tools that are available but this was offset by other and well known advantages of open source.