The Management of ICT Investments
JISC is aware that traditional methods of evaluation and risk assessment have not been particularly useful when applied to the implementation of Information and Communications Technology (ICT) for teaching and learning. The need to increase investments and demonstrate their value along with other factors, such as increasing legislation, makes the task even more difficult for educational institutions.
Version 1.1 December 2005
Foreword
Many of us will have encountered managers who rely heavily on intuition, and may have wondered how they seem to get away with making strange decisions. Whilst following hunches or ‘flying by the seat on one’s pants’ may be necessary at times there is, in my experience, a case for assembling evidence to substantiate courses of action, particularly investment decisions.
Investment decisions concerning ICT initiatives are often difficult. Problems arise from a variety of sources including predispositions to follow a particular course of action regardless of the facts and risks involved, difficulty in identifying all the relevant costs and over-optimism regarding the perceived benefits. All too often ICT projects are delivered late, overspend and underachieve.
The work of JISC in developing methodologies for cost benefit analysis and risk management is therefore very welcome. The Insight Frameworks described in this document are the output of extensive work and they set out easy to follow practical processes.
The Cost Benefit Analysis Toolkit provides the means to arrive at well-informed and therefore better decisions. The decision to use the Toolkit is the first step towards success. The Risk Assessment Toolkit sets out a framework to identify and evaluate key risks and to determine how those risks are to be managed. This is an essential process and shortcuts are not advisable.
I welcome the production of these Toolkits and commend them to you.
Ray Corner
Director of Finance, University of Salford
Institutions in further and higher education rely heavily upon Information and Communications Technology (ICT) for the delivery of their teaching and research, the maintenance of their operational infrastructure and ‘back office’ functions, and their daily communication requirements.
In many cases, significant investments are being made in Managed or Virtual Learning Environments. Pressure to remain attractive and competitive to prospective students and staff leads to a wish to update and augment systems and equipment more frequently as user expectations and demands rise.
The pervasive and incremental nature of ICT investment has led to a requirement for new approaches to assess and evaluate the cost and benefits associated with planned investments. It is now widely recognised as essential to include ICT investments and management within the risk management framework.
JISC has therefore commissioned and published twoToolkits to assist institutions in these areas. They are summarised in this brief publication, and the full versions are freely available from the websites listed at the end. We hope that you will find them useful.
David House
Chair, JISC Organisational Support Committee
Introduction
JISC is aware that traditional methods of evaluation and risk assessment have not been particularly useful when applied to the implementation of Information and Communications Technology (ICT) for teaching and learning. The need to increase investments and demonstrate their value along with other factors, such as increasing legislation, makes the task even more difficult for educational institutions.
Now two decision-support toolkits have been created by JISC to make the decision-making process easier and more rewarding. The toolkits are based on sound research and are designed to support good governance in the complex environment in which educational institutions operate.
The Higher Education Funding Council for England (HEFCE) recognises the need for sound investment decision processes. The need for the use of a more defined approach to investments is also supported by the Adult Learning Inspectorate (ALI).
The toolkits include a comprehensive suite of materials to help you identify the best investment options and identify the risks. We have prepared the ground for you by creating a model you can adapt to your institution’s unique circumstances. Each toolkit provides a framework to aid the decision-making process and to build on existing processes, improving rather than replacing them.
Cost Benefit Analysis and Risk Assessment Toolkits
The Cost Benefit and Risk Assessment toolkits provide guidance for institution-wide teams led by senior managers as they seek to address the challenges that arise in a sector experiencing rapid change. They enable teams to undertake effective cost benefit analysis and risk assessment of investments in e-learning and ICT. Each toolkit includes a range of guidance sheets, software (including a simple costing model) and documentation for running workshops.They will help participants define objectives and find appropriate ways of understanding the outcomes from the
See the ‘Insight’ and ‘Risk’ toolkits
evaluation and risk assessment processes.
There are limited opportunities when conditions are right for an institution to apply this process. It is therefore vitally important that it is introduced at an appropriate moment during your institution’s planning cycle. By building on existing processes rather than replacing them, each framework will improve your institution’s approach to decision making.
Cost Benefit Analysis toolkit
Institutions need to evaluate the true impact of technology as investment continues to grow. Assessing the value to an institution rather than the cost is known to be a more useful measure.
The Cost Benefit Analysis Toolkit helps you do this by bringing together people from different departments, faculties and functions. The framework helps to structure the discussions and document the process, producing qualitative rather than quantitative results. It is vitally important that this exercise is introduced at an appropriate moment during your institution’s planning cycle. By improving existing decision-making processes it will be particularly helpful to larger and more complex institutions.
There are five key stages in the framework. The following diagram and key steps outline the process
Context
Before starting, however, set the context by defining the decision to be evaluated. Is this a set of potential options or a set of existing projects/investments? Identify the decisionmakers who should be involved in the discussions. The process will lead to a relative evaluation (ie a comparison between two or more options).
Step 1: Define and Prioritise Objectives
Identify the objectives of the investment and prioritise these in the context of the institution’s strategic objectives. Identify indicators that can be used to measure success against these objectives.
Step 2: Evaluate Benefits
Rate each option according to how well it addresses each objective. Their overall value provides a ‘score’ for the evaluation.
Step 3: Identify Costs
Decide on the costing methodology for collecting costs, and then identify the costs of each option.
Step 4: Compare Costs and Benefits
The costs and benefits of each option are expressed as numeric values for comparison on a graph.
Step 5: Modelling
As this process involves the application of subjective numeric values, the group needs to explore how changes to the numeric weightings for priority and evaluation affect the final outcome.
Risk Assessment Toolkit
As e-learning transforms the teaching and learning landscape, institutions need to identify their exposure to risks and plan how to deal with them. The toolkit will help your institution reduce risks as it manages the change.
The implementation of large-scale e-learning across your institution should involve a wide range of staff drawn from academic, teaching support, ICT and other functions right across the institution. Larger and more complex institutions may find this type of institution-wide change more challenging, but the risk assessment framework will make the decision-making process easier to manage. Again, it is vitally important that the process is introduced at an appropriate moment during the institution’s planning cycle.
The toolkit is based on sound research and the key stages are:
Context
The Risk Framework and associated tools are designed to be used when the e-learning strategy is being developed, reviewed or implemented.
Step 1: Identify a Facilitator
A facilitator approved by senior management is required to coordinate the use of the Risk Framework and tools.
Step 2: Identify the Key Stakeholders
The facilitator needs to identify appropriate representatives from across the institution with the aim of gathering the greatest range of views. These should therefore include sceptics.
Step 3: Plan the Risk Workshops
Four workshops plans have been designed to facilitate an open debate about e-learning objectives, the perceived risks and ways in which the risk can be managed or reduced. Institutions can ‘pick and mix’ from these workshops to suit their own requirements
Step 4: Encourage Ownership of the Process
Prior to running the workshops it is important that the facilitator encourages ‘buy-in’ to the risk review process.
Step 5: Run The Workshops
The objectives of the workshop should be made clear and the event should be presented in the context of the institution’s e-learning strategy. Participants are asked to identify the risks, risk management strategies and e-learning activities. Risk Worksheets prompt issues and questions and provide a checklist to help inform the sessions, and the facilitator must collect as much information as possible.
Step 6: Report and Follow-up Activities
The workshop should end with a clear indication of how the outputs will be fed into the future planning processes.
Where do I start?
There is a trend in higher education for institutions to demonstrate good governance through self assessment. This has already been occurring in Human Resources and Estates Management. The Cost Benefit and Risk Analysis toolkits provide some measure of self-assessment and JISC is currently developing a dedicated self-assessment toolkit for the management and governance of ICT systems; this should be available in 2006.
Effective use of the Cost Benefit Analysis and Risk Assessment toolkits depends on them being introduced at the right time during your institution’s planning cycle and equally importantly, that discussions include key representatives such as senior managers, academic practitioners, representatives from relevant support areas and, potentially, other stakeholders such as students.
The toolkits can be applied to proposed investments or used retrospectively. They include a comprehensive suite of downloadable materials to help you achieve a practical and reliable outcome. The materials can be downloaded from a number of websites and the URL addresses are given at the end of this document. The models are flexible, allowing you to adapt them to your institution’s unique circumstances.
The workshops cover the following themes:
1. Identifying Risks
2. Managing Risks
3. Constructing an Implementation Plan
4. Conducting a Risk Assessment
What lessons can be learned from other institutions?
The ‘Management of ICT Investments’ programme of work includes a report that is based on case studies from across the UK. ‘The Impact of Network Learning on Institutions’ describes what has been happening during the implementation of e-learning in FE and HE institutions across the UK. The work is comprehensively illustrated with case studies that highlight important factors that are useful for comparison. The report gives examples of where resources have been applied successfully or could have been applied more effectively. The case studies will give you a better understanding of the issues you need to manage by reflecting how other institutions have handled them.
The Impact of Networked Learning on Educational Institutions (INLEI)
New technologies have the potential for managing new modes of teaching, growing student numbers and the increasing diversity of student needs. The ‘Impact of Networked Learning on Educational Institutions’ (INLEI) project was set up to explore the impact of large-scale networked learning on institutions. The report reveals the progress made through the perceptions of staff at 20 institutions as they relate to key issues such as institutional development, infrastructure and support services, staffing and collaboration, learning, teaching and assessment, quality assurance, monitoring and evaluation procedures and access. The case studies show how institutions have managed the change, where they succeeded and where they met difficulties.
Links
References
Supporting documents
Organisations