JISC

There are two levels of evaluation that will occur in the Repositories and Preservation programme.

• Project evaluations - carried out by the project manager and should prove the worth of the project and draw out value.
• Programme evaluations - carried out by JISC programme managers and should assess whether the programme has realised the benefits it was designed to achieve.

The programme evaluation will rely on a variety of sources of information, including the project evaluation reports. This does not mean that there will be extra evaluation responsibilities for project managers, it just means that JISC programme managers will make sure that the project evaluations are aligned with the programme evaluation and that project managers are aware of how their project is contributing to the programme evaluation.

Project Evaluation

Evaluation is a vital part of the project plan, it is the mechanism which proves what the project has contributed. There will be training and support for evaluation available to all project managers in the Repositories and Preservation programme.

There is an evaluation handbook available to all project and programme managers for the Repositories and Preservation Programme. This handbook has been produced by Glenaffric Ltd who have been employed by JISC to help with programme evaluation. The link to the handbook can be found at the bottom of the page.

There will be further evaluation activities available to project managers involved in the Repositories and Preservation Programme. These activities are likely to take to form of sessions at the programme meeting and workshops. Details of upcoming activities can be found toward the bottom of this page in the section labeled: Training and Activities.

JISC programme managers will be happy to answer any questions about evaluation, whether it refers to project or programme evaluation.

Programme Evaluation

Overview

This overview of the Repositories and Preservation Programme evaluation just focuses on the blueprint and benefits section of the plan and is designed to give an insight into how JISC is approaching evaluation for this programme. The full plan is currently at the JISC internal planning stage. It will be released for public consumption when we have it in a stable state.

JISC is using the Managing Successful Programmes (MSP) methodology to manage the Repositories and Preservation Programme. The key focus of this approach is that programmes are initiated to realise benefits through change. MSP does not include an evaluation methodology so JISC employed Glenaffric Ltd to design an evaluation approach for the programme. The link between these two approaches is that both focus on the realisation of benefits.

Blueprint

The blueprint is a useful tool in evaluation planning. It is a picture of the future that the programme should produce. As such, it helps to focus the evaluation on assessing whether the programme has been a success in achieving its outcome. The blueprint that is being used for the Repositories and Preservation programme can be expressed in words and as a picture.

At the end of this Programme:

• Every UK institution will have repository capability to support research and learning: MORE REPOSITORIES
• Repositories will be embedded into workflows: EMBEDDED INTO WORKFLOWS
• Repository services will be interoperating to avoid duplication of effort, and to increase assess to content: INTEROPERABLE SERVICES
• Repository services will be integrated into the Information Environment: INTEGRATION INTO THE INFORMATION ENVIRONMENT
• Better support and better methods for sharing information will be available: MORE SUPPORT AND BETTER INFORMATION SHARING
• Institutions will be better equipped and skilled to manage their information assets: INCREASED SKILLS
• A greater proportion of institutional assets will be preserved for future access: MORE PRESERVED CONTENT
• The UK academic community will have increased access to a wider body of content produced by their peers: INCREASED ACCESS

 

Benefits

The benefits that the programme hopes to achieve are the backbone of the evaluation. The evaluation plan is based around measuring whether the benefits have been achieved. These benefits are:

Section 1 - Development of repositories:

1a Increased availability of content for learning and research.
1b Improved institutional infrastructure.
1c Better understanding of how repositories can be used for learning and research as well links to other systems.

Section 2 - Support for the repository community

2a Improved skills and knowledge within the sector.
2b Better sharing of information and advice based on JISC activities and other relevant initiatives.
2c Increased capacity to develop and use content into the future.

Section 3 - Resource discovery

3a Research and learning outputs are more easily accessible by subject and other appropriate means.
3b motivation for the deposit of content is created
3c increased impact of institutional digital assets

Section 4 - Standards

4a Better and more widely available common standards for content therefore offering improved interoperability within a heterogeneous environment.
4b Improved accessibility and longevity of digital content.

Section 5 - Tools and innovation

5a More effective methods, practices and software to support a network of repositories and their use for learning and research as well as preservation.
5b Tools to help improve and embed repositories within universities and colleges.
5c Better understanding of challenging issues and how they can be addressed, for example rights management and the economics of digital repositories.
5d Use cases and tools to populate the e-Framework.

Section 6 - Shared infrastructure services

6a Identification of shared infrastructure to support resource discovery and curation.
6b More efficient and interoperable resources and services based on a suite of shared infrastructure services.
6c Improved consistency between services.
6d Population of the e-Framework in the area of resource discovery and curation.

Section 7 - Preservation

7a Better understanding of what technologies and organisations need to do throughout the information lifecycle.
7b A sound basis for the development of an effective repository and preservation infrastructure.

Training and Activities

November 2007

• The gathering of the baseline data for the evaluation will be complete by the end of October. A baseline is a snapshot of the environment in which the programme is operating, in this case the repositories and preservation environment in the UK. We will use the baseline to assess, through subsequent measurement, what effect the programme has had on the repositories and preservation environment.
• 28th November 2007 - The November Repositories and Preservation Programme Meeting will have a session which will address the engagement of the programme's projects with the evaluation plan.

January 2008

• It is possible that evaluation workshops will be put on for projects in January 2008. The exact nature and date of these workshops is yet to be determined. the workshops will be run by Glenaffric.