Project planning: Evaluation plan
The project plan template has a table to help you develop your evaluation plans. Plan evaluation in consultation with the programme manager. Also report on evaluation activities in progress reports and the final report.
Think about what the project is aiming to achieve and discuss evaluation factors with your programme manager. Factors to consider include:
- Stakeholder engagement
- Outcomes and impacts
- Effectiveness of the project
For the project, focus on whether the project was effective and achieved its objectives. For project outputs, focus on whether the outputs are useful, meet user needs and perform well.
Focus on questions that really need to be answered to demonstrate success. Think about what stakeholders want to know. Make sure that the questions can be answered unambiguously.
Formative Evaluation questions:
- Have milestones been met on schedule?
- What is holding up progress?
- What should we do to correct this?
- Is project management effective?
- Are stakeholders on board?
- Do they agree with interim findings?
- Is our dissemination effective?
Summative Evaluation questions:
- Have objectives been met?
- Have outcomes been achieved?
- What are the key findings?
- What impact did the project have?
- What benefits are there for stakeholders?
- Was our approach effective?
- What lessons have we learned?
- What would we do differently?
Formative evaluation let you reflect on what you’ve done so far, what’s going well and what you could do to change or improve things. Formative evaluation is also used to improve the programme so tell the programme manager, personally or in progress reports, what you feel could be improved. Other projects may be saying similar things, and the programme manager can decide what action to take at programme level.
Summative Evaluation will demonstrate whether you’ve achieved your aims and objectives, the work was useful, and there are benefits for the community but remember there may be circumstances beyond your control that affect what the project can achieve. Demonstrating that the work was useful and has benefits for the community relates to sustainability so if you plan to carry the work forward, include evaluation results in your business plan.
Evaluation methods are well-documented, so even if you haven’t conducted an evaluation before you should find sufficient information to choose appropriate methods and use them successfully.
Quantitative methods could include:
- Usage logs
- Web server logs
Qualitative methods could include:
- Focus groups
- Observation Peer review
Whatever methods used, involve stakeholders, to increase their commitment to the project, confidence in the results, and likelihood they will act on the findings. Involve user to increase the likelihood of them using of outputs. You may choose to involve an expert to help plan studies or advise on analysing results. When planning evaluation of the project, it’s important to get independent views. Projects collecting personal data during evaluations should ensure that data protection policies are followed.
For project outputs, performance indicators may relate to user demand, user satisfaction, efficiency, effectiveness, take-up, etc. For the project, they will relate to achieving your objectives. Use SMART objectives (specific, measurable, achievable, realistic, timed), you to demonstrate whether they have been achieved. Discuss how you will measure success with stakeholders to understand success from their point of view.
Think about the level of success you hope to achieve and try to quantify success in some way, to ensure that your evaluation results are objective, valid, reliable, repeatable, etc. For example:
- 1,000 users per day will visit the website
- Usage of the portal will increase by 200% from year 2 to year 3
- 90% of users questioned will say the process/method saved them time
Jisc evaluates its programmes to ensure knowledge and results are shared with the wider community and to improve the programme. For most programmes, there will be formative and summative evaluations. These studies may be undertaken by an external consultant or group of consultants, a designated project within the programme, the programme’s advisory board, or a mixture of these approaches.
Projects are required to participate in any evaluation studies at programme-level and the programme manager will let you know about plans for programme evaluations and how to participate. Programme evaluations focus on what the programme is achieving and are not to ‘check up on’ projects. What projects are achieving is relevant, but individual projects are not evaluated.