- To allow you to set the agenda for system demonstrations
- To ensure you see how the system would operate your business scenarios
- To empower you as an active tester not a passive viewer
- To ensure fairness and objectivity in decision making
- To avoid evaluators being diverted by sales talk.
- Script test scenarios based on real life business scenarios
- Prepare test data
- Organise the evaluation events
- Brief suppliers and evaluators.
- Agenda for a set of tested scripts
- Supplier briefing pack
- Evaluators’ scores and product summaries.
Depending on the procurement process you have chosen you may either shortlist suppliers on the basis of their tender responses or move straight to a full evaluation of interested suppliers.
Where compilation of a shortlist is part of the process you are likely to look at:
- The company’s financial standing
- The company’s market position
- What proportion of your requirements is the company able to meet? This is one of the most difficult areas to evaluate from a tender response alone. Company responses are understandably positive in their approach and will tend to gloss over any shortfalls against your requirements. Don’t worry – the next stage of the evaluation will help you differentiate the optimistic hopefuls from the real contenders
You should take care that any shortlisting decisions are fully documented and can be justified against your selection criteria should the decision be challenged.
In some cases you may be able to come to a selection decision on the basis of the responses to the Invitation to Tender (ITT) but, in the case of significant investment decisions, you are likely to wish to conduct some kind of evaluation event.
A formal evaluation event is an effective way to test the extent to which potential suppliers can meet your requirements. In the case of replacing a mission critical system the supplier evaluation is one of the most critical elements of the selection process. It is based on the premise that you as the customer should be able to see hard evidence of what the system can do and, in particular, what it can do for your business.
This means that, rather than simply attending a sales demonstration, you set the agenda based on your key business issues. All suppliers will be required to follow the same demonstration agenda making direct comparison far easier than would otherwise be the case.
Setting the agenda
The agenda will depend on what type of system you are buying and what the scale of the implementation is likely to be. This model was originally developed by the University of Northumbria during selection of a new integrated student and HR system. The scale of that project meant that the evaluation of each supplier took place over three days and had up to three streams running in parallel.
A simpler system will require a more straightforward evaluation event. The key thing is to ensure that your agenda covers your main business issues.
You may choose to look at: your most critical processes eg, in looking for an HR/payroll system you may decide that payroll and recruitment are critical to your organisation but training management and time and attendance are not hence you would focus the evaluation on the first two.
Your highest volume processes eg, in a student system might be admissions and enrolment.
Processes which give rise to particular problems in your organisation – this may stem from weaknesses in your current systems or the unique nature of your requirements.
Criteria which support your strategic objectives and/or which help differentiate between suppliers. For instance if your institution wishes to offer self-service functionality to students and only one potential supplier can meet this requirement this may distinguish between otherwise similar offerings.