JISC e-Learning Framework Assessment Demonstrator
The aim of this project was to integrate a variety of forms of automated assessment questions and engines that are pre-existing outputs of internal development and other JISC projects.
Executive Summary
The approach was to demonstrate a system that incorporated results from three JISC projects: APIS (‘Assessment Provision through Interoperable Segments’), ISIS (Integrating Simple Sequencing) and ASAP (Automated System for the Assessment of Programming). The demonstration system provided a means of delivering assessment items embedded in a learning context; they were stored and presented IMS SCORM and QTI 2.0 using open standards. To provide the required demonstration, some ‘Player’ software was developed to orchestrate the delivery of the content, since this component was not available from third party sources at the required time. Content was authored for three high stakes trials in August 2005, in which students underwent retake examinations. Throughout the project there was close consultation with staff involved in the JISC funded Reference Model for Assessment (FREMA). A demonstration of the system, content and software for download are all available via the project website.
The project found that successful delivery of innovative content was possible using the components of the JISC e-Learning Framework. There were some difficulties and limitations, such as the lack of authoring tools for QTI2.0, the relatively small range of QTI questions that were successfully rendered and processed by the APIS software, and the scope for varied and adaptive sequences of learning activities was limited by the support provided by the sequencing engine. Also, the project team encountered a frustrating but understandable ‘institutional reluctance’ to allow innovative components to be.
In conclusion, the project was successful in highlighting some capabilities and deficiencies of the components that it demonstrated. It successfully deployed a system built from these components in a high-stakes test environment, and made the results available to the wider community.