JISC

This project has completed. See the project website and download the final report.

Overview

Formative assessment aims to provide appropriate feedback to learners, helping them gauge more accurately their understanding of the material set.  It is also used as a learning activity in its own right to form understanding/knowledge.  It is something lecturers/teachers would love to do more of but do not have the time to develop, set, and then mark as often as they like.  A formative e-assessment system allows lecturers/teachers to develop and set the work once, allows the learner to take the formative test at a time and place of their convenience, possibly as often as they like, obtain meaningful feedback, and see how well they are progressing in their understanding of the material.  McAlpine (2002, p6) [1] also suggests that formative assessment can be used by learners to “highlight areas of further study and hence improve future performance”. Steve Draper (2005) distinguishes different types of feedback, highlighting the issue that although a system may provide feedback, its level and quality is still down to the author.

At last year’s JISC/CETIS conference it was recommended that the community needed to ‘kick start’ the use of the IMS Question and Test Interoperability version 2 (QTIv2) specifications. At this meeting it was felt that in order to achieve this, there needed to be a robust set of tools and services that conformed to the QTIv2 specification.  R2Q2 is a recently funded JISC project that successfully implemented a rendering and response engine for a single question (also termed an item), for which there are sixteen types described in the specification and implemented in R2Q2.  While this is useful it does not implement the whole of the specification regarding the test process.  The specification details how a test is to be presented to candidates, the order of the questions, the time allowed etc.

Aims and objectives

In this project we aim to build an assessment delivery engine to the IMS Question and Test Interoperability version 2.1 specifications that can be deployed as a stand-alone web application or as part of a Service Oriented Architecture enabled Virtual Learning environment or portal framework.

The engine will provide for:

• delivery of an assessment consisting of an assembly of QTI items
• scheduling of assessments against users and groups
• delivery of items using a web interface
• marking and feedback
• a Web service API for retrieving assessment results

In the second phase, the project will integrate with the other projects in this call on item banking (Cambridge: Minibix) and item authoring (Kingston: AQuRate) to provide a demonstrator, and will contribute to its evaluation and the evaluation of the project and its integration with the other projects under the Assessment call.

Project methodology

The ASDEL project will deliver a test engine for the QTIv2 specification.  We will not be considering anything beyond the specification.  The project will take an agile approach to developing the ASDEL System and the language of choice is Java.  This choice of Java as a language is mainly to allow for cross platform deployment.  The project will use a Service Oriented Architecture (SOA) when designing the ASDEL system; major components of the system will be designed as internal Web services.

Anticipated impact

The outcome of this project will be the first open source test delivery system for tests written in the QTIv2 format.  ASDEL will be freely available to those working in the FE and HE sector in the UK.

The test engine will be designed to work either as a stand-alone Web service or could be integrated into a Virtual Learning Environment (or other Web based course delivery system).

It is envisioned that such a system as ASDEL will enable early adaptors and those researching into assessment for e-learning an opportunity to experiment with alternative ways of presenting a test that QTIv2 offers.

Currently there is no test authoring tool available to the community as a whole, but QTIv2 will allow for the sharing of items and test amongst the community.  This will break the ‘chicken and egg situation’, in that people will not write test until there is a delivery system, and ASDEL will now provide that system.  So hopefully the community can start to write tests and share these amongst communities of practice (academic disciplines).

Lead institution

• Learning Society Lab, School of Electronics and Computer Science, University of Southampton

Project partners

• University of Hull
  

[1] Mhairi McAlpine: (2002) Principles of Assessment, Bluepaper Number 1CAA Centre, University of Luton, February.