AQuRate: A QTI Authoring Tool
This final report describes the design and implementation of software for the creation and editing of assessment questions.
Executive Summary
The software was designed to work in conjunction with two other projects, Minibix and Asdel. The assessment questions can be stored in (and retrieved from) a Minibix repository. The Asdel project provides components to deliver a test composed of such assessment questions.
A major motivation for the trio of projects is version 2.1 of the QTI (Question Test Interoperability) standard. This standard has addressed several shortcomings in versions 1.x that limit the interoperability of software and content used by different institutions and companies. By implementing several important components of software that use QTI 2.1, the projects have provided a platform for further adoption of the standard. Furthermore, in using the standard to implement a complex ‘Assessment Scenario’ (creation, editing, storage, selection and delivery), some minor flaws in the standard were identified. The associated remedies were also identified and earmarked for inclusion in the next release of the standard.
For the AQuRate project, two key requirements were that the software is ‘open source’, and that the software can be used across the common platforms (Windows, Mac, Linux, etc). The identification of the ‘New BSD’ form of license, to be appropriate for all projects, was an early project activity. Towards the end of the project, the source and executable releases were examined to ensure that all elements complied with the various third party licences. The requirement to make the project results accessible from multiple platforms was one factor in deciding to use Java as the main software development environment. In the main part, this requirement was satisfied: the software could be executed on any platform for which there is a Java Run-Time Environment. However, several small issues and problems persisted on some platforms. This was partly as a consequence of detailed testing taking place largely on Windows platforms only.
For the other two projects, the main integration requirement was to provide input and output functions with the Minibix repository. This was achieved, and demonstrated at the Assessment Special Interest Group meeting in Cambridge on 15th February 2008. The joint result of the two projects is an open source system that can create, store, retrieve, and edit Assessment questions, using a web-based repository and a desktop-based authoring tool. It is perhaps not unexpected that some of the requirements for this interaction were identified fairly late in the project timeline. In particular, the AQuRate team had initially failed to appreciate the requirements for meta-data (such as author name, institution etc) and appropriate elements were subsequently added to the authoring tool interface.
All three projects used a two-phase planning process. A progress review, between phases, identified the most useful directions for the second phase. It was decided that during this second phase the number of supported interaction types would not be extended beyond the commonly used ‘core’ set of interactions. The extended types could be accommodated at a future development stage. For the second phase, the focus was developing the interaction with the Minibix repository, and eliminating the small problems associated with the graphical user interface controls. Collaborative Design Sessions provided usability testing.
In conjunction with Dr. Graham Smith, The AQuRate project successfully re-used the ‘JAssess’ software to provide ‘rendering and processing’ of Assessment Items, in a browser, without requiring any server (i.e. using the AQuRate desktop application alone). In addition to the Desktop tool, server-based prototypes of the Authoring Tool were developed, all using a common API.
By the end of the project, there were over 1,500 downloads of the software from the project site, with numerous expressions of interest to continue the development. The project team were pleased with the final results, and look forward to continued involvement in the development of tools using the QTI standard.