Constructing Assessments using Tools and Services
This project (CATS) developed an exemplary system for harvesting assessment items from item bank systems on the basis of an assessment request profile that not only comprised the subject domain query and the target assessment bank, but also included additional "pedagogical" parameters, i.e. the number of items to be retrieved and the algorithm to be used to traverse the target assessment bank. The resulting collection of assessment items would be packaged according to the content/assessment packaging specification indicated in the profile and then written to a files structure, zipped, etc. according to the profile request.
Summary
Although a functional system, the CATS system was conceived as a proof of concept demonstrating how item banks might not only be queried using web services but the items actually harvested. The design was deliberately extensible and explicitly facilitates the creation of new "flavours" of the services. This was most evident in the connections between the CATS system and item banks. The potential variability in the interfaces offered by assessment banking systems requires a special connector for each system that is able to translate between the common language of the assessment request and the particular language and format required by the assessment bank. Beyond accommodating the development of new item bank connectors, the system permits the development of new packaging services for new and different content/assessment packaging standards or the development of new writing services that could populate assessment management systems. The architecture also allowed the incorporate of new steps in the harvesting process; among the options discussed was a service for comparing the list of items returned against existing previously created collections to avoid duplicating the same assessment pool or the incorporation of a step where the user was able to preview and screen the collection of items retuned before submitting the to the packaging and writing services.
The Services
Beyond a coherent functioning system, CATS was conceived as a toolkit of services that could have a wider application. The project took services developed for previous projects and extended their functionality making them more amenable to use in new context. For example, the SPAID zipping service could only handle one assessment item; through CATS the zipping service was extended to process not only multiple assessment items, but also the items' dependent files (e.g. images) in a complex hierarchical files structure. Similarly, the D+ services were developed to search across data targets and return pointers/references to records that matched the query criteria. Through CATS the service was extensively modified so that, rather than simply returning a reference, it retrieved the actual item from the assessment bank along with dependent files. These retrieved items are returned in an organised stream allowing the file structure to be (re)created.