JISC

This project utilised a version of the QTI toolkit (ASDEL, AQuRate, Minibix, R2Q2), extended to include the MathQTI specification in item authoring and rendering to handle mathematical elements. To enable the use of algebraic expressions within questions, a Computer Algebra System was incorporated into the item rendering and template processing calculations. Tools for integration with the gradebooks at least on the Moodle VLE were created and tested.

Executive Summary

JISC has made a significant investment in QTI, in particular and most recently Version 2.1, as an open standard in the e-assessment arena and supported the development of QTItools a collection of open source tool kits. By significantly enhancing these toolkits and integrating with Maxima a computer algebra system (CAS), MathAssess has:

• Handled the specific needs of mathematics: Unusual symbols and well-formatted equations and expressions and Delivery of truly randomised questions and tests, a major technical achievement
• Specified the necessary extensions to the QTI2.1 standard
• Carefully attended to user needs as expressed by the mathematics support community: Diagnostic, formative and summative assessments and Hints, solutions and feedback available to students at all appropriate stages
• Showed how long-standing problems with lock-in and obsolesence of mathematics e-assessment systems can be overcome by: Commitment to open standards and use of agile open source software development and Laying the foundations for development of appropriate open educational resources
• Created freely-available code which can be taken up by other subject groups

The strengths of this project included:

• An excellent team of users and developers supported by adequate financial resources
• QTI Version 2.1, a well-established open standard
• e-Learning and e-Assessment firmly embedded as part of mathematics support
• Addressing a real and intractable curriculum issue – the maths problem

The opportunities offered to the partners by the project included:

• Pioneering work in e-assessment
• High profile of 'any-time, anywhere learning'
• Making mathematics accessible is strategically important and stubbornly difficult
• Raising the profile of mathematics support activities

The challenges facing the project included:

• Avoiding the default conclusion It cannot be done in six months
• Complexity of the management task with seven partners across the UK and the very wide range of knowledge, skills and experience needed for success
• Development of content was out of scope, apart from limited needs of software testing

The threats to success which the project had to address included:

• Rendering mathematical formulae in web browsers is tiresome, requiring highly-skilled programmers to provide multi-functional software libraries
• Handling algebraic issues, such as - Is 2(x+4) - 3x = 8 – x ? - is tricky, requiring a detailed understanding of how syntax relates to semantics in mathematics and of the variety of ways in which this can be encapsulated in software
• Randomising questions is essential - Is 2(x+4) - 3x = 8 – x ? and Is -3(p-5) + 4p = p + 15 ? -  are 'the same question'. This is complicated, especially when hints, solutions and feedback to students are required. The generality and significance of this user requirement is sometimes hard to communicate
• Difficulty in giving end users access to complex technology; MathAssess has not been entirely successful here