Technology-enhanced assessment
Materials are now available from the series of workshops based around the JISC Effective Assessment in a Digital Age publication. Find out more
Assessment lies at the heart of the learning experience: how learners are assessed shapes their understanding of the curriculum and determines their ability to progress. At the same time, assessment and feedback form a significant part of practitioners' workloads and, with increased numbers, reduced budgets and higher learner expectations, continue to be a matter of concern for many institutions delivering higher education.
JISC has been working in support of technology-enhanced assessment for over a decade, looking at issues to do with pedagogy and institutional context for the appropriate use of a wide range of technology to support the assessment and feedback process, as well as technologies and interoperability standards for the delivery of on-screen tests.
Technology can support nearly every aspect of assessment in one way or another, from the administration of individual tests and assignments to the management of assessment across a faculty or institution; from automatically marked on-screen tests to tools to support human marking and feedback. Clearly, though, for technology-enhanced assessment to be effective, pedagogically sound developments need to be supported by robust and appropriate technology, within a supportive institutional or departmental context.
'Technology-enhanced assessment' refers to the wide range of ways in which technology can be used to support assessment and feedback. It includes on-screen assessment, often called e-assessment.
JISC-funded assessment developments
Mapping the terrain
Advanced e-Assessment Techniques
The aim of the project was to provide JISC and its community with a review of state-of-the-art techniques in e-assessment which should be considered for application in UK HE and FE settings.
This JISC study was designed to build a significant body of information about who is using different techniques, the associated issues and the benefits of advanced e-assessments. This review will contribute to subsequent work to define and develop a roadmap of future e-assessment research and development activities within the JISC e-learning programme.
Report on Summative e-Assessment Quality
The report on summative e-Assessment Quality (REAQ) project surveyed quality assurance (QA) activities commonly undertaken in summative e-assessment by UK Higher Education (HE) Practitioners and others. The project focused on what denotes high quality in summative e-assessment for the interviewees and the stops that they take to meet their own standards.
Formative e-Assessment
This project developed a domain map for formative e-assessment based on a review of relevant literature in the field of formative assessment and e-assessment as well as an analysis of aspects of prevailing assessment practice in technology-enhanced post-16 contexts nationally and beyond. Project outcomes include a set of recommendations, a literature review as well as a set of case studies of existing practice in formative e-assessment. On the basis of the case studies the project delineated a set of key processes involved in effective formative (e-) assessment practice as well as a number of technical requirements for formative assessment systems, components and processes.
e-Assessment roadmap
This project (completed in April 2006) provided an overview of the current drivers and barriers to e-assessment in post-16 education, covering summative (or high stakes) assessment, formative (or low stakes) assessment and diagnostic testing.
FREMA: Framework Reference Model for Assessment
The FREMA project provides an Assessment Reference Model for systems in the assessment domain of e-learning that are built on top of Service-Oriented Architectures; such as Web Services and the Grid, and in particular the JISC e-Framework.
The Assessment Reference Model begins to describe how the assessment domain maps to the e-Framework and thus acts as a driver for implementation and evaluation. FREMA aims to ease the development of further web services and promote the re-use of existing ones and will be fully integrated into the e-Framework knowledge base.
Tools and interoperability
A significant focus of JISC's work in the assessment area has been on interoperability standards to enable efficient and re-usable on-screen tests without vendor lock-in. As part of this work, several tools have been developed to provide a demonstration of these interoperability standards (IMS QTI). Profiling work on IMS QTI 2.1 is currently underway, and several tools have successfully demonstrated that they work with the new profile. Further information on this work can be found in the assessment topic on the JISC-CETIS website Tools have also been developed to support other aspects of technology-enhanced assessment.
This project will support the profiling and implementation of IMS QTI 2.1 by assisting with the develoment of the main profile, and by providing online support to users of the specification.
AQuRate A QTI 2 Authoring Tool
The AQuRate project has developed an open source, QTI v2 standards compliant, platform independent question authoring tool with API.
This project developed a web-based system to provide staff and students across different institutions with analysis and visual feedback of data captured using 'clickers' (electronic voting system, EVSs) in their lectures or other teaching eisodes. This allows students to view questions with their own responses after a lecture for feedback and revision purposes. For staff, the tool provides anonymised aggregate statistics and feedback as to the effectiveness of the clicker questions used on their courses.
The Minibix project has developed an open source item banking system for QTI v2 items capable of supporting both high-stakes private item banks and low-stakes item banks for sharing questions. The tool interoperates with the HEA Physical sciences item bank. The project team will provide a QTI SUM to the e-Framework.
ASDEL: A QTI2 Item and Assessment Delivery Engine
The ASDEL project has developed an open source QTI v2 assessment delivery engine that can be deployed as a stand-alone web application or as part of a SOA enabled VLE. The delivery engine includes assessment developer tools such as a QTI v2 validator. The tools interoperate with ASDEL and Minibix and provide a Moodle connector.
MathAssess is a composite application demonstrating join up between ASDEL, AQuRATE, Minibix and SnuggleTex to cater for specific mathematics e-assessment requirements.
WebPA is an open source online peer assessment system which has been developing in maturity since 2006. It now supports peer assessment of group work, SOA design and integration with VLEs.
The project produced a simple web service enabling very easy production of a popular subset of QTI questions from any MCQFM consuming application thus supporting the community of QTI development practitioners to further disseminate the standard.
The Xmarks project has developed components to allow the exchange of assessment information between an institution’s student records system and virtual learning environment, using a web services model. The project is aimed at developers within further and higher education institutions who are looking for a way to synchronise assessment and marks data.
This project further developed OpenMentor, a learning support tool for teachers in further and higher education, which helps them by providing reflective comments on their assessment and feedback of student assignments and coursework.
PyAssess provides an open source toolkit for implementing QTI v2-based assessment (particularly marking) services in Python, building on the Python QTI v1.2 to v2.0 migration tool produced earlier. The project also produced a command line demonstrator to illustrate the use of the toolkit and support the testing and debugging of complex questions.
This project scoped and implemented services to enable the packaging of assessment items for storage in an item bank and the search and retrieval services necessary for the use of item banks.
A software design project to define functional requirements for an item bank tool to support storage, exchange and analysis of high-stakes assessment items for use in UK further and higher education.
Technology-enhanced assessment in an institutional context
JISC Assessment and Feedback Programme, 2011 - 2014
20 projects have recently been funded as part of the Assessment and Feedback Programme, involving 8 institutional change projects, 8 evaluation projects, and 4 technology transfer projects. For further details on all projects please select the link above.
If students are to develop critical thinking and autonomy in assignment production then they should be provided with opportunities to evaluate and provide feedback on their peers' work. This project will identify educational designs and evaluate software tools that support student peer-review processes, and will pilot-test these in different classroom scenarios.
This project will explore the use and effectiveness of digitally supported patchwork text assessments in a range of traditional academic subjects within different HEIs. Conditions required for successful implementation of patchwork text assessment will be identified.
The ESCAPE project involves the re-engineering of nine modules across two schools (Business and Life Sciences) from an assessment for learning perspective. The project uses a range of curriculum development activities and change management processes to investigate, formulate, pilot and embed the use of a range of blended learning solutions in order to achieve pedagogical change. The objective is to improve the effectiveness and efficiency of assessment practices.
Building on previous work at the University of Southampton, this project provided an on-line environment for administration, marking and feedback of electronically-submitted open-ended student assignments, such as essays.
Simple e-based technologies (specifically blogs and online questionnaires) were used to develop an integrated process which collates feedback, guides student reflections and facilitates their use of feedback to improve performance and inform their ensuing aspirations. The process was rolled out within the School of Biosciences at the University of Westminster.
This Building Capacity project, funded by the JISC Operational Support Committee, aims to develop a portfolio of assessment and feedback process that take advantage of technology, and to support their adoption by academic schools and individual staff. This includes promoting greater take-up of existing services, as well as exploring new approaches.
A research project investigating the benefits and drawbacks for staff and students of three forms of audio feedback currently being used by lecturers around the UK: audio-only, asynchronous audio-visual, and synchronous audio-visual.
The key aim of this JISC-funded project was to develop an innovative, interactive Web 2.0 resource, 'ASSET', to encourage staff to experiment with the use of video media to provide feed-forward and feedback to students on their assignments.
This project aims to derive a personalised e learning assessment and reporting model which meets the needs of work based learners, local employers and the training provider, based on e-assessment of engineering students employed by Rolls Royce and studying at Derby College.
This project is exploring the role of e-assessment in enhancing the student experience of HE students in an FE college.
www.jisc.ac.uk/eassesswales
Publications
Guide to Effective Practice with e-Assessment
This document combines:
- The e-assessment roadmap
- The e-assessment glossary
A set of e-assessment case studies of innovative and effective practice involving 55 institutions across the UK and large-scale assessment outside the
FE and
HE sectors. The studies cover issues such as innovation, effectiveness, maturity, accessibility, confidence-based testing and single champions in an institution.
This guide explores how technology can improve the experience of assessment and feedback for practitioners and learners, demonstrating through ten newly researched case studies how both generic and more complex technologies can increase learner autonomy, improve teaching efficiency and enhance the quality of the experience of assessment and feedback. The supplementary online resources which accompany the guide include video case studies, a planning tool and expert podcasts designed to support individuals and curriculum teams in harnessing the potential of technology to transform their assessment and feedback practice.
|
This document (from 2008) describes work on e-assessment within the
JISC e-learning programme.
Assessment Symposium - Where next for assessment in a digital age?
The JISC e-Learning Programme recently ran an Assessment Symposium entitled – Where next for assessment in a digital age. This event was held with invited experts to inform the development of the Effective Assessment in a Digital Age publication and also to feed in ideas for future funded activity in the area of technology-enhanced assessment and feedback.
Presentations from this event
JISC Services to support e-assessment
Established to monitor interoperability initiatives in e-assessment in FE and HE, the Special Interest Group (SIG) for assessment also monitors wider developments in the technology and application of e-assessment. The SIG is run by the JISC-CETIS innovation service and includes a mailing list and regular open meetings.
JISC infoNet offers information on e-assessment through its Effective Use of VLEs infoKit. JISC infoNet is an Advisory Service promoting good practice and innovation within the education sector.
Netskills offers workshops and support materials on assessment and communications for e-Learning.
TechDis provides advice on accessibility in relation to e-assessment.
Contact
For further information about JISC's work in assessment, contact Sarah Davies.