Inclusive assessment
Accessibility must be considered from the outset when designing assessments, otherwise disabled learners could be unintentionally disadvantaged.
Organisations must ensure that the assessment evaluates learning outcomes and not the speed, manual dexterity, vision, hearing, or physical endurance of the learner. There are a number of things you can do to maximise the inclusiveness and accessibility of assessment.
Personalised learning and assessment
Involving students in both their learning and assessment provides opportunities for them to influence their progress, and use their own preferred methods to access the curriculum. Contributing to the choice and design of assessments ensures that they are able to contribute fully no matter what their access needs are.
Self-assessment and peer assessment are also powerful motivational tools which can be used with most assessment techniques.
E-assessment
This allows learners to use their own assistive technologies to access questions and evidence their answers. This benefits different students in different ways, for example:
- A student with motor impairments independently handling the exam paper
- Blind students accessing the assessment using a screen reader
- Students can personalise their view of the questions (eg, font types, sizes, colours)
- Those with visual difficulties or dyslexia can adapt the appearance and layout of the text to make their reading more efficient and productive
- Text-to-speech software can read questions out loud - students who are dyslexic or visually impaired can independently access the meaning of the questions or accompanying resources.
Students can also check their own progress and get instant objective feedback which will help reinforce and embed learning. This may be particularly useful for those with short term memory problems and students on the autistic spectrum who may prefer automated feedback with limited personal interaction.
Evidence-based assessment
Many professional associations such as those for medical, health and engineering now use evidence of achievement as qualification criteria for professional progression. This evidence is collated within a digital portfolio system or e-portfolio.
Using an e-portfolio enables individuals to demonstrate their competence in a personal and customised way. The restrictions of formalised testing are no longer barriers to the demonstration of achievement.
Sources of evidence can include documents produced as part of work activity, records as photographs or videos of the product or the process. Witness testimonies, assessor observations or authenticated candidate reports can be recorded using audio or video to validate the activity.
Barriers
Although e-assessment has great potential for accessible experiences the potential is not always realised. This can be due to design faults in the questions, the quiz software or the delivery platform on which the quiz sits.
Different disabled learners experience different barriers; an e-assessment that works well for a dyslexic learner may be completely inaccessible to a blind learner. Some barriers (such as poor question design) can be addressed quite quickly but others, such as accessibility issues in the quiz software or the virtual learning environment on which it sits, can be more difficult to address.
Ideally your organisation will address these issues during the procurement process when you decide what sort of systems you intend to use.
Policy and procedures
There are a series of strategic issues that the organisation needs to consider if accessibility is going to work. These include procurement, standards, user testing and the different roles and responsibilities within the organisation that make these elements work together.
Digital assessment tools are beneficial for organisations when installed across the whole provision so policies and procedures need to be in place.
These tools provide constructive data to inform an organisation’s self-assessment and quality improvement process. They enable tracking of learner engagement, achievement and progression and allow learning assessment to take place more often without increasing the burden of manual marking.
System procurement
Students with disabilities can present a range of accessibility challenges for software assessment systems.
Take a coordinated approach
The procurement, installation and development of assessment systems needs to be managed in a coordinated and structured way. It is important to include the following when evaluating assessment systems:
- Accessibility
- Interoperability
- Privacy and data protection.
Anticipating these challenges should provide more time to address them properly and ensure that they are supported by informed policies and procedures.
Anticipate and accommodate
Creating engaging assessment materials that are accessible to all learners, with or without disabilities, is considerably more difficult than creating an accessible website. It is therefore likely that even the best designed tools will have some functions that don’t work with particular learners and/or certain assistive technologies.
It is essential to identify any accessibility issues in advance so that any additional support costs can be incorporated into the budget. Questions to ask before agreeing to use one particular system include:
- What built-in accessibility features that exist;
- Where users would find the accessibility documentation;
- Which assistive technologies has the system been tested with.
If the answers are unsatisfactory you may opt for a different product rather than spend more on disability support.
Compatibility
You need to consider whether the e-assessment tool is compatible with existing tools within your organisation. Compliance with IMS global standards is crucial - interoperability may not sound exciting but can prevent an organisation locked into proprietary systems.
Roles and responsibilities
In order for accessible assessment to be embedded across the organisation, a number of different roles need to be involved and take responsibility.
Disability support
This department needs to be involved as they will provide the initial support for learners adjusting to the technology, and will also provide personal support as a result of accessibility failings in any system.
Network and systems managers
They will need to advise on interoperability and long-term trends in standards (for example the coming together of IMS global standards and EPUB3 in EDUPUB).
Quality assurance
This team needs to be involved because the choice of system may have a big impact on progression data available and integration with learning analytics tools.
Staff development and e-learning
They will need to provide guidance on the suitability of the system and the necessary training for staff.
Recording and recognising progress and achievement (RARPA)
There are many occasions when assessment needs to be summative but is not part of an accredited course.
Organisations need to have procedures in place to track, monitor and evaluate achievement and to include them in their quality assurance procedures.
Practice
Here we touch on generic good practice in feedback, question design, assistive technologies and use of rich media in quizzes. We explore RARPA and a range of assessment tools.
Well-designed assessments will allow you to assess knowledge and understanding.
Reducing barriers
All types of assessment have potential accessibility issues. A good way of exploring how to reduce barriers is to refer to the awarding bodies and how they are addressing these challenges. In 2006, we worked with a number of awarding bodies to provide accessibility in e-assessment guidelines. Although the work is old the awarding bodies are still using this guidance and Appendix 2 of the report has a list of helpful recommendations that would be relevant to a tutor.
Principles
The Higher Education Academy's summary resource on assessment is helpful for helping you think about what you want to assess and why. The publication 'A marked improvement – Transforming assessment in Higher Education' outlines excellent principles that apply across any sector. These are:
- Assessment for learning (not of learning)
- Ensuring assessment is fit for purpose (are you assessing achievement of intended programme outcomes?)
- There are limits to the extent that standards can be articulated explicitly (there are important benefits of education which are not amenable either to the precise specification of standards or to objective assessment)
- Constructing standards in communities (emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards)
- Integrating assessment literacy into course design (allow students to develop their own, internalised conceptions of standards and to monitor and supervise their own learning
- Ensuring professional judgements are reliable (establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities)
These principles are valid for all sectors and levels of achievement, and are as useful for those teaching learners with complex needs as they are for those working in higher education.
Types and design
Feedback
Giving effective feedback is an essential part of the assessment process. It is crucial that feedback is prompt and supportive. E-assessment should have the information incorporated into the system to enable students to learn and progress as a result of the assessment.
Good constructive feedback, even in a simple question, should help the learner not only know what the best answer is but why it was best and why the others were weaker (or wrong).
One of the benefits of well-written, objective, computer-based feedback is that it removes one layer of human interaction - this can be helpful for people with mental health problems or autistic spectrum disorders as they may prefer to have factual feedback with no personal comments.
Questions
Writing effective questions for assessment is an important skill. The nature of the question you are writing demands different skills. For example, writing a good multiple choice question requires different considerations from writing a good assertion-reason question or a good gap fill question.
The Computer Aided Assessment Centre has an excellent set of guides with a range of examples that would be very effective for staff development. The design of the question will depend on the nature of the assessment.
Rich media
Embedding audio and video in assessments can add significant value for some learners (for example those with reading impairments) but create barriers for others (for example blind learners accessing visual content of videos or deaf learners accessing the audio content).
There are different ways of reducing these barriers. Audio files can have a linked transcript. Video files can have audio captions on screen or (if the audio narrative works independent of the visual content) a separate transcript of the audio. Blind learners may need an audio description of the video scenes. This is quite a specialist job and must be done carefully in order to avoid undermining the assessment objectives.
RARPA in practice
RARPA is essentially just good practice in teaching and assessment. There is a need for clear and accurate recording of achievements to benefit both the learner and the organisation.
Data on the tracking of progress will vary according to the learner and the type of achievements that they are working towards. There are a number of case studies on how to quality assure RARPA in the provision for learners with learning difficulties available on the Excellence Gateway.
Assistive technologies
Not all disabled learners will need assistive technologies in order to meet their access needs. It is helpful to think first in terms of the accessibility services that learners require. These services may be delivered via the assessment tool or separately via assistive technology.
We outline a number of different accessibility requirements and how to achieve them:
Change font and background colours
- Check whether there are options in the software to do this and test those options
- Check whether the change options in the operating system work in the software - find out how to change the colour scheme in Windows 7 or Windows 10 and Mac OS
- Try third-party commercial or free and open source tools.
Increase the font size
- Are there options within the software to do this?
- If the software runs in a web browser, can you use the browser settings to increase the font size or zoom in?
- Can you use the magnification options in Windows 7, Windows 10 or Mac OS?
- Try third-party magnification tools.
Ensure automatic text reflow when font size is increased
- Test in the software or web browser if it is browser-based
- Try third-party commercial tools with optical character recognition functionality.
Navigate pages by sight and select text to read out loud
- Is there a text-to-speech option in the software?
- Is there a browser based text to speech plugin?
- Try third-party commercial or free text-to-speech tools like Balabolka or the Southampton ATbar.
Navigate pages by audio and have text read out in an appropriate order
Third party commercial or free screenreader tools like NVDA or WindowEyes. How effectively these tools work depends on the quality of accessible design in the software.
Navigate the test and answer questions without using a mouse
Keyboard shortcuts or third party tools like switch, Eyegaze or voice recognition systems. Again, these tools are dependent on the accessibility of the software.
Describe all non-text elements (eg, images or formulae)
This is dependent on both the software and the accessibility levels specified. Describing images requires care so that the assessment objectives are not undermined. Describing formulae requires specialist knowledge and may or may not be appropriate depending on whether a blind user accesses maths through Braille or audio.
Tools
The tools a tutor might use will vary depending on the organisational policies, procedures and facilities and their intentions for the assessment.
For instance, does everything have to be tracked? Is it a summative test or a more formative opportunity to consolidate and reinforce learning? Does it need to work on a mobile phone? Are the learners creating questions and challenges for each other?
Institutional virtual learning environments like Moodle or Blackboard will provide inbuilt assessment tools. There are also a range of commercial systems available as well as free and open source tools such as Rogo from the University of Nottingham.
Other tools designed for creating courses often have interactive functionality such as Moodle and Xerte OnlineToolkits.