JISC

The aims of this project were to:

• Develop a specification for a web service based on the use of Bloom’s Taxonomy (a classification of different learning objectives) as a language for determining the qualifications level (at the Scottish Qualification and Credit Level) of assessment questions. The web service aimed to return a list of metadata, utilising a Bloom’s query approach, in a standard set format (XML) that can be read by standard searches in applications.
• The specified web service aimed to provide a platform for potentially populating metadata items for assessment questions and for automatic updating of existing question banks.
• Investigate the potential for Bloom’s taxonomy as a supportive tool in mapping the language used in question stems to identify the SQCF / QCF level.
• To engage with college practitioners, who have an interest in pedagogy, e-learning and resource development, to ensure dissemination of findings and to encourage embedding of project findings into assessment approaches.

 An Example of the how the web service might work:

Question stem:  Critically assess George Simmel’s account of the characteristic features of modern social relations.

Output from web service as follows -
Verbs in Blooms Taxonomy

Highest Blooms Level verb in question stem:            6 - Evaluation
SCQF Generic Outcome :                                               Generic Cognitive Skills

Outcomes

After evaluation of a range of electronic assessments across a number of curricular areas, it was determined that the relationship between the language used in assessment stems and SCQF / QCF levels could not be accurately or consistently identified through a software application. The following reasons were identified –

• There was often a limited amount of text within existing question stems for the software to parse and identify the level.
• The use of language within the question stems did not always reflect the learning domain / level being addressed and, as a consequence, provided no indication of the potential SCQF / QCF level.
• The language used in assessments at different SCQF / QCF levels was often similar / the same therefore providing no discrimination of SCQF / QCF level.
• As a consequence, the SCQF / QCF level could only be consistently and accurately identified through reference to programme / unit descriptors.

However, where assessments were well designed for the level, it was possible, though parsing the question stem, to identify the Bloom’s Domain and associated Level and the relevant SCQF / QCF generic outcome / indicator. Through discussion in practitioner focus groups and the delivery of workshops in assessment design, it was clear that there was a value to practitioners in identifying these characteristics to support assessment design, alignment with other aspects of the curriculum (such as programme and unit descriptions and learning materials) and to ensure that assessments are deployed to address specific learning objectives.