JISC

Download the full report

Building on very small-scale work using MP3 files for summative feedback on one programme, this project widened the focus to both formative and summative feedback in various disciplines at different educational levels. The experimentation included delivering digital sound files containing feedback to students via a virtual learning environment, email and mobile devices such as widely-available MP3 players.

Executive Summary

This project (Sounds Good) was a small project exploring the use of digital audio to give assessment feedback. Part of the JISC Users and Innovation programme, it was based at Leeds Metropolitan University and led by Bob Rotheram, National Teaching Fellow. Initially, it was funded for the period January to July 2008. During this time a team of Leeds Met lecturers experimented with digital audio to give formative and summative feedback on students’ coursework. Later, funding was provided under JISC’s ‘benefits realisation’ initiative for a second stage, ‘Sounds Good 2’, which ran until February 2009. In this phase the work at Leeds Met expanded and audio feedback was introduced to three other higher education institutions: Newman University College, Birmingham; University of Northampton; York St John University.

The main aim was to test the hypothesis that using digital audio for feedback can benefit staff and students by:

• saving assessors’ time (speaking the feedback rather than writing it)
• providing richer feedback to students (speech is a richer medium than written text)

The project has mainly been a qualitative study. Even so, it has produced a few statistics. Taking the two phases of Sounds Good together, 38 lecturers in four institutions have supplied audio feedback to at least 1,201 students at all educational levels from foundation degree and first-year undergraduate to doctoral. The staff were located as follows: Leeds Met 23, Newman University College 8, University of Northampton 4, York St John University 3. In the first phase the student numbers on the various modules ranged from six to 151, with at least 463 students receiving one or more items of audio or video feedback. In Sounds Good 2 the numbers on modules ranged from three to 150 and at least 738 students received one or more items of audio feedback.

Sounds Good has received considerable positive publicity, including being shortlisted for a Times Higher award. The project manager gave 24 presentations about audio feedback in the UK, US and Canada, most of them by invitation.

Early in the project it was agreed with JISC that five key questions would be explored. Here they are, along with the answers:

(Without reducing the amount of feedback) in what circumstances can using digital audio save assessors’ time?

The most favourable circumstances would appear to be:

• The assessor is comfortable with the technology
• The assessor writes or types slowly but records their speech quickly
• A substantial amount of feedback is given
• A quick and easy method of delivering the audio file to the student is available

Does digital audio feedback improve students’ learning experience?

Students were overwhelmingly positive about receiving audio feedback on their coursework. They frequently remarked approvingly about its personal nature and the detail provided, evidence that the lecturer had carefully considered their work. On the other hand, a small minority of students said they preferred written feedback; a few asked for both audio and written comments on their work.

What do assessors think of digital audio as a medium for providing feedback to students?

The Sounds Good staff team were strongly in favour of audio feedback; most have clearly said that they intend to continue using it. Even if they didn’t manage to save time, several members of the team commented that they were able to give more, and higher-quality, feedback using audio, which they felt was worthwhile.

What recommendations are there for improved practice?

Practice guidelines have been produced about using digital audio for feedback on students’ work. They are grouped under four headings: saving time; technical matters; administration; feedback structure. The main points are:

• How much time you eventually save will depend on various factors, including how much feedback you give and how quickly you write, type and speak.
• Consider accepting a longer pay-back period. Experiment with spending more time in the short term, using audio to give your students more extensive advice and richer feedback. It may save you and your colleagues work in the long term.
• Make your audio files as small as possible, so they can be sent quickly and stored economically.
• Aim for the minimum acceptable sound quality for the particular purpose.
• Keep the files short – don’t ‘overdo it’. Only go beyond five minutes if there is a good reason.
• Make sure key administrative and quality-assurance staff accept that you are giving audio rather than written feedback.

What should be explored next?

Some caution is appropriate. It is difficult to generalise from the project. Sounds Good has generated case studies, some of them very small, about the use of digital audio or video. On the other hand, comments from students and staff have been remarkably consistent throughout.

Here are some issues which seem to be worth exploring, if resources permit:

• Can staff become quicker in providing audio feedback if they persist?
• Does using audio to provide more extensive guidance and richer feedback lead to saving staff time in the medium-to-long term?
• How may the practice guidelines be improved?
• How successfully can audio feedback be combined with electronic submission of assignments?
• Can speech-to-text software be used to generate written feedback automatically (and sufficiently faithfully) at the same time as audio feedback?
• Can ways be found to automate the transmission of feedback files to students?
• What happens if tutors and students use audio to engage in dialogue about assessed work?
• Does the novelty of audio – for students and staff – wear off with repeated use?

Of these issues, the most urgent is probably automating the transmission of feedback files to students.

External evaluator’s report

Dr Isobel Falconer of Glasgow Caledonian University acted as external evaluator for Sounds Good 2. In her report she notes that both stages of the project have been “highly effective” and “have had a significant effect on changing the practice of participants.” She repeats the view of one participant that “the project represented great value for money.”

Conclusions

Sounds Good worked very well. It ran almost entirely to plan in the first phase and encountered only minor problems in the second phase. It broadly achieved all it set out to achieve. It did some valuable exploration and produced useful practice guidelines. Using audio for assessment feedback has proved popular with students and staff. It is probably fair to say most UK higher education lecturers would find it worth giving audio feedback an extended trial with at least with some of their assessment work.

Sounds Good has obtained considerable positive publicity through presentations and publications. There is reason to expect that, as a result, a substantial number of people will begin to use audio for feedback, in the UK and elsewhere. All in all, it has delivered an excellent return on JISC’s modest investment.

Report available electronically only