JISC

This project assessed the effectiveness and impact of the Learning Activity Management System (LAMS) at the University of Sheffield in relation to: acceptability to practitioners; learner outcomes; effectiveness for the organisation; capacity-building across the organisation.

Executive Summary

This report presents the findings of an evaluation project funded by JISC as part of its Design for Learning programme and ran from May 2006 to October 2007.

The main research question was: to what extent does a tool such as LAMS add value to the practice and impact of designing for inquiry-based learning (IBL), and to the dissemination of IBL pedagogy?

Key findings

LAMS was useful in bringing the concept of design for learning to the fore at institutional and individual levels and in supporting the practice of process-aware design for IBL.

LAMS (in the version tested, i.e. version 1) offered a good fit with some practitioners’ pedagogical purposes and values in relation to IBL. It was seen to provide well for the design of linear forms of inquiry and relatively tightly-structured, teacher-controlled pedagogy.

It appeared considerably less well-suited to the design of more flexible and open-ended forms of inquiry and despite its orientation towards activity it did not tend to direct pedagogical thinking and practice towards student-led pedagogy.

The ease with which activity-design is supported by LAMS was perceived as bringing a risk of unreflective, mechanistic approaches to design.

The important role for pedagogical guidance and for opportunities for practitioner reflection and dialogue in supporting design for IBL, when a design system such as LAMS is being used, was confirmed.

Practitioners often wanted to use LAMS in conjunction with the institution’s VLE, but there was limited opportunity to do this during the project because of interoperability constraints.

LAMS was experienced as relatively easy to use. Institutional commitment to supporting the system was identified as critical to practitioner willingness to invest time in developing LAMS-based designs.

Staff satisfaction with LAMS was mixed. Practitioners were enthusiastic about the capacity to guide students through structured, relatively linear sequences of inquiry activity, and welcomed LAMS as more effective than the institutional VLE in this respect. Pilot users were in general satisfied with impact on the student experience. However, practitioners perceived LAMS as unsuited to supporting design for more open-ended, independent forms of student inquiry and some questioned its ‘added value’ in relation to the institutional VLE for orchestration of student learning.

Students’ responses to using LAMS were mixed but there were clear indications of positive engagement and beneficial impact on learning experiences. Negative responses often related to limitations on the ability to independently move freely back and forward through a sequence.

Introductory workshops followed by opportunities for one-to-one educational development support were welcomed. A need was identified for a pedagogical planning resource to support IBL uses of LAMS, and for case study presentations of LAMS implementations by other practitioner-users.

The project demonstrated that systems integration with the institution’s VLE would be essential to practitioner acceptance of LAMS beyond the pilot.

It also suggested that most practitioners would be unlikely to look beyond the institution for technical support if the system were to be rolled out in the longer term, making provision of in-house ‘help-desk’ style support (supplemented by in-classroom support where needed) essential.

LAMS proved amenable to a range of approaches to design, including reuse of previously authored sequences. However, cultural and other factors appeared to constrain the practice of reuse, and the project confirmed the need for local, community-focused strategies to encourage development of active sharing and reuse practices.

Practitioners’ attitudes to reuse suggested that they might be more open to reusing whole-sequence LAMS-based activity designs when the content is perceived as generic and therefore also directly transferable.

Practitioners responded positively to the prospect of using LAMS to ‘cherry-pick’ design ideas (components of whole sequences) for adaptation to their own needs.

The project findings suggest that practitioners may be more likely to use local rather than remote repositories of LAMS-based designs.

There was some indication that repositories of LAMS-based designs should be as closely integrated as possible with other practice-sharing platforms in the context of community-focused development and innovation for IBL, so that practitioners have a ‘one-stop’ interface to exploring and sharing different pedagogical resources and tools.

The project suggested the value of ‘decoupled’ design and orchestration functionality in design for learning tools, and of future exploration of requirements for tools for students as (co)designers of their own learning.

Main project deliverables were: a report on the impact of using LAMS in design for IBL; recommendations for effective embedding and use of LAMS for IBL, as regards implications for: academic practice; educational development and support; technical support; institutional policy and strategy; development of learning design systems; a portfolio of LAMS-based IBL designs, embedded into case studies.

Download the full report below