JISC

This project will make an open source version of an existing online peer moderated marking system, for the assessment of group work, already in use at Loughborough University since 1998. WebPA will also include a number of significant enhancements that will help integrate good practice being developed locally and at our partner site at the University of Hull and will benefit the entire JISC community. It will build upon existing evaluation and developments to the system, across a range of subject disciplines.

The project will use a service-orientated approach within the e-Framework for Education and Research. The new system will be piloted at Loughborough University and the University of Hull and be developed within the context of each institution’s VLE and online delivery (Moodle at Loughborough and portal/learning services at Hull).

Aims and objectives

The project aims to:

• Make significant enhancements to the online peer assessment system ‘webPA’ (WebPA v0.8), designed at Loughborough University, to incorporate changes based on evidence from pedagogic research undertaken at Loughborough and Hull University; and technical evaluation to address the need for more learner centred provision.
• Provide a mature open source peer assessment application (WebPa OS v2.0) with the support of an active development community provided through the Higher Education Academy Engineering and Physical Sciences Subject Centres.
• Create and disseminate models of pedagogic use based on a current literature review, findings from student and tutor evaluations regarding the best practice in terms of formative and summative assessment methodologies and with the engagement of employers.
• Advance the service oriented approach for application development within the JISC e-Learning Framework.

The project will carry out the following tasks.

1. Literature review and exploration of potential pedagogical models to evaluate
2. Ongoing updates to existing application based on research and feedback.
3. Work closely with Hull University in developments, both pedagogic and technical implementation and further developments.
4. Re-coding and re-purposing of application to make it open source and service oriented.
5. Student and tutor qualitative and quantitative evaluation.
6. Documentation of technical approach and ongoing evaluation
7. Guides and case studies for wider community
8. Support for adoption in at least 3 further institutions

Project methodology

• Literature review and exploration of pedagogical models to evaluate The project will perform a literature review on current practices on the use of online peer and self-assessment, with respect to the provision of flexible delivery.  Also reflect on current reports eg. Cox Review and Roberts Report to make recommendations to the development of the application in light of the government recommendations on making graduates more employable. The project will explore various pedagogical models and research questions about online peer assessment related to flexible delivery, learner-centred provision and the engagement of employers.
• Host the online peer assessment for Hull University In the first year of the project, Loughborough University will host a version of the existing WebPA software for Hull University.  Hull will use it with students across two departments in Semester 2 of the 2006/2007 academic year.  They will pilot models of use based on current research-informed models by both partner institutions. 
• Ongoing updates to existing application based on research and feedback Throughout the project the application will be revised to support the outcomes of the pedagogic research being undertaken into the different models for use.  Each addition and enhancement will be documented and tested before being released.
• Re-coding and re-purposing of application to make it Open Source and Service Oriented To ensure flexibility and reuse, the existing application (WebPa v0.9) will be refactored to decouple it from the specific technical environment at Loughborough University.  The core functionality of the application (personal info, group management, etc) will be re-modelled to map to suitable service specifications, including the IMS Enterprise Services Specification.
• Student and Tutor qualitative and quantitative evaluation During the first year of the project the existing version (webPA v0.9) will be evaluated with both Loughborough and Hull’s students and tutors to provide a large quantity of qualitative and quantitative data.
• Documentation of technical approach and ongoing evaluation At the end of each year the project staff will document the technical approach used and throughout the project will contribute to community SIG’s, blogs, wiki’s and the e-Framework knowledge base on the successes and failures of integrating within a service based environment as well as information on starting and sustaining open source projects
• Guides and case studies for wider community During Year 1 the Subject Centres will help to build up and disseminate resources about online peer assessment.  During Year 2, with the help of the Subject Centres, the project will provide formal documentation in support of the application and its use in various models of delivery and across disciplines.
• Support for adoption in at least 2 further institutions The Higher Education Academy Subject Centre partners in this project are ideally placed to ensure dissemination and further uptake of this tool.  The Centres will identify at least 2 other institutions willing to pilot the WebPA v1.0 software and with the help of the project team, support the process. 

Technological approach

The redevelopment of the existing system (WebPA v0.8) will address two main technical goals.  At the application level an object-oriented design will facilitate maintenance and future updates to the code. This will be particularly useful for institutional adoption and localisation where functional changes are needed for local versions of the application. We intend all code developments to be incorporated into the community code base built up around the software.  At the enterprise level, the service-oriented approach will help institutions with the deployment of the application, specifically when integrating with a VLE, or other information systems, where no existing interoperability has been developed. The system APIs will be available using both REST and Web Services technologies and will restrict any integration effort to those services that require institution specific data (such as student info, module cohorts, etc).  Specific integration experiences for local institutional needs will also be shared with the community so others with similar technical environments can quickly deploy based on that knowledge.

A version control system such as Subversion will be used to help us manage and incorporate updates and additions made to the source code of the application, particularly those from third parties. To validate any changes a rigorous testing framework will be employed. This should produce a more robust system, as well as promoting the open source nature of the project by allowing us to rapidly incorporate additional code from the community.

The project team will contribute to the national knowledge base on coding for the e-Framework environment and coding to the interface specifications laid out by IMS.  We will also make contributions on how easy was it to tailor the service implementations to the institutions back-end systems.

Anticipated impact

This project will benefit the community greatly, producing a mature open source application for online peer and self-assessment available for free, with the potential to have an impact on all students within all disciplines.  The application will be written in PHP and use a mySQL database making it within the technical and financial capability of many institutions and also be optimised for a LINUX server set up. The project intends to have the following deliverables:

• Open source web based peer assessment application (WebPA OS v2.0)
• Documented APIs and web services descriptions for integration of software
• Online documentation and supporting information
• Transferable models of good practice for embedding within curricula across multi-disciplinary module assessments
• Evaluation report analysing areas in which the project has been successful; identify opportunities for further development; and highlighting the implications and challenges of implementing online peer assessment.
• Evaluation of learner’s experiences of using online peer assessment
• Use cases describing various implications of different marking schemes and group sizes when using online peer assessment.
• Documentation and evaluation of service definitions used and developed to go into the e-Framework knowledge base.

Lead institution 
Engineering Centre for Excellence in Teaching & Learning (engCETL) Loughborough University

Project partners

• HEA Engineering Subject Centre Loughborough University
• HEA Physical Sciences Subject Centre University of Hull
• eServices Integration Group Academic Services University of Hull