JISC

The JISC reference model projects ran initially for 12 months from March 2005, with several of the projects being given a 6-month extension. The outputs from the reference model projects were both diverse and complex, to make the outputs of the reference model projects more accessible and to aid understanding of the reference mode projects, JISC commissioned this project.

Executive Summary

The reference model projects were:

• Course Validation Reference Model (COVARM)
• Framework Reference Model for Assessment (FREMA)
• Learning Activity Design in Education (LADiE)
• e-Portfolio for Lifelong Learning (eP4LL)
• Personal Learning Environments (PLE)
• Exchanging Course Related Information (XCRI)

Only 4 of the 6 projects included in this work were originally funded under the reference model banner. PLE and XCRI were funded through other mechanisms and then added to the reference model programme as they included some effort aimed in the direction of reference models. The result is that the artefacts that those two projects produced have the least in common with the other reference model projects.

Given the level of funding available to the synthesis project it has had to focus closely on the deliverables
defined in the invitation to tender (ITT):

Collation of the approaches and outputs from the six Reference Model Projects into a preserved collection that can be browsed with high degree of openness and ease of access.

Feedback recommendations on approaches used by Reference Models to current set of Domain Maps projects.

We have been in discussions with the P-SPEX and ADOM domain model projects, and this has helped to lead to greater commonality of approach between the projects. We will also be sending them copies of the final report.

We have also been working with the FREMA team in the development of the e-Framework upper levels (eFUL), which builds on the approach taken by the FREMA project and the HILDA project to develop a coherent, community based model of the higher education domain that can provide a method of exploring, locating and understanding the information in the e-Framework

Advice and guidance aimed at institutions and stakeholders on how to use domain maps to support the design, development and implementation of ICT systems to support the delivery of learning and teaching. There are three levels at which our advice and guidance apply:

Firstly, in model driven development, a domain model for the problem domain has to be generated as the underpinning activity for any production of code. In such development, particularly where it follows the OMG standard for Model Driven Architecture, the model has the potential to become the operational system.

The second level of application of these guidelines is in the use of an existing model-based knowledgebase that addresses the problem-domain and its context. For the UK HE sector such a knowledgebase may be provided by a rich population of the High Level Domain Architecture (HILDA) model.

Thirdly, the development process will yield deliverables for the project itself and for the HILDA knowledgebase, for which guidelines are provided to place them correctly within the knowledgebase.

Synthesis of reference model work undertaken into a coherent accessible whole for wider dissemination. The outputs from the reference model projects were analysed and two kinds of syntheses produced.

The first synthesis was a methodological synthesis. As a result of analysis, we derived a single lifecycle method from the four most methodologically similar reference model projects. We then used this lifecycle method to compare and contrast the methods and design artefacts used in the four projects.

The lifecycle method provides a general method that may be adopted by future reference model projects.

The second synthesis approach was at the domain level where High Level Domain Map (HILDA) was used to supply a framework that positioned the individual projects in a higher-level domain map of Higher Education.

Evaluation of the outcome of the reference model programme against its aims, objectives and assumptions.

The 6 reference projects exhibit considerably diversity in origin, methodology and nature of deliverables. This makes it is difficult to provide a fully levelled evaluation of results across all projects yet also reflect the true value of their outcomes within the e-learning framework.

To introduce a degree of conformity across the project profiles a small online survey was performed designed to capture views of a key stakeholder from each project. Questions were structured in line with the requirements for a reference project as detailed in JISC Circular 10/04. The goal was to gain insight to the projects against these common criteria.

Results are analysed by question across projects, and the respondents original answers are supplied in an appendix to this report. In summary we conclude that the core question to answer is 'to what extent have the projects collectively advanced the e-Learning Framework (ELF)?' This is best answered in two parts:

• Furthering Domain Knowledge - Each project developed a body of work that delves deeply into its chosen domains and provided improved understanding in its area of the HE environment.
• Adding further substance to the ELF – Through the development of knowledge within each domain the ELF gained greater depth.

Our judgement is that the sum of the effort across the projects was to further advance the ELF.

Extraction of further SUMs from the reference model projects. The state of this activity both inside and outside the synthesis project is:

• In the time available the synthesis extracted two draft SUMs from the COVARM project and submitted them to the UK e-Framework editor
• The UK e-Framework editor has already extracted a SUM from the eP4LL project.
• The UK e-Framework editor has undertaken, as a result of negotiation with the synthesis project, to document the XCRI Interoperability standard in the e-Framework.

Recommendations to JISC on the:

• Value of a domain model approach
• On further development activity
• General topics associated with the reference model projects

Overall we are supportive of a domain model approach to underpin development in the e-learning and allied domains. We considered alternative ways to represent domain models in our work, and decided that adopting the approach espoused by the High Level Domain Map (HILDA) represented the best available approach to representing the work of the projects at domain level.

The recommendations for future work are divided into 3 parts:

• Work that might be undertaken to ensure that reference model (and related work in SUMs, domain mapping etc) might be made more useful.
• Work in general areas where it might be appropriate to undertake more reference model creation that would help to move forward the e-Framework as a whole.
• Existing work in the individual projects that could be extended to provide greater value and build on the existing work.

We note that much of the information about future work information was derived, as agreed with the JISC, by consulting staff who worked on the reference model projects. Time limits on the synthesis project precluded deeper investigation of these topics, and we strongly suggest that that the suggestions need to be further validated before any of them are selected for future funding.

The synthesis project produced 42 recommendations to JISC. These recommendations a broad spread of aspects of the reference model projects and the reference model programme. To simplify scanning of these recommendations we have reduced them to four overarching summary recommendations, and centralised all recommendations at the beginning of the full report together with page references to the recommendations’ original context.

The 3 summary recommendations are:

• Recommendation Synthesis projects should be funded alongside, rather than after, programmes to maximise the benefit. This would mean providing time for individual projects to work with the synthesis project and ensuring that the synthesis project was providing benefit to the individual projects.
• Recommendation Reference modelling, domain modelling, and the production of SUMs and other project outputs is far more effective where these are provided by or elicited from community of practice, and JISC should therefore ensure that such work is closely tied to existing CoPs.
• Recommendation More guidance should be given to projects working towards part of e-Framework (including reference models and domain models) so that the outcomes of projects can be combined and re-used more effectively. This includes guidance as to effective design, development and deployment lifecycle methods, and guidance as to project outputs.