News and Information

About the ACETS Project

The following is a paper submitted for the first IVIMEDS conference in St Andrews June 2002.

A strategy for encouraging the use of digital Re-usable Learning Objects (RLOs) in medical education.

DAVID DEWHURST

The Joint Information Systems Committee (JISC) recently invited bids under a programme ‘Exchange for Learning (X4L)’ the primary focus of which was to explore the re-purposing of existing digital learning objects for use in both higher and further education. The total funding for this programme was ~£700k over three years. One successful bid (ACETS: Assembling, Cataloguing, Exemplifying/Embedding RLOs in practice. Testing with students and Sharing outcomes with the HE and FE communities; £197,000 over three years), concerns the use of RLO’s in medical and healthcare education and clearly has synergies with the IVIMEDS project. The project is led by the University of Edinburgh Medical School, and also involves a number of partner institutions (the Medical Schools of the universities of Cambridge, Birmingham, Newcastle (LTSN-01) and two FE Colleges: Suffolk and Edinburgh Telford College).

This paper briefly describes some of the background and the strategy proposed in the ACETS bid.

Introduction

"Within less than two student generations, communication and information technology (C&IT) has been repositioned as an integral component of the medical school environment". Medicine and health sciences have often provided the groundbreakers and proving grounds for learning technology practice and tools, which have later proved to be of great benefit to the human and animal healthcare sector as a whole. Thus, most medical schools in the UK have developed specialist VLEs and have been early adopters of advanced metadata systems such as the National Library of Medicine’s Medical Subheadings "MeSH" for cataloguing and describing learning resources.

However, despite very significant investment in developing, and widening access, to digital resources (e.g. Computers in Teaching Initiative (CTI); Teaching & Learning Technology Programme (TLTP); Learning and Teaching Support Network (LTSN)) and access to national collections of learning resources such as BIOME (OMNI) at Nottingham and the Bristol Biomedical Image Archive, the actual use of these resources by teachers and students has been patchy at best. Each project carries with it the enthusiasts and the early adopters but persuading the vast majority of teachers to engage with these resources at more than just a superficial level (i.e. informing students of their availability over the network) has always been the problem. This proposal seeks to address this by exploring strategies which will facilitate the integration of digital resources into practice and investigating the impact that this has on the students learning experiences. A number of factors contribute to the problem and it is clear that any solution must have technological, pedagogical and semantic components and that these must be both integrated and mutually compatible.

a lack of cognitive awareness and expectation by both individuals and institutions as to how these resources can be of use to them
inadequate descriptors of RLOs - lack of common language, terminology and other semantic aspects
teachers do not have the time and, in some instances, the creative ability, to integrate existing resources into their teaching
a lack of documentation, exemplars and other expository material to better enable teachers to help teachers to embed RLOs into their practice
technological incompatibilities across the sector which inhibits access to RLOs by teachers

All of the partners are actively involved in developing local solutions to these issues and loose collaborative structures are already in place between the lead HE institutions and the two FE institutions.

The ACETS Project

This collaborative project will develop and evaluate processes to achieve sustainable use of RLOs by practitioners across medicine and related disciplines in the HE and FE sectors. It will build on existing and emerging experience and expertise in both the HE and FE medicine and health science communities to deliver mechanisms for effective repurposing of existing digital resources.

The project recognises that although enhancement of student learning is the major focus, this can be best-achieved through persuading teachers to embrace technology (existing digital resources and delivery mechanisms) and embed it into their practice. Central to this is making this process easier, less time consuming, intellectually satisfying and rewarding through positive student feedback.

Developing much more useful, teacher-centred, descriptors of RLOs and ensuring they are readily available is essential. This will be achieved, in part, by further developing existing JISC-funded databases and dynamic uptake technologies to explore, with practitioners, innovative ways in which RLOs can be (a) described using basic and extended metadata and educational and subject relevant taxonomies and (b) catalogued to make them more useful in a wide variety of applications and scenarios. The project will also build on existing and ongoing work both within the consortia institutions and the wider FE/HE sectors to develop approaches and means to import and export digital resources as RLOs and to broker them within their own communities.

Recognising that facilitating identification and selection is only one step along the path to ensuring that the use of RLOs is embedded into practice and used to enhance student learning, the project will also investigate creative ways of unlocking the content of a variety of these objects across different disciplines and levels in the HE and FE sectors. Teachers will be commissioned to develop ‘wrap-arounds’ to facilitate the process of embedding selected RLOs, appropriate to their local needs, in their practices. The process and effectiveness of using the repurposed RLOs in a variety of student learning activities will be evaluated with student groups and the results of this evaluation added to the database as further reflective guides to practice and use. The process will be fully documented and the derived exemplars, which themselves will become new RLOs will be catalogued and shared with the wider community where the transferability of the process can be investigated further.

Aims and Objectives

a) Aims

Define methods of cataloguing, describing and making RLOs more accessible both at their semantic and technical levels to teachers in both the HE and FE sectors, embracing local, national and international practice and developments in this area as appropriate
Investigate ways in which RLOs can be embedded into a range of educational practices across the HE and FE sectors i.e. methods of repurposing RLOs
Explore how the re-purposed RLOs can support student learning activities in a variety of contexts and for a variety of types of learning outcome
Evaluate and document the outcomes (both processes and products) of the project and disseminate these to the wider HE and FE communities.

b) Objectives

To achieve these aims a number of specific objectives have been identified for each year of the project:

Year 1 [Assemble; Catalogue]

Identify, by consultation with practitioners in the consortium constituency, a representative subset of learning objects covering different interaction levels, media types and levels of aggregation. This sub-set will draw on resources from JISC-funded (e.g. on-line journals, Bristol BioMed, Derweb, Visible Human Project, LifeSign, MAAS learning resources and even EDINA resources such as maps which could be used for 'innovative' uptake e.g. in epidemiology) and other projects and will comprise a minimum of 100 learning objects from each of: peer reviewed courseware packages; text; images; video, and a minimum of 25 learning objects from each of: animations; audio; simulations; 3D models.
Adopt, and where necessary adapt, existing taxonomies and metadata (in particular those from the IMS Project) to provide a ‘rich’ description of the RLOs which will better enable teachers to make informed selections of RLOs to embed in their local practice. Thus the metadata will include an expanded range of properties (including pedagogical e.g. learning outcomes, student activities, level, peer review) of the identified RLOs.
Catalogue the RLOs using the identified taxonomies, developing cataloguing and good practice guidelines for project participants.
Work with the BIOME project to extend the existing BIOME database (adding the identified sub-set of RLOs) to include deeper metadata, that both catalogues technical and semantic aspects of the object in question and associated pedagogical uses and interventions.
Create a project infrastructure including a web-based file server, a web-site with online tools to support the project’s learning community such as discussion boards and group workspaces.

Year 2 [Exemplify/Embed; Test]

Commission and prioritise proposals from practitioners in the wider consortium community to develop exemplars of ways in which the RLOs from the sub-set of relevance to their ‘local’ needs can be transformed (repurposed) into a form which they can then embed in their own practice.
Manage the process of creating exemplars by engaging with the identified developers, linking them into the project and with each other and seeking, by the use of online and face-to-face communication, to develop a community of practice around the theme of RLOs.
Produce an extensive and representative series of exemplar ‘wrap arounds’ linked to learning outcomes and embedded into educational practice. Thus an image may be transformed to a useful learning object by creating a ‘wrap-around’ comprising objectives or outcomes, one or more student activities, assessment and feedback. The complexity of the ‘wrap-around’ will depend on the practitioners own needs This might be achieved through individual practitioners working alone or by small groups of practitioners from different institutions.
Trial the new sets of RLOs (original RLO(s) plus ‘wrap-around’) with students to provide first stage evaluation.
Explore how a limited set of RLOs can support teaching and learning in related disciplines such as veterinary medicine, dentistry, nursing, biomedical science and more distantly related topics such as beauty therapy and health studies.
Document, through reflective diaries, the process of transformation to provide a minimum of three case-studies.

Year 3 [Test; Share]

Evaluate the transferability of the process to other institutions (HE and FE) who have not been involved in the project. This would include assessment across different levels in the same discipline e.g. undergraduate and postgraduate medicine, and across disciplines e.g. veterinary medicine, PAMS.
Organise a series of workshops with staff from a wider group of HE and FE institutions with an ‘enabling’ focus. These would be hosted in a number of institutions (HE and FE) and be led by experienced teachers from the consortium. These would be ‘show and tell’ events where sharing experiences, and developing skills would be the primary objectives.
Provide ongoing support for practitioners both individually and as a community.
Ensure, through involvement of the relevant LTSN centres and the Regional JISC Support Centres, that all heads of the education boards are made aware of outcomes of the project.

Throughout the three years of the project lifespan key members of the project team will liase with, and where necessary partake in, the various CETIS special interest groups looking at IMS specifications in order to inform the development of the project and to feedback the formative outcomes of this project to these groups.

Project deliverables

A content-rich database of selected RLOs using enhanced metadata tagging and cataloguing, and made accessible through the web by integrated database searching and passing XML data between systems.
Extended or newly created taxonomies (FAILTE http://www.failte.ac.uk/) of metadata tagging protocols, which build on existing taxonomies and use existing and emerging IMS specifications, and which address the technical, semantic and pedagogical information required to best reuse RLOs - they would include areas such as learning outcomes, support resources, assessment level, as well as content. They will integrate with existing taxonomic protocols such as MESH, Best Evidence Medicine, SeSDL and the British Education Index and will look at how they, IMS and any needed extensions can be developed and applied meaningfully with additional tables established in BIOME and a methodology for maintaining them agreed with the user community.
An associated and expanded database of peer-evaluated exemplars of how RLOs can be embedded in practice and used to deliver enhanced quality student learning. This database which will be accessible from the ACETS central web server and will be searchable at both subject and pedagogical levels.
Detailed case studies of practitioners’ engagement with the processes of reusing RLOs within their teaching practice, covering a representative sample of disciplines and types of object. This will include material from existing collaborative work on selecting and identifying processes, issues and outcomes in this area. The studies will also examine the issues surrounding the levels of RLO aggregation and the cataloguing of components as well as the aggregated whole and investigate the relationship between an objects degree of inherent process and its reusability.
A significant contribution to the publicly available banks of learning objects (including clinical/diagnostic/illustration images, video and other materials) from several of the consortia institutions, both as a substantive contribution in their own right and as part of the research process of integrating both existing and new materials into a common reusable learning object framework. This will also be part of the definition and application of interoperability methodologies for VLEs and their embedded content.
Testing, evaluation and development specifications and standards for delivery mechanisms (such as VLEs) to ensure interoperability. This is particularly an issue in medicine where almost all institutions have purpose-built VLEs for their medical courses and need to adopt an open-standards approach to facilitate the sharing and reuse of RLOs. The project will also need to ensure integration with the commonly available VLE systems found in both HE and FE sectors and with wider developments such as IMS, which is gaining wide acceptance with developers of VLE systems such as WebCT and Blackboard. Technical issues beyond VLEs will also be examined such as access from home over dial-up and broadband and institutional computing clusters.
The collective result of 1-6 will be a ‘re-usable learning object framework’, which as an entity will draw the three areas of technology including interoperability and system transparency, pedagogy and semantics together to provide an integrated approach to the use of RLOs. This framework must tie in directly with work performed in other X4L projects both in strands A and B to achieve a sector-wide approach.
Development and publication of a "Users Guide for Reusing Learning Objects". This will be a step-by-step guide for selecting and repurposing RLOs in a wide variety of settings and applications, covering both pedagogical and technical issues of reuse, in a for appropriate to the target audience of teachers and academics at all levels and in all disciplines within the tertiary sector.
A range of dissemination activities including a report detailing major project outcomes and highlighting the research questions which the project has sought to answer. This will include independent project evaluation and pointers to the technical, pedagogical and semantic developments achieved by the project.

Pedagogic Context

The medical and health science communities now have access to a wide range of digital resources with the potential to be used in a number of teaching and learning situations ranging in complexity from courseware packages down to single images and text files. Many of these have resulted from significant funding through Government agencies such as the JISC, and TLTP and are provided either free or at low cost. Despite this the perception and evidence is that teachers make relatively little use of them and opportunities for using these to improve the quality of the student learning experience are missed. The strategy of simply making resources available to students without contextualising their use, embedding them into practice and assessing them does not work. Inherently teachers are conservative and will only embrace innovative teaching opportunities if they require minimal time commitment to explore and evaluate what is available, and little effort to implement.

Current metadata descriptions of RLOs lack the necessary levels of detail and transparency which would help teachers to decide if a resource was likely to be useful to them. In our experience this information needs to do much more than simply describe the content and should also contain pedagogical information such as learning outcomes, level, student activities, self-, and formative assessments. However, even if this were available and they could select a range of resources which would augment learning they still need to be able to embed them into their teaching. This again requires significant time and some creativity. One strategy to enable this is to develop ‘wrap-around’ materials aligned to local needs which, in addition to facilitating implementation also gives the teacher ‘ownership’ (which avoids the ‘not-invented-here’ issue) of the new RLOs. If materials can be developed to satisfy a range of local needs e.g. different levels of the same course, different courses then we have a rich variety of exemplars of ways in which RLOs may be repurposed. Once these materials have been created they can either act as templates or examples of good practice for teachers in other institutions who can then create their own or use the ready-made ‘off-the-shelf’ materials if they fit their needs. Local editing will be facilitated by the wrap-arounds being produced in standard formats such as web, pdf, MS Office (Word, Excel, PowerPoint). There is also an in-built multiplier effect where a sub-set of RLOs are repurposed to provide a further, often larger, sub-set.

For example, a RLO such as a image of a chest X-ray might have a wrap-around which described outcomes (e.g. having worked through the student activities you will be able to describe the primary features of a normal chest X-ray), level (e.g. suitable for first year undergraduate medical students, second year nursing, physiotherapy students etc), student activities (e.g. list the main features of the x-ray and compare to the ‘expert’ description; name the features labelled a, b, c; a case-based learning activity on respiratory medicine with the RLO as one component; a student-centred activity on clinical examination with the RLO as one component); self-assessments (could be MCQs or more sophisticated assessments), and a formative assessment (could be tutor- or peer-assessed examples).

The major thrust in year two is to involve as many teachers in as many institutions (across both HE and FE sectors) as possible to explore strategies for developing ‘wrap-around’ materials and integrating the repurposed RLOs into their teaching. The success of these strategies will be evaluated with students who are the recipients of this learning activity. Mini-project proposals for funding from teachers wishing to engage in these activities will be sought and prioritised against agreed criteria. Some of these may well take the form of student projects (e.g. special study modules) and these will be encouraged. The aim will be to fund as many mini-projects as possible. These might range in complexity from an example of a teacher who wishes to repurpose a small number of similar RLOs (perhaps a series of anatomical images to address a specific course need) to one who wishes to combine a large number of different types of RLOs to provide a highly sophisticated medical ‘case’ which integrates a large number of strands of a course (e.g. anatomy, physiology, pathology, diagnosis, treatment, ethics, social sciences etc). The project will also seek to investigate strategies for repurposing different types of RLOs (e.g. images, simulations, video or audioclips) and for repurposing the same sub-set of RLOs for different but closely related HE disciplines e.g. veterinary medicine or dentistry.

The emphasis will probably be on relatively low-level learning objects such that the repurposing process (RLO plus wrap-around) results in new low-level but much more usable RLOs. The two lead FE Colleges who will be responsible for exploring ways in which RLOs from the same sub-set can be repurposed for local use in related disciplines in the FE sector (e.g. beauty therapy, sports therapy). It is expected that the JISC Regional Support Centre network will be involved in recruiting other FE institutions into the project either as developers of ‘wrap-arounds’ or as evaluators of effectiveness/impact on learning.

Evaluation Strategy

An evaluation strategy will be developed early in the project in consultation with JISC and evaluation will be managed by LTSN-01. It will focus on monitoring indicators for formative improvement of the project methodology and outcomes, and determining the pedagogic impact of the outcomes. The project will conform to any overarching programme evaluation and monitoring, and all reports will be reviewed by the Advisory Group to ensure that it is meeting its operational targets.

Broadly, the strategy will seek to explore the following: the merits and shortcomings of the project (including gaps or omissions), what aspects of the project worked well and why (and, conversely, which were less successful), lessons to be learned and growth points to be developed, how the project was interpreted by the partners and how this affected the outcomes. Reflective diaries, feedback forms and focus groups will play a significant role in testing, formative and summative evaluation.

The evaluation process will be used to help us to support judgements on what is generalisable across a range of subjects, in FE and HE context. As we plan to adopt a user-centred approach in the design of the pedagogic process, qualitative user feedback and the quantitative behaviour of users of learning objects will be recorded from the website. This allows the sampling and analysing of qualitative and quantitative evaluation data. Taken together, these data should be suitable to evaluate the project output, e.g. to find out which users benefit most from the service, which users cannot be served satisfactorily, and where the system does not work properly.

The educational effectiveness of the repurposed RLOs will be evaluated with students in the institutions in which they were developed. The project team will develop suitable evaluation tools targeted at both teachers and students which address both process and product issues e.g. usefulness of the newly developed RLO database, accessibility of the RLOs, features of the repurposing process, methods of integration into practice, student perspectives, impact on learning, learning outcomes). The centralisation of the evaluation process (development of appropriate tools, dissemination, data collection, analysis, reporting) will ensure standardisation of the evaluation approach, and collection of comparable data.

Dissemination Strategy

Dissemination of outcomes will be managed by the LTSN Centre for Medicine, Dentistry and Veterinary Medicine who are uniquely positioned to co-operate with key players in the disciplines they represent and related disciplines (LTSN Centre for Health Sciences and Practice; LTSN Biosciences) ensuring that outcomes will be rolled out to the whole medical/health education community. The project has sought support from the JISC Regional Support Centres and several of these have already responded positively. The project will also seek to involve JISCAssist, Netskills and other successful JISC X4L projects in joint dissemination activities. Early in the project, advice will be sought from consultants with experience in the field to define an optimal dissemination strategy, which is likely to include: distribution of marketing information (leaflets and posters), eCommunications and eNews, presentations by project members at existing high profile events, and publications.

Dissemination within the wider project consortium will be through hands-on workshops which will focus on ensuring that as many practitioners as possible develop the necessary skill-set to repurpose RLOs and embed them into their teaching. The hope is that many of these will become evangelists in their own institution.

Project outcomes, case studies etc will be made available through the project web site, mounted on the ACETS server in Edinburgh, which will seek, through interactive features such as discussion boards and chat areas, to develop an on-line learning community to support teachers. The full project report, details of the evaluation, exemplar RLO wrap-arounds will also be widely disseminated via the website.

Technical and metadata/taxonomic developments will also be disseminated through bodies such as CETIS and it’s special interest groups (SIGs), JISC, ALT, and made publicly available. Aspects such as integration with VLEs will also need to be disseminated to practitioners and systems managers through special meetings perhaps organised to be coincident with national conferences such as ALT-C

The project will host a dedicated email list for those interested in receiving regular updates, and special interest lists for particular topics. Significant advances/events will be publicised via the email list as well as through a range of relevant lists throughout Europe in order to continue to identify potential sources of learning objects and promote the project outcomes. A number of high quality conferences are devoted to the publication and dissemination of the study of pedagogy in health education and of new technologies in learning. Partners will be encouraged to document and reflect on process and findings throughout the lifetime of the project and to submit papers for relevant conferences. For example: The Association for the Study of Medical Education (ASME) Association for Medical Education in Europe (AMEE), EdMedia, Online Educa, Improving Student Learning Symposium, ALT-C and CETIS.

Contact

The ACETS project Director is Dr David Dewhurst , Director of Learning Technology, Faculty Group of Medicine & Veterinary Medicine, University of Edinburgh, Hugh Robson Link Building, 15 George Square, Edinburgh EH8 9XD.Tel: 0131 651 1564: Fax: 0131 651 3011: e-mail: d.dewhurst@ed.ac.uk

Abstracts submitted, accepted plus conference reports

ACETS is a JISC funded project