Developing a standardised evaluation methodology for computer based learning materials

The UK-based EASEIT-Eng project (http://www.easeit-eng.ac.uk/) aims to encourage or enhance the usage of existing engineering computer-based learning materials by enabling an academic tutor to make an informed choice from a range of evaluated software. This is being achieved through: (1) establishing standardised evaluation criteria and hence an evaluation methodology, (2) conducting evaluations, focusing primarily on producing case studies of actual implementations of software and (3) providing web access to these evaluations through a dedicated, fully searchable database. The evaluation methodology described in this paper was developed with the intention that the resource demanded by an evaluation would be: (1) manageable from the perspective of the evaluator, (2) acceptable from the perspective of the academic tutor delivering the software and (3) sustainable in the sense that further evaluations could be conducted with minimal funding beyond the life of the project. The development of this unique methodology also drew on established good practice and procedures, including particularly the ANSI Dental Informatics Guidelines ^[1] and the NEEDS database ^[2]. The resulting Evaluation Manual, which has been in use since September 2000, offers full guidance on the principles behind the approach to evaluation, details on how to conduct each part of the evaluation and access to the evaluation tools. The Manual should enable new evaluators to train themselves to conduct evaluations and use the tools developed. To date, EASEIT-Eng has 42 case studies and 39 reviews covering 53 different software implementations (over 40 different products) in all the major Engineering subject areas, either completed or in progress. Academics from over 20 different UK universities have participated in the project.