Development of a Mixed Reality Game for Simulation Based Education

Mixed reality allows for new ways for learners to enrich their learning experience. By enabling the use of 3D virtual content in a tangible space, it offers a more immersive digital learning environment than traditional on-screen methods. This paper reports on the use of the Mixed Reality (MR) Microsoft HoloLens to create a mixed reality Vocational Educational Training (VET) serious game. The main challenges of this investigation are the use of tangible objects as an interface to a virtual learning environment, which is achieved through low-latency, real-time tracking, allowing for believable interaction with virtual content. This is examined through the creation of a stone grinding application for training college-level stonemasons, whereby the learner attempts to cut a dogleg into a virtual stone block using a real grinder. Cyber-physical systems (CPS) are utilised to realise this as part of the digital pedagogy. Currently, the vocational training is strung by regulations which do not allow students to use power tools such as an angle grinder within the college. By using a virtual stone block, a physical grinder as a tangible interface can be used without the associated safety risks. The application has been co-designed and developed in association with several further education colleges in Scotland and has received appraisals from teachers and learners. On-site demonstrations have been carried out, with the mixed reality operation alongside a PC-based, mouse-operated application. Early feedback has suggested that this is a system that the colleges are interested in and could provide a real use in training. This work builds upon research carried out for the BEACONING program, part of the Horizon 2020 framework.