Towards Multimodal Sensing and Interaction for Assistive Autonomous Robots

Assistive environments that combine sensors, actuators, and robotic devices have gained attention in recent years, with the potential to provide assistance and support to individuals with disabilities or to the ageing population. One key challenge in designing autonomous assistive environments is to accurately recognise and understand the user’s goals and intentions. To address these challenges, we propose a system with a goal recognition and planning loop that uses data collected from sensors embedded in an assistive environment. We present the general architecture of the system, highlighting the key components available at the time of writing, plus the proposed extensions for the long-term goal of the project. We present data collected in a smart kitchen used by the activity recognition system, as well as a custom tool to simulate and test the data.