Adding sign language animation to the ECHOES multimodal technology enhanced learning environment

Elaine Farrow, Oliver Lemon

Research output: Contribution to conferencePaper

Abstract

ECHOES (Foster et al., 2010) is a multimodal technology enhanced learning (TEL) environment which uses a child-like virtual human avatar to interact with young children (age 5-7) in a series of social games. It provides opportunities for both typically developing (TD) children and those with autistic spectrum disorders (ASD) to practise social interaction and communication skills such as turn-taking and collaboration in a safe and predictable environment. The virtual character is driven by an intelligent agent based on the FAtiMA architecture (Dias and Paiva, 2005), capable of planning, goal-directed behaviour and emotional affect. The system uses recordings of a real child's voice in conjunction with facial expressions and purposefully directed gaze to interact with the child, encourage joint attention, and create emergent narratives. The avatar can also perform a small set of manual gestures in Makaton ("your turn", "my turn", and "all done"). We propose to translate the voice-recorded phrases into both British Sign Language(BSL) and Sign Supported English (SSE) and integrate them with the other aspects of the character's communication including gaze and pointing. This will allow the system to be used by deaf children, both ASD and TD, providing us with insights into this under-researched population and an indication of the potential of technology-enhanced learning to aid their development.
Original languageEnglish
Publication statusPublished - 23 Oct 2011
EventSecond International Workshop on Sign Language Translation and Avatar Technology - Dundee, United Kingdom
Duration: 23 Oct 201123 Oct 2011

Conference

ConferenceSecond International Workshop on Sign Language Translation and Avatar Technology
Abbreviated titleSLTAT
CountryUnited Kingdom
CityDundee
Period23/10/1123/10/11

Fingerprint Dive into the research topics of 'Adding sign language animation to the ECHOES multimodal technology enhanced learning environment'. Together they form a unique fingerprint.

  • Cite this

    Farrow, E., & Lemon, O. (2011). Adding sign language animation to the ECHOES multimodal technology enhanced learning environment. Paper presented at Second International Workshop on Sign Language Translation and Avatar Technology, Dundee, United Kingdom. http://vhg.cmp.uea.ac.uk/demo/SLTAT2011Dundee/3.pdf