A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids

Abhijeet Bishnu*, Ankit Gupta, Mandar Gogate, Kia Dashtipour, Ahsan Adeel, Amir Hussain, Mathini Sellathurai, Tharmalingam Ratnarajah*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In this paper, we design a first of its kind transceiver (PHY layer) prototype for cloud-based audio-visual (AV) speech enhancement (SE) complying with high data rate and low latency requirements of future multimodal hearing assistive technology. The innovative design needs to meet multiple challenging constraints including up/down link communications, delay of transmission and signal processing, and real-time AV SE models processing. The transceiver includes device detection, frame detection, frequency offset estimation, and channel estimation capabilities. We develop both uplink (hearing aid to the cloud) and downlink (cloud to hearing aid) frame structures based on the data rate and latency requirements. Due to the varying nature of uplink information (audio and lip-reading), the uplink channel supports multiple data rate frame structure, while the downlink channel has a fixed data rate frame structure. In addition, we evaluate the latency of different PHY layer blocks of the transceiver for developed frame structures using LabVIEW NXG. This can be used with software defined radio (such as Universal Software Radio Peripheral) for real-time demonstration scenarios.

Original languageEnglish
Title of host publication2022 IEEE International Conference on E-Health Networking, Application and Services
PublisherIEEE
Pages75-80
Number of pages6
ISBN (Electronic)9781665480161
DOIs
Publication statusPublished - 21 Dec 2022
Event2022 IEEE International Conference on E-health Networking, Application and Services - Genoa, Italy
Duration: 17 Oct 202219 Oct 2022

Conference

Conference2022 IEEE International Conference on E-health Networking, Application and Services
Abbreviated titleHealthCom 2022
Country/TerritoryItaly
CityGenoa
Period17/10/2219/10/22

Keywords

  • Audio-Visual Speech Enhancement
  • Downlink
  • Frame structure
  • Hearing Technology
  • Physical layer
  • Uplink

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Health Informatics

Fingerprint

Dive into the research topics of 'A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids'. Together they form a unique fingerprint.

Cite this