5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids

Ankit Gupta; Abhijeet Bishnu; Mandar Gogate; Kia Dashtipour; Tughrul Arslan; Ahsan Adeel; Amir Hussain; Tharmalingam Ratnarajah; Mathini Sellathurai

5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids

Ankit Gupta
, Abhijeet Bishnu
, Mandar Gogate
, Kia Dashtipour
, Tughrul Arslan
, Ahsan Adeel
, Amir Hussain
, Tharmalingam Ratnarajah
, Mathini Sellathurai

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Over twenty percent of the world's population suffers from some form of hearing loss, making it one of the most significant public health challenges. Current hearing aids commonly amplify noises while failing to improve speech comprehension in crowded social settings. In this demonstration, we showcase a proof-of-concept implementation of the world's first 5G and Internet of Things (IoT) enabled multi-modal hearing aid (MM HA) prototype. This integrates an innovative 5G cloud-radio access network (C-RAN) and IoT based transceiver model for real-time audio-visual speech enhancement (AVSE). Specifically, we demonstrate a transceiver model for Cloud-based AVSE which satisfies high data rate and low latency requirements for future MM HAs. The innovative 5G-IoT transceiver application is shown to satisfy HA latency limitations while transmitting raw noisy AV data from an MM HA prototype device to the cloud for deep learning-based real-time AVSE processing and obtaining a clean audio signal.

Original language	English
Title of host publication	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023
Pages	686-687
Number of pages	2
Publication status	Published - Aug 2023
Event	24th International Speech Communication Association Conference 2023 - Dublin, Ireland Duration: 20 Aug 2023 → 24 Aug 2023 https://interspeech2023.org/

Conference

Conference	24th International Speech Communication Association Conference 2023
Abbreviated title	INTERSPEECH 2023
Country/Territory	Ireland
City	Dublin
Period	20/08/23 → 24/08/23
Internet address	https://interspeech2023.org/

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Signal Processing
Software
Modelling and Simulation

Cite this

@inproceedings{5e612e8ef5914221bfef3a0555d5c17b,

title = "5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids",

abstract = "Over twenty percent of the world's population suffers from some form of hearing loss, making it one of the most significant public health challenges. Current hearing aids commonly amplify noises while failing to improve speech comprehension in crowded social settings. In this demonstration, we showcase a proof-of-concept implementation of the world's first 5G and Internet of Things (IoT) enabled multi-modal hearing aid (MM HA) prototype. This integrates an innovative 5G cloud-radio access network (C-RAN) and IoT based transceiver model for real-time audio-visual speech enhancement (AVSE). Specifically, we demonstrate a transceiver model for Cloud-based AVSE which satisfies high data rate and low latency requirements for future MM HAs. The innovative 5G-IoT transceiver application is shown to satisfy HA latency limitations while transmitting raw noisy AV data from an MM HA prototype device to the cloud for deep learning-based real-time AVSE processing and obtaining a clean audio signal.",

author = "Ankit Gupta and Abhijeet Bishnu and Mandar Gogate and Kia Dashtipour and Tughrul Arslan and Ahsan Adeel and Amir Hussain and Tharmalingam Ratnarajah and Mathini Sellathurai",

note = "Funding Information: This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) programme grant: COG-MHEAR (Grant reference EP/T021063/1). Website: https: //cogmhear.org/index.html Funding Information: This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) programme grant: COGMHEAR (Grant reference EP/T021063/1). Website: https://cogmhear.org/index.html Publisher Copyright: {\textcopyright} 2023 International Speech Communication Association. All rights reserved.; 24th International Speech Communication Association Conference 2023, INTERSPEECH 2023 ; Conference date: 20-08-2023 Through 24-08-2023",

year = "2023",

month = aug,

language = "English",

pages = "686--687",

booktitle = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023",

url = "https://interspeech2023.org/",

}

Gupta, A, Bishnu, A, Gogate, M, Dashtipour, K, Arslan, T, Adeel, A, Hussain, A, Ratnarajah, T & Sellathurai, M 2023, 5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids. in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023. pp. 686-687, 24th International Speech Communication Association Conference 2023, Dublin, Ireland, 20/08/23.

5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids. / Gupta, Ankit; Bishnu, Abhijeet; Gogate, Mandar et al.
Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023. 2023. p. 686-687.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - 5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids

AU - Gupta, Ankit

AU - Bishnu, Abhijeet

AU - Gogate, Mandar

AU - Dashtipour, Kia

AU - Arslan, Tughrul

AU - Adeel, Ahsan

AU - Hussain, Amir

AU - Ratnarajah, Tharmalingam

AU - Sellathurai, Mathini

N1 - Funding Information: This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) programme grant: COG-MHEAR (Grant reference EP/T021063/1). Website: https: //cogmhear.org/index.html Funding Information: This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) programme grant: COGMHEAR (Grant reference EP/T021063/1). Website: https://cogmhear.org/index.html Publisher Copyright: © 2023 International Speech Communication Association. All rights reserved.

PY - 2023/8

Y1 - 2023/8

N2 - Over twenty percent of the world's population suffers from some form of hearing loss, making it one of the most significant public health challenges. Current hearing aids commonly amplify noises while failing to improve speech comprehension in crowded social settings. In this demonstration, we showcase a proof-of-concept implementation of the world's first 5G and Internet of Things (IoT) enabled multi-modal hearing aid (MM HA) prototype. This integrates an innovative 5G cloud-radio access network (C-RAN) and IoT based transceiver model for real-time audio-visual speech enhancement (AVSE). Specifically, we demonstrate a transceiver model for Cloud-based AVSE which satisfies high data rate and low latency requirements for future MM HAs. The innovative 5G-IoT transceiver application is shown to satisfy HA latency limitations while transmitting raw noisy AV data from an MM HA prototype device to the cloud for deep learning-based real-time AVSE processing and obtaining a clean audio signal.

AB - Over twenty percent of the world's population suffers from some form of hearing loss, making it one of the most significant public health challenges. Current hearing aids commonly amplify noises while failing to improve speech comprehension in crowded social settings. In this demonstration, we showcase a proof-of-concept implementation of the world's first 5G and Internet of Things (IoT) enabled multi-modal hearing aid (MM HA) prototype. This integrates an innovative 5G cloud-radio access network (C-RAN) and IoT based transceiver model for real-time audio-visual speech enhancement (AVSE). Specifically, we demonstrate a transceiver model for Cloud-based AVSE which satisfies high data rate and low latency requirements for future MM HAs. The innovative 5G-IoT transceiver application is shown to satisfy HA latency limitations while transmitting raw noisy AV data from an MM HA prototype device to the cloud for deep learning-based real-time AVSE processing and obtaining a clean audio signal.

UR - https://www.scopus.com/pages/publications/85171571544

M3 - Conference contribution

AN - SCOPUS:85171571544

SP - 686

EP - 687

BT - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2023

T2 - 24th International Speech Communication Association Conference 2023

Y2 - 20 August 2023 through 24 August 2023

ER -

5G-IoT Cloud based Demonstration of Real-Time Audio-Visual Speech Enhancement for Multimodal Hearing-aids

Abstract

Conference

UN SDGs

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this