American Sign Language Recognition Using Deep Learning Models

Gauri D. Revankar; Smitha S. Kumar

doi:10.1007/978-981-96-1758-6_31

American Sign Language Recognition Using Deep Learning Models

Gauri D. Revankar^*, Smitha S. Kumar

^*Corresponding author for this work

School of Mathematical & Computer Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Sign language is a mode of communication that enables individuals with hearing or speech impairment, or both, to express themselves. Sign language recognition from videos has become a new challenge in this research field. This paper focuses on isolated sign language recognition, which involves recognizing and interpreting phrases or words expressed through gestures and hand movements through a short video. With the advancement of convolutional and recurrent neural network architectures in computer vision, this paper proposes efficient deep learning models to recognize American Sign Language (ASL). The models implemented in this paper are ResNet50, ResNet50 + BiLSTM, Xception, and Xception + BiLSTM. Overall, ResNet50 + BiLSTM performed the best, with training, validation, and test accuracies of 79.37%, 69.56%, and 52.17%, respectively. The models were trained and evaluated on a 10-gloss subset of the WLASL dataset. Furthermore, a comparative analysis was performed with the models proposed in other research papers implemented for the same purpose.

Original language	English
Title of host publication	Proceedings of International Conference on Information Technology and Applications. ICITA 2024
Editors	Abrar Ullah, Sajid Anwar
Publisher	Springer
Pages	371-382
Number of pages	12
ISBN (Electronic)	9789819617586
ISBN (Print)	9789819617579
DOIs	https://doi.org/10.1007/978-981-96-1758-6_31
Publication status	Published - 15 May 2025
Event	18th International Conference on Information Technology and Applications 2024 - Sydney, Australia Duration: 17 Oct 2024 → 19 Oct 2024 https://2024.icita.world/#/

Publication series

Name	Lecture Notes in Networks and Systems
Volume	1248
ISSN (Print)	2367-3370
ISSN (Electronic)	2367-3389

Conference

Conference	18th International Conference on Information Technology and Applications 2024
Abbreviated title	ICITA 2024
Country/Territory	Australia
City	Sydney
Period	17/10/24 → 19/10/24
Internet address	https://2024.icita.world/#/

Keywords

American sign language
Computer vision
Deep learning
Isolated sign language recognition
Sign language recognition
Transfer learning

ASJC Scopus subject areas

Control and Systems Engineering
Signal Processing
Computer Networks and Communications

Access to Document

10.1007/978-981-96-1758-6_31

Cite this

@inproceedings{48574d6ea37f4047b78aadc31edc5c96,

title = "American Sign Language Recognition Using Deep Learning Models",

abstract = "Sign language is a mode of communication that enables individuals with hearing or speech impairment, or both, to express themselves. Sign language recognition from videos has become a new challenge in this research field. This paper focuses on isolated sign language recognition, which involves recognizing and interpreting phrases or words expressed through gestures and hand movements through a short video. With the advancement of convolutional and recurrent neural network architectures in computer vision, this paper proposes efficient deep learning models to recognize American Sign Language (ASL). The models implemented in this paper are ResNet50, ResNet50 + BiLSTM, Xception, and Xception + BiLSTM. Overall, ResNet50 + BiLSTM performed the best, with training, validation, and test accuracies of 79.37\%, 69.56\%, and 52.17\%, respectively. The models were trained and evaluated on a 10-gloss subset of the WLASL dataset. Furthermore, a comparative analysis was performed with the models proposed in other research papers implemented for the same purpose.",

keywords = "American sign language, Computer vision, Deep learning, Isolated sign language recognition, Sign language recognition, Transfer learning",

author = "Revankar, \{Gauri D.\} and Kumar, \{Smitha S.\}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.; 18th International Conference on Information Technology and Applications 2024, ICITA 2024 ; Conference date: 17-10-2024 Through 19-10-2024",

year = "2025",

month = may,

day = "15",

doi = "10.1007/978-981-96-1758-6\_31",

language = "English",

isbn = "9789819617579",

series = "Lecture Notes in Networks and Systems",

publisher = "Springer",

pages = "371--382",

editor = "Abrar Ullah and Sajid Anwar",

booktitle = "Proceedings of International Conference on Information Technology and Applications. ICITA 2024",

url = "https://2024.icita.world/\#/",

}

Revankar, GD & Kumar, SS 2025, American Sign Language Recognition Using Deep Learning Models. in A Ullah & S Anwar (eds), Proceedings of International Conference on Information Technology and Applications. ICITA 2024. Lecture Notes in Networks and Systems, vol. 1248, Springer, pp. 371-382, 18th International Conference on Information Technology and Applications 2024, Sydney, New South Wales, Australia, 17/10/24. https://doi.org/10.1007/978-981-96-1758-6_31

American Sign Language Recognition Using Deep Learning Models. / Revankar, Gauri D.; Kumar, Smitha S.
Proceedings of International Conference on Information Technology and Applications. ICITA 2024. ed. / Abrar Ullah; Sajid Anwar. Springer, 2025. p. 371-382 (Lecture Notes in Networks and Systems; Vol. 1248).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - American Sign Language Recognition Using Deep Learning Models

AU - Revankar, Gauri D.

AU - Kumar, Smitha S.

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

PY - 2025/5/15

Y1 - 2025/5/15

N2 - Sign language is a mode of communication that enables individuals with hearing or speech impairment, or both, to express themselves. Sign language recognition from videos has become a new challenge in this research field. This paper focuses on isolated sign language recognition, which involves recognizing and interpreting phrases or words expressed through gestures and hand movements through a short video. With the advancement of convolutional and recurrent neural network architectures in computer vision, this paper proposes efficient deep learning models to recognize American Sign Language (ASL). The models implemented in this paper are ResNet50, ResNet50 + BiLSTM, Xception, and Xception + BiLSTM. Overall, ResNet50 + BiLSTM performed the best, with training, validation, and test accuracies of 79.37%, 69.56%, and 52.17%, respectively. The models were trained and evaluated on a 10-gloss subset of the WLASL dataset. Furthermore, a comparative analysis was performed with the models proposed in other research papers implemented for the same purpose.

AB - Sign language is a mode of communication that enables individuals with hearing or speech impairment, or both, to express themselves. Sign language recognition from videos has become a new challenge in this research field. This paper focuses on isolated sign language recognition, which involves recognizing and interpreting phrases or words expressed through gestures and hand movements through a short video. With the advancement of convolutional and recurrent neural network architectures in computer vision, this paper proposes efficient deep learning models to recognize American Sign Language (ASL). The models implemented in this paper are ResNet50, ResNet50 + BiLSTM, Xception, and Xception + BiLSTM. Overall, ResNet50 + BiLSTM performed the best, with training, validation, and test accuracies of 79.37%, 69.56%, and 52.17%, respectively. The models were trained and evaluated on a 10-gloss subset of the WLASL dataset. Furthermore, a comparative analysis was performed with the models proposed in other research papers implemented for the same purpose.

KW - American sign language

KW - Computer vision

KW - Deep learning

KW - Isolated sign language recognition

KW - Sign language recognition

KW - Transfer learning

UR - https://www.scopus.com/pages/publications/105009405420

U2 - 10.1007/978-981-96-1758-6_31

DO - 10.1007/978-981-96-1758-6_31

M3 - Conference contribution

AN - SCOPUS:105009405420

SN - 9789819617579

T3 - Lecture Notes in Networks and Systems

SP - 371

EP - 382

BT - Proceedings of International Conference on Information Technology and Applications. ICITA 2024

A2 - Ullah, Abrar

A2 - Anwar, Sajid

PB - Springer

T2 - 18th International Conference on Information Technology and Applications 2024

Y2 - 17 October 2024 through 19 October 2024

ER -

American Sign Language Recognition Using Deep Learning Models

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this