An Intelligent Implementation of Multi-Sensing Data Fusion With Neuromorphic Computing for Human Activity Recognition

Zheqi Yu, Adnan Zahid, Ahmad Taha, William Taylor, Julien Le Kernec, Hadi Heidari, Muhammad Ali Imran, Qammer H. Abbasi*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
62 Downloads (Pure)

Abstract

The increasing demand for considering multisensor data fusion technology has drawn attention for precise human activity recognition (HAR) over standalone technology due to its reliability and robustness. This article presents a framework that fuses data from multiple sensing systems and applies neuromorphic computing to sense and classify human activities. The data is collected by utilizing inertial measurement unit (IMU) sensors, software-defined radios, and radars, and feature extraction and selection are performed on the data. For each of the actions, such as sitting and standing, an activity matrix is generated, which is then fed into a discrete Hopfield neural network as a binary feature pattern for one-shot learning. Following the Hopfield network neurons' feedback output, the conformity to the standard activity feature pattern is also determined. Following the Hopfield network neurons' feedback output, the training of neurons is completed after two steps under the Hebbian learning law, and the conformity to the standard activity feature pattern is also determined. According to the probabilistic statistics on inference predictions, the proposed method, that is the neuromorphic computing of the three data fused framework, achieved the box plot for the highest lower quartile output of 95.34%, while the confusion matrix classification accuracy of the two activities was 98.98%. The results have shown that neuromorphic computing is most capable of multisensor data-fusion-based HAR. Furthermore, the proposed method can be enhanced by incorporating additional hardware signal processing in the system to enable the flexible integration of human activity data.

Original languageEnglish
Pages (from-to)1124-1133
Number of pages10
JournalIEEE Internet of Things Journal
Volume10
Issue number2
Early online date6 Sept 2022
DOIs
Publication statusPublished - 15 Jan 2023

Keywords

  • Artificial intelligence
  • data fusion
  • human activity recognition (HAR)
  • signal processing

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'An Intelligent Implementation of Multi-Sensing Data Fusion With Neuromorphic Computing for Human Activity Recognition'. Together they form a unique fingerprint.

Cite this