Deep learning model for human-intuitive shoeprint reconstruction

Muhammad Hassan, Yan Wang, Di Wang, Wei Pang, Daixi Li, You Zhou, Dong Xu, Sami Ur-Rahman, Amin Ur Rahman, Ahmed Ameen Fateh, Peiwu Qin

Research output: Contribution to journalArticlepeer-review

Abstract

Shoeprint reconstruction is essential in forensic science, but it is also challenging due to various inconsistencies in the patterns, textures, sizes, abrasions, etc. of shoeprints. The computational reconstruction of sharper and more complete shoeprints is conventionally conducted using handcrafted features, and it often requires human intervention. Prior studies using end-to-end machine learning approaches are limited in number and have not achieved a high level of performance. In this paper, we propose a model named ShoeRec, which employs variational autoencoder (VAE) as a component in a U-Net-like architecture to reconstruct missing regions and borders in shoeprint images. ShoeRec incorporates skip connections to preserve key patterns and employs VAE in the bottleneck to facilitate the reconstruction of desired shoeprints with the restoration of detail as perceived by humans. As a U-Net, the model skips the contextual information from the encoder to the decoder, and the compressed features in the latent space via VAE optimize the probabilistic distribution for reconstructing complete shoeprints. The reconstruction operation is automatically tuned according to the objective function, so as to reduce the structural correlation between the original and projected shoeprint and restore the absent information in the desired shoeprints. To the best of our knowledge, ShoeRec is the first deep learning infusion model that specializes in shoeprint reconstruction. The shoeprints reconstructed by ShoeRec have a close match with the originals in terms of both structures and patterns, and ShoeRec outperforms state-of-the-art generative models in human evaluation.
Original languageEnglish
Article number123704
JournalExpert Systems with Applications
Volume249
Issue numberPart C
Early online date24 Mar 2024
DOIs
Publication statusE-pub ahead of print - 24 Mar 2024

Keywords

  • Abrasion
  • Forensics
  • Generation
  • Patterns
  • Reconstruction
  • Shoeprint

ASJC Scopus subject areas

  • General Engineering
  • Artificial Intelligence
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Deep learning model for human-intuitive shoeprint reconstruction'. Together they form a unique fingerprint.

Cite this