Restorable-Inpainting: A Novel Deep Learning Approach for Shoeprint Restoration

Shoeprints are important information collected at the crime scene and are of great value for forensic analysis. Shoeprints collected in real-world scenarios are normally unclear, abrasive, and lack contextual and other kinds of missing information. In this research, we apply a novel deep learning technique called restorable inpainting to repair shoeprint contours and missing parts. Existing inpainting methods aim to fill artificially occluded areas with plausible pixels, but these methods may not restore missing information for occlusions in shoeprint images. In addition, because no ground-truth shoeprints exist for training samples, inpainting occluded regions becomes challenging. In this paper, we propose DeepShoePaint, a novel deep learning approach to perform restorable inpainting by restoring synthetic information resembling desirable shoeprint images necessary for forensics. DeepShoePaint novelly adapts a probabilistic distribution borrowed from the variational autoencoder into a U-Net-like structure forming a unified architecture trained in an unsupervised fashion to restore occluded and masked regions to produce human-verifiable shoeprints. The experimental results reveal that DeepShoePaint achieves outstanding human inspection and statistical assessment results and outperforms conventional inpainting models. We believe that this study can provide valuable insights, not limited to inpainting, into restoring desirable shoeprints to automate and facilitate the forensic investigation and examination process instead of using handcrafted methods.