Spatio-Temporal Point Process for Multiple Object Tracking

Tao Wang, Kean Chen, Weiyao Lin*, John See, Zenghui Zhang, Qian Xu, Xia Jia

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    15 Citations (Scopus)


    Multiple object tracking (MOT) focuses on modeling the relationship of detected objects among consecutive frames and merge them into different trajectories. MOT remains a challenging task as noisy and confusing detection results often hinder the final performance. Furthermore, most existing research are focusing on improving detection algorithms and association strategies. As such, we propose a novel framework that can effectively predict and mask-out the noisy and confusing detection results before associating the objects into trajectories. In particular, we formulate such 'bad' detection results as a sequence of events and adopt the spatio-temporal point process to model such events. Traditionally, the occurrence rate in a point process is characterized by an explicitly defined intensity function, which depends on the prior knowledge of some specific tasks. Thus, designing a proper model is expensive and time-consuming, with also limited ability to generalize well. To tackle this problem, we adopt the convolutional recurrent neural network (conv-RNN) to instantiate the point process, where its intensity function is automatically modeled by the training data. Furthermore, we show that our method captures both temporal and spatial evolution, which is essential in modeling events for MOT. Experimental results demonstrate notable improvements in addressing noisy and confusing detection results in MOT data sets. An improved state-of-the-art performance is achieved by incorporating our baseline MOT algorithm with the spatio-temporal point process model.

    Original languageEnglish
    Pages (from-to)1777-1788
    Number of pages12
    JournalIEEE Transactions on Neural Networks and Learning Systems
    Issue number4
    Early online date8 Jun 2020
    Publication statusPublished - Apr 2023


    • Multiple object tracking
    • recurrent neural networks
    • spatio-temporal point processes

    ASJC Scopus subject areas

    • Software
    • Computer Science Applications
    • Computer Networks and Communications
    • Artificial Intelligence


    Dive into the research topics of 'Spatio-Temporal Point Process for Multiple Object Tracking'. Together they form a unique fingerprint.

    Cite this