Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition

Huabin Liu; Jianguo Li; Dian Li; John See; Weiyao Lin

doi:10.1109/TMM.2021.3090274

Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition

Huabin Liu, Jianguo Li, Dian Li, John See, Weiyao Lin

School of Mathematical & Computer Sciences

Research output: Contribution to journal › Article › peer-review

31 Downloads (Pure)

Abstract

This paper proposes a Scale-consistent Attention Part Network (SCAPNet) to address that issue, which seamlessly integrates three novel modules: grid gate attention unit (gGAU), scale-consistent attention part selection (SCAPS), and part relation modeling (PRM). The gGAU module represents the grid region at a certain fine-scale with middle layer CNN features and produces hard attention maps with the lightweight Gumbel-Max based gate. The SCAPS module utilizes attention to guide part selection across multi-scales and keep the selection scale-consistent. The PRM module utilizes the self-attention mechanism to build the relationship among parts based on their appearance and relative geo-positions. SCAPNet can be learned in an end-to-end way and demonstrates state-of-the-art accuracy on several publicly available fine-grained recognition datasets (CUB-200-2011, FGVC-Aircraft, Veg200, and Fru92).

Original language	English
Journal	IEEE Transactions on Multimedia
DOIs	https://doi.org/10.1109/TMM.2021.3090274
Publication status	E-pub ahead of print - 17 Jun 2021

Keywords

attention part
fine-grained image recognition
Image recognition
Location awareness
Logic gates
Object detection
scale-consistent
Task analysis
Training
Visualization

ASJC Scopus subject areas

Signal Processing
Media Technology
Computer Science Applications
Electrical and Electronic Engineering

Access to Document

10.1109/TMM.2021.3090274

Download pdf
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 2.19 MB

Cite this

@article{f9262ce090914220a44cdc9529eb2ae5,

title = "Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition",

abstract = "This paper proposes a Scale-consistent Attention Part Network (SCAPNet) to address that issue, which seamlessly integrates three novel modules: grid gate attention unit (gGAU), scale-consistent attention part selection (SCAPS), and part relation modeling (PRM). The gGAU module represents the grid region at a certain fine-scale with middle layer CNN features and produces hard attention maps with the lightweight Gumbel-Max based gate. The SCAPS module utilizes attention to guide part selection across multi-scales and keep the selection scale-consistent. The PRM module utilizes the self-attention mechanism to build the relationship among parts based on their appearance and relative geo-positions. SCAPNet can be learned in an end-to-end way and demonstrates state-of-the-art accuracy on several publicly available fine-grained recognition datasets (CUB-200-2011, FGVC-Aircraft, Veg200, and Fru92).",

keywords = "attention part, fine-grained image recognition, Image recognition, Location awareness, Logic gates, Object detection, scale-consistent, Task analysis, Training, Visualization",

author = "Huabin Liu and Jianguo Li and Dian Li and John See and Weiyao Lin",

note = "Publisher Copyright: IEEE",

year = "2021",

month = jun,

day = "17",

doi = "10.1109/TMM.2021.3090274",

language = "English",

journal = "IEEE Transactions on Multimedia ",

issn = "1520-9210",

publisher = "IEEE",

}

TY - JOUR

T1 - Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition

AU - Liu, Huabin

AU - Li, Jianguo

AU - Li, Dian

AU - See, John

AU - Lin, Weiyao

N1 - Publisher Copyright: IEEE

PY - 2021/6/17

Y1 - 2021/6/17

N2 - This paper proposes a Scale-consistent Attention Part Network (SCAPNet) to address that issue, which seamlessly integrates three novel modules: grid gate attention unit (gGAU), scale-consistent attention part selection (SCAPS), and part relation modeling (PRM). The gGAU module represents the grid region at a certain fine-scale with middle layer CNN features and produces hard attention maps with the lightweight Gumbel-Max based gate. The SCAPS module utilizes attention to guide part selection across multi-scales and keep the selection scale-consistent. The PRM module utilizes the self-attention mechanism to build the relationship among parts based on their appearance and relative geo-positions. SCAPNet can be learned in an end-to-end way and demonstrates state-of-the-art accuracy on several publicly available fine-grained recognition datasets (CUB-200-2011, FGVC-Aircraft, Veg200, and Fru92).

AB - This paper proposes a Scale-consistent Attention Part Network (SCAPNet) to address that issue, which seamlessly integrates three novel modules: grid gate attention unit (gGAU), scale-consistent attention part selection (SCAPS), and part relation modeling (PRM). The gGAU module represents the grid region at a certain fine-scale with middle layer CNN features and produces hard attention maps with the lightweight Gumbel-Max based gate. The SCAPS module utilizes attention to guide part selection across multi-scales and keep the selection scale-consistent. The PRM module utilizes the self-attention mechanism to build the relationship among parts based on their appearance and relative geo-positions. SCAPNet can be learned in an end-to-end way and demonstrates state-of-the-art accuracy on several publicly available fine-grained recognition datasets (CUB-200-2011, FGVC-Aircraft, Veg200, and Fru92).

KW - attention part

KW - fine-grained image recognition

KW - Image recognition

KW - Location awareness

KW - Logic gates

KW - Object detection

KW - scale-consistent

KW - Task analysis

KW - Training

KW - Visualization

UR - http://www.scopus.com/inward/record.url?scp=85110788461&partnerID=8YFLogxK

U2 - 10.1109/TMM.2021.3090274

DO - 10.1109/TMM.2021.3090274

M3 - Article

AN - SCOPUS:85110788461

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

ER -

Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this