![]() |
Introduction
Tutorials
Survey papers
Benchmark datasets
Fine-grained image recognition
Fine-grained recognition by localization-classification subnetworks
Employing detection or segmentation techniques
Utilizing deep filters / activations
Leveraging attention mechanisms
Other methods
Fine-grained recognition by end-to-end feature encoding
High-order feature interactions
Specific loss functions
Other methods
Fine-grained recognition with external information
Fine-grained recognition with web data / auxiliary data
Fine-grained recognition with multi-modality data
Fine-grained recognition with humans in the loop
Fine-grained image retrieval
Content-based fine-grained image retrieval
Sketch-based fine-grained image retrieval
Future directions of FGIA
Fine-grained few/zero shot learning
Fine-grained hashing
Fine-grained domain adaptation
Fine-grained image generation
FGIA within more realistic settings
Toolbox
Recognition leaderboard
This homepage lists some representative papers/codes/datasets all about deep learning based fine-grained image analysis, including fine-grained image recognition, fine-grained image retrieval, etc. If you have any questions, please feel free to contact Prof. Xiu-Shen Wei.
Fine-Grained Visual Analysis with Deep Learning
Xiu-Shen Wei, Serge Belongie, Piotr Koniusz, Lei Wang, Yi-Zhe Song, Jiashi Feng, Jian Yang, and Dian Shao. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Fine-Grained Image Analysis.
Xiu-Shen Wei. IEEE International Conference on Multimedia and Expo (ICME), 2019.
Fine-Grained Image Analysis.
Xiu-Shen Wei, and Jianxin Wu. Pacific Rim International Conference on Artificial Intelligence (PRICAI), 2018.
Fine-Grained Image Analysis with Deep Learning: A Survey
Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha, Jianxin Wu, Yuxin Peng, Jinhui Tang, Jian Yang and Serge Belongie. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Deep Learning for Fine-Grained Image Analysis: A Survey.
Xiu-Shen Wei, Jianxin Wu, and Quan Cui. arXiv: 1907.03069, 2019.
A Survey on Deep Learning-based Fine-Grained Object Classification and Semantic Segmentation.
Bo Zhao, Jiashi Feng, Xiao Wu, and Shuicheng Yan. International Journal of Automation and Computing, 2017.
Summary of popular fine-grained image datasets. Note that ‘‘BBox’’ indicates whether this dataset provides object bounding box supervisions. ‘‘Part anno.’’ means providing the key part localizations. ‘‘HRCHY’’ corresponds to hierarchical labels. ‘‘ATR’’ represents the attribute labels (e.g., wing color, male, female, etc). ‘‘Texts’’ indicates whether fine-grained text descriptions of images are supplied. Several datasets are listed here twice since they are commonly used in both recognition and retrieval tasks.
Dataset name | Year | Meta-class | ![]() | ![]() | BBox | Part anno. | HRCHY | ATR | Texts |
Oxford flower | 2008 | Flowers | 8,189 | 102 | ![]() | ||||
CUB200 | 2011 | Birds | 11,788 | 200 | ![]() | ![]() | ![]() | ![]() | |
Stanford Dog | 2011 | Dogs | 20,580 | 120 | ![]() | ||||
Stanford Car | 2013 | Cars | 16,185 | 196 | ![]() | ||||
FGVC Aircraft | 2013 | Aircrafts | 10,000 | 100 | ![]() | ![]() | |||
Birdsnap | 2014 | Birds | 49,829 | 500 | ![]() | ![]() | ![]() | ||
Food-101 | 2014 | Food dishes | 101,000 | 101 | |||||
NABirds | 2015 | Birds | 48,562 | 555 | ![]() | ![]() | |||
Food-975 | 2016 | Foods | 37,885 | 975 | ![]() | ||||
DeepFashion | 2016 | Clothes | 800,000 | 1,050 | ![]() | ![]() | ![]() | ||
Fru92 | 2017 | Fruits | 69,614 | 92 | ![]() | ||||
Veg200 | 2017 | Vegetable | 91,117 | 200 | ![]() | ||||
iNat2017 | 2017 | Plants & Animals | 859,000 | 5,089 | ![]() | ![]() | |||
Dog-in-the-Wild | 2018 | Dogs | 299,458 | 362 | |||||
RPC | 2019 | Retail products | 83,739 | 200 | ![]() | ![]() | |||
Products-10K | 2020 | Retail products | 150,000 | 10,000 | ![]() | ![]() | |||
UFG | 2021 | Leaf | 47,114 | 3,526 | ![]() | ||||
iNat2021 | 2021 | Plants & Animals | 3,286,843 | 10,000 | ![]() |
Dataset name | Year | Meta-class | ![]() | ![]() | BBox | Part anno. | HRCHY | ATR | Texts |
Oxford flower | 2008 | Flowers | 8,189 | 102 | ![]() | ||||
CUB200 | 2011 | Birds | 11,788 | 200 | ![]() | ![]() | ![]() | ![]() | |
Stanford Car | 2013 | Cars | 16,185 | 196 | ![]() | ||||
SBIR2014 | 2014 | Multiple | 1,120/7,267 | 14 | ![]() | ![]() | ![]() | ||
DeepFashion | 2016 | Clothes | 800,000 | 1,050 | ![]() | ![]() | ![]() | ||
QMUL-Shoe | 2016 | Shoes | 419/419 | 1 | ![]() | ||||
QMUL-Chair | 2016 | Chairs | 297/297 | 1 | ![]() | ||||
Sketchy | 2016 | Multiple | 75,471/12,500 | 125 | |||||
QMUL-Handbag | 2017 | Handbags | 568/568 | 1 | |||||
SBIR2017 | 2017 | Shoes | 912/304 | 1 | ![]() | ![]() | |||
QMUL-Shoe-V2 | 2019 | Shoes | 6,730/2,000 | 1 | |||||
FG-Xmedia | 2019 | Birds | 11,788 | 200 | ![]() |
Employing detection or segmentation techniques
Part-based R-CNNs for Fine-Grained Category Detection.
Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Darrell. ECCV, 2014. [code]
Fine-Grained Recognition without Part Annotations.
Jonathan Krause, Hailin Jin, Jianchao Yang, and Li Fei-Fei. CVPR, 2015. [code]
Deep LAC: Deep Localization, Alignment and Classification for Fine-grained Recognition.
Di Lin, Xiaoyong Shen, Cewu Lu, and Jiaya Jia. CVPR, 2015.
Part-Stacked CNN for Fine-Grained Visual Categorization.
Shaoli Huang, Zhe Xu, Dacheng Tao, and Ya Zhang. CVPR, 2016.
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-grained Recognition.
Han Zhang, Tao Xu, Mohamed Elhoseiny, Xiaolei Huang, Shaoting Zhang, Ahmed Elgammal, and Dimitris Metaxas. CVPR, 2016.
Weakly Supervised Fine-Grained Categorization with Part-Based Image Representation.
Yu Zhang, Xiu-Shen Wei, Jianxin Wu, Jianfei Cai, Jiangbo Lu, Viet-Anh Nguyen, and Minh N. Do. IEEE TIP, 2016.
Coarse-to-Fine Description for Fine-Grained Visual Categorization.
Hantao Yao, Shiliang Zhang, Yongdong Zhang, Jintao Li, and Qi Tian. IEEE TIP, 2016.
Fine-Grained Recognition as HSnet Search for Informative Image Parts.
Michael Lam, Behrooz Mahasseni, and Sinisa Todorovic. CVPR, 2017.
Weakly Supervised Learning of Part Selection Model with Spatial Constraints for Fine-Grained Image Classification.
Xiangteng He, and Yuxin Peng. AAAI, 2017.
Mask-CNN: Localizing Parts and Selecting Descriptors for Fine-Grained Bird Species Categorization.
Xiu-Shen Wei, Chen-Wei Xie, Jianxin Wu, and Zhi-Hua Zhou. Pattern Recognition, 2018.
Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up.
Weifeng Ge, Xiangru Lin, and Yizhou Yu. CVPR, 2019.
Graph-Propagation Based Correlation Learning
for Weakly Supervised Fine-Grained Image Classification.
Zhihui Wang, Shijie Wang, Haojie Li, Zhi Dou, and Jianjun Li. AAAI, 2020.
Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization.
Chuanbin Liu, Hongtao Xie, Zheng-Jun Zha, Lingfeng Ma, Lingyun Yu, and Yongdong Zhang. AAAI, 2020.
Utilizing deep filters / activations
The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification.
Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, and Zheng Zhang. CVPR, 2015.
The Treasure beneath Convolutional Layers: Cross-convolutional-layer Pooling for Image Classification.
Lingqiao Liu, Chunhua Shen, and Anton van den Hengel. CVPR, 2015.
Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks.
Marcel Simon, and Erik Rodner. ICCV, 2015. [code]
Picking Deep Filter Responses for Fine-grained Image Recognition.
Xiaopeng Zhang, Hongkai, Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. CVPR, 2016.
Learning a Discriminative Filter Bank within a CNN for Fine-grained Recognition.
Yaming Wang, Vlad I. Morariu, and Larry S. Davis. CVPR, 2018. [code]
Selective Sparse Sampling for Fine-grained Image Recognition.
Yao Ding, Yanzhao Zhou, Yi Zhu, Qixiang Ye, and Jianbin Jiao. ICCV, 2019. [code]
Interpretable and Accurate Fine-grained Recognition via Region Grouping.
Zixuan Huang, and Yin Li. CVPR, 2020.
Neural Prototype Trees for Interpretable Fine-grained Image Recognition.
Meike Nauta, Ron van Bree, Christin Seifert. CVPR, 2021.
Dynamic Position-Aware Network for Fine-Grained Image Recognition.
Shijie Wang, Haojie Li, Zhihui Wang, and Wanli Ouyang. AAAI, 2021.
Fine-Grained Classification via Categorical Memory Networks.
Weijian Deng, Joshua Marsh, Stephen Gould, and Liang Zheng. IEEE TIP, 2022.
P-CNN: Part-Based Convolutional Neural Networks for Fine-Grained Visual Categorization.
Junwei Han, Xiwen Yao, Gong Cheng, Xiaoxu Feng, and Dong Xu. IEEE TPAMI, 2022.
Cross-Part Learning for Fine-Grained Image Classification.
Man Liu, Chunjie Zhang, Huihui Bai, Riquan Zhang, and Yao Zhao. IEEE TIP, 2022.
Leveraging attention mechanisms
Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition.
Jianlong Fu, Heliang Zheng, and Tao Mei. CVPR, 2017. [code]
Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition.
Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. ICCV, 2017. [code]
Localizing by Describing: Attribute-Guided Attention Localization for Fine-Grained Recognition.
Xiao Liu, Jiang Wang, Shilei Wen, Errui Ding, and Yuanqing Lin. AAAI, 2017.
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition.
Ming Sun, Yuchen Yuan, Feng Zhou, and Errui Ding. ECCV, 2018. [code]
Object-Part Attention Model for Fine-Grained Image Classification.
Yuxin Peng, Xiangteng He, and Junjie Zhao. IEEE TIP, 2018.
Learning a Mixture of Granularity-Specific Experts for Fine-Grained Categorization.
Lianbo Zhang, Shaoli Huang, Wei Liu, and Dacheng Tao. ICCV, 2019.
Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-Grained Image Recognition.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, and Jiebo Luo. CVPR, 2019. [code]
Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization.
Ruyi Ji, Longyin Wen, Libo Zhang, Dawei Du, Yanjun Wu, Chen Zhao, Xianglong Liu, and Feiyue Huang. CVPR, 2020. [code]
Learning Rich Part Hierarchies With Progressive
Attention Networks for Fine-Grained Image Recognition.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, and Tao Mei. IEEE TIP, 2020.
Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions.
Xinhai Liu, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. IEEE TIP, 2020. [code]
Weakly Supervised Attention Pyramid Convolutional Neural Network for Fine-Grained Visual Classification.
Yifeng Ding, Shaoguo Wen, Jiyang Xie, Dongliang Chang, Zhanyu Ma, Zhongwei Si, and Haibin Ling. IEEE TIP, 2021.
Context-Aware Attentional Pooling (CAP) for Fine-Grained Visual Classification.
Ardhendu Behera, Zachary Wharton, Pradeep R. P. G. Hewage, and Asish Bera. AAAI, 2021.
Fine-Grained 3D Shape Classification With Hierarchical Part-View Attention.
Xinhai Liu, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. IEEE TIP, 2021.
Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-Identification.
Yongming Rao, Guangyi Chen, Jiwen Lu, and Jie Zhou. ICCV, 2021.
Part-Guided Relational Transformers for Fine-Grained Visual Recognition.
Yifan Zhao, Jia Li, Xiaowu Chen, and Yonghong Tian. IEEE TIP, 2021.
SR-GNN: Spatial Relation-Aware Graph Neural Network for Fine-Grained Image Categorization.
Asish Bera, Zachary Wharton, Yonghuai Liu, Nik Bessis, and Ardhendu Behera. IEEE TIP, 2022.
TransFG: A Transformer Architecture for Fine-Grained Recognition.
Ju He, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, and Changhu Wang. AAAI, 2022.
Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism.
Yangyang Shu, Baosheng Yu, Haiming Xu, and Lingqiao Liu. ECCV, 2022.
Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification.
Haowei Zhu, Wenjing Ke, Dong Li, Ji Liu, Lu Tian, and Yi Shan. CVPR, 2022.
Other methods
Spatial Transformer Networks.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. NeurIPS, 2015. [code]
Mining Discriminative Triplets of Patches for Fine-Grained Classification.
Yaming Wang, Jonghyun Choi, Vlad I. Morariu, and Larry S. Davis. CVPR, 2016.
Learning to Navigate for Fine-grained Classification.
Ze Yang, Tiange Luo, Dong Wang, Zhiqiang Hu, Jun Gao, and Liwei Wang. ECCV, 2018. [code]
Which and How Many Regions to Gaze: Focus Discriminative Regions for Fine-Grained Visual Categorization.
Xiangteng He, Yuxin Peng, and Junjie Zhao. IJCV, 2019.
Weakly Supervised Fine-grained Image Classification via Guassian Mixture Model Oriented Discriminative Learning.
Zhihui Wang, Shijie Wang, Shuhui Yang, Haojie Li, Jianjun Li, and Zezhou Li. CVPR, 2020.
Fine-Grained Object Classification via Self-Supervised Pose Alignment.
Xuhui Yang, Yaowei Wang, Ke Chen, Yong Xu, and Yonghong Tian. CVPR, 2022.
High-order feature interactions
Bilinear CNN Models for Fine-grained Visual Recognition.
Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. ICCV, 2015. [code]
Compact Bilinear Pooling.
Yang Gao, Oscar Beijbom, Ning Zhang, and Trevor Darrell. CVPR, 2016. [code]
Kernel Pooling for Convolutional Neural Networks.
Yin Cui, Feng Zhou, Jiang Wang, Xiao Liu, Yuanqing Lin, and Serge Belongie. CVPR, 2017.
Low-rank Bilinear Pooling for Fine-Grained Classification.
Shu Kong, and Charless Fowlkes. CVPR, 2017. [code]
Higher-order Integration of Hierarchical Convolutional Activations for Fine-Grained Visual Categorization.
Sijia Cai, Wangmeng Zuo, and Lei Zhang. ICCV, 2017. [code]
Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization.
Peihua Li, Jiangtao Xie, Qilong Wang, and Zilin Gao. CVPR, 2018. [code]
DeepKSPD: Learning Kernel-matrix-based SPD Representation for Fine-Grained Image Recognition.
Melih Engin, Lei Wang, Luping Zhou, and Xinwang Liu. ECCV, 2018.
Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition.
Chaojian Yu, Xinyi Zhao, Qi Zheng, Peng Zhang, and Xinge You. ECCV, 2018. [code]
Grassmann Pooling as Compact Homogeneous Bilinear Pooling for Fine-Grained Visual Classification.
Xing Wei, Yue Zhang, Yihong Gong, Jiawei Zhang, and Nanning Zheng. ECCV, 2018.
Learning Deep Bilinear Transformation for Fine-grained Image Representation.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, and Jiebo Luo. NeurIPS, 2019.
Multi-Objective Matrix Normalization for Fine-Grained Visual Recognition.
Shaobo Min, Hantao Yao, Hongtao Xie, Zheng-Jun Zha, and Yongdong Zhang. IEEE TIP, 2020.
Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification.
Piotr Koniusz, Hongguang Zhang. IEEE TPAMI, 2021.
Graph-based High-Order Relation Discovery for Fine-grained Recognition.
Yifan Zhao, Ke Yan, Feiyue Huang, and Jia Li. CVPR, 2021.
Specific loss functions
Maximum-Entropy Fine Grained Classification.
Abhimanyu Dubey, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. NeurIPS, 2018.
Pairwise Confusion for Fine-Grained Visual Classification.
Abhimanyu Dubey, Otkrist Gupta, Pei Guo, Ramesh Raskar, Ryan Farrell, and Nikhil Naik. ECCV, 2018. [code]
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition.
Ming Sun, Yuchen Yuan, Feng Zhou, and Errui Ding. ECCV, 2018. [code]
Channel Interaction Networks for Fine-Grained Image Categorization.
Yu Gao, Xintong Han, Xun Wang, Weilin Huang, and Matthew R. Scott. AAAI, 2020.
Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes.
Guolei Sun, Hisham Cholakkal, Salman Khan, Fahad Shahbaz Khan, and Ling Shao. AAAI, 2020.
Learning Attentive Pairwise Interaction for Fine-Grained Classification.
Peiqin Zhuang, Yali Wang, and Yu Qiao. AAAI, 2020.
The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification.
Dongliang Chang, Yifeng Ding, Jiyang Xie, Ayan Kumar Bhunia, Xiaoxu Li, Zhanyu Ma, Ming Wu, Jun Guo, and Yi-Zhe Song. IEEE TIP, 2020.
Discrimination-Aware Mechanism for Fine-grained Representation Learning.
Furong Xu, Meng Wang, Wei Zhang, Yuan Cheng, and Wei Chu. CVPR, 2021.
Convolutional Fine-Grained Classification With Self-Supervised Target Relation Regularization.
Kangjun Liu, Ke Chen, and Kui Jia. IEEE TIP, 2022.
Other methods
Fine-Grained Image Classification by Exploring Bipartite-Graph Labels.
Feng Zhou, and Yuanqing Lin. CVPR, 2016. [project page]
Destruction and Construction Learning for Fine-grained Image Recognition.
Yue Chen, Yalong Bai, Wei Zhang, and Tao Mei. CVPR, 2019. [code]
Cross-X Learning for Fine-Grained Visual Categorization.
Wei Luo, Xiong Yang, Xianjie Mo, Yuheng Lu, Larry S. Davis, Jun Li, Jian Yang, and Ser-Nam Lim. ICCV, 2019. [code]
Fine-grained Image-to-Image Transformation towards Visual Recognition.
Wei Xiong, Yutong He, Yixuan Zhang, Wenhan Luo, Lin Ma, and Jiebo Luo. CVPR, 2020.
Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches.
Ruoyi Du, Dongliang Chang, Ayan Kumar Bhunia, Jiyang Xie, Yi-Zhe Song, Zhanyu Ma, and Jun Guo. ECCV, 2020. [code]
Learning Canonical 3D Object Representation for Fine-Grained Recognition.
Sunghun Joung, Minsu Kim, Seungryong Kim, and Ig-Jae Kim. ICCV, 2021.
Guided Zoom: Zooming into Network Evidence to Refine Fine-Grained Model Decisions.
Sarah Adel Bargal, Andrea Zunino, Vitali Petsiuk, Jianming Zhang, Kate Saenko, Vittorio Murino, and Stan Sclaroff. IEEE TPAMI, 2021.
SnapMix: Semantically Proportional Mixing for Augmenting Fine-Grained Data.
Shaoli Huang, Xinchao Wang, and Dacheng Tao. AAAI, 2021.
Stochastic Partial Swap: Enhanced Model Generalization and Interpretability for Fine-Grained Recognition.
Shaoli Huang, Xinchao Wang, and Dacheng Tao. ICCV, 2021.
Fine-grained recognition with web data
Hyper-Class Augmented and Regularized Deep Learning for Fine-Grained Image Classification.
Saining Xie, Tianbao Yang, Xiaoyu Wang, and Yuanqing Lin. CVPR, 2015.
Augmenting Strong Supervision Using Web Data for Fine-Grained Categorization.
Zhe Xu, Shaoli Huang, Ya Zhang, and Dacheng Tao. ICCV, 2015.
The unreasonable effectiveness of noisy data for fine-grained recognition.
Jonathan Krause, Benjamin Sapp, Andrew Howard, Howard Zhou, Alexander Toshev, Tom Duerig, James Philbin, and Li Fei-Fei. ECCV, 2016.
Webly Supervised Learning Meets Zero-shot Learning: A Hybrid Approach for Fine-grained Classification.
Li Niu, Ashok Veeraraghavan, and Vshu Sabbarwal. CVPR, 2018.
Fine-Grained Visual Categorization using Meta-Learning Optimization with Sample Selection of Auxiliary Data.
Yabin Zhang, Hui Tang, and Kai Jia. ECCV, 2018. [code]
Recognition From Web Data: A Progressive Filtering Approach.
Jufeng Yang, Xiaoxiao Sun, Yu-Kun Lai, Liang Zheng, and Ming-Ming Cheng. IEEE TIP, 2018.
Webly-Supervised Fine-Grained Visual Categorization via Deep Domain Adaptation.
Zhe Xu, Shaoli Huang, Ya Zhang, and Dacheng Tao. IEEE TPAMI, 2018.
Learning from Web Data using Adversarial Discriminative Neural Networks for Fine-Grained Classification.
Xiaoxiao Sun, Liyi Chen, and Jufeng Yang. AAAI, 2019.
Web-Supervised Network with Softly Update-Drop Training for Fine-Grained Visual Classification.
Chuanyi Zhang, Yazhou Yao, Huafeng Liu, Guo-Sen Xie, Xiangbo Shu, Tianfei Zhou, Zheng Zhang, Fumin Shen, and Zhenmin Tang. AAAI, 2020.
Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach.
Zeren Sun, Yazhou Yao, Xiu-Shen Wei, Yongshun Zhang, Fumin Shen, Jianxin Wu, Jian Zhang, and Heng Tao Shen. ICCV, 2021. [code]
Webly-Supervised Fine-Grained Recognition with Partial Label Learning.
Yu-Yan Xu, Yang Shen, Xiu-Shen Wei, and Jian Yang. IJCAI, 2022.
Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information.
Lingfeng Yang, Xiang Li, Renjie Song, Borui Zhao, Juntian Tao, Shihao Zhou, Jiajun Liang, and Jian Yang. CVPR, 2022.
Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition.
Jun Yu, Min Tan, Hongyuan Zhang, Yong Rui, and Dacheng Tao. IEEE TPAMI, 2022.
Fine-grained recognition with multi-modality data
Fine-Grained Image Classification via Combining Vision and Language.
Xiangteng He, and Yuxin Peng. CVPR, 2017.
Audio Visual Attribute Discovery for Fine-Grained Object Recognition.
Hua Zhang, Xiaochun Cao, and Rui Wang. AAAI, 2018.
Fine-Grained Image Classification by Visual-Semantic Embedding.
Huapeng Xu, Guilin Qi, Jingjing Li, Meng Wang, Kang Xu, and Huan Gao. IJCAI, 2018.
Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition.
Tianshui Chen, Liang Lin, Riquan Chen, Yang Wu, and Xiannan Luo. IJCAI, 2018.
Bi-Modal Progressive Mask Attention for Fine-Grained Recognition.
Kaitao Song, Xiu-Shen Wei, Xiangbo Shu, Ren-Jie Song, and Jianfeng Lu. IEEE TIP, 2020.
GLAVNet: Global-Local Audio-Visual Cues for Fine-Grained Material Recognition.
Fengmin Shi, Jie Guo, Haonan Zhang, Shan Yang, Xiying Wang, and Yanwen Guo. CVPR, 2021.
Exploring Fine-Grained Audiovisual Categorization with the SSW60 Dataset.
Grant Van Horn, Rui Qian, Kimberly Wilber, Hartwig Adam, Oisin Mac Aodha, and Serge Belongie. ECCV, 2022.
Fine-grained recognition with humans in the loop
Fine-grained Categorization and Dataset Bootstrapping using Deep Metric Learning with Humans in the Loop.
Yin Cui, Feng Zhou, Yuanqing Lin, and Serge Belongie. CVPR, 2016.
Leveraging the Wisdom of the Crowd for Fine-Grained Recognition.
Jia Deng, Jonathan Krause, Michael Stark, and Li Fei-Fei. IEEE TPAMI, 2016.
Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval.
Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, and Zhi-Hua Zhou. IEEE TIP, 2017. [project page]
Centralized Ranking Loss with Weakly Supervised Localization for Fine-Grained Object Retrieval.
Xiawu Zheng, Rongrong Ji, Xiaoshuai Sun, Yongjian Wu, Feiyue Huang, and Yanhua Yang. IJCAI, 2018.
Towards Optimal Fine Grained Retrieval via Decorrelated Centralized Loss with Normalize-Scale layer.
Xiawu Zheng, Rongrong Ji, Xiaoshuai Sun, Baochang Zhang, Yongjian Wu, and Feiyue Huang. AAAI, 2019.
Fine-Grained Image Retrieval via Piecewise Cross Entropy loss.
Xianxian Zeng, Yun Zhang, Xiaodong Wang, Kairui Chen, Dong Li, and Weijun Yang. Image and Vision Computing, 2020.
Category-Specific Nuance Exploration Network for Fine-Grained Object Retrieval.
Shijie Wang, Zhihui Wang, Haojie Li, and Wanli Ouyang. AAAI, 2022.
Fine-Grained Sketch-Based Image Retrieval by Matching Deformable Part Models.
Yi Li, Timothy M. Hospedales, Yi-Zhe Song, and Shaogang Gong. BMVC, 2014.
Sketch Me That Shoe.
Qian Yu, Feng Liu, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, and Chen Change Loy. CVPR, 2016.
Cross-domain Generative Learning for Fine-Grained Sketch-Based Image Retrieval.
Kaiyue Pang, Yi-Zhe Song, Tao Xiang, and Timothy M. Hospedales. BMVC, 2017.
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval.
Jifei Song, Qian Yu, Yi-Zhe Song, Tao Xiang, and Timothy M. Hospedales. ICCV, 2017.
Synergistic Instance-Level Subspace Alignment for Fine-Grained Sketch-Based Image Retrieval.
Ke Li, Kaiyue Pang, Yi-Zhe Song, Timothy M. Hospedales, Tao Xiang, and Honggang Zhang. IEEE TIP, 2017.
Generalising Fine-Grained Sketch-Based Image Retrieval.
Kaiyue Pang, Ke Li, Yongxin Yang, Honggang Zhang, Timothy M. Hospedales, Tao Xiang, and Yi-Zhe Song. CVPR, 2019.
Solving Mixed-modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image Retrieval.
Kaiyue Pang, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, and Yi-Zhe Song. CVPR, 2020.
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval.
Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Yongxin Yang, Tao Xiang, and Yi-Zhe Song. CVPR, 2021.
Toward Fine-Grained Sketch-Based 3D Shape Retrieval.
Anran Qi, Yulia Gryaditskaya, Jifei Song, Yongxin Yang, Yonggang Qi, Timothy M. Hospedales, Tao Xiang, and Yi-Zhe Song. IEEE TIP, 2021.
Fine-Grained Instance-Level Sketch-Based Image Retrieval.
Qian Yu, Jifei Song, Yi-Zhe Song, Tao Xiang, and Timothy M. Hospedales. IJCV, 2021.
Adaptive Fine-Grained Sketch-Based Image Retrieval.
Ayan Kumar Bhunia, Aneeshan Sain, Parth Hiren Shah, Animesh Gupta, Pinaki Nath Chowdhury, Tao Xiang, and Yi-Zhe Song. ECCV, 2022.
Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples.
Xiu-Shen Wei, Peng Wang, Lingqiao Liu, Chunhua Shen, and Jianxin Wu. IEEE TIP, 2019.
Meta-Reinforced Synthetic Data for One-Shot Fine-Grained Visual Recognition.
Satoshi Tsutsui, Yanwei Fu, and David Crandall. NeurIPS, 2019.
Revisiting Pose-Normalization for Fine-Grained Few-Shot Recognition.
Luming Tang, Davis Wertheimer, and Bharath Hariharan. CVPR, 2020. [code]
Multi-attention Meta Learning for Few-shot Fine-grained Image Recognition.
Yaohui Zhu, Chenlong Liu, and Shuqiang Jiang. IJCAI, 2020.
Compositional Zero-Shot Learning via Fine-Grained Dense Feature Composition.
Dat Huynh, Ehsan Elhamifar. NeurIPS, 2020.
Fine-Grained Zero-Shot Learning with DNA as Side Information.
Sarkhan Badirli, Zeynep Akata, George Mohler, Christine J.Picard, and Murat Dundar. NeurIPS, 2021.
Weak-shot Fine-grained Classification via Similarity Transfer.
Junjie Chen, Li Niu, Liu Liu, and Liqing Zhang. NeurIPS, 2021.
BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification.
Xiaoxu Li, Jijie Wu, Zhuo Sun, Zhanyu Ma, Senior Member, Jie Cao, and Jing-Hao Xue. IEEE TIP, 2021.
Variational Feature Disentangling for Fine-Grained Few-Shot Classification.
Jingyi Xu, Mingzhen Huang, ShahRukh Athar, and Dimitris Samaras. ICCV, 2021. [code]
Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition.
James Hong, Matthew Fisher, Michael Gharbi, and Kayvon Fatahalian. ICCV, 2021.
Dual Attention Networks for Few-Shot Fine-Grained Recognition.
Shu-Lin Xu, Faen Zhang, Xiu-Shen Wei, and Jianhua Wang. AAAI, 2022.
Task Discrepancy Maximization for Fine-Grained Few-Shot Classification.
SuBeen Lee, WonJun Moon, and Jae-Pil Heo. CVPR, 2022.
ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval.
Quan Cui, Qing-Yuan Jiang, Xiu-Shen Wei, Wu-Jun Li, and Osamu Yoshie. ECCV, 2020.
Deep Saliency Hashing for Fine-Grained Retrieval.
Sheng Jin, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhu, Lei Zhang, and Xiansheng Hua. IEEE TIP, 2020.
A2-NET: Learning Attribute-Aware Hash Codes for Large-Scale Fine-Grained Image Retrieval.
Xiu-Shen Wei, Yang Shen, Xuhao Sun, Han-Jia Ye, and Jian Yang. NeurIPS, 2021.
Deep Listwise Triplet Hashing for Fine-Grained Image Retrieval.
Yuchen Liang, Yan Pan, Hanjiang Lai, Wei Liu, and Jian Yin. IEEE TIP, 2022.
Sub-Region Localized Hashing for Fine-Grained Image Retrieval.
Xinguang Xiang, Yajie Zhang, Lu Jin, Zechao Li, and Jinhui Tang. IEEE TIP, 2022.
Fine-Grained Hashing With Double Filtering.
Zhen-Duo Chen, Xin Luo, Yongxin Wang, Shanqing Guo, and Xin-Shun Xu. IEEE TIP, 2022.
SEMICON: A Learning-to-hash Solution for Large-Scale Fine-Grained Image Retrieval.
Yang Shen, Xuhao Sun, Xiu-Shen Wei, Qing-Yuan Jiang, and Jian Yang. ECCV, 2022.
Grafit: Learning Fine-Grained Image Representations With Coarse Labels.
Hugo Touvron, Alexandre Sablayrolles, Matthijs Douze, Matthieu Cord, and Hervé Jégou. ICCV, 2021.
Weakly Supervised Representation Learning with Coarse Labels.
Yuanhong Xu, Qi Qian, Hao Li, Rong Jin, and Juhua Hu.
Fine-grained Angular Contrastive Learning with Coarse Labels.
Guy Bukchin, Eli Schwartz, Kate Saenko, Ori Shahar, Rogerio Feris, Raja Giryes, and Leonid Karlinsky. CVPR, 2021.
Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approach.
Timnit Geru, Judy Hoffman, and Li Fei-Fei. ICCV, 2017.
Progressive Adversarial Networks for Fine-Grained Domain Adaptation.
Sinan Wang, Xinyang Chen, Yunbo Wang, Mingsheng Long, and Jianmin Wang. CVPR, 2020.
An Adversarial Domain Adaptation Network for Cross-Domain Fine-Grained Recognition.
Yimu Wang, Ren-Jie Song, Xiu-Shen Wei, and Lijun Zhang. WACV, 2020.
GrainSpace: A Large-Scale Dataset for Fine-Grained and Domain-Adaptive Recognition of Cereal Grains.
Lei Fan, Yiwen Ding, Dongdong Fan, Donglin Di, Maurice Pagnucco, and Yang Song. CVPR, 2022.
CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training.
Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. ICCV, 2017. [code]
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks.
Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. CVPR, 2018. [code]
FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery.
Krishna Kumar Singh, Utkarsh Ojha, and Yong Jae Lee. CVPR, 2019. [code]
Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation.
Tianyi Chen, Yi Liu, Yunfei Zhang, Si Wu, Yong Xu, Feng Liangbing, and Hau San Wong. ICCV, 2021.
The iNaturalist Species Classification and Detection Dataset.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. CVPR, 2018.
RPC: A Large-Scale Retail Product Checkout Dataset.
Xiu-Shen Wei, Quan Cui, Lei Yang, Peng Wang, and Lingqiao Liu. arXiv: 1901.07249, 2019. [project page]
Presence-Only Geographical Priors for Fine-Grained Image Classification.
Oisin Mac Aodha, Elijah Cole, and Pietro Perona. ICCV, 2019.
N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras.
Junho Kim, Jaehyeok Bae, Gangin Park, Dongsu Zhang, Young Min Kim. ICCV, 2021.
A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification.
Jong-Chyi Su, Zezhou Cheng, Subhransu Maji. CVPR, 2021.
Learning Deep Classifiers Consistent with Fine-Grained Novelty Detection.
Jiacheng Cheng, Nuno Vasconcelos. CVPR, 2021.
Your “Flamingo” is My “Bird”: Fine-Grained, or Not.
Dongliang Chang, Kaiyue Pang, Yixiao Zheng, Zhanyu Ma, Yi-Zhe Song, and Jun Guo. CVPR, 2021.
The section is being continually updated. Since CUB200-2011 is the most popularly used fine-grained dataset, we list the fine-grained recognition leaderboard by treating it as the test bed.
Method | Published | BBox? | Part? | External information? | Base model | Image resolution | Accuracy |
PB R-CNN | ECCV 2014 | ![]() | Alex-Net | 224x224 | 73.9% | ||
MaxEnt | NeurIPS 2018 | GoogLeNet | TBD | 74.4% | |||
PB R-CNN | ECCV 2014 | ![]() | ![]() | Alex-Net | 224x224 | 76.4% | |
PS-CNN | CVPR 2016 | ![]() | ![]() | CaffeNet | 454x454 | 76.6% | |
MaxEnt | NeurIPS 2018 | VGG-16 | TBD | 77.0% | |||
Mask-CNN | PR 2018 | ![]() | Alex-Net | 448x448 | 78.6% | ||
PC | ECCV 2018 | ResNet-50 | TBD | 80.2% | |||
DeepLAC | CVPR 2015 | ![]() | ![]() | Alex-Net | 227x227 | 80.3% | |
MaxEnt | NeurIPS 2018 | ResNet-50 | TBD | 80.4% | |||
Triplet-A | CVPR 2016 | ![]() | Manual labour | GoogLeNet | TBD | 80.7% | |
Multi-grained | ICCV 2015 | WordNet etc. | VGG-19 | 224x224 | 81.7% | ||
Krause et al. | CVPR 2015 | ![]() | CaffeNet | TBD | 82.0% | ||
Multi-grained | ICCV 2015 | ![]() | WordNet etc. | VGG-19 | 224x224 | 83.0% | |
TS | CVPR 2016 | VGGD+VGGM | 448x448 | 84.0% | |||
Bilinear CNN | ICCV 2015 | VGGD+VGGM | 448x448 | 84.1% | |||
STN | NeurIPS 2015 | GoogLeNet+BN | 448x448 | 84.1% | |||
LRBP | CVPR 2017 | VGG-16 | 224x224 | 84.2% | |||
PDFS | CVPR 2016 | VGG-16 | TBD | 84.5% | |||
Xu et al. | ICCV 2015 | ![]() | ![]() | Web data | CaffeNet | 224x224 | 84.6% |
Cai et al. | ICCV 2017 | VGG-16 | 448x448 | 85.3% | |||
RA-CNN | CVPR 2017 | VGG-19 | 448x448 | 85.3% | |||
MaxEnt | NeurIPS 2018 | Bilinear CNN | TBD | 85.3% | |||
GZ | IEEE TPAMI 2021 | ResNet-101 | 448x448 | 85.4% | |||
PC | ECCV 2018 | Bilinear CNN | TBD | 85.6% | |||
CVL | CVPR 2017 | Texts | VGG | TBD | 85.6% | ||
Mask-CNN | PR 2018 | ![]() | VGG-16 | 448x448 | 85.7% | ||
Peer-learning | ICCV 2021 | Web data | ResNet-32 | 224x224 | 85.7% | ||
GP-256 | ECCV 2018 | VGG-16 | 448x448 | 85.8% | |||
KP | CVPR 2017 | VGG-16 | 224x224 | 86.2% | |||
T-CNN | IJCAI 2018 | ResNet | 224x224 | 86.2% | |||
MA-CNN | ICCV 2017 | VGG-19 | 448x448 | 86.5% | |||
MaxEnt | NeurIPS 2018 | DenseNet-161 | TBD | 86.5% | |||
DeepKSPD | ECCV 2018 | VGG-19 | 448x448 | 86.5% | |||
OSME+MAMC | ECCV 2018 | ResNet-101 | 448x448 | 86.5% | |||
StackDRL | IJCAI 2018 | VGG-19 | 224x224 | 86.6% | |||
DFL-CNN | CVPR 2018 | VGG-16 | 448x448 | 86.7% | |||
Bi-Modal PMA | IEEE TIP 2020 | VGG-16 | 448x448 | 86.8% | |||
PC | ECCV 2018 | DenseNet-161 | TBD | 86.9% | |||
KERL | IJCAI 2018 | Attributes | VGG-16 | 224x224 | 87.0% | ||
HBP | ECCV 2018 | VGG-16 | 448x448 | 87.1% | |||
SAM | ECCV 2022 | DBTNet-50 | 224x224 | 87.26% | |||
Mask-CNN | PR 2018 | ![]() | ResNet-50 | 448x448 | 87.3% | ||
P-CNN | IEEE TPAMI 2022 | VGG-19 | 448x448 | 87.3% | |||
DFL-CNN | CVPR 2018 | ResNet-50 | 448x448 | 87.4% | |||
NTS-Net | ECCV 2018 | ResNet-50 | 448x448 | 87.5% | |||
HSnet | CVPR 2017 | ![]() | ![]() | GoogLeNet+BN | TBD | 87.5% | |
Bi-Modal PMA | IEEE TIP 2020 | ResNet-50 | 448x448 | 87.5% | |||
CIN | AAAI 2020 | ResNet-50 | 448x448 | 87.5% | |||
ProtoTree | CVPR 2021 | ResNet-32 | 224x224 | 87.5% | |||
MetaFGNet | ECCV 2018 | Auxiliary data | ResNet-34 | TBD | 87.6% | ||
Cross-X | CVPR 2020 | ResNet-50 | 448x448 | 87.7% | |||
GZ | IEEE TPAMI 2021 | MA-CNN | 448x448 | 87.7% | |||
DCL | CVPR 2019 | ResNet-50 | 448x448 | 87.8% | |||
ACNet | CVPR 2020 | VGG-16 | 448x448 | 87.8% | |||
TASN | CVPR 2019 | ResNet-50 | 448x448 | 87.9% | |||
ACNet | CVPR 2020 | ResNet-50 | 448x448 | 88.1% | |||
CIN | AAAI 2020 | ResNet-101 | 448x448 | 88.1% | |||
DBTNet-101 | NeurIPS 2019 | ResNet-101 | 448x448 | 88.1% | |||
Bi-Modal PMA | IEEE TIP 2020 | Texts | VGG-16 | 448x448 | 88.2% | ||
CMN | IEEE TIP 2022 | ResNet-50 | 448x448 | 88.2% | |||
GCL | AAAI 2020 | ResNet-50 | 448x448 | 88.3% | |||
AP-CNN | IEEE TIP 2021 | ResNet-50 | 448x448 | 88.4% | |||
LC3DOR | ICCV 2021 | ResNet-50 | 512x512 | 88.4% | |||
S3N | CVPR 2020 | ResNet-50 | 448x448 | 88.5% | |||
Sun et al. | AAAI 2020 | ResNet-50 | 448x448 | 88.6% | |||
FDL | AAAI 2020 | ResNet-50 | 448x448 | 88.6% | |||
Bi-Modal PMA | IEEE TIP 2020 | Texts | ResNet-50 | 448x448 | 88.7% | ||
SPS | ICCV 2021 | Resnet-50 | 448x448 | 88.70% | |||
DF-GMM | CVPR 2020 | ResNet-50 | 448x448 | 88.8% | |||
PMG | ECCV 2020 | VGG-16 | 550x550 | 88.8% | |||
FDL | AAAI 2020 | DenseNet-161 | 448x448 | 89.1% | |||
DP-Net | AAAI 2021 | ResNet-50 | 448x448 | 89.3% | |||
SnapMix | AAAI 2021 | ResNet-101 | 448x448 | 89.32% | |||
PMG | ECCV 2020 | ResNet-50 | 550x550 | 89.6% | |||
GHORD | CVPR 2021 | ResNet-50 | 448x448 | 89.6% | |||
API-Net | AAAI 2020 | DenseNet-161 | 512x512 | 90.0% | |||
PART | IEEE TIP 2021 | ResNet-101 | 448x448 | 90.1% | |||
DTRG | IEEE TIP 2022 | DenseNet-161 | 448x448 | 90.1% | |||
P2P-Net | CVPR 2022 | ResNet-50 | 448x448 | 90.2% | |||
Ge et al. | CVPR 2019 | GoogLeNet+BN | Shorter side is 800 px | 90.3% | |||
CAL | ICCV 2021 | ResNet-101 | 448x448 | 90.6% | |||
CP-CNN | IEEE TIP 2022 | ResNet-50 | 448x448 | 91.4% | |||
DCAL | CVPR 2022 | ViT-Base | 448x448 | 91.4% | |||
TransFG | AAAI 2022 | ViT-B_16 | 448x448 | 91.7% | |||
CAP | AAAI 2021 | Xception | 224x224 | 91.8% | |||
SR-GNN | IEEE TIP 2022 | Xception | 224x224 | 91.9% | |||
DCAL | CVPR 2022 | R50-ViT-Base | 448x448 | 92.0% |