Fine-Grained Image Analysis Tutorial

PRICAI 2018 Tutorial T3

Tuesday, August 28th, 10:30 to 12:10

Location: Room Revolution

Organizers

• Xiu-Shen Wei (Megvii Nanjing Research, China)

• Jianxin Wu (Nanjing University, China)

Abstract

Fine-grained image recognition vs. traditional image recognition (coarse-grained) as an example.

In traditional computer vision research, the categories of the target objects in image analysis are usually coarse-grained categories, such as “dog”, “car” and “bird”. However, in many real-world applications, the target objects often belong to fine-grained categories which are from one common / specific coarse-grained category. For example, there are several images belonging to Husky, several ones belonging to Alaska, several belonging to Satsuma, and so on. Yet, all of these images are from the common coarse-grained category, i.e., “dog”.

Fine-grained image analysis is a research direction focusing on this kind of image tasks, which is a hot topic in computer vision and pattern recognition. The goals of fine-grained image analysis are localizing the fine-grained objects in these fine-grained images, recognizing the fine-grained object categories, retrieving the fine-grained objects and so on. Fine-grained image analysis is practical and valuable in diverse real-world applications such as biological research, bio-diversity protection, product recognition, clothes retrieval, etc.

In this tutorial, we will introduce the applications, models (especially for deep models), practical methods, and new developments of fine-grained image analysis in various computer vision scenarios.

Outline

1. Background

   1. A brief introduction of computer vision

   2. Traditional image recognition and image retrieval

   3. Deep learning and convolutional neural networks

2. Introduction

   1. Fine-grained images vs. generic images

   2. Various real-world applications of fine-grained images

   3. Challenges of fine-grained image analysis

   4. Fine-grained benchmark datasets

3. Fine-grained image retrieval

   1. Fine-grained image retrieval based on hand-crafted features

   2. Fine-grained image retrieval based on deep learning

4. Fine-grained image recognition

   1. Fine-grained image recognition with powerful representation learning

   2. Fine-grained image recognition with part-based approaches

5. Other computer vision tasks related to fine-grained image analysis

   1. Person / Vehicle re-identification

   2. Clothes retrieval

   3. Product recognition

6. New developments of fine-grained image analysis

   1. Fine-grained images with languages

   2. Few-shot fine-grained image recognition

Slides

Tutorial slides (25.8MB).

References / Resources

• X.-S. Wei, J.-H. Luo, J. Wu, and Z.-H. Zhou. Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval. IEEE Transactions on Image Processing (TIP), 2017, 26(6): 2868-2881. [project page]

• X.-S. Wei, C.-L. Zhang, Y. Li, C.-W. Xie, J. Wu, C. Shen, and Z.-H. Zhou. Deep Descriptor Transforming for Image Co-Localization. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI’17), Melbourne, Australia, 2017, pp. 3048-3054. [slides] [poster] [code]

• X.-S. Wei, C.-W. Xie, J. Wu, and C. Shen. Mask-CNN: Localizing Parts and Selecting Descriptors for Fine-Grained Bird Species Categorization. Pattern Recognition, 2018, 76:704-714. [project page]

• Y. Zhang, X.-S. Wei, J. Wu, J. Cai, J. Lu, V.-A. Nguyen, and M. N. Do. Weakly Supervised Fine-Grained Categorization with Part-Based Image Representation. IEEE Transactions on Image Processing (TIP), 2016, 25(4): 1713-1725.

• X.-S. Wei, P. Wang, L. Liu, C. Shen, and J. Wu. Piecewise Classifier Mappings: Learning Fine-Grained Learners for Novel Categories with Few Examples. arXiv preprint, arXiv:1805.04288, 2018.

• ...