Skip to content

Latest commit

 

History

History
163 lines (104 loc) · 7.67 KB

README.md

File metadata and controls

163 lines (104 loc) · 7.67 KB
Raw

Ranked-List-Loss-for-Deep-Metric-Learning

[Paper] [Slides] [Poster]

This work is being extended for TPAMI submission, with the main target to improve this work further.

In deep metric learning, The improvements over time have been marginal?

Recently, in paper A Metric Learning Reality Check, it is reported that the improvements over time have been marginal at best. Is it true? I present my personal viewpoints as follows:

  • First of all, acedemic research progress is naturally slow, continuous and tortuous. Beyond, it is full of flaws on its progress. For example,

    • In person re-identification, several years ago, some researchers vertically split one image into several parts for alignment, which is against the design of CNNs and non-meaningful. Because deep CNNs are designed to be invariant against translation, so that hand-crafted alignment is unnecessary.
    • The Adam optimiser is found to be very sensitive to the setting of delta recently.
    • ...

  • There are some vital breakthroughs over the time although they seem to be trivial now.

    • Multibatch Method, after which people rarely use rigid input formats.

      • Before this milestone, we heard a lot about siamese networks, triplet networks, etc.
      • After Multibatch Method, we construct doublets, triplets, or high-order tuples directly in the embedding space.

    • The importance of sample mining/weighting becomes clearer for our community. Of course, there exist many variants of sample mining/weighting for different scenarios.

    • Our community become much more open-minded: all methods which learn discriminative representations can be categorised into deep metric learning, e.g., softmax + categorical cross entropy.

    • ...

Citation

If you find our code and paper help your research, please kindly cite our work:

InProceedings{Wang_2019_CVPR,
author = {Wang, Xinshao and Hua, Yang and Kodirov, Elyor and Hu, Guosheng and Garnier, Romain and Robertson, Neil M.},
title = {Ranked List Loss for Deep Metric Learning},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}

To Visualise the Repository Tree Structure

cd ./Ranked-List-Loss-for-Deep-Metric-Learning
tree

Dependencies and Setup

The core functions are implemented using C++ in the caffe framework. We use matlab interfaces matcaffe for data preparation.

  • Clone our repository: Simply copy and execute following commands in the command line

    git clone [email protected]:XinshaoAmosWang/Ranked-List-Loss-for-D
eep-Metric-Learning.git
cd Ranked-List-Loss-for-Deep-Metric-Learning/

  • Install dependencies on Ubuntu 16.04

    sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libopenblas-dev
sudo apt-get install python-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

  • Install MATLAB 2017b

    Download and Run the install binary file

    ./install

  • Compile Caffe and matlab interface

    Note you may need to change some paths in Makefile.config according your system environment and MATLAB path

    cd CaffeMex_RLL_GR_V03_Simp
make -j8  && make matcaffe

Usage

Training on Stanford Online Product dataset.

  • Data preparation for SOP

    Downlaod Stanford_Online_Products dataset from ftp://cs.stanford.edu/cs/cvgl/Stanford_Online_Products.zip

    For simplicity, you can use the data mat file in pre_post_process directory, which is ready training and testing scripts. To solve the data path, you can do eithor a or b:

      a. Changing the path within the mat files. 
  b. A Simpler way: Create a soft link of your data
  e.g sudo ln -s /.../Stanford_Online_Products /home/xinshao/Papers_Projects/Data/Stanford_Online_Products

  • Train & Test

    Run the training and testing scripts in the training folder of a specific setting defined by its corresponding prototxt folder.

Training on In-shop Clothes

Our trained models on SOP, In-shop Clothes

Training on custom datasets

You only need to create training/testing mat files with the same structure as SOP_TrainImagePathBoxCell.mat and SOP_TestImagePathBoxCell.mat in directory SOP_GoogLeNet_Ori_V05/pre_pro_process.

e.g. SOP_TrainImagePathBoxCell.mat contains , TrainImagePathBoxCell storing all image paths and class_ids storing their corresponding semantic labels.

Application on person re-identification

Common questions

1. What does ranking mean?

  • Given a query, the objective is to rank its postive set in front of its negative set by a distance margin.

  • We do not need to consider the exact order of examples within the positive and negative sets.

2. How is a loss function related with deep metric learning?

Please see our discussion in the paper.

Acknowledgements

Our work benefits from:

Licence

BSD 3-Clause "New" or "Revised" License

Contact

Xinshao Wang (You can call me Amos as well) xinshao dot wang at eng dot ox dot ac dot uk