Skip to content

[ICLR 2024 Spotlight] "Negative Label Guided OOD Detection with Pretrained Vision-Language Models"

License

Notifications You must be signed in to change notification settings

XueJiang16/NegLabel

Repository files navigation

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Official PyTorch implementation of the ICLR 2024 (Spotlight) paper:

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, Bo Han

Abstract: In this paper, we propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases. We design a novel scheme for the OOD score collaborated with negative labels. Theoretical analysis helps to understand the mechanism of negative labels. Extensive experiments demonstrate that our method NegLabel achieves state-of-the-art performance on various OOD detection benchmarks and generalizes well on multiple VLM architectures. Furthermore, our method NegLabel exhibits remarkable robustness against diverse domain shifts.

Installation

The project is based on MMClassification. MMClassification is an open source image classification toolbox based on PyTorch. It is a part of the OpenMMLab project.

Below are quick steps for installation:

conda create -n open-mmlab python=3.8 pytorch==1.10 torchvision==0.11.2 cudatoolkit=11.3 -c pytorch -y
conda activate open-mmlab
pip install openmim
cd NegLabel
mim install -e .
pip install ftfy regex tqdm
pip install git+https://github.com/openai/CLIP.git

Dataset Preparation

In-distribution dataset

Please download ImageNet-1k and place the training data (not necessary) and validation data in ./data/id_data/imagenet_train and ./data/id_data/imagenet_val, respectively.

Out-of-distribution dataset

Following MOS, we use the following 4 OOD datasets for evaluation: iNaturalist, SUN, Places, and Textures.

Please refer to MOS, download OOD datasets and put them into ./data/ood_data/.

About Negative Labels

We provide the wordnet database in ./txtfiles. We dynamically calculate and select negative labels based on the given ID labels and the database before the inference stage. We do not directly provide txt files for the selected negative labels. For details, please refer to ./mmcls/models/classifiers/multi_modal.py/#L140.

OOD Detection Evaluation

To reproduce our results, please run:

bash ./run.sh

Citation

@inproceedings{
jiang2024negative,
title={Negative Label Guided {OOD} Detection with Pretrained Vision-Language Models},
author={Xue Jiang and Feng Liu and Zhen Fang and Hong Chen and Tongliang Liu and Feng Zheng and Bo Han},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=xUO1HXz4an}
}

About

[ICLR 2024 Spotlight] "Negative Label Guided OOD Detection with Pretrained Vision-Language Models"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published