Object detection has been expanded from a limited number of categories to open vocabulary. Moving forward, a complete intelligent vision system requires understanding more fine-grained object descriptions, object parts. In this work, we propose a detector with the ability to predict both open-vocabulary objects and their part segmentation. This ability comes from two designs:
- We train the detector on the joint of part-level, object-level and image-level data.
- We parse the novel object into its parts by its dense semantic correspondence with the base object.
[arXiv]
See installation instructions.
See Preparing Datasets and Preparing Models.
See Getting Started for demo, training and inference.
We provide a large set of baseline results and trained models in the Model Zoo.
The majority of this project is licensed under a MIT License. Portions of the project are available under separate license of referred projects, including CLIP, Detic and dino-vit-features. Many thanks for their wonderful works.
If you use VLPart in your research or wish to refer to the baseline results published here, please use the following BibTeX entries:
@article{peize2023vlpart,
title = {Going Denser with Open-Vocabulary Part Segmentation},
author = {Sun, Peize and Chen, Shoufa and Zhu, Chenchen and Xiao, Fanyi and Luo, Ping and Xie, Saining and Yan, Zhicheng},
journal = {arXiv preprint arXiv:2305.11173},
year = {2023}
}Grounded Segment Anything: From Objects to Parts: A dialogue system to detect, segment and edit anything in part-level in the image.
Semantic-SAM: A universal image segmentation model to enable segment and recognize anything at any desired granularity.