Add document-retrieval to Hub as a task

[merveenoyan]

This PR adds document retrieval to Hub for following models that are used heavily and now there's a lot of them:

• MrLight/dse-qwen2-2b-mrl-v1
• vidore/colpali-v1.2
• llamaindex/vdr-2b-multi-v1

Icon looks like this:

Other names I thought of:

• multimodal-feature-extraction (ambiguous, also overlaps with zero shot image classification)
• document-feature-extraction (sounds like plain document embeddings for a document backbone, best if this doesn't happen and those models are covered under image-feature-extraction instead)
• zero-shot-document-classification (too long, sounds odd but would be accurate, but we use these models often for retrieval and not this purpose)
  this name is very to the point so best if it stays like it

[merveenoyan]

@NielsRogge

[merveenoyan]

@pcuenca can you leave a review? 👀💗

[julien-c]

is it not already very close to image-feature-extraction? How many models would be covered by this? (i'm wondering if it isn't too specific)

[julien-c]

(no strong opinion though)

[pcuenca]

Sounds a bit NLP-y to me. Could we use something like "visual-document-retrieval", for symmetry with "visual-question-answering"?

[pcuenca]

How many models would be covered by this? (i'm wondering if it isn't too specific)

I have the same question. Would, for example, all the ColPali models be included here?

[pcuenca]

We'd also have to tag multiple models before merging, as usual.

[merveenoyan]

@pcuenca I wanted to wait for the naming consensus before opening PRs to them. we can do visual-document-retrieval yes.

@julien-c it's actually not. those are singular image backbones used to train traditional vision models. these models on the contrast are zero shot models built on VLMs to do document retrieval on multimodal RAG pipelines. they're a bit like CLIP, but for documents, and they're not used for classification (they have long context length and have fine grained image understanding). the number of models keep increasing as number of VLMs increase and they're all wrongly tagged hence this PR. (there's ColPali, ColQwen, ColSmolVLM, DSE models and more now)

here's an tldr explainer on how they're used https://x.com/mervenoyann/status/1831409380040044762?s=46

[julien-c]

ok, sounds good

[Vaibhavs10]

maybe image-document-retrieval so that it's more in sync with the current task vision/ audio related tasks names?

[pcuenca]

Ok so the items in discussion were:

• Name: @merveenoyan thinks image-document-retrieval is not ideal, so can we move forward with visual-document-retrieval?
• hideInDatasets: if there are not many, I'd hide and we can "unhide" later on, but deferring to @merveenoyan to open PRs and decide once the name has been confirmed.

[merveenoyan]

@pcuenca sorry I was busy with smolagents sprint. I'll edit with this visual-document-retrieval, and hide in datasets for now

[merveenoyan]

locally lint passes with format and check too, idk why this is happening (I also did pnpm i just in case something wasn't matching the CI's linting setup)

[Vaibhavs10]

ah yes! the failing CI is unrelated! (other PRs suffer from the same thing)

[Vaibhavs10]

green CI yay 🙏