Release SynthImgCap on Hugging Face

[NielsRogge]

Hello @jianjieluo 🤗

I'm Niels and work as ML engineer at Hugging Face. I discovered your work through ECCV, congrats, I indexed your paper here: https://huggingface.co/papers/2409.13592. The paper page lets people discuss about your paper and lets them find artifacts about it (your dataset for instance) you can also claim the paper as yours which will show up on your public profile at HF.

Would you like to host the datasets on https://huggingface.co/datasets? I see you're using Google Drive for it. Hosting on Hugging Face will give you more visibility, and will also allow people to do:

from datasets import load_dataset

dataset = load_dataset("your-hf-org/synthimgcap")

If you're down, leaving a guide here: https://huggingface.co/docs/datasets/image_dataset. We also support Webdataset, useful for image/video datasets: https://huggingface.co/docs/datasets/en/loading#webdataset.

Besides that, there's the dataset viewer which allows people to quickly explore the first few rows of the data in the browser.

After uploaded, we can also link the datasets to the paper page (read here) so people can discover your work.

What do you think?

Kind regards,

Niels

[jianjieluo]

Hi Niels @NielsRogge ,

Thank you so much for your invitation! I’m truly honored to be considered for hosting my dataset on Hugging Face, and I would be more than happy to proceed with this.

However, I have a couple of concerns I would like to address:

Timeline for Uploading: Do you have a specific deadline for starting the upload process? I have to submit my doctoral thesis for external review by October 9, which is a critical step for my graduation. Would it be acceptable if I begin uploading the SynthImgCap dataset to Hugging Face starting October 10?

Potential NSFW Content: While I have run a checker on the synthetic images to filter out potential NSFW content, there is still a possibility that some may slip through. I am concerned that Hugging Face’s excellent dataset viewer functionality might expose this content easier and lead to negative implications. Does Hugging Face have any effective strategies to manage this concern? If it requires a manual review of all images, it would be quite time-consuming for me.

I appreciate your understanding and look forward to your guidance on these matters.

Best regards,
Jianjie

[NielsRogge]

Hi,

There's no specific timeline for uploading, feel free to upload it whenever you prefer :)

Regarding potential NSFW Content, we do support adding the following in your dataset card:

---
tags:
- nsfw
---

This ensures that people then see a disclaimer before seeing the viewer, see e.g.: https://huggingface.co/UnfilteredAI/NSFW-gen-v2

[jianjieluo]

Hi,

Thank you for the flexibility and your guidance regarding both the timeline and the NSFW content. I’m glad to hear that there is no set deadline for uploading, which gives me peace of mind to prioritize my thesis submission. I will plan to start the upload process after October 10 as discussed.

Regarding the NSFW content, I appreciate the solution with the tag in the dataset card to show a disclaimer before accessing the viewer. This is very helpful and addresses my concern perfectly. I’ll be sure to add the necessary tags when I upload the dataset.

I look forward to hosting the dataset on Hugging Face and linking it to the paper page. Thanks again for your support!

Best regards,
Jianjie