diff --git a/REPRODUCE_RESULTS.md b/REPRODUCE_RESULTS.md new file mode 100644 index 0000000..47e6530 --- /dev/null +++ b/REPRODUCE_RESULTS.md @@ -0,0 +1,63 @@ +# Reproduce Results +This is a guide for how to reproduce the results published in the paper. +## 1. Download the pretrained models +```bash +wget https://dl.fbaipublicfiles.com/dinov2/dinov2_vitb14/dinov2_vitb14_reg4_pretrain.pth +cd notebook/models +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/unvalidated-dino-cls-b-crosswalk-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/unvalidated-dino-cls-b-curbramp-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/unvalidated-dino-cls-b-obstacle-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/unvalidated-dino-cls-b-surfaceproblem-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-clip-cls-b-crosswalk-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-clip-cls-b-curbramp-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-clip-cls-b-obstacle-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-clip-cls-b-surfaceproblem-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-dino-cls-b-curbramp-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-dino-cls-b-obstacle-tags-best.pth?download=true +wget https://huggingface.co/projectsidewalk/sidewalk-tagger-ai-models/resolve/main/validated-dino-cls-b-surfaceproblem-tags-best.pth?download=true +cd ../.. +``` +## 2. Setup Conda Environment +Make sure your system has NVIDIA drivers and CUDA installed and then run these commands: +```bash +conda create -n sidewalk-tagger-ai python=3.10 +conda activate sidewalk-tagger-ai +conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia +pip install -r requirements.txt +pip install xformers --index-url https://download.pytorch.org/whl/cu118 +``` +## 3. Download & Preprocess Test Dataset +Make sure you are still in the conda environment that we just created. +```bash +chmod +x download_and_process_test_dataset.sh +./download_and_process_test_dataset.sh +``` +This can take a long time depending on your system specs. +## 4. Test +Make sure you are still in the conda environment that we created. + +In the `notebooks/test.py` file, there is a section called `params`. + +```python +params = { + 'label_type': 'crosswalk', # 'crosswalk' 'curbramp' 'surfaceproblem' 'obstacle' + 'pretrained_model_prefix': MODEL_PREFIXES['DINO'], # 'DINO' or 'CLIP' + 'dataset_type': 'validated', # 'unvalidated' or 'validated' + # ... +} +``` + +You should change these options to whatever you want to test. + +Now, we just run the test script! +```bash +cd notebooks +python test.py +``` + +Results are visible here: + +![image](https://github.com/user-attachments/assets/52d19021-00c0-454a-aced-1cf15f9feaee) +![image](https://github.com/user-attachments/assets/e3c16ce4-ffb2-44a6-ab6d-e44928011dac) + +You can repeat this process of specifying the params and running the test script until you are satisfied.