From 745b12c565e2826a4148caf56c8f66b5da53cab9 Mon Sep 17 00:00:00 2001 From: Nick Stracke Date: Wed, 4 Dec 2024 23:59:45 +0100 Subject: [PATCH] update --- README.md | 16 ++-- docs/index.html | 213 ++++++++++++++++++++++++++---------------------- 2 files changed, 121 insertions(+), 108 deletions(-) diff --git a/README.md b/README.md index 5fd088a..44f2e7a 100644 --- a/README.md +++ b/README.md @@ -11,30 +11,29 @@ * Equal Contribution

-[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://compvis.github.io/CleanDIFT/) -[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://compvis.github.io/CleanDIFT/static/pdfs/cleandift.pdf) +[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://compvis.github.io/cleandift/) +[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://compvis.github.io/cleandift/static/pdfs/cleandift.pdf) [![Weights](https://img.shields.io/badge/HuggingFace-Weights-orange)](https://huggingface.co/CompVis/cleandift) - - This repository contains the official implementation of the paper "CleanDIFT: Diffusion Features without Noise". We propose CleanDIFT, a novel method to extract noise-free, timestep-independent features by enabling diffusion models to work directly with clean input images. Our approach is efficient, training on a single GPU in just 30 minutes. ![teaser](./docs/static/images/teaser_fig.png) - ## 🚀 Usage + ### Setup + Just clone the repo and install the requirements via `pip install -r requirements.txt`, then you're ready to go. ### Training -In order to train a feature extractor on your own, you can run `python train.py`. The training script expects your data to be stored in `./data` with the following format: Single level directory with images named `filename.jpg` and corresponding json files `filename.json` that contain the key `caption`. +In order to train a feature extractor on your own, you can run `python train.py`. The training script expects your data to be stored in `./data` with the following format: Single level directory with images named `filename.jpg` and corresponding json files `filename.json` that contain the key `caption`. ### Feature Extraction -For feature extraction, please refer to one of the notebooks at [`notebooks`](https://github.com/CompVis/CleanDIFT/tree/main/notebooks). We demonstrate how to extract features and use them for semantic correspondence detection and depth prediction. +For feature extraction, please refer to one of the notebooks at [`notebooks`](https://github.com/CompVis/cleandift/tree/main/notebooks). We demonstrate how to extract features and use them for semantic correspondence detection and depth prediction. Our checkpoints are fully compatible with the `diffusers` library. If you already have a pipeline using SD 1.5 or SD 2.1 from `diffusers`, you can simply replace the U-Net state dict: @@ -48,7 +47,6 @@ state_dict = load_file(ckpt_pth) unet.load_state_dict(state_dict, strict=True) ``` - ## 🎓 Citation If you use this codebase or otherwise found our work valuable, please cite our paper: @@ -62,4 +60,4 @@ If you use this codebase or otherwise found our work valuable, please cite our p archivePrefix={arXiv}, primaryClass={cs.CV} } -``` \ No newline at end of file +``` diff --git a/docs/index.html b/docs/index.html index f68192e..0dcee60 100644 --- a/docs/index.html +++ b/docs/index.html @@ -6,18 +6,20 @@ - - - + + + - - - - + + + + - + @@ -25,10 +27,10 @@ - + CleanDIFT: Diffusion Features without Noise - + @@ -124,7 +126,7 @@

🧹 CleanDIFT: Diffusion Features with
- @@ -136,7 +138,7 @@

🧹 CleanDIFT: Diffusion Features with - @@ -146,7 +148,7 @@

🧹 CleanDIFT: Diffusion Features with -
@@ -302,7 +309,7 @@

Semantic Segmentation

- +
@@ -332,7 +339,7 @@

Semantic Segmentation

- +
@@ -365,56 +372,56 @@

Semantic Segmentation

- - - - + + +