Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
kliyer-ai committed Dec 4, 2024
1 parent eefe7e2 commit 745b12c
Show file tree
Hide file tree
Showing 2 changed files with 121 additions and 108 deletions.
16 changes: 7 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,30 +11,29 @@
<sup>*</sup> Equal Contribution
</p>

[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://compvis.github.io/CleanDIFT/)
[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://compvis.github.io/CleanDIFT/static/pdfs/cleandift.pdf)
[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://compvis.github.io/cleandift/)
[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://compvis.github.io/cleandift/static/pdfs/cleandift.pdf)
[![Weights](https://img.shields.io/badge/HuggingFace-Weights-orange)](https://huggingface.co/CompVis/cleandift)



This repository contains the official implementation of the paper "CleanDIFT: Diffusion Features without Noise".

We propose CleanDIFT, a novel method to extract noise-free, timestep-independent features by enabling diffusion models to work directly with clean input images. Our approach is efficient, training on a single GPU in just 30 minutes.

![teaser](./docs/static/images/teaser_fig.png)


## 🚀 Usage

### Setup

Just clone the repo and install the requirements via `pip install -r requirements.txt`, then you're ready to go.

### Training

In order to train a feature extractor on your own, you can run `python train.py`. The training script expects your data to be stored in `./data` with the following format: Single level directory with images named `filename.jpg` and corresponding json files `filename.json` that contain the key `caption`.
In order to train a feature extractor on your own, you can run `python train.py`. The training script expects your data to be stored in `./data` with the following format: Single level directory with images named `filename.jpg` and corresponding json files `filename.json` that contain the key `caption`.

### Feature Extraction

For feature extraction, please refer to one of the notebooks at [`notebooks`](https://github.com/CompVis/CleanDIFT/tree/main/notebooks). We demonstrate how to extract features and use them for semantic correspondence detection and depth prediction.
For feature extraction, please refer to one of the notebooks at [`notebooks`](https://github.com/CompVis/cleandift/tree/main/notebooks). We demonstrate how to extract features and use them for semantic correspondence detection and depth prediction.

Our checkpoints are fully compatible with the `diffusers` library. If you already have a pipeline using SD 1.5 or SD 2.1 from `diffusers`, you can simply replace the U-Net state dict:

Expand All @@ -48,7 +47,6 @@ state_dict = load_file(ckpt_pth)
unet.load_state_dict(state_dict, strict=True)
```


## 🎓 Citation

If you use this codebase or otherwise found our work valuable, please cite our paper:
Expand All @@ -62,4 +60,4 @@ If you use this codebase or otherwise found our work valuable, please cite our p
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
```
Loading

0 comments on commit 745b12c

Please sign in to comment.