Tweaks for training people

A few tweaks to the repo to help training faces, though I've also used these same settings to train objects and styles.

Also adding a notebook to make it easier to run the code on vast.ai / runpod / other cloud computing platforms.

configs/stable-diffusion/v1-finetune_unfrozen.yaml

@@ -117,4 +117,4 @@ lightning:
 
   trainer:
     benchmark: True
-    max_steps: 800
+    max_steps: 3000

dreambooth_runpod_joepenna.ipynb

@@ -0,0 +1,272 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "aa2c1ada",
+   "metadata": {},
+   "source": [
+    "# Dreambooth\n",
+    "### Notebook implementation by Joe Penna (@MysteryGuitarM on Twitter)\n",
+    "https://github.com/JoePenna/Dreambooth-Stable-Diffusion\n",
+    "\n",
+    "### If on runpod / vast.ai / etc, spin up an A6000 or A100 pod using a Stable Diffusion template with Jupyter pre-installed."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c4d54876",
+   "metadata": {},
+   "source": [
+    "## Check GPU"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a3d20275-7c9b-447f-b160-dc556fcaee9a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# mostly for Google Colab -- you should know what you're running on Runpod / Vast.ai / etc\n",
+    "from IPython.display import HTML\n",
+    "from subprocess import getoutput\n",
+    "s = getoutput('nvidia-smi')\n",
+    "print(s)\n",
+    "# or simply\n",
+    "#!nvidia-smi -L"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b8347baa",
+   "metadata": {},
+   "source": [
+    "## Installation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ed803d51-d42f-4b07-9582-5cf1c9b52274",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# GIT THE REPO\n",
+    "!git clone https://github.com/JoePenna/Dreambooth-Stable-Diffusion\n",
+    "\n",
+    "# OR THE ORIGINAL\n",
+    "# !git clone https://github.com/XavierXiao/Dreambooth-Stable-Diffusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7b971cc0",
+   "metadata": {},
+   "source": [
+    "## Build Environment"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#BUILD ENV\n",
+    "%cd /workspace/Dreambooth-Stable-Diffusion\n",
+    "!pip install omegaconf einops pytorch-lightning==1.6.5 test-tube transformers kornia -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers -e git+https://github.com/openai/CLIP.git@main#egg=clip\n",
+    "!pip install setuptools==59.5.0\n",
+    "!pip install pillow==9.0.1\n",
+    "!pip install torchmetrics==0.6.0\n",
+    "!pip install -e ."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "17d1d11a",
+   "metadata": {},
+   "source": [
+    "## Generate regularization images (you should change the code below)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ed07a5df",
+   "metadata": {},
+   "source": [
+    "This repo simultaneously trains both your token **and** retrains your class.\n",
+    "\n",
+    "From cursory testing, it does not seem like the number of reg images affects the model too much.\n",
+    "\n",
+    "However, it does affect your class greatly."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67f9ff0c-b529-4c7c-8e26-8388d70a5d91",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#GENERATE 50 images\n",
+    "!python scripts/stable_txt2img.py --seed 10 --ddim_eta 0.0 --n_samples 1 --n_iter 50 --scale 10.0 --ddim_steps 50  --ckpt /workspace/stable-diffusion-webui/model.ckpt \\\n",
+    "--prompt \"person\"\n",
+    "\n",
+    "# IMPORTANT!! Change \"person\" above, and on the cell below:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "00717137",
+   "metadata": {},
+   "source": [
+    "## Upload your samples"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c90ed3ab",
+   "metadata": {},
+   "source": [
+    "Drag your training images into /workspace/Dreambooth-Stable-Diffusion/training_samples\n",
+    "\n",
+    "At the moment, it seems like 10-20 images of someone's face is enough."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ad4e50df",
+   "metadata": {},
+   "source": [
+    "## Prep Training Variables"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a26964af",
+   "metadata": {},
+   "source": [
+    "Navigate to:\n",
+    "/workspace/Dreambooth-Stable-Diffusion/ldm/data/personalized.py\n",
+    "\n",
+    "Change \"sks\" in line 11 to whatever you want your token to be.\n",
+    "\n",
+    "e.g. I changed mine to\n",
+    "```python\n",
+    "training_templates_smallest = [\n",
+    "    'joepenna {}',\n",
+    "]\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "37612c32",
+   "metadata": {},
+   "source": [
+    "The last line of this file trains for `3000` steps.\n",
+    "/workspace/Dreambooth-Stable-Diffusion/configs/stable-diffusion/v1-finetune_unfrozen.yaml\n",
+    "\n",
+    "If training a person or subject, keep an eye on your project's `/images/train/samples_scaled_gs-00xxxx` generations.\n",
+    "\n",
+    "If training a style, keep an eye on your project's `/images/train/samples_gs-00xxxx` generations."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "12f19de7",
+   "metadata": {},
+   "source": [
+    "## Start Training (you should also change the code below)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# START THE TRAINING\n",
+    "!python \"main.py\" \\\n",
+    " --base configs/stable-diffusion/v1-finetune_unfrozen.yaml \\\n",
+    " -t \\\n",
+    " --actual_resume \"/workspace/stable-diffusion-webui/model.ckpt\" \\\n",
+    " --reg_data_root \"/workspace/Dreambooth-Stable-Diffusion/outputs/txt2img-samples/samples\" \\\n",
+    " -n \"project_name\" \\\n",
+    " --gpus 0, \\\n",
+    " --data_root \"/workspace/Dreambooth-Stable-Diffusion/training_imgs\" \\\n",
+    " --class_word \"person\" # << match this word to the class word from regularization images above"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3ab885a3",
+   "metadata": {},
+   "source": [
+    "## Generate Images With Trained Model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b955a68e",
+   "metadata": {},
+   "source": [
+    "Be sure to change `PROJECT_PATH` below to match the folder in:\n",
+    "/workspace/Dreambooth-Stable-Diffusion/logs\n",
+    "\n",
+    "Also change `sks person` in the prompt to the token *and* class_word."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b6e40a7a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!python scripts/stable_txt2img.py \\\n",
+    "--ddim_eta 0.0 --n_samples 1 --n_iter 4 \\\n",
+    "--scale 7.0 --ddim_steps 60 --ckpt \"/workspace/Dreambooth-Stable-Diffusion/logs/PROJECT_PATH_GOES_HERE/last.ckpt\" \\\n",
+    "--prompt \"sks person as a masterpiece portrait painting by John Singer Sargent in the style of Rembrandt\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e77031d7",
+   "metadata": {},
+   "source": [
+    "Trained generated images at /workspace/Dreambooth-Stable-Diffusion/outputs/"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.6 64-bit",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}