Skip to content

Commit

Permalink
expand documentation to offer two paths for weights
Browse files Browse the repository at this point in the history
Signed-off-by: ❤️ <[email protected]>
  • Loading branch information
cuuupid authored Jan 11, 2024
1 parent 22e677b commit 78b1a1f
Showing 1 changed file with 16 additions and 2 deletions.
18 changes: 16 additions & 2 deletions docs/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,22 @@ Prepare the model so multiple predictions run efficiently.

Use this _optional_ method to include any expensive one-off operations in here like loading trained models, instantiate data transformations, etc.

This method is also where you should download the weights for your model (e.g. using [`pget`](https://github.com/replicate/pget)). Storing your weights with your image directly can cause long `cog build` times and large images.
> If this is not an option for you, you can use the `--separate-weights` flag on `cog build` to store weights in a [separate layer](https://github.com/replicate/cog/blob/12ac02091d93beebebed037f38a0c99cd8749806/docs/getting-started.md?plain=1#L219).
Many models use this method to download their weights (e.g. using [`pget`](https://github.com/replicate/pget)).

This has some advantages:
- Smaller image sizes
- Faster build times
- Faster pushes and inference on [Replicate](https://replicate.com)

However, this may also significantly increase your `setup()` time.

As an alternative, some choose to store their weights directly in the image. You can simply leave your weights in the directory alongside your `cog.yaml` and ensure they are not excluded in your `.dockerignore` file.

While this will increase your image size and build time, it offers other advantages:
- Faster `setup()` time
- Ensures idempotency and reduces your model's reliance on external systems- Preserves reproducibility as your model will be self-contained in the image

> When using this method, you should use the `--separate-weights` flag on `cog build` to store weights in a [separate layer](https://github.com/replicate/cog/blob/12ac02091d93beebebed037f38a0c99cd8749806/docs/getting-started.md?plain=1#L219).
### `Predictor.predict(**kwargs)`

Expand Down

0 comments on commit 78b1a1f

Please sign in to comment.