done

examples/inference-deployments/mpt/README.md

@@ -7,11 +7,11 @@ Check out [this blog post](https://www.mosaicml.com/blog/mpt-7b) for more inform
 You’ll find in this folder:
 
 - Model YAMLS - read [docs](https://docs.mosaicml.com/projects/mcli/en/latest/inference/inference_schema.html) for an explanation of each field.
-    - `mpt_7b_simple.yaml` - an optimized no-code yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).
-    - `mpt_30b_simple.yaml` - an optimized no-code yaml to deploy [MPT-30B Base](https://huggingface.co/mosaicml/mpt-30b).
+    - `mpt_7b.yaml` - an optimized no-code yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).
+    - `mpt_30b.yaml` - an optimized no-code yaml to deploy [MPT-30B Base](https://huggingface.co/mosaicml/mpt-30b).
     - `mpt_30b_ft.yaml` - a yaml to deploy [MPT-30B Base](https://huggingface.co/mosaicml/mpt-30b).
     - `mpt_30b_instruct_ft.yaml` - a yaml to deploy [MPT-30B Instruct](https://huggingface.co/mosaicml/mpt-30b-instruct).
-    - `mpt_7b.yaml` - a yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).
+    - `mpt_7b_custom.yaml` - a custom yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).
     - `mpt_7b_instruct.yaml` - a yaml to deploy [MPT-7B Intstruct](https://huggingface.co/mosaicml/mpt-7b-instruct).
     - `mpt_7b_storywriter.yaml` - a yaml to deploy [MPT-7B StoryWriter](https://huggingface.co/mosaicml/mpt-7b-storywriter).
 - Model handlers - these define how your model should be loaded and how the model should be run when receiving a request. You can use the default handlers here or write your custom model handler as per instructions [here](https://docs.mosaicml.com/projects/mcli/en/latest/inference/deployment_features.html#custom-model-handlers).

examples/inference-deployments/mpt/mpt_7b.yaml

@@ -1,19 +1,12 @@
-name: mpt-7b
+name: mpt-7b-simple
 compute:
   gpus: 1
   instance: oci.vm.gpu.a10.1
 image: mosaicml/inference:0.1.16
-replicas: 1
 command: |
-  export PYTHONPATH=$PYTHONPATH:/code/examples
-integrations:
-- integration_type: git_repo
-  git_repo: mosaicml/examples
-  ssh_clone: false
-  git_commit: df65ce9448f2e4c7803f7082930f80c8dc4e8fe1
-model:
-  download_parameters:
+  export PYTHONPATH=/code
+replicas: 1
+default_model:
+  model_type: mpt-7b
+  checkpoint_path:
     hf_path: mosaicml/mpt-7b
-  model_handler: examples.inference-deployments.mpt.mpt_handler.MPTModelHandler
-  model_parameters:
-    model_name: mosaicml/mpt-7b

examples/inference-deployments/mpt/mpt_7b_custom.yaml

@@ -0,0 +1,19 @@
+name: mpt-7b
+compute:
+  gpus: 1
+  instance: oci.vm.gpu.a10.1
+image: mosaicml/inference:0.1.16
+replicas: 1
+command: |
+  export PYTHONPATH=$PYTHONPATH:/code/examples
+integrations:
+- integration_type: git_repo
+  git_repo: mosaicml/examples
+  ssh_clone: false
+  git_commit: df65ce9448f2e4c7803f7082930f80c8dc4e8fe1
+model:
+  download_parameters:
+    hf_path: mosaicml/mpt-7b
+  model_handler: examples.inference-deployments.mpt.mpt_handler.MPTModelHandler
+  model_parameters:
+    model_name: mosaicml/mpt-7b