wip

examples/inference-deployments/mpt/README.md

@@ -7,6 +7,8 @@ Check out [this blog post](https://www.mosaicml.com/blog/mpt-7b) for more inform
 You’ll find in this folder:
 
 - Model YAMLS - read [docs](https://docs.mosaicml.com/projects/mcli/en/latest/inference/inference_schema.html) for an explanation of each field.
+    - `mpt_7b_simple.yaml` - an optimized no-code yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).
+    - `mpt_30b_simple.yaml` - an optimized no-code yaml to deploy [MPT-30B Base](https://huggingface.co/mosaicml/mpt-30b).
     - `mpt_30b_ft.yaml` - a yaml to deploy [MPT-30B Base](https://huggingface.co/mosaicml/mpt-30b).
     - `mpt_30b_instruct_ft.yaml` - a yaml to deploy [MPT-30B Instruct](https://huggingface.co/mosaicml/mpt-30b-instruct).
     - `mpt_7b.yaml` - a yaml to deploy [MPT-7B Base](https://huggingface.co/mosaicml/mpt-7b).

examples/inference-deployments/mpt/mpt_30b_simple.yaml

@@ -0,0 +1,17 @@
+name: mpt-30b-simple
+compute:
+  gpus: 2
+  gpu_type: a100_40gb
+image: mosaicml/inference:0.1.16
+replicas: 1
+command: |
+  export PYTHONPATH=/code/llm-foundry:/code
+integrations:
+- integration_type: git_repo
+  git_repo: mosaicml/llm-foundry
+  git_commit: 496b50bd588b1a7231fe54b05d70babb3620fc72
+  ssh_clone: false
+default_model:
+  model_type: mpt-30b
+  checkpoint_path:
+    hf_path: mosaicml/mpt-30b

examples/inference-deployments/mpt/mpt_7b_simple.yaml

@@ -0,0 +1,12 @@
+name: mpt-7b-simple
+compute:
+  gpus: 1
+  instance: oci.vm.gpu.a10.1
+image: mosaicml/inference:0.1.16
+ommand: |
+  export PYTHONPATH=/code
+replicas: 1
+default_model:
+  model_type: mpt-7b
+  checkpoint_path:
+    hf_path: mosaicml/mpt-7b