premerge

examples/end-to-end-examples/support_chatbot/README.md

@@ -2,7 +2,7 @@
 
 In this tutorial, we will be creating an application that answers questions about the MosaicML composer codebase. The basic structure of this application will be a retrieval question answering system where the user will provide the chatbot with a question, and then a language model will answer the question based on the retrieved text. See some [great](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html#retrieval-question-answering) [materials](https://blog.langchain.dev/langchain-chat/) from [LangChain](https://python.langchain.com/en/latest/index.html) for more exploration on this type of application.
 
-By default the model that is used throughout is [MPT-7b](https://huggingface.co/mosaicml/mpt-7b), a 7-billion parameter large language model trained by MosaicML. See [our blog](https://www.mosaicml.com/blog/mpt-7b) for more details. We have also released a more powerful, 30-billion parameter model, which you can swap in to this example very easily. See [MPT-30b](https://huggingface.co/mosaicml/mpt-30b) for more details. To swap in the 30b model, all you need to do is change all occurrences of `mpt-7b` to `mpt-30b`. The important ones to change are the `model.pretrained_model_name_or_path` and the `tokenizer.name`, in the various [finetune](./mcli-yamls/finetune/) yamls. The other occurrences are just in names of runs and save folders. Depending on your hardware, and particularly if you get a CUDA `c10` error, you may also need to change `device_train_microbatch_size` from `auto` to `1` in the [finetune](./mcli-yamls/finetune/) yamls.
+By default the model that is used throughout is [MPT-30b-chat](https://huggingface.co/mosaicml/mpt-30b-chat), a 7-billion parameter large language model trained by MosaicML. See [our blog](https://www.mosaicml.com/blog/mpt-30b) for more details. Depending on your hardware, and particularly if you get a CUDA `c10` error, you may also need to change `device_train_microbatch_size` from `auto` to `1` in the [finetune](./mcli-yamls/finetune/) yamls.
 
 ![demo](web_app_screenshot.png)
 

examples/end-to-end-examples/support_chatbot/app_demo.py

@@ -31,7 +31,6 @@ def parse_args() -> Namespace:
         '--endpoint_url',
         type=str,
         default='https://models.hosted-on.mosaicml.hosting/mpt-30b-chat/v1/predict',
-        #default='https://mpt-30b-composer-finetuned-dmhpmi.inf.hosted-on.mosaicml.hosting/predict',
         required=False,
         help='The endpoint of our MosaicML LLM Model')
     parser.add_argument(

examples/end-to-end-examples/support_chatbot/mcli_yamls/conversion/convert_PyPi_stream_to_mds.yaml

@@ -10,7 +10,7 @@ compute:
 integrations:
 # Clone the examples repository so that we have access to the code in sec_10k_qa
 - integration_type: git_repo
-  git_repo: KuuCi/examples
+  git_repo: YOUR_GITHUB_USERNAME/examples
   git_branch: support-bot
   ssh_clone: false # Should be true if using a private repo
   path: /workspace/examples # Tell MCLI what path to clone the repo to
@@ -22,6 +22,6 @@ command: |
   cd /workspace/examples/examples/end-to-end-examples/support_chatbot/
   pip install -r requirements.txt
   python scripts/conversion/convert_PyPi_to_stream.py \
-    --out_root oci://mosaicml-internal-checkpoints/support-bot-demo/data/PyPi_mds/ \
+    --out_root CLOUD://BUCKET/support-bot-demo/data/PyPi_mds/ \
 
 image: mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04 # Use the Docker image provided by MosaicML

examples/end-to-end-examples/support_chatbot/mcli_yamls/conversion/convert_txt_to_stream.yaml

@@ -10,8 +10,8 @@ compute:
 integrations:
 # Clone the examples repository so that we have access to the code in sec_10k_qa
 - integration_type: git_repo
-  git_repo: KuuCi/examples
-  git_branch: support-bot
+  git_repo: YOUR_GITHUB_USERNAME/examples
+  #git_branch: support-bot
   ssh_clone: false # Should be true if using a private repo
   path: /workspace/examples # Tell MCLI what path to clone the repo to
 
@@ -22,7 +22,7 @@ command: |
   cd /workspace/examples/examples/end-to-end-examples/support_chatbot/
   pip install -r requirements.txt
   python scripts/conversion/convert_txt_to_stream.py \
-    --out_root oci://mosaicml-internal-checkpoints/support-bot-demo/data/composer_30b/ \
+    --out_root CLOUD://BUCKET/support-bot-demo/data/composer_30b/ \
     --in_root retrieval_data/composer
 
 image: mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04 # Use the Docker image provided by MosaicML

examples/end-to-end-examples/support_chatbot/mcli_yamls/deploy_llm.yaml

@@ -10,7 +10,7 @@ replicas: 1
 integrations:
 # Clone and install the examples repo so we can use the deployment helper from it
 - integration_type: git_repo
-  git_repo: KuuCi/examples
+  git_repo: YOUR_GITHUB_USERNAME/examples
   git_branch: support-bot
   ssh_clone: false
 # Clone the llm-foundry repo so we can use the HF to FasterTransformer convert script from it
@@ -38,7 +38,7 @@ model:
   model_parameters:
     ft_lib_path: /code/FasterTransformer/build/lib/libth_transformer.so
     # FT checkpoint path is hardcoded in MPTFTModelHandler at /tmp/mpt
-    model_name_or_path: mosaicml/mpt-30b # This is used for the tokenzier
+    model_name_or_path: mosaicml/mpt-30b-chat # This is used for the tokenzier
     gpus: 4
 
 image: mosaicml/inference:0.1.29