From 7296d61b90fcc546a601f39a6b3794448f7a286d Mon Sep 17 00:00:00 2001
From: Armando Daniel Diaz Gonzalez
<61255126+arm-diaz@users.noreply.github.com>
Date: Thu, 13 Jun 2024 14:56:23 -0400
Subject: [PATCH 1/7] Native function calling with converse API
---
...rock_Mistral_native_function_calling.ipynb | 763 ++++++++++++++++++
1 file changed, 763 insertions(+)
create mode 100644 notebooks/Bedrock_Mistral_native_function_calling.ipynb
diff --git a/notebooks/Bedrock_Mistral_native_function_calling.ipynb b/notebooks/Bedrock_Mistral_native_function_calling.ipynb
new file mode 100644
index 0000000..733b520
--- /dev/null
+++ b/notebooks/Bedrock_Mistral_native_function_calling.ipynb
@@ -0,0 +1,763 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "e3eed142-6144-4ba2-a693-2bcfdeeae823",
+ "metadata": {},
+ "source": [
+ "# Amazon Bedrock Function Calling (Tool Use) with Converse API using Mistral Large"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4c7a66db-4d22-4e0f-883c-8e8daf6a4291",
+ "metadata": {},
+ "source": [
+ "In this Jupyter Notebook, we walkthrough an implementation of native function calling and agentic workflows with with the Converse API for Amazon Bedrock and Mistral Large. Function Calling is a powerful technique that allows large language models to connect to external tools, systems, or APIs to enable, which can be executed to perform actions based on user's input."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3c58ce86-3cb5-4a4b-9735-dc4684db731f",
+ "metadata": {},
+ "source": [
+ "The Converse or ConverseStream API is a unified structured text action for simplifying the invocations to Bedrock LLMs. It includes the possibility to define tools for implementing external functions that can be called or triggered from the LLMs."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9a5fcd65-01b7-418d-988e-f80d05ebb4a5",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "## Mistral Model Selection\n",
+ "\n",
+ "Today, Mistral Large supports native function calling with Converse API for Amazon Bedrock:\n",
+ "\n",
+ "### 3. Mistral Large\n",
+ "\n",
+ "- **Description:** A cutting-edge text generation model with top-tier reasoning capabilities. It can be used for complex multilingual reasoning tasks, including text understanding, transformation, and code generation.\n",
+ "- **Max Tokens:** 8,196\n",
+ "- **Context Window:** 32K\n",
+ "- **Languages:** English, French, German, Spanish, Italian\n",
+ "- **Supported Use Cases:** Synthetic Text Generation, Code Generation, RAG, or Agents\n",
+ "\n",
+ "### Performance and Cost Trade-offs\n",
+ "\n",
+ "The table below compares the model performance on the Massive Multitask Language Understanding (MMLU) benchmark and their on-demand pricing on Amazon Bedrock.\n",
+ "\n",
+ "| Model | MMLU Score | Price per 1,000 Input Tokens | Price per 1,000 Output Tokens |\n",
+ "|-----------------|------------|------------------------------|-------------------------------|\n",
+ "| Mistral Large | 81.2% | \\$0.008 | \\$0.024 |\n",
+ "\n",
+ "For more information, refer to the following links:\n",
+ "\n",
+ "1. [Mistral Model Selection Guide](https://docs.mistral.ai/guides/model-selection/)\n",
+ "2. [Amazon Bedrock Pricing Page](https://aws.amazon.com/bedrock/pricing/)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f3978ae9-57d0-4dca-b94b-b6493f8eade0",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "## Supported papameters\n",
+ "\n",
+ "The Mistral AI models have the following inference parameters.\n",
+ "\n",
+ "\n",
+ "```\n",
+ "{\n",
+ " \"prompt\": string,\n",
+ " \"max_tokens\" : int,\n",
+ " \"stop\" : [string], \n",
+ " \"temperature\": float,\n",
+ " \"top_p\": float,\n",
+ " \"top_k\": int\n",
+ "}\n",
+ "```\n",
+ "\n",
+ "The Mistral AI models have the following inference parameters:\n",
+ "\n",
+ "- **Temperature** - Tunes the degree of randomness in generation. Lower temperatures mean less random generations.\n",
+ "- **Top P** - If set to float less than 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation.\n",
+ "- **Top K** - Can be used to reduce repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation.\n",
+ "- **Maximum Length** - Maximum number of tokens to generate. Responses are not guaranteed to fill up to the maximum desired length.\n",
+ "- **Stop sequences** - Up to four sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.\n",
+ "\n",
+ "---"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "95db24c2-384a-4d1e-b6b6-a832ac002500",
+ "metadata": {},
+ "source": [
+ "### Local Setup (Optional)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "11710041-4077-4700-a47a-2459d1be4fb9",
+ "metadata": {},
+ "source": [
+ "For a local server, follow these steps to execute this jupyter notebook:\n",
+ "\n",
+ "1. **Configure AWS CLI**: Configure [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) with your AWS credentials. Run `aws configure` and enter your AWS Access Key ID, AWS Secret Access Key, AWS Region, and default output format.\n",
+ "\n",
+ "2. **Install required libraries**: Install the necessary Python libraries for working with SageMaker, such as [sagemaker](https://github.com/aws/sagemaker-python-sdk/), [boto3](https://github.com/boto/boto3), and others. You can use a Python environment manager like [conda](https://docs.conda.io/en/latest/) or [virtualenv](https://virtualenv.pypa.io/en/latest/) to manage your Python packages in your preferred IDE (e.g. [Visual Studio Code](https://code.visualstudio.com/)).\n",
+ "\n",
+ "3. **Create an IAM role for SageMaker**: Create an AWS Identity and Access Management (IAM) role that grants your user [SageMaker permissions](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html). \n",
+ "\n",
+ "By following these steps, you can set up a local Jupyter Notebook environment capable of deploying machine learning models on Amazon SageMaker using the appropriate IAM role for granting the necessary permissions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7c22cf2e-971a-4df9-9e27-1cd7a05d8307",
+ "metadata": {},
+ "source": [
+ "## Setup and Requirements"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1ba91a5b-4ceb-4e60-9f89-b31a4e446c00",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "1. Create an Amazon SageMaker Notebook Instance - [Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/gs-setup-working-env.html)\n",
+ " - For Notebook Instance type, choose ml.t3.medium.\n",
+ "2. For Select Kernel, choose [conda_python3](https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-prepare.html).\n",
+ "3. Install the required packages."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5b7ef5d8-904d-4cce-8522-e4b21279ceb1",
+ "metadata": {},
+ "source": [
+ " \n",
+ "\n",
+ "
NOTE:\n",
+ "\n",
+ "- For
Amazon SageMaker Studio, select Kernel \"
Python 3 (ipykernel)\".\n",
+ "\n",
+ "- For
Amazon SageMaker Studio Classic, select Image \"
Base Python 3.0\" and Kernel \"
Python 3\".\n",
+ "\n",
+ "
\n",
+ "\n",
+ "
NOTE:\n",
+ "\n",
+ "- For
Amazon SageMaker Studio, select Kernel \"
Python 3 (ipykernel)\".\n",
+ "\n",
+ "- For
Amazon SageMaker Studio Classic, select Image \"
Base Python 3.0\" and Kernel \"
Python 3\".\n",
+ "\n",
+ "
\n",
- "\n",
- "
NOTE:\n",
- "\n",
- "- For
Amazon SageMaker Studio, select Kernel \"
Python 3 (ipykernel)\".\n",
- "\n",
- "- For
Amazon SageMaker Studio Classic, select Image \"
Base Python 3.0\" and Kernel \"
Python 3\".\n",
- "\n",
- "