1_download_save.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download and Save the Model\n",
    "\n",
    "To save this model so that you can use it from various locations, including other notebooks or the model server, upload it to s3-compatible storage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Install the required packages and define a function for the upload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install boto3 botocore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download from Huggingface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "git_repo = \"https://huggingface.co/ibm-granite/granite-7b-instruct\"\n",
    "# git_repo = \"https://huggingface.co/ibm/merlinite-7b\"\n",
    "\n",
    "# git_repo = \"https://huggingface.co/instructlab/merlinite-7b-lab\"\n",
    "# git_repo = \"https://huggingface.co/instructlab/granite-7b-lab\"\n",
    "\n",
    "# git_repo = \"https://huggingface.co/google/flan-t5-small\"\n",
    "\n",
    "# git_repo = \"https://huggingface.co/codellama/CodeLlama-7b-hf\"\n",
    "\n",
    "# git_repo = \"https://huggingface.co/mosaicml/mpt-7b-chat\"\n",
    "# git_repo = \"https://huggingface.co/mosaicml/mpt-7b-instruct\"\n",
    "\n",
    "\n",
    "# These models will require Huggingface user/token authentication\n",
    "# set the environment variables for HF_USER and HF_TOKEN\n",
    "\n",
    "# HF_USER = os.getenv(\"HF_USER\") or os.getenv(\"HUGGINGFACE_USER\")\n",
    "# HF_TOKEN = os.getenv(\"HF_TOKEN\") or os.getenv(\"HUGGINGFACE_TOKEN\")\n",
    "\n",
    "# git_repo = f\"https://{HF_USER}:{HF_TOKEN}@huggingface.co/mistralai/Mistral-7B-Instruct-v0.2\"\n",
    "\n",
    "# git_repo = f\"https://{HF_USER}:{HF_TOKEN}@huggingface.co/meta-llama/Llama-2-7b-chat-hf\"\n",
    "# git_repo = f\"https://{HF_USER}:{HF_TOKEN}@huggingface.co/meta-llama/Llama-2-13b-chat-hf\"\n",
    "\n",
    "# git_repo = f\"https://{HF_USER}:{HF_TOKEN}@huggingface.co/meta-llama/Meta-Llama-3-8B\"\n",
    "# git_repo = f\"https://{HF_USER}:{HF_TOKEN}@huggingface.co/meta-llama/Meta-Llama-3.1-8B\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!git clone $git_repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "model_name = os.path.basename(git_repo)\n",
    "model_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper functions for upload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import boto3\n",
    "import botocore\n",
    "\n",
    "aws_access_key_id = os.environ.get('AWS_ACCESS_KEY_ID')\n",
    "aws_secret_access_key = os.environ.get('AWS_SECRET_ACCESS_KEY')\n",
    "endpoint_url = os.environ.get('AWS_S3_ENDPOINT')\n",
    "region_name = os.environ.get('AWS_DEFAULT_REGION')\n",
    "bucket_name = os.environ.get('AWS_S3_BUCKET')\n",
    "\n",
    "session = boto3.session.Session(aws_access_key_id=aws_access_key_id,\n",
    "                                aws_secret_access_key=aws_secret_access_key)\n",
    "\n",
    "s3_resource = session.resource(\n",
    "    's3',\n",
    "    config=botocore.client.Config(signature_version='s3v4'),\n",
    "    endpoint_url=endpoint_url,\n",
    "    region_name=region_name)\n",
    "\n",
    "bucket = s3_resource.Bucket(bucket_name)\n",
    "\n",
    "#upload the model directory without git\n",
    "def upload_directory_to_s3(local_directory, s3_prefix, remove_safetensors=True):\n",
    "    for root, dirs, files in os.walk(local_directory):\n",
    "        for filename in files:\n",
    "            file_path = os.path.join(root, filename)\n",
    "            relative_path = os.path.relpath(file_path, local_directory)\n",
    "            if \".git\" in relative_path:\n",
    "                print(f\"skipping {relative_path}\")\n",
    "                continue\n",
    "            s3_key = os.path.join(s3_prefix, relative_path)\n",
    "            print(f\"{file_path} -> {s3_key}\")\n",
    "            bucket.upload_file(file_path, s3_key)\n",
    "\n",
    "\n",
    "def list_objects(prefix):\n",
    "    filter = bucket.objects.filter(Prefix=prefix)\n",
    "    for obj in filter.all():\n",
    "        print(obj.key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Check the Storage Bucket\n",
    "\n",
    "In your S3 bucket, under the `models` upload prefix, run the `list_object` command. As best practice, to avoid mixing up model files, keep only one model and its required files in a given prefix or directory. This practice allows you to download and serve a directory with all the files that a model requires. \n",
    "\n",
    "If this is the first time running the code, this cell will have no output or the fraud model from the predictive AI/ML exercise.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "list_objects(\"models\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Upload and check again"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the function to upload the `models` folder in a rescursive fashion:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "upload_directory_to_s3(model_name, f\"models/{model_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "To confirm this worked, run the `list_objects` function again:\n",
    "\n",
    "This time, you should see files listed in the directory/prefix\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list_objects(\"models\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Next Step\n",
    "\n",
    "Now that you've saved the model to s3 storage, you can refer to the model by using the same data connection to serve the model as an API.\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3.9",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}