3-mlops/deploy-attrition-model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploying a web service to Azure Container Instance (ACI)\n",
    "\n",
    "This notebook shows the steps for deploying a model as service to ACI. The workflow is similar no matter where you deploy your model:\n",
    "\n",
    "1. Register the model.\n",
    "2. Prepare to deploy. (Specify assets, usage, compute target.)\n",
    "3. Deploy the model to the compute target.\n",
    "4. Test the deployed model, also called a web service.\n",
    "5. Consume the model using Power BI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
    "from azureml.core.compute import AksCompute, ComputeTarget\n",
    "from azureml.core.webservice import Webservice, AksWebservice\n",
    "from azureml.core.model import Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "print(azureml.core.VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get workspace\n",
    "Load existing workspace from the config file info."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.workspace import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.authentication import InteractiveLoginAuthentication\n",
    "interactive_auth = InteractiveLoginAuthentication(tenant_id=\"72f988bf-86f1-41af-91ab-2d7cd011db47\")\n",
    "\n",
    "ws = Workspace(subscription_id=\"3c8972d9-f541-46b2-b70b-d81baba3595d\",\n",
    "                  resource_group=\"amlworkshop\",\n",
    "                  workspace_name=\"amlworkshop\",\n",
    "                  auth=interactive_auth)\n",
    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Register the model\n",
    "Register an existing trained model, add description and tags.\n",
    "\n",
    "This is the model you've already trained using manual training or using [Automated Machine Learning](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-create-portal-experiments).\n",
    "\n",
    "In the code snippet below we're using the already trained model `original_model.pkl` that is saved in the folder that contains this notebook. We're registering this model with the name `IBM-attrition-model`. Later on we will use the same name in the scoring script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Register the model\n",
    "from azureml.core.model import Model\n",
    "\n",
    "# if the model is already registered as part of training then uncomment the line below. Make sure model is registered with the name \"IBM_attrition_model\"\n",
    "# attrition_model = Model(ws, 'IBM_attrition_model')\n",
    "\n",
    "# if the model is not already registered as part of training register the original_model.pkl file provided in the same folder as this notebook\n",
    "model = Model.register(model_path = \"original_model.pkl\", # this points to a local file\n",
    "                       model_name = \"IBM_attrition_model\", # this is the name the model is registered as\n",
    "                       tags = {'area': \"HR\", 'type': \"attrition\"},\n",
    "                       description = \"Attrition model to understand attrition risk\",\n",
    "                       workspace = ws)\n",
    "\n",
    "print(model.name, model.description, model.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare to deploy\n",
    "\n",
    "To deploy the model, you need the following items:\n",
    "\n",
    "- **An entry script**, this script accepts requests, scores the requests by using the model, and returns the results.\n",
    "- **Dependencies**, like helper scripts or Python/Conda packages required to run the entry script or model.\n",
    "- **The deployment configuration** for the compute target that hosts the deployed model. This configuration describes things like memory and CPU requirements needed to run the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Define your entry script and dependencies\n",
    "\n",
    "### Entry script\n",
    "\n",
    "We will first write the entry script as shown below. Note a few points in the entry script.\n",
    "\n",
    "The script contains two functions that load and run the model:\n",
    "\n",
    "**init()**: Typically, this function loads the model into a global object. This function is run only once, when the Docker container for your web service is started.\n",
    "\n",
    "When you register a model, you provide a model name that's used for managing the model in the registry. You use this name with the Model.get_model_path() method to retrieve the path of the model file or files on the local file system. If you register a folder or a collection of files, this API returns the path of the directory that contains those files.\n",
    "\n",
    "**run(input_data)**: This function uses the model to predict a value based on the input data. Inputs and outputs of the run typically use JSON for serialization and deserialization. You can also work with raw binary data. You can transform the data before sending it to the model or before returning it to the client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile score.py\n",
    "\n",
    "import pandas as pd\n",
    "import joblib\n",
    "from azureml.core.model import Model\n",
    "\n",
    "import json\n",
    "import pickle\n",
    "import numpy as np\n",
    "\n",
    "from inference_schema.schema_decorators import input_schema, output_schema\n",
    "from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType\n",
    "from inference_schema.parameter_types.pandas_parameter_type import PandasParameterType\n",
    "\n",
    "\n",
    "input_sample = pd.DataFrame(data=[{'Age': 41, 'BusinessTravel': 'Travel_Rarely', 'DailyRate': 1102, 'Department': 'Sales', 'DistanceFromHome': 1, 'Education': 2, 'EducationField': 'Life Sciences', 'EnvironmentSatisfaction': 2, 'Gender': 'Female', 'HourlyRate': 94, 'JobInvolvement': 3, 'JobLevel': 2, 'JobRole': 'Sales Executive', 'JobSatisfaction': 4, 'MaritalStatus': 'Single', 'MonthlyIncome': 5993, 'MonthlyRate': 19479, 'NumCompaniesWorked': 8, 'OverTime': 'No', 'PercentSalaryHike': 11, 'PerformanceRating': 3, 'RelationshipSatisfaction': 1, 'StockOptionLevel': 0, 'TotalWorkingYears': 8, 'TrainingTimesLastYear': 0, 'WorkLifeBalance': 1, 'YearsAtCompany': 6, 'YearsInCurrentRole': 4, 'YearsSinceLastPromotion': 0, 'YearsWithCurrManager': 5}])\n",
    "output_sample = np.array([0])\n",
    "\n",
    "\n",
    "def init():\n",
    "    global model\n",
    "    # This name is model.id of model that we want to deploy deserialize the model file back\n",
    "    # into a sklearn model\n",
    "    model_path = Model.get_model_path('IBM_attrition_model')\n",
    "    model = joblib.load(model_path)\n",
    "\n",
    "\n",
    "@input_schema('data', PandasParameterType(input_sample))\n",
    "@output_schema(NumpyParameterType(output_sample))\n",
    "def run(data):\n",
    "    try:\n",
    "        result = model.predict(data)\n",
    "        return json.dumps({\"result\": result.tolist()})\n",
    "    except Exception as e:\n",
    "        result = str(e)\n",
    "        return json.dumps({\"error\": result})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automatic schema generation\n",
    "To automatically generate a schema for your web service, provide a sample of the input and/or output in the constructor for one of the defined type objects. The type and sample are used to automatically create the schema. Azure Machine Learning then creates an OpenAPI (Swagger) specification for the web service during deployment.\n",
    "To use schema generation, include the _inference-schema_ package in your Conda environment file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define dependencies\n",
    "\n",
    "The following YAML is the Conda dependencies file we will use for inference. If you want to use automatic schema generation, your entry script must import the inference-schema packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.makedirs(\"deployment-config/\", exist_ok = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile deployment-config/inference-env.yml\n",
    "\n",
    "name: project_environment\n",
    "dependencies:\n",
    "- python=3.6.2\n",
    "\n",
    "- pip:\n",
    "  - sklearn-pandas\n",
    "  - azureml-defaults\n",
    "  - azureml-core\n",
    "  - inference-schema[numpy-support]\n",
    "- scikit-learn\n",
    "- pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Environment\n",
    "\n",
    "# Instantiate environment\n",
    "inference_env = Environment.from_conda_specification(name = \"inference-env\",\n",
    "                                                     file_path = \"deployment-config/inference-env.yml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Define your inference configuration\n",
    "\n",
    "The inference configuration describes how to configure the model to make predictions. This configuration isn't part of your entry script. It references your entry script and is used to locate all the resources required by the deployment. It's used later, when you deploy the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.model import InferenceConfig\n",
    "\n",
    "inference_config = InferenceConfig(entry_script='score.py', environment=inference_env)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Define your deployment configuration\n",
    "\n",
    "Before deploying your model, you must define the deployment configuration. The deployment configuration is specific to the compute target that will host the web service. The deployment configuration isn't part of your entry script. It's used to define the characteristics of the compute target that will host the model and entry script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.webservice import AciWebservice\n",
    "\n",
    "aciconfig = AciWebservice.deploy_configuration(cpu_cores=1,\n",
    "                                               memory_gb=1,\n",
    "                                               enable_app_insights=True,\n",
    "                                               auth_enabled=True,\n",
    "                                               tags = {'area': \"HR\", 'type': \"attrition\"}, \n",
    "                                               description='Explain predictions on employee attrition')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Deploy Model as Webservice on Azure Container Instance\n",
    "\n",
    "Deployment uses the inference configuration deployment configuration to deploy the models. The deployment process is similar regardless of the compute target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "service = Model.deploy(ws, name='predictattritionsvc1', models=[model], \n",
    "                       inference_config=inference_config, deployment_config=aciconfig)\n",
    "service.wait_for_deployment(True)\n",
    "print(service.state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Web service schema\n",
    "\n",
    "If you used automatic schema generation with your deployment, you can get the address of the OpenAPI specification for the service by using the `swagger_uri` property (For example, `print(service.swagger_uri)`). Use a `GET` request or open the URI in a browser to retrieve the specification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Swagger:\", service.swagger_uri)\n",
    "print(\"Scoring URL:\", service.scoring_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test the deployed model\n",
    "\n",
    "Every deployed web service provides a REST API, so you can create client applications in a variety of programming languages. If you've enabled key authentication for your service, you need to provide a service key as a `Bearer` token in your request header. If you've enabled token authentication for your service, you need to provide an Azure Machine Learning JWT token as a bearer token in your request header."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "\n",
    "# the sample below contains the data for an employee that is not an attrition risk\n",
    "sample = pd.DataFrame(data=[{'Age': 41, 'BusinessTravel': 'Travel_Rarely', 'DailyRate': 1102, 'Department': 'Sales', 'DistanceFromHome': 1, 'Education': 2, 'EducationField': 'Life Sciences', 'EnvironmentSatisfaction': 2, 'Gender': 'Female', 'HourlyRate': 94, 'JobInvolvement': 3, 'JobLevel': 2, 'JobRole': 'Sales Executive', 'JobSatisfaction': 4, 'MaritalStatus': 'Single', 'MonthlyIncome': 5993, 'MonthlyRate': 19479, 'NumCompaniesWorked': 8, 'OverTime': 'No', 'PercentSalaryHike': 11, 'PerformanceRating': 3, 'RelationshipSatisfaction': 1, 'StockOptionLevel': 0, 'TotalWorkingYears': 8, 'TrainingTimesLastYear': 0, 'WorkLifeBalance': 1, 'YearsAtCompany': 6, 'YearsInCurrentRole': 4, 'YearsSinceLastPromotion': 0, 'YearsWithCurrManager': 5}])\n",
    "\n",
    "# the sample below contains the data for an employee that is an attrition risk\n",
    "# sample = pd.DataFrame(data=[{'Age': 49, 'BusinessTravel': 'Travel_Rarely', 'DailyRate': 1098, 'Department': 'Research & Development', 'DistanceFromHome': 4, 'Education': 2, 'EducationField': 'Medical', 'EnvironmentSatisfaction': 4, 'Gender': 'Female', 'HourlyRate': 21, 'JobInvolvement': 3, 'JobLevel': 2, 'JobRole': 'Laboratory Technician', 'JobSatisfaction': 3, 'MaritalStatus': 'Single', 'MonthlyIncome': 711, 'MonthlyRate': 2124, 'NumCompaniesWorked': 8, 'OverTime': 'Yes', 'PercentSalaryHike': 8, 'PerformanceRating': 4, 'RelationshipSatisfaction': 3, 'StockOptionLevel': 0, 'TotalWorkingYears': 2, 'TrainingTimesLastYear': 0, 'WorkLifeBalance': 3, 'YearsAtCompany': 2, 'YearsInCurrentRole': 1, 'YearsSinceLastPromotion': 0, 'YearsWithCurrManager': 1}])\n",
    "\n",
    "\n",
    "# converts the sample to JSON string\n",
    "sample = pd.DataFrame.to_json(sample)\n",
    "\n",
    "# deserializes sample to a python object \n",
    "sample = json.loads(sample)\n",
    "\n",
    "# serializes sample to JSON formatted string as expected by the scoring script\n",
    "sample = json.dumps({\"data\":sample})\n",
    "\n",
    "# Will automatically send API key\n",
    "prediction = service.run(sample)\n",
    "\n",
    "print(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using plain HTTP Requests\n",
    "\n",
    "We can also just use regular HTTP `POST` request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "url = service.scoring_uri\n",
    "key1, key2 = service.get_keys()\n",
    "print(\"URL:\", url)\n",
    "print(\"Key:\", key1)\n",
    "\n",
    "headers = {'Content-Type':'application/json',\n",
    "           'Authorization': 'Bearer ' + key1}\n",
    "resp = requests.post(url, sample, headers=headers)\n",
    "\n",
    "print(\"prediction:\", resp.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now navigate to the Azure Portal, navigate to the Application Insights instance that is associated with the workspace, goto `Logs` and analyze for example the `requests` data. Your results should look simliar to this:\n",
    "\n",
    "![](images/app_insights.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deployment to AKS\n",
    "\n",
    "For deployment to AKS, we could also use Python, but let's use the Azure ML CLI to create a Continous Deployment pipeline!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Consume the model using Power BI\n",
    "You can also consume the model from Power BI. See details [here](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-consume-web-service#consume-the-service-from-power-bi).\n"
   ]
  }
 ],
 "metadata": {
  "file_extension": ".py",
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "mimetype": "text/x-python",
  "name": "python",
  "npconvert_exporter": "python",
  "pygments_lexer": "ipython3",
  "version": 3
 },
 "nbformat": 4,
 "nbformat_minor": 4
}