Project-3.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "CNDKREiQRJJX",
      "metadata": {
        "id": "CNDKREiQRJJX"
      },
      "source": [
        "<font size=\"+3\"><b>Assignment 3: Non-Linear Models and Validation Metrics</b></font>\n",
        "\n",
        "***\n",
        "* **Full Name** = Aarsh Shah\n",
        "* **UCID** = 30150079 \n",
        "***"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ce31b39a",
      "metadata": {
        "id": "ce31b39a"
      },
      "source": [
        "<font color='Blue'>\n",
        "In this assignment, you will need to write code that uses non-linear models to perform classification and regression tasks. You will also be asked to describe the process by which you came up with the code. More details can be found below. Please cite any websites or AI tools that you used to help you with this assignment.\n",
        "</font>\n",
        "\n",
        "|                **Question**                | **Point** |\n",
        "|:------------------------------------------:|:---------:|\n",
        "|           **Part 1: Regression**           |  **14.5** |\n",
        "|          Step 0: Import Libraries          |           |\n",
        "|             Step 1: Data Input             |    0.5    |\n",
        "|           Step 2: Data Processing          |     0     |\n",
        "| Step 3: Implement   Machine Learning Model |    0.5    |\n",
        "|           Step 4: Validate Model           |    0.5    |\n",
        "|         Step 5: Visualize   Results        |     3     |\n",
        "|                  Questions                 |     6     |\n",
        "|             Process Description            |     4     |\n",
        "|         **Part 2: Classification**         |  **17.5** |\n",
        "|             Step 1: Data Input             |     2     |\n",
        "|           Step 2: Data Processing          |    1.5    |\n",
        "| Step 3: Implement   Machine Learning Model |           |\n",
        "|            Step 4: Validate Mode           |           |\n",
        "|         Step 5: Visualize   Results        |     4     |\n",
        "|                  Questions                 |     6     |\n",
        "|             Process Description            |     4     |\n",
        "|   **Part 3: Observations/Interpretation**  |   **3**   |\n",
        "|           **Part 4: Reflection**           |   **2**   |\n",
        "|                  **Total**                 |   **37**  |\n",
        "|                                            |           |\n",
        "|                  **Bonus**                 |           |\n",
        "|         **Part 5: Bonus Question**         |   **3**   |"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cf275ca7",
      "metadata": {
        "id": "cf275ca7"
      },
      "source": [
        "### Import Libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "2b67a661",
      "metadata": {
        "id": "2b67a661"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5ee2d2c3",
      "metadata": {
        "id": "5ee2d2c3"
      },
      "source": [
        "# **Part 1: Regression (14.5 marks)**\n",
        "\n",
        "For this section, we will be continuing with the concrete example from yellowbrick. You will need to compare these results to the results from the previous assignment. Please use the results from the solution if you were unable to complete Assignment 2"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8219f163",
      "metadata": {
        "id": "8219f163"
      },
      "source": [
        "## **Step 1:** Data Input (0.5 marks)\n",
        "\n",
        "The data used for this task can be downloaded using the yellowbrick library:\n",
        "https://www.scikit-yb.org/en/latest/api/datasets/concrete.html\n",
        "\n",
        "Use the yellowbrick function `load_concrete()` to load the concrete dataset into the feature matrix `X` and target vector `y`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "2af8bd32",
      "metadata": {
        "id": "2af8bd32"
      },
      "outputs": [],
      "source": [
        "# TO DO: Import concrete dataset from yellowbrick library\n",
        "\n",
        "from yellowbrick.datasets import load_concrete\n",
        "X, y = load_concrete(data_home=None, return_dataset=False)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "42fea4cc",
      "metadata": {
        "id": "42fea4cc"
      },
      "source": [
        "## **Step 2:** Data Processing (0 marks)\n",
        "\n",
        "Data processing was completed in the previous assignment. No need to repeat here.\n",
        "\n",
        "<font color='red'>\n",
        "This is just for your information and no action is required from you for this step.\n",
        "</font>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2a245d00",
      "metadata": {
        "id": "2a245d00"
      },
      "source": [
        "## **Step 3:** Implement Machine Learning Model (0.5 marks)\n",
        "\n",
        "1. Import the Decision Tree, Random Forest and Gradient Boosting Machines regression models from sklearn\n",
        "2. Instantiate the three models with `max_depth = 5`. Are there any other parameters that you will need to set?\n",
        "3. Implement each machine learning model with `X` and `y`"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3f994e31",
      "metadata": {
        "id": "3f994e31"
      },
      "source": [
        "## **Step 4:** Validate Model (0.5 marks)\n",
        "\n",
        "Calculate the average training and validation accuracy using mean squared error with cross-validation. To do this, you will need to set `scoring='neg_mean_squared_error'` in your `cross_validate` function and negate the results (multiply by -1)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5fc3f7a8",
      "metadata": {
        "id": "5fc3f7a8"
      },
      "source": [
        "## **Step 5:** Visualize Results (3 marks)\n",
        "\n",
        "1. Create a pandas DataFrame `results` with columns: Training accuracy and Validation accuracy, and index: DT, RF and GB\n",
        "2. Add the accuracy results to the `results` DataFrame\n",
        "3. Print `results`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "fdc93a78",
      "metadata": {
        "id": "fdc93a78"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Training Score</th>\n",
              "      <th>Validation Score</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>DT</th>\n",
              "      <td>47.918561</td>\n",
              "      <td>163.087775</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>RF</th>\n",
              "      <td>32.055432</td>\n",
              "      <td>156.404972</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>GB</th>\n",
              "      <td>3.73927</td>\n",
              "      <td>99.360259</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Training Score Validation Score\n",
              "DT      47.918561       163.087775\n",
              "RF      32.055432       156.404972\n",
              "GB        3.73927        99.360259"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE FOR STEPS 3-5\n",
        "# Note: for any random state parameters, you can use random_state = 0\n",
        "# HINT: USING A LOOP TO STORE THE DATA IN YOUR RESULTS DATAFRAME WILL BE MORE EFFICIENT\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.tree import DecisionTreeRegressor\n",
        "from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor\n",
        "from sklearn.model_selection import cross_validate\n",
        "\n",
        "decision_tree_model = DecisionTreeRegressor(max_depth=5, random_state=0)\n",
        "random_forest_model = RandomForestRegressor(max_depth=5, random_state=0)\n",
        "gradient_boosting_model = GradientBoostingRegressor(max_depth=5, random_state=0)\n",
        "\n",
        "# Fit the models\n",
        "decision_tree_model.fit(X, y)\n",
        "random_forest_model.fit(X, y)\n",
        "gradient_boosting_model.fit(X, y)\n",
        "\n",
        "\n",
        "# Configuartion for K-Fold Cross-Validation\n",
        "n_folds = 5\n",
        "cv_config = {'cv': n_folds, 'scoring': 'neg_mean_squared_error', 'return_train_score': True}\n",
        "\n",
        "# DataFrame to store results\n",
        "results = pd.DataFrame(columns=['Training Score', 'Validation Score'], index=['DT', 'RF', 'GB'])\n",
        "\n",
        "# Perform K-Fold CV and evaluate the models\n",
        "def evaluate_model(model, X, y, cv_config, model_name, pos):\n",
        "    cv_results = cross_validate(model, X, y, **cv_config)\n",
        "    \n",
        "    \n",
        "    if pos:\n",
        "        mean_train_score = np.mean(cv_results['train_score'])\n",
        "        mean_test_score = np.mean(cv_results['test_score'])\n",
        "    else:\n",
        "        mean_train_score = np.mean(-cv_results['train_score'])  # Convert from negative MSE to MSE\n",
        "        mean_test_score = np.mean(-cv_results['test_score'])    # Convert from negative MSE to MSE\n",
        "        \n",
        "    results.loc[model_name] = [mean_train_score, mean_test_score]\n",
        "\n",
        "# Evaluate each model\n",
        "model_names = ['DT', 'RF', 'GB']\n",
        "models = [decision_tree_model, random_forest_model, gradient_boosting_model]\n",
        "\n",
        "for model, name in zip(models, model_names):\n",
        "    evaluate_model(model, X, y, cv_config, name, False)\n",
        "\n",
        "# Print the results DataFrame\n",
        "results"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "31715a9d",
      "metadata": {
        "id": "31715a9d"
      },
      "source": [
        "Repeat the step above to print the R2 score instead of the mean-squared error. For this case, you can use `scoring='r2'`.\n",
        "\n",
        "<font color='red'>\n",
        "Due to the similarity of this to the main part of step 5, this part is 0.5 and the main part of step 5 is 2.5 of the total 3 points for this step.\n",
        "</font>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "83539f47",
      "metadata": {
        "id": "83539f47"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Training Score</th>\n",
              "      <th>Validation Score</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>DT</th>\n",
              "      <td>0.822887</td>\n",
              "      <td>0.17621</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>RF</th>\n",
              "      <td>0.881221</td>\n",
              "      <td>0.173748</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>GB</th>\n",
              "      <td>0.986436</td>\n",
              "      <td>0.473701</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Training Score Validation Score\n",
              "DT       0.822887          0.17621\n",
              "RF       0.881221         0.173748\n",
              "GB       0.986436         0.473701"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE\n",
        "# This would be similar to the main step, the main difference is the scoring.\n",
        "results = pd.DataFrame(columns=['Training Score', 'Validation Score'], index=['DT', 'RF', 'GB'])\n",
        "cv_config = {'cv': n_folds, 'scoring': 'r2', 'return_train_score': True}\n",
        "\n",
        "for model, name in zip(models, model_names):\n",
        "    evaluate_model(model, X, y, cv_config, name, True)\n",
        "\n",
        "# Print the results DataFrame with R^2 scores\n",
        "results"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a5257a98",
      "metadata": {
        "id": "a5257a98"
      },
      "source": [
        "## Questions (6 marks)\n",
        "1. How do these results compare to the results using a linear model in the previous assignment? Use values.\n",
        "1. Out of the models you tested, which model would you select for this dataset and why?\n",
        "1. If you wanted to increase the accuracy of the tree-based models, what would you do? Provide two suggestions."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2PRnpiFjVDzv",
      "metadata": {
        "id": "2PRnpiFjVDzv"
      },
      "source": [
        "<font color='Green'><b>YOUR ANSWERS HERE</b></font>\n",
        "\n",
        "1. Comparing the mean squared error scores for the Decision Tree, Random Forest, and Gradient Booster it is clear that in general, their training scores were exceptionally better than the linear models from the previous assignment (especially the gradient booster). In this case, the Training MSE Scores were 47.9, 32.1, and 3.7 respectively compared to the 111.4 for the linear model. Thus in the best case, the Gradient booster was over 30x better than the linear model, which as described in the previous assignment is considerably good considering the y data is between 30-80, thus in the worst case (for the gradient booster) it only deviated by about 10%. However when we start to look at the MSE for the validation scores it's clear the models don't perform quite as well under the test data, and this is a clear indicator of an overfit model that is trained too closely to the training data. To compare the data (163.1, 156.4, 99.4 vs 95.9 for the linear model), all 3 of the validation scores from this assignment are higher than that of the last assignment, thus indicating major deviation from the expected results. This is a very similar case when we look at the R2 Score metrics. The training scores are very high (0.82, 0.88, and 0.98 vs 0.61) but as soon as the test data is used the validation scores drop to 0.18, 0.17, 0.47 compared to 0.62.\n",
        "\n",
        "\n",
        "2. In general, I definitely choose the Linear Regression model because looking solely at the metrics and reasons listed above it seems as if it performs vastly better at predicting the y-values. It is also important to note that I tested different hyperparameters and in general, found that the linear model still performed better. \n",
        "\n",
        "\n",
        "3. \n",
        "    - Hyperparameter Tuning: Adjust the hyperparameters of the tree-based models to find a better balance between bias and variance. For Decision Trees, you could experiment with max_depth, min_samples_split, and min_samples_leaf to prevent overfitting. For Random Forests, in addition to the Decision Tree parameters, you can adjust n_estimators (the number of trees in the forest) and max_features (the number of features to consider when looking for the best split). For Gradient Boosting, parameters like learning_rate, n_estimators, and subsample can significantly impact performance.\n",
        "    - Feature Engineering: Improving or adding new features can significantly increase the predictive power of your model. This could involve creating interaction terms between features, applying transformations (logarithmic, square root, etc.) to certain features to normalize their distribution, or even generating new features that capture more complexity in the data. Additionally, feature selection methods can be applied to remove irrelevant or redundant features from the dataset, which can help in reducing the complexity of the model and improving its generalization capability.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "37b238f4",
      "metadata": {
        "id": "37b238f4"
      },
      "source": [
        "## Process Description (4 marks)\n",
        "Please describe the process you used to create your code. Cite any websites or generative AI tools used. You can use the following questions as guidance:\n",
        "1. Where did you source your code?\n",
        "1. In what order did you complete the steps?\n",
        "1. If you used generative AI, what prompts did you use? Did you need to modify the code at all? Why or why not?\n",
        "1. Did you have any challenges? If yes, what were they? If not, what helped you to be successful?"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93097bfe",
      "metadata": {
        "id": "93097bfe"
      },
      "source": [
        "<font color='Green'><b>DESCRIBE YOUR PROCESS HERE</b></font>\n",
        "\n",
        "I sourced all of the code from the sklearn (e.g. https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_validate.html), pandas docs, Github Co-pilot and from the last assignment (1). When I used Co-pilot it initially generated code that was incorrect because it was attempting to train and fit the data before the cross-validation function. I was skeptical about this, and thus referenced the docs and found that the cross_validation function already handles all of that for us, thus I removed those unnecessary steps. I also used ChatGPT to help me interpret the results and understand how the models could be improved, and why specifically they weren’t performing as well as I had expected.\n",
        "\n",
        "I completed the steps in order on a basic level and then followed the suggestion of using a loop by going back and creating a reusable function called the evaluate model that allows for the passing of all the configurations.\n",
        "\n",
        "Most of my challenges regarding this section had to do with understanding what cross-validation was, I was successful by going back and reading through the lecture slides from the start of the semester as well as reading through the docs mentioned above to fully understand the purpose of the function."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f7c6de86",
      "metadata": {
        "id": "f7c6de86"
      },
      "source": [
        "# **Part 2: Classification (17.5 marks)**\n",
        "\n",
        "You have been asked to develop code that can help the user classify different wine samples. Following the machine learning workflow described in class, write the relevant code in each of the steps below:"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5f9d33a8",
      "metadata": {
        "id": "5f9d33a8"
      },
      "source": [
        "## **Step 1:** Data Input (2 marks)\n",
        "\n",
        "The data used for this task can be downloaded from UCI: https://archive.ics.uci.edu/dataset/109/wine\n",
        "\n",
        "Use the pandas library to load the dataset. You must define the column headers if they are not included in the dataset\n",
        "\n",
        "You will need to split the dataset into feature matrix `X` and target vector `y`. Which column represents the target vector?\n",
        "\n",
        "Print the size and type of `X` and `y`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "33583c67",
      "metadata": {
        "id": "33583c67"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "(178, 13) <class 'pandas.core.frame.DataFrame'>\n",
            "(178, 1) <class 'pandas.core.frame.DataFrame'>\n"
          ]
        }
      ],
      "source": [
        "# TO DO: Import wine dataset\n",
        "## **Step 1:** Data Input (2 marks)\n",
        "\n",
        "from ucimlrepo import fetch_ucirepo \n",
        "  \n",
        "# fetch dataset \n",
        "wine = fetch_ucirepo(id=109) \n",
        "  \n",
        "# data (as pandas dataframes) \n",
        "X_wine = wine.data.features \n",
        "y_wine = wine.data.targets \n",
        "\n",
        "# X represents the feature matrix and y represents the target vector\n",
        "print(X_wine.shape, type(X_wine))\n",
        "print(y_wine.shape, type(y_wine))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "156db208",
      "metadata": {
        "id": "156db208"
      },
      "source": [
        "## **Step 2:** Data Processing (1.5 marks)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a28af110",
      "metadata": {
        "id": "a28af110"
      },
      "source": [
        "Print the first five rows of the dataset to inspect:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "ea266921",
      "metadata": {
        "id": "ea266921"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Alcohol</th>\n",
              "      <th>Malicacid</th>\n",
              "      <th>Ash</th>\n",
              "      <th>Alcalinity_of_ash</th>\n",
              "      <th>Magnesium</th>\n",
              "      <th>Total_phenols</th>\n",
              "      <th>Flavanoids</th>\n",
              "      <th>Nonflavanoid_phenols</th>\n",
              "      <th>Proanthocyanins</th>\n",
              "      <th>Color_intensity</th>\n",
              "      <th>Hue</th>\n",
              "      <th>0D280_0D315_of_diluted_wines</th>\n",
              "      <th>Proline</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>14.23</td>\n",
              "      <td>1.71</td>\n",
              "      <td>2.43</td>\n",
              "      <td>15.6</td>\n",
              "      <td>127</td>\n",
              "      <td>2.80</td>\n",
              "      <td>3.06</td>\n",
              "      <td>0.28</td>\n",
              "      <td>2.29</td>\n",
              "      <td>5.64</td>\n",
              "      <td>1.04</td>\n",
              "      <td>3.92</td>\n",
              "      <td>1065</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>13.20</td>\n",
              "      <td>1.78</td>\n",
              "      <td>2.14</td>\n",
              "      <td>11.2</td>\n",
              "      <td>100</td>\n",
              "      <td>2.65</td>\n",
              "      <td>2.76</td>\n",
              "      <td>0.26</td>\n",
              "      <td>1.28</td>\n",
              "      <td>4.38</td>\n",
              "      <td>1.05</td>\n",
              "      <td>3.40</td>\n",
              "      <td>1050</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>13.16</td>\n",
              "      <td>2.36</td>\n",
              "      <td>2.67</td>\n",
              "      <td>18.6</td>\n",
              "      <td>101</td>\n",
              "      <td>2.80</td>\n",
              "      <td>3.24</td>\n",
              "      <td>0.30</td>\n",
              "      <td>2.81</td>\n",
              "      <td>5.68</td>\n",
              "      <td>1.03</td>\n",
              "      <td>3.17</td>\n",
              "      <td>1185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>14.37</td>\n",
              "      <td>1.95</td>\n",
              "      <td>2.50</td>\n",
              "      <td>16.8</td>\n",
              "      <td>113</td>\n",
              "      <td>3.85</td>\n",
              "      <td>3.49</td>\n",
              "      <td>0.24</td>\n",
              "      <td>2.18</td>\n",
              "      <td>7.80</td>\n",
              "      <td>0.86</td>\n",
              "      <td>3.45</td>\n",
              "      <td>1480</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>13.24</td>\n",
              "      <td>2.59</td>\n",
              "      <td>2.87</td>\n",
              "      <td>21.0</td>\n",
              "      <td>118</td>\n",
              "      <td>2.80</td>\n",
              "      <td>2.69</td>\n",
              "      <td>0.39</td>\n",
              "      <td>1.82</td>\n",
              "      <td>4.32</td>\n",
              "      <td>1.04</td>\n",
              "      <td>2.93</td>\n",
              "      <td>735</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Alcohol  Malicacid   Ash  Alcalinity_of_ash  Magnesium  Total_phenols  \\\n",
              "0    14.23       1.71  2.43               15.6        127           2.80   \n",
              "1    13.20       1.78  2.14               11.2        100           2.65   \n",
              "2    13.16       2.36  2.67               18.6        101           2.80   \n",
              "3    14.37       1.95  2.50               16.8        113           3.85   \n",
              "4    13.24       2.59  2.87               21.0        118           2.80   \n",
              "\n",
              "   Flavanoids  Nonflavanoid_phenols  Proanthocyanins  Color_intensity   Hue  \\\n",
              "0        3.06                  0.28             2.29             5.64  1.04   \n",
              "1        2.76                  0.26             1.28             4.38  1.05   \n",
              "2        3.24                  0.30             2.81             5.68  1.03   \n",
              "3        3.49                  0.24             2.18             7.80  0.86   \n",
              "4        2.69                  0.39             1.82             4.32  1.04   \n",
              "\n",
              "   0D280_0D315_of_diluted_wines  Proline  \n",
              "0                          3.92     1065  \n",
              "1                          3.40     1050  \n",
              "2                          3.17     1185  \n",
              "3                          3.45     1480  \n",
              "4                          2.93      735  "
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE\n",
        "X_wine.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "c6892a92",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   class\n",
              "0      1\n",
              "1      1\n",
              "2      1\n",
              "3      1\n",
              "4      1"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y_wine.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "834fc8fe",
      "metadata": {
        "id": "834fc8fe"
      },
      "source": [
        "Check to see if there are any missing values in the dataset. If necessary, select an appropriate method to fill-in the missing values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "97c6e9dc",
      "metadata": {
        "id": "97c6e9dc"
      },
      "outputs": [],
      "source": [
        "# TO DO: ADD YOUR CODE HERE\n",
        "if (y_wine.isnull().values.any() == True):\n",
        "    print(\"There are missing values in the target vector, fill them with the mean value\")\n",
        "    y_wine.fillna(y.mean(), inplace=True)\n",
        "    \n",
        "if (X_wine.isnull().values.any() == True):\n",
        "    print(\"There are missing values in the feature matrix, fill them with the mean value\")\n",
        "    X_wine.fillna(X.mean(), inplace=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "070956af",
      "metadata": {
        "id": "070956af"
      },
      "source": [
        "How many samples do we have of each type of wine?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "b37a6fd9",
      "metadata": {
        "id": "b37a6fd9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "class\n",
            "2        71\n",
            "1        59\n",
            "3        48\n",
            "dtype: int64\n"
          ]
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE\n",
        "print(y_wine.value_counts())"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "70e6c46f",
      "metadata": {
        "id": "70e6c46f"
      },
      "source": [
        "## **Step 3:** Implement Machine Learning Model\n",
        "\n",
        "1. Import `SVC` and `DecisionTreeClassifier` from sklearn\n",
        "2. Instantiate models as `SVC()` and `DecisionTreeClassifier(max_depth = 3)`\n",
        "3. Implement the machine learning model with `X` and `y`"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0870b0d2",
      "metadata": {
        "id": "0870b0d2"
      },
      "source": [
        "## **Step 4:** Validate Model\n",
        "\n",
        "Calculate the average training and validation accuracy using `cross_validate` for the two different models listed in Step 3. For this case, use `scoring='accuracy'`"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bb0bbd83",
      "metadata": {
        "id": "bb0bbd83"
      },
      "source": [
        "## **Step 5:** Visualize Results (4 marks)\n",
        "\n",
        "<font color='red'>\n",
        "There is no individual mark for Steps 3 and 4 and those grades are included within the four points.\n",
        "\n",
        "</font>\n",
        "\n",
        "### **Step 5.1:** Compare Models (2 out of total 4 marks)\n",
        "1. Create a pandas DataFrame `results` with columns: Training accuracy and Validation accuracy\n",
        "2. Add the data size, training and validation accuracy for each dataset to the `results` DataFrame\n",
        "3. Print `results`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "id": "be4b5c0a",
      "metadata": {
        "id": "be4b5c0a"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "    Training Score Validation Score\n",
            "SVC       0.703743         0.663492\n",
            "DT        0.974756         0.893175\n"
          ]
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE FOR STEPS 3-5\n",
        "# Note: for any random state parameters, you can use random_state = 0\n",
        "# HINT: USING A LOOP TO STORE THE DATA IN YOUR RESULTS DATAFRAME WILL BE MORE EFFICIENT\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as  plt\n",
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "\n",
        "svc_model = SVC(random_state=0)\n",
        "decision_tree_model = DecisionTreeClassifier(max_depth=3, random_state=0)\n",
        "\n",
        "svc_model.fit(X_wine, y_wine)\n",
        "decision_tree_model.fit(X_wine, y_wine)\n",
        "\n",
        "\n",
        "results = pd.DataFrame(columns=['Training Score', 'Validation Score'],  index=['SVC', 'DT'])\n",
        "cv_config = { 'scoring': 'accuracy', 'return_train_score': True}\n",
        "models = [svc_model, decision_tree_model]\n",
        "model_names = ['SVC', 'DT']\n",
        "\n",
        "\n",
        "for model, name in zip(models, model_names):\n",
        "    evaluate_model(model, X_wine, y_wine, cv_config, name, True)\n",
        "\n",
        "# Print the results DataFrame with R^2 scores\n",
        "print(results)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f2e17878",
      "metadata": {
        "id": "f2e17878"
      },
      "source": [
        "### **Step 5.2:** Visualize Classification Errors  (2 out of total 4 marks)\n",
        "Which method gave the highest accuracy? Use this method to print the confusion matrix and classification report:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "id": "44b091a4",
      "metadata": {
        "id": "44b091a4"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style>#sk-container-id-1 {\n",
              "  /* Definition of color scheme common for light and dark mode */\n",
              "  --sklearn-color-text: black;\n",
              "  --sklearn-color-line: gray;\n",
              "  /* Definition of color scheme for unfitted estimators */\n",
              "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
              "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
              "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
              "  --sklearn-color-unfitted-level-3: chocolate;\n",
              "  /* Definition of color scheme for fitted estimators */\n",
              "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
              "  --sklearn-color-fitted-level-1: #d4ebff;\n",
              "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
              "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
              "\n",
              "  /* Specific color for light theme */\n",
              "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
              "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
              "  --sklearn-color-icon: #696969;\n",
              "\n",
              "  @media (prefers-color-scheme: dark) {\n",
              "    /* Redefinition of color scheme for dark theme */\n",
              "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
              "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
              "    --sklearn-color-icon: #878787;\n",
              "  }\n",
              "}\n",
              "\n",
              "#sk-container-id-1 {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 pre {\n",
              "  padding: 0;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 input.sk-hidden--visually {\n",
              "  border: 0;\n",
              "  clip: rect(1px 1px 1px 1px);\n",
              "  clip: rect(1px, 1px, 1px, 1px);\n",
              "  height: 1px;\n",
              "  margin: -1px;\n",
              "  overflow: hidden;\n",
              "  padding: 0;\n",
              "  position: absolute;\n",
              "  width: 1px;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-dashed-wrapped {\n",
              "  border: 1px dashed var(--sklearn-color-line);\n",
              "  margin: 0 0.4em 0.5em 0.4em;\n",
              "  box-sizing: border-box;\n",
              "  padding-bottom: 0.4em;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-container {\n",
              "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
              "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
              "     so we also need the `!important` here to be able to override the\n",
              "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
              "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
              "  display: inline-block !important;\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-text-repr-fallback {\n",
              "  display: none;\n",
              "}\n",
              "\n",
              "div.sk-parallel-item,\n",
              "div.sk-serial,\n",
              "div.sk-item {\n",
              "  /* draw centered vertical line to link estimators */\n",
              "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
              "  background-size: 2px 100%;\n",
              "  background-repeat: no-repeat;\n",
              "  background-position: center center;\n",
              "}\n",
              "\n",
              "/* Parallel-specific style estimator block */\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel-item::after {\n",
              "  content: \"\";\n",
              "  width: 100%;\n",
              "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
              "  flex-grow: 1;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel {\n",
              "  display: flex;\n",
              "  align-items: stretch;\n",
              "  justify-content: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  position: relative;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel-item {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
              "  align-self: flex-end;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
              "  align-self: flex-start;\n",
              "  width: 50%;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
              "  width: 0;\n",
              "}\n",
              "\n",
              "/* Serial-specific style estimator block */\n",
              "\n",
              "#sk-container-id-1 div.sk-serial {\n",
              "  display: flex;\n",
              "  flex-direction: column;\n",
              "  align-items: center;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  padding-right: 1em;\n",
              "  padding-left: 1em;\n",
              "}\n",
              "\n",
              "\n",
              "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
              "clickable and can be expanded/collapsed.\n",
              "- Pipeline and ColumnTransformer use this feature and define the default style\n",
              "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
              "*/\n",
              "\n",
              "/* Pipeline and ColumnTransformer style (default) */\n",
              "\n",
              "#sk-container-id-1 div.sk-toggleable {\n",
              "  /* Default theme specific background. It is overwritten whether we have a\n",
              "  specific estimator or a Pipeline/ColumnTransformer */\n",
              "  background-color: var(--sklearn-color-background);\n",
              "}\n",
              "\n",
              "/* Toggleable label */\n",
              "#sk-container-id-1 label.sk-toggleable__label {\n",
              "  cursor: pointer;\n",
              "  display: block;\n",
              "  width: 100%;\n",
              "  margin-bottom: 0;\n",
              "  padding: 0.5em;\n",
              "  box-sizing: border-box;\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
              "  /* Arrow on the left of the label */\n",
              "  content: \"▸\";\n",
              "  float: left;\n",
              "  margin-right: 0.25em;\n",
              "  color: var(--sklearn-color-icon);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
              "  color: var(--sklearn-color-text);\n",
              "}\n",
              "\n",
              "/* Toggleable content - dropdown */\n",
              "\n",
              "#sk-container-id-1 div.sk-toggleable__content {\n",
              "  max-height: 0;\n",
              "  max-width: 0;\n",
              "  overflow: hidden;\n",
              "  text-align: left;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-toggleable__content pre {\n",
              "  margin: 0.2em;\n",
              "  border-radius: 0.25em;\n",
              "  color: var(--sklearn-color-text);\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
              "  /* Expand drop-down */\n",
              "  max-height: 200px;\n",
              "  max-width: 100%;\n",
              "  overflow: auto;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
              "  content: \"▾\";\n",
              "}\n",
              "\n",
              "/* Pipeline/ColumnTransformer-specific style */\n",
              "\n",
              "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator-specific style */\n",
              "\n",
              "/* Colorize estimator box */\n",
              "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
              "#sk-container-id-1 div.sk-label label {\n",
              "  /* The background is the default theme color */\n",
              "  color: var(--sklearn-color-text-on-default-background);\n",
              "}\n",
              "\n",
              "/* On hover, darken the color of the background */\n",
              "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "/* Label box, darken color on hover, fitted */\n",
              "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
              "  color: var(--sklearn-color-text);\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Estimator label */\n",
              "\n",
              "#sk-container-id-1 div.sk-label label {\n",
              "  font-family: monospace;\n",
              "  font-weight: bold;\n",
              "  display: inline-block;\n",
              "  line-height: 1.2em;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-label-container {\n",
              "  text-align: center;\n",
              "}\n",
              "\n",
              "/* Estimator-specific */\n",
              "#sk-container-id-1 div.sk-estimator {\n",
              "  font-family: monospace;\n",
              "  border: 1px dotted var(--sklearn-color-border-box);\n",
              "  border-radius: 0.25em;\n",
              "  box-sizing: border-box;\n",
              "  margin-bottom: 0.5em;\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-0);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-estimator.fitted {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-0);\n",
              "}\n",
              "\n",
              "/* on hover */\n",
              "#sk-container-id-1 div.sk-estimator:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-2);\n",
              "}\n",
              "\n",
              "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-2);\n",
              "}\n",
              "\n",
              "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
              "\n",
              "/* Common style for \"i\" and \"?\" */\n",
              "\n",
              ".sk-estimator-doc-link,\n",
              "a:link.sk-estimator-doc-link,\n",
              "a:visited.sk-estimator-doc-link {\n",
              "  float: right;\n",
              "  font-size: smaller;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1em;\n",
              "  height: 1em;\n",
              "  width: 1em;\n",
              "  text-decoration: none !important;\n",
              "  margin-left: 1ex;\n",
              "  /* unfitted */\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted,\n",
              "a:link.sk-estimator-doc-link.fitted,\n",
              "a:visited.sk-estimator-doc-link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
              ".sk-estimator-doc-link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover,\n",
              "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
              ".sk-estimator-doc-link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "/* Span, style for the box shown on hovering the info icon */\n",
              ".sk-estimator-doc-link span {\n",
              "  display: none;\n",
              "  z-index: 9999;\n",
              "  position: relative;\n",
              "  font-weight: normal;\n",
              "  right: .2ex;\n",
              "  padding: .5ex;\n",
              "  margin: .5ex;\n",
              "  width: min-content;\n",
              "  min-width: 20ex;\n",
              "  max-width: 50ex;\n",
              "  color: var(--sklearn-color-text);\n",
              "  box-shadow: 2pt 2pt 4pt #999;\n",
              "  /* unfitted */\n",
              "  background: var(--sklearn-color-unfitted-level-0);\n",
              "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link.fitted span {\n",
              "  /* fitted */\n",
              "  background: var(--sklearn-color-fitted-level-0);\n",
              "  border: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "\n",
              ".sk-estimator-doc-link:hover span {\n",
              "  display: block;\n",
              "}\n",
              "\n",
              "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
              "\n",
              "#sk-container-id-1 a.estimator_doc_link {\n",
              "  float: right;\n",
              "  font-size: 1rem;\n",
              "  line-height: 1em;\n",
              "  font-family: monospace;\n",
              "  background-color: var(--sklearn-color-background);\n",
              "  border-radius: 1rem;\n",
              "  height: 1rem;\n",
              "  width: 1rem;\n",
              "  text-decoration: none;\n",
              "  /* unfitted */\n",
              "  color: var(--sklearn-color-unfitted-level-1);\n",
              "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
              "  /* fitted */\n",
              "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
              "  color: var(--sklearn-color-fitted-level-1);\n",
              "}\n",
              "\n",
              "/* On hover */\n",
              "#sk-container-id-1 a.estimator_doc_link:hover {\n",
              "  /* unfitted */\n",
              "  background-color: var(--sklearn-color-unfitted-level-3);\n",
              "  color: var(--sklearn-color-background);\n",
              "  text-decoration: none;\n",
              "}\n",
              "\n",
              "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
              "  /* fitted */\n",
              "  background-color: var(--sklearn-color-fitted-level-3);\n",
              "}\n",
              "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeClassifier(max_depth=3, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;DecisionTreeClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.4/modules/generated/sklearn.tree.DecisionTreeClassifier.html\">?<span>Documentation for DecisionTreeClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeClassifier(max_depth=3, random_state=0)</pre></div> </div></div></div></div>"
            ],
            "text/plain": [
              "DecisionTreeClassifier(max_depth=3, random_state=0)"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# TO DO: Implement best model\n",
        "decision_tree_model = DecisionTreeClassifier(max_depth=3, random_state=0)\n",
        "X_wine_train, X_wine_test, y_wine_train, y_wine_test = train_test_split(X_wine, y_wine, random_state=0)\n",
        "decision_tree_model.fit(X_wine_train, y_wine_train)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "id": "09d21b59",
      "metadata": {
        "id": "09d21b59"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAooAAAIhCAYAAADJr5uhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABYDElEQVR4nO3deVxUdfv/8fegAioiKribBiXuiBsumEuaqS1qi2mKZlq5UUIuaO77vpGmqWVp7paVrXqXWbll7juQ+wYpKMoiMr8//DnfRo4JyjhD83rej/O4m898OOdiZoSL67Mck9lsNgsAAAC4g4u9AwAAAIBjIlEEAACAIRJFAAAAGCJRBAAAgCESRQAAABgiUQQAAIAhEkUAAAAYIlEEAACAIRJF4D+AffMBALZAoogs2bdvn/r376/GjRurWrVqatasmYYOHapTp07Z7Joff/yxGjRooGrVqmnOnDnZcs5t27bJ399f27Zty5bzZeZa/v7++vXXXw37REdHW/qcPn060+dOTU3VuHHj9NVXX92zr7+/v2bPnp3pcxtZv369mjRpoipVqmjYsGEPdC4jt1+D20elSpUUFBSkbt266aeffsr260nSoEGD1LRpU5v1v1+dO3fO8HrceQwaNMjmcfybv/76SyNGjFCzZs1UrVo1NW7cWGFhYTp8+LBVv6ZNmz70WNeuXWv17yktLU2DBg1SYGCgatSooa1bt2bLvwngvy63vQNAzrF06VKNGzdOQUFBCg8PV9GiRXXixAktXLhQP/zwgxYvXqwKFSpk6zUTExM1ceJENW7cWN26dVPp0qWz5byVK1fWihUr9Nhjj2XL+TLDxcVF3333nYKDgzM8980339zXOS9evKjFixdr/Pjx9+y7YsUKFS9e/L6uc9uoUaNUrlw5TZgwQcWKFXugc93Niy++qJdeekmSdOPGDcXGxmrNmjV66623NGTIEIWEhGTr9Xr16pWlc2a1//0aPny4EhMTLY9Hjhxpab+tcOHCNo/jbn744QcNGDBAjz/+uHr27KnSpUvr/PnzWrx4sV5++WXNnTtXDRo0sFt8jRs31ooVK1S0aFFJ0ubNm/X555+rV69eql+/vipVqpQt/yaA/zoSRWTKzp07NXbsWL366qsaMmSIpT0oKEjNmjVTmzZtNHjwYK1duzZbr5uQkKD09HQ1a9ZMtWvXzrbzenh4qHr16tl2vsyoUaOGfvzxR40YMUK5c1v/0/vmm29UsWJFHTp0yGbXz47vNz4+Xg0aNFBQUNCDB3QXxYsXzxBrq1at1LdvX02aNElNmzbNtj8YJOmRRx6xaf/7decfMR4eHpKy5318UCdPntTAgQPVsGFDzZgxQ7ly5bI899RTT6lDhw4aOHCg/ve//8nV1dUuMRYuXNgqkY6Pj5cktWvXTmXKlJHkGK8l4OgYekamLFy4UAUKFFBYWFiG5woXLqxBgwbpySef1PXr1yVJN2/e1NKlS/Xss89ahqSmTJmilJQUy9cNGjRIXbt21Zo1a9SiRQtVqVJFzz//vH755RdJt4aObg/xDR48WP7+/pKMh7HuHGZKTk7WiBEj9MQTT6hKlSp6+umntXDhQkt/o6Hnffv26fXXX1dQUJBq1Kiht956S8eOHcvwNVu2bFG3bt0UEBCgBg0aaPLkybp58+Y9X8NWrVopPj5eW7dutWo/fPiwjh8/rpYtW2b4mg0bNqhjx44KDAy0fB9Lly6VJJ0+fVpPPvmkJCkiIsLyWg0aNEhdunTR8OHDVaNGDbVq1Uo3b960Gmbr06ePqlatqpiYGMu1Zs+erYoVK2r79u0Z4rj9vUvS+++/b/Va//bbb+rYsaNq1qxpqTafO3fO6r2pVKmSVq1apQYNGqhOnTqKioq65+t1p379+unGjRtavXq1pS0lJUWTJk1So0aNVKVKFT377LMZqrNms1kff/yxWrZsqWrVqql58+ZauHChZV7nnUPJ+/fvV5cuXVSzZk0FBgaqa9eu2r17t+X5O/tnx2f9QcyePVvNmzdXZGSk6tSpo+DgYCUkJEiSVq1apdatW6tKlSpq3LixZs+eneGz+scff6hTp04KCAhQnTp1NHDgQF26dOlfr/npp58qNTVV7733nlWSKEl58+bVwIED9cILL1jiuNPp06c1YMAABQcHq3LlyqpXr54GDBigy5cvW/rc6324dOmSwsPD1aBBA1WtWlXPP/+8vvjiC8vz//yZMGjQIMvPjGbNmqlz586SMk7HiI+P17Bhw1S/fn1VrVpVL7/8srZs2WIVu7+/vyIjI9WuXTtVq1ZNkZGR//paATkdFUXck9ls1q+//qqmTZsqb968hn1atWpl9XjYsGFat26devTooVq1aungwYN6//33dejQIS1YsEAmk0nSrV8GFy9eVGhoqDw8PDRz5kz17dtXv/zyixo3bqzIyEj16dNHPXv2VOPGjTMd87hx4/Trr79q4MCB8vb21i+//KJJkybJy8tLL7zwQob+W7duVffu3RUUFKRx48YpJSVF8+bN0yuvvKKVK1fKz8/P0vfdd99Vx44d1aNHD/38889asGCBypQpo1deeeVfY3rsscf0+OOPZxh+Xr9+verUqSMfHx+r/j///LN69+6tkJAQ9e3bV8nJyfrss880atQoValSRRUrVrR6fZ566inL1/7xxx9yc3PT+++/r+vXr2f4ZT5ixAi1bt1aw4cP16effqr9+/frgw8+ULdu3VSnTp0Msd8eqm/fvr1laLho0aL64osvNHDgQD3zzDN68803dfnyZc2aNUvt27fX559/riJFiki6lUwtWrRIY8eO1eXLl61ez8zy9fVVyZIltXPnTkm3Ppe9e/fWn3/+qdDQUPn5+enHH39Uv379lJqaqjZt2kiSJk2apMWLF+u1115TgwYNtG/fPk2ZMkVpaWl68803ra6RmJio7t27q27dupo9e7ZSU1M1d+5cvf766/r5559VoECBDHFlx2e9YMGCWX49/uns2bPatGmTpk+frvj4eBUsWFDz5s3T9OnT1alTJ0VEROjQoUOaPXu2zp07p3HjxkmSduzYoddee01169bVjBkzlJCQoJkzZyokJESrV6+Wu7u74fU2b96sSpUq3XX6Qb169VSvXj3D55KSkhQSEqJChQpp+PDhKlCggHbt2qXIyEi5u7tr1KhRmXof+vfvr7///lsjR46Uh4eH1q1bp4EDB6p48eKqW7eu1TV79eql4sWLa+7cuYqMjNSjjz6aIa6UlBR16dJFcXFx6tevn4oWLao1a9aoe/fuWrBggdX388EHHyg8PFyPPvqoSpUqlan3CMipSBRxT5cvX1ZKSkqmh/uioqK0evVqhYeH64033pAkNWjQQEWLFtWAAQP0yy+/qFGjRpKkq1evau3atZbhvHz58qlTp07aunWrWrRooYoVK0q6NdyXlWGi7du3q0GDBmrdurWkW0Pk+fLlsyQud5o6darKli2r+fPnW5Kq4OBgNW/eXLNmzdLMmTMtfV966SX17t1b0q1fiBs2bNDPP/98z0RRklq2bKlPPvnEavj5m2++0VtvvZWhb1RUlNq2bWs11B8YGKigoCBt27ZNAQEBVq9PpUqVLP3S0tI0atSou86/8vb21vDhw9WvXz+tWrVKixcvVvny5fX2228b9v/nUP3toeH09HRNmTJFwcHBmjp1qqXv7SrmwoULNWDAAEv7W2+9laVk/25xx8XFSZJ+//13bd68WdOnT7f8odKwYUMlJSVpypQpeuaZZ3T9+nV98skn6tSpk/r37y9Jql+/vmJjY7Vjx44MiWJUVJQuX76skJAQ1ahRQ9KtBHXFihW6du1ahkQxOz/rDyItLU0DBw5UrVq1LNeaM2eO2rdvr/fee0/Src+zl5eX3nvvPb322mt6/PHHNXXqVD366KOaN2+e5XMfEBCg1q1ba82aNXr11VcNr3f+/HnLZy+rjh8/ruLFi2vixImWIeC6detqz549lmp2Zt6H7du3q3fv3mrWrJkkqU6dOvLy8jIc6n7kkUcsr3vFihUNf5atW7dOhw8f1sqVKxUQECBJeuKJJ9S5c2dNmTJFa9assfStVauWXnvttfv6/oGchqFn3NPtXyCZGV6VZPlhfztJu61169bKlSuX1XBv4cKFreZ83U5skpKSHijmoKAgrVy5Uj169NCSJUt06tQp9e7d2zBRuX79uvbt26eWLVtaVd48PT3VpEmTDEOxgYGBVo+LFy9uGXK/lzuHn/fs2aMLFy5YVQNv6969uyZMmKBr165p//79+uabbzRv3jxJt1Y7/xsvL697TtJv1aqVWrRooWHDhunUqVOaMmVKluaT/fXXX4qNjdUzzzxj1f7II48oMDAww+t2v4nFP5nNZkuFbsuWLTKZTGrUqJHS0tIsR9OmTRUbG6tjx45p9+7dSktLy/D6vvfee1qwYEGG8z/++OMqXLiw3nrrLQ0bNkw//vijvL291b9/f8PX0xE+67f98/XdtWuXkpOT1bRp0wyvjXRrukBSUpL27NmjRo0ayWw2W/qUKVNGfn5++u233+56rVy5cmX654FRnJ999plKlSql48ePa9OmTVq4cKFiYmIsn+vMvA9BQUGaPXu2QkNDtWrVKsXFxWngwIGWxDKrtmzZIh8fH1WuXNnyWty8eVNNmjTR/v37rYbRs+OzDOQUVBRxTwULFlT+/Pl19uzZu/a5fv26bty4oYIFC1p+oN45lJo7d24VKlRIV69etbTdOZR9OwlIT09/oJiHDBmi4sWL68svv9To0aM1evRoBQYGasSIERlWZl+9elVms1ne3t4ZzuPt7W0Vr6QMw3EuLi6Z3sfw0UcfVcWKFS3Dz998842Cg4MNhx4vXbqk4cOHa8OGDTKZTCpbtqylYnSv6+XPnz9T8bRt21bff/+9ypUrZzgc929uLw642+t28OBBq7Z8+fJl6fxGzp8/r/Lly1uubzab75oYXLx40fJZzOzq4Pz582vp0qWaO3euvv32W61YsULu7u56/vnn9d5772VIpB3hs/7P2G+7/d7crnLe6eLFi7py5YrS09P14Ycf6sMPP8zQx83N7a7XKlmy5L/+PLhx44YSEhIMPxuS9NFHH+mDDz5QfHy8vL29VaVKFeXNm9fyemXmfZg+fbo++OADffvtt/r+++/l4uKi+vXra9SoUfc1HBwfH6/Y2FhVrlzZ8PnY2FjLv9Ps+CwDOQWJIjIlODhY27ZtU0pKiuEvkJUrV2rixIlavXq15YdpbGys1Q/sGzdu6PLlyypUqNADx3NnNePOip6rq6t69uypnj176uzZs/rpp580Z84chYeHa/369VZ9CxQoIJPJZBnS/KfY2Fh5eXk9cLz/dHtYdvjw4fruu+/07rvvGvZ79913FRMTo48//liBgYFydXVVUlKSVq5cmS1xJCUlafz48SpfvryOHj2qRYsWqXv37pn++tuvy91et+x4n/8pKipKsbGxluHQAgUKKF++fPrkk08M+5ctW1Z//vmnpFtJt6+vr+W5s2fP6uTJk6pZs2aGr/P19bUsUNq7d6/WrVunZcuW6ZFHHsnw+jyMz/r98PT0lCRNmTJF5cqVy/C8t7e38ufPL5PJpK5du2aoiEoZE9t/Cg4O1uLFixUbG5shSZakTZs2qXfv3oqMjFTz5s2tnvvqq680YcIE9e/fX+3atbMk8W+//bb27dtn6Xev9+H2PMX+/fsrJiZGGzdu1Jw5czRy5EjNnz8/U6/TPxUoUEDlypXTlClTDJ/PzpX2QE7C0DMypVu3boqPj9eMGTMyPBcbG6tFixbpscceU+XKlS2LIe5MyNavX6+bN28a/nLOCg8PD50/f96q7fYCB+nWiucWLVpo0aJFkm5VP1599VW1bt3asAqSL18+ValSRd9++61VAnr16lX9/PPPDxzvnVq2bKn4+Hh98MEHSkhIsKxcvtPOnTv11FNPKSgoyFLJur1K9nYV6s5FKlkxdepUnT9/XrNnz1anTp00a9YsRUdHZ/rrH330Ufn4+Ojrr7+2aj916pR2795930OAdzNr1iy5u7urbdu2km7NSbt+/brMZrOqVq1qOY4ePar3339faWlpqlatmvLkyZNhs+5FixYpLCwsw+v33XffqW7duoqNjVWuXLksVWhPT0/Dz46tP+v3KyAgQHny5NGFCxesXpvcuXNr2rRpOn36tDw8PFSpUiXFxMRY9Xn88cc1e/bsf92M/tVXX1WePHk0duxYwz/aZs2apUKFCumJJ57I8LU7d+6Up6enunfvbkkSr127pp07d1o+1/d6H86cOaNGjRrpu+++k3QrqezRo4fq16//r5XOf1OnTh2dO3dORYoUsXo9fvvtNy1YsOCB/q0BORkVRWRK9erV9fbbb2vGjBmKjo5WmzZtVKhQIR07dkwLFy5USkqKJYl87LHH1LZtW82aNUtJSUmqXbu2Dh06pMjISAUFBalhw4YPFEuTJk00b948zZs3TwEBAfrf//5nteWMu7u7KleurMjISOXJk0f+/v7666+/9Pnnn9910UB4eLhef/11vfHGG+rYsaNu3Lih+fPnKzU11bJwJbuUKVNGVatW1bx589S8efO7DmNVq1ZNX331lSpXrqzixYvrzz//1Pz582UymSzz2m4vrtiyZYv8/Pwsk/DvZfv27VqyZIn69euncuXK6Z133tGPP/6oQYMGafny5Zn6peji4qKwsDBFREQoPDxczz33nC5fvqzIyEgVLFjwvif7nz9/3rINSlpami5cuKDPP/9cv/76q9UCnUaNGql27drq1auXevXqJT8/P+3du1ezZs1Sw4YNLUlISEiIPv74Y7m6uqpOnTras2ePli1bpgEDBsjFxfpv5Ro1aig9PV29e/fWG2+8ofz58+vbb7/V1atXDeeR2vqzfr8KFSqk7t27a+bMmUpMTFRQUJAuXLigmTNnymQyWaZfhIWF6Y033rC8f7dXp+/Zs0e9evW66/lLly6tESNGaMiQIXr11Vf1yiuvqESJEjp58qQ++ugjnTp1SgsXLjQcfahWrZqWLVumCRMmqEmTJrp48aIWLlyouLg4S4X2Xu9DqVKlVLx4cY0ZM0aJiYl65JFHtH//fm3atCnDAqXMateunZYsWaLXXntNb731lkqUKKHff/9dH374oTp16qQ8efLc13mBnI5EEZnWs2dPVapUyXKHloSEBJUoUUKNGze2/GC9bezYsSpbtqzWrFmjDz/8UEWLFlVISIh69eqV4ZdzVr355pu6dOmSFi5cqBs3bqhx48YaO3asevbsaekzatQozZgxQ4sWLVJsbKyKFCmiF1988a6reuvVq6ePPvpIs2bNUlhYmFxdXVWrVi1NnDhRjz/++APFa6RVq1bat2+f4ZDfbRMmTLDMr5SkcuXKaeTIkfryyy/1xx9/SLpVXX3ttde0YsUKbdq06V8XINx2/fp1RUREqHz58nr99dcl3ZoTNmzYMPXs2VMLFizI9C/bdu3aKX/+/Jo3b5569+4tDw8PNWzYUGFhYYZDkpmxevVqy16JLi4u8vLyUkBAgD766COrLUpcXFw0f/58zZw5U/PmzdPff/+tYsWK6bXXXrNK7vv3768iRYpo+fLlWrBggUqXLq2hQ4carlIvWrSoFixYoJkzZ2rIkCFKSkqyVNju3HLlNlt+1h/EO++8Ix8fH3322WdasGCBChYsqHr16iksLMzyB0ZwcLAWLlyoyMhIhYaGKk+ePKpcubI++uije+4y0LZtW5UtW1aLFy/WjBkz9Pfff8vHx0c1atTQ7Nmz77oFUtu2bXX69GmtWbNGn332mYoVK6ZGjRqpY8eOGjp0qKKjo+Xn53fP9yEyMlLTpk3TzJkzdfnyZZUoUUJ9+vS567zMe8mXL5+WLl2qqVOnavLkybp69apKlSql8PBwdevW7b7OCfwXmMyZnYUPAAAAp8IcRQAAABgiUQQAAIAhEkUAAAAHcuHCBYWGhqpOnTpq2LChxo8fb7l//KlTp9S1a1dVr15drVq10q+//vqv5/r666/VrFkzBQQEqHfv3ve8l/udSBQBAAAchNlsVmhoqJKSkrR06VJNnz5dP/30k2bMmGG5x723t7fWrFmj559/Xn369LnrtlB79+7VkCFD1KdPH61YsUJXrlxRREREluJh1TMAAICDiImJ0e7du/Xbb79Z7m4UGhqqiRMn6oknntCpU6e0fPly5cuXT35+ftqyZYvWrFmjvn37ZjjXkiVL1LJlS7Vp00aSNGnSJDVp0kSnTp2y3Gv9XqgoAgAAOAgfHx8tWLAgwy0wExMTtWfPHlWqVMlq/92aNWta9p690549eyy3fpWkEiVKqGTJktqzZ0+m4yFRBAAAcBCenp5Wm/Wnp6dryZIllrsVFS1a1Kp/kSJFMtyt7LaLFy9mqb+R/+TQc2LYc/YOAcigxqf3d2sxwFZiEs7ZOwTASlrqGbtd+0ZcjM3Oncfb996d7mLy5Mk6ePCgVq9ebbnL1D+5uroqNTXV8GuTk5Oz1N8IFUUAAAAHNHnyZC1evFiTJ09W+fLl5ebmliHJS01Nlbu7u+HX361/3rx5Mx3Df7KiCAAAkCXpN+0dgZXRo0dr2bJlmjx5slq0aCFJKlasmKKioqz6xcXFZRhevq1YsWKKi4vL0D8rt1iloggAAGBOt92RRZGRkVq+fLmmTZum1q1bW9oDAgJ04MABJScnW9p27typgIAAw/MEBARo586dlsfnzp3TuXPn7trfCIkiAACAg4iOjtacOXPUo0cP1axZU7GxsZajTp06KlGihCIiInTs2DHNnz9fe/fu1Ysvvijp1rBybGysbt68VR3t0KGD1q1bp1WrVunw4cMaMGCAGjdunOmtcSQSRQAAACk93XZHFmzcuFE3b97U3LlzFRwcbHXkypVLc+bMUWxsrNq1a6cvv/xS77//vkqWLClJ2rVrl4KDg3Xu3K2FaoGBgRo1apTef/99dejQQQULFtT48eOzFI/JbDabs/QVOQCrnuGIWPUMR8OqZzgau656PnfIZufOU6Kizc5tayxmAQAATs98H3MJnQFDzwAAADBERREAACCLcwmdBRVFAAAAGKKiCAAAwBxFQySKAAAADnZnFkfB0DMAAAAMUVEEAABg6NkQFUUAAAAYoqIIAADA9jiGqCgCAADAEBVFAADg9LiFnzEqigAAADBERREAAIA5ioZIFAEAABh6NsTQMwAAAAxRUQQAAOAWfoaoKAIAAMAQFUUAAADmKBqioggAAABDVBQBAADYHscQFUUAAAAYoqIIAADAHEVDJIoAAAAMPRti6BkAAACGqCgCAACnZzaz4bYRKooAAAAwREURAACAxSyGqCgCAADAEBVFAAAAVj0boqIIAAAAQ1QUAQAAmKNoiEQRAAAgne1xjDD0DAAAAENUFAEAABh6NkRFEQAAAIaoKAIAALA9jiEqigAAADBERREAAIA5ioaoKAIAAMAQFUUAAADmKBoiUQQAACBRNMTQMwAAAAxRUQQAAE7PbOYWfkaoKAIAAMAQFUUAAADmKBoiUQQAAHBAqampateunYYOHaqgoCANGjRIn3/+eYZ+QUFB+uSTTzK0JyQkqE6dOlZtXl5e2rZtW6ZjIFEEAABwsA23U1JSFB4ermPHjlnahgwZovDwcMvjM2fOqHPnzgoJCTE8R1RUlLy8vPT1119b2lxcsjbrkEQRAADAgURFRSk8PFxms9mqvUCBAipQoIDl8aBBg/T000+rWbNmhueJiYnRo48+Kh8fn/uOhcUsAAAA6em2O7Jo+/btCgoK0ooVK+7aZ8uWLdqxY4fCwsLu2icqKkrlypXL8vX/iYoiAACAAw09d+zY8Z595s+fr7Zt26pEiRJ37RMdHa20tDS9+OKLunDhgmrVqqWIiAgVLVo007FQUQQAAMhBTp06pa1bt6pz587/2i8mJkaJiYmKiIjQ9OnTdfHiRb311lu6eTPze0ZSUQQAAMhB2+N8//33qlixoh577LF/7bd+/XqZTCa5u7tLkmbNmqXg4GDt2bNHNWrUyNS1SBQBAABykM2bN+vJJ5+8Z7+8efNaPS5SpIi8vLx04cKFTF+LoWcAAABzuu2O7AzTbNa+ffvuWRFMTExU7dq1tXXrVkvbhQsXdPnyZfn6+mb6eiSKAAAAOcSZM2d07do1w2Hn5ORkxcbGSpI8PDxUs2ZNjR8/Xnv37tWBAwfUr18/NWzYUP7+/pm+HokiAACAA22P82/+/vtvSVLBggUzPPfNN98oODjY8njixImqVKmS3njjDXXu3FmlSpXSlClTsnQ9k/nO3Rz/AxLDnrN3CEAGNT49a+8QACsxCefsHQJgJS31jN2unfTtLJudO2/LUJud29ZYzAIAAJCDVj0/TCSKAAAADrThtiNhjiIAAAAMUVEEAABg6NkQFUUAAAAYIlGEFVPBIso/9jPl8qty1z55Gj4rj2lfylQo8zcVBx6EyWTSK11e0Jc/L9Ou479o444vFDE6TPk98ts7NDi55s2e0Jbf1+tKfJSOHdmisH5v2jsk3K8csuH2w0aiCAuTl7fyvjlSprwed+/jU1KurUMeYlSA1KNviIZN6K9NG35V75B3tXDOErV5uZUiP5pk79DgxILq1NC6LxbryJFovfRydy1b/rkmjH9PA/r3tndoQLZhjiIkk0m5azWR27PdJNO/9XOR+ytvy3ztikyuPg8tPDg3k8mkHn27aMUnazV1zPuSpN9/2a74ywma8eF4VQmoqP17Dtk5Sjij4cPCtXv3fnV97dYeed//8LPy5MmtQQP7atbshUpOTrZzhMgS5igaoqIIuZQoJ7cXe+nGH/9T8mfT79ovT5M2MhXwUurG1Q8xOjg7jwL5tW7VN/pqzfdW7THHjkuSHnm0tB2igrNzdXVVo0b19MW676za16xZL0/PAgpuUNtOkQHZyyEqipcvX1Zqaqry5s0rT09Pe4fjdNLjY3V93JsyJ/x917mJLsXKyLVFByXNHyGXwsUecoRwZlevJGrM4Iy3nGrWsrEk6djhmIccESD5+j4iNzc3HT1m/fmLij4uSSpf3k8bNm62Q2S4bzl8LqGt2C1R/OGHH7RkyRLt3btXKSkplnZ3d3dVqVJFXbp0UbNmzewVnnO5niizEu/+vIuL3Dr2042tPyo9+gCJIuyuWo3KeiO0izZ+94uOHY62dzhwQgX/f1Hj6hXrn51Xr9567OlZ4KHHBNiCXRLFjz76SJGRkerevbv69OmjIkWKyNXVVampqYqLi9Mff/yhQYMG6e2331bnzp3tESL+IU+zl2XKm1+p6xfbOxRANeoEaN7S6Tp98qwiQkfaOxw4KReXf5+5lc58t5yH98yQXRLFRYsWaeLEiYYVQz8/PwUFBcnf31+jR48mUbQzl1K+cm32kpI/HCml3ZBcXCTT//8Befu/KdfjIWnVprkmzBqu49En9forfRV/OcHeIcFJJVy5IunWHNp/ul1JTEi4+tBjwgMiUTRkl0QxOTlZpUv/+wT0YsWK6epV/qHZW+4qQTLlzqO8PcdkeC7/kPm6GbVPSXOG2CEyOJtuvTppwPBQbfttp3p3eVeJV6/ZOyQ4sejoE0pLS9NjfuWs2m8/Pnz42MMPCrABuySKzZs316BBg/Tee++pevXqyp37/8JIT0/X7t27NXz4cLVo0cIe4eEfbmz5XmkHdli15a5c+9bClgWjlR571k6RwZm0D2mnQSPf0frPf9CA3sN040aavUOCk0tJSdHmzdvUtk0rTZ32gaW9XbtWio9P0PYdu+wYHe6L2WzvCBySXRLFESNGaOLEiXr99dd18+ZNeXl5WeYoxsfHK3fu3Hr++ecVERFhj/DwD+Yrl2S+csmqLb3EI7f+/9wJmS9ftEdYcCLeRYto8OgwnTpxRksWrlClahWsnj95/LQu/x1vn+Dg1MaNn6nvv1uu5cvm6eOPl6tevVoKD+upwUPGKSmJPRTx32CXRNHV1VVDhw7Vu+++q8OHDys2NlZJSUlyc3NTsWLFVLFiRbm7u9sjNAAOplGzBsqbz11lypbSsq8XZnh+YN8R+nz513aIDM7up59/00vte2j4sHCtWb1QZ86c18BBYzR9xjx7h4b7wRxFQyaz+b9Xa00Me87eIQAZ1PiUYXo4lpiEc/YOAbCSlnrGbtdOWjbcZufO2yHn7tDgEBtuAwAA2BUVRUPcwg8AAACGqCgCAACwJ7AhEkUAAACGng0x9AwAAABDVBQBAAD+e5vAZAsqigAAADBERREAAIA5ioaoKAIAAMAQFUUAAAAqioaoKAIAAMAQFUUAAAA23DZEoggAAJyeOZ3tcYww9AwAAABDVBQBAABYzGKIiiIAAAAMUVEEAABgMYshKooAAAAwREURAACAVc+GqCgCAADAEBVFAAAAVj0bIlEEAAAgUTTE0DMAAAAMUVEEAAAws5jFCBVFAAAAGKKiCAAAwBxFQ1QUAQAAYIhEEQAAIN1su+M+paam6plnntG2bdssbWPGjJG/v7/VsWTJkrue4+OPP1bDhg0VGBiowYMHKykpKUsxMPQMAADgYFJSUhQeHq5jx45ZtUdHRys8PFxt27a1tHl4eBie4/vvv1dkZKQmT56sIkWKKCIiQpMnT9awYcMyHQcVRQAAAHO67Y4sioqK0ssvv6yTJ09meC46OlqVKlWSj4+P5cibN6/heT755BN16dJFTZo0UbVq1TRy5EitWbMmS1VFEkUAAAAHGnrevn27goKCtGLFCqv2xMREXbhwQeXKlbvnOW7evKl9+/apVq1alrbq1avrxo0bOnz4cKZjYegZAADAgXTs2NGwPTo6WiaTSR988IF++eUXeXl56bXXXrMahr7typUrSklJUdGiRS1tuXPnlpeXl86fP5/pWEgUAQCA0zPngO1xYmJiZDKZ5Ovrq06dOmnHjh0aOnSoPDw81Lx5c6u+ycnJkiRXV1erdldXV6Wmpmb6miSKAAAAOUCbNm3UpEkTeXl5SZIqVKig48ePa9myZRkSRTc3N0nKkBSmpqbedU6jEeYoAgAAONAcxbsxmUyWJPE2X19fXbhwIUNfLy8vubm5KS4uztKWlpam+Ph4+fj4ZPqaJIoAAAA5wMyZM9W1a1ertsOHD8vX1zdDXxcXF1WtWlU7d+60tO3evVu5c+dWhQoVMn1NEkUAAAAH2h7nbpo0aaIdO3Zo4cKFOnnypD777DN98cUX6tatm6Rb8xJjY2Mt/Tt27KiFCxdqw4YN2rt3r0aMGKGXX345S0PPzFEEAADIAapVq6aZM2dq1qxZmjlzpkqVKqWpU6cqMDBQkvTNN98oIiJCR44ckSS1bt1aZ86c0bBhw5SamqqnnnpK/fv3z9I1TWazOfsGzx1EYthz9g4ByKDGp2ftHQJgJSbhnL1DAKykpZ6x27WvjXrVZufOP2ypzc5ta1QUAQAAcsD2OPbAHEUAAAAYoqIIAACQjdvY/JdQUQQAAIAhKooAAADZuI3NfwkVRQAAABiioggAAMAcRUNUFAEAAGCIiiIAAHB6ZvZRNESiCAAAwNCzIYaeAQAAYIiKIgAAABVFQ1QUAQAAYIiKIgAAABtuG6KiCAAAAENUFAEAAJijaIiKIgAAAAxRUQQAAE7PTEXREIkiAAAAiaIhhp4BAABgiIoiAAAA93o2REURAAAAhqgoAgAAMEfREBVFAAAAGKKiCAAAQEXREBVFAAAAGKKiCAAAnJ7ZTEXRCBVFAAAAGKKiCAAAwBxFQySKAAAAJIqGGHoGAACAISqKAADA6ZmpKBr6TyaKXpE77R0CkEHS2c32DgGwUsL3aXuHAMDB/ScTRQAAgCyhomiIOYoAAAAwREURAAAg3d4BOCYqigAAADBERREAADg9Vj0bI1EEAAAgUTTE0DMAAAAMUVEEAABgMYshKooAAAAwREURAAA4PRazGKOiCAAAAENUFAEAAJijaIiKIgAAAAyRKAIAAKdnTjfb7LhfqampeuaZZ7Rt2zZL2+7du/XKK68oMDBQLVq00KpVq/71HLVq1ZK/v7/Vce3atUzHwNAzAACAgw09p6SkKDw8XMeOHbO0xcbGqkePHurQoYMmTJigAwcOKCIiQj4+PmrcuHGGc1y4cEFXr17Vhg0b5O7ubmnPly9fpuMgUQQAAHAgUVFRCg8Pl9lsXY3csGGDvL29FRYWJkkqV66ctm3bpq+++sowUYyOjpaPj4/KlClz37GQKAIAAKdndqCK4vbt2xUUFKR+/fqpevXqlvaGDRuqYsWKGfonJiYanicqKkqPPvroA8VCoggAAOBAOnbsaNheunRplS5d2vL477//1vr169W3b1/D/tHR0UpKSlLnzp31119/qWLFiho8eHCWkkcWswAAAKTb8LCB5ORk9e3bV97e3mrfvr1hn5iYGCUkJKhnz56aM2eO3N3d1bVr17tWII1QUQQAAMhBrl27pl69eun48eP67LPPlDdvXsN+Cxcu1I0bN5Q/f35J0pQpU9SoUSP99NNPevbZZzN1LRJFAADg9BxpjuK/SUxMVPfu3XXy5EktXrxY5cqVu2tfV1dXubq6Wh67ubmpdOnSunDhQqavx9AzAABADpCenq4+ffro9OnT+vTTT/X444/fta/ZbFazZs20du1aS9v169d14sQJ+fr6ZvqaVBQBAAByQEVx9erV2rZtm+bOnStPT0/FxsZKkvLkySMvLy+lpqYqISFBhQsXVq5cudS4cWPNnj1bpUqVUuHChTVz5kwVL15cjRo1yvQ1SRQBAIDTywlDz99//73S09P15ptvWrXXqVNHn376qXbt2qWQkBBt3LhRpUuXVv/+/ZU7d26Fh4crMTFRdevW1fz585UrV65MX9NkvnM3x/+A3K6l7B0CkEHS2c32DgGwUsL3aXuHAFiJu3LUbteObZ75KltW+fy4yWbntjUqigAAwOnlhIqiPbCYBQAAAIaoKAIAAKdHRdEYFUUAAAAYoqIIAABgNtk7AodERREAAACGqCgCAACnxxxFYySKAADA6ZnTGXo2wtAzAAAADFFRBAAATo+hZ2NUFAEAAGCIiiIAAHB6ZrbHMURFEQAAAIaoKAIAAKfHHEVjVBQBAABgiIoiAABweuyjaIxEEQAAOD2z2d4ROKZMJYo7duzI9Alr165938EAAADAcWQqUezcubNMJpPM90i3TSaTDh06lC2BAQAAPCwMPRvLVKK4ceNGW8cBAAAAB5OpRLFUqVIZ2lJTU3X69Gk98sgjMpvNypMnT7YHBwAA8DBQUTSW5e1xzGazpkyZotq1a+uZZ57RuXPnNHDgQA0ZMkQ3btywRYwAAACwgywnip9++qnWrVun4cOHy9XVVZLUrFkzbdiwQZGRkdkeIAAAgK2ZzbY7crIsJ4orVqzQsGHD1K5dO5lMt8q0rVq10pgxY/TVV19le4AAAACwjyzvo3j69GlVrFgxQ3uFChUUGxubLUEBAAA8TMxRNJblimKpUqW0b9++DO2//PKLypQpky1BAQAAPExms8lmR06W5Yri66+/rpEjRyo2NlZms1lbtmzRihUr9Omnn2rQoEG2iBEAAAB2kOVE8YUXXlBaWprmzp2r5ORkDRs2TIULF9Y777yjDh062CJGAAAAmzKn2zsCx3Rf93pu37692rdvr0uXLslsNqtIkSLZHRcAAADs7L4SxdjYWH322Wc6duyYXF1dVb58eXXs2FGenp7ZHR8AAIDNpefwuYS2kuXFLNu2bVPz5s21bt06mUwmJScn67PPPtNTTz2lw4cP2yJGAAAA2EGWK4qTJk3Ss88+qxEjRihXrlySbt3Ob9CgQRozZoyWLFmS7UECAADYUk5fnWwrWa4oHj16VN26dbMkiZLk6uqqXr16ae/evdkaHAAAAOwny4nio48+qqNHj2ZoP3HihEqVKpUtQQEAADxM5nSTzY6cLFNDzzt27LD8d+vWrTVs2DDFxcWpRo0acnFx0YEDBzR16lT17dvXZoECAADYSk6/J7OtmMzme780FSpUkMlk0r26mkwmHTp0KNuCu1+5XalswvEknd1s7xAAKyV8n7Z3CICVuCsZRywflkOPt7LZuSse+8Zm57a1TFUUN27caOs4AAAA7CanDxHbSqYSxczOPUxJSXmgYAAAAOA4srw9zuXLl/XBBx/o6NGjunnzpiTJbDbrxo0bioqK0h9//JHtQQIAANgSG24by/Kq55EjR+qLL75QoUKF9Mcff6hYsWK6du2adu/erTfeeMMWMQIAAMAOslxR3LJliyZOnKjGjRvryJEjev3111WhQgUNHTpUUVFRtogRAADApthw21iWK4rXrl2Tv7+/JMnX19dy275OnTpp27Zt2RsdAAAA7CbLiWKxYsV05swZSVK5cuV05MgRSVLevHmVkJCQvdEBAAA8BGaz7Y6cLMuJ4lNPPaWIiAjt3LlT9evX1+eff67vvvtOs2bNUtmyZW0RIwAAAOwgy4liv3791LhxY509e1b16tXTU089pXfeeUebNm3SwIEDbREj7KB5sye05ff1uhIfpWNHtiis35v2DglOJD09XSs+X6+2IT1Vu1lbPf3Sa5o4c54Sr12z9Dl5+qx6Dxiuei1eVHCr9ho1ebbV88DDVKJkMUWf/EMNguvYOxTcp3SzyWZHTpblxSyurq4aMmSI5fGoUaMUFhYmDw8P5c6d5dPBAQXVqaF1XyzWylVfacSIyWrQoI4mjH9PuXPn1qTJ79s7PDiBRUtXa/aHi9W1w4uqW6u6Tpw6o9kffqJjMSf04Yyxupp4Td1CB8m7cCGNfS9cly7Ha9qcRTpz7oLmTRtj7/DhZEqWKq5Vny9SQS9Pe4eCB8BiFmNZriga8fLy0q5du/Tkk09mx+lgZ8OHhWv37v3q+lqovv/hZw0bPklTp83VoIF95e7ubu/w8B+Xnp6uRUtX6aXnW6lfz9dUr3agXmn3jN4L762tf+zSgcPHtOLz9UpIuKK5U0apacN6evG5lpo4fIB+27ZTf+49YO9vAU7CZDLplY5t9dOv6+RTtIi9w8F/UGpqqp555hmrxcKnTp1S165dVb16dbVq1Uq//vrrv57j66+/VrNmzRQQEKDevXvr0qVLWYohWxJFSUpOTtbZs2ez63SwE1dXVzVqVE9frPvOqn3NmvXy9Cyg4Aa17RQZnEXitet6pkVTtWre2Kr90bJlJEmnzpzTb9t3qkZAFRXyKmh5vn6dGsqfL682b9nxMMOFE6tcpYKmzBillcu+UK83Btg7HDwgR1vMkpKSorCwMB07duwfMZrVu3dveXt7a82aNXr++efVp0+fu+Zfe/fu1ZAhQ9SnTx+tWLFCV65cUURERJbiYKwYVnx9H5Gbm5uOHouxao+KPi5JKl/eTxs2brZDZHAWngU8NLhfzwzt/9u8RZL0mG9ZxRw/paeffMLq+Vy5cqlUyeI6fvL0Q4kTOH36rGpXb6ZzZy8wNxHZKioqSuHh4TLfkWVu3bpVp06d0vLly5UvXz75+flpy5YtWrNmjfr27ZvhPEuWLFHLli3Vpk0bSdKkSZPUpEkTnTp1SmXKlMlULNlWUcR/Q0HPW3Nsrl5JtGq/evXWY0/PAg89JmDvgcNa+OlKNW4QpMd9yynx2jV55M+XoV/+fHmVeO26HSKEM4q/nKBzZy/YOwxkE0dazLJ9+3YFBQVpxYoVVu179uxRpUqVlC/f//38q1mzpnbv3m14nj179qhWrVqWxyVKlFDJkiW1Z8+eTMdCRRFWXFz+/W+H9PT0hxQJcMufew+oz4ARKlWymMYMCZMkpafffSznXp9hAHB0HTt2NGyPjY1V0aJFrdqKFCmi8+fPG/a/ePFilvobyVSiGBkZec8+J06cyPRFJWnHjszPI6pdm3lxD0vClSuSJI8C+a3ab1cSExKuPvSY4Ly+3bBJ742dprKPlNK8qWPkVfBWxbuARz5du56UoX/itesq6u39sMME8B+QE1Y9JyUlydXV1arN1dVVqamphv2Tk5Oz1N9IphLFtWvXZupkJUqUyPSFR40aZbk39J1j8P9kMpl06NChTJ8XDyY6+oTS0tL0mF85q/bbjw8fPpbxiwAb+Oiz1Zo2Z5FqB1bTzPFDVcDj//54KfdIaZ08bT15++bNmzpz9oKaNWrwsEMFgIfCzc1N8fHxVm2pqal33ZHEzc0tQ1KYmpqqvHnzZvqamUoU//e//2X6hJm1Zs0ahYWF6fTp01qxYoXc3Nyy/RrIupSUFG3evE1t27TS1GkfWNrbtWul+PgEbd+xy47RwVms/OIbTX1/oZ5+8gmNH/qu8uTJY/V8/do1tOiz1bp0OV6FC3lJkn7f/qeuJyWpfp0adogYQE6XEzbGLlasmKXIdltcXFyG4eV/9o+Li8vQ38fHJ9PXtNtkHldXV02bNk2SNGPGDHuFAQPjxs9UnTqBWr5snp5u0UQjR/RXeFhPTZg4W0lJyfYOD/9xcX9f0qRZ81WqRDF1fOFZHTwSpT37D1mOS5fj1b5ta7m7uarHO0O0YdNvWv3ldxo4cpIa1q2lwKqV7P0tAMiBzDY8sktAQIAOHDig5OT/+128c+dOBQQE3LX/zp07LY/PnTunc+fO3bW/EbsuZnF1ddXUqVO1fft2e4aBO/z08296qX0PDR8WrjWrF+rMmfMaOGiMps+YZ+/Q4AR+2bJDySkpOnPugkJ69c/w/JjBYWrTurkWzZ6oiTPnadDIycqXL69aNG2od3t3t0PEAPBw1KlTRyVKlFBERIR69eqln376SXv37tX48eMl3RpWTkhIUOHChZUrVy516NBBnTt3VvXq1VW1alWNHTtWjRs3zvTWOJJkMv/bBMEcKrdrKXuHAGSQdJb9J+FYSvg+be8QACtxV47a7dq/l3jBZueuf27NfX+tv7+/PvnkEwUFBUm6tXh4yJAh2rNnj8qWLavBgwerfv36kqRt27YpJCREGzduVOnSpSXdWmcya9YsJSQkqEGDBho9erQKFSqU6euTKAIPCYkiHA2JIhwNiaLjeaCh59TU1AzLrgEAAHKanLA9jj3c12KWZcuWqWnTpqpevbpOnTql4cOHa86cOdkdGwAAAOwoy4niV199palTp6pt27aWLSv8/Pz0wQcfaNGiRdkeIAAAgK2l2/DIybKcKC5atEhDhgxR3759LbfKCgkJ0bBhwzLckxAAAAA5V5YTxb/++svqBtO3BQUF6dy5c9kSFAAAwMNklslmR06W5UTR29tbf/31V4b2Xbt23XVncAAAAEeWbrbdkZNlOVFs3769Ro0apY0bN0qSYmJitGzZMo0dO1bt2rXL9gABAABgH1neHqdHjx66evWqwsLClJKSojfffFO5c+fWK6+8orfeessWMQIAANhUeg4fIraV+9pHMSwsTD179lRUVJTMZrN8fX3l4eGR3bEBAADAjrKcKJ49e9by30WKFJEkXblyRVeuXJEklSxZMptCAwAAeDhy+qITW8lyoti0aVOZTHd/MQ8dOvRAAQEAAMAxZDlR/OSTT6we37x5U3/99Zc+/vhjDRo0KNsCAwAAeFhy+sbYtpLlRLFOnToZ2urVq6cyZcpo9uzZatq0abYEBgAAAPu6r8UsRsqVK6fDhw9n1+kAAAAeGuYoGnugxSy3JSYmat68eSpdunS2BAUAAPAwMfRsLFsWs5jNZuXLl0+TJ0/OtsAAAABgXw+8mEWS8uTJo/Llyyt//vzZEhQAAMDDREXR2H0liv369ZOfn58t4gEAAICDyHKiuHXrVrm5udkiFgAAALtgMYsxl6x+Qdu2bTVlyhQdO3ZMqamptogJAAAADiDLFcVNmzbp5MmT+v777w2f584sAAAgp0mnoGgoy4liz549bREHAAAAHEymEsWKFSvq119/VZEiRdS2bVtbxwQAAPBQpTNH0VCmEkWz2WzrOAAAAOyGTMdYlhezAAAAwDlkeo7it99+Kw8Pj3v2a9OmzYPEAwAA8NCx4baxTCeKY8aMuWcfk8lEoggAAPAfkelE8bffflORIkVsGQsAAIBdpJtYzGIkU3MUTbx4AAAATodVzwAAwOmR6RjLVEWxbdu23N8ZAADAyWSqojh+/HhbxwEAAGA3rHo2luVb+AEAAPzXcK9nY2y4DQAAAENUFAEAgNPjXs/GqCgCAADAEBVFAADg9NgexxgVRQAAABiioggAAJweq56NUVEEAACAISqKAADA6bHhtjESRQAA4PRYzGKMoWcAAAAYoqIIAACcHotZjFFRBAAAgCEqigAAwOmxmMUYiSIAAICDWLt2rSIiIjK0m0wmHT58OEP7c889pyNHjli1ffXVVypfvny2xEOiCAAAnJ6jVBRbtWqlhg0bWh6npaWpS5cuaty4cYa+N2/e1PHjx7VkyRKVK1fO0l6oUKFsi4dEEQAAwEG4u7vL3d3d8njevHkym8169913M/Q9ffq0bty4oWrVqsnNzc0m8ZAoAgAAp2d2wFXP8fHx+vDDDzVmzBi5urpmeD4qKkolSpSwWZIoseoZAABA6TY87teyZctUtGhRPf3004bPR0dHK0+ePHrzzTfVoEEDderUSXv37n2AK2ZEoggAAOBgzGazVq1apU6dOt21z19//aWEhAS99NJLmj9/vvz8/NSlSxedO3cu2+Jg6BkAADg9R1nMctu+fft04cIFtW7d+q59Ro8ereTkZHl4eEiSRowYoT///FPr1q3TW2+9lS1xUFEEAABwMJs3b1atWrVUsGDBu/bJnTu3JUmUbm2h4+vrqwsXLmRbHCSKAADA6ZlteNyPvXv3qkaNGv/ap3PnzoqMjLQ8Tk9P15EjR+Tr63ufV82IRBEAAMDBHDt2TI899phV282bNxUbG6vU1FRJUtOmTfXxxx9r48aNiomJ0ahRo3T16lW1bds22+JgjiIAAHB66Q62PU5cXJw8PT2t2s6dO6cnn3xSn3zyiYKCgtS1a1elpKRozJgxiouLU0BAgD766COr4egHZTKbzfdbFXVYuV1L2TsEIIOks5vtHQJgpYSv8ZYbgL3EXTlqt2vPfOTuq4sf1Nsnl9js3LZGRREAADg9R1v17ChIFAEAgNMjUTTGYhYAAAAYoqIIAACc3n9uwUY2oaIIAAAAQ1QUAQCA03O07XEcBRVFAAAAGKKiCAAAnB6rno1RUQQAAIAhKooAAMDpserZGBVFAAAAGKKiCAAAnF46NUVDJIrAQ5K3ZEN7hwBYWeTTxN4hAA6DxSzGGHoGAACAISqKAADA6THwbIyKIgAAAAxRUQQAAE6POYrGqCgCAADAEBVFAADg9NJN9o7AMVFRBAAAgCEqigAAwOmx4bYxEkUAAOD0SBONMfQMAAAAQ1QUAQCA02N7HGNUFAEAAGCIiiIAAHB6LGYxRkURAAAAhqgoAgAAp0c90RgVRQAAABiioggAAJweq56NkSgCAACnx2IWYww9AwAAwBAVRQAA4PSoJxqjoggAAABDVBQBAIDTYzGLMSqKAAAAMERFEQAAOD0zsxQNUVEEAACAISqKAADA6TFH0RiJIgAAcHpsuG2MoWcAAAAYoqIIAACcHvVEY1QUAQAAYIiKIgAAcHrMUTRGRREAAACGSBQBAIDTS7fhkVU//vij/P39rY7Q0FDDvr///rueeeYZBQQEKCQkRKdOnbqPK94dQ88AAAAOJCoqSk2aNNHo0aMtbW5ubhn6nT17Vr1791bfvn3VsGFDvf/+++rVq5e+/PJLmUymbImFRBEAADg9R7qFX3R0tMqXLy8fH59/7bdq1SpVqVJF3bp1kySNHz9eDRo00Pbt2xUUFJQtsTD0DAAAnJ4jDT1HR0erXLly9+y3Z88e1apVy/I4b968qly5snbv3n0fVzVGoggAAOAgzGaz/vrrL/36669q0aKFmjVrpilTpig1NTVD39jYWBUtWtSqrUiRIjp//ny2xcPQMwAAcHqOMvR89uxZJSUlydXVVTNmzNDp06c1ZswYJScn67333rPqe7vfP7m6uhomlfeLRBEAAMBBlCpVStu2bVPBggVlMplUsWJFpaenq3///oqIiFCuXLksfd3c3DIkhampqfL09My2eBh6BgAATs+R5ih6eXlZrVr28/NTSkqKEhISrPoVK1ZMcXFxVm1xcXH3XASTFSSKAAAADmLz5s0KCgpSUlKSpe3QoUPy8vJS4cKFrfoGBARo586dlsdJSUk6ePCgAgICsi0eEkUAAOD00s1mmx1ZERgYKDc3N7333nuKiYnRpk2bNGnSJHXv3l03b95UbGysZbj5hRde0J9//qn58+fr2LFjioiIUOnSpbNtaxyJRBEAAMBheHh4aOHChbp06ZJeeOEFDRkyRO3bt1f37t117tw5BQcHa9euXZKk0qVLa/bs2VqzZo1efPFFxcfH6/3338+2zbYlyWQ2ZzHVzQFyu5aydwgA4PAW+TSxdwiAlZAzS+x27U5l29ns3EtOrLXZuW2NVc8AAMDppTvI9jiOhqFnAAAAGKKiCAAAnJ6jbLjtaKgoAgAAwBAVRQAA4PTuZ2NsZ0BFEQAAAIaoKAIAAKfHqmdjVBQBAABgiIoiAABweqx6NkaiCAAAnB6LWYwx9AwAAABDVBQBAIDTM5sZejZCRREAAACGqCgCAACnx/Y4xqgoAgAAwBAVRQAA4PRY9WyMiiIAAAAMUVEEAABOjw23jZEoAgAAp8diFmMMPQMAAMAQFUUAAOD02HDbGBVFAAAAGKKiCAAAnB7b4xijoggAAABDVBQBAIDTY3scY1QUYah5sye05ff1uhIfpWNHtiis35v2DglOjs8kHNHjHRvruf9NUIdjC/TczxPl36WZvUMCshWJIjIIqlND675YrCNHovXSy921bPnnmjD+PQ3o39veocFJ8ZmEI3qsQ2PVm9xd5349oJ9em64TX25TnTEhqvRmS3uHhvuQLrPNjpzMZP4PrgfP7VrK3iHkaN98vVReXp6qH/yspW38uMF6840QlSxdXcnJyXaMDs6Iz6RtLPJpYu8QcrSn1w2T+aZZ37cbbWlr+H5vedfw0+f1wuwYWc4VcmaJ3a7drEwLm517w6nvbXZuW6OiCCuurq5q1Kievlj3nVX7mjXr5elZQMENatspMjgrPpNwVLncXHUjMcmqLeXyVbkVKmCniPAgzGazzY6cjEQRVnx9H5Gbm5uOHouxao+KPi5JKl/ezw5RwZnxmYSjOrTwe5VsVFWPtmugPAXyqmSjqvJ7qaFi1vxq79BwHxh6NmaXVc+pqamaOXOmvv76a129elX169dXv3795Of3fz/w4+Li1LBhQx06dMgeITqtgp6ekqSrVxKt2q9evfXY05O/lPFw8ZmEozr+xe8qXq+iGs7uaWk789Ne7Rhuv+FTILvZpaI4bdo0bdiwQQMGDNCoUaMUFxenF154QRs2bLDql9PLtTmRi8u/fyTS09mSFA8Xn0k4qiaLwlS2dW3tHL1M378wRtuGLFaRgEfVaF5fe4eG+2C24f9yMrtUFL/99ltNmzZNNWvWlCS1bt1akyZN0jvvvKPJkyerZctbK8ZMJpM9wnNqCVeuSJI8CuS3ar9dtUlIuPrQY4Jz4zMJR+RT63GVahqg399doKhlP0uSLmw9rMSTF/Xkp/1Vqll1ndmw264xAtnBLolicnKyvLy8LI9NJpMGDhwoFxcX9e/fX7lz51ZgYKA9QnN60dEnlJaWpsf8ylm13358+PCxhx8UnBqfSTii/KW8JUmxO45atV/YeliS5FW+NIliDpPOKKYhuww9BwUFadKkSbp06ZJVe//+/dW+fXv169dPn332mT1Cc3opKSnavHmb2rZpZdXerl0rxccnaPuOXXaKDM6KzyQc0ZWos5KkokH+Vu1Fa5eXJCWevPjQYwJswS4VxSFDhig0NFQNGjTQggUL1KBBA8tzQ4cOVaFChTR37lx7hAZJ48bP1PffLdfyZfP08cfLVa9eLYWH9dTgIeOUlMR+dXj4+EzC0Vw6cEIn1m9XreGvyrVgfsXtipZX+VIKCG+nuD0xOvntH/YOEVlEPdGYXTfcjomJkY+PjwoUyLhqMTo6Whs3btQbb7yR5fOy4faDe/75pzV8WLj8y/vpzJnzmvvBYk2fMc/eYcGJ8ZnMfmy4/WBc8uRS1bfbyPeFBspXrJCunf1bJ7/9Q3unf6606yn2Di9HsueG2w1LPWmzc28+s9Fm57Y17swCAE6KRBGOxp6JYoNSTW127t/O/M9m57Y1uww9AwAAOJKcvjG2rXBnFgAAABiioggAAJzef3AmXragoggAAABDVBQBAIDTY46iMSqKAAAAMERFEQAAOD0zFUVDVBQBAABgiEQRAAA4PbPZbLMjqy5cuKDQ0FDVqVNHDRs21Pjx45WSYny3n549e8rf39/q+Omnnx705bBg6BkAADg9R1nMYjabFRoaKk9PTy1dulQJCQkaPHiwXFxcNHDgwAz9o6OjNXnyZNWrV8/SVrBgwWyLh0QRAADAQcTExGj37t367bff5O3tLUkKDQ3VxIkTMySKqampOn36tKpWrSofHx+bxEOiCAAAnJ6jbLjt4+OjBQsWWJLE2xITEzP0jYmJkclkUpkyZWwWD3MUAQAAHISnp6caNmxoeZyenq4lS5aobt26GfrGxMTIw8NDAwYMUHBwsF588UVt2rQpW+MhUQQAAE4vXWabHQ9i8uTJOnjwoPr165fhuZiYGCUnJys4OFgLFixQo0aN1LNnT+3bt++BrvlPDD0DAAA4oMmTJ2vx4sWaPn26ypcvn+H5Xr16qXPnzpbFKxUqVNCBAwe0cuVKVa1aNVtiIFEEAABOz9E23B49erSWLVumyZMnq0WLFoZ9XFxcMqxw9vX1VVRUVLbFwdAzAACAA4mMjNTy5cs1bdo0tW7d+q79Bg0apIiICKu2w4cPy9fXN9tioaIIAACcXrqDrHqOjo7WnDlz9MYbb6hmzZqKjY21POfj46PY2FgVKFBA7u7uatq0qcLCwhQUFKTAwEB99dVX2rlzp0aNGpVt8ZAoAgAAp+coQ88bN27UzZs3NXfuXM2dO9fquSNHjig4OFjjx49Xu3bt9NRTT2n48OGaO3euzp49q8cff1wLFixQ6dKlsy0ek9lRNg7KRrldS9k7BABweIt8mtg7BMBKyJkldrt25WJBNjv3gQvbbHZuW6OiCAAAnJ6jDD07GhazAAAAwBAVRQAA4PQcZY6io6GiCAAAAENUFAEAgNNjjqIxKooAAAAwREURAAA4PeYoGiNRBAAATo+hZ2MMPQMAAMAQFUUAAOD0GHo2RkURAAAAhqgoAgAAp2c2p9s7BIdERREAAACGqCgCAACnl84cRUNUFAEAAGCIiiIAAHB6ZvZRNESiCAAAnB5Dz8YYegYAAIAhKooAAMDpMfRsjIoiAAAADFFRBAAATi+diqIhKooAAAAwREURAAA4PTOrng1RUQQAAIAhKooAAMDpserZGIkiAABwemy4bYyhZwAAABiioggAAJweQ8/GqCgCAADAEBVFAADg9Nhw2xgVRQAAABiioggAAJwecxSNUVEEAACAISqKAADA6bGPojESRQAA4PQYejbG0DMAAAAMUVEEAABOj+1xjFFRBAAAgCEqigAAwOmZWcxiiIoiAAAADFFRBAAATo85isaoKAIAAMAQFUUAAOD02EfRGBVFAAAAGKKiCAAAnB6rno2RKAIAAKfH0LMxhp4BAAAcSEpKigYPHqxatWopODhYixYtumvfgwcP6qWXXlJAQIBeeOEF7d+/P1tjIVEEAABOz2w22+zIqkmTJmn//v1avHixhg8frsjISH333XcZ+l2/fl1vvPGGatWqpbVr1yowMFBvvvmmrl+/nh0viSQSRQAAAIdx/fp1rVq1SkOGDFHlypXVvHlzde/eXUuXLs3Q95tvvpGbm5sGDBggPz8/DRkyRPnz5zdMKu8XiSIAAHB6ZhseWXH48GGlpaUpMDDQ0lazZk3t2bNH6enpVn337NmjmjVrymQySZJMJpNq1Kih3bt3Z/Gqd0eiCAAA4CBiY2NVqFAhubq6Wtq8vb2VkpKi+Pj4DH2LFi1q1VakSBGdP38+2+L5T656Tks9Y+8QAABADuIouUNSUpJVkijJ8jg1NTVTfe/s9yCoKAIAADgINze3DIne7cfu7u6Z6ntnvwdBoggAAOAgihUrpsuXLystLc3SFhsbK3d3d3l6emboGxcXZ9UWFxeXYTj6QZAoAgAAOIiKFSsqd+7cVgtSdu7cqapVq8rFxTptCwgI0K5duyxb8JjNZv35558KCAjItnhIFAEAABxE3rx51aZNG40YMUJ79+7Vhg0btGjRIoWEhEi6VV1MTk6WJD399NO6cuWKxo4dq6ioKI0dO1ZJSUlq2bJltsVjMnPPGgAAAIeRlJSkESNG6IcffpCHh4def/11de3aVZLk7++v8ePHq127dpKkvXv3avjw4YqOjpa/v79GjhypSpUqZVssJIoAAAAwxNAzAAAADJEoAgAAwBCJIgAAAAyRKMJQSkqKBg8erFq1aik4OFiLFi2yd0iApFubyT7zzDPatm2bvUOBk7tw4YJCQ0NVp04dNWzYUOPHj1dKSoq9wwKy1X/yFn54cJMmTdL+/fu1ePFinT17VgMHDlTJkiX19NNP2zs0OLGUlBSFh4fr2LFj9g4FTs5sNis0NFSenp5aunSpEhISNHjwYLm4uGjgwIH2Dg/INiSKyOD69etatWqVPvzwQ1WuXFmVK1fWsWPHtHTpUhJF2E1UVJTCw8PFRg1wBDExMdq9e7d+++03eXt7S5JCQ0M1ceJEEkX8pzD0jAwOHz6stLQ0BQYGWtpq1qypPXv2KD093Y6RwZlt375dQUFBWrFihb1DAeTj46MFCxZYksTbEhMT7RQRYBtUFJFBbGysChUqJFdXV0ubt7e3UlJSFB8fr8KFC9sxOjirjh072jsEwMLT01MNGza0PE5PT9eSJUtUt25dO0YFZD8SRWSQlJRklSRKsjxOTU21R0gA4NAmT56sgwcPavXq1fYOBchWJIrIwM3NLUNCePuxu7u7PUICAIc1efJkLV68WNOnT1f58uXtHQ6QrUgUkUGxYsV0+fJlpaWlKXfuWx+R2NhYubu7y9PT087RAYDjGD16tJYtW6bJkyerRYsW9g4HyHYsZkEGFStWVO7cubV7925L286dO1W1alW5uPCRAQBJioyM1PLlyzVt2jS1bt3a3uEANsFvfWSQN29etWnTRiNGjNDevXu1YcMGLVq0SCEhIfYODQAcQnR0tObMmaMePXqoZs2aio2NtRzAfwlDzzAUERGhESNGqEuXLvLw8FDfvn311FNP2TssAHAIGzdu1M2bNzV37lzNnTvX6rkjR47YKSog+5nM7F4LAAAAAww9AwAAwBCJIgAAAAyRKAIAAMAQiSIAAAAMkSgCAADAEIkiAAAADJEoAgAAwBCJIgAAAAyRKAL/cU2bNpW/v7/lqFChgmrUqKFOnTppx44d2X69bdu2yd/fX6dPn5Ykde7cWYMGDcrU116/fl1Lly59oOufPn1a/v7+2rZtW6biux+zZ89W06ZN7/vrs+scAGBr3MIPcALdunVTt27dJElms1nx8fGaNm2aunfvrm+//VYlS5a02bVnz56tXLlyZarvokWLtHbtWr366qs2iwcAkHlUFAEnkC9fPvn4+MjHx0dFixZV+fLlNXLkSCUnJ+vHH3+06bW9vLxUoECBTPXljqIA4FhIFAEnlTv3rQEFV1dXSbeGqCdOnKhWrVopKChI27dvl9ls1ocffqgnn3xSAQEBev755/Xll19aneePP/7QSy+9pGrVqum5557T4cOHrZ6/c+h579696tq1qwIDA1W/fn0NHz5cSUlJmj17tiIjI3XmzBmroeE1a9aoZcuWqlatmlq2bKnFixcrPT3dcr6jR48qJCRE1atXV/PmzbVly5YHfm2OHj2qN998U7Vr11aVKlX05JNPatGiRRn6vf/++woKClKNGjX07rvvKj4+3vLc1atXNXToUNWtW1c1a9ZUSEiI9u3b98CxAcDDxNAz4IQuXLigcePGKV++fGrUqJGlfcmSJZo3b54KFCggf39/TZ8+XV9//bWGDRsmX19f7dixQyNGjNDVq1f16quv6tSpU+rWrZvatGmjCRMmKCoqSsOGDbvrdU+dOqUuXbqoefPmWrFiha5evaqBAwdq5MiRGjp0qK5fv65vvvlGq1evVuHChbVixQpNmzZNw4YNU7Vq1XTw4EGNHj1aFy5c0IABA3T16lVL0rlq1SpdvHhRQ4cOfaDXJikpSd26dVODBg20fPly5cqVS6tWrdLEiRNVr149VaxYUZJ05swZbd26VR999JESExM1dOhQRUREaO7cuTKbzerRo4fc3d01b948eXh4aN26derQoYNWrlypSpUqPVCMAPCwkCgCTmDevHmWilhaWppSU1Pl5+enGTNmWM1PbNSokerXry/p1sKSjz/+WNOmTVPjxo0lSY888ojOnDmjhQsX6tVXX9XKlSvl7e2t4cOHK1euXPLz89O5c+c0fvx4wzhWrlwpLy8vjRs3zlLRHDNmjHbt2qX8+fMrX758ypUrl3x8fCRJc+bMUc+ePdW6dWtJUpkyZZSYmKiRI0fq7bff1vr165WUlKQJEyaoQIECevzxxzV48GD17t37vl+rpKQkhYSE6NVXX1X+/PklSaGhoVqwYIGOHDliSRTd3Nw0ffp0eXt7S5KGDRumbt266cSJEzp79qx2796trVu3ysvLS5IUFhamP//8U5988okmTJhw3/EBwMNEogg4gVdeeUWdO3eWJLm4uNx13mDZsmUt/x0VFaWUlBSFh4fLxeX/ZqncTjSTk5N19OhRVapUyWqxSo0aNe4ax9GjR1W5cmVLkihJdevWVd26dTP0vXTpks6fP69p06Zp5syZlvb09HSlpKTo9OnTOnr0qMqVK2f1vQQGBt7r5fhXhQsXVseOHfX111/r4MGDOnnypGU4/Z9D3mXLlrUkiZIUEBAgSTp27JiOHz8us9msJk2aWJ07NTVVKSkpDxQfADxMJIqAEyhYsKBVEng37u7ulv++vbBkxowZ8vX1zdDX1dVVJpPJKnmSZJUE3unfnrvT7fNGRERYqpz/VKJEiSxfPzNiY2PVvn17FS5cWE2bNlVwcLCqVq1qNUQvKcNK7ps3b0qS8uTJo/T0dHl4eGjt2rUZzn97TigA5AQsZgFgyNfXV7lz59bZs2dVtmxZy7Fp0yYtXLhQLi4uqlChgvbv36/U1FTL1+3fv/+u53zsscd08OBBS1IlST/++KOaNm2qlJQUmUwmS3uRIkVUuHBhnTp1yur6Bw4c0IwZMyRJFSpU0PHjx3Xp0qVMXT8zvv76a8XHx2vZsmXq1auXmjdvroSEBEnWq7KPHz+uxMREy+OdO3fKZDLpscceU/ny5ZWYmKgbN25Yxf7hhx9q48aNDxQfADxMJIoADBUoUECvvPKKZs6cqXXr1unUqVNavXq1Jk+erKJFi0qSOnTooKSkJA0ePFjR0dH66aefNHv27Lues2PHjrp8+bKGDx+u6Oho7dixQ5MmTVLdunXl5uamfPnyKSEhQX/99ZfS0tLUo0cPffrpp1qyZIlOnjypH3/8USNGjJC7u7tcXV3VunVrFSlSROHh4Tp8+LC2b9+usWPHZur727Fjh3755Rer48SJEypevLiSkpL03Xff6ezZs/r1118VFhYmSVYJcUpKit555x0dPHhQv/32m0aPHq02bdqoVKlSatiwoSpWrKh+/fpp69atOnHihMaPH6+1a9fKz8/vAd4VAHi4GHoGcFcREREqVKiQZs6cqYsXL6pEiRIKDQ1V9+7dJUnFihXT4sWLNW7cOLVt21YlSpRQz549NXLkSMPzFStWTIsWLdLkyZPVpk0bFSxYUK1atbIkYk899ZRWrlyp5557TkuWLFG3bt3k5uamTz/9VBMmTJC3t7defvllhYaGSrq1P+TixYs1evRodejQQQULFlRoaKgiIiLu+b0Z3S2mT58+6tOnjw4cOKAJEyYoMTFRpUqV0ksvvaSNGzdq37596tChgySpSpUqqlixokJCQmQymdSqVSvLOXPlymX5Pt955x0lJSXJz89PkZGRqlevXtbfCACwE5OZHW4BAABggKFnAAAAGCJRBAAAgCESRQAAABgiUQQAAIAhEkUAAAAYIlEEAACAIRJFAAAAGCJRBAAAgCESRQAAABgiUQQAAIAhEkUAAAAY+n+sFQfUrPaCmQAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 800x600 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# TO DO: Print confusion matrix using a heatmap\n",
        "# Predictions\n",
        "y_wine_pred = decision_tree_model.predict(X_wine_test)\n",
        "\n",
        "# Generate confusion matrix\n",
        "cm = confusion_matrix(y_wine_test, y_wine_pred)\n",
        "    \n",
        "# Plot the confusion matrix using a heatmap\n",
        "plt.figure(figsize=(8,6))\n",
        "sns.heatmap(cm, annot=True, fmt=\"d\", xticklabels=True, yticklabels=True)\n",
        "plt.xlabel('Predicted Label')\n",
        "plt.ylabel('True Label')\n",
        "plt.title('Confusion Matrix for Decision Tree Classifier')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "id": "5ef95947",
      "metadata": {
        "id": "5ef95947"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           1       1.00      0.88      0.93        16\n",
            "           2       0.91      0.95      0.93        21\n",
            "           3       0.89      1.00      0.94         8\n",
            "\n",
            "    accuracy                           0.93        45\n",
            "   macro avg       0.93      0.94      0.93        45\n",
            "weighted avg       0.94      0.93      0.93        45\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# TO DO: Print classification report\n",
        "\n",
        "from sklearn.metrics import classification_report\n",
        "\n",
        "report = classification_report(y_wine_test, y_wine_pred)\n",
        "print(report)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bf319621",
      "metadata": {
        "id": "bf319621"
      },
      "source": [
        "## Questions (6 marks)\n",
        "1. How do the training and validation accuracy change depending on the method used? Explain with values.\n",
        "1. What are two reasons why the support vector machines model did not work as well as the tree-based model?\n",
        "1. How many samples were incorrectly classified in step 5.2?\n",
        "1. In this case, is maximizing precision or recall more important? Why?"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1FQstcwnXXng",
      "metadata": {
        "id": "1FQstcwnXXng"
      },
      "source": [
        "\n",
        "<font color='Green'><b>YOUR ANSWERS HERE</b></font>\n",
        "\n",
        "1. The difference between the training and validation accuracy is drastic. The decision tree classifier performs much better in both the training and test set with SVC performing at 0.70 and 0.66 vs DT at 0.97 and 0.89 respectively. This means that the Descison tree has almost no deviations from its expected results on its training set but is also predicted very well on new data. The DT performed 23% better on the test set. This is a reflection of clear overfitting on both cases, but especially DT due to the large difference.\n",
        "\n",
        "2. Two reasons as to why the SVM didn't work as well as the the tree-based model include:\n",
        "    - SVMs are sensitive to the feature space and scale, especially in cases where the feature dimensions have different scales or the data is not linearly separable. The effectiveness of SVM is largely dependent on the tuning of its hyperparameters, and the scaling of input features. If these aspects are not optimally configured, SVM might not capture the complexity of the specific distributions of the data as effectively as tree-based models, which inherently handle varying feature scales and non-linear separability by partitioning the data space into simpler regions. Thus if we were instructed to use more specific parameters it may have performed better.\n",
        "    - Generalization of SVC: SVC aims to find a decision boundary with the maximum margin, which can help in better generalization to unseen data. However, if the SVC model is too simple, or if the hyperparameters are not optimally tuned, it might not capture the underlying data structure well, leading to underfitting. This underfitting results in poorer performance on both training and validation datasets compared to a model that captures the data's complexity more accurately, like a Decision Tree in this scenario.\n",
        "    \n",
        "\n",
        "3. 2 samples were labeled as 1 but should have been 0, and 1 sample was labeled as 1 but should have been 2. Thus 3 samples were incorrectly labeled in total.\n",
        "\n",
        "4. In this context maximizing one over the other comes down to these factors:\n",
        "\n",
        "    - If maximizing precision: You're aiming to ensure that when a wine is classified as belonging to a certain cultivar, it is very likely correct. This could be important in contexts where misclassification could lead to significant economic loss or damage to reputation, such as in the marketing of high-end wines where customers expect a certain quality and origin.\n",
        "\n",
        "    - If maximizing recall: You're aiming to ensure that as many wines of a specific cultivar are correctly identified as possible. This could be important in scenarios where missing a wine of a particular cultivar is costly, such as in scientific research where accurately identifying the diversity and characteristics of wines from different cultivars is crucial.\n",
        "\n",
        "In this case, it is likely better to <b>maximize for recall</b> because the dataset doesn't include the same amount for each class (particularly, 59 for 1, 71 for 2, and 48 for 3) thus we want the minority classes to be correctly classified as much as possible. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "664ff8ae",
      "metadata": {
        "id": "664ff8ae"
      },
      "source": [
        "## Process Description (4 marks)\n",
        "Please describe the process you used to create your code. Cite any websites or generative AI tools used. You can use the following questions as guidance:\n",
        "1. Where did you source your code?\n",
        "1. In what order did you complete the steps?\n",
        "1. If you used generative AI, what prompts did you use? Did you need to modify the code at all? Why or why not?\n",
        "1. Did you have any challenges? If yes, what were they? If not, what helped you to be successful?"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d0e837da",
      "metadata": {
        "id": "d0e837da"
      },
      "source": [
        "<font color='Green'><b>DESCRIBE YOUR PROCESS HERE</b></font>\n",
        "\n",
        "Similar to the first part, I used a combination of resources. For this one specifically, I primarily used the sklearn Decision Tree Classifier and SVC  class documentation, as well as the code from the previous part and previous labs. I also used ChatGPT to generate the code, especially around the confusion matrix. For the generative AI portion of my code, my prompts were quite basic, they followed the instructions pretty closely, but I had to describe to it the purpose of certain things for the AI to understand. I completed the steps pretty much in order but I found it helpful to go to the first section actually and compare and contrast what was being done there and then update the evaluate_model function I had made slightly so that I could reuse it with these new models.\n",
        "\n",
        "Most of the challenges surrounding the second part came from interpreting what it meant to have high precision and recall, and the combination of those values can maximize a certain desired outcome. In addition, I also found it a little challenging to understand how to read the confusion matrix for a non-binary problem. I used research, notes, and ChatGPT to help me understand how to properly interpret the results.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4cd7358d",
      "metadata": {
        "id": "4cd7358d"
      },
      "source": [
        "# **Part 3: Observations/Interpretation (3 marks)**\n",
        "\n",
        "Describe any pattern you see in the results. Relate your findings to what we discussed during lectures. Include data to justify your findings."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "F3ifv218XL62",
      "metadata": {
        "id": "F3ifv218XL62"
      },
      "source": [
        "<font color='Green'><b>\n",
        "ADD YOUR FINDINGS HERE\n",
        "</b></font>\n",
        "\n",
        "One of the key topics we've discussed in lectures and classes is the sensitivity of Machine Learning models to certain conditions for optimal performance. This includes the necessity of clean data, properly set hyperparameters, and the importance of trial and error to achieve the most ideal results. This assignment has reinforced this concept. The approach involves understanding the dataset and precisely identifying what we aim to predict. Then, by applying different models to the dataset and employing complex metrics and cross-validation, we determine which model performs best under specific parameters, ultimately selecting the most suitable one. Over time, and even within this assignment, it's expected that data scientists will develop a clearer intuition about which approaches are more likely to succeed.\n",
        "\n",
        "In the first part, there was a clear demonstration of the lesson that a model's performance on training data doesn't necessarily predict its success on test data, highlighting the significance of being mindful of underfitting and overfitting. For instance, the gradient booster exhibited a nearly perfect R2 score at 98.6% on training data, but its accuracy on test data dramatically fell to 47.4%, indicating overfitting and a failure to generalize to unseen data. We used 4 different models and compared and contrasted many metrics to understand which model would work better. Concluding that the Linear Regression from assignment 1 was much better considering its validation accuracy was 62% compared to the Decision Trees 18%, the Random Forests 17% and the Gradient Boosteds 47%. Thus it's important to test multiple different methods to get the most ideal results.\n",
        "\n",
        "The second part was also a testament to the pattern explained above. We used three different methods on the wine dataset: SVC, Decision Tree, and LinearSVC, and saw that each model performed incrementally better, with SVC there was a test accuracy of 66%, then DT with 89%, and finally with LSVC with 92%. Additionally, this was a practical application of the metrics we've learned in class. A high precision score signifies that the model is highly accurate in its positive predictions, indicating a high likelihood of correctness when it predicts a positive class. Conversely, a high recall score suggests that the model is effective at identifying positive instances, capturing a significant portion of the actual positives.\n",
        "\n",
        "It is also important to notice that as seen above without propely configuration you will overfit your data.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cd97b6ac",
      "metadata": {
        "id": "cd97b6ac"
      },
      "source": [
        "## **Part 4:** Reflection (2 marks)\n",
        "Include a sentence or two about:\n",
        "- what you liked or disliked,\n",
        "- found interesting, confusing, challangeing, motivating\n",
        "while working on this assignment."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "tDFYc89YXQGJ",
      "metadata": {
        "id": "tDFYc89YXQGJ"
      },
      "source": [
        "<font color='Green'><b>\n",
        "ADD YOUR THOUGHTS HERE\n",
        "</b></font>\n",
        "\n",
        "In this assignment, comparing the different models was quite enlightening. It helped me see why certain models work better for specific tasks by putting their performances next to each other. This comparison made it easier to understand the strengths and weaknesses of each model in a more practical context, rather than just looking at numbers without comparison. However, I found the concepts of precision and recall more challenging because they involve understanding the trade-off between correctly identifying true positives and avoiding false positives. This balance is not always straightforward and requires a deeper look into how the models make their predictions, which can be complex to grasp fully."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fa21e53b",
      "metadata": {
        "id": "fa21e53b"
      },
      "source": [
        "## **Part 5:** Bonus Question (3 marks)\n",
        "\n",
        "Repeat Part 2 and compare the support vector machines model used to `LinearSVC(max_iter=5000)`. Does using `LinearSVC` improve the results? Why or why not?\n",
        "\n",
        "Is `LinearSVC` a good fit for this dataset? Why or why not?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "id": "30fea72e",
      "metadata": {
        "id": "30fea72e"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\utils\\validation.py:1229: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
            "  y = column_or_1d(y, warn=True)\n",
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "     Training Score Validation Score\n",
            "SVC        0.703743         0.663492\n",
            "DT         0.974756         0.893175\n",
            "LSVC       0.880725         0.921746\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "c:\\Users\\aarsh\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\sklearn\\svm\\_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAn4AAAIhCAYAAADQCLdCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABGI0lEQVR4nO3deXyM9/r/8XciIojYYo3WHmLfY8up2rfW1tZWtLQUlZK0JVTs+1JbLSVKS+1KbV3oUW0PsZRStSS2RqMaJSQViZj5/eFnvh2hTWTGjLlfz/O4H8d87ns+c81kUpfrs9xuZrPZLAAAALg8d0cHAAAAgMeDxA8AAMAgSPwAAAAMgsQPAADAIEj8AAAADILEDwAAwCBI/AAAAAyCxA8AAMAgSPwAOAR7xwPA40fiB5d37NgxvfPOO2rUqJGqVKmipk2bauTIkYqJibHbay5btkwNGjRQlSpVNH/+fJv0GRkZqXLlyikyMtIm/aXntcqVK6fvv//+gdecOXPGcs3FixfT3XdKSoomTpyoLVu2/Ou15cqV09y5c9Pd94Ns27ZNzz77rCpVqqTw8PBM9fUg6YmxcePGGjZsmM1f2x6++eYb9erVS7Vq1VLlypXVrFkzTZgwQX/++aflmhYtWqhNmzYP7SMlJUWBgYF69913rdod8bsIwBqJH1zaypUr1aVLF/35558KDQ3V4sWL1bdvX+3fv18vvPCCTp48afPXTExM1JQpU1SlShVFRESoQ4cONum3YsWKWrNmjSpWrGiT/tLD3d1dX3zxxQPPbd++/ZH6/OOPP7R8+XKlpqb+67Vr1qzRiy+++Eivc8/YsWNVsGBBRUREqHfv3pnq61HNmzdPAwYMcMhrZ8Rnn32mAQMGqGTJkpo2bZoWL16sXr166csvv1Tnzp11/fp1SVLHjh0VHR2tEydOPLCf3bt3Kz4+3upn54jfRQAPYAZc1MGDB80BAQHm8ePHpzn3559/moOCgswdOnSw+etevHjR7O/vb16/fr3N+35c9u3bZ/b39zd369bNXKdOHfPt27fTXNOyZUtzu3btzP7+/uaYmJh09x0TE2P29/c3b9iwwZYhP5S/v7959uzZdu1/zpw5duv/cWrSpIk5JCQkTfuJEyfM/v7+5sWLF5vNZrP5999/NwcEBJgnT578wH7eeOMNc/PmzS2PHfW7CCAtKn5wWREREcqVK5dCQkLSnMuXL5+GDRumJk2a6ObNm5KkO3fuaOXKlXruuedUpUoVNWrUSNOnT1dycrLlecOGDdMrr7yiDRs2qEWLFqpUqZLatWunPXv2SJI2btyoxo0bS5KGDx+ucuXKSXrwUN/GjRuthklv3bql0aNH6z//+Y8qVaqkli1bKiIiwnL9g4Z6jx07pj59+igwMFA1atTQG2+8oaioqDTP2bt3r3r37q2qVauqQYMGmjZtmu7cufOvn2Hr1q0VHx+vffv2WbWfPHlS58+fV6tWrdI8Z+fOnerWrZuqV69ueR8rV66UJF28eFFNmjSRJIWFhVk+q2HDhqlXr14aNWqUatSoodatW+vOnTtWw6hvvvmmKleurLNnz1pea+7cuQoICND+/fvTxHHvvUvSBx98YPVZ//DDD+rWrZtq1qypwMBAhYaG6tKlS1Y/mwoVKmjdunVq0KCB6tSpo+jo6H/9vB7m7z//ixcvqly5ctqxY4eCg4NVvXp11alTR++9957lu3jPunXr1KZNG1WqVEmNGjXS3Llz0/zc1q1bp44dO6patWqqUqWK2rVrpx07djzSe7ly5coD516WL19eYWFhqlSpkiSpUKFCCgoK0rZt22QymayuvXr1qr777jt16tTJ0pbR30UA9kPiB5dkNpv1/fffq169esqePfsDr2ndurUGDhyoHDlySJLCw8M1adIkNW3aVAsWLFD37t21YsUKDRgwwOovw59//lkREREKDg7WBx98oCxZsmjQoEG6fv26GjVqpHnz5kmS+vfvrzVr1qQ75okTJ2rPnj0aOnSoIiIi1KRJE02dOlUbNmx44PX79u1T165dLc8dP368Ll26pC5duujMmTNW17799tuqWbOmFi5cqLZt22rJkiVat27dv8ZUpkwZlS1bNs1w77Zt21SnTh0VKFDAqn337t0aOHCgKlasqPnz52vu3Ll66qmnNHbsWP30008qWLCg1edz78+SdPDgQV26dEkffPCBQkNDlSVLFqu+R48erRw5cmjUqFGS7v4cFi5cqN69e6tOnTppYr83NC5JL7zwgtasWaOCBQtq06ZN6t27t4oUKaKZM2cqLCxMhw8fVufOna3msd25c0dLly7VhAkTFBYWptKlS//r55URo0aNkp+fn+bPn68+ffpo/fr1WrBggeX8okWLNHLkSNWrV08LFy5U9+7dtXjxYo0cOdJyzcqVKxUeHq6mTZtq0aJFmj59ujw9PfX222/r999/z/B7adSokbZt26aBAwdq69atunz5suXcK6+8orp161oed+rUSZcvX06TdG/dulVms9kyxeFRfhcB2I+HowMA7OHatWtKTk5WsWLF0nV9dHS01q9fr9DQUPXt21eS1KBBAxUsWFDvvvuu9uzZo2eeeUaSlJCQoI0bN+rpp5+WJOXIkUMvv/yy9u3bpxYtWiggIECS9PTTT6tatWrpjnn//v1q0KCBZdJ8YGCgcuTIofz58z/w+hkzZqh48eL68MMPLUlSw4YN1axZM82ZM0ezZ8+2XPviiy9q4MCBkqR69epp586d2r17t7p06fKvcbVq1Uoff/yxRo8eLQ+Pu//J2L59u954440010ZHR6tDhw4aMWKEpa169eoKDAxUZGSkqlatavX5VKhQwXJdamqqxo4dq8KFCz8wDl9fX40aNUpDhgzRunXrtHz5cvn7++utt9564PXe3t6Wz79w4cKqVq2aTCaTpk+froYNG2rGjBmWa+9VGSMiIqwWJLzxxhtq1KjRv35Gj+KZZ57R0KFDJd39mfzwww/avXu3QkNDlZCQoPnz56tz58567733JN392ebJk0fvvfeeXn31VZUtW1YxMTHq06eP1fxBPz8/dezYUYcOHbJagJGe9zJu3DiZTCZ99dVX2rlzp6S7P6cmTZro1VdfVaFChSzXPvvss8qXL5+2bNlilRBu2rRJ//nPfyz/KMjo7yIA+6LiB5d0LxFKz3CmJEvV4v6Vim3atFGWLFmshlfz5ctnSfokWRKVpKSkTMUcGBiotWvX6vXXX9eKFSsUExOjgQMHPvAv65s3b+rYsWNq1aqVVWXMx8dHzz77bJoqTPXq1a0eFy5cON3DavcP9/7000+6fPmymjdvnuba1157TZMnT9Zff/2ln3/+Wdu3b9eiRYsk3V3p+U/y5Mnz0KTv77G0aNFC4eHhiomJsVS40uvcuXOKi4tT27ZtrdqffvppVa9ePc3ndi9JtYf7/1Hw95/J4cOHdevWLTVu3FipqamW497Q+A8//CDp7hD522+/rRs3bujIkSPavHmzZVj9/s87Pe8lV65cmjNnjnbu3Knw8HC1aNFCN27c0EcffaSWLVvq8OHDlmuzZs2qdu3a6auvvrK8VlRUlI4fP64XXnjBcl1GfxcB2BeJH1xS7ty5lTNnTsXGxj70mps3b1pWKd77//uHLj08PJQ3b14lJCRY2u4frnJzc5OkNHOdMmrEiBEaPHiwLl68qHHjxqlp06bq0qXLA1c7JiQkyGw2y9fXN805X19fq3glycvLy+qxu7t7uvfRK1mypAICAizDvdu3b1fDhg2VO3fuNNdevXpVgwYNUq1atfTSSy9p7ty5SkxMlPTv+/blzJkzXfF06NBBJpNJJUqUUMmSJdP1nHvi4+MlKd2fmz2HHu//Hv39Z3Ivzr59+6pixYqWo379+pLuroyWpF9//VWvvPKKateurZdfflkRERGW1dL3f94ZeS/FihVT9+7dNWfOHO3du1dz586Vm5ubxo0bZ3Vdp06ddOPGDe3evVvS3VXBBQoUsPrHSkZ/FwHYF4kfXFbDhg0VGRlptTjj79auXau6devq+PHjliQmLi7O6prbt2/r2rVryps3b6bjub/icX/FzdPTU/3799eOHTv03//+11LVCg0NTdNXrly55ObmpitXrqQ5FxcXpzx58mQ63r9r3bq1vv76a92+fVtffPHFQ/dwe/vtt3Xs2DEtW7ZMR44c0Y4dOzR8+HCbxZGUlKRJkybJ399fp0+f1tKlSzP0/Hufy8M+N1v8nG3Bx8dHkjR9+nStX78+zdGjRw+ZTCb17dtXf/75p9avX68jR47o888/t0xVyKgvv/xSdevW1blz56za3d3d1bx5c3Xq1CnN3NGyZcuqatWq2rp1q0wmk7Zs2aIOHTqkmZ+Zkd9FAPZF4geX1bt3b8XHx2vWrFlpzsXFxWnp0qUqU6aMKlasaFkcsG3bNqvrtm3bpjt37qhmzZqZisXb29tqsr0kHTp0yPLnW7duqUWLFpZEpmjRourevbvatGnzwEpJjhw5VKlSJe3YscMqoUxISNDu3bszHe/9WrVqpfj4eC1cuFDXr1+3rMy936FDh9S8eXMFBgZahmDvrXi+VxG9PynIiBkzZuj333/X3Llz9fLLL2vOnDlpkpF/UrJkSRUoUEBbt261ao+JidGRI0dUo0aNR47NlqpWraqsWbPq8uXLqly5suXw8PDQzJkzdfHiRV27dk3nzp3TCy+8YDknpf2806ts2bKKj4/X8uXLH3j+/Pnz8vf3T9PeqVMn7dmzR99//73++OMPq9W892TkdxGAfbG4Ay6rWrVqeuuttzRr1iydOXNG7du3V968eRUVFaWIiAglJydb/iIqU6aMOnTooDlz5igpKUm1a9fWiRMnNG/ePAUGBiooKChTsTz77LNatGiRFi1apKpVq+qbb76x2iLFy8tLFStW1Lx585Q1a1aVK1dO586d02effaYWLVo8sM/Q0FD16dNHffv2Vbdu3XT79m19+OGHSklJsSzksJWnnnpKlStX1qJFi9SsWbOHDhtWqVJFW7ZsUcWKFVW4cGH9+OOP+vDDD+Xm5maZA5krVy5J0t69e1W6dGlVrVo1XTHs379fK1as0JAhQ1SiRAkNHjxYX3/9tYYNG6bVq1enK6F0d3dXSEiIwsLCFBoaqueff17Xrl3TvHnzlDt3br366qvp/ESsHTlyRMuWLUvTHhQU9EirgfPmzavXXntNs2fPVmJiogIDA3X58mXNnj1bbm5uKl++vHLlyiU/Pz+tXLlShQsXlo+Pj7777jt9/PHHkjI+57RUqVLq27evFi1apNjYWD3//PMqXLiw/vzzT23evFl79+7VRx99lOZ5bdq00aRJkzRhwgTVqVNHJUqUSHNNRn4XAdgXiR9cWv/+/VWhQgWtXLlSEydO1PXr11WkSBE1atRIb7zxhooUKWK5dsKECSpevLg2bNigxYsXq2DBgurZs6cGDBggd/fMFcf79eunq1evKiIiQrdv31ajRo00YcIE9e/f33LN2LFjNWvWLC1dulRxcXHKnz+/XnjhhYeuWq1Xr54++ugjzZkzRyEhIfL09FStWrU0ZcoUlS1bNlPxPkjr1q117Nixf7xV1+TJkzVu3DjLXLASJUpozJgx+vzzz3Xw4EFJd6ufr776qtasWaNvv/3WslDhn9y8eVNhYWHy9/dXnz59JN2dExgeHq7+/ftryZIl6tevX7reR8eOHZUzZ04tWrRIAwcOlLe3t4KCghQSEpJmjmd6ff/99w+8tV3evHkfeRuYwYMHq0CBAvr000+1ZMkS5c6dW/Xq1VNISIgleZ4/f74mTJigYcOGydPTU2XKlNGCBQs0ceJEHTx4UD169MjQa4aEhCggIEDr1q3T+PHjlZiYKB8fH9WqVUvr169X+fLl0zzH29tbLVq00KZNm6y+z/fLyO8iAPtxM6d3hjcAAACeaMzxAwAAMAgSPwAAAIMg8QMAAHBCKSkpatu2rdVNBGJjY/X666+ratWqatasmbZv356hPkn8AAAAnExycrJCQkIUFRVlaUtNTVW/fv3k4eGhzz77TH369NG7776r06dPp7tfVvUCAAA4kejoaIWGhqa5A8+3336rS5cuadWqVfL29lapUqW0Z88eHT58+IH7bD4IiR8AAIAT2b9/vwIDAzVkyBCr+3rv379f9erVk7e3t6Vt/vz5GeqbxA8AAMCJdOvW7YHtMTEx8vPz0/Tp07V582blzZtXwcHBatq0abr7dsnEL/n4LkeHAKSRs3pPR4cAWCmQI7ejQwCsXIr/xWGvffvKWbv1ndW3lE36uXnzpj777DO1bt1aCxcuVGRkpIKDg7VmzRpVrlw5XX24ZOIHAADgarJkyaI8efJo9OjRcnd3V8WKFXXw4EGtXbuWxA8AACDdTHccHcG/KliwoNzc3KxuI1qyZEmdOnUq3X2wnQsAAIDZZL/DRqpWraqoqCjdufN/SeqZM2fk5+eX7j5I/AAAAJ4Abdu2lclk0pgxY3ThwgWtXLlS3333nV566aV098FQLwAAgMl2lTl78fb21kcffaTRo0erbdu2Klq0qN5//31VrFgx3X2Q+AEAADip++fvlSlTRitWrHjk/kj8AACA4ZltOBfPmTHHDwAAwCCo+AEAADwBc/xsgYofAACAQVDxAwAAMMgcPxI/AACAJ+DOHbbAUC8AAIBBUPEDAAAwyFAvFT8AAACDoOIHAADAdi4AAABwJVT8AACA4XHLNgAAALgUKn4AAAAGmeNH4gcAAMBQLwAAAFwJFT8AAABu2QYAAABXQsUPAACAOX4AAABwJVT8AAAADLKdCxU/AAAAg6DiBwAAYJA5fiR+AAAADPUCAADAlVDxAwAAhmc2s4EzAAAAXAgVPwAAAIMs7qDiBwAAYBBU/AAAAFjVCwAAAFdCxQ8AAMAgc/xI/AAAAExs5wIAAAAXQsUPAADAIEO9VPwAAAAMgoofAAAA27kAAADAlVDxAwAAYI4fAAAAXAkVPwAAAIPM8SPxAwAAMEjix1AvAACAQVDxAwAAhmc2c8s2AAAAuBAqfgAAAMzxAwAAgKOkpKSobdu2ioyMTHMuISFBQUFB2rhxY4b6pOIHAADgZBs4JycnKzQ0VFFRUQ88P23aNP3xxx8Z7peKHwAAgBOJjo7WSy+9pF9//fWB5w8ePKh9+/apQIECGe6bxA8AAMBkst+RQfv371dgYKDWrFmT5lxKSopGjhyp8PBweXp6ZrhvhnoBAACcaKi3W7duDz23cOFCVahQQQ0bNnykvkn8AAAAngDR0dFavXq1Pv/880fug8QPAADAybdzMZvNeu+99xQcHCxfX99H7ofEDwAAwMnFxsbq8OHDOnXqlKZMmSJJSkpK0qhRo7R9+3YtWbIkXf2Q+AEAADjRHL8HKVSokL766iurth49eqhHjx56/vnn090PiR8AAICT8/DwUPHixdO05c+fX4UKFUp/P7YODAAA4Inj5HP8bIXEDwAAwEmdOnXqoee++eabDPdH4gcAAEDFDwAAwCCcfHGHrXDLNgAAAIOg4gcAAGCQoV4qfgAAAAZB4gcrv1+5pgYvh+rAz6et2nsNn6EqHQekOY5HX3BQpDCaZk3/o73/26Yb8dGKOrVXIUP6OTokwKJI0UI6eWGf6jWs7ehQ8KjMJvsdToShXlj8fuWq3hg7Twk3k6zazWazTl/4TT2fa6Jm9WtYnStZrPDjDBEGFVinhjZvWq6167Zo9OhpatCgjiZPek8eHh6aOu0DR4cHgyvqV1irNixW7tw+jg4F+FckfpDJZNKW3ZGasXyjzOa052N+j9NfSbfUsGZFVS1X8vEHCMMbFR6qI0d+1iuvBkuSvvxqt7Jm9dCwoYM0Z26Ebt265eAIYURubm56sWs7jRr3juTm5uhwkFnM8YNRnL7wm8YtWqXnGgVq4lu90pw/ee6iJKl8yWKPOzRAnp6eeuaZetq0+Qur9g0btsnHJ5caNmBoDY5RoVI5TZk5SutWb9agfsMcHQ6QLk5R8bt27ZpSUlKUPXt2+fhQKn/civjm09YPxqiwb940c/sk6dT5i8rhlU0zlm3UtweP6eatZNWpXE7vvPqCSvql//6AwKMoVeppZcuWTaejzlq1R585L0ny9y+tnbu+c0BkMLrfYi6pfo2WuhR7mbl9rsDJ5uLZi8MSv6+++korVqzQ0aNHlZycbGn38vJSpUqV1KtXLzVt2tRR4RlK7lw5lTtXzoeeP3Xuom7eSpaPdw69P7SfLsX9qYVrt+uVETO0buZwFcyX5/EFC8PJ/f//MZhwI9GqPSHh7mMfn1yPPSZAkuLjrys+/rqjwwAyxCGJ30cffaR58+bptdde05tvvqn8+fPL09NTKSkpunLlig4ePKhhw4bprbfeUo8ePRwRIv5mULfn9Ur7ZqpVsez/bymjauVKq13wWK3c+l8N6dnBofHBtbm7//OMFJNB5uUAsDOD/LfEIYnf0qVLNWXKlAdW9EqXLq3AwECVK1dO48aNI/FzAuUeMLevWGFflSpWWKfO/+aAiGAk12/ckCR531eVvlfpu3494bHHBMAFGSTxc8jijlu3bqlYsX9eKFCoUCElJPAfdEdLvXNHm/+7Tz+dOpvm3K2UFOXL7e2AqGAkZ85cUGpqqsqULmHVfu/xyZNRjz8oAHhCOSTxa9asmYYNG6aDBw8qNTXV6pzJZNKPP/6o4cOHq0WLFo4ID3/jkSWLFq7dppnLP7Nq/+XMr4r5PU61K/k7KDIYRXJysr77LlId2re2au/YsbXi469r/4HDDooMgEsxm+13OBGHDPWOHj1aU6ZMUZ8+fXTnzh3lyZPHMscvPj5eHh4eateuncLCwhwRHu7T/6U2em/uxxo+e5meeyZQsXFXNX/1VpUrUUzPN6rr6PBgABMnzdaXX6zW6lWLtGzZatWrV0uhIf01fMREJSWxhx8ApJdDEj9PT0+NHDlSb7/9tk6ePKm4uDglJSUpW7ZsKlSokAICAuTl5eWI0PAAzz9bV9k8s+qjTV/rrSmLlD2bp5oEVtNbL7dTlixsBQn7++/uH/Ri59c1KjxUG9ZH6LffftfQYeP1/qxFjg4NgKswyBw/N7PZyWqQNpB8fJejQwDSyFm9p6NDAKwUyJHb0SEAVi7F/+Kw105aNcpufWfvOsZufWeUU2zgDAAA4FAGqfgxTgcAAGAQVPwAAAC4ZRsAAIBBMNQLAAAAV0LFDwAAwPU2OXkgKn4AAAAGQcUPAACAOX4AAABwJVT8AAAAqPgBAADAlVDxAwAAYANnAAAAYzCb2M4FAAAALoSKHwAAAIs7AAAA4Eqo+AEAABhkcQcVPwAAAIOg4gcAAMCqXgAAALgSKn4AAAAGWdVL4gcAAGCQxI+hXgAAAIOg4gcAAGBmcQcAAABcCBU/AAAA5vgBAADAlZD4AQAAmMz2Ox5RSkqK2rZtq8jISEvbkSNH1KVLF1WvXl0tWrTQunXrMtQniR8AAICTSU5OVkhIiKKioixtcXFxev3111WnTh199tlnCg4O1rhx47R79+5098scPwAAALPzzPGLjo5WaGiozPetNN65c6d8fX0VEhIiSSpRooQiIyO1ZcsWNWrUKF19k/gBAAA40b169+/fr8DAQA0ZMkTVqlWztAcFBSkgICDN9YmJienum8QPAADAiXTr1u2B7cWKFVOxYsUsj//8809t27ZNgwYNSnffJH4AAMDwzE/Ydi63bt3SoEGD5Ovrq86dO6f7eSR+AAAAT5C//vpLAwYM0Pnz5/Xpp58qe/bs6X4uiR8AAIATzfH7J4mJiXrttdf066+/avny5SpRokSGnk/iBwAA8AQwmUx68803dfHiRX3yyScqXbp0hvsg8QMAAHCi7VweZv369YqMjNSCBQvk4+OjuLg4SVLWrFmVJ0+edPVB4gcAAPAE+PLLL2UymdSvXz+r9jp16uiTTz5JVx8kfgAAAE46x+/UqVOWP0dERGS6PxI/AACAJ2w7l0fFvXoBAAAMgoofAACAkw712hoVPwAAAIOg4gcAAPAEbOdiC1T8AAAADIKKHwAAAHP8AAAA4Eqo+AEAAMMzG2QfPxI/AAAAhnoBAADgSqj4AQAAUPEDAACAK6HiBwAAwAbOAAAAcCVU/AAAAJjjBwAAAFdCxQ8AABie2SAVPxI/AAAAgyR+DPUCAAAYBBU/AAAAg9yrl4ofAACAQVDxAwAAYI4fAAAAXAkVPwAAACp+AAAAcCVU/AAAgOGZzVT8AAAA4EKo+AEAABhkjh+JHwAAgEESP4Z6AQAADIKKHwAAMDyzQSp+Lpn4lWgwyNEhAGkkxX7n6BAAK/1qvevoEAA8Zi6Z+AEAAGSIQSp+zPEDAAAwCCp+AAAAJkcH8HhQ8QMAADAIKn4AAMDwWNULAABgFAZJ/BjqBQAAMAgqfgAAACzuAAAAgCuh4gcAAAzPKIs7qPgBAAAYBBU/AAAA5vgBAADAlZD4AQAAwzObzHY7HlVKSoratm2ryMhIS1tMTIxeeeUVVatWTa1bt9b333+foT5J/AAAAEx2PB5BcnKyQkJCFBUVZWkzm80aOHCgfH19tWHDBrVr105vvvmmYmNj090vc/wAAACcSHR0tEJDQ2U2W1cL9+3bp5iYGK1evVo5cuRQ6dKltXfvXm3YsEGDBg1KV99U/AAAgOGZTfY7Mmr//v0KDAzUmjVrrNp/+uknVahQQTly5LC01axZU0eOHEl331T8AAAAnEi3bt0e2B4XF6eCBQtateXPn1+///57uvsm8QMAAHgCtnNJSkqSp6enVZunp6dSUlLS3QdDvQAAAE+AbNmypUnyUlJS5OXlle4+qPgBAADDe5S5eI9boUKFFB0dbdV25cqVNMO//4SKHwAAwBOgatWqOn78uG7dumVpO3TokKpWrZruPkj8AAAAnGwfvwepU6eOihQporCwMEVFRenDDz/U0aNH9cILL6S7DxI/AABgeM60ncvDZMmSRfPnz1dcXJw6duyozz//XB988IGKFi2a7j6Y4wcAAOCkTp06ZfW4ePHiWrFixSP3R+IHAAAM70lY3GELDPUCAAAYBBU/AABgeFT8AAAA4FKo+AEAAJjdHB3BY0HFDwAAwCCo+AEAAMMzyhw/Ej8AAGB4ZhNDvQAAAHAhVPwAAIDhGWWol4ofAACAQVDxAwAAhmdmOxcAAAC4Eip+AADA8JjjBwAAAJdCxQ8AABieUfbxI/EDAACGZzY7OoLHI12J34EDB9LdYe3atR85GAAAANhPuhK/Hj16yM3NTeZ/SYfd3Nx04sQJmwQGAADwuDDU+ze7du2ydxwAAACws3Qlfn5+fmnaUlJSdPHiRT399NMym83KmjWrzYMDAAB4HIxS8cvwdi5ms1nTp09X7dq11bZtW126dElDhw7ViBEjdPv2bXvECAAAABvIcOL3ySefaPPmzRo1apQ8PT0lSU2bNtXOnTs1b948mwcIAABgb2az/Q5nkuHEb82aNQoPD1fHjh3l5na3LNq6dWuNHz9eW7ZssXmAAAAAsI0M7+N38eJFBQQEpGkvX7684uLibBIUAADA48Qcv4fw8/PTsWPH0rTv2bNHTz31lE2CAgAAeJzMZje7Hc4kwxW/Pn36aMyYMYqLi5PZbNbevXu1Zs0affLJJxo2bJg9YgQAAIANZDjx69Spk1JTU7VgwQLdunVL4eHhypcvnwYPHqyuXbvaI0YAAAC7MpscHcHj8Uj36u3cubM6d+6sq1evymw2K3/+/LaOCwAAADb2SIlfXFycPv30U0VFRcnT01P+/v7q1q2bfHx8bB0fAACA3ZmcbC6evWR4cUdkZKSaNWumzZs3y83NTbdu3dKnn36q5s2b6+TJk/aIEQAAADaQ4Yrf1KlT9dxzz2n06NHKkiWLpLu3bxs2bJjGjx+vFStW2DxIAAAAe3K21bf2kuGK3+nTp9W7d29L0idJnp6eGjBggI4ePWrT4AAAAGA7GU78SpYsqdOnT6dpv3Dhgvz8/GwSFAAAwONkNrnZ7XAm6RrqPXDggOXPbdq0UXh4uK5cuaIaNWrI3d1dx48f14wZMzRo0CC7BQoAAGAvznZPXXtxM5v//a2WL19ebm5u+rdL3dzcdOLECZsF96iK5Kng6BCANH6N3uroEAAr/Wq96+gQACtLz6932GufKNvabn0HRG23W98Zla6K365du+wdBwAAgMM425CsvaQr8Uvv3L3k5ORMBQMAAAD7yfB2LteuXdPChQt1+vRp3blzR5JkNpt1+/ZtRUdH6+DBgzYPEgAAwJ7YwPkhxowZo02bNilv3rw6ePCgChUqpL/++ktHjhxR37597REjAAAAbCDDFb+9e/dqypQpatSokU6dOqU+ffqofPnyGjlypKKjo+0RIwAAgF2xgfND/PXXXypXrpwkqVSpUpbbtL388suKjIy0bXQAAACwmQwnfoUKFdJvv/0mSSpRooROnTolScqePbuuX79u2+gAAAAeA7PZfoczyXDi17x5c4WFhenQoUOqX7++PvvsM33xxReaM2eOihcvbo8YAQAAYAMZTvyGDBmiRo0aKTY2VvXq1VPz5s01ePBgffvttxo6dKg9YoQDFSlaSCcv7FO9hrUdHQoM6vc/4lSvxQva/6P1vcAvx13R0NFT1KDVSwps1lGvvRWmE6eZZwzHGLjwHU39fr6jw0AmmMxudjucSYYXd3h6emrEiBGWx2PHjlVISIi8vb3l4ZHh7uDEivoV1qoNi5U7t4+jQ4FBXbocp34hI5SQ+JdV+19/3VSvAe/I0zOrRr07SJ6enlq0bJVeHzxCn328QAV88zkoYhhR3fZBqtkyUFcu/uHoUJAJLO7IgDx58ujw4cNq0qSJLbqDg7m5uemlbu319Z4N8i2Y39HhwIBMJpM2bftaL776pv68Gp/m/CdrN+n6jQRFzJ6s5s8GqVGDQM2dHC7PrFl14PDRtB0CdpKnYF51G91bV2OvODoUuJBLly6pX79+qlGjhho3bqxly5bZrG+blehu3bql2NhYW3UHB6pQqZymzByl5RGrtGf3Pq1ct9DRIcFgTkef09jpc9WlQ1vVrVVNA94ZZXX+q93fq1mjhlaVPd/8+fTN5hWPO1QY3CtT+uv4d0d1OzlF5etWdHQ4yARnWoQxePBgFS1aVBs3blR0dLTefvtt+fn5qVmzZpnu2yYVP7iW32IuqX6Nlho9YqqSkpIcHQ4MqEjhgtq+JkLvBveVl5eX1bnbqak6e+5XlXi6mOZ++LEaPd9N1f7TVq++OVTRZy84KGIYUVDnJipeqZRWhi9xdChwIdevX9eRI0fUv39/lShRQk2bNlVQUJD27t1rk/5J/JBGfPx1XYq97OgwYGC5fXKpcMECDzx3IyFRqXfu6JM1n2n/4Z80ZthgTRsbpmvx1/XKm+/qj7g/H3O0MKL8fr7q8l4vrRi5WInXEhwdDmzAWRZ3eHl5KXv27Nq4caNu376ts2fP6scff1RAQIBN3ieJH4Anyu3bty1/XjRjvJ6pX0fNGjXQgulj9dfNm/p0wxYHRgejeHXqQB3dfViHvuDGBbCtbNmyKTw8XGvWrFHVqlXVqlUr/ec//9GLL75ok/7TNcdv3rx5/3rNhQsZG2I5cOBAuq+tXZutRADclTNHDklS7RpVlCNHdkt7kcIFVar40zoZdcZRocEgGvdsqafKP62RLUPlnuVu/cTN7W5Vxz2Lu8wms8zONGEM6eJMq3rPnDmjZ599Vq+++qqioqI0btw41atXT88//3ym+05X4rdx48Z0dVakSJF0v/DYsWMt9/b9p18QNzc3nThxIt39AnBtubxzKl+e3EpJuZ3mXGpqqrJ5ejogKhhJrdb1lCt/bs06kHZu35Iza7V51t0DeBR79+7V+vXr9e2338rLy0uVK1fW5cuXtWDBgseX+H3zzTeZfqH7bdiwQSEhIbp48aLWrFmjbNmy2fw1ALimoHq1tWvP/3Qt/rry5sktSTp34aLOx1xUp+daOjg6uLqPhy+SV87sVm3Pv/WiSlQupTmvTVH85asOigyZ4SwbLf/8888qXry41cK2ChUqaOFC2+yw4bA5fp6enpo5c6YkadasWY4KA8AT6I1Xu8nNzU19h4zQrj3/0xe79mjgu6NUuGABdXquhaPDg4v7/Wyszh87Y3Ukxico9Xaqzh87o/g/rjk6RDwCsx2PjChYsKAuXLiglJQUS9vZs2dVrFixR31rVhy6uMPT01MzZszQ008/7cgwADxhnvIrohULZ6hggfwKGztdY6bOUfmypbR8/jTlzJnD0eEBwCNr3LixsmbNqvfee0/nzp3TN998o4ULF6pHjx426d/N7IIzUIvkqeDoEIA0fo3e6ugQACv9ar3r6BAAK0vPr3fYa/+vSCe79V3/0oYMXR8dHa0JEybo6NGjypcvn7p3765evXpZFhFlBjfXBQAAcCJlypTRRx99ZJe+M5X4paSkyJMVdAAA4AnnTNu52NMjzfFbtWqVGjdurGrVqikmJkajRo3S/PnzbR0bAAAAbCjDid+WLVs0Y8YMdejQQVmzZpUklS5dWgsXLtTSpUttHiAAAIC9mex4OJMMJ35Lly7ViBEjNGjQILm73316z549LbcXAQAAgHPKcOJ37tw51apVK017YGCgLl26ZJOgAAAAHiez3Ox2OJMMJ36+vr46d+5cmvbDhw+rYMGCNgkKAADgcTKZ7Xc4kwwnfp07d9bYsWO1a9cuSXd3k161apUmTJigjh072jxAAAAA2EaGt3N5/fXXlZCQoJCQECUnJ6tfv37y8PBQly5d9MYbb9gjRgAAALsyOdmQrL080j5+ISEh6t+/v6Kjo2U2m1WqVCl5e3vbOjYAAADYUIYTv9jYWMuf8+fPL0m6ceOGbty4IUkqWrSojUIDAAB4PJxtEYa9ZDjxa9y48T/eK+7EiROZCggAAAD2keHE7+OPP7Z6fOfOHZ07d07Lli3TsGHDbBYYAADA4+JsGy3bS4YTvzp16qRpq1evnp566inNnTtXjRs3tklgAAAAsK1HWtzxICVKlNDJkydt1R0AAMBjwxy/h/j74o57EhMTtWjRIhUrVswmQQEAADxODPU+xIMWd5jNZuXIkUPTpk2zWWAAAACwrUwv7pCkrFmzyt/fXzlz5rRJUAAAAI8TFb+H+PjjjzVkyBCVLl3aHvEAAADATjKc+O3bt0/ZsmWzRywAAAAOYZTFHe4ZfUKHDh00ffp0RUVFKSUlxR4xAQAAwA4yXPH79ttv9euvv+rLL7984Hnu3AEAAJ40JmMU/DKe+PXv398ecQAAAMDO0pX4BQQE6Pvvv1f+/PnVoUMHe8cEAADwWJkMMscvXYmf2Wy2dxwAAAAOY5RMJ8OLOwAAAPBkSvccvx07dsjb2/tfr2vfvn1m4gEAAHjs2MD5PuPHj//Xa9zc3Ej8AAAAnFS6E78ffvhB+fPnt2csAAAADmFyM8bijnTN8XMzyIcBAADgyljVCwAADM8omU66Kn4dOnTg/rwAAABPuHRV/CZNmmTvOAAAAByGVb0AAAAGYZR79bKBMwAAgEFQ8QMAAIZnlHv1UvEDAAAwCCp+AADA8NjOBQAAAC6Fih8AADA8VvUCAADApVDxAwAAhscGzgAAAAbB4g4AAAC4FCp+AADA8FjcAQAAAJdCxQ8AABieURZ3UPEDAABwIikpKRozZoxq166t+vXra+bMmTKbbbP8hIofAAAwPGeq+I0fP16RkZGKiIjQX3/9pSFDhqho0aLq0qVLpvum4gcAAOAk4uPjtWHDBo0bN05VqlRRvXr11Lt3b/3000826Z+KHwAAMDyzk6zqPXTokLy9vVWnTh1LW9++fW3WPxU/AABgeCY7HhkRExMjPz8/bdq0SS1btlSTJk30wQcfyGSyzWA0FT8AAAAncfPmTV24cEGrV6/WpEmTFBcXp/DwcGXPnl29e/fOdP8kfgAAwPCcZXGHh4eHEhMTNWPGDPn5+UmSYmNjtWrVKpskfgz1AgAAOIkCBQooW7ZslqRPkkqWLKlLly7ZpH8SPwAAYHhmOx4ZUbVqVSUnJ+vcuXOWtrNnz1olgplB4gcAAOAkSpUqpUaNGiksLEwnT57Ud999pw8//FBdu3a1Sf/M8QMAAIZncpLtXCRp+vTpGjdunLp27ars2bOre/fu6tGjh036JvEDAABwIrly5dLUqVPt0jeJHwAAMDxnWdVrbyR+AADA8IyS+LG4AwAAwCCo+AEAAMPL6LYrTyoqfgAAAAZBxQ8AABieM23nYk9U/AAAAAyCih8AADA8VvUCAADApVDxAwAAhseqXgAAALgUKn4AAMDwTAap+blk4hd387qjQwDSyF40yNEhAFb2Fazt6BAAp8HiDgAAALgUl6z4AQAAZIQxBnqp+AEAABgGFT8AAGB4zPEDAACAS6HiBwAADM/k5ugIHg8qfgAAAAZBxQ8AABgeGzgDAAAYhDHSPoZ6AQAADIOKHwAAMDy2cwEAAIBLoeIHAAAMzyiLO6j4AQAAGAQVPwAAYHjGqPdR8QMAADAMKn4AAMDwjLKql8QPAAAYHos7AAAA4FKo+AEAAMMzRr2Pih8AAIBhUPEDAACGZ5TFHVT8AAAADIKKHwAAMDyzQWb5UfEDAAAwCCp+AADA8Iwyx4/EDwAAGB4bOAMAAMClUPEDAACGZ4x6HxU/AAAAw6DiBwAADI85fgAAAHApJH4AAMDwTHY8HlXfvn01bNiwTPSQFokfAACAk9m2bZu+/fZbm/fLHD8AAGB4znTLtvj4eE2dOlWVK1e2ed8kfgAAwPCc6c4dU6ZMUbt27fTHH3/YvG+GegEAAJzE3r17dfDgQQ0YMMAu/VPxAwAAhucMQ73JyckaNWqUwsPD5eXlZZfXoOIHAADgBObNm6dKlSopKCjIbq9BxQ8AABieM8zx27Ztm65cuaLq1atLklJSUiRJX375pQ4fPmyT1yDxAwAAcAKffPKJUlNTLY+nT58uSXr77bdt9hokfgAAwPBMZsfP8fPz87N6nDNnTklS8eLFbfYazPEDAAAwCCp+AADA8Bxf70tr8uTJNu+TxA8AABieySlTP9tjqBcAAMAgqPgBAADDc4YNnB8HKn4AAAAGQcUPAAAYnjNs4Pw4UPEDAAAwCCp+AADA8FjVCwAAAJdCxQ8AABieUVb1kvgBAADDY3EHAAAAXAoVPwAAYHhmszGGeqn4AQAAGAQVPwAAYHhs5wIAAACXQsUPAAAYHqt6AQAA4FKo+AEAAMNjA2cAAACDYHEHAAAAXAoVPwAAYHhs4AwAAACXQsUPAAAYHtu5AAAAwKVQ8QMAAIZnlO1cqPjhgZo1/Y/2/m+bbsRHK+rUXoUM6efokGBwfCfhjHy7NVPFXXNU/fRqVfzvXBXo1crRIQH/iMQPaQTWqaHNm5br1KkzevGl17Rq9WeaPOk9vfvOQEeHBoPiOwln5Nu1qUpMHagbPxxV9KsTdW3LD3p63Osq1Ledo0PDIzDJbLfDmbiZXXD9soenn6NDeKJt37pSefL4qH7D5yxtkyYOV7++PVW0WDXdunXLgdHBiPhO2se+grUdHcITrfymyTLfMelUp+GWtpLzQuRdo5yO1aci/ShqXdzksNdu+lQLu/W9M+ZLu/WdUVT8YMXT01PPPFNPmzZ/YdW+YcM2+fjkUsMG/EWBx4vvJJyVW7asupN406rtTnyCPPLmclBEyAyz2Wy3w5mQ+MFKqVJPK1u2bDodddaqPfrMeUmSv39pB0QFI+M7CWf1R8RW5X6muvJ1fEZZcuWQzzPVlP+Fxvpzw25Hh4ZHYJShXoes6k1JSdHs2bO1detWJSQkqH79+hoyZIhKl/6//4BfuXJFQUFBOnHihCNCNKzcPj6SpIQbiVbtCQl3H/v48C9ZPF58J+Gsrm7+TrnqVVKpOUMsbdd3/6iY0REOjAr4Zw6p+M2cOVM7d+7Uu+++q7Fjx+rKlSvq1KmTdu7caXWds5VHjcDd/Z+/EiaTUba4hLPgOwlnVWZpmPK2qa+Y8ct08oURuvDeh8pRpYxKLXzX0aHhEZjt+D9n4pCK344dOzRz5kzVrFlTktSmTRtNnTpVgwcP1rRp09Sq1d3l8G5ubo4Iz9Cu37ghSfLOldOq/V5V5fr1hMceE4yN7yScUc6a5ZT72Zo6/848XVl1t2iRuO+4Un69rLIfj1TuJrV0fddBB0cJpOWQxO/WrVvKkyeP5bGbm5uGDh0qd3d3vfPOO/Lw8FD16tUdEZrhnTlzQampqSpTuoRV+73HJ09GPf6gYGh8J+GMshUrKElKPHDSqj1h33FJUvZyT5H4PWFMBhlldMhQb2BgoKZOnaqrV69atb/zzjvq3LmzhgwZok8//dQRoRlecnKyvvsuUh3at7Zq79ixteLjr2v/gcMOigxGxXcSzigp+qIkyTuwglW7d+3ykqTkC5cfe0xAejik4jdixAgFBwerQYMGWrJkiRo0aGA5N3LkSOXNm1cLFixwRGiQNHHSbH35xWqtXrVIy5atVr16tRQa0l/DR0xUUhL7peHx4zsJZ5N0/Jyubvufngp/VR65c+qvw1Hy8n9KRUO66K+fonXti32ODhEZZIx6n4M3cD579qwKFCigXLnSrso7c+aMdu3apb59+2a4XzZwzrx27VpqVHioyvmX1m+//a4FC5fr/VmLHB0WDIzvpO2xgXPmuGX1UJG3XlT+jo2UtVA+pcTGKf6LSMW+v0amm/yD5FE4cgPnIL8mduv7u9922a3vjOLOHQBgUCR+cDaOTPwa+DW2W98//PaN3frOKIcM9QIAADgTZ9to2V64cwcAAIBBUPEDAACG54Iz3x6Iih8AAIBBUPEDAACGxxw/AAAAuBQqfgAAwPDMVPwAAADgSkj8AACA4ZnNZrsdGXX58mUFBwerTp06CgoK0qRJk5ScnGyT98lQLwAAMDxnWdxhNpsVHBwsHx8frVy5UtevX9fw4cPl7u6uoUOHZrp/Kn4AAABO4uzZszpy5IgmTZqksmXLqlatWgoODtbWrVtt0j8VPwAAYHjOsoFzgQIFtGTJEvn6+lq1JyYm2qR/Kn4AAABOwsfHR0FBQZbHJpNJK1asUN26dW3SPxU/AABgeM4yx+9+06ZN0y+//KL169fbpD8SPwAAACc0bdo0LV++XO+//778/f1t0ieJHwAAMDxn28B53LhxWrVqlaZNm6YWLVrYrF8SPwAAACcyb948rV69WjNnzlTLli1t2jeJHwAAMDyTk6zqPXPmjObPn6++ffuqZs2aiouLs5wrUKBApvsn8QMAAIbnLEO9u3bt0p07d7RgwQItWLDA6typU6cy3b+b2Vk2rrEhD08/R4cAAE5vX8Hajg4BsFLr4iaHvXbFQoF26/v45Ui79Z1RVPwAAIDhOctQr72xgTMAAIBBUPEDAACG5yxz/OyNih8AAIBBUPEDAACGxxw/AAAAuBQqfgAAwPCMMsePxA8AABgeQ70AAABwKVT8AACA4RllqJeKHwAAgEFQ8QMAAIZnNpscHcJjQcUPAADAIKj4AQAAwzMxxw8AAACuhIofAAAwPLNB9vEj8QMAAIbHUC8AAABcChU/AABgeEYZ6qXiBwAAYBBU/AAAgOGZqPgBAADAlVDxAwAAhmdmVS8AAABcCRU/AABgeEZZ1UviBwAADI8NnAEAAOBSqPgBAADDM8pQLxU/AAAAg6DiBwAADI8NnAEAAOBSqPgBAADDY44fAAAAXAoVPwAAYHhG2cePxA8AABgeQ70AAABwKVT8AACA4bGdCwAAAFwKFT8AAGB4ZoMs7qDiBwAAYBBU/AAAgOExxw8AAAAuhYofAAAwPPbxAwAAgEuh4gcAAAzPKKt6SfwAAIDhMdQLAACAxy45OVnDhw9XrVq11LBhQy1dutRmfVPxAwAAhudMFb+pU6fq559/1vLlyxUbG6uhQ4eqaNGiatmyZab7JvEDAABwEjdv3tS6deu0ePFiVaxYURUrVlRUVJRWrlxpk8SPoV4AAGB4ZjseGXHy5EmlpqaqevXqlraaNWvqp59+kslketS3Z0HiBwAA4CTi4uKUN29eeXp6Wtp8fX2VnJys+Pj4TPfvkkO9qSm/OToEAADwBHGW3CEpKckq6ZNkeZySkpLp/qn4AQAAOIls2bKlSfDuPfby8sp0/yR+AAAATqJQoUK6du2aUlNTLW1xcXHy8vKSj49Ppvsn8QMAAHASAQEB8vDw0JEjRyxthw4dUuXKleXunvm0jcQPAADASWTPnl3t27fX6NGjdfToUe3cuVNLly5Vz549bdK/m9mZdiwEAAAwuKSkJI0ePVpfffWVvL291adPH73yyis26ZvEDwAAwCAY6gUAADAIEj8AAACDIPEDAAAwCBI/PFBycrKGDx+uWrVqqWHDhlq6dKmjQwIk3d3ItG3btoqMjHR0KDC4y5cvKzg4WHXq1FFQUJAmTZqk5ORkR4cF/COXvGUbMm/q1Kn6+eeftXz5csXGxmro0KEqWrSoWrZs6ejQYGDJyckKDQ1VVFSUo0OBwZnNZgUHB8vHx0crV67U9evXNXz4cLm7u2vo0KGODg94KBI/pHHz5k2tW7dOixcvVsWKFVWxYkVFRUVp5cqVJH5wmOjoaIWGhoqNCOAMzp49qyNHjuiHH36Qr6+vJCk4OFhTpkwh8YNTY6gXaZw8eVKpqamqXr26pa1mzZr66aefZDKZHBgZjGz//v0KDAzUmjVrHB0KoAIFCmjJkiWWpO+exMREB0UEpA8VP6QRFxenvHnzytPT09Lm6+ur5ORkxcfHK1++fA6MDkbVrVs3R4cAWPj4+CgoKMjy2GQyacWKFapbt64DowL+HYkf0khKSrJK+iRZHqekpDgiJABwatOmTdMvv/yi9evXOzoU4B+R+CGNbNmypUnw7j328vJyREgA4LSmTZum5cuX6/3335e/v7+jwwH+EYkf0ihUqJCuXbum1NRUeXjc/YrExcXJy8tLPj4+Do4OAJzHuHHjtGrVKk2bNk0tWrRwdDjAv2JxB9IICAiQh4eHjhw5Ymk7dOiQKleuLHd3vjIAIEnz5s3T6tWrNXPmTLVp08bR4QDpwt/iSCN79uxq3769Ro8eraNHj2rnzp1aunSpevbs6ejQAMApnDlzRvPnz9frr7+umjVrKi4uznIAzoyhXjxQWFiYRo8erV69esnb21uDBg1S8+bNHR0WADiFXbt26c6dO1qwYIEWLFhgde7UqVMOigr4d25mdkMFAAAwBIZ6AQAADILEDwAAwCBI/AAAAAyCxA8AAMAgSPwAAAAMgsQPAADAIEj8AAAADILEDwAAwCBI/AAX17hxY5UrV85ylC9fXjVq1NDLL7+sAwcO2Pz1IiMjVa5cOV28eFGS1KNHDw0bNixdz71586ZWrlyZqde/ePGiypUrp8jIyHTF9yjmzp2rxo0bP/LzbdUHAGQUt2wDDKB3797q3bu3JMlsNis+Pl4zZ87Ua6+9ph07dqho0aJ2e+25c+cqS5Ys6bp26dKl2rhxo7p37263eADAyKj4AQaQI0cOFShQQAUKFFDBggXl7++vMWPG6NatW/r666/t+tp58uRRrly50nUtd5AEAPsi8QMMysPjbsHf09NT0t0h4SlTpqh169YKDAzU/v37ZTabtXjxYjVp0kRVq1ZVu3bt9Pnnn1v1c/DgQb344ouqUqWKnn/+eZ08edLq/P1DvUePHtUrr7yi6tWrq379+ho1apSSkpI0d+5czZs3T7/99pvVUOyGDRvUqlUrValSRa1atdLy5ctlMpks/Z0+fVo9e/ZUtWrV1KxZM+3duzfTn83p06fVr18/1a5dW5UqVVKTJk20dOnSNNd98MEHCgwMVI0aNfT2228rPj7eci4hIUEjR45U3bp1VbNmTfXs2VPHjh3LdGwAkBkM9QIGdPnyZU2cOFE5cuTQM888Y2lfsWKFFi1apFy5cqlcuXJ6//33tXXrVoWHh6tUqVI6cOCARo8erYSEBHXv3l0xMTHq3bu32rdvr8mTJys6Olrh4eEPfd2YmBj16tVLzZo105o1a5SQkKChQ4dqzJgxGjlypG7evKnt27dr/fr1ypcvn9asWaOZM2cqPDxcVapU0S+//KJx48bp8uXLevfdd5WQkGBJItetW6c//vhDI0eOzNRnk5SUpN69e6tBgwZavXq1smTJonXr1mnKlCmqV6+eAgICJEm//fab9u3bp48++kiJiYkaOXKkwsLCtGDBApnNZr3++uvy8vLSokWL5O3trc2bN6tr165au3atKlSokKkYAeBRkfgBBrBo0SJLxSo1NVUpKSkqXbq0Zs2aZTW/75lnnlH9+vUl3V1osWzZMs2cOVONGjWSJD399NP67bffFBERoe7du2vt2rXy9fXVqFGjlCVLFpUuXVqXLl3SpEmTHhjH2rVrlSdPHk2cONFScRw/frwOHz6snDlzKkeOHMqSJYsKFCggSZo/f7769++vNm3aSJKeeuopJSYmasyYMXrrrbe0bds2JSUlafLkycqVK5fKli2r4cOHa+DAgY/8WSUlJalnz57q3r27cubMKUkKDg7WkiVLdOrUKUvily1bNr3//vvy9fWVJIWHh6t37966cOGCYmNjdeTIEe3bt0958uSRJIWEhOjHH3/Uxx9/rMmTJz9yfACQGSR+gAF06dJFPXr0kCS5u7s/dN5d8eLFLX+Ojo5WcnKyQkND5e7+f7NC7iWOt27d0unTp1WhQgWrxRs1atR4aBynT59WxYoVLUmfJNWtW1d169ZNc+3Vq1f1+++/a+bMmZo9e7al3WQyKTk5WRcvXtTp06dVokQJq/dSvXr1f/s4/lG+fPnUrVs3bd26Vb/88ot+/fVXy/D134eYixcvbkn6JKlq1aqSpKioKJ0/f15ms1nPPvusVd8pKSlKTk7OVHwAkBkkfoAB5M6d2yqpexgvLy/Ln+8ttJg1a5ZKlSqV5lpPT0+5ublZJUOSrJK6+/3Tufvd6zcsLMxShfy7IkWKZPj10yMuLk6dO3dWvnz51LhxYzVs2FCVK1e2GhKXlGal8p07dyRJWbNmlclkkre3tzZu3Jim/3tzKgHAEVjcAeCBSpUqJQ8PD8XGxqp48eKW49tvv1VERITc3d1Vvnx5/fzzz0pJSbE87+eff35on2XKlNEvv/xiSZIk6euvv1bjxo2VnJwsNzc3S3v+/PmVL18+xcTEWL3+8ePHNWvWLElS+fLldf78eV29ejVdr58eW7duVXx8vFatWqUBAwaoWbNmun79uiTrVcfnz59XYmKi5fGhQ4fk5uamMmXKyN/fX4mJibp9+7ZV7IsXL9auXbsyFR8AZAaJH4AHypUrl7p06aLZs2dr8+bNiomJ0fr16zVt2jQVLFhQktS1a1clJSVp+PDhOnPmjP773/9q7ty5D+2zW7duunbtmkaNGqUzZ87owIEDmjp1qurWrats2bIpR44cun79us6dO6fU1FS9/vrr+uSTT7RixQr9+uuv+vrrrzV69Gh5eXnJ09NTbdq0Uf78+RUaGqqTJ09q//79mjBhQrre34EDB7Rnzx6r48KFCypcuLCSkpL0xRdfKDY2Vt9//71CQkIkySrBTU5O1uDBg/XLL7/ohx9+0Lhx49S+fXv5+fkpKChIAQEBGjJkiPbt26cLFy5o0qRJ2rhxo0qXLp2JnwoAZA5DvQAeKiwsTHnz5tXs2bP1xx9/qEiRIgoODtZrr70mSSpUqJCWL1+uiRMnqkOHDipSpIj69++vMWPGPLC/QoUKaenSpZo2bZrat2+v3Llzq3Xr1pbEqnnz5lq7dq2ef/55rVixQr1791a2bNn0ySefaPLkyfL19dVLL72k4OBgSXf3J1y+fLnGjRunrl27Knfu3AoODlZYWNi/vrcH3U3kzTff1Jtvvqnjx49r8uTJSkxMlJ+fn1588UXt2rVLx44dU9euXSVJlSpVUkBAgHr27Ck3Nze1bt3a0meWLFks73Pw4MFKSkpS6dKlNW/ePNWrVy/jPwgAsBE3MzumAgAAGAJDvQAAAAZB4gcAAGAQJH4AAAAGQeIHAABgECR+AAAABkHiBwAAYBAkfgAAAAZB4gcAAGAQJH4AAAAGQeIHAABgECR+AAAABvH/AFgn+gOPYPRiAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 800x600 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           1       0.94      0.94      0.94        16\n",
            "           2       1.00      0.76      0.86        21\n",
            "           3       0.62      1.00      0.76         8\n",
            "\n",
            "    accuracy                           0.87        45\n",
            "   macro avg       0.85      0.90      0.85        45\n",
            "weighted avg       0.91      0.87      0.87        45\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# TO DO: ADD YOUR CODE HERE\n",
        "from sklearn.svm import LinearSVC\n",
        "\n",
        "svc_model = LinearSVC(random_state=0, max_iter=5000)    \n",
        "svc_model.fit(X_wine, y_wine)\n",
        "evaluate_model(svc_model, X_wine, y_wine, cv_config, \"LSVC\", True)\n",
        "\n",
        "# Print the results DataFrame with R^2 scores\n",
        "print(results)\n",
        "\n",
        "\n",
        "svc_model = LinearSVC(random_state=0, max_iter=5000)    \n",
        "X_wine_train, X_wine_test, y_wine_train, y_wine_test = train_test_split(X_wine, y_wine, random_state=0)\n",
        "svc_model.fit(X_wine_train, y_wine_train)\n",
        "\n",
        "y_wine_pred = svc_model.predict(X_wine_test)\n",
        "\n",
        "# Generate confusion matrix\n",
        "cm = confusion_matrix(y_wine_test, y_wine_pred)\n",
        "    \n",
        "# Plot the confusion matrix using a heatmap\n",
        "plt.figure(figsize=(8,6))\n",
        "sns.heatmap(cm, annot=True, fmt=\"d\", xticklabels=True, yticklabels=True)\n",
        "plt.xlabel('Predicted Label')\n",
        "plt.ylabel('True Label')\n",
        "plt.title('Confusion Matrix for Linear SVC')\n",
        "plt.show()\n",
        "\n",
        "report = classification_report(y_wine_test, y_wine_pred)\n",
        "print(report)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aabc68a4",
      "metadata": {
        "id": "aabc68a4"
      },
      "source": [
        "*ANSWER HERE*\n",
        "\n",
        "The LinearSVC actually makes a very good fit for this dataset. Solely comparing the numbers, the training accuracy is 88.1% which is quite good, but it actually out performs the SVC and DT for the validation score (which is what we truly care about) with a score of 92.1% compared to 66% and 89% respectively. In addition looking at the confusion matrix its clear that the middle diagonal is populated the most and just barely under peforms compared to the DT with (15, 16, 8 for the LinearSVC and 14, 20, 8 with DT)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}