hw2_classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "hw2_classification.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Iallen520/lhy_DL_Hw/blob/master/hw2_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OSL2CMEzmQvB",
        "colab_type": "text"
      },
      "source": [
        "#**Homework 2 - Classification**\n",
        "若有任何問題，歡迎來信至助教信箱 ntu-ml-2020spring-ta@googlegroups.com"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ox7joE3aZkh-",
        "colab_type": "text"
      },
      "source": [
        "Binary classification is one of the most fundamental problem in machine learning. In this tutorial, you are going to build linear binary classifiers to predict whether the income of an indivisual exceeds 50,000 or not. We presented a discriminative and a generative approaches, the logistic regression(LR) and the linear discriminant anaysis(LDA). You are encouraged to compare the differences between the two, or explore more methodologies. Although you can finish this tutorial by simpliy copying and pasting the codes, we strongly recommend you to understand the mathematical formulation first to get more insight into the two algorithms. Please find [here](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Logistic%20Regression%20(v3).pdf) and [here](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Classification%20(v3).pdf) for more detailed information about the two algorithms.\n",
        "\n",
        "二元分類是機器學習中最基礎的問題之一，在這份教學中，你將學會如何實作一個線性二元分類器，來根據人們的個人資料，判斷其年收入是否高於 50,000 美元。我們將以兩種方法: logistic regression 與 generative model，來達成以上目的，你可以嘗試了解、分析兩者的設計理念及差別。針對這兩個演算法的理論基礎，可以參考李宏毅老師的教學投影片 [logistic regression](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Logistic%20Regression%20(v3).pdf) 與 [generative model](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Classification%20(v3).pdf)。\n",
        "\n",
        "若有任何問題，歡迎來信至助教信箱 ntu-ml-2020spring-ta@googlegroups.com"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nkNW5cQmohoo",
        "colab_type": "text"
      },
      "source": [
        "#Dataset\n",
        "\n",
        "This dataset is obtained by removing unnecessary attributes and balancing the ratio between positively and negatively labeled data in the [**Census-Income (KDD) Data Set**](https://archive.ics.uci.edu/ml/datasets/Census-Income+(KDD)), which can be found in [**UCI Machine Learning Repository**](https://archive.ics.uci.edu/ml/index.php). Only preprocessed and one-hot encoded data (i.e. *X_train*,  *Y_train* and *X_test*) will be used in this tutorial. Raw data (i.e. *train.csv* and *test.csv*) are provided to you in case you are interested in it.\n",
        "\n",
        "這個資料集是由 [**UCI Machine Learning Repository**](https://archive.ics.uci.edu/ml/index.php) 的 [**Census-Income (KDD) Data Set**](https://archive.ics.uci.edu/ml/datasets/Census-Income+(KDD)) 經過一些處理而得來。為了方便訓練，我們移除了一些不必要的資訊，並且稍微平衡了正負兩種標記的比例。事實上在訓練過程中，只有 X_train、Y_train 和 X_test 這三個經過處理的檔案會被使用到，train.csv 和 test.csv 這兩個原始資料檔則可以提供你一些額外的資訊。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ww4-VJoJqE-_",
        "colab_type": "code",
        "outputId": "398da30b-8e74-4e94-fe37-6e9f2c6af2b2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "!gdown --id '1KSFIRh0-_Vr7SdiSCZP1ItV7bXPxMD92' --output data.tar.gz\n",
        "!tar -zxvf data.tar.gz\n",
        "!ls"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1KSFIRh0-_Vr7SdiSCZP1ItV7bXPxMD92\n",
            "To: /content/data.tar.gz\n",
            "\r0.00B [00:00, ?B/s]\r6.11MB [00:00, 53.5MB/s]\n",
            "data/\n",
            "data/sample_submission.csv\n",
            "data/test_no_label.csv\n",
            "data/train.csv\n",
            "data/X_test\n",
            "data/X_train\n",
            "data/Y_train\n",
            "acc.png  data.tar.gz  output_generative.csv  sample_data\n",
            "data\t loss.png     output_logistic.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WRXI0kf0W4Bd",
        "colab_type": "text"
      },
      "source": [
        "#Logistic Regression\n",
        "\n",
        "In this section we will introduce logistic regression first. We only present how to implement it here, while mathematical formulation and analysis will be omitted. You can find more theoretical detail in [Prof. Lee's lecture](https://www.youtube.com/watch?v=hSXFuypLukA).\n",
        "\n",
        "首先我們會實作 logistic regression，針對理論細節說明請參考[李宏毅老師的教學影片](https://www.youtube.com/watch?v=hSXFuypLukA)\n",
        "\n",
        "###Preparing Data\n",
        "\n",
        "Load and normalize data, and then split training data into training set and development set.\n",
        "\n",
        "下載資料，並且對每個屬性做正規化，處理過後再將其切分為訓練集與發展集。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7NzAmkzU2MAS",
        "colab_type": "code",
        "outputId": "61610be3-295e-4ff8-befe-8044938141c1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "np.random.seed(0)\n",
        "X_train_fpath = './data/X_train'\n",
        "Y_train_fpath = './data/Y_train'\n",
        "X_test_fpath = './data/X_test'\n",
        "output_fpath = './output_{}.csv'\n",
        "\n",
        "# Parse csv files to numpy array\n",
        "with open(X_train_fpath) as f:\n",
        "    next(f)\n",
        "    X_train = np.array([line.strip('\\n').split(',')[1:] for line in f], dtype = float)\n",
        "with open(Y_train_fpath) as f:\n",
        "    next(f)\n",
        "    Y_train = np.array([line.strip('\\n').split(',')[1] for line in f], dtype = float)\n",
        "with open(X_test_fpath) as f:\n",
        "    next(f)\n",
        "    X_test = np.array([line.strip('\\n').split(',')[1:] for line in f], dtype = float)\n",
        "\n",
        "def _normalize(X, train = True, specified_column = None, X_mean = None, X_std = None):\n",
        "    # This function normalizes specific columns of X.\n",
        "    # The mean and standard variance of training data will be reused when processing testing data.\n",
        "    #\n",
        "    # Arguments:\n",
        "    #     X: data to be processed\n",
        "    #     train: 'True' when processing training data, 'False' for testing data\n",
        "    #     specific_column: indexes of the columns that will be normalized. If 'None', all columns\n",
        "    #         will be normalized.\n",
        "    #     X_mean: mean value of training data, used when train = 'False'\n",
        "    #     X_std: standard deviation of training data, used when train = 'False'\n",
        "    # Outputs:\n",
        "    #     X: normalized data\n",
        "    #     X_mean: computed mean value of training data\n",
        "    #     X_std: computed standard deviation of training data\n",
        "\n",
        "    if specified_column == None:\n",
        "        specified_column = np.arange(X.shape[1])\n",
        "    if train:\n",
        "        X_mean = np.mean(X[:, specified_column] ,0).reshape(1, -1)\n",
        "        X_std  = np.std(X[:, specified_column], 0).reshape(1, -1)\n",
        "\n",
        "    X[:,specified_column] = (X[:, specified_column] - X_mean) / (X_std + 1e-8)\n",
        "     \n",
        "    return X, X_mean, X_std\n",
        "\n",
        "def _train_dev_split(X, Y, dev_ratio = 0.25):\n",
        "    # This function spilts data into training set and development set.\n",
        "    train_size = int(len(X) * (1 - dev_ratio))\n",
        "    return X[:train_size], Y[:train_size], X[train_size:], Y[train_size:]\n",
        "\n",
        "# Normalize training and testing data\n",
        "X_train, X_mean, X_std = _normalize(X_train, train = True)\n",
        "X_test, _, _= _normalize(X_test, train = False, specified_column = None, X_mean = X_mean, X_std = X_std)\n",
        "    \n",
        "# Split data into training set and development set\n",
        "dev_ratio = 0.1\n",
        "X_train, Y_train, X_dev, Y_dev = _train_dev_split(X_train, Y_train, dev_ratio = dev_ratio)\n",
        "\n",
        "train_size = X_train.shape[0]\n",
        "dev_size = X_dev.shape[0]\n",
        "test_size = X_test.shape[0]\n",
        "data_dim = X_train.shape[1]\n",
        "print('Size of training set: {}'.format(train_size))\n",
        "print('Size of development set: {}'.format(dev_size))\n",
        "print('Size of testing set: {}'.format(test_size))\n",
        "print('Dimension of data: {}'.format(data_dim))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size of training set: 48830\n",
            "Size of development set: 5426\n",
            "Size of testing set: 27622\n",
            "Dimension of data: 510\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "imgCeBDoApdb",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "###Some Useful Functions\n",
        "\n",
        "Some functions that will be repeatedly used when iteratively updating the parameters.\n",
        "\n",
        "這幾個函數可能會在訓練迴圈中被重複使用到。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hSDAw5LTAs2o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def _shuffle(X, Y):\n",
        "    # This function shuffles two equal-length list/array, X and Y, together.\n",
        "    randomize = np.arange(len(X))\n",
        "    np.random.shuffle(randomize)\n",
        "    return (X[randomize], Y[randomize])\n",
        "\n",
        "def _sigmoid(z):\n",
        "    # Sigmoid function can be used to calculate probability.\n",
        "    # To avoid overflow, minimum/maximum output value is set.\n",
        "    return np.clip(1 / (1.0 + np.exp(-z)), 1e-8, 1 - (1e-8))\n",
        "\n",
        "def _f(X, w, b):\n",
        "    # This is the logistic regression function, parameterized by w and b\n",
        "    #\n",
        "    # Arguements:\n",
        "    #     X: input data, shape = [batch_size, data_dimension]\n",
        "    #     w: weight vector, shape = [data_dimension, ]\n",
        "    #     b: bias, scalar\n",
        "    # Output:\n",
        "    #     predicted probability of each row of X being positively labeled, shape = [batch_size, ]\n",
        "    return _sigmoid(np.matmul(X, w) + b)\n",
        "\n",
        "def _predict(X, w, b):\n",
        "    # This function returns a truth value prediction for each row of X \n",
        "    # by rounding the result of logistic regression function.\n",
        "    return np.round(_f(X, w, b)).astype(np.int)\n",
        "    \n",
        "def _accuracy(Y_pred, Y_label):\n",
        "    # This function calculates prediction accuracy\n",
        "    acc = 1 - np.mean(np.abs(Y_pred - Y_label))\n",
        "    return acc"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OxJdfhEEOYwg",
        "colab_type": "text"
      },
      "source": [
        "### Functions about gradient and loss\n",
        "\n",
        "Please refers to [Prof. Lee's lecture slides](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Logistic%20Regression%20(v3).pdf)(p.12) for the formula of gradient and loss computation.\n",
        "\n",
        "請參考[李宏毅老師上課投影片](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Logistic%20Regression%20(v3).pdf)第 12 頁的梯度及損失函數計算公式。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DqYkUgLjOWi1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def _cross_entropy_loss(y_pred, Y_label):\n",
        "    # This function computes the cross entropy.\n",
        "    #\n",
        "    # Arguements:\n",
        "    #     y_pred: probabilistic predictions, float vector\n",
        "    #     Y_label: ground truth labels, bool vector\n",
        "    # Output:\n",
        "    #     cross entropy, scalar\n",
        "    cross_entropy = -np.dot(Y_label, np.log(y_pred)) - np.dot((1 - Y_label), np.log(1 - y_pred))\n",
        "    return cross_entropy\n",
        "\n",
        "def _gradient(X, Y_label, w, b):\n",
        "    # This function computes the gradient of cross entropy loss with respect to weight w and bias b.\n",
        "    y_pred = _f(X, w, b)\n",
        "    pred_error = Y_label - y_pred\n",
        "    w_grad = -np.sum(pred_error * X.T, 1)\n",
        "    b_grad = -np.sum(pred_error)\n",
        "    return w_grad, b_grad\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XXEFuqydaA34",
        "colab_type": "text"
      },
      "source": [
        "### Training\n",
        "\n",
        "Everything is prepared, let's start training! \n",
        "\n",
        "Mini-batch gradient descent is used here, in which training data are split into several mini-batches and each batch is fed into the model sequentially for losses and gradients computation. Weights and bias are updated on a mini-batch basis.\n",
        "\n",
        "Once we have gone through the whole training set,  the data have to be re-shuffled and mini-batch gradient desent has to be run on it again. We repeat such process until max number of iterations is reached.\n",
        "\n",
        "我們使用小批次梯度下降法來訓練。訓練資料被分為許多小批次，針對每一個小批次，我們分別計算其梯度以及損失，並根據該批次來更新模型的參數。當一次迴圈完成，也就是整個訓練集的所有小批次都被使用過一次以後，我們將所有訓練資料打散並且重新分成新的小批次，進行下一個迴圈，直到事先設定的迴圈數量達成為止。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s6yNUeG9aBR1",
        "colab_type": "code",
        "outputId": "ab2d6731-1d43-4298-b278-5bc4f907b096",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Zero initialization for weights ans bias\n",
        "w = np.zeros((data_dim,)) \n",
        "b = np.zeros((1,))\n",
        "\n",
        "# Some parameters for training    \n",
        "max_iter = 10\n",
        "batch_size = 8\n",
        "learning_rate = 0.2\n",
        "\n",
        "# Keep the loss and accuracy at every iteration for plotting\n",
        "train_loss = []\n",
        "dev_loss = []\n",
        "train_acc = []\n",
        "dev_acc = []\n",
        "\n",
        "# Calcuate the number of parameter updates\n",
        "step = 1\n",
        "\n",
        "# Iterative training\n",
        "for epoch in range(max_iter):\n",
        "    # Random shuffle at the begging of each epoch\n",
        "    X_train, Y_train = _shuffle(X_train, Y_train)\n",
        "        \n",
        "    # Mini-batch training\n",
        "    for idx in range(int(np.floor(train_size / batch_size))):\n",
        "        X = X_train[idx*batch_size:(idx+1)*batch_size]\n",
        "        Y = Y_train[idx*batch_size:(idx+1)*batch_size]\n",
        "\n",
        "        # Compute the gradient\n",
        "        w_grad, b_grad = _gradient(X, Y, w, b)\n",
        "            \n",
        "        # gradient descent update\n",
        "        # learning rate decay with time\n",
        "        w = w - learning_rate/np.sqrt(step) * w_grad\n",
        "        b = b - learning_rate/np.sqrt(step) * b_grad\n",
        "\n",
        "        step = step + 1\n",
        "            \n",
        "    # Compute loss and accuracy of training set and development set\n",
        "    y_train_pred = _f(X_train, w, b)\n",
        "    Y_train_pred = np.round(y_train_pred)\n",
        "    train_acc.append(_accuracy(Y_train_pred, Y_train))\n",
        "    train_loss.append(_cross_entropy_loss(y_train_pred, Y_train) / train_size)\n",
        "\n",
        "    y_dev_pred = _f(X_dev, w, b)\n",
        "    Y_dev_pred = np.round(y_dev_pred)\n",
        "    dev_acc.append(_accuracy(Y_dev_pred, Y_dev))\n",
        "    dev_loss.append(_cross_entropy_loss(y_dev_pred, Y_dev) / dev_size)\n",
        "\n",
        "print('Training loss: {}'.format(train_loss[-1]))\n",
        "print('Development loss: {}'.format(dev_loss[-1]))\n",
        "print('Training accuracy: {}'.format(train_acc[-1]))\n",
        "print('Development accuracy: {}'.format(dev_acc[-1]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training loss: 0.27135543524640593\n",
            "Development loss: 0.2896359675026287\n",
            "Training accuracy: 0.8836166291214418\n",
            "Development accuracy: 0.8733873940287504\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RJuoQ_R2jUmX",
        "colab_type": "text"
      },
      "source": [
        "### Plotting Loss and accuracy curve"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DH3AJtvHjVJ7",
        "colab_type": "code",
        "outputId": "f3fc5d1b-ddcc-4cf6-eea5-ebf23026edc5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 545
        }
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Loss curve\n",
        "plt.plot(train_loss)\n",
        "plt.plot(dev_loss)\n",
        "plt.title('Loss')\n",
        "plt.legend(['train', 'dev'])\n",
        "plt.savefig('loss.png')\n",
        "plt.show()\n",
        "\n",
        "# Accuracy curve\n",
        "plt.plot(train_acc)\n",
        "plt.plot(dev_acc)\n",
        "plt.title('Accuracy')\n",
        "plt.legend(['train', 'dev'])\n",
        "plt.savefig('acc.png')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhU5dnH8e+dfSckbIGwhH0RQQiI\nCwoIiKigrxtara0L9UVcqm9bat1bW6vWtlZaxWKtrYpaa0VFcUERiiigoET2PeyEJSEL2e73j3MS\nJhDIBCY5k5n7c11zZeYswz0BfueZ55zzPKKqGGOMCV0RXhdgjDGmYVnQG2NMiLOgN8aYEGdBb4wx\nIc6C3hhjQpwFvTHGhDgLemOMCXEW9CasichGERnpdR3GNCQLemOMCXEW9MbUQkRuFpG1IrJXRGaK\nSFt3uYjI70Vkl4jki8i3InKKu26siHwnIgUislVE/s/bT2GMw4LemCOIyAjgN8CVQAawCZjhrh4N\nnAN0B5q52+S566YDP1LVZOAUYE4jlm3MMUV5XYAxQeh7wPOq+hWAiPwc2CcinYAyIBnoCXypqit8\n9isDeovIMlXdB+xr1KqNOQZr0RtztLY4rXgAVPUgTqu9narOAZ4GpgK7RGSaiKS4m14GjAU2ichc\nETmjkes2plYW9MYcbRvQseqFiCQC6cBWAFV9SlUHAr1xunB+4i5fpKrjgVbAf4DXGrluY2plQW8M\nRItIXNUDeAX4oYj0F5FY4NfAF6q6UUQGicjpIhINFAIlQKWIxIjI90SkmaqWAflApWefyBgfFvTG\nwCyg2OcxDLgPeAPYDnQBJrjbpgDP4fS/b8Lp0nncXXcdsFFE8oFbcPr6jfGc2MQjxhgT2qxFb4wx\nIc6C3hhjQpwFvTHGhDgLemOMCXFBd2dsixYttFOnTl6XYYwxTcqSJUv2qGrL2tYFXdB36tSJxYsX\ne12GMcY0KSKy6VjrrOvGGGNCnAW9McaEOAt6Y4wJcUHXR2+MMSeirKyM3NxcSkpKvC6lQcXFxZGZ\nmUl0dLTf+1jQG2NCQm5uLsnJyXTq1AkR8bqcBqGq5OXlkZubS1ZWlt/7WdeNMSYklJSUkJ6eHrIh\nDyAipKen1/tbiwW9MSZkhHLIVzmRzxg6QV+SDx89BHnrvK7EGGOCSugEfVkRfPEsfPyQ15UYY8LQ\n/v37+fOf/1zv/caOHcv+/fsboKLDQifok9vAWXfAd2/B5oVeV2OMCTPHCvry8vLj7jdr1ixSU1Mb\nqizAz6AXkTEiskpE1orIlFrW3yIi34rIUhGZLyK9j1jfQUQOisj/BarwWp05GZIzYPYvwCZUMcY0\noilTprBu3Tr69+/PoEGDGDp0KOPGjaN3bycOL7nkEgYOHEifPn2YNm1a9X6dOnViz549bNy4kV69\nenHzzTfTp08fRo8eTXFxcUBqq/PyShGJxJnxfhSQCywSkZmq+p3PZi+r6jPu9uOAJ4ExPuufBN4L\nSMXHE5MII+6DtybB8jeg7+UN/kcaY4LPQ2/n8N22/IC+Z++2KTxwcZ9jrn/00UdZvnw5S5cu5dNP\nP+XCCy9k+fLl1ZdBPv/886SlpVFcXMygQYO47LLLSE9Pr/Eea9as4ZVXXuG5557jyiuv5I033uDa\na6896dr9adEPBtaq6npVLQVmAON9N1BV399oIlDdnBaRS4ANQM5JV+uPfhOgdV/nxGxZaN84YYwJ\nXoMHD65xrftTTz1Fv379GDJkCFu2bGHNmjVH7ZOVlUX//v0BGDhwIBs3bgxILf7cMNUO2OLzOhc4\n/ciNRORW4C4gBhjhLksCfobzbeCY3TYiMhGYCNChQwc/Sz+GiEg4/1fw4nj48lmn394YE1aO1/Ju\nLImJidXPP/30Uz766CM+//xzEhISGDZsWK3XwsfGxlY/j4yMDFjXTcBOxqrqVFXtghPs97qLHwR+\nr6oH69h3mqpmq2p2y5a1DqdcP52HQbfz4bPfQWHeyb+fMcbUITk5mYKCglrXHThwgObNm5OQkMDK\nlStZuLBxLxjxJ+i3Au19Xme6y45lBnCJ+/x04DER2QjcCdwjIpNPoM76G/1LKD0Icx9tlD/OGBPe\n0tPTOeusszjllFP4yU9+UmPdmDFjKC8vp1evXkyZMoUhQ4Y0am2idVydIiJRwGrgPJyAXwRco6o5\nPtt0U9U17vOLgQdUNfuI93kQOKiqTxzvz8vOztaATTzyzl3w1d9h0kJo0S0w72mMCUorVqygV69e\nXpfRKGr7rCKy5MjcrVJni15Vy4HJwGxgBfCaquaIyMPuFTYAk0UkR0SW4vTTX38yHyJghv0couLh\nw/u9rsQYYzzj1+iVqjoLmHXEsvt9ntd5xlNVH6xvcSctqSUM/TF8/DBsmAdZQxu9BGOM8Vro3Bl7\nLEMmQbP28MEvoLLS62qMMabRhX7QR8fDeffD9mXw7WteV2OMMY0u9IMe4JTLoe1pThdOaZHX1Rhj\nTKMKj6CPiIDRj0D+Vlg41etqjDGmUYVH0AN0Ogt6XgTz/wAFO72uxhgT4h588EGeeOK4V5M3mvAJ\neoCRD0F5CXz6G68rMcaYRhNeQd+iKwy6ybmJatcKr6sxxoSYRx55hO7du3P22WezatUqANatW8eY\nMWMYOHAgQ4cOZeXKlRw4cICOHTtS6V4JWFhYSPv27SkrK2uQuvy6jj6knPszWPYKfHAfXPsvr6sx\nxjSE96bAjm8D+55t+sIFxx5SZcmSJcyYMYOlS5dSXl7OgAEDGDhwIBMnTuSZZ56hW7dufPHFF0ya\nNIk5c+bQv39/5s6dy/Dhw3nnnXc4//zziY6ODmzNrvAL+oQ0OOcn8MG9sG4OdBnhdUXGmBAwb948\nLr30UhISEgAYN24cJSUlLFiwgCuuuKJ6u0OHDgFw1VVX8eqrrzJ8+HBmzJjBpEmTGqy28At6gMET\n4cvnYPa9cMs8Z2hjY0zoOE7LuzFVVlaSmprK0qVLj1o3btw47rnnHvbu3cuSJUsYMaLhGp3h1Udf\nJSoWRj4Iu3Jg6cteV2OMCQHnnHMO//nPfyguLqagoIC3336bhIQEsrKyeP311wFQVZYtWwZAUlIS\ngwYN4o477uCiiy4iMrLhGpzhGfQAfS6FzMEw51dw6LjD5RtjTJ0GDBjAVVddRb9+/bjgggsYNGgQ\nAC+99BLTp0+nX79+9OnTh7feeqt6n6uuuop//vOfXHXVVQ1aW53DFDe2gA5TXJctX8L0UXDuFBj+\n88b5M40xDcKGKT6JYYpDWvvBTst+wVOQv93raowxpkGEd9CD01dfWe504RhjTAiyoG/eybkKZ+lL\nsP0br6sxxpyEYOuKbggn8hkt6AHO+T+IT3WurQ+DfyjGhKK4uDjy8vJCOuxVlby8POLi4uq1X3he\nR3+k+ObOCdn3fwZrPoTuo72uyBhTT5mZmeTm5rJ7926vS2lQcXFxZGZm1msfC/oq2TfAl9OcVn2X\nERBpvxpjmpLo6GiysrK8LiMo+dV1IyJjRGSViKwVkSm1rL9FRL4VkaUiMl9EervLB7vLlorIMhG5\nNNAfIGCiYmDUw7BnlTPomTHGhIg6g15EIoGpwAVAb+DqqiD38bKq9lXV/sBjwJPu8uVAtrt8DPCs\niARvU7nnhdDxLPjk11CS73U1xhgTEP606AcDa1V1vaqWAjOA8b4bqKpvKiYC6i4vUtVyd3lc1fKg\nJQKjfwVFe+C/f/C6GmOMCQh/gr4dsMXnda67rAYRuVVE1uG06G/3WX66iOQA3wK3+AS/774TRWSx\niCz2/ERKuwHQ90r4fCocyPW2FmOMCYCAXV6pqlNVtQvwM+Ben+VfqGofYBDwcxE56rogVZ2mqtmq\nmt2yZctAlXTizrvf+fnxw97WYYwxAeBP0G8F2vu8znSXHcsM4JIjF6rqCuAgcEp9CvREansYMgm+\neRW2fuV1NcYYc1L8CfpFQDcRyRKRGGACMNN3AxHp5vPyQmCNuzyr6uSriHQEegIbA1B3wzv7x5DQ\nwm6iMsY0eXUGvdunPhmYDawAXlPVHBF5WETGuZtNFpEcEVkK3AVc7y4/G1jmLn8TmKSqewL+KRpC\nXIozouWm/8KqWV5XY4wxJyy8hymuS0U5/OVM0AqYtBAiG2Y+R2OMOVk2TPGJioyC0b+EvLWw+Hmv\nqzHGmBNiQV+XbqMh61z49FEo3u91NcYYU28W9HURgfMfgeJ9MO8Jr6sxxph6s6D3R5u+0P978MWz\nsG+j19UYY0y9WND7a8QvICIKPnrI60qMMaZeLOj9ldIWzrwNcv4NWxZ5XY0xxvjNgr4+zrwdklrD\n7HvsJipjTJNhQV8fsUkw4l7I/RK++4/X1RhjjF8s6Our//egVR/48AEoP+R1NcYYUycL+vqKiHRu\notq/Cb58zutqjDGmThb0J6LredB1JHz2GBTt9boaY4w5Lgv6EzX6V3CoAOY+5nUlxhhzXBb0J6pV\nLxjwfVj0HOSt87oaY4w5Jgv6kzH8FxAVBx/e73UlxhhzTBb0JyOpFZx9J6x8BzYt8LoaY4yplQX9\nyRpyKyS3hdm/gMpKr6sxxpijWNCfrJgEZzLxbV/B8je8rsYYY45iQR8Ip14FGf3g44egrNjraowx\npgYL+kCIiIDRj8CBLbDwL15XY4wxNfgV9CIyRkRWichaEZlSy/pbRORbEVkqIvNFpLe7fJSILHHX\nLRGREYH+AEEjayj0GAvznoSDu72uxhhjqtUZ9CISCUwFLgB6A1dXBbmPl1W1r6r2Bx4DnnSX7wEu\nVtW+wPXAPwJWeTAa9TCUFcHcR72uxBhjqvnToh8MrFXV9apaCswAxvtuoKr5Pi8TAXWXf62q29zl\nOUC8iMSefNlBqkU3yL4BFv8N1n7kdTXGGAP4F/TtgC0+r3PdZTWIyK0isg6nRX97Le9zGfCVqh41\n5KOITBSRxSKyePfuJt7tMfweaNUbXroSFk33uhpjjAncyVhVnaqqXYCfAff6rhORPsBvgR8dY99p\nqpqtqtktW7YMVEneSEiDG96HbqPg3bvg/XugssLrqowxYcyfoN8KtPd5nekuO5YZwCVVL0QkE3gT\n+L6qhsegMLFJMOFlOP1/YeFUePVaOHTQ66qMMWHKn6BfBHQTkSwRiQEmADN9NxCRbj4vLwTWuMtT\ngXeBKar638CU3ERERMIFj8LYJ2D1+/C3CyB/W937GWNMgNUZ9KpaDkwGZgMrgNdUNUdEHhaRce5m\nk0UkR0SWAnfhXGGDu19X4H730sulItIq8B8jiA2+Ga55HfZugOdGwPZlXldkjAkzokE2yXV2drYu\nXrzY6zICb2cOvHwVFOXBZdOh51ivKzLGhBARWaKq2bWtsztjG0vrPnDTx9CyJ8y4Bj6fCkF2kDXG\nhCYL+saU3Bp+8C70uhhm3wPv3g0V5V5XZYwJcRb0jS0mAa74O5z9Y1g8HV6+AkoOeF2VMSaEWdB7\nISICRj4I456GDZ/B9PNh3yavqzLGhCgLei8NuA6uexMKtsFfz4PcEDwJbYzxnAW917LOgRs/gphE\neOFCyHnT64qMMSHGgj4YtOwON82BjP7w+g9g3u/sihxjTMBY0AeLxHT4/lvQ9wr4+GF4azKUl3pd\nlTEmBER5XYDxER0H//McpHeFT38D+zfBlS86A6UZY8wJshZ9sBGBYVOcwN/yBUwfBXnhMRacMaZh\nWNAHq1OvhO/PhOJ9zhU5mxZ4XZExpomyoA9mHc+Amz6ChBbw4nhY9qrXFRljmiAL+mCX1hlu+hDa\nnw5vToQ5j9gVOcaYerGgbwrim8O1/4bTroPPHoM3boKyEq+rMsY0EXbVTVMRFQPj/uRckfPRA3Bg\nizOLVWILryszxgQ5a9E3JSJw9p3OJZfblzkTmexe5XVVxpggZ0HfFPUeDz+YBWXF8NdRsP5Trysy\nxgQxC/qmKnMg3DwHmmXCPy+DJS94XZExJkhZ0Ddlqe3hhveh8zB4+w744D6orPS6KmNMkPEr6EVk\njIisEpG1IjKllvW3iMi37uTf80Wkt7s8XUQ+EZGDIvJ0oIs3QFwKXP0qDLoJFjwFr10HpUVeV2WM\nCSJ1Br2IRAJTgQuA3sDVVUHu42VV7auq/YHHgCfd5SXAfcD/Ba7kY9txoISKyjC8xjwyCsY+AWN+\nC6tmwQtjoWCH11UZY4KEPy36wcBaVV2vqqXADGC87waqmu/zMhFQd3mhqs7HCfwGtW73QYY/8Skv\nfRGmMzWJwJBbYMIrsHu1c0XOjm+9rsoYEwT8Cfp2wBaf17nushpE5FYRWYfTor+9PkWIyEQRWSwi\ni3fv3l2fXat1bpHIgI6pPD57FbsLDp3Qe4SEHmOcfntVZ4rCb173uiJjjMcCdjJWVaeqahfgZ8C9\n9dx3mqpmq2p2y5YtT+jPFxEeGncKJWUV/Oa9FSf0HiEj41TnipyMU+HfN8Hbd9qdtMaEMX+CfivQ\n3ud1prvsWGYAl5xMUSeqa6skbhramX9/tZUvN+z1ooTgkZIB178DZ90JS/4G00facMfGhCl/gn4R\n0E1EskQkBpgAzPTdQES6+by8EFgTuBLr57YRXWmXGs/9by2nvCLMLzWMjIJRD8E1r8GBXHj2XJuT\n1pgwVGfQq2o5MBmYDawAXlPVHBF5WETGuZtNFpEcEVkK3AVcX7W/iGzEuQrnByKSW8sVOwGVEBPF\nfRf1ZuWOAl5YsLEh/6imo/v58KN50KqnMyftrJ9AeRifxzAmzIgG2ZC32dnZunjx4pN6D1Xlhy8s\nYvHGfXx897m0TokLUHVNXHkpfPwQfP40tD0NrngBmnfyuipjTACIyBJVza5tXUjeGeucmO1DaUUl\nv3o3zE/M+oqKgfMfcUa93LsenjkHVrzjdVXGmAYWkkEP0DE9kVvO7cLby7axYO0er8sJLj0vhB99\nBuld4NXvwfv3OK19Y0xICtmgB5g0rAvt0+K5763llJaH+YnZIzXv5FxvP/hHsHAq/O0C2L/Z66qM\nMQ0gpIM+LjqSh8b1Yd3uQqbP3+B1OcEnKhbGPgZX/B32rIZnhsKq972uyhgTYCEd9AAjerZmVO/W\nPPXxGrbuL/a6nODU5xKY+KkzGuYrVzmjYFaUeV2VMSZAQj7oAR64uDeK8su3v/O6lOCV3gVu/Aiy\nb3RGwXzhIjhwvPvijDFNRVgEfWbzBG4b0Y33c3bw6apdXpcTvKLj4KIn4bLpsHM5PDsU1nzkdVXG\nmJMUFkEPcNPQLDq3SOSBmTmUlFV4XU5w63u505WT1AZeugw+fhgqyr2uyhhzgsIm6GOjInlofB82\n5RUx7bP1XpcT/Fp0g5s/hgHfh3m/gxfHQ/52r6syxpyAsAl6gKHdWnJh3wymfrKWLXttFqY6RcfD\nuD/Bpc/Ctq+crhybiNyYJiesgh7g3ot6ERkhPDgzx+tSmo5+E+DmTyAhHV68BD75DVRa95cxTUXY\nBX1Gs3juHNmNj1fu4sPvdnpdTtPRqqczxn2/CTD3UfjHpXDQTmwb0xSEXdAD/PCsLLq1SuLBmTkU\nl1rL1G8xiXDpMzB+Kmz5Ap45GzbM87oqY0wdwjLooyMjeHj8KWzdX8yfP13rdTlNz2nXOq372BR4\ncRx89jhU2hATxgSrsAx6gDO6pHNJ/7Y8O3c9G/YUel1O09O6D0z8BE65DOb8Cl66HApt8DhjglHY\nBj3APRf2IjYqgvvfWk6wjcvfJMQmw/88Bxf9HjbOd8bK2fS511UZY44Q1kHfKjmOu0Z3Z96aPby3\nfIfX5TRNIpB9A9z0oXNn7QsXwvw/WFeOMUEkrIMe4LohHemdkcLDb39H4SG7+/OEZfSDiXOh18Xw\n0QPwygQoCvMJ2o0JEmEf9FGREfzykj7syC/hqTmezWkeGuJSnOkJxz4B6z9xunK2LPK6KmPCXpQ/\nG4nIGOCPQCTwV1V99Ij1twC3AhXAQWCiqn7nrvs5cKO77nZVnR248gNjYMc0rhiYyfR5G7h8QCbd\nWid7XVLTJQKDb4Z2A52JyP82xunaSWoNMUkQk+Bcphmd6Pz0fUQnONtExXj9KYwJKXVODi4ikcBq\nYBSQCywCrq4KcnebFFXNd5+PAyap6hgR6Q28AgwG2gIfAd1V9ZgXrwdicvATkXfwECN+N5deGcm8\ncvMQRKTRawg5xfvh7Ttgxdtw7L/yo0VEuweEJDf8jzwgVD2vWlfLdrUdSKJiG+6zGuOx400O7k+L\nfjCwVlXXu282AxgPVAd9Vci7EoGqo8d4YIaqHgI2iMha9/2C7tKM9KRYfnJ+D+79z3JmLtvG+P7t\nvC6p6YtPhSv/DqpQUQqlhYcfZVXPi6D0oLus6nnREdu42x3c5W7js7yyHhOkpHaA7mOg+/nQaagF\nvwkb/gR9O2CLz+tc4PQjNxKRW4G7gBhghM++C4/Y96gEFZGJwESADh06+FN3g7h6cAdeW7yFX727\nguE9W5ESF+1ZLSFFxAnVqFhISAvse5eXHn3QOPJgUFoIpQWw9Sv46h/w5TSnxd9luBP83UZDcuvA\n1mVMEPGrj94fqjoVmCoi1wD3AtfXY99pwDRwum4CVVN9RUYIvxx/Cpf8+b/8/sPVPHBxH69KMf6K\ninEe8c39276s2Bm2YfX7sHo2rHzHWd5u4OHWfptTnYOTMSHCn6DfCrT3eZ3pLjuWGcBfTnBfz/Vr\nn8rVgzvw9wUbuWJge3q3TfG6JBNI0fHQfbTzUHVm0qoK/U9+DZ88AsltncDvPgY6n+vsY0wT5s/J\n2Cick7Hn4YT0IuAaVc3x2aabqq5xn18MPKCq2SLSB3iZwydjPwa6BePJWF/7i0oZ8bu5dG6RyGs/\nOoOICGvdhYWDu2DNh07wr5vjdANFxTthXxX8KW29rtKYWp3UyVhVLReRycBsnMsrn1fVHBF5GFis\nqjOBySIyEigD9uF227jbvYZz4rYcuPV4IR8sUhNimDKmJz994xve+CqXK7Lb172TafqSWsFp33Me\n5Ydg03+dlv6q95zw58dOt073MdBjDGScBhFhfyuKaQLqbNE3tmBo0QNUViqXP7OATXlFzLl7GM0S\n7MRs2FKF3avcLp73nSGatRISW7ndQGOg83CITfK6UhPGjteit6A/jpxtB7j4T/O55vQO/OqSvl6X\nY4JF0V5Y+5HT0l/7MRw6AJExziWbVSd0m3f0ukoTZizoT8KDM3P4++cbeevWszg1M9XrckywqSiD\nzQsPt/bz3PkNWvV2+/UvgMxsiIj0tk4T8izoT0J+SRkjnphLu9Q43px0lp2YNce3Z+3h0N/8OVSW\nQ3yac61+9/Oh63kQ18zrKk0IsqA/SW9+ncuPX13Gry/tyzWne3dDl2liivfDuo+dE7prPoDifRAR\nBR3OgK4jodsop+Vv1+ybALCgP0mqyoRpC1m1s4A5dw8jLdEG3TL1VFkBW750WvprP3Ku3wdIaXc4\n9LPOdUYANeYEWNAHwOqdBYz94zwuG5DJby8/1etyTFN3YKsT+Gs/hHWfOkM0VLX2u42CrqOgVS9r\n7Ru/WdAHyK9nrWDaZ+t543/PZGBHP2+5N6YuFWXOJZtrPoA1H8Eu915Ea+2berCgD5CDh8oZ+bu5\npCXG8PZtZxNpJ2ZNQ7DWvjkBFvQB9M4325j88tc8NK4P15/ZyetyTKgrL3Va+2s/PKK1n+lcwdNt\nFHQe5kzUbsKaBX0AqSrXTf+SZbn7mXP3MFom25jmphEd2OqG/oewfq7b2o+GDkOstR/mLOgDbN3u\ng4z5w2dcfGpbnryqv9flmHB1vNZ+t5FO6Hc+15vWfkU5lOx3Likt2gvFe2t/XlEKbU9zuqXaDXRm\nDTMnxIK+ATw+eyVTP1nHqxOHcHrndK/LMQYO5Dp9+4Fs7avCofwjAnrfsYO76nnJgWO/p0Q68wdU\nTUKzZw2gTq1t+zuh3/FMaH964CeqCWEW9A2guLSCkU/OJTE2kndvH0p0pI1iaIJIjdb+h7DLnfmz\nqrXfeZjzujqg9x0juPcdf77fuGZOaMenOaFc6/PmNZfHptQ82BTvg81fOHcSb/7cmQmsaorIlr2g\n4xnQ4UznZ7PMhvhthQQL+gbyQc4OJv5jCfde2Iubhnb2uhxjjq221r6v6ITaQ/l4z+NSITJgk9Qd\nVlYMW5fAps9h8wLnRrPSg866Zh3c4HcfLXvY+QiXBX0DUVVueGERX27Yy8d3D6NNszivSzKmbuWl\nzp25UbFOcMc3h+gg/rdbUe7Uu/lz2LTA+Vm421kXn+Z29bit/oxTITI8hxS3oG9Am/IKGfX7zxjd\nuzVPXzPA63KMCX2qsHf94dDftAD2bXDWRSc4o4VWdfVkDoKYRG/rbSQnNcOUOb6O6YlMGtaFP3y0\nhqsH7+Gsri28LsmY0CYC6V2cx4DrnGX52w/38W/+HOb+FucEbxRk9Dvc1dPhDEgMv4snrEUfACVl\nFZz/h8+IjBDeu2MosVE29rgxnio54PTtV7X6ty5xLuUEaNGj5gne1NAYkda6bhrBJ6t28cO/LeIn\n5/fg1uFdvS7HGOOrrAS2fe2c3N30uXNF0qF8Z11KJmQOhLTO0LzT4UdKZsOcbG4gJ911IyJjgD/i\nTA7+V1V99Ij1dwE34UwAvhu4QVU3uet+C1zobvpLVX31hD5FkBveoxXn92nNn+as4ZLT2tEuNd7r\nkowxVaLjnNZ7xzNgKM6w0TtznNnBNi+A7ctg5bvORDFVJBJS29cMf99HfNMZ2LDOFr2IRAKrgVFA\nLrAIuFpVv/PZZjjwhaoWicj/AsNU9SoRuRC4E7gAiAU+Bc5T1fxj/XlNtUUPkLuviJFPzuWcbi35\ny7UDbdAzY5qSygrI3wr7Ntb+KMqruX1cs2MfBJq1b/Srf062RT8YWKuq6903mwGMB6qDXlU/8dl+\nIXCt+7w38JmqlgPlIvINMAZ4rd6fognIbJ7AbSO68fjsVfS87z3apyWQlZ5IpxaJdEpPcH8m0jY1\n3g4CxgSbiEinvz61A2Sdc/T6knzYv+noA8DOHGei+KpzAAAS4dzcddRBIOvwt4FGvP7fn6BvB2zx\neZ0LnH6c7W8E3nOfLwMeEJHfAQnAcHwOEKHolnO7kNk8nhXbC9i4p5CNeYUsWJdHcdnhuwtjIiNo\nnxZPlhv8HVskugeEBDKa2QV4SxYAABC7SURBVEHAmKAUlwJt+jqPI1VWQMH22r8JrHrv8HX/VWKb\nQfOORx8I0jpDWlbASw/omQYRuRbIBs4FUNUPRGQQsACn7/5z4Kj7qUVkIjARoEOHpn0GPDJCGN+/\nHeN9xjpTVXbmH2JjXiEb9xSywf25Ka+I+Wv3UFJWWb1tTFQEHdIS6JR++FtAVotEOqYn0LZZvE1O\nbkwwioh0WvDNMqHT2UevP3Sw9m8Du1c6cwpXHHK2a3saTPw04OX5E/RbgfY+rzPdZTWIyEjgF8C5\nqnqoarmqPgI84m7zMk5/fw2qOg2YBk4ffT3qbxJEhDbN4mjTLI4hRwyAVlmp7CwoYeOeouoDgfOz\niHlrdnOovOZBoGNaQo2uoCz3G0FGSpwdBIwJVrFJ0LqP8zhSZSUc3OEEf+VxxhU6Cf4E/SKgm4hk\n4QT8BOAa3w1E5DTgWWCMqu7yWR4JpKpqnoicCpwKfBCo4kNBRISQ0SyejGbxnNHl6IPAjvyS6uD3\nPRB8trrmQSA2KoKO6Ql0TE+s7hLq264ZfTObNfZHMsbUR0QEpLR1Hg2kzqBX1XIRmQzMxrm88nlV\nzRGRh4HFqjoTeBxIAl4X5wTDZlUdB0QD89xl+cC17olZ44eICKFtajxtU+M5s0vNddUHAbcraFNe\nERv2OAeCuat3U+oeBK4/oyM/H9uLuGi7icuYcGU3TIWgykple34Jz8/fwPT5G+jTNoWnrxlAVovw\nGPPDmHB0vMsrbRD1EBQRIbRLjee+i3rz1+9ns3V/MRc9NY+3lh51asUYEwYs6EPcyN6tmXX7UHq3\nTeGOGUv56b+WUVRqvWfGhBML+jDQNjWeV24ewm0juvL6klzGPf1fVu0oqHtHY0xIsKAPE1GREdw9\nugf/vPF09heVMe7p+bzy5WaC7RyNMSbwLOjDzFldW/DeHUMZnJXGz//9Lbe98jUFJWVel2WMaUAW\n9GGoZXIsf//hYH46pgfvLd/BhU/N55vc/V6XZYxpIBb0YSoiQpg0rCuvThxCeUUll/1lAdPnb7Cu\nHGNCkAV9mMvulMasO4ZybvdW/PKd77j5xSXsKyyte0djTJNhQW9ITYjhue8P5IGLe/PZ6t2MfWoe\nizbu9bosY0yAWNAbwBl47YdnZfHG/55JTFQEE6Yt5Ok5a6iotK4cY5o6C3pTQ9/MZrxz29mM7ZvB\nEx+s5vrnv2RXQYnXZRljToIFvTlKclw0T03oz6P/05fFm/Yy9o/zmLdmd907GmOCkgW9qZWIMGFw\nB9669WyaJ8Tw/ee/5PHZKymvqKx7Z2NMULGgN8fVo00yMyefzZUD2zP1k3VMmLaQbfuLvS7LGFMP\nFvSmTvExkfz28lP544T+rNiez9in5vHhdzu9LssY4ycLeuO38f3b8c7tQ2mXGs/NLy7mobdzOFTe\nMFOfGWMCx4Le1EtWi0T+PelMfnBmJ/72341c9pcFbNxT6HVZxpjjsKA39RYbFcmD4/rw7HUD2bK3\nmIv+NJ+Zy7Z5XZYx5hgs6M0JO79PG2bdMZQebZK5/ZWvmfLGNxSXWleOMcHGr6AXkTEiskpE1orI\nlFrW3yUi34nINyLysYh09Fn3mIjkiMgKEXlK3JnCTWholxrPjIlDmDSsCzMWbWH81Pms2WmTmhgT\nTOoMehGJBKYCFwC9gatFpPcRm30NZKvqqcC/gMfcfc8EzgJOBU4BBgHnBqx6ExSiIyP46ZievHjD\nYPYWlnLx0/N5bdEWGwnTmCDhT4t+MLBWVderaikwAxjvu4GqfqKqRe7LhUBm1SogDogBYoFowK7L\nC1HndG/JrNuHMqBDc376xjfc+epSDh6y+WmN8Zo/Qd8O2OLzOtdddiw3Au8BqOrnwCfAdvcxW1VX\nnFippilolRLHP248nbtHdeftZdu46Kl5zF+zhwPFNouVMV6JCuSbici1QDZu94yIdAV6cbiF/6GI\nDFXVeUfsNxGYCNChQ4dAlmQ8EBkh3HZeNwZnpXHHjKVcO/0LwJnZqnOLRLq0SqJLyyQ6t0yka8sk\n2qbGExlhp26MaSj+BP1WoL3P60x3WQ0iMhL4BXCuqh5yF18KLFTVg+427wFnADWCXlWnAdMAsrOz\nrWM3RJzeOZ0P7jqHhevyWL+nkHW7DrJ+TyHvfrO9Rgs/NiqCrBaJdGmZRJeWiXRuefhAkBgb0LaI\nMWHJn/9Fi4BuIpKFE/ATgGt8NxCR04BngTGqustn1WbgZhH5DSA4Lf0/BKJw0zSkxEUzuk+bGstU\nlb2FpdXhv273QdbvLiRn2wHeW74d3yHw26TE0aWVcxDw/TbQJiWOCPsWYIxf6gx6VS0XkcnAbCAS\neF5Vc0TkYWCxqs4EHgeSgNfdqyc3q+o4nCtwRgDf4pyYfV9V326Yj2KaChEhPSmW9KRYBnVKq7Hu\nUHkFm/OKWLf7IOt2F1b/fPOrrRT4nNiNj46kc3XrP7H6G0DnFknEx0Q29kcyJqhJsF0Cl52drYsX\nL/a6DBNkVJXdBw+xbldh9TcA5yBwkK37i/H9Z9wuNZ7Obvh3aZVEF/ebQKvkWOw2DhOqRGSJqmbX\nts46QE2TICK0So6jVXIcZ3RJr7GupKyCDXuOPgC8tngLRT536qbERdEzI4VebZLplZFCz4wUurdO\nIiHG/huY0Gb/wk2TFxcdSa+MFHplpNRYrqrsyC+p/hawemcBK7bn868luRS6BwARyEpPpGdGMr3a\nOOHfs00ymc3jrfVvQoYFvQlZIkJGs3gymsVzdrcW1csrK5Ut+4pYsb2AlTvyWbE9n5xt+cz6dkf1\nNsmxUfTMSKZnmxS39Z9MzzbJ1vo3TZL10RvjOnionFU7Dof/yu0FrNxRUH13rwh0TEuoEf69M1Jo\nlxpvVwAZz1kfvTF+SIqNYmDH5gzs2Lx6maqSu6+YFdvzq78BrNxRwOzvdlSfAE6KjaJHm2R6+XwD\n6NEmmSS7B8AECWvRG3MCCg+Vu33+bvhvL2DFjnwKSg5fAtohLaFG+PfKSKZ98wRr/ZsGYS16YwIs\nMTaK0zo057QONVv/W/cXO6G/3Wn5r9iezwff7axu/SfERNIqOZakuCiSY6Pdn1Ekx0U5z+OiSXJf\nJ8dFkRQb7f6Mqv4ZFWnTSJj6saA3JkBEhMzmCWQ2T2Bk79bVy4tLK6qv+Fm5o4C9haUcPFROQUkZ\nW/YWUVBSXv260o8v2PHRkYcPDLE1Dw5VB4vk2KrnVQeJmgeMxJgo+2YRRizojWlg8TGR9GufSr/2\nqcfdTlUpLqvgYEk5+T7hf7CknIJD5c4BocRddsh3WRk780vc7cv9Gho6QiCjWTwd0xPcRyId0xLo\n4D638wuhxf42jQkSIkJCTBQJMVG0Sql7+2OprFQKS8trfFM4/Nw5WOwvLiV3XzGb8oqYnbOTvYWl\nNd4jPTHGCf009yDgHhA6pCXSIinG7jFoYizojQkxERHidtVE+71PfkkZm/OK2JRXxKa9hdXPF23c\nx1vLttUYYiIxJpL2aQl0cg8AzgHBeZ7RLM7OIQQhC3pjDClx0ZzSrhmntGt21LpD5RVs2VvM5r2F\nzoEgr4jNe4tYvauAOSt3UVpRWb1tVITQPi2BDmlV3wAOfyPokJZAXLQNOOcFC3pjzHHFRkXStVUS\nXVslHbWuotIZZmJTnvstYG8Rm/OK2JhXyFeb9tUYcRScYacPdwklkNEsHhEor1QqK9X5qUpF5eFH\n1bqKI5ZXHGtZfbZV571joyPp3jqJnm2cITC6t0kmpR7fiIKdXUdvjGkQqsq+ojLnILDX+Saw0eeA\nsLvgUN1vcgQR51tDhIjzM0KIjJBal0VGCJFy+Hn1Op9lVY+CknJW7yiocWBqlxpPjzbJ9GjjDH/R\ns00KnVsmEh2kXVN2Hb0xptGJCGmJMaQlxtS436BKUWk5O/MPESE4IR1ZM4QjjgjwyAhp0JPAqsq2\nAyWscu9+XrWjgJXbC/hs9W7K3eteoyOFLi2TahwAerRJoW2zuKA+QW0temOMOY7S8krW7znojoPk\nHABW7Shg6/7i6m2S46Lc0HeCv+p5Y3b/WIveGGNOUExUhNt3n8J4n+UHistYvbMq/PNZtaOAt5Zu\no6Bkc/U2bZvFOS3/jMPh37lFEjFRjdv9Y0FvjDEnoFl8NIM6pdWYDrO27p9VOwqYv3YPZRVO70lU\nRM3un14ZDd/9Y0FvjDEBIiK0S42nXWo8I3oeHgajtLySDXsKq0c/XbWjgCWb9jFz2bbqbZLjoji3\ne0uevmZAwOuyoDfGmAYWExVR3YL37f7JLyljtU/ff3Jcw0SyX+8qImOAPwKRwF9V9dEj1t8F3ASU\nA7uBG1R1k4gMB37vs2lPYIKq/icQxRtjTFOWEhdNdqc0sn26fxpCnWcERCQSmApcAPQGrhaR3kds\n9jWQraqnAv8CHgNQ1U9Utb+q9gdGAEXABwGs3xhjTB38OfU7GFirqutVtRSYATW+fVQFepH7ciGQ\nWcv7XA6857OdMcaYRuBP0LcDtvi8znWXHcuNwHu1LJ8AvFLbDiIyUUQWi8ji3bt3+1GSMcYYfwX0\nYk4RuRbIBh4/YnkG0BeYXdt+qjpNVbNVNbtly5aBLMkYY8KePydjtwLtfV5nustqEJGRwC+Ac1X1\nyEEsrgTeVNWyEy3UGGPMifGnRb8I6CYiWSISg9MFM9N3AxE5DXgWGKequ2p5j6s5RreNMcaYhlVn\n0KtqOTAZp9tlBfCaquaIyMMiMs7d7HEgCXhdRJaKSPWBQEQ64XwjmBvg2o0xxvjBr+voVXUWMOuI\nZff7PB95nH03cvyTt8YYYxpQ0I1eKSK7gU0n8RYtgD0BKqeps99FTfb7qMl+H4eFwu+io6rWejVL\n0AX9yRKRxccaqjPc2O+iJvt91GS/j8NC/XcRnFOlGGOMCRgLemOMCXGhGPTTvC4giNjvoib7fdRk\nv4/DQvp3EXJ99MYYY2oKxRa9McYYHxb0xhgT4kIm6EVkjIisEpG1IjLF63q8JCLtReQTEflORHJE\n5A6va/KaiESKyNci8o7XtXhNRFJF5F8islJEVojIGV7X5CUR+bH7/2S5iLwiInFe1xRoIRH0fk6O\nEk7KgbtVtTcwBLg1zH8fAHfgDOFhnNni3lfVnkA/wvj3IiLtgNtxJk46BWcWvQneVhV4IRH0+DE5\nSjhR1e2q+pX7vADnP3LYDkMhIpnAhcBfva7FayLSDDgHmA6gqqWqut/bqjwXBcSLSBSQAGyrY/sm\nJ1SCvr6To4QNd1C504AvvK3EU38AfgpUel1IEMjCmdf5b25X1l9FJNHroryiqluBJ4DNwHbggKqG\n3HSnoRL0phYikgS8Adypqvle1+MFEbkI2KWqS7yuJUhEAQOAv6jqaUAhELbntESkOc63/yygLZDo\nTqAUUkIl6P2aHCWciEg0Tsi/pKr/9roeD50FjBORjThdeiNE5J/eluSpXCBXVau+4f0LJ/jD1Uhg\ng6rudidG+jdwpsc1BVyoBH2dk6OEExERnD7YFar6pNf1eElVf66qmaraCeffxRxVDbkWm79UdQew\nRUR6uIvOA77zsCSvbQaGiEiC+//mPELw5LRf49EHO1UtF5GqyVEigedVNcfjsrx0FnAd8K2ILHWX\n3ePOK2DMbcBLbqNoPfBDj+vxjKp+ISL/Ar7CuVrta0JwOAQbAsEYY0JcqHTdGGOMOQYLemOMCXEW\n9MYYE+Is6I0xJsRZ0BtjTIizoDfGmBBnQW+MMSHu/wH1SbcjKy5ssgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEICAYAAAC0+DhzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3hUZfbA8e9JgZBAKAEikABRQUCB\nAKGo6CqgIihgQarKiourov5cV1ddd0VXV9fVtayoiwoqIkWwAKKCFAsoEqr0Dgk19BJC2vn98Q44\nkCCBlDvJnM/z5GHm3jvvPXdI5sxbr6gqxhhjjL8QrwMwxhgTeCw5GGOMycOSgzHGmDwsORhjjMnD\nkoMxxpg8LDkYY4zJw5KDMcaYPCw5mKAmIrNEZK+IlPc6FmMCiSUHE7REpD5wGaBAtxI8b1hJncuY\ns2XJwQSz24CfgPeA249tFJEKIvKSiGwSkf0i8oOIVPDtay8ic0Rkn4ikiMgA3/ZZInKnXxkDROQH\nv+cqIveKyBpgjW/bq74yDojIfBG5zO/4UBF5XETWichB3/54ERkqIi/5X4SITBSRB4vjDTLBy5KD\nCWa3AaN8P9eISKxv+4tAK+ASoBrwCJArIvWAL4H/AjWARGDRGZyvB9AWaOJ7Ps9XRjXgI+BjEYnw\n7fsT0AfoAkQDdwDpwPtAHxEJARCR6kAn3+uNKTKWHExQEpH2QD1gnKrOB9YBfX0funcAD6jqFlXN\nUdU5qnoU6At8o6qjVTVLVXer6pkkh+dUdY+qHgFQ1Q99ZWSr6ktAeeAC37F3Ak+o6ip1FvuO/RnY\nD3T0HdcbmKWqOwr5lhhzAksOJljdDkxV1V2+5x/5tlUHInDJ4mTxp9heUCn+T0TkzyKywtd0tQ+o\n7Dv/6c71PtDf97g/MLIQMRmTL+sYM0HH139wCxAqItt9m8sDVYBaQAZwHrD4pJemAG1OUexhINLv\n+Tn5HHN8CWRf/8IjuBrAMlXNFZG9gPid6zxgaT7lfAgsFZHmQGPgs1PEZMxZs5qDCUY9gBxc23+i\n76cx8D2uH2I48B8Rqe3rGL7YN9R1FNBJRG4RkTARiRGRRF+Zi4AbRSRSRM4HBp4mhkpANpAGhInI\n33F9C8e8A/xDRBqI00xEYgBUNRXXXzESmHCsmcqYomTJwQSj24ERqrpZVbcf+wFeB/oBjwK/4D6A\n9wD/AkJUdTOug/gh3/ZFQHNfmS8DmcAOXLPPqNPE8DXwFbAa2ISrrfg3O/0HGAdMBQ4A7wIV/Pa/\nDzTFmpRMMRG72Y8xpY+IXI5rXqqn9kdsioHVHIwpZUQkHHgAeMcSgykulhyMKUVEpDGwD9dx/orH\n4ZgyzJqVjDHG5GE1B2OMMXmUiXkO1atX1/r163sdhjHGlCrz58/fpao18ttXJpJD/fr1SU5O9joM\nY4wpVURk06n2WbOSMcaYPCw5GGOMycOSgzHGmDzKRJ9DfrKyskhNTSUjI8PrUIpVREQEcXFxhIeH\nex2KMaYMKbPJITU1lUqVKlG/fn1E5PQvKIVUld27d5OamkpCQoLX4RhjypAy26yUkZFBTExMmU0M\nACJCTExMma8dGWNKXplNDkCZTgzHBMM1GmNKXpltVjKmLEjZk87kJduoXCGcWpUjiI2O4JzKEVSN\nDLcvBh7KzVVEyvaXM0sOxWTfvn189NFH3HPPPWf0ui5duvDRRx9RpUqVYorMlAa5ucqHczfx/Jcr\nSc/MybO/XFgIsdHlqRVdgdjKEZwTXf544jiWRGpWiqBcWJluHChxK7cfYOy8FD5duIUDR7KIrhBO\ndEQ40RXCqHzscUQ4lSPDiY4II7pC+K/bK4S5fRXCia4QTvmwkIBOLpYcism+fft444038iSH7Oxs\nwsJO/bZPmTKluEMzAW7jrsM8MmEJP2/Yw2UNqvNMj4sICw1h+/4M93Mggx0Hfn28JHUfX+/PIDM7\nN09Z1SuWIzb614RxTnSEL5n4tlWOoFL5sID+kPLawYwsJi/Zxph5KSxO2Ue50BCuujCWhJgoDmRk\nceBIFvuPZHEgI5udBw75HmeRkZX3/8NfudAQl1xOShrREWF+j49tP/GYShFhhIcWb+K35FBMHn30\nUdatW0diYiLh4eFERERQtWpVVq5cyerVq+nRowcpKSlkZGTwwAMPMGjQIODXpUAOHTrEtddeS/v2\n7ZkzZw516tTh888/p0KFCqc5symtcnKVEbM38OLUVYSHhvDCTc3omRR3/IO7TpVT/9+rKvvSs9h+\nwJc89mewbb8viRzIIHXvEeZv2sve9Kw8r40sF+qShl/COPb8WE2kesXyhIYETwJRVRZs3suYn1OY\nvGQbR7JyaBhbkb9f14QbWtShalS505ZxNDuHA0ey8ySQXx9nuf2+x/vSM9m0+zAHMrLZfySLnNzf\nXjE7qlwo0RXCeaTzBdzQIq6oLv24oEgOT01axvKtB4q0zCa1o3ny+gtPuf/5559n6dKlLFq0iFmz\nZtG1a1eWLl16fMjp8OHDqVatGkeOHKF169bcdNNNxMTEnFDGmjVrGD16NG+//Ta33HILEyZMoH//\n/kV6HSYwrN15iEfGL2bB5n10bFSTZ29oyjmVIwr8ehGhalQ5qkaVo3Gt6FMel5GVc0Ktwz0+yo4D\nGWzbf4S5G/aw40AG2Sd9MIWGCPWqRfK7C2rQsVEsbRKqlckmq92HjvLJgi2MmbeZdWmHiSoXSvfE\n2vRqHU9ifJUzqmGVDwulRqVQalQqf8ZxqCrpmTnHE8j+I1nHk4h7nH38cWx0wX9PzkRQJIdA0KZN\nmxPmIrz22mt8+umnAKSkpLBmzZo8ySEhIYHERHf/+latWrFx48YSi9eUjOycXN7+fgMvf7OayHKh\nvNIrke6JtYutmSciPJR6MVHUi4k65TG5ucruw5m+hPFrTWTZ1v2MmruZEbM3UrF8GJc1qE7HxrFc\ncUENqlc88w/AQJGTq3y/Jo1xySlMW76DrBylZd0qvHBTM7o2q0VU+ZL/mBQRosqHEVU+jFqVS/z0\nQJAkh9/6hl9SoqJ+/WOcNWsW33zzDT/++CORkZFcccUV+c5VKF/+1z+40NBQjhw5UiKxmpKxcvsB\nHhm/hCWp++l84Tk83eNCalYqnm+BZyIkRKhRqTw1KpXnojonfjKlZ2YzZ+1upq/cyYyVO/hy6XZE\nIDG+Ch0b1aRDo1ga16pUKvowUvem83FyKh8np7B1fwbVospx+8X16dU6ngaxlbwOz3NBkRy8UKlS\nJQ4ePJjvvv3791O1alUiIyNZuXIlP/30UwlHZ7yUlZPLGzPX8frMNURHhDO0b0u6NqvldVgFElku\njE5NYunUJBbVi1i29QDTV7hE8eLU1bw4dTW1KkfQoVFNOjauySXnVSciPNTrsI87mp3DN8t3Mmbe\nZn5YuwuAyxrU4K9dm9CpSU3KhwVOrF6z5FBMYmJiuPTSS7nooouoUKECsbGxx/d17tyZt956i8aN\nG3PBBRfQrl07DyM1JWnplv08PH4JK7YdoFvz2gzpdiHVCtC5GYhEhIvqVOaiOpV5oFMDdh7MYNbK\nNKav3MGnC7cwau5mIsJDaH9+dTo0iqVDo5pn1I9SlFbvOHh8COqew5nUrhzB/R0a0DMpjriqkZ7E\nFOjKxD2kk5KS9OSb/axYsYLGjRt7FFHJCqZrLa2OZufw3+lrefPbdVSLKsezPS7i6gvP8TqsYnM0\nO4e56/cwY+VOvlmxg9S9rkn0wtrRdGxUk46NY2lapzIhxTgC6vDRbCYv2crYeSks2LyP8FDhqiax\n9Gpdl/bnVw+q0VenIiLzVTUpv31WczCmmC1K2cfDHy9mzc5D3NQyjr9f14TKkWV7Fd3yYaFc3rAG\nlzeswZPXN2HNzkPHm59en7mW12aspXrF8nRoVIMOjWJp36A6FYug41dVWZiyj3HzUpi0eCuHM3M4\nv2ZFnujamBta1CGmFHeclzRLDsYUk4ysHF6etpq3v19PbHQEI37fmisvqOl1WCVORGgYW4mGsZW4\n+4rz2Hs4k29XpzF95U6+XLqdccmplAsNoe251Y7XKuKrnVlTz57DmXy6cAtj521m9Y5DVAgP5frm\ntejVOp6WdauWig7yQGPJwZhikLxxD4+MX8L6XYfp0yaex7o0JjqibNcWCqpqVDl6tKhDjxZ1yMrJ\nJXnjXmas3MH0lTsZMmk5QyYtp0HNinRsHEvHxjVpEV+FsHxmA+fmKrPX7WLMvBSmLdtBZk4uzeOr\n8NyNTbmuWS0q2ftdKJYcjClC6ZnZvPDVKt7/cSN1qlTgw4Ftad+gutdhBazw0BAuPi+Gi8+L4a9d\nm7Bh12Fm+IbJvvP9et76dh1VIsO5omENOjSO5XcNanA4M9sNQZ2fQureI1SJDKdfu7r0ah1Po3NO\nPQHQnJkCJQcR6Qy8CoQC76jq8yftrwu8D1TxHfOoqk4RkXDgHaCl71wfqOpzIhIPfADEAgoMU9VX\nfWUNAf4ApPmKf1xVbcEhE/DmrNvFXyYsIWXPEW6/uB6PdG7kyQSq0iyhehQD2ycwsH0CBzKy+H71\nLqav3MGsVWl8tmgroSFCriqq0P786jzSuRFXN4kNqOGyZcVpf3NFJBQYClwFpALzRGSiqi73O+wJ\nYJyqvikiTYApQH2gJ1BeVZuKSCSwXERGA0eBh1R1gYhUAuaLyDS/Ml9W1ReL6iKNKU6Hjmbz3JQV\njJq7mfoxkYwd1I6258ac/oXmN0VHhNO1WS26NqtFTq6yKGUfM1fuJDREuKllHHVjbAhqcSrI15o2\nwFpVXQ8gImOA7oB/clDgWH2uMrDVb3uUiIQBFYBM4ICq7gG2AajqQRFZAdQ5qcwyZ8iQIVSsWJE/\n//nPXodiisi3q9N4/JNf2Lr/CHe2T+Chqy+gQjn7FlvUQkOEVvWq0qpeVa9DCRoFSQ51gBS/56lA\n25OOGQJMFZH7gCigk2/7eFwi2QZEAg/6EsNxIlIfaAHM9ds8WERuA5JxNYy9JwclIoOAQQB169Yt\nwGUYU3T2H8ni2S+WMy45lfNqRDH+j5fYB5cpU4pqWcU+wHuqGgd0AUaKSAiu1pED1AYSgIdE5Nxj\nLxKRisAE4P9U9diyqW8C5wGJuKTyUn4nVNVhqpqkqkk1atQoossoes8++ywNGzakffv2rFq1CoB1\n69bRuXNnWrVqxWWXXcbKlSvZv38/9erVIzfXrQF/+PBh4uPjycrKu8Sy8db0FTu4+uVvmbBgC3df\ncR5f3H+ZJQZT5hSk5rAFiPd7Hufb5m8g0BlAVX8UkQigOtAX+EpVs4CdIjIbSALW+zqrJwCjVPWT\nYwWp6o5jj0XkbWDyGV/Vyb58FLb/UuhiTnBOU7j2+d88ZP78+YwZM4ZFixaRnZ1Ny5YtadWqFYMG\nDeKtt96iQYMGzJ07l3vuuYcZM2aQmJjIt99+y5VXXsnkyZO55pprCA+34XiBYu/hTJ6atIzPFm2l\n0TmVePu2JJrF2R37TNlUkOQwD2ggIgm4pNAb96HvbzPQEXhPRBoDEbjRRpuBDriaRBTQDnhF3IyU\nd4EVqvof/4JEpJaqbvM9vQFYelZXFgC+//57brjhBiIjXcdZt27dyMjIYM6cOfTs2fP4cUePHgWg\nV69ejB07liuvvJIxY8ac8S1Gy4p96ZmkZ+ZQMSKMiuXCinWJhYL6auk2nvhsGfvSM3mgYwPuvfL8\nMnk/A2OOOW1yUNVsERkMfI0bpjpcVZeJyNNAsqpOBB4C3haRB3Gd0ANUVUVkKDBCRJYBAoxQ1SUi\n0h64FfhFRBb5TnVsyOoLIpLoK2cjcFehr/I03/BLUm5uLlWqVGHRokV59nXr1o3HH3+cPXv2MH/+\nfDp06OBBhN4aPz+Vxz/95YRbXlYsH0aliLBf/41wt0msdHx7uG97GNEnPXfHhRMRfnb369116ChP\nfr6ML37ZxoW1o/ngjjY0qW1j6U3ZV6BB2L4P7Sknbfu73+PlwKX5vO4Qbjjrydt/wCWL/M51a0Fi\nKg0uv/xyBgwYwGOPPUZ2djaTJk3irrvuIiEhgY8//piePXuiqixZsoTmzZtTsWJFWrduzQMPPMB1\n111HaGjwjHrJysnl2S9W8N6cjVx8bgzdE2tzMCObg0ezOZiRxaGMbA5mZHPoaDb70zNJ3Zvunmdk\ncyQr57Tlh4XI8WRxLHkcTy4RYVSKCKdieV9y8SWUXYeO8q+vVnL4aA4PX3MBgy4/t9jv22tMoLAZ\nOsWoZcuW9OrVi+bNm1OzZk1at24NwKhRo7j77rt55plnyMrKonfv3jRv3hxwTUs9e/Zk1qxZHkZe\nsnYdOso9oxbw84Y9DGyfwGPXNsp3uYRTycrJ5fBRlzzcTxaHjj3PJ7kczMjiYEY22w9ksGbnr9uy\ncvKuUNw8vgr/vrkZDe3mLybI2JLdZUBpvtYlqfv448j57D6cyfM3NS2WG6UXhKpyNDv3hASSlZNL\nYnxVW9rZlFm2ZLcJSBPmp/LYp79Qo2J5Jtx9SZ5bUpYkESEiPJSI8LO7IbwxZY0lB1PiTu5feL1v\nC1tn35gAU6aTg6qW+XXcS1uz4K5DR7l31ALmnmX/gjGmZJTZ5BAREcHu3buJiYkpswlCVdm9ezcR\nEd7cl/dM+fcvvNyruWf9C8aY0yuzySEuLo7U1FTS0tJOf3ApFhERQVxc4H/IBlL/gjHm9MpscggP\nDychIcHrMIKe9S8YUzqV2eRgvGf9C8aUXpYcTLGw/gVjSjdLDgHgn1NW8MOaXfRrV5cbW8SV+pvF\nWP+CMaWf1fE9NnHxVoZ9t55dh47y10+XcvHz03nhq5Vs35/hdWhnLCsnlyETl/HQx4tpVbcqEwdf\naonBmFLKag4e2rDrMI9NWEKrelUZM6gdCzbtZfjsDbz57TqGfbeers1qccelCTSPD/x7Blj/gjFl\niyUHj2Rk5XDvqAWEh4Xw3z4tCA8Noe25MbQ9N4bNu9N5b85GxiWn8PmirSTVq8od7RO4uklsQH7g\n/pK6n7tGJlv/gjFlSJldeC/Q/e2zpYz8aRPv3p5Ex8ax+R5zMCOLj5NTGTFnAyl7jlCnSgUGXFKf\nXm3iiY4IjDvE+fcv/O/WVtaMZEwp8lsL71ly8MDkJVsZ/NFCBl1+Lo93Of1qqjm5yjcrdjD8hw3M\n3bCHqHKh9EyKZ8Al9alfPaoEIs7L5i8YU/pZcgggm3YfputrP9AgtiLj7rr4jG8es3TLfobP3sCk\nxVvJzlU6Norljvb1ufjcklsmxL9/4Y5LE3i8i/UvGFMaWXIIEEezc7jpzTmk7DnCF/e3J65q5FmX\ntfNABh/+tIkP525mz+FMGteK5o5L63N989pEhBffUFj//oXnbmzKjS2tf8GY0sqSQ4AYMnEZ783Z\nyNu3JXFVk/z7Gc5URlYOny/awvAfNrJqx0GqVyxHv7b16N+uXpHfl8D6F4wpWyw5BIAvf9nG3aMW\nMLB9An+7rkmRl6+qzFm3m3d/2MCMlTspFxpCt8Ta3HFpAk1qRxeq7KycXP45ZQUjZm+k3bnVGNq3\npfUvGFMG/FZyKFBDsYh0FpFVIrJWRB7NZ39dEZkpIgtFZImIdPFtDxeR90XkFxFZISKPna5MEUkQ\nkbm+7WNFpNyZX3Jg2bw7nUcmLKF5fBX+0rlRsZxDRLj0/OoMH9CaGQ/9jl6t4/liyTa6vPY9fYb9\nxLTlO8jNPfMvArsPHeXWd+cyYvZG7rg0gQ8HtrXEYEwQOG3NQURCgdXAVUAqMA/oo6rL/Y4ZBixU\n1TdFpAkwRVXri0hfoJuq9haRSGA5cAWQcqoyRWQc8ImqjhGRt4DFqvrmb8UYyDWHzOxcer41h/W7\nDjPl/suIr3b2/Qxnan96FmPmbeb9ORvZuj+D+jGRDLikPj2T4okqf/opLta/YEzZVtiaQxtgraqu\nV9VMYAzQ/aRjFDjWdlEZ2Oq3PUpEwoAKQCZw4FRlihtu0wEY73v9+0CPAsQYsJ77cgWLU/fz75ub\nl2hiAKgcGc5dvzuPbx+5ktf7tqBqVDmGTFpOu+em8+wXy0ndm37K136yIJWb35qDiDDh7kssMRgT\nZAoyQ7oO7pv+MalA25OOGQJMFZH7gCigk2/7eFwi2QZEAg+q6h4ROVWZMcA+Vc32214nv6BEZBAw\nCKBu3boFuIyS9/Wy7YyYvZEBl9Sn80XneBZHeGgI1zWrzXXNarNg815GzN7I8NkbefeHDXS+6Bzu\nuDSBVvWqIiLWv2CMAYpu+Yw+wHuq+pKIXAyMFJGLcDWEHKA2UBX4XkS+KYoTquowYBi4ZqWiKLMo\npexJ5+GPF9O0TmUe61I8/Qxno2XdqrSsW5XHrm3EBz9uYvTPm5nyy3aax1Wmf7t6TFiQyk/rbf6C\nMcGuIMlhCxDv9zzOt83fQKAzgKr+KCIRQHWgL/CVqmYBO0VkNpCEqzXkV+ZuoIqIhPlqD/mdK+Bl\nZucyePRCVGFo35aUDwu8JbhrV6nAo9c24v6O5zNhwRZGzN7Aw+OXUD4shP/c0tyakYwJcgVJDvOA\nBiKSgPug7o370Pe3GegIvCcijYEIIM23vQOuJhEFtANewXVM5ylTVVVEZgI34/ohbgc+L9wllrwX\nvlrJ4pR9vNmvJXVjSraf4UxFlgvj1nb16NemLrPX7SI2OoKGsZW8DssY47HTthn4vsEPBr4GVgDj\nVHWZiDwtIt18hz0E/EFEFgOjgQHqhkENBSqKyDJckhmhqktOVaavrL8AfxKRtbg+iHeL6mJLwjfL\nd/DODxu47eJ6XNu0ltfhFFhIiHBZgxqWGIwxgE2CK1Jb9h2hy6vfE1e1AhPuvqRYl7EwxpjCKvQk\nOHN6WTm5DP5oATm5ytC+LS0xGGNKNbvZTxF58etVLNy8j9f7tvBsGW1jjCkqVnMoAjNW7uB/362n\nX9u6XNesttfhGGNMoVlyKKSt+47wp3GLaVwrulgW1DPGGC9YciiErJxc7h+9kKzsXN7oZ/0Mxpiy\nw/ocCuE/01aTvGkvr/ZOJMH6GYwxZYjVHM7SzFU7eXPWOvq0iad7Yr7LPxljTKllyeEsbN+fwUPj\nFtPonEo8ef2FXodjjDFFzpLDGcr29TNkZOUw1PoZjDFllPU5nKFXvlnDzxv38HKv5pxXo6LX4Rhj\nTLGwmsMZ+G51GkNnraVXUjw3tLBVS40xZZclhwLacSCDB8cuomHNSgzpZv0MxpiyzZJDARzrZ0jP\nzGFovxZUKGf9DMaYss36HArgtelrmLthDy/2bM75NW1Ja2NM2Wc1h9P4Yc0u/jtzLTe3iuPmVtbP\nYIwJDpYcfsPOgxn839hFnF+jIk93t34GY0zwsGalU8jJVR4YvYhDR7P46A9tiSxnb5UxJnjYJ94p\n/HfGGn5cv5sXbmpmt840xgQda1bKx5x1u3h1+hpubFGHnknWz2CMCT6WHE6SdvAoD4xZxLnVo/hH\nj4sQEa9DMsaYEleg5CAinUVklYisFZFH89lfV0RmishCEVkiIl182/uJyCK/n1wRSRSRSidt3yUi\nr/heM0BE0vz23Vm0l3xqObnKg2MXceBIFkP7tSSqvLW6GWOC02mTg4iEAkOBa4EmQB8ROfmWZ08A\n41S1BdAbeANAVUepaqKqJgK3AhtUdZGqHjy23bdvE/CJX3lj/fa/U+irLKA3Zq7lh7W7eKrbhTQ6\nJ7qkTmvMqaXOhw9vhrRVXkdigkxBag5tgLWqul5VM4ExQPeTjlHg2KdpZWBrPuX08b32BCLSEKgJ\nfF/QoIvDT+t38/I3q+meWJtereO9DMUYJycLPr8X1k6Dd6+GzT95HZEJIgVJDnWAFL/nqb5t/oYA\n/UUkFZgC3JdPOb2A0fls742rKajftpt8zVPjRSTfT2oRGSQiySKSnJaWVoDLOLVdh45y/+iF1I+J\n4tkbmlo/gwkMc/8HaSug878gMgY+6A4rJnsdlQkSRdUh3Qd4T1XjgC7ASBE5XraItAXSVXVpPq/t\nzYlJYxJQX1WbAdOA9/M7oaoOU9UkVU2qUaPGWQee6+tn2Hcki9f7tqSi9TOYQHBgK8x6DhpcA23v\ngoHTIPYiGHcrzCuxllYTxAqSHLYA/t/e43zb/A0ExgGo6o9ABFDdb//JCQAAEWkOhKnq/GPbVHW3\nqh71PX0HaFWAGM/am9+u4/s1u3jy+iY0qW39DCZATH3CNStd+zyIQFQM3D4JGlwNXzwE05+GEyrb\nxhStgiSHeUADEUkQkXK4D/qJJx2zGegIICKNcckhzfc8BLiFfPobcDWOE5KGiNTye9oNWFGAGM/K\nzxv28NLUVVzXrBZ929QtrtMYc2bWz4KlE+CyP0G1c3/dXi4Seo2ClrfD9y/BZ/e4BGJMMThtG4qq\nZovIYOBrIBQYrqrLRORpIFlVJwIPAW+LyIO4zukBfn0IlwMpqro+n+JvwTVD+btfRLoB2cAeYMBZ\nXFeBpOxJ5/yaFXnuRutnMAEiOxOmPAxV68OlD+TdHxoG178K0XVg1j/h0A645QMob3clNEVLtAxU\nTZOSkjQ5OfmsXpudk0tYqM0FNAHih5fhmyHQdxw0vOa3j53/Pkx+EM5pCv0+hoo1SyREU3aIyHxV\nTcpvX9B/KlpiMAFjfyp8+wJc0PX0iQGg1e3Q+yM3B+Ldq2D3uuKP0TiHd5f5Jj37ZDQmUHz1mOtk\n7vxcwV9zQWcYMBkyDrgEkTr/9K8xhbN9KbyW6IYWZ2d6HU2xseRgTCBY+w2smAiXPwRV653Za+OS\n3FDXchXh/etg9dTiidHA/i0wqidICGyaDVP+XGZHjVlyMMZr2UdhyiNQ7Ty45P6zK6P6+S5BVG8A\no3vDgpFFG6NxtbOPboGjB11t7bKHYMH78PMwryMrFjbjyxivzXkN9qyD/p9AWPmzL6dSLAz4Asbd\nBhMHw8FtcPnDbp6EKZycLPj4dti5AvqNc4MAal4IO1fCV4+6pHxeB6+jLFJWczDGS3s3wXcvQZPu\ncH7HwpdXvhL0GQvNesPMZ91oppzswpcbzFTd+7huBlz/CpzfyW0PCYEb/wc1GsPHA2DXWk/DLGqW\nHIzx0lePufbra/5ZdGWGlYMb3oL2f4L5I9ySG5npRVd+sPn+RVg40tXCWt524r7ylaDPaAgJg9G9\n4Mg+b2IsBpYcjPHK6q9h1eA0dMwAABxESURBVBfwu0egchHfcVAEOj0J1/4bVn3pRtak7ynacwSD\nJeNgxjPQrBdc+df8j6laD24ZCXs3wvg7ykxNzZKDMV7IOuJmQle/ANrdU3znaTvIzaDettgt+713\nU/Gdq6zZ8J1boqT+ZdDt9d/uu6l/KXT9D6ybDtP+XnIxFiNLDsZ44YdXYN8m6PJv1wxUnJp0g9s+\nh8M73VyIbYuL93xlwc6VMKa/W9uq18iC/R+1uh3a/hF+GlomRotZcjCmpO1Z75bJuOgmOPd3JXPO\nehfDHVMhJBxGdIV1M0vmvKXRwR1uLkNYebcsSYWqBX/t1c/CuVe6DuxNPxZfjCXAkoMxJUkVvvwL\nhIa7D5KSVLMR3DkNqtSFUTe79nRzoqOH3FyG9F3Qd+yZT0gMDYOeI9zrxvaHfZuLJ84SYMnBmJK0\nagqsmQpXPAbRtU5/fFGLrg13fAl1L4ZP/uCat8roDN8zlpMNEwbC9iVw8wio0/LsyqlQFfqMcXMj\nRvdxCacUsuRgTEnJTIcvH4WaTdzd3bwSURn6T4ALb4RvnnSTuHJzvIsnEKjCV3+B1V+5fqALOheu\nvOoNoOdw2LkcPr0LcnOLJs4SZMnBmJLy/UuwfzN0edE1K3kprDzc9C60uxfmvgXjfw9ZGd7G5KU5\n/3W3X73kfmh9Z9GUeX4n13S4crK790YpY8tnGFMSdq11y2Q06+2GPQaCkBDo/E/X1DT1r24Z6t6j\noEIVryMrWUs/gWl/gwtvgE5PFW3Z7e52tYfv/g01G7tBCKWE1RyMKW6q8OXDEBYBVz3tdTR5XTLY\n1SJS5sLwzm7l0WCx+Sf49I8Q3w56vOUSZlEScfMf6l7s5kxsWVC05RcjSw7GFLfln7t1eTo84RbH\nC0RNb3b9EPtT3VyIHcu9jqj47VrrVrCtHOeWwAiPKJ7zhJVzM6ijasCYvnBgW/Gcp4hZcjCmOB09\nBF8/7lbxTBrodTS/7dzfuZFMuTkwojNsnO11RMXn8C4YdRNIKPQfD5HVivd8FWu4BJRxAMb2czPk\nA5wlB2OK03cvwIEtrmkhtBR08Z3T1M2FqBgLI3vAss+8jqjoZR1xNYaD292Q02rnlsx5z2nqVnHd\nMh8m3h/wQ4gLlBxEpLOIrBKRtSLyaD7764rITBFZKCJLRKSLb3s/EVnk95MrIom+fbN8ZR7bV9O3\nvbyIjPWda66I1C+6yzWmBKWtgh+HQov+EN/G62gKrkpduONrqN3CLUU9939eR1R0cnNgwp2Qmgw3\nvQPxrUv2/I2vd82Lv4yD2a+U7LnP0GmTg4iEAkOBa4EmQB8RaXLSYU8A41S1BdAbeANAVUepaqKq\nJgK3AhtUdZHf6/od26+qO33bBgJ7VfV84GXgX4W4PmO8oQpfPORu3VnUI2BKQmQ1tx5To67w5SNu\nMblSOFY/j6lPuKGl1/zTfVB74bI/u1FL3zzlVswNUAWpObQB1qrqelXNBMYA3U86RoFo3+PKwNZ8\nyunje+3pdAfe9z0eD3QUsVtZmVJm6QTY+D10/DtEVfc6mrMTXsGt6Jo0EGa/CuMHuA7r0uqnt+Cn\nN9zieBcX40q4pyMC3YdC7URXiwnQzv+CJIc6QIrf81TfNn9DgP4ikgpMAe7Lp5xewOiTto3wNSn9\nzS8BHD+fqmYD+4GYAsRpTGDIOABf/9U1y7Qa4HU0hRMSCl1fgk5D3Lfc11q4pcZLyYib41ZMdjPB\nG11XtDdWOlvhFaD3R65mObqXm2MSYIqqQ7oP8J6qxgFdgJEicrxsEWkLpKvqUr/X9FPVpsBlvp9b\nz+SEIjJIRJJFJDktLa3wV2BMUfn2X3Boh/tQDQn1OprCE4H2D8J9CyCxLyQPh9cS3V3sDu08/eu9\nljrffUOv0wpufDtw/k+ia7sEcXCHu+93dqbXEZ2gIMlhCxDv9zzOt83fQGAcgKr+CEQA/nXp3pxU\na1DVLb5/DwIf4ZqvTjifiIThmqnypFVVHaaqSaqaVKNGjQJchjElYMcy+OlNt7Z/nVZeR1O0qsTD\n9a/CffPhoptdR/WrzV1/RAB+8wVgzwa3ymqlWDcyqVyk1xGdKK4VdH8dNv3gJkoG0AimgiSHeUAD\nEUkQkXK4D/qJJx2zGegIICKNcckhzfc8BLgFv/4GEQkTkeq+x+HAdcCxWsVE4Hbf45uBGaoB9I4Z\ncyqq8MWf3cJ2HZ/0OpriU7U+9BgKg+e5Tt3Zr8GrzWD6PwLrVqTpe9x9GTQH+o13cw0CUbNbXM1s\n/nvw89teR3PcaZODr91/MPA1sAI3KmmZiDwtIt18hz0E/EFEFuNqCAP8PtAvB1JUdb1fseWBr0Vk\nCbAIV1s49q68C8SIyFrgT0CeobPGBKQlY2HzHNc+X9yTqgJBzHlw4zC4dy40uBq+f9HVJGY9Dxn7\nvY0tK8PNRt63yTXdVG/gbTyn0+Hv0PBa1y8SIDdikrLwpTwpKUmTk5O9DsMEsyP74PUkqFIPBk4r\n+jV6SoMdy2DWc7BiEkRUgUvuc0uTl69UsnHk5rr7Miz7BG4eXnoWuzt60N3n+8AW+MNMl3yLmYjM\nV9Wk/PYF4W+wMcVg5j8hfbevEzpI/6xiL4ReH8Jd37mF5mb8w9UkZr8KmYdLLo7pT7nE0Omp0pMY\nwCXRPqPdkh4f9XJfODwUpL/FxhShbUtg3ttuPkDtRK+j8V6t5tB3DPxhhhvOO+3vLkn8+EbxrymU\nPNzNPE66Ay59oHjPVRyq1odeI2HvBlf78fAmTJYcjCmM3Fw3E7pCNejwV6+jCSx1WrmVXu+Y6u5+\n9/Vjbp7Ez29D9tGiP9/qqe7/osE1cO2/3RDc0qh+e3dDqLXfuMTqEUsOxhTG4o8g9We4+h/u3sEm\nr7pt4faJMOALqJoAU/4Mr7WE5BFFN7Z/60K3DtQ5TV0/Q2lY5PC3JP0e2twFP74OC0Z6EoIlB2PO\nVvoe980uvp27w5v5bfXbw++nwK2fQXQtmPx/8HorWPgh5GSffbn7Nrs2+shq0HcclK9YdDF76Zp/\nwrlXwOQH3U2JSpglB2PO1oxnXKdhMHdCnykROO9KN6Kr33iIjIHP74WhrWHx2DNvYz+yz81lyMqA\nfh9DpXOKJ24vhIZBz/fcKrlj+7skWILsN9qYs7Flgev8bDMIzrnI62hKHxFocJUbstl7NIRHwaeD\n4I12btHCgqwAm53pPjR3r4PeH7p7NJc1Faq6md3ZmTC6r7t5VAmx5GDMmTrWCV2xJlz5mNfRlG4i\n0KiLG/56ywduGOf4O+CtS2H5xFMnCVWYeJ9b+bb765BwecnGXZJqNHT9KDuXwWd/LLGl0y05mMCS\nm+tukLPoo8BaisHfgvdh6wK4+hm3VIYpvJAQaNId7p4DN70LOVkw7lYYdrlbDfbkybqznoMlY+DK\nv0LzIOjvadDJ/b6tmOSuvQSU8i59U6aowld/gZ+HuecS6joxG1/vllqOruVtfOAWmJv+FNRrD017\neh1N2RMSAk1vhiY9YOl4txTH6N5Qu6VLBOd3dB3Y3/7L3WHv8oe9jrjktLvH3fvhuxegZqNin+Bn\ny2eYwDHrefet6OLB7hd/xST3s3uN2x/X+tdEUQJLC+Rr4n2uVvPHH8pmG3egycmCxWPg2xdg/2Y3\nqW77L1D/MtcBHRrudYQlK/sovN8Nti2CO75y70ch/NbyGZYcTGCYO8wtWZzYz90ly38CU9oqWDHR\n3bBlm+8uszUvhMbXuWQRe1HJTHhKmQfvdnJrBl39TPGfz/wqOxMWfQjfvehGOA34AiKiT/+6suhQ\nGrx9pRvZNWhmoUZoWXIwgW3Jx/DJnXBBF7hl5G9PYNq32SWJlZNh0xxA3ZIDja6Dxt1c7aI4hpXm\n5rg/yEM73VLVJb2YnHFyc1zzY2mf5FZY25bA8GvczPMBX0B4xFkVY8nBBK7VU2FMHzeRrP+EM/sl\nP5QGq6a4pqf1syA3CyrGQqOuLlkkXF50zQ4/v+1m9t48Ai66sWjKNKYwlk90nfYdn4TL/nRWRVhy\nMIFp80/wQQ83VO/2yYVrJsjYD2umueanNdMgK92NJGp4rWt+Oq/j2d8F7FCam8lbu4Wb3Vta1+wx\nZc+aaW4W9Vl+Cfqt5BDkdTPjme1L3e0bo2tDvwmFbz+OqOxGuTS92a38uW6mq1Gs/tINeQyr4IYD\nNroeGl4DFaoUvOxvnoTM9NK9mJspmxpcVWxFW3IwJW/PBvjwRjcr9rbPiv72jeEV3MSqRl3caJdN\ns3/tp1gxCULCXJNT4+vhgq7u/sKnsvknWDTK3caxRsOijdOYAGbNSqZkHdzuOtIy9sPvv3LjtUtK\nbq6bvLZioksSe9YDAvFtXdNTo+ugWsKvx+dkw7DfufV7Bv8M5aJKLlZjSoA1K5nAcGQvjLzRteHf\nPrFkEwO4UUxxSe6n01Owc4VLEisnwdQn3M85TV3TU+PrYcO3sGOpG0FlicEEGas5mJKRmQ4jb4At\n86HfODivg9cRnWjPBlj5hUsWKXMB39/F+Z3c6qHW12DKIKs5GG/lZMHHt7sP3Z4jAi8xgGtOumSw\n+zm4A1Z94fobrvyrJQYTlAo0W0hEOovIKhFZKyKP5rO/rojMFJGFIrJERLr4tvcTkUV+P7kikigi\nkSLyhYisFJFlIvK8X1kDRCTN7zV3Ft3lmhKXmwuf3Q1rpsJ1L8OFN3gd0elVinX3IL5xGFSt53U0\nxnjitMlBREKBocC1QBOgj4g0OemwJ4BxqtoC6A28AaCqo1Q1UVUTgVuBDarqW/+AF1W1EdACuFRE\nrvUrb+yx16nqO4W5QOOhYwvp/fIxdPy7u/WhMaZUKEjNoQ2wVlXXq2omMAboftIxChwbqF4Z2JpP\nOX18r0VV01V1pu9xJrAAiDvz8E1A+/ZfboXViwdD+7ObwWmM8UZBkkMdIMXveapvm78hQH8RSQWm\nAPflU04vYPTJG0WkCnA9MN1v802+5qnxIhKfX1AiMkhEkkUkOS0trQCXYUrU3GFuhdXEfm6ROmu3\nN6ZUKaoVyvoA76lqHNAFGCkix8sWkbZAuqou9X+RiIThEsZrqrret3kSUF9VmwHTgPfzO6GqDlPV\nJFVNqlGjiCdRmcJZ8rFbYfWCLnD9a5YYjCmFCpIctgD+397jfNv8DQTGAajqj0AEUN1vf2/yqTUA\nw4A1qvrKsQ2qultVj/qevgO0KkCMJlCsmeZuZVivvVukLthXzzSmlCpIcpgHNBCRBBEph/ugn3jS\nMZuBjgAi0hiXHNJ8z0OAW/D1NxwjIs/g+if+76Tt/rf76gasKOjFGI9t/gnG3gqxF0Kf0We9jLAx\nxnun/VqnqtkiMhj4GggFhqvqMhF5GkhW1YnAQ8DbIvIgrnN6gP46u+5yIMWv2QgRiQP+CqwEFohr\ndnjdNzLpfhHpBmQDe4ABRXOpplgV9UJ6xhhP2QxpU3h7Nrj1kiQUBn4NVep6HZExpgBshrQpPge3\nw8gekJPpFtKzxGBMmWDJwZw9rxfSM8YUG0sO5uxkpsNHvWHXareQXly+NVNjTCllycGcuRMW0nsv\nMBfSM8YUiiUHc2ZOWEjvFbiwh9cRGWOKQVHNkDbBwBbSMyZoWHIwBfftC7aQnjFBwpKDKZi5w2DW\nP20hPWOChCUHc3rHF9LragvpGRMkLDmY33bCQnrDbSE9Y4KEJQdzaraQnjFBy5KDyd+xhfQq17GF\n9IwJQpYcTF57NsCHN0J4FNz6KVS0mykZE2ysAdmcyBbSM8ZgycH4O2EhvUm2kJ4xQcyalYyzaQ6M\n6OoW0us9CuLs7qzGBDOrOQS7fZth2t9h2acQXceNSjrvSq+jMsZ4zJJDsDp6CGa/AnP+Cwhc8Rhc\ncj+Ui/Q6MmNMALDkEGxyc2HJWJj+FBzcBk17QqchUDnO68iMMQHEkkMwSfkZvnoUtsyH2i3hlg8g\nvo3XURljAlCBOqRFpLOIrBKRtSLyaD7764rITBFZKCJLRKSLb3s/EVnk95MrIom+fa1E5Bdfma+J\nuAV7RKSaiEwTkTW+f6sW5QUHpf2pMOFOePcqOLAVbvgf3DndEoMx5pROmxxEJBQYClwLNAH6iEiT\nkw57Ahinqi2A3sAbAKo6SlUTVTURuBXYoKqLfK95E/gD0MD309m3/VFguqo2AKb7npuzkZkOM5+D\n/ybBiklw+cMwOBma94YQG6hmjDm1gjQrtQHWqup6ABEZA3QHlvsdo8Cx9RUqA1vzKacPMMZXRi0g\nWlV/8j3/AOgBfOkr+wrfa94HZgF/KegFGdxNeX4ZD988CQe2wIU3wFVP24Q2Y0yBFSQ51AFS/J6n\nAm1POmYIMFVE7gOigE75lNML98F/rMzUk8qs43scq6rbfI+3A7H5BSUig4BBAHXr2ofecanzXb9C\n6s9QKxFuehfqXex1VMaYUqao2hb6AO+pahzQBRgpIsfLFpG2QLqqLj2TQlVVcbWS/PYNU9UkVU2q\nUcPW/uHAVvjkLninA+zbBN2Hwh9mWmIwxpyVgtQctgDxfs/jfNv8DcTXZ6CqP4pIBFAd2Onb3xsY\nfVKZ/mMn/cvcISK1VHWbr/lpJ+bUso7AnNfhh/9Abja0fxAuewjKV/I6MmNMKVaQmsM8oIGIJIhI\nOdwH/cSTjtkMdAQQkcZABJDmex4C3IKvvwHA12x0QETa+UYp3QZ87ts9Ebjd9/h2v+3Gnyos/QRe\nbwMzn4HzO8K9P7s5C5YYjDGFdNqag6pmi8hg4GsgFBiuqstE5GkgWVUnAg8Bb4vIg7hmoAG+JiGA\ny4GUYx3afu4B3gMq4Dqiv/Rtfx4YJyIDgU24xGL8bV0IXz0Gm3+E2KbQYzIkXOZ1VMaYMkR+/Qwv\nvZKSkjQ5OdnrMIrfwR0w/WlYNAoiY6Dj36DFrRAS6nVkxphSSETmq2pSfvtshnRpkJUBP70B378E\n2UfhksFuzkJEZa8jM8aUUZYcApkqrJgIU//mRiBd0BWu/gfEnOd1ZMaYMs6SQ6DatsT1K2z6AWo2\ngds+h3Ov8DoqY0yQsOQQaA7thBnPwIIPoEJV6PoStBwAofZfZYwpOfaJEyiyj8Lct+Dbf0P2EWh3\nD/zuYZcgjDGmhFlyCARrv4Ev/gx7N0CDa+CaZ6F6A6+jMsYEMUsOXspMh2l/g3nvQPWG0H8CnJ/f\nslTGGFOyLDl4ZetCmPAH2L0GLh4MHf4G4RFeR2WMMYAlh5KXmwM/vAyznoOomjYKyRgTkCw5lKS9\nG93KqSk/uXssdP0PRFbzOipjjMnDkkNJUIXFo2HKIyACNwyDZre4x8YYE4AsORS39D0w6QE307le\ne7jhTbsjmzEm4FlyKE5rv4HP7oX03dDpKbjkPlskzxhTKlhyKA5ZR2Dak/Dz/6BGI+j3MdRq5nVU\nxhhTYJYcitq2xW6I6q5V0PZu6PQkhFfwOipjjDkjlhyKSm4OzHkNZjwLUdXh1k/hvA5eR2WMMWfF\nkkNR2LsJPv0jbJ4DTbrDda/YEFVjTKlmyaEwVGHJWJjysHvc4y1o3tuGqBpjSj1LDmcrfQ9MfhCW\nfwZ1L4Yb/gdV63kdlTHGFAlLDmdj3Uz47G44nAYdn4RLH7AhqsaYMiWkIAeJSGcRWSUia0Xk0Xz2\n1xWRmSKyUESWiEgXv33NRORHEVkmIr+ISISIVBKRRX4/u0TkFd/xA0QkzW/fnUV3uYWUleHuzjay\nB5SPhjunw2V/ssRgjClzTltzEJFQYChwFZAKzBORiaq63O+wJ4BxqvqmiDQBpgD1RSQM+BC4VVUX\ni0gMkKWqGUCi3znmA5/4lTdWVQcX9uKK1PZf3BDVtBXQ5i646ikbomqMKbMK0qzUBlirqusBRGQM\n0B3wTw4KRPseVwa2+h5fDSxR1cUAqrr75MJFpCFQE/j+bC6g2OXmwo+vw4x/uLuy2T0XjDFBoCDN\nSnWAFL/nqb5t/oYA/UUkFVdruM+3vSGgIvK1iCwQkUfyKb83rqagfttu8jVPjReR+PyCEpFBIpIs\nIslpaWkFuIyzsC8FPujmbsjT8Bq4+0dLDMaYoFCgPocC6AO8p6pxQBdgpIiE4Gom7YF+vn9vEJGO\nJ722NzDa7/kkoL6qNgOmAe/nd0JVHaaqSaqaVKNGjSK6DD9LPoY3L3U35ek+FG4ZCVExRX8eY4wJ\nQAVJDlsA/2/vcb5t/gYC4wBU9UcgAqiOq2V8p6q7VDUdV6toeexFItIcCFPV+ce2qepuVT3qe/oO\n0OqMrqiwjuyF8XfAJ3dCzcbwxx+gRX+bu2CMCSoFSQ7zgAYikiAi5XDf9CeedMxmoCOAiDTGJYc0\n4GugqYhE+jqnf8eJfRV9OLHWgIjU8nvaDVhR8MsppPXfutrC8s/dbTt/PwWqJZTY6Y0xJlCctkNa\nVbNFZDDugz4UGK6qy0TkaSBZVScCDwFvi8iDuM7pAb4+hL0i8h9cglFgiqp+4Vf8LbhmKH/3i0g3\nIBvYAwwo1BUWRPZRmP6063iOaQADp0Gdlqd/nTHGlFFyYj9w6ZSUlKTJycln9+Idy9wQ1Z3LoPWd\ncNU/oFxk0QZojDEBSETmq2pSfvuCe4b0wlEw+f8gogr0/RgaXu11RMYYExCCOznEnOeGqF73iltm\n2xhjDBDsyaFuO/djjDHmBEU1z8EYY0wZYsnBGGNMHpYcjDHG5GHJwRhjTB6WHIwxxuRhycEYY0we\nlhyMMcbkYcnBGGNMHmVibSURSQM2neXLqwO7ijCc0s7ejxPZ+/Erey9OVBbej3qqmu8NccpEcigM\nEUk+1cJTwcjejxPZ+/Erey9OVNbfD2tWMsYYk4clB2OMMXlYcoBhXgcQYOz9OJG9H7+y9+JEZfr9\nCPo+B2OMMXlZzcEYY0welhyMMcbkEdTJQUQ6i8gqEVkrIo96HY9XRCReRGaKyHIRWSYiD3gdUyAQ\nkVARWSgik72OxWsiUkVExovIShFZISIXex2TV0TkQd/fyVIRGS0iEV7HVByCNjmISCgwFLgWaAL0\nEZEm3kblmWzgIVVtArQD7g3i98LfA8AKr4MIEK8CX6lqI6A5Qfq+iEgd4H4gSVUvAkKB3t5GVTyC\nNjkAbYC1qrpeVTOBMUB3j2PyhKpuU9UFvscHcX/4dbyNylsiEgd0Bd7xOhaviUhl4HLgXQBVzVTV\nfd5G5akwoIKIhAGRwFaP4ykWwZwc6gApfs9TCfIPRAARqQ+0AOZ6G4nnXgEeAXK9DiQAJABpwAhf\nM9s7IhLldVBeUNUtwIvAZmAbsF9Vp3obVfEI5uRgTiIiFYEJwP+p6gGv4/GKiFwH7FTV+V7HEiDC\ngJbAm6raAjgMBGUfnYhUxbUwJAC1gSgR6e9tVMUjmJPDFiDe73mcb1tQEpFwXGIYpaqfeB2Pxy4F\nuonIRlxzYwcR+dDbkDyVCqSq6rHa5HhcsghGnYANqpqmqlnAJ8AlHsdULII5OcwDGohIgoiUw3Uq\nTfQ4Jk+IiODak1eo6n+8jsdrqvqYqsapan3c78UMVS2T3w4LQlW3AykicoFvU0dguYcheWkz0E5E\nIn1/Nx0po53zYV4H4BVVzRaRwcDXuBEHw1V1mcdheeVS4FbgFxFZ5Nv2uKpO8TAmE1juA0b5vkit\nB37vcTyeUNW5IjIeWIAb5beQMrqMhi2fYYwxJo9gblYyxhhzCpYcjDHG5GHJwRhjTB6WHIwxxuRh\nycEYY0welhyMMcbkYcnBGGNMHv8PpgU3tzJbFv0AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HzIcYAfvkUZ_",
        "colab_type": "text"
      },
      "source": [
        "###Predicting testing labels\n",
        "\n",
        "Predictions are saved to *output_logistic.csv*.\n",
        "\n",
        "預測測試集的資料標籤並且存在 *output_logistic.csv* 中。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZEAKhugPkUyH",
        "colab_type": "code",
        "outputId": "97c3eb12-a9c5-4c43-bc62-54f7f5f4797d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# Predict testing labels\n",
        "predictions = _predict(X_test, w, b)\n",
        "with open(output_fpath.format('logistic'), 'w') as f:\n",
        "    f.write('id,label\\n')\n",
        "    for i, label in  enumerate(predictions):\n",
        "        f.write('{},{}\\n'.format(i, label))\n",
        "\n",
        "# Print out the most significant weights\n",
        "ind = np.argsort(np.abs(w))[::-1]\n",
        "with open(X_test_fpath) as f:\n",
        "    content = f.readline().strip('\\n').split(',')\n",
        "features = np.array(content)\n",
        "for i in ind[0:10]:\n",
        "    print(features[i], w[i])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " Not in universe -4.031960278019251\n",
            " Spouse of householder -1.625403958705141\n",
            " Other Rel <18 never married RP of subfamily -1.4195759775765404\n",
            " Child 18+ ever marr Not in a subfamily -1.2958572076664745\n",
            " Unemployed full-time 1.1712558285885912\n",
            " Other Rel <18 ever marr RP of subfamily -1.167791807296237\n",
            " Italy -1.093458143800618\n",
            " Vietnam -1.0630365633146415\n",
            "num persons worked for employer 0.938992277356651\n",
            " 1 0.8226614922117185\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1C6sqhUbLMGe",
        "colab_type": "text"
      },
      "source": [
        "# Porbabilistic generative model\n",
        "\n",
        "In this section we will discuss a generative approach to binary classification. Again, we will not go through the formulation detailedly. Please find [Prof. Lee's lecture](https://www.youtube.com/watch?v=fZAZUYEeIMg) if you are interested in it.\n",
        "\n",
        "接者我們將實作基於 generative model 的二元分類器，理論細節請參考[李宏毅老師的教學影片](https://www.youtube.com/watch?v=fZAZUYEeIMg)。\n",
        "\n",
        "### Preparing Data\n",
        "\n",
        "Training and testing data is loaded and normalized as in logistic regression. However, since LDA is a deterministic algorithm, there is no need to build a development set.\n",
        "\n",
        "訓練集與測試集的處理方法跟 logistic regression 一模一樣，然而因為 generative model 有可解析的最佳解，因此不必使用到 development set。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "czWXO7qML8DU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Parse csv files to numpy array\n",
        "with open(X_train_fpath) as f:\n",
        "    next(f)\n",
        "    X_train = np.array([line.strip('\\n').split(',')[1:] for line in f], dtype = float)\n",
        "with open(Y_train_fpath) as f:\n",
        "    next(f)\n",
        "    Y_train = np.array([line.strip('\\n').split(',')[1] for line in f], dtype = float)\n",
        "with open(X_test_fpath) as f:\n",
        "    next(f)\n",
        "    X_test = np.array([line.strip('\\n').split(',')[1:] for line in f], dtype = float)\n",
        "\n",
        "# Normalize training and testing data\n",
        "X_train, X_mean, X_std = _normalize(X_train, train = True)\n",
        "X_test, _, _= _normalize(X_test, train = False, specified_column = None, X_mean = X_mean, X_std = X_std)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L8NGKl-EPvok",
        "colab_type": "text"
      },
      "source": [
        "### Mean and Covariance\n",
        "\n",
        "In generative model, in-class mean and covariance are needed.\n",
        "\n",
        "在 generative model 中，我們需要分別計算兩個類別內的資料平均與共變異。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iQrzXXKUPwHT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Compute in-class mean\n",
        "X_train_0 = np.array([x for x, y in zip(X_train, Y_train) if y == 0])\n",
        "X_train_1 = np.array([x for x, y in zip(X_train, Y_train) if y == 1])\n",
        "\n",
        "mean_0 = np.mean(X_train_0, axis = 0)\n",
        "mean_1 = np.mean(X_train_1, axis = 0)  \n",
        "\n",
        "# Compute in-class covariance\n",
        "cov_0 = np.zeros((data_dim, data_dim))\n",
        "cov_1 = np.zeros((data_dim, data_dim))\n",
        "\n",
        "for x in X_train_0:\n",
        "    cov_0 += np.dot(np.transpose([x - mean_0]), [x - mean_0]) / X_train_0.shape[0]\n",
        "for x in X_train_1:\n",
        "    cov_1 += np.dot(np.transpose([x - mean_1]), [x - mean_1]) / X_train_1.shape[0]\n",
        "\n",
        "# Shared covariance is taken as a weighted average of individual in-class covariance.\n",
        "cov = (cov_0 * X_train_0.shape[0] + cov_1 * X_train_1.shape[0]) / (X_train_0.shape[0] + X_train_1.shape[0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kifW1pFxXXA5",
        "colab_type": "text"
      },
      "source": [
        "### Computing weights and bias\n",
        "\n",
        "Directly compute weights and bias from in-class mean and shared variance. [Prof. Lee's lecture slides](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Classification%20(v3).pdf)(p.33) gives a concise explanation.\n",
        "\n",
        "權重矩陣與偏差向量可以直接被計算出來，算法可以參考[李宏毅老師教學投影片](http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/Classification%20(v3).pdf)第 33 頁。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UghOxYrUXXPU",
        "colab_type": "code",
        "outputId": "c71a5d93-4c19-48e5-b0ea-38c3e1e132ca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Compute inverse of covariance matrix.\n",
        "# Since covariance matrix may be nearly singular, np.linalg.inv() may give a large numerical error.\n",
        "# Via SVD decomposition, one can get matrix inverse efficiently and accurately.\n",
        "u, s, v = np.linalg.svd(cov, full_matrices=False)\n",
        "inv = np.matmul(v.T * 1 / s, u.T)\n",
        "\n",
        "# Directly compute weights and bias\n",
        "w = np.dot(inv, mean_0 - mean_1)\n",
        "b =  (-0.5) * np.dot(mean_0, np.dot(inv, mean_0)) + 0.5 * np.dot(mean_1, np.dot(inv, mean_1))\\\n",
        "    + np.log(float(X_train_0.shape[0]) / X_train_1.shape[0]) \n",
        "\n",
        "# Compute accuracy on training set\n",
        "Y_train_pred = 1 - _predict(X_train, w, b)\n",
        "print('Training accuracy: {}'.format(_accuracy(Y_train_pred, Y_train)))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training accuracy: 0.8693232084930699\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RDKWzBy0bi3c",
        "colab_type": "text"
      },
      "source": [
        "###Predicting testing labels\n",
        "\n",
        "Predictions are saved to *output_generative.csv*.\n",
        "\n",
        "預測測試集的資料標籤並且存在 *output_generative.csv* 中。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "T3QjToT_Sq9J",
        "outputId": "955f3181-82a9-4701-d57f-5e524cacd34e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# Predict testing labels\n",
        "predictions = 1 - _predict(X_test, w, b)\n",
        "with open(output_fpath.format('generative'), 'w') as f:\n",
        "    f.write('id,label\\n')\n",
        "    for i, label in  enumerate(predictions):\n",
        "        f.write('{},{}\\n'.format(i, label))\n",
        "\n",
        "# Print out the most significant weights\n",
        "ind = np.argsort(np.abs(w))[::-1]\n",
        "with open(X_test_fpath) as f:\n",
        "    content = f.readline().strip('\\n').split(',')\n",
        "features = np.array(content)\n",
        "for i in ind[0:10]:\n",
        "    print(features[i], w[i])\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " 29 -9.5791015625\n",
            " Forestry and fisheries 9.53662109375\n",
            " Retail trade 8.13671875\n",
            " 7 -7.81689453125\n",
            " Finance insurance and real estate 7.6845703125\n",
            " 41 -7.138671875\n",
            " Agriculture 6.975830078125\n",
            " 34 -6.2109375\n",
            " 37 -5.69384765625\n",
            " 33 -5.6201171875\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}