DataScience short tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n",
    "\n",
    "<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this notebook we try to practice all the classification algorithms that we learned in this course.\n",
    "\n",
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-04-03 20:09:48--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 23101 (23K) [text/csv]\n",
      "Saving to: ‘loan_train.csv’\n",
      "\n",
      "100%[======================================>] 23,101      --.-K/s   in 0.002s  \n",
      "\n",
      "2019-04-03 20:09:48 (11.1 MB/s) - ‘loan_train.csv’ saved [23101/23101]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching package metadata .............\n",
      "Solving package specifications: .\n",
      "\n",
      "# All requested packages already installed.\n",
      "# packages in environment at /opt/conda/envs/DSX-Python35:\n",
      "#\n",
      "seaborn                   0.9.0                    py35_0    anaconda\n"
     ]
    }
   ],
   "source": [
    "# notice: installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9b75cb2ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9b75ef5208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9b75c8fc88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'], dtype=object)"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_rate = []\n",
    "for i in range(1,40):\n",
    "    knn = KNeighborsClassifier(n_neighbors=i)\n",
    "    knn.fit(X_train,y_train)\n",
    "    knn_pred = knn.predict(X_test)\n",
    "    error_rate.append(np.mean(knn_pred != y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'Error Rate')"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmsAAAGDCAYAAAB0s1eWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xl8VNX9//HXJwtbMC4J0lYFhaJV+KLWVEHb2vp1w7Zoq4JSXKpoAaGWqnXt8rV1qfzUlqpYi7ZSS9WqtVihKl30q0ArrriUlqAg4gKuEMIQyOf3x5l8iSEZZpK5984k7+fjMY/JzJxz7mfm3kw+Ofeec8zdEREREZHCVJJ0ACIiIiLSNiVrIiIiIgVMyZqIiIhIAVOyJiIiIlLAlKyJiIiIFDAlayIiIiIFTMmaiEgnZ2ZuZp9MOg4RaR8layLSJjN71czqzWxds9sNMcfwBTNrTG97rZktMbNv5FD/h2Z2R5Qx5srMTjezx5s9rjSzJ8zsXjMrb1H2F2Y2s5U2hppZysx2iiNmEUmOkjUR2ZavuHvvZrdJrRUys7JsnsskQ/lV7t4bqASmAL80s71yabtQmdmOwDxgOTDa3RtaFPk18DUzq2jx/KnAn9z93eijFJEkKVkTkXZJ9w49YWbXm9m7wA/beK7EzC4zs+Vm9raZzTSz7dNt7J4+RXemma0A/pppmx7MAd4FhjaL5Wdm9pqZfWhmT5nZ59LPHw1cAoxO98w9l35+ezO71czeMLPXzezHZlbaynv8RLpncadmz+1vZmvMrNzMPmlmj5rZB+nn7srxM6xOv+cXgbHuvqmV97wAeB04vlm9UmAMcHv68YFmtsDM3k+/pxvMrFsb2/y7mY1r9rhlL9+nzOwRM3s33Ys5Kpf3JCL5p2RNRDriIGAZsDNwRRvPnZ6+fREYAPQGWp5KPRTYGzgq08bSid9IoBpY2uylJ4H9gJ2AWcDvzayHu/8ZuBK4K90ruG+6/O3AJuCTwP7AkcA4WnD3VcACmiVKhCTpnnQP2I+Ah4EdgV2Bn2eKv4WdgEeBfwBnuHtjhrIzCT1pTQ4HyoG56cebCT2O1cBw4L+BiTnEAkC69+4Rwme4M3AycJOZDc61LRHJHyVrIrIt96d7bJpuZzV7bZW7/9zdN7l7fRvPfR24zt2Xufs64GLgpBanPH/o7nXN2mjpE2b2PlAP/AH4jrs/0/Siu9/h7u+kt3kt0B1o9TSpmfUFRgDfTm/zbeB64KQ2tj2LkLRgZpYuNyv9WgPQH/iEu29w98dbb6JVuwF7Ar/ybS/S/BvgUDPbNf34VGBW0ylTd3/K3Rem3/+rwC8ICXCuvgy86u6/Srf1NHAvcEI72hKRPFGyJiLbcpy779Ds9stmr73WSvmWz32CcD1Wk+VAGdB3G+00t8rddyBcszYNOKz5i2Z2npm9nD4d+T6wPaGXqTX9Cb1SbzQloITkZuc2yt8DDDezTwCfBxz43/Rr3wUM+KeZvWhmZ2zjfTT3HHA+MNfM9s9U0N1XAI8BY82sN3Ac6VOgAGa2p5n9yczeNLMPCb2Jbb3/TPoDBzVPzgnJ9sfa0ZaI5ElOF/+KiLTQWo9Qy+dWEZKAJv0IpyDfIpw6bKudrRt2T5nZhcASMzvO3e9PX592IeHU34vu3mhm7xGSqNbafg1IAdWtXSPWyjbfN7OHgVGEU7W/a+oJc/c3gbMAzOyzwDwze8zdl7bZ4Efb/pmZdQceMbMvuPsLGYrfDlwEvAG8ku71ajIdeAY42d3Xmtm3abs3rA7o1exx80TsNeBRdz8im/hFJB7qWRORqP0OmGJme6R7hZquIdtmotQad98IXAt8P/3UdoTkbzVQZmbfJ/TANXkL2N3MStL13yBcZ3ZtesqMEjMbaGaZThvOIpx6PJ4tp0AxsxObnZp8j5AYbs7x/VwD/IyQ6GUa4Xov4dTp/9CsVy1tO+BDYJ2ZfQqYkKGdZwmjS3tZmHvtzGav/QnY08xOSQ+gKDezz5jZ3rm8JxHJLyVrIrItD9hH51n7Q471byNcc/UY8AqwAZjcwZhuA/qZ2VeAhwgX2v+bcIp1Ax89rfr79P07ZtbUG3Uq0A14iZBk3QN8PMP2ZgODgLfc/blmz38G+IeZrUuXOdfdXwFInxb9ejZvxt1/BMwA/mJmA9soU8eWhO23LV4+nzDwYS3wSyDTqNTrgY2EJPb25m25+1rCYIuTCD2ibwI/IVwDKCIJsW1f1yoiIiIiSVHPmoiIiEgBU7ImIiIiUsCUrImIiIgUMCVrIiIiIgVMyZqIiIhIAes0k+JWV1f77rvvnnQYIiIiItv01FNPrXH3PtmU7TTJ2u67786iRYuSDkNERERkm8xs+bZLBToNKiIiIlLAlKyJiIiIFDAlayIiIiIFTMmaiIiISAFTsiYiIiJSwJSsiYiIiBQwJWsiIiIiBUzJWhZqa2HKxBR9K+spLWmkb2U9UyamqK1NOjIRERHp7JSsbcPcuTBsaB09Z0xj/tohpLwb89cOoeeMaQwbWsfcuUlHKCIiIp2ZuXvSMeRFTU2N53sFg9rakKjNXn84w1m41esLGMbIXvNY+HwFAwfmddMiIiLSiZnZU+5ek01Z9axlcMO1Kc5quKnVRA1gOAsZ1zCdG69PxRyZiIiIdBVK1jKYdUcjZzbcnLHMuIbpzPrN5pgiEhERka5GyVoGa9Z1pz+Z11ntxwrWrOsRU0QiIiLS1ShZy6C6d4rl9M9YZgX9qO69IaaIREREpKtRspbBmLEl3Fo+PmOZGeUTGHNKaUwRiYiISFejZC2DSed155flE1nAsFZfX8AwZpRP4Jwp3WOOTERERLoKJWsZDBwIM++pYGSveVxcPpVaBtBAGbUM4KLyqYzsNY+Z92jaDhEREYmOkrVtGDECFj5fQersyRxSuZielmK/0sWsOWkyC5+vYMSIpCMUERGRzkyT4oqIiIjETJPixuCddyCluXBFREQkYpEma2Z2tJktMbOlZnZRK6+PN7PFZvasmT1uZvs0e22omS0wsxfTZQpmMrOnnoKPfQzmzEk6EhEREensIkvWzKwUuBEYAewDnNw8GUub5e7/5e77AdcA16XrlgF3AOPdfTDwBaAhqlhzte++sMMOcNddSUciIiIinV2UPWsHAkvdfZm7bwTuBI5tXsDdP2z2sAJouoDuSOB5d38uXe4ddy+YNZ3KyuD44+GBB2D9+qSjERERkc4symRtF+C1Zo9Xpp/7CDM7x8xqCT1r30o/vSfgZvaQmT1tZt+NMM52GTUqJGoPPph0JCIiItKZRZmsWSvPbTX01N1vdPeBwIXAZemny4DPAl9P33/VzP57qw2YnW1mi8xs0erVq/MXeRYOPRT69oW77451syIiItLFRJmsrQR2a/Z4V2BVhvJ3Asc1q/uou69x9/XAHODTLSu4+y3uXuPuNX369MlT2NkpLYWZM+EnP4l1syIiItLFRJmsPQkMMrM9zKwbcBIwu3kBMxvU7OGXgP+kf34IGGpmvdKDDQ4FXoow1nY58kgYMCDpKERERKQzK4uqYXffZGaTCIlXKXCbu79oZpcDi9x9NjDJzA4njPR8DzgtXfc9M7uOkPA5MMfdC/LqsAcfhJdeggsuSDoSERER6Yy0gkEHnXsu/OIX8PbbUFkZ++ZFRESkCGkFgxiNHh1WMnjggaQjERERkc5IyVoHDRsGu+6qCXJFREQkGkrWOqikBE48ER56CN5/P+loREREpLNRspYHo0fD7rvDq68mHYmIiIh0NpGNBu1KDjwQ/vUvsNamARYRERHpAPWs5YFZuKVSsGFD0tGIiIhIZ6JkLU9WrAjLT82alXQkIiIi0pkoWcuT3XaDqiqtFSoiIiL5pWQtT8xg1CiYNw/eeSfpaERERKSzULKWR6NHw+bNcN99SUciIiIinYWStTzad18YNEinQkVERCR/NHVHHpnBT38KO+2UdCQiIiLSWShZy7Njjkk6AhEREelMdBo0AosWwfXXJx2FiIiIdAZK1iLwpz/BeefBG28kHYmIiIgUOyVrETjxRHCHe+9NOhIREREpdkrWIjB4cLhpVKiIiIh0lJK1iIweDY8/Dq+/nnQkIiIiUsyUrEXkoIOgd7cU++5ZT2lJI30r65kyMUVtbdKRiYiISDFRshaBuXPh61+t45zGafxj/RBS3o35a4fQc8Y0hg2tY+7cpCMUERGRYmHunnQMeVFTU+OLFi1KOgxqa2HY0Dpmrz+c4Szc6vUFDGNkr3ksfL6CgQMTCFBEREQSZ2ZPuXtNNmXVs5ZnN1yb4qyGm1pN1ACGs5BxDdO58fpUzJGJiIhIMVKylmez7mjkzIabM5YZ1zCdWb/ZHFNEIiIiUsyUrOXZmnXd6c/yjGX6sYI163rEFJGIiIgUMyVreVbdO8Vy+mcss4J+VPfeEFNEIiIiUsyUrOXZmLEl3Fo+PmOZGeUTGHNKaUwRiYiISDFTspZnk87rzi/LJ7KAYa2+voBhzCifwDlTusccmYiIiBQjJWt5NnAgzLyngpG95nFx+VRqGUADZdQygIvLpzKy1zxm3qNpO0RERCQ7StYiMGIELHy+gtTZkzmkcjE9LcUhlYtJnT2Zhc9XMGJE0hGKiIhIsdCkuBFbtQp22QVuvhm++c2koxEREZFCoElxC8hOO4X7NWuSjUNERESKk5K1iPXoARUV8M47SUciIiIixSjSZM3MjjazJWa21MwuauX18Wa22MyeNbPHzWyfFq/3M7N1ZnZ+lHFGrapKyZqIiIi0T2TJmpmVAjcCI4B9gJNbJmPALHf/L3ffD7gGuK7F69cDc6OKMS5K1kRERKS9yiJs+0BgqbsvAzCzO4FjgZeaCrj7h83KVwD/N9rBzI4DlgF1EcYYi3POgZ49k45CREREilGUydouwGvNHq8EDmpZyMzOAb4DdAMOSz9XAVwIHAEU9SlQgDPPTDoCERERKVZRXrNmrTy31Twh7n6juw8kJGeXpZ/+H+B6d1+XcQNmZ5vZIjNbtHr16g4HHJV162DZsqSjEBERkWIUZbK2Etit2eNdgVUZyt8JHJf++SDgGjN7Ffg2cImZTWpZwd1vcfcad6/p06dPfqKOwNSp8MlPwubNSUciIiIixSbK06BPAoPMbA/gdeAkYEzzAmY2yN3/k374JeA/AO7+uWZlfgisc/cbIow1UlVV4A7vvQfV1UlHIyIiIsUksmTN3Tele8MeAkqB29z9RTO7HFjk7rOBSWZ2ONAAvAecFlU8SaqqCvfvvKNkTURERHITZc8a7j4HmNPiue83+/ncLNr4Yf4ji1fzZE1EREQkF1rBIAZK1kRERKS9lKzFYOBAmDYNBg9OOhIREREpNpGeBpVgp51g8uSkoxAREZFipJ61mLz0ErzyStJRiIiISLFRshaTww6Dq65KOgoREREpNkrWYlJdrQEGIiIikjslazGpqoI1a5KOQkRERIqNkrWYVFWpZ01ERERyp2QtJjoNKiIiIu2hqTtiMm4cfOlLSUchIiIixUbJWkwOPDDpCERERKQY6TRoTNasgUcegbVrk45EREREiomStZgsWABHHglLliQdiYiIiBQTJWsx0WLuIiIi0h5K1mKiZE1ERETaQ8laTJqSNU2MKyIiIrlQshaTHXcEM/WsiYiISG40dUdMSkthzhzYc8+kIxEREZFiomQtRkcfnXQEIiIiUmx0GjRGTzwBDz2UdBQiIiJSTNSzFqOrroJVq+Coo5KORERERIqFetZiVFWlAQYiIiKSGyVrMVKyJiIiIrlSshajqiqoq4MNG5KORERERIqFkrUYVVeHe/WuiYiISLaUrMXouOPg6aehT5+kIxEREZFiodGgMerbN9xEREREsqWetRh9+CH84hfw8stJRyIiIiLFQslajNauhfHj4bHHko5EREREioWStRhVVYV7DTAQERGRbClZi1GPHlBRAWvWJB2JiIiIFAslazHTxLgiIiKSi0iTNTM72syWmNlSM7uoldfHm9liM3vWzB43s33Szx9hZk+lX3vKzA6LMs44KVkTERGRXEQ2dYeZlQI3AkcAK4EnzWy2u7/UrNgsd785XX4kcB1wNLAG+Iq7rzKzIcBDwC5RxRqn++8Pp0JFREREshHlPGsHAkvdfRmAmd0JHAv8X7Lm7h82K18BePr5Z5o9/yLQw8y6u3sqwnhj0a9f0hGIiIhIMYnyNOguwGvNHq+kld4xMzvHzGqBa4BvtdLO8cAznSFRA/j73+HKK5OOQkRERIpFlMmatfKcb/WE+43uPhC4ELjsIw2YDQZ+Anyz1Q2YnW1mi8xs0erVq/MQcvT+9je47DLYvDnpSERERKQYRJmsrQR2a/Z4V2BVhvJ3Asc1PTCzXYE/AKe6e21rFdz9FnevcfeaPkWy4GZVFbjDe+8lHYmIiIgUgyiTtSeBQWa2h5l1A04CZjcvYGaDmj38EvCf9PM7AA8CF7v7ExHGGLvq6nCvudZEREQkG5Ela+6+CZhEGMn5MnC3u79oZpenR34CTDKzF83sWeA7wGlNzwOfBL6XntbjWTPbOapY46RVDERERCQXUY4Gxd3nAHNaPPf9Zj+f20a9HwM/jjK2pChZExERkVxEmqzJ1vbdN1yvVlmZdCQiIiJSDJSsxay8HHbYIekoREREpFhobdAEfP/7cN99SUchIiIixUDJWgJ++UuYOzfpKERERKQYKFlLgBZzFxERkWwpWUuAkjURERHJlpK1BFRXa1JcERERyY6StQRUVcH69UlHISIiIsVAU3ckYPp0KC1NOgoREREpBupZS4ASNREREcmWkrUE/POfcMop8MYbSUciIiIihU7JWgLefBPuuANWrkw6EhERESl0StYSUF0d7jV9h4iIiGyLkrUEVFWFeyVrIiIisi1K1hKgZE1ERESypWQtATvuCDvtBI2NSUciIiIihU7zrCWgtFS9aiIiIpId9ayJiIiIFDAlawn53vfgu99NOgoREREpdDoNmpBnnoFVq5KOQkRERAqdetYSUl2t69ZERERk25SsJaSqSsmaiIiIbJuStYRUVUFdHaRSSUciIiIihUzJWkL694d99oF165KORERERAqZkrWEfP3r8OKLW1YzEBEREWmNkjURERGRApZVsmZmPc1sr6iD6UqWL4fPfhYefjjpSERERKSQbTNZM7OvAM8Cf04/3s/MZkcdWGdXXg5PPAGvvJJ0JCIiIlLIsulZ+yFwIPA+gLs/C+weXUhdQ9O1amvWJBuHiIiIFLZskrVN7v5B5JF0Md27Q0WF5loTERGRzLJZbuoFMxsDlJrZIOBbwPxow+oatIqBiIiIbEs2PWuTgcFACpgFfACcG2VQXcVnPwu77ZZ0FCIiIlLIsknWvuTul7r7Z9K3y4CR2TRuZkeb2RIzW2pmF7Xy+ngzW2xmz5rZ42a2T7PXLk7XW2JmR2X/lorHHXfAj3+cdBQiIiJSyLJJ1i7O8rmPMLNS4EZgBLAPcHLzZCxtlrv/l7vvB1wDXJeuuw9wEqFH72jgpnR7IiIiIl1Km9esmdkI4BhgFzOb1uylSmBTFm0fCCx192Xp9u4EjgVeairg7h82K18BePrnY4E73T0FvGJmS9PtLchiu0Xjmmtg1ix49tmkIxEREZFClWmAwSpgEeGU51PNnl8LTMmi7V2A15o9Xgkc1LKQmZ0DfAfoBhzWrO7CFnV3yWKbRWX9enjuOdi0CcqyGeohIiIiXU6bKYK7Pwc8Z2az3L2hHW1ba822sp0bgRvTI04vA07Ltq6ZnQ2cDdCvX792hJisprnW3nsP+vRJNhYREREpTNlcs7a7md1jZi+Z2bKmWxb1VgLNxzruSuita8udwHG51HX3W9y9xt1r+hRhttOUrGn6DhEREWlLNsnar4DphOvUvgjMBH6TRb0ngUFmtoeZdSMMGPjIMlXpeduafAn4T/rn2cBJZtbdzPYABgH/zGKbRUXJmoiIiGxLNldK9XT3v5iZufty4Idm9r/ADzJVcvdNZjYJeAgoBW5z9xfN7HJgkbvPBiaZ2eFAA/Ae4RQo6XJ3EwYjbALOcffN7X2Thap/f/jyl6FXr6QjERERkUJl7ltdCvbRAmZPAJ8D7gH+CrwOXO3ue0UfXvZqamp80aJFSYchIiIisk1m9pS712RTNpvToN8GehGWmToAOIV0D5iIiIiIRGubyZq7P+nu69x9pbt/w92/BrwRQ2xdwh57wPe/n3QUIiIiUqgyJmtmNtzMTjCzndOPh5rZLODxWKLrAjZsgDeU+oqIiEgb2kzWzGwqcBtwPPCgmf0AeAT4B2F0puRBVZVGg4qIiEjbMo0G/RKwv7tvMLMdCfOcDXX3/2SoIzlSsiYiIiKZZDoNWu/uGwDc/T1giRK1/KuqgjVrko5CREREClWmnrWBZtZ8Etvdmz9295HRhdV1HHkkDNJJZREREWlDpmTt2BaPr40ykK5q/PikIxAREZFClmkh90fjDKQr27wZSkrAWlu+XkRERLq0bCbFlQj99rdQXg6vvpp0JCIiIlKIlKwlrLIS3DXIQERERFq3rUlxS9PzrUlEqqrCvabvEBERkdZkTNbcfTNwgJmupoqKkjURERHJJNNo0CbPAH80s98DdU1Puvt9kUXVhVRXh3udBhUREZHWZHPN2k7AO8BhwFfSty9HGVRXssMOMHkyDB2adCRdV20tTJmYom9lPaUljfStrGfKxBS1tUlHJiIiAubuSceQFzU1Nb5o0aKkw5AiM3cunHpCHWc13MSZDTfTn+Uspz+3lo/nl+UTmXlPBSNGJB2liIh0Nmb2lLvXZFN2mz1rZrarmf3BzN42s7fM7F4z27XjYUqTVArefz/pKLqe2tqQqM1efzhXNnyXgSyjjM0MZBlXNnyX2esP59QT6tTDJiIiicrmNOivgNnAJ4BdgAfSz0mefPGLcMIJSUfR9dxwbYqzGm5iOAtbfX04CxnXMJ0br0/FHJmIiMgW2SRrfdz9V+6+KX37NdAn4ri6lKoqjQZNwqw7Gjmz4eaMZcY1TGfWbzbHFJGIiMjWsknW1pjZ2PSca6VmNpYw4EDyRMlaMtas605/lmcs048VrFnXI6aIREREtpZNsnYGMAp4E3gDOCH9nOSJkrVkVPdOsZz+GcusoB/VvTfEFJGIiMjWtrmCAXC8u4909z7uvrO7H+fumbsjJCdVVbB+PWxQThCrMWNLmFE+PmOZGeUTGHNKaUwRiYiIbC2bFQyOjSmWLuuww+Dqq6GxMelIupZJ53XnJp/IAoa1+voChjGjfALnTOkec2QiIiJbZLOCwRNmdgNwFx9dweDpyKLqYoYNCzeJ16uvwoebKji6bB4TbTrjGqbTjxWsoB8zyicwo3wCM++pYODApCMVEZGuLJtk7eD0/eXNnnPCigaSBw0NsHJlWHpqu+2SjqbrWLYsrBwxa1YFt06fzCG/mcjqD3uwfY8NnH5mKQundFeiJiIiicu4goGZlQAnuPvd8YXUPsW8gsHixSFpuPtuOPHEpKPpWjZtgrJs/mURERHJo7ytYODujcCkvEQlbaqqCvcaERqPRx6Be+8NP7eWqG3aBJs1tZqIiBSIbKbueMTMzjez3cxsp6Zb5JF1IUrW4vPee3D66fCDH4TTzy397W9QUQFPPhl7aCIiIq3K5gRQ05xq5zR7zoEB+Q+na+reHXr3VrIWh299C956C2bPhvLyrV/v1w82boQXX9SgDxERKQzbTNbcfY84AunqqqpgzZqko+jc7r8f7rgDvv99OOCA1svssQf07BmSNRERkULQ5mlQM/tus59PbPHalVEG1RVdcUU4PSfR+OAD+OY3Yb/94NJL2y5XUgL77AMvvBBfbCIiIplkumbtpGY/X9zitaMjiKVL+/rXw+S4Eo3KSvjpT+H226Fbt8xlBw9Wz5qIiBSOTKdBrY2fW3ssHfT66/DGG1CT1SBeyUUqFa4LPPnk7MqfeCLstVdYUaIkmyE4IiIiEcr0p8jb+Lm1x60ys6PNbImZLTWzi1p5/Ttm9pKZPW9mfzGz/s1eu8bMXjSzl81smpl16gTx6qvhyCOTjqL41dbClIkp+lbWU1rSyM7b1fOJqhTTpmXfxpe/DJdcokRNREQKQ6Y/R/ua2YdmthYYmv656fF/bavh9CLwNwIjgH2Ak81snxbFngFq3H0ocA9wTbruwcAhwFBgCPAZ4NDc3lpxqaoK00ps2pR0JMVr7lwYNrSOnjOmMX/tEFLejQXrhvCNumlcfmEdc+dm1447vPkmrFoVbbwiIiLZaDNZc/dSd6909+3cvSz9c9PjViY92MqBwFJ3X+buG4E7abEovLv/zd3Xpx8uBHZtegnoAXQDugPlwFu5vbXi0jTX2nvvJRtHsaqthVNPqGP2+sO5suG7DGQZZWxmIMv4f3yXBzYczqkn1FFbm117gwaF3k4REZGkRXmiZxfgtWaPV6afa8uZwFwAd18A/A14I317yN1fblnBzM42s0Vmtmj16tV5CzwJ1dXhXnOttc8N16Y4q+EmhrOw1deHs5BxDdO58frUNtsyCyNCNchAREQKQZTJWmvXmLV6rZuZjQVqgKnpx58E9ib0tO0CHGZmn9+qMfdb3L3G3Wv69OmTt8CToFUMOmbWHY2c2XBzxjLjGqYz6zfZrSM1eLCm7xARkcIQZbK2Etit2eNdga2uAjKzw4FLgZHu3tTt8VVgobuvc/d1hB63Tj2f/H77wV13wZ57Jh1JcVqzrjv9WZ6xTD9WsGZdj6zaGzIE3n5bExWLiEjyokzWngQGmdkeZtaNMG/b7OYFzGx/4BeERO3tZi+tAA41szIzKycMLtjqNGhnsvPOMGoUFHkHYWKqe6dYTv+MZVbQj+reG7Jqb/DgcK9ToSIikrTIkjV33wRMAh4iJFp3u/uLZna5mY1MF5sK9AZ+b2bPmllTMncPUAssBp4DnnP3B6KKtRA0NsJf/gJLliQdSXEaM7aEW8vHZywzo3wCY04pzaq9z3wmTKD7qU/lIzoREZH2M/espkwreDU1Nb5o0aKkw2g39zBx63e+o1GI7VFbG6btmL3+8FYHGSxgGCN7zWPh8xUMHJhAgCIiIs2Y2VPuntVU+Jr2s0CYhUEGGmBvQmuiAAAgAElEQVTQPgMHwsx7KhjZax4X2FRqGUADZdQygIvLpzKy1zxm3pNbovavf8Gf/xxdzCIiItlQslZAqqt1QXtHjBgBC5+vYGblZPYvW0zPkhSHVC4mdfZkFj5fwYgRubV33XUwdmzo9RQREUmKkrUCop61jtt9d3i/vjsTvtOLTZtLePODXlx3Q/d2nfocPDjsj7ff3nZZERGRqChZKyBK1jquvh5Gj4ZDDul4WxoRKiIihaAs6QBki+9/H1LbnmBfMujdG2bOzE9bzZO1ww7LT5siIiK5UrJWQPbdN+kIit/GjVBeHgZsdNTHPgY77qieNRERSZZOgxaQZcvgN78Jp/KkfX74wzBQY3N2q0plZAYPPww/+lHH2xIREWkvJWsF5LHH4NRT4Y03ko6keC1bFnrDSrOb+3abamq0qoSIiCRLyVoB0WLuHVdbCwMG5K+9V16Bq67SlCoiIpIcJWsFpClZU2LQfsuWkdcVCl59FS65BJ55Jn9tioiI5ELJWgGprg736llrn/ffh3ffzW/P2pAh4f6FF/LXpoiISC6UrBWQYjgNWlsLUyam6FtZT2lJI30r65kyMUVtbdKRQWNj6AU79ND8tdmnT7hpRGjxKuRjVkQkG0rWCsiOO8I//xmWOCpEc+eGxdJ7zpjG/LVDSHk35q8dQs8Z0xg2tI65c5ONb6ed4Ior4MAD89vu4MFK1opVoR+zIiLZMO8kCx/W1NT4okWLkg6j06qtDX/0Zq8/nOEs3Or1BQxjZK95LHw+t8XS8+ntt6FbN9hhh/y2O2kS3HVXaD8f87dJPIrhmBWRrsvMnnL3mmzKqmetwNxzD/zxj0lHsbUbrk1xVsNNrf7RAxjOQsY1TOfG65NbguGyy2DQoPy3e+WVYToVJWrFpRiOWRGRbKhnrcAcfDD06gXz5iUdyUf1raxn/tohDGRZm2VqGcAhlYt584NeMUa2xeGHw7p1sLD1v83SxRTDMSsiXZd61opYoS7mvmZdd/qzPGOZfqxgzboeMUW0tdra/E7b0WTTpnAq9O6789+2RKcYjlkRkWwoWSswhZqsVfdOsZz+GcusoB/VvTfEFNFHNTTAihX5nbajSVkZ3Hsvuhi9yBT6MSsiki0lawWmUJO1MWNLuLV8fMYyM8onMOaUPK3zlKPly8PUHVFdKK4RocWn0I9ZEZFsKVkrMFVVsH49bCiwf/YnndedX5ZPZAHDWn19AcOYUT6Bc6Z0jzmyYMcdYfp0+Nznomm/KVlrbIymfcm/Sed155aywj1mRUSypWStwIwfDytXhikoCsnAgTDzngqOKp3H+UyllgE0UEYtA7i4fCoje81j5j3JTYFQVRU+u6i2P2RISKKXZ74ESgrIwIFw5HEVHM48Lir76DF7PlM5wuZx0681bYeIFD4lawVmp51gl12gpAD3zLBhsJ4KHt9vMjXdF9OdFIdULiZ19mQWPl/BiBHJxfbii/DSS9G1P2QI7LZbmGtNisdvfwuz51Ww8ZuTOaRyMT1LwjH75gmTqbcKHnww6QhFRLZNU3cUmFWrYMYMOOkk2HPPpKP5qA8/hF/9KkyR8cILMG4cPP10NHOb5eprX4MlS3RdmQTL0rN1ZBpw8uMfh8EjF16oOfREJH65TN2hZK3ALF4MQ4eGaSJOPDHpaNq2eXPo/SuUP3L77gv9+sEDDyQdiSRt8+Zw7eJrr4XpXArtkgIREdA8a0WtujrcF9qI0HfegdtvD71rAKWlhZOouUc3x1pzl18eevCksF1/PSxYAFddlV2idv/9MHly9HGJiLSXkrUCU1UV7tesSTaOlu67D04/fcvpJQgTxV5/fWIh/Z/Vq6GuLpo51pr74IMw19rmzdFuR9rvpZfCsmPHHQdf/3p2dV54AW64ISz1JiJSiJSsFZhu3aB378LrWbvrrnBt2r77bnluwQJ4+OHkYmpSWxvuo+5ZGzw4TKnyyivRbkfaZ9Om8A9F795w883Z9/xedBHU1MCECRpAIiKFSclaAaquLqxk7e234W9/g1GjPvoHcODALYlSkvbeGx58EA46KNrtDB4c7jWIoTA1NISka/p06Ns3+3plZVtO8Y8fH06ri4gUEiVrBejZZ8Ooy0Jx331hMtjRoz/6/IAB8OqryZ8W3GEHOOaYLdf7RWWffcK9krXC1LMn3HRT+wbm7LMP/OhH8Ic/wN//nvfQREQ6RMlaAdp++3ABf6F4+mn41KfCXGPNDRwYejNWrkwmribz5sXzB3a77cK1UH36RL8tyd7GjWGqm44OBj/vvDCa+AtfyEtYIiJ5U5Z0ALK1u++G558P80AVgltuCRfXt7wGaM89wynIDz5IJq4m//M/YRqRRx+Nflt/+EP025DcXHFFuKby5JPDadD2Ki2FL385/LxyZZiculBGPItI16aetQJTWwvX/CjFdVfUU1rSSN/KeqZMTCV2bVjT9Tvbb7/1a4ceGkbfDR0ab0wt1dZGPxK0ucbG9l/XVFsLUyam6FsZ7f5t73baUy+uOq3Vq+5dz08uT3HccXDssZnrZuv558NgmhH/Hf1+gviOCREpXpEma2Z2tJktMbOlZnZRK69/x8xeMrPnzewvZta/2Wv9zOxhM3s5XWb3KGMtBHPnwrChdfz3S9NYzBBS3o35a4fQc8Y0hg2tY+7c+GM65hi45JL4t5ut9evhjTeiHwna5IEHwunQJUtyr9u0f3vOmMb8tdHt3/Zupz314qrTVr1/1A1hMtN4/OH8fX6vvQZlG+sY8rdo9xPEd0yISJFz90huQClQCwwAugHPAfu0KPNFoFf65wnAXc1e+ztwRPrn3k3l2rodcMABXsyWLnWv7rXO5zPMPXTcfOQ2n2Fe3WudL10aX0wrV4bNX35522VOPdV90qT4YmrphRdCjL/9bTzbW7QobO+ee3KrF9f+be922lMvrjrF8PkV+rZEpPAAizzLnCrKnrUDgaXuvszdNwJ3Ah85UeHuf3P39emHC4FdAcxsH6DM3R9Jl1vXrFyndMO1Kc5quInhLGz19eEsZFzDdG68PhVbTE2ThI4a1XaZt94K860lpWmS3rh61vbeO1zHlOuI0Lj2b3u30556cdXpSL1cxfl7WIi/8yJSoLLN6nK9AScAM5o9PgW4IUP5G4DL0j8fB/wJuA94BpgKlLZS52xgEbCoX79+ec1447bzdut9KQNa/Q+76baUAd63si62mIYPd99338xlJkxw33HHeOJpzfr17s89F+7jMmCA+6hRudWJa/9mu53tyrZs54QT3HuVZFevoqTO99rL/ayzst9W73SdvfbKfjs7dA/xrV+fW724Pr98/B4W4u+8iMSHAulZa20cVauXZZvZWKCGkJRBGKX6OeB84DOEU6mnb9WY+y3uXuPuNX2KfD6FNeu605/lGcv0YwVr1vWIJZ4VK0KPWaZeNQg9Wu+9F25J6NkzDHDo2TO+bQ4ZknvPWlz7N9vt1G3asp0994T6xuzq1Tf2YL/9woCObLe1Pl1nv/2y387ajSG+kpLc6sX1+eXj97DQfudFpHBFmaytBHZr9nhXYFXLQmZ2OHApMNLdU83qPuPhFOom4H7g0xHGmrjq3imW0z9jmRX0o7r3hljiKS8Py/C0nAi3pabTj83XDI3TbbeFSXvjdNJJYZqIXMS1f7PdTp/KLdu54gros1329e68MxwbuWzrzjvhzjuz3071diG+7t1zrBfT55eP38NC+50XkcIVZbL2JDDIzPYws27AScDs5gXMbH/gF4RE7e0WdXc0s6bussOAlyKMNXFjxpZwa/n4jGVmlE9gzCnxzJb78Y/DVVdt+1qwvfeGI4+MJaRWTZ0Ks2bFu82TT4ZLL82tzpixJcwoi37/tvc4ak+9uOp0pF6u4vw9LLTfeREpYNmeL23PDTgG+DdhVOil6ecuJyRnAPOAt4Bn07fZzeoeATwPLAZ+DXTLtC2NBs2f1193nzvXfePG6LfVEZs3u3fv7n7BBfFv+5133Fevzr78f/7jvl1pPKMZq3pqNGhHPr84R4Pu1KMwfudFJH7kcM1apMlanLdiT9bc3efMCX8oLiqf6ksZ4Bsp86UM8IvKp3p1r3U+Z048cfzkJ+HIWLYs+zqNjdHF05bXXgtx3nxzvNvdsMG9pMT9e9/Lvs4dd4RYty/fev+eX5Lf/fv5z7v3Yp1/tyy346g9x19cdTpSL1dtbeeC0qnek3V+xhn52Y67+/nnu1fYOr+w5bbyfEyISOFRslbEli51n3LOBu9bWefGZt++W51POWdDrP9dH3CA+4EHZl9+1Cj3o46KLp62/P3v4Qh+5JH4tz1okPvXvpZd2ddfd99hB/eDD3ZfsmTL/i0t2ex9K/O7f+++O3wmU6a0bzvNj79s68VVpyP1ctXadr49cYMfcUTozV2yJH/bevnlj26rd2md990x3t95EYlfLsmahfLFr6amxhd1dCXnAnPwwWG9wv/93/i2uXRpWGrn//2/sLB1Nk45JcT46quRhraVmTPhtNPiX24K4KtfhZdfhn/9a9tla2th3LiwxuqgQa2XefppmDwZ7r0XPvax9sf1xz/CL34Bs2dDmVb+zbu334bf/AbOPbdjn++CBWHEdWsDeG68ESZNghdegMGD278NESlsZvaUu2e1orHWBi1gX/4y9O8fLmCJy+9/H+5PPDH7OgMGhCV6Nm6MJqa2nHoqvPsu7L57vNuF8Ed06VJIZTFf6cCB8Le/tZ2oQZh65KmnYPz4ju3vY4+FOXOUqEVl553DPzFlZe0/3tevD/9kXHgh1Ndv/fro0fDYY2HwjogIKFkraJdcAnfcEWbMj8vf/w7Dh0O/ftnXGTgwLG6+PPOUUZHYcccwF1fchgyBzZszrxG6YgWccQasWbPt9vbeO0yh8cc/hn2eqzvvhGuvDTFJ9BYuDMf9c8/lXveSS+A//4Ff/ar1+QGrq+Fzn0vmuBaRwqSvgyIQZ4/V3LnhVFwumk5B1tbmP55Mvvc9uP32eLfZ5LOfDac1P/7x1l93hzPPhLvvhg8/zK7Nb38bDjkknA59/fXsY3n9dZgwIf755rqyQYOgoSH0kOXy+/noo/Czn4XTnF/8YtvlVq2Cb30r98mXRaRzUrJW4L7wBRg7Nr7tlZS0nYC0Za+9QmIS9yISN92U3Lqku+4KZ53V9nu++WaYNy9c+5ft9XSlpaG3ZeNGuP767Oq4hzhSKfj1r0MbEr2qqpCsP/cc/PjH2dXZsAG+8Q345Cfh6qszly0rC9eu/e53HY9VRIqfkrUC9/GPw/z58WzrqKPCH4hc9ekDM2bAAQfkP6a2vP9+uF4trgXcW/Pvf7c++GPZMrjgAjjiCPjmN3Nrc9Cg0Oa2/pg3ue220Bt69dWZr4mT/Bs5Mlw3eeWVkM3Yph49wn66/XaoqMhcdued4bDD4K674r1mVUQKk5K1AnfwweE012uvRbudl16Chx9u/x+GxsZ41wdtWt4q7lGgzV16abgmraXzzgs9XLfe2r7rDQ84IPSsrF6d+XTohx/C+eeH3tdJk3LfjnTcz34WRu/eeWfmcg0N4X7UqPA7nY1Ro8Iglmef7ViMIlL8lKwVuKYv9qh71+6+OyQWxx/fvvonngif/3x+Y8qkKVlLsmdt8OBwnV7LEX3Tp4fr/nbbrfV62di0KQz0OPXUkAi3prISHngg9K7pYvRk7LADPPlkWPasLR98EI6VmTNza/trXwtJ/113dSxGESl++oovcEOHhhFjUSZr7iFZO/TQ3K9Xa9KvX0ig4jpl88EH4VRSkj1rgweH99s019qaNWE05sc+Bocf3rG2y8rgu9+Fv/41JH8trV4d7j/7Wdhjj45tSzrm4x8P/+gsXQrPPLP161OmhKR+r71ya7eqKvwTpGlYRETJWoErL4fLLw/Xk+VTbS1MmZiib2U9ZaWNvPJyPWWNqXaP6BwwIMwf9dZb+Y2zLWeeCWvXht6lJNTWwp/uS9GDemo+3Ujfyno+PSSV1/101llhv59/Ppzx9bCvSksa6bNdPXvskuLKK/O3LemYxsYwx91XvwqTzt6yr6p61fPbX6U4+2w46KDc2/3d77IfwBCV5t8VpSXhWJ8ycdvfFe2p195tSfzi2r86/tKyXeqg0G+dZbmpODStfXhx+TW+lAHeQGlYY7HsmnavR/jgg2GZoyeeyH+8hWbL2pEf/fzO4xrfoVt+13OcOTOs83k+W2+rqqfWjiwkV17p3pN1fkHJR/fV+bT/98o9rLv7xhv5jTVbbX1XXFye+T21p157tyXxi2v/dvbjD60N2rls3uz+wgthjcmOWro0HJDzGRZ2f4vbfIZ5da91Oa9L+PLLoYmZMzseYzZGjXKfPj2ebTUX1eeX9LakY6LcV+PGuffrF5K2OLX3PbWnno714hHX/u0Kx5+StU5m9eqwp66+uuNtfXvCBr+4/JpWD8im20XlU33KORtyanfDBvcf/tD92Wc7HuO2bNzoXlrqfuml0W+rpag+v6S3JR0T5b66/fbQxIIFEQSeQTbv6cKyqT7+jA2+enX4nqqrC/Uu2la98o/W++Y3NvhFZTrWi0Gux8WaNdn/fkwcl9sxcWHZVJ945pZj4t13s6/XFN8HHyT3XatkrRMaNMh95MiOt7Pzdut9KQMyHpRLGeB9K+s6vrGILF0aQr3ttvi3Hefn1xn2VVcR5b56/333bt3cp0yJIPAMsn1PPan7v6euvrp99XqgY71Y5Lp/d9gh+zrbleV+TPQu3XJMDB2afb2m+I47Lrnv2lySNQvli19NTY0vymZmyiJ1+ulhge633urYWqGlJY2kvBtltL2IZANl9CxJsWlzbuNP3n0X3ngjjJKM0sMPhwvvH3003ulCINrPL8ltScdEva+OPRaeeiqsNxvXNC1ZvydL8dNpIajhw+HAz+Re79zJjaTQsV4Mcj0uuneH8d+M8JiwFJsaQ53f/Q7Gjsm+3k+nlTBgAHzly8l815rZU+5ek01ZHfVF4uCDw3QNHR2VUt07xXL6Zyyzgn5U996Qc9sXXNDxKSuykeQca1F+fkluSzom6n01alSYIDnO5dWyfk/bbWDSpDAx8wEHtK9e9XY61otFrvv3rLMiPia223JMnHxybvUmTYJjjimO71ola0Vi+PBw39H51saMLeHW8vEZy8won8CYU3JfZHLgQHjzzTCFR5TKy8P8c+2dE64jovz8ktyWdEzU+2rkSPj972H//dtVvV3a+57aU0/HevEYM7aEGWXR718dfy1ke7600G+d/Zq1TZvcH3ggXEDZEVGOevnd70Izixd3LMZCptGg0prOuK+WLnWv6tm5R+NJ7pYudd+uVKNB8wENMJBMmuaTubB8qi9lgG+kLMyzVj61Q/PJ/POf4Yj64x/zG2+h2TLPWn4/v6S3JR0T9b56/333K65wf/LJ/MSbjTlz3Hfqkft3RXs+i7bqXFimY73Q3H9/OC7ysX/zXSfubXWEkrVOatky96uuCkONO2rpUvcp52zwvpV1Xlqy2ftW1vmUczZ06D+Hd94JR9R113U8vrY0NroPHuz+859Ht41sRPH5FcK2pGOi3Fdr17r37Ok+cWLH28rGokXhvr3vqT31dKwXtmXL3D/8MPwc1/7tzMdfLsmaRoMWkaZRkI880rEL+Rsb4StfgTPOaP/C7a1xh9tvD4Mh9twzf+02t3o17Lwz/PSncO650WxDpFCdeCI89hisWhUWeY/KvffCCSfA/feHkahJ2rgxjIAvL082jq5u40Y48MAta1V3ZFYCCTQatJM66KDwC9LRQQYLF4ZpQOrr8xNXE7MwxUhUiRpsGQ2bxEhQkaSNHg1vvx2mrYnK22/D+PFhZOcxx0S3nWwsWQJ9+8If/5hsHBLWqH3uObjoIiVqSVCyVkS23z7MYdbRZO2uu6B79zDCLN9qa0MiGBUla9KVHXMMVFTA3XdH0747TJwIH34YesmT7s365CehW7fwnSXJeeopuPJKOOWU5Htauyola0Xm4INDz1hjY/vqNzaGKQBGjIDKyvzGBvDLX8Jxx8HmtucW7JCmOdZ23z2a9kUKWa9e8LWvRTc9zp13hlOgl18e/eTW2SgtDadjH3wQ1q1LOpquKZWC004LPZw/+1nS0XRdStaKzPDhsGEDvPpq++o//nhYZWD06LyG9X8GDICGBli5Mpr2+/ULsffsGU37IoXu9tth5szo2j/qKDj//Ojaz9Xo0eGSjQcfTDqSrun996G6Gm69FXbcMeloui4NMCgy9fVhuZnu3dtXf8ECuOoqmDULevfOb2wAf/lLGPzw17/CF7+Y//ZFJFi3LprfYffCuiZp82bYbTcYNgzuuy/paLqmQjsmOgsNMOjEevZsf6IGoWdu9uxovuQh9KxBx5fFaktUp1dFismPfgR77BF6sfNh1iz4xS8K849yaSn8/OdhOTuJz/r1cM454UxMoR0TXZGStSL061/DmDG511uxIgz5j9Juu0FZ2ZZry/Kpvj5cs/Pzn+e/bZFist9+sGZN6MnuqNdegwkTQsJWqCdajj9+y5J7Eo9LL4WbboKXX046EgEla0XpjTfgd7+Dd97Jrd5VV8Fee4ULRqNSVhb+gEyenP+2X3klzPVTVZX/tkWKycCB0LtbitEj6yktaaRvZT1TJqa22aNdWwtTJqboW7ml3hcPTtHQAL/6VbjEolD985+h9y9fWvsstvUZxlUn6fhGH5fipz8NPWuHHZY5TolHAf9qSlsOPjjcL1yYfZ1Nm8Iory99qWOnUbPx+c9Hs8h60xdO06lWka5o7lw49DN1TGiYxtMNQ0h5N+avHULPGdMYNrSOuXPbrjdsaB09Z0xj/tot9Y5bOY3um+tYsiTe95GrO++Eb30rXPDeUW19Fpk+w7jqFEJ8/f44jQqr47//O7fPVSKU7VIHhX7rCstNNVm3zr201P3SS7Ov88gjYSmoe++NLq4mzz7rPm1a/tv96U/De3j77fy3LVIM4lzcutAsWBDCvf32jrUT10Lfhb4QeWc4JoodOSw3FWnPmpkdbWZLzGypmV3UyuvfMbOXzOx5M/uLmfVv8Xqlmb1uZjdEGWexqagI16zkMjnuXXeFQQUjRkQXV5NHHgn/Ab/3Xn7bra0N76G6Or/tihSLG65NcVbDTQyn9W714SxkXMN0brw+lZd6heSgg8LUPR2dILc9n0VcdYohPklItlldrjegFKgFBgDdgOeAfVqU+SLQK/3zBOCuFq//DJgF3LCt7XWlnjV39+99z/3kk7Mru2mTe3W1+5gx0cbU5L77wj9nTQtB58tdd4X3LdJV7bzdel/KgFZ7QppuSxngVb3q/J13Qp3ly9136pldvb6Vdcm+wW04/3z3sjL3d99tfxvZfoZ9etf5/fe7339/9p9fVa8tn19VRfZ1mrYzb15u+zjX+KortsTXWY6JYkYhLORuZsOBH7r7UenHF6eTw6vaKL9/Oik7JP34AOAC4M9AjbtPyrS9rjLPWnu9/nqYTDeOZZqeey70/N19d1h4WkTyo7SkkZR3o4y257BpoIzupJi/oIRhw8LAgTPPaGQj267XsyTFps2Feynzk0/C0UeHtUI/+9n2tZHtZ9iDFI3pk09Gdp9fD1Js9lCnxLKr050UTSe59t4blvyrsOIr9GOimBXKPGu7AK81e7wy/VxbzgTmAphZCXAtIVlrk5mdbWaLzGzR6tWrOxhucdq0Kbtyu+wS33qaUcy11tgYph2J6H8LkaJQ3TvFcvpnLLOCflRVbOC//is8HjkSqiqyq1fde0O+Qo1ETQ28+Wb7EzXI/jOs7r2Bp5+Gp5/O/vOrqtjy+VXlsK+atnPvvTns4/bE12z/dpZjoquIMllrbRq9Vv/UmtlYoAaYmn5qIjDH3V9rrfz/NeZ+i7vXuHtNnz59OhRsMfrCF+D00zOX2bgxzFH02GNxRBRstx3svHN+k7VVq0LCecst+WtTpNiMGVvCreXjM5aZUT6BU04vpaIiPK6qgrGnZldvzCml+Qo1EmZhcXn37P9RbSnbz/Drp5Wy//6w//7Zf35jT9/y+Y09Jft91bSdvffOPr6x7YnvtGbxdZJjosvI9nxprjdgOPBQs8cXAxe3Uu5w4GVg52bP/RZYAbwKrAE+BK7OtL2uds2au/vxx7vvsUfmMn/6U7j84E9/iiemJq+84l5fn7/2Hn00vI+HH85fmyLFpiuPBm3y1lvuAwe633JL++oX+mjLQo9P8occrlmLMlkrA5YBe7BlgMHgFmX2JwxCGJShndPRAINWXXtt2IOrVrVd5tRT3XfYwT2Vii+uKNx2W3iv+uKQrm7OnPBH9qLyqb6UAb6RMl/KAL+ofKpX91rnc+bkt16haWx0HzTI/fDD21d/xQr33r3dd+iW22fRns8vzn0VZ3ySHwWRrIU4OAb4dzohuzT93OXAyPTP84C3gGfTt9mttKFkrQ3z54c92NbcafX17pWV7t/4RrxxubsvXOj+7W/nL0m87LIwt9zGjflpT6SYLV3qPuWcDd63ss5LSzZ738o6n3LOhm3+M9PeeoXm0kvdS0pyn3OxsdH9iCPcKyrc//rX3D+L9nx+ce6rOOOTjsslWYtsNGjcuuJo0FQKKivDnGZTp279+uzZcOyxYZbqo4+ON7bbbw/X0/373zBoUMfbGzMmrNgQxZqjIlJcFi+GoUNh+nQYn/myq4+4+eawDmqu9USiUCijQSVi3bvD974Hhx7a+utmYRBCEkuG5HtE6CmnwA9+kJ+2RKS4DRkCn/pUmB4oW8uWwfnnwxFHwDe/GV1sIlEoSzoA6ZjLLmv7ta98JdyS0DRNSL56wuJYeUFEioMZ/OhHUJrDQMVFi8LqL7feGuqLFBP1rBU593Cq8a23Pvr8669DXV0yMUFYyL1Hj/z0rG3YAP/4B6xb1/G2RKRzOOEE+OpXsy8/alT453G33aKLSSQqStaK3Jtvwl57wWAW4sIAAA2cSURBVG9/+9Hnzz8fBg9ObhJZs3AqNB9zFb/8MgwbBn/+c8fbEpHOo7YWfv3rzGX+/W+4777wc9PccyLFRslakfv4x2H33WHBgi3PrV8PDzwQBhUk2d3/9NMwc2bH22nqnYtrBQYRKQ6zZsEZZ4QzCa3ZvDkMdBo3Dt57L9bQRPJKyVoncPDBMH/+ll60OXPCKdDRo5ONq3v3/LTTdN1b06AFEREIpzbdwzJNrbnuuvCP7A03wI47xhubSD4pWesEhg8PyzGtWBEe33UX9O0Ln/98snE98URYyL2jp0Jra8OSOdtvn5+4RKRz2Gsv2Hff8J3X0ksvhdHyX/0qnHxy/LGJ5JOStU7g4IPD/YIFoUftwQfDxbe5jJSKwvvvwz33wH/+07F2li3TKVARad2oUeHMwmvNVpLetAlOOy2sU3zzzRr9KcVPyVonUFEBI49K8a2z6qncrpGKknrWv5fK60Lq7dGR6Ttqa2HKxBR9K+v5618aqX2hnikTk39PIlJYDjoIeliK/faqp7Skkb6V9Zz/rRQnnwwzZsDOOycdoUjHKVkrcnPnwmc/Xcfgv05jwbohpLwbC+uG8LHfT2PY0Drmzk0utt13D//R5ppgzZ0Lw4bW0XPGNOavDe/pH+uH0HNG8u9JRArH3Llw0sg6vl02jX/Wh++K+WuH0GvGNK76Xh3duiUdoUh+aLmpIlZbG5Ka2esPZzgLt3p9AcMY2WseC5+vSOw04m67wWGHheWnslEM70lEkqfvCil2Wm6qi7jh2hRnNdzU6hcVwHAWMq5hOjden4o5si0+/WkoL8++fDG8JxFJnr4rpCtRz1oR61tZz/y1QxhI2xeF1TKAQyoX8+YHvWKMrP0643sSkfzTd4UUO/WsdRFr1nWnP8szlunHCtas6xFTRB3XGd+TiOSfviukK1GyVsSqe6dYTv+MZVbQj+reG2KKaGsLFoSlov71r+zKF8N7EpHk6btCuhIla0VszNgSbi0fn7HMjPIJjDkluQnXSkvDIuz//nd25YvhPYlI8vRdIV2JkrUiNum87vyyfCILGNbq6wsYxozyCZwzJU/rPrVDrnOtFcN7EpHk6btCuhIla0Vs4ECYeU8FI3vN4+LyqdQygAbKqGUAF5dPZWSvecy8J9lh6zvtFJaJynautYED4cTTKvhKj8J9TyKSvGL4/hPJFyVrRW7ECFj4fAWpsydzSOViepakOKRyMamzJ7Pw+QpGjEg2PrOwAHu2ydrjj4flYUacULjvSUQKQ6F//4nki6bukMhdcAFs2AA//3nmcnV1YVHmxkZ4/nno3Tue+EREROKWy9QdZVEHIzJ1anblLroo9MD9/e9K1ERERJroNKgUhP/9X7jhBjj3XDj00KSjERERKRxK1iRyixbBLruEHrO21NTAFVfAlVfGFpaIiEhRULImkauqglWr2h5ksHEj9OwJl1wCvbQqjIiIyEcoWZPI7bYblJW1nqz9+c+w997ZT5orIiLS1ShZk8iVlUH//ltPjPv++zBuHPTo8f/bu/9Yr+o6juPPl5cLAcbMIOdE8MfYkMywIcNRjcy5CzmwrR9qlDWMNHWU9gNMV7n5RzmTaWYp/sIiM0q8q9wStOkmmBgEmhlcE0SQCzMXiF2v8u6Pc9j3u8v3e5Pl+Z5z7+f12Ng95/M938t77733/b7v+XzOOTBuXDmxmZmZVZ2vBrWWaHSvtQUL4OWXYcWKrGEzMzOzg7lZs5Y45xx46aXafmcnLF0KV12VXVxgZmZmjXka1ArX1QWbnu5hyU2v03bYfo4a9ToLL+9h4kS4+uqyozMzM6s2N2tWqAcfhGmnvMbwJTfy+J6T6YmhPL7nZOZsvZFdW15j1aqyIzQzM6s2P27KCtPVlTVqnfvO5HTWHPT6aqYxe8RK1mzww5bNzCwth/K4KZ9Zs8L8+Poevtz7k4aNGsDprOHC3lu4+YaeFkdmZmY2cBTarEnqkPScpM2SFjZ4/XJJf5O0QdIqSePz8cmSVkt6Jn/ts0XGacVY9vP9zOv9ab/HXNh7C8vueatFEZmZmQ08hTVrktqAm4GZwCTgPEmT+hy2DpgSEacAy4Ef5uP7gC9ExPuBDmCxpCOKitWKsXvvMMazpd9jxrGV3Xt93w4zM7NmijyzNhXYHBHPR8QbwL3AnPoDIuKRiNiX764Bxubj/4iITfn2dqAbGFNgrFaA0Yf3sIXx/R6zlXGMPvw/LYrIzMxs4CmyWTsGeLFuf1s+1sw84MG+g5KmAkOBgx5WJGm+pLWS1u7atev/DNfeaefPPYzb2y/q95gl7Rdz/ufbWhSRmZnZwFNks6YGYw0vPZU0F5gCXNdn/GjgHuBLEbH/oF8WcWtETImIKWPG+MRb1Vx6xTBua/8qq5nW8PXVTGNJ+8Vc8vVhLY7MzMxs4CiyWdsGHFu3PxbY3vcgSWcC3wFmR0RP3fgo4PfAVRHR+HJCq7QTT4Sly0cye8RKFrVfRxcn0MsQujiBRe3XMXvESpYu9207zMzM+lNks/YkMEHS8ZKGAucCnfUHSDoV+BlZo9ZdNz4UuB9YGhG/LjBGK9jMmbBmw0h65l/G9FEbGX5YD9NHbaRn/mWs2TCSmTPLjtDMzKzaCr0prqRZwGKgDbgjIq6VdA2wNiI6Ja0EPgDsyN+yNSJm59OidwLP1P26L0bE+mb/l2+Ka2ZmZgPFodwU108wMDMzM2sxP8HAzMzMbJBws2ZmZmZWYW7WzMzMzCrMzZqZmZlZhblZMzMzM6swN2tmZmZmFeZmzczMzKzCBs191iTtArYcwltGA7sLCmegcS5qnIsa5yLjPNQ4FzXORY1zkTnUPIyPiLf1YPNB06wdKklr3+7N6AY756LGuahxLjLOQ41zUeNc1DgXmSLz4GlQMzMzswpzs2ZmZmZWYSk3a7eWHUCFOBc1zkWNc5FxHmqcixrnosa5yBSWh2TXrJmZmZkNBCmfWTMzMzOrvCSbNUkdkp6TtFnSwrLjKZOkFyRtlLRe0tqy42klSXdI6pb0dN3YkZIekrQp//meMmNshSZ5+J6kl/K6WC9pVpkxtoqkYyU9IulZSc9IWpCPp1gXzXKRVG1IepekP0v6a56H7+fjx0t6Iq+JX0kaWnasResnF3dJ+mddTUwuO9ZWkdQmaZ2k3+X7hdRFcs2apDbgZmAmMAk4T9KkcqMq3cciYnKCl17fBXT0GVsIrIqICcCqfH+wu4uD8wBwQ14XkyPiDy2OqSxvAldExEnANOCS/PMhxbpolgtIqzZ6gDMi4oPAZKBD0jTgB2R5mAD8C5hXYoyt0iwXAN+sq4n15YXYcguAZ+v2C6mL5Jo1YCqwOSKej4g3gHuBOSXHZCWIiEeBV/oMzwHuzrfvBs5paVAlaJKHJEXEjoj4S769h+xD+BjSrItmuUhKZPbmu+35vwDOAJbn46nURLNcJEnSWOATwJJ8XxRUFyk2a8cAL9btbyPBD6A6AfxR0lOS5pcdTAUcFRE7IPuyAt5XcjxlulTShnyadNBP+/Ul6TjgVOAJEq+LPrmAxGojn+paD3QDDwFdwKsR8WZ+SDLfI31zEREHauLavCZukDSsxBBbaTHwLWB/vv9eCqqLFJs1NRhL9i8DYHpEfIhsWvgSSR8tOyCrhFuAE8mmOnYA15cbTmtJOhz4DfC1iPh32fGUqUEukquNiHgrIiYDY8lmZ05qdFhroypH31xIOhlYBEwETgOOBL5dYogtIelsoDsinqofbnDoO1IXKTZr24Bj6/bHAttLiqV0EbE9/9kN3E/2QZSynZKOBsh/dpccTykiYmf+obwfuI2E6kJSO1lz8ouI+G0+nGRdNMpFyrUREa8CfyJbw3eEpCH5S8l9j9TloiOfMo+I6AHuJI2amA7MlvQC2XKqM8jOtBVSFyk2a08CE/IrNoYC5wKdJcdUCkkjJb37wDZwFvB0/+8a9DqBC/LtC4AHSoylNAcak9wnSaQu8jUntwPPRsSP6l5Kri6a5SK12pA0RtIR+fZw4Eyy9XuPAJ/KD0ulJhrl4u91f8iIbI3WoK4JgIhYFBFjI+I4sj7i4Yj4HAXVRZI3xc0vNV8MtAF3RMS1JYdUCkknkJ1NAxgCLEspF5J+CcwARgM7ge8CK4D7gHHAVuDTETGoF983ycMMsmmuAF4AvnJgzdZgJunDwGPARmrrUK4kW6uVWl00y8V5JFQbkk4hWyjeRnaC476IuCb//LyXbNpvHTA3P7M0aPWTi4eBMWTTgOuBi+ouRBj0JM0AvhERZxdVF0k2a2ZmZmYDRYrToGZmZmYDhps1MzMzswpzs2ZmZmZWYW7WzMzMzCrMzZqZmZlZhblZMzNrQNLeuu1ZkjZJGldmTGaWpiH/+xAzs3RJ+jhwE3BWRGwtOx4zS4+bNTOzJiR9hOyRSrMioqvseMwsTb4prplZA5J6gT3AjIjYUHY8ZpYur1kzM2usF3gcmFd2IGaWNjdrZmaN7Qc+A5wm6cqygzGzdHnNmplZExGxT9LZwGOSdkbE7WXHZGbpcbNmZtaPiHhFUgfwqKTdEfFA2TGZWVp8gYGZmZlZhXnNmpmZmVmFuVkzMzMzqzA3a2ZmZmYV5mbNzMzMrMLcrJmZmZlVmJs1MzMzswpzs2ZmZmZWYW7WzMzMzCrsv7Bvt8oE8DVbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9b75d66198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,6))\n",
    "plt.plot(range(1,40),error_rate,color='blue', linestyle='dashed', marker='o',\n",
    "         markerfacecolor='red', markersize=10)\n",
    "plt.title('Error Rate vs. K Value')\n",
    "plt.xlabel('K')\n",
    "plt.ylabel('Error Rate')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=7, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=7)\n",
    "knn.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Best k value to build the model is 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtree = DecisionTreeClassifier()\n",
    "dtree.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = SVC()\n",
    "model.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logmodel = LogisticRegression()\n",
    "logmodel.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-04-03 20:10:37--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "100%[======================================>] 3,642       --.-K/s   in 0s      \n",
      "\n",
      "2019-04-03 20:10:37 (652 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   50              Bechalor  female  \n",
       "1 2016-09-15   35       Master or Above    male  \n",
       "2 2016-10-09   43  High School or Below  female  \n",
       "3 2016-10-09   26               college    male  \n",
       "4 2016-09-25   29              Bechalor    male  "
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  \n",
       "0 2016-10-07   50              Bechalor  female          3  \n",
       "1 2016-09-15   35       Master or Above    male          4  \n",
       "2 2016-10-09   43  High School or Below  female          5  \n",
       "3 2016-10-09   26               college    male          5  \n",
       "4 2016-09-25   29              Bechalor    male          6  "
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor  female          3        0  \n",
       "1 2016-09-15   35       Master or Above    male          4        1  \n",
       "2 2016-10-09   43  High School or Below  female          5        1  \n",
       "3 2016-10-09   26               college    male          5        1  \n",
       "4 2016-09-25   29              Bechalor    male          6        1  "
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 \n",
    "\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor       1          3        0  \n",
       "1 2016-09-15   35       Master or Above       0          4        1  \n",
       "2 2016-10-09   43  High School or Below       1          5        1  \n",
       "3 2016-10-09   26               college       0          5        1  \n",
       "4 2016-09-25   29              Bechalor       0          6        1  "
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lets convert male to 0 and female to 1:\n",
    "\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "feature = pd.concat([feature,pd.get_dummies(test_df['education'])], axis=1)\n",
    "feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "feature.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature and Label selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = feature\n",
    "y1 = test_df['loan_status']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalize Data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = preprocessing.StandardScaler().fit(x1).transform(x1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [],
   "source": [
    "p1 = knn.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [],
   "source": [
    "v1 = jaccard_similarity_score(y1,p1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "v2 = f1_score(y1,p1,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "p2 = dtree.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "v3 = jaccard_similarity_score(y1,p2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [],
   "source": [
    "v4 = f1_score(y1,p2,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "p3 = model.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "v5 = jaccard_similarity_score(y1,p3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "v6 = f1_score(y1,p3,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "p4 = logmodel.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "p5 = logmodel.predict_proba(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "v7 = jaccard_similarity_score(y1,p4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "v8 = f1_score(y1,p4,average='micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "v9 = log_loss(y1,p5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "You should be able to report the accuracy of the built model using different evaluation metrics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard | F1-score | LogLoss |\n",
    "|--------------------|---------|----------|---------|\n",
    "| KNN                | ?       | ?        | NA      |\n",
    "| Decision Tree      | ?       | ?        | NA      |\n",
    "| SVM                | ?       | ?        | NA      |\n",
    "| LogisticRegression | ?       | ?        | ?       |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = pd.DataFrame([[v1,v2,'NA'],[v3,v4,'NA'],[v5,v6,'NA'],[v7,v8,v9]],['KNN','Decision Tree','SVM','LogisticRegression'],[ 'F1-score', 'Jaccard', 'LogLoss'])\n",
    "res.columns.names = ['Algorithm']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Algorithm</th>\n",
       "      <th>F1-score</th>\n",
       "      <th>Jaccard</th>\n",
       "      <th>LogLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>KNN</th>\n",
       "      <td>0.722222</td>\n",
       "      <td>0.722222</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Decision Tree</th>\n",
       "      <td>0.685185</td>\n",
       "      <td>0.685185</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVM</th>\n",
       "      <td>0.722222</td>\n",
       "      <td>0.722222</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogisticRegression</th>\n",
       "      <td>0.759259</td>\n",
       "      <td>0.759259</td>\n",
       "      <td>0.479193</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Algorithm           F1-score   Jaccard   LogLoss\n",
       "KNN                 0.722222  0.722222        NA\n",
       "Decision Tree       0.685185  0.685185        NA\n",
       "SVM                 0.722222  0.722222        NA\n",
       "LogisticRegression  0.759259  0.759259  0.479193"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2>Want to learn more?</h2>\n",
    "\n",
    "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n",
    "\n",
    "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n",
    "\n",
    "<h3>Thanks for completing this lesson!</h3>\n",
    "\n",
    "<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n",
    "<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n",
    "\n",
    "<hr>\n",
    "\n",
    "<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}