lecture_6_statistics.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from scipy.stats import norm\n",
    "\n",
    "sns.set(color_codes=True)\n",
    "\n",
    "import torch\n",
    "import torch.distributions as dst\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Definition\n",
    "\n",
    "<b>Statistics</b> is the discipline that concerns the collection, organization, displaying, analysis, interpretation and presentation of data <i>(Wikipedia)</i>.\n",
    "\n",
    "In statistics and probability we usually consider two main concepts:<br />\n",
    "1) <b>Sample Data</b> - The data we work with (usually, some numerical values). <br />\n",
    "2) <b>Data Generating Process</b> - Some abstract or concrete process that we assume have generated our data. Often we describe such process with probability distribution or some parametric family of probability distributions (e.g Bernoulli with potential parameter $p\\in[0.4, 0.7]$). <br /><br />\n",
    "Examples:<br />\n",
    "100 coin flip results generated from flipping a fair coin (Bernoulli(p=0.5))<br />\n",
    "Heights of 500 individuals from population of Georgian citizens.<br />\n",
    "10000 Celebrity faces from all the celebrity face pictures over the Internet.<br />\n",
    "...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probability VS Statistics\n",
    "\n",
    "<img src='images/stats/stats-vs-prob.png' style='height: 200px'>\n",
    "<br />\n",
    "<p><big>\n",
    "    Given a <b>data generating process (e.g. probability distribution)</b> with known parameters, the goal of <b>probability</b> is to make statements about data to be observed.\n",
    "</big></p>\n",
    "<ul><li>What is the probability of observing heads in case of flipping a fair coin?</li></ul>\n",
    "<p><big>\n",
    "    Given <b>sample data</b>, the goal of <b>Statistics</b> is to make statements about the <b>unknown properties</b> of the underlying <b>data generating process</b>.\n",
    "</big></p>\n",
    "<ul><li>Given 100 coin flip results, what's the bias of the coin (is coin fair)?</li>\n",
    "</ul>\n",
    "\n",
    "### Machine Learning\n",
    "\n",
    "Machine learning is closely related to Statistics. ML can be seen as a bunch of computer algorithms for learning data generating process properties from the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gaussian Distribution & Basic Concepts\n",
    "\n",
    "Gaussian Distribution is a continuous probability distribution which has a PDF of the following form:\n",
    "\n",
    "$f(x \\mid \\mu, \\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^2} } e^{ -\\frac{(x-\\mu)^2}{2\\sigma^2} }\n",
    "$\n",
    "\n",
    "parameters of the distribution are:\n",
    "\n",
    "$\\mu$ - is a mean (a.k.a. center) of the distribution.<br />\n",
    "$\\sigma^2$ - is a variance of the distribution.\n",
    "\n",
    "Gaussian with zero mean and unit variance is usually called <b>standard normal<b/> distribution.\n",
    "\n",
    "Below is the visualization of the PDF with different parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxcd33v/9fMaLTv0mi1Zcvbx/Iqy/JuJyF2GpxAQkhCSnIhKU1TLtDeFi7pvU24JeUX2h9dSIEkP3rTNlBjCgQSAlkAx45jZ/Mq7/56XyTL2mVZu2b5/TGjoMiLZqSRziyf5+Ohh3Q2zVtHM5858z3nfL82n8+HUkqp2Ge3OoBSSqmJoQVfKaXihBZ8pZSKE1rwlVIqTmjBV0qpOJFgdYBrSAKWAPWAx+IsSikVLRxAMbAT6Bu+MFIL/hJgm9UhlFIqSq0Btg+fGakFvx6gra0Lr3d09wnk5aXT0tIZ1lDhoLlCo7lCo7lCE2u57HYbOTlpEKihw0VqwfcAeL2+URf8we0jkeYKjeYKjeYKTYzmumpTuJ60VUqpOKEFXyml4kRQTToicj/wOOAEnjLGPH2N9W4HvmeMKQ9MZwM/AqYBTcCnjDEXwxFcKaVUaEY8wheRUuBJYDVQCTwiInOusl4h8I+Abcjs/wfYZoypAP4v8C/hCK2UUip0wTTprAM2G2NajTFdwAvAPVdZ7zngiWHzbsd/hA/wY2C9iDhHG1YppdToBVPwS/jwJT71wKShK4jInwN7gPeuta0xxg10AK7RhlXKal6fz//l9aFdi6toE0wbvh0Y+sy2Ad7BCRGZB9wNrGXYGwEfbt65YtuR5OWlB7vqVblcGWPafrxortBYnat/wMP2fXW88vZpjp1r/2B+eoqTtUvKuG3VVEryx/ZcDSer99e1aK7QjEeuYAp+Lf67tgYVAReGTN+L/1beXUAiUCIi24wxa4C6wPq1IpIAZAAtwYZraekc9bWoLlcGTU2XR7XteNJcobEyl8/nY/OeOn65/TSdPQMU56Vy+4opOB12UtOSOH62lV9vP8Uv3zrJwul5PLR+NlnpSZZkHaT/x9DEWi673XbdA+VgCv4m4Osi4gK68B/NPzK40BjzN8DfAIjIVODNQLEHeBX4LPBN4D78J3AHQv4rlJpgbo+X//yNYdv+euZMzeG25VOomJKDzeb/0Dr4gmzv7GNrzQVee/8sf/uDXfz53QuYUhSZR4xKjdiGb4ypAx4DtgA1wEZjzA4ReVVEqkfY/GvAchE5BHwB+OJYAys13jq6+/nHH+9l2/56Pr5yKl++r5I5U3M/KPZDZacncefqcv76vy3GZoO/27CbXUcbLUit1MhsEXriaSpwWpt0Jo7m8uvpc/N3G3bT0NbD526rYNmcwqBzXerq53u/2M/Jug6+8Il5VM8umIjII+aKBJorNGFo0ikHzlyxfMzJlIoRXp+P5359mLrmLv7s7vnXLPbXkpWWyKOfXsS0kkyee+Uw5xsjr1MuFd+04CsV8PL20+w93sx9N89kXnneqH6HM8HBlz45n9SkBL778/1c7u4Pc0qlRk8LvlLArqONvPz2GVbNL+KW6uFXF4cmOz2JP7t7Ae2d/Tzz4kHcnqCvRFZqXGnBV3HvUmcfz792lGklmXz21tlXPTkbqvLiTB5aL5jz7fxmx7kwpFRq7LTgq7j34zeO0+/28Me3V+BMCN9LYuW8YhbPcvHy22dobOsO2+9VarS04Ku4duBUCzuONHL7iqkU56WF/ffff8ssHHYb//nbY9oVg7KcFnwVt/oGPPznbwxFuanctnzKuDxGTkYSd984nUOnW3n/cMO4PIZSwdKCr+LWy2+fpvlSLw9+VMLalDPcRxaVUl6cwY/fOE5nj95orqyjBV/FpeZLPfx2x3lWzStCynLG9bHsdhsPfnQ2nd0DvPbe2XF9LKWuRwu+iku/fucMNht8Ys20CXm8ssIMls0t5I3dtVzq7JuQx1RqOC34Ku40tHWzff9FbqwsJS8recIe987V5bg9Pl55V4/ylTW04Ku488vtp0lw2PjYivE5UXsthTmprF5QxJs1dbRc6p3Qx1YKtOCrOFPX1Mn7hxpYu3iSJX3Xf3xlOQC/eufMhD+2UlrwVVx5aftpkhIdrB+nyzBHkpeVzI0LS9m+v54GvRlLTTAt+Cpu1Ld0sds0sa56MukpTsty3L5yCna7jd/sOG9ZBhWftOCruPG7XbUkOOysWzy2ztHGKjs9iRVzC3nnQL32pqkmVDBDHCIi9wOPA07gKWPM08OW3wU8ATiAncAjxph+EXkQ+Htg8BbDV4wxj4UrvFLButzdz9sH6lk5r5DMtESr4/AHSyazbX89b+6t4+Oryq2Oo+LEiEf4IlIKPAmsBiqBR0RkzpDlacD3gFuMMXOBZOChwOJq4MvGmMrAlxZ7ZYk399Yx4PZyS/Vkq6MAUOpKZ960XN7Y48+l1EQIpklnHbDZGNNqjOkCXgDuGVwYmDfVGNMgIqlAAdAWWLwEeFBEDojIBhEZ31salbqKAbeXN/bUMW9aLqWudKvjfODWJWV0dPVrHztqwgRT8EuA+iHT9cCHGkGNMQMish44D+QDvx2y7jeABYFl3xtrYKVC9f7hBjq6+rl1SZnVUT5kztQcSl1p/HbnOe1JU02IYNrw7cDQZ6MNuOIzqDHmNSBPRL4JPAvcb4y5a3C5iHwLOBlKuMBgvKPmcmWMafvxorlCM5ZcPp+PzXvrmFqcyY1LysIyuEk4cg265+aZ/MtPaqhr62WRhGfQ81j8P46neMoVTMGvBdYMmS4CLgxOiEguUG2MGTyq/xHwExHJAj5njPl2YL4NcIcSrqWlE693dEc+sTYa/XiL1VzHzrdzpr6Dh9bPprk5fIOKh2t/zZmcTUaqk5fePMGk3JSIyRVumis0o81lt9uue6AcTJPOJmCtiLgCbfR3A68PWW4DNojI4Ofle4HtQCfwqIgsC8z/EvBiiPmVGpOtNXWkJDlYVlFodZSrcibYWTW/mJrjzbRd1k7V1PgaseAbY+qAx4AtQA2w0RizQ0ReFZFqY0wL8AjwaxHZBwjwV8YYD/Ap4FkROQIsBh4drz9EqeE6ewbYebSJ5XOLSEp0WB3nmm5cWILX52P7gfqRV1ZqDIK6Dt8YsxHYOGzebUN+fgl46SrbbQOqxphRqVF59+BF3B4vNy4ssTrKdRXmplIxJYe3ai5w+4op2MN4nkGpofROWxWTfD4fW/ddoLw4k7LCyDwpN9SNlSW0dPRy+HSr1VFUDNOCr2LS8dpLXGju4sbKyD66H7Ropov0FCdv1lwYeWWlRkkLvopJW2sukJzoYGlFeC51HG/OBDurAydv23VELDVOtOCrmNPdO8Au08iKuUUkJwZ1mioi3FAZOHm7X0/eqvGhBV/FnB1HGxlwe1m9oNjqKCEpyk1l1qQs3jl4Ue+8VeNCC76KOe8cvEhxXipTiyL/ZO1wK+cXc7G1m9P1kXczkIp+WvBVTGls6+ZE7SVWzisKazcKE6VaCkhw2Hn34EWro6gYpAVfxZR3DzVgA1bMLbI6yqikJidQNSuf94804PZot8kqvLTgq5jh8/l452A9s6fkkJuZbHWcUVs5r4jOngEOnGyxOoqKMVrwVcw4UXeJpvZeVs6LzqP7QXPLc8lMdfKONuuoMNOCr2LGOwcvkui0s1hcVkcZE4fdzvK5RdScaKazZ8DqOCqGaMFXMWHA7WHnkUYWz3JF1bX317JyXhEer4+dR3Q0LBU+WvBVTNh/spXuPnfUnqwdbnJBOqWuNN7T4Q9VGGnBVzFhx5EGMlKdVEyNjWGTbTYbyyoKOV57idaOXqvjqBihBV9Fvd5+N/tONFM9uwCHPXae0oP9AO040mhxEhUrYufVoeJWzfFm+t3eiB3VarQKclIpL87gfW3HV2ES1NktEbkfeBxwAk8ZY54etvwu4AnAAewEHjHG9AeGPdwAFAAGeMAYE76BRZXCfwSck5HEjElZVkcJu6UVhfxk8wkaWrspzE21Oo6KciMe4YtIKfAksBqoBB4RkTlDlqcB3wNuMcbMBZKBhwKLnwGeMcbMBnYBXwtrehX3unoHOHCqhaUVBTE5UtTSikJs+M9RKDVWwTTprAM2G2NajTFdwAvAPYMLA/OmGmMaAoOcFwBtIuIEbgisD/A8/gHOlQqbPaYJj9fH0hhrzhmUk5HEzMnZ2o6vwiKYgl8CDO2gux6YNHQFY8yAiKwHzgP5wG8D3zuMMe5rbafUWO040kBBdkpU9owZrGUVBdQ1d1HbpK2hamyCacO3A0M757YBV/TqZIx5DcgTkW8CzwJfHbYdV9vuevLy0kNZ/QouV2QWAc0Vmmvlar/cx5GzbdyzdhYFBZkTnGri9tetq6bxo03HOXCmjUVzRu7jP9r+j1aLp1zBFPxaYM2Q6SLgg4E3RSQXqDbG/DYw60fAT4BGIEtEHMYYD1A8dLtgtLR04vWObiAIlyuDpqbI61Ncc4Xmerm27KnF64N5ZdkTnn2i91fFlBy27qnlo9WTrtvtczT+H60Ua7nsdtt1D5SDadLZBKwVEVegjf5u4PUhy23AhsAVOeBvp99ujBkAtgH3BeZ/FngtxPxKXdMu00RRbiqlrjSro4y7anHR2NbD+UZt1lGjN2LBN8bUAY8BW4AaYKMxZoeIvCoi1caYFuAR4Ncisg8Q4K8Cm38B/1U9h/F/Snh8PP4IFX86uvo5eq6N6tkFUTnQSaiqZrmw22zsMnryVo1eUNfhG2M2AhuHzbttyM8vAS9dZbuzwE1ji6jUlfYcb8Ln8x/5xoOM1ESkLJudR5u4a820uHiTU+Gnd9qqqLTraCOFOSlMLhjbif1oUj27gIbWbuqauqyOoqKUFnwVdS5393P0bHvcNOcMqprlwmZDm3XUqGnBV1Fn7/FmvD4f1VJgdZQJlZWWiEzOZudRLfhqdLTgq6iz62gjruxkygrjpzlnUPXsAupbuqlr1mYdFTot+CqqdPYMcORs/FydM1zVLBc2/G96SoVKC76KKjXHm/F44685Z1B2ehIzJ2VpO74aFS34KqrsOdZEXmZSTPedM5LFUkBdUxcNrd1WR1FRRgu+iho9fW4Onm6lalZ8NucMqprlv/dg97Emi5OoaKMFX0WNA6dacHu8LI6Tm62uJS8rmfLiDHYbLfgqNFrwVdTYbZrITEtkRmnsjWwVqqpZLk7Xd+gA5yokWvBVVBhwe9h/soWqmfnY7fHbnDNoceCk9R5t1lEh0IKvosKh0230DXioivPmnEFFuamU5qdpwVch0YKvosJu00hqUgKzy3KsjhIxFosLc76dju5+q6OoKKEFX0U8t8dLzYlmFs7IJ8GhT9lBVbNc+Hz+exOUCoa+elTEM+fb6ep1x/3VOcNNLkjHlZ2sN2GpoGnBVxFvj2ki0Wlnbnmu1VEiis1mY/GsAo6caaO71211HBUFghoARUTuxz9alRN4yhjz9LDldwJP4B/u8DTwR8aYNhF5EPh7oCGw6ivGmMfCFV7FPq/Xx57jTcwvzyPJ6bA6TsSpmuXi9R3n2H+qmeVziqyOoyLciEf4IlIKPAmsBirxD1k4Z8jyTOBZ4HZjzEJgP/D1wOJq4MvGmMrAlxZ7FZJj59u41Nn/wd2l6sOmlWaSlZbIHr0JSwUhmCaddcBmY0yrMaYLeAG4Z8hyJ/DFwNi34C/4gwOaLwEeFJEDIrJBRPQSCxWSd/fX47DbWDgjz+ooEclus7FolosDp1rpH/BYHUdFuGCadEqA+iHT9cDSwYnAIOYvAohICvC/gO8OWfcfgXeAbwLfAx4INlxe3tj6O3e5IrODLc0VHJ/Px7sH61kwI58pkyOv/T5S9tdHlpTx5t466tp6KS2JnFzDaa7QjEeuYAq+HfANmbYB3uEriUgW/sK/zxjzAwBjzF1Dln8LOBlKuJaWTrxe38grXoXLlUFT0+VRbTueNFfwaps6qW/uYt3iSRGXLZL2V3FWEilJCWzeeZalc4siJtdQkbS/hoq1XHa77boHysE06dQCxUOmi4ALQ1cQkWJgG/7mnIcD87JE5C+HrGYD9FICFbQ9x5qw2WDRzHyro0S0BIedyhl57DvRgsdzxbGYUh8IpuBvAtaKiEtEUoG7gdcHF4qIA/gV8FNjzF8YYwYPyTuBR0VkWWD6SwSafpQKxh7TxOwpuWSnJ1kdJeJVzXLR2TPAodMtVkdREWzEgh84GfsYsAWoATYaY3aIyKsiUg3cAVQB94hITeDrOWOMB/gU8KyIHAEWA4+O21+iYkpTew/nGjtZPq945JUV88rzcCbYeXd//cgrq7gV1HX4xpiNwMZh824L/LiLa7xxGGO24X8zUCokewOdgq2YXww+baYYSVKig3nlubx7sJ5PrJ6KPY4HiFHXpnfaqoi0+1gTk1zpFOenWR0lalTNctFyqZcz9ZF3ElJFBi34KuJc6urnRO0lqmbpydpQLJzhHytAu0xW16IFX0WcmuNN+Pj9IB8qOOkpThZMz2f3sSZ8vtFdzqximxZ8FXF2H2vClZ3MJJc254Rq+fxiGlq7udDSbXUUFYG04KuI0t3r5siZNhbPKsCmJx5DtnyevwM1bdZRV6MFX0WU/aea8Xh92lnaKOVlpTC9JFMLvroqLfgqouwxTWSlJTKtNNPqKFGrapaLsxcv03ypx+ooKsJowVcRo3/Aw4FTrSya5dLryMdg8NPR3mM69KH6MC34KmIcOtNK34CHxdqcMyaFuamUutLYrc06ahgt+Cpi7DnWRGpSAlKWbXWUqFc108Xx2nY6uvqtjqIiiBZ8FRHcHi81x5upnJlPgkOflmO1WFz4fLD3uB7lq9/TV5aKCOZcO129bm3OCZPJBem4spO1WUd9iBZ8FRF2H2siyelgbnnkjWwVjWw2G4ulgCNn2ujuHbA6jooQWvCV5bxeH3uONTF/eh6JTofVcWLG4lkuPF4f+05oH/nKTwu+styJukt0dPVTLdqcE07lJZnkZCSxyzRaHUVFCC34ynK7TRMJDjvzp+VZHSWm2G02qma6OHi6lb5+j9VxVAQIagAUEbkfeBxwAk8ZY54etvxO4An849aeBv7IGNMmImXABqAAMMADxpjOMOZXUc7n87HnWCPzynNJSQrq6ahCsFhcvLGnlgOnWqierb2PxrsRj/BFpBR4ElgNVAKPiMicIcszgWeB240xC/EPZP71wOJngGeMMbPxj4z1tbCmV1HvzMXLtHT0ad8542Tm5CzSU5x6tY4CgmvSWQdsNsa0GmO6gBeAe4YsdwJfDIx9C/6CXyYiTuCGwPoAzwP3hiW1ihm7TRN2m43KmTrYyXhw2O1Uzcpn34lmBtzarBPvgvkMXQIMHRm5Hlg6OGGMaQFeBBCRFOB/Ad8F8oEOY4x7yHaTQgmXl5ceyupXcLkyxrT9eNFcfj6fj70nmlkwM5/ysmtfjqn7KzTDc928dApv7auntrWXpXOLLEoVPfsrUoxHrmAKvh0YOnyODbhiVGkRycJf+PcZY34QaAoaPuxOSKNRt7R04vWObuQelyuDpqbIG9tTc/3euYbL1Dd38QfVk6752Lq/QnO1XCXZyaQlJ7Dp/bOUF1gzqEw07a9IMNpcdrvtugfKwTTp1ALFQ6aLgAtDVxCRYmAb/uachwOzG4EsERm8sLp4+HYqvu0yjdhtNhZp+/24SnDYqZyZT82JJgbcIR1zqRgTTMHfBKwVEZeIpAJ3A68PLgwU9F8BPzXG/IUxxgdgjBnA/yZwX2DVzwKvhTO8il4+n4+dR5uQsmwyUxOtjhPzlswuoKfPw+EzrVZHURYasUnHGFMnIo8BW4BE4DljzA4ReRX4P8BkoApIEJHBk7m7jDEPA18AfiAijwPngE+Pxx+hok9tUxcNrd3cumSy1VHiwpypuaQmJbDraCMLZ+gJ8ngV1IXPxpiNwMZh824L/LiLa3xSMMacBW4aQz4Vo3YdbcRmQy/HnCAJDjuLZuaz93gzbo9XeySNU/pfVxPO5/OxyzQik7PJTNPmnImyeHYB3X1ubdaJY1rw1YS70NxFfUs3S/TOzwk1d2ouKUkOdh3Vm7DilRZ8NeF2Hm3EBlSJFvyJ5EywUznDxd7jTbg9erVOPNKCryaU/+qcRmZNziZLm3Mm3JLZBXT1ujl8ps3qKMoCWvDVhKpt8jfnLJtTaHWUuDS33H+1zo4jDVZHURbQgq8m1I4jDdhtNhZr3/eWcCbYqRIXe441ad86cUgLvpowPp+P9w83MGdqDhl6s5VlllUU0tvvYf9JvVon3mjBVxPmdP1lmi/1srRCm3OsNHtKNhmpTm3WiUNa8NWE2XGkgQSHjapZeqenlRx2O9VSwL6TzToSVpzRgq8mhNfnY8eRBuaV55Ga7LQ6TtxbWlFA/4CXmhPNVkdRE0gLvpoQx8+3097Zz9I5eu19JJg5OZvs9ERt1okzWvDVhNhxpJHEBDuV2nFXRLDbbCytKOTAqRa6ewesjqMmiBZ8Ne7cHi87A700JifqQOWRYmlFIW6Pj91Gu1qIF1rw1bg7eLqVzp4BVlg4vJ66UnlxBoU5Kbx76KLVUdQE0YKvxt17hy6SnuJk3rRrj1urJp7NZmPF3CLMuXZaO3qtjqMmQFAFX0TuF5HDInJcRL54nfV+KCIPDZl+UETqRaQm8PVkGDKrKNLT52bv8WaWVhRoH+wRaPncQnzA+4f15G08GLFBNTAY+ZPAYqAPeEdEthhjDg9ZpwT4PrAW2Dxk82rgy8aYH4c1tYoau41/HFVtzolMBTmpTC/N5N1DF1m/fIrVcdQ4C+aQax2w2RjTaozpAl4A7hm2zgPAL4GfDpu/BHhQRA6IyAYRyRlzYhVV3j10kYLsFKaVZFodRV3DirlF1DZ1ca7hstVR1DgLpuCXAPVDpuuBSUNXMMb8gzHmuatsWw98A1gAnAe+N8qcKgq1dvRy9Gwby+cWYrPZrI6jrmFpRSEOu433DmmzTqwL5ho5O+AbMm0Dgho9wRhz1+DPIvIt4GQo4fLy0kNZ/QouV8aYth8v8ZJr28GL+IDb10zH5Rr9/zJe9le4hJrLBVRXFLLjaCOfv7cSh3183pxjZX9NlPHIFUzBrwXWDJkuAi6MtJGIZAGfM8Z8OzDLBrhDCdfS0onX6xt5xatwuTJoaoq8j6jxksvn8/G7988yrSQTJ75R/+542V/hMtpcVTPzef/QRbbtOsfc8vBfTRVr+2u8jTaX3W677oFyME06m4C1IuISkVTgbuD1ILbrBB4VkWWB6S8BLwaxnYoBZxsuU9vUxap5erI2GlTOyCM1KYG3D9SPvLKKWiMWfGNMHfAYsAWoATYaY3aIyKsiUn2d7TzAp4BnReQI/qt8Hg1PbBXptu+vx5lg15GtooQzwcGyuYXsPtakXS3EsKDuczfGbAQ2Dpt321XWe2jY9Dagagz5VBQacHt471ADVbNc2jNmFFk9v5gte+p4/0gjH1lUanUcNQ70ThgVdnuPN9Pd52b1gmKro6gQTC3KYJIrje37tVknVmnBV2G3fX89eZlJVEzR2y6iic1mY/X8Yk7Xd1DX1Gl1HDUOtOCrsGrt6OXQ6VZWzS/GrtfeR53l84pw2G1s15O3MUkLvgqrtwPX3q+ar8050SgzNZGFM/J59+BF3J6gbrdRUUQLvgobr8/H2/vrmV2WjSs7xeo4apRWLyimo3uAfSdarI6iwkwLvgqbI2faaGzv4YaFJVZHUWMwf1ouORlJbK2pszqKCjMt+Cps3txbR3qKk8Wi49ZGM4fdzg0LSzh4upXG9h6r46gw0oKvwqLtch97jzezZkExzgR9WkW7GxaWYLfZ9Cg/xugrU4XFtn0X8Pp83FipzTmxICcjiYUz8ti+v54Bt568jRVa8NWYebxetu67wNzyXApyUq2Oo8LkI4tKudw9wJ5jOsh5rNCCr8Zs/8kW2i73cVOl3o4fS+aU55KflcyWvdqsEyu04Ksx27K3juz0RCpn5lkdRYWR3WbjpkWlHDvfTl1zl9VxVBhowVdj0tDazaFTrdywsASHXZ9OsWb1gmISHDY27661OooKA32FqjHZtKsWh8OmvSvGqMzURJbNKeTtg/V09mi3ydFOC74ate7eAbYfqGdZRSFZ6UlWx1Hj5JbqyfQPeNm2b8SB7lSE04KvRu2tffX0DXhYVz3Z6ihqHJUVZjC7LJtNu2u1f50oF1TBF5H7ReSwiBwXkS9eZ70fishDQ6bLROQtETkqIr8UkbGNSq4ihsfr5Y3d55HJ2UwpisxBoFX43LJkMm2X+/QSzSg3YsEXkVLgSWA1UAk8IiJzhq1TIiK/Au4ZtvkzwDPGmNnALuBrYUmtLLfnWDMtHX38wRI9uo8HC2fkU5Cdwu92nrc6ihqDYI7w1wGbjTGtxpgu4AWuLOwPAL8Efjo4Q0ScwA2B9QGeB+4da2AVGX638zyu7GQWzsi3OoqaAHabjXXVkzh5oYOTdZesjqNGKZiCXwIMHQ2hHpg0dAVjzD8YY54btl0+0GGMcV9rOxWdjte2c6LuEuuqJ2O36yAn8WL1gmJSkhJ4/f1zVkdRoxTMIOZ2wDdk2gYEc+Zm+HYEud0H8vLG1uTvckVm23K053rml4fITEvkkzfPIjkpmKfQ2ET7/ppo45nrjjXT+MmmY/R4fJQVZYa0bTzur7EYj1zBvFprgTVDpouAYK7PagSyRMRhjPEAxUFu94GWlk683uHvGcFxuTJoaro8qm3HU7TnOtdwmV1HGrjrhmlc7uhhvP+SaN9fE228c62cU8CLW0+w4dUj/MnH54y8wQTlGq1Yy2W32657oBxMk84mYK2IuEQkFbgbeH2kjYwxA8A24L7ArM8CrwXxeCqCvfLuWVKSHKyt0hut4lFGaiI3VZby/uEGmrSv/KgzYsE3xtQBjwFbgBpgozFmh4i8KiLVI2z+BfxX9RzG/ynh8bEGVtapb+li19FGbq6aRGqy0+o4yiK3Li3DbofXtC0/6gTVAGuM2QhsHDbvtqus99Cw6bPATaOPpyLJa++dw5lg5xa90Squ5WQksWp+Mdv3X+COVVPJ1ruso4beaauC0tzew7uHLnLDwhIy0xKtjqWOQiEAABGWSURBVKMstn5ZGR6vT6/YiTJa8FVQfrn9NHa7jfXLp1gdRUWAgpxUVs4rYvOeOlo7eq2Oo4KkBV+NqK65i3cOXWRt1SRyMvTju/K7c1U5Pp+Pl98+Y3UUFSQt+GpEL751iiSng/XLy6yOoiJIfnYKN1WWsn1/PQ2t3VbHUUHQgq+u63R9B3uONfHRpWVkpGrbvfqwj62cQkKCjZe2n7Y6igqCFnx1XT/fepL0FCe3aCdp6iqy0pO4pXoy7x9u4FxD5N3ApD5MC766pkOnWzl8po3bV0whZQK6UFDR6aPLykhNSuCFN0/i843uzng1MbTgq6tye7xs3HSMguwUbq7SPu/UtaUlO7lj1VQOnm5l38kWq+Oo69CCr65qy5466lu6uW/tDJwJ+jRR13fz4kkU56XyX28cZ8Cto2JFKn0lqyt0dPfz0vbTzC3PpVL7u1dBSHDY+fTamTS29bBplw6SEqm04KsrvPjWKfr6Pfzh2pnYbNrfvQrOvGl5LJyex8vvnKG9s8/qOOoqtOCrDzlzsYO3ai5w8+JSSvPTrI6joswfrp2J2+3lZ1tOWB1FXYUWfPUBt8fLf7x6lMy0RD6xutzqOCoKFeamsn55Ge8eauDAKT2BG2m04KsPvP7+Oc43dvKZW0W7P1aj9vGVUynOS+WHrx+lp8898gZqwmjBVwCcb7jMy2+fpnp2AVWzXFbHUVHMmeDgj9ZX0NrRxy+2nrI6jhpCC77C6/XxnZ/sJcnp4IFbZlkdR8WAGZOyWLt4Epv31HLsfLvVcVRAULdPisj9+EercgJPGWOeHra8EngOyATeAj5vjHGLyIPA3wMNgVVfMcY8Fq7wKjx+s/McR8+28fDHKsjSvu5VmHzyxmnUnGjm3189wqI5RVbHUQRxhC8ipcCTwGqgEv+QhcNHL94AfMkYMwuwAX8SmF8NfNkYUxn40mIfYU5d6OAXW0+xckExK+bqi1KFT3JiAg9/bA5N7T38f7/Yb3UcRXBNOuuAzcaYVmNMF/ACcM/gQhGZAqQYY94LzHoeuDfw8xLgQRE5ICIbRCQnfNHVWHX3uvn+ywfJTk/kz+6t1GvuVdjNmpzNHavK2bK7lrcP1FsdJ+4FU/BLgKH/qXpgUpDL64FvAAuA88D3Rp1UhZXP5+OHvzlKy6U+/vSOeaRr18dqnHx85VTmTc9jw2+PcVH7zbdUMG34dmBoF3g2wBvMcmPMXYMzReRbwMlQwuXlpYey+hVcrowxbT9eIiHXa++eYceRRj6zvoIVi/zvz5GQ62o0V2giMddX7l/Mn//TFv7vrw/zrS+tITmCel+NxP0F45MrmL1eC6wZMl0EXBi2vHj4chHJAj5njPl2YL4NCOmi3JaWTrze0XW36nJl0NQUef1zR0KuI2fb+P4v9jN/Wh43zi+iqelyROS6Gs0VmkjO9fDH5vDUz/bx98/v4L/fNQ97BDQhRvL+Gk0uu9123QPlYJp0NgFrRcQlIqnA3cDrgwuNMWeBXhFZFZj1GeA1oBN4VESWBeZ/CXgx5L9AhVVDWzfPvHiAgpwU/vSOudjt1r/oVHyYPy2P+z4yg93Hmnhpm46QZYURC74xpg54DNgC1AAbjTE7RORVEakOrPYA8G0ROQqkA98xxniATwHPisgRYDHw6Hj8ESo43b1uvvOC/2qJ/3HPAlKTI+djtYoPtyyZzJoFxfz6nTO8d/ii1XHiTlCveGPMRmDjsHm3Dfl5H7D0KtttA6rGmFGFQd+Ah+/8fD+NbT185b5KCnJSrY6k4pDNZuMztwoNbT38+ytHyEhNZO7UXKtjxQ290zYODLi9PP3iAY6fb+ePP1bB7Cl6dayyToLDzpc+OZ+i3FS++/P9HK/VO3Enihb8GOfxevnXlw9x8FQrD66fzXK941FFgPQUJ1/5w0XkZCTz1M/2ceZih9WR4oIW/Bg24Pby/ZcPs/tYE59eO5MbFpZYHUmpD2SlJfLVP6wkNcnJP/1XDScvXLI6UszTgh+jevrcPPWzfew62sh9N8/gliWTrY6k1BVyM5P56v2LSE1O4B9+vJeD2of+uNKCH4M6uvr51o/3Ys6188e3V3Dr0jKrIyl1TQXZKfz1f1tMUU4q//LCft47pFfvjBct+DHmdH0H3/jBTuqbu/jze+azan7xyBspZbGs9CQevb+KmZOy+NdfHebnW0+O+qZLdW1a8GPIW/su8Hcb9gDwVw9UsWB6vsWJlApeanICf/mphdywsIRX3j3Lt39aw+XufqtjxRQt+DGgu3eAf/v1YZ5/7SgyOYv/89ASyoszrY6lVMicCQ4eWj+bh9bPxpy/xN8+v5OjZ9usjhUz9FbLKHfgVAvPv3aU9s4+Pr5yKneuLtfuElTUu2FhCZML0vn+y4f41o/3cnNVKffcNJ3kRC1ZY6F7L0q1d/bx8zdP8vbBi5Tkp/GlT87Xo3oVU8qLM3nic0v5xdZTbNp1nv0nW/j0uplUzsjXsRtGSQt+lOkb8PCbHed47b1zuD1ebl8xhTtWTcWZ4LA6mlJhl+R08Ol1M1ksLn7w+lG++/MDVEzJ4b6bZ1BWGJndGkcyLfhRorffzZt7L/Cbnee41NnPYnFx703TtU8cFRdmTc7mic8tZWvNBV7adoon/mMn1bMLuH3FFC38IdCCH+FaO3rZWnOBzXtq6ep1UzElh8/fMRcp0/5wVHxJcNhZu3gSy+cW8vr759i8p5adRxtZMD2PdYsnMac8NyL62I9kWvAj0IDbw4FTrby17wIHTrWADxbOyOf2FVOYXppldTylLJWW7OTuG6ezflkZm/fU8btd5/nnn+4jPyuZNQuKWTG3iPzsFKtjRiQt+BGiu3eAw2fa2HOsiZoTzfT2e8hKT+T2FVNYs6AElz6BlfqQ1GQnH1s5lVuXlrH3eBNbay7w4rbTvLjtNFOLMqieXcDC6XmU5KfpSd4ALfgW6eod4NSFDk7UXuLwmVZO1Xfg8/l7EVxaUcBiKWDO1Bwcdr1VQqnrcSbYWVpRyNKKQprae9hlGtl1tIkX3jzJC2+eJDvd3+f+rLJsppdkUZSXGrdNP0EVfBG5H3gccAJPGWOeHra8EngOyATeAj5vjHGLSBmwASgADPCAMaYzjPkjntvjpam9h4ut3dQ2ddHc0cfxc21cbO0GwGbzX352+4qpzJ2aw4xJWVrklRolV3YK65dNYf2yKbRc6uXQmVYOnW5l38kW3j7o76MnNSmBKUUZTHKlUzEtj4wkB4W5qaSnOC1OP/5GLPgiUgo8iX+Iwj7gHRHZYow5PGS1DcDDxpj3ROTfgD8BngWeAZ4xxvyXiHwN+BrwV+H+I6zg8Xrp6nXT1TPA5e4BOrr66ejup72zj9aOPlo7emm+1EtLRy++IV2CFOSmUpKbysp5RUwvyWRqcSYpSfpBS6lwy8tK5oaFJdywsASvz8fFlm5OXrjEyboOzjdeZmuNv/1/UFpyAq7sFHIzk8nNSCInM4nM1ESy0hPJTE0kLdlJeoqTRKc9apuIgqk064DNxphWABF5AbgH+NvA9BQgxRjzXmD954EnROQ54AbgE0Pmb2WcC77P5+PouXaO1F7i0qUevD4fPh94vT58gMfrw+v14fH68Hi9/p89PtxeL263D7fHy4DHy4DbS7/bS/+Ah/4BD30DHnr7PfT0uenp99DX77nq49tskJORRG5GMtNKMlk+t4jCnBQKc1MpzU+jbFLOqEajV0qNnt1moyQ/jZL8NNYs8I8L4fX6cNvtHDreSGNbDw1tPTQHPo0fOtN6zde4w24jJSmBlCQHKYkJJCY6SHL6vxIT7CQk2P3fHYNfNhx2Gw6H3f/dbsM++GXzf9lsYLcHvtts3JCWNC77IZiCXwLUD5mu58Pj115t+SQgH+gwxriHzQ9aXl56KKsD0H65j3/6SU3IPe3ZbZCQ4MDpsOF0Okh0Okhy2kl0OkhOTCAnNZGUxARSkxNIS3GSmpRARloi6an+d/+czCSy05PITEvE4bh+k4zLFZnXDWuu0Giu0ERqrlLXlXXG5/PR0+em/XIfbZf7uNTZx+XuATq7+7nc3U9Pn5vuXv9Xb7+bvn4PnT29HzpIdAcOHN2e0Hv9bOns57O3zQnHn/chwRR8OzA0sQ3wBrF8+HyGbTeilpbOUXWR+g//fSUpaUm0t3djA2x2G/bAO6fDbsMWeJf9/Zc9LP3PuPsGaO0buO46LldGRB7ha67QaK7QRGsuJ1CQkUhBRuKoH8PnG2xR8LcmeAPT3sCXz+f7fUtE4Ps8KRzV/rLbbdc9UA6m4NcCa4ZMFwEXhi0vvsryRiBLRBzGGE9gnaHbjZucjCRcrgyS9dynUspiNpuNBIeNBAf+d5AgOMapA8RgSuImYK2IuEQkFbgbeH1woTHmLNArIqsCsz4DvGaMGQC2AfcF5n8WeC1syZVSSoVkxIJvjKkDHgO2ADXARmPMDhF5VUSqA6s9AHxbRI4C6cB3AvO/ADwiIofxf0p4PNx/gFJKqeAEdT2gMWYjsHHYvNuG/LyPD5/IHZx/FrhpbBGVUkqFg7ZyK6VUnNCCr5RScUILvlJKxYlIvaffAYz52vhIHdtVc4VGc4VGc4UmlnIN2eaqQ+DZfL7Qb2yaAKvxX9KplFIqdGuA7cNnRmrBTwKW4O+O4eodWiillBrOgf8m1534O7v8kEgt+EoppcJMT9oqpVSc0IKvlFJxQgu+UkrFCS34SikVJ7TgK6VUnNCCr5RScUILvlJKxYlI7VohbERkEfCeMWZ8RgUOkYisAZ4CEoHTwIPGmDZrU0FgAJtv48/VAnwu0L11RBCRbwAeY8zXLc5xP/5xHZzAU8aYp63MM5SIZALvAB8zxpyxOA4AIvI3wKcCk68YYx61Ms8gEflb4B78w7D+mzHmny2O9CEi8o9AvjHmoXD+3pg+wg+M0PVd/EUsUvwH8BljzHzgMPBVi/MM+hHwsDGmMvDzd0ZYf0KISJaI/BvwlQjIUgo8ib/rj0r8g/uEf6TpURCRZfhvpZ9ldZZBIrIO+ANgEf79tVhE7rI2FYjIjcDNwAKgGvgzERFrU/2eiKwFHhyP3x3TBR/4J/xH05GkwhhzWEScQCkQCUf3ScDjxpj9gVn7gTILIw11J3Ac///SauuAzcaYVmNMF/AC/qPESPAnwBeZoHGjg1QPfMUY0x8Y8vQIEfC8MsZsBT5ijHEDBfhbOrqsTeUnIrn4Dyq+OR6/P2abdETkDiDVGPNCBL15Y4wZEJH5+McKHgD+2uJIGGP6gA0AImIHvg68ZGWmQcaYHwKIyNctjgJQgr+IDarnKiO9WcEY8zBAhD3XDw3+LCIz8TftrLr2FhMn8Dp8AvifwM+AOosjDfo+/iFlJ4/HL4/6gi8i9+Jvex7qKJCJ/4jMEtfKZYxZZ4w5ABSKyJ8CPwFWRkIuEUkEfoD/eTEuRxijyTWROUZgx9/mO8gGeC3KEjVEZC7wCvBVY8xxq/MMMsb8jYj8v8Cv8H9C+lcr84jIw8B5Y8wbIvLQeDxG1Bd8Y8zP8L9DfyCw4/438NbgEY+I1ABrjDGXLcyVLCKfMMYMHj1vYIKbKq6WK5AtHXgZ/wnbOwMfwS3PFWFq8Xc7O6iIyGpCiTiBiwF+DvyFMea/rM4DICKzgWRjTI0xpltEfoG/Pd9q9wHFgVqVC6SLyLeNMX8ZrgeI+oJ/NcaY54DnBqdFxBc4GWm1AeBpETlvjNmN/yPuFX1WW2QDcAL4vDFGj1qvbhPwdRFx4W/zvRt4xNpIkUtEJuNvGrzPGLPZ6jxDTAOeEJHV+D+x3Qn8u7WRwBhzy+DPgSP8m8JZ7CH2T9pGFGOMB/+7+L8G3sXvAR62NtUHl67eib99dY+I1IjIqxbHijjGmDr87atbgBpgozFmh7WpItr/BJKBfw48p2pE5PNWhzLGvIq/iWkvsBt4J1I+fYw37Q9fKaXihB7hK6VUnNCCr5RScUILvlJKxQkt+EopFSe04CulVJzQgq+UUnFCC75SSsUJLfhKKRUn/n+pxvTOXgu1FAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def create_gaussian_pdf(mu, sigma):\n",
    "    \n",
    "    return lambda x: 1 / np.sqrt(2 * np.pi * sigma ** 2) * np.exp(- (x - mu) ** 2 / (2 * sigma ** 2))\n",
    "\n",
    "def plot_dist(pdf):\n",
    "    \n",
    "    x = np.linspace(-4, 4, num=100)\n",
    "    y = pdf(x)\n",
    "\n",
    "    plt.plot(x, y)\n",
    "    \n",
    "plot_dist(create_gaussian_pdf(0, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXzU5b33/9dM9h0IE0hCAiTAh0UwIODCLosLKrIr1IXWWn+tPaftXftrb+1995wup+3xVE89djnVU7UYFXEDBRQEEUVAlsjqFcISSAgQEiAkIevM/cdMbBqWzGT7zvJ5Ph48HvNd5z1Dks93u67L5nK5UEopFbrsVgdQSillLS0ESikV4rQQKKVUiNNCoJRSIU4LgVJKhbhwqwP4KAoYA5QAjRZnUUqpQBEGpAKfA7UtFwZaIRgDbLI6hFJKBagJwCctZwZaISgBOHu2Cqezbe0fkpPjKSur7NBQHUFz+UZz+UZz+SbYctntNrp3jwPP39CWAq0QNAI4na42F4Km7f2R5vKN5vKN5vJNkOa67CV1vVmslFIhTguBUkqFOK8uDYnIIuAJIAJ42hjzbIvls4B/AWzAEWCJMeasiGQCS4EUwACLjTGVItINeBnIAkqBBcaYkx30mZRSSvmg1TMCEUkHfgmMB3KAh0VkaLPlicAfgZnGmGuB3cDPPIv/APzBGDMY2A781DP/F8AmY8wQ4C/Af3bIp1FKKeUzby4NTQPWG2PKjTFVwHJgXrPlEcB3jDHFnundQKaIRAATPesDvADM97yeifuMAOAV4DbP+koppbqYN4UgjX985KgE6NM0YYwpM8a8BSAiMcCPgbeBnkCFMabhMtt9tU/P8grA0faPoZS1nC6X+5/ThXbtrgKNN/cI7EDzn2wb4Gy5kogkAW8BXxhjXvRcUmr5G9G0na3F/Mvu80qSk+O9XfWyHI6Edm3fWTSXb6zIVVPbwJeF5ew9XMbRExWcPltN6dmLVF6s/2qd8DAbPbvF4OgWS2rPOIb068E12cn06hGLzdbyR7/r6P+jb0IplzeFoAh3a7QmvYETzVcQkVTgfWA98H3P7NNAkoiEGWMacTdvbtqu2LOfIhEJBxKAMm9Dl5VVtvlZWocjgdLSC23atjNpLt90Za7qmgZ2mNN8tu8kB4vO0+h0YbNBWnIcyUnR9OudQGJsJDYgNi6Ks+erKa+opex8DZ9+UcwHWwsB6JEYxdjBvbhhWC8yUuK7tCjo/6Nvgi2X3W676gG0N4VgHfAzEXEAVcBc4OGmhSISBqwElhljftE03xhTLyKbgIVALnA/sNqzeJVn+lee5ZuMMX8/pFLKDxSXVrJqyzG2m9PUNzjp1T2GW6/PRDK7kZ2WREzUpb8+LX9RnS4XJWeqyD9+jj2Hy1m7/Thrth2jjyOOGWMyuWFYL8LD9CluZa1WC4ExplhEHgc2AJHAc8aYbSKyCvg/QAYwCggXkaabyNuNMQ8B3wZeFJEngGPAvZ7lPwVeEJF9wDlgcUd+KKXao/DkBVZuPsrO/FKiIsKYMCKVm65JpX9qgs9H8XabjXRHPOmOeKaM6kPlxXo+P3CKj/JO8D+rDvDOJ4e59fq+TMpJ04KgLGMLsBtb/YAjemmo64RSrgvVdbyx8TCbvjhBTFQ400b3YdroDOJjvH+gzdtcLpeLPYfLeHdzIQXF5+nVI5bF0wdyTf/k9nyEdufqaprLNx1waag/cLTl8kDra0ipDudyufgo7wRvbjxETV0jM8ZmcOdN/YmN7rxfD5vNxojsngzPSmbP4XJeWZfP7177glGDHCyePojuCVGd9t5KtaSFQIW085W1PL/qAHsPlzM4sxuLZwjpPeO67P3dBSGZIX2v54PPj7Fy81H+z/NbeeDWwYwenNJlOVRo00KgQlbewTP8z6oD1NY38rUZg5gyMt2yxzsjwu3MvLEfoyWF/165jz+8vZdxw3uzaNqgy96UVqoj6U+YCjlOl4t3Nh1h5eajZPaK5+E7h5HWhWcBV9OrRyw/+dp1rPj0CO99VsjRkgt8d+5wUrrHWh1NBTF9TEGFlIu1DTz75h5Wbj7K+BGpPH7faL8pAk3Cw+zMmZjNDxbmcK6ylp+/uJ39R8utjqWCmBYCFTLKK2r41dIdfFFQxqJpA1ly22Aiwv33V2BYvx789IHRdIuP4nevfcFHecWtb6RUG/jvb4FSHaikrIpfLd1BeUUN3194LdNGZ1ja3YO3UrrH8r/vu45rsnrw0hrDys1HtS8j1eG0EKigd/RkBf+2dCcNDU5+dO8ohvXrYXUkn8REhfPonOHcOKwXb318mNfWF+DUYqA6kN4sVkGtoOg8v1uWR1x0BD+8J4dePQLzpmt4mJ1v3DGUuJgIPvj8OBdrG3jgtsHYA+CsRvk/LQQqaB0+UcHvluWRFBfJjxaNCvhGWnabjXunDiQmMpyVm48SZrdx3y0SEJe4lH/TQqCCUuHJC/zHa3kkxEbw2L0jA74INLHZbNw9oT9Ol4v3PiskzG5n0fSBWgxUu2ghUEGnuLSSJ1/dRWxUOI/dO5IeidFWR+pQNpuNOROzaGh08v6240RE2FkwZYDVsVQA00Kggkp5RQ2/W/YF4eF2Hls0kp5JMVZH6hQ2m40FUwZQ1+BkzdZjdIuLZMbYTKtjqQClhUAFjaqaep5a9gU1dQ38/4tGkdItOItAE5vNxuJpg7hQVcer6wtIio/i+qG9rI6lApA+PqqCQn1DI8+8sYeT5dU8Ons4mb38c5jBjma32/jmnUMZlNGN597dzwFtgazaQAuBCngul4sXVhvyj5/joTuGMiTA2gm0V0R4GP80dzi9e8Ty7Ft7OVlebXUkFWC0EKiAt2brMT7bd5LZE/qH7KWR2OgI/nneCOx2G/+5fDdVNTryq/KeFgIV0PIKzrD8o0OMHZLCHTf1szqOpXp2i+HROcM5c+4if3pnH41Op9WRVIDQQqACVvGZKv68Yh+ZvRNYcvsQfZYeGJTRjftuEfYdKWfZ+kNWx1EBQp8aUgGpuqaeZ9/cQ1REGN+dM5yoiDCrI/mNidemUVRaydrtx8lKSwzZy2XKe14VAhFZBDwBRABPG2OevcJ6LwHrjTEviEgK8EGzxUmAwxgTLyKTgDeB455lu4wxS9r6IVRocblc/H5ZHqfPXuSxe3OCrsFYR1gwZQBHT17ghdVfkpESj8MRGk9RqbZptRCISDrwS+A6oBbYLCIbjDH7m62TBvwZmAqsBzDGnAZyPMvtwIfA455NRgNPGmP+reM+igoV67YX8ekXJ5g/ORvJ7G51HL8UHmbn/5t1Df/y1208+9Ye/jOrp9WRlB/z5h7BNNxH+eXGmCpgOTCvxTqLgXeAZVfYxxKg2hiT65keA8wQkd0iskJEMtqQXYWggqLzLNtQwA3X9ObW67Ul7dV0T4jiW7Ou4WR5Nf+1LE/HMVBX5E0hSANKmk2XAH2ar2CM+XdjzHOX21hEwnCfCfy42exzwDPGmBHAKuBVX0Kr0FRVU8+fV+ylR2IU37tnlN4c9sKQvt2ZMzGLj/OK2bS7pPUNVEjy5h6BHWh+KGEDfHku7VbgoDFmT9MMY8wjzV7/SUR+LSJJxpjz3uwwOTneh7e/lL9eL9VcV+ZyuXjupc85V1nHb787gbiYCOJiIqyOdVn+8H01d/8d13DoxAVe+fAgY4enkeFnra797ftqEkq5vCkERcCEZtO9gRM+vMfdNDvi99wv+Anwa2NMY7P1GrzdYVlZJU5n205zHY4ESksvtGnbzqS5rm5jXjGbd5cwf0o23WPcP7b+kKslf/m+Wvr+olF857fr+bcXtvHE/dcREe4fT1n56/cVbLnsdttVD6C9uTS0DpgqIg4RiQXmAmt8yHAjsKlpwhjjBGZ79oOI3A9s9dx/UOoSxWeqeGXdQYb1684t2sNmm/RIjOahO4Zw/HQlyzZo+wL1j1otBMaYYtzX+DcAeUCuMWabiKwSkdFevEcW7rOK5h4Avici+3DfSH7It9gqVDQ0OvnLin1ERYbx0B1DdWjGdhiR3ZPpozP4cEcRew+XWR1H+RGv2hF4nvbJbTHv9sus9+Bl5l0ySKwxZh9wk9cpVch655MjHDtdyXfnDCcpPjhGGbPSvMlZ7D9azvOrDvDzb1xPvJ/eZ1FdS7uYUH6roOg8q7YUMn54KiMHOayOExQiwt1nVpXV9Sz9wFgdR/kJLQTKL9XUNfDcu/tJTozm3mkDrY4TVPr2TmDW+P5sO3CaLftPWh1H+QEtBMovvb7hEKXnLvKNmUOIidIusTrabTdkkp2eyNL38zl7odbqOMpiWgiU3zlwtJwNu4qZNjpDu5DoJGF2Ow/NHEpDo5O/vW+01XGI00Kg/EptXSN/Xf0lKd1jmDMpy+o4Qa1Xj1hmT8wir+AMW/efsjqOspAWAuVX3th4iDPna1hy22DtWroLTB+dQXZaIi+vzed8VZ3VcZRFtBAov5F//Bwf7iji5lHpekmoi9jtNpbcPoTaeqc+RRTCtBAov1Df4L4klJwUzbzJ2VbHCSlpPeOYNb4fO0wpO8xpq+MoC2ghUH7h3c2FnCqv5v5bhehIfUqoq916fSaZKfEsXZtPdY3X3X6pIKGFQFmuuLSSVVsKuXFYL67pn2x1nJAUZrfzwG2DqaiqY/lG7Yso1GghUJZyuly8uMYQExXOwqnacMxK/VMTmT46g492FZN//JzVcVQX0kKgLLVxVzEFxedZePMAEmMjrY4T8u6e0J/kxCheXPMl9Q2+DDuiApkWAmWZ85W1LN94iCF9u3PTNb2tjqOA6Mhw7rtFKCmrZs22Y1bHUV1EC4GyzGvrC6hvcHLfLaLDTvqREdk9uU4cvLv5KKfPXbQ6juoCWgiUJfYfLWfL/lPcfkNfeve4pKdyZbF7pw7EbreRuzZfu58IAVoIVJerb3Dytw/ySekWw8wb+1odR11Gj8RoZo/vz+5DZezMP2N1HNXJtBCoLrdmq7vNwNdmDPKbsXPVpaaO7kNGSjy56/KpqdO2BcFMC4HqUmfOXeTdzwoZLQ6uydI2A/4szG7nvluEsxdqWbn5qNVxVCfSQqC61CsfHsRmg3u0zUBAGJCexPjhqXyw7TglZVVWx1GdRAuB6jJ7Dpex6+AZ7rypHz0So62Oo7w0b3I2kRFheuM4iHnVqYuILAKeACKAp40xz15hvZeA9caYFzzTDwC/Bpo6O3/PGPO4iGQCS4EUwACLjTGV7fkgyr/VNzjJXZtPr+4xzBiTaXUc5YPEuEhmT+hP7rqD7Mwv5TpJsTqS6mCtnhGISDrwS2A8kAM8LCJDW6yTJiIrgXktNh8N/MAYk+P597hn/h+APxhjBgPbgZ+283MoP/fB58c4dfYii6cPIiJcT0QDzZRR6fRxxPPqhweprW+0Oo7qYN78Rk7DfZRfboypApZz6R/8xcA7wLIW88cAD4jIHhFZKiLdRSQCmOjZD8ALwPy2fgDl/8orali5+SgjB/bUG8QBKsxu52szBlFWUcuqzwqtjqM6mDeXhtKAkmbTJcDY5isYY/4dQETGt9i2BHgS2Az8Cvgv4IdAhTGmodk6fXwJnZwc78vql3A4Etq1fWcJ1lwvvp+PywXfnp+DIzmug1IF7/fVWdqby+FIYOL+U7y/7RizpgykVwc1BAzW76uzdEYubwqBHWh+h8gGeNUblTFmdtNrEfktcAj4UYv94e3+mpSVVeJ0tu2mlcORQGnphTZt25mCNdfBonNs3FXEHTf1I8zp7LDPGKzfV2fpqFx33diXLXtL+OPyPL4ze7jf5OpowZbLbrdd9QDam0tDRUBqs+newInWNhKRJBH5frNZNqABOA0kiUhTS6JUb/anAo/T6SJ37UG6J0Qx8wZtQRwMeiRGM/OGvuwwpRwoPGt1HNVBvCkE64CpIuIQkVhgLrDGi+0qgR+JyPWe6UeBt4wx9cAmYKFn/v3Aat9iq0DwyZ4SCk9dYP6UbKIitQVxsLhlbCY9k6LJXZdPo1O7qg4GrRYCY0wx8DiwAcgDco0x20RklYiMvsp2jcAC4I8icgC4DvdlIYBv4376aD8wAfejqSqIVNc08MbGQwzsk8T1Q3pZHUd1oMiIMBbePJDi0io+2qUn88HAq3YExphcILfFvNsvs96DLaY3AaMus14hMNmHnCrAvLv5KJXV9dy7YKB2MR2ERg3qyZC+3Xl702FuGNaLuOgIqyOpdtAHulWHO3W2mrXbjzNueCr9eidaHUd1ApvNxj1TB1Jd28CKT45aHUe1kxYC1eGWrS8gPNzOnElZVkdRnSgjJZ6J16axfmeR9kMU4LQQqA514Gg5uw6e4Y4b+9ItPsrqOKqTzZ6QRUS4nWXrC6yOotpBC4HqME6ni1c+LCA5MZoZYzKsjqO6QGJcJHeO68cXh8rYe6TM6jiqjbQQqA7zyZ4SikormT8lWwecCSHTrsvA0S2a1z4s0MdJA5QWAtUhLtY28ObHhxmQnsSYwdo7ZSiJCLczf/IAis9UsWl3SesbKL+jhUB1iNVbj1FRVcfCqQP0cdEQdJ04GNgnibc/PszFWh3WMtBoIVDtVl5Rw/vbjnHD0F5kpyVZHUdZoOlx0orqelZt0d5JA40WAtVub2w8BMDcSdkWJ1FW6p+ayI3DevH+tuOcOX/R6jjKB1oIVLscKangs32nmDEmg+QkHX4y1M2dlI3NBm9sPGx1FOUDLQSqzVwuF6+tLyAxNoLbtXdRhbt30lvGZrB1/ykOn6iwOo7ykhYC1WZ5B8+Qf/wcsyZkERPlVbdVKgTcdn1fEmMjWLb+oA52HyC0EKg2aWh0suyjQ6QmxzLx2tTWN1AhIyYqnFkTssgvOs+ug2esjqO8oIVAtcnGvBOcKq9m/pQBhNn1x0j9o4nXppKaHMvrGwpoaNRGZv5Of4OVz6prGnjnkyMM6duda7N1MHp1qTC7nQVTBnDq7EU+2lVsdRzVCi0EymfvbTlK1cV6FkzRxmPqykZkJzOkb3dWfHqU6hptZObPtBAon5Sdr2Ht50XcMKw3fXsnWB1H+TGbzcaCKQOovKiNzPydFgLlkzc/dj8fPmeijjWgWte3dwI3DuvF2u3HKTtfY3UcdQVaCJTXCk9e4LN9J7XxmPLJnInZuFx/P4hQ/serh79FZBHuAeYjgKeNMc9eYb2XgPXGmBc80+OAp4BIoAz4ujGmUEQmAW8Cxz2b7jLGLGnPB1Gdy+VysWxDAfEx2nhM+SY5KZrpY/qwZssxZozJ0EuKfqjVMwIRSQd+CYwHcoCHRWRoi3XSRGQlMK/F5i8DDxljcjyvf++ZPxp40hiT4/mnRcDP7TlcxoHCs9w1rh+x0dp4TPlm5g39iIuJYNmGAm1k5oe8uTQ0DfdRfrkxpgpYzqV/8BcD7wDLmmaISBTwhDFmt2fWbiDT83oMMENEdovIChHR4az8mNPp4vUNh0jpHsPkkelWx1EBKDY6nDvH9eNA4Vn2Him3Oo5qwZtCkAY0H22iBOjTfAVjzL8bY55rMa/WGLMUQETswM+Atz2LzwHPGGNGAKuAV9uUXnWJT/aUUHyminmTsgkP09tKqm2mjEwnpVsMyzYU4HTqWYE/8eYc3w40/1+zAV43FRSRSOBFz3v9CsAY80jTcmPMn0Tk1yKSZIw5780+k5PjvX37y3I4/PMapT/mqqltYMWnRxnctzu3js/yq3YD/vh9gea6miV3DeM3L21nT+FZpo1132vyh1yXE0q5vCkERcCEZtO9gRPe7FxE4oEVuG8UzzLG1HvODn4C/NoY09hsda9bnJSVVbb5iMLhSKC09EKbtu1M/pprfd4Jyitq+NZdQzlzptLqOF/x1+9Lc13doNQEstISefG9/Qzuk0SftG5+kaslf/m+WmprLrvddtUDaG/O89cBU0XEISKxwFxgjZfvvxQoABYaY2oBjDFOYLZnP4jI/cBWz/0H5Ucqqup4Y8NBRg1yMLBPN6vjqCDQ1MjsXGUdH3x+vPUNVJdotRAYY4qBx4ENQB6Qa4zZJiKrRGT0lbYTkZHALGAcsFNE8kRklWfxA8D3RGQfsAR4qJ2fQ3WCdz49Ql29k3mTdeQx1XEGZXRj5MCerN5SyLkLtVbHUXjZjsAYkwvktph3+2XWe7DZ61247ydcbn/7gJt8Caq6VklZFRt3neC2m/rRu0es1XFUkJk3OZufPreN19Ya5kzob3WckKePgKjLemPjYSIi7NwzXayOooJQanIck3LSWP3ZUU6WV1sdJ+RpIVCXOFh0jp35pdx+fSbdEqKsjqOC1F3j+xMZYeeNjYesjhLytBCof9DUlUS3+EhmjM1sfQOl2igpLpI5Uwayw5RSUOTVk+Oqk2ghUP9ghynlUHEFd0/IIioizOo4KsjdPTGbpPhI7XrCYloI1FcaGp0s33iI9J5xjB+u4xCrzhcdFc7sCVkUFJ9nZ36p1XFClhYC9ZWPdhVz+uxF5k/Jxm73nxbEKriNG96btJ5xLP/okI5vbBEtBApwj0O84tOjDOnbneFZOg6x6jphdjvzJ2dz6uxFNuZ51WmB6mBaCBQAq7YUUqnjECuLjMhOZnBmN9755IiOb2wBLQSK8ooa1m4/zo3DeumgIcoSNpuNBTe7xzdevVXHN+5qWggUb358GJcLZus4xMpC/XoncsPQXnzw+XHKK3R8466khSDEFZ68wGd7TzJ9dB96JsVYHUeFuDkTs3C54C0d37hLaSEIYU2Nx+JiIph5o45DrKzXs1sM00f3YfPekxw75X/dQAcrLQQhbM/hcg4UnuXOcf2IjY6wOo5SAMy8sS9xMRG8tl4bmXUVLQQhqtHpZNmGAlK6xzBFxyFWfiQ2OuKr8Y33HNbxjbuCFoIQ9cnuEk7oOMTKT00ZmU5K9xhe31BAo1MbmXU2/QsQgi7WNvDWpiMMSE/iOnFYHUepS4SH2Zk3KZviM1Vs2l1idZygp4UgBK3ZeoyKqjoWTtXGY8p/XScOBvZJ4u1NR7hYq43MOpMWghBTXlHD+9uOMXZICtlpSVbHUeqKmhqZVVTVsXrrMavjBDUtBCHmrY8P43S5mDtJxyFW/i87LYmxQ1L4YNsxbWTWibQQhJDCkxfYvPck00Zn4OimjcdUYJg3KRuny90CXnUOrwavF5FFwBNABPC0MebZK6z3ErDeGPOCZzoTWAqkAAZYbIypFJFuwMtAFlAKLDDGnGznZ1FX4XK5eG39QeJiIrhDG4+pANLUyGz11mNMG92Hfr0TrY4UdFo9IxCRdOCXwHggB3hYRIa2WCdNRFYC81ps/gfgD8aYwcB24Kee+b8ANhljhgB/Af6zXZ9CtSrv4Bm+PHaOWeP7a+MxFXBm3tiPhNgIXv1QG5l1Bm8uDU3DfZRfboypApZz6R/8xcA7wLKmGSISAUz0rA/wAjDf83om7jMCgFeA2zzrq07Q0OhuPJaaHMvkkWlWx1HKZ7HR4dw9IYv84+fYmX/G6jhBx5tCkAY0f5C3BOjTfAVjzL8bY55rsV1PoMIY03CZ7b7ap2d5BaAPtHeSDTuLOXX2IgtvHkCYXW8LqcA08dpU0nrG8fqGAh3JrIN5c4/ADjQ/F7MB3vwvtNyOZtu1fHjd230CkJwc7+2ql+Vw+Gef+52R60J1HSs3HyVnkIObr+/XpnYDofR9dQTN5Rtfcj08ezg/+8sWtppS7p40oBNTBcf35S1vCkERMKHZdG/Am/HkTgNJIhJmjGkEUpttV+zZT5GIhAMJQJm3ocvKKnE623ad0OFIoLTU/3o17KxcuevyqaqpZ874/pw5U+k3udpLc/kmWHJlJsdyTVYPct83jOjXnYTYSL/I1VXamstut131ANqb6wTrgKki4hCRWGAusKa1jYwx9cAmYKFn1v3Aas/rVZ5pPMs3edZXHaikrIoNO4uZeG0afVLadxallL9YOGUAtXWNvPPJEaujBI1WC4Exphh4HNgA5AG5xphtIrJKREa3svm3cT9ltB/3WcUTnvk/BW4QkX2edb7T1g+gruy19QVERtiZPUFHHlPBI90Rz+SRaXy06wTFpb6f5apLedWOwBiTC+S2mHf7ZdZ7sMV0ITD5MuuVA3f5kFP5aM/hMnYfKmPBlAEkxnXO6bNSVrl7QhZb9p3i1Q8P8oOFOdpnVjvpIyRBqNHp5NUPD5LSPYZpo/u0voFSASY+JoJZ4/uz7+hZvjjk9e1FdQVaCILQR7tOUFJWzcKbB+hYAypoTRmVTu8esbz24UF9nLSd9K9EkKm8WM/bmw4zpG93cgb0tDqOUp0mPMzOPVMHcOrsRdZtL7I6TkDTQhBk3tp0mIu1jdw7baBeN1VBb0R2T4ZnJbNy8xHOV9VZHSdgaSEIIsdPV/LRrmKmjEynj0MfF1Wh4Z6pA6ird/LGxkNWRwlYWgiChMvlIndtPnHREcya0N/qOEp1mdTkOKaPzuDT3SUcKamwOk5A0kIQJLabUszxc8yemEV8jPbfp0LLneP6kRAXSe66fO2dtA20EASB2vpGlq0/SEZKPJOu1d5FVeiJiQpn7qQsDhVX8Nk+HdrEV1oIgsCqzwopq6hl0bSB2O16g1iFpnHDU+mfmsjrGw7pYPc+0kIQ4E6frWb11mPcMKwXktnd6jhKWcZus/G1GYOoqKpjxafaD5EvtBAEuFc/LCAszMb8yZ3bJa9SgaB/aiITrk1j3fYiis9UWR0nYGghCGBfFJwhr+AMs8b1p3tClNVxlPILcydlER0ZRu5avXHsLS0EAaq+oZFX1h0kNTlW+xNSqpmE2EhmT8ziQOFZPv/ytNVxAoIWggC1essxTp+7yKLpg7Q/IaVamJyTTmaveF5bX6A3jr2gf0EC0Omz1bz7WSFjh6QwrF8Pq+Mo5Xfsdhv3zRDOXqjVG8de0EIQYFwuF7nrDhIWZmPhzQOtjqOU38pOT2Litams/byIIh3A5qq0EASYXQfPsPtQGbPH6w1ipVozb/IAYqPDWfq+0RvHV6GFIIDU1jXyyrp8+jjimKo3iJVqVXxMBPMmZ5NfdJ7Ne7XF8ZVoIQgg73xyhLKKWu67RQiz63+dUt4YPyKV7PREXltfQOXFeqvj+CX9axIgjp26wAefH2fitWkM7NPN6jhKBQy7zcYDt5kNs4IAABPRSURBVAzmYm0DyzYUWB3HL3k1eL2ILAKeACKAp40xz7ZYngM8ByQCHwOPAD2AD5qtlgQ4jDHxIjIJeBM47lm2yxizpD0fJJg5nS5eet8QFxPOvMnZVsdRKuD0SYlnxpgMVm89xvjhqQzK0IOp5lo9IxCRdOCXwHggB3hYRIa2WG0p8KgxZhBgA75pjDltjMkxxuQAo4CjwMOe9UcDTzYt1yJwdRvzijl8ooJ7pg7ULqaVaqO7xvWnZ1I0L675Usc4bsGbS0PTgPXGmHJjTBWwHJjXtFBE+gIxxpgtnlkvAPNb7GMJUG2MyfVMjwFmiMhuEVkhIhnt+RDB7FxlLcs3HmZov+7cMLSX1XGUClhRkWF8bcYgSsqqWb2l0Oo4fsWbS0NpQEmz6RJgbCvLv3qkRUTCgMeBWc3WOQcsM8a8KSKPAK8C47wNnZzcvmEYHY6Edm3fWS6X67lVB2hsdPLP94wixaLhJwPp+/IHmss3XZlrqiOB7flnWLm5kBk39adPypXfO5S+L28KgR1o/gCuDXD6sPxW4KAxZk/TDGPMI81e/0lEfi0iScaY896ELiurxOls2zPBDkcCpaUX2rRtZ7pcrh2mlM27S5g7KYsIXJbkDqTvyx9oLt9YkWvuxCx2mdM8lbuTHy0aid126RgewfZ92e22qx5Ae3NpqAhIbTbdGzjhw/K7cR/xAyAidhF53HOm0Jx2CNJMdU0DS9caMlLiuWVsptVxlAoaSXGRLJgygPzj5/j4ixOtbxACvCkE64CpIuIQkVhgLrCmaaExphCoEZGmSzv3AaubbX8jsKnZ+k5gtmc/iMj9wFbP/QflsXzjISqq6njwtsHaqZxSHWz8iFSG9O3O6xsKOHuh1uo4lmv1L4wxphj3Nf4NQB6Qa4zZJiKrRGS0Z7XFwFMi8iUQD/y+2S6ycJ81NPcA8D0R2Yf7RvJD7fsYwcUcO8tHu4qZPjqD/qmJVsdRKujYbDbuv1VoaHSx9APtfsKrdgSep31yW8y7vdnrL/jHG8jN14u9zLx9wE0+JQ0RtfWN/HXVlzi6RTN7QpbVcZQKWr26xzJ7QhbLNhSw7cBprg/hp/L0moOfeevjw5w+d5Eltw0hKrLlbRSlVEeaMcZ91v3y2nwqquqsjmMZLQR+pKDoPGs/P86UkekM7qsD0SvV2ex2G1+/fTA1dQ28vDbf6jiW0ULgJ+rqG/mfVQfokRit3Ugo1YXSHfHcOa4/n395mh0mNIe21ELgJ/62+gAny6t58LbBxER5detGKdVBbrs+k769EnjpfUNFdehdItJC4Afyj5/jnY8PMXlkOsP669CTSnW18DA737hjCBdrG/jbmtB7ikgLgcVq6hp4/r399OoRy4IpeklIKav0ccQze0IWO/JL2biz5RPvwU0LgcVe33CIM+dq+N49o4iO1EtCSlnplrGZZKcn8qe39oRUQzMtBBbae7iMDbuKmTE2g2FZyVbHUSrk2e02Hpo5lPoGJ39ddSBkLhFpIbDIheo6nn/vAOk947ThmFJ+pFePWL5+5zD2Hiln/c5iq+N0CS0EFnC5XLy4xlBVU8837xxKZIQ2HFPKn9x+Uz9GZCezbEMBxWeCvxs0LQQW+GR3CTvzS5kzMZvMXv7Z57lSocxms7Hk9iFER4bxlxX7qG8I7hHNtBB0sdNnq8ldd5DBmd2YMVYHZlPKXyXFRfLgbYM5drqStzYdtjpOp9JC0IUaGp386Z19hNltPHTH0MsOiKGU8h8jBzqYnJPGmq3H2Hek3Oo4nUYLQRd6c+Nhjp68wJLbB9MjMdrqOEopLyycOpC0nnH85d39nA/Sjum0EHSRPYfLWLPtGFNGpnOdpFgdRynlpaiIMB6ZNYyLtQ08/+5+nEH4SKkWgi5wvrKW59/dTx9HHAtvHmB1HKWUj/o44rln6kD2Hinng23HrY7T4bQQdDKn08WfV+yjpq6Rb826Rh8VVSpATc5J4zpx8MbGQxQUn7c6TofSQtDJ3v7kCF8eO8d9twjpPeOsjqOUaiObzcaS2wbTIzGKP769lwtB1EupFoJOtOdwGe9uPsqEEamMG55qdRylVDvFRkfw7buHc6G6jr8E0f0Cr3o5E5FFwBNABPC0MebZFstzgOeAROBj4BFjTIOIPAD8GjjlWfU9Y8zjIpIJLAVSAAMsNsZUdsQH8hflFTX8ZeV++jjiWTx9kNVxlFIdpG/vBO6dNoi/vW9477NC7rypn9WR2q3VMwIRSQd+CYwHcoCHRWRoi9WWAo8aYwYBNuCbnvmjgR8YY3I8/x73zP8D8AdjzGBgO/DT9n8U/1Hf4OTZt/bS0Ojk27P1voBSwWZyTho3DO3F25sOB0X7Am8uDU0D1htjyo0xVcByYF7TQhHpC8QYY7Z4Zr0AzPe8HgM8ICJ7RGSpiHQXkQhgomc/LdcPCi+vNRwpqeAbM4fSu0es1XGUUh3MZrPxwK2DSesZx5/e2UvpuYtWR2oXbwpBGlDSbLoE6OPl8hLg58AI4DjwX0BPoMIY03CF/QW0j/KK+fiLEmbe2JfrxGF1HKVUJ4mKDOPROcNxueDZN/dQW99odaQ28+YegR1ofkfEBji9WW6Mmd00U0R+CxwCftRifVrsr1XJyfG+rH4Jh6NzOnr7srCc3LX5jJIUvjnnWsLsvnUh0Vm52ktz+UZz+SaQczkcCTx232j+9fktvPbRIX5w7yhsndx1TGd8X94UgiJgQrPp3sCJFstTWy4XkSTg68aYpzzzbUADcBpIEpEwY0yjZ9vm+2tVWVklTmfb7tY7HAmUll5o07ZXU15Rwy9e3E63+CgevFUoL/Pt3ndn5WovzeUbzeWbYMjVt2csd4/vz1ubjpCSGM2t12f6Ra7m7HbbVQ+gvbk0tA6YKiIOEYkF5gJrmhYaYwqBGhEZ55l1H7AaqAR+JCLXe+Y/CrxljKkHNgELPfPv96wfsGrrG3nmzT3U1DfyT/NGEB8TYXUkpVQXmnlTP0YPTuH1DQXsPnTG6jg+a7UQGGOKgceBDUAekGuM2SYiq0RktGe1xcBTIvIlEA/83nO0vwD4o4gcAK7DfVkI4Nu4nz7aj/ts44mO/FBdyeVy8ddVBzh28gLfunMYfRztu2yllAo8dpuNb8wcQkaveP68Yl/ADWZjC7AxOfsBR/zp0tCKT4/w9qYjzJ+czW039G3zfoLhFLkraS7faC7ftDVXeUUN//ridqIjwnj8/utIiI30i1zNLg31B45esrzdyULYZ/tO8vamI9w4rHenXhdUSgWGHonRfHfOcMov1PLMG3uobwiMJ4m0ELSROXaWv646wODMbiy5fXCnPymglAoM2elJfPPOoRQUn+e5dw8ERDcUWgjaoKSsiv96cw+ObjF8Z85wwsP0a1RK/d2YwSnMn5LN51+e5o2Nh6yO0yqv+hpSf3f2Qi1PLfuCMLuN782/lrhofUJIKXWpW8dmUnquhtVbjtEjIZqp1/lvu1ktBD6orqnnqWV5XLhYz4/uHYmjW4zVkZRSfspms7F4+kDOXagld20+CbERjB3Sy+pYl6XXNLxUV9/I75fvpqSsmkdnD6d/aqLVkZRSfi7MbueRWcMY0CeJv6zcz/6j/tlBnRYCLzQ0Ovnzin0cLDrPQ3cMZVj/HlZHUkoFiMiIMP5p3gh6J8fyzJt7OFJSYXWkS2ghaIXT6eL59w6w6+AZFk0fxPVD/fPUTinlv+KiI/jBghwSYiL43Wt5FJ32r+FXtBBchdPl4sU1X7J1/ynmTsry65s9Sin/1j0hih/eO5LIiDCefHUXJWX+0/pYC8EVuFwuXll3kE27S7jjpn7MvLGf1ZGUUgEupVsMP7wnBxfw5Kt5nPaTcQy0EFxGUxH4cEcRM8ZkMHtCf6sjKaWCRGpyHD+8ZyR19Y38NnenXxQDLQQtuFwuctcdZJ2nCCy8eYC2GlZKdaiMlHgeu3cktXWN/OblnZw+W21pHi0EzThdLl5em8+HO4q4ZawWAaVU58nslcBj946kvsHJb3J3carcumKghcCj0enkr6sOsH5nMbeOzWTBFC0CSqnO1bwY/NvLOy17mkgLAVDf4ORP7+zj0z0nuXt8f+ZPydYioJTqEhkp8fx48SjC7DZ+k7uTQyfOd3mGkC8ENXUNPPPGbnaYUu6dOpC7xvfXIqCU6lJpPeP4yeJRxEVH8OQreezr4hbIIV0IzlfV8ZvcXew7Ws6S2wYzfUyG1ZGUUiGqZ7cYfvy1UTi6RfP0si/4bN/JLnvvkC0EJ8ur+dXftlNyporvzh3BhGvTrI6klApx3eKj+PHiUQz09E20akshXTGKZEgWgvzj5/jV33ZwsbaRxxaNJGdAT6sjKaUUALHREXx/QQ5jh6Sw/KNDLP0gn4ZGZ6e+Z8h1Q/3h58d4ZlkePZOi+d78a+nVI9bqSEop9Q8iwu08fNcwkhOjWb31GKfOVvPtu6/ptPfzqhCIyCLgCSACeNoY82yL5TnAc0Ai8DHwiDGmQUTGAU8BkUAZ8HVjTKGITALeBI57drHLGLOkIz7Q1Sz/6BCrthQypG93vj37Gh1URinlt+w2G/OnDKB3ciwvrTH84qUd/PyRmwjrjPdqbQURSQd+CYwHcoCHRWRoi9WWAo8aYwYBNuCbnvkvAw8ZY3I8r3/vmT8aeNIYk+P51+lF4EJ1He9vO8atN/bj+wt0ZDGlVGCYMCKNH96TQ+XFej78/HjrG7SBN/cIpgHrjTHlxpgqYDkwr2mhiPQFYowxWzyzXgDmi0gU8IQxZrdn/m4g0/N6DDBDRHaLyAoR6fTHdRJiI/n9P0/gO/Ou1TGGlVIBRTK78x/fGceCaYM6Zf/e/EVMA0qaTZcAfVpbboypNcYsBRARO/Az4G3POueAZ4wxI4BVwKttSu+jmKiQuyWilAoSEeF2IsI75yDWm7+MdqD580s2wOntchGJBF70vNevAIwxjzQtN8b8SUR+LSJJxhivmtQlJ8d7s9oVORwJ7dq+s2gu32gu32gu34RSLm8KQREwodl0b+BEi+Wpl1suIvHACtw3imcZY+o9Zwc/AX5tjGlstl2Dt6HLyipxOtv2bK3DkUBp6YU2bduZNJdvNJdvNJdvgi2X3W676gG0N+cZ64CpIuIQkVhgLrCmaaExphCo8TwhBHAfsNrzeilQACw0xtR61ncCsz37QUTuB7Z67j8opZTqYq0WAmNMMfA4sAHIA3KNMdtEZJWIjPasthh4SkS+BOKB34vISGAWMA7YKSJ5IrLKs/4DwPdEZB+wBHioQz+VUkopr3l199QYkwvktph3e7PXXwBjW2y2C/f9gsvtbx9wk09JlVJKdQp9jlIppUJcoD1PGQbuGx/t0d7tO4vm8o3m8o3m8k0w5Wq2zWUbJtu6ome7DjQe2GR1CKWUClATgE9azgy0QhCFu1VyCdDYyrpKKaXcwnA/5v85UNtyYaAVAqWUUh1MbxYrpVSI00KglFIhTguBUkqFOC0ESikV4rQQKKVUiNNCoJRSIU4LgVJKhbhA62Kiw3h6R91ijImyOguAiEwAngYigSPAA8aYs9amAk/34k/hzlUGfN3T9bhfEJGfA43GmJ9ZnGMR8AQQATxtjHnWyjzNiUgisBm4wxhz1OI4AIjI/wUWeCbfM8b8yMo8TUTkX3EPxesCnjfG/M7iSP9ARJ4EehpjHuzI/YbkGYFnXIVncP9x8xd/Be4zxgwH9gOPWZynycvAQ8aYHM/r31ucBwARSRKR54H/5QdZ0oFf4u4CJQd4WESGWpvKTUSux92lQOcMdtsGIjINmAGMxP19XScis61NBSIyCbgZGAGMBr4rImJtqr8Tkam4u/DvcCFZCID/wH307U+GGGP2i0gEkA74w9lAFPCEMWa3Z9ZuINPCSM3NAg7i/r+02jRgvTGm3DPA0nLcR5X+4JvAd/jHUQWtVgL8L2NMnTGmHjiAH/xcGWM2AlOMMQ1ACu4rJn4xYJaI9MB9sPGrzth/yF0aEpG7gFhjzHI/KvZ4hvEcjntEuHrgf1scCc+ocksBPEOM/gx428pMTYwxLwGIyM8sjgKQhvuPW5MSLh2fwxLGmIcA/OxnfV/TaxEZiPsS0bgrb9F1PL+H/wL8EHgdKLY4UpM/4x4gLKMzdh60hUBE5uO+tt3cl0Ai7iM4S1wplzFmmjFmD9BLRL4FvEYXDt5ztVwiEgm8iPvnpVOOSNqSqytztMKO+5pyExvgtChLwBCRYcB7wGPGmINW52lijPm/IvIbYCXuM6r/tjKPiDwEHDfGfCgiD3bGewRtITDGvI67on/F84X+BPi46QhJRPKACcaYLhmp+gq5okXkbmNM09H2Urr4ksflcnmyxQMrcN8onuU5lbc8l58pwt29b5Pe+NelGL/jeQjhDeB7xphXrc4DICKDgWhjTJ4xplpE3sR9v8BqC4FUz9+qHkC8iDxljPl+R71B0BaCyzHGPAc81zQtIi7PTVCr1QPPishxY8wO3KfKl/QZbpGlQAHwiDFGj3Ivbx3wMxFx4L6mPBd42NpI/ktEMnBfYlxojFlvdZ5msoB/EZHxuM/wZgH/Y20kMMZMb3rtOSOY3JFFAEL3ZrFfMcY04q76/+2p+vOAh6xN9dUjtrNwX7/dKSJ5IrLK4lh+xxhTjPv67QYgD8g1xmyzNpVf+yEQDfzO8zOVJyKPWB3KGLMK96WqXcAOYLO/nK10Nh2PQCmlQpyeESilVIjTQqCUUiFOC4FSSoU4LQRKKRXitBAopVSI00KglFIhTguBUkqFOC0ESikV4v4fJcb6X7cA3c8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_dist(create_gaussian_pdf(0, 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXhb13nn8e8FCO6LuIALKErWYr2WvEmWnNVOnEhJWyeNk9ppU6edun1cNdNMZ6ZLutnT2M04M909M/V00jiZNhOzi52lTaI4qSI9Xmq7imXJdiz5WKIsiQQpEgR3cCcwfwCQYVgSAQjAvbh4P4/9iBf3AvgRIF8enHvuOVYsFkMppZR7eOwOoJRSKr+0sCullMtoYVdKKZfRwq6UUi6jhV0ppVymwubnrwJuBIaAFZuzKKVUqfACXcAPgYX0nXYX9huBp2zOoJRSpepm4On0G+0u7EMA4+MRotHsx9O3ttYTDs/kPdTlcmoucG42zZUdzZUdt+XyeCyam+sgUUPT2V3YVwCi0VhOhT15Xydyai5wbjbNlR3NlR2X5rpgF7aePFVKKZfRwq6UUi6TUVeMiNwJ3Av4gAeNMQ+l7b8B+AJQCfQDP2eMmchzVqWUUhlYtcUuIt3AA8BNwHZgr4hsSzvsfwB/YIy5HjDAb+U7qFJKqcxk0hWzBzhgjBkzxkSAx4A70o7xAo2Jr2uBufxFVEoplY1MumICvHlIzRDwtrRjfgP4vog8CESAt+cnnlL2iCams45GY8RiMSzLsjmRUpnLpLB7gNTxOBYQTW6ISA3wJWCPMeaQiPwG8BXgQ5mGaG2tz/TQt/D7G3K+byE5NRc4N5tTcv3lo0f53nNnzm9ftb6Zz//qu/FVeG1M9VZOeb3Saa7sFCJXJoV9gPjVTUmdwGDK9jXAnDHmUGL7C8DnsgkRDs/kNJbT728gFJrO+n6F5tRc4NxsTsn1o9fDfO+5M9x4VTvdbXVELYt/fuoUj+w7xofeeYXd8c5zyuuVTnNlJ9dcHo91yQZxJoV9P3CfiPiJd7PcDuxN2X8S6BERMcYY4Dbi8xcoVVKWlqM88v3X6Giu4e4Pb8NX4cHvb2BgeJpvPXOad2zrpLWp2u6YSq1q1ZOnxpggcA9wEDgK9Ca6XPaJyC5jzDhwF/CPIvIS8EvALxYws1IF8b1DZxken+OTH9yCr+KNX41P7N4MMfj7H5ywMZ1SmctoHLsxphfoTbvt1pSvvwt8N7/RlCqe0Yk5vv3MaXaJn2s2tL5pX1tTDT/57iv42hOnePlUmGs3tl7kUZRyBr3yVCngsSf6sCyLT+y+8oL7P3jjOjpaaundfwJdAF45nRZ2VfYWFlc4cmKUm67roqXxwn3ovgoPH37neobHZjl9znkn4ZRKpYVdlb0fvR5maTnKzi3+Sx53/eY2PJbFkROhIiVTKjda2FXZe+G1UeqqK7iyp+mSx9XX+NjS08SR10aLlEyp3GhhV2VteSXKiydH2b65Da9n9V+HG7b4CY5GGB6bLUI6pXKjhV2Vtdf6J5hdWGbHKt0wSTuujB/3gnbHKAfTwq7K2pHXRqms8HD1hpaMjm9tqmZ9R4N2xyhH08KuylYsFuOFEyGu3tBClS/zeWB2bGmjLzjJ5MxbFodXyhG0sKuydfrcNOPTC9yQYTdM0g1b/MSAIye11a6cSQu7KltHToSwrPgwxmx0t9XRvqZGu2OUY2lhV2Xr6IkwW9auob7Gl9X9LMtix5Y2jp0eY2HxgovEK2UrLeyqLM3OLxMMzbB1fXNO979qXTMr0Rinz03lOZlSl08LuypLp4YmiQGbui99UdLFJO93MjiZx1RK5YcWdlWW+oJTWMDGQOOqx15IfY2PjpZa+oLaYlfOo4VdlaW+4CTd/jpqqjKaufqCNgca6Ruc1NkeleNoYVdlJxqL0Tc4lXM3TNKm7iamZ5cITczlKZlS+ZFRc0VE7gTuBXzAg8aYh1L2bQf+JuVwPzBujLkmjzmVypuh8CxzC8tsClx+YYd4t057c20+oimVF6u22EWkG3gAuAnYDuwVkW3J/caYo8aY7caY7cC7gHHgUwXKq9Rl60uc8NzUnVv/elJ3Wx3VlV5ODuoJVOUsmXTF7AEOGGPGjDER4DHgjosc+3vAE8aYp/MVUKl8OxmcpK66gs6Wy2tlezwWGwON9A1oYVfOkklXTAAYStkeAt6WfpCINAF7gWuzDdHaWp/tXc7z+xtyvm8hOTUXODdbsXKdGZ5m64ZW2tsza7FfKte1V/p5dP9r1DfWXNaJ2FyU+/uYrXLKlclPogdIPe1vAdELHPdzwDeNMSPZhgiHZ4hGsx9Z4Pc3EAo5b5kyp+YC52YrVq7I/BL9wzPskvaMnm+1XF1raojG4IcvD+Z8sVMuyv19zJbbcnk81iUbxJl0xQwAXSnbncDgBY77KPD3WaVTqshODcbHnW/Ocfx6umQ/fZ9eqKQcJJMW+37gPhHxAxHgduJdLueJiAXsBJ7Ne0Kl8qgvOIllwYY8Ffa6ah9drbVa2JWjrNpiN8YEgXuAg8BRoNcYc0hE9onIrsRhfmDRGDNfuKhKXb6+4CRr/fVUV+avP3xToIm+wSm9UEk5RkY/3caYXqA37bZbU74eId5Fo5RjxWIxTg1N8/at7Xl93I3djTz98hChiTkdz64cQa88VWUjPDnP3MIy6zryOwphfeLx+kdm8vq4SuVKC7sqG8nC29Oe+/DaC+luq8OytLAr59DCrspG/8gMFrDWn9/CXunz0tlSq4VdOYYWdlU2+kdmaG+uoaoy84WrM9XTXq+FXTmGFnZVNvpHZvLeDZPU017P6OQ8s/PLBXl8pbKhhV2VhfnFZUYm5gpa2AEGQtpqV/bTwq7KwkAoAsDaAhX2ZL+9dscoJ9DCrspCoUbEJDU3VFFXXaGFXTmCFnZVFvpHZqitqqC1sbogj29Zlp5AVY6hhV2Vhf6Rada212NZVsGeo6e9gWAot5lKlconLezK9aKxGAMjkYJ1wyT1tNezuBxleHy2oM+j1Gq0sCvXC03MsbC0UpTCDnoCVdlPC7tyvf7hwp44TQq01eH1WFrYle20sCvX6x+ZwbLic7oUkq/CQ2erTi2g7KeFXble/8gMnS21VPryP5VAOh0Zo5xAC7tyvUJOJZCup72e8ekFZuaWivJ8Sl1IRgttiMidwL2AD3jQGPNQ2n4BvgA0A+eATxhjxvOcVamszS0sE56a55YdgaI8X0/iCtRgaAZZV7zFrZVKtWqLXUS6gQeAm4DtwF4R2Zay3wL+GfjvxpjrgSPA7xYmrlLZGQzHpxIIFLh/PSn5PIOjkaI8n1IXkklXzB7ggDFmzBgTAR4D7kjZfwMQMcY8ntj+PPAQSjlAssAWq7A3N1RRXellcFTHsiv7ZNIVEwCGUraHgLelbG8GzonIl4AdwHHg17IJ0dqae/+n35/fZc7yxam5wLnZCpFrYnYZX4WHrZvb8Xpyu+o021zrOxsJTc0X/HUup/cxH8opVyaF3QOkXiNtAdG0x7gFeI8x5nkR+Rzw58BdmYYIh3O7DNvvbyAUms76foXm1Fzg3GyFynXy7DidLbWMhXMbqZJLLn9TNS+dChf0dS639/FyuS2Xx2NdskGcSVfMANCVst0JDKZsnwNOGGOeT2z/HW9u0Stlm8HRSMHHr6cLtNUxFVnUkTHKNpkU9v3AbhHxi0gtcDvweMr+ZwC/iFyf2P5J4HB+YyqVvfnF+IiYLhsKO+gJVGWfVQu7MSYI3AMcBI4CvcaYQyKyT0R2GWPmgI8BXxSRV4D3A79ZyNBKZWIoHD+BGWgtdmGvBd4YkaNUsWU0jt0Y0wv0pt12a8rX/4Z2vyiHeWNETG1Rn7elsZoqn5fBkBZ2ZQ+98lS51uBohAqvRXtzTVGf12NZBNpqtcWubKOFXbnW4GiEzpZavJ7i/5gHWuu0j13ZRgu7cq3BcKRoFyalC7TVMTGzyOy8joxRxaeFXbnSwtIKoxPzRT9xmtR1fmSMXoGqik8Lu3Klc+FZYhRvKoF054c8aj+7soEWduVKxZ4jJl1bUzWVFR7tZ1e20MKuXGkwHMHrKf6ImCSPZdGlJ1CVTbSwK1cKhiJ0tNRS4bXvRzzQVktQC7uygRZ25UqD4QiB1uJemJQu0FbH+PQCcwvLtuZQ5UcLu3KdpeUVQhNztvWvJ+mcMcouWtiV65wbmyMWs+/EaVJyqGVyzhqlikULu3KdocQQw84We7ti2tZUU+G1zudRqli0sCvXGRyNYGF/Yfd6PHQ012qLXRWdFnblOufGZmlbU02lz2t3FLpaa7XFropOC7tyncHRWbpsmkogXWdrHSMTcywtR1c/WKk8yWg+dhG5E7gX8AEPGmMeStv/WeCXgPHETV9MP0apYohGYwyPz3L1hma7owAQaK0lFoOR8Vm6/bkv2q5UNlYt7CLSDTwA7AQWgGdE5KAx5ljKYbuATxhjni1MTKUyMzo1z9Jy1DEt9q6UkTFa2FWxZNIVswc4YIwZM8ZEgMeAO9KO2QX8voi8JCJ/KSLV+Q6qVCaGEmPGu2y+OCkpeQJXJwNTxZRJYQ8AQynbQ8Da5IaI1ANHgM8ANwBrgP+Sx4xKZSw5AsUpLfaqSi+tjdWc05Exqogy6WP3ALGUbQs4fybIGDMDnF//VET+DPgy8QWwM9LamvtHVL+/Ief7FpJTc4Fzs+Uj13hkkab6Sjasa8lDorjLzbW+q5GRyfm8v+5ufh8LoZxyZVLYB4CbU7Y7gcHkhoisA/YYY76cuMkCslo2JhyeIRqNrX5gGr+/gVBoOuv7FZpTc4Fzs+Ur1+uDk3Q01+bte8xHrtaGKn50apThkSk8luWYXIWgubKTay6Px7pkgziTrpj9wG4R8YtILXA78HjK/jngj0Vkg4hYwKeBb2SdVKnLFIvFGBq1f/KvdF1ttSwuRRmbmrc7iioTqxZ2Y0yQeLfKQeAo0GuMOSQi+0RklzEmBPwK8C3AEG+x/1kBMyt1QdOzS0Tml+l0SP96UlfiBKpegaqKJaNx7MaYXqA37bZbU77+GvC1/EZTKjvJKzyd12J/Y8jjtRtbbU6jyoFeeapcw2kjYpIaayupr/Hp1AKqaLSwK9cYCs9S6fPQ3Fhld5S36GzVycBU8WhhV64xFI7Q1VKXt5En+RTQycBUEWlhV64xFI445orTdJ0tdUzPLjEzl9VIYKVyooVducLC4grhqQXHFvZAW3JkjLbaVeFpYVeucG7MmSdOk7p0mTxVRFrYlSskJ9nqsnmd04tpbaqmssKjC1urotDCrlxhcDSCx7LoaK6xO8oFeSyLzpZaneVRFYUWduUKQ+FZ2ptrqPA690e6q62OoVHtilGF59zfAqWyMBSOEHBoN0xSoLWW8NQ8C4srdkdRLqeFXZW85ZUoI+Nzjh0Rk5Q8gZo80atUoWhhVyVveHyOlWiMgENHxCQlT+zqCVRVaFrYVck7vxxem7Nb7B3NNXgsS0+gqoLTwq5KXvKin64WZ7fYK7we2ptrdCy7Kjgt7KrkDYVnaW2spqrSa3eUVQXa6vTqU1VwWthVyRscjTi+Gyapq7WW4bE5lleiqx+sVI4yKuwicqeIHBOREyLy6Usc9yEReT1/8ZS6tGgsxrmxWcefOE0KtNYRjcUYHp+zO4pysVULu4h0Aw8ANwHbgb0isu0Cx3UAf0p8aTyliiI8Oc/ictTxQx2Tkp8shnRkjCqgTFrse4ADxpgxY0wEeAy44wLHPQzcn89wSq3m/HJ4Dr84KSl5glf72VUhZVLYA8BQyvYQsDb1ABH5j8ALwHP5i6bU6gZHnT2rY7qqSi+tjdUM6sgYVUCZLGbtAWIp2xZw/syPiFwD3A7sJq3gZ6q1tT6XuwHg9zfkfN9CcmoucG62XHKNRxZZU1/FhnUtBUgUl+/Xa32gkdDE/GU/rpvex2Iop1yZFPYB4OaU7U5gMGX740AX8DxQCQRE5CljTOp9LikcniEaja1+YBq/v4FQaDrr+xWaU3OBc7PlmutUcIKO5pqCfU+FeL3aGqr40clRhkemcl7Gz23vY6G5LZfHY12yQZxJV8x+YLeI+EWklnjr/PHkTmPMZ40xW4wx24FbgcFsirpSuYrFYgyNzpZM/3pSoK2OxeUo4cl5u6Mol1q1sBtjgsA9wEHgKNBrjDkkIvtEZFehAyp1MZORRWYXlktmRExSMq/OGaMKJZOuGIwxvUBv2m23XuC408AV+Qim1GoGR529atLFJD9hDIYjXL+5zeY0yo30ylNVsoKJwr62xAp7XbWPpvpKBkPaYleFoYVdlaxgKEJ9jY/Gukq7o2RtbVsdA9oVowpEC7sqWcHRGbrb6rByHFlip25/PUOjkZxGgym1Gi3sqiTFYjGCoQjd/tLqhknqToyMCU3qnDEq/7Swq5I0NrXA/OIK3f7cL26zUzJ3UPvZVQFoYVclKTg6A8RbvqUokJgMLBiasTmJciMt7KokJUfElGpXTHVlBW1N1ee/D6XySQu7KknBUIQ19ZXUVfvsjpKztf56LeyqILSwq5IUP3Famv3rSd3+Os6FZ3U1JZV3WthVyYlGYwyGIyXbv57U3VbHSjTG8JhO4avySwu7KjmhiTmWlqMl27+elJxaQLtjVL5pYVclZyAxRHBtiXfFdLXW4rGs89+PUvmihV2VnMHEUMdSWcD6YnwVXjpaanSWR5V3WthVyQmORmhrqqaq0mt3lMvW3VanY9lV3mlhVyUnGIqUfDdMUre/npHxORaXVuyOolxEC7sqKcsrUc6NzZb8idOk7rY6YsCQLm6t8iijhTZE5E7gXsAHPGiMeSht/8eA+wEv8ENgrzFmMc9ZleLc2Cwr0VjJD3VMSv6BGgjNsL7TmYstq9KzaotdRLqBB4CbgO3AXhHZlrK/DvhL4APGmKuBauCugqRVZW9gJDFHjEu6Ytqba/BVeBjQfnaVR5l0xewBDhhjxowxEeAx4I7kzsRtVxhjhhOLXbcD4wVJq8re2ZEZKryeklvn9GK8Hg9r/XWcHdbCrvInk8IeAIZStoeAtakHGGOWROQngH6gDfh+3hIqleLs8DTd/joqvO45PdTT3sDZ4WliMV10Q+VHJn3sHiD1J84C3jK5hTHmu0CriHwe+CvgzkxDtLbm/rHa73dmv6RTc4Fzs62WKxaL0T8S4R3XdBb1eyj0c129uY0nXxwEXwX+5sw/iZTq+2iXcsqVSWEfAG5O2e4EBpMbItIC7DLGJFvpjwD/kE2IcHgmpyXC/P4GQqHprO9XaE7NBc7Nlkmusal5pmcXaW+qLtr3UIzXq6U2PkPl0WPn2LHFn9F9Svl9tIPbcnk81iUbxJl8nt0P7BYRf6IP/Xbg8ZT9FvBVEVmX2P448HTWSZVaRbIfen2HM1teuVrrr8cifv5AqXxYtbAbY4LAPcBB4CjQa4w5JCL7RGSXMSYM7AW+LSIvAgL8TiFDq/J0dngaC1jb7o6hjklVlV46W2s5O+y8FqUqTRmNYzfG9AK9abfdmvL1N4Fv5jeaUm92dmSG9pZaqisz+rEtKT3t9fQFJ+2OoVzCPUMLlOudHZ5mfYc7xq+nW9/RQHhqgZm5JbujKBfQwq5KQmR+idHJeXra3VnY1yXOG/Rrd4zKAy3sqiT0u/TEaVJP4pPIGb1QSeWBFnZVEpInFntcWtgbaytpbqji7Ii22NXl08KuSsKZ4Rma6itpqqu0O0rB9LTXn/9kotTl0MKuSkL/yLRru2GS1nU0MBSe1bnZ1WXTwq4cb2l5hcHRWda5dERM0vqOeqKxmK6Bqi6bFnbleAOhCNFYjHXt7m6xJ88f6IVK6nJpYVeOdyZR6NzeYvc3VVNbVcHpc1rY1eXRwq4c71RwivoaH/41NXZHKSjLstgQaOTU4JTdUVSJ08KuHK9vcJJNgUYsy7I7SsFtCjQSHJ1hbmHZ7iiqhGlhV442O7/EUHiWjd1Ndkcpio2BJmIxtDtGXRYt7MrRTg3FuyU2BRptTlIcGxPf56lBnRBM5U4Lu3K0vuAUFrChqzwKe32Nj86WWvqC2s+ucqeFXTla3+Ak3f46aqrcN1XvxWwKNHJqcFLXQFU508KuHCsai/H64BQbA+XRv560sbuJqdn4bJZK5SKjZpCI3AncC/iAB40xD6Xtvw24n/gyea8Dv2iMGc9zVlVmhsdmicwvl03/elLy++0bnHT9EE9VGKu22EWkG3gAuAnYDuwVkW0p+xuBvwI+ZIy5HngJuK8gaVVZSY7nLpcRMUnd/joqfR5OaT+7ylEmXTF7gAPGmDFjTAR4DLgjZb8P+HRibVSIF/Z1KHWZ+ganqKny0tVaa3eUovJ6PGzobKRPL1RSOcqksAeAoZTtIWBtcsMYEzbGfANARGqA30XXP1V5cCo4ycauRjxlcGFSuo3djZwdnmZpWWd6VNnLpI/dA6SenreAaPpBItIEfAN40Rjzt9mEaG3NfQ4Qv9+ZE0M5NRc4N1tqrrmFZQZCM/z0HrE9rx3Pf8PWTr773Fkm56Ns3bDmgsfY/bpcjObKTiFyZVLYB4CbU7Y7gcHUA0SkC/gecAD49WxDhMMzRKPZD+3y+xsIhZx3hZ5Tc4Fzs6XnevXMONEYdK6ptjWvXa9XW50PgMPHhmir971lf6m8j07htlwej3XJBnEmhX0/cJ+I+IEIcDuwN7lTRLzAt4B/NMb816wTKnUBJ4LxKy83ltmImKSm+iramqo5MTDJj73N7jSq1Kxa2I0xQRG5BzgIVAIPG2MOicg+4A+AHuAGoEJEkidVnzfG3F2o0Mr9jp8eY117PfU1b22tlour1jdz5LUQ0WgMj6f8zjOo3GU0jt0Y0wv0pt12a+LL59ELnVQeLS6tcDI4xe6d3XZHsdW29c08/dIQZ0emuaKzPD+5qNxoQVaOcyI4yfJKlK3rW+yOYqut65sBOH5ar/VT2dHCrhzn+OlxvB6LLT3ldWFSuqb6Krrb6jh2Rgu7yo4WduU4x8+MsTHQSHVl+Uz8dTFb1zdzon+CpeW3jDBW6qK0sCtHicwvcXpo+nw3RLnbekUzi8tR+oI6P7vKnBZ25SivnpkgBmy7orz715OkpxnLQrtjVFa0sCtHOX5mjEqfp2zHr6erra5gQ1cjx8+M2R1FlRAt7MpRjp8ZZ0vPGiq8+qOZtHV9M68PTusC1ypj+tujHGN8eoGh8CzbynyYY7pt65uJxmKY/gm7o6gSoYVdOUayu0FPnL7Z5rVN+Co8Op5dZUwLu3KMF0+Gaaz10dOR+2yfbuSr8LKlZw0v9o3qOqgqI1rYlSMsLK3wUl+YHVv8ZTn/+mp2bvEzMj7HQChidxRVArSwK0c4YkZYWFphp/jtjuJIO7b4sYDDZsTuKKoEaGFXjvDMS4PUVVdw1TrtX7+QprpKruxZw2ETsjuKKgFa2JXtlleiHHrlHNs3t+kwx0vYKX6CoxGGwtodoy5Nf4uU7Y6fGScyv8xOabc7iqPt3BLvptJWu1qNFnZlu8NmhJoqL1dv0G6YS2lprGZjoFELu1pVRoVdRO4UkWMickJEPn2J474iInflLZ1yvWg0xguvjXLj1k58FV674zjeTvFzZnia4bFZu6MoB1u1sItIN/AAcBOwHdgrItvSjgmIyLeAOy7wEEpd1Gv9E8zMLfHO67rsjlISkt0xz748uMqRqpxl0mLfAxwwxowZYyLAY7y1gH8S+CfgH/OcT7ncoePD+Co87Lyqw+4oJaG9uZae9nqePBK0O4pysEwKewAYStkeAtamHmCM+RNjzMP5DKbcb25hmWePDXPjVe3UVOmiGpm66douTvRPcPrclN1RlENl8tvkAVKvY7aAvC7n0tqa+yXkfn9DHpPkj1NzgXOyfedfX2dhcYXbd28BnJMrndNyfeR9V/L1p07x7LERbrzWeQt+O+31SiqnXJkU9gHg5pTtTiCvHXzh8AzRaPZzYPj9DYRC0/mMkhdOzQXOyRaLxfjWk32s72xgTXX8pKkTcqVzyuuV7pYb1nLw+X4+8q711FX77I5znlNfL7fl8nisSzaIM+mK2Q/sFhG/iNQCtwOPZ51EqRSv9U8QHI3w/h3dWDo3TNZufdcGFpej/OvL5+yOohxo1cJujAkC9wAHgaNArzHmkIjsE5FdhQ6o3OngkSC1VRW8bZueNM3Fxu4mNnU3cvCFAaI646NKk9EZK2NML9CbdtutFzjurvzEUm42ObPAYRNi9861VPl07Hqu3r9jLV/89jGOnxnnal0jVqXQK09V0T3x4iAr0Ri37HDeib9SsusqP/U1Pg4cHrA7inIYLeyqqGbmlvj+oX6u29RKZ0ut3XFKmq/Cy/t2dHPkxCivD+nQR/UGLeyqqPY9e4a5hWXueO8mu6O4wo+/fR31NT4ePXhSV1dS52lhV0UzOjHH/sP9vPvaLta26/J3+VBTVcFtN23g1bMTvHwqbHcc5RBa2FXRfOOpU1iWxUdv3mB3FFd57/YA7c01PHqwL6frQZT7aGFXRXHm3DTPvjLMB2/soaWx2u44rlLh9XDHezcRHI3wry8PrX4H5Xpa2FXBrUSjPPIvr1Ff4+Mn3r7e7jiutFP8bAw08vUnTzE1u2h3HGUzLeyq4L751OucDE5y5weupLZaJ/sqBMuy+Hc/JkTml3n428f0oqUyp4VdFdSPToX5zrNneM/1Ad6xrdPuOK62rqOBn91zJT86Ncbj/3bW7jjKRlrYVcGMTy/wxW8fo9tfx8/uudLuOGXhlu0Bdl3VztefOMWJgQm74yibaGFXBTG3sMz//sbLLCyt8O9vu0anDigSy7K468evorWpiv/zT68wMq5L6JUjLewq72bmlviTvzvC6XPT/PKHtxFoq7M7Ulmpra7gP/zUdSwtR/lvj7xAcDRidyRVZFrYVV5NRhb5494XGAhF+PRPXctOabc7Ulnqaa/nt+/cATH4o0de4Oyw8+YiV4WjhV3lzatnxnngK88zMjHHf/r4dWzf3GZ3pLK21l/P737yBip9Hv6o9wWefmlIpx0oE1rY1WWbW1jm/33P8Md/dwSPZfGZn92h08g6REdLLb/3yZ2s9dfz5X3HefDRlxibmrc7liowHVSscjYzt8QTR4P84B4PcUEAAAhCSURBVPAAkzOLfPDGHj72no16otRhWpuq+Z1P3sCBwwM89kQf9zz8b7z3+gB7dq6lbU2N3fFUAWRU2EXkTuBewAc8aIx5KG3/duBhoBF4EviUMWY5z1mVAywurfDq2QmOnAjx7CvnWFyKsnV9M7/60WvZvLbJ7njqIjyWxZ5dPVy3uY1vPHmK/c8P8C/P93PDlX52XdXONRtbHLV2qro8qxZ2EekGHgB2AgvAMyJy0BhzLOWwrwJ3G2OeE5EvAb8M/FUhAqviWVhaITw5z0Bohv6RGc4MT/Pa2QkWl6NUVni48ap2PnBjD+s6nLn6u3qr9jU1/MpHrubjt2ziB4cHeOqlIQ6/FsKyYFN3Exs6G+lpr6envR7/mmpqqip0TdoSlEmLfQ9wwBgzBiAijwF3AH+Y2F4P1Bhjnksc/zfA/RS4sEdjMZ4/PszI6EwhnyYnjYNTTOXQj5ntia30w2PESPwHsfhrFIvFHzcaixGNQV1tJROTcyxHYywvR1lcjrK4tMLC0gqz88tE5peYmVtifHqByPwbH7q8HovO1lrec32A6za1sqVnDZXa5VKyWhqr+fj7NnP7ezdxamiKl/vCvHJ6jCeOBllcjp4/rqrSS0tDFQ01PmqrfdTVVFDtq6DS56HS56XCa1Hh9VDh9eDxWDQ1VhOZWcCyLCyL+P9YJP5L/PvGH4ps/2bk+kcm19/JQvJYcEtTYbrCMinsASB1yrgh4G2r7F+bTYjW1uzn5j7ZP8H9Dz+3+oHqkiq8HqoqvVT5vNTX+qiv8dHd3sC1V/rxr6mhbU0N6zoaWNfZgK+isIXc73dmy9/tuTo6Gnnn9viv7Eo0xtDoDGeGphkZn2V0Yo7QxBzTs4tMRBYZGI2wsLjMwuLKm/4AqNxYFV5+7B1X5P1xMynsHhKNwGQWIJrF/lWFwzNZzyPdVO3l4Xs+wNC5yazuVwzNLXWMj+V4UUgGLZLUIy50uGXF20SWJ/GvFe9j9Xgs/G0NTE5E8Ho9VHgtvJ7MBkZNFPgKRr+/gVDIeWOtyzFXlQVbAg1sCVz6D0c0GmN5JcpKNMbSSpRoNEZzcx2h0en4J0USn0JTv06RupnRb/9lDNW8rN/JAvF4LK6VjpzeR4/HumSDOJPCPgDcnLLdCQym7e+6xP6C6WipxbOyUoynyorf30Ct15n9kmsaqlia12ld1eXzeCwqPfFPcckOhbY1NcSWnDduwqm/k4U6f5FJc20/sFtE/CJSC9wOPJ7caYw5A8yLyLsTN/088N28J1VKKZWRVQu7MSYI3AMcBI4CvcaYQyKyT0R2JQ77JPAXIvIqUA/8z0IFVkopdWkZjWM3xvQCvWm33Zry9Yu8+YSqUkopm+iUAkop5TJa2JVSymW0sCullMvYPQmYF+LDpnJ1OfctJKfmAudm01zZ0VzZcVOulPtc8KpBy+b5mW8CnrIzgFJKlbCbgafTb7S7sFcBNxKfhsB5VxoppZQzeYlfGPpD4pMzvondhV0ppVSe6clTpZRyGS3sSinlMlrYlVLKZbSwK6WUy2hhV0opl9HCrpRSLqOFXSmlXMbuKQXyRkR2AM8ZY6rszgIgIjcDDwKVwOvALxhjxu1NBYkFUf6CeK4w8EuJxVIcQUQ+B6wYY+6zOcedwL2AD3jQGPOQnXlSiUgj8AzwYWPMaZvjACAinwV+OrH5HWPMb9uZJ0lE/hC4g/jqe18yxvy5zZHeRET+FGgzxtyVz8d1RYs9sbLT/yJerJzi/wI/b4y5FjgGfMbmPEmPAHcbY7YnvnbEoigi0iQiXwJ+0wFZuoEHiE95sR3YKyLb7E0VJyJvJ34J+Ra7sySJyB7gg8AO4q/XThH5mL2pQETeC7wfuA7YBfyaiIi9qd4gIruBXyjEY7uisAN/Rrx17CRbjTHHRMQHdANOaK1XAfcaY15K3PQSsM7GSKluA04Qfy/ttgc4YIwZM8ZEgMeIt/qc4JeBT1OkdYUzNAT8pjFm0RizBBzHAT9XxpgngPcZY5aBduI9FI5Y0VpEWog3Hj5fiMcv+a4YEfkIUGuMecxBf4wxxiyJyLXE14xdAn7f5kgYYxaArwKIiAe4D/imnZmSjDFfARCR+2yOAhAgXqyShnDICmHGmLsBHPaz/kryaxG5kniXzLsvfo/iSfwe3g/8FvAoELQ5UtIXiC852lOIBy+Zwi4iHyfeN5zqVaCReAvLFhfLZYzZY4x5GegQkV8B/gF4lxNyiUgl8LfE3/+CtBhyyVXMHKvwEO+TTbKAqE1ZSoaIXA18B/iMMeaE3XmSjDGfFZE/Ar5F/BPPX9uZR0TuBvqNMT8QkbsK8RwlU9iNMY8S/4t7XuIF+j3gyWQLRkSOAjcbY6ZtzFUtIh81xiRbw1+lyF0MF8qVyFYP/DPxE6e3JT46257LYQaIT4ea1Imzuj4cJ3FS/mvAfzbG/L3deQBE5Cqg2hhz1BgzKyJfJ97fbrefAboStaoFqBeRvzDG/Hq+nqBkCvuFGGMeBh5ObotILHFS0G5LwEMi0m+MOUz8o+lb5ky2yVeBk8CnjDHaCr2w/cB9IuIn3id7O7DX3kjOJSI9xLv0fsYYc8DuPCk2AveLyE3EP4HdBnzZ3khgjPlA8utEi/2WfBZ1cM/JU0cxxqwQ/6v814m/yncAd9ub6vyQ0NuI93++ICJHRWSfzbEcxxgTJN7/eRA4CvQaYw7Zm8rRfguoBv488TN1VEQ+ZXcoY8w+4l1DR4DDwDNO+TRRaDofu1JKuYy22JVSymW0sCullMtoYVdKKZfRwq6UUi6jhV0ppVxGC7tSSrmMFnallHIZLexKKeUy/x8SG1hRbP0uUAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_dist(create_gaussian_pdf(0, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXhb13nn8e8FwA3cCYIkSJESLcuvLcuybCveIjlurEwSp63bsbLUbadOJ1XzTJ52pk0609ZuY7d1nplucdt48qR10jZNOW3qNJ6kld2MLaXxEkfeZDtydCxZpMQFpEhwB3cC8wcAGaYlEqQA3IuL9/OHHl7cC+AnAHx5cM6551rxeByllFLu4bE7gFJKqezSwq6UUi6jhV0ppVxGC7tSSrmMFnallHIZn83PXwa8CwgDyzZnUUqpQuEFQsDzwPzKnXYX9ncBT9mcQSmlCtVe4OmVN9pd2MMAY2NRYrH1z6cPBKqIRKazHupiOTUXODeb5lofzbU+bsvl8VjU11dCsoauZHdhXwaIxeIbKuyp+zqRU3OBc7NprvXRXOvj0lzn7cLWwVOllHIZLexKKeUyWtiVUsplMupjF5G7gHuBEuBBY8xDK/ZfC3wJKAV6gZ8zxoxnOatSSqkMrNliF5E24AFgD7ALOCAi21cc9mfA7xpjrgYM8JlsB1VKKZWZTLpi9gGHjDGjxpgo8Aiwf8UxXqAm+bMfmM1eRKWUUuuRSVdMK2+fKxkGrl9xzK8D3xGRB4EocMN6QgQCVes5/G2CweoN3zeXnJoLnJtNc63tZO84X/znVyjxefnd/3wD/vISuyO9g5Ner3TFlCuTwu4B0idaWkAstSEiFcCXgX3GmCMi8uvAV4EPZRoiEpne0FzOYLCa4eGpdd8v15yaC5ybTXOtbm5hiW9+r5snXuyluqKE6bklfueLz/BrH91FWYnX7njnOOX1WsltuTwea9UGcSZdMX0k1iRIaQEG0rZ3ALPGmCPJ7S8Bt64vplLqQuLxOJ//+is88UIvt+5q43MHbuIzd13Hif4JvvCNV1lc0mWW1NtlUtifAG4TkaCI+IE7gcfT9p8E2kVEktt3kFiYRimVBSf6JjjRN8HP7NvGz79f8Jf72HtNGx//4BUc6xnjKweP2x1ROcyahd0Y0w/cAxwGjgJdyS6XgyKy2xgzBtwNfF1EXgV+Efh4DjMrVVT+7cgZKst97L269W2379kZ4vYbN/OD14cYGdf5CuotGc1jN8Z0AV0rbrs97efHgMeyG00pNTQ2w9ETI3zo5s3n7Uu/9ZpWHnvuNM/8cJA79nTakFA5kZ55qpSDPfF8Hx6PxXuv3XTe/Y21FVyxpZ6nXw0TiztzkSuVf1rYlXKo6NwiT702wI3bm6mrKrvgcXt2hohMzmFOj+UxnXIyLexKOdS/Hx1gYTHG+97Vvupx124LUlHm4+nXzrs0typCWtiVcqB4PM53X+7nis31dDSvfgJLaYmXG7Y384IZZmZuKU8JlZNpYVfKgYbGZhmZmGP35U0ZHb93Z4jFpRhHjg/lOJkqBFrYlXKgY92jAFy5pT6j47e0VNPWWMnTr2p3jNLCrpQjHesepbG2nKZ6f0bHW5bF9Vc0cWpgkunZxRynU06nhV0ph1lajnH8zBg7OhvWdb/LNyda9+aMXgqh2GlhV8phusOTzC0ss33L+gp7Z6iGUp8Hc0anPRY7LexKOcyx7lEsC67IsH89xef1sLWtluPaYi96WtiVcphjPaN0hmqo3MBa65d31NE/PK397EVOC7tSDjIzt0T3wNS6u2FSpKOeOPBGr7bai5kWdqUc5PiZMWLxeMbTHFdK9bMf1372oqaFXSkHOdY9Slmpl61ttRu6f4kv0c/+hvazFzUt7Eo5yOs9o1zeXofPu/FfTemoo/es9rMXs4zWYxeRu4B7gRLgQWPMQ2n7dgF/k3Z4EBgzxuzIYk6lXG9yZoGhsVlu2dW69sGruLyjnkfp5kTvONdcFsxSOlVI1mwWiEgb8ACwB9gFHBCR7an9xpijxphdxphdwM3AGPDJHOVVyrVODyYuatzZUnNRj9MZqqHE58HoAGrRyuT73j7gkDFm1BgTBR4B9l/g2N8C/t0Y83S2AipVLLrDk1jA5pbVV3NcS4nPw9bWGh1ALWKZFPZWIH1loTDwjsu5iEgtcAC4PzvRlCouPeEpWgJ+Ksoy6iFd1WXtdfQOTTM7r8v4FqNMPkEeIP2aWxYQO89xPwc8aow5u94QgUDVeu9yTjB4ca2bXHFqLnButmLPdebsFFdvC2b8fKsdt+vyZr71TA9TCzE6NuX3dS3293G9cpErk8LeB+xN224BBs5z3E8Bn9tIiEhkmlhs/ddrDAarGR6e2shT5pRTc4FzsxV7rrGpeUYn52mpr8jo+dbKVVeR+NV+xQzRVF2atZxrKfb3cb02msvjsVZtEGdS2J8A7hORIBAF7iTR5XKOiFjAdcD3151QKUVPeBJIDHxmQ11VGbWVpZwZdF4xU7m3Zh+7MaYfuAc4DBwFuowxR0TkoIjsTh4WBBaMMXO5i6qUe3UPTuKxLDqaNt4tudLmlmpOD2lhL0YZjdIYY7qArhW33Z7281kSXTRKqQ3oCU/RFqyktMSbtcfsaK7mh6dGWVhczurjKufTM0+Vslk8Hqc7PElnKLuDaJubq4nF4/QNR7P6uMr5tLArZbORiTmic0tsucgTk1ba3JLo1tHumOKjhV0pm3VneeA0JVBTTmW5j9ODk1l9XOV8WtiVslnP4BQ+r0VbsDKrj2tZVmIAdXA6q4+rnE8Lu1I26wlP0t5UfVErOl7I5uZq+oanWVo+3zmFyq20sCtlo1g8Ts/gFFuyPHCasrmlmuVYnH4dQC0qWtiVstHI+CxzC8tsbs5RYU8+rg6gFhct7ErZKDUVMdv96ynB+goqyrxa2IuMFnalbNQ/nBjYbGvMTWFPnM1arUsLFBkt7ErZqG84SrCunPLSi1+q90I2t1TTe3aa5ZgOoBYLLexK2ahveJpNweytD3M+7U1VLCzFODs2m9PnUc6hhV0pmywuxRganc1Z/3pK6g+HzowpHlrYlbJJOBIlFo/nvMXeEvBjAf0jWtiLhRZ2pWzSlxo4zXFhLyvxEqyvODdQq9xPC7tSNukbjuLzWjTXV+T8udoaK7XFXkS0sCtlk/7hKKFAZU6WElipLVjJ0Ogsi0s6M6YYZDTHSkTuAu4FSoAHjTEPrdgvwJeAemAQ+JgxZizLWZVylb7haS7vqMvLc7U1VhGLxxkcnaE9i1dpUs60ZlNBRNqAB4A9wC7ggIhsT9tvAd8C/qcx5mrgZeA3cxNXKXeIzi0yNjWf8/71lNTMG+1nLw6ZfAfcBxwyxowaY6LAI8D+tP3XAlFjzOPJ7c8BD6GUuqDU1MNNOZ7qmNLS4MfrsbSfvUhk0hXTCoTTtsPA9WnblwKDIvJl4BrgR8CvZC2hUi6UmhGT66mOKT6vh+YGv85lLxKZFHYPEE/btoD0ERgfcCtwizHmBRH5feBPgbszDREIbPzDHQzmZlW8i+XUXODcbMWUKzK1QGW5j8suacSyrA09xnpzdbbV8mbfeM5f52J6H7MhF7kyKex9wN607RZgIG17EDhhjHkhuf1/SHTXZCwSmSYWi6994ArBYDXDw85b3MipucC52Yot14neMVobKxkZ2Vif90ZyBavLeDYyQ1//OGWl3g09by5y5YPbcnk81qoN4kz62J8AbhORoIj4gTuBx9P2PwsEReTq5PZPAC+uO6lSRSIeT1z4Il/dMCltwUriwEBEu2Pcbs3CbozpB+4BDgNHgS5jzBEROSgiu40xs8BPA38lIseA9wKfzmVopQrZ2NQ8s/NLOV8jZqXWxtTMGC3sbpfRPHZjTBfQteK229N+/gFvH1BVSl1AqsXcGshvYW+qr8Dn9TCgM2NcT888VSrPwiMzwFst6Hzxejy0Bvz0bbBfXxUOLexK5dlAJEpluY9qf0nen7s1WKldMUVAC7tSeRYeidLaWLnhaY4Xo62xkrGpeWbmlvL+3Cp/tLArlWcDkRlCee5fT2lrrEpm0Fa7m2lhVyqPJmcWmJ5dzHv/ekqo0Q8kvjUo99LCrlQepQpqa8Bvy/MHaxMzY8KRGVueX+WHFnal8mggWVDt6orxeCxaGiq0K8bltLArlUcDI1HKSr001JTZliEUqCSshd3VtLArlUfhSJRQg9+WGTEpoYCfkfE5FhaXbcugcksLu1J5FI7M2DZwmtLamFgzZnBU+9ndSgu7UnkyM7fE2NQ8IZsGTlNS/fs6gOpeWtiVypPwaHJGjM0t9paGCiwL7Wd3MS3sSuXJwIg9i3+tVOLzEqytODdDR7mPFnal8iQcmcHn9dBYV253FEIBv7bYXUwLu1J5MjASpaWhAq/H/l+7UGMlQ6MzLMdiax+sCo79nzClikQ4ErW9fz0lFPCztBxnZHzO7igqBzK60IaI3AXcC5QADxpjHlqx/7PALwJjyZv+auUxShWzhcVlRsbnuHlHyO4owFsDuAORKM0N9s7SUdm3ZmEXkTbgAeA6YB54VkQOG2NeTztsN/AxY8z3cxNTqcI2ODpDHGyf6pgSanhryuM122wOo7Iuk66YfcAhY8yoMSYKPALsX3HMbuC3ReRVEfmCiNg/OqSUg9h1ObwL8Zf7qKsq1VUeXSqTwt4KhNO2w8Cm1IaIVAEvA78BXAvUAb+TxYxKFbzByAyWhaO6PUKBSp3y6FKZ9LF7gHjatgWcG0o3xkwD5y5sLSJ/AnwFuCfTEIFAVaaHvkMwWL3h++aSU3OBc7O5OVdkeoGWQCWtodosJEq42FxbN9Xx5Au9NDZWZXXtGje/j7mQi1yZFPY+YG/adgswkNoQkQ5gnzHmK8mbLGBxPSEikWlisfjaB64QDFYzPDy17vvlmlNzgXOzuT1Xz8AETbXlWfs/ZiNXXWUJs/NLvHFqhIaa7PSeuv19zLaN5vJ4rFUbxJl0xTwB3CYiQRHxA3cCj6ftnwX+UEQ6RcQCPgV8c91JlXKp5ViModEZQg6Z6piia8a415qF3RjTT6Jb5TBwFOgyxhwRkYMistsYMwz8MvBtwJBosf9JDjMrVVBGJuZYWo47ZkZMSuoqTnoGqvtkNI/dGNMFdK247fa0n78BfCO70ZRyh/BIokXslBkxKTWVpfjLfNpidyE981SpHEu1iJ3WYrcsi1CjrhnjRlrYlcqxcGSG2spS/OUldkd5h1CDTnl0Iy3sSuVYOBJ1XGs9JdToZzK6QHRuXRPZlMNpYVcqh+LxOAMR582ISTk3M2ZEW+1uooVdqRyaiC4wO79EyEFnnKZLzYwZ0H52V9HCrlQOpWacOLXF3lhbgc/r0QFUl9HCrlQOhR22+NdKHo9FS4Nfpzy6jBZ2pXIoPDJDeamXuqpSu6NcUKtOeXQdLexK5dBAckZMNhfZyrZQoJKR8TkWFpftjqKyRAu7Ujk0ODpzbuaJU4UCfuIksip30MKuVI7Mzi8xNjXv2DnsKboYmPtoYVcqR87NiHF4i72loQLL0sXA3EQLu1I5cm5GjEOnOqaU+LwEayt0aQEX0cKuVI70j0TxeS2Cdc6/BHAooDNj3EQLu1I5MjASpaXBj9fj/F+zUGMlQ6MzLMdiax+sHM/5nzilCtTASNTx3TApoYCfpeU4I+NzdkdRWZBRYReRu0TkdRE5ISKfWuW4D4lId/biKVWY5heWiUzMFUxhT50Zq2vGuMOahV1E2oAHgD3ALuCAiGw/z3HNwB+TuDSeUkVtcHSGOM5dSmCl1MydgREt7G6QSYt9H3DIGDNqjIkCjwD7z3Pcw8D92QynVKFKFchCabH7y33UV5dpYXeJTK552gqE07bDwPXpB4jIrwIvAc9tJEQgULWRuwEQDFZv+L655NRc4Nxsbso1PtuL12Nx5WVN+Ly5GcrK9uvV2VrL0PjcRT+um97HfMhFrkwKuweIp21bwLmhcxHZAdwJ3AZs2kiISGSaWCy+9oErBIPVDA9PbeQpc8qpucC52dyW6+SZMZob/IyN5qYFnIvXq7GmjNfeHGFoaBKPZ2M9qm57H3Nto7k8HmvVBnEmTYk+IJS23QIMpG1/OLn/BeAg0CoiT607qVIu0j8SPXcRi0LR2ljJ4lKM4YlZu6Ooi5RJYX8CuE1EgiLiJ9E6fzy10xjzWWPMZcaYXcDtwIAxZm9u4irlfAuLywyPzxZM/3pKW6MOoLrFmoXdGNMP3AMcBo4CXcaYIyJyUER25zqgUoVmcHSGeLxwBk5TWrWwu0YmfewYY7qArhW33X6e43qALdkIplShGnD4VZMupKLMR0NNGf1a2AuennmqVJYNjMzgsSyaHXoB69W0NlYyMKyFvdBpYVcqy8IjUZrqKyjxFd6vV1tjJeHRmQ3NUlPOUXifPKUcbiBSOGvErNQa0JkxbqCFXaksWlyKMTQ6S2tj4XXDALQGkwOo2h1T0LSwK5VFQ2MzxOLxghs4TUnl1gHUwqaFXaksKrQ1YlaqKPMRqNE1YwqdFnalsqhveBqPZTn+OqeraW2s0sJe4LSwK5VFfWejtAT8BTkjJqW10a8zYwpc4X76lHKgvuFpNgULt7UOaWvGjOvMmEKlhV2pLJmdX2JkYo5NwY0vQ+0EbY2J/DqAWri0sCuVJf3JKYKFXthbG/1YJL59qMKkhV2pLOlNFsJNTYXdFVNe6iNYX0HvWS3shUoLu1JZ0jc8TUWZl0BNud1RLlp7U5UW9gKmhV2pLOk7O01bsArLKvzrubc3VTE8NsvcwpLdUdQGaGFXKgvi8Th9w1HaC7x/PaW9qYo4b40bqMKS0XrsInIXcC9QAjxojHloxf6fBu4HvMDzwAFjzEKWsyrlWKOT88zOLxX8VMeU1B+o3rPTbG2rtTmNWq81W+wi0gY8AOwBdgEHRGR72v5K4AvA+4wxVwLlwN05SauUQ/WdGzh1R4s9UFtORZlP+9kLVCZdMfuAQ8aYUWNMFHgE2J/ambxtizFmKHlN1CZgLCdplXKoVGFPzQEvdJZl0R6sPDfTRxWWTAp7KxBO2w4Dm9IPMMYsisgHgV6gEfhO1hIqVQD6hqMEasrxl2fUu1kQ2puq6Ts7TSyuSwsUmkw+hR4g/Z21gNjKg4wxjwEBEfkc8EXgrkxDBAIbb+UEg9Ubvm8uOTUXODdbIecKj85wyabavP4fcv1cV2wN8ORLfcS9XoLrWNSskN9HO+QiVyaFvQ/Ym7bdAgykNkSkAdhtjEm10v8e+Mf1hIhEpje04FAwWM3w8NS675drTs0Fzs1WyLkWl2L0n53mqs6GvP0f8vF61VWUAHD0R0NcJ8GM7lPI76MdNprL47FWbRBn0hXzBHCbiASTfeh3Ao+n7beAr4lIR3L7w8DT606qVIEKR6Isx+K0u2TgNKUtWIll6dIChWjNwm6M6QfuAQ4DR4EuY8wRETkoIruNMRHgAPAvIvIKIMD/yGVopZzk3MCpS+awp5SVeGmu9+vMmAKU0UiPMaYL6Fpx2+1pPz8KPJrdaEoVhtOD05T6PLQ0VNgdJes2NVVxenDS7hhqnfTMU6Uu0unBSdqbq/B63Pfr1N5UxfD4HLPzurRAIXHfJ1GpPIrF4pwemmZLS43dUXIiNW6gSwsUFi3sSl2EwdEZ5heX2dLizKl0F6sjWdjPnHXejBJ1YVrYlboIpwcTBW+zSwt7fXUZVRUl9AxqYS8kWtiVugg9g1OU+jyEAn67o+SEZVl0hmroCesAaiHRwq7URegZnKSjudqVA6cpnaFq+keizC8s2x1FZci9n0alciwWi3NmaNq13TApnaEa4nE4PaTdMYVCC7tSG+T2gdOUzlBixk+3dscUDC3sSm1QT/LEHbcX9prKUgI1ZVrYC4gWdqU2qGdwitISD6F1rHxYqLaEaugJa1dModDCrtQGnR6coqOpGo+n8C9evZbOUA1nx2eZnl20O4rKgBZ2pTagWAZOUzqT/88eXTemIGhhV2oDwkUycJqyuSU1gKrdMYVAC7tSG3C6SAZOU/zlPloa/HQPaIu9EGhhV2oD3hyYpKzUWxQDpymdoWq6tSumIGhhV2oDTvROcGlbbVEMnKZ0hmqYmF5gbGre7ihqDRldaENE7gLuBUqAB40xD63YfwdwP4nL5HUDHzfGjGU5q1KOMDO3SP/wNLsv77Q7Sl6ln6hUX53ZNVCVPdZssYtIG/AAsAfYBRwQke1p+2uALwIfMsZcDbwK3JeTtEo5wMn+SeLAtrZau6PkVUdzFV6PxSntZ3e8TLpi9gGHjDGjxpgo8AiwP21/CfCp5LVRIVHYO1DKpU70jeOxLC5pLa7CXuLzsqWlmjf6xu2OotaQSWFvBcJp22FgU2rDGBMxxnwTQEQqgN9Er3+qXOxE3wSbW6ooK/XaHSXvLuuoo3tgkvlFXenRyTLpY/cA8bRtC4itPEhEaoFvAq8YY/52PSECgY1f3T0YdOZ0M6fmAudmK4Rci0vL9IQn+eDNnbbnteP5r9/RymPPnSESXeTqbXXnPcbu1+VCiilXJoW9D9ibtt0CDKQfICIh4N+AQ8CvrTdEJDJNLBZf+8AVgsFqhoedd8KEU3OBc7MVSq6T/RMsLMXYFKiwNa9dr1dTdSmWBUdeG6C1rtwxudbitlwej7VqgziTwv4EcJ+IBIEocCdwILVTRLzAt4GvG2P+YN0JlSogJ5L9y5duOn9r1e0qynx0NFfzRq/2szvZmoXdGNMvIvcAh4FS4GFjzBEROQj8LtAOXAv4RCQ1qPqCMeYTuQqtlF1O9k3QXF9BbWWp3VFsI+11HH65n8WlGCU+PRXGiTKax26M6QK6Vtx2e/LHF9ATnVQRiMfjnOibYNeljXZHsZW01/Gd53vpDk9yWXtxfnNxOi3ISmVocHSG6dlFtm0qrmmOK21LFnOj3TGOpYVdqQyd6JsA3ipsxaqqooRNwUreOKMnlzuVFnalMvR6zyg1laU011fYHcV20l7Pyf5JlpbfMfNZOYAWdqUyEIvFOdY9yo7OBiyreBb+uhDpqGN+cZnTQ86bQqi0sCuVke7wJNG5Ja66JGB3FEdIdUe9cUb72Z1IC7tSGXjtVATLgis7G+yO4gi1laW0NVbyw+5Ru6Oo89DCrlQGXjs1yiWhGqoqSuyO4hg7Lw3wRu84s/NLdkdRK2hhV2oNUzML9IQntRtmhau3NrIci2ur3YG0sCu1hmPdo8SBHVrY3+bStloqy30cPTFidxS1ghZ2pdbw2qlRqipK2BJy5uqAdvF4LHZuDfDaqciGFvFTuaOFXalVxGJxftgdYUdnAx6d5vgOV1/ayPTsIm8OTNgdRaXRwq7UKt7sH2dqZlH71y9gR2cAr8filZMRu6OoNFrYlVrFi8fPAjrN8UL85T4ua6/jlZPaz+4kWtiVWsUzrwywtbWGmiJepnctV28N0D8SZXh81u4oKkkLu1IX0Dc8TU94khu2N9sdxdGu3pZYxlhb7c6hhV2pC3ju2BAej8X1V2hhX01zvZ9QwM8LZtjuKCopo8IuIneJyOsickJEPrXKcV8Vkbuzlk4pm8TicX7w+iC7LgtqN0wGbryyhTd6xxmMRO2OosigsItIG/AAsAfYBRwQke0rjmkVkW8D+8/zEEoVnJN9E0Qm57n12k12RykI797RggU8+Xyv3VEUmbXY9wGHjDGjxpgo8AjvLOA/C/xf4OtZzqeULZ57fYjSEg837gjZHaUgNNSUs72zgUMvnCEW15OV7JZJYW8FwmnbYeBtzRhjzB8ZYx7OZjCl7LK0HOP5Hw1xzbYgFWUZXRZYAXuuCnF2bBZzWq+sZLdMPrUeIP1PsAVk9bIpgUDVhu8bDDrzNG+n5gLnZnNKriPHBonOLfH+m7YAzsm1ktNyve9mP1/7juH5EyPc8q7Ndsd5B6e9Xim5yJVJYe8D9qZttwAD2QwRiUxvaK2JYLCa4WHnXcHFqbnAudmclOvgM6cS1/VsSFwCzym50jnp9Up3yzWbePL5M3z4lksc9W3Hqa/XRnN5PNaqDeJMumKeAG4TkaCI+IE7gcfXnUSpAnB2bIaX3hhm784QPq/OBl6vfdd3sLAU4/nkGbvKHmt+co0x/cA9wGHgKNBljDkiIgdFZHeuAyqVT//2fC8ey2Lf7na7oxSkbe11tDZWcvilfuI6iGqbjL4rGWO6gK4Vt91+nuPuzk4spfJvcmaBZ14Nc9OOFuqry+yOU5Asy+L972rnrx87zmunRtm5VRdPs4N+11Qq6dCLfSwsxfjA9R12RyloN+1oIVBTzree6dZWu020sCsFzC8uc+ilfnZd2khrY6XdcQqaz+vhQzdv5tTAJMd69LJ5dtDCrhTw9KthpmcX+cAN2lrPhnfvCNFQU8a3nu7RVrsNtLCrojc7v8S/fr+HrW01bNtUa3ccVyjxebj9xs2c7J/guJ6wlHda2FXR++ZTp5iYXuBnbrsMSy9/lzV7d4aoqyrln586pcsM5JkWdlXUTg9O8eSLfbznmjYuaa2xO46rlPi8/MdbtvJm/ySHX+q3O05R0cKuilYsHufvvmOorijhzvdcYnccV3r3VS3suKSBf/ruSc7qFZbyRgu7KlrfOzrAqYFJPvLeS6ksL7E7jitZlsXdH7gcr8fibw7+SLtk8kQLuypKZ4am+MdDJ7m8o46brmyxO46rNdSU89H3buP4mXG++7J2yeSDFnZVdMan5/mzR17FX+7jl37iSh0wzYO9O0Ps6GzgH548iTmjs2RyTQu7Kirzi8v8xTdeZWZuif+6f6cuHZAnlmVx4CevJFhXzp9/41VODzpvpUU30cKuisbi0jJ/+a1j9ISnOPCT2+lodub63G5VVVHCpz+6C3+Zj89//ShDozN2R3ItLeyqKExMz/OHXS/z8okRPrZvG9dsC9odqSg11JTz6x/dRSwOf/QPL9MdnrQ7kitpYVeud2Zoit//6gv0Dk/zX35qB+/TJXltFQpU8pmP7cICPvd3L/Lki3267ECWaWFXrjU7v8Qj332TP/jqi8Tj8Fs/ex27L2+yO5YCOpqr+ezHr+fKzgb+/v+9wf9+9IeM6Dz3rHHOtauUypL5hWW+f2yQR5/uZjK6wE1Xtk8vJDsAAAgySURBVPCRH9tKbZUOlDpJVUUJv7p/J4//4AyPPtXN0RMj3HJ1Kz9+8xYd1L5IGRV2EbkLuBcoAR40xjy0Yv8u4GGgBvge8EljzFKWsyp1QUvLMd7sn+DZHw7y/PGzzC0ss7Wthl+9c6cuFeBgHsvi9hs3c+P2Zv7l+6f53isDfO+VAa7sbOD6K5q4ZlvQUddOLRRrvmIi0gY8AFwHzAPPishhY8zraYd9DfiEMeY5Efky8EvAF3MRWKnFpWWGxmYZjMwwMBLlRN84J/snmV9cpqzEy+7Lg+y5KsRl7XU6R71ANNSU85/eL3zwhg4Ov9TP88eHePjNCF7PcTqaq7gkVMslrTW0BPwE6yqoLPfpe7uKTP4U7gMOGWNGAUTkEWA/8HvJ7c1AhTHmueTxfwPcT44Leywe54UfDXF2ZDqXT7MhNQOTTE7O2R3jvC4m23oHuFKHxzn3A/F4YjseT7yH8TjEYnEq/KVMTM6xvBxjcSnG4nKM+cVl5uaXmV1YYmZuicmZBSamF5ieXTz3HBbQFqxKFPKOOq66pIHyUm3hFapgXQUfee+l7P+xrZwamOToiRFODUzw1GsDPPlS37njKsp81FSWUu0vobqihIoyH2WlXspLvZR4PZT4PPi8HrweC6/Hora2guj0PJZlYVmJbwok/y5YyX+s5A3r/Xux0T8wHgtura3Y0H3XkslvQCsQTtsOA9evsX/TekIEAlXrORyAk73j3P/wc2sfqAqSz2tR4vNSWe6joryEynIf7c3V7Ly0nPqaclobK2lrqqItWJXzr+rBoDPnu7s9V3NTDTftSpSS5eUYfWenCUeiDEaiDEVmGJueZ3J6gdGpeWZHoszOLzE7v8zSciwrz58Pls/L+2/ckvXHzeQ3wgOkN9UsILaO/WuKRKaJxdbXGqwt9/LwPe8jPDixrvvlQ31DJWOjUbtjnNdFZ8ugdZJ+xMrDUy0ly7LwJFtOlmXR3FTN+FgUr8eDz2fh9WQ2YWt6cpZcfmcLBqsZHnbeWZLFmMvvs9jaXMXW5tUbgrF4PPnNL574ORanrs7PyMg08Xj83DdFSBSu9G+i6V9KM6pIFzFN0+OxuEqaN/R6eTzWqg3iTAp7H7A3bbsFGFixP7TK/pxpbvDjWV7Ox1OtSzBYjd/rzP4/p2arrSpjYXbB7hjKBTyWhcfnpSStujXWVRBfdN58jlyNE2TSLHoCuE1EgiLiB+4EHk/tNMacBuZE5N3Jm34eeCzrSZVSSmVkzcJujOkH7gEOA0eBLmPMERE5KCK7k4f9LPB5ETkOVAF/nqvASimlVpfRqJMxpgvoWnHb7Wk/v8LbB1SVUkrZRJcUUEopl9HCrpRSLqOFXSmlXMbuU/S8kJiTuVEXc99ccmoucG42zbU+mmt93JQr7T7e8+23bF4HeQ/wlJ0BlFKqgO0Fnl55o92FvQx4F4llCJx3ppFSSjmTl8SJoc+TWJzxbewu7EoppbJMB0+VUspltLArpZTLaGFXSimX0cKulFIuo4VdKaVcRgu7Ukq5jBZ2pZRyGbuXFMgaEbkGeM4YU2Z3FgAR2Qs8CJQC3cAvGGPG7E0FyQuifJ5Ergjwi8mLpTiCiPw+sGyMuc/mHHcB9wIlwIPGmIfszJNORGqAZ4EfN8b02BwHABH5LPCR5Oa/GmP+u515UkTk94D9JK5092VjzJ/aHOltROSPgUZjzN3ZfFxXtNiTV3b6CxLFyin+Gvh5Y8xVwOvAb9icJ+XvgU8YY3Ylf3bERVFEpFZEvgx82gFZ2oAHSCx5sQs4ICLb7U2VICI3kDiF/DK7s6SIyD7gPwDXkHi9rhORn7Y3FYjIe4D3AjuB3cCviIjYm+otInIb8Au5eGxXFHbgT0i0jp3kCmPM6yJSArQBTmitlwH3GmNeTd70KtBhY6R0dwAnSLyXdtsHHDLGjBpjosAjJFp9TvBLwKfI03WFMxQGPm2MWTDGLAI/wgGfK2PMvwM/ZoxZAppI9FA44irzItJAovHwuVw8fsF3xYjITwJ+Y8wjDvpjjDFmUUSuInHN2EXgt22OhDFmHvgagIh4gPuAR+3MlGKM+SqAiNxncxSAVhLFKiWMQ64QZoz5BIDDPuvHUj+LyDYSXTLvvvA98if5e3g/8Bngn4B+myOlfInEJUfbc/HgBVPYReTDJPqG0x0Haki0sGxxoVzGmH3GmNeAZhH5ZeAfgZudkEtESoG/JfH+56TFsJFc+cyxBg+JPtkUC4jZlKVgiMiVwL8Cv2GMOWF3nhRjzGdF5H8B3ybxjecv7cwjIp8Aeo0xT4rI3bl4joIp7MaYfyLxF/ec5Av0W8D3Ui0YETkK7DXGTNmYq1xEfsoYk2oNf408dzGcL1cyWxXwLRIDp3ckvzrbnsth+kgsh5rSgrO6PhwnOSj/DeC/GWP+we48ACJyOVBujDlqjJkRkX8m0d9ut48CoWStagCqROTzxphfy9YTFExhPx9jzMPAw6ltEYknBwXttgg8JCK9xpgXSXw1fceayTb5GnAS+KQxRluh5/cEcJ+IBEn0yd4JHLA3knOJSDuJLr2PGmMO2Z0nzSXA/SKyh8Q3sDuAr9gbCYwx70v9nGyx35rNog7uGTx1FGPMMom/yn+Z/Ku8H/iEvanOTQm9g0T/50siclREDtocy3GMMf0k+j8PA0eBLmPMEXtTOdpngHLgT5OfqaMi8km7QxljDpLoGnoZeBF41infJnJN12NXSimX0Ra7Ukq5jBZ2pZRyGS3sSinlMlrYlVLKZbSwK6WUy2hhV0opl9HCrpRSLqOFXSmlXOb/A1Z8W82d5mlvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_dist(create_gaussian_pdf(1, 0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Central Limit Theoreom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let, $X_{1}, X_{2}, ..., X_{N}$ be <b>iid (identically and independently distributed)</b> random variables with the mean ( $\\operatorname{E}[X_{i}]=\\mu$ ) and finite variance ( $Var[X_{i}]=\\sigma^2<\\infty$ )\n",
    "\n",
    "Let, $S_n := \\frac{X_1+\\cdots+X_n}{n}$ be a sample mean (which itself is a random variable)\n",
    "\n",
    "<br />\n",
    "\n",
    "<b>Theorem (CLT):</b><br />\n",
    "As $n\\rightarrow{\\infty}$, \n",
    "$\\sqrt{n}\\left(S_n - \\mu\\right)\\ \\xrightarrow{d}\\ N\\left(0,\\sigma^2\\right)$\n",
    "\n",
    "or, equivalently\n",
    "\n",
    "$\\frac{\\sqrt{n}}{\\sigma}\\left(S_n - \\mu\\right)\\ \\xrightarrow{d}\\ N\\left(0,1\\right)$\n",
    "\n",
    "\n",
    "<br />\n",
    "\n",
    "Let's visualize the result of this theorem on coin flip example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAATf0lEQVR4nO3df4xlZX3H8fcM4O7oziqOl+4CgrG4X61F1iiYCIg/tiYkki2RlRSK0gpI1IYmqGnrWlhaNSZlJRiJVvnVblEiiAgLpnEhoiBoVaBx5RvargS6k3QzEtlFdt1lpn+cO+2ddX6cOzN3zvDM+5UQ5j73OXO/82X3M4fn3HuevrGxMSRJZepvugBJUu8Y8pJUMENekgpmyEtSwQx5SSrYoU0X0GEZcCIwDDzfcC2S9EJxCLAa+DGw7+AnF1PInwh8v+kiJOkF6lTgBwcPLqaQHwZ4+ulnGR1dnO/dHxpawcjInqbLWDTsx0T2YyL7MVGv+tHf38fhh78E2hl6sMUU8s8DjI6OLdqQBxZ1bU2wHxPZj4nsx0Q97seky9xeeJWkghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCL6X3ykl6gBlcOsHzZ78ZJqzU44fHefQfY/cxzC1WWMOQlzYPlyw7ljEtvn3HeHVeuZ/cC1KP/53KNJBXMkJekghnyklQwQ16SCmbIS1LBDHlJKljtt1BGxD8Ar8jM8yNiLfBVYCVwH3BxZh6IiGOALcARQALnZqa7BkhSQ2qdyUfEu4APdAxtAT6amWuAPuDC9vg1wDWZ+Vrg34BPzWOtkqQuzRjyEfFy4NPAZ9qPjwUGMvPB9pQbgA0RcRjwNuCWzvF5rleS1IU6Z/JfBj4JPN1+fCQT9xIcBo4GXgE8k5kHDhqXJDVk2jX5iLgAeDIzt0XE+e3hfqBzo8I+YHSScdrjXRkaWtHtIQvq4HtxLHX2YyL7MbOl3KMmfvaZLryeDayOiIeBlwMrqIJ8dcecVcBO4H+Al0bEIZn5fHvOzm4LGhnZs2g3/221Btm1yztvjLMfEy3lfnQTXku5R7342fv7+6Y9OZ52uSYz/ygz/zAz1wJ/C3w7M/8M2BsRJ7ennQfcnZn7ge9T/WIAeD9w91x/AEnS7M32ffLnAp+PiMeozu6vbo9/GLgoIrYDpwIb516iJGm2ar9PPjNvoHrHDJn5CHDSJHOeAN4+P6VJkubKT7xKUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpWa9OQiLgCOItqf9drM3NzRFwPnAI82562KTNvi4h1wGZgALg5M90dSpIaMmPIR8RpwDuBNwCHAdsjYivwZuBtmTncMXcAuA44DXgS2BoRp2eme71KUgNmXK7JzO8B78jMA8ARVL8YngOOAa6LiEcjYlNE9FNtCfh4Zu5oz98CbOhd+ZKk6dRarsnM/RGxCfgY8A2qM/p7qDbu/jVwJ/BBYA8w3HHoMHB0NwUNDa3oZvqCa7UGmy5hUbEfE9mPmS3lHjXxs3ezkfdlEfE54A7gXZl55vhzEfEF4P3ALVTr9uP6gNFuChoZ2cPo6NjMExvQag2ya9fupstYNOzHREu5H92E11LuUS9+9v7+vmlPjmdcromI10bEWoDM/A3wTeDsiHhvx7Q+YD/wFLC6Y3wVsHMWdUuS5kGdM/lXA5si4hSqs/T1wPeAqyLiHqolmouAG4GHgIiI44AdwDlUF2IlSQ2oc+H1LmAr8DPgJ8ADmXkF8FngfmA78HBmfi0z9wLnA7e2xx+jWsKRJDWg7oXXy4HLDxq7BrhmkrnbgBPmoTZJ0hz5iVdJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBat9q2FJUvcGVw6wfFkVtdPdknnvvgPsfua5eX99Q16Semj5skM549LbZ5x3x5Xr6cWd9l2ukaSCGfKSVDBDXpIKVmtNPiKuAM6i2hnq2szcHBHrgM3AAHBzZm5sz10LfBVYCdwHXJyZB3pRvCRpenX2eD0NeCfwBuDNwF9ExAlU2/qtB14HnBgRp7cP2QJ8NDPXUO39emEvCpckzazO9n/fA97RPhs/gurs/2XA45m5oz2+BdgQEccCA5n5YPvwG4ANPalckjSjWmvymbk/IjZR7du6DTgSGO6YMgwcPc24JKkBtd8nn5mXRcTngDuANVTr8+P6gFGqXxqTjdc2NLSim+kLbroPMyxF9mMi+zEzezS1XvRmxpCPiNcCyzPz4cz8TUR8k+oi7PMd01YBO4GngNWTjNc2MrKH0dGxmSc2oNUaZNeuXnxc4YXJfky0lPvRTTgttR71ujf9/X3TnhzXWa55NfCViFgWES+iutj6ZSAi4riIOAQ4B7g7M58A9kbEye1jzwPu7rpqSdK8qHPh9S5gK/Az4CfAA5n5deB84FaqdfrHgFvah5wLfD4iHgNWAFfPf9mSpDpqrcln5uXA5QeNbQNOmGTuI8BJ81CbJGmO/MSrJBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalgtTYNiYjLgPe1H27NzE9ExPXAKcCz7fFNmXlbRKwDNgMDwM2ZuXG+i5Yk1VNnI+91wLuBNwJjwHci4kzgzcDbMnO4Y+4AcB1wGvAksDUiTs9M93mVpAbUOZMfBi7NzN8CRMQvgGPa/1wXEUcBtwGbqLb9ezwzd7TnbgE24GbektSIGUM+M38+/nVEvIZq2eZU4O3Ah4FfA3cCHwT2UP1SGDcMHN1NQUNDK7qZvuBarcGmS1hU7MdE9mNm9mhqvehNrTV5gIh4PbAV+HhmJnBmx3NfAN4P3EK1pDOuDxjtpqCRkT2Mjo7NPLEBrdYgu3btbrqMRcN+TLSU+9FNOC21HvW6N/39fdOeHNd6d01EnAxsA/4qM2+MiOMj4r0dU/qA/cBTwOqO8VXAzq6rliTNizoXXl8JfAs4OzPvaQ/3AVdFxD1USzQXATcCD1WHxHHADuAcqguxkqQG1Fmu+RiwHNgcEeNjXwI+C9wPHAbcmplfA4iI84Fb28fcRbWEI0lqQJ0Lr5cAl0zx9DWTzN8GnDDHuiRJ88BPvEpSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklSwWnu8RsRlVBt4A2zNzE9ExDpgMzAA3JyZG9tz1wJfBVYC9wEXZ+aBea9ckjSjGc/k22H+buCNwFrgTRHxJ1Tb+q0HXgecGBGntw/ZAnw0M9dQbRN4YS8KlyTNrM5yzTBwaWb+NjP3A78A1gCPZ+aO9ln6FmBDRBwLDGTmg+1jbwA29KBuSVINdbb/+/n41xHxGqplmy9Qhf+4YeBo4MgpxiVJDai1Jg8QEa8HtgIfBw5Qnc2P6wNGqf7PYGyS8dqGhlZ0M33BtVqDTZewqNiPiezHzOzR1HrRm7oXXk8GbgX+MjO/HhGnAas7pqwCdgJPTTFe28jIHkZHx2ae2IBWa5Bdu3Y3XcaiYT8mWsr96CacllqPet2b/v6+aU+O61x4fSXwLeCczPx6e/ih6qk4LiIOAc4B7s7MJ4C97V8KAOcBd3ddtSRpXtQ5k/8YsBzYHBHjY18Czqc6u18O3AXc0n7uXOArEbES+Clw9TzWKzVmcOUAy5dN/1em1Rpk774D7H7muQWqSppenQuvlwCXTPH0CZPMfwQ4aY51SYvO8mWHcsalt884744r17O0FiS0mPmJV0kqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgrWzR6vK4EHgPdk5i8j4nrgFODZ9pRNmXlbRKwDNgMDwM2ZuXG+i5Yk1VN3j9e3AF9h4ubdbwbelpnDHfMGgOuA04Anga0RcXpmugWgJDWg7pn8hcBHgH8GiIgXA8cA10XEUcBtwCaqHaEez8wd7XlbgA24z6skNaJWyGfmBQAde7yuAu4BPgz8GrgT+CCwBxjuOHQYOHqeapUkdan2mnynzPwv4MzxxxHxBeD9VJt5j3VM7QNGu/neQ0MrZlPSgmm1BpsuYVGxH5OzL1OzN1PrRW9mFfIRcTywJjNvbQ/1AfuBp4DVHVNXATu7+d4jI3sYHR2beWIDWq1Bdu1yi+ZxS60f3fwFXEp9AXsznV73pr+/b9qT41mFPFWoXxUR91At0VwE3Ag8BEREHAfsAM6huhArSWrArN4nn5mPAp8F7ge2Aw9n5tcycy9wPnBre/wxqiUcSVIDujqTz8xXdXx9DXDNJHO2ASfMuTJJ0pz5iVdJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsFqbxoSESuBB4D3ZOYvI2IdsBkYAG7OzI3teWuBrwIrgfuAizPzwLxXLkmaUa0z+Yh4C/ADYE378QDV3q3rgdcBJ0bE6e3pW4CPZuYaqr1gL5zvoiVJ9dRdrrkQ+Aiws/34JODxzNzRPkvfAmyIiGOBgcx8sD3vBmDDPNYrSepCreWazLwAICLGh44EhjumDANHTzNe29DQim6mL7hWa7DpEhYV+zE5+zI1ezO1XvSmq428O/QDYx2P+4DRacZrGxnZw+jo2MwTG9BqDbJr1+6my1g0llo/uvkLuJT6AvZmOr3uTX9/37Qnx7N9d81TwOqOx6uolnKmGpckNWC2If8QEBFxXEQcApwD3J2ZTwB7I+Lk9rzzgLvnoU5J0izMKuQzcy9wPnArsB14DLil/fS5wOcj4jFgBXD13MuUJM1GV2vymfmqjq+3ASdMMucRqnffSJIa5ideJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBZrv9n15ABlcOsHxZvf/Ue/cdYPczz/W4IkkLZU4hHxH3AkcA+9tDHwJ+H9gIHAZclZlfnFOFmrPlyw7ljEtvrzX3jivXs7R24JTKNuuQj4g+YA1wbGYeaI8dBXwdeBOwD3ggIu7NzO3zUawkqTtzOZOP9r//NSKGgK8Au4F7MvNXABFxC3AWcMWcqpQkzcpcLrweDmwDzgTeBVwMHAMMd8wZBo6ew2tIkuZg1mfymflD4IfjjyPiWmAz8Pcd0/qA0W6+79DQitmWtCBarcGmS+i5bn7GpdCP2bAvU7M3U+tFb+ayJn8KsKy9oTdUgf5LYHXHtFXAzm6+78jIHkZHx2ZbVk+1WoPs2vXCuyzZ7R+cuj/jC7Ufs9VNH5dSX8DeTKfXvenv75v25Hgua/IvA66IiLdSvZPmA8CfAlsiogU8C7wXuGgOryFJmoNZr8ln5p3AVuBnwE+A6zLzfuCTwL3Aw8BNmfmj+ShUktS9Ob1PPjM/BXzqoLGbgJvm8n0lSfPD2xpIUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgo2p01DphIR5wAbqbYFvCozv9iL1+k0uHKA5cvq/Th79x1g9zPP9bgiSWrevId8RBwFfBp4E7APeCAi7s3M7fP9Wp2WLzuUMy69vdbcO65cz9LaSljSUtWLM/l1wD2Z+SuAiLgFOAu4YobjDoFq5/HZOuLwgdpzZ/s6c6mvSb3qzQu1H7NVt49LrS9gb6bTy950HHPIZM/3jY2Ndf1NpxMRfw28JDM3th9fAJyUmRfNcOgpwPfntRhJWjpOBX5w8GAvzuT7gc7fHH3AaI3jfkxV5DDwfA/qkqQSHQKspsrQ39GLkH+KKqzHrQJ21jhuH5P8FpIkzeg/p3qiFyH/XeDyiGgBzwLvBWZaqpEk9cC8v08+M/8b+CRwL/AwcFNm/mi+X0eSNLN5v/AqSVo8/MSrJBXMkJekghnyklQwQ16SCtaTG5SVKiJOBa4CXgTsAD6QmU83W1VzIuJk4PNU/RgB/jwzn2i2quZFxN8Bz2fm5U3X0oQmblC42EXESuAB4D2Z+cuFfG3P5LtzPXBeZh4PbAc+3nA9TfsX4ILMXNv++uqG62lURLw0Iq4FLm26lqZ03KDwFGAtcFFE/EGzVTUrIt5C9UHPNU28viHfnddl5vaIOAw4CljKZ/HLgI2Z+Wh76FHgmAZLWgzWA48DVzZdSIP+7waFmfksMH6DwqXsQuAj1Pvk/7xzuaYLmbk/Io6n+lTvfuBvGi6pMZm5D9gCEBH9wOXAt5qsqWmZ+U8AEXF5w6U06Uiq+0+NGwZOaqiWRSEzLwCIiEZe35CfRERsoFpr7vRYZq7LzH8Hfi8iPgTcDLx1wQtcYNP1IyJeBNxI9WfpMwteXAOm60cT9Swys71BoXrEkJ9EZn4D+EbnWEQsj4g/zszxs9UtLJH/LZ+sHwARsQL4NtVF1/WZuX+ha2vCVP0QMPsbFKpHXJOvbz/wxYh4U/vx+/CumVuA/wDObi/fSN8F3hURrYh4MdUNCr/TcE1LmiFfU2Y+D5wN/GNEPEx1MemCZqtqTkS8kepC48nATyPi4Yi4q+Gy1DBvULj4eIMySSqYZ/KSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekgv0vRrIthfL+MlUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAROUlEQVR4nO3df4zkdX3H8efuHe6u3m4lm7F3gGhavHepoWAUTATEthcTEumVyGl6FiUVkKgNTU5NrWcE+sP0D04C0dqCp7RXhHhXpHhc0/QgRUXQqGjjyTu0PYj09o/LauSOcucdu/1jvmtG3Nv5zs7Mzc6nz0dyYeczn+/O+z3svfZzn+/Md0bm5+eRJJVpdNAFSJL6x5CXpIIZ8pJUMENekgpmyEtSwVYPuoAWY8D5wAzwwoBrkaRhsQpYB3wLOPriO1dSyJ8PfHXQRUjSkLoY+NqLB1dSyM8A/OQnzzE3V/+1+9PTa5idPdy3ogbJ3oaTvQ2nYe1tdHSEU099GVQZ+mIrKeRfAJibm+8o5BeOKZW9DSd7G05D3tui29yeeJWkghnyklQwQ16SCmbIS1LBDHlJKpghL0kFq/USyoi4CbgCmAc+l5nbImIDsA2YAO7JzK3V3POAO4Ap4GHgusw83o/iJUlLa7uSj4hLgN8Bfgt4A/DHEXEusB3YCJwNnB8Rl1aH7AA+mJnrgRHgmn4ULqm/JqcmaDQmf+EP8Etjk1MTA65US2m7ks/Mf4+I387M4xFxenXMy4EnM3M/QETsADZFxD5gIjMfrQ7/AnAj8Dd9qV5S34yPreayLfe1nXf/zRs5dBLq0fLU2q7JzGMRcSPwIeBLwGn84ltoZ4AzlhivbXp6TSfTAX6+wiiRvQ2nkntbTCn9ltJHq9qXNcjMT0TEXwP3A+tp7s8vGAHmaG7/LDZe2+zs4Y7eWtxoTHLwYJnrCHsbTqX01kngldLvMPYxOjqy5OK4zp78b1QnU8nM/wX+CXgLzUtbLlgLHACeOcG4JGkA6ryE8teA2yNiLCJeQvNk698CERFnRcQqYDOwJzOfBo5ExIXVsVcCe/pRuCSpvbYhn5kPALuB7wLfBh7JzLuBq4BdwD7gCWBndci7gE9FxBPAGuDW3pctSaqj7onXG4AbXjS2Fzh3kbnfAy7oQW2SpC75jldJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSpY7atQStJKMzk1wfhY+xg7cvQ4h559/iRUtPIY8pKGlh9s0p7bNZJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYLUuNRwRnwDeUd3cnZkfiYjPAxcBz1XjN2bmvRGxAdgGTAD3ZObWXhctSaqnbchXof1W4HXAPPAvEXE58AbgzZk50zJ3AtgOXAL8CNgdEZdm5p5+FC9JWlqdlfwMsCUzfwYQET8Ezqz+bI+I04F7gRuBC4AnM3N/NXcHsAkw5CVpANqGfGb+YOHriHgNzW2bi4G3AO8Hfgp8BXgvcJjmL4UFM8AZnRQ0Pb2mk+kANBqTHR8zLOxtOJXc22KGod86NQ5DH52q/fF/EfFaYDfw4cxM4PKW+24D3g3spLmls2AEmOukoNnZw8zNzbefWGk0Jjl4sMwP9rK34VRKb50E3qD67WWNw/r/bXR0ZMnFca1X10TEhcBe4E8z886IOCci3t4yZQQ4BjwDrGsZXwsc6LhqSVJP1Dnx+krgy8A7M/PBangEuCUiHqS5RXMtcCfwWPOQOAvYD2ymeSJWkjQAdbZrPgSMA9siYmHss8Anga8DpwC7MvOLABFxFbCrOuYBmls4kqQBqHPi9Xrg+hPc/ZlF5u8Fzu2yLklSD/iOV0kqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkq2Oo6kyLiE8A7qpu7M/MjEbEB2AZMAPdk5tZq7nnAHcAU8DBwXWYe73nlkqS22q7kqzB/K/A64Dzg9RHxB8B2YCNwNnB+RFxaHbID+GBmrgdGgGv6Ubgkqb062zUzwJbM/FlmHgN+CKwHnszM/dUqfQewKSJeBUxk5qPVsV8ANvWhbklSDW23azLzBwtfR8RraG7b3EYz/BfMAGcAp51gXJI0ALX25AEi4rXAbuDDwHGaq/kFI8AczX8ZzC8yXtv09JpOpgPQaEx2fMywsLfhVHJvixmGfuvUOAx9dKruidcLgV3An2Tm3RFxCbCuZcpa4ADwzAnGa5udPczc3Hz7iZVGY5KDBw918hBDw96GUym9dRJ4g+q3lzUO6/+30dGRJRfHdU68vhL4MrA5M++uhh9r3hVnRcQqYDOwJzOfBo5UvxQArgT2dNOAJGn56qzkPwSMA9siYmHss8BVNFf348ADwM7qvncBt0fEFPAd4NYe1isVY3JqgvGx9n8Fjxw9zqFnnz8JFalEdU68Xg9cf4K7z11k/veAC7qsSyre+NhqLttyX9t599+8keHbRNBK4TteJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalgq+tOjIgp4BHgbZn5VER8HrgIeK6acmNm3hsRG4BtwARwT2Zu7XXRkqR6aoV8RLwRuB1Y3zL8BuDNmTnTMm8C2A5cAvwI2B0Rl2bmnt6VLEmqq+5K/hrgA8A/AETES4Ezge0RcTpwL3AjcAHwZGbur+btADYBhrwkDUCtkM/MqwEiYmFoLfAg8H7gp8BXgPcCh4GZlkNngDN6VKskqUO19+RbZeZ/A5cv3I6I24B3AzuB+ZapI8BcJ997enpNx/U0GpMdHzMs7G049bq3lf5crfT6oF6Nw9BHp5YV8hFxDrA+M3dVQyPAMeAZYF3L1LXAgU6+9+zsYebm5ttPrDQakxw8eKiThxga9jac6vbWSaAM4rla6fVBb2sc1p/J0dGRJRfHywp5mqF+S0Q8SHOL5lrgTuAxICLiLGA/sJnmiVhJ0gAs63Xymfl94JPA14F9wOOZ+cXMPAJcBeyqxp+guYUjSRqAjlbymfnqlq8/A3xmkTl7gXO7rkyS1DXf8SpJBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwVYPugBJKtnk1ATjY+2j9sjR4xx69vmeP74hL0l9ND62msu23Nd23v03b+RQHx6/VshHxBTwCPC2zHwqIjYA24AJ4J7M3FrNOw+4A5gCHgauy8zjfahbklRD2z35iHgj8DVgfXV7AtgObATOBs6PiEur6TuAD2bmemAEuKYfRUuS6qlz4vUa4APAger2BcCTmbm/WqXvADZFxKuAicx8tJr3BWBTj+uVJHWg7XZNZl4NEBELQ6cBMy1TZoAzlhiXJA3Ick68jgLzLbdHgLklxjsyPb2m44IajcmOjxkW9jacet3bSn+uVnp9UK/GQffRj8dfTsg/A6xrub2W5lbOicY7Mjt7mLm5+fYTK43GJAcP9uOc9ODZ23Cq21snf6EH8Vyt9PqgtzX262ey38/j6OjIkovj5bwZ6jEgIuKsiFgFbAb2ZObTwJGIuLCadyWwZxnfX5LUIx2HfGYeAa4CdgH7gCeAndXd7wI+FRFPAGuAW3tTpiRpOWpv12Tmq1u+3gucu8ic79F89Y0kaQXw2jWSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFW86HhkgDMzk1wfhY+x/bI0ePc+jZ509CRdLKZshrqIyPreayLfe1nXf/zRsp83OnpM64XSNJBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYF1d1iAiHgJeARyrht4H/DqwFTgFuCUzP91VhZKkZVt2yEfECLAeeFVmHq/GTgfuBl4PHAUeiYiHMnNfL4qVJHWmm5V8VP/914iYBm4HDgEPZuaPASJiJ3AFcFNXVUqSlqWbPflTgb3A5cDvAtcBZwIzLXNmgDO6eAxJUheWvZLPzG8A31i4HRGfA7YBf9EybQSY6+T7Tk+v6biWRmOy42OGhb2t3O9/Mh97pf8crPT6oF6Ng+6jH4/fzZ78RcBYZu6thkaAp4B1LdPWAgc6+b6zs4eZm5uvPb/RmOTgwTKvHG5vix9X16Ceu7q9rfReVnp90Nsa+/X3rd/P4+joyJKL42725F8O3BQRb6L5Spr3AH8I7IiIBvAc8Hbg2i4eQ5LUhWXvyWfmV4DdwHeBbwPbM/PrwMeAh4DHgbsy85u9KFSS1LmuXiefmR8HPv6isbuAu7r5vpKk3vAdr5JUMENekgpmyEtSwQx5SSqYIS9JBevq1TUqy+TUBONj9X4kjhw9zqFnn+9zRZK6Zcjr58bHVnPZlvtqzb3/5o2U+V5cqSxu10hSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kF68tnvEbEZmArcApwS2Z+uh+PM2zqflC2H5ItqVd6HvIRcTrwl8DrgaPAIxHxUGbu6/VjDZu6H5Tth2RL6pV+rOQ3AA9m5o8BImIncAVwU5vjVgGMjo50/IDLOWZQXnHqRK15Cz2d7N7q1gfd17bc4zt9Dgeh7mOv9F5Wen3Q2xr71Uc/n8eWY1Ytdv/I/Px8x990KRHxUeBlmbm1un01cEFmXtvm0IuAr/a0GEn6/+Ni4GsvHuzHSn4UaP3NMQLM1TjuWzSLnAFe6ENdklSiVcA6mhn6S/oR8s/QDOsFa4EDNY47yiK/hSRJbf3Xie7oR8j/G3BDRDSA54C3A+22aiRJfdDz18ln5v8AHwMeAh4H7srMb/b6cSRJ7fX8xKskaeXwHa+SVDBDXpIKZshLUsEMeUkqWF8uUDYIEfE64NHMHBt0Lb0SERcDtwAvAfYD78nMnwy2qt6IiAuBT9HsbRb4o8x8erBV9VZE/DnwQmbeMOhaulX6RQcjYgp4BHhbZj414HJ6qoiVfES8FLiNZmCU5PPAlZl5DrAP+PCA6+mlfwSuzszzqq9vHXA9PRMRvxIRnwO2DLqWXmi56OBFwHnAtRHxm4Otqnci4o0034i5ftC19EMRIQ/cTHPFW5qzM3NfRJwCnA6UsoofA7Zm5veroe8DZw6wpF7bCDxJ8+eyBD+/6GBmPgcsXHSwFNcAH6DeO/OHztBv10TE7wEvzcydETHocnoqM49FxDk030V8DPizAZfUE5l5FNgBEBGjwA3AlwdZUy9l5t8DRMQNAy6lV06jeU2pBTPABQOqpecy82qA0vJjwdCEfERsormH2+oJYIrmSmNonai3zNyQmf8B/GpEvA+4B3jTSS+wC0v1FhEvAe6k+XP4Vye9uC4t1dsg6umj5V50UCvA0IR8Zn4J+FLrWHUZ448CDy/8Fo6Ix4GLM3NoPnfjBL2NR8TvZ+bCCncHQ/jP/8V6A4iINcA/0zzpujEzj53s2rp1ot4KtNyLDmoFGJqQX0xm3gHcsXA7IuarE3klOAZ8OiJ+lJnfBt5BWVfp3AH8J3BdZroqXNm86OAQK+XEa3Ey8wXgncDfVf86uQK4erBV9Ub1cteNwIXAdyLi8Yh4YMBl6QS86OBw8wJlklQwV/KSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekgv0fyOmDkHwGnL4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAASu0lEQVR4nO3dfYxldX3H8ffMAjsrO2t1vLqLFIzB/YpWXEuBRHxqXW1IIKupKxGkUrOLRGg1XTBaV6E22toUMDQ+A8VmayVCKyDSGvEBKcH6UNCIfEMbIMWdppspLbvIPs70j3vH3u7DzLlPe+797fuVbDLnnN/8ft+5c+9nzv7Oub87Njc3hySpTON1FyBJGhxDXpIKZshLUsEMeUkqmCEvSQU7qu4C2iwFTgOmgX011yJJo2IJsAr4PrBr/4PDFPKnAd+tuwhJGlGvBu7Zf+cwhfw0wBNPPMXs7IH37k9NLWdmZsdhL6pfrL9e1l+fUa4dhr/+8fExnvWsY6GVofsbppDfBzA7O3fQkJ8/Nsqsv17WX59Rrh1Gpv6DTnN74VWSCmbIS1LBKk3XRMQK4F7g7Mx8tG3/pcBbMvN1re0TgC3Ac4EEzs/M4Z3MkqTCLXomHxFn0Lxiu3q//S8B3r9f808Bn8rMFwM/AD7UpzolSV2oMl2zEbgE2Dq/IyKWAp8FPty272jgNcDNrV03Auv7VagkqXOLTtdk5gaAiGjf/afADcAjbfueAzyZmXtb29PA8f0pU5LUjY5voYyINwAnZOYfRsTr2g6NA/vfZzTbaf9TU8sPeazRmOy0u6Fi/fWy/vqMcu0w2vV3c5/824CXRsT9wHJgZUTcBLwdeGZELMnMfTTfZrt1gX4OamZmx0HvSW00Jtm2bXsX5Q4H66/XoeqfXLGMiaXVXgY7d+1l+5NP97u0Skb58R/l2mH46x8fH1vw5LjjkM/Md85/3TqTvzIzz21tfxc4F/gi8LvAnZ32Lx1OE0uP4pxNt1Zqe/tV6xjel7p0cP2+T/7dwEUR8SDNdRQ297l/SVIHKp/JZ+YLDrLv28Dr2rYfa9+WJNXLd7xKUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklSwYfr4Px3Bqi4vUHVpgYP1N8rrj0jdMuQ1FKouL1B1aYFO+pNK5nSNJBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpWeYGyiFgB3AucnZmPRsRFwB8Ac8APgHdl5u6IWANcB6wA7gYuzsy9/S9dkrSYSmfyEXEGcA+wurW9GrgceCVwSqufS1rNtwCXZuZqYAzY2OeaJUkVVZ2u2UgzxLe2tncB787MJzNzDvgJcEJEnAgsy8z7Wu1uBNb3sV5JUgcqTddk5gaAiJjffgx4rLWvAVwKXAgcB0y3fes0cHzfqpUkdaSnDw2JiOcDdwLXZ+a3I+JMmnP088aA2U76nJpafshjo/7JPtbfH3XWcaSO3atRrh1Gu/6uQz4iXgz8I3BtZl7V2v04sKqt2Ur+b4qnkpmZHczOzh2wv9GYZNu2Kp8JNJysf/H+q6pSx6BelHX9Dkf5+TPKtcPw1z8+PrbgyXFXt1BGxCTwdWBzW8DPT+PsbJ3RA1xA80xfklSDbs/kNwDPAzZFxKbWvtsy88PA+cDnW7dc/gi4tvcypdHR7w8ll3rRUchn5gtaX17T+newNg8Ap/dWljS6+v2h5FIvfMerJBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUsJ7Wk5cWU3WxLkmD4atPA9XJYl2S+s/pGkkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBKr8ZKiJWAPcCZ2fmoxGxFrgaWAbclJmbW+3WANcBK4C7gYszc2/fK5ckLarSmXxEnAHcA6xubS8DbgDWAScDp0XEWa3mW4BLM3M1MAZs7HfRkqRqqk7XbAQuAba2tk8HHs7MR1pn6VuA9RFxIrAsM+9rtbsRWN/HeiVJHag0XZOZGwAiYn7XccB0W5Np4PgF9lc2NbX8kMcajclOuho61t8fddbR77E76W9YHv9ujHLtMNr1d7tA2Tgw17Y9BswusL+ymZkdzM7OHbC/0Zhk27btnVc6JI7U+gfx4qhSx6BelP0eu+pjOsrPn1GuHYa//vHxsQVPjru9u+ZxYFXb9kqaUzmH2i9JqkG3If89ICLipIhYApwH3JmZjwE7I+LMVrsLgDv7UKckqQtdhXxm7gQuBG4BHgQeAm5uHT4fuCYiHgKWA9f2XqYkqRsdzcln5gvavr4LePlB2jxA8+4bSVLNfMerJBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIK1tEHee8vIt4OfKC1eWdmXhYRa4DrgBXA3cDFmbm3tzKlI9fkimUANBqTC7bbuWsv2598+nCUpBHSdchHxDOAa4HVwH8D/xQRa4FPABsy876IuB7YCHy6H8VKR6KJpUdxzqZbF213+1Xr2H4Y6tFo6WW6Zknr+48Fjm792wMsy8z7Wm1uBNb3UqAkqXtdh3xmbgc+BDwEPA48CuwGptuaTQPH91CfJKkHvUzXnAK8EzgR+B9gC/BGYK6t2Rgw20m/U1PLD3lssTnJYWf9/VFnHf0ee9j765dhrauqUa6/lwuvvw3clZn/CRARNwKXAava2qwEtnbS6czMDmZn5w7Y32hMsm3b6M44Hqn1D+LFUaWOQb0o+z12Hf0dbkfqc/9wGR8fW/DkuJc5+QeAtRFxbESMAecA3wF2RsSZrTYXAHf2MIYkqQe9zMl/Hfhb4IfAj2leeP0z4Hzgmoh4CFhO8w4cSVINerpPPjM/Dnx8v90PAKf30q8kqT98x6skFcyQl6SCGfKSVLCe5uRVnskVy5hYevCnRfutfK6TIo0GQ17/j+ukSGVxukaSCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVLCePjQkIs4BrgCOBb6eme+JiLXA1cAy4KbM3Nx7mZKkbnR9Jh8RLwQ+A7wJOAX49Yg4C7gBWAecDJzW2idJqkEv0zVvpnmm/nhm7gHOBX4BPJyZj2TmXmALsL4PdUqSutDLdM1JwO6IuA04Afgq8FNguq3NNHB8D2NIknrQS8gfBbwGeB2wA7gNeBqYa2szBsx20unU1PJDHms0JjutcaiMev37q+vnqfNx7PfYw95fvwxrXVWNcv29hPx/AN/IzG0AEfH3NKdm9rW1WQls7aTTmZkdzM7OHbC/0Zhk27bt3Vdbs1Gpv5Mnc5WfZxAvjrrGHcTYdfR3uI3Kc/9Qhr3+8fGxBU+Oewn5rwJfiIhfAbYDZwE3A++PiJOAR4DzaF6IlSTVoOsLr5n5PeDPgXuAB4HHgE8DFwK3tPY9RDP4JUk16Ok++cy8gQPP1O8CXt5Lv5Kk/vAdr5JUMENekgrW03SNpNEzuWIZE0sXf+nv3LWX7U8+fRgq0iAZ8tIRZmLpUZyz6dZF291+1TqG98ZBVeV0jSQVzJCXpIIZ8pJUMOfkR5gX0CQtxpAfYV5Ak7QYp2skqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVLC+LFAWEX8BPCczL4yINcB1wArgbuDizNzbj3EkSZ3p+Uw+Il4PvKNt1xbg0sxcDYwBG3sdQ5LUnZ5CPiKeDXwU+Fhr+0RgWWbe12pyI7C+lzEkSd3r9Uz+s8AHgSda28cB023Hp4HjexxDktSlrufkI2ID8O+ZeVdEXNjaPQ7MtTUbA2Y76XdqavkhjzUakx1WOVzqrH8QY9f185T0OB4p/fnarU8vF17PBVZFxP3As4HlNAN+VVublcDWTjqdmdnB7OzcAfsbjUm2bRvdzzcaRP2dPPGqjt3vPgfx4qhr3EGMXUJ/VcbztTs44+NjC54cdz1dk5lvyMxfy8w1wIeB2zLz94CdEXFmq9kFwJ3djiFJ6s0g7pM/H7gmIh6ieXZ/7QDGkCRV0Jf75DPzRpp30pCZDwCn96NfSVJvfMerJBXMkJekghnyklQwQ16SCmbIS1LB+nJ3jaqZXLGMiaWLP+Q7d+1l+5NPH4aKJJXOkD+MJpYexTmbbl203e1XrWN4318naZQ4XSNJBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwV6GUdFC79+yj0Zis1NblsYdXTyEfEVcAb21t3pGZ74uItcDVwDLgpszc3GONkmpwzNFLKi2NDS6PPcy6nq5phfkbgVcAa4BTI+JtwA3AOuBk4LSIOKsfhUqSOtfLnPw0sCkzd2fmHuBnwGrg4cx8JDP3AluA9X2oU5LUha6nazLzp/NfR8SLaE7b/CXN8J83DRzfdXWSpJ70fOE1Il4K3AFcDuyleTY/bwyY7aS/qanlhzxW9SLQ4bZ7zz6OOXpJX/vs9886iMeurt9Hnc+DYf+9DOtjM6yv3apGuf5eL7yeCdwCvDczvxQRrwVWtTVZCWztpM+ZmR3Mzs4dsL/RmGTbtuG8tNNoTFb+7NaqqvysnTzxqj52/e5zEC+OusYdxNjD3l8nDjX2ML92qxj2+sfHxxY8Oe465CPiV4GvAOdm5jdbu7/XPBQnAY8A59G8ECtJqkEvZ/KXARPA1RExv+8zwIU0z+4ngK8BN/cwhiSpB71ceH0P8J5DHH55t/1KkvrHZQ0kqWCGvCQVzLVrJB02kyuWMbF08dhxLZz+MeQlHTYTS4+qfLvx8N60OFqcrpGkghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUsCNyWQPXz5B0pDgiQ971MyQdKZyukaSCGfKSVDBDXpIKZshLUsGKufBa9Y4ZSTqSDCQVI+I8YDNwNPCJzPzkIMZpV/WOGWjeNSNpeO3es49GY3LRdt7mvLi+h3xEPB/4KHAqsAu4NyK+lZkP9nssSWU65ugl3ubcJ4M4k18LfDMz/wsgIm4G3gJ8ZJHvWwIwPj52yAYLHQN47rOWVS6yatvFxuy0v7rGrdrfIPoc9semk+dNXb+XUX9s2o/V8XxYvnyCpRWmc3ft2suOHTu7GmNQYy+mrbYlBzs+Njc313GnC4mIDwDHZubm1vYG4PTMvGiRb30V8N2+FiNJR45XA/fsv3MQZ/LjQPtfjjFgtsL3fZ9mkdPAvgHUJUklWgKsopmhBxhEyD9OM6znrQS2Vvi+XRzkr5AkaVH/dqgDgwj5bwBXRkQDeAr4HWCxqRpJ0gD0/c1Qmflz4IPAt4D7gS9m5j/3exxJ0uL6fuFVkjQ8XNZAkgpmyEtSwQx5SSqYIS9JBRupZRsj4tXAJ4BjgEeAd2TmE/VWVU1EnAlcQ7P2GeCdmflYvVV1LiL+BNiXmVfWXUsVdSyW128RsQK4Fzg7Mx+tuZyORMQVwFtbm3dk5vvqrKdTEfERmsuyzAHXZ+bVNZfUsVE7k/8r4ILMfBnwIHB5zfV04m+ADZm5pvX1tTXX05GIeGZEXA9sqruWqtoWy3sVsAa4KCJeUm9VnYmIM2i+SXB13bV0KiLWAm8EXkHz8T81It5cb1XVRcRrgd8CTgF+A/j9iIh6q+rcqIX8yZn5YEQcDTwfGJWz+KXA5sz8cWvXj4ETaiypG+uAh4Gr6i6kA79cLC8znwLmF8sbJRuBS6j2rvFhMw1syszdmbkH+Bkj9LzPzO8Av5mZe4Hn0pz5eKreqjo3UtM1mbknIl5G8121e4A/qrmkSjJzF7AFICLGgSuBr9RZU6cy868BIuLKmkvpxHE0g2beNHB6TbV0JTM3AIzgCSSZ+dP5ryPiRTSnbc6sr6LOtTLnj4HLgC8DP6+5pI4NZchHxHqa89ftHsrMtZn5E+B5EfEu4CbglYe9wAUsVHtEHAN8gebj/rHDXlwFC9VfRz096naxPPVRRLwUuAO4PDMfrrueTmXmFRHxceB2mv+z+lzNJXVkKEM+M79M86/mL0XERES8KTPnz4C3MIRTBwerHSAilgO30bzouq7139ehc6j6R1S3i+WpT1o3HNwCvDczv1R3PZ2IiBcDE5l5f2b+IiL+jub8/EgZpTn5PcAnI+LU1vZbGa1VK7cA/wqc25q+0eB9A3h9RDQi4hk0F8v7h5prOmJExK/SnJY8b9QCvuWFwOcjYmnrf+HrGK3MAUYo5DNzH3Au8LmIuJ/mBbQN9VZVTUS8guYT5EzgRxFxf0R8reayiudiebW7DJgArm495++PiIvrLqqqzPwazWmmfwF+CNw7in+sXKBMkgo2MmfykqTOGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXsfwFCfCtvcWshggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPQElEQVR4nO3df4xlZ13H8ffM/phZurNa1sEuFEgI8BUR2lpbE0tFZa0hsVmIto3FhopbbGJMSbYQkdUWFGKNLQRD+CHFYlaFAEpb2hpCIUBtigRta1r6TTWVCB3jZkW7Xdr9NeMf9w4Zl9255965d869375f/+zeO8/M+T53zvnMc59zznOnlpaWkCRNtum2C5AkrZ1hLkkFGOaSVIBhLkkFGOaSVMDGFrY5A5wHLADHW9i+JE2iDcAO4OvA4RO/2EaYnwd8tYXtSlIFFwL3nPhkG2G+APDd7x5icbH3Ne7bt2/lwIEnR17UKFXoA9ToR4U+QI1+VOgDrF8/pqenOP3006CboSdqI8yPAywuLjUK8+W2k65CH6BGPyr0AWr0o0IfYN37cdLpaU+ASlIBhrkkFWCYS1IBhrkkFWCYS1IBhrkkFWCYS1IBbVxnLg3N3LYtzM703o2fPnyMg088tQ4VSe0wzDXRZmc2cvGeW3u2u/3GXRxch3qktjjNIkkFGOaSVIBhLkkFGOaSVIBhLkkFGOaSVIBhLkkFGOaSVIBhLkkFGOaSVIBhLkkFGOaSVIALbWksNV0NUVKHR4vGUj+rIUpymkWSSmg0Mo+IXwfe3n14V2ZeGxFnAx8FtgFfAa7OzGOjKVNamyNHjzM/Pwfw/X9Pxg+x0KTqGeYR8Szg/cBLgf8B/iEidgLvA3Zn5n0RcTNwFfDBURYrDWrzpg1+iIVKazIy30BnOuY04BCwCTgKbMnM+7ptbgHeiWGuCbdyBN+Lo3iNk55hnpkHI+L3gUeA7wFfBo4ACyuaLQBnjqRCaR01HcGDo3iNlybTLK8E3gS8EPhfYB9wEbC0otkUsNjPhrdv39q4bdOR0jir0Aeo049hafP1qPC7qNAHGI9+NJlm+SXg7sz8L4CIuAW4Ftixos0ZwOP9bPjAgSdZXFzq2W5+fo79+yd7/FOhD7C+/RiHg6OJtn6vFfapCn2A9evH9PTUqoPgJpcmPgDsjIjTImIKuJjOVMvTEXFBt80VwF1rLVaSNJieYZ6Znwf+BvgG8CCdE6B/DLwBeG9EPAJspXPFiySpBY2uM8/MG4AbTnj6AeD8oVckSeqbd4BKUgGGuSQV4EJb0oCa3mDkzUVaD4a5NCCXCNA4cZpFkgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgowzCWpAMNckgrwk4Y0FHPbtjA703t38iPUpNEwzDUUszMb/Qg1qUVOs0hSAYa5JBVgmEtSAYa5JBVgmEtSAV7NIo3YkaPHmZ+f69nOyza1Foa5NGKbN23wsk2NnNMsklSAYS5JBRjmklSAYS5JBRjmklSAYS5JBRjmklSA15lrVSeuU97k5hdJ669RmEfExcB1wGnA5zPzmojYCdwEbAE+mZl7R1em2tLPOuWS2tNzmiUiXgR8CHgd8ErgJyPitcDHgF3Ay4Dzus9JklrQZM789XRG3t/OzKPAZcD3gEcz87HMPAbsAy4ZYZ2SpFU0mWZ5MXAkIm4DXgB8DngIWFjRZgE4c/jlSZKaaBLmG4GfBX4OeBK4DXgKWFrRZgpY7GfD27dvbdy2wkm3Cn0YFl+LU+vntanwOlboA4xHP5qE+X8CX8jM/QAR8Xd0plSOr2hzBvB4Pxs+cOBJFheXerabn59j//7JXktukvswip20yWsxDgdHG5ruJ5O8Ty2r0AdYv35MT0+tOghuEuafAz4eET8MHAReC3wa+N2IeDHwGHA5nROikqQW9DwBmplfA/4EuAd4GPgW8EHgSuAz3eceoRPwkqQWNLrOPDM/xg+OvO8Gzhp6RZKkvnk7vyQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgGGuSQVYJhLUgEb2y5AUn/mtm0BYH5+btV2h48cZ2bzhp4/7+nDxzj4xFNDqU3tMcylCTM7s5GL99zas93tN+5q3O7gMApTq5xmkaQCDHNJKqDxNEtE/CnwI5l5ZUScDXwU2AZ8Bbg6M4+NqEZJUg+NwjwiXgO8Ebij+9Q+YHdm3hcRNwNXAR8cTYkatrltW5idaed0yZGjx3ueuJPUv55HdEQ8G3g38B7grIh4IbAlM+/rNrkFeCeG+cRoegINOifHhmnzpg2NT8pJaq7J8OzDwDuA53cfPxdYWPH1BeDMfje8ffvWxm0rjOQq9EGj19Z+8kzb7rCNQz9WDfOI2A38R2beHRFXdp+eBpZWNJsCFvvd8IEDT7K4uNSz3fz8HPv3T/aFU+PWh3HY8XRyTfaTUfz+2tg/x+24GNR69WN6emrVQXCvkfllwI6IuB94NrCVTpDvWNHmDODxNdYpSVqDVS9NzMxfzMyfyMyzgT8AbsvM3wCejogLus2uAO4acZ2SpFUMep35G4D3RsQjdEbr7x9eSZKkfjW+Pi0zb6Fz5QqZ+QBw/mhKkiT1yztAJakAw1ySCjDMJakAw1ySCnA9c2lMuG6N1sIwl8aE69ZoLZxmkaQCDHNJKsBplkLaXKdcUrs88gvp54N+JdXiNIskFWCYS1IBhrkkFWCYS1IBhrkkFeDVLNIzXNNlBJ4+fIyDTzy1DhVpEIa59AzXzzICk//xy3U5zSJJBRjmklSAYS5JBRjmklSAYS5JBRjmklSAYS5JBRjmklSAYS5JBRjmklSAt/NPAD8OTlIvJsQE8OPgJPXiNIskFWCYS1IBTrNIaqTpuufg2udtMMwlNdJ03XNw7fM2OM0iSQUY5pJUgGEuSQUY5pJUQKMToBFxHXBp9+Edmfm2iNgJ3ARsAT6ZmXtHVKMkqYeeI/NuaF8EnAOcDZwbEb8GfAzYBbwMOC8iXjvKQiVJp9ZkmmUB2JOZRzLzKPBN4KXAo5n5WGYeA/YBl4ywTknSKnpOs2TmQ8v/j4iX0Jlu+TM6Ib9sAThz6NVJkhppfNNQRLwcuAN4K3CMzuh82RSw2M+Gt2/f2rht07vOxlmFPkj9aLLPVzkuxqEfTU+AXgB8BnhLZn4iIl4N7FjR5Azg8X42fODAkywuLvVsNz8/x/79k30v2Vr7MA47itSvXvt8hWMb1q8f09NTqw6Ce4Z5RDwf+CxwWWZ+sfv01zpfihcDjwGX0zkhKklqQZOR+bXALHBTRCw/9yHgSjqj9VngTuDTI6hPktRAkxOg1wDXnOLLZw23HEnSILwDVJIKMMwlqQDDXJIKMMwlqQDDXJIKMMwlqQDDXJIKMMwlqQDDXJIKaLxqopqb27aF2Zn//9KebLGspw8f4+ATT61XWdK6OXL0eKMF4ua2bfEYGBLDfARmZzZy8Z5be7a7/cZdTP6acdIP2rxpg8fAOnOaRZIKMMwlqQDDXJIKMMwlqQDDXJIKMMwlqQDDXJIKMMwlqQDDXJIK8A5QSa1petu/S1/0ZphLao23/Q+P0yySVIAj8xY1fYspSb0Y5i3q5y2mJK3GMO/DydYpl6RxYDL1oZ91yiVpPRnmOOKWNPlMMBxxS5p8XpooSQUY5pJUgGEuSQUY5pJUgGEuSQWUvZrFyw2lOlxdsbeyadf0ckPwkkNp3Lm6Ym9Os0hSARM3Mnf6RNKpDHs6pmnezM/PtT7Fs6ZUjIjLgb3AJuB9mfmBoVS1Cu/WlHQqw56O6Xe6ts0pnoHDPCKeB7wbOBc4DNwbEV/KzIeHVZwkjcIoPkug7ZO0axmZ7wS+mJn/DRARnwZ+FXhXj+/bADA9PdV4Qye2fc7pWxp9X9N2o/iZVdq1ue1xb9fmtse9XZvbbtJu86YN/OYffb5nu5v3XtR4u/38zEN95N+yFTm44WRfn1paWur7hwJExNuB0zJzb/fxbuD8zHxzj299FfDVgTYqSboQuOfEJ9cyMp8GVv4lmAIWG3zf17vFLADH17B9SXom2QDsoJOhP2AtYf5tOqG87Azg8Qbfd5iT/FWRJPX0b6f6wlrC/AvA9RExDxwCfgXoNcUiSRqBgW8ayszvAO8AvgTcD/x1Zv7jsAqTJDU38AlQSdL48HZ+SSrAMJekAgxzSSrAMJekAsZ++cGIuBB4H7AZeAx4Y2Z+t92q+hcRFwDvpdOPA8CbMvNb7VY1mIj4Q+B4Zl7fdi39aGNhuFGIiG3AvcAvZ+a/t1xO3yLiOuDS7sM7MvNtbdYzqIh4F50lTJaAmzPzpjbrmYSR+V8AV2TmK4CHgbe2XM+g/grYnZlnd////pbr6VtE/FBE3AzsabuWfq1YGO5VwNnAmyPix9utqn8R8dN0brp7adu1DCIidgIXAefQ+T2cGxGvb7eq/kXEq4FfAF4J/BTwOxERbdY0CWH+ssx8OCI2Ac8DJnFUPgPszcwHu089CLygxZIGtQt4FLix7UIG8P2F4TLzELC8MNykuQr4bZrdbT2OFoA9mXkkM48C32QCj4XM/DLw85l5DHgOnVmOQ23WNPbTLJl5NCJeQeeO06PA77VcUt8y8zCwDyAipoHrgc+2WdMgMvMvASLi+pZLGcRz6QTJsgXg/JZqGVhm7gZoeRA4sMx8aPn/EfESOtMtF7RX0eC62fRO4FrgU8B32qxnbMI8Ii6hM6e80iOZuTMz/wX40Yj4LeCTwM+se4ENrdaPiNgMfJzO6/6edS+uodX60EY9QzLownAagYh4OXAH8NbMfLTtegaVmddFxA3A7XTeNX2krVrGJswz81N0/rp9X0TMRsTrMnN5FLuPMX+Lf7J+AETEVuA2Oic/d3XfYo6lU/Vhwg26MJyGrHsxwGeAt2TmJ9quZxAR8WPAbGben5nfi4i/pTN/3ppxnzM/CnwgIs7tPr6UyV1xcR/wr8Bl3WkXra8vAK+JiPmIeBadheH+vuWannEi4vl0phgvn9Qg73oR8OcRMdN9x72LlrNprMM8M48DlwEfiYj76Zyw2t1uVf2LiHPo/LIvAP4pIu6PiDtbLusZxYXhxsa1wCxwU/c4uD8irm67qH5l5p10pon+GfgGcG/bf5xcaEuSChjrkbkkqRnDXJIKMMwlqQDDXJIKMMwlqQDDXJIKMMwlqQDDXJIK+D/5qAPKQha7WwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def sample_bernoulli(p):\n",
    "    return int(np.random.rand() < p)\n",
    "\n",
    "def generate_bernoulli_sample(p, sample_size):\n",
    "    \n",
    "    return [sample_bernoulli(p) for _ in range(sample_size)]\n",
    "\n",
    "def sample_sn(p, n):\n",
    "    \n",
    "    return np.mean(generate_bernoulli_sample(p, n))\n",
    "\n",
    "def scale(x, n, mu, sigma):\n",
    "    \n",
    "    return np.sqrt(n) * (x - mu) / sigma\n",
    "\n",
    "# plotting sample mean distribution for different sample sizes.\n",
    "p = 0.8\n",
    "for n in [5, 10, 50, 500]:\n",
    "    plt.hist([scale(sample_sn(p, n=n), n, p, np.sqrt(p * (1 - p))) for _ in range(1000)], bins=30)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's perform the same visualization experiment with Exponential random variables.\n",
    "\n",
    "Recall that exponential random variable is a continuous random variable with PDF:\n",
    "\n",
    "$f(x;\\lambda) = \\begin{cases}\n",
    "\\lambda e^{-\\lambda x} & x \\ge 0, \\\\\n",
    "0 & x < 0.\n",
    "\\end{cases}$\n",
    "\n",
    "$\\operatorname{E}[X]=\\frac{1}{\\lambda}$<br />\n",
    "$\\operatorname{Var}[X]=\\frac{1}{\\lambda^2}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQnklEQVR4nO3df4hlZ33H8ffM/pgZd2fRrFOyaTRFYr5YS7JRkv4Ro4KrEmi6SruGRsWtJBJaIZZEaTBppKCoNYlEigppusJaDSao+eFCMBajSFKxJELifklLDKYZcZkGM6vZnzP9496xk3V355x77p177jPvFyzMPfPcc77P3jOfOfOc5z53bHFxEUlSmcaHXYAkaXAMeUkqmCEvSQUz5CWpYIa8JBVs/bALWGYCuAiYBY4PuRZJGhXrgG3Aj4HDJ36zTSF/EfCDYRchSSPqUuCHJ25sU8jPAjz//G9YWKg+d3/r1s3MzR0cWFGrzf60m/1pt7XYn/HxMV7xik3QzdATtSnkjwMsLCzWCvml55TE/rSb/Wm3Ndyfkw5ze+NVkgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCtWmevIZsessUkxPVTolDh48x/8KLA65IUlOVfqIjYgvwI+DPMvPnEbEDuBWYAu7KzBu77bYDdwBbgIeBazLz2EAqV99NTqzn8uu+XantfbfsZH7A9UhqbsXhmoj4UzrrIZzXfTwF3AnsBF4HXBQRl3Wb7wU+nJnnAWPA1YMoWpJUTZUx+auBvwWe6z6+GHgqM5/uXqXvBXZFxDnAVGY+0m23B9jV53olSTWsOFyTmVcBRMTSprN46UI4s8DZp9ley9atm+s+hZmZ6drPabNR6U/VOkelP1XZn3azPy/Vy43XcWD5ijljwMJpttcyN3ew1gJDMzPTHDhQzujwMPtT92SqUqevT7vZn3ar0p/x8bHTXhz3MoXyWToL1C85k85Qzqm2S5KGpJeQfxSIiDg3ItYBVwL7MvMZ4FBEXNJt935gX5/qlCT1oHbIZ+YhYDdwD/AksB+4u/vt9wK3RcR+YDNwe3/KlCT1ovKYfGb+0bKvHwIuOEmbx+nMvpEktYDLGkhSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQXzg7zVkyNHj1daf/7I0eOrUI2kUzHk1ZONG9ZV+tDv+27ZuQrVSDoVh2skqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsFcaniETW+ZYnJi5Zfw8JHjTGxctwoVSWobQ36ETU6sr7ymu2u/S2uTwzWSVDBDXpIK1mi4JiLeB9zQfbgvM6+PiO3AHcAW4GHgmsw81qxMSVIver6Sj4iXAbcDbwEuAC6NiB3AXuDDmXkeMAZc3Y9CJUn1NRmuWdd9/iZgQ/ffUWAqMx/pttkD7GpSoCSpdz2HfGbOAzcB+4FngZ8DR4DZZc1mgbMb1CdJaqDnMfmIOB/4IHAO8Gs6wzTvABaXNRsDFursd+vWzbVrmZmZrv2cNrM/7WZ/2s3+vFSTG6/vBB7KzF8BRMQe4Hpg27I2ZwLP1dnp3NxBFhYWV27YNTMzzYED83UO0Wp1+jMqJ/NafX1Ggf1ptyr9GR8fO+3FcZMx+ceBHRGxKSLGgMuB7wOHIuKSbpv3A/saHEOS1ECTMfkHga8BPwF+SufG66eB9wK3RcR+YDOdGTiSpCFoNE8+Mz8DfOaEzY8DFzfZrySpP3zHqyQVzAXKNFBHjh6vdIP40OFjzL/w4ipUJK0thrwGauOGdZVXwCxnToTUHg7XSFLBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgrmm6E0Uqa3TDE5sfJp6ztopQ5DXiNlcmK976CVanC4RpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMKZQtVHUuuCStxCRpoTpzwSXpdByukaSCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCtZo7ZqIuBy4GdgEPJiZ10bEDuBWYAq4KzNvbF6mJKkXPV/JR8RrgC8B7wLOB94QEZcBdwI7gdcBF3W3SZKGoMlwzbvpXKk/m5lHgSuA3wJPZebTmXkM2Avs6kOdkqQeNBmuORc4EhH3Aq8G7geeAGaXtZkFzq6z061bN9cuZGZmuvZz1D79fh0HdV6Udr7Zn3Zr2p8mIb8eeDPwVuAgcC/wIrC4rM0YsFBnp3NzB1lYWFy5YdfMzDQHDszXOUSrlXaC1lHldazz/zOI86LE883+tFeV/oyPj5324rhJyP8S+G5mHgCIiG/SGZo5vqzNmcBzDY4hSWqgScjfD3wlIl4OzAOXAXcDfx8R5wJPA1fSuRErSRqCnm+8ZuajwGeBHwJPAs8AXwR2A/d0t+2nE/ySpCFoNE8+M+/k96/UHwIuaLJfSVJ/+I5XSSpYoyt51TO9ZYrJCf/LJa0eE2cVTU6s5/Lrvr1iu/tu2bkK1UhaCxyukaSCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFc6lhtcKRo8eZmZkedhlScQx5tcLGDetca18aAIdrJKlghrwkFcyQl6SCGfKSVDBvvKpIVWfrHDp8jPkXXlyFiqThMORVpDqzdeZXoR5pWByukaSCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQVrPIUyIj4HvDIzd0fEduAOYAvwMHBNZh5reoy2m94yxeSEs1EltU+jZIqItwEfAB7obtoLXJWZj0TEvwBXA19sVmL7TU6sdwVFSa3U83BNRJwBfBL4VPfxOcBUZj7SbbIH2NW0QElS75qMyX8Z+DjwfPfxWcDssu/PAmc32L8kqaGehmsi4irgF5n5UETs7m4eBxaXNRsDFurue+vWzbXr8ROF1ETd86e0883+tFvT/vQ6Jn8FsC0iHgPOADbTCfhty9qcCTxXd8dzcwdZWFhcuWHXzMw0Bw4Md/WR0k6qtabO+dOG862f7E+7VenP+PjYaS+Oexquycy3Z+afZOZ24B+AezPzr4FDEXFJt9n7gX297F+S1B/9nif/XuC2iNhP5+r+9j7vX5JUQ+PJ3Zm5h85MGjLzceDipvuUJPWH73iVpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIK5geTak07cvR4paWiDx0+xvwLL65CRVJ/GfJa0zZuWFf583nLWaVca4nDNZJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalgTqE8jektU0xO+F+kwah6fjlHX02YYKcxObG+8hxqqa4655dz9NUrh2skqWCGvCQVzJCXpII5Ji9VsHwhs5UWNPNGqdrEkJcqqLqQGXijVO3icI0kFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqWKMplBFxM/Ce7sMHMvNjEbEDuBWYAu7KzBsb1ihJ6lHPV/LdMH8HcCGwHXhjRPwVcCewE3gdcFFEXNaPQiVJ9TW5kp8FrsvMIwAR8TPgPOCpzHy6u20vsAvY17TQfnH5YElrSc9pl5lPLH0dEa+lM2zzBTrhv2QWOLvn6gag6vKu4BLCkkZf40vaiHg98ADwUeAYnav5JWPAQp39bd26uXYNK60lIq22fp+TgzzHS/v5sT8v1fTG6yXAPcBHMvPrEfEWYNuyJmcCz9XZ59zcQRYWFiu3n5mZ5sCB6iuFlHYCqJ2qnJN1zsU653gddX9+2m4t9md8fOy0F8c9h3xEvAr4FnBFZn6vu/nRzrfiXOBp4Eo6N2KlNWP5ipXSsDW5kr8emARujYilbV8CdtO5up8EvgPc3eAY0sipumKl93y0GprceL0WuPYU376g1/1KkvrHd7xKUsEMeUkqmCEvSQUz5CWpYIa8JBXMRVyklqs67/7Q4WPMv/DiKlSkUWLISy1XZ959Oe/1VL84XCNJBTPkJalghrwkFcyQl6SCGfKSVDBDXpIK5hRKaY058XOOTzUH33n3ZTDkpTWm6uccO+++DIa8VAg/kUonY8hLhfATqXQy3niVpIIZ8pJUMENekgpmyEtSwbzxKumkqs7WOXzkOBMb11Xap3PvV58hL+mk6szWqdJuqa1z71eXwzWSVDBDXpIK5nCNpOKduF7PqZR4z8CQl1S8tbxeTzEhX/U3taT2q/rzfLKZPa7f81LFpGKd39SS2q3Oz7M/96fnjVdJKpghL0kFK2a4RlL7lbLmfZ17gFXfETyomT0DCfmIuBK4EdgAfD4z/3kQx5E0WkpZ877qPQOod99gEDN7+j5cExF/CHwSeBOwHfhQRPxxv48jSVrZIK7kdwDfy8z/BYiIu4G/BP5xheetAxgfH6t9wKXn/MErpiq1r9puEPsspd0wj932dsM8dtvbDfPYVdtVzaBB9Plkx16pnmXfP+mY0Nji4mKlg1cVETcAmzLzxu7jq4CLM/NDKzz1TcAP+lqMJK0dlwI/PHHjIK7kx4HlvznGgIUKz/sxnSJngeMDqEuSSrQO2EYnQ3/PIEL+WTphveRM4LkKzzvMSX4LSZJW9N+n+sYgQv67wCciYgb4DfAXwEpDNZKkAej77JrM/B/g48C/A48B/5aZ/9Hv40iSVtb3G6+SpPZwWQNJKpghL0kFM+QlqWCGvCQVrJhVKCPiQuCRzJwYdi1NRMQlwG3ARmAO+GBmPjPcquorbZG6iLgZeE/34QOZ+bFh1tMvEfE54JWZuXvYtTQVEZcDNwObgAcz89ohl9RIRLwPuKH7cF9mXt/Lfoq4ko+IlwFfoBOMo+6rwFWZub379e1Drqe20hapi4gdwDuAC+n0540R8e7hVtVcRLwN+MCw6+iHiHgN8CXgXcD5wBsi4rLhVtW7bqbdDrwFuAC4tHse1lZEyAO3AJ8fdhFNRcQEcGNm/rS76afAq4dYUq9+t0hdZv4GWFqkblTNAtdl5pHMPAr8jNF8XX4nIs6g84v4U8OupU/eDdyVmc92X6MrgEeHXFMT6+jk8yY6fw1vAHpabH7kh2si4s+Bl2Xm3REx7HIayczDwF6AiBgHPgF8a5g19egsOsG4ZBa4eEi1NJaZTyx9HRGvpTNsc8nwKuqLL9N50+Krhl1In5wLHImIe+n8Ar4fuGm4JfUuM+cj4iZgP/Bb4PvAj3rZ18iEfETsojNWvdx+YAudK8eRcqr+ZOaOiNgIfIXO6zOKV1q9LlLXahHxeuAB4KOZ+dSw6+lVd2XYX2TmQxGxe9j19Ml64M3AW4GDwL10hqL2DK+k3kXE+cAHgXOAX9O5+Lse+Ke6+xqZkM/MbwDfWL6te7LeADy8dBUfEY8Bl2bmID5kpW9O1h+AiNhM5wSdA3Z2//QcNb0uUtda3Rvi9wAfycyvD7uehq4AtnV/Vs4ANkfEbZn5d0Ouq4lfAt/NzAMAEfFNOn897hlmUQ28E3goM38FEBF7gL+h5JA/mcy8A7hj6XFELHZvWI6yvcB/Addk5qhe/Ra1SF1EvIrOsNkVmfm9YdfTVGa+fenr7pX8W0c84KEzPPOViHg5MA9cxmgOdS55HPhsRGyiM1xzOadYSnglpdx4LUJ3GuhOOuO9/xkRj0XEd4ZcVm0FLlJ3PTAJ3Np9TR6LiGuGXZT+X2Y+CnyWznLlTwLPAP861KIayMwHga8BP6EzAWMD8Ole9uUCZZJUMK/kJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQX7P/KCUH8s58FFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPiklEQVR4nO3dbYxc5XnG8f++2LvGXrdgtsIUEikh3EoTgQmCViJOkOJS8cF1SEusQiJcShCKkEgLSYtiStQ2UYliiIiqgkQoqLQJEqi8BokGoiYogkZFgArhFqoIDWWjuNuorAn22t7th5mNto49c2Z3Z8/MM/+fhOSZfcbn0jBz7ePnvA3Nz88jSepvw3UHkCQtn2UuSQWwzCWpAJa5JBXAMpekAozWsM0x4BxgCjhcw/YlqR+NAJuBHwAHjvxhHWV+DvC9GrYrSSXYCjx15JN1lPkUwM9+9hZzc/Ue475p0wamp/fVmmEp+jF3P2aG/szdj5nB3O0MDw9x/PHrodmhR6qjzA8DzM3N117mCzn6UT/m7sfM0J+5+zEzmLuioy5PuwNUkgpgmUtSASxzSSqAZS5JBbDMJakAlrkkFcAyl6QC1HGcubRiJjauY3ys/cd4/4FDzLz59iokkuphmauvjY+Nsv3aB9uOe3jPDmZWIY9UF5dZJKkAlrkkFcAyl6QCWOaSVADLXJIK4NEs0iJHO9RxcnLiqGM93FG9xDKXFql6qCN4uKN6i8ssklQAy1ySCmCZS1IBLHNJKoA7QKUlmj14+JhHuizmUS9aDZa5tERr14x4kS/1DJdZJKkAlrkkFcAyl6QCuGaugVB1Z6XUryxzDYROdlZK/chlFkkqgGUuSQWwzCWpAJa5JBXAHaBSl3nav1aDZS51maf9azW4zCJJBag0M4+ITwDXNx8+lpnXRcQW4A5gI/Bd4KrMPNSdmJKkVtrOzCPiOOBW4MPAmcDWiNgG3ANcnZmnA0PAp7oZVJJ0bFWWWUaa49YDa5r/HQTWZebTzTF3ARd3I6Akqb22ZZ6ZM8ANwMvA68CPgFlgatGwKeCULuSTJFXQds08Is4ALgfeCfwvjeWVC4D5RcOGgLlONrxp04ZOhndNv158qR9z92Pm1bZS71G/vtfmXroqO0B/B3giM38KEBF3AdcBmxeNOQl4o5MNT0/vY25uvv3ALpqcnGDv3v47GKwfc3crcy98iVbSSrxH/fj5AHO3Mzw81HISXGXN/HlgW0Ssj4ghYDvwL8D+iDivOeaTwGPLDStJWpoqa+aPA98A/g14gcYO0L8GLgVuiYiXgQ00jniRJNWg0nHmmXkTcNMRTz8PnLviiSRJHfMMUEkqgGUuSQWwzCWpAJa5JBXAMpekAljmklQAb06hnjSxcR3jY348par8tqgnjY+NVr47jySXWSSpCJa5JBXAMpekAljmklQAy1ySCmCZS1IBLHNJKoBlLkkFsMwlqQCWuSQVwDKXpAJY5pJUAMtckgpgmUtSASxzSSqAZS5JBbDMJakAlrkkFcAyl6QCWOaSVADLXJIKYJlLUgEsc0kqgGUuSQWwzCWpAKN1B1AZJjauY3ys9cdpcnKC/QcOMfPm26uUShoclrlWxPjYKNuvfbDtuIf37GBmFfJIg6ZSmUfEduBGYD3weGZeExHbgJuBdcC9mbm7ezElSa20XTOPiHcBtwEfBc4APhARFwJ3AjuA9wLnNJ+TJNWgyg7Qi2jMvF/PzIPATuDnwCuZ+WpmHgLuAS7uYk5JUgtVlllOA2Yj4iHgHcAjwIvA1KIxU8ApKx9PklRFlTIfBT4EnA/sAx4C3gbmF40ZAuY62fCmTRs6Gd41k5MTdUdYkn7NDf2dvZtmDx6u9N7MHjzM2jUjLcf063ts7qWrUuY/Ab6dmXsBIuKfaCypHF405iTgjU42PD29j7m5+fYDu2hycoK9e/vv2IpezN3Jh7lK9l74cqy2tWtGKh8R1Oo97MXPRxXmbm14eKjlJLhKmT8C3B0RvwrMABcC9wF/FhGnAa8Cl9DYISpJqkHbHaCZ+QzwZeAp4CXgNeBvgV3A/c3nXqZR8JKkGlQ6zjwz7+SXZ95PAGeueCJJUsc8A1SrqupOPkmdscy1qjrZySepOq+aKEkFsMwlqQCWuSQVwDKXpAK4A1TqM1WOCPJGIIPHMpf6TCdHBPXfyfFaKpdZJKkAlrkkFcAyl6QCWOaSVADLXJIKYJlLUgEsc0kqgGUuSQWwzCWpAJa5JBXAMpekAljmklQAy1ySCmCZS1IBvASuVKgq1z0HvO55ISxzqVBe93ywuMwiSQWwzCWpAJa5JBXAMpekAljmklQAy1ySCmCZS1IBLHNJKoBlLkkFsMwlqQCWuSQVwGuzqKWJjesYH/NjIvW6yt/SiPgKcGJm7oqILcAdwEbgu8BVmXmoSxlVo/Gx0coXa5JUn0rLLBHxEeCyRU/dA1ydmacDQ8CnupBNklRR2zKPiBOALwJfaj5+J7AuM59uDrkLuLhbASVJ7VVZZrkd+DxwavPxycDUop9PAad0uuFNmzZ0+pKuqHLx/l7Ur7nVm3rl89QrOTrVC7lblnlEXAH8ODOfiIhdzaeHgflFw4aAuU43PD29j7m5+fYDu2hycoK9e/vvsvyrmbsXPqTqvl74Hvh9bG14eKjlJLjdzHwnsDkingNOADbQKPLNi8acBLyxzJySpGVouWaemb+dme/PzC3AnwMPZeYfAvsj4rzmsE8Cj3U5pySphaWeNHQpcEtEvExjtn7rykWSJHWq8nHmmXkXjSNXyMzngXO7E0mS1ClP55ekAljmklQAy1ySCmCZS1IBLHNJKoBlLkkFsMwlqQCWuSQVwDKXpAJY5pJUAMtckgpgmUtSAbztujTgZg8ernQTkv0HDjHz5turkEhLYZlLA27tmhG2X/tg23EP79lB/90HaHC4zCJJBbDMJakAlrkkFcAyl6QCWOaSVADLXJIKYJlLUgEsc0kqgGUuSQWwzCWpAJa5JBXAMpekAljmklQAy1ySCmCZS1IBvJ75AJrYuI7xMf/XSyXxGz2AxsdGK92MABo3JJDU+1xmkaQCWOaSVADLXJIKYJlLUgEq7QCNiBuBjzcfPpqZn4uIbcDNwDrg3szc3aWMkqQ22s7Mm6V9AXAWsAU4OyL+ALgT2AG8FzgnIi7sZlBJ0rFVWWaZAq7NzNnMPAj8EDgdeCUzX83MQ8A9wMVdzClJaqHtMktmvrjw54h4D43llq/RKPkFU8ApnWx406YNnQzvmsnJibojLEm/5lb/mj14uPLnbvbgYdauGel4G/36ue6F3JVPGoqI9wGPAp8FDtGYnS8YAuY62fD09D7m5uY7ecmKm5ycYO/emVozLMVyc/fCB0/9Z+2akY5ONuv0Mzqo38eqhoeHWk6Cq+4APQ+4H/hMZn4zIj4MbF405CTgjeUElVSOqrP4/QcOMfPm26uQqHxtyzwiTgUeAHZm5pPNp59p/ChOA14FLqGxQ1SSKs/iH96zg/6bi/emKjPz64Bx4OaIWHjuNmAXjdn6OPAt4L4u5JMkVVBlB+g1wDXH+PGZKxtHkrQUngEqSQWwzCWpAJa5JBXAMpekAljmklQAy1ySCmCZS1IBvKFzH5jYuI7xsf//v+pop0p7arQ0uCzzPjA+Nuqp0ZJacplFkgrgzLwgnVxvWlJZLPOCdHKlOkllcZlFkgpgmUtSASxzSSqAZS5JBbDMJakAlrkkFcAyl6QCWOaSVADLXJIKYJlLUgEsc0kqgGUuSQWwzCWpAF41sUZHu4OQNEiOvGzzsS7hfGD2MGNrR9r+fYN8ty2bpEad3EFIKlEnl232blutWeaSilH1Bi0lzuAtc0nF6GSmX9oM3jLvAtfCJa02G6cLXAuXtNo8NFGSCmCZS1IBLHNJKoBr5lTfYVn1xAVJWm3LKvOIuATYDawBvpqZf7MiqVZZJzss3bEpqRctucwj4teBLwJnAweA70fEdzLzpZUKdzRVZ9ElnhQgaWVUPbmo6r/GJycnar/kwHJm5tuAJzPzfwAi4j7g94G/aPO6EYDh4aElbXR8bJQ/+qvH2477+u4LeKvCNhZy/Nrx6yptf9DG1bntXh9X57Z7fVyd264ybu2akco9UmVcJ2OrdtORFnXmUX9jDM3Pz3f8lwJExPXA+szc3Xx8BXBuZl7Z5qUfBL63pI1KkrYCTx355HJm5sPA4t8EQ8Bchdf9oBlmCji8jO1L0iAZATbT6NBfspwyf51GKS84CXijwusOcJTfKpKktv7jWD9YTpl/G/hCREwCbwG/B7RbYpEkdcGSTxrKzP8CPg98B3gO+MfM/NeVCiZJqm7JO0AlSb3D0/klqQCWuSQVwDKXpAJY5pJUgIG+amJEbAbuAE4Gfg5cmpk/qjVUByLiLODpzByrO0sVEXEecAuwFpgGLs/M1+pNdXT9ehG5iLgR+Hjz4aOZ+bk683QiIr4CnJiZu+rOUkVEbAduBNYDj2fmNXXmGfSZ+d8DD2fmWc0/31Rznsoi4jjgazSKsV/8A3BFZm5p/vnWmvMc1aKLyH0Q2AJcGRG/UW+q9iJiG3ABcBaN3GdHxEX1pqomIj4CXFZ3jqoi4l3AbcBHgTOAD0TEhXVmGtgyj4gTgTOB25tP/R2NmVi/2AN8te4QVUXEGLA7M19oPvUC8I4aI7Xyi4vIZeZbwMJF5HrdFHBtZs5m5kHgh/Tue/wLEXECjV+eX6o7SwcuAu7NzNeb7/VO4Jk6Aw3yMsu7gf8E9kTEVuAnwNX1RqomIn4XOC4z74uIuuNUkpkHgHsAImIY+ALwQJ2ZWjiZRjEumALOrSlLZZn54sKfI+I9NJZbzqsvUWW30zgB8dS6g3TgNGA2Ih6i8QvzEeCGOgMNRJlHxMU01moXe4XGP0dvzMw/aV718W7g/FWOd0zHyP0ysJHG7LEnHSt3Zm6LiLU03udRencmttSLyPWEiHgf8Cjw2cx8pe48rTS/dz/OzCciYlfdeTowCnyIRl/sAx6isUx0V12BBvYM0Ih4N/BsZv5K8/FxwH9n5nH1Jmut+eG/HphpPnUm8DywNTNnjvnCHhARG2h86KeBTzRn6z0nIi6j8X5e0Xx8AzCUme2u1V+75k7m+4HPZOY3687TTkT8M40rAR4CTgA2AHdn5h/XGqyNiPhLYOPCTs+I+DTw/sz8dF2ZBrbMASLiReC6zHwsInYCV2fm1nav6yURMZ+ZS7vTxyqLiAeAnwJXZWbPznSbO0CforG08hbwfeDKXr/2UEScCjwL7MzMJ+vO06nmzPz8fjiaJSJ+k8a/MH+LxsTqAeCBzPx6XZkGdgdo08eAP42IfweuAS6vOU+xmodR7qCxhvtsRDwXEd+qOdZR9fFF5K4DxoGbm+/vcxFxVd2hSpSZzwBfpvFL/yXgNRoHUdRmoGfmklSKQZ+ZS1IRLHNJKoBlLkkFsMwlqQCWuSQVwDKXpAJY5pJUAMtckgrwfxlBAzXk/jUeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPj0lEQVR4nO3df4wc5X3H8ffd2b4z9rkFcxWmkEgJyVdpImyCoJWIE6S4VPxBHdoSq5AIlxKEIiTSQtKimJL+SFSiABFRFZCAgErbIIFK+REkGoiaoAgaFQEqhK9QRWkoF8W9RsUG7LN91z92Lzph7nb2bvdm9rn3S7LknX125zu3s5977pmZZ4ZmZ2eRJA224boLkCQtn2EuSQUwzCWpAIa5JBXAMJekAqypYZ2jwBnAJHCkhvVL0iAaAbYAPwIOvv3JOsL8DOAHNaxXkkqwHXji7QvrCPNJgJ///A1mZo4+x33z5o1MTe1f8aJ6ze1oFrejWdyO7g0PD3HssRugnaFvV0eYHwGYmZl9xzCfe64EbkezuB3N4nYs2TsOT3sAVJIKYJhLUgEMc0kqgGEuSQUwzCWpAIa5JBXAMJekAtRxnrm04qYPHWFiYrxjuwMHD7Pv9bdWoCKptwxzrQrr1o5w3lX/1LHdgzfsZN8K1CP1msMsklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQVwDCXpAIY5pJUAMNckgpgmEtSAQxzSSqAYS5JBTDMJakAhrkkFcAwl6QCeHMKNdL4pvWMjXbePb0zkNRimKuRxkbXeGcgqQsOs0hSAQxzSSqAwywaaNOHjjAxMV53GVLtDHMNtHVrRyqPrUslc5hFkgpgmEtSASoNs0TEp4Br2g8fycyrI2IbcBuwCfg+cHlmHu5PmZKkxXTsmUfEMcDNwMeArcD2iNgB3A1ckZnvB4aAz/SzUEnSwqoMs4y0220A1rb/HQLWZ+aT7TZ3Ahf0o0BJUmcdh1kyc19EXAu8CLwJ/AswDUzOazYJnNTNijdv3rjgc6WcauZ2DKamb2/T66vK7eitjmEeEacClwDvBv6P1vDKOcDsvGZDwEw3K56a2s/MzOxRyycmxtm7d/Av0HY7lr/eujT5c3O/apaV3I7h4aFFO8FVhll+C3gsM3+WmQdpDamcDWyZ1+YE4LWllylJWo4qYf4ssCMiNkTEEHAeraGWAxFxVrvNp4FH+lSjJKmDjmGemY8C/wD8G/AcrQOgfw1cBNwUES8CG2md8SJJqkGl88wz83rg+rctfhY4s+cVSZK65hWgklQAw1ySCmCYS1IBnAJXK6rqvT0ldcdvlVZUN/f2lFSdwyySVADDXJIKYJhLUgEMc0kqgGEuSQUwzCWpAIa5JBXA88zVE14MJNXLb596wouBpHoZ5lKfVf2r5cDBw+x7/a0VqEglMsylPuvmr5bBvyum6uIBUEkqgGEuSQUwzCWpAI6ZS/NMHzrCxMR4pbYesFSTGObSPOvWjlQ6WAkesFSzOMwiSQUwzCWpAIa5JBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQVwDCXpAIY5pJUAMNckgpQaQrciDgPuA7YADyamVdGxA7gRmA9cE9m7ulfmZKkxXTsmUfEe4BbgE8ApwIfjohzgTuAncAHgDPayyRJNagyzHI+rZ73q5l5CNgFvAm8lJkvZ+Zh4G7ggj7WKUlaRJVhllOA6Yh4AHgX8BDwPDA5r80kcFLvy5MkVVElzNcAHwXOBvYDDwBvAbPz2gwBM92sePPmjQs+V/UejE1XwnZ0c0/M1ajXP5sq71fK5+F29FaVMP8p8N3M3AsQEf9Ia0jlyLw2JwCvdbPiqan9zMzMHrV8YmKcvXsH/86KJW1HlXtiPnjDzhWopnmqfMbdfNk7vV9J+5Xb0Z3h4aFFO8FVwvwh4K6I+GVgH3AucC/wpxFxCvAycCGtA6KSpBp0PACamU8BXwWeAF4AXgG+CewG7msve5FWwEuSalDpPPPMvIOje96PAVt7XpE0IDyeoCapFOaSjrZu7YjHE9QYXs4vSQUwzCWpAIa5JBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKoBhLkkF8ApQqSGqTg8wvmk9+15/awUq0iAxzKWG6GZ6gMGfPFa95jCLJBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQVwDCXpAIY5pJUAMNckgpgmEtSAQxzSSqAU+CuQuOb1jM26kcvlcRv9Co0Nrqm0rzZ0Jo7W1LzOcwiSQUwzCWpAIa5JBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKkDli4Yi4mvA8Zm5OyK2AbcBm4DvA5dn5uE+1ShJ6qBSmEfEx4GLgYfbi+4GLs3MJyPiduAzwDf7U6Kk+aYPHWFiYrxjuwMHD7Pv9bdWoCI1Qccwj4jjgC8DXwG2RsS7gfWZ+WS7yZ3An2OYSyti3dqRStMxPHjDTvatQD1qhio981uBLwIntx+fCEzOe34SOKnbFW/evHHB56r0OgZBKduhwdXkfbDJtXWjKduxaJhHxKXATzLzsYjY3V48DMzOazYEzHS74qmp/czMzB61fGJinL17B78/0eTtaMrOp/5r8j7Y1Nq6sZLbMTw8tGgnuFPPfBewJSKeAY4DNtIK8i3z2pwAvLbMOiVJy7DoqYmZ+ZuZ+aHM3Ab8GfBAZv4BcCAizmo3+zTwSJ/rlCQtYqnnmV8E3BQRL9Lqrd/cu5IkSd2qfJ55Zt5J68wVMvNZ4Mz+lCRJ6pZXgEpSAQxzSSqAYS5JBTDMJakAhrkkFcAwl6QCGOaSVADDXJIKYJhLUgEMc0kqgGEuSQUwzCWpAJUn2lLzjW9az9ioH6m0GvnNL8jY6JrK94aUVBaHWSSpAIa5JBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQVwDCXpAI4N4tUqOlDR5iYGO/Y7sDBw+x7/a0VqEj9ZJhLhVq3dqTyxGv7VqAe9ZfDLJJUAMNckgpgmEtSAQxzSSqAYS5JBfBslhpVvWenp46pn6qewnhw+gij60Yqvaf77MozzGvUzT07PXVM/dLNKYxV2s21dZ9dWQ6zSFIBKvXMI+I64JPthw9n5hciYgdwI7AeuCcz9/SpRklSBx175u3QPgc4DdgGnB4Rvw/cAewEPgCcERHn9rNQSdLCqgyzTAJXZeZ0Zh4Cfgy8H3gpM1/OzMPA3cAFfaxTkrSIjsMsmfn83P8j4n20hlu+QSvk50wCJ/W8OklSJZXPZomIDwIPA58HDtPqnc8ZAma6WfHmzRsXfK7KaVKDoJfbUcrPRKtHp322lH26KdtR9QDoWcB9wOcy89sR8TFgy7wmJwCvdbPiqan9zMzMHrV8YmKcvXsH/6SmKtvRzU5Q5WfSlJ1KgsX32dX0Pe+V4eGhRTvBHcM8Ik4G7gd2Zebj7cVPtZ6KU4CXgQtpHRCVJNWgSs/8amAMuDEi5pbdAuym1VsfA74D3NuH+iRJFVQ5AHolcOUCT2/tbTmSpKXwClBJKoBhLkkFMMwlqQCGuSQVwDCXpAIY5pJUAMNckgpgmEtSAbxt3ACoeo9GqSmq7LMTE+PeK7SHDPMB0M09GqUm6GafHfzptprBYRZJKoBhLkkFMMwlqQCGuSQVwDCXpAIY5pJUAMNckgrgeeZ9ML5pPeANliWtHMO8D8ZG13iRj6QV5TCLJBXAMJekAhjmklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQVwCtAJdWm6v1tD04fYXTdSMd2q/meooa5pNp0c69Q7ym6OMOc1sRYY6OdfxRVeweSmq3qd36QevqGOd1NjOUEWtLg6+Y7Pyg9fQ+ASlIBDHNJKoBhLkkFKHbMvOoBDkkqwbLSLiIuBPYAa4GvZ+bf9KSqHqh6gAM8YClp8C05zCPiV4EvA6cDB4EfRsT3MvOFXhX3TuxxS1pI1YuQevl+c8/XfWHTclJxB/B4Zv4vQETcC/we8BcdXjcCMDw8tGCDxZ4bG13DH/7Vox2Lu33POfzKses7tptTte1qa1fnupvers51N71dXetet3akp/lQ9f3m3rPqut9YJOMWMi8X3/E3xtDs7GzXbwoQEdcAGzJzT/vxpcCZmXlZh5d+BPjBklYqSdoOPPH2hcvpmQ8D838TDAEzFV73o3Yxk8CRZaxfklaTEWALrQw9ynLC/FVaoTznBOC1Cq87yDv8VpEkdfQfCz2xnDD/LvCliJgA3gB+F+g0xCJJ6oMlXzSUmf8NfBH4HvAM8PeZ+a+9KkySVN2SD4BKkprDy/klqQCGuSQVwDCXpAIY5pJUgMZNchIRW4DbgBOBN4GLMvM/ay1qGSLiNODJzBytu5aliIizgJuAdcAUcElmvlJvVdU1eTK4bkTEdcAn2w8fzswv1FnPckXE14DjM3N33bUsRUScB1wHbAAezcwray6pkT3zvwUezMzT2v+/vuZ6liwijgG+QSsIB9XfAZdm5rb2/2+uuZ7K5k0G9xFgG3BZRPxavVV1LyJ2AOcAp9HajtMj4vx6q1q6iPg4cHHddSxVRLwHuAX4BHAq8OGIOLfeqhoW5hFxPLAVuLW96Fu0elWD6gbg63UXsVQRMQrsyczn2oueA95VY0nd+sVkcJn5BjA3GdygmQSuyszpzDwE/JjB+hx+ISKOo/UL9it117IM5wP3ZOar7c9jF/BUzTU1bpjlvcB/ATdExHbgp8AV9Za0NBHx28AxmXlvRNRdzpJk5kHgboCIGAa+BNxfZ01dOpFWEM6ZBM6sqZYly8zn5/4fEe+jNdxyVn0VLcuttC42PLnuQpbhFGA6Ih6g9Uv1IeDaekuqMcwj4gJaY7HzvUTrT8nrMvOP2zMx3gWcvcLlVbbAdrwIbKLVMxwIC21HZu6IiHW0Poc1DFaPaqmTwTVSRHwQeBj4fGa+VHc93Wp/n3+SmY9FxO6661mGNcBHaeXSfuABWsNGd9ZXUsOuAI2I9wJPZ+YvtR8fA/xPZh5Tb2Xdae+01wD72ou2As8C2zNz34IvbKCI2EhrZ50CPtXurQ+EiLiY1s/80vbja4GhzOw0537jtA9E3wd8LjO/XXc9SxER/0xr1r/DwHHARuCuzPyjWgvrUkT8JbBp7qBnRHwW+FBmfrbOuhoV5gAR8TxwdWY+EhG7gCsyc3un1zVZRMxmZvez0TdARNwP/Ay4PDMHqlfbPgD6BK2hlTeAHwKXDdocQhFxMvA0sCszH6+7nl5o98zPHsSzWSLi12n9pfobtDps9wP3Z+btddbVqAOgbb8D/ElE/DtwJXBJzfWsWu3TKnfSGp99OiKeiYjv1FxWZQVNBnc1MAbc2P4MnomIy+suarXKzKeAr9LqKLwAvELrZI1aNa5nLknqXhN75pKkLhnmklQAw1ySCmCYS1IBDHNJKoBhLkkFMMwlqQCGuSQV4P8BbiABMNCJXRAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# same visualizations but not for Exponential random variable with parameter lambda=2 \n",
    "\n",
    "def generate_exponential_sample(l, sample_size):\n",
    "    dist = torch.distributions.Exponential(l)\n",
    "    return [dist.sample().item() for _ in range(sample_size)]\n",
    "\n",
    "def sample_sn(p, n):\n",
    "    \n",
    "    return np.mean(generate_exponential_sample(p, n))\n",
    "\n",
    "def scale(x, n, mu, sigma):\n",
    "    \n",
    "    return np.sqrt(n) * (x - mu) / sigma\n",
    "\n",
    "# plotting sample mean distribution for different sample sizes.\n",
    "for n in [5, 100, 500]:\n",
    "    plt.hist([scale(sample_sn(2, n=n), n, 1/2, 1/4) for _ in range(1000)], bins=30)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Application of CLT - Asymptotic Confidence Intervals\n",
    "\n",
    "In statistics, confidence intervals are used to measure the \"quality of estimation\".\n",
    "\n",
    "Let's say we have iid sample $X_{1}, X_{2}, ..., X_{n} \\sim Bernoulli(p)$ with some unknown $p$.\n",
    "\n",
    "We want to estimate unknown $p$ by using empirical average $\\overline{X}_n=\\sum_{k=1}^n X_k$<br /> (by the low of large numbers $\\overline{X}_n\\ \\xrightarrow{P}\\ \\mu \\qquad\\textrm{when}\\ n \\to \\infty$)\n",
    "\n",
    "After calculating the empirical average, we want to compute the <b>confidence interval of level $1-\\alpha$</b> for our estimator. Mathematically, we want to calculate interval $I$ from our data, so that $\\lim_{n\\to\\infty} P(\\overline{X}_n\\in I)= 1-\\alpha$\n",
    "\n",
    "(Note that, confidence interval $I$ is itself a random variable and is calculated solely from the data.)\n",
    "\n",
    "In this use-case, we will be using confidence interval centered around empirical average of the form $[\\overline{X}_n-s; \\overline{X}_n+s]$ with some $s>0$.\n",
    "\n",
    "We have,<br />\n",
    "$\\lim_{n\\to\\infty}P(p\\in [\\overline{X}_n-s; \\overline{X}_n+s])= 1-\\alpha$<br />\n",
    "$\\lim_{n\\to\\infty}P(\\overline{X}_n-s\\le p \\le \\overline{X}_n+s)= 1-\\alpha$<br />\n",
    "$\\lim_{n\\to\\infty}P(|\\overline{X}_n-p|\\le s)= 1-\\alpha$<br />\n",
    "$\\lim_{n\\to\\infty}P(\\frac{\\sqrt{n}}{\\sigma}|\\overline{X}_n-p|\\le \\frac{\\sqrt{n}}{\\sigma} s)= 1-\\alpha$<br />\n",
    "$\\lim_{n\\to\\infty}P(|Z|\\le \\frac{\\sqrt{n}}{\\sigma} s)= 1-\\alpha, \\qquad Z\\sim N(0, 1)$<br />\n",
    "\n",
    "Note that, $P(|Z|\\le t)=CDF(t)-CDF(-t)=CDF(t)-(1-CDF(t))=2CDF(t)-1=\\alpha$<br />\n",
    "$CDF(t)=1-\\frac{1}{\\alpha}$<br />\n",
    "$t = Q(1-\\frac{\\alpha}{2})=q_{\\frac{\\alpha}{2}}$, where $Q$ is quantile function of standard gaussian.<br />\n",
    "\n",
    "From above, if  $t=\\frac{\\sqrt{n}}{\\sigma} s$, $s=\\frac{\\sigma}{\\sqrt{n}} t=\\frac{\\sigma}{\\sqrt{n}}q_{\\frac{\\alpha}{2}}$<br />\n",
    "\n",
    "Finally, by substituting we get<br />\n",
    "$\\lim_{n\\to\\infty}P(p\\in [\\overline{X}_n-\\frac{\\sigma}{\\sqrt{n}}q_{\\frac{\\alpha}{2}}; \\overline{X}_n+\\frac{\\sigma}{\\sqrt{n}}q_{\\frac{\\alpha}{2}}])= 1-\\alpha$, where $\\sigma^2=p(1-p)\\approx \\overline{X}_n(1-\\overline{X}_n)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = generate_bernoulli_sample(0.5, sample_size=5000)\n",
    "x[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.40320208300821747, 0.5991979169917825]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def bern_conf_interval(x, n, alpha):\n",
    "    \"\"\"Calculate asymptotic confidence interval of level 1-alpha for Bernoulli parameter p.\"\"\"\n",
    "    \n",
    "    avg = np.mean(x)\n",
    "    \n",
    "    # can also use torch.distributions.Normal(0, 1).icdf()\n",
    "    q = norm.ppf(1 - alpha / 2)\n",
    "\n",
    "    s = np.sqrt(avg * (1 - avg) / n) * q\n",
    "    \n",
    "    return [avg - s, avg + s]\n",
    "\n",
    "I = bern_conf_interval(x, 100, 0.05)\n",
    "I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.958"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculating the proportion of cases when true parameter falls within our confidence interval \n",
    "\n",
    "p = 0.5 # true mean of the distribution\n",
    "alpha = 0.05 # confidence interval of level 95%\n",
    "\n",
    "count = 0\n",
    "for i in range(500):\n",
    "    x = generate_bernoulli_sample(p, sample_size=500)\n",
    "    I = bern_conf_interval(x, 500, alpha)\n",
    "    if I[0] <= p <= I[1]:\n",
    "        count += 1\n",
    "        \n",
    "count / 500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Regression Model (Theory)\n",
    "<br /><br />\n",
    "<b>Linear Regression</b> is one of the fundamental concepts in Machine Learning and Data Science. It is used when there is an assumption of <b>linear relationship</b> between predictors/features $X_i$ and a dependent variable (label) $Y_i$\n",
    "\n",
    "Examples:\n",
    "<ul>\n",
    "    <li>Predict human <b>height</b> $Y$ given features: <b>$X_1$ = Age</b>, <b>$X_2$ = Gender</b>, <b>$X_3$ = Father's height</b></li>\n",
    "    <li>Predict <b>house price</b> $Y$ given features: <b>$X_1$ = Area</b>, <b>$X_2$ = # of bedrooms</b>, <b>$X_3$ = location</b>, <b>$X_3$ = # year built</b></li>\n",
    "</ul>\n",
    "\n",
    "In the above cases, we assume that there are coefficients $\\alpha, \\beta, \\gamma$, such that $Y \\approx \\alpha * X_1 + \\beta * X_2 + \\gamma * X_3$. Our goal is to find these coefficients.\n",
    "\n",
    "<br /><br /><br />\n",
    "<b>Task Formulation:</b><br /><br />\n",
    "Lets, $(X_1, Y_1), (X_2, Y_2), ..., (X_{n}, Y_{n}) \\overset{iid}{\\sim} \\mathbb{R}^d\\times\\mathbb{R}$ and the relationship between $X_{i}$ and $Y_{i}$ is given by linearity (with random noise)\n",
    "$Y_{i}=\\theta^TX_{i}+\\epsilon_{i}$\n",
    "<br /><br />\n",
    "Lets also assume that $X_{i}\\sim\\mathcal{N}(0,\\,I_d)$ and $\\epsilon_i\\sim\\mathcal{N}(0,\\,1)$\n",
    "<br /><br />\n",
    "Together with these assumptions, the above linearity is called <b>linear regression model</b> parametrized by vector $\\theta\\in\\mathbb{R}^d$\n",
    "<br /><br />\n",
    "The above definitions imply that the conditional density of $Y$ is defined as follows: $P_\\theta(Y_{i}|x)=\\mathcal{N}(\\theta^Tx_{i}, 1)$\n",
    "<br /><br />\n",
    "Consequently, the data generating process can be thought as if we first sample $X$ from multivariate standard normal distribution and, afterwards, use above conditional density to sample corresponding $Y$\n",
    "<br /><br /><br />\n",
    "<b>The Maximum Likelihood Estimator for Linear Regression Model</b>\n",
    "\n",
    "Generally, the true parameter $\\theta$, that generated our observations, is unknown to us. Therefore, our goal is to find an estimate $\\hat{\\theta}$ from our data.\n",
    "\n",
    "Method of <b>maximum likelihood estimation</b> tries to find estimator $\\hat{\\theta}_{MLE}$ so that our observed data becomes most probable under the assumed statistical model (data generating process) if we used $\\hat{\\theta}_{MLE}$ as a parameter. \n",
    "\n",
    "The probability of our observed data under a particular $\\theta$ can be described as a function of $\\theta$: $L(\\theta)=\\prod_{i=1}^{n} P_\\theta(Y_{i}|x_i)$ \n",
    "\n",
    "Then, $\\hat{\\theta}_{MLE}=\\underset{\\theta\\in\\mathbb{R}^d}{\\mathrm{argmax}} L(\\theta)=\\underset{\\theta\\in\\mathbb{R}^d}{\\mathrm{argmax}} \\ln(L(\\theta))$<br />\n",
    "The latter is valid because $ln(x)$ is increazing function on its domain.\n",
    "<br /><br />\n",
    "\n",
    "$\\ln{L(\\theta)}=\\sum_{i=1}^{n}\\ln{\\mathcal{N}(\\theta^Tx_{i}, 1)}=\\sum_{i=1}^{n}\\ln{\\frac{1}{\\sqrt{2\\pi\\sigma^2} } exp( -\\frac{(y_i-\\theta^T x_i)^2}{2\\sigma^2})}=\\ln{\\frac{n}{\\sqrt{2\\pi\\sigma^2}}}-\\frac{1}{2\\sigma^2}\\sum_{i=1}^n(y_i-\\theta^T x_i)^2$<br /><br />\n",
    "$\\hat{\\theta}_{MLE}=\\underset{\\theta\\in\\mathbb{R}^d}{\\mathrm{argmax}} L(\\theta)=\\underset{\\theta\\in\\mathbb{R}^d}{\\mathrm{argmin}}\\sum_{i=1}^n(y_i-\\theta^T x_i)^2$<br />\n",
    "\n",
    "NOTE 1: we end up optimizing something very similar to so called <b>Quadratic Loss</b> which is standard loss function used for training linear models in Machine Learning. <br />\n",
    "NOTE 2: Optimization function is <b>convex</b> so there exists global unique minimizer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Some Concepts of Information Theory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Statistics and Machine Learning there is often a requirement of having some kind of <b>distance</b> notion between probability distributions. In other words, we want to know how similar or different are two probability distributions and, moreover, we want to quantify this similarity.\n",
    "\n",
    "<br /><br />\n",
    "<b><big>Measure of Information Content</big></b>\n",
    "\n",
    "Given some probability space $(\\Omega, \\mathcal{F}, P)$, let $A\\subset \\mathcal{F}$ is some event. <b>Information content / self-information / surprisal</b> of an event $A$ is defined as follows:<br /><br />\n",
    "$I(A)=-logP(A)=log\\frac{1}{P(A)}$<br /><br />\n",
    "Based on the definition, information of event $A$ measures \"how much unlikely\" is event $A$, i.e \"how much surprised\" shall we be by observing $A$.<br /><br />\n",
    "Examples:<br />\n",
    "Let's assume we toss 6-sided fair dice and observe the outcome. Let's define events: A = {result <= 6}, B = {result is odd}, C = {result is 6}\n",
    "<ul>\n",
    "    <li>$I(A)=-log(1)=0$ - event $A$ has \"no surprise\" because we already knew that A is sure event. No information content</li>\n",
    "    <li>$I(B)=-log(0.5)=1$ - event $B$ has \"surpriseness\"/\"information content\" of 1. We get some information - we can rule out that observation is even.</li>\n",
    "    <li>$I(C)=-log(\\frac{1}{6})\\approx2.58$ - event $C$ has the smallest probability so carries the most \"surprise\".</li>\n",
    "</ul>\n",
    "\n",
    "<b>Conclusion:</b> Rare events have high information content.<br /><br />\n",
    "\n",
    "<b><big>Entropy</big></b>\n",
    "<br /><br />\n",
    "Let $p(X)$ be a probability mass function for some random variable $X$.\n",
    "\n",
    "The <b>entropy</b> of a random variable $X$ is defined as an expectation of the information content of its outcomes:<br /><br />\n",
    "$H(X)=\\mathbb{E}_p[I_p(X)]=-\\sum_{i=1}^{n}p(x_i)logp(x_i)$   ($x_i$ are possible values of $X$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def entropy(x):\n",
    "    return -np.sum(x * np.log2(x))\n",
    "\n",
    "def bar_plot(x):\n",
    "    plt.bar(list(range(len(x))), x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQOElEQVR4nO3cf6zd9V3H8ee9bRFGO8TLwbawijHZ2+AidRSMUrYY6hKLBheKzuJ+EVZQFjPdpnPtRpnWGCYZmXTEgAgGbxyWFFLtFtO0U8jA0owfmsW3m4LS9mZrbhvXNtuo3Osf55zx5XDvPd9777n3nn76fCRN7vf7+Xy/fZ9vP+fVz/2e7/kMjI+PI0kqx+BCFyBJ6i2DXZIKY7BLUmEMdkkqjMEuSYVZvMB//w8BVwAjwKsLXIsknS4WASuAZ4DvdzYudLBfATyxwDVI0unqauDJzp0LHewjAMeOnWRsrH+fpx8aWsro6ImFLqMr6+y906VW6+ytfq9zcHCA888/F1oZ2mmhg/1VgLGx8b4OdqDv62uzzt47XWq1zt46Teqc8Ba2H55KUmEMdkkqjMEuSYUx2CWpMLU+PI2IjcAWYAlwd2Zu72i/HbgJONbadV9nH0nS/Oga7BFxEbANuJzmg/BfjYh9mfn1Src1wHsy86m5KVOSVFedWzHrgL2ZeTQzTwI7gA0dfdYAn4yIFyLinog4u9eFSpLqqXMrZiWvfwh+BLiyvRERS4FngY8D3wQeBD4FbK5bxNDQ0rpdX+eVU69y1pJFMzp2uhqNZfPy98yWdb6mV+NjtrXO1zj13763Tqcx2qlOsA8C1Sf1B4Cx9kZmngDWt7cj4i7gAaYR7KOjJ2b0ZYBGYxm/8tHHp32czgy77rqOI0eOz+ocjcaynpzDcaqJzHSMDg4OTDkhrnMr5iDNxWbalgOH2xsRsSoibqq0DwCnplmnJKlH6szY9wBbI6IBnASuBzZV2r8L3BkR+4CXgNuAnT2uU5JUU9cZe2YeonlbZR/wHDCcmfsjYndErMnMI8AtwC4gac7Y75rDmiVJU6j1HHtmDgPDHfvWV35+FHi0t6VJkmbCb55KUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SClMr2CNiY0R8PSK+ERG3TdHv2oh4sXflSZKmq2uwR8RFwDZgLbAa2BQRl07Q70eBPwMGel2kJKm+OjP2dcDezDyamSeBHcCGCfrdD9zRy+IkSdNXJ9hXAiOV7RHg4mqHiPgd4GvA070rTZI0E4tr9BkExivbA8BYeyMi3gZcD1xDR+DXNTS0dCaHSV01Gsv64hzSZOZifNUJ9oPA1ZXt5cDhyvYNwArgAHAWsDIinsjM6jFTGh09wdjYePeOHXzDqZsjR47P6vhGY1lPziFNZibja3BwYMoJcZ1g3wNsjYgGcJLm7HxTuzEzbwduB4iIS4CvTCfUJUm91fUee2YeAjYD+4DngOHM3B8RuyNizVwXKEmanjozdjJzGBju2Ld+gn4vAZf0ojBJ0sz4zVNJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYRbX6RQRG4EtwBLg7szc3tH+buAOYBHwDLApM1/pca2SpBq6ztgj4iJgG7AWWA1siohLK+3nAvcAv5iZPwWcDXxgTqqVJHVV51bMOmBvZh7NzJPADmBDu7G175LM/FZEvAm4EDg2J9VKkrqqE+wrgZHK9ghwcbVDZp6KiF8CXgYuAP6xZxVKkqalzj32QWC8sj0AjHV2yswvAUMR8SfAvcDGukUMDS2t21WalkZjWV+cQ5rMXIyvOsF+ELi6sr0cONzeiIgfAdZkZnuW/jfAF6dTxOjoCcbGxrt37OAbTt0cOXJ8Vsc3Gst6cg5pMjMZX4ODA1NOiOvcitkDXBMRjdY99OuBL1faB4CHI2JVa/sG4MlpVypJ6omuwZ6Zh4DNwD7gOWA4M/dHxO6IWJOZo8Am4O8j4nkggD+Yy6IlSZOr9Rx7Zg4Dwx371ld+fgx4rLelSZJmwm+eSlJhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYRbX6RQRG4EtwBLg7szc3tF+HXAHMAC8CHwwM4/1uFZJUg1dZ+wRcRGwDVgLrAY2RcSllfY3A/cC12bmZcALwNY5qVaS1FWdWzHrgL2ZeTQzTwI7gA2V9iXAbZl5qLX9ArCqt2VKkuqqcytmJTBS2R4BrmxvZOYosBMgIs4BPgH8+XSKGBpaOp3uUm2NxrK+OIc0mbkYX3WCfRAYr2wPAGOdnSLiPJoB/3xmPjSdIkZHTzA2Nt69YwffcOrmyJHjszq+0VjWk3NIk5nJ+BocHJhyQlznVsxBYEVlezlwuNohIlYAT9C8DXPztKuUJPVMnRn7HmBrRDSAk8D1wKZ2Y0QsAnYBj2TmH89JlZKk2roGe2YeiojNwD7gLOD+zNwfEbuBTwNvAd4OLI6I9oeqBzLTmbskLYBaz7Fn5jAw3LFvfevHA/hFJ0nqGwayJBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTCL63SKiI3AFmAJcHdmbp+k318DezPzwZ5VKEmalq4z9oi4CNgGrAVWA5si4tKOPisjYhewYU6qlCTVVudWzDqas/CjmXkS2MEbA/xG4HHgkR7XJ0mapjq3YlYCI5XtEeDKaofM/CxARKztXWmSpJmoE+yDwHhlewAY62URQ0NLe3k66QcajWV9cQ5pMnMxvuoE+0Hg6sr2cuBwL4sYHT3B2Nh4944dfMOpmyNHjs/q+EZjWU/OIU1mJuNrcHBgyglxnWDfA2yNiAZwErge2DTtSiRJ86Lrh6eZeQjYDOwDngOGM3N/ROyOiDVzXaAkaXpqPceemcPAcMe+9RP0+0BvypIkzZTfPJWkwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIsrtMpIjYCW4AlwN2Zub2jfTVwP/Bm4J+BWzPz/3pcqySphq4z9oi4CNgGrAVWA5si4tKObg8DH87MtwIDwId6XagkqZ46M/Z1wN7MPAoQETuADcBnWts/BpyTmU+3+j8I3AHcW+PciwAGBwemV3XFheefM+NjVb7ZjK1ensNxqsnMZHxVjlk0UXudYF8JjFS2R4Aru7RfXLO+FQDnn39uze5v9Jdb3jXjY1W+oaGlfXEOx6kmM8vxtQL4z86ddYJ9EBivbA8AY9Non8ozwNU0/zN4teYxknSmW0Qz1J+ZqLFOsB+kGb5ty4HDHe0rpmifyveBJ2v2lSS95g0z9bY6jzvuAa6JiEZEvAm4HvhyuzEz/xv4XkRc1dr1XuBLsyhWkjQLXYM9Mw8Bm4F9wHPAcGbuj4jdEbGm1e1G4HMR8e/AUuDzc1WwJGlqA+Pj4917SZJOG37zVJIKY7BLUmEMdkkqjMEuSYWptQjYmSAiVtFc8+ZCIIEbM/NER58VwF/RfFZ/DPhYZu6NiCXAKPBfle6XZ2bPvnQ104XY6ryuXqpR53U0l5wYAF4EPpiZxyLi/cCfAt9qdf2HzNy8gHXeDtwEHGvtui8zty/EgndT1dqq58FK9wZwLDPfNt/XtFXPm4GvAr+cmS91tPXFGK1RZ1+M0dlwxv6aLwBfyMyfBA4An5qgz2eBXZm5GvgNYDgiFgE/DTyVmasrf3oZ6rNZiK3O65qXOltvpnuBazPzMuAFYGureQ3we5XrN5ehXud6rgHeU6mnHabzuuBdt1oz87l2jcDP0/yP6NbKa5iXa9qq9WdpfuHwrZN0WfAx2q3Ofhmjs2WwA60Z9zuAHa1dDwI3TNB1JzDc+vmbwNk0n9u/AmhExIGIeDoi3tnjEn+wEFtmnmzVuaFS/0QLsd0wjdc1L3XSnHHe1vpuBDTfNKtaP18BvD8i/jUiHo6I8xewTmi+iT8ZES9ExD0RcfZk13kO66xba9sfAv+Ume1vc8/nNYVmUN/GBN8876MxOmWd9M8YnRWDvekC4DuVX6knXMgsMx/NzPav5h8Dns3M/6W5Vs5jwM8BvwV8MSIu6GF93RZam6y91uuarzozczQzdwJExDnAJ2het3bfP6L528/LwD0LVWdELAWeBT4OvB34YZqzyNkseDcntbZFxHnAJpq3EKp95+uakpk3Z+YTkzT3yxidss4+GqOzcsbdY4+IG4DPdez+Bq9fyAymWMgsIj4C3AK8EyAz/6LS/GxE/AtwFfD4rAtumulCbJ37of4CbTNRa0G4VgjtBJ7PzIcAMvPdlfY7mWIdjLmus3V/d32lnruAB4DdUx03R+ousvebwGOZ+e32jnm+pt30yxitpQ/G6KyccTP2zPy7zLy4+gd4F3Be6345NBc1m3Ahs9Y/6IeAd2Tmy619742In6h0GwBO9bDsbgutTdb+bWq+rnmqs/0B9BM0f8W9ubXvvIj43Uq3AWAuP5Ccss6IWBURN3XUc6rbcXOk7t/5q8DftjcW4Jp20y9jtKs+GaOzcsYF+0Qy8xTNf8hfb+16HxMsZNaaqf8CcFVmHqw0XQZ8tNUngJ9pna9XZrQQW93XNV91tt68u4BHMvMjmdmeqZ0Afr/1oRbAh2nOlhakTuC7wJ0R8eMRMUDzfuzOBVrwrluttGq8HHiqsnu+r+mU+miMTqmPxuisnHG3Yqbw28BDEbEF+B+aT70QEbfSvD94e+vPd4CvNPMbaP7K/hnggYj4N5q/Vr4vM4/3qrDMPBQR7YXYzgLuby/EBnw6Mw/QXIjtvtan+l/jtYXYJnxdc6FbncBbaN6zXhwR7Q8AD2TmzRHxa8C9rfua/0HzDb4gdWbmgYi4heYb/CyaT1Dc1Tp8suu8YLXSfMTxlcz8XuW4V+fzmk6m38ZotzrpkzE6Wy4CJkmF8VaMJBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTD/DzYWbd9y6Y37AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = [0.5, 0.5]\n",
    "\n",
    "bar_plot(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entropy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQhElEQVR4nO3df4zk9V3H8efu3cHR3kFxO8gdFDEmfRts2hMOjBbamJ4kHjbYcGi92p+hB0pjWttq7V0LVDGGSkoqlBgQweBG6zVATq/VkLsqTVvhUn5oGt9WBe0dm/ayXOTuAhTY9Y+ZKV+H3Z3v7n5nd/bD85Fcsp/v5/P97nvmPvPa73xn5jMj09PTSJLKMbrcBUiSmmWwS1JhDHZJKozBLkmFMdglqTCrl/n3nwicD0wALy5zLZK0UqwCNgAPAc/1di53sJ8PPLDMNUjSSnUR8LXejcsd7BMAR44cZ2pqeN9PPza2jsnJY8tdRl/W2byVUqt1NmvY6xwdHeHUU18NnQzttdzB/iLA1NT0UAc7MPT1dVln81ZKrdbZrBVS54yXsH3xVJIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwiz3+9il4q0/+STWnjj4h1qrtX7gv6MJ1vmSZ597gaNPP9P4cQ12acDWnriat3/0vuUuQ0Noz42XcnQAx/VSjCQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmFqre4YEduBXcAa4KbMvKWn/xrgA8CRzqbbesdIkpZG32CPiDOA64HzgOeAr0fE/sz8dmXYZuCdmfmNwZQpSaqrzqWYLcC+zHwqM48Du4FtPWM2A5+MiMci4uaIWNt0oZKkeuoE+0ZgotKeAM7sNiJiHfAw8HHgXOA1wKcarFGSNA91rrGPAtOV9ggw1W1k5jFga7cdETcCdwA76xYxNrau7tBl49d5NWul1Akrq1atPIOYX3WC/SBwUaV9OvBktxERZwFbMvOOzqYR4Pn5FDE5eYypqen+A5dJq7Wew4cH8QVWzbLO5jVRq38YNJeFzK/R0ZE5T4jrBPv9wLUR0QKOA5cBOyr9zwA3RMR+4AngauCeeVcqSWpE32vsmXmI9mWV/cAjwHhmPhgReyNic2YeBq4E9gBJ+4z9xgHWLEmaQ633sWfmODDes21r5ecvAV9qtjRJ0kL4yVNJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVJhay/YOq/Unn8TaE5fmJqyUb8Gxzpc8+9wLHH36mYH/HmnYrOhgX3viat7+0fuWuwwNqT03XsrK+AI+qVleipGkwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSpMrWCPiO0R8e2I+E5EXD3HuEsi4vHmypMkzVffYI+IM4DrgQuBTcCOiDhnhnE/CvwxMNJ0kZKk+uqcsW8B9mXmU5l5HNgNbJth3O3AdU0WJ0mavzrBvhGYqLQngDOrAyLit4BvAd9srjRJ0kLU+QalUWC60h4BprqNiHgDcBnwNnoCv66xsXUL2U3qq4mv4FspXzeolWkQ86tOsB8ELqq0TweerLQvBzYAB4ATgI0R8UBmVveZ0+TkMaampvsP7OEDTv0cPry4L8drtdY3cgxpNguZX6OjI3OeENcJ9vuBayOiBRynfXa+o9uZmdcA1wBExNnAV+cT6pKkZvW9xp6Zh4CdwH7gEWA8Mx+MiL0RsXnQBUqS5qfOGTuZOQ6M92zbOsO4J4CzmyhMkrQwfvJUkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwq+sMiojtwC5gDXBTZt7S0/8O4DpgFfAQsCMzf9BwrZKkGvqesUfEGcD1wIXAJmBHRJxT6X81cDPwC5n5U8Ba4H0DqVaS1FedSzFbgH2Z+VRmHgd2A9u6nZ1tZ2fm9yLiVcBpwJGBVCtJ6qvOpZiNwESlPQFcUB2Qmc9HxC8CdwOHgH+YTxFjY+vmM1yqrdVaPxTHkGYziPlVJ9hHgelKewSY6h2UmV8GxiLiD4Fbge11i5icPMbU1HT/gT18wKmfw4ePLmr/Vmt9I8eQZrOQ+TU6OjLnCXGdSzEHgQ2V9unAk91GRPxIRFxc6f9L4I3zrFOS1JA6wX4/8LaIaHWuoV8GfKXSPwLcHRFnddqXA19rtkxJUl19gz0zDwE7gf3AI8B4Zj4YEXsjYnNmTgI7gL+NiEeBAH53kEVLkmZX633smTkOjPds21r5+V7g3mZLkyQthJ88laTCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmFW1xkUEduBXcAa4KbMvKWn/1LgOmAEeBx4f2YeabhWSVINfc/YI+IM4HrgQmATsCMizqn0nwzcClySmW8CHgOuHUi1kqS+6lyK2QLsy8ynMvM4sBvYVulfA1ydmYc67ceAs5otU5JUV51LMRuBiUp7Arig28jMSeAegIg4CfgE8CcN1ihJmoc6wT4KTFfaI8BU76CIOIV2wD+amXfNp4ixsXXzGS7V1mqtH4pjSLMZxPyqE+wHgYsq7dOBJ6sDImID8PfAPuAj8y1icvIYU1PT/Qf28AGnfg4fPrqo/Vut9Y0cQ5rNQubX6OjInCfEdYL9fuDaiGgBx4HLgB3dzohYBewBvpiZfzDvCiVJjeob7Jl5KCJ2AvuBE4DbM/PBiNgLfBp4HXAusDoiui+qHsjMKwZVtCRpdrXex56Z48B4z7atnR8P4AedJGloGMiSVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYVZXWdQRGwHdgFrgJsy85ZZxv0FsC8z72ysQknSvPQ9Y4+IM4DrgQuBTcCOiDinZ8zGiNgDbBtIlZKk2upcitlC+yz8qcw8Duzm5QH+LuA+4IsN1ydJmqc6l2I2AhOV9gRwQXVAZn4WICIubK40SdJC1An2UWC60h4BpposYmxsXZOHk36o1Vo/FMeQZjOI+VUn2A8CF1XapwNPNlnE5OQxpqam+w/s4QNO/Rw+fHRR+7da6xs5hjSbhcyv0dGROU+I6wT7/cC1EdECjgOXATvmXYkkaUn0ffE0Mw8BO4H9wCPAeGY+GBF7I2LzoAuUJM1PrfexZ+Y4MN6zbesM497XTFmSpIXyk6eSVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYVZXWdQRGwHdgFrgJsy85ae/k3A7cDJwD8BV2XmCw3XKkmqoe8Ze0ScAVwPXAhsAnZExDk9w+4GPpSZrwdGgA82XagkqZ46Z+xbgH2Z+RRAROwGtgGf6bR/DDgpM7/ZGX8ncB1wa41jrwIYHR2ZX9UVp5160oL3VfkWM7eaPIbzVLNZyPyq7LNqpv46wb4RmKi0J4AL+vSfWbO+DQCnnvrqmsNf7s92XbzgfVW+sbF1Q3EM56lms8j5tQH4z96NdYJ9FJiutEeAqXn0z+Uh4CLafwxerLmPJL3SraId6g/N1Fkn2A/SDt+u04Ene/o3zNE/l+eAr9UcK0l6ycvO1LvqvN3xfuBtEdGKiFcBlwFf6XZm5n8Dz0bEmzub3g18eRHFSpIWoW+wZ+YhYCewH3gEGM/MByNib0Rs7gx7F/C5iPg3YB3w+UEVLEma28j09HT/UZKkFcNPnkpSYQx2SSqMwS5JhTHYJakwtRYBeyWIiLNor3lzGpDAuzLzWM+YDcCf036v/hTwsczcFxFrgEngvyrDz8vMxj50tdCF2OrcribVqPNS2ktOjACPA+/PzCMR8V7gj4DvdYb+XWbuXMY6rwE+ABzpbLotM29ZjgXv5qq1U8+dleEt4EhmvmGp79NOPScDXwd+KTOf6Okbijlao86hmKOL4Rn7S74AfCEzfxI4AHxqhjGfBfZk5ibg14DxiFgFvBH4RmZuqvxrMtQXsxBbndu1JHV2Hky3Apdk5puAx4BrO92bgd+u3H+DDPU69+dm4J2VerphuqQL3vWrNTMf6dYI/BztP0RXVW7DktynnVp/hvYHDl8/y5Bln6P96hyWObpYBjvQOeN+C7C7s+lO4PIZht4DjHd+/g9gLe337Z8PtCLiQER8MyLe2nCJP1yILTOPd+rcVql/poXYLp/H7VqSOmmfcV7d+WwEtB80Z3V+Ph94b0T8S0TcHRGnLmOd0H4QfzIiHouImyNi7Wz38wDrrFtr1+8B/5iZ3U9zL+V9Cu2gvpoZPnk+RHN0zjoZnjm6KAZ722uBpytPqWdcyCwzv5SZ3afmHwMezsz/pb1Wzr3AzwK/Afx1RLy2wfr6LbQ2W3+t27VUdWbmZGbeAxARJwGfoH2/dcf+Pu1nP98Fbl6uOiNiHfAw8HHgXOA1tM8iF7Pg3UBq7YqIU4AdtC8hVMcu1X1KZl6RmQ/M0j0sc3TOOodoji7KK+4ae0RcDnyuZ/N3+P8LmcEcC5lFxIeBK4G3AmTmn1a6H46IfwbeDNy36ILbFroQW+92qL9A20LUWhCuE0L3AI9m5l0AmfmOSv8NzLEOxqDr7Fzf3Vqp50bgDmDvXPsNSN1F9n4duDczv9/dsMT3aT/DMkdrGYI5uiivuDP2zPybzDyz+g+4GDilc70c2ouazbiQWec/9IPAWzLzu51t746In6gMGwGeb7Dsfgutzdb/fWreriWqs/sC9AO0n+Je0dl2SkR8pDJsBBjkC5Jz1hkRZ0XEB3rqeb7ffgNS93f+MvBX3cYy3Kf9DMsc7WtI5uiivOKCfSaZ+Tzt/8hf7Wx6DzMsZNY5U/954M2ZebDS9Sbgo50xAfx053hNWdBCbHVv11LV2Xnw7gG+mJkfzszumdox4Hc6L2oBfIj22dKy1Ak8A9wQET8eESO0r8fes0wL3vWrlU6N5wHfqGxe6vt0TkM0R+c0RHN0UV5xl2Lm8JvAXRGxC/gf2u96ISKuon198JrOv6eBr7bzG2g/Zf8McEdE/Cvtp5XvycyjTRWWmYciorsQ2wnA7d2F2IBPZ+YB2gux3dZ5Vf9bvLQQ24y3axD61Qm8jvY169UR0X0B8EBmXhERvwLc2rmu+e+0H+DLUmdmHoiIK2k/wE+g/Q6KGzu7z3Y/L1uttN/i+IPMfLay34tLeZ/OZtjmaL86GZI5ulguAiZJhfFSjCQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakw/wc8wXUeEWpilAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = [0.45, 0.55]\n",
    "\n",
    "bar_plot(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9927744539878083"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entropy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPuUlEQVR4nO3df5Dc9V3H8eddEkhoEsxct5Lwwzo6fTsMY1Ia6Cg/WiXTP0AGOwSt9CcMTVGqUwEdNJEfrTidKpZqU4YBKjidUzEORcbAOEyo0mlpiIXiz7dUqZJw2MwRJckAAnf+sbuy3V5uv3e7t7v3yfMxczP3+X4+3++995vvvvK57+5+bmR6ehpJUjlGB12AJKm3DHZJKozBLkmFMdglqTAGuyQVZumAf/6xwBnABPD6gGuRpMViCbAWeBx4pb1z0MF+BvDogGuQpMXqHOCr7RsHHewTAAcOHGZqanjfTz82tpLJyUODLqMj6+y9xVKrdfbWsNc5OjrCmjVvgkaGtht0sL8OMDU1PdTBDgx9fU3W2XuLpVbr7K1FUueMt7B98VSSCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIM+n3sXVm1egXLj+3PQ6jVVvXl53TLOt/w8iuvcfDFlxb850jDZlEH+/Jjl3LhNfcPugwNqQduuYiDgy5CGgBvxUhSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKkylP40XEZcC24BlwK2Zub2t/3TgduAY4FngA5n53z2uVZJUQccZe0ScCNwMnA1sALZExKltwz4HXJ+Z64EEru11oZKkaqrcitkE7MrMFzLzMLAD2Nw2ZgmwuvH9cYB/Gl6SBqTKrZh1wERLewI4s23M1cBfR8StwGHgnXMpYmxs5VyGS5XVaquG4hj9YJ29tVjqnEmVYB8FplvaI8BUsxERK4C7gE2ZuTsirgb+GLigahGTk4eYmpruPLDNYj7x6o/9+w92tX+ttqrrY/SDdfbWsNc5Ojoy64S4yq2YvcDalvYJwHMt7dOAlzJzd6N9O/DuuZUpSeqVKsH+MHBeRNQi4jjgYuChlv5vAydHRDTaFwGP97ZMSVJVHYM9M/cBW4FHgCeB8cYtl50RsTEzDwAfAe6NiKeAy4HLFrBmSdIsKr2PPTPHgfG2bee3fP8g8GBvS5MkzYefPJWkwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSrM0iqDIuJSYBuwDLg1M7e39QdwO7AGeB54X2Ye6HGtkqQKOs7YI+JE4GbgbGADsCUiTm3pHwH+Evh0Zq4HngCuW5hyJUmdVLkVswnYlZkvZOZhYAewuaX/dOBwZj7UaP8OsB1J0kBUuRWzDphoaU8AZ7a0fxR4PiLuAt4O/DPwy3MpYmxs5VyGS5XVaquG4hj9YJ29tVjqnEmVYB8FplvaI8BU2zHeDZybmXsi4lPA7wMfqVrE5OQhpqamOw9ss5hPvPpj//6DXe1fq63q+hj9YJ29Nex1jo6OzDohrnIrZi+wtqV9AvBcS/t54OnM3NNo/wnfO6OXJPVRlWB/GDgvImoRcRxwMfBQS//XgFpErG+0LwT+rrdlSpKq6hjsmbkP2Ao8AjwJjGfm7ojYGREbM/Ml4L3AHRHxj8BPA9csZNGSpCOr9D72zBwHxtu2nd/y/Tfw9oskDQU/eSpJhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklSYSsEeEZdGxD9FxNMRcdUs4y6IiGd6V54kaa46BntEnAjcDJwNbAC2RMSpM4z7QeD3gJFeFylJqq7KjH0TsCszX8jMw8AOYPMM4+4EbuplcZKkuVtaYcw6YKKlPQGc2TogIn4F+Cbw2HyKGBtbOZ/dpI5qtVVDcYx+sM7eWix1zqRKsI8C0y3tEWCq2YiI04CLgfOAk+ZTxOTkIaampjsPbLOYT7z6Y//+g13tX6ut6voY/WCdvTXsdY6Ojsw6Ia5yK2YvsLalfQLwXEv7kkb/HmAnsC4iHp17qZKkXqgyY38YuDEiasBh6rPzLc3OzLwBuAEgIt4KfCUzz+l9qZKkKjrO2DNzH7AVeAR4EhjPzN0RsTMiNi50gZKkuakyYyczx4Hxtm3nzzDuO8Bbe1GYJGl+/OSpJBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYZZWGRQRlwLbgGXArZm5va3/IuAmYAR4BrgsMw/0uFZJUgUdZ+wRcSJwM3A2sAHYEhGntvSvBm4DLsjM9cBTwI0LUq0kqaMqt2I2Absy84XMPAzsADa39C8DrsrMfY32U8ApvS1TklRVlVsx64CJlvYEcGazkZmTwH0AEbECuA74w7kUMTa2ci7DpcpqtVVDcYx+sM7eWix1zqRKsI8C0y3tEWCqfVBEHE894L+VmffMpYjJyUNMTU13HthmMZ949cf+/Qe72r9WW9X1MfrBOntr2OscHR2ZdUJc5VbMXmBtS/sE4LnWARGxFniU+m2YK+ZepiSpV6rM2B8GboyIGnAYuBjY0uyMiCXAA8C9mfnbC1KlJKmyjsGemfsiYivwCHAMcGdm7o6IncD1wMnA6cDSiGi+qLonM525S9IAVHofe2aOA+Nt285vfLsHP+gkSUPDQJakwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCVPoLSpLmb9XqFSw/duGfarXaqgX/Gb1gnW94+ZXXOPjiSz0/rsEuLbDlxy7lwmvuH3QZGkIP3HIRBxfguN6KkaTCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUmEp/Gi8iLgW2AcuAWzNze1v/BuBOYDXwt8CVmflaj2uVJFXQccYeEScCNwNnAxuALRFxatuwLwEfz8y3ASPAR3tdqCSpmioz9k3Arsx8ASAidgCbgU822j8ErMjMxxrj7wZuAm6rcOwlAKOjI3OrusVb1qyY974qXzfXVi+P4XWqI5nP9dWyz5KZ+qsE+zpgoqU9AZzZof+kivWtBViz5k0Vh3+/u7a9Z977qnxjYyuH4hhepzqSLq+vtcC/tW+sEuyjwHRLewSYmkP/bB4HzqH+n8HrFfeRpKPdEuqh/vhMnVWCfS/18G06AXiurX/tLP2zeQX4asWxkqQ3fN9MvanK2x0fBs6LiFpEHAdcDDzU7MzM/wBejoizGps+CDzYRbGSpC50DPbM3AdsBR4BngTGM3N3ROyMiI2NYe8HPhsR/wKsBP5goQqWJM1uZHp6uvMoSdKi4SdPJakwBrskFcZgl6TCGOySVJhKi4AdDSLiFOpr3rwFSOD9mXmobcxa4I+ov1d/Crg2M3dFxDJgEvj3luHvyMyefehqvguxVXlcvVShzouoLzkxAjwDXJaZByLiw8Cngf9qDP2rzNw6wDpvAC4HDjQ23ZGZ2wex4N1stTbqubtleA04kJmn9fucNupZDXwN+JnM/E5b31BcoxXqHIprtBvO2N/wBeALmfljwB7gt2YY87vAA5m5AfgFYDwilgA/Dnw9Mze0fPUy1LtZiK3K4+pLnY0n023ABZm5HngKuLHRvRG4uuX8LWSoVzmfG4H3tdTTDNO+LnjXqdbMfLJZI/CT1P8jurLlMfTlnDZqfSf1Dxy+7QhDBn6NdqpzWK7RbhnsQGPGfS6wo7HpbuCSGYbeB4w3vv82sJz6+/bPAGoRsSciHouId/W4xP9fiC0zDzfq3NxS/0wLsV0yh8fVlzqpzzivanw2AupPmlMa358BfDgi/j4ivhQRawZYJ9SfxL8ZEU9FxOcjYvmRzvMC1lm11qbfAP4mM5uf5u7nOYV6UF/FDJ88H6JrdNY6GZ5rtCsGe92bgRdbfqWecSGzzPyLzGz+an4t8ERm/g/1tXK+DPwE8IvAn0XEm3tYX6eF1o7UX+lx9avOzJzMzPsAImIFcB3189Yc+ynqv/08C3x+UHVGxErgCeDXgNOBH6A+i+xmwbsFqbUpIo4HtlC/hdA6tl/nlMy8IjMfPUL3sFyjs9Y5RNdoV466e+wRcQnw2bbNT/O9C5nBLAuZRcQngI8B7wLIzNtbup+IiG8AZwH3d11w3XwXYmvfDtUXaJuPSgvCNULoPuBbmXkPQGa+t6X/M8yyDsZC19m4v3t+Sz23AF8Eds623wKpusjeB4AvZ+Z3mxv6fE47GZZrtJIhuEa7ctTN2DPzzzPzpNYv4D3A8Y375VBf1GzGhcwa/6AfBc7NzGcb2z4YET/SMmwEeLWHZXdaaO1I/d+l4uPqU53NF6Afpf4r7hWNbcdHxK+2DBsBFvIFyVnrjIhTIuLytnpe7bTfAqn6M38W+NNmYwDntJNhuUY7GpJrtCtHXbDPJDNfpf4P+fONTR9ihoXMGjP1nwLOysy9LV3rgWsaYwJ4e+N4vTKvhdiqPq5+1dl48j4A3JuZn8jM5kztEPDrjRe1AD5OfbY0kDqBl4DPRMQPR8QI9fux9w1owbtOtdKo8R3A11s29/uczmqIrtFZDdE12pWj7lbMLH4JuCcitgH/Sf1dL0TEldTvD97Q+HoR+Eo9v4H6r+yfBL4YEf9A/dfKD2XmwV4Vlpn7IqK5ENsxwJ3NhdiA6zNzD/WF2O5ovKr/Td5YiG3Gx7UQOtUJnEz9nvXSiGi+ALgnM6+IiJ8Dbmvc1/xX6k/wgdSZmXsi4mPUn+DHUH8HxS2N3Y90ngdWK/W3OP5vZr7cst/r/TynRzJs12inOhmSa7RbLgImSYXxVowkFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMP8HmfUz7MZfQAAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = [0.9, 0.1]\n",
    "\n",
    "bar_plot(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4689955935892812"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entropy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.321928094887362"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXiElEQVR4nO3df4xd5X3n8feMMVlcG4cOw2IbiITUfAgrsU4wSRt+7tqLWrIbgswPFSv8iAyyCJXYtkuTtdkmaZyghq0dUjuJ4qxMZCYoi0hIFIO6jq3EFSKULcaRST5Ko60Xm0kZjUPAjkKxZ/aPe6Z7uRl7zp1f19zn85IsnfM8zzn3+cJoPufHnHt6RkdHiYiIcvV2egIREdFZCYKIiMIlCCIiCpcgiIgoXIIgIqJwp3R6Am16G3AJMAgc6/BcIiLeKuYAi4C/A15v7XyrBcElwO5OTyIi4i3qcuBvWxvfakEwCPCLXxxhZOTkff6hr28+w8OHOz2Njii5dii7/pJrh5O7/t7eHs4447eg+h3a6q0WBMcARkZGT+ogAE76+c2kkmuHsusvuXZ4S9Q/7iX13CyOiChcgiAionAJgoiIwiUIIiIKV+tmsaSbgXXAXGCj7U0t/dcCnwR6gP8D3G77F5LOA7YBZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3Y1H868EXgA7b/LbAX+ETVvRnYbPsC4Fngvqr908Bu2+8CvgJ8flqqiYiIttW5NLQC2Gn7kO0jwKPA9U39c4GP2j5Yre8FzpM0F7iiGg+wFbihWv4AjTMCgK8Df1CNj4iIWVbn0tBi3vwQwiDw3rEV28PANwEknQZ8DPgCcCbwqu2jTdud07pP20clvQr0Ay/VmXRf3/w6w8b1z28c49S5cya9fV39/QtmdP+TqaNbaof2a5mt2iH/72fSyVw7nJz111EnCHqB5qckeoCR1kGSFtIIhOdtP1RdUmp9umJsu56W9nH3eTzDw4cn/eBGf/8C/tOfPD6pbU8m3/nv1zI09Fpb23RL7dB+/SXXDt1Tf8m1w+Tqh8aTxSc6gK5zaegAjS8rGnM2LUfukhbR+A6gvcDqqvllYKGksfha1LTdwWo/SDoFWAAM15hLRERMszpBsANYLqlf0jxgJfDkWGf1i/47wDds32N7FMD2GzTC4aZq6C3AE9Xy9mqdqn93NT4iImbZhJeGbB+UtBbYBZwKbLH9jKTtwH8DzgXeA5wiaewm8rO2VwN3AQ9JWgf8X+APq/77gK2S9gGvAKums6iIiKiv1nMEtgeAgZa2a6rFZznOmYXt/cBV47QfAj7YzkQjImJm5MniiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLVeTCPpZmAdMBfYaHvTccZ9Ddhpe6uks4C/aepeCPTbni/pSuAx4MWq7znbt0+2iIiImLwJg0DSEmA9cDHwOvCUpF22X2gasxj4MrAc2Alg+2VgadXfC3wPWFttsgx4wPZnp6+UiIiYjDpnBCtoHOUfApD0KHA98KmmMauAx4Hh4+zjduBX1SsvAS4B/rWkPwT+Efio7RePs21ERMygOvcIFgODTeuDwDnNA2x/zvaW8TaWNIfGmcDHmppfAb5g+yJgO/BIO5OOiIjpU+eMoBcYbVrvAUba+IzfB35q+0djDbbXNC1/SdL9khba/mWdHfb1zW/j47tXf/+CTk+ho0quP7WXaybqrxMEB4DLm9bPBl5q4zM+RNMRf3W/4OPA/baPNY07WneHw8OHGRkZnXjgOLrph2ho6LW2xndT7dBe/SXXDt1Vf8m1Q/v1A/T29pzwALrOpaEdwHJJ/ZLmASuBJ9uYw+8Bu8dWbI8A11X7QdItwA9tH2ljnxERMU0mDALbB2lc498F7AEGbD8jabukZTU+43waZxXNbgXukbSPxo3k1e1NOyIipkut5wiqv/YZaGm7Zpxxt43TNm+ctn3A+2vPMiIiZkyeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtredJxxXwN22t5ard8K3A/8UzXku7bXSjoP2AacBRhYZfvwVAqJiIjJmfCMQNISYD1wGbAUuFPShS1jFkv6DnB9y+bLgD+2vbT6t7Zq3wxstn0B8Cxw3xTriIiISapzRrCCxlH+IQBJj9L4hf+ppjGrgMeB4ZZtLwF+R9J/BZ4H/gg4DFwBfKgasxX4PvBnkyshIiKmos49gsXAYNP6IHBO8wDbn7O9ZZxtB4G/AC4CXgT+GjgTeNX20ePtLyIiZk+dM4JeYLRpvQcYqbNz29eNLUv6S+BnwL0t+6Pu/sb09c1vZ3jX6u9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLE20kaSHwEdsbqqYe4CjwMrBQ0hzbx4BFdfbXbHj4MCMjrVlSTzf9EA0NvdbW+G6qHdqrv+TaobvqL7l2aL9+gN7enhMeQNe5NLQDWC6pX9I8YCXwZI3tDgP3SnpftX438E3bbwC7gZuq9luAJ2rsLyIiZsCEQWD7ILAW2AXsAQZsPyNpu6RlJ9juGHAj8EVJPwYupnFZCOAuGn999AKNs411UysjIiImq9ZzBLYHgIGWtmvGGXdby/pu4D3jjNsPXNXGPCMiYobkyeKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMLVekOZpJtpvE5yLrDR9qbjjPsasNP21mr9UmADcCowTONl9vslXQk8BrxYbfqc7dunUkhEREzOhEEgaQmwnsY7h18HnpK0y/YLTWMWA18GlgM7mzZ/GPig7b2SPgI8CFwLLAMesP3ZaaskIiImpc6loRU0jvIP2T4CPApc3zJmFfA48I2xBklvA9bZ3ls17QXOq5YvAa6WtFfStyWdO5UiIiJi8uoEwWJgsGl9EDineYDtz9ne0tL2uu1tAJJ6gU8A36q6XwG+YPsiYDvwyKRmHxERU1bnHkEvMNq03gOM1P0ASacCD1Wf9RkA22vG+m1/SdL9khba/mWdffb1za/78V2tv39Bp6fQUSXXn9rLNRP11wmCA8DlTetnAy/V2bmk+cC3adwovtb2G9XZwceB+20faxp+tN6UYXj4MCMjoxMPHEc3/RANDb3W1vhuqh3aq7/k2qG76i+5dmi/foDe3p4THkDXuTS0A1guqV/SPGAl8GTNz98G/ANwk+3XAWyPANdV+0HSLcAPq/sPERExyyYMAtsHgbXALmAPMGD7GUnbJS073naS3k3jL4QuBf5e0h5J26vuW4F7JO0DbgdWT7GOiIiYpFrPEdgeAAZa2q4ZZ9xtTcvP0bifMN7+9gHvb2eiERExM/JkcURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4Wq9oUzSzcA6YC6w0fam44z7GrDT9tZq/Twa7y0+CzCwyvZhSW8HHgbOB4aAG23/fIq1RETEJEx4RiBpCbAeuAxYCtwp6cKWMYslfQe4vmXzzcBm2xcAzwL3Ve2fBnbbfhfwFeDzU6oiIiImrc6loRU0jvIP2T4CPMpv/sJfBTwOfGOsQdJc4IpqPMBW4IZq+QM0zggAvg78QTU+IiJmWZ0gWAwMNq0PAuc0D7D9OdtbWrY7E3jV9tFxtvuXfVb9rwL97U09IiKmQ517BL3AaNN6DzAyie1o2q6npb3uPgHo65tfd2hX6+9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLNbZ7GVgoaY7tY8Cipu0OVvs5IOkUYAEwXHfSw8OHGRlpzZh6uumHaGjotbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2nPAAus6loR3Ackn9kuYBK4EnJ9rI9hvAbuCmqukW4IlqeXu1TtW/uxofERGzbMIgsH0QWAvsAvYAA7afkbRd0rIJNr+Lxl8ZvUDjrGJd1X4f8LuS9lVjPjrZAiIiYmpqPUdgewAYaGm7Zpxxt7Ws7weuGmfcIeCDbcwzIiJmSJ4sjogoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXK03lEm6mcZrJucCG21vaulfCmwBTgd+AKwBfhv4m6ZhC4F+2/MlXQk8BrxY9T1n+/apFBIREZMzYRBIWgKsBy4GXgeekrTL9gtNw7YBq20/LemrwB22vwgsrfbRC3yPxruPAZYBD9j+7PSVEhERk1Hn0tAKYKftQ7aPAI8C1491SnoHcJrtp6umrcANLfu4HfhV9e5jgEuAqyXtlfRtSedOpYiIiJi8OkGwGBhsWh8EzqnbL2kOjTOBjzWNeQX4gu2LgO3AI+1NOyIipkudewS9wGjTeg8w0kb/7wM/tf2jsQbba5qWvyTpfkkLbf+yzqT7+ubXGdb1+vsXdHoKHVVy/am9XDNRf50gOABc3rR+NvBSS/+iE/R/iKYj/up+wceB+20faxp3tOacGR4+zMjI6MQDx9FNP0RDQ6+1Nb6baof26i+5duiu+kuuHdqvH6C3t+eEB9B1Lg3tAJZL6pc0D1gJPDnWaXs/8GtJl1ZNHwaeaNr+94DdTeNHgOuq/SDpFuCH1f2HiIiYZRMGge2DNK7x7wL2AAO2n5G0XdKyatgqYIOknwDzgQebdnE+jbOGZrcC90jaR+NG8uqplREREZNV6zmC6q99Blrarmlafh5473G2nTdO2z7g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtre1NK/FNgCnA78AFhj+6ikW4H7gX+qhn7X9lpJ5wHbgLMAA6tsH56OgiIioj0TnhFIWgKsBy4DlgJ3SrqwZdg24G7b7wR6gDuq9mXAH9teWv1bW7VvBjbbvgB4Frhv6qVERMRk1Lk0tALYafuQ7SPAo8D1Y52S3gGcZvvpqmkrcEO1fAlwq6QfSdom6QxJc4Erqv20jo+IiFlWJwgWA4NN64PAOTX7B4G/AC4CXgT+GjgTeNX20ePsLyIiZlGdewS9wGjTeg8wUqff9nVjjZL+EvgZcG/LeFr2N6G+vvntDO9a/f0LOj2Fjiq5/tRerpmov04QHAAub1o/G3ippX9Ra7+khcBHbG+o2nuAo8DLwEJJc2wfq7Zt3t+EhocPMzLSmiX1dNMP0dDQa22N76baob36S64duqv+kmuH9usH6O3tOeEBdJ1LQzuA5ZL6Jc0DVgJPjnXa3g/8WtKlVdOHgSeAw8C9kt5Xtd8NfNP2G8Bu4Kaq/ZZqfEREdMCEQWD7ILAW2AXsAQZsPyNpu6Rl1bBVwAZJPwHmAw9WR/s3Al+U9GPgYhqXhQDuovHXRy/QONtYN51FRUREfbWeI7A9AAy0tF3TtPw88N5xttsNvGec9v3AVW3ONSIiZkCeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJupnG6yTnAhttb2rpXwpsAU4HfgCssX20eo/xBuBUYJjGy+z3S7oSeAx4sdrFc7Zvn46CIiKiPROeEUhaAqwHLgOW0njX8IUtw7YBd9t+J9AD3FG1Pwystr20Wn6wal8GPGB7afUvIRAR0SF1Lg2tAHbaPmT7CPAocP1Yp6R3AKfZfrpq2grcIOltwDrbe6v2vcB51fIlwNWS9kr6tqRzp6GWiIiYhDqXhhYDg03rg7z5RfXj9Z9j+3UaZwpI6gU+AXyrGvMK8A3bj0laAzwCXFp30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN6z3ASN1+SacCD1Wf9RkA22vG+m1/SdL9khba/mWdSQ8PH2ZkZHTigePoph+ioaHX2hrfTbVDe/WXXDt0V/0l1w7t1w/Q29tzwgPoOpeGDgCLmtbPBl6q0y9pPvAkjRC41vYbknolrZU0p+VzjtaYS0RETLM6QbADWC6pX9I8YCWNX+4A2N4P/Lr6CyGADwNPVMvbgH8AbqouFWF7BLiu2g+SbgF+WN1/iIiIWTZhENg+CKwFdgF7gAHbz0jaLmlZNWwVsEHST4D5wIOS3g1cS+Pa/99L2iNpezX+VuAeSfuA24HV01pVRETUVus5AtsDwEBL2zVNy8/z5hvIAM/RuF8w3v72Ae9va6YRETEj8mRxREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThar2hTNLNwDpgLrDR9qaW/qXAFuB04AfAGttHJZ1H473FZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3YMmwbcLftd9J4PeUdVftmYLPtC4Bngfuq9k8Du22/C/gK8PmpFhIREZNT54xgBbDT9iEASY8C1wOfqtbfAZxm++lq/Fbgk5K2AFcAH2pq/z7wZ8AHqj6ArwObJM21/cYEc5kD0Ns77quQazvrjNOmtP3JYjL/Hbqldmi//pJrh+6pv+TaYXL1N20zZ7z+OkGwGBhsWh/kzS+qH6//HOBM4FXbR1va37RNdQnpVaAfeGmCuSwCOOOM36ox7eP76rqrp7T9yaKvb37b23RL7dB+/SXXDt1Tf8m1w+Tqb7II+FlrY50g6AVGm9Z7gJEa/a3tNG3XGmmt+zyevwMupxEix2qMj4iIxpnAIhq/Q39DnSA4QOOX75izefOR+4HqA1r7XwYWSppj+1g1Zmy7g9W4A5JOARYAwzXm8jrwtzXGRUTEm/3GmcCYOn8+ugNYLqlf0jxgJfDkWKft/cCvJV1aNX0YeKK63r8buKlqvwV4olreXq1T9e+ucX8gIiJmwIRBYPsgsBbYBewBBmw/I2m7pGXVsFXABkk/AeYDD1btd9H4K6MXaJxVrKva7wN+V9K+asxHp6ugiIhoT8/oaOtl/IiIKEmeLI6IKFyCICKicAmCiIjCJQgiIgpX60vnop6Jvpyv20k6HXgK+I+2/7HD05lVkv4cuLFa/a7tezs5n9km6VM0vnpmFPiq7b/q8JRmnaQHgDNt39bpubQrZwTTpOaX83UtSe+j8bDfOzs9l9kmaQVwNfBuGv/vL5Z0XWdnNXskXQn8e+AiYBnwR5LU2VnNLknLgVs7PY/JShBMn3/5cj7bR4CxL+crxR00ngeZ6PuiutEg8Ce2/7l6MPLHwHkdntOssf194N9V3yt2Fo0rDUc6O6vZI+m3aRwEfqbTc5msXBqaPhN9OV9Xs70aoLADQQBs7xtblvQ7NC4RXXr8LbqP7TckfRL4U+B/0vgamVJ8mcZDt+d2eiKTlTOC6TPRl/NFl5P0b4D/BfwX2z/t9Hxmm+0/p/Etwufy/99J0tUkrQZetP29Ts9lKhIE0+d4X74XBai+a+t7wMdsP9Tp+cwmSRdUbynE9q+Ax2jcLyjBTcDVkvbQeEfLByVt6PCc2pZLQ9NnB/AJSf00ro+uBO7s7JRiNkg6F/gWcJPtnZ2eTwecT+NlVJfROCu+FvgfnZ3S7LD9H8aWJd0GXGX7P3duRpOTM4Jpcrwv5+vsrGKW/Cnwr4C/krSn+rem05OaLba3A98FngP+N/CU7Uc6O6toR750LiKicDkjiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCvf/AD2wLMTkQwMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# entropy is maximal here because all outcomes are equiprobable - we don't know nothing which has high chance to come.\n",
    "\n",
    "x = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
    "\n",
    "bar_plot(x)\n",
    "\n",
    "entropy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.2464393446710154"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAR8klEQVR4nO3db4xc1XnH8e+uvTQ4NpazDMLmT6pKzYOIRF0w8CJAae0glTalyAZUW8EkAisiVE1CQiLZJCERUZQ0AaVgFIlUJnI3FbUKiGJo69hKHKHIuCokKsnTCqUU7BVYCxLYVcD2bl/M3XQy2J4769mdzZ7vR0Kae8859vOM8W/vnr1zPTAxMYEkae4b7HcBkqSZYeBLUiEMfEkqhIEvSYUw8CWpEPP7XcBx/BZwMTAKHO1zLZL0m2IesBR4BnirfXC2Bv7FwO5+FyFJv6EuB37UfnK2Bv4owOuvH2J8fPZ+TmB4eCFjYwf7XUZflNw7lN2/vc/e3gcHB1iy5N1QZWi72Rr4RwHGxydmdeADs76+6VRy71B2//Y+6x1zK9wf2kpSIQx8SSqEgS9JhTDwJakQtX5oGxFrgU3AEHBvZt7fNn4tcBfNe0CfATZk5tsRcS6wFTgDSGBdZs7eH3FL0hzW8Qo/Is4C7gYuA5YDGyLi/JbxdwP3AR/MzPcD7wJuqoY3A5sz8zxgL3BnT6uXJNVWZ0tnFbAzM1/LzEPANmDN5GB17rcz85WIWEDzav71iBgCrqjmA2wBrutl8ZKk+ups6Szj12/iHwUuaZ2QmYcj4o9pbt/sA/4FOB14IzOPtKw7u5vihocXdjO9LxqNRf0uoW9K7h3K7n829v724aOcMjRv2n+fmeh9unqpE/iDQOsnDQaA8fZJmfkkMBwRXwEeAD7Tto5jrTuRsbGDs/pDDo3GIg4ceLPfZfRFyb1D2f3P1t4bjUV86PbH+l1GTzz+jWum9B4PDg6c8EK5zpbOyzQfxjPpTGD/5EFEvCcirmoZ/zvgAuBVYHFETH6ZWtq6TpI0s+oE/g5gZUQ0qj361cBTLeMDwNbqjhxo7tP/KDMP03wA2g3V+RuBJ3tTtiSpWx0DPzP3ARuBXcCzwEhm7omI7RGxIjPHgA3AP0XEc0AAn62W30rzrp7naT69bdN0NCFJ6qzWffiZOQKMtJ27uuX1o8Cjx1j3InDlyZUoSeoFP2krSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYWYX2dSRKwFNgFDwL2ZeX/b+DXAXcAA8AvgI5n5ekSsB74KvFJNfSIzN/aqeElSfR0DPyLOAu4GLgLeAp6OiF2Z+Xw1fhrwAHBxZu6LiC8BXwT+ClgBfCozvzdN9UuSaqqzpbMK2JmZr2XmIWAbsKZlfAj4eGbuq45/Apxbvb4YWB8RP42IrRGxpFeFS5K6UyfwlwGjLcejwNmTB5k5lpmPAETEqcDngEdb5n4ZuAB4CbivBzVLkqagzh7+IDDRcjwAjLdPiojFwCPAc5n5EEBmXtsy/jXghW6KGx5e2M30vmg0FvW7hL4puXcou/+Se58p0/Ee1wn8l4HLW47PBPa3ToiIpcA/AzuBT1bnFgMfzcx7qmkDwJFuihsbO8j4+ETniX3SaCziwIE3+11GX5TcO5Td/2ztfa59EZrKezw4OHDCC+U6Wzo7gJUR0YiIBcBq4KnJwYiYBzwOPJyZn8jMyYQ+CNwREZdWx7fR/A5AktQHHa/wqztvNgK7gFOABzNzT0RsBz4PnANcCMyPiMkf5u7NzJsj4nrggWpv/z+BG6elC0lSR7Xuw8/MEWCk7dzV1cu9HOc7hczcTfOLgSSpz/ykrSQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpELMrzMpItYCm4Ah4N7MvL9t/BrgLmAA+AXwkcx8PSLOBbYCZwAJrMvMgz2sX5JUU8cr/Ig4C7gbuAxYDmyIiPNbxk8DHgD+JDN/D/gJ8MVqeDOwOTPPA/YCd/a0eklSbXW2dFYBOzPztcw8BGwD1rSMDwEfz8x91fFPgHMjYgi4opoPsAW4ridVS5K6VmdLZxkw2nI8ClwyeZCZY8AjABFxKvA54G+A04E3MvNIy7qzuylueHhhN9P7otFY1O8S3uHtw0c5ZWjetP8+M9F7t73MVO8w/f1PpZe58mc/k3+Os9V0vMd1An8QmGg5HgDG2ydFxGKawf9cZj5UbQVNtE17x7oTGRs7yPh4+y8xezQaizhw4M1+l/EOjcYiPnT7Y/0uoyce/8Y1Xb3HJfcOc6f/qfY+l0wlWwYHB054oVxnS+dlYGnL8ZnA/tYJEbEU2E1zO+fm6vSrwOKImPwyvbR9nSRp5tQJ/B3AyohoRMQCYDXw1ORgFeiPAw9n5icycwIgMw/T/CJwQzX1RuDJXhYvSaqv45ZOZu6LiI3ALuAU4MHM3BMR24HPA+cAFwLzI2Lyh7l7M/Nm4FbgoYjYBPwP8BfT0YQkqbNa9+Fn5ggw0nbu6urlXo7znUJmvghceRL1SZJ6xE/aSlIhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1Ih5teZFBFrgU3AEHBvZt5/nHnfBXZm5pbqeD3wVeCVasoTmbnxZIuWJHWvY+BHxFnA3cBFwFvA0xGxKzOfb5mzDPg2sBLY2bJ8BfCpzPxeT6uWJHWtzpbOKppX7a9l5iFgG7Cmbc464DHg4bbzFwPrI+KnEbE1IpacdMWSpCmpE/jLgNGW41Hg7NYJmfn1zHzwGGtHgS8DFwAvAfdNsU5J0kmqs4c/CEy0HA8A43V+8cy8dvJ1RHwNeKGb4oaHF3YzvS8ajUX9LmHOK/k9tvdyTUf/dQL/ZeDyluMzgf2dFkXEYuCjmXlPdWoAONJNcWNjBxkfn+g8sU8ajUUcOPBmv8t4h7n2F6Wb97jk3mFu9V9y79B9/wCDgwMnvFCus6WzA1gZEY2IWACsBp6qse4gcEdEXFod3wY8UmOdJGkadAz8zNwHbAR2Ac8CI5m5JyK2R8SKE6w7ClwPPBARP6N5l88dvSlbktStWvfhZ+YIMNJ27upjzLup7Xg3cOFJ1CdJ6hE/aStJhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQ8+tMioi1wCZgCLg3M+8/zrzvAjszc0t1fC6wFTgDSGBdZh7sQd2SpC51vMKPiLOAu4HLgOXAhog4v23Osoh4HFjTtnwzsDkzzwP2Anf2pGpJUtfqbOmsonnV/lpmHgK28c5gXwc8Bjw8eSIihoArqvkAW4DrTrZgSdLU1NnSWQaMthyPApe0TsjMrwNExGUtp08H3sjMIy3rzu6muOHhhd1M/5W3Dx/llKF5U1rbrUZj0bT++jPZy2w13e/xbGbv5ZqO/usE/iAw0XI8AIxPYR011/3K2NhBxsfbf4nOGo1FfOj2x7peNxs9/o1rOHDgza7WzLW/KN30X3LvMLf6L7l36L5/gMHBgRNeKNfZ0nkZWNpyfCawv8a6V4HFETF5ebq05jpJ0jSoE/g7gJUR0YiIBcBq4KlOizLzMLAbuKE6dSPw5FQLlSSdnI6Bn5n7gI3ALuBZYCQz90TE9ohY0WH5rTTv6nkeuJzmrZ2SpD6odR9+Zo4AI23nrj7GvJvajl8Erpx6eZKkXvGTtpJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUiPl1JkXEWmATMATcm5n3t40vBx4ETgN+CHwsM49ExHrgq8Ar1dQnMnNjr4qXJNXX8Qo/Is4C7gYuA5YDGyLi/LZpW4HbMvN9wABwS3V+BfCpzFxe/WfYS1Kf1NnSWQXszMzXMvMQsA1YMzkYEe8FTs3MH1entgDXVa8vBtZHxE8jYmtELOld6ZKkbtQJ/GXAaMvxKHB2zfFR4MvABcBLwH1TrlSSdFLq7OEPAhMtxwPAeJ3xzLx28mREfA14oZvihocXdjN9zmo0FvW7hL4quX97L9d09F8n8F8GLm85PhPY3za+tH08IhYDH83Me6rzA8CRboobGzvI+PhE54lt5tr/KAcOvNnV/JL7L7l3mFv9l9w7dN8/wODgwAkvlOts6ewAVkZEIyIWAKuBpyYHM/NF4JcR8YHq1IeBJ4GDwB0RcWl1/jbgka47kCT1RMfAz8x9wEZgF/AsMJKZeyJie0SsqKatA+6JiJ8DC4FvZeZR4HrggYj4GXARcMd0NCFJ6qzWffiZOQKMtJ27uuX1c8Alx1i3G7jwJGuUJPWAn7SVpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUiPl1JkXEWmATMATcm5n3t40vBx4ETgN+CHwsM49ExLnAVuAMIIF1mXmwh/VLkmrqeIUfEWcBdwOXAcuBDRFxftu0rcBtmfk+YAC4pTq/GdicmecBe4E7e1W4JKk7da7wVwE7M/M1gIjYBqwBvlQdvxc4NTN/XM3fAtwVEQ8CVwB/3nL+B8Bna/ye8wAGBwdqNXEsZyw5dcprZ5upvA8l919y7zB3+i+5d5ha/y1r5h1rvE7gLwNGW45HgUs6jJ8NnA68kZlH2s7XsRRgyZJ315z+Tt/ZdNWU1842w8MLu15Tcv8l9w5zp/+Se4ep9d9iKfBC+8k6gT8ITLQcDwDjNcbbz9O27kSeAS6n+UXiaM01klS6eTTD/pljDdYJ/Jdphu+kM4H9beNLjzH+KrA4IuZl5tFqTuu6E3kL+FHNuZKk//eOK/tJdW7L3AGsjIhGRCwAVgNPTQ5m5ovALyPiA9WpDwNPZuZhYDdwQ3X+RuDJKRQvSeqBjoGfmfuAjcAu4FlgJDP3RMT2iFhRTVsH3BMRPwcWAt+qzt9K866e52l+l7Cp1w1IkuoZmJho32aXJM1FftJWkgph4EtSIQx8SSqEgS9Jhaj18DT9uk4Pk5vrIuI04GngTzPzv/tczoyKiC8A11eHT2TmHf2sZyZFxJdoPlZlAvhOZn6zzyXNuIj4a+D0zLyp37VMhVf4Xar5MLk5KyIupfmhuPf1u5aZFhGrgKuA36f5Z39RRFzb36pmRkT8AfBHwAXACuAvIyL6W9XMioiVwPp+13EyDPzu/ephcpl5CJh8mFwpbgE+Tv1PTc8lo8Dtmfl29cHCnwHn9rmmGZGZPwD+sHo21hk0dwcO9beqmRMR76F5ofeVftdyMtzS6V6nh8nNaZl5M0BhF3cAZOZ/TL6OiN+lubXzgeOvmFsy83BE3AV8GvgHYF+fS5pJ36b5AdRz+l3IyfAKv3udHianOS4i3g/8K/CZzPyvftczkzLzC0CDZvDd0mH6nBARNwMvZeb3+13LyTLwu3e8h8WpANUzo74PfC4zH+p3PTMlIs6r/mU7MvN/gX+kuZ9fghuAqyLiWZr/DsifRcQ9fa5pStzS6d4O4IsR0aC5h7ka2NDfkjQTIuIc4FHghszc2e96Ztjv0PyHjS6j+R3uNcDf9rekmZGZH5x8HRE3AVdm5if7V9HUeYXfpeM9TK6/VWmGfBp4F/DNiHi2+u9j/S5qJmTmduAJ4N+BfwOezsy/729V6pYPT5OkQniFL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSrE/wG7rcLRX5jFdQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = [0.1, 0.2, 0.2, 0.2, 0.3]\n",
    "\n",
    "bar_plot(x)\n",
    "\n",
    "entropy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.170950594454669"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAR/UlEQVR4nO3dfYxc1XnH8e+uvTQQG8tZBvkFnKpS8yAiURcM/BGgtHaQSptSZF5UWwESgRURqiYhIZFskpCIKEqaYKXYKBKpTORuKooKiGK7rWMrIUIRuKpJVJKnFUopmBVYaySwq4Dt3f4xs+lksD139u1u9nw/kqW595xjP8/Y+u3ds3eu+8bGxpAkzX39dRcgSZoZBr4kFcLAl6RCGPiSVAgDX5IKMb/uAk7it4CLgWHgeM21SNJvinnAUuBZ4K3Owdka+BcDT9VdhCT9hroc+FHnydka+MMAr79+hNHR2fs5gcHBBYyMHK67jFqU3DuU3b+9z97e+/v7WLz43dDK0E6zNfCPA4yOjs3qwAdmfX3TqeTeoez+7X3WO+FWuD+0laRCGPiSVAgDX5IKYeBLUiEq/dA2ItYBm4ABYHNmbukYvxa4h+Y9oM8CGzLz7YhYAWwHzgYSWJ+Zs/dH3JI0h3W9wo+I5cC9wGXASmBDRJzfNv5u4H7gg5n5fuBdwC2t4a3A1sw8D9gH3D2l1UuSKquypbMG2JOZhzLzCPAIcN34YOvcb2fmqxFxBs2r+dcjYgC4ojUfYBtw/VQWL0mqrsqWzjJ+/Sb+YeCS9gmZeTQi/pjm9s0B4F+As4A3MvNY27pzeilucHBBL9Nr0WgsrLuE2pTcO8zO/t8+epzTBuZN+58z3b1PpI+50jtMXy9VAr8faP+kQR8w2jkpM3cCgxHxFeAB4DMd6zjRulMZGTk8qz/k0Ggs5ODBN+suoxYl9w6zt/9GYyEfuvPxusuYtCe+cU3P7+9c6R0m1j80P2l7qgvlKls6L9N8GM+4JcAr4wcR8Z6IuKpt/O+AC4DXgEURMf5lamn7OknSzKoS+LuB1RHRaO3RrwV2tY33Adtbd+RAc5/+R5l5lOYD0G5snb8J2Dk1ZUuSetU18DPzALAR2AvsB4Yy85mI2BERqzJzBNgA/FNEPAcE8NnW8ttp3tXzPM2nt22ajiYkSd1Vug8/M4eAoY5zV7e9fgx47ATrXgSunFyJkqSp4CdtJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQ86tMioh1wCZgANicmVs6xq8B7gH6gF8AH8nM1yPiZuCrwKutqU9m5sapKl6SVF3XwI+I5cC9wEXAW8DTEbE3M59vjZ8JPABcnJkHIuJLwBeBvwJWAZ/KzO9NU/2SpIqqbOmsAfZk5qHMPAI8AlzXNj4AfDwzD7SOfwKsaL2+GLg5In4aEdsjYvFUFS5J6k2VwF8GDLcdDwPnjB9k5khmPgoQEacDnwMea5v7ZeAC4CXg/imoWZI0AVX28PuBsbbjPmC0c1JELAIeBZ7LzIcAMvPatvGvAS/0Utzg4IJeptei0VhYdwm1Kbl3sP/pVvr7Ox39Vwn8l4HL246XAK+0T4iIpcA/A3uAT7bOLQI+mpn3tab1Acd6KW5k5DCjo2PdJ9ak0VjIwYNv1l1GLUruHWZv/3MpJHt9f+dS79B7/wD9/X2nvFCusqWzG1gdEY2IOANYC+waH4yIecATwMOZ+YnMHE/ow8BdEXFp6/gOmt8BSJJq0PUKv3XnzUZgL3Aa8GBmPhMRO4DPA+cCFwLzI2L8h7n7MvPWiLgBeKC1t/+fwE3T0oUkqatK9+Fn5hAw1HHu6tbLfZzkO4XMfIrmFwNJUs38pK0kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCzK8yKSLWAZuAAWBzZm7pGL8GuAfoA34BfCQzX4+IFcB24GwggfWZeXgK65ckVdT1Cj8ilgP3ApcBK4ENEXF+2/iZwAPAn2Tm7wE/Ab7YGt4KbM3M84B9wN1TWr0kqbIqWzprgD2ZeSgzjwCPANe1jQ8AH8/MA63jnwArImIAuKI1H2AbcP2UVC1J6lmVLZ1lwHDb8TBwyfhBZo4AjwJExOnA54C/Ac4C3sjMY23rzumluMHBBb1Mr0WjsbDuEmozG3t/++hxThuYNyN/1nT3P5O9zEaz8d/XTJqO/qsEfj8w1nbcB4x2ToqIRTSD/7nMfKi1FTTWMe0d605lZOQwo6Odv8Xs0Wgs5ODBN+suoxaztfdGYyEfuvPxusuYEk9845qe3+O5FJIl9w699w/Q3993ygvlKls6LwNL246XAK+0T4iIpcBTNLdzbm2dfg1YFBHjlyhLO9dJkmZOlcDfDayOiEZEnAGsBXaND7YC/Qng4cz8RGaOAWTmUZpfBG5sTb0J2DmVxUuSquu6pZOZByJiI7AXOA14MDOfiYgdwOeBc4ELgfkRMf7D3H2ZeStwO/BQRGwC/gf4i+loQpLUXaX78DNzCBjqOHd16+U+TvKdQma+CFw5ifokSVPET9pKUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiHmV5kUEeuATcAAsDkzt5xk3neBPZm5rXV8M/BV4NXWlCczc+Nki5Yk9a5r4EfEcuBe4CLgLeDpiNibmc+3zVkGfBtYDexpW74K+FRmfm9Kq5Yk9azKls4amlfthzLzCPAIcF3HnPXA48DDHecvBm6OiJ9GxPaIWDzpiiVJE1Il8JcBw23Hw8A57RMy8+uZ+eAJ1g4DXwYuAF4C7p9gnZKkSaqyh98PjLUd9wGjVX7zzLx2/HVEfA14oZfiBgcX9DK9Fo3GwrpLqE3Jvc+Ukt/jknuH6em/SuC/DFzedrwEeKXboohYBHw0M+9rneoDjvVS3MjIYUZHx7pPrEmjsZCDB9+su4xazNbe51pI9Poez6X+S+4deu8foL+/75QXylW2dHYDqyOiERFnAGuBXRXWHQbuiohLW8d3AI9WWCdJmgZdAz8zDwAbgb3AfmAoM5+JiB0RseoU644DNwAPRMTPaN7lc9fUlC1J6lWl+/AzcwgY6jh39Qnm3dJx/BRw4STqkyRNET9pK0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRDzq0yKiHXAJmAA2JyZW04y77vAnszc1jpeAWwHzgYSWJ+Zh6egbklSj7pe4UfEcuBe4DJgJbAhIs7vmLMsIp4ArutYvhXYmpnnAfuAu6ekaklSz6ps6ayhedV+KDOPAI/wzmBfDzwOPDx+IiIGgCta8wG2AddPtmBJ0sRU2dJZBgy3HQ8Dl7RPyMyvA0TEZW2nzwLeyMxjbevO6aW4wcEFvUz/lbePHue0gXkTWturRmPhtP7+E+llpvqf7t5hZv8uZ6OZeI9nq5J7h+npv0rg9wNjbcd9wOgE1lFx3a+MjBxmdLTzt+iu0VjIh+58vOd1s9ET37iGgwff7GlNyf3PtZCYyN/9XFFy79B7/wD9/X2nvFCusqXzMrC07XgJ8EqFda8BiyJi/PJsacV1kqRpUCXwdwOrI6IREWcAa4Fd3RZl5lHgKeDG1qmbgJ0TLVSSNDldAz8zDwAbgb3AfmAoM5+JiB0RsarL8ttp3tXzPHA5zVs7JUk1qHQffmYOAUMd564+wbxbOo5fBK6ceHmSpKniJ20lqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRDzq0yKiHXAJmAA2JyZWzrGVwIPAmcCPwQ+lpnHIuJm4KvAq62pT2bmxqkqXpJUXdcr/IhYDtwLXAasBDZExPkd07YDd2Tm+4A+4LbW+VXApzJzZeuXYS9JNamypbMG2JOZhzLzCPAIcN34YES8Fzg9M3/cOrUNuL71+mLg5oj4aURsj4jFU1e6JKkXVQJ/GTDcdjwMnFNxfBj4MnAB8BJw/4QrlSRNSpU9/H5grO24DxitMp6Z146fjIivAS/0Utzg4IJeps9ZjcbCukuoVcn923u5pqP/KoH/MnB52/ES4JWO8aWd4xGxCPhoZt7XOt8HHOuluJGRw4yOjnWf2GGu/UM5ePDNnuaX3H/JvcPc6r/k3qH3/gH6+/tOeaFcZUtnN7A6IhoRcQawFtg1PpiZLwK/jIgPtE59GNgJHAbuiohLW+fvAB7tuQNJ0pToGviZeQDYCOwF9gNDmflMROyIiFWtaeuB+yLi58AC4FuZeRy4AXggIn4GXATcNR1NSJK6q3QffmYOAUMd565ue/0ccMkJ1j0FXDjJGiVJU8BP2kpSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSrE/CqTImIdsAkYADZn5paO8ZXAg8CZwA+Bj2XmsYhYAWwHzgYSWJ+Zh6ewfklSRV2v8CNiOXAvcBmwEtgQEed3TNsO3JGZ7wP6gNta57cCWzPzPGAfcPdUFS5J6k2VK/w1wJ7MPAQQEY8A1wFfah2/Fzg9M3/cmr8NuCciHgSuAP687fwPgM9W+DPnAfT391Vq4kTOXnz6hNfONhN5H0ruv+TeYe70X3LvMLH+29bMO9F4lcBfBgy3HQ8Dl3QZPwc4C3gjM491nK9iKcDixe+uOP2dvrPpqgmvnW0GBxf0vKbk/kvuHeZO/yX3DhPrv81S4IXOk1UCvx8YazvuA0YrjHeep2PdqTwLXE7zi8TximskqXTzaIb9sycarBL4L9MM33FLgFc6xpeeYPw1YFFEzMvM46057etO5S3gRxXnSpL+3zuu7MdVuS1zN7A6IhoRcQawFtg1PpiZLwK/jIgPtE59GNiZmUeBp4AbW+dvAnZOoHhJ0hToGviZeQDYCOwF9gNDmflMROyIiFWtaeuB+yLi58AC4Fut87fTvKvneZrfJWya6gYkSdX0jY11brNLkuYiP2krSYUw8CWpEAa+JBXCwJekQlR6eJp+XbeHyc11EXEm8DTwp5n53zWXM6Mi4gvADa3DJzPzrjrrmUkR8SWaj1UZA76Tmd+suaQZFxF/DZyVmbfUXctEeIXfo4oPk5uzIuJSmh+Ke1/dtcy0iFgDXAX8Ps2/+4si4tp6q5oZEfEHwB8BFwCrgL+MiKi3qpkVEauBm+uuYzIM/N796mFymXkEGH+YXCluAz5O9U9NzyXDwJ2Z+Xbrg4U/A1bUXNOMyMwfAH/YejbW2TR3B47UW9XMiYj30LzQ+0rdtUyGWzq96/YwuTktM28FKOziDoDM/I/x1xHxuzS3dj5w8hVzS2YejYh7gE8D/wAcqLmkmfRtmh9APbfuQibDK/zedXuYnOa4iHg/8K/AZzLzv+quZyZl5heABs3gu63L9DkhIm4FXsrM79ddy2QZ+L072cPiVIDWM6O+D3wuMx+qu56ZEhHntf5nOzLzf4F/pLmfX4IbgasiYj/N/wfkzyLivpprmhC3dHq3G/hiRDRo7mGuBTbUW5JmQkScCzwG3JiZe+quZ4b9Ds3/2Ogymt/hXgP8bb0lzYzM/OD464i4BbgyMz9ZX0UT5xV+j072MLl6q9IM+TTwLuCbEbG/9etjdRc1EzJzB/Ak8O/AvwFPZ+bf11uVeuXD0ySpEF7hS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgrxf9OtwtG10/cbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = [0.1, 0.1, 0.2, 0.3, 0.3]\n",
    "\n",
    "bar_plot(x)\n",
    "\n",
    "entropy(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><big>Cross-Entropy</big></b>\n",
    "<br /><br />\n",
    "Let $p(X)$ and $q(X)$ be two probability mass functions over the same set of events.\n",
    "<br /><br />\n",
    "<b>Cross Entropy</b> between distributions $p$ and $q$ is defined as the following expectation:<br /><br />\n",
    "$H(p, q)=\\mathbb{E}_p[I_q(X)]=-\\sum_{i=1}^{n}p(x_i)logq(x_i)$   ($x_i$ are possible values of $X$)\n",
    "<br /><br />\n",
    "\n",
    "Note that, $p==q$ implies $H(p, q)=H(p)=H(q)$, i.e. cross-entropy becomes just entropy.\n",
    "\n",
    "<br /><br />\n",
    "\n",
    "Lets see examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cross_entropy(p, q):\n",
    "    return -np.sum(p * np.log2(q))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXiElEQVR4nO3df4xd5X3n8feMMVlcG4cOw2IbiITUfAgrsU4wSRt+7tqLWrIbgswPFSv8iAyyCJXYtkuTtdkmaZyghq0dUjuJ4qxMZCYoi0hIFIO6jq3EFSKULcaRST5Ko60Xm0kZjUPAjkKxZ/aPe6Z7uRl7zp1f19zn85IsnfM8zzn3+cJoPufHnHt6RkdHiYiIcvV2egIREdFZCYKIiMIlCCIiCpcgiIgoXIIgIqJwp3R6Am16G3AJMAgc6/BcIiLeKuYAi4C/A15v7XyrBcElwO5OTyIi4i3qcuBvWxvfakEwCPCLXxxhZOTkff6hr28+w8OHOz2Njii5dii7/pJrh5O7/t7eHs4447eg+h3a6q0WBMcARkZGT+ogAE76+c2kkmuHsusvuXZ4S9Q/7iX13CyOiChcgiAionAJgoiIwiUIIiIKV+tmsaSbgXXAXGCj7U0t/dcCnwR6gP8D3G77F5LOA7YBZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3Y1H868EXgA7b/LbAX+ETVvRnYbPsC4Fngvqr908Bu2+8CvgJ8flqqiYiIttW5NLQC2Gn7kO0jwKPA9U39c4GP2j5Yre8FzpM0F7iiGg+wFbihWv4AjTMCgK8Df1CNj4iIWVbn0tBi3vwQwiDw3rEV28PANwEknQZ8DPgCcCbwqu2jTdud07pP20clvQr0Ay/VmXRf3/w6w8b1z28c49S5cya9fV39/QtmdP+TqaNbaof2a5mt2iH/72fSyVw7nJz111EnCHqB5qckeoCR1kGSFtIIhOdtP1RdUmp9umJsu56W9nH3eTzDw4cn/eBGf/8C/tOfPD6pbU8m3/nv1zI09Fpb23RL7dB+/SXXDt1Tf8m1w+Tqh8aTxSc6gK5zaegAjS8rGnM2LUfukhbR+A6gvcDqqvllYKGksfha1LTdwWo/SDoFWAAM15hLRERMszpBsANYLqlf0jxgJfDkWGf1i/47wDds32N7FMD2GzTC4aZq6C3AE9Xy9mqdqn93NT4iImbZhJeGbB+UtBbYBZwKbLH9jKTtwH8DzgXeA5wiaewm8rO2VwN3AQ9JWgf8X+APq/77gK2S9gGvAKums6iIiKiv1nMEtgeAgZa2a6rFZznOmYXt/cBV47QfAj7YzkQjImJm5MniiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLVeTCPpZmAdMBfYaHvTccZ9Ddhpe6uks4C/aepeCPTbni/pSuAx4MWq7znbt0+2iIiImLwJg0DSEmA9cDHwOvCUpF22X2gasxj4MrAc2Alg+2VgadXfC3wPWFttsgx4wPZnp6+UiIiYjDpnBCtoHOUfApD0KHA98KmmMauAx4Hh4+zjduBX1SsvAS4B/rWkPwT+Efio7RePs21ERMygOvcIFgODTeuDwDnNA2x/zvaW8TaWNIfGmcDHmppfAb5g+yJgO/BIO5OOiIjpU+eMoBcYbVrvAUba+IzfB35q+0djDbbXNC1/SdL9khba/mWdHfb1zW/j47tXf/+CTk+ho0quP7WXaybqrxMEB4DLm9bPBl5q4zM+RNMRf3W/4OPA/baPNY07WneHw8OHGRkZnXjgOLrph2ho6LW2xndT7dBe/SXXDt1Vf8m1Q/v1A/T29pzwALrOpaEdwHJJ/ZLmASuBJ9uYw+8Bu8dWbI8A11X7QdItwA9tH2ljnxERMU0mDALbB2lc498F7AEGbD8jabukZTU+43waZxXNbgXukbSPxo3k1e1NOyIipkut5wiqv/YZaGm7Zpxxt43TNm+ctn3A+2vPMiIiZkyeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtredJxxXwN22t5ard8K3A/8UzXku7bXSjoP2AacBRhYZfvwVAqJiIjJmfCMQNISYD1wGbAUuFPShS1jFkv6DnB9y+bLgD+2vbT6t7Zq3wxstn0B8Cxw3xTriIiISapzRrCCxlH+IQBJj9L4hf+ppjGrgMeB4ZZtLwF+R9J/BZ4H/gg4DFwBfKgasxX4PvBnkyshIiKmos49gsXAYNP6IHBO8wDbn7O9ZZxtB4G/AC4CXgT+GjgTeNX20ePtLyIiZk+dM4JeYLRpvQcYqbNz29eNLUv6S+BnwL0t+6Pu/sb09c1vZ3jX6u9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLE20kaSHwEdsbqqYe4CjwMrBQ0hzbx4BFdfbXbHj4MCMjrVlSTzf9EA0NvdbW+G6qHdqrv+TaobvqL7l2aL9+gN7enhMeQNe5NLQDWC6pX9I8YCXwZI3tDgP3SnpftX438E3bbwC7gZuq9luAJ2rsLyIiZsCEQWD7ILAW2AXsAQZsPyNpu6RlJ9juGHAj8EVJPwYupnFZCOAuGn999AKNs411UysjIiImq9ZzBLYHgIGWtmvGGXdby/pu4D3jjNsPXNXGPCMiYobkyeKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMLVekOZpJtpvE5yLrDR9qbjjPsasNP21mr9UmADcCowTONl9vslXQk8BrxYbfqc7dunUkhEREzOhEEgaQmwnsY7h18HnpK0y/YLTWMWA18GlgM7mzZ/GPig7b2SPgI8CFwLLAMesP3ZaaskIiImpc6loRU0jvIP2T4CPApc3zJmFfA48I2xBklvA9bZ3ls17QXOq5YvAa6WtFfStyWdO5UiIiJi8uoEwWJgsGl9EDineYDtz9ne0tL2uu1tAJJ6gU8A36q6XwG+YPsiYDvwyKRmHxERU1bnHkEvMNq03gOM1P0ASacCD1Wf9RkA22vG+m1/SdL9khba/mWdffb1za/78V2tv39Bp6fQUSXXn9rLNRP11wmCA8DlTetnAy/V2bmk+cC3adwovtb2G9XZwceB+20faxp+tN6UYXj4MCMjoxMPHEc3/RANDb3W1vhuqh3aq7/k2qG76i+5dmi/foDe3p4THkDXuTS0A1guqV/SPGAl8GTNz98G/ANwk+3XAWyPANdV+0HSLcAPq/sPERExyyYMAtsHgbXALmAPMGD7GUnbJS073naS3k3jL4QuBf5e0h5J26vuW4F7JO0DbgdWT7GOiIiYpFrPEdgeAAZa2q4ZZ9xtTcvP0bifMN7+9gHvb2eiERExM/JkcURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4Wq9oUzSzcA6YC6w0fam44z7GrDT9tZq/Twa7y0+CzCwyvZhSW8HHgbOB4aAG23/fIq1RETEJEx4RiBpCbAeuAxYCtwp6cKWMYslfQe4vmXzzcBm2xcAzwL3Ve2fBnbbfhfwFeDzU6oiIiImrc6loRU0jvIP2T4CPMpv/sJfBTwOfGOsQdJc4IpqPMBW4IZq+QM0zggAvg78QTU+IiJmWZ0gWAwMNq0PAuc0D7D9OdtbWrY7E3jV9tFxtvuXfVb9rwL97U09IiKmQ517BL3AaNN6DzAyie1o2q6npb3uPgHo65tfd2hX6+9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLNbZ7GVgoaY7tY8Cipu0OVvs5IOkUYAEwXHfSw8OHGRlpzZh6uumHaGjotbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2nPAAus6loR3Ackn9kuYBK4EnJ9rI9hvAbuCmqukW4IlqeXu1TtW/uxofERGzbMIgsH0QWAvsAvYAA7afkbRd0rIJNr+Lxl8ZvUDjrGJd1X4f8LuS9lVjPjrZAiIiYmpqPUdgewAYaGm7Zpxxt7Ws7weuGmfcIeCDbcwzIiJmSJ4sjogoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXK03lEm6mcZrJucCG21vaulfCmwBTgd+AKwBfhv4m6ZhC4F+2/MlXQk8BrxY9T1n+/apFBIREZMzYRBIWgKsBy4GXgeekrTL9gtNw7YBq20/LemrwB22vwgsrfbRC3yPxruPAZYBD9j+7PSVEhERk1Hn0tAKYKftQ7aPAI8C1491SnoHcJrtp6umrcANLfu4HfhV9e5jgEuAqyXtlfRtSedOpYiIiJi8OkGwGBhsWh8EzqnbL2kOjTOBjzWNeQX4gu2LgO3AI+1NOyIipkudewS9wGjTeg8w0kb/7wM/tf2jsQbba5qWvyTpfkkLbf+yzqT7+ubXGdb1+vsXdHoKHVVy/am9XDNRf50gOABc3rR+NvBSS/+iE/R/iKYj/up+wceB+20faxp3tOacGR4+zMjI6MQDx9FNP0RDQ6+1Nb6baof26i+5duiu+kuuHdqvH6C3t+eEB9B1Lg3tAJZL6pc0D1gJPDnWaXs/8GtJl1ZNHwaeaNr+94DdTeNHgOuq/SDpFuCH1f2HiIiYZRMGge2DNK7x7wL2AAO2n5G0XdKyatgqYIOknwDzgQebdnE+jbOGZrcC90jaR+NG8uqplREREZNV6zmC6q99Blrarmlafh5473G2nTdO2z7g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtre1NK/FNgCnA78AFhj+6ikW4H7gX+qhn7X9lpJ5wHbgLMAA6tsH56OgiIioj0TnhFIWgKsBy4DlgJ3SrqwZdg24G7b7wR6gDuq9mXAH9teWv1bW7VvBjbbvgB4Frhv6qVERMRk1Lk0tALYafuQ7SPAo8D1Y52S3gGcZvvpqmkrcEO1fAlwq6QfSdom6QxJc4Erqv20jo+IiFlWJwgWA4NN64PAOTX7B4G/AC4CXgT+GjgTeNX20ePsLyIiZlGdewS9wGjTeg8wUqff9nVjjZL+EvgZcG/LeFr2N6G+vvntDO9a/f0LOj2Fjiq5/tRerpmov04QHAAub1o/G3ippX9Ra7+khcBHbG+o2nuAo8DLwEJJc2wfq7Zt3t+EhocPMzLSmiX1dNMP0dDQa22N76baob36S64duqv+kmuH9usH6O3tOeEBdJ1LQzuA5ZL6Jc0DVgJPjnXa3g/8WtKlVdOHgSeAw8C9kt5Xtd8NfNP2G8Bu4Kaq/ZZqfEREdMCEQWD7ILAW2AXsAQZsPyNpu6Rl1bBVwAZJPwHmAw9WR/s3Al+U9GPgYhqXhQDuovHXRy/QONtYN51FRUREfbWeI7A9AAy0tF3TtPw88N5xttsNvGec9v3AVW3ONSIiZkCeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJupnG6yTnAhttb2rpXwpsAU4HfgCssX20eo/xBuBUYJjGy+z3S7oSeAx4sdrFc7Zvn46CIiKiPROeEUhaAqwHLgOW0njX8IUtw7YBd9t+J9AD3FG1Pwystr20Wn6wal8GPGB7afUvIRAR0SF1Lg2tAHbaPmT7CPAocP1Yp6R3AKfZfrpq2grcIOltwDrbe6v2vcB51fIlwNWS9kr6tqRzp6GWiIiYhDqXhhYDg03rg7z5RfXj9Z9j+3UaZwpI6gU+AXyrGvMK8A3bj0laAzwCXFp30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN6z3ASN1+SacCD1Wf9RkA22vG+m1/SdL9khba/mWdSQ8PH2ZkZHTigePoph+ioaHX2hrfTbVDe/WXXDt0V/0l1w7t1w/Q29tzwgPoOpeGDgCLmtbPBl6q0y9pPvAkjRC41vYbknolrZU0p+VzjtaYS0RETLM6QbADWC6pX9I8YCWNX+4A2N4P/Lr6CyGADwNPVMvbgH8AbqouFWF7BLiu2g+SbgF+WN1/iIiIWTZhENg+CKwFdgF7gAHbz0jaLmlZNWwVsEHST4D5wIOS3g1cS+Pa/99L2iNpezX+VuAeSfuA24HV01pVRETUVus5AtsDwEBL2zVNy8/z5hvIAM/RuF8w3v72Ae9va6YRETEj8mRxREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThar2hTNLNwDpgLrDR9qaW/qXAFuB04AfAGttHJZ1H473FZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3YMmwbcLftd9J4PeUdVftmYLPtC4Bngfuq9k8Du22/C/gK8PmpFhIREZNT54xgBbDT9iEASY8C1wOfqtbfAZxm++lq/Fbgk5K2AFcAH2pq/z7wZ8AHqj6ArwObJM21/cYEc5kD0Ns77quQazvrjNOmtP3JYjL/Hbqldmi//pJrh+6pv+TaYXL1N20zZ7z+OkGwGBhsWh/kzS+qH6//HOBM4FXbR1va37RNdQnpVaAfeGmCuSwCOOOM36ox7eP76rqrp7T9yaKvb37b23RL7dB+/SXXDt1Tf8m1w+Tqb7II+FlrY50g6AVGm9Z7gJEa/a3tNG3XGmmt+zyevwMupxEix2qMj4iIxpnAIhq/Q39DnSA4QOOX75izefOR+4HqA1r7XwYWSppj+1g1Zmy7g9W4A5JOARYAwzXm8jrwtzXGRUTEm/3GmcCYOn8+ugNYLqlf0jxgJfDkWKft/cCvJV1aNX0YeKK63r8buKlqvwV4olreXq1T9e+ucX8gIiJmwIRBYPsgsBbYBewBBmw/I2m7pGXVsFXABkk/AeYDD1btd9H4K6MXaJxVrKva7wN+V9K+asxHp6ugiIhoT8/oaOtl/IiIKEmeLI6IKFyCICKicAmCiIjCJQgiIgpX60vnop6Jvpyv20k6HXgK+I+2/7HD05lVkv4cuLFa/a7tezs5n9km6VM0vnpmFPiq7b/q8JRmnaQHgDNt39bpubQrZwTTpOaX83UtSe+j8bDfOzs9l9kmaQVwNfBuGv/vL5Z0XWdnNXskXQn8e+AiYBnwR5LU2VnNLknLgVs7PY/JShBMn3/5cj7bR4CxL+crxR00ngeZ6PuiutEg8Ce2/7l6MPLHwHkdntOssf194N9V3yt2Fo0rDUc6O6vZI+m3aRwEfqbTc5msXBqaPhN9OV9Xs70aoLADQQBs7xtblvQ7NC4RXXr8LbqP7TckfRL4U+B/0vgamVJ8mcZDt+d2eiKTlTOC6TPRl/NFl5P0b4D/BfwX2z/t9Hxmm+0/p/Etwufy/99J0tUkrQZetP29Ts9lKhIE0+d4X74XBai+a+t7wMdsP9Tp+cwmSRdUbynE9q+Ax2jcLyjBTcDVkvbQeEfLByVt6PCc2pZLQ9NnB/AJSf00ro+uBO7s7JRiNkg6F/gWcJPtnZ2eTwecT+NlVJfROCu+FvgfnZ3S7LD9H8aWJd0GXGX7P3duRpOTM4Jpcrwv5+vsrGKW/Cnwr4C/krSn+rem05OaLba3A98FngP+N/CU7Uc6O6toR750LiKicDkjiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCvf/AD2wLMTkQwMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
    "\n",
    "bar_plot(p)\n",
    "\n",
    "cross_entropy(p, p) == entropy(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXiElEQVR4nO3df4xd5X3n8feMMVlcG4cOw2IbiITUfAgrsU4wSRt+7tqLWrIbgswPFSv8iAyyCJXYtkuTtdkmaZyghq0dUjuJ4qxMZCYoi0hIFIO6jq3EFSKULcaRST5Ko60Xm0kZjUPAjkKxZ/aPe6Z7uRl7zp1f19zn85IsnfM8zzn3+cJoPufHnHt6RkdHiYiIcvV2egIREdFZCYKIiMIlCCIiCpcgiIgoXIIgIqJwp3R6Am16G3AJMAgc6/BcIiLeKuYAi4C/A15v7XyrBcElwO5OTyIi4i3qcuBvWxvfakEwCPCLXxxhZOTkff6hr28+w8OHOz2Njii5dii7/pJrh5O7/t7eHs4447eg+h3a6q0WBMcARkZGT+ogAE76+c2kkmuHsusvuXZ4S9Q/7iX13CyOiChcgiAionAJgoiIwiUIIiIKV+tmsaSbgXXAXGCj7U0t/dcCnwR6gP8D3G77F5LOA7YBZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3Y1H868EXgA7b/LbAX+ETVvRnYbPsC4Fngvqr908Bu2+8CvgJ8flqqiYiIttW5NLQC2Gn7kO0jwKPA9U39c4GP2j5Yre8FzpM0F7iiGg+wFbihWv4AjTMCgK8Df1CNj4iIWVbn0tBi3vwQwiDw3rEV28PANwEknQZ8DPgCcCbwqu2jTdud07pP20clvQr0Ay/VmXRf3/w6w8b1z28c49S5cya9fV39/QtmdP+TqaNbaof2a5mt2iH/72fSyVw7nJz111EnCHqB5qckeoCR1kGSFtIIhOdtP1RdUmp9umJsu56W9nH3eTzDw4cn/eBGf/8C/tOfPD6pbU8m3/nv1zI09Fpb23RL7dB+/SXXDt1Tf8m1w+Tqh8aTxSc6gK5zaegAjS8rGnM2LUfukhbR+A6gvcDqqvllYKGksfha1LTdwWo/SDoFWAAM15hLRERMszpBsANYLqlf0jxgJfDkWGf1i/47wDds32N7FMD2GzTC4aZq6C3AE9Xy9mqdqn93NT4iImbZhJeGbB+UtBbYBZwKbLH9jKTtwH8DzgXeA5wiaewm8rO2VwN3AQ9JWgf8X+APq/77gK2S9gGvAKums6iIiKiv1nMEtgeAgZa2a6rFZznOmYXt/cBV47QfAj7YzkQjImJm5MniiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLVeTCPpZmAdMBfYaHvTccZ9Ddhpe6uks4C/aepeCPTbni/pSuAx4MWq7znbt0+2iIiImLwJg0DSEmA9cDHwOvCUpF22X2gasxj4MrAc2Alg+2VgadXfC3wPWFttsgx4wPZnp6+UiIiYjDpnBCtoHOUfApD0KHA98KmmMauAx4Hh4+zjduBX1SsvAS4B/rWkPwT+Efio7RePs21ERMygOvcIFgODTeuDwDnNA2x/zvaW8TaWNIfGmcDHmppfAb5g+yJgO/BIO5OOiIjpU+eMoBcYbVrvAUba+IzfB35q+0djDbbXNC1/SdL9khba/mWdHfb1zW/j47tXf/+CTk+ho0quP7WXaybqrxMEB4DLm9bPBl5q4zM+RNMRf3W/4OPA/baPNY07WneHw8OHGRkZnXjgOLrph2ho6LW2xndT7dBe/SXXDt1Vf8m1Q/v1A/T29pzwALrOpaEdwHJJ/ZLmASuBJ9uYw+8Bu8dWbI8A11X7QdItwA9tH2ljnxERMU0mDALbB2lc498F7AEGbD8jabukZTU+43waZxXNbgXukbSPxo3k1e1NOyIipkut5wiqv/YZaGm7Zpxxt43TNm+ctn3A+2vPMiIiZkyeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtredJxxXwN22t5ard8K3A/8UzXku7bXSjoP2AacBRhYZfvwVAqJiIjJmfCMQNISYD1wGbAUuFPShS1jFkv6DnB9y+bLgD+2vbT6t7Zq3wxstn0B8Cxw3xTriIiISapzRrCCxlH+IQBJj9L4hf+ppjGrgMeB4ZZtLwF+R9J/BZ4H/gg4DFwBfKgasxX4PvBnkyshIiKmos49gsXAYNP6IHBO8wDbn7O9ZZxtB4G/AC4CXgT+GjgTeNX20ePtLyIiZk+dM4JeYLRpvQcYqbNz29eNLUv6S+BnwL0t+6Pu/sb09c1vZ3jX6u9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLE20kaSHwEdsbqqYe4CjwMrBQ0hzbx4BFdfbXbHj4MCMjrVlSTzf9EA0NvdbW+G6qHdqrv+TaobvqL7l2aL9+gN7enhMeQNe5NLQDWC6pX9I8YCXwZI3tDgP3SnpftX438E3bbwC7gZuq9luAJ2rsLyIiZsCEQWD7ILAW2AXsAQZsPyNpu6RlJ9juGHAj8EVJPwYupnFZCOAuGn999AKNs411UysjIiImq9ZzBLYHgIGWtmvGGXdby/pu4D3jjNsPXNXGPCMiYobkyeKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMLVekOZpJtpvE5yLrDR9qbjjPsasNP21mr9UmADcCowTONl9vslXQk8BrxYbfqc7dunUkhEREzOhEEgaQmwnsY7h18HnpK0y/YLTWMWA18GlgM7mzZ/GPig7b2SPgI8CFwLLAMesP3ZaaskIiImpc6loRU0jvIP2T4CPApc3zJmFfA48I2xBklvA9bZ3ls17QXOq5YvAa6WtFfStyWdO5UiIiJi8uoEwWJgsGl9EDineYDtz9ne0tL2uu1tAJJ6gU8A36q6XwG+YPsiYDvwyKRmHxERU1bnHkEvMNq03gOM1P0ASacCD1Wf9RkA22vG+m1/SdL9khba/mWdffb1za/78V2tv39Bp6fQUSXXn9rLNRP11wmCA8DlTetnAy/V2bmk+cC3adwovtb2G9XZwceB+20faxp+tN6UYXj4MCMjoxMPHEc3/RANDb3W1vhuqh3aq7/k2qG76i+5dmi/foDe3p4THkDXuTS0A1guqV/SPGAl8GTNz98G/ANwk+3XAWyPANdV+0HSLcAPq/sPERExyyYMAtsHgbXALmAPMGD7GUnbJS073naS3k3jL4QuBf5e0h5J26vuW4F7JO0DbgdWT7GOiIiYpFrPEdgeAAZa2q4ZZ9xtTcvP0bifMN7+9gHvb2eiERExM/JkcURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4Wq9oUzSzcA6YC6w0fam44z7GrDT9tZq/Twa7y0+CzCwyvZhSW8HHgbOB4aAG23/fIq1RETEJEx4RiBpCbAeuAxYCtwp6cKWMYslfQe4vmXzzcBm2xcAzwL3Ve2fBnbbfhfwFeDzU6oiIiImrc6loRU0jvIP2T4CPMpv/sJfBTwOfGOsQdJc4IpqPMBW4IZq+QM0zggAvg78QTU+IiJmWZ0gWAwMNq0PAuc0D7D9OdtbWrY7E3jV9tFxtvuXfVb9rwL97U09IiKmQ517BL3AaNN6DzAyie1o2q6npb3uPgHo65tfd2hX6+9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLNbZ7GVgoaY7tY8Cipu0OVvs5IOkUYAEwXHfSw8OHGRlpzZh6uumHaGjotbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2nPAAus6loR3Ackn9kuYBK4EnJ9rI9hvAbuCmqukW4IlqeXu1TtW/uxofERGzbMIgsH0QWAvsAvYAA7afkbRd0rIJNr+Lxl8ZvUDjrGJd1X4f8LuS9lVjPjrZAiIiYmpqPUdgewAYaGm7Zpxxt7Ws7weuGmfcIeCDbcwzIiJmSJ4sjogoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXK03lEm6mcZrJucCG21vaulfCmwBTgd+AKwBfhv4m6ZhC4F+2/MlXQk8BrxY9T1n+/apFBIREZMzYRBIWgKsBy4GXgeekrTL9gtNw7YBq20/LemrwB22vwgsrfbRC3yPxruPAZYBD9j+7PSVEhERk1Hn0tAKYKftQ7aPAI8C1491SnoHcJrtp6umrcANLfu4HfhV9e5jgEuAqyXtlfRtSedOpYiIiJi8OkGwGBhsWh8EzqnbL2kOjTOBjzWNeQX4gu2LgO3AI+1NOyIipkudewS9wGjTeg8w0kb/7wM/tf2jsQbba5qWvyTpfkkLbf+yzqT7+ubXGdb1+vsXdHoKHVVy/am9XDNRf50gOABc3rR+NvBSS/+iE/R/iKYj/up+wceB+20faxp3tOacGR4+zMjI6MQDx9FNP0RDQ6+1Nb6baof26i+5duiu+kuuHdqvH6C3t+eEB9B1Lg3tAJZL6pc0D1gJPDnWaXs/8GtJl1ZNHwaeaNr+94DdTeNHgOuq/SDpFuCH1f2HiIiYZRMGge2DNK7x7wL2AAO2n5G0XdKyatgqYIOknwDzgQebdnE+jbOGZrcC90jaR+NG8uqplREREZNV6zmC6q99Blrarmlafh5473G2nTdO2z7g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtre1NK/FNgCnA78AFhj+6ikW4H7gX+qhn7X9lpJ5wHbgLMAA6tsH56OgiIioj0TnhFIWgKsBy4DlgJ3SrqwZdg24G7b7wR6gDuq9mXAH9teWv1bW7VvBjbbvgB4Frhv6qVERMRk1Lk0tALYafuQ7SPAo8D1Y52S3gGcZvvpqmkrcEO1fAlwq6QfSdom6QxJc4Erqv20jo+IiFlWJwgWA4NN64PAOTX7B4G/AC4CXgT+GjgTeNX20ePsLyIiZlGdewS9wGjTeg8wUqff9nVjjZL+EvgZcG/LeFr2N6G+vvntDO9a/f0LOj2Fjiq5/tRerpmov04QHAAub1o/G3ippX9Ra7+khcBHbG+o2nuAo8DLwEJJc2wfq7Zt3t+EhocPMzLSmiX1dNMP0dDQa22N76baob36S64duqv+kmuH9usH6O3tOeEBdJ1LQzuA5ZL6Jc0DVgJPjnXa3g/8WtKlVdOHgSeAw8C9kt5Xtd8NfNP2G8Bu4Kaq/ZZqfEREdMCEQWD7ILAW2AXsAQZsPyNpu6Rl1bBVwAZJPwHmAw9WR/s3Al+U9GPgYhqXhQDuovHXRy/QONtYN51FRUREfbWeI7A9AAy0tF3TtPw88N5xttsNvGec9v3AVW3ONSIiZkCeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJupnG6yTnAhttb2rpXwpsAU4HfgCssX20eo/xBuBUYJjGy+z3S7oSeAx4sdrFc7Zvn46CIiKiPROeEUhaAqwHLgOW0njX8IUtw7YBd9t+J9AD3FG1Pwystr20Wn6wal8GPGB7afUvIRAR0SF1Lg2tAHbaPmT7CPAocP1Yp6R3AKfZfrpq2grcIOltwDrbe6v2vcB51fIlwNWS9kr6tqRzp6GWiIiYhDqXhhYDg03rg7z5RfXj9Z9j+3UaZwpI6gU+AXyrGvMK8A3bj0laAzwCXFp30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN6z3ASN1+SacCD1Wf9RkA22vG+m1/SdL9khba/mWdSQ8PH2ZkZHTigePoph+ioaHX2hrfTbVDe/WXXDt0V/0l1w7t1w/Q29tzwgPoOpeGDgCLmtbPBl6q0y9pPvAkjRC41vYbknolrZU0p+VzjtaYS0RETLM6QbADWC6pX9I8YCWNX+4A2N4P/Lr6CyGADwNPVMvbgH8AbqouFWF7BLiu2g+SbgF+WN1/iIiIWTZhENg+CKwFdgF7gAHbz0jaLmlZNWwVsEHST4D5wIOS3g1cS+Pa/99L2iNpezX+VuAeSfuA24HV01pVRETUVus5AtsDwEBL2zVNy8/z5hvIAM/RuF8w3v72Ae9va6YRETEj8mRxREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThar2hTNLNwDpgLrDR9qaW/qXAFuB04AfAGttHJZ1H473FZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3YMmwbcLftd9J4PeUdVftmYLPtC4Bngfuq9k8Du22/C/gK8PmpFhIREZNT54xgBbDT9iEASY8C1wOfqtbfAZxm++lq/Fbgk5K2AFcAH2pq/z7wZ8AHqj6ArwObJM21/cYEc5kD0Ns77quQazvrjNOmtP3JYjL/Hbqldmi//pJrh+6pv+TaYXL1N20zZ7z+OkGwGBhsWh/kzS+qH6//HOBM4FXbR1va37RNdQnpVaAfeGmCuSwCOOOM36ox7eP76rqrp7T9yaKvb37b23RL7dB+/SXXDt1Tf8m1w+Tqb7II+FlrY50g6AVGm9Z7gJEa/a3tNG3XGmmt+zyevwMupxEix2qMj4iIxpnAIhq/Q39DnSA4QOOX75izefOR+4HqA1r7XwYWSppj+1g1Zmy7g9W4A5JOARYAwzXm8jrwtzXGRUTEm/3GmcCYOn8+ugNYLqlf0jxgJfDkWKft/cCvJV1aNX0YeKK63r8buKlqvwV4olreXq1T9e+ucX8gIiJmwIRBYPsgsBbYBewBBmw/I2m7pGXVsFXABkk/AeYDD1btd9H4K6MXaJxVrKva7wN+V9K+asxHp6ugiIhoT8/oaOtl/IiIKEmeLI6IKFyCICKicAmCiIjCJQgiIgpX60vnop6Jvpyv20k6HXgK+I+2/7HD05lVkv4cuLFa/a7tezs5n9km6VM0vnpmFPiq7b/q8JRmnaQHgDNt39bpubQrZwTTpOaX83UtSe+j8bDfOzs9l9kmaQVwNfBuGv/vL5Z0XWdnNXskXQn8e+AiYBnwR5LU2VnNLknLgVs7PY/JShBMn3/5cj7bR4CxL+crxR00ngeZ6PuiutEg8Ce2/7l6MPLHwHkdntOssf194N9V3yt2Fo0rDUc6O6vZI+m3aRwEfqbTc5msXBqaPhN9OV9Xs70aoLADQQBs7xtblvQ7NC4RXXr8LbqP7TckfRL4U+B/0vgamVJ8mcZDt+d2eiKTlTOC6TPRl/NFl5P0b4D/BfwX2z/t9Hxmm+0/p/Etwufy/99J0tUkrQZetP29Ts9lKhIE0+d4X74XBai+a+t7wMdsP9Tp+cwmSRdUbynE9q+Ax2jcLyjBTcDVkvbQeEfLByVt6PCc2pZLQ9NnB/AJSf00ro+uBO7s7JRiNkg6F/gWcJPtnZ2eTwecT+NlVJfROCu+FvgfnZ3S7LD9H8aWJd0GXGX7P3duRpOTM4Jpcrwv5+vsrGKW/Cnwr4C/krSn+rem05OaLba3A98FngP+N/CU7Uc6O6toR750LiKicDkjiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCvf/AD2wLMTkQwMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXxElEQVR4nO3df4xd5X3n8feMMQTXxqHDsNgGIiE1H8JKrBNM0oafu/ailuyGIPNDxQrgCJBFqETaLE3WZptkQ4IathBSO4lCViYyE5RFJBDFsF3HVuIKEZctxpFJPkqjrRfbkzIah4DJ4mLP7B/3THq5GTPnzq9r7vN5SZbueZ7nnPt87ZE/95wz5z49o6OjREREuXo7PYGIiOisBEFEROESBBERhUsQREQULkEQEVG44zo9gTadAJwPDAJHOjyXiIi3ijnAIuDvgEOtnW+1IDgf2N7pSUREvEVdBPxta+NbLQgGAX75y1cZGTl2n3/o65vP8PDBTk+jI0quHcquv+Ta4diuv7e3h5NP/h2o/g9t9VYLgiMAIyOjx3QQAMf8/GZSybVD2fWXXDu8Jeof95J6bhZHRBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4d5qzxHEFCw46UTedsLM/5P39y+Y8fd47dBhXnn5/834+0SUIEFQkLedcBz/8c8e6/Q0psV3/9sVvNLpSUR0iVwaiogoXIIgIqJwCYKIiMIlCCIiClfrZrGk64B1wFzgPtvrW/qvAD4N9AD/B1ht+5eSzgQ2AacCBlbZPijp7cBDwFnAEHCN7V9MU00REdGGCc8IJC0B7gIuBJYCt0g6p6n/JODLwAds/xtgF/CpqnsDsMH22cAzwJ1V+2eB7bbfBXwN+OK0VBMREW2rc2loBbDV9gHbrwKPAFc19c8FPmp7X7W9CzhT0lzg4mo8wEbg6ur1B2icEQB8E/ijanxERMyyOkGwmDeuajMInD62YXvY9rcBJJ0IfAL4DnAK8LLtw+Ps95tjVv0vA/2TLyMiIiarzj2CXqB52Z0eYKR1kKSFwLeB52w/WF1Sal2uZ2y/npb2cY95NH198+sO7ZjZeLq2dO38Hf/z60c4fu6cGZzNv5jpf/vJ1DJb9ZdcOxyb9ddRJwj20ljweMxpwP7mAZIWAf8T2Ap8rGp+EVgoaY7tI8Cipv32VcfZK+k4YAEwXHfSw8MHj+kl4fr7FzA0dOw999pt4dTO33F//4Kueqq63Z+vbqm/5NphcvVDY83iN/sAXefS0BZguaR+SfOAlcCTY52S5gDfBb5l+3bbowC2Xwe2A9dWQ68Hnqheb662qfq3V+MjImKWTXhGYHufpLXANuB44AHbOyRtBv4LcAbwHuA4SWM3kZ+xfRNwK/CgpHXA/wX+uOq/E9goaTfwErBqOouKiIj6aj1HYHsAGGhpu7x6+QxHObOwvQe4dJz2A8AH25loRETMjDxZHBFRuARBREThEgQREYVLEEREFC5BEBFRuOKWquyWdXuzZm9ETJfigqBb1u3Nmr0RMV1yaSgionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionC1niyWdB2wDpgL3Gd7/VHGfQPYanujpFOBv2nqXgj0254v6RLgUeCFqu9Z26snW0REREzehEEgaQlwF3AecAh4StI22883jVkMfBVYTmMBe2y/CCyt+nuB7wNrq12WAffY/vz0lRIREZNR54xgBY1P+QcAJD0CXAV8pmnMKuAxYPgox1gN/Lpa8hLgfOBfSfpj4B+Bj9p+4Sj7RkTEDKpzj2AxMNi0PQic3jzA9hdsPzDezpLm0DgT+ERT80vAl2yfC2wGHm5n0hERMX3qnBH0AqNN2z3ASBvv8YfAz2z/eKzB9pqm11+RdLekhbZ/VeeAfX3z23j77jXTX3V9rCu5/tRerpmov04Q7AUuato+Ddjfxnt8iKZP/NX9gk8Cd9s+0jTucN0DDg8fZGRkdOKB4+imH6Khofa+iLqbaof26i+5duiu+kuuHdqvH6C3t+dNP0DXuTS0BVguqV/SPGAl8GQbc/gDYPvYhu0R4MrqOEi6HviR7VfbOGZEREyTCYPA9j4a1/i3ATuBAds7JG2WtKzGe5xF46yi2Q3A7ZJ207iRfFN7046IiOlS6zmC6rd9BlraLh9n3I3jtM0bp2038P7as4yIiBmTJ4sjIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjC1VqYRtJ1wDpgLnCf7fVHGfcNYKvtjdX2DcDdwD9VQ75ne62kM4FNwKmAgVW2D06lkIiImJwJzwgkLQHuAi4ElgK3SDqnZcxiSd8FrmrZfRnwp7aXVn/WVu0bgA22zwaeAe6cYh0RETFJdc4IVtD4lH8AQNIjNP7D/0zTmFXAY8Bwy77nA78n6T8DzwF/AhwELgY+VI3ZCPwA+PPJlRAREVNRJwgWA4NN24PAe5sH2P4CgKQLW/YdBO4BngI+B/w18HHgZduHm8ac3s6k+/rmtzO8a/X3L+j0FDqq5PpTe7lmov46QdALjDZt9wAjdQ5u+8qx15L+Evg5cEfL8ah7vDHDwwcZGWk9RD3d9EM0NPRKW+O7qXZor/6Sa4fuqr/k2qH9+gF6e3ve9AN0nd8a2gssato+Ddg/0U6SFkr6WFNTD3AYeBFYKGlO1b6ozvEiImJm1AmCLcBySf2S5gErgSdr7HcQuEPS+6rt24Bv234d2A5cW7VfDzzR3rQjImK6TBgEtvcBa4FtwE5gwPYOSZslLXuT/Y4A1wBflvQT4Dwal4UAbqXx20fPAxfR+NXUiIjogFrPEdgeAAZa2i4fZ9yNLdvbgfeMM24PcGkb84yIiBmSJ4sjIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLXWI5B0HY3FY+YC99lef5Rx3wC22t5YbV8A3AscDwwDH7G9R9IlwKPAC9Wuz9pePZVCIiJiciYMAklLgLtorDB2CHhK0jbbzzeNWQx8FVgObG3a/SHgg7Z3SfoIcD9wBbAMuMf256etkoiImJQ6l4ZW0PiUf8D2q8AjwFUtY1YBjwHfGmuQdAKwzvauqmkXcGb1+nzgMkm7JD0u6YypFBEREZNXJwgWA4NN24PA6c0DbH/B9gMtbYdsbwKQ1At8CvhO1f0S8CXb5wKbgYcnNfuIiJiyOvcIeoHRpu0eYKTuG0g6Hniweq/PAdheM9Zv+yuS7pa00Pav6hyzr29+3bfvav39Czo9hY4quf7UXq6ZqL9OEOwFLmraPg3YX+fgkuYDj9O4UXyF7ders4NPAnfbPtI0/HC9KcPw8EFGRkYnHjiObvohGhp6pa3x3VQ7tFd/ybVDd9Vfcu3Qfv0Avb09b/oBus6loS3Ackn9kuYBK4Ena77/JuAfgGttHwKwPQJcWR0HSdcDP6ruP0RExCybMAhs7wPWAtuAncCA7R2SNktadrT9JL2bxm8IXQD8vaSdkjZX3TcAt0vaDawGbppiHRERMUm1niOwPQAMtLRdPs64G5teP0vjfsJ4x9sNvL+diUZExMzIk8UREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThai1MI+k6YB0wF7jP9vqjjPsGsNX2xmr7TBrLVZ4KGFhl+6CktwMPAWcBQ8A1tn8xxVoiImISJjwjkLQEuAu4EFgK3CLpnJYxiyV9F7iqZfcNwAbbZwPPAHdW7Z8Fttt+F/A14ItTqiIiIiatzqWhFTQ+5R+oFph/hN/+D38V8BjwrbEGSXOBi6vxABuBq6vXH6BxRgDwTeCPqvERETHL6gTBYmCwaXsQOL15gO0v2H6gZb9TgJdtHx5nv98cs+p/Gehvb+oRETEd6twj6AVGm7Z7gJFJ7EfTfq2L2tc9JgB9ffPrDu1q/f0LOj2Fjiq5/tRerpmov04Q7AUuato+DdhfY78XgYWS5tg+Aixq2m9fdZy9ko4DFgDDdSc9PHyQkZHWjKmnm36IhoZeaWt8N9UO7dVfcu3QXfWXXDu0Xz9Ab2/Pm36ArnNpaAuwXFK/pHnASuDJiXay/TqwHbi2aroeeKJ6vbnapurfXo2PiIhZNmEQ2N4HrAW2ATuBAds7JG2WtGyC3W+l8VtGz9M4q1hXtd8J/L6k3dWYj062gIiImJpazxHYHgAGWtouH2fcjS3be4BLxxl3APhgG/OMiIgZkieLIyIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwtVamEbSdTRWF5sL3Gd7fUv/UuAB4CTgh8Aa4HeBv2kathDotz1f0iXAo8ALVd+ztldPpZCIiJicCYNA0hLgLuA84BDwlKRttp9vGrYJuMn205K+Dtxs+8vA0uoYvcD3aSx5CbAMuMf256evlIiImIw6l4ZWAFttH7D9KvAIcNVYp6R3ACfafrpq2ghc3XKM1cCvqyUvAc4HLpO0S9Ljks6YShERETF5dS4NLQYGm7YHgfdO0H/62IakOTTOBK5oGvMS8C3bj0paAzwMXFB30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN2z3ASBv9fwj8zPaPxxpsr2l6/RVJd0taaPtXdSY9PHyQkZHRiQeOo5t+iIaGXmlrfDfVDu3VX3Lt0F31l1w7tF8/QG9vz5t+gK5zaWgvsKhp+zRgfxv9H6LxiR9o3C+QtLY6U2h2uMZcIiJimtUJgi3Ackn9kuYBK4Enxzpt7wFekzR2aefDwBNN+/8BsL1p/AhwZXUcJF0P/Ki6/xAREbNswiCwvY/GNf5twE5gwPYOSZslLauGrQLulfRTYD5wf9MhzqJx1tDsBuB2Sbtp3Ei+aWplRETEZNV6jqD6bZ+BlrbLm14/xxtvIDePmzdO227g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjC1VqPQNJ1wDpgLnCf7fUt/UuBB4CTgB8Ca2wflnQDcDfwT9XQ79leK+lMYBNwKmBgle2D01FQRES0Z8IzAklLgLuAC4GlwC2SzmkZtgm4zfY7aSxef3PVvgz4U9tLqz9rq/YNwAbbZwPPAHdOvZSIiJiMOpeGVgBbbR+o1hV+BLhqrFPSO4ATbT9dNW0Erq5enw/cIOnHkjZJOlnSXODi6jit4yMiYpbVCYLFwGDT9iBwes3+QeC/AucCLwB/DZwCvGz78FGOFxERs6jOPYJeYLRpuwcYqdNv+8qxRkl/CfwcuKNlPC3Hm1Bf3/x2hnet/v4FnZ5CR5Vcf2ov10zUXycI9gIXNW2fBuxv6V/U2i9pIfAR2/dW7T3AYeBFYKGkObaPVPs2H29Cw8MHGRlpzZJ6uumHaGjolbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2vOkH6DqXhrYAyyX1S5oHrASeHOu0vQd4TdIFVdOHgSeAg8Adkt5Xtd8GfNv268B24Nqq/fpqfEREdMCEQWB7H7AW2AbsBAZs75C0WdKyatgq4F5JPwXmA/dXn/avAb4s6SfAeTQuCwHcSuO3j56ncbaxbjqLioiI+mo9R2B7ABhoabu86fVzwHvH2W878J5x2vcAl7Y514iImAF5sjgionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcrYVpJF1HYxWxucB9tte39C8FHgBOAn4IrLF9uFq+8l7geGCYxhrGeyRdAjwKvFAd4lnbq6ejoIiIaM+EZwSSlgB3ARcCS2ksMXlOy7BNwG2230ljkfqbq/aHgJtsL61e31+1LwPusb20+pMQiIjokDqXhlYAW20fsP0q8Ahw1VinpHcAJ9p+umraCFwt6QRgne1dVfsu4Mzq9fnAZZJ2SXpc0hnTUEtERExCnSBYDAw2bQ8Cp0/Ub/uQ7U0AknqBTwHfqca8BHzJ9rnAZuDhSc0+IiKmrM49gl5gtGm7Bxip2y/peODB6r0+B2B7zVi/7a9IulvSQtu/qjPpvr75dYZ1vf7+BZ2eQkeVXH9qL9dM1F8nCPYCFzVtnwbsb+lfNF6/pPnA4zRuFF9h+/Xq7OCTwN22jzTtd7jupIeHDzIyMjrxwHF00w/R0NArbY3vptqhvfpLrh26q/6Sa4f26wfo7e150w/QdS4NbQGWS+qXNA9YCTw51ml7D/Ba9RtCAB8GnqhebwL+AbjW9qFq/AhwZXUcJF0P/Ki6/xAREbNswiCwvQ9YC2wDdgIDtndI2ixpWTVsFXCvpJ8C84H7Jb0buAK4APh7STslba7G3wDcLmk3sBq4aVqrioiI2mo9R2B7ABhoabu86fVzwHtbdnuWxv2C8Y63G3h/WzONiIgZkSeLIyIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwtVamEbSdcA6YC5wn+31Lf1LgQeAk4AfAmtsH5Z0Jo3lKk8FDKyyfVDS24GHgLOAIeAa27+YppoiIqINE54RSFoC3AVcCCwFbpF0TsuwTcBttt9JY1Wym6v2DcAG22cDzwB3Vu2fBbbbfhfwNeCLUy0kIiImp84ZwQpgq+0DAJIeAa4CPlNtvwM40fbT1fiNwKclPQBcDHyoqf0HwJ8DH6j6AL4JrJc01/brE8xlDkBv77grYNZ26sknTmn/Y8Vk/h66pXZov/6Sa4fuqb/k2mFy9TftM2e8/jpBsBgYbNoe5I3rE4/XfzpwCvCy7cMt7W/Yp7qE9DLQD+yfYC6LAE4++XdqTPvovr7usintf6zo65vf9j7dUju0X3/JtUP31F9y7TC5+pssAn7e2lgnCHqB0abtHmCkRn9rO037tUZa6zGP5u+Ai2iEyJEa4yMionEmsIjG/6G/pU4Q7KXxn++Y03jjJ/e91Ru09r8ILJQ0x/aRaszYfvuqcXslHQcsAIZrzOUQ8Lc1xkVExBv91pnAmDq/ProFWC6pX9I8YCXw5Fin7T3Aa5IuqJo+DDxRXe/fDlxbtV8PPFG93lxtU/Vvr3F/ICIiZsCEQWB7H7AW2AbsBAZs75C0WdKyatgq4F5JPwXmA/dX7bfS+C2j52mcVayr2u8Efl/S7mrMR6eroIiIaE/P6GjrZfyIiChJniyOiChcgiAionAJgoiIwiUIIiIKV+tL56Keib6cr9tJOgl4CvgPtv+xw9OZVZL+Arim2vye7Ts6OZ/ZJukzNL56ZhT4uu2/6vCUZp2ke4BTbN/Y6bm0K2cE06Tml/N1LUnvo/Gw3zs7PZfZJmkFcBnwbhr/9udJurKzs5o9ki4B/h1wLrAM+BNJ6uysZpek5cANnZ7HZCUIps9vvpzP9qvA2JfzleJmGs+DTPR9Ud1oEPgz2/9cPRj5E+DMDs9p1tj+AfBvq+8VO5XGlYZXOzur2SPpd2l8CPxcp+cyWbk0NH0m+nK+rmb7JoDCPggCYHv32GtJv0fjEtEFR9+j+9h+XdKngY8D/4PG18iU4qs0Hro9o9MTmaycEUyfib6cL7qcpH8N/C/gP9n+WafnM9ts/wWNbxE+g39Zk6SrSboJeMH29zs9l6lIEEyfo335XhSg+q6t7wOfsP1gp+czmySdXa1SiO1fA4/SuF9QgmuByyTtpLFGywcl3dvhObUtl4amzxbgU5L6aVwfXQnc0tkpxWyQdAbwHeBa21s7PZ8OOIvGYlQX0jgrvgL4752d0uyw/e/HXku6EbjU9sc6N6PJyRnBNDnal/N1dlYxSz4OvA34K0k7qz9rOj2p2WJ7M/A94FngfwNP2X64s7OKduRL5yIiCpczgoiIwiUIIiIKlyCIiChcgiAionAJgoiIwiUIIiIKlyCIiChcgiAionD/Hzf2MEMjIkL+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "eps = 1e-2\n",
    "p = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
    "q = [0.2-eps, 0.2+eps, 0.2, 0.2, 0.2]\n",
    "\n",
    "\n",
    "bar_plot(p)\n",
    "plt.show()\n",
    "bar_plot(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0007222507104756026"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abs(cross_entropy(p, q) - entropy(p))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXiElEQVR4nO3df4xd5X3n8feMMVlcG4cOw2IbiITUfAgrsU4wSRt+7tqLWrIbgswPFSv8iAyyCJXYtkuTtdkmaZyghq0dUjuJ4qxMZCYoi0hIFIO6jq3EFSKULcaRST5Ko60Xm0kZjUPAjkKxZ/aPe6Z7uRl7zp1f19zn85IsnfM8zzn3+cJoPufHnHt6RkdHiYiIcvV2egIREdFZCYKIiMIlCCIiCpcgiIgoXIIgIqJwp3R6Am16G3AJMAgc6/BcIiLeKuYAi4C/A15v7XyrBcElwO5OTyIi4i3qcuBvWxvfakEwCPCLXxxhZOTkff6hr28+w8OHOz2Njii5dii7/pJrh5O7/t7eHs4447eg+h3a6q0WBMcARkZGT+ogAE76+c2kkmuHsusvuXZ4S9Q/7iX13CyOiChcgiAionAJgoiIwiUIIiIKV+tmsaSbgXXAXGCj7U0t/dcCnwR6gP8D3G77F5LOA7YBZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3Y1H868EXgA7b/LbAX+ETVvRnYbPsC4Fngvqr908Bu2+8CvgJ8flqqiYiIttW5NLQC2Gn7kO0jwKPA9U39c4GP2j5Yre8FzpM0F7iiGg+wFbihWv4AjTMCgK8Df1CNj4iIWVbn0tBi3vwQwiDw3rEV28PANwEknQZ8DPgCcCbwqu2jTdud07pP20clvQr0Ay/VmXRf3/w6w8b1z28c49S5cya9fV39/QtmdP+TqaNbaof2a5mt2iH/72fSyVw7nJz111EnCHqB5qckeoCR1kGSFtIIhOdtP1RdUmp9umJsu56W9nH3eTzDw4cn/eBGf/8C/tOfPD6pbU8m3/nv1zI09Fpb23RL7dB+/SXXDt1Tf8m1w+Tqh8aTxSc6gK5zaegAjS8rGnM2LUfukhbR+A6gvcDqqvllYKGksfha1LTdwWo/SDoFWAAM15hLRERMszpBsANYLqlf0jxgJfDkWGf1i/47wDds32N7FMD2GzTC4aZq6C3AE9Xy9mqdqn93NT4iImbZhJeGbB+UtBbYBZwKbLH9jKTtwH8DzgXeA5wiaewm8rO2VwN3AQ9JWgf8X+APq/77gK2S9gGvAKums6iIiKiv1nMEtgeAgZa2a6rFZznOmYXt/cBV47QfAj7YzkQjImJm5MniiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLVeTCPpZmAdMBfYaHvTccZ9Ddhpe6uks4C/aepeCPTbni/pSuAx4MWq7znbt0+2iIiImLwJg0DSEmA9cDHwOvCUpF22X2gasxj4MrAc2Alg+2VgadXfC3wPWFttsgx4wPZnp6+UiIiYjDpnBCtoHOUfApD0KHA98KmmMauAx4Hh4+zjduBX1SsvAS4B/rWkPwT+Efio7RePs21ERMygOvcIFgODTeuDwDnNA2x/zvaW8TaWNIfGmcDHmppfAb5g+yJgO/BIO5OOiIjpU+eMoBcYbVrvAUba+IzfB35q+0djDbbXNC1/SdL9khba/mWdHfb1zW/j47tXf/+CTk+ho0quP7WXaybqrxMEB4DLm9bPBl5q4zM+RNMRf3W/4OPA/baPNY07WneHw8OHGRkZnXjgOLrph2ho6LW2xndT7dBe/SXXDt1Vf8m1Q/v1A/T29pzwALrOpaEdwHJJ/ZLmASuBJ9uYw+8Bu8dWbI8A11X7QdItwA9tH2ljnxERMU0mDALbB2lc498F7AEGbD8jabukZTU+43waZxXNbgXukbSPxo3k1e1NOyIipkut5wiqv/YZaGm7Zpxxt43TNm+ctn3A+2vPMiIiZkyeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtredJxxXwN22t5ard8K3A/8UzXku7bXSjoP2AacBRhYZfvwVAqJiIjJmfCMQNISYD1wGbAUuFPShS1jFkv6DnB9y+bLgD+2vbT6t7Zq3wxstn0B8Cxw3xTriIiISapzRrCCxlH+IQBJj9L4hf+ppjGrgMeB4ZZtLwF+R9J/BZ4H/gg4DFwBfKgasxX4PvBnkyshIiKmos49gsXAYNP6IHBO8wDbn7O9ZZxtB4G/AC4CXgT+GjgTeNX20ePtLyIiZk+dM4JeYLRpvQcYqbNz29eNLUv6S+BnwL0t+6Pu/sb09c1vZ3jX6u9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLE20kaSHwEdsbqqYe4CjwMrBQ0hzbx4BFdfbXbHj4MCMjrVlSTzf9EA0NvdbW+G6qHdqrv+TaobvqL7l2aL9+gN7enhMeQNe5NLQDWC6pX9I8YCXwZI3tDgP3SnpftX438E3bbwC7gZuq9luAJ2rsLyIiZsCEQWD7ILAW2AXsAQZsPyNpu6RlJ9juGHAj8EVJPwYupnFZCOAuGn999AKNs411UysjIiImq9ZzBLYHgIGWtmvGGXdby/pu4D3jjNsPXNXGPCMiYobkyeKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMLVekOZpJtpvE5yLrDR9qbjjPsasNP21mr9UmADcCowTONl9vslXQk8BrxYbfqc7dunUkhEREzOhEEgaQmwnsY7h18HnpK0y/YLTWMWA18GlgM7mzZ/GPig7b2SPgI8CFwLLAMesP3ZaaskIiImpc6loRU0jvIP2T4CPApc3zJmFfA48I2xBklvA9bZ3ls17QXOq5YvAa6WtFfStyWdO5UiIiJi8uoEwWJgsGl9EDineYDtz9ne0tL2uu1tAJJ6gU8A36q6XwG+YPsiYDvwyKRmHxERU1bnHkEvMNq03gOM1P0ASacCD1Wf9RkA22vG+m1/SdL9khba/mWdffb1za/78V2tv39Bp6fQUSXXn9rLNRP11wmCA8DlTetnAy/V2bmk+cC3adwovtb2G9XZwceB+20faxp+tN6UYXj4MCMjoxMPHEc3/RANDb3W1vhuqh3aq7/k2qG76i+5dmi/foDe3p4THkDXuTS0A1guqV/SPGAl8GTNz98G/ANwk+3XAWyPANdV+0HSLcAPq/sPERExyyYMAtsHgbXALmAPMGD7GUnbJS073naS3k3jL4QuBf5e0h5J26vuW4F7JO0DbgdWT7GOiIiYpFrPEdgeAAZa2q4ZZ9xtTcvP0bifMN7+9gHvb2eiERExM/JkcURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4Wq9oUzSzcA6YC6w0fam44z7GrDT9tZq/Twa7y0+CzCwyvZhSW8HHgbOB4aAG23/fIq1RETEJEx4RiBpCbAeuAxYCtwp6cKWMYslfQe4vmXzzcBm2xcAzwL3Ve2fBnbbfhfwFeDzU6oiIiImrc6loRU0jvIP2T4CPMpv/sJfBTwOfGOsQdJc4IpqPMBW4IZq+QM0zggAvg78QTU+IiJmWZ0gWAwMNq0PAuc0D7D9OdtbWrY7E3jV9tFxtvuXfVb9rwL97U09IiKmQ517BL3AaNN6DzAyie1o2q6npb3uPgHo65tfd2hX6+9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLNbZ7GVgoaY7tY8Cipu0OVvs5IOkUYAEwXHfSw8OHGRlpzZh6uumHaGjotbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2nPAAus6loR3Ackn9kuYBK4EnJ9rI9hvAbuCmqukW4IlqeXu1TtW/uxofERGzbMIgsH0QWAvsAvYAA7afkbRd0rIJNr+Lxl8ZvUDjrGJd1X4f8LuS9lVjPjrZAiIiYmpqPUdgewAYaGm7Zpxxt7Ws7weuGmfcIeCDbcwzIiJmSJ4sjogoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXK03lEm6mcZrJucCG21vaulfCmwBTgd+AKwBfhv4m6ZhC4F+2/MlXQk8BrxY9T1n+/apFBIREZMzYRBIWgKsBy4GXgeekrTL9gtNw7YBq20/LemrwB22vwgsrfbRC3yPxruPAZYBD9j+7PSVEhERk1Hn0tAKYKftQ7aPAI8C1491SnoHcJrtp6umrcANLfu4HfhV9e5jgEuAqyXtlfRtSedOpYiIiJi8OkGwGBhsWh8EzqnbL2kOjTOBjzWNeQX4gu2LgO3AI+1NOyIipkudewS9wGjTeg8w0kb/7wM/tf2jsQbba5qWvyTpfkkLbf+yzqT7+ubXGdb1+vsXdHoKHVVy/am9XDNRf50gOABc3rR+NvBSS/+iE/R/iKYj/up+wceB+20faxp3tOacGR4+zMjI6MQDx9FNP0RDQ6+1Nb6baof26i+5duiu+kuuHdqvH6C3t+eEB9B1Lg3tAJZL6pc0D1gJPDnWaXs/8GtJl1ZNHwaeaNr+94DdTeNHgOuq/SDpFuCH1f2HiIiYZRMGge2DNK7x7wL2AAO2n5G0XdKyatgqYIOknwDzgQebdnE+jbOGZrcC90jaR+NG8uqplREREZNV6zmC6q99Blrarmlafh5473G2nTdO2z7g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtre1NK/FNgCnA78AFhj+6ikW4H7gX+qhn7X9lpJ5wHbgLMAA6tsH56OgiIioj0TnhFIWgKsBy4DlgJ3SrqwZdg24G7b7wR6gDuq9mXAH9teWv1bW7VvBjbbvgB4Frhv6qVERMRk1Lk0tALYafuQ7SPAo8D1Y52S3gGcZvvpqmkrcEO1fAlwq6QfSdom6QxJc4Erqv20jo+IiFlWJwgWA4NN64PAOTX7B4G/AC4CXgT+GjgTeNX20ePsLyIiZlGdewS9wGjTeg8wUqff9nVjjZL+EvgZcG/LeFr2N6G+vvntDO9a/f0LOj2Fjiq5/tRerpmov04QHAAub1o/G3ippX9Ra7+khcBHbG+o2nuAo8DLwEJJc2wfq7Zt3t+EhocPMzLSmiX1dNMP0dDQa22N76baob36S64duqv+kmuH9usH6O3tOeEBdJ1LQzuA5ZL6Jc0DVgJPjnXa3g/8WtKlVdOHgSeAw8C9kt5Xtd8NfNP2G8Bu4Kaq/ZZqfEREdMCEQWD7ILAW2AXsAQZsPyNpu6Rl1bBVwAZJPwHmAw9WR/s3Al+U9GPgYhqXhQDuovHXRy/QONtYN51FRUREfbWeI7A9AAy0tF3TtPw88N5xttsNvGec9v3AVW3ONSIiZkCeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJupnG6yTnAhttb2rpXwpsAU4HfgCssX20eo/xBuBUYJjGy+z3S7oSeAx4sdrFc7Zvn46CIiKiPROeEUhaAqwHLgOW0njX8IUtw7YBd9t+J9AD3FG1Pwystr20Wn6wal8GPGB7afUvIRAR0SF1Lg2tAHbaPmT7CPAocP1Yp6R3AKfZfrpq2grcIOltwDrbe6v2vcB51fIlwNWS9kr6tqRzp6GWiIiYhDqXhhYDg03rg7z5RfXj9Z9j+3UaZwpI6gU+AXyrGvMK8A3bj0laAzwCXFp30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN6z3ASN1+SacCD1Wf9RkA22vG+m1/SdL9khba/mWdSQ8PH2ZkZHTigePoph+ioaHX2hrfTbVDe/WXXDt0V/0l1w7t1w/Q29tzwgPoOpeGDgCLmtbPBl6q0y9pPvAkjRC41vYbknolrZU0p+VzjtaYS0RETLM6QbADWC6pX9I8YCWNX+4A2N4P/Lr6CyGADwNPVMvbgH8AbqouFWF7BLiu2g+SbgF+WN1/iIiIWTZhENg+CKwFdgF7gAHbz0jaLmlZNWwVsEHST4D5wIOS3g1cS+Pa/99L2iNpezX+VuAeSfuA24HV01pVRETUVus5AtsDwEBL2zVNy8/z5hvIAM/RuF8w3v72Ae9va6YRETEj8mRxREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThar2hTNLNwDpgLrDR9qaW/qXAFuB04AfAGttHJZ1H473FZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3YMmwbcLftd9J4PeUdVftmYLPtC4Bngfuq9k8Du22/C/gK8PmpFhIREZNT54xgBbDT9iEASY8C1wOfqtbfAZxm++lq/Fbgk5K2AFcAH2pq/z7wZ8AHqj6ArwObJM21/cYEc5kD0Ns77quQazvrjNOmtP3JYjL/Hbqldmi//pJrh+6pv+TaYXL1N20zZ7z+OkGwGBhsWh/kzS+qH6//HOBM4FXbR1va37RNdQnpVaAfeGmCuSwCOOOM36ox7eP76rqrp7T9yaKvb37b23RL7dB+/SXXDt1Tf8m1w+Tqb7II+FlrY50g6AVGm9Z7gJEa/a3tNG3XGmmt+zyevwMupxEix2qMj4iIxpnAIhq/Q39DnSA4QOOX75izefOR+4HqA1r7XwYWSppj+1g1Zmy7g9W4A5JOARYAwzXm8jrwtzXGRUTEm/3GmcCYOn8+ugNYLqlf0jxgJfDkWKft/cCvJV1aNX0YeKK63r8buKlqvwV4olreXq1T9e+ucX8gIiJmwIRBYPsgsBbYBewBBmw/I2m7pGXVsFXABkk/AeYDD1btd9H4K6MXaJxVrKva7wN+V9K+asxHp6ugiIhoT8/oaOtl/IiIKEmeLI6IKFyCICKicAmCiIjCJQgiIgpX60vnop6Jvpyv20k6HXgK+I+2/7HD05lVkv4cuLFa/a7tezs5n9km6VM0vnpmFPiq7b/q8JRmnaQHgDNt39bpubQrZwTTpOaX83UtSe+j8bDfOzs9l9kmaQVwNfBuGv/vL5Z0XWdnNXskXQn8e+AiYBnwR5LU2VnNLknLgVs7PY/JShBMn3/5cj7bR4CxL+crxR00ngeZ6PuiutEg8Ce2/7l6MPLHwHkdntOssf194N9V3yt2Fo0rDUc6O6vZI+m3aRwEfqbTc5msXBqaPhN9OV9Xs70aoLADQQBs7xtblvQ7NC4RXXr8LbqP7TckfRL4U+B/0vgamVJ8mcZDt+d2eiKTlTOC6TPRl/NFl5P0b4D/BfwX2z/t9Hxmm+0/p/Etwufy/99J0tUkrQZetP29Ts9lKhIE0+d4X74XBai+a+t7wMdsP9Tp+cwmSRdUbynE9q+Ax2jcLyjBTcDVkvbQeEfLByVt6PCc2pZLQ9NnB/AJSf00ro+uBO7s7JRiNkg6F/gWcJPtnZ2eTwecT+NlVJfROCu+FvgfnZ3S7LD9H8aWJd0GXGX7P3duRpOTM4Jpcrwv5+vsrGKW/Cnwr4C/krSn+rem05OaLba3A98FngP+N/CU7Uc6O6toR750LiKicDkjiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCvf/AD2wLMTkQwMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAR8klEQVR4nO3dfYxc1XnH8e+uvTQ4NpazDPIbpKrUPIhI1AUDfwQorR2k0qQU2YBqK5hEYEWEqklISCSbJCQiipImWCkYRSIVRO6moqiAKLbbOrYSRygyrgqJSvK0QikFewXWggR2FbC92z9mNpmMX+bOvswse74fCWnuPeeY5xnbv7179s5139jYGJKk2a+/1wVIkrrDwJekQhj4klQIA1+SCmHgS1Ih5va6gFP4HeASYBg43uNaJOmdYg6wBHgGeKt1cKYG/iXA3l4XIUnvUFcAP249OVMDfxjg9dePMDo6cz8nMDg4n5GRw70uoydK7h3K7t/eZ27v/f19LFr0bmhkaKuZGvjHAUZHx2Z04AMzvr7pVHLvUHb/9j7jnXQr3B/aSlIhDHxJKoSBL0mFMPAlqRCVfmgbEeuAzcAAsCUz728Zvw64m/o9oM8AGzPz7Yg4D9gGnAMksD4zZ+6PuCVpFmt7hR8Ry4B7gMuBFcDGiLigafzdwH3ABzPz/cC7gJsbw1uBrZl5PrAfuGtKq5ckVVZlS2c1sDszX8vMI8CjwNrxwca5383MVyJiHvWr+dcjYgC4sjEf4CHg+qksXpJUXZUtnaX89k38w8ClzRMy82hE/Cn17ZsDwL8CZwNvZOaxpnXLOylucHB+J9N7olZb0OsSTvD20eOcMTBn2v8/3ei9W71MxEz8ve8We39nqhL4/UDzJw36gNHWSZm5AxiMiK8CDwCfbVnHydadzsjI4Rn9IYdabQGHDr3Z6zJOUKst4MN3PNHrMqbEk9+8dsa+xzOxrm6w95nbe39/32kvlKts6bxM/WE84xYDB8cPIuI9EXF10/jfAxcCrwILI2L88mxJ8zpJUndVCfxdwKqIqDX26NcAO5vG+4BtjTtyoL5P/+PMPEr9AWg3Ns7fBOyYmrIlSZ1qG/iZeQDYBOwBngWGMnNfRGyPiJWZOQJsBP45Ip4DAvhcY/lt1O/qeZ7609s2T0cTkqT2Kt2Hn5lDwFDLuWuaXj8OPH6SdS8CV02uREnSVPCTtpJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUiLlVJkXEOmAzMABsycz7W8avBe4G+oBfAh/NzNcjYgPwNeCVxtSnMnPTVBUvSaqubeBHxDLgHuBi4C3g6YjYk5nPN8bPAh4ALsnMAxHxZeBLwF8DK4FPZ+b3p6l+SVJFVbZ0VgO7M/O1zDwCPAqsbRofAD6RmQcaxz8Fzmu8vgTYEBE/i4htEbFoqgqXJHWmSuAvBYabjoeB5eMHmTmSmY8BRMSZwOeBx5vmfgW4EHgJuG8KapYkTUCVPfx+YKzpuA8YbZ0UEQuBx4DnMvNhgMy8rmn868ALnRQ3ODi/k+k9Uast6HUJs95MfY9nal3dYO/vTFUC/2XgiqbjxcDB5gkRsQT4F2A38KnGuYXAxzLz3sa0PuBYJ8WNjBxmdHSs/cQeqdUWcOjQm70u4wTv5D+QJzNT3+OZWFc32PvM7b2/v++0F8pVtnR2AasiohYR84A1wM7xwYiYAzwJPJKZn8zM8YQ+DNwZEZc1jm+n/h2AJKkH2l7hN+682QTsAc4AHszMfRGxHfgCcC5wETA3IsZ/mLs/M2+JiBuABxp7+/8F3DQtXUiS2qp0H35mDgFDLeeuabzczym+U8jMvdS/GEiSesxP2kpSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSrE3CqTImIdsBkYALZk5v0t49cCdwN9wC+Bj2bm6xFxHrANOAdIYH1mHp7C+iVJFbW9wo+IZcA9wOXACmBjRFzQNH4W8ADwZ5n5B8BPgS81hrcCWzPzfGA/cNeUVi9JqqzKls5qYHdmvpaZR4BHgbVN4wPAJzLzQOP4p8B5ETEAXNmYD/AQcP2UVC1J6liVLZ2lwHDT8TBw6fhBZo4AjwFExJnA54G/Bc4G3sjMY03rlndS3ODg/E6m90SttqDXJcx6nbzHbx89zhkDc6axmt+Y7t/7ifTSrf7tfXpNVy9VAr8fGGs67gNGWydFxELqwf9cZj7c2Aoaa5l2wrrTGRk5zOho6y8xc9RqCzh06M1el3GC2fZFqJP3uFZbwIfveGIaq+meJ795bcd/vmZL/yX3DhPrH6C/v++0F8pVtnReBpY0HS8GDjZPiIglwF7q2zm3NE6/CiyMiPEvU0ta10mSuqdK4O8CVkVELSLmAWuAneODjUB/EngkMz+ZmWMAmXmU+heBGxtTbwJ2TGXxkqTq2m7pZOaBiNgE7AHOAB7MzH0RsR34AnAucBEwNyLGf5i7PzNvAW4DHo6IzcD/An85HU1IktqrdB9+Zg4BQy3nrmm83M8pvlPIzBeBqyZRnyRpivhJW0kqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqxNwqkyJiHbAZGAC2ZOb9p5j3PWB3Zj7UON4AfA14pTHlqczcNNmiJUmdaxv4EbEMuAe4GHgLeDoi9mTm801zlgLfAVYBu5uWrwQ+nZnfn9KqJUkdq7Kls5r6VftrmXkEeBRY2zJnPfAE8EjL+UuADRHxs4jYFhGLJl2xJGlCqgT+UmC46XgYWN48ITO/kZkPnmTtMPAV4ELgJeC+CdYpSZqkKnv4/cBY03EfMFrlF8/M68ZfR8TXgRc6KW5wcH4n03uiVlvQ6xJmvZLfY3sv13T0XyXwXwauaDpeDBxstygiFgIfy8x7G6f6gGOdFDcycpjR0bH2E3ukVlvAoUNv9rqME8y2vyidvMcl9w6zq/+Se4fO+wfo7+877YVylS2dXcCqiKhFxDxgDbCzwrrDwJ0RcVnj+HbgsQrrJEnToG3gZ+YBYBOwB3gWGMrMfRGxPSJWnmbdceAG4IGI+Dn1u3zunJqyJUmdqnQffmYOAUMt5645ybybW473AhdNoj5J0hTxk7aSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKMbfKpIhYB2wGBoAtmXn/KeZ9D9idmQ81js8DtgHnAAmsz8zDU1C3JKlDba/wI2IZcA9wObAC2BgRF7TMWRoRTwJrW5ZvBbZm5vnAfuCuKalaktSxKls6q6lftb+WmUeARzkx2NcDTwCPjJ+IiAHgysZ8gIeA6ydbsCRpYqps6SwFhpuOh4FLmydk5jcAIuLyptNnA29k5rGmdcs7KW5wcH4n03/t7aPHOWNgzoTWdqpWWzCtv343e5mppvs9nsnsvVzT0X+VwO8HxpqO+4DRCayj4rpfGxk5zOho6y/RXq22gA/f8UTH62aiJ795LYcOvdnRmtn2F6WT/kvuHWZX/yX3Dp33D9Df33faC+UqWzovA0uajhcDByusexVYGBHjl6dLKq6TJE2DKoG/C1gVEbWImAesAXa2W5SZR4G9wI2NUzcBOyZaqCRpctoGfmYeADYBe4BngaHM3BcR2yNiZZvlt1G/q+d54Arqt3ZKknqg0n34mTkEDLWcu+Yk825uOX4RuGri5UmSpoqftJWkQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQsytMiki1gGbgQFgS2be3zK+AngQOAv4EfDxzDwWERuArwGvNKY+lZmbpqp4SVJ1ba/wI2IZcA9wObAC2BgRF7RM2wbcnpnvA/qAWxvnVwKfzswVjf8Me0nqkSpbOquB3Zn5WmYeAR4F1o4PRsR7gTMz8yeNUw8B1zdeXwJsiIifRcS2iFg0daVLkjpRJfCXAsNNx8PA8orjw8BXgAuBl4D7JlypJGlSquzh9wNjTcd9wGiV8cy8bvxkRHwdeKGT4gYH53cyfdaq1Rb0uoSeKrl/ey/XdPRfJfBfBq5oOl4MHGwZX9I6HhELgY9l5r2N833AsU6KGxk5zOjoWPuJLWbbH5RDh97saH7J/ZfcO8yu/kvuHTrvH6C/v++0F8pVtnR2AasiohYR84A1wM7xwcx8EfhVRHygceojwA7gMHBnRFzWOH878FjHHUiSpkTbwM/MA8AmYA/wLDCUmfsiYntErGxMWw/cGxG/AOYD387M48ANwAMR8XPgYuDO6WhCktRepfvwM3MIGGo5d03T6+eAS0+ybi9w0SRrlCRNAT9pK0mFMPAlqRAGviQVwsCXpEIY+JJUCANfkgph4EtSIQx8SSqEgS9JhTDwJakQBr4kFcLAl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXCwJekQhj4klQIA1+SCmHgS1IhDHxJKoSBL0mFMPAlqRBzq0yKiHXAZmAA2JKZ97eMrwAeBM4CfgR8PDOPRcR5wDbgHCCB9Zl5eArrlyRV1PYKPyKWAfcAlwMrgI0RcUHLtG3A7Zn5PqAPuLVxfiuwNTPPB/YDd01V4ZKkzlS5wl8N7M7M1wAi4lFgLfDlxvF7gTMz8yeN+Q8Bd0fEg8CVwF80nf8h8LkK/885AP39fZWaOJlzFp054bUzzUTeh5L7L7l3mD39l9w7TKz/pjVzTjZeJfCXAsNNx8PApW3GlwNnA29k5rGW81UsAVi06N0Vp5/ou5uvnvDamWZwcH7Ha0ruv+TeYfb0X3LvMLH+mywBXmg9WSXw+4GxpuM+YLTCeOt5WtadzjPAFdS/SByvuEaSSjeHetg/c7LBKoH/MvXwHbcYONgyvuQk468CCyNiTmYeb8xpXnc6bwE/rjhXkvQbJ1zZj6tyW+YuYFVE1CJiHrAG2Dk+mJkvAr+KiA80Tn0E2JGZR4G9wI2N8zcBOyZQvCRpCrQN/Mw8AGwC9gDPAkOZuS8itkfEysa09cC9EfELYD7w7cb526jf1fM89e8SNk91A5KkavrGxlq32SVJs5GftJWkQhj4klQIA1+SCmHgS1IhKj08Tb+t3cPkZruIOAt4GvhQZv5Pj8vpqoj4InBD4/CpzLyzl/V0U0R8mfpjVcaA72bmt3pcUtdFxN8AZ2fmzb2uZSK8wu9QxYfJzVoRcRn1D8W9r9e1dFtErAauBv6Q+u/9xRFxXW+r6o6I+CPgT4ALgZXAX0VE9Laq7oqIVcCGXtcxGQZ+5379MLnMPAKMP0yuFLcCn6D6p6Znk2Hgjsx8u/HBwp8D5/W4pq7IzB8Cf9x4NtY51HcHjvS2qu6JiPdQv9D7aq9rmQy3dDrX7mFys1pm3gJQ2MUdAJn5n+OvI+L3qW/tfODUK2aXzDwaEXcDnwH+ETjQ45K66TvUP4B6bq8LmQyv8DvX7mFymuUi4v3AvwGfzcz/7nU93ZSZXwRq1IPv1jbTZ4WIuAV4KTN/0OtaJsvA79ypHhanAjSeGfUD4POZ+XCv6+mWiDi/8S/bkZn/B/wT9f38EtwIXB0Rz1L/d0D+PCLu7XFNE+KWTud2AV+KiBr1Pcw1wMbelqRuiIhzgceBGzNzd6/r6bLfo/4PG11O/Tvca4G/621J3ZGZHxx/HRE3A1dl5qd6V9HEeYXfoVM9TK63ValLPgO8C/hWRDzb+O/jvS6qGzJzO/AU8B/AvwNPZ+Y/9LYqdcqHp0lSIbzCl6RCGPiSVAgDX5IKYeBLUiEMfEkqhIEvSYUw8CWpEAa+JBXi/wG7rcLRg6Z+3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "eps = 1e-1\n",
    "p = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
    "q = [0.2-eps, 0.2+eps, 0.2, 0.2, 0.2]\n",
    "\n",
    "\n",
    "bar_plot(p)\n",
    "plt.show()\n",
    "bar_plot(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.08300749985576905"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abs(cross_entropy(p, q) - entropy(p))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXiElEQVR4nO3df4xd5X3n8feMMVlcG4cOw2IbiITUfAgrsU4wSRt+7tqLWrIbgswPFSv8iAyyCJXYtkuTtdkmaZyghq0dUjuJ4qxMZCYoi0hIFIO6jq3EFSKULcaRST5Ko60Xm0kZjUPAjkKxZ/aPe6Z7uRl7zp1f19zn85IsnfM8zzn3+cJoPufHnHt6RkdHiYiIcvV2egIREdFZCYKIiMIlCCIiCpcgiIgoXIIgIqJwp3R6Am16G3AJMAgc6/BcIiLeKuYAi4C/A15v7XyrBcElwO5OTyIi4i3qcuBvWxvfakEwCPCLXxxhZOTkff6hr28+w8OHOz2Njii5dii7/pJrh5O7/t7eHs4447eg+h3a6q0WBMcARkZGT+ogAE76+c2kkmuHsusvuXZ4S9Q/7iX13CyOiChcgiAionAJgoiIwiUIIiIKV+tmsaSbgXXAXGCj7U0t/dcCnwR6gP8D3G77F5LOA7YBZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3Y1H868EXgA7b/LbAX+ETVvRnYbPsC4Fngvqr908Bu2+8CvgJ8flqqiYiIttW5NLQC2Gn7kO0jwKPA9U39c4GP2j5Yre8FzpM0F7iiGg+wFbihWv4AjTMCgK8Df1CNj4iIWVbn0tBi3vwQwiDw3rEV28PANwEknQZ8DPgCcCbwqu2jTdud07pP20clvQr0Ay/VmXRf3/w6w8b1z28c49S5cya9fV39/QtmdP+TqaNbaof2a5mt2iH/72fSyVw7nJz111EnCHqB5qckeoCR1kGSFtIIhOdtP1RdUmp9umJsu56W9nH3eTzDw4cn/eBGf/8C/tOfPD6pbU8m3/nv1zI09Fpb23RL7dB+/SXXDt1Tf8m1w+Tqh8aTxSc6gK5zaegAjS8rGnM2LUfukhbR+A6gvcDqqvllYKGksfha1LTdwWo/SDoFWAAM15hLRERMszpBsANYLqlf0jxgJfDkWGf1i/47wDds32N7FMD2GzTC4aZq6C3AE9Xy9mqdqn93NT4iImbZhJeGbB+UtBbYBZwKbLH9jKTtwH8DzgXeA5wiaewm8rO2VwN3AQ9JWgf8X+APq/77gK2S9gGvAKums6iIiKiv1nMEtgeAgZa2a6rFZznOmYXt/cBV47QfAj7YzkQjImJm5MniiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicLVeTCPpZmAdMBfYaHvTccZ9Ddhpe6uks4C/aepeCPTbni/pSuAx4MWq7znbt0+2iIiImLwJg0DSEmA9cDHwOvCUpF22X2gasxj4MrAc2Alg+2VgadXfC3wPWFttsgx4wPZnp6+UiIiYjDpnBCtoHOUfApD0KHA98KmmMauAx4Hh4+zjduBX1SsvAS4B/rWkPwT+Efio7RePs21ERMygOvcIFgODTeuDwDnNA2x/zvaW8TaWNIfGmcDHmppfAb5g+yJgO/BIO5OOiIjpU+eMoBcYbVrvAUba+IzfB35q+0djDbbXNC1/SdL9khba/mWdHfb1zW/j47tXf/+CTk+ho0quP7WXaybqrxMEB4DLm9bPBl5q4zM+RNMRf3W/4OPA/baPNY07WneHw8OHGRkZnXjgOLrph2ho6LW2xndT7dBe/SXXDt1Vf8m1Q/v1A/T29pzwALrOpaEdwHJJ/ZLmASuBJ9uYw+8Bu8dWbI8A11X7QdItwA9tH2ljnxERMU0mDALbB2lc498F7AEGbD8jabukZTU+43waZxXNbgXukbSPxo3k1e1NOyIipkut5wiqv/YZaGm7Zpxxt43TNm+ctn3A+2vPMiIiZkyeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtredJxxXwN22t5ard8K3A/8UzXku7bXSjoP2AacBRhYZfvwVAqJiIjJmfCMQNISYD1wGbAUuFPShS1jFkv6DnB9y+bLgD+2vbT6t7Zq3wxstn0B8Cxw3xTriIiISapzRrCCxlH+IQBJj9L4hf+ppjGrgMeB4ZZtLwF+R9J/BZ4H/gg4DFwBfKgasxX4PvBnkyshIiKmos49gsXAYNP6IHBO8wDbn7O9ZZxtB4G/AC4CXgT+GjgTeNX20ePtLyIiZk+dM4JeYLRpvQcYqbNz29eNLUv6S+BnwL0t+6Pu/sb09c1vZ3jX6u9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLE20kaSHwEdsbqqYe4CjwMrBQ0hzbx4BFdfbXbHj4MCMjrVlSTzf9EA0NvdbW+G6qHdqrv+TaobvqL7l2aL9+gN7enhMeQNe5NLQDWC6pX9I8YCXwZI3tDgP3SnpftX438E3bbwC7gZuq9luAJ2rsLyIiZsCEQWD7ILAW2AXsAQZsPyNpu6RlJ9juGHAj8EVJPwYupnFZCOAuGn999AKNs411UysjIiImq9ZzBLYHgIGWtmvGGXdby/pu4D3jjNsPXNXGPCMiYobkyeKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMLVekOZpJtpvE5yLrDR9qbjjPsasNP21mr9UmADcCowTONl9vslXQk8BrxYbfqc7dunUkhEREzOhEEgaQmwnsY7h18HnpK0y/YLTWMWA18GlgM7mzZ/GPig7b2SPgI8CFwLLAMesP3ZaaskIiImpc6loRU0jvIP2T4CPApc3zJmFfA48I2xBklvA9bZ3ls17QXOq5YvAa6WtFfStyWdO5UiIiJi8uoEwWJgsGl9EDineYDtz9ne0tL2uu1tAJJ6gU8A36q6XwG+YPsiYDvwyKRmHxERU1bnHkEvMNq03gOM1P0ASacCD1Wf9RkA22vG+m1/SdL9khba/mWdffb1za/78V2tv39Bp6fQUSXXn9rLNRP11wmCA8DlTetnAy/V2bmk+cC3adwovtb2G9XZwceB+20faxp+tN6UYXj4MCMjoxMPHEc3/RANDb3W1vhuqh3aq7/k2qG76i+5dmi/foDe3p4THkDXuTS0A1guqV/SPGAl8GTNz98G/ANwk+3XAWyPANdV+0HSLcAPq/sPERExyyYMAtsHgbXALmAPMGD7GUnbJS073naS3k3jL4QuBf5e0h5J26vuW4F7JO0DbgdWT7GOiIiYpFrPEdgeAAZa2q4ZZ9xtTcvP0bifMN7+9gHvb2eiERExM/JkcURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4RIEERGFSxBERBQuQRARUbgEQURE4Wq9oUzSzcA6YC6w0fam44z7GrDT9tZq/Twa7y0+CzCwyvZhSW8HHgbOB4aAG23/fIq1RETEJEx4RiBpCbAeuAxYCtwp6cKWMYslfQe4vmXzzcBm2xcAzwL3Ve2fBnbbfhfwFeDzU6oiIiImrc6loRU0jvIP2T4CPMpv/sJfBTwOfGOsQdJc4IpqPMBW4IZq+QM0zggAvg78QTU+IiJmWZ0gWAwMNq0PAuc0D7D9OdtbWrY7E3jV9tFxtvuXfVb9rwL97U09IiKmQ517BL3AaNN6DzAyie1o2q6npb3uPgHo65tfd2hX6+9f0OkpdFTJ9af2cs1E/XWC4ABwedP62cBLNbZ7GVgoaY7tY8Cipu0OVvs5IOkUYAEwXHfSw8OHGRlpzZh6uumHaGjotbbGd1Pt0F79JdcO3VV/ybVD+/UD9Pb2nPAAus6loR3Ackn9kuYBK4EnJ9rI9hvAbuCmqukW4IlqeXu1TtW/uxofERGzbMIgsH0QWAvsAvYAA7afkbRd0rIJNr+Lxl8ZvUDjrGJd1X4f8LuS9lVjPjrZAiIiYmpqPUdgewAYaGm7Zpxxt7Ws7weuGmfcIeCDbcwzIiJmSJ4sjogoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXK03lEm6mcZrJucCG21vaulfCmwBTgd+AKwBfhv4m6ZhC4F+2/MlXQk8BrxY9T1n+/apFBIREZMzYRBIWgKsBy4GXgeekrTL9gtNw7YBq20/LemrwB22vwgsrfbRC3yPxruPAZYBD9j+7PSVEhERk1Hn0tAKYKftQ7aPAI8C1491SnoHcJrtp6umrcANLfu4HfhV9e5jgEuAqyXtlfRtSedOpYiIiJi8OkGwGBhsWh8EzqnbL2kOjTOBjzWNeQX4gu2LgO3AI+1NOyIipkudewS9wGjTeg8w0kb/7wM/tf2jsQbba5qWvyTpfkkLbf+yzqT7+ubXGdb1+vsXdHoKHVVy/am9XDNRf50gOABc3rR+NvBSS/+iE/R/iKYj/up+wceB+20faxp3tOacGR4+zMjI6MQDx9FNP0RDQ6+1Nb6baof26i+5duiu+kuuHdqvH6C3t+eEB9B1Lg3tAJZL6pc0D1gJPDnWaXs/8GtJl1ZNHwaeaNr+94DdTeNHgOuq/SDpFuCH1f2HiIiYZRMGge2DNK7x7wL2AAO2n5G0XdKyatgqYIOknwDzgQebdnE+jbOGZrcC90jaR+NG8uqplREREZNV6zmC6q99Blrarmlafh5473G2nTdO2z7g/W3NNCIiZkSeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJuhlYB8wFNtre1NK/FNgCnA78AFhj+6ikW4H7gX+qhn7X9lpJ5wHbgLMAA6tsH56OgiIioj0TnhFIWgKsBy4DlgJ3SrqwZdg24G7b7wR6gDuq9mXAH9teWv1bW7VvBjbbvgB4Frhv6qVERMRk1Lk0tALYafuQ7SPAo8D1Y52S3gGcZvvpqmkrcEO1fAlwq6QfSdom6QxJc4Erqv20jo+IiFlWJwgWA4NN64PAOTX7B4G/AC4CXgT+GjgTeNX20ePsLyIiZlGdewS9wGjTeg8wUqff9nVjjZL+EvgZcG/LeFr2N6G+vvntDO9a/f0LOj2Fjiq5/tRerpmov04QHAAub1o/G3ippX9Ra7+khcBHbG+o2nuAo8DLwEJJc2wfq7Zt3t+EhocPMzLSmiX1dNMP0dDQa22N76baob36S64duqv+kmuH9usH6O3tOeEBdJ1LQzuA5ZL6Jc0DVgJPjnXa3g/8WtKlVdOHgSeAw8C9kt5Xtd8NfNP2G8Bu4Kaq/ZZqfEREdMCEQWD7ILAW2AXsAQZsPyNpu6Rl1bBVwAZJPwHmAw9WR/s3Al+U9GPgYhqXhQDuovHXRy/QONtYN51FRUREfbWeI7A9AAy0tF3TtPw88N5xttsNvGec9v3AVW3ONSIiZkCeLI6IKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFyCICKicAmCiIjCJQgiIgqXIIiIKFytN5RJupnG6yTnAhttb2rpXwpsAU4HfgCssX20eo/xBuBUYJjGy+z3S7oSeAx4sdrFc7Zvn46CIiKiPROeEUhaAqwHLgOW0njX8IUtw7YBd9t+J9AD3FG1Pwystr20Wn6wal8GPGB7afUvIRAR0SF1Lg2tAHbaPmT7CPAocP1Yp6R3AKfZfrpq2grcIOltwDrbe6v2vcB51fIlwNWS9kr6tqRzp6GWiIiYhDqXhhYDg03rg7z5RfXj9Z9j+3UaZwpI6gU+AXyrGvMK8A3bj0laAzwCXFp30n198+sO7Wr9/Qs6PYWOKrn+1F6umai/ThD0AqNN6z3ASN1+SacCD1Wf9RkA22vG+m1/SdL9khba/mWdSQ8PH2ZkZHTigePoph+ioaHX2hrfTbVDe/WXXDt0V/0l1w7t1w/Q29tzwgPoOpeGDgCLmtbPBl6q0y9pPvAkjRC41vYbknolrZU0p+VzjtaYS0RETLM6QbADWC6pX9I8YCWNX+4A2N4P/Lr6CyGADwNPVMvbgH8AbqouFWF7BLiu2g+SbgF+WN1/iIiIWTZhENg+CKwFdgF7gAHbz0jaLmlZNWwVsEHST4D5wIOS3g1cS+Pa/99L2iNpezX+VuAeSfuA24HV01pVRETUVus5AtsDwEBL2zVNy8/z5hvIAM/RuF8w3v72Ae9va6YRETEj8mRxREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThEgQREYVLEEREFC5BEBFRuARBREThar2hTNLNwDpgLrDR9qaW/qXAFuB04AfAGttHJZ1H473FZwEGVtk+LOntwMPA+cAQcKPtn09TTRER0YYJzwgkLQHWA5cBS4E7JV3YMmwbcLftd9J4PeUdVftmYLPtC4Bngfuq9k8Du22/C/gK8PmpFhIREZNT54xgBbDT9iEASY8C1wOfqtbfAZxm++lq/Fbgk5K2AFcAH2pq/z7wZ8AHqj6ArwObJM21/cYEc5kD0Ns77quQazvrjNOmtP3JYjL/Hbqldmi//pJrh+6pv+TaYXL1N20zZ7z+OkGwGBhsWh/kzS+qH6//HOBM4FXbR1va37RNdQnpVaAfeGmCuSwCOOOM36ox7eP76rqrp7T9yaKvb37b23RL7dB+/SXXDt1Tf8m1w+Tqb7II+FlrY50g6AVGm9Z7gJEa/a3tNG3XGmmt+zyevwMupxEix2qMj4iIxpnAIhq/Q39DnSA4QOOX75izefOR+4HqA1r7XwYWSppj+1g1Zmy7g9W4A5JOARYAwzXm8jrwtzXGRUTEm/3GmcCYOn8+ugNYLqlf0jxgJfDkWKft/cCvJV1aNX0YeKK63r8buKlqvwV4olreXq1T9e+ucX8gIiJmwIRBYPsgsBbYBewBBmw/I2m7pGXVsFXABkk/AeYDD1btd9H4K6MXaJxVrKva7wN+V9K+asxHp6ugiIhoT8/oaOtl/IiIKEmeLI6IKFyCICKicAmCiIjCJQgiIgpX60vnop6Jvpyv20k6HXgK+I+2/7HD05lVkv4cuLFa/a7tezs5n9km6VM0vnpmFPiq7b/q8JRmnaQHgDNt39bpubQrZwTTpOaX83UtSe+j8bDfOzs9l9kmaQVwNfBuGv/vL5Z0XWdnNXskXQn8e+AiYBnwR5LU2VnNLknLgVs7PY/JShBMn3/5cj7bR4CxL+crxR00ngeZ6PuiutEg8Ce2/7l6MPLHwHkdntOssf194N9V3yt2Fo0rDUc6O6vZI+m3aRwEfqbTc5msXBqaPhN9OV9Xs70aoLADQQBs7xtblvQ7NC4RXXr8LbqP7TckfRL4U+B/0vgamVJ8mcZDt+d2eiKTlTOC6TPRl/NFl5P0b4D/BfwX2z/t9Hxmm+0/p/Etwufy/99J0tUkrQZetP29Ts9lKhIE0+d4X74XBai+a+t7wMdsP9Tp+cwmSRdUbynE9q+Ax2jcLyjBTcDVkvbQeEfLByVt6PCc2pZLQ9NnB/AJSf00ro+uBO7s7JRiNkg6F/gWcJPtnZ2eTwecT+NlVJfROCu+FvgfnZ3S7LD9H8aWJd0GXGX7P3duRpOTM4Jpcrwv5+vsrGKW/Cnwr4C/krSn+rem05OaLba3A98FngP+N/CU7Uc6O6toR750LiKicDkjiIgoXIIgIqJwCYKIiMIlCCIiCpcgiIgoXIIgIqJwCYKIiMIlCCIiCvf/AD2wLMTkQwMyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAOpUlEQVR4nO3df2xd91mA8cdO0qZrXNR5t7TpWioEvKhDEFhWhJaKH80m0VGqKR1DmRBl6sykAhLqBoMWmkx0f7B1q2BVheimDg0LRsdahWUTZAnQqkxpYVklBq8GWtGSWChyKy0J/ZHW5g/b6OIm9rnOvfe4730+UiWfe47t93uTPjk5vvdkbH5+HklSHeNtDyBJ6i/DLknFGHZJKsawS1Ixhl2SitnY8ve/EHgLMAO80vIskvRasQG4AngSeHH5zrbD/hbgsZZnkKTXquuBx5c/2HbYZwCee+40c3Pr9/X0k5NbmJ091fYYrRjltcNor9+1r9+1j4+PcemlF8NiQ5drO+yvAMzNza/rsAPrfr5BGuW1w2iv37Wve2e9hO0PTyWpGMMuScUYdkkqxrBLUjGNfngaEbuBu4BNwH2Zef+y/XcD7wWeW3zoT5cfI0kajlXDHhFXAvcAb2bhhfBPRMShzPxG12HbgV/MzH8azJiSpKaaXIrZCRzMzGcz8zTwMHDLsmO2A78bEU9HxCcjYnO/B5UkNdPkUsxW/v+L4GeA65Y2ImIL8DXgg8B/AA8Bvwfc2XSIycktTQ9tTacz0fYIrRnltcNor389rv2lM69wwaYNA/8+w1j7oNbSJOzjQPcr9ceAuaWNzDwF3Li0HRH3Ap+mh7DPzp5a128G6HQmOHHiZNtjtGKU1w6jvf71uvZOZ4Kb7ni07TH6Yt+9N6/pOR4fH1vxhLjJpZijLNxsZsnlwPGljYi4OiLe27V/DDjT45ySpD5pcsZ+ANgTER3gNLALmOra/zzwhxFxCHgGuB34Qp/nlCQ1tOoZe2YeY+GyyiHgCDCdmYcjYn9EbM/ME8CvAvuAZOGM/d4BzixJWkGj17Fn5jQwveyxG7s+/jzw+f6OJklaC995KknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkoox7JJUjGGXpGIMuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpmEZhj4jdEfGNiPhmRNy+wnHviIhv9W88SVKvVg17RFwJ3APsALYBUxFx7VmO+27gY8BYv4eUJDXX5Ix9J3AwM5/NzNPAw8AtZznuQWBvP4eTJPWuSdi3AjNd2zPAG7sPiIjfAP4F+Gr/RpMkrcXGBseMA/Nd22PA3NJGRPwQsAu4gWXBb2pycstaPm2oOp2JtkdozSivHUZ7/aO89mEZxHPcJOxHgeu7ti8Hjndtvwu4AngKuADYGhGPZWb356xodvYUc3Pzqx/Ykk5nghMnTrY9RitGee0w2utfr2uv9ofNWp7j8fGxFU+Im4T9ALAnIjrAaRbOzqeWdmbm3cDdABFxDfD3vURdktRfq15jz8xjwJ3AIeAIMJ2ZhyNif0RsH/SAkqTeNDljJzOngellj914luOeAa7px2CSpLXxnaeSVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkoox7JJUjGGXpGIMuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkorZ2OSgiNgN3AVsAu7LzPuX7X8nsBfYADwJTGXmS32eVZLUwKpn7BFxJXAPsAPYBkxFxLVd+y8GPgm8LTPfBGwGbh3ItJKkVTW5FLMTOJiZz2bmaeBh4JalnYuPXZOZ/x0RrwMuA54byLSSpFU1CftWYKZrewZ4Y/cBmXkmIn4W+DbwBuBv+zahJKknTa6xjwPzXdtjwNzygzLzS8BkRHwEeADY3XSIycktTQ9tTacz0fYIrRnltcP6XP9LZ17hgk0bBv59Br32Ya1jPRvEc9wk7EeB67u2LweOL21ExOuB7Zm5dJb+58Bf9jLE7Owp5ubmVz+wJZ3OBCdOnGx7jFaM8tph/a6/05ngpjsebXuM87bv3pt7fn7X4x+052Mtv7/Gx8dWPCFucinmAHBDRHQWr6HvAr7ctX8M+GxEXL24/S7g8Z4nlST1xaphz8xjwJ3AIeAIMJ2ZhyNif0Rsz8xZYAr4m4j4OhDAbw9yaEnSuTV6HXtmTgPTyx67sevjR4BH+juaJGktfOepJBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkoox7JJUjGGXpGIMuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFbOxyUERsRu4C9gE3JeZ9y/bfzOwFxgDvgX8SmY+1+dZJUkNrHrGHhFXAvcAO4BtwFREXNu1/xLgAeAdmfkjwNPAnoFMK0laVZNLMTuBg5n5bGaeBh4Gbunavwm4PTOPLW4/DVzd3zElSU01uRSzFZjp2p4BrlvayMxZ4AsAEXER8CHgj3sZYnJySy+Ht6LTmWh7hNaM8trB9Q/aqD+/g1h/k7CPA/Nd22PA3PKDIuK7WAj81zPzM70MMTt7irm5+dUPbEmnM8GJEyfbHqMVo7x2WL/rrxTDXp/fSmuH3tcPMD4+tuIJcZNLMUeBK7q2LweOdx8QEVcAj7FwGea2nqeUJPVNkzP2A8CeiOgAp4FdwNTSzojYAOwDPpeZfzCQKSVJja0a9sw8FhF3AoeAC4AHM/NwROwHfh+4CvgxYGNELP1Q9anM9MxdklrQ6HXsmTkNTC977MbFD5/CNzpJ0rphkCWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkoox7JJUjGGXpGIMuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFbOxyUERsRu4C9gE3JeZ95/juD8DDmbmQ32bUJLUk1XP2CPiSuAeYAewDZiKiGuXHbM1IvYBtwxkSklSY00uxexk4Sz82cw8DTzMqwP+HuBR4HN9nk+S1KMml2K2AjNd2zPAdd0HZOZHASJiR/9GkyStRZOwjwPzXdtjwFw/h5ic3NLPLzcQnc5E2yO0Zj2u/aUzr3DBpg1D+V6DXv8w17IercffX8M0iPU3CftR4Pqu7cuB4/0cYnb2FHNz86sf2JJOZ4ITJ062PUYr1uvaO50Jbrrj0bbH6It9997c83NcKYajvHboff0A4+NjK54QNwn7AWBPRHSA08AuYKrnSSRJQ7HqD08z8xhwJ3AIOAJMZ+bhiNgfEdsHPaAkqTeNXseemdPA9LLHbjzLcbf2Zyw1MXHJRWy+sNEv4XkZxl99X3jxZU5+5/mBfx9pFAy+ChqYzRduLHWdef1dyZdem7ylgCQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYgy7JBVj2CWpGMMuScUYdkkqxrBLUjGGXZKKMeySVIxhl6RiDLskFWPYJakYwy5JxRh2SSrGsEtSMYZdkoox7JJUjGGXpGIMuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklTMxrYHOB8Tl1zE5guHs4ROZ2KgX/+FF1/m5HeeH+j3kDQaGlUxInYDdwGbgPsy8/5l+7cBDwKXAP8IvD8zX+7zrK+y+cKN3HTHo4P+NkOx796bOdn2EJJKWPVSTERcCdwD7AC2AVMRce2ywz4L/Fpm/gAwBryv34NKkpppcsa+EziYmc8CRMTDwC3Ahxe3vwe4KDO/unj8Q8Be4IEGX3sDwPj4WG9Td7ns0ovW/LnrzVqeh1Fe/yivHeqsf5TXDmtbf9fnbDjb/rH5+fkVv0BE/A5wcWbetbh9G3BdZk4tbv8E8NHM3LG4/X3A/sWz99XsAB5rcJwk6dWuBx5f/mCTM/ZxoLv+Y8BcD/tX8uTiYDPAKw0/R5JG3QbgChYa+ipNwn6UhfguuRw4vmz/FSvsX8mLnOVPG0nSqv7zXDuavI79AHBDRHQi4nXALuDLSzsz87+AFyLirYsP/RLwpfMYVpJ0HlYNe2YeA+4EDgFHgOnMPBwR+yNi++Jh7wE+ERH/DmwB/mhQA0uSVrbqD08lSa8t3lJAkoox7JJUjGGXpGIMuyQV85q+u+OgrXbzs+oi4hLgCeDnMvOZlscZqoi4G/iFxc0vZuZvtTnPMEXEh1m4bcg88KnM/HjLIw1dRHwMeENm3tr2LGvhGfs5NLz5WVkR8eMsvHmsya0hSomIncDbgR9l4df+zRHxznanGo6I+EngZ4AfBrYDvx4R0e5UwxURNwC/3PYc58Own9v/3fwsM08DSzc/GxXvA26n+buIK5kB7sjMlzLzDPBvwNUtzzQUmfkPwE8v3nb7Mhb+Vn+63amGJyJez8IJ3UfanuV8eCnm3Lay8D/4khngupZmGbrMvA1gxE7WAMjMf136OCK+n4VLMm8992fUkplnImIv8AHgr4BjLY80TH/Cwhsyr2p7kPPhGfu5nc/NzVRARLwJ+Dvgg5n5zbbnGabMvBvosBC4kfj3FRbvXPvtzPxK27OcL8N+budzczO9xi3e++grwIcy8zNtzzMsEfGDi/8iGpn5P8Bfs3C9fRS8G3h7RBxh4d+b+PmI+ETLM62Jl2LO7QCwJyI6LFxj3AVMtTuShiEirgIeAd6dmQfbnmfIvhfYGxE7WPgb683Ap9sdaTgy821LH0fErcBPZeZvtjfR2nnGfg7nuvlZu1NpSD4AbAY+HhFHFv97f9tDDUNm7ge+CHwN+Gfgicz8i3anUq+8CZgkFeMZuyQVY9glqRjDLknFGHZJKsawS1Ixhl2SijHsklSMYZekYv4X3ZRcz3YqMCMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "eps = 1e-2\n",
    "p = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
    "q = [0.01, 0.09, 0.1, 0.3, 0.5]\n",
    "\n",
    "\n",
    "bar_plot(p)\n",
    "plt.show()\n",
    "bar_plot(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9134081185447793"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abs(cross_entropy(p, q) - entropy(p))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Intuition:</b> Cross entropy of \"similar\" distributions is smaller, and for \"dissimilar\" - larger\n",
    "<br /><br />\n",
    "It can be thought as a distance measure between distributions, it isn't a valid metric though as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# H(p, q) != H(q, p)\n",
    "cross_entropy(q, p) == cross_entropy(p, q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cross entropy has many applications in <b>Machine Learning</b>, for example, it is used to measure the similarity between predictions and ground-truth labels (loss function)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><big>Kullback-Leibler (KL) Divergence</big></b>\n",
    "<br /><br />\n",
    "Let $p(X)$ and $q(X)$ be two probability mass functions over a same set of events.\n",
    "<br /><br />\n",
    "The <b>Kullback-Leibler divergence</b> between distributions $p$ and $q$ is defined as following expectation:<br /><br />\n",
    "$KL(p, q)=\\mathbb{E}_p[log\\frac{p(X)}{q(X)}]=\\sum_{i=1}^{n}p(x_i)log\\frac{p(x_i)}{q(x_i)}$   ($x_i$ are possible values of $X$)\n",
    "<br /><br />\n",
    "\n",
    "Note that, the above formula is valid where $q(x)\\neq0$ or both $p(x)=q(x)=0$ Otherwise, $KL(p, q)=\\infty$<br /><br />\n",
    "\n",
    "Also, if $p=q$, $KL(p, q)=0$\n",
    "\n",
    "<br /><br />\n",
    "\n",
    "There is a relation between <b>Entropy</b>, <b>Cross-Entropy</b> and <b>KL divergence:</b><br /><br />\n",
    "    \n",
    "$H(p, q)=H(P)+KL(p, q)$   \n",
    "\n",
    "<br /><br /> \n",
    "\n",
    "Some final notes:\n",
    "<ul>\n",
    "    <li>Similarly to Cross-Entropy, KL divergence is used in Statistics and Machine Learning to measure similarity between probability distributions.</li>\n",
    "    <li>KL Divergence is used as a loss function in approximate variational inference - powerful unsupervised technique in ML to learn complex distributions</li>\n",
    "    \n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}