013 LSTM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `013` Long short-term memory (LSTM) networks\n",
    "\n",
    "Requirements: 012 Recurrent neural networks\n",
    "\n",
    "As we saw in the previous RNN notebook, the layers that update the hidden state in RNNs suffer from very long term dependencies that make the gradients vanish or explode:\n",
    "* If a cell has a weight of 0.5, and the sequence is 50 steps long, some output will be multiplied by $0.5^{50}$, which is almost zero (or exactly zero if the floating point precision is not enough).\n",
    "* If a cell has a weight of 2, and the sequence is 50 steps long, some output will be multiplied by $2^{50}$, which is roughly one quadrillion.\n",
    "\n",
    "Some previous attempts to solve this problem used activation functions that were able to keep the gradients in a reasonable range, like the `tanh` function, that cramps an input in $(-\\infty, \\infty)$ into $(-1, 1)$. However, since $tanh(-4) \\approx -0.999$, the gradients can still vanish, and long-term dependencies are still hard to learn.\n",
    "\n",
    "Which is why LSTM was proposed. LSTM is a type of RNN that maintains two cell states: a long term and a short term memory. The architecture is designed in a way that the gradients can flow through the network without vanishing or exploding, so that the network can know what to remember and what to forget. It is composed of three so-called gates: the input gate, the forget gate, and the output gate.\n",
    "\n",
    "To understand how all of this works, we first have to think of the semantics of the two used activation functions: sigmoid and tanh."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from json import loads\n",
    "from matplotlib import pyplot as plt\n",
    "from random import shuffle\n",
    "from string import ascii_letters, digits\n",
    "from time import time\n",
    "from unicodedata import category, normalize\n",
    "import torch\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9UAAAEpCAYAAACdhQFUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVU0lEQVR4nO3deXzTRf4/8FeSNknvg9435SqVo1BsLYp4VKsoHqsuCiuHCqtSv0DxqqtU1LUqiPhDVjw4dJUVdVnEBYuIVBeoIIVyg7QUCqV3adMzSZP5/VEaCD0gpc0naV/PxyO2n8lM8s5H0sk7M58ZmRBCgIiIiIiIiIgsJpc6ACIiIiIiIiJ7xaSaiIiIiIiIqJOYVBMRERERERF1EpNqIiIiIiIiok5iUk1ERERERETUSUyqiYiIiIiIiDqJSTURERERERFRJzGpJiIiIiIiIuokJtVEREREREREncSkmshGRUREYOrUqVKH0aFVq1ZBJpPh5MmTl61rD6+HiIioN2npx3fv3i11KER2jUk1kQQOHDiABx98EOHh4VCr1QgODsZtt92GJUuWSB0aERERSUAmk13RLTMzU+pQiegSDlIHQNTb7NixAzfffDPCwsIwffp0BAQE4PTp0/jtt9/w/vvv45lnngEAHDt2DHK5bX/v9eijj+Lhhx+GSqWSOhQiIiK79s9//tPs+PPPP8fmzZtblQ8ePNiaYRHRFWBSTWRlf//73+Hh4YHff/8dnp6eZveVlpaafreHRFWhUEChUEgdBhERkd37y1/+Ynb822+/YfPmza3Kicj22PYwGFEPlJeXh2uuuaZVQg0Afn5+pt/bugZ5//79GDt2LJycnBASEoI33ngDK1eubHVdc0REBO6++25kZmZi1KhRcHJywtChQ01TxtauXYuhQ4dCrVYjNjYWe/fubRXLzz//jDFjxsDFxQWenp649957ceTIEbM6bV1TLYTAG2+8gZCQEDg7O+Pmm2/GoUOHLD5PREREZG7lypW45ZZb4OfnB5VKhejoaHz44Yet6rV8Dti2bRvi4uKgVqsRGRmJzz//vM3H1Wq1SElJga+vL1xcXHD//fejrKysu18OUY/BpJrIysLDw5GdnY2DBw9a1K6wsNCUoKampmLOnDn48ssv8f7777dZPzc3FxMnTsT48eORnp6Oc+fOYfz48fjyyy8xZ84c/OUvf8H8+fORl5eHP//5zzAajaa2P/30E5KSklBaWopXX30VKSkp2LFjB66//vrLLko2b948vPLKKxg+fDgWLFiAyMhI3H777airq7Po9RIREZG5Dz/8EOHh4XjppZfw7rvvIjQ0FE8//TSWLl3aqm5ubi4efPBB3HbbbXj33Xfh5eWFqVOntvlF9zPPPIN9+/YhLS0NTz31FL7//nskJydb4yUR9QyCiKzqxx9/FAqFQigUCpGQkCCef/55sWnTJqHT6czqhYeHiylTppiOn3nmGSGTycTevXtNZRUVFcLb21sAEPn5+WZtAYgdO3aYyjZt2iQACCcnJ3Hq1ClT+UcffSQAiK1bt5rKYmJihJ+fn6ioqDCV7du3T8jlcjF58mRT2cqVK82eu7S0VCiVSnHXXXcJo9FoqvfSSy8JAGavh4iIiNo3c+ZMcelH9fr6+lb1kpKSRGRkpFlZy+eAX3/91VRWWloqVCqVmDt3rqmspR9PTEw067fnzJkjFAqFqKqq6qqXQ9SjcaSayMpuu+02ZGVl4Z577sG+ffvwzjvvICkpCcHBwVi/fn277TIyMpCQkICYmBhTmbe3NyZNmtRm/ejoaCQkJJiO4+PjAQC33HILwsLCWpWfOHECAFBUVIScnBxMnToV3t7epnrDhg3Dbbfdho0bN7Yb408//QSdTodnnnkGMpnMVD579ux22xAREdGVcXJyMv1eXV2N8vJyjB07FidOnEB1dbVZ3ejoaIwZM8Z07Ovri0GDBpn6+4vNmDHDrN8eM2YMDAYDTp061Q2vgqjnYVJNJIFrr70Wa9euxblz57Br1y6kpqaipqYGDz74IA4fPtxmm1OnTqF///6tytsqA2CWOAOAh4cHACA0NLTN8nPnzpmeBwAGDRrU6jEHDx6M8vLydqdyt7QdMGCAWbmvry+8vLzabENERERXZvv27UhMTDStd+Lr64uXXnoJAFol1Zd+DgAALy8vU3/fUd2WPrutukTUGpNqIgkplUpce+21ePPNN/Hhhx9Cr9fjm2++6ZLHbm9V7vbKhRBd8rxERETU9fLy8nDrrbeivLwcixYtwoYNG7B582bMmTMHAMzWRgEs6+/52YDo6nBLLSIbMWrUKADN06/bEh4ejtzc3FblbZVdjfDwcADN+2Rf6ujRo/Dx8YGLi0uHbY8fP47IyEhTeVlZGb/tJiIiugrff/89tFot1q9fbzayvHXrVgmjIiKAI9VEVrd169Y2v/ltuVa5rWnXAJCUlISsrCzk5OSYyiorK/Hll192aXyBgYGIiYnBZ599hqqqKlP5wYMH8eOPP2LcuHHttk1MTISjoyOWLFli9hoXL17cpTESERH1Ni2jyRf3r9XV1Vi5cqVUIRHReRypJrKyZ555BvX19bj//vsRFRUFnU6HHTt2YM2aNYiIiMC0adPabPf888/jiy++wG233YZnnnkGLi4u+PTTTxEWFobKykqzBUau1oIFC3DnnXciISEBjz/+OBoaGrBkyRJ4eHjg1Vdfbbedr68vnn32WaSnp+Puu+/GuHHjsHfvXvzwww/w8fHpsviIiIh6m9tvvx1KpRLjx4/HX//6V9TW1uKTTz6Bn59fu7PciMg6OFJNZGULFy7EzTffjI0bNyIlJQUpKSnYtWsXnn76aezcuROenp5ttgsNDcXWrVsxePBgvPnmm1i8eDGmTJmCxx57DACgVqu7LMbExERkZGSgT58+mDdvHhYuXIjrrrsO27dvR9++fTts+8Ybb2D+/PnYu3cvnnvuOeTl5eHHH39sd8o4ERERXd6gQYPw7bffQiaT4dlnn8WyZcswY8YMzJo1S+rQiHo9meAKBER2bfbs2fjoo49QW1vb7kIjRERERETUPThSTWRHGhoazI4rKirwz3/+EzfccAMTaiIiIiIiCfCaaiI7kpCQgJtuugmDBw9GSUkJli9fDo1Gg1deeUXq0IiIiIiIeiUm1UR2ZNy4cfj222/x8ccfQyaTYeTIkVi+fDluvPFGqUMjIiIiIuqVeE01ERERERERUSfxmmoiIiIiIiKiTmJSTURERERERNRJdnFNtdFoxNmzZ+Hm5gaZTCZ1OERE1MsJIVBTU4OgoCDI5fx+uiuwryciIltzpf29XSTVZ8+eRWhoqNRhEBERmTl9+jRCQkKkDqNHYF9PRES26nL9vV0k1W5ubgCaX4y7u7vE0RDZtwkTJmDNmjVSh0Fk1zQaDUJDQ039E1099vVEXYd9PVHXuNL+3i6S6pZpYO7u7uxoia6So6Mj30dEXYTTlLsO+3qirsO+nqhrXa6/54VgRERERERERJ3EpJqIiIiIiIiok5hUExEREREREXWSxUn1r7/+ivHjxyMoKAgymQzr1q27bJvMzEyMHDkSKpUK/fv3x6pVqzoRKhEREVkD+3oiIqIrZ3FSXVdXh+HDh2Pp0qVXVD8/Px933XUXbr75ZuTk5GD27Nl44oknsGnTJouDJSIiou7Hvp6IiOjKWbz695133ok777zziusvW7YMffv2xbvvvgsAGDx4MLZt24b33nsPSUlJlj49ERERdTP29URERFeu27fUysrKQmJiollZUlISZs+e3d1PTUREFxFCwCgAg1E030TzT6NRwCiaj8X5+43nfxcCMIrzx+cfo7kMEBAwGpt/CtH8HKZ2Fz1nczugpbSl7oU65nXNY26pe0mj9l5jO+0vdk2QO7xclB0/EFmEfT0R2RIhBPQGAb3BeP7W3N81GY3nfwoIIWAwXtTHmfq7C30ZLurfWvqyi/sqIS7qn0xPfpnYuvaldvxc1nwyGxPkqUakr6vVnq/bk+ri4mL4+/ublfn7+0Oj0aChoQFOTk6t2mi1Wmi1WtOxRqPp7jCJiKyqUW+AplGP2sYm1DQ2oVbbfKvXNaFOa0C9rgn1OgMa9UY06g1o0BmgbTJA22SErsl44afBCH2TETqDEU3nPzjoDc0fGkw/W5JoYy/uXS/y2WNxGDvQV+owehT29UTU1fQGI0o0jSiqbr5V1Gpxrk6Hc/V6VNbrUNvYhDpT32lAg94Ard6AxvP9I/VuT9zQFy/fHW215+v2pLoz0tPTMX/+fKnDICK6YkIIVNTpUKJpPH/TolSjRUWdFhV1uvMfBvSobtCjqkGHRr3tdfgyGSCXySA3/Wz+XSaTme6TyQAZWn6/cNz88+JjmekxW+5rOQYu1JGZnvyiOEzxyC69y+wxLtSXdXj/pVyUio4rkFWwryciADAaBf4orcHBQg2Ol9TgWEkNjpfU4mx1Q5eOtDoqZFDIZXCQyyGXAQr5+X5O3tyLtBwDgFzeTp92/j/t9VPt9T+X9lOt7r9Mv0WW83NXWfX5uj2pDggIQElJiVlZSUkJ3N3d2/zmGgBSU1ORkpJiOtZoNAgNDe3WOImILsdgFCg814C88lqcKKvDyfI6nD5XjzPnGnDmXL3FibJMBriqHOCmcoCb2hHOKgVcVQ5wVirgrHSAk1IBJ8fmm9pRDrWjAioHOVQOCigd5M03hRyODnI4KmRQKuRwUDT/7iCXw0Ehg6NcDoVCBoVMBofzP1uOWz5ANP+88OGAyFLs64nIEgUV9dh6rBRZeRXYmV+Bc/X6NuspFXIEeKgR4KGGr5sK3s5KeDk7wtNZCTe1A1xVDnBROcBFpYDa8cJNdXEfqZBDIWf/Rt2r25PqhIQEbNy40axs8+bNSEhIaLeNSqWCSmXdbxeIiC7WqDfg0NlqHDqrweGzGhwp0uBocQ20HUwpk8mAPi4q+Lur4O+uhp+bCj6uKni7KNHHVQlvFyU8nZTwdHaEh7MjXJUOkLOjpx6AfT0RXU5Nox4bDxTh39mF2HWy0uw+Z6UCw0I8MMjfDQMD3DDQ3w0RfVzQx0XJfpLsgsVJdW1tLXJzc03H+fn5yMnJgbe3N8LCwpCamorCwkJ8/vnnAIAnn3wSH3zwAZ5//nk89thj+Pnnn/H1119jw4YNXfcqiIiuUnWDHll5Fdh9shLZBedwsLAaekPreWdKhRwRPs7o5+uKvj4uCO/jjBAvZ4R4OSHQwwlKB4t3KiSyOezriairlNVo8f+2HMc32adNM7pkMiC+rzdu6O+DhH59MCzEE44K9p9kvyxOqnfv3o2bb77ZdNwydWvKlClYtWoVioqKUFBQYLq/b9++2LBhA+bMmYP3338fISEh+PTTT7nFBhFJSgiBfWeqkXmsFP87Xo6c01WtFvLycVViWIgnogPdER3kjsGB7gjzduY0Murx2NcT0dVq0Bnw6f9OYNkveajTGQAA/Xxd8EBsCO4fEYxAj7YvDSGyRzIhbH+xdY1GAw8PD1RXV8Pd3V3qcIjs2j333IP169dLHYYkhBDYe7oKG/cX4YeDxSisajC7v5+vCxL69UFsuBdiw7wR6u3E64ypTeyXuh7PKVHXkbqv//FQMV757iBKNM0r/A8L8cCLd0QhoV8f9qtkV660b7LJ1b+JiLpSWY0W32Sfxle7TqOgst5U7qxUYOxAX4wd6IsbBvggxMtZwiiJiIjsm9Eo8N5Pf2DJz82Xj4R4OeH5O6Jw99BAXhtNPRqTaiLqsXblV2LVjnz8eKgETeendrsoFbh1sD/GDQ3ETYN8oXbk9kpERERXq7pBjzlrcvDz0VIAwOM39MXzdwyCyoH9LPV8TKqJqEcRQuCXP8qwdGsufj95zlQ+IswTE+PCcPewIDhxn2IiIqIuc7K8Do+t+h0nyuugcpDjrQeG4v4RIVKHRWQ1TKqJqMfIPFaKd3/8AwcKqwE0r9T94KgQPHpdOAYH8hpNIiKirlZeq8WjK3bidGUDgjzU+HjyKAwJ9pA6LCKrYlJNRHYvr6wWb/z3MLYeKwMAODkqMCk+DNNvjIS/u1ri6IiIiHqmRr0B0z/fjdOVDQjzdsa3TyXAz439LvU+TKqJyG7VNOrx/k/HsWrHSTQZBRzkMkwdHYGnb+4Pbxel1OERERH1WEajwJw1OdhbUAUPJ0esnHYtE2rqtZhUE5FdysqrwLPf7DNti3VrlB/+dtdgRPq6ShwZERFRz/d2xlH8cLAYSoUcHz8ai37sf6kXY1JNRHalUW/Agk3HsHxbPgAg1NsJr987BDcN8pM4MiIiot7hu5xCfPTrCQDAOw8OQ3xkH4kjIpIWk2oisht/lNRg5pd7cLy0FgDwSFwo/nZXNFxV/FNGRERkDdX1erz+38MAgGdu6Y/7RgRLHBGR9PhJlIjswubDJZj91V7U6QzwcVXhnQeH4pYof6nDIiIi6lUW/ngM5bU69PN1wTO3DJA6HCKbwKSaiGyaEAL/yMzDwh+PQQjgukhvLJ04En1cVVKHRkRE1KvsP1OFL3aeAgC8ft8QKB3kEkdEZBuYVBORzWrUG/Dct/vx/b6zAIBHrwvHvPHRcFSwEyciIrImg1Hg5XUHIQRwX0wQRvfzkTokIpvBpJqIbFK9rgkzPs/GttxyOMhleO3eIZgYHyZ1WERERL3S6l0F2H+mGm4qB7x012CpwyGyKUyqicjm1DTq8fiq3dh1shLOSgU+nTKK34gTERFJpLxWiwUZRwEAzyYN4n7URJdgUk1ENqW6Xo/JK3dh3+kquKkcsOqxOMSGe0kdFhERUa+1cns+NI1NuCbIHX+5LlzqcIhsDpNqIrIZ1Q16TPz0Nxw6q4GnsyP++Vg8hoZ4SB0WERFRr9WgM+DLnQUAmrfQUshlEkdEZHuYVBORTdA2GTDj8904dFYDH1clvngiHlEB7lKHRURE1Kut3XsGVfV6hHo74bboAKnDIbJJXEKXiCRnNAo8+81+7MyvhKvKAZ89FseEmoiISGJGo8CKbfkAgKmj+3KUmqgdTKqJSHJvZxzF9/vOwkEuw4d/GYlrgjjlm4iISGq/HC9DXlkdXFUO+POoEKnDIbJZTKqJSFKrtufjo19PAADeeXAYxgzwlTgiIiIiAmAapZ5wbSjc1I4SR0Nku5hUE5FkduSW47X/HgYAPJc0CH8ayW/BiYiIbMEfJTX43/FyyGXA1NERUodDZNOYVBORJEo0jfi/r/bCKIAHRobg6Zv6SR0SERERndcySn17dABCvZ0ljobItjGpJiKrazIY8cy/9qK8VoeoADe8cd8QyGRc/ISIiMgWVNbpsHZvIQDg8TF9JY6GyPYxqSYiq3t38x/YdX6l739MGgknpULqkIiIiOi8jQeKoGsyIjrQHaPCvaQOh8jmMakmIqvacqQEH2bmAQDefmAYIn1dJY6IiIiILvbDwSIAwPjhQZxJRnQFmFQTkdWU1Wgx95t9AJoXPblrWKDEEREREdHFKmq1yMqrAADcNZT9NNGVYFJNRFYhhMAr6w6iql6P6EB3pI6LkjokIiIiusSmQyUwCmBIsDvC+nCBMqIrwaSaiKxiw4EiZBwqhoNchoUPDYfKgddRExER2ZqWqd93DuEoNdGVYlJNRN2uolaLed8dAgA8fXN/RAe5SxwRERERXaqyTocdnPpNZDEm1UTU7eatP4TKuubts5Jv7i91OERERNSGzYeLYTAKRAe6I8LHRepwiOwGk2oi6lY/HCjChv1FUMhlWPDgcCgd+GeHiIjIFm04UAwAGDc0QOJIiOxLpz7dLl26FBEREVCr1YiPj8euXbs6rL948WIMGjQITk5OCA0NxZw5c9DY2NipgInIftRqm5C2vnna95NjIzE0xEPiiIiIiKgtVfU67MgtBwCM49RvIotYnFSvWbMGKSkpSEtLw549ezB8+HAkJSWhtLS0zfqrV6/Giy++iLS0NBw5cgTLly/HmjVr8NJLL1118ERk2z74ORelNVqE93HGM7cMkDocIiIiasePh0vQZBSICnBDpK+r1OEQ2RWLk+pFixZh+vTpmDZtGqKjo7Fs2TI4OztjxYoVbdbfsWMHrr/+ekycOBERERG4/fbb8cgjj1x2dJuI7NvJ8jqs2JYPAHj5rmioHbnaNxERka3aeKB51W+OUhNZzqKkWqfTITs7G4mJiRceQC5HYmIisrKy2mwzevRoZGdnm5LoEydOYOPGjRg3btxVhE1Etu6NDUegMxgxZoAPEgf7SR0OERERtUPTqMd2Tv0m6jQHSyqXl5fDYDDA39/frNzf3x9Hjx5ts83EiRNRXl6OG264AUIINDU14cknn+xw+rdWq4VWqzUdazQaS8IkIon9+kcZfjpSAge5DGnjoyGTyaQOiYiIiNqx60Ql9AaBiD7O6O/Hqd9Elur2ZXgzMzPx5ptv4h//+Af27NmDtWvXYsOGDXj99dfbbZOeng4PDw/TLTQ0tLvDJKIuojcY8dp/DwMAJidEoL+fm8QRERERUUe25zWPUo/u7yNxJET2yaKRah8fHygUCpSUlJiVl5SUICCg7aX3X3nlFTz66KN44oknAABDhw5FXV0dZsyYgb/97W+Qy1vn9ampqUhJSTEdazQaJtZEduKfWaeQW1qLPi5KzErk4mRERES2LiuvAgBwfT8m1USdYdFItVKpRGxsLLZs2WIqMxqN2LJlCxISEtpsU19f3ypxViiaFywSQrTZRqVSwd3d3exGRLavplGPJT8fBwDMvX0QPJwcJY6IiIiIOlJeq8XR4hoAwHWR3hJHQ2SfLBqpBoCUlBRMmTIFo0aNQlxcHBYvXoy6ujpMmzYNADB58mQEBwcjPT0dADB+/HgsWrQII0aMQHx8PHJzc/HKK69g/PjxpuSaiHqGFdtO4ly9HpE+LvjzqBCpwyEiIqLL+O1E8yh1VIAb+riqJI6GyD5ZnFRPmDABZWVlmDdvHoqLixETE4OMjAzT4mUFBQVmI9Mvv/wyZDIZXn75ZRQWFsLX1xfjx4/H3//+9657FUQkuap6HT793wkAwOzbBsJB0e1LNhAREdFV2p7bnFSP5tRvok6zOKkGgOTkZCQnJ7d5X2ZmpvkTODggLS0NaWlpnXkqIrITH/16AjXaJkQFuOFubsdBRERkF7LOL1J2ff8+EkdCZL84lEREV62sRotV208CAFJuGwi5nFtoEdm7pUuXIiIiAmq1GvHx8di1a1e7dVetWgWZTGZ2U6vVVoyWiDqjsKoBJyvqoZDLENeX11MTdRaTaiK6av/IzEWD3oDhIR64Ldr/8g2IyKatWbMGKSkpSEtLw549ezB8+HAkJSWhtLS03Tbu7u4oKioy3U6dOmXFiImoM3bkNo9SDw32gJuai4sSdRaTaiK6KmerGvDlbwUAmlf8lsk4Sk1k7xYtWoTp06dj2rRpiI6OxrJly+Ds7IwVK1a020YmkyEgIMB0a1lrhYhsl2krLU79JroqTKqJ6Kr8IzMXOoMRcX29MWYAFzkhsnc6nQ7Z2dlITEw0lcnlciQmJiIrK6vddrW1tQgPD0doaCjuvfdeHDp0qMPn0Wq10Gg0Zjcish4hBHbkcZEyoq7ApJqIOq2sRouvd58BAMxOHMBRaqIeoLy8HAaDodVIs7+/P4qLi9tsM2jQIKxYsQLfffcdvvjiCxiNRowePRpnzpxp93nS09Ph4eFhuoWGhnbp6yCijuWX16FY0wilQo7YcC+pwyGya0yqiajTVm7Ph67JiJhQTyREcuoYUW+VkJCAyZMnIyYmBmPHjsXatWvh6+uLjz76qN02qampqK6uNt1Onz5txYiJaPv5UeqR4Z5QOyokjobIvnVqSy0ioppGPf75W/NCRE/d1I+j1EQ9hI+PDxQKBUpKSszKS0pKEBAQcEWP4ejoiBEjRiA3N7fdOiqVCiqV6qpiJaLOM22lxanfRFeNI9VE1CmrdxagprEJ/XxdcNtgLkhE1FMolUrExsZiy5YtpjKj0YgtW7YgISHhih7DYDDgwIEDCAzknvVEtkgIYVqkbDQXKSO6ahypJiKLNeoN+HRbPgDgr2P7cV9qoh4mJSUFU6ZMwahRoxAXF4fFixejrq4O06ZNAwBMnjwZwcHBSE9PBwC89tpruO6669C/f39UVVVhwYIFOHXqFJ544gkpXwYRtSO/vA7n6vVQOsgxNNhT6nCI7B6TaiKy2H/2FqKsRosAdzXuiwmWOhwi6mITJkxAWVkZ5s2bh+LiYsTExCAjI8O0eFlBQQHk8guT3c6dO4fp06ejuLgYXl5eiI2NxY4dOxAdHS3VSyCiDuScrgIADAlyh9KBE1eJrhaTaiKyiMEo8NEveQCAJ8b0ZWdM1EMlJycjOTm5zfsyMzPNjt977z289957VoiKiLpCS1IdE8pVv4m6Aj8NE5FFNh0qxsmKeng4OeKRuDCpwyEiIiILmZLqME9J4yDqKZhUE5FFVpy/lnpyQjhcVJzsQkREZE8a9QYcKdIAAEaEekobDFEPwaSaiK7YwcJq7D51Dg5yGR69LlzqcIiIiMhCh85qoDcI9HFRIsTLSepwiHoEJtVEdMVW7TgJABg3NBB+7mppgyEiIiKLXbie2hMyGXfvIOoKTKqJ6IqU12qxPucsAGDq9RHSBkNERESdcnFSTURdg0k1EV2Rr3YVQGcwYniIB6/BIiIislM5p88B4CJlRF2JSTURXZbeYMQ/fzsFoHmUmtPFiIiI7E9FrRanKxsAAMNCPKUNhqgHYVJNRJeVcbAYJRotfFxVGDc0UOpwiIiIqBNapn7393OFh5OjtMEQ9SBMqonosj47v0DZpPgwqBwU0gZDREREncLrqYm6B5NqIupQyzZajgoZJsWHSR0OERERdRKTaqLuwaSaiDr0xflrqe8cwm20iIiI7JXRKJhUE3UTJtVE1K6aRj3W72veRouj1ERERPbrRHkdahqboHaUIyrATepwiHoUJtVE1K7vcs6iXmdAP18XxPX1ljocIiIi6qSWUeqhwR5wUDAFIOpKfEcRUZuEEFi9swAA8EhcGLfRIiIismOm/ak59ZuoyzGpJqI27T9TjcNFGigd5HgwNkTqcIiIiOgqXLie2kvaQIh6ICbVRNSmllHqu4YGwtNZKXE0RERE1Fm6JiOOFdcAAIaFeEgcDVHPw6SaiFq5eIGyR+K4QBkREZE9O15aA71BwF3tgBAvJ6nDIepxmFQTUSvrcs6iQW9Afz9XXBvBaWJERET27NBZDQAgOsida6QQdQMm1URkhguUERER9SyHzyfV1wRx6jdRd2BSTURm9p2pxpHzC5Q9MDJY6nCIiIjoKh06Ww0AuCbIXeJIiHqmTiXVS5cuRUREBNRqNeLj47Fr164O61dVVWHmzJkIDAyESqXCwIEDsXHjxk4FTETd6+vdpwEAdw4J4AJlREREds5oFBypJupmDpY2WLNmDVJSUrBs2TLEx8dj8eLFSEpKwrFjx+Dn59eqvk6nw2233QY/Pz98++23CA4OxqlTp+Dp6dkV8RNRF2rQGfB9TvMCZRNGhUocDREREV2tU5X1qNMZoHKQo5+vi9ThEPVIFifVixYtwvTp0zFt2jQAwLJly7BhwwasWLECL774Yqv6K1asQGVlJXbs2AFHR0cAQERExNVFTUTdYtOhYtRomxDi5YTrIvtIHQ4RERFdpZap31EBbnBQ8MpPou5g0TtLp9MhOzsbiYmJFx5ALkdiYiKysrLabLN+/XokJCRg5syZ8Pf3x5AhQ/Dmm2/CYDBcXeRE1OW+yW6e+v1gbAjkci5QRkREZO8urPzNqd9E3cWikery8nIYDAb4+/ublfv7++Po0aNttjlx4gR+/vlnTJo0CRs3bkRubi6efvpp6PV6pKWltdlGq9VCq9WajjUajSVhElEnnK6sx/bcCshkzUk1ERER2b9DpuupuUgZUXfp9jkgRqMRfn5++PjjjxEbG4sJEybgb3/7G5YtW9Zum/T0dHh4eJhuoaG8tpOou32bfQYAMLpfH4R4OUscDREREV0tIQQOc+Vvom5nUVLt4+MDhUKBkpISs/KSkhIEBAS02SYwMBADBw6EQqEwlQ0ePBjFxcXQ6XRttklNTUV1dbXpdvr0aUvCJCILGY3ClFT/mQuUERER9QhlNVqU1+oglwFRAUyqibqLRUm1UqlEbGwstmzZYiozGo3YsmULEhIS2mxz/fXXIzc3F0aj0VT2xx9/IDAwEEpl29v1qFQquLu7m92IqPtknahAYVUD3NQOSLqm7S/IiIiIyL60TP3u5+sKJ6XiMrWJqLMsnv6dkpKCTz75BJ999hmOHDmCp556CnV1dabVwCdPnozU1FRT/aeeegqVlZWYNWsW/vjjD2zYsAFvvvkmZs6c2XWvgoiuSsve1PcMD4LakZ0uERFRT9Cy8nc0p34TdSuLt9SaMGECysrKMG/ePBQXFyMmJgYZGRmmxcsKCgogl1/I1UNDQ7Fp0ybMmTMHw4YNQ3BwMGbNmoUXXnih614FEXVadYMeGQeLAXDqNxERUU/CRcqIrMPipBoAkpOTkZyc3OZ9mZmZrcoSEhLw22+/deapiKibbTxQBG2TEQP9XTEshNttEBER9RQXkmr270TdiTvAE/Vy/z6/QNkDI0Mgk3FvaiIiop5A06hHQWU9AI5UE3U3JtVEvdjJ8jrsPnUOchlw34hgqcMhIiKiLnL4/Ch1sKcTPJ3bXhyYiLoGk2qiXmztnuZR6hsG+MLfXS1xNERERNRVWqZ+c5Eyou7HpJqolzIaBdbuLQQAPDCSo9REREQ9ScvK35z6TdT9mFQT9VK7TlbizLkGuKm4NzUREVFP0zL9OzqQSTVRd2NSTdRLtSxQNm5oIPemJiIi6kGMkCOvrBYAp38TWQOTaqJeqEFnwMYDRQCAB2JDJI6GiIiIupJW5Qm9QcBd7YBgTyepwyHq8ZhUE/VCmw4Vo05nQJi3M66N8JI6HCIiIupCjeo+AICoQHdul0lkBUyqiXqhf59f9ftPI4PZ2RIREfUwDeeTal5PTWQdTKqJehm9gzO25ZYDAP40glO/iYiIeppGVXNSPTjQTeJIiHoHJtVEvcw5jwEQAoiL8EZYH2epwyEiG7V06VJERERArVYjPj4eu3bt6rD+N998g6ioKKjVagwdOhQbN260UqREdDEhhGmkejBHqomsgkk1US8ihMA5j4EAgPu5NzURtWPNmjVISUlBWloa9uzZg+HDhyMpKQmlpaVt1t+xYwceeeQRPP7449i7dy/uu+8+3HfffTh48KCVIyei0hotDA5OkMuAgf4cqSayBibVRL3IobMaaNXeUDrIMW5ooNThEJGNWrRoEaZPn45p06YhOjoay5Ytg7OzM1asWNFm/ffffx933HEHnnvuOQwePBivv/46Ro4ciQ8++MDKkRPR4aLm/akjfV25ZSaRlTCpJupF1u4pBADcFu0PDydHiaMhIluk0+mQnZ2NxMREU5lcLkdiYiKysrLabJOVlWVWHwCSkpLarU9E3efI+aSaU7+JrMdB6gCIyDr0BiPW72tOqh/g1G8iakd5eTkMBgP8/f3Nyv39/XH06NE22xQXF7dZv7i4uN3n0Wq10Gq1pmONRnMVURNRiyNFNQC4SBmRNTGpJuol/ne8DOW1Ojg01WPMAF+pwyGiXi49PR3z589vVT5hwgQ4OnImDVFnHev3Z0Dlha8/fg8Zi05LHQ6RXdPr9VdUj0k1US/RMvXbszoPjgpe+UFEbfPx8YFCoUBJSYlZeUlJCQICAtpsExAQYFF9AEhNTUVKSorpWKPRIDQ0FGvWrIG7O6etEnVGo96A6HkZgADWLHsX/u5qqUMismsajQYeHh6XrcdP1kS9QHWDHj8ebv7A61n9h8TREJEtUyqViI2NxZYtW0xlRqMRW7ZsQUJCQpttEhISzOoDwObNm9utDwAqlQru7u5mNyK6On+U1MAoAEVTA/zcVFKHQ9RrcKSaqBf44UARdE1GDPR3hepwudThEJGNS0lJwZQpUzBq1CjExcVh8eLFqKurw7Rp0wAAkydPRnBwMNLT0wEAs2bNwtixY/Huu+/irrvuwldffYXdu3fj448/lvJlEPU6LYuUOTVWQCaTSRwNUe/BpJqoF2iZ+v2nkSHYuFXiYIjI5k2YMAFlZWWYN28eiouLERMTg4yMDNNiZAUFBZDLL0x2Gz16NFavXo2XX34ZL730EgYMGIB169ZhyJAhUr0Eol6pZZEytbZC4kiIehcm1UQ9XEFFPXadrIRMBtwbE4SNUgdERHYhOTkZycnJbd6XmZnZquyhhx7CQw891M1REVFHDl80Uk1E1sNrqol6uLV7zwAAbujvg0APJ4mjISIiou4ghDBN/1YzqSayKibVRD2YEMI09fuBkSESR0NERETdpbCqATWNTXBUyKDSVkkdDlGvwqSaqAf7/eQ5FFTWw0WpQNI17W9tQ0RERPat5Xrqfr6ukMMocTREvQuTaqIebO2e5qnf44YGwkmpkDgaIiIi6i4tU7+jA7k9HZG1Makm6qEa9QZs2F8EAHggllO/iYiIerLDZ5uT6sFMqomsjkk1UQ+16VAxarRNCPFyQlyEt9ThEBERUTc6VFQNALgmiEk1kbUxqSbqof7dsjf1iGDI5TKJoyEiIqLuUt2gx+nKBgBANJNqIqtjUk3UA5VoGrHteBkA4E9c9ZuIiKhHa5n6HezpBE9npcTREPU+TKqJeqB1ewthFMCocC9E+LhIHQ4RERF1o0NnOfWbSEpMqol6GCEEvs1uXvWbo9REREQ93+HzK39fE+QhcSREvVOnkuqlS5ciIiICarUa8fHx2LVr1xW1++qrryCTyXDfffd15mmJ6ArsO1ON46W1UDnIcffwQKnDISIiom7WMv2bI9VE0rA4qV6zZg1SUlKQlpaGPXv2YPjw4UhKSkJpaWmH7U6ePIlnn30WY8aM6XSwRHR5X+8+DaB5b2p3taPE0RAREVF3atQbcLy0FgAXKSOSisVJ9aJFizB9+nRMmzYN0dHRWLZsGZydnbFixYp22xgMBkyaNAnz589HZGTkVQVMRO1r0Bnwfc5ZAMBD3JuaiIiox/ujpAYGo4CXsyMCPdRSh0PUK1mUVOt0OmRnZyMxMfHCA8jlSExMRFZWVrvtXnvtNfj5+eHxxx/vfKREdFkX7019XWQfqcMhIiKibnbo7IXrqWUybqFJJAUHSyqXl5fDYDDA39/frNzf3x9Hjx5ts822bduwfPly5OTkXPHzaLVaaLVa07FGo7EkTKJeq2Xq90OxodybmoiIqBfgyt9E0uvW1b9ramrw6KOP4pNPPoGPj88Vt0tPT4eHh4fpFhoa2o1REvUMpyvrsSOvAjIZ8EBssNThEBERkRW0jFTzemoi6Vg0Uu3j4wOFQoGSkhKz8pKSEgQEBLSqn5eXh5MnT2L8+PGmMqPR2PzEDg44duwY+vXr16pdamoqUlJSTMcajYaJNdFltGyjdX0/H4R4OUscDREREXU3g1HgaFENAG6nRSQli5JqpVKJ2NhYbNmyxbQtltFoxJYtW5CcnNyqflRUFA4cOGBW9vLLL6Ompgbvv/9+u4mySqWCSqWyJDSiXs1ovLA39UOjuEAZERFRb5BfXosGvQFOjgr09XGROhyiXsuipBoAUlJSMGXKFIwaNQpxcXFYvHgx6urqMG3aNADA5MmTERwcjPT0dKjVagwZMsSsvaenJwC0KieiztuRV4HCqga4qR2QdE3rWSNERETU87RM/R4c6AYF11IhkozFSfWECRNQVlaGefPmobi4GDExMcjIyDAtXlZQUAC5vFsv1SaiS7QsUHZvTBDUjgqJoyEiIiJruHjlbyKSjsVJNQAkJye3Od0bADIzMztsu2rVqs48JRG1o7JOh4yDxQCACaPCJI6GiIiIrIUrfxPZBg4pE9m5f2efgc5gxNBgDwwN4TfVREREvYEQgiPVRDaCSTWRHRNC4F+7CgAAj8RxlJqIiKi3OFvdiKp6PRRyGQb4u0odDlGvxqSayI79dqISJ8rr4KJU4J6YIKnDISIiIis5VNg89XuAnyvXUyGSGJNqIju2+vwo9b0jguGq6tQSCURERGSHWqZ+R/N6aiLJMakmslMVtVpsOr9A2URO/SYiIupVck5XAQCGh3hKGgcRMakmslv/3tO8QNmwEA8MCeYCJURERL2FEAL7zlQBAGJCPSWNhYiYVBPZpeYFypr3puYoNRERUe9ysqIeVfV6KB3kGBzI6d9EUmNSTWSHsk5UIL+8Dq4qB4wfzgXKiIiIepOc0+cANO9PrXTgx3kiqfFdSGSHvvjtFADg3pgguHCBMiIiol4lp6AKAKd+E9kKJtVEdqawqgGbDpUAACYnREgbDBEREVldyyJlTKqJbAOTaiI788Vvp2AwCozu1weDAtykDoeIiIisqFFvwOGi5u20RoR6SRwNEQFMqonsSqPegH+d35t66ugIaYMhIiIiqztcpIHeIODtokSot5PU4RARmFQT2ZXvcgpRVa9HiJcTbh3sL3U4REREZGUXX08tk8mkDYaIADCpJrIbQgis3H4SADA5IRwKOTtSIiKi3obXUxPZHibVRHZiV34ljhbXwMlRgQmjuDc1ERFRb9SSVI8I85Q0DiK6gEk1kZ1YteMkAOD+kcHwcHaUNhgiIiKyuopaLQoq6wEAw0I8pQ2GiEyYVBPZgeZttIoBcIEyIiKi3mrfmSoAQD9fF3g48Qt2IlvBpJrIDqzclg+jAEb364OB/txGi4iIqDe6sEgZt9IisiVMqolsXFW9DqvPb6M1/cZIiaMhIiIiqextWaSM11MT2RQm1UQ27p9Zp1CvMyAqwA03DfSVOhwiIiKSgNEosK9lkTKu/E1kU5hUE9mwBp0BK88vUPbUTf24HyUREVEvlV9RB01jE1QOcgwK4KVgRLaESTWRDft692lU1ukQ6u2Eu4YGSh0OEfUClZWVmDRpEtzd3eHp6YnHH38ctbW1Hba56aabIJPJzG5PPvmklSIm6h32nr+eemiwBxwV/AhPZEscpA6AiNqmNxjx8a8nAAAzxkTCgR0oEVnBpEmTUFRUhM2bN0Ov12PatGmYMWMGVq9e3WG76dOn47XXXjMdOzs7d3eoRL1KVl4FAODavt4SR0JEl2JSTWSjNuwvQmFVA/q4KPHQqFCpwyGiXuDIkSPIyMjA77//jlGjRgEAlixZgnHjxmHhwoUICgpqt62zszMCAgKsFSpRryKEwI68cgDA9f18JI6GiC7FoS8iGySEwIeZeQCAx27oC7WjQuKIiKg3yMrKgqenpymhBoDExETI5XLs3Lmzw7ZffvklfHx8MGTIEKSmpqK+vr7D+lqtFhqNxuxGRG07WVGPoupGKBVyxIZzOy0iW8ORaiIb9NORUhwrqYGLUoG/xIdLHQ4R9RLFxcXw8/MzK3NwcIC3tzeKi4vbbTdx4kSEh4cjKCgI+/fvxwsvvIBjx45h7dq17bZJT0/H/Pnzuyx2op6sZZR6RJgnnJT8op3I1jCpJrIxRqPAuz8eAwA8mhABD2dHiSMiInv34osv4u233+6wzpEjRzr9+DNmzDD9PnToUAQGBuLWW29FXl4e+vXr12ab1NRUpKSkmI41Gg1CQ3mpC1FbduQ2X089mlO/iWwSk2oiG7PhQBGOFtfATeWAJ8dGSh0OEfUAc+fOxdSpUzusExkZiYCAAJSWlpqVNzU1obKy0qLrpePj4wEAubm57SbVKpUKKpXqih+TqLcyGgWyTjQn1df37yNxNETUFibVRDakyWDEez/9AQB4YkwkPJ2VEkdERD2Br68vfH19L1svISEBVVVVyM7ORmxsLADg559/htFoNCXKVyInJwcAEBjIrQCJrtaxkhpU1ungrFRgWIin1OEQURu4UBmRDfnP3kKcKKuDl7MjHrshQupwiKiXGTx4MO644w5Mnz4du3btwvbt25GcnIyHH37YtPJ3YWEhoqKisGvXLgBAXl4eXn/9dWRnZ+PkyZNYv349Jk+ejBtvvBHDhg2T8uUQ9Qjbc5uvp742whtKB350J7JFfGcS2QhdkxHvbzkOAHhybD+4qXktNRFZ35dffomoqCjceuutGDduHG644QZ8/PHHpvv1ej2OHTtmWt1bqVTip59+wu23346oqCjMnTsXDzzwAL7//nupXgJRj9KyPzWnfhPZrk5N/166dCkWLFiA4uJiDB8+HEuWLEFcXFybdT/55BN8/vnnOHjwIAAgNjYWb775Zrv1iXqrNbtP48y5Bvi6qTA5IULqcIiol/L29sbq1avbvT8iIgJCCNNxaGgofvnlF2uERtTrNBmM2JlfCYCLlBHZMotHqtesWYOUlBSkpaVhz549GD58OJKSklotbNIiMzMTjzzyCLZu3YqsrCyEhobi9ttvR2Fh4VUHT9RTNOoN+ODn5lHq5Jv7c7sMIiIiwoHCatRqm+Dh5IjBge5Sh0NE7bA4qV60aBGmT5+OadOmITo6GsuWLYOzszNWrFjRZv0vv/wSTz/9NGJiYhAVFYVPP/0URqMRW7ZsuergiXqKj345gRKNFsGeTng4jlvKEBEREbDj/NTv6yK9oZDLJI6GiNpjUVKt0+mQnZ2NxMTECw8glyMxMRFZWVlX9Bj19fXQ6/Xw9va2LFKiHqqwqgEf/pILAHjxziioHDhKTURERMCOvOZFyq7vz6nfRLbMomuqy8vLYTAY4O/vb1bu7++Po0ePXtFjvPDCCwgKCjJLzC+l1Wqh1WpNxxqNxpIwiexK+sYjaNQbEdfXG3cP4/YzRERE1Hxp2O6T5wAAo/txkTIiW2bV1b/feustfPXVV/jPf/4DtVrdbr309HR4eHiYbqGhnA5LPdPOExX47/4iyGVA2vhoyGSc2kVERETA7pPnoG0ywtdNhX6+rlKHQ0QdsCip9vHxgUKhQElJiVl5SUkJAgICOmy7cOFCvPXWW/jxxx8vu29lamoqqqurTbfTp09bEiaRXTAYBeZ/fxgA8HBcGK4J8pA4IiIiIrIVPxwsAgDcMsiPX7oT2TiLkmqlUonY2FizRcZaFh1LSEhot90777yD119/HRkZGRg1atRln0elUsHd3d3sRtTTrPn9NA4XaeCmdsDc2wZKHQ4RERHZCINRYNOhYgDAOF4aRmTzLN6nOiUlBVOmTMGoUaMQFxeHxYsXo66uDtOmTQMATJ48GcHBwUhPTwcAvP3225g3bx5Wr16NiIgIFBc3/4FwdXWFqyunslDvVFmnw8IfjwEA5iQORB9XlcQRERERka3YlV+J8lodPJwceT01kR2wOKmeMGECysrKMG/ePBQXFyMmJgYZGRmmxcsKCgogl18YAP/www+h0+nw4IMPmj1OWloaXn311auLnshOpa0/hMo6HQb5u+HRhHCpwyEiIiIbsvFA89Tv26P94aiw6hJIRNQJFifVAJCcnIzk5OQ278vMzDQ7PnnyZGeegqjH2nSoGN/vOwuFXIYFDw1jZ0lEREQmBqNABqd+E9kVfponsqKqeh3+9p+DAIAZN0ZiWIintAERERGRTdl9shJlNVq4qx1wfT/uT01kD5hUE1nR/O8Po7xWi/5+rph16wCpwyEiIiIb0zL1+7boACgd+FGdyB7wnUpkJVuOlOA/ewshlwHvPDgMakeF1CERERGRDTEaBX442Dz1+65hHW9XS0S2g0k1kRWU1Wjx4toDAIAnxkRiZJiXxBERERGRrckuOIfSGi3c1A64vj+nfhPZCybVRN3MYBSY9dVelNVoMcDPFSnck5qIiIjaYJr6PdgfKgfOaCOyF0yqibrZ4p/+wI68CjgrFfjwLyM57ZuIiIhaMRoFfjhwftXvoVz1m8ieMKkm6kaZx0qx5OdcAED6n4aiv5+bxBERERGRLdqWW45iTSPcVA4YM5BTv4nsCZNqom5ytqoBc9bkAAAmxYfh3phgaQMiIiIim7V8Wz4A4IHYEE79JrIzTKqJukG9rglPfZGNc/V6DAl2xyt3R0sdEhEREdmo3NIa/PJHGWQyYNr1EVKHQ0QWYlJN1MWaDEY8s3ov9p2phqezI/4xMZbXURMREVG7lm87CaB5gbLwPi7SBkNEFmNSTdSFhBB45btD2HK0FCoHOZZPGYWwPs5Sh0VEREQ2qrJOh7V7zgAAHr+hr8TREFFnMKkm6kL/yMzDv3YVQCYD3n94BGLDvaUOiYiIiGzY6p2noG0yYkiwO+L68nMDkT1iUk3URb7efRoLNh0DAMy/5xrcMSRA4oiIiIjIlumajPg86xSA5lFqmUwmcURE1BlMqom6wOqdBXjh3/sBAH8dG4nJCRHSBkREREQ277/7z6K0Rgs/NxXuGhokdThE1EkOUgdAZO9Wbs/H/O8PAwCmjo7AC0lREkdEREREtk4IYdpGa8roCCgdONZFZK+YVBNdhQ8z8/B2xlEAzSPUL94RxalbREREdFn/3lOIQ2c1cHJUYGJcmNThENFVYFJN1AkGo8A7GUfx0a8nAACzbh2A2YkDmFATERHRZVXV65C+8QgA4P9uHQAvF6XEERHR1WBSTWQhTaMes/61F1uPlQEAnr9jEJ6+qb/EUREREZG9WLDpGCrqdBjg58pttIh6ACbVRBY4UVaLJz7fjRNldVA7yvHOg8Nxz3AuLEJERERXJud0FVbvKgAAvH7fEF5LTdQDMKkmukIZB4vw3Lf7UdPYhEAPNT6ZPApDgj2kDouIiIjshMEo8PK6AxAC+NOIYFwX2UfqkIioCzCpJrqM6gY95q8/hLV7CwEAseFeWPaXWPi6qSSOjIiIiOzJF7+dwsFCDdzUDkgdN1jqcIioizCpJurAtuPleO7bfSiqboRcBjx1Uz/MunUgp2oRERGRRfafqcJbPzTvGPJ80iB+OU/UgzCpJmpDUXUD3v7hKNblnAUAhPdxxqI/D0dsuLfEkREREZG9OXOuHo+t2o0GvQFjB/piYny41CERURdiUk10kUa9AZ/8egL/yMxDg94AmQyYFB+G1DsHw0XFtwsRERFZprpBj8dW/Y7yWi2iAtzwwcQRUMi5BSdRT8IsgQjNyfQ32WewLDMPhVUNAJqvnU4bH41hIZ7SBkdERER2SddkxNNfZuOPklr4u6uwctq1cFM7Sh0WEXUxJtXUq9XrmrB6ZwE+/vUESmu0AIBADzVSxw3G+GGBkMn4TTIRERFZrkFnwLPf7MP23Ao4KxVYPuVaBHo4SR0WEXUDJtXUK/1RUoN/7SrA2j2FqG7QAwCCPNSYcWMkHo4Lg9pRIXGEREREZK/OnKvHX/+ZjUNnNVDIZfhg4ghuw0nUgzGppl6jql6HHw+X4OvfT2P3qXOm8og+znjqpn64f0QIV/UmIiKiq7IjrxzJq/eisk4Hbxcllk4ciYR+3I+aqCdjUk09WkWtFj8dKcHGA8XYnluOJqMAACjkMiQO9sPE+HCM6e8DORcMISIioqtQp23CR7/kYWlmHgxGgWuC3PHRo7EI8XKWOjQi6mZMqqlHadQbsLegCr8eL8P/jpfhYKHG7P6oADeMHx6Eh2JD4OeulihKIiIi6imaDEas2X0a720+jvLa5vVZ7h8RjPQ/DeXlZES9BJNqsltCCJw514CDhdXIPnUO2QXncKhQA53BaFYvOtAd44YG4M6hgejn6ypRtERERNSTaBr12LC/CMu35SO3tBYAEN7HGS/cEYU7hwRwsVOiXqRTSfXSpUuxYMECFBcXY/jw4ViyZAni4uLarf/NN9/glVdewcmTJzFgwAC8/fbbGDduXKeDpt5FCIFiTSPyy+qQV16HvNJaHC7S4EiRBjWNTa3q+7qpMKa/D8YM9MH1/X3g58YRaSKiK/X3v/8dGzZsQE5ODpRKJaqqqi7bRgiBtLQ0fPLJJ6iqqsL111+PDz/8EAMGDOj+gImsqFFvwG8nKrB2TyE2HSqGtqn5i3wvZ0f8360DMCk+nOuzEPVCFifVa9asQUpKCpYtW4b4+HgsXrwYSUlJOHbsGPz8/FrV37FjBx555BGkp6fj7rvvxurVq3Hfffdhz549GDJkSJe8CLJfQgjU6wwordGiRNOIEk0jiqsbUVjVgDPnGnDmXD3OnGtAvc7QZntHhQwD/d0wIswTseFeiA3zRqi3E78dJiLqJJ1Oh4ceeggJCQlYvnz5FbV555138P/+3//DZ599hr59++KVV15BUlISDh8+DLWaX2yS/aqq1+FIUQ125Vci60Q59hRUQdd0YUbcAD9XPBgbgofjwuDhxP2niXormRBCWNIgPj4e1157LT744AMAgNFoRGhoKJ555hm8+OKLrepPmDABdXV1+O9//2squ+666xATE4Nly5Zd0XNqNBp4eHiguroa7u7uloRL3cxoFGhsMqBBZ0C9zoA6XRPqtAbUaZtQ09iEWq0eNY1N0DToUd2gR1WDHlX1epyr16GiVoeKOi0a9cbLPo9CLkOYtzMifVwQ6euCqAB3RAe5o5+vK78RttA999yD9evXSx0GkV3rDf3SqlWrMHv27MuOVAshEBQUhLlz5+LZZ58FAFRXV8Pf3x+rVq3Cww8/fEXP1xvOKdkWIQTqdAacq9OhRNOIs9WNKKpqwNmqBuSV1eFYSQ3KarSt2vm6qXDnkAA8GBuCocEeNvlFPvt6oq5xpX2TRSPVOp0O2dnZSE1NNZXJ5XIkJiYiKyurzTZZWVlISUkxK0tKSsK6desseeou0ag3YGd+ZYd1Lv2OodU3DuLSQ3G+Hcx/XvJ44qL7AAEhLq7T/DgtZaY2F5UbRXN5y/0tZUYhIIQw/W4wttwnYBACRqOAwdh8bBQCTcbmOi03vcF4/qdAk9GIJkNzWfNNQNdkhM5gNP3UNhmg1RvRqDegsclo9m3t1XBWKhDgroafuwr+7moEezohxMsZIV5OCPZyQqiXM5NnIiIblJ+fj+LiYiQmJprKPDw8EB8fj6ysrCtOqrtS9qlzaGhnhhN1nmj9qcj8/lafkVrKhdkxTJ+Vzn/2uejzz4XPS5d81gHOf6YRps80BiNgMBpNn210BiP0Tc2fY3SG5s8qWr0RjU3nv/jXNqHu/E9NQ/MX/HrD5ceWgj2dMCLME9dF9kFCvz6I9HGxyUSaiKRjUVJdXl4Og8EAf39/s3J/f38cPXq0zTbFxcVt1i8uLm73ebRaLbTaC98MajSadutaoqpejykrdnXJY1FrTo4KuKgUcFY6wFmpgLvaEa5qB7idv3k6KeHp7Ah3J0d4OSvh46qEj6sK3i5KuKi4Zh4RkT1q6c9tpa8HgBf/vR/Hzy8cRXQ5Kgc5fN1UCPJwQpCnGoGeTujbxwUD/F0xwN8NrvyMQkSXYZN/JdLT0zF//vxW5RMmTICjY+evV2lSqKEOu6uNe8y/pbz8d4+X+1az+X6ZaF3W6rlE6/tkwEVf97aUtQxtX3zc1u9GQAhTmdlPCMhEy/1GyIQRMiEA0+8X3VrKjAbIhAFyYYRMNEEuDJAZmyATRsiNTZCLJsiNTZAJw/nnuKDx/K38MmeLrGvXrl245557pA6DyK7p9XqpQ7DIiy++iLfffrvDOkeOHEFUVJSVImq/r+8KfX1coJBzJNEaLh2xvfSsXzqg23IsO19TJjvfRtZc0nIsk8mgOH8glzXXV8hlkMtlkMsAhaz52EEhg0Iuh0IGKB3kcFS03GRQOyqgdlRA5SCH2lEBN7UDXJQOcFE1f9nv7aKEl7MSTkpue0VEV8eipNrHxwcKhQIlJSVm5SUlJQgICGizTUBAgEX1ASA1NdVsyrhGo0FoaCjWrFnD66yIrhKvsyK6ei3XWNmLuXPnYurUqR3WiYyM7NRjt/TnJSUlCAwMNJWXlJQgJiam3Xbt9fVd4ePJo7rkcYiIiK6ERUm1UqlEbGwstmzZgvvuuw9A80JlW7ZsQXJycpttEhISsGXLFsyePdtUtnnzZiQkJLT7PCqVCiqVypLQiIiIqB2+vr7w9fXtlsfu27cvAgICsGXLFlMSrdFosHPnTjz11FPttmNfT0REPYXFKz+lpKTgk08+wWeffYYjR47gqaeeQl1dHaZNmwYAmDx5stlCZrNmzUJGRgbeffddHD16FK+++ip2797dbhJORERE0ikoKEBOTg4KCgpgMBiQk5ODnJwc1NZeuEY5KioK//nPfwA0T9OdPXs23njjDaxfvx4HDhzA5MmTERQUZPoCnoiIqCez+JrqCRMmoKysDPPmzUNxcTFiYmKQkZFhWqCkoKAAcvmFXH306NFYvXo1Xn75Zbz00ksYMGAA1q1bxz2qiYiIbNC8efPw2WefmY5HjBgBANi6dStuuukmAMCxY8dQXV1tqvP888+jrq4OM2bMQFVVFW644QZkZGRwj2oiIuoVLN6nWgrcu5Ko6/CaaqKrx36p6/GcEnUd9vVEXeNK+yZu/EtERERERETUSUyqiYiIiIiIiDqJSTURERERERFRJ1m8UJkUWi771mg0EkdCZP/0ej3fS0RXqeU9ZAfLktgN9vVEXYd9PVHXuNL+3i6S6pqaGgBAaGioxJEQ9QweHh5Sh0DUI9TU1PD91EXY1xN1Lf5tIuo6l+vv7WL1b6PRiLNnz8LNzQ0ymeyqHkuj0SA0NBSnT5+2y9VFGb+0GL+0GL+0GP8FQgjU1NQgKCjIbBtJ6jz29RcwfmkxfmkxfmnZe/yANP29XYxUy+VyhISEdOljuru72+0/FIDxS43xS4vxS4vxN+MoUNdiX98a45cW45cW45eWvccPWLe/59frRERERERERJ3EpJqIiIiIiIiok3pdUq1SqZCWlgaVSiV1KJ3C+KXF+KXF+KXF+Mle2Pv/a8YvLcYvLcYvLXuPH5DmNdjFQmVEREREREREtqjXjVQTERERERERdRUm1URERERERESdxKSaiIiIiIiIqJOYVBMRERERERF1Uo9Lqv/+979j9OjRcHZ2hqenZ5t1CgoKcNddd8HZ2Rl+fn547rnn0NTU1OHjVlZWYtKkSXB3d4enpycef/xx1NbWdsMrMJeZmQmZTNbm7ffff2+33U033dSq/pNPPtnt8bYlIiKiVSxvvfVWh20aGxsxc+ZM9OnTB66urnjggQdQUlJipYgvOHnyJB5//HH07dsXTk5O6NevH9LS0qDT6TpsJ+X5X7p0KSIiIqBWqxEfH49du3Z1WP+bb75BVFQU1Go1hg4dio0bN1olzkulp6fj2muvhZubG/z8/HDffffh2LFjHbZZtWpVq/OsVqutFLG5V199tVUsUVFRHbaxlXMPtP0+lclkmDlzZpv1pT73v/76K8aPH4+goCDIZDKsW7fO7H4hBObNm4fAwEA4OTkhMTERx48fv+zjWvr+IWmwr2/Gvr5rsK+3Hnvv6wH29+zv2yF6mHnz5olFixaJlJQU4eHh0er+pqYmMWTIEJGYmCj27t0rNm7cKHx8fERqamqHj3vHHXeI4cOHi99++03873//E/379xePPPJIN72KC7RarSgqKjK7PfHEE6Jv377CaDS2227s2LFi+vTpZu2qq6u7Pd62hIeHi9dee80sltra2g7bPPnkkyI0NFRs2bJF7N69W1x33XVi9OjRVor4gh9++EFMnTpVbNq0SeTl5YnvvvtO+Pn5iblz53bYTqrz/9VXXwmlUilWrFghDh06JKZPny48PT1FSUlJm/W3b98uFAqFeOedd8Thw4fFyy+/LBwdHcWBAwe6PdZLJSUliZUrV4qDBw+KnJwcMW7cOBEWFtbhv5WVK1cKd3d3s/NcXFxsxagvSEtLE9dcc41ZLGVlZe3Wt6VzL4QQpaWlZrFv3rxZABBbt25ts77U537jxo3ib3/7m1i7dq0AIP7zn/+Y3f/WW28JDw8PsW7dOrFv3z5xzz33iL59+4qGhoZ2H9PS9w9Jh319M/b1XYN9vfXYe18vBPt79vdt63FJdYuVK1e22dFu3LhRyOVys38MH374oXB3dxdarbbNxzp8+LAAIH7//XdT2Q8//CBkMpkoLCzs8tg7otPphK+vr3jttdc6rDd27Fgxa9Ys6wR1GeHh4eK999674vpVVVXC0dFRfPPNN6ayI0eOCAAiKyurGyK0zDvvvCP69u3bYR2pzn9cXJyYOXOm6dhgMIigoCCRnp7eZv0///nP4q677jIri4+PF3/961+7Nc4rUVpaKgCIX375pd067b3PpZCWliaGDx9+xfVt+dwLIcSsWbNEv3792v1Ab0vn/tJO1mg0ioCAALFgwQJTWVVVlVCpVOJf//pXu49j6fuHpMe+nn19d2Ffbx321tcLwf5eSrbc3/e46d+Xk5WVhaFDh8Lf399UlpSUBI1Gg0OHDrXbxtPTE6NGjTKVJSYmQi6XY+fOnd0e88XWr1+PiooKTJs27bJ1v/zyS/j4+GDIkCFITU1FfX29FSJs21tvvYU+ffpgxIgRWLBgQYdT8LKzs6HX65GYmGgqi4qKQlhYGLKysqwRboeqq6vh7e192XrWPv86nQ7Z2dlm500ulyMxMbHd85aVlWVWH2h+P9jKeQZw2XNdW1uL8PBwhIaG4t577233fWwNx48fR1BQECIjIzFp0iQUFBS0W9eWz71Op8MXX3yBxx57DDKZrN16tnTuL5afn4/i4mKz8+vh4YH4+Ph2z29n3j9ku9jXS4N9Pft6S9ljXw+wv5f6/Lewpf7eodMt7VRxcbFZJwvAdFxcXNxuGz8/P7MyBwcHeHt7t9umuyxfvhxJSUkICQnpsN7EiRMRHh6OoKAg7N+/Hy+88AKOHTuGtWvXWinSC/7v//4PI0eOhLe3N3bs2IHU1FQUFRVh0aJFbdYvLi6GUqlsdZ2cv7+/1c/3pXJzc7FkyRIsXLiww3pSnP/y8nIYDIY2/30fPXq0zTbtvR+kPs9GoxGzZ8/G9ddfjyFDhrRbb9CgQVixYgWGDRuG6upqLFy4EKNHj8ahQ4cu+x7pavHx8Vi1ahUGDRqEoqIizJ8/H2PGjMHBgwfh5ubWqr6tnnsAWLduHaqqqjB16tR269jSub9Uyzm05Px25v1Dtot9Pfv6q8G+3jrssa8H2N9Lff4vZkv9vV0k1S+++CLefvvtDuscOXLksosE2JLOvKYzZ85g06ZN+Prrry/7+DNmzDD9PnToUAQGBuLWW29FXl4e+vXr1/nAz7Mk/pSUFFPZsGHDoFQq8de//hXp6elQqVRXHUtndOb8FxYW4o477sBDDz2E6dOnd9i2u89/Tzdz5kwcPHgQ27Zt67BeQkICEhISTMejR4/G4MGD8dFHH+H111/v7jDN3Hnnnabfhw0bhvj4eISHh+Prr7/G448/btVYrtby5ctx5513IigoqN06tnTuqWdgX9+MfX3XYV9v2+yxrwfY30t9/m2VXSTVc+fO7fAbFACIjIy8oscKCAhotbpby0qTAQEB7bYpLS01K2tqakJlZWW7bS6nM69p5cqV6NOnD+655x6Lny8+Ph5A87evXfGH/mr+n8THx6OpqQknT57EoEGDWt0fEBAAnU6Hqqoqs2+wS0pKOn2+L2Vp/GfPnsXNN9+M0aNH4+OPP7b4+br6/LfFx8cHCoWi1cqpHZ23gIAAi+pbQ3JyMv773//i119/tfgbUEdHR4wYMQK5ubndFN2V8/T0xMCBA9uNxRbPPQCcOnUKP/30k8UjLbZ07lvOYUlJCQIDA03lJSUliImJabNNZ94/1LXY1zdjX8++viPs622rvwHY30vJpvr7Tl+NbeMut3jJxau7ffTRR8Ld3V00Nja2+Vgti5fs3r3bVLZp0yarLl5iNBpF3759L7sSZXu2bdsmAIh9+/Z1cWSW++KLL4RcLheVlZVt3t+yeMm3335rKjt69Khki5ecOXNGDBgwQDz88MOiqampU49hrfMfFxcnkpOTTccGg0EEBwd3uHjJ3XffbVaWkJAgyeIZRqNRzJw5UwQFBYk//vijU4/R1NQkBg0aJObMmdPF0VmupqZGeHl5iffff7/N+23p3F8sLS1NBAQECL1eb1E7Kc892lm4ZOHChaay6urqK1q4xJL3D0mPfb059vWdx77eOnpaXy8E+3trsuX+vscl1adOnRJ79+4V8+fPF66urmLv3r1i7969oqamRghxYZuN22+/XeTk5IiMjAzh6+trts3Gzp07xaBBg8SZM2dMZXfccYcYMWKE2Llzp9i2bZsYMGCAVbbZaPHTTz8JAOLIkSOt7jtz5owYNGiQ2LlzpxBCiNzcXPHaa6+J3bt3i/z8fPHdd9+JyMhIceONN1ot3hY7duwQ7733nsjJyRF5eXniiy++EL6+vmLy5Mntxi9E8zYbYWFh4ueffxa7d+8WCQkJIiEhwerxnzlzRvTv31/ceuut4syZM2bbCbQXv5Tn/6uvvhIqlUqsWrVKHD58WMyYMUN4enqaVsB99NFHxYsvvmiqv337duHg4CAWLlwojhw5ItLS0iTb5uGpp54SHh4eIjMz0+w819fXm+pcGv/8+fNNW6BkZ2eLhx9+WKjVanHo0CGrxz937lyRmZkp8vPzxfbt20ViYqLw8fERpaWlbcZuS+e+hcFgEGFhYeKFF15odZ+tnfuamhrT33cAYtGiRWLv3r3i1KlTQojmLTY8PT3Fd999J/bv3y/uvffeVlts3HLLLWLJkiWm48u9f8h2sK9nX9+V2Ndbj7339UKwv2d/37Yel1RPmTJFAGh1u3jvtZMnT4o777xTODk5CR8fHzF37lyzb2m2bt0qAIj8/HxTWUVFhXjkkUeEq6urcHd3F9OmTTN13tbwyCOPtLt3Y35+vtlrLCgoEDfeeKPw9vYWKpVK9O/fXzz33HOS7F2ZnZ0t4uPjhYeHh1Cr1WLw4MHizTffNBspuDR+IYRoaGgQTz/9tPDy8hLOzs7i/vvvN+vcrGXlypVt/nu6eJKHrZ3/JUuWiLCwMKFUKkVcXJz47bffTPeNHTtWTJkyxaz+119/LQYOHCiUSqW45pprxIYNG6wS56XaO88rV6401bk0/tmzZ5teq7+/vxg3bpzYs2eP9YMXQkyYMEEEBgYKpVIpgoODxYQJE0Rubq7pfls+9y02bdokAIhjx461us/Wzn3L3+lLby0xGo1G8corrwh/f3+hUqnErbfe2up1hYeHi7S0NLOyjt4/ZDvY10vf11yMfT37+itl7329EOzv2d+3TSaEEJ2fPE5ERERERETUe/W6faqJiIiIiIiIugqTaiIiIiIiIqJOYlJNRERERERE1ElMqomIiIiIiIg6iUk1ERERERERUScxqSYiIiIiIiLqJCbVRERERERERJ3EpJqIiIiIiIiok5hUExEREREREXUSk2oiIiIiIiKiTmJSTURERERERNRJTKqJiIiIiIiIOun/A3yVUY/QxGLmAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1200x300 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.linspace(-10, 10, 100)\n",
    "y1 = torch.sigmoid(x)\n",
    "y2 = torch.tanh(x)\n",
    "plt.figure(figsize=(12, 3))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.axhline(0, color='black', lw=.5)\n",
    "plt.axvline(0, color='black', lw=.5)\n",
    "plt.plot(x, y1)\n",
    "plt.title('Sigmoid')\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.axhline(0, color='black', lw=.5)\n",
    "plt.axvline(0, color='black', lw=.5)\n",
    "plt.plot(x, y2)\n",
    "plt.title('Tanh');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sigmoid activation function takes an input and transforms it into a value between 0 and 1. If the preactivation is obtained from learnable weights, you can think as the output of the sigmoid as a faucet valve, that learns to open or close depending on the input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output before the new data flows: tensor([0, 1, 0])\n",
      "Flow rate: 0.11920291930437088\n",
      "Output after the new data flows: tensor([0.1192, 1.1192, 0.1192])\n"
     ]
    }
   ],
   "source": [
    "data = torch.tensor([-3, 2, 10])  # this is our input, the water coming from the pipe to the faucet\n",
    "gate = torch.tensor([0, -1, 0])  # this is our faucet, a gate that, given an input, let's it flow or not\n",
    "new_hidden_state = torch.tensor([0, 1, 0])  # this is what we have already at the other side of the faucet\n",
    "\n",
    "print(f'Output before the new data flows: {new_hidden_state}')\n",
    "flow_rate = torch.sigmoid(data @ gate)  # based on the input data, we decide how much water to let flow\n",
    "print(f'Flow rate: {flow_rate.item()}')\n",
    "new_hidden_state = new_hidden_state + flow_rate  # we add the new water to the existing water\n",
    "print(f'Output after the new data flows: {new_hidden_state}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, you can think of $tanh$ as a function that squashes the input into a value between -1 and 1. This is useful because it can learn to keep the gradients in a reasonable range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output after the activation function: tensor([0.1186, 0.8073, 0.1186])\n"
     ]
    }
   ],
   "source": [
    "new_hidden_state = new_hidden_state.tanh()\n",
    "print(f'Output after the activation function: {new_hidden_state}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Understanding these building blocks is crucial to understand how the LSTM works. Let's see the architecture of the LSTM cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "short_term = torch.zeros(3)  # internal state of the LSTM\n",
    "long_term = torch.zeros(3)  # internal state of the LSTM\n",
    "input = torch.tensor([1, 2, 3], dtype=torch.float32)  # new input\n",
    "\n",
    "# forget gate: decide how much will be preserved from the long term, using learned weights\n",
    "W_f = torch.randn(3, 3)  # weights for the input\n",
    "U_f = torch.randn(3, 3)  # weights for the memory\n",
    "b_f = torch.randn(3)  # bias\n",
    "preserve_rate = torch.sigmoid(input @ W_f + short_term @ U_f + b_f)\n",
    "\n",
    "# input gate: decide how much of the input will be added to the memory, using learned weights\n",
    "W_i = torch.randn(3, 3)  # weights for the input\n",
    "U_i = torch.randn(3, 3)  # weights for the memory\n",
    "b_i = torch.randn(3)  # bias\n",
    "add_rate = torch.sigmoid(input @ W_i + short_term @ U_i + b_i)\n",
    "\n",
    "# state candidate: decide what to add to the memory from the input, using learned weights\n",
    "W_c = torch.randn(3, 3)  # weights for the input\n",
    "U_c = torch.randn(3, 3)  # weights for the memory\n",
    "b_c = torch.randn(3)  # bias\n",
    "state_candidate = torch.tanh(input @ W_c + short_term @ U_c + b_c)  # note that tanh means that the values are between -1 and 1\n",
    "\n",
    "# output gate: build an output based on the input and memory\n",
    "W_o = torch.randn(3, 3)  # weights for the input\n",
    "U_o = torch.randn(3, 3)  # weights for the memory\n",
    "b_o = torch.randn(3)  # bias\n",
    "output_rate = torch.sigmoid(input @ W_o + short_term @ U_o + b_o)\n",
    "\n",
    "# update the memory\n",
    "new_long_term = long_term * preserve_rate + state_candidate * add_rate\n",
    "new_short_term = output_rate * torch.tanh(new_long_term)\n",
    "\n",
    "# build an output\n",
    "W_out = torch.randn(3, 3)  # weights for the output\n",
    "b_out = torch.randn(3)  # bias\n",
    "output = new_short_term @ W_out + b_out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few things to note from the previous code:\n",
    "* The forget gate is a \"learnable faucet\" that regulates how much of the long term memory should be kept.\n",
    "* The input gate is another learnable faucet that regulates how much of the new information should make it to the long term memory.\n",
    "* The state candidate is the new information that will be added to the long term memory. In this model, information ranges in $(-1, 1)$ and faucets range in $(0, 1)$. Since this is just the input info transformed, we use $tanh$ and not sigmoid.\n",
    "* The memory will be updated by the forget gate and the input gate. The forget gate will decide how much of the old memory should be kept, and the input gate will decide how much of the new memory should be added.\n",
    "* The output gate will decide how much of the long term memory should be outputted. This is useful because the long term memory can be used to predict the next token in a sequence, but it can also be used to predict the next hidden state in a sequence.\n",
    "\n",
    "This is a complicated architecture, but it is very powerful. It can learn long-term dependencies and can be used in a variety of tasks, like language modeling, machine translation, and speech recognition. The folklore says that minimal RNNs usually don't work well with sequences longer than ~10 steps, while LSTMs can work well until ~1000 steps. Even though more modern architetures like Transformers can take inputs of 32k tokens and more, as of 2024, growing the sequence length is still an active field of research and remains and open challenge.\n",
    "\n",
    "Let's replicate the testing environment from the previous notebook, but using LSTMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocabulary = ascii_letters + digits + ' .,;\\''\n",
    "c2i = {c: i for i, c in enumerate(vocabulary)}\n",
    "i2c = {i: c for i, c in enumerate(vocabulary)}\n",
    "\n",
    "def vectorize_sentence(s):\n",
    "\treturn [c2i[c] for c in normalize('NFD', s) if category(c) != 'Mn' and c in vocabulary]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 6750521 sentences from 8 languages using 67 different characters\n",
      "tensor([11, 62, 27,  0, 15, 19,  8, 18, 19, 62, 37, 20,  8, 62, 38,  8],\n",
      "       device='cuda:0') tensor(4, device='cuda:0') -> l Baptist Lui Mi en\n"
     ]
    }
   ],
   "source": [
    "with open('custom-data/sentences.json', encoding='utf-8') as f:\n",
    "\tdata = loads(f.read())\n",
    "\n",
    "languages = list(data.keys())\n",
    "\n",
    "X, Y = [], []\n",
    "sentence_size = 16\n",
    "for language, sentences in data.items():\n",
    "\tfor sentence in sentences:\n",
    "\t\tsentence = vectorize_sentence(sentence)\n",
    "\t\tfor i in range(len(sentence) - sentence_size + 1):\n",
    "\t\t\tX.append(sentence[i:i+sentence_size])\n",
    "\t\t\tY.append(languages.index(language))\n",
    "ix = torch.randperm(len(X))\n",
    "X = torch.tensor([X[i] for i in ix], device=device)\n",
    "Y = torch.tensor([Y[i] for i in ix], device=device)\n",
    "\n",
    "print(f'Loaded {len(X)} sentences from {len(languages)} languages using {len(vocabulary)} different characters')\n",
    "print(X[0], Y[0], '->', ''.join(i2c[i.item()] for i in X[0]), languages[Y[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created model with 333832 parameters\n"
     ]
    }
   ],
   "source": [
    "def get_model(input_size, hidden_size, output_size):\n",
    "\t# forget gate\n",
    "\tW_f = torch.randn(input_size, hidden_size, device=device)\n",
    "\tU_f = torch.randn(hidden_size, hidden_size, device=device)\n",
    "\tb_f = torch.zeros(hidden_size, device=device)\n",
    "\t# input gate\n",
    "\tW_i = torch.randn(input_size, hidden_size, device=device) * .1\n",
    "\tU_i = torch.randn(hidden_size, hidden_size, device=device) * .1\n",
    "\tb_i = torch.zeros(hidden_size, device=device)\n",
    "\t# state candidate\n",
    "\tW_c = torch.randn(input_size, hidden_size, device=device) * .1\n",
    "\tU_c = torch.randn(hidden_size, hidden_size, device=device) * .1\n",
    "\tb_c = torch.zeros(hidden_size, device=device)\n",
    "\t# output gate\n",
    "\tW_o = torch.randn(input_size, hidden_size, device=device) * .1\n",
    "\tU_o = torch.randn(hidden_size, hidden_size, device=device) * .1\n",
    "\tb_o = torch.zeros(hidden_size, device=device)\n",
    "\t# output transformation\n",
    "\tW_out = torch.randn(hidden_size, output_size, device=device) * .1\n",
    "\tb_out = torch.zeros(output_size, device=device)\n",
    "\t# parameters\n",
    "\tparameters = W_f, U_f, b_f, W_i, U_i, b_i, W_c, U_c, b_c, W_o, U_o, b_o, W_out, b_out\n",
    "\tfor p in parameters: p.requires_grad = True\n",
    "\treturn parameters\n",
    "\n",
    "def forward_character(model, input, memory):\n",
    "\tW_f, U_f, b_f, W_i, U_i, b_i, W_c, U_c, b_c, W_o, U_o, b_o, W_out, b_out = model\n",
    "\t# compute rates and candidate long term\n",
    "\tshort_term, long_term = memory.chunk(2)\n",
    "\tpreserve_rate = torch.sigmoid(input @ W_f + short_term @ U_f + b_f)\n",
    "\tadd_rate = torch.sigmoid(input @ W_i + short_term @ U_i + b_i)\n",
    "\tstate_candidate = torch.tanh(input @ W_c + short_term @ U_c + b_c)\n",
    "\toutput_rate = torch.sigmoid(input @ W_o + short_term @ U_o + b_o)\n",
    "\t# update memory\n",
    "\tnew_long_term = long_term * preserve_rate + state_candidate * add_rate\n",
    "\tnew_short_term = output_rate * new_long_term.tanh()\n",
    "\tmemory = torch.cat([new_short_term, new_long_term])\n",
    "\t# compute output\n",
    "\toutput = new_short_term @ W_out + b_out\n",
    "\treturn output, memory\n",
    "\n",
    "one_hot = torch.eye(len(vocabulary), device=device)\n",
    "def forward_sequence(model, input):\n",
    "\tmemory = torch.zeros(2 * model[0].shape[1], device=device)\n",
    "\tfor i in range(input.size(-1)):\n",
    "\t\ti = one_hot[input[..., i]]\n",
    "\t\toutput, memory = forward_character(model, i, memory)\n",
    "\treturn output\n",
    "\n",
    "model = get_model(len(vocabulary), 256, len(languages))\n",
    "print(f'Created model with {sum(p.numel() for p in model)} parameters')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0 loss: 2.1045 remaining: 12:10\n",
      "Epoch  500 loss: 2.0945 remaining: 2:42\n",
      "Epoch 1000 loss: 1.6269 remaining: 2:22\n",
      "Epoch 1500 loss: 1.4186 remaining: 2:03\n",
      "Epoch 2000 loss: 1.2514 remaining: 1:47\n",
      "Epoch 2500 loss: 1.5482 remaining: 1:29\n",
      "Epoch 3000 loss: 1.3566 remaining: 1:11\n",
      "Epoch 3500 loss: 1.3483 remaining: 0:54\n",
      "Epoch 4000 loss: 1.2110 remaining: 0:36\n",
      "Epoch 4500 loss: 1.1295 remaining: 0:18\n",
      "Epoch    0 loss: 1.2418 remaining: 2:30\n",
      "Epoch  500 loss: 1.0290 remaining: 2:48\n",
      "Epoch 1000 loss: 0.9744 remaining: 2:26\n",
      "Epoch 1500 loss: 0.8595 remaining: 2:07\n",
      "Epoch 2000 loss: 0.8843 remaining: 1:47\n",
      "Epoch 2500 loss: 0.9464 remaining: 1:31\n",
      "Epoch 3000 loss: 0.8728 remaining: 1:12\n",
      "Epoch 3500 loss: 0.7613 remaining: 0:54\n",
      "Epoch 4000 loss: 0.7326 remaining: 0:36\n",
      "Epoch 4500 loss: 0.9882 remaining: 0:18\n"
     ]
    }
   ],
   "source": [
    "def train(model, iterations=5000, batch_size=32, lr=0.05):\n",
    "\tstart = time()\n",
    "\tcriterion = torch.nn.CrossEntropyLoss()\n",
    "\tlosses = []\n",
    "\tfor epoch in range(iterations):\n",
    "\t\tix = torch.randint(len(X), (batch_size,))\n",
    "\t\txb, yb = X[ix], Y[ix]\n",
    "\t\toutput = forward_sequence(model, xb)\n",
    "\t\tloss = criterion(output, yb)\n",
    "\t\tfor p in model: p.grad = None\n",
    "\t\tloss.backward()\n",
    "\t\tfor p in model: p.data -= lr * p.grad\n",
    "\t\tlosses.append(loss.item())\n",
    "\t\tif epoch % 500 == 0:\n",
    "\t\t\tremaining = (time() - start) * (iterations - epoch) / (epoch + 1)\n",
    "\t\t\tprint(f'Epoch {epoch:4} loss: {loss.item():.4f} remaining: {remaining//60:.0f}:{remaining%60:02.0f}')\n",
    "\treturn losses\n",
    "\n",
    "print('Training with default learning rate')\n",
    "losses = train(model)\n",
    "print('Training with lower learning rate')\n",
    "losses.extend(train(model, lr=0.005))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAF4CAYAAAC4pwM6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAACylUlEQVR4nOzdd3gUVfcH8O/upveQnhBCgBAIhCS0kFDEl24DCyCiFAV/IigICKKA9QWxIBYURREbigjyKiCCSOgtEGogJBAIkEpI79md3x83u2w2W2Z3Z2vO53nmSbI7O3N3tmTOnHvPFXEcx4EQQgghhBBCiFHElm4AIYQQQgghhNgDCq4IIYQQQgghRAAUXBFCCCGEEEKIACi4IoQQQgghhBABUHBFCCGEEEIIIQKg4IoQQgghhBBCBEDBFSGEEEIIIYQIgIIrQgghhBBCCBEABVeEEEIIIYQQIgAKrgghhBBCCCFEABYNrpYvX44+ffrA09MTgYGBGDNmDDIyMnQ+btOmTejSpQtcXFwQGxuLHTt2NLuf4zgsXboUISEhcHV1xdChQ5GZmWmqp0EIIYQQQgghlg2u9u3bh5kzZ+Lo0aPYvXs3GhoaMHz4cFRVVWl8zOHDhzFhwgQ888wzSEtLw5gxYzBmzBicP39esc57772HTz75BGvWrMGxY8fg7u6OESNGoLa21hxPixBCCCGEENIKiTiO4yzdCLmioiIEBgZi3759GDRokNp1xo8fj6qqKmzbtk1xW79+/RAfH481a9aA4ziEhoZi3rx5mD9/PgCgrKwMQUFBWL9+PR5//HGzPBdCCCGEEEJI6+Jg6QYoKysrAwC0adNG4zpHjhzB3Llzm902YsQIbN26FQCQnZ2N/Px8DB06VHG/t7c3EhMTceTIEbXBVV1dHerq6hR/y2Qy3LlzB35+fhCJRMY8JUIIIYQQQogN4zgOFRUVCA0NhVisveOf1QRXMpkMc+bMQf/+/dG9e3eN6+Xn5yMoKKjZbUFBQcjPz1fcL79N0zqqli9fjjfffNOY5hNCCCGEEELs2I0bN9C2bVut61hNcDVz5kycP38eBw8eNPu+Fy1a1CwbVlZWhnbt2uHGjRvw8vIye3t0+e233/DMM8/AwcMDjd98A7i7m70NJ3r2RJ9Tp3SuVzZwoM51vA8cAAC4iMWolckUt1/r1w++jo6K+1UdSUiAi0SCDq6uOvexvbgYb2Vn45suXdDdw0Pn+uraB/B7PvrKqq5Gr5MnTbZ9U1E+LqUDBlCWlxBCCCF2qby8HOHh4fD09NS5rlUEV7NmzcK2bduwf/9+ndFgcHAwCgoKmt1WUFCA4OBgxf3y20JCQpqtEx8fr3abzs7OcHZ2bnG7l5eXVQZXkydPxvvvv49Lly4Bf/8NPPWU2dvg4eXFK6jjdfyatiMSiwGl4MrLywtejo4a95N0+TIA4FSvXkjQ8WZ/4tQpQCzG5Bs3kJmYqLtNatonb5MhOI7D1tu3keDhgfYqwWBmba1iH9b4ftNI5bhQcEUIIYQQITQ0NGDt2rUAgOnTp8PR0dHCLWL4nOtYtFogx3GYNWsWfv/9d/z777+IjIzU+ZikpCTs2bOn2W27d+9GUlISACAyMhLBwcHN1ikvL8exY8cU69g6iUSCJUuWsD82bQK0VFe0Jp/evIlRZ8+iRioVdLs9m7I+fFTqsW+pgLVefisqwiMXLiDy2DHBtkkIIYQQYo84jkNRURGKiopgRbX3eLFocDVz5kz8+OOP2LBhAzw9PZGfn4/8/HzU1NQo1pk0aRIWLVqk+Hv27NnYuXMnPvzwQ1y6dAlvvPEGUlNTMWvWLAAsopwzZw7eeecd/PHHHzh37hwmTZqE0NBQjBkzxtxP0WTGjx+P6OhooKKCBVhmZkiW4sWsLOy8cwdf5eWZoEXCulZTA+8DBzA3K0uQ7e1vKtZCCCGEEELsl0WDqy+++AJlZWUYPHgwQkJCFMvGjRsV6+Tk5CBP6WQ8OTkZGzZswFdffYW4uDj89ttv2Lp1a7MiGAsWLMALL7yAZ599Fn369EFlZSV27twJFxcXsz4/U5JIJHjrrbfYHxs3AnfuWLZBWlQ0NiJZaXyWpuyRNV2XWJ6TgyqZDB/dvCnI9rRddaHOdIQQQggh9sHi3QLVLVOmTFGsk5KSgvXr1zd73NixY5GRkYG6ujqcP38e9913X7P7RSIR3nrrLeTn56O2thb//PMPOnfubIZnZF5jx44FunQBamuB776zdHM0+uzWLRwpL1f8vTg7W+16ysUsANMFHanl5Vh14wZkNpRmvtPQgNKGBovsu6ShAcmnTmH1rVsW2T8hhBBCiK2waHBFjCMSifDjqlXsj23bgJwcs+37bz0yZdUqQRMA1Km5zVz6nDqFl65cwQ8FBdhTUoL9paUt1jEksCtuaMD9Z89iU2Fhi/uMCeNqpVL4HToE30OHLBIQLs/JwZHycszKzNS4ju2EqYQQQgghpkPBlY2bOGIEkJTEqux9/bXZ9jtbj7FI1joQ8WBZGYaeOYN7Tp9GgwDB3pLsbOy4cwfj0tMFaN1dt+rrFb83WOBYVglcgIQQQgghxF5RcGUHXnrrLUAsBg4cAM6ft3RzeLGGgOuOUje7RgHaU2ShbnuEEEIIIfZEJBLB29sb3t7eNjfVCwVXdmBm//7AyJHsjy+/BKwgcJH7vagIBVYUdGgK6ix5xKz9K8Pa20cIIYQQ++Lo6Ig5c+Zgzpw5VjPHFV8UXNkBEQBMmQI4O7PM1aFDFm7RXY9cuICvbaD0+qaiIuTV1Sn+trWrJPpokMnwyPnzWHnjhqWbQgghhBBiVyi4shcBAcBjj7Hf164FWtk4maM855HSFDRNuXQJ3U+c0Pn4T27eRIHSGChD7CkpQeTRo/inqSiIpjCO4zhcrKoSdDJjgE1o/Pvt25h35Yqg2yWEEEIIae0ouLID/vJ06eOPA15erGrgjh2WbZSZJaWlGb2NO42NOteZnZWFkWfPGrWfoWfO4FptLYY1bUdT6PTxzZuIOXECUy9dMmp/qpTnGZuTmak1MN1XWoqzVVWC7p8QQgghRJuGhgasXbsWa9euRYMVDS/hg4IrO+Dl4MB+8fAAJk1iv69fD9TUWKxN1kqIQhqnKyv1368B+3n7+nUAwGGlOcKELgTy8a1bGgPTnNpaDD59Ggd4ZgUJIYQQQoTAcRxyc3ORm5trFUXQ9EHBlZ34IiqK/fLQQ0BoKHDnDrBpk2UbZSHpVVWYn5WFIgO6742/cAH5dXVmLeJg7tFdfPeXXVvLe5u29bVHCCGEEGIaFFzZiefCwtgvjo7AM8+w33/5hQVZNsrQohLdT5zAhzdv4umMDL23+WtREZ7XMlmuoSj4MJ/dd+6gz8mTOGNAhpEQQgghxBgUXNmjwYOBLl1Yt8B33gF4jCWyhAOlpZinZTJiQ9PA8kelVlQY9PjremRshKAp4LPfeoWmNfzsWaRWVOCBc+cs3RRCCCGEtDIUXNkjsRhYsABwdQXS0oDPP7d0i1r4Oi8Pg06fxsqbN826X+WAzRazSaZqcw7PgPJydTUyqqtN1Ar9XampwYc3bqBKTXXMOzY2AJYQQgghto+CK3sVGQm8+ir7/fffge3bLdseFS9oyVjJmXquKW1b13fPtjTYUt1xjTh6FDU8yvdHHz+OLseP81rXHLocP475V67g1atXW9xnO68IIYQQQuwFBVf2bMAA4Omn2e+rVrEJhongShsa0OnYMfxWVKRxHVsIvm7rkekptZKupo1Nx3U/VTQkhBBC7Iqbmxvc3Nws3Qy9OVi6AcTEnnwSuHIF2LcPWLoUWLMGCAy0dKt4OVxWhlF+fkZt45ObN+HrcPdtzicbJoJ+mauv8vJwVU23ukaZDA5i3dcv/rx9m/e+zBWiXampweDTp820N+NZKnhtlMnwfUEB7vHxQUdXV4u0gRBCCLE3Tk5OePnlly3dDINQ5sreiUTAwoVAhw5ASQmwZAlQV2fpVvHysJGZtvz6eszOysIkpUl4lU/ChaomJ1NzYr8+Lw9O+/djG4/A6buCAsXvr169ijeyswVplzH+T02lRVtjjnDri9xcPJORgU7Hjplhb4QQQgixdhRc2ZFAR0f1d7i6Av/9L+DlBVy+DLz/PmAD3dRkJt7+FYGqAqrLhk3NyAAH4EE9A8TlOTl48/p1lGvodmfKV0152xV6jqmyhW6PpkDdEQkhhBCijIIrO/JXjx6a7wwOBt58E5BIgD17gI0bzdcwM8q1QFaOTxdCfUOPkMOHUWyicU2maK+lqWuvOUrZW1tQWdzQgD9u30ajzNSXJgghhBDTaWhowPr167F+/Xo02Fj1Xwqu7IjOFzM+Hpg1i/3+1VdsHJYVM+S0NezIEUH2rU+lQlOcxFdrODm2tpN5a9Yaj1S/U6cw+vx5vH/jhqWbQgghhBiM4zhcv34d169ft7lzHwquWpvRo4EHHmDdAt9+26oDLI7jMO7CBd7rP3XxoiD71bcEvDVN9ptWUQFRSgqeNWDMlDU9D1th6ukC9JVVUwMAWitXEkIIIcR0KLiyI534VCsTiYA5c4ChQwGp1OoDrE16nCT+qFQYwlhXmk5S+TDnCTYH7dmrnidPAgDW5uUZtG2+DtJYIwCUSSSEEEJIcxRc2REPBwcUJCfjXO/e2leUSIBXXrkbYL31llUGWKaYpraAZ7/dv+7c4b1Nc+YuZly+jM7Hj6PKiEl8hWjvuPR0AbYiHHUhTmsOfFrvMyeEEEIsi4IrOxPo5ITuHh7IS0rSvqI8wBo2DJDJrDbAsgW6gpVDZWWCnez+VFiIrJoabBAwS0cIIcS+vZSVhSVWMM0HIa0BBVd2KtjZWfdKEgmbA0s5wEpJMXnbrF1qRYVe6+sKrgakpRneGA1MkZmgbIf5lTc2IqWkRO1cadamuKEB87KycE6g+eEIIeaRU1uLVTdv4p3r19FAlUQJMTkKrlo7eYA1fDgLsN5+mwIsPVlbUQMh6Huqb/2hgeX9fecOEk+exIWqKsVtA9PScO+ZM/giN9eCLeNnxuXLWHnzJnqkplq6KYQQPdRRQEVslKOjIxw1zeFqxRws3QBiBSQSYMECVuzi779ZgLVnD5CYyJaAAEu30KrZS2h1q64O7ZydzRYschyHzJoadHR1hcQOA1RVI8+eBQA8ev48LiUmAgDONgVaPxYUYGZYmMXapmpzURE6u7oi1sNDcdtJPTO6hBBCiKGcnJzw6quvWroZBqHgijASCfDyy4BYDPz1F3DwIFsAoEMHoF8/Fmh168bWJQqWmJR39a1b2Hr7NjZ36wZXPV8PTe3tn5aG2WFhWBUVZXwDeXjj2jW8df06AIAbPNiobdlS5kzd5NBCF98wZmsHSkvxWNMUCMa+LoQQQkhrQ90CyV3yAOvLL4Gnn2aBlEgEXL0KbNgAzJ4NPPwwsHOnpVva6p2tqsJfd+5gWU6OoNv9+NYtQbenjTywMhVbCrisqa1nlbotEkIIIUQ/Fg2u9u/fjwcffBChoaEQiUTYunWr1vWnTJkCkUjUYunWrZtinTfeeKPF/V26dDHxM7EjIhHQuTPw1FPAZ58Bv/8OvPYaK9vu5QVUVAArVgBff83GaLViJQ0NeDI9HbtLSizWhneuX8cpE3TXsrUy5rbUXvvvAEkIsSa28+1IyF2NjY3YsGEDNmzYgEY1PT6smUWDq6qqKsTFxWH16tW81v/444+Rl5enWG7cuIE2bdpg7Nixzdbr1q1bs/UOyru3Ef15e7PA6rXXgC1bWNAFAD/9xMZm1dVZtn0WwnEcFmdn46fCQvxZXGzRtnypRzGEDQUFmJqRoXUda65c9+ft21hvwATJpmKtR8qWgk1CiPnQxR1iK2QyGTIzM5GZmQmZjV3Mt+iYq1GjRmHUqFG81/f29oa3t7fi761bt6KkpARTp05ttp6DgwOCg4N5b7eurg51SkFCeXk578e2KhIJ6y4YGgp8+CGrKlhQALzzDtCmjaVbZ1ZHysuRo0dgaQ0nux/duIG5V67oXK9CKrXaCogPnT8PABjk44MOrq4Wbo1whH53nKmqwn+vX8drEREAgHOVlfCQSBDJ45hZ5ytPCCGE2AabHnP1zTffYOjQoYhoOoGQy8zMRGhoKDp06ICJEyciR8e4lOXLlysCN29vb4SHh5uy2bZv5Ejg/fcBT0/g4kXg+eeBVjY5YYVUajUnoXzbwSewkjNlMCjEtosaGgRoiXEOlZVh6+3bir/PVVY2m0OG4ziLZgEXZ2fjSk0NCurr0SM1FR2OHbNYWwghhJDWwmaDq9zcXPz111+YNm1as9sTExOxfv167Ny5E1988QWys7MxcOBAVGgZl7Jo0SKUlZUplhs3bpi6+bYvPh5YvRoIC2PZqxdeAE6csHSrzEbfwMqSeauKxkac1nNclr4JeL7P751r1xBy+DCu1dS0uO9wWRmWZmejXqD0v6mP+YC0tGbHqUdqqqLKHgAMOXMGPU6cQKPK81H33jFVWyulUmSpOdaEkNbJ8n0oCLF/NluK/bvvvoOPjw/GjBnT7HblboY9evRAYmIiIiIi8Ouvv+KZZ55Ruy1nZ2c4Ozubsrn2KTycBVhLlwJnzwKvvAIsWQK0gvLNf9y+jVob6QMce+IEruvRhfGZS5eQVllpkrYsuXYNAPCamkxn/7Q0AICPgwPm6sgeq2a/rOWE4Q+l8Xd7S0sBABerq5vNF0UIIYQQ+2WTmSuO47Bu3To89dRTcHJy0rquj48POnfujKysLDO1znr8ExeHbm5u+KN7d9PtxNubdREcNoxVD3zvPaAVZP5W5+ZatEqgMk3jo+Rd0vQJrABgs1JXN0vIqK422ba/zM3FQ+fOoVYqNdk+VPU7dQppSplDc3cntZbuq4QQy6PvA0JMzyaDq3379iErK0tjJkpZZWUlrly5gpCQEDO0zLoM8fXF+b598aC/P8oHDDDdjpycgIULWVfBmhrgzTeB+nrT7c8GWSKzIuM41JgxiDAnPgU35Mc8tbwc4UeOYENBAZ67fBl/FhfjSzNWHKyWyTDszBmt6+gzDk3Gcfjv9evYY+bgnk7KCCGEEN0sGlxVVlbi9OnTOH36NAAgOzsbp0+fVhSgWLRoESZNmtTicd988w0SExPRXU1GZv78+di3bx+uXbuGw4cP4+GHH4ZEIsGECRNM+lysnaeDiXuASiSsXLuPD3DlCusuSBS+zc+3yH4LraDwgynoE4yMTU/Hzbo6TLx4UXFbuYnnzFBtX7GA+9tUVITF2dkYqiNgU6eAx0UPTUGUtXS9JLatzka6UxNCLMvJyQmvv/46Xn/9dZ291KyNRYOr1NRUJCQkICEhAQAwd+5cJCQkYOnSpQCAvLy8FpX+ysrKsHnzZo1Zq5s3b2LChAmIjo7GuHHj4Ofnh6NHjyIgIMC0T4YA/v7AokXs9z/+APbutWx7WpnzJhonZQvUnfjLgwR1BTJMGSjcaWhAlZ4ZQ33ac1XPAhXKwVLw4cPIa6Vz0xHL+72oCC7792P1rVuWbgohhJiMRQtaDB48WOsV6PXr17e4zdvbG9VaxmT88ssvQjSNGKpvX+CJJ4ANG9hcWJ07s4qCRK2hTVlbY5yurMTE9HSEqRRlsbZMg3KZcmO7mPHpFtjAcXhZj/LzQvE7dEjr/ULOIZZWUYEoV1d46JGZPlhWhrGBgYK1gRC+xqWnAwBmZWZiJv1fMBtr+19AiL2zyTFXxMo9/TTQvTtQVQW89RaNv9JiT1NFOWMcLS/HhsJCvK9SSMSc/1D57EvIeZY4jkMqj8m+P7hxA7lGvv84jsMft2/jk5s3mwWI1qDnyZPoffKkXo+hEy1CCCHWrrGxEZs2bcKmTZvQaOKu/EKj4IoITyJhJdm9vIDLl4Evv7R0i1otaypCcFOpO9otLQFPKY8v0eLGRvQ5dUrxtykDhi9yczH6/HnMzsrCJybqznTKiC6dGTSPlUa36urwfxkZrbrLLCGE2CKZTIb09HSkp6dDZmUXNnWh4IqYRmAgm/cKALZsAQ4csGx7WiEO1pulSNGSsfu1qAgLdXTnyzXBuKFDZWVYfetWi67Kb1+/rvj9KI9smbXRN8DWtL41Bep8PZ6ejq/y8hCfmmrpphBiFaz1fwIh9oSCK2I6SUnA+PHs9/feA5omkCXmUWZjaXRl75l4rjR1JxgD0tIwKzMTO+7cMem+LR2kKD93mR5VF23R6aaMlX1OSGB7LP3eb63ouBNiXhRcEdOaNg3o2hWorGRjsV5/HWga1ExM6+lLl2z+n+qmwkK8fe1ai2xSpgm7wplyEmNrkl9Xh6DDh/ESzwnW7TsMI+ZA7yHLoONOiHlRcEVMy8GBTSqcmAhwHLB/PzBzJvDCC6yroJ1OcmsNtt+5Y/GSx9VSKfqcPIlXr17V+7H3pKVhXHo6ll67hoNlZc3ua1G8w4YyMNYS8H508yZuNzRg1c2bzW4XspohIYQQ0tpYtBQ7sYwe7u44W1Vlvh0GBADvvgtkZwObNgH//AOcP8+WsDBg7Fhg5EhApZQ4MZ6pu9fp8kNBAVIrKpBaUaH3Y/crBVT59fVWV6nPXmkKVK/W1pq5JcTeUNhOCGkNKHPVClnsH1xkJLBgAfDzz8DEiYCnJ3DrFrBqFTBhAvDLLwBVPrMr6ibwNcSl6mq47N8vyLYA7Zku28mB8aOaibKlLB8hhBBiayi4aoXEarr99PTwMF8D/PzYWKyNG1n3wKAgoKSElWx//HHgxx/ZGC1iM0x9wv7W9euwprzVTwUFJt/HlZoaHFHpDikkjVUBrbRb4JnKSryQmYkimjePEELsnqOjIxYtWoRFixbB0dHR0s3RCwVXrZC60+DJwcEIcXIyb0NcXYFHHmHB1IIFrItgeTnwzTcsyFq3jv1NbNK1mhrM01FSnS9zZluu1tTg9exs3NZwEt8ok+HJixcN3j7f0KXTsWNITktDlpEFNlT3Z6ojeaGqCmdNeFEkPjUVn926hWcvX+b9GMrSWRfrDNvtH30OiC0SiURwcnKCk5OT1V7004SCq1bISSSCv6MjXMRifBsdjYmBgfi/0FDLNcjBARg1CvjuO+C114CICKCqCvjhBxZk/fwzFb6wAWWNjbgnLQ1fNBXRGH72LBoE+qeuaytCnjp8npuLt65fx+RLl9Te32jiE5Xcurpm3Skv2ED1wnqZDN1PnEBcaiqqTPxZ1RTA1dB3hNWjU3xCSGtAwVUrJAJwKykJJf37Y0pICH6MiYGzWKz1H9+1fv1M3zCJBBg6lGWs3ngD6NSJjcH66itg9mw2PotYpW3FxXg/Jwf7y8rwfGYmAP3KpWvKEsnp6hJoipO2HXfuYGNhYbNCGkJcAdZ2Be5CVRXCjhwx6aS3up6BIdcHlQOqUhPPr6au/YuvXoXbgQPYW1LS7HZbu9pJiKlRgEtsRWNjI7Zu3YqtW7ei0cbm7aTgqhXp5OoKABgXGAgnsRguEgnvx7qKzfhWEYuBe+5hQdXChYCbG3DhAhuntW0bK+lOrMrY9HRUGJE56Hz8uFH7z6ypwdysLEh5vjc4ABWNjTivoxvb4+npKDbjl/rGwkIAwEUt2Sp14YK+QV+1VIo/iovV3meLn67/5uQAAOZombOrgMZqkVaKLjIQWySTyXDmzBmcOXMGMhurFkzBVStytGdPbIuNxYthYXo/1iJfzSIRK9H+zTdAXBxQWwt8+CHrOnjnjiVaRLQw5qS8RIAA5qObN/GjHoUmOh8/jtjUVBwsLdVrP6YMPjR9IZ+sqMBLWVkobWhQe7/foUPYpcdn4pmMDI0BnLGZJ2sa36HcluDDh1FhY1c/7Q2d4hNCWgMKrloRP0dH3O/nBwcDslAW/acYHAysXAnMmAE4OgJHjgBTp7IJiYnV0CewMZXcujre6+Y3ZTL+pyGDYwnqKnkCQO+TJ7Hq5k28pKFASEljI0acPdvi9oL6erzY1E1T2S9NGTJVfxUX4xUDJnxW1+oqqRRTLl7Ettu39d6eNsaEbtdori7SClnTBQ9CWgMKrohtEIuBceNYufaOHVkVwddfByZPBhYvZrfv2AGcOweYsHw10UyI7JMtMPY0RduFCl0XMc7rOfn3lEuXcExlAmdt7X81O5v3tg+VleGVK1dQq6E76Ls5OfiuoAAPnj/Pe5vGOltVZdQcZlcNKH+/vbgYXY4dwwmqbEoIIQSAg6UbQKzHYwEB+OzWLbiIxahV6d9qNX22IyOBL74A1q9nkw7n5LBFlZcX4O7OuhaKRCw4k/8ukQDDhwPjx5u9+a3FTD3KZVtCjlKG62ZdHT43Y7GUGxqya/+7fRt1OvqV63sFOs+E44wGpKUBADwkEsxU09X4ph5ZRCGdqaxEvKcnAP2/tzoeOwYAyOjbF53d3Hg95oFz5wAA9507h6L+/fXaHyGEEPtDwRVReK9DB/T19MTINm1wvKICtTIZHrtwwdLNasnREZg+HXj0UeDKFeDmTeDGjbtLQQHLbGm7krxmDQuyHnvMfO1uRT7PzbXIfvmeSq/Pz1f8/kthocZucuro0/VQH2PMlOERuotQRk0NXhZoPjM+dLW/RoDqjueqqngHV3KVVAqeEEIIKLgiSlwlEjwVHAwAuN/PDzlK4xOssv9omzZs6dOn+e21tUBuLvvJcYD8ZEsmY3+fPMkmLv78cyAwEBg0yPxtJyaxW6UUt9AaOA5TMzJMug9b9I1SsCpnqVy3EKEjjVEhhBCG4zhMvHgR7V1csKxDB0s3xyZQcEV4UT5RchSJBJsc1iRcXABtXwBxcSyr9ccfwH//C/j5Ad26ma99xGT+1bPyn77+KC6Gg7V0kTXQbC3lyk0VVBwrL8fCK1fwUadOSGjqsmfNDDkKtv2uMA86RpZhxf+tiQ04UVGBn5t6d5gzuHJ0dMT8+fMVv9sSq0xIEOug/IWsPHZhZJs2WNSunfkbJBSRCHjxRSApCaivZ6XdaYJiwlOjNV9Y4OGOiQuPFDY04NmMDKQqFdLod+oU9pWV4d7Tp5utW1Bfj0YTzl+iOubKtl85QoxHnwGiL11jgU1FJBLB3d0d7u7u1jPunycKrohGylexVd/WtvU2V0MiAZYsATp3ZtUFFy6kKoPE6nEwbXEZIbb9bEYG1ubl4ZyayoZlSuOSTldUIPjwYSSnpSGnthaHTPD5M2f3Ppv/TjQDOrG3DHpvEmJeFFwRjZplrlTus4sva1dXYPlyICiIZa5eew2wUIUzQiyN4zicrqzkte66vDyN96VrmJxY1XdN86KdqKhAxNGjGJCWhjSVsvFq28lr68bRtI93rl3DGspyE2JRHMfh2YwMfHjjhqWbQkyosbER27dvx/bt29FoY1O9UHBFeLGLYEqdNm2AFSsADw/gwgVg2bK7BTAIaUXW6FHh8RkTFfXoefIk0vWcy0uVqYKvzOpqLLl2DTPUTMpM+LHb/yN24FJVFXbduWPpZvBysKwMa/PyMN+MVUpbM0tlnGUyGVJTU5GamgqZjZ2XUUELopGmMVfq/rZpERHA228DCxYA+/cDM2cCAQEss+XqCri53f09Ohro3p2N2yLECMrVOK3B+wZeBTbkk1BYX6/xRO6ZjAwc6dnToLaoMvR7St3JRLmOUuv0jUCsFZ+T464nTgAATvXqZfVFZ2jaA2LtKLgiGmnqFmjr1dLUio9nwdV//wtcusQWTaKigLFjgcGD2ZxbhBhgcXa23o+xlzErnY8dazb+SpmQg6eppDox1s3aWky5dAmz27bFg/7+lm6OyZ2tqrL64IqYlx2e8ZkcBVeEF7sPrgBg6FAgMhK4ehWorgZqau7+rKlh5duPHwcyM1n3wa++AsaMAR58EPDysnTriY2xpqqDQlbsq+GxLU2BlamZMtiyq2y+iYhEIjbXoA35v8uXsae0FHtKS8ENHmzp5hA7VieTYV9pKQZ6e8NVIrF0c4gRLDrmav/+/XjwwQcRGhoKkUiErVu3al0/JSUFIpGoxZKvMoHl6tWr0b59e7i4uCAxMRHHjx834bOwX5qqBTqKRPZ7JaNjR2DYMGD0aODxx4Gnn2bdBOfPB956C9i4EXjmGTY31u3bwNdfA+PGAR99BBiQiSBEH2mVldjQVAhCSP3T0gTfpqHSKiuNCoJKGhqMboOu/e818WTV9soWM4mFAryfbIktvkb24oXMTIw4exZTtPWcsQB6R+jPosFVVVUV4uLisHr1ar0el5GRgby8PMUSGBiouG/jxo2YO3cuXn/9dZw6dQpxcXEYMWIECpsmQCP8qY65SvDwAABMDQ623+BKF29v4MkngZ9/BhYtYl0E6+rYhMRPP80Crw0bAC3V1AgBDO9qcdbIgg/qHK+oQLYVjQHbr6Usu65/9A+eP2/0/nXtY6ea8WKt9jvRznAchzOVlRab24e0Xmubzht+LSqycEuIsSzaLXDUqFEYNWqU3o8LDAyEj4+P2vtWrlyJ6dOnY+rUqQCANWvWYPv27Vi3bh1eeeUVY5rb6qiOuTqckICcujp0dnPDgdY+J5SjIzB8OMtynTkDbNkCHDnCuhRevQqsXQvExAD/+Q8bm+XnZ+kWEytDJ+Oa5ZpoSgQOwMWqKrhJJIhwcTHJPohmttAtcG1eHv7v8mUM9fXF7rg4SzdHcLoyU+Z6daqlUuwoLsbwNm3g5UAjVIh9sclS7PHx8QgJCcGwYcNw6NAhxe319fU4efIkhg4dqrhNLBZj6NChOHLkiMbt1dXVoby8vNlCmhMBcJFI0NnNTfE3AasaGB/Pugxu3sy6D/bsCYjFQHo68NlnrNvgq68C169burWEWFwpj25WHIDDZWV46uJF5AsYaF2vrUXMiRNof/Qor/UvVFXhiJoLSdYdHhBjfNo0j9k/1PXTpGZmZmJsejoevXCB1/r/u30bqSY8N0uvqsLwM2dwuLVfOLYijo6OmD17NmbPng1HGyseZlPBVUhICNasWYPNmzdj8+bNCA8Px+DBg3Hq1CkAwO3btyGVShEUFNTscUFBQS3GZSlbvnw5vL29FUt4eLhJn4et0DqJsMrg7aG+viZvj9Xz8gLuvx/48ENg0ybghReAbt3YvFlHjrAug6tXAzwnaiX2jW8BBGu/kFGm5+SOt+rrea3XPy0NPxYU4P8uX252uzGBzcM8T+Tkup84geS0NJNl0uzF9dpadD1+HF/qMVeataIxR+axvumcjE8Qe76yEmPOn0efpnM9Uxh19ix2l5RY1fjT1k4kEsHHxwc+Pj42VzDIpoKr6Oho/N///R969eqF5ORkrFu3DsnJyfjoo4+M2u6iRYtQVlamWG7QrN8AVApaaHljb4uNxZZu3czRJNvRpg3wyCMsc/X990D//oBUCvz2G/DUU8C2bexv0ir9kJ9v9UETX6auephVU6P1fpkJ9q+6xes8xqPZy+tpiLlZWbhUXY3nVAJhVa35GBHDZer4DhBCjp4XUKQch1r6H040sKngSp2+ffsiKysLAODv7w+JRIIClWpaBQUFCA4O1rgNZ2dneHl5NVuIjsyV0u/3+/nBU0uf6bykJCGbZXvCw4F33gHeew9o1w4oLWXZrRkzgHPnLN06YgGTLl1CbSsdMD9IzyvDMgAT09M13v+BmothUiMDLtVHCxm+cRxnd9kRTe/lXwsLMevyZcXrYYvPmgJCy7PG903P1FS4HTiACj0z94Q/qVSKXbt2YdeuXZDaWCBr86MIT58+jZCQEACAk5MTevXqhT179mDMmDEAAJlMhj179mDWrFkWbKVtCnd2Vvyu+g+mndJ9cpu7dVPbfzpYzbqtUp8+wDffAFu3AuvXs/myXnwRSEgAgoNZ0Ys2bdhP+eLs3HK+Lfnv/v5A795s3BexOZt5VoSyxhMLY9zR82TkUnU1LlVXK/5WDUw+unmzxWNc9+/HuuhowxqoxguZmVjQrp3Wdfh8CjmOw+DTpyECsDc+3ua6uuhrfFNQ3M/LC09qucCpas2tW3jn+nXsjotDV3d3UzWPEIPJq7YeLCvDKCpYZRJSqVRRL2Hw4MGQ2NDcXxYNriorKxVZJwDIzs7G6dOn0aZNG7Rr1w6LFi3CrVu38P333wMAVq1ahcjISHTr1g21tbX4+uuv8e+//2LXrl2KbcydOxeTJ09G79690bdvX6xatQpVVVWK6oGEPw8HB9xMSmLzWqmcBDwVHIz06moMVqra+EhAAD6LisKWoiK8ExmJpy5exFuRkWZutZVzcAAeewwYMoQFWjt2AMb08e7fH5g3D6AxbzbHlvJWV8zQLUeZsQFlA8fhqUuX4C4WpnPGqcpKPK6UPTO0fYUNDYoy88UNDfB3chKgdZan63gU6DlX1IzMTADA9IwMHOzZ08BWCe+da9dwpqoKG2NiILahwFif96u9XcwhxqP3hP4sGlylpqbi3nvvVfw9d+5cAMDkyZOxfv165OXlIScnR3F/fX095s2bh1u3bsHNzQ09evTAP//802wb48ePR1FREZYuXYr8/HzEx8dj586dLYpcEH7CNGSdJCIR3uvYscXtM8PCMDMsDACQ1a+fSdtm03x9WWXBRx9lVQXv3AGKi+/+lP/e0AC4uACuroCbG1tcXVlGKy0NOHSIPX7ePBZoEWICnzdVUFPHdk4x+dPVba+4oQHdjx/Hk3r+X+E7jtXeyJ+3vs/YVB2BzlVWwlUsRqem6rfKtL3yS65dAwDsLinBiDZtTNM4DfLr6nC+qgpDfH1b1XvH2lHgQdSxaHA1ePBgrf/E1q9f3+zvBQsWYMGCBTq3O2vWLOoGaOW6uLk16+rTakVGskUdjmOVBjWlwrOygGXLgOxsYPFiVqnw+edZAEaIAFLLy9FbxxjU7Wom1LU1JQ0NcNejy8m3TZXOFmVnm6pJNsXaT/V3Fhfj6YwMfNulC/p4eqJHaioAgBs82KDtWWK8ZPujR1HHcdjcrRseCQgw+/5J66X8+S6or4e7WAwPmptMK5svaEFs07sdOjT7u3LgQAu1xIqJRJoDKwDo1AlYs4bNoyUSAdu3A9OmUZEMO5PDo1KdqfQ5dQq779zBSjXjmqyFsSf2BfX1aHPoEDoeO2ZcO1pxNsFUV++FOqKjzp1DXn09Rp49ixs6qsLxeS7yi8K1UqnRxVP4qmvaz99GXszQ1VoObH63LTzHhJLWJfjwYbRRml+WqEfBFbEK+lw1JkqcnFjVwZUrgaAgIC8PmDOH/b15M7B7N3D8OHDpEpCby+bYksmAxkagro4VxqisBMrKWDdEM4+tIbq9qDQu1RKGnz1r0f1rkl9Xh4VXriCP57xZql6+cgUcx+Hfpnl2biqddJvjdNnYwEHKcciy8+z/kfJyxJ04YdELDJpUS6XwOngQ3U+cMHpbV2tqUKznuDR96Pte637iBB69cAH7SktN0Rxi4xp4XlCokUrt/jtKE8rrEYuwt1LEFhcfD3z9NZtX6++/gT//NGw7EgnrXvjkkwB1PSEWou37QX7P+PR0RXEIQ3xw4wYe8fcXrDCBufNWE9PTsbGoCF9HR+OZpoq51sqYb/uzVVWYnZWF37t357X+pzdv4lh5OT6JioKU4xBgoqIhJyoq0MBxRnVvTykpQX59PSZcvAjA8G6Kuhh6/E9XVuIepaJVpHUy9P3TIzUVWTU1OJSQgGRvb0HbZO0ouCJmFe7sDBexGCO1DAZO9PTEsYoKM7bKTnh4AK+8AtxzD3D0KMtGlZc3X3RNlCiVAn/8AezcCYweDTzxBED/XIkVOsgjsNI1MqZcKlXbfcMWLv1sbOq29W5OjkmCKxnH4avcXCR5eyPOw0Pw7eujWo85buSZ3p8KCwEAFQMG6LUv5dd+UFoaThj5v2h2ZiZSSktxtGdPuCr10ChrbMS9Z84YtW1iOg0yGRwFqjbaWskngN9YWGhQcOXo6IgZM2YofrclFFwRs/o5JgZJXl5arxa35rELgkhKYos68q6AEgkgFrf8efYsKxF/7hywaROwbRsrHT9uHAveCLESfL4langUHhDq24bXPFcC7cscNhQUKEqiC5VRMfRYG3PcsvXsUqiciTpgRGZU7pOmSpu/FhVhstJcX6U0+Sxv5v7cXKyqQsyJE5gdFoZVUVFa17Wlz7StEYlECAwMtHQzDEJhOTErEWB0N5y327fHh2rKwBMenJ1ZJsrTE3B3Z2XenZxYcCUSAXFxwMcfAytWAJ07s0Dshx9YBuvnn1lmixALkp/MCHER5vH0dKu7mCPTs8s0x3Gol8mwv7QUDUZUseM4DjMvX8ZnTcVL0iorDd6WvTPkhFqfMVUcx1ms67ylu+xzHIeJ6emY3RTYW8Jb168DAD5uCoylHIeHz5/H202l+K1Zdk0NHjt/HsfKywXbpnV9Q9oGCq6IWRn7IT2ckIDF7dsL0RSiiUgE9O3LKhG++SYQEQFUVABffcVuI8ROlDY2qv0nqG+AI5S/79yB98GD2NjUpY2vZzMycM/p03oXP5FxHGZcvoy1ubnYX1aGz3Nz8ULTNkxxQqXtqKZXVWGrCSrUadunMYGEvo+dd+UKr/VkHId+p05h6JkzFg90LHFSfbG6GhsKCxUZP2uwo7gYW2/fxlILB1cyjsNNHZnYcenp2Hz7NvqdOiXYfi31LpRKpUhJSUFKSgqkNnZhl4IrYjHfRkcDANZ36dLsdm3/UOgKihmJRMCgQayb4OzZ7LbffgP27rVsu4jdu8CjSIBQ3wXqMgqWqhY48uxZVEqleDw9Xa9tfVdQAABYk5ur1+N2FBdjTW4unr18GeUCdVP7+OZNlPLM0uTX1WFCejpSSkrQ7cQJPHzhAva38gp12bW1OF5RgX9LS1Enk+FGbW2zaomGvO+NfT+bq9w8oLsS3eGyMlyoqjJTa5hqLRnhM2bM8I67cAHhR49qvQiRZUcVf6VSKfbt24d9+/ZRcEWINh1dXRW/TwkJQc3Agc36oetibV14WgWJBBgzBhg/nv393ntAU7cJQkzh3ZwcjfcJfZr3lpr3sgzAC3p2S7LFb6YSpYBKqOM6JysLUzMy1N6neoyez8zEL4WFzQo7CN0dkW/ZaGtUz3Fod/QoIo4eVdxmyLMp0GO6AtXtX6qqgveBA3jDSibM7p+WhpevXrV0MxRey85GhZnGz22+fRsA8P6NGxrXscbvIWtsk6lRcEXM4nq/frjQpw8CVcriuqiZ3yrKzU3jdsKdnQG0zg+rxU2fzkq+19YCS5cCrXT+CmIdhPoOqFBzRZTjOHwmYLekzOpqzMnMbDaXljVQPoaqJ9XGXMjaVlzc7G9NAYG+xSbk6mUy1PK8kt375EmN9xkTdhl7oU9TDw3lrQpV9OJZpWBX3+e88OpVVMlkeJMuqGlkyjnK9GWN50a2e3nDcBRcEbNo5+KCGHd3retc7NMHBxMS0K4pgFK1Lz4eoU33CTU3DdGDRAIsWQL4+wM5OSyDZcNXhYltM2WVP0Pe1dpOtpPT0vDxrVt49MIFA7as3RU1AcqczEzMunxZ8H2pOlpWhlevXtWrVLouX6gJapVfD47jEHr4MHwOHkS9EQU8jGGqb73oY8dwR8eJuvxdVieTIaWkhNcxUPce0UT5uVnLf1lLjz3TJfLYMZwVOOPK9+KBKmt5zVo7Cq6I1eji7o7+GuZC2B4bi0FK8y1NCgpCFy0ZLmIibdoAr7/OAq19+9gYLEJsmLqTEaFP2W83nTDfEChzVaflhLq8sREf37qF1bm5KNSjO5g2Uo7Dd/n5yFLJVielpWF5Tg728SxZrnqs1R37DB1jRmplMhQ3NqKO44w+njKOw8IrV/CbngVEhKAuGL9cU4MPm6o1AtpPlKdnZODeM2fwogmr6lk6pLGGoCrp1Cl8qvSaaPLw+fOC7vcNG6hMSDSj4IpYHXVfp6qTDvs6OuJi377maRBprnt3oGliP6xZw+bGIsTMhBp/qTa4soKTOm2evnRJ433KxQd0FSJQPoarVE4glY/LN3l5mHLpEqKOH9evoSpadD3k+ThN6xly8r1QqWrfH8XFeO/GDYzVs4CIoft+nce4JeXXTNsefmgqYvJlXp7e7dCGMh/NHS0vxyEeZc2FHtv31507VhFcEsNQcEWsjrqvE/rCtzKPPAL85z+ATAa89RZw507z+0tKgFOngC1bWAn3P/9kExMLMCknad2EPuEoVjOuxRydzQz5TvsqNxfTLl3CBp6ZFn2OVIqWKn3GTqZr7Csm5Cv+d0mJ4vd8gTJ7mtxQ6Y4nL55yvLwcM8zQbdMQqt0CLfm/tzWHFvn19Qg7cgSL9CzeYchFp9y6OpwzYdXD1nj+5mDpBhAipAf8/CAC8KfKgGoiMJEImD8fuHKFVQ587TWgSxfg2jUgO1t7EOXry+bOiogAIiOBAQMAPz+zNZ3YB1P+wzYkgDPHCcT/8Tgh19SOW3V1CHVyEiTjV9rQgDMGlMO2tpMsU5+833/uXIvbPsjJsapqd6Qla8gYFTZ1JX43JwfLO3Tg/ThDPmNhR44AAK4mJiJSqaKznLmPR0Z1NdKrqjDazw/Tpk0DADg42Fa4YlutJa2Cug+yphOCddHReFpD2V9iYq6uLGs1YwZw6RJb5EQiICQEaN8eCAgA8vJYEFZQwLJaJSXA6dNs3U8+AZKSgPvvZ5MXq6kgSYgqU56oC5m5Us1eaNKoMo6qWiqFmwGfhS1N5ZoBNsYr1NlZcUK/NCICb0ZGKu7XdAwbZLJm96lOaux76JBebVqTm4v+Xl6oUnmOpwy4Wm4NARoHfhmCc2oCUF2BlbYKjsQ8NppgMmtz0ffzcUtp3OLpykq1wZWx9H0fd2nqfry7Rw8MDQsTvD3mQMEVsTr6fBCnhoTgh4IC7G3lE09aTLt2LMD644+7wVRkJMtKubi0XL+mhlUavHaN/TxzBrhwATh0iC3+/sCoUcB99wF6zH9GWo+Cpiu6pjzJNmRCXU3t0TQB6YXq6mYFfNaqjJ1JKS3FfQZkdJ9RutjU79QpcIMHK07o37p+vVlwpcn9584hwcND8bfyeBLVohbqNHIc5mVlKf6+VluLgfKLKUSrZsGVBTIofPd4paYGYoDXyXhxQwM8JRI4ifUbicKp/LQG+XV12K7aDd7MGnm+Lx4+fx5fdu6smALnSk0NMqqrm32vnNeQgf46Nxdhzs4Y5ednsflFT1ZWYqjKeHtbQcEVsTt8vna+jo7GNMp4CaNXL7bw4eoKREezRe7aNWD7dmDXLuD2beCHH4Aff7y73S5dgM6dAWOrQ0qlwK1bQFAQoKHcP7ENFY2NLbIgQvoiN1fvx+jbmrSKimbB1SWVoGV3SQnu8/PD33fuILumRuv8f4bSdMq0u6QEPZWCK2V8i1qs5FFhzZZd0VHVUAj6nNTm1dWBAxTTlZjCpaoqvH7tGuaFhyPx1CkAQMOgQXDQEjTl1NYi4uhRRLm64nJiosnaZi6DTp9Gphlee22OV1RovK9QqZT/1tu34SIW4+eYGABAp2PHAAB74uLwH19fjdu4UFWF6U1dkLnBgwVosWFkjY041JQl79evHyQ21KuFgitidYy9StVRXcZExTMhIYIEVynx8Ris5YrssyEh+Ergak52p317YOZMNknxwYMs0Dp1CkhNZQvAuhlGRLBAS75oyo4pq6oCTpwADh8Gjh0DyssBLy+WHXvoISA01ORPjwivQ9NJgqnkGlDo4LaG+YkMzT7IT6tHNlXjpOpTLRldKMOIzNAzZrg4p6vao1yDTIbQpnEztYMGwVlLsKNri9qOSdcTJwAAvyp1m6vjOK0nkvLxz4YEJBzHASIRryx1WkUFTlVW4ung4BZB6fLr17GrpAR/xcbCxcgTdEsHVtqUqvkOuqVmuoJj5eVag6tclcfw/Zzcrq9HYUNDizlNDc57yWT4559/AAB9+vSh4IoQc1L94L4VGYmP1UxEKTQfBwckenpqXUffbhCtmpMTq0D4n/8AublsHq2LF4GMDKCwkGW4rl0Ddu68+5jAQKBtWyA8nP1s147NxXX2LHDkCBvXpdzFSyxmAdbGjcCvv7IxXmPGsJ/0WtkMTYGMLdFVllz1b0PzdMqlx22VNRQYsIT+aWm81qtUmnD2TkMDQrRkr5JPncJ/fH3xUadOOrcrgnBTHgDATwUFuFFbi1ciIvR6nLZXn+M4iEQi9Dx5EgAQ4OiIh/z9m63zalMJ/O8KCvB/ShfUMqqr4SIWI4LHBVlbwPc7QohPU1ljI443BWmSpvdIwOHDAIBLffsiWo9Me0lDA9IqKzHYxwdiC3VBFBoFV8Tq6PvBlyh9GF8OD4eXgwOO9+yJPSUlWKQ0r8ij/v7YrDTYWwi6/vHYx9eEBYSGAhMm3P27uPhu0YxLl4DLl1mQVFjIlqYuKmqFh7OCGcnJQNeuLBu2dSvLaB07xpaQEJbJ6taNVTP08QHc3VnGjBAjaPqOuFFXh223b+N+DWMahAon3rtxQ3PbtDzOUuMsbIEpgz3l426KUvFnq6pwtqoKH3XqBCnH4XBZGXopXSRU3r9qV1VjPXnxIgBgFM+xhE9duoRbdXWYqUdRg3NVVS2CK7ktRUWK4KqkoUFROEGIrm+t7dMyMC0N56qq8H6HDpjfrl2z+w6WlakNrlbfuoXzVVX4PCqq2fus58mTuFZbi6+jo/FMSIjJ224OFFwRm/dpVBTuPX0az4aEYJCPDwCgj5cX+nh5NQuuhNZar6ZahJ8f0L8/W+TKyoAbN9hy8yb7mZMDFBUBnTqxYCopiWWzlCUns+XWLeB//wP++otVM/zyy+brOTiwIMvbmwVcDzwA3HOPyZ8qaR3ev3ED79+4gd+6dcOjAQGWbo5V21Naijeys/FGZKSgxQ3s8Rtcn+f0fk4OFmVnY0jT/02g+f+11bm5GKMhUDFGsZbMs3KQ8ktTlco+WnqIcADA83/xLqU5zm4odX0z5f/yA6WliHBxQTszZMf4Bnibioqwt7QUP3btavC+5JUwfyosbBFcqZIf3VmZmQCAxwICMESpW+K1poqqmwoL0V7pONny55OCK2J1lL/oipKTdXati3Zzw62kJLNfaeXQ+q5WWRVvb7Z0727Y48PCgOefB55+Gvj3X2D3bpYFKy0FqqtZd8Lbt9kCsIzXzJnAY48J9hSI/dN14ra3pASPBgRQpkiHN69fxxs8Kh3aO3XvE23vnMlN2SJ1Pm8q3LLHzNV2v9JSMEbdp6VWS/GaIWfOoMGA4jbKZxVCl8bZfecOzldVYYC3NwY1jck2JjvWKJNpLRqir9NNUyC8fOUKJgYFaV33qqFj5bQo01CN9e+SkmaTfNsyCq6I1VH+WPo3lRDVhc+JCZ+rIL4ODigxoAyzJnS6ZANcXFjp9/vuu3tbfT0LsuTL4cMsy7V6NeuOOHUqdRkkLXQ4ehTroqMxWMtgcVWavrs+unkTkSa82l0rlWr93rSFd7ctX9kWUqqG6nG36+vxfUGBxsfxKem9VeCu9ID+80jVa2lnioGBofLYHpkAmSvlLQxvKkLzoAFTKajzQ0EBpmroLlfW2IhtxcV4yIB9acsgyr2YmYn54eF6b7u1oxHcxOpECTiJ3YcdO+q1/gY90uT0j92OOTmxYhmdO7NiF7NnA888w+774Qfg008BE5YCJ7Ypu7YW9545o/hbxnH4Jj/f4O29qDRXlJDmZ2XB9cABnCgv17iOtQZXyt+7GUaOCRLyO7xOJsNTFy/iRyNeb8Cw4z6s6WRelVTtrXepC66s8f+aEMGPKlNmruTklRIBILO6GhVNF27z6uqw684d3t0RizQEQWcrK3FPWhqevHgRky9dUruO1mIg0P1+u8JzEvR6pf+HygVWwGMfmljje5EvCq6I1ZkWEoIlERFIiY83eluTlFLefD/gr+joPyxnyx98oieRCHjySRZkAcDvvwPvvsvmziJEg18KC/GBloISwN1/wvLxJebwYdMcVB9acC6qTwXY90Pnz6PaiM+g6mvTyPOCibrv/q/z8vBjQQGe0nCSa434TkZrj5SztvMEuIih6/yi8/HjiDx6FCklJQg9cgQjzp7F/3hmBS9WV2Nm07xTcpnV1YhLTcWZprFPv2vYljyAq9LwOVF+BzSoeT/wLWpSo/TZKRPq/6JEgsmTJ2Py5MlwcLCtjnYUXBGr4yAW463ISNyjNMjWlJK8vBS/i0QiLIuMxLV+/XA4IaHFugOVJv0EKMBqdcaMAV59lZVt370bWLqUdSE0RFkZwPOqoE6NjcD58yyrtn077wHexDQaZTKcqazU2F1LmfykzBSV4Yyx1sTz8xmSlVN3pd+Ybtw3VObz0TY5qy5CTQ9gzoyhuoyQvt8c6p73rbo6/K1HZkaZsc9/W3ExPucxFYvyye9qAyYNN0RxY2OzzPZunuOL1ufnK8bHyal7r2o72nfUvE6q649PT8d3BmZelcdRvXHtmtasMt93hUgsRvv27dG+fXuIbWyqFIu2dv/+/XjwwQcRGhoKkUiErVu3al1/y5YtGDZsGAICAuDl5YWkpCT8/fffzdZ54403IBKJmi1dunQx4bMg1ozPh7jF/DIiESJcXNBdZSI8APgsKurutjkOTiIRvLVMbGetXWuIEYYNA95+G3B0ZGOxXnmFTVbMh1TKJkpeuBB4+GHg2WdZkKUvjgOys4HffmPB3kMPAS+8AKxbB3zwAfDzz/pvkwhmWkYG4lNT8RGP7IxIJDJo0LipaeqKZEkcTNNFTHn7fIxLT2/2d5+TJxVFAszVBj6PN8f/n8ijR1vc1vbIEYw8exbblLrF8aXu+etzTI6Wl2NmZiZOawiU/75zBxerquxmPiU+OI5TlMHXZYqBmdcIlffBdKVJtlvjpT6LBldVVVWIi4vD6tWrea2/f/9+DBs2DDt27MDJkydx77334sEHH0SaykR73bp1Q15enmI5ePCgKZpPbIymD3izzJXS7Z4ODihKTlb8/VhAAHp4eDTbnkgkQoFyeXDSOiQnA++9B7i6AmlpwKOPAvPnAz/+yDJIqlfT8/NZ4DN+PLBkCXD8OAuQbtxg2S++J7INDayoxmOPsSqHq1ezyZJragAvL0CebV27lmWwiEV8p6WIgCoRWDBGdCttbER7lZM4oU6RixsaMNXAE8vUigrBCj9o6r6lzJDqeOqo+5+o7/HUduL8r5mrECrL05AJHnn2LGJOnLD5blv6vE4nKyqwX8NFPHXbESIYOmDIRUMVMpkMx48fx/HjxyG1sS74Fu3EOGrUKIwaNYr3+qtWrWr297Jly/C///0Pf/75JxKUunA5ODggODhYqGYSO6H6JXKpb1/8cfs2ZoaFaRx7oFytUP5l/EVUFGZkZmJTt24AAGct6Wq+5ZUPxMdjYFPJVmIj4uOBjz4CXn8dKCgATp5kC8AqEMbGsjLx6el3gymAzZ01ciQQF8cyYGfPAitXAgsWaK9AWF8PvPkmy5YBgLMz0KMH0LMn0KsX0LEj6664di2wYQPbpqcnMGiQKY8CMdLhsrJm4xWIZuvz81EsYDVXZXOzspBpBRnEP3Rke06Ul8Np/36N9xsbbBp7Yq2cWaS5ILUz5ujo8zrXWfh1UG0r7/eFVIq//voLABAfHw+Jll5C1sa2RoipkMlkqKioQJs2bZrdnpmZidDQULi4uCApKQnLly9HOy1FCurq6lCn1Pe6XEsFJWI/ot3c8LLK+8KLx4f3ubAwPBMSAkcb6wNMTCA6mgUy168Dp0+zLNaZM6xc+4kTbJHr2ZNNRDxgAOtSCLDAbNEiYOdOIDwceOIJ9fupq2MZruPHWSXDhQvZdtRNVTBtGutquH078M47wIoVdzNaxOocM2KcjyGMKQBhaabsEphlBYEVHyd1dD80d7dAVd8bWS1RHUNedXvv9Cfl+VlQHVeoTFOQo8+xowBaPZsOrj744ANUVlZi3LhxitsSExOxfv16REdHIy8vD2+++SYGDhyI8+fPw1PDLN/Lly/Hm2++aa5mEwvR9hXwWVQUrtTUIFGpi6A2QgdWNIGoDROLgchItjz8MCvRfu0aC7TS04GgIDaHVtu2LR/bty8waxbwyScs49S2bctMU00NsHgxcOoUy1b9978sU6WJSAS89BIL8A4cYI/96CNWVp60eu4HDli6CQYzZVhI38DGkXIcJCIRjipdnBYqKDLktbkmVLEgE1J+XvpeONh55w6v9XLq6ngHYppQ+KQ/m730vmHDBrz55pv49ddfERgYqLh91KhRGDt2LHr06IERI0Zgx44dKC0txa+//qpxW4sWLUJZWZliuaGjdC6xPzPDwrCyUyfBJ9U09z/s0X5+gk1cSAwkFgMdOrBxWEuWsKIV6gIruYcfZgsALFsGKI+/qa5mBTNOnWLju1as0B5YyUkkLKhKSGDbWLiQje8ixIapOwGloEg9Q8eQzczM1Psxz1++jDYHDyJPS5aEL2MLWsjxKSZjLWQcB8m+fXo9Rp+ufqYMrgy5MMy3NXstOGbPWDYZXP3yyy+YNm0afv31VwwdOlTruj4+PujcuTOytJR9dXZ2hpeXV7OFEFu0NTYWk2m8oe2ZORNITGTd/157DSgqAior2Tiss2cBd3dWQCMujv82nZzYmK6oKKC0FHj5ZbZdQmyUKa+gH7Kz4QAvZWVhu5bsRs/UVJQL1EX0i9xclEul+ERNCXQhuo2ZoqT7Pis4cb/elF27YkCXVHN2x7PUsRKiKIal2Fxw9fPPP2Pq1Kn4+eefcf/99+tcv7KyEleuXEFISIgZWkdIc6pf8E5Gdv/7oGNHox5PrJREwrJckZFAcTErr/7yy8CFC4CHByuv3r27/tt1d2fZrvBwVnRj3jxAZTJKQmyFKcdc2RtdY8jSBCodr4zjuGZdN0sNKD6itlugCbrNTxW4QmdOXR0ijhzBZZ6T7gLQGvzqos8n4aHz5/XaxsKrV5v9XaklCD9dWYlpBmRI7f2zbNHgqrKyEqdPn8bppipp2dnZOH36NHJycgCw7nqTJk1SrL9hwwZMmjQJH374IRITE5Gfn4/8/HyUKUW38+fPx759+3Dt2jUcPnwYDz/8MCQSCSZMmGDW50aIqtlhYThkZGEB6gJjx9zdWbdAX18gKwu4dImVV1+5EjBmrj5fX5b18vdnXQNnzAC+/trwyY8JsZAl1661uE1+imZNA+uFKpVuiM9u3UJ+XZ3Fxsk0Kr0ORQ0NFhtPbIn95tTV4QUDulUa8lqpe4dp2o624EjdUVINvFfq6GL5DY8iJsqTGE/LyEDk0aOoMFHlT2tg0eAqNTUVCQkJijLqc+fORUJCApYuXQoAyMvLUwRaAPDVV1+hsbERM2fOREhIiGKZPXu2Yp2bN29iwoQJiI6Oxrhx4+Dn54ejR48iICDAvE+OWB1jv2qN/bJeFRWFTq6uau+zuRQyMY3gYFbhz8WFBUUrV7JufUJs96uvgHvuYQU3fvqJjQVTmQyVEFvU/9QpiPftw04DJq01hZDDh1FnoQDr3Zwc/OfMGaPH2RiCQ8sg15Auby22a0WBsy6NZmqrumNySM9udBzMV6zi58JCxe/lUily6urwk475AKViMX7q3h0/de8OBwfbqr9n0dYOHjxY64dm/fr1zf5OSUnRuc1ffvnFyFYRe+Kt9IF8yN8fmwWa6JGPddHROF9V1ew25Xf7Bx07Yv6VKwCAvgKO86Pslo2LiQF+/plVBtQQjBvE1xd44w1g/35g1SpWPn7WrLsTEru4CLcvQsxEBOBw03ipUefOWbYxTYobG3HWBN3u+LpYXY1oIb87eLpZVwdHlYuQ+haWsJ0wSj1zVSlUd5xGa+j+Z610XbCWiUTIbCrQJbaxqW9sq7WE6MlJLMaNfv2Q068fngoKwqaYGFxNTDTLvqeGhLQIdNyV5tFq6+ys+F0iYBcGW//nRMAmGjbVydGgQcD69cDw4Wxi402bgGeeAVJT7050LISyMuDcOUCpOwghrUX/tDSL7t8SebMNStkJADincnHRULb0P+2qnsGVjOMsNok4XYg1HdvKsxFigLZKV+QfUyrbbwlOYjEu9ukDGQysECR8k7R6r0MHLFAZ3ErsgJcXm7z43ntZ18PcXFZAo21bFnQNG8a6EhrqzBngzTeBkhI2liwxEejfn83r5eEh3PMgxEo1WLgrW6YVTIp8k2dp9ty6OoQ2XWwU6oRfvp0yKx/X0z8trdncYHwJ8e46Wl6OQjOMvTV0SIVYJkOPpoBdKpVConRx2tpRcEUIT0J96XdxdwcgTF90dfpomCzbEC+3a4dPbt3i/U+S2Jh+/YBvvwW++Qb46y/g5k1g3Tq2xMWxQOuee1iAxIc8E/bll2xsl0QCVFUB//7LFgcHID4eSE4GBgwAaCwsIXbFkJP+sCNH8FxoKJZFRgrWjovV1Rh+5gx2l5QItk1TMCSwAoA/BRhfWC6V4ikDKv3pi+M4g4q8SDgOY5qqOtpacEXdAgkxIUMmJV4dFYXHjciwtXNxQUbfvjrXmx0Wxmt71HXAzrm7Ay++CGzZwiYs7tkTEIlY9un994FHHmFFNs6e1d5tsLqaZau++IIFVsOGAX/+CXz2GTBhAtCuHdDYyLoffvIJ8MQTbPwXIUawpS5jrcF+A+cmWpObi7lNY5CFYu2BVWtxpLwcL6l5bXPt+KItZa4I4UnoIGOgjw8AoKNKIYE4Dw88HxaGX1T6r+ujs5ubznXCqYABUebqCowYwZbCQuCff4C//wZycoA9e9gSGQk8+CDLaClns65fB5YuZes6OADPPw+MGcOCtG7d2PLss6wU/OHDQEoKKzX/3/+yQhuxsZZ61sTGUXBlXYwp6KCpCMj3OqrKEet2vKJC7e1vXb+OVyMizNwa86DMFSE8GRJcaXuMt4MDKgcOxCWVLJO2xzgIWPgiyNFRsG0ROxMYyDJL69ezTNT997NqgtnZLOv02GNsrFZWFguUZsxggZW/P6tE+PDDLLBSFR4OjB/PslnJyWyurddeY48lxAC2VKabaHehqgpxqamWbgYxI0MmmrYFFFwRosM7kZEIcXLCfzt0EHzb7hIJHPQoMfqMMUUGVEwICtJ43xvt2/PqWkjsnEjEJjCeP5+NpZo1i3Xvq61lXf6mT2ddAWtqgIQENpdWt266tyuRAEuWAF27AhUVwIIFgJXMUURsC4VW9qOOAuVWx14vjlBwRYgOr0VE4FZSEiJ4dKPzd3TEPd7eONqzp6BtONWrF2T33AMPBwfMDw8XZJvayr/PCA1VdC00Jle2smNHIx5NrIqHB/Dooyyb9dFHwODBLEgC2Jiq999nXfz4cnEBli0DwsKAggJWvbC62hQtJ4QQQsyGxlwRwgPfUqK/xMRgiD4nmOr2pea2BKUKgEJ2DVSV1qsX2jo7w9/JSZDtvRQejgqpFK9fuybI9ogVEIlYxb/4eODOHaC8HGjf3rBt+fgAK1YAL7wAZGYCr78OLF/Oxm3po6QEOHiQdVPs0gXo04d1USR2zz6vexNCbBkFV4SYkNDjtABgUlAQ3jXRGJV4NWXcjQ3mFoSHU3Blr9q0YYsxwsJYBuull1glwfffZ1ULdb3vbt8GDhxgy5kzrEIhAPzxB/vZoQObV6tvX6B7d4DGGNqlgwZWpyOEWJ62iyNSsRi/xsQAAJboe8HNwmyrtYRYOdXTQYOCKx0nlV2VqrQ9FRSEHzRUUjqckIDktDQDWmC4JC8vHDFw3g7SinXpwrJWr70G7NrFuhcOHgw0NLDy7Q0NgFTKfubns4DqwoXmpeE7d2ZBVHo6kJEBXL3Kll9+YZUQe/YE7ruPze2lxzhHYt0mXrxo6SYQQgz0b2mpxvtkIhHSm+ZCFNvYdzYFV4RYmWhXV97rzm7bVmNwleTtLVSTCDG9fv2AuXOBDz4ANm5kiy4xMWyS44EDgZCQu7eXlbEs2PHjwIkTrNvgoUNsCQ9n1Q5HjACcnU33fAghhGj1pJ1eHKHgihArUdK/P2plMvjo2X3p927dMD49HdvUzBX0eVQUns/MFKqJhJjW/fezSoSbNrG/HRzY4uh493d3d6B3b2DQIFYyXh1vb2DIELbIZMCVK2yerm3b2FxbH30ErFsHPPQQm4/L2K6NhBBCBCXmOHS5fRsAIJPJbCp7RcEVIQIyZnSSPkGVh0SCSqkUXdzc0MvTE3X33KN2vRlhYWYNrkxXaoO0Go8+yhahiMVAVBRbJk0CduwAfvuNVSj84QfWbXDYMGDaNP2qHRJCCDEZiUyGcenpAIDG0aPhJFChLXOwnTCQECvWxc0NYgB9VApC8K0yqK+i5GSUDxgAd3kpbAsa6O2NFR06YEdsrNrgylTHgBC9ubmxLoE//cTGeHXtysZx7dgBLFzI5usihBBCjEDBFSECON+nD6oGDoSHmSrauEgk8OS5r/Vdupi0LfsTErCgXTuM8vNTe78pJwl83wQTO5NWQCJhBTM+/xz45BNWEj4zk5WBl1cdJIQQQgxAwRUhApCIRHCxgiySOpODgy2y375qyroLrZ+Xl8n3QexcbCzw1ltsXNeBA8A331i6RYQQQmwYBVeEtGIPaMg2GUq5C2BgU/9oW+kWOCkoyNJNIJYSGwu8/DL7fcMG4O+/LdseQgghNouCK0JMyNrDih9M3GWQEJsxbBjw5JPs9w8+AM6ds2x7CCGE2CQKrggxIWsPrnwcHfGEpnLWAjHlmCtCBDV1Kps3q7ERWLIEyM21dIsIIYTYGAquCDGhYBsqHSoE5WDyKTN0s+PT5ZDvSDgKAQnEYuCVV4DOndlExK++ClRWWrpVhBDS6khFImyNjsbW6GhIrHRMuyYUXBFiQv8XGopngoPxW7dulm6KWSiHOmMDAthtOgKg+eHhGu/7oGNHo9tUMXAgvuzcWed6FFwRAICLC/DOO4C/P3D9Oit2IZVaulWEENKqyMRinA4OxungYAquCCF3OYnF+LpLFzzaFGi0JnwLWThoWW+elsCLL1eJBE482kLdF4lCQADw3/+yQOvECTYGiwIsQgghPBgUXH333XfYvn274u8FCxbAx8cHycnJuH79umCNI6Q1cxezj6epr4CYupy5qcaduYk1H5mRbdoAABKbysHzCfQotCLNdO4MvPYa6yq4cyewYgUFWIQQYiZijkNUcTGiioshs7H5Bw06b1u2bBlcXV0BAEeOHMHq1avx3nvvwd/fHy+99JKgDSSktdqfkIDBPj442rOnSfczIzQUX0RF4WKfPkZvS10I46QSBD0kQPn3Yb6+uN6vn8b7Y9zcUJScjIMJCby3qSv8Gubry3tbxE4MGMAKW4jFwO7dwLJlFGARQogZSGQyTDx/HhPPn0djY6Olm6MXg4KrGzduoFOnTgCArVu34tFHH8Wzzz6L5cuX48CBA4I2kJDWqqenJ/bGx6OPiTNLDmIxngsLQxd3d6O3pStDJAKwicf4s+khITisJTBaHRUFfycnjQERB8DfyQkOWrJbqpzEYrzWrh3mtG3b4j5HkQi74uJ4b4vYkcGDgTfeABwcgH//Bd5+m1UTNNStW8BHH7FtUVdUQgixOwYFVx4eHiguLgYA7Nq1C8OGDQMAuLi4oKamRrjWEUJapS86d0aSt7fG+6Pc3Eyy33c6dMBHnTrxGqOlr5T4eAQ4Ogq+XWIGAwcCb74JODoC+/ax3xsa9NuGVAr8+ivwzDPAH3+wIO3VV4GiItO0mTBSKdB0vkIIIeZgUHA1bNgwTJs2DdOmTcPly5dx3333AQAuXLiA9u3bC9k+QogdEUF397uZoaGQaAlulMdaKa81qmmclRBKBgxASf/+ir/V5Rce0LN74z0+PvB1cDCyZcRikpNZQOToCBw8CLz+OlBfz++xV68Cs2YBX3wB1NUBUVFsO0ePsrm1/vyTslimkJXFgtnHHgN+/tnSrSGEtBIGBVerV69GUlISioqKsHnzZvg1nWScPHkSEyZM4L2d/fv348EHH0RoaChEIhG2bt2q8zEpKSno2bMnnJ2d0alTJ6xfv15t+9q3bw8XFxckJibi+PHjvNtEiD37qGNH9PfywmYTlYY3JN+jWqVvakiI4PuI07PLo5tEAh8dWaY/Y2MNaAmxaYmJrIqgkxNw5Agbj1VcrDkwqq8H1q0Dnn0WuHQJcHcH5s8HvvwS+OoroGtXoKoKWLkSmDePdRnUl1TKHnf0KHD7tv6P5zj7G0cmk7Es4fPPs3L6ADveX39NQSwhxOQMuozq4+ODzz77rMXtb775pl7bqaqqQlxcHJ5++mk88sgjOtfPzs7G/fffj+eeew4//fQT9uzZg2nTpiEkJAQjRowAAGzcuBFz587FmjVrkJiYiFWrVmHEiBHIyMhAYGCgXu0jxN60d3HBQSMKZKT26oWVN27Az9ERnxpyIoiWwZHqOC1dwZOm6obaTpliPTx0tsscp1ymqpxIzKhPH2D5clZJ8PhxlhXx8ADatWu+ODoCn39+9+R+wABg9mw2fxYAtG8PfPopsGUL8M03QFoay7I88wwwdOjdoEcmu/u7VArk5QHZ2XeXnByWDQMANzeWUevbl99zOXuWBYulpaw6YkzM3cWY6SM4jrXbEnPT3L4NvPsucPIk+zs5mT239euBn34CqqtZFlGP8ZhGq68HLl8Gzp9nr1fXrsCgQYCWrs+EENtlUHC1c+dOeHh4YMCAAQBYpmjt2rWIiYnB6tWr4cuzqtaoUaMwatQo3vtds2YNIiMj8eGHHwIAunbtioMHD+Kjjz5SBFcrV67E9OnTMXXqVMVjtm/fjnXr1uGVV17R52kSQlT08vTETzExWHnjhtr7h/r6Ym9pqda5q3RRfeQgb2/sLyvDv3Fx2FZcjJd5zH3FJ1B60M8Pfxo5FqO3pydSKyp4r0/XzO1Ez56sNPtHH7GT5cpKID2dLap8fVlQNWgQoPq5kEiAsWNZAPDhhyzA+vxztujDyQnw8mKBxaJFwIsvAqNHa3/Mn38CH398N2t1/jxb5AICWJDVrx8wbBj/QOnYMWD1aqC2Fli4EOjVS7/nYowDB9icZOXlgLMzy1w9+CA77j4+7Pn+/jvLFi5YYLrgr6SEHcsLF9jPy5ebj9H76y9g1Sr2Prr3XhZ4m7hwESHEfAwKrl5++WWsWLECAHDu3DnMmzcPc+fOxd69ezF37lx8++23gjZS7siRIxg6dGiz20aMGIE5c+YAAOrr63Hy5EksWrRIcb9YLMbQoUNx5MgRjdutq6tDnfzKH4Dy8nJhG05IKzE/PBzBTk74j4+PxnVUM1W6Ju/dGx+PGpkM7hIJ7jWwHLq6UG91VBTKGxuxr6yMVzvU+bN7d4Ro+W4hdqxHD+Dbb1lW4uZNFmQpL0VFLDB57jndGYqwMBZcbdvGslhlZSyzIhKxn8pLQAAQGckyX5GRbAkJYZmiDz8E/v6bnbjfugX83/+1DCAaG1nwI++GP3gwMGkSkJEBXLzIAsSrV1n79+1jy08/sbFhgwdrzvjk5wOffQYcOnT3tpdfBiZMYI/lM96wspItwcG611VWXc0CUvn8m1FRwOLFLIMoN3o0y+y9+y6waxdQU8PWcXJSv82yMvac2rdngZou5eVASgor2a8cpMr5+gLduwNt2wKpqUBmJvuZmsqC9F692PG9915++yPEzklFImxvqky+2BJZcCMYFFxlZ2cjJiYGALB582Y88MADWLZsGU6dOqUobmEK+fn5CAoKanZbUFAQysvLUVNTg5KSEkilUrXrXLp0SeN2ly9frneXRkJai65ubrhaW8trXSexGE/rGDOli2ogJBaJ4M7ji1Xf4CjcxQUpCQkQpaQYvO1gnidB53r31qdpxJY4OQEdOrDFGCIRy7I88MDdv/UhkbBMUdu2LEDbtIl1IXz1VaBpXkqUlbFKh2lp7O9nngEmTmT7iowERo5kt9fUsGDrzBkWhN28yYp5bNjAHtOv39321dcDv/zCArD6etaORx9l2/jzT/aYtDQ2Pk3Td0NhIWvvtm0s4xURAfznP2xRMzUCANYV8tgxYO9eNt6stpa1afx44OmnWbdMVcOGsQDrzTdZluu114C33mLHp7CQdZOUL/LunA4OrBtfXBwQH8+yefLjWV/P9r17N/spL9EvErGgrHt3oFs39jM09O4xe/ZZdkxTUlj7r15lz+XYMdZ9ceZMls0yQdVSQmyFTCzGibAwAICkNQRXTk5OqK6uBgD8888/mDRpEgCgTZs2Npn1WbRoEebOnav4u7y8HOE8uh4RYmt0zUOlztfR0VicnY3/Cw29ux1D98/jsYa0UZWznuO4TK07jzFfpjA2IACbqNS3bTHm/S8SAU8+yYKYFStYVcOXXmLjqsrKWKYmL48FB6+9BihVxGzG1ZUFEvHxrNvi5s3Axo3AlSssWOvWDZg2jQVQn30G5OayxyUksC6J8qrBvXqxbnoXLwLTpwNz57KASS4nh1Xx++ef5oHJ9essK/jttywL9Z//sIyOry9w4gQLSA4fZvuXa9uWPVddY0r792dj5hYvZlmj//s/FiQVFLRc19MTqKgAzp1jy48/suCxSxd2jI8eZZk2uY4dgeHDWXvlY+s0aduWvVZPPsmeb0oKCy4LCoClS4HevYEXXmiefdMHxwGnTrEAt1+/u4GzueXns+A6LY0FrW3bsoDWRNNptCoyGftMRkRozsASizAouBowYADmzp2L/v374/jx49i4cSMA4PLly2ir6SqTAIKDg1Gg8gVYUFAALy8vuLq6QiKRQCKRqF0nWEs3A2dnZzhTGp60AoactgU7O+PrLl0Eb4sm+rRRNZ/0QceO+LGgAAsMPSExQG5SEq7U1GDg6dM61zX3mKtfu3XD8DNnsLukxMx7JhY1ZAgQGMgCiIwM1jWxqooFI6GhwDvvsEwVH25uwFNPAQ89xDJUW7awsUQvvXR3HX9/YMYMFgApB4f33ANER7Pg7vx5lv1KTWUn+r/9xoI/eVY4IYF1Iezald2+d+/d7nOZmazCoosLy1DJBQWxrnSDB7P98A1Me/ViXSgXLgTk40fFYhbIxcayLp+xsaw7Z24ucPo0y+KdOcMyXBcusEX+3IcMYUGVodnLiAhg8mRg3DiWAfz1V/bcn36aZQEnTWKVJvmQB1XffccCQoB17czOZoGkqQt53LnDiomkpbHjlpfX/P6CAhagr1hhXPdHqZQFxfX17GKAvsEFx7GAns/cg1VV7PmcOMGW8nJgxAiWJbVUoTSplH2e9u1j4/WGDQNGjWIBvp0QcRwimrrty2QyiM1ZhMZIBgVXn332GZ5//nn89ttv+OKLLxDWlLb766+/MNKEV0eSkpKwY8eOZrft3r0bSUlJAFhGrVevXtizZw/GjBkDgL0ge/bswaxZs0zWLkKI4RxVvjCNyTLNCw/HPAOzzoYGPiHOzggx8uJMZt++iDLRlBE0cXErFRvLxiEtWnQ3gEhIYNUEDalS5+3NTs4ffRT44Ye745see4yd/GvKRAQHszFg333HMj9//cUWuQEDWFDVNNQAAAu+Ro5k2bZ9+1igdeYMC6z8/e8GVF27Gh4sxMSwsWcHD7Kgqls39c8hLIwt99/PTsjz81nQcPMmy5LFxwtXGMPVlWUER41ibTtyhAVa//zDuhLee6/mIEJdUOXoyALJo0fZduTdRF1cdLelqIjti+97JTOT7WPv3ual/eWZvvh4loX75BP2Wr7++t1547QpKWGFSC5cuBtM1dezrI2cqysL/seOBXTNP1hfz7pxbtzIPhf+/uyCQ2goe53lPzmOBbgnTrB9q05XsGULmwx8+HD2/jU2scBx/C8OSKUs+7pvH/u7vJxllzdvZhcZ7ruPZU8t1GNCKA4yGaacOQMAaLzvPjjZUHbOoOCqXbt22LZtW4vbP/roI722U1lZiaysLMXf2dnZOH36NNq0aYN27dph0aJFuHXrFr7//nsAwHPPPYfPPvsMCxYswNNPP41///0Xv/76K7bLv+QBzJ07F5MnT0bv3r3Rt29frFq1ClVVVYrqgYQQ4xkTACk/1l0sxqSgILzTNL7BRSxGFxN0F1HtavhV586C78MYnUzwnOVVFalCYSsWFsa67X3xBbvC/tRT/ApLaOPvz7JWTz7JTgj5XLmXSFgWpmdPlsUqKWHl5h9//G4XQnW8vdlJ80MPsUqIpaUsOyTUFex27YAnnuC/vkjEugMaOa5Up7AwYNkyFhR99hkrTvLuu2zx92cBa0gI+xkczIKL339vHlQ9+CA76ff3Z8HE+++zcWbybqKaJl3PyGAB2pEj7DjHxQEDB7IgWLU8P8ex4GPjRhbYyUVFsdc6IYEF+crfbyEhrNDJsWOsHUuWaA5Oz51jXQh1zd9WU8PasGULyyg9/jg7hsqqqlgXyd9+Y3PTyd2+zZazZ7Xvo21bNg1Dnz6svT//zILsHTuAnTtZlnbiRMMyRz//zLLCY8ey10xbsC6Tsa62e/aw9V5/nQXBO3awrrIZGWz5/HNWobRvXzbmLyiIxvCZkcHfslKpFFu3bsXFixcBAN26dcNDDz2k16Cz1NRU3HvvvYq/5eOeJk+ejPXr1yMvLw85OTmK+yMjI7F9+3a89NJL+Pjjj9G2bVt8/fXXijLsADB+/HgUFRVh6dKlyM/PR3x8PHbu3NmiyAUhxPKO9eqF9kpXUUsHDGiRyeKLbxCxIzYWo9Rc3bS1IOSltm3x0c2bGu9/18AuSq5iMWqUrwoT2+blxbq/Cc2QebDi41n2qrZW/+yZv7/ucUz2pl8/FqRs3sxOvsvL7wYD6ioSqgZVcsOGsZPrJUvYZNYzZ7LMh3Jgm5nJimkcPsz+FonYibx8vNQnn7BM4cCBbOqAixdZpio7m60vFrNs4rhxLHuiSWwsy1i99hrLvLi6smBL+Xuf41iwtHYta0O7dmw6gzZtWCChvDg6svnmNmxgx2TbNhZo3HMPC5zbtGHH73//YwEWwI7NuHGsvbdvs+A1N7f5z4YGFhz27s0CKtWAum9ftr8NG1gguncvW5KT2YTgmoJXVf/+yya4BlghmhMnWLZZ3VAWjmNZ4J072fFasoS9HgCb4Ly0lFXC3LGDjePbvZst8ufcrdvdJSqKX5dIvuRz8AUG6t9Fs76eVfzUUmXY1hgUXGVlZeG+++7DrVu3EN30IVq+fDnCw8Oxfft2dOQZuQ8ePFhrha/169erfUyavNKRBrNmzaJugISoYenrViKRqEUWyVEsRuXAgRABcDbiijTf4MjTgC481hh4rezUCduLi3FZeVC/EjFdpSTWyNmZSo3rw8mJBUuPP363PHxeHvsp/724mAWuqkGVsh49WBZs0SIWPMyaxaomenuzTNXBg2w9sZiNIZNnOQ8cYMuFCyygunjxbjAAsC6GDzzAuovyLaHfpw8LDN54gwUKrq6seIdIxAqIvPvu3SBvyBAWrMgrNKqTlMSWs2dZsCOvIrl3L3sO8mIp7dqxYzRkyN3AIiCABY2G6N6dZRizsth+U1JYu3NzgZUrWQEWbS5eZGPP5M/h9Gn2HKZNYxnGIUPurstxbNLxP/9kx2nRIhZAKvPxYUHj2LFs23v3sgAwM5MFkfKpFQD2/IOCWDdK+YUL5d99fdn23N3VZ7zkc/vJ58e7eJFdNAkIYPPL3XOP7kwZx7E2fvUVm+j7rbe0r29DDAquXnzxRXTs2BFHjx5Fm6bovLi4GE8++SRefPHFZt30CCGW91RQEE5VVGA436tpZhLZlLXiU2pdKEJUI9RmanAwvs3PV3ufIUHakogIvC0vC63CkECREGKD5BMh+/iwMUyGCA9nY7kWL2YnxAsW3B27JBKxcTqTJjWvUDhuHFuKi9kcZgcOsC6Avr4soHrgAVZVUV8DBwKvvMKCk99/ZyfxAwawgCs/n538z5p1dxJoPnr0YEtWFutql5LCAquuXVkWKznZNAU9OnViFR6feood02vXgPnzWYClKUNbVMQCzPp6Fli9/TZ73suWsaDlnXdYt9DZs9mx+fJLdpwAtg+VOV+bEYnYmEL5OMbaWjaRtXxS6/PnWRb05k22aOPoeDfQ8vVlbbl2jWUsVZMjYjF7Xm++ycb6vfii5mqX58+zrotNvd8gk7E22clk2gYFV/v27WsWWAGAn58f3n33XfTXVNqVEGIx33ftCo7jBAss9N3OAG9vHCwrw5NN3XMrBw5Eg0wGNyODA+Wv9in6TjxqIuMCAuDr4ICVuv5p8dDfywv3+/lpDK740Dego3wXIXbM25tVSlyxgnVJE4nuTiStbfybn9/d8W/y+cyMvbgzbBjrDrZqFesuumEDO8kODWVjiQwdG9upEwtcnn2Wbb99e/OMN4qMZAHVSy+xucvmzWPHWjXAqq1lAW5xMWvba6+xYxkWxrpf/vgj8P33rJDJuXMsUJEXc5s7V/+y+i4udwNP4G5hlsJC1gZ5V9Pbt+/+XVLCxrI1NLD1Cgtbbjc09O5cbrGxLHO5cSN7HU+eZHPijR3Lgk555jE3l2Wq5Bk0FxcW+I4dy6/Qio0wKLhydnZGRUVFi9srKyttqpoHIa2JkBmbYU3dHVx4XgX8s3t37LxzBw81dVlxF+Ifs4pII7+YtQUhQnULVPcKzNdS3dCQ16yHuzs6autCo6STqyuyVLoVPhcaKkhgSAixUk5O7IR+yBB2Qh8Rof/jhTJ6NAuAvvqKBVYDB7LMjBCV7iwx1j48nAVYc+awOahefpkVoJBnZDiOBbaXL7Pbli1rXmZfImFl+Xv3ZgU/8vLuBlYvvsgyecbiW5iltpaN4yotZcFWaSnrmioPqtT1hJkyhQXNn33GMm/yeeymT2cZxd9/ZwGbWMyqYk6dqrvCow0yKLh64IEH8Oyzz+Kbb75B3759AQDHjh3Dc889h4ceekjQBhJCrE+Muzsy+vZFEM9/sj6OjnjcBP/oQgz4J69tnKepqdvzA1r+scxQmriZ7/ZO9+7dLCh7MSwMv6i76gggvU8fOO3fr/j77x49MNjHh4IrQuydWMy6yVmDCRNYICSTsYDP1seLtmvHAqy5c9l4J3mA5enJMlIpKWws2FtvaQ5wunVjBT0++4xlGKdNAx5+2KxPAy4udytS6iMsjBVMOXyYjROTd3eU692bzb2noz6DTCTCrqbCTIttrAu8QcHVJ598gsmTJyMpKQmOTYMCGxoaMHr0aKxatUrI9hFCrFRnE5QP11cHnhkaPrrznaTTTBaGh2NCYCCOq+kloI1qtivJ2xuFyck4W1WFSqkUY5qqjEnAiol0dnXF5ZoaOItEFhmT5+vggK5ubjhcXm72fRNCrMR//mPpFgirfXvWJfCll1iWasECNj5NXqhtzhxW5l4bd3dW6XP+fMF7ephFcjLr0rhhA8tghYayycb79uUVQEvFYhxu6tmhTyVya2BQcOXj44P//e9/yMrKUpRi79q1Kzp16iRo4wghRGiqwceJnj3x1507eEnLJJCqGaJgPTJmf8v7uUN9N7w4DUFdN3d3wbpyBjg5YYiGNm+PjcXSa9fwiqaBxya2s0cPfHbrFgVXhBD7ojwG69IltgBs4u377+e/HRsLLJpxdmZd/554gnUntfWsJE+8gyv5HFSa7N27V/H7ypUrDW8RIYQYwM/AOTt6e3mht54ViobrKLGrHIwpZ4O+iY5G2JEjir/v9O8Pn6Z2L46IwO47d3BMJVNlqm6M8sCtk5sbNsirSlmACWp3EUKIdejQgWWw5s1j1fD69mVd4lobA6ZfEHEcQpr+H8pkMohNUenRRHgHV7rmlpIzdZljQghR9mtMDAobGszaTdHQb7lQlX8wvkoB4duRkXg7MhKilBQA/IpoxHt44FRlpV5t+CIqCjMyM7FJwIBqkLc39peVGfTYSFdXqlBICLFfnTqxsVPHj7MiDraciTIjB5kMzzbFHo0jR9pUwTzewZVyZooQQqzF2MBASzehBXMFCxODgrBOw5xamjwXFoanQ0LgJOBVQF3Pt52zM3Lq6lrcfqlvX/g5OlrlJM2EECKY8HC2kFbBdnJshBBiB3RVAJTjE6CpfoHzzUYJGVjxsSQiAkPVdKWMFjjb2M9OJqAkhBBiuyi4IoQQPRnT/dmUX7qPWSiLt7ypXK6hhMpc9fX0FGhLhBBCiGEouCKEEAEJ3cXNHJMbG+IBPz/09/LCgfh4JHl7a11XVzsnWWKyT0IIIcQEDCrFTgghRLOeWjIopgqIdiqVfDeHOW3bYoiOqonKHg8MxD8lJWrvG9amDS726YObdXUYdvasUE0khBBCzI6CK0IIEdiEwEBUS6UmHwOk3DlxhJknANanRLyHRILxgYHIq6vDx7du4b+RkRjj799snS7u7oh2c8OEwED8XFhoUJuo6iAhhBBLo+CKEEL0pO0kXgRALBJhOs/CFbpoKzFvK1X2xgYEQCwSYXH79ljcvr3G9UQiETbExBgcXBFCCLEPMpEIKRERAIDFNla+noIrQggRkNABj7rJkVcYWUDCFFzEYtTKZGrvczBTdUJ1hUbu9fHB3tJSs+yfEEKIMKRiMVKaLsZJbCy4ooIWhBCiJ3N3P3NXCk6u9euHBe3ambkFuh2Ij1d7+4cdO5q3ISpGq3Q/JIQQQkyJgitCCNFBHtzIq9rNM/NkkMrZsAgXF7W3W5qm8vQxAs9lpcnSpu4j+mrr7CxwSwghhBhLxHEIqKpCQFWVXmN8rQEFV4QQosHOHj3Qxc0N/8TFAQDWd+mC6oEDEePubvA2+f6L6OHhYfA+hPJYQIDG+1Sfh8yC//yu9+uHN7SM5dL12AQrONaEEELucpDJMDM1FTNTU9HQ0GDp5uiFgitCSKsSoUemYkSbNrjYty/6Nc3jJBKJ4Gpk329dIUhG377YExeHOKUTfktVwWvjYPywXGMmXFYmD3CVjQ0IwF+xsWjn4gKRSITpISF6bTPQ0RFigdpHiDa9KIAnpNWg4IoQ0iqc6d0b++Lj0VapW5016uzmhv/oMX+UKUW5umJlx454W01WSDVI1BQ0CtWdQ92cWmP8/THSz0/xd4y7O/pomWNM1XI9C4PsVRPgEaLL6qgoHO/Vy9LNIISYCQVXhJBWoYeHBwb5+Jh8P/aWB3kpPBxPNI01A4D727RBVzc3DFY5lkJ2C0zUI0BSxXc83DuRkXhaz0zXYB1B70tt2+q1PdI6yKdnIIS0DhRcEUKIAJ4JDkZvT0+1GRZTscQop03duuFCnz5wUimvrqktQnULVEfdPsepjBPTtPdQJye99hXWtH7doEEa11nZqZNe2ySEEGJ/aJ4rQggRwNdduphs26YMUAyhrj2agisHM7fdVMcq0tUVAFoElXJxRhQ5IfbNuj69hBBTo8wVIYRYOW8LT6DI5+RQXXA1wte3RfdBS5P/01PuIsrn+elaZ3poqIEtIvbOtopIE0KMRcEVIYSYkSEFHv6MjUWsuzu2x8aaoEW6hTVVWHQSieCsIXPTy8OjRRC4My4OEgMySXyzT48YMEHwnQEDcCUxER2bMlFA87nDNBH6BDmnXz+k9+kj8FaJNZK/11w1fHYIIS3JRCIcatsWh9q2hcTCFxj1RZ90Qgixcgmenjjbpw/uU6qMZw7ygMJJLEblwIEoHTBA48B8F4kEhf3744WwMJO3653ISDQMGmRQWXxvBwd0UAqsAODzqCihmtbMCC3j78JdXBBKExi3CsOa3gfUPZCYU1czTeBuKlKxGLs7dsTujh0puCKEEGJ/3CUSncGMk1gsyD8VTSehVxMTsTcuDq9FRMBBwCxAsLMzriQmItjJCSs0lGfXdWKs7n5dGTgXymS0CvL3AVUMJOaU3revpZvQalnFN/vq1avRvn17uLi4IDExEcePH9e47uDBgyESiVos999/v2KdKVOmtLh/5MiR5ngqhBBiNkLNIWUrIl1ddZZDN1QHV1fkJiVhQbt2gm1T9fUZ1aYNVnXqhGM9ewIAnMViHO3ZE2s6dxZsn6b0v+7dLd0Em9bNxjMJhJiTiOPgU1sLn9pam/tfZ/HgauPGjZg7dy5ef/11nDp1CnFxcRgxYgQKCwvVrr9lyxbk5eUplvPnz0MikWDs2LHN1hs5cmSz9X7++WdzPB1CCNHKlv5FtHF01PsxtvT8VBlTaZBvUYzZbduir5eX4rZELy/8n5HFMPQtK2+ofkrtJvozd+VMIrwFPOfRI8ZzkMkw59gxzDl2DA0NDZZujl4sHlytXLkS06dPx9SpUxETE4M1a9bAzc0N69atU7t+mzZtEBwcrFh2794NNze3FsGVs7Nzs/V8tVztrKurQ3l5ebOFEEJaq2+jo/FUUBAmKU0ebKsMOZ3tbcQkxsrbMFcJ/ZtJSSbfx5nevRFopiDOHjymMt8aIOw0AS/TST4hVsuiwVV9fT1OnjyJoUOHKm4Ti8UYOnQojhw5wmsb33zzDR5//HG4q8wxkpKSgsDAQERHR2PGjBkoLi7WuI3ly5fD29tbsYTTlxYhxESmh4QAAO7x9rZwSzSbEhKC77t2haOFxgSpnoImeHiYdf+HExKM3kaiAAGaJtEqBTnMEcT1MPNroK9XBOzOKYQ6mcyk23+vY0esN+Hceq2NM2UViYAsGlzdvn0bUqkUQSpXR4OCgpCfn6/z8cePH8f58+cxbdq0ZrePHDkS33//Pfbs2YMVK1Zg3759GDVqFKRSqdrtLFq0CGVlZYrlxo0bhj8pQgjRoreXFwqSk7EnPt7STbEJFQMG4ESvXgY9Vlu1Pm0sEVQe0OP98K+adS/26YNpTYE7X+15lKBX1YtnkLUsMlLvbWsT4eystQDIYwEBWBIRIeg+jRGn5jgJffo+Xk12zNIOJSRQ99FWjIJUxuLdAo3xzTffIDY2Fn1VKqI8/vjjeOihhxAbG4sxY8Zg27ZtOHHiBFJSUtRux9nZGV5eXs0WQggxlUAnJ4Pmf2otlI+Mh4ODwcfKnMdYbbVAlb+1jUcb4OOD37t1M3hfXdzd9S6MoW8wBgC/8yxqMbeV9wBp6+yMjL59UZicrLhN6BMuc3U71Ueyt7dNnlha47G0ZwEGjOe1JRb9DPj7+0MikaCgoKDZ7QUFBQgODtb62KqqKvzyyy945plndO6nQ4cO8Pf3R1ZWllHtJYQQa2KNxSPsdaJUQ069TPX6aGqLPsHkv3FxBnWlC+eZ7RL6VPVRK8jSfKbHfGhd3NzQ2c0NAUrj1OgEntg7vu9xdWMS7YlF/ws6OTmhV69e2LNnj+I2mUyGPXv2IEnHAN1Nmzahrq4OTz75pM793Lx5E8XFxQgx4CodIYQQ/ha2a4fenp74qGNHSzdFUNYUyArRlnt9fW0qeypUl7/sxESDHhfu7IyZekyQrW6coNBH29DtPWjmycgJaW0sfolx7ty5WLt2Lb777jtcvHgRM2bMQFVVFaZOnQoAmDRpEhYtWtTicd988w3GjBkDP5UvicrKSrz88ss4evQorl27hj179mD06NHo1KkTRowYYZbnRAgh5mCNk5L6OjriRK9emGPj3cI8dUyYzIc+3QINZUjXPm1CnZzwhUqG5ikLVI1UHV/lJBZrnetG2ydBuQtSe5ViIHz5ODgY9DhlT1pB9c0oV1f8ERtr0n1Y37dSc45GfG8+5O8vYEuINjKRCMdDQxEUGwuxjfWIMP7bwkjjx49HUVERli5divz8fMTHx2Pnzp2KIhc5OTktDmpGRgYOHjyIXbt2tdieRCLB2bNn8d1336G0tBShoaEYPnw43n77bTg7O5vlORFCiDnc4+2NRE9PxKhUS7V11tB9SrUFulokRJv12cZfsbFYn5+Pdzt0MHq/yob6+jabj2lTTAzuMyDTYezRUA2kjNleqJMTinTMkxPk6IgCAefSURcGTgkORmdXV3R1d0fP1FTk1NXx3t722Fjcf+5cs9ss/ykxzpZu3fDIhQtm369EJEKDAe+vc717o7uVV82UW2jlF7f4HG+pWIwdUVGY3aMHHAS4uGFOVtHaWbNmYdasWWrvU1eEIjo6WuMVLFdXV/z9999CNo8QQqySg1iMowZW0msthDoB1TfrJBKJ9N53pB7jmUb6+WGkmqDnx65d8eTFi4j38MDpyko9W9AyY/dYYKDe21AlgnV1q/wrNhajVAIVcxCLRBjg4wMAGB8YiPf1qEzM97r9md698eTFizhXVaX2fmt6HSzF0O8EXYGVq1iMTzp1wvTLlw3cg/4+7NgR865caXG7NVygas1sK89GCCHE7tnDaYGTASc3PTw8MJlH1zFtJ8gTg4JQM3AgZoaG6r1/XdvWh/KzP9+nj8nboc/RVheUmpu+z4/v+j08PCDT0n2SqCfEd04XNzezzwdnTLtHtmnT7O/3BM6C6/KIri6WHAe3+no01NRo7RJsjSi4IoQQ0iocNGJyYL4nMSs6dEA3NzcsUqnE5yGR4AMeRT4eEaCKlotEYlSQZI1XvXVlvwKVqvJZA2NPBpfzmCdM06tkqdPQeW3bArDO948yU7XOEs9a330OV5r774327Zvd97KG6qH6VBXVpz2bunVDTr9+Gu93lMmw4MgRpH79NRoE7LJrDhRcEUIIsSqmOknp7+1tcBtilca19VczF6J8/QXt2uF8377wVznZLx0wAN14jI2rl8la3DY2IADxSlfEhTw+33Xp0qxYg7VcH9a3HWFGjqnWFRAI/Z7UNdHuKD8/tLHgOJOTenY3zk9OxvtNFw96WPkY0Clqpvqxlve9vvQJZLd2745flebSM8X3LN/jKALrJst3agdbQ8EVIYQQuxLUVB1utL+/IFfRF4SHY7lSl5l/4uL03gbfsufqTrpfadcOewzYp1ywSqCX6Omp+H1ScDCK+/fXa3sbY2J0rmPscXcTsDoY37Z0cXPTuY486HipKUujia6TzEf8/fFT1654Ws2J/h/duyPOw0Nn0KvpefE5wdU1vo/PsZDr4OKCICcnRXuWd+iAl8PDkaolQBPzbKcq2T33GPCo5tSV1Bci0LDufB37PrTXeQitDR1lQgghduVcnz7YERtrVJly5RPXFR07wlPpRNdFIkF7lZNToU6s2rq4IDsxESUqAY8xY3R0lYTXt6T/uMBAJOnIvBjrqeBgBCqVUDcmWFPXRe+v2NgWY0z2xsXhcw0TBT/fdELe09MT9YMGYWWnTtgYE4PflDIBsXpkbEQiEZ4ICsIwpW5aAOAuFuPBprEoW7p3R5y7O/7s3l39NjRsm897JaNvX+3t03KfrvGEXg4OeK9jR/RSCuJVHTKwi64QF0vUnfjqs9UjCQl4s317jFfThdeYDNibKt30zKEgORnPBAfjWM+egmzP2gNMc6HgihBCiF0JcHLCKD8/iEUig6/UDm066Q3hOZYnXsBJY9u7usJHKbAwhKEneaYoaGEIF7EYKfHxzW4TsuvWSD+/FmNMgp2dMSMsDEsjItDZ1RUDvL3xR/fuSOvVC88qBeqOTe+pcYGBeFTpBNtf6TXjO0/auMBAvKphTEuchwdO9+mDBzQM/NcUFGsraCEPNB0tnMHop0cXXX3M0mOiZ2X6zGPWz9sbS9u3b3EMNQV+z/MsLrPUgOCqt5YAlo9AJyd83aUL+jZdLFEdKwro91lWXndt5868HmPuQhrmQMEVIYQQqyLk1c+JQUH4j48P3tbzxOWrzp2xPDISRzRc0X28qUS5o0iEnT16oI+JMzn6HhPlrI/qqbYQx1ef8UCGBEV82tjBxQVPBwcrTuKEet+8GRmJjMREHEhIwIP+/oj39OSVMXEUiXCnf3+U9O8PB57Bi1gkwn+VTi71PVbhJpq/01WASbQt4VOlzKM+7wc3gSYNd1DzPhF6om9lfTQEV6rdOjupmTxb3fFZZmSgo/z+naYlqFT+PA1Vyd7aA6uY54oQQggxBWexGHtUMiB8+Do64pWICI33v9m+PXp7emKwjw/8jMwy6cJB/5Pu0f7+mB0Whr5eXi3mwdG1rQgeJ+yfd+6MiKNH9WyVdr/ExODx9HTF37raKRGJ8E2XLjrX5xMYCRWY+ZrovaDP62+rxRmEps/8aurWO9mrF3qdPKnXPhMsXIr9cEICDpSV4SmVKR30GUNHjEeZK0IIIVbF2ks5A4CTWIxHAwJMHlgBLa+G8zk+YpEIq6Ki8ASPebOUcRyHIb6++LBjR+zq0UPjeu10FEQw5BUcrzRhserjrf8dYT1FEZQDBdWulUJQbaMQ0wcYw6sp65Sskj029ljyqe6puj9d4xf16X6oylHNtkW4O1ZrVadOSPL2xoJ27TS2wxY+R3IykQing4IQ0KULxDZWiMO2WksIIcTudaWrrACAF8PC8Ki/f4vS1vrOoaQ6WaguIpEIc8PDMUyPx11JTFRUaeSDTycsXYU4+AZgfMbNWctJpxBZJ+X3xz0+PgJsUbMNXbvq3eXWWB1cXOAmFuOzqCg8FRSEW0lJ2BQTg+2xsc3WU3cRQp/JvYV8T8i776nOLaUPde0RiURY2r498pOTMVtHBUtzMOSYaXqMVCzG1i5dEDVsGBwsOC2BIWyrtYQQQuze8g4d4CgSYYJSJsMWGZuB+9jA8SOqPunUCevz83mta+jJfQeVMR26nntGYiI6HTtmkrao+qpzZzx7+bLJTz5NmXHVFFCru1XbcRO6y+AEPTOjygxtS0bfvpCCdfmVl1V/TM13hQjA8Z49sfTaNbzXoQNu1NUhwcMD5VKpwW3WRtur/29cHNIqK9Hf2xtzsrIM2n5/b2/sLS1Vu88gHRcQwqxskm1Nx2pCYCCWRESgo6srnPfvB2Cb3Vwpc0UIIcSqeDs44OOoKJNVFLNFxpxgeDo4tOgypXFdExQyUD3RmB0Who5qBtgrM6RbYIBS5uy5psH0r7Vrh7YuLtjRowdG6JnBs1VCn4yqvidM3W1XuRjLIDXfAQ5iMZx5dBMTAejj5YW/evRArIcH7vPzQ4iJCoDo4uHggIE+PhCLRDhqYNnzn2NisDA8HFv0mAh4R2wsxgcEGF2owlw2xMSgq7s7nMRigOPgKJVC2tCgd7be0ii4IoQQQmyIsSe3yWpOWL/q3BmDvL3xmpYiHrqotmtKcDAe8PNDFzc3POTnB4DNEbRKw1xSxpoeGoqngoKwoWtXfBYVhTO9e+OtyEiT7Mua8T0NfaSpxPv5Pn20ricRiZpVJfQwMAA3ZGLoZR06oK2BAZHGecDUnKi7q2mbvp8yvp/LRAMriwY5OeHdjh3VVv7TZJSfH37p1g1tmgJWa+n+yued4CiT4bWDB3F8zRo0NDSYvE1CouCKEEIIsXPKp5PvqAk4poeGYl9CglHV7lRPWr/t0gV/xsZCJBLh9+7dcbt/f72ykaqnwKrbVz1RdBaL8X3XrpgQFASJSIQeHh56T5BsD/he5f+tWzeUDxigs3CD6va2x8ais6urxsmNNVENCvrymKPJkIzFx506AWCZHr5+iolBVzc3zNWj62ggz8+K0O/AZhOEG/H+Vlc2Xh3ltZbruFgh4bnNV5Tm07KFAkb6ouCKEEIIsXLK2QIvI7vuuVtgDiOxSGRUZUVTn35Z++md8gm1rhM3vmOuRCIRPHkUClDdXl8vL2QkJmqc3FgT1WMc7uKCrMREXOvXj/e++XixbVvUDByIhzVUMVR3Mt/N3R3pfftqrVipaklEBMYoHQMh3kOHEhLwe7duGNvUdnXzyQnVQU5d9UGgeeDT4r6ICJzt3Vvj/Y8HBqKrm5vWyZwrBw5EsFI2Up8582wFBVeEEEKICQh5wu4sFiO9Tx9c6NPHaiZ4PZiQgDh3dxxoKvct5BVokUhk9QGPKnO1NysxUev9XzZNqvxWU2U6fdvVQU2ZfUNP6DfqyB51dHVFhIuL4PMwuRj4GVE9Vn/FxsJNLMbPXbu2WNfH0RG/65m90yXZ2xtjAgKwvksX7IiNxedNr6WySB3TIGij/Do6auimubxDh2bvgSiVjGOshweu9eunNnPpJhYjvW9fxWTOiyMi0N/Lq9n2VC/uhLu4YF10NDYrjSWzdRRcEUIIITagq7s7YvSce0fOFMPB+3t743SfPhhgonLftjWE3bQG+/jAUSRCb09PRCqd7I5Vk50Z6eeH6oEDscTA4GpfU7A8NTgYAPC6EeXDe/Ho+gdobmMPE0zKq0/YNdLPDxUDB+JxLVURg5sq8Y1uymL9GhOD+4wsnuImkWCUnx+c1Vyw8HJwwI1+/VCQnKz3dpW7yfKdOuGp4GC81b59sznTIlxckKSmi6/qBZa3IyNxsGdPnQVIpoaEtJgv7aSBhT+sgf3l4gghhBBidkJW9GpRLVAkajnPlR2O1dDE08EBFQMHtujKtSwyEjvv3MHF6upmtxuT3WzblGX4OjoaC9u1Q2dXV/xcWIibdXUGb9NQ3iboMtbR1RWP+vtj8+3bWteTv790jds73bs3DpaVKYq2jA0MxNjAQIhSUoxuq6ZPVFsDs1cSkQgne/VCvUwGH57BlUQkUgTqyvwcHZGXlAQ3iQSPnD+PPaWleKYpIBdCtIEXkqwBZa4IIYQQE7CmU39bK2XMx1BfX8G2ZQuBmrNY3OJE30Ui0Ws+OH3eBWKRCNFubhCJRPi5a1fc6+ODXT166LEFwFUpY2Etx1gkEuG37t15F6TQJcjJCY8GBGjsZmdtenp6CjbNRbCzM7wcHLArLg53+vdHdxNkGm2RbbwTCCGEEGLVhD55Vg0IlQtifNqpE961srl7rCN0aEmIsLqTmxv+jY/HMD27u4U6O+Ot9u3xQceOcNLy/lDuPhihkpUx1XF9tmkutMEaurVaw+tpK5dExCKR1kqjz4SE6L9NsRgxMTGIiYmB2EYCVznqFkgIIYSYwNL27fFHcbFiQltLssaTtGWRkXg1O5vXuiIAO3v0wLMZGVjRsSOGCJS1mte2LT68eRMfduxo9LZUgwJD8C2PbUvkXcp+LSzUuM4nnTohxMkJTwYFwUMiweysLCwIDzdpu5ZGRGCQtzeSDJx3Sij9vLw0PldLfW6jXF1xtbZWsO3NadsW6/Pz1U4KrYmDgwPGjh0rWBvMiYIrQgghxAR6eXqiauBAuFlBdT9znKS1dXZGfn097/UXRUSgqKEBH9282eI+EVpmwnp6eiJVSxloQ7zfsSMWRUQYVSb+n7g4fJ+fj/8aMWHxppgYzM7KwiY7qpimD19HR7ynFOD+GRtr8n06isV6Z+KE5iwS4YiWwg2W6s67rksXLLx6FTMFujAkEYlwTsdk1faEgitCCCHERKwhsDKXX2JiMCszU+s8Oao05Wn8HR3NcmIpMnL+LQAY4utrdCbtscBAPKbH2Cllk4KDsfTaNY1j0ES4G1x7tqL3o7GEyiFay1gzfYQ6O+MHNeXnCT8UXBFCCCF2zhzXvzu6uuIvPQseqNrQtSu2FRdjRmgosmpqFLfb3ump+US4uKBiwABek0P/omPeqdbOFO8zeywmYw719fVYvnw5AGDRokVwaip5bwsouCKEEELsnKuNDAifEBSECVrmFCLqefAsWW7oPGnENHSFXRSW2Sbb+LYlhBBCiMHWRkcj2tUV67t0sXRTmvEXqBw2IUITqjufMdsJc3YWpA3EvChzRQghhNi5aDc3XEpMtHQzWpjdti1OVVZijL+/pZtityQiEWTUNc0q6XpVkr298WHHjohydTVLe4gwrCJztXr1arRv3x4uLi5ITEzE8ePHNa67fv16iESiZouLSvlTjuOwdOlShISEwNXVFUOHDkVmZqapnwYhhBBC9OAmkWBTt26YqKMroC0WBbAWu3r0QICjIzbReCuLiDSyRP/c8HA8SBcfbIrFg6uNGzdi7ty5eP3113Hq1CnExcVhxIgRKNQyH4KXlxfy8vIUy/Xr15vd/9577+GTTz7BmjVrcOzYMbi7u2PEiBGoFbBmPyGEEEKItRvs64uC5GSDqxFamq2G1cX9+yM3KQlePMfDEfth8eBq5cqVmD59OqZOnYqYmBisWbMGbm5uWLduncbHiEQiBAcHK5YgpSteHMdh1apVWLx4MUaPHo0ePXrg+++/R25uLrZu3WqGZ0QIIYQQY1FHNuFQ5s/82jg6IoTGTLVKFg2u6uvrcfLkSQwdOlRxm1gsxtChQ3HkyBGNj6usrERERATCw8MxevRoXLhwQXFfdnY28vPzm23T29sbiYmJGrdZV1eH8vLyZgshhBBCCLGsCCO71RHbJBaLERUVhaioKIhtpNqpnEVbe/v2bUil0maZJwAICgpCfn6+2sdER0dj3bp1+N///ocff/wRMpkMycnJuNk0w7v8cfpsc/ny5fD29lYs4eHhxj41QgghhBBipB+6dsVoPz/si4+3dFOIGTk4OOCJJ57AE088AQcb61ppW6EggKSkJEyaNAnx8fG45557sGXLFgQEBODLL780eJuLFi1CWVmZYrlx44aALSaEEEKIvgJtaNJQYjoRLi7YGhuLQT4+lm4KIbxYNBT09/eHRCJBQUFBs9sLCgoQHBzMaxuOjo5ISEhAVlYWACgeV1BQgJCQkGbbjNdw1cPZ2RnO1C+WEEIIsRpBTk7Y2aMH3G2sSxCxXeYem0bjCu2TRb+xnJyc0KtXL+zZs0dxm0wmw549e5CUlMRrG1KpFOfOnVMEUpGRkQgODm62zfLychw7doz3NgkhhBBieSPatMEAyljYhUcCAgAA4XQxW8HPxrq7mVN9fT2WLVuGZcuWob6+3tLN0YvFX9W5c+di8uTJ6N27N/r27YtVq1ahqqoKU6dOBQBMmjQJYWFhWL58OQDgrbfeQr9+/dCpUyeUlpbi/fffx/Xr1zFt2jQA7KrDnDlz8M477yAqKgqRkZFYsmQJQkNDMWbMGEs9TUIIIYSQVmte27aIdnVFf29vSzfF4vbExeGVq1fxZefOlm6KVWtoaLB0Ewxi8eBq/PjxKCoqwtKlS5Gfn4/4+Hjs3LlTUZAiJyenWZWQkpISTJ8+Hfn5+fD19UWvXr1w+PBhxChNjrdgwQJUVVXh2WefRWlpKQYMGICdO3e2mGyYEEIIIYSYnoNYjDFN2StrJTHTfv7j64vjvXqZaW/E3EQcx1GXTxXl5eXw9vZGWVkZvLy8LN0cQgghhBCTEaWkKH7nBg+2WDssTcZxeODcOYQ6OeHrLl0s3ZxWrb6+XtFrbdGiRXCycIEbfWIDi2euCCGEEEKI5XwbHY2pGRlY38oDCrFIhB09eli6GcTGUXBFCCGEENKKTQkJweOBgXCRmKtjHCH2i+qbEkIIIYS0chRYESIMylwRQgghhBBCrIZIJEJERITid1tCwRUhhBBCCCHEajg6OmLKlCmWboZBqFsgIYQQQgghhAiAgitCCCGEEEIIEQB1CySEEEIIIYRYjfr6enz88ccAgNmzZ1t8nit9UHBFCCGEEEIIsSrV1dWWboJBqFsgIYQQQgghhAiAgitCCCGEEEIIEQAFV4QQQgghhBAiAAquCCGEEEIIIUQAFFwRQgghhBBCiACoWiAhhBBCCCHEaohEIoSGhip+tyUUXBFCCCGEEEKshqOjI6ZPn27pZhiEgisjSKVSNDQ0WLoZxMo4OjpCIpFYuhmEEEIIIcTMKLgyAMdxyM/PR2lpqaWbQqyUj48PgoODbS6VTQghhBBCDEfBlQHkgVVgYCDc3NzoBJoocByH6upqFBYWAgBCQkIs3CJCCCGEENvS0NCA1atXAwBmzpwJR0dHC7eIPwqu9CSVShWBlZ+fn6WbQ6yQq6srAKCwsBCBgYHURZAQQgghRA8cx6GsrEzxuy2hUux6ko+xcnNzs3BLiDWTvz9oTB4hhBBCSOtBwZWBqCsg0YbeH4QQQgghrQ8FV4QQQgghhBAiAAquWpHBgwdjzpw5lm4GIYQQQgghdomCK0IIIYQQQggRAFULJIQQQgghhFgNkUiEgIAAxe+2hDJXAuA4DlVSqdkXY0pTlpSUYNKkSfD19YWbmxtGjRqFzMxMxf3Xr1/Hgw8+CF9fX7i7u6Nbt27YsWOH4rETJ05EQEAAXF1dERUVhW+//dbo40gIIYQQQoijoyOef/55PP/88zY1xxVgJZmr1atX4/3330d+fj7i4uLw6aefom/fvmrXXbt2Lb7//nucP38eANCrVy8sW7as2fpTpkzBd9991+xxI0aMwM6dO03S/mqZDB4HDphk29pUDhwIdwPnUJoyZQoyMzPxxx9/wMvLCwsXLsR9992H9PR0ODo6YubMmaivr8f+/fvh7u6O9PR0eHh4AACWLFmC9PR0/PXXX/D390dWVhZqamqEfGqEEEIIIYTYHIsHVxs3bsTcuXOxZs0aJCYmYtWqVRgxYgQyMjIQGBjYYv2UlBRMmDABycnJcHFxwYoVKzB8+HBcuHABYWFhivVGjhzZLJvi7OxsludjC+RB1aFDh5CcnAwA+OmnnxAeHo6tW7di7NixyMnJwaOPPorY2FgAQIcOHRSPz8nJQUJCAnr37g0AaN++vdmfAyGEEEIIIdbG4sHVypUrMX36dEydOhUAsGbNGmzfvh3r1q3DK6+80mL9n376qdnfX3/9NTZv3ow9e/Zg0qRJitudnZ0RHBxs2sY3cROLUTlwoFn2pbpfQ1y8eBEODg5ITExU3Obn54fo6GhcvHgRAPDiiy9ixowZ2LVrF4YOHYpHH30UPXr0AADMmDEDjz76KE6dOoXhw4djzJgxiiCNEEIIIYQQYzQ0NGDt2rUAgOnTp9tU10CLjrmqr6/HyZMnMXToUMVtYrEYQ4cOxZEjR3hto7q6Gg0NDWjTpk2z21NSUhAYGIjo6GjMmDEDxcXFGrdRV1eH8vLyZos+RCIR3CUSsy+mHOA3bdo0XL16FU899RTOnTuH3r1749NPPwUAjBo1CtevX8dLL72E3NxcDBkyBPPnzzdZWwghhBBCSOvBcRyKiopQVFRkVI0BS7BocHX79m1IpVIEBQU1uz0oKAj5+fm8trFw4UKEhoY2C9BGjhyJ77//Hnv27MGKFSuwb98+jBo1ClKpVO02li9fDm9vb8USHh5u+JOyAV27dkVjYyOOHTumuK24uBgZGRmIiYlR3BYeHo7nnnsOW7Zswbx58xRXEAAgICAAkydPxo8//ohVq1bhq6++MutzIIQQQgghxNpYvFugMd5991388ssvSElJgYuLi+L2xx9/XPF7bGwsevTogY4dOyIlJQVDhgxpsZ1FixZh7ty5ir/Ly8vtOsCKiorC6NGjMX36dHz55Zfw9PTEK6+8grCwMIwePRoAMGfOHIwaNQqdO3dGSUkJ9u7di65duwIAli5dil69eqFbt26oq6vDtm3bFPcRQgghhBDSWlk0c+Xv7w+JRIKCgoJmtxcUFOgcL/XBBx/g3Xffxa5duxRjgTTp0KGDoqqdOs7OzvDy8mq22Ltvv/0WvXr1wgMPPICkpCRwHIcdO3Yo+rRKpVLMnDkTXbt2xciRI9G5c2d8/vnnAAAnJycsWrQIPXr0wKBBgyCRSPDLL79Y8ukQQgghhBBicRbNXDk5OaFXr17Ys2cPxowZAwCQyWTYs2cPZs2apfFx7733Hv773//i77//VlSs0+bmzZsoLi5GSEiIUE23SSkpKYrffX198f3332tcVz6+Sp3Fixdj8eLFQjaNEEIIIYQQm2fxSYTnzp2LtWvX4rvvvsPFixcxY8YMVFVVKaoHTpo0CYsWLVKsv2LFCixZsgTr1q1D+/btkZ+fj/z8fFRWVgIAKisr8fLLL+Po0aO4du0a9uzZg9GjR6NTp04YMWKERZ4jIYQQQgghxP5ZfMzV+PHjUVRUhKVLlyI/Px/x8fHYuXOnoshFTk4OxEolx7/44gvU19fjsccea7ad119/HW+88QYkEgnOnj2L7777DqWlpQgNDcXw4cPx9ttv01xXhBBCCCGEWDmRSARvb2/F77ZExNlafUMzKC8vh7e3N8rKylqMv6qtrUV2djYiIyObFdEgRBm9TwghhBBC7IO22ECVxbsFEkIIIYQQQog9oOCKEEIIIYQQQgRg8TFXhBBCCCGEECLX0NCA9evXAwCmTJmimCrIFlBwRQghhBBCCLEaHMchNzdX8bstoW6BhBBCCCGEECIACq4IIYQQQgghRAAUXBG7lZKSApFIhNLSUks3hRBCCCGEtAIUXBFCCCGEEEKIACi4IiYjlUohk8ks3QxCCCGEEELMgoIrAXAch6qqKrMv+lRP2blzJwYMGAAfHx/4+fnhgQcewJUrVxT3JycnY+HChc0eU1RUBEdHR+zfvx8AUFdXh/nz5yMsLAzu7u5ITExESkqKYv3169fDx8cHf/zxB2JiYuDs7IycnBycOHECw4YNg7+/P7y9vXHPPffg1KlTzfZ16dIlDBgwAC4uLoiJicE///wDkUiErVu3Kta5ceMGxo0bBx8fH7Rp0wajR4/GtWvX+L9QADZv3oxu3brB2dkZ7du3x4cfftjs/s8//xxRUVFwcXFBUFAQHnvsMcV9v/32G2JjY+Hq6go/Pz8MHToUVVVVeu2fEEIIIYTo5ubmBjc3N0s3Q29Uil0A1dXV8PDwMPt+Kysr4e7uzmvdqqoqzJ07Fz169EBlZSWWLl2Khx9+GKdPn4ZYLMbEiRPx3nvv4d1334VIJAIAbNy4EaGhoRg4cCAAYNasWUhPT8cvv/yC0NBQ/P777xg5ciTOnTuHqKgoAOxYrFixAl9//TX8/PwQGBiIq1evYvLkyfj000/BcRw+/PBD3HfffcjMzISnpyekUinGjBmDdu3a4dixY6ioqMC8efOatb+hoQEjRoxAUlISDhw4AAcHB7zzzjsYOXIkzp49CycnJ53H4OTJkxg3bhzeeOMNjB8/HocPH8bzzz8PPz8/TJkyBampqXjxxRfxww8/IDk5GXfu3MGBAwcAAHl5eZgwYQLee+89PPzww6ioqMCBAwdsrjwoIYQQQoi1c3Jywssvv2zpZhiGIy2UlZVxALiysrIW99XU1HDp6elcTU2N4rbKykoOgNmXyspKg59jUVERB4A7d+4cx3EcV1hYyDk4OHD79+9XrJOUlMQtXLiQ4ziOu379OieRSLhbt241286QIUO4RYsWcRzHcd9++y0HgDt9+rTWfUulUs7T05P7888/OY7juL/++otzcHDg8vLyFOvs3r2bA8D9/vvvHMdx3A8//MBFR0dzMplMsU5dXR3n6urK/f3332r3s3fvXg4AV1JSwnEcxz3xxBPcsGHDmq3z8ssvczExMRzHcdzmzZs5Ly8vrry8vMW2Tp48yQHgrl27pvW5yal7nxBCCCGEENujLTZQRZkrAbi5uaGystIi++UrMzMTS5cuxbFjx3D79m3FWKicnBx0794dAQEBGD58OH766ScMHDgQ2dnZOHLkCL788ksAwLlz5yCVStG5c+dm262rq4Ofn5/ibycnJ/To0aPZOgUFBVi8eDFSUlJQWFgIqVSK6upq5OTkAAAyMjIQHh6O4OBgxWP69u3bbBtnzpxBVlYWPD09m91eW1vbrHujNhcvXsTo0aOb3da/f3+sWrUKUqkUw4YNQ0REBDp06ICRI0di5MiRePjhh+Hm5oa4uDgMGTIEsbGxGDFiBIYPH47HHnsMvr6+vPZNCCGEEELsHwVXAhCJRLy751nKgw8+iIiICKxduxahoaGQyWTo3r076uvrFetMnDgRL774Ij799FNs2LABsbGxiI2NBcC6IEokEpw8eRISiaTZtpW7RLq6uiq6FcpNnjwZxcXF+PjjjxEREQFnZ2ckJSU127culZWV6NWrF3766acW9wUEBPDejjaenp44deoUUlJSsGvXLixduhRvvPEGTpw4AR8fH+zevRuHDx/Grl278Omnn+K1117DsWPHEBkZKcj+CSGEEEIIGw4iP+ebOHEiHB0dLdwi/qigRStQXFyMjIwMLF68GEOGDEHXrl1RUlLSYr3Ro0ejtrYWO3fuxIYNGzBx4kTFfQkJCZBKpSgsLESnTp2aLcoZJ3UOHTqEF198Effdd5+imMTt27cV90dHR+PGjRsoKChQ3HbixIlm2+jZsycyMzMRGBjYYv/e3t68jkPXrl1x6NChFm3r3LmzImB0cHDA0KFD8d577+Hs2bO4du0a/v33XwAsiO7fvz/efPNNpKWlwcnJCb///juvfRNCCCGEEH44jsP169dx/fp1mxvfTsFVK+Dr6ws/Pz989dVXyMrKwr///ou5c+e2WM/d3R1jxozBkiVLcPHiRUyYMEFxX+fOnTFx4kRMmjQJW7ZsQXZ2No4fP47ly5dj+/btWvcfFRWFH374ARcvXsSxY8cwceJEuLq6Ku4fNmwYOnbsiMmTJ+Ps2bM4dOgQFi9eDACKLNjEiRPh7++P0aNH48CBA8jOzkZKSgpefPFF3Lx5k9dxmDdvHvbs2YO3334bly9fxnfffYfPPvsM8+fPBwBs27YNn3zyCU6fPo3r16/j+++/h0wmQ3R0NI4dO4Zly5YhNTUVOTk52LJlC4qKitC1a1de+ybk/9u796CorjsO4F9Y2F0oWVZBXgaERAQCKCKKCKmTui01TFqaTkGHCEZTm8Q0+IgaQbHVpDCmdFoTI0knlWaaSLFV2ig1JfiqFFEIGFGKWG3IZLKQBHkoCsj++ofDjdcHEbOwgN/PzI7rOb+9ew77G9jfnHvPJSIiotGPxdU9wN7eHgUFBaiqqkJYWBiWL1+OV1555ZaxKSkpOHHiBB5++GH4+fmp+rZv347U1FSsXLkSQUFBSExMxPHjx2+Ku9Fbb72FCxcuIDIyEgsWLMDzzz8PDw8PpV+j0aCoqAgXL17E9OnT8dRTTyEzMxMAoNfrAVy7vuzw4cPw8/PD448/jpCQECxevBhXrlyBwWC4o59DZGQkCgsLUVBQgLCwMGRlZWHjxo1YuHAhAMBoNGLXrl34zne+g5CQEOTl5WHHjh0IDQ2FwWDA4cOH8eijj2LSpElYt24dcnNzMXfu3Dt6byIiIiIa/exkpK21DYH29na4urqira3tpi/uV65cwfnz5xEQEKB88SfrKysrQ1xcHM6ePYsHH3zQ1sMZMOYJERER0d3p7u5GdnY2AGDt2rV3dMudwdRfbXAjbmhBw8Lu3bvh4uKCwMBAnD17Funp6YiNjR2RhRURERER3ZtYXNGw0NHRgTVr1qCxsRHu7u4wmUzIzc219bCIiIiIiO4YiysaFlJTU5GammrrYRARERHRMDCStl+/HosrIiIiIiIaNrRaLTIyMmw9jLvC3QLvEvcBof4wP4iIiIjuPSyuBqhvibKzs9PGI6HhrC8/RuqSNhERERENHE8LHCCNRgOj0Yjm5mYA1+6/1HejWyIRQWdnJ5qbm2E0GqHRaGw9JCIiIqIR5erVqygsLAQAJCUlwcFh5JQsI2ekw4iXlxcAKAUW0Y2MRqOSJ0RERER05ywWCxoaGpTnIwmLq7tgZ2cHb29veHh4oKenx9bDoWHG0dGRK1ZERERE9yAWV9+ARqPhl2giIiIiIgIwTDa02Lp1K/z9/aHX6xEdHY1jx471G79z504EBwdDr9cjPDwcxcXFqn4RQVZWFry9veHk5ASTyaQsLRIREREREQ0GmxdXf/7zn7FixQps2LABH374IaZMmYL4+PjbXs/073//G/Pnz8fixYtRXV2NxMREJCYmora2VonZvHkztmzZgry8PFRUVOBb3/oW4uPjceXKlaGaFhERERER3WPsxMY35ImOjsb06dPx2muvAbh20Zqvry9+/vOf48UXX7wpPjk5GZcuXcKePXuUtpkzZyIiIgJ5eXkQEfj4+GDlypV44YUXAABtbW3w9PREfn4+5s2b97Vjam9vh6urK9ra2mAwGKw0UyIiIiIi+jrd3d3Izs4GAKxduxZardam4xlIbWDTa666u7tRVVWFtWvXKm329vYwmUwoLy+/5WvKy8uxYsUKVVt8fDyKiooAAOfPn4fZbIbJZFL6XV1dER0djfLy8lsWV11dXejq6lL+39bWBuDaD5KIiIiIiIZOd3e3csZZe3v7sCiugGuXHn0dmxZXX3zxBXp7e+Hp6alq9/T0xH/+859bvsZsNt8y3mw2K/19bbeLuVF2djZ++ctf3tTu6+t7ZxMhIiIiIiKry8nJsfUQFB0dHXB1de03hrsF4tpy4/WrYRaLBS0tLXBzc7P5DYLb29vh6+uLTz75hKco0h1hztBAMWdooJgzNFDMGRqo4ZQzIoKOjg74+Ph8baxNiyt3d3doNBo0NTWp2puamm57A1YvL69+4/v+bWpqgre3tyomIiLilsfU6XTQ6XSqNqPROJCpDDqDwWDzxKKRhTlDA8WcoYFiztBAMWdooIZLznzdilUfm+4WqNVqMW3aNJSWliptFosFpaWliImJueVrYmJiVPEAUFJSosQHBATAy8tLFdPe3o6KiorbHpOIiIiIiOibsvlpgStWrEBaWhqioqIwY8YM/Pa3v8WlS5fw5JNPAgBSU1Mxfvx4ZceQ9PR0zJ49G7m5uUhISEBBQQEqKyvx5ptvAgDs7OywbNkyvPTSSwgMDERAQADWr18PHx8fJCYm2mqaREREREQ0ytm8uEpOTsbnn3+OrKwsmM1mREREYN++fcqGFI2NjbC3/2qBbdasWXj33Xexbt06ZGRkIDAwEEVFRQgLC1NiVq9ejUuXLmHJkiVobW1FXFwc9u3bB71eP+Tz+6Z0Oh02bNhw02mLRLfDnKGBYs7QQDFnaKCYMzRQIzVnbH6fKyIiIiIiotHAptdcERERERERjRYsroiIiIiIiKyAxRUREREREZEVsLgiIiIiIiKyAhZXw9zWrVvh7+8PvV6P6OhoHDt2zNZDoiGQnZ2N6dOn47777oOHhwcSExNRX1+virly5QqWLl0KNzc3uLi44Mc//vFNN9hubGxEQkICnJ2d4eHhgVWrVuHq1auqmIMHDyIyMhI6nQ4TJ05Efn7+YE+PBllOTo5yW4o+zBe60aeffoonnngCbm5ucHJyQnh4OCorK5V+EUFWVha8vb3h5OQEk8mEhoYG1TFaWlqQkpICg8EAo9GIxYsX4+LFi6qYjz76CA8//DD0ej18fX2xefPmIZkfWVdvby/Wr1+PgIAAODk54cEHH8SmTZtw/b5ozJl72+HDh/HYY4/Bx8cHdnZ2KCoqUvUPZX7s3LkTwcHB0Ov1CA8PR3FxsdXne1tCw1ZBQYFotVr5wx/+IKdOnZKf/vSnYjQapampydZDo0EWHx8v27dvl9raWqmpqZFHH31U/Pz85OLFi0rM008/Lb6+vlJaWiqVlZUyc+ZMmTVrltJ/9epVCQsLE5PJJNXV1VJcXCzu7u6ydu1aJebcuXPi7OwsK1askNOnT8urr74qGo1G9u3bN6TzJes5duyY+Pv7y+TJkyU9PV1pZ77Q9VpaWmTChAmycOFCqaiokHPnzsn7778vZ8+eVWJycnLE1dVVioqK5MSJE/KDH/xAAgIC5PLly0rM97//fZkyZYocPXpU/vWvf8nEiRNl/vz5Sn9bW5t4enpKSkqK1NbWyo4dO8TJyUneeOONIZ0vfXMvv/yyuLm5yZ49e+T8+fOyc+dOcXFxkd/97ndKDHPm3lZcXCyZmZmya9cuASC7d+9W9Q9VfpSVlYlGo5HNmzfL6dOnZd26deLo6CgnT54c9J+BiAiLq2FsxowZsnTpUuX/vb294uPjI9nZ2TYcFdlCc3OzAJBDhw6JiEhra6s4OjrKzp07lZi6ujoBIOXl5SJy7Zecvb29mM1mJWbbtm1iMBikq6tLRERWr14toaGhqvdKTk6W+Pj4wZ4SDYKOjg4JDAyUkpISmT17tlJcMV/oRmvWrJG4uLjb9lssFvHy8pJXXnlFaWttbRWdTic7duwQEZHTp08LADl+/LgS849//EPs7Ozk008/FRGR119/XcaMGaPkUN97BwUFWXtKNMgSEhJk0aJFqrbHH39cUlJSRIQ5Q2o3FldDmR9JSUmSkJCgGk90dLT87Gc/s+ocb4enBQ5T3d3dqKqqgslkUtrs7e1hMplQXl5uw5GRLbS1tQEAxo4dCwCoqqpCT0+PKj+Cg4Ph5+en5Ed5eTnCw8OVG3IDQHx8PNrb23Hq1Ckl5vpj9MUwx0ampUuXIiEh4abPlPlCN/r73/+OqKgo/OQnP4GHhwemTp2K3//+90r/+fPnYTabVZ+3q6sroqOjVTljNBoRFRWlxJhMJtjb26OiokKJ+fa3vw2tVqvExMfHo76+HhcuXBjsaZIVzZo1C6WlpThz5gwA4MSJEzhy5Ajmzp0LgDlD/RvK/LD13yoWV8PUF198gd7eXtUXHQDw9PSE2Wy20ajIFiwWC5YtW4bY2FiEhYUBAMxmM7RaLYxGoyr2+vwwm823zJ++vv5i2tvbcfny5cGYDg2SgoICfPjhh8jOzr6pj/lCNzp37hy2bduGwMBAvP/++3jmmWfw/PPP449//COArz7z/v4Gmc1meHh4qPodHBwwduzYAeUVjQwvvvgi5s2bh+DgYDg6OmLq1KlYtmwZUlJSADBnqH9DmR+3ixmq/HEYknchoru2dOlS1NbW4siRI7YeCg1Tn3zyCdLT01FSUgK9Xm/r4dAIYLFYEBUVhV/96lcAgKlTp6K2thZ5eXlIS0uz8ehoOCosLMQ777yDd999F6GhoaipqcGyZcvg4+PDnCG6Dleuhil3d3doNJqbdvNqamqCl5eXjUZFQ+25557Dnj17cODAAdx///1Ku5eXF7q7u9Ha2qqKvz4/vLy8bpk/fX39xRgMBjg5OVl7OjRIqqqq0NzcjMjISDg4OMDBwQGHDh3Cli1b4ODgAE9PT+YLqXh7e+Ohhx5StYWEhKCxsRHAV595f3+DvLy80NzcrOq/evUqWlpaBpRXNDKsWrVKWb0KDw/HggULsHz5cmW1nDlD/RnK/LhdzFDlD4urYUqr1WLatGkoLS1V2iwWC0pLSxETE2PDkdFQEBE899xz2L17N/bv34+AgABV/7Rp0+Do6KjKj/r6ejQ2Nir5ERMTg5MnT6p+UZWUlMBgMChfqmJiYlTH6Ithjo0sc+bMwcmTJ1FTU6M8oqKikJKSojxnvtD1YmNjb7q9w5kzZzBhwgQAQEBAALy8vFSfd3t7OyoqKlQ509raiqqqKiVm//79sFgsiI6OVmIOHz6Mnp4eJaakpARBQUEYM2bMoM2PrK+zsxP29uqvjRqNBhaLBQBzhvo3lPlh879VQ7JtBt2VgoIC0el0kp+fL6dPn5YlS5aI0WhU7eZFo9Mzzzwjrq6ucvDgQfnss8+UR2dnpxLz9NNPi5+fn+zfv18qKyslJiZGYmJilP6+rbW/973vSU1Njezbt0/GjRt3y621V61aJXV1dbJ161ZurT1KXL9boAjzhdSOHTsmDg4O8vLLL0tDQ4O888474uzsLH/605+UmJycHDEajfK3v/1NPvroI/nhD394y22Tp06dKhUVFXLkyBEJDAxUbZvc2toqnp6esmDBAqmtrZWCggJxdnbmttojUFpamowfP17Zin3Xrl3i7u4uq1evVmKYM/e2jo4Oqa6ulurqagEgv/nNb6S6ulo+/vhjERm6/CgrKxMHBwf59a9/LXV1dbJhwwZuxU5fefXVV8XPz0+0Wq3MmDFDjh49aush0RAAcMvH9u3blZjLly/Ls88+K2PGjBFnZ2f50Y9+JJ999pnqOP/73/9k7ty54uTkJO7u7rJy5Urp6elRxRw4cEAiIiJEq9XKAw88oHoPGrluLK6YL3Sj9957T8LCwkSn00lwcLC8+eabqn6LxSLr168XT09P0el0MmfOHKmvr1fFfPnllzJ//nxxcXERg8EgTz75pHR0dKhiTpw4IXFxcaLT6WT8+PGSk5Mz6HMj62tvb5f09HTx8/MTvV4vDzzwgGRmZqq2xGbO3NsOHDhwy+8uaWlpIjK0+VFYWCiTJk0SrVYroaGhsnfv3kGb943sRK67tTYRERERERHdFV5zRUREREREZAUsroiIiIiIiKyAxRUREREREZEVsLgiIiIiIiKyAhZXREREREREVsDiioiIiIiIyApYXBEREREREVkBiysiIiIiIiIrYHFFRERkRQcPHoSdnR1aW1ttPRQiIhpiLK6IiIiIiIisgMUVERERERGRFbC4IiKiUcVisSA7OxsBAQFwcnLClClT8Je//AXAV6fs7d27F5MnT4Zer8fMmTNRW1urOsZf//pXhIaGQqfTwd/fH7m5uar+rq4urFmzBr6+vtDpdJg4cSLeeustVUxVVRWioqLg7OyMWbNmob6+fnAnTkRENsfiioiIRpXs7Gy8/fbbyMvLw6lTp7B8+XI88cQTOHTokBKzatUq5Obm4vjx4xg3bhwee+wx9PT0ALhWFCUlJWHevHk4efIkfvGLX2D9+vXIz89XXp+amoodO3Zgy5YtqKurwxtvvAEXFxfVODIzM5Gbm4vKyko4ODhg0aJFQzJ/IiKyHTsREVsPgoiIyBq6urowduxYfPDBB4iJiVHan3rqKXR2dmLJkiV45JFHUFBQgOTkZABAS0sL7r//fuTn5yMpKQkpKSn4/PPP8c9//lN5/erVq7F3716cOnUKZ86cQVBQEEpKSmAymW4aw8GDB/HII4/ggw8+wJw5cwAAxcXFSEhIwOXLl6HX6wf5p0BERLbClSsiIho1zp49i87OTnz3u9+Fi4uL8nj77bfx3//+V4m7vvAaO3YsgoKCUFdXBwCoq6tDbGys6rixsbFoaGhAb28vampqoNFoMHv27H7HMnnyZOW5t7c3AKC5ufkbz5GIiIYvB1sPgIiIyFouXrwIANi7dy/Gjx+v6tPpdKoC6245OTndUZyjo6Py3M7ODsC168GIiGj04soVERGNGg899BB0Oh0aGxsxceJE1cPX11eJO3r0qPL8woULOHPmDEJCQgAAISEhKCsrUx23rKwMkyZNgkajQXh4OCwWi+oaLiIiIoArV0RENIrcd999eOGFF7B8+XJYLBbExcWhra0NZWVlMBgMmDBhAgBg48aNcHNzg6enJzIzM+Hu7o7ExEQAwMqVKzF9+nRs2rQJycnJKC8vx2uvvYbXX38dAODv74+0tDQsWrQIW7ZswZQpU/Dxxx+jubkZSUlJtpo6ERENAyyuiIhoVNm0aRPGjRuH7OxsnDt3DkajEZGRkcjIyFBOy8vJyUF6ejoaGhoQERGB9957D1qtFgAQGRmJwsJCZGVlYdOmTfD29sbGjRuxcOFC5T22bduGjIwMPPvss/jyyy/h5+eHjIwMW0yXiIiGEe4WSERE94y+nfwuXLgAo9Fo6+EQEdEow2uuiIiIiIiIrIDFFRERERERkRXwtEAiIiIiIiIr4MoVERERERGRFbC4IiIiIiIisgIWV0RERERERFbA4oqIiIiIiMgKWFwRERERERFZAYsrIiIiIiIiK2BxRUREREREZAUsroiIiIiIiKzg/6GwJougOZzeAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "average_losses = torch.tensor(losses).view(-1, 100).mean(1)\n",
    "plt.figure(figsize=(10, 4))\n",
    "plt.plot(losses, c='c', label='loss')\n",
    "plt.plot(range(0, len(losses), 100), average_losses, c='k', label='average loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('loss')\n",
    "plt.legend()\n",
    "plt.ylim(0, 2)\n",
    "plt.axvline(10000, c='gray', linestyle='--');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, training is really slow due to recurrency not letting us use vectorized operations. This is the main reason why, despite of it great effectiveness, RNN/LSTMs are not being used in practice that much anymore. Let's revisit the confusion matrix as in the previous notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.53%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbQAAAG0CAYAAABaNNJGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmxElEQVR4nO3df1RVdaL38c/h15EEMQWFFEWNTCsVNRUTpRwnq5keq9WTXicmx8Lp+gMxs6jMxK54m/yR3Sn1moo1YzOZUz1ls0xvaKlRmqJl4w9EIaWhskBSD8o5zx9dz3gCFY7A3ufb+7XWXouz9z77fGAdzofv3pu9HR6PxyMAAAJckNUBAABoCBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIIVYHaGxut1tHjx5VZGSkHA6H1XEAAPXg8Xh0/PhxXXHFFQoKuvAYzPhCO3r0qOLj462OAQC4BCUlJWrfvv0F1zG+0CIjIyVJN62+TyHNwyxOUz9n/u8PVkfwi+dUldUR/BYUH2d1BL94viqzOoJf3AH6XjlxW5LVEfwW+cE+qyPUyxlPlTaW/8X7WX4hxhfa2d2MIc3DFBpghSbHaasT+MXjCNzLgwYFO62O4BePI8De2//LHaDvlZDQZlZH8FtIgL5X6nLIiJNCAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGCJhCS0hI0IIFC6yOAQCwqYApNAAALsQ2hZaamqoJEyZowoQJioqKUnR0tKZPny6Px6PU1FQdPnxYmZmZcjgcdboVNwDg5yXE6gDnys3N1dixY/Xxxx9r27ZtSk9PV4cOHbRmzRr17NlT6enpeuCBBy64DZfLJZfL5X1cUVHR2LEBADZgq0KLj4/X/Pnz5XA41LVrV+3evVvz58/XAw88oODgYEVGRio2NvaC28jJydHMmTObKDEAwC5ss8tRkgYMGOCzOzE5OVn79+9XdXV1nbeRlZWl8vJy71RSUtIYUQEANmOrEVpDcDqdcjqdVscAADQxW43Q8vPzfR5/9NFHSkxMVHBwsMLCwuo1UgMA/LzYqtCKi4s1ZcoU7d27V6tWrdLzzz+vjIwMST/+H9qmTZt05MgRffPNNxYnBQDYja12OaalpenkyZPq16+fgoODlZGRofT0dElSdna2xo0bpy5dusjlcsnj8VicFgBgJ7YqtNDQUC1YsEAvvvhijWUDBgxQQUGBBakAAIHAVrscAQDwF4UGADCCbXY55uXlWR0BABDAGKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMYJuLEze207d/J48j1OoY9TJn3wdWR/BLVrchVkfwW/XBYqsj+CWkbYzVEfziSGhhdQS/NH97h9UR/Oa2OkA9uT2n67wuIzQAgBEoNACAESg0AIARKDQAgBEoNACAESg0AIARKDQAgBEoNACAESg0AIARKDQAgBEoNACAESg0AIARKDQAgBEoNACAESg0AIARKDQAgBEoNACAESg0AIARbFdoqampmjx5stUxAAABxnaFBgCAPyg0AIARLC20H374QWlpaYqIiFBcXJzmzp3rs9zlcmnq1Klq166dmjdvrv79+ysvL8+asAAAW7O00B5++GFt3LhRb775ptatW6e8vDx9+umn3uUTJkzQ1q1b9eqrr2rXrl26++67NXz4cO3fv/+823S5XKqoqPCZAADms6zQKisr9dJLL+nZZ5/V0KFDdd111yk3N1dnzpyRJBUXF2v58uV67bXXlJKSoi5dumjq1KkaNGiQli9fft7t5uTkKCoqyjvFx8c31bcEALBQiFUvXFhYqKqqKvXv3987r1WrVurataskaffu3aqurtZVV13l8zyXy6XWrVufd7tZWVmaMmWK93FFRQWlBgA/A5YV2sVUVlYqODhY27dvV3BwsM+yiIiI8z7P6XTK6XQ2djwAgM1YVmhdunRRaGio8vPz1aFDB0nSd999p3379mnIkCFKSkpSdXW1ysrKlJKSYlVMAECAsKzQIiIiNHbsWD388MNq3bq12rRpo8cff1xBQT8e1rvqqqs0evRopaWlae7cuUpKStLXX3+tDRs2qEePHrrtttusig4AsCFLdzn+4Q9/UGVlpX79618rMjJSDz30kMrLy73Lly9frqeffloPPfSQjhw5oujoaA0YMEC/+tWvLEwNALAjh8fj8VgdojFVVFQoKipKN4berRBHqNVx6mXOvg+sjuCXrG5DrI7gN3fVaasj+CWkbYzVEfziubyF1RH84t5/yOoIPxtnPKf1/unXVF5erhYtLvx+4UohAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1h6g88mFeSQHA6rU9TLY9ffanUEv7x7cIPVEfx2S+INVkfwT1Bg/m1a/Y9CqyP4x11tdQL/BQVbnaBePJ66/6wD87cAAICfoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARgioQktISNCCBQusjgEAsKGAKjQAAM6HQgMAGCHE6gDnSk1N1bXXXitJevnllxUaGqoHH3xQ2dnZcjgcFqcDANiZ7UZoubm5CgkJ0ccff6znnntO8+bN09KlS+v8fJfLpYqKCp8JAGA+2xVafHy85s+fr65du2r06NGaOHGi5s+fX+fn5+TkKCoqyjvFx8c3YloAgF3YrtAGDBjgs3sxOTlZ+/fvV3V1dZ2en5WVpfLycu9UUlLSWFEBADZiq2NoDcHpdMrpdFodAwDQxGw3QsvPz/d5/NFHHykxMVHBwcEWJQIABALbFVpxcbGmTJmivXv3atWqVXr++eeVkZFhdSwAgM3ZbpdjWlqaTp48qX79+ik4OFgZGRlKT0+3OhYAwOZsV2ihoaFasGCBXnzxxRrLDh061PSBAAABwXa7HAEA8AeFBgAwgq12Oebl5VkdAQAQoBihAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMYKur7Tem4LbRCg5yWh2jXjwVlVZH8MvNV/SyOoLf7thzyOoIfnkjqb3VEfziCA3Mj6DgmFirI/jNEx5gn4PVLqmwbusyQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYwdaFlpqaqsmTJ1sdAwAQAGx9//M1a9YoNDRUkpSQkKDJkydTcACAWtm60Fq1amV1BABAgAiIXY6pqak6fPiwMjMz5XA45HA4rI4GALAZWxfaWWvWrFH79u2VnZ2t0tJSlZaWnnddl8uliooKnwkAYL6AKLRWrVopODhYkZGRio2NVWxs7HnXzcnJUVRUlHeKj49vwqQAAKsERKHVR1ZWlsrLy71TSUmJ1ZEAAE3A1ieF+MPpdMrpdFodAwDQxAJmhBYWFqbq6mqrYwAAbCpgCi0hIUGbNm3SkSNH9M0331gdBwBgM3Xe5XjnnXfWeaNr1qzxK8yFZGdna9y4cerSpYtcLpc8Hk+DvwYAIHDVudCioqIaM0et8vLyvF8PGDBABQUFTZ4BABAY6lxoy5cvb8wcAABckoA5hgYAwIX4fdr+6tWr9de//lXFxcWqqqryWfbpp59ecjAAAOrDrxHawoULNWbMGLVt21Y7duxQv3791Lp1ax08eFC33HJLQ2cEAOCi/Cq0F154QUuWLNHzzz+vsLAwTZs2Te+9954mTZqk8vLyhs4IAMBF+VVoxcXFGjhwoCQpPDxcx48flyTde++9WrVqVcOlAwCgjvwqtNjYWB07dkyS1KFDB3300UeSpKKiIv4/DABgCb8K7aabbtJbb70lSRozZowyMzM1bNgw3XPPPbrjjjsaNCAAAHXh11mOS5YskdvtliSNHz9erVu31pYtW3T77bdr3LhxDRoQAIC68KvQgoKCFBT0r8HdyJEjNXLkyAYLBQBAfflVaJs2bbrg8sGDB/sVBgAAf/lVaKmpqTXmORwO79fc5gUA0NT8Oinku+++85nKysr097//Xddff73WrVvX0BkBALgov0ZotV15f9iwYQoLC9OUKVO0ffv2Sw4GAEB9NOjFidu2bau9e/c25CYBAKgTv0Zou3bt8nns8XhUWlqqOXPmqFevXg2RCwCAevGr0Hr16iWHw1HjqiADBgzQsmXLGiRYQ6v+5zdyOEKtjlE/115pdQK/BP2j6uIr2dSb1ze3OoJf+n/8vdUR/PJJSmurI/jlzNGvrI7gt+Cuna2OUD+O4Dqv6lehFRUV+TwOCgpSTEyMmjVr5s/mAAC4ZH4dQ9u4caNiY2PVsWNHdezYUfHx8WrWrJmqqqq0cuXKhs4IAMBF+VVoY8aMqfU2McePH9eYMWMuORQAAPXlV6F5PB6ff6Q+68svv6z1lH4AABpbvY6hJSUlyeFwyOFwaOjQoQoJ+dfTq6urVVRUpOHDhzd4SAAALqZehTZixAhJ0s6dO3XzzTcrIiLCuywsLEwJCQm66667GjQgAAB1Ua9CmzFjhiQpISFBI0eOlNPpbJRQAADUl1/H0Lp3766dO3fWmJ+fn69t27ZdaiYAAOrNr0IbP368SkpKasw/cuSIxo8ff8mhAACoL78Kbc+ePerdu3eN+UlJSdqzZ88lhwIAoL78KjSn06l//vOfNeaXlpb6nPkIAEBT8avQfvnLXyorK8vnn6u///57PfbYYxo2bFiDhQMAoK78Gk49++yzGjx4sDp27KikpCRJP57K37ZtW7388ssNGhAAgLrwq9DatWunXbt26U9/+pMKCgoUHh6uMWPGaNSoUQoNDbAr2gMAjOD3Aa/mzZtr0KBB6tChg6qqfrxdyLvvvitJuv322xsmHQAAdeRXoR08eFB33HGHdu/e7b0v2rnXdqyurm6wgAAA1IVfJ4VkZGSoU6dOKisr02WXXabPPvtMGzduVN++fZWXl9fAEQEAuDi/Rmhbt27V//zP/yg6OlpBQUEKDg7WoEGDlJOTo0mTJmnHjh0NnRMAgAvya4RWXV2tyMhISVJ0dLSOHj0qSerYsaP27t3bcOkAAKgjv0Zo1157rQoKCtSpUyf1799fzzzzjMLCwrRkyRJ17ty5oTMCAHBRfo3QnnjiCbndbklSdna2ioqKlJKSorVr12rhwoV13o7b7VZOTo46deqk8PBw9ezZU6tXr5Yk5eXlyeFwaMOGDerbt68uu+wyDRw4kBEgAKBWfo3Qbr75Zu/XV155pf7xj3/o2LFjuvzyy2u9k/X55OTk6JVXXtGiRYuUmJioTZs26Te/+Y1iYmK86zz++OOaO3euYmJi9Pvf/16/+93vtHnzZn9iAwAM1mAXXmzVqlW91ne5XJo9e7bWr1+v5ORkSVLnzp314YcfavHixUpPT5ck/cd//IeGDBkiSXr00Ud122236dSpU2rWrNl5t+tyubyPKyoq/Pl2AAABxrIrCR84cEAnTpyoce3Hqqoq7+W0JKlHjx7er+Pi4iRJZWVl6tChQ63bzcnJ0cyZMxshMQDAziwrtMrKSknSO++8o3bt2vksczqdKiwslCSfS2md3Z159vhdbbKysjRlyhTv44qKCsXHxzdYbgCAPVlWaN27d5fT6VRxcbF3l+K5zhZafTmdTjmdzkuNBwAIMJYVWmRkpKZOnarMzEy53W4NGjRI5eXl2rx5s1q0aKGOHTtaFQ0AEIAsvRvnrFmzFBMTo5ycHB08eFAtW7ZU79699dhjj11wtyIAAD9laaE5HA5lZGQoIyOj1uUej8fnca9evWrMAwBA8vMfqwEAsBsKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYARLb/DZlDynz8jjcFgdo14qukZaHcEvLfcF7t9JnlMuqyP4Zdv/6WJ1BL+cfD2wfifPCrv5B6sj+O9YudUJ6sddVedVA/eTBwCAc1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACMEXKE5HA698cYbVscAANhMwBUaAAC1odAAAEYIsTrAT6WmpqpHjx5q1qyZli5dqrCwMP3+97/XU089ZXU0AICN2XKElpubq+bNmys/P1/PPPOMsrOz9d5779XpuS6XSxUVFT4TAMB8tiy0Hj16aMaMGUpMTFRaWpr69u2rDRs21Om5OTk5ioqK8k7x8fGNnBYAYAe2LbRzxcXFqaysrE7PzcrKUnl5uXcqKSlpjIgAAJux3TE0SQoNDfV57HA45Ha76/Rcp9Mpp9PZGLEAADZmyxEaAAD1RaEBAIxAoQEAjGC7Y2h5eXk15p17qSuPx9N0YQAAAYMRGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAwAi2ux9aYzl2bx8FhzWzOka9tPl/hVZH8Ev1DyesjuC34C4drY7glzOFh62O4JfwkVFWR/DLF0uSrI7gt+4zjlodoV4c7uo6r8sIDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYAQKDQBgBAoNAGAECg0AYARbFJrb7VZOTo46deqk8PBw9ezZU6tXr5Ykfffddxo9erRiYmIUHh6uxMRELV++3OLEAAC7CbE6gCTl5OTolVde0aJFi5SYmKhNmzbpN7/5jWJiYvTaa69pz549evfddxUdHa0DBw7o5MmT592Wy+WSy+XyPq6oqGiKbwEAYDHLC83lcmn27Nlav369kpOTJUmdO3fWhx9+qMWLF6uyslJJSUnq27evJCkhIeGC28vJydHMmTMbOzYAwGYs3+V44MABnThxQsOGDVNERIR3WrlypQoLC/Xggw/q1VdfVa9evTRt2jRt2bLlgtvLyspSeXm5dyopKWmi7wQAYCXLR2iVlZWSpHfeeUft2rXzWeZ0OhUfH6/Dhw9r7dq1eu+99zR06FCNHz9ezz77bK3bczqdcjqdjZ4bAGAvlhda9+7d5XQ6VVxcrCFDhtS6TkxMjH7729/qt7/9rVJSUvTwww+ft9AAAD9PlhdaZGSkpk6dqszMTLndbg0aNEjl5eXavHmzWrRoocLCQvXp00fXXHONXC6X3n77bXXr1s3q2AAAm7G80CRp1qxZiomJUU5Ojg4ePKiWLVuqd+/eeuyxx1RSUqKsrCwdOnRI4eHhSklJ0auvvmp1ZACAzdii0BwOhzIyMpSRkVFj2eDBg/XEE09YkAoAEEgsP8sRAICGQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMQKEBAIxAoQEAjEChAQCMYIsbfDaFmLcPKCQozOoY9VPttjqBXxzBwVZH8F9FpdUJ/BIUFmp1BP+EBOZHUPenSq2O4Lc7139qdYR6OVl5Ruv71m1dRmgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI1BoAAAjUGgAACNQaAAAI9i20Dwej9LT09WqVSs5HA7t3LnT6kgAABuz7f3P//73v2vFihXKy8tT586dFR0dbXUkAICN2bbQCgsLFRcXp4EDB9a6vKqqSmFhYU2cCgBgV7YstPvuu0+5ubmSJIfDoY4dOyohIUHXXnutQkJC9Morr+i6667T+++/b3FSAIBd2PIY2nPPPafs7Gy1b99epaWl+uSTTyRJubm5CgsL0+bNm7Vo0aJan+tyuVRRUeEzAQDMZ8sRWlRUlCIjIxUcHKzY2Fjv/MTERD3zzDMXfG5OTo5mzpzZ2BEBADZjyxHa+fTp0+ei62RlZam8vNw7lZSUNEEyAIDVbDlCO5/mzZtfdB2n0ymn09kEaQAAdhJQIzQAAM6HQgMAGIFCAwAYwbbH0CZPnqzJkyd7H+fl5VmWBQBgf4zQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARqDQAABGoNAAAEag0AAARrDt/dAaisfjkSSdcVdZnMQPHrfVCfzi9py2OoLfggLxfSLJ4wnM3I4A/Xk73IH5uylJJyvPWB2hXk79b96zn+UX4vDUZa0A9uWXXyo+Pt7qGACAS1BSUqL27dtfcB3jC83tduvo0aOKjIyUw+Fo0G1XVFQoPj5eJSUlatGiRYNuuzGRu2kFam4pcLOTu2k1Zm6Px6Pjx4/riiuuUFDQhY+SGb/LMSgo6KKtfqlatGgRUG++s8jdtAI1txS42cndtBord1RUVJ3W46QQAIARKDQAgBEotEvgdDo1Y8YMOZ1Oq6PUC7mbVqDmlgI3O7mbll1yG39SCADg54ERGgDACBQaAMAIFBoAwAgUGgDACBQaAMAIFBoAoF5Onz6toUOHav/+/VZH8UGh/YwUFhbqiSee0KhRo1RWViZJevfdd/X5559bnAx2U1VVpS+//FLFxcU+Expebm6u3nnnHe/jadOmqWXLlho4cKAOHz5sYbLzCw0N1a5du6yOUQP/h1ZPwcHBKi0tVZs2bXzmf/vtt2rTpo2qq6stSnZhGzdu1C233KIbbrhBmzZt0hdffKHOnTtrzpw52rZtm1avXm11RK+FCxfWed1JkyY1YpJLE4jvlf379+t3v/udtmzZ4jPf4/HI4XDYLvNbb71V53Vvv/32Rkziv65du+rFF1/UTTfdpK1bt+oXv/iF5s+fr7ffflshISFas2aN1RFrlZmZKafTqTlz5lgdxcv4ixM3tPP1v8vlUlhYWBOnqbtHH31UTz/9tKZMmaLIyEjv/Jtuukn/9V//ZWGymubPn1+n9RwOh60LLRDfK/fdd59CQkL09ttvKy4ursHvUNHQRowYUaf17FjGZ5WUlOjKK6+UJL3xxhu66667lJ6erhtuuEGpqanWhruAM2fOaNmyZVq/fr369Omj5s2b+yyfN29ek2ei0Oro7KjB4XBo6dKlioiI8C6rrq7Wpk2bdPXVV1sV76J2796tP//5zzXmt2nTRt98840Fic6vqKio1vlnC8LuH7KB/F7ZuXOntm/fbtt8P+UO4BttnhUREaFvv/1WHTp00Lp16zRlyhRJUrNmzXTy5EmL0/natWuXrr32WgUFBemzzz5T7969JUn79u3zWc+q31EKrY7Ojho8Ho8WLVqk4OBg77KwsDAlJCRo0aJFVsW7qJYtW6q0tFSdOnXymb9jxw61a9fOolR189JLL2n+/PneA9CJiYmaPHmy7r//fouT1S6Q3yvdu3e33R849bFhwwZt2LBBZWVlPmXncDj00ksvWZjs/IYNG6b7779fSUlJ2rdvn2699VZJ0ueff66OHTtanM5XUlKSdzf64cOH9cknn6h169ZWx/Ki0Oro7Kjhxhtv1N/+9je1bNkyYEYMkjRy5Eg98sgjeu211+RwOOR2u7V582ZNnTpVaWlpVsc7ryeffFLz5s3TxIkTlZycLEnaunWrMjMzVVxcrOzsbIsT1lTbeyVQ/Od//qemTZum2bNn67rrrlNoaKjPcjvfo2vmzJnKzs5W3759A2J36Vl//OMfNX36dJWUlGjNmjXegti+fbv+7d/+zeJ0vlq2bKmioiK1adNGhw4dst0ImZNC/BBoIwbpx7PWxo8frxUrVqi6ulohISE6c+aMRo8erRUrVviMIuwkJiZGCxcu1KhRo3zmr1q1ShMnTrTdaGLKlCmaNWuWmjdvrszMzAt+qFpxjOFizr0j8LnZ7XpSyLni4uL0zDPP6N5777U6Sr1t2rRJixcv1sGDB7V69Wq1a9dOK1euVOfOnTVo0CCr43mlp6dr5cqViouLU3Fxsdq3b3/ez46DBw82cTpGaPU2Y8YMzZ07N6BGDNKPu7r++7//W08++aR2796tyspKJSUlKTEx0epoF3T69Gn17du3xvw+ffrozJkzFiS6sB07duj06dOSfjwedT52HT28//77VkfwW1VVlQYOHGh1jHp7/fXXde+992r06NHasWOHXC6XJKmiokKzZ8/W2rVrLU74L0uWLNGdd96pAwcOaNKkSXrggQd8TjKzGiO0egqkEcPZg8t1YcfRgiRNnDhRoaGhNfJNnTpVJ0+e1B//+EeLkpnrgw8+0OLFi1VYWOgdLbz88svq1KmTrUYLP/XII48oIiJC06dPtzpKvSQlJSkzM1NpaWmKjIxUQUGBOnfurB07duiWW27RV199ZXXEWo0ZM0YLFy60VaExQqunQBox7Nixw+fxp59+qjNnzqhr166SfjwzKTg4WH369LEiXp299NJLWrdunQYMGCBJys/PV3FxsdLS0nxK266lHEjON1ooLy+33Wjhp06dOqUlS5Zo/fr16tGjR43jf3Z9f+zdu1eDBw+uMT8qKkrff/990weqo+XLl1sdoQYKrZ7uvfdevfjiizV+OZYsWaLRo0dblKp25+4+mjdvniIjI5Wbm6vLL79ckvTdd99pzJgxSklJsSriRZ17anBhYaEkKTo6WtHR0frss8+869l1F16gefrpp7Vo0SKlpaXp1Vdf9c6/4YYb9PTTT1uY7OJ27dqlXr16SZLPe0Oy9/sjNjZWBw4cUEJCgs/8Dz/8UJ07d7YmVIBil2M9TZw4UStXrlR8fHytI4Zz/yq001+E7dq107p163TNNdf4zP/ss8/0y1/+UkePHrUoGezksssu0549e5SQkOCz++vgwYPq3r27Tp06ZXVE4+Tk5OiVV17RsmXLNGzYMK1du1aHDx9WZmampk+frokTJ1odMWAwQqunQB0xVFRU6Ouvv64x/+uvv9bx48ctSAQ7YrTQ9B599FG53W4NHTpUJ06c0ODBg+V0OjV16lTKrJ4Yof1MpKWl6YMPPtDcuXPVr18/ST+OLB9++GGlpKQoNzfX4oSwA0YL1qmqqtKBAwdUWVmp7t27+1xhBnVDof1MnDhxQlOnTtWyZcu8p5WHhIRo7Nix+sMf/lDjOmz4efJ4PJo9e7ZycnJ04sQJSfKOFmbNmmVxOuDCKLSfmR9++MG7q7RLly4UGWrFaAGBiEIDABiBG3wCAIxAoQEAjEChAQCMQKEBASYhIUELFizwPnY4HHrjjTeaPMdTTz3lvTIHYAcUGhDgSktLdcstt9RpXUoIJuNKIYAFqqqqFBYW1iDbio2NbZDtAIGOERrQAFJTUzVhwgRNmDBBUVFRio6O1vTp0713NU9ISNCsWbOUlpamFi1aKD09XdKPl5RKSUlReHi44uPjNWnSJP3www/e7ZaVlenXv/61wsPD1alTJ/3pT3+q8do/3eX45ZdfatSoUWrVqpWaN2+uvn37Kj8/XytWrNDMmTNVUFAgh8Mhh8OhFStWSJK+//573X///YqJiVGLFi100003qaCgwOd15syZo7Zt2yoyMlJjx47luo6wHQoNaCC5ubkKCQnRxx9/rOeee07z5s3T0qVLvcufffZZ9ezZUzt27ND06dNVWFio4cOH66677tKuXbv0l7/8RR9++KEmTJjgfc59992nkpISvf/++1q9erVeeOEFlZWVnTdDZWWlhgwZoiNHjuitt95SQUGBpk2bJrfbrXvuuUcPPfSQrrnmGpWWlqq0tFT33HOPJOnuu+9WWVmZ3n33XW3fvl29e/fW0KFDdezYMUnSX//6Vz311FOaPXu2tm3bpri4OL3wwguN9JME/OQBcMmGDBni6datm8ftdnvnPfLII55u3bp5PB6Pp2PHjp4RI0b4PGfs2LGe9PR0n3kffPCBJygoyHPy5EnP3r17PZI8H3/8sXf5F1984ZHkmT9/vneeJM/f/vY3j8fj8SxevNgTGRnp+fbbb2vNOWPGDE/Pnj1rvGaLFi08p06d8pnfpUsXz+LFiz0ej8eTnJzs+fd//3ef5f3796+xLcBKjNCABjJgwACfuywkJydr//79qq6ulqQaN4YtKCjQihUrFBER4Z1uvvlmud1uFRUV6YsvvlBISIjPDVivvvpqtWzZ8rwZdu7cqaSkJLVq1arOuQsKClRZWanWrVv7ZCkqKvJeJu2LL75Q//79fZ6XnJxc59cAmgInhQBN5KfXzaysrNS4ceM0adKkGut26NBB+/btq/drhIeH1/s5lZWViouLU15eXo1lFypPwG4oNKCB5Ofn+zz+6KOPlJiYqODg4FrX7927t/bs2aMrr7yy1uVXX321zpw5o+3bt+v666+XJO3du1fff//9eTP06NFDS5cu1bFjx2odpYWFhXlHjOfm+OqrrxQSElLjPmhndevWTfn5+UpLS/P5/gA7YZcj0ECKi4s1ZcoU7d27V6tWrdLzzz+vjIyM867/yCOPaMuWLZowYYJ27typ/fv368033/SeFNK1a1cNHz5c48aNU35+vrZv367777//gqOwUaNGKTY2ViNGjNDmzZt18OBBvf7669q6daukH8+2LCoq0s6dO/XNN9/I5XLpF7/4hZKTkzVixAitW7dOhw4d0pYtW/T4449r27ZtkqSMjAwtW7ZMy5cv1759+zRjxgx9/vnnDfjTAy4dhQY0kLS0NJ08eVL9+vXT+PHjlZGR4T09vzY9evTQxo0btW/fPqWkpCgpKUlPPvmkrrjiCu86y5cv1xVXXKEhQ4bozjvvVHp6utq0aXPebYaFhWndunVq06aNbr31Vl133XWaM2eOd5R41113afjw4brxxhsVExOjVatWyeFwaO3atRo8eLDGjBmjq666SiNHjtThw4fVtm1bSdI999yj6dOna9q0aerTp48OHz6sBx98sIF+ckDD4PYxQANITU1Vr169fC5JBaBpMUIDABiBQgMAGIFdjgAAIzBCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGIFCAwAYgUIDABiBQgMAGOH/Ayd8S1RiXqynAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ix = torch.randint(len(X), (1000,))\n",
    "X_test, Y_test = X[ix], Y[ix]\n",
    "\n",
    "confusion_matrix = torch.zeros(len(languages), len(languages), device=device)\n",
    "output = forward_sequence(model, X_test).argmax(1)\n",
    "for true_label, predicted_label in zip(Y_test, output):\n",
    "\tconfusion_matrix[true_label, predicted_label] += 1\n",
    "confusion_matrix /= confusion_matrix.sum(1, keepdim=True)\n",
    "\n",
    "plt.imshow(confusion_matrix.cpu().numpy())\n",
    "plt.xticks(range(len(languages)), languages, rotation=90)\n",
    "plt.yticks(range(len(languages)), languages)\n",
    "plt.xlabel('predicted')\n",
    "plt.ylabel('actual')\n",
    "\n",
    "accuracy = confusion_matrix.diag().sum().item() / len(Y_test)\n",
    "print(f'Accuracy: {100 * accuracy:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see a similarly good performance as with RNNs, probably because the dataset is simple and the sequences are short, so none of the models have a hard time learning the patterns.\n",
    "\n",
    "The performance could be furthermore improved in a number of ways. For instance, all $W_*$ and $U_*$ weights could be merged, so that in the input the short term memory and the input are concatenated and multiplied by a single weight matrix. This would reduce the number of parameters and make the model more efficient. However, it still poses the problem of having a time complexity of $O(n^2)$, where $n$ is the sequence length. Furthermore, there are other RNN architectures that are more efficient like GRU or BiGRU."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}