ARTM_example_EN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple experiment using BigARTM Python API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Author - **Murat Apishev** (great-mel@yandex.ru)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BigARTM is an open library for topic modeling of text collections. It's based on the ARTM theory. The projects main site\n",
    " is http://bigartm.org/.\n",
    "\n",
    "The example of BigARTM Python API usage is showen below. We'll proceed a model experiment.\n",
    "\n",
    "#### Description:\n",
    "Let's learn two topic models of text collections, ARTM and PLSA, and compare them.\n",
    "\n",
    "#### Quality measures:\n",
    "One of the important measures is the perplexity. Nevertheless it's not the only way to check the quality of the model learning. The following qualities are implemented in library:\n",
    "\n",
    "- $\\Phi$ (token-topic) matrix sparsity;\n",
    "- $\\Theta$ (topic-doc) matrix sparsity;\n",
    "- topic kernel characteristics (purity, contrast, size);\n",
    "- perplexity;\n",
    "- topic mass;\n",
    "- the ratio of background tokens;\n",
    "- coherency of top-tokens and tokens from topic kernels.\n",
    "\n",
    "We'll use the first four ones. The higher values of sparsities and average purity and contrast means the more interpretable model.\n",
    "\n",
    "#### Goals of the experiment:\n",
    "We'll try to learn the ARTM model in the way to obtain better values of sparsities and kernel characteristics, than in PLSA, without significant decline of the perplexity.\n",
    "\n",
    "The main tool to control the learning process is the regularization. Here's the list of currently implemented regularizers:\n",
    "\n",
    "- $\\Phi$ matrix sparsing (+ semi-supervised learning);\n",
    "- $\\Theta$ matrix sparsing (+ semi-supervised learning);\n",
    "- $\\Phi$ matrix topics decorrelation;\n",
    "- forced $\\Phi$ matrix sparsing;\n",
    "- classes balancing (Label Regularization);\n",
    "- coherency improving.\n",
    "\n",
    "We'll use the first three regularizers in ARTM model in this experiment. ARTM without the regularization corresponds the PLSA model.\n",
    "\n",
    "#### Text collection:\n",
    "Let's use the small 'kos' collection from the UCI repository https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/. The collection has following parameters:\n",
    "\n",
    "- 3430 documents;\n",
    "- 6906 tokens in the dictionary;\n",
    "- 467714 non-zero values in Bag-of-Words."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At first let's import all necessary modules (make sure you have the BigARTM Python API in your PATH variable):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8.0\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import glob\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import artm\n",
    "\n",
    "print artm.version()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First of all you need to prepare the input data. BigARTM has its own documents format for processing called batches. BigARTM has tools for their creation from Bag-of-Words UCI and Vowpal Wabbit formats (see more at http://docs.bigartm.org/en/latest/formats.html).\n",
    "\n",
    "Library Python API similarly to scikit-learn algorithms represents input data in the form of one class called BatchVectorizer. This class object get batches or UCI / VW files as inputs and is used as input parameter in all methods. If the given data is not batches, the object will create them and store to disk.\n",
    "\n",
    "So let's create the object of BatchVectorizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_vectorizer = None\n",
    "if len(glob.glob(os.path.join('kos', '*.batch'))) < 1:\n",
    "    batch_vectorizer = artm.BatchVectorizer(data_path='', data_format='bow_uci', collection_name='kos', target_folder='kos')\n",
    "else:\n",
    "    batch_vectorizer = artm.BatchVectorizer(data_path='kos', data_format='batches')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "ARTM is a class, that represents BigARTM Python API. Allows to use almost all library abilities in scikit-learn style. Let's create two topic models for our experiments. The most important parameter of the model is the number of topics. Optionally the user can define the list of regularizers and quality measures (scores) to be used in this model. This step can be done later. Note, that each model defines its own namespace for names of regularizers and scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dictionary = artm.Dictionary()\n",
    "\n",
    "model_plsa = artm.ARTM(topic_names=['topic_{}'.format(i) for i in xrange(15)],\n",
    "                       scores=[artm.PerplexityScore(name='PerplexityScore',\n",
    "                                                    use_unigram_document_model=False,\n",
    "                                                    dictionary=dictionary)],\n",
    "                       cache_theta=True)\n",
    "\n",
    "model_artm = artm.ARTM(topic_names=['topic_{}'.format(i) for i in xrange(15)],\n",
    "                       scores=[artm.PerplexityScore(name='PerplexityScore',\n",
    "                                                    use_unigram_document_model=False,\n",
    "                                                    dictionary=dictionary)],\n",
    "                       regularizers=[artm.SmoothSparseThetaRegularizer(name='SparseTheta', tau=-0.15)],\n",
    "                       cache_theta=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to initialize models. It can be done using dictionary. It means:\n",
    "- the $\\Phi$ matrix with the name 'pwt' will be created with numbers of rows and columns corresponding the numbers of tokens in the dictionary and topics in the model;\n",
    "- this matrix will be filled with random values from (0, 1) and normalized.\n",
    "\n",
    "Note this step is optional: the model will be auto-initialized during the calls of fit_offline() / fit_online().\n",
    "\n",
    "Dictionary is the object of BigARTM, containing the information about the collection (vocabulary, different counters and values, linked to tokens)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "if not os.path.isfile('kos/dictionary.dict'):\n",
    "    dictionary.gather(data_path=batch_vectorizer.data_path)\n",
    "    dictionary.save(dictionary_path='kos/dictionary.dict')\n",
    "\n",
    "dictionary.load(dictionary_path='kos/dictionary.dict')\n",
    "dictionary.load(dictionary_path='kos/dictionary.dict')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then dictionary can be used to initialize topic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_plsa.initialize(dictionary=dictionary)\n",
    "model_artm.initialize(dictionary=dictionary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it was said earlier, ARTM provides the ability to use all the scores of BigARTM. Once the score was included into model, the model will save all its values, obtained at the time of each $\\Phi$ matrix update. Let's add the scores we need for our experiment (only ones, missed in the constructors):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.scores.add(artm.SparsityPhiScore(name='SparsityPhiScore'))\n",
    "model_plsa.scores.add(artm.SparsityThetaScore(name='SparsityThetaScore'))\n",
    "model_plsa.scores.add(artm.TopicKernelScore(name='TopicKernelScore', probability_mass_threshold=0.3))\n",
    "\n",
    "model_artm.scores.add(artm.SparsityPhiScore(name='SparsityPhiScore'))\n",
    "model_artm.scores.add(artm.SparsityThetaScore(name='SparsityThetaScore'))\n",
    "model_artm.scores.add(artm.TopicKernelScore(name='TopicKernelScore', probability_mass_threshold=0.3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll do the same thing with the regularizers for artm_model (let's set their start coefficients of the regularization, these values can be changed later):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_artm.regularizers.add(artm.SmoothSparsePhiRegularizer(name='SparsePhi', tau=-0.1))\n",
    "model_artm.regularizers.add(artm.DecorrelatorPhiRegularizer(name='DecorrelatorPhi', tau=1.5e+5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll try to learn the model in offline mode (e.g. with one $\\Phi$ matrix update during one path through the whole collection). Let's start with 15 passes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.num_document_passes = 1\n",
    "model_artm.num_document_passes = 1\n",
    "\n",
    "model_plsa.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=15)\n",
    "model_artm.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the results of first part of learning process by comparing the values of scores of both models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sparsity Phi: 0.000 (PLSA) vs. 0.469 (ARTM)\n",
      "Sparsity Theta: 0.000 (PLSA) vs. 0.001 (ARTM)\n",
      "Kernel contrast: 0.466 (PLSA) vs. 0.525 (ARTM)\n",
      "Kernel purity: 0.215 (PLSA) vs. 0.359 (ARTM)\n",
      "Perplexity: 2058.027 (PLSA) vs. 1950.717 (ARTM)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX9//HXJyxBtoQ1bLIKERANVFBBaty3b3Fr3dqv\nRW3tt260thawttpfWxWtrdrWWmsVtFql7rYIqCSi1gUVBAEDiixBCCAQQCCQ5PP749yBIZlJZiYz\nc+cOn+fjcR+ZOXNm7jvYzsk9555zRFUxxhhj4pXjdwBjjDHBZA2IMcaYhFgDYowxJiHWgBhjjEmI\nNSDGGGMSYg2IMcaYhKS0ARGRQSIyX0Q+9H5Wisj1ItJBRGaLSJmIzBKRvLD3TBaR5SKyVEROCysf\nISILRWSZiNyTytzGGGMaJ+maByIiOUA5cAxwLfClqt4pIhOBDqo6SUSGAI8DI4FewKvAQFVVEXkX\nuFZV54nIDOBeVZ2VlvDGGGPqSWcX1inAZ6q6BjgHmOaVTwPO9R6PA55U1WpVXQksB0aJSDegnarO\n8+o9GvYeY4wxPkhnA3IR8IT3uEBVKwBUdT3Q1SvvCawJe89ar6wn7uolpNwrM8YY45O0NCAi0gJ3\ndfEvr6huv5mtp2KMMQHTPE3nORP4QFU3ec8rRKRAVSu87qkNXvla4NCw9/XyyqKV1yMi1hgZY0wC\nVFXiqZ+uLqxLgH+GPX8RGO89/i7wQlj5xSLSUkT6AYcB73ndXJUiMkpEBLgs7D31qGpgj1tuucX3\nDAdjdsvv/2H5/T0SkfIrEBFpjRtAvyqseAowXUSuAFYBFwKo6hIRmQ4sAfYCV+v+3+waYCrQCpih\nqjNTnd0PK1eu9DtCwoKcHSy/3yx/8KS8AVHVnUCXOmWbcY1KpPq3A7dHKP8AGJaKjMYYY+JnM9Ez\nzPjx4/2OkLAgZwfL7zfLHzxpm0iYLiKi2fY7GWNMqokImqGD6Gm1o+IrvyMkrLS01O8ICQtydrD8\nfrP8wZOVDUh56ad+RzDGmKyXlV1Yb014itH3XOh3FGOMCQzrwvJULVrmdwRjjMl6WdmAFN1/VeOV\nMlSQ+1GDnB0sv98sf/BkZQPSobBr45WMMcY0SVaOgWTb72SMMalmYyDGGGPSxhqQDBPkftQgZwfL\n7zfLHzzWgBhjjElIVo6BLMk9ii9GnsvJb9zqdxxjjAmERMZA0rWhVFoNrvqIyk/7+h3DGGOyWtZ2\nYXXeHMzJhEHuRw1ydrD8frP8wZO1DUjvPZ9Svbva7xjGGJO1snIMZG2zXvSoKWflq5/S9+QBfkcy\nxpiMZ/NAPOvzCgHY9N9gdmMZY0wQZGUD0mX6/WxZVM7RN5/hd5S4BbkfNcjZwfL7zfIHT1behXXo\nyYP8jmCMMVkvK8dAsu13MsaYVEv6PBARaQX8DzAW6AHsAj4G/qOqixMNaowxJviijoGIyK+At4Dj\ngHeBvwLTgWrgDhF5RUSOTEvKg0iQ+1GDnB0sv98sf/A0dAXynqreEuW134tIV6B3CjIlTdXOGqit\nJbdtC7+jGGNM1snaMZCSI65j9OIHef/6xxhzr+2PbowxDUnqGIiIvAREbV1UdVw8J0o3adOaXPaw\nZ1GZ31GMMSYrNTQP5HfA3cDnuMHzv3nHDuCz1EdrmmZD3K28LT4P1mTCIPejBjk7WH6/Wf7giXoF\noqqvA4jI3ap6dNhLL4nI+ylP1kR5IwthKnTYYFcgxhiTCo2OgYjIUuBsVV3hPe8HzFDVwTGdQCQP\neAg4AqgFrgCWAU8BfYCVwIWqWunVn+zVqQYmqOpsr3wEMBVo5Z3/R1HOp6rKxsUb6HJEAVvJI69m\nC5ITV9eeMcYcVFK1FtaPgVIRKRWR14ESIOKXdxT3sr/BOQr4BJgEvKqqhcAcYLL3CwwBLgQGA2cC\n94tI6Bf6C3Clqg4CBonI6Q2dtPPgLmyVfHJEqVy1NY64xhhjYtFoA6KqM4GBwATgeqBQVWfF8uEi\n0h4Yq6qPeJ9V7V1pnANM86pNA871Ho8DnvTqrQSWA6NEpBvQTlXnefUeDXtP5HPnCC1Wr6B9zVby\n+3WIJW5GCHI/apCzg+X3m+UPnkYbEBFpDdwIXKuqHwG9ReR/Yvz8fsAmEXlERD4UkQe9zytQ1QoA\nVV0PdPXq9wTWhL1/rVfWEygPKy/3yhrUplcHEOu6MsaYVIilC+sRYA9uRjq4L/XfxPj5zYERwJ9V\ndQTwFa77qu7AS3ZNRmmC4uJivyMkLMjZwfL7zfIHTyyr8Q5Q1YtE5BIAVd0ZNi7RmHJgjaqG7tp6\nBteAVIhIgapWeN1TG7zX1wKHhr2/l1cWrTyi8ePH07dvXwDy8/MpKira9x83dJlpz+25PbfnB/Pz\n0tJSpk6dCrDv+zJuqtrgAfwXOAT40Hs+ALfMSaPv9eq/DgzyHt8CTPGOiV7ZROAO7/EQYD7QEtf9\n9Sn77xR7BxgFCDADOCPK+TTISkpK/I6QsCBnV7X8frP8/vK+O2P6Xg8dsVyB3ALMBA4VkceBMcD4\nONqo64HHRaQFsAK4HGgGTBeRK4BVuDuvUNUlIjIdWALsBa72fjGAazjwNt6ZsZx817a9rJ67ksNO\n60+zls3iiG2MMaYhDc4D8bqqegE7gWNxf/2/o6qb0hMvfnX3A1ndvD+9az63/dGNMaYBSZ8H4n0T\nz1DVL1X1P6r670xuPCLZ1L4/ABvftBnpxhiTTLHchfWhiIxMeZIU2dHDrYn11fxgrIkVGuQKoiBn\nB8vvN8sfPLGMgRwDfFtEVuFuwxXcxUkgNpPSQYWwGHKW2xWIMcYkUyxrYfWJVK6qq1KSqInqjoG8\n/+uXOfqXZ/Fhh5MYsfk1H5MZY0zmSvZ+IG1VdUdDDUWoTjwnTLeuYwvZnNOJ6jZ5fkcxxpis0tAY\nyAsicreIfF1E2oQKRaS/iFwpIrOAM1IfsWl6n9CPjjWbGLXmWb+jxCTI/ahBzg6W32+WP3ga2g/k\nZBE5C/gBMEZEOuCWWC8D/gN8V906VpnN1sIyxpiUyNo90Y0xxsQuVfuBGGOMMfVYA5JhgtyPGuTs\nYPn9ZvmD56BoQPburmHJM0t5785Sv6MYY0zWOCjGQDYt3UjnIV2ppD3ta7ba/ujGGFNH2sZAROTf\nibzPL50KO7NV8sljG5uWbGj8DcYYYxqVaBfW95OaIsUkRyhvXQjA2pLMXhMryP2oQc4Olt9vlj94\nYmpARKSliBwpIsNEpKWqrkt1sGTbWuAWVdz+vq2JZYwxyRDLWlhnAw8An+EWUuwH/EBVX059vPhF\nmwdSespvKH7tF5SOvJHi9+70IZkxxmSupK6FFeZu4ERV/dQ7yQDcTPSMbECiaTt2OIs+GEuLwyKu\nDWmMMSZOsXRhbQ81Hp4VwPYU5UmZo285m2Fb5jLmiWv8jtKgIPejBjk7WH6/Wf7gieUK5H0RmQFM\nBxT4FjBPRM4HUNVgrFJojDEmqWIZA3mkgZdVVa9IbqSmsbWwjDEmfkkfAxGRZsBCVf1Dk5IZY4zJ\nOg2OgahqDXBJmrIYgt2PGuTsYPn9ZvmDJ5YxkLdE5E/AU7g90QFQ1Q9TlipFKhZtoOzB18nt1JZj\nbj3T7zjGGBNosYyBlEQoVlU9KTWRmqahMZB5v57JyF+eyfz8Exm+ZU6akxljTOZKyTwQVT0x8UiZ\npevxbjZ6t22ZvZyJMcYEQaPzQESkQET+LiIve8+HiMiVqY+WfL3G9KGKlnSvXcuO9Tv8jhNRkPtR\ng5wdLL/fLH/wxDKRcCowC+jhPV8G/ChVgVKpWctmrGl5GADlJct9TmOMMcEWyxjIPFUdKSLzVXW4\nV7ZAVYvSkjBOjc0DeafH+Ry77jn+e90/GX3fxWlMZowxmStVa2F9JSKdcLPQEZFjgcoE8mWEmtPP\n4q0PCug4vK/fUYwxJtBi6cK6AXgRGCAibwGPAtfFegIRWSkiH4nIfBF5zyvrICKzRaRMRGaJSF5Y\n/ckislxElorIaWHlI0RkoYgsE5F7Yv4N6xjzyPcYs/AvDL782EQ/IqWC3I8a5Oxg+f1m+YOn0QbE\nm+9xAjAa+AEwVFUXxnGOWqBYVYer6iivbBLwqqoWAnOAyeAG6IELgcHAmcD9IhK6pPoLcKWqDgIG\nicjpcWQwxhiTZLGMgbQCrgaOx3VjvQE8oKq7YzqByOfA0ar6ZVjZJ8AJqlohIt2AUlU9XEQm4eaY\nTPHqvQzcCqwC5qjqEK/8Yu/9P4xwPlsLyxhj4pSqPdEfBYYCfwT+5D1+LI5zKPCKiMwTke95ZQWq\nWgGgquuBrl55T2BN2HvXemU9gfKw8nKvzBhjjE9iGUQ/IvSXv6dERJbEcY4xqrpORLoAs0WkDG9A\nPkxSLxnGjx9P3759AcjPz6eoqIji4mJgfz9lpj6/5557ApU3/Hl4H3Am5LH8mZXP8mfW89LSUqZO\nnQqw7/sybqra4AH8Azg27PkxwKONvS/KZ90C/ARYirsKAegGLPUeTwImhtWf6Z1vXx2v/GLgL1HO\noY2Z/5e39ZXjb9X375rTaN10Kykp8TtCwoKcXdXy+83y+8v77ozrOz2WMZClQCGw2ivqDZQB1d4J\nj2zgva2BHFXdISJtgNnAr4CTgc2qOkVEJgIdVHWSN4j+uNdo9AReAQaqqorIO8D1wDzclrr3qerM\nCOfUxn6nklN+y4mv3Uzp0T+leN5dDdY1xpiDQarmgZyRYB6AAuA5EVHvXI+r6mwReR+YLiJX4AbI\nLwRQ1SUiMh1YAuwFrg5rDa7BzYpvBcyI1HjEKveoQngNDim3NbGMMSZRsdzGu6qho5H3fq6qRepu\n4R2mqnd45ZtV9RRVLVTV01R1a9h7blfVw1R1sKrODiv/wPuMgao6oSm/dKdj3aKKXTaXNeVjUiK8\nHzVogpwdLL/fLH/wxHIXVtbpVezWwzp0z2dU7672OY0xxgRTo2MgQRPrPJDy5n3oVbOaVa8so88p\nA9OQzBhjMleqxkCy0spv3sgqgcGH5vsdxRhjAilqF5aIbBeRbWHH9vCf6QyZCsc/eS1j/nktHQu7\n+B3lAEHuRw1ydrD8frP8wdPQFchruPkXzwJPqurqBuoaY4w5yDQ4BuKtkns+buJeK+ApXGOyOT3x\n4mdrYRljTPwSGQOJaRBdRHJwjch9wG2q+vvEIqaeNSDGGBO/pC+mKCKjReSPwIe45dzPy+TGIxsE\nuR81yNnB8vvN8gdPQ4Poq4D7cSviXgU8jNudcISIjEhTvpSac8GfebPnRXz60lK/oxhjTOBE7cIS\nkVL2r5KrQPiljarqSamNlph4urD27Y9+/T8Zfa/tj26MOXglex7IJFV9p4mZMtquPoWwDvYssjWx\njDEmXg2NgdyfthQ+aT7YrYnV4vPMWRMryP2oQc4Olt9vlj94GmpA4rqUCaK8ka4B6bDBrkCMMSZe\nDY2BbAXmRnujqo5LVaimiGcMZNMnm+g8uAvbaEe7mkokJ+vbTGOMiSjZYyAbgbubFimzdRrUibcv\nf5B2wwcwpFatATHGmDg01IW1XVVfj3akLWEKSY5w3MPf54jrTiKneWasbB/kftQgZwfL7zfLHzwN\nfWuuTFcIY4wxwRP3fiAicirwM1U9NTWRmsaWMjHGmPgldSkTETlRRJaJyA4R+YeIDPP2Mr8D+EtT\nwxpjjAm2hrqwfo9bwqQT8DTwNjBVVb+mqs+mI9zBKMj9qEHODpbfb5Y/eBpqQFRVS1W1SlWfB9aq\n6p/SFSxdaqtreav7N1naqojqXXv9jmOMMYHR0DyQFcBPw4ruAm4MPcnUq5BExkBsf3RjzMEu2fNA\nXge+EfZ8bthzxe1UmBUq8grptXk1G94oswbEGGNiFLULS1Uvj3YAk9OYMeV29HBLmuxc4P+SJkHu\nRw1ydrD8frP8wRPz7DkRyReRK0XkNWB+CjOlnQ4qBEA+9b8BMcaYoGhsT/RDgHOAS4HhQDvgXGCu\nqtamJWGcEhkDmfebWYz8xRnMzy9m+JaSFCUzxpjMldQ90UXkCWAsMBt4EpgDfKqq/ZoaNJUSaUAq\nV1fyxcsf0eOkw8kb2DVFyYwxJnMle0/0IcAWYCmwVFVr2L9DYVbJ653H4B98PSMajyD3owY5O1h+\nv1n+4GloEL0IuBDXbfWqiLwJtBORgnhPIiI5IvKhiLzoPe8gIrNFpExEZolIXljdySKyXESWishp\nYeUjRGShNzv+nngzGGOMSa6Y18ISka8Bl+AalXJVHR3zSUR+DHwNaK+q40RkCvClqt4pIhOBDqo6\nSUSGAI8DI4FewKvAQFVVEXkXuFZV54nIDOBeVZ0V4Vy2FpYxxsQp2V1YB1DVD1T1p0AfYFIcoXoB\nZwEPhRWfA0zzHk/DDcwDjAOeVNVqVV0JLAdGiUg3oJ2qzvPqPRr2HmOMMT6IexMMdaLuVBjBH3Az\n2MMvCwpUtcL7vPVAaPChJ7AmrN5ar6wnUB5WXu6VJV1Ntb9XL0HuRw1ydrD8frP8wdPQTPQmE5Gz\ngQpVXSAixQ1UTeq39vjx4+nbty8A+fn5FBUVUVzsTh/6j1z3ec/t3cg5/1xKmzdjwMt/brR+qp4v\nWLAgreez5/bcnh+cz0tLS5k6dSrAvu/LeMW9H0hcHy5yG/AdoBo4BDcg/xxwNFCsqhVe91SJqg4W\nkUm4i5wp3vtnArcAq0J1vPKLgRNU9YcRzpnQGMjGpZvoMsT2RzfGHJxSOgYSdpJzROSYWOqq6k2q\n2ltV+wMXA3NU9X+Bl4DxXrXvAi94j18ELhaRliLSDzgMeM/r5qoUkVEiIsBlYe9Jis6Hd2azdKQ9\n29n08fpkfrQxxmSlRDYCPwa4WURebsJ57wBOFZEy4GTvOaq6BJgOLAFmAFeHXU5cA/wdWAYsV9WZ\nTTh/PSKwtrVbE2vd6/4taRK6xAyiIGcHy+83yx88cY+BqOpNiZxIVV/HrfCLqm4GTolS73bg9gjl\nHwDDEjl3rLYWFMKKd6icVwackMpTGWNM4DU6BiIiXYExQA9gF/Ax8H42rYUVUnLqbZz46s954/jJ\njH3jtiQnM8aYzJXstbBOxM336IhbfXcD0AoYBAzAbXN7t6pua0roZGtKA1L5+WZyRGnXt1OSUxlj\nTGZL9iD6WcD3VXWkql6lqjer6k9VdRxwFK5RObUJeTNOXr+OvjceQe5HDXJ2sPx+s/zBE3UMRFVv\nbOC1auD5lCQyxhgTCLGMgRQAtwE9VfUMb72q41T17+kIGC9bC8sYY+KXqnkgU4FZQHfv+TLgR/FF\nM8YYk21iaUA6q+p0oBb2dV/VpDSVz77asocvP9vqy7mD3I8a5Oxg+f1m+YMnlgbkKxHphLdelYgc\nC1SmNJWPSi+fSquOh7DkGxP9jmKMMRktljGQEcAfgSNwc0C6AN9U1YWpjxe/po6BzPvtbEbefDoL\n8ospsv3RjTEHiUTGQBqdia6qH4rICUAhIECZqu5NMGPGKxjrljPptq3M5yTGGJPZGu3CEpHLgEtx\nOwqOAC7xyrJSz2MPZTe5dKtdx45129N+/iD3owY5O1h+v1n+4IllDGRk2DEWuBW3c2BWatayGatz\nBwJQPse/RRWNMSbTxb0fiIjk47adPSM1kZomGfNA3ul5AQPWv8na2x+j6GenJSmZMcZkrqSuhdXA\nSVoAH6tqYVxvTJNkNCC1VXvJyW2RpETGGJP5UjKRUEReEpEXvePfQBluV8Gs5WfjEeR+1CBnB8vv\nN8sfPLHsB/K7sMfVwCpVLU9RHmOMMQGR0j3R/WBrYRljTPySOg9ERLbjzT6v+xKgqto+znzGGGOy\nSNQxEFVtp6rtIxztDobGY1vFLhY/uYgNizem9bxB7kcNcnaw/H6z/METyzwQRGSEiFwvIteJyPBU\nh8oEH43+IUMvOZKyKVl9v4AxxiQslrWwfgl8C3jWKzoX+Jeq/ibF2RKSrDGQktNu58RXbuL1o3/C\nCfN+1/gbjDEmwFKyFhbwbeAoVd3tneQOYAGQkQ1IsuQOGwSvwCFrbE0sY4yJJJYurC+AVmHPc4G1\nqYmTOTod5xZV7LIlvcuZBLkfNcjZwfL7zfIHT0N3Yf0RdxdWJbBYRF7xnp8KvJeeeP7pVXwYtQi9\n9qygetdemh9iM9ONMSZc1DEQEfluQ29U1WkpSdREyZwH8nH746hp3opD355Ox8IuSflMY4zJRGlZ\nCyvT2URCY4yJX1LXwvLWwPqGt3hi3df6i8j/E5ErEglqogtyP2qQs4Pl95vlD56G7sL6PnADcI+I\nbAY24gbT+wGfAn9S1RdSH9EYY0wmiqkLS0T6At2BXcAyVd0Z04eL5AJzgZa4xuppVf2ViHQAngL6\nACuBC1W10nvPZOAK3MKNE1R1tlc+ApiKa8RmqOqPopzTurCMMSZOGTkGIiKtVXWniDQD3gKuBy4A\nvlTVO0VkItBBVSeJyBDgcdzuh72AV4GBqqoi8i5wrarOE5EZwL2qOivC+awBMcaYOKVkP5CmCrta\nycVdhShwDhC6i2sabnY7uK1yn1TValVdCSwHRolIN6Cdqs7z6j0a9p6U+viJhcy95il2bvwqHacL\ndD9qkLOD5feb5Q+elDcgIpIjIvOB9cArXiNQoKoVAKq6HujqVe8JrAl7+1qvrCcQvgdJuVeWci2v\n+DZfv/9iVs/+JB2nM8aYwIhlKZMmUdVaYLiItAeeE5Gh1F8mPql9TuPHj6dv374A5OfnU1RURHFx\nMbD/r4RYn89u25EvqiD3nTL49tfifn+8z0Nlqfr8VD4vLi7OqDyWP7PyWf7Mel5aWsrUqVMB9n1f\nxiuhMRARuVVVb03gfb8AdgLfA4pVtcLrnipR1cEiMgm318gUr/5M4BZgVaiOV34xcIKq/jDCOZI6\nBjLnuJs46Z3beb34Fk4ouTVpn2uMMZkknWMgH8RSSUQ6i0ie9/gQ3DIoS4EXgfFete8CoduBXwQu\nFpGWItIPOAx4z+vmqhSRUSIiwGVh70mp5kMKAWjxeXrWxAr9hRBEQc4Olt9vlj94EurCUtWXYqza\nHZgmIjm4xuopVZ0hIu8A072JiKuAC73PXSIi04ElwF7g6rDLiWs48DbemYlkj1feyEHwMHTYaKvy\nGmNMuFj2A+kP3AscB9QCbwM/VtUVqY8Xv2R3YX25fDOrR1/EV4NGcPxbU5L2ucYYk0lSMg/Eu1r4\nM/BPr+hi4DpVPSahlClm80CMMSZ+qRoDaa2qj3lzM6pV9R8cuD+ISaIg96MGOTtYfr9Z/uCJZQzk\nZe/uqCdxt9teBMwQkY4Aqro5hfmMMcZkqFi6sD5v4GVV1f7JjdQ01oVljDHxS/oYiHf31HGq+lZT\nw6WLNSDGGBO/pI+BeLPI/9SkVFlg55e7KL3qceZ84w8pP1eQ+1GDnB0sv98sf/DEMgbymohcADx7\nsP5prwpf/9v/UksO1buuTc7+6KtXwyefwLJlsHy5+7l2LYwfD2HLmhhjTKaKZQxkO9AGqMHtByK4\nsY/2qY8Xv1R1Ya1u3o/eNStZNbuMPqcOavwNqrBuHbRvD23b1n997Fh488365R98ACNG1C+/+WbY\ntQuGDoUhQ9zRPiP/ExhjAiiRLqxGr0BUtV3ikbLHhrxB9N68ko1vRmlAZs+GuXP3X1EsXw5ffQXP\nPAPnn1+//rHHQrNmMHAgDBrkfvbsCUccETnA1KnuCiVcr15QUgKHHdbk388YY+LVaAPirT31baCf\nqv5aRA4FuqvqeylPl0F29BwEm2ejzzwLv/pGvde3PvwM+U89eEDZ3rxO7CzfTl6Ez1t93V1s2gQi\nBx4rZ5YyblzxgZVVqfzNH6n5eAktly+m5adLaPHZJ7B2LTvzetAmwudXf+tiajoXoIOHoIPdVUtO\n5440b+7arQOD7nUN365dsHPn/p85OXDOOfU/fPNmuPHGevVLd+ygeP78CGGqYd486NIFOneGvDz3\ny2aY0rBVkIPI8vsr6PkTEcsYyP24JUxOAn4N7MDNTB+ZwlwZRwsHwyKgpjri649uGcdWCljGIJYz\nkOUMZEtlR/7aCq6KUP/Xv4aHHqpffsMNMG5cnUIRfvrWeTz00Hn7inKooTermfxca66qe4KtW2n+\n9FP1/uMuYTBv/nVJ/fobN0a88tkgBcyceg6XXVbnhb174eGH69Xf3KwLzz4b4YJr40YYPXrf0+qc\nFmzP7UxF3iAqnizlhBPq1K+q4r93vcXCdV2oateZvXmdycltQYsWcOKJkS/Sli1zp8nNhZYt3ZGb\n69qsSD2Ixpimi6UBOUZVR3ibQqGqW0SkZYpzZZwj7vgOMz4qo9u5YyK+XnXK2by662xUoXktHK5u\nGKSgIPLn9e4NRUWuTvhx3HHFEet37gx9+0JtratXW9uMPdqPNpEuP1q14oGzX2LtK0sorF3C4JrF\nFOpSurOO5pH+i7dpw7q8w/misjW7OISduJ+btDO6J0L9/HymHf83Zr15YP3tNe24dkukf5wqVnQZ\nBRs30plNtK/dTodd61i/K4/ly6nfgKxZw+hfnMzosKKt5PE+R/P2g6/Wb0DKy9l2xg2s+DyXKvYf\n5fTi8L/eUL/B3LaNv180m1ffyKW6RSu0RS61LVtR1ao7G2+Hb32r/q/w9NMwfz60bn3gMXo0DBhQ\nv/727e5n69YRrvhSJOh//Vr+4IllEP1dYDQwz2tIugCzVXV4OgLGy+aBRFFbC19+6f4kj6Cqyl1Y\n1NS4qrW17nHbtu5LsK5162Dr1v31a2rc0adP5FN8+CF8/rnrzarduZvmlV/Czp0cecFACgvrVF62\njE3nX0VtxUZa79xI611fkqO1rO00jNUvLeS44+rU/+gj1xrX8UmLYSx6fGH9BmHhQjjqqHr1yxjE\n24+UMX58nRcqKph50p3MWVLABrpSQQEVFLCO7tz2cHcuv7z+73vllfsv0lq2dP+GbdvC734HF11U\nv/6sWbBihevdCz/69LF7JUx6pGQQHbgPeA4oEJHfAt8Ebk4gn4lByvpRc3KiNh7guntyc2P/uO7d\n3RGutLSULl2KI9YfMSL85rJWNLgj8aBBdP64dP/z2lrYsoWeu3bRs1eE+r17w1NPuVZw9273s6qK\nwzt04PDjO2jbAAAUoUlEQVQIVxO0bUvNOedTs6sK3bUb3V0Fu6tY1iavfvchwIoVnLHk95xRp3hl\nh+GUD/ywfv21axn30d9p07KA8j1dWb+ngIo9BVRsLUA1cn/atGnwz39GLq/XhQjccQf8978HNjZb\ntpQyYUJx/QY5III+hhD0/ImI5S6sx0XkA+Bkr+hcVV2a2ljGhMnJgU6dor/eoQNceGHsn9e/P82e\nf4a6PUvtSkvp2DFC/Z49YcoU2LABKircsWEDfY8YQt/jI9T/5BPO+eAW6t5+sPtro9l7doRFHdav\n56q81xkxrgdranqwak93NuxoTWUldOsW+Vd49114KcKuPCedRMQG5Gc/c+/p2vXA48wzXdeoMYmI\naUtbERkBHI9bTPEtVY3wZ1dmsC4s47uyMnj00X0Nzb6fxx8Pjz1Wv/5LL9W/cyI/392N8Pe/16+/\nbRtL3viST7/qzpZdraishMpK10N51VVuilBdp5wCr71Wv/w//4GzzqpffsMNsGTJgY1N9+5w6qnR\nGzUTbCnpwhKRXwLfAp7BTSJ8RET+paq/SSymMVmusBB++9vY63fsCBdcAF984Y7QAFN15Dv+mDGD\nIZdcwpDQe3v0cMeZZ8KQH0V8ywMPwJo1rh0LPwYOjHyKt9+Gd96pXz5nTuQG5MEHYccO15t46KHu\n6NbNXTya7BXLIHoZcJSq7vaeHwIsUNWM7GkN+hVIkPtRg5wdMii/qptrU10d+Ta+J56AiRNh/foD\nGpnSceMofuGF+vVfecW957DD3DFggPuZnx81wqJFBzY4FRVuHutdd7nGoa5hw+Djjw8sa9HCNUKR\nFlZYscL1PObn758SlDH//gkKev5UDaJ/gRv13O09zwXWRq9ujGkSkYbHfC691B21tbBp0/4rly++\niFz/rbfcSgZ1TZzoRuPr2ruXYUc0Z9iw2L9LfvAD13O3Zo07Vq920aJ1d512Gnz2GbRp4xqk3r3d\n3WqDB0e/9d1knliuQJ7HTRp8BTcGcirwHlAOoKrXpzhjXIJ+BWJM0n38sbtl69NP9x+ffeZuDLj2\n2vr1f/5z+POf3VXKwIH7r1zGjoX+sW//s2sXtGpVf9EBVfja19xqPzt2HPja5s3uyqSua65xVyv9\n+7sLqP793b0N6ZpjczBI1Z7o323odVWdFs8JU80aEGNioOq6v1pEWFn6sssiD/Y/8IC71KhryRI3\n2NG/v7uMiCPC1q37r1jKy+H//q9+vaoqOOQQVz9cbi5s2eJeq2vnzsjzl0x0KWlAgiboDUiQ+1GD\nnB0s/z6qrv8p/Ipl+XL4yU/cpUNd550Hzz/vLgf694fDD3c3Enzve5HvKY4z/+7dMH26u2hasWL/\nTxF3v0FdO3ZAu3au+yx0xTJggLuYuvTSOP4d4hT0//2kagzEGHMwEXGTTrt0of60/wgKCtxkklWr\n9q9EHbo1OVID8vTTrrEpLHRdY41ctbRqFXky5e7d9cvADfa3aOHuMVi/3vXegRtnidSAbNvmhoj6\n93dHv36Rr2pMfXYFYoxJjl273NVKWZnbLO2aayIPaBQWutUvYX/XV2Eh3H+/+5ZPgpoa15CEX7W0\nagW//GX9uu+/DyPrLA3bo4fb1+3xx5MSJxDS0oUlIq2Ab6jqv+J6Y5pYA2JMhvvZz9zAflkZrFzp\n7iYDNxMy0lIAoYZo0KD9R8QlAxJTVgb33OMamhUrXKTqajj9dJg5s379d991a52FBvNDR+iCKqhS\n1oCISDPgdOAS4DTgDVX9ZkIpUyzoDUiQ+1GDnB0svy+qqvaNs5Tm5dXPX1XlRsNDjUxIp05u9D0F\nfU3V1e7qparKtVV1PfZY5C61448v5Y03iuuVL1sGpaVu/7dDD3U/w+e/ZIqkj4GIyAnApcBZuFt3\nx+A2ltqZcEpjjAnJzXXbNA8d6r5l61J1K0ouW3bg0aJF9NuvBg/ev9Nn6CgsjLzufgTNm7tVkKM5\n/3y3J02oeyx0ROt9mzu3/s1rbdq4srvvrl9/2zbXBZeJjUxdUa9ARKQcWA38BXheVbeLyOeq2i+d\nAeMV9CsQY0wjQjP1I022jLJUP336uL6punbscNtRh2YzdumS9PVXSkrgH/9wtymHJlru2OF68qZM\nqV//vvtgwgR34dWr1/7jvPPg3HPr11dNTkOT7CuQp4FzgYuAGhF5ATeRMJ5AvYBHgQLcroZ/U9X7\nRKQD8BTQB1gJXKiqld57JgNXANXABFWd7ZWPAKbiZsXPUNXIi/4YY7JbQzP1hw51XWJ1r1iijZl8\n8olbhywkN9d9W48dC488Ur9+At/WJ57ojnChq4xIqqrc3jE7duyPD+7usEgNyG23uSVm6q60fP75\nbsZ/Sqlq1AO3eOKJwIO4mefbgQuBtg29L+z93YAi73FboAw4HJgC/Mwrnwjc4T0eAszHNWx9gU/Z\nf5X0LjDSezwDOD3KOTXISkpK/I6QsCBnV7X8fvMl/4IFquPGqRYVqXbsuH9z0NNOi1x/7lzVvDzV\nYcNUzz5b9Yc/VL3tNtUZM5Kev7JSdfFi1ZkzVR96SPX99yPXu/76uvuauuOuuyLXv/lm1YIC9yuc\nfLLqJZeo/utfqt53Z6Pf6+FHg2MgqqpACVAiIi3YP5B+P9A5hsZpPbDee7xDRJYCvYBzgNBGptOA\nUmASMA54UlWrgZUishwYJSKrgHaqOs97z6O4q6NZjWUwxpiojjoKwheg/Oor18dUd9A+ZM0at3b+\nokXuCDnnHPhRhE6RuXPhV79yc2W6dt3/c+hQOPbYBqO1b++W5o+0PH+4P/zB3Z4cvsryxo1uu+VI\n1q3bv61NyOGHN3yOaGK+jddrQI7ALaS4XVV3xXUikb64huIIYI2qdgh7bbOqdhSRPwJvq+oTXvlD\nuKuNVcDtqnqaV3487gqm3v5xNgZijEkZVXe78erV7gitwzJ0KPX3Qsbta3zllfXLv/1tNzBS15w5\n8Kc/7e+HCjU4Q4a4cyTBnj1uoYHwxuaII6CoKIljICLyAPBHVV0sInnA20AN0BH4KRBhA86on9UW\nN6YywbsSqfsNb9/4xpjMJwKdO7sj0jr1dZ11ltvwPnxjsYoKGDMmcv3Fi+G55+qXX3UV/PWv9cuf\nfRbuvdeN8YSODh3cCgJ1B148LVvu30KmqRrqwhqrqqGlzS4HlqnquSLSDXiZGBsQEWmOazweU9XQ\ntWKFiBSoaoX3eRu88rVA+G4DvbyyaOURjR8/nr7ePp35+fkUFRXtu7+81LtVMFOf33PPPYHKG/48\n9DhT8lj+zMp3UOb/5BNo2ZLi73wnts/r3BluvZXiLl2gooLS+fNhyxaKvany9erPmgVz51LsnTeU\noPiGG+DEE+vXnzABpk6luKCA0ubNmVpZCbm59D36aBISbXAEmB/2+D/A+EivNXbgxit+X6dsCjBR\now+itwT6ceAg+jvAKNzA/gzgjCjnizYmFQhBHggNcnZVy+83y5+AtWtV58xRfeYZ1b/9TXXKFNWJ\nE1X//e/I9X/5y8gj7r/6VUKD6A3NAykB7sb9pV8CHK6q670rio9VtdFhFxEZA8wFFuG6qRS4CTcp\ncTruqmIV7jberd57JgNXAns58Dber3HgbbwTopxTo/1OxhhzUNu7162Bv3mzO0KPhw1Dhg9P3lIm\nIjIIuA93K+49qjrVKz8dOE1Vf9K03yQ1rAExxpj4JTKRMOqUS1VdpqpnqGpRqPHwymcBaxKPaRoS\n3o8aNEHODpbfb5Y/eBKds39DUlMYY4wJnIT2AxGRNap6aOM108+6sIwxJn5J7cJqhH1DG2PMQS5q\nAyIi20Vkm/cz9HibiGwHkjAFxUQS5H7UIGcHy+83yx88UScSqmq7dAYxxhgTLA3dxtsK+D/gMGAh\n8LC6RQ4zmo2BGGNM/JI9BjINOBo3CfAs3KRCY4wxBmi4ARmiqt9R1b8C3wTGpinTQS3I/ahBzg6W\n32+WP3gaakD2hh4EoevKGGNMejU0BlIDfBV6ChwC7PQeq6q2T0vCONkYiDHGxC+pe6KrarOmRzLG\nGJOtEp1IaFIkyP2oQc4Olt9vlj94rAExxhiTkITWwspkNgZijDHxS+daWMYYYw5y1oBkmCD3owY5\nO1h+v1n+4LEGxBhjTEJsDMQYY4yNgRhjjEkfa0AyTJD7UYOcHSy/3yx/8FgDYowxJiE2BmKMMcbG\nQIwxxqSPNSAZJsj9qEHODpbfb5Y/eKwBMcYYkxAbAzHGGGNjIMYYY9InpQ2IiPxdRCpEZGFYWQcR\nmS0iZSIyS0Tywl6bLCLLRWSpiJwWVj5CRBaKyDIRuSeVmf0W5H7UIGcHy+83yx88qb4CeQQ4vU7Z\nJOBVVS0E5gCTAURkCHAhMBg4E7hfREKXU38BrlTVQcAgEan7mVljwYIFfkdIWJCzg+X3m+UPnpQ2\nIKr6JrClTvE5wDTv8TTgXO/xOOBJVa1W1ZXAcmCUiHQD2qnqPK/eo2HvyTpbt271O0LCgpwdLL/f\nLH/w+DEG0lVVKwBUdT3Q1SvvCawJq7fWK+sJlIeVl3tlxhhjfJQJg+h2y1SYlStX+h0hYUHODpbf\nb5Y/eFJ+G6+I9AFeUtUjvedLgWJVrfC6p0pUdbCITAJUVad49WYCtwCrQnW88ouBE1T1h1HOZw2S\nMcYkIN7beJunKkgY8Y6QF4HxwBTgu8ALYeWPi8gfcF1UhwHvqaqKSKWIjALmAZcB90U7Wbz/AMYY\nYxKT0gZERJ4AioFOIrIad0VxB/AvEbkCd3VxIYCqLhGR6cASYC9wddiMwGuAqUArYIaqzkxlbmOM\nMY3Lupnoxhhj0iMTBtGTQkTOEJFPvMmGE/3OEw8R6SUic0RksYgsEpHr/c6UCBHJEZEPReRFv7PE\nS0TyRORf3iTWxSJyjN+ZYiUiPxaRj73Jto+LSEu/MzUm3knGmSRK9ju9/+0sEJFnRKS9nxkbEil/\n2Gs/EZFaEekYy2dlRQMiIjnAn3CTFocCl4jI4f6miks1cIOqDgWOA64JWP6QCbguyCC6F9c9Ohg4\nCljqc56YiEgP4DpghHejSnPgYn9TxSTmScYZKFL22cBQVS3CzWHL1OwQOT8i0gs4FTe0EJOsaECA\nUcByVV2lqnuBJ3ETFgNBVder6gLv8Q7cl1eg5rp4/+M7C3jI7yzx8v5aHKuqjwB4k1m3+RwrHs2A\nNiLSHGgNfOFznkbFOck4o0TKrqqvqmqt9/QdoFfag8Uoyr89wB+AG+P5rGxpQOpOQgzsZEMR6QsU\nAe/6myRuof/xBXFQrR+wSUQe8brgHhSRQ/wOFQtV/QK4G1iNm3y7VVVf9TdVwqJNMg6aK4CX/Q4R\nDxEZB6xR1UXxvC9bGpCsICJtgaeBCd6VSCCIyNlAhXcVVfe27SBoDowA/qyqI4CduO6UjCci+bi/\n3PsAPYC2InKpv6mSJnB/jIjIz4G9qvqE31li5f2xdBPuLtl9xbG8N1sakLVA77DnvbyywPC6H54G\nHlPVFxqrn2HGAONEZAXwT+BEEXnU50zxKMf99fW+9/xpXIMSBKcAK1R1s6rWAM8Co33OlKgKESkA\n8CYZb/A5T1xEZDyuGzdoDfgAoC/wkYh8jvv+/EBEGr0CzJYGZB5wmIj08e5AuRg3MTFIHgaWqOq9\nfgeJl6repKq9VbU/7t9+jqpe5neuWHndJmtEZJBXdDLBuRlgNXCsiLTyVq8+mYDcAED0ScZw4CTj\nTHRAdhE5A9eFO05Vq3xLFbt9+VX1Y1Xtpqr9VbUf7g+q4araaAOeFQ2I95fXtbg7IRbjVvUNyv+J\nEJExwLeBk0RkvtcPf4bfuQ4y1+NWQliAuwvrNp/zxERV38NdMc0HPsJ9KTzoa6gYeJOM/4vbnmG1\niFyOm2R8qoiU4RrCO/zMGE2U7H8E2gKveP//vd/XkA2Ikj+cEmMXlk0kNMYYk5CsuAIxxhiTftaA\nGGOMSYg1IMYYYxJiDYgxxpiEWANijDEmIdaAGGOMSYg1ICbriMh272cfEbkkyZ89uc7zN5P5+emU\nin8fc3CxBsRko9Dkpn7EuayEiDRrpMpNB5xI9fh4Pj/DxP3vY0w4a0BMNrsdON6bGTzB2/DqThF5\n19v45/sAInKCiMwVkRdwKxkgIs+JyDxvg6/veWW3A4d4n/eYV7Y9dDIRucur/5GIXBj22SVhm1U9\nFlb/Dm8jqAUicmfd8CLSRkQe9jaKWiAi53nll3hlC0XkjrD64VkuEJFHvMePiMi9IvKWiHwqIudH\n+vdJyr+4OaikdE90Y3w2CfiJqo4D8BqMrap6jLdm2lsiMturOxy3IdBq7/nlqrpVRFoB80TkGVWd\nLCLXeCv2hqj32RcAR6rqMG8Runki8rpXpwgYAqz3zjka+AQ4V1UP994faQe7X3h5j/Tq5IlId9wS\nH8OBrbilM8ap6ovUX702/Hk3VR0jIoNxa049W/ffx5h42RWIOZicBlwmIvNx+610BAZ6r70X1ngA\n/MhbFyu0OdBAGjYGtxIx3iJ0pcDIsM9ep27doAW4lU8rgV0i8pB3ZbErwmeeAvw59ERVK73PLPFW\n360FHge+7lVpaP2i573PWEpw99kwGcYaEHMwEeA6VR3uHQPCNl/6al8lkROAk4BjvC1KFwCtwj4j\n1nOFhK/OWgM09xYAHYVbCPF/gJkRPiPaQnXRMoTXb1XntfAMQduvxWQoa0BMNgp9QW4H2oWVzwKu\n9vZeQUQGikjrCO/PA7aoapW4vemPDXttT+j9dc71BnCRN87SBRgLvBc1oDtvvqrOBG4AjoxQ7RXg\nmrD35Huf+XUR6egN+F+Cu9oBWC8ihSKSA5wX7dxE//cxJi7WgJhsFPpLfCFQ6y2RP0FV/4bb5+ND\nEVkEPIDbT7yumUALEVmMW9b97bDXHgQWhg2GK4CqPued7yPgVeDGKPsphLK1B/4tIh8Bc4EfR6j7\nW6CjNzA/Hyj2tnqdhGs05gPzVPXfXv3JwH+ANzlwX/RoYyMH/PtEOL8xDbLl3I0xxiTErkCMMcYk\nxBoQY4wxCbEGxBhjTEKsATHGGJMQa0CMMcYkxBoQY4wxCbEGxBhjTEKsATHGGJOQ/w8h6retZ287\nSgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbc7a3c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def print_measures(model_plsa, model_artm):\n",
    "    print 'Sparsity Phi: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['SparsityPhiScore'].last_value,\n",
    "        model_artm.score_tracker['SparsityPhiScore'].last_value)\n",
    "\n",
    "    print 'Sparsity Theta: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['SparsityThetaScore'].last_value,\n",
    "        model_artm.score_tracker['SparsityThetaScore'].last_value)\n",
    "\n",
    "    print 'Kernel contrast: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['TopicKernelScore'].last_average_contrast,\n",
    "        model_artm.score_tracker['TopicKernelScore'].last_average_contrast)\n",
    "\n",
    "    print 'Kernel purity: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['TopicKernelScore'].last_average_purity,\n",
    "        model_artm.score_tracker['TopicKernelScore'].last_average_purity)\n",
    "\n",
    "    print 'Perplexity: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['PerplexityScore'].last_value,\n",
    "        model_artm.score_tracker['PerplexityScore'].last_value)\n",
    "\n",
    "    plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['PerplexityScore'].value, 'b--',\n",
    "             xrange(model_artm.num_phi_updates), model_artm.score_tracker['PerplexityScore'].value, 'r--', linewidth=2)\n",
    "    plt.xlabel('Iterations count')\n",
    "    plt.ylabel('PLSA perp. (blue), ARTM perp. (red)')\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "    \n",
    "print_measures(model_plsa, model_artm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see, that we have an improvement of sparsities and kernel measures, and the downgrade of the perplexion isn't big. Let's try to increase the absolute values of regularization coefficients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_artm.regularizers['SparsePhi'].tau = -0.2\n",
    "model_artm.regularizers['SparseTheta'].tau = -0.2\n",
    "model_artm.regularizers['DecorrelatorPhi'].tau = 2.5e+5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides that let's include into each model the TopTokenScore measure, which allows to look at the most probable tokens in each topic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_plsa.scores.add(artm.TopTokensScore(name='TopTokensScore', num_tokens=6))\n",
    "model_artm.scores.add(artm.TopTokensScore(name='TopTokensScore', num_tokens=6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll continue the learning process with 25 passes through the collection, and than will look at the values of the scores:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=25)\n",
    "model_artm.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sparsity Phi: 0.093 (PLSA) vs. 0.841 (ARTM)\n",
      "Sparsity Theta: 0.000 (PLSA) vs. 0.023 (ARTM)\n",
      "Kernel contrast: 0.640 (PLSA) vs. 0.740 (ARTM)\n",
      "Kernel purity: 0.674 (PLSA) vs. 0.822 (ARTM)\n",
      "Perplexity: 1619.031 (PLSA) vs. 1644.220 (ARTM)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVNX9//HXhwWkSJciRUHFXhAEJRhZNdbEkkRREwvR\nVPVri1EwBZMYu8YWyzfmJ+rXhibWICoJq9EoooCoIGABAQULRTrs7vv3x7kLw+7s7szsnZ256+f5\neMyDuWfvnfveq8zhnnPPOSYJ55xzLlvNCh3AOedcMnkF4pxzLidegTjnnMuJVyDOOedy4hWIc865\nnHgF4pxzLid5rUDMbGczm2ZmU6M/V5jZeWbWycyeN7PZZvacmXVIOWa0mc01s1lmdnhK+UAzm2Fm\nc8zspnzmds45Vz9rrHEgZtYMWAjsD5wLfCnpWjO7FOgkaZSZ7Q48AAwGegMTgf6SZGaTgXMlTTGz\n8cDNkp5rlPDOOedqaMwmrG8BH0haABwH3BuV3wscH70/FnhYUrmkecBcYIiZ9QDaSZoS7XdfyjHO\nOecKoDErkJOAB6P33SUtAZC0GOgWlfcCFqQcsygq60W4e6myMCpzzjlXII1SgZhZC8LdxaNRUfV2\nM59PxTnnEqZ5I53nKOBNSV9E20vMrLukJVHz1GdR+SKgT8pxvaOy2sprMDOvjJxzLgeSLJv9G6sJ\n6xTgoZTtp4CR0fszgCdTyk82s5Zm1g/YCXg9auZaYWZDzMyA01OOqUFS0b/GjBlT8AxNIaPn9JzF\n/kpKzlzk/Q7EzNoQOtB/mlJ8DTDOzM4E5gMjACTNNLNxwExgI3C2Nv9m5wBjgVbAeEkT8p09n+bN\nm1foCPVKQkbwnHHznPFKSs5c5L0CkbQG6FqtbCmhUkm3/1XAVWnK3wT2ykdG55xz2fOR6AUycuTI\nQkeoVxIygueMm+eMV1Jy5qLRBhI2FjNTU/udnHMu38wMFWknuqumrKys0BHqlYSM4Dnj5jnjlZSc\nufAKxDnnXE68Ccs555w3YTnnnGs8TbICUWXx34EkoV00CRnBc8bNc8YrKTlz0SQrEGuW1V2Yc865\nHHgfiHPOOe8Dcc4513i8AimQJLSLJiEjeM64ec54JSVnLrwCcc45l5Mm2Qcy5Zp/sd8lhxQ6inPO\nJYb3gUTWvPNBoSM451yT1yQrkMoVKwsdoV5JaBdNQkbwnHHznPFKSs5cNMkKhBVfFTqBc841eU2y\nD2TSoIsofeOGQkdxzrnE8D6QSMkqvwNxzrl8a5IVSO/rLyh0hHoloV00CRnBc8bNc8YrKTlz0SQr\nkH7f2aPQEZxzrslrkn0gTe13cs65fMulD6R5PR/YCvgO8E2gJ7AWeAf4p6R3cw3qnHMu+WptwjKz\n3wOvAEOBycBdwDigHLjazF4ws70bJWUTlIR20SRkBM8ZN88Zr6TkzEVddyCvSxpTy89uNLNuwHZ5\nyOSccy4BmmQfyOROhzNk6XOFjuKcc4mRSx9IrRWImT0N1Fq7SDo2u3iNw8wkYMPK9bTcumWh4zjn\nXCLEPZDweuAG4CNC5/lfo9cqoOhnK1z1aXHPh5WEdtEkZATPGTfPGa+k5MxFrX0gkl4EMLMbJO2X\n8qOnzeyNvCdroNWffkXn/l0KHcM555qsevtAzGwW8G1JH0bb/YDxknbL6ARmHYC7gT2BSuBMYA7w\nCLA9MA8YIWlFtP/oaJ9y4HxJz0flA4GxQKvo/GmHm1c1Yc159C12PsEfEnPOuUzkay6sC4EyMysz\nsxeBSUA2c4XczOYKZx/gPWAUMFHSLsC/gdHRL7A7MALYDTgKuN3Mqn6hO4CzJO0M7GxmR9R10nWf\n+XxYzjmXT/VWIJImAP2B84HzgF0kZfSIk5m1B74p6Z7os8qjO43jgHuj3e4Fjo/eHws8HO03D5gL\nDDGzHkA7SVOi/e5LOaaGt294nr7fLu7pTJLQLpqEjOA54+Y545WUnLmotwIxszbAr4BzJb0FbGdm\n38nw8/sBX5jZPWY21cz+N/q87pKWAEhaDHSL9u8FLEg5flFU1gtYmFK+MCpLa6+LDqP99p0yjOic\ncy4XmfSBPAK8CZwuac+oAvivpAH1frjZIOA1YKikN8zsz8BKQmXUOWW/LyV1MbNbgVclPRiV3w2M\nB+YDV0k6PCo/ELgk3aPEPheWc85lL/a5sCI7SjrJzE4BkLQmpV+iPguBBZKqntr6O6H/Y4mZdZe0\nJGqe+iz6+SKgT8rxvaOy2srTGjlyJH379gWgY8eODBgwgNLSUmDz7aRv+7Zv+/bXebusrIyxY8cC\nbPq+zJqkOl/Af4HWwNRoe0fCNCf1Hhvt/yKwc/R+DHBN9Lo0KrsUuDp6vzswDWhJaP56n813Sa8B\nQwAj3JUcWcv5lASTJk0qdIR6JSGj5Dnj5jnjlZSc0XdnRt/rVa9M7kDGABOAPmb2ADAMGJlFHXUe\n8ICZtQA+BH4ElADjzOxMQvPUiOibf6aZjQNmAhuBs6NfDOActnyMd0IWGZxzzsWszj6QqKmqN7AG\nOIDwr//XJH3ROPGyZ2Z6veNhfDX4UA59/tJCx3HOuUSIvQ9EksxsvKS9gH82KF0jGrz8Bf4720eh\nO+dcPmUykHCqmQ3Oe5KYtVhb3AMJqzqzilkSMoLnjJvnjFdScuYikz6Q/YEfmtl8YDWhGUuSinqe\nkJbri3syReecS7pMxoFsn65c0vy8JGqgqrmwZrfah13WTi90HOecS4RY+0DMbGtJq+qqKKr2yeaE\njaVVud+BOOdcPtXVB/Kkmd1gZgeZWduqQjPbwczOMrPngCPzHzF7068cz/p7Hip0jDoloV00CRnB\nc8bNc8YrKTlzUdd6IIea2dHAz4BhZtaJMMX6bMITWWcozGNVdAaMPqrQEZxzrslrkmuiN7XfyTnn\n8i1f64E455xzNXgFUiBJaBdNQkbwnHHznPFKSs5ceAXinHMuJ02yD+Rf+/6SznMns9X/3spup9S7\nbIlzzn3tNVofiJk9k8txjaX9x28zYNXLrHq/KB8Sc865JiHXJqyfxJoiZhtbtQ9/Li3e+bCS0C6a\nhIzgOePmOeOVlJy5yGQuLMysJbArIGC2pE/zmqqBytu0C38u9dHozjmXL5nMhfVt4E7gA8JEiv2A\nn0l6Nv/xsmdmmjTgfEqn30zZcTdS+sSFhY7knHNFL19rot8AHCzp/egkOxJGohdlBQLA1uEORCv8\nDsQ55/Ilkz6QlVWVR+RDoKi/mXe6YiTv3jaJ3a/7UaGj1CoJ7aJJyAieM26eM15JyZmLTO5A3jCz\n8cA4Qh/IicAUM/segKR/5DFfTnoP35Hew3csdAznnGvSMukDuaeOH0vSmfFGahifC8s557IXex+I\nmZUAMyT9uUHJnHPONTl19oFIqgBOaaQsXytJaBdNQkbwnHHznPFKSs5cZNIH8oqZ3QY8QlgTHQBJ\nU/OWyjnnXNHLpA9kUppiSTokP5Eaxsz06dRPWFT6Aza06sDQJU8UOpJzzhW9vIwDkXRw7pEKo6RF\nMwZ9VcbnK7sWOopzzjVZ9Y4DMbPuZvY3M3s22t7dzM7Kf7Tcte0RBhK2k8+F1RBJyAieM26eM15J\nyZmLTAYSjgWeA3pG23OAC/IVKA6tO7emnBJasZ6NazYWOo5zzjVJmfSBTJE02MymSdo3KpsuqSgX\n2qgaB7K8WSc6ajnL3v+STjt2LnQs55wravlaD2S1mXUhjELHzA4AVuSQr1GtahamdF/9afE2Yznn\nXJJlUoFcBDwF7GhmrwD3Af+T6QnMbJ6ZvWVm08zs9aisk5k9b2azzew5M+uQsv9oM5trZrPM7PCU\n8oFmNsPM5pjZTfWdd9VfH2bOfa/Rda8emUZtVEloF01CRvCccfOc8UpKzlxk8hTWVDMbDuxCmM59\ntqRsOhYqgVJJy1LKRgETJV1rZpcCo4FRZrY7MALYDegNTDSz/tHcJHcAZ0maYmbjzewISc/VdtJd\nfzQ0i4jOOeeylUkfSCvgbOBAQjPWf4A7Ja3L6ARmHwH7Sfoypew9YLikJWbWAyiTtKuZjSKMMbkm\n2u9Z4HJgPvBvSbtH5SdHx/8izfl8LiznnMtSvvpA7gP2AG4Fbove35/FOQS8YGZTzOzHUVl3SUsA\nJC0GukXlvYAFKccuisp6AQtTyhdGZc455wokk6lM9qz6l39kkpnNzOIcwyR9amZdgefNbDZRh3yK\nWG8ZRo4cSd++fQHo2LEjAwYMoLS0FNjcHlno7aqyYsmTbrt61kLnqW17+vTpXHDBBUWTp7Ztv55+\nPYshT9V2WVkZY8eOBdj0fZk1SXW+gP8DDkjZ3h+4r77javmsMcAvgVmEuxCAHsCs6P0o4NKU/SdE\n59u0T1R+MnBHLedQEkyaNKnQEeqVhIyS54yb54xXUnJG351Zfadn0gcyi9CB/nFUtB0wGyiPTrh3\nHce2AZpJWmVmbYHngd8DhwJLJV0TdaJ3klTVif5AVGn0Al4A+kuSmb0GnAdMISype4ukCWnOKUm8\neOa9dHrkDpYecwalD9foKnHOOZciX2uiH5ljHoDuwONmpuhcD0h63szeAMaZ2ZmEDvIRAJJmmtk4\nYCawEThbm2u4cwij4lsB49NVHqm0eAl7r5nMix8c2ID4zjnnalNvJ7qk+XW96jn2I0kDJO0raS9J\nV0flSyV9S9Iukg6XtDzlmKsk7SRpN0nPp5S/GX1Gf0nn15fbOoSBhLaqOJdvT22/LVZJyAieM26e\nM15JyZmLTJ7CSqTmncKEiiVri7MCcc65pKu3DyRpqvpAJv/mafb/07G83u3bDFnyTKFjOedcUcvX\nOJBEatkl3IG0XO93IM45lw+1ViBmttLMvkp5rUz9szFD5mLHE/blvbtfpusTdxc6SlpJaBdNQkbw\nnHHznPFKSs5c1PUU1r8I4y/+ATws6eM69i067ft0oP1Zwwodwznnmqw6+0CiWXK/Rxi41wp4hFCZ\nLG2ceNnzubCccy57ufSBZNSJbmbNCJXILcCVkm7MLWL+eQXinHPZi70T3cy+YWa3AlOBbwDfLebK\nI0mS0C6ahIzgOePmOeOVlJy5qLUPxMzmA8uAh4GfEqYuwcwGQlgnpDECOuecK061NmGZWRmbZ8kV\nYTGpKpJ0SH6j5Sa1CeuVbU+gy9K5dHj5n2w7uHeBkznnXPGKey6sUZJea2Cmguq27D36b3iXuQuW\ngVcgzjkXq7r6QG5vtBR5sq5FGEy47vPiG0yYhHbRJGQEzxk3zxmvpOTMRV0VSFa3MsVoQ8tQgWz4\noujHPTrnXOLU1QeyHHiptgMlHZuvUA2R2gfyaq8TGPrJ33n1wnEMvfHEAidzzrniFXcfyOfADQ2L\nVFjlbcIdSPkyvwNxzrm41dWEtVLSi7W9Gi1hA/T766/58LGp7HP59wodpYYktIsmISN4zrh5zngl\nJWcu6roDmddYIfKld+lOhY7gnHNNVtbrgZjZYcAlkg7LT6SG8alMnHMue7FOZWJmB5vZHDNbZWb/\nZ2Z7RWuZXw3c0dCwzjnnkq2uPpAbCVOYdAEeA14FxkoaJOkfjRGuKUtCu2gSMoLnjJvnjFdScuai\nrj4QSSqL3j9hZosk3dYImZxzziVAXeNAPgQuTim6DvhV1Uax3oWk9oHMemg6lWeexdJuu/HN+f9X\n4GTOOVe84h4H8iJwTMr2SynbIqxUWNwqKthj3VRmfe6d6s45F7da+0Ak/ai2FzC6ETPmrFXXMJCw\nTXnxDSRMQrtoEjKC54yb54xXUnLmos4FpVKZWUczO8vM/gVMy2Om2LTdtj0AbcqLbzJF55xLuvrW\nRG8NHAf8ANgXaAccD7wkqbJREmYptQ9k9Weradt9a9bSitZaW+BkzjlXvOIeB/IgMAc4DLgV6Ass\nk1RWrJVHdW22aUMFzWjNOsrXbix0HOeca1LqasLanbCk7SxglqQKNq9QmAjWzPjoocksfGEWzVqU\nFDrOFpLQLpqEjOA54+Y545WUnLmoqxN9ADCC0Gw10cxeBtqZWfdsT2Jmzcxsqpk9FW13MrPnzWy2\nmT1nZh1S9h1tZnPNbJaZHZ5SPtDMZkSj42/K9Nw7nbwfvb+1K82aZ9zd45xzLgMZz4VlZoOAUwiV\nykJJ38j4JGYXAoOA9pKONbNrgC8lXWtmlwKdJI0ys92BB4DBQG9gItBfksxsMnCupClmNh64WdJz\nac7lc2E551yWYu0DqU7Sm5IuBrYHRmURqjdwNHB3SvFxwL3R+3sJHfMAxwIPSyqXNA+YCwwxsx5A\nO0lTov3uSznGOedcAWTdrqOg1pUK0/gzYQR76m1Bd0lLos9bDHSLynsBC1L2WxSV9QIWppQvjMoS\nKwntoknICJ4zbp4zXknJmYu6RqI3mJl9G1giabqZldaxa6xtTiNHjqRv374AdOzYkQEDBlBaGk5f\n9R+z0NtViiVPkrenT59eVHmSvu3X8+txPcvKyhg7dizApu/LbGW9HkhWH252JXAqUA60JnTIPw7s\nB5RKWhI1T02StJuZjSLc5FwTHT8BGAPMr9onKj8ZGC7pF2nOuUUfyL9K/8h2/32IJb+4nANvHpG3\n39U555Isr30gKSc5zsz2z2RfSZdJ2k7SDsDJwL8lnQY8DYyMdjsDeDJ6/xRwspm1NLN+wE7A61Ez\n1wozG2JmBpyeckydmi/7nP4bZ1Gx4NNMf0XnnHMZyOXZ1v2B35jZsw0479XAYWY2Gzg02kbSTGAc\nMBMYD5ydcjtxDvA3wuDGuZImZHKiyq3DdCZaUVzzYVXdShazJGQEzxk3zxmvpOTMRdZ9IJIuy+VE\nkl4kzPCLpKXAt2rZ7yrgqjTlbwJ7ZX3iDqECYaXPh+Wcc3Gqtw/EzLoBw4CewFrgHeCNYp3OpHof\nyIsn38HwR87mpd1/xkHv3lnAZM45V7xiXQ/EzA4mjPfoTJh99zOgFWH8xY5m9hhwg6TiahuqpqRT\nuAMpWe13IM45F6e6+kCOBn4iabCkn0r6jaSLJR0L7EOoVA5rlJQNsNclRzHvqRnsPuHGQkfZQhLa\nRZOQETxn3DxnvJKSMxe13oFI+lUdPysHnshLoph16NeZDv06FzqGc841OZn0gXQHrgR6SToymq9q\nqKS/NUbAbPlcWM45l718jQMZCzwHbBttzwEuyC6ac865piaTCmQbSeOAStjUfFWR11RfA0loF01C\nRvCccfOc8UpKzlxkUoGsNrMuRPNVmdkBwIq8pnLOOVf0MukDGUhY0nZPwhiQrsAJkmbkP172qveB\nqFLMajOQrcuX03PVXJq3yuv8kc45l0i59IFkNJmimTUHdgEMmC2paBcYT9eJvtw60pEVrPhoKR36\ndipQMuecK1556UQ3s9OBHxBWFBwInBKVJcaaknYArF5cPIMJk9AumoSM4Dnj5jnjlZScucikPWdw\nyvtWhMkPpxJWBUyENSXtoQLWLC7qQfPOOZcoWa8HYmYdCcvOHpmfSA2Trgnrna0PYM/Vk3n7rv+y\n10+HFiiZc84Vr0ZZDwRYDfTL4biCWb9VmA9rwxd+B+Kcc3HJpA/kaTN7Kno9A8wmrCqYGD2fvotP\nX5rLXuccVOgomyShXTQJGcFzxs1zxispOXORSR/I9Snvy4H5khbmKU9ebPuNRN0wOedcIuR1TfRC\n8LmwnHMue3GvB7KSaPR59R8BktQ+y3zOOeeakFr7QCS1k9Q+zaudVx4Nl4R20SRkBM8ZN88Zr6Tk\nzEVG83pE05kcSLgjeVnStLymcs45V/QymQvrd8CJwD+iouOBRyVdkedsOUnXB/La7/5J96suYP7O\nh1P67l8KlMw554pXrH0gKX4I7CNpXXSSq4HpQFFWIGltLKdf+ft8/uWuhU7inHNNRiYDCT8hTGFS\nZStgUX7i5MdW24S5sFqu87mwspGEjOA54+Y545WUnLmo6ymsWwl9HiuAd83shWj7MOD1xokXj5bb\nhD7/rTYWTwXinHNJV2sfiJmdUdeBku7NS6IGStcH8uGzs9nh6F2Z12In+m6YW6BkzjlXvGLtAynW\nCiIXbXqEO5A2FX4H4pxzcam1DySaA+sYM2uR5mc7mNkfzOzM/MaLR9c9urFo4ixazJha6CibJKFd\nNAkZwXPGzXPGKyk5c1HXU1g/AS4CbjKzpcDnhM70fsD7wG2Snsx/xIYraVlCr0P9CSznnItTpkva\n9gW2BdYCcyStyejDzbYCXgJaEiqrxyT93sw6AY8A2wPzgBGSVkTHjAbOJEzceL6k56PygcBYQiU2\nXtIFtZzT58Jyzrks5W1N9IYwszaS1phZCfAKcB7wfeBLSdea2aVAJ0mjzGx34AHCKoi9gYlAf0ky\ns8nAuZKmmNl44GZJz6U5n1cgzjmXpcZaUCorKXcrWxHuQgQcB1R10t9LGN0OcCxhtcNySfOAucAQ\nM+sBtJM0JdrvvpRjEikJ7aJJyAieM26eM15JyZmLvFcgZtbMzKYBi4EXokqgu6QlAJIWA92i3XsB\nC1IOXxSV9QJS1yBZGJU555wrkIwmU2wISZXAvmbWHnjczPag5jTxsbY5jRw5kr59+wLQsWNHBgwY\ngP34fnb+cALP/OyX9D9pIKWlpcDmfx34ds3t0tLSospT13aVYsnj1zP/2349G7ZdVlbG2LFjATZ9\nX2Yrpz4QM7tc0uU5HPdbYA3wY6BU0pKoeWqSpN3MbBRhrZFrov0nAGOA+VX7ROUnA8Ml/SLNOdL2\ngbzW83sc8OnjvHbxoxxw3QnZRnfOuSatMftA3sxkJzPbxsw6RO9bE6ZBmQU8BYyMdjsDqHoc+Cng\nZDNraWb9gJ2A16NmrhVmNsTMDDg95ZiMbGwTBhOWLy2OwYTV/2VSjJKQETxn3DxnvJKSMxc5NWFJ\nejrDXbcF7jWzZoTK6hFJ483sNWBcNBBxPjAi+tyZZjYOmAlsBM5OuZ04hy0f452QTebKNmFCxcrl\nX2VzmHPOuVpksh7IDsDNwFCgEngVuFDSh/mPl73amrAmDfs1B//3SsoO/SOlE39TgGTOOVe88tWE\n9SAwDugB9AQeBR7KPl5hWbtwB8LK4mjCcs65pMukAmkj6f5obEa5pP9jy/VBEmHgnT/l8zfmc8A/\nf1voKEAy2kWTkBE8Z9w8Z7ySkjMXmfSBPBs9HfUw4XHbk4DxZtYZQNLSPOaLTfu+naFv50LHcM65\nJiOTPpCP6vixJO0Qb6SG8alMnHMue7HPhRU9PTVU0isNDddYvAJxzrnsxd6JHo0iv61BqVxaSWgX\nTUJG8Jxx85zxSkrOXGTSif4vM/t+NICvaXvuOTjoILjuOn9ayznn6pFJH8hKoC1QQVgPxAh9H+3z\nHy97tTVhfTbzC9bvtR/rS1qz04ZZ6Q9etAi23x4qKqBjRzj3XDjvPOjaNc+pnXOusPIyDkRSO0nN\nJLWQ1D7aLsrKoy6tO7WiT+V8tt34MSxbBukqzl694J574JvfhOXL4YorQoVy++2NH9g554pcvRWI\nBadGEyFiZn3MbEj+o8Wrbbe2VGK0ZQ0VPXry2WMvpd3vk0NPY9ZdL7HgoZdZc8h3YO1alnXekbVr\nq+24YQNMnQpPPw133gm//S386EfwixrzO6aVhHbRJGQEzxk3zxmvpOTMRSbjQG4nTGFyCPBHYBXw\nF8KqgYnRrMRYZh3ppGWUbFjHB/e9QrcTh9fY709/qrrhGAY8za7M4r1TduXWW0OL1iaffgqDBtU4\n/suSrhz5xh386lcwYkTKD1atgn/8g1f0Dd5YtiOLl4Sizp3Dq1cvqBos75xzSZBJBbK/pIHRolBI\nWmZmLfOcKy+mnnEzS8re49nuIznhx/3T7tO9O+yyC5SXw8aN8FX5bnTdCG3bVttx221h7715Z2lP\nXl3Ym0X0YhG9WFjRmzfeCC1gW3jtNTjjDIYB/enKx2zHF2zDExzCdVxSs4KKPPMMzJ4NffpA797h\nz223heZ5X8ll8xoCxc5zxstzxispOXORydfQxmg9cwGYWVfCHUniHHrPaQD8oI59fve78KpXy5bw\n1ltsvxI6r4T168Nrwwa4fG3oOtlCmzZw3HGsm/Rfun31Od34HIBmXTrzRHQHsoV//xvOO4+WGw/m\nlTkH8yLDWUqXcEyz0Gr2k59k9Gs751x+SKrzBfyQsE7HIuBPwGzgxPqOK9Qr/EpFrLJSmjdPk26/\nXXrmGWny5PT7/fa3Uujq3/Sa03pvndf+HplJEyemP+zCC6VjjpFGj5aeeEL68svco06aNCn3gxuR\n54yX54xXUnJG351Zfd/Wewci6QEzexM4NCo6XlItz8G6epmF25PddoO6bm1//Ws47DCYNAnKyuC/\n/6X/2hncfO0qrvtpuAtJ54UX4J13Qt9+1en23hvuuy/86ZxzccloSVszGwgcSGjGekXS1HwHy1WT\nncpk3TqYPBn694eePWv+/JJL4IsvWLT3UUzpeBhvvN+R//wndL1s2ACffw7bbFPzsIoKKCnJf3zn\nXHGLfS6s6EN/B5wI/J0wiPB44FFJV+QaNJ+abAVSl8rKUKksWRK2S0pg2DA45RTWnnAab73flgMO\nqHnY+vWwww5h1xNOgKOPhq23btzozrnikK8FpX4IDJZ0uaQxwAHAabkEdJvF+my4WWi7uvrqMBUL\nwEsvwbnn0nrjV2krDwh3J598Ao8+CiedFAbcf+97YTv2jHnkOePlOeOVlJy5yOQprE8IC0iti7a3\nInSou2JhBnvtFV6XXhqeIX7mGXj//fDMby2GD4ePPoJ//AMeewxefRUefzw0a514YiPmd84lUiZN\nWE8QBg2+QOgDOQx4HVgIIOm8PGfMyteyCStbL70EDz4IP/85DBiwqXjhwlCB7LILHH54zcM2bAhP\nLzvnmp589YGcUdfPJd2bzQnzzSuQDIwYsbmdavhwuOACOOaYenvTTzsN5syBs84KTV4dOjRCVudc\no8ilAin4uI24XxT7OJBIQZ8Nf/dd6bzzpHbtNo8z6ddPmjp1i91SM27YIPXosXn31q2l00+XysrC\n0JZCSspz9p4zXp4zXuQwDiSTTnTX1Oy+O9x8c2izuumm8CjWF1/AjjvWekiLFvDBB2E8SWkprF0b\n3h95JHz65jQ+AAAWA0lEQVT1VeNFd84Vj4zGgSSJN2HloKICZs2CPfes+bPKytBJX209sQ8+CDPf\nr1sH11/fSDmdc3mTlz6QNCdpBRwj6dGsDmwkXoHEbNw4+P3vw0yPp52W8UCR8ePh2WfhjDPCpMVf\ng/UsnUu0fI0DwcxKzOxoM7sfmA+clEtAt1kSng0vKyuDhx+GmTPh7LPDjI/nnx960utx++1w220w\neDD06xf66V98Mdzs5CVnAnjOeHnOwquzAjGz4WZ2FzAPOIvwCG8/SSc0QjZXDB55JLwOPDB0dtxy\nS3jOt56/FFdcEVYD7tED5s8PXS6lpeGuxDnXNNTahGVmC4GPgTuAJyStNLOPJPVrzIDZ8iasPJo2\nDf7yl3ArMXNm6FmvR2VlmMLr8cfDYPlXX4VWrWru99FH0LevN3U5Vyix9oGY2U2Eea/eAR4EngTe\nlrRDFoF6A/cB3QlriPxV0i1m1gl4BNiecHczQtKK6JjRwJlAOXC+pOej8oHAWMKo+PGSLqjlnF6B\n5Ft5efoVrZYsCf0lJ54Y1pXPcNWrpUuhS5ewmNdBB4WhKcOHh4fFapt12DkXr9jHgRAmTzwY+F/C\nyPOVwAhg60yeEQZ6AAOi91sT1hLZFbgGuCQqvxS4Onq/OzCNMMVKX+B9NldykwlzcgGMB46o5ZwN\nfh66MSTh2fCsM95+++aBIl27Sj/7mTRhgrRqVZ2HTZkide+++dCq18CBecpZIJ4zXp4zXsQ9DiT6\n3EmSfgr0A04BjiPcNWRSOS2WND16vwqYBfSOPqNqBPu9hDsdgGOBhyWVS5oHzAWGmFkPoJ2kKdF+\n96Uc44rFwQfD6NGw005h/vi77goDRX7/+zoP22+/sMT8e++FQ374w7B8b7qniiG0nl1/feiG8TEo\nzhVOxo/xmlkLYE/CRIorJa3N6kRmfYGy6DMWSOqU8rOlkjqb2a3Aq5IejMrvJtxtzAeuknR4VH4g\n4Q7m2DTnUaa/k8sTCWbMCNOlPPdc6FE/4oia+z3/fOj0GDwYOnas8RHr1kHr1jUP+/Of4aKLNm/3\n7x/mkfzBD+D734/5d3HuayKXJqxaG6nN7E7gVknvmlkH4FWgAugMXAw8lEWwrYHHCH0aq8ys+je8\nf+M3JWawzz7hdUUdy8aMGRPmlIcwCn7QoPA64wyse/e0lQfAvvuGeSDffBPeegvmzg2vgQPT7z9t\nGnz2WVgEsk8f76h3Li519XJ+U9LPo/c/AuZIOj5qTnqWDCsQM2tOqDzul/RkVLzEzLpLWhJ93mdR\n+SKgT8rhvaOy2srTGjlyJH379gWgY8eODBgwgNJo+diqZ7ILvV1VVix50m1Xzxrr50c95WXLlsGH\nH1L6wQfwwQeUjRsHvXpR+sMf1jz+888pe+cdMOOOO8LnvfBCGc8/P5199rmAffdNf77rroPx48N2\nq1ZlbLcdDBlSyi9+ARs2NJHrGeP29OnTueCCC4omT23bfj0bfv3Gjh0LsOn7Mmu1dY4A01Le/xMY\nme5n9b0I/RU3Viu7BrhUtXeityT0uaR2or8GDCF07I8HjqzlfDl3IjWmJHSsNVrGDRuk6dOlv/1N\nOv/89LMzVlaGyR87dZKGD5f+53+kv/5VmjxZkyZOrPPjr71WOuig0K+f2kn/+OPp93/wQemOO6R/\n/UtasECqqGj4rygl47+55DnjlpSc5NCJXtdjvJOAGwj/0p8E7CppcXRH8Y6kXeurnMxsGPAS8Dah\nmUrAZYT1RMYR7irmEx7jXR4dM5owaHEjWz7GO4gtH+M9v5ZzqrbfySXYkiXhud6lS7csb90aVq6s\nORV9ZSUsWBDarJptflbkyy/DtF+zZoUZ7Hv0qHmqb3wjjFep0qZNeC7gnntqbyZzLuniHgeyM3AL\n4VHcmySNjcqPAA6X9MuGxc0Pr0CaMCk8rjVjxuZXZWVYHKu6efPCHCqtW8POO8Ouu4ZOkAED4Ljj\n6jzNbbfB1KlhxpY5c8IDZQCzZ4ePqu6734VVq8Kkxn37bn7tu2/6QZPOFaNGWw8EuCCX4xrjhTdh\nxSYJGaVack6enH5wyaBB6T/kiy+kiROluXOldeu2+NGyZdLrr4eWtuoqK6UOHWqeBqT589PnfOyx\n0Dw2Z460Zk32v2++Jfq/exFKSk5yaMLKbKhwTRcBN+V4rHP5N2QILF4c1oefPXtzu1W6NisIbVbH\nHLN5u0cP2H57OOooOo4Zw+DBtZ/qtdfC9Pbz5m1+ffwx9OxZc18JTj01PKJcZZttQkvbSy+ln+x4\nxQpo396fHnPFJ6f1QMxsgaQ+9e/Z+LwJy+Vk4kT4wx/CzI+LFm2eNvj00+HeNKs2P/44XHxxGPHY\np0949egRHkM+8MBaT7NuHfz4x6F75uOPw5pe5eWhpW316pqVRHk5bLVVaArr2RO23TacZtttw1pg\nXqm4uDTKeiDRiT6WtF3WBzYCr0Bcg5WXwyefhG/4du3CeJbqbrwRfpmmG/DnP4c77qhZ/sQTcOed\n4Xaja9dNf1bsM5Alffbjs89C90x1S5aEDvxVq7Ys79w5PBBQ3Vdfha6fbt3Cq2vX8Gfv3nDZZZn9\n+u7rKe5O9JWEp6aqPrBqRwNaS8q1+SuvklKBlJWVbXo2u1glISMUKOfataGCWbAg3EYsWBCazIYP\nhxEjau7/pz9R9pvfUCPlxRfDddfV3P/uu+HWW8MI/U6d2LB1J1aWdGThHkcys88RrF8PI0em7L98\nOaxfz4dftGfHPVux+a9t0KdPiFvdJ5+EBwO6dAl1WpcuUF5exv77l3LVVTX3Ly8Pv2bHjtC2bWHv\ngPz/z3jFOhJdUruGR3KuiWrdOqyLsssume1/6qnhUeOePcP6859/Hv4cOjT9/vPnh6fMIi2BLkCX\nMR3Y55Q008LccguMGcMOQGVJCZVttmZjq3bMOvpiXtnv/BpPOfPCC/DiizRb2ZYzV7dl9eq2rP64\nLW+xD+8RKpYtKpC1a6Gyko8WtmbnXcNj0SUl0KFDqEz23BOefJIali0Li1q2bx/2bd8+vDp1CpWa\nS7a67kBaAT8HdgJmAP9PUnkjZstJUu5AnKvTF1+EO5tly7Z8DR0Kw4bV3P/KK8MkYV99BRs2bC6/\n+mq49NKa+48aBddcU6N4zulXMOXwX9OyZZiVf5Nf/zqcA1hPS9bSmnW04kou41bOY999w6PPmzzw\nADz1FEtXb8Xf/7kV6wmvZ/gOZRzMXnttUT+G+WZmz+aTL1py/S0tadG2Jc3btGTFNjuyoXsfdtoJ\nLrkkZf+VK2HtWlaua8HUt1vQun0LWrdrTpt2JbRtGx5GyHD1ZReJ9Q6EMEvuRuA/wNHAHkDawXvO\nuZhts014ZeqyyzZ3cmzYEDpNVq2q/Vv0iCNCG9Tq1Vu8dv7enuycbpiMWbjrWruWrdjAVmwAVnDV\n5Rs474dpliqeNg3GjaMz8JOU4tbbd+fL9gfTv3+1/R98EK6/np7AjSnFl3ANf+ESBg+uVoH88Y9w\n3XW0A4ZX2/86LmHQIHjjjZQfXHkl3H036yubs2hJcyqbtaCypDnP7HgBk3c+jd12g8svT9n/nnvg\nqadYt7GERYtLsBbNadaihE9LT2HFN45im23CLNKbPPMM/Oc/VFoJGytLKGkZXnb4YXDAATWv56uv\nhhq0pCQMdC0pCa9Bg8J4pepmzQqP9zVrtuVrp53S38otWhTucqv223lnaNmy5n4NVdvzvYTFo6re\nNwemZvuMcCFe+DiQ2CQho+Q541ZnzspKae3aMDjm00+lFSvS7zd9uvTQQ9LYsdJdd0m33CJdd10Y\nn5PO/fdLJ52kDd85XkuHHq0v9v2WPttjuF4652Hdeaf0yCPV9v/tbzWpfXuVt+ugNc3aaIO1kEBX\ndL5B3btLRx5Zbf8LL0w7WOdCbhBIw4Zltv9FXC+Qhg7NbP9LSq5Xq1aTMs5zz57X66STpN//vtr+\nF12Udv//HH+97rpLeuaZuvdfO/PD9Nc9BTGPA9mYUsmUmz8v6JwzC88U1zfEvmo25kydeiqceiot\ngE4pxV2Bb6bb/w9/gEMOoaS0lNRJm38t8et0X1W/+Q2cey6rV5Tz0dxy1q8pZ8OqjRy+dS/2b11j\nNQE480w48EA+WVDBP5+qoGJDBRUbylnfZj+O2CrNTcJ3vgPdujHvwwoeuLcClVdAZQUvVxzAuoqN\nrK2++MXQobBqFZ8uqmTC+ApKqKAZlYx7ZzeefSeszPm736Xsv8sucOSRLP2ykjenVNKMSkqo4PYn\n+vDoE2EB0G9/O2X/nj1hn31YtbKSeR9W0m5ZS7ZPdx0bqK4+kApgddUm0BpYE72XpPZ5yNNg3gfi\nnCsGUmhNXL8+vO/QoeY+y5bB9OmwcWPYt+rVpQscdljN/T/8EP72t837V/3Zv3+1Jr7I1Klw/vnw\n2GNhyei6NNo4kGLmFYhzzmUvlwqkziVtXf6krmVQrJKQETxn3DxnvJKSMxdegTjnnMuJN2E555zz\nJiznnHONxyuQAklCu2gSMoLnjJvnjFdScubCKxDnnHM58T4Q55xz3gfinHOu8XgFUiBJaBdNQkbw\nnHHznPFKSs5ceAXinHMuJ94H4pxzzvtAnHPONR6vQAokCe2iScgInjNunjNeScmZC69AnHPO5cT7\nQJxzznkfiHPOucaT1wrEzP5mZkvMbEZKWScze97MZpvZc2bWIeVno81srpnNMrPDU8oHmtkMM5tj\nZjflM3NjSUK7aBIygueMm+eMV1Jy5iLfdyD3AEdUKxsFTJS0C/BvYDSAme0OjAB2A44CbrfNC7Hf\nAZwlaWdgZzOr/pmJM3369EJHqFcSMoLnjJvnjFdScuYirxWIpJeBZdWKjwPujd7fCxwfvT8WeFhS\nuaR5wFxgiJn1ANpJmhLtd1/KMYm1fPnyQkeoVxIygueMm+eMV1Jy5qIQfSDdJC0BkLQY6BaV9wIW\npOy3KCrrBSxMKV8YlTnnnCugYuhE/1o+MjVv3rxCR6hXEjKC54yb54xXUnLmIu+P8ZrZ9sDTkvaO\ntmcBpZKWRM1TkyTtZmajAEm6JtpvAjAGmF+1T1R+MjBc0i9qOd/XskJyzrmGyvYx3ub5CpLColeV\np4CRwDXAGcCTKeUPmNmfCU1UOwGvS5KZrTCzIcAU4HTgltpOlu0FcM45l5u8ViBm9iBQCnQxs48J\ndxRXA4+a2ZmEu4sRAJJmmtk4YCawETg7ZUTgOcBYoBUwXtKEfOZ2zjlXvyY3Et0551zjKIZO9FiY\n2ZFm9l402PDSQuepjZnNM7O3zGyamb1e6DxVsh30WSi15BxjZgvNbGr0OrLAGXub2b/N7F0ze9vM\nzovKi+p6psn5P1F5sV3PrcxscvR35m0zGxOVF9v1rC1nUV3PKmbWLMrzVLSd9fVsEncgZtYMmAMc\nCnxC6Cs5WdJ7BQ2Whpl9CAySVH18TEGZ2YHAKuC+lAcergG+lHRtVCl3kjSqCHOOAVZKurGQ2apE\nD4f0kDTdzLYG3iSMf/oRRXQ968h5EkV0PQHMrI2kNWZWArwCnAd8nyK6nnXkPIoiu54AZnYhMAho\nL+nYXP6+N5U7kCHAXEnzJW0EHib8RShGRhFe9ywHfRZMLTlhywc1CkrSYknTo/ergFlAb4rsetaS\ns2qMVdFcTwBJa6K3WxH6bkWRXU+oNScU2fU0s97A0cDdKcVZX8+i+yLLUfVBiMU82FDAC2Y2xcx+\nUugw9aht0GcxOtfMppvZ3YVuykhlZn2BAcBrQPdivZ4pOSdHRUV1PaPmlmnAYuCFaGaKorueteSE\nIruewJ+BX7HlOLysr2dTqUCSZJikgYTa/5yoSSYpirW983ZgB0kDCH9xi6KpIGoWegw4P/oXfvXr\nVxTXM03Ooruekiol7Uu4kxtiZntQhNczTc7dKbLraWbfBpZEd5913RnVez2bSgWyCNguZbt3VFZ0\nJH0a/fk58Dih+a1YLTGz7rCpvfyzAudJS9LnKY98/xUYXMg8AGbWnPClfL+kqrFORXc90+UsxutZ\nRdJXQBlwJEV4Pauk5izC6zkMODbqj30IOMTM7gcWZ3s9m0oFMgXYycy2N7OWwMmEgYlFxczaRP/a\nw8zaAocD7xQ21RZqG/QJWw76LLQtckb/s1f5HsVxTf8fMFPSzSllxXg9a+QstutpZttUNfuYWWvg\nMEJ/TVFdz1pyvlds11PSZZK2k7QD4bvy35JOA54my+vZJJ7CgvAYL3AzoVL8m6SrCxypBjPrR7jr\nEKGD7YFiyWkpgz6BJYRBn08AjwJ9iAZ9Siro1KK15DyY0H5fCcwDflbVllsIZjYMeAl4m/DfWsBl\nwOvAOIrketaR8wcU1/Xci9Cp2yx6PSLpT2bWmeK6nrXlvI8iup6pzGw48MvoKaysr2eTqUCcc841\nrqbShOWcc66ReQXinHMuJ16BOOecy4lXIM4553LiFYhzzrmceAXinHMuJ16BuCbHzFZGf25vZqfE\n/Nmjq22/HOfnN6Z8XB/39eIViGuKqgY39SMMistYNA13XS7b4kRSkuYyqy7r6+NcKq9AXFN2FXBg\ntGjO+dFMqddGi/5Mr5oN2cyGm9lLZvYk8G5U9ng0Y/LbZvbjqOwqoHX0efdHZSurTmZm10X7v2Vm\nI1I+e5KZPWpms6qOi352tZm9E2W5tnp4M2trZv/PzGZE+3w3Kj8lKpthZlen7J+a5ftmdk/0/h4z\nu9nMXjGz983se+muTyxX3H2t5HVNdOcKbBTRNA0AUYWxXNL+0Zxpr5jZ89G++wJ7SPo42v6RpOVm\n1gqYYmZ/lzTazM6JZlOuouizvw/sLWkvM+sWHfNitM8AYHfCTKyvmNk3gPeA4yXtGh3fPk3+30Z5\nqxbO6mBm2wJXR3mXE5YGOFbSU9Q9O20PScPMbDfCHFL/qH59nMuW34G4r5PDgdOj9RomA52B/tHP\nXk+pPAAuMLPphHU8eqfsV5thhJlNkfQZYSbWqllXX5f0aTQj63SgL7ACWButD/FdYG2az/wW8Jeq\nDUkros+cJGmppErgAeCgaJe6puZ+IvqMWRTBuhmuafAKxH2dGPA/kvaNXjtKmhj9bPWmncIEc4cA\n+0drOEwHWqV8RqbnqrI+5X0F0FxSBWEq/8eA7wAT0nxGbRPV1ZYhdf9W1X6WmqGoVsdzyeUViGuK\nqr4gVwLtUsqfA86O1sDAzPqbWZs0x3cAlklab2a7Agek/GxD1fHVzvUf4KSon6Ur8E3C7LvpA4bz\ndpQ0AbgI2DvNbi8A56Qc0zH6zIPMrHPU4X8K4W4HwnoOu5hZM+C7tZ2b2q+Pc1nxCsQ1RVX/Ep8B\nVJrZNDM7X9JfgZnAVDN7G7gTSPfU1QSghZm9C1wJvJrys/8FZqR0hgtA0uPR+d4CJgK/ipqyasvW\nHnjGzN4iTKl+YZp9/wR0jjrmpwGl0VKjowiVxjRgiqRnov1HA/8EXgY+SXPO6ttbXJ8053euTj6d\nu3POuZz4HYhzzrmceAXinHMuJ16BOOecy4lXIM4553LiFYhzzrmceAXinHMuJ16BOOecy4lXIM45\n53Ly/wExWgTaHayOKQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf35b780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_measures(model_plsa, model_artm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides let's plot the changes of matrices sparsities by iterations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFMX5x/HPA16AwoIKGBBXQFHwWFTQCOp6At6KURGi\nq0k0XlG8QKNiPBIwapBojHiBRyTGeKD+gifrQVQQWBDkMtxyeIECoq7s8/ujemB2dmZ2ZrZnunv3\neb9e/dqtnpqeL706NVPVXSWqijHGGBPTKOgAxhhjwsUaBmOMMdVYw2CMMaYaaxiMMcZUYw2DMcaY\naqxhMMYYU03eGwYR6Ssic0VkvogMSfJ4kYg8JyIzROQDEema70zGGGNSy2vDICKNgPuAPkA3YICI\n7JVQ7QZguqruD5wHjMpnJmOMMenl+xtDT2CBqi5R1UpgHHBKQp2uwFsAqjoPKBaRnfOcyxhjTAr5\nbhjaAcviysu9ffFmAKcDiEhPoAPQPs+5jDHGpBCGwefhQEsRmQZcCkwHNgUbyRhjGq6t8nz8z3Df\nAGLae/s2U9V1wAWxsogsAhYmHkhEbFInY4zJgapKNvXz/Y1hCtBZRHYTkW2As4Hx8RVEpIWIbO39\n/hvgbVVdn+xgqhr6bdiwYYFnsJyWM6oZLaf/Wy7y+o1BVTeJyGXAa7hG6BFVnSMiF7mHdTSwNzBW\nRKqA2cCv8pkp3xYvXhx0hIxYTn9FIWcUMoLlDIN8dyWhqhOALgn7Hoz7/YPEx40xxgQnDIPP9UpZ\nWVnQETJiOf0VhZxRyAiWMwwk1z6oQhMRjUpWY4wJCxFBQzb43OCUl5cHHSEjltNfUcgZhYxgOcPA\nGgZjjDHVWFeSMcbUY9aVZIwxps6sYfBZVPodLae/opAzChnBcoaBNQzGGGOqsTEGY4ypx3IZY8j7\nnc/GGGN8NHkyPPUUrFoFhx0Gl13m+0tYV5LPotLvaDn9FYWcUcgIDTDnvHkwfDgMHgznnANHHw37\n7ANXXJG8/oIFMGoUPPMMTJrkT4YE9o3BGGP8tHIlTJwIn33mPtXHtu7d4a67atafMweuv77m/l13\nTX78nj3hnnugbVvYK3GlZH/YGIMxxtRGFb76ChYuhEWL3NaiBVx8cc26r70GffrU3H/44fD22zX3\nL1gADz0Ebdps2Vq3hp/9DHau+yrHuYwxWMNgjDHg3vwlyfvnlClw1FGwPmGZmAMOgKlTa9ZfuBCG\nDoV27WCXXdwn+7ZtoUOHvH3CT8f3wWcR+TkwCDgM2AXYCMwCXgGeVNVvcsxab5WXl1NaWhp0jFpZ\nTn9FIWcUMkIBcn7/Pbz/vvukPn/+lm377eGjj2rWb93aNQotWsDuu2/eyhs3JmnKjh1d/3+EpWwY\nROQ/wArgReAO4HNgO2BP4EjgRRG5R1XHpzqGMcYEYuNGWLoUuiRZ6mXNGvcNIFGTJlBVBY0SrsnZ\ndVf4+mto2bL6/ogMkuciZVeSiOykql+mfXJmdfoCI9mygtuIhMebA0/i1oZuDNytqmOSHMe6kowx\nNW3a5C7f/OQTmD3b/Vy0CLbaCr77zv2MpwrHHuve8PfYA/bc0/3s3BmaNQvm35BHoRtjEJFGwHzg\naNy3jynA2ao6N67O9UBzVb1eRHYC5gFtVPWnhGNZw2BMQ1VVBZ9+6t68Ez/Rq0KrVrB27ZZ9jRtD\np05usLdt28JmDRlfJ9ETkXUi8m2qLcPj9wQWqOoSVa0ExgGnJNRRYAfv9x2ArxIbhShpcNdg55nl\n9E8UMoKXc8YMeOQRd/NWr17QvLnrFvr005pPEHH1brnF9e3Pnu2+Kcybl9dGISrnMxcpxxhUdQcA\nEbkNWAk8AQgwEDcQnYl2wLK48nJcYxHvPmC8iKwAtgfOyvDYxpj66uKL3QBxvHbtYPVq1/WT6Lbb\nCpOrgcjkBreTVXX/uPIDIjIDuNmnDH2A6ap6lIh0Al4Xkf1UdX1ixbKyMoqLiwEoKiqipKRk89UL\nsdbbypmVY/vCkifq5di+sORJVY7PWvDX/+knSps3hylTKH/5ZZg/n9J774UTT6xWv7S0lPKuXaFJ\nE0r79IHu3SnfsAGKiig97LDg8icpx4QlT2lpKeXl5YwZMwZg8/tltmodYxCR/wL347qBFBgAXKqq\nh9Z6cJFDgFtUta9XHgpo/AC0iLwM/ElVJ3nlN4EhqvpRwrFsjMGYqLrjDrdt3Fh9/003wa23BpOp\ngcjXQj3nAGcCq73tF96+TEwBOovIbiKyDXA2kHh56xLgGAARaYO7HHZhhscPncRPEmFlOf0VhZx5\nz7h0qZveIZkdd3SNQpcucN55bq6fSZNgyJDC5/RJVHLmotauJFVdTM0B44yo6iYRuQx4jS2Xq84R\nkYvcwzoauB0YIyIzvaddp6pf5/J6xpgCWrXKzQn01lvu5//+B/37w7PP1qw7YACccQbstFPhc5qs\nZdKVtCfwAO4S0n1EZD/cuMPthQgYl8O6kowJi3ffdXP/xGve3L35P/JIMJlMUnm5j0FE3gauBR5U\n1e7evlmquk/OSXNgDYMxBabq5v3p1KnmYxs2QPv2cPDBcOSR7k7i7t1r3kxmApevMYamqjo5YV9k\n7zPIt6j0O1pOf0UhZ0YZN2yA8ePhoovcpG9dulS/cSymWTP48kuYMMGNE/To4VujEIVzCdHJmYtM\n/pJfepeRKoCInIG7r8EYU58MHOjGB378ccu+tm3dZHM9etSs37hx4bKZgsqkK6kjMBo4FFgDLAIG\nquqS/MerlsO6kozJp/PPh7FjXffQ8cfDCSdASUnNKShMpPg+xuDNdXSGqj4jIs2ARqq6ro45c2IN\ngzF1oAoffwzPPw/77QennVazztKlboZRHxaHMeHh+xiDqlYB13m/bwiqUYiSqPQ7Wk5/hTJnrDG4\n6Sbo0oXy/fd38wk9+GDy+h06hKJRCOW5TCIqOXORyRjDGyJyDfBPYENsp91rYEzIvfsuHHHElnLz\n5nDmmfCLXwSXyURCJmMMi5LsVlXtmJ9IKXNYV5Ix2fjpJ9hnH3e/wZlnQmmpXU7aAIVuPQY/WcNg\nTIJvv3XTTD/1lPuZrBso1TrGpsHwez2G3rW8WHMRKehNblEQlX5Hy+mvguXctAneeAMGDXKXkv7m\nN26JyaefTl4/rlGwc+mvqOTMRbrvlf1F5E5gAjAV+AK35nNn3JrPuwFX5z2hMWaL666De+7ZUi4t\nhbIyN0eRMT6p7XLVVkB/oBducZ6NwBzgFVV9ryAJt2SxriRjJk2Cc891M5Seey7kON++aThsjMGY\n+uC77+Cdd6Bv35qPqbrNbjozGcrXXEkmC1Hpd7Sc/vIl54oV8Pvfu/sJjj/eTWOdSCTnRqFBncsC\niErOXNi1a8YEbeZMuPNO+Oc/3SWmAAcdBGvWBJvLNFjWlWRM0C6/HO67z30TOP10uPJKOPRQu8zU\n+CKXrqScvjGISFtVXZVh3b7ASLas4DYi4fFrgIG42Vu3BvYGdlLVJHP9GlMPDRniZiq98kobTDah\nkOsYQ0ZLNHmT8N0H9AG6AQNEZK/4Oqp6l6p2V9UDgOuB8ig3ClHpd7Sc/soo59y5yfe3bw8jR+a9\nUahX5zIEopIzFzk1DKp6QoZVewILVHWJqlYC40i/fvQAIMWdOsZE1IcfQr9+sPfe8MEHQacxplYZ\njTGIyAFAb1x3zyRVnZbRwUX6A31U9UKvPAjoqaq/S1K3CbAc6JTsG4ONMZjIWbIEhg6FceNcuVkz\nuP9+dw+CMQWSlzEGEbkZ+AXwnLfrMRH5l6renkPGdE4C3kvXjVRWVkax93W7qKiIkpISSktLgS1f\n66xs5VCU77wTbryR0spK2HZbyk8/Hc46i9JTTglHPivX23J5eTljxowB2Px+mTVVTbsB84Dt4spN\ngHm1Pc+rewgwIa48FBiSou5zwNlpjqVRMHHixKAjZMRy+qtGzjVrVHfeWXXAANXFiwPJlCiy5zKk\nopLTe++s9f06fsvkqqQVuDmSvvfK2wKfZdjuTAE6i8huuHWiz8aNI1QjIi2AI3BXJxkTfUVFMGcO\n7Lhj0EmMyVom6zG8APQAXseNMRwLTMaNB6BJxgsSnt8XuJctl6sOF5GL3FN1tFfnPNxYxDlpjqO1\nZTWm4JYuhS+/hAMOCDqJMUnlZa4k7007JVUdm80L5soaBhMqqvDYY+7eg913h2nT3L0IxoRMXuZK\nUtWxsQ0YD0xP2GfixAaBws5y1sHKlXDSSfCrX8G6ddCpE+Wvvhp0qlqF8lwmYTmDV2vDICLl3qI8\nrYBpwEMick9tzzOmXnr2WejWDV55xY0jPPkk/Pvf0LRp0MmM8U0mXUnTVbW7iPwa2FVVh4nITFXd\nrzARN+ewriQTvLFj3cI4ffvCww9Du3ZBJzImrXzNlbSViOwCnAn8PqdkxtQX554LrVu7hsEmuTP1\nVCZTYtwKvAp8qqpTRKQjsCC/saIrKv2OljNHIm56i4RGIXQ5k4hCRrCcYZDJ4PO/VHU/Vb3EKy9U\nVVtg1tRvCxbASy8FncKYQNh6DMYkev55N45QWQmTJ8M++wSdyJic2dKextRFZSVce61bLOfbb12X\nUYcOQacypuCsYfBZVPodLWeCFSvg6KPhrrvcjWp33+0uTW3ePKOnR+F8RiEjWM4wSHlVkogMUtUn\nReSqZI+rqt3LYOqPr76Cjz6CXXaBZ56B3r2DTmRMYFKOMYjIRar6oIgMS/a4qv4hr8lq5rExBpNf\nr74KJSXQpk3QSYzxTV7mSgoLaxiMMSZ7eRl8FpGdReQGERktIo/Gttxj1m9R6Xds0DkrK30/ZBTO\nZxQyguUMg0wGn18EWgBvAK/EbcZEz4oVcPDB4K1wZYypKZO5kipUtaRAedLlsK4kUzezZ7tLUJct\ng65doaICtt466FTG5FW+7mN4WUSOzzGTMeFQXg69erlG4ec/h7fftkbBmBRSNgwisk5EvgWuwDUO\nG0Xk27j9GRGRviIyV0Tmi8iQFHVKRWS6iMwSkYnZ/zPCIyr9jg0q53PPQZ8+8M03cNpp8OabsNNO\ndT9unCiczyhkBMsZBinvY1DVHep6cBFpBNwHHI1bO3qKiLyoqnPj6rQA7geOU9XPRMTf/2ON6dIF\nmjSB3/4W7rnHVlozphbp7mNoDdwAdAZmAsNVNeNvCt4xDgGGqWo/rzwUt9bziLg6FwO7qOrNtRzL\nxhhM7pYtg113DTqFMQXn9xjD48AG4K/ADsCoHDK1A5bFlZd7++LtCbQSkYkiMkVEfpnD6xiTnjUK\nxmQs3UI9u6hqbGGeV0VkWh4zHAAcBTQD3heR91X108SKZWVlFBcXA1BUVERJSQmlpaXAlv6+oMux\nfWHJk6o8cuTIUJ4/O5/5K1dUVHDllVeGJk+qcuLfPug8qcphPZ/l5eWM8S7Hjr1fZk1Vk27ADKAl\n0MrbqpVTPS/hGIcAE+LKQ4EhCXWG4LqbYuWHgf5JjqVRMHHixKAjZKTe5rzlFtX7789LlnSicD6j\nkFHVcvrNe++s9f06fks3xrAYqAKS9U2pqnasrdERkcbAPNzg80pgMjBAVefE1dkL113VF9gW+BA4\nS1U/STiWpspqDAC33QY33+wGl+fOhc6dg05kTOB8XfNZVYvrGkhVN4nIZcBruPGMR1R1johc5B7W\n0ao6V0RexQ1wbwJGJzYKxtTqj390jUKjRvD449YoGFMHeV+PQVUnqGoXVd1DVYd7+x5U1dFxde5S\n1W7qlhD9a74z5VN8/2iY1aucd94Jv/+9W4d57Fg455y850oUhfMZhYxgOcPAFuox0fbVV25xHRF4\n7DEYNCjoRMZEnk27baLvk09g6lT4pV3pbEwiX9djEJFW6Z6oql9n80J1ZQ2DMcZkz+8b3L4EKoCP\nvG1q3PZRriHru6j0O1pOf0UhZxQyguUMg3Q3uI0CjgQmAU8D79lHdhM4VTeeYIzJm7RjDCIiQCkw\nAOiJu+z0AVVdVJB01bNYu9TQqcKFF0KnTjBkiDUQxmQgb2s+i0gRcDZwG3CDqj6UW8TcWcNg+Pvf\n4eKL3Uyps2ZBx1rvsTSmwfN1jEFEmonIOSLyIvB/wPbAgUE0ClESlX7HyOV8/3343e/c7w8/HLpG\nIQrnMwoZwXKGQboxhs+BBcA476cCB4nIQQCq+lz+4xkDrF4NZ5wBlZVwxRWB3MBmTEOS7nLVx9I8\nT1X1gvxESs66khqwM86Af/8bDjvMrb5mS3IakzFf50oCXrJvBSYURo6En35yYwzWKBiTd+nuY7ix\nYCnqkaj0O0YqZ/v28MIL0LZt0HFSisL5jEJGsJxhYHMlGWOMqSbdGMN3QI1V1HDrM6iq7pfPYEny\n2BiDMcZkye8xhkXASXWLZEwOPv8cdt7ZbmAzJiDpupJ+VNUlqbZMX0BE+orIXBGZLyJDkjx+hIis\nFZFp3hbpsY2o9DuGNmdlJZSWwlFHwcqV4c2ZIAo5o5ARLGcYpPvGMKmuBxeRRsB9uKU9VwBTRORF\nVZ2bUPUdVT25rq9n6oEHHoA5c1wD0SrtBL/GmDypba6kxkBLVf3SK28DlAGDVXXvWg8ucggwTFX7\neeWhuPGJEXF1jgCuUdW03VY2xtAAfPkl7LEHrF0L48fDSdaTaUxd+T0lxlnA18BMEXlbRI4DFgL9\ngIEZHr8dsCyuvNzbl+jnIlIhIq+ISNcMj23qm5tvdo3CccfBiScGncaYBivdGMNNuLmRfgYMBl4C\nLlbV01R1mo8ZpgIdVLUE1+30go/HLrio9DuGLuesWfDgg9C4MfzlL5sHnkOXM4Uo5IxCRrCcYZBu\njOFHVf0UQFWnicgCVX0py+N/BnSIK7f39m2mquvjfv+PiPxNRFolWyGurKyM4uJiAIqKiigpKaG0\ntBTY8kcKuhwTljypyhUVFaHKU75yJVx4IaWtWkHXrnY+81CuqKgIVZ6ol8N6PsvLyxkzZgzA5vfL\nbKW7j2E5cE/crqviy6p6T40n1TxGY2AebvB5JTAZGKCqc+LqtFHV1d7vPYFnVLU4ybFsjMEYY7Lk\n930MDwE7pChn9A6tqptE5DLcAj+NgEdUdY6IXOQe1tHAGSJyMVAJbATOyuYfYIwxxl8ZLdRT40ki\nPVR1Sh7ypHvNSHxjKC8v3/z1Lswsp7+ikDMKGcFy+s3vbwyJB++KW+JzALAWOCi7eMYYY6KgtvsY\nitnSGFQCuwEHqeriAmRLzBKJbwwmC9ddB/vv7xbesekvjMkLX9d8FpH3gea4FdzGqeoCEVmkqrvX\nPWr2rGGoZ95/Hw49FLbdFubPhw4dan+OMSZrvt7gBqzGDTa3AXb29tk7cy0SL7MMq0BzLlzoVmUD\nuOqqtI2CnU//RCEjWM4wSNkwqOqpwL64G9BuEZFFQEvvklJjcrN8ORx9NKxYAYcfDjfdFHQiY0yC\njK9KEpHWwJm48YYOqrprPoMleX3rSoo6VTdz6jvvwMEHw+uvww471Po0Y0zufB1jqOWFdstm6m0/\nWMNQT8ybB1dfDU88AS1bBp3GmHrP7zGGlArdKERJVPodA8vZpQu8/HLGjYKdT/9EISNYzjCwNZ+N\nMcZUk1NXUhCsKymCqqqgkX32MCZIBelKEpFLROQsEcn4rmnTAFVWuktSR4yova4xJlRy+TgnQG/g\nOZ+z1AtR6XfMa86qKrjgAnj+eRg+HFatyvlQdj79E4WMYDnDIOtP/ap6fz6CmHrk97+HJ5+EZs3g\nP/+Btm2DTmSMyUKtYwwichBwGPAz3LTYs4DXVXVN/uNVy2FjDFHw4IPw29+6ldheeQX69Ak6kTEN\nmt9zJZ0PXA4swt39/DmwHbAn0AvXQNykqkvrEjrjoNYwhN9337nLUZcvh4cfhl/9KuhExjR4fg8+\nNwV6qWp/Vf2jqj6sqvep6u9U9UDgL8AedQlcH0Wl3zEvOZs2hUmT4L77fGsUGvT59FkUMoLlDIN0\ncyXdr6ob0zxeoapv1vYCItJXROaKyHwRGZKmXg8RqRSR02uPbUKrQwe49NKgUxhj6iCTMYY9gQeA\nNqq6j4jsB5ysqrfXenCRRsB83JrPK4ApwNmqOjdJvddxYxiPqmqNK56sK8kYY7KXr/sYHgKuxy3U\ng6rOBM7O8Pg9gQWqukRVK3FrO5ySpN7lwLO4cQwTFVVVQScwxuRBJg1DU1WdnLDvpwyP3w5YFlde\n7u3bTER+Bpyqqg/g7pGItKj0O9Y5pyqcf767NDWP3+QazPksgChkBMsZBpncx/CliHTCW6RHRM4A\nVvqYYSQQP/YQ+cahQbjlFnj8cXevwgUXQKdOQScyxvgkk4bhUmA0sJeIfIa7fHVQhsf/DIhfnqu9\nty/eQcA4ERFgJ6CfiFSq6vjEg5WVlVFcXAxAUVERJSUllJaWAltabytnVo7ty+n5Y8dSfuutIELp\nM89Ap06B/3uCLsf2hSVPqnJ81jDkSVYuLS0NVZ505Ziw5ImduzFjxgBsfr/MVjYL9TQDGqnquowP\nLtIYmIcbfF4JTAYGqOqcFPUfA16ywecQe/ddtwJbZSX87W9w8cVBJzLGpJGXwWcRuVlEbgauBgbH\nlWulqpuAy4DXgNnAOFWdIyIXiciFyZ6SRfZQSvwkEVY55VSFwYNdo3DllQVpFOr1+SywKGQEyxkG\nmXQlbYj7fTvgRCDpJ/5kVHUC0CVh34Mp6l6Q6XFNAETcNBcjR8LttV6tbIyJqKzXYxCRbYFXVbU0\nL4lSv651JRljTJYKtbRnU9wgsjHGmHookzGGj0VkprfNxg0mj8x/tGiKSr+j5fRXFHJGISNYzjDI\nZIzhxLjffwJWq2qmN7iZKHv5ZXjzTbjrLjeNtjGmQUg37XardE9U1a/zkigFG2MosJkzoVcvWL/e\nLbozcGDQiYwxOchljCHdN4apuMtHkx1QgY7ZvJCJkFWr4KSTXKMwYACcc07QiYwxBZRu2u3dVbWj\n9zNxs0Yhhaj0O6bM+d13cMopsHQp/Pzn8Oij7jLVgET+fIZIFDKC5QyDjNZ89tZI6I37pvCuqr6Q\n11QmODfeCJMnQ3ExPP88bLdd0ImMMQWWyXoMfwM6A097u84C/qeqBV2NxcYYCmTNGrf62h13wN57\nB53GGFNHvq75HHfQucDesXdlb1Gd2apa0HcNaxiMMSZ7+brB7VOqz5C6q7fPJBGVfkfL6a8o5IxC\nRrCcYZByjEFEXsKNKewAzBGRyV75YNwsqcYYY+qhdPcxHJHuiar6dl4SpWBdSXmweDHcey+MGAHb\nbBN0GmNMHvh9H8M7tb0Ti71bR9fatXD88TBnDjRpAn/8Y9CJjDEhkW6MYaKIXC4i8eMLiMg2InKU\niIwFzstvvOiJRL/jjz9SfuSRrlHo1g2uuy7oRClF4nwSjZxRyAiWMwzSfWPoC1wAPC0iuwNrgSa4\nxuQ1YKSqTs9/ROMrVbjwQqiogLZt3foKRUVBpzLGhEhG6zGIyNa49Zg3qurarF5ApC9uNtZGwCOq\nOiLh8ZOB24AqoBIYrKqTkhzHeq38cP/9cNll0LQpvPMOHHhg0ImMMXmUl/sY6sK752E+bs3nFcAU\n4GxVnRtXp6mqfuf9vi/wTLJ7JKxh8MmGDXD22XDJJdCvX9BpjDF5VqiFerLRE1igqktUtRIYB5wS\nXyHWKHi2x31ziKzQ9zs2awbjx1PepEnQSTIS+vPpiULOKGQEyxkG+W4Y2gHL4srLvX3ViMipIjIH\neAk3rmHyKcBJ8Ywx4ZfRJHr55k3K94KI9AZuB45NVq+srIzi4mIAioqKKCkpobS0FNjSels5s3Js\nX1jyRL0c2xeWPKnK8VnDkCdZubS0NFR50pVjwpIndu7GjBkDsPn9Mls5jTGIyGhVvTCDeocAt6hq\nX688FNDEAeiE5/wP6JG4EJCNMeRg0iQYNQrGjHH3KhhjGpxCjjE8mGG9KUBnEdlNRLYBzgbGx1cQ\nkU5xvx8AbFPo1eH8lPhJIjAzZsAJJ8Azz7grkRKEJmctLKd/opARLGcYZNyVJCLNcZ/216nq1Eye\no6qbROQy3H0PsctV54jIRd6xRgP9ReRc4EdgI3Bm1v8KU92CBdCnD3zzDfTvD4MHB53IGBMhmUy7\n3QN4FDeZnuBudLsg08bBL9aVlKFly+Cww2DJEjjmGHj5Zdh226BTGWMC4vdcSTGPAJeo6rvei/QG\nHgP2yz6iybvhw12jcPDBbgU2axSMMVnKZIxhU6xRAFDV94Cf8hcp2gLvd7z7brjqKvjPf2D77VNW\nCzxnhiynf6KQESxnGGTyjeFtEXkQt7Sn4pb2LPcGilHVaXnMZ7K13XaucTDGmBxlMsYwMc3DqqpH\n+RspZQ4bYzDGmCyFbq4kP1nDkMSbb7qxhDRdRsaYhi0v9zGIyBUi0lych0Vkmogcl3vM+q1g/Y6P\nP+4uST3lFPjxx6yfHpX+UcvpnyhkBMsZBpkMPl+gqt8CxwE7Ar8Ehuc1lUnvr3+F886DTZugVy/Y\neuugExlj6pFMxhhmqup+InIvUK6qz4vIdFXtXpiIm3NYV1JVFfzhD3Drra58111w9dXBZjLGhFpe\nxhhE5DHcjKi7A/sDjXENREFXeLGGATfn0fnnu9lRR4+GX/866ETGmJDL11xJvwKG4ia2+w7YBjg/\nh3wNQl77HQcNclNcvPxynRuFqPSPWk7/RCEjWM4wqPU+BlWtAqbFlb8CvspnKJPCVlvBs88GncIY\nU8/Z5arGGFOPhXFpT5OLtWtdV9Hq1UEnMcY0QCkbBm+abUSkVbKtcBGjpc79jrNnQ8+e8Mgj8Nvf\n+pIpmaj0j1pO/0QhI1jOMEg3xvAP4ERgKm6OpPivIgp0zGOuhmncOPdNYcMGKCmBe+4JOpExpgGy\nMYYwUIUrrnA3rgGccw489BA0bRpsLmNM5OVtjEFE2onIoSJyeGzLIlRfEZkrIvNFZEiSx88RkRne\n9p6I7JvNP6BeEIEWLdwdzPffD08+aY2CMSYwmcyVNAKYBNwIXOtt12RycBFpBNwH9AG6AQNEZK+E\naguBw1V1f+B24KGM04dQzv2Ot9wC06fDJZe4hiLPotI/ajn9E4WMYDnDIJP1GE4FuqjqDzkcvyew\nQFWXAIj2KbwzAAAT4klEQVTIOOAUYG6sgqp+EFf/A9xd1g1P48bQrVvQKYwxJqMpMf4D/EJV12d9\ncJH+QB9VvdArDwJ6qurvUtS/BtgzVj/hsfoxxvD557B0KRx0UNBJjDENgK9rPovIX3FXH30HVIjI\nm8Dmbw2p3txzJSJH4qba6J2qTllZGcXFxQAUFRVRUlJCaWkpsOVrXajLM2ZQeuedsGkT5fffDzvu\nGK58VraylSNfLi8vZ8yYMQCb3y+zpqpJN+C8dFuq5yUc4xBgQlx5KDAkSb39gAVApzTH0iiYOHFi\nzZ2VlarDhqk2aqQKqr17q65YUeho1STNGUKW0z9RyKhqOf3mvXfW+n4dv6X8xqCqY0WkBOgMzFbV\nOTm0O1OAziKyG7ASOBsYEF9BRDoA/wZ+qar/y+E1wm3ZMhg4EN591w0q33CDG2i2NRSMMSGVcoxB\nRG7CLcozFTgY+JOqZn3FkIj0Be7FXQH1iKoOF5GLcK3YaBF5CDgdWIK7ia5SVXsmOY6myhpqb70F\nxxwDbdvCE0/A0UcHncgY04D4uh6DiMzGm2pbRHbEdQn18CFnTiLbMAA89RQcdxzsvHPQSYwxDYzf\nN7j9oG79hdhU2zbhXgZig0DVDBwYukYhac4Qspz+iUJGsJxhkO4+ho4iMt77XYBOcWVU9eS8Josa\nVZg6NegUxhhTZ+m6ko5I90RVfTsviVIIdVfS2rVw4YVuEZ033oCjjgo6kTHGAD7fx1DoN/7I+uAD\nGDAAFi+GHXaAb74JOpExxtSJjRvkqqoKRoyAww5zjcJBB8H06ZS3bBl0soxEpX/UcvonChnBcoaB\nNQy5+vprt17CTz/B1VfDpEnQqVPQqYwxps6yXo9BRLYDTlLVf+UnUsrXDd8Yw5tvwg8/wPHHB53E\nGGOS8vU+hoQDN8ZNnT0AOA54V1XPyClljkLZMBhjTMj5vlCPiBwhIg8Ci4FfAccCuxe6UQjc6tWw\naVNGVaPS72g5/RWFnFHICJYzDFI2DCKyHPgT8B7QVVX7AxtjN701GG+8AfvuC7ffHnQSY4wpiHT3\nMYzELdIzC/gH8CLwsap2LFy8ankK25VUVeUag1tucTev9ekDr7ziFtQxxpiI8H2MQUQEKMWNLRwP\ntMB1Kf2f5rBwT10UtGH44gsYNAhee83NiHrzzXDTTdYoGGMix/cxBm8674nqVlTbHddAnIIbc6i/\nrrjCNQo77QQTJrhvDRk2ClHpd7Sc/opCzihkBMsZBpms+RzvM2AwsC4PWcLj7rvh++9h1Cho3z7o\nNMYYU1Dpxhj+DvxVVWeLSAvgfWAT0Aq4RlWfLlxMu1zVGGNy4XdX0mGqOtv7/XxgvqruCxwIXJdj\nRmOMMSGXrmH4Me73Y4EXAFR1VTYvICJ9RWSuiMwXkSFJHu8iIv8Vke9F5Kpsjl1nr73mJsDL8B6F\nTESl39Fy+isKOaOQESxnGKRrGNaKyIki0h3oBUwAEJGtgCaZHFxEGgH34e6a7gYMEJG9Eqp9BVwO\n/DnL7LlThT//Gfr1g3Hj4PHHC/bSxhgTdunGGPYERgFtgZGqOsbb3wc4TlWvrvXgIocAw1S1n1ce\nirvYaUSSusOAdap6T4pj+TPGsGED/PrXrkEAdynqsGHQyOYTNMbUP36vxzAf6Jtk/6sisneGx28H\nLIsrLwd6ZhPQV59/7tZenjEDtt8enngCTj01sDjGGBNG2V6uGnMVMNLPIJkoKyujuLgYgKKiIkpK\nSigtLQW29PelLW/aROmOO8Iee1B+ww1QVESpd+yMnp9BObbPr+Plqzxy5Mjsz18A5di+sOSJ8vms\nqKjgyiuvDE2eVOXEv33QeVKVgz6f69dDjx6ltGlT8/yNGTMGYPP7ZdZUNesNWJZhvUOACXHlocCQ\nFHWHAVelOZb64ssvVdes8edYSUycODFvx/aT5fRXFHJGIaOq5UznhRdUzzlHdc89VUH10ktrf473\n3pnVe3zW6zEAiMhSVe2QQb3GwDzgaGAlMBkYoKpzktQdBqxX1btTHEtzyWqMMVGyYQNs3OgmXkh0\n881w223u9222gYED4dFH0x/P17mSRGQdoEDsgLGKAjRR1Yy6oUSkL3Av7gqoR1R1uIhchGvFRotI\nG+AjYAegCliPm811fcJxsmsYvv3Wra7WqlXmzzHGmALasAGmTYOpU7f8nDsXLr4Y7ruvZv2KCpg8\n2a0kvM8+rnGoTS4NQ05dSUFsZNOVtGiR6j77qB5xhOoPP2T+PB/Y12B/WU7/RCGjasPK+fTTrkso\nfttqK9Vzz617vhhy6EpK+anfW8Lzt0BnYCbwqKr+lFWrE4T333dXGn3+Oey1F3z5JfzsZ0GnMsY0\nEGvWwKxZ8PHHbps503VcvPRSzboHHQQHHOC2Aw902777wnbbFT53vHRdSf8EKoF3gX7AElW9ooDZ\nEvNoqqyb/eMfcMEFbh3mY4+FZ56BoqLCBDTGNHhz58LeSS7mb9kSvvrKzeJfaH6PMXysbm6k2N3O\nk1X1gLrHzE2tDcOrr0Jf77aLSy6Be++FrXK9GtcYY5yNG90b/iefwOzZbluxwvX1J77RV1ZC69bQ\nubMbA9hvP7ftu6/bHwRfxxiAaenKhd6obYzhp59UTz1VddSojPrd8qUh9Y8WguX0TxQyqoYrZ2Wl\n6nbb1RwHANV//3ti0udUVRU2Y23wc4wB2F9Evo01OkATryzeCzXPsuHKr8aN4bnngvmuZoyJDFX3\niT9+DODjj12nQ5s21etutZUbqvz+e+jWDbp2dT+7dYPVq5Mfvz68BeV0H0MQ7D4GY4wfevWC//63\n5v7XX4djjqm5v6oq2lOp+TpXUqgtWQJNmgTXaWeMCY1Fi1y//8KF8L//uZ/z58MDD8BRR9Ws3769\nGwzed98t/f+xsYBkotwo5CzbvqegNmJjDDNmqO6yi2qPHqrr1tW5/81vYeofTcdy+isKOaOQUbV6\nzqoq1VWrVP/7X9Vly5LXHzgw+RhAquHGjRv9GQeIyvnE5zGG8HnnHTj5ZPjmG+jSxdcFdowx4fDW\nWzBypPvkv3ChuzsY4C9/AW/OumoOPhi++AI6doROnbb83HPP5McP+h6BKIjWGMO227p7FM44w02Z\nbX9hY0Lvu+9cF8+iRe6NfvFi1xt88slw/vk1699xB9x445Zyy5buzf6SS9xtSiY79X+M4Ycf3H8d\no0a5q5CMMYGrqoLly10Hzm671Xz8b3+Da6+tub916+QNw2mnuU/7HTu6rWVL/zOb9KI1rHLrrW5m\nqRA3CvFzyYeZ5fRXFHL6lXHGDBg82H3i79oVmjZ1DcLNNyevv+eerue3b1/3ue7OO92kBIMHJ6//\n+efl/OIXbnqIMDcKUfib5ypa3xhuuinoBMbUexs3uqt8vv8eeveu+fjy5W4MIF7bttCsWfLjnXyy\n20x0RGuMISJZjYmSVavg4YfdTV4zZsCCBa57qEcPN+1DohUr4Mkn3QBv587u5/bbFz63yYyvcyWF\njTUMxuRG1U02/NlnbhbPRIsWub78mMaN3d2+hxziGgwTbbk0DHkfYxCRviIyV0Tmi8iQFHVGicgC\nEakQkZJ8Z8qnqPQ7Wk5/hSnnDz+4Vb5++Uvo2dP107dtC716lVNVVbP+brvBddfBY4+5xWLWr3fT\nRgfVKITpXKYTlZy5yGvDICKNgPuAPkA3YICI7JVQpx/QSVX3AC4C/p7PTPlWUVERdISMWE5/5Tvn\nN9+42T1ffx3GjHGXdF5yiWsEEm2zDYwY4bp7pkxxzy0qgtatK1izpmb9Ro1c/bIy6N49+KvA7W8e\nvHwPPvcEFqjqEgARGQecAsyNq3MK8DiAqn4oIi1EpI2qppiiKtzWrl0bdISMWE5/ZZvziy/cGlJf\nf+22r75y3T0XXww77FCz/t57w8qVNfdfdx0UF1ffJ+K+MTRv7q4G6tLFrR/8hz+sZccds4oZiPr6\nN4+SfDcM7YBlceXluMYiXZ3PvH01GoapU2u+wF57Jb8aYs4cd2NNoev/+GPNfUHmSVX/iy8a1vmc\nNcvdQZs4ccL++ycfOH3/ffdJu6rK3WD/449u69s3+SWUH34I11zjruj5/nv3c+NGd/VOsmv7e/d2\n8/kkOvFEdwlooo4dXYPRrp1bkLBdO7cla0Qg9aWgxmQk2zk0stmA/sDouPIgYFRCnZeAQ+PKbwAH\nJDlW0vlQPvoo+fwgBxyQfP6UfNc/4YTzQpUnVf1Wrc4LVZ58n8/u3f2pP3Vq8votWyY/n1OmJK/f\nr59qly6qhxyievzxqoMGqV59terSpcnr++G8887L38F9ZDn9RQ5zJeX1qiQROQS4RVX7euWhXsgR\ncXX+DkxU1X965bnAEZrQlSQidkmSMcbkQEM2JcYUoLOI7AasBM4GBiTUGQ9cCvzTa0jWJjYKkP0/\nzBhjTG7y2jCo6iYRuQx4DXcF1COqOkdELnIP62hV/T8ROV5EPgU2AElmTzHGGFMokbnBzRhjTGFE\nYhK9TG6SCwMRWSwiM0RkuogkmUwgGCLyiIisFpGZcftaishrIjJPRF4VkRZBZvQyJcs5TESWi8g0\nb+sbcMb2IvKWiMwWkY9F5Hfe/lCdzyQ5L/f2h+18bisiH3r/z3wsIsO8/WE7n6lyhup8epkaeVnG\ne+Wsz2XovzF4N8nNB44GVuDGLc5W1blpnxgAEVkIHKiqSW4jCo6I9AbWA4+r6n7evhHAV6p6p9fY\ntlTVoSHMOQxYp6r3BJktRkTaAm1VtUJEtgem4u7FOZ8Qnc80Oc8iROcTQESaqup3ItIYmAT8DndF\nY2jOZ5qc/Qjf+RwMHAg0V9WTc/l/PQrfGDbfJKeqlUDsJrkwEkJ4TlX1PSCxsToFGOv9PhY4taCh\nkkiRE9x5DQVVXaWqFd7v64E5QHtCdj5T5GznPRya8wmgqrE7UrbFjXsqITufkDInhOh8ikh74Hgg\nfkKTrM9l6N7Ekkh2k1y7FHWDpsDrIjJFRH4TdJhatI5d/aWqq4DWAedJ5zJvHq2Hg+5SiCcixUAJ\n8AHQJqznMy7nh96uUJ1Pr+tjOrAKeF1VpxDC85kiJ4TrfP4FuJYtjRbkcC6j0DBESS9VPQDXYl/q\ndY1ERVj7FP8GdFTVEtz/kKH4yu51zzwLXOF9Ik88f6E4n0lyhu58qmqVqnbHffPqKSLdCOH5TJKz\nKyE6nyJyArDa+6aY7ltMrecyCg3DZ0CHuHJ7b1/oqOpK7+cXwPPUnP4jTFaLSBvY3B/9ecB5klLV\nL3TLQNhDQI8g8wCIyFa4N9snVPVFb3fozmeynGE8nzGq+i1QDvQlhOczJj5nyM5nL+Bkb6zzaeAo\nEXkCWJXtuYxCw7D5JjkR2QZ3k9z4gDPVICJNvU9niEgz4DhgVrCpqhGqf4oYD5R5v58HvJj4hIBU\ny+n9hxxzOuE4p48Cn6jqvXH7wng+a+QM2/kUkZ1i3S8i0gQ4FjceEqrzmSLn3DCdT1W9QVU7qGpH\n3PvkW6r6S9y0Q2VetYzOZeivSgJ3uSpwL1tukhsecKQaRGR33LcExQ1MPRWWnCLyD6AU2BE3OeEw\n4AXgX8CuwBLgTFUNdLrIFDmPxPWPVwGLgYuS3RlfKCLSC3gH+Bj3t1bgBmAy8AwhOZ9pcp5DuM7n\nvrgB0Ube9k9VvUNEWhGu85kq5+OE6HzGiMgRwNXeVUlZn8tINAzGGGMKJwpdScYYYwrIGgZjjDHV\nWMNgjDGmGmsYjDHGVGMNgzHGmGqsYTDGGFONNQwmMkRknfdzNxFJXAmwrse+PqH8np/HL6R8nB/T\nsFjDYKIkdtPN7rgbtTLmTZWczg3VXkg1SvNcJcr6/BgTzxoGE0V/Anp7i5Fc4c16eae3kEpFbGZb\nETlCRN4RkReB2d6+573Zbz8WkV97+/4ENPGO94S3b13sxUTkz179GSJyZtyxJ4rIv0RkTux53mPD\nRWSWl+XOxPAi0kxEHhWRmV6d07z9A7x9M0VkeFz9+Cz9ReQx7/fHROReEZkkIp+KyOnJzo8vZ9w0\nLKpqm22R2IBvvZ9HAOPj9v8GuMH7fRvc/Fq7efXWAR3i6hZ5P7fDTRfRMv7YSV6rP/Cq93tr3JQC\nbbxjrwF2wc3t9F/gUKAVMDfuOM2T/DuGA/fElVt4x1niPb8R8CZwcmI2L8+j3u+P4aZmANgbt25J\njfNjm23ZbvaNwdQHxwHnenPlf4h7c93De2yyqi6Nq3uliFTg1lBoH1cvlV64mSpR1c9xs2rGZtCc\nrKorVVWBCqAY+AbY6M3NfxqwMckxjwHujxVU9RvvmBNV9WtVrQKeAg73qqSbQvkF7xhzCMGaBaZ+\nsIbB1AcCXK6q3b2tk6q+4T22YXMlN7HYUcDB6ubPr8B9c4gdI9PXivkh7vdNwFaqugk33fqzwInA\nhCTHSDVBWaoM8fW3S3gsPkNoVhIz0WYNg4mS2BvfOmCHuP2vApd46w8gInuISNMkz28BrFHVH0Rk\nL+CQuMd+jD0/4bXeBc7yxjF2Bg7DzaSaPKB73SJVnQBcBeyXpNrrwKVxzynyjnm4iLTyBsoH4L6d\ngJtPv4u49c9PS/XapD4/xmTFGgYTJbFPzjOBKhGZLiJXqOpDwCfANBH5GPg7kOwqpAnA1iIyG/gj\n8H7cY6OBmXGDyAqgqs97rzcDeAO41utSSpWtOfCyiMzATXs9OEndO4BW3oD2dKBU3ZKLQ3GNwXRg\niqq+7NW/HngFeA9YkeQ1E8vVzk+S1zcmLZt22xhjTDX2jcEYY0w11jAYY4ypxhoGY4wx1VjDYIwx\nphprGIwxxlRjDYMxxphqrGEwxhhTjTUMxhhjqvl/O5RW4uIE7LAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf9283c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEPCAYAAACHuClZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8HuP9//HXO7Ek1oi1giSNSGqJYylRVLRCpCVKG0IR\nlFQtRWtJ0KjWEopIU5IQgtZWRexCJRRFSI4ECfEjWiqxfJEFkZx8fn/MHO7cOfd97mXmnplzPs/H\n436ce+a+Zu53rta5zlzXzHXJzHDOOeeq0SbpAM4557LPGxPnnHNV88bEOedc1bwxcc45VzVvTJxz\nzlXNGxPnnHNVi70xkdRP0mxJb0g6u0CZUZLmSKqXVBfu20zSE5JelTRT0qk55YdLelfStPDVL+5/\nh3POucJWifPkktoAo4EfAv8DpkqaaGazc8rsD3Qzs+6SdgXGAL2BZcAZZlYvaS3gJUmTco690syu\njDO/c8650sR9ZbILMMfM3jGzpcDtwIC8MgOAmwHM7HlgXUkbm9k8M6sP9y8CZgGdco5TzNmdc86V\nKO7GpBPw35ztd1mxQWiqzHv5ZSR1AeqA53N2nxx2i10vad2oAjvnnCtf6gfgwy6uu4Bfh1coANcA\n3zazOmAe4N1dzjmXoFjHTAiuMrbI2d4s3JdfZvOmykhahaAhucXMJjYWMLMPc8pfB9zf1JdL8onH\nnHOuAmZW1lBC3FcmU4EtJXWWtBpwGHBfXpn7gKMAJPUGPjWz+eFnNwCvmdnVuQdI2iRn82DglUIB\nzCz1r+HDhyeewXN6Rs/pORtflYj1ysTMGiSdDEwiaLjGm9ksSUOCj22cmT0kqb+kN4HFwGAASbsD\nRwAzJU0HDBhmZo8Al4W3EC8H5gJD4vx3xG3u3LlJRyiJ54xOFjKC54xaVnJWIu5uLsJf/j3y9o3N\n2z65ieOeAdoWOOdRUWZ0zjlXnWYbE0kbAbsDmwJfEHQpvWhmy2PO1moMHjw46Qgl8ZzRyUJG8JxR\ny0rOSqhQ/5ikvYFzgI7AdOADoB2wFdCNYGD8CjNbUJuo5ZNklfb/OedcayUJi3AAvj9wvJl918xO\nMLPzzOy3ZnYgsD1BA9O3irwuNGXKlKQjlMRzRicLGcFzRi0rOStRsJvLzM4s8tky4N5YEjnnnMuc\nYt1cZxQ70DIwL5Z3cznnXPkq6eYqNgC/dvizB/Bdvnk+5ADghfLjOeeca6kKjpmY2e/N7PcET6Tv\naGa/MbPfADux4lPtrkpZ6Uf1nNHJQkbwnFHLSs5KlPIE/MbAVznbX4X7nHPOOaDImMnXBaRzgYHA\nPeGug4A7zezimLNVzcdMnHMuNHcunH8+XHghdO1atGjUYyYAmNlFkh4G9gx3HWNm08v5Eueccwn5\n7DO45BIYORKWLAEz+OtfI/+aUid6XANYYMGEi+9KKt6subJkpR/Vc0YnCxnBc0at5jn/9jfYcksY\nMSJoSI44Ai6Op1OplOlUhgM7E9zVdSOwKvBXgilWnHPOpdWyZfDRR7DHHnDFFbDLLrF9VSljJvXA\nDsA0M9sh3DfDzHrFlioiPmbinGvVli+HSZNgv/1ApQ+BRD2dSqOvwt/IFn7JmuV8gXPOuZgtXx6M\nheRr0wb69SurIalUKY3JnZLGAh0kHQ88TrC6oYuI9/dGKws5s5ARPGfUYsk5bx7svz9ce2305y5D\nKXdz/UlSX2ABwbjJ78zssdiTOeecK+7hh2HwYPjgA3jtNTj2WGjXLpEoRcdMJLUFHjezvWsXKTo+\nZuKca5GWLIFhw+DKcIrEvfeGW26BTp0iOX3kYyZm1gAsl7RuVcmcc85F55e/DBqStm2DW30feyyy\nhqRSpYyZLCJYh328pFGNr7iDtSatur83BlnImYWM4DmjFlnOoUNhu+3g6aeD922bXOG8pkpZA/7u\n8OWccy4NttoK6uuDu7VSotnnTLLMx0ycc5m2dCksXAgdO9b0ayMdM5F0v6QDJK3axGfflnShpGMr\nCeqcc64Zzz4LO+4Ixx2XdJKSFLtGOp5gcsfZkqZKekjSE5LeBsYCL5nZDTVJ2cK1uv7emGUhZxYy\ngueMWkk5P/kEhgyB3XeHV16BGTPg449jz1atYmvAzwPOAs6S1AX4FvAF8IaZfV6TdM4515rccQec\nemrw3Miqq8JZZ8G550L79kkna5aPmTjnXFqcdx5cdBF8//swZgx85zuJxKhkzMQbE+ecS4svvoB7\n7oFBg2oyn1YhcU306GLWovp7UyALObOQETxn1JrN2b49HH54og1Jpbwxcc65Wlu8GGbPTjpFpCrq\n5pJ0gZldEH2caHk3l3MudV5/HQ45BBYsgGnTYIMNkk60klp2c71U4XHOOdd63Xkn7LwzvPoqrLFG\ncBtwC1FRY2Jm90cdpDVrMf29KZGFnFnICJ4zMosXwymnMOXQQ2HRIjj0UJg6Fbp3TzpZZJptTMKn\n3e+X9JGkDyRNlPTtWoRzzrkWob4eRo8OJmT885/htttg7bWTThWpUtaAfw74C3BbuOsw4BQz2zXm\nbFXzMRPnXGpcein07Qs77ZR0kmbF8pyJpBlm1itv38tmtn0FGWvKGxPnnCtfXAPwD0s6R1IXSZ0l\nnQU8JKmjpNpOZdlCpb6/N+Q5o5OFjOA5y/bSS3D11QU/Tk3OGJSynsnA8OeQvP2HAQb4+IlzrnVb\ntCiYQ2v06GB7jz0y0Z0VpdinU5HUDxhJcBU03sxGNFFmFLA/sBgYbGb1kjYDbgY2BpYD15nZqLD8\nesAdQGdgLjDQzD5r4rzezeWci9e0aXDYYTBnTjDAftppcMEFsNZaSSerWCzdXJJ+Jmnt8P15ku6W\ntEOJgdoAo4H9gG2AQZJ65pXZH+hmZt0Jrn7GhB8tA84ws22A3YCTco49B3jczHoATwBDS8njnHOR\nuvde6N07aEi22w5efBH+9KdMNySVKmXM5HwzWyhpD2AfYDzf/MJvzi7AHDN7x8yWArcDA/LKDCC4\nAsHMngfWlbSxmc0zs/pw/yJgFtAp55ibwvc3AQeVmCeVstKP6jmjk4WM4DmbtdtuwSqIJ50Ezz8P\ndXVFi2elPitRyphJQ/jzR8A4M3tQ0h9LPH8n4L852+8SNDDFyrwX7pvfuCNcT6UOeC7ctZGZzYdg\n3RVJG5WYxznnorPxxsHT7Ouvn3SSxJXSmLwnaSzQFxghaXVqOEGkpLWAu4Bfm9niAsUKDowMHjyY\nLl26ANChQwfq6uro06cP8M1fCb5d2nbjvrTkyfJ2nz59UpWn2HajtOTx+ox+e8qUKUyYMAHg69+X\n5SrlOZM1gH7ATDObI+lbwHZmNqnZk0u9gQvMrF+4fQ5guYPwksYAk83sjnB7NrCXmc2XtArwAPCw\nmV2dc8wsoE9YZpPw+JVWkfEBeOdcJGbNgr/8Jbjtt23bpNPELpYBeDP73MzuNrM54fb7pTQkoanA\nluHzKasR3E58X16Z+4Cj4OvG59PGLizgBuC13IYk55jB4fujgYkl5kml/L9Y0spzRicLGcFz8u9/\nBzP8brPNN41JFbJSn5UopZurYmbWIOlkYBLf3Bo8S9KQ4GMbZ2YPSeov6U3CW4MBJO0OHAHMlDSd\noCtrmJk9AowA7pR0LPAO3zwL45xz1fvXv2DoUHjmmWB7tdVg8GA49thEY6WZL9vrnHP5Jk6Egw6C\nDh3gxBPhlFPgW99KOlXN+Brwebwxcc5VZPlyuOGG4GHEVvjMSFwPLfaWNFXSIklfSWqQtKDymC5f\nVvpRPWd0spARWnjOhga4+eZgKpR8bdrAL34ReUOSlfqsRCm3+I4GBgFzgPbALwimpHfOuWz65z+D\nubOOPjp4Yt1VrZRbg180s51zp6KXNN3MSppSJUnezeWcW8GsWXDmmfDgg8H25psHjclAv4cnVyXd\nXKXczfV5eFtvvaTLgPep4UOLzjkXicb5sxoaglUOhw4NJmVs3z7pZC1CKY3CkWG5kwlu3d0cODjO\nUK1NVvpRPWd0spARWljO7t3hgANgyJCgYRk6tOYNSVbqsxKlXJkcFD40+CXwewBJvwaqe3rHOedq\n7a67WsUT7EkoZcxkmpntmLfPx0ycc+m1bBmsEusz2S1apLcGSxok6X6gq6T7cl5TgP+rMqtzzsVj\nyhTYait45ZWkk7QqxcZMngWuAGaHPxtfZxAsduUikpV+VM8ZnSxkhIzlNIOrroJ99oG33656Hq04\nZKU+K1HwOtDM3iGY92o3SZ2B7mb2uKT2BM+bLKxRRuecK+6LL2DQILjjjmD7rLPgoouSzdTKlDJm\ncjxwAtDRzLpJ6g6MMbMf1iJgNXzMxLlWYPnyYMXDF14InlifMCGY6ddVLK7nTE4iWB3xeYBwTRNf\n2dA5lw5t2gTL5n72GdxzD3xnpaWNXA2U8pzJEjP7qnEjXLDK/9yPUFb6UT1ndLKQETKUc4stoL4+\n9Q1JVuqzEqU0Jk9KGga0l9QX+Dtwf7yxnHOuTO3aJZ2gVStlzKQNcBywLyDgUeD6LAxG+JiJcy3M\n55/DtGmwxx5JJ2nRfD2TPN6YONeCvP8+HHhg8PzIk0/CLrsknajFims9k90lPSbpDUlvSXpb0luV\nx3T5stKP6jmjk4WMkKKcL78cNB4vvgibbhpM1JgjNTmbkZWclSjlbq7xwOnAS0BDvHGccy7PAw8E\nKx4uXgy77x7csbXhhkmncnlKGTN53sx2rVGeSHk3l3MZ9/HH0LUrLFwIRxwB11/vA+01EOmYiaTG\nyR0HAm2Bu4EljZ+b2bQKc9aMNybOtQD33ht0c/3ud6Cyfr+5CkU9ZtI4F9euwM7AxTn7fJ3LCGWl\nH9VzRicLGSElOQ86CIYPL9qQpCJnCbKSsxLFxkxGm9k/apbEOedcZhXr5lppHZOs8W4u5zLkueeC\n2X4HDUo6SasX19xczjkXrzvugKOPDiZt3Gor2GmnpBO5MhUbM+kpaUYTr5mSZtQsYSuQlX5Uzxmd\nLGSEGuQ0C6aKP+wwWLIEjjkGevUq+zRen8krdmXyNnBArYI451qZ5cvhxBNh3LhgcP1Pf4LTT/c7\ntjKq2JhJJtZ5L8bHTJxLsTfeCLqzli2D22+HAQOSTuRCUY+ZPFNlHuecK2yrrWDSJFi0CPr2TTqN\nq1LBMRMzO7mWQVqzrPSjes7oZCEj1CDnbrtF0pB4fSavlPVMnHPOuaJ8CnrnXPzMgilR6uqSTuJK\nEOmYiaSDix1oZneX80XOuVbKDM4/Hy65BG66CX7+86QTuRgU6+a6CzgP+HH4OiDn9eP4o7UeWelH\n9ZzRyUJGiCjnhRcGz5JIsc3426rqM6WK3c11MHAY0AuYCNxmZm/WJJVzrmW46CK44AJo0wb+9jf4\n6U+TTuRiUsp6JmsCA4BDgfWBc83syZK/QOoHjCS4ChpvZiOaKDMK2B9YDBxjZtPD/eMJroLmm1mv\nnPLDgeOBD8Jdw8zskSbO62MmziVlzJjgoUQJbrklWI/EZUIsy/YCXwKfAQuAtYCSr1MltQFGA/sB\n2wCDJPXMK7M/0M3MugNDgGtzPr4xPLYpV5rZjuFrpYbEOZew7beHjh3huuu8IWkFCjYmkn4gaRzB\ncr17A1ebWZ2ZPVrG+XcB5pjZO2a2FLid4Con1wDgZgAzex5YV9LG4fbTwCeFIpaRI9Wy0o/qOaOT\nhYxQZc7ddoM5c+C44yLLU0irqM+UK3Zl8jhBY/A0sDpwlKRRja8Sz98J+G/O9rvhvmJl3muiTFNO\nllQv6XpJ65aYxzlXSx07Jp3A1UixubmOLnagmd3U7MmlQ4D9zOyEcPvnwC5mdmpOmfuBS8zs2XD7\nceCsxmWBJXUG7s8bM9kQ+MjMTNIfgW+Z2Up//viYiXPOlS/qubl6mNmwKjO9B2yRs71ZuC+/zObN\nlFmBmX2Ys3kdcH+hsoMHD6ZLly4AdOjQgbq6Ovr06QN8c8np277t21Vuv/8+U8aNg732Skce3y5r\ne8qUKUyYMAHg69+XZTOzJl/AtEKflfoC2gJvAp2B1YB64Dt5ZfoDD4bvewPP5X3eBZiZt2+TnPen\nA7cW+H7LgsmTJycdoSSeMzpZyGhWYs5PPjHbbjszMLv11tgzNaVF1WcKhL87y/p9X+zKpK2k9Sgw\n0G1m/1dCQ9Ug6WRgEt/cGjxL0pAw7Dgze0hSf0lvEt4a3Hi8pFuBPsD6kv4DDDezG4HLJNUBy4G5\nBHeBOedq7Ysv4MADYeZM6NHDZ/9txYqNmSwh6G5qqjExM/t2nMGi4GMmzsVo2bLgIcSJE6FTJ3jm\nGejcOelULgJRj5m8ZhlfHMs5F6Mzzggakg4d4JFHvCFp5XwK+hRoHAhLO88ZnSxkhGZy/uIX0LUr\nPPggbLttzTI1pUXUZ8YVuzK5utAHkrYws//EkMc5lxW9esHrr8OqqyadxKVA0bm5JO1G8ADhU2b2\ngaRewDnAnma2ecEDU8LHTJxzrnyRzs0l6TLgBuAQ4MHw4cBJwPNA92qCOueca1mKjZn8GNjBzAYB\n+wKnAb3N7Goz+7Im6VqJrPSjes7oZCEj5OS8/HIYPTrRLMVkrj5boGJjJl82Nhpm9omkOWY2tzax\nnHOp8ec/w1lnBVPJ/+AHsPXWSSdyKVTsOZNPgadydn0/d9vMDow3WvV8zMS5Kl13HZxwQvB+7Nhv\n3rsWrZIxk2KNyV7FDrQyFshKijcmzlXhllvg6KODNdyvvhpOPbX5Y1yLEOkAvJk92dQLeItganoX\nkaz0o3rO6KQ+4+LFMHQoU8zg0ktT35Ckvj5DWclZiZIeWpS0oaRfSfoXMAXYONZUzrlkrbkmTJoE\nJ50EZ5+ddBqXAcW6udYGDgYOB7YC7gYONbPNahevOt7N5Zxz5Yt6zOQL4AXgPOBpMzNJb2VhgsdG\n3pg451z5Ih0zAYYSLNd7DTBUUrdqwrnCstKP6jmjk6qMy5fDww83+VGqchbhOZNXbAB+pJn1BgaE\nu+4FNpV0tqStapLOORevhgY47jjo3x9Gjkw6jcuwonNzrVRY2hYYRDB2smVsqSLi3VzOFfHVV3Dk\nkXDnnbDGGsF08vvsk3QqlwKRjpm0BN6YOFfAsmVw8MFw//2wzjrBNPJ77JF0KpcSUY+ZuBrJSj+q\n54xO4hkvvDBoSDp2hH/+s2BDknjOEnnO5Hlj4lxrNHAgbLcd3H037Lxz0mlcC+DdXM61Vg0N0LZt\n0ilcCtWkm0vSTZKuDQfjnXNZ5Q2Ji1Al3VyjgceBIyPO0mplpR/Vc0YnCxnBc0YtKzkrUWw9kyaZ\n2VRgKvCP6OM452LxySew3npJp3AtWHNrwG8GHAbsCWwKfAG8AjwIPGxmy2sRslI+ZuIcsHAh7Lgj\nfO97cM01wSSOzhVRyZhJwSsTSTcCnYAHgBHAB0A7gkkf+wHnSjrHzJ4qdA7nXAr86lfw5ptBI+Lj\nJC4mxcZMrjCzfc1slJk9a2ZvmtkrZna3mZ0C9AH+V5uYLVtW+lE9Z3RqlvHmm+Gvfw2ecL/9dmjX\nrqzDs1CX4DnToNjcXK80vpfUXlKPvM+/MrM34wznnKvCG28EVyUAo0ZBz57J5nEtWrPPmUg6ELgc\nWM3MukqqAy70NeCdS7nDDoM77oBDD4XbbgOV1QXuWrFIx0xyDCdYpncKgJnVS+pafjznXE2NHw+d\nO8OwYd6QuNiV8pzJUjP7LG+f/7kfoaz0o3rO6NQk45prwogRsO66FZ8iC3UJnjMNSrkyeVXS4UBb\nSd2BU4Fn443lnHMuS0oZM1kDOBfYFxDwKPAHM/sy/njV8TET55wrn69nkscbE9dqmAV3bB13HKy1\nVtJpXMbFMtGjpMmSnsh/VR7T5ctKP6rnjE7kGS+/HE47Dfr1CxqWiGShLsFzpkEpYya/zXnfDjgE\nWBZPHOdc2SZPhqFDg/dnneV3brlEVNTNJekFM9slhjyR8m4u1+K9+24w79aHHwa3AF90UdKJXAsQ\nVzdXx5zXBpL2A0q+11BSP0mzJb0h6ewCZUZJmiOpXtIOOfvHS5ovaUZe+fUkTZL0uqRHJVV+76Nz\nWfXVV/CznwUNyT77BEvxOpeQUp4zeQl4Mfz5b+A3wHGlnFxSG4L1T/YDtgEGSeqZV2Z/oJuZdQeG\nANfmfHxjeGy+c4DHzawH8AQwtJQ8aZWVflTPGZ1IMjY0QI8esPnmcOutsUzimIW6BM+ZBs2OmZhZ\nNU+77wLMMbN3ACTdDgwAZueUGQDcHH7X85LWlbSxmc03s6cldW7ivAOAvcL3NxE8nX9OFTmdy572\n7eHGG4Mrkw03TDqNa+UKjplIOrjYgWZ2d7Mnlw4B9jOzE8LtnwO7mNmpOWXuBy4xs2fD7ceBs8xs\nWrjdGbjfzHrlHPN/Ztax0HbOfh8zcc65MkU9N9cBRT4zoNnGpIa8xXDOuQQVbEzM7JgIzv8esEXO\n9mbhvvwymzdTJt/8xq4wSZsQLNzVpMGDB9OlSxcAOnToQF1dHX369AG+6b9MertxX1ryFNoeOXJk\nKusvi/WZn7Wk4ydPhi+/pM/++9csb319PaeddlrNvq/S7YrqM4HttNbnlClTmDBhAsDXvy/LZmZF\nX8D6wChgGsEg/NXA+s0dFx7bFngT6AysBtQD38kr0x94MHzfG3gu7/MuwMy8fSOAs8P3ZwOXFvh+\ny4LJkycnHaEknjM6FWW89FKznj3NXnst8jyFZKEuzTxn1MLfnc3+js99lTI312PAU8Bfw11HAH3M\nbJ9SGitJ/cIGqA0w3swulTQkDDsuLDOaYCngxcAx9s14ya0EKzquD8wHhpvZjZI6AncSXNG8Aww0\ns0+b+G5r7t/nXCY88QT07QvLl8N998EBxXqhnatOLHNzSXrFzLbN2zfTzLarIGNNeWPiWoTcBxPP\nPRf++MekE7kWLpaHFoFJkg6T1CZ8DSSYOdhFJLe/N808Z3RKzpj7YGLfvvD738eaK18W6hI8ZxoU\nHICXtJDgLikBpwG3hB+1BRax4pxdzrk43H03PPdcrA8mOhcFn4LeubS74QbYdlvYJfXT4bkWItIx\nE0ldzGxukS8T0MnM3i0rZQ15Y+Kcc+WLeszkckn/kHSUpG0kbSRpC0k/kPQH4BngO1UldkB2+lE9\nZ3SykBE8Z9SykrMSxR5a/JmkrQluBT4W+BbwOTALeAi4yDKwdK9zzrn4+ZiJc2lhBr/5DRx1FNTV\nJZ3GtWJx3RrsnKuF886Dq66C/v3hS7/od9nijUkKZKUf1XNGZ6WM48bBxRcHt/6OHw/t2iWSK18W\n6hI8Zxp4Y+Jc0h56CE48MXg/ZgyEEzk6lyU+ZuJckj74ALp1g0WLgm6uP/wh6UTOxTM3V4EvmmZm\nO5Z9YI15Y+Iy4dZbYfLkoKtLZf3361wsajYAn4WGJEuy0o/qOaOzQsbDD4frrktlQ5KFugTPmQbN\nrgEPEC5AtQvBXF1TzWxerKmcc85lSilT0P8C+B3wBMGkj3sBF5rZDfHHq453cznnXPniWs/kdeB7\nZvZxuL0+8KyZ9ag4aY14Y+JS55ZboEcPn7TRpVpcYyYfAwtztheG+1xEstKP6jmr9NxzcNxxsOee\nTLn11qTTlCS1dZnHcyavlDGTN4HnJU0kGDMZAMyQdAaAmV0ZYz7nWoYPPwwWuVq6FE49FTbdNOlE\nzkWqlG6u4cU+N7PaLv1WBu/mcqnQ0AD77Qf//Cd873vBbcCrrZZ0KucKiv05E0ltgLXMbEG54ZLg\njYlLhWHD4JJLYKONYNo06NQp6UTOFRXLmImkWyWtI2lN4BXgNUlnVhrSrSwr/aies0Lf/S6stx7c\nccfXDUnqMhbgOaOVlZyVKGUAfuvwSuQg4GGgK3BkrKmca0l+8hOYOxf69Ek6iXOxKWXM5FWgDrgV\nGG1mT0p62cy2r0XAang3l3POlS+uW4PHAnOBNYGnJHUGMjFm4pxzrjaabUzMbJSZdTKz/uGf+f8B\n9o4/WuuRlX5Uz1mijz5qtkjiGUvkOaOVlZyVKHuiRwssiyOMc5n3+OPQuTOMHp10EudqytczcS4q\nM2bAHnvAwoXw29/C5Zcnnci5itRsPZOs8MbE1cy770Lv3vDeezBwINx2G7TxhUxdNsW2nomkbSUN\nlHRU46uyiK4pWelH9ZwFfPYZ9O8fNCR77gk33dRsQ+J1GS3Pmbxm5+YKp1PpA2wNPATsDzwN3Bxr\nMuey4q23giuTHj3g3nuhXbukEzlXc6U8ZzIT2B6YbmbbS9oY+KuZ9a1FwGp4N5ermdmzYfXVoWvX\npJM4V7VKurlKmTX4CzNbLmmZpHWAD4DNK0roXEvVs2fSCZxLVCljJi9K6gBcB7wETAP+HWuqViYr\n/aieMzpZyAieM2pZyVmJZq9MzOxX4dsxkh4B1jGzGfHGci7FPvoINtgg6RTOpUopYyb/NLMfNrcv\njXzMxEXutddgt93gd7+DM84AldWt7FwmRDpmIqkdsAawgaT1gMYTrwP4ggyu9Vm2DI46ChYsgBdf\nTDqNc6lSbMxkCMEYSU+CcZKXwtdEoOS5IiT1kzRb0huSzi5QZpSkOZLqJdU1d6yk4ZLelTQtfPUr\nNU8aZaUftdXn/NOf4KWXYIstYNy4qq5KWn1dRsxzJq/glYmZXQ1cLekUM/tzJScPV2YcDfwQ+B8w\nVdJEM5udU2Z/oJuZdZe0KzAG6F3CsVf6+vOuZl57DYaHK1hfdx2svXayeZxLmVLGTNYETge2MLMT\nJHUHepjZA82eXOoNDDez/cPtcwjmihyRU2YMMNnM7gi3ZxE8JNm10LHhg5SLzOyKZr7fx0xcNPbb\nDyZNguOOg+uvTzqNc7GKazqVG4CvgO+F2+8Bfyzx/J2A/+Zsv8vK4y2FyjR37Mlht9j1ktYtMY9z\nlbn+ehg8GK4o+veLc61WKQ8tdjOzQyUNAjCzz6VYb2Ep5dzXABeamUn6I3AlcFxTBQcPHkyXLl0A\n6NChA3V1dfQJl09t7L9MertxX1ryFNoeOXJkKuuvJvW5+eZMOfpomD49kvPlZ43y3x/ldn19Paed\ndlpq8hTa9vqsvv4mTJgA8PXvy7KZWdEX8CzQHpgWbncDXmjuuLBsb+CRnO1zgLPzyowBDs3Zng1s\nXMqx4f50QyagAAAPaklEQVTOwIwC329ZMHny5KQjlMRzRicLGc08Z9SykjP83dns7/jcVyljJn2B\n8wgmepwE7A4MNrMpzTVUktoCrxMMor8PvAAMMrNZOWX6AyeZ2Y/CMZaRZta72LGSNjGzeeHxpwPf\nNbPDm/h+a+7f55xzbkWxzM1lZo9JmkZwpSDg12bW/LqkwbENkk4maITaAOPDxmBI8LGNM7OHJPWX\n9CawGDim2LHhqS8LbyFeTrA+/ZAy/s3ONe/TT2GddXxNEudKVOp/Ke2AT4AFwNaSvl/qF5jZI2bW\nw8y6m9ml4b6xZjYup8zJZralmW1vZtOKHRvuP8rMeplZnZkdZGbzS82TRrn9vWnWanIuXw4DBsA+\n+wRTy8eg1dRljXjO5JWynskI4FDgVYIrAQADnooxl3PJufZaeOop2GgjaN8+6TTOZUIpYyavA73M\nbEltIkXHx0xc2d5+G7bbDhYvhrvugkMOSTqRczUX13MmbwGrVhbJuQxZsgSOPDJoSH72M29InCtD\nwcZE0p8ljQI+B+oljQ3n0BoV7ncRyUo/aovPef318Mwz0KkTjC55+rmKtPi6rDHPmbxiYyaN06K+\nBNxXgyzOJevEE2H+fPjJT4LxEudcyQqOmUiaYGaDaxsnWj5m4pxz5Yt6zKRXlXmcc861EsUakzUk\n7SBpx6ZeNUvYCmSlH9VzRicLGcFzRi0rOStRbMykE3AFTU+8aMAPYknkXC0sXAgnnQSXXBIMuDvn\nqlJszGS6me1Q4zyR8jET16SGBjjoIHjgAdh7b3jiiaQTOZcqcT1n4lzLcuaZQUPSsSOMHZt0Guda\nhGKNSZPrtbvoZaUftUXkHDsWrroKVl0V7r4bunevWa5cLaIuU8RzJq9gY2Jmk2oZxLnYzZwZjJMA\njBsHe+2VbB7nWpBm5+bKMh8zcSswg4svhs8/h4suSjqNc6lVyZhJ2Y2JpHbAAWb297IOTIA3Js45\nV77YBuAltQ0XsLoFeIdgSnoXkaz0o3rO6GQhI3jOqGUlZyWKrmciaS/gcKA/wbK5uwNdzezzGmRz\nzjmXEcWeM3kX+A9wLXCvmS2U9LaZda1lwGp4N1crtnw5DBsGxx8P3bolnca5TIm6m+suYFOCLq0D\nJK1J8OS7c+l37rkwYgTsuy8sXZp0GudavGK3Bp8GdCWYUqUP8DqwoaSBktaqTbzWISv9qJnJecYZ\ncOml0LYt/OUvwTMlKZOZuvSckcpKzkoUHYC3wGQzO4GgYRkEDADm1iCbc+V74AEYOTJ4P3Ys9OuX\nbB7nWomSbw2WtCqwLfAesNDMvogzWBR8zKSV+X//D3r1Cp4jOf98uPDCpBM5l0mRPmciaQzwZzN7\nVdK6wL+BBqAj8Fszu63awHHzxqSVMYNrr4Xp04Mn3FXWfwvOuVDUA/B7mtmr4ftjgDfMbDtgJ+Cs\nCjO6JmSlHzX1OSX41a+YcsQRqW9IUl+XIc8ZrazkrESxxuSrnPd9gXsBzGxerImcc85lTrFurskE\nd3K9B0wGeprZPEmrAK+YWc/axayMd3M551z5KunmKvYE/BBgFLAJcFrOFckPgQcri+hchD75BD76\nKLFp5J1z3yj2nMkbZtbPzOrMbELO/keB/9YiXGuRlX7UVOU0g2OOgZ12gkkrrpaQqpwFZCEjeM6o\nZSVnJSpdafGMSFM4V66rroKJE4MHE7faKuk0zrV6Fa1nIum/ZrZ5DHki5WMmLdRzz8Gee8KyZXDv\nvTBgQNKJnGtRarkGvP+Gdsn4+GMYODBoSM44wxsS51KiYGMiaaGkBeHPxvcLJC0kmADSRSQr/aip\nyPncczB/PvTuHcy/1YRU5GxGFjKC54xaVnJWouDdXGa2di2DOFeSH/0Inn0WNtgglRM4OtdaFXvO\npB3wS2BLYAZwg5ktq2G2qvmYiXPOlS/qMZObgJ2BmQQrLV5RRTbnyjNrFvzyl8GzJM651CvWmGxt\nZj83s7HAT4E9K/kCSf0kzZb0hqSzC5QZJWmOpHpJdc0dK2k9SZMkvS7p0XAiyszKSj9q7DnN4Mkn\n4YADYOutgynkx4wp+zRZqM8sZATPGbWs5KxEscbk6+XpKu3ektQGGA3sB2wDDJLUM6/M/kA3M+tO\n8NT9mBKOPQd43Mx6AE8AQyvJlxb19fVJRyhJrDmffhp23BH69AnWJGnXLrgy+elPyz5VFuozCxnB\nc0YtKzkrUWw6le0lLQjfC2gfbotg3ax1Sjj/LsAcM3sHQNLtBItrzc4pMwC4meCkz0taV9LGBItx\nFTp2ALBXePxNwBSCBiaTPv3006QjlKSqnEuXwv/+B/Pmwa67rvz5aqtBfT2svz6cfDKcdBJsuGHt\nc9ZIFjKC54xaVnJWotjdXG0jOH8nVpx65V2CBqa5Mp2aOXZjM5sf5pwnaaOCCV56acXtNm1ghx1W\nLtfQEPwyy1eL8k2JO89XXwVXAEuXBu8bGoL9q6wCP//5yuWXLAnq8pprgvJLlwavtm3h7CZ6Lz/8\nEPbeGz79NHgtXhzs79wZ5s5dufz228Ojj8Iee8Aaa6z8uXMu1YpdmSSlkoUoCt+ytfPOK2wuZC3W\ntoUrl/v885XK1qr83KMPqX2eL7+EQ1b+3oWsxdoFGpO5DzwQNEA5FrA26zTVmLRvD6+++vVmA234\niA2of6cH+y1ZAquvvmL51VdH++278nkIhlKaUmjJkqOPnltW+XLPH0X5uXPnpipP4fJzueCCNOXx\n+qy2fGw3uJpZbC+gN/BIzvY5wNl5ZcYAh+ZszwY2LnYsMIvg6gSCWY1nFfh+85e//OUvf5X/Kvf3\nfdxXJlOBLSV1Bt4HDgMG5ZW5DzgJuENSb+BTM5sv6aMix94HDAZGAEcDE5v68nLvk3bOOVeZWBsT\nM2uQdDIwieDOsfFmNkvSkOBjG2dmD0nqL+lNYDHBEsEFjw1PPQK4U9KxwDvAwDj/Hc4554qraNZg\n55xzLlelswanWikPSqaBpLmSXpY0XdILSedpJGm8pPmSZuTsS92DogVyDpf0rqRp4atfkhnDTJtJ\nekLSq5JmSjo13J+qOm0i5ynh/tTUqaTVJT0f/jczU9LwcH/a6rJQztTUZS5JbcI894XbZddni7sy\nCR92fINgeeH/EYzbHGZms4semABJbwE7mVmq5gyRtAewCLjZzHqF+0YAH5vZZWEDvZ6ZJfpsT4Gc\nw4GFZnZlktlySdoE2MTM6iWtBbxE8KzUMaSoTovkPJQU1amkNczsc0ltgWeAU4FDSFFdFsm5Pymq\ny0aSTgd2AtYxswMr+e+9JV6ZfP2gpJktBRofdkwjkcL/DczsaSC/gRtA8IAo4c+DahqqCQVyQmW3\nl8fGzOaZWX34fhHB3YibkbI6LZCzU/hxaurUzD4P365OMO5rpKwuoWBOSFFdQnBFSjD/4vU5u8uu\nz9T9IotAoYcg08iAxyRNlXR80mGasVHug6JA4QdFk3dyOM/b9Ul3d+ST1AWoA54j7+FbUlSnOTmf\nD3elpk7DLpnpwDzgMTObSgrrskBOSFFdhq4CzuSbxg4qqM+W2Jhkye5mtiPBXwUnhd02WZHW/tFr\ngG+bWR3Bf8Sp6U4Iu47uAn4d/uWfX4epqNMmcqaqTs1suZntQHB1t4ukbUhhXTaRc2tSVpeSfgTM\nD69Ii10xNVufLbExeQ/YImd7s3Bf6pjZ++HPD4F7WHmqmTSZr2DOtMa+9Q8SztMkM/vQvhkIvA74\nbpJ5GklaheAX9C1m1vhcVOrqtKmcaa1TM1tAMC9fP1JYl41yc6awLncHDgzHb28DfiDpFmBeufXZ\nEhuTrx+UlLQawcOO9yWcaSWS1gj/AkTSmsC+wCvJplqBWPEvlcYHRaHIg6IJWCFn+H/8RgeTnjq9\nAXjNzK7O2ZfGOl0pZ5rqVNIGjV1DktoDfQnGdlJVlwVyzk5TXQKY2TAz28LMvk3wu/IJMzsSuJ8y\n67PF3c0Fwa3BwNV887Bj04uFJ0hSV4KrESMYnPtbWnJKuhXoA6wPzAeGA/cCfwc2J3xQ1MwSnQK1\nQM69Cfr6lwNzgSGNfb9JkbQ78BTBQnON01UMA14A7iQldVok5+GkpE4lbUcwINwmfN1hZhdJ6ki6\n6rJQzptJSV3mk7QX8Jvwbq6y67NFNibOOedqqyV2cznnnKsxb0ycc85VzRsT55xzVfPGxDnnXNW8\nMXHOOVc1b0ycc85VzRsT16JJWhj+7Cwpf5XPas89NG/76SjPX0tx1I9rXbwxcS1d44NUXQkevitZ\nOHV4McNW+CKzLM2tlq/s+nEulzcmrrW4BNgjXADo1+GMrpeFCxjVN87aLGkvSU9Jmgi8Gu67J5zZ\neaakX4T7LgHah+e7Jdy3sPHLJF0eln9Z0sCcc0+W9HdJsxqPCz+7VNIrYZbL8sNLWlPSDZJmhGV+\nEu4fFO6bIenSnPK5WQ6RdGP4/kZJV0t6RtKbkg5uqn4iqXHXqsS6BrxzKXIO4VQRAGHj8amZ7RrO\n4faMpElh2R2AbczsP+H2MWb2qaR2wFRJ/zCzoZJOCmd9bmThuQ8BepnZdpI2Co95MixTB2xNMGPs\nM5K+B8wGDjKznuHx6zSR//wwb+MiYOtK+hZwaZj3U4LlDA40s/soPovuJma2u6TvEMxpdXd+/ThX\nLr8yca3VvsBR4XoTzwMdge7hZy/kNCQAp0mqJ1iDZLOccoXsTjADK2b2AcGMsY2zw75gZu+HM8fW\nA12Az4AvwvUtfgJ80cQ59wH+0rhhZp+F55xsZv9nZsuBvwHfD4sUm0783vAcs0jBuh+uZfDGxLVW\nAk4xsx3CVzczezz8bPHXhYLJ734A7BquQVEPtMs5R6nf1WhJzvsGYBUzayBYfuAu4MfAI02co9Ak\neoUy5JZvl/dZboZUrfrnsssbE9fSNf6yXAisnbP/UeBX4fodSOouaY0mjl8X+MTMlkjqCfTO+eyr\nxuPzvutfwKHhuMyGwJ4EMwQ3HTD43g5m9ghwBtCriWKPASflHNMhPOf3JXUMbxYYRHAVBMF6FD0k\ntQF+Uui7KVw/zpXFGxPX0jX+hT4DWC5puqRfm9l1wGvANEkzgTFAU3dvPQKsKulV4GLg3zmfjQNm\n5AykG4CZ3RN+38vA48CZYXdXoWzrAA9IeplgCvjTmyh7EdAxHNSfDvQJl1M9h6ABmQ5MNbMHwvJD\ngQeBp4H/NfGd+dsr1E8T3+9cUT4FvXPOuar5lYlzzrmqeWPinHOuat6YOOecq5o3Js4556rmjYlz\nzrmqeWPinHOuat6YOOecq5o3Js4556r2/wGzal9fiBR69gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa1237f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['SparsityPhiScore'].value, 'b--',\n",
    "                 xrange(model_artm.num_phi_updates), model_artm.score_tracker['SparsityPhiScore'].value, 'r--', linewidth=2)\n",
    "plt.xlabel('Iterations count')\n",
    "plt.ylabel('PLSA Phi sp. (blue), ARTM Phi sp. (red)')\n",
    "plt.grid(True)\n",
    "plt.show()\n",
    "\n",
    "plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['SparsityThetaScore'].value, 'b--',\n",
    "                 xrange(model_artm.num_phi_updates), model_artm.score_tracker['SparsityThetaScore'].value, 'r--', linewidth=2)\n",
    "plt.xlabel('Iterations count')\n",
    "plt.ylabel('PLSA Theta sp. (blue), ARTM Theta sp. (red)')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "It seems that achieved result is enough. The regularization helped us to improve all scores with quite small perplexity downgrade. Let's look at top-tokens:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "topic_0:  [u'year', u'tax', u'jobs', u'america', u'president', u'issues']\n",
      "topic_1:  [u'people', u'war', u'service', u'military', u'rights', u'vietnam']\n",
      "topic_2:  [u'november', u'electoral', u'account', u'polls', u'governor', u'contact']\n",
      "topic_3:  [u'republican', u'gop', u'senate', u'senator', u'south', u'conservative']\n",
      "topic_4:  [u'people', u'time', u'country', u'speech', u'talking', u'read']\n",
      "topic_5:  [u'dean', u'democratic', u'edwards', u'primary', u'kerry', u'clark']\n",
      "topic_6:  [u'state', u'party', u'race', u'candidates', u'candidate', u'elections']\n",
      "topic_7:  [u'administration', u'president', u'years', u'bill', u'white', u'cheney']\n",
      "topic_8:  [u'campaign', u'national', u'media', u'local', u'late', u'union']\n",
      "topic_9:  [u'house', u'million', u'money', u'republican', u'committee', u'delay']\n",
      "topic_10:  [u'republicans', u'vote', u'senate', u'election', u'democrats', u'house']\n",
      "topic_11:  [u'iraq', u'war', u'american', u'iraqi', u'military', u'intelligence']\n",
      "topic_12:  [u'kerry', u'poll', u'percent', u'voters', u'polls', u'numbers']\n",
      "topic_13:  [u'news', u'time', u'asked', u'political', u'washington', u'long']\n",
      "topic_14:  [u'bush', u'general', u'bushs', u'kerry', u'oct', u'states']\n"
     ]
    }
   ],
   "source": [
    "for topic_name in model_plsa.topic_names:\n",
    "    print topic_name + ': ',\n",
    "    print model_plsa.score_tracker['TopTokensScore'].last_tokens[topic_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "topic_0:  [u'party', u'political', u'issue', u'tax', u'america', u'issues']\n",
      "topic_1:  [u'people', u'military', u'official', u'officials', u'service', u'public']\n",
      "topic_2:  [u'electoral', u'governor', u'account', u'contact', u'ticket', u'experience']\n",
      "topic_3:  [u'gop', u'convention', u'senator', u'debate', u'south', u'sen']\n",
      "topic_4:  [u'country', u'speech', u'bad', u'read', u'end', u'talking']\n",
      "topic_5:  [u'democratic', u'dean', u'john', u'edwards', u'primary', u'clark']\n",
      "topic_6:  [u'percent', u'race', u'candidates', u'candidate', u'win', u'nader']\n",
      "topic_7:  [u'administration', u'years', u'white', u'year', u'bill', u'jobs']\n",
      "topic_8:  [u'campaign', u'national', u'media', u'press', u'local', u'ads']\n",
      "topic_9:  [u'house', u'republican', u'million', u'money', u'elections', u'district']\n",
      "topic_10:  [u'november', u'poll', u'senate', u'republicans', u'vote', u'election']\n",
      "topic_11:  [u'iraq', u'war', u'american', u'iraqi', u'security', u'united']\n",
      "topic_12:  [u'bush', u'kerry', u'general', u'president', u'voters', u'bushs']\n",
      "topic_13:  [u'time', u'news', u'long', u'asked', u'washington', u'political']\n",
      "topic_14:  [u'state', u'states', u'people', u'oct', u'fact', u'ohio']\n"
     ]
    }
   ],
   "source": [
    "for topic_name in model_artm.topic_names:\n",
    "    print topic_name + ': ',\n",
    "    print model_artm.score_tracker['TopTokensScore'].last_tokens[topic_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "We can see, that topics are approximatelly equal in terms of interpretability, but they are more different in ARTM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's extract the $\\Phi$ matrix as pandas.DataFrame and print it (to do this operation with more options use ARTm.get_phi()):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   topic_0   topic_1   topic_2   topic_3   topic_4   topic_5  \\\n",
      "parentheses       0.000000  0.000000  0.000000  0.000000  0.000000  0.000277   \n",
      "opinion           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "attitude          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "held              0.000000  0.000385  0.000000  0.000000  0.000000  0.000000   \n",
      "impeachment       0.000000  0.000115  0.000000  0.000000  0.000000  0.000000   \n",
      "platform          0.001717  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "msnbc             0.000000  0.000000  0.000000  0.000000  0.000000  0.000194   \n",
      "assault           0.000000  0.000000  0.000000  0.000000  0.001202  0.000000   \n",
      "tools             0.000000  0.000000  0.000000  0.000000  0.000502  0.000000   \n",
      "vance             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "contingent        0.000000  0.000544  0.000000  0.000000  0.000000  0.000000   \n",
      "drove             0.000000  0.000146  0.000000  0.000000  0.000374  0.000000   \n",
      "stripped          0.000000  0.000111  0.000000  0.000000  0.000410  0.000000   \n",
      "air               0.000000  0.000593  0.000000  0.000000  0.000124  0.000000   \n",
      "baby              0.000139  0.000003  0.000032  0.000000  0.000000  0.000000   \n",
      "highlight         0.000000  0.000000  0.000000  0.000000  0.000019  0.000000   \n",
      "honored           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kerry             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "nethercutt        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kansas            0.000007  0.000000  0.000000  0.000076  0.000000  0.000000   \n",
      "annual            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "disaster          0.000076  0.000000  0.000000  0.000000  0.001151  0.000000   \n",
      "operatives        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "fluid             0.000030  0.000000  0.000000  0.000000  0.000000  0.000344   \n",
      "aug               0.000043  0.000000  0.000000  0.001610  0.003245  0.000000   \n",
      "monitoring        0.000000  0.000000  0.000000  0.000000  0.000000  0.000490   \n",
      "initially         0.000000  0.000113  0.000000  0.000000  0.000000  0.000000   \n",
      "kysen             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "action            0.000904  0.000107  0.000000  0.000000  0.000488  0.000147   \n",
      "wide              0.000000  0.000862  0.000000  0.000000  0.000000  0.000164   \n",
      "...                    ...       ...       ...       ...       ...       ...   \n",
      "drake             0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "freeway           0.000000  0.000000  0.000000  0.000000  0.000297  0.000000   \n",
      "playbook          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "hoodies           0.000000  0.000000  0.002087  0.000000  0.000000  0.000000   \n",
      "vaantirepublican  0.000000  0.000000  0.002060  0.000000  0.000000  0.000000   \n",
      "suck              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "thirdplace        0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "wink              0.000000  0.000000  0.000176  0.000000  0.000408  0.000000   \n",
      "inching           0.000000  0.000000  0.000000  0.000000  0.000000  0.000312   \n",
      "intangibles       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "sawyer            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "composite         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "geps              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postiowa          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "brush             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "caucusgoers       0.000000  0.000000  0.000000  0.000000  0.000000  0.000667   \n",
      "countylevel       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "outreach          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "rncs              0.000297  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "uaw               0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "deanclark         0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "samarra           0.000000  0.000000  0.000000  0.000000  0.000466  0.000000   \n",
      "postdebate        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "schwarz           0.000534  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "asserted          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "aft               0.000000  0.000000  0.000000  0.000000  0.000000  0.000419   \n",
      "spindizzy         0.000000  0.000000  0.000647  0.000000  0.000000  0.000000   \n",
      "barnes            0.000000  0.000000  0.000000  0.000792  0.000000  0.000000   \n",
      "barbour           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kroll             0.000000  0.000317  0.000000  0.000000  0.000000  0.000000   \n",
      "\n",
      "                   topic_6   topic_7   topic_8   topic_9  topic_10  topic_11  \\\n",
      "parentheses       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "opinion           0.000394  0.000000  0.000193  0.000000  0.000000  0.000355   \n",
      "attitude          0.000499  0.000000  0.000000  0.000000  0.000000  0.000187   \n",
      "held              0.000000  0.000000  0.000000  0.006317  0.000000  0.000790   \n",
      "impeachment       0.000000  0.000000  0.000000  0.000000  0.000094  0.000000   \n",
      "platform          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "msnbc             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "assault           0.000000  0.000000  0.000000  0.000000  0.000000  0.000471   \n",
      "tools             0.000000  0.000000  0.000329  0.000000  0.000000  0.000000   \n",
      "vance             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "contingent        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "drove             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "stripped          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "air               0.000000  0.000025  0.000000  0.000084  0.000000  0.000048   \n",
      "baby              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "highlight         0.000000  0.000590  0.000371  0.000000  0.000000  0.000000   \n",
      "honored           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kerry             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "nethercutt        0.000244  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kansas            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "annual            0.000000  0.001851  0.000000  0.000000  0.000000  0.000000   \n",
      "disaster          0.000000  0.000000  0.000000  0.000000  0.000000  0.000804   \n",
      "operatives        0.000000  0.000008  0.000149  0.000000  0.000000  0.000431   \n",
      "fluid             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "aug               0.000183  0.000035  0.001008  0.000504  0.000000  0.000000   \n",
      "monitoring        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "initially         0.000000  0.000726  0.000314  0.000000  0.000000  0.000094   \n",
      "kysen             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "action            0.000000  0.001463  0.000212  0.002019  0.000000  0.000034   \n",
      "wide              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "...                    ...       ...       ...       ...       ...       ...   \n",
      "drake             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "freeway           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "playbook          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "hoodies           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "vaantirepublican  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "suck              0.000000  0.000000  0.000368  0.000000  0.000000  0.000000   \n",
      "thirdplace        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "wink              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "inching           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "intangibles       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "sawyer            0.000000  0.000243  0.000000  0.000000  0.000107  0.000000   \n",
      "composite         0.000391  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "geps              0.000535  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postiowa          0.000000  0.000000  0.000471  0.000000  0.000000  0.000000   \n",
      "brush             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "caucusgoers       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "countylevel       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "outreach          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "rncs              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "uaw               0.000000  0.000000  0.000334  0.000000  0.000000  0.000000   \n",
      "deanclark         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "samarra           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postdebate        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "schwarz           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "asserted          0.000000  0.000178  0.000000  0.000000  0.000000  0.000000   \n",
      "aft               0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "spindizzy         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "barnes            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "barbour           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kroll             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "\n",
      "                  topic_12      topic_13  topic_14  \n",
      "parentheses       0.000000  0.000000e+00  0.000000  \n",
      "opinion           0.002908  0.000000e+00  0.000346  \n",
      "attitude          0.000000  0.000000e+00  0.000000  \n",
      "held              0.000526  0.000000e+00  0.000000  \n",
      "impeachment       0.000000  0.000000e+00  0.000396  \n",
      "platform          0.000000  0.000000e+00  0.000000  \n",
      "msnbc             0.000000  0.000000e+00  0.002042  \n",
      "assault           0.000000  4.839028e-05  0.000000  \n",
      "tools             0.000000  3.548828e-04  0.000000  \n",
      "vance             0.000000  0.000000e+00  0.000803  \n",
      "contingent        0.000000  0.000000e+00  0.000000  \n",
      "drove             0.000000  0.000000e+00  0.000000  \n",
      "stripped          0.000000  0.000000e+00  0.000000  \n",
      "air               0.000000  4.631828e-05  0.010024  \n",
      "baby              0.000000  1.057148e-03  0.000000  \n",
      "highlight         0.000000  8.327506e-07  0.000000  \n",
      "honored           0.000000  4.265937e-04  0.000000  \n",
      "kerry             0.138169  0.000000e+00  0.000000  \n",
      "nethercutt        0.000143  0.000000e+00  0.000000  \n",
      "kansas            0.000100  0.000000e+00  0.000969  \n",
      "annual            0.000000  8.087014e-06  0.000000  \n",
      "disaster          0.000000  3.991361e-05  0.000000  \n",
      "operatives        0.000000  7.908880e-04  0.000000  \n",
      "fluid             0.000000  0.000000e+00  0.000000  \n",
      "aug               0.003553  0.000000e+00  0.000000  \n",
      "monitoring        0.000000  0.000000e+00  0.000000  \n",
      "initially         0.000000  1.245606e-04  0.000000  \n",
      "kysen             0.000000  7.598700e-04  0.000000  \n",
      "action            0.000000  0.000000e+00  0.000411  \n",
      "wide              0.000000  0.000000e+00  0.000487  \n",
      "...                    ...           ...       ...  \n",
      "drake             0.000000  0.000000e+00  0.000000  \n",
      "freeway           0.000000  0.000000e+00  0.000000  \n",
      "playbook          0.000378  0.000000e+00  0.000000  \n",
      "hoodies           0.000000  0.000000e+00  0.000000  \n",
      "vaantirepublican  0.000000  0.000000e+00  0.000000  \n",
      "suck              0.000000  0.000000e+00  0.000000  \n",
      "thirdplace        0.000000  0.000000e+00  0.000000  \n",
      "wink              0.000000  0.000000e+00  0.000000  \n",
      "inching           0.000000  0.000000e+00  0.000000  \n",
      "intangibles       0.000000  0.000000e+00  0.000310  \n",
      "sawyer            0.000000  0.000000e+00  0.000000  \n",
      "composite         0.000000  0.000000e+00  0.000000  \n",
      "geps              0.000000  0.000000e+00  0.000000  \n",
      "postiowa          0.000000  0.000000e+00  0.000000  \n",
      "brush             0.000000  2.548060e-04  0.000139  \n",
      "caucusgoers       0.000000  0.000000e+00  0.000000  \n",
      "countylevel       0.000000  4.599213e-04  0.000000  \n",
      "outreach          0.000000  0.000000e+00  0.000698  \n",
      "rncs              0.000000  0.000000e+00  0.000000  \n",
      "uaw               0.000000  0.000000e+00  0.000000  \n",
      "deanclark         0.000000  0.000000e+00  0.000000  \n",
      "samarra           0.000000  0.000000e+00  0.000000  \n",
      "postdebate        0.000086  0.000000e+00  0.000763  \n",
      "schwarz           0.000000  0.000000e+00  0.000000  \n",
      "asserted          0.000118  0.000000e+00  0.000000  \n",
      "aft               0.000000  0.000000e+00  0.000000  \n",
      "spindizzy         0.000000  0.000000e+00  0.000000  \n",
      "barnes            0.000000  0.000000e+00  0.000000  \n",
      "barbour           0.000000  0.000000e+00  0.000451  \n",
      "kroll             0.000000  0.000000e+00  0.000000  \n",
      "\n",
      "[6906 rows x 15 columns]\n"
     ]
    }
   ],
   "source": [
    "print model_artm.phi_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's additionally extract $\\Theta$ mtrix and print it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              3001      3002      3003      3004      3005      3006  \\\n",
      "topic_0   0.076121  0.035935  0.069378  0.136764  0.083617  0.011377   \n",
      "topic_1   0.095387  0.029030  0.034951  0.043516  0.064787  0.022496   \n",
      "topic_2   0.004087  0.048284  0.012102  0.006649  0.015708  0.520682   \n",
      "topic_3   0.057739  0.029823  0.122427  0.085727  0.060414  0.016142   \n",
      "topic_4   0.122660  0.046273  0.084217  0.077540  0.064804  0.014840   \n",
      "topic_5   0.025646  0.010737  0.024132  0.051743  0.059580  0.017807   \n",
      "topic_6   0.044074  0.017331  0.072042  0.042680  0.036825  0.017084   \n",
      "topic_7   0.061899  0.059855  0.022353  0.029254  0.002857  0.005222   \n",
      "topic_8   0.049225  0.048494  0.043635  0.095823  0.025636  0.016371   \n",
      "topic_9   0.070107  0.492084  0.014166  0.131111  0.200206  0.025601   \n",
      "topic_10  0.056218  0.023523  0.158903  0.092290  0.059759  0.239756   \n",
      "topic_11  0.158816  0.031754  0.028020  0.024615  0.041939  0.006004   \n",
      "topic_12  0.054255  0.027387  0.191633  0.078728  0.125391  0.052281   \n",
      "topic_13  0.056038  0.056568  0.025291  0.041948  0.110980  0.032068   \n",
      "topic_14  0.067729  0.042923  0.096752  0.061612  0.047496  0.002270   \n",
      "\n",
      "              3007      3008      3009      3010    ...         991   \\\n",
      "topic_0   0.153216  0.031591  0.066032  0.036219    ...     0.053795   \n",
      "topic_1   0.041968  0.009980  0.024485  0.055780    ...     0.119861   \n",
      "topic_2   0.010361  0.000000  0.012824  0.004855    ...     0.201032   \n",
      "topic_3   0.140113  0.095807  0.090738  0.080190    ...     0.015301   \n",
      "topic_4   0.028386  0.042167  0.044592  0.068055    ...     0.048995   \n",
      "topic_5   0.039975  0.080582  0.077005  0.067891    ...     0.014937   \n",
      "topic_6   0.186580  0.291586  0.159585  0.068007    ...     0.005185   \n",
      "topic_7   0.033841  0.019672  0.033055  0.071855    ...     0.035326   \n",
      "topic_8   0.021736  0.015335  0.030470  0.059429    ...     0.030472   \n",
      "topic_9   0.158546  0.344387  0.276788  0.279663    ...     0.025226   \n",
      "topic_10  0.032311  0.000000  0.048914  0.031847    ...     0.080713   \n",
      "topic_11  0.024322  0.002047  0.013785  0.018584    ...     0.272355   \n",
      "topic_12  0.044409  0.017861  0.013088  0.019473    ...     0.023370   \n",
      "topic_13  0.032618  0.034590  0.043202  0.063385    ...     0.040395   \n",
      "topic_14  0.051617  0.014395  0.065438  0.074768    ...     0.033035   \n",
      "\n",
      "              992       993       994       995       996       997   \\\n",
      "topic_0   0.057835  0.078560  0.037073  0.170305  0.023683  0.094523   \n",
      "topic_1   0.121933  0.180116  0.112602  0.032622  0.034982  0.143123   \n",
      "topic_2   0.000000  0.057726  0.002886  0.000632  0.007803  0.014957   \n",
      "topic_3   0.040474  0.066918  0.036633  0.043454  0.012304  0.051498   \n",
      "topic_4   0.071596  0.133851  0.082915  0.049283  0.021307  0.059609   \n",
      "topic_5   0.017675  0.021395  0.015159  0.030125  0.042343  0.039248   \n",
      "topic_6   0.009005  0.029403  0.040560  0.104222  0.106431  0.031618   \n",
      "topic_7   0.018613  0.005084  0.109385  0.062973  0.036562  0.036915   \n",
      "topic_8   0.185597  0.034914  0.050408  0.026470  0.023620  0.130382   \n",
      "topic_9   0.012500  0.009456  0.067889  0.088528  0.025921  0.030995   \n",
      "topic_10  0.040815  0.000000  0.018810  0.124785  0.104889  0.031240   \n",
      "topic_11  0.281438  0.232306  0.255559  0.022806  0.116224  0.068760   \n",
      "topic_12  0.014712  0.000000  0.058669  0.136899  0.352839  0.069516   \n",
      "topic_13  0.054435  0.081143  0.079657  0.036140  0.027119  0.173720   \n",
      "topic_14  0.073375  0.069129  0.031795  0.070756  0.063973  0.023897   \n",
      "\n",
      "              998       999       1000  \n",
      "topic_0   0.077524  0.081823  0.066142  \n",
      "topic_1   0.185704  0.103474  0.126820  \n",
      "topic_2   0.008666  0.007206  0.016567  \n",
      "topic_3   0.043518  0.090904  0.129180  \n",
      "topic_4   0.077166  0.076402  0.115954  \n",
      "topic_5   0.032495  0.027419  0.030624  \n",
      "topic_6   0.019817  0.011535  0.043325  \n",
      "topic_7   0.060655  0.054204  0.052605  \n",
      "topic_8   0.059885  0.065536  0.083408  \n",
      "topic_9   0.031605  0.023796  0.024791  \n",
      "topic_10  0.011905  0.001742  0.000308  \n",
      "topic_11  0.196675  0.146919  0.169095  \n",
      "topic_12  0.011672  0.079587  0.036432  \n",
      "topic_13  0.134102  0.135609  0.059862  \n",
      "topic_14  0.048609  0.093843  0.044887  \n",
      "\n",
      "[15 rows x 3430 columns]\n"
     ]
    }
   ],
   "source": [
    "theta_matrix = model_artm.get_theta()\n",
    "print theta_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model can be used to find $\\theta_d$ vectors for new documents via ARTM.transform() method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Empty DataFrame\n",
      "Columns: []\n",
      "Index: []\n"
     ]
    }
   ],
   "source": [
    "test_batch_vectorizer = artm.BatchVectorizer(data_format='batches', data_path='kos_test', batches=['test_docs.batch'])\n",
    "test_theta_matrix = model_artm.transform(batch_vectorizer=test_batch_vectorizer)\n",
    "print test_theta_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion\n",
    "\n",
    "Topic modeling task has an infinite set of solutions. It gives us freedom in our choice. Regularizers give an opportunity to get the result, that satisfacts several criteria (such as sparsity, interpretability) at the same time.\n",
    "\n",
    "Given example is a demonstrative one, one can choose more flexible strategies of regularization to get better result. The experiments with other, bigger collection can be proceeded in the same way as it was described above."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}