examples/02_memoized_agent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training a Simple Memoized Agent With Simulation Cache\n",
    "This notebook shows how to use the ```phyre``` API and simulation cache to train a simple memoized agent.\n",
    "\n",
    "For introduction to the ```phyre``` API, see [PHYRE intro notebook](./01_phyre_intro.ipynb)\n",
    "\n",
    "In the random agent example, we explored creating an agent that sampled actions at random from the action space of the tier. An alternate strategy is to have agents select or rank actions from a list of candidates. For this purpose, PHYRE provides a cache of the simulation results of 100k actions per task instance. The 100k actions in the cache are the same across all tasks. By only interacting with cached results of simulations, we can speed up the training process. This notebook shows how to build a simple memoized agent that selects actions based on the historical sucess rate of the action during training.\n",
    "\n",
    "### Run this in colab\n",
    "You can open and play with this notebook in colab. \n",
    "<a href=\"https://colab.research.google.com/github/facebookresearch/phyre/blob/master/examples/02_memoized_agent.ipynb\" target=\"_parent\">Open In Colab</a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install phyre and auxiliary packages.\n",
    "!pip install phyre matplotlib tqdm ipywidgets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import functools\n",
    "import multiprocessing\n",
    "import random\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tqdm\n",
    "\n",
    "import phyre"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "tier = 'ball'\n",
    "eval_setup = 'ball_cross_template'\n",
    "fold_id = 0\n",
    "random.seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1: The simulation cache\n",
    "Let's explore how to use the simulation cache. To start, let's use the ball tier and evaluate our agent across templates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We only use train and dev for building agents. In general test should be reserved for final evaluation of agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train, dev, test = phyre.get_fold(eval_setup, fold_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's load the cache. The cache contains 100k actions each simulated on each task in the ball tier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "cache = phyre.get_default_100k_cache(tier)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see which actions are cached by looking at the num-cached-actions X action-space ```action_array```. This array is built by `simulator.build_discrete_action_space` with a fixed seed=42."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache.action_array() and shape: (100000, 3) [[0.5986585  0.15601864 0.15599452]\n",
      " [0.83244264 0.21233912 0.18182497]\n",
      " [0.1834045  0.30424225 0.52475643]\n",
      " ...\n",
      " [0.7356543  0.7556098  0.0149308 ]\n",
      " [0.8846895  0.6563959  0.8496121 ]\n",
      " [0.5163206  0.3238494  0.6627394 ]]\n"
     ]
    }
   ],
   "source": [
    "print('cache.action_array() and shape:', cache.action_array.shape,\n",
    "      cache.action_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each task, we can can get a corresponding 1D array of the status of each of the actions in ```action_array``` simulated on the task. The values -1, 0, and 1 corresponds to NOT_SOLVED, INVALID, and SOLVED statuses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Randomly selected task: 00012:198\n",
      "Cached simulation status of actions on task 00012:198 : (100000,) [-1 -1  0 ... -1 -1  0]\n",
      "Share of SOLVED statuses: 0.12186\n"
     ]
    }
   ],
   "source": [
    "task_id = random.choice(train)\n",
    "print('Randomly selected task:', task_id)\n",
    "statuses = cache.load_simulation_states(task_id)\n",
    "print('Cached simulation status of actions on task', task_id, ':',\n",
    "      statuses.shape, statuses)\n",
    "print('Share of SOLVED statuses:', (statuses == phyre.SimulationStatus.SOLVED).mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's verify the status of a random action in the cache."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cached status is: -1\n",
      "Simulated status is: SimulationStatus.NOT_SOLVED\n",
      "Simulator considers task solved? False\n"
     ]
    }
   ],
   "source": [
    "cached_status = phyre.simulation_cache.INVALID\n",
    "while cached_status == 0:  # Let's make sure we chose a valid action.\n",
    "    action_index = random.randint(0, len(cache))\n",
    "    action = cache.action_array[action_index]\n",
    "    # Get the status for this action from the cache.\n",
    "    cached_status = statuses[action_index]\n",
    "\n",
    "# Now let's create a simulator for this task to simulate the action.\n",
    "simulator = phyre.initialize_simulator([task_id], tier)\n",
    "simulation = simulator.simulate_action(0,\n",
    "                                                     action,\n",
    "                                                     need_images=True)\n",
    "\n",
    "# Let's compare.\n",
    "print('Cached status is:', cached_status)\n",
    "print('Simulated status is:', simulation.status)\n",
    "print('Simulator considers task solved?', simulation.status.is_solved())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at an image of the last state to visually verify if the task is solved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWUAAAEXCAYAAAB1b1VxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debwcVZ338c+XQABJhITECGEJYBgMPBogIuOKgsr6BDcGUInbBBVGHXABXEBBR3F7dFA0KAKKLCrIIrghqIyPSkBlFYgQJIGEsCeALOE3f5zTpNLp9d5equ/9vl+vft2+daqqf11d/atTp06dVkRgZmblsFa/AzAzs1WclM3MSsRJ2cysRJyUzcxKxEnZzKxEnJTNzErESdnMrER6mpQlHSfp+718zW6QdIKkeyUt6fLrHCPp211Y73GSnpS0QtIGnV6/ma1O0rb5+7ZS0rsbzds0KUtaKOmxvMLK46TOhdtcjmGPNuY/TdIJXYplC+BIYEZEPLeD691N0qLitIj4bEQ0/ACH4ZyIGBcRj+TXl6TPS7ovPz4vSXVi3UfSlZIelLRE0rcljS+UT5V0gaT7JS2S9J6q5feTdH3el34vaUah7O15xy3ub7sVyqdJulzSo5L+VtwvGi0raYuq6SskhaQjW9lYbW6f3SQ9XfVacwrlEyWdL+kRSXdIOrhq+f+QdLukhyXNl/SyQlnxgFp5bF0of7Wka/Kyt0maW7XuyZJ+IOkhSQ9IOrNG/BMlLZN0ZWHaW6pe89G8/Xausw3WlXRqjmOJpCOabNsTJC3OcV0haftC+QF5P3lU0hU1lo+8LSuxfbtQtq6kb0pamvfHiyRNLZQ/X9Kv8+sukPT6OjF+Mr9OcX+7oWqbPCXpolrLR8QtETEO+F297VCcueEDWAjs0Wy+Vh7AccD3h7BcWzEApwEndCLmGut+GbCoC+vdrRvrbfVzAA4FbgY2A6YCNwLvqbP8wcCewLOACcClwDcL5ZcD/w9YB3ghcD/wqlw2HXg4b8e1gaOBBcDaufztwJUNYv//wJeB9YE3Ag8Ck1tZtmo9WwErgWktzt/O9mn4WQJnAecA4/J2eAjYPpe9GHgE2BkQ8F5gGTCm3mdXWO86eV2H5mVfBKwAXliY53d5+22Y59+xxnpOAX7b5HN4O/B3QHXK/yu/1gTg+cASYM868x4A3AVsDYzJy15TKN8jz/NJ4IoaywfwvDrr/gjwV2AKsB5wBnBeLlsbuAU4Ir/uq/O237ZqHdsA1+UYa+ahvL1vBw5psh9dAby74Twt7IwLGwTyduBK4IvAAzmovap2/N8Ay4FfAic12KEmAReTvmT35w90LeB7wNPAY3kH+0ie/4f5g34o70CVnXou8CTwRJ7/ojx9U+DHeQe/HXh/g/e8Yf7wlgF3AB/PseyR43g6r/u0GstOyO9jWd4mFwObFconAt/NH/ADwE+ADarWuyLHe1xxewH/F7ghb6MrgOdXfU4fAq7N2+QcYL0672+19eZpvwfmFv5/F/CHZvtHnvcNwHX5+TjSl2RyoXwe8L38/HDgp4WytfJ73724T9V5nW2Bx4HxVUnmPc2WrbGuY4HLW5m33e1Dg6ScP+snKHzxSfv45/LzfwP+VDV/AJvU++wK807J8z6rMO0q4KD8/LV5PxnT4H2+hHTge0ejbUk68B7boPwu4LWF/48Hzq4z70eBcwv/bw/8s8Z876b9pHwycGLh/32Am/PzHUjfNRXKfwEcX7WOnwF70zgXvpKU5zZosh9dQZOk3Ik25ReTahCTgBOB7xRO634AXJ3Ljgfm1FxDciSwCJhM2rmOASIi3gb8A9gv0un2iXn+S0m1rucA1wBnkhaYl5+fmOffT9JawEWkI+ZUYHfgg5JeVyeW/yYl5q1JG/sQ4B0R8StgL+CuvO6311h2LVLS3RLYgpRwis093yPVMLfPsX8lUhNCcb3jIuKu4kolbUuqYX0wb6NLgIskjS3MdgCpBrsV8AJSkmrV9qTtU/HXPK0VryAdLCDVGIp/K893qPq/+Ly6fEelNvtbJH1C0tqFGG+LiOUN4qy37KoXTPvnIcDpLb6/ymu3s32ek0+Zb5f0Fa1qu98WeCoibqmzrkuBMZJeLGkM8E7gL6QKSMV++VT8BknvrUyMiKWkfeQdksZI+lfSflhphtiV9F09PTfBXCXplZXl8+udRDpw1h0UR9KWpM/8jDrlE4BNaH17nQ1so9Tuug4pT/ys3uvX8dvcTHKepGmF6d8BXippU0nPAt5C2sb1rLYvSnoz8HhEXNLk9ecAP87f5eFpoYawkHQ0ebDw+PdYVTNZUJj3WaQP87mkhPQUhSMHKUnXO8p/GriAGkc8mjRfABvl190w/38aheYL0oHjH1XLHA18t8a6xpBqMjMK0w4lH6Fps5kBmAk8kJ9vQqoNT6gx3xrrpVArAj7B6rWJtYDFwG6FbfTWQvmJFJoU6q23MG0lsF3h/+l5m9Y8PS3M9xpSjb9Y87uSdGBbD9iJdOZTqZ1sRzpF3A0Ym9/X08DRuXxr0kFlLeD/kJoJKmVvo6p2CnyGfMbSaNmqZV5O2qfHtfE5trx9SPv/jBzHVqQzuW8VXntJ1fz/Xti/RKqQPEn6/twLvKgw7wzSWdQYUq32bnJNOJfvByzNyz5F/q7msnk55neRmi4OJH2fJ+Xy/wROLny3652xfIIaNdZC+eb5ddYrTHsNsLDO/GOBr+ZlniKdyW5VY756NeVX5HVsRDqoXM+q5rANSUm/su4/AxNz2TrAbaQmjnVIZxJPAD/P5eOBW8lNXNTJQ6S89zD5u9hkP7qCDtWU94+IjQqPUwplzxzBI+LR/HQcacd5IFY/ctzR4DW+QGpb/EW+QHFUvRlzLeBzkv4u6eG8sSDVyGvZEthU6cLUg5IeJO34U2rMO4n0ARVjvYNUw25K0rMkfUvpAs7DpC/kRrkWsjlwf0Q80Mq6qmxajCkingburIqrWJt6lPQ5tGoF8OzC/88GVkTek2qRtCvpQPumWL3m9xZSMrqTdPr4fdJZEBHxN1Kt4iRSQplESp6V8tsi4vaIeDoiriMdrN9UJ8ZKnMtbWLaoUqtZ0WB7VGt5+0TEkoi4McdxO+lL/8ZW3gMpYb6DVKscC7wVuFjSpnndN0bEXRGxMiJ+T0pmbwKQtB0pAR2Sl90e+IikffK6HyMlxu9ExJMRcTbpM3ppXv/7gY+1sC2anWVUtmv19lpeY15IbcUvIn0/1gM+Bfw612ybiojfRsQTEfEg8AHSvvf8XPx1YF1gY1JT0HnkmnJEPAnsT2rSWEI6Wz+XvC+SKi/fi4iFTUJ4A6ni8ZtW4m2mm13i7gYmaPUuV1vUmzkilkfEkRGxNant9AhJu1eKq2Y/GJhNauPdEJiWp6vO/HcCt1cdWMZHxN41QrmXVEvZsiruxfVir3Ik8C/AiyPi2aSjeCW2O4GJkjaqsVzd5JfdVYwpn4Jv3kZczdxAuihX8UJWNUmsQdKOwIXAOyPismJZRNwREftGxOSIeDEp8f6pUP6jiNghIjYmte1OI7V91hKs+lxvALZWoadHkziLy1biXh94M+01XVReu+XtUyOOynftFmBtSdPrrGsmcHGkq/VPR8TPSN+llzRYd+U97gDcEhE/z8veDPyU1DQG6XpD9X5W+X8X0pncjUpdPb8K7JKbBMZUZpb0UlIF4Ud132yqdNxN69trJqk30KKIeCoiTiNdm5lRZ/5mittkJulM6v6IeJx0BreLpEk51msj4pURsXFEvI50tlXZV3cH3p+3wRLS9+1cSR+ter05wBmNKjDt6FpSjog7gPnApySNVerWs1+9+SXtK+l5Odk8RDpdfDoXLyVtrIrxpAs+95FOHT5btbrq+f8ELJf0UUnr55r2DpJeVCPulaSj5Wckjc/tZ0eQanutGE+qkTwoaSIp6VTWfTfpKP0NSRMkrSOpkrSXAhtL2rDOes8F9pG0e253OzJvg9+3GFczZ5AOhFNzrelIUjPQGiTtQGrz+4+IWKMLUO5mND5/7m8lnRZ+uVC+c/4MJpNOqS/MNWgk7SVpSn6+HelU+QJI3YpI7avHSlovd196AekCbsNlC15Pam65vIvb51WStlSyOfC5wnt4hFRb+7SkDXKSm0261gDp4LSPpK3z8q8htUNfn9c9O+87krQLqXZbeY9/BqYrdYuTpG2AfUnJGOB8UkVpTt7+byL1Jvkf0n45jZTEZpJqr38GZubvREXlLKNerbe4vT6eY92O1ERTc3vl9/xmSVMkrSXpbaSz1QX5PY+RtB6pt8Ra+bNfJ5dtL2lmnmcc8CVSReWmwroPkbRhXuZ9pGs39+blX5DX9yxJHyIdmCpx7k460FW2yV2kpsyvVwKXtBnwKto/yNfXQhvIQlb1fKg8zo867U4UroSSEuPv8jLNel/8Z36tR0inD58olM0mXex7kNTDYBxpR1xOOqU/pOp1p5O+vA8CP8nTNiVdBFlC+lL+gfpXUieQkvAyUu32k8BauWw3Gnd32pTUbrSCVCs6NMdWaeOaSPoAl+Y4zisseyrpQPMgtXtfvJ50qv8Q6VRp+6rPaY/C/6stWxXjGmWkmsWJpNOw+/Pz4lXpFcDL8/PvsnpPkRXADYV5P5i33SOk9uVZVa91Zf7s7ge+xerXHb6Yt80jpPa+TwPrFMqn5e37GOmi1R6tLpvn+TlVV9dbebS5fY4gJYZH8/7zNVbvMTKR1OvmEdJ+fXDV63w6T19OSi5vK5SflfeRFcDfqOpFRLrYe31edhHwefK+m8tfTuretYJUaXp5nff7dtb8bq9H2jd3b2F7rUvanx/On8kRhbIt8utvUVjv10m164dJF+73rIolqh6n5bJX5/3gEeCevF2nF5bdmHTh/54c+5XALoXyL5C+hytIB6aavThqfcfytKOB37WxH11BkzZl5RltFJH0cdLO9CQwNTpxxdjM6srNVVeR2vrfF6mJpva8TspmZuXhAYnMzErESdnMrESclA3o7iBOI51WDXY0pvncayy7xkBUbSw7TWmQnLXz/1eoyQhkVctfqsJASVYOTsoDbDhf6GG+bsMRwHK3vb8pjep1ee5W2HTZ3IXuR0qjAoYKo8Pl8g8rjS63XOn25Q83ifN4Sdcpjd51XFWZJH1M0j9yLGdLenahfKKkc5RuR75X0pnF8qKI+EekW+NX1iovA9UYNjci9oqIznXlso5wUrahOI7U7XBLUh/Nj0jaE0CpU/55pD7CE0ndrs5pZdnsStJdbLXGqq6MWTGBNMbH4ZIObBDnAtLddD+tUXYI6bbtys0Q65NuLKg4Ib/OVqRRwqbk2Ec91RhPxDqo3f6afgz/Qbo76ypSf+OrgJdU9WM8ntShfzlp1KpJNdZRb2S5XUijfD1I6vd5EjA2LyPgK6Q+mw+T+qvukMtOI48XQroB5nJS/9paYzvUHQGMNErf72vEuV2zZateYxFNxhLI8f13C9v7+8BxVdN+BHy46jP5J3mENVKf1fcVyg8jj4lQY/3TWL0vekufYZ53t/xejyHdTboQeEuhfB/STRwPk/o8H9fkddfoA0s6gD1B6gK5AvhrrflJgx/dROq3+3Ngy0JZ5G1wK+nu2L5/j0bqwzXlHlO6y++npISyMelOt59K2rgw28Gk8Q+eQ+rX+KHq9UT9keVWkm7EmQT8K+mupPflxV5Luu17W9Lt6QeQbkQoxrcxcBnwPxHx/sjfyEJ5sxHAVhtNLcf5d2D7FpZtmSSRboS4oTDtYjUYM6XWaqqer0uqxUO6mWHffEfaBNLYFY1GF6vW9DMseC7p85pKumNunqR/yWWPkGr1G5ES9Hsl7d9GHES6VfuzrPphgxdWzyNpNunA8AbSKIS/I92oUrQ/aXCvod7+bC1wUu69fYBbI+J7ke7zP4t0Z1bxFvTvRhr74DHS7dUzW115RFwdEX/I615IumPulbn4SVIteDtSDfimSLd+V2xKulPwhxHx8TovURnk6KHCtIfyeivlD7G6SnmzZdtxHKuGSQUg0ngbn2tx+Z8B784XyzYkjekL6bZ9SHeVjSUdtO4jHey+0UZ87X6Gn4iIxyPiN6SD9gEAEXFFRFwXaSyLa0mJ8pWNVjRE7wH+K+8TT5GS+Mzi9YBcfn9+T9YlTsq9t9pob1n1KHRDHu1NaUzai/NFtIdJX67K4Cu/JjVnfB24R9K8qotX+5DaVr/Z4CWajQDWaBS0dkcPq0nS4aTa4z6RBpkZilNJCe4KUm27MhZG5cLpuaTb5MfnGP9O6+OfQHufYa3RFDcFUBpX+XKln2d6iJQ8642GOBxbAl/VqlEU7yedPRT3yzu78LpWxUm591Yb7S1rZxS6olq3Y55MqnlPjzRK3TEUTtMj4msRsTPpFHRboNiD4RRSDfIS1flB1Wg+Athqo6nl9WxDGhuj3dHD1iDpncBRpPEXhtzzJNc8j42IaRGxWY5hMas+h5mkMZAfiTTE5zdJvz7RDbVGU6z8yMEPSKPxbR4RG+Y4av42YBPNbt29Ezg0Vh9Jcf1Iw4O2ug7rACfl3rsE2FbSwZLWlvRvpAR58RDWVWtkufGki0Ir8uhcz/wyhaQX5ZrXOqS2yn+yaiS+isNJA7xcpDTMZS2NRgA7H9hB0huVRvb6JHBt5FHgmixb6TK3Xv53bB7BS7nsLaSa/2si4rZmG0dpFL71SPv52nldY3LZREnb5K5xM0ht+5+ONE41pAuw71YaVXB90gXMa2u9Tod8KncJfDlpZLcf5unjSWNw/1NpVLiD666hsaXANKVf4anlm8DRyj9YqjSq2puH+Fo2DE7KPRYR95G+dEeS2io/AuwbeSjBNtf1N9Ip+G35tHNT0gWlg0lNAqewene0Z+dpD5BOke8jjZJVXGeQEtAi4IJCgiw6lnQ6fwepDfoL+WISEbGMdFHsM/l1Xkz6hYumy2Y3k3prTCX1AHiMVWcWJ5Aujl6lVb8g/ExTi9LNEMcU1nVKXv4g0uDtj5G6wUFqAriEdHC6FDg10k+JVbyT1LthEan2vDWNf85sOCojF95FGtHsPYWD2PtIw3wuJx3gzh3ia1SS/H2SrqkujIjzSSPKnZ2bva5n1TjM1kMekMjMrERcUzYzKxEnZTOzEhlVSVnSnpJulrSgzZsMzMx6YtS0Keer7reQfup8Eenq+kERcWNfAzMzKxhNA4vsAiyodKWSdDbpt/9qJuVJkybFtGnTehed2Sh09dVX3xsRk/sdR5mMpqQ8ldXvSFpE6q71DElzSd3B2GKLLZg/f37vojMbhSRV39066o2qNuVmImJeRMyKiFmTJ/vgbWa9N5qS8mJg88L/mzG0W5vNzLpmNCXlq4DpkraSNJZ0l9mFfY7JzGw1o6ZNOSKeyqOL/RwYQ7qttuWBcMzMemHUJGWAiLiENN6BmVkpjabmCzOz0nNSNjMrESdlM7MScVI2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM7MScVI2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM7MScVI2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM7MScVI2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM7MScVI2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM7MSWbvfAXSapIXAcmAl8FREzJI0ETgHmAYsBA6IiAf6FaOZWT0jtab8qoiYGRGz8v9HAZdFxHTgsvy/mVnpjNSkXG02cHp+fjqwfx9jMTOrayQm5QB+IelqSXPztCkRcXd+vgSYUmtBSXMlzZc0f9myZb2I1cxsNSOuTRl4WUQslvQc4JeS/lYsjIiQFLUWjIh5wDyAWbNm1ZzHzKybRlxNOSIW57/3AOcDuwBLJW0CkP/e078IzczqG1FJWdIGksZXngOvBa4HLgTm5NnmABf0J0Izs8ZGWvPFFOB8SZDe2w8i4meSrgLOlfQu4A7ggD7GaGZW14hKyhFxG/DCGtPvA3bvfURmZu0ZUc0XZmaDzknZzKxEnJTNzErESdnMrESclM3MSsRJ2cysRJyUzcxKxEnZzKxEnJTNzErESdnMrESclM3MSmREjX1hvZUHfnpGhIegNhsu15RtSKoTspl1hpOyta1eQnaiNhs+J2XrKCdms+FxUjYzKxEnZWuLa8Jm3eXeFyNcrSQ6nF4SEeHEbNZFrimPYL4gZzZ4XFPusk7XVDtF0pDjaFRbLsN7Mxtkril30WirqTohmw2fa8pd0izxDqem2inDrS2bWefJX67aJD2zYdrdRq3WhLu97ZvF4c/e+k3S1RExq99xlImbL1owUpsbzKx8nJRHKdeSzcrJSbkLypLw6sVRlvjMbE2+0NcnvUqMTsBmg8U15S5plAydKM2sHteUW+BuY2bWK64pm5mViGvKDbima2a95ppyHTvvvHO/QzCzUchJ2cysRJyUzcxKZCCTsqRTJd0j6frCtImSfinp1vx3Qp4uSV+TtEDStZJ26l/kZmaNDWRSBk4D9qyadhRwWURMBy7L/wPsBUzPj7nAyT2K0cysbQOZlCPit8D9VZNnA6fn56cD+xemnxHJH4CNJG3Sm0jNzNozkEm5jikRcXd+vgSYkp9PBe4szLcoT1uDpLmS5kuav2zZsu5FamZWx0hKys+I1MG47U7GETEvImZFxKzJkyd3ITIzs8ZGUlJeWmmWyH/vydMXA5sX5tssTzMzK52RlJQvBObk53OACwrTD8m9MHYFHio0c5iZlcpA3mYt6SxgN2CSpEXAscDngHMlvQu4Azggz34JsDewAHgUeEfPAzYza9FAJuWIOKhO0e415g3gsO5GZGbWGSOp+cLMbOA5KZuZlYiTsplZiTgpm5mViJOymVmJOCmbmZWIk7KZWYk4KZuZlYiTsplZiTgpm5mViJOymVmJDOTYF2bWeZKeeZ6GjLF+cFI266Fi4qvodwKsFZOkvsc1Wrn5wqxHaiW/RtN7odFr9zOu0cw1ZbMeaJbgWqmZNltHN2q2teIqY21/JHFS7oEV+7+ubtm4n/y8h5FYP/SqxtmLJodGtX0n5s5wUu6CRkm40bxO0FbPIDQlODF3hpNyB7WTjJst7wQ9ckTEQCTVZkbCexgEvtDXASv2f92wE3KtdZrZ6OOkPEzdTJ5OzCNHs9P6fp329ysuSa551+HmiyHqVcJ0k8bIUUlwlWTUTsJrpQlkqAm01eU60QzjRNyck/IQuAZrwzHc5FnWLmlliGEkcPNFm/qZkH0wMEjJr/rRy9duZ7q1z0m5DWVIimWIwUa3fh4URgMn5RaVKRmWKRYz6ywnZTOzEnFSbkEZa6bd6Btt1m1u6mjOSdnMesqJuTEn5SbKXhste3xmtfgCYX1OymZmJeKkbGZWIk7KDbhpwMx6zUnZzKxEBjIpSzpV0j2Sri9MO07SYkl/yY+9C2VHS1og6WZJLVV/n15wazdC7wrX6M1GjoFMysBpwJ41pn8lImbmxyUAkmYABwLb52W+IWlMzyI1M2vDQCbliPgtcH+Ls88Gzo6IxyPidmABsEvXgjMzG4aBTMoNHC7p2ty8MSFPmwrcWZhnUZ62BklzJc2XNP/eJ57odqxmZmsYSUn5ZGAbYCZwN/CldlcQEfMiYlZEzJo0dmyn4zMza2rEJOWIWBoRKyPiaeAUVjVRLAY2L8y6WZ5mZlY6IyYpS9qk8O/rgUrPjAuBAyWtK2krYDrwp17HZ2bWioH8OShJZwG7AZMkLQKOBXaTNBMIYCFwKEBE3CDpXOBG4CngsIhY2Y+4zcyaGcikHBEH1Zj8nQbzfwb4TPciMjPrjBHTfNFpaz1ver9DaJl/5dps5HBSNjMrkYFsvrDRR9Izzz0Or41krik34GaBcigm5Mr/1dPMRgonZSu1RsnXidlGIiflJspeWy57fMPhpGujkZOydZ2Tq1nrfKFvgPWyltwosda68FarHbjR/GaWuKbcgjI2EZQppkYJuJX5zWwVJ+UWlSkJ9jqWfiVR16htNHJSbkMZEvNwY+hWd7LKOnuZwJ20bSRyUm5TGRLzUFQn40Hp61sv8Toh20jlC31DUEnMvfzB0qEeDFpt3x1ukqssHxEdT/ZOwDaauKY8APpdO283KTab30nWrD7XlIeh2zXmfifjVtVKsvVqzE7IZo25ptwB3UieZUvIEbFGQq01rVZ58WFmjbmm3CGdqjWXLRlXc2I16y75S1abpBjutmk1QXc7ETe78OZ9wPpF0tURMavfcZSJa8oNtNIzod3bj2suV7WOTifJyvqqY3UyNisftym3oF7iHertxP26Ddntu2bl56RsZlYiTspdNgh3zZlZeTgpm5mViJNyl7nt1sza4aTcgqEOiuPBdMysXU7Kw9SNhO2kbTZ6OSnXsfPOO7ecIIvztZNUi/M5GZsZ+OaRjhpKUnUiNrMi15TNzErESdnMrESclM3MSsRJ2cysRJyUzcxKZCCTsqTNJV0u6UZJN0j6QJ4+UdIvJd2a/07I0yXpa5IWSLpW0k79fQdmZrUNZFIGngKOjIgZwK7AYZJmAEcBl0XEdOCy/D/AXsD0/JgLnNz7kM3MmhvIpBwRd0fENfn5cuAmYCowGzg9z3Y6sH9+Phs4I5I/ABtJ2qTHYZuZNTWQSblI0jRgR+CPwJSIuDsXLQGm5OdTgTsLiy3K08zMSmWgk7KkccCPgQ9GxMPFsvwDe23dLidprqT5kuYvW7asg5GambVmYJOypHVICfnMiDgvT15aaZbIf+/J0xcDmxcW3yxPW01EzIuIWRExa/Lkyd0L3sysjoFMyko/5/Ed4KaI+HKh6EJgTn4+B7igMP2Q3AtjV+ChQjOHmVlpDOqARC8F3gZcJ+kvedoxwOeAcyW9C7gDOCCXXQLsDSwAHgXe0dtwzUa2w//8qzWmnbTjHn2IZPDJo5TVNmvWrJg/f36/wzArrVqJuJnqRC3p6oiY1amYRoKBbL4ws/4aSkIeznKjiZOymbXFibW7nJTNrGWdSMhO6o05KZuZlYiTspn1nGvL9Tkpm1lLOp1InZhrc1I2MysRJ2UzsxJxUjYzKxEnZTOzEnFSNjMrESdlM2tJpwcY8oBFtTkpm5mViKZx6+AAAAWSSURBVJOymfWca8n1OSmbjVLptyKsbJyUzUYZSc8k5MrzVhN0J2q4riU3Nqi/PGLWEY2SUdl/AKIYeyuxNku8kuqup9ayh13zy6avWe2kHfdwDb0J15Rt1Brk5FAdezu13XbWW28awNd3ek1b63ZCbo1rymYDplFia1Tb7fRrQUrMEVF3cCE3VbTPSdmsjk4nuJHMybdz3HxhViLNmiF6efrvpob+cFI2q6OXteTqZNypNuKiVt6Pzwz6z0nZRq0yJKB2a8bDjbne8hGxRlkZts9o5DZlG9UiomZSHMkJqZPvrd111dvetopryjbqFRNLrRpj2fQyvkavNdQ4yr59+801ZTMGL1H0soZfXGeneqRU1uFa85qclM0GVD8OJIN28BpEbr4w6yMnOavmmrJZn9U7lXfCHp2clM1KwknYwM0XZmal4qRsZlYiTspmZiXipGxmViIDmZQlbS7pckk3SrpB0gfy9OMkLZb0l/zYu7DM0ZIWSLpZ0uv6F72ZWX2D2vviKeDIiLhG0njgakmV36b5SkR8sTizpBnAgcD2wKbAryRtGxErexq1mVkTA1lTjoi7I+Ka/Hw5cBMwtcEis4GzI+LxiLgdWADs0v1IzczaM5BJuUjSNGBH4I950uGSrpV0qqQJedpU4M7CYouokcQlzZU0X9L8ZcuWdTFqM7PaBrX5AgBJ44AfAx+MiIclnQwcD0T++yXgna2uLyLmAfMANtpw69h3nzM7H7SZWQMDW1OWtA4pIZ8ZEecBRMTSiFgZEU8Dp7CqiWIxsHlh8c3yNDOzUhnIpKw0SMB3gJsi4suF6ZsUZns9cH1+fiFwoKR1JW0FTAf+1Kt4zcxaNajNFy8F3gZcJ+kvedoxwEGSZpKaLxYChwJExA2SzgVuJPXcOMw9L8ysjORBUGqTtAx4BLi337G0YBKDEScMTqyOs/NqxbplREzuRzBl5aTcgKT5ETGr33E0MyhxwuDE6jg7b5Bi7aeBbFM2MxupnJTNzErESbmxef0OoEWDEicMTqyOs/MGKda+cZuymVmJuKZsZlYiTspmZiXipFyDpD3zuMsLJB3V73iqSVoo6bo8ZvT8PG2ipF9KujX/ndBsPV2I61RJ90i6vjCtZlxKvpa38bWSdipBrKUbj7vB2OGl2q4e47yDIsKPwgMYA/wd2BoYC/wVmNHvuKpiXAhMqpp2InBUfn4U8Pk+xPUKYCfg+mZxAXsDlwICdgX+WIJYjwM+VGPeGXk/WBfYKu8fY3oU5ybATvn5eOCWHE+ptmuDOEu3Tcv+cE15TbsACyLitoh4AjibNB5z2c0GTs/PTwf273UAEfFb4P6qyfXimg2cEckfgI2qxi7pqjqx1tO38bij/tjhpdquDeKsx2Oc1+GkvKaWxl7uswB+IelqSXPztCkRcXd+vgSY0p/Q1lAvrrJu5yGPx91tVWOHl3a7dnKM89HISXkwvSwidgL2Ag6T9IpiYaTzw9L1dSxrXAUnA9sAM4G7SeNxl0L12OHFsjJt1xpxlnablpWT8ppKP/ZyRCzOf+8Bzied9i2tnKbmv/f0L8LV1IurdNs5Sjoed62xwynhdvUY553hpLymq4DpkraSNJb0g6sX9jmmZ0jaQOnHYpG0AfBa0rjRFwJz8mxzgAv6E+Ea6sV1IXBI7i2wK/BQ4XS8L8o4Hne9scMp2XatF2cZt2np9ftKYxkfpCvYt5CuCH+s3/FUxbY16ar1X4EbKvEBGwOXAbcCvwIm9iG2s0inqE+S2gjfVS8uUu+Ar+dtfB0wqwSxfi/Hci0paWxSmP9jOdabgb16GOfLSE0T1wJ/yY+9y7ZdG8RZum1a9odvszYzKxE3X5iZlYiTsplZiTgpm5mViJOymVmJOCmbmZWIk7KZWYk4KZuZlcj/AjnybneXiZkSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "img = phyre.vis.observations_to_float_rgb(simulation.images[-1])\n",
    "plt.imshow(img)\n",
    "plt.title(f'End state of action {action}\\n on task {task_id} in {tier} tier')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, we can get cached results on a list of tasks. We can also limit the number of actions to the first N in the cache."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of actions in our small cache is (100, 3)\n"
     ]
    }
   ],
   "source": [
    "example_tasks = dev[:100]\n",
    "cache_dict = cache.get_sample(example_tasks, 100)\n",
    "print('The shape of actions in our small cache is',\n",
    "      cache_dict['actions'].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number and tasks in our small cache are 100 ('00000:000', '00000:001', '00000:002', '00000:003', '00000:004', '00000:005', '00000:006', '00000:007', '00000:008', '00000:009', '00000:010', '00000:011', '00000:012', '00000:013', '00000:014', '00000:015', '00000:016', '00000:017', '00000:018', '00000:019', '00000:020', '00000:021', '00000:022', '00000:023', '00000:024', '00000:025', '00000:026', '00000:027', '00000:028', '00000:029', '00000:030', '00000:031', '00000:032', '00000:033', '00000:034', '00000:035', '00000:036', '00000:037', '00000:038', '00000:039', '00000:040', '00000:041', '00000:042', '00000:043', '00000:044', '00000:045', '00000:046', '00000:047', '00000:048', '00000:049', '00000:050', '00000:051', '00000:052', '00000:053', '00000:054', '00000:055', '00000:056', '00000:057', '00000:058', '00000:059', '00000:060', '00000:061', '00000:062', '00000:063', '00000:064', '00000:065', '00000:066', '00000:067', '00000:068', '00000:069', '00000:070', '00000:071', '00000:072', '00000:073', '00000:074', '00000:075', '00000:076', '00000:077', '00000:078', '00000:079', '00000:080', '00000:081', '00000:082', '00000:083', '00000:084', '00000:085', '00000:086', '00000:087', '00000:088', '00000:089', '00000:090', '00000:091', '00000:092', '00000:093', '00000:094', '00000:095', '00000:096', '00000:097', '00000:098', '00000:099')\n"
     ]
    }
   ],
   "source": [
    "print('The number and tasks in our small cache are',\n",
    "      len(cache_dict['task_ids']), cache_dict['task_ids'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also receive a status matrix simulation_statuses of size ```len(cache_dict['task_ids'])``` X ```len(cache_dict['actions'])``` where ```entry[i][j]``` is the status of simulating action ```cache_dict['actions'][j]``` on task id ```cache_dict['task_ids'][i]```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Simulation cache and shape (100, 100) [[-1 -1 -1 ... -1 -1 -1]\n",
      " [-1 -1  0 ... -1 -1 -1]\n",
      " [-1 -1 -1 ...  1 -1 -1]\n",
      " ...\n",
      " [-1 -1 -1 ... -1 -1 -1]\n",
      " [-1 -1  0 ... -1 -1 -1]\n",
      " [-1 -1 -1 ... -1 -1 -1]]\n"
     ]
    }
   ],
   "source": [
    "print('Simulation cache and shape', cache_dict['simulation_statuses'].shape,\n",
    "      cache_dict['simulation_statuses'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize the status of the different actions on the tasks. In the visualization, each row corresponds to a specific task and each column to an action.\n",
    "\n",
    "|Color|Simulation Status|\n",
    "| :-------------: | :----------:|\n",
    "|Purple|Not Solved|\n",
    "|Yellow|Invalid Input|\n",
    "|Green|Solved|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfFklEQVR4nO2de/QlVXXnP9/w6vAQaMwgDZ2ANuAijGhWB0EzEyJmQCTqZBFBjBKHLFaiRkAdB51H0ElcmgkBZkxMDBPHGBB8TTBE7ZgWJmOWgE1gNILIU7p5KCKvQGxp3fNHVUF1UY9TVace93f3Z61e/bv1OGfXuffU3mefffaRmeE4zsrnx6YWwHGccfDO7jhLgnd2x1kSvLM7zpLgnd1xlgTv7I6zJHhnnxmSzpX0FyPU8y5JFw1cx/+S9Dvp3/9K0s0D1PFaSX8Tu9yViHf2Hkg6VdImSf8k6V5Jn5P0c1PLFYKZvdfMfn3E+v6vmR3apwxJB0oySTvmyr3YzP5NfwlXPt7ZOyLprcAFwHuBfYGfBP4IeOWUcjlOFd7ZOyBpT+A9wJvM7NNm9piZPWFmf2Vm/z695khJX5b0UKr1PyBp51wZPy3pC5K+J+nbkt6Vq2JnSX8u6VFJX5e0PnffGkmfknS/pDskvaVCxhdKuk/SDrlj/1bSV9O/nxwuSFol6S8kPZDK+xVJ+6bn7pT00lwZ2w0zJH0iredhSX8n6acr5DlG0pb075NTayj7t1XSVem5l0u6XtIjkjZLOjdXzN+l/z+U3ne0pF+T9KVcPS9K5X84/f9FuXNXSfqvkv4+bdu/kfTMMnlXIt7Zu3E0sAr43zXX/BA4G3hmev2xwBsBJO0B/C3weWANsA7YmLv3FcClwF7AZ4APpPf9GPBXwP8D9k/LPEvSccXKzewa4DHgJbnDpwKXlMh6GrAnsBbYB/gN4J9rni3P54CDgX8B/ANwcdMNZnaZme1uZruTPP/twMfS048Bryd59pcDvynpVem5f53+v1d6/5fz5UpaDfw18N/T5/gD4K8l7ZO77FTgDam8OwNvD3zOhcc7ezf2Ab5rZtuqLjCz68zsajPbZmZ3An8C/Hx6+kTgPjM7z8y+b2aPpp0z40tm9lkz+yHwUeCI9PjPAj9hZu8xsx+Y2e3AnwKnVIjxMeA18OQL5gSe6lR5nkifaZ2Z/TCV/ZHmZgAz+7NU/q3AucARqeXTSPryugS4ysz+JC3vKjP7mpn9yMy+msr783Xl5Hg5cIuZfTRt948B3wB+KXfNh83sm2b2z8DHgecHlr3weGfvxgPAM/OOoiKSDpF0RWriPkIyts9MxrXAbTXl35f7+3FgVVrXTwFrUlP7IUkPAe8i8RmUcQnwy5J2AX4Z+Acz+1bJdR8FNgCXSrpH0u9J2qlGvuwZd5D0Pkm3pc94Z3oq1DT+XWAP4MmhSDr8uDIdpjxMYmWElrcGKD7ft0isoIxi2+4eWPbC4529G18GtgKvqrnmgyRa5WAzewZJp1R6bjPw7A71bgbuMLO9cv/2MLMTyi42sxtJfuwvo9qEJ/U3vNvMDgNeRGJ5vD49/Riwa+7yZ+X+PpXEIflSkmHAgelx0YCkU0isjpPM7IncqUtIhi5rzWxP4I9z5TUt0byH5IWY5yeBu5vkWQa8s3fAzB4G/gvwh5JeJWlXSTtJepmk30sv2wN4BPgnSc8FfjNXxBXAfpLOkrSLpD0kvTCg6muBRyX9B0k/nmrWwyX9bM09lwBnkox3P1F2gaRfkPQvU2feIyRm/Y/S0zcAp6TPtx44KXfrHiQvvQdIXgjvDXgGJL0A+B/Aq8zs/sLpPYDvmdn3JR1J8kLJuD+Vq+pF+VngECVTojtKOhk4jKS9lx7v7B0xs/OAtwL/ieRHuBl4M/CX6SVvJ/mhPkoyrr4sd++jwC+SjCXvA24BfiGgzh+SaN3nA3cA3wUuItGqVWRj3i+a2XcrrnkW8EmSjn4T8H9ITHuA/ww8B3gQeDfbWwd/TmI53A3cCFzd9AwprwT2Br6U88h/Lj33RuA9kh4leaF+PLvJzB4nMf3/Ph3GHJUv1MweIGmft5G8gN4BnFjz3EuFPHmF4ywHrtkdZ0nwzu44S0Kvzi7peEk3S7pV0jmxhHIcJz6dx+yp5/abJI6mLcBXgNek0z2O48yMyqCQAI4Ebk2juJB0KYmXtbKz76xdbBW7lZ7bujY5vsvmx1oLUnVvdjxP1TXZ8fw9XWQZm7p2q3q2uuc65HmPA/DNr+5aeU0M2ULqiSnLotD0HTWd/z6P8QPbWhrn0Kez708y3ZSxBXjaXLGkM4AzAFaxKy/UsaWF3fq2ZBZl3dmhszfN92bH81Rdkx3P39NFlrGpa7eqZ6t7rg0bbgDguDX9o0jr6gupJ6Ysi0LTd9R0/hrbWHoc+pnxJwHHZ2uiJb0OeKGZvbnqnmdoteU7+63nP71jZccWoaN1oeyZi+fqnr1P+6z0tg1hpbfBNbaRR+x7pZq9j4PubpIY74wD8LBEx5ktfTT7jiQOumNJOvlXgFPN7OtV9xQ1+6KwSNpgwz03PPl3lfmbXVN2vu4cbG+Z3HbyHwPwnMt+A3h6+9SVVWzTsjaeut2z+rPnhG5tWnVtWbtVnSu2RdX3XKfZO4/ZzWybpDeTrJbaAfizuo7uOM609HHQYWafJVl84DjOzBk1Nr5oxoeYnCG0MaG6kDddodpczVNl2tbR5jmKJmafZ69zlBYJeZ425nex7fLP0dQebdq9eG12XaicoTLFomqY0/R9D+WgcxxngZhEs5dpjblMvZVpuTZOqya5h55661t+F6rap+57noqpf19NhH6/dfPsrtkdZ8mZdMxeR5c38FDjqaYxe1/ajE2HGCuGBDeVfR9FueumlMYm5m+hrv2rfqdlU5R92qmuP+Sf1TW74zjz1eyLRJlXOKZGaeOh7kLI2Dr2WLfJ2wzDteHcCPH5hD6Ha3bHcVyzhzB3D+6YtJl5mCJcNvSekOcY+nuvK7/qXJOGd83uOM58NHvoW7SNZ7rOI1osLzseS3P1oc9ClT711ZU7h7FvlwjDlWaNNcWo3HPeBWy9a7NrdsdZZryzO86SMBszvkiTgwLamZRVJmBVYEhI+VOY8TGpWuySp25hTJ8Q5zma2SFOw6Z7iseb7o+NO+gcx5nfEtc5OIKK9JGpqBHzQSPFcMmQevrIUmXF5MuKEaYcImOboJqu01Ahsra9NtRCjE2MTDWu2R1nSeiVqaYrdW/BOWn0jNCxe9k9RW103NlPlbWusOlpiLZoI0voNFrpFOXZTz9XJHu2Jy2U9HnK/B+h8odYGX20aJt7is9Vdv9T7XNDxfFuVtIQi4lcszvOkjBbb3wVXbzledwb/3RCFsKMPSYtY5H8OWW/ozFmIHzM7jjOtJo9ZDzYhioPb9fyMmJqlC5po4ZOXlHG0J7uGLRZaNN0PKSeqrqmxpNXOI6zHd7ZHWdJWDgH3ZjE2EQxY05hoUXmvrZ7inLGJGRoV3W+iJvxjuOMG1RzyPMeZ8OGG2Y1zVVHVYBJyI4wxSCLfDBNKH2fOWaYbyyqZBoqi3CbEOQugSwxFsKEZKqJ8X24ZnecJWE2U299xlgxx2ltdoQpe9tW5Qhvk12lSJ8dYuZO3R5mofn0QxbcZJTV06YNx2rvrjn5fczuOM7yeeO7vMUzuozlioy9m8yQDBFsVLbjTKydU/J0DbuOvcNMU5l1S6TL7nfN7jjO8mn2GAy1O0pGrHLnsHhlbJkyFt2X0RXX7I7jTJO8IqPOm53RJkVTnxzwZTJVeeNDkjA2yRhaXh+G1p5dlsEOke9+6MU6+e9sqh1qY1hEjZpd0lpJV0q6UdLXJZ2ZHl8t6QuSbkn/37uzFI7jDE6IGb8NeJuZHQYcBbxJ0mHAOcBGMzsY2Jh+dhxnprR20Em6HPhA+u8YM7tX0n7AVWZ2aN29Qzno5uiIGouhnn2szDGx5Q/NW9dlm69YcoZMKTZl3+2ysWOrMbukA4EXANcA+5rZvemp+4B9K+45AzgDYBW7tqnOcZyIBHd2SbsDnwLOMrNHpKdeHmZmkkpNBDP7EPAhSDR7P3HL6bPQY47aOoRM/uPWhN/TJpijaalrvu4uWqgof6wtm6vo8xtp08ZtZClmF4a81j9qOxmK93YhaOpN0k4kHf1iM/t0evjbqflO+v93OkvhOM7gNI7ZlajwjwDfM7Ozcsf/G/CAmb1P0jnAajN7R11ZQwfVTDl+7cPUWVPLxq+LbvmMRcgS1yoLJ1Ybh+agCzHjXwy8DviapMwOfBfwPuDjkk4HvgW8upfEjuMMyooKlx3as9tluepYC1/a7EtWR6jXtytVnui6vd7aMFYI71yz73q4rOM404bLtqHurVg1hxr77V6291f+3qb7Y8hSt49a1bVNlI3Zi2m0YlkqRU/0U89Kab0rgSavfpuEKX2sA9fsjrMkzHZ/9qkWHJQRI3Fjn0i0RdsRJmNoWdtYeyH39K1zSOp+R/lzPmZ3HMc7u+MsC7Mz46sI2cQvI0amWug2/VTlOJsiYCaGU22uU0xNdVcNAxclWKgpV0MVbsY7jjOuZl9/xCq7dsPaVlvsFqmzBoqBGdnbPU/Tm75MM1e9Xbs4iEKIpRlDA2RCLKypQ3pD6eLgHeK7mmqbZ9fsjuPMf8wee1+vPrTb6217QnLphVw7BHULYYrEXLTRl7J2zzR6lTUzxdRbjGcOtXxdszuOM224bMibbujxcR3F8V8Vfeudehxcpy2GTkvVh9J6z685V3F86KCmKuuizq8wRNu6ZnecJWGSMftQc52LModapI1mafIy1+WyH7t9QvLqF4nlxV7U30ITTc/lY3bHcbyzO86yMPtMNW3MsakdRl0Ze9VbSJuOtbV1XVkrzQRvQ1P7V7WTm/GO48w/qGYoQqyALiG8GVXTdbGz58QI8qjTpmPlFyhr6y51h+YNnPK3V6Su/cty8xXJy++a3XGclTX1ViRk6qdqN5Oya+dA17FcyL2x6TL15tTjU2+O4zSytGP2oeiT6XYOswl9cuVn9MlX38WnkadPEpQ2dY69v6AvhHEcJ5jZjdlXaphjRuwxdRufg4/Z5//7CvXJ+JjdcZxKZjdmn8O4tUifMWiRKZeMxkiSEfKddYkP6DLPPrQ/Idb+eW2oi4GA8vRpPs/uOM52eGd3nCVhtgthYm+blBHT/Bo6Q87Q5ZbVM9aWx1WOxb5bNsekj8k/9LRyVY4DN+Mdx5mfg26RCJlaauMUq8p5t6wZfao0a8wltbFkik2bTMbuoHMcZzuCs8tK2gHYBNxtZidKOgi4FNgHuA54nZn9oE3leQ22ju7hmTEpC0rpI1ObpZUhSzljtM/cNXpG1TOOvXddnie/I4Ztw6pnLN3liKf8HlvPq5arjWY/E7gp9/n9wPlmtg54EDi9RVmO44xM0Jhd0gHAR4DfBd4K/BJwP/AsM9sm6WjgXDM7rq6cLmmpisxpnL+oWXLrym/SfHW7x8SQd44htlMwZVqqC4B3AD9KP+8DPGRm29LPW4D9y26UdIakTZI2PcHWwOocx4lNo2aXdCJwgpm9UdIxwNuBXwOuTk14JK0FPmdmh9eVVafZY+ziWlUmhO/9NYfkFVP7LcqYg0xtZFiUGYe2NO0LcM95F7D1rs2lmj3EQfdi4BWSTgBWAc8ALgT2krRjqt0PAO7uJr7jOGPQaMab2TvN7AAzOxA4Bfiimb0WuBI4Kb3sNODywaR0HKc3rYJqMjM+nXp7NsnU22rgeuBXzax2UB7DQTcnYpmKTSudpgghHSvbzNDMYfgxNPlnrHPQtdrF1cyuAq5K/74dOLKXlI7jjMZsF8LEpk/+8zFyh01JiFNyDs+xSFq6zpFcDHndLrisZ/t7uKzjOMuj2edCyBt/6CwosXZ3GWK3mDZTn0NZG3Ocji3ie705jlPJpJq9LECguKdVnzxmXXaECamnDcsYCDLW2HqKGYOp/Aah9bpmdxxnPmP2GG/MRfLWzok5jkmdbrhmdxxnPpq9DzHSFE2xs+zQO7asFB9AF5b12V2zO47jnd1xloWFNuOnzhs/h0w1MR2bfcvp0x4hWy3N0QEbkgU25vbOTW3gZrzjOPMJqllkR0qs54ixZfOYDFH33H8TUzv++mSqcc3uOEvCQo/ZM4beunfO1D1H044qc5jii73oJPR7nSJLcZfQ6arciVX4mN1xnMXT7G3eyCHe+D67bw7tjc8YenwY8sxzsISaFj4NRZ21UdUuU+1v4JrdcZz5afYhEiL0ZWhN22an1yGo2+WlSP7Zq6yiuu8upoXQxV/Rpdw6LV1lXYTsWdDGMgndxcc1u+M43tkdZ1mYnRm/SMzdQTe2U23sgBPfBPLpuBnvOM58NPtUGzvW3ROa3TRPzAyrQzu45kis7LtTh7UOhYfLOo7TyGw0+xAhr1MFNrRlpWqhKVnpbep54x3HqaTVxo59OeR5j7NhQ7m27aJ5m8ImY2vzocbLMZfFttljrC4YJaSMsqCTuuNl1F079SxCm+87VoBP07VVwU4huGZ3nCVhNmP2jD7hskNp3pjhsnWacSrfg4fL1t/r4bKO4ywUk4zZQ96YbcZPQy19LCYQyO+jHXpvRp0sTdqnqxYM1W5lsoW0XbHc7PNt96Sa7OzxZ0Cydj9uTfn5kMSWbfYXrGqn/PHnkFo6XF15TYicVTxpSVH/nblmd5wlwTu74ywJQQ46SXsBFwGHAwb8O+Bm4DLgQOBO4NVm9mBdOSttIUwdVdtHTxnkEWPdfBsHY9+h19S5DaYITW5ql6YpyhgOuguBz5vZc4EjgJuAc4CNZnYwsDH97DjOTGnU7JL2BG4Anm25iyXdDBxjZvdK2g+4yswOrStrmTR7F4bW/rE11RyslUWkz/cw9EKYg4D7gQ9Lul7SRZJ2A/Y1s3vTa+4D9i27WdIZkjZJ2vQEWwOqcxxnCEI0+3rgauDFZnaNpAuBR4DfMrO9ctc9aGZ715W10rLLtgnxjLV/Wxet0FR3yFLdoa2C4jRnWV1zXN7bJrvsGJZQ3zH7FmCLmV2Tfv4k8DPAt1PznfT/78QQ1nGcYWgMqjGz+yRtlnSomd0MHAvcmP47DXhf+v/lfQQZ4q2XD4LJa4zQe5qCFMqCbKruafNcoUESoTTV3SWApi/FOrPPWQAK9Hv+sfwJVe1SdjxmYpOycm89/yi2nlddR2gE3W8BF0vaGbgdeAOJVfBxSacD3wJe3Upqx3FGZXYLYaqoG6/FeIsX53ShOZVV2Vu2aEH08biGpNJa5hRWZZR9j1D/25jjrELXvel9IYzjON7ZHWdZWBgzvo6hzdU5mnkZQ6/hH2tTyaEDfWJvDR2Dujbumj3HzXjHccZdzz4UMbKRlNFHu83ZGghhLLlDFsLE0PrFMrYL3plg3T3Ut/EQC6hcszvOkrAixuwZi65Nq8iHXrZZ7jn2Mts5TfWt1N9CGZ6DznGc7VhRmt3pRohneugdU8s0cRftHHpPXUbdGPX19fZ3tUxcszuO45p9pbBMY9SMWNZAaD2xyw2p0zW74zitmVSztxk3hSyEKZbVl5i7imTUaYnYWmkZtX1G01h6Dok/h5i1cM3uOI53dsdZFiY147tschiLNuGyIevKq+7N6Jt7bpGIlScvdMPL/PkmM71vG4/9HRV/R02/RTfjHceZRrOHbK4X4khps9VxRsw3cpu3/FjTRHXZcWMs/umyI0yequ+3LrtsHwurDyHt1Sa77FDk29I1u+M48w2qmcMUSQxiWChDM4dEDmMxxZg9xvcbmujCNbvjOPMJqsnoMy4bKvVQ05u5TT11O4VUlTHm+K9YZ5vxfZ8xbhe6pHUKoW6XoC47yg5tufkSV8dxtmNUzb7+iFV27Ya1CzeP3GXOfA4+hzE1ylxk6tLubWZ15jSbU3bONbvjOCtLs89Rk82ZWGPeLjvUdhn7hjDVnHwdxXiDvpq8DtfsjuN4Z3ecZWG2QTVzINSUKgsLbbOx4NS0yUE31HPEyhs/B8doW2LmOHAz3nGcaRfC5JmDI6WKRdr3LGSaqK6epgVJIdmFqoJtQsodeqFQG9pYG7Gct1UOzJA2BdfsjuMwozF7l83nxw6GmFrTjMFUz+htG6dc1+yO44RpdklnA78OGPA14A3AfsClwD7AdcDrzOwHdeUsmjd+CPp6XhdpZ9ll0NZj09SmvTS7pP2BtwDrzexwYAfgFOD9wPlmtg54EDi9i/CO44xD6P7sOwI/LukJYFfgXuAlwKnp+Y8A5wIfjC1gCMukQdpYA8vULqHMvU2GlK9Rs5vZ3cDvA3eRdPKHScz2h8xsW3rZFmD/svslnSFpk6RNT7A1jtSO47QmxIzfG3glcBCwBtgNOD60AjP7kJmtN7P1O7FLZ0Edx+lHo4NO0q8Ax5vZ6enn1wNHA78CPMvMtkk6GjjXzI6rK6vN1Fsbxs6QMjRtnicjRn4z6Ld9cRfafC99suMOlcVo7C2bm9qg79TbXcBRknaVJOBY4EbgSuCk9JrTgMsDynIcZyJCp97eDZwMbAOuJ5mG259k6m11euxXzax2UJ5p9lihkV1CO5vKmDtN7dRmaq8sbDkjRvaZshyDTfnq+ua9L9Y9lSOua0BX6Pdbll8f6jV7kDfezH4b+O3C4duBI0PudxxnemYTLtuHsfKaZcwhB93QGVnmOGYPIVTuOeTKbzNtGvp9e7is4zjz1+xjeWvraLuTZtk9dUsTY3rWY7HoMxwxZ3f6LqjyHWEcxxmV2Wv2WIS+Xdvs/lFXZp/kDENpzBCveZdn7ZLQIWTX3i6ZaKf2wsem6TcCnrzCcZwCoQthZkOb8VPdPHLxmqys7co8v4OAExM7LVWxrCn2ncuYo48g9v7sQz6ja3bHWRK8szvOkjCJGV9mqoQ4bJruqaPqmrp7m8JN29xb5jCqciJNaaY+me8+HcIUZcznw1/H1dsdyz4XKTNpq+6pdZCeHPwYUQhx9jX9Xoesuy2u2R1nSZh06q2Ls62NU6kNZWUUQxSrpoBi7QjTJYCoSBfLp27qrcv011iOtLp2bzOF2Kfdx9pENFTT+9Sb4zjTavahkyaULbEshrqGLIuNuQtIyBLUsbcb7jpdN/ZUWJe9BYr0WUpdVlfsqbe+uGZ3HGd5wmWdbuPvMcuDdktPxwqNncNy2CJVMrlmdxxnZWj2RU20kNFG/q5e2rbMMTS1DW3aZZEWzzT5oVyzO47jnd1xloVRzfj1R6yyazesjRL8At1WdbUpv1hPH3N7CkIDQEKmQBfF1I2RpThGfVM59dyMdxxnfg66GFlP6miTHabMGVJ2b109xXti5UUvMkXQy9TWS913FlOz1+UN9HBZx3Fmx+w0+9S0CXNs2p2jK0NPp2VBMLHCcmNq0zkGsNQxlS/Dg2ocx6lk4TT70AsM+ua4iyHT0Mt5i9TtyVa8ZqjkDENZSYtATGvGNbvjOPNJSxVyru54KE0aKiQHfBtZqnaTgacvIAnR2m2sjVBvf1ku+yJ1O78OEe+QZ+z8+sXy8wzxrF1CertYuK7ZHWdJWLgxex1T7Qga+946hk6LNPay0WLKr75j9qmsgBDGSLbhY3bHcbyzO86ysKJz0M2RkBx0dc8+x6HFXOoLYQ4yDbHwJsPNeMdx5uOgC33jdglnzdOkTecerrlImt0ZH9fsjuOMq9kl3Q88Bnx3tEr78UwWR1ZYLHkXSVZYHHl/ysx+ouzEqJ0dQNImM1s/aqUdWSRZYbHkXSRZYfHkLcPNeMdZEryzO86SMEVn/9AEdXZlkWSFxZJ3kWSFxZP3aYw+ZnccZxrcjHecJcE7u+MsCaN1dknHS7pZ0q2Szhmr3lAkrZV0paQbJX1d0pnp8dWSviDplvT/vaeWNUPSDpKul3RF+vkgSdekbXyZpJ2nljFD0l6SPinpG5JuknT0XNtW0tnpb+AfJX1M0qo5t20oo3R2STsAfwi8DDgMeI2kw8aouwXbgLeZ2WHAUcCbUhnPATaa2cHAxvTzXDgTuCn3+f3A+Wa2DngQOH0Sqcq5EPi8mT0XOIJE7tm1raT9gbcA683scGAH4BTm3bZhmNng/4CjgQ25z+8E3jlG3T1kvhz4ReBmYL/02H7AzVPLlspyAEkHeQlwBSCSCK8dy9p8Yln3BO4gdQjnjs+ubYH9gc3AapK0bVcAx821bdv8G8uMzxowY0t6bJZIOhB4AXANsK+Z3Zueug/YdyKxilwAvAP4Ufp5H+AhM9uWfp5TGx8E3A98OB12XCRpN2bYtmZ2N/D7wF3AvcDDwHXMt22DcQddAUm7A58CzjKzR/LnLHmtTz5XKelE4Dtmdt3UsgSyI/AzwAfN7AUk6yO2M9ln1LZ7A68keUGtAXYDjp9UqEiM1dnvBtbmPh+QHpsVknYi6egXm9mn08PflrRfen4/4DtTyZfjxcArJN0JXEpiyl8I7CUpyxg8pzbeAmwxs2vSz58k6fxzbNuXAneY2f1m9gTwaZL2nmvbBjNWZ/8KcHDq0dyZxOHxmZHqDkKSgP8J3GRmf5A79RngtPTv00jG8pNiZu80swPM7ECStvyimb0WuBI4Kb1sFrICmNl9wGZJh6aHjgVuZIZtS2K+HyVp1/Q3kck6y7ZtxYiOjxOAbwK3Af9xamdFiXw/R2JGfhW4If13AslYeCNwC/C3wOqpZS3IfQxwRfr3s4FrgVuBTwC7TC1fTs7nA5vS9v1LYO+5ti3wbuAbwD8CHwV2mXPbhv7zcFnHWRLcQec4S4J3dsdZEryzO86S4J3dcZYE7+yOsyR4Z3ecJcE7u+MsCf8fiSYjQBqp2JgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(cache_dict['simulation_statuses'], interpolation='nearest')\n",
    "plt.title('Cache visualization')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "# Part 2: Memoized Agent\n",
    "\n",
    "We now build our agent. We will build a simple memoized agent. At training, the agent ranks actions based on the percentage of training tasks it solves. At test time, the agent uses the first ```max_attempts_per_task``` valid actions on each task in order of their rank.\n",
    "\n",
    "As we are evaluating the performance across templates, the agent will have had no experience on any of the task templates in the test set, so the ranking of actions is the same accross templates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_memo_agent(train_tasks, cache, max_actions_to_rank=None):\n",
    "    cache_sample = cache.get_sample(train_tasks, max_actions_to_rank)\n",
    "    sim_statuses = cache_sample['simulation_statuses']\n",
    "    positive = (sim_statuses.astype('float32') > 0).sum(0)\n",
    "    negative = (sim_statuses.astype('float32') < 0).sum(0)\n",
    "    denominators = positive + negative + 1\n",
    "    action_scores = positive / denominators\n",
    "    # Return indices in order of the % tasks in the training set that the action solves.\n",
    "    ordered_actions = action_scores.argsort()[::-1]\n",
    "    return ordered_actions\n",
    "\n",
    "\n",
    "def eval_memo_agent(eval_task_ids,\n",
    "                    cache,\n",
    "                    ordered_actions,\n",
    "                    max_attempts_per_task,\n",
    "                    n=1):\n",
    "    # Evaluation of this agent is simple: record the status of the first max_attempts_per_task\n",
    "    # valid actions on each task.\n",
    "    evaluator = phyre.Evaluator(eval_task_ids)\n",
    "    for task_index, task_id in tqdm.tqdm_notebook(\n",
    "            enumerate(eval_task_ids),\n",
    "            total=len(eval_task_ids),\n",
    "            position=n,\n",
    "            desc=f'Evaluate memo agent'):\n",
    "        statuses = cache.load_simulation_states(task_id)\n",
    "        for action_index in ordered_actions:\n",
    "            status = phyre.SimulationStatus(statuses[action_index])\n",
    "            evaluator.maybe_log_attempt(task_index, status)\n",
    "            if evaluator.get_attempts_for_task(\n",
    "                    task_index) >= max_attempts_per_task:\n",
    "                break\n",
    "    return evaluator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can train and evaluate our agent, ranking all 100k actions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d121ac18a38247d9a8d0f55682c22482",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Evaluate memo agent', max=400, style=ProgressStyle(descriptio…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUCESS after 100 attempts on dev set 0.09299969491290087\n"
     ]
    }
   ],
   "source": [
    "ordered_actions = train_memo_agent(train, cache)\n",
    "evaluator = eval_memo_agent(dev, cache, ordered_actions,\n",
    "                            phyre.MAX_TEST_ATTEMPTS)\n",
    "\n",
    "print('AUCESS after 100 attempts on dev set', evaluator.get_aucess())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now explore the relationship between the number of actions that we rank and the performance of the agent.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c4c9186d5a5c4bb1ab40078f5cc45c7c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Evaluate all memo sizes', max=10, style=ProgressStyle(descrip…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "memo_sizes = np.logspace(0, 5, 10).astype(int)\n",
    "\n",
    "\n",
    "def worker(memo_size, train, dev, cache, max_attempts):\n",
    "    index = 1 + np.where(memo_sizes == memo_size)[0][0]\n",
    "    ordered_actions = train_memo_agent(train, cache, memo_size)\n",
    "    evaluator = eval_memo_agent(dev, cache, ordered_actions, max_attempts,\n",
    "                                index)\n",
    "    return memo_size, evaluator.get_aucess()\n",
    "\n",
    "\n",
    "# We use 10 threads to speed up the training process.\n",
    "pool = multiprocessing.Pool(10)\n",
    "partial_worker = functools.partial(\n",
    "    worker,\n",
    "    train=train,\n",
    "    dev=dev,\n",
    "    cache=cache,\n",
    "    max_attempts=phyre.MAX_TEST_ATTEMPTS,\n",
    ")\n",
    "# NOTE: This can take ~ 2 minutes to run.\n",
    "results = list(\n",
    "    tqdm.tqdm_notebook(\n",
    "        pool.imap(partial_worker, memo_sizes),\n",
    "        total=len(memo_sizes),\n",
    "        desc='Evaluate all memo sizes'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We graph the relationship between the number of ranked action, and AUCESS of the trained agent on the dev set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5hcVZnv8e+PJoaWW0OS4ZjmkiAYBgaGaIMgHgYETECBkIPDRQUxA94QGI8ZyaODgOMkTLwwHhklKIIMiIgxRgUjCkEHBdMhkXCLhksgHUYDJMhoE3N5zx97dahUqqt3d3pXVap/n+epJ7vW2pd3dXXq7b3X3mspIjAzMyu3Xb0DMDOzxuQEYWZmFTlBmJlZRU4QZmZWkROEmZlV5ARhZmYVbV/vAAbLyJEjY8yYMfUOw8xsm7Jw4cLnI2JUpbqmSRBjxoyhs7Oz3mGYmW1TJC3vrc6XmMzMrCInCDMzq8gJwszMKnKCMDOzipwgzMysIicIMzOryAnCzMwqKjRBSJooaamkZZIurVD/MUmPSnpI0s8k7VNSt0HS4vSaW2ScZma2pcIelJPUAlwDnACsABZImhsRj5astgjoiIg/S/oQ8G/AGamuOyIOLSo+MzOrrsgziMOBZRHxZET8BbgVOLV0hYi4JyL+nN7eD+xZYDxmZtYPRSaIduDZkvcrUllvpgB3lrzfQVKnpPslTaq0gaQL0jqdq1at2vqIzcxskz4ThKSxecq2hqT3AB3AzJLifSKiAzgbuFrS68u3i4hZEdERER2jRlUca8rMzAYozxnEdyuU3Z5juy5gr5L3e6ayzUg6HvgkcEpErO0pj4iu9O+TwHxgfI5jmpnZIOm1k1rSAcBBwK6SJpdU7QLskGPfC4D909lGF3Am2dlA6THGA9cCEyPiDyXluwF/joi1kkYCR5F1YJuZWY1Uu4tpHPBOoA04uaT8ZeD8vnYcEeslXQjMA1qA6yPiEUlXAp0RMZfsktJOwHckATwTEacAfw1cK2kj2VnOjLK7n8zMrGCKiOorSEdGxK9qFM+AdXR0hOeDsGY0Z1EXM+ctZeWabka3tTJ1wjgmja92v4dZfpIWpv7eLeR5DuIFST8D9oiIv5F0CFl/wb8MapRmDaYRvpjnLOpi2uwldK/bAEDXmm6mzV4C4CRhhcvTSX0dMA1YBxARD5H1J5g1rZ4v5q413QSvfjHPWbTFfRaFmjlv6abk0KN73QZmzlta0zhsaMqTIF4bEb8uK1tfRDBmjaJRvphXrunuV7nZYMqTIJ5PzyAEgKTTgecKjcqszhrli3l0W2u/ys0GU54E8RGyW1EPkNQFXAJ8qNCozOqsUb6Yp04YR+uwls3KWoe1MHXCuJrGYUNTnwkijaV0PDAKOCAi3hoRTxcemVkdNcoX86Tx7UyffDDtba0IaG9rZfrkg91BbTXR511Mki4GvkH2/MN1kt4IXBoRPyk6OLN66fkCrvddTD2xOCFYPeS5zfX9EfHvkiYAI4D3AjcBThDW1PzFbENdnj4IpX9PAr4ZEY+UlJmZWZPKkyAWSvoJWYKYJ2lnYGOxYZmZWb3lucQ0BTgUeDLN/DYCOK/YsMzMrN76TBARsRF4sOT9C8ALRQZlZmb1V9ic1GYD1QhjIJmZE4Q1GA9OZ9Y48kw5+iVJb6lFMGaNMgaSmeW8iwn4lKQnJH1OUsVxw80GQ6OMgWRm+YbauDEiTgIOA5YCV0n6XeGR2ZDUKGMgmVm+M4ge+wEHAPsAjxcTjg11jTIGkpnlG4vp34DTgCeAW4HPRMSaogOzoamRxkAyG+ry3MX0BHBkRDxfdDBm4DGQzBpF3ilHJ0q6DEDS3pIOLzYsMzOrtzwJ4hrgSOCs9P7lVGZmZk0szyWmN0fEGyUtAoiI1ZJeU3BcZmZWZ3nOINZJauHVOalH4dFczcyaXp4ziC8B3wP+StJngdOBTxUalZk1HI+RNfTkGc31ZkkLgePIJgqaFBGPFR6ZmTUMj5E1NPV6iUnS7j0v4A/At4BbgN+nMjMbIjxG1tBU7QxiIVm/g4C9gdVpuQ14BhhbeHRm1hA8RtbQ1OsZRESMjYh9gZ8CJ0fEyIgYAbwT+EmtAjSz+vMYWUNTnruYjoiIO3reRMSdgIf/NhtCPEbW0JQnQayU9ClJY9Lrk8DKPDuXNFHSUknLJF1aof5jkh6V9JCkn0nap6TuXEm/S69z8zfJzAbbpPHtTJ98MO1trQhob2tl+uSD3UHd5BQR1VfIOqQ/DRxN1ifxc+DKiHixj+1agN8CJwArgAXAWRHxaMk6xwIPRMSfJX0IOCYizkjH7AQ60jEXAm+KiNW9Ha+joyM6Ozv7aq+ZmZWQtDAiKs7zk+c21xeBiwdw3MOBZRHxZAriVuBUYFOCiIh7Sta/H3hPWp4A3NWThCTdBUwku5PKzMxqoD/zQfRXO/BsyfsVqaw3U4A7B7itmZkNsjxPUhdO0nvILif9XT+3uwC4AGDvvfcuIDIzs6GryDOILmCvkvd7prLNSDoe+CRwSkSs7c+2ETErIjoiomPUqFGDFriZmeWbUW4UcD4wpnT9iHh/H5suAPaXNJbsy/1M4OyyfY8HrgUmRsQfSqrmAf8qabf0/u3AtL5iNTOzwZPnEtP3gV+QPTC3oY91N4mI9ZIuJPuybwGuj4hHJF0JdEbEXGAmsBPwHUkAz0TEKRHxoqTPkCUZyHHXlJmZDa48t7kujohDaxTPgPk2VzOz/qt2m2uePogfSjppkGMyM7MGlydBXEyWJF6R9HJ6/bHowMzMrL7yPCi3cy0CMTOzxpLrOQhJp5ANtQEwPyJ+WFxIZmbWCPq8xCRpBtllpkfT62JJ04sOzMzM6ivPGcRJwKERsRFA0o3AIvxcgplZU8v7JHVbyfKuRQRiZmaNJc8ZxHRgkaR7yKYcPRrYYm4HMzNrLnnuYvqWpPnAYanoExHx34VGZWZmdZfrLqaIeA6YW3AsZmbWQIoczdXMzLZhThBmZlZRnucgXi9peFo+RtJFktr62s7MzLZtec4gvgtskLQfMItsIp9bCo3KzMzqLk+C2BgR64HTgP8XEVOB1xUblpmZ1VueBLFO0lnAuUDPGEzDigvJzMwaQZ4EcR5wJPDZiHgqTSF6U7FhmZlZveV5DuKEiLio501KEq8UGJOZmTWAPGcQ51Yoe98gx2FmZg2m1zOI1O9wNjBWUulT1DsDLxYdmJmZ1Ve1S0y/BJ4DRgKfLyl/GXioyKDMzKz+ek0QEbEcWA4cKel/AYcDASxNt72amVkTy/Mk9RTg18Bk4HTgfknvLzowMzOrrzx3Mf0TMD4iXgCQNILs8tP1RQZmZmb1lecuphfI+h16vJzKzMysieU5g1gGPCDp+2R9EKcCD0n6GEBEfKHA+MzMrE7yJIgn0qvH99O/Ow9+OGZm1ijyTDl6RS0CMTOzxtJngpDUAXwS2Kd0/Yg4pMC4zMyszvJcYroZmAosATYWG46ZmTWKPAliVUTM7Xs1MzNrJnkSxKclfQ34GbC2pzAiZve1oaSJwL8DLcDXImJGWf3RwNXAIcCZEXF7Sd0GsrMWgGci4pQcsZpZE5uzqIuZ85ayck03o9tamTphHJPGt9c7rKaVJ0GcBxxANklQzyWmAKomCEktwDXACcAKYIGkuRHxaMlqz5CNDPvxCrvojohDc8RnZkPAnEVdTJu9hO51GwDoWtPNtNnZ35BOEsXIkyAOi4hxA9j34cCyiHgSQNKtZM9QbEoQEfF0qnPfhplVNXPe0k3JoUf3ug3MnLfUCaIgeZ6k/qWkAwew73bg2ZL3K1JZXjtI6pR0v6RJlVaQdEFap3PVqlUDCNHMthUr13T3q9y2Xp4ziCOAxZKeIuuDEBA1uM11n4jokrQvcLekJRFR+sAeETELmAXQ0dERBcdjZnU0uq2VrgrJYHRbax2iGRryJIiJA9x3F7BXyfs9U1kuEdGV/n1S0nxgPJs/0W1mQ8jUCeM264MAaB3WwtQJA7kCbnn0eYkpzQuxF/C2tPznPNsBC4D9JY2V9BrgTCDX7bKSdpM0PC2PBI6ipO/CzIaeSePbmT75YNrbWhHQ3tbK9MkHu/+hQHmepP400AGMA75BdjfTf5J9afcqItZLuhCYR3ab6/UR8YikK4HOiJgr6TDge8BuwMmSroiIg4C/Bq5NndfbATPK7n4ysyFo0vh2J4QaynOJ6TSyyzsPAkTESkm5BuqLiDuAO8rKLitZXkB26al8u18CB+c5hpmZFSPPpaK/RESQPfuApB2LDcnMzBpBnjOI2yRdC7RJOh94P/C1YsOyWvMTqmZWLs9w35+TdALwR7J+iMsi4q7CI7Oa8ROqZlZJn5eYJF0VEXdFxNSI+HhE3CXpqloEZ7VR7QlVMxu68vRBnFCh7MTBDsTqx0+omlklvV5ikvQh4MPA6yU9VFK1M/DLogOz2vETqmZWSbUziFuAk8nmoD655PWmiHh3DWKzGpk6YRytw1o2K/MTqmbW6xlERLwEvCRpfXqCehNJN0XEewuPzmqipyPadzGZWak8t7keVPpG0vbAm4oJx+rFT6iaWbleLzFJmibpZeAQSX9Mr5eB35NddjIzsybWa4KIiOkRsTMwMyJ2Sa+dI2JEREyrYYxmZlYHeR6UmyZpN2B/YIeS8p8XGZiZmdVXntFc/wG4mGxQvcVkEwj9CnhbsaGZmTWmoTI0TZ4H5S4GDgOWR8SxZCO7rik0KjOzBtUzNE3Xmm6CV4emmbMo93xo24w8CeKViHgFQNLwiHicbEwmM7MhZygNTZPnNtcVktqAOcBdklYDy/vYxsysKQ2loWnydFKflhYvl3QPsCvw40KjMjNrUENpaJo8l5g2iYh7I2JuRPylqIDMzBrZUBqaJs8lJjMzS4bS0DROEGZm/TRUhqbJ8xzEHkDPT6IrIn5fbEhmZtYIqs0HcSjwVbJO6Z4bfPeUtAb4cEQ8WIP4zMysTqqdQdwAfCAiHigtlHQE8A3gbwuMy8zM6qzaXUw7licHgIi4H9ixuJDMzKwRVDuDuFPSj4BvAs+msr2Ac/BzEGZmTa/ajHIXSToROJWSTmrgmoi4oxbBmZlZ/VS9iyki7gTurFEsZmbWQKrNKLerpBmSHpP0oqQX0vKMNDaTmZk1sWqd1LcBq4FjI2L3iBgBHEs21PdttQjOzMzqp1qCGBMRV0XEf/cURMR/R8QMYJ/iQzMzs3qqliCWS/qn9CQ1kD1VLekTvHpXU1WSJkpaKmmZpEsr1B8t6UFJ6yWdXlZ3rqTfpde5eRtkZmaDo1qCOAMYAdyb+iBeBOYDuwN/39eOJbUA1wAnAgcCZ0k6sGy1Z4D3AbeUbbs78GngzcDhwKfTvNhmZlYj1W5zXQ18Ir0G4nBgWUQ8CSDpVrJbZh8tOcbTqW5j2bYTgLsi4sVUfxcwEfjWAGMxM7N+6td8ED0knZdjtXY2vxS1glefpyhyWzMzGwQDShDAFYMaxQBJukBSp6TOVatW1TscM7OmUm0014d6qwL26KWuVBfZ0Bw99uTVUWHzbHtM2bbzy1eKiFnALICOjo7IuW8zM8uh2pPUe5D1BawuKxfwyxz7XgDsL2ks2Rf+mcDZOeOaB/xrScf024FpObc1MxsS5izqKnRmu2oJ4ofAThGxuLxC0vy+dhwR6yVdSPZl3wJcHxGPSLoS6IyIuZIOA74H7AacLOmKiDgoIl6U9BmyJANwZU+HtZmZZclh2uwldK/bAEDXmm6mzV4CMGhJQhHNcWWmo6MjOjs76x2GmVlNHDXjbrrWdG9R3t7Wyn2Xvi33fiQtjIiOSnUD7aQ2M7M6WlkhOVQrHwgnCDOzbdDottZ+lQ+EE4SZ2TZo6oRxtA5r2aysdVgLUyeMG7RjVE0Qklok3TNoRzMzs0ExaXw70ycfTHtbKyLre5g++eCa3cVERGyQtFHSrhHx0qAd1czMttqk8e2DmhDKVU0Qyf8AS9J4SH/qKYyIiwqLyszM6i5PgpidXmZmNoT0mSAi4kZJrcDeEbG0BjGZmVkD6PMuJkknA4uBH6f3h0qaW3RgZmZWX3luc72cbG6HNQBp6I19C4zJzMwaQJ4Esa7CHUzlE/yYmVmTydNJ/Yiks4EWSfsDF5FvNFczM9uG5TmD+ChwELCWbO7ol4CLiwzKzMzqL88ZxDsi4pPAJ3sKJL0L+E5hUZmZWd3lOYOoNFGPJ+8xM2ty1aYcPRE4CWiX9KWSql2A9UUHZmZm9VXtElMX0AmcAiwsKX8Z+McigzIzs/qrliC+GBHHSfrbiLixZhGZmVlDqJYgXifpLcCJkm4CVFoZEQ8WGpmZmdVVtQRxGfDPwJ7AF8rqAsg/6amZmW1zek0QEXE7cLukf46Iz9QwJjMzawB5RnP9jKTdgP2BHUrKf15kYGZmVl99JghJ/0D25PSeZKO6HgH8Cl9iMjNrankelLsYOAxYHhHHAuNJI7uamVnzypMgXomIVwAkDY+Ix4FxxYZlZmb1lmcsphWS2oA5wF2SVgPLiw3LzMzqLU8n9Wlp8XJJ9wC7kmaXMzOz5pXnDGKTiLi3qEDMzKyx5OmDMDOzIcgJwszMKsqdICSNkHSapDcVGZCZmTWGXhOEpB9K+pu0/DrgYeD9wE2SLsmzc0kTJS2VtEzSpRXqh0v6dqp/QNKYVD5GUrekxen11QG0rWHNWdTFUTPuZuylP+KoGXczZ1FXvUMyM9tCtU7qsRHxcFo+D7grIs6RtDNwH3B1tR1LagGuAU4AVgALJM2NiEdLVpsCrI6I/SSdCVwFnJHqnoiIQ/vfpMY2Z1EX02YvoXvdBgC61nQzbfYSACaNb69naGZmm6l2iWldyfJxwB0AEfEysDHHvg8HlkXEkxHxF+BW4NSydU4FeuaauB04TpJoYjPnLd2UHHp0r9vAzHlL6xSRmVll1RLEs5I+Kuk04I2kZx8ktQLDcuy7HXi25P2KVFZxnYhYD7wEjEh1YyUtknSvpP9d6QCSLpDUKalz1apVOUKqv5VruvtVbmZWL9USxBTgIOB9wBkR0TP+0hHANwqO6zlg74gYD3wMuEXSLuUrRcSsiOiIiI5Ro0YVHNLgGN3W2q9yM7N66TVBRMQfIuKDEXFqRPwEIA37PT8iPpdj313AXiXv90xlFdeRtD3ZU9ovRMTaiHghxbEQeAJ4Q842NbSpE8bROqxls7LWYS1MneDhrcyssVS7i+kySQek5eFpmI0ngN9LOj7HvhcA+0saK+k1wJnA3LJ15gLnpuXTgbsjIiSNSp3cSNqXbC6KJ/vTsEY1aXw70ycfTHtbKwLa21qZPvlgd1CbWcOpdhfTGUDPTHI9X+KjyP6SvxH4abUdR8R6SRcC84AW4PqIeETSlUBnRMwFvk522+wy4EWyJAJwNHClpHVkHeIfjIgX+926BjVpfLsTgpk1vGoJ4i8REWl5AnBrRGwAHkuXg/oUEXeQ7n4qKbusZPkV4F0Vtvsu8N08xzAzs2JU66ReK+lvJI0CjgV+UlL32mLDMjOzeqt2JnAJ2bMJo4AvRsRTAJJOAhbVIDYzM6ujXhNERNwPHFChfIvLRmZm1nyq3cV0dcnyxWV1NxQYk5mZNYBqfRBHlyyfW1Z3SAGxmJlZA6mWINTLspmZDQHVOqm3S09Ob1ey3JMoWnrfzMzMmkG1BLErsJBXk8KDJXWx5epmZtZMqt3FNKaGcZiZWYOpdhfTBEmnVyj/P5JOKDYsMzOrt2qd1JcB91Yovxe4sphwzMysUVRLEMMjYotZeCLieWDH4kIyM7NGUC1B7FJpUD5JwwDPbmNm1uSqJYjZwHWSNp0tSNoJ+GqqMzOzJlYtQXwK+D2wXNJCSQ8CTwGrUp2ZmTWxare5rgculXQFsF8qXhYR3TWJzMzM6qrXBCFpcllRAG2SFkfEy8WGZWZm9VbtSeqTK5TtDhwiaUpE3F1QTDU1Z1EXM+ctZeWabka3tTJ1wjhPB2pmRvVLTOdVKpe0D3Ab8OaigqqVOYu6mDZ7Cd3rNgDQtaababOXADhJmNmQV62TuqKIWA4MKyCWmps5b+mm5NCje90GZs5bWqeIzMwaR78ThKRxwNoCYqm5lWsq97f3Vm5mNpRU66T+AVuO2ro78DrgvUUGVSuj21rpqpAMRrf5OUAzs2qd1J8rex/AC8DvIuIvxYVUO1MnjNusDwKgdVgLUyeMq2NUZmaNoVondaWB+pD0VklnRcRHigurNno6on0Xk5nZlqqdQWwiaTxwNvAusqepm2aojUnj250QzMwqqNYH8QbgrPR6Hvg2oIg4tkaxmZlZHVU7g3gc+AXwzohYBiDpH2sSlZmZ1V2121wnA88B90i6TtJxvDo/tZmZNbleE0REzImIM4EDgHuAS4C/kvQVSW+vVYBmZlYffT4oFxF/iohbIuJkYE9gEfCJwiMzM7O66teT1BGxOiJmRcRxRQVkZmaNod9DbfSHpImSlkpaJunSCvXDJX071T8gaUxJ3bRUvlTShCLjNDOzLRWWICS1ANcAJwIHAmdJOrBstSnA6ojYD/gicFXa9kDgTOAgYCLwH2l/ZmZWI0WeQRxONgPdk2lojluBU8vWORW4MS3fDhwnSan81ohYGxFPAcvS/szMrEaKTBDtwLMl71eksorrpClOXwJG5NwWSRdI6pTUuWrVqkEM3czMCu2DKFrqMO+IiI5Ro0bVOxwzs6ZSZILoAvYqeb9nKqu4jqTtgV3JRozNs62ZmRWoyASxANhf0lhJryHrdJ5bts5c4Ny0fDpwd0REKj8z3eU0Ftgf+HWBsZqZWZlco7kORESsl3QhMA9oAa6PiEckXQl0RsRc4OvATZKWAS+SJRHSercBjwLrgY9ExIaKBzIzs0Io+4N929fR0RGdnZ31DsPMbJsiaWFEdFSq26Y7qc3MrDhNcwYhaRWwnKyj+6WSqmrvS5dHks17sbXKj7c161aqz1O2rba5tzq3uXJZb20sfT9Y7e0tpoGsV0Sbi/iMe4tpIOs1cpv3iYjKt4FGRFO9gFl535ctdxZx/K1Zt1J9nrJttc291bnN/Wtj6fvBam9/2jyQ3+utbXMRn/FQbXPpqxkvMf2gH+/L64o4/tasW6k+T9m22ube6tzmymXV2ljPNg/k97q38rxtLqK9/dlvM7V5k6a5xLS1JHVGLx01zcptbn5Drb3gNg+mZjyDGKhZ9Q6gDtzm5jfU2gtu86DxGYSZmVXkMwgzM6vICcLMzCpygjAzs4qcICqQtKOkGyVdJ+nd9Y6nFiTtK+nrkm6vdyy1ImlS+oy/Lent9Y6nFiT9taSvSrpd0ofqHU+tpP/TnZLeWe9YakHSMZJ+kT7rYwa6nyGTICRdL+kPkh4uK680b/Zk4PaIOB84pebBDpL+tDmymf+m1CfSwdPPNs9Jn/EHgTPqEe9g6GebH4uIDwJ/DxxVj3gHQz//PwN8ArittlEOrn62OYD/AXYgm3BtYIp4+q4RX8DRwBuBh0vKWoAngH2B1wC/IZs/expwaFrnlnrHXos2l9TfXu+469DmzwNvrHfstWoz2R89dwJn1zv2WrQZOIFspOj3Ae+sd+w1avN2qX4P4OaBHnPInEFExM/JhhQv1du82SvIJimCbfgsq59tbgr9abMyVwF3RsSDtY51sPT3c46IuRFxIrDNXj7tZ5uPAY4AzgbOl7RN/p/uT5sjYmOqXw0MH+gxC5sPYhtRae7rNwNfAr4s6R3U4HH2GqvYZkkjgM8C4yVNi4jpdYmuGL19zh8Fjgd2lbRfRHy1HsEVpLfP+RiyS6jDgTvqEFeRKrY5Ii4EkPQ+4PmSL89m0NvnPBmYALQBXx7ozod6gqgoIv4EnFfvOGopIl4guxY/ZETEl8j+GBgyImI+ML/OYdRFRNxQ7xhqJSJmA7O3dj/b5KnWIBqKc1+7zW5zs3KbB7nNQz1B5Jk3u9m4zW5zs3KbB7nNQyZBSPoW8CtgnKQVkqZExHqgZ97sx4DbIuKResY5mNxmtxm32W3emmOmW6HMzMw2M2TOIMzMrH+cIMzMrCInCDMzq8gJwszMKnKCMDOzipwgzMysIicIA0BSSPp8yfuPS7p8kPZ9g6TTB2NffRznXZIek3TPVu5nkqQDS95fKen4rY+wGJIul/Txrdj+fyqUjSkfVrrRSLpE0mvrHUczc4KwHmuByZJG1juQUpL6M17YFOD8iDh2Kw87iWzIZAAi4rKI+OlW7rNPaXRZ/5/M7xLACaJA/mW0HuuBWcA/lleUnwH0/MWZZq26V9L3JT0paYakd0v6taQlkl5fspvjlc3o9VulWb0ktUiaKWmBpIckfaBkv7+QNBd4tEI8Z6X9P5yG60bSZcBbga9Lmlm2/k6SfibpwbTdqSV156Rj/0bSTZLeQjZfwkxJiyW9vrT9ko6TtCjt53pJw1P505KuKDnGAan879J+Fqftdi6LbYyyyV6+CTwM7CXpK+ln9YikK0rWrXiMsv2dL+lOSa0p9h9LWph+nj0xjZX0q7SPfynfR4ntJd2czspul/RaSW+TNKfkeCdI+l6FOC5Ln+vDkmZJUio/LP28F6fP/uEcvwvz0/EfT/FI0kXAaOAebeUZo1VR70kw/GqMF9nsU7sATwO7Ah8HLk91NwCnl66b/j0GWAO8jmz46C7gilR3MXB1yfY/JvuDZH+yIYl3AC4APpXWGQ50AmPTfv8EjK0Q52jgGWAU2WjEdwOTUt18oKPCNtsDu6TlkcAyQMBBwG+Bkalu917aewNweor5WeANqfybwCVp+Wngo2n5w8DX0vIPgKPS8k7A9mWxjQE2AkeUlPXE0ZLadEgfx7g8fV4XAt8HhqfynwH7p+U3A3en5bnAOWn5Iz2fZ4W4oiT269MxBDwOjErltwAnV9h+95Llm3rWIUuCR6blGaTJb/r4XXiJbBC67ciGmnhryc9jZL3/7zTzy2cQtklE/JHsS++ifmy2ICKei4i1ZDNb/SSVLyH7kulxW0RsjIjfAU8CBwBvB86RtBh4ABhBlkAAfh0RT1U43mHA/IhYFdk4NDeTzbRVjYB/lfQQ8FOyMfT3AN4GfCcingeIiPLJWMqNA56KiN+m9zeWHbtneOWFvNr2+4AvpL9421LM5fZbksMAAAMtSURBVJZHxP0l7/9e0oPAIrIkdmBJXaVjAJwDnEiW2NZK2gl4C/Cd9PO9liyRQzbV6LfS8k1V2vtsRNyXlv+T7Is50jbvkdQGHEk2O125YyU9IGkJ2c/5oLT+zhHxq7TOLSXr9/W7sCKyeRwWl7XbCuT5IKzc1cCDwDdKytaTLkcqu0b+mpK6tSXLG0veb2Tz36/yQb+C7Iv7oxExr7RC2aQ2fxpY+BW9m+yM400RsU7S02RnA4Otp+0bSG2PiBmSfgScBNwnaUJEPF623aa2ShpL9pf6YRGxWtINZbFucYxkCXAo2V/aT5F9Xmsi4tBeYs0zCFulzwyy340fAK+QJdjNkp6kHYD/IDube1bZzQ59/byr/S6U/o6Vt9sK5DMI20z6K/o2sg7fHk8Db0rLpwDDBrDrd0naTlm/xL7AUrIRKD8kaRiApDdI2rGP/fwa+DtJIyW1AGcB9/axza7AH1JyOBbYJ5XfneIakY6/eyp/Gdh5y92wFBgjab/0/r19HVvS6yNiSURcRTY08xb9BmV2IUsYL0nag+ysII9FwAeAuZJGp7PBpyS9K8UhSX+b1r2PbFhoqD7t6N6SjkzLZwP/BRARK4GVwKfY/A+JHj3J4Pl0JnN62m4N8LKkN6f6M0u2GcjvQm+fkw0SJwir5PNk1+p7XEf2pfwbsksKA/nr/hmyL/c7gQ9GxCvA18g6oR9MnZXX0sdfhxHxHHApcA/ZBO0LI+L7fRz7ZqAjXe44h+waOpENi/xZ4N7Uti+k9W8FpqZO5U0d7Snm88gu2ywhO0vqa5rSS1JH7UPAOipfjilt32/IvuwfJ7sEc1+19cu2/S+ys48fKbsb7d3AlNS2R3h17vGLgY+kNrRX2eXStN5jwG7AV0rqbia7BPVYhTjWkP3OPEz2xb+gpHoKcF26lLQjWf8CDOB3geymih+7k7o4Hu7bzPpN0peBRRHx9X5ut1NE9NwFdynwuoi4uIgYbev5Wp6Z9YukhWRnkf93AJu/Q9I0su+e5cD7BjE0G2Q+gzAzs4rcB2FmZhU5QZiZWUVOEGZmVpEThJmZVeQEYWZmFTlBmJlZRf8fZXVfwOcKp2wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sizes, aucess = zip(*results)\n",
    "plt.scatter(sizes, aucess)\n",
    "plt.xscale('log')\n",
    "plt.xlabel('Number of actions ranked by agent')\n",
    "plt.ylabel('AUCESS after 100 attempts on dev set')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We note that the AUCESS plateaus, and eventually decrases after ranking a number of actions. This is due to a lack of diversity in the solutions required to solve some subsets of tasks. Some actions solve a large number of (easy) tasks, but the same subset of tasks. By ranking too many actions from the cache we risk having all the top performing actions (the 100 actions the memo agent chooses) all solve the same subset of tasks in the training set."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,py:light",
   "main_language": "python",
   "notebook_metadata_filter": "jupytext,-kernelspec"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}