lab04.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions and Visualizations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welcome to lab 4! This week, we'll learn about functions and the table method `apply` from [Section 8.1](https://www.inferentialthinking.com/chapters/08/1/applying-a-function-to-a-column.html).  We'll also learn about visualization from [Chapter 7](https://www.inferentialthinking.com/chapters/07/visualization.html).\n",
    "\n",
    "First, set up the tests and imports by running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=====================================================================\n",
      "Assignment: Functions and Visualizations\n",
      "OK, version v1.13.11\n",
      "=====================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from datascience import *\n",
    "\n",
    "# These lines set up graphing capabilities.\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "import warnings\n",
    "warnings.simplefilter('ignore', FutureWarning)\n",
    "\n",
    "from ipywidgets import interact, interactive, fixed, interact_manual\n",
    "import ipywidgets as widgets\n",
    "\n",
    "from client.api.notebook import Notebook\n",
    "ok = Notebook('lab04.ok')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Functions and CEO Incomes\n",
    "\n",
    "Let's start with a real data analysis task.  We'll look at the 2015 compensation of CEOs at the 100 largest companies in California.  The data were compiled for a Los Angeles Times analysis [here](http://spreadsheets.latimes.com/california-ceo-compensation/), and ultimately came from [filings](https://www.sec.gov/answers/proxyhtf.htm) mandated by the SEC from all publicly-traded companies.  Two companies have two CEOs, so there are 102 CEOs in the dataset.\n",
    "\n",
    "We've copied the data in raw form from the LA Times page into a file called `raw_compensation.csv`.  (The page notes that all dollar amounts are in millions of dollars.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>Rank</th> <th>Name</th> <th>Company (Headquarters)</th> <th>Total Pay</th> <th>% Change</th> <th>Cash Pay</th> <th>Equity Pay</th> <th>Other Pay</th> <th>Ratio of CEO pay to average industry worker pay</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>1   </td> <td>Mark V. Hurd*     </td> <td>Oracle (Redwood City)         </td> <td>$53.25   </td> <td>(No previous year)</td> <td>$0.95   </td> <td>$52.27    </td> <td>$0.02    </td> <td>362                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>2   </td> <td>Safra A. Catz*    </td> <td>Oracle (Redwood City)         </td> <td>$53.24   </td> <td>(No previous year)</td> <td>$0.95   </td> <td>$52.27    </td> <td>$0.02    </td> <td>362                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>3   </td> <td>Robert A. Iger    </td> <td>Walt Disney (Burbank)         </td> <td>$44.91   </td> <td>-3%               </td> <td>$24.89  </td> <td>$17.28    </td> <td>$2.74    </td> <td>477                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>4   </td> <td>Marissa A. Mayer  </td> <td>Yahoo! (Sunnyvale)            </td> <td>$35.98   </td> <td>-15%              </td> <td>$1.00   </td> <td>$34.43    </td> <td>$0.55    </td> <td>342                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>5   </td> <td>Marc Benioff      </td> <td>salesforce.com (San Francisco)</td> <td>$33.36   </td> <td>-16%              </td> <td>$4.65   </td> <td>$27.26    </td> <td>$1.45    </td> <td>338                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>6   </td> <td>John H. Hammergren</td> <td>McKesson (San Francisco)      </td> <td>$24.84   </td> <td>-4%               </td> <td>$12.10  </td> <td>$12.37    </td> <td>$0.37    </td> <td>222                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>7   </td> <td>John S. Watson    </td> <td>Chevron (San Ramon)           </td> <td>$22.04   </td> <td>-15%              </td> <td>$4.31   </td> <td>$14.68    </td> <td>$3.05    </td> <td>183                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>8   </td> <td>Jeffrey Weiner    </td> <td>LinkedIn (Mountain View)      </td> <td>$19.86   </td> <td>27%               </td> <td>$2.47   </td> <td>$17.26    </td> <td>$0.13    </td> <td>182                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>9   </td> <td>John T. Chambers**</td> <td>Cisco Systems (San Jose)      </td> <td>$19.62   </td> <td>19%               </td> <td>$5.10   </td> <td>$14.51    </td> <td>$0.01    </td> <td>170                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>10  </td> <td>John G. Stumpf    </td> <td>Wells Fargo  (San Francisco)  </td> <td>$19.32   </td> <td>-10%              </td> <td>$6.80   </td> <td>$12.50    </td> <td>$0.02    </td> <td>256                                            </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>\n",
       "<p>... (92 rows omitted)</p>"
      ],
      "text/plain": [
       "Rank | Name               | Company (Headquarters)         | Total Pay | % Change           | Cash Pay | Equity Pay | Other Pay | Ratio of CEO pay to average industry worker pay\n",
       "1    | Mark V. Hurd*      | Oracle (Redwood City)          | $53.25    | (No previous year) | $0.95    | $52.27     | $0.02     | 362\n",
       "2    | Safra A. Catz*     | Oracle (Redwood City)          | $53.24    | (No previous year) | $0.95    | $52.27     | $0.02     | 362\n",
       "3    | Robert A. Iger     | Walt Disney (Burbank)          | $44.91    | -3%                | $24.89   | $17.28     | $2.74     | 477\n",
       "4    | Marissa A. Mayer   | Yahoo! (Sunnyvale)             | $35.98    | -15%               | $1.00    | $34.43     | $0.55     | 342\n",
       "5    | Marc Benioff       | salesforce.com (San Francisco) | $33.36    | -16%               | $4.65    | $27.26     | $1.45     | 338\n",
       "6    | John H. Hammergren | McKesson (San Francisco)       | $24.84    | -4%                | $12.10   | $12.37     | $0.37     | 222\n",
       "7    | John S. Watson     | Chevron (San Ramon)            | $22.04    | -15%               | $4.31    | $14.68     | $3.05     | 183\n",
       "8    | Jeffrey Weiner     | LinkedIn (Mountain View)       | $19.86    | 27%                | $2.47    | $17.26     | $0.13     | 182\n",
       "9    | John T. Chambers** | Cisco Systems (San Jose)       | $19.62    | 19%                | $5.10    | $14.51     | $0.01     | 170\n",
       "10   | John G. Stumpf     | Wells Fargo  (San Francisco)   | $19.32    | -10%               | $6.80    | $12.50     | $0.02     | 256\n",
       "... (92 rows omitted)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_compensation = Table.read_table('raw_compensation.csv')\n",
    "raw_compensation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.1.** <br/> We want to compute the average of the CEOs' pay. Try running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "cannot perform reduce with flexible type",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-10-f97fab5a8083>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0maverage\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mraw_compensation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Total Pay\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/numpy/lib/function_base.py\u001b[0m in \u001b[0;36maverage\u001b[0;34m(a, axis, weights, returned)\u001b[0m\n\u001b[1;32m   1126\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1127\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mweights\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1128\u001b[0;31m         \u001b[0mavg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1129\u001b[0m         \u001b[0mscl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mavg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mavg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1130\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/numpy/core/_methods.py\u001b[0m in \u001b[0;36m_mean\u001b[0;34m(a, axis, dtype, out, keepdims)\u001b[0m\n\u001b[1;32m     68\u001b[0m             \u001b[0mis_float16_result\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     69\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 70\u001b[0;31m     \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mumr_sum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkeepdims\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     71\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mret\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmu\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndarray\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     72\u001b[0m         ret = um.true_divide(\n",
      "\u001b[0;31mTypeError\u001b[0m: cannot perform reduce with flexible type"
     ]
    }
   ],
   "source": [
    "np.average(raw_compensation.column(\"Total Pay\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see an error. Let's examine why this error occured by looking at the values in the \"Total Pay\" column. Use the `type` function and set `total_pay_type` to the type of the first value in the \"Total Pay\" column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "total_pay_type =type('Total Pay')\n",
    "total_pay_type\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q1_1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.2.** <br/>You should have found that the values in \"Total Pay\" column are strings (text). It doesn't make sense to take the average of the text values, so we need to convert them to numbers if we want to do this. Extract the first value in the \"Total Pay\" column.  It's Mark Hurd's pay in 2015, in *millions* of dollars.  Call it `mark_hurd_pay_string`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'$53.25 '"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mark = raw_compensation.column(\"Total Pay\")\n",
    "mark_hurd_pay_string =  mark[0]\n",
    "mark_hurd_pay_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q1_2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.3.** <br/>Convert `mark_hurd_pay_string` to a number of *dollars*.  The string method `strip` will be useful for removing the dollar sign; it removes a specified character from the start or end of a string.  For example, the value of `\"100%\".strip(\"%\")` is the string `\"100\"`.  You'll also need the function `float`, which converts a string that looks like a number to an actual number.  Last, remember that the answer should be in dollars, not millions of dollars."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "53250000.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mark_hurd_pay = float(mark_hurd_pay_string.strip(\"$\"))*1000000\n",
    "mark_hurd_pay"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q1_3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute the average pay, we need to do this for every CEO.  But that looks like it would involve copying this code 102 times.\n",
    "\n",
    "This is where functions come in.  First, we'll define a new function, giving a name to the expression that converts \"total pay\" strings to numeric values.  Later in this lab we'll see the payoff: we can call that function on every pay string in the dataset at once.\n",
    "\n",
    "**Question 1.4.** <br/>Copy the expression you used to compute `mark_hurd_pay` as the `return` expression of the function below, but replace the specific `mark_hurd_pay_string` with the generic `pay_string` name specified in the first line of the `def` statement.\n",
    "\n",
    "*Hint*: When dealing with functions, you should generally not be referencing any variable outside of the function. Usually, you want to be working with the arguments that are passed into it, such as `pay_string` for this function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "for_assignment_type": "student"
   },
   "outputs": [],
   "source": [
    "def convert_pay_string_to_number(pay_string):\n",
    "    pay = float(pay_string.strip(\"$\"))*1000000\n",
    "    return pay"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q1_4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running that cell doesn't convert any particular pay string. Instead, it creates a function called `convert_pay_string_to_number` that can convert any string with the right format to a number representing millions of dollars.\n",
    "\n",
    "We can call our function just like we call the built-in functions we've seen. It takes one argument, a string, and it returns a number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42000000.0"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "convert_pay_string_to_number('$42')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "53250000.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "convert_pay_string_to_number(mark_hurd_pay_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "53240000.0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can also compute Safra Catz's pay in the same way:\n",
    "convert_pay_string_to_number(raw_compensation.where(\"Name\", are.containing(\"Safra\")).column(\"Total Pay\").item(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, what have we gained by defining the `convert_pay_string_to_number` function? \n",
    "Well, without it, we'd have to copy that `10**6 * float(pay_string.strip(\"$\"))` stuff each time we wanted to convert a pay string.  Now we just call a function whose name says exactly what it's doing.\n",
    "\n",
    "Soon, we'll see how to apply this function to every pay string in a single expression. First, let's take a brief detour and introduce `interact`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using `interact`\n",
    "\n",
    "We've included a nifty function called `interact` that allows you to\n",
    "call a function with different arguments.\n",
    "\n",
    "To use it, call `interact` with the function you want to interact with as the\n",
    "first argument, then specify a default value for each argument of the original\n",
    "function like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f16b46dcc6e943a5820e44600375deda",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='$42', description='pay_string'), Output()), _dom_classes=('widget-interact',…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = interact(convert_pay_string_to_number, pay_string='$42')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now change the value in the textbox to automatically call\n",
    "`convert_pay_string_to_number` with the argument you enter in the `pay_string`\n",
    "textbox. For example, entering in `'$49'` in the textbox will display the result of\n",
    "running `convert_pay_string_to_number('$49')`. Neat!\n",
    "\n",
    "Note that we'll never ask you to write the `interact` function calls yourself as\n",
    "part of a question. However, we'll include it here and there where it's helpful\n",
    "and you'll probably find it useful to use yourself.\n",
    "\n",
    "Now, let's continue on and write more functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Defining functions\n",
    "\n",
    "Let's write a very simple function that converts a proportion to a percentage by multiplying it by 100.  For example, the value of `to_percentage(.5)` should be the number 50.  (No percent sign.)\n",
    "\n",
    "A function definition has a few parts.\n",
    "\n",
    "##### `def`\n",
    "It always starts with `def` (short for **def**ine):\n",
    "\n",
    "    def\n",
    "\n",
    "##### Name\n",
    "Next comes the name of the function.  Let's call our function `to_percentage`.\n",
    "    \n",
    "    def to_percentage\n",
    "\n",
    "##### Signature\n",
    "Next comes something called the *signature* of the function.  This tells Python how many arguments your function should have, and what names you'll use to refer to those arguments in the function's code.  `to_percentage` should take one argument, and we'll call that argument `proportion` since it should be a proportion.\n",
    "\n",
    "    def to_percentage(proportion)\n",
    "\n",
    "We put a colon after the signature to tell Python it's over.\n",
    "\n",
    "    def to_percentage(proportion):\n",
    "\n",
    "##### Documentation\n",
    "Functions can do complicated things, so you should write an explanation of what your function does.  For small functions, this is less important, but it's a good habit to learn from the start.  Conventionally, Python functions are documented by writing a triple-quoted string:\n",
    "\n",
    "    def to_percentage(proportion):\n",
    "        \"\"\"Converts a proportion to a percentage.\"\"\"\n",
    "    \n",
    "    \n",
    "##### Body\n",
    "Now we start writing code that runs when the function is called.  This is called the *body* of the function.  We can write anything we could write anywhere else.  First let's give a name to the number we multiply a proportion by to get a percentage.\n",
    "\n",
    "    def to_percentage(proportion):\n",
    "        \"\"\"Converts a proportion to a percentage.\"\"\"\n",
    "        factor = 100\n",
    "\n",
    "##### `return`\n",
    "The special instruction `return` in a function's body tells Python to make the value of the function call equal to whatever comes right after `return`.  We want the value of `to_percentage(.5)` to be the proportion .5 times the factor 100, so we write:\n",
    "\n",
    "    def to_percentage(proportion):\n",
    "        \"\"\"Converts a proportion to a percentage.\"\"\"\n",
    "        factor = 100\n",
    "        return proportion * factor\n",
    "Note that `return` inside a function gives the function a value, while `print`, which we have used before, is a function which has no `return` value and just prints a certain value out to the console. The two are very different. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 2.1.** <br/>Define `to_percentage` in the cell below.  Call your function to convert the proportion .2 to a percentage.  Name that percentage `twenty_percent`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "for_assignment_type": "student"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20.0"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def to_percentage(number):\n",
    "    \"\"\" Converts a proportion to a percentage. \"\"\"\n",
    "    factor = 100\n",
    "    return number * factor\n",
    "\n",
    "twenty_percent = to_percentage(0.2)\n",
    "twenty_percent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2_1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like the built-in functions, you can use named values as arguments to your function.\n",
    "\n",
    "**Question 2.2.** <br/>Use `to_percentage` again to convert the proportion named `a_proportion` (defined below) to a percentage called `a_percentage`.\n",
    "\n",
    "*Note:* You don't need to define `to_percentage` again!  Just like other named things, functions stick around after you define them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "70.71067811865476"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_proportion = 2**(.5) / 2\n",
    "a_percentage = to_percentage(a_proportion)\n",
    "a_percentage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2_2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's something important about functions: the names assigned within a function body are only accessible within the function body. Once the function has returned, those names are gone.  So even though you defined `factor = 100` inside `to_percentage` above and then called `to_percentage`, you cannot refer to `factor` anywhere except inside the body of `to_percentage`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'factor' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-27-a219be0dab32>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# You should see an error when you run this.  (If you don't, you might\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;31m# have defined factor somewhere above.)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mfactor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'factor' is not defined"
     ]
    }
   ],
   "source": [
    "# You should see an error when you run this.  (If you don't, you might\n",
    "# have defined factor somewhere above.)\n",
    "factor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we've seen with the built-in functions, functions can also take strings (or arrays, or tables) as arguments, and they can return those things, too.\n",
    "\n",
    "**Question 2.3.** <br/>Define a function called `disemvowel`.  It should take a single string as its argument.  (You can call that argument whatever you want.)  It should return a copy of that string, but with all the characters that are vowels removed.  (In English, the vowels are the characters \"a\", \"e\", \"i\", \"o\", and \"u\".)\n",
    "\n",
    "*Hint:* To remove all the \"a\"s from a string, you can use `that_string.replace(\"a\", \"\")`.  The `.replace` method for strings returns another string, so you can call `replace` multiple times, one after the other. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Cn y rd ths wtht vwls?'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def disemvowel(a_string):\n",
    "    a= a_string.replace(\"a\",\"\")\n",
    "    b= a.replace(\"e\",\"\")\n",
    "    c= b.replace(\"u\",\"\")\n",
    "    d= c.replace(\"i\",\"\")\n",
    "    e= d.replace(\"o\",\"\")\n",
    "    return e\n",
    "# An example call to your function.  (It's often helpful to run\n",
    "# an example call from time to time while you're writing a function,\n",
    "# to see how it currently works.)\n",
    "disemvowel(\"Can you read this without vowels?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5e9bd845a7fa41d486480a40a93de76f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='Hello world', description='a_string'), Output()), _dom_classes=('widget-inte…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Alternatively, you can use interact to call your function\n",
    "_ = interact(disemvowel, a_string='Hello world')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2_3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Calls on calls on calls\n",
    "Just as you write a series of lines to build up a complex computation, it's useful to define a series of small functions that build on each other.  Since you can write any code inside a function's body, you can call other functions you've written.\n",
    "\n",
    "If a function is a like a recipe, defining a function in terms of other functions is like having a recipe for cake telling you to follow another recipe to make the frosting, and another to make the sprinkles.  This makes the cake recipe shorter and clearer, and it avoids having a bunch of duplicated frosting recipes.  It's a foundation of productive programming.\n",
    "\n",
    "For example, suppose you want to count the number of characters *that aren't vowels* in a piece of text.  One way to do that is this to remove all the vowels and count the size of the remaining string.\n",
    "\n",
    "**Question 2.4.** <br/>Write a function called `num_non_vowels`.  It should take a string as its argument and return a number.  The number should be the number of characters in the argument string that aren't vowels.\n",
    "\n",
    "*Hint:* The function `len` takes a string as its argument and returns the number of characters in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def num_non_vowels(a_string):\n",
    "    \"\"\"The number of characters in a string, minus the vowels.\"\"\"\n",
    "    a= a_string.replace(\"a\",\"\")\n",
    "    b= a.replace(\"e\",\"\")\n",
    "    c= b.replace(\"u\",\"\")\n",
    "    d= c.replace(\"i\",\"\")\n",
    "    e= d.replace(\"o\",\"\")\n",
    "    ss= len(e)\n",
    "    return ss\n",
    "    \n",
    "num_non_vowels(\"Go bears!\")\n",
    "# Try calling your function yourself to make sure the output is what\n",
    "# you expect. You can also use the interact function if you'd like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2_4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions can also encapsulate code that *does things* rather than just computing values.  For example, if you call `print` inside a function, and then call that function, something will get printed.\n",
    "\n",
    "The `movies_by_year` dataset in the textbook has information about movie sales in recent years.  Suppose you'd like to display the year with the 5th-highest total gross movie sales, printed in a human-readable way.  You might do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Year number 5 for total gross movie sales was: 2010\n"
     ]
    }
   ],
   "source": [
    "movies_by_year = Table.read_table(\"movies_by_year.csv\")\n",
    "rank = 5\n",
    "fifth_from_top_movie_year = movies_by_year.sort(\"Total Gross\", descending=True).column(\"Year\").item(rank-1)\n",
    "print(\"Year number\", rank, \"for total gross movie sales was:\", fifth_from_top_movie_year)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After writing this, you realize you also wanted to print out the 2nd and 3rd-highest years.  Instead of copying your code, you decide to put it in a function.  Since the rank varies, you make that an argument to your function.\n",
    "\n",
    "**Question 2.5.** <br/>Write a function called `print_kth_top_movie_year`.  It should take a single argument, the rank of the year (like 2, 3, or 5 in the above examples).  It should print out a message like the one above.  It shouldn't have a `return` statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "for_assignment_type": "student"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Year number 2 for total gross movie sales was: 2013\n",
      "Year number 3 for total gross movie sales was: 2012\n"
     ]
    }
   ],
   "source": [
    "def print_kth_top_movie_year(k):\n",
    "    # Our solution used 2 lines.\n",
    "    year = movies_by_year.sort(\"Total Gross\", descending=True).column(\"Year\").item(k-1)\n",
    "    print(\"Year number\", k, \"for total gross movie sales was:\", year)\n",
    "    return \n",
    "# Example calls to your function:\n",
    "print_kth_top_movie_year(2)\n",
    "print_kth_top_movie_year(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9b55fb4e2c9e4a67bdf92ba7918ced4f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Dropdown(description='k', options=(1, 2, 3, 4, 5, 6, 7, 8, 9), value=1), Output()), _dom…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# interact also allows you to pass in an array for a function argument. It will\n",
    "# then present a dropdown menu of options.\n",
    "_ = interact(print_kth_top_movie_year, k=np.arange(1, 10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2_5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. `apply`ing functions\n",
    "\n",
    "Defining a function is a lot like giving a name to a value with `=`.  In fact, a function is a value just like the number 1 or the text \"the\"!\n",
    "\n",
    "For example, we can make a new name for the built-in function `max` if we want:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "our_name_for_max = max\n",
    "our_name_for_max(2, 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The old name for `max` is still around:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max(2, 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try just writing `max` or `our_name_for_max` (or the name of any other function) in a cell, and run that cell.  Python will print out a (very brief) description of the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function max>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is this useful?  Since functions are just values, it's possible to pass them as arguments to other functions.  Here's a simple but not-so-practical example: we can make an array of functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([<built-in function max>, <function average at 0x7f8393d2ea60>,\n",
       "       <function are.equal_to at 0x7f8383bf2158>], dtype=object)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_array(max, np.average, are.equal_to)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 3.1.** <br/>Make an array containing any 3 other functions you've seen.  Call it `some_functions`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([<built-in function max>, <built-in function min>,\n",
       "       <built-in function abs>], dtype=object)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_functions = make_array (max,min,abs)\n",
    "some_functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 4\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Working with functions as values can lead to some funny-looking code.  For example, see if you can figure out why this works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_array(max, np.average, are.equal_to).item(0)(4, -2, 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a simpler example that's actually useful: the table method `apply`.\n",
    "\n",
    "`apply` calls a function many times, once on *each* element in a column of a table.  It produces an array of the results.  Here we use `apply` to convert every CEO's pay to a number, using the function you defined:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5.325e+07, 5.324e+07, 4.491e+07, 3.598e+07, 3.336e+07, 2.484e+07,\n",
       "       2.204e+07, 1.986e+07, 1.962e+07, 1.932e+07, 1.876e+07, 1.861e+07,\n",
       "       1.836e+07, 1.809e+07, 1.710e+07, 1.663e+07, 1.633e+07, 1.614e+07,\n",
       "       1.610e+07, 1.602e+07, 1.510e+07, 1.498e+07, 1.463e+07, 1.451e+07,\n",
       "       1.444e+07, 1.436e+07, 1.431e+07, 1.409e+07, 1.400e+07, 1.367e+07,\n",
       "       1.234e+07, 1.220e+07, 1.218e+07, 1.213e+07, 1.205e+07, 1.184e+07,\n",
       "       1.171e+07, 1.163e+07, 1.116e+07, 1.111e+07, 1.111e+07, 1.073e+07,\n",
       "       1.050e+07, 1.043e+07, 1.037e+07, 1.028e+07, 1.027e+07, 1.018e+07,\n",
       "       1.016e+07, 9.970e+06, 9.960e+06, 9.860e+06, 9.740e+06, 9.420e+06,\n",
       "       9.390e+06, 9.220e+06, 9.060e+06, 9.030e+06, 8.860e+06, 8.760e+06,\n",
       "       8.570e+06, 8.380e+06, 8.360e+06, 8.350e+06, 8.230e+06, 7.860e+06,\n",
       "       7.700e+06, 7.580e+06, 7.510e+06, 7.230e+06, 7.210e+06, 7.120e+06,\n",
       "       6.880e+06, 6.770e+06, 6.640e+06, 6.560e+06, 6.140e+06, 5.920e+06,\n",
       "       5.900e+06, 5.890e+06, 5.730e+06, 5.420e+06, 5.040e+06, 4.920e+06,\n",
       "       4.920e+06, 4.470e+06, 4.250e+06, 4.080e+06, 3.930e+06, 3.720e+06,\n",
       "       2.880e+06, 2.830e+06, 2.820e+06, 2.450e+06, 1.790e+06, 1.680e+06,\n",
       "       1.530e+06, 9.400e+05, 8.100e+05, 7.000e+04, 4.000e+04, 0.000e+00])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_compensation.apply(convert_pay_string_to_number, \"Total Pay\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an illustration of what that did:\n",
    "\n",
    "<img src=\"apply.png\" alt=\"For each value in the column 'Total Pay', the function `convert_pay_string_to_number` was applied.\"/>\n",
    "\n",
    "Note that we didn't write something like `convert_pay_string_to_number()` or `convert_pay_string_to_number(\"Total Pay\")`.  The job of `apply` is to call the function we give it, so instead of calling `convert_pay_string_to_number` ourselves, we just write its name as an argument to `apply`.\n",
    "\n",
    "**Question 3.2.** <br/>Using `apply`, make a table that's a copy of `raw_compensation` with one more column called \"Total Pay (\\$)\".  It should be the result of applying `convert_pay_string_to_number` to the \"Total Pay\" column, as we did above, and creating a new table which is the old one, but with the \"Total Pay\" column redone.  Call the new table `compensation`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>Rank</th> <th>Name</th> <th>Company (Headquarters)</th> <th>Total Pay</th> <th>% Change</th> <th>Cash Pay</th> <th>Equity Pay</th> <th>Other Pay</th> <th>Ratio of CEO pay to average industry worker pay</th> <th>Total Pay ($)</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>1   </td> <td>Mark V. Hurd*     </td> <td>Oracle (Redwood City)         </td> <td>$53.25   </td> <td>(No previous year)</td> <td>$0.95   </td> <td>$52.27    </td> <td>$0.02    </td> <td>362                                            </td> <td>5.325e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>2   </td> <td>Safra A. Catz*    </td> <td>Oracle (Redwood City)         </td> <td>$53.24   </td> <td>(No previous year)</td> <td>$0.95   </td> <td>$52.27    </td> <td>$0.02    </td> <td>362                                            </td> <td>5.324e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>3   </td> <td>Robert A. Iger    </td> <td>Walt Disney (Burbank)         </td> <td>$44.91   </td> <td>-3%               </td> <td>$24.89  </td> <td>$17.28    </td> <td>$2.74    </td> <td>477                                            </td> <td>4.491e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>4   </td> <td>Marissa A. Mayer  </td> <td>Yahoo! (Sunnyvale)            </td> <td>$35.98   </td> <td>-15%              </td> <td>$1.00   </td> <td>$34.43    </td> <td>$0.55    </td> <td>342                                            </td> <td>3.598e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>5   </td> <td>Marc Benioff      </td> <td>salesforce.com (San Francisco)</td> <td>$33.36   </td> <td>-16%              </td> <td>$4.65   </td> <td>$27.26    </td> <td>$1.45    </td> <td>338                                            </td> <td>3.336e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>6   </td> <td>John H. Hammergren</td> <td>McKesson (San Francisco)      </td> <td>$24.84   </td> <td>-4%               </td> <td>$12.10  </td> <td>$12.37    </td> <td>$0.37    </td> <td>222                                            </td> <td>2.484e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>7   </td> <td>John S. Watson    </td> <td>Chevron (San Ramon)           </td> <td>$22.04   </td> <td>-15%              </td> <td>$4.31   </td> <td>$14.68    </td> <td>$3.05    </td> <td>183                                            </td> <td>2.204e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>8   </td> <td>Jeffrey Weiner    </td> <td>LinkedIn (Mountain View)      </td> <td>$19.86   </td> <td>27%               </td> <td>$2.47   </td> <td>$17.26    </td> <td>$0.13    </td> <td>182                                            </td> <td>1.986e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>9   </td> <td>John T. Chambers**</td> <td>Cisco Systems (San Jose)      </td> <td>$19.62   </td> <td>19%               </td> <td>$5.10   </td> <td>$14.51    </td> <td>$0.01    </td> <td>170                                            </td> <td>1.962e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>10  </td> <td>John G. Stumpf    </td> <td>Wells Fargo  (San Francisco)  </td> <td>$19.32   </td> <td>-10%              </td> <td>$6.80   </td> <td>$12.50    </td> <td>$0.02    </td> <td>256                                            </td> <td>1.932e+07    </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>\n",
       "<p>... (92 rows omitted)</p>"
      ],
      "text/plain": [
       "Rank | Name               | Company (Headquarters)         | Total Pay | % Change           | Cash Pay | Equity Pay | Other Pay | Ratio of CEO pay to average industry worker pay | Total Pay ($)\n",
       "1    | Mark V. Hurd*      | Oracle (Redwood City)          | $53.25    | (No previous year) | $0.95    | $52.27     | $0.02     | 362                                             | 5.325e+07\n",
       "2    | Safra A. Catz*     | Oracle (Redwood City)          | $53.24    | (No previous year) | $0.95    | $52.27     | $0.02     | 362                                             | 5.324e+07\n",
       "3    | Robert A. Iger     | Walt Disney (Burbank)          | $44.91    | -3%                | $24.89   | $17.28     | $2.74     | 477                                             | 4.491e+07\n",
       "4    | Marissa A. Mayer   | Yahoo! (Sunnyvale)             | $35.98    | -15%               | $1.00    | $34.43     | $0.55     | 342                                             | 3.598e+07\n",
       "5    | Marc Benioff       | salesforce.com (San Francisco) | $33.36    | -16%               | $4.65    | $27.26     | $1.45     | 338                                             | 3.336e+07\n",
       "6    | John H. Hammergren | McKesson (San Francisco)       | $24.84    | -4%                | $12.10   | $12.37     | $0.37     | 222                                             | 2.484e+07\n",
       "7    | John S. Watson     | Chevron (San Ramon)            | $22.04    | -15%               | $4.31    | $14.68     | $3.05     | 183                                             | 2.204e+07\n",
       "8    | Jeffrey Weiner     | LinkedIn (Mountain View)       | $19.86    | 27%                | $2.47    | $17.26     | $0.13     | 182                                             | 1.986e+07\n",
       "9    | John T. Chambers** | Cisco Systems (San Jose)       | $19.62    | 19%                | $5.10    | $14.51     | $0.01     | 170                                             | 1.962e+07\n",
       "10   | John G. Stumpf     | Wells Fargo  (San Francisco)   | $19.32    | -10%               | $6.80    | $12.50     | $0.02     | 256                                             | 1.932e+07\n",
       "... (92 rows omitted)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "compensation = raw_compensation.with_column(\n",
    "    \"Total Pay ($)\",raw_compensation.apply(convert_pay_string_to_number, 'Total Pay'))\n",
    "\n",
    "compensation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the pay in numbers, we can compute things about them.\n",
    "\n",
    "**Question 3.3.**<br/>Compute the average total pay of the CEOs in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11445294.11764706"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "average_total_pay = np.average(compensation.column('Total Pay ($)'))\n",
    "average_total_pay"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 3.4.** <br/>Companies pay executives in a variety of ways: directly in cash; by granting stock or other \"equity\" in the company; or with ancillary benefits (like private jets).  Compute the proportion of each CEO's pay that was cash.  (Your answer should be an array of numbers, one for each CEO in the dataset.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: RuntimeWarning: invalid value encountered in true_divide\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0.01784038, 0.01784373, 0.55421955, 0.02779322, 0.13938849,\n",
       "       0.48711755, 0.19555354, 0.12437059, 0.25993884, 0.35196687,\n",
       "       0.3075693 , 0.22138635, 0.13126362, 0.1708126 , 0.23099415,\n",
       "       0.06734817, 0.13043478, 0.28004957, 0.33229814, 0.15355805,\n",
       "       0.29337748, 0.21829105, 0.31100478, 0.25086147, 0.2299169 ,\n",
       "       0.16991643, 0.31795947, 0.26188786, 0.28357143, 0.15654718,\n",
       "       0.38168558, 0.28934426, 0.20361248, 0.47650453, 0.45643154,\n",
       "       0.36402027, 0.2177626 , 0.24763543, 0.42562724, 0.2610261 ,\n",
       "       0.18361836, 0.1444548 , 0.33333333, 0.10834132, 0.20925747,\n",
       "       0.97276265, 0.22979552, 0.22789784, 0.37893701, 0.25175527,\n",
       "       0.73895582, 0.37018256, 0.2412731 , 0.2133758 , 0.20553781,\n",
       "       0.23318872, 0.33664459, 0.3875969 , 0.56094808, 0.11757991,\n",
       "       0.35239207, 0.24463007, 0.25      , 0.23712575, 0.43377886,\n",
       "       0.31424936, 0.46363636, 0.32585752, 0.24766977, 0.98755187,\n",
       "       0.27184466, 0.96207865, 0.31831395, 0.81979321, 0.23795181,\n",
       "       0.17530488, 0.21172638, 0.37162162, 0.27288136, 0.26994907,\n",
       "       0.55148342, 0.3597786 , 0.        , 0.47154472, 0.47154472,\n",
       "       0.29753915, 0.16235294, 0.48529412, 0.46819338, 0.32526882,\n",
       "       0.98958333, 0.61130742, 0.67021277, 0.75510204, 0.50837989,\n",
       "       0.98809524, 0.98039216, 0.9893617 , 0.87654321, 0.        ,\n",
       "       1.        ,        nan])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "compensation2 = raw_compensation.with_columns(\n",
    "    \"Total Pay($)\",raw_compensation.apply(convert_pay_string_to_number, 'Total Pay'),\n",
    "    \"Cash Pay($)\",raw_compensation.apply(convert_pay_string_to_number, 'Cash Pay')\n",
    ")\n",
    "\n",
    "cash_proportion = compensation2.column('Cash Pay($)')/compensation.column(\"Total Pay ($)\")\n",
    "cash_proportion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check out the \"% Change\" column in `compensation`.  It shows the percentage increase in the CEO's pay from the previous year.  For CEOs with no previous year on record, it instead says \"(No previous year)\".  The values in this column are *strings*, not numbers, so like the \"Total Pay\" column, it's not usable without a bit of extra work.\n",
    "\n",
    "Given your current pay and the percentage increase from the previous year, you can compute your previous year's pay.  For example, if your pay is \\$100 this year, and that's an increase of 50% from the previous year, then your previous year's pay was $\\frac{\\$100}{1 + \\frac{50}{100}}$, or around \\$66.66.\n",
    "\n",
    "**Question 3.5.** <br/>Create a new table called `with_previous_compensation`.  It should be a copy of `compensation`, but with the \"(No previous year)\" CEOs filtered out, and with an extra column called \"2014 Total Pay ($)\".  That column should have each CEO's pay in 2014.\n",
    "\n",
    "*Hint:* This question takes several steps, but each one is still something you've seen before.  Take it one step at a time, using as many lines as you need.  You can print out your results after each step to make sure you're on the right track.\n",
    "\n",
    "*Hint 2:* You'll need to define a function.  You can do that just above your other code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "for_assignment_type": "student"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "    <thead>\n",
       "        <tr>\n",
       "            <th>Rank</th> <th>Name</th> <th>Company (Headquarters)</th> <th>Total Pay</th> <th>% Change</th> <th>Cash Pay</th> <th>Equity Pay</th> <th>Other Pay</th> <th>Ratio of CEO pay to average industry worker pay</th> <th>Total Pay ($)</th> <th>2014 Total Pay ($)</th>\n",
       "        </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "        <tr>\n",
       "            <td>3   </td> <td>Robert A. Iger    </td> <td>Walt Disney (Burbank)         </td> <td>$44.91   </td> <td>0.97    </td> <td>$24.89  </td> <td>$17.28    </td> <td>$2.74    </td> <td>477                                            </td> <td>4.491e+07    </td> <td>4.35627e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>4   </td> <td>Marissa A. Mayer  </td> <td>Yahoo! (Sunnyvale)            </td> <td>$35.98   </td> <td>0.85    </td> <td>$1.00   </td> <td>$34.43    </td> <td>$0.55    </td> <td>342                                            </td> <td>3.598e+07    </td> <td>3.0583e+07        </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>5   </td> <td>Marc Benioff      </td> <td>salesforce.com (San Francisco)</td> <td>$33.36   </td> <td>0.84    </td> <td>$4.65   </td> <td>$27.26    </td> <td>$1.45    </td> <td>338                                            </td> <td>3.336e+07    </td> <td>2.80224e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>6   </td> <td>John H. Hammergren</td> <td>McKesson (San Francisco)      </td> <td>$24.84   </td> <td>0.96    </td> <td>$12.10  </td> <td>$12.37    </td> <td>$0.37    </td> <td>222                                            </td> <td>2.484e+07    </td> <td>2.38464e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>7   </td> <td>John S. Watson    </td> <td>Chevron (San Ramon)           </td> <td>$22.04   </td> <td>0.85    </td> <td>$4.31   </td> <td>$14.68    </td> <td>$3.05    </td> <td>183                                            </td> <td>2.204e+07    </td> <td>1.8734e+07        </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>8   </td> <td>Jeffrey Weiner    </td> <td>LinkedIn (Mountain View)      </td> <td>$19.86   </td> <td>1.27    </td> <td>$2.47   </td> <td>$17.26    </td> <td>$0.13    </td> <td>182                                            </td> <td>1.986e+07    </td> <td>2.52222e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>9   </td> <td>John T. Chambers**</td> <td>Cisco Systems (San Jose)      </td> <td>$19.62   </td> <td>1.19    </td> <td>$5.10   </td> <td>$14.51    </td> <td>$0.01    </td> <td>170                                            </td> <td>1.962e+07    </td> <td>2.33478e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>10  </td> <td>John G. Stumpf    </td> <td>Wells Fargo  (San Francisco)  </td> <td>$19.32   </td> <td>0.9     </td> <td>$6.80   </td> <td>$12.50    </td> <td>$0.02    </td> <td>256                                            </td> <td>1.932e+07    </td> <td>1.7388e+07        </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>11  </td> <td>John C. Martin**  </td> <td>Gilead Sciences (Foster City) </td> <td>$18.76   </td> <td>0.99    </td> <td>$5.77   </td> <td>$12.98    </td> <td>$0.01    </td> <td>117                                            </td> <td>1.876e+07    </td> <td>1.85724e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "        <tr>\n",
       "            <td>13  </td> <td>Shantanu Narayen  </td> <td>Adobe Systems (San Jose)      </td> <td>$18.36   </td> <td>1.03    </td> <td>$2.41   </td> <td>$15.85    </td> <td>$0.09    </td> <td>125                                            </td> <td>1.836e+07    </td> <td>1.89108e+07       </td>\n",
       "        </tr>\n",
       "    </tbody>\n",
       "</table>\n",
       "<p>... (71 rows omitted)</p>"
      ],
      "text/plain": [
       "Rank | Name               | Company (Headquarters)         | Total Pay | % Change | Cash Pay | Equity Pay | Other Pay | Ratio of CEO pay to average industry worker pay | Total Pay ($) | 2014 Total Pay ($)\n",
       "3    | Robert A. Iger     | Walt Disney (Burbank)          | $44.91    | 0.97     | $24.89   | $17.28     | $2.74     | 477                                             | 4.491e+07     | 4.35627e+07\n",
       "4    | Marissa A. Mayer   | Yahoo! (Sunnyvale)             | $35.98    | 0.85     | $1.00    | $34.43     | $0.55     | 342                                             | 3.598e+07     | 3.0583e+07\n",
       "5    | Marc Benioff       | salesforce.com (San Francisco) | $33.36    | 0.84     | $4.65    | $27.26     | $1.45     | 338                                             | 3.336e+07     | 2.80224e+07\n",
       "6    | John H. Hammergren | McKesson (San Francisco)       | $24.84    | 0.96     | $12.10   | $12.37     | $0.37     | 222                                             | 2.484e+07     | 2.38464e+07\n",
       "7    | John S. Watson     | Chevron (San Ramon)            | $22.04    | 0.85     | $4.31    | $14.68     | $3.05     | 183                                             | 2.204e+07     | 1.8734e+07\n",
       "8    | Jeffrey Weiner     | LinkedIn (Mountain View)       | $19.86    | 1.27     | $2.47    | $17.26     | $0.13     | 182                                             | 1.986e+07     | 2.52222e+07\n",
       "9    | John T. Chambers** | Cisco Systems (San Jose)       | $19.62    | 1.19     | $5.10    | $14.51     | $0.01     | 170                                             | 1.962e+07     | 2.33478e+07\n",
       "10   | John G. Stumpf     | Wells Fargo  (San Francisco)   | $19.32    | 0.9      | $6.80    | $12.50     | $0.02     | 256                                             | 1.932e+07     | 1.7388e+07\n",
       "11   | John C. Martin**   | Gilead Sciences (Foster City)  | $18.76    | 0.99     | $5.77    | $12.98     | $0.01     | 117                                             | 1.876e+07     | 1.85724e+07\n",
       "13   | Shantanu Narayen   | Adobe Systems (San Jose)       | $18.36    | 1.03     | $2.41    | $15.85     | $0.09     | 125                                             | 1.836e+07     | 1.89108e+07\n",
       "... (71 rows omitted)"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For reference, our solution involved more than just this one line of code\n",
    "...\n",
    "compensation_new= compensation.where('% Change',are.not_equal_to(\"(No previous year)\"))\n",
    "\n",
    "def convert_percent(pay_string):\n",
    "    pay = float(pay_string.strip(\"%\"))/100+1\n",
    "    return pay\n",
    "compensation3 = compensation_new.with_column(\n",
    "    \"% Change\",compensation_new.apply(convert_percent, '% Change'))\n",
    "value = compensation3.column(9)*compensation3.column(4)\n",
    "\n",
    "with_previous_compensation = compensation3.with_column(\"2014 Total Pay ($)\",value)\n",
    "with_previous_compensation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 3.6.** <br/>What was the average pay of these CEOs in 2014?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11339681.481481481"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "average_pay_2014 = np.average(with_previous_compensation.column('2014 Total Pay ($)'))\n",
    "average_pay_2014"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3_6')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Histograms\n",
    "Earlier, we computed the average pay among the CEOs in our 102-CEO dataset.  The average doesn't tell us everything about the amounts CEOs are paid, though.  Maybe just a few CEOs make the bulk of the money, even among these 102.\n",
    "\n",
    "We can use a *histogram* to display more information about a set of numbers.  The table method `hist` takes a single argument, the name of a column of numbers.  It produces a histogram of the numbers in that column.\n",
    "\n",
    "**Question 4.1.** <br/>Make a histogram of the pay of the CEOs in `compensation`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "The column \"Total Pay($)\" is not in the table. The table contains these columns: Rank, Name, Company (Headquarters), Total Pay, % Change, Cash Pay, Equity Pay, Other Pay, Ratio of CEO pay to average industry worker pay, Total Pay ($)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-100-7d96f1ddb708>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mcompensation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Total Pay($)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/datascience/tables.py\u001b[0m in \u001b[0;36mhist\u001b[0;34m(self, overlay, bins, bin_column, unit, counts, group, side_by_side, width, height, *columns, **vargs)\u001b[0m\n\u001b[1;32m   2444\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mgroup\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2445\u001b[0m                 \u001b[0mcolumns_included\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2446\u001b[0;31m             \u001b[0mself\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mselect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mcolumns_included\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2447\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mgroup\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2448\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mbin_column\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/datascience/tables.py\u001b[0m in \u001b[0;36mselect\u001b[0;34m(self, *column_or_columns)\u001b[0m\n\u001b[1;32m    629\u001b[0m         \u001b[0mtable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    630\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mlabel\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mlabels\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 631\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_add_column_and_format\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlabel\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    632\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mtable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    633\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/datascience/tables.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, index_or_label)\u001b[0m\n\u001b[1;32m    166\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    167\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__getitem__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex_or_label\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 168\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindex_or_label\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    169\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    170\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__setitem__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex_or_label\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalues\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/datascience/tables.py\u001b[0m in \u001b[0;36mcolumn\u001b[0;34m(self, index_or_label)\u001b[0m\n\u001b[1;32m    273\u001b[0m                 \u001b[0;34m'The column \"{}\" is not in the table. The table contains '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    274\u001b[0m                 \u001b[0;34m'these columns: {}'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 275\u001b[0;31m                 \u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindex_or_label\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m', '\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    276\u001b[0m             )\n\u001b[1;32m    277\u001b[0m         if (isinstance(index_or_label, int)\n",
      "\u001b[0;31mValueError\u001b[0m: The column \"Total Pay($)\" is not in the table. The table contains these columns: Rank, Name, Company (Headquarters), Total Pay, % Change, Cash Pay, Equity Pay, Other Pay, Ratio of CEO pay to average industry worker pay, Total Pay ($)"
     ]
    }
   ],
   "source": [
    "compensation.hist('Total Pay($)')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 4.2.** <br/>Looking at the histogram, how many CEOs made more than \\$30 million?  (Answer the question by filling in your answer manually.  You'll have to do a bit of arithmetic; feel free to use Python as a calculator.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_ceos_more_than_30_million = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 4.3.**<br/> Answer the same question with code.  *Hint:* Use the table method `where` and the property `num_rows`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_ceos_more_than_30_million_2 =compensation.where('Total Pay ($)', are.above(30000000)).num_rows\n",
    "num_ceos_more_than_30_million_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q4_3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Submission"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great job! :D You're finished with lab 4! Be sure to...\n",
    "- **run all the tests and verify that they all pass** (the next cell has a shortcut for that), \n",
    "- **Review the notebook one last time, we will be grading the final state of your notebook after the deadline**,\n",
    "- **Save and Checkpoint** from the `File` menu,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 4\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 3\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# For your convenience, you can run this cell to run all the tests at once!\n",
    "import os\n",
    "_ = [ok.grade(q[:-3]) for q in os.listdir(\"tests\") if q.startswith('q')]"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}