unit04_model_sel/model_sel_inclass.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "okgQCCkHoywI"
   },
   "source": [
    "# Model Selection In-Class Exercise\n",
    "\n",
    "In this example, you will a linear model to data and select the model order by model order selection.  First load the standard packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PNpSxz4ZoywK"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pickle\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qV_kpz8koywO"
   },
   "source": [
    "## Load the Data\n",
    "\n",
    "The data in this exercise is completely synthetic.  You can load it with the following commmand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jcsWD_KQoywP",
    "outputId": "02b6ac6f-523e-434c-e249-a23faf7784b0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File synth_data.p is already downloaded\n"
     ]
    }
   ],
   "source": [
    "fn_src = 'https://raw.githubusercontent.com/sdrangan/introml/master/unit04_model_sel/synth_data.p'\n",
    "fn_dst = 'synth_data.p'\n",
    "\n",
    "import os\n",
    "from six.moves import urllib\n",
    "\n",
    "if os.path.isfile(fn_dst):\n",
    "    print('File %s is already downloaded' % fn_dst)\n",
    "else:        \n",
    "    urllib.request.urlretrieve(fn_src, fn_dst)\n",
    "    print('File %s downloaded' % fn_dst)\n",
    "\n",
    "with open(fn_dst,'rb') as fp:\n",
    "    x,y = pickle.load(fp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "A1pl4tZDoywU"
   },
   "source": [
    "Plot the data `y` vs. `x` using a scatter plot. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JXTCfEdVoywV"
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qMdo0iqroywY"
   },
   "source": [
    "## Fit the Data\n",
    "\n",
    "We will now try to fit the data.  First, split the data into training and test.  You can use the `train_test_split()` method in the `sklearn` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5_vV75MzoywY"
   },
   "outputs": [],
   "source": [
    "# TODO\n",
    "# xtr, xts, ytr, yts = train_test_split(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Pi2lYVmboywb"
   },
   "source": [
    "Now try to fit a linear model to the data.  You can pick any linear model with a variable number `d` of basis functions.  For example, you can use the polynomials up to degree `d-1`.  \n",
    "\n",
    "For each model order `d`:\n",
    "* Fit the model on the training data\n",
    "* Test the model on the test data\n",
    "\n",
    "Plot the test error vs. `d`.  Select the model order `dopt` with the lowest test error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6hhCAy6Eo4Gf"
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IiAiaNoco6QH"
   },
   "source": [
    "Select the optimal model order `dopt`.  Re-train the model for that model order.  On one plot:\n",
    "* Plot the predicted value `yhat` vs. `x` for your model for `x in [0,35]`\n",
    "* Plot a scatter plot of the test data `xts` vs. `yts`\n",
    "\n",
    "Does your model fit the test data well?\n",
    "Does it extrapolate reasonably in the range `x >= 30`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GNtzaacQpEHA"
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "w4YprWBPpe_c"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "model_sel_inclass.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}