unit03_mult_lin_reg/linreg_inclass.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiple Linear Regression In-class Exercise\n",
    "\n",
    "Before starting the lab, the following exercise will perform a simple multiple variable linear fitting on a civil engineering dataset.  In doing this exercise, you will learn to:\n",
    "\n",
    "* Load data from a `csv` file using the `pandas` package\n",
    "* Visualize relations between different variables with a scatter plot.\n",
    "* Fit a multiple variable linear model using the `sklearn` package\n",
    "* Evaluate the fit.\n",
    "\n",
    "We begin by loading the packages we will need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import linear_model\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Data\n",
    "\n",
    "Concrete is one of the most basic construction materials.  In this exercise, you will download a simple dataset for predicting the strength of concrete from the attributes of concrete.  The data set comes from this very nice\n",
    "[kaggle competition](https://www.kaggle.com/maajdl/yeh-concret-data).  Kaggle has many excellent dataset for your project.  \n",
    "You can download the data with the following command.  After running this command, you should have the file `data.csv` in your local folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File data.csv is already downloaded\n"
     ]
    }
   ],
   "source": [
    "fn_src = 'https://raw.githubusercontent.com/sdrangan/introml/master/unit03_mult_lin_reg/Concrete_Data_Yeh.csv'\n",
    "fn_dst = 'data.csv'\n",
    "\n",
    "import os\n",
    "from six.moves import urllib\n",
    "\n",
    "if os.path.isfile(fn_dst):\n",
    "    print('File %s is already downloaded' % fn_dst)\n",
    "else:        \n",
    "    urllib.request.urlretrieve(fn_src, fn_dst)\n",
    "    print('File %s downloaded' % fn_dst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `pandas` package has excellent methods for loading `csv` files.  YThe following command loads the `csv` file into a dataframe `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('data.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ues the `df.head()` to print the first few rows of the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, the target variable will be the concrete strength in Megapascals, `csMPa`.  We will use the other 8 attributes as predictors to predict the strength.  \n",
    "\n",
    "Create a list called `xnames` of the 8 names of the predictors.  You can do this as follows:\n",
    "* Get the list of names of the columns from `df.columns.tolist()`.  \n",
    "* Remove the last items from the list using indexing.\n",
    "\n",
    "Print the `xnames`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#   xnames = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the data matrix `X` and target vector `y` from the dataframe `df`.  \n",
    "\n",
    "Recall that to get the items from a dataframe, you can use syntax such as\n",
    "\n",
    "    s = np.array(df['slag'])  \n",
    "        \n",
    "which gets the data in the column `slag` and puts it into an array `s`.  You can also get multiple columns with syntax like\n",
    "\n",
    "    X12 = np.array(df['cement', 'slag'])  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#    X = ...\n",
    "#    y = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the subplot command, create two plots, side-by-side with:\n",
    "* `y` vs. the `cement` on the left (attribute 0)\n",
    "* `y` vs. the `water` on the right (attribute 3)\n",
    "Label the axes and use the `plt.tight_layout()` to adjust the plots nicely at the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Training and Test\n",
    "\n",
    "Split the data into training and test.  Use 30% for test and 70% for training.  You can do the splitting manually or use the `sklearn` package `train_test_split`.   Store the training data in `Xtr,ytr` and test data in `Xts,yts`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# TODO\n",
    "#  Xtr,Xts,ytr,yts = train_test_split(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fit a Linear Model\n",
    "\n",
    "Create a linear regression model object `reg` and fit the model on the training data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#   reg = ...\n",
    "#   reg.fit(...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute the predicted values `yhat_tr` on the training data and print the `R^2` value on the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#    yhat_tr = ...\n",
    "#    rsq_tr = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now compute the predicted values `yhat_ts` on the test data and print the `R^2` value on the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#    yhat_ts = ...\n",
    "#    rsq_ts = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a scatter plot of the actual vs. predicted values of `y` on the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating Different Variables\n",
    "\n",
    "One way to see the importance of different variables is to compute the *normalized* coefficients:\n",
    "\n",
    "    coeff_norm[j]  = reg.coef_[j] * std(Xtr[:,j]) / std(ytr) \n",
    "    \n",
    "which represents the change in the target for a change of one standard deviation in the attribute `j`.  The change in the target is normalized by its standard deviation.  \n",
    "\n",
    "Compute the `coeff_norm` for the 8 attributes and plot the values using a `plt.stem()` plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n",
    "#  coeff_norm = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which variable has the highest normalized coefficient, and hence most influence on the concrete strength?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}