cs230_notes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"week4a.png\">\n",
    "<div>L=4, L = layers</div>\n",
    "<div align='left'>$\\begin{align}x^{[l]}\\end{align}$ is the number of nodes in layer l. For l=0 there are 3 nodes, l=1 there are 5 nodes, l=2 there\n",
    "are 5 nodes, l=3 there are 3 nodes, l=4 there is one node. $n_x=3$</div>\n",
    "<div>$\\begin{align} a^{[l]} = activations\\space  in\\space layer\\space l. \\end{align}$</div>\n",
    "<div>$\\begin{align} a^{[l]} = g^{[l]}(z^{l})\\space\\space w^{[l]} = weights for z^[l]\\end{align}$</div>\n",
    "\n",
    "<div>WHAT DOES HE USE TO MAKE THE NN DIAGRAM? NOT PP?</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"week4b.png\">\n",
    "<h6>First step develop the notation for forward propagation from the input to output as the input is being multiplied by weights\n",
    "and a bias added, progress to vectorized notation for performance with the exception the \n",
    "layers are a for loop and not vectorized. </h6>\n",
    "<div>The weight matrix is capitalized. The input and output vectors are lower case. All vectors are lower case</div>\n",
    "<div>The input is x or $\\begin{align}a^{[0]}\\end{align}$. The output of the first layer before the activation is a linear function\n",
    "$z^{[1]} = W^{[1]}x+b^{[1]}$, The output of the first linear function is $a^{[2]}=g^{[2]}(z^[2])$ where g is the activation function\n",
    "and z is the output after the input is multiplied by a weight and a bias added</div>\n",
    "<div>Each layer follows the same pattern, multiply the output of the previous activation stage with the weights\n",
    "add a bias and put the output of linear function through an activation. </div>\n",
    "<div>$\\begin{align} z^{2} = W^{[2]a^{[1]}+b^{[2]}}\\end{align}$ and $a^{[2]}=g^{[2]}(z^{[2]})$</div>\n",
    "<div>skip layer 3 and use the pattern we see in the previous 2 layers wherae a is from the l-1 where l is the current layer\n",
    "</div>\n",
    "<div>layer 4 is  $z^{[4]}=W^{[4]}a^{[3]}+b^{[4]}$ for the linear layer and the activation layer is $a^{[4]} = g^{[4]}(Z^{[4]})$</div>\n",
    "<div>The general is z^{[l]}= W^{[l]}a^{[l-1]}+b{[l]} for linear function layer and activation as $a^{[l]}=g^{[l]}(z^{l})$ </div>\n",
    "<h6>Vectorized notatiopn</h6>\n",
    "<div>$\\begin{align} Z^{[1]} = W^{[1]}X+b^{[1]} \\end{align}$ where $X=A^{[0]}$ and the activation is\n",
    "$A^{[1]} = g^{[1]}(Z^{[1]})$</div>\n",
    "<div>same pattern as for nonvectorized notation</div>\n",
    "<div>$\\begin{align} \\hat{y} = g(Z^{[4]})=A^{[4]} \\end{align}$</div>\n",
    "<img src=\"week4c.png\">\n",
    "<h6>Adding dimensions constrained by the number of weights in each layer and the number of inputs.</h6>\n",
    "<div>There are 5 layers in this network, L=5, $z^{[1]}=W^{[1]}x+b[1]$ we know x shape is (2,1) because there are 2 features or inputs from the diagram\n",
    "and we know z has shape (3,1) bc there are 3 nodes from the picture. So (3,1) = W(2,1) + b. W has to be 3,2 bc 2,3 wont work with \n",
    "the matrix multiplication rules. </div>\n",
    " <img src=\"week4d.png\">\n",
    "<div>ARE ALL THE NONVECTORIZED LOWER CASE? OR is W upper regardless of vectorization? bc it is a matrix? Is z lower case \n",
    "or upper case? (3,1) is a vector or matrix? </div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}