1st June 2019

Mini-Hackathon: The objective of this experiment is to help a leading retailer forecast the sales.

You need to help predict the RMSE values for the test set; there are 4 datasets:

1. Macro Economic Dataset
2. Events and Holidays Dataset
3. Weather Data Set
4. Train Data (Sales and the Year/Month)

---

2nd June 2019

Lectures:

1. Lecture 21: Beyond AlexNet
2. Lecture 22: BP Revisited
3. Lecture 23: Siamese Networks
4. Lecture 24: GANs

GD: Variations

* Batch GD: Update the parameters after the gradients are computed for the entire training set.
* Stochastic GD: Randomly shuffle the training set,and update the parameters after gradients are computed for each training example.
* Mini-Batch Stochastic GD: Update the parameters after gradients are computed for a randomly drawn mini-batch of training examples.

Momentum

* Nesterov Momentum

Optimization Algorithms

* SGD vs Adam
* Adagrad
* Adadelta
* RMSProp
* Adam

• Regularization
• Data Normalization
• Data Augmentation or Jittering
• Dropout
• Batch Normalization

Beyond AlexNet

* What changed over time? 
* VGG-Nets
* GoogleNet and Inception
* Residual Net
* DenseNet
* Performance Indices
    • Accuracy
    • Model complexity
    • Memory usage
    • Computational complexity
    • Inference time

Siamese Networks

* Siamese and Triplet networks/losses are popular for solving fine grain classification (e.g. Face), capturing subjective needs (eg. Invariance to rotation), rankings etc.
* Recognition Vs Verification
* Siamese: Loss

Face processing pipeline

1. Input Image
2. Detect
3. Transform
4. Crop
5. Recognition

* Hyperface: state of art

GANs

* Generative Adversarial Networks
    * Discriminator D
    * Generator G
* Extension: Deep Convolutional GAN(DCGAN)    

---