A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.
Traditional Machine Learning
Title
Dataset
Description
Notebooks
Perceptron
TBD
TBD
Logistic Regression
TBD
TBD
Softmax Regression (Multinomial Logistic Regression)
TBD
TBD
Softmax Regression with MLxtend's plot_decision_regions on Iris
TBD
TBD
Title
Dataset
Description
Notebooks
Multilayer Perceptron
TBD
TBD
Multilayer Perceptron with Dropout
TBD
TBD
Multilayer Perceptron with Batch Normalization
TBD
TBD
Multilayer Perceptron with Backpropagation from Scratch
TBD
TBD
Convolutional Neural Networks
Title
Dataset
Description
Notebooks
Convolutional Neural Network
TBD
TBD
CNN with He Initialization
TBD
TBD
Title
Dataset
Description
Notebooks
Replacing Fully-Connnected by Equivalent Convolutional Layers
TBD
TBD
Title
Dataset
Description
Notebooks
AlexNet Trained on CIFAR-10
TBD
TBD
AlexNet with Grouped Convolutions Trained on CIFAR-10
TBD
TBD
Title
Description
Daset
Notebooks
DenseNet-121 Digit Classifier Trained on MNIST
TBD
TBD
DenseNet-121 Image Classifier Trained on CIFAR-10
TBD
TBD
Title
Dataset
Description
Notebooks
"All Convolutionl Net" -- A Fully Convolutional Neural Network
TBD
TBD
Title
Dataset
Description
Notebooks
LeNet-5 on MNIST
TBD
TBD
LeNet-5 on CIFAR-10
TBD
TBD
LeNet-5 on QuickDraw
TBD
TBD
Title
Dataset
Description
Notebooks
MobileNet-v2 on Cifar-10
TBD
TBD
MobileNet-v3 small on Cifar-10
TBD
TBD
MobileNet-v3 large on Cifar-10
TBD
TBD
Title
Dataset
Description
Notebooks
Network in Network Trained on CIFAR-10
TBD
TBD
Title
Dataset
Description
Notebooks
Convolutional Neural Network VGG-16 Trained on CIFAR-10
TBD
TBD
VGG-16 Smile Classifier Trained on CelebA
TBD
TBD
VGG-16 Dogs vs Cats Classifier
TBD
TBD
Convolutional Neural Network VGG-19
TBD
TBD
Title
Dataset
Description
Notebooks
ResNet and Residual Blocks
TBD
TBD
ResNet-18 Digit Classifier Trained on MNIST
TBD
TBD
ResNet-18 Gender Classifier Trained on CelebA
TBD
TBD
ResNet-34 Digit Classifier Trained on MNIST
TBD
TBD
ResNet-34 Object Classifier Trained on QuickDraw
TBD
TBD
ResNet-34 Gender Classifier Trained on CelebA
TBD
TBD
ResNet-50 Digit Classifier Trained on MNIST
TBD
TBD
ResNet-50 Gender Classifier Trained on CelebA
TBD
TBD
ResNet-101 Gender Classifier Trained on CelebA
TBD
TBD
ResNet-101 Trained on CIFAR-10
TBD
TBD
ResNet-152 Gender Classifier Trained on CelebA
TBD
TBD
Ordinal Regression and Deep Learning
Please note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.
Title
Dataset
Description
Notebooks
BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier
TBD
TBD
Filter Response Normalization for Network-in-Network CIFAR-10 Classifier
TBD
TBD
Title
Dataset
Description
Notebooks
Siamese Network with Multilayer Perceptrons
TBD
TBD
Fully-connected Autoencoders
Title
Dataset
Description
Notebooks
Autoencoder (MNIST)
TBD
TBD
Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier
TBD
TBD
Convolutional Autoencoders
Title
Dataset
Description
Notebooks
Convolutional Autoencoder with Deconvolutions / Transposed Convolutions
TBD
TBD
Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance
TBD
TBD
Convolutional Autoencoder with Deconvolutions (without pooling operations)
TBD
TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation
TBD
TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA
TBD
TBD
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw
TBD
TBD
Title
Dataset
Description
Notebooks
Variational Autoencoder
TBD
TBD
Convolutional Variational Autoencoder
TBD
TBD
Conditional Variational Autoencoders
Title
Dataset
Description
Notebooks
Conditional Variational Autoencoder (with labels in reconstruction loss)
TBD
TBD
Conditional Variational Autoencoder (without labels in reconstruction loss)
TBD
TBD
Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss)
TBD
TBD
Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss)
TBD
TBD
Generative Adversarial Networks (GANs)
Title
Dataset
Description
Notebooks
Fully Connected GAN on MNIST
TBD
TBD
Fully Connected Wasserstein GAN on MNIST
TBD
TBD
Convolutional GAN on MNIST
TBD
TBD
Convolutional GAN on MNIST with Label Smoothing
TBD
TBD
Convolutional Wasserstein GAN on MNIST
TBD
TBD
Deep Convolutional GAN (DCGAN) on Cats and Dogs Images
TBD
TBD
Deep Convolutional GAN (DCGAN) on CelebA Face Images
TBD
TBD
Graph Neural Networks (GNNs)
Title
Dataset
Description
Notebooks
Most Basic Graph Neural Network with Gaussian Filter on MNIST
TBD
TBD
Basic Graph Neural Network with Edge Prediction on MNIST
TBD
TBD
Basic Graph Neural Network with Spectral Graph Convolution on MNIST
TBD
TBD
Recurrent Neural Networks (RNNs)
Many-to-one: Sentiment Analysis / Classification
Title
Dataset
Description
Notebooks
A simple single-layer RNN (IMDB)
TBD
TBD
A simple single-layer RNN with packed sequences to ignore padding characters (IMDB)
TBD
TBD
RNN with LSTM cells (IMDB)
TBD
TBD
RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors
TBD
TBD
RNN with LSTM cells and Own Dataset in CSV Format (IMDB)
TBD
TBD
RNN with GRU cells (IMDB)
TBD
TBD
Multilayer bi-directional RNN (IMDB)
TBD
TBD
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News)
TBD
TBD
Many-to-Many / Sequence-to-Sequence
Title
Dataset
Description
Notebooks
A simple character RNN to generate new text (Charles Dickens)
TBD
TBD
Title
Dataset
Description
Notebooks
Baseline CNN
MNIST
A simple baseline with traditional train/validation/test splits
K-fold with pl_cross
MNIST
A 5-fold cross-validation run using the pl_cross
library
Title
Dataset
Description
Notebooks
Cyclical Learning Rate
TBD
TBD
Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet)
TBD
TBD
Gradient Clipping (w. MLP on MNIST)
TBD
TBD
Title
Dataset
Description
Notebooks
Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10)
TBD
TBD
Visualization and Interpretation
Title
Dataset
Description
Notebooks
Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)
TBD
TBD
Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images)
TBD
TBD
PyTorch Workflows and Mechanics
PyTorch Lightning Examples
Title
Dataset
Description
Notebooks
MLP in Lightning with TensorBoard -- continue training the last model
TBD
TBD
MLP in Lightning with TensorBoard -- checkpointing best model
TBD
TBD
Title
Dataset
Description
Notebooks
Custom Data Loader Example for PNG Files
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD)
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images
TBD
TBD
Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST
TBD
TBD
Training and Preprocessing
Title
Dataset
Description
Notebooks
Generating Validation Set Splits
TBD
TBD
Dataloading with Pinned Memory
TBD
TBD
Standardizing Images
TBD
TBD
Image Transformation Examples
TBD
TBD
Char-RNN with Own Text File
TBD
TBD
Sentiment Classification RNN with Own CSV File
TBD
TBD
Improving Memory Efficiency
Title
Dataset
Description
Notebooks
Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10)
TBD
TBD
Title
Description
Notebooks
Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA
TBD
Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example)
TBD
Title
Dataset
Description
Notebooks
PyTorch with and without Deterministic Behavior -- Runtime Benchmark
TBD
TBD
Sequential API and hooks
TBD
TBD
Weight Sharing Within a Layer
TBD
TBD
Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib
TBD
TBD
Title
Dataset
Description
Notebooks
Getting Gradients of an Intermediate Variable in PyTorch
TBD
TBD
TensorFlow Workflows and Mechanics
Title
Description
Notebooks
Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives
TBD
Storing an Image Dataset for Minibatch Training using HDF5
TBD
Using Input Pipelines to Read Data from TFRecords Files
TBD
Using Queue Runners to Feed Images Directly from Disk
TBD
Using TensorFlow's Dataset API
TBD
Training and Preprocessing
Title
Dataset
Description
Notebooks
Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives
TBD
TBD
Title
Description
Notebooks
TorchMetrics
How do we use it, and what's the difference between .update() and .forward()?