human_remains_one_shot_learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "human-remains-one-shot-learning.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/bonetrade/one-shot-learning/blob/master/human_remains_one_shot_learning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RWeTgO_e6Y9e",
        "colab_type": "text"
      },
      "source": [
        "# One Shot Learning with Siamese Networks\n",
        "\n",
        "Code adapated from Harshvardhan Gupta, 2017 [One Shot Learning With Siamese Networks](https://github.com/harveyslash/Facial-Similarity-with-Siamese-Networks-in-Pytorch/blob/master/Siamese-networks-medium.ipynb) and discussed in \n",
        "Graham and Huffer, Can One-Shot Learning Identify Origins of Human Remains Shown on Social Media? \n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "We're assuming you have loaded this notebook with Google Colab and that you are signed into a Google account. You run each block of code in sequential order the first time through. If after having trained a network you wish to resume testing at a later date, we show you how to save your model and reload it, at the appropriate blocks.\n",
        "\n",
        "## Hardware Acceleration\n",
        "\n",
        "Under 'Edit' -> 'Notebook Settings' make sure to select 'GPU' for Hardware Acceleration. The next block of code double-checks to make sure you've got GPU selected. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fjVyWDJAF4-m",
        "colab_type": "code",
        "outputId": "72e81140-d294-4b30-8fb6-8b4b84025aca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import tensorflow as tf\n",
        "device_name = tf.test.gpu_device_name()\n",
        "if device_name != '/device:GPU:0':\n",
        "  raise SystemError('GPU device not found')\n",
        "print('Found GPU at: {}'.format(device_name))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found GPU at: /device:GPU:0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jrQFGF1T8wgI",
        "colab_type": "text"
      },
      "source": [
        "## Load your data\n",
        "\n",
        "There are a variety of ways to get your data into this notebook. The easiest are to upload directly, or to connect your Google Drive account. If you connect your Google Drive account, it will appear as a folder within the tray. You can right-click on a folder within your drive, copy the path, and then paste that into your code. Note that you'll have to delete the leading `/content/` from the path. \n",
        "\n",
        "**We are assuming that you have zipped a folder called `data` with two subfolders `testing` and `training`, and within each of those folders, one folder per class of _item_ you wish to train on/test.**\n",
        "\n",
        "**To upload** open the tray at left by clicking on the `>` button. Select 'Files' and then 'Upload'. You can then select a zip file from your machine. The tray does not refresh automatically, so you'll need to hit 'refresh' periodically.\n",
        "\n",
        "Then, unzip the data by running the next block. If you're using Google drive, skip to the next block."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "byy27-Z_EHgx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# if I'm uploading\n",
        "!unzip data.zip -d data/"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cp4KELeH9wcp",
        "colab_type": "text"
      },
      "source": [
        "**To mount your Google drive** run the next block of code. The results block will display a URL. Click on this, and a new window will open, confirming that you want to connect your drive. Once you've confirmed, an authorization code will be displayed. Copy this code, and paste into the results block. If all goes well, the results block will shortly display the text, 'Mounted at /content/drive'. Refresh the files pane in the tray at left, and you will see it there."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ne9mUU_TQFLJ",
        "colab_type": "code",
        "outputId": "b9436b69-0519-4e14-f2cd-16d265c971bf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser:\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ft2c2VM-Zx7",
        "colab_type": "text"
      },
      "source": [
        "In the two blocks below, we show you how to copy a file from your drive to this space, how to make a new directory (`mkdir`) and how to unzip a folder into that new directory."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x0rfcjBnQbVg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#if you have data already on google drive\n",
        "!cp \"drive/My Drive/one-shot-test/data.zip\" data.zip"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wiy0t5z3J1Is",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#upload data from local machine then:\n",
        "!mkdir data && unzip data.zip -d data/\n",
        "\n",
        "#or copy it over from google drive"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T5oU-S9i-mkz",
        "colab_type": "text"
      },
      "source": [
        "---\n",
        "## Imports\n",
        "\n",
        "We now need to import some libraries that we will use to build the neural network.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7l92t8_-FOeK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%matplotlib inline\n",
        "import torchvision\n",
        "import torchvision.datasets as dset\n",
        "import torchvision.transforms as transforms\n",
        "from torch.utils.data import DataLoader,Dataset\n",
        "import matplotlib.pyplot as plt\n",
        "import torchvision.utils\n",
        "import numpy as np\n",
        "import random\n",
        "from PIL import Image\n",
        "import torch\n",
        "from torch.autograd import Variable\n",
        "import PIL.ImageOps    \n",
        "import torch.nn as nn\n",
        "from torch import optim\n",
        "import torch.nn.functional as F"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K0yyYaOq-031",
        "colab_type": "text"
      },
      "source": [
        "## Helper functions\n",
        "\n",
        "Here we create two helper functions, `imshow` and `show_plot`. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gE-TfRPnGGxd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def imshow(img,text=None,should_save=False):\n",
        "    npimg = img.numpy()\n",
        "    plt.axis(\"off\")\n",
        "    if text:\n",
        "        plt.text(75, 8, text, style='italic',fontweight='bold',\n",
        "            bbox={'facecolor':'white', 'alpha':0.8, 'pad':10})\n",
        "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
        "    plt.show()    \n",
        "\n",
        "def show_plot(iteration,loss):\n",
        "    plt.plot(iteration,loss)\n",
        "    plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6hh2Y_l8_bpY",
        "colab_type": "text"
      },
      "source": [
        "## Configuration class\n",
        "\n",
        "We now create a class to configure some variables we will be reusing. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ml-XlyHEGKB7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Config():\n",
        "    training_dir = \"data/training/\"\n",
        "    testing_dir = \"data/testing/\"\n",
        "    train_batch_size = 64\n",
        "    train_number_epochs = 100"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RC_2Oc5L_8V4",
        "colab_type": "text"
      },
      "source": [
        "## Custom dataset class\n",
        "\n",
        "Here we set up the code to generates a pair of images from our training dataset, 0 for geniune pair and 1 for imposter pair. It goes through the training directory, pairing images and assuming that images within a subfolder are similar and images from different subfolders are different. This is also where we attach filenames to the images so that when we are testing later we know which pairs of images we're dealing with. This addition to the original code is courtesy Tim Sherratt."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C49BrRm0GdTY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SiameseNetworkDataset(Dataset):\n",
        "    \n",
        "    def __init__(self,imageFolderDataset,transform=None,should_invert=True):\n",
        "        self.imageFolderDataset = imageFolderDataset    \n",
        "        self.transform = transform\n",
        "        self.should_invert = should_invert\n",
        "        \n",
        "    def __getitem__(self,index):\n",
        "        img0_tuple = random.choice(self.imageFolderDataset.imgs)\n",
        "        #we need to make sure approx 50% of images are in the same class\n",
        "        should_get_same_class = random.randint(0,1) \n",
        "        if should_get_same_class:\n",
        "            while True:\n",
        "                #keep looping till the same class image is found\n",
        "                img1_tuple = random.choice(self.imageFolderDataset.imgs) \n",
        "                if img0_tuple[1]==img1_tuple[1]:\n",
        "                    break\n",
        "        else:\n",
        "            while True:\n",
        "                #keep looping till a different class image is found\n",
        "                \n",
        "                img1_tuple = random.choice(self.imageFolderDataset.imgs) \n",
        "                if img0_tuple[1] !=img1_tuple[1]:\n",
        "                    break\n",
        "\n",
        "        img0 = Image.open(img0_tuple[0])\n",
        "        img1 = Image.open(img1_tuple[0])\n",
        "        img0 = img0.convert(\"L\")\n",
        "        img1 = img1.convert(\"L\")\n",
        "        \n",
        "        if self.should_invert:\n",
        "            img0 = PIL.ImageOps.invert(img0)\n",
        "            img1 = PIL.ImageOps.invert(img1)\n",
        "\n",
        "        if self.transform is not None:\n",
        "            img0 = self.transform(img0)\n",
        "            img1 = self.transform(img1)\n",
        "        \n",
        "          # return img0, img1 , torch.from_numpy(np.array([int(img1_tuple[1]!=img0_tuple[1])],dtype=np.float32))\n",
        "\t\t# Courtesy of Tim Sherrat\n",
        "    # The new return line\n",
        "\t\t# Note the addition of img0_tuple[0], img1_tuple[0] which contain the paths\n",
        "        return img0, img1 , torch.from_numpy(np.array([int(img1_tuple[1]!=img0_tuple[1])],dtype=np.float32)), img0_tuple[0], img1_tuple[0]\n",
        "\n",
        "    \n",
        "    def __len__(self):\n",
        "        return len(self.imageFolderDataset.imgs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6k0ZQeUoA3-C",
        "colab_type": "text"
      },
      "source": [
        "## Setting the image folder to be used by the custom dataset\n",
        "\n",
        "In the first block, we specify the location of the training data. In the second block, we pass the images through the custom dataset class,  resizing them to 100 x 100 pixels, transforming them into tensors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5gAhHc3_A3ED",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "folder_dataset = dset.ImageFolder(root=Config.training_dir)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PqQBMCGIKdsS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "siamese_dataset = SiameseNetworkDataset(imageFolderDataset=folder_dataset,\n",
        "                                        transform=transforms.Compose([transforms.Resize((100,100)),\n",
        "                                                                      transforms.ToTensor()\n",
        "                                                                      ])\n",
        "                                       ,should_invert=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m_KJxHsaBfU0",
        "colab_type": "text"
      },
      "source": [
        "## Visualising some of the training data\n",
        "\n",
        "The top row and the bottom row of any column is one pair. The 0s and 1s correspond to the column of the image. 1 indiciates dissimilar, and 0 indicates similar.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DVBSCKdIKhFz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "vis_dataloader = DataLoader(siamese_dataset,\n",
        "                        shuffle=True,\n",
        "                        num_workers=8,\n",
        "                        batch_size=8)\n",
        "dataiter = iter(vis_dataloader)\n",
        "\n",
        "\n",
        "example_batch = next(dataiter)\n",
        "concatenated = torch.cat((example_batch[0],example_batch[1]),0)\n",
        "imshow(torchvision.utils.make_grid(concatenated))\n",
        "print(example_batch[2].numpy())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WoUIsQJ5BpZp",
        "colab_type": "text"
      },
      "source": [
        "## Create the neural network\n",
        "\n",
        "We will use a standard convolutional neural network. Each convolutional layer has batch normalisation and then dropout. As Gupta says, 'There is nothing special about this network. It accepts an input of 100px by 100px and has 3 full connected layers after the convolution layers'. This might be where you want to experiment, eventually, with adding more layers and so on."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0qtvqaHjKjSR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class SiameseNetwork(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(SiameseNetwork, self).__init__()\n",
        "        self.cnn1 = nn.Sequential(\n",
        "            nn.ReflectionPad2d(1),\n",
        "            nn.Conv2d(1, 4, kernel_size=3),\n",
        "            nn.ReLU(inplace=True),\n",
        "            nn.BatchNorm2d(4),\n",
        "            \n",
        "            nn.ReflectionPad2d(1),\n",
        "            nn.Conv2d(4, 8, kernel_size=3),\n",
        "            nn.ReLU(inplace=True),\n",
        "            nn.BatchNorm2d(8),\n",
        "\n",
        "\n",
        "            nn.ReflectionPad2d(1),\n",
        "            nn.Conv2d(8, 8, kernel_size=3),\n",
        "            nn.ReLU(inplace=True),\n",
        "            nn.BatchNorm2d(8),\n",
        "\n",
        "\n",
        "        )\n",
        "\n",
        "        self.fc1 = nn.Sequential(\n",
        "            nn.Linear(8*100*100, 500),\n",
        "            nn.ReLU(inplace=True),\n",
        "\n",
        "            nn.Linear(500, 500),\n",
        "            nn.ReLU(inplace=True),\n",
        "\n",
        "            nn.Linear(500, 5))\n",
        "\n",
        "    def forward_once(self, x):\n",
        "        output = self.cnn1(x)\n",
        "        output = output.view(output.size()[0], -1)\n",
        "        output = self.fc1(output)\n",
        "        return output\n",
        "\n",
        "    def forward(self, input1, input2):\n",
        "        output1 = self.forward_once(input1)\n",
        "        output2 = self.forward_once(input2)\n",
        "        return output1, output2\n",
        "        "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rtETbBhICRle",
        "colab_type": "text"
      },
      "source": [
        "## Contrastive loss function\n",
        "\n",
        "Here we define the contrastive loss."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "as_ZMSzVKnHj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class ContrastiveLoss(torch.nn.Module):\n",
        "    \"\"\"\n",
        "    Contrastive loss function.\n",
        "    Based on: http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self, margin=2.0):\n",
        "        super(ContrastiveLoss, self).__init__()\n",
        "        self.margin = margin\n",
        "\n",
        "    def forward(self, output1, output2, label):\n",
        "        euclidean_distance = F.pairwise_distance(output1, output2, keepdim = True)\n",
        "        loss_contrastive = torch.mean((1-label) * torch.pow(euclidean_distance, 2) +\n",
        "                                      (label) * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))\n",
        "\n",
        "\n",
        "        return loss_contrastive"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QIv7hO3UCdWf",
        "colab_type": "text"
      },
      "source": [
        "## Training the neural network\n",
        "\n",
        "The next three blocks configure all of the variables and settings for training the neural network. You might want to experiment with changing the values of the Adam optimizer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Or4xt6VDKp2H",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_dataloader = DataLoader(siamese_dataset,\n",
        "                        shuffle=True,\n",
        "                        num_workers=8,\n",
        "                        batch_size=Config.train_batch_size)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xnRiUE3nKuqe",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net = SiameseNetwork().cuda()\n",
        "criterion = ContrastiveLoss()\n",
        "optimizer = optim.Adam(net.parameters(),lr = 0.00006 )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "04O3bd3SCiXG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "counter = []\n",
        "loss_history = [] \n",
        "iteration_number= 0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I7YqWETMC1Zl",
        "colab_type": "text"
      },
      "source": [
        "### Start the training\n",
        "\n",
        "This next block will start the training for the number of epochs set at the start of the notebook in the configuration block. The code is slightly modified so that filenames get stored for the images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fA0y-hd7K5et",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "for epoch in range(0,Config.train_number_epochs):\n",
        "    for i, data in enumerate(train_dataloader,0):\n",
        "\t\t# Modifications by Tim Sherrat, to enable path printing\n",
        "    # Note the underscores! They're the path values, which we don't actually want here\n",
        "\t\t# Underscores are used as variable names for things we don't actually want\n",
        "        img0, img1 , label, _, _ = data\n",
        "        img0, img1 , label = img0.cuda(), img1.cuda() , label.cuda()\n",
        "        optimizer.zero_grad()\n",
        "        output1,output2 = net(img0,img1)\n",
        "        loss_contrastive = criterion(output1,output2,label)\n",
        "        loss_contrastive.backward()\n",
        "        optimizer.step()\n",
        "        if i %10 == 0 :\n",
        "            print(\"Epoch number {}\\n Current loss {}\\n\".format(epoch,loss_contrastive.item()))\n",
        "            iteration_number +=10\n",
        "            counter.append(iteration_number)\n",
        "            loss_history.append(loss_contrastive.item())\n",
        "show_plot(counter,loss_history)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sKVPWER5DJVS",
        "colab_type": "text"
      },
      "source": [
        "## Save the model\n",
        "\n",
        "The block below saves the state dictionary, and the model. This is handy so that once you *have* a trained model, you can return to it if your notebook connection to Colab is broken, or if you have to set the project aside for a while. The second block copies (`cp`) the file to a location on Google drive. You can also download the file to your machine directly by right-clicking the filename in the tray at left (you might need to hit 'refresh' to see it, first)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wXTopDCEhyy5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#save the model!\n",
        "torch.save(net.state_dict(),'net_params.pkl')\n",
        "torch.save(net, 'net.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f2QLT90P6eNb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "cp net_params.pkl \"drive/My Drive/one-shot-test\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sizl8QETDvAg",
        "colab_type": "text"
      },
      "source": [
        "## Reloading the model at a later time\n",
        "\n",
        "If this is your first time through the notebook, you don't need to worry about this; skip down to **Testing**. If you're returning to the project, make sure\n",
        "+ that you've got hardware acceleration turned on\n",
        "+ that you've run all of the code again to import the necessary libraries, and set the various configurations \n",
        "  + _including_ the SiameseNetwork class\n",
        "\n",
        "The block below assumes you've connected Google drive. Alternatively you can upload directly."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oCbuAtMOigQ6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#copy the model back from your drive\n",
        "!cp \"drive/My Drive/one-shot-test/net_params.pkl\" data/net_params.pkl\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KLQ_aEcCERPI",
        "colab_type": "text"
      },
      "source": [
        "...then tell the machine to load the model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YvKhEPhr7oqu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#load the model\n",
        "net=SiameseNetwork()\n",
        "net.load_state_dict(torch.load('data/net_params.pkl'))\n",
        "dp = nn.DataParallel(net) #https://github.com/pytorch/pytorch/issues/3805\n",
        "\n",
        "#the incompatiblekeys message might not be an issue - see https://gpytorch.readthedocs.io/en/latest/examples/00_Basic_Usage/Saving_and_Loading_Models.html \n",
        "#which replicates that incompatiblekeys message without any kind of comment, seems to be hunkydory"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2LLQYb1LErDI",
        "colab_type": "text"
      },
      "source": [
        "## Testing\n",
        "\n",
        "You will end up running this block over and over. This block loads pairs of images from different subfolders in your `testing` folder and compares the results using euclidean distance. It will print out the images with the dissimilarity distance, as well as printing out the filenames for each pair. You can experiment with printing these results to a file."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oA4QpoRCLgSK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "folder_dataset_test = dset.ImageFolder(root=Config.testing_dir)\n",
        "siamese_dataset = SiameseNetworkDataset(imageFolderDataset=folder_dataset_test,\n",
        "                                        transform=transforms.Compose([transforms.Resize((100,100)),\n",
        "                                                                      transforms.ToTensor()\n",
        "                                                                      ])\n",
        "                                       ,should_invert=False)\n",
        "\n",
        "test_dataloader = DataLoader(siamese_dataset,num_workers=6,batch_size=1,shuffle=False)\n",
        "dataiter = iter(test_dataloader)\n",
        "# This is unpacking values from SiameseNetworkDataset\n",
        "# So the 4th value should now be the path of image 1\n",
        "\n",
        "x0,_,_,p0,_ = next(dataiter)\n",
        "\n",
        "for i in range(20):\n",
        "    # The 5th value should now be the path of image 2\n",
        "    _,x1,label2,_,p1 = next(dataiter)\n",
        "\n",
        "    concatenated = torch.cat((x0,x1),0)\n",
        "\n",
        "    # Again getting rid of cuda for my Mac\n",
        "    output1,output2 = net(Variable(x0).cuda(),Variable(x1).cuda())\n",
        "    #output1,output2 = net(Variable(x0),Variable(x1))\n",
        "    euclidean_distance = F.pairwise_distance(output1, output2)\n",
        "\n",
        "    # show the images:\n",
        "    imshow(torchvision.utils.make_grid(concatenated),'Dissimilarity: {:.2f}'.format(euclidean_distance.item()))\n",
        "    \n",
        "    # Show the paths for the two images\n",
        "    print('Image 1: {}'.format(p0[0])) \n",
        "    print('Image 2: {}'.format(p1[0]))\n",
        "    print('Dissimilarity: {:.2f}'.format(euclidean_distance.item()))\n",
        "    print('-----')\n",
        "    # just print out the results as a kind of table.\n",
        "    # from tabulate import tabulate\n",
        "    # print(tabulate([[p0, p1,euclidean_distance.item()]], headers=['image1', 'image2','euclidean_distance']))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVt-evEFFP62",
        "colab_type": "text"
      },
      "source": [
        "---\n",
        "# The End\n",
        "\n",
        "The two code blocks below print out the structure of the neural network, and the version info of all of the loaded packages in this environment. This information is useful for replicating this notebook in the future.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YDnSSRJpUuzH",
        "colab_type": "code",
        "outputId": "4282f4a3-1383-44e2-9add-1d331951dc19",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 421
        }
      },
      "source": [
        "from torchvision import models\n",
        "model = net\n",
        "print(model)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "SiameseNetwork(\n",
            "  (cnn1): Sequential(\n",
            "    (0): ReflectionPad2d((1, 1, 1, 1))\n",
            "    (1): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
            "    (2): ReLU(inplace)\n",
            "    (3): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (4): ReflectionPad2d((1, 1, 1, 1))\n",
            "    (5): Conv2d(4, 8, kernel_size=(3, 3), stride=(1, 1))\n",
            "    (6): ReLU(inplace)\n",
            "    (7): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (8): ReflectionPad2d((1, 1, 1, 1))\n",
            "    (9): Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1))\n",
            "    (10): ReLU(inplace)\n",
            "    (11): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "  )\n",
            "  (fc1): Sequential(\n",
            "    (0): Linear(in_features=80000, out_features=500, bias=True)\n",
            "    (1): ReLU(inplace)\n",
            "    (2): Linear(in_features=500, out_features=500, bias=True)\n",
            "    (3): ReLU(inplace)\n",
            "    (4): Linear(in_features=500, out_features=5, bias=True)\n",
            "  )\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K2x_rKjj7UxC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "outputId": "ff4b6dc3-9b76-4652-eddf-af0f6ae898cb"
      },
      "source": [
        "# watermark is not installed by default.\n",
        "# the first time through, uncomment the two lines below\n",
        "# then run the block.\n",
        "#\n",
        "#\n",
        "# !pip install watermark\n",
        "# %load_ext watermark\n",
        "%watermark -v -m -p numpy,scipy,torchvision,PIL,tensorflow,torch -g"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CPython 3.6.8\n",
            "IPython 5.5.0\n",
            "\n",
            "numpy 1.16.4\n",
            "scipy 1.3.0\n",
            "torchvision 0.3.0\n",
            "PIL 4.3.0\n",
            "tensorflow 1.14.0\n",
            "torch 1.1.0\n",
            "\n",
            "compiler   : GCC 8.0.1 20180414 (experimental) [trunk revision 259383\n",
            "system     : Linux\n",
            "release    : 4.14.79+\n",
            "machine    : x86_64\n",
            "processor  : x86_64\n",
            "CPU cores  : 2\n",
            "interpreter: 64bit\n",
            "Git hash   :\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}