Firebase_ML_on_device_recommentations.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Firebase ML on-device recommentations.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "TUfAcER1oUS6",
        "scrolled": true
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gb7qyhNL1yWt"
      },
      "source": [
        "# On-device recommendations with Firebase ML and TensorFlow Lite"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fw5Y7snSuG51"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/FirebaseExtended/codelab-contentrecommendation-android/blob/master/Firebase_ML_on_device_recommentations.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/FirebaseExtended/codelab-contentrecommendation-android/blob/master/Firebase_ML_on_device_recommentations.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fyYiyNxVp6mS"
      },
      "source": [
        "## Overview"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CShg7PXmqGUJ"
      },
      "source": [
        "This is the notebook for step 11 of the codelab [**Add recommendations to your app with TensorFlow Lite and Firebase**](https://codelabs.developers.google.com/codelabs/contentrecommendation-android). Before running the code in this notebook, complete steps 1-10 of the codelab to get your app and console projects set up.\n",
        "\n",
        "This code base provides a toolkit to train an on-device recommendation\n",
        "tensorflow model with user data collected in your app with Firebase Analytics. This model will then be deployed with Firebase ML to serve movie recommendations in the sample app FireFlix. \n",
        "\n",
        "This Notebook shows an end-to-end example that 1) imports Firebase Analytics data from BigQuery 2) preprocesses that data to prepare it for training 3) trains a recommendations model using the data and 4) exports the model in tflite format, ready to use in apps to run inference and serve recommendations.\n",
        "\n",
        "Since the app we use in the codelab is just a sample app, it doesn't have the usage necessary to generate a significant amount of analytics events. Since training accurate models requires a large amount of data, for the purposes of this codelab and notebook, we will be simulating a larger analytics event store by using the public [movielens](https://grouplens.org/datasets/movielens/) dataset, but you could\n",
        "adapt the data processing script for your dataset and train your own\n",
        "recommendation model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bcLF2PKkSbV3"
      },
      "source": [
        "## Prerequisites\n",
        "\n",
        "Run the cell below to clone the tensorflow recommendations model sample from Github. This is the model we will use, with our analytics training data, to create the recommendations model.\n",
        "\n",
        "The model uses a Convolutional neural-network encoder (CNN): applying multiple layers of convolutional neural-network to generate an encoding of the user history analytics data. For more details, refer to the [documentation]() for the underlying tensorflow model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6cv3K3oaksJv",
        "scrolled": true
      },
      "source": [
        "!git clone https://github.com/tensorflow/examples\n",
        "%cd /content/examples/lite/examples/recommendation/ml/\n",
        "!pip install -r requirements.txt\n",
        "!pip install --upgrade google-cloud-storage google-cloud-bigquery[bqstorage]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "joSTkeEWN01m"
      },
      "source": [
        "## Set up authentication\n",
        "\n",
        "In this notebook, we use analytics data from BigQuery to generate training data for our recommendations model. To access BigQuery data from the Colab notebook, you need to upload the service account file that you downloaded in step 10 of the codelab.\n",
        "\n",
        "Note: If this step is throwing an error, you can either:\n",
        "1. Manually upload the json file to the /content folder using the Folder icon in the left menu. Then set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the file path.\n",
        "i.e. If file was uploaded to /content, run:\n",
        "`os.environ[\"GOOGLE_APPLICATION_CREDENTIALS\"]='/content/<your_service_acct_file_name>`\n",
        "OR,\n",
        "2. Try disabling third party cookies in your browser, as [suggested here](https://stackoverflow.com/a/61494336)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mqN2Qro5sN5f"
      },
      "source": [
        "import os\n",
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()\n",
        "\n",
        "for fn in uploaded.keys():\n",
        "  print('User uploaded file \"{name}\" with length {length} bytes'.format(\n",
        "      name=fn, length=len(uploaded[fn])))\n",
        "  with open('/content/' + fn, 'wb') as f:\n",
        "    f.write(uploaded[fn])\n",
        "  os.environ[\"GOOGLE_APPLICATION_CREDENTIALS\"]='/content/' + fn\n",
        "  projectID = fn.rsplit(\"-\", 1)[0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4dy0D_RQK2D"
      },
      "source": [
        "# Import app analytics data from BigQuery\n",
        "\n",
        "In this step, we will load the analytics data we collected in the app with Firebase Analytics and sent to BigQuery. We will load the data into the pandas data processing library and then preprocess this data to be the appropriate format for input for the model training step."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JUc7uJXMRLNb"
      },
      "source": [
        "## Enable BigQuery IPython magics\n",
        "\n",
        "BigQuery provides several convenience IPython magics that we will use to fetch data with the %load_ext magic below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5J2plSCksN5h"
      },
      "source": [
        "%reload_ext google.cloud.bigquery"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pq46MEB5RlUR"
      },
      "source": [
        "## Import data\n",
        "\n",
        "We use the following SQL statement to get items from the table we created in BigQuery. Firebase Analytics exports a lot of additional information, such as device type, platform version, etc, that we don't need for the purposes of training this model. Initially, we only get a limited amount of rows to briefly explore the form of this data and select which fields are important.\n",
        "\n",
        "Notice that a row in the dataframe is created for each analytics event logged in the app. This row has many properties, but the ones that are of importance for this notebook are the fields:\n",
        "* event_name\n",
        "* event_timestamp\n",
        "* items\n",
        "* user_pseudo_id\n",
        "\n",
        "Notice that some fields, such as the **items** field is actually an object. We will extract the subfield of interest below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qoAKLXkZsN5j"
      },
      "source": [
        "%%bigquery analytics_test_import\n",
        "SELECT\n",
        "    *\n",
        "FROM `firebase_recommendations_dataset.recommendations_table`\n",
        "LIMIT 10"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RLXfzEckVMZ1"
      },
      "source": [
        "analytics_test_import"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_wysL9g8WW5U"
      },
      "source": [
        "All of the columns included in each analytics event entry."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V9jHxMTqU8E2"
      },
      "source": [
        "analytics_test_import.columns"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ImvkOzFwWQN4"
      },
      "source": [
        "Of the information logged under 'items', we are only interested in 'item_id',which corresponds to the ID of the movie the user interacted with."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UVazzlpLVwTN"
      },
      "source": [
        "analytics_test_import['items'][0][0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HF3vnWJXThab"
      },
      "source": [
        "Now we run the following command to import the whole dataset into a variable. Note how we only import the fields which we are interested in for training purposes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rXK2PAK5AMpM"
      },
      "source": [
        "%%bigquery analytics_data_real\n",
        "SELECT\n",
        "    items,user_pseudo_id,event_timestamp\n",
        "FROM `firebase_recommendations_dataset.recommendations_table`"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OdM67AvOsN5p"
      },
      "source": [
        "analytics_data_real.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UTdiPeVjUADI"
      },
      "source": [
        "# Preprocess the dataset\n",
        "\n",
        "In this step, we create a lambda function to extract a subfield 'item_id' from the items object. This represents the movie_id, so we also rename the columns to match."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BJzJb0AsDQlk"
      },
      "source": [
        "analytics = analytics_data_real\n",
        "def getMovieID(row):\n",
        "  items_obj = row['items'][0]\n",
        "  return items_obj['item_id']\n",
        "analytics['movie_id'] = analytics.apply(lambda row: getMovieID(row), axis=1)\n",
        "analytics"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qXi0XmaJUff6"
      },
      "source": [
        "We drop the 'items' column since we don't need anything else from it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7hUZGrBhErs9"
      },
      "source": [
        "analytics.rename(columns={'user_pseudo_id': 'user_id', 'event_timestamp': 'timestamp'}, inplace=True)\n",
        "analytics.drop(['items'], axis=1, inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aQW3-22MUnrH"
      },
      "source": [
        "Here is our processed dataframe containing only the data we want to use in training.\n",
        "\n",
        "The data has the following properties:\n",
        "*   UserIDs range between 1 and 6040\n",
        "*   MovieIDs range between 1 and 3952\n",
        "*   Timestamp is represented in seconds since the epoch as returned by time(2)\n",
        "*   Each user has at least 20 ratings"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D8S7hp8FFxg6"
      },
      "source": [
        "analytics"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pS2IXA2IW2Xd"
      },
      "source": [
        "## Sort and group training data to create training examples\n",
        "\n",
        "Our analytics events need to be reorganized in the format required for the model training step. We will create an object that maps key user_id to a list of movies that user has seen. We use the timestamp data to create the sequential context."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E2gKMD_4sN57"
      },
      "source": [
        "import collections\n",
        "def convert_to_timelines(df):\n",
        "  \"\"\"Convert ratings data to user.\"\"\"\n",
        "  timelines = collections.defaultdict(list)\n",
        "  movie_counts = collections.Counter()\n",
        "  for user_id, timestamp, movie_id in df.values:\n",
        "    timelines[user_id].append([movie_id, int(timestamp)])\n",
        "    movie_counts[movie_id] += 1\n",
        "  # Sort per-user timeline by timestamp\n",
        "  for (user_id, timeline) in timelines.items():\n",
        "    timeline.sort(key=lambda x: x[1])\n",
        "    timelines[user_id] = [movie_id for movie_id, _ in timeline]\n",
        "  return timelines, movie_counts\n",
        "timelines, counts = convert_to_timelines(analytics)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0ASLyOYZqCH"
      },
      "source": [
        "The timelines object contains a list of movie_id's keyed on user_id to indicate the sequence of movies that user has interacted with."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l6Zm23tgsN59"
      },
      "source": [
        "import itertools\n",
        "\n",
        "for key, val in sorted(timelines.items())[0:10]:\n",
        "  print(key, val)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GcxmkcOzZ7Zj"
      },
      "source": [
        "## Generate training examples\n",
        "\n",
        "We use the timelines data to generate tensorflow training examples. We discard any timeline with less than 3 context items, and we consider context lengths of 100 items. We perform the following steps:\n",
        "\n",
        "* Groups movie records by user, and orders per-user movie records by timestamp.\n",
        "* Generates Tensorflow examples with features: 1) \"context\": time-ordered sequential movie IDs 2) \"label\": next movie ID user viewed as label. \"max_history_length\" is taken in as parameter to define \"context\" feature shape, if not enough history found, right padding with out-of-vocab ID 0 will be performed.\n",
        "* Then partition the available data into a training and test set.\n",
        "\n",
        "Sample generated training example with max user history as 10:\n",
        "```\n",
        "0 : {   # (tensorflow.Example)\n",
        "  features: {   # (tensorflow.Features)\n",
        "    feature: {\n",
        "      key  : \"context\"\n",
        "      value: {\n",
        "        int64_list: {\n",
        "          value: [ 595, 2687, 745, 588, 1, 2355, 2294, 783, 1566, 1907 ]\n",
        "        }\n",
        "      }\n",
        "    }\n",
        "    feature: {\n",
        "      key  : \"label\"\n",
        "      value: {\n",
        "        int64_list: {\n",
        "          value: [ 48 ]\n",
        "        }\n",
        "      }\n",
        "    }\n",
        "  }\n",
        "}\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hwWgd41asN5_"
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "# used to pad when user doesn't have enough context\n",
        "OOV_MOVIE_ID = 0\n",
        "\n",
        "def generate_examples_from_timelines(timelines,\n",
        "                                     min_timeline_len=3,\n",
        "                                     max_context_len=100):\n",
        "  \"\"\"Convert user timelines to tf examples.\n",
        "\n",
        "  Convert user timelines to tf examples by adding all possible context-label\n",
        "  pairs in the examples pool.\n",
        "\n",
        "  Args:\n",
        "    timelines: the user timelines to process.\n",
        "    min_timeline_len: minimum length of the user timeline.\n",
        "    max_context_len: maximum length of context signals.\n",
        "\n",
        "  Returns:\n",
        "    train_examples: tf example list for training.\n",
        "    test_examples: tf example list for testing.\n",
        "  \"\"\"\n",
        "  train_examples = []\n",
        "  test_examples = []\n",
        "  for timeline in timelines.values():\n",
        "    # Skip if timeline is shorter than min_timeline_len.\n",
        "    if len(timeline) < min_timeline_len:\n",
        "      continue\n",
        "    for label_idx in range(1, len(timeline)):\n",
        "      start_idx = max(0, label_idx - max_context_len)\n",
        "      context = timeline[start_idx:label_idx]\n",
        "      # Pad context with out-of-vocab movie id 0.\n",
        "      while len(context) < max_context_len:\n",
        "        context.append(OOV_MOVIE_ID)\n",
        "      label = timeline[label_idx]\n",
        "      feature = {\n",
        "          \"context\":\n",
        "              tf.train.Feature(int64_list=tf.train.Int64List(value=context)),\n",
        "          \"label\":\n",
        "              tf.train.Feature(int64_list=tf.train.Int64List(value=[label]))\n",
        "      }\n",
        "      tf_example = tf.train.Example(features=tf.train.Features(feature=feature))\n",
        "      if label_idx == len(timeline) - 1:\n",
        "        test_examples.append(tf_example.SerializeToString())\n",
        "      else:\n",
        "        train_examples.append(tf_example.SerializeToString())\n",
        "  return train_examples, test_examples\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "3v0XcJ3OsN6A"
      },
      "source": [
        "train_examples, test_examples = generate_examples_from_timelines(timelines)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3SS4ZYJCjRvO"
      },
      "source": [
        "Write examples to tfrecords, to be loaded in the model training step."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KVOe4EXUeyjw"
      },
      "source": [
        "def write_tfrecords(tf_examples, filename):\n",
        "  \"\"\"Write tf examples to tfrecord file.\"\"\"\n",
        "  with tf.io.TFRecordWriter(filename) as file_writer:\n",
        "    for example in tf_examples:\n",
        "      file_writer.write(example)\n",
        "\n",
        "output_dir = 'data/examples'\n",
        "OUTPUT_TRAINING_DATA_FILENAME = \"train_movielens_1m.tfrecord\"\n",
        "OUTPUT_TESTING_DATA_FILENAME = \"test_movielens_1m.tfrecord\"\n",
        "\n",
        "if not tf.io.gfile.exists(output_dir):\n",
        "  tf.io.gfile.makedirs(output_dir)\n",
        "write_tfrecords(\n",
        "    tf_examples=train_examples,\n",
        "    filename=os.path.join(output_dir, OUTPUT_TRAINING_DATA_FILENAME))\n",
        "write_tfrecords(\n",
        "    tf_examples=test_examples,\n",
        "    filename=os.path.join(output_dir, OUTPUT_TESTING_DATA_FILENAME))\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XQcQ6AssuBN8"
      },
      "source": [
        "# Train model\n",
        "\n",
        "The training launcher script uses TensorFlow keras compile/fit APIs and performs\n",
        "the following steps to kick start training and evaluation process:\n",
        "\n",
        "*   Set up both train and eval dataset input function.\n",
        "*   Construct keras model according to provided configs, please refer to sample.config file in the source code to config your model architecture, such as embedding dimension, convolutional neural network params, LSTM units etc.\n",
        "*   Setup loss function. In this code base, we leverages customized batch softmax loss function.\n",
        "*   Setup optimizer, with flag specified learning rate and gradient clip if needed.\n",
        "*   Setup evaluation metrics, we provided recall@k metrics by default.\n",
        "*   Compile model with loss function, optimizer and defined metrics.\n",
        "*   Setup callbacks for tensorboard and checkpoint manager.\n",
        "*   Run model.fit with compiled model, where you could specify number of epochs to train, number of train steps in each epoch and number of eval steps in each epoch.\n",
        "\n",
        "## Model training parameters\n",
        "\n",
        "### Encoder type\n",
        "\n",
        "You can train the model using three different encoder types: a convolutional neural net (cnn), a recurrent neural net (rnn), or a bag of words (bow). You can select between the various types with the **--encoder_type** parameter supplying **cnn**, **rnn**, or **bow**. Different encoders have strengths and weakensses depending on the input / output characteristics of your dataset.\n",
        "\n",
        "For example: If the input context (here, the user history length) is long, cnn and rnn would be more suitable as they have better summarization ability with longer user histories.\n",
        "\n",
        "### Training time / size\n",
        "\n",
        "Another consideration is training time. Rnn generally requires the longer training times, followed by cnn, and finally bow with the shortest training times. Bag of words will also be a smaller sized model if space is a consideration.\n",
        "\n",
        "To start training, execute the following command. Please note that we are using a very small number of epochs (**num_epochs** parameter below) of 10 to speed up training time at the expense of model quality. Generating a high quality model often requires a much higher number. For this model, setting num_epochs to at least 100 should provide a model of sufficient quality. \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3gPKz5InxEbF"
      },
      "source": [
        "!python -m model.recommendation_model_launcher_keras \\\n",
        "  --run_mode \"train_and_eval\" \\\n",
        "  --encoder_type \"cnn\" \\\n",
        "  --training_data_filepattern \"data/examples/train_movielens_1m.tfrecord\" \\\n",
        "  --testing_data_filepattern \"data/examples/test_movielens_1m.tfrecord\" \\\n",
        "  --model_dir \"model/model_dir\" \\\n",
        "  --params_path \"model/sample_config.json\"\\\n",
        "  --batch_size 64 \\\n",
        "  --learning_rate 0.1 \\\n",
        "  --steps_per_epoch 1000 \\\n",
        "  --num_epochs 10 \\\n",
        "  --num_eval_steps 1000 \\\n",
        "  --gradient_clip_norm 1.0 \\\n",
        "  --max_history_length 10"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ObH_mcGcxS96"
      },
      "source": [
        "# Export model\n",
        "\n",
        "Now we export the trained model to a tflite file suitable for on-device inference on mobile devices.\n",
        "Note that here we use the latest checkpoint, number 10000 in the **checkpoint_path**. This results from num_epochs (10) x steps_per_epoch (1000). If you change either parameter in the previous training step, you should update this parameter to accordingly export the latest checkpoint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SH5r6AxHzGrS",
        "scrolled": true
      },
      "source": [
        "!python -m model.recommendation_model_launcher_keras \\\n",
        "  --run_mode \"export\" \\\n",
        "  --encoder_type \"cnn\" \\\n",
        "  --params_path \"model/sample_config.json\"\\\n",
        "  --model_dir \"model/model_dir\" \\\n",
        "  --checkpoint_path \"model/model_dir/ckpt-10000\" \\\n",
        "  --num_predictions 100"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qXMQ5D5JzSgv"
      },
      "source": [
        "# Model inference (Optional)\n",
        "\n",
        "You could verify your model's performance by running inference with test examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "og0qkYavz3Nt",
        "scrolled": true
      },
      "source": [
        "import tensorflow as tf\n",
        "import os\n",
        "import json\n",
        "\n",
        "# Use [0, 1, ... 9] as example input to represent 10 movies that user interacted with.\n",
        "#context = [1196, 1210, 2628]\n",
        "# context = tf.range(10)\n",
        "context = tf.constant([1196, 1210, 2628, 260, 480, 2571, 589, 1240, 1, 10])\n",
        "\n",
        "# Directory to exported TensorFlow Lite model.\n",
        "export_dir = \"model/model_dir/export\"\n",
        "tflite_model_path = os.path.join(export_dir, 'model.tflite')\n",
        "f = open(tflite_model_path, 'rb')\n",
        "interpreter = tf.lite.Interpreter(model_content=f.read())\n",
        "interpreter.allocate_tensors()\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "print(input_details)\n",
        "print(output_details)\n",
        "\n",
        "interpreter.set_tensor(input_details[0]['index'], context)\n",
        "interpreter.invoke()\n",
        "tflite_top_predictions_ids = interpreter.get_tensor(\n",
        "    output_details[0]['index'])\n",
        "tflite_top_prediction_scores = interpreter.get_tensor(\n",
        "    output_details[1]['index'])\n",
        "print(\"results >>>>>\")\n",
        "print(\"input >>>>>\")\n",
        "print(input_details[0])\n",
        "print(\"output >>>>>\")\n",
        "print(tflite_top_predictions_ids)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A_omMjoT035u"
      },
      "source": [
        "# Deploy model to the Firebase Console\n",
        "\n",
        "We now deploy the model to the Firebase Console. From there, it can be automatically downloaded to your user's devices with Firebase ML."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "awx9Q73aFyfS"
      },
      "source": [
        "Step 1. Initialize Firebase App Instance"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NxHAv1OaNo-X"
      },
      "source": [
        "import firebase_admin\n",
        "\n",
        "firebase_admin.initialize_app(options={'projectId': projectID, \n",
        "             'storageBucket': projectID + '.appspot.com' })"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7PxvbLcNNoxZ"
      },
      "source": [
        "Step 2. Upload the model file to Cloud Storage"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8WPegXrvdnth"
      },
      "source": [
        "from firebase_admin import ml\n",
        "\n",
        "# This uploads it to your bucket as recommendation.tflite\n",
        "source = ml.TFLiteGCSModelSource.from_saved_model(export_dir, 'model.tflite')\n",
        "print (source.gcs_tflite_uri)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cjUgFOCCF4Ro"
      },
      "source": [
        "Step 3. Deploy the model to Firebase"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EbPwOU1iF75O"
      },
      "source": [
        "# Create a Model Format\n",
        "model_format = ml.TFLiteFormat(model_source=source)\n",
        "\n",
        "# Create a Model object\n",
        "sdk_model_1 = ml.Model(display_name=\"recommendations\", model_format=model_format)\n",
        "\n",
        "# Make the Create API call to create the model in Firebase\n",
        "firebase_model_1 = ml.create_model(sdk_model_1)\n",
        "print(firebase_model_1.as_dict())\n",
        "\n",
        "# Publish the model\n",
        "model_id = firebase_model_1.model_id\n",
        "firebase_model_1 = ml.publish_model(model_id)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GvdDw6pXyDIT"
      },
      "source": [
        "# Return to the Firebase Console\n",
        "At this point, we have deployed the trained model to the Firebase console. You can go to Develop > Machine Learning > Custom to check it out!\n",
        "\n",
        "Note that for the purposes of this codelab, in order to have a quick training time, we intentionally chose suboptimal training parameters (as described in the model training step above) that sacrifice model quality. To get better results, please use the pre-trained model included in the Github code repo [here](https://github.com/FirebaseExtended/codelab-contentrecommendation-android/blob/master/recommendation_cnn_i10o100.tflite).\n",
        "To replace the model we just published:\n",
        "1. In the Firebase console, go to Develop > Machine Learning > Custom\n",
        "1. Select the settings dropdown under the model named \"recommendations\"\n",
        "1. Choose \"Replace model\" and upload the model file from the Github repo.\n",
        "\n",
        "Finally, please return to the [codelab](https://codelabs.developers.google.com/codelabs/contentrecommendation-android) and complete the last steps to see the app in action!"
      ]
    }
  ]
}