01_01_ingest_and_clustering.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\r\n",
    "\r\n",
    "from nbdev.showdoc import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 - Data Ingestion and Clustering Process\r\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Section 1 outlines the **ingest** and **clustering** processes as parts of _SC2 Training Grounds'_ computer model.\n",
    "\n",
    "> Note: this sections' notebooks and modules use sequence diagrams and annotated code to illustrate the development of one of the primary processes that make up _SC2 Training Grounds'_ computer model. I chose to develop this part of the document in the most specific level of detail because the base concept of _SC2 Training Grounds_ is to use player experience modelling to generate recommendations. Player experience modelling requires defining a way to abstract and transform the players' experience into quantifiable data, which I do in this module (see dissertation chapter 5.2).\n",
    "\n",
    "The following sequence diagram shows the proposed flow of orders and data between modules that would compose the application's ingest and clustering process. In particular, it shows how the library's main module (i.e. sc_training) can orchestrate the following steps:\n",
    "\n",
    "1. Load the replay files generated by StarCraft II.\n",
    "2. Call upon the _ingest_ sub-package to inventory the replays and store their relevant indicators into various collections within a document-based database using a local MongoDB client. \n",
    "3. Use the _profiler_ sub-package to build the player profiles from the replay data collections.\n",
    "4. Process the player profiles by game race to build each race classification clusters using the _cluster_ sub-package. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img alt=\"Clustering Process Sequence Diagram\" caption=\"Clustering Process Sequence Diagram\" id=\"clustering_seq_diagram\" height=\"900\" src=\"images/clustering_sequence.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on this proposed flow of orders, I first need to implement the ingest process for StarCraft II replay files. This ingestion process is essential to structure the input for the clustering process. Furthermore, it also determines the input of the application's classification modelling process, as shown in SC2 Training Grounds activity diagram (see Section 1's introduction).  \n",
    "\n",
    "In the sequence diagram, I show that the ingestion process is, on a top-level, composed of two main functions,  `inventory_replays` and `get_replay_indicators`. `inventory_replays` registers the replays so that a user can use their basic meta-data to reference specific replays and identify their players. Meanwhile, `get_replay_indicators` compiles all performance indicators of each player in each replay. Thus, the system can loop through various replays and extract and store the performance indicators for each player in each game. \n",
    "\n",
    "Having understood how the functions are supposed to interact, I implement them in the following modules. The most important outcome of this implementation is the structure of the players' performance indicators and how this data is indexed. In other words, this definition determines the information vectors that will become the inputs for the clustering and classification modelling process afterwards. \n",
    "\n",
    "> Note: To review the three-step process that resulted in the final version of the vectors, see the dissertation document Chapter 5.\n",
    "\n",
    "This implementation implies defining multiple functions that extract, organise and return different bits of information contained in a _SC2Replay_ file. Since different pieces relate to different categories of information, I group the functions in separate modules according to those categories. \n",
    "\n",
    "The following sequence diagram shows the interaction between these modules in the ingest process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img alt=\"Instion Sequence Diagram\" caption=\"Ingestion Sequence Diagram\" id=\"ingest_seq_diagram\" height=\"900\" src=\"images/ingest_sequence_diagram.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.7 64-bit ('sctraining_env': conda)",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}