Language identifier - SVM-v3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1e98460d",
   "metadata": {},
   "source": [
    "## <p style=\"text-align:center\" color=\"red\"><span style=\"color:red\">Language identifier with Support Vector Machine</span></p>\n",
    "\n",
    "\n",
    "<table align=\"center\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/labrijisaad/Language-Identifier-SVM\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "</table>\n",
    "\n",
    "- 🎯 In this notebook, I tried to write a script capable of identifying the language used in a text. \n",
    "- 🛠️ The supported languages are:  **`Swahili`**, **`Wolof`**, **`French`**, **`English`**, **`Arabic`** and **`Dyula`**.\n",
    "> The code in this notebook can easily be modified to add or modify the languages you want to detect: if you want to add other languages, just add a training dataset on the targeted languages, this dataset can be found by example on [HuggingFace Datesets](https://huggingface.co/datasets?sort=downloads).\n",
    "- 📫 Feel free to contact me if anything is wrong or if anything needs to be changed 😎!  **labrijisaad@gmail.com**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c69cd7aa",
   "metadata": {},
   "source": [
    "### Importing the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "77623b86",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip install datasets # to import the Hugging Face dataset (community dataset containing labeled phrases)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "701a6eef",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "import codecs\n",
    "import seaborn as sns \n",
    "import re\n",
    "import glob\n",
    "import unidecode\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "from sklearn import svm\n",
    "from datetime import datetime\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "tqdm.pandas()\n",
    "\n",
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ded3d7ec",
   "metadata": {},
   "source": [
    "### Importing the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e06e4e2a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using custom data configuration papluca--language-identification-088edc04c6573c74\n",
      "Reusing dataset csv (C:\\Users\\SAAD\\.cache\\huggingface\\datasets\\papluca___csv\\papluca--language-identification-088edc04c6573c74\\0.0.0\\652c3096f041ee27b04d2232d41f10547a8fecda3e284a79a0ec4053c916ef7a)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "148ebfaa11f84fe080d9d5519ffc5625",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>labels</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>58155</th>\n",
       "      <td>ru</td>\n",
       "      <td>Боюсь , что так , сказал Мистер Картер серьезно .</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46273</th>\n",
       "      <td>fr</td>\n",
       "      <td>Je vous ferais part de mon commentaire lorsque...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42982</th>\n",
       "      <td>ru</td>\n",
       "      <td>И еще в 1994 году правительство тюрьму в тюрьм...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      labels                                               text\n",
       "58155     ru  Боюсь , что так , сказал Мистер Картер серьезно .\n",
       "46273     fr  Je vous ferais part de mon commentaire lorsque...\n",
       "42982     ru  И еще в 1994 году правительство тюрьму в тюрьм..."
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = load_dataset(\"papluca/language-identification\") # Dataset downloaded from huggingface\n",
    "df_languages = pd.DataFrame.from_dict(dataset['train'])\n",
    "df_languages.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7bc20169",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>59218</th>\n",
       "      <td>baadhi ya wanachama wa simba na wadau wa mpira...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47485</th>\n",
       "      <td>the Plateau of the Cumberland, and here the la...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25226</th>\n",
       "      <td>katika mchezo huo wa ligi kuu bara yanga ilish...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence     lang\n",
       "59218  baadhi ya wanachama wa simba na wadau wa mpira...  swahili\n",
       "47485  the Plateau of the Cumberland, and here the la...  english\n",
       "25226  katika mchezo huo wa ligi kuu bara yanga ilish...  swahili"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file = pd.read_csv('corpus_id.csv') # Dataset that contains labelled sentences from :Wolof, Swahili, English, Arabic and French \n",
    "languages_file.sample(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c2116f18",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>40146</th>\n",
       "      <td>Ala ka kɔgɔji danfɛn barakamanw dan, ani jiraf...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54704</th>\n",
       "      <td>Ala ka o furancɛ wele ko «sankolo». Ayiwa, su ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37492</th>\n",
       "      <td>Minw bɛ jija koɲuman na, ka tandori ni bonya n...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence\n",
       "40146  Ala ka kɔgɔji danfɛn barakamanw dan, ani jiraf...\n",
       "54704  Ala ka o furancɛ wele ko «sankolo». Ayiwa, su ...\n",
       "37492  Minw bɛ jija koɲuman na, ka tandori ni bonya n..."
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dyula_list = [] \n",
    "with codecs.open('bible_dyula.txt', encoding='utf-8') as file:\n",
    "    for text in file:\n",
    "        dyula_list.append(text)\n",
    "\n",
    "df_dyula = pd.DataFrame (dyula_list, columns = ['sentence'])\n",
    "df_dyula.sample(3) # Dataset containing Dyula sentences extracted with selenium from the bible"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf9ec406",
   "metadata": {},
   "source": [
    "### Data exploration analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5adce60b",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "# Collecting Arabic phrases from huggingface dataset.\n",
    "df_ar = df_languages[df_languages.labels == \"ar\"]\n",
    "df_ar.rename(columns = {'labels':'lang', 'text':'sentence'}, inplace = True)\n",
    "df_ar[\"lang\"] = \"arabic\"\n",
    "\n",
    "# Taking a random sample of 14000 sentences (from the english Hugging face datset) to properly balance our training dataset.\n",
    "df_en = df_languages[df_languages.labels == \"en\"].sample(1000)\n",
    "df_en.rename(columns = {'labels':'lang', 'text':'sentence'}, inplace = True)\n",
    "df_en[\"lang\"] = \"english\"\n",
    "df_en"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ceacbd70",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>41416</th>\n",
       "      <td>Su kora, ka o dugu gbɛ. O kɛra lon naaninan ye.</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8316</th>\n",
       "      <td>Kɛrɛkɛden kanbelen minw tun bɛ Yohabu kɔ, olug...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11749</th>\n",
       "      <td>O bɛɛ tagara ɲɔgɔn lajɛn Zeruzalɛmu, Asa ta ma...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40008</th>\n",
       "      <td>Ne ko: aw minw b’a fɛ ka sariya* sira tagama, ...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47181</th>\n",
       "      <td>Dugukolo bɛganw bɛɛ, ani kɔnɔw bɛɛ, ani fɛnɲan...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37958</th>\n",
       "      <td>Ala ka baraka don o ra, ko o ye woro, ka caya ...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51673</th>\n",
       "      <td>A ka kelen sigi terefɛwagati kunna, ka kelen s...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22119</th>\n",
       "      <td></td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9478</th>\n",
       "      <td>Ala ka o furancɛ kɛ. Ji minw bɛ san fɛ, ani ji...</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12375</th>\n",
       "      <td>Su kora, ka o dugu gbɛ. O kɛra lon naaninan ye.</td>\n",
       "      <td>dyula</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>14000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence   lang\n",
       "41416   Su kora, ka o dugu gbɛ. O kɛra lon naaninan ye.   dyula\n",
       "8316   Kɛrɛkɛden kanbelen minw tun bɛ Yohabu kɔ, olug...  dyula\n",
       "11749  O bɛɛ tagara ɲɔgɔn lajɛn Zeruzalɛmu, Asa ta ma...  dyula\n",
       "40008  Ne ko: aw minw b’a fɛ ka sariya* sira tagama, ...  dyula\n",
       "47181  Dugukolo bɛganw bɛɛ, ani kɔnɔw bɛɛ, ani fɛnɲan...  dyula\n",
       "...                                                  ...    ...\n",
       "37958  Ala ka baraka don o ra, ko o ye woro, ka caya ...  dyula\n",
       "51673  A ka kelen sigi terefɛwagati kunna, ka kelen s...  dyula\n",
       "22119                                                     dyula\n",
       "9478   Ala ka o furancɛ kɛ. Ji minw bɛ san fɛ, ani ji...  dyula\n",
       "12375   Su kora, ka o dugu gbɛ. O kɛra lon naaninan ye.   dyula\n",
       "\n",
       "[14000 rows x 2 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_dyula[\"sentence\"] = df_dyula[\"sentence\"].str.replace(\"\\n\",\" \") \n",
    "df_dyula[\"sentence\"] = df_dyula[\"sentence\"].str.replace(\"\\t\",\" \") \n",
    "df_dyula[\"sentence\"] = df_dyula[\"sentence\"].str.replace(\"\\r\",\" \") \n",
    "df_dyula['lang'] = 'dyula'\n",
    "\n",
    "# Taking a random sample of 14000 sentences (from the Dyula datset) to properly balance our training dataset.\n",
    "df_dyula = df_dyula.sample(n = 14000) \n",
    "df_dyula"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "21af3eea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mu ne dina fi jaar buy dellu</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mimi ningekuwa naipenda yanga si ningekuwa nac...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Pencil skirts are nice until you have to go up...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Kingsport , TN #Transportation #Job: Rolloff ...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>daan sa doole</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023</th>\n",
       "      <td>just ok. inner plastic curtain keeps falling o...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49318</th>\n",
       "      <td>Great for daily devotions!</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46698</th>\n",
       "      <td>This is not the Jason shampoo with MSM. I look...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33705</th>\n",
       "      <td>It charges extremely slowly for some reason, a...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42648</th>\n",
       "      <td>Came quickly and are working great! I thought ...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>83195 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence     lang\n",
       "0                           mu ne dina fi jaar buy dellu    wolof\n",
       "1      mimi ningekuwa naipenda yanga si ningekuwa nac...  swahili\n",
       "2      Pencil skirts are nice until you have to go up...  english\n",
       "3      #Kingsport , TN #Transportation #Job: Rolloff ...  english\n",
       "4                                          daan sa doole    wolof\n",
       "...                                                  ...      ...\n",
       "2023   just ok. inner plastic curtain keeps falling o...  english\n",
       "49318                         Great for daily devotions!  english\n",
       "46698  This is not the Jason shampoo with MSM. I look...  english\n",
       "33705  It charges extremely slowly for some reason, a...  english\n",
       "42648  Came quickly and are working great! I thought ...  english\n",
       "\n",
       "[83195 rows x 2 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file = pd.concat([languages_file, df_dyula, df_ar, df_en]) \n",
    "languages_file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0cbb1f9",
   "metadata": {},
   "source": [
    "- **83195**  is the number of records in our dataset.\n",
    "- **2** is the number of columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "607af6b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 83195 entries, 0 to 42648\n",
      "Data columns (total 2 columns):\n",
      " #   Column    Non-Null Count  Dtype \n",
      "---  ------    --------------  ----- \n",
      " 0   sentence  83195 non-null  object\n",
      " 1   lang      83195 non-null  object\n",
      "dtypes: object(2)\n",
      "memory usage: 1.9+ MB\n"
     ]
    }
   ],
   "source": [
    "languages_file.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b3ae41b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sentence    0\n",
       "lang        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6943f72",
   "metadata": {},
   "source": [
    "- Good news so far, there are no missing values in our dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8ca69a77",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dealing with capitalized values :\n",
    "languages_file['sentence'] = languages_file['sentence'].str.lower()\n",
    "\n",
    "# Dealing with trailing spaces :\n",
    "languages_file['sentence'] = languages_file['sentence'].str.strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e9807040",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dealing with empty sentences :\n",
    "languages_file['sentence'].replace('', np.nan, inplace=True)\n",
    "languages_file.dropna(subset=['sentence'], inplace=True)\n",
    "\n",
    "# Dealing with the ponctuation :\n",
    "def remove_ponctuation(text):\n",
    "    return re.sub(r'[^\\w\\s]', '', text)\n",
    "\n",
    "languages_file['sentence'] = languages_file['sentence'].apply(lambda x: remove_ponctuation(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37eda943",
   "metadata": {},
   "source": [
    "> **`Here is the dataset that we are going to use to train our language identification model:`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "610e8e41",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mu ne dina fi jaar buy dellu</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mimi ningekuwa naipenda yanga si ningekuwa nac...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pencil skirts are nice until you have to go up...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>kingsport  tn transportation job rolloff drive...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>daan sa doole</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023</th>\n",
       "      <td>just ok inner plastic curtain keeps falling of...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49318</th>\n",
       "      <td>great for daily devotions</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46698</th>\n",
       "      <td>this is not the jason shampoo with msm i looke...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33705</th>\n",
       "      <td>it charges extremely slowly for some reason al...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42648</th>\n",
       "      <td>came quickly and are working great i thought i...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>83124 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence     lang\n",
       "0                           mu ne dina fi jaar buy dellu    wolof\n",
       "1      mimi ningekuwa naipenda yanga si ningekuwa nac...  swahili\n",
       "2      pencil skirts are nice until you have to go up...  english\n",
       "3      kingsport  tn transportation job rolloff drive...  english\n",
       "4                                          daan sa doole    wolof\n",
       "...                                                  ...      ...\n",
       "2023   just ok inner plastic curtain keeps falling of...  english\n",
       "49318                          great for daily devotions  english\n",
       "46698  this is not the jason shampoo with msm i looke...  english\n",
       "33705  it charges extremely slowly for some reason al...  english\n",
       "42648  came quickly and are working great i thought i...  english\n",
       "\n",
       "[83124 rows x 2 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "21689ddb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAhEElEQVR4nO3deZhcRb3/8feHhNVAQkguF5JAEHJRlqvCsAkqEn8QVEy8P9aLEjAaFwQRN1A0qOAVF7giAgaILCKrIEFBCJugEMKELWGTMSyZEMiQhLAvge/9o2qSw9CTmTmZ7p7JfF7P08+cU1WnTp3unv6eqnO6WhGBmZlZGavVuwFmZtZ7OYiYmVlpDiJmZlaag4iZmZXmIGJmZqU5iJiZWWkOIlZ3ks6U9P1uqmsTSS9K6pfXb5H0+e6oO9d3raTx3VVfF/Z7gqRnJT1dIW93Sc21bpMZQP96N8BWbZIeBzYElgJvAg8C5wOTI+ItgIj4Uhfq+nxE3NBemYh4Ehiwcq1etr/jgS0i4jOF+vfujrq72I5NgG8Am0bEglrv32xF3BOxWtgnItYFNgV+CnwHOKe7dyJpVT0p2gRY6ABiPZGDiNVMRCyJiKnAAcB4SdsASDpX0gl5eYikP0t6TtIiSbdJWk3SBaQP06vzcNW3JY2UFJImSHoSuKmQVgwom0uaIel5SVdJGpz39Y5hIEmPS/qYpDHAd4ED8v7uy/nLhsdyu46T9ISkBZLOlzQw57W2Y7ykJ/NQ1Pfae24kDczbt+T6jsv1fwyYBmyc23FuR8+zpGMk/UvSC5IelPTpQt6hkv4u6ReSFkt6TNLehfzNJN2at71B0m8k/b6j5ysv7yjpjvzazZd0mqQ1CmX3lPSIpCWSTpf0t+JQo6TPSXoot+s6SZvmdEk6JT/Hz0ua1fresfpzELGai4gZQDPwoQrZ38h5Q0nDYN9Nm8RngSdJvZoBEfGzwjYfAd4L7NXOLg8BPgdsRBpWO7UTbfwr8BPgkry/91Uodmh+fBR4N2kY7bQ2ZXYDtgRGAz+Q9N52dvlrYGCu5yO5zYflobu9gadyOw7tqO3Av0jP7UDgh8DvJW1UyN8JeAQYAvwMOEeSct4fgBnABsDxwGc7sb9WbwJfz/XuQjrmr0A6OQAuB47NdT8CfLB1Q0ljSa/1f5Fe+9uAi3L2nsCHgf/Ix7Q/sLAL7bIqchCxenkKGFwh/Q3Sh/2mEfFGRNwWHU/wdnxEvBQRr7STf0FEzI6Il4DvA/u3XnhfSQcDJ0fEnIh4kfQBeWCbXtAPI+KViLgPuA94RzDKbTkQODYiXoiIx4Ff0rUP8GUi4rKIeCoi3oqIS4BHgR0LRZ6IiLMi4k3gPNLzvWG+9rID8IOIeD0i/g5M7cJ+Z0bE9IhYmo/ht6SACPBx4IGIuCIiWgN58SaBLwH/ExEP5fyfAO/PvZE3gHWB9wDKZeZ39Xmx6nAQsXoZBiyqkP5zoAm4XtIcScd0oq65Xch/AliddLa8sjbO9RXr7k/qQbUqflC+TOWL/kNym9rWNaxMoyQdIunePKz0HLANbz/eZW2KiJfz4gDS8SwqpEHHz21xv/+RhyKflvQ8KRC07nfjYl35xKA4NLYp8KtCmxcBAoZFxE2kHt5vgAWSJktar7PtsupyELGak7QD6QPy723z8pn4NyLi3cCngKMljW7NbqfKjnoqIwrLm5DObJ8FXgLWKbSrH2kopbP1PkX68CvWvRR4poPt2no2t6ltXfO6WA/5zP0s4KvABhExCJhN+kDuyHxgsKR1CmnF566j5+sM4GFgVESsRxqeat3vfGB4YVsV10kB5osRMajwWDsibgeIiFMjYntgK9Kw1rc6cTxWAw4iVjOS1pP0SeBi4PcRMatCmU9K2iJ/yCwhjbO/lbOfIV0z6KrPSNoqfzj+CLg8D+X8E1hL0ickrQ4cB6xZ2O4ZYKSk9v5PLgK+ni9GD2D5NZSlXWlcbsulwImS1s2B4Gjg912pJ3sXKfi1AEg6jNQT6Uw7ngAageMlrSFpF2CfQpGOnq91geeBFyW9B/hyIe8vwLaSxuXhvsOBfy/knwkcK2nr3O6BkvbLyztI2inv8yXgVZa/J6zOHESsFq6W9ALpbPN7wMnAYe2UHQXcALwI3AGcHhE357z/AY7LQx7f7ML+LwDOJQ3jrAUcCeluMdKF37NJZ/0v8fYhlsvy34WS7q5Q75Rc963AY6QPtyO60K6iI/L+55B6aH/I9XdJRDxIup5yBykIbgv8owtVHEy6KL4QOAG4BHgt193R8/VN4L+BF0i9oUsK7XoW2I90IX8hqUfRWKj7SuAk4OI8FDabdEMBwHq5vsWkYb6FpGFP6wHkH6Uys/ZIugR4OCImdXO9q5EC0MGFkwTrhdwTMbNl8tDR5vk7KmOAscCfuqnuvSQNkrQmy6+XTO+Ouq1+VtVv+JpZOf8OXEH6Lkcz8OWIuKeb6t6FNEy3Bmn6m3EruC3begkPZ5mZWWkezjIzs9L63HDWkCFDYuTIkfVuhplZrzJz5sxnI2Jo2/Q+F0RGjhxJY2NjvZthZtarSHqiUrqHs8zMrDQHETMzK81BxMzMSnMQMTOz0hxEzMysNAcRMzMrzUHEzMxKcxAxM7PSHETMzKw0BxHr1YaN2ARJq8Rj2IhN6v10mnVZn5v2xFYtTzXP5YDf3l7vZnSLS774wXo3wazL3BMxM7PSHETMzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK81BxMzMSqtaEJE0RdICSbMr5H1DUkgaktcl6VRJTZLul7Rdoex4SY/mx/hC+vaSZuVtTpWkah2LmZlVVs2eyLnAmLaJkkYAewJPFpL3Bkblx0TgjFx2MDAJ2AnYEZgkaf28zRnAFwrbvWNfZmZWXVULIhFxK7CoQtYpwLeBKKSNBc6PZDowSNJGwF7AtIhYFBGLgWnAmJy3XkRMj4gAzgfGVetYzMyssppeE5E0FpgXEfe1yRoGzC2sN+e0FaU3V0hvb78TJTVKamxpaSndfk/2Z2b2djWbgFHSOsB3SUNZNRURk4HJAA0NDdFB8XZ5sj+zzhk2YhOeap7bccFeoN/qa/LmG6/VuxkrbePhI5g398mOC3ZRLWfx3RzYDLgvXwMfDtwtaUdgHjCiUHZ4TpsH7N4m/ZacPrxCeTPrAVa1E65V4ViqdeJYs+GsiJgVEf8WESMjYiRpCGq7iHgamAocku/S2hlYEhHzgeuAPSWtny+o7wlcl/Oel7RzvivrEOCqWh2LmZkl1bzF9yLgDmBLSc2SJqyg+DXAHKAJOAv4CkBELAJ+DNyVHz/KaeQyZ+dt/gVcW43jMDOz9lVtOCsiDuogf2RhOYDD2yk3BZhSIb0R2GblWmlmZivDv2zYV63WH38/08xWloNIX/XWUl8sNLOV5rmzzMysNAcRMzMrzUHEzMxKcxAxM7PSHETMzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK83Tnpj1FJ7PzHohBxGznmIVmc8MPKdZX+LhLDMzK81BxMzMSnMQMTOz0hxEzMysNAcRMzMrrWpBRNIUSQskzS6k/VzSw5Lul3SlpEGFvGMlNUl6RNJehfQxOa1J0jGF9M0k3ZnTL5G0RrWOxczMKqtmT+RcYEybtGnANhHxn8A/gWMBJG0FHAhsnbc5XVI/Sf2A3wB7A1sBB+WyACcBp0TEFsBiYEIVj8XMzCqoWhCJiFuBRW3Sro+IpXl1OjA8L48FLo6I1yLiMaAJ2DE/miJiTkS8DlwMjFX6RtYewOV5+/OAcdU6FjMzq6ye10Q+B1ybl4cBcwt5zTmtvfQNgOcKAak1vSJJEyU1SmpsaWnppuabmVldgoik7wFLgQtrsb+ImBwRDRHRMHTo0Frs0sysT6j5tCeSDgU+CYyOiMjJ84ARhWLDcxrtpC8EBknqn3sjxfJmZlYjNe2JSBoDfBv4VES8XMiaChwoaU1JmwGjgBnAXcCofCfWGqSL71Nz8LkZ2DdvPx64qlbHYWZmSTVv8b0IuAPYUlKzpAnAacC6wDRJ90o6EyAiHgAuBR4E/gocHhFv5l7GV4HrgIeAS3NZgO8AR0tqIl0jOadax2JmZpVVbTgrIg6qkNzuB31EnAicWCH9GuCaCulzSHdvmZlZnfgb62ZmVpqDiJmZleYgYmZmpTmImJlZaQ4iZmZWmoOImZmV5iBiZmalOYiYmVlpDiJmZlaag4iZmZXmIGJmZqU5iJiZWWkOImZmVpqDiJmZleYgYmZmpTmImJlZaQ4iZmZWmoOImZmVVs3fWJ8iaYGk2YW0wZKmSXo0/10/p0vSqZKaJN0vabvCNuNz+UcljS+kby9pVt7mVEmq1rGYmVll1eyJnAuMaZN2DHBjRIwCbszrAHsDo/JjInAGpKADTAJ2Iv2e+qTWwJPLfKGwXdt9mZlZlVUtiETErcCiNsljgfPy8nnAuEL6+ZFMBwZJ2gjYC5gWEYsiYjEwDRiT89aLiOkREcD5hbrMzKxGan1NZMOImJ+XnwY2zMvDgLmFcs05bUXpzRXSzcyshup2YT33IKIW+5I0UVKjpMaWlpZa7NLMrE+odRB5Jg9Fkf8uyOnzgBGFcsNz2orSh1dIrygiJkdEQ0Q0DB06dKUPwszMkloHkalA6x1W44GrCumH5Lu0dgaW5GGv64A9Ja2fL6jvCVyX856XtHO+K+uQQl1mZlYj/atVsaSLgN2BIZKaSXdZ/RS4VNIE4Alg/1z8GuDjQBPwMnAYQEQskvRj4K5c7kcR0Xqx/iukO8DWBq7NDzMzq6GqBZGIOKidrNEVygZweDv1TAGmVEhvBLZZmTaamdnK8TfWzcysNAcRMzMrzUHEzMxKcxAxM7PSHETMzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK81BxMzMSnMQMTOz0joVRCTt2pk0MzPrWzrbE/l1J9PMzKwPWeFU8JJ2AT4IDJV0dCFrPaBfNRtmZmY9X0e/J7IGMCCXW7eQ/jywb7UaZWZmvcMKg0hE/A34m6RzI+KJGrXJzMx6ic7+suGakiYDI4vbRMQe1WiUmZn1Dp0NIpcBZwJnA29WrzlmZtabdPburKURcUZEzIiIma2PsjuV9HVJD0iaLekiSWtJ2kzSnZKaJF0iaY1cds283pTzRxbqOTanPyJpr7LtMTOzcjobRK6W9BVJG0ka3Poos0NJw4AjgYaI2IZ0l9eBwEnAKRGxBbAYmJA3mQAszumn5HJI2ipvtzUwBjhdku8YMzOroc4GkfHAt4DbgZn50bgS++0PrC2pP7AOMB/YA7g8558HjMvLY/M6OX+0JOX0iyPitYh4DGgCdlyJNpmZWRd16ppIRGzWXTuMiHmSfgE8CbwCXE8KSs9FxNJcrBkYlpeHAXPztkslLQE2yOnTC1UXt3kbSROBiQCbbLJJdx2KmVmf16kgIumQSukRcX5XdyhpfVIvYjPgOdJF+zFdracrImIyMBmgoaEhqrkvM7O+pLN3Z+1QWF4LGA3cDXQ5iAAfAx6LiBYASVcAuwKDJPXPvZHhwLxcfh4wAmjOw18DgYWF9FbFbczMrAY6O5x1RHFd0iDg4pL7fBLYWdI6pOGs0aTrKzeTvgV/MekazFW5/NS8fkfOvykiQtJU4A+STgY2BkYBM0q2yczMSuhsT6Stl0jDUV0WEXdKupzUk1kK3EMaavoLcLGkE3LaOXmTc4ALJDUBi0h3ZBERD0i6FHgw13N4RPg7LGZmNdTZayJXA63XEvoB7wUuLbvTiJgETGqTPIcKd1dFxKvAfu3UcyJwYtl2mJnZyulsT+QXheWlwBMR0VyF9piZWS/Sqe+J5IkYHybN5Ls+8Ho1G2VmZr1DZ3/ZcH/SRev9gP2BOyV5Kngzsz6us8NZ3wN2iIgFAJKGAjew/BvmZmbWB3V22pPVWgNItrAL25qZ2Sqqsz2Rv0q6Drgorx8AXFOdJpmZWW/R0W+sbwFsGBHfkvRfwG456w7gwmo3zszMeraOeiL/CxwLEBFXAFcASNo25+1TxbaZmVkP19F1jQ0jYlbbxJw2siotMjOzXqOjIDJoBXlrd2M7zMysF+ooiDRK+kLbREmfJ/0GiJmZ9WEdXRM5CrhS0sEsDxoNwBrAp6vYLjMz6wVWGEQi4hngg5I+CmyTk/8SETdVvWVmZtbjdfb3RG4m/d6HmZnZMv7WuZmZleYgYmZmpTmImJlZaQ4iZmZWmoOImZmVVpcgImmQpMslPSzpIUm7SBosaZqkR/Pf9XNZSTpVUpOk+yVtV6hnfC7/qKTx9TgWM7O+rF49kV8Bf42I9wDvAx4CjgFujIhRwI15HWBvYFR+TATOAJA0GJgE7ATsCExqDTxmZlYbNQ8ikgYCHwbOAYiI1yPiOWAscF4udh4wLi+PBc6PZDowSNJGwF7AtIhYFBGLgWnAmJodiJmZ1aUnshnQAvxO0j2Szpb0LtKMwfNzmaeBDfPyMGBuYfvmnNZe+jtImiipUVJjS0tLNx6KmVnfVo8g0h/YDjgjIj4AvMTyoSsAIiKA6K4dRsTkiGiIiIahQ4d2V7VmZn1ePYJIM9AcEXfm9ctJQeWZPExF/tv6m+7zgBGF7YfntPbSzcysRmoeRCLiaWCupC1z0mjgQWAq0HqH1Xjgqrw8FTgk36W1M7AkD3tdB+wpaf18QX3PnGZmZjXSqQkYq+AI4EJJawBzgMNIAe1SSROAJ4D9c9lrgI8DTcDLuSwRsUjSj4G7crkfRcSi2h2CmZnVJYhExL2k3yVpa3SFsgEc3k49U4Ap3do4MzPrNH9j3czMSnMQMTOz0hxEzMysNAcRMzMrzUHEzMxKcxAxM7PSHETMzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK81BxMzMSnMQMTOz0hxEzMysNAcRMzMrzUHEzMxKcxAxM7PS6hZEJPWTdI+kP+f1zSTdKalJ0iX599eRtGZeb8r5Iwt1HJvTH5G0V50Oxcysz6pnT+RrwEOF9ZOAUyJiC2AxMCGnTwAW5/RTcjkkbQUcCGwNjAFOl9SvRm03MzPqFEQkDQc+AZyd1wXsAVyei5wHjMvLY/M6OX90Lj8WuDgiXouIx4AmYMeaHICZmQH164n8L/Bt4K28vgHwXEQszevNwLC8PAyYC5Dzl+Tyy9IrbPM2kiZKapTU2NLS0o2HYWbWt9U8iEj6JLAgImbWap8RMTkiGiKiYejQobXarZnZKq9/Hfa5K/ApSR8H1gLWA34FDJLUP/c2hgPzcvl5wAigWVJ/YCCwsJDeqriNmZnVQM17IhFxbEQMj4iRpAvjN0XEwcDNwL652Hjgqrw8Na+T82+KiMjpB+a7tzYDRgEzanQYZmZGfXoi7fkOcLGkE4B7gHNy+jnABZKagEWkwENEPCDpUuBBYClweES8Wftmm5n1XXUNIhFxC3BLXp5DhburIuJVYL92tj8ROLF6LTQzsxXxN9bNzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK81BxMzMSnMQMTOz0hxEzMysNAcRMzMrzUHEzMxKcxAxM7PSHETMzKw0BxEzMyvNQcTMzEpzEDEzs9IcRMzMrDQHETMzK81BxMzMSqt5EJE0QtLNkh6U9ICkr+X0wZKmSXo0/10/p0vSqZKaJN0vabtCXeNz+Uclja/1sZiZ9XX16IksBb4REVsBOwOHS9oKOAa4MSJGATfmdYC9gVH5MRE4A1LQASYBOwE7ApNaA4+ZmdVGzYNIRMyPiLvz8gvAQ8AwYCxwXi52HjAuL48Fzo9kOjBI0kbAXsC0iFgUEYuBacCY2h2JmZnV9ZqIpJHAB4A7gQ0jYn7OehrYMC8PA+YWNmvOae2lV9rPREmNkhpbWlq67wDMzPq4ugURSQOAPwJHRcTzxbyICCC6a18RMTkiGiKiYejQod1VrZlZn1eXICJpdVIAuTAirsjJz+RhKvLfBTl9HjCisPnwnNZeupmZ1Ug97s4ScA7wUEScXMiaCrTeYTUeuKqQfki+S2tnYEke9roO2FPS+vmC+p45zczMaqR/Hfa5K/BZYJake3Pad4GfApdKmgA8Aeyf864BPg40AS8DhwFExCJJPwbuyuV+FBGLanIEZmYG1CGIRMTfAbWTPbpC+QAOb6euKcCU7mudmZl1hb+xbmZmpTmImJlZaQ4iZmZWmoOImZmV5iBiZmalOYiYmVlpDiJmZlaag4iZmZXmIGJmZqU5iJiZWWkOImZmVpqDiJmZleYgYmZmpTmImJlZaQ4iZmZWmoOImZmV5iBiZmalOYiYmVlpDiJmZlZarw8iksZIekRSk6Rj6t0eM7O+pFcHEUn9gN8AewNbAQdJ2qq+rTIz6zt6dRABdgSaImJORLwOXAyMrXObzMz6DEVEvdtQmqR9gTER8fm8/llgp4j4aptyE4GJeXVL4JGaNrRrhgDP1rsR3cTH0vOsKscBPpZa2zQihrZN7F+PltRaREwGJte7HZ0hqTEiGurdju7gY+l5VpXjAB9LT9Hbh7PmASMK68NzmpmZ1UBvDyJ3AaMkbSZpDeBAYGqd22Rm1mf06uGsiFgq6avAdUA/YEpEPFDnZq2sXjHs1kk+lp5nVTkO8LH0CL36wrqZmdVXbx/OMjOzOnIQMTOz0hxEeiBJj0sa0kGZIyU9JOnCWrWrLEnn5u/0tE3fWNLleXl3SX/Oy5/qaVPYSBopaXZebpB06grKLjuWnkrSLZLecUtpPZ77ar+X23v/1Yqk4yV9s8R2h0o6rRpt6k69+sJ6H/cV4GMR0VzvhpQVEU8B7/jnjoip9OC77CKiEWisdzs6IqlfRLzZlW3q9Ny/470sqX9ELK1xO6wE90SqSNK3JB2Zl0+RdFNe3kPShZIOkjRL0mxJJ7VTx9E5f7ako3LamcC7gWslfb1KbX+XpL9Iui/v+zuSrsh5YyW9ImkNSWtJmpPTvyDprrzNHyWtU6jyw5JulzSn9ayweHbfZt/degYm6TOSZki6V9JvJfWT9KKkE3Nbp0vaMJfdPK/PknSCpBcr1FfsNX0k13uvpHskrZuLDZB0uaSH82ut7jqeQjv+JGmmpAfyrAzk4/qlpPuAXST9IL8msyVNbtOOz+Z2z5a0Y95+2XMvaUNJV+bn6D5JH6zCMRTfy0skXSDpH8AFkobm99Fd+bFr3uZ4SVNyb2pO6/9YzjtE0v25vRcUdvWO9181SfqepH9K+jtplox+ku4u5I9qXVdh5EGpl3tLhfr2kXRnfo/d0Pp+7REiwo8qPYCdgcvy8m3ADGB1YFJ+PAkMJfUIbwLG5bKPk6ZB2B6YBbwLGAA8AHygWKaKbf//wFmF9YHAnLz8C9J3dHYFPgJclNM3KJQ/ATgiL58LXEY6admKNN8ZwEhgdl7eHfhzXj4UOK2bjuO9wNXA6nn9dOAQIIB9ctrPgOPy8p+Bg/Lyl4AXO2jr1cCueXlAfi13B5aQvvy6GnAHsFsVXqPB+e/awGxgg3xc+7ctk5cvKBzzLa2vL/DhwrEte+6BS4Cj8nI/YGCV3mut7/fjgZnA2jn9D63PG7AJ8FBePh64HVgzb7eQ9H+1NfBP8v9F4fmp+P6r4v9O6//tOsB6QBPwTeBm4P25zE9Y/v/xeKHNDcAtFV6L9Vl+N+3ngV9W8xi68nBPpLpmAttLWg94jfRh0gB8CHiO9GZpidRtv5D0z1y0G3BlRLwUES8CV+Rta2EW8P8knSTpQxGxBPiXpPeSJr48Obf3Q6QACbCNpNskzQIOJv1Tt/pTRLwVEQ8CtTyLGk36p75L0r15/d3A66SAAel1GpmXdyF94ED6EOvIP4CT89nwoFg+BDMjIpoj4i3g3kL93enI3OOYTpq5YRTwJvDHQpmP5jPYWcAevP01uQggIm4F1pM0qE39ewBn5DJv5vdAtU2NiFfy8seA0/LrNjW3cUDO+0tEvBYRzwILSO+pPUgnbc/mNi8q1FvL99+HSP+3L0fE8ywfHjwbOExp9vED6Nz7q9Vw4Lr8On6Lt7+OdeUgUkUR8QbwGOmM4nbSh+1HgS1IZx89VkT8E9iOFExOkPQD4FbStPtvADeQgtxuLA8i5wJfjYhtgR8CaxWqfK2w3O1DOysg4LyIeH9+bBkRxwNvRD6tI33wlro+GBE/JZ0Zrg38Q9J7clbxeEvX3x5Ju5M+ZHeJiPcB95Ce71cjXweRtBap57Vvfk3O4u2vSdsvifWEL429VFheDdi58NoNyydT0PXnt17vv6I/kv5/PgnMjIiFOX0pyz+L16q0IfBrUq9kW+CLKyhXcw4i1XcbqSt7a17+EukffgbwEUlD8pnJQcDfKmw7TtI6kt4FfJrlH9hVJWlj4OWI+D3wc1JAuQ04CrgjIlpIwydbkoZSANYF5ktandQT6QluBPaV9G8AkgZL2nQF5aeThvIgTaOzQpI2j4hZEXESaYjvPR1t000GAosj4uUcuHauUKb1g+bZfAbf9lrAAQCSdgOWVOhp3Ah8OZfpJ2lgt7W+c64HjmhdkfT+DsrfBOwnaYNcfnD1mrZCt5L+b9fO18j2AYiIV0mza5wB/K5Q/nFSbxmWv/faGsjyeQHHd3eDV4aDSPXdBmxE+uB9BngVuC0i5gPHkMZJ7yOdmVxV3DAi7iad3c8A7gTOjoh7atTubYEZeShhEukax52koYBbc5n7gVmFM/rv5zL/AB6uUTtXKA9fHAdcL+l+YBrp9WjPUcDRuewWpGsbK3JUvjB9P6mHdu3Kt7pT/gr0l/QQ8FNS8HubiHiO1PuYTfrwuqtNkVcl3QOcCUyosI+vkYbDZpGG/Gr9g29HAg35QvmDpBOwdkWa8uhE4G95mO/kGrSxUjvuJl1Puo/0fig+7xcCb5ECZKsfAr+S1EjqVVVyPHCZpJn0sCnjPe2JWYHSHWWvRERIOpB0kd0/dGbdQun7IgMj4vv1bkt38fdEzN5ue9LFXJFufvhcfZtjqwpJVwKbk24AWGW4J2JmZqX5moiZmZXmIGJmZqU5iJiZWWkOImZVpgrzb5mtKhxEzMysNAcRsxqRNEDSjZLuVpoleGxOH6n0expnKc3Ie72ktXPeDvnLdvdK+rkqzHpsVk8OIma18yrw6YjYjjSH2i/z91EgTZ74m4jYmvT9lNbpL34HfDEi3k/732Y2qxsHEbPaEfCTPEXKDcAwls8o+1hE3JuXZwIj86y660bEHTm9K7O+mtWEv7FuVjsHk34/ZvuIeEPS4yyfJLHtrLRr17htZqW4J2JWOwOBBTmAfBRY0WzCrRMoviBpp5zU4azCZrXmnohZ7VwIXJ1nxW2kczMdTwDOkvQW6acCavHDUGad5rmzzHowSQNaf4hJ0jHARhHxtTo3y2wZ90TMerZPSDqW9L/6BOlXMs16DPdEzMysNF9YNzOz0hxEzMysNAcRMzMrzUHEzMxKcxAxM7PS/g8a9zxVGZUNYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the distribution of the languages column :\n",
    "sns.histplot(languages_file['lang'])\n",
    "plt.title('Distribution of languages')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4e2864ad",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lang</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>arabic</th>\n",
       "      <td>12700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dyula</th>\n",
       "      <td>13929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>english</th>\n",
       "      <td>14974</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>french</th>\n",
       "      <td>13928</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>swahili</th>\n",
       "      <td>13569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wolof</th>\n",
       "      <td>14024</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         sentence\n",
       "lang             \n",
       "arabic      12700\n",
       "dyula       13929\n",
       "english     14974\n",
       "french      13928\n",
       "swahili     13569\n",
       "wolof       14024"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file.groupby('lang').count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a254c886",
   "metadata": {},
   "source": [
    "> **`Our dataset is quite balanced and has enough data, normally we expect good results after training the model`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7c325b4d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mu ne dina fi jaar buy dellu</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mimi ningekuwa naipenda yanga si ningekuwa nac...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pencil skirts are nice until you have to go up...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>kingsport  tn transportation job rolloff drive...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>daan sa doole</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023</th>\n",
       "      <td>just ok inner plastic curtain keeps falling of...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49318</th>\n",
       "      <td>great for daily devotions</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46698</th>\n",
       "      <td>this is not the jason shampoo with msm i looke...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33705</th>\n",
       "      <td>it charges extremely slowly for some reason al...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42648</th>\n",
       "      <td>came quickly and are working great i thought i...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>83124 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence     lang\n",
       "0                           mu ne dina fi jaar buy dellu    wolof\n",
       "1      mimi ningekuwa naipenda yanga si ningekuwa nac...  swahili\n",
       "2      pencil skirts are nice until you have to go up...  english\n",
       "3      kingsport  tn transportation job rolloff drive...  english\n",
       "4                                          daan sa doole    wolof\n",
       "...                                                  ...      ...\n",
       "2023   just ok inner plastic curtain keeps falling of...  english\n",
       "49318                          great for daily devotions  english\n",
       "46698  this is not the jason shampoo with msm i looke...  english\n",
       "33705  it charges extremely slowly for some reason al...  english\n",
       "42648  came quickly and are working great i thought i...  english\n",
       "\n",
       "[83124 rows x 2 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60552aea",
   "metadata": {},
   "source": [
    "- The lang column is divided into 5 languages: **``swahili``**, **``wolof``**, **``french``**, **``english``**, **``arabic``** and **``dyula``**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "be661657",
   "metadata": {},
   "outputs": [],
   "source": [
    "# WORD-COUNT\n",
    "dataframe = languages_file.copy(deep=True)\n",
    "dataframe['sentence']= dataframe['sentence'].apply(lambda x: len(str(x).split()))\n",
    "\n",
    "dict_languages_wordcount = {}\n",
    "languages_names = list(dataframe.lang.unique())\n",
    "\n",
    "for language in languages_names:\n",
    "    dict_languages_wordcount[language] = dataframe[dataframe['lang'].str.contains(language)].sentence.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "970aaa03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "wolof  :  6.573302909298346\n",
      "swahili  :  25.032426855332005\n",
      "english  :  10.996260184319487\n",
      "arabic  :  12.337322834645668\n",
      "french  :  13.728388856978748\n",
      "dyula  :  27.564074951539954\n"
     ]
    }
   ],
   "source": [
    "for key, value in dict_languages_wordcount.items():\n",
    "    print(key, ' : ', value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "580f13e1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABI8AAAEVCAYAAABpMMWAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABXQ0lEQVR4nO3de7xcVX3//9fbhJvcQjBQIMREiVKgcvEUQm0pJRICWkNbRNBKIrGpX4OXaiugPhoC4hd/PyvCT8VGEwnWJiBeSBXBlItWa4CA3AIiIYBJBBJICCAKBj+/P9bnkJ2TmXPJmTNnJuf9fDzmcfZee82eteesPbNn7bU+SxGBmZmZmZmZmZlZLa8Y7AKYmZmZmZmZmVnrcuORmZmZmZmZmZnV5cYjMzMzMzMzMzOry41HZmZmZmZmZmZWlxuPzMzMzMzMzMysLjcemZmZmZmZmZlZXW48MjMzs5Yh6WZJ7x3scpiZmZnZJm48MjMzMxsgko6VtGqwy2FmZmbWH248MjMzs6ZTMWjXIZKGD9Zrm5mZmbUbNx6ZmZlZtyS9R9J/VdYflPTNyvpKSYfl8p9Juk3Shvz7Z5V8N0u6UNJPgeeB10g6XtIvMv8XAHVTjvMkXS3pSknPSrpD0qGV7ftK+paktZIelvTBGs/9D0nPANNq7P8kSfflvldL+ufKtrdKulPS05L+V9IbKtsekfTPku7O47hS0o6SdgZ+AOwr6bl87CvpFZLOkfSQpKckXSVpZO5rrKSQNFXSryQ9KekTldcaJunj+dxnJd0uaf/cdqCkxZLWSXpA0qk9/W/NzMzMesONR2ZmZtaTHwF/kY0e+wLbA0cDSHoNsAtwdzaAfB+4FNgT+BzwfUl7Vvb1bmAGsCuwAfg28EngVcBDwJt6KMsU4JvASOA/ge9K2i57Mf0XcBewHzAR+LCkE7o892pgBPCNGvueC/xjROwKHALcmMd4ODAP+Mc8rn8HFknaofLcU4HJwDjgDcC0iPgNcCLw64jYJR+/Bj4AnAz8JbAvsB74Ypey/Dnw+jyOf5X0x5n+EeB04CRgN+BM4PlsqFqc78lewGnAlyQd1N2baWZmZtYbbjwyMzOzbkXECuBZ4DDgGOB64NeSDqQ0gPxPRPwBeAvwYER8PSI2RsQC4BfAX1d2d3lELIuIjZSGlWURcXVE/B74PPB4D8W5vZL/c8COwATgT4FREXF+RLyYZf4KpRGl088i4rsR8YeI+G2Nff8eOEjSbhGxPiLuyPQZwL9HxC0R8VJEzAdeyNftdGlE/Doi1lEasQ7r5hjeB3wiIlZFxAvAecApXYbSzY6I30bEXZQGsc4eVu8FPhkRD0RxV0Q8BbwVeCQivpbv/c+BbwFv76YcZmZmZr3i8f5mZmbWGz8CjgUOyOWnKQ1HR+c6lF40j3Z53qOUnkCdVlaW962uR0RIqm6vpZr/DxmMel8gKMPDnq7kHQb8T53XruXvKL2gLpJ0N3BORPwMeDUwVdIHKnm3z9ftVG30er7Ltq5eDXxH0h8qaS8Be3ezv11yeX9KD61a+zyqy/EPB77eTTnMzMzMesWNR2ZmZtYbP6L0IBoHfJrSePQuSuPRFzLPrymNGFVjgOsq61FZfozSGAKUINrV9Tqq+V8BjM7X3Qg8HBHju3ludLONiLgNmCJpO+As4Kp8vZXAhRFxYQ9l6+1rrgTOjIifdt0gaWwP+1sJvBa4t0b6jyLi+K0oo5mZmVm3PGzNzMzMeuNHwF8BO0XEKkqPnsmUGEA/zzzXAq+T9E5JwyW9AzgI+F6dfX4fOFjS3+aQrQ8Cf9RDOd5Yyf9hyvCxJcCtwLOSzpa0UwaWPkTSn/bm4CRtL+ldknbPIXHPAJ09g74CvE/SUSp2lvQWSbv2YtdPAHtK2r2S9mXgQkmvztceJWlKb8oJfBW4QNL4LMsbMqbU9yjv/bszBtR2kv60EivJzMzMbKu58cjMzMx6FBG/BJ4jh4FFxDPACuCnEfFSpnXG3vko8BTwMeCtEfFknX0+SYnJc1HmHw9s0Runi2uAd1CCTL8b+NuI+H2W4a2UWEMPA09SGlp2r7OfWt4NPJKzsb2P0rOKiFgK/AOlh9V6YDk1ZmurJSJ+ASwAVuRMbfsClwCLgB9KepbS+HVUL8v4OUqPqB9SGrjmUhr0ngUmUWI8/Zoy7O0zwA519mNmZmbWa4rotge3mZmZWUuQdB5wQET8/WCXxczMzGwocc8jMzMzMzMzMzOry41HZmZmZmZmZmZWlxuPmkTSWEmRAT5rbf+4pK/WyivpB5KmNrO8Zo0g6WZJ7+1l3r+RtFLSc5IOH+iymVVJer2kOyU9K+mDTXzdaZJ+0qzXa3cRcZ6HrLU2ScdKWlVZXybp2B6e0+01klm7kPSIpDfX2fYXkh5odpnM+kLS5ZI+1YD9nCfpPxpRJmsd/pJuERHx6W62ndjMsuQ0wQ8D20XExma+tg1pnwXOiohrmvWCki4HVkXEJ5v1mtayPgbcFBGHDXZBms1xhGwgRcTBg12G3sgGrv+IiNGDXBTbRkXE/wCvH+xygK/1beiRNA14b0T8+WCXpZ2555GZtYpXA8sGuxA2ZHVb/yQNa2JZzMysxbhnnJkNdW48qpB0tqTVOWzhAUkTJe0o6beSXpV5PiFpo6Tdcv0CSZ/P5bdI+rmkZ3L4zXk1XuZdkn4l6UlJn6i8dt2ufd0N/ZF0pKSl+ZpPSPpcZdsESf+bUwPfVe02nvu8QNJP83h/2HmMwI/z79M5hOjofM6Zku6XtF7S9ZJeXdlfSHqfpAfz9b4oSZXt/5DPfVbSfZKOyPR9JX1L0lpJD1eHi3R3bDawJL1H0n9V1h+U9M3K+kpJh+Xyn0m6TdKG/Ptndfb5CkmflPSopDWSrpC0u6QdJD0HDAPukvRQjedK0sX5vGck3SPpkNy2g6TP5nn1hKQvS9optx0raZWkj+ZzH5P0ntw2gzIN98eynv9XpndXJ8+TdFWW/VmV4Rgdle37S/p2PvcpSV+obKt5/nR3bNYckm4E/gr4QtaF16l0275M0rWSfgP81UDVjdz+2awbD0uq29tUNb6nMv0Vks6R9FDu/ypJI3Nb55Cgqery/SNpMvBx4B157Hdl+u6S5uY5s1rSp5QNaMqhdvXKLGmkpK9J+nVu/25l21tVhgc+rfL99Iaejs0GRj/r8xEq1zvPSvqmpCtVZ5iDKsN41PP3es1rpBr7PEnlWuLZrDP/XNnWXR17RNI/S7pb5TvrSpXrvJ2BHwD75nnwXL4/W3Ve5fZhKiEJHspy3i5p/9x2oKTFktZlXT+1N8dmjVf5/3Zen/5NZds0levkiyU9BZwn6bWSbsz68KSkb0ga0WW3f5r7Wp+fhTvm/roO6ez2e6GSz9f6NmAkHS7pjvy/XQnsWNl2r6S/rqxvl/X+8K71Obd3N2zzm5Iez8/eH0uq2ys1z70VWaaHJb2rsq3PdVTSHwNfBo7O+v505t+q3xC5fSdJ/6byu2aDynVR53O7Oy/rHltbiAg/IqB0I10J7JvrY4HX5vKPgb/L5R8CDwEnVrb9TS4fC/wJpVHuDcATwMmV/QXwFWAn4FDgBeCPc/t5lO7S1bzDc/1mSje7WuX+GfDuXN4FmJDL+wFPASdleY7P9VGVfT4EvC7LczNwUa3Xz7QpwHLgjynDHT8J/G9lewDfA0YAY4C1wOTc9nZgNfCngIADKHf5XwHcDvwrsD3wGmAFcEJ3x+ZHU86H1wBP5/9oX+BRyvCuzm3rc9vIXH531ovTc33PrnUXODPr0Gvy//lt4Otd6tABdcpzQtaVEVmH/hjYJ7ddDCzKsuwK/Bfwfyvn5EbgfGC7PB+eB/bI7ZcDn6q8Tk918jzgd7mfYcD/BZbktmHAXVmenSlfvn/e0/nT3bH50dQ6/3JdrdSNDcCbsl68coDqxjTg98A/ZL7/A/waUI0ydvc99SFgCTAa2AH4d2BBJV+vvn8qr/Wd3MfOwF7ArcA/9qbMwPeBK4E9KOfdX2b64cAa4Kh83lTgkSxv3WPzY0Dqe38+67anfCd8KP+/fwu8SH6WUj53V1Ve6xHgzblc75ql2zpao/yPAX+Ry3sAR/RUxypluZXyvTYSuB94X61yN+C8+hfgnqzbyu17Us6plcB7KN8HhwNPAgd1d2x+DNi58PasD68A3gH8hk3XF9Mo1xAfyP/VTpRr2OOzPoyi/A74fJf6fi+wf9axn9Y6N+jme6FGGX2t78dA1f/Oz/N/onyen0L5fu+ssx8DruxSR+7pWp+71P/Oz/vzqFxbUH4H7JrnzueBO+uUaWfgGeD1ub4PcHAD6ug04CddXqs/vyG+mOfUfpTz+c/y2Oqel90dW7s8Br0ArfLID7k1wJsp43+r2y4ALs1K+jjlYuIiygf9b8kfyjX2+Xng4lwemxV6dGX7rcBpufzyCUbfGo9+DMwGXtUl/WwqP8wz7XpgamWfn6xsez9wXa3Xz7QfANMr66/IE+jVuR5UvvSAq4BzKq/7oRplPwr4VZe0c4GvdXdsfjTtnFgJHAGcBszJ+nog5YJ3UeZ5N3Brl+f9DJhWqWedjUc3AO+v5Hs95Quqs54H9RuPjgN+CUwAXlFJF+VC77WVtKOBh3P52DxHq3V5DZsuvC5n88ajnurkecB/V7YdBPy28rprq69VyVf3/Kl3bH40vb6/XFcrdeOKJtSNacDyyvor81z4oxp5u/ueuh+YWFnfp/P8og/fP7m+N+VH8E6VtNMpMaG6LXO+7h/Ii6suZbwMuKBL2gPAX3Z3bH4MSH3vT30+hvIjUZXtP6F3jUf1rlm6raM1yv8r4B+B3Xpbxypl+fvKtv8H+HKtcmdaf86rB4ApNcr+DuB/uqT9OzCru2Pzo2nnxp2d/7f8rPtVD/lPBn5eWX+EbJDM9ZOAh7rWMbr5XqjxGr7W92Og6vsxdLlhBfwvmz7P9wWe7fw8Aq4GPpbLtT4zH6FO41GXfCOyPu1eY9vOlBvYf0flOiT6X0enUWk8oh+/IfJ1fwscWqP8dc/L7o6tXR4etpYiYjnwYUpFXyNpoaR9c/OPKBXoCMpdpMWUi90JlAvopwAkHSXppuyWuQF4H/AqNvd4Zfl5Sit7f0yn3FH4hcqQobdm+quBt2d3uaeze96fUy58tqYsrwYuqexrHeWk268X+9ufcuej1j737VLGj1N+uHR3bNYcnfX+mFy+mVLv/zLXYVOvpKpH2bxeUCfvo5QL8L1r5N1MRNwIfIHSyr9G0hyVoaOjyB4hlTp0XaZ3eio2DwbZXV3vqU7ClvV8R5U4CPsDj0btwJN1z59ujs0G38rK8kDVjc2eFxHP5+IWdbSH76lXA9+plO1+4KUeytfdebAd8Fhlf/9O6YHUU5n3B9ZFxPo6+/1ol/dwf0pvo+6OzRqvP/V5X2B15BVxqp4r3enpe723dfTvKD/KH5X0I+WQG7qpY1vxGp3729rzqrtrn6O6lPFdlMbX7o7NBoCkM7RpmOPTwCFsfu2+skv+vfPzabWkZ4D/YMtr/epzHmXz+tepp++FKl/r20Cp9Xn+8rV6RPya0nvu71SGZ54IfKOvL6IyjPcilSGiz1AamWDLc4eI+A2lkf19lOuQ70s6MDf3p4521Z/fEK+idCKpV+drnpc9HFtbcONRRUT8Z5QI7K+mtFx+Jjf9L6WXxN8AP4qI+yhd4U5i049ogP+kdH3bPyJ2p4ytFAMoIh6MiNMpF/WfAa5WGbu/ktLqOaLy2DkiLurNbmukraQMWajub6eI+N9e7G8l8No66Q932eeuEXFSD8dmzdHZePQXufwjtmw8+jXlfKkaQ7kr3VXXvGMo3UGf6E1hIuLSiHgj5Q746yhDAp6ktPwfXKlDu0dEbxtlu9b1butkD1YCY1Q7oGa350+dY7PB1/XH8UDUjb4VqP731ErKcOpq+XaMiFrn4ha7rVHeFyh3gjv3tVv0btaslcBIbRkDpHPbhV3K+MqIWNDDsVnj9ac+PwbsJ6l6fbN/b160Ud/rEXFbREzJ/XyXcncZeqhjPe22Rlp/zqvurn1+1GWfu0TE/+nh2KzBVGKlfAU4izKKYARlyFm1bnetF5/OtD+JiN2Av2fLa/3q+TCGcv3TVa+/F3ytbwOo1uf5mC555lPq+duBn1U+/35DaXwBXp5YZBS1vZMy5OzNwO6Unm9Q53dyRFwfEcdTGkJ/QTlPoX91tGud789viCcpQ7vr1fm652U3x9YW3HiUJL1e0nGSdqBUht9Sut533lW9HZjJph/N/0tpNaw2Hu1KueP6O0lHUk6UgS7330saFRF/oHSDI8v9H8BfSzohW3t3VAn81ZspaNfmPl5TSfsycK4yuJlKMNW397KYXwX+WdIbVRyQX9i3As+qBEndKct5iKQ/7eHYrDl+RAkivFNErAL+B5hMidnw88xzLfA6Se+UNFzSOygNIN+rsb8FwD9JGidpF8oF2JXRi7tukv5UpWffdpQvq98Bf8i68RXgYkl7Zd79JJ3Qy2N8gs3rebd1sge3Ur6EL5K0c55zb8ptdc+fesfWy/Jb8wxU3ei17r6nKHXsQm0KxD5K0pRe7voJYKykVwBExGOU+H7/Jmk3laDBr5X0lz3tKJ/7A+BLkvZQCa55TG7+CvC+rO/K9+Itknbt4dis8fpTn39G6X1zVn7uTwGO7M2LNuJ7XdL2kt4lafeI+D0lfkTnPurWsV7s+glgT0m7V9L6c159FbhA0vgsyxsk7Un5fnydpHfn+bFdfg/8cQ/HZo23M+UH5Vook4VQeh51Z1fgOWCDpP2ofbNnpqTRKsHVP0GJAddVr78XujlvfK1v/fUzyo3cD+Zn0d+y5ef5dymjbz4EXFFJ/yWlR+pb8hr2k5SYP7XsSrkp9RSlwenT9Qqk0rtvikoj4guU8616rbO1dfQJYLSk7QH68xsinzsP+JxKQPhhko7Oa5i652UPx9YW3Hi0yQ6UOEZPUrq77UUZk9vpR5Ru/LdW1ndl02wFUMYSny/pWUpguGbcLZoMLFOZreoSylj730bESkoL78cpXxArKV9wPf7Ps7HsQuCnKt3tJkTEdyh3BBaqdDe8l9J1sUcR8c3c339Sxs1+FxgZES8BbwUOAx6mvPdfpbRI1z223rym9V9E/JLyofY/uf4MJcjhT/N/R5Qhm28FPkr5QvgY8NaIeLLGLucBX6ecMw9TfiB+oJfF2Y3yAb+e0p32KeD/zW1nU4LnLcm6+d+UnoK9MRc4KOv5d3tRJ+vK5/41JXbLr4BVlK6p9HD+dHds1iIGqm70UXffU5dQer7+ML+DllBiTfRG50yKT0m6I5fPoATSvI9SN69m86EQ3Xk3JS7MLyjxAT4MEBFLKUG2v5D7XE6JQdDTsVmD9bM+v0gJkj2d8mPv7ykNIi/04qUb9b3+buCR/Dx9H2XYV091rFsR8QvKTY4V+Z2wL/07rz5HuQ78IaURaC7lZsyzwCRKPMFfU+r7Z9j0o6vmsVnjRRlJ8G+UH9BPUCa9+WkPT5tN+SG9gTI5wLdr5PlPyv99BWVYyxYzEfbxe8HX+jYgKp/n0yhDwN5Blzqd/49vAeOq2yJiA+W371cpIw5+Q6nHtVxBucZdTbmuWNJNsV4BfITy+biOMuKhs2fmVtdR4EZgGfC4pM7fKf35DfHPlHA2t2U5P0OJXdrdeVn32NpF58woZmZmZmZ9JukWSuDprw12WczMrLEk/Svwuoj4+8Euiw0u9zwyMzMzs16T9JeS/iiHrU0F3kAJNGpmZtuQHH45nTLzsg1xbjwyMzMzs754PXAXZdjaR4FTMt6VmVlbytg0t0q6S9IySbMz/XJJD6vMynenpMMyXZIulbRc0t2Sjqjsa6qkB/MxdZAOqd8k/QNl2NUPIuLHPeW3bZ+HrZmZmZmZmdmQJUnAzhHxXAaA/gklSPT7gO9FxNVd8p9Eid15EiUW2iURcVT21FkKdFACst8OvDEi1jfvaMwGhnsemZmZmZmZ2ZAVxXO5ul0+uutlMQW4Ip+3BBghaR/gBGBxRKzLBqPFlMDgZm1v+GAXoDuvetWrYuzYsYNdDBuibr/99icjYtRgvb7rvw0W130bqlz3bShz/behqrPuSxpG6Sl0APDFiLhF0v8BLsyg0TcA50TEC8B+lCFdnVZlWr30zUiaAcwA2Hnnnd944IEHDsCRmXWvr5/7Ld14NHbsWJYuXTrYxbAhStKjg/n6rv82WFz3bahy3behzPXfhqrOuh8RLwGHSRoBfEfSIcC5wOPA9pSg0WcD5/f3NSNiTu6Pjo6OcN23wdDXz/1eDVuT9E8ZOOxeSQsyoNg4SbdkkLArJW2feXfI9eW5fWxlP+dm+gOSTujTkZmZmZmZmZkNoIh4GrgJmBwRj+XQtBeArwFHZrbVwP6Vp43OtHrpZm2vx8YjSfsBHwQ6IuIQYBhwGvAZ4OKIOABYT5nCj/y7PtMvznxIOiifdzBl3OeXsmugmZmZmZmZ2aCQNCp7HCFpJ+B44BcZx6gzoPbJwL35lEXAGTnr2gRgQ846eT0wSdIekvYAJmWaWdvrbcDs4cBOkoYDrwQeA44DOqPOz6ecTFCCh83P5auBiXmyTQEWRsQLEfEwsJxNLbdmZmZmZmZmg2Ef4CZJdwO3UYJefw/4hqR7gHuAVwGfyvzXAisov2m/ArwfICLWARfkPm4Dzs80s7bXY8yjiFgt6bPAr4DfAj+kBBJ7OiI2ZrZqILCXg4RFxEZJG4A9M31JZdc1g4eZmZmZmZmZNUtE3A0cXiP9uDr5A5hZZ9s8YF5DC2jWAnozbG0PSq+hccC+wM4M4HSDkmZIWipp6dq1awfqZczMzMzMzMzMrBd6M2ztzcDDEbE2In4PfBt4EzAih7HB5oHAXg4Sltt3B56il8HDImJORHRERMeoUYM2W6iZmZmZmZmZmdG7xqNfARMkvTJjF00E7qNEoD8l80wFrsnlRblObr8xu/UtAk7L2djGAeOBWxtzGGZmZmZmZmZmNhB6E/PoFklXA3cAG4GfA3OA7wMLJX0q0+bmU+YCX5e0HFhHmWGNiFgm6SpKw9NGYGZEvNTg4zEzMzMzMzMzswbqsfEIICJmAbO6JK+gxmxpEfE74O119nMhcGEfy2hmZmZmZmZmZoOkV41H2xrN1lY/N2ZFA0titnW0lVU4XH1tSNvaz36fONYaZmv2Vj1vVnS9/2fWZnzhY9Zns2dv5XfGLH9nWG29iXlkZmZmZmZmZmZDlBuPzMzMzMwMSf8kaZmkeyUtkLSjpHGSbpG0XNKVkrbPvDvk+vLcPrayn3Mz/QFJJwzaAZmZWcO48cjMzMzMzLYDPgh0RMQhwDDKxDefAS6OiAOA9cD0zD8dWJ/pF2c+JB2UzzsYmAx8SdKwZh6ImZk1nhuPzMzMbMg588wz2WuvvTjkkEO22Cbpo5JC0qtyXZIuzZ4Ud0s6opJ3qqQH8zG1kv5GSffkcy6VtjZoi1lTDQd2kjQceCXwGHAccHVunw+cnMtTcp3cPjHr+RRgYUS8EBEPA8upMcmOmZm1lyEZMNvMzMyGtmnTpnHWWWdxxhlnbJYuaX9gEvCrSvKJwPh8HAVcBhwlaSRlNtoOSmT12yUtioj1mecfgFuAayk9MH4woAdl1j+/Bz5Lqfu/BX4I3A48HREbM88qYL9c3g9YCRARGyVtAPbM9CWV/VafsxlJM4AZAGPGjGnksZjZVtraQNvgYNvbOvc8MjMzsyHnmGOOYeTIkbU2XQx8jM2n2ZsCXBHFEmCEpH2AE4DFEbEuG4wWA5Nz224RsSQiAriCTb01zFrVMEpdHwfsC+xMafQcMBExJyI6IqJj1KhRA/lSZmbWT248MjMzMytGAKsj4q4u6S/3sEidPSm6S19VI92sle0GPBwRayPi98C3gTdRGks7RyuMBlbn8mpgf4DcvjvwVDW9xnPMzKxNufHIzMzMhrznn38e4I+Af23m60qaIWmppKVr165t5kubdfUiMEHSKzN20UTgPuAm4JTMMxW4JpcX5Tq5/cbsabcIOC1nYxtHGe55a5OOwczMBogbj8zMzGzIe+ihhwB2AO6S9Ailt8Qdkv6I+j0puksfXSN9Cx62Yy3kN5TA13cA91B+J8wBzgY+Imk5JabR3Mw/F9gz0z8CnAMQEcuAqygNT9cBMyPipSYeh5mZDQAHzDYzM7Mh70/+5E8A7oqIDoBsQOqIiCclLQLOkrSQEjB7Q0Q8Jul64NOS9sjdTALOjYh1kp6RNIESMPsM4P9r8iGZ9VlEzKIEga9aQY3Z0iLid8Db6+znQuDChhfQzMwGjRuPzMzMbMg5/fTTufnmm3nyyScZPXp0T7PLXAucRJly/HngPQDZSHQBcFvmOz8i1uXy+4HLgZ0os6x5pjUzMzNrW248MjMzsyFnwYIFW6S9973vfXk5IsZWlgOYWWs/ETEPmFcjfSlwSP9LamZmZjb4HPPIhrwzzzyTvfbai0MO2XSN/y//8i8AB0u6W9J3JI3o3CbpXEnLJT0g6YRK+uRMWy7pnEr6OEm3ZPqVkrZvzpGZmZmZmZmZ9Z8bj2zImzZtGtddd91maccffzzAsoh4A/BL4FwASQcBpwEHA5OBL0kaJmkY8EXgROAg4PTMC/AZ4OKIOABYD0wf8IMyMzMzMzMzaxA3HtmQd8wxxzBy5MjN0iZNmlRdXcKmWXOmAAsj4oWIeJgS/+LIfCyPiBUR8SKwEJiSU90eR5m9BGA+cPIAHYqZmZmZmZlZw7nxyKxnZ7Ip0Ol+wMrKtlWZVi99T+DpiNjYJb0mSTMkLZW0dO3atQ0qvpmZmZmZmdnWc+ORWTckfQLYCHyjGa8XEXMioiMiOkaNGtWMlzQzMzMzG9Ik7SjpVkl3SVomaXam14xdKmmHXF+e28dW9lUzPqpZu+ux8UjS6yXdWXk8I+nDkkZKWizpwfy7R+aXpEvzhLlb0hGVfU3N/A9KmjqQB2bWAHsCbwXelTPtAKwG9q/kGZ1p9dKfAkZIGt4l3czMzMzMWsMLwHERcShwGDBZ0gTqxy6dDqzP9IszX934qM08ELOB0mPjUUQ8EBGHRcRhwBuB54HvAOcAN0TEeOCGXIcSMHh8PmYAlwFIGgnMAo6ixIeZ1dngZNZqMoD2HwFvi4jnK5sWAafl3YZxlHp+K3AbMD7vTmxP+dJYlI1ONwGn5POnAtc06TDMzMzMzKwHUTyXq9vlI6gfu3RKrpPbJ2as03rxUc3aXl+HrU0EHoqIR9n8hOl6Il2RJ+ASSq+LfYATgMURsS4i1gOLKa2xZoPq9NNP5+ijj+aBBx5g9OjRzJ07l7POOgtgGLA4e9x9GSAilgFXAfcB1wEzI+KljGl0FnA9cD9wVeYFOBv4iKTllN5Mc5t6gGZmZmZm1q2cQflOYA3lt+pD1I9d+nK809y+gXKdXy8OatfXcpxTazvDe86ymdOABbm8d0Q8lsuPA3vncl8DCm9G0gxKjyXGjBnTx+KZ9d2CBQu2SJs+fTqS7o6Ijq7bIuJC4MIa6dcC19ZIX4HvOJiZmZmZtayIeAk4TNIIykibAwfwteYAcwA6Ojqih+xmLaHXPY9yKM7bgG923ZZDcxpS6R0w2MzMzMzMzAZDRDxNCTtxNPVjl74c7zS3706JdVovDqpZ2+vLsLUTgTsi4olcfyKHo5F/12R6XwMKm5mZmZmZmQ0KSaOyxxGSdgKOp4SiqBe7dFGuk9tvzA4V9eKjmrW9vjQenc6mIWuw+QnT9UQ6I2ddmwBsyOFt1wOTJO2RgbInZZqZmZmZmZnZYNkHuEnS3ZSJcBZHxPeoH7t0LrBnpn+EnDyqXnzUph6J2QDpVcwjSTtTWl//sZJ8EXCVpOnAo8CpmX4tcBIlsvzzwHsAImKdpAsoJyPA+RGxrt9HYGZmZmZmZraVIuJu4PAa6TVjl0bE74C319lXzfioZu2uV41HEfEbSktrNe0pyuxrXfMGMLPOfuYB8/peTDMzMzMzMzMzGwx9GbZmZmZmZmbbph0k3Vl5PCPpw5JGSlos6cH8uwdAhqi4VNJySXdLOqJzR5KmZv4HJU2t/5JmZtYu3HhkZmY1SRom6eeSvpfr4yTdkj8UrsxZOMmgkFdm+i2Sxlb2cW6mPyDphEE6FDMz69kLEXFYRBwGvJESfuI7lFguN0TEeOCGXIcymc74fMwALgOQNBKYBRxFGe4zq7PByczM2pcbj8zMrJ4PUWYa6fQZ4OKIOABYD0zP9OnA+ky/OPMh6SDgNOBgYDLwJUnDmlR2MzPbehOBhyLiUWAKMD/T5wMn5/IU4IoollCmNN8HOIESbHhdRKwHFlO+A8zMrI258cjMzLYgaTTwFuCruS7gOODqzNL1B0TnD4urgYmZfwqwMCJeiIiHKRMpbBF00szMWs5pbJplee+cORngcWDvXN4PWFl5zqpMq5e+BUkzJC2VtHTt2rWNKruZmQ0ANx6ZmVktnwc+Bvwh1/cEno6Ijble/THw8g+F3L4h8/f6B4SZmbWGHJL8NuCbXbflxDjRqNeKiDkR0RERHaNGjWrUbs3MbAC48cjMzLraHVgTEbc36wV999nMrGWcCNwREU/k+hM5HI38uybTVwP7V543OtPqpZuZWRtz45GZmXW1C/A2SY8ACynD1S6hxLMYnnmqPwZe/qGQ23cHnqIPPyB899ma7cwzz2SvvfbikEMOqSaPlvSLnDnqO5JGdG6oF/xd0uRMWy7pnEp6zQDzZm3gdDYNWQNYBHTOmDYVuKaSfkbOujYB2JDD264HJknaIwNlT8o0MzNrY248MjOzrlZHxOiIGEuJe3FjRLwLuAk4JfN0/QHR+cPilMwfmX5azsY2jjIjz61NOgazbk2bNo3rrruua/IzwCER8Qbgl8C5UD/4ewaA/yKlp8ZBwOmZF+oHmDdrWZJ2Bo4Hvl1Jvgg4XtKDwJtzHeBaYAUlnt1XgPcDRMQ64ALgtnycn2lmZtbGhvecxczMDICzgYWSPgX8HJib6XOBr0taDqyj/MgmIpZJugq4D9gIzIyIl5pfbLMtHXPMMTzyyCNdk5+pxPVawqbG0peDvwMPZ13vDP6+PCJWAEhaCEyRdD+lx947M8984DxyKnOzVhURv6HErKumPUWZfa1r3gBm1tnPPGDeQJTRzMwGhxuPzMysroi4Gbg5l1dQY7a0iPgd8PY6z78QuHDgSmg2YM4Erszl/SiNSZ2qwd+7BoU/iu4DzG9G0gxgBsCYMWMaUnAzMzOzRvOwNTMzM7MKSZ+g9Jb7xkC/luN9mZmZWTtwzyMzMzOzJGka8FZgYg7Lge6Dv9dKf4oMMJ+9jzzblJmZmbU19zwyMzMzK3YDPga8LSKer6TXC/5+GzA+Z1bbnhLva1E2OtULMG9mZmbWdtx4ZGZmZkPO6aefztFHH80DDzzA6NGjmTt3LsAYYFdgsaQ7JX0ZSvB3oDP4+3Vk8PfsVXQWZRry+4GrMi+UAPMfyeDae7IpwLyZmZlZ2/GwNTMzMxtyFixYsEXae9/73nsjoqNW/nrB3yPiWsqU5V3TawaYNzMzM2tH7nlkZmZmZmZmZmZ1ufHIzMzMzMzMzMzq6lXjkaQRkq6W9AtJ90s6WtJISYslPZh/98i8knSppOWS7pZ0RGU/UzP/g5KmDtRBmfXFmWeeyV577cUhhxzyctq6deugBEHtd/2W9EZJ9+RzLpWkJh6emZmZmZmZWb/0tufRJcB1EXEgcCglKOQ5wA0RMR64IdcBTqTMQjIemAFcBiBpJDALOIoSA2BW5w9ys8E0bdo0rrvuus3SLrroIoBnG1S/LwP+ofK8yQN4OGZmZmZmZmYN1WPjkaTdgWPIWUIi4sWIeBqYAszPbPOBk3N5CnBFFEuAEZL2AU4AFkfEuohYDyzGP6KtBRxzzDGMHDlys7RrrrkG4Klc3er6ndt2i4glOXXzFZV9mZmZmZmZmbW83sy2Ng5YC3xN0qHA7cCHgL0j4rHM8ziwdy7vB6ysPH9VptVLN2s5TzzxBMDvc7U/9Xu/XO6abmZmZmZmts2YPXv2Vj1v1qxZDS6JDYTeDFsbDhwBXBYRhwO/YdMQHgCyR0U0okCSZkhaKmnp2rVrG7FLs35pZP3uieu/mZmZmVlTbSfpJkn3SVom6UMAks6TtFrSnfk4qfMJks7NeKYPSDqhkj4505ZLOqfWi5m1q940Hq0CVkXELbl+NaUx6YkckkP+XZPbVwP7V54/OtPqpW8mIuZEREdEdIwaNaovx2LWMHvvvTfAdtDv+r06l7um1+T6b2ZmZmbWdB+NiIOACcBMSQdl+sURcVg+rgXIbacBB1PCsHxJ0jBJw4AvUmKkHgScXtmPWdvrsfEoIh4HVkp6fSZNBO4DFgGdM0pNBa7J5UXAGTkr1QRgQw5vux6YJGmPDCQ8KdPMWs7b3vY2gD1zdavrd257RtKEnGXtjMq+zMzMzMxscP0+Iu4AiIhnKZNDdRdmYgqwMCJeiIiHgeWUCXOOBJZHxIqIeBFYmHnNtgm9iXkE8AHgG5K2B1YA76E0PF0laTrwKHBq5r0WOIlyEj2feYmIdZIuAG7LfOdHxLqGHIVZP5x++uncfPPNPPnkk4wePZrZs2dzzjnn8NnPfnY3SQ/S//r9fuByYCfgB/kwMzMzM7MWImkscDhwC/Am4CxJZwBLKb2T1lMalpZUnlaNado1BupRA13mbcHWxkoCx0tqpl41HkXEnUBHjU0Ta+QNYGad/cwD5vWhfGYDbsGCBfU2/TIiNqv3W1O/I2IpcEg/i2lmZmY2oCSNAL5KuW4J4EzgAeBKYCzwCHBqRKzPHtWXUG6qPQ9M6+y9IWkq8Mnc7aciYj5mLU7SLsC3gA9HxDOSLgMuoJwLFwD/RjknGvFaM4AZAGPGjGnELs0GXG9iHpmZmZmZ2bbvEuC6iDgQOJQyfOcc4IaIGA/cwKaJc04ExudjBnAZgKSRwCxKj4sjgVk5pN+sZUnajtJw9I2I+DZARDwRES9FxB+Ar1DqM/Qzxm/u23FOre248cjMzMzMzIYBxwBzASLixYh4mhKzpbPn0Hzg5FyeAlwRxRJgRE4ycgKwOCLW5RCfxZSgwmatbC5wf0R8rjOhc3Ko9DfAvbm8CDhN0g6SxlEaUG+lhK8YL2lchns5LfOabRN6G/PIzMzMzMy2XdsDa4GvSToUuB34ELB3TgAC8Diwdy7vx5bxXfbrJn0LHrpjLWIX4N3APZLuzLSPU2ZLO4wybO0R4B8BImKZpKsok0htBGZGxEsAks6iTKQzDJgXEcuadxhmA8uNR2ZmZmZmJuAI4AMRcYukS9g0RA0osR8lRaNeMCLmAHMAOjo6GrZfsz56LiJUI/3aek+IiAuBC2ukX9vd88zamYetmZmZdUv9eJiZtY0XgVURcUuuX01pTHqic/hO/l2T2/sd98XMzNqHG4/MzMzMzGwjsFLS63N9ImVYziJgaqZNBa7J5UXAGSomABtyeNv1wCRJe2Sg7EmZZmZmbczD1szMzMzMDOADwDcy2O8K4D2Um81XSZoOPAqcmnmvBU4ClgPPZ14iYp2kCyjBgwHOj4h1zTsEMzMbCG48MjMzMzMzIuJOoKPGpok18gYws85+5gHzGlo4MzMbVB62ZmZmZkPOmWeeyV577cUhhxxSTR4mabGkB/PvHgA5LOdSScsl3S3piM4nSJqa+R+UNLWS/kZJ9+RzLpXkIFhmZmbWttx4ZGZmZkPOtGnTuO6667om7wPcEBHjgRvYNNPUicD4fMwALgOQNBKYBRwFHAnM6mxwyjz/UHne5AE7GDMzM7MB5sYjMzMzG3KOOeYYRo4c2TV5BDA/l+cDJ+fyFOCKKJYAI3LWqROAxRGxLiLWA4uBybltt4hYkkN7rqjsy8zMzKztuPHIzMzMrBies0UBPA7sncv7ASsr+VZlWnfpq2qkm5mZmbUlNx6ZmZmZdZE9hmKgX0fSDElLJS1du3btQL+cmZmZ2VZx45GZmZlZsTGHnJF/12T6amD/Sr7RmdZd+uga6VuIiDkR0RERHaNGjWrIQZiZmZk1mhuPzMzMzIqngc4Z06YC1+TyIuCMnHVtArAhh7ddD0yStEcGyp4EXJ/bnpE0IWdZO6OyLzMzM7O2M3ywC2BmZmbWbKeffjo333wzTz75JKNHj2b27NkAjwHHS5oOPAqcmtmvBU4ClgPPA+8BiIh1ki4Abst850fEulx+P3A5sBPwg3yYmZmZtSU3HpmZmdmQs2DBgi3S3vve974UERO7pmf8o5m19hMR84B5NdKXAof0v6RmZmZmg8/D1szMzMzMzMzMrK5eNR5JekTSPZLulLQ000ZKWizpwfy7R6ZL0qWSlku6W9IRlf1MzfwPSppa7/XMzMzMzMzMzKw19KXn0V9FxGER0ZHr5wA3RMR44IZcBzgRGJ+PGcBlUBqbgFnAUcCRwKzOBiczM2stknaUdKukuyQtkzQ708dJuiVvEFwpaftM3yHXl+f2sZV9nZvpD0g6YZAOyczMzMzMtlJ/hq1NAebn8nzg5Er6FVEsAUbkdLcnAIsjYl1ErAcWA5P78fpmZjZwXgCOi4hDgcOAyTnL1GeAiyPiAGA9MD3zTwfWZ/rFmQ9JBwGnAQdTPvO/JGlYMw/EzMzMzMz6p7eNRwH8UNLtkmZk2t45FS3A48DeubwfsLLy3FWZVi99M5JmSFoqaenatWt7WTwzM2ukvAHwXK5ul48AjgOuzvSuNw46byhcDUzMKcqnAAsj4oWIeJgyW9WRA38EZmZmZmbWKL1tPPrziDiCMiRtpqRjqhtzFpJoRIEiYk5EdEREx6hRoxqxSzMz2wqShkm6E1hD6S36EPB0RGzMLNWbAC/fIMjtG4A98Y0DMzMzM7O216vGo4hYnX/XAN+h3DV+IoejkX/XZPbVwP6Vp4/OtHrpZi1L0j9lvJd7JS3IODCO+WJDQkS8FBGHUT6vjwQOHMDX8o0DMzMzM7MW1WPjkaSdJe3auQxMAu4FFgGdM6ZNBa7J5UXAGTnr2gRgQw5vux6YJGmPDJQ9KdPMWtV2wAeBjog4BBhGid3imC82pETE08BNwNGUOHbDc1P1JsDLNwhy++7AU/jGgZlZ2/AMy2ZmVk9veh7tDfxE0l3ArcD3I+I64CLgeEkPAm/OdYBrgRWUuBZfAd4PEBHrgAuA2/JxfqaZtbLhwE75Y/iVwGM45osNAZJGSRqRyzsBxwP3UxqRTslsXW8cdP5AOAW4MYc0LwJOy5554ygzcd7alIMwM7Ot4RmWzcxsC8N7yhARK4BDa6Q/BUyskR7AzDr7mgfM63sxzQbF74HPAr8Cfgv8ELidXsZ8kVSN+bKkst+aMV/MWsw+wPzsJfcK4KqI+J6k+4CFkj4F/ByYm/nnAl+XtBxYR+ltR0Qsk3QVcB+wEZgZES81+VjMzGzrTQGOzeX5wM3A2VRmWAaWSOqcYflYcoZlAEmdMywvaG6xzXpP0v7AFZSOEwHMiYhLsjH0SmAs8AhwakSszxvElwAnAc8D0yLijtzXVOCTuetPRcR8zLYBPTYemQ1hwygXRuOAp4FvUi5+BkzOZjgDYMyYMQP5Umbdioi7gcNrpK+gRs+5iPgd8PY6+7oQuLDRZTQzs4brnGE5gH+PiDkM0AzLZi1mI/DRiLgjQ7bcng2f0yg97y6SdA6l593ZbN7z7ihKz7ujKj3vOijn0+2SFkXE+qYfkVmD9Xa2NbOhaDfg4YhYGxG/B74NvIkBjPnioMFmZmY2iJo2wzJ4pk1rHRHxWGfPoYh4ljJUfz82D0vRNVzFFVEsofw+2Ac4gex5lw1GnT3vzNqeG4/M6nsRmCDpldk1dSJl6I1jvpiZmdk2p9kzLPummbWinDH5cOAW3PPO7GVuPDKr7zeUwNd3APdQzpc5lK6qH8nYLnuyecyXPTP9I2RAyYhYBnTGfLkOx3wxMzOz1vMKz7BsQ52kXYBvAR+OiGeq2xrZ88697qwdOeaRWTciYhZl3HKVY76YDRoNdgHMzLZVwykzLHcu/2dEXCfpNuAqSdOBR4FTM/+1lGDByykBg98DZYZlSZ0zLINnWLY2IWk7SsPRNyLi25n8hKR9IuKxPvS8O7ZL+s1dXyvjic0B6OjoaNhQULOB5MYjMzMzMzN7MSI6uiZ6hmUbCjJExVzg/oj4XGVTZ8+7i9iy591ZkhZSAmZvyAam64FPZ687KD3vzm3GMZgNNDcemZmZmZmZ2VD2JuDdwD2S7sy0j1Majdzzzgw3HpmZmZmZmdkQFhE/of7YePe8M8MBs83MzMzMzMzMrBtuPDIzMzMzMzMzs7rceGRmZmZWIemfJC2TdK+kBZJ2lDRO0i2Slku6UtL2mXeHXF+e28dW9nNupj8g6YRBOyAzMzOzfnLjkZmZmVmStB/wQaAjIg4BhgGnAZ8BLo6IA4D1wPR8ynRgfaZfnPmQdFA+72BgMvAlScOaeSxmZmZmjeLGIzMzM7PNDQd2kjQceCXwGHAccHVunw+cnMtTcp3cPjGnfJ4CLIyIFyLiYcqMPEc2p/hmZmZmjeXGIzMzM7MUEauBzwK/ojQabQBuB56OiI2ZbRWwXy7vB6zM527M/HtW02s852WSZkhaKmnp2rVrG39AZmZmZg3gxiMzMzOzJGkPSq+hccC+wM6UYWcDIiLmRERHRHSMGjVqoF7GzMzMrF/ceGRmZma2yZuBhyNibUT8Hvg28CZgRA5jAxgNrM7l1cD+ALl9d+CpanqN55iZmZm1FTcemZmZmW3yK2CCpFdm7KKJwH3ATcApmWcqcE0uL8p1cvuNERGZflrOxjYOGA/c2qRjMDMzM2uoXjceSRom6eeSvpfrnrLWzMzMtikRcQsl8PUdwD2Ua6U5wNnARyQtp8Q0mptPmQvsmekfAc7J/SwDrqI0PF0HzIyIl5p4KGZmZmYNM7znLC/7EHA/sFuud05Zu1DSlylT1V5GZcpaSZ1T276jy5S1+wL/Lel1vpAyMzOzVhIRs4BZXZJXUGO2tIj4HfD2Ovu5ELiw4QU0MzMza7Je9TySNBp4C/DVXBeestbMzMzMzMzMbJvX22Frnwc+Bvwh1/dkgKasNTMzMzMzMzOz1tFj45GktwJrIuL2JpQHSTMkLZW0dO3atc14STMzMzMzMzMzq6M3PY/eBLxN0iPAQspwtUsYoClrI2JORHRERMeoUaP6fEBmZmatQ1v5MDMzMzNrHT02HkXEuRExOiLGUgJe3xgR78JT1pqZmZmZbVM8w7KZmdXS25hHtXjKWjMzMzOzbUvnDMudOmdYPgBYT5lZGSozLAMXZz66zLA8GfiSpGFNKruZmQ2Q4T1n2SQibgZuzuVBn7JWs92138zMzMysESozLF9IuUncOcPyOzPLfOA84DLKTMrnZfrVwBe6zrAMPJw3lI8EftakwzAzswHQn55HZmZmZma27fg8TZxh2RPlmJm1DzcemZmZmZnZ7jRxhmXwRDlmZu3EjUdm3ZA0QtLVkn4h6X5JR0saKWmxpAfz7x6ZV5IuzQCRd0s6orKfqZn/QUlT67+imZmZ2aDYhSbOsGxmZu3FjUdm3bsEuC4iDgQOpQSQPAe4ISLGAzfkOsCJlFkExwMzKPEAkDQSmAUcRRnzP6uzwcnMzMysRaz2DMtmZlaPG4/M6hsGHEPOJBgRL0bE05RAkPMzz3zg5FyeAlwRxRLKnbp9gBOAxRGxLiLWA4sps4+YmZmZtbr2nWFZ2vqHDSmS5klaI+neStp5klZLujMfJ1W2nZujDR6QdEIlfXKmLZd0TtfXMWtnfZptzWyI2R5YC3xN0qHA7ZTpa/eOiMcyz+PA3rlcL0BknwJHUnotMWbMmMYchZmZmVkftNoMy2ZNcDnwBeCKLukXR8RnqwmSDqL0zjsY2Bf4b0mvy81fBI6nXO/fJmlRRNw3kAU3axb3PDKrT8ARwGURcTjwGzYNUQMgu2dHo17QgSPNzMzMzJorIn4MrOtl9inAwoh4ISIeBpZTGliPBJZHxIqIeJESO2zKgBTYbBC48cisvheBVRFxS65fTWlMeiKHo5F/1+T2egEiHTjSzMzMzKz9nJUT4cyrxCzt92gDs3bkxiOz+jYCKyW9PtcnUsbvVwNEdg0ceUbOujYB2JDD264HJknaI790JmWamZmZmZm1psuA1wKHAY8B/9aoHUuaIWmppKVr165t1G7NBpRjHpl17wPANyRtD6wA3kNpdL1K0nTgUeDUzHstcBKl6+rzmZeIWCfpAuC2zHd+RPS2W6yZmZmZmTVZRDzRuSzpK8D3crW7UQW9Gm0QEXOAOQAdHR0NC4FhNpDceGTWjYi4E+iosWlijbwBzKyzn3nAvIYWzmwASdqfEjRyb0pcrzkRcYmkkcCVwFjgEeDUiFgvScAllAbU54FpEXFH7msq8Mnc9aciYj5mZmZmLUzSPpVJcv4G6JyJbRHwn5I+RwmYPR64lRIvdbykcZRGo9OAd/anDLNnz+7P080ayo1HZmZWy0bgoxFxh6RdgdslLQamATdExEU5Be05lGmcT6RcPI0HjqJ09T4qG5tmURphI/ezKCLWN/2IzMzMzGqQtAA4FniVpFWUa5djJR1GuX55BPhHgIhYJukqSjiLjcDMiHgp93MWJTzFMGBeRCxr7pGYDRw3HpmZ2RbyTttjufyspPspQR+nUC6uAOZTpnI+O9OvyB54SySNyIDyxwKLO4dqZgPUZGBB0w7GrI8kjQC+ChxC+dFwJvAA7nVnZrZNiojTayTP7Sb/hcCFNdKvpYSyMNvmOGC2mZl1S9JY4HDgFmDvShfuxynD2qCfM484cKS1mEuA6yLiQOBQ4H5KL7sbImI8cEOuw+a97mZQet1R6XV3FGX65lmVmXrMzMzM2oobj8zMrC5JuwDfAj4cEc9Ut2Uvo4YEeYyIORHREREdo0aNasQuzbaKpN2BY8g7zhHxYkQ8Teld19lzaD5wci6/3OsuIpYAnb3uTiB73eUwzc5ed2ZmZmZtx41HZmZWk6TtKA1H34iIb2fyE/nDmPy7JtPrzTzS3YwkZq1oHLAW+Jqkn0v6qqSdca87MzMzG8LceGRmZlvIOC5zgfsj4nOVTYuAqbk8Fbimkn6GignAhvyhfT0wSdIeOWRnUqaZtarhwBHAZRFxOPAbNg1RA9zrzszMzIYeNx6ZmVktbwLeDRwn6c58nARcBBwv6UHgzbkOJTjkCmA58BXg/QAZKPsC4LZ8nN8ZPNusRa0CVkXELbl+NaUxyb3uzMzMbMjqcbY1STsCPwZ2yPxXR8QsSeOAhcCewO3AuyPiRUk7AFcAbwSeAt4REY/kvs4FpgMvAR+MCN99NjNrQRHxE0B1Nk+skT+AmXX2NQ+Y17jSmQ2ciHhc0kpJr4+IByj1/b58TKU0mHbtdXeWpIWU4NgbIuIxSdcDn64EyZ4EnNvMYzEzMzNrlB4bj4AXgOMi4rmMf/ETST8APgJcHBELJX2Z0ih0Wf5dHxEHSDoN+AzwDkkHAacBBwP7Av8t6XUR8dIAHJeZmZnZ1voA8A1J21N61L2H0lv7KknTgUeBUzPvtcBJlF53z2deImKdpM5ed+Bed2ZmZtbGemw8yrvJz+XqdvkI4DjgnZk+HziP0ng0JZehdPX+QsbOmAIsjIgXgIclLadMXfuzRhyImZmZWSNExJ1AR41N7nVnZmZmQ1KvYh5JGibpTsr4/sXAQ8DTEbExs1RnEHl5dpHcvoEytM2zjpiZmZmZmZmZtZleNR5FxEsRcRgl2OORwIEDVSDPOmJmZmZm1nSSdKukuyQtkzQ7E8dJukXScklX5nBOJO2Q68tz+9jKjs7N9AcknTBIx2NmZg3Up9nWIuJp4CbgaGCEpM5hb9UZRF6eXSS3704JnO1ZR8zMzMzMWlNQ4pweChwGTJY0gRK/9OKIOABYT4lvCpU4p8DFmY8ucU4nA1+SNKyZB2JmZo3XY+ORpFGSRuTyTsDxwP2URqRTMlvXWUem5vIpwI0ZD2ARcFrepRgHjAdubdBxmJmZmZlZP0REvTinV2f6fODkXJ6S6+T2iV3jnEbEw5Rg8kcOfOnNzGwg9Wa2tX2A+XnH4BXAVRHxPUn3AQslfQr4OTA3888Fvp4BsddR7jwQEcskXUWZ6nYjMNMzrZmZmZmZtYa83r8dOAD4In2IcyqpGud0SWW3NeOc5uvNAGYAjBkzpqHHYmZmjdWb2dbuBg6vkb6CGncRIuJ3wNvr7OtC4MK+F9PMzMzMzAZS3tg9LEcdfIcBjHOarzcHmAPQ0dERA/laZmbWP32KeWRmZmZmZts2xzk1M7Ou3HhkZmZmZmbDHefUzMzq6U3MIzMzMzMz27ZtB9zkOKdmZlaLG4/MzMzMzOy3EdHRNdFxTs3MDDxszczMzMzMzMzMuuHGIzMzMzMzMzMzq8uNR2ZmZmZmZmZmVpcbj8x6IGmYpJ9L+l6uj5N0i6Tlkq6UtH2m75Dry3P72Mo+zs30BySdMEiHYmZmZmZmZtZnbjwy69mHKFPVdvoMcHFEHACsB6Zn+nRgfaZfnPmQdBBlBpKDgcnAl3ImEzMzMzMzG2SS5klaI+neStpISYslPZh/98h0Sbo0bwzfLemIynOmZv4HJU0djGMxGyhuPDLrhqTRwFuAr+a6gOOAqzPLfODkXJ6S6+T2iZl/CrAwIl6IiIeB5dSYtcTMzMzMzAbF5ZSbvFXnADdExHjghlwHOBEYn48ZwGVQGpuAWcBRlGv9WZ0NTmbbAjcemXXv88DHgD/k+p7A0xGxMddXAfvl8n7ASoDcviHzv5xe4zmbkTRD0lJJS9euXdvAwzAzMzMzs1oi4sfAui7J1RvDXW8YXxHFEmCEpH2AE4DFEbEuItYDi9myQcqsbbnxyKy+3YE1EXF7s14wIuZEREdEdIwaNapZL2tmZmZmZpvbOyIey+XHgb1zud6N4V7fMDZrR8MHuwBmLWwX4G2STgJ2BHYDLqHcXRievYtGA6sz/2pgf2CVpOGUxqenKumdqs8xMzMzM7MWFhEhKRq1P0kzKEPeGDNmTKN2azag3PPIrL7VETE6IsZSAl7fGBHvAm4CTsk8U4FrcnlRrpPbb4yIyPTTcja2cZTx0bc26RjMzMzMzKzvnsjhaOTfNZle78Zwr28Ye7SBtSM3Hpn13dnARyQtp8Q0mpvpc4E9M/0jZFC9iFgGXAXcB1wHzIyIl5peajMz6zVJwyT9XNL3cn2cpFtydp0rJW2f6Tvk+vLcPrayj3Mz/QFJJwzSoZiZ2dap3hjuesP4jJx1bQKwIYe3XQ9MkrRHBsqelGlm2wQPWzPrhYi4Gbg5l1dQY7a0iPgd8PY6z78QuHDgSmhmZg32IeB+ypBlgM8AF0fEQklfBqZTZtiZDqyPiAMknZb53iHpIEqv1YOBfYH/lvQ63zwwM2s9khYAxwKvkrSKMmvaRcBVkqYDjwKnZvZrgZMoMyg/D7wHICLWSboAuC3znR8RXYNwm7UtNx6ZmZmZVUgaDbyF0uj/EUkCjgPemVnmA+dRGo+m5DLA1cAXMv8UYGFEvAA8nL1SjwR+1qTDMDOzXoqI0+tsmlgjbwAz6+xnHjCvgUUzaxk9DluTtL+kmyTdJ2mZpA9l+khJiyU9mH/3yHRJujS7ad8t6YjKvqZm/gclTa33mmZmZmaD6PPAx4A/5PqewNM5UQJsPoPOy7Pr5PYNmb9Xs+5ImiFpqaSla9eubfBhmJmZmTVGb2IebQQ+GhEHAROAmdkV+xzghogYD9yQ6wAnUgICj6dEkL8MSmMTpfvfUZQ7b7M6G5zMzMzMWoGktwJrIuL2Zryeg6aamZlZO+ix8SgiHouIO3L5Wcr4//0o3bHnZ7b5wMm5PAW4IoollGnN9wFOABZHxLqIWA8sBiY38mDMzMzM+ulNwNskPQIspAxXu4RyPdM53L86g87Ls+vk9t2Bp+jDrDtmZmZmra5Ps63lDCKHA7cAe2dUeYDHgb1zuV437V513zYzMzMbLBFxbkSMjoixlIDXN0bEu4CbgFMyW9dZdzqH4p+S+SPTT8vZ2MZRemTf2qTDMNsa2zlUhZmZ1dPrxiNJuwDfAj4cEc9Ut+VFUjSiQB77b2ZmZi3obErw7OWUmEZzM30usGemf4Qcxh8Ry4CrgPuA64CZnmnN2oBDVZiZWU29ajyStB2l4egbEfHtTH4ih6ORf9dker1u2r3qvu2x/2ZmZtYKIuLmiHhrLq+IiCMj4oCIeHvOokZE/C7XD8jtKyrPvzAiXhsRr4+IHwzWcZj10u8dqsLMzOrpzWxrotxVuz8iPlfZVO2m3bX79hnZlXUCsCGHt10PTJK0R959mJRpZmZmZmbWIpoVqsIjDszM2sfwnrPwJuDdwD2S7sy0jwMXAVdJmg48Cpya264FTgKWA88D7wGIiHWSLgBuy3znR8S6RhyEmZmZmZn1X9dQFeU+chERIakhoSpyf3OAOQAdHR0N26+ZmTVej41HEfETQHU2T6yRP4CZdfY1D5jXlwKamZmZmdnA6y5URUQ81odQFcd2Sb95IMttZmYDr0+zrZmZmZmZ2TbLoSrMzKym3gxbswrNrtcJq3sxyz1xzax9SJoHvBVYExGHZNpI4EpgLPAIcGpErM/YeJdQhiw/D0zrDLqaUzR/Mnf7qYiYj5mZtaJdcKgKMzOrw41HZmZWy+XAF4ArKmmd0zVfJOmcXD+bzadrPooyXfNRlemaO4AAbpe0KGffMTOz1vJcRDhUhZmZ1eRha2ZmtoWI+DHQ9U6xp2s2MzMzMxuC3HhkZma95emazczMzMyGIDcemZlZn+VwhYZO1xwRHRHRMWrUqEbt1szMzMzMGsCNR2Zm1ltP5HA0+jBdc610MzMzMzNrI248MjOz3vJ0zWZmZmZmQ5BnWzMzsy1IWgAcC7xK0irKrGmertnMzMzMbAhy45GZmW0hIk6vs8nTNZuZmZmZDTFuPDIzMzMzMzOztjN79uytet6sWbMaXJJtn2MemZmZmZmZmZlZXW48MjMzMzMzMzOzutx4ZFbfdpJuknSfpGWSPgQgaaSkxZIezL97ZLokXSppuaS7JR3RuSNJUzP/g5Km1ntBMzMzMzNrHZIekXSPpDslLc20Pv8eMGt3bjwy695HI+IgYAIwU9JBwDnADRExHrgh1wFOBMbnYwZwGZQvF8pMVUcBRwKzOr9gzMzMzMys5f1VRBwWER253qffA2bbAjcemdX3+4i4AyAingXuB/YDpgDzM8984ORcngJcEcUSYISkfYATgMURsS4i1gOLgcnNOwwzMzMzM2ugvv4eMGt7bjwy6wVJY4HDgVuAvSPisdz0OLB3Lu8HrKw8bVWm1Uuv9TozJC2VtHTt2rWNOwAzM+sVSft7yLKZmVUE8ENJt0uakWl9/T1g1vbceGTWA0m7AN8CPhwRz1S3RURQvlAaIiLmRERHRHSMGjWqUbs1M7Pe24iHLJuZ2SZ/HhFHUD7vZ0o6prpxa34P+IaxtaMeG48kzZO0RtK9lTTffbMhQdJ2lIajb0TEtzP5ic7up/l3TaavBvavPH10ptVLNzOzFhMRj3nIspmZdYqI1fl3DfAdyg2Bvv4e6LpP3zC2ttObnkeXs+XFju++2VAxF7g/Ij5XSVsEdDaATgWuqaSfkY2oE4AN2Z31emCSpD2y3k/KNDMza2HNGLLsu8/WSnzT2GxzknaWtGvnMuU6/l76/nvArO312HgUET8G1nVJ9t03Gwp2Ad4NHJdTc94p6STgIuB4SQ8Cb851gGuBFcBy4CvA+wEiYh1wAXBbPs7PNDMza1HNGrLsu8/WYi7HN43NqvYGfiLpLuBW4PsRcR19/D1gti0YvpXPG9CAwZQvIMaMGbOVxTNriOciQnW2TeyakD8mZtbKHBHzgHkNLJuZmQ2Q7oYsR8RjfRiyfGyX9JsHstxm/RURP84ed1VT2FSX51Pq8dlUbhoDSyR13jQ+lrxpDCCp86bxgoEuv1mjRcQK4NAa6U/Rx98DZu2u3wGzHTDYzMzMthWShIcsm1UN2E1jMzNrH1vb88h338zMzAZMvU6PvdGw+zlD1ZsoQ5bvkXRnpn2cMiThKknTgUeBU3PbtcBJlCEKzwPvgTJkWVLnkGXwkGXbBkRESGrYh4xHHJiZtY+tbTzqvPt2EVvefTtL0kLKOOcN2cB0PfDpynjnScC5W19sMzMzs8aLiJ9Qv/XOQ5ZtKBqwm8YRMQeYA9DR0eGWbzOzFtbjsDVJC4CfAa+XtCrvuDlgsJmZmZnZts9DNs3MrOeeRxFxep1NvvtmZmZmZraNyJvGxwKvkrSKMmuah2yamdlWD1szMzMzM7NtiG8am5lZPf2ebc3MzMzMzMzMzLZdbjwyMzMzMzMzM7O63HhkZmZmZmZmZmZ1ufHIzMzMzMzMzMzqcuORmZmZmZmZmZnV5cYjMzMzMzMzMzOry41HZmZmZmZmZmZW1/DBLoCZmZmZmVlLkbbueRGNLYeZWYtw45HZELK110HgayEzMzMzM7OhysPWzMzMzMzMzMysLjcemZmZmZmZmZlZXW48MjMzMzMzMzOzuhzzyMzMzMzMzMyGjNmzZ2/1c2fNmtXAkrQP9zwyMzMzMzMzM7O63HhkZmZmZmZmZmZ1ediamZmZ2TZstvrRNT+GZtd8MzMz21zTG48kTQYuAYYBX42Ii5pdhsGg2drq58asaGBJbLAM1bpv5rpvQ5nrvw1Vrvs2VLnu27aqqY1HkoYBXwSOB1YBt0laFBH3NbMcZs22LdR9bWX7Z7jtc0jbFuq+2dZy/behakjX/a29YAJfNG0DhnTdH0K2Nth2uwfabnbMoyOB5RGxIiJeBBYCU5pcBrPB4LpvQ5Xrvg1lrv82VLnu21Dlum/brGYPW9sPWFlZXwUc1eQytJ2tHfLm4W4tZcjWfd+AG/KGbN0fPFt70vmEGwBtX//7Ey9pazjG0jaj7ev+oHA3722B677VtbU9lvqjkb2dWi5gtqQZwIxcfU7SA3Wyvgp4sjml2iqDXj6d1+MX0KCXsRcGs4yvbvYL9rL+t8P/rRFeJQ2J44TW+5+2at2H1nuv6mmHcnYpYz9aegdOs99H1/3G2OqynqfzGluS7g2J97QPWrX+t9P/aWv0//j6c6euOVr9f+i633suU++1RLnOO++86mrXMvWp7je78Wg1sH9lfXSmvSwi5gBzetqRpKUR0dHY4jVOq5cPXMYm67HuQ+/q/zb0nnRrqBwnbPPH2rC6D+3zXrVDOV3Gphgy1z1V7VLWdikntFdZk697emlbPz4YGsdYsU3XfZep91qxXP0tU7NjHt0GjJc0TtL2wGnAoiaXwWwwuO7bUOW6b0OZ678NVa77NlS57ts2q6k9jyJio6SzgOspUxfOi4hlzSyD2WBw3behynXfhjLXfxuqXPdtqHLdt21Z02MeRcS1wLUN2FWPXbwHWauXD1zGphpCdb9RhspxwjZ+rA2s+9A+71U7lNNlbIIh+tnfLmVtl3JCe5UVGLJ1f2ts68cHQ+MYX7aN132XqfdasVz9KpPC0fnNzMzMzMzMzKyOZsc8MjMzMzMzMzOzNtJ2jUeSJkt6QNJySecMdnk6SXpE0j2S7pS0NNNGSlos6cH8u0eTyzRP0hpJ91bSapZJxaX5vt4t6YhBLON5klbne3mnpJMq287NMj4g6YRmlLFVtGrdb5RWPIcaoR3Ow3bRiueApP0l3STpPknLJH0o01uu7koaJunnkr6X6+Mk3ZLv55UZ2HOwyzhC0tWSfiHpfklHt+J72WytVPfb6TOtr+fnIF4L7SjpVkl3ZTlnZ3rNc1TSDrm+PLePbUY5m62V6n0jbYvXO+30udAOWqHud/P5Wfd3WhPL1lLnkKTXV96POyU9I+nDg/FeDfS52FaNR5KGAV8ETgQOAk6XdNDglmozfxURh1WmvzsHuCEixgM35HozXQ5M7pJWr0wnAuPzMQO4bBDLCHBxvpeH5bhh8n99GnBwPudLWSe2eW1Q9xul1c6hRric1j8PW14LnwMbgY9GxEHABGBmlqsV6+6HgPsr65+hfNYeAKwHpg9KqTZ3CXBdRBwIHEopbyu+l03TgnX/ctrnM62v5+dglfcF4LiIOBQ4DJgsaQL1z9HpwPpMvzjzbVNasN432rZ2vXM57fO50NJaqO7X+/yEGr/TBkHLnEMR8UDn+wG8EXge+E5ubvZ7dTkDeC62VeMRcCSwPCJWRMSLwEJgyiCXqTtTgPm5PB84uZkvHhE/Btb1skxTgCuiWAKMkLTPIJWxninAwoh4ISIeBpZT6sRQ0G51v1EG9RxqhHY4D9tES54DEfFYRNyRy89SGjv2o8XqrqTRwFuAr+a6gOOAqzNLK5Rxd+AYYC5ARLwYEU/TYu/lIGiput9On2lbcX4O1rVQRMRzubpdPoL652i1/FcDE/Oc3pa0VL1vgrb+nGunz4U20BJ1v5vPz1bVKufQROChiHh0MF58oM/Fdms82g9YWVlfRetU4gB+KOl2STMybe+IeCyXHwf2HpyibaZemVrtvT0ru8/Nq3Q7bLUyNtNQOPZ2OYcaoV3Ow1bS8u9NDh05HLiF1qu7nwc+Bvwh1/cEno6IjbneCu/nOGAt8DWV4XVflbQzrfdeNlvL133a4DOtl+fnoJVXZVjpncAaYDHwEPXP0ZfLmds3UM7pbUnL1J0BMFSud1ruPGsTLff+dPn8hNq/05qplc+h04AFlfXBfq+ggediuzUetbI/j4gjKN2/Zko6proxIoJS0VtGK5YpXQa8ltJ1+zHg3wa1NNYsbXcONcK2elxDjaRdgG8BH46IZ6rbBvt/LOmtwJqIuH2wytBLw4EjgMsi4nDgN3Tpdj7Y76X1rBX/R618flbK8VIOeRhN6Xlw4OCWyAbQkLve2RaPaaio8fnZCr/TWvIcUolL9zbgm5nUCu/VZvr73rRb49FqYP/K+uhMG3QRsTr/rqGMcTwSeKKz61f+XTN4JXxZvTK1zHsbEU/kRdQfgK+waWhay5RxEGzzx95G51AjtPx52IJa9r2RtB3lwuobEfHtTG6luvsm4G2SHqF0fz+OEltohKThmacV3s9VwKqI6LyzeTWlMamV3svB0LJ1v6JlP9P6eH4OenlzqOZNwNHUP0dfLmdu3x14qpnlbIJB/18MlCF0vdOy51mLa5n3p9bnZze/05qmhc+hE4E7IuKJLN+gv1epYediuzUe3QaMV5l9YntKt7BFg1wmJO0sadfOZWAScC+lbFMz21TgmsEp4WbqlWkRcEZGXZ8AbKh0b2uqLmMt/4byXkIp42kqs4yMowT3urXZ5RskLVn3G6XNzqFGaPnzsAW15DmQcUbmAvdHxOcqm1qm7kbEuRExOiLGUt63GyPiXZQfqKdktkE/vyLicWClpNdn0kTgPlrovRwkLVn3u2jJz7StOD8HpbySRkkakcs7AcdT4ovUO0er5T+Fck5va7062qHe99kQu95pqfOsjbRE3a/3+dnN77RmlauVz6HTqQxZG+z3qqJx52JEtNUDOAn4JWUs+CcGuzxZptcAd+VjWWe5KOPPbwAeBP4bGNnkci2gdJH7PeWO7vR6ZQJEiez/EHAP0DGIZfx6luHurNT7VPJ/Isv4AHDiYP/vm/z/bLm638Bja8lzqEHH1vLnYbs8WvEcAP6c0v33buDOfJzUqnUXOBb4Xi6/htIAv5zSxXqHFijfYcDSfD+/C+zRqu9lk9+Xlqn77fSZ1tfzc7DKC7wB+HmW817gXzO95jkK7Jjry3P7awa7jg7Q+9Iy9b6Bx7RNXu+00+dCOzxaoe538/lZ93dak8rVkucQsDOlB+julbSmv1cDfS4qn2hmZmZmZmZmZraFdhu2ZmZmZmZmZmZmTeTGIzMzMzMzMzMzq8uNR2ZmZmZmZmZmVpcbj8zMzMzMzMzMrC43HpmZmZmZmZmZWV1uPDIzMzMzMzMzs7rceGRmZmZmZmZmZnW58cjMzMzMzMzMzOr6/wFctI5hIAzPlwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x288 with 6 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# PLOTTING WORD-COUNT\n",
    "fig,(ax1, ax2, ax3, ax4, ax5, ax6)=plt.subplots(1,6,figsize=(20,4))\n",
    "\n",
    "swahili_words = dataframe[dataframe['lang'].str.contains(\"swahili\")].sentence\n",
    "ax1.hist(swahili_words,color='green')\n",
    "ax1.set_title('swahili sentences')\n",
    "\n",
    "\n",
    "wolof_words = dataframe[dataframe['lang'].str.contains(\"wolof\")].sentence\n",
    "ax2.hist(wolof_words,color='blue')\n",
    "ax2.set_title('wolof sentences')\n",
    "\n",
    "\n",
    "french_words = dataframe[dataframe['lang'].str.contains(\"french\")].sentence\n",
    "ax3.hist(french_words,color='yellow')\n",
    "ax3.set_title('french sentences')\n",
    "\n",
    "\n",
    "english_words = dataframe[dataframe['lang'].str.contains(\"english\")].sentence\n",
    "ax4.hist(english_words,color='purple')\n",
    "ax4.set_title('english sentences')\n",
    "\n",
    "\n",
    "shikomori_words = dataframe[dataframe['lang'].str.contains(\"arabic\")].sentence\n",
    "ax5.hist(shikomori_words,color='red')\n",
    "ax5.set_title('arabic sentences')\n",
    "\n",
    "shikomori_words = dataframe[dataframe['lang'].str.contains(\"dyula\")].sentence\n",
    "ax6.hist(shikomori_words,color='gray')\n",
    "ax6.set_title('dyula sentences')\n",
    "\n",
    "\n",
    "fig.suptitle('word per sentence')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9b00a8",
   "metadata": {},
   "source": [
    "- Below we can see a quick comparison of the average number of words in a sentence of each language in our dataset.\n",
    "- In our case, the most wordy language is **``dyula``** and the less one is **``wolof``**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67722520",
   "metadata": {},
   "source": [
    "### Model training: **`SVM classifier with TfidfVectorizer`**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b28ced",
   "metadata": {},
   "source": [
    "#### Importing libraries needed for model training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f88db7c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from sklearn import svm\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from yaspin import yaspin "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2b04d37",
   "metadata": {},
   "source": [
    "#### Quick overview of the dataset we use for training the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "10b68b29",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mu ne dina fi jaar buy dellu</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mimi ningekuwa naipenda yanga si ningekuwa nac...</td>\n",
       "      <td>swahili</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pencil skirts are nice until you have to go up...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>kingsport  tn transportation job rolloff drive...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>daan sa doole</td>\n",
       "      <td>wolof</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2023</th>\n",
       "      <td>just ok inner plastic curtain keeps falling of...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49318</th>\n",
       "      <td>great for daily devotions</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46698</th>\n",
       "      <td>this is not the jason shampoo with msm i looke...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33705</th>\n",
       "      <td>it charges extremely slowly for some reason al...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42648</th>\n",
       "      <td>came quickly and are working great i thought i...</td>\n",
       "      <td>english</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>83124 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                sentence     lang\n",
       "0                           mu ne dina fi jaar buy dellu    wolof\n",
       "1      mimi ningekuwa naipenda yanga si ningekuwa nac...  swahili\n",
       "2      pencil skirts are nice until you have to go up...  english\n",
       "3      kingsport  tn transportation job rolloff drive...  english\n",
       "4                                          daan sa doole    wolof\n",
       "...                                                  ...      ...\n",
       "2023   just ok inner plastic curtain keeps falling of...  english\n",
       "49318                          great for daily devotions  english\n",
       "46698  this is not the jason shampoo with msm i looke...  english\n",
       "33705  it charges extremely slowly for some reason al...  english\n",
       "42648  came quickly and are working great i thought i...  english\n",
       "\n",
       "[83124 rows x 2 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "languages_file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "583faf3c",
   "metadata": {},
   "source": [
    "#### Data separation\n",
    "> Because we are going to export our model we are going to use 95% of the data on traing and the rest for the test, i've already tested the model on 20% of the data and still got good results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "4877cce5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separating the 95% data for training data and 10% for testing data\n",
    "X = languages_file.sentence\n",
    "y = languages_file.lang\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.05, random_state=8888) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38ec86cc",
   "metadata": {},
   "source": [
    "#### Feature Vectorizing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b424a9c9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No. of feature_words:  18121\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\SAAD\\anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# Fit the TF-IDF Vectorizer :\n",
    "vectorizer = TfidfVectorizer(min_df = 5,\n",
    "                             max_df = 0.8,\n",
    "                             sublinear_tf = True,\n",
    "                             use_idf = True)\n",
    "\n",
    "train_vectors = vectorizer.fit_transform(X_train)\n",
    "test_vectors = vectorizer.transform(X_test)\n",
    "\n",
    "print('No. of feature_words: ', len(vectorizer.get_feature_names()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69d0230a",
   "metadata": {},
   "source": [
    "#### Performing classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b8cc699b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[K0m elapsed time (0:07:08.91)\u001b[K\u001b[?25h\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K\u001b[K"
     ]
    }
   ],
   "source": [
    "# Perform classification with SVM, kernel=linear\n",
    "with yaspin(text=\"elapsed time\", timer=True) as sp:\n",
    "    classifier_linear = svm.SVC(kernel='linear', probability=True)\n",
    "    t0 = time.time()\n",
    "    classifier_linear.fit(train_vectors, y_train)\n",
    "    t1 = time.time()\n",
    "    prediction_linear = classifier_linear.predict(test_vectors)\n",
    "    t2 = time.time()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "524985e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training time: 426.20s; Prediction time: 2.72s\n"
     ]
    }
   ],
   "source": [
    "time_linear_train = t1-t0 # The time needed to train our linear classifier\n",
    "time_linear_predict = t2-t1 # The time needed to predict with our trained linear classifier\n",
    "\n",
    "print(\"Training time: {:.2f}s; Prediction time: {:.2f}s\".format(time_linear_train, time_linear_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3251a1c",
   "metadata": {},
   "source": [
    "#### Model testing "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "8dffb077",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random as rd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "aa6a09e9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "wolof precision: 99.56%\n",
      "french precision: 99.71%\n",
      "swahili precision: 100.00%\n",
      "english precision: 96.83%\n",
      "arabic precision: 93.63%\n",
      "dyula precision: 100.00%\n",
      "\n",
      "\n",
      "wolof:  {'precision': 0.9956011730205279, 'recall': 0.9883551673944687, 'f1-score': 0.9919649379108838, 'support': 687}\n",
      "french:  {'precision': 0.9971264367816092, 'recall': 0.9788434414668548, 'f1-score': 0.9879003558718862, 'support': 709}\n",
      "swahili:  {'precision': 1.0, 'recall': 0.9849108367626886, 'f1-score': 0.9923980649619903, 'support': 729}\n",
      "english:  {'precision': 0.9683195592286501, 'recall': 0.9736842105263158, 'f1-score': 0.9709944751381215, 'support': 722}\n",
      "arabic:  {'precision': 0.9363354037267081, 'recall': 0.9741518578352181, 'f1-score': 0.9548693586698337, 'support': 619}\n",
      "dyula:  {'precision': 1.0, 'recall': 1.0, 'f1-score': 1.0, 'support': 691}\n"
     ]
    }
   ],
   "source": [
    "report = classification_report(y_test, prediction_linear, output_dict=True)\n",
    "\n",
    "print('wolof precision: {:.2f}%'.format(report['wolof'][\"precision\"]*100))\n",
    "print('french precision: {:.2f}%'.format(report['french'][\"precision\"]*100))\n",
    "print('swahili precision: {:.2f}%'.format(report['swahili'][\"precision\"]*100))\n",
    "print('english precision: {:.2f}%'.format(report['english'][\"precision\"]*100))\n",
    "print('arabic precision: {:.2f}%'.format(report['arabic'][\"precision\"]*100))\n",
    "print('dyula precision: {:.2f}%'.format(report['dyula'][\"precision\"]*100))\n",
    "\n",
    "print(\"\\n\")\n",
    "print('wolof: ', report['wolof'])\n",
    "print('french: ', report['french'])\n",
    "print('swahili: ', report['swahili'])\n",
    "print('english: ', report['english'])\n",
    "print('arabic: ', report['arabic'])\n",
    "print('dyula: ', report['dyula'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "37267988",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Defining a function to simplify the prediction process\n",
    "def predict_language(text):\n",
    "    serie = pd.Series(text)\n",
    "    vector = vectorizer.transform(serie)\n",
    "    return classifier_linear.predict(vector)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "3534b914",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mu dem ci ñaari xale yu góor yi ni leen ñaari xale yu góor yi du ngéen ma fetalal yëkku kuuy yëkku kuuy dee ma luqi bët bi diw ko asamaan asamaan taw wereyaan sax xar lekk ko dee mool ngemboo der ba dindil ma saa wujj wi ci biir déeg bi ba ma woli soq sa déegu jonn\n",
      "\n",
      " -> LABELED LANGUAGE: wolof\n",
      "-> PREDICTED LANGUAGE: wolof\n"
     ]
    }
   ],
   "source": [
    "random_number = rd.randint(0, len(languages_file)+1) # Picking a random sentence from the dataset\n",
    "text = languages_file.sentence[random_number]\n",
    "language = languages_file.lang[random_number]\n",
    "\n",
    "print(text)\n",
    "print(\"\\n -> LABELED LANGUAGE: \" + language)\n",
    "print(\"-> PREDICTED LANGUAGE: \" + predict_language(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7014b628",
   "metadata": {},
   "source": [
    "#### Saving the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "38f0b3c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "pickle.dump(classifier_linear, open('model/SVM_model_language_identifier.pkl', 'wb')) # saving the model\n",
    "with open('model/SVM_vectorizer.pk', 'wb') as fin: # saving the vectorizer\n",
    "    pickle.dump(vectorizer, fin)\n",
    "exit()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0af3c78",
   "metadata": {},
   "source": [
    "#### Loading the model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7332e15",
   "metadata": {},
   "source": [
    "##### meth 1\n",
    "> via model and vectorizer import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "9465bda1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "import pandas as pd\n",
    "\n",
    "SVM_model = pickle.load(open('model/SVM_model_language_identifier.pkl', 'rb'))\n",
    "SVM_vectorizer = pickle.load(open(\"model/SVM_vectorizer.pk\",\"rb\"))\n",
    "\n",
    "\n",
    "def predict_language(text):\n",
    "    serie = pd.Series(text)\n",
    "    vector = SVM_vectorizer.transform(serie)\n",
    "    return str(SVM_model.predict(vector)[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "68aba391",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "wolof\n"
     ]
    }
   ],
   "source": [
    "text = \"Na nga def ?\" \n",
    "print(predict_language(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3b6a067",
   "metadata": {},
   "source": [
    "##### meth 2\n",
    "> by calling a python script that does all the work for us"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "52fb59d0",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "english\n"
     ]
    }
   ],
   "source": [
    "text = \"I'm not really into the birthday thing honestly but I admit this was a really chill\"\n",
    "var = !python model/language_identifier.py $text \n",
    "print(var[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "402ca1f8",
   "metadata": {},
   "source": [
    "> - 🙌 Notebook made by [@labriji_saad](https://github.com/labrijisaad)\n",
    "> - 🔗 Linledin [@labriji_saad](https://www.linkedin.com/in/labrijisaad/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}