Traducao_TMX.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mKw2I57kIcOO"
      },
      "source": [
        "# Modelo de tradução em pares\n",
        "As traduções abordadas aqui são:\n",
        "- português brasileiro <-> espanhol\n",
        "- português brasileiro <-> inglês\n",
        "- italiano <-> latim\n",
        "- italiano <-> espanhol\n",
        "- italiano <-> inglês"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7QN3muszIxT8"
      },
      "source": [
        "## Bibliotecas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "78Ii2OB-JTXu",
        "outputId": "f54f6ca7-5c3b-4f51-f67f-1b3b7b67ad5f"
      },
      "outputs": [],
      "source": [
        "import nltk\n",
        "from nltk.translate.bleu_score import corpus_bleu\n",
        "nltk.download('punkt')\n",
        "\n",
        "import torch\n",
        "from torch import optim\n",
        "from torch.utils.data import DataLoader\n",
        "\n",
        "from IPython.display import Image"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QqopyKbUBTGg",
        "outputId": "0274d891-d154-4d21-f004-5c3607914cd6"
      },
      "outputs": [],
      "source": [
        "!pip3 install sentencepiece\n",
        "!pip3 install transformers\n",
        "!pip3 install translate-toolkit"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u8DRkbdxX0DS"
      },
      "outputs": [],
      "source": [
        "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n",
        "from translate.storage.tmx import tmxfile"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "12SFsRXEJIV_"
      },
      "source": [
        "## Corpora\n",
        "Vamos utilizar corpora retirados do website [OPUS - an open source parallel corpus](https://opus.nlpl.eu/). Eles estão todos presentes na pasta 'Dados'."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "Image(filename = 'Imagens/proto-indo-eu.jpg')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Primeiramente, lemos o córpus."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "file_path = 'Dados/'\n",
        "\n",
        "def read_corpus(filename, language_1, language_2):\n",
        "    '''\n",
        "    Read corpus function.\n",
        "\n",
        "    Params:\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "\n",
        "    Return:\n",
        "    - f_lang1_to_lang2: file containing language_1 as source of translation and language_2 as target\n",
        "    - f_lang2_to_lang1: file containing language_2 as source of translation and language_1 as target\n",
        "    '''\n",
        "\n",
        "    with open(file_path + filename, 'rb') as f_input:\n",
        "        f_output = tmxfile(f_input, language_1, language_2)\n",
        "\n",
        "    return f_output"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Depois, o preparamos para o formato necessário para que a tradução ocorra."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def prepare_data(prefix_lang, filename, language_1, language_2):\n",
        "    '''\n",
        "    Format the files correctly for the translation\n",
        "\n",
        "    Params:\n",
        "    - prefix_lang: prefix for the target language, e.g. '>>pt_br<<'\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "    \n",
        "    Return:\n",
        "    - data: data formatted for the translation from language_1 to language_2\n",
        "    '''\n",
        "\n",
        "    file = read_corpus(filename, language_1, language_2)\n",
        "\n",
        "    data = [\n",
        "        { 'src': prefix_lang + ' ' + w.source, 'trg': w.target }\n",
        "        for w in file.unit_iter()\n",
        "    ]\n",
        "\n",
        "    # print(\"Total sentences in the file: \" + str(len(data)))\n",
        "\n",
        "    return data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o7LoeCUQJWtI"
      },
      "source": [
        "Então, separamos em conjuntos de treino e teste."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def train_test(prefix_lang, filename, language_1, language_2):\n",
        "    '''\n",
        "    Split the data in train and test by 80/20\n",
        "\n",
        "    Params:\n",
        "    - prefix_lang: prefix for the target language, e.g. '>>pt_br<<'\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "\n",
        "    Return:\n",
        "    - train: first 80% of occurrences in data\n",
        "    - test: last 20% of occurrences in data\n",
        "    '''\n",
        "\n",
        "    data = prepare_data(prefix_lang, filename, language_1, language_2)\n",
        "\n",
        "    size = int(len(data) * 0.2)\n",
        "\n",
        "    train = data[size:]\n",
        "    test = data[:size]\n",
        "\n",
        "    # print(train[0])\n",
        "    # print(test[0])\n",
        "\n",
        "    return train, test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xEIDFHIjJwK5"
      },
      "source": [
        "## Treinamento"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Hiperparâmetros"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eZulD4cPJl_Q"
      },
      "outputs": [],
      "source": [
        "LEARNING_RATE = 1e-5 \n",
        "EPOCHS = 1\n",
        "BATCH_SIZE = 16\n",
        "BATCH_STATUS = 32\n",
        "EARLY_STOP = 3\n",
        "TOKEN_MAX_LENGTH = 128\n",
        "NUM_BEAMS = 4\n",
        "DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Prefixos"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "PREFIX_PT_BR = '>>pt_br<<'\n",
        "PREFIX_LATIN = '>>la<<'\n",
        "PREFIX_SPANISH = '>>es<<'\n",
        "PREFIX_ITALIAN = '>>it<<'\n",
        "PREFIX_ENGLISH = '>>en<<'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aNKiTB-GKDdp"
      },
      "source": [
        "Iniciamos o treinamento separando os dados em batches (lotes)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def batch_train_test(prefix_lang, filename, language_1, language_2):\n",
        "    '''\n",
        "    Put the data in batches\n",
        "\n",
        "    Params:\n",
        "    - prefix_lang: prefix for the target language, e.g. '>>pt_br<<'\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "\n",
        "    Return:\n",
        "    - train_data: train data in batches\n",
        "    - test_data: test data in batches\n",
        "    '''\n",
        "\n",
        "    train, test = train_test(prefix_lang, filename, language_1, language_2)\n",
        "\n",
        "    train_data = DataLoader(train, batch_size = BATCH_SIZE)\n",
        "    test_data = DataLoader(test, batch_size = BATCH_SIZE)\n",
        "\n",
        "    return train_data, test_data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zXrOnb8hKgF6"
      },
      "source": [
        "Avaliamos então nosso modelo com base na [pontuação BLEU (BiLingual Evaluation Understudy)](https://cloud.google.com/translate/automl/docs/evaluate)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-l_Cd8L1KfVC"
      },
      "outputs": [],
      "source": [
        "def evaluate(prefix_lang, filename, language_1, language_2, model, tokenizer):\n",
        "    '''\n",
        "    Evaluate the model\n",
        "\n",
        "    Params:\n",
        "    - prefix_lang: prefix for the target language, e.g. '>>pt_br<<'\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "    - model: neural network model\n",
        "    - tokenizer: tokenizer, can be pre-trained\n",
        "\n",
        "    Return:\n",
        "    - bleu: BLEU score\n",
        "    '''\n",
        "\n",
        "    # Evaluate the model\n",
        "    model.eval()\n",
        "    \n",
        "    y_real = []\n",
        "    y_pred = []\n",
        "\n",
        "    _, test_data = batch_train_test(prefix_lang, filename, language_1, language_2)\n",
        "    \n",
        "    for batch_idx, inp in enumerate(test_data):\n",
        "        y_real.extend(inp['trg'])\n",
        "        \n",
        "        # tokenization\n",
        "        model_inputs = tokenizer(\n",
        "            inp['src'], \n",
        "            truncation = True, \n",
        "            padding = True, \n",
        "            max_length = TOKEN_MAX_LENGTH, \n",
        "            return_tensors = \"pt\"\n",
        "        ).to(DEVICE)\n",
        "        \n",
        "        # Translation\n",
        "        generated_ids = model.generate(**model_inputs, num_beams = NUM_BEAMS)\n",
        "        \n",
        "        # Translation post processing\n",
        "        output = tokenizer.batch_decode(generated_ids, skip_special_tokens = True)\n",
        "        y_pred.extend(output)\n",
        "    \n",
        "        # Print results\n",
        "        if (batch_idx + 1) % BATCH_STATUS == 0:\n",
        "            print(\n",
        "                'Evaluation: [{}/{} ({:.0f}%)]'.format(\n",
        "                    batch_idx + 1, len(test_data), 100. * batch_idx/ len(test_data)\n",
        "                )\n",
        "            )\n",
        "\n",
        "    # Calculate BLUE score\n",
        "    hyps, refs = [], []\n",
        "    \n",
        "    for i, snt_pred in enumerate(y_pred):\n",
        "        hyps.append(nltk.word_tokenize(snt_pred))\n",
        "        refs.append([nltk.word_tokenize(y_real[i])])\n",
        "    \n",
        "    bleu = corpus_bleu(refs, hyps)\n",
        "\n",
        "    return bleu * 100"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tH2wZ22zLFHY"
      },
      "source": [
        "Por fim, criamos nosso fluxo de treinamento."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ntr4XkyFLEJa"
      },
      "outputs": [],
      "source": [
        "def train(prefix_lang, filename, language_1, language_2, model, tokenizer, optimizer):\n",
        "    '''\n",
        "    Training loop\n",
        "\n",
        "    Params:\n",
        "    - prefix_lang: prefix for the target language, e.g. '>>pt_br<<'\n",
        "    - filename (string): name of the file, e.g. 'en-pt_br.tmx'\n",
        "    - language_1 (string): abbreviation of the first language, e.g. 'en'\n",
        "    - language_2 (string): abbreviation of the second language, e.g. 'pt'\n",
        "    - model: neural network model\n",
        "    - tokenizer: tokenizer, can be pre-trained\n",
        "    - optimizer: neural network optimizer\n",
        "    '''\n",
        "\n",
        "    train_data, test_data = batch_train_test(prefix_lang, filename, language_1, language_2)\n",
        "    \n",
        "    # Calculate initial BLEU score\n",
        "    max_bleu = evaluate(prefix_lang, filename, language_1, language_2, model, tokenizer)\n",
        "    print('Initial BLEU score:', max_bleu)\n",
        "    \n",
        "    # Train model\n",
        "    model.train()\n",
        "    repeat = 0\n",
        "    \n",
        "    for epoch in range(EPOCHS):\n",
        "        losses = []\n",
        "\n",
        "        for batch_idx, inp in enumerate(train_data):\n",
        "            # Inicialize with the gradient equals to zero\n",
        "            optimizer.zero_grad()\n",
        "\n",
        "            # Tokenization\n",
        "            model_inputs = tokenizer(\n",
        "                inp['src'], \n",
        "                truncation = True,\n",
        "                padding = True, \n",
        "                max_length = TOKEN_MAX_LENGTH, \n",
        "                return_tensors = \"pt\"\n",
        "            ).to(DEVICE)\n",
        "            \n",
        "            with tokenizer.as_target_tokenizer():\n",
        "                labels = tokenizer(\n",
        "                    inp['trg'], \n",
        "                    truncation = True, \n",
        "                    padding = True, \n",
        "                    max_length = TOKEN_MAX_LENGTH, \n",
        "                    return_tensors = \"pt\"\n",
        "                ).input_ids.to(DEVICE)\n",
        "            \n",
        "            # Translation and Forward pass\n",
        "            output = model(**model_inputs, labels = labels)\n",
        "\n",
        "            # Calculate loss\n",
        "            loss = output.loss\n",
        "            losses.append(float(loss))\n",
        "\n",
        "            # Backpropagation\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            # Print results\n",
        "            if (batch_idx + 1) % BATCH_STATUS == 0:\n",
        "                print(\n",
        "                    'Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}\\tTotal Loss: {:.6f}'.format(\n",
        "                        epoch + 1, batch_idx + 1, len(train_data), 100. * batch_idx/ len(train_data), \n",
        "                        float(loss), round(sum(losses)/ len(losses), 5)\n",
        "                    )\n",
        "                )\n",
        "\n",
        "        # Calculate epoch BLEU score\n",
        "        bleu = evaluate(prefix_lang, filename, language_1, language_2, model, tokenizer)\n",
        "        print('BLEU:', bleu)\n",
        "        \n",
        "        if bleu > max_bleu:\n",
        "            max_bleu = bleu\n",
        "            repeat = 0\n",
        "\n",
        "            # print('Saving best model...')\n",
        "            # torch.save(model, write_path)\n",
        "        else:\n",
        "            repeat += 1\n",
        "\n",
        "        if repeat == EARLY_STOP:\n",
        "            break"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5xjqXQTnL8Ie"
      },
      "source": [
        "## Modelo\n",
        "Para mais detalhes, confira [aqui](https://huggingface.co/docs/transformers/model_doc/marian)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0bwoZUFzL1hb"
      },
      "outputs": [],
      "source": [
        "model = AutoModelForSeq2SeqLM.from_pretrained(\"Helsinki-NLP/opus-mt-en-ROMANCE\").to(DEVICE)\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"Helsinki-NLP/opus-mt-en-ROMANCE\")\n",
        "optimizer = optim.AdamW(model.parameters(), lr = LEARNING_RATE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Experimentos"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def experiment(batch_input_sent, model, tokenizer):\n",
        "    '''\n",
        "    Given sentence, print translation\n",
        "\n",
        "    Params:\n",
        "    - batch_input_sent: input sentences to translate\n",
        "    - model: neural network model\n",
        "    - tokenizer: tokenizer, can be pre-trained\n",
        "    '''\n",
        "\n",
        "    # Tokenize sentences\n",
        "    encoded = tokenizer(batch_input_sent, return_tensors = 'pt', padding = True).to(DEVICE)\n",
        "\n",
        "    # Translation\n",
        "    translated = model.generate(**encoded)\n",
        "\n",
        "    # Prepare output\n",
        "    print(tokenizer.batch_decode(translated, skip_special_tokens = True))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Português brasileiro <-> Espanhol"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Português brasileiro -> Espanhol"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_SPANISH, 'es-pt_BR.tmx', 'pt_br', 'es', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>es<< Será que isso vai funcionar?\"),\n",
        "    (\">>es<< Teste número 2. Você consegue traduzir isso que eu sei!\"),\n",
        "    (\">>es<< Acho que eu preciso deixar você rodando por mais tempo, né?\"),\n",
        "    (\">>es<< Eu sei que eu deveria ser mais criativo nos meus testes, mas quero traduzir de português brasileiro para espanhol, mesmo que o BLEU seja baixo.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Espanhol -> Português brasileiro"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_PT_BR, 'es-pt_BR.tmx', 'es', 'pt_br', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>pt_br<< Buenos días.\"),\n",
        "    (\">>pt_br<< No sé cómo resultará esto.\"),\n",
        "    (\">>pt_br<< Por favor trabaja.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Português brasileiro <-> Inglês"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Português brasileiro -> Inglês"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_ENGLISH, 'en-pt_br.tmx', 'pt_br', 'en', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>en<< Será que isso vai funcionar?\"),\n",
        "    (\">>en<< Teste número 2. Você consegue traduzir isso que eu sei!\"),\n",
        "    (\">>en<< Acho que eu preciso deixar você rodando por mais tempo, né?\"),\n",
        "    (\">>en<< Eu sei que eu deveria ser mais criativo nos meus testes, mas não acredito que consegui traduzir de português brasileiro para inglês, mesmo com um BLEU tão baixo.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Inglês -> Português brasileiro"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_PT_BR, 'en-pt_br.tmx', 'en', 'pt_br', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>pt_br<< Please, don't fail me now.\"), \n",
        "    (\">>pt_br<< Who is a good translator? You are!\"), \n",
        "    (\">>pt_br<< I hope you are able to translate a big sentence, because people nowadays love texting. And I want to present this to my teacher and colleagues, so you have to work!\"),\n",
        "    (\">>pt_br<< I really don't want to study tonight but I have to do it because I want to graduate and get a job and have a lot of money.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Italiano <-> Latim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Italiano -> Latim"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_LATIN, 'it-la.tmx', 'it', 'la', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>la<< Allora molti morirono combattendo per la libertà.\"),\n",
        "    (\">>la<< Gli alunni ascoltavano i maestri per imparare molte cose.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Latim -> Italiano"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_ITALIAN, 'it-la.tmx', 'la', 'it', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>it<< Libertatis causa pugnantes multi tum ceciderunt.\"),\n",
        "    (\">>it<< Discipuli magistros audiebant ut multa discerent.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Italiano <-> Espanhol"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Italiano -> Espanhol"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_SPANISH, 'es-it.tmx', 'it', 'es', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>es<< Oggi devo studiare ancora per due ore.\"),\n",
        "    (\">>es<< Mi piace disegnare paesaggi di montagna.\"),\n",
        "    (\">>es<< Il mio computer non funziona tanto bene.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Espanhol -> Italiano"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_ITALIAN, 'es-it.tmx', 'es', 'it', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>it<< Buenos días.\"),\n",
        "    (\">>it<< No sé cómo resultará esto.\"),\n",
        "    (\">>it<< Por favor trabaja.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Italiano <-> Inglês"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Italiano -> Inglês"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_ENGLISH, 'en-it.tmx', 'it', 'en', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>en<< Oggi devo studiare ancora per due ore.\"),\n",
        "    (\">>en<< Mi piace disegnare paesaggi di montagna.\"),\n",
        "    (\">>en<< Il mio computer non funziona tanto bene.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Inglês -> Italiano"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train(PREFIX_ITALIAN, 'en-it.tmx', 'en', 'it', model, tokenizer, optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "batch_input_sent = (\n",
        "    (\">>it<< Please work. I'm exhausted already.\"),\n",
        "    (\">>it<< Now I just need one more sentence so I can finally save this file.\"),\n",
        "    (\">>it<< And I couldn't be creative, even at the very end.\")\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment(batch_input_sent, model, tokenizer)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "Traducao TED PT ENG",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}