5 Document translation batch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "313da351",
   "metadata": {},
   "source": [
    "# Document translation batch (asynchronous mode)\n",
    "\n",
    "https://learn.microsoft.com/en-us/azure/ai-services/translator/document-translation/how-to-guides/use-rest-api-programmatically?tabs=csharp\n",
    "\n",
    "<img src=\"https://github.com/retkowsky/azure-ai-translator/blob/main/logo.jpg?raw=true\" width=200>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0e17c0d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import json\n",
    "import os\n",
    "import requests\n",
    "import sys\n",
    "import time\n",
    "\n",
    "from azure.core.credentials import AzureKeyCredential\n",
    "from azure.ai.translation.document import DocumentTranslationClient\n",
    "from azure.storage.blob import BlobServiceClient\n",
    "from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "326398ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.10.11 (main, May 16 2023, 00:28:57) [GCC 11.2.0]'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sys.version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5e96ffdb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Today is 11-Apr-2024 12:52:28\n"
     ]
    }
   ],
   "source": [
    "print(f\"Today is {datetime.datetime.today().strftime('%d-%b-%Y %H:%M:%S')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8f284bc",
   "metadata": {},
   "source": [
    "## Azure AI Translator credentials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6dd102a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(\"azure.env\")\n",
    "\n",
    "azure_ai_translator_key = os.getenv(\"AZURE_AI_TRANSLATION_KEY\")\n",
    "azure_ai_translator_endpoint = os.getenv(\"AZURE_AI_TRANSLATION_ENDPOINTDOCUMENT\")\n",
    "azure_ai_translator_region = os.getenv(\"AZURE_AI_TRANSLATION_REGION\")\n",
    "\n",
    "blob_results_connection_string = os.getenv(\"connection_string\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac11c907",
   "metadata": {},
   "source": [
    "## Shared Access Signatures (SAS)\n",
    "\n",
    "In order to read and write from your blobs in your Azure storage account you need to do these steps:<br><br>\n",
    "1 Go the container from your storage account<br>\n",
    "2 Go to **\"Settings / Shared Access Tokens\"**<br>\n",
    "3 Check the **\"permissions\"**<br>\n",
    "4 Check the **\"expiry\"** informations<br>\n",
    "5 Then click on **\"Generate SAS Token and URL\".**<br>\n",
    "6 You will have your **\"Blob SAS token\"** and the **\"Blob SAS URL\"**.<br>\n",
    "\n",
    "- Below we have two blob storage (one for the source documents, another one for the translated documents).\n",
    "- For the source blob storage you should define the **\"Read\"** and **\"List\"** permissions (step 3).\n",
    "- For the results blob strage you should define the **\"Write\"** and **\"List\"** permissions (step 3)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94645048",
   "metadata": {},
   "source": [
    "<img src=\"https://github.com/retkowsky/azure-ai-translator/blob/main/SAS.jpg?raw=true\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5f8d1884",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Source blob SAS\n",
    "sourceblobsasurl = os.getenv(\"sourceblobsasurl\")\n",
    "sourceblobsastoken = os.getenv(\"sourceblobsastoken\")\n",
    "targetblobsasurl = os.getenv(\"targetblobsasurl\")\n",
    "targetblobsastoken = os.getenv(\"targetblobsastoken\")\n",
    "\n",
    "sourceUri = sourceblobsasurl + sourceblobsastoken\n",
    "targetUri = targetblobsasurl + targetblobsastoken"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33abd56a",
   "metadata": {},
   "source": [
    "## Azure AI Translator client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bfdd6495",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize a new instance of the DocumentTranslationClient object to interact with the Document Translation feature\n",
    "client = DocumentTranslationClient(azure_ai_translator_endpoint, AzureKeyCredential(azure_ai_translator_key))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3f4bf75d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<azure.ai.translation.document._client.DocumentTranslationClient at 0x7fb3fdcf6a70>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c7ce4d7",
   "metadata": {},
   "source": [
    "## Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4178533b",
   "metadata": {},
   "outputs": [],
   "source": [
    "targetLanguage = \"fr\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f4aace0",
   "metadata": {},
   "source": [
    "> https://aka.ms/TranslatorLanguageCodes "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "59145268",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting to translate the documents to language:fr ...\n",
      "\n",
      "Elapsed time: 00:00:24.395009\n"
     ]
    }
   ],
   "source": [
    "# include source and target locations and target language code for the begin translation operation\n",
    "print(\n",
    "    f\"Starting to translate the documents to language:{targetLanguage} ...\")\n",
    "\n",
    "start = time.time()\n",
    "\n",
    "poller = client.begin_translation(sourceUri, targetUri, targetLanguage)\n",
    "result = poller.result()\n",
    "\n",
    "elapsed = time.time() - start\n",
    "print(\"\\nElapsed time: \" + time.strftime(\"%H:%M:%S.{}\".format(str(elapsed %\n",
    "      1)[2:])[:15], time.gmtime(elapsed)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d22d0e2",
   "metadata": {},
   "source": [
    "## Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1bf61768",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31;34m\n",
      "Status: Succeeded\n",
      "Created on: 2024-04-11 12:52:57.009659+00:00\n",
      "Last updated on: 2024-04-11 12:53:20.207304+00:00\n",
      "\n",
      "Total number of translations on documents: 4\n",
      "  0 failed\n",
      "  4 succeeded\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[1;31;34m\")\n",
    "print(\"Status: {}\".format(poller.status()))\n",
    "print(\"Created on: {}\".format(poller.details.created_on))\n",
    "print(\"Last updated on: {}\".format(poller.details.last_updated_on))\n",
    "print(\n",
    "    \"\\nTotal number of translations on documents: {}\".format(\n",
    "        poller.details.documents_total_count\n",
    "    )\n",
    ")\n",
    "\n",
    "print(\"  {} failed\".format(poller.details.documents_failed_count))\n",
    "print(\"  {} succeeded\".format(poller.details.documents_succeeded_count))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8529bcda",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- Document 1\n",
      "Document ID: 004e0bf3-0000-0000-0000-000000000000\n",
      "Document status: Succeeded\n",
      "Source document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs/termsconditions_en.pdf\n",
      "Translated document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs-translated/termsconditions_en.pdf\n",
      "Translated to language: fr\n",
      "\n",
      "- Document 2\n",
      "Document ID: 004e0bf2-0000-0000-0000-000000000000\n",
      "Document status: Succeeded\n",
      "Source document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs/releasenotes_en.pdf\n",
      "Translated document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs-translated/releasenotes_en.pdf\n",
      "Translated to language: fr\n",
      "\n",
      "- Document 3\n",
      "Document ID: 004e0bf1-0000-0000-0000-000000000000\n",
      "Document status: Succeeded\n",
      "Source document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs/Microsoft%20Translator%20Customer%20Ready%20Deck%20v4.1.pptx\n",
      "Translated document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs-translated/Microsoft%20Translator%20Customer%20Ready%20Deck%20v4.1.pptx\n",
      "Translated to language: fr\n",
      "\n",
      "- Document 4\n",
      "Document ID: 004e0bf0-0000-0000-0000-000000000000\n",
      "Document status: Succeeded\n",
      "Source document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs/AutoGen%20A%20Multi-Agent%20Framework%20for%20Enabling%20Next-Gen%20AI%20Applications.msg\n",
      "Translated document location: https://azurestorageaccountsr.blob.core.windows.net:443/docs-translated/AutoGen%20A%20Multi-Agent%20Framework%20for%20Enabling%20Next-Gen%20AI%20Applications.msg\n",
      "Translated to language: fr\n",
      "\n"
     ]
    }
   ],
   "source": [
    "nb = 1\n",
    "\n",
    "for document in result:\n",
    "    print(\"- Document\", nb)\n",
    "    print(\"Document ID: {}\".format(document.id))\n",
    "    print(\"Document status: {}\".format(document.status))\n",
    "\n",
    "    if document.status == \"Succeeded\":\n",
    "        print(\"Source document location: {}\".format(\n",
    "            document.source_document_url))\n",
    "        print(\n",
    "            \"Translated document location: {}\".format(\n",
    "                document.translated_document_url)\n",
    "        )\n",
    "        print(\"Translated to language: {}\\n\".format(document.translated_to))\n",
    "        nb += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e2ebd87",
   "metadata": {},
   "source": [
    "## Translated files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ceb3b4b9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31;34m\n",
      "1 Document: AutoGen A Multi-Agent Framework for Enabling Next-Gen AI Applications.msg\n",
      "Date: 2024-04-11 12:53:15\n",
      "Size: 128512 bytes (0.12 MB)\n",
      "\n",
      "2 Document: Microsoft Translator Customer Ready Deck v4.1.pptx\n",
      "Date: 2024-04-11 12:53:18\n",
      "Size: 16978375 bytes (16.19 MB)\n",
      "\n",
      "3 Document: releasenotes_en.pdf\n",
      "Date: 2024-04-11 12:53:19\n",
      "Size: 257870 bytes (0.25 MB)\n",
      "\n",
      "4 Document: termsconditions_en.pdf\n",
      "Date: 2024-04-11 12:53:20\n",
      "Size: 387618 bytes (0.37 MB)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "container_name = \"docs-translated\"\n",
    "\n",
    "blob_service_client = BlobServiceClient.from_connection_string(blob_results_connection_string)\n",
    "container_client = blob_service_client.get_container_client(container_name)\n",
    "fileslist = container_client.list_blobs()\n",
    "\n",
    "nb = 0\n",
    "print(\"\\033[1;31;34m\")\n",
    "\n",
    "for file in fileslist:\n",
    "    print(f\"{nb+1} Document: {file.name}\")\n",
    "    print(f\"Date: {file.last_modified.strftime('%Y-%m-%d %H:%M:%S')}\")\n",
    "    size_mb = file.size / (1024 * 1024)\n",
    "    print(f\"Size: {file.size} bytes ({size_mb:.2f} MB)\\n\")\n",
    "    nb += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9afaf5d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of translated files = 4\n"
     ]
    }
   ],
   "source": [
    "print(f\"Total number of translated files = {nb}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d0f6780",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}