Video2BlogPost.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hae8NChJE8B_"
      },
      "source": [
        "# Using Claude 3 to transform a video tutorial in a blog post\n",
        "\n",
        "This notebook provides a baseline to reproduce Anthropic's solution to Karpathy's challenge of converting a video tutorial in a blog post. See associated Medium article [here](https://medium.com/@ya-lb/using-claude-3-to-transform-a-video-tutorial-in-a-blog-post-d2c1e04e7a7b).\n",
        "\n",
        "### Original data\n",
        "\n",
        "- Karpathy's video tutorial on tokenization : https://www.youtube.com/watch?v=zduSFxRajkE\n",
        "- Hand-written tutorial summary : https://github.com/karpathy/minbpe/blob/master/lecture.md\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ezEbNf6YF2ri"
      },
      "source": [
        "# Install and import libraries\n",
        "\n",
        "- `pytube`: used to download a Youtube video\n",
        "- `youtube-transcript-api`: used to directly download the video transcript from Youtube, if available\n",
        "- `faster_whisper`: used to get transcript from audio\n",
        "- `anthropic`: used to access Claude 3.0 large multimodal model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "ZDNNuLFdF2cf"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!pip install -q pytube\n",
        "!pip install -q youtube-transcript-api\n",
        "!pip install -q anthropic\n",
        "!pip install faster_whisper"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gx6v-ap8GK5X"
      },
      "source": [
        "Let us load the libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "SZueZdDuE5F0"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import glob\n",
        "from pathlib import Path\n",
        "import re\n",
        "\n",
        "import pytube\n",
        "from youtube_transcript_api import YouTubeTranscriptApi\n",
        "from faster_whisper import WhisperModel\n",
        "import torch\n",
        "import anthropic\n",
        "\n",
        "import cv2 #Used to extract frames from video\n",
        "import base64 #Used to convert JPG image in base64 format\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6MHgGIObGUst"
      },
      "source": [
        "Put your Anthropic API key here:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "2KTQK1usGUzu"
      },
      "outputs": [],
      "source": [
        "ANTHROPIC_API_KEY = \"sk-...\"\n",
        "os.environ[\"ANTHROPIC_API_KEY\"] = ANTHROPIC_API_KEY\n",
        "\n",
        "client = anthropic.Anthropic(api_key=os.environ.get(\"ANTHROPIC_API_KEY\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Define Youtube video ID, folder to store video, chapters, and resulting blog post."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "5jXfMMx1H1Ag"
      },
      "outputs": [],
      "source": [
        "# Andrej Karpathy : Let's build the GPT Tokenizer - https://www.youtube.com/watch?v=zduSFxRajkE\n",
        "youtube_video_id = \"zduSFxRajkE\"\n",
        "\n",
        "DATA_DIR = youtube_video_id\n",
        "CHAPTERS_DIR = DATA_DIR+\"/chapters\"\n",
        "MERGE_DIR = DATA_DIR+\"/final_output\"\n",
        "\n",
        "if not os.path.exists(DATA_DIR):\n",
        "    os.makedirs(DATA_DIR)\n",
        "\n",
        "if not os.path.exists(CHAPTERS_DIR):\n",
        "    os.makedirs(CHAPTERS_DIR)\n",
        "\n",
        "if not os.path.exists(MERGE_DIR):\n",
        "    os.makedirs(MERGE_DIR)\n",
        "    "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Is2gMyoUG7Pr"
      },
      "source": [
        "## Download video and get transcript\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pJzyyDGqHNho"
      },
      "source": [
        "### Download video"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "6oRHJWUfHHXO"
      },
      "outputs": [],
      "source": [
        "def download_youtube_video(video_id, output_path):\n",
        "    \"\"\"\n",
        "    Download a YouTube video given its ID, stores it in output_path, and returns the output path with the video ID as filename.\n",
        "    \"\"\"\n",
        "\n",
        "    # Create a YouTube object with the video ID\n",
        "    youtube = pytube.YouTube(f\"https://www.youtube.com/watch?v={video_id}\")\n",
        "    # Get the highest resolution video stream\n",
        "    stream = youtube.streams.get_highest_resolution()\n",
        "    # Download the video\n",
        "    video_path = stream.download(output_path=output_path, filename=video_id+\".mp4\")\n",
        "\n",
        "    return video_path\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mMSL7OY3HSts",
        "outputId": "1e65c49d-eceb-4ee1-ad8d-e4c7e5a1ab5a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: user 483 ms, sys: 51.7 ms, total: 534 ms\n",
            "Wall time: 3.44 s\n"
          ]
        }
      ],
      "source": [
        "# About 20 seconds for 330MB video\n",
        "%time video_path=download_youtube_video(youtube_video_id, DATA_DIR)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7l10eoDUI5MB"
      },
      "source": [
        "### Get transcript\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### With YouTubeTranscriptApi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'text': \"hi everyone so in this video I'd like us\",\n",
              "  'start': 0.04,\n",
              "  'duration': 4.04},\n",
              " {'text': 'to cover the process of tokenization in',\n",
              "  'start': 2.04,\n",
              "  'duration': 4.4},\n",
              " {'text': 'large language models now you see here',\n",
              "  'start': 4.08,\n",
              "  'duration': 4.2},\n",
              " {'text': \"that I have a set face and that's\",\n",
              "  'start': 6.44,\n",
              "  'duration': 3.88}]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "transcript = YouTubeTranscriptApi.get_transcript(youtube_video_id, languages=[\"en\"])\n",
        "transcript[0:4]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "3422"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "len(transcript)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### With Whisper"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 307,
          "referenced_widgets": [
            "7f303e51420542c3966b88f22bc511cc",
            "69eb3b58035c49e1aae52d54ee6b49b1",
            "db49681c0ba74ec2be054455e190ff2f",
            "d368691a0cc64f39bfe8deba1af56ee9",
            "0fda4859069c44cb91aa7cd1cb8c135d",
            "5f4a43ef05d7486c860ace1905eb4432",
            "2670fa9d7b3042faa16115111b13fa0d",
            "448769a71b744fdab9ebfa979a6037f7",
            "149cbcbaba50491aad252eeb6a5f0939",
            "34f1101be13d4b0283a6188da0e7865f",
            "cedafecaaf6f44078f7a307501251fec",
            "67160c2a907746c9bea0e7411cc55dc6",
            "5288fa17fb6a4bab8cc030d5c8e6a940",
            "68f36a6e21ec4da499d34bcf1c3077af",
            "c93728be9a504a6592025d76da71efa2",
            "46871f1aa7854e04832202ac6a6bf4b3",
            "d5e8c54e07e74cc294e21957935c17e3",
            "d3a51af668594c27b321da04aea460c3",
            "dca8daac0b2342fdb0e0d6f44c1f2b0a",
            "160f14e3285b42b186a0046740125241",
            "6630197f929e4c0caf42bde46d1bed72",
            "a9e6cfff03d649c1ad770606fa088570",
            "279ab02ab34e4cfd93398afe6ab3fe52",
            "2f6ee52dc62246f4b0ea4a19e7fc4052",
            "f50184a769324dfb8dbd87f6412e008e",
            "1e70ac856ea24a398994917b8347f7fa",
            "b06ea66f71704df99c5241c11b0c97f4",
            "5f87845a6a884a639e96b0dfd1b07cc4",
            "6ff4a774f9324cf39ac009120c1f0fb2",
            "0f0c45cb5fd844f5ade26e0d4b855667",
            "8b67eb7b7ef247e78d6b4580ab31a411",
            "add73905264a415d8545d2dc01e73f1b",
            "2c411521bf92481a9d2327d40f7afd4e",
            "a45061bae8c74a67bb0aa9fc732aec03",
            "791829eec9ce4c7585715a455c91a345",
            "0c2ce7531ec14be1b634514368304208",
            "a6732d8f540745e9b5c13087cfe4646b",
            "2a50fc0ee9174fe79f074d1e6da221fd",
            "e95c4fee6da14edab08641a30a4cae67",
            "54a6b1bd00634613860c8f738a0cd57f",
            "c90b67b82c4147aab3de015b061fb6ed",
            "f9fd3b70b2434d7f8857b3bdc0e1fb08",
            "8edd5efdf38e4344886c86614adb3a45",
            "b48cc4e148aa47bba31070c945a4ee4b",
            "2192c498dca64b0ebc25e97183cda8b1",
            "0609ce95360a445c8559f1f148410310",
            "4c728e60bc5f4b18a5d4b82eccf6779e",
            "bca4a6320f314908928e6aed7cbabd57",
            "962a179711754bf995e1b8f2d3bc8c33",
            "f585a4d03ce64fda8a282ee2fd120609",
            "480d172265504ae59a7fdd35aa8c845f",
            "21fb03d8db644eeb99002c8ff12dccef",
            "e9e75822ea5d4218ab8429c7589d225e",
            "411a8154a01e42eda163d9c9552017ec",
            "87ae6af5def049928177a93f7c1ca148"
          ]
        },
        "id": "WoCYKdXIm6t2",
        "outputId": "583d3cae-fd37-48d3-a4e7-4a71a092d7d4"
      },
      "outputs": [],
      "source": [
        "whisper_model = WhisperModel(\"large-v3\",\n",
        "                              device=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n",
        "                              compute_type=\"float16\",\n",
        "                            )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "sYZT9QZHn19t"
      },
      "outputs": [],
      "source": [
        "def speech_to_text(whisper_model, audio_file, initial_prompt=\"\", language=\"en\", segments=None):\n",
        "\n",
        "        segments, transcript_info = whisper_model.transcribe(audio_file,  initial_prompt=initial_prompt, language=language)\n",
        "        segments = list(segments)\n",
        "        segments = [\n",
        "            {\n",
        "                \"start\": round(s.start,2),\n",
        "                \"duration\": round(s.end-s.start,2),\n",
        "                \"text\": s.text,\n",
        "            }\n",
        "            for s in segments\n",
        "        ]\n",
        "\n",
        "        return segments"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "#25 minutes for a 2h13 video on T4\n",
        "transcript = speech_to_text(whisper_model, video_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "geO0dtBmHHfv"
      },
      "source": [
        "## Chop up in chapters of aligned text and screenshots"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [],
      "source": [
        "def get_text_chapter(transcript, chapter_start_time, chapter_end_time, output_dir):\n",
        "    \"\"\"\n",
        "    Extract and save a specified chapter's text from a transcript.\n",
        "\n",
        "    This function iterates through a transcript, extracting text that falls within the specified start and end times\n",
        "    of a chapter. The extracted text is concatenated into a single string, which is then saved to a file named\n",
        "    'transcript.txt' within the specified output directory.\n",
        "\n",
        "    Args:\n",
        "        transcript (list of dicts): The transcript from which to extract text, where each entry in the list\n",
        "            represents a segment of the transcript with a start time, end time, and text.\n",
        "        chapter_start_time (int): The start time of the chapter, used to identify which segments of the transcript to include.\n",
        "        chapter_end_time (int): The end time of the chapter, used to identify which segments of the transcript to include.\n",
        "        output_dir (str): The directory where the extracted chapter text will be saved.\n",
        "\n",
        "    The function does not return any value but writes the extracted chapter text to 'transcript.txt' in the specified directory.\n",
        "    \"\"\"\n",
        "    text_chapter = \"\"\n",
        "\n",
        "    for i in range(len(transcript)):\n",
        "        transcript_i = transcript[i]\n",
        "\n",
        "        # Check if the current transcript segment falls within the chapter's start and end times\n",
        "        if int(transcript_i['start']) >= chapter_start_time and int(transcript_i['start']) <= chapter_end_time:\n",
        "            # Concatenate text from the audio transcript, removing any new lines and leading/trailing whitespace\n",
        "            text_chapter += transcript_i['text'].replace('\\n', ' ').strip() + \" \"\n",
        "\n",
        "    # Define the path to the output transcript file\n",
        "    transcript_file = output_dir + '/transcript.txt'\n",
        "\n",
        "    # Save the concatenated chapter text to the specified file\n",
        "    with open(transcript_file, \"w\") as f:\n",
        "        f.write(text_chapter)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [],
      "source": [
        "def get_frames_chapter(video_path, chapter_start_time, chapter_end_time, output_dir, timestamps_screenshots=None):\n",
        "    \"\"\"\n",
        "    Extract and save frames from a specified chapter of a video at given timestamps or at regular intervals.\n",
        "\n",
        "    This function calculates a list of timestamps to take screenshots if not provided, defaulting to 10 evenly spaced\n",
        "    intervals within the chapter duration. If the calculated interval is less than 60 seconds, it defaults to 60 seconds.\n",
        "    It then opens the video file, iterates over the calculated or provided timestamps, captures frames at these timestamps,\n",
        "    and saves them as JPEG files in the specified output directory.\n",
        "\n",
        "    Args:\n",
        "        video_path (str): The path to the video file.\n",
        "        chapter_start_time (int): The start time of the chapter in seconds.\n",
        "        chapter_end_time (int): The end time of the chapter in seconds.\n",
        "        output_dir (str): The directory where the extracted frames will be saved.\n",
        "        timestamps_screenshots (list of int, optional): Specific timestamps to capture screenshots. If None,\n",
        "            screenshots will be taken at regular intervals within the chapter.\n",
        "\n",
        "    The function does not return any value but saves the captured frames in the specified output directory.\n",
        "    \"\"\"\n",
        "    # Calculate default timestamps if not provided\n",
        "    if timestamps_screenshots is None:\n",
        "        screenshot_interval = int((chapter_end_time - chapter_start_time) / 10)\n",
        "        # Ensure a minimum interval of 60 seconds between screenshots\n",
        "        if screenshot_interval < 60:\n",
        "            screenshot_interval = 60\n",
        "        timestamps_screenshots = list(range(chapter_start_time, chapter_end_time, screenshot_interval))\n",
        "    else:\n",
        "        #Make sure timestamps are integers\n",
        "        timestamps_screenshots = [int(ts) for ts in timestamps_screenshots]\n",
        "\n",
        "    # Open the video file using OpenCV\n",
        "    video = cv2.VideoCapture(video_path)\n",
        "\n",
        "    # Determine the frames per second (FPS) of the video for frame index calculation\n",
        "    fps = video.get(cv2.CAP_PROP_FPS)\n",
        "\n",
        "    # Capture and save frames at each specified timestamp\n",
        "    for timestamp in timestamps_screenshots:\n",
        "        # Calculate the frame index based on the timestamp and video FPS\n",
        "        index = int(timestamp * fps)\n",
        "        video.set(cv2.CAP_PROP_POS_FRAMES, index)\n",
        "\n",
        "        # Attempt to read the frame at the calculated index\n",
        "        success, frame = video.read()\n",
        "\n",
        "        # If the frame is successfully read, save it as a JPEG file\n",
        "        if success:\n",
        "            # Format the timestamp for the output filename\n",
        "            timestamp_str = \"{:05d}\".format(timestamp)\n",
        "            output_path = f\"{output_dir}/{timestamp_str}.jpg\"\n",
        "            # Save the frame to the output directory\n",
        "            cv2.imwrite(output_path, frame)\n",
        "\n",
        "    # Release the video file resources\n",
        "    video.release()\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "pdYRDKiLhcFy"
      },
      "outputs": [],
      "source": [
        "def chop_up_in_chapters(chapters_list, video_path, transcript, timestamps_screenshots_list_seconds=None):\n",
        "    \"\"\"\n",
        "    Split the video in chapters based on the video chapters list.\n",
        "    \"\"\"\n",
        "\n",
        "    n_chapters=len(chapters_list)-1\n",
        "    print(f\"Number of chunks: {n_chapters}\")\n",
        "\n",
        "    # Iterate over the timestamps and topics\n",
        "    for current_chapter in range(n_chapters):\n",
        "\n",
        "        output_dir=CHAPTERS_DIR+\"/\"+str(current_chapter)\n",
        "\n",
        "         # Create the output directory if it does not exist\n",
        "        if not os.path.exists(output_dir):\n",
        "            os.makedirs(output_dir)\n",
        "\n",
        "        # Get the current and next timestamp\n",
        "        current_chunk_start_time=chapters_list[current_chapter]['timestamp']\n",
        "        current_chunk_end_time=chapters_list[current_chapter+1]['timestamp']-1\n",
        "\n",
        "        print(f\"Chapter {current_chapter}; Start: {current_chunk_start_time}, End: {current_chunk_end_time}\")\n",
        "\n",
        "        # Extract text and frames for the current chapter\n",
        "        get_text_chapter(transcript, current_chunk_start_time, current_chunk_end_time, output_dir)\n",
        "        \n",
        "        if timestamps_screenshots_list_seconds is not None:\n",
        "            get_frames_chapter(video_path, current_chunk_start_time, current_chunk_end_time, output_dir,timestamps_screenshots_list_seconds[current_chapter])\n",
        "        else:\n",
        "            get_frames_chapter(video_path, current_chunk_start_time, current_chunk_end_time, output_dir)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'timestamp': 0,\n",
              "  'topic': 'intro: Tokenization, GPT-2 paper, tokenization-related issues'},\n",
              " {'timestamp': 350,\n",
              "  'topic': 'tokenization by example in a Web UI (tiktokenizer)'},\n",
              " {'timestamp': 896, 'topic': 'strings in Python, Unicode code points'},\n",
              " {'timestamp': 1095,\n",
              "  'topic': 'Unicode byte encodings, ASCII, UTF-8, UTF-16, UTF-32'},\n",
              " {'timestamp': 1367, 'topic': 'daydreaming: deleting tokenization'},\n",
              " {'timestamp': 1430,\n",
              "  'topic': 'Byte Pair Encoding (BPE) algorithm walkthrough'},\n",
              " {'timestamp': 1622, 'topic': 'starting the implementation'},\n",
              " {'timestamp': 1715,\n",
              "  'topic': 'counting consecutive pairs, finding most common pair'},\n",
              " {'timestamp': 1836, 'topic': 'merging the most common pair'},\n",
              " {'timestamp': 2098,\n",
              "  'topic': 'training the tokenizer: adding the while loop, compression ratio'},\n",
              " {'timestamp': 2360,\n",
              "  'topic': 'tokenizer/LLM diagram: it is a completely separate stage'},\n",
              " {'timestamp': 2567, 'topic': 'decoding tokens to strings'},\n",
              " {'timestamp': 2901, 'topic': 'encoding strings to tokens'},\n",
              " {'timestamp': 3456,\n",
              "  'topic': 'regex patterns to force splits across categories'},\n",
              " {'timestamp': 4298,\n",
              "  'topic': 'tiktoken library intro, differences between GPT-2/GPT-4 regex'},\n",
              " {'timestamp': 4499,\n",
              "  'topic': 'GPT-2 encoder.py released by OpenAI walkthrough'},\n",
              " {'timestamp': 4706,\n",
              "  'topic': 'special tokens, tiktoken handling of, GPT-2/GPT-4 differences'},\n",
              " {'timestamp': 5128,\n",
              "  'topic': 'minbpe exercise time! write your own GPT-4 tokenizer'},\n",
              " {'timestamp': 5322,\n",
              "  'topic': 'sentencepiece library intro, used to train Llama 2 vocabulary'},\n",
              " {'timestamp': 6207,\n",
              "  'topic': 'how to set vocabulary set? revisiting gpt.py transformer'},\n",
              " {'timestamp': 6491,\n",
              "  'topic': 'training new tokens, example of prompt compression'},\n",
              " {'timestamp': 6598,\n",
              "  'topic': 'multimodal [image, video, audio] tokenization with vector quantization'},\n",
              " {'timestamp': 6701,\n",
              "  'topic': 'revisiting and explaining the quirks of LLM tokenization'},\n",
              " {'timestamp': 7820, 'topic': 'final recommendations'},\n",
              " {'timestamp': 7970, 'topic': '??? :)'}]"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chapters_24=\"\"\"\n",
        "00:00:00 intro: Tokenization, GPT-2 paper, tokenization-related issues\n",
        "00:05:50 tokenization by example in a Web UI (tiktokenizer)\n",
        "00:14:56 strings in Python, Unicode code points\n",
        "00:18:15 Unicode byte encodings, ASCII, UTF-8, UTF-16, UTF-32\n",
        "00:22:47 daydreaming: deleting tokenization\n",
        "00:23:50 Byte Pair Encoding (BPE) algorithm walkthrough\n",
        "00:27:02 starting the implementation\n",
        "00:28:35 counting consecutive pairs, finding most common pair\n",
        "00:30:36 merging the most common pair\n",
        "00:34:58 training the tokenizer: adding the while loop, compression ratio\n",
        "00:39:20 tokenizer/LLM diagram: it is a completely separate stage\n",
        "00:42:47 decoding tokens to strings\n",
        "00:48:21 encoding strings to tokens\n",
        "00:57:36 regex patterns to force splits across categories\n",
        "01:11:38 tiktoken library intro, differences between GPT-2/GPT-4 regex\n",
        "01:14:59 GPT-2 encoder.py released by OpenAI walkthrough\n",
        "01:18:26 special tokens, tiktoken handling of, GPT-2/GPT-4 differences\n",
        "01:25:28 minbpe exercise time! write your own GPT-4 tokenizer\n",
        "01:28:42 sentencepiece library intro, used to train Llama 2 vocabulary\n",
        "01:43:27 how to set vocabulary set? revisiting gpt.py transformer\n",
        "01:48:11 training new tokens, example of prompt compression\n",
        "01:49:58 multimodal [image, video, audio] tokenization with vector quantization\n",
        "01:51:41 revisiting and explaining the quirks of LLM tokenization\n",
        "02:10:20 final recommendations\n",
        "02:12:50 ??? :)\n",
        "\"\"\"\n",
        "\n",
        "def chapters_to_list(chapters):\n",
        "    chapters_list = chapters.strip().split('\\n')\n",
        "    chapters_dict_list = []\n",
        "\n",
        "    for chapter in chapters_list:\n",
        "        time_str, topic = chapter.split(' ', 1)\n",
        "        hours, minutes, seconds = map(int, time_str.split(':'))\n",
        "        total_seconds = hours * 3600 + minutes * 60 + seconds\n",
        "        chapters_dict_list.append({\"timestamp\": total_seconds, \"topic\": topic})\n",
        "\n",
        "    return chapters_dict_list\n",
        "\n",
        "chapters_list = chapters_to_list(chapters_24)\n",
        "last_timestamp=int(transcript[-1]['start']+transcript[-1]['duration'])\n",
        "#chapters_list.append({\"timestamp\": last_timestamp, \"topic\": \"end\"})\n",
        "chapters_list"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XZj-HZkxhcFy",
        "outputId": "0077689e-877f-4a5b-f4c2-54e541202918"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of chunks: 24\n",
            "Chapter 0; Start: 0, End: 349\n",
            "Chapter 1; Start: 350, End: 895\n",
            "Chapter 2; Start: 896, End: 1094\n",
            "Chapter 3; Start: 1095, End: 1366\n",
            "Chapter 4; Start: 1367, End: 1429\n",
            "Chapter 5; Start: 1430, End: 1621\n",
            "Chapter 6; Start: 1622, End: 1714\n",
            "Chapter 7; Start: 1715, End: 1835\n",
            "Chapter 8; Start: 1836, End: 2097\n",
            "Chapter 9; Start: 2098, End: 2359\n",
            "Chapter 10; Start: 2360, End: 2566\n",
            "Chapter 11; Start: 2567, End: 2900\n",
            "Chapter 12; Start: 2901, End: 3455\n",
            "Chapter 13; Start: 3456, End: 4297\n",
            "Chapter 14; Start: 4298, End: 4498\n",
            "Chapter 15; Start: 4499, End: 4705\n",
            "Chapter 16; Start: 4706, End: 5127\n",
            "Chapter 17; Start: 5128, End: 5321\n",
            "Chapter 18; Start: 5322, End: 6206\n",
            "Chapter 19; Start: 6207, End: 6490\n",
            "Chapter 20; Start: 6491, End: 6597\n",
            "Chapter 21; Start: 6598, End: 6700\n",
            "Chapter 22; Start: 6701, End: 7819\n",
            "Chapter 23; Start: 7820, End: 7969\n"
          ]
        }
      ],
      "source": [
        "chop_up_in_chapters(chapters_list, video_path, transcript) "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2e9CowTGK_QY"
      },
      "source": [
        "## LLM transform\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-mdP9bU_LEXO"
      },
      "source": [
        "This is the core step. For each chapter, the audio transcript and selected screenshots are provided to the LMM, with the goal of transforming these input data into an output suitable for inclusion in a textbook.\n",
        "\n",
        "Documentation for querying Claude with images: https://docs.anthropic.com/claude/docs/vision\n",
        "\n",
        "Prompt inspired by https://github.com/hundredblocks/transcription_demo/tree/main"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [],
      "source": [
        "prompt_instructions = f\"\"\"\n",
        "<instructions>\n",
        "You have been given images of a video at different timestamps, followed by the audio transcript in <transcript>\n",
        "The transcript was generated by an AI speech recognition tool and may contain some errors/infelicities.\n",
        "Your task is to transform the transcript into a markdown blog post.\n",
        "This transcript is noisy. Please rewrite it using the following guidelines:\n",
        "- output valid markdown\n",
        "- insert section headings and other formatting where appropriate\n",
        "- you are given only part of a transcript, so do not include introductory or concluding paragraphs. Only include the main topics discussed in the transcript\n",
        "- use styling to make images, text, code, callouts and the page layout and margins look like a typical blog post or textbook\n",
        "- remove any verbal tics\n",
        "- if there are redundant pieces of information, only present it once\n",
        "- keep the conversational content in the style of the transcript. Including headings to make the narrative structure easier to follow along\n",
        "- the transcript includes too many images, so you should only include the most important 1-2 images in your output\n",
        "- choose images that provide illustrations that are relevant to the transcript\n",
        "- prefer to include images which display complete code, rather than in progress\n",
        "- when relevant transcribe important pieces of code and other valuable text\n",
        "- if an image would help illustrate a part of a transcript, include it\n",
        "- to include an image, insert a tag with <img src=\"xxxxx.jpg\"/> where xxxxx is replaced by the exact image timestamp inserted above the image data\n",
        "- do not add any extraneous information: only include what is either mentioned in the transcript or the images\n",
        "\n",
        "Your final output should be suitable for inclusion in a textbook.\n",
        "</instructions>\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Transform the JPG screenshots in a format suitable for Anthorpic's API. \n",
        "\n",
        "The function iterates over all screenshots in order to describe each of them with two messages:\n",
        "\n",
        "- a text message that specifies the timestamp for the screenshot, \n",
        "- and an image message containing its base64-encoded representation. \n",
        "\n",
        "The text message with the timestamp will allow later to add a hyperlink from the final document to the original video."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "PyDpiSUKLRz1"
      },
      "outputs": [],
      "source": [
        "def get_screenshots_as_messages(screenshots):\n",
        "\n",
        "\tscreenshots_as_messages = []\n",
        "\n",
        "\tfor i in range(len(screenshots)):\n",
        "\t\tscreenshots_as_messages.extend([\n",
        "\t\t{\n",
        "\t\t\t\"type\": \"text\",\n",
        "\t\t\t\"text\": f\"The timestamp for the following image is {Path(screenshots[i]).stem}.\"\n",
        "\t\t},\n",
        "\t\t{\n",
        "\t\t\"type\": \"image\",\n",
        "\t\t\"source\": {\n",
        "\t\t\t\"type\": \"base64\",\n",
        "\t\t\t\"media_type\": \"image/jpeg\",\n",
        "\t\t\t\"data\": base64.b64encode(open(screenshots[i], \"rb\").read()).decode(\"utf-8\"),\n",
        "\t\t}\n",
        "\t\t}\n",
        "\t\t])\n",
        "\n",
        "return screenshots_as_messages\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Bring together the screenshots, transcript and instructions. \n",
        "\n",
        "The function additionally prefills Claude's output to make it start its answer with a markdown title - https://docs.anthropic.com/claude/docs/prefill-claudes-response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {},
      "outputs": [],
      "source": [
        "def get_prompt_as_messages(chapter_id):\n",
        "\n",
        "    folder_path=CHAPTERS_DIR+'/'+str(chapter_id)\n",
        "\n",
        "    with open(folder_path+'/transcript.txt', \"r\") as f:\n",
        "        transcript = f.read()\n",
        "\n",
        "    screenshots=sorted(glob.glob(folder_path+'/*.jpg'))\n",
        "    \n",
        "    screenshots_as_messages=get_screenshots_as_messages(screenshots)\n",
        "\n",
        "    prompt_as_messages = [\n",
        "        {\n",
        "            \"role\": \"user\",\n",
        "            \"content\": screenshots_as_messages+\n",
        "            [\n",
        "                {\n",
        "                    \"type\": \"text\",\n",
        "                    \"text\": f\"<transcript>\\n{transcript}\\n</transcript>\"\n",
        "                },\n",
        "                {\n",
        "                    \"type\": \"text\",\n",
        "                    \"text\": prompt_instructions\n",
        "                }\n",
        "            ],\n",
        "        },\n",
        "        {\n",
        "            \"role\": \"assistant\",\n",
        "            \"content\": [\n",
        "                {\n",
        "                    \"type\": \"text\",\n",
        "                    \"text\": \"#\"\n",
        "                }\n",
        "            ]\n",
        "        }\n",
        "    ]\n",
        "\n",
        "    return prompt_as_messages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "owv_YpF7hcFz"
      },
      "outputs": [],
      "source": [
        "# Check content\n",
        "prompt_as_messages = get_prompt_as_messages(0)\n",
        "prompt_as_messages"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Iteratively call Claude, and writing the result as a markdown file in the corresponding chapter folder."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l0FJLgk0hcFz",
        "outputId": "ad039dc0-9699-4562-b3b0-44748967a4fc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Processing chunk 0\n",
            "Processing chunk 1\n",
            "Processing chunk 2\n",
            "Processing chunk 3\n",
            "Processing chunk 4\n",
            "Processing chunk 5\n",
            "Processing chunk 6\n",
            "Processing chunk 7\n",
            "Processing chunk 8\n",
            "Processing chunk 9\n",
            "Processing chunk 10\n",
            "Processing chunk 11\n",
            "Processing chunk 12\n",
            "Processing chunk 13\n",
            "Processing chunk 14\n",
            "Processing chunk 15\n",
            "Processing chunk 16\n",
            "Processing chunk 17\n",
            "Processing chunk 18\n",
            "Processing chunk 19\n",
            "Processing chunk 20\n",
            "Processing chunk 21\n",
            "Processing chunk 22\n",
            "Processing chunk 23\n"
          ]
        }
      ],
      "source": [
        "\n",
        "# Iterate through the list of chapters\n",
        "for chapter in range(len(chapters_list)-1): \n",
        "\n",
        "    # Display the current processing chapter number to the console.\n",
        "    print(f\"Processing chunk {chapter}\")\n",
        "\n",
        "    # Generate the prompt for the current chapter (list of messages with screenshots, transcript and instructions).\n",
        "    prompt_generate_markdown = get_prompt_as_messages(chapter)\n",
        "\n",
        "    # Create a message by invoking Claude with the prompt.\n",
        "    message = client.messages.create(\n",
        "        model=\"claude-3-opus-20240229\",\n",
        "        system=\"You are an expert at writing markdown blog post.\",\n",
        "        temperature=0,\n",
        "        max_tokens=4000,\n",
        "        messages=prompt_generate_markdown\n",
        "    )\n",
        "\n",
        "    # Extract the generated markdown content from the response.\n",
        "    answer = message.content[0].text\n",
        "    markdown = \"#\"+answer  # Prepend a header tag to the markdown content.\n",
        "    \n",
        "    # Define the path for the markdown file corresponding to the current chapter.\n",
        "    markdown_file = CHAPTERS_DIR + '/' + str(chapter) + '/markdown.md'\n",
        "\n",
        "    # Write the generated markdown content to the file.\n",
        "    with open(markdown_file, \"w\") as f:\n",
        "        f.write(markdown)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Merge all chapters and finalize blog post\n",
        "\n",
        "The final and last step of the workflow consists in two main tasks. First, it merges together the different markdown outputs. Second, it adds hyperlinks to chapter titles and images. This allows to connect the final markdown file to the original YouTube video at relevant timestamps."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "0MdV06shhcFz"
      },
      "outputs": [],
      "source": [
        "merged_markdown=\"\"\n",
        "\n",
        "# Iterate over the chapter folders to merge the markdown files\n",
        "for chapter in range(len(chapters_list)-1):\n",
        "\n",
        "    markdown_file=CHAPTERS_DIR+'/'+str(chapter)+'/markdown.md'\n",
        "\n",
        "    with open(markdown_file, \"r\") as f:\n",
        "        markdown = f.readlines()\n",
        "\n",
        "    # Let us add, for each chapter title, a hyperlink to the video at the right timestamp\n",
        "    url_chapter = f\"https://www.youtube.com/watch?v={youtube_video_id}&t={chapters_list[chapter]['timestamp']}s\"\n",
        "    markdown[0] = f\"# [{chapter+1}) {markdown[0][2:].strip()}]({url_chapter})\"\n",
        "    markdown = '\\n'.join(markdown)\n",
        "\n",
        "    merged_markdown+=\"\\n\"+markdown\n",
        "\n",
        "# Find all <img> tags with timestamps in the src attribute, so we can add a hyperlink to the video at the right timestamp\n",
        "timestamps_screenshots = re.findall(r'<img src=\"(\\d+)\\.jpg\"/>', merged_markdown)\n",
        "timestamps_screenshots = [timestamp for timestamp in timestamps_screenshots]\n",
        "\n",
        "# Add a hyperlink to the video at the right timestamp for each image\n",
        "for timestamp in timestamps_screenshots:\n",
        "    video_link = f'<a href=\"https://www.youtube.com/watch?v={youtube_video_id}&t={int(timestamp)}s\">Link to video</a>'\n",
        "    merged_markdown = merged_markdown.replace(f'<img src=\"{timestamp}.jpg\"/>', f'<img src=\"{timestamp}.jpg\"/>\\n\\n{video_link}')\n",
        "\n",
        "# Get frames based on screenshots effectively selected in the merged markdown and save in merge folder\n",
        "get_frames_chapter(video_path, None, None, MERGE_DIR, timestamps_screenshots=timestamps_screenshots)\n",
        "\n",
        "# Save the merged markdown to a markdown blogpost.md file\n",
        "markdown_file=MERGE_DIR+'/blogpost.md'\n",
        "with open(markdown_file, \"w\") as f:\n",
        "        f.write(merged_markdown)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vbw-lUaYKUR9"
      },
      "source": [
        "## Useful links\n",
        "\n",
        "- Companion Medium article\n",
        "- [Claude 3 - Vision documentation](https://docs.anthropic.com/claude/docs/vision)\n",
        "- Karpathy's challenge and Ameisen and colleague's repository\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.5"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0609ce95360a445c8559f1f148410310": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f585a4d03ce64fda8a282ee2fd120609",
            "placeholder": "​",
            "style": "IPY_MODEL_480d172265504ae59a7fdd35aa8c845f",
            "value": "preprocessor_config.json: 100%"
          }
        },
        "0c2ce7531ec14be1b634514368304208": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c90b67b82c4147aab3de015b061fb6ed",
            "max": 1068114,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f9fd3b70b2434d7f8857b3bdc0e1fb08",
            "value": 1068114
          }
        },
        "0f0c45cb5fd844f5ade26e0d4b855667": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0fda4859069c44cb91aa7cd1cb8c135d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "149cbcbaba50491aad252eeb6a5f0939": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "160f14e3285b42b186a0046740125241": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1e70ac856ea24a398994917b8347f7fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_add73905264a415d8545d2dc01e73f1b",
            "placeholder": "​",
            "style": "IPY_MODEL_2c411521bf92481a9d2327d40f7afd4e",
            "value": " 2.39k/2.39k [00:00&lt;00:00, 97.5kB/s]"
          }
        },
        "2192c498dca64b0ebc25e97183cda8b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0609ce95360a445c8559f1f148410310",
              "IPY_MODEL_4c728e60bc5f4b18a5d4b82eccf6779e",
              "IPY_MODEL_bca4a6320f314908928e6aed7cbabd57"
            ],
            "layout": "IPY_MODEL_962a179711754bf995e1b8f2d3bc8c33"
          }
        },
        "21fb03d8db644eeb99002c8ff12dccef": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2670fa9d7b3042faa16115111b13fa0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "279ab02ab34e4cfd93398afe6ab3fe52": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2f6ee52dc62246f4b0ea4a19e7fc4052",
              "IPY_MODEL_f50184a769324dfb8dbd87f6412e008e",
              "IPY_MODEL_1e70ac856ea24a398994917b8347f7fa"
            ],
            "layout": "IPY_MODEL_b06ea66f71704df99c5241c11b0c97f4"
          }
        },
        "2a50fc0ee9174fe79f074d1e6da221fd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2c411521bf92481a9d2327d40f7afd4e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2f6ee52dc62246f4b0ea4a19e7fc4052": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f87845a6a884a639e96b0dfd1b07cc4",
            "placeholder": "​",
            "style": "IPY_MODEL_6ff4a774f9324cf39ac009120c1f0fb2",
            "value": "config.json: 100%"
          }
        },
        "34f1101be13d4b0283a6188da0e7865f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "411a8154a01e42eda163d9c9552017ec": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "448769a71b744fdab9ebfa979a6037f7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46871f1aa7854e04832202ac6a6bf4b3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "480d172265504ae59a7fdd35aa8c845f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4c728e60bc5f4b18a5d4b82eccf6779e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_21fb03d8db644eeb99002c8ff12dccef",
            "max": 340,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e9e75822ea5d4218ab8429c7589d225e",
            "value": 340
          }
        },
        "5288fa17fb6a4bab8cc030d5c8e6a940": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d5e8c54e07e74cc294e21957935c17e3",
            "placeholder": "​",
            "style": "IPY_MODEL_d3a51af668594c27b321da04aea460c3",
            "value": "model.bin: 100%"
          }
        },
        "54a6b1bd00634613860c8f738a0cd57f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5f4a43ef05d7486c860ace1905eb4432": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5f87845a6a884a639e96b0dfd1b07cc4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6630197f929e4c0caf42bde46d1bed72": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67160c2a907746c9bea0e7411cc55dc6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5288fa17fb6a4bab8cc030d5c8e6a940",
              "IPY_MODEL_68f36a6e21ec4da499d34bcf1c3077af",
              "IPY_MODEL_c93728be9a504a6592025d76da71efa2"
            ],
            "layout": "IPY_MODEL_46871f1aa7854e04832202ac6a6bf4b3"
          }
        },
        "68f36a6e21ec4da499d34bcf1c3077af": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dca8daac0b2342fdb0e0d6f44c1f2b0a",
            "max": 3087284237,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_160f14e3285b42b186a0046740125241",
            "value": 3087284237
          }
        },
        "69eb3b58035c49e1aae52d54ee6b49b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f4a43ef05d7486c860ace1905eb4432",
            "placeholder": "​",
            "style": "IPY_MODEL_2670fa9d7b3042faa16115111b13fa0d",
            "value": "tokenizer.json: 100%"
          }
        },
        "6ff4a774f9324cf39ac009120c1f0fb2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "791829eec9ce4c7585715a455c91a345": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e95c4fee6da14edab08641a30a4cae67",
            "placeholder": "​",
            "style": "IPY_MODEL_54a6b1bd00634613860c8f738a0cd57f",
            "value": "vocabulary.json: 100%"
          }
        },
        "7f303e51420542c3966b88f22bc511cc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_69eb3b58035c49e1aae52d54ee6b49b1",
              "IPY_MODEL_db49681c0ba74ec2be054455e190ff2f",
              "IPY_MODEL_d368691a0cc64f39bfe8deba1af56ee9"
            ],
            "layout": "IPY_MODEL_0fda4859069c44cb91aa7cd1cb8c135d"
          }
        },
        "87ae6af5def049928177a93f7c1ca148": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8b67eb7b7ef247e78d6b4580ab31a411": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8edd5efdf38e4344886c86614adb3a45": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "962a179711754bf995e1b8f2d3bc8c33": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a45061bae8c74a67bb0aa9fc732aec03": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_791829eec9ce4c7585715a455c91a345",
              "IPY_MODEL_0c2ce7531ec14be1b634514368304208",
              "IPY_MODEL_a6732d8f540745e9b5c13087cfe4646b"
            ],
            "layout": "IPY_MODEL_2a50fc0ee9174fe79f074d1e6da221fd"
          }
        },
        "a6732d8f540745e9b5c13087cfe4646b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8edd5efdf38e4344886c86614adb3a45",
            "placeholder": "​",
            "style": "IPY_MODEL_b48cc4e148aa47bba31070c945a4ee4b",
            "value": " 1.07M/1.07M [00:00&lt;00:00, 4.31MB/s]"
          }
        },
        "a9e6cfff03d649c1ad770606fa088570": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "add73905264a415d8545d2dc01e73f1b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b06ea66f71704df99c5241c11b0c97f4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b48cc4e148aa47bba31070c945a4ee4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bca4a6320f314908928e6aed7cbabd57": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_411a8154a01e42eda163d9c9552017ec",
            "placeholder": "​",
            "style": "IPY_MODEL_87ae6af5def049928177a93f7c1ca148",
            "value": " 340/340 [00:00&lt;00:00, 8.51kB/s]"
          }
        },
        "c90b67b82c4147aab3de015b061fb6ed": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c93728be9a504a6592025d76da71efa2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6630197f929e4c0caf42bde46d1bed72",
            "placeholder": "​",
            "style": "IPY_MODEL_a9e6cfff03d649c1ad770606fa088570",
            "value": " 3.09G/3.09G [00:13&lt;00:00, 221MB/s]"
          }
        },
        "cedafecaaf6f44078f7a307501251fec": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d368691a0cc64f39bfe8deba1af56ee9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_34f1101be13d4b0283a6188da0e7865f",
            "placeholder": "​",
            "style": "IPY_MODEL_cedafecaaf6f44078f7a307501251fec",
            "value": " 2.48M/2.48M [00:00&lt;00:00, 8.35MB/s]"
          }
        },
        "d3a51af668594c27b321da04aea460c3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d5e8c54e07e74cc294e21957935c17e3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db49681c0ba74ec2be054455e190ff2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_448769a71b744fdab9ebfa979a6037f7",
            "max": 2480617,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_149cbcbaba50491aad252eeb6a5f0939",
            "value": 2480617
          }
        },
        "dca8daac0b2342fdb0e0d6f44c1f2b0a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e95c4fee6da14edab08641a30a4cae67": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e9e75822ea5d4218ab8429c7589d225e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f50184a769324dfb8dbd87f6412e008e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0f0c45cb5fd844f5ade26e0d4b855667",
            "max": 2394,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8b67eb7b7ef247e78d6b4580ab31a411",
            "value": 2394
          }
        },
        "f585a4d03ce64fda8a282ee2fd120609": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f9fd3b70b2434d7f8857b3bdc0e1fb08": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}