explore_ami_corpus.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explore AMI corpus for speaker diarization\n",
    "\n",
    "In this notebook, the characteristics of the [AMI](https://groups.inf.ed.ac.uk/ami/corpus/overview.shtml) corpus will be analyzed to see whether it is adequate for comparing different speaker diarization [pipelines](https://hackmd.io/xr3m0hiWRLeh9TzYlMNAeA). The comparison requires the corpus to have many segments of speech and non-speech with differing lengths. The speech segments should be distributed equally across speakers. The corpus includes four different speakers per file (differing across files)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Reusing dataset ami (C:\\Users\\MalteLuken\\.cache\\huggingface\\datasets\\ami\\microphone-single\\1.6.2\\2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e)\n",
      "100%|██████████| 1/1 [00:00<00:00, 250.45it/s]\n"
     ]
    }
   ],
   "source": [
    "ami = load_dataset(\"ami\", \"microphone-single\", split=[\"test\"])[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_sample_length(sample):\n",
    "    audio = sample[\"audio\"]\n",
    "    return audio[\"array\"].shape[0]/audio[\"sampling_rate\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_nr_segments(sample):\n",
    "    return len(sample[\"segment_ids\"]) + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_non_speech_ratio(sample):\n",
    "    inter_segment_lengths = []\n",
    "    for i, _ in enumerate(sample[\"segment_ids\"]):\n",
    "        if i < len(sample[\"segment_ids\"])-1 and sample[\"segment_start_times\"][i+1]>=sample[\"segment_end_times\"][i]:\n",
    "            inter_segment_lengths.append(sample[\"segment_start_times\"][i+1]-sample[\"segment_end_times\"][i])\n",
    "    total_length = np.sum(inter_segment_lengths)\n",
    "    non_speech_ratio = total_length/get_sample_length(sample)\n",
    "    return non_speech_ratio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_segment_lengths(sample):\n",
    "    return np.array(sample[\"segment_end_times\"]) - np.array(sample[\"segment_start_times\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_segment_stats(sample):\n",
    "    segment_lengths = get_segment_lengths(sample)\n",
    "    return round(np.mean(segment_lengths), 2), round(np.std(segment_lengths), 2), round(np.min(segment_lengths), 2), round(np.max(segment_lengths), 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_equal_speakers_index(sample):\n",
    "    spks = list(set(sample[\"segment_speakers\"]))\n",
    "    prob = np.array([sample[\"segment_speakers\"].count(spk) for spk in spks])/get_nr_segments(sample)\n",
    "    sh_index = -np.sum(prob*np.log(prob))\n",
    "    rndm = np.array([0.25, 0.25, 0.25, 0.25])\n",
    "    return sh_index/-np.sum(rndm*np.log(rndm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_equal_speech_time_index(sample):\n",
    "    segment_lengths = get_segment_lengths(sample)\n",
    "    spks = list(set(sample[\"segment_speakers\"]))\n",
    "    spk_times = []\n",
    "    for spk in spks:\n",
    "        spk_times.append(np.sum([segment_lengths[i] for i, ref_spk in enumerate(sample[\"segment_speakers\"]) if ref_spk == spk]))\n",
    "    \n",
    "    sample_length = get_sample_length(sample)\n",
    "    rel_spk_times = np.array(spk_times)/sample_length\n",
    "    sh_index = -np.sum(rel_spk_times*np.log(rel_spk_times))\n",
    "    rndm = np.array([0.25, 0.25, 0.25, 0.25])\n",
    "    return sh_index/-np.sum(rndm*np.log(rndm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_dict = {'sample_length_min': [], 'nr_segments': [], 'seg_length_secs_M_SD_Min_Max': [], 'non_speech_ratio': [], 'equal_speakers_index': [], 'equal_speech_time_index': []}\n",
    "\n",
    "for row in ami:\n",
    "    df_dict['sample_length_min'].append(get_sample_length(row)/60)\n",
    "    df_dict['nr_segments'].append(get_nr_segments(row))\n",
    "    df_dict['seg_length_secs_M_SD_Min_Max'].append(get_segment_stats(row))\n",
    "    df_dict['non_speech_ratio'].append(get_non_speech_ratio(row))\n",
    "    df_dict['equal_speakers_index'].append(get_equal_speakers_index(row))\n",
    "    df_dict['equal_speech_time_index'].append(get_equal_speech_time_index(row))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sample_length_min</th>\n",
       "      <th>nr_segments</th>\n",
       "      <th>seg_length_secs_M_SD_Min_Max</th>\n",
       "      <th>non_speech_ratio</th>\n",
       "      <th>equal_speakers_index</th>\n",
       "      <th>equal_speech_time_index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17.49</td>\n",
       "      <td>284</td>\n",
       "      <td>(3.72, 5.77, 0.27, 43.44)</td>\n",
       "      <td>0.45</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>39.09</td>\n",
       "      <td>488</td>\n",
       "      <td>(4.94, 8.28, 0.27, 59.36)</td>\n",
       "      <td>0.41</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38.91</td>\n",
       "      <td>522</td>\n",
       "      <td>(4.68, 7.13, 0.29, 42.93)</td>\n",
       "      <td>0.42</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>37.04</td>\n",
       "      <td>668</td>\n",
       "      <td>(3.39, 5.88, 0.3, 55.06)</td>\n",
       "      <td>0.49</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13.98</td>\n",
       "      <td>223</td>\n",
       "      <td>(3.48, 4.85, 0.19, 31.58)</td>\n",
       "      <td>0.38</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>34.21</td>\n",
       "      <td>410</td>\n",
       "      <td>(5.07, 10.3, 0.27, 58.61)</td>\n",
       "      <td>0.49</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>30.35</td>\n",
       "      <td>334</td>\n",
       "      <td>(5.05, 10.08, 0.21, 128.4)</td>\n",
       "      <td>0.40</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>32.41</td>\n",
       "      <td>538</td>\n",
       "      <td>(3.52, 5.85, 0.19, 48.68)</td>\n",
       "      <td>0.37</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>25.09</td>\n",
       "      <td>310</td>\n",
       "      <td>(3.91, 7.61, 0.22, 56.22)</td>\n",
       "      <td>0.50</td>\n",
       "      <td>0.93</td>\n",
       "      <td>0.59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>36.84</td>\n",
       "      <td>440</td>\n",
       "      <td>(4.58, 5.76, 0.22, 27.58)</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.91</td>\n",
       "      <td>0.89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>42.83</td>\n",
       "      <td>417</td>\n",
       "      <td>(5.02, 6.21, 0.54, 26.63)</td>\n",
       "      <td>0.26</td>\n",
       "      <td>0.94</td>\n",
       "      <td>0.89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>43.64</td>\n",
       "      <td>772</td>\n",
       "      <td>(3.11, 4.03, 0.24, 28.3)</td>\n",
       "      <td>0.42</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>35.71</td>\n",
       "      <td>826</td>\n",
       "      <td>(3.53, 4.41, 0.24, 36.69)</td>\n",
       "      <td>0.40</td>\n",
       "      <td>0.99</td>\n",
       "      <td>1.04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>29.78</td>\n",
       "      <td>551</td>\n",
       "      <td>(3.95, 5.79, 0.26, 39.41)</td>\n",
       "      <td>0.44</td>\n",
       "      <td>0.98</td>\n",
       "      <td>0.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>49.54</td>\n",
       "      <td>678</td>\n",
       "      <td>(5.25, 7.86, 0.17, 62.17)</td>\n",
       "      <td>0.41</td>\n",
       "      <td>0.79</td>\n",
       "      <td>0.79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>36.83</td>\n",
       "      <td>802</td>\n",
       "      <td>(3.8, 5.12, 0.19, 49.3)</td>\n",
       "      <td>0.38</td>\n",
       "      <td>0.98</td>\n",
       "      <td>1.03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    sample_length_min  nr_segments seg_length_secs_M_SD_Min_Max  \\\n",
       "0               17.49          284    (3.72, 5.77, 0.27, 43.44)   \n",
       "1               39.09          488    (4.94, 8.28, 0.27, 59.36)   \n",
       "2               38.91          522    (4.68, 7.13, 0.29, 42.93)   \n",
       "3               37.04          668     (3.39, 5.88, 0.3, 55.06)   \n",
       "4               13.98          223    (3.48, 4.85, 0.19, 31.58)   \n",
       "5               34.21          410    (5.07, 10.3, 0.27, 58.61)   \n",
       "6               30.35          334   (5.05, 10.08, 0.21, 128.4)   \n",
       "7               32.41          538    (3.52, 5.85, 0.19, 48.68)   \n",
       "8               25.09          310    (3.91, 7.61, 0.22, 56.22)   \n",
       "9               36.84          440    (4.58, 5.76, 0.22, 27.58)   \n",
       "10              42.83          417    (5.02, 6.21, 0.54, 26.63)   \n",
       "11              43.64          772     (3.11, 4.03, 0.24, 28.3)   \n",
       "12              35.71          826    (3.53, 4.41, 0.24, 36.69)   \n",
       "13              29.78          551    (3.95, 5.79, 0.26, 39.41)   \n",
       "14              49.54          678    (5.25, 7.86, 0.17, 62.17)   \n",
       "15              36.83          802      (3.8, 5.12, 0.19, 49.3)   \n",
       "\n",
       "    non_speech_ratio  equal_speakers_index  equal_speech_time_index  \n",
       "0               0.45                  0.97                     0.93  \n",
       "1               0.41                  0.99                     0.98  \n",
       "2               0.42                  0.99                     0.98  \n",
       "3               0.49                  1.00                     0.96  \n",
       "4               0.38                  0.97                     0.81  \n",
       "5               0.49                  0.99                     0.99  \n",
       "6               0.40                  0.98                     0.95  \n",
       "7               0.37                  0.99                     0.90  \n",
       "8               0.50                  0.93                     0.59  \n",
       "9               0.25                  0.91                     0.89  \n",
       "10              0.26                  0.94                     0.89  \n",
       "11              0.42                  0.97                     0.88  \n",
       "12              0.40                  0.99                     1.04  \n",
       "13              0.44                  0.98                     0.99  \n",
       "14              0.41                  0.79                     0.79  \n",
       "15              0.38                  0.98                     1.03  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(df_dict).round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "segment_lengths = []\n",
    "\n",
    "for row in ami:\n",
    "    for seg_len in get_segment_lengths(row):\n",
    "        if seg_len < 60: # Only less than 1 min\n",
    "            segment_lengths.append(seg_len)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQaklEQVR4nO3dbaycZZ3H8e9P6tOqa4t0G9I2WzY2azBZHtIARrNRiFDQWF6owZi1MU36pptgYqJlN1niAwm8ETG7kjTStRhXZH1YGjRit2A2+0LgVBCByvaIENoAPdiC6xrJFv/7Yq6ake3hnNNOz+mc6/tJJnPf//uameuf3v3N3XvumaaqkCT14VULPQFJ0vwx9CWpI4a+JHXE0Jekjhj6ktSRJQs9gVdyxhln1Jo1axZ6GpI0Vvbs2fNcVS0/1rZTOvTXrFnDxMTEQk9DksZKkien2+bpHUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6sgp/Y3cE7Vm6/eO+7FPXP++Ec5Ekk4NHulLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SerIrEI/yRNJfpbkwSQTrXZ6kl1J9rX7Za2eJF9KMpnkoSTnDz3PxjZ+X5KNJ6clSdJ05nKk/56qOreq1rX1rcDuqloL7G7rAJcDa9ttM3AzDN4kgGuBC4ELgGuPvlFIkubHiZze2QDsaMs7gCuH6rfWwI+BpUnOBC4DdlXVoao6DOwC1p/A60uS5mi2oV/AD5PsSbK51VZU1dNt+RlgRVteCTw19Nj9rTZd/Y8k2ZxkIsnE1NTULKcnSZqNJbMc966qOpDkz4BdSX4+vLGqKkmNYkJVtQ3YBrBu3bqRPKckaWBWR/pVdaDdHwS+y+Cc/LPttA3t/mAbfgBYPfTwVa02XV2SNE9mDP0kb0jypqPLwKXAw8BO4OgVOBuBO9ryTuBj7Sqei4AX2mmgu4BLkyxrH+Be2mqSpHkym9M7K4DvJjk6/l+q6gdJ7gduT7IJeBL4cBv/feAKYBL4LfBxgKo6lORzwP1t3Ger6tDIOpEkzWjG0K+qx4FzjlH/FXDJMeoFbJnmubYD2+c+TUnSKPiNXEnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHVk1qGf5LQkDyS5s62fleTeJJNJvpnkNa3+2rY+2bavGXqOa1r9sSSXjbwbSdIrmsuR/tXA3qH1G4Abq+qtwGFgU6tvAg63+o1tHEnOBq4C3g6sB76c5LQTm74kaS5mFfpJVgHvA77S1gNcDHyrDdkBXNmWN7R12vZL2vgNwG1V9WJV/RKYBC4YQQ+SpFma7ZH+F4FPAb9v628Bnq+qI219P7CyLa8EngJo219o4/9QP8Zj/iDJ5iQTSSampqZm34kkaUYzhn6S9wMHq2rPPMyHqtpWVeuqat3y5cvn4yUlqRtLZjHmncAHklwBvA74U+AmYGmSJe1ofhVwoI0/AKwG9idZArwZ+NVQ/ajhx0iS5sGMR/pVdU1VraqqNQw+iL27qj4K3AN8sA3bCNzRlne2ddr2u6uqWv2qdnXPWcBa4L6RdSJJmtFsjvSn82ngtiSfBx4Abmn1W4CvJZkEDjF4o6CqHklyO/AocATYUlUvncDrS5LmaE6hX1U/An7Ulh/nGFffVNXvgA9N8/jrgOvmOklJ0mj4jVxJ6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1ZMbQT/K6JPcl+WmSR5J8ptXPSnJvkskk30zymlZ/bVufbNvXDD3XNa3+WJLLTlpXkqRjms2R/ovAxVV1DnAusD7JRcANwI1V9VbgMLCpjd8EHG71G9s4kpwNXAW8HVgPfDnJaSPsRZI0gxlDvwZ+01Zf3W4FXAx8q9V3AFe25Q1tnbb9kiRp9duq6sWq+iUwCVwwiiYkSbMzq3P6SU5L8iBwENgF/AJ4vqqOtCH7gZVteSXwFEDb/gLwluH6MR4z/Fqbk0wkmZiamppzQ5Kk6c0q9Kvqpao6F1jF4Oj8bSdrQlW1rarWVdW65cuXn6yXkaQuzenqnap6HrgHeAewNMmStmkVcKAtHwBWA7TtbwZ+NVw/xmMkSfNgNlfvLE+ytC2/HngvsJdB+H+wDdsI3NGWd7Z12va7q6pa/ap2dc9ZwFrgvhH1IUmahSUzD+FMYEe70uZVwO1VdWeSR4HbknweeAC4pY2/BfhakkngEIMrdqiqR5LcDjwKHAG2VNVLo21HkvRKZgz9qnoIOO8Y9cc5xtU3VfU74EPTPNd1wHVzn6YkaRT8Rq4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6MmPoJ1md5J4kjyZ5JMnVrX56kl1J9rX7Za2eJF9KMpnkoSTnDz3XxjZ+X5KNJ68tSdKxzOZI/wjwyao6G7gI2JLkbGArsLuq1gK72zrA5cDadtsM3AyDNwngWuBC4ALg2qNvFJKk+TFj6FfV01X1k7b838BeYCWwAdjRhu0ArmzLG4Bba+DHwNIkZwKXAbuq6lBVHQZ2AetH2Ywk6ZXN6Zx+kjXAecC9wIqqerptegZY0ZZXAk8NPWx/q01Xf/lrbE4ykWRiampqLtOTJM1g1qGf5I3At4FPVNWvh7dVVQE1iglV1baqWldV65YvXz6Kp5QkNbMK/SSvZhD4X6+q77Tys+20De3+YKsfAFYPPXxVq01XlyTNk9lcvRPgFmBvVX1haNNO4OgVOBuBO4bqH2tX8VwEvNBOA90FXJpkWfsA99JWkyTNkyWzGPNO4G+AnyV5sNX+DrgeuD3JJuBJ4MNt2/eBK4BJ4LfAxwGq6lCSzwH3t3GfrapDo2hCkjQ7M4Z+Vf0nkGk2X3KM8QVsmea5tgPb5zJBSdLo+I1cSeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I6YuhLUkcMfUnqiKEvSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSRwx9SeqIoS9JHTH0Jakjhr4kdWTG0E+yPcnBJA8P1U5PsivJvna/rNWT5EtJJpM8lOT8ocdsbOP3Jdl4ctqRJL2SJbMY81XgH4Fbh2pbgd1VdX2SrW3908DlwNp2uxC4GbgwyenAtcA6oIA9SXZW1eFRNTJqa7Z+77gf+8T17xvhTCRpdGY80q+q/wAOvay8AdjRlncAVw7Vb62BHwNLk5wJXAbsqqpDLeh3AetHMH9J0hwc7zn9FVX1dFt+BljRllcCTw2N299q09X/nySbk0wkmZiamjrO6UmSjuWEP8itqmJwymYkqmpbVa2rqnXLly8f1dNKkjj+0H+2nbah3R9s9QPA6qFxq1pturokaR4db+jvBI5egbMRuGOo/rF2Fc9FwAvtNNBdwKVJlrUrfS5tNUnSPJrx6p0k3wDeDZyRZD+Dq3CuB25Psgl4EvhwG/594ApgEvgt8HGAqjqU5HPA/W3cZ6vq5R8OS5JOshlDv6o+Ms2mS44xtoAt0zzPdmD7nGYnSRopv5ErSR0x9CWpI4a+JHXE0Jekjhj6ktQRQ1+SOmLoS1JHDH1J6oihL0kdMfQlqSOz+Z+zNEf+r1uSTlUe6UtSRwx9SeqIoS9JHTH0Jakjhr4kdcTQl6SOGPqS1BFDX5I64pezTjF+sUvSyeSRviR1xNCXpI4Y+pLUEc/pLyIn8nkA+JmA1AOP9CWpIx7p6w+8ckha/DzSl6SOeKSvkfBfCdJ4mPfQT7IeuAk4DfhKVV0/33PQqcU3DGn+zGvoJzkN+CfgvcB+4P4kO6vq0fmchxYPr1iS5ma+j/QvACar6nGAJLcBGwBDXwviRN80erNQb5L+a3B05jv0VwJPDa3vBy4cHpBkM7C5rf4myWPH+VpnAM8d52NPNYupF1hc/XTVS26Yp5mMxhnAc2M25+nMdT/78+k2nHIf5FbVNmDbiT5PkomqWjeCKS24xdQLLK5+7OXUtZj6GWUv833J5gFg9dD6qlaTJM2D+Q79+4G1Sc5K8hrgKmDnPM9Bkro1r6d3qupIkr8F7mJwyeb2qnrkJL3cCZ8iOoUspl5gcfVjL6euxdTPyHpJVY3quSRJpzh/hkGSOmLoS1JHFl3oJ1mf5LEkk0m2LvR85irJ9iQHkzw8VDs9ya4k+9r9soWc42wlWZ3kniSPJnkkydWtPnb9JHldkvuS/LT18plWPyvJvW1/+2a7QGFsJDktyQNJ7mzrY9lPkieS/CzJg0kmWm3s9rOjkixN8q0kP0+yN8k7RtXPogr9oZ95uBw4G/hIkrMXdlZz9lVg/ctqW4HdVbUW2N3Wx8ER4JNVdTZwEbCl/XmMYz8vAhdX1TnAucD6JBcBNwA3VtVbgcPApoWb4nG5Gtg7tD7O/bynqs4dup59HPezo24CflBVbwPOYfBnNJp+qmrR3IB3AHcNrV8DXLPQ8zqOPtYADw+tPwac2ZbPBB5b6DkeZ193MPjdpbHuB/gT4CcMvk3+HLCk1f9o/zvVbwy+J7MbuBi4E8i49gM8AZzxstpY7mfAm4Ff0i60GXU/i+pIn2P/zMPKBZrLKK2oqqfb8jPAioWczPFIsgY4D7iXMe2nnQp5EDgI7AJ+ATxfVUfakHHb374IfAr4fVt/C+PbTwE/TLKn/ZQLjOl+BpwFTAH/3E69fSXJGxhRP4st9Be9GrzNj9V1tkneCHwb+ERV/Xp42zj1U1UvVdW5DI6QLwDetrAzOn5J3g8crKo9Cz2XEXlXVZ3P4NTuliR/PbxxnPYzBt+fOh+4uarOA/6Hl53KOZF+FlvoL9afeXg2yZkA7f7gAs9n1pK8mkHgf72qvtPKY9sPQFU9D9zD4PTH0iRHv+Q4TvvbO4EPJHkCuI3BKZ6bGNN+qupAuz8IfJfBm/K47mf7gf1VdW9b/xaDN4GR9LPYQn+x/szDTmBjW97I4Nz4KS9JgFuAvVX1haFNY9dPkuVJlrbl1zP4bGIvg/D/YBs2Fr0AVNU1VbWqqtYw+Htyd1V9lDHsJ8kbkrzp6DJwKfAwY7ifAVTVM8BTSf6ylS5h8PPzo+lnoT+0OAkfglwB/BeD861/v9DzOY75fwN4GvhfBu/4mxica90N7AP+HTh9oec5y17exeCfoA8BD7bbFePYD/BXwAOtl4eBf2j1vwDuAyaBfwVeu9BzPY7e3g3cOa79tDn/tN0eOfr3fhz3s6GezgUm2v72b8CyUfXjzzBIUkcW2+kdSdIrMPQlqSOGviR1xNCXpI4Y+pLUEUNfkjpi6EtSR/4P/51CTrOmLY0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 1)\n",
    "\n",
    "ax.hist(segment_lengths, bins=20)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The corpus contains multiple files with differing numbers of speech segments. The segments have different lengths ranging from less than a second (probably hard to identify the speaker) to two minutes (easy to identify) of speech. All files contain a sufficient amount of speech and non-speech to test voice activity detection components of the pipelines. The equal speakers index gives the entropy of the multinomial speaker distribution relative to a distribution with all speakers speaking an equal amount of segments. The closer to 1 the index is, the more equally segments are distributed across speakers. This does not take into account the length of the segments, however, a short subset of a segment (5-10 secs) is usually sufficient to identify the speaker. Thus, it is more important to have equally distributed segments rather than equally distributed speaking times. In contrast, the equal speech time index is closer to one if the speakers speak an equal amount of time taken all segments together (it is sometimes larger than 1 because of overlapping segments). Both indices are close to 1 for most files, indicating that the dataset is fit for comparison."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "7347d991c5e2f391a3d7d79e5ee26bb4d126111ca53b3542519de53e7706b949"
  },
  "kernelspec": {
   "display_name": "Python 3.9.7 ('navigating-stories')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}