RepoAnalysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"font-size:250%\">But actually, we want to analyze repos ...</span>\n",
    "\n",
    "This notebook was aimed at developing the methodology to analyze repositories fast. Most of the early development history has been put into the [RepoAnalysis History](./RepoAnalysis_Historical.ipynb) notebook. This notebook mainly includes the actual three big analysis runs and the improvements to the analysis methods. From the results of the two notebooks, the [repoAnalysis.py](./repoAnalysis.py) module was created.\n",
    "\n",
    "Additionally the [RepoLibrarian](./RepoLibrarian.ipynb) notebook that aims at managing downloaded repos was split off this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from datetime import datetime\n",
    "from multiprocessing import Pool\n",
    "from multiprocessing import Semaphore\n",
    "import multiprocessing\n",
    "import sqlalchemy\n",
    "import pandas\n",
    "from IPython.utils import io\n",
    "from IPython.display import Audio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%aimport repoAnalysis\n",
    "%aimport repoLibrarian\n",
    "%aimport dbUtils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Preparation: Set repos work folder\n",
    "(Initially I downloaded the repositories to a more local folder, which I later used a small test set for developing analysis routines)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Local folder:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'./repos/'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.setReposFolder('./repos/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Shared folder:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/mnt/brick/crm20/repos/'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.setReposFolder('/mnt/brick/crm20/repos/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/mnt/brick/crm20/repos/'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.getReposFolder()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recap: Current state of analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point in time, the [repoAnalysis](./repoAnalysis.py) module already included the `calculateMetrics` method to calculate a given set of metrics for all files of a repository. A set of metrics has been developed (`repoAnalysis.metricSuite`). The result data should be written to database, which is why the analysis for each repository returns a pandas dataframe which can easily be written to table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prerequisites to run suites and generate database"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Metric suites could already be run on single repos. Now means had to be developed to do this for all repos and write the data into a given table. Note that column names have later been renamed to use snake case, because there were some errors with camel case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used for ('bptlab', 'scylla', 123456789): 65.44256091117859\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>file_count</th>\n",
       "      <th>num_methods</th>\n",
       "      <th>num_lambdas</th>\n",
       "      <th>num_comment_lines</th>\n",
       "      <th>num_reflection</th>\n",
       "      <th>num_snakes</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>37bcca3d0bc7b1f03818ecf24b15951b444b4f76</td>\n",
       "      <td>71e5cb71cd634f5170a4285caf2636308d5eb999</td>\n",
       "      <td>1590332677</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29844</td>\n",
       "      <td>25672</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4235</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57175.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>71e5cb71cd634f5170a4285caf2636308d5eb999</td>\n",
       "      <td>a49e5be01918f8012875befe18821676dc3ce98d</td>\n",
       "      <td>1590329536</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29859</td>\n",
       "      <td>25687</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4233</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57268.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>a49e5be01918f8012875befe18821676dc3ce98d</td>\n",
       "      <td>54ac50b59b456cdc156b4aa18c7be17db1511950</td>\n",
       "      <td>1588277133</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29849</td>\n",
       "      <td>25675</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4233</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57249.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>54ac50b59b456cdc156b4aa18c7be17db1511950</td>\n",
       "      <td>0c42536c5657adf2e910833cc72fea577d8ee1eb</td>\n",
       "      <td>1588096131</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29825</td>\n",
       "      <td>25648</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4230</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57069.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0c42536c5657adf2e910833cc72fea577d8ee1eb</td>\n",
       "      <td>81171bf7a031077d010518344a28b0d89c6f1e90</td>\n",
       "      <td>1588095924</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29781</td>\n",
       "      <td>25610</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4223</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>56867.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>475</th>\n",
       "      <td>411d51771ee6c714db13c23e8912086840b84f51</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1485386460</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12972</td>\n",
       "      <td>10850</td>\n",
       "      <td>158</td>\n",
       "      <td>691</td>\n",
       "      <td>15</td>\n",
       "      <td>1454</td>\n",
       "      <td>31</td>\n",
       "      <td>117</td>\n",
       "      <td>25773.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>476</th>\n",
       "      <td>e36dd96a1201ad35578a5e5d65a5db02135f4614</td>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>1484674213</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12966</td>\n",
       "      <td>10836</td>\n",
       "      <td>155</td>\n",
       "      <td>687</td>\n",
       "      <td>15</td>\n",
       "      <td>1463</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25851.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477</th>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1484673978</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12966</td>\n",
       "      <td>10836</td>\n",
       "      <td>155</td>\n",
       "      <td>687</td>\n",
       "      <td>15</td>\n",
       "      <td>1463</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25851.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>478</th>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>e1ca54916ff570656eb1d40c13b7e58c10d8bc0a</td>\n",
       "      <td>1484305822</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12922</td>\n",
       "      <td>10795</td>\n",
       "      <td>155</td>\n",
       "      <td>686</td>\n",
       "      <td>15</td>\n",
       "      <td>1466</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25716.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>479</th>\n",
       "      <td>e1ca54916ff570656eb1d40c13b7e58c10d8bc0a</td>\n",
       "      <td>None</td>\n",
       "      <td>1484305765</td>\n",
       "      <td>123456789</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>480 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          sha  \\\n",
       "0    37bcca3d0bc7b1f03818ecf24b15951b444b4f76   \n",
       "1    71e5cb71cd634f5170a4285caf2636308d5eb999   \n",
       "2    a49e5be01918f8012875befe18821676dc3ce98d   \n",
       "3    54ac50b59b456cdc156b4aa18c7be17db1511950   \n",
       "4    0c42536c5657adf2e910833cc72fea577d8ee1eb   \n",
       "..                                        ...   \n",
       "475  411d51771ee6c714db13c23e8912086840b84f51   \n",
       "476  e36dd96a1201ad35578a5e5d65a5db02135f4614   \n",
       "477  0d893a471f932b921473f72563a540da46f15fa5   \n",
       "478  9e8faf6f0b516334451c53960dee001a6e9e42c4   \n",
       "479  e1ca54916ff570656eb1d40c13b7e58c10d8bc0a   \n",
       "\n",
       "                                       parent   timestamp    repo_id    loc  \\\n",
       "0    71e5cb71cd634f5170a4285caf2636308d5eb999  1590332677  123456789  29844   \n",
       "1    a49e5be01918f8012875befe18821676dc3ce98d  1590329536  123456789  29859   \n",
       "2    54ac50b59b456cdc156b4aa18c7be17db1511950  1588277133  123456789  29849   \n",
       "3    0c42536c5657adf2e910833cc72fea577d8ee1eb  1588096131  123456789  29825   \n",
       "4    81171bf7a031077d010518344a28b0d89c6f1e90  1588095924  123456789  29781   \n",
       "..                                        ...         ...        ...    ...   \n",
       "475  9e8faf6f0b516334451c53960dee001a6e9e42c4  1485386460  123456789  12972   \n",
       "476  0d893a471f932b921473f72563a540da46f15fa5  1484674213  123456789  12966   \n",
       "477  9e8faf6f0b516334451c53960dee001a6e9e42c4  1484673978  123456789  12966   \n",
       "478  e1ca54916ff570656eb1d40c13b7e58c10d8bc0a  1484305822  123456789  12922   \n",
       "479                                      None  1484305765  123456789      0   \n",
       "\n",
       "      cloc  file_count  num_methods  num_lambdas  num_comment_lines  \\\n",
       "0    25672         276         1627          226               4235   \n",
       "1    25687         276         1627          226               4233   \n",
       "2    25675         276         1627          226               4233   \n",
       "3    25648         276         1627          226               4230   \n",
       "4    25610         276         1627          226               4223   \n",
       "..     ...         ...          ...          ...                ...   \n",
       "475  10850         158          691           15               1454   \n",
       "476  10836         155          687           15               1463   \n",
       "477  10836         155          687           15               1463   \n",
       "478  10795         155          686           15               1466   \n",
       "479      0           0            0            0                  0   \n",
       "\n",
       "     num_reflection  num_snakes  total_indent  \n",
       "0                93        1741      57175.75  \n",
       "1                93        1741      57268.75  \n",
       "2                93        1741      57249.75  \n",
       "3                93        1741      57069.75  \n",
       "4                93        1741      56867.00  \n",
       "..              ...         ...           ...  \n",
       "475              31         117      25773.50  \n",
       "476              30         117      25851.00  \n",
       "477              30         117      25851.00  \n",
       "478              30         117      25716.50  \n",
       "479               0           0          0.00  \n",
       "\n",
       "[480 rows x 13 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "suite = repoAnalysis.metricSuite\n",
    "data = repoAnalysis.calculateMetrics(('bptlab', 'scylla', 123456789), suite)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring database creation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sqlalchemy import MetaData, Table, Column, Integer, String"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sqlalchemy provides direct methods to created tables. As columns, the metric function names can be used. Attention, the schema name has to be set!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = [Column('sha', String), Column('parent', String), Column('timestamp', Integer), Column('repo_id', Integer)]\n",
    "columns = columns + list(map(lambda func: Column(func.__name__, Integer), suite))\n",
    "meta = MetaData(schema='crm20')\n",
    "tableName = 'lb_test2'\n",
    "table = Table(\n",
    "    tableName, meta,\n",
    "    *columns\n",
    ")\n",
    "meta.create_all(dbUtils.engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The table has been created successfully:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 355,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.3105020523071289\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>data_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>totalIndent</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>numReflection</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>numSnakes</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>timestamp</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>repoId</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>loc</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>cloc</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>fileCount</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>numMethods</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>numLambdas</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>numCommentLines</td>\n",
       "      <td>integer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>parent</td>\n",
       "      <td>character varying</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>sha</td>\n",
       "      <td>character varying</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        column_name          data_type\n",
       "0       totalIndent            integer\n",
       "1     numReflection            integer\n",
       "2         numSnakes            integer\n",
       "3         timestamp            integer\n",
       "4            repoId            integer\n",
       "5               loc            integer\n",
       "6              cloc            integer\n",
       "7         fileCount            integer\n",
       "8        numMethods            integer\n",
       "9        numLambdas            integer\n",
       "10  numCommentLines            integer\n",
       "11           parent  character varying\n",
       "12              sha  character varying"
      ]
     },
     "execution_count": 355,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT column_name, data_type \n",
    "    FROM information_schema.columns\n",
    "    WHERE table_name = '''+\"'\"+tableName+'''';\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because of the pandas dataframe, data can easily be written to the database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_sql(tableName, schema='crm20', con=dbUtils.engine, if_exists='append', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And has successfully been created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.005092144012451172\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>file_count</th>\n",
       "      <th>num_methods</th>\n",
       "      <th>num_lambdas</th>\n",
       "      <th>num_comment_lines</th>\n",
       "      <th>num_reflection</th>\n",
       "      <th>num_snakes</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>37bcca3d0bc7b1f03818ecf24b15951b444b4f76</td>\n",
       "      <td>71e5cb71cd634f5170a4285caf2636308d5eb999</td>\n",
       "      <td>1590332677</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29844</td>\n",
       "      <td>25672</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4235</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>71e5cb71cd634f5170a4285caf2636308d5eb999</td>\n",
       "      <td>a49e5be01918f8012875befe18821676dc3ce98d</td>\n",
       "      <td>1590329536</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29859</td>\n",
       "      <td>25687</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4233</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>a49e5be01918f8012875befe18821676dc3ce98d</td>\n",
       "      <td>54ac50b59b456cdc156b4aa18c7be17db1511950</td>\n",
       "      <td>1588277133</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29849</td>\n",
       "      <td>25675</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4233</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>54ac50b59b456cdc156b4aa18c7be17db1511950</td>\n",
       "      <td>0c42536c5657adf2e910833cc72fea577d8ee1eb</td>\n",
       "      <td>1588096131</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29825</td>\n",
       "      <td>25648</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4230</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>57070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0c42536c5657adf2e910833cc72fea577d8ee1eb</td>\n",
       "      <td>81171bf7a031077d010518344a28b0d89c6f1e90</td>\n",
       "      <td>1588095924</td>\n",
       "      <td>123456789</td>\n",
       "      <td>29781</td>\n",
       "      <td>25610</td>\n",
       "      <td>276</td>\n",
       "      <td>1627</td>\n",
       "      <td>226</td>\n",
       "      <td>4223</td>\n",
       "      <td>93</td>\n",
       "      <td>1741</td>\n",
       "      <td>56867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>475</th>\n",
       "      <td>411d51771ee6c714db13c23e8912086840b84f51</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1485386460</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12972</td>\n",
       "      <td>10850</td>\n",
       "      <td>158</td>\n",
       "      <td>691</td>\n",
       "      <td>15</td>\n",
       "      <td>1454</td>\n",
       "      <td>31</td>\n",
       "      <td>117</td>\n",
       "      <td>25774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>476</th>\n",
       "      <td>e36dd96a1201ad35578a5e5d65a5db02135f4614</td>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>1484674213</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12966</td>\n",
       "      <td>10836</td>\n",
       "      <td>155</td>\n",
       "      <td>687</td>\n",
       "      <td>15</td>\n",
       "      <td>1463</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477</th>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1484673978</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12966</td>\n",
       "      <td>10836</td>\n",
       "      <td>155</td>\n",
       "      <td>687</td>\n",
       "      <td>15</td>\n",
       "      <td>1463</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25851</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>478</th>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>e1ca54916ff570656eb1d40c13b7e58c10d8bc0a</td>\n",
       "      <td>1484305822</td>\n",
       "      <td>123456789</td>\n",
       "      <td>12922</td>\n",
       "      <td>10795</td>\n",
       "      <td>155</td>\n",
       "      <td>686</td>\n",
       "      <td>15</td>\n",
       "      <td>1466</td>\n",
       "      <td>30</td>\n",
       "      <td>117</td>\n",
       "      <td>25717</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>479</th>\n",
       "      <td>e1ca54916ff570656eb1d40c13b7e58c10d8bc0a</td>\n",
       "      <td>None</td>\n",
       "      <td>1484305765</td>\n",
       "      <td>123456789</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>480 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          sha  \\\n",
       "0    37bcca3d0bc7b1f03818ecf24b15951b444b4f76   \n",
       "1    71e5cb71cd634f5170a4285caf2636308d5eb999   \n",
       "2    a49e5be01918f8012875befe18821676dc3ce98d   \n",
       "3    54ac50b59b456cdc156b4aa18c7be17db1511950   \n",
       "4    0c42536c5657adf2e910833cc72fea577d8ee1eb   \n",
       "..                                        ...   \n",
       "475  411d51771ee6c714db13c23e8912086840b84f51   \n",
       "476  e36dd96a1201ad35578a5e5d65a5db02135f4614   \n",
       "477  0d893a471f932b921473f72563a540da46f15fa5   \n",
       "478  9e8faf6f0b516334451c53960dee001a6e9e42c4   \n",
       "479  e1ca54916ff570656eb1d40c13b7e58c10d8bc0a   \n",
       "\n",
       "                                       parent   timestamp    repo_id    loc  \\\n",
       "0    71e5cb71cd634f5170a4285caf2636308d5eb999  1590332677  123456789  29844   \n",
       "1    a49e5be01918f8012875befe18821676dc3ce98d  1590329536  123456789  29859   \n",
       "2    54ac50b59b456cdc156b4aa18c7be17db1511950  1588277133  123456789  29849   \n",
       "3    0c42536c5657adf2e910833cc72fea577d8ee1eb  1588096131  123456789  29825   \n",
       "4    81171bf7a031077d010518344a28b0d89c6f1e90  1588095924  123456789  29781   \n",
       "..                                        ...         ...        ...    ...   \n",
       "475  9e8faf6f0b516334451c53960dee001a6e9e42c4  1485386460  123456789  12972   \n",
       "476  0d893a471f932b921473f72563a540da46f15fa5  1484674213  123456789  12966   \n",
       "477  9e8faf6f0b516334451c53960dee001a6e9e42c4  1484673978  123456789  12966   \n",
       "478  e1ca54916ff570656eb1d40c13b7e58c10d8bc0a  1484305822  123456789  12922   \n",
       "479                                      None  1484305765  123456789      0   \n",
       "\n",
       "      cloc  file_count  num_methods  num_lambdas  num_comment_lines  \\\n",
       "0    25672         276         1627          226               4235   \n",
       "1    25687         276         1627          226               4233   \n",
       "2    25675         276         1627          226               4233   \n",
       "3    25648         276         1627          226               4230   \n",
       "4    25610         276         1627          226               4223   \n",
       "..     ...         ...          ...          ...                ...   \n",
       "475  10850         158          691           15               1454   \n",
       "476  10836         155          687           15               1463   \n",
       "477  10836         155          687           15               1463   \n",
       "478  10795         155          686           15               1466   \n",
       "479      0           0            0            0                  0   \n",
       "\n",
       "     num_reflection  num_snakes  total_indent  \n",
       "0                93        1741         57176  \n",
       "1                93        1741         57269  \n",
       "2                93        1741         57250  \n",
       "3                93        1741         57070  \n",
       "4                93        1741         56867  \n",
       "..              ...         ...           ...  \n",
       "475              31         117         25774  \n",
       "476              30         117         25851  \n",
       "477              30         117         25851  \n",
       "478              30         117         25717  \n",
       "479               0           0             0  \n",
       "\n",
       "[480 rows x 13 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT *\n",
    "    FROM crm20.'''+tableName+'''\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mistakes will be made and experiments need to be undertaking, so cleaning up is necessary:<br>\n",
    "Note: Because dropping a table does not return values, an error is thrown, this can be safely ignored for now "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "ename": "ResourceClosedError",
     "evalue": "This result object does not return rows. It has been closed automatically.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/result.py\u001b[0m in \u001b[0;36m_fetchall_impl\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1218\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1219\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfetchall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1220\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mAttributeError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'fetchall'",
      "\nThe above exception was the direct cause of the following exception:\n",
      "\u001b[0;31mResourceClosedError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-25-b93e26bec95c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m dbUtils.runQuery('''\n\u001b[0m\u001b[1;32m      2\u001b[0m     \u001b[0mDROP\u001b[0m \u001b[0mTABLE\u001b[0m \u001b[0mcrm20\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m'''+tableName+'''\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m ''')\n",
      "\u001b[0;32m~/jupyterNotebook/dbUtils.py\u001b[0m in \u001b[0;36mrunQuery\u001b[0;34m(query, mute)\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mrunQuery\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmute\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m     \u001b[0mstart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m     \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpandas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_sql_query\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     27\u001b[0m     \u001b[0;32mglobal\u001b[0m \u001b[0mlastResult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m     \u001b[0mlastResult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/pandas/io/sql.py\u001b[0m in \u001b[0;36mread_sql_query\u001b[0;34m(sql, con, index_col, coerce_float, params, parse_dates, chunksize)\u001b[0m\n\u001b[1;32m    324\u001b[0m     \"\"\"\n\u001b[1;32m    325\u001b[0m     \u001b[0mpandas_sql\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpandasSQL_builder\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcon\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 326\u001b[0;31m     return pandas_sql.read_query(\n\u001b[0m\u001b[1;32m    327\u001b[0m         \u001b[0msql\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    328\u001b[0m         \u001b[0mindex_col\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mindex_col\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/pandas/io/sql.py\u001b[0m in \u001b[0;36mread_query\u001b[0;34m(self, sql, index_col, coerce_float, parse_dates, params, chunksize)\u001b[0m\n\u001b[1;32m   1229\u001b[0m             )\n\u001b[1;32m   1230\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1231\u001b[0;31m             \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfetchall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1232\u001b[0m             frame = _wrap_result(\n\u001b[1;32m   1233\u001b[0m                 \u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/result.py\u001b[0m in \u001b[0;36mfetchall\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1275\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1276\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mBaseException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1277\u001b[0;31m             self.connection._handle_dbapi_exception(\n\u001b[0m\u001b[1;32m   1278\u001b[0m                 \u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcontext\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1279\u001b[0m             )\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_handle_dbapi_exception\u001b[0;34m(self, e, statement, parameters, cursor, context)\u001b[0m\n\u001b[1;32m   1483\u001b[0m                 )\n\u001b[1;32m   1484\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1485\u001b[0;31m                 \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc_info\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwith_traceback\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mexc_info\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1486\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1487\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/util/compat.py\u001b[0m in \u001b[0;36mraise_\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m    176\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    177\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 178\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mexception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    179\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    180\u001b[0m             \u001b[0;31m# credit to\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/result.py\u001b[0m in \u001b[0;36mfetchall\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1271\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1272\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1273\u001b[0;31m             \u001b[0ml\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprocess_rows\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_fetchall_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1274\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_soft_close\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1275\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/result.py\u001b[0m in \u001b[0;36m_fetchall_impl\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1219\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfetchall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1220\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mAttributeError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1221\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_non_result\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1222\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1223\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_non_result\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdefault\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/engine/result.py\u001b[0m in \u001b[0;36m_non_result\u001b[0;34m(self, default, err)\u001b[0m\n\u001b[1;32m   1223\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_non_result\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdefault\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1224\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_metadata\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1225\u001b[0;31m             util.raise_(\n\u001b[0m\u001b[1;32m   1226\u001b[0m                 exc.ResourceClosedError(\n\u001b[1;32m   1227\u001b[0m                     \u001b[0;34m\"This result object does not return rows. \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/sqlalchemy/util/compat.py\u001b[0m in \u001b[0;36mraise_\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m    176\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    177\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 178\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mexception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    179\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    180\u001b[0m             \u001b[0;31m# credit to\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mResourceClosedError\u001b[0m: This result object does not return rows. It has been closed automatically."
     ]
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    DROP TABLE crm20.'''+tableName+'''\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create utility functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the results from the previous section can be extracted to some utility functions. These can later be extracted to an appropriate module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the core, the `runSuite` function stands: Calculating metrics for a repo and then writing them to table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataBaseSemaphore = multiprocessing.Semaphore()\n",
    "def runSuite(repo):\n",
    "    with io.capture_output() as output:\n",
    "        data = repoAnalysis.calculateMetrics(repo, suite)\n",
    "        with dataBaseSemaphore:\n",
    "            data.to_sql(tableName, schema='crm20', con=dbUtils.engine, if_exists='append', index=False)\n",
    "            dbUtils.engine.dispose()\n",
    "    log(output)\n",
    "    return len(data) > 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this, the table has to be created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def createTable():\n",
    "    columns = [Column('sha', String), Column('parent', String), Column('timestamp', Integer), Column('repoId', Integer)]\n",
    "    columns = columns + list(map(lambda func: Column(func.__name__, Integer), suite))\n",
    "    meta = MetaData(schema='crm20')\n",
    "    table = Table(\n",
    "        tableName, meta,\n",
    "        *columns\n",
    "    )\n",
    "    meta.create_all(dbUtils.engine)\n",
    "    dbUtils.engine.dispose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, table deletion is needed for clean up of failed runs (which there were a lot). Exception handling is put around, because an error is returned as the querey does not return data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def deleteTable():\n",
    "    try:\n",
    "        dbUtils.runQuery('''\n",
    "            DROP TABLE crm20.'''+tableName+'''\n",
    "        ''')\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, it had shown that for longer running processes, display sessions are not adequate to hold outputs. Therefore, output was written to a log file. This function was later extracted to the [dbUtils](dbUtils.py) module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "logSemaphore = multiprocessing.Semaphore()\n",
    "\n",
    "def log(text):\n",
    "    with logSemaphore:\n",
    "        with open('log.txt', 'a') as file:\n",
    "            file.write('========= '+str(datetime.datetime.now())+' ==========\\n'+str(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# First iteration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After preparing a first iteration of analysis, the first run started with approx. 200 users (100 in each group)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "tableName = 'lb_results1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/mnt/brick/crm20/repos/'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.setReposFolder('/mnt/brick/crm20/repos/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now all the results from the other sub-projects are brought in:<br>\n",
    "The projects to analyze are taken from the [data explorer](DataExplorer.ipynb):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.37739133834838867\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.36337733268737793\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "polyglotProjects = dbUtils.runQuery('''SELECT * FROM lb_polyglotProjects''')\n",
    "controlgroupProjects = dbUtils.runQuery('''SELECT * FROM lb_controlgroupProjects''')\n",
    "bothProjects = pandas.concat([polyglotProjects, controlgroupProjects], axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repos are downloaded and managed with the help of the [repoLibrarian](repoLibrarian.py):<br>\n",
    "For this, table rows must however be preprocessed: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rowToTuple(x):\n",
    "    (index, row) = x\n",
    "    split = row['url'].split('/')\n",
    "    user = split[-2]\n",
    "    project = split[-1]\n",
    "    return (user, project, row['repo_id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Downloading repos takes longer time than expected and shows that there are many repos that are not accessible anymore. Many repos have shown to just have been renamed, however, implementing a github rename detection strategy is out of scope of this project. Also it is useful to filter the repositories by Java, so no unnecessary downloads occur."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Failed to check ('thiagoruis', 'dotNET-Grupo-2'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"tananaev/traccar-client\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/tananaev/traccar-client.git /mnt/brick/crm20/repos/tananaev/traccar-client.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/tananaev/traccar-client.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('tananaev', 'traccar-client'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"topicusonderwijs/wicket-openid\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/topicusonderwijs/wicket-openid.git /mnt/brick/crm20/repos/topicusonderwijs/wicket-openid.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/topicusonderwijs/wicket-openid.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('topicusonderwijs', 'wicket-openid'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dav009/graficadora-tweets\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dav009/graficadora-tweets.git /mnt/brick/crm20/repos/dav009/graficadora-tweets.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dav009/graficadora-tweets.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dav009', 'graficadora-tweets'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dav009/MWE-DictionaryExtractor\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dav009/MWE-DictionaryExtractor.git /mnt/brick/crm20/repos/dav009/MWE-DictionaryExtractor.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dav009/MWE-DictionaryExtractor.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dav009', 'MWE-DictionaryExtractor'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('OpenGamma', 'OG-RStats'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/jbehave-web\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/jbehave-web.git /mnt/brick/crm20/repos/sayems/jbehave-web.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/jbehave-web.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'jbehave-web'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/Selenium2\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/Selenium2.git /mnt/brick/crm20/repos/sayems/Selenium2.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/Selenium2.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'Selenium2'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/wd\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/wd.git /mnt/brick/crm20/repos/sayems/wd.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/wd.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'wd'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"leviwilson/bigshitfancy\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/leviwilson/bigshitfancy.git /mnt/brick/crm20/repos/leviwilson/bigshitfancy.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/leviwilson/bigshitfancy.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('leviwilson', 'bigshitfancy'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/tomorrow-theme\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/tomorrow-theme.git /mnt/brick/crm20/repos/sayems/tomorrow-theme.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/tomorrow-theme.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'tomorrow-theme'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"lejingw/jmscm\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/lejingw/jmscm.git /mnt/brick/crm20/repos/lejingw/jmscm.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/lejingw/jmscm.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('lejingw', 'jmscm'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/selenium-pageloadmetrics\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/selenium-pageloadmetrics.git /mnt/brick/crm20/repos/sayems/selenium-pageloadmetrics.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/selenium-pageloadmetrics.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'selenium-pageloadmetrics'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('sayems', 'jbehave-tutorial'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Frotty/Buttermilk2D\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Frotty/Buttermilk2D.git /mnt/brick/crm20/repos/Frotty/Buttermilk2D.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Frotty/Buttermilk2D.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Frotty', 'Buttermilk2D'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"joaoSakai/mendes-project\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/joaoSakai/mendes-project.git /mnt/brick/crm20/repos/joaoSakai/mendes-project.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/joaoSakai/mendes-project.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('joaoSakai', 'mendes-project'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"OpenGamma/OG-DotNet\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/OpenGamma/OG-DotNet.git /mnt/brick/crm20/repos/OpenGamma/OG-DotNet.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/OpenGamma/OG-DotNet.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('OpenGamma', 'OG-DotNet'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/webdriverjs\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/webdriverjs.git /mnt/brick/crm20/repos/sayems/webdriverjs.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/webdriverjs.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'webdriverjs'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sergiofbsilva/pyechonest\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sergiofbsilva/pyechonest.git /mnt/brick/crm20/repos/sergiofbsilva/pyechonest.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sergiofbsilva/pyechonest.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sergiofbsilva', 'pyechonest'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"topicusonderwijs/openedge\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/topicusonderwijs/openedge.git /mnt/brick/crm20/repos/topicusonderwijs/openedge.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/topicusonderwijs/openedge.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('topicusonderwijs', 'openedge'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/AlarmClock\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/AlarmClock.git /mnt/brick/crm20/repos/MaximAlien/AlarmClock.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/AlarmClock.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'AlarmClock'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/NewPageFactory\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/NewPageFactory.git /mnt/brick/crm20/repos/sayems/NewPageFactory.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/NewPageFactory.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'NewPageFactory'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ntietz/quines\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ntietz/quines.git /mnt/brick/crm20/repos/ntietz/quines.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ntietz/quines.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ntietz', 'quines'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/SmtpClient-for-Qt\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/SmtpClient-for-Qt.git /mnt/brick/crm20/repos/MaximAlien/SmtpClient-for-Qt.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/SmtpClient-for-Qt.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'SmtpClient-for-Qt'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('sayems', 'Test-Automation-Framework'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/automated-pageloadmetrics\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/automated-pageloadmetrics.git /mnt/brick/crm20/repos/sayems/automated-pageloadmetrics.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/automated-pageloadmetrics.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'automated-pageloadmetrics'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/BrowserMob\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/BrowserMob.git /mnt/brick/crm20/repos/sayems/BrowserMob.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/BrowserMob.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'BrowserMob'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/FluentLenium\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/FluentLenium.git /mnt/brick/crm20/repos/sayems/FluentLenium.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/FluentLenium.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'FluentLenium'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"topicusonderwijs/som-examenstatus\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/topicusonderwijs/som-examenstatus.git /mnt/brick/crm20/repos/topicusonderwijs/som-examenstatus.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/topicusonderwijs/som-examenstatus.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('topicusonderwijs', 'som-examenstatus'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"sayems/Ebselen\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/sayems/Ebselen.git /mnt/brick/crm20/repos/sayems/Ebselen.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/sayems/Ebselen.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('sayems', 'Ebselen'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/Android-ViewPagerIndicator\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/Android-ViewPagerIndicator.git /mnt/brick/crm20/repos/MaximAlien/Android-ViewPagerIndicator.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/Android-ViewPagerIndicator.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'Android-ViewPagerIndicator'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('MaximAlien', 'ExpenseManager'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/android-actionbarstylegenerator\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/android-actionbarstylegenerator.git /mnt/brick/crm20/repos/MaximAlien/android-actionbarstylegenerator.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/android-actionbarstylegenerator.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'android-actionbarstylegenerator'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"leviwilson/SpecFlow\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/leviwilson/SpecFlow.git /mnt/brick/crm20/repos/leviwilson/SpecFlow.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/leviwilson/SpecFlow.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('leviwilson', 'SpecFlow'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"slonka/project2_asm\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/slonka/project2_asm.git /mnt/brick/crm20/repos/slonka/project2_asm.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/slonka/project2_asm.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('slonka', 'project2_asm'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"topicusonderwijs/Font-Awesome\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/topicusonderwijs/Font-Awesome.git /mnt/brick/crm20/repos/topicusonderwijs/Font-Awesome.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/topicusonderwijs/Font-Awesome.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('topicusonderwijs', 'Font-Awesome'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/Spoon-Knife\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/Spoon-Knife.git /mnt/brick/crm20/repos/byronyi/Spoon-Knife.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/Spoon-Knife.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'Spoon-Knife'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/Spoon-Knife\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/Spoon-Knife.git /mnt/brick/crm20/repos/MaximAlien/Spoon-Knife.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/Spoon-Knife.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'Spoon-Knife'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"leviwilson/ruby_koans\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/leviwilson/ruby_koans.git /mnt/brick/crm20/repos/leviwilson/ruby_koans.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/leviwilson/ruby_koans.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('leviwilson', 'ruby_koans'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/linux\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/linux.git /mnt/brick/crm20/repos/MaximAlien/linux.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/linux.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'linux'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/android\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/android.git /mnt/brick/crm20/repos/MaximAlien/android.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/android.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'android'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/ActionBarSherlock\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/ActionBarSherlock.git /mnt/brick/crm20/repos/MaximAlien/ActionBarSherlock.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/ActionBarSherlock.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'ActionBarSherlock'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ntietz/Stonetolb\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ntietz/Stonetolb.git /mnt/brick/crm20/repos/ntietz/Stonetolb.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ntietz/Stonetolb.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ntietz', 'Stonetolb'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('MaximAlien', 'LazyList'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ntietz/ntietz.com\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ntietz/ntietz.com.git /mnt/brick/crm20/repos/ntietz/ntietz.com.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ntietz/ntietz.com.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ntietz', 'ntietz.com'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ascariandrea/JSON-js\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ascariandrea/JSON-js.git /mnt/brick/crm20/repos/ascariandrea/JSON-js.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ascariandrea/JSON-js.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ascariandrea', 'JSON-js'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dashorst/Font-Awesome\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dashorst/Font-Awesome.git /mnt/brick/crm20/repos/dashorst/Font-Awesome.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dashorst/Font-Awesome.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dashorst', 'Font-Awesome'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"NightDreamer/Kegelverein\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/NightDreamer/Kegelverein.git /mnt/brick/crm20/repos/NightDreamer/Kegelverein.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/NightDreamer/Kegelverein.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('NightDreamer', 'Kegelverein'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"jcarvalho/bennu\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/jcarvalho/bennu.git /mnt/brick/crm20/repos/jcarvalho/bennu.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/jcarvalho/bennu.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('jcarvalho', 'bennu'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('joaoSakai', 'tapa-buraco-app'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('raonicaselli', 'testeAula'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"CavC/testeAula\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/CavC/testeAula.git /mnt/brick/crm20/repos/CavC/testeAula.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/CavC/testeAula.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('CavC', 'testeAula'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Ankoul/gilson\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Ankoul/gilson.git /mnt/brick/crm20/repos/Ankoul/gilson.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Ankoul/gilson.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Ankoul', 'gilson'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"rsbarata/fenix\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/rsbarata/fenix.git /mnt/brick/crm20/repos/rsbarata/fenix.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/rsbarata/fenix.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('rsbarata', 'fenix'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LukaszRados/SolidOfRevolution\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LukaszRados/SolidOfRevolution.git /mnt/brick/crm20/repos/LukaszRados/SolidOfRevolution.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LukaszRados/SolidOfRevolution.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LukaszRados', 'SolidOfRevolution'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"btaylor1/FormsConversion\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/btaylor1/FormsConversion.git /mnt/brick/crm20/repos/btaylor1/FormsConversion.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/btaylor1/FormsConversion.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('btaylor1', 'FormsConversion'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"papegaaij/nl.topicus.m2e.javassist\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/papegaaij/nl.topicus.m2e.javassist.git /mnt/brick/crm20/repos/papegaaij/nl.topicus.m2e.javassist.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/papegaaij/nl.topicus.m2e.javassist.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('papegaaij', 'nl.topicus.m2e.javassist'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"PinkieSwirl/EscapeBuilder\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/PinkieSwirl/EscapeBuilder.git /mnt/brick/crm20/repos/PinkieSwirl/EscapeBuilder.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/PinkieSwirl/EscapeBuilder.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('PinkieSwirl', 'EscapeBuilder'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/caffe\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/caffe.git /mnt/brick/crm20/repos/byronyi/caffe.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/caffe.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'caffe'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('iamharleen', 'markup'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"iamharleen/SocialNetworkArtist_CMPE272_Group6\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/iamharleen/SocialNetworkArtist_CMPE272_Group6.git /mnt/brick/crm20/repos/iamharleen/SocialNetworkArtist_CMPE272_Group6.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/iamharleen/SocialNetworkArtist_CMPE272_Group6.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('iamharleen', 'SocialNetworkArtist_CMPE272_Group6'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouqianli/mdwiki-examples\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouqianli/mdwiki-examples.git /mnt/brick/crm20/repos/zhouqianli/mdwiki-examples.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouqianli/mdwiki-examples.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouqianli', 'mdwiki-examples'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouqianli/mdwiki\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouqianli/mdwiki.git /mnt/brick/crm20/repos/zhouqianli/mdwiki.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouqianli/mdwiki.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouqianli', 'mdwiki'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/Specs\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/Specs.git /mnt/brick/crm20/repos/MaximAlien/Specs.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/Specs.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'Specs'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MaximAlien/MMSliderPopover\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MaximAlien/MMSliderPopover.git /mnt/brick/crm20/repos/MaximAlien/MMSliderPopover.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MaximAlien/MMSliderPopover.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MaximAlien', 'MMSliderPopover'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Cherobin', 'AndroidBasico'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Ivesf1m/Jogo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Ivesf1m/Jogo.git /mnt/brick/crm20/repos/Ivesf1m/Jogo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Ivesf1m/Jogo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Ivesf1m', 'Jogo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Ivesf1m/LangApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Ivesf1m/LangApp.git /mnt/brick/crm20/repos/Ivesf1m/LangApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Ivesf1m/LangApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Ivesf1m', 'LangApp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Ivesf1m/WatchApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Ivesf1m/WatchApp.git /mnt/brick/crm20/repos/Ivesf1m/WatchApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Ivesf1m/WatchApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Ivesf1m', 'WatchApp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/Burn_CPU_Burn\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/Burn_CPU_Burn.git /mnt/brick/crm20/repos/byronyi/Burn_CPU_Burn.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/Burn_CPU_Burn.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'Burn_CPU_Burn'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('fkysly', 'k-vim'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/Treebook\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/Treebook.git /mnt/brick/crm20/repos/LarryAsanteBoateng/Treebook.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/Treebook.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'Treebook'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"skyrising/password-safe\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/skyrising/password-safe.git /mnt/brick/crm20/repos/skyrising/password-safe.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/skyrising/password-safe.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('skyrising', 'password-safe'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/HelloWorld\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/HelloWorld.git /mnt/brick/crm20/repos/dracojx/HelloWorld.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/HelloWorld.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'HelloWorld'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/ProjectManagement\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/ProjectManagement.git /mnt/brick/crm20/repos/dracojx/ProjectManagement.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/ProjectManagement.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'ProjectManagement'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Cherobin', 'AndroidAvancado'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/PIJavaMapping\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/PIJavaMapping.git /mnt/brick/crm20/repos/dracojx/PIJavaMapping.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/PIJavaMapping.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'PIJavaMapping'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/PITool\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/PITool.git /mnt/brick/crm20/repos/dracojx/PITool.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/PITool.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'PITool'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"RaiffRamalho/RaiffRamalho.github.io\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/RaiffRamalho/RaiffRamalho.github.io.git /mnt/brick/crm20/repos/RaiffRamalho/RaiffRamalho.github.io.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/RaiffRamalho/RaiffRamalho.github.io.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('RaiffRamalho', 'RaiffRamalho.github.io'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/free-programming-books-zh_CN\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/free-programming-books-zh_CN.git /mnt/brick/crm20/repos/byronyi/free-programming-books-zh_CN.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/free-programming-books-zh_CN.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'free-programming-books-zh_CN'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ekryvoshey/pllug\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ekryvoshey/pllug.git /mnt/brick/crm20/repos/ekryvoshey/pllug.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ekryvoshey/pllug.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ekryvoshey', 'pllug'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ist-dsi/fenixedu-docs\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ist-dsi/fenixedu-docs.git /mnt/brick/crm20/repos/ist-dsi/fenixedu-docs.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ist-dsi/fenixedu-docs.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ist-dsi', 'fenixedu-docs'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/ScrapWeibo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/ScrapWeibo.git /mnt/brick/crm20/repos/byronyi/ScrapWeibo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/ScrapWeibo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'ScrapWeibo'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('sergiofbsilva', 'test'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ntietz/plato\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ntietz/plato.git /mnt/brick/crm20/repos/ntietz/plato.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ntietz/plato.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ntietz', 'plato'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dav009/juegogit\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dav009/juegogit.git /mnt/brick/crm20/repos/dav009/juegogit.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dav009/juegogit.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dav009', 'juegogit'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/html\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/html.git /mnt/brick/crm20/repos/dracojx/html.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/html.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'html'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsante/PaperHacks\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsante/PaperHacks.git /mnt/brick/crm20/repos/LarryAsante/PaperHacks.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsante/PaperHacks.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsante', 'PaperHacks'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Gineton2/udacity\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Gineton2/udacity.git /mnt/brick/crm20/repos/Gineton2/udacity.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Gineton2/udacity.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Gineton2', 'udacity'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dracojx/projmgmt\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dracojx/projmgmt.git /mnt/brick/crm20/repos/dracojx/projmgmt.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dracojx/projmgmt.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dracojx', 'projmgmt'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"WonShaw/lianmeng2\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/WonShaw/lianmeng2.git /mnt/brick/crm20/repos/WonShaw/lianmeng2.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/WonShaw/lianmeng2.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('WonShaw', 'lianmeng2'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('EmmaBYPeng', 'Memory'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"PauloRobertoM/POO_SisBanc_CristianeLeandro_PauloRobertoMedeiros\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/PauloRobertoM/POO_SisBanc_CristianeLeandro_PauloRobertoMedeiros.git /mnt/brick/crm20/repos/PauloRobertoM/POO_SisBanc_CristianeLeandro_PauloRobertoMedeiros.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/PauloRobertoM/POO_SisBanc_CristianeLeandro_PauloRobertoMedeiros.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('PauloRobertoM', 'POO_SisBanc_CristianeLeandro_PauloRobertoMedeiros'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouliqiang/openfire-push-notification-plugin\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouliqiang/openfire-push-notification-plugin.git /mnt/brick/crm20/repos/zhouliqiang/openfire-push-notification-plugin.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouliqiang/openfire-push-notification-plugin.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouliqiang', 'openfire-push-notification-plugin'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/Holler\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/Holler.git /mnt/brick/crm20/repos/LarryAsanteBoateng/Holler.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/Holler.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'Holler'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsante/HollerApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsante/HollerApp.git /mnt/brick/crm20/repos/LarryAsante/HollerApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsante/HollerApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsante', 'HollerApp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsante/Ours\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsante/Ours.git /mnt/brick/crm20/repos/LarryAsante/Ours.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsante/Ours.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsante', 'Ours'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Connorcpu/Carbs-Alleviate-Hunger\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Connorcpu/Carbs-Alleviate-Hunger.git /mnt/brick/crm20/repos/Connorcpu/Carbs-Alleviate-Hunger.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Connorcpu/Carbs-Alleviate-Hunger.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Connorcpu', 'Carbs-Alleviate-Hunger'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('MaximAlien', 'QVideoWidget'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"RaiffRamalho/EroticRoulette\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/RaiffRamalho/EroticRoulette.git /mnt/brick/crm20/repos/RaiffRamalho/EroticRoulette.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/RaiffRamalho/EroticRoulette.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('RaiffRamalho', 'EroticRoulette'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"obs619/less-sugar\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/obs619/less-sugar.git /mnt/brick/crm20/repos/obs619/less-sugar.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/obs619/less-sugar.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('obs619', 'less-sugar'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"RaiffRamalho/testJenkins\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/RaiffRamalho/testJenkins.git /mnt/brick/crm20/repos/RaiffRamalho/testJenkins.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/RaiffRamalho/testJenkins.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('RaiffRamalho', 'testJenkins'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"chuckiefan/ShopingSystem\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/chuckiefan/ShopingSystem.git /mnt/brick/crm20/repos/chuckiefan/ShopingSystem.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/chuckiefan/ShopingSystem.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('chuckiefan', 'ShopingSystem'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"chuckiefan/ShoppingSystem\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/chuckiefan/ShoppingSystem.git /mnt/brick/crm20/repos/chuckiefan/ShoppingSystem.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/chuckiefan/ShoppingSystem.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('chuckiefan', 'ShoppingSystem'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fkysly/myblog\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fkysly/myblog.git /mnt/brick/crm20/repos/fkysly/myblog.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fkysly/myblog.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fkysly', 'myblog'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('byronyi', 'cubes'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"jbalves/mobile\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/jbalves/mobile.git /mnt/brick/crm20/repos/jbalves/mobile.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/jbalves/mobile.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('jbalves', 'mobile'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/bubbles\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/bubbles.git /mnt/brick/crm20/repos/byronyi/bubbles.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/bubbles.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'bubbles'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/CircularImageView\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/CircularImageView.git /mnt/brick/crm20/repos/LarryAsanteBoateng/CircularImageView.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/CircularImageView.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'CircularImageView'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouliqiang/lehoo-app-android\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouliqiang/lehoo-app-android.git /mnt/brick/crm20/repos/zhouliqiang/lehoo-app-android.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouliqiang/lehoo-app-android.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouliqiang', 'lehoo-app-android'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/EmailHotlistNew\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/EmailHotlistNew.git /mnt/brick/crm20/repos/muthutent/EmailHotlistNew.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/EmailHotlistNew.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'EmailHotlistNew'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/T3LM\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/T3LM.git /mnt/brick/crm20/repos/muthutent/T3LM.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/T3LM.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'T3LM'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/vim_bridge\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/vim_bridge.git /mnt/brick/crm20/repos/byronyi/vim_bridge.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/vim_bridge.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'vim_bridge'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/vimpdb\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/vimpdb.git /mnt/brick/crm20/repos/byronyi/vimpdb.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/vimpdb.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'vimpdb'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('muthutent', 'T3LMNew'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/Demo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/Demo.git /mnt/brick/crm20/repos/muthutent/Demo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/Demo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'Demo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/cdc.war\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/cdc.war.git /mnt/brick/crm20/repos/muthutent/cdc.war.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/cdc.war.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'cdc.war'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/CDC\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/CDC.git /mnt/brick/crm20/repos/muthutent/CDC.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/CDC.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'CDC'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/DataExportLib\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/DataExportLib.git /mnt/brick/crm20/repos/muthutent/DataExportLib.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/DataExportLib.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'DataExportLib'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/EmailHotlist2014\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/EmailHotlist2014.git /mnt/brick/crm20/repos/muthutent/EmailHotlist2014.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/EmailHotlist2014.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'EmailHotlist2014'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/AssignmentFive\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/AssignmentFive.git /mnt/brick/crm20/repos/LarryAsanteBoateng/AssignmentFive.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/AssignmentFive.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'AssignmentFive'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dav009/holaMundo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dav009/holaMundo.git /mnt/brick/crm20/repos/dav009/holaMundo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dav009/holaMundo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dav009', 'holaMundo'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('caskdata', 'cdap-apps'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('LarryAsante', 'CollegeBarter'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/hickle\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/hickle.git /mnt/brick/crm20/repos/byronyi/hickle.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/hickle.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'hickle'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/ImageProcessing\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/ImageProcessing.git /mnt/brick/crm20/repos/byronyi/ImageProcessing.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/ImageProcessing.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'ImageProcessing'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ponydevs/LoE-PrivateServer\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ponydevs/LoE-PrivateServer.git /mnt/brick/crm20/repos/ponydevs/LoE-PrivateServer.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ponydevs/LoE-PrivateServer.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ponydevs', 'LoE-PrivateServer'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouliqiang/CustomViewsSet\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouliqiang/CustomViewsSet.git /mnt/brick/crm20/repos/zhouliqiang/CustomViewsSet.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouliqiang/CustomViewsSet.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouliqiang', 'CustomViewsSet'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/byronyi.github.com\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/byronyi.github.com.git /mnt/brick/crm20/repos/byronyi/byronyi.github.com.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/byronyi.github.com.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'byronyi.github.com'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('muthutent', 'LMMobile'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/CDCear\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/CDCear.git /mnt/brick/crm20/repos/muthutent/CDCear.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/CDCear.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'CDCear'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/csc207-hw6\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/csc207-hw6.git /mnt/brick/crm20/repos/LarryAsanteBoateng/csc207-hw6.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/csc207-hw6.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'csc207-hw6'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fkysly/hicoder\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fkysly/hicoder.git /mnt/brick/crm20/repos/fkysly/hicoder.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fkysly/hicoder.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fkysly', 'hicoder'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"iltzortz/imrep\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/iltzortz/imrep.git /mnt/brick/crm20/repos/iltzortz/imrep.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/iltzortz/imrep.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('iltzortz', 'imrep'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fandul/Dice_Game_01\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fandul/Dice_Game_01.git /mnt/brick/crm20/repos/fandul/Dice_Game_01.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fandul/Dice_Game_01.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fandul', 'Dice_Game_01'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"chengzhanzhan/hello-world\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/chengzhanzhan/hello-world.git /mnt/brick/crm20/repos/chengzhanzhan/hello-world.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/chengzhanzhan/hello-world.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('chengzhanzhan', 'hello-world'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('chengzhanzhan', 'CrossValidation-for-libSVM'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"chengzhanzhan/ImageRetrieval\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/chengzhanzhan/ImageRetrieval.git /mnt/brick/crm20/repos/chengzhanzhan/ImageRetrieval.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/chengzhanzhan/ImageRetrieval.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('chengzhanzhan', 'ImageRetrieval'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/LMEJB\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/LMEJB.git /mnt/brick/crm20/repos/muthutent/LMEJB.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/LMEJB.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'LMEJB'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"chengzhanzhan/Online-Bookstore\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/chengzhanzhan/Online-Bookstore.git /mnt/brick/crm20/repos/chengzhanzhan/Online-Bookstore.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/chengzhanzhan/Online-Bookstore.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('chengzhanzhan', 'Online-Bookstore'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"obs619/beacon\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/obs619/beacon.git /mnt/brick/crm20/repos/obs619/beacon.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/obs619/beacon.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('obs619', 'beacon'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('fkysly', 'blogwritebymyself'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/PrivateLabel-WS\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/PrivateLabel-WS.git /mnt/brick/crm20/repos/muthutent/PrivateLabel-WS.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/PrivateLabel-WS.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'PrivateLabel-WS'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/PrivateLabel-wsbean\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/PrivateLabel-wsbean.git /mnt/brick/crm20/repos/muthutent/PrivateLabel-wsbean.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/PrivateLabel-wsbean.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'PrivateLabel-wsbean'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"muthutent/PrivateLabel-wslib\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/muthutent/PrivateLabel-wslib.git /mnt/brick/crm20/repos/muthutent/PrivateLabel-wslib.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/muthutent/PrivateLabel-wslib.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('muthutent', 'PrivateLabel-wslib'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/spark-bootstrap\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/spark-bootstrap.git /mnt/brick/crm20/repos/byronyi/spark-bootstrap.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/spark-bootstrap.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'spark-bootstrap'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('me2app', 'me2-assets'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"LarryAsanteBoateng/Nect\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/LarryAsanteBoateng/Nect.git /mnt/brick/crm20/repos/LarryAsanteBoateng/Nect.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/LarryAsanteBoateng/Nect.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('LarryAsanteBoateng', 'Nect'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fkysly/HTML4.01Spec_Chinese\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fkysly/HTML4.01Spec_Chinese.git /mnt/brick/crm20/repos/fkysly/HTML4.01Spec_Chinese.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fkysly/HTML4.01Spec_Chinese.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fkysly', 'HTML4.01Spec_Chinese'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zhouliqiang/zhouliqiang.github.io\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zhouliqiang/zhouliqiang.github.io.git /mnt/brick/crm20/repos/zhouliqiang/zhouliqiang.github.io.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zhouliqiang/zhouliqiang.github.io.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zhouliqiang', 'zhouliqiang.github.io'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"iltzortz/sample\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/iltzortz/sample.git /mnt/brick/crm20/repos/iltzortz/sample.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/iltzortz/sample.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('iltzortz', 'sample'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"byronyi/avazu\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/byronyi/avazu.git /mnt/brick/crm20/repos/byronyi/avazu.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/byronyi/avazu.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('byronyi', 'avazu'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mattflax/PhenotypeArchive\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mattflax/PhenotypeArchive.git /mnt/brick/crm20/repos/mattflax/PhenotypeArchive.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mattflax/PhenotypeArchive.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mattflax', 'PhenotypeArchive'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ascariandrea/react-semantic-ui\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ascariandrea/react-semantic-ui.git /mnt/brick/crm20/repos/ascariandrea/react-semantic-ui.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ascariandrea/react-semantic-ui.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ascariandrea', 'react-semantic-ui'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('obs619', 'procurement'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Armandi/BattleShip\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Armandi/BattleShip.git /mnt/brick/crm20/repos/Armandi/BattleShip.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Armandi/BattleShip.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Armandi', 'BattleShip'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"pedrorijo91/fenixedu-academic-thesis\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/pedrorijo91/fenixedu-academic-thesis.git /mnt/brick/crm20/repos/pedrorijo91/fenixedu-academic-thesis.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/pedrorijo91/fenixedu-academic-thesis.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('pedrorijo91', 'fenixedu-academic-thesis'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"gaoyangyiqiao/elearningWeb\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/gaoyangyiqiao/elearningWeb.git /mnt/brick/crm20/repos/gaoyangyiqiao/elearningWeb.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/gaoyangyiqiao/elearningWeb.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('gaoyangyiqiao', 'elearningWeb'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"favl/VersionControl\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/favl/VersionControl.git /mnt/brick/crm20/repos/favl/VersionControl.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/favl/VersionControl.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('favl', 'VersionControl'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"vsegouin/musichall\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/vsegouin/musichall.git /mnt/brick/crm20/repos/vsegouin/musichall.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/vsegouin/musichall.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('vsegouin', 'musichall'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"shadowinlife/Log-Analysis\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/shadowinlife/Log-Analysis.git /mnt/brick/crm20/repos/shadowinlife/Log-Analysis.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/shadowinlife/Log-Analysis.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('shadowinlife', 'Log-Analysis'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"linagora/james\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/linagora/james.git /mnt/brick/crm20/repos/linagora/james.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/linagora/james.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('linagora', 'james'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"kgmyh/test\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/kgmyh/test.git /mnt/brick/crm20/repos/kgmyh/test.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/kgmyh/test.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('kgmyh', 'test'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"kevinobruno/bananinhas\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/kevinobruno/bananinhas.git /mnt/brick/crm20/repos/kevinobruno/bananinhas.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/kevinobruno/bananinhas.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('kevinobruno', 'bananinhas'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"kevinobruno/foodSurvival\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/kevinobruno/foodSurvival.git /mnt/brick/crm20/repos/kevinobruno/foodSurvival.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/kevinobruno/foodSurvival.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('kevinobruno', 'foodSurvival'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"kevinobruno/foodSurvival2\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/kevinobruno/foodSurvival2.git /mnt/brick/crm20/repos/kevinobruno/foodSurvival2.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/kevinobruno/foodSurvival2.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('kevinobruno', 'foodSurvival2'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"rsbarata/bennu\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/rsbarata/bennu.git /mnt/brick/crm20/repos/rsbarata/bennu.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/rsbarata/bennu.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('rsbarata', 'bennu'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Dirichi/BikeParkingSF\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Dirichi/BikeParkingSF.git /mnt/brick/crm20/repos/Dirichi/BikeParkingSF.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Dirichi/BikeParkingSF.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Dirichi', 'BikeParkingSF'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"huobiapi/API_Docs\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/huobiapi/API_Docs.git /mnt/brick/crm20/repos/huobiapi/API_Docs.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/huobiapi/API_Docs.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('huobiapi', 'API_Docs'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('VilsonLu', 'TeddyPooFudz'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"syedsayem/syedsayem.github.io\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/syedsayem/syedsayem.github.io.git /mnt/brick/crm20/repos/syedsayem/syedsayem.github.io.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/syedsayem/syedsayem.github.io.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('syedsayem', 'syedsayem.github.io'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"huobiapi/API_Docs_en\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/huobiapi/API_Docs_en.git /mnt/brick/crm20/repos/huobiapi/API_Docs_en.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/huobiapi/API_Docs_en.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('huobiapi', 'API_Docs_en'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('momo-ozawa', 'library-catalog'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"TortueHuppee/starwars_garage\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/TortueHuppee/starwars_garage.git /mnt/brick/crm20/repos/TortueHuppee/starwars_garage.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/TortueHuppee/starwars_garage.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('TortueHuppee', 'starwars_garage'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"favl/ChatApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/favl/ChatApp.git /mnt/brick/crm20/repos/favl/ChatApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/favl/ChatApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('favl', 'ChatApp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"siddra/siddra.github.io\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/siddra/siddra.github.io.git /mnt/brick/crm20/repos/siddra/siddra.github.io.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/siddra/siddra.github.io.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('siddra', 'siddra.github.io'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"siddra/webdriver\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/siddra/webdriver.git /mnt/brick/crm20/repos/siddra/webdriver.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/siddra/webdriver.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('siddra', 'webdriver'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"patrickmlima/watter-bee\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/patrickmlima/watter-bee.git /mnt/brick/crm20/repos/patrickmlima/watter-bee.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/patrickmlima/watter-bee.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('patrickmlima', 'watter-bee'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Hanifor/SEandC-II\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Hanifor/SEandC-II.git /mnt/brick/crm20/repos/Hanifor/SEandC-II.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Hanifor/SEandC-II.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Hanifor', 'SEandC-II'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('momo-ozawa', 'thirsty-meetups'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"momo-ozawa/thirsty-calagator\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/momo-ozawa/thirsty-calagator.git /mnt/brick/crm20/repos/momo-ozawa/thirsty-calagator.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/momo-ozawa/thirsty-calagator.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('momo-ozawa', 'thirsty-calagator'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"favl/CA3semesterSeedSP\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/favl/CA3semesterSeedSP.git /mnt/brick/crm20/repos/favl/CA3semesterSeedSP.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/favl/CA3semesterSeedSP.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('favl', 'CA3semesterSeedSP'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"michael-munoz/ProjectCECS323\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/michael-munoz/ProjectCECS323.git /mnt/brick/crm20/repos/michael-munoz/ProjectCECS323.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/michael-munoz/ProjectCECS323.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('michael-munoz', 'ProjectCECS323'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"favl/CoolMcMondoThingy\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/favl/CoolMcMondoThingy.git /mnt/brick/crm20/repos/favl/CoolMcMondoThingy.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/favl/CoolMcMondoThingy.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('favl', 'CoolMcMondoThingy'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"favl/BizzAirline\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/favl/BizzAirline.git /mnt/brick/crm20/repos/favl/BizzAirline.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/favl/BizzAirline.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('favl', 'BizzAirline'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('NIAEFEUP', 'MindStorms'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('piotr-sarna', 'RESTnessMonsters'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Etto91/ProntoPane\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Etto91/ProntoPane.git /mnt/brick/crm20/repos/Etto91/ProntoPane.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Etto91/ProntoPane.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Etto91', 'ProntoPane'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Sermalenk', 'homework-4'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"The-Nico26/lorann\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/The-Nico26/lorann.git /mnt/brick/crm20/repos/The-Nico26/lorann.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/The-Nico26/lorann.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('The-Nico26', 'lorann'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Etto91/Hitter-Web\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Etto91/Hitter-Web.git /mnt/brick/crm20/repos/Etto91/Hitter-Web.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Etto91/Hitter-Web.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Etto91', 'Hitter-Web'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"HKUST-SING/librdmacm-cpp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/HKUST-SING/librdmacm-cpp.git /mnt/brick/crm20/repos/HKUST-SING/librdmacm-cpp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/HKUST-SING/librdmacm-cpp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('HKUST-SING', 'librdmacm-cpp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"flaxsearch/elasticsearch-training\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/flaxsearch/elasticsearch-training.git /mnt/brick/crm20/repos/flaxsearch/elasticsearch-training.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/flaxsearch/elasticsearch-training.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('flaxsearch', 'elasticsearch-training'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"d1ngn1gefe1/ActionPrediction\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/d1ngn1gefe1/ActionPrediction.git /mnt/brick/crm20/repos/d1ngn1gefe1/ActionPrediction.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/d1ngn1gefe1/ActionPrediction.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('d1ngn1gefe1', 'ActionPrediction'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"abraaobb/Zero-Diabetes-ANDROID\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/abraaobb/Zero-Diabetes-ANDROID.git /mnt/brick/crm20/repos/abraaobb/Zero-Diabetes-ANDROID.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/abraaobb/Zero-Diabetes-ANDROID.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('abraaobb', 'Zero-Diabetes-ANDROID'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"gleidesigner/GameGalaxian\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/gleidesigner/GameGalaxian.git /mnt/brick/crm20/repos/gleidesigner/GameGalaxian.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/gleidesigner/GameGalaxian.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('gleidesigner', 'GameGalaxian'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('at-anhlq', 'Webservice'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"FCNY/OrdersCSVImporter\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/FCNY/OrdersCSVImporter.git /mnt/brick/crm20/repos/FCNY/OrdersCSVImporter.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/FCNY/OrdersCSVImporter.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('FCNY', 'OrdersCSVImporter'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"FCNY/set-order-status-tool\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/FCNY/set-order-status-tool.git /mnt/brick/crm20/repos/FCNY/set-order-status-tool.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/FCNY/set-order-status-tool.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('FCNY', 'set-order-status-tool'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"aburasali/CS362W17Section-001\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/aburasali/CS362W17Section-001.git /mnt/brick/crm20/repos/aburasali/CS362W17Section-001.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/aburasali/CS362W17Section-001.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('aburasali', 'CS362W17Section-001'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"FCNY/base-api-client\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/FCNY/base-api-client.git /mnt/brick/crm20/repos/FCNY/base-api-client.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/FCNY/base-api-client.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('FCNY', 'base-api-client'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"pjw6193/caliber\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/pjw6193/caliber.git /mnt/brick/crm20/repos/pjw6193/caliber.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/pjw6193/caliber.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('pjw6193', 'caliber'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"jrmatos/an-unity-platform-game\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/jrmatos/an-unity-platform-game.git /mnt/brick/crm20/repos/jrmatos/an-unity-platform-game.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/jrmatos/an-unity-platform-game.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('jrmatos', 'an-unity-platform-game'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"luisnguyen/School-Mate\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/luisnguyen/School-Mate.git /mnt/brick/crm20/repos/luisnguyen/School-Mate.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/luisnguyen/School-Mate.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('luisnguyen', 'School-Mate'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fernandodantasfilho/heroes-of-the-jungle\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fernandodantasfilho/heroes-of-the-jungle.git /mnt/brick/crm20/repos/fernandodantasfilho/heroes-of-the-jungle.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fernandodantasfilho/heroes-of-the-jungle.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fernandodantasfilho', 'heroes-of-the-jungle'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('hoanghaockpt195', 'AdminShip'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"benthehorn/Neo4JAssignment\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/benthehorn/Neo4JAssignment.git /mnt/brick/crm20/repos/benthehorn/Neo4JAssignment.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/benthehorn/Neo4JAssignment.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('benthehorn', 'Neo4JAssignment'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"jakubjurena/git-workshop\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/jakubjurena/git-workshop.git /mnt/brick/crm20/repos/jakubjurena/git-workshop.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/jakubjurena/git-workshop.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('jakubjurena', 'git-workshop'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"jrmatos/vampire-runner\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/jrmatos/vampire-runner.git /mnt/brick/crm20/repos/jrmatos/vampire-runner.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/jrmatos/vampire-runner.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('jrmatos', 'vampire-runner'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dakeenekid/mokuvision\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dakeenekid/mokuvision.git /mnt/brick/crm20/repos/dakeenekid/mokuvision.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dakeenekid/mokuvision.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dakeenekid', 'mokuvision'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"optimizme/inversify-express-utils\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/optimizme/inversify-express-utils.git /mnt/brick/crm20/repos/optimizme/inversify-express-utils.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/optimizme/inversify-express-utils.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('optimizme', 'inversify-express-utils'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('JulienMrgrd', 'JhipsterSpring5NoReact'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Herklos/ProjetX\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Herklos/ProjetX.git /mnt/brick/crm20/repos/Herklos/ProjetX.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Herklos/ProjetX.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Herklos', 'ProjetX'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"liuyong1352/javaweb-sercurtiy\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/liuyong1352/javaweb-sercurtiy.git /mnt/brick/crm20/repos/liuyong1352/javaweb-sercurtiy.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/liuyong1352/javaweb-sercurtiy.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('liuyong1352', 'javaweb-sercurtiy'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"CatShaq/catshaq.github.io\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/CatShaq/catshaq.github.io.git /mnt/brick/crm20/repos/CatShaq/catshaq.github.io.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/CatShaq/catshaq.github.io.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('CatShaq', 'catshaq.github.io'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"brunofrassetto/calculo-matriz-univali\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/brunofrassetto/calculo-matriz-univali.git /mnt/brick/crm20/repos/brunofrassetto/calculo-matriz-univali.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/brunofrassetto/calculo-matriz-univali.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('brunofrassetto', 'calculo-matriz-univali'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Skalldihor/jugebra\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Skalldihor/jugebra.git /mnt/brick/crm20/repos/Skalldihor/jugebra.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Skalldihor/jugebra.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Skalldihor', 'jugebra'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Yrds', 'ComputacaoGrafica'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"salemove/twilio-auth\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/salemove/twilio-auth.git /mnt/brick/crm20/repos/salemove/twilio-auth.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/salemove/twilio-auth.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('salemove', 'twilio-auth'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Jean-GuillaumePonsard/PimpMyFridge\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Jean-GuillaumePonsard/PimpMyFridge.git /mnt/brick/crm20/repos/Jean-GuillaumePonsard/PimpMyFridge.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Jean-GuillaumePonsard/PimpMyFridge.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Jean-GuillaumePonsard', 'PimpMyFridge'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mintingle/teachable-machine\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mintingle/teachable-machine.git /mnt/brick/crm20/repos/mintingle/teachable-machine.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mintingle/teachable-machine.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mintingle', 'teachable-machine'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('calvin-liang', 'CalSupplyChain-UI'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"raouldeheer/TMRO-CCS\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/raouldeheer/TMRO-CCS.git /mnt/brick/crm20/repos/raouldeheer/TMRO-CCS.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/raouldeheer/TMRO-CCS.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('raouldeheer', 'TMRO-CCS'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"yutingzhao1991/umi\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/yutingzhao1991/umi.git /mnt/brick/crm20/repos/yutingzhao1991/umi.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/yutingzhao1991/umi.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('yutingzhao1991', 'umi'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"shaba81/JavaHeist\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/shaba81/JavaHeist.git /mnt/brick/crm20/repos/shaba81/JavaHeist.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/shaba81/JavaHeist.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('shaba81', 'JavaHeist'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"microstrongruan/project\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/microstrongruan/project.git /mnt/brick/crm20/repos/microstrongruan/project.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/microstrongruan/project.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('microstrongruan', 'project'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"abaines-phx/airfoil\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/abaines-phx/airfoil.git /mnt/brick/crm20/repos/abaines-phx/airfoil.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/abaines-phx/airfoil.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('abaines-phx', 'airfoil'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('AlfredHeller', 'Trailing'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"wsingleton/1803-MAR26-Java\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/wsingleton/1803-MAR26-Java.git /mnt/brick/crm20/repos/wsingleton/1803-MAR26-Java.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/wsingleton/1803-MAR26-Java.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('wsingleton', '1803-MAR26-Java'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"itepifanio/365TI\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/itepifanio/365TI.git /mnt/brick/crm20/repos/itepifanio/365TI.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/itepifanio/365TI.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('itepifanio', '365TI'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('danielsouzaaf', '365ti'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"steffres/parthenos_discovery\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/steffres/parthenos_discovery.git /mnt/brick/crm20/repos/steffres/parthenos_discovery.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/steffres/parthenos_discovery.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('steffres', 'parthenos_discovery'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"liankuGit/ColdVerify\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/liankuGit/ColdVerify.git /mnt/brick/crm20/repos/liankuGit/ColdVerify.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/liankuGit/ColdVerify.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('liankuGit', 'ColdVerify'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"huobiapi/REST-Java-demo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/huobiapi/REST-Java-demo.git /mnt/brick/crm20/repos/huobiapi/REST-Java-demo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/huobiapi/REST-Java-demo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('huobiapi', 'REST-Java-demo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"huobiapi/REST-Python3-demo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/huobiapi/REST-Python3-demo.git /mnt/brick/crm20/repos/huobiapi/REST-Python3-demo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/huobiapi/REST-Python3-demo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('huobiapi', 'REST-Python3-demo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"luchizina/AplicacionWebEq2\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/luchizina/AplicacionWebEq2.git /mnt/brick/crm20/repos/luchizina/AplicacionWebEq2.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/luchizina/AplicacionWebEq2.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('luchizina', 'AplicacionWebEq2'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"wsingleton/1810-Oct08-Java-USF\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/wsingleton/1810-Oct08-Java-USF.git /mnt/brick/crm20/repos/wsingleton/1810-Oct08-Java-USF.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/wsingleton/1810-Oct08-Java-USF.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('wsingleton', '1810-Oct08-Java-USF'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('jluctamar', 'KevinsCraftBacon-Back'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"maqiang34/smrt_verify\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/maqiang34/smrt_verify.git /mnt/brick/crm20/repos/maqiang34/smrt_verify.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/maqiang34/smrt_verify.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('maqiang34', 'smrt_verify'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"lugkhast/article-classifier\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/lugkhast/article-classifier.git /mnt/brick/crm20/repos/lugkhast/article-classifier.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/lugkhast/article-classifier.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('lugkhast', 'article-classifier'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"SiteView/Realtime-Panel\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/SiteView/Realtime-Panel.git /mnt/brick/crm20/repos/SiteView/Realtime-Panel.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/SiteView/Realtime-Panel.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('SiteView', 'Realtime-Panel'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"gjherbiet/textmate\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/gjherbiet/textmate.git /mnt/brick/crm20/repos/gjherbiet/textmate.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/gjherbiet/textmate.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('gjherbiet', 'textmate'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"klinovp/elk\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/klinovp/elk.git /mnt/brick/crm20/repos/klinovp/elk.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/klinovp/elk.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('klinovp', 'elk'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('luck3y', 'jboss-openjdk-orb'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/android-cropimage\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/android-cropimage.git /mnt/brick/crm20/repos/soundcloud/android-cropimage.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/android-cropimage.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'android-cropimage'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/Android-PullToRefresh\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/Android-PullToRefresh.git /mnt/brick/crm20/repos/soundcloud/Android-PullToRefresh.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/Android-PullToRefresh.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'Android-PullToRefresh'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"MercuryGroup/Project\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/MercuryGroup/Project.git /mnt/brick/crm20/repos/MercuryGroup/Project.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/MercuryGroup/Project.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('MercuryGroup', 'Project'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"gjherbiet/homebrew\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/gjherbiet/homebrew.git /mnt/brick/crm20/repos/gjherbiet/homebrew.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/gjherbiet/homebrew.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('gjherbiet', 'homebrew'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"bjoern-thalheim/captaincasa-multiplesortui\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/bjoern-thalheim/captaincasa-multiplesortui.git /mnt/brick/crm20/repos/bjoern-thalheim/captaincasa-multiplesortui.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/bjoern-thalheim/captaincasa-multiplesortui.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('bjoern-thalheim', 'captaincasa-multiplesortui'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('jselbo', 'SCMTest'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"gjherbiet/gitflow\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/gjherbiet/gitflow.git /mnt/brick/crm20/repos/gjherbiet/gitflow.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/gjherbiet/gitflow.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('gjherbiet', 'gitflow'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Parquet/parquet-mr\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Parquet/parquet-mr.git /mnt/brick/crm20/repos/Parquet/parquet-mr.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Parquet/parquet-mr.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Parquet', 'parquet-mr'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Buzz2Buzz/TestGitHub\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Buzz2Buzz/TestGitHub.git /mnt/brick/crm20/repos/Buzz2Buzz/TestGitHub.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Buzz2Buzz/TestGitHub.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Buzz2Buzz', 'TestGitHub'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Buzz2Buzz/ProjetJava\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Buzz2Buzz/ProjetJava.git /mnt/brick/crm20/repos/Buzz2Buzz/ProjetJava.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Buzz2Buzz/ProjetJava.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Buzz2Buzz', 'ProjetJava'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"AleksandarFaraj/psychic-octo-avenger\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/AleksandarFaraj/psychic-octo-avenger.git /mnt/brick/crm20/repos/AleksandarFaraj/psychic-octo-avenger.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/AleksandarFaraj/psychic-octo-avenger.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('AleksandarFaraj', 'psychic-octo-avenger'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ashleyj/RxJava\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ashleyj/RxJava.git /mnt/brick/crm20/repos/ashleyj/RxJava.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ashleyj/RxJava.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ashleyj', 'RxJava'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('dursunkoc', 'Miops'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/SDWebImage\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/SDWebImage.git /mnt/brick/crm20/repos/soundcloud/SDWebImage.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/SDWebImage.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'SDWebImage'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ZonalYewHD/VeritasVoting\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ZonalYewHD/VeritasVoting.git /mnt/brick/crm20/repos/ZonalYewHD/VeritasVoting.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ZonalYewHD/VeritasVoting.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ZonalYewHD', 'VeritasVoting'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ZonalYewHD/MechSuits\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ZonalYewHD/MechSuits.git /mnt/brick/crm20/repos/ZonalYewHD/MechSuits.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ZonalYewHD/MechSuits.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ZonalYewHD', 'MechSuits'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ZonalYewHD/BlockedCommands\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ZonalYewHD/BlockedCommands.git /mnt/brick/crm20/repos/ZonalYewHD/BlockedCommands.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ZonalYewHD/BlockedCommands.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ZonalYewHD', 'BlockedCommands'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud-cookbooks/icinga-cookbook\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud-cookbooks/icinga-cookbook.git /mnt/brick/crm20/repos/soundcloud-cookbooks/icinga-cookbook.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud-cookbooks/icinga-cookbook.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud-cookbooks', 'icinga-cookbook'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Zarkoix', 'ArborealWizardry'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/statiker\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/statiker.git /mnt/brick/crm20/repos/soundcloud/statiker.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/statiker.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'statiker'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/NineOldAndroids\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/NineOldAndroids.git /mnt/brick/crm20/repos/soundcloud/NineOldAndroids.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/NineOldAndroids.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'NineOldAndroids'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ZonalYewHD/Temp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ZonalYewHD/Temp.git /mnt/brick/crm20/repos/ZonalYewHD/Temp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ZonalYewHD/Temp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ZonalYewHD', 'Temp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"zombiepig333/Runes-of-Wizardry\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/zombiepig333/Runes-of-Wizardry.git /mnt/brick/crm20/repos/zombiepig333/Runes-of-Wizardry.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/zombiepig333/Runes-of-Wizardry.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('zombiepig333', 'Runes-of-Wizardry'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"frogsmelody/opensource\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/frogsmelody/opensource.git /mnt/brick/crm20/repos/frogsmelody/opensource.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/frogsmelody/opensource.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('frogsmelody', 'opensource'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Xilef11/GitTest\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Xilef11/GitTest.git /mnt/brick/crm20/repos/Xilef11/GitTest.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Xilef11/GitTest.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Xilef11', 'GitTest'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Carlos-Descalzi', '6502-ide'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/libcontainer\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/libcontainer.git /mnt/brick/crm20/repos/soundcloud/libcontainer.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/libcontainer.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'libcontainer'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"wuchaofei/themjn\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/wuchaofei/themjn.git /mnt/brick/crm20/repos/wuchaofei/themjn.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/wuchaofei/themjn.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('wuchaofei', 'themjn'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"reptar25/DragonGame\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/reptar25/DragonGame.git /mnt/brick/crm20/repos/reptar25/DragonGame.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/reptar25/DragonGame.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('reptar25', 'DragonGame'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/doozer-backup\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/doozer-backup.git /mnt/brick/crm20/repos/soundcloud/doozer-backup.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/doozer-backup.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'doozer-backup'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Nishnilanka/FundAdminstartionSystem\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Nishnilanka/FundAdminstartionSystem.git /mnt/brick/crm20/repos/Nishnilanka/FundAdminstartionSystem.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Nishnilanka/FundAdminstartionSystem.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Nishnilanka', 'FundAdminstartionSystem'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('soundcloud', 'xctool'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ashleyj/robovm-gradle-plugin\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ashleyj/robovm-gradle-plugin.git /mnt/brick/crm20/repos/ashleyj/robovm-gradle-plugin.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ashleyj/robovm-gradle-plugin.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ashleyj', 'robovm-gradle-plugin'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/specta\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/specta.git /mnt/brick/crm20/repos/soundcloud/specta.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/specta.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'specta'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Cosijopiii', 'Javandmore'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"dursunkoc/Miops_android\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/dursunkoc/Miops_android.git /mnt/brick/crm20/repos/dursunkoc/Miops_android.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/dursunkoc/Miops_android.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('dursunkoc', 'Miops_android'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ljr900723/Android_ZhouWidgets\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ljr900723/Android_ZhouWidgets.git /mnt/brick/crm20/repos/ljr900723/Android_ZhouWidgets.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ljr900723/Android_ZhouWidgets.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ljr900723', 'Android_ZhouWidgets'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/scrooge-thrift-gen-demo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/scrooge-thrift-gen-demo.git /mnt/brick/crm20/repos/soundcloud/scrooge-thrift-gen-demo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/scrooge-thrift-gen-demo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'scrooge-thrift-gen-demo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"fwu1/subProj\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/fwu1/subProj.git /mnt/brick/crm20/repos/fwu1/subProj.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/fwu1/subProj.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('fwu1', 'subProj'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"soundcloud/HttpResponseCache\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/soundcloud/HttpResponseCache.git /mnt/brick/crm20/repos/soundcloud/HttpResponseCache.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/soundcloud/HttpResponseCache.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('soundcloud', 'HttpResponseCache'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('LightningPig333', 'Runes-of-Wizardry'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"suxlv2/small-4f-proj\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/suxlv2/small-4f-proj.git /mnt/brick/crm20/repos/suxlv2/small-4f-proj.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/suxlv2/small-4f-proj.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('suxlv2', 'small-4f-proj'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"suxlv2/miExpenses\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/suxlv2/miExpenses.git /mnt/brick/crm20/repos/suxlv2/miExpenses.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/suxlv2/miExpenses.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('suxlv2', 'miExpenses'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"AkselBH/Test\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/AkselBH/Test.git /mnt/brick/crm20/repos/AkselBH/Test.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/AkselBH/Test.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('AkselBH', 'Test'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('Yashwa', 'Prematix'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"torthefurious/Resource-Conductor\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/torthefurious/Resource-Conductor.git /mnt/brick/crm20/repos/torthefurious/Resource-Conductor.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/torthefurious/Resource-Conductor.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('torthefurious', 'Resource-Conductor'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"schneidm17/CarStereo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/schneidm17/CarStereo.git /mnt/brick/crm20/repos/schneidm17/CarStereo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/schneidm17/CarStereo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('schneidm17', 'CarStereo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"RatherFunky/Group_5_Project_2015\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/RatherFunky/Group_5_Project_2015.git /mnt/brick/crm20/repos/RatherFunky/Group_5_Project_2015.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/RatherFunky/Group_5_Project_2015.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('RatherFunky', 'Group_5_Project_2015'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mhollingsworthAub/VidyaMap\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mhollingsworthAub/VidyaMap.git /mnt/brick/crm20/repos/mhollingsworthAub/VidyaMap.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mhollingsworthAub/VidyaMap.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mhollingsworthAub', 'VidyaMap'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"marcesfe/metodos\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/marcesfe/metodos.git /mnt/brick/crm20/repos/marcesfe/metodos.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/marcesfe/metodos.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('marcesfe', 'metodos'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('mduretete', 'cs371-GitLabStarter-master-gp4'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mduretete/NewGitLab\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mduretete/NewGitLab.git /mnt/brick/crm20/repos/mduretete/NewGitLab.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mduretete/NewGitLab.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mduretete', 'NewGitLab'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mhollingsworthAub/NLPtest\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mhollingsworthAub/NLPtest.git /mnt/brick/crm20/repos/mhollingsworthAub/NLPtest.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mhollingsworthAub/NLPtest.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mhollingsworthAub', 'NLPtest'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"mduretete/SJProj\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/mduretete/SJProj.git /mnt/brick/crm20/repos/mduretete/SJProj.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/mduretete/SJProj.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('mduretete', 'SJProj'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"amchristi/cs362w16\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/amchristi/cs362w16.git /mnt/brick/crm20/repos/amchristi/cs362w16.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/amchristi/cs362w16.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('amchristi', 'cs362w16'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"JStehouwer/RollingHamster\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/JStehouwer/RollingHamster.git /mnt/brick/crm20/repos/JStehouwer/RollingHamster.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/JStehouwer/RollingHamster.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('JStehouwer', 'RollingHamster'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ratlifri/urlval\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ratlifri/urlval.git /mnt/brick/crm20/repos/ratlifri/urlval.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ratlifri/urlval.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ratlifri', 'urlval'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Gigigo-Android-Devs/fabmenu\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Gigigo-Android-Devs/fabmenu.git /mnt/brick/crm20/repos/Gigigo-Android-Devs/fabmenu.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Gigigo-Android-Devs/fabmenu.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Gigigo-Android-Devs', 'fabmenu'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Gigigo-Android-Devs/SampleFabMenu\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Gigigo-Android-Devs/SampleFabMenu.git /mnt/brick/crm20/repos/Gigigo-Android-Devs/SampleFabMenu.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Gigigo-Android-Devs/SampleFabMenu.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Gigigo-Android-Devs', 'SampleFabMenu'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"matezz55/os_project\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/matezz55/os_project.git /mnt/brick/crm20/repos/matezz55/os_project.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/matezz55/os_project.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('matezz55', 'os_project'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"callesandstrom/BeaconBooking\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/callesandstrom/BeaconBooking.git /mnt/brick/crm20/repos/callesandstrom/BeaconBooking.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/callesandstrom/BeaconBooking.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('callesandstrom', 'BeaconBooking'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"guoxiaoqink/Instrument\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/guoxiaoqink/Instrument.git /mnt/brick/crm20/repos/guoxiaoqink/Instrument.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/guoxiaoqink/Instrument.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('guoxiaoqink', 'Instrument'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"GigigoGreenLabs/Communicator\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/GigigoGreenLabs/Communicator.git /mnt/brick/crm20/repos/GigigoGreenLabs/Communicator.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/GigigoGreenLabs/Communicator.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('GigigoGreenLabs', 'Communicator'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('GigigoGreenLabs', 'insightVisionDemo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"vaadin/vaadin-demo\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/vaadin/vaadin-demo.git /mnt/brick/crm20/repos/vaadin/vaadin-demo.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/vaadin/vaadin-demo.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('vaadin', 'vaadin-demo'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"P3tan/FJP_Project\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/P3tan/FJP_Project.git /mnt/brick/crm20/repos/P3tan/FJP_Project.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/P3tan/FJP_Project.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('P3tan', 'FJP_Project'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"C-S-D/amqp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/C-S-D/amqp.git /mnt/brick/crm20/repos/C-S-D/amqp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/C-S-D/amqp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('C-S-D', 'amqp'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('ball14886', 'JohnWick'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"GigigoGreenLabs/DeviceInfoLib\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/GigigoGreenLabs/DeviceInfoLib.git /mnt/brick/crm20/repos/GigigoGreenLabs/DeviceInfoLib.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/GigigoGreenLabs/DeviceInfoLib.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('GigigoGreenLabs', 'DeviceInfoLib'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"GigigoGreenLabs/AndroidSDKLib\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/GigigoGreenLabs/AndroidSDKLib.git /mnt/brick/crm20/repos/GigigoGreenLabs/AndroidSDKLib.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/GigigoGreenLabs/AndroidSDKLib.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('GigigoGreenLabs', 'AndroidSDKLib'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"nojdavid/MilkyWayApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/nojdavid/MilkyWayApp.git /mnt/brick/crm20/repos/nojdavid/MilkyWayApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/nojdavid/MilkyWayApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('nojdavid', 'MilkyWayApp'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('nojdavid', 'TheArtOfMilkProjectApp'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('soundcloud', 'build-time-tracker-plugin'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Mkean/SevenStarTechnology\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Mkean/SevenStarTechnology.git /mnt/brick/crm20/repos/Mkean/SevenStarTechnology.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Mkean/SevenStarTechnology.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Mkean', 'SevenStarTechnology'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"AvantikaYellapantula/chess\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/AvantikaYellapantula/chess.git /mnt/brick/crm20/repos/AvantikaYellapantula/chess.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/AvantikaYellapantula/chess.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('AvantikaYellapantula', 'chess'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ser316asu-2018/Hohenzollern\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ser316asu-2018/Hohenzollern.git /mnt/brick/crm20/repos/ser316asu-2018/Hohenzollern.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ser316asu-2018/Hohenzollern.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ser316asu-2018', 'Hohenzollern'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('agiamona', 'agiamona_review'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ViviKulu/SweetSeedsApp\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ViviKulu/SweetSeedsApp.git /mnt/brick/crm20/repos/ViviKulu/SweetSeedsApp.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ViviKulu/SweetSeedsApp.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ViviKulu', 'SweetSeedsApp'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Yvanovits/PrototypeVaavioV3\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Yvanovits/PrototypeVaavioV3.git /mnt/brick/crm20/repos/Yvanovits/PrototypeVaavioV3.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Yvanovits/PrototypeVaavioV3.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Yvanovits', 'PrototypeVaavioV3'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"Hector7500/Final-Project-Android-Watch\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/Hector7500/Final-Project-Android-Watch.git /mnt/brick/crm20/repos/Hector7500/Final-Project-Android-Watch.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/Hector7500/Final-Project-Android-Watch.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('Hector7500', 'Final-Project-Android-Watch'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ZeroPoints/FirstMod\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ZeroPoints/FirstMod.git /mnt/brick/crm20/repos/ZeroPoints/FirstMod.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ZeroPoints/FirstMod.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ZeroPoints', 'FirstMod'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"ViviKulu/TownhallNYC\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/ViviKulu/TownhallNYC.git /mnt/brick/crm20/repos/ViviKulu/TownhallNYC.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/ViviKulu/TownhallNYC.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('ViviKulu', 'TownhallNYC'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"krisztinamat/finalPractice\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/krisztinamat/finalPractice.git /mnt/brick/crm20/repos/krisztinamat/finalPractice.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/krisztinamat/finalPractice.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('krisztinamat', 'finalPractice'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"HCRenegade/OTren\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/HCRenegade/OTren.git /mnt/brick/crm20/repos/HCRenegade/OTren.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/HCRenegade/OTren.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('HCRenegade', 'OTren'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('souch3', 'group-19-native'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"souch3/group-19-web\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/souch3/group-19-web.git /mnt/brick/crm20/repos/souch3/group-19-web.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/souch3/group-19-web.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('souch3', 'group-19-web'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"krisztinamat/trie\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/krisztinamat/trie.git /mnt/brick/crm20/repos/krisztinamat/trie.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/krisztinamat/trie.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('krisztinamat', 'trie'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"tosyag/tiscon4\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/tosyag/tiscon4.git /mnt/brick/crm20/repos/tosyag/tiscon4.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/tosyag/tiscon4.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('tosyag', 'tiscon4'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"huangcs3/cloudtest\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/huangcs3/cloudtest.git /mnt/brick/crm20/repos/huangcs3/cloudtest.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/huangcs3/cloudtest.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('huangcs3', 'cloudtest'): Reference at 'refs/heads/master' does not exist\n",
      "Could not download repo \"AdrianDanlos/E-SportsProject\": Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git clone --bare -v https://github.com/AdrianDanlos/E-SportsProject.git /mnt/brick/crm20/repos/AdrianDanlos/E-SportsProject.git\n",
      "  stderr: 'Cloning into bare repository '/mnt/brick/crm20/repos/AdrianDanlos/E-SportsProject.git'...\n",
      "fatal: could not read Username for 'https://github.com': No such device or address\n",
      "'\n",
      "Failed to check ('AdrianDanlos', 'E-SportsProject'): Reference at 'refs/heads/master' does not exist\n"
     ]
    }
   ],
   "source": [
    "repos = list(filter(lambda tupl: repoLibrarian.isJavaRepo(tupl[0], tupl[1]), map(rowToTuple, bothProjects.iterrows())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compare how many repos of the experiment groups can actually be downloaded and are Java repos: Approx. half"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 700,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1377\n",
      "620\n"
     ]
    }
   ],
   "source": [
    "print(len(bothProjects))\n",
    "print(len(repos))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then the run started. The commented line shows that there were precursive runs with the reduced local dataset (and random repo ids).<br>\n",
    "The results of this run can be found in table `lb_results1` and their evaluation in the [Results_Iteration#1](Results_Iteration#1.ipynb) notebook. <br>\n",
    "The log for this run can be found under [./results/log_first_data_run.txt](./results/log_first_data_run.txt).\n",
    "It shows that approx. 2d were taken, which can surely be improved but which is good for a first large run.\n",
    "\n",
    "The Evaluation of the results can be found in the [Results_Iteration#1](Results_Iteration#1.ipynb) notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deleteTable()\n",
    "createTable()\n",
    "#repos = list(map(lambda tupl: (*tupl, int.from_bytes(bytearray(str(tupl), 'utf-8'), byteorder='big', signed=False) % 10000000), repoLibrarian.managedRepos()))\n",
    "start = time.time()\n",
    "with Pool(int(multiprocessing.cpu_count()/4)) as pool:\n",
    "    allMetrics = pool.map(runSuite, repos)\n",
    "end = time.time()\n",
    "dbUtils.log('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Second iteration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The analysis of the data of the first run has shown several shortcomings that need to be addressed before scaling up:\n",
    "\n",
    "Low severity:\n",
    "* Camel case column names lead to errors or need to be escaped\n",
    "* If the lifecycle analysis approach should be further pursued, last written date should be added to avoid warping the results of retired developers\n",
    "\n",
    "High severity:\n",
    "* `iter_commits` was missing the `--all` parameter so the analysis just ran on master branches - ignoring other branches!\n",
    "* Using commit size information from the raw_patches table decreases our data set significantly\n",
    "* The run took 2 days for 620 repositories. While this is already quite good, this makes it unfeasible to scale up by order of magnitude. If possible, opportunities for optimization should be taken"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The camel case problem can easily be solved by renaming the functions used in the `repoAnalysis.suite` (e.g. `num_lambdas` instead of `numLambdas`).\n",
    "\n",
    "As suggested, the lifeycycle problem can be solved by adding a column last_written to the `lb_languages` and dependent views. However, this second iteration should run on the same data as the first (to increase comparability of results). Recreating all views and accepting the losses therefore does not seem to be worth and will be postponed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding a new way to get commit changes information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "testRepoId = ('tfox12', 'REST-Debugger')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.hasRepo(*testRepoId)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from git.db import GitDB\n",
    "from git.db import GitCmdObjectDB\n",
    "from git import Repo "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "testRepo = Repo.init(repoLibrarian.pathFor(*testRepoId), bare=True, odbt=GitCmdObjectDB)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use gitpython commit object functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GitPython commit objects provide a stats attribute where diff information can be queried from.<br>\n",
    "However, this is order of magnitude worse, so unfeasible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used: 0.47133684158325195\n",
      "Total Time used: 5.964926481246948\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "for commit in testRepo.iter_commits():\n",
    "    for obj in commit.tree.traverse():\n",
    "        if obj.type == 'blob' and obj.name.endswith('.java'):\n",
    "            contentWithHeader = obj.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")\n",
    "end = time.time()\n",
    "print('Total Time used: '+str(end - start))\n",
    "\n",
    "start = time.time()\n",
    "res = []\n",
    "for commit in testRepo.iter_commits():\n",
    "    for obj in commit.tree.traverse():\n",
    "        if obj.type == 'blob' and obj.name.endswith('.java'):\n",
    "            contentWithHeader = obj.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")\n",
    "    res.append(sum(map(lambda file: file[1]['lines'], (filter(lambda file: file[0].endswith('.java'), commit.stats.files.items())))))\n",
    "end = time.time()\n",
    "print('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gitpython commits can also diff on the fly with parent commits. This takes approximately the same time so it does not work either. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used: 6.147816896438599\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "res = []\n",
    "for commit in testRepo.iter_commits():\n",
    "    for obj in commit.tree.traverse():\n",
    "        if obj.type == 'blob' and obj.name.endswith('.java'):\n",
    "            contentWithHeader = obj.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")\n",
    "    if len(commit.parents) == 1:\n",
    "        diff = testRepo.git.diff(commit.parents[0].hexsha, commit.hexsha, '--', numstat=True)\n",
    "        # can be further parsed\n",
    "end = time.time()\n",
    "print('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dissect gitpython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Using gitpython interface methods works, but the needed time is far too bad\n",
    "- It might be worth to look into internals of gitpython, iterating over commits (and all of them files!) is fast\n",
    "- Does this mean, gitpython uses a fast way to apply deltas? (when iterating over files is fast)\n",
    "- For this I looked up the gitpython sources at https://github.com/gitpython-developers/GitPython\n",
    "\n",
    "Aggregated Insights:\n",
    "- Iterating over objects of commit needs irritatingly little time -> iter_commits needs 99% of time\n",
    "- `iter_commits` iterates over revlist and creates commit for each; subcalls are\n",
    "    - `testRepo.git.rev_list(testRepo.head.commit))`\n",
    "    - `Commit._iter_from_process_or_stream`\n",
    "- The time is spend in `cmd.py>>_call_process` which is aliased with `__getattr__` --> this means gets called on attribute getting\n",
    "    - The call stack is `repo.iter_commits -> Commit.iter_items -> repo.git.rev_list` so it is the rev-list call that takes all the time\n",
    "    - Repo.git is of type `GitCommandWrapperType` which is `cmd.py>>Git`\n",
    "    - However, diff information is not included in the rev-list call (which just translates to a git command call)\n",
    "- `_iter_from_process_or_stream` parses hexshas from a stream and creates commits out of them\n",
    "- That means the file contents come directly from `data_stream` which calls odb.stream(binsha)\n",
    "- So file contents are accessed directly via a hash - no way for me to get the diff that way..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "81.9 ms ± 1.62 ms per loop (mean ± std. dev. of 7 runs, 20 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit -n 20\n",
    "commits = testRepo.iter_commits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "commits = testRepo.iter_commits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "171 ns ± 86.3 ns per loop (mean ± std. dev. of 7 runs, 20 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit -n 20\n",
    "for commit in commits:\n",
    "    for obj in commit.tree.traverse(predicate=lambda obj, depth: obj.type == 'blob' and obj.name.endswith('.java')):\n",
    "        contentWithHeader = obj.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " "
     ]
    },
    {
     "data": {
      "text/plain": [
       "         885 function calls (882 primitive calls) in 0.092 seconds\n",
       "\n",
       "   Ordered by: cumulative time\n",
       "   List reduced from 156 to 15 due to restriction <15>\n",
       "\n",
       "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
       "        1    0.000    0.000    0.092    0.092 {built-in method builtins.exec}\n",
       "        1    0.000    0.000    0.092    0.092 <string>:1(<module>)\n",
       "        1    0.000    0.000    0.091    0.091 base.py:504(iter_commits)\n",
       "        1    0.000    0.000    0.089    0.089 commit.py:186(iter_items)\n",
       "        1    0.000    0.000    0.089    0.089 cmd.py:542(<lambda>)\n",
       "        1    0.000    0.000    0.089    0.089 cmd.py:940(_call_process)\n",
       "        1    0.000    0.000    0.089    0.089 cmd.py:580(execute)\n",
       "        1    0.000    0.000    0.087    0.087 subprocess.py:732(__init__)\n",
       "        1    0.001    0.001    0.087    0.087 subprocess.py:1550(_execute_child)\n",
       "        1    0.063    0.063    0.063    0.063 {built-in method _posixsubprocess.fork_exec}\n",
       "        1    0.020    0.020    0.020    0.020 {built-in method posix.read}\n",
       "        1    0.000    0.000    0.001    0.001 symbolic.py:192(_get_commit)\n",
       "        1    0.000    0.000    0.001    0.001 symbolic.py:183(_get_object)\n",
       "        1    0.000    0.000    0.001    0.001 os.py:705(copy)\n",
       "        1    0.000    0.000    0.001    0.001 subprocess.py:1097(_close_pipe_fds)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%prun -l 15 -s cumulative testRepo.iter_commits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 469,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<git.cmd.Git.AutoInterrupt object at 0x7f54bb7e1970>\n",
      "ea393258a29f93539e371156cb77ea8eb2755b31\n",
      "8a94a36e5448dd83ef4dafc76cc7650e8b20f791\n",
      "cf84234e66b5a9940a3654b3bf6c17e8ec23e03c\n",
      "468a8e7939c7328d3af5b95fe8c6832966df1b52\n",
      "e78ae0397b209312ba80b709ba6996b558bb13bf\n",
      "d85a3642c97d44d6c4b0bcd145c68f420b3c2d96\n",
      "9dcfe3d3e498dfb06ae0f8c00dd1eda72bda070d\n",
      "81a242f338713512b8d6b52d2a228a2be2f70d27\n",
      "6fcc68689f3772ecaa1a6a011510ff26d7ee48d6\n",
      "3f93813edef9786f6cb8ced6a7441783da3f8884\n",
      "34503c976c764ce103baf2c10721ffa3656f20f8\n",
      "1b3e586b70006b9a21d9a35c5029c1e48cd95837\n",
      "a198e51031eadf8a013cc3711e37656f9edbe27b\n",
      "3682dc8523e86a00251a237258047429c1bb6cf9\n",
      "58c4dcce5ee8f618c0b12fd9053a79df8558b5a6\n",
      "0f4e0bb6b43d088baca4e874d2641c6bc3e427cf\n",
      "49f326042cf26b2b9ead9b675b6a830704092fa3\n",
      "5be9b9fa33c509896fc1ad3c0e9d09680135153f\n",
      "e7576bd7f72e7e5286d23bbad167cf3f394bed9c\n",
      "ed4dd39de749f287ad3036832ebcf24126140b72\n",
      "6c51645d3341163b45552ccfc47c162c8a0d0dc8\n",
      "377ccc918b4b7d3f390bcba84777e1555e501622\n",
      "0c5ddda62b6911aaec177be1f0e3d028dd94f94c\n",
      "9c6d5dd48018dc9d72d952913f3c3f9c92530574\n",
      "4048f135cdc7a3de2f3dc8408aec7994011bb58d\n",
      "c610b258b68939a57c7cf7a2359eb83335e10d4f\n",
      "eb10265604da5d0071123b90dcd6164e1aa3da7c\n",
      "6370564b9f337a427c1e6214e9e5ffe4a06014a6\n",
      "d85ff85719a0b836a9e4af66924be9c42ea19874\n",
      "5c778f7a888a37ada4a78f50ae55bd58cbd43694\n",
      "6afa0c165e08e8eb88909379676f8bb16f574308\n",
      "aeca813af17971b015dffd130e10fcbae260765f\n",
      "7cac0a6db617672c6f921766811f41f3dbf8e563\n",
      "7c54ff26f3e40d2d62aa3b9770ad69adc8f32d50\n",
      "026f1a12fd8a1823f2f75dbc29283923fcf990c2\n",
      "e0028014808b37e3150d20893d48ca1987df7c99\n",
      "cdd18557f374cd3dba16351f19c3f9cc426c9d54\n",
      "922fa79af5a4e47fac20316342d15f90639a641a\n",
      "104ee8cd45c19bcedc59af53ff7f054471a4cda9\n",
      "e2ea49d732d4a01f9cd81553f8cccae198b1ec96\n",
      "d57568287f74b37c9e21e8954530980ca89ea226\n",
      "72c76bea96266b27372408032b2a6e7fb24e6a14\n",
      "a7c393f22994febb066ebed97c0b104d561b9c1b\n",
      "0564e66d6840dec7a3e399bd02e10225b8012db2\n",
      "93e97fd82487e1b0bcd843e70b9766bd012cb0ea\n",
      "dd9a0063c05feb0259ea9dd0dbd99fe7ab7030ee\n",
      "6a0b3f3788374004f41aa6f58e3e294c4db5a4f1\n",
      "d6badf4a0f1eeba4f0399130f4f3010855610af6\n",
      "b7ce69459d073f51ce9c46e5733227cca4dfd165\n",
      "b33d6d11a09679022d947581cc1c37dc5ced1e09\n",
      "07c611facfda722498aa25dd35229d0d23c546da\n",
      "f88d9120ea06e396e00fb303749a5086489656e8\n",
      "eb95c1102d4e232d04290e07ddc335f356d111d0\n",
      "755ccb5f87eed83dac506a2f8fa46a902d027f24\n",
      "37174a63863ec6a01bd46f7c368aded38f84e8ea\n",
      "c83136da25c57dce83bdf74bc601ccd6f4dfa376\n",
      "873dda0f2315859d380abe8f33a83a88c85529e1\n",
      "0094f94b18f743c002e6bdba619186529aae27e0\n",
      "0b98da27c93f95b9c3638845b054c8eb1608e699\n",
      "7b4ccaf48a1fc7461d2fbccc10455903a537888e\n",
      "ce9f05e3a03cc052b2b75a86eb72ad5d3d705d24\n",
      "b6645b87f5a76cbf024938c2ccac2a6707b62ec7\n",
      "252234fd02a8338affaa419ccc05bd0db87a6759\n",
      "b09d0e5d97ed695511d7ac5b4962b435c0f564fc\n",
      "ebdd6a5d9f3ea6a469f82a563c58540ec540c045\n",
      "1e6327afa9d941e650a282455f664935d2780045\n",
      "73a12ce433514697679814d0f561c707fd224ba6\n",
      "33b3223c19a2b34895b5ff15115c5372b3a43bed\n",
      "432d4819a443134fda2aa5bac9f46b2e4868adc5\n",
      "d0b53e76978d835e64daffe71beb9adc110deae3\n",
      "2b4447896126df4feceb23b1f94d2fe95374e132\n",
      "4b26d1da08f8c1ae76cf1a8b77e63b871069cc44\n",
      "c45e0b6f242fabc7dc001943261c4256ff343836\n",
      "8e05c6a9309cf9178427d545dc313c9b984c1cc2\n",
      "536fee8cf83098689d7177ce79823c6c62984a2c\n"
     ]
    }
   ],
   "source": [
    "from git.objects import Commit\n",
    "proc = testRepo.git.rev_list(testRepo.head.commit, as_process=True)\n",
    "print(proc)\n",
    "commits = Commit._iter_from_process_or_stream(testRepo, proc)\n",
    "for commit in commits:\n",
    "    print(commit.hexsha)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manual diff approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As I now know that gitpython subprocesses are what costs most of the time, there are alternative approaches: Diffing manually can be fast and avoids cmds.<br>\n",
    "Therefore, I took an implementation of the myers algorithm (which is also used in git) Source: https://gist.github.com/tonyg/2361e3bfe4e92a1fc6f7 <br>\n",
    "Despite sounding like a bad idea this approach actually is quite good. One problem however is that the algorithm provides slightly different results than the git internal one. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Source: https://gist.github.com/tonyg/2361e3bfe4e92a1fc6f7\n",
    "\n",
    "# Copyright (c) 2015 Tony Garnock-Jones <tonyg@leastfixedpoint.com>\n",
    "#\n",
    "# Permission is hereby granted, free of charge, to any person\n",
    "# obtaining a copy of this software and associated documentation files\n",
    "# (the \"Software\"), to deal in the Software without restriction,\n",
    "# including without limitation the rights to use, copy, modify, merge,\n",
    "# publish, distribute, sublicense, and/or sell copies of the Software,\n",
    "# and to permit persons to whom the Software is furnished to do so,\n",
    "# subject to the following conditions:\n",
    "#\n",
    "# The above copyright notice and this permission notice shall be\n",
    "# included in all copies or substantial portions of the Software.\n",
    "#\n",
    "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\n",
    "# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n",
    "# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n",
    "# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\n",
    "# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\n",
    "# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n",
    "# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n",
    "# SOFTWARE.\n",
    "\n",
    "# Text diff algorithm after Myers 1986 and Ukkonen 1985, following\n",
    "# Levente Uzonyi's Squeak Smalltalk implementation at\n",
    "# http://squeaksource.com/DiffMerge.html\n",
    "#\n",
    "# E. W. Myers, \"An O(ND) difference algorithm and its variations,\"\n",
    "# Algorithmica, vol. 1, no. 1-4, pp. 251-266, Nov. 1986.\n",
    "#\n",
    "# E. Ukkonen, \"Algorithms for approximate string matching,\" Inf.\n",
    "# Control, vol. 64, no. 1-3, pp. 100-118, Jan. 1985.\n",
    "\n",
    "def longest_common_subsequence(xs, ys):\n",
    "    totallen = len(xs) + len(ys)\n",
    "    frontier = [0] * (2 * totallen + 1)\n",
    "    candidates = [None] * (2 * totallen + 1)\n",
    "    for d in range(totallen + 1):\n",
    "            for k in range(-d, d+1, 2):\n",
    "                    if k == -d or (k != d and frontier[totallen + k - 1] < frontier[totallen + k + 1]):\n",
    "                            index = totallen + k + 1\n",
    "                            x = frontier[index]\n",
    "                    else:\n",
    "                            index = totallen + k - 1\n",
    "                            x = frontier[index] + 1\n",
    "                    y = x - k\n",
    "                    chain = candidates[index]\n",
    "                    while x < len(xs) and y < len(ys) and xs[x] == ys[y]:\n",
    "                            chain = ((x, y), chain)\n",
    "                            x = x + 1\n",
    "                            y = y + 1\n",
    "                    if x >= len(xs) and y >= len(ys):\n",
    "                            result = []\n",
    "                            while chain:\n",
    "                                    result.append(chain[0])\n",
    "                                    chain = chain[1]\n",
    "                            result.reverse()\n",
    "                            return result\n",
    "                    frontier[totallen + k] = x\n",
    "                    candidates[totallen + k] = chain\n",
    "\n",
    "def myer_diff(xs, ys):\n",
    "        i = -1\n",
    "        j = -1\n",
    "        matches = longest_common_subsequence(xs, ys)\n",
    "        matches.append((len(xs), len(ys)))\n",
    "        result = []\n",
    "        for (mi, mj) in matches:\n",
    "                if mi - i > 1 or mj - j > 1:\n",
    "                        result.append((i + 1, mi - i - 1, j + 1, mj - j - 1))\n",
    "                i = mi\n",
    "                j = mj\n",
    "        return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used: 0.5856478214263916\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "for commit in testRepo.iter_commits():\n",
    "    if len(commit.parents) == 1:\n",
    "        #print(commit.hexsha)\n",
    "        for obj in commit.tree.traverse(predicate=lambda obj, depth: obj.type == 'blob' and obj.name.endswith('.java')):\n",
    "            try:\n",
    "                obj2 = commit.parents[0].tree / obj.path\n",
    "                content1 = obj.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")\n",
    "                content2 = obj2.data_stream.read().decode(\"CP437\")#.decode(\"utf-8\")\n",
    "                if not (content1 == content2):\n",
    "                    #print('\\tchanged '+obj.path)\n",
    "                    diff = myer_diff(content1.split('\\n'), content2.split('\\n'))\n",
    "                    additions = sum(map(lambda change: change[1], diff))\n",
    "                    deletions = sum(map(lambda change: change[3], diff))\n",
    "                    #print(str(additions)+'\\t'+str(deletions)+'\\t'+obj.path)\n",
    "            except KeyError:\n",
    "                pass\n",
    "                #print('\\tadded '+obj.path)\n",
    "        #print('---\\n'+testRepo.git.diff(commit.parents[0].hexsha, commit.hexsha, '--', numstat=True))\n",
    "        #print(testRepo.git.diff(commit.parents[0].hexsha, commit.hexsha))\n",
    "        #print('\\n')\n",
    "end = time.time()\n",
    "print('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### git log approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that iter_commits takes 90% of time, because it calls rev-list. Calling a git command for every commit or even every file increases cost by magnitude. So I opened a [stackoverflow thread](https://stackoverflow.com/questions/63115815/fast-way-to-get-number-of-changed-deleted-lines-per-subset-of-files-per-commit) which resulted in a tip to use `git log --numstat`. \n",
    "\n",
    "I used this approach to generate one machine readable full history all files commit diff log:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "//ea393258a29f93539e371156cb77ea8eb2755b31\n",
      "\n",
      "144\t0\tweb/lib/debugger.js\n",
      "//8a94a36e5448dd83ef4dafc76cc7650e8b20f791\n",
      "\n",
      "1\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "11\t5\tweb/test.html\n",
      "//cf84234e66b5a9940a3654b3bf6c17e8ec23e03c\n",
      "\n",
      "22\t2\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "//468a8e7939c7328d3af5b95fe8c6832966df1b52\n",
      "\n",
      "2\t1\tmakefile\n",
      "11\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "1\t1\tsrc/capstone/wrapper/PdbWrapper.java\n",
      "2\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "20\t108\tweb/test.html\n",
      "//e78ae0397b209312ba80b709ba6996b558bb13bf\n",
      "\n",
      "10\t6\tsrc/capstone/daemon/DaemonHandler.java\n",
      "231\t0\tsrc/capstone/wrapper/PdbWrapper.java\n",
      "129\t123\tsrc/capstone/wrapper/Wrapper.java\n",
      "31\t0\ttest/capstone/wrapper/PdbWrapperTest.java\n",
      "//d85a3642c97d44d6c4b0bcd145c68f420b3c2d96\n",
      "\n",
      "1\t0\t.gitignore\n",
      "//9dcfe3d3e498dfb06ae0f8c00dd1eda72bda070d\n",
      "\n",
      "5\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//81a242f338713512b8d6b52d2a228a2be2f70d27\n",
      "\n",
      "12\t2\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "1\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//6fcc68689f3772ecaa1a6a011510ff26d7ee48d6\n",
      "\n",
      "1\t1\tsrc/capstone/util/StackFrame.java\n",
      "//3f93813edef9786f6cb8ced6a7441783da3f8884\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "0\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "//34503c976c764ce103baf2c10721ffa3656f20f8\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//1b3e586b70006b9a21d9a35c5029c1e48cd95837\n",
      "\n",
      "1\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//a198e51031eadf8a013cc3711e37656f9edbe27b\n",
      "\n",
      "57\t3\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "2\t2\tsrc/capstone/wrapper/Wrapper.java\n",
      "5\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//3682dc8523e86a00251a237258047429c1bb6cf9\n",
      "\n",
      "28\t15\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "120\t13\tsrc/capstone/wrapper/Wrapper.java\n",
      "//58c4dcce5ee8f618c0b12fd9053a79df8558b5a6\n",
      "\n",
      "6\t3\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//0f4e0bb6b43d088baca4e874d2641c6bc3e427cf\n",
      "\n",
      "2\t2\tsrc/capstone/daemon/Daemon.java\n",
      "1\t1\tsrc/capstone/daemon/DaemonInitializer.java\n",
      "27\t9\tsrc/capstone/wrapper/Wrapper.java\n",
      "//49f326042cf26b2b9ead9b675b6a830704092fa3\n",
      "\n",
      "3\t2\tsrc/capstone/wrapper/Wrapper.java\n",
      "2\t2\tweb/test.html\n",
      "//5be9b9fa33c509896fc1ad3c0e9d09680135153f\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/Daemon.java\n",
      "1\t1\tweb/test.html\n",
      "//e7576bd7f72e7e5286d23bbad167cf3f394bed9c\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/Daemon.java\n",
      "1\t1\tweb/test.html\n",
      "//ed4dd39de749f287ad3036832ebcf24126140b72\n",
      "\n",
      "85\t4\tweb/test.html\n",
      "//6c51645d3341163b45552ccfc47c162c8a0d0dc8\n",
      "\n",
      "4\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "0\t54\ttest.html\n",
      "246\t0\tweb/lib/codemirror.css\n",
      "5516\t0\tweb/lib/codemirror.js\n",
      "160\t0\tweb/mode/apl/apl.js\n",
      "61\t0\tweb/mode/apl/index.html\n",
      "183\t0\tweb/mode/asterisk/asterisk.js\n",
      "142\t0\tweb/mode/asterisk/index.html\n",
      "302\t0\tweb/mode/clike/clike.js\n",
      "103\t0\tweb/mode/clike/index.html\n",
      "767\t0\tweb/mode/clike/scala.html\n",
      "223\t0\tweb/mode/clojure/clojure.js\n",
      "76\t0\tweb/mode/clojure/index.html\n",
      "22\t0\tweb/mode/coffeescript/LICENSE\n",
      "346\t0\tweb/mode/coffeescript/coffeescript.js\n",
      "728\t0\tweb/mode/coffeescript/index.html\n",
      "101\t0\tweb/mode/commonlisp/commonlisp.js\n",
      "165\t0\tweb/mode/commonlisp/index.html\n",
      "567\t0\tweb/mode/css/css.js\n",
      "58\t0\tweb/mode/css/index.html\n",
      "145\t0\tweb/mode/css/scss.html\n",
      "80\t0\tweb/mode/css/scss_test.js\n",
      "113\t0\tweb/mode/css/test.js\n",
      "205\t0\tweb/mode/d/d.js\n",
      "262\t0\tweb/mode/d/index.html\n",
      "32\t0\tweb/mode/diff/diff.js\n",
      "105\t0\tweb/mode/diff/index.html\n",
      "192\t0\tweb/mode/ecl/ecl.js\n",
      "39\t0\tweb/mode/ecl/index.html\n",
      "463\t0\tweb/mode/erlang/erlang.js\n",
      "64\t0\tweb/mode/erlang/index.html\n",
      "96\t0\tweb/mode/gfm/gfm.js\n",
      "74\t0\tweb/mode/gfm/index.html\n",
      "112\t0\tweb/mode/gfm/test.js\n",
      "165\t0\tweb/mode/go/go.js\n",
      "74\t0\tweb/mode/go/index.html\n",
      "210\t0\tweb/mode/groovy/groovy.js\n",
      "73\t0\tweb/mode/groovy/index.html\n",
      "242\t0\tweb/mode/haskell/haskell.js\n",
      "62\t0\tweb/mode/haskell/index.html\n",
      "429\t0\tweb/mode/haxe/haxe.js\n",
      "90\t0\tweb/mode/haxe/index.html\n",
      "73\t0\tweb/mode/htmlembedded/htmlembedded.js\n",
      "49\t0\tweb/mode/htmlembedded/index.html\n",
      "104\t0\tweb/mode/htmlmixed/htmlmixed.js\n",
      "73\t0\tweb/mode/htmlmixed/index.html\n",
      "98\t0\tweb/mode/http/http.js\n",
      "32\t0\tweb/mode/http/index.html\n",
      "88\t0\tweb/mode/javascript/index.html\n",
      "437\t0\tweb/mode/javascript/javascript.js\n",
      "48\t0\tweb/mode/javascript/typescript.html\n",
      "38\t0\tweb/mode/jinja2/index.html\n",
      "42\t0\tweb/mode/jinja2/jinja2.js\n",
      "741\t0\tweb/mode/less/index.html\n",
      "266\t0\tweb/mode/less/less.js\n",
      "23\t0\tweb/mode/livescript/LICENSE\n",
      "446\t0\tweb/mode/livescript/index.html\n",
      "267\t0\tweb/mode/livescript/livescript.js\n",
      "266\t0\tweb/mode/livescript/livescript.ls\n",
      "74\t0\tweb/mode/lua/index.html\n",
      "140\t0\tweb/mode/lua/lua.js\n",
      "344\t0\tweb/mode/markdown/index.html\n",
      "526\t0\tweb/mode/markdown/markdown.js\n",
      "636\t0\tweb/mode/markdown/test.js\n",
      "74\t0\tweb/mode/meta.js\n",
      "149\t0\tweb/mode/mirc/index.html\n",
      "177\t0\tweb/mode/mirc/mirc.js\n",
      "33\t0\tweb/mode/ntriples/index.html\n",
      "170\t0\tweb/mode/ntriples/ntriples.js\n",
      "131\t0\tweb/mode/ocaml/index.html\n",
      "113\t0\tweb/mode/ocaml/ocaml.js\n",
      "7\t0\tweb/mode/pascal/LICENSE\n",
      "48\t0\tweb/mode/pascal/index.html\n",
      "94\t0\tweb/mode/pascal/pascal.js\n",
      "19\t0\tweb/mode/perl/LICENSE\n",
      "62\t0\tweb/mode/perl/index.html\n",
      "816\t0\tweb/mode/perl/perl.js\n",
      "51\t0\tweb/mode/php/index.html\n",
      "129\t0\tweb/mode/php/php.js\n",
      "42\t0\tweb/mode/pig/index.html\n",
      "171\t0\tweb/mode/pig/pig.js\n",
      "41\t0\tweb/mode/properties/index.html\n",
      "63\t0\tweb/mode/properties/properties.js\n",
      "21\t0\tweb/mode/python/LICENSE.txt\n",
      "135\t0\tweb/mode/python/index.html\n",
      "340\t0\tweb/mode/python/python.js\n",
      "131\t0\tweb/mode/q/index.html\n",
      "124\t0\tweb/mode/q/q.js\n",
      "24\t0\tweb/mode/r/LICENSE\n",
      "74\t0\tweb/mode/r/index.html\n",
      "141\t0\tweb/mode/r/r.js\n",
      "19\t0\tweb/mode/rpm/changes/changes.js\n",
      "53\t0\tweb/mode/rpm/changes/index.html\n",
      "99\t0\tweb/mode/rpm/spec/index.html\n",
      "5\t0\tweb/mode/rpm/spec/spec.css\n",
      "66\t0\tweb/mode/rpm/spec/spec.js\n",
      "21\t0\tweb/mode/rst/LICENSE.txt\n",
      "524\t0\tweb/mode/rst/index.html\n",
      "550\t0\tweb/mode/rst/rst.js\n",
      "24\t0\tweb/mode/ruby/LICENSE\n",
      "173\t0\tweb/mode/ruby/index.html\n",
      "195\t0\tweb/mode/ruby/ruby.js\n",
      "48\t0\tweb/mode/rust/index.html\n",
      "432\t0\tweb/mode/rust/rust.js\n",
      "54\t0\tweb/mode/sass/index.html\n",
      "349\t0\tweb/mode/sass/sass.js\n",
      "65\t0\tweb/mode/scheme/index.html\n",
      "230\t0\tweb/mode/scheme/scheme.js\n",
      "51\t0\tweb/mode/shell/index.html\n",
      "118\t0\tweb/mode/shell/shell.js\n",
      "19\t0\tweb/mode/sieve/LICENSE\n",
      "81\t0\tweb/mode/sieve/index.html\n",
      "183\t0\tweb/mode/sieve/sieve.js\n",
      "57\t0\tweb/mode/smalltalk/index.html\n",
      "139\t0\tweb/mode/smalltalk/smalltalk.js\n",
      "83\t0\tweb/mode/smarty/index.html\n",
      "148\t0\tweb/mode/smarty/smarty.js\n",
      "42\t0\tweb/mode/sparql/index.html\n",
      "143\t0\tweb/mode/sparql/sparql.js\n",
      "68\t0\tweb/mode/sql/index.html\n",
      "268\t0\tweb/mode/sql/sql.js\n",
      "98\t0\tweb/mode/stex/index.html\n",
      "246\t0\tweb/mode/stex/stex.js\n",
      "117\t0\tweb/mode/stex/test.js\n",
      "129\t0\tweb/mode/tcl/index.html\n",
      "131\t0\tweb/mode/tcl/tcl.js\n",
      "142\t0\tweb/mode/tiddlywiki/index.html\n",
      "14\t0\tweb/mode/tiddlywiki/tiddlywiki.css\n",
      "353\t0\tweb/mode/tiddlywiki/tiddlywiki.js\n",
      "81\t0\tweb/mode/tiki/index.html\n",
      "26\t0\tweb/mode/tiki/tiki.css\n",
      "309\t0\tweb/mode/tiki/tiki.js\n",
      "39\t0\tweb/mode/turtle/index.html\n",
      "145\t0\tweb/mode/turtle/turtle.js\n",
      "21\t0\tweb/mode/vb/LICENSE.txt\n",
      "88\t0\tweb/mode/vb/index.html\n",
      "260\t0\tweb/mode/vb/vb.js\n",
      "42\t0\tweb/mode/vbscript/index.html\n",
      "26\t0\tweb/mode/vbscript/vbscript.js\n",
      "103\t0\tweb/mode/velocity/index.html\n",
      "144\t0\tweb/mode/velocity/velocity.js\n",
      "121\t0\tweb/mode/verilog/index.html\n",
      "182\t0\tweb/mode/verilog/verilog.js\n",
      "45\t0\tweb/mode/xml/index.html\n",
      "328\t0\tweb/mode/xml/xml.js\n",
      "20\t0\tweb/mode/xquery/LICENSE\n",
      "221\t0\tweb/mode/xquery/index.html\n",
      "64\t0\tweb/mode/xquery/test.js\n",
      "450\t0\tweb/mode/xquery/xquery.js\n",
      "68\t0\tweb/mode/yaml/index.html\n",
      "95\t0\tweb/mode/yaml/yaml.js\n",
      "39\t0\tweb/mode/z80/index.html\n",
      "113\t0\tweb/mode/z80/z80.js\n",
      "78\t0\tweb/test.html\n",
      "//377ccc918b4b7d3f390bcba84777e1555e501622\n",
      "\n",
      "2\t2\tsrc/capstone/wrapper/Wrapper.java\n",
      "14\t27\ttest.html\n",
      "//0c5ddda62b6911aaec177be1f0e3d028dd94f94c\n",
      "\n",
      "23\t11\tsrc/capstone/daemon/DaemonHandler.java\n",
      "31\t14\tsrc/capstone/wrapper/DebuggerCommand.java\n",
      "2\t2\tsrc/capstone/wrapper/Wrapper.java\n",
      "16\t3\ttest.html\n",
      "//9c6d5dd48018dc9d72d952913f3c3f9c92530574\n",
      "\n",
      "11\t5\tsrc/capstone/daemon/DaemonHandler.java\n",
      "25\t11\ttest.html\n",
      "//4048f135cdc7a3de2f3dc8408aec7994011bb58d\n",
      "\n",
      "1\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "1\t0\ttest.html\n",
      "//c610b258b68939a57c7cf7a2359eb83335e10d4f\n",
      "\n",
      "6\t5\ttest.html\n",
      "//eb10265604da5d0071123b90dcd6164e1aa3da7c\n",
      "\n",
      "38\t0\ttest.html\n",
      "//6370564b9f337a427c1e6214e9e5ffe4a06014a6\n",
      "\n",
      "8\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "0\t38\ttest.html\n",
      "//d85ff85719a0b836a9e4af66924be9c42ea19874\n",
      "\n",
      "18\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "1\t1\tsrc/capstone/daemon/DaemonInitializer.java\n",
      "22\t4\ttest.html\n",
      "//5c778f7a888a37ada4a78f50ae55bd58cbd43694\n",
      "\n",
      "12\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//6afa0c165e08e8eb88909379676f8bb16f574308\n",
      "\n",
      "-\t-\tlib/json-simple-1.1.1.jar\n",
      "//aeca813af17971b015dffd130e10fcbae260765f\n",
      "\n",
      "0\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//7cac0a6db617672c6f921766811f41f3dbf8e563\n",
      "\n",
      "4\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//7c54ff26f3e40d2d62aa3b9770ad69adc8f32d50\n",
      "\n",
      "1\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//026f1a12fd8a1823f2f75dbc29283923fcf990c2\n",
      "\n",
      "1\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "20\t11\tsrc/capstone/wrapper/Wrapper.java\n",
      "//e0028014808b37e3150d20893d48ca1987df7c99\n",
      "\n",
      "10\t13\tsrc/capstone/wrapper/Wrapper.java\n",
      "//cdd18557f374cd3dba16351f19c3f9cc426c9d54\n",
      "\n",
      "20\t0\ttest.html\n",
      "//922fa79af5a4e47fac20316342d15f90639a641a\n",
      "\n",
      "5\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//104ee8cd45c19bcedc59af53ff7f054471a4cda9\n",
      "\n",
      "3\t0\tmakefile\n",
      "1\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "0\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//e2ea49d732d4a01f9cd81553f8cccae198b1ec96\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//d57568287f74b37c9e21e8954530980ca89ea226\n",
      "\n",
      "4\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//72c76bea96266b27372408032b2a6e7fb24e6a14\n",
      "\n",
      "3\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "6\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//a7c393f22994febb066ebed97c0b104d561b9c1b\n",
      "\n",
      "1\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//0564e66d6840dec7a3e399bd02e10225b8012db2\n",
      "\n",
      "31\t1\tsrc/capstone/daemon/DaemonHandler.java\n",
      "1\t2\tsrc/capstone/wrapper/DebuggerRequest.java\n",
      "1\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//93e97fd82487e1b0bcd843e70b9766bd012cb0ea\n",
      "//dd9a0063c05feb0259ea9dd0dbd99fe7ab7030ee\n",
      "\n",
      "30\t3\tsrc/capstone/daemon/DaemonHandler.java\n",
      "//6a0b3f3788374004f41aa6f58e3e294c4db5a4f1\n",
      "\n",
      "6\t0\tsrc/capstone/util/ProgramError.java\n",
      "1\t2\tsrc/capstone/wrapper/Wrapper.java\n",
      "//d6badf4a0f1eeba4f0399130f4f3010855610af6\n",
      "\n",
      "4\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "80\t4\tsrc/capstone/wrapper/Wrapper.java\n",
      "//b7ce69459d073f51ce9c46e5733227cca4dfd165\n",
      "\n",
      "48\t16\tsrc/capstone/wrapper/Wrapper.java\n",
      "//b33d6d11a09679022d947581cc1c37dc5ced1e09\n",
      "\n",
      "17\t0\tsrc/capstone/daemon/DaemonInitializer.java\n",
      "//07c611facfda722498aa25dd35229d0d23c546da\n",
      "//f88d9120ea06e396e00fb303749a5086489656e8\n",
      "\n",
      "23\t11\tsrc/capstone/daemon/Daemon.java\n",
      "8\t6\tsrc/capstone/daemon/DaemonHandler.java\n",
      "0\t19\tsrc/capstone/daemon/DaemonPipelineFactory.java\n",
      "//eb95c1102d4e232d04290e07ddc335f356d111d0\n",
      "\n",
      "57\t30\tsrc/capstone/wrapper/DebuggerCommand.java\n",
      "33\t0\tsrc/capstone/wrapper/DebuggerRequest.java\n",
      "0\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "17\t16\tsrc/capstone/wrapper/Wrapper.java\n",
      "//755ccb5f87eed83dac506a2f8fa46a902d027f24\n",
      "\n",
      "0\t1\t.gitignore\n",
      "//37174a63863ec6a01bd46f7c368aded38f84e8ea\n",
      "\n",
      "2\t0\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "19\t4\tsrc/capstone/wrapper/Wrapper.java\n",
      "//c83136da25c57dce83bdf74bc601ccd6f4dfa376\n",
      "\n",
      "17\t18\tsrc/capstone/daemon/Daemon.java\n",
      "22\t0\tsrc/capstone/daemon/DaemonHandler.java\n",
      "19\t0\tsrc/capstone/daemon/DaemonPipelineFactory.java\n",
      "//873dda0f2315859d380abe8f33a83a88c85529e1\n",
      "\n",
      "18\t8\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "47\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//0094f94b18f743c002e6bdba619186529aae27e0\n",
      "\n",
      "1\t3\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "28\t3\tsrc/capstone/wrapper/Wrapper.java\n",
      "41\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//0b98da27c93f95b9c3638845b054c8eb1608e699\n",
      "\n",
      "10\t2\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//7b4ccaf48a1fc7461d2fbccc10455903a537888e\n",
      "\n",
      "1\t1\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//ce9f05e3a03cc052b2b75a86eb72ad5d3d705d24\n",
      "\n",
      "1\t0\tbuild.xml\n",
      "0\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "85\t11\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//b6645b87f5a76cbf024938c2ccac2a6707b62ec7\n",
      "\n",
      "95\t34\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "13\t12\tsrc/capstone/wrapper/Wrapper.java\n",
      "1\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//252234fd02a8338affaa419ccc05bd0db87a6759\n",
      "\n",
      "20\t1\tsrc/capstone/daemon/Daemon.java\n",
      "19\t8\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "//b09d0e5d97ed695511d7ac5b4962b435c0f564fc\n",
      "\n",
      "40\t9\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "27\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//ebdd6a5d9f3ea6a469f82a563c58540ec540c045\n",
      "\n",
      "10\t0\tmakefile\n",
      "2\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "0\t0\ttest/capstone/wrapper/GdbWrapperTest.java\n",
      "//1e6327afa9d941e650a282455f664935d2780045\n",
      "\n",
      "-\t-\textern/hamcrest-core-1.3.jar\n",
      "-\t-\textern/junit-4.11.jar\n",
      "63\t9\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "//73a12ce433514697679814d0f561c707fd224ba6\n",
      "\n",
      "22\t3\tsrc/capstone/wrapper/DebuggerCommand.java\n",
      "5\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "11\t1\tsrc/capstone/wrapper/Wrapper.java\n",
      "//33b3223c19a2b34895b5ff15115c5372b3a43bed\n",
      "\n",
      "10\t10\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "//432d4819a443134fda2aa5bac9f46b2e4868adc5\n",
      "\n",
      "-\t-\tlib/netty-all-4.0.0.CR1-sources.jar\n",
      "-\t-\tlib/netty-all-4.0.0.CR1.jar\n",
      "//d0b53e76978d835e64daffe71beb9adc110deae3\n",
      "\n",
      "43\t1\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "22\t20\tsrc/capstone/wrapper/Wrapper.java\n",
      "//2b4447896126df4feceb23b1f94d2fe95374e132\n",
      "\n",
      "42\t0\tsrc/capstone/wrapper/DebuggerCommand.java\n",
      "10\t0\tsrc/capstone/wrapper/GdbWrapper.java\n",
      "6\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//4b26d1da08f8c1ae76cf1a8b77e63b871069cc44\n",
      "\n",
      "15\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//c45e0b6f242fabc7dc001943261c4256ff343836\n",
      "\n",
      "3\t2\t.gitignore\n",
      "14\t0\tsrc/capstone/util/ProgramError.java\n",
      "16\t0\tsrc/capstone/util/ProgramVariable.java\n",
      "18\t0\tsrc/capstone/util/StackFrame.java\n",
      "10\t0\tsrc/capstone/wrapper/Wrapper.java\n",
      "//8e05c6a9309cf9178427d545dc313c9b984c1cc2\n",
      "\n",
      "94\t0\tbuild.xml\n",
      "10\t0\tsrc/capstone/daemon/Daemon.java\n",
      "//536fee8cf83098689d7177ce79823c6c62984a2c\n",
      "\n",
      "6\t0\t.gitignore\n",
      "4\t0\tREADME.md\n",
      "\n",
      "Total Time used: 0.15308237075805664\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "start = time.time()\n",
    "print(subprocess.check_output('git -C '+repoLibrarian.pathFor(*testRepoId)+' log --numstat --format=//%H', shell=True).decode('utf-8'))\n",
    "end = time.time()\n",
    "print('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding this to our current approach will double the run time, which would be ok. But, with all the knowledge about the inner workings of gitpython, rev-log is just used as text input to get all commit shas - this data is also included in the log! If we replace that with a git log --numstats, it's not too much slower than before but we get detailed diff information - we can filter that for java files and sum up and even pursue a diff-based approach instead of calculating the whole project metrics every time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used for new method: 13.67722201347351\n",
      "Total Time used for old method: 9.329468011856079\n"
     ]
    }
   ],
   "source": [
    "from git.util import hex_to_bin\n",
    "from git import Commit, Repo\n",
    "\n",
    "repo = Repo.init('./repos/bptlab/scylla.git', bare=True)\n",
    "\n",
    "#New code in comparison:\n",
    "start = time.time()\n",
    "log = repo.git.log('--numstat', '--format=//%H', '--all')\n",
    "blocks = log.split('//')[1:]\n",
    "commits = map(lambda block: block.split('\\n')[0], blocks)\n",
    "for hexsha in commits:\n",
    "    commit = Commit(repo, hex_to_bin(hexsha))\n",
    "    for obj in commit.tree.traverse(predicate=lambda obj, depth: obj.type == 'blob' and obj.name.endswith('.java')):\n",
    "        content = obj.data_stream.read().decode(\"CP437\")\n",
    "end = time.time()\n",
    "print('Total Time used for new method: '+str(end - start))\n",
    "\n",
    "#Old code in comparison:\n",
    "start = time.time()\n",
    "log = repo.git.log('--numstat', '--format=//%H', '--all')\n",
    "blocks = log.split('//')[1:]\n",
    "commits = map(lambda block: block.split('\\n')[0], blocks)\n",
    "for commit in repo.iter_commits():\n",
    "    for obj in commit.tree.traverse(predicate=lambda obj, depth: obj.type == 'blob' and obj.name.endswith('.java')):\n",
    "        content = obj.data_stream.read().decode(\"CP437\")\n",
    "end = time.time()\n",
    "print('Total Time used for old method: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delta based analysis approach "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the insights from the previous section, a great opportunity has risen: Only analyzing those files of a commit that have actually changed. The implications of this tuning are huge: the time complexity of the analysis run is reduced from quadratic in repository size (number of commits times number of files) to linear (number of commits times (more or less constant) commit size). \n",
    "\n",
    "To actually implement this, several sub functions must be implemented: `block_to_stats` extracts the changed java files and sum of changes from blocks of the commit log. `file_contents` is the new way to access files outside of a commit tree iterator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "def safeToInt(string):\n",
    "    return 0 if string == '-' else int(string)\n",
    "\n",
    "def block_to_stats(block):\n",
    "    lines = block.split('\\n')\n",
    "    header = lines[0]\n",
    "    lines = filter(lambda line: line.endswith('.java'), lines)\n",
    "    changed_files = list(map(lambda line: line.split('\\t'), lines))\n",
    "    #changed_files = list(changed_files)\n",
    "    additions = sum(map(lambda file: safeToInt(file[0]), changed_files))\n",
    "    deletions = sum(map(lambda file: safeToInt(file[1]), changed_files))\n",
    "    return (header, (changed_files, additions, deletions))\n",
    "\n",
    "def file_contents(tree, path):\n",
    "    try: \n",
    "        obj = tree / path\n",
    "        return obj.data_stream.read().decode(\"CP437\")\n",
    "    except KeyError:\n",
    "        return ''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following run demonstrates how the new approach can be used; the file contents and parent commit file contents are queried to simulate normal load.<br>\n",
    "This is \"only\" twice as fast as the original approach, but the effect turned out to be huge for larger repos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used for new method: 5.317599773406982\n"
     ]
    }
   ],
   "source": [
    "testRepo = Repo.init('./repos/bptlab/scylla.git', bare=True)\n",
    "start = time.time()\n",
    "log = testRepo.git.log('--numstat', '--format=//%H', '--all')\n",
    "#print(log)\n",
    "commits = log.split('//')[1:]\n",
    "changes = map(block_to_stats, commits)\n",
    "\n",
    "for hexsha, change in changes:\n",
    "    commit = Commit(testRepo, hex_to_bin(hexsha))\n",
    "    if len(commit.parents) == 1:\n",
    "        changed_files, additions, deletions = change\n",
    "        for added, removed, file in changed_files:\n",
    "            content1 = file_contents(commit.tree, file)\n",
    "            content2 = file_contents(commit.parents[0].tree, file)\n",
    "\n",
    "end = time.time()\n",
    "print('Total Time used for new method: '+str(end - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Time used for old method: 10.103673458099365\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "for commit in repo.iter_commits('--all'):\n",
    "    for obj in commit.tree.traverse(predicate=lambda obj, depth: obj.type == 'blob' and obj.name.endswith('.java')):\n",
    "        content = obj.data_stream.read().decode(\"CP437\")\n",
    "end = time.time()\n",
    "print('Total Time used for old method: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another test run with a bigger repository:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "testRepoTuple = ('brockn', 'incubator-parquet-mr', 11108627)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 672,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used for ('brockn', 'incubator-parquet-mr', 11108627): 435.4707381725311\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repoId</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>fileCount</th>\n",
       "      <th>numMethods</th>\n",
       "      <th>numLambdas</th>\n",
       "      <th>numCommentLines</th>\n",
       "      <th>numReflection</th>\n",
       "      <th>numSnakes</th>\n",
       "      <th>totalIndent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69ba4844730426a212c609facd93b33bf6692b3a</td>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>1412699955</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71466</td>\n",
       "      <td>56538</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1966</td>\n",
       "      <td>54355.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>1412198064</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71461</td>\n",
       "      <td>56534</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1964</td>\n",
       "      <td>54351.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>1412196285</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71454</td>\n",
       "      <td>56527</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1960</td>\n",
       "      <td>54340.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>1412017203</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71157</td>\n",
       "      <td>56235</td>\n",
       "      <td>489</td>\n",
       "      <td>5197</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1849</td>\n",
       "      <td>53845.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>3a082e8e390898646c094d20f4ec1eeba45b79ac</td>\n",
       "      <td>1411688756</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71153</td>\n",
       "      <td>56231</td>\n",
       "      <td>489</td>\n",
       "      <td>5197</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1849</td>\n",
       "      <td>53840.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1830</th>\n",
       "      <td>dbbf9443ffe219862c9744806f34deb2606d21c7</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346885521</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1831</th>\n",
       "      <td>24028799609ef606359d6bee39acd34d8d67a067</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346454047</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1832</th>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>1346452819</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1833</th>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>576c709724551a5122ae9b9e314b6c400f5f778d</td>\n",
       "      <td>1346452652</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1834</th>\n",
       "      <td>576c709724551a5122ae9b9e314b6c400f5f778d</td>\n",
       "      <td>None</td>\n",
       "      <td>1346451425</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1835 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           sha  \\\n",
       "0     69ba4844730426a212c609facd93b33bf6692b3a   \n",
       "1     be1222ef4a3260ddcf516d73c6ceecd144a134cb   \n",
       "2     da9129927bce90feb6d2860745263f4d74d0dfa8   \n",
       "3     0b17cbee9541998df66d33c8a99b675ced80d9aa   \n",
       "4     bf20abbf4825fa5892d8e15c066e768671a39289   \n",
       "...                                        ...   \n",
       "1830  dbbf9443ffe219862c9744806f34deb2606d21c7   \n",
       "1831  24028799609ef606359d6bee39acd34d8d67a067   \n",
       "1832  f3cdad30930c2cd73404804cf4ceeca237509a2c   \n",
       "1833  a8c10efccf35977193cab80b0f17d6a2f7d066d9   \n",
       "1834  576c709724551a5122ae9b9e314b6c400f5f778d   \n",
       "\n",
       "                                        parent   timestamp    repoId  \\\n",
       "0     be1222ef4a3260ddcf516d73c6ceecd144a134cb  1412699955  11108627   \n",
       "1     da9129927bce90feb6d2860745263f4d74d0dfa8  1412198064  11108627   \n",
       "2     0b17cbee9541998df66d33c8a99b675ced80d9aa  1412196285  11108627   \n",
       "3     bf20abbf4825fa5892d8e15c066e768671a39289  1412017203  11108627   \n",
       "4     3a082e8e390898646c094d20f4ec1eeba45b79ac  1411688756  11108627   \n",
       "...                                        ...         ...       ...   \n",
       "1830  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346885521  11108627   \n",
       "1831  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346454047  11108627   \n",
       "1832  a8c10efccf35977193cab80b0f17d6a2f7d066d9  1346452819  11108627   \n",
       "1833  576c709724551a5122ae9b9e314b6c400f5f778d  1346452652  11108627   \n",
       "1834                                      None  1346451425  11108627   \n",
       "\n",
       "      additions  deletions    loc   cloc  fileCount  numMethods  numLambdas  \\\n",
       "0             0          0  71466  56538        489        5214          11   \n",
       "1             0          0  71461  56534        489        5214          11   \n",
       "2             0          0  71454  56527        489        5214          11   \n",
       "3             0          0  71157  56235        489        5197          11   \n",
       "4             0          0  71153  56231        489        5197          11   \n",
       "...         ...        ...    ...    ...        ...         ...         ...   \n",
       "1830          0          0   2858   2502         40         323           3   \n",
       "1831          0          0   2858   2502         40         323           3   \n",
       "1832          0          0   2858   2502         40         323           3   \n",
       "1833          0          0   2858   2502         40         323           3   \n",
       "1834          0          0      0      0          0           0           0   \n",
       "\n",
       "      numCommentLines  numReflection  numSnakes  totalIndent  \n",
       "0                6229            360       1966     54355.75  \n",
       "1                6229            360       1964     54351.75  \n",
       "2                6229            360       1960     54340.75  \n",
       "3                6229            360       1849     53845.25  \n",
       "4                6229            360       1849     53840.75  \n",
       "...               ...            ...        ...          ...  \n",
       "1830               71              7          2      2190.50  \n",
       "1831               71              7          2      2190.50  \n",
       "1832               71              7          2      2190.50  \n",
       "1833               71              7          2      2190.50  \n",
       "1834                0              0          0         0.00  \n",
       "\n",
       "[1835 rows x 15 columns]"
      ]
     },
     "execution_count": 672,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoAnalysis.calculateMetrics(testRepoTuple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the git log approach only to determine the commits still results in a comparable run time - while providing the needed diff information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 673,
   "metadata": {},
   "outputs": [],
   "source": [
    "def alternativeCalculateMetrics(repoTuple, metricSuite=repoAnalysis.metricSuite):\n",
    "    (user, project, repoId) = repoTuple\n",
    "    repo = repoLibrarian.getRepo(user, project)\n",
    "    columns = ['sha', 'parent', 'timestamp', 'repoId', 'additions', 'deletions'] + list(map(lambda fun: fun.__name__, metricSuite))\n",
    "    results = []\n",
    "    try:\n",
    "        start = time.time()\n",
    "        log = repo.git.log('--numstat', '--format=//%H', '--all')\n",
    "        blocks = log.split('//')[1:]\n",
    "        commits = map(lambda block: block.split('\\n')[0], blocks)\n",
    "        for hexsha in commits:\n",
    "            commit = Commit(repo, hex_to_bin(hexsha))\n",
    "            results.append(repoAnalysis.metricsForCommit(commit, metricSuite, repoId))\n",
    "        df = pandas.DataFrame(results, columns=columns)\n",
    "        end = time.time()\n",
    "        print('Time used for '+str(repoTuple)+': '+str(end - start))\n",
    "        return df\n",
    "    except Exception as e:\n",
    "        print('Failed to analyze '+str(repoTuple)+': '+str(e))\n",
    "        return []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 674,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used for ('brockn', 'incubator-parquet-mr', 11108627): 439.26439571380615\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repoId</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>fileCount</th>\n",
       "      <th>numMethods</th>\n",
       "      <th>numLambdas</th>\n",
       "      <th>numCommentLines</th>\n",
       "      <th>numReflection</th>\n",
       "      <th>numSnakes</th>\n",
       "      <th>totalIndent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69ba4844730426a212c609facd93b33bf6692b3a</td>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>1412699955</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71466</td>\n",
       "      <td>56538</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1966</td>\n",
       "      <td>54355.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>1412198064</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71461</td>\n",
       "      <td>56534</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1964</td>\n",
       "      <td>54351.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>1412196285</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71454</td>\n",
       "      <td>56527</td>\n",
       "      <td>489</td>\n",
       "      <td>5214</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1960</td>\n",
       "      <td>54340.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>1412017203</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71157</td>\n",
       "      <td>56235</td>\n",
       "      <td>489</td>\n",
       "      <td>5197</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1849</td>\n",
       "      <td>53845.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>3a082e8e390898646c094d20f4ec1eeba45b79ac</td>\n",
       "      <td>1411688756</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>71153</td>\n",
       "      <td>56231</td>\n",
       "      <td>489</td>\n",
       "      <td>5197</td>\n",
       "      <td>11</td>\n",
       "      <td>6229</td>\n",
       "      <td>360</td>\n",
       "      <td>1849</td>\n",
       "      <td>53840.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1830</th>\n",
       "      <td>dbbf9443ffe219862c9744806f34deb2606d21c7</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346885521</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1831</th>\n",
       "      <td>24028799609ef606359d6bee39acd34d8d67a067</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346454047</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1832</th>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>1346452819</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1833</th>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>576c709724551a5122ae9b9e314b6c400f5f778d</td>\n",
       "      <td>1346452652</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2858</td>\n",
       "      <td>2502</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1834</th>\n",
       "      <td>576c709724551a5122ae9b9e314b6c400f5f778d</td>\n",
       "      <td>None</td>\n",
       "      <td>1346451425</td>\n",
       "      <td>11108627</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1835 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           sha  \\\n",
       "0     69ba4844730426a212c609facd93b33bf6692b3a   \n",
       "1     be1222ef4a3260ddcf516d73c6ceecd144a134cb   \n",
       "2     da9129927bce90feb6d2860745263f4d74d0dfa8   \n",
       "3     0b17cbee9541998df66d33c8a99b675ced80d9aa   \n",
       "4     bf20abbf4825fa5892d8e15c066e768671a39289   \n",
       "...                                        ...   \n",
       "1830  dbbf9443ffe219862c9744806f34deb2606d21c7   \n",
       "1831  24028799609ef606359d6bee39acd34d8d67a067   \n",
       "1832  f3cdad30930c2cd73404804cf4ceeca237509a2c   \n",
       "1833  a8c10efccf35977193cab80b0f17d6a2f7d066d9   \n",
       "1834  576c709724551a5122ae9b9e314b6c400f5f778d   \n",
       "\n",
       "                                        parent   timestamp    repoId  \\\n",
       "0     be1222ef4a3260ddcf516d73c6ceecd144a134cb  1412699955  11108627   \n",
       "1     da9129927bce90feb6d2860745263f4d74d0dfa8  1412198064  11108627   \n",
       "2     0b17cbee9541998df66d33c8a99b675ced80d9aa  1412196285  11108627   \n",
       "3     bf20abbf4825fa5892d8e15c066e768671a39289  1412017203  11108627   \n",
       "4     3a082e8e390898646c094d20f4ec1eeba45b79ac  1411688756  11108627   \n",
       "...                                        ...         ...       ...   \n",
       "1830  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346885521  11108627   \n",
       "1831  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346454047  11108627   \n",
       "1832  a8c10efccf35977193cab80b0f17d6a2f7d066d9  1346452819  11108627   \n",
       "1833  576c709724551a5122ae9b9e314b6c400f5f778d  1346452652  11108627   \n",
       "1834                                      None  1346451425  11108627   \n",
       "\n",
       "      additions  deletions    loc   cloc  fileCount  numMethods  numLambdas  \\\n",
       "0             0          0  71466  56538        489        5214          11   \n",
       "1             0          0  71461  56534        489        5214          11   \n",
       "2             0          0  71454  56527        489        5214          11   \n",
       "3             0          0  71157  56235        489        5197          11   \n",
       "4             0          0  71153  56231        489        5197          11   \n",
       "...         ...        ...    ...    ...        ...         ...         ...   \n",
       "1830          0          0   2858   2502         40         323           3   \n",
       "1831          0          0   2858   2502         40         323           3   \n",
       "1832          0          0   2858   2502         40         323           3   \n",
       "1833          0          0   2858   2502         40         323           3   \n",
       "1834          0          0      0      0          0           0           0   \n",
       "\n",
       "      numCommentLines  numReflection  numSnakes  totalIndent  \n",
       "0                6229            360       1966     54355.75  \n",
       "1                6229            360       1964     54351.75  \n",
       "2                6229            360       1960     54340.75  \n",
       "3                6229            360       1849     53845.25  \n",
       "4                6229            360       1849     53840.75  \n",
       "...               ...            ...        ...          ...  \n",
       "1830               71              7          2      2190.50  \n",
       "1831               71              7          2      2190.50  \n",
       "1832               71              7          2      2190.50  \n",
       "1833               71              7          2      2190.50  \n",
       "1834                0              0          0         0.00  \n",
       "\n",
       "[1835 rows x 15 columns]"
      ]
     },
     "execution_count": 674,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alternativeCalculateMetrics(testRepoTuple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the approach to its full extend sped up to a factor of more than 25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "def deltaMetricsForCommit(commit, metricSuite, repoId, change):\n",
    "    \n",
    "    resultTuple = {\n",
    "        'sha' : commit.hexsha,\n",
    "        'parent' : commit.parents[-1].hexsha if len(commit.parents) == 1 else None,\n",
    "        'timestamp' : commit.committed_date,\n",
    "        'repoId' : repoId\n",
    "    }    \n",
    "    for metricFunction in metricSuite:\n",
    "        resultTuple[metricFunction.__name__] = 0\n",
    "    \n",
    "    changed_files, additions, deletions = change\n",
    "    resultTuple['additions'] = additions\n",
    "    resultTuple['deletions'] = deletions\n",
    "    for added, removed, file in changed_files:\n",
    "        \n",
    "        contentWithHeader = file_contents(commit.tree, file)\n",
    "        content = repoAnalysis.removeHeader(contentWithHeader)\n",
    "        contentWithoutStrings = repoAnalysis.stringRemoveRegex.sub(\"\\\"...\\\"\", content)\n",
    "        contentWithoutComments = repoAnalysis.commentRegex.sub(\"/*...*/\", contentWithoutStrings)\n",
    "        for metricFunction in metricSuite:\n",
    "            metric = metricFunction(content=content, contentWithHeader=contentWithHeader, contentWithoutComments=contentWithoutComments)\n",
    "            resultTuple[metricFunction.__name__] = resultTuple[metricFunction.__name__] + metric\n",
    "            \n",
    "        contentWithHeader = file_contents(commit.parents[0].tree, file)\n",
    "        content = repoAnalysis.removeHeader(contentWithHeader)\n",
    "        contentWithoutStrings = repoAnalysis.stringRemoveRegex.sub(\"\\\"...\\\"\", content)\n",
    "        contentWithoutComments = repoAnalysis.commentRegex.sub(\"/*...*/\", contentWithoutStrings)\n",
    "        for metricFunction in metricSuite:\n",
    "            metric = metricFunction(content=content, contentWithHeader=contentWithHeader, contentWithoutComments=contentWithoutComments)\n",
    "            resultTuple[metricFunction.__name__] = resultTuple[metricFunction.__name__] - metric\n",
    "            \n",
    "    return resultTuple\n",
    "    \n",
    "\n",
    "def calculateDeltaMetrics(repoTuple, metricSuite=repoAnalysis.metricSuite):\n",
    "    (user, project, repoId) = repoTuple\n",
    "    repo = repoLibrarian.getRepo(user, project)\n",
    "    columns = ['sha', 'parent', 'timestamp', 'repo_id', 'additions', 'deletions'] + list(map(lambda fun: fun.__name__, metricSuite))\n",
    "    results = []\n",
    "    try:\n",
    "        start = time.time()\n",
    "        log = repo.git.log('--numstat', '--format=//%H', '--all')\n",
    "        commits = log.split('//')[1:]#First 'part'\n",
    "        changes = map(block_to_stats, commits)\n",
    "\n",
    "        for hexsha, change in changes:\n",
    "            commit = Commit(repo, hex_to_bin(hexsha))\n",
    "            if len(commit.parents) == 1:\n",
    "                results.append(deltaMetricsForCommit(commit, metricSuite, repoId, change))\n",
    "                \n",
    "        df = pandas.DataFrame(results, columns=columns)\n",
    "        end = time.time()\n",
    "        print('Time used for '+str(repoTuple)+': '+str(end - start))\n",
    "        return df\n",
    "    except Exception as e:\n",
    "        print('Failed to analyze '+str(repoTuple)+': '+str(e))\n",
    "        return []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used for ('brockn', 'incubator-parquet-mr', 11108627): 16.90212655067444\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>file_count</th>\n",
       "      <th>num_methods</th>\n",
       "      <th>num_lambdas</th>\n",
       "      <th>num_comment_lines</th>\n",
       "      <th>num_reflection</th>\n",
       "      <th>num_snakes</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69ba4844730426a212c609facd93b33bf6692b3a</td>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>1412699955</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>4.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>be1222ef4a3260ddcf516d73c6ceecd144a134cb</td>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>1412198064</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19</td>\n",
       "      <td>12</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>11.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>da9129927bce90feb6d2860745263f4d74d0dfa8</td>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>1412196285</td>\n",
       "      <td>NaN</td>\n",
       "      <td>346</td>\n",
       "      <td>49</td>\n",
       "      <td>297</td>\n",
       "      <td>292</td>\n",
       "      <td>0</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111</td>\n",
       "      <td>495.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0b17cbee9541998df66d33c8a99b675ced80d9aa</td>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>1412017203</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34</td>\n",
       "      <td>30</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>bf20abbf4825fa5892d8e15c066e768671a39289</td>\n",
       "      <td>3a082e8e390898646c094d20f4ec1eeba45b79ac</td>\n",
       "      <td>1411688756</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1449</th>\n",
       "      <td>5fb63e9a0743a23735f5b6043abd202a85173202</td>\n",
       "      <td>7ed952872d68eacc02d5a13e6b6ded4cf8babc55</td>\n",
       "      <td>1347128023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304</td>\n",
       "      <td>146</td>\n",
       "      <td>158</td>\n",
       "      <td>149</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>2</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>83.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1450</th>\n",
       "      <td>dbbf9443ffe219862c9744806f34deb2606d21c7</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346885521</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1451</th>\n",
       "      <td>24028799609ef606359d6bee39acd34d8d67a067</td>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>1346454047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1452</th>\n",
       "      <td>f3cdad30930c2cd73404804cf4ceeca237509a2c</td>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>1346452819</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1453</th>\n",
       "      <td>a8c10efccf35977193cab80b0f17d6a2f7d066d9</td>\n",
       "      <td>576c709724551a5122ae9b9e314b6c400f5f778d</td>\n",
       "      <td>1346452652</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2818</td>\n",
       "      <td>0</td>\n",
       "      <td>2818</td>\n",
       "      <td>2462</td>\n",
       "      <td>40</td>\n",
       "      <td>323</td>\n",
       "      <td>3</td>\n",
       "      <td>71</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2190.50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1454 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           sha  \\\n",
       "0     69ba4844730426a212c609facd93b33bf6692b3a   \n",
       "1     be1222ef4a3260ddcf516d73c6ceecd144a134cb   \n",
       "2     da9129927bce90feb6d2860745263f4d74d0dfa8   \n",
       "3     0b17cbee9541998df66d33c8a99b675ced80d9aa   \n",
       "4     bf20abbf4825fa5892d8e15c066e768671a39289   \n",
       "...                                        ...   \n",
       "1449  5fb63e9a0743a23735f5b6043abd202a85173202   \n",
       "1450  dbbf9443ffe219862c9744806f34deb2606d21c7   \n",
       "1451  24028799609ef606359d6bee39acd34d8d67a067   \n",
       "1452  f3cdad30930c2cd73404804cf4ceeca237509a2c   \n",
       "1453  a8c10efccf35977193cab80b0f17d6a2f7d066d9   \n",
       "\n",
       "                                        parent   timestamp  repo_id  \\\n",
       "0     be1222ef4a3260ddcf516d73c6ceecd144a134cb  1412699955      NaN   \n",
       "1     da9129927bce90feb6d2860745263f4d74d0dfa8  1412198064      NaN   \n",
       "2     0b17cbee9541998df66d33c8a99b675ced80d9aa  1412196285      NaN   \n",
       "3     bf20abbf4825fa5892d8e15c066e768671a39289  1412017203      NaN   \n",
       "4     3a082e8e390898646c094d20f4ec1eeba45b79ac  1411688756      NaN   \n",
       "...                                        ...         ...      ...   \n",
       "1449  7ed952872d68eacc02d5a13e6b6ded4cf8babc55  1347128023      NaN   \n",
       "1450  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346885521      NaN   \n",
       "1451  f3cdad30930c2cd73404804cf4ceeca237509a2c  1346454047      NaN   \n",
       "1452  a8c10efccf35977193cab80b0f17d6a2f7d066d9  1346452819      NaN   \n",
       "1453  576c709724551a5122ae9b9e314b6c400f5f778d  1346452652      NaN   \n",
       "\n",
       "      additions  deletions   loc  cloc  file_count  num_methods  num_lambdas  \\\n",
       "0             7          2     5     4           0            0            0   \n",
       "1            19         12     7     7           0            0            0   \n",
       "2           346         49   297   292           0           17            0   \n",
       "3            34         30     4     4           0            0            0   \n",
       "4            34          1    33    33           0           11            0   \n",
       "...         ...        ...   ...   ...         ...          ...          ...   \n",
       "1449        304        146   158   149           0           30            2   \n",
       "1450          0          0     0     0           0            0            0   \n",
       "1451          0          0     0     0           0            0            0   \n",
       "1452          0          0     0     0           0            0            0   \n",
       "1453       2818          0  2818  2462          40          323            3   \n",
       "\n",
       "      num_comment_lines  num_reflection  num_snakes  total_indent  \n",
       "0                     0               0           2          4.00  \n",
       "1                     0               0           4         11.00  \n",
       "2                     0               0         111        495.50  \n",
       "3                     0               0           0          4.50  \n",
       "4                     0               0           0         15.00  \n",
       "...                 ...             ...         ...           ...  \n",
       "1449                 32               1           0         83.75  \n",
       "1450                  0               0           0          0.00  \n",
       "1451                  0               0           0          0.00  \n",
       "1452                  0               0           0          0.00  \n",
       "1453                 71               7           2       2190.50  \n",
       "\n",
       "[1454 rows x 15 columns]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "calculateDeltaMetrics(testRepoTuple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above developed functions have been integrated into the [repoAnalysis](repoAnalysis.py) module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysis Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the listed problems solved and a new approach developed, the second run can now start. Because of the big changes, it is interesting to run this analyis on the same data scale (and therefore developers) as the first one. Therefore, the approaches can be compared. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "tableName = 'lb_results2'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Utility function updates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create table had to be revised, as firstly additional columns `additions` and `deletions` had been introduced (these were formerly extracted from the table, now directly calculated during analysis) and secondly column names had been switched from camel to snake case, as there were errors when running queries on attributes in camel case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "def createTable():\n",
    "    columns = [Column('sha', String), Column('parent', String), Column('timestamp', Integer), Column('repo_id', Integer), Column('additions', Integer), Column('deletions', Integer)]\n",
    "    columns = columns + list(map(lambda func: Column(func.__name__, Integer), suite))\n",
    "    meta = MetaData(schema='crm20')\n",
    "    table = Table(\n",
    "        tableName, meta,\n",
    "        *columns\n",
    "    )\n",
    "    meta.create_all(dbUtils.engine)\n",
    "    dbUtils.engine.dispose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing to the database gained it's own function for better decomposition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataBaseSemaphore = multiprocessing.Semaphore()\n",
    "def writeDataToDb(data):\n",
    "    with dataBaseSemaphore:\n",
    "        data.to_sql(tableName, schema='crm20', con=dbUtils.engine, if_exists='append', index=False)\n",
    "        dbUtils.engine.dispose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suite running was changed from the old total-count-calculation to calculating deltas only"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "def runDeltaSuite(repo):\n",
    "    with io.capture_output() as output:\n",
    "        data = repoAnalysis.calculateDeltaMetrics(repo, suite)\n",
    "        writeDataToDb(data)\n",
    "    dbUtils.log(output)\n",
    "    return len(data) > 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the repos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/mnt/brick/crm20/repos/'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repoLibrarian.setReposFolder('/mnt/brick/crm20/repos/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.4349958896636963\n",
      "Time used: 0.34889984130859375\n"
     ]
    }
   ],
   "source": [
    "polyglotProjects = dbUtils.runQuery('''SELECT * FROM lb_polyglotProjects''', mute=True)\n",
    "controlgroupProjects = dbUtils.runQuery('''SELECT * FROM lb_controlgroupProjects''', mute=True)\n",
    "bothProjects = pandas.concat([polyglotProjects, controlgroupProjects], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Failed to check ('thiagoruis', 'dotNET-Grupo-2'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('tananaev', 'traccar-client'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('topicusonderwijs', 'wicket-openid'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('dav009', 'graficadora-tweets'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('dav009', 'MWE-DictionaryExtractor'): Reference at 'refs/heads/master' does not exist\n",
      "Failed to check ('OpenGamma', 'OG-RStats'): Reference at 'refs/heads/master' does not exist\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-24-781f35691424>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mrepos\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mtupl\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mrepoLibrarian\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0misJavaRepo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtupl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtupl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrowToTuple\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbothProjects\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miterrows\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mlog\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Total Time used: '\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mend\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mstart\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-24-781f35691424>\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(tupl)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mrepos\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mlambda\u001b[0m \u001b[0mtupl\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mrepoLibrarian\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0misJavaRepo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtupl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtupl\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrowToTuple\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbothProjects\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miterrows\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mlog\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Total Time used: '\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mend\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mstart\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyterNotebook/repoLibrarian.py\u001b[0m in \u001b[0;36misJavaRepo\u001b[0;34m(user, project)\u001b[0m\n\u001b[1;32m     84\u001b[0m         \u001b[0mrepo\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgetRepo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0muser\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mproject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     85\u001b[0m         \u001b[0mcommit\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrepo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miter_commits\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 86\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0misJavaFile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mobj\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcommit\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtree\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtraverse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     87\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     88\u001b[0m         \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Failed to check '\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0muser\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mproject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;34m': '\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/gitdb/util.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, attr)\u001b[0m\n\u001b[1;32m    251\u001b[0m         \u001b[0mto\u001b[0m \u001b[0mbe\u001b[0m \u001b[0mcreated\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m.\u001b[0m \u001b[0mNext\u001b[0m \u001b[0mtime\u001b[0m \u001b[0mthe\u001b[0m \u001b[0msame\u001b[0m \u001b[0mattribute\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mreqeusted\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mit\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0msimply\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    252\u001b[0m         returned from our dict/slots. \"\"\"\n\u001b[0;32m--> 253\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_set_cache_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mattr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    254\u001b[0m         \u001b[0;31m# will raise in case the cache was not created\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    255\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mobject\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__getattribute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/objects/commit.py\u001b[0m in \u001b[0;36m_set_cache_\u001b[0;34m(self, attr)\u001b[0m\n\u001b[1;32m    140\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mattr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mCommit\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__slots__\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    141\u001b[0m             \u001b[0;31m# read the data in a chunk, its faster - then provide a file wrapper\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 142\u001b[0;31m             \u001b[0m_binsha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_typename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrepo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0modb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstream\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbinsha\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    143\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_deserialize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mBytesIO\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstream\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    144\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/db.py\u001b[0m in \u001b[0;36mstream\u001b[0;34m(self, sha)\u001b[0m\n\u001b[1;32m     40\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msha\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     41\u001b[0m         \u001b[0;34m\"\"\"For now, all lookup is done by git itself\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 42\u001b[0;31m         \u001b[0mhexsha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtypename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_git\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstream_object_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbin_to_hex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msha\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     43\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mOStream\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhex_to_bin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhexsha\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtypename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     44\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/cmd.py\u001b[0m in \u001b[0;36mstream_object_data\u001b[0;34m(self, ref)\u001b[0m\n\u001b[1;32m   1082\u001b[0m         \u001b[0;34m:\u001b[0m\u001b[0;32mreturn\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mhexsha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtype_string\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msize_as_int\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1083\u001b[0m         :note: This method is not threadsafe, you need one independent Command instance per thread to be safe !\"\"\"\n\u001b[0;32m-> 1084\u001b[0;31m         \u001b[0mcmd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_persistent_cmd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"cat_file_all\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"cat_file\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1085\u001b[0m         \u001b[0mhexsha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtypename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msize\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__get_object_header\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcmd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mref\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1086\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mhexsha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtypename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mCatFileContentStream\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcmd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstdout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/cmd.py\u001b[0m in \u001b[0;36m_get_persistent_cmd\u001b[0;34m(self, attr_name, cmd_name, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1048\u001b[0m         \u001b[0moptions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1049\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1050\u001b[0;31m         \u001b[0mcmd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_process\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcmd_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1051\u001b[0m         \u001b[0msetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattr_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcmd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1052\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mcmd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/cmd.py\u001b[0m in \u001b[0;36m_call_process\u001b[0;34m(self, method, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1003\u001b[0m         \u001b[0mcall\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1004\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1005\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mexec_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1006\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1007\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_parse_object_header\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheader_line\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/jupyter/lib/python3.8/site-packages/git/cmd.py\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, command, istream, with_extended_output, with_exceptions, as_process, output_stream, stdout_as_string, kill_after_timeout, with_stdout, universal_newlines, shell, env, max_chunk_size, **subprocess_kwargs)\u001b[0m\n\u001b[1;32m    719\u001b[0m                   command, cwd, universal_newlines, shell, istream_ok)\n\u001b[1;32m    720\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 721\u001b[0;31m             proc = Popen(command,\n\u001b[0m\u001b[1;32m    722\u001b[0m                          \u001b[0menv\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0menv\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    723\u001b[0m                          \u001b[0mcwd\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcwd\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.8/subprocess.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)\u001b[0m\n\u001b[1;32m    852\u001b[0m                             encoding=encoding, errors=errors)\n\u001b[1;32m    853\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 854\u001b[0;31m             self._execute_child(args, executable, preexec_fn, close_fds,\n\u001b[0m\u001b[1;32m    855\u001b[0m                                 \u001b[0mpass_fds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcwd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0menv\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    856\u001b[0m                                 \u001b[0mstartupinfo\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreationflags\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshell\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.8/subprocess.py\u001b[0m in \u001b[0;36m_execute_child\u001b[0;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session)\u001b[0m\n\u001b[1;32m   1635\u001b[0m                     \u001b[0mfds_to_keep\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpass_fds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1636\u001b[0m                     \u001b[0mfds_to_keep\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merrpipe_write\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1637\u001b[0;31m                     self.pid = _posixsubprocess.fork_exec(\n\u001b[0m\u001b[1;32m   1638\u001b[0m                             \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexecutable_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1639\u001b[0m                             \u001b[0mclose_fds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msorted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfds_to_keep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "repos = list(filter(lambda tupl: repoLibrarian.isJavaRepo(tupl[0], tupl[1]), map(rowToTuple, bothProjects.iterrows())))\n",
    "end = time.time()\n",
    "dbUtils.log('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1377\n",
      "620\n"
     ]
    }
   ],
   "source": [
    "print(len(bothProjects))\n",
    "print(len(repos))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dry test to see if everything is up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "dbUtils.log('test')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('twitter', 'scalding', 1182)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repos[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "deleteTable()\n",
    "createTable()\n",
    "runDeltaSuite(repos[0])\n",
    "display(Audio('./beep.mp3', autoplay=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(The results have been checked by simply running a database query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's go"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deleteTable()\n",
    "createTable()\n",
    "start = time.time()\n",
    "with Pool(int(multiprocessing.cpu_count()*3/4)) as pool:\n",
    "    allMetrics = pool.map(runDeltaSuite, repos)\n",
    "end = time.time()\n",
    "dbUtils.log('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The run log can be found under [./results/log_second_data_run.txt](./results/log_second_data_run.txt). \n",
    "It shows that this run analyzed the 620 repositories in 27 minutes ... that is an improvement by factor 106. This exceeded my expectations by far and motivates to scale up a lot for the next run.\n",
    "\n",
    "Evaluation of the results can be found in the [Results_Iteration#2](Results_Iteration#2.ipynb) notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Emergency Recovery"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, the `lb_results2` table has been lost. <br>\n",
    "The set of polyglots used as input for this table could, against the expectations, not be reconstructed by simply filtering the extended set by `MOD(user_id, 113) = 5` (yields 1697 results compared to 1366 in the [historical commit](https://github.com/LeonBein/CRM20/blob/d58061b46925d8c72498ef623784cfeef16ee52b/DataExplorer.ipynb)). <br>\n",
    "Therefore, a full recovery run has been started that hopefully yields the same results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.13672184944152832\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_id</th>\n",
       "      <th>language</th>\n",
       "      <th>loc</th>\n",
       "      <th>first_written</th>\n",
       "      <th>last_written</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>118</td>\n",
       "      <td>java</td>\n",
       "      <td>4458.0</td>\n",
       "      <td>2014-07-12 13:58:27</td>\n",
       "      <td>2016-10-01 16:22:09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1361</td>\n",
       "      <td>java</td>\n",
       "      <td>54192.0</td>\n",
       "      <td>2014-08-23 09:58:05</td>\n",
       "      <td>2019-03-19 10:18:42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4073</td>\n",
       "      <td>java</td>\n",
       "      <td>3228.0</td>\n",
       "      <td>2014-08-14 02:10:11</td>\n",
       "      <td>2015-09-19 21:27:06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5655</td>\n",
       "      <td>java</td>\n",
       "      <td>3107.0</td>\n",
       "      <td>2012-07-15 22:32:06</td>\n",
       "      <td>2017-09-19 20:08:39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5768</td>\n",
       "      <td>java</td>\n",
       "      <td>17628.0</td>\n",
       "      <td>2010-09-30 21:37:07</td>\n",
       "      <td>2016-03-09 18:27:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1692</th>\n",
       "      <td>50573155</td>\n",
       "      <td>java</td>\n",
       "      <td>72624.0</td>\n",
       "      <td>2019-04-10 13:08:48</td>\n",
       "      <td>2019-05-31 10:14:29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1693</th>\n",
       "      <td>50591009</td>\n",
       "      <td>java</td>\n",
       "      <td>12170.0</td>\n",
       "      <td>2019-04-10 11:37:35</td>\n",
       "      <td>2019-05-10 03:30:22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1694</th>\n",
       "      <td>51592189</td>\n",
       "      <td>java</td>\n",
       "      <td>3697.0</td>\n",
       "      <td>2019-05-27 11:02:37</td>\n",
       "      <td>2019-05-27 11:54:27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1695</th>\n",
       "      <td>50858593</td>\n",
       "      <td>java</td>\n",
       "      <td>13814.0</td>\n",
       "      <td>2019-05-03 22:52:34</td>\n",
       "      <td>2019-05-28 19:04:28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1696</th>\n",
       "      <td>51593658</td>\n",
       "      <td>java</td>\n",
       "      <td>7122.0</td>\n",
       "      <td>2019-05-27 12:59:14</td>\n",
       "      <td>2019-05-31 13:34:42</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1697 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      author_id language      loc       first_written        last_written\n",
       "0           118     java   4458.0 2014-07-12 13:58:27 2016-10-01 16:22:09\n",
       "1          1361     java  54192.0 2014-08-23 09:58:05 2019-03-19 10:18:42\n",
       "2          4073     java   3228.0 2014-08-14 02:10:11 2015-09-19 21:27:06\n",
       "3          5655     java   3107.0 2012-07-15 22:32:06 2017-09-19 20:08:39\n",
       "4          5768     java  17628.0 2010-09-30 21:37:07 2016-03-09 18:27:03\n",
       "...         ...      ...      ...                 ...                 ...\n",
       "1692   50573155     java  72624.0 2019-04-10 13:08:48 2019-05-31 10:14:29\n",
       "1693   50591009     java  12170.0 2019-04-10 11:37:35 2019-05-10 03:30:22\n",
       "1694   51592189     java   3697.0 2019-05-27 11:02:37 2019-05-27 11:54:27\n",
       "1695   50858593     java  13814.0 2019-05-03 22:52:34 2019-05-28 19:04:28\n",
       "1696   51593658     java   7122.0 2019-05-27 12:59:14 2019-05-31 13:34:42\n",
       "\n",
       "[1697 rows x 5 columns]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT *\n",
    "    FROM crm20.lb_java_users\n",
    "    WHERE MOD(author_id, 113) = 5\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First step is to reconstruct the basic tables, especially language information per author. All queries are based on an [old version](https://github.com/LeonBein/CRM20/blob/d58061b46925d8c72498ef623784cfeef16ee52b/DataExplorer.ipynb) of the data explorer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dbUtils.runQuery('''\n",
    "    CREATE MATERIALIZED VIEW crm20.temp_languages AS (\n",
    "        WITH temp_sample_users AS (\n",
    "            SELECT id \n",
    "            FROM ght.users \n",
    "            WHERE MOD(id, 113) = 5 \n",
    "            AND type = 'USR'\n",
    "        ),\n",
    "\n",
    "        temp_projects AS (\n",
    "            SELECT DISTINCT repo_id\n",
    "            FROM temp_sample_users, ght.project_members \n",
    "            WHERE id = user_id\n",
    "        ),\n",
    "\n",
    "        temp_all_commits AS (\n",
    "            SELECT commit_id \n",
    "            FROM ght.project_commits, temp_projects\n",
    "            WHERE project_id = repo_id\n",
    "        ),\n",
    "\n",
    "        temp_commits AS (\n",
    "            SELECT commit_id, sha, author_id, created_at, project_id \n",
    "            FROM temp_all_commits, ght.commits\n",
    "            WHERE commit_id = id AND author_id IN (SELECT * FROM temp_sample_users)\n",
    "        ),\n",
    "\n",
    "        temp_changes AS (\n",
    "            SELECT commit_id, temp_commits.sha, SUM(changes-deletions) AS loc, SUBSTRING(name from '\\.([^\\.]*)$') AS language\n",
    "            FROM temp_commits, ght.raw_patches\n",
    "            WHERE temp_commits.sha = raw_patches.sha\n",
    "            GROUP BY commit_id, temp_commits.sha, language\n",
    "        )\n",
    "\n",
    "        SELECT author_id, language, SUM(loc) AS loc, MIN(created_at) as first_written\n",
    "        FROM temp_commits, temp_changes\n",
    "        WHERE temp_commits.commit_id = temp_changes.commit_id\n",
    "        GROUP BY author_id, language\n",
    "    );\n",
    "\n",
    "    SELECT * FROM crm20.temp_languages\n",
    "''')\n",
    "\n",
    "dbUtils.log('lets see', 'resolved.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting data is the same compared to the historical commit; this is an intermediate success"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 3.0308587551116943\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_id</th>\n",
       "      <th>language</th>\n",
       "      <th>loc</th>\n",
       "      <th>first_written</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2014-04-07 12:09:46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>au</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2011-03-13 16:26:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>cn</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2011-03-13 16:26:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>com/bluele/slack/LICENSE</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2018-11-16 11:26:22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>com/google/go-querystring/LICENSE</td>\n",
       "      <td>27.0</td>\n",
       "      <td>2017-02-09 10:59:19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332356</th>\n",
       "      <td>51673097</td>\n",
       "      <td>css</td>\n",
       "      <td>569.0</td>\n",
       "      <td>2019-05-30 21:09:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332357</th>\n",
       "      <td>51673097</td>\n",
       "      <td>html</td>\n",
       "      <td>168.0</td>\n",
       "      <td>2019-05-30 21:09:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332358</th>\n",
       "      <td>51673097</td>\n",
       "      <td>js</td>\n",
       "      <td>121.0</td>\n",
       "      <td>2019-05-30 21:09:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332359</th>\n",
       "      <td>51673097</td>\n",
       "      <td>png</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2019-05-30 21:09:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332360</th>\n",
       "      <td>51696488</td>\n",
       "      <td>py</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2019-05-31 22:21:38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>332361 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        author_id                           language    loc  \\\n",
       "0               5                                  2    6.0   \n",
       "1               5                                 au    1.0   \n",
       "2               5                                 cn    1.0   \n",
       "3               5           com/bluele/slack/LICENSE    0.0   \n",
       "4               5  com/google/go-querystring/LICENSE   27.0   \n",
       "...           ...                                ...    ...   \n",
       "332356   51673097                                css  569.0   \n",
       "332357   51673097                               html  168.0   \n",
       "332358   51673097                                 js  121.0   \n",
       "332359   51673097                                png    0.0   \n",
       "332360   51696488                                 py    4.0   \n",
       "\n",
       "             first_written  \n",
       "0      2014-04-07 12:09:46  \n",
       "1      2011-03-13 16:26:02  \n",
       "2      2011-03-13 16:26:02  \n",
       "3      2018-11-16 11:26:22  \n",
       "4      2017-02-09 10:59:19  \n",
       "...                    ...  \n",
       "332356 2019-05-30 21:09:03  \n",
       "332357 2019-05-30 21:09:03  \n",
       "332358 2019-05-30 21:09:03  \n",
       "332359 2019-05-30 21:09:03  \n",
       "332360 2019-05-31 22:21:38  \n",
       "\n",
       "[332361 rows x 4 columns]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT * FROM crm20.temp_languages\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then java users and polyglot and control subgroups are determined"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.12581706047058105\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_id</th>\n",
       "      <th>language</th>\n",
       "      <th>loc</th>\n",
       "      <th>first_written</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>118</td>\n",
       "      <td>java</td>\n",
       "      <td>4458.0</td>\n",
       "      <td>2014-07-12 13:58:27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1361</td>\n",
       "      <td>java</td>\n",
       "      <td>52662.0</td>\n",
       "      <td>2014-08-23 09:58:05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4073</td>\n",
       "      <td>java</td>\n",
       "      <td>5481.0</td>\n",
       "      <td>2014-08-14 02:10:11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5768</td>\n",
       "      <td>java</td>\n",
       "      <td>191889.0</td>\n",
       "      <td>2010-08-04 18:11:50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10401</td>\n",
       "      <td>java</td>\n",
       "      <td>4762.0</td>\n",
       "      <td>2012-08-30 09:36:51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1361</th>\n",
       "      <td>49736842</td>\n",
       "      <td>java</td>\n",
       "      <td>3194.0</td>\n",
       "      <td>2019-03-01 08:29:23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1362</th>\n",
       "      <td>49975159</td>\n",
       "      <td>java</td>\n",
       "      <td>12942.0</td>\n",
       "      <td>2019-03-18 09:09:26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1363</th>\n",
       "      <td>50518011</td>\n",
       "      <td>java</td>\n",
       "      <td>11297.0</td>\n",
       "      <td>2019-04-09 18:20:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1364</th>\n",
       "      <td>50573155</td>\n",
       "      <td>java</td>\n",
       "      <td>72624.0</td>\n",
       "      <td>2019-04-10 13:08:48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1365</th>\n",
       "      <td>50591009</td>\n",
       "      <td>java</td>\n",
       "      <td>11968.0</td>\n",
       "      <td>2019-04-10 11:37:35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1366 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      author_id language       loc       first_written\n",
       "0           118     java    4458.0 2014-07-12 13:58:27\n",
       "1          1361     java   52662.0 2014-08-23 09:58:05\n",
       "2          4073     java    5481.0 2014-08-14 02:10:11\n",
       "3          5768     java  191889.0 2010-08-04 18:11:50\n",
       "4         10401     java    4762.0 2012-08-30 09:36:51\n",
       "...         ...      ...       ...                 ...\n",
       "1361   49736842     java    3194.0 2019-03-01 08:29:23\n",
       "1362   49975159     java   12942.0 2019-03-18 09:09:26\n",
       "1363   50518011     java   11297.0 2019-04-09 18:20:00\n",
       "1364   50573155     java   72624.0 2019-04-10 13:08:48\n",
       "1365   50591009     java   11968.0 2019-04-10 11:37:35\n",
       "\n",
       "[1366 rows x 4 columns]"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    DROP VIEW IF EXISTS crm20.temp_java_users;\n",
    "    \n",
    "    CREATE VIEW crm20.temp_java_users AS (\n",
    "        SELECT *\n",
    "        FROM crm20.temp_languages\n",
    "        WHERE language = 'java' AND loc > 3000\n",
    "    );\n",
    "        \n",
    "    SELECT * FROM crm20.temp_java_users\n",
    "''', mute=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.21452021598815918\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_id</th>\n",
       "      <th>javaloc</th>\n",
       "      <th>javastart</th>\n",
       "      <th>secondary_languages</th>\n",
       "      <th>secloc</th>\n",
       "      <th>secstart</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>55149</td>\n",
       "      <td>48533.0</td>\n",
       "      <td>2012-08-18 19:14:30</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[13340]</td>\n",
       "      <td>[2015-08-10 01:20:42]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>61025</td>\n",
       "      <td>231507.0</td>\n",
       "      <td>2010-12-17 19:50:34</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[1494]</td>\n",
       "      <td>[2013-10-09 16:20:00]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>89727</td>\n",
       "      <td>90236.0</td>\n",
       "      <td>2012-03-01 13:56:58</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[32537]</td>\n",
       "      <td>[2013-02-10 15:10:56]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100010</td>\n",
       "      <td>4449.0</td>\n",
       "      <td>2013-08-21 21:01:11</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[1214]</td>\n",
       "      <td>[2014-05-16 19:11:29]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100236</td>\n",
       "      <td>143991.0</td>\n",
       "      <td>2010-04-11 12:32:30</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[42598]</td>\n",
       "      <td>[2015-01-31 21:20:39]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>41410889</td>\n",
       "      <td>11749.0</td>\n",
       "      <td>2018-04-02 00:46:50</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[1263]</td>\n",
       "      <td>[2018-04-13 04:04:22]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>45852806</td>\n",
       "      <td>4813.0</td>\n",
       "      <td>2018-10-23 03:57:45</td>\n",
       "      <td>[py]</td>\n",
       "      <td>[1928]</td>\n",
       "      <td>[2018-11-09 02:44:09]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>45985807</td>\n",
       "      <td>69531.0</td>\n",
       "      <td>2018-10-23 08:41:21</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[29569]</td>\n",
       "      <td>[2018-10-23 08:57:52]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>46354752</td>\n",
       "      <td>12717.0</td>\n",
       "      <td>2018-10-08 22:59:11</td>\n",
       "      <td>[js]</td>\n",
       "      <td>[2197]</td>\n",
       "      <td>[2018-10-09 13:36:55]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>46576797</td>\n",
       "      <td>10508.0</td>\n",
       "      <td>2018-10-20 15:30:47</td>\n",
       "      <td>[py]</td>\n",
       "      <td>[2417]</td>\n",
       "      <td>[2018-11-13 19:37:04]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>108 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     author_id   javaloc           javastart secondary_languages   secloc  \\\n",
       "0        55149   48533.0 2012-08-18 19:14:30                [js]  [13340]   \n",
       "1        61025  231507.0 2010-12-17 19:50:34                [js]   [1494]   \n",
       "2        89727   90236.0 2012-03-01 13:56:58                [js]  [32537]   \n",
       "3       100010    4449.0 2013-08-21 21:01:11                [js]   [1214]   \n",
       "4       100236  143991.0 2010-04-11 12:32:30                [js]  [42598]   \n",
       "..         ...       ...                 ...                 ...      ...   \n",
       "103   41410889   11749.0 2018-04-02 00:46:50                [js]   [1263]   \n",
       "104   45852806    4813.0 2018-10-23 03:57:45                [py]   [1928]   \n",
       "105   45985807   69531.0 2018-10-23 08:41:21                [js]  [29569]   \n",
       "106   46354752   12717.0 2018-10-08 22:59:11                [js]   [2197]   \n",
       "107   46576797   10508.0 2018-10-20 15:30:47                [py]   [2417]   \n",
       "\n",
       "                  secstart  \n",
       "0    [2015-08-10 01:20:42]  \n",
       "1    [2013-10-09 16:20:00]  \n",
       "2    [2013-02-10 15:10:56]  \n",
       "3    [2014-05-16 19:11:29]  \n",
       "4    [2015-01-31 21:20:39]  \n",
       "..                     ...  \n",
       "103  [2018-04-13 04:04:22]  \n",
       "104  [2018-11-09 02:44:09]  \n",
       "105  [2018-10-23 08:57:52]  \n",
       "106  [2018-10-09 13:36:55]  \n",
       "107  [2018-11-13 19:37:04]  \n",
       "\n",
       "[108 rows x 6 columns]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "        CREATE MATERIALIZED VIEW crm20.temp_polyglots AS (\n",
    "            SELECT temp_java_users.author_id, temp_java_users.loc AS javaLoC,  temp_java_users.first_written as javaStart, ARRAY_AGG(temp_languages.language) AS secondary_languages, ARRAY_AGG(temp_languages.loc) AS secLoC, ARRAY_AGG(temp_languages.first_written) as secStart\n",
    "            FROM crm20.temp_java_users, crm20.temp_languages\n",
    "            WHERE temp_java_users.author_id = temp_languages.author_id \n",
    "            AND (temp_languages.language = 'py' OR temp_languages.language = 'js') AND temp_languages.loc > 1000\n",
    "            GROUP BY temp_java_users.author_id, temp_java_users.loc, temp_java_users.first_written\n",
    "            HAVING temp_java_users.loc / 2 > ALL(ARRAY_AGG(temp_languages.loc))\n",
    "            AND temp_java_users.first_written < ALL(ARRAY_AGG(temp_languages.first_written))\n",
    "        );\n",
    "        SELECT * FROM crm20.temp_polyglots\n",
    "''', mute=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.2006971836090088\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_id</th>\n",
       "      <th>javaloc</th>\n",
       "      <th>javastart</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>16503</td>\n",
       "      <td>106978.0</td>\n",
       "      <td>2010-09-30 11:48:03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>34131</td>\n",
       "      <td>174060.0</td>\n",
       "      <td>2010-07-22 11:55:16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>99671</td>\n",
       "      <td>20371.0</td>\n",
       "      <td>2012-07-14 12:27:07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>153911</td>\n",
       "      <td>5064.0</td>\n",
       "      <td>2013-11-19 15:26:37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>338779</td>\n",
       "      <td>13707.0</td>\n",
       "      <td>2012-07-09 07:15:19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>46946759</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>2019-05-10 07:02:55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>49175119</td>\n",
       "      <td>16300.0</td>\n",
       "      <td>2019-02-03 02:28:33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>49975159</td>\n",
       "      <td>12942.0</td>\n",
       "      <td>2019-03-18 09:09:26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>50518011</td>\n",
       "      <td>11297.0</td>\n",
       "      <td>2019-04-09 18:20:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>50573155</td>\n",
       "      <td>72624.0</td>\n",
       "      <td>2019-04-10 13:08:48</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>106 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     author_id   javaloc           javastart\n",
       "0        16503  106978.0 2010-09-30 11:48:03\n",
       "1        34131  174060.0 2010-07-22 11:55:16\n",
       "2        99671   20371.0 2012-07-14 12:27:07\n",
       "3       153911    5064.0 2013-11-19 15:26:37\n",
       "4       338779   13707.0 2012-07-09 07:15:19\n",
       "..         ...       ...                 ...\n",
       "101   46946759   10000.0 2019-05-10 07:02:55\n",
       "102   49175119   16300.0 2019-02-03 02:28:33\n",
       "103   49975159   12942.0 2019-03-18 09:09:26\n",
       "104   50518011   11297.0 2019-04-09 18:20:00\n",
       "105   50573155   72624.0 2019-04-10 13:08:48\n",
       "\n",
       "[106 rows x 3 columns]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "        DROP MATERIALIZED VIEW IF EXISTS crm20.temp_controlgroup;\n",
    "        CREATE MATERIALIZED VIEW crm20.temp_controlgroup AS (\n",
    "            SELECT temp_java_users.author_id, temp_java_users.loc AS javaLoC,  temp_java_users.first_written as javaStart\n",
    "            FROM crm20.temp_java_users\n",
    "            WHERE author_id NOT IN (SELECT author_id FROM crm20.temp_languages WHERE (language = 'py' OR language = 'js'))\n",
    "            AND MOD(author_id, 4) = 3 \n",
    "            AND loc > 5000\n",
    "        );\n",
    "        SELECT * FROM crm20.temp_controlgroup\n",
    "''', mute=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 7.6404218673706055\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>repo_id</th>\n",
       "      <th>url</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1182</td>\n",
       "      <td>https://api.github.com/repos/twitter/scalding</td>\n",
       "      <td>scalding</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1372</td>\n",
       "      <td>https://api.github.com/repos/twitter/finagle</td>\n",
       "      <td>finagle</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3732</td>\n",
       "      <td>https://api.github.com/repos/ceylon/ceylon-ide...</td>\n",
       "      <td>ceylon-ide-eclipse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>17827</td>\n",
       "      <td>https://api.github.com/repos/twitter/flockdb</td>\n",
       "      <td>flockdb</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>34552</td>\n",
       "      <td>https://api.github.com/repos/thiagoruis/dotNET...</td>\n",
       "      <td>dotNET-Grupo-2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>989</th>\n",
       "      <td>131699833</td>\n",
       "      <td>https://api.github.com/repos/suimn416/2019.Spr...</td>\n",
       "      <td>2019.Spring.AI_BOSS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>990</th>\n",
       "      <td>131979956</td>\n",
       "      <td>https://api.github.com/repos/capturetheworld/c...</td>\n",
       "      <td>cs151project</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>991</th>\n",
       "      <td>132564033</td>\n",
       "      <td>https://api.github.com/repos/idio/langdetect</td>\n",
       "      <td>langdetect</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>992</th>\n",
       "      <td>134135475</td>\n",
       "      <td>https://api.github.com/repos/luchizina/phpLuna</td>\n",
       "      <td>phpLuna</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>993</th>\n",
       "      <td>136724786</td>\n",
       "      <td>https://api.github.com/repos/ramesesinc/module...</td>\n",
       "      <td>modules-partner</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>994 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       repo_id                                                url  \\\n",
       "0         1182      https://api.github.com/repos/twitter/scalding   \n",
       "1         1372       https://api.github.com/repos/twitter/finagle   \n",
       "2         3732  https://api.github.com/repos/ceylon/ceylon-ide...   \n",
       "3        17827       https://api.github.com/repos/twitter/flockdb   \n",
       "4        34552  https://api.github.com/repos/thiagoruis/dotNET...   \n",
       "..         ...                                                ...   \n",
       "989  131699833  https://api.github.com/repos/suimn416/2019.Spr...   \n",
       "990  131979956  https://api.github.com/repos/capturetheworld/c...   \n",
       "991  132564033       https://api.github.com/repos/idio/langdetect   \n",
       "992  134135475     https://api.github.com/repos/luchizina/phpLuna   \n",
       "993  136724786  https://api.github.com/repos/ramesesinc/module...   \n",
       "\n",
       "                    name  \n",
       "0               scalding  \n",
       "1                finagle  \n",
       "2     ceylon-ide-eclipse  \n",
       "3                flockdb  \n",
       "4         dotNET-Grupo-2  \n",
       "..                   ...  \n",
       "989  2019.Spring.AI_BOSS  \n",
       "990         cs151project  \n",
       "991           langdetect  \n",
       "992              phpLuna  \n",
       "993      modules-partner  \n",
       "\n",
       "[994 rows x 3 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 1.937997817993164\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>repo_id</th>\n",
       "      <th>url</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7589</td>\n",
       "      <td>https://api.github.com/repos/nxtbgthng/OAuth2C...</td>\n",
       "      <td>OAuth2Client</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8612</td>\n",
       "      <td>https://api.github.com/repos/jboss-switchyard/...</td>\n",
       "      <td>components</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>17345</td>\n",
       "      <td>https://api.github.com/repos/jboss-switchyard/...</td>\n",
       "      <td>tools</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>23282</td>\n",
       "      <td>https://api.github.com/repos/cloudera/crunch</td>\n",
       "      <td>crunch</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24828</td>\n",
       "      <td>https://api.github.com/repos/jboss-switchyard/...</td>\n",
       "      <td>quickstarts</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>378</th>\n",
       "      <td>128921855</td>\n",
       "      <td>https://api.github.com/repos/huangcs3/cloudtest</td>\n",
       "      <td>cloudtest</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>379</th>\n",
       "      <td>129200651</td>\n",
       "      <td>https://api.github.com/repos/wer345/vscodegit</td>\n",
       "      <td>vscodegit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>380</th>\n",
       "      <td>129732012</td>\n",
       "      <td>https://api.github.com/repos/MatteoCultrera/Fo...</td>\n",
       "      <td>Foody_User</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>134529452</td>\n",
       "      <td>https://api.github.com/repos/mibaldi/Architect...</td>\n",
       "      <td>ArchitectCodersG3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>382</th>\n",
       "      <td>134744493</td>\n",
       "      <td>https://api.github.com/repos/AdrianDanlos/E-Sp...</td>\n",
       "      <td>E-SportsProject</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>383 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       repo_id                                                url  \\\n",
       "0         7589  https://api.github.com/repos/nxtbgthng/OAuth2C...   \n",
       "1         8612  https://api.github.com/repos/jboss-switchyard/...   \n",
       "2        17345  https://api.github.com/repos/jboss-switchyard/...   \n",
       "3        23282       https://api.github.com/repos/cloudera/crunch   \n",
       "4        24828  https://api.github.com/repos/jboss-switchyard/...   \n",
       "..         ...                                                ...   \n",
       "378  128921855    https://api.github.com/repos/huangcs3/cloudtest   \n",
       "379  129200651      https://api.github.com/repos/wer345/vscodegit   \n",
       "380  129732012  https://api.github.com/repos/MatteoCultrera/Fo...   \n",
       "381  134529452  https://api.github.com/repos/mibaldi/Architect...   \n",
       "382  134744493  https://api.github.com/repos/AdrianDanlos/E-Sp...   \n",
       "\n",
       "                  name  \n",
       "0         OAuth2Client  \n",
       "1           components  \n",
       "2                tools  \n",
       "3               crunch  \n",
       "4          quickstarts  \n",
       "..                 ...  \n",
       "378          cloudtest  \n",
       "379          vscodegit  \n",
       "380         Foody_User  \n",
       "381  ArchitectCodersG3  \n",
       "382    E-SportsProject  \n",
       "\n",
       "[383 rows x 3 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(dbUtils.runQuery('''\n",
    "    CREATE VIEW crm20.temp_polyglotProjects AS (\n",
    "        SELECT DISTINCT repo_id, url, name\n",
    "        FROM temp_polyglots, ght.project_members, ght.projects\n",
    "        WHERE author_id = user_id\n",
    "        AND repo_id = id\n",
    "    );\n",
    "    SELECT * FROM  crm20.temp_polyglotProjects\n",
    "''', mute=True))\n",
    "display(dbUtils.runQuery('''\n",
    "    CREATE VIEW crm20.temp_controlgroupProjects AS (\n",
    "        SELECT DISTINCT repo_id, url, name\n",
    "        FROM temp_controlgroup, ght.project_members, ght.projects\n",
    "        WHERE author_id = user_id\n",
    "        AND repo_id = id\n",
    "    );\n",
    "    SELECT * FROM crm20.temp_controlgroupProjects\n",
    "''', mute=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the end, another analysis run is started:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "recoveryPolyglotProjects = dbUtils.runQuery('''SELECT * FROM temp_polyglotProjects''', mute=True)\n",
    "recoveryControlgroupProjects = dbUtils.runQuery('''SELECT * FROM temp_controlgroupProjects''', mute=True)\n",
    "recoveryBothProjects = pandas.concat([recoveryPolyglotProjects, recoveryControlgroupProjects], axis=0)\n",
    "start = time.time()\n",
    "repos = list(filter(lambda tupl: repoLibrarian.isJavaRepo(tupl[0], tupl[1]), map(rowToTuple, recoveryBothProjects.iterrows())))\n",
    "end = time.time()\n",
    "dbUtils.log('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1377\n",
      "619\n"
     ]
    }
   ],
   "source": [
    "print(len(recoveryBothProjects))\n",
    "print(len(repos))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows that one repository has been lost compared to the 620 of the original run. This is bearable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "To simplify the code, the final module is used here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "repoAnalysis.runFullAnalysis(repos, repoFolder='/mnt/brick/crm20/repos/', tableName='lb_results2', logfile='recovery_log.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 3.0898277759552\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>file_count</th>\n",
       "      <th>num_methods</th>\n",
       "      <th>num_lambdas</th>\n",
       "      <th>num_comment_lines</th>\n",
       "      <th>num_reflection</th>\n",
       "      <th>num_snakes</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>bfbb10eafcbe7ce74af7ff418f4ea2cf277b3996</td>\n",
       "      <td>fc9846d62801bc744b5dc9fc338e9532db580c0e</td>\n",
       "      <td>1597130908</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>347</td>\n",
       "      <td>-347</td>\n",
       "      <td>-257</td>\n",
       "      <td>-4</td>\n",
       "      <td>-36</td>\n",
       "      <td>-8</td>\n",
       "      <td>-5</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>-331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>25b81f4a68b9ec5bd694cebc92c562c4e55a0fbf</td>\n",
       "      <td>7df2c345badc749d7043293922e79e6206fd2351</td>\n",
       "      <td>1597083792</td>\n",
       "      <td>613</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>fc9846d62801bc744b5dc9fc338e9532db580c0e</td>\n",
       "      <td>ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2</td>\n",
       "      <td>1597069608</td>\n",
       "      <td>613</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2</td>\n",
       "      <td>19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f</td>\n",
       "      <td>1597064617</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f</td>\n",
       "      <td>3857bef92641cf923a014732810a9155d36daa7d</td>\n",
       "      <td>1597009837</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>539272</th>\n",
       "      <td>4dab5ec84ea69ff8db6699527c67971440753c1d</td>\n",
       "      <td>983d36fd357a75e7693057f76ceeda16932d57e9</td>\n",
       "      <td>1418149334</td>\n",
       "      <td>13395535</td>\n",
       "      <td>122</td>\n",
       "      <td>1</td>\n",
       "      <td>121</td>\n",
       "      <td>87</td>\n",
       "      <td>5</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>539273</th>\n",
       "      <td>983d36fd357a75e7693057f76ceeda16932d57e9</td>\n",
       "      <td>65c847e0d2be48038f3483fa2718f27d5ba7740c</td>\n",
       "      <td>1417386728</td>\n",
       "      <td>13395535</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>539274</th>\n",
       "      <td>65c847e0d2be48038f3483fa2718f27d5ba7740c</td>\n",
       "      <td>f0605d5b9c3e830c35619625de69437bd67a42a3</td>\n",
       "      <td>1417030624</td>\n",
       "      <td>13395535</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>539275</th>\n",
       "      <td>f0605d5b9c3e830c35619625de69437bd67a42a3</td>\n",
       "      <td>96e54c833612e1f52b056cbbcb2f85d9e732fbc3</td>\n",
       "      <td>1416338568</td>\n",
       "      <td>13395535</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>539276</th>\n",
       "      <td>96e54c833612e1f52b056cbbcb2f85d9e732fbc3</td>\n",
       "      <td>bcb394f5458ad2fc125065372d10872796f12d55</td>\n",
       "      <td>1416338527</td>\n",
       "      <td>13395535</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>539277 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             sha  \\\n",
       "0       bfbb10eafcbe7ce74af7ff418f4ea2cf277b3996   \n",
       "1       25b81f4a68b9ec5bd694cebc92c562c4e55a0fbf   \n",
       "2       fc9846d62801bc744b5dc9fc338e9532db580c0e   \n",
       "3       ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2   \n",
       "4       19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f   \n",
       "...                                          ...   \n",
       "539272  4dab5ec84ea69ff8db6699527c67971440753c1d   \n",
       "539273  983d36fd357a75e7693057f76ceeda16932d57e9   \n",
       "539274  65c847e0d2be48038f3483fa2718f27d5ba7740c   \n",
       "539275  f0605d5b9c3e830c35619625de69437bd67a42a3   \n",
       "539276  96e54c833612e1f52b056cbbcb2f85d9e732fbc3   \n",
       "\n",
       "                                          parent   timestamp   repo_id  \\\n",
       "0       fc9846d62801bc744b5dc9fc338e9532db580c0e  1597130908       613   \n",
       "1       7df2c345badc749d7043293922e79e6206fd2351  1597083792       613   \n",
       "2       ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2  1597069608       613   \n",
       "3       19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f  1597064617       613   \n",
       "4       3857bef92641cf923a014732810a9155d36daa7d  1597009837       613   \n",
       "...                                          ...         ...       ...   \n",
       "539272  983d36fd357a75e7693057f76ceeda16932d57e9  1418149334  13395535   \n",
       "539273  65c847e0d2be48038f3483fa2718f27d5ba7740c  1417386728  13395535   \n",
       "539274  f0605d5b9c3e830c35619625de69437bd67a42a3  1417030624  13395535   \n",
       "539275  96e54c833612e1f52b056cbbcb2f85d9e732fbc3  1416338568  13395535   \n",
       "539276  bcb394f5458ad2fc125065372d10872796f12d55  1416338527  13395535   \n",
       "\n",
       "        additions  deletions  loc  cloc  file_count  num_methods  num_lambdas  \\\n",
       "0               0        347 -347  -257          -4          -36           -8   \n",
       "1               7          2    5     3           0            1            0   \n",
       "2               6          3    3     1           0            0            0   \n",
       "3               0          0    0     0           0            0            0   \n",
       "4               0          0    0     0           0            0            0   \n",
       "...           ...        ...  ...   ...         ...          ...          ...   \n",
       "539272        122          1  121    87           5           13            0   \n",
       "539273          0          0    0     0           0            0            0   \n",
       "539274          1          1    0     0           0            0            0   \n",
       "539275          0          0    0     0           0            0            0   \n",
       "539276          0          0    0     0           0            0            0   \n",
       "\n",
       "        num_comment_lines  num_reflection  num_snakes  total_indent  \n",
       "0                      -5               0          -8          -331  \n",
       "1                       0               0           0            18  \n",
       "2                       0               0           0             6  \n",
       "3                       0               0           0             0  \n",
       "4                       0               0           0             0  \n",
       "...                   ...             ...         ...           ...  \n",
       "539272                  0               0           8            86  \n",
       "539273                  0               0           0             0  \n",
       "539274                  0               0           0             0  \n",
       "539275                  0               0           0             0  \n",
       "539276                  0               0           0             0  \n",
       "\n",
       "[539277 rows x 15 columns]"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT *\n",
    "    FROM crm20.lb_results2\n",
    "''', mute=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So most of the data has been reconstructed. The original cardinalities can still be found in [Results_Iteration#2](Results_Iteration#2.ipynb) notebook. The log for this run can be found in [./results/log_second_reconstruction_run.txt](./results/log_second_reconstruction_run.txt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Third (final) iteration "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As now the technology has matured, scaling up is possible. The underlying dataset for this run is a direct superset of the two preseeding runs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This run mainly uses the same technology as the run before, but scales the sample users up by factor 10 and decreases the factor of java to secondary code that polyglots needed. It is meant to revise and refactor the methodology and run a big stress test on it - so it can be released. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "tableName = 'lb_results3'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the repos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At first, again, the repos to analyze have to be collected, downloaded, and filtered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "count = 0\n",
    "total = len(repoTuples)\n",
    "def processRepoTuple(tupl):\n",
    "    isJavaRepo = False\n",
    "    with io.capture_output() as output:\n",
    "        global count\n",
    "        count = count + 1\n",
    "        print('Looking at '+str(tupl)+' - ('+str(count)+'/'+str(total)+')')\n",
    "        isJavaRepo = repoLibrarian.isJavaRepo(tupl[0], tupl[1])\n",
    "    dbUtils.log(output)\n",
    "    return isJavaRepo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepareRepos():\n",
    "    dbUtils.log('Starting to download repos')\n",
    "    start = time.time()\n",
    "    repos = list(filter(processRepoTuple, repoTuples))\n",
    "    end = time.time()\n",
    "    dbUtils.log('Total Time used: '+str(end - start))\n",
    "    return repos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.5708019733428955\n",
      "Time used: 0.42449331283569336\n"
     ]
    }
   ],
   "source": [
    "polyglotProjects = dbUtils.runQuery('''SELECT * FROM lb_polyglotProjects''', mute=True)\n",
    "controlgroupProjects = dbUtils.runQuery('''SELECT * FROM lb_controlgroupProjects''', mute=True)\n",
    "bothProjects = pandas.concat([polyglotProjects, controlgroupProjects], axis=0)\n",
    "repoTuples = list(map(rowToTuple, bothProjects.iterrows()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "repos = prepareRepos()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The log file for repo preparation, can be found under [./results/log_third_repos.txt](./results/log_third_repos.txt). It shows that approx. 16h were taken to process all repositories - that is much more than the time to actually analyze them.\n",
    "\n",
    "Again, less than half of the repositories are available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16692\n",
      "7676\n"
     ]
    }
   ],
   "source": [
    "print(len(repoTuples))\n",
    "print(len(repos))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dry run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "deleteTable()\n",
    "createTable()\n",
    "runDeltaSuite(repos[0])\n",
    "display(Audio('./beep.mp3', autoplay=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.06114506721496582\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>cloc</th>\n",
       "      <th>file_count</th>\n",
       "      <th>num_methods</th>\n",
       "      <th>num_lambdas</th>\n",
       "      <th>num_comment_lines</th>\n",
       "      <th>num_reflection</th>\n",
       "      <th>num_snakes</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>bfbb10eafcbe7ce74af7ff418f4ea2cf277b3996</td>\n",
       "      <td>fc9846d62801bc744b5dc9fc338e9532db580c0e</td>\n",
       "      <td>1597130908</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>347</td>\n",
       "      <td>-347</td>\n",
       "      <td>-257</td>\n",
       "      <td>-4</td>\n",
       "      <td>-36</td>\n",
       "      <td>-8</td>\n",
       "      <td>-5</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>-331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>25b81f4a68b9ec5bd694cebc92c562c4e55a0fbf</td>\n",
       "      <td>7df2c345badc749d7043293922e79e6206fd2351</td>\n",
       "      <td>1597083792</td>\n",
       "      <td>613</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>fc9846d62801bc744b5dc9fc338e9532db580c0e</td>\n",
       "      <td>ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2</td>\n",
       "      <td>1597069608</td>\n",
       "      <td>613</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2</td>\n",
       "      <td>19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f</td>\n",
       "      <td>1597064617</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f</td>\n",
       "      <td>3857bef92641cf923a014732810a9155d36daa7d</td>\n",
       "      <td>1597009837</td>\n",
       "      <td>613</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9169</th>\n",
       "      <td>f39852a32b6cd7d4172408cbcc9bd757ae65f4d7</td>\n",
       "      <td>7074a81040c31cf8cb239a5e946dd2609761e49e</td>\n",
       "      <td>1300984314</td>\n",
       "      <td>613</td>\n",
       "      <td>146</td>\n",
       "      <td>119</td>\n",
       "      <td>27</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9170</th>\n",
       "      <td>7074a81040c31cf8cb239a5e946dd2609761e49e</td>\n",
       "      <td>4037881e9c1da2832ad58af52f4e0b60f2441ca0</td>\n",
       "      <td>1300923433</td>\n",
       "      <td>613</td>\n",
       "      <td>117</td>\n",
       "      <td>135</td>\n",
       "      <td>-18</td>\n",
       "      <td>-20</td>\n",
       "      <td>0</td>\n",
       "      <td>-6</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>-33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9171</th>\n",
       "      <td>4037881e9c1da2832ad58af52f4e0b60f2441ca0</td>\n",
       "      <td>4bc9c5d53d138286134b17a730d59793f5795513</td>\n",
       "      <td>1299441062</td>\n",
       "      <td>613</td>\n",
       "      <td>132</td>\n",
       "      <td>107</td>\n",
       "      <td>25</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>34</td>\n",
       "      <td>-1</td>\n",
       "      <td>-19</td>\n",
       "      <td>-27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9172</th>\n",
       "      <td>4bc9c5d53d138286134b17a730d59793f5795513</td>\n",
       "      <td>8119b86c903994007f4d7fd886585e2a85c7c45b</td>\n",
       "      <td>1299324097</td>\n",
       "      <td>613</td>\n",
       "      <td>168</td>\n",
       "      <td>107</td>\n",
       "      <td>61</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9173</th>\n",
       "      <td>8119b86c903994007f4d7fd886585e2a85c7c45b</td>\n",
       "      <td>1c465aa99502694c99e50ed1a6d1435f3668acaa</td>\n",
       "      <td>1299246034</td>\n",
       "      <td>613</td>\n",
       "      <td>106</td>\n",
       "      <td>87</td>\n",
       "      <td>19</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9174 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           sha  \\\n",
       "0     bfbb10eafcbe7ce74af7ff418f4ea2cf277b3996   \n",
       "1     25b81f4a68b9ec5bd694cebc92c562c4e55a0fbf   \n",
       "2     fc9846d62801bc744b5dc9fc338e9532db580c0e   \n",
       "3     ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2   \n",
       "4     19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f   \n",
       "...                                        ...   \n",
       "9169  f39852a32b6cd7d4172408cbcc9bd757ae65f4d7   \n",
       "9170  7074a81040c31cf8cb239a5e946dd2609761e49e   \n",
       "9171  4037881e9c1da2832ad58af52f4e0b60f2441ca0   \n",
       "9172  4bc9c5d53d138286134b17a730d59793f5795513   \n",
       "9173  8119b86c903994007f4d7fd886585e2a85c7c45b   \n",
       "\n",
       "                                        parent   timestamp  repo_id  \\\n",
       "0     fc9846d62801bc744b5dc9fc338e9532db580c0e  1597130908      613   \n",
       "1     7df2c345badc749d7043293922e79e6206fd2351  1597083792      613   \n",
       "2     ad1b1b38a1c5cbd811bdb7f9ee58f19b18216cd2  1597069608      613   \n",
       "3     19b3ee2d164aa063a9a66f9fb12fc043a1c47a4f  1597064617      613   \n",
       "4     3857bef92641cf923a014732810a9155d36daa7d  1597009837      613   \n",
       "...                                        ...         ...      ...   \n",
       "9169  7074a81040c31cf8cb239a5e946dd2609761e49e  1300984314      613   \n",
       "9170  4037881e9c1da2832ad58af52f4e0b60f2441ca0  1300923433      613   \n",
       "9171  4bc9c5d53d138286134b17a730d59793f5795513  1299441062      613   \n",
       "9172  8119b86c903994007f4d7fd886585e2a85c7c45b  1299324097      613   \n",
       "9173  1c465aa99502694c99e50ed1a6d1435f3668acaa  1299246034      613   \n",
       "\n",
       "      additions  deletions  loc  cloc  file_count  num_methods  num_lambdas  \\\n",
       "0             0        347 -347  -257          -4          -36           -8   \n",
       "1             7          2    5     3           0            1            0   \n",
       "2             6          3    3     1           0            0            0   \n",
       "3             0          0    0     0           0            0            0   \n",
       "4             0          0    0     0           0            0            0   \n",
       "...         ...        ...  ...   ...         ...          ...          ...   \n",
       "9169        146        119   27    29           0            3            0   \n",
       "9170        117        135  -18   -20           0           -6            0   \n",
       "9171        132        107   25    24           0            6            1   \n",
       "9172        168        107   61    63           0            9            0   \n",
       "9173        106         87   19    19           0            1            0   \n",
       "\n",
       "      num_comment_lines  num_reflection  num_snakes  total_indent  \n",
       "0                    -5               0          -8          -331  \n",
       "1                     0               0           0            18  \n",
       "2                     0               0           0             6  \n",
       "3                     0               0           0             0  \n",
       "4                     0               0           0             0  \n",
       "...                 ...             ...         ...           ...  \n",
       "9169                 -7               0           0           -77  \n",
       "9170                  2               0          11           -33  \n",
       "9171                 34              -1         -19           -27  \n",
       "9172                  3               0           0           -20  \n",
       "9173                  2               0           1            90  \n",
       "\n",
       "[9174 rows x 15 columns]"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT * FROM crm20.'''+tableName+'''\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start the run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def runFullAnalysis(repos):\n",
    "    createTable()\n",
    "    start = time.time()\n",
    "    with Pool(int(multiprocessing.cpu_count()*3/4)) as pool:\n",
    "        allMetrics = pool.map(runDeltaSuite, repos)\n",
    "    end = time.time()\n",
    "    dbUtils.log('Total Time used: '+str(end - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "runFullAnalysis(repos)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Total Time used: 9503.501674413681` for almost 8k repositories this is really nice.<br>\n",
    "The full log file, as always, can be found under [./results/log_third_data_run.txt](./results/log_third_data_run.txt)\n",
    "\n",
    "Teh evaluation of the results can be found in the [Results_Iteration#3](Results_Iteration#3.ipynb) notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Extracting to Module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To release the code for running analysis suites, the code should be extracted to the [repoAnalysis](repoAnalysis.py) and [dbUtils](dbUtils.py) modules. Especially parameters have been added to replace the global state used in the preceding runs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section aims at testing if the extracted methods work and to showcase their usage to potential future users."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "repos = list(filter(lambda tupl: repoLibrarian.isJavaRepo(*tupl), [('bptlab', 'scylla', 13377331), ('bptlab', 'fcm2cpn', 4242)]))\n",
    "customSuite = [repoAnalysis.loc, repoAnalysis.total_indent]\n",
    "testTableName = 'lb_module_test'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "repoAnalysis.runFullAnalysis(repos, tableName='lb_module_test', repoFolder='./local/', logfile='test_log.txt', suite=customSuite)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time used: 0.02479243278503418\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/mpeg;base64,SUQzAwAAAAAAJlRQRTEAAAAcAAAAU291bmRKYXkuY29tIFNvdW5kIEVmZmVjdHMA//uSwAAAAAABLBQAAALeM2u3K0AIA4eAoGAoEAAABAAFQ28DVSOAzwcO4GThIAaMPAxgEgMBgnwPxmA6Sb8BcGApNA0af/DNgatGDWAB9o338DgnQxMF1AcQHu/s20DEgQMkEAwQIRYA4oKF/++HLjOibgDgAGlHiAQep//+NIZgph+YGBBhY+J9Eplf//f+GWxzhBAXEHyFoSgQ0TuOEUH///f/xigDBwyEAUIC2QAQQbINggPRAOBhiUGwWPQNgsQG////8oBCgnAZcBADdzRTAHMoAAAAAAADqwzMXAErIXlCE8wyPGWtbgWN05gMOGKRVbq1aMwRBI4tIzhQQtoDEGEEEYSisYZiG3k1ce/9mbCZmMSamsQBHVua73YCoq6/gqPJqWESxwuAS2AMCQYPVLfzyCAGhOeFUhBMwfD08dMIxfDIqgMAganQQAJoQJREBFHlfEIOGTJVsSwvJpr5T0AQxmWIJCAAWzc7ZCogp7RW3ukgMkE8SEyTZPrArrDAZmGI1DQFsyYmgl5/xgSGV1O4c/Hiixg2BNrXNM9WFf/7ksBqgCRJrU3ZzpACX60pu7VQAMCHXAUqghys44/etXP7rndZ61/b/d8/nLzIYi5NSUwYtJ2n+p4ItsRfGM/3D9fnh+/1hnh/P1/ceb1///NY8rata19k7/jIuLGv/TDSgAWZ2kgBAAAA5FDWdwpuyeG53GvlyniK5VIyyQ0NlAwICwMbB4VsSTIurUhZkZxKQ0XKFroBqLACCCzY1RQSdqKqZ4xLx8pDJhb6AQhwNvjwDEoIC4IW8ipNGKKk3pKz6nRFmiCoARwBsKOaQWpKyt/qVHyKKCIjC5iert2+1XRUlTrWun2qq6TalWt/9az+YXXTl//U+4cAADlnWwAEAAAGUOfWPYpEnzEc4mp+5k8AQCgh3qZTuMfWVY4fv//Hv3dfjr8OxBYc7JwJJUiRMqTSWkpTus2ZMtJF4csMHALLMDMhSC6I5Q41GCaDm6107nrnVkAAyCHAxNOWZCnsgy15xGCQEChSY421S99WvlsM1NUF1jVJXrdVvWt79y2Tl5z7LamOaTJgBQ0PYACAAADKHRkEFcNSiVjG5UmVNm3/+5LAU4AR7Q9Lymargkeh6XlN1XCUX/dRRcyMwh+pnrXP//33LL98zuVWFGGUAAADEfEFKyLrap60EVE6ZFckROQGFBOB0wDAsERby4bufrNkFoMt1LQQMB1gYuIYcqgZsyaqVk1LdR+cZIGgWBxGRSQ219ar9azRiFtK0qUjrGfetznPYptrJ/2Mt2UtckAAF2eLAAQAACvBr/w+dfu3S0UESl6X1hQYZvxrC2yg14VaBT4d73fe1/uf3ueq9I+6EwyuCIqfVTfMO3sfZpaL5gdNSyJ0DA4GClQBw4ngKAQQuLgJsvmyaKC5q63TsbJi0AYVNAdpI0SRRSdOiippx0ziKgaC4FiEipvUv9660DG1mVVerrWp859f7r+rPXrONSw8+cc521llQcIChJ6wBIVlkAAAAAK8G/+eqP3JbMzPao79AvNZEA45XnKNlBWpTu+fv+8/+Y/rf41bTWTLYszEGWq/07qplnzZ5JSJkxJGpOjOA3qBgFiAa1LoWFDKjgMTqDomlR5FbGk8XTgvwHm8QYSJ9BjiqrLVpPMjxkEA//uSwIkAE/FrSc1urYJzrWk5rdWw6BYRI0a3pra+tVW5WXVrqt2ZVb7ttX1/VdlVHlOGTg1ppzb3dm+ggmhtAAAGZGgAAAAABnDWuS4yhNEe6iQJApRAiCaDLGVI6mlHzi02/5r//n57/dTuE2+gjAzGNUFSD8z9m1c+/l9ajdIsk8YlQYohoGJQIAzfwcLhBhfJw8a5VSOJqnjdSJkiLUBiw6B+5eKqSKak01HkUkk03WZMYBImAoAlpmrpn9eyq2rUqTi1u9VqaS3a6K3adWprLq/spCucXegt6VkFddJSp65O8la0RkiJJBFJsARkV6AAAAACvhh/0mdqB71eQ0MrhmH0JBatHjdSSlQEIqMEV9d/f87l+ev3lztNGGbnEkHaCrWf2mzyxy/I3rPmhTKhsTQlMCgAAwS2QOBmgAYDiOBS5MFw1YyXUbJqNULmKiGgZPKYY8QUnkH3NNSVJb0i6cCZOGYRTPUT6C1tatzy1MpRZRe9dkXdDW1d7bLrdN1fV1qUbra1VfXelek5tZpFGpTQSMGSDWpAABZmWQAAAP/7ksCxABWxl0fKbq2CszLo+a1VsAAr4Z81a3FZfLrdPZrQ9ZfFB5iWrdRHc1M6afYz1hrf5fzDX9r8r0EGjgAM9RjoOwaKz2OWN7erW91KvYFpp58S+xhmqBw4yAFAUWUONI4iyRugs+yW7nESSAzEHAslKxM0zNPVW7uYJJJHjgQkoFAgbonlLetbvrSWtaSkSeQStQo0alpXZFFnOMpkN+1q1aDJKP3sgykLoqrWhXzWzP2vXsr1mcZaxl9AG2fgAGIRedYliCEyUaY2RzELk/G+8ttMOMA1aJ+xvd/fPz5h/63hXvP+hINTJAV7NNgKlwqZXt44a+ixoIzlG2gKjMsGANAAAQWBwG/kUIOfLdZWWtBNzdNZuZl8cwDDqBAGwxiQ5NIuOgit1OhOM1EwBMJwKBMWdPIredXRPLpOmhniWS2ZOjN61oWZFk1pt910amd2RsqdNlstp9SSnUvSXQWg5e6vSZV+yrqOtLkBv6AAF2NoAAAAACuEw/7GWE9NRGc1I4rdbrHFVbOVKpUZ/Fazr2ePL2uY4fd397Wd6bj/+5LAygAWDcNHzW6tysy4KJFd2bniLphmBgkII4u9IdVKmet0Wrl6/SRedlKpBkCmBIYBuJBgAAIP3EeE2XyeSMUlLPMs+XFoGhgK+BmIrgHAFRTZA2ZDU7z7oUTUwBCPgWOBudRqetbrRrTPrXOoF/VoOzoLrZSVkc6t3ap6bVMy6r0DqGtkVTta1Ulv2mNH/a9a7M1F1zqaTsPgBu7NAAAAAAUhH8+mwddrbw0sgtx6/SusngyrK7EiAFM0WHdpu/rLWHf7n3W6mdSnfQKgJo06UdtqS6z1y/r0VqK5eMRxGJIjHALAMDDbeA7mawBg8I9HAanXTRNHUWlHXJuxsdE2AY/QQFgEfJhNjFSC1zk0rZMxQNAQiYFkYfMUaDoHU6PTUtSjqyLLUp70Ji6rKWyDL3vZLq23ZddZaoLZSNBTIrW7HlrVcyuHWtS7IiYhNigAFXVpAAAAACuE/95/I6SGpbRYTdukcJibmY6myQEMoW3Yr1O93/Msvwy525jlNTq0jOs4zoTRycWXXc7OVfpvTK8lSkYlwRuGMAMWlED+//uSwN6AFu3LRc3yrcLaMui5vdWwoeAsOw5QgZEDx5pVWpFa2MHPF0oCXgYES4aQSxPs1SCS0HnpguYmrAVBAUTBXMXdR9dbpspS2akixLHa3ZGgo2Und1VXmVnqSpr1Vtay1qKtNenS1K616pio3f8tZoYuKl7Kjmm0AG6scgAAAABSE4d+pqBYrN0T8Se5SUjD2vNhwzqJpm8mS/5ZXw5r8ea5hq3qvhnHG5iAGMr6AVfu9SX8MOXuYn1HVFwzG8XiaEFwgBABsXA0eiAFwGIQCdyYLhVQOGqCnQmiKCBSTFYAw0lAxWTxUsxx1onlKQcsKRLLMKDCidSRSZR9q9Gqh0EivrakyqKC1pOuhZJ23VZbL00HW1aqjzCa7Xlue17xv/d3TnHtHHW/7SSgAAAWVUkAAAAAK4T8ML+WcPx7Gc3U5OQCiUsNlZnm1CRhWmf/u6u9bz/PWXKHDdWfasae6mxhSSLvS3XLlnXoLYqGpSHCTRLiyQblAYgYIHoC0AUJRBw7z6jyKSa1H3UeOnJRMhaQMmnkNEHaSSB44p1WTf/7ksDuABadl0XN7q2Kzi1oub3VsUazJpTQNQQhUHJktomzq0lukiknXrUsy2ZbWukhQWhW6bFl3srQbTd6Z5ynGD9Mc1TWQf8xz3TShlzZNd3e75OABQ6tIAAAAAUhP71nizd4YDjdHL4TF5cslfDTccpSjMZpILNqXuP6/me9/vW6lvCPwAMgcCQsxkCFcyKdy3hfy0VETAxMygTai+HuBc+Big/AfcHIDAwDbyAEXNnTUV6z61ImSTlBiDAZqIgN0SeJ1JE7dNE81BFdE6giCEXgshUkT6KmqRUp1aSmqYkU7vdSDIpOt3T3VRtr6a3W9zqFkkXsu45SZoW4UA6UaUmf0TpWS8e/22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB/SUAFITrP5ZTus1mCpXGrErsS3rHe1s44VAOYELkA5fzG3hv+bwx5nUz1bhhIsHKUBPFez+01fPdrOtj+r9u9CqWlYAWAOORoDESLBoAhExOhmblZNM2danSPmKj5mmTgDDvANABdIskkg6SqKaTKQT/+5LA/4AWTV9Fze6titgraLm+VbGYxUYgmRQWMps59Sj0wukp3ZmZaUk23oqZHZezLdNRzZGpSF3M0rHWvdIt13upe70r1VzvdVlnqf+TR15nYhp2gAUNEkAAAAAK4Lf67VuSeZzq15VJaFhQKA6QO7E2OAIycUWuW6meOGu81+Ot6y5nJI+SAAwZCB0IpXPLPb7d5v7+W5+tJYepZA05AKYlTgHxSOAwLhGpIFpE+cOoHlmiM+ZJpGhoMuBmwbg2ZNiig70z+i62LC0zsugmUQUHqlFZFR5SmUqtd1IIqRLi71pKRZb16aq1Mm1G6Ck0XdO8xtXWVqqrKUuiqnQWnaYUKFtHezGd+sRmkvEYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAChWaQAEAAAr4d5vHjcYGqXZdPTkbk0PJJQJjnSM7NzEGP0mesP3lhvW8s+V8LtJDaCQz1uHs6KyTDmGsOY39XfqTcs7cWukIY3HgeBHwDAfD5CBkUPIqWVFKNVLdNlpGYpADIpcCyo+WE5xrU2WpBTT//uSwP+AFkmnRI3yrcrltui5vlW4qR0JksFACkgbugedBnV2SZFdA1X2RrWtddmZ9ii9kKqC9dqL6lGaCDOrqVrrSermamUyfoorZNaNfsk6tnoPNgA5ZXkAAAAAD8qGCYDXE1KIAizBnS15dBWxrEbZmyswURMPFzGxd/H8ZQKAZugFeawzhyHfledBKIxGNU8MQ5DlJJVpoJzBRczvTErOPtgWIsRrkOSmNu45E5QwGytd7pwplYJAphMKmMZeZSXBjwFFkJKu9IhFBTRrkmm43D8bs3Ybct233jSCQxsiQcFFeSyxuG6evfzt16evX7LIw7bW1TkhVSIby3G43GJZSYcz7bz5zmedfuVHXz7X539fnvvOcy5rWWFjL+bzqYc/889581/f73d7P957/+4Z97/P1hl/6qakbECIPeUXrPnf/JRwAQMAAjDYNEnKkKg6oKmC4tiUv6/r+v6/rEXJisy/ogAmMDkowzJIOBP08K5kxlBmJP1XdlnK7XKkT6o8lpQIFCBJIHN1nOW8NKWMaASuiamKKqRLEadlKJpclP/7ksD/gBZB20fNbq3DxrLoua3xsQCpq6BdYwkYMSITLC8yQ7N8xz5sk4JDMqLTJhcwwLQ2k7AUJKDqgrizDAlAUxV0xVuRcIwMMMfMjQT4IIYDQCqCrph2drRqMy3laGn+f5/n+pbCwxIBmKC4GFVBXFs1Yy/r+w7LbOq0pjMtpearU1rHHWVNTU2W61NTU1rtWllMal1NTdxxxx/8aWlpbPMsccaXHn5ZY4/rLLLL/1lllvHHHLLfN41aWydFBRUEFBWPimhQ3woAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA9AAiAZ89DGttiAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//uSwP+AAAABLAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7ksD/gAAAASwAAAAAAAAlgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAD/+5LA/4AAAAEsAAAAAAAAJYAAAAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAA\" type=\"audio/mpeg\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>parent</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>repo_id</th>\n",
       "      <th>additions</th>\n",
       "      <th>deletions</th>\n",
       "      <th>loc</th>\n",
       "      <th>total_indent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>517d90b38b862f157d08a462f2cc9ed4d64e6c74</td>\n",
       "      <td>7361decd7afe0a558ba9e61e8743f23ee52dcb24</td>\n",
       "      <td>1596288222</td>\n",
       "      <td>4242</td>\n",
       "      <td>57</td>\n",
       "      <td>43</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7361decd7afe0a558ba9e61e8743f23ee52dcb24</td>\n",
       "      <td>f37c6829849431618c5b4c011d5704ce25be8f3f</td>\n",
       "      <td>1596285127</td>\n",
       "      <td>4242</td>\n",
       "      <td>13</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>f37c6829849431618c5b4c011d5704ce25be8f3f</td>\n",
       "      <td>3adb5ef52051bedb9c6de7b27b21de9f0d551cc8</td>\n",
       "      <td>1596279743</td>\n",
       "      <td>4242</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3adb5ef52051bedb9c6de7b27b21de9f0d551cc8</td>\n",
       "      <td>1773ba29816ff7dfeec8075e5a7fcf7e100d1f03</td>\n",
       "      <td>1596273974</td>\n",
       "      <td>4242</td>\n",
       "      <td>42</td>\n",
       "      <td>16</td>\n",
       "      <td>26</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1773ba29816ff7dfeec8075e5a7fcf7e100d1f03</td>\n",
       "      <td>7a2e0afbfd49e8d719a684cafe23f8ef4fb2fddb</td>\n",
       "      <td>1595599925</td>\n",
       "      <td>4242</td>\n",
       "      <td>48</td>\n",
       "      <td>20</td>\n",
       "      <td>28</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>529</th>\n",
       "      <td>3a525dd8c0178158275815e47e6f7970372489df</td>\n",
       "      <td>a019eafb85e1d093b5a9ac75792d3c3793e0fcfe</td>\n",
       "      <td>1485388416</td>\n",
       "      <td>13377331</td>\n",
       "      <td>373</td>\n",
       "      <td>26</td>\n",
       "      <td>345</td>\n",
       "      <td>740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>530</th>\n",
       "      <td>411d51771ee6c714db13c23e8912086840b84f51</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1485386460</td>\n",
       "      <td>13377331</td>\n",
       "      <td>237</td>\n",
       "      <td>189</td>\n",
       "      <td>47</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>531</th>\n",
       "      <td>e36dd96a1201ad35578a5e5d65a5db02135f4614</td>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>1484674213</td>\n",
       "      <td>13377331</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>532</th>\n",
       "      <td>0d893a471f932b921473f72563a540da46f15fa5</td>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>1484673978</td>\n",
       "      <td>13377331</td>\n",
       "      <td>66</td>\n",
       "      <td>22</td>\n",
       "      <td>44</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>533</th>\n",
       "      <td>9e8faf6f0b516334451c53960dee001a6e9e42c4</td>\n",
       "      <td>e1ca54916ff570656eb1d40c13b7e58c10d8bc0a</td>\n",
       "      <td>1484305822</td>\n",
       "      <td>13377331</td>\n",
       "      <td>12786</td>\n",
       "      <td>0</td>\n",
       "      <td>12767</td>\n",
       "      <td>25717</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>534 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          sha  \\\n",
       "0    517d90b38b862f157d08a462f2cc9ed4d64e6c74   \n",
       "1    7361decd7afe0a558ba9e61e8743f23ee52dcb24   \n",
       "2    f37c6829849431618c5b4c011d5704ce25be8f3f   \n",
       "3    3adb5ef52051bedb9c6de7b27b21de9f0d551cc8   \n",
       "4    1773ba29816ff7dfeec8075e5a7fcf7e100d1f03   \n",
       "..                                        ...   \n",
       "529  3a525dd8c0178158275815e47e6f7970372489df   \n",
       "530  411d51771ee6c714db13c23e8912086840b84f51   \n",
       "531  e36dd96a1201ad35578a5e5d65a5db02135f4614   \n",
       "532  0d893a471f932b921473f72563a540da46f15fa5   \n",
       "533  9e8faf6f0b516334451c53960dee001a6e9e42c4   \n",
       "\n",
       "                                       parent   timestamp   repo_id  \\\n",
       "0    7361decd7afe0a558ba9e61e8743f23ee52dcb24  1596288222      4242   \n",
       "1    f37c6829849431618c5b4c011d5704ce25be8f3f  1596285127      4242   \n",
       "2    3adb5ef52051bedb9c6de7b27b21de9f0d551cc8  1596279743      4242   \n",
       "3    1773ba29816ff7dfeec8075e5a7fcf7e100d1f03  1596273974      4242   \n",
       "4    7a2e0afbfd49e8d719a684cafe23f8ef4fb2fddb  1595599925      4242   \n",
       "..                                        ...         ...       ...   \n",
       "529  a019eafb85e1d093b5a9ac75792d3c3793e0fcfe  1485388416  13377331   \n",
       "530  9e8faf6f0b516334451c53960dee001a6e9e42c4  1485386460  13377331   \n",
       "531  0d893a471f932b921473f72563a540da46f15fa5  1484674213  13377331   \n",
       "532  9e8faf6f0b516334451c53960dee001a6e9e42c4  1484673978  13377331   \n",
       "533  e1ca54916ff570656eb1d40c13b7e58c10d8bc0a  1484305822  13377331   \n",
       "\n",
       "     additions  deletions    loc  total_indent  \n",
       "0           57         43     14            14  \n",
       "1           13          8      5            18  \n",
       "2            2          1      1             3  \n",
       "3           42         16     26            48  \n",
       "4           48         20     28            65  \n",
       "..         ...        ...    ...           ...  \n",
       "529        373         26    345           740  \n",
       "530        237        189     47            57  \n",
       "531          0          0      0             0  \n",
       "532         66         22     44           135  \n",
       "533      12786          0  12767         25717  \n",
       "\n",
       "[534 rows x 8 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dbUtils.runQuery('''\n",
    "    SELECT * FROM crm20.'''+testTableName+'''\n",
    "''')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "dbUtils.deleteTable(testTableName)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Contents of test_log.txt:\n",
    "```\n",
    "========= 2020-08-16 00:59:01.439596 ==========\n",
    "Time used for ('bptlab', 'fcm2cpn', 4242): 2.717569589614868\n",
    "\n",
    "========= 2020-08-16 00:59:06.756431 ==========\n",
    "Time used for ('bptlab', 'scylla', 13377331): 7.9743125438690186\n",
    "\n",
    "========= 2020-08-16 00:59:07.256119 ==========\n",
    "Total Time used: 50.08342528343201\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  },
  "toc-autonumbering": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}