Udacity_ML_Capstone_Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Udacity ML Capstone: Predicting Device Failures\n",
    "### by Michael Lin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Our challenge is to build a machine learning model to predict the probability of a device failure. Success criteria is to correctly identify the true failure cases, while at the same time, minimize false positives and false negatives."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First Look at Data\n",
    "\n",
    "We'll start by looking at the dataset, getting a feel for the challenge, before formulating our game plan. \n",
    "\n",
    "Initial analysis of the data reveals the following: \n",
    "\n",
    "- The dataset is telemetry logs of 124,494 records, collected from 1,169 devices. \n",
    "\n",
    "- The device failure rate is about 9.07%.  \n",
    "\n",
    "- The data sources possilby comes from 3 device types.\n",
    "\n",
    "- The data coverage spans time period of 2015/1/1 to 2015/11/2. \n",
    "\n",
    "- The duration of the device in service also varies widely, ranging from 1 day to 304 days.\n",
    "\n",
    "- In terms of features, we have 9 attributes which are numerical in nature.   \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# install dependencies\n",
    "\n",
    "# !conda install -c numpy pandas matplotlib seaborn scikit-learn -y\n",
    "# !conda install -c conda-forge imbalanced-learn -y\n",
    "# !conda install -c conda-forge xgboost -y\n",
    "# !conda install -c conda-forge lightgbm -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>attribute1</th>\n",
       "      <th>attribute2</th>\n",
       "      <th>attribute3</th>\n",
       "      <th>attribute4</th>\n",
       "      <th>attribute5</th>\n",
       "      <th>attribute6</th>\n",
       "      <th>attribute7</th>\n",
       "      <th>attribute8</th>\n",
       "      <th>attribute9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>215630672</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "      <td>6</td>\n",
       "      <td>407438</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>61370680</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>403174</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>173295968</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>237394</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>79694024</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>410186</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>135970480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>313173</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure  attribute1  attribute2  attribute3  \\\n",
       "0  2015-01-01  S1F01085        0   215630672          56           0   \n",
       "1  2015-01-01  S1F0166B        0    61370680           0           3   \n",
       "2  2015-01-01  S1F01E6Y        0   173295968           0           0   \n",
       "3  2015-01-01  S1F01JE0        0    79694024           0           0   \n",
       "4  2015-01-01  S1F01R2B        0   135970480           0           0   \n",
       "\n",
       "   attribute4  attribute5  attribute6  attribute7  attribute8  attribute9  \n",
       "0          52           6      407438           0           0           7  \n",
       "1           0           6      403174           0           0           0  \n",
       "2           0          12      237394           0           0           0  \n",
       "3           0           6      410186           0           0           0  \n",
       "4           0          15      313173           0           0           3  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load in sample dataset\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "url = 'https://udacity-data-michlin0825.s3-ap-northeast-1.amazonaws.com/device_failure.csv'\n",
    "df = pd.read_csv(url, header = 0, delimiter = ',', encoding = \"ISO-8859-1\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "124494"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# total number of daily logs\n",
    "\n",
    "df.count()[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "106"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# daily logs with failure record\n",
    "\n",
    "len(df[df['failure']==1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0851446656063746"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# percentage of daily logs with failure\n",
    "\n",
    "len(df[df['failure']==1]) / df.count()[0] * 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1169"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# total number of device \n",
    "\n",
    "len(df.groupby('device'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9.067579127459368"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# percentage of device with failure\n",
    "\n",
    "len(df[df['failure']==1]) / len(df.groupby('device')) * 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "S1F    0.440648\n",
       "W1F    0.347551\n",
       "Z1F    0.211801\n",
       "Name: device, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# device types\n",
    "\n",
    "df['device'].str[:3].value_counts(normalize = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2015-01-01\n",
       "Name: date, dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dataset timeframe\n",
    "\n",
    "df['date'].head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "124493    2015-11-02\n",
       "Name: date, dtype: object"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['date'].tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "304\n",
      "106.4961505560308\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# device service days\n",
    "\n",
    "print(df.groupby('device')['date'].count().max())\n",
    "print(df.groupby('device')['date'].count().mean())\n",
    "print(df.groupby('device')['date'].count().min())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1098b5208>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAXCElEQVR4nO3dfYxc1XnH8e8vYF7EUhsCWbm21YXiViG4cfAIqKiiXUgTY6KaSNA6ssAkrjZtiUQUUmESqSFNUZ22Dm0USrqJKSZJs1AShEWgLTWMkP8AYhPjFxzKgrfU2LVFMIZxElI7T/+YYxgvszvvu7OH30dazb3nnjv3eebOPHvnzJ25igjMzCwv75rqAMzMrP1c3M3MMuTibmaWIRd3M7MMubibmWXo+KkOAOCMM86Ivr6+htc7dOgQp5xySvsDmgK55OI8uksueUA+ubQzj82bN78cEWdWW9YVxb2vr49NmzY1vF6xWKS/v7/9AU2BXHJxHt0llzwgn1zamYek/x5vmYdlzMwy5OJuZpYhF3czswy5uJuZZcjF3cwsQy7uZmYZcnE3M8uQi7uZWYZc3M3MMlT3N1QlHQdsAl6KiI9KOgsYBk4HngKujohfSjoRuAtYBPwU+KOIGG175Enfqh+2tP7o6svbFImZWfdo5Mj9emBnxfxXgFsjYj5wAFiZ2lcCByLiHODW1M/MzCZRXcVd0lzgcuBbaV7AJcC9qcs64Io0vTTNk5ZfmvqbmdkkUT3XUJV0L/DXwKnA54BrgcfT0TmS5gEPRcR5krYDiyNid1r2PHBhRLw85j4HgUGA3t7eRcPDww0HXyqV2HXwSMPrVVowZ2ZL67dLqVSip6dnqsNomfPoLrnkAfnk0s48BgYGNkdEodqymmPukj4K7I+IzZL6jzZX6Rp1LHurIWIIGAIoFArRzK+kFYtF1mw81PB6lUaXN77dTvAv3nUX59F9csllsvKo5wPVi4E/kLQEOAn4NeDvgVmSjo+Iw8BcYE/qvxuYB+yWdDwwE3il7ZGbmdm4ao65R8RNETE3IvqAZcAjEbEceBS4MnVbAdyfptenedLyR6KesR8zM2ubVs5zvxH4rKQR4N3A2tS+Fnh3av8ssKq1EM3MrFENXYkpIopAMU2/AFxQpc8vgKvaEJuZmTXJ31A1M8uQi7uZWYZc3M3MMuTibmaWIRd3M7MMubibmWXIxd3MLEMu7mZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDLm4m5llyMXdzCxDLu5mZhmqWdwlnSTpSUlPS9oh6Uup/U5JuyRtSX8LU7skfU3SiKStks7vdBJmZnaseq7E9AZwSUSUJM0ANkp6KC3784i4d0z/y4D56e9C4PZ0a2Zmk6SeC2RHRJTS7Iz0N9EFr5cCd6X1HgdmSZrdeqhmZlYvRUxUp1Mn6ThgM3AOcFtE3CjpTuB3KR/ZbwBWRcQbkh4AVkfExrTuBuDGiNg05j4HgUGA3t7eRcPDww0HXyqV2HXwSMPrVVowZ2ZL67dLqVSip6dnqsNomfPoLrnkAfnk0s48BgYGNkdEodqyui6QHRFHgIWSZgH3SToPuAn4X+AEYAi4EfhLQNXuosp9DqX1KBQK0d/fX08oxygWi6zZeKjh9SqNLm98u51QLBZp5jHoNs6ju+SSB+STy2Tl0dDZMhHxKlAEFkfE3jT08gbwz8AFqdtuYF7FanOBPW2I1czM6lTP2TJnpiN2JJ0MfAj4ydFxdEkCrgC2p1XWA9eks2YuAg5GxN6ORG9mZlXVMywzG1iXxt3fBdwTEQ9IekTSmZSHYbYAf5L6PwgsAUaAnwGfaH/YZmY2kZrFPSK2Ah+o0n7JOP0DuK710MzMrFn+hqqZWYZc3M3MMuTibmaWIRd3M7MMubibmWXIxd3MLEMu7mZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDLm4m5llyMXdzCxDLu5mZhlycTczy5CLu5lZhuq5hupJkp6U9LSkHZK+lNrPkvSEpOck3S3phNR+YpofScv7OpuCmZmNVc+R+xvAJRHxfmAhsDhd+PorwK0RMR84AKxM/VcCByLiHODW1M/MzCZRzeIeZaU0OyP9BXAJcG9qXwdckaaXpnnS8kslqW0Rm5lZTSpfz7pGJ+k4YDNwDnAb8LfA4+noHEnzgIci4jxJ24HFEbE7LXseuDAiXh5zn4PAIEBvb++i4eHhhoMvlUrsOnik4fUqLZgzs6X126VUKtHT0zPVYbTMeXSXXPKAfHJpZx4DAwObI6JQbdnx9dxBRBwBFkqaBdwHvLdat3Rb7Sj9bf9BImIIGAIoFArR399fTyjHKBaLrNl4qOH1Ko0ub3y7nVAsFmnmMeg2zqO75JIH5JPLZOXR0NkyEfEqUAQuAmZJOvrPYS6wJ03vBuYBpOUzgVfaEayZmdWnnrNlzkxH7Eg6GfgQsBN4FLgydVsB3J+m16d50vJHop6xHzMza5t6hmVmA+vSuPu7gHsi4gFJzwDDkv4K+DGwNvVfC3xb0gjlI/ZlHYjbzMwmULO4R8RW4ANV2l8ALqjS/gvgqrZEZ2ZmTfE3VM3MMuTibmaWIRd3M7MMubibmWXIxd3MLEMu7mZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDLm4m5llyMXdzCxDLu5mZhlycTczy5CLu5lZhuq5zN48SY9K2ilph6TrU/vNkl6StCX9LalY5yZJI5KelfSRTiZgZmZvV89l9g4DN0TEU5JOBTZLejgtuzUi/q6ys6RzKV9a733ArwP/Kem3IuJIOwM3M7Px1Txyj4i9EfFUmn6d8sWx50ywylJgOCLeiIhdwAhVLsdnZmado4iov7PUBzwGnAd8FrgWeA3YRPno/oCkrwOPR8R30jprgYci4t4x9zUIDAL09vYuGh4ebjj4UqnEroOtvSFYMGdmS+u3S6lUoqenZ6rDaJnz6C655AH55NLOPAYGBjZHRKHasnqGZQCQ1AN8H/hMRLwm6Xbgy0Ck2zXAJwFVWf1t/0EiYggYAigUCtHf319vKG8qFous2Xio4fUqjS5vfLudUCwWaeYx6DbOo7vkkgfkk8tk5VHX2TKSZlAu7N+NiB8ARMS+iDgSEb8CvslbQy+7gXkVq88F9rQvZDMzq6Wes2UErAV2RsRXK9pnV3T7GLA9Ta8Hlkk6UdJZwHzgyfaFbGZmtdQzLHMxcDWwTdKW1PZ54OOSFlIechkFPgUQETsk3QM8Q/lMm+t8poyZ2eSqWdwjYiPVx9EfnGCdW4BbWojLzMxa4G+ompllyMXdzCxDLu5mZhlycTczy5CLu5lZhlzczcwy5OJuZpYhF3czswy5uJuZZcjF3cwsQy7uZmYZcnE3M8uQi7uZWYZc3M3MMuTibmaWIRd3M7MMubibmWWonmuozpP0qKSdknZIuj61ny7pYUnPpdvTUrskfU3SiKStks7vdBJmZnaseo7cDwM3RMR7gYuA6ySdC6wCNkTEfGBDmge4jPJFsecDg8DtbY/azMwmVLO4R8TeiHgqTb8O7ATmAEuBdanbOuCKNL0UuCvKHgdmSZrd9sjNzGxcioj6O0t9wGPAecCLETGrYtmBiDhN0gPA6nRhbSRtAG6MiE1j7muQ8pE9vb29i4aHhxsOvlQqsevgkYbXq7RgzsyW1m+XUqlET0/PVIfRMufRXXLJA/LJpZ15DAwMbI6IQrVlx9d7J5J6gO8Dn4mI1ySN27VK29v+g0TEEDAEUCgUor+/v95Q3lQsFlmz8VDD61UaXd74djuhWCzSzGPQbZxHd8klD8gnl8nKo66zZSTNoFzYvxsRP0jN+44Ot6Tb/al9NzCvYvW5wJ72hGtmZvWo52wZAWuBnRHx1YpF64EVaXoFcH9F+zXprJmLgIMRsbeNMZuZWQ31DMtcDFwNbJO0JbV9HlgN3CNpJfAicFVa9iCwBBgBfgZ8oq0Rm5lZTTWLe/pgdLwB9kur9A/guhbjMjOzFvgbqmZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDNX98wOWj75VP2x63dHVl7cxEjPrFB+5m5llyMXdzCxDLu5mZhlycTczy5CLu5lZhlzczcwy5OJuZpYhF3czswy5uJuZZaiey+zdIWm/pO0VbTdLeknSlvS3pGLZTZJGJD0r6SOdCtzMzMZXz5H7ncDiKu23RsTC9PcggKRzgWXA+9I6/yjpuHYFa2Zm9alZ3CPiMeCVOu9vKTAcEW9ExC7K11G9oIX4zMysCSpf8rRGJ6kPeCAizkvzNwPXAq8Bm4AbIuKApK8Dj0fEd1K/tcBDEXFvlfscBAYBent7Fw0PDzccfKlUYtfBIw2vV2nBnJktrd8upVKJnp6eSdnWtpcONr1urcdrMvPoJOfRfXLJpZ15DAwMbI6IQrVlzf4q5O3Al4FIt2uAT1L9QtpV/3tExBAwBFAoFKK/v7/hIIrFIms2Hmp4vUqjyxvfbicUi0WaeQyacW0rvwpZ4/GazDw6yXl0n1xymaw8mjpbJiL2RcSRiPgV8E3eGnrZDcyr6DoX2NNaiGZm1qimiruk2RWzHwOOnkmzHlgm6URJZwHzgSdbC9HMzBpVc1hG0veAfuAMSbuBLwL9khZSHnIZBT4FEBE7JN0DPAMcBq6LiNYGxc3MrGE1i3tEfLxK89oJ+t8C3NJKUGZm1hp/Q9XMLEMu7mZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDLm4m5llyMXdzCxDLu5mZhlycTczy5CLu5lZhlzczcwy5OJuZpYhF3czsww1ew1Ve4fqq3H91RsWHB73Gq2jqy/vREhmVoWP3M3MMlSzuEu6Q9J+Sdsr2k6X9LCk59Ltaaldkr4maUTSVknndzJ4MzOrrp4j9zuBxWPaVgEbImI+sCHNA1xG+aLY84FB4Pb2hGlmZo2o5xqqj0nqG9O8lPJFswHWAUXgxtR+V0QE8LikWZJmR8TedgVsZtZtan0WVWns51Kd+ixK5Tpco1O5uD8QEeel+VcjYlbF8gMRcZqkB4DVEbExtW8AboyITVXuc5Dy0T29vb2LhoeHGw6+VCqx6+CRhtertGDOzJbWb5dSqURPT8+kbGvbSwc7dt+9J8O+n1df1i2PdT0mc390Ui55QHfn0shrauxrpJXXxcDAwOaIKFRb1u6zZVSlrep/j4gYAoYACoVC9Pf3N7yxYrHImo2HGl6v0ujyxrfbCcVikWYeg2aMdzZLO9yw4DBrtlV/WnXLY12PydwfnZRLHtDduTTymhr7GunU66LZs2X2SZoNkG73p/bdwLyKfnOBPc2HZ2ZmzWi2uK8HVqTpFcD9Fe3XpLNmLgIOerzdzGzy1RyWkfQ9yh+eniFpN/BFYDVwj6SVwIvAVan7g8ASYAT4GfCJDsRsZmY11HO2zMfHWXRplb4BXNdqUGZm1hp/Q9XMLEMu7mZmGXJxNzPLkIu7mVmGXNzNzDLk4m5mliEXdzOzDPlKTGbWVo38QuJYvlpX+/jI3cwsQy7uZmYZcnE3M8uQi7uZWYb8gaqZHaOVD0Ste/jI3cwsQy7uZmYZcnE3M8tQS2PukkaB14EjwOGIKEg6Hbgb6ANGgT+MiAOthWlmZo1ox5H7QEQsjIhCml8FbIiI+cCGNG9mZpOoE8MyS4F1aXodcEUHtmFmZhNQ+bKnTa4s7QIOAAH8U0QMSXo1ImZV9DkQEadVWXcQGATo7e1dNDw83PD2S6USuw4eaTp+gAVzZra0fruUSiV6enomZVvbXjrYsfvuPRn2/bz6sm55rOsxmfujk5rJo5PPj1omeo508z5p5DEb+xpp5XUxMDCwuWLU5Bitnud+cUTskfQe4GFJP6l3xYgYAoYACoVC9Pf3N7zxYrHImo2HGl6v0ujyxrfbCcVikWYeg2Zc28HzmG9YcJg126o/rbrlsa7HZO6PTmomj04+P2qZ6DnSzfukkcds7GukU6+LloZlImJPut0P3AdcAOyTNBsg3e5vNUgzM2tM08Vd0imSTj06DXwY2A6sB1akbiuA+1sN0szMGtPKsEwvcJ+ko/fzLxHxb5J+BNwjaSXwInBV62GamVkjmi7uEfEC8P4q7T8FLm0lKDMza41/OGyKjP1xphsWHK77QxlfrcbMavHPD5iZZcjF3cwsQx6WMbOuMdFvydcauvRw5bFc3M3sHS/HC5R4WMbMLEPv+CP3Vv5j+22gmXUrH7mbmWXIxd3MLEMu7mZmGXJxNzPL0Dv+A1WbPP7w2mzyuLjbtOB/DGaNcXGfhnL8woWZtZeLu1kHTdU7jqPbbeTXRi0vLu4t8BH09NDMfqosilM1rOPnl7XCZ8uYmWWoY0fukhYD/wAcB3wrIlZ3altmZn6nc6yOFHdJxwG3Ab8P7AZ+JGl9RDzTie2ZdZKLhk1HnRqWuQAYiYgXIuKXwDCwtEPbMjOzMRQR7b9T6UpgcUT8cZq/GrgwIj5d0WcQGEyzvw0828SmzgBebjHcbpFLLs6ju+SSB+STSzvz+I2IOLPagk6NuatK2zH/RSJiCBhqaSPSpogotHIf3SKXXJxHd8klD8gnl8nKo1PDMruBeRXzc4E9HdqWmZmN0ani/iNgvqSzJJ0ALAPWd2hbZmY2RkeGZSLisKRPA/9O+VTIOyJiRwc21dKwTpfJJRfn0V1yyQPyyWVS8ujIB6pmZja1/A1VM7MMubibmWVo2hZ3SYslPStpRNKqqY6nEZJGJW2TtEXSptR2uqSHJT2Xbk+b6jirkXSHpP2Stle0VY1dZV9L+2irpPOnLvJjjZPHzZJeSvtli6QlFctuSnk8K+kjUxP120maJ+lRSTsl7ZB0fWqfVvtkgjym1T6RdJKkJyU9nfL4Umo/S9ITaX/cnU40QdKJaX4kLe9rWzARMe3+KH9I+zxwNnAC8DRw7lTH1UD8o8AZY9r+BliVplcBX5nqOMeJ/YPA+cD2WrEDS4CHKH/v4SLgiamOv0YeNwOfq9L33PQcOxE4Kz33jpvqHFJss4Hz0/SpwH+leKfVPpkgj2m1T9Lj2pOmZwBPpMf5HmBZav8G8Kdp+s+Ab6TpZcDd7Ypluh655/jzBkuBdWl6HXDFFMYyroh4DHhlTPN4sS8F7oqyx4FZkmZPTqQTGyeP8SwFhiPijYjYBYxQfg5OuYjYGxFPpenXgZ3AHKbZPpkgj/F05T5Jj2spzc5IfwFcAtyb2sfuj6P76V7gUknVvgTasOla3OcA/1Mxv5uJnwjdJoD/kLQ5/QwDQG9E7IXyEx14z5RF17jxYp+O++nTabjijoqhsWmRR3pL/wHKR4vTdp+MyQOm2T6RdJykLcB+4GHK7ypejYjDqUtlrG/mkZYfBN7djjima3Gv+fMGXe7iiDgfuAy4TtIHpzqgDplu++l24DeBhcBeYE1q7/o8JPUA3wc+ExGvTdS1SlvX5FIlj2m3TyLiSEQspPzN/AuA91brlm47lsd0Le7T+ucNImJPut0P3Ef5CbDv6NvjdLt/6iJs2HixT6v9FBH70gvzV8A3eettflfnIWkG5YL43Yj4QWqedvukWh7TdZ8ARMSrQJHymPssSUe/NFoZ65t5pOUzqX+4cELTtbhP2583kHSKpFOPTgMfBrZTjn9F6rYCuH9qImzKeLGvB65JZ2hcBBw8OlTQjcaMPX+M8n6Bch7L0pkNZwHzgScnO75q0vjsWmBnRHy1YtG02ifj5THd9omkMyXNStMnAx+i/PnBo8CVqdvY/XF0P10JPBLp09WWTfWnyy18Kr2E8ifqzwNfmOp4Goj7bMqf8j8N7DgaO+Vxtg3Ac+n29KmOdZz4v0f57fH/UT7qWDle7JTfct6W9tE2oDDV8dfI49spzq3pRTe7ov8XUh7PApdNdfwVcf0e5bfxW4Et6W/JdNsnE+QxrfYJ8DvAj1O824G/SO1nU/7nMwL8K3Biaj8pzY+k5We3Kxb//ICZWYam67CMmZlNwMXdzCxDLu5mZhlycTczy5CLu5lZhlzczcwy5OJuZpah/wcZ0+ZR7emJtwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby('device')['date'].count().hist(bins = 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>failure</th>\n",
       "      <th>attribute1</th>\n",
       "      <th>attribute2</th>\n",
       "      <th>attribute3</th>\n",
       "      <th>attribute4</th>\n",
       "      <th>attribute5</th>\n",
       "      <th>attribute6</th>\n",
       "      <th>attribute7</th>\n",
       "      <th>attribute8</th>\n",
       "      <th>attribute9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>124494.000000</td>\n",
       "      <td>1.244940e+05</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "      <td>124494.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000851</td>\n",
       "      <td>1.223881e+08</td>\n",
       "      <td>159.484762</td>\n",
       "      <td>9.940455</td>\n",
       "      <td>1.741120</td>\n",
       "      <td>14.222669</td>\n",
       "      <td>260172.657726</td>\n",
       "      <td>0.292528</td>\n",
       "      <td>0.292528</td>\n",
       "      <td>12.451524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.029167</td>\n",
       "      <td>7.045933e+07</td>\n",
       "      <td>2179.657730</td>\n",
       "      <td>185.747321</td>\n",
       "      <td>22.908507</td>\n",
       "      <td>15.943028</td>\n",
       "      <td>99151.078547</td>\n",
       "      <td>7.436924</td>\n",
       "      <td>7.436924</td>\n",
       "      <td>191.425623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>6.128476e+07</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>221452.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.227974e+08</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>249799.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.833096e+08</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>310266.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.441405e+08</td>\n",
       "      <td>64968.000000</td>\n",
       "      <td>24929.000000</td>\n",
       "      <td>1666.000000</td>\n",
       "      <td>98.000000</td>\n",
       "      <td>689161.000000</td>\n",
       "      <td>832.000000</td>\n",
       "      <td>832.000000</td>\n",
       "      <td>18701.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             failure    attribute1     attribute2     attribute3  \\\n",
       "count  124494.000000  1.244940e+05  124494.000000  124494.000000   \n",
       "mean        0.000851  1.223881e+08     159.484762       9.940455   \n",
       "std         0.029167  7.045933e+07    2179.657730     185.747321   \n",
       "min         0.000000  0.000000e+00       0.000000       0.000000   \n",
       "25%         0.000000  6.128476e+07       0.000000       0.000000   \n",
       "50%         0.000000  1.227974e+08       0.000000       0.000000   \n",
       "75%         0.000000  1.833096e+08       0.000000       0.000000   \n",
       "max         1.000000  2.441405e+08   64968.000000   24929.000000   \n",
       "\n",
       "          attribute4     attribute5     attribute6     attribute7  \\\n",
       "count  124494.000000  124494.000000  124494.000000  124494.000000   \n",
       "mean        1.741120      14.222669  260172.657726       0.292528   \n",
       "std        22.908507      15.943028   99151.078547       7.436924   \n",
       "min         0.000000       1.000000       8.000000       0.000000   \n",
       "25%         0.000000       8.000000  221452.000000       0.000000   \n",
       "50%         0.000000      10.000000  249799.500000       0.000000   \n",
       "75%         0.000000      12.000000  310266.000000       0.000000   \n",
       "max      1666.000000      98.000000  689161.000000     832.000000   \n",
       "\n",
       "          attribute8     attribute9  \n",
       "count  124494.000000  124494.000000  \n",
       "mean        0.292528      12.451524  \n",
       "std         7.436924     191.425623  \n",
       "min         0.000000       0.000000  \n",
       "25%         0.000000       0.000000  \n",
       "50%         0.000000       0.000000  \n",
       "75%         0.000000       0.000000  \n",
       "max       832.000000   18701.000000  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# nature of attributes: continuous vs. categorical\n",
    "\n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "123877\n",
      "558\n",
      "47\n",
      "115\n",
      "60\n",
      "44838\n",
      "28\n",
      "28\n",
      "65\n"
     ]
    }
   ],
   "source": [
    "# check unique values/levels of the attributes\n",
    "\n",
    "print(df['attribute1'].nunique())\n",
    "print(df['attribute2'].nunique())\n",
    "print(df['attribute3'].nunique())\n",
    "print(df['attribute4'].nunique())\n",
    "print(df['attribute5'].nunique())\n",
    "print(df['attribute6'].nunique())\n",
    "print(df['attribute7'].nunique())\n",
    "print(df['attribute8'].nunique())\n",
    "print(df['attribute9'].nunique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save raw data to file\n",
    "df.to_csv('device_failure.csv', index = None, header=True) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Game plan\n",
    "\n",
    "We envision the assignment as a classification challenge. \n",
    "\n",
    "In designing the solution, we keep the following considerations in mind: \n",
    "\n",
    "- The distribution of the label, failure, is highly skewed. We may need case weighting or data resampling to enable our model with sufficient exposure to both failure and non-failure classes . This is required so model can produce robust parameter estimates. \n",
    "\n",
    "- Devices do not have equal number of log records. We have to reconstruct a dataset fixing granularity at the device level, with proper labelling of failure or non failure at each device's last point of observation. \n",
    "\n",
    "- In term of feature engineering, we can explore patterns along weekday pattern, seasonality, and/or device types. \n",
    "\n",
    "- In terms of attributes, we need to align sensor reading of the previous 1 day, previous 2 days, and previous 3 days to eventual failure incidence. \n",
    "\n",
    "- As for model selection, we use area under the curve (AUC) to guide our model search, balancing precision vs. recall at the same time. Simple accuracy measurement is not effective in the face of imbalanced dataset. Naive prediction of all devices never fail gets us to 91% accuracy.  \n",
    "\n",
    "- As for model evalaution, we manually hold out 20% of data for validation. Our model's performance is measured against the hold-out dataset. \n",
    "\n",
    "- Depending on the natrue of the underlying relationship between features and label, more complex models may be required to pick up non-linearity and/or feature interactions. We'll gradually consider more complex models as result of the simpler models come in. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysis Flow\n",
    "\n",
    "Our analysis will stick to the following steps:  \n",
    "\n",
    "1. clean data\n",
    "2. generate features and construct working dataset \n",
    "3. split dataset into training and validation\n",
    "4. resample minority class in training dataset  \n",
    "5. establish prediction baseline \n",
    "6. fit classification models for comparisons\n",
    "7. fit ensemble models to enhance predictions \n",
    "8. tune ensemble hyperparameters for better performance \n",
    "9. adjust prediction cut-off point\n",
    "10. explore other avenues to make continous improvement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Data Cleaning \n",
    "\n",
    "In this phase of data prepration, we achieve the following objectives: \n",
    "\n",
    "- We identify a duplicate record in the device log, and decide to drop it. \n",
    "\n",
    "- We find attribute8 and attribute9 with perfect correlation, reflecting possibly duplicate entries. We opt to drop attribute9 to avoid perfect collineraity. \n",
    "\n",
    "- We do not find missing data points, so there's no need to impute missing values.  \n",
    "\n",
    "- Features attribute1 to attribute9 do not conform to normal distribution, and varies widely in terms of scale. We have options to either normalize the scale to range between 0 and 1 and avoid potential outlier(s), or standardize the scale constrainting to unit variance of one. The latter will make attributes across the board with the same order of magnitude. Without context of what these attributes mean in practice, it's difficult to interpret outlier. On the other hand, feature variance across attributes do pose challenge to some of the models we plan to fit in the later stage, e.g. KNN, SVM. Hence our plan is to standardize those numerical features in due course. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# delete duplicate rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date          124494\n",
       "device        124494\n",
       "failure       124494\n",
       "attribute1    124494\n",
       "attribute2    124494\n",
       "attribute3    124494\n",
       "attribute4    124494\n",
       "attribute5    124494\n",
       "attribute6    124494\n",
       "attribute7    124494\n",
       "attribute8    124494\n",
       "attribute9    124494\n",
       "dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date          124493\n",
       "device        124493\n",
       "failure       124493\n",
       "attribute1    124493\n",
       "attribute2    124493\n",
       "attribute3    124493\n",
       "attribute4    124493\n",
       "attribute5    124493\n",
       "attribute6    124493\n",
       "attribute7    124493\n",
       "attribute8    124493\n",
       "attribute9    124493\n",
       "dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop_duplicates().count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date          124493\n",
       "device        124493\n",
       "failure       124493\n",
       "attribute1    124493\n",
       "attribute2    124493\n",
       "attribute3    124493\n",
       "attribute4    124493\n",
       "attribute5    124493\n",
       "attribute6    124493\n",
       "attribute7    124493\n",
       "attribute8    124493\n",
       "attribute9    124493\n",
       "dtype: int64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop_duplicates()\n",
    "df.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check column correlation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>failure</th>\n",
       "      <th>attribute1</th>\n",
       "      <th>attribute2</th>\n",
       "      <th>attribute3</th>\n",
       "      <th>attribute4</th>\n",
       "      <th>attribute5</th>\n",
       "      <th>attribute6</th>\n",
       "      <th>attribute7</th>\n",
       "      <th>attribute8</th>\n",
       "      <th>attribute9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>failure</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.001984</td>\n",
       "      <td>0.052902</td>\n",
       "      <td>-0.000948</td>\n",
       "      <td>0.067398</td>\n",
       "      <td>0.002270</td>\n",
       "      <td>-0.000550</td>\n",
       "      <td>0.119055</td>\n",
       "      <td>0.119055</td>\n",
       "      <td>0.001622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute1</th>\n",
       "      <td>0.001984</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.004249</td>\n",
       "      <td>0.003702</td>\n",
       "      <td>0.001836</td>\n",
       "      <td>-0.003373</td>\n",
       "      <td>-0.001518</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.001121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute2</th>\n",
       "      <td>0.052902</td>\n",
       "      <td>-0.004249</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.002617</td>\n",
       "      <td>0.146593</td>\n",
       "      <td>-0.013999</td>\n",
       "      <td>-0.026350</td>\n",
       "      <td>0.141367</td>\n",
       "      <td>0.141367</td>\n",
       "      <td>-0.002736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute3</th>\n",
       "      <td>-0.000948</td>\n",
       "      <td>0.003702</td>\n",
       "      <td>-0.002617</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.097452</td>\n",
       "      <td>-0.006697</td>\n",
       "      <td>0.009027</td>\n",
       "      <td>-0.001884</td>\n",
       "      <td>-0.001884</td>\n",
       "      <td>0.532366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute4</th>\n",
       "      <td>0.067398</td>\n",
       "      <td>0.001836</td>\n",
       "      <td>0.146593</td>\n",
       "      <td>0.097452</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.009773</td>\n",
       "      <td>0.024870</td>\n",
       "      <td>0.045631</td>\n",
       "      <td>0.045631</td>\n",
       "      <td>0.036069</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute5</th>\n",
       "      <td>0.002270</td>\n",
       "      <td>-0.003373</td>\n",
       "      <td>-0.013999</td>\n",
       "      <td>-0.006697</td>\n",
       "      <td>-0.009773</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.017051</td>\n",
       "      <td>-0.009384</td>\n",
       "      <td>-0.009384</td>\n",
       "      <td>0.005949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute6</th>\n",
       "      <td>-0.000550</td>\n",
       "      <td>-0.001518</td>\n",
       "      <td>-0.026350</td>\n",
       "      <td>0.009027</td>\n",
       "      <td>0.024870</td>\n",
       "      <td>-0.017051</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.012207</td>\n",
       "      <td>-0.012207</td>\n",
       "      <td>0.021152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute7</th>\n",
       "      <td>0.119055</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.141367</td>\n",
       "      <td>-0.001884</td>\n",
       "      <td>0.045631</td>\n",
       "      <td>-0.009384</td>\n",
       "      <td>-0.012207</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.006861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute8</th>\n",
       "      <td>0.119055</td>\n",
       "      <td>0.000151</td>\n",
       "      <td>0.141367</td>\n",
       "      <td>-0.001884</td>\n",
       "      <td>0.045631</td>\n",
       "      <td>-0.009384</td>\n",
       "      <td>-0.012207</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.006861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>attribute9</th>\n",
       "      <td>0.001622</td>\n",
       "      <td>0.001121</td>\n",
       "      <td>-0.002736</td>\n",
       "      <td>0.532366</td>\n",
       "      <td>0.036069</td>\n",
       "      <td>0.005949</td>\n",
       "      <td>0.021152</td>\n",
       "      <td>0.006861</td>\n",
       "      <td>0.006861</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             failure  attribute1  attribute2  attribute3  attribute4  \\\n",
       "failure     1.000000    0.001984    0.052902   -0.000948    0.067398   \n",
       "attribute1  0.001984    1.000000   -0.004249    0.003702    0.001836   \n",
       "attribute2  0.052902   -0.004249    1.000000   -0.002617    0.146593   \n",
       "attribute3 -0.000948    0.003702   -0.002617    1.000000    0.097452   \n",
       "attribute4  0.067398    0.001836    0.146593    0.097452    1.000000   \n",
       "attribute5  0.002270   -0.003373   -0.013999   -0.006697   -0.009773   \n",
       "attribute6 -0.000550   -0.001518   -0.026350    0.009027    0.024870   \n",
       "attribute7  0.119055    0.000151    0.141367   -0.001884    0.045631   \n",
       "attribute8  0.119055    0.000151    0.141367   -0.001884    0.045631   \n",
       "attribute9  0.001622    0.001121   -0.002736    0.532366    0.036069   \n",
       "\n",
       "            attribute5  attribute6  attribute7  attribute8  attribute9  \n",
       "failure       0.002270   -0.000550    0.119055    0.119055    0.001622  \n",
       "attribute1   -0.003373   -0.001518    0.000151    0.000151    0.001121  \n",
       "attribute2   -0.013999   -0.026350    0.141367    0.141367   -0.002736  \n",
       "attribute3   -0.006697    0.009027   -0.001884   -0.001884    0.532366  \n",
       "attribute4   -0.009773    0.024870    0.045631    0.045631    0.036069  \n",
       "attribute5    1.000000   -0.017051   -0.009384   -0.009384    0.005949  \n",
       "attribute6   -0.017051    1.000000   -0.012207   -0.012207    0.021152  \n",
       "attribute7   -0.009384   -0.012207    1.000000    1.000000    0.006861  \n",
       "attribute8   -0.009384   -0.012207    1.000000    1.000000    0.006861  \n",
       "attribute9    0.005949    0.021152    0.006861    0.006861    1.000000  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20b8ceb8>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEjCAYAAAA2Uaa4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df3yT9b3//0faxEo9drSVFNF9Hfu4QXEFLEO0k4rttFSpNSk//NX2bNqAUDhlQ1cgg64C6pyyIWwzlEnFeqB8Ca2eIqXgYTubvYW5LwNBRKQr0ipNLbUUGkp+XN8/+JhDbGlCIQlpXvfb7brdcl258n5eed8g777f14+3SlEUBSGEEOL/igj2AQghhLi6SMMghBDCgzQMQgghPEjDIIQQwoM0DEIIITxIwyCEEMKDNAxCCBHiTp8+zZQpU2hqaurx3qFDh9Dr9WRkZLB48WIcDofX8qRhEEKIELZv3z4effRRGhsbe33/mWeeYcmSJdTW1qIoCpWVlV7LlIZBCCGuQqdOnaKpqanHcurUKY/9KisrWbp0KVqttkcZzc3NnD17lrFjxwKg1+vZvn2712z1lfkKoWmW6jsBz1xp+zjgmQCqMLvBXVGpgn0IYoAZdO21l13Gpfzm3Lbq56xevbrH9sLCQubOneteX758+UXLsFqtDBkyxL0+ZMgQWlpavGaHdcMghBBXq/z8fHQ6XY/tMTExPpfhcrlQXfBHkqIoHusXIw2DEEJchWJiYi6pEejN0KFDaW1tda9/+eWXvQ45fZOcYxBCiACJVPm+XAk33XQTUVFR/OMf/wCgurqa1NRUr5+ThkEIIQIkUqXyebkcBQUFfPjhhwD85je/4fnnn2fy5Ml0dXWRl5fn9fOqcH7stpx8Hrjk5LO40q7Eyed5kcN93neV81+Xnddfco5BCCEC5EoNEfmbDCUJIYTwID0GIYQIkMs9dxAo/e4x5Obmul8vXLiQ5ubmHvu0tLRQUFAAQHFxMWaz2efyOzs7mTNnjs/7P/vss5dUvhBCBFqgr0rqr343DHv27HG/tlgs9HYOOyEhgbVr1/ar/I6ODg4dOuR1v5aWFmbNmkVtbW2/coQQQnjy2jA4HA6MRiMzZswgPT2d2bNnYzQaAZg2bRomkwmr1YrBYKC9vZ20tDSKiorIyMhg//79pKWlucvavXs3er2erKwstm3bBoDZbKa4uNi9T25uLhaLhWXLlmG1Wt29hqqqKnQ6HdnZ2SxatIju7m4A3nnnHdLT08nMzLxytSKEEH4QqMtVL5fXhmHv3r1oNBo2bdpEXV0dnZ2dTJw4EYDNmzdjMBjQarWYTCZiY2MBSE1Npba2lri4OI+ybDYblZWVlJWVsWLFCo878r7JaDSi1WpZs2YNR44cobKyko0bN1JdXU18fDzr1q0D4KmnnmLatGn9rgAhhBCevJ58Hj9+PIMHD6aiooKGhgYaGxvp6urq8zNjxozpdbtOp0OtVpOQkMDYsWPZt2+fTwdpsVg4duwY06dPB8ButzNq1CifPiuEEFeLYJ878JXXhmHXrl2sWrWKvLw89Ho97e3tvZ5PuFBUVFSv2yMjI92vXS4XGo0GlUrlUZ7dbu/xOafTSWZmpnsI68yZMzidTm+HLoQQV5VgDxH5yutQUn19PZmZmeTk5BATE4PFYsHpdBIZGemeCSgyMtKnH+qamhoURaG5uZkDBw6QlJREbGwsR48eRVEUjh8/zuHDhwFQq9Xu8idMmEBdXR1tbW0oikJJSQnl5eWX872FEEJchNcew7Rp01iwYAE1NTVoNBqSk5NpamoiPT2d7OxszGYzkyZNwmAwUFZW1mdZ0dHR6PV6HA4HpaWlxMXFkZKSwpYtW5g8eTLDhw9n3LhxAMTHxzNs2DByc3PZsGEDhYWF5Ofn43K5SExMxGAwXJkaEEKIAAmVO4rlWUkBJs9KCgx5VpK40q7Es5JKB93q875LbJ9edl5/yZ3PQggRIAPmHIMQQojwIj0GIYQIkAFzuaoQQogrQ4aShBBChCTpMQghRIDIUFIICMalo/MHjQx4JgTvMlnNl0eDknsu/v8EJdcZhMuCB7U3BjwzmOzxvk+PKfonrBsGIYQIpFA5xyANgxBCBEioDCXJyWchhBAepMcghBABIkNJQgghPMhQkhBCiJAkPQYhhAiQUOkxSMMghBABEirnGPo9lJSbm+t+vXDhQpqbm3vs09LSQkFBAQDFxcWYzWafy+/s7GTOnDle99u5cyfZ2dk89NBDzJ49m46ODp8zhBBC9NTvhmHPnj3u1xaLpdd5oBMSEli7dm2/yu/o6ODQoUN97nP69GlKSkowmUy8/fbbjBgxgldffbVfeUII4W+RKt+XYPLaMDgcDoxGIzNmzCA9PZ3Zs2djNBqB89N+mkwmrFYrBoOB9vZ20tLSKCoqIiMjg/3795OWluYua/fu3ej1erKysti2bRsAZrOZ4uJi9z65ublYLBaWLVuG1Wp19xqqqqrQ6XRkZ2ezaNEiuru7sdvtLF26lISEBABGjBjBF198ceVqRwghwpDXhmHv3r1oNBo2bdpEXV0dnZ2dTJw4EYDNmzdjMBjQarWYTCZiY2MBSE1Npba2lri4OI+ybDYblZWVlJWVsWLFClpbWy+aazQa0Wq1rFmzhiNHjlBZWcnGjRuprq4mPj6edevWERsby3333QfA2bNnMZlM/PjHP+53ZQghhD9FqlQ+L8Hk9eTz+PHjGTx4MBUVFTQ0NNDY2EhXV1efnxkzZkyv23U6HWq1moSEBMaOHcu+fft8OkiLxcKxY8eYPn06AHa7nVGjRrnf//p8xMiRI9HpdD6VKYQQgRbsISJfeW0Ydu3axapVq8jLy0Ov19Pe3t7r+YQLRUVF9bo9MjLS/drlcqHRaFCpVB7l2e32Hp9zOp1kZma6h7DOnDmD0+kEwGq18uSTT3LnnXeyaNEib19HCCGEF16Hkurr68nMzCQnJ4eYmBgsFgtOp5PIyEgcDgdw/gf/6x/qvtTU1KAoCs3NzRw4cICkpCRiY2M5evQoiqJw/PhxDh8+DIBarXaXP2HCBOrq6mhra0NRFEpKSigvL8fpdDJr1iwyMzNZvHgxqhC5FEwIEZ4GzFDStGnTWLBgATU1NWg0GpKTk2lqaiI9PZ3s7GzMZjOTJk3CYDBQVlbWZ1nR0dHo9XocDgelpaXExcWRkpLCli1bmDx5MsOHD2fcuHEAxMfHM2zYMHJzc9mwYQOFhYXk5+fjcrlITEzEYDDw3nvv8dFHH+F0OqmtrQXgBz/4AcuXL78CVSOEEKHhnXfe4Q9/+AMOh4P8/Hwef/xxj/cPHjzIkiVLsNvt3Hjjjbz00kvExMRctDyV4m1caACznT0b8EyZqCcwZKKegStYE/UMuvbayy5j69DbfN5Xd+KgT/u1tLTw6KOPYjabueaaa3jkkUd45ZVXuPXWW937PPbYY8ycOZN77rmHF154gaioKObPn3/RMuXOZyGECJBLGSI6deoUp06d6rE9JibG46/9999/nzvvvJPBgwcDkJGRwfbt2yksLHTv43K5OHPmDHD+6tBvfetbfWZLwyCEEFeh8vJyVq9e3WN7YWEhc+fOda9brVaGDBniXtdqtezfv9/jM8XFxfz0pz9lxYoVDBo0iMrKyj6zpWEQQogAibiEHkN+fn6vl99/89yAy+XyuPBGURSP9bNnz7J48WLWr1/P6NGjef311/nFL36ByWS6aLY0DEIIcRX65pDRxQwdOpQPPvjAvd7a2opWq3Wvf/LJJ0RFRTF69GgAZsyYwe9+97s+y5T5GIQQIkBUkSqfF1+lpKRQX1/PyZMnsdls7Nixg9TUVPf7t9xyCydOnKChoQE4f29aUlJSn2VKj0EIIUJYQkIC8+fPJy8vD7vdztSpUxk9ejQFBQXMmzePpKQknn/+eYqKilAUhfj4eFasWNFnmWF9uepZmy3gmUqQblwJ1mWyr7bsDkqu/fqhQcmNIPD/nYL1bypYWpbODErud14sv+wy3v1O748L6k1mo2+PDPIH6TEIIUSAqCJDY/Q+NI5SCCFEwEiPQQghAuRSTioHk/QYhBBCeJAegxBCBEhEiPQYpGEQQogAUUWExiBNaBylEEKIgJEegxBCBIgMJQkhhPAw4K9Kys3Ndb9euHAhzc3NPfZpaWmhoKAAOP/YV7PZ7HP5nZ2dzJkzx+t+dXV1ZGVl8eCDD1JcXMy5c+d8zhBCCNFTvxuGPXv2uF9bLBZ6e7JGQkICa9eu7Vf5HR0dHDp0qM99urq6KC0t5fXXX6empobu7m62bt3arzwhhPA3VWSEz0sweU13OBwYjUZmzJhBeno6s2fPxmg0AufngzaZTFitVgwGA+3t7aSlpVFUVERGRgb79+8nLS3NXdbu3bvR6/VkZWWxbds2AMxmM8XFxe59cnNzsVgsLFu2DKvV6u41VFVVodPpyM7OZtGiRXR3dxMdHc17773HDTfcgM1mo62tzafH1AohRDBERKp8XoJ6nN522Lt3LxqNhk2bNlFXV0dnZycTJ04EYPPmzRgMBrRaLSaTidjYWABSU1Opra0lLi7OoyybzUZlZSVlZWWsWLGC1tbWi+YajUa0Wi1r1qzhyJEjVFZWsnHjRqqrq4mPj2fdunUAaDQa/vznPzNp0iTa29u5++67+10ZQgghfDj5PH78eAYPHkxFRQUNDQ00NjbS1dXV52fGjOn9CYI6nQ61Wk1CQgJjx45l3z7fnh5osVg4duwY06dPB8ButzNq1Cj3+/fccw8Wi4VXXnmFkpISXn75ZZ/KFUKIQFJFhMbJZ68Nw65du1i1ahV5eXno9Xra29t7PZ9woaioqF63R0ZGul+7XC40Gg0qlcqjPLvd3uNzTqeTzMxM9xDWmTNncDqdfPXVVxw4cMDdS8jKymL+/PnevpIQQog+eB1Kqq+vJzMzk5ycHGJiYrBYLDidTiIjI3E4HMD5H3yn0+k1rKamBkVRaG5u5sCBAyQlJREbG8vRo0dRFIXjx49z+PBhANRqtbv8CRMmUFdXR1tbG4qiUFJSQnl5OYqi8Mwzz/D5558DsH37dpKTk/tdGUII4U8RkRE+L8Hktccwbdo0FixYQE1NDRqNhuTkZJqamkhPTyc7Oxuz2cykSZMwGAyUlZX1WVZ0dDR6vR6Hw0FpaSlxcXGkpKSwZcsWJk+ezPDhwxk3bhwA8fHxDBs2jNzcXDZs2EBhYSH5+fm4XC4SExMxGAxERUXx3HPPMXPmTFQqFbfeeiu/+tWvrkzNCCHEFRYq9zHIDG4BJjO4BYbM4DZwhfIMbu/fk+p9p/8r5c9/uey8/pI7n4UQIkBCpccgD9ETQgjhQXoMQggRIME+qeyr0DhKIYQQASM9BiGECJBQOccgDUOAab48GpTcYF0dNDdhUlByX+n6OCi5Kpcj8JnOnjeFDmQ3Gn8X7EPot4gQufNZhpKEEEJ4kB6DEEIESLAfp+2r0DhKIYQQASM9BiGECJBgz7PgK2kYhBAiQELlqiQZShJCCOFBegxCCBEgcvJZCCFESJIegxBCBEionHzud48hNzfX/XrhwoU0Nzf32KelpYWCggIAiouLMZvNPpff2dnJnDlzfN5/9+7dpKWl+by/EEIEmipC5fNyKd555x0eeOAB7r//fioqKnq839DQQG5uLg899BBPPvkkHR0dfZbX74Zhz5497tcWi6XXeaATEhJYu3Ztv8rv6Ojg0KFDPu375Zdf8uKLL/YrRwghQllLSwsrV67krbfeoqqqik2bNvHpp5+631cUhaeffpqCggLefvttEhMTMZlMfZbptWFwOBwYjUZmzJhBeno6s2fPxmg0Auen/TSZTFitVgwGA+3t7aSlpVFUVERGRgb79+/3+Ct+9+7d6PV6srKy2LZtGwBms5ni4mL3Prm5uVgsFpYtW4bVanX3GqqqqtDpdGRnZ7No0SK6u7vdnzEajRQWFvpSh0IIETSXMufzqVOnaGpq6rGcOnXKo8z333+fO++8k8GDBxMdHU1GRgbbt293v3/w4EGio6NJTT0/e9ysWbN4/PHH+z5Ob19k7969aDQaNm3aRF1dHZ2dnUycOBGAzZs3YzAY0Gq1mEwmYmNjAUhNTaW2tpa4uDiPsmw2G5WVlZSVlbFixQpaW1svmms0GtFqtaxZs4YjR45QWVnJxo0bqa6uJj4+nnXr1gHwxhtvMGrUKMaMGePtqwghRFCpIlU+L+Xl5aSnp/dYyss9pxi1Wq0MGTLEva7VamlpaXGvf/bZZ9xwww0sWrQInU7H0qVLiY6O7vM4vZ58Hj9+PIMHD6aiooKGhgYaGxvp6urq8zMX+5HW6XSo1WoSEhIYO3Ys+/bt8xYPnB+qOnbsGNOnTwfAbrczatQoPvnkE3bs2MH69es5ceKET2UJIUQoyM/PR6fT9dgeExPjse5yuVBdMO+3oige6w6Hgz179vDmm2+SlJTEb3/7W1544QVeeOGFi2Z7bRh27drFqlWryMvLQ6/X097e3uv5hAtFRUX1uj0yMtLjy2g0GlQqlUd5dnvPRwg7nU4yMzPdQ1hnzpzB6XSyfv16WltbycnJwW63Y7Vaeeyxx3jrrbe8fS0hhAi4S7mPISYmpkcj0JuhQ4fywQcfuNdbW1vRarXu9SFDhnDLLbeQlJQEwJQpU5g3b16fZXo9yvr6ejIzM8nJySEmJgaLxYLT6SQyMhKH4/yz5yMjI3E6nV6/QE1NDYqi0NzczIEDB0hKSiI2NpajR4+iKArHjx/n8OHDAKjVanf5EyZMoK6ujra2NhRFoaSkhPLycubNm0dtbS3V1dWYTCa0Wq00CkKIq5YqIsLnxVcpKSnU19dz8uRJbDYbO3bscJ9PALj99ts5efIkH398fo6S9957j9tuu63PMr32GKZNm8aCBQuoqalBo9GQnJxMU1MT6enpZGdnYzabmTRpEgaDgbKysj7Lio6ORq/X43A4KC0tJS4ujpSUFLZs2cLkyZMZPnw448aNAyA+Pp5hw4aRm5vLhg0bKCwsJD8/H5fLRWJiIgaDwWuFCSHEQJeQkMD8+fPJy8vDbrczdepURo8eTUFBAfPmzSMpKYk1a9ZgNBqx2WwMHTqUX//6132WqVK8jQsNYGdttoBnqtsaAp4JoFxzXVByw20Gt0hXEGZTC7MZ3FAF54ENUdcPvuwyjjw91ed9v/eH//ey8/pLHokhhBDCgzwSQwghAiRUHqInDYMQQgRIqDQMoXGUQgghAkZ6DEIIESCXchlqMIXGUQohhAiYsO4xKKrAPxv9XPz/CXhmMAXrstGfRY8MSu5vzvj2ROArKUKjCXhmMEUQulfYqy54+sPVTHoMQgghPIR1j0EIIQIpVK5KkoZBCCECJEJOPgshhAhF0mMQQogACZWhpNA4SiGEEAEjPQYhhAiQUOkxSMMghBABMuDvfM7NzXW/XrhwIc3NzT32aWlpoaCgAIDi4mLMZrPP5Xd2djJnzhyv+61evZp7772X7OxssrOzqaio8DlDCCFET/3uMezZs8f92mKx9PojnpCQwNq1a/tVfkdHB4cOeb+L9MCBA7zyyivcfvvt/coRQohACZWhJK9H6XA4MBqNzJgxg/T0dGbPno3RaATOT/tpMpmwWq0YDAba29tJS0ujqKiIjIwM9u/fT1pamrus3bt3o9frycrKYtu2bQCYzWaKi4vd++Tm5mKxWFi2bBlWq9Xd4FRVVaHT6cjOzmbRokV0d3cD5xuG1157jaysLEpLS93bhRBC9I/XhmHv3r1oNBo2bdpEXV0dnZ2dTJw4EYDNmzdjMBjQarWYTCZiY2MBSE1Npba2lri4OI+ybDYblZWVlJWVsWLFClpbWy+aazQa0Wq1rFmzhiNHjlBZWcnGjRuprq4mPj6edevWcebMGRITE3nmmWfYunUrp06d4ve///3l1IcQQviNKjLC5yWYvA4ljR8/nsGDB1NRUUFDQwONjY10dXX1+ZkxY8b0ul2n06FWq0lISGDs2LHs27fPp4O0WCwcO3aM6dOnA2C32xk1ahTXXXedx1DVT3/6UxYtWsT8+fN9KlcIIQIpIkSGkrw2DLt27WLVqlXk5eWh1+tpb29HUfp+umFUVFSv2yMveLKgy+VCo9GgUqk8yrPbe05s7nQ6yczMdA9hnTlzBqfTyeeff87777/P1KnnJ9hWFAW1Wi60EkKIy+G1+aqvryczM5OcnBxiYmKwWCw4nU4iIyNxOBzA+R98p9PpNaympgZFUWhububAgQMkJSURGxvL0aNHURSF48ePc/jwYQDUarW7/AkTJlBXV0dbWxuKolBSUkJ5eTnXXnstL730EsePH0dRFCoqKrjvvvsupz6EEMJvVBERPi/B5PXP62nTprFgwQJqamrQaDQkJyfT1NREeno62dnZmM1mJk2ahMFgoKysrM+yoqOj0ev1OBwOSktLiYuLIyUlhS1btjB58mSGDx/OuHHjAIiPj2fYsGHk5uayYcMGCgsLyc/Px+VykZiYiMFgICoqitLSUp5++mnsdjvJycn85Cc/uTI1I4QQV1iwzx34SqV4GxcawGxnzwY8M3xrO7DCaqKeIEw4FUzBmqjn2kGDLruM1pW+n/8cMn/lZef1lwzICyFEgIRKj0EaBiGECJBgnzvwVWgcpRBCiICRHoMQQgRIxAWX7F/NpMcghBDCg/QYhBAiQOTks+iVM0jXq2qCdEWjyuUISm4wLhsFWHBdYsAzX+n6OOCZwRSsf1NXQqg0DKFxlEIIIQJGegxCCBEgcrmqEEKIgHjnnXd44IEHuP/++/ucxXL37t0ec+RcjPQYhBAiQPxxjqGlpYWVK1diNpu55ppreOSRR5gwYQK33nqrx35ffvklL774ok9lSo9BCCGuQqdOnaKpqanHcurUKY/93n//fe68804GDx5MdHQ0GRkZbN++vUd5RqORwsJCn7KlxyCEEAFyKT2G9eXlrF69usf2wsJC5s6d6163Wq0MGTLEva7Vatm/f7/HZ9544w1GjRp10UnUvkkaBiGECJBLOfmcn5+PTqfrsT0mJsZj3eVyobrgCbuKonisf/LJJ+zYsYP169dz4sQJn7KlYRBCiKtQTExMj0agN0OHDuWDDz5wr7e2tqLVat3r27dvp7W1lZycHOx2O1arlccee4y33nrromX2+xxDbm6u+/XChQtpbm7usU9LSwsFBQUAFBcXYzabfS6/s7OTOXPmeN2voaGB3NxcHnroIZ588kk6Ojp8zhBCiEBSRUT6vPgqJSWF+vp6Tp48ic1mY8eOHaSmprrfnzdvHrW1tVRXV2MymdBqtX02CnAZDcOePXvcry0WS6/zQCckJLB27dp+ld/R0cGhQ33fvaooCk8//TQFBQW8/fbbJCYmYjKZ+pUnhBChKCEhgfnz55OXl8fDDz/MlClTGD16NAUFBXz44Yf9KtPrUJLD4aCkpIQjR47w5ZdfMmLECOLi4oDz037ed999WK1WDAYDFRUV5OTkMHr0aA4dOsRLL71EUVER7733HnD+Gto333wTu93O008/zQMPPIDZbGbPnj288MILwPmeSGFhIa+//jpWq5U5c+awZs0aqqqqKC8vx+Vycdttt7F06VKOHDlCdHS0u3WcNWtWjzP2Qghx1biEnsClyMrKIisry2Nbb3+U33zzze7f47547THs3bsXjUbDpk2bqKuro7Ozk4kTJwKwefNmDAYDWq0Wk8lEbGwsAKmpqdTW1robkK/ZbDYqKyspKytjxYoVtLa2XjTXaDSi1WpZs2YNR44cobKyko0bN1JdXU18fDzr1q3js88+44YbbmDRokXodDqWLl1KdHS01y8thBBBERHh+xJEXnsM48ePZ/DgwVRUVNDQ0EBjYyNdXV19fuZil0TpdDrUajUJCQmMHTuWffv2+XSQFouFY8eOMX36dADsdjujRo3i5ptvZs+ePbz55pskJSXx29/+lhdeeMHd+xBCCHHpvDYMu3btYtWqVeTl5aHX62lvb+/1fMKFoqKiet0eecEkFS6XC41Gg0ql8ijPbrf3+JzT6SQzMxOj0QjAmTNncDqdHDx4kFtuuYWkpCQApkyZwrx587x9JSGECArVQJmop76+nszMTHJycoiJicFiseB0OomMjMThOP/428jISJxOp9ewmpoaFEWhubmZAwcOkJSURGxsLEePHkVRFI4fP87hw4cBUKvV7vInTJhAXV0dbW1tKIpCSUkJ5eXl3H777Zw8eZKPPz7/2OH33nuP2267rd+VIYQQwocew7Rp01iwYAE1NTVoNBqSk5NpamoiPT2d7OxszGYzkyZNwmAwUFZW1mdZ0dHR6PV6HA4HpaWlxMXFkZKSwpYtW5g8eTLDhw9n3LhxAMTHxzNs2DByc3PZsGEDhYWF5Ofn43K5SExMxGAwEBUVxZo1azAajdhsNoYOHcqvf/3rK1MzQghxpfnp5POVplK8jQsNYLazZwOe6XDJRD2BYFcF595NmajH/yJdPYebAyHquusvuwzbf63xed9BU7zfx+Uv8hA9IYQQHuSRGEIIESChMlGPNAxCCBEoIXKOITSaLyGEEAEjPQYhhAiUEOkxSMMghBABIucYRK8GtTcGJdcePzwouSpncC4tjNBogpIbjEtHfxY9MuCZwfTbrr6fuiwunzQMQggRKCEylBQa/RohhBABIz0GIYQIlBDpMUjDIIQQATJgnq4qhBAivEiPQQghAiVELlcNjaMUQggRMNJjEEKIQAmRk8/97jHk5ua6Xy9cuJDm5uYe+7S0tFBQUABAcXExZrPZ5/I7OzuZM6fv55EfOnSI7Oxs9zJx4kSmTJnic4YQQoie+t1j2LNnj/u1xWLp9Uc8ISGBtWvX9qv8jo4ODh3q+w7HxMREqqurAbDZbEybNo2SkpJ+5QkhhL+pBkqPweFwYDQamTFjBunp6cyePRuj0Qicn/bTZDJhtVoxGAy0t7eTlpZGUVERGRkZ7N+/n7S0NHdZu3fvRq/Xk5WVxbZt2wAwm80UFxe798nNzcVisbBs2TKsVqu7wamqqkKn05Gdnc2iRYvo7u72OM7XXnuN8ePH88Mf/vDya0UIIfwhIsL3JZiH6W2HvXv3oqsye+8AABrzSURBVNFo2LRpE3V1dXR2djJx4kQANm/ejMFgQKvVYjKZiI2NBSA1NZXa2lri4uI8yrLZbFRWVlJWVsaKFStobW29aK7RaESr1bJmzRqOHDlCZWUlGzdupLq6mvj4eNatW+fet7Ozk8rKSgoLC/tVCUIIIf6X16Gk8ePHM3jwYCoqKmhoaKCxsZGurq4+PzNmzJhet+t0OtRqNQkJCYwdO5Z9+/b5dJAWi4Vjx44xffp0AOx2O6NGjXK///bbb/PjH/+Y+Ph4n8oTQohgCJWhJK8Nw65du1i1ahV5eXno9Xra29tRlL4ntI+Kiup1e+QFd/25XC40Gg0qlcqjPLu959M4nU4nmZmZ7iGsM2fO4HQ63e/v3LmTmTNnevsqQgghfOB1KKm+vp7MzExycnKIiYnBYrHgdDqJjIzE4XAA53/wL/yhvpiamhoURaG5uZkDBw6QlJREbGwsR48eRVEUjh8/zuHDhwFQq9Xu8idMmEBdXR1tbW0oikJJSQnl5eUAKIrCwYMHuf322/tdCUIIERARkb4vQeS1xzBt2jQWLFhATU0NGo2G5ORkmpqaSE9PJzs7G7PZzKRJkzAYDJSVlfVZVnR0NHq9HofDQWlpKXFxcaSkpLBlyxYmT57M8OHDGTduHADx8fEMGzaM3NxcNmzYQGFhIfn5+bhcLhITEzEYDACcPHkSjUZz0V6KEEJcNULkzmeV4m1caACznT0b8ExN278CngnBm6gn4lzf56P8xamJDkpuMMhEPYFx7aBBl12GY98On/dVj7n/svP6S+58FkKIAJGnqwohhAhJ0mMQQohAGSiXqwohhLhCQqRhkKEkIYQIce+88w4PPPAA999/PxUVFT3e37lzJ9nZ2Tz00EPMnj2bjo6OPsuThkEIIQJEFRHh8+KrlpYWVq5cyVtvvUVVVRWbNm3i008/db9/+vRpSkpKMJlMvP3224wYMYJXX321zzKlYRBCiEC5hBvcTp06RVNTU4/l1KlTHkW+//773HnnnQwePJjo6GgyMjLYvn27+3273c7SpUtJSEgAYMSIEXzxxRd9HqacYxBCiKtQeXk5q1ev7rG9sLCQuXPnutetVitDhgxxr2u1Wvbv3+9ej42N5b777gPg7NmzmEwmj/l0eiMNgxBCBIrK90Ga/Px8dDpdj+0xMTEe6y6XC5VK5V5XFMVj/WtfT342cuTIXsu9kDQMQggRKJfQMMTExPRoBHozdOhQPvjgA/d6a2srWq3WYx+r1cqTTz7JnXfeyaJFi7yWKecYhBAihKWkpFBfX8/Jkyex2Wzs2LGD1NRU9/tOp5NZs2aRmZnJ4sWLe+1NfJP0GIQQIkCUS+gx+CohIYH58+eTl5eH3W5n6tSpjB49moKCAubNm8eJEyf46KOPcDqd1NbWAvCDH/yA5cuXX7RMeYhegMlD9AJDHqI3cIXyQ/Sc//r/fN43cnjyZef1l/QYhBAiUPzQY/CH0DhKIYQQAdPvhuHC62AXLlxIc3Nzj31aWlooKCgAoLi4GLPZ7HP5X19a5c3BgwfJycnhoYceYubMmT1u/hBCiKuGSuX7EkT9bhj27Nnjfm2xWHqdBzohIYG1a9f2q/yOjg4OHfI+lrh8+XLmzZvH22+/zfDhw1m3bl2/8oQQwu8iInxfgnmY3nZwOBwYjUZmzJhBeno6s2fPxmg0Auen/TSZTFitVgwGA+3t7aSlpVFUVERGRgb79+8nLS3NXdbu3bvR6/VkZWWxbds2AMxmM8XFxe59cnNzsVgsLFu2DKvV6u41VFVVodPpyM7OZtGiRXR3dwPnb+44c+YMADabjWuvvfYKVY0QQoQnrw3D3r170Wg0bNq0ibq6Ojo7O5k4cSIAmzdvxmAwoNVqMZlMxMbGApCamkptbS1xcXEeZdlsNiorKykrK2PFihW0trZeNNdoNKLValmzZg1HjhyhsrKSjRs3Ul1dTXx8vLtnUFxcjNFo5O677+b999/nkUce6XdlCCGEPymqCJ+XYPJ6VdL48eMZPHgwFRUVNDQ00NjYSFdX35cgjhkzptftOp0OtVpNQkICY8eOZd++fT4dpMVi4dixY0yfPh04/1CoUaNGcfbsWRYvXsz69esZPXo0r7/+Or/4xS8wmUw+lSuEEKInrw3Drl27WLVqFXl5eej1etrb23s9n3ChqKioXrdHXjDfqcvlQqPRoFKpPMqz2+09Pud0OsnMzHQPYZ05cwan08knn3xCVFQUo0ePBmDGjBn87ne/8/aVhBAiOAbK5ar19fVkZmaSk5NDTEwMFosFp9NJZGQkDocDOP+D73Q6vYbV1NSgKArNzc0cOHCApKQkYmNjOXr0KIqicPz4cQ4fPgyAWq12lz9hwgTq6upoa2tDURRKSkooLy/nlltu4cSJEzQ0NADnG7GkpKR+V4YQQviVKsL3JYi89himTZvGggULqKmpQaPRkJycTFNTE+np6WRnZ2M2m5k0aRIGg4GysrI+y4qOjkav1+NwOCgtLSUuLo6UlBS2bNnC5MmTGT58OOPGjQMgPj6eYcOGkZuby4YNGygsLCQ/Px+Xy0ViYiIGg4GoqCief/55ioqKUBSF+Ph4VqxYcWVqRgghwpQ8EiPA5JEYgSGPxBi4QvmRGI4vjvi8r/rG7112Xn+FxoCXEEKIgJFnJQkhRIAE+zJUX0nDIIQQgRIiDUNoHKUQQoiAkR6DEEIESpAfjucr6TEIIYTwIJerhomWpTODknujMTh3oivq3u++9zeVyxHwTCUivDr+RdGJQcn9o9J42WXYWz/zeV/NkP/nsvP6S3oMQgghPITXnxpCCBFEcrmqEEIIT0GegMdX0jAIIUSghEiPITSOUgghRMBIj0EIIQIlRHoM0jAIIUSghEjDEBpHKYQQImCkxyCEEAESKper9vsoc3Nz3a8XLlxIc3Nzj31aWlooKCgAoLi4GLPZ7HP5nZ2dzJkzx+t+f/7zn8nKyiIrK4uf//znnDlzxucMIYQQPfW7YdizZ4/7tcViobcnayQkJLB27dp+ld/R0cGhQ33P1HTq1CmKi4tZuXIl77zzDiNHjmTlypX9yhNCCL8LkTmfvaY7HA6MRiMzZswgPT2d2bNnYzQagfPzQZtMJqxWKwaDgfb2dtLS0igqKiIjI4P9+/eTlpbmLmv37t3o9XqysrLYtm0bAGazmeLiYvc+ubm5WCwWli1bhtVqdfcaqqqq0Ol0ZGdns2jRIrq7u2lsbGTYsGHceuutANx7773s3LnzytWOEEKEIa8Nw969e9FoNGzatIm6ujo6OzuZOHEiAJs3b8ZgMKDVajGZTMTGxgKQmppKbW0tcXFxHmXZbDYqKyspKytjxYoVtLa2XjTXaDSi1WpZs2YNR44cobKyko0bN1JdXU18fDzr1q3jO9/5DidOnODjjz8G4N133+XLL7/sd2UIIYRfqVS+L0Hk9eTz+PHjGTx4MBUVFTQ0NNDY2EhXV98TvI8ZM6bX7TqdDrVaTUJCAmPHjmXfvn0+HaTFYuHYsWNMnz4dALvdzqhRo4iJieHFF1/kl7/8JS6Xi+nTp6PRaHwqUwghAi5ETj57bRh27drFqlWryMvLQ6/X097e3uv5hAtFRfX+yOPIyEj3a5fLhUajQaVSeZRnt9t7fM7pdJKZmekewjpz5gxOpxOn08nQoUPZvHkzAPv37+fb3/62t68khBCiD16br/r6ejIzM8nJySEmJgaLxYLT6SQyMhKH4/yz5yMjI3E6nV7DampqUBSF5uZmDhw4QFJSErGxsRw9ehRFUTh+/DiHDx8GQK1Wu8ufMGECdXV1tLW1oSgKJSUllJeXo1Kp+OlPf0pLSwuKorB+/XoeeOCBy6kPIYTwG0UV4fNyKd555x0eeOAB7r//fioqKnq8f+jQIfR6PRkZGSxevNj923oxXtOnTZtGTU0NWVlZ/Md//AfJyck0NTWRnp5OdnY23d3dTJo0CYPBwPHjx/ssKzo6Gr1ez6xZsygtLSUuLo6UlBRuvPFGJk+ezPLlyxk3bhwA8fHxDBs2jNzcXEaOHElhYSH5+fk8+OCDuFwuDAYDERERlJaW8tRTTzF58mRiYmJ48sknvX0lIYQYMFpaWli5ciVvvfUWVVVVbNq0iU8//dRjn2eeeYYlS5ZQW1uLoihUVlb2WabM4BYmZAa3wJAZ3PwvlGdwO2uz+bzvObudU6dO9dgeExNDTEyMe33r1q38/e9/Z8WKFQCsWbMGRVEoLCwEoLm5mfz8fPcVmx988AGrVq3ijTfeuGh2eP2LEkKIIFIu4Wqj8vJyVq9e3WN7YWEhc+fOda9brVaGDBniXtdqtezfv/+i7w8ZMoSWlpY+s6VhEEKIq1B+fj46na7H9gt7C3D+Qh7VBQ2Ooige697e7400DEIIESCXMnD/zSGjixk6dCgffPCBe721tRWtVuvx/oX3jH355Zce7/cmNC6qFUII0auUlBTq6+s5efIkNpuNHTt2kJqa6n7/pptuIioqin/84x8AVFdXe7zfG2kYhBAiQFyK4vPiq4SEBObPn09eXh4PP/wwU6ZMYfTo0RQUFPDhhx8C8Jvf/Ibnn3+eyZMn09XVRV5eXp9lylVJYUKuSgoMuSrJ/0L5qqTOLt+vSro+etBl5/VXWDcMQgghepKhJCGEEB6kYRBCCOFBGgYhhBAepGEQQgjhQRoGIYQQHqRhEEII4UEaBiGEEB6kYRBCCOFBGgYhhBAepGEQQgjhQRoGIYQQHqRhEGHrrbfeAuDcuXP89re/Ra/XM336dEwmk9fJ0oUYyKRhuEp1dnayfPly5s6dS3V1tcd7v/zlL/2W63A42Lx5M7W1tZw+fZqf//znZGVlUVJSwunTp/2W25snnnjCr+Vv3rwZgBdffJHm5maWL19OSUkJx44dY+nSpX7L7ezs5OWXX+bEiROcOnWKhQsXMmXKFH7xi19w8uRJv+VmZWWxd+9ev5V/MadPn+Z3v/sdr732Gh0dHcycOZPbb7+dvLw8mpub/ZZrs9lYtmwZ9957L0lJSSQnJ/Pwww9jMplwOp1+yx0I5OmqXlRVVfX5/sMPP+yX3MLCQr7//e8zYsQITCYTo0aN4rnnngNAp9OxdetWv+QuXLiQ7u5u2tra+Oqrr5g0aRJZWVls376df/3rX7z88st+yU1PT++xraWlhYSEBAB27dp1xTO/rseHHnqIqqoqIiLO/52kKAqZmZls3779imcCPPXUU4waNQqDwcCvfvUrbr75ZqZMmcKuXbvYs2cPZWVlfslNTU3lW9/6FnfccQdz5swhLi7OLznfNGvWLIYPH47NZqO+vp5HH32UGTNmsHPnTrZu3cqf/vQnv+Q+88wzjBgxgnvvvZdt27YxdOhQRo4cyfr16/nWt77FkiVL/JI7ICiiT88++6wyduxYpbi4uNfFX7Kzs92vbTabkpeXpzz//PM93rvSpkyZoiiKonR1dSl33HHHRY/pSvvv//5vJTMzU6mpqVGampqU48ePKw888IDS1NSkNDU1+SXz3nvvVf75z38qs2bNUv71r3+5tzc1NbnrwR8eeugh92udTufxnj9zH374YcVmsykrV65UUlJSlCVLligWi0Xp7u72W6aieP67ufvuuz3eu7AurrRvlj116lRFURTF5XIpGRkZfssdCMJrho9+ePHFF+no6GDcuHFMnTo1oNmtra0MGTKEa6+9ltWrV/P444/zxz/+0etE3pdDpVJx8uRJ4uLieOmll9zbT5w4gcvl8lvupEmTSEpKYvHixTQ0NDB79myuueYabrrpJr9l6vV6/vCHP3DgwAGef/55XnvtNbZs2cJLL73k7p35Q0xMDH/729/40Y9+RGJiIh999BGjRo3i8OHDXHvttX7LBbj22mspKiriqaeeorq6mj/+8Y8cPHgQjUbDX//6V79kqtVq/vrXv9LZ2UlXVxcHDhzgBz/4AY2NjX7J+5qiKDQ0NPDd736Xw4cPu//ftLa2otFo/Jod8oLdMoWClpYWpaysLKCZdXV1yt13363U1dV5HIdOp1MSExP9lrtjxw4lNTVVcTgc7m1//etflbvuukvZuXOn33Iv9MYbbyhPPPFEQP+qO3PmjKIoivL5558rbW1tfs06evSokpGRoeh0OqWgoEAZM2aMkp2drUycOFHZt2+f33L76vH58zsfPHhQeeyxx5RHHnlE+ec//6k8+OCDSk5OjvKjH/1I+ctf/uK33N27dyt33XWXMnXqVOVHP/qR8re//U359NNPldTUVGX37t1+yx0I5BzDVez06dM4HA4GDx7s3uZyuXjvvff48Y9/7Ldcm83GoEH/O61gR0cHiqJ4HIe/HTlyhO3btzN37ly/Z507d44//elPNDQ0sGTJEtavX4/BYOCaa67xa+7Bgwc5duwYTqeTG264geTkZKKi/Dcl6ccff8zIkSP9Vr6vuru7+eSTT7jllluIiYnxa1ZnZyeNjY185zvf4frrr8flcqFSqfza6x4I5KokH507d44//OEPPPvss5w+fZrVq1dz7tw5v2b+27/9G9HR0R65v//970lNTfVr7qBBgzy+b2RkJG+++abfvy/8bz2vXbuWf//3fw9IPZeWltLV1cVHH31EZGQkn332GYsWLfJrJsD3vvc9jh07xv/8z/+QlJTE2rVr/fpdv24UgvFv+cLcJUuWMHz4cN544w2/515//fWMGDGCiooKnn32Wbq6ulizZk1Avm8ok4bBR6WlpdhstoD/ePSWu3jx4qDkBvr7qtXqgOQePHiQn/3sZ6jVagYNGsSLL77Ixx9/7NdMuDrqOBi5Bw8eDHhuMBr+kBbkoayQ8fDDDyuK8r/jtC6XS3nwwQcldwDk6nQ6pbu7253d1tbm16uDvhZOdRyOuaFMrkrykUql4ty5c+6xyfb29oCMU0qu/3Pz8vL4yU9+QmtrK8uXL2fnzp3Mnj3br5kQXnUcjrmhLLKkpKQk2AcRCqKiolixYgWff/45X3zxBS+88AJPPfUUt912m+SGeO7IkSMZM2YMN954I9dffz0zZ87knnvu8Vve18KpjsMxN5TJVUmX4NNPP8ViseB0OrnjjjsCdoWH5Po3d+7cubz66qse2/Lz8ykvL/drLoRPHYdrbsgK9lhWqCgsLOyxLS8vT3JDOHfOnDlKWlqacttttylpaWnu5Z577lFmzJjhl8wLhUMdh3NuKJNzDF4UFhZy6NAhWlpaPJ7n43Q6GTp0qOSGcO4LL7zAV199xfLlyzEaje7tarWa+Ph4v2RCeNVxOOYOBDKU5MXp06f7/PFQq/3TtkpuYHIB/v73v/e6ffz48X7JC7c6DrfcgUAaBh8F+sdDcgOXm5ub637tcDg4fPgwP/zhDzGZTH7LhPCq43DMDWXSZPpo1apV7tcX/nj4+x+X5Po/d8OGDR7rx48f5/nnn/db3tfCqY7DMTekBfcUR+j67LPPlKefflpyB2huMB7LHG51HG65oUR6DP307W9/m4aGBskdALkLFy70WD969Cjf//73/ZrZm4Fcx5IbWqRh8FGwfjwk1/+5d9xxh/u1SqVi8uTJ3HXXXX7NhPCq43DMDWVy8tlHF06lqVKpiI2N5a677vL7o5klNzC5bW1t7Nu3D7VazejRowPyiPFwq+Nwyw1l0jBcgmD8eEiu/3Pfffddli9fTnJyMk6nk4MHD1JaWur3x5tD+NRxuOaGKnlWko/effddZs+eTUdHBx9++CGvvvoq3/3ud7nlllskN8Rzf/7zn1NRUcHUqVN58MEHue+++1i8eDGPPvqo3zIhvOo4HHNDWnDPfYeOrKwspaWlxb3e1NSkZGVlSe4AyNXpdIrL5eqxzd/CqY7DMTeUyclnH6nVaoYMGeJev+mmmwJy56Tk+i+3qqoKgJtvvplZs2bx8MMPo1ar+a//+i9GjBjhl8wLhUMdh3NuKJPa8SJYPx6S6/9ci8UCwHXXXcd1113HX/7yFwCio6P9kve1cKrjcMwdCKRh8CJYPx6S6//cQNzd3JtwquNwzB0I5KokEbZmzpzJa6+9RlpaWq8zeu3atSsIRyVE8EnD4EWwfjwk1/+5VqsVrVbL/v37e33M9k033XTFMyG86jgccwcCaRi8CNaPh+QGJhcgMzOTd99912/lf1O41XG45Q4IwbwkKpRMnjxZcgdoblFRkbJ161bl6NGjSnNzs3vxt3Cq43DMDWVyg5uP/vGPf3D27FmuueYabDYbnZ2ddHZ2cv3110tuiOf++te/xmKx8Pbbb2M2mzGbzWzdupX8/Hy/ZUJ41XE45oYyGUryUVpaWo9tKpXK7+OUkuv/3K+++qrHIxKampq4+eab/ZYJ4VXH4ZgbyqRh8FGwfjwk13+5X3zxBYqiYDAYWLt2LV//V3A6nRQUFLB9+/YrnnmhcKjjcM4NZRHBPoCr3RdffMHnn3/OE0884X79+eefc/z4cZ566inJDeHcVatW8cQTT9DY2Mjjjz/OE088wRNPPMGTTz7p1wfohVMdh2PuQCA3uHmxatUqLBYLVquVxx9/3L1drVYzadIkyQ3h3K9vcDOZTBgMBo/3zp0755dMCK86DsfcASGYZ75DyWuvvdZjW3d3t+QOgNzp06d7rDudTmXKlCl+zVSU8KrjcMwNZTKU5KNvnqhyuVzk5ORIbgjn5uXlMXLkSPbt20diYqJ7GT16NMOHD/dL5oXCoY7DOTeUyVCSF3l5eezZsweAxMRE9/bIyMher3aQ3NDJfeONNwD45S9/ycSJE+nq6kJRFJxOJ83NzX7JhPCq43DMHQikYfAiWD8ekhuYXICTJ0+yYcMGPvvsM374wx9isVhITk72W1641XG45Q4E0jD4KNA/HpIbuNxPPvmEHTt2sHz5cnJycigqKqKoqMivmRBedRyOuaFMzjH46JNPPuGNN97gvvvu46mnnuI///M/A/JXh+T6Pzc+Ph6VSsXw4cM5fPgw3/72t7Hb7X7NhPCq43DMDWXSMPgoWD8ekuv/3O9973s899xzTJgwgfXr12Mymdw3u/lTONVxOOaGMhlK8tHXPx6PPvooCxYswGq1BuTHQ3L9n1tSUsLevXu59dZbmTt3LvX19bz88st+zYTwquNwzA1pgb4+NlQ5HA7l73//u6IoirJz507lueeeUw4fPiy5AyQ3GMKtjsMtN5TJs5KEEEJ4kHMMQgghPEjDIIQQwoM0DEIIITxIwyCEEMLD/w/AZu9EY3YVRwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_att = df[['attribute1', 'attribute2', 'attribute3', 'attribute4', 'attribute5', \n",
    "             'attribute6', 'attribute7', 'attribute8', 'attribute9']]\n",
    "corr_pearson = df_att.corr() \n",
    "\n",
    "import seaborn as sns\n",
    "sns.set()\n",
    "sns.heatmap(corr_pearson, center = 0, cmap = 'RdBu_r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# attribute8 and attribute9 perfectly correlated. dropping attribute9 to avoid perfect collineraity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(['attribute9'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>attribute1</th>\n",
       "      <th>attribute2</th>\n",
       "      <th>attribute3</th>\n",
       "      <th>attribute4</th>\n",
       "      <th>attribute5</th>\n",
       "      <th>attribute6</th>\n",
       "      <th>attribute7</th>\n",
       "      <th>attribute8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>215630672</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>52</td>\n",
       "      <td>6</td>\n",
       "      <td>407438</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>61370680</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>403174</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>173295968</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>237394</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>79694024</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>410186</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>135970480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>313173</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure  attribute1  attribute2  attribute3  \\\n",
       "0  2015-01-01  S1F01085        0   215630672          56           0   \n",
       "1  2015-01-01  S1F0166B        0    61370680           0           3   \n",
       "2  2015-01-01  S1F01E6Y        0   173295968           0           0   \n",
       "3  2015-01-01  S1F01JE0        0    79694024           0           0   \n",
       "4  2015-01-01  S1F01R2B        0   135970480           0           0   \n",
       "\n",
       "   attribute4  attribute5  attribute6  attribute7  attribute8  \n",
       "0          52           6      407438           0           0  \n",
       "1           0           6      403174           0           0  \n",
       "2           0          12      237394           0           0  \n",
       "3           0           6      410186           0           0  \n",
       "4           0          15      313173           0           0  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date          0\n",
       "device        0\n",
       "failure       0\n",
       "attribute1    0\n",
       "attribute2    0\n",
       "attribute3    0\n",
       "attribute4    0\n",
       "attribute5    0\n",
       "attribute6    0\n",
       "attribute7    0\n",
       "attribute8    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check feature distribution and evaluate if normalize/standardize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20d10c88>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEUCAYAAAA/Yh00AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZQU5b3w8W/v+6z0zACCIKKAEcElmg3k5MUhDgRF43pjriS43CgnxAshyMEED2qUq1cvMQd5jZpcog4ooLlR9EoSX8W4IEFGWUUQGJh96X2t94+ebmlm6e6Znunt9zmHw/RTVf08tfSvnnrqqadUiqIoCCGEyBvqTBdACCFEeklgF0KIPCOBXQgh8owEdiGEyDMS2IUQIs9IYBdCiDyTNYHd6XQye/Zsjh071q/lm5ubueOOO5g9ezbXX389O3fuTHMJhRAiN2RFYN+1axc33ngjhw8f7vd3PPTQQ0yaNIk///nPrF69msWLF+P1etNXSCGEyBFZEdhra2u57777qKioiKVt3ryZq6++mrlz57Js2TJ8Pl+f37Fnzx6+973vATBq1ChKSkqk1i6EKEhZEdhXrVrFxRdfHPt84MABamtreeGFF9iyZQvl5eU8/fTTfX7HpEmT+J//+R8A9u/fz8GDB2lubh7UcgshRDbSZroAPXn//fc5cuQI1113HQCBQIBJkyZx8uTJWNqpXn31VX75y19y//33M2fOHC644AIuvfRSdDrdUBddCCEyLisDeygU4nvf+x7Lly8HwOVyEQqFKCoq4u233+5xmWPHjnH//fdjtVoBqKmpYfTo0UNWZiGEyBZZ0RRzuksvvZQ333yTlpYWFEXhV7/6Fc8991yfy/z3f/83L7zwAgBvv/02oVCICRMmDEVxhRAiq2RljX3ChAncdddd/OhHPyIcDjNx4kRuu+22Ppe57bbbuOeee9iyZQsWi4U1a9agVmfleUsIIQaVSobtFUKI/CJVWiGEyDMS2IUQIs9IYBdCiDyTFTdP29pchMPZ1dRfXm6lpcWZ6WKklaxTbkh1nUJh8AdDcWl6rQZNFlXbelsnjz/EroPxDxJecPYwTHrNUBWt3zJ57KnVKkpLLb1Oz4rAHg4rWRfYgaws00AN9joFw+ALBOPSDDot2kEMMoW+n9y+IB/uaYhLu2RiJRZDVvy8Y3pap2AojNsb6JYWDmfRWakP2XrsZdeeFznPF+g5yGizLMiI9MjEiVwkJr+2AiY/yuxw+n5QWt24fcGc2BdyIs9OsvULWK78KAdyAsqFk9fp+8FmNeJwervtC38gRKfbjy8QJhgMA6DXqQkpkSYBtVo15GXvjaIoeHwhnN4ALk+AL1vc1J/sxOkJ4PQEcHmCBMNhVCoVLR0eLCYdRWY9pTZDpoueF7LrFyxEDwZyAsqVkxdAKKzgcPtp7PBystnFZ4fb6HT66HAH6HT58PhCvS6rAkxGLRajFotRR32zi+FlZsqLTQwrNlJebMSg6/mGZF8nv2AojNcfwusL4vGHcHsDseDs9ARod/o5fLITfyCMLxDC5w+x6e1DuL1BQn20P5sNWrRaNf5ACF8gxKmPSb6z+wTnjCphyvhhjBlehFqlyrqTcbbLvqM7x/T0o4ChqxXmQo1UdKcoCi2dXg4e62DPl+18+kULbQ5fXIArtRmwl5g4s9JKkbmMYqseg17LsUZHrHYeDClUDbOw/0gbLm+kJtzS6eXYTifBUHxgNRk06LUadFo1Bl3kf5UK/MEwTneAsKIQ6urIoFapCITCBLquDHqj06gj36fXYNBpKLbqGVVhpcRqwGLUYTVF/pWVmQn4g1iMWsxGHZqu8ocV+HDPSdyeIJ1uP80dXlzeIH//Zz1//fg4ZqOWcSOKuHr6OMZU2tK7E/KYBPYU9BREwwrs2NvQbd6eaoWDEYR7qpF+/bwqfIH4H7VBJ7saIBQO4/IGu4KogkGvwajTYDbqB/Vk6A+EONro5PP6Tg4e7+DgsXbanX4g0pxSVmTkvLFllFj1DLfb0KrgWxeMQDmt1htWQKuJb3K54Bw75tO6B140oYJgIExLh5fmDg8tnV46XH4CwTD+QAh/MIw/0BW0VaDVqFGrIt3oNGoVw8stmAxaTHoNRoMWo16DSqXiaKMDgy4SxA16DRdNrGTX/qa4vHvqkaNoNPx9x5fdtssF59hRq1RYzTqsZh0jhlm44Bw7H352kmONTr444aDuUCu7D7Vy7qgSvnPBcC46t6LXqw8RIb/2Xjjcfly+xEH8gnPsSX/nYDQL+AMh6ptdNLZ56HD56XT52fL/vsDrD6GgYNBpMOq1jBtZxPnj7dhtBsZU2dAOcSdnRVFQALVq8NuBFUWhzeGjvsXFFyccfHKwmXaXH5cngNcfAg50W8Zs0FJqM1Bs1VNsMVBi1VNijXyO/m8z6dHr1L1uO0VR8AVCtDl8tHR6Odni5shJB4cbHJxodhPuqo6XFxk5d3QpZ48s5uyRxZQWG/l4X2Pse6Jt7L5AqFvQTPZ4U6tUlNoMlNoMnH1GcZ/zupLsLunyBQmG+q7Bp4teq+GsEcWcNaIYtzeANxDmg08b+L9/3sP6N/dz6aQqvjN5OGOqbKiSOKYK7cpWAnsvPN7uB3sqQXwwdbr97NzfxEd7G9n7ZTuhsIIKsJp1FFn0jLRb6XBGXiXo84fw+IPUHWrlH59G1seo1zBhdCmTxpYRDIUw6tNzGCiKwslWN3u/bKOpzYPDHcDhDrD+jf2x9la9To2xq7Zn0EVqggadGoNe23US0qDRRGqNapUKtVpFKKxwosWFSqVCpYoELacngEmvpbjISHOrG7cvQJvDx4kWN/XNrq4AHmEyaCixGiirsGIxahk7spiTzS4g0pThD4YosuhxeYJ0OH3sb22jw+Xv1pQRpVGr0OvU6LUa1GoV4bBCMBTG4wvFgndUkVnHmVVFTB1v58xKG2OH2ygrMsbNc3oFIhuo1KoeKzaZYDbqmD61kqu/PZb9R9t5e9cJtu8+wd92HucMu4XvTB7BN75WhdXU+4t1kq1U5csJIOlf9G9+8xva2tp46KGH4tLr6+tZvHgxLS0tjB07ltWrV2Ox9P5EVCHZsa+J9z87SVOHl3anjyKzHo06UpOqKDVRXmxM/CVdWju97Pq8pSuYt6EoUFFq4vKpI4HI37quo++Cc+zdanoXT6iguNjM+7uO89nhyKXtPw82o1JBZamZ0ZVWRldaU17HsKJw6HgnH+1rZMe+Jlo6Iy8QNxm0FFv1jK60MmZEEZquWpU/GMbnDxEMRZoEvP7Iv3aXH68/FJsWViJtvWGl54dAduyLXz+VCorMekYMs/Ctrw1nxDAzI4ZZKLYZ2HukLW7enrbP18+rimv2UBQFtzeIyxuitdNDh9OH0xskEAwRCoHXH8QfCKEQuXGp12ki7ccGbeQkUmSgrNiMyaCOq1HqtNqMBcze7gf1lP9ArhZ6OiloBvgyM5Vahdsf4oxKGzddYePq6Wex62AL23fX8/xbB9jwt4NMOXsY3548nK+dVd7vK8Ncutnel6RK+95777Fp0yYuv/zybtN+/etfc9NNN1FTU8Nvf/tbnnzySRYvXpzucuYUnz/Es6/tZcfeRkptBqrKzAwvN9Pu9HOs0cnhkw4gEoze3X2Sc84oYfgwM8OKjFhMOkIhBV8wREOrm+PNLvZ/2c7xrhpmRamJKy4ZxZRz7IwcZkFB1WMb/+lUKhVlRUYunlDBxRMqUBSFfcc6eO0fR/jypIMP9jTywZ5Gduxr5uIJdiaMLmVUhbXHZgenJ8CBY+18+kUrH+9vot3pR6tRMWlMGTMvGYUvEMRq0sUCWk+BtLdL/dN/VNHl/7mvEUWJnEgUBaaMt2PUqSkpteDo8GA0aHr8MSdbG+4pkEXzPnKyA7VaRZFZB+iSOjFAJGB+1EPTXX8DZip6q3H3dKykO/+etuXF5w1P+3dedl4l371wJMcanbyz+wTb607y0b4mRlVYufbycXxtbFlSzTT5KGFgb29v57HHHuOOO+5g7969cdMCgQAffvghv/3tbwGYN28e//Iv/1LQgd0fCPHn7Yfx+ENMObs8UntQxwc4rz9Ec7uHxnYPXl+Iv/3zeK+9D0wGDWOHF3HJxArCYYViqx6VSsWJZhcnml0p1aIaux58iRpptzJ1/DCmjh9Gu9PH0QYnzR1eNvz1cyByk66y1IzFpMOo1+D0BOhw+mO1cr1WzflnlXPRuXYmjxuG2ajtNTgPVKwZhsi2NBu1WAxayoqMhHyBBEsPvoHUcAdDtpVnMJ1RYeWG747n2svH8eHeRja9fYjHandx4Tl2bpszCX0vN1qzqbkp3RIG9hUrVrBo0SJOnDjRbVpbWxtWqxWtNvI1drudhob0/6hzyc4Dzbi9Qf7tmvNxuPw9zmPUazijwsoZFVYumViJUaemwxnp6uX2BdGqVei0aipKzZR0BfKBBkxfIMSeI404nN5Y2qk/9BKrgRKrgUsmVuLzBfm8vpND9R00tnlwe4N0OP1YTFoqRhUzvXwE54wqYUyVrdcfzWCL/iiVU05WudgWKtJHq1FzycQqJo0t468fH+fVd77g0dpd3Pb989D1cJzm88mvz8C+YcMGhg8fzje+8Q1efvnlbtMVRel2qdOfS5/y8tTbdgdbY6sbmzW+DVyn0/aZ1tDqZt+X7Uw+exjnjiln32ltuz0tbzYbqCgzU1UJ53alOdx+PN5THjEn0kZ5+rLJlOnUNIj/jt7KM+aMUs4d178DXElyu+kNOpTTmnlSWccwKvYdbY9Lu/DcCuxl5n6Vp6e0VOZNd5rNahySfIZqfaLrlM68ezqGguEQB+sdjKoq4ruXjOatD7/kP17YyT03X9TvfMxmQ7fjKspuz86+9X0G9r/85S80NTUxd+5cOjo6cLvdPPDAAyxbtgyAsrIyHA4HoVAIjUZDU1MTFRUVKReipcWZfaOkaTRxtVuAQCDYa1o4rLDtoy8xGTRMGlPS57yncrt9NIXinyjsqXZ+wTn2bssmKtPpaUBcerLlSYXbl1x5nG5fj7WlVNcx2jUQwOP1c/iYL26+sEJK+zGVvAcrLbpOQ5H3UK0jdN8PA/3ORMfQiHIT37lgOG/vOsH/fnCEihJTv/Lp7Tdht9toanJ0Sx8KarWqzwpxn4H9mWeeif398ssv88EHH8SCOoBOp+Piiy/mL3/5C3PmzGHz5s1MmzYtDcXOPV+c6KS108e0KSPQa/Pr4YlMP12brHy+tBb9M2Z4EYdPOti24xjf/9ZYzMbc6t3SX/36Wd5777289dZbANx3333U1tZy5ZVX8tFHH/Gzn/0srQXMFV+ccGA16TizH10Gs120C9jp//yhEC5fMO5ftl14CXHRuXZCIYWdB7r3espXSZ++5s2bx7x58wBYtWpVLH3kyJH88Y9/TH/JcojPH+JEi4tJY0pTvseQy3fmpYYscoHNrOc7U0bwt4+PM2F0aUrPj+SqwrguGWRHG50oCpxZlfqNlGwLjrl8ohGiN//nklFs332CT79oZdqUEZkuzqCTwJ4GR05GmmHKi3K/JpBtJxoh0sFk0DKmysah+k6CofCQj5U01PJ77YaA2xvkRIuL0ZXWgn3KTYhcMLrSRjCkUN/1FHc+k8A+QHWHWggrMKYfzTBCiKFTVWbGoNNw5GRmuigOJQnsA7T7YDMWo7YgbsgIkcvUahWjKq0ca3QRGqLhhzNFAvsAKIrCFyccjBhmkWYYIXLAmZU2AqEw9S3uTBdlUElgHwCHO4DHF5TauhA5Yni5Gb1OnffNMRLYByA6yqEEdiFyg1qtYlSFlaONWTiMSRpJYB+Alg4vWo2aUqsh00URQiSpqsxMIBimsS1/m2MksA9Ac4eXkXZLbLx1IUT2G9Z1hf1lgzPDJRk8Etj7KawotHZ6GZWHY8MIkc+KLHp0WjVHG/K3nV0Cez91OiMvOx5VIf3XhcglKpWK8iKj1NhFd80dkRunUmMXIvcMKzZyojl/+7NLYO+nlk4vOo0ae6kp8cxCiKxSXmwkFFZodfgSz5yDJLD3U3OHl7JiA2p5MEmInDOsJHIDNXrlnW8ksPdDKKzQ1umL3V0XQuQWs0FLkUVPS54G9qSG7X388cfZunUrKpWKa6+9lltvvTVu+po1a3jppZcoKioC4LrrruPmm29Of2mzRIfTR1hRKMuDYXqFKEQq1VcPKuWjhIH9gw8+4B//+AevvPIKwWCQK6+8kunTp3PWWWfF5qmrq+PRRx9l6tSpg1rYbNHp8gNQbNFnuCRCiP4aXWXj0y9a8QdC6HX59Z7ihE0xX//61/nDH/6AVqulpaWFUCiE2WyOm6euro61a9cyZ84cVq5cic+XnzckojrdASDyyi0hRG6K9mjLx3b2pNrYdTodTzzxBDU1NXzjG9+gsrIyNs3lcjFx4kQWL17Mpk2b6Ozs5Mknnxy0AmcDh8uPyaBFp5VbFELkqpH2SGBvd+ZfRTTpV+MtXLiQBQsWcMcdd1BbW8v1118PgMViYd26dbH55s+fz7Jly1i0aFHShSgvz76+4I2tbmzW+DZ0nU6LzWrE5QtSajNgsxpjaafrKT3daakuD8SlD2Xeg5lP9P9Mb990pvV2bOXqOkbXKVN595ReYjNi0Gnw+sN9HkNmswF7WXwrRZTdnp0PKCYM7J9//jl+v5+JEydiMpm44oor2LdvX2x6fX0927dv59prrwUiY5Rrtam9SrWlJQtHWtNocDjjL9ECgSAOp5d2h48zKqw4nN5Y2ul6Sk93WqrLA3HpQ5n3YOVjsxpj0zK9fdOVFl2nbNi+6cwn246hYDCEzayjucPT5zHkdvtoCoW6fafdbqOpKTPDEqjVqj4rxAnbEo4dO8by5cvx+/34/X7eeustLrrooth0o9HII488wtGjR1EUhfXr1zNz5sz0lD4L+YMhvP4QRWZdposihBigIos+1hkinyQM7NOnT+fyyy/nqquu4pprrmHq1KnU1NSwYMECdu/eTVlZGStXruTOO+9k1qxZKIrSrTtkPnG4IjdOi6RHjBA5r8iix+0NEsyzoQWSajO5++67ufvuu+PSTm1Xr66uprq6Or0ly1Kd7sjZXXrECJH7olfeDneAUlv+vFdBunWkyOGKBnZpihEi19m6rrzzrTlGAnuKOt0BzEYtWo1sOiFyXZFZArsgcgAUSTOMEHlBp1VjMmhjTaz5QgJ7ihzugDTDCJFHiiw6qbEXMrc3iC8Qkh4xQuSRIrOezq7ebvlCAnsKmts9gNw4FSKfFFn0+AIhfIHuDyHlKgnsKWjqCuxSYxcif0R/z448ao6RwJ6CWI3dJDV2IfJFtC97Pt1AlcCeglaHD7NRi0a6OgqRN6xmPSrIq3Z2iVApaHf4sBhTG+BMCJHdNGoVVnN+9YyRwJ6CdqcPs1GaYYTINzazDoc0xRQeRVFod/ilxi5EHrIYdbi8wUwXI20ksCfJFwgTDIUxS2AXIu9YTDq8/hCBYH6M8iiBPUlub+TGikWaYoTIO9Er8Y48eU2eBPYkRS/TpClGiPwTrbC1OySwF5RojV1ungqRf6JNrG1SYy8sLm8QtVqF0aDJdFGEEGkWvRIvqBr7448/zpVXXklNTQ3PPPNMt+l79uxh3rx5VFdXc++99xIM5s/d5Si3N0ixRY9apcp0UYQQaabRqDHqNbQ786PLY8LA/sEHH/CPf/yDV155hZdeeok//vGPHDp0KG6exYsXs2LFCrZu3YqiKNTW1g5agTPF5QlQbM2fV2cJIeJZjNrCqbF//etf5w9/+ANarZaWlhZCoRBmszk2/fjx43i9XqZMmQLAvHnzeP311wevxBni9gUpscngX0LkK4tJlzeBPakuHjqdjieeeILf//73zJo1i8rKyti0xsZG7HZ77LPdbqehoSGlQpSXW1Oafyg0trqxWY1A5OEktzdIeZEplhal02m7pfWWnu60VJcH4tKHMu/BzCf6f6a3bzrTbFZj1mzfdOWTzccQQInNyL4v27rNZzYbsJeZ6YndbusxPdOS7ru3cOFCFixYwB133EFtbS3XX389AOFwGNUp7c6KosR9TkZLi5NwWElpmUGn0eBwegHw+IKEwgo2iy6WFhUIBLul9Zae7rRUlwfi0ocy78HKx2Y1xqZlevumKy26TtmwfdOZT7YeQ1E6jQqfP0RLmwu97qtOEm63j6ZQ97Ha7XYbTU2ObulDQa1W9VkhTtgU8/nnn7Nnzx4ATCYTV1xxBfv27YtNr6qqoqmpKfa5ubmZioqKgZQ567i7+rCXWKUpRoh8Fe0Zkw9DCyQM7MeOHWP58uX4/X78fj9vvfUWF110UWz6yJEjMRgM7NixA4AtW7Ywbdq0wStxBri6+rCXyM1TIfKWpes9C9Hfey5LGNinT5/O5ZdfzlVXXcU111zD1KlTqampYcGCBezevRuA1atX8+CDDzJr1izcbje33HLLoBd8KMVq7DYJ7ELkq2iN3e3J/Rp7Um3sd999N3fffXdc2rp162J/T5gwgY0bN6a3ZFnE5Q2iVn11RhdC5B+jQYtarSqMGruIDCdgNurk4SQh8phapaLYoi+MNnYRqbHL4F9C5L8Sm0Fq7IXC7Q3KOOxCFIASmwFXHrSxS2BPIPJwUkBGdRSiAJRYDbi9QRQly56rSZEE9gR8gRBhBcwGqbELke9KrHrCioLX3/2BpFwigT0Bjy+yg00yXK8QeS/apTnXb6BKYE/A44vsYJPU2IXIe0WWyNPl0d99rpLAnoAEdiEKR5FZAntBkMAuROGwmSOdJCSw5zmPL4RWo0KnlU0lRL6LvknJLW3s+c3jC0ptXYgCYjJopcae7ySwC1FYIoFdujvmNbcEdiEKillq7PnP6wvJw0lCFBCTQYPHHyScw0+fSmDvQyAYJhAKy8NJQhQQk0GLooAvh58+lcDeB+nqKEThif7ec7k5JqmItWbNGl577TUg8kalJUuWdJv+0ksvUVRUBMB1113HzTffnOaiDj0J7EIUnuhIrm5fkLIMl6W/Ekas7du3884777Bp0yZUKhU/+clPePPNN5k5c2Zsnrq6Oh599FGmTp06qIUdah5/dJwYCexCFIqCqLHb7XaWLl2KXh951HbcuHHU19fHzVNXV8fatWs5fvw4l1xyCb/4xS8wGHL//aAeb7TGLm3sQhSK6O89l7s8JmxjHz9+PFOmTAHg8OHDvPbaa0yfPj023eVyMXHiRBYvXsymTZvo7OzkySefHLwSDyGPL/KuU4NOArsQhUKjVqPXqfO7xh514MABbr/9dpYsWcKYMWNi6RaLJe7F1vPnz2fZsmUsWrQo6UKUl1uTnneoNLa6CYYVzEYdRTYTADqdFpvVGDdfT2mpzDuQtFSXB+LShzLvwcwn+n+mt28602xWY9Zs33Tlk83H0OlpVpMefzCMzWrEbDZgLzN3+04Au93WY3qmJRXYd+zYwcKFC1m2bBk1NTVx0+rr69m+fTvXXnstEHnjkFabWpt0S4uTcDjL+oxqNHS6fBj0GhxOLwCBQDD2d1RPaanMO5C0VJcH4tKHMu/BysdmNaa8f7JtHU9Pi65TNmzfdOaTrcdQT2kGnRqHy4/D6cXt9tEU6t4sY7fbaGpydEsfCmq1qs8KccKmmBMnTvDTn/6U1atXdwvqAEajkUceeYSjR4+iKArr16+Pu7Gayzy+kNw4FaIA5fp4MQmj1tNPP43P5+Ohhx6Kpd1www1s27aNhQsXcv7557Ny5UruvPNOAoEAF154IbfeeuugFnqoeHxB7CXdL+uEEPktGthz9d2nCQP78uXLWb58ebf0G2+8MfZ3dXU11dXV6S1ZhgVDYbz+EEa91NiFKDRmg5awEnnncS6SJ0974XD5AXmJtRCFyGSM9mWXwJ5XOpw+4KsdLIQoHF/1Zc/NdnYJ7L1od0Zq7PJwkhCFJ3qlnqtvUpLA3osOV1eNXdrYhSg4uT6sgAT2XnR2tbEbpcYuRMHRatTotLn79KkE9l50Ov3odWo0atlEQhQik0EbGwgw10jU6kWHyyfNMEIUMJNBIzX2fNPp8kszjBAFzKTP3adPJbD3otPllxq7EAUsl4cVkMDeiw6nT8aJEaKAmQwagiElJ999KoG9B75AKDKcgDTFCFGwohW7jq4ecrlEAnsPol0dpSlGiMIVDeydEtjzQyywS41diIIlgT3PRC+9ZGRHIQpXtGLX6ZbAnhe+qrFLYBeiUBl0GtQqqbHnja9q7NIUI0ShUqlUGA3a/A3sa9asoaamhpqaGh5++OFu0/fs2cO8efOorq7m3nvvJRjMzb6fUZ0uP1aTDrValemiCCEyyKTP08C+fft23nnnHTZt2sTmzZv59NNPefPNN+PmWbx4MStWrGDr1q0oikJtbe2gFXgodLj8FFn1mS6GECLDTAZNfrax2+12li5dil6vR6fTMW7cOOrr62PTjx8/jtfrZcqUKQDMmzeP119/ffBKPAQ6XX6KLIZMF0MIkWGmHG2KSXh3cPz48bG/Dx8+zGuvvcbzzz8fS2tsbMRut8c+2+12Ghoa0lzModXh8nHWyJJMF0MIkWEmgxanO0AoHM6pkV6T7vZx4MABbr/9dpYsWcKYMWNi6eFwGJXqq7ZoRVHiPiejvNya0vyDzeEOUGTRY7Ma49J1Om1SaanMO5C0VJcH4tKHMu/BzCf6f6a3bzrTbFZj1mzfdOWTzcdQb2mlRUYUQG8yUFbU/Xvtdlu3tGyQVGDfsWMHCxcuZNmyZdTU1MRNq6qqoqmpKfa5ubmZioqKlArR0uIkHFZSWmaweP1BvP4QxVYDDqc3blogEEwqLZV5B5KW6vJAXPpQ5j1Y+disxti0TG/fdKVF1ykbtm8688nWY6ivNBWRuHToSCuhqvggbrfbaGpydMtrKKjVqj4rxAmvLU6cOMFPf/pTVq9e3S2oA4wcORKDwcCOHTsA2LJlC9OmTRtAkTMr2p5WZJGbp0IUuq/Gi/FluCSpSVhjf/rpp/H5fDz00EOxtBtuuIFt27axcOFCzj//fFavXs3y5ctxOp2cd9553HLLLYNa6MEU7cNebNHT0p7b3TaFEAMTDezRl9vnioSBffny5Sxfvrxb+o033hj7e8KECWzcuN9I/xUAABfZSURBVDG9JcuQr2rsBlra3RkujRAik0xdDynm2giPuXObd4jEauzSj12IgqfRqDEbtHQ4c6spRgL7aTpdflSA1azLdFGEEFmgyKKXGnuu63D5sZl1OdVnVQgxeIosejpyrI1dotdpIk+dSjOMECKiyKLLuV4xEthP0yGBXQhxCps5UmNXlOx41iYZEthP0+H0USzjxAghuhRb9PiDYbw59FJrCeynUBSFDpefEukRI4ToEr2Cb8+hnjES2E/h8gYJhhSKrVJjF0JERAN7Lo3yKIH9FNG+qsXSxi6E6FJkjsSDXOryKIH9FNEdJ00xQoio6Et3cmlYAQnsp4j2VZWmGCFElNmgRatR5VSXRwnsp2h3SVOMECKeSqWiOMceUpLAfooOpx+9To2xa+AfIYSAyKCA0saeozpcfkoshpTfACWEyG8lVn1ODQQmgf0UHU5f7EaJEEJEFefYQGAS2E8RqbFLYBdCxCu2GnC4AwRD4UwXJSkS2E/R7vRLjxghRDfRDhUOdyDDJUlO0oHd6XQye/Zsjh071m3amjVrmDFjBnPnzmXu3LmsX78+rYUcCv5ACI8vKD1ihBDdFOfYsAIJX40HsGvXLpYvX87hw4d7nF5XV8ejjz7K1KlT01m2ISVvThJC9CZ6JZ8r7exJ1dhra2u57777qKio6HF6XV0da9euZc6cOaxcuRKfLzfOaqeK9lEtkaYYIcRpojX2XOkZk1SNfdWqVb1Oc7lcTJw4kcWLF3PmmWeydOlSnnzySRYtWpR0IcrLrUnPO1j2n3AAMOaMUux2G42tbmxWY9w8Op02qbRU5h1IWqrLA3HpQ5n3YOYT/T/T2zedaTarMWu2b7ryyeZjqK80s9nAyKoiAIKosNttsWmn/p1NkgrsfbFYLKxbty72ef78+SxbtiylwN7S4iQczuwg9kfrOwAIB4I0NTlAo8Hh9MbNEwgEk0pLZd6BpKW6PBCXPpR5D1Y+NqsxNi3T2zddadF1yobtm858svUYSpTmdvtQhUJYjFrqGxyR+EAkqEf/HmpqtarPCvGAe8XU19ezcePG2GdFUdBqB3y+GHIdLh9qlQqbSV5iLYTorsSaO0+fDjiwG41GHnnkEY4ePYqiKKxfv56ZM2emo2xDqt3px2bRoVbLU6dCiO6KLPqcGQis34F9wYIF7N69m7KyMlauXMmdd97JrFmzUBSFW2+9NZ1lHBIdzshwAkII0ZPIsAK5UWNPqc1k27Ztsb9PbVevrq6muro6faXKgA6XT3rECCF6Vdw1EJiiKFk/npQ8edqlw+mXh5OEEL0qtuoJBMN4fMFMFyUhCexAOKzQ6ZbhBIQQvYv1Zc+BG6gS2AGH24+iyCvxhBC9i1b8cuEVeRLY+WpHFcvNUyFEL3Lp6VMJ7EBrZ+SBhLIiCexCiJ6V2iLxoc0hgT0ntHbtqDKbBHYhRM9MBi0mg5bWTgnsOaHV4UWjVmGTXjFCiD6U2Qy0OroPWZBtJLATubQqtRlQZ3nfVCFEZpUWGaTGnitaO33SDCOESKjMZqRNauy5oc3hpbSo+1CfQghxqrIiA53uAIFgdr/7tOADe1hRaHNIjV0IkViZLVIBzPZae8EHdqc7QDCkxLoyCSFEb0q7ukRnezt7wQf26B3uMmmKEUIkEL2yz/aeMQUf2Nu6zrxSYxdCJBKtAGb7Q0oFH9hjDydJjV0IkYBBp8FizP6HlCSwRx9OMssr8YQQiZUVGWPDkGSrpAO70+lk9uzZHDt2rNu0PXv2MG/ePKqrq7n33nsJBrN/vOKotk55OEkIkbzI06d5UGPftWsXN954I4cPH+5x+uLFi1mxYgVbt25FURRqa2vTWcZB1SpdHYUQKSjNlxp7bW0t9913HxUVFd2mHT9+HK/Xy5QpUwCYN28er7/+enpLOYhaO73Svi6ESFqZzYDLG8Trz96WiaTeebpq1apepzU2NmK322Of7XY7DQ0NAy/ZEAgrCu1On/SIEUIkLTq8d0uHl2wdNjCll1n3JBwOx73YtT8vei0vtw60GP3S5vASDCmMHlGM3W6Lm9bY6sZmja/J63TapNJSmXcgaakuD8SlD2Xeg5lP9P9Mb990ptmsxqzZvunKJ5uPob7SzGYD9jJz7PNZoyLt681tHi44x042GnBgr6qqoqmpKfa5ubm5xyabvrS0OAmHlYEWJWWHT3YCoFNBU5MjfqJGg8MZ344WCASTSktl3oGkpbo8EJc+lHkPVj42qzE2LdPbN11p0XXKhu2bznyy9RhKlOZ2+2gKhWKf1eHI303tnu5xY4io1ao+K8QD7u44cuRIDAYDO3bsAGDLli1MmzZtoF87JKJ9UeXNSUKIZEWbbps7PBkuSe/6HdgXLFjA7t27AVi9ejUPPvggs2bNwu12c8stt6StgIMp+vRYqU1ungohkqPTarCZdTS3Z29gT6kpZtu2bbG/161bF/t7woQJbNy4MX2lGiKtnV60Gnk4SQiRmjKbkaYsDuwF/eRpY7uHYcUmeThJCJGSYSVGGlpcmS5Grwo7sLd5qCg1ZboYQogcU1lqpqHVTSicnS/cKNjAriiKBHYhRL9UlJoIhhRasnQwsIIN7B0uP75AiMpSc+KZhRDiFJVdFcLGNneGS9Kzgg3sjW2RGx+VUmMXQqSosuuBpYbW7LyBWrCBvaHrTCtNMUKIVBVb9Bj1mlgcyTYFG9gb2zxo1CrKi6UPuxAiNSqViuHDLLEr/2xTsIG9oc3DsGIjGnXBbgIhxAAMH2ahQQJ7dmlsdVMhN06FEP00YpiV5nZPVnZ5LMjArigKDe3S1VEI0X8jhlkIhbOzy2NBBvZOdwCfPyQ9YoQQ/TZ8mAWIXP1nm4IM7A2t0R4x0hQjhOifEfbIsLnZ2M5ekIFd+rALIQaq1GbAoMvOLo+FGdjb3ahV0tVRCNF/KpWKilJTVnZ5LMjA3tAa6eqo1RTk6gsh0qSy1CRNMdlCBv8SQqRDZZk5K7s8JhXYX331Va688kquuOIK1q9f3236mjVrmDFjBnPnzmXu3Lk9zpMtFEWhoc0tg38JIQasotREKKzQ3NH9PauZlPANSg0NDTz22GO8/PLL6PV6brjhBi699FLOPvvs2Dx1dXU8+uijTJ06dVALmw6N7R68/hCjKnt/EawQQiRjVEUkjnzZ4MyqymLCGvv27du57LLLKCkpwWw2U11dzeuvvx43T11dHWvXrmXOnDmsXLkSny/7OuxHHT4Reav4mZW2DJdECJHrRg6zolGrOHyyM9NFiZMwsDc2NmK322OfKyoqaGhoiH12uVxMnDiRxYsXs2nTJjo7O3nyyScHp7RpcOSkA61GxUi7JdNFEULkOJ1WzRkV1liFMVskbIoJh8OoTnknqKIocZ8tFkvci63nz5/PsmXLWLRoUdKFKC8fumaR+lY3Y0YUM7yquM/5Glvd2Kzx3SF1Om1SaanMO5C0VJcH4tKHMu/BzCf6f6a3bzrTbFZj1mzfdOWTzcdQX2lmswF7Wc/NLHa7jQljynhnVz3DhlnjYmMmJQzsVVVVfPTRR7HPTU1NVFRUxD7X19ezfft2rr32WiAS+LXahF8bp6XFSTispLRMfyiKwoGj7Vw6sYKmpgRnWI0GhzP+hkggEEwqLZV5B5KW6vJAXPpQ5j1Y+disxti0TG/fdKVF1ykbtm8688nWYyhRmtvtoykU6vaddruNpiYHlSVGXJ4Anx1oHLKn2dVqVZ8V4oRNMd/85jd57733aG1txePx8MYbbzBt2rTYdKPRyCOPPMLRo0dRFIX169czc+bM9JQ+zRrbPXh8Qc6skvZ1IUR6jK0qAuDwyexpjkkY2CsrK1m0aBG33HILV111FbNnz2by5MksWLCA3bt3U1ZWxsqVK7nzzjuZNWsWiqJw6623DkXZU3aka8OP6doRQggxUCPtFrQaVSy+ZIOk2kzmzJnDnDlz4tJObVevrq6muro6vSUbBIdPyI1TIUR6aTVqRtqtuVVjzyeHT3Zyht0qQwkIIdJqTJWNIycdKMrg3ytMRsFEOEVRONLgZIy0rwsh0mxMlQ23L0hTe3aMG1MwgV1unAohBsuYLLuBWjCB/VB95MkwuXEqhEi3yA1UNZ8fz44nUAsmsO862IzNrIuN7SCEEOmi1aiZMLqEXQebs6KdvSACezAUZvehFqacPQy1OjueDBNC5Jep59hpbPdQ3+zKdFEKI7Dv/bINjy/E1PH2xDMLIUQ/TDl7GAA7DzRnuCQFEth3HmhGr1MzaUxpposihMhTpTYDY4fbJLAPBUVR+OeBZs4bU4Zep8l0cYQQeWzqeDtfnOikzZHZocvzPrAfaXDQ5vBJM4wQYtBNHR9pjvnnwczW2vM+sH+8vxmVCi44uzzTRRFC5LkRwyxUlJrYeaApo+XI68AeCIbYXneCc0eVYDPrM10cIUSeU6lUXHSunc++aKOhzZ2xcuR1YP/rznpaO33UfHNMposihCgQMy8ehVarYvP/+yJjZcjbwO7xBfnz9sNMPLOU88aUZbo4QogCUWI1MPPiUbz/WQNfNmRmiIG8DexvfHgUpyfANdPHZbooQogCM+vS0ZgNWl5++1BG8s/LwN7Q6mbrB19y0Tl2zhohY8MIIYaWxajjym+cySeft2TkRmreBfaGVje/+dPHaDVqrp0htXUhRGZ896IzOLPSxu8217H7UMuQ5p1UYH/11Ve58sorueKKK1i/fn236Xv27GHevHlUV1dz7733EgwG017QZBw56eA3f/qYYEhhyU1TqRyiF8sKIcTpDDoN99wwhRHDLPzXS7v5cG/jkA0QljCwNzQ08Nhjj/GnP/2JzZs38+KLL3Lw4MG4eRYvXsyKFSvYunUriqJQW1s7aAU+ndMTYPehFv5zwy5+/eyHhMKRoH6GXUZxFEJkltWk499vmMrIYRZ+t7mOXz/7Ie99epLWTu+gBvmE7zzdvn07l112GSUlJUDk/aavv/46d911FwDHjx/H6/UyZcoUAObNm8cTTzzBTTfdlHQh+jPiYkunl6de+TT26K7ZqOOH1efwza8Nx2RI6lWufVLUKsxGXVyaVqNOKi2VeQeSluryJoOWUFDXr2WzdR1PXadMb990pUXXKRu2b/rSuv+ehirvdHxnbzEqmdhVZNGz4taL+Xh/E3/dWc+WdyLdIM0GLTfPPIfxo0oSfkeq+aqUBKeNtWvX4na7WbRoEQAbNmzgk08+4f777wdg586dPPzwwzz//PMAHDlyhNtuu42tW7emXFghhBADl7ApJhwOo1J9dXZQFCXuc6LpQgghhlbCwF5VVUVT01fddZqamqioqOh1enNzc9x0IYQQQythYP/mN7/Je++9R2trKx6PhzfeeINp06bFpo8cORKDwcCOHTsA2LJlS9x0IYQQQythGztEujuuXbuWQCDAtddey4IFC1iwYAELFy7k/PPPZ+/evSxfvhyn08l5553Hgw8+iF4vg24JIUQmJBXYhRBC5I68e/JUCCEKnQR2IYTIMxLYhRAiz0hgF0KIPFPwgT1XBjhLRaJ1WrNmDTNmzGDu3LnMnTu3x3mykdPpZPbs2Rw7dqzbtFzcT9D3OuXiflqzZg01NTXU1NTw8MMPd5uei/sp0Tpl5X5SCtjJkyeVGTNmKG1tbYrL5VLmzJmjHDhwIG6empoaZefOnYqiKMovf/lLZf369ZkoatKSWafbb79d+fjjjzNUwv755z//qcyePVs577zzlKNHj3abnmv7SVESr1Ou7ad3331Xuf766xWfz6f4/X7llltuUd544424eXJtPyWzTtm4nwq6xn7qAGdmszk2wFlUTwOcnTo9GyVaJ4C6ujrWrl3LnDlzWLlyJT6fL0OlTV5tbS333Xdfj0815+J+gr7XCXJvP9ntdpYuXYper0en0zFu3Djq6+tj03NxPyVaJ8jO/VTQgb2xsRG73R77XFFRQUNDQ6/T7XZ73PRslGidXC4XEydOZPHixWzatInOzk6efPLJTBQ1JatWreLiiy/ucVou7ifoe51ycT+NHz8+FrQPHz7Ma6+9xvTp02PTc3E/JVqnbN1PBR3Y83GAs0RltlgsrFu3jnHjxqHVapk/fz5///vfM1HUtMnF/ZRILu+nAwcOMH/+fJYsWcKYMWNi6bm8n3pbp2zdTwUd2PNxgLNE61RfX8/GjRtjnxVFQasd+Pj1mZSL+ymRXN1PO3bs4F//9V+55557uPrqq+Om5ep+6mudsnU/FXRgz8cBzhKtk9Fo5JFHHuHo0aMoisL69euZOXNmBks8cLm4nxLJxf104sQJfvrTn7J69Wpqamq6Tc/F/ZRonbJ2P2Xopm3WeOWVV5SamhrliiuuUJ566ilFURTlJz/5ifLJJ58oiqIoe/bsUa655hqlurpa+fnPf674fL5MFjcpidbp9ddfj01funRpTqxT1IwZM2I9SHJ9P0X1tk65tp/uv/9+ZcqUKcr3v//92L8//elPOb2fklmnbNxPMgiYEELkmYJuihFCiHwkgV0IIfKMBHYhhMgzEtiFECLPSGAXQogM6GsAuFN9+umnXHPNNXz/+9/n9ttvp7OzM+F3S2AXeeeTTz5hxYoVsc/z58+ntbW1x3kXLFjAwYMHef/995k9e3bKeW3YsCGl0fz27t3Lt7/97ZTzEfll165d3HjjjRw+fDjhvKtWrWLhwoW88sorjB07lqeffjrhMhLYRd45ePBg3Bgk7777bq/zrlu3jrPPPrvfee3YsQOv15twvmAwyLPPPsuPf/xjXC5Xv/MT+aGnAeA2b97M1Vdfzdy5c1m2bFlsMLFwOBw7ZjweD0ajMeH3Z/7ZVyH6KRwO88ADD7Br1y5cLheKovDrX/+aJ554AofDwS9/+cvYvD/60Y946qmnuPnmm5k8eTL79u3j5z//OQ8++CCPP/44AG63m4ULF3LkyBGKiopYuXIlY8eOZenSpYwfP54f//jHALHPo0ePZtu2bbz77rsYjUZuvvlmfve73/HGG28QDocZOXIk9913H5WVlXz22Wfs27ePNWvWMH/+/IxsL5E9Vq1aFff5wIED1NbW8sILL2AwGPiP//gPnn76af7t3/6NpUuXMn/+fB544AFMJhO1tbUJv18Cu8hZu3btorGxkRdffBG1Ws1TTz3F73//exYuXMjWrVt58MEHAXj55Zd57rnnKCsrAyIj9v3nf/4nQGweiDw+vnr1ai688EJefPFFlixZwoYNG3rNf+bMmbz11luMHz+em2++mc2bN7N//342bNiAVqvlxRdfZPny5axbt47JkyczefLkhO2pojC9//77HDlyhOuuuw6AQCDApEmT8Hq93HvvvTz77LNMnjyZZ555hl/84hc89dRTfX6fBHaRs6ZOnUpxcTEvvPACR48e5f3338disSRcrrehcs8991wuvPBCAK6++mp+9atf4XA4ki7PX//6V3bv3s0111wDRK4oPB5P0suLwhUKhfje977H8uXLgchwwKFQiP3792MwGJg8eTIA119/fewKsy/Sxi5y1t/+9jduv/12AL773e9y4403JrWc2WzuMV2tjv85qFQqtFotKpWKU0feCAQCPS4fDof5yU9+wpYtW9iyZQsvvfQSzz//fFJlEoXt0ksv5c0336SlpQVFUfjVr37Fc889x5lnnsnJkyc5dOgQAG+99Rbnn39+wu+TwC5y1rvvvsuMGTO46aab+NrXvsb//u//EgqF0Gg0ce/SPP1zb/bt28eePXsAePHFF7noooswmUyUlpZSV1cHQENDAx988EGP3/3tb3+bjRs34nQ6AXj88cdZsmRJ2tZX5K8JEyZw11138aMf/YiamhrC4TC33XYbxcXFPPjgg/zsZz9jzpw5vPTSSzzwwAMJv0+aYkTOuuGGG7jnnnuYM2cOwWCQb33rW7zxxhtMnjyZ3/72t9x1112sWbOGWbNm8cMf/pD/+q//6vP7zjrrLNasWcPRo0cpLy/noYceAuCHP/wh//7v/051dTVnnHEGl112WWyZadOmxeZbsGABDQ0NXHfddahUKoYPHx6bJkRPtm3bFvv7Bz/4AT/4wQ+6zTN9+vS4tzYlQ0Z3FEKIPCNNMUIIkWcksAshRJ6RwC6EEHlGArsQQuQZCexCCJFnJLALIUSekcAuhBB5RgK7EELkmf8PQFkefU86mdUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute1'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a21312f98>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEJCAYAAACOr7BbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3TU9Z3/8eckE6JhIjF0hlDc7tGuv8rhsrg7tsj6Sw5eMiEX0RSqSxRdbNhyavGyolFo3FRjWpcCRaWLW6utm7SJbpkYTzJw2i5igR+G/FbR3aiNWokJTiYkmDuZZD6/PyjzI+QbJwlTQsrrcU57/Ny+eX+GGV7z/X71G5sxxiAiInKamIkuQEREzk0KCBERsaSAEBERSwoIERGxpIAQERFLCggREbGkgBAREUv2iS4g2trbuwmFxv6fdkyf7uDo0a4/QUV/epO19slaN6j2iaLaoysmxsbFF08dcfzPLiBCITOugDi5drKarLVP1rpBtU8U1X726BKTiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhY+rP77yDGq7Onn+7jA8P64+Ps2BWjInIeUkD8UW/fALX1/mH9V82egT1eL5OInH/03VhERCwpIERExJICQkRELCkgRETEkgJCREQsKSBERMSSAkJERCwpIERExJICQkRELCkgRETE0qieIVFVVcWPf/xjBgYGuOOOO8jLyxsyXl9fz/r16+nu7sbtdlNUVITdbqe5uZl169Zx9OhRLr30UjZu3MjUqVPp6OjggQceoLGxkeTkZLZs2YLT6eRb3/oWR44cASAUCvH+++/z8ssvM2/evOjvXEREPlfEMwi/38/mzZspKyvD6/VSXl5OQ0PDkDnr1q2jsLCQnTt3YoyhoqICgKKiIlasWIHP52Pu3Lls27YNgC1btuB2u6mpqWH58uUUFxcD8K//+q9UVlZSWVnJ9ddfzze+8Q2Fg4jIBIkYEPv27WPhwoUkJSWRkJCAx+PB5/OFx5uamujr62PBggUA5Obm4vP5CAaD1NbW4vF4hvQD7N69m5ycHACys7PZs2cPwWAwfMwPP/wQr9fLQw89FL2diojImES8xNTS0oLT6Qy3XS4Xhw4dGnHc6XTi9/tpb2/H4XBgt9uH9J++xm6343A4aGtrY8aMGQBs27aNu+66C4fDMeYNTZ8+9jUALW09JDouGNafkBCPMzlhXMc8m5zOxIkuYVwma92g2ieKaj97IgZEKBTCZrOF28aYIe2Rxk+fBwxrn7omJubEycxnn33G3r17w5edxuro0S5CITP2hbGxdHb1Devu6TlOYHBwXLWcLU5nIoFA50SXMWaTtW5Q7RNFtUdXTIztc79UR7zElJKSQiAQCLcDgQAul2vE8dbWVlwuF8nJyXR2djL4x79cT13ncrlobW0FYGBggO7ubpKSkgB47bXXSE1NJT4+fiz7FBGRKIsYEIsWLWL//v20tbXR29vLrl27SE1NDY/PmjWL+Ph46urqAKisrCQ1NZW4uDjcbjfV1dUAeL3e8Lq0tDS8Xi8A1dXVuN1u4uLiAHjzzTdxu93R3aWIiIxZxICYMWMG9913HytXruSmm24iOzub+fPnk5+fz9tvvw3Axo0bKSkpISMjg56eHlauXAnAo48+SkVFBZmZmRw8eJB7770XgHvuuYc333yTrKwsysrKKCwsDP+8xsbG8L0IERGZODZjzDgu2J+7xnsPwsTG8lrd4WH9V82ewdRz/FeOnovXNkdjstYNqn2iqPboOuN7ECIicn5SQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIilUQVEVVUVmZmZpKenU1paOmy8vr6e3NxcPB4P69evZ2BgAIDm5mby8vLIyMhgzZo1dHd3A9DR0cHq1atZsmQJeXl5BAIBAPr7+3n88ce56aabyMrK4ne/+1209ikiImMUMSD8fj+bN2+mrKwMr9dLeXk5DQ0NQ+asW7eOwsJCdu7ciTGGiooKAIqKilixYgU+n4+5c+eybds2ALZs2YLb7aampobly5dTXFwMwE9+8hPa29vZsWMHW7Zs4eGHH+bP7Fdmi4hMGhEDYt++fSxcuJCkpCQSEhLweDz4fL7weFNTE319fSxYsACA3NxcfD4fwWCQ2tpaPB7PkH6A3bt3k5OTA0B2djZ79uwhGAxSU1NDfn4+NpuNyy+/nOeff14BISIyQSIGREtLC06nM9x2uVz4/f4Rx51OJ36/n/b2dhwOB3a7fUj/6WvsdjsOh4O2tjY+/vhjamtrWbFiBbfccgutra3ExOg2iYjIRLBHmhAKhbDZbOG2MWZIe6Tx0+cBw9qnromJiWFwcJBPP/2U0tJS3nvvPb75zW9SU1NDYmLiqDc0fbpj1HNP1dLWQ6LjgmH9CQnxOJMTxnXMs8npHP1rdC6ZrHWDap8oqv3siRgQKSkpHDx4MNwOBAK4XK4h4ydvMgO0trbicrlITk6ms7OTwcFBYmNjh6xzuVy0traSkpLCwMAA3d3dJCUl8YUvfIGsrCxsNhtXXHEFKSkpfPTRR8yfP3/UGzp6tItQaByXpWJj6ezqG9bd03OcwODg2I93FjmdiQQCnRNdxphN1rpBtU8U1R5dMTG2z/1SHfH6zaJFi9i/fz9tbW309vaya9cuUlNTw+OzZs0iPj6euro6ACorK0lNTSUuLg632011dTUAXq83vC4tLQ2v1wtAdXU1brebuLg4Fi9eHJ7f2NjIkSNHuPTSS8e5dRERORM2M4q7wFVVVWzfvp1gMMiyZcvIz88nPz+ftWvXMm/ePN599102bNhAV1cXc+bMoaSkhClTptDU1ERBQQFHjx5l5syZbNq0iWnTpnHs2DEKCgpobGwkMTGRjRs3cskll9DV1cX3vvc9/vu//xuABx54gMWLF49pQ+M9gzCxsbxWd3hY/1WzZzA1PuKJ1oQ6F7+ZjMZkrRtU+0RR7dEV6QxiVAExmSggJo/JWjeo9omi2qPrjC8xiYjI+UkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYmlUAVFVVUVmZibp6emUlpYOG6+vryc3NxePx8P69esZGBgAoLm5mby8PDIyMlizZg3d3d0AdHR0sHr1apYsWUJeXh6BQACApqYmrrzySpYuXcrSpUu56667orVPEREZo4gB4ff72bx5M2VlZXi9XsrLy2loaBgyZ926dRQWFrJz506MMVRUVABQVFTEihUr8Pl8zJ07l23btgGwZcsW3G43NTU1LF++nOLiYgDeeecdcnJyqKyspLKykueeey7a+xURkVGKGBD79u1j4cKFJCUlkZCQgMfjwefzhcebmpro6+tjwYIFAOTm5uLz+QgGg9TW1uLxeIb0A+zevZucnBwAsrOz2bNnD8FgkLfffpv333+fpUuXsnLlSt57772ob1hEREbHHmlCS0sLTqcz3Ha5XBw6dGjEcafTid/vp729HYfDgd1uH9J/+hq73Y7D4aCtrY34+HhuvPFGbr31Vl5//XW+/e1vU11dzZQpU0a9oenTHaOeO2SfbT0kOi4Y1p+QEI8zOWFcxzybnM7EiS5hXCZr3aDaJ4pqP3siBkQoFMJms4Xbxpgh7ZHGT58HDGufuiYmJobvfOc74b60tDR++MMf8uGHH3LFFVeMekNHj3YRCplRzw+LjaWzq29Yd0/PcQKDg2M/3lnkdCYSCHROdBljNlnrBtU+UVR7dMXE2D73S3XES0wpKSnhm8gAgUAAl8s14nhraysul4vk5GQ6OzsZ/ONfrqeuc7lctLa2AjAwMEB3dzdJSUm8+OKLtLe3h49ljAmfgYiIyNkVMSAWLVrE/v37aWtro7e3l127dpGamhoenzVrFvHx8dTV1QFQWVlJamoqcXFxuN1uqqurAfB6veF1aWlpeL1eAKqrq3G73cTFxVFbW8vLL78MwBtvvEEoFOKyyy6L7o5FRGRUbMaYiNdjqqqq2L59O8FgkGXLlpGfn09+fj5r165l3rx5vPvuu2zYsIGuri7mzJlDSUkJU6ZMoampiYKCAo4ePcrMmTPZtGkT06ZN49ixYxQUFNDY2EhiYiIbN27kkksuwe/3U1BQQCAQID4+nuLi4jFdXoLxX2IysbG8Vnd4WP9Vs2cwNf7cPos5F09dR2Oy1g2qfaKo9uiKdIlpVAExmSggJo/JWjeo9omi2qPrjO9BiIjI+UkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpZGFRBVVVVkZmaSnp5OaWnpsPH6+npyc3PxeDysX7+egYEBAJqbm8nLyyMjI4M1a9bQ3d0NQEdHB6tXr2bJkiXk5eURCASGHK+rq4vrr7+eAwcOnOn+RERknCIGhN/vZ/PmzZSVleH1eikvL6ehoWHInHXr1lFYWMjOnTsxxlBRUQFAUVERK1aswOfzMXfuXLZt2wbAli1bcLvd1NTUsHz5coqLi4cc77HHHqOjoyNaexQRkXGIGBD79u1j4cKFJCUlkZCQgMfjwefzhcebmpro6+tjwYIFAOTm5uLz+QgGg9TW1uLxeIb0A+zevZucnBwAsrOz2bNnD8FgEIDq6mqmTp3KV77ylejuVERExiRiQLS0tOB0OsNtl8uF3+8fcdzpdOL3+2lvb8fhcGC324f0n77GbrfjcDhoa2ujubmZn/3sZzz44IPR2Z2IiIybPdKEUCiEzWYLt40xQ9ojjZ8+DxjWPnUNwPr16/nud7/LBRdcMLZdnGL6dMe41rW09ZDoGP5zExLicSYnjLues8XpTJzoEsZlstYNqn2iqPazJ2JApKSkcPDgwXA7EAjgcrmGjJ96k7m1tRWXy0VycjKdnZ0MDg4SGxs7ZJ3L5aK1tZWUlBQGBgbo7u6mvb2dDz/8kPXr1wNw+PBhNmzYwGOPPcbChQtHvaGjR7sIhcyo54fFxtLZ1Tesu6fnOIHBwbEf7yxyOhMJBDonuowxm6x1g2qfKKo9umJibJ/7pTriJaZFixaxf/9+2tra6O3tZdeuXaSmpobHZ82aRXx8PHV1dQBUVlaSmppKXFwcbreb6upqALxeb3hdWloaXq8XOHHPwe12c8UVV/Daa69RWVlJZWUlc+fO5fHHHx9TOIiISPREDIgZM2Zw3333sXLlSm666Says7OZP38++fn5vP322wBs3LiRkpISMjIy6OnpYeXKlQA8+uijVFRUkJmZycGDB7n33nsBuOeee3jzzTfJysqirKyMwsLCP+EWRURkPGzm5A2APxPjvcRkYmN5re7wsP6rZs9ganzEK3ET6lw8dR2NyVo3qPaJotqj64wvMYmIyPlJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJiSQEhIiKWRhUQVVVVZGZmkp6eTmlp6bDx+vp6cnNz8Xg8rF+/noGBAQCam5vJy8sjIyODNWvW0N3dDUBHRwerV69myZIl5OXlEQgEAGhpaeHOO+/kxhtvZPny5dTX10drnyIiMkYRA8Lv97N582bKysrwer2Ul5fT0NAwZM66desoLCxk586dGGOoqKgAoKioiBUrVuDz+Zg7dy7btm0DYMuWLbjdbmpqali+fDnFxcUAbN68GY/HwyuvvMJ3vvMdioqKor1fEREZpYgBsW/fPhYuXEhSUhIJCQl4PB58Pl94vKmpib6+PhYsWABAbm4uPp+PYDBIbW0tHo9nSD/A7t27ycnJASA7O5s9e/YQDAYpLi7mlltuAeCTTz7hoosuiu5uRURk1OyRJrS0tOB0OsNtl8vFoUOHRhx3Op34/X7a29txOBzY7fYh/aevsdvtOBwO2tramDFjBgAZGRk0NTWFzzhEROTsixgQoVAIm80WbhtjhrRHGj99HjCsfeqamJj/fzLj8/mor69n1apV1NTUkJSUNOoNTZ/uGPXcU7W09ZDouGBYf0JCPM7khHEd82xyOhMnuoRxmax1g2qfKKr97IkYECkpKRw8eDDcDgQCuFyuIeMnbzIDtLa24nK5SE5OprOzk8HBQWJjY4esc7lctLa2kpKSwsDAAN3d3SQlJbF7926uuuoqpk6dyuzZs/niF79IY2PjmALi6NEuQiEz6vlhsbF0dvUN6+7pOU5gcHDsxzuLnM5EAoHOiS5jzCZr3aDaJ4pqj66YGNvnfqmOeA9i0aJF7N+/n7a2Nnp7e9m1axepqanh8VmzZhEfH09dXR0AlZWVpKamEhcXh9vtprq6GgCv1xtel5aWhtfrBaC6uhq3201cXBw7duwI3+BuaGigtbWVyy67bJxbFxGRM2EzxkT8ul1VVcX27dsJBoMsW7aM/Px88vPzWbt2LfPmzePdd99lw4YNdHV1MWfOHEpKSpgyZQpNTU0UFBRw9OhRZs6cyaZNm5g2bRrHjh2joKCAxsZGEhMT2bhxI5dccgl+v59HHnmEQCBAfHw8Dz30EG63e0wbGu8ZhImN5bW6w8P6r5o9g6nxEU+0JtS5+M1kNCZr3aDaJ4pqj65IZxCjCojJRAExeUzWukG1TxTVHl1nfIlJRETOTwoIERGxpIAQERFLCggREbGkgBAREUsKCBERsaSAEBERSwoIERGxpIAQERFLCggREbGkgBAREUsKCBERsaSAEBERSwoIERGxpIAQERFLCggREbGkgBAREUsKCBERsaSAEBERS6MKiKqqKjIzM0lPT6e0tHTYeH19Pbm5uXg8HtavX8/AwAAAzc3N5OXlkZGRwZo1a+ju7gago6OD1atXs2TJEvLy8ggEAgC0tLRw1113sXTpUm6++Wb2798frX2KiMgYRQwIv9/P5s2bKSsrw+v1Ul5eTkNDw5A569ato7CwkJ07d2KMoaKiAoCioiJWrFiBz+dj7ty5bNu2DYAtW7bgdrupqalh+fLlFBcXA/Dkk09y7bXXUllZyQ9/+EMeeOABBgcHo71nEREZhYgBsW/fPhYuXEhSUhIJCQl4PB58Pl94vKmpib6+PhYsWABAbm4uPp+PYDBIbW0tHo9nSD/A7t27ycnJASA7O5s9e/YQDAa54YYbyM7OBuAv//IvOX78OD09PdHdsYiIjIo90oSWlhacTme47XK5OHTo0IjjTqcTv99Pe3s7DocDu90+pP/0NXa7HYfDQVtbWzhMAJ577jlmz55NYmLimDY0fbpjTPPD+2jrIdFxwbD+hIR4nMkJ4zrm2eR0ju11OldM1rpBtU8U1X72RAyIUCiEzWYLt40xQ9ojjZ8+DxjWPnVNTMz/P5l54YUXKC8v59///d9Hv5M/Onq0i1DIjHkdsbF0dvUN6+7pOU7gHL/M5XQmEgh0TnQZYzZZ6wbVPlFUe3TFxNg+90t1xEtMKSkp4ZvIAIFAAJfLNeJ4a2srLpeL5ORkOjs7w/cQTl3ncrlobW0FYGBggO7ubpKSkoAT9yFeeuklSktLmTlz5lj2KiIiURQxIBYtWsT+/ftpa2ujt7eXXbt2kZqaGh6fNWsW8fHx1NXVAVBZWUlqaipxcXG43W6qq6sB8Hq94XVpaWl4vV4AqqurcbvdxMXF8cILL3DgwAF+8YtfkJKSEvXNiojI6NmMMRGvx1RVVbF9+3aCwSDLli0jPz+f/Px81q5dy7x583j33XfZsGEDXV1dzJkzh5KSEqZMmUJTUxMFBQUcPXqUmTNnsmnTJqZNm8axY8coKCigsbGRxMRENm7cyKxZs/jqV7+Kw+HgoosuCv/sZ599lhkzZox6Q+O9xGRiY3mt7vCw/qtmz2BqfMQrcRPqXDx1HY3JWjeo9omi2qMr0iWmUQXEZKKAmDwma92g2ieKao+uM74HISIi5ycFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFgaVUBUVVWRmZlJeno6paWlw8br6+vJzc3F4/Gwfv16BgYGAGhubiYvL4+MjAzWrFlDd3c3AB0dHaxevZolS5aQl5dHIBAYcry9e/dyxx13nOneRETkDEQMCL/fz+bNmykrK8Pr9VJeXk5DQ8OQOevWraOwsJCdO3dijKGiogKAoqIiVqxYgc/nY+7cuWzbtg2ALVu24Ha7qampYfny5RQXFwMQCoX46U9/yv33308oFIr2XkVEZAwiBsS+fftYuHAhSUlJJCQk4PF48Pl84fGmpib6+vpYsGABALm5ufh8PoLBILW1tXg8niH9ALt37yYnJweA7Oxs9uzZQzAY5IMPPuCDDz7gsccei/pGRURkbCIGREtLC06nM9x2uVz4/f4Rx51OJ36/n/b2dhwOB3a7fUj/6WvsdjsOh4O2tjYuv/xyiouLmTZtWnR2JyIi42aPNCEUCmGz2cJtY8yQ9kjjp88DhrVPXRMTE5375dOnO8a1rqWth0THBcP6ExLicSYnnGlZf3JOZ+JElzAuk7VuUO0TRbWfPREDIiUlhYMHD4bbgUAAl8s1ZPzUm8ytra24XC6Sk5Pp7OxkcHCQ2NjYIetcLhetra2kpKQwMDBAd3c3SUlJUdnQ0aNdhEJm7AtjY+ns6hvW3dNznMDgYBQq+9NxOhMJBDonuowxm6x1g2qfKKo9umJibJ/7pTri1/ZFixaxf/9+2tra6O3tZdeuXaSmpobHZ82aRXx8PHV1dQBUVlaSmppKXFwcbreb6upqALxeb3hdWloaXq8XgOrqatxuN3FxcePfpYiIRF3EgJgxYwb33XcfK1eu5KabbiI7O5v58+eTn5/P22+/DcDGjRspKSkhIyODnp4eVq5cCcCjjz5KRUUFmZmZHDx4kHvvvReAe+65hzfffJOsrCzKysooLCz8E25RRETGw2aMGcf1mHPXeC8xmdhYXqs7PKz/qtkzmBof8UrchDoXT11HY7LWDap9oqj26DrjS0wiInJ+UkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYUkCIiIglBYSIiFhSQIiIiCUFhIiIWFJAiIiIJQWEiIhYGlVAVFVVkZmZSXp6OqWlpcPG6+vryc3NxePxsH79egYGBgBobm4mLy+PjIwM1qxZQ3d3NwAdHR2sXr2aJUuWkJeXRyAQAKC/v59169axZMkSbr75Zj744INo7XPMggOhIe33DrezzfsOP/e9S9Xej/C39UxQZSIiZ0fEgPD7/WzevJmysjK8Xi/l5eU0NDQMmbNu3ToKCwvZuXMnxhgqKioAKCoqYsWKFfh8PubOncu2bdsA2LJlC263m5qaGpYvX05xcTEAL774IhdeeCE1NTU88sgjPPzww9Heb0SDIcP/fT/AL3/9e3bVNvLRkQ5q/s/HPPmL/+Ldj9s5+F6AHa9/xKPPv8HrbzVjjBnxWJ91HefXBxvZVPEm3tc/5Hj/4FnciYjImbGZz/sbDtixYwe1tbU88cQTADzzzDMYY7j77rsBaGpq4o477uDXv/41AAcPHmTr1q0899xzfO1rX+ONN97Abrdz5MgRbrvtNn7zm99w7bXXUlpaysyZMxkYGOCrX/0qBw4cYNWqVdxzzz243W4Arr/+en7+85/zxS9+cdQbam/vJhT63C0N89qbzfjbe+noOY6/rZfPuo4zc/pU2jr6OB488Zf6nEuTWfq/LyU+zs5n3f1493zAh80dpEyfyuBgiN7jAyRcYCfJEc/AYIi2juN81nUcA1ycGE9753EumjqFeV+eTqCth5ZjfTgujOMLSRcyNd4ONuvabDYbsTYbMTE2bDEQG2Mj5o/t4MAgza09BD7rJWGKnS9+YSpJjikMhAyDg4bYWBv22BhsNsCAATAGc/KfOfk62bDBiXm2E/98sp7BwRADg+aPtZyox2Y7OXzq/xOeY7GJ4V1AQsIUenr6hw+cayzeTpa1T7RRvHY2bGdce8gYevoG6O7tx2azkZgwhQumxGKMYdAYQiFDKASh0Kltw2DIEGeP4cL4WKbExf7xXTc2CQlxdE/Q6z6eek914YVxdHcfD3/+DH/8LBpzyufKRsxpn8OBwRADAyfehHb7iRqOdR7nWFc/8XGxXJwYz+y/vJgvz5o25ppiYmxcfPHUEcftkQ7Q0tKC0+kMt10uF4cOHRpx3Ol04vf7aW9vx+FwYLfbh/SfvsZut+NwOGhra7M81qeffjqmgPi8zY4k97r/Nab5lwBz/soZcZ6IyGQW8RJTKBTCdso3wJNpF2n89HnAsPapa2JiYoatOdkvIiJnX8S/fVNSUsI3kQECgQAul2vE8dbWVlwuF8nJyXR2djI4ODhsncvlorW1FYCBgQG6u7tJSkpixowZtLS0DDuWiIicfREDYtGiRezfv5+2tjZ6e3vZtWsXqamp4fFZs2YRHx9PXV0dAJWVlaSmphIXF4fb7aa6uhoAr9cbXpeWlobX6wWguroat9tNXFwcaWlpVFZWAifuZcTHx4/p8pKIiERPxJvUcOJfc92+fTvBYJBly5aRn59Pfn4+a9euZd68ebz77rts2LCBrq4u5syZQ0lJCVOmTKGpqYmCggKOHj3KzJkz2bRpE9OmTePYsWMUFBTQ2NhIYmIiGzdu5JJLLuH48eMUFhbyzjvvMGXKFB5//HHmzJlzNl4HERE5zagCQkREzj+6AywiIpYUECIiYkkBISIilhQQIiJi6bwPiEgPIjzburq6yM7O5pNPPgFg37595OTkkJ6ezubNm8PzovWAxM9i/18AAAl7SURBVGh4+umnycrKIisriyeffHLS1A3wox/9iMzMTLKysnj++ecnVe0n/eAHP6CgoCCqNf6pH5x5++23k5WVxdKlS1m6dClvvfXWiJ/FaP15RMtvf/tbcnNzWbJkCY8//nhUazxb75lRM+exTz/91CxevNi0t7eb7u5uk5OTY37/+99PWD1vvvmmyc7ONnPmzDGNjY2mt7fXpKWlmcOHD5tgMGhWrVpldu/ebYwxJisry/zXf/2XMcaYhx9+2JSWlhpjjFm9erV59dVXjTHGPP300+bJJ580xhhTVFRktm/fbowxZseOHeaee+6JSs179+41t9xyizl+/Ljp7+83K1euNFVVVed83cYYc+DAAXPrrbeaYDBoent7zeLFi019ff2kqP2kffv2ma997WvmoYceimqNP/nJT8x3v/tdY4wxb7zxhlm+fHnUag6FQuaaa64xwWAw3DfSZzGan4FoOHz4sLnmmmvMkSNHTH9/v/n7v/97s3v37kn1nhmL8zogfvWrX5mHH3443H766afNU089NWH1PPLII6a2ttYsXrzYNDY2mgMHDpiVK1eGx3fs2GEKCgrMJ598Yq677rpwf21trbn99ttNf3+/ufLKK8MfvObmZnPttdcaY4xZvHixaW5uNsYYEwwGzZVXXmn6+/vPuOb3338//AEw5sQb/Kmnnjrn6z7p5LE++eQTk5qaOile85Pa29vN8uXLzfPPP28eeuihqNZ42223mdra2vCxrrvuOtPU1BSVuhsaGsw111xjbr/9dpOTk2NefPHFET+L0fzziIbnnnvOPPHEE+H2p59+OqneM2N1Xl9isnoQ4ckHCk6E4uLi8JNsYeT6ovmAxDN1+eWXs2DBAgD+8Ic/UFNTg81mO+frPikuLo6tW7eSlZXF1VdfPSle85MKCwu57777uOiii4b9vDOtcaQHZ0ZDR0cHV199Nc888wwvvPACv/zlL2lubh7V634mfx7R8PHHHzM4OMi3vvUtli5dSllZ2aR6z4zVeR0QkR5EONFGqi+aD0iMlt///vesWrWKBx98kL/4i7+YNHUDrF27lv3793PkyBH+8Ic/TIraX3rpJWbOnMnVV18d7pssD8688sorefLJJ0lMTCQ5OZlly5axdevWMb3u0djreAwODrJ//36eeOIJysvLOXToEI2NjZPiPTMeER/3/ecsJSWFgwcPhtunP4hwoo30oMTRPCAxNjbW8gGJKSkpQx6QGA11dXWsXbuWRx55hKysLN54441JUfcHH3xAf38/s2fP5sILLyQ9PR2fz0dsbOw5X3t1dTWBQIClS5fy2Wef0dPTg81mi1qNJx+c+aUvfWnIsaLh4MGDBIPBcLgZY5g1a9ao3jNn8ucRDV/4whe4+uqrSU5OBk78zprJ8p4Zj/P6DCLSgwgn2l//9V/z0UcfhU9rX331VVJTU6P6gMQzdeTIEb797W+zceNGsrKyJk3dAJ988gkbNmygv7+f/v5+fvOb33DrrbdOitqff/55Xn31VSorK1m7di3XXnstJSUlk+LBmZ2dnTz55JMcP36crq4uduzYwb/8y79Yfhaj+V6KhsWLF/O73/2Ojo4OBgcHef3118nIyJgU75lxOUv3Os5Zr7zyisnKyjLp6enm2WefnehyjDEmfJPamBP/lkpOTo5JT083xcXFJhQKGWOMqa+vN1//+teNx+Mx999/vzl+/Lgx5sTN1ttuu80sWbLErFq1yhw7dswYc+KG5j/+4z+azMxMc8stt4SPf6Yee+wxs2DBAnPjjTeG/1dWVnbO133S1q1bzZIlS0x2drbZunWrMebcf81P9x//8R/hf4spWjX29fWZBx980GRmZpqbbrrJvPPOO1GtefPmzSYjI8Okp6ebF154wRgz8mcxWn8e0fLSSy+F6ywqKjKDg4OT7j0zWnpYn4iIWDqvLzGJiMjIFBAiImJJASEiIpYUECIiYkkBISIilhQQIiM4dOgQhYWF4faqVatGfOxBfn4+DQ0NHDhwgOzs7DH/rJdeemlUTxNua2vj7rvvJicnh8zMTH7wgx8QCoXG/PNERkMBITKChoaGIc/x2bt374hz/+3f/o2/+qu/GvfPqquro6+vL+K8J554gi9/+ctUVVWxY8cODh06xK9+9atx/1yRz3NeP2pDBE48w+iJJ57grbfeoru7G2MMRUVFbN26lc7OTh5++OHw3DvuuINnn32WvLw85s+fz3vvvcf9999PSUkJP/rRjwDo6elh7dq1fPzxx1x00UV873vf49JLL6WgoIDLL7+cu+66CyDc/tKXvsRvf/tb9u7dywUXXEBeXh4//vGP2bVrF6FQiFmzZvHoo48yY8YMbrjhBv7mb/4GgPj4eC6//HKam5vP/osm5wWdQch576233qKlpYXy8nKqq6u5+eab+elPf8ratWtxu92UlJRQUlICwM9+9jNmzpwJnHiSbU1NDTfccMOQ4x05coQ777yTyspKsrOzefDBBz/3599www1ce+213HnnneTl5eH1enn//fd56aWXqKysJC0tjQ0bNgDg8XjCT/v8n//5H1599dVhP18kWnQGIee9K6+8kmnTpvHLX/6SxsZGDhw4wNSpUyOuO/XR7Kf6yle+Ev6Wf/PNN/PP//zPdHZ2jrqe//zP/+Ttt9/m61//OnDiDKe3t3fInNdff51169axYcMGZs+ePepji4yFAkLOe7t376a4uJh/+Id/4LrrruOyyy7jlVdeibguISHBsv/0xzPbbDbsdnv4Uc8nBYNBy/WhUIhvfvObrFixAjjx6z8/++yz8Pjzzz/Ps88+y6ZNm1i0aFHEOkXGS5eY5Ly3d+9eFi9ezIoVK5g7dy6//vWvw49iPvk7hIFh7ZG899571NfXA1BeXs7f/u3fcuGFF3LxxRfzzjvvAOD3+3njjTcsj33NNdfw8ssv09XVBZz4vdknL1OVlpZSWlpKRUWFwkH+5HQGIee9W2+9lX/6p38iJyeHgYEB/u7v/o5du3Yxf/58nnnmGe6++26efvppMjIyuP3223nqqac+93iXXXYZTz/9NI2NjUyfPp3vf//7ANx+++088MADeDweLrnkEhYuXBhek5qaGp6Xn5+P3+/nG9/4BjabjZkzZ/L973+f/v5+Nm7ciMPh4O677w6vPfmL70WiTU9zFRERS7rEJCIilhQQIiJiSQEhIiKWFBAiImJJASEiIpYUECIiYkkBISIilhQQIiJi6f8BK21cHuSAIEEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute2'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20d3eba8>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEJCAYAAABsc6siAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfXSU9Z3//2cyEwZCUmP8zSSI4OKenla5WdCwpRx2slVkQgikpEBZUqIrG1uOLcqu0QTYYHpEhI1gPRVr9+hxi8Em2Dpz0hMm7G4X3CM5ksQ7OBuDQFWawGSSiTU3ECaZ6/cHdb4MSTq5YiRiXo8j5/C5u67rfc0xr7muIdfEGIZhICIiYkLsaB+AiIhcexQeIiJimsJDRERMU3iIiIhpCg8RETFN4SEiIqYpPERExDTraB/A1dLe3kUoZP5XWm64IYG2ts4v4Ii+vFTz2KCax4bh1BwbG8P110/8i3PGTHiEQsawwuOztWONah4bVPPY8EXUrNtWIiJimsJDRERMU3iIiIhpQwqPyspKMjMzWbRoEWVlZf3GGxoayMnJweVysXnzZnp7ewFobm4mNzeXjIwM1q9fT1dXFwCnTp0iNzeX7Oxsvv/979PQ0ADAxYsXKSgoYPHixSxfvpxTp04BYBgGO3bsICMjg8zMTOrr60ekeBERGZ6o4eHz+di9ezf79u3D7XZTXl7OyZMnI+YUFBRQXFxMdXU1hmFQUVEBQElJCWvWrMHr9TJjxgz27NkDwJYtW8jPz8fj8fDQQw/x6KOPArB3714mTJjAgQMH2LRpE0VFRQBUV1dz6tQpqqqqePbZZykqKgoHlIiIXH1Rw+PIkSPMmzePpKQk4uPjcblceL3e8HhTUxMXLlxg9uzZAOTk5OD1egkGg9TW1uJyuSL6AVauXMnf/d3fAfCNb3yDs2fPAnDo0CGWLVsGwNy5cwkEAjQ3N3P48GEyMzOJjY1l2rRpTJo0ibfffnsET4OIiJgRNTxaWlqw2+3htsPhwOfzDTput9vx+Xy0t7eTkJCA1WqN6IdLQWKxWAB45plnWLhw4aDbOnfuHC0tLTgcjn79IiIyOqL+nkcoFCImJibcNgwjoj3Y+JXzgH7zdu7cybvvvsuvfvWrAbdtGAaxsbED7iM21txn/TfckGBq/mc6ui9i/DnoLjdhvJXE+HHD2ua1wG5PHO1DuOpU89igmkdG1PBITU2lrq4u3Pb7/RFXAampqfj9/nC7tbUVh8NBcnIyHR0d9PX1YbFYItb19vby6KOP4vP5+NWvfkVi4qXCUlJSaGlpYerUqRHbSk1NpaWlpd8+zGhr6xzWL8oYFguH6z/u1z/31hQudPWY3t61wG5PxO/vGO3DuKpU89igmocmNjYm6hvuqG/f58+fT01NDYFAgPPnz3Pw4EGcTmd4fPLkydhstvC/gPJ4PDidTuLi4khLS6OqqgoAt9sdXrdjxw46Ozt58cUXw8EBkJ6ejsfjAaCurg6bzcaNN96I0+mksrKSvr4+PvroIz788ENmzpxp6mSIiMjIiRnKd5hXVlby/PPPEwwGWbFiBfn5+eTn57NhwwZmzpzJ+++/z5YtW+js7GT69Ols376dcePG0dTURGFhIW1tbUyaNIldu3bR19fHggULuOmmm5gwYUJ4Hx6Ph56eHoqLizl+/Djjxo3j8ccfZ/r06eFbXK+//joARUVFLFiwwFShX8SVx0TbV/PpLnp3Njao5rHhi7ryGFJ4fBUoPIZO/4ONDap5bBi121YiIiJXUniIiIhpCg8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhpCg8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMG9JX4VVWVvLcc8/R29vLPffcQ25ubsR4Q0MDmzdvpquri7S0NEpKSrBarTQ3N1NQUEBbWxvTpk2jtLSUiRMnhtft37+f+vp6nnzySQB+9KMfcfbsWQBCoRAnTpzg1Vdf5Zvf/Cbf+ta3mDJlSnjtb3/7WywWy+c+ASIiYl7UKw+fz8fu3bvZt28fbreb8vJyTp48GTGnoKCA4uJiqqurMQyDiooKAEpKSlizZg1er5cZM2awZ88eAHp6eigtLeWJJ56I2M4vfvELPB4PHo+HhQsXsmrVKmbOnEljYyNz5swJj3k8HgWHiMgoihoeR44cYd68eSQlJREfH4/L5cLr9YbHm5qauHDhArNnzwYgJycHr9dLMBiktrYWl8sV0Q9QW1tLKBSioKBgwH2ePn0at9vNo48+CsCxY8cIBALk5OSwatUqjh49+vmqFhGRzyVqeLS0tGC328Nth8OBz+cbdNxut+Pz+WhvbychIQGr1RrRD7BgwQIeeeQRxo8fP+A+9+zZw7p160hIuPQF7DExMdx1112Ul5fz2GOPsXHjRgKBwDDKFRGRkRD1M49QKERMTEy4bRhGRHuw8SvnAf3aA/nTn/7EG2+8wbZt28J9q1evDv/9tttuY9asWbz11lssXLgw6vY+c8MNCUOee7mWQDeJCf1DLj7ehj05fljbvBbY7YmjfQhXnWoeG1TzyIgaHqmpqdTV1YXbfr8fh8MRMe73+8Pt1tZWHA4HycnJdHR00NfXh8Vi6bduMIcPH8bpdGKz2cJ9breb22+/nalTpwKXAiouLm5oFf5ZW1snoZBhag0AFgsdnRf6dXd39+Dv6zO/vWuA3Z6I398x2odxVanmsUE1D01sbEzUN9xRb1vNnz+fmpoaAoEA58+f5+DBgzidzvD45MmTsdls1NfXA+DxeHA6ncTFxZGWlkZVVRVwKQAuXzeYd955h7S0tIi+xsZGXnzxReDS5yENDQ3ccccdUbclIiJfjKjhkZKSwsaNG8nLy+O73/0uWVlZzJo1i/z8fI4dOwZAaWkp27dvJyMjg+7ubvLy8gDYunUrFRUVZGZmUldXx0MPPRT1gM6cOUNKSkpE3wMPPEAgECArK4sHH3yQHTt2hD8PERGRqy/GMIxh3Mu59gz3tpVhsXC4/uN+/XNvTWGibUi/JnPN0aX92KCax4ZRu20lIiJyJYWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMU3hISIipik8RETENIWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMU3hISIipik8RETEtCGFR2VlJZmZmSxatIiysrJ+4w0NDeTk5OByudi8eTO9vb0ANDc3k5ubS0ZGBuvXr6erqyti3f79+yksLAy3m5qamDNnDtnZ2WRnZ7Nu3ToALl68SEFBAYsXL2b58uWcOnVq2AWLiMjnFzU8fD4fu3fvZt++fbjdbsrLyzl58mTEnIKCAoqLi6mursYwDCoqKgAoKSlhzZo1eL1eZsyYwZ49ewDo6emhtLSUJ554ImI7x48fZ+nSpXg8HjweDy+88AIAe/fuZcKECRw4cIBNmzZRVFQ0IsWLiMjwRA2PI0eOMG/ePJKSkoiPj8flcuH1esPjTU1NXLhwgdmzZwOQk5OD1+slGAxSW1uLy+WK6Aeora0lFApRUFAQsa9jx45x4sQJsrOzycvLo7GxEYBDhw6xbNkyAObOnUsgEKC5uXkEyhcRkeGIGh4tLS3Y7fZw2+Fw4PP5Bh232+34fD7a29tJSEjAarVG9AMsWLCARx55hPHjx0fsy2azsWzZMl577TXWrVvHAw88wMWLFwfcx7lz54ZZsoiIfF7WaBNCoRAxMTHhtmEYEe3Bxq+cB/RrX+knP/lJ+O/p6ek89dRTnD59ut+2DMMgNtbcZ/033JBgav5nWgLdJCaM79cfH2/Dnhw/rG1eC+z2xNE+hKtONY8NqnlkRA2P1NRU6urqwm2/34/D4YgY9/v94XZraysOh4Pk5GQ6Ojro6+vDYrH0WzeQvXv3kpWVxfXXXw9cCgmr1UpKSgotLS1MnTo1Yh9mtLV1EgoZptYAYLHQ0XmhX3d3dw/+vj7z27sG2O2J+P0do30YV5VqHhtU89DExsZEfcMd9e37/PnzqampIRAIcP78eQ4ePIjT6QyPT548GZvNRn19PQAejwen00lcXBxpaWlUVVUB4Ha7I9YNpLa2lldffRWAo0ePEgqFuOWWW0hPT8fj8QBQV1eHzWbjxhtvjHboIiLyBYkaHikpKWzcuJG8vDy++93vkpWVxaxZs8jPz+fYsWMAlJaWsn37djIyMuju7iYvLw+ArVu3UlFRQWZmJnV1dTz00EN/cV+bN2/myJEjZGVlsWPHDp566iliY2NZu3YtFy9eZMmSJWzbto2dO3eOQOkiIjJcMYZhDONezrVnuLetDIuFw/Uf9+ufe2sKE21R7/pdk3RpPzao5rFh1G5biYiIXEnhISIipik8RETENIWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMU3hISIipik8RETENIWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMW1I4VFZWUlmZiaLFi2irKys33hDQwM5OTm4XC42b95Mb28vAM3NzeTm5pKRkcH69evp6uqKWLd//34KCwvD7ZaWFtatW0d2djbLly+npqYGgGAwyO233052dnb4T19f37CLFhGRzydqePh8Pnbv3s2+fftwu92Ul5dz8uTJiDkFBQUUFxdTXV2NYRhUVFQAUFJSwpo1a/B6vcyYMYM9e/YA0NPTQ2lpKU888UTEdnbu3Mmdd96Jx+Phqaee4uGHH6avr4/GxkbmzJmDx+MJ/7FYLCN1DkRExKSo4XHkyBHmzZtHUlIS8fHxuFwuvF5veLypqYkLFy4we/ZsAHJycvB6vQSDQWpra3G5XBH9ALW1tYRCIQoKCiL2dffdd5OVlQXAzTffTE9PD93d3Rw7doxAIEBOTg6rVq3i6NGjI1O9iIgMS9TwaGlpwW63h9sOhwOfzzfouN1ux+fz0d7eTkJCAlarNaIfYMGCBTzyyCOMHz8+Yl8ul4vrrrsOgBdeeIFbb72VxMREYmJiuOuuuygvL+exxx5j48aNBAKBz1G2iIh8HtZoE0KhEDExMeG2YRgR7cHGr5wH9GsP5qWXXqK8vJyXX34ZgNWrV4fHbrvtNmbNmsVbb73FwoULh7Q9gBtuSBjy3Mu1BLpJTBjfrz8+3oY9OX5Y27wW2O2Jo30IV51qHhtU88iIGh6pqanU1dWF236/H4fDETHu9/vD7dbWVhwOB8nJyXR0dNDX14fFYum3bjA7d+7k8OHDlJWVkZqaCoDb7eb2229n6tSpwKWAiouLG3qVQFtbJ6GQYWoNABYLHZ0X+nV3d/fg/4p+aG+3J+L3d4z2YVxVqnlsUM1DExsbE/UNd9TbVvPnz6empoZAIMD58+c5ePAgTqczPD558mRsNhv19fUAeDwenE4ncXFxpKWlUVVVBVwKgMvXDeSll17izTff5JVXXgkHB0BjYyMvvvgiAKdPn6ahoYE77rgj2qGLiMgXJOqVR0pKChs3biQvL49gMMiKFSuYNWsW+fn5bNiwgZkzZ1JaWsqWLVvo7Oxk+vTp5OXlAbB161YKCwt57rnnmDRpErt27Rp0P4Zh8Oyzz5KQkMDatWvD/b/85S954IEH2LRpE1lZWcTExLBjxw4SEoZ3G0pERD6/GMMwhnEv59oz3NtWhsXC4fqP+/XPvTWFibao2XtN0qX92KCax4ZRu20lIiJyJYWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMU3hISIipik8RETENIWHiIiYpvAQERHTFB4iImKawkNERExTeIiIiGkKDxERMU3hISIipik8RETEtCGFR2VlJZmZmSxatIiysrJ+4w0NDeTk5OByudi8eTO9vb0ANDc3k5ubS0ZGBuvXr6erqyti3f79+yksLAy3L168SEFBAYsXL2b58uWcOnUKuPQVtTt27CAjI4PMzMzw96WLiMjoiBoePp+P3bt3s2/fPtxuN+Xl5Zw8eTJiTkFBAcXFxVRXV2MYBhUVFQCUlJSwZs0avF4vM2bMYM+ePQD09PRQWlrKE088EbGdvXv3MmHCBA4cOMCmTZsoKioCoLq6mlOnTlFVVcWzzz5LUVFROKBEROTqixoeR44cYd68eSQlJREfH4/L5cLr9YbHm5qauHDhArNnzwYgJycHr9dLMBiktrYWl8sV0Q9QW1tLKBSioKAgYl+HDh1i2bJlAMydO5dAIEBzczOHDx8mMzOT2NhYpk2bxqRJk3j77bdH5gyIiIhpUcOjpaUFu90ebjscDnw+36Djdrsdn89He3s7CQkJWK3WiH6ABQsW8MgjjzB+/Pi/uC+73c65c+doaWnB4XD06xcRkdFhjTYhFAoRExMTbhuGEdEebPzKeUC/9pWuXGMYBrGxsQPuIzbW3Gf9N9yQYGr+Z1oC3SQmjO/XHx9vw54cP6xtXgvs9sTRPoSrTjWPDap5ZEQNj9TUVOrq6sJtv98fcRWQmpqK3+8Pt1tbW3E4HCQnJ9PR0UFfXx8Wi6XfuoGkpKTQ0tLC1KlTI7aVmppKS0tLv32Y0dbWSShkmFoDgMVCR+eFft3d3T34+/rMb+8aYLcn4vd3jPZhXFWqeWxQzUMTGxsT9Q131Lfv8+fPp6amhkAgwPnz5zl48CBOpzM8PnnyZGw2W/hfQHk8HpxOJ3FxcaSlpVFVVQWA2+2OWDeQ9PR0PB4PAHV1ddhsNm688UacTieVlZX09fXx0Ucf8eGHHzJz5sxohy4iIl+QqFceKSkpbNy4kby8PILBICtWrGDWrFnk5+ezYcMGZs6cSWlpKVu2bKGzs5Pp06eTl5cHwNatWyksLOS5555j0qRJ7Nq16y/ua+3atRQXF7NkyRLGjRvHzp07AcjIyOC9994Lf5i+bdu2fp+XiIjI1RNjGMYw7uVce4Z728qwWDhc/3G//rm3pjDRFjV7r0m6tB8bVPPYMGq3rURERK6k8BAREdMUHiIiYprCQ0RETFN4iIiIaQoPERExTeEhIiKmKTxERMQ0hYeIiJim8BAREdMUHiIiYprCQ0RETFN4iIiIaQoPERExTeEhIiKmKTxERMQ0hYeIiJg2pK/Cq6ys5LnnnqO3t5d77rmH3NzciPGGhgY2b95MV1cXaWlplJSUYLVaaW5upqCggLa2NqZNm0ZpaSkTJ07k008/5eGHH+bMmTMkJyfz9NNPY7fb+dGPfsTZs2cBCIVCnDhxgldffZVvfvObfOtb32LKlCnhff72t7/FYrGM4KkQEZGhinrl4fP52L17N/v27cPtdlNeXs7Jkycj5hQUFFBcXEx1dTWGYVBRUQFASUkJa9aswev1MmPGDPbs2QPA008/TVpaGgcOHGDlypVs27YNgF/84hd4PB48Hg8LFy5k1apVzJw5k8bGRubMmRMe83g8Cg4RkVEUNTyOHDnCvHnzSEpKIj4+HpfLhdfrDY83NTVx4cIFZs+eDUBOTg5er5dgMEhtbS0ulyuiH+DQoUMsXboUgKysLF5//XWCwWB4m6dPn8btdvPoo48CcOzYMQKBADk5OaxatYqjR4+OUPkiIjIcUcOjpaUFu90ebjscDnw+36Djdrsdn89He3s7CQkJWK3WiP4r11itVhISEggEAuFt7Nmzh3Xr1pGQcOkL2GNiYrjrrrsoLy/nscceY+PGjRHzRUTk6or6mUcoFCImJibcNgwjoj3Y+JXzgH7ty9fExl7KsT/96U+88cYb4VtZAKtXrw7//bbbbmPWrFm89dZbLFy4MNrhh91wQ8KQ516uJdBNYsL4fv3x8TbsyfHD2ua1wG5PHO1DuOpU89igmkdG1PBITU2lrq4u3Pb7/Tgcjohxv98fbre2tuJwOEhOTqajo4O+vj4sFkvEOofDQWtrK6mpqfT29tLV1UVSUhIAhw8fxul0YrPZwtt0u93cfvvtTJ06FbgUNnFxcaYKbWvrJBQyTK0BwGKho/NCv+7u7h78fX3mt3cNsNsT8fs7RvswrirVPDao5qGJjY2J+oY76m2r+fPnU1NTQyAQ4Pz58xw8eBCn0xkenzx5Mjabjfr6egA8Hg9Op5O4uDjS0tKoqqoCLgXAZ+vS09Nxu90AVFVVkZaWFg6Dd955h7S0tIhjaGxs5MUXXwQufR7S0NDAHXfcMaSTICIiIy9qeKSkpLBx40by8vL47ne/S1ZWFrNmzSI/P59jx44BUFpayvbt28nIyKC7u5u8vDwAtm7dSkVFBZmZmdTV1fHQQw8B8OCDD/LOO++wZMkS9u3bR3FxcXh/Z86cISUlJeIYHnjgAQKBAFlZWTz44IPs2LEj/HmIiIhcfTGGYQzjXs61Z7i3rQyLhcP1H/frn3trChNtQ/o1mWuOLu3HBtU8NozabSsREZErKTxERMQ0hYeIiJim8BAREdMUHiIiYprCQ0RETFN4iIiIaQoPERExTeEhIiKmKTxERMQ0hYeIiJim8BAREdMUHiIiYprCQ0RETFN4iIiIaQoPERExTeEhIiKmDSk8KisryczMZNGiRZSVlfUbb2hoICcnB5fLxebNm+nt7QWgubmZ3NxcMjIyWL9+PV1dXQB8+umn3H///SxevJjc3Fz8fj8ATU1NzJkzh+zsbLKzs1m3bh0AFy9epKCggMWLF7N8+XJOnTo1IsWLiMjwRA0Pn8/H7t272bdvH263m/Lyck6ePBkxp6CggOLiYqqrqzEMg4qKCgBKSkpYs2YNXq+XGTNmsGfPHgCefvpp0tLSOHDgACtXrmTbtm0AHD9+nKVLl+LxePB4PLzwwgsA7N27lwkTJnDgwAE2bdpEUVHRiJ4EERExJ2p4HDlyhHnz5pGUlER8fDwulwuv1xseb2pq4sKFC8yePRuAnJwcvF4vwWCQ2tpaXC5XRD/AoUOHWLp0KQBZWVm8/vrrBINBjh07xokTJ8jOziYvL4/Gxsbw/GXLlgEwd+5cAoEAzc3NI3gaRETEDGu0CS0tLdjt9nDb4XDw3nvvDTput9vx+Xy0t7eTkJCA1WqN6L9yjdVqJSEhgUAggM1mY9myZaxevZr//d//5YEHHqCqqmrAfZw7d44bb7xxyIVG+zL3QesPdJOYML5ff3y8DXty/LC2eS2w2xNH+xCuOtU8NqjmkRE1PEKhEDExMeG2YRgR7cHGr5wH9GtfviY2Npaf/OQn4b709HSeeuopTp8+3W9bn803o62tk1DIMLUGAIuFjs4L/bq7u3vw9/WZ3941wG5PxO/vGO3DuKpU89igmocmNjYm6hvuqD+BU1NTwx9oA/j9fhwOx6Djra2tOBwOkpOT6ejooO/PP2AvX+dwOGhtbQWgt7eXrq4ukpKS2Lt3L+3t7eFtGYaB1WolJSWFlpaWfvsQEZHRETU85s+fT01NDYFAgPPnz3Pw4EGcTmd4fPLkydhsNurr6wHweDw4nU7i4uJIS0ujqqoKALfbHV6Xnp6O2+0GoKqqirS0NOLi4qitreXVV18F4OjRo4RCIW655RbS09PxeDwA1NXVYbPZTN2yEhGRkRVjGEbUezmVlZU8//zzBINBVqxYQX5+Pvn5+WzYsIGZM2fy/vvvs2XLFjo7O5k+fTrbt29n3LhxNDU1UVhYSFtbG5MmTWLXrl1cd911fPLJJxQWFnLmzBkSExMpLS3lpptuwufzUVhYiN/vx2azsW3bNr75zW/S09NDcXExx48fZ9y4cTz++ONMnz7dVKHDvW1lWCwcrv+4X//cW1OYaIt61++apEv7sUE1jw1f1G2rIYXHV4HCY+j0P9jYoJrHhlH7zENERORKCg8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhpCg8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhpQwqPyspKMjMzWbRoEWVlZf3GGxoayMnJweVysXnzZnp7ewFobm4mNzeXjIwM1q9fT1dXFwCffvop999/P4sXLyY3Nxe/3w9AS0sL69atIzs7m+XLl1NTUwNAMBjk9ttvJzs7O/ynr69vRE6AiIiYFzU8fD4fu3fvZt++fbjdbsrLyzl58mTEnIKCAoqLi6mursYwDCoqKgAoKSlhzZo1eL1eZsyYwZ49ewB4+umnSUtL48CBA6xcuZJt27YBsHPnTu688048Hg9PPfUUDz/8MH19fTQ2NjJnzhw8Hk/4j8ViGelzISIiQxQ1PI4cOcK8efNISkoiPj4el8uF1+sNjzc1NXHhwgVmz54NQE5ODl6vl2AwSG1tLS6XK6If4NChQyxduhSArKwsXn/9dYLBIHfffTdZWVkA3HzzzfT09NDd3c2xY8cIBALk5OSwatUqjh49OrJnQURETLFGm9DS0oLdbg+3HQ4H77333qDjdrsdn89He3s7CQkJWK3WiP4r11itVhISEggEAuGgAXjhhRe49dZbSUxMJCYmhrvuuosf/vCHfPDBB+Tn51NZWUlycvKQC432Ze6D1h/oJjFhfL/++Hgb9uT4YW3zWmC3J472IVx1qnlsUM0jI2p4hEIhYmJiwm3DMCLag41fOQ/o1758TWzs/7sIeumllygvL+fll18GYPXq1eGx2267jVmzZvHWW2+xcOHCaIcf1tbWSShkDHl+mMVCR+eFft3d3T34v6Kfu9jtifj9HaN9GFeVah4bVPPQxMbGRH3DHfW2VWpqavgDbQC/34/D4Rh0vLW1FYfDQXJyMh0dHeEPti9f53A4aG1tBaC3t5euri6SkpKAS5977N+/n7KyMiZNmgSA2+3m448/Du/DMAzi4uKiHbqIiHxBoobH/PnzqampIRAIcP78eQ4ePIjT6QyPT548GZvNRn19PQAejwen00lcXBxpaWlUVVUBlwLgs3Xp6em43W4AqqqqSEtLIy4ujpdeeok333yTV155hdTU1PA+GhsbefHFFwE4ffo0DQ0N3HHHHSN0CkRExKwYwzCi3suprKzk+eefJxgMsmLFCvLz88nPz2fDhg3MnDmT999/ny1bttDZ2cn06dPZvn0748aNo6mpicLCQtra2pg0aRK7du3iuuuu45NPPqGwsJAzZ86QmJhIac0Gcx8AAA0ESURBVGkpkydP5m//9m9JSEjga1/7Wnjfv/zlL5k4cSKbNm3i9OnTxMTEsHnzZubNm2eq0OHetjIsFg7Xf9yvf+6tKUy0Rb3rd03Spf3YoJrHhi/qttWQwuOrQOExdPofbGxQzWPDqH3mISIiciWFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhpCg8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhpCg8RETFN4SEiIqYpPERExLQhhUdlZSWZmZksWrSIsrKyfuMNDQ3k5OTgcrnYvHkzvb29ADQ3N5Obm0tGRgbr16+nq6sLgE8//ZT777+fxYsXk5ubi9/vB+DixYsUFBSwePFili9fzqlTpwAwDIMdO3aQkZFBZmZm+PvSRURkdEQND5/Px+7du9m3bx9ut5vy8nJOnjwZMaegoIDi4mKqq6sxDIOKigoASkpKWLNmDV6vlxkzZrBnzx4Ann76adLS0jhw4AArV65k27ZtAOzdu5cJEyZw4MABNm3aRFFREQDV1dWcOnWKqqoqnn32WYqKisIB9WX2h7OfUvH7k/zfhwGG+22/PRf7ONvWNez1IiJfhKhfwn3kyBHmzZtHUlISAC6XC6/Xy49//GMAmpqauHDhArNnzwYgJyeHZ555hpUrV1JbW8uzzz4b7v/BD35AQUEBhw4dCl/BZGVl8dOf/pRgMMihQ4d48MEHAZg7dy6BQIDm5mYOHz5MZmYmsbGxTJs2jUmTJvH2228zd+7cIRcaGxtj4rRccvwPAc4Gumn2dxLsDdHe0UNH90Xix1tp7+jh/7tuAgYGf/4P/vwD3jDg9NlPOXHmEwDe+sDPzZO+xjduuo4LF/sI9oWwxVmYMM4CQG/IwDAMLLGxWGJjuNjbR08wRJO/k4/OddAXMkhKsDF9WjIJE6wEew1CIYO4uFjirJfl/2fHcdmx9BPz/85DvzPy546J8ePo6r5ITP8ZX1kTJ7bR1XVxtA/jqlLNX31THBOx2xNN//wbyvyo4dHS0oLdbg+3HQ4H77333qDjdrsdn89He3s7CQkJWK3WiP4r11itVhISEggEAgNu69y5c7S0tOBwOPr1m3H99RNNzQdIj/IF8CIi14IbvoCfZVFvW4VCIWIue7dqGEZEe7DxK+cB/dqXr4mNje235rP+gfYRG6vP+kVERkvUn8CpqanhD7QB/H5/xFXAleOtra04HA6Sk5Pp6Oigr6+v3zqHw0FraysAvb29dHV1kZSUREpKCi0tLf22lZqaOmC/iIiMjqjhMX/+fGpqaggEApw/f56DBw/idDrD45MnT8Zms4X/BZTH48HpdBIXF0daWhpVVVUAuN3u8Lr09HTcbjcAVVVVpKWlERcXR3p6Oh6PB4C6ujpsNhs33ngjTqeTyspK+vr6+Oijj/jwww+ZOXPmyJ4JEREZshhjCP+Mp7Kykueff55gMMiKFSvIz88nPz+fDRs2MHPmTN5//322bNlCZ2cn06dPZ/v27YwbN46mpiYKCwtpa2tj0qRJ7Nq1i+uuu45PPvmEwsJCzpw5Q2JiIqWlpdx000309PRQXFzM8ePHGTduHI8//jjTp0/HMAx27tzJ66+/DkBRURELFiz4wk+OiIgMbEjhISIicjl96iwiIqYpPERExDSFh4iImKbwEBER0xQeg4j2MMhrzdq1a1myZAnZ2dlkZ2fz7rvvDlrjkSNHWLp0KYsWLWL37t3h/sEegPll09nZSVZWFn/84x8B8/WYfaDnl8GVNRcVFbFo0aLw6/2f//mfwMidi9H285//nCVLlrBkyRJ27twJfPVf54FqHtXX2ZB+zp07Z3znO98x2tvbja6uLmPp0qXGBx98MNqHNWyhUMhYsGCBEQwGw32D1Xj+/HkjPT3d+Pjjj41gMGjcd999xqFDhwzDMIwlS5YYb7/9tmEYhlFUVGSUlZWNSj1/yTvvvGNkZWUZ06dPN86cOTOseu6//37jd7/7nWEYhvHzn//c2Llzp2EYhlFSUmI8//zzhmEYxmuvvWY8+OCDV7u8AV1Zs2EYRlZWluHz+SLmjeS5GE1vvPGG8f3vf9/o6ekxLl68aOTl5RmVlZVf6dd5oJoPHjw4qq+zrjwGcPnDIOPj48MPg7xWnT59GoD77ruPZcuW8fLLLw9a43vvvcfNN9/MlClTsFqtLF26FK/XO+ADML+M56SiooKtW7eGn0Bgtp5gMEhtbS0ulyuiH+DQoUMsXboUuPRAz9dff51gMDgKVUa6subz58/T3NzMpk2bWLp0Kc888wyhUGhEz8VostvtFBYWMm7cOOLi4vjrv/5rPvzww6/06zxQzc3NzaP6Okd9MOJYFO1hkNeaTz/9lG9/+9v867/+K8FgkLy8PBYvXjxgjQPV7vP5Bn0A5pfNZ4/3/4zZeobzQM+UlJQvuqy/6MqaW1tbmTdvHlu3biUxMZEf/vCHvPrqq8THx4/YuRhNX//618N///DDDzlw4AA/+MEPvtKv80A1l5WVcfTo0VF7nXXlMYBoD4O81syZM4edO3eSmJhIcnIyK1as4JlnnhmwxsFqv1bPidl6Bqor2gM9v2ymTJnCs88+i8PhYMKECaxdu5bDhw9/oediNHzwwQfcd999PPLII0yZMmVMvM6X13zLLbeM6uv85TgjXzLRHgZ5ramrq6OmpibcNgyDyZMnD1jjYLUP9gDMLzuz9QzngZ5fNo2NjVRXV4fbhmFgtVpH9FyMtvr6eu69917+5V/+heXLl4+J1/nKmkf7dVZ4DCDawyCvNR0dHezcuZOenh46Ozt57bXX+Ld/+7cBa/ybv/kb/vCHP/DRRx/R19fH7373O5xO56APwPyyM1vPcB7o+WVjGAZPPPEEf/rTnwgGg5SXl3P33XeP6LkYTWfPnuWBBx6gtLSUJUuWAF/913mgmkf7ddazrQYx0MMgr2VPP/001dXVhEIh1qxZwz333DNojTU1NWzfvp2enh7S09MpKioiJiZm0Adgfhndeeed/OpXv+Kmm24yXY/ZB3p+WVxec1lZGWVlZfT29rJo0SIefvhhwPxrO9i5GE2PP/44v/nNb5g6dWq4b/Xq1fzVX/3VV/Z1HqzmUCg0aq+zwkNEREzTbSsRETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIia99957FBcXh9v33XcfgUBgwLn5+fmcPHmSN998k6ysLNP72r9//5Ce6nz27Nnws8uysrJ47bXXTO9LxAw920rEpJMnT0Y8++eNN94YdO6///u/A9DW1jasfdXX10c812gwJSUlOJ1O7r33XlpbW1m0aBHf/va3SU1NHdZ+RaJReIgMIhQK8cQTT/Duu+/S1dWFYRiUlJTwzDPP0NHRQVFRUXjuPffcwy9/+Utyc3OZNWsWjY2N/PM//zPbt2/nZz/7GQDd3d1s2LCBjz76iK997Wv89Kc/Zdq0aRQWFvL1r3+ddevWAYTbU6dO5fe//z1vvPEG48ePJzc3l+eee46DBw8SCoWYPHkyW7duJSUlhT179vDZr2w1NzdjtVqx2WxX/6TJmKHbViKDePfdd2lpaaG8vJyqqiqWL1/Oiy++yIYNG0hLS2P79u1s374dgP/4j/9g0qRJwKUnoB44cIC77747Yntnz57l3nvvxePxkJWVxSOPPPIX93/33Xdz5513cu+995Kbm4vb7ebEiRPs378fj8dDeno6W7ZsASA2NhaLxcLatWtZvXo1K1as4Prrr/8CzorIJbryEBnEnDlzuO666/j1r3/NmTNnePPNN5k4cWLUdWlpaQP2f+Mb3+D2228HYPny5Tz22GN0dHQM+Xj+53/+h2PHjvG9730PuHRldP78+Yg5e/fuJRAI8I//+I/85je/Cc8VGWm68hAZxKFDh/jhD38IwF133cU//MM/DGldfHz8gP1XPtY7JiYGq9Uafiz2Zwb74qFQKMQ//dM/4fF48Hg8/OY3v+GVV14BwOv10tnZCUBycjILFy7k//7v/4Z0vCLDofAQGcQbb7zBd77zHdasWcOMGTP4r//6L/r6+rBYLBHf335lezCNjY00NDQAUF5ezh133MGECRO4/vrrOX78OAA+n4+jR48OuO0FCxbw6quvhkPiZz/7WfjW1yuvvMLLL78MXHqK8n//938zb968ETgLIgNTeIgMYvXq1Rw9epSlS5eyfPlypkyZwh//+EdmzZrFmTNn+PGPfwxARkYGa9eu5cSJE39xe7fccgs///nPWbZsGb///e958sknAVi7di1+vx+Xy8WmTZsifug7nU5+/etf8/zzz7Ny5Ur+/u//nlWrVrFkyRIaGxvD23jyySepr69n6dKlrFmzhu9973v9PnMRGUl6qq6IiJimKw8RETFN4SEiIqYpPERExDSFh4iImKbwEBER0xQeIiJimsJDRERMU3iIiIhp/z/VZKANHujIHgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute3'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11cdc4ba8>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df1RU953/8efAAGaEhOjOQA5Nk01sk6yV2ISurKcHkmwE5UdQqm0iKya0kOS0q8v2kMVosXE1NJYsNjXxuJ7U05ziCklPmaVHgaxdk0Y8UdhNoieuKdtuQ9EOg5g4/J5h7vcPv84JIg4gDpD7epyTEz8/7p335zrwmnsvXC2GYRiIiIhphU11ASIiMrUUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnIKAhERk7NOdQETcf58D37/xH79Ye7caM6d657kiq6vmVgzqO5Qmok1g+oOlbAwCzffPHvU8RkZBH6/MeEguLT9TDMTawbVHUozsWZQ3dOBLg2JiJicgkBExOTGFAR1dXVkZGSQlpZGVVXViPFTp06Rm5tLeno6GzduxOfzAdDc3Exubi7Z2dk89dRTfPrppwBcuHCBoqIili1bRl5eHm63exKXJCIi4xE0CFwuF5WVlezbt4/a2lqqq6tpbW0dNqekpISysjIaGhowDIOamhoANmzYwPbt26mrq2PevHm8+uqrAOzYsYOkpCQOHjzIqlWr2LZt23VYmoiIjEXQIGhqaiI5OZnY2FhsNhvp6enU19cHxtvb2+nv72fhwoUA5ObmBsYPHDjAvHnz8Hq9uFwubrzxRgAOHz5MdnY2AFlZWbz99tt4vd5JX5yIiAQXNAg6Ojqw2+2BtsPhwOVyjTput9sD4xEREZw+fZrU1FTeffddMjMzR2xjtVqJjo6mq6trclYkIiLjEvTHR/1+PxaLJdA2DGNYO9j4XXfdRVNTE/v376e4uJj9+/ePeA3DMAgLG/t967lzo8c890rs9phr2n4qzMSaQXWH0kysGVT3dBA0COLj42lubg603W43Dodj2Phnb/Z2dnbicDgYGBjgt7/9LQ8//DAAjzzyCC+88AJw8ayis7OT+Ph4fD4fPT09xMbGjrnoc+e6J/wzvLNmR3Guq2dEf1SEFes0/Rkquz0Gt9sz1WWMm+oOnZlYM6juUAkLs1z1A3TQb32LFy/m6NGjdHV10dfXR2NjIykpKYHxhIQEoqKiaGlpAcDpdJKSkoLVauW5557j5MmTABw8eJD77rsPgNTUVGpra4GL9xGSkpKIiIiY+CrHoa/fx/FTrhH/DXh9IXl9EZHpJugZQVxcHMXFxeTn5+P1elm5ciWJiYkUFhaybt06FixYQEVFBZs2baK7u5v58+eTn59PeHg4lZWVlJWVMTQ0RFxcXOCng9avX09paSmZmZnExMRQUVFx3RcqIiJXZpmJ/2bxtVwaMsLDeavl4xH9X7snjtlR0/OJGzPtNPQS1R06M7FmUN2hcs2XhkRE5PNNQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImpyAQETE5BYGIiMkpCERETE5BICJicgoCERGTUxCIiJicgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnJjCoK6ujoyMjJIS0ujqqpqxPipU6fIzc0lPT2djRs34vP5AGhpaWHlypXk5OSwdu1a2tvbATh27BiLFi0iJyeHnJwcNmzYMIlLEhGR8QgaBC6Xi8rKSvbt20dtbS3V1dW0trYOm1NSUkJZWRkNDQ0YhkFNTU2gf+vWrTidTrKzs9m6dSsAJ0+epKCgAKfTidPppLy8/DosTURExiJoEDQ1NZGcnExsbCw2m4309HTq6+sD4+3t7fT397Nw4UIAcnNzqa+vZ3BwkPXr13P33XcDcNddd3H27FkATpw4wTvvvEN2djZPPfVUoF9ERELPGmxCR0cHdrs90HY4HHzwwQejjtvtdlwuF5GRkeTk5ADg9/vZuXMnDz/8MAAxMTEsW7aMtLQ0/u3f/o3i4mL2798/5qLnzo0e89wR6+nqJSZ61oh+my0K+xzbhPd7vdntMVNdwoSo7tCZiTWD6p4OggaB3+/HYrEE2oZhDGsHGx8cHKS0tBSfz8eTTz4JwJYtWwLjjz32GC+++CIej4eYmLEd2HPnuvH7jTHNHSE8HE93/4ju3t4B3ENDE9vndWa3x+B2e6a6jHFT3aEzE2sG1R0qYWGWq36ADnppKD4+HrfbHWi73W4cDseo452dnYHxnp4evvOd7+Dz+di1axcRERH4/X527drF0GXfdMPDw8e+KhERmTRBg2Dx4sUcPXqUrq4u+vr6aGxsJCUlJTCekJBAVFQULS0tADidzsB4SUkJt912Gzt27CAyMvLiC4aF8eabb9LQ0ABAbW0t9957Lzbb9L0sIyLyeRb00lBcXBzFxcXk5+fj9XpZuXIliYmJFBYWsm7dOhYsWEBFRQWbNm2iu7ub+fPnk5+fz4cffsihQ4eYN28eK1asAC7eX9izZw8vvPACP/jBD3j55ZeZM2cO27dvv+4LFRGRK7MYhjHBi+1T51ruERjh4bzV8vGI/q/dE8fsqKC5OCVm2vXIS1R36MzEmkF1h8o13yMQEZHPNwWBiIjJKQhERExOQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImpyAQETE5BYGIiMkpCERETE5BICJicgoCERGTUxCIiJicgkBExOQUBCIiJqcgEBExuTEFQV1dHRkZGaSlpVFVVTVi/NSpU+Tm5pKens7GjRvx+XwAtLS0sHLlSnJycli7di3t7e0AXLhwgaKiIpYtW0ZeXh5ut3sSlyQiIuMRNAhcLheVlZXs27eP2tpaqquraW1tHTanpKSEsrIyGhoaMAyDmpqaQP/WrVtxOp1kZ2ezdetWAHbs2EFSUhIHDx5k1apVbNu27TosTURExiJoEDQ1NZGcnExsbCw2m4309HTq6+sD4+3t7fT397Nw4UIAcnNzqa+vZ3BwkPXr13P33XcDcNddd3H27FkADh8+THZ2NgBZWVm8/fbbeL3eSV+ciIgEFzQIOjo6sNvtgbbD4cDlco06brfbcblcREZGkpOTA4Df72fnzp08/PDDI7axWq1ER0fT1dU1OSsSEZFxsQab4Pf7sVgsgbZhGMPawcYHBwcpLS3F5/Px5JNPXvE1DMMgLGzs963nzo0e89zLdXT1EhM9a0S/zRaFfY5twvu93uz2mKkuYUJUd+jMxJpBdU8HQYMgPj6e5ubmQNvtduNwOIaNf/Zmb2dnZ2C8p6eHp59+mtjYWHbt2kVERARw8ayis7OT+Ph4fD4fPT09xMbGjrnoc+e68fuNMc8fJjwcT3f/iO7e3gHcQ0MT2+d1ZrfH4HZ7prqMcVPdoTMTawbVHSphYZarfoAO+jF88eLFHD16lK6uLvr6+mhsbCQlJSUwnpCQQFRUFC0tLQA4nc7AeElJCbfddhs7duwgMjIysE1qaiq1tbUAHDhwgKSkpEBIiIhIaAU9I4iLi6O4uJj8/Hy8Xi8rV64kMTGRwsJC1q1bx4IFC6ioqGDTpk10d3czf/588vPz+fDDDzl06BDz5s1jxYoVwMUzgT179rB+/XpKS0vJzMwkJiaGioqK675QERG5MothGBO8xjJ1ruXSkBEezlstH4/o/9o9ccyOCpqLU2KmnYZeorpDZybWDKo7VK750pCIiHy+KQhERExOQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImpyAQETE5BYGIiMkpCERETE5BICJicgoCERGTUxCIiJicgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExuTEFQV1dHRkYGaWlpVFVVjRg/deoUubm5pKens3HjRnw+37DxHTt28NOf/jTQPnbsGIsWLSInJ4ecnBw2bNhwjcsQEZGJChoELpeLyspK9u3bR21tLdXV1bS2tg6bU1JSQllZGQ0NDRiGQU1NDQAej4dnn32WvXv3Dpt/8uRJCgoKcDqdOJ1OysvLJ3FJIiIyHkGDoKmpieTkZGJjY7HZbKSnp1NfXx8Yb29vp7+/n4ULFwKQm5sbGD906BC33347TzzxxLB9njhxgnfeeYfs7Gyeeuopzp49O5lrEhGRcQgaBB0dHdjt9kDb4XDgcrlGHbfb7YHx5cuXU1RURHh4+LB9xsTEsGbNGurq6khNTaW4uPiaFyIiIhNjDTbB7/djsVgCbcMwhrWDjV/Jli1bAn9+7LHHePHFF/F4PMTExIyp6Llzo8c070o6unqJiZ41ot9mi8I+xzbh/V5vdvvYjs10o7pDZybWDKp7OggaBPHx8TQ3Nwfabrcbh8MxbNztdgfanZ2dw8Yv5/f72b1794gzhcvPGq7m3Llu/H5jzPOHCQ/H090/oru3dwD30NDE9nmd2e0xuN2eqS5j3FR36MzEmkF1h0pYmOWqH6CDXhpavHgxR48epauri76+PhobG0lJSQmMJyQkEBUVRUtLCwBOp3PY+MiCwnjzzTdpaGgAoLa2lnvvvRebbfp+GhcR+TwLGgRxcXEUFxeTn5/P8uXLycrKIjExkcLCQk6cOAFARUUF5eXlLF26lN7eXvLz86+6zxdeeIHXXnuNzMxMfvnLX7J169bJWY2IiIybxTCMCV5jmTrXcmnICA/nrZaPR/R/7Z44ZkcFvVI2JWbaaeglqjt0ZmLNoLpD5ZovDYmIyOebgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImpyAQETE5BYGIiMkpCERETE5BICJicgoCERGTUxCIiJicgkBExOTGFAR1dXVkZGSQlpZGVVXViPFTp06Rm5tLeno6GzduxOfzDRvfsWMHP/3pTwPtCxcuUFRUxLJly8jLy8Ptdl/jMkREZKKCBoHL5aKyspJ9+/ZRW1tLdXU1ra2tw+aUlJRQVlZGQ0MDhmFQU1MDgMfj4dlnn2Xv3r3D5u/YsYOkpCQOHjzIqlWr2LZt2yQuSURExiNoEDQ1NZGcnExsbCw2m4309HTq6+sD4+3t7fT397Nw4UIAcnNzA+OHDh3i9ttv54knnhi2z8OHD5OdnQ1AVlYWb7/9Nl6vd9IWJSIiYxc0CDo6OrDb7YG2w+HA5XKNOm632wPjy5cvp6ioiPDw8FH3abVaiY6Opqur69pWIiIiE2INNsHv92OxWAJtwzCGtYONj4VhGISFjf2+9dy50ePa/2d1dPUSEz1rRL/NFoV9jm3C+73e7PaYqS5hQlR36MzEmkF1TwdBgyA+Pp7m5uZA2+1243A4ho1/9mZvZ2fnsPErcTgcdHZ2Eh8fj8/no6enh9jY2DEXfe5cN36/Meb5w4SH4+nuH9Hd2zuAe2hoYvu8zuz2GNxuz1SXMW6qO3RmYs2gukMlLMxy1Q/QQT+GL168mKNHj9LV1UVfXx+NjY2kpKQExhMSEoiKiqKlpQUAp9M5bPxKUlNTqa2tBeDAgQMkJSURERExpgWJiMjkChoEcXFxFBcXk5+fz/Lly8nKyiIxMZHCwkJOnDgBQEVFBeXl5SxdupTe3l7y8/Ovus/169fz3nvvkZmZyb59+ygrK5uc1YiIyLhZDMOY4DWWqXMtl4aM8HDeavl4RP/X7oljdlTQK2VTYqadhl6iukNnJtYMqjtUrvnSkIiIfL4pCERETE5BICJicgoCERGTUxCIiJicgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImpyAQETE5BYGIiMkpCERETG5MQVBXV0dGRgZpaWlUVVWNGD916hS5ubmkp6ezceNGfD4fAGfOnCEvL4+lS5fy9NNP09PTA8CxY8dYtGgROTk55OTksGHDhklckoiIjEfQIHC5XFRWVrJv3z5qa2uprq6mtbV12JySkhLKyspoaGjAMAxqamoAeO6551i9ejX19fV85Stf4ZVXXgHg5MmTFBQU4HQ6cTqdlJeXX4eliYjIWAQNgqamJpKTk4mNjcVms5Genk59fX1gvL29nf7+fhYuXAhAbm4u9fX1eL1ejh8/Tnp6+rB+gBMnTvDOO++QnZ3NU089xdmzZ6/H2kREZAyCBkFHRwd2uz3QdjgcuFyuUcftdjsul4vz588THR2N1Wod1g8QExPDmjVrqKurIzU1leLi4klbkIiIjI812AS/34/FYgm0DcMY1h5t/PJ5QKC9ZcuWQN9jjz3Giy++iMfjISYmZkxFz50bPaZ5V9LR1UtM9KwR/TZbFPY5tgnv93qz28d2bKYb1R06M7FmUN3TQdAgiI+Pp7m5OdB2u904HI5h4263O9Du7OzE4XAwZ84cPB4PQ0NDhIeHB7bz+/3s3r2boqIiwsPDA9t99s/BnDvXjd9vjHn+MOHheLr7R3T39g7gHhqa2D6vM7s9BrfbM9VljJvqDp2ZWDOo7lAJC7Nc9QN00EtDixcv5ujRo3R1ddHX10djYyMpKSmB8YSEBKKiomhpaQHA6XSSkpJCREQESUlJHDhwAIDa2lpSUlIICwvjzTffpKGhIdB/7733YrNN30/jIiKfZ0GDIC4ujuLiYvLz81m+fDlZWVkkJiZSWFjIiRMnAKioqKC8vJylS5fS29tLfn4+AJs3b6ampoaMjAyam5v5h3/4BwBeeOEFXnvtNTIzM/nlL3/J1q1br+MSRUTkaiyGYUzwGsvUuZZLQ0Z4OG+1fDyi/2v3xDE7KuiVsikx005DL1HdoTMTawbVHSrXfGlIREQ+3xQEIiImpyAQETE5BYGIiMkpCERETE5BICJicgoCERGTUxCIiJicgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnIKAhERk1MQiIiYnIJARMTkFAQiIianIBARMTkFgYiIySkIRERMTkEgImJyCgIREZNTEIiImJyCQETE5BQEIiImN6YgqKurIyMjg7S0NKqqqkaMnzp1itzcXNLT09m4cSM+nw+AM2fOkJeXx9KlS3n66afp6ekB4MKFCxQVFbFs2TLy8vJwu92TuCQRERmPoEHgcrmorKxk37591NbWUl1dTWtr67A5JSUllJWV0dDQgGEY1NTUAPDcc8+xevVq6uvr+cpXvsIrr7wCwI4dO0hKSuLgwYOsWrWKbdu2XYelBdc/6KPd3cOn3YMMDflHnef3GxiGMayvb8BH14V+Oj7po7vPe011eHoH6e33XdM+REQmymJc/h3uMr/61a84fvw4zz//PAAvv/wyhmHwve99D4D29nbWrl3Lf/zHfwDQ3NzMSy+9xKuvvsqiRYs4duwYVquVs2fP8nd/93ccOnSIhx56iKqqKm655RZ8Ph9//dd/zbvvvktERMSYij5/vge//6pljzDk9/P2e2cY8Bn8uauHrgv9uLp6ubR6iwXCwiz4/QYR1nDm3jiLm6Ij+cQzgPuTPqIirdx5y43E3hjFH85coN3dzaUKLMAX42KY94WbmBVpHXNN/YM+Pmr7hLaObixA3FwbX/iL2URGhBMRHo7VaiHSGkZ09Cx6egfA4OJr/v+iL/+bs1i4Ykeg23Lpf5dPvHZ9gz56+rz4hvzMnhXB7FlWYmJm0dMzOOmvdb3ZbJH09s6sumdizWDeuo3AF/PIr2lj+CwAwi0WLGEWFtwxl7k3zhr364WFWbj55tmjjgf9rtXR0YHdbg+0HQ4HH3zwwajjdrsdl8vF+fPniY6Oxmq1Duu/fBur1Up0dDRdXV3ExcWNaVFXW9DVrFxy44S2ExH5PAt6acjv92P5zEdNwzCGtUcbv3weMKL92W3CwnTfWkRkKgT97hsfHz/sZq7b7cbhcIw63tnZicPhYM6cOXg8HoaGhkZs53A46OzsBMDn89HT00NsbOzkrEhERMYlaBAsXryYo0eP0tXVRV9fH42NjaSkpATGExISiIqKoqWlBQCn00lKSgoREREkJSVx4MABAGprawPbpaamUltbC8CBAwdISkoa8/0BERGZXEFvFsPFHx/dvXs3Xq+XlStXUlhYSGFhIevWrWPBggX8z//8D5s2baK7u5v58+dTXl5OZGQk7e3tlJaWcu7cOW655Rb+5V/+hZtuuolPPvmE0tJS2traiImJoaKigi984QuhWK+IiFxmTEEgIiKfX7pDKyJicgoCERGTUxCIiJicgkBExORMEwTBHpw3lXbu3ElmZiaZmZls374dgA0bNpCWlkZOTg45OTm8+eabADQ1NZGdnU1aWhqVlZVTWTZr1qwhMzMzUOP7778/6nGeLnW//vrrgXpzcnK4//772bJly7Q93t3d3WRlZfGnP/3pqvWM98GPoa67urqarKwssrOz2bBhA4ODFx/PsHPnTh588MHAcb/0nhltPaGue7zvi6mq+5oZJvDnP//ZePDBB43z588bPT09RnZ2tvG73/1uqssyDMMwjhw5YnzrW98yBgYGjMHBQSM/P99obGw0srKyDJfLNWxuX1+fkZqaanz88ceG1+s1CgoKjMOHD09J3X6/3/j6179ueL3eQN9ox3k61f1ZH330kbFkyRLj3Llz0/J4v/fee0ZWVpYxf/58o62t7ar1ZGZmGv/93/9tGIZhbNiwwaiqqjIMwzCKioqMX//614ZhGMbOnTuN7du3h7zu3//+98aSJUsMj8dj+P1+45lnnjH27t1rGIZhPPnkk8Z//dd/jdjHaOsJZd2GYYz7fTEVdU8GU5wRNDU1kZycTGxsLDabjfT0dOrr66e6LODiM5hKS0uJjIwkIiKCO++8kzNnznDmzBmeffZZsrOzeemll/D7/XzwwQfcdttt3HrrrVitVrKzs6dsHb///e8BKCgo4JFHHuEXv/jFqMd5OtX9WT/84Q8pLi7mhhtumJbHu6amhs2bNwd+I3+0etrb2+nv72fhwoUA5ObmUl9fj9fr5fjx46Snpw/rD3XdkZGRbN68mejoaCwWC1/+8pc5c+YMACdPnmT37t1kZ2ezZcsWBgYGRl1PqOvu6+sb1/tiquqeDGN/VOYMFuzBeVPpS1/6UuDP//d//8fBgwepqqri2LFjbN68mZiYGJ588kneeOMNbDbbiHVcepBfqF24cIG/+Zu/4Qc/+AFer5f8/HyWLVt2xeN8peM/VXVf0tTURH9/P8uWLaOtrY3k5ORpd7wvfzz7aMdxIg9+DGXdCQkJJCQkANDV1UVVVRXl5eX09PRwzz33UFJSwm233UZpaSmvvPIKDzzwwBXXE+q6Ozs7x/W+GO3vYSYwxRlBsAfnTQe/+93vKCgo4JlnnuGOO+7g5ZdfxuFwcMMNN7BmzRreeuutabWOr371q2zfvp2YmBjmzJnDypUreemll65Y33Sq+5L9+/fzxBNPAHDrrbdO++MNo7+PJ+PBj6HgcrlYu3Yt3/jGN1i0aBGzZ89mz5493HnnnVitVgoKCqbVcR/v+2K61D0RpgiCYA/Om2otLS08/vjjfP/732fFihWcPn2ahoaGwLhhGFit1mm1jubmZo4ePRpoG4ZBQkLCFeubTnUDDA4Ocvz4cR566CGAGXG8YfT38UQe/Bhq//u//8ujjz7KihUr+O53vwtcvJH9xhtvBOaMdtwvrSfUxvu+mC51T4QpgiDYg/Om0tmzZ/nud79LRUUFmZmZwMU33PPPP8+nn36K1+ulurqaJUuWcO+99/KHP/yBP/7xjwwNDfHrX/96ytbh8XjYvn07AwMDdHd386tf/Yof//jHVzzO06luuPgFfvvtt2Oz2YCZcbyBUeuZyIMfQ6m7u5tvf/vbrF+/noKCgkD/rFmz+PGPf0xbWxuGYVBVVcWSJUtGXU+ojfd9MV3qnghT3COIi4ujuLiY/Pz8wIPzEhMTp7osAF599VUGBgb40Y9+FOh79NFHKSoq4rHHHsPn85GWlkZWVhYAP/rRj/j7v/97BgYGSE1NZenSpVNS94MPPsj777/P8uXL8fv9rF69mvvvv3/U4zxd6gZoa2sjPj4+0L777run/fEGiIqKGrWeioqKYQ9+zM/PB2Dz5s2Ulpaya9euwIMfQ+2NN96gs7OTvXv3snfvXgAeeugh1q9fz5YtW3j66afxer3cd999gct1o60nlCbyvpgOdU+EHjonImJyprg0JCIio1MQiIiYnIJARMTkFAQiIianIBARMTkFgZjeBx98QFlZWaBdUFBAV1fXFecWFhbS2trKu+++G/hRwvF4/fXXx/X028HBQVatWsWrr7467tcSGSsFgZhea2vrsGfCHDlyZNS5e/bsYd68eRN+rZaWFvr7+8c8//nnn6etrW3CrycyFqb4hTIRuPisnueff57333+fnp4eDMPgueee46WXXsLj8bBhw4bA3LVr1/Kv//qv5OXlkZiYyOnTp/nHf/xHysvL+clPfgJAb28v69at449//CM33ngjW7Zs4S//8i8pLS3lS1/6Et/+9rcBAu0vfvGL/OY3v+HIkSPMmjWLvLw8du3aRWNjI36/n4SEBDZv3kxcXBxw8TeBPR4PDzzwQMiPlZiLzgjENN5//306Ojqorq7mwIEDrFixgp/97GesW7eOpKQkysvLKS8vB+DnP/85t9xyC3DxCbEHDx5kyZIlw/Z39uxZHn/8cZxOJ1lZWTzzzDNXff0lS5bw0EMP8fjjj5OXl0dtbS0fffQRr7/+Ok6nk9TUVDZt2gRcfAzGa6+9xj//8z9fhyMhMpzOCMQ0vvrVr3LTTTexf/9+2traePfdd5k9e3bQ7ZKSkq7Yf9ddd3HfffcBsGLFCn74wx/i8XjGXM9//ud/crXXACQAAAG0SURBVOLECb7xjW8AF89Y+vr68Hg8/NM//RMVFRWB5yGJXE8KAjGNw4cPs23bNp544gn+9m//ljvuuIN///d/D7rdaN+Mw8KGn1BbLBasVmvgEdCXeL3eK27v9/v5zne+w+rVq4GLN4Y//fRTfvvb33LhwgW+//3vAxfPPI4cOUJ3dzfr168f01pFxkNBIKZx5MgRHnzwQVavXk1/fz979uxhaGiI8PDwYf+27OXt0Zw+fZpTp05xzz33UF1dzf33388NN9zAzTffzMmTJ4GLz+A/duwYf/VXfzVi31//+teprq7mkUceITo6mp/85Cd8+OGH7N27l4yMjMDrXH7PQWSy6R6BmMajjz7KsWPHyM7OZsWKFdx666386U9/IjExkba2Nr73ve8BsHTpUtasWcNHH3101f3dcccd7Ny5k0ceeYTf/OY3gSfIrlmzBrfbTXp6Os8++yzJycmBbVJSUti/fz+7d+9m1apVPPDAA3zzm98kMzOT06dPD3sKrUio6OmjIiImpzMCERGTUxCIiJicgkBExOQUBCIiJqcgEBExOQWBiIjJKQhERExOQSAiYnL/D4XdadNKB/+0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute4'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11d0fc0b8>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXhU5d34//esSSYJhMAkgbiiCJX1qVEjWmhFCFsa5cFWSAVLRbELNs8lPLhUhJ8W8YdiWwUttf22FioIkhi/GBAVK4sCkSUIgljWJEwm+2SyzHLO94/JDIQsk5BJMsl8Xtfl5Zxz7rPcM2Q+c+8aVVVVhBBChCxtVz+AEEKIriWBQAghQpwEAiGECHESCIQQIsRJIBBCiBAngUAIIUKcBAIhhAhx+q5+gCtRVmZHUdo+/KFv3yhKSqo64ImCWyjmOxTzDJLvUNKWPGu1Gvr0iWz2eLcMBIqiXlEg8J4bikIx36GYZ5B8h5JA5VmqhoQQIsRJIBBCiBAngUAIIUKcBAIhhAhxEgiEECLESSAQQogQJ4EgQNyKwqdfncflVrr6UYQQok0kEATIyfMVvL3tBN+er+jqRxFCiDaRQBAgzvqSgNMlJQIhRPcigSBA3G61/v8SCIQQ3YsEggBx1w/1dofgMHchRPcmgSBAvAFAGouFEN2NBIIA8VYJudxSIhBCdC+tCgTZ2dlMnjyZCRMmsHbt2kbHt2/fTlpaGj/+8Y/55S9/SUWFp+dMQUEB6enpTJw4kcceewy73Q5AZWUljzzyCJMmTSI9PR2r1RrALHWNi1VDUiIQQnQvfgOBxWJh5cqVrFu3jszMTNavX8/Jkyd9x6uqqnjuuef485//zPvvv8/gwYP505/+BMCSJUuYOXMmOTk5DBs2jFWrVgHw6quvkpSUxIcffsj999/PCy+80EHZ6zwXq4akRCCE6F78BoLdu3eTnJxMTEwMJpOJlJQUcnJyfMedTieLFy8mPj4egMGDB1NYWIjT6WTfvn2kpKQAMG3aNN95O3bsIDU1FYCpU6fy73//G6fTGfDMdSZv1ZA0Fgshuhu/gaCoqAiz2ezbjouLw2Kx+Lb79OnD+PHjAaitreXPf/4z99xzD2VlZURFRaHXe9a+MZvNvvMuvaZerycqKorS0tLA5aoLuKT7qBCim/K7QpmiKGg0Gt+2qqoNtr1sNhu/+tWvGDJkCPfddx8Wi6VRuqbO815Tq219u3XfvlGtTns5szn6is9tSXiEEYCwcEOH3aM9gvGZOloo5hkk36EkUHn2GwgSEhLYv3+/b9tqtRIXF9cgTVFREb/4xS9ITk7mqaeeAiA2NhabzYbb7Uan0zU4Ly4ujuLiYhISEnC5XNjtdmJiYlr90CUlVVe0RJvZHI3Vamvzea1Raaup/39th93jSnVkvoNVKOYZJN+hpC151mo1Lf6A9vszfPTo0ezZs4fS0lJqamrYtm0bY8aM8R13u93MmzePSZMm8fTTT/t+9RsMBpKSktiyZQsAmZmZvvPGjh1LZmYmAFu2bCEpKQmDwdCqDAUr78hiaSwWQnQ3fksE8fHxZGRkMGvWLJxOJ9OnT2fEiBHMnTuX+fPnc+HCBY4ePYrb7Wbr1q0ADBs2jBdeeIHFixezaNEiVq9eTf/+/XnllVcAePzxx1m0aBFTpkwhOjqaFStWdGwuO4FLuo8KIbopjaqq3e4nbDBWDb274yQffnGWcbdcRfr4mzrkHldKis2hQ/IdOjq1aki0jkw6J4ToriQQBIhvQJmMIxBCdDMSCALEN6BMSgRCiG5GAkGAuGSKCSFENyWBIEB8bQRSNSSE6GYkEASIt9uorEcghOhuJBAEiKxQJoToriQQBIh0HxVCdFcSCAJE1iMQQnRXEggCxNtGIFNMCCG6GwkEASKTzgkhuisJBAHikl5DQohuSgJBgMg4AiFEdyWBIEB83UelRCCE6GYkEASITDonhOiuJBAEyMVJ5yQQCCG6F78rlAFkZ2ezevVqXC4Xs2fPJj09vcl0CxcuJDk5mWnTplFSUsKcOXN8x2w2G2VlZRw4cIC9e/fym9/8hoSEBABuvvlmli1bFoDsdJ2L4wikakgI0b34DQQWi4WVK1fy3nvvYTQaeeCBB7j99tu58cYbG6RZvHgxe/bsITk5GYC+ffuSlZUFgKIozJ49m4yMDACOHDnCnDlzePTRRzsiT11CppgQQnRXfquGdu/eTXJyMjExMZhMJlJSUsjJyWmQJjs7m3HjxjFp0qQmr7Fp0yYiIiJITU0FIC8vj507d5Kamsq8efMoLCwMQFa6lrdqSEoEQojuxm+JoKioCLPZ7NuOi4vj8OHDDdI8/PDDAOTm5jY63+1288Ybb7Bq1SrfvujoaCZNmsSECRP417/+RUZGBu+8806rH7qltTf9MZujr/jclnibBlQVYvtGodNqOuQ+V6qj8h3MQjHPIPkOJYHKs99AoCgKGs3FLzVVVRts+/P5559z3XXXMXjwYN++pUuX+l7PmDGDl19+GZvNRnR06zIVjIvXu1wXSwIWSwUGva5D7nMlZGHv0CH5Dh2dunh9QkICVqvVt221WomLi2vVzQG2b9/O5MmTfduKorB69WrcbneDdDpd8HxxXgm3ovhKATLNhBCiO/EbCEaPHs2ePXsoLS2lpqaGbdu2MWbMmFbf4ODBgyQlJV28oVbLRx99xNatWwHIzMxk5MiRmEymK3j84OF2qxgNnmAmDcZCiO7EbyCIj48nIyODWbNmce+99zJ16lRGjBjB3LlzycvL83uDc+fO+bqJei1fvpx//OMfTJkyhU2bNvH8889feQ6CgKKoqECYwfN2SoOxEKI70aiq2u1+vgZbG4HT5ebRFZ8R3ycCS1kNLz12B/16RwT8PldK6k9Dh+Q7dHRqG4Hwz9smIFVDQojuSAJBAHi/+MPqA4E0FgshuhMJBAHgDQTG+jYCmYFUCNGdSCAIAO8Xf5hUDQkhuiEJBAHgnXo6zOitGpISgRCi+5BAEADeEoGxfjSxTEUthOhOJBAEQKPGYkVKBEKI7kMCQQB4SwBhRu+AMikRCCG6DwkEAeDrNSRVQ0KIbkgCQQC4lct7DUnVkBCi+5BAEAAXq4ak15AQovuRQBAA3sbhiwPKpGpICNF9SCAIAF+JwNdrSAKBEKL7kEAQABenmPA2FkvVkBCi+5BAEAAy6ZwQojtrVSDIzs5m8uTJTJgwgbVr1zabbuHChbz33nu+7c2bN3PXXXeRlpZGWloaK1euBKCgoID09HQmTpzIY489ht1ub2c2ulbjuYakRCCE6D78BgKLxcLKlStZt24dmZmZrF+/npMnTzZKM2/ePN/yk15Hjhxh0aJFZGVlkZWVRUZGBgBLlixh5syZ5OTkMGzYMFatWhXALHW+xrOPSolACNF9+A0Eu3fvJjk5mZiYGEwmEykpKeTk5DRIk52dzbhx45g0aVKD/Xl5eWzevJnU1FSeeOIJKioqcDqd7Nu3j5SUFACmTZvW6HrdjTcQGHRatBqNTDEhhOhW/AaCoqIizGazbzsuLg6LxdIgzcMPP8z999/f6Fyz2cwvf/lL3n//ffr378/SpUspKysjKioKvV7vS3P59bobb9WQTqtBp9NIG4EQolvR+0ugKAoajca3rapqg+2WvP76677XDz/8MOPHj2fhwoWNzm/t9bxaWnvTH7M5+orPbU64yQhAXFwvDHotxjB9h9ynPYLteTpDKOYZJN+hJFB59hsIEhIS2L9/v2/barUSFxfn98I2m41Nmzbx0EMPAZ4AotPpiI2NxWaz4Xa70el0rb7epYJt8fqKiloAysvsaDUabFV1QbWQtizsHTok36GjUxevHz16NHv27KG0tJSamhq2bdvGmDFj/N7YZDLxl7/8hUOHDgHwz3/+k/Hjx2MwGEhKSmLLli0AZGZmtup6wczbS0iv81QNSWOxEKI78RsI4uPjycjIYNasWdx7771MnTqVESNGMHfuXPLy8po9T6fT8eqrr/Lcc88xadIkvv76axYsWADA4sWL2bBhA5MnT2b//v389re/DVyOuoD3i1+n1aLXamVAmRCiW/FbNQSQmppKampqg31r1qxplO7FF19ssJ2UlMTmzZsbpUtMTOTtt99uy3MGNZeiosFT/NLpNDLFhBCiW5GRxQHgVhR0Ok+Dt17Xcongi6MXyN51qrMeTQgh/JJAEABut4pO63kr9dqWu4/u/8bKrrwLnfVoQgjhlwSCAHArKjqtp0TgqRpqvkTgcLlxShuCECKISCAIALei+qqGdDpti72GXC4Fp0sCgRAieEggCACXW/GVCPRaTYttBA6XIiUCIURQkUAQAG63il7neSt1Oq1v7qGmOJwKLikRCCGCiASCAHArDUsELTUWO11u3IqKokoXUyFEcJBA0E4uxfMrX6PRYK9zoWo8DcLN/ej3VgtJqUAIESwkELRTndNFcWUtdU43+45ZqKxyYK91Ued0NZne4fQEAGknEEIECwkEAaAqKtr6GVS1Wk2LE+J5ewxJiUAIESwkEASAoqrUjydDq2ldIJAupEKIYCGBIACUBiUCmm0IdrkV3zGpGhJCBAsJBAGgqCoarf+qoUtLAVIiEEIECwkEAaAoXCwRaDTNlggclwYCKREIIYKEBIIA8LQRtKZE4Pa9lsZiIUSwkEAQAIqqUh8H6ksEnqU5L+eUEoEQIgi1KhBkZ2czefJkJkyYwNq1a5tNt3DhQt577z3fdm5uLtOnTyctLY3Zs2eTn58PwN69e7n99ttJS0sjLS2NJ598sp3Z6FrKZd1Hvfsu5x1DAOByychiIURw8LtCmcViYeXKlbz33nsYjUYeeOABbr/9dm688cYGaRYvXsyePXtITk727V+wYAGrVq1iyJAhbNy4keeff57Vq1dz5MgR5syZw6OPPtoxuepkqkqDqiGgyVXKLi0FSIlACBEs/JYIdu/eTXJyMjExMZhMJlJSUsjJyWmQJjs7m3HjxjFp0iTfPofDweOPP86QIUMAGDx4MIWFhQDk5eWxc+dOUlNTmTdvnm9/d9WgRFBfRdTUDKROp7QRCCGCj99AUFRUhNls9m3HxcVhsVgapHn44Ye5//77G+wzGo2kpaUBoCgKr732Gvfccw8A0dHRPPjgg2RnZzN27FgyMjLanZGu1GBAWX0kaGoGUuk1JIQIRn6rhhTFM6Gal6qqDbb9cTgcLFq0CJfL5asKWrp0qe/4jBkzePnll7HZbERHR7fqmn37RrX6/pczm1t3j9ZSS6tRVQgz6omOCicywgiAwWhodK+IQpvvdXh44+MdqTPvFSxCMc8g+Q4lgcqz30CQkJDA/v37fdtWq5W4uLhWXdxut/PYY48RExPD6tWrMRgMKIrCm2++ySOPPIJOp/OlvfS1PyUlVS1O49Acszkaq9XmP2EbVNe5cCueNQZsVbU4HJ7J5qrstY3uVVJi970uLa8J+LM0pyPyHexCMc8g+Q4lbcmzVqtp8Qe036qh0aNHs2fPHkpLS6mpqWHbtm2MGTOmVTdfsGAB1157La+++ipGo7H+gbR89NFHbN26FYDMzExGjhyJyWRq1TWDkaJc0lhcX1pqarlKxyXjCKRqSAgRLPyWCOLj48nIyGDWrFk4nU6mT5/OiBEjmDt3LvPnz2f48OFNnnf06FE+/vhjbrzxRu677z7A076wZs0ali9fzu9+9ztef/11YmNjeemllwKbq052+YAy8N9GII3FQohg4TcQAKSmppKamtpg35o1axqle/HFF32vb775Zo4fP97k9QYNGsQ777zTlucMWt4qKt+AMm8gaOIXv0sai4UQQUhGFreT2xcIGlYNNTWOwFsiCDfqpEQghAgaEgjaya14vtAvVg3V72/iF7/D5Uav02A06KREIIQIGhII2qm5EkFTbQROl4JBr8Og00qJQAgRNCQQtJO3d5DmsgFlriZ6DTldCka9Fr1eKyUCIUTQkEDQTr6qoctKBM1NOmfQazHotLIwjRAiaEggaCdvieDy2UddTc015HJ7AoFeIyUCIUTQkEDQTr42gssHlDXTRmCUNgIhRJCRQNBOjQJBi72GPFVD0kYghAgmEgjayX3ZgDJdS+sRuKSNQAgRfCQQtNPljcWaFuYaurTXUFO9ioQQoiu0aooJ0TzF3bBqSOeba6jpAWW+qqFLJqATQoiuJCWCdmo0oEx7sURQ63Bx7EyZL22DAWVSIhBCBAkJBO3kDQS+AWWX9Br6v3vOsOKdA9TUedYocLgUjAZviUDaCIQQwUECQTt5xwtcbCO4uP/At8WoKthqnMBljcXSa0gIESQkELTT5VVDGo0GrUaDpayGgmLPimR2XyDwDiiTcQRCiOAhjcXtpFw2jsDzGo6eKvVt22udKIqKy61i1OvQ4AkgiqI2OE8IIbpCq0oE2dnZTJ48mQkTJrB27dpm0y1cuJD33nvPt11QUEB6ejoTJ07ksccew273/EKurKzkkUceYdKkSaSnp2O1WtuZja5zeYkAPEGhzukmzOBZh7mkso6KagcAOp2njQBkcRohRHDwGwgsFgsrV65k3bp1ZGZmsn79ek6ePNkozbx583zrEHstWbKEmTNnkpOTw7Bhw1i1ahUAr776KklJSXz44Yfcf//9vPDCCwHMUufyjiDWXvJOeoPCVXGRABw7XcqXRy2Apw3BoKsPBFI9JIQIAn4Dwe7du0lOTiYmJgaTyURKSgo5OTkN0mRnZzNu3DgmTZrk2+d0Otm3bx8pKSkATJs2zXfejh07fEtfTp06lX//+984nc6AZaozNVciALhhQG8AHE63b1yBt40Amp6YTgghOpvfNoKioiLMZrNvOy4ujsOHDzdI8/DDDwOQm5vr21dWVkZUVBR6vecWZrMZi8XS6Jp6vZ6oqChKS0uJj49v1UP37RvVqnRNMZujr/jcpuj0nuqf6OhwIsMNAOh1WiLD9dxwTR8MB/JR0RAeZgQgyhRGZIQnXXSvCMx9IwP6PM0JdL67g1DMM0i+Q0mg8uw3ECiK4ps2AUBV1QbbzWkqXXPnqaqKVtv6DkwlJVVNzvfvj9kcjdVqa/N5Lamp9ZRkqqsdKPWjhaNNBm5I7I3dXodRr8VW7aCyqhYAVXFTW+N59iKrDV0TI5ADrSPyHexCMc8g+Q4lbcmzVqtp8Qe032/fhISEBo25VquVuLg4vzeOjY3FZrPhdrsbnRcXF0dxcTEALpcLu91OTEyM32sGo6baCMbdchU//sH1ABgNOk/VUH06vU6LXtoIhBBBxG8gGD16NHv27KG0tJSamhq2bdvGmDFj/F7YYDCQlJTEli1bAMjMzPSdN3bsWDIzMwHYsmULSUlJGAyG9uSjyzTVRgAXSz9hBh11TrdvNtJL2wik15AQIhj4DQTx8fFkZGQwa9Ys7r33XqZOncqIESOYO3cueXl5LZ67ePFiNmzYwOTJk9m/fz+//e1vAXj88cc5ePAgU6ZMYd26dTz77LOByU0XaC4QeIUZdTicim82Us/CNPVTVUuJQAgRBFo1oCw1NdXXy8drzZo1jdK9+OKLDbYTExN5++23G6WLiYnhjTfeaMtzBi3fXEPNNJuEGbTUXdJrSK/Xoq9PLCUCIUQwkJHF7eR2q2g1mmYbwo31VUPeEoFBr6W+QCBtBEKIoCBzDbWTW1FoqcNTmEGHqkKtw9NoHmbU+aagtte6sNd5/pOYIIToKlIiaCdFUZttHwBPiQCgus7TzVRRVY6d8cxDdPxsmW9Q2a3fi0cfJh+HEKLzSYmgndx+Jo4LM3je4upaz5oEBp324uI1VzAWQgghAk0CQTu53EqLA+y8E8/5AoFe61vO8koGxQkhRKBJIGgnt6LS0kzSF6uGXGg0ntlHdfWNClIiEEIEAwkE7VReVUd4C3X73hJBTZ3LVxLQSdWQECKISCBoB1VVybfaiY0OazaN0eCdaVT1lQS8NUkSCIQQwUACQTuU2eqornXRp4VAoNddbBPQ6S4uZ6nTalA6YcI5IYTwRwJBO5wtqgIgtlfzgQA8YwfgYpWQ97V3kJkQQnQlCQTtcM7imQK2T3R4i+m87QTeWUfBUzqQqiEhRDCQQNAO54qq6Nc73DebaHO87QQNSwRaCQRCiKAggaAdzhZVkWj2v1qat0TgbSMAz0IRMo5ACBEMJBBcoZo6F9ayGq4y+19q0juWQHfJpEQ6rVQNCSGCgwSCK5RvtaNCm0oEet1ljcXSa0gIEQQkEFyhc0WehuLEVpQIwppsI5ASgRAiOLRqusvs7GxWr16Ny+Vi9uzZpKenNzh+7Ngxnn76aex2O0lJSSxZsoSKigrmzJnjS2Oz2SgrK+PAgQPs3buX3/zmNyQkJABw8803s2zZsgBmq+OdLarCFKZvcQyBV1NVQ1qtRtYjEEIEBb+BwGKxsHLlSt577z2MRiMPPPAAt99+OzfeeKMvzYIFC3j++ecZNWoUTz31FBs2bGDmzJlkZWUBoCgKs2fPJiMjA4AjR44wZ84cHn300Q7KVsc7V1TFNfFRLU4459VUY7FOq6FOlRKBEKLr+a0a2r17N8nJycTExGAymUhJSSEnJ8d3PD8/n9raWkaNGgXAtGnTGhwH2LRpExEREb7lLvPy8ti5cyepqanMmzePwsLCQOapwymqyvmiKq6K898+AJcEAhlQJoQIQn5LBEVFRZjNZt92XFwchw8fbva42WzGYrH4tt1uN2+88QarVq3y7YuOjmbSpElMmDCBf/3rX2RkZPDOO++0+qH79m3dF3BTzOboKz7Xq6KqDodL4fqrYjCZwoiOajygzGDQ+/bXuTxf+KYIo29/WJgBtcrhS2MyhWGONbX72ZoTiHx3N6GYZ5B8h5JA5dlvIFCUhvPtq6raYNvf8c8//5zrrruOwYMH+/YtXbrU93rGjBm8/PLL2Gw2oqNbl6mSkqor6oNvNkdjtdrafN7lLpRWe164Faqr67BV1TZK43S6fPtdLs9aBG6X27dfURScLsWXprq6Dqvb3e5na0qg8t2dhGKeQfIdStqSZ61W0+IPaL9VQwkJCVitVt+21WolLi6u2ePFxcUNjm/fvp3Jkyf7thVFYfXq1bgv+9LT6XT+HiVo2Gs8y05GhhtalT7cqKd3pJGYSxqWdTKgTAgRJPwGgtGjR7Nnzx5KS0upqalh27ZtjBkzxnc8MTGRsLAwcnNzAcjKympw/ODBgyQlJV28oVbLRx99xNatWwHIzMxk5MiRmEwdVy0SaPba+kAQ0bo1hnVaDWk/uJ6rL2lTkHEEQohg4TcQxMfHk5GRwaxZs7j33nuZOnUqI0aMYO7cueTl5QGwYsUKli1bxsSJE6murmbWrFm+88+dO+frJuq1fPly/vGPfzBlyhQ2bdrE888/H+BsdSx7jaeqJ6qVJYKmyDgCIUSwaNVP2tTUVF+PH681a9b4Xg8ZMoSNGzc2ee6hQ4ca7Rs0aFCbGoeDTZW3aiiifYFAVT3rFmtbWutSCCE6mIwsvgL2WicawNTCEpX+aGW5SiFEkJBAcAXsNS5M4fp2/ZL3jjKWBmMhRFeTQHAF7LXOVvcYao4sYC+ECBYSCK5AVa2z1T2GmuOdbsKtKFTXuqh1uALxaEII0WYSCK6AvcbZroZiuNhGUFJZR9bOU2z+7D+BeDQhhGgzCQRXwF7jalfXUbhYNbTrcCFOl0KZrS4QjyaEEG3WvvqNEBWYNgJPDNZqNERFGnxdUoUQorNJiaCNFEWlutbV7jaCXpEGok0Gfvj9AfTtHS6BQAjRZaRE0EbVdS5UWj/PUHOiTUbuGzMQ8Cx7aZdAIIToIlIiaCPvF3ZUOxuLLxVm1OFwKdQ5O2b2USGEaIkEgjaqauOEc60RXr9wTVW1lAqEEJ1PAkEbeSeca2/V0KXCjPWBQKqHhBBdQAJBG12cgjpwgSC8PhDYahwBu6YQQrSWBII26pA2AoOnmskmVUNCiC4ggaCNvNU37Zl59HLeEoG0EQghukKrAkF2djaTJ09mwoQJrF27ttHxY8eOMW3aNFJSUnj66ad9a/Ru3ryZu+66i7S0NNLS0li5ciUABQUFpKenM3HiRB577DHsdnsAs9Sx7LUuTGHtm3n0ckaDFo0GbNJGIIToAn4DgcViYeXKlaxbt47MzEzWr1/PyZMnG6RZsGABzz77LFu3bkVVVTZs2ADAkSNHWLRoEVlZWWRlZZGRkQHAkiVLmDlzJjk5OQwbNoxVq1Z1QNY6hj0AE85dTqPREBluoKpa2giEEJ3PbyDYvXs3ycnJxMTEYDKZSElJIScnx3c8Pz+f2tpaRo0aBcC0adN8x/Py8ti8eTOpqak88cQTVFRU4HQ62bdvHykpKY3Sdwf2GldAewx5RUUYpEQghOgSfgNBUVERZrPZtx0XF4fFYmn2uNls9h03m8388pe/5P3336d///4sXbqUsrIyoqKi0Ov1jdJ3B1U1zoA2FHtFRRikjUAI0SX81nEoioJGc7E+XFXVBtstHX/99dd9+x9++GHGjx/PwoULG6QHGm3707dvVJvSX8psjr6i82zVDmpqXdhrncTFmlB1ngZenQGio8IbpTcY9G3a3ysqDGt5zRU/nz8ddd1gFop5Bsl3KAlUnv0GgoSEBPbv3+/btlqtxMXFNThutVp928XFxcTFxWGz2di0aRMPPfQQ4AkQOp2O2NhYbDYbbrcbnU7X6HqtUVJSdUVLPJrN0VittjafB2Cvc7HvmIXyqjrKbbV8lnsWgJE3mbFV1TZK73S62rQ/wqijwlZ7xc/Xkvbku7sKxTyD5DuUtCXPWq2mxR/QfquGRo8ezZ49eygtLaWmpoZt27YxZswY3/HExETCwsLIzc0FICsrizFjxmAymfjLX/7CoUOHAPjnP//J+PHjMRgMJCUlsWXLFgAyMzMbXC+YqaqKw6kQVj8lRCBFReipqnGhqLJ0pRCic/ktEcTHx5ORkcGsWbNwOp1Mnz6dESNGMHfuXObPn8/w4cNZsWIFzzzzDFVVVQwdOpRZs2ah0+l49dVXee6556itreW6667jpZdeAmDx4sUsWrSI1atX079/f1555ZUOz2ggOFwK4OnuGWiREQYU1TPFdUe0QQghRHNa1Q8yNTWV1NTUBpyGqDMAAB4ISURBVPvWrFnjez1kyBA2btzY6LykpCQ2b97caH9iYiJvv/12W5+1y9U5PLODdkyJwPPl31GN0UII0RwZWdwGDmcnBALpOSSE6GQSCNqgzumpGuqIQOCdxM4mg8qEEJ1MAkEbeBeOMXZgiUAGlQkhOpsEgjaodXjmUAozBv5tu7SNQAghOpMEgjYoq6wj3KjrkKoho0GHUa+VqiEhRKeTxevboLiiln69w9s8Ero1NFoNkREGyqoc2Otcvv1hBj16CddCiA4kgaCVaupcVNgdXD+gV4dcv87pRqOBfGsVu/IK2HX4AsMGxjL5juvQB3DtAyGEuJx8w7TSWYtnKHe/3o3nCQqUcKOOOoebAyeKOVdU1aH3EkIILwkErXTmgicQ9O3VcV/OYQYdRWU1FFd45iLy9lISQoiOJIGglc5csBFtMhBmDHxDsVe4UY/LrRIZrsetqNQ6JBAIITqeNEO2gqqqnL5gwxwT0aH3iQjzBJnkofGYwvVSIhBCdAoJBK1QZquj0u6gbwfX2d94VQzjbrmKRHMUYQadb24jIYToSBIIWuFUYSXQsQ3F4GksTjRHAhBm1EmJQAjRKSQQtMJ/CivRaTXERod12j2lRCCE6CwSCFrhVEElieYodLrOe7vCjTocLgX3FazEJoQQbSGBoBUsZTUk9DV16j2901hU18rcQ0KIjtWqQJCdnc3kyZOZMGECa9eubXT82LFjTJs2jZSUFJ5++mlcLs8UCbm5uUyfPp20tDRmz55Nfn4+AHv37uX2228nLS2NtLQ0nnzyyQBmKbAUVaXS7qB3pLFT7+vtpmqvcflJKYQQ7eM3EFgsFlauXMm6devIzMxk/fr1nDx5skGaBQsW8Oyzz7J161ZUVWXDhg2+/c8//zxZWVmkpqby/PPPA3DkyBHmzJlDVlYWWVlZLFu2rAOyFhhV1U7citr5gaC+RGCXEoEQooP5DQS7d+8mOTmZmJgYTCYTKSkp5OTk+I7n5+dTW1vLqFGjAJg2bRo5OTk4HA4ef/xxhgwZAsDgwYMpLCwEIC8vj507d5Kamsq8efN8+4NReVUdAL2juiYQXDottSLtBUKIDuA3EBQVFWE2m33bcXFxWCyWZo+bzWYsFgtGo5G0tDQAFEXhtdde45577gEgOjqaBx98kOzsbMaOHUtGRkbAMhRo5VWeaaF7dVHVkLeN4KzFxryXP6Og2N6pzyGE6Pn8TjGhKEqDaZdVVW2w7e+4w+Fg0aJFuFwuHn30UQCWLl3qOz5jxgxefvllbDYb0dHRrXrovn2jWpWuKWZz6+7hpfynFICEftE01ZvTYNATHdV4fEF794eHexaqqXOpmM3RfPVdCS63wtniakZ+L6FNeYC257snCMU8g+Q7lAQqz34DQUJCAvv37/dtW61W4uLiGhy3Wq2+7eLiYt9xu93OY489RkxMDKtXr8ZgMKAoCm+++SaPPPIIOt3FeXsufe1PSUnVFVWTmM3RWK22Np1zrrACAIMWbFW1jY47na4O2a+qKjqthvLKWqxWG6fOlwNw+EQRyUPMjc5vyZXku7sLxTyD5DuUtCXPWq2mxR/QfquGRo8ezZ49eygtLaWmpoZt27YxZswY3/HExETCwsLIzc0FICsry3d8wYIFXHvttbz66qsYjcb6B9Ly0UcfsXXrVgAyMzMZOXIkJlPnds9srXK7g8hwPYZOXh1Go9EQZtD5Gou9M5J+V1DRqc8hhOj5/JYI4uPjycjIYNasWTidTqZPn86IESOYO3cu8+fPZ/jw4axYsYJnnnmGqqoqhg4dyqxZszh69Cgff/wxN954I/fddx/gaV9Ys2YNy5cv53e/+x2vv/46sbGxvPTSSx2e0StVbqsjJqrzRhRfKsyo83UfLa6oAcBaXktltYNeps5tsxBC9FytmoY6NTWV1NTUBvvWrFnjez1kyBA2btzY4PjNN9/M8ePHm7zeoEGDeOedd9r6rF2iwu4gppN7DHk1KBGU1xLfJwJLWQ3/Kahk1I39uuSZhBA9j4ws9qO8qo7eXVgiqKpx4nC6qbA7SBoSh1aj4T9SPSSECCAJBC1QVJWKKkfXVQ0ZdFTXujhX32W0b+9wEs2RnDhXgUvpkkcSQvRAEghaUFVTP6q4q6qGjJ6qoT15ngF3RWXVRITpOVVQSU2djDgWQgSGBIIWVNQPJuvTRSWCcIMOVYVSm2d0c1SEEXNMOE63woXS6i55JiFEzyOBoAVdNb2EV5jR8/GUVNSi1WqICNP5lss8c6GyS55JCNHzSCBogTcQdGUbAXgCQVS4Ho1GQ7TJQJhBx/5vrCiqzD0khGg/CQQt8M4z1GXdR73zDdW5iDJ5ppzQaDSMGtSPE+fK+WDX6S55LiFEz9KqcQShqqKqrn5UsQ6Hu/PXBfCWCACiIgy+1zdd3RtFVcnaeYrrB/Ri+MC+nf5sQoieQ0oELSjvwq6jcLFEAA0DgUaj4YFxg0g0R7Em+ygut/QlFUJcOQkEl/n0QD5vvv81qqrWDybruqkcDDotWq1nJtdLAwGA0aDjx3deR1WNkzOWixNP2WudVNodnfqcQoi2s5bXUBEkf6sSCC6z40A+Xx618MXXFiqqum6eIfD88o8M99TeXR4IAG5I7A3Ad/kXexC99cExlq39ShqSQ1BReQ3/yZdR592BqqqseOcAf9tyrKsfBZBA0ECl3cG5oio0Gnh3x0nKqxxdWiIAiKxfl8DbWHypPtFh9Osdzsn6P36ny83Xp0uxlFbzzZmyTn1O0fXe+uAoi9fskZXsugFLWQ3W8lqOnSnD6WpioZNOJoHgEsfqvzynjbmB8ioHbkXFFG7AXueiq/62TOF69DpNg4bjS92Y2JuT58tRVZUT5ytw1s898e9DBZ35mKKLldnqOHm+gnJbHd/Wr13RlIqqOv7ntZ18edTSbBrR8b4+5VnwyulS+PZ815fiJBBc4ujpUiLCdESE67i+v2flH2t5DfuOWXApXdMg2y8mgj7RYQ1WfbvUDYm9Ka9yUFpZx9enStFpNfxgRH++OmHFVt36+sei8hrWfnSCnC/PcjK/okMboPOtVb4lOP1xuRX+uPEw6z460WD95lCkqCpltjoKS+xcKK2muvZiT7avTlhRAZ1Ww/5vrM1eY+u+c5RXOXjv39/h7qJ/0z3dkVMlvPD2fmrqmu9pePR0KTFRRnRaDV+fLu3Ep2tayHcf9S6tqaoqR0+XMejqGLQaDbcMjkNRIb5PRJc+371jB3LoROM/bI1Wg73ORWKcZ9Whr8+UcuQ/JdyQ2Ju7Rg7g88OF7DhUwJ3D+vPBF2e5PiGaodf1aXQdlwLFFdW88s5BSitrfSWfaJOBO4YmMHbUAPr3jQxYfk6er2D5uq9IiDXxv+nfb7Lt41IffnmWgyeL0Whg95ELJA9LQANoNDD2vxKJ7xNJJ68Z1OmKymt4I/MI+cV2X4kPICJMz3M/vxVzTAT7vykisV8k1/Tvxf4TRcwYPwjtZT8eqmqcfHogn7g+ERSV1fDF1xbuHN6/s7PToymqyoZPvuO8tYrPDxUw4bZrGqVxKwrfnC3j1iHxXCit5uipMvhh89e0lFYT1yei2R+DgdDD/4RatvNwIf/7xh4ulFZjLa+hpLKWwVd7vixN4XrGjhpARFjXxsowg67JaqE6p5t9xywUWKvQ6zR8+lU+5612briqN/nWKvr1Dmfrl2d59q0veW/HSVZtzqO0svESmZXVdbz8zkHKbHWk3HYN9//oBsaOGsDAAb35OPc8S/62D0uA5jWqrHawOusIvSKNWMqq+cO7h6hraiHoepayarJ3nSZpSByLfnYLvaOMfJp7nh0H8vk49zz//7oDFFd4nq2wxM72/ed6ZFfaDZ+cpLC0mnHfv4r08YOZPWkI6RNuwu1W+OdHJygsrebEuXJuGWzmrpEDqKhycLKJ6obt+89R53Dz6/uGc01cFB/sPt3qUkFhiZ3/8+Ex3t52vMvqtNUg7ADxzZky/r+/7+P4WU+18sFvizlvrSIiTM/23PNNttecKrBRU+dm6PWxDL2uD2cttmZL758dzOfJP3/BjgP5HZqPVgWC7OxsJk+ezIQJE1i7dm2j48eOHWPatGmkpKTw9NNP43J5ikQFBQWkp6czceJEHnvsMex2z3TKlZWVPPLII0yaNIn09PQGax53FFVV2XWowFec9laFFFfUsjrzCIdOlgAw+JqYDn+WQNJqNfTrHcGZC54upN7nv+nqGKprXYQZdIy/7RrcisJfPjjaoDfRWYuNP208TGlFLWNGDcDcJ4KIMD3XJkTzcOrNLHs0GZ1Owz+2Hr/iP0KXAvY6F7YaJ29kfY2t2sHcHw9lbuow/lNYyUv/+opPvzpPQbGdMxdsHPy2mMPflWApreafW49j0GuYMW4QA/pFMv7Wq/lZyk2kT7iJCbdejb3GyWub8tj02Xcs/ute1m3/ljeyvu5RweDb8+V8dcLK5Nuv4Sd330jysHjfetZDB8aS910Jb2YdQQWGDuzLtQm90Os0fHHUQn6JnVWZR/jTpsN8dqiA7fvPM/yGvvTpHc6U0ddjKathd96FJj9bh9PN8bNlbNt7lj9tOswza75kz9cWPv0qnxfXHqCsfiJEr88PF7A684hvIaVAUlWVt7ce58k/f0FxeU3Ar99axeU1PPvWl7y+OY+zFhtfHL3AKxsOcqrQxmvv5WEpq+b9XaeIi4lg9sTBFFfU8lV9af7zwwVs+OQkLrfC16dL0QDfu7YPN18fi8rF9skzF2zUOjzfURV2B+9++h0aYONn3zV6zwNJ99xzzz3XUgKLxcITTzzBhg0bSE9PZ8mSJdx6663Exsb60vz85z9nyZIlLFiwgI8//hir1crw4cN54oknmD59OkuXLuXMmTPs3r2bO++8k+XLl3PTTTexcuVKdDodb7/9NhMnTmz1Q9fUOGjr95K91sXit74g94SVYQNj+T9bvqGy2sHM8Tfx+aFCvjlbRu+oMKaOvo6C+vn/L5XQNxJLSeNfxsGwv7LaSVFZDWEGHfeOGUhRaTV9osPo38/EqBv7kRgXzcD+vfj0QD61DjcllbV8drCAt7eewOVWuGNYAtfERze4fqI5ipjIMEzhBj7OPU+/3hFcEx+Noqgoqtqo2uFyLrdCrcNFSWUtGz/9jg2ffEt+sZ3bbo4nzKBj1KB+XNUvkoPflvDFUQuffJXPZwcL+PKYhS+OWvg49zzW8lruHTOQGxJ7o6hQWGz3FY+jIgz06x3O16dKOX62nFuGxPH9wWY+O1jAuaIq4vuaOJVfQZmtjtjocDQaDbZqBx9+cYbvCiq4yhzV6etQt4VLgeo6J3/JPoaiqjw4cQiKiu99AOjXO5wzhTaKymsx94kgLiYcW42LAmsVx8+W89nBfCyl1ZRU1pJ73IrTrZA0xEy5rY4RN/bl6/+U8tmhArbuPce+b4o4dqaMMxdsfJx7nn9sPc6/DxVy5FQpdU43P/z+VcyeNITrB/Ri5+ECdh2+QK9II1eZI9m+/zxvbztBQbGdo6fKSBoSh7GZzg1tpaoq6z85yce556lzujlwsphbvxdPuLHh9SMjw6huQ5tYa9Q53fVBV0uZrY7l//qKCruDorIatueeJ/e4lYEDevPovcP54ugFdh4u9P2bHTXIzL5viigssVNpd7Bu+7eczK/grMXGmaIqekUauHPEAEzhBj47WICiqHxzrpz/8+E37D9uZcg1MWz+9384b63iN/89gi+OFlFcUcutQ+KuKM8ajQZTC8vbalQ/P/U2b97Mvn37+P3vfw/A66+/jqqq/PrXvwYgPz+f2bNns337dgD279/PH//4R9566y1uv/129u7di16vp7CwkJ/97Gd8/PHH3H333axdu5b+/fvjcrm47bbb+PLLLzEYWq4v9iors19RF7kiWx2r3j1EnUvB7VZI+8FAvn+TmW37zrHrcAEjB5m59wcDyfuuuNG537s+lmOnGjfqBMN+S6mdXXkXSDRH8dCU7zVKHxUZxg0Dolm//VuO1jdMaTXw/cFx/OiWq/iuiWqEkTeZof5L/2//9xjWshqujo/izAUbKnBd/2gS+0ZRXFFDYWk1EUY9A/qZMBp0nC60kW+talD66BMVxk3XxDCgXyQajcZ3fVVVKams45zFRphRR7TJwLfnyrHXOnG5Va7r3wutRtPs+9Arysjp/Epie4cDnl/Qed+VNEjTOyqMGwb04sipUpxONypgCtNzy5A4HE43ZVV1hBv09IkOI8yoparaib3WRUSYnuj6brvVtS4cTjcRYXpMYXpcikKtQ0FVVMLCPNV3LreCw6mg0YDRoEWv1eJWVF/1i1arQYsGjVaDRgOKW8WlKGg0GvRaDYoKtmoHtmonOp2WMlstZy7Y+P5NZq7r36vJfw+W0mp25RVy58gBxMdEEBUZxtH/FLP/myL6RIXxwISbKCy2U1JRi8PpJtEc5ft8bVV1HDtTRklFLaWVtRRX1FJeVUe0ychNV8cw6KreDOgXSbTJiKLi+7uosNex/xsrFVV19O0dTklFLUOu68PIG/ux8dOTmGNMDOwfTXGFpyoytlc4vUwGKqudlFfVYdBp6RMdhtGgpbSyjoqqOkwRRvr1CselKBQW2ym3O+jXOxy9VsuRUyXcdnM8Iwb25e853xDbK5zEfpEUlHgC4oC+kQyIj+b8BZtnzQ6jjvhYE+FGPRdKqymtrKVXpJH+sZG4FIWCEjvltjriY030j42kwl7HWYuNGoebq+OiSOhj4lyRjdMXbGi0Ggb270WprY5Ku4PRwxKINhk4mV+B06kwbGAsQ2/ox78P5LPzcAERYQYm3Ho1Wq2G6joXOV+cAWD4wL5cFR/Fh3s82zddE8Ow6z1Tw3zx9QXfj8+Rg8x8d76CWocLl1vhh/+VSMpt17LjwHm27j3LnCnfY8g1nurrvn2jKCmpavQ30RStVkOfPi209al+vPHGG+orr7zi296wYYP6zDPP+La/+uor9YEHHvBtnz59Wp0wYYJqsVjUH/zgB779TqdTHTp0qKqqqjp06FDV6XT6jv3gBz9QL1y44O9RhBBCdAC/ZWOl/hfLJYGjwXZzxy9PBzTb6q2qKlpt8BbThRCiJ/P77ZuQkNCgMddqtRIXF9fs8eLiYuLi4oiNjcVms+F2uxudFxcXR3Gxp5jpcrmw2+3ExHSvRlohhOgp/AaC0aNHs2fPHkpLS6mpqWHbtm2MGTPGdzwxMZGwsDByc3MByMrKYsyYMRgMBpKSktiyZQsAmZmZvvPGjh1LZmYmAFu2bCEpKanV7QNCCCECy29jMXi6j7755ps4nU6mT5/O3LlzmTt3LvPnz2f48OF88803PPPMM1RVVTF06FCWLVuG0WgkPz+fRYsWUVJSQv/+/XnllVfo3bs35eXlLFq0iHPnzhEdHc2KFSu46qqrOiO/QgghLtOqQCCEEKLnkhZaIYQIcRIIhBAixEkgEEKIECeBQAghQlzIBAJ/E+f1FK+99hpTpkxhypQpvPTSSwDs3r2b1NRUJkyYwMqVK7v4CTvO8uXLWbRoEdD8RIg9ySeffMK0adOYNGkSzz//PBAan3VWVpbv3/jy5cuBnvt5V1VVMXXqVM6fPw80//m2O/9dN6i581y4cEH90Y9+pJaVlal2u11NTU1Vv/32265+rIDbtWuX+tOf/lStq6tTHQ6HOmvWLDU7O1sdO3asevbsWdXpdKpz5sxRd+zY0dWPGnC7d+9Wb7/9dvV///d/VVVV1SlTpqgHDhxQVVVVn3zySXXt2rVd+XgBd/bsWfWuu+5SCwsLVYfDoc6YMUPdsWNHj/+sq6ur1VtvvVUtKSlRnU6nOn36dHXXrl098vM+ePCgOnXqVHXo0KHquXPn1JqammY/3/bmPyRKBLt37yY5OZmYmBhMJhMpKSnk5OR09WMFnNlsZtGiRRiNRgwGAzfccAOnT5/m2muv5eqrr0av15Oamtrj8l5eXs7KlSuZN28e4JkIsba2llGjRgEwbdq0Hpfnjz76iMmTJ5OQkIDBYGDlypVERET0+M/a7XajKAo1NTW4XC5cLhd6vb5Hft4bNmxg8eLFvhkZDh8+3OTnG4h/7yGxQllRURFms9m3HRcXx+HDh7vwiTrGoEGDfK9Pnz7Nhx9+yM9+9rNGebdYetZ6tc8++ywZGRkUFhYCjT9vs9nc4/J85swZDAYD8+bNo7CwkB/+8IcMGjSox3/WUVFRPP7440yaNImIiAhuvfVWDAZDj/y8X3jhhQbbTX2PWSyWgPx7D4kSgb+J83qab7/9ljlz5rBw4UKuvvrqHp33d999l/79+3PHHXf49oXC5+12u9mzZw+///3vWb9+PYcPH+bcuXM9Pt/ffPMNmzZt4tNPP+Xzzz9Hq9Wya9euHp9vaP7fdSD+vYdEiSAhIYH9+/f7ti+fOK8nyc3NZf78+Tz11FNMmTKFvXv3tjhpYHe3ZcsWrFYraWlpVFRUUF1djUajaXIixJ6kX79+3HHHHb4Fou655x5ycnLQ6S4u2NLTPmuAnTt3cscdd9C3r2cu/2nTpvHWW2/1+M8bmp8AtLmJP9siJEoE/ibO6ykKCwv51a9+xYoVK5gyZQoAI0eO5NSpU5w5cwa3280HH3zQo/L+t7/9jQ8++ICsrCzmz5/P3XffzbJly5qcCLEn+dGPfsTOnTuprKzE7Xbz+eefM3HixB79WQMMGTKE3bt3U11djaqqfPLJJ9x22209/vOG5v+Wm5v4sy1CokQQHx9PRkYGs2bN8k2cN2LEiK5+rIB76623qKur48UXX/Tte+CBB3jxxRf5zW9+Q11dHWPHjm3TsqDd1YoVKxpMhDhr1qyufqSAGjlyJA8//DAzZ87E6XRy5513MmPGDAYOHNijP+u77rqLo0ePMm3aNAwGA8OHD+eRRx5h/PjxPfrzBggLC2v2b7m9/95l0jkhhAhxIVE1JIQQonkSCIQQIsRJIBBCiBAngUAIIUKcBAIhhAhxEghEyDt8+DDPPvusb3vOnDmUlpY2mXbu3LmcPHmSL7/8kqlTp7b5Xu+++26rZr+12WwMHz6ctLQ0339ffPFFm+8nRGuExDgCIVpy8uTJBnOz7Nq1q9m0a9asAaCkpOSK7pWbm9tgTqjmHDx4kFtvvZW//vWvV3QfIdpCAoEIGYqi8Pvf/55Dhw5ht9tRVZUlS5bwxz/+EZvNxpNPPulLO3v2bP785z+Tnp7OiBEjOH78OP/zP//DsmXL+MMf/gBAdXU18+fP58yZM/Tq1YulS5dy/fXXs2jRIgYNGsQvfvELAN/2NddcwyeffMKuXbsIDw8nPT2d1atXs23bNhRFITExkcWLFxMfH8+BAwcoLy/nJz/5CQ6Hg5/85CfMnDmzS9430fNJ1ZAIGYcOHaKoqIj169ezZcsW7rvvPv76178yf/58kpKSWLZsGcuWLQPg73//O/379wc8s7p++OGHjB8/vsH1CgsLeeihh8jKymLq1KksXLiwxfuPHz+eu+++m4ceeoj09HQyMzM5ceIE7777LllZWYwdO5ZnnnkGAJ1Ox913380///lP3nzzTf7+97+zffv2DnhXhJASgQgh//Vf/0Xv3r155513OHfuHF9++SWRkZF+z0tKSmpy/+DBg/n+978PwH333cdzzz2HzWZr9fN8+umn5OXl8d///d8Avnn2AX71q1/50sXHx/PTn/6Ujz76iHvuuafV1xeitSQQiJCxY8cOXnjhBX7+858zbtw4Bg4cyPvvv+/3PJPJ1OR+rbZhgVqj0aDX69FoNFw6c4vT6WzyfEVRfPMFATgcDioqKgB4++23GTduHAMGDAA8Uwvr9fLnKjqGVA2JkLFr1y5+9KMfMXPmTIYNG8b27dtxu93odLoGa7xevt2c48ePc+zYMQDWr1/PLbfcQkREBH369OHIkSMAWCwW9u7d2+S177rrLjZu3EhVVRUAf/jDH3zVS7m5ubz11luAZwW2jRs3Mnny5AC8C0I0JoFAhIwHHniAvXv3kpqayn333cfVV1/N+fPnGTFiBOfOnePXv/41ABMnTuTBBx/kxIkTLV5v4MCBvPbaa/z4xz/mk08+8c36+uCDD2K1WklJSeGpp54iOTnZd86YMWN45513ePPNN7n//vv54Q9/yE9+8hOmTJnC8ePHfdd49tlnsVgsTJkyhZ/+9KfMmDGDO++8s4PeGRHqZPZRIYQIcVIiEEKIECeBQAghQpwEAiGECHESCIQQIsRJIBBCiBAngUAIIUKcBAIhhAhxEgiEECLE/T+ZQKYYtY4wAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute5'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11d1c9748>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEJCAYAAACzPdE9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzde3iU9Z34/fecZ5JJCAkzOXEQFU8QxBoUXBvWthJIQBYWnrpQ6G53cddnd/FyW1YQF5du6f7gxyWsVmm3XdunrGylVoO0SaDbNrIrqAFRRKPIKUBCMkkmJJnJHO6ZuZ8/JjMSkpDTHZIJn9d1ecncx2/uZOYz39Pnq1NVVUUIIYQYJP1wF0AIIcToIAFFCCGEJiSgCCGE0IQEFCGEEJqQgCKEEEITElCEEEJoQgKKEEIITRiHuwDDrbnZSySi/VScjAw7TU0eza+rNSmnthKhnIlQRpByak2rcur1OsaOTe52X58Cyr59+9i5cyehUIhvfvObrFixotP+qqoqNmzYgNfrJT8/n02bNmE0GqmtrWXt2rU0NTUxefJktm3bRnJyMq2trXznO9/hwoULpKens2PHDhwOB8FgkA0bNnDixAmsVivbtm3jlltuid/ns88+4x/+4R/4zW9+E9/28ssvs2fPHlRV5dvf/jZz587t18OJRNQhCSixaycCKae2EqGciVBGkHJqbajL2WuTV319Pdu3b2f37t2UlJTw6quvcurUqU7HrF27lo0bN7J//35UVWXPnj0AbNq0ieXLl1NeXs60adN46aWXANixYwf5+fmUlZWxbNkyNm/eDMCuXbuw2WyUlZXx9NNPs379+vg9SkpK+Ku/+it8Pl982/Hjx3nzzTfZu3cvu3fvZuvWrVy+fHnwT0UIIUS/9RpQDh06xKxZs0hLSyMpKYnCwkLKy8vj+2tqavD7/cyYMQOAJUuWUF5ejqIoVFZWUlhY2Gk7QEVFBQsXLgRgwYIFHDx4EEVRqKio4JFHHgFg5syZuN1uamtraWtr43e/+x3PPfdcp7IdPHiQhx9+GIvFQkZGBvfddx8VFRWDfypCCCH6rdcmL5fLhcPhiL92Op0cP368x/0Oh4P6+nqam5ux2+0YjcZO268+x2g0Yrfbcbvd3V6rrq6OnJwcXnjhBS5evNilbHl5eV2O74+MDHu/ju8PhyNlyK6tJSmnthKhnIlQRpByam2oy9lrQIlEIuh0uvhrVVU7ve5p/9XHAV1eX3mOXq/vck5s+7XKdrVrHd+dpibPkLQrOhwpNDS0aX5drUk5tZUI5UyEMoKUU2talVOv1/X4RbzXT9+srCwaGhrirxsaGnA6nT3ub2xsxOl0kp6eTltbG+FwuMt5TqeTxsZGAEKhEF6vl7S0NDIzM3G5XF2uNdCyCSGEuH56DSgPPPAAhw8fxu124/P5OHDgAAUFBfH9ubm5WCwWjh49CsDevXspKCjAZDKRn59PaWkpEO1Uj503Z84cSkpKACgtLSU/Px+TycScOXPYu3cvAEeOHMFisZCTk9Nj2QoKCjhw4AA+nw+3280777zD7NmzB/gohBBCDEavTV6ZmZk8+eSTrFq1CkVRWLp0KdOnT2f16tWsWbOGvLw8tm3bxjPPPIPH42Hq1KmsWrUKgGeffZZ169axc+dOsrOz453qTzzxBOvWraO4uJiUlBS2bdsGwMqVK9m4cSPFxcWYzWa2bt16zbJNnz6dRx55hKVLlxIKhVizZg2ZmZmDfSZCCCEGQHejL7AlfSiju5yhCASUUJftFpMR4xDkiUiE55kIZQQpp9auRx/KDT9TXoxuASVEZVV9l+0z78zEaJE/fyG0JLm8hBBCaEICihBCCE1IQBFCCKEJCSjihhFRVfzBrh30QghtSEARN4zPL7TwxsGzhMJdMywIIQZPAoq4YTS3+VFCEVq9weEuihCjkgQUccPw+KLNXRJQhBgaElDEDcPrUwBobVeGuSRCjE4SUMQNQVVVPLGAIjUUIYaEBBRxQ/AHw4Q7UuxIQBFiaEhAETcErz9aO0myGmnxBrnBU9gJMSQkoIgbQqxDPicjGSUUiTd/CSG0IwFF3BBiASR7XBIA9W7fcBZHiFFJAoq4IXh9CmajnnFjrAA0XG4f5hIJMfpIQBE3BI9PIdlmItlmQq/TSQ1FiCEgAUXcELw+BXtHMElJNuG6LAFFCK1JQBGjXmwOit1mAmBMshlXszR5CaE1CShi1AsoEUJhlWRbdIXGlCQzjZf9hCOSJFIILUlAEaNeLOXKlTWUcESlqcU/nMUSYtSRgCJGvdiQ4eSOgJKaHP1/nVuavYTQkgQUMerFZsnbrdFAkpJkBsDVLB3zQmhJAooY9Tw+BZNBj9kU/XO3mAwAtAdk9UYhtCQBRYx6Hl+IZJsRnU4HgF6vw2zU45OAIoSmJKCIUc97xZDhGKvZKAFFCI1JQBGjntevkGS9KqBYDLQHwsNUIiFGJwkoYlQLKmGCSoRkq7HTdptFaihCaE0Cikh4EVXl5+WfUl3X1mXfZU90Ma2kqwOK2SABRQiNGXs/RIiRzd3qp+KDWkxGA5OyUjrtu+wJAJDcpcnLSItX5qEIoSWpoYiE526NBo3qutYu+y63RfddXUOxSg1FCM1JQBEJL5ZCpdrlIXLV0r6xGkqXJi+LEZ90yguhKQkoIuE1tUYDSiAY7jL7/bIniNmkx2jo/KduNRsJKGFJECmEhiSgiITnbv0iyePVHfOXPQGSLF27Cm2W6Gx5qaUIoZ0+BZR9+/ZRVFTE3LlzeeWVV7rsr6qqYsmSJRQWFrJhwwZCoWjbdG1tLStWrGDevHk8/vjjeL1eAFpbW3nssceYP38+K1asoKGhAYBgMMjatWuZP38+ixcv5vTp00B0PYstW7Ywb948ioqKOHr0aPzeP/nJT5g/fz5FRUX89Kc/HdzTEAmpsdXPeIcdo0FHdf1VAaUt0KVDHqJNXgB+6UcRQjO9BpT6+nq2b9/O7t27KSkp4dVXX+XUqVOdjlm7di0bN25k//79qKrKnj17ANi0aRPLly+nvLycadOm8dJLLwGwY8cO8vPzKSsrY9myZWzevBmAXbt2YbPZKCsr4+mnn2b9+vUA7N+/n9OnT1NaWsqLL77I+vXrCYVCVFdXs3v3bt544w1ee+01fv7zn1NdXa3pAxIjn7s1gHOsjfEOe/c1FGvXGorVLPm8hNBarwHl0KFDzJo1i7S0NJKSkigsLKS8vDy+v6amBr/fz4wZMwBYsmQJ5eXlKIpCZWUlhYWFnbYDVFRUsHDhQgAWLFjAwYMHURSFiooKHnnkEQBmzpyJ2+2mtraWt956i6KiIvR6PZMnTyY7O5tjx44RiURQFIVAIEAwGERVVYxGGQl9I1HV6LomGalWJmWlcL6+DbWjYz4UjtDWrvQQUKLbZKSXENrpNaC4XC4cDkf8tdPppL6+vsf9DoeD+vp6mpubsdvt8Q/42ParzzEajdjtdtxud7fXqqurw+Vy4XQ6u2yfPHkyCxYs4Ctf+QoPPfQQc+fOJTc3d6DPQiQgrz9EQAmTkWphUmYKXn+Ixo5RXz0NGYYvmrykD0UI7fT6dT4SicSztEL0G+GVr3vaf/VxQJfXV56j1+u7nBPb3t099Ho9Bw8e5MSJExw8eBBVVVm9ejWlpaUUFRX14UePysiw9/nY/nI4Uno/aARI5HK21bQAcNOEsTjSbLD/M5p9Ie6akoKrLTpLflxaEil2a6fzxo6xAWC0GDX/+RPheSZCGUHKqbWhLmevASUrK4sjR47EXzc0NHSqLWRlZcU71QEaGxtxOp2kp6fT1tZGOBzGYDB0Os/pdNLY2EhWVhahUAiv10taWhqZmZm4XC4mTpzY6VpZWVm4XK4u9ygtLaWwsJDk5GQg2nxWWVnZr4DS1OQhElF7P7CfHI4UGhq6pgIZaRK9nKfONQFgQsVu0mHQ6/jopIvbslM4d7EZAJ2q0ua5arnf9GhAqW/waPrzJ8LzTIQygpRTa1qVU6/X9fhFvNcmrwceeIDDhw/jdrvx+XwcOHCAgoKC+P7c3FwsFkt85NXevXspKCjAZDKRn59PaWkpACUlJfHz5syZQ0lJCQClpaXk5+djMpmYM2cOe/fuBeDIkSNYLBZycnIoKChg3759hMNhqqurOXfuHHl5edxxxx384Q9/IBwOoygKBw8eJC8vbxCPSiSa2ByUjFQrJqOBnHHJ8ZFesRn03fahdAwb9gelD0UIrfRaQ8nMzOTJJ59k1apVKIrC0qVLmT59OqtXr2bNmjXk5eWxbds2nnnmGTweD1OnTmXVqlUAPPvss6xbt46dO3eSnZ3Nc889B8ATTzzBunXrKC4uJiUlhW3btgGwcuVKNm7cSHFxMWazma1btwIwb948jh8/Hu+w37x5M1arlWXLlnHmzBmKi4sxGo3MmTOHxYsXD8mDEiNTU6sfk1FPSlJ0aPAtOakc/riegBKmuS2AxWTAZOz6vclk0GPQ62SUlxAa0qmqqn17TwKRJq/ELudLJSe44PLwr4/NAqCqupn/+1/H+OtHpnL0MxcXGjwU3jexy3kz78xk/Q8Pk3+Hk1WFtw95OUeSRCgjSDm1dj2avGSMrUho0SHDlvjr2yemMTbFwjsf1+HxKaTZLT2emyRrogihKUm9IhKauzU6ByVGr9Nx/52ZnDjrps7dzphrBBRZZEsIbUlAEQlLCUVo8QY7BRSAWVMzCUdUvP4QY+3mHs+3WSSFvRBakoAiEpa7LTrCK/2qgDLBaSdnXHQo+bWavKSGIoS2JKCIhOXumBGfMaZzQNHpdMy6KxOAtBQJKEJcLxJQRMJqjM9B6Ro0Cmbk8GBeNjfnjOnxfJvFSLukXhFCMxJQRMJq9UZTq3TX8Z6aZOZbxXd2O6kxxmYx4g+EuqzyKIQYGAkoImH5g2EMeh3mbiYu9oXNYkAlutKjEGLwJKCIhOUPhLGaDT0mHe3NFxmHpR9FCC3IxEaRsHzBUHxdk1AEAkrXwHCtJAhJElCE0JQEFJGw/MFwPMljQAlRWVXf5Zi7b3N02RYja6IIoS1p8hIJyxcIYTMP/DtRLKBIgkghtCEBRSQsfzAcXxt+IGwd50qTlxDakIAiEk4oAt5AiPaAgsmoxxsIXbOvpCfxJi9ZE0UITUhAEQkn1l/S6lVoa1eorKonFIn0+zoyyksIbUlAEQkrFIp0u3hWX0WHHEtAEUIrElBEQlJVFSU8uICi0+mwmY34/DLKSwgtSEARCSkUjnaaDCagQHS2vIzyEkIbElBEQlJC0VqFyTDYgGLEL53yQmhCAopISEoo2gk/+BqKpLAXQisSUERCGmxA0el1eAMhzCYDHn8IbyD6X6j/g8WEEB0k9YpISEp4cAEloIT58GQDvkCI5lZ/PG3LzDszMVrkbSHEQEgNRSQkrZq8LCYDAUVGeQmhBQkoIiFpFVCsZgOhsEooLG1dQgyWBBSRkLQMKBDNCyaEGBwJKCIhxQPKIIcNWySgCKEZCSgiISmhCHod6PUDW60xJrZAV0DmoggxaBJQREJSwhGMRv2Al/+NkSYvIbQjAUUkJCUUwWwc+FooMRJQhNCOBBSRkJRQBKNhcLUTiHbq63QQkIAixKBJQBEJSQlFMGlQQ9HpdFjNBvwyF0WIQZOAIhKSMsi1UK5kMRmkyUsIDUhAEQlpsGuhXMlqNsooLyE00Kd35L59+ygqKmLu3Lm88sorXfZXVVWxZMkSCgsL2bBhA6FQ9M1ZW1vLihUrmDdvHo8//jherxeA1tZWHnvsMebPn8+KFStoaGgAIBgMsnbtWubPn8/ixYs5ffo0EF1MacuWLcybN4+ioiKOHj0av/eePXtYvHgxhYWF/PjHPx7c0xAJQwmFNQwoUkMRQgu9viPr6+vZvn07u3fvpqSkhFdffZVTp051Ombt2rVs3LiR/fv3o6oqe/bsAWDTpk0sX76c8vJypk2bxksvvQTAjh07yM/Pp6ysjGXLlrF582YAdu3ahc1mo6ysjKeffpr169cDsH//fk6fPk1paSkvvvgi69evJxQKceTIEV5++WV+/vOf8/rrr/PLX/6yS9nE6KSEIoOe1BhjkYAihCZ6fUceOnSIWbNmkZaWRlJSEoWFhZSXl8f319TU4Pf7mTFjBgBLliyhvLwcRVGorKyksLCw03aAiooKFi5cCMCCBQs4ePAgiqJQUVHBI488AsDMmTNxu93U1tby1ltvUVRUhF6vZ/LkyWRnZ3Ps2DHKyspYvnw5KSkpJCcn8/LLL5OVlaXtExIjTiSiEgqrmtZQlFCEcETV5HpC3Kh6fUe6XC4cDkf8tdPppL6+vsf9DoeD+vp6mpubsdvtGI3GTtuvPsdoNGK323G73d1eq66uDpfLhdPp7LK9uroat9vNX/7lX7Jo0SL+8Ic/YLfbB/osRIKIZQfWMqCADB0WYrB6XfghEol0mo2sqmqn1z3tv/o4oMdZzaqqotfru5wT297dPfR6PeFwmPfff5+dO3cSCoX4xje+wZQpU5g1a1YffvSojIyhC0AOR8qQXVtLiVbOJq8CQEqyhRS7FQCTyRj/95X6sj0t1QaAwWggKcmCIz1Jk3KOZIlQRpByam2oy9lrQMnKyuLIkSPx1w0NDZ1qC1lZWfFOdYDGxkacTifp6em0tbURDocxGAydznM6nTQ2NpKVlUUoFMLr9ZKWlkZmZiYul4uJEyd2ulZWVhYul6vLPcaNG8fUqVNJTk4G4Mtf/jIfffRRvwJKU5OHyBA0dTgcKTQ0tGl+Xa0lYjmbW9oBCIfDtHn8AChKKP7vK/Vlu9qRut7d0k57e4CG8MBrKonwPBOhjCDl1JpW5dTrdT1+Ee+1zeCBBx7g8OHDuN1ufD4fBw4coKCgIL4/NzcXi8USH3m1d+9eCgoKMJlM5OfnU1paCkBJSUn8vDlz5lBSUgJAaWkp+fn5mEwm5syZw969ewE4cuQIFouFnJwcCgoK2LdvH+FwmOrqas6dO0deXh4PPfQQv/3tbwkGg/j9ft555x2mTZs2iEclEkGsaUrrJi/pmBdicHqtoWRmZvLkk0+yatUqFEVh6dKlTJ8+ndWrV7NmzRry8vLYtm0bzzzzDB6Ph6lTp7Jq1SoAnn32WdatW8fOnTvJzs7mueeeA+CJJ55g3bp1FBcXk5KSwrZt2wBYuXIlGzdupLi4GLPZzNatWwGYN28ex48fj3fYb968GavVSlFREefPn2fx4sWEQiEWLVrE7Nmzh+RBiZEj9sGv5SivK68rhBgYnaqqN/TQFmnySrxyvn2ijv/49ScseGAS6anRfpC7b3Pw4cmGLuf1ZXtEVfnP/SeZfksGqx+ZSvIg1pRPhOeZCGUEKafWRkSTlxAjjb9jVrtWTV56na4j/YrMlhdiMCSgiISjdR8KyGx5IbQgAUUkHP8QBBSL2SDzUIQYJAkoIuH4g2H0Oh0GvcY1FElhL8SgSEARCccfDGlaO4FoCnupoQgxOBJQRMLxB7XLNBxj7WjyitzYgx6FGBQJKCLhBIYkoBhRgXa/jPQSYqAGPuBeiGEyJE1eHZMbvX4Fb8DUeZ/JiMa3E2JUkoAiEs5QNHnZbdG3wtlLrZyv6zz5a+admRgHMdlRiBuFfO8SCccfDGuWdiXGkWbDbjPxzok6Ta8rxI1EAopIOL5ACLNJ2z9dnU7HbRPGcLqmlcuegKbXFuJGIQFFJBRVVWn3K5hNBs2vfUvuGAx6HScvXNb82kLcCCSgiITiD4aJqNF5I1qzWYxMv3Ucp2taCXWskSKE6DsJKCKheP3R1Rq1bvKKmZ2XhRKKcPZS65BcX4jRTAKKSCixeSJmo/Y1FICbc1JJT7Vw9LMG2tqDQ3IPIUYrCSgioXg7AspQNHlBtHO+4O4cAP7wfg1KSJq+hOgrCSgioXh9Q9vkBZCabGbOjBxavEHe/ujSkN1HiNFGAopIKO2BjiavIaqhxGRnJDNtcjrn6z14OoKYEOLaJKCIhBLrlB+qJq8rOdJsADQ0+4b8XkKMBhJQREJp94fQ68Bo0A35vVKTzQC4mtuH/F5CjAYSUERC8fpDJFlN6HRDH1DsNhM6HbguSw1FiL6QgCISSrtfIcl6fRI16vU6UpLMuNwSUIToCwkoIqF4fQpJ1zHzb2qSSWooQvSRBBSRULz+ELbrVEOBaD9KQ7NPVnIUog8koIiE0u4PXd8aSrIZJRyhuVUyEAvRG1k1aIi0tQfxBrouJyur/w2O9zr2ocAXI73qmtvJGGO9bvcVIhFJQBkiPn+Iyqr6Lttl9b+Bi6gq7YHoKK/rJTWpI6A0tTP1pvTrdl8hEpF8somE4Q+EUFWua5OXzWLAYjJQ7+77XJRQBAKK1E7FjUcCikgYscSQ17PJS6fT4Rxro64fkxsDyhe106OfNdDqDfLH9+Rw311ZUjsVo5p8XxIJI5a63nadP5QdY239qqFc6dylVi64PLIKpLghSEARCSOWxyv5OtZQADLH2mhs8fc7lb0vEMLrD2HQ6zj6WQPuVv8QlVCIkUECikgY7fEmr+vXKQ/gHJuEqvY/BUtDx/Gzp2UB8F+/PYkq81nEKCYBRSQMT0cN5Xo3eTk7sg73t9mrqcWPTgcTM+3cM8XBp+cvc66ubSiKKMSIIAFFJIz2YeiUBxjXEVBc/Uxj39DiZ2yKBaNBz3hnMgAXXB7NyyfESNGngLJv3z6KioqYO3cur7zySpf9VVVVLFmyhMLCQjZs2EAoFH3j19bWsmLFCubNm8fjjz+O1+sFoLW1lccee4z58+ezYsUKGhoaAAgGg6xdu5b58+ezePFiTp8+DYCqqmzZsoV58+ZRVFTE0aNHu5RhzZo1vPDCCwN7CiIheP0KBr0O83Uee5tkNWKzGGhq6XsfiKqqNLX4GTcmGozsNhMmo56aBu9QFVOIYdfrO7O+vp7t27eze/duSkpKePXVVzl16lSnY9auXcvGjRvZv38/qqqyZ88eADZt2sTy5cspLy9n2rRpvPTSSwDs2LGD/Px8ysrKWLZsGZs3bwZg165d2Gw2ysrKePrpp1m/fj0A+/fv5/Tp05SWlvLiiy+yfv36eNACeO2113j33Xe1eSJixGr3h0i2XZ/U9VfLSLXS1I9O9RZvECUUwZEWnV2v0+nIzkiiplFqKGL06jWgHDp0iFmzZpGWlkZSUhKFhYWUl5fH99fU1OD3+5kxYwYAS5Ysoby8HEVRqKyspLCwsNN2gIqKChYuXAjAggULOHjwIIqiUFFRwSOPPALAzJkzcbvd1NbW8tZbb1FUVIRer2fy5MlkZ2dz7NgxAM6fP88bb7zB17/+dQ0fixiJvP7QdR/hFZORaqWxHzWUxsvRY69M15KdkSw1FDGq9frudLlcOByO+Gun08nx48d73O9wOKivr6e5uRm73Y7RaOy0/epzjEYjdrsdt9vd7bXq6upwuVw4nc4u20OhEBs2bGDTpk2UlZUN6AFkZNgHdF5vXO52Uuxdcz8lJVlwpCcNyT0HyuFIGe4i9IkSjjDGbiEpydLl2ZpMxm6ftxbbk5IsjM9K5VTNhT49q6QkCy3tCmaTnvGZqfEa1aTsVN79pB6zzcwYu6XX6wylRPmdSzm1NdTl7DWgRCKRTk0Mqqp2et3T/quPA3psqlBVFb1e3+Wc2Pbu7qHX63nhhRd4+OGHufXWW/vwo3avqclDJDIEQzkNBto8Xb/RtrcHaAiHtb/fADkcKTQ0jPyRRw5HCpdbA4yxm2lvD3R5tooS6vZ5a7G9vT1AklmP1x+i+kLzNQcFOBwptLcHqG3wkJFqxeP9IkvxuDHRIHL803rumDT22j/wEEqk37mUUztalVOv1/X4RbzXgJKVlcWRI0firxsaGjrVFrKysuKd6gCNjY04nU7S09Npa2sjHA5jMBg6ned0OmlsbCQrK4tQKITX6yUtLY3MzExcLhcTJ07sdK2srCxcLleXe7zwwguYzWZ+9atf0djYCIDNZuOv/uqv+vN8RILw+hWyxw1P7S4jNVpraWr1k2S9dq02qIS57AkwbXLnZJI5GdGRXjWN3mENKEIMlV77UB544AEOHz6M2+3G5/Nx4MABCgoK4vtzc3OxWCzxkVd79+6loKAAk8lEfn4+paWlAJSUlMTPmzNnDiUlJQCUlpaSn5+PyWRizpw57N27F4AjR45gsVjIycmhoKCAffv2EQ6Hqa6u5ty5c+Tl5VFeXs6bb77J3r17efTRR3n00UclmIxi7f4Qydd5UmNMrC+kLyO96pt9qCqkp3ZuOktNNpNsNVLTIB3zYnTqtYaSmZnJk08+yapVq1AUhaVLlzJ9+nRWr17NmjVryMvLY9u2bTzzzDN4PB6mTp3KqlWrAHj22WdZt24dO3fuJDs7m+eeew6AJ554gnXr1lFcXExKSgrbtm0DYOXKlWzcuJHi4mLMZjNbt24FYN68eRw/fjzeYb9582asVlmb4kYSiaj4AsPXKT/uihpKb1wdiSRTkzsHP51OR+64ZC42Sse8GJ369O5cuHBhfFRWzI9//OP4v++44w5ee+21Lufl5uaya9euLtvT0tL44Q9/2GW7xWJhy5YtXbbrdDqeeuopnnrqqR7L+Pd///fX/BlEYmv3K6hc/7QrMSnJZowGfZ9qKLGUKykda6lcKddh551P6rvtYxQi0clMeZEQ2tqHJzFkjF6nIyPVQmOfaig+kqxGjIaub6/xjmR8gRDNbbKksBh9JKCIhODxBQGGrQ8Fov0ofaqhNPviKz1eLdcR7dC/KPNRxCgkAUUkhFgerbSU7j+or4e+zpZ3XfZ16T8B0Ol1pKVGhw6frWvFGwjRz4z4QoxoElBEQjhb04K+o1N7uGSMsdLqDaKEep5H1OoN0u4PkZrcNfAFlDCfnHWTZDFy/FQjlVX13S4VLESikoAiEsLZ2layM5IwGQ3DVoYv5qL03P9R2zEkuKcmL4D0VAvua1xDiEQlAUUkhDO1LUxwDk2anL4a14e5KLE5Jt3VUGLSU63x5JFCjCYSUMSI5/EpNF72DXtAyejDXJTaRi96XTRdfY/X6QhMMtJLjDYSUIZIJKLS3BYgIku+DlpsUaoJmcMbUNJSLOh0XDPrcE2Dh4wxNvT6nueYpHd0zMsa89rS3tsAACAASURBVGK0GZ5B/aOY16/w28oL/O9Hdbhb/RTcnc1N2anDXayEFg8ozuHN6Go06BmbYrlmk1dtgwfHWNs1r5NkMWI1G/q1vooQiUBqKBo78N4F3nz7HM6OFPVev4ziGawL9W2MTbEw5hr9EtfLtYYOq6pKbaM3vgZ9T3Q6nXTMi1FJAorGWtuDpCaZ+M7yL6HTQSA4clLVJ6oLLg+Tc8YMdzEAoqsuNni6bcq87AkSCIZx9lJDgWhguuwJSMe8GFUkoGjMe8UytRaTgYAiAWUwQuEINY1eJueMjGbDm3PG4PWHqHe3d9lX17GtLwElPdWKqsIlSRQpRhEJKBrz+pR4ehCLyUBQAsqgXGpqJxxRR0wN5dbcaDlO1bR02RcLMo5emrzgi475WP+QEKOBBBSNef1KfEU/s8lAQJEmjcE4Xx9dYe7m3OELKDq9Dm8ghDcQIsVuxmYx8tn5y13Sppy91EqS1UhaSu/L+9ptJsxGPRdcI3+lPyH6SkZ5aazdHyJ3XHR4q8Wkl075Qbrg8mAy6skZl4zbPTzNQwElzIcnv1iVdGyKhU/OuQkoIYyW6FtIVVVOnHUz4zYH+j6kpY92zFulhiJGFamhaMzrV0i2RT9kpA9l8D6tbmZyVgqGblLBDxdHmpXLniC+wBdfFmoavDS3Bbj3jsw+XydjjIXaRi/t8qVDjBIj5106CoQjEXyBMPZYH4pZ+lAGo7ktwHmXh+m3jhvuonQS6yOprvuiueqjM00A3HuHs8/XmZSVQiis8t6n9doWUIhhIgFFQ7Fvmlf2oYTCKuGI9KMMxIenGwG4+5aMYS5JZ+PSoqlTzl5qjW/76EwT4x12Msb03iEfk5FqJTsjif89fknzMgoxHCSgaCjWX5Jsi43yij7eQFACykAcP9VERqqVnGFMWd8ds9FAmt0cDyi+QIjPL7aQd3N6v66j0+mYNTWLM7Wt1MjwYTEKSEDRkNcXW6b2i2HDgDR7DYASCvNJtZu7JqfTHgzjcrfHR1pFRkB6NEeajXOX2oioKlXVzYQjKnk3978mNfNOJwa9jrelliJGARnlpaF4DeWKJi9AOuYH4NPzlwkqEcxGHZVV9aTYrbR5oilP7r7NMcyliwaUzy+28JNff4LHp2A1G7h1fP+HNqckmZl+SwaHPq5jyZybu12HXohEIQFFQ15/Rw3F1rmGIgGl/z481YjZqCerIyfaSHNTdgo6nY73T7rwBcLce5tjQMFAp9cx885Mjn3eyJHPGsjr6C+ymIwYJbaIBCMBRUNfNHl9MWwYkMmN/aSqKsdPN3HbxLQRNVz4SkaDnke/NoWVc2/j43NuJg4wE3JACdPuVzAb9fzu6AX8wWgtd+admfE5LkIkipH5bk1Q3qtGeVnM0ocyEG0+hcYWP1MmpA13UXplNhm4Z4ojvmjWQOj1OnIdyVx0eYmMhA4iIQZIAoqGvH4Fm8WAQR99rEaDTjIOD0BsvZFxg/iQTjQTMlMIKGEaLvuGuyhCDJgEFA15faH4CC9AMg4PUCygpKeM7IByZY6v2H8ud/uARqHljktGr9NJKhaR0KSRVkNev9IpoIBkHB6I2BK76alWLjWN3PkZV+f4AkixW7k5p//9KSajnuyMJM7Xe7j39uEfxSbEQEgNRUPt/lC8/yRGMg73X1OrH5vF0OVZjnYTMu14fAqXPcHhLooQAyIBRUPRxJBX11D00uTVT00tfjJSR3Zz11AY74hmqZZmL5GoJKBoyOtTsF/1rdpilj6U/mq8QQNKktXIuDFWahokoIjEJAFFI6qq4vWHSJI+lEFravUzrh9JFkcT51gbTa0BQmFpJhWJRwKKRgJKmHBEja+FEiMZh/un3a/gC4QGNa8jkWWMsRKJqLLWvEhIElA04vXF8nh17UMByTjcV7ERXjdqQInNvamul6WBReLpU0DZt28fRUVFzJ07l1deeaXL/qqqKpYsWUJhYSEbNmwgFIp+uNbW1rJixQrmzZvH448/jtcb/dbV2trKY489xvz581mxYgUNDdGhl8FgkLVr1zJ//nwWL17M6dOngWhz0pYtW5g3bx5FRUUcPXoUgHA4zLPPPsuCBQsoLi7mZz/72aAfyEDF83hd3YciGYf7pan1xpvUeCW7zYTFZOB8vfSjiMTTa0Cpr69n+/bt7N69m5KSEl599VVOnTrV6Zi1a9eyceNG9u/fj6qq7NmzB4BNmzaxfPlyysvLmTZtGi+99BIAO3bsID8/n7KyMpYtW8bmzZsB2LVrFzabjbKyMp5++mnWr18PwP79+zl9+jSlpaW8+OKLrF+/nlAoxOuvv87ly5d58803+eUvf8mePXv4+OOPNX1AffVFpuHONRTJONw/8RrKDdgpD9HJsBljrJ1WgxQiUfQaUA4dOsSsWbNIS0sjKSmJwsJCysvL4/tramrw+/3MmDEDgCVLllBeXo6iKFRWVlJYWNhpO0BFRQULFy4EYMGCBRw8eBBFUaioqOCRRx4BYObMmbjdbmpra3nrrbcoKipCr9czefJksrOzOXbsGFOmTOFv//Zv0ev1JCUlMWHCBC5dGp51JeKJIbsMG5aA0h9NLX7MRj0pSabeDx6lxo2JTuiUlD0i0fQaUFwuFw7HFzN3nU4n9fX1Pe53OBzU19fT3NyM3W7HaDR22n71OUajEbvdjtvt7vZadXV1uFwunE5nl+0zZszgtttuA+D999/n+PHjzJw5c0APYrDaA53XQomJJYiUyY1909TqJz3Vik6nG+6iDJuMMVZUVfpRROLpdSpyJBLp9OZWVbXT6572X30c0OOHhKqq6PX6LufEtnd3D73+i1hYWVnJk08+ybZt2xgzpn+LHGVk2Pt1fI86yjNp/FisFiMudzspdisWS8c3bZ2OFLuVpCQLjhG2xofDMbDU60OhpV0hx2HH4UhB7XiGMbF/m0zGTtt72qbV9ut97UlGA7xfQ0NbkD8aot/NSPqdX4uUU1tDXc5eA0pWVhZHjhyJv25oaOhUW8jKyop3qgM0NjbidDpJT0+nra2NcDiMwWDodJ7T6aSxsZGsrCxCoRBer5e0tDQyMzNxuVxMnDix07WysrJwuVxd7gFw4MAB/vmf/5nt27dz//339/sBNDV5NEkZXt/kwWjQ0drSTptOBwYDbR5/R5CENk+ANo+f9vYADeGR05ThcKTQ0DByvgnXNXrJvT2JhoY22gOh+CqNV67YqChfbI/pbptW2/tzbIrdqsm10+xmTpxq4I/ucnbZN1gj7XfeEymntrQqp16v6/GLeK9NXg888ACHDx/G7Xbj8/k4cOAABQUF8f25ublYLJb4yKu9e/dSUFCAyWQiPz+f0tJSAEpKSuLnzZkzh5KSEgBKS0vJz8/HZDIxZ84c9u7dC8CRI0ewWCzk5ORQUFDAvn37CIfDVFdXc+7cOfLy8jh+/Dj//M//zMsvvzygYKKldn8003B3tTLJONw3gWAYj0+5YTvkrzQpK5Wzl1qHuxhC9EuvNZTMzEyefPJJVq1ahaIoLF26lOnTp7N69WrWrFlDXl4e27Zt45lnnsHj8TB16lRWrVoFwLPPPsu6devYuXMn2dnZPPfccwA88cQTrFu3juLiYlJSUti2bRsAK1euZOPGjRQXF2M2m9m6dSsA8+bN4/jx4/EO+82bN2O1Wtm5cyfhcJinnnoqXt41a9bw1a9+Vdun1Aden9JjMkOZLd83N/qQ4StNzLTz4alGPD4Fu+3GHaAgEkuf0rkuXLgwPior5sc//nH833fccQevvfZal/Nyc3PZtWtXl+1paWn88Ic/7LLdYrGwZcuWLtt1Oh1PPfVUp8ABsHPnzr4U/7rw+kNdRnjFSMbhvrnRJzVeaXJOKgCnLrYwY8q4YS6NEH0jM+U14vUr2K3dBxSr2RBfK1z0LFZDkSYvuCkrFaNBz6fnm4e7KEL0mQQUjXh9XddCibFZDPgC0uTVm6YWPwa9jjS7ZbiLMuxMRj235qbyabUEFJE4JKBo5Fpt3TaLkYAS1mQ02WgUioA3EKKuuZ2xKRZ8ShhvIDSgpXRHkzsmjeWCy4OnY9LsUIv9Hq7+LySttaKPbqwl8YaIPxgioIQZk2zudr/NbIwfJ7oKKCEqq+q5UN+G0aCnsio6Afbu227spXDvnDSWkv85y2fnm7n3du2HD18t9nu42sw7MzFa5KNC9E5qKBpo9UaXbE3tIaBYLdHZ8tLsdW0eX6hL+v8b2eTsVMwmPZ9WXx7uogjRJxJQNNDqjTZJ9FhD6fh255MaSo/CkQi+QEiGyF7BaNAzZXyadMyLhCEBRQMtvdRQYk1eUkPpWU/rydzo7piYRk2jN14LFmIkk4CigVZvAOi9ycsfkBpKT2LryUgNpbM7Jo0FuK61FFVVOVfXJssQi36TgKKBFm8QHfSYct1o0GMy6vFJQOmRJ57+X/pQrnRTVgo2i4ETZ9zX7Z4XG7wc/KCWYycbr9s9xeggAUUDre0K9iQTBn3Pj9NmMeKT9S165PWF0CFNXjE6vQ5vIIRfiTDt5gyOnnRx2Ru8LkN4P78QHQTw6flmmtsCQ39DMWpIQNFAiyfQY3NXjM1skCava/D4FGxWI3r9jbsOypUCSpjKqnoqq+pJTTLhC4R583/PEFCG9m/I61eoafBy24QxmI0G3v2kHlW9wScEiT6TgKKB1vYgqUnXDihWi1GavK7B61OkdtKD7IxkLCYD5y4NfYr00xdbUIGpk9O557ZxuJp9HPnU1et5QoAEFE20eoM9DhmOsVkM0uR1DdFMA9J/0h29XsekLDsXXJ4hXRY4oqp8frGF7IwkUpLM3Dp+DGl2Mwc/rNXsHs1tASo+qOFnZVVc9khz2mgj72ANtHiDfWjyMqKEIgSVMMky67iTcESlXeagXNPk7FROXmjhozNNzLk7Z0ju8Vl1M15/iHtvj2Yo0Ot05IxL5uSFy4TCEYyGgX//DEXg1d+d5HdHL8a3KWGVvyi6C6N8rR015Fc5SP5giKAS6UMNJRpEWttlPsHVWjwBVJUe0/8LcI61kWQ1cnQIm5/eq6rHYjIwIfOL1fgcaTZCYZULLs+grv3Ox5f43dGL3JyTyiN/dBO3T0zjnY/ruJgAKx2KvpOAMki9pV2JsXXMRWnzXp9Ef4nE3ZG2XmooPdPpdEzOTuWTc25qGr2aX18JRThxxs2ETHun0Yqxxc7O1A589UhXczu7f3uScWOsPDAti7QUC3k3Z6DX6Sg7XD3osouRQwLKIMXSrvQWUKxSQ+mRuzXali6d8tc2dfJYLGYjv/jvk5qPvKqqduMPhpmU2Xmt8CSrkdRkM2dqWwZ03UhE5UdvfoxOp6Pg7pz4KL4kq5E7JqVx5FPXkARIMTwkoAxSS8cs+V6bvDrSr7RJCo0u3G3RGopMarw2q9lI0axJfHyumQ9PNWl67SOfNWA1G8jKSOq0XafTcVNWyoBrKEdPNnD2UhvLHroF+1UTf6dOTsdsMvCbQ+cGWmwxwkhAGaS+NnlZzdEmL6mhdOVuDWA1GwbV6Xuj+PLd2WRnJPGL332OotEsx3AkwgefNzLt5oxuJ+dOykqhvtnX73VZVFXlN4fOkZme1G36favZyP1TMznyWQPtfmkKHg3kHTxIvaVdidHrdVjNhngTmfiCu9Uv/Sd9ZDDo+bOvTcF12cf/eeUo5+sH36l98vxlPD6Fu2/tfu36SVkpAJy91L9aykdnmjjv8lA0a2KPE1bvvyuTUDjCe1Uy12U0kIAySK3eIMm2a6ddibGaDVJDuYqqqtQ1tXdpDhE9mzY5g79ZNJWmFj/f/dkRXv3954Oan3L0ZANmo567bhrb7f6JmSno6FvHfGzVR49fYe//nmNsioW7bx3X4+qbE5x2ch3JvP3RpQGXX4wcElAGqcUbZIz92s1dMTaLUfpQrlLb1E6LN0hWelLvB4t4jq+pN2ew4Zv5zJqayf73LrDhJ+/w4an+J3NUQmGOftZA3s0ZmE2Gbo+xWYzkjEvuU0CJrfr460PnOHuplSnjx/D+yQZCke6b53Q6HX80LZvTta1capLO+UQnAWWQ+pJ2JcZmMUoN5Sofn41m0c3JSB7mkiSGK3N8fXzWza3jx1B4/wTMRgP/9trxfs9qL3v3PC3eIF+5d/w1j5uck8rZS619Hl1Wda4Zi8nArePH9Hrs7KmZ6HU63v6ork/XFiOXBJRBavH0nnYlxmYx0OZVJNneFT4558aRZpMmr0HIHJvEulX3cueksfy8/FPerarHGwj1mpm4scVH6eFq8u9wcuek7pu7Ym7JScXjU6htau+1PB6fwkWXhykTxvRpoMUYu4W8m9M5dOKSrMGS4CSgDFJre+9pV2JsZiNKOCIrN3ZQQhE+Pd8cX0RKDFw4ojJjyjjGplj4yb5PKD18rtfMxHt+fwqArz90a6/Xz7s5A6BPzWqfnb8MOrh9QlrvBe/w1XvHc9kT5LWK030+R4w8ElAGoa9pV2JikxtjM8NvdKdqWggqkV6/HYu+MRn1fOXe8SRZjfz+aA317p5rE+9V1XPkswaKZ08io2M2/LWkp1qZlJnCsc8brnlcMBTm84uXmeC09ymVTqxPaHLuGL58dw4HKi/wzif112XdF6E9CSiD0Nc5KDHONBsAJ85ev9X3RrJPzrnR63R9amcXfWOzGPla/nh0OnjpjY+6zehbdvgs//7mx0zOTuXLM3LxBkJ4A6EeR2LF3DNlHGdqWmm5xsCSo5+6CCqRPtc6r+wTmuhMJj3Vws/KqiTHV4KSgDIILf0MKPYkE+Oddt7v5VvejeLEWTe35KbGE2cKbaQkmfnKvePx+BS27j7GR2eaUFUVr19h36Fz7HztOM6xScyamskHnzfEP9B7GokVM2PKOFR6bvYKhSP895GLpNnNZI619bvcBoOegrtzUFX4P/95lH3/c4ZIb1FOjCjyTh4EV7MPgLF2S5/PmX5LBmWHq6PDjfsYiEajFm+Q83VtLPry5OEuyqg0boyVv140jf/67Um27/mQCU47de52lFCEGbc5uGtSWr8zE0xw2slItfLB540UdJNCv/zd87iafXzlS7nodANbeTM12czCB26iqrqZfy/5iD+MH8PjfzKNtH68x8TwkRrKIHx4qpExdjM5jr4Peb371ui3vA9u4FqKPxjiB68fR6/X8aXbHMNdnFHrtglpfP+xWax4+DZ0wAPTsnj2z2fy98tmDCjNjU6nY8aUcXx8zt1lIqXrso99h84x49ZxjHfae7hC39iTTPy/f5rHXy6cSnVdG5t+WsnH1W7pV0kAElAGSAmF+eiMm3tuHYe+H9/GsjOScKRZOfZ5/yehjUShcHTRsL4KKmGef+04Z2vb+JtFUxnvGNyHj+iZTq8jEIowa1oWa1d8iaUP3cq4sTYC/fh9xa4T62e586axKKEIH5z+Ijmlqqq8cuAker2OJX98iyZlD4YiWC1GCu+fQDiisv3VDyk9fFaawEY4afIaoKrqZgJKmHv6+Q1bp9NxzxQHv3//Ir5AKGH7D8KRCG9/VEfJ/5yl3a9Q9MBN/PE9uRg6cjZZTMYuK/E1tgT44d6POFvbysp5t3PHTel96gwWAxNQwnx4smtNOH9q9oCvE4moJFmN/LT0EyLhCLdPTOP/K/+Mj8408WdfncLYFG2bpsamWCmaPYnDJ6J/ax+eit7nltzUATeriaGTmJ9mI8Cxzxuxmg3cMbH/Q16/dJuDA5UX+OhME/fdmTkEpRtalz0Bnnv1Ay42eLkpO4Vkm5GSg2c4+EENf5SXxdgUKzPvzMR4RbD8tLqZnXtP4AuEePDu6AdaZVU9AHdLs1fC0Ot1FM2ayLHPG/nxrz/BZNSj08GffW0KX7t3PO1DsOa91Wzgj+/JwWDQ89ofTvP9/zyKI81K/u1OJmWlMMFpxznW1qd8emJoSUAZgIiqxtN9mwawIPatuWPISLXwym9P4kizMTk7dQhK2ZkSCnPyQgvjnfZBDQZo9Qb5v/91DHdbgL9dPI3bJ42lsqqe8/Ue3v2knt8cqmbGlHHxdOXuVj+/eus0hz+uxznWxkP35JKm8bdYcX0lWU2s+X9mUHboHDWNXhYX3IwjzUZ7MDxktU2dTkf+nZncdVM6H55q5OinLva/dz5+P6NBT864JG7KSmXa5HTuumksSbJg23XXp4Cyb98+du7cSSgU4pvf/CYrVqzotL+qqooNGzbg9XrJz89n06ZNGI1GamtrWbt2LU1NTUyePJlt27aRnJxMa2sr3/nOd7hw4QLp6ens2LEDh8NBMBhkw4YNnDhxAqvVyrZt27jllltQVZWtW7fyhz/8Ab1ez7/8y79w7733AvDyyy+zZ88eVFXl29/+NnPnztX+KV3lbG10LP6XpnSf7vtadHodPiXM44vzeOmNj9iy+33+ouhO7r09s0sTkRba2oP899EaKo5dpK1dQaeLBrRZU7P4o7yuI3WupbktwPY9H9DU6ufJZXdz+8SxeAMhdDodk7JSyEy38c7H9bx/spHjp/8Xq9mIPxjdXzx7Eg99aTzHT4+OvqMbXSgcwTnWhnOsjXOXWjnXkdp+KGubASXMiTNNGPQ67rsrk3tvd3DZGyQ91UpNg5faBg/vVdVz8MNa9DqYnDOGvJvTybs5g0lZKf3q6xQD02tAqa+vZ/v27bz++uuYzWYeffRR7r//fm699Yt0DWvXruV73/seM2bM4Omnn2bPnj0sX76cTZs2sXz5coqLi3nxxRd56aWXWLt2LTt27CA/P59///d/p6SkhM2bN7Njxw527dqFzWajrKyMyspK1q9fz549e9i/fz+nT5+mtLSU6upq/vqv/5rS0lI++eQT3nzzTfbu3YvH4+HrX/869913H2lpfU/5MBDHPm/EoNcx/ZaMfp97ZXv0V740nt8dvciP9n7MbRNqWPjATdw+sf/DObu9TzDMgSMXKHunGn8wTK4jmS/d5qC5LcC5ujZ27f+M/z5ygW8WT2VChu2afTlBJcz+987zm3eqUVV4Yul0bu+mqc9qNjJnRg7n6z2YjHrCERWTQc/D+RPIGGPFG7h2KhAh+sNg0JORauXu2xzogPGOZPLvcNJw2Udto5fLniAl/3OWkv85i91mYurkdKZ1/DdGhiEPiV4DyqFDh5g1a1b8Q7qwsJDy8nL+7u/+DoCamhr8fj8zZswAYMmSJTz//PMsW7aMyspKXnzxxfj2b3zjG6xdu5aKigpeeeUVABYsWMB3v/tdFEWhoqKCJ554AoCZM2fidrupra3lrbfeoqioCL1ez+TJk8nOzubYsWO8++67PPzww1gsFiwWC/fddx8VFRX8yZ/8SZ8fQE8L/1yLM93Gggduwn6NLMOqXtdtldto0Me3J1lN/Okf38LZ2lbO1rWx68Bn6HU6xqZaSE02YzLoMRr0mIx6TAYd6HTogOgXLR06HUT/qev4fzQ/lqvZR527naASZva0LP74nlwuXZHUb/a0LGobvZyqaeHlfSfQ6SA7I5mxKRZsZgMGg55wOEJAidDQ4qPhso9wWGX2tGzmzpxAeqqVQMcYTr1e3+XnvPOm9Og31SvaPwKhSLfHXv1MetpusxgJh0w9Ht+Xawx0e3+OtVmMQ3Zt7crd+9/mcJY7tv3K33l/rmFPMjM5Zwx33+bA4w1wpraVzy+2cLqmhX2HzrHv0DmSrCbS7GaSLEbCqkpEBTWiEo6oqBGVCCqgw2YxkGQxYrMYSbaYMJl06HWx9170/0nJFny+IDp0qKioKtCRAFZVQQVUov+I7orui789Ov6vqtF9asc2FZX4IbFjYhfhqnM7/qFesQ0dGPR6jAYdBr2OFLsFn08hHIlgt5mZdVfmgD7/rnVOrwHF5XLhcHxRjXU6nRw/frzH/Q6Hg/r6epqbm7Hb7RiNxk7brz7HaDRit9txu93dXquurg6Xy4XT6ex2e15eXpft/TF2bP/Tpv/pV2/v03HFX+5+COXN40dX7qqJOX1PndLTsT09k/5s1+IaN8q1J2R23283Usqt1T0B7rhZBn1cL722rUQikU7D81RV7fS6p/1XHwf0OMxPVVX0en2Xc2Lbu7tHbHuXH0hGegghxLDo9dM3KyuLhoYvxrI3NDR0qi1cvb+xsRGn00l6ejptbW2Ew+Eu5zmdThobo52zoVAIr9dLWloamZmZuFyuLtfKysrqcfu1yiaEEOL66TWgPPDAAxw+fBi3243P5+PAgQMUFBTE9+fm5mKxWDh69CgAe/fupaCgAJPJRH5+PqWlpQCUlJTEz5szZw4lJSUAlJaWkp+fj8lkYs6cOezduxeAI0eOYLFYyMnJoaCggH379hEOh6murubcuXPk5eVRUFDAgQMH8Pl8uN1u3nnnHWbPnq3tExJCCNEnOrUPywfu27ePH/3oRyiKwtKlS1m9ejWrV69mzZo15OXl8emnn/LMM8/g8XiYOnUq//qv/4rZbKampoZ169bR1NREdnY2zz33HGPGjOHy5cusW7eOCxcukJKSwrZt2xg/fjyBQICNGzdy4sQJzGYz3/ve95g6dWp82PDBgwcBWL9+PQ8++CAQHTb8q1/9ilAoxOOPP96vDnkhhBDa6VNAEUIIIXojPdhCCCE0IQFFCCGEJiSgCCGE0IQEFCGEEJqQgDIE9u3bR1FREXPnzo2nmBkqHo+HBQsWcPHiRSCaKmfhwoXMnTuX7du3x4+rqqpiyZIlFBYWsmHDBkKhaF6t2tpaVqxYwbx583j88cfxer0AtLa28thjjzF//nxWrFgRn+8TDAZZu3Yt8+fPZ/HixZw+fbrXMv7gBz+guLiY4uJitm7dOmLL+W//9m8UFRVRXFzMT3/60xFbToAtW7awbt2661IWVVXZsmUL8+bNo6ioKD5F4FpWrlxJcXExixYtYtGiRXz44Yc9vi+G+hlfy+9//3uWLFnC/Pnz+d73vnddytPf3/kvf/nL+HNctGgR9957L9/97ndHXDkB3k7ILQAACapJREFUUIWm6urq1Iceekhtbm5WvV6vunDhQvXzzz8fknt98MEH6oIFC9SpU6eqFy5cUH0+nzpnzhz1/PnzqqIo6re+9S21oqJCVVVVLS4uVo8dO6aqqqquX79efeWVV1RVVdXHHntM/fWvf62qqqr+4Ac/ULdu3aqqqqpu2rRJ/dGPfqSqqqq+8cYb6hNPPKGqqqr+5Cc/Uf/pn/5JVVVVfe+999Rly5Zds4xvv/22+vWvf10NBAJqMBhUV61ape7bt2/ElfPdd99VH330UVVRFNXn86kPPfSQWlVVNeLKqaqqeujQIfX+++9Xn3rqqetSlrKyMnX16tVqOBxWz5w5oz788MOqoig9li8SiagPPvhgp2N6el9cj7/Znpw/f1598MEH1UuXLqnBYFD9sz/7M7WiomJE/s5jTp48qT788MNqbW3tiCynBBSNvf766+r69evjr3/wgx+oL7zwwpDc6+mnn1YrKyvVhx56SL1w4YL67rvvqqtWrYrvf+ONN9R169apFy9eVL/61a/Gt1dWVqorV65Ug8Gges8998Tf+LW1tepXvvIVVVVV9aGHHlJra2tVVVVVRVHUe+65Rw0Gg+o3vvENtbKyMn6tr371q2pNTU2PZTx58mT8j1tVo3+8L7zwwogrp6qqajAYVFVVVS9evKgWFBSMyOfZ3NysLlu2TP3pT3+qPvXUU9elLOvWrVPfeOON+PZVq1ap7733Xo9lPHXqlPrggw+qK1euVBcuXKju2rWrx/fF9XjGPfmP//gP9fvf/378dV1d3Yj8nV9p+fLlamlp6YgtpzR5aay7ZJqxpJha27x5M/n5+b3e+3ok8OzJlClT4pmoz507R1lZGTqdbsSVE8BkMvH8889TXFzM7NmzR+Tz3LhxI08++SSpqaldrjtUZekpOWtPWltbmT17Ni+++CI/+9nP+MUvfkFtbW2fnuVQPOOeVFdXEw6H+Zu/+RsWLVrE7t27R+TvPObQoUP4/X7mz58/YsspAUVjvSXTHI57X48Enr35/PPP+da3vsU//uM/MmHChBFbzjVr1nD48GEuXbrEuXPnRlQ5f/nLX5Kdnd0pvdBwJmftyT333MPWrVtJSUkhPT2dpUuX8vzzz/frWWr5c/UkHA5z+PBhvv/97/Pqq69y/PhxLly4MKJ+51f6xS9+wV/8xV8AI/e9LgFFY8OZsLKne1+PBJ7XcvToUf78z/+cb3/72yxevHhElvP06dNUVVUBYLPZmDt3Lu++++6IKmdpaSlvv/02ixYt4vnnn+f3v/89r7322rAlZ+3JkSNHOHz4cPy1qqrk5ub26VkOxTPuybhx45g9ezbp6elYrVa+9rWvcejQoRH1O48JBoP8/+3dX0hT/x/H8WcUYgVJEESEUVJ0E/b3oouwXLjN/ZGGVOKSLSsiiiFe2B+kstBJBeEMIoX+gikrcssKojSwXTgJWgkxGUQoSHVTzUon2+d7MRzfvr/89/sev9/9+L0fsIsj53zO27Md3nzOOXutt7cXnU4HpO+5Lg1FY1OFac6m9evX8/79+9RUvqOjg7y8vH8kwHMiQ0NDHD16lEuXLmE2m9O2zsHBQaqrq4nFYsRiMZ4/f05JSUla1Xnjxg06Ojrw+Xy4XC50Oh1ut/tfC2edSDQa5cKFC4yOjjI8PMyDBw+4ePHib8+Lf+KzMJH8/HxevnzJt2/fiMfjdHd3YzQa0+o9HxcOh1m5ciULFiwA0vMcAuQpr9ng9/uV2WxWer1eNTU1zfr+xm/KK5V8AshqtSq9Xq9qa2tVIpFQSin17t07VVxcrAwGg6qsrFSjo6NKqeQN6H379qnCwkJVXl6uvnz5opRK3vw9fPiwMplMau/evanxR0ZGVFVVlTKZTGrXrl2qr69v0trOnz+vNmzYoIqKilKvlpaWtKtTKaU8Ho8qLCxUFotFeTyetDye4+7fv596ymu2a0kkEqq+vl6ZTCZlMplUd3f3lPVdvnxZGY1Gpdfr1c2bN5VSE58Xs32MJ+P1elM11dTUqHg8npbv+aNHj1RFRcUvf0vHOiUcUgghhCbkkpcQQghNSEMRQgihCWkoQgghNCENRQghhCakoQghhNCENBQhNPLmzRtOnz6dWi4vL58w+uPQoUNEIhF6enqwWCwz3pfX651WknU8HqexsRGbzYbBYKCurg55sFPMFmkoQmgkEon8ktsWCAQmXLe5uZnVq1f/1/t69eoVIyMjU653+/ZtgsEgd+/e5eHDh7x+/Tr15TYhtDbv3y5AiP81iUSCuro6QqEQ379/RylFTU0NHo+HaDTKyZMnU+s6HA6ampqw2+3k5uYSDoeprKzE7XbT0NAAwI8fP3C5XHz48IFFixZx7tw5Vq1axYkTJ1izZg0HDhwASC2vWLGCzs5OAoEAmZmZ2O12rl69ytOnT0kkEixfvpwzZ86wdOlS2tvbOX78OJmZmQA0NjZO+u1xIf4OmaEIMUOhUIhPnz7R1tbG48ePsdlsXL9+HZfLxZYtW3C73bjdbgBu3brFsmXLgGTy8pMnTygoKPhlvKGhIZxOJz6fD4vFQlVV1aT7LygoQKfT4XQ6sdvttLe309/fj9frxefzsX37dqqrq4FkwnMkEsHhcGC1WmlpaSErK2sWjooQMkMRYsY2btxIVlYWra2tDAwM0NPTw8KFC6fc7s8/NfBna9euZdOmTQDYbDbOnj1LNBqddj1dXV28ffuW4uJiIDmD+vnzJ5AM+wuFQjQ3NxOLxThy5Ah37tzB6XROe3whpksaihAz9OLFC2pra9m/fz87d+4kJycHv98/5XbjwX5/9ddI8Dlz5jBv3rxU7Pi4sbGx326fSCQ4ePAgpaWlQDKZ9uvXr0AySdZsNpORkUFGRgZGo5He3t5p/Z9CzJRc8hJihgKBAPn5+ZSWlrJu3TqePXtGPB5n7ty5qd/vBv5jeSLhcDgVnd/W1sbmzZuZP38+ixcvpq+vD4CPHz8SDAZ/O/a2bdu4d+8ew8PDADQ0NKQumxkMBvx+P4lEgrGxMbq6uiZNChbi75CGIsQMlZSUEAwGsVqt2Gw2srOzGRwcJDc3l4GBAY4dOwaA0WikrKyM/v7+ScfLycnhypUrFBUV0dnZSX19PQBlZWV8/vwZg8HAqVOn2Lp1a2qbvLw8WltbuXbtGrt372bHjh3s2bMHs9lMOBxOjVFRUcGSJUuwWCxYLBays7NxOByzdGTE/ztJGxZCCKEJmaEIIYTQhDQUIYQQmpCGIoQQQhPSUIQQQmhCGooQQghNSEMRQgihCWkoQgghNCENRQghhCb+AFJn8gyU+o5IAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute6'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11e76a518>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEJCAYAAACXCJy4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAb40lEQVR4nO3de1Bc9f3/8deyEOKGKCbdhZSJWhtHnRhKralIHWKssshFkjW2aaJErVjbWiydYnMbvDSGmKHFS2zGW7UXMoFoBHEioSbGamASYTomjhjlN2oiTZcFNIUACbDn94ff7gQJOUBCgnyejxlHPud9ztnPec/xxeETOXFYlmUJADDuhZ3pCQAATg8CHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABgi/ExP4EQ+//ywgsGR/ZrA1KlRam3tOMUzGl/okT16ZI8e2TtdPQoLc+jccycNWh/TgR8MWiMO/P8djxOjR/bokT16ZG8s9IglHQAwBIEPAIYg8AHAEEMK/MrKSqWlpSklJUUlJSUD6g0NDfL5fPJ6vVqxYoV6e3slSc3Nzbrrrrs0b948LVy4UJ999tmpnT0AYMhsA9/v96u4uFgbNmxQeXm5SktL1djY2G+f/Px8FRQUaOvWrbIsS2VlZZKk++67T3PnzlV5ebmysrJUVFQ0OlcBALBlG/g1NTVKTExUdHS0XC6XvF6vqqqqQvWmpiZ1d3crISFBkuTz+VRVVaW2tjZ98MEHWrhwoSTppptu0q9//etRugwAgB3bwG9ubpbb7Q6NPR6P/H7/oHW32y2/368DBw7om9/8ptasWaObbrpJubm5ioiIOMXTBwAMle3/hx8MBuVwOEJjy7L6jQer9/b26v3339evfvUrLVu2TJs2bdLSpUv1t7/9bciTmzo1asj7flV751FZTueA7WdNDNdk14QRn3e8cbsnn+kpjHn0yB49sjcWemQb+LGxsaqrqwuNA4GAPB5Pv3ogEAiNW1pa5PF45Ha7NWnSJM2dO1eSlJGRoVWrVg1rcq2tHSP+ZQXL6dSb9fsHbJ99aYy6Dx8Z0TnHG7d7sgKB9jM9jTGNHtmjR/ZOV4/CwhwnfFC2XdJJSkpSbW2t2tra1NXVperqaiUnJ4fqcXFxioyMVH19vSSpoqJCycnJOu+88xQbG6s333xTkvTGG29o5syZJ3s9AIARsg38mJgY5eXlKTs7W/PmzVNGRobi4+OVk5OjvXv3SpKKiopUWFio1NRUdXZ2Kjs7W5L0xBNP6Nlnn1VGRob++te/avXq1aN7NQCAQTnG8l9iPlpLOpMix/QrhE4bfhS3R4/s0SN7X5slHQDA+EDgA4AhCHwAMASBDwCGIPABwBAEPgAYgsAHAEMQ+ABgCAIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQxD4AGAIAh8ADEHgA4AhhhT4lZWVSktLU0pKikpKSgbUGxoa5PP55PV6tWLFCvX29kqSXn75ZV199dXKyspSVlaWiouLT+3sAQBDFm63g9/vV3FxsTZv3qwJEyZo4cKFuvLKKzVjxozQPvn5+Vq1apUSEhK0fPlylZWVadGiRXrvvfe0dOlSZWRkjOpFAADs2T7h19TUKDExUdHR0XK5XPJ6vaqqqgrVm5qa1N3drYSEBEmSz+cL1ffu3auXX35ZmZmZ+u1vf6tDhw6N0mUAAOzYBn5zc7Pcbndo7PF45Pf7B6273e5Q3e126xe/+IVeeeUVTZs2TQ899NCpnDsAYBhsl3SCwaAcDkdobFlWv/GJ6k8++WRo+5133qnrr79+WJObOjVqWPsfq7mtU5OjJg7Y7nJFyj3FNeLzjjdu9+QzPYUxjx7Zo0f2xkKPbAM/NjZWdXV1oXEgEJDH4+lXDwQCoXFLS4s8Ho/a29v10ksv6bbbbpP05TcCp9M5rMm1tnYoGLSGdUyI06n2ju4Bmzs7jyjQ1zeyc44zbvdkBQLtZ3oaYxo9skeP7J2uHoWFOU74oGy7pJOUlKTa2lq1tbWpq6tL1dXVSk5ODtXj4uIUGRmp+vp6SVJFRYWSk5Plcrn07LPP6t1335Uk/f3vfx/2Ez4A4NSxfcKPiYlRXl6esrOz1dPTowULFig+Pl45OTnKzc3VrFmzVFRUpJUrV6qjo0MzZ85Udna2nE6nHn30UT3wwAPq7u7WBRdcoLVr156OawIAHIfDsqwRrpmMvpNZ0rGcTr1Zv3/A9tmXxmhSpO33OSPwo7g9emSPHtn72izpAADGBwIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQxD4AGAIAh8ADEHgA4AhCHwAMASBDwCGIPABwBAEPgAYgsAHAEMQ+ABgCAIfAAwxpMCvrKxUWlqaUlJSVFJSMqDe0NAgn88nr9erFStWqLe3t1/9/fff12WXXXZqZgwAGBHbwPf7/SouLtaGDRtUXl6u0tJSNTY29tsnPz9fBQUF2rp1qyzLUllZWajW1dWl3//+9+rp6Tn1swcADJlt4NfU1CgxMVHR0dFyuVzyer2qqqoK1ZuamtTd3a2EhARJks/n61dfs2aNlixZMgpTBwAMR7jdDs3NzXK73aGxx+PRnj17Bq273W75/X5J0rZt29Td3a3U1NQRTW7q1KgRHSdJzW2dmhw1ccB2lytS7imuEZ93vHG7J5/pKYx59MgePbI3FnpkG/jBYFAOhyM0tiyr33iweiAQ0Pr16/XCCy+MeHKtrR0KBq2RHex0qr2je8Dmzs4jCvT1jXhO44nbPVmBQPuZnsaYRo/s0SN7p6tHYWGOEz4o2y7pxMbGKhAIhMaBQEAej2fQektLizwej3bs2KEvvvhCixcvVlZWliQpKytLHR0dI7oQAMDJsQ38pKQk1dbWqq2tTV1dXaqurlZycnKoHhcXp8jISNXX10uSKioqlJycrJtvvlmvv/66KioqVFFREapFRY18mQYAMHK2gR8TE6O8vDxlZ2dr3rx5ysjIUHx8vHJycrR3715JUlFRkQoLC5WamqrOzk5lZ2eP+sQBAMPjsCxrhIvko+9k1vAtp1Nv1u8fsH32pTGaFGn7RxdGYO3VHj2yR4/sfW3W8AEA4wOBDwCGIPABwBAEPgAYgsAHAEMQ+ABgCAIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQxD4AGAIAh8ADEHgA4AhCHwAMMSQAr+yslJpaWlKSUlRSUnJgHpDQ4N8Pp+8Xq9WrFih3t5eSVJdXZ18Pp8yMzN1991369ChQ6d29gCAIbMNfL/fr+LiYm3YsEHl5eUqLS1VY2Njv33y8/NVUFCgrVu3yrIslZWVSZKWLVumtWvXqrKyUjNmzNBzzz03OlcBALBlG/g1NTVKTExUdHS0XC6XvF6vqqqqQvWmpiZ1d3crISFBkuTz+UL1LVu2aMaMGerp6ZHf79fZZ589SpcBALBjG/jNzc1yu92hscfjkd/vH7TudrtD9YiICO3bt09z5szRrl27lJ6efirnDgAYhnC7HYLBoBwOR2hsWVa/sV394osvVk1NjTZu3Ki8vDxt3LhxyJObOjVqyPt+VXNbpyZHTRyw3eWKlHuKa8TnHW/c7slnegpjHj2yR4/sjYUe2QZ+bGys6urqQuNAICCPx9OvHggEQuOWlhZ5PB4dOXJEb731lq677jpJ0o033qhHHnlkWJNrbe1QMGgN65gQp1PtHd0DNnd2HlGgr29k5xxn3O7JCgTaz/Q0xjR6ZI8e2TtdPQoLc5zwQdl2SScpKUm1tbVqa2tTV1eXqqurlZycHKrHxcUpMjJS9fX1kqSKigolJycrPDxcDz74oN577z1J0muvvabLL7/8ZK8HADBCtk/4MTExysvLU3Z2tnp6erRgwQLFx8crJydHubm5mjVrloqKirRy5Up1dHRo5syZys7OltPpVHFxsQoKCtTX16eYmBg9/PDDp+OaAADH4bAsa4RrJqPvZJZ0LKdTb9bvH7B99qUxmhRp+33OCPwobo8e2aNH9r42SzoAgPGBwAcAQxD4AGAIAh8ADEHgA4AhCHwAMASBDwCGIPABwBAEPgAYgsAHAEMQ+ABgCAIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQwwp8CsrK5WWlqaUlBSVlJQMqDc0NMjn88nr9WrFihXq7e2VJNXX12vBggXKysrSkiVL1NTUdGpnDwAYMtvA9/v9Ki4u1oYNG1ReXq7S0lI1Njb22yc/P18FBQXaunWrLMtSWVlZaPuqVatUUVGhzMxMrVq1anSuAgBgyzbwa2pqlJiYqOjoaLlcLnm9XlVVVYXqTU1N6u7uVkJCgiTJ5/OpqqpKR48e1b333qtLLrlEknTxxRfr4MGDo3QZAAA74XY7NDc3y+12h8Yej0d79uwZtO52u+X3+zVhwgRlZWVJkoLBoNatW6frrrtuWJObOjVqWPv3m3dbpyZHTRyw3eWKlHuKa8TnHW/c7slnegpjHj2yR4/sjYUe2QZ+MBiUw+EIjS3L6je2qx89elRLly5Vb2+vfvaznw1rcq2tHQoGrWEdE+J0qr2je8Dmzs4jCvT1jeyc44zbPVmBQPuZnsaYRo/s0SN7p6tHYWGOEz4o2y7pxMbGKhAIhMaBQEAej2fQektLS6h++PBh3Xnnnert7dX69esVERExoosAAJw828BPSkpSbW2t2tra1NXVperqaiUnJ4fqcXFxioyMVH19vSSpoqIiVM/Pz9f555+vRx99VBMmTBilSwAADIXtkk5MTIzy8vKUnZ2tnp4eLViwQPHx8crJyVFubq5mzZqloqIirVy5Uh0dHZo5c6ays7P1/vvva9u2bZoxY4bmz58v6cv1/2eeeWbULwoAMJDDsqwRLpKPvpNZw7ecTr1Zv3/A9tmXxmhSpO33OSOw9mqPHtmjR/a+Nmv4AIDxgcAHAEMQ+ABgCAIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQxD4AGAIAh8ADEHgA4AhCHwAMASBDwCGIPABwBAEPgAYYkiBX1lZqbS0NKWkpKikpGRAvaGhQT6fT16vVytWrFBvb2+/+qOPPqonnnji1MwYADAitoHv9/tVXFysDRs2qLy8XKWlpWpsbOy3T35+vgoKCrR161ZZlqWysjJJUnt7u5YvX67nn39+dGYPABgy28CvqalRYmKioqOj5XK55PV6VVVVFao3NTWpu7tbCQkJkiSfzxeqb9u2TRdccIFuv/32UZo+AGCobAO/ublZbrc7NPZ4PPL7/YPW3W53qD5v3jzdddddcjqdp3LOAIARCLfbIRgMyuFwhMaWZfUb29VPxtSpUSM+trmtU5OjJg7Y7nJFyj3FdTLTGlfc7slnegpjHj2yR4/sjYUe2QZ+bGys6urqQuNAICCPx9OvHggEQuOWlpZ+9ZPR2tqhYNAa2cFOp9o7ugds7uw8okBf30nObHxwuycrEGg/09MY0+iRPXpk73T1KCzMccIHZdslnaSkJNXW1qqtrU1dXV2qrq5WcnJyqB4XF6fIyEjV19dLkioqKvrVAQBjg23gx8TEKC8vT9nZ2Zo3b54yMjIUHx+vnJwc7d27V5JUVFSkwsJCpaamqrOzU9nZ2aM+cQDA8DgsyxrhmsnoO5klHcvp1Jv1+wdsn31pjCZF2q5kGYEfxe3RI3v0yN7XZkkHADA+EPgAYAgCHwAMQeADgCEIfAAwBIEPAIYg8AHAEAQ+ABiCwAcAQxD4AGAIAh8ADEHgA4AhCHwAMASBDwCGIPABwBAEPgAYgsAHAEMQ+ABgCAIfAAxB4AOAIQh8ADAEgQ8AhiDwAcAQBD4AGILABwBDEPgAYIghBX5lZaXS0tKUkpKikpKSAfWGhgb5fD55vV6tWLFCvb29kqR///vfWrx4sVJTU/Xzn/9chw8fPrWzBwAMmW3g+/1+FRcXa8OGDSovL1dpaakaGxv77ZOfn6+CggJt3bpVlmWprKxMkvTggw9q0aJFqqqq0mWXXaY//elPo3MVJ2BZlj5r7tCu9/3a+/9aVb+vWW3/7T7t8wCAM81hWZZ1oh1efvllvfPOO1q9erUk6cknn5RlWbrnnnskSU1NTVqyZIlef/11SVJdXZ0ef/xxPffcc7ryyiu1e/duhYeH6+DBg7rlllu0bdu2IU/u888PKxg84fQG6AsGteNf/9bRvqD8rYf175ZOfdFxRM4wh/qOOVfcN6J0XkyUgpalvqClyAinIic4JUlHevrU12fJGeZQeFiYnE6Hwp0OORwOOSQ5HF/50P/b/uXX//vXl19YsqT/+1hLkgZr9zEn/erpj93gGFjtx9Ig5z/OZpdrgjo7j57wfINParDdhrjj18SkSRN0+PAQe2QoemRvuD267MIpmnr2xGF/TliYQ+eeO2nQerjdCZqbm+V2u0Njj8ejPXv2DFp3u93y+/36/PPPFRUVpfDw8H7bh+NEEz+RH6WcPaLjAGA8s13SCQaDchzz9GlZVr/xYPWv7idpwBgAcPrYBn5sbKwCgUBoHAgE5PF4Bq23tLTI4/FoypQpam9vV19f33GPAwCcXraBn5SUpNraWrW1tamrq0vV1dVKTk4O1ePi4hQZGan6+npJUkVFhZKTkxUREaErrrhCW7ZskSSVl5f3Ow4AcHrZ/qGt9OX/lvnUU0+pp6dHCxYsUE5OjnJycpSbm6tZs2bpgw8+0MqVK9XR0aGZM2eqsLBQEyZMUFNTk5YuXarW1lZNmzZNf/zjH3XOOeecjusCAHzFkAIfAPD1x2/aAoAhCHwAMASBDwCGIPABwBDjLvDtXvRmknXr1ik9PV3p6elau3atJKmmpkaZmZlKSUlRcXFxaN/BXoBnikceeURLly6VxMsAv2r79u3y+Xy64YYbtGrVKkncR19VUVER+m/tkUcekTRG7yNrHPnPf/5jzZ071/r888+tw4cPW5mZmdZHH310pqd1RuzcudP68Y9/bB05csQ6evSolZ2dbVVWVlpz5syx9u/fb/X09Fh33HGHtWPHDsuyLCs9Pd3617/+ZVmWZS1btswqKSk5k9M/rWpqaqwrr7zS+t3vfmdZ1uC9uOuuu6xXX33VsizLWrdunbV27dozM+HTaP/+/dbVV19tHTx40Dp69Kj1k5/8xNqxYwf30TE6Ozut2bNnW62trVZPT4+1YMECa+fOnWPyPhpXT/g1NTVKTExUdHS0XC6XvF6vqqqqzvS0zgi3262lS5dqwoQJioiI0Le//W198sknOv/88zV9+nSFh4crMzNTVVVVampqUnd3txISEiRJPp/PmL598cUXKi4u1t133y1Jg/aip6dH77zzjrxeb7/t490//vEPpaWlKTY2VhERESouLtZZZ53FfXSMvr4+BYNBdXV1qbe3V729vQoPDx+T95Hty9O+Tuxe9GaSiy66KPT1J598otdee0233HLLgP74/f5BX4BngoKCAuXl5engwYOSRvdlgF9Hn376qSIiInT33Xfr4MGDuuaaa3TRRRdxHx0jKipK9957r2644QadddZZmj17tiIiIsbkfTSunvDtXvRmoo8++kh33HGH7rvvPk2fPv24/TG1b5s2bdK0adN01VVXhbbxMsD++vr6VFtbq9WrV6u0tFR79uzRgQMHuI+O8cEHH+ill17SG2+8obfeekthYWHauXPnmLyPxtUTfmxsrOrq6kJj01/YVl9fr9zcXC1fvlzp6enavXv3cV+EN9gL8Ma7LVu2KBAIKCsrS4cOHVJnZ6ccDoftywCdTqcx99Y3vvENXXXVVZoyZYok6brrrlNVVZWcTmdoH9Pvo7fffltXXXWVpk6dKunLZZrnnntuTN5H4+oJ3+5FbyY5ePCgfvnLX6qoqEjp6emSpO985zv6+OOP9emnn6qvr0+vvvqqkpOTB30B3nj3/PPP69VXX1VFRYVyc3N17bXXqrCwkJcBHmPu3Ll6++239d///ld9fX166623lJqayn10jEsuuUQ1NTXq7OyUZVnavn27vv/974/J+2jcvUvneC96M9GqVav00ksv6bzzzgttW7hwoS644AIVFhbqyJEjmjNnjpYtWyaHwzHoC/BMsXnzZu3evVtr1qzhZYBf8eKLL+qFF15QT0+PfvCDH2jlypXatWsX99Exnn76aW3evFkRERGaNWuW7r//fn388cdj7j4ad4EPADi+cbWkAwAYHIEPAIYg8AHAEAQ+ABiCwAcAQxD4MMaePXtUUFAQGt9xxx1qa2s77r45OTlqbGzUrl27lJGRMezP2rRp05De1rpw4UJlZWWF/omPjw+9kRI41cbVb9oCJ9LY2NjvvSU7d+4cdN9nnnlGktTa2jqiz6qvr+/3PqPBbNy4MfT1tm3b9Ic//EH33nvviD4TsEPgY9wJBoNavXq13n33XR0+fFiWZenBBx/U448/rvb2di1btiy075IlS/T0009r8eLFio+P1759+/Sb3/xGhYWFeuyxxyRJnZ2dys3N1aeffqqzzz5bDz30kL71rW9p6dKluuiii/TTn/5UkkLj8847T9u3b9fOnTs1ceJELV68WOvXr1d1dbWCwaDi4uJ0//33KyYmJjSPL774Qvfff7/Wr1+vyZMnn96GwRgs6WDceffdd9Xc3KzS0lJt2bJF8+fP15///Gfl5ubqiiuuUGFhoQoLCyVJf/nLXzRt2jRJX75h9LXXXtP111/f73wHDx7UbbfdpoqKCmVkZOi+++474edff/31uvbaa3Xbbbdp8eLFKi8v14cffqhNmzapoqJCc+bM0cqVK/sd88wzz2jOnDmaNWvWKewE0B9P+Bh3vvvd7+qcc87Rxo0bdeDAAe3atUuTJk2yPe6KK6447vaLL75Yl19+uSRp/vz5euCBB9Te3j7k+bzxxhvau3evbrrpJkkKvTv9f44cOaKysjJt3rx5yOcERoLAx7izY8cOPfzww7r99tv1wx/+UBdeeKFeeeUV2+NcLtdxt4eF9f9B2OFwKDw8PPS62//p6ek57vHBYFB33nmnFi1aJEk6evSoDh06FKr/85//1CWXXKLp06fbzhE4GSzpYNzZuXOn5s6dq0WLFumyyy7T66+/Hnod7bF/x+pXx4PZt2+fGhoaJEmlpaX63ve+p7POOkvnnnuu3nvvPUmS3+/X7t27j3vuq6++Wi+++KI6OjokSY899li/ZaHdu3f3eyc/MFoIfIw7Cxcu1O7du5WZman58+dr+vTp+uyzzxQfH68DBw7onnvukSSlpqbq1ltv1YcffnjC81144YVat26dbrzxRm3fvl1r1qyRJN16660KBALyer1avny5EhMTQ8ckJydr48aNeuqpp3TzzTfrmmuu0Y9+9COlp6dr3759oXNIX/6tUnFxcaPQCaA/3pYJAIbgCR8ADEHgA4AhCHwAMASBDwCGIPABwBAEPgAYgsAHAEMQ+ABgiP8P1cADgxAfyjoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute7'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a2144c898>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEJCAYAAACXCJy4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAcLUlEQVR4nO3dfVCVdf7/8efhgNhRy3TPAZfpZlubak2iNjeyFrMtDgGGnqx1tbBaadvdlpZmab0bulkTdZilG9umG6u9wRHtBsJRZNNsTBgNZidtIovfVBrrHg5QLrcKnOv3R989IyFegCLE5/X4Jz/X+7qu87nec/Xi4oNcOizLshARkREvbKgnICIiZ4YCX0TEEAp8ERFDKPBFRAyhwBcRMYQCX0TEEAp8ERFDhA/1BE7mq69aCAYH9msCEyeOpaGh+TTPaGRRj+ypR/bUI3tnqkdhYQ7OPXdMr/VhHfjBoDXgwP/f8XJy6pE99cieemRvOPRISzoiIoZQ4IuIGEKBLyJiiD4FfklJCcnJySQmJlJQUNCjXl1djc/nw+v1smzZMjo7OwGoq6vjvvvuY/bs2cybN48vv/zy9M5eRET6zDbw/X4/+fn5rF+/nqKiIgoLC6mpqem2T3Z2Njk5OWzbtg3Lsti4cSMADz/8MDNnzqSoqIi0tDTy8vIG5ypERMSWbeCXl5cTHx/P+PHjcblceL1eSktLQ/Xa2lra29uJi4sDwOfzUVpaSmNjIx9//DHz5s0D4LbbbuP3v//9IF2GiIjYsQ38uro63G53aOzxePD7/b3W3W43fr+fQ4cO8f3vf59Vq1Zx2223kZmZSURExGmevoiI9JXt38MPBoM4HI7Q2LKsbuPe6p2dnXz00Uf87ne/Y8mSJWzatInFixfz97//vc+TmzhxbJ/3/bam1mNYTmeP7WeNDmeca9SAzzvSuN3jhnoKw556ZE89sjccemQb+NHR0VRWVobGgUAAj8fTrR4IBELj+vp6PB4PbrebMWPGMHPmTABSU1NZsWJFvybX0NA84F9WsJxO3q062GP7tMuiaG85OqBzjjRu9zgCgaahnsawph7ZU4/snakehYU5TvqgbLukM336dCoqKmhsbKStrY2ysjISEhJC9ZiYGCIjI6mqqgKguLiYhIQEzj//fKKjo3n33XcBeOedd5gyZcqpXo+IiAyQbeBHRUWRlZVFeno6s2fPJjU1ldjYWDIyMti/fz8AeXl55ObmkpSURGtrK+np6QA888wzvPTSS6SmpvK3v/2NlStXDu7ViIhIrxzD+R8xH6wlnTGRw/oVQmeMvhW3px7ZU4/sfWeWdEREZGRQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGEKBLyJiCAW+iIghFPgiIoZQ4IuIGKJPgV9SUkJycjKJiYkUFBT0qFdXV+Pz+fB6vSxbtozOzk4A3nzzTa6//nrS0tJIS0sjPz//9M5eRET6LNxuB7/fT35+Pm+88QajRo1i3rx5XHPNNUyePDm0T3Z2NitWrCAuLo6lS5eyceNG5s+fz4cffsjixYtJTU0d1IsQERF7tk/45eXlxMfHM378eFwuF16vl9LS0lC9traW9vZ24uLiAPD5fKH6/v37efPNN5k1axZ/+MMfOHLkyCBdhoiI2LEN/Lq6Otxud2js8Xjw+/291t1ud6judrv5zW9+w1tvvcWkSZN4/PHHT+fcRUSkH2yXdILBIA6HIzS2LKvb+GT1Z599NrR90aJF3Hzzzf2a3MSJY/u1//HqGlsZN3Z0j+0uVyTuCa4Bn3ekcbvHDfUUhj31yJ56ZG849Mg28KOjo6msrAyNA4EAHo+nWz0QCITG9fX1eDwempqaeP3117n77ruBb74QOJ3Ofk2uoaGZYNDq1zEhTidNze09Nre2HiXQ1TWwc44wbvc4AoGmoZ7GsKYe2VOP7J2pHoWFOU76oGy7pDN9+nQqKipobGykra2NsrIyEhISQvWYmBgiIyOpqqoCoLi4mISEBFwuFy+99BIffPABAP/4xz/6/YQvIiKnj+0TflRUFFlZWaSnp9PR0cHcuXOJjY0lIyODzMxMpk6dSl5eHsuXL6e5uZkpU6aQnp6O0+nkySef5NFHH6W9vZ0LL7yQNWvWnIlrEhGRE3BYljXANZPBdypLOpbTybtVB3tsn3ZZFGMibb/OGUHfittTj+ypR/a+M0s6IiIyMijwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQM0afALykpITk5mcTERAoKCnrUq6ur8fl8eL1eli1bRmdnZ7f6Rx99xOWXX356ZiwiIgNiG/h+v5/8/HzWr19PUVERhYWF1NTUdNsnOzubnJwctm3bhmVZbNy4MVRra2vjT3/6Ex0dHad/9iIi0me2gV9eXk58fDzjx4/H5XLh9XopLS0N1Wtra2lvbycuLg4An8/Xrb5q1SoWLlw4CFMXEZH+CLfboa6uDrfbHRp7PB727dvXa93tduP3+wHYvn077e3tJCUlDWhyEyeOHdBxAHWNrYwbO7rHdpcrEvcE14DPO9K43eOGegrDnnpkTz2yNxx6ZBv4wWAQh8MRGluW1W3cWz0QCPDcc8/x6quvDnhyDQ3NBIPWwA52Omlqbu+xubX1KIGurgHPaSRxu8cRCDQN9TSGNfXInnpk70z1KCzMcdIHZdslnejoaAKBQGgcCATweDy91uvr6/F4POzcuZOvv/6aBQsWkJaWBkBaWhrNzc0DuhARETk1toE/ffp0KioqaGxspK2tjbKyMhISEkL1mJgYIiMjqaqqAqC4uJiEhARuv/123n77bYqLiykuLg7Vxo4d+DKNiIgMnG3gR0VFkZWVRXp6OrNnzyY1NZXY2FgyMjLYv38/AHl5eeTm5pKUlERrayvp6emDPnEREekfh2VZA1wkH3ynsoZvOZ28W3Wwx/Zpl0UxJtL2RxdG0NqrPfXInnpk7zuzhi8iIiODAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETGEAl9ExBAKfBERQyjwRUQMocAXETFEnwK/pKSE5ORkEhMTKSgo6FGvrq7G5/Ph9XpZtmwZnZ2dAFRWVuLz+Zg1axb3338/R44cOb2zFxGRPrMNfL/fT35+PuvXr6eoqIjCwkJqamq67ZOdnU1OTg7btm3Dsiw2btwIwJIlS1izZg0lJSVMnjyZdevWDc5ViIiILdvALy8vJz4+nvHjx+NyufB6vZSWlobqtbW1tLe3ExcXB4DP5wvVt2zZwuTJk+no6MDv93P22WcP0mWIiIgd28Cvq6vD7XaHxh6PB7/f32vd7XaH6hERERw4cIAZM2awZ88eUlJSTufcRUSkH8LtdggGgzgcjtDYsqxuY7v6JZdcQnl5ORs2bCArK4sNGzb0eXITJ47t877fVtfYyrixo3tsd7kicU9wDfi8I43bPW6opzDsqUf21CN7w6FHtoEfHR1NZWVlaBwIBPB4PN3qgUAgNK6vr8fj8XD06FF27drFTTfdBMCtt97K6tWr+zW5hoZmgkGrX8eEOJ00Nbf32NzaepRAV9fAzjnCuN3jCASahnoaw5p6ZE89snemehQW5jjpg7Ltks706dOpqKigsbGRtrY2ysrKSEhICNVjYmKIjIykqqoKgOLiYhISEggPD+exxx7jww8/BGDr1q1cddVVp3o9IiIyQLZP+FFRUWRlZZGenk5HRwdz584lNjaWjIwMMjMzmTp1Knl5eSxfvpzm5mamTJlCeno6TqeT/Px8cnJy6OrqIioqiieeeOJMXJOIiJyAw7KsAa6ZDL5TWdKxnE7erTrYY/u0y6IYE2n7dc4I+lbcnnpkTz2y951Z0hERkZFBgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYog+BX5JSQnJyckkJiZSUFDQo15dXY3P58Pr9bJs2TI6OzsBqKqqYu7cuaSlpbFw4UJqa2tP7+xFRKTPbAPf7/eTn5/P+vXrKSoqorCwkJqamm77ZGdnk5OTw7Zt27Asi40bN4a2r1ixguLiYmbNmsWKFSsG5ypERMSWbeCXl5cTHx/P+PHjcblceL1eSktLQ/Xa2lra29uJi4sDwOfzUVpayrFjx3jwwQe59NJLAbjkkks4fPjwIF2GiIjYCbfboa6uDrfbHRp7PB727dvXa93tduP3+xk1ahRpaWkABINB1q5dy0033dSvyU2cOLZf+3ebd2Mr48aO7rHd5YrEPcE14POONG73uKGewrCnHtlTj+wNhx7ZBn4wGMThcITGlmV1G9vVjx07xuLFi+ns7ORXv/pVvybX0NBMMGj165gQp5Om5vYem1tbjxLo6hrYOUcYt3scgUDTUE9jWFOP7KlH9s5Uj8LCHCd9ULZd0omOjiYQCITGgUAAj8fTa72+vj5Ub2lpYdGiRXR2dvLcc88RERExoIsQEZFTZxv406dPp6KigsbGRtra2igrKyMhISFUj4mJITIykqqqKgCKi4tD9ezsbC644AKefPJJRo0aNUiXICIifWG7pBMVFUVWVhbp6el0dHQwd+5cYmNjycjIIDMzk6lTp5KXl8fy5ctpbm5mypQppKen89FHH7F9+3YmT57MnDlzgG/W/1988cVBvygREenJYVnWABfJB9+prOFbTifvVh3ssX3aZVGMibT9OmcErb3aU4/sqUf2vjNr+CIiMjIo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQCnwREUP0KfBLSkpITk4mMTGRgoKCHvXq6mp8Ph9er5dly5bR2dnZrf7kk0/yzDPPnJ4Zi4jIgNgGvt/vJz8/n/Xr11NUVERhYSE1NTXd9snOziYnJ4dt27ZhWRYbN24EoKmpiaVLl/LKK68MzuxFRKTPbAO/vLyc+Ph4xo8fj8vlwuv1UlpaGqrX1tbS3t5OXFwcAD6fL1Tfvn07F154Iffcc88gTV9ERPrKNvDr6upwu92hscfjwe/391p3u92h+uzZs7nvvvtwOp2nc84iIjIA4XY7BINBHA5HaGxZVrexXf1UTJw4dsDH1jW2Mm7s6B7bXa5I3BNcpzKtEcXtHjfUUxj21CN76pG94dAj28CPjo6msrIyNA4EAng8nm71QCAQGtfX13ern4qGhmaCQWtgBzudNDW399jc2nqUQFfXKc5sZHC7xxEINA31NIY19cieemTvTPUoLMxx0gdl2yWd6dOnU1FRQWNjI21tbZSVlZGQkBCqx8TEEBkZSVVVFQDFxcXd6iIiMjzYBn5UVBRZWVmkp6cze/ZsUlNTiY2NJSMjg/379wOQl5dHbm4uSUlJtLa2kp6ePugTFxGR/nFYljXANZPBdypLOpbTybtVB3tsn3ZZFGMibVeyjKBvxe2pR/bUI3vfmSUdEREZGRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKGUOCLiBhCgS8iYggFvoiIIRT4IiKG6FPgl5SUkJycTGJiIgUFBT3q1dXV+Hw+vF4vy5Yto7OzE4B///vfLFiwgKSkJH7961/T0tJyemcvIiJ9Zhv4fr+f/Px81q9fT1FREYWFhdTU1HTbJzs7m5ycHLZt24ZlWWzcuBGAxx57jPnz51NaWsrll1/OX/7yl8G5ipOwLIsv65rZ85Gf/f+vgaoDdTT+t/2Mz0NEZKg5LMuyTrbDm2++yfvvv8/KlSsBePbZZ7EsiwceeACA2tpaFi5cyNtvvw1AZWUlTz/9NOvWreOaa65h7969hIeHc/jwYe688062b9/e58l99VULweBJp9dDVzDIzn/9m2NdQfwNLfy7vpWvm4/iDHPQddy5Yr43lvOjxhK0LLqCFpERTiJHOQE42tFFV5eFM8xBeFgYTqeDcKcDh8OBA3A4vvWh/7f9mz//7z/f/MHCgv/7WAugt3Yfd9Jvn/74DY6e1W4sejn/CTa7XKNobT120vP1Pqneduvjjt8RY8aMoqWljz0ylHpkr789uvyiCUw8e3S/PycszMG5547ptR5ud4K6ujrcbndo7PF42LdvX691t9uN3+/nq6++YuzYsYSHh3fb3h8nm/jJ3JF49oCOExEZyWyXdILBII7jnj4ty+o27q3+7f2AHmMRETlzbAM/OjqaQCAQGgcCATweT6/1+vp6PB4PEyZMoKmpia6urhMeJyIiZ5Zt4E+fPp2KigoaGxtpa2ujrKyMhISEUD0mJobIyEiqqqoAKC4uJiEhgYiICK6++mq2bNkCQFFRUbfjRETkzLL9oS1889cyn3/+eTo6Opg7dy4ZGRlkZGSQmZnJ1KlT+fjjj1m+fDnNzc1MmTKF3NxcRo0aRW1tLYsXL6ahoYFJkybx5z//mXPOOedMXJeIiHxLnwJfRES++/SbtiIihlDgi4gYQoEvImIIBb6IiCFGXODbvejNJGvXriUlJYWUlBTWrFkDQHl5ObNmzSIxMZH8/PzQvr29AM8Uq1evZvHixYBeBvhtO3bswOfzccstt7BixQpA99G3FRcXh/5fW716NTBM7yNrBPnPf/5jzZw50/rqq6+slpYWa9asWdann3461NMaErt377Z+/vOfW0ePHrWOHTtmpaenWyUlJdaMGTOsgwcPWh0dHda9995r7dy507Isy0pJSbH+9a9/WZZlWUuWLLEKCgqGcvpnVHl5uXXNNddYf/zjHy3L6r0X9913n7V582bLsixr7dq11po1a4ZmwmfQwYMHreuvv946fPiwdezYMesXv/iFtXPnTt1Hx2ltbbWmTZtmNTQ0WB0dHdbcuXOt3bt3D8v7aEQ94ZeXlxMfH8/48eNxuVx4vV5KS0uHelpDwu12s3jxYkaNGkVERAQ//OEP+fzzz7ngggs477zzCA8PZ9asWZSWllJbW0t7eztxcXEA+Hw+Y/r29ddfk5+fz/333w/Qay86Ojp4//338Xq93baPdP/85z9JTk4mOjqaiIgI8vPzOeuss3QfHaerq4tgMEhbWxudnZ10dnYSHh4+LO8j25enfZfYvejNJBdffHHoz59//jlbt27lzjvv7NEfv9/f6wvwTJCTk0NWVhaHDx8GBvdlgN9FX3zxBREREdx///0cPnyYG264gYsvvlj30XHGjh3Lgw8+yC233MJZZ53FtGnTiIiIGJb30Yh6wrd70ZuJPv30U+69914efvhhzjvvvBP2x9S+bdq0iUmTJnHttdeGtullgN11dXVRUVHBypUrKSwsZN++fRw6dEj30XE+/vhjXn/9dd555x127dpFWFgYu3fvHpb30Yh6wo+OjqaysjI0Nv2FbVVVVWRmZrJ06VJSUlLYu3fvCV+E19sL8Ea6LVu2EAgESEtL48iRI7S2tuJwOGxfBuh0Oo25t773ve9x7bXXMmHCBABuuukmSktLcTqdoX1Mv4/ee+89rr32WiZOnAh8s0yzbt26YXkfjagnfLsXvZnk8OHD/Pa3vyUvL4+UlBQArrjiCj777DO++OILurq62Lx5MwkJCb2+AG+ke+WVV9i8eTPFxcVkZmZy4403kpubq5cBHmfmzJm89957/Pe//6Wrq4tdu3aRlJSk++g4l156KeXl5bS2tmJZFjt27OAnP/nJsLyPRty7dE70ojcTrVixgtdff53zzz8/tG3evHlceOGF5ObmcvToUWbMmMGSJUtwOBy9vgDPFG+88QZ79+5l1apVehngt7z22mu8+uqrdHR0cN1117F8+XL27Nmj++g4L7zwAm+88QYRERFMnTqVRx55hM8++2zY3UcjLvBFROTERtSSjoiI9E6BLyJiCAW+iIghFPgiIoZQ4IuIGEKBL8bYt28fOTk5ofG9995LY2PjCffNyMigpqaGPXv2kJqa2u/P2rRpU5/e1trS0sKDDz5IamoqycnJrFu3rt+fJdJXI+o3bUVOpqamptt7S3bv3t3rvi+++CIADQ0NA/qsqqqqbu8z6s3LL7/M6NGj2bx5M83NzaSkpDBt2jRiY2MH9LkiJ6PAlxEnGAyycuVKPvjgA1paWrAsi8cee4ynn36apqYmlixZEtp34cKFvPDCCyxYsIDY2FgOHDjAQw89RG5uLk899RQAra2tZGZm8sUXX3D22Wfz+OOP84Mf/IDFixdz8cUX88tf/hIgND7//PPZsWMHu3fvZvTo0SxYsIDnnnuOsrIygsEgMTExPPLII0RFRdHV1UVLSwudnZ0cPXqUYDBoxC8qydDQko6MOB988AF1dXUUFhayZcsW5syZw8svv0xmZiZXX301ubm55ObmAvDXv/6VSZMmAd+8YXTr1q3cfPPN3c53+PBh7r77boqLi0lNTeXhhx8+6efffPPN3Hjjjdx9990sWLCAoqIiPvnkEzZt2kRxcTEzZsxg+fLlACxatIja2lp++tOfcsMNN5CSksKll146CF0R0RO+jEBXXnkl55xzDhs2bODQoUPs2bOHMWPG2B539dVXn3D7JZdcwlVXXQXAnDlzePTRR2lqaurzfN555x3279/PbbfdBhB6dzrA448/znXXXcdDDz1EfX0999xzD1deeWXofekip5MCX0acnTt38sQTT3DPPffws5/9jIsuuoi33nrL9jiXy3XC7WFh3b8RdjgchIeHh153+z8dHR0nPD4YDLJo0SLmz58PwLFjxzhy5AjwzT8w8tZbbxEWFobH4yEpKYk9e/Yo8GVQaElHRpzdu3czc+ZM5s+fz+WXX87bb78deh3t8f/G6rfHvTlw4ADV1dUAFBYW8uMf/5izzjqLc889lw8//BAAv9/P3r17T3ju66+/ntdee43m5mYAnnrqqdCy0I9+9CO2bt0KfPOzgl27dnHFFVechi6I9KTAlxFn3rx57N27l1mzZjFnzhzOO+88vvzyS2JjYzl06BAPPPAAAElJSdx111188sknJz3fRRddxNq1a7n11lvZsWMHq1atAuCuu+4iEAjg9XpZunQp8fHxoWMSEhLYsGEDzz//PLfffjs33HADd9xxBykpKRw4cCB0jtWrV1NZWUlycjJ33HEHM2bMIC0tbZA6I6bT2zJFRAyhJ3wREUMo8EVEDKHAFxExhAJfRMQQCnwREUMo8EVEDKHAFxExhAJfRMQQ/x/OkySkm9GK4gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['attribute8'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we'll standardize in later cross-validation loop to avoid possible 'data leakage'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Feature Engineering\n",
    "\n",
    "- We hypothesize that device failure may reflect scheduling pattern. We explicitely model weekday, Monday to Sunday, to simulate the effect.\n",
    "\n",
    "- We also suspect seasonable effect. We model month, Janaury to December, to gauge the impact. \n",
    "\n",
    "- We calculate the days the device is in service before failure. We use this to pick up effect of usage on failure. \n",
    "\n",
    "- In terms of device type, namely 'S1F', 'W1F', 'Z1F'. We model it to explore influence of device family on failure. \n",
    "\n",
    "- For attributes, we extract sensor readings of the past 1 day, past 2 days, and past 3 days to align with the label for model training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>date2</th>\n",
       "      <th>day_of_week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure      date2  day_of_week\n",
       "0  2015-01-01  S1F01085        0 2015-01-01            3\n",
       "1  2015-01-01  S1F0166B        0 2015-01-01            3\n",
       "2  2015-01-01  S1F01E6Y        0 2015-01-01            3\n",
       "3  2015-01-01  S1F01JE0        0 2015-01-01            3\n",
       "4  2015-01-01  S1F01R2B        0 2015-01-01            3"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# day of week\n",
    "\n",
    "df1 = df.copy()\n",
    "df1 = df1.drop(['attribute1', 'attribute2', 'attribute3', 'attribute4', 'attribute5', 'attribute6', 'attribute7', 'attribute8'], axis = 1)\n",
    "\n",
    "df1['date2'] = pd.to_datetime(df1['date'])\n",
    "df1['day_of_week'] = df1['date2'].dt.weekday\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>date2</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>month_of_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure      date2  day_of_week  month_of_year\n",
       "0  2015-01-01  S1F01085        0 2015-01-01            3              1\n",
       "1  2015-01-01  S1F0166B        0 2015-01-01            3              1\n",
       "2  2015-01-01  S1F01E6Y        0 2015-01-01            3              1\n",
       "3  2015-01-01  S1F01JE0        0 2015-01-01            3              1\n",
       "4  2015-01-01  S1F01R2B        0 2015-01-01            3              1"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# month of year\n",
    "\n",
    "df1['month_of_year'] = df1['date2'].dt.month\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>date2</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>month_of_year</th>\n",
       "      <th>service_days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>222</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure      date2  day_of_week  month_of_year  \\\n",
       "0  2015-01-01  S1F01085        0 2015-01-01            3              1   \n",
       "1  2015-01-01  S1F0166B        0 2015-01-01            3              1   \n",
       "2  2015-01-01  S1F01E6Y        0 2015-01-01            3              1   \n",
       "3  2015-01-01  S1F01JE0        0 2015-01-01            3              1   \n",
       "4  2015-01-01  S1F01R2B        0 2015-01-01            3              1   \n",
       "\n",
       "   service_days  \n",
       "0             5  \n",
       "1             5  \n",
       "2            47  \n",
       "3             5  \n",
       "4           222  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# service days\n",
    "\n",
    "service_days_map = (df1.groupby('device')['date'].count() -1).to_dict()\n",
    "df1['service_days'] = df1['device'].map(service_days_map)\n",
    "\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>device</th>\n",
       "      <th>failure</th>\n",
       "      <th>date2</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>month_of_year</th>\n",
       "      <th>service_days</th>\n",
       "      <th>device_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01085</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F0166B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01E6Y</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01JE0</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>S1F01R2B</td>\n",
       "      <td>0</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>222</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date    device  failure      date2  day_of_week  month_of_year  \\\n",
       "0  2015-01-01  S1F01085        0 2015-01-01            3              1   \n",
       "1  2015-01-01  S1F0166B        0 2015-01-01            3              1   \n",
       "2  2015-01-01  S1F01E6Y        0 2015-01-01            3              1   \n",
       "3  2015-01-01  S1F01JE0        0 2015-01-01            3              1   \n",
       "4  2015-01-01  S1F01R2B        0 2015-01-01            3              1   \n",
       "\n",
       "   service_days device_type  \n",
       "0             5         S1F  \n",
       "1             5         S1F  \n",
       "2            47         S1F  \n",
       "3             5         S1F  \n",
       "4           222         S1F  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# device type\n",
    "\n",
    "df1['device_type'] = df1['device'].str[:3]\n",
    "\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reindex using device and date to faciliate device/time specific feature extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = df1.drop(['date2'], axis = 1)\n",
    "df1.sort_values(by=['device', 'date'])\n",
    "df1 = df1.set_index(['device', 'date'], drop = True)\n",
    "df1 = df1.sort_index(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>failure</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>month_of_year</th>\n",
       "      <th>service_days</th>\n",
       "      <th>device_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>S1F01085</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F013BB</th>\n",
       "      <th>2015-05-11</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F0166B</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01E6Y</th>\n",
       "      <th>2015-02-17</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>47</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01JE0</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     failure  day_of_week  month_of_year  service_days  \\\n",
       "device   date                                                            \n",
       "S1F01085 2015-01-06        0            1              1             5   \n",
       "S1F013BB 2015-05-11        0            0              5             5   \n",
       "S1F0166B 2015-01-06        0            1              1             5   \n",
       "S1F01E6Y 2015-02-17        0            1              2            47   \n",
       "S1F01JE0 2015-01-06        0            1              1             5   \n",
       "\n",
       "                    device_type  \n",
       "device   date                    \n",
       "S1F01085 2015-01-06         S1F  \n",
       "S1F013BB 2015-05-11         S1F  \n",
       "S1F0166B 2015-01-06         S1F  \n",
       "S1F01E6Y 2015-02-17         S1F  \n",
       "S1F01JE0 2015-01-06         S1F  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1_cut = df1.groupby(level=0).tail(1).dropna()\n",
    "df1_cut.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# construct sensor reading from previous day, previous 2 days, and previous 3 days as features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "df2 = df.copy()\n",
    "df2.sort_values(by=['device', 'date'])\n",
    "df2 = df2.set_index(['device', 'date'], drop = True)\n",
    "df2 = df2.sort_index(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "df2.drop(['failure'], axis =1, inplace = True)\n",
    "df_pre1day = df2.groupby(level=0).shift(periods = 1, fill_value = np.NaN)\n",
    "df_pre2day = df2.groupby(level=0).shift(periods = 2, fill_value = np.NaN)\n",
    "df_pre3day = df2.groupby(level=0).shift(periods = 3, fill_value = np.NaN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>att1_pre1day</th>\n",
       "      <th>att2_pre1day</th>\n",
       "      <th>att3_pre1day</th>\n",
       "      <th>att4_pre1day</th>\n",
       "      <th>att5_pre1day</th>\n",
       "      <th>att6_pre1day</th>\n",
       "      <th>att7_pre1day</th>\n",
       "      <th>att8_pre1day</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">S1F01085</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>215630672.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-03</th>\n",
       "      <td>1650864.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-04</th>\n",
       "      <td>124017368.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>128073224.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407439.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     att1_pre1day  att2_pre1day  att3_pre1day  att4_pre1day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN   \n",
       "         2015-01-02   215630672.0          56.0           0.0          52.0   \n",
       "         2015-01-03     1650864.0          56.0           0.0          52.0   \n",
       "         2015-01-04   124017368.0          56.0           0.0          52.0   \n",
       "         2015-01-05   128073224.0          56.0           0.0          52.0   \n",
       "\n",
       "                     att5_pre1day  att6_pre1day  att7_pre1day  att8_pre1day  \n",
       "device   date                                                                \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN  \n",
       "         2015-01-02           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-03           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-04           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-05           6.0      407439.0           0.0           0.0  "
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_pre1day = df_pre1day.rename(columns = {'attribute1': 'att1_pre1day', 'attribute2': 'att2_pre1day', 'attribute3': 'att3_pre1day', 'attribute4': 'att4_pre1day', 'attribute5': 'att5_pre1day', 'attribute6': 'att6_pre1day', 'attribute7': 'att7_pre1day', 'attribute8': 'att8_pre1day'})\n",
    "df_pre1day.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>att1_pre2day</th>\n",
       "      <th>att2_pre2day</th>\n",
       "      <th>att3_pre2day</th>\n",
       "      <th>att4_pre2day</th>\n",
       "      <th>att5_pre2day</th>\n",
       "      <th>att6_pre2day</th>\n",
       "      <th>att7_pre2day</th>\n",
       "      <th>att8_pre2day</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">S1F01085</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-03</th>\n",
       "      <td>215630672.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-04</th>\n",
       "      <td>1650864.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>124017368.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     att1_pre2day  att2_pre2day  att3_pre2day  att4_pre2day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN   \n",
       "         2015-01-02           NaN           NaN           NaN           NaN   \n",
       "         2015-01-03   215630672.0          56.0           0.0          52.0   \n",
       "         2015-01-04     1650864.0          56.0           0.0          52.0   \n",
       "         2015-01-05   124017368.0          56.0           0.0          52.0   \n",
       "\n",
       "                     att5_pre2day  att6_pre2day  att7_pre2day  att8_pre2day  \n",
       "device   date                                                                \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN  \n",
       "         2015-01-02           NaN           NaN           NaN           NaN  \n",
       "         2015-01-03           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-04           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-05           6.0      407438.0           0.0           0.0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_pre2day = df_pre2day.rename(columns = {'attribute1': 'att1_pre2day', 'attribute2': 'att2_pre2day', 'attribute3': 'att3_pre2day', 'attribute4': 'att4_pre2day', 'attribute5': 'att5_pre2day', 'attribute6': 'att6_pre2day', 'attribute7': 'att7_pre2day', 'attribute8': 'att8_pre2day'})\n",
    "df_pre2day.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>att1_pre3day</th>\n",
       "      <th>att2_pre3day</th>\n",
       "      <th>att3_pre3day</th>\n",
       "      <th>att4_pre3day</th>\n",
       "      <th>att5_pre3day</th>\n",
       "      <th>att6_pre3day</th>\n",
       "      <th>att7_pre3day</th>\n",
       "      <th>att8_pre3day</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">S1F01085</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-02</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-03</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-04</th>\n",
       "      <td>215630672.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015-01-05</th>\n",
       "      <td>1650864.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     att1_pre3day  att2_pre3day  att3_pre3day  att4_pre3day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN   \n",
       "         2015-01-02           NaN           NaN           NaN           NaN   \n",
       "         2015-01-03           NaN           NaN           NaN           NaN   \n",
       "         2015-01-04   215630672.0          56.0           0.0          52.0   \n",
       "         2015-01-05     1650864.0          56.0           0.0          52.0   \n",
       "\n",
       "                     att5_pre3day  att6_pre3day  att7_pre3day  att8_pre3day  \n",
       "device   date                                                                \n",
       "S1F01085 2015-01-01           NaN           NaN           NaN           NaN  \n",
       "         2015-01-02           NaN           NaN           NaN           NaN  \n",
       "         2015-01-03           NaN           NaN           NaN           NaN  \n",
       "         2015-01-04           6.0      407438.0           0.0           0.0  \n",
       "         2015-01-05           6.0      407438.0           0.0           0.0  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_pre3day = df_pre3day.rename(columns = {'attribute1': 'att1_pre3day', 'attribute2': 'att2_pre3day', 'attribute3': 'att3_pre3day', 'attribute4': 'att4_pre3day', 'attribute5': 'att5_pre3day', 'attribute6': 'att6_pre3day', 'attribute7': 'att7_pre3day', 'attribute8': 'att8_pre3day'})\n",
    "df_pre3day.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>att1_pre1day</th>\n",
       "      <th>att2_pre1day</th>\n",
       "      <th>att3_pre1day</th>\n",
       "      <th>att4_pre1day</th>\n",
       "      <th>att5_pre1day</th>\n",
       "      <th>att6_pre1day</th>\n",
       "      <th>att7_pre1day</th>\n",
       "      <th>att8_pre1day</th>\n",
       "      <th>att1_pre2day</th>\n",
       "      <th>att2_pre2day</th>\n",
       "      <th>...</th>\n",
       "      <th>att7_pre2day</th>\n",
       "      <th>att8_pre2day</th>\n",
       "      <th>att1_pre3day</th>\n",
       "      <th>att2_pre3day</th>\n",
       "      <th>att3_pre3day</th>\n",
       "      <th>att4_pre3day</th>\n",
       "      <th>att5_pre3day</th>\n",
       "      <th>att6_pre3day</th>\n",
       "      <th>att7_pre3day</th>\n",
       "      <th>att8_pre3day</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>S1F01085</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>97393448.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>408114.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>128073224.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>124017368.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F013BB</th>\n",
       "      <th>2015-05-11</th>\n",
       "      <td>85127128.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>689062.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>243346080.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>214383720.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>688964.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F0166B</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>224339296.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>403812.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>82195688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>77681360.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>403174.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01E6Y</th>\n",
       "      <th>2015-02-17</th>\n",
       "      <td>182876688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>259486.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>227370128.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>83214688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>259468.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01JE0</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>158246712.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>410888.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>235473648.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>235562856.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>410186.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     att1_pre1day  att2_pre1day  att3_pre1day  att4_pre1day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06    97393448.0          56.0           0.0          52.0   \n",
       "S1F013BB 2015-05-11    85127128.0           0.0           0.0           0.0   \n",
       "S1F0166B 2015-01-06   224339296.0           0.0           3.0           0.0   \n",
       "S1F01E6Y 2015-02-17   182876688.0           0.0           0.0           0.0   \n",
       "S1F01JE0 2015-01-06   158246712.0           0.0           0.0           0.0   \n",
       "\n",
       "                     att5_pre1day  att6_pre1day  att7_pre1day  att8_pre1day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06           6.0      408114.0           0.0           0.0   \n",
       "S1F013BB 2015-05-11           5.0      689062.0           0.0           0.0   \n",
       "S1F0166B 2015-01-06           6.0      403812.0           0.0           0.0   \n",
       "S1F01E6Y 2015-02-17          12.0      259486.0           0.0           0.0   \n",
       "S1F01JE0 2015-01-06           6.0      410888.0           0.0           0.0   \n",
       "\n",
       "                     att1_pre2day  att2_pre2day  ...  att7_pre2day  \\\n",
       "device   date                                    ...                 \n",
       "S1F01085 2015-01-06   128073224.0          56.0  ...           0.0   \n",
       "S1F013BB 2015-05-11   243346080.0           0.0  ...           0.0   \n",
       "S1F0166B 2015-01-06    82195688.0           0.0  ...           0.0   \n",
       "S1F01E6Y 2015-02-17   227370128.0           0.0  ...           0.0   \n",
       "S1F01JE0 2015-01-06   235473648.0           0.0  ...           0.0   \n",
       "\n",
       "                     att8_pre2day  att1_pre3day  att2_pre3day  att3_pre3day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06           0.0   124017368.0          56.0           0.0   \n",
       "S1F013BB 2015-05-11           0.0   214383720.0           0.0           0.0   \n",
       "S1F0166B 2015-01-06           0.0    77681360.0           0.0           3.0   \n",
       "S1F01E6Y 2015-02-17           0.0    83214688.0           0.0           0.0   \n",
       "S1F01JE0 2015-01-06           0.0   235562856.0           0.0           0.0   \n",
       "\n",
       "                     att4_pre3day  att5_pre3day  att6_pre3day  att7_pre3day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06          52.0           6.0      407438.0           0.0   \n",
       "S1F013BB 2015-05-11           0.0           5.0      688964.0           0.0   \n",
       "S1F0166B 2015-01-06           0.0           6.0      403174.0           0.0   \n",
       "S1F01E6Y 2015-02-17           0.0          12.0      259468.0           0.0   \n",
       "S1F01JE0 2015-01-06           0.0           6.0      410186.0           0.0   \n",
       "\n",
       "                     att8_pre3day  \n",
       "device   date                      \n",
       "S1F01085 2015-01-06           0.0  \n",
       "S1F013BB 2015-05-11           0.0  \n",
       "S1F0166B 2015-01-06           0.0  \n",
       "S1F01E6Y 2015-02-17           0.0  \n",
       "S1F01JE0 2015-01-06           0.0  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# merge sensor reading of all 8 attributes, from previous 1 day to 3 days. \n",
    "\n",
    "df2_total_cut = pd.concat([df_pre1day, df_pre2day, df_pre3day], axis = 1).dropna().groupby(level = 0).tail(1).dropna()\n",
    "df2_total_cut.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>failure</th>\n",
       "      <th>day_of_week</th>\n",
       "      <th>month_of_year</th>\n",
       "      <th>service_days</th>\n",
       "      <th>device_type</th>\n",
       "      <th>att1_pre1day</th>\n",
       "      <th>att2_pre1day</th>\n",
       "      <th>att3_pre1day</th>\n",
       "      <th>att4_pre1day</th>\n",
       "      <th>att5_pre1day</th>\n",
       "      <th>...</th>\n",
       "      <th>att7_pre2day</th>\n",
       "      <th>att8_pre2day</th>\n",
       "      <th>att1_pre3day</th>\n",
       "      <th>att2_pre3day</th>\n",
       "      <th>att3_pre3day</th>\n",
       "      <th>att4_pre3day</th>\n",
       "      <th>att5_pre3day</th>\n",
       "      <th>att6_pre3day</th>\n",
       "      <th>att7_pre3day</th>\n",
       "      <th>att8_pre3day</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>S1F01085</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "      <td>97393448.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>124017368.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>407438.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F013BB</th>\n",
       "      <th>2015-05-11</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "      <td>85127128.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>214383720.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>688964.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F0166B</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "      <td>224339296.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>77681360.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>403174.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01E6Y</th>\n",
       "      <th>2015-02-17</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>47</td>\n",
       "      <td>S1F</td>\n",
       "      <td>182876688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>83214688.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>259468.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S1F01JE0</th>\n",
       "      <th>2015-01-06</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>S1F</td>\n",
       "      <td>158246712.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>235562856.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>410186.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                     failure  day_of_week  month_of_year  service_days  \\\n",
       "device   date                                                            \n",
       "S1F01085 2015-01-06        0            1              1             5   \n",
       "S1F013BB 2015-05-11        0            0              5             5   \n",
       "S1F0166B 2015-01-06        0            1              1             5   \n",
       "S1F01E6Y 2015-02-17        0            1              2            47   \n",
       "S1F01JE0 2015-01-06        0            1              1             5   \n",
       "\n",
       "                    device_type  att1_pre1day  att2_pre1day  att3_pre1day  \\\n",
       "device   date                                                               \n",
       "S1F01085 2015-01-06         S1F    97393448.0          56.0           0.0   \n",
       "S1F013BB 2015-05-11         S1F    85127128.0           0.0           0.0   \n",
       "S1F0166B 2015-01-06         S1F   224339296.0           0.0           3.0   \n",
       "S1F01E6Y 2015-02-17         S1F   182876688.0           0.0           0.0   \n",
       "S1F01JE0 2015-01-06         S1F   158246712.0           0.0           0.0   \n",
       "\n",
       "                     att4_pre1day  att5_pre1day  ...  att7_pre2day  \\\n",
       "device   date                                    ...                 \n",
       "S1F01085 2015-01-06          52.0           6.0  ...           0.0   \n",
       "S1F013BB 2015-05-11           0.0           5.0  ...           0.0   \n",
       "S1F0166B 2015-01-06           0.0           6.0  ...           0.0   \n",
       "S1F01E6Y 2015-02-17           0.0          12.0  ...           0.0   \n",
       "S1F01JE0 2015-01-06           0.0           6.0  ...           0.0   \n",
       "\n",
       "                     att8_pre2day  att1_pre3day  att2_pre3day  att3_pre3day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06           0.0   124017368.0          56.0           0.0   \n",
       "S1F013BB 2015-05-11           0.0   214383720.0           0.0           0.0   \n",
       "S1F0166B 2015-01-06           0.0    77681360.0           0.0           3.0   \n",
       "S1F01E6Y 2015-02-17           0.0    83214688.0           0.0           0.0   \n",
       "S1F01JE0 2015-01-06           0.0   235562856.0           0.0           0.0   \n",
       "\n",
       "                     att4_pre3day  att5_pre3day  att6_pre3day  att7_pre3day  \\\n",
       "device   date                                                                 \n",
       "S1F01085 2015-01-06          52.0           6.0      407438.0           0.0   \n",
       "S1F013BB 2015-05-11           0.0           5.0      688964.0           0.0   \n",
       "S1F0166B 2015-01-06           0.0           6.0      403174.0           0.0   \n",
       "S1F01E6Y 2015-02-17           0.0          12.0      259468.0           0.0   \n",
       "S1F01JE0 2015-01-06           0.0           6.0      410186.0           0.0   \n",
       "\n",
       "                     att8_pre3day  \n",
       "device   date                      \n",
       "S1F01085 2015-01-06           0.0  \n",
       "S1F013BB 2015-05-11           0.0  \n",
       "S1F0166B 2015-01-06           0.0  \n",
       "S1F01E6Y 2015-02-17           0.0  \n",
       "S1F01JE0 2015-01-06           0.0  \n",
       "\n",
       "[5 rows x 29 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# merge all features\n",
    "\n",
    "df_w = pd.merge(left = df1_cut, left_index = True, right = df2_total_cut, right_index = True, how = 'inner')\n",
    "df_w.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# assemble categorical features (aka dummy coding variables)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dfw_1    2\n",
       "dfw_2    2\n",
       "dfw_3    2\n",
       "dfw_4    2\n",
       "dfw_5    2\n",
       "dfw_6    2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dummy coding day of week\n",
    "\n",
    "day_of_week = pd.get_dummies(df_w['day_of_week'], prefix = 'dfw', drop_first = True)\n",
    "day_of_week.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "mfy_2     2\n",
       "mfy_3     2\n",
       "mfy_4     2\n",
       "mfy_5     2\n",
       "mfy_6     2\n",
       "mfy_7     2\n",
       "mfy_8     2\n",
       "mfy_9     2\n",
       "mfy_10    2\n",
       "mfy_11    2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dummy coding month of year\n",
    "\n",
    "month_of_year = pd.get_dummies(df_w['month_of_year'], prefix = 'mfy', drop_first = True)\n",
    "month_of_year.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device_W1F    2\n",
       "device_Z1F    2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dummy coding device type\n",
    "\n",
    "device_type = pd.get_dummies(df_w['device_type'], prefix = 'device', drop_first = True)\n",
    "device_type.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1167, 18)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_columns = pd.concat([day_of_week, month_of_year, device_type], axis = 1)\n",
    "categorical_columns.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# assemble numerical features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['service_days', 'att1_pre1day', 'att2_pre1day', 'att3_pre1day', 'att4_pre1day', 'att5_pre1day', 'att6_pre1day', 'att7_pre1day', 'att8_pre1day', 'att1_pre2day', 'att2_pre2day', 'att3_pre2day', 'att4_pre2day', 'att5_pre2day', 'att6_pre2day', 'att7_pre2day', 'att8_pre2day', 'att1_pre3day', 'att2_pre3day', 'att3_pre3day', 'att4_pre3day', 'att5_pre3day', 'att6_pre3day', 'att7_pre3day', 'att8_pre3day']\n"
     ]
    }
   ],
   "source": [
    "numerical_features = (df_w.columns[3:4]).append(df_w.columns[5:]).tolist()\n",
    "print(numerical_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1167, 25)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numerical_columns = df_w[numerical_features]\n",
    "numerical_columns.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1167, 1)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# identify label \n",
    "\n",
    "label = df_w[['failure']]\n",
    "label.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1167, 44)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create working dataset for analysis\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.concat([categorical_columns, numerical_columns, label], axis=1)\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# enforicng numerical datatype so it works with scikit-learn \n",
    "\n",
    "df = df.astype(float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1167, 44)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['dfw_1', 'dfw_2', 'dfw_3', 'dfw_4', 'dfw_5', 'dfw_6', 'mfy_2', 'mfy_3',\n",
       "       'mfy_4', 'mfy_5', 'mfy_6', 'mfy_7', 'mfy_8', 'mfy_9', 'mfy_10',\n",
       "       'mfy_11', 'device_W1F', 'device_Z1F', 'service_days', 'att1_pre1day',\n",
       "       'att2_pre1day', 'att3_pre1day', 'att4_pre1day', 'att5_pre1day',\n",
       "       'att6_pre1day', 'att7_pre1day', 'att8_pre1day', 'att1_pre2day',\n",
       "       'att2_pre2day', 'att3_pre2day', 'att4_pre2day', 'att5_pre2day',\n",
       "       'att6_pre2day', 'att7_pre2day', 'att8_pre2day', 'att1_pre3day',\n",
       "       'att2_pre3day', 'att3_pre3day', 'att4_pre3day', 'att5_pre3day',\n",
       "       'att6_pre3day', 'att7_pre3day', 'att8_pre3day', 'failure'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Model Validation Setup\n",
    "\n",
    "- We purposely set aside 20% of data as hold-out. We reserve this pool to validate our model performance. \n",
    "- We use stratified sampling so for each fold of parameter estimation, keeping percentage of non-failure vs. failure classes more less fixed. \n",
    "- We also fix random seeding so results are reproducible. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Splitting validation dataset\n",
    "\n",
    "X = df.drop(['failure'], axis = 1).values \n",
    "Y = df['failure'].values\n",
    "\n",
    "validation_size = 0.20 \n",
    "seed = 7 \n",
    "np.random.seed(7)\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_validation, Y_train, Y_validation = train_test_split(X, Y, \n",
    "                                                                test_size=validation_size, \n",
    "                                                                random_state=seed, \n",
    "                                                                stratify=Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1167, 43)\n",
      "(1167,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(Y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(933, 43)\n",
      "(933,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape)\n",
    "print(Y_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(234, 43)\n",
      "(234,)\n"
     ]
    }
   ],
   "source": [
    "print(X_validation.shape)\n",
    "print(Y_validation.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    852\n",
       "1.0     81\n",
       "dtype: int64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(Y_train).value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Resampling Training Data\n",
    "\n",
    "- With the training dataset (80% of the whole), we over-sample minority class (failure instance) to address the challenge of imbalnced dataset. Numerically the training dataset has 933 cases, of which 852 are no failure, and 81 are failure cases. After resampling, the training dataset has equal number of 848 no failure and 848 failure cases. The size of the augmented trainig dataset increases from 1,167 to 1,704. \n",
    "- With resampling algorithms, we have few options, ranging from over-sampling of the minority class, under-sampling of the majority class, to ensemble of combining both variants. For intial iteration, we keep model simple per principle of Occam's razor. We'll phase in more complex resampling scheme should initial models fail to yield satisfactory result. \n",
    "- Notice we do not need to resample the validation dataset. Ultimatley our model needs to correctly classify cases in the validation dataset as is. Numerically we need to correctly (or as many as possible) pick out the 20 cases that are actual failure cases, from the pool of 234, without mis-classifying others as false positives or false negatives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# over-sampling minority class to deal with imbalanced dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "# option1: SMOTE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1704, 43), (1704,))"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from imblearn.over_sampling import SMOTE\n",
    "from collections import Counter\n",
    "\n",
    "smote = SMOTE(ratio = 'minority', random_state = seed)\n",
    "X_train, Y_train = smote.fit_sample(X_train, Y_train)\n",
    "X_train.shape, Y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0    852\n",
       "0.0    852\n",
       "dtype: int64"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(Y_train).value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "# option 2: ADASYN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from imblearn.over_sampling import ADASYN\n",
    "\n",
    "# adasyn = ADASYN(ratio = 'minority', random_state = seed)\n",
    "# X_train, Y_train = adasyn.fit_sample(X_train, Y_train)\n",
    "# X_train.shape, Y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# unique_elements, counts_elements = np.unique(Y_train, return_counts=True)\n",
    "# print(\"Frequency of unique values of the said array:\")\n",
    "# print(np.asarray((unique_elements, counts_elements)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# option 3: TOMELINKS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from imblearn.under_sampling import TomekLinks\n",
    "# from collections import Counter\n",
    "\n",
    "# tome = TomekLinks(return_indices=True, ratio='auto', random_state = seed)\n",
    "\n",
    "# # Fit data\n",
    "# X_train, Y_train, id_tome = tome.fit_sample(X_train, Y_train)\n",
    "# X_train.shape, Y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Find removed indices\n",
    "# idx_samples_removed = np.setdiff1d(np.arange(X_train.shape[0]),id_tome)\n",
    "\n",
    "# len(idx_samples_removed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "# unique_elements, counts_elements = np.unique(Y_train, return_counts=True)\n",
    "# print(\"Frequency of unique values of the said array:\")\n",
    "# print(np.asarray((unique_elements, counts_elements)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "# OPTION 4: ensemble via SMOTETOMEK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from imblearn.over_sampling import SMOTE\n",
    "# from imblearn.combine import SMOTETomek\n",
    "\n",
    "# smote = SMOTE(ratio = 'minority', random_state = seed)\n",
    "# smto = SMOTETomek(ratio='auto', smote=smote, random_state = seed)\n",
    "\n",
    "# X_train, Y_train = smto.fit_sample(X_train, Y_train)\n",
    "# X_train.shape, Y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import numpy as np\n",
    "\n",
    "# unique_elements, counts_elements = np.unique(Y_train, return_counts=True)\n",
    "# print(\"Frequency of unique values of the said array:\")\n",
    "# print(np.asarray((unique_elements, counts_elements)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "# class distribution in the validation dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    214\n",
       "1.0     20\n",
       "dtype: int64"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(Y_validation).value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save cleaned working dataset to file\n",
    "\n",
    "np.save('X_train.npy', X_train)\n",
    "np.save('X_validation.npy', X_validation)\n",
    "\n",
    "np.save('Y_train.npy', Y_train)\n",
    "np.save('Y_validation.npy', Y_validation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load in cleaned working dataset to notebook \n",
    "\n",
    "X_train = np.load('X_train.npy')\n",
    "X_validation = np.load('X_validation.npy')\n",
    "\n",
    "Y_train = np.load('Y_train.npy') \n",
    "Y_validation = np.load('Y_validation.npy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Fitting Baseline Model\n",
    "\n",
    "- We use AUC as the scoring criteria to guide our model search, as we move across folds of the training dataset to estimate model parameters.  \n",
    "- We use confusion matrix to provide diagnostics of how well the model is performing in validation, balancing false positives vs. false negatives.\n",
    "- Our baseline is that of naive prediction, using majority class of the training dataset (non-failure) to predict cases in validation dataset. In essence naive model says all devices in validation are not faulty. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get baseline prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set cross-validation option and model evaluation metric\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "num_folds = 10\n",
    "seed = 7\n",
    "np.random.seed(7)\n",
    "\n",
    "scoring = 'roc_auc'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most frequent class (dummy classifier)\n",
      " [[214   0]\n",
      " [ 20   0]]\n"
     ]
    }
   ],
   "source": [
    "# predict class using majority class from training dataset\n",
    "\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "dummy_majority = DummyClassifier(random_state = seed, strategy = 'most_frequent').fit(X_train, Y_train)\n",
    "Y_majority_predicted = dummy_majority.predict(X_validation)\n",
    "confusion = confusion_matrix(Y_validation, Y_majority_predicted)\n",
    "\n",
    "print('Most frequent class (dummy classifier)\\n', confusion)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.5\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, Y_majority_predicted))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Fitting Classification Models with Feature Standardization\n",
    "- We fit KNN and SVM models for first batch of comparisons with baseline model. AUC for KNN 0.90, and for SVM is 0.92. Both models perform significanlty better than baseline, with AUC of only 0.5.  \n",
    "- As KNN and SVM are distance-based models, and sensitive to feature scaling, we standardize all numerical features to yield robust results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "# single model with feature standardization\n",
    "\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "\n",
    "pipelines = []\n",
    "pipelines.append(('ScaledLR', Pipeline([('Scaler', StandardScaler()),('LR', LogisticRegression())])))\n",
    "pipelines.append(('ScaledLDA', Pipeline([('Scaler', StandardScaler()),('LDA', LinearDiscriminantAnalysis())])))\n",
    "pipelines.append(('ScaledKNN', Pipeline([('Scaler', StandardScaler()),('KNN', KNeighborsClassifier())])))\n",
    "pipelines.append(('ScaledCART', Pipeline([('Scaler', StandardScaler()),('CART', DecisionTreeClassifier())])))\n",
    "pipelines.append(('ScaledNB', Pipeline([('Scaler', StandardScaler()),('NB', GaussianNB())])))\n",
    "pipelines.append(('ScaledSVM', Pipeline([('Scaler', StandardScaler()),('SVM', SVC())]))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ScaledLR: 0.934008 (0.016232)\n",
      "ScaledLDA: 0.930512 (0.015550)\n",
      "ScaledKNN: 0.992901 (0.006504)\n",
      "ScaledCART: 0.961457 (0.054322)\n",
      "ScaledNB: 0.876912 (0.040798)\n",
      "ScaledSVM: 0.996514 (0.003938)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "results = []\n",
    "names = []\n",
    "for name, model in pipelines:\n",
    "  kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "  cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "  results.append(cv_results)\n",
    "  names.append(name)\n",
    "  msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "  print(msg) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEjCAYAAADQeG38AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdeVwV9f7H8Rcg4AKEyzngiluApahJgaaQZZLIkoRpaWYialdFbfmJoVYqmgqB2q00NdK01GJNQdN71bwQauV6VdxQXNhEYxFlOfP7w4dzJRcWETj0eT4ePXLObN8PM+e8Z75zzoyBoigKQggh/tYMa7sBQgghap+EgRBCCAkDIYQQEgZCCCGQMBBCCIGEgRBCCCQM9F5JSQkrVqzAzc2Nrl270rt3b9555x3OnTtXLctftmwZPj4+VZ7/22+/5fnnny93uqioKOzs7FixYsVd4wIDAwkICKhyG8rz1xp//vlnLl++DEBycjJ2dnYUFBQ8svUDXL9+nfDwcNzc3HBwcOD555/nk08+ITc395Gut7o96m0lHh0JAz336aefsnHjRmbMmEFCQgLLly+noKCAESNG6NUHSWxsLO3bt+fHH3+s8XWPGTOGVatWAXDx4kUmTZpEXl5eja0/Pz+fYcOGkZSURFBQEJs3b+bjjz9mz549jBkzhhs3btRYWx5WUFAQ8+bNq+1miCqQMNBzP/74I5MmTeK5556jTZs2ODg4EB4eTkFBAdu3b6/t5lVIZmYmycnJTJkyhdTUVPbv31+j62/SpAlNmzYFoDZ+gxkSEoJOp+Obb77BxcWFtm3b0q9fP7766iuOHz9eKwFZVebm5lhYWNR2M0QVSBjoOQMDA5KTkykpKVFfa9SoEdHR0bz44ovqa+vXr8fNzY3u3bvj6+tb5gM3IiJC7WZ65plneP/99+/bLXL27Fn8/Pzo3r07/fv3Z9GiRRQVFanjDx06xKuvvkr37t0ZOXIkGRkZ5dbw008/0aRJE9zc3LCzsyv3wy8pKQlvb28cHBx4/fXXWbp0KW+88YY6/vDhw4waNYqePXvSt29fFi9eTHFxMQCRkZEMGTKE9957j6eeeorPP/+8TDfRCy+8AICnpyfLli1TlxkbG8vzzz+Pg4MDb731ltqNdOHCBezs7PjXv/7FwIEDcXBwYMKECaSnpzNt2jR69OjBiy++yJ49e+5ZS1FREXFxcYwcOZKGDRuWGdeyZUvWrFmDu7s7cCuo1qxZg5ubG926dcPb25tdu3ap0wcGBjJv3jxmzJhBjx49cHFxYcuWLWzbto0BAwbQs2dP3nnnHXV7LVu2jAkTJjBv3jx69uyJi4sLERERZdrwoH1j2bJljB07Fj8/P3r16kVUVFSZbqL8/HzeffddnJyc6NGjB35+fqSmplZoOyUnJ+Pk5ER0dDTPP/88vXr1YsKECWRnZz9w3xBVJ2Gg58aMGUNkZCTPPfccH3zwAbGxseTk5GBjY4O5uTlw6+xh4cKF+Pv7Exsbi5OTE+PHj+fq1av89NNPLFu2jMDAQLZu3cqCBQvYsWMHGzZsuGtdN2/exM/Pj7Zt2xIVFcWiRYv45Zdf1G6Ba9eu4e/vj729PZGRkXh7e9/14XIvsbGx9O/fHyMjI1588UUSEhLuG0ZpaWmMHz8eV1dXoqOjcXNzY/ny5er4s2fP8sYbb9C5c2d++OEH5s6dS0xMDJ9++qk6zX//+1+aNGlCVFQUL7/8cpnlb9q0CYC1a9cyZswY9fXo6GiWLl3KunXruHDhAosXLy4z35IlSwgJCWH16tUkJyfj5eVFz549+fHHH+nSpQszZ868bz35+fl069btnuOfeuop9azlyy+/ZNmyZQQEBBAbG8uAAQN4++23OX78uDr9999/T7t27YiLi6Nv374EBQWxatUqlixZQlhYGDt27CA2Nladfs+ePVy+fJmNGzfyzjvv8OmnnxIVFQVQoX3jl19+wdHRkY0bN+Lq6lqm7eHh4Vy4cIE1a9YQGRmJoaEhH3zwQYW3U15eHt9//z1Lly7ln//8JwcPHuSLL764599JVANF6L2EhATlzTffVJ588knF1tZWeeKJJ5R58+YpJSUliqIoio+PjxIcHKxOr9PplEWLFimpqalKcnKysm3btjLLGz9+vDJjxgxFURRl6dKlypAhQxRFUZQffvhBGThwoKLT6dRpf/vtN8Xe3l7Jy8tT1q1bpzz77LNKUVGROn7WrFlK//7979v2kydPKra2tsrPP/+sKIqiHD9+XLG1tVU2btyoTjN9+nRl8uTJiqIoSmhoqPLyyy+XWcbUqVOVkSNHKoqiKJ988oni4eFRpo2bN29WnnzySaWgoED58ccfFVtbWyUzM1Mdf2eNaWlpiq2trXLixAlFURTl119/VWxtbZUjR46o0//zn/9UBg0aVGb6LVu2qOPHjh2rDB8+XB2+vYy8vLy76v/tt98UW1tbJTU19b5/I0W5tc2cnJyUlStXlnndz89Peeedd9S/k7u7uzru4MGDiq2trfKf//xHfW3kyJHKggUL1Lp79epVpl3z5s1TfH19FUVRKrRvODg4KKWlper4O7fVhAkTlLfeekvJz89XFEVR0tPTld9++01RlPK30+2/2R9//KGOnz9/vjJs2LAH/p1E1TWo7TASD8/NzQ03NzcKCgrYu3cv0dHRrFmzBo1Gw7hx4zh9+jSjR49WpzcwMOD9998HwMbGhsOHDxMWFsbZs2c5efIkZ8+eveuIGeDUqVOkpaXx1FNPqa8pioJOpyM1NZWTJ09iZ2eHsbGxOt7BweG+XSQAMTExNG7cmL59+wJgZ2enXkgeOnToXdOfOHHirqPoHj16qNdHTp48Sffu3TEwMFDH9+rVi+LiYvUbVo0aNUKj0dy3TffSrl079d8WFhZ3XdS1sbFR/92oUSP1aB7A1NQUoEx32m23p/vzzz8fuP4rV65w9epVevToUeb1Xr16kZCQcM923u52atu2rfqaiYlJmXZ06dIFMzMzddjBwUE9O3rmmWfK3TfatGmDoeG9OxjGjx/P+PHj6d27N08//TQDBgzA29sbqNh2Amjfvr36bzMzM7UbSVQ/6SbSY8ePH2fOnDnqcJMmTejfvz9LlizB3d2dX375BQBjY+Myb7o7RUZGMmLECP7880/69etHWFjYfb8KWlJSQo8ePYiOjlb/i4mJYdu2bXTu3Bm4+wLsncHwV4qisHnzZq5fv85TTz3FE088wRNPPMG5c+f4448/OH369F3zNGjQAJ1Od99l/rXf/c423Z7v9odzZdzvA+/OdlVm+ttsbGywtLTk8OHD9xy/YMECVq5cec+64H9hfNu9/t732/YARkZGZYZ1Op3a9orsGw/6W/bo0YMdO3YQHBxMs2bNCAsLY9iwYdy8ebNC2+le9fx1/xLVR8JAj+l0OtatW8fevXvvGmdmZqYedbZv356jR4+WGT948GC2bt3KunXr8PPz46OPPmLo0KHY2dlx7ty5e77pOnXqxLlz57C2tsbGxgYbGxv+/PNPQkNDKS4uxs7OjuPHj5c58vzvf/973/bv27ePixcvMnfu3DIBExERgaGh4T0vJD/++ON31XLnB2mnTp04ePBgmfb//vvvGBsblzlqvp8HfXA+CoaGhnh7e/Ptt99y8+bNMuMuXLjAhg0bMDExwczMDK1Wy4EDB8pM88cff9CxY8cqr//kyZNlttfhw4ext7cHqNS+cS/Lly/n0KFDeHp6snjxYjZs2EBKSgonTpx46O0kqp+EgR574oknGDhwIFOmTGHTpk2cP3+eY8eOsWrVKuLi4njrrbcAGD16NN9//z2xsbGcP3+e0NBQsrOzefrpp7G0tCQ5OZlTp05x8uRJZs6cyalTp+7ZpeHl5YWhoSHTp08nJSWFP/74gxkzZnD9+nXMzc0ZPHgwhoaGzJw5k9OnTxMXF6d2OdxLbGws1tbWvPLKK9ja2qr/OTs74+rqSmxsbJlvSQG89tprnD59Wu26+O6774iPj1fHv/7661y4cIG5c+dy+vRpdu3axcKFC3n55ZfVC+oP0rhxYwCOHTtWY781+Mc//oFOp2PUqFHs2bOHtLQ0tm/fjp+fH3Z2dgwbNgyAcePG8eWXX7J582ZSU1P5/PPP2bNnT5lvUlVWdnY2H3/8MWfOnCEqKoqNGzfy5ptvAlRq37iX9PR05s2bx++//05aWhpRUVGYmZnRvn37h95OovrJNQM9FxoaysqVK/nmm2+YN28ehoaGdO/enZUrV9KzZ0/g1llAVlYW4eHhXLlyhS5duvDVV1/RrFkzgoKCmDVrFq+88grm5ub07t2b8ePHs3Xr1rvW1bhxY1avXs2CBQsYOnQoDRs2pH///syYMQO49R3ziIgIPvroI4YMGUKnTp0YPXo00dHRdy2rqKiIrVu38uabb97VVQEwcuRI/Pz8ynx1EsDKyorPP/+cBQsWsGrVKnr06IGXlxeZmZnq+JUrV7J48WK8vb1p2rQpPj4+TJw4sUJ/z6ZNm+Lr68vMmTMZPnw4AwYMqNB8D8PS0pL169fz+eefM3v2bLKzs9FqtQwcOJC3335b7YoZOXIk169fZ/HixVy5cgVbW1u+/PJLHB0dq7xuOzs7TExM8PHxoUWLFnz44Ye4ubkBVGrfuJf333+f+fPnqz/i69KlCytWrMDCwgILC4uH2k6i+hko0gkn9EhKSgo3btzAwcFBfe3jjz/mxo0bLFiwoBZbpn+WLVvGv//9byIjI2u7KaIOkG4ioVcuXLjA6NGj2blzJxcvXiQhIYHo6Gj1h1lCiKqRbiKhV55//nkmTJjAnDlzyMrKok2bNsyePZt+/frVdtOE0GvSTSSEEEK6iYQQQkgYCCGEQMJACCEEEgZCCCGQMBBCCIGEgRBCCCQMhBBCIGEghBACCQMhhBBIGAghhEDCQAghBBIGQgghkDAQQgiBhIEQQggkDIQQQqCnD7e5erUAna5mHsPQvLkZV67k18i6aoPUp9+kPv1Vk7UZGhrQtGmTB06jl2Gg0yk1Fga311efSX36TerTX3WpNukmEkIIIWEghBBCwkAIIQSVCIP8/Hw8PDy4cOHCXeOOHTuGj48Pbm5uBAUFUVJSAsClS5cYMWIEL730Em+//TYFBQUA5ObmMm7cOAYNGsSIESPIysqqpnKEEEJURYXC4ODBg7z22mukpqbec/z777/P7Nmz2bp1K4qisHHjRgA+/vhjXn/9dRISEujatSuff/45AOHh4Tg6OhIfH8/QoUMJDg6unmqEEEJUSYXCYOPGjXz44Ydotdq7xl28eJEbN27Qo0cPAHx8fEhISKC4uJh9+/bh5uZW5nWAnTt34unpCYCHhwe7d++muLi4WgoSQghReRX6aumDjtwzMzPRaDTqsEajISMjg6tXr2JmZkaDBg3KvP7XeRo0aICZmRk5OTlYWVlVuRAhhBBV99C/M9DpdBgYGKjDiqJgYGCg/v9Ofx2+cx5Dw4pfy27e3Kxqja0ijca8RtdX0/S9vq5du3L06NFKz/fkk09y5MiRR9CimqXv2688+lyfPu2bDx0G1tbWZS4AZ2dno9VqadasGXl5eZSWlmJkZERWVpbazaTVasnOzsba2pqSkhIKCgqwtLSs8DqvXMmvsR9raDTmZGXl1ci6akN9qO/f/0667zit1oLMzNz7jtf32uvD9nsQfajP1rYd165dq9ZlHj169L4Hz5aWlqSknK/U8gwNDco9iH7oMGjdujWmpqb89ttv9OrVi5iYGFxcXDA2NsbR0ZEtW7bg6elJdHQ0Li4uALi6uhIdHc2ECRPYsmULjo6OGBsbP2xThBCixn33Ri9srR6rsfWlZPz5SJZroChKhQ+xn3/+edasWUObNm3w9/cnICCAbt26cfz4cWbOnEl+fj5PPvkkCxYswMTEhIsXLxIYGMiVK1do2bIln376KY899hjXrl0jMDCQtLQ0zM3NCQkJoU2bNhVutJwZVB99qe9RHH09SFWOvmqDvmy/qtKH+rRaixpd36M6M6hUGNQVEgbVR1/qK6+7536qWl9V11fT9GX7VVV9rq8ma6tIGMgvkIUQogZFRm7CxcUJIyMjXFyciIzcVNtNAvT0rqXi7yd+8gvkrRhd6fmqetwVP/mFKs5Z/VxcnDh+/Fil57O378Lu3cmPoEWiqiIjNzF//lzCwz/Dw2MgP/20jalTJwHg4zO0Vtsm3UTlqM+nqaA/9Uk30b3pSzurSl/2z4pycXFi/vzF9O3rota2Z89uPvjg/Uca3DXybSIhakpNXqirzFedhaiolJQTODn1LvOak1NvUlJO1FKL/kfCQOiFqh791rcjS6HfbG3tSE5Oom9fF/W15OQkbG3tarFVt0gYCCFEDZk69T3GjRtNo0aNuXAhjTZt2lJYeJ158xbWdtPk20RCCFGTbl+lvf0L47py1VbODISoAx7mR3VVuZaiLz+qq2/Cw0P46quIe15Aru1vE0kYCFEHXLt2rca/LSVqXl2+gCzdREIIUUNuX0C+U125gCxhIIQQNWTq1PeYOnUSe/bceqDXnj27mTp1ElOnvlfbTZNuIiGEqCm3rwt88MH7+Pp6YWtrxwcfzKr16wUgYSCEEDXKx2coPj5D69xvYKSbSAghhISBEEIICQMhhBBU8JpBXFwcX3zxBSUlJbz55puMGDGizPhdu3YREhICgK2tLXPmzKFJkyb4+PhQWloKwI0bN0hLS2P37t3cvHkTDw8P2rVrB0CLFi1YtWpVddYlhBCiEso9M8jIyCAsLIz169cTHR3Nhg0bOHXqlDo+NzeXwMBAwsLCiIuLw97enrCwMAAiIyOJiYkhJiaG7t27ExAQQIsWLThy5Aienp7qOAkCUd3q6gNEhKiryj0zSExMxNnZWb2lr5ubGwkJCUyadOuBDKmpqbRq1YrOnTsD0L9/f8aOHcvMmTPVZSQlJXH8+HEWLFgAwOHDh0lJScHb25vHHnuMoKAg7Oxq/0cXon6oyw8QuZ+/88N7RN1QbhhkZmai0WjUYa1Wy6FDh9Th9u3bk56ezvHjx7G3tyc+Pp7s7Owyy1i6dCnTpk3DyMgIAFNTU7y8vBg+fDi//PILEydOZMuWLZiYmFRXXeJvLDw8hPDwz+jb1wVjY2P69nUhPPyzOnH/l/sZtGxHjd6OYpDWgsxZlZ5N1GPlhoFOp1PvrgegKEqZYQsLCxYuXMisWbPQ6XS8+uqrGBsbq+NPnjzJ1atX6d+/v/ra5MmT1X+7uroSGhrKmTNnsLe3r1Cjy3tiT3XTaMxrdH01rb7Vl5JyAg+Pgep+qNGY4+ExEF9frzpda1XbVtPz1TR9aWdV1KXayg0Da2tr9u/frw5nZWWh1WrV4dLSUqytrdm06Vaf7KFDh2jbtq06fvv27bi7u5dZ5tq1a/Hw8KBp06bArYBp0KDiv3+Tx15Wn/pYn62tHT/9tO2uO0Pa2trV6Vqr0raH2X51+W9xW33cP2+rydoq8tjLci8g9+nTh6SkJHJycigsLGTbtm24uPzvKT0GBgaMGTOGjIwMFEUhIiKizIf/gQMHcHR0LLPMffv28cMPPwCwd+9edDodHTt2rFRxQtxPXb7/ixB1VbmH41ZWVkybNo1Ro0ZRXFyMr68vDg4O+Pv7ExAQQLdu3ZgzZw5jx46lqKiI3r174+fnp86flpaGlZVVmWUGBQURGBhITEwMpqamhIaGYmgoP3kQ1aMu3/9FiLrKQFHqynN2Kk66iaqP1Fc3aLUWNf48g6o+V7om6cv2qwq96yYSQghR/0kYCCGEkDAQQgghYSCEEAJ5uI0QdUZNPqT+9u1l6qrIyE2Eh4eQknICW1s7pk59T74N9ohJGAhRB1T1mz368q2gytDHe0vVB9JNJISoU+53b6nw8JDablq9JmEghKhTUlJO4OTUu8xrTk69SUk5UUst+nuQMBBC1Cm2tnYkJyeVeS05OQlbW7nN/aMkYSCEqFPk3lK1Qy4gAy4uThw/fqzS89nbd2H37uRH0CIh/r7k3lK1Q+5NVI76+G2NO9Xne79A/ajv73ywUh+23/3UtXsTyZmBEHXcgz7Q6/OHpahZcs1ACCGEhIEQQggJAyGEEFQwDOLi4nB3d2fgwIGsW7furvG7du3C09MTT09P3n33XQoKCoBbj7R0cnLC29sbb29vZsyYAUBubi7jxo1j0KBBjBgxgqysrGosSQghRGWVGwYZGRmEhYWxfv16oqOj2bBhA6dOnVLH5+bmEhgYSFhYGHFxcdjb2xMWFgbAkSNHGDNmDDExMcTExLBgwQIAwsPDcXR0JD4+nqFDhxIcHPyIyhNCCFER5YZBYmIizs7OWFpa0rhxY9zc3EhISFDHp6am0qpVKzp37gxA//792b59OwCHDx9mz549eHp6MmHCBC5fvgzAzp078fT0BMDDw4Pdu2/9uEQ8Gi4uTmi1Fvf8z8DA4L7jXFycarvpQogaUu5XSzMzM9FoNOqwVqvl0KFD6nD79u1JT0/n+PHj2NvbEx8fT3Z2NgDm5uYMGjSIgQMH8t133zFt2jS+//77Msts0KABZmZm5OTkYGVlVd31CR781cT6/jsKIUTFlBsGOp0OAwMDdVhRlDLDFhYWLFy4kFmzZqHT6Xj11VcxNjYGYM6cOep0r732GqGhoeTl3f2daEVRMDSs+LXs8n48Ud00GvMaXV9Nk/r0m9Snv+pSbeWGgbW1Nfv371eHs7Ky0Gq16nBpaSnW1tZs2rQJgEOHDtG2bVt0Oh3Lly9n3LhxGBkZqdMbGRmh1WrJzs7G2tqakpISCgoKKvWwjZr8BTJQ73/UU5/rq+8/ypL69Fdd+wVyuYfjffr0ISkpiZycHAoLC9m2bRsuLi7qeAMDA8aMGUNGRgaKohAREYG7uzuGhob8/PPPbN26FYDo6Gi6d+9O48aNcXV1JTo6GoAtW7bg6Oionk0IIYSoeRW6N1FcXBzLly+nuLgYX19f/P398ff3JyAggG7durFz505CQ0MpKiqid+/eBAUFYWxszMmTJ5k1axZ5eXk0a9aMRYsW0bJlS65du0ZgYCBpaWmYm5sTEhJCmzZtKtzoqpwZ2Nq249q1a5Wa52FYWlqSknK+xtZXVfX9mkF9PrIEqU+f1bUzg7/Njeqq+qFX1Q2mLx+y+tLOqqrPHyYg9emzuhYG8gtkIYQQEgZCCCHkFtb1xsNcE9FqLSo9j75cExFCVIyEQT1x7dq1Gr8mIoSoP6SbSAghhISBEEIICQMhhBBIGAghhEDCQAghBBIGQgghkDAQQgiBhIEQQgjkR2f1RvzkF8hbMbrS81X1Nlnxk1+o4pxCiLpIwqCeGLRsR43+AnmQ1oLMWZWeTQhRR0k3kRBCCAkDIYQQFQyDuLg43N3dGThwIOvWrbtr/K5du/D09MTT05N3332XgoICAE6fPs2IESPw9vZm2LBhHDt2DICLFy/Ss2dPvL298fb2xs/PrxpLEkIIUVnlXjPIyMggLCyMyMhITExMGD58OE5OTnTu3BmA3NxcAgMDWbt2LZ07d+arr74iLCyMmTNnMnPmTMaPH89zzz1HUlIS06dPJzY2liNHjuDp6cmcOXMeeYG3yQVWIYS4v3LDIDExEWdnZywtLQFwc3MjISGBSZMmAZCamkqrVq3UcOjfvz9jx45l5syZDB06lH79+gFgZ2fH5cuXATh8+DApKSl4e3vz2GOPERQUhJ2d3SMp8Da5wCqEEPdXbjdRZmYmGo1GHdZqtWRkZKjD7du3Jz09nePHjwMQHx9PdnY2AD4+PhgZGQGwdOlSBgwYAICpqSleXl5ERUXh5+fHxIkTKSoqqr6qhBBCVEq5ZwY6nQ4DAwN1WFGUMsMWFhYsXLiQWbNmodPpePXVVzE2Ni4z/aJFizh48CBr1qwBYPLkyep4V1dXQkNDOXPmDPb29hVqdHkPdr6fmnwgS9OmTdFozGtsfUCV11fT89U0fWlnVUl9+qsu1VZuGFhbW7N//351OCsrC61Wqw6XlpZibW3Npk2bADh06BBt27YFoKSkhOnTp5ORkcGaNWswN79V+Nq1a/Hw8KBp06bArcBo0KDiP3m4ciUfnU6p8PRAlbqI4FaAVHXeqnQvPYyqrK+q3WBVXV9Ne5j69IHUp79qsjZDQ4NyD6LL7Sbq06cPSUlJ5OTkUFhYyLZt23BxcVHHGxgYMGbMGDIyMlAUhYiICNzd3QFYuHAh+fn5rF69Wg0CgH379vHDDz8AsHfvXnQ6HR07dqxSkUIIIR5euYfjVlZWTJs2jVGjRlFcXIyvry8ODg74+/sTEBBAt27dmDNnDmPHjqWoqIjevXvj5+dHTk4O69ato02bNgwdOlRdXkxMDEFBQQQGBhITE4OpqSmhoaEYGspPHoQQorYYKIpSuf6WOqAq3URV9TDdRDWpqu2s6qmqvvxd6nM3A0h9+qyudRPJvYmEELXKxcWJ48ePVXo+e/su7N6d/Aha9PckYVCP1OS3pW7/7kSIh/WgD3R9OQOtDyQM6ona+LaUEKL+kKu2QgghJAyEEEJIGAghhEDCQAghBBIGQgghkDAQQgiBhIEQQggkDIQQQiBhIIQQAgkDIYQQyO0ogPJvlHW/e/7IjbKEEPWFhAEPvlFWfb6FrhBC3CbdREIIISp2ZhAXF8cXX3xBSUkJb775JiNGjCgzfteuXYSEhABga2vLnDlzaNKkCbm5ubz33nukpaXRrFkzwsPD0Wg0FBUVERQUxJEjR2jYsCEhISF06tSp+qsTgHSDCSHKV24YZGRkEBYWRmRkJCYmJgwfPhwnJyc6d+4MQG5uLoGBgaxdu5bOnTvz1VdfERYWxsyZMwkPD8fR0ZEVK1YQHR1NcHAw4eHhrF27lkaNGhEfH8++ffuYMWMGGzdufOTF/l1JN5gQojzldhMlJibi7OyMpaUljRs3xs3NjYSEBHV8amoqrVq1UsOhf//+bN++HYCdO3fi6ekJgIeHB7t376a4uJidO3fi5eUFwNNPP01OTg6XLl2q9uKEEEJUTLlhkJmZiUajUYe1Wi0ZGRnqcPv27UlPT+f48eMAxMfHk52dfde8DRo0wMzMjJycnLuWqdFoSE9Pr56KhBBCVFq53bMdZb8AACAASURBVEQ6nQ4DAwN1WFGUMsMWFhYsXLiQWbNmodPpePXVVzE2Nr7nshRFwdDQ8K5l3H69osp7sHN102jMa3R9NU3q029Sn/6qS7WVGwbW1tbs379fHc7KykKr1arDpaWlWFtbs2nTJgAOHTpE27ZtgVtnEdnZ2VhbW1NSUkJBQQGWlpZYWVmRmZlJu3btAMjOzi6zzPJcuZKPTqdUePqHUd/71KU+/Vbf6wPqbX01ue0MDQ3KPYgu93C8T58+JCUlkZOTQ2FhIdu2bcPFxUUdb2BgwJgxY8jIyEBRFCIiInB3dwfA1dWV6OhoALZs2YKjoyPGxsa4uroSExMDwP79+zE1NaVVq1ZVLlQIIcTDKTcMrKysmDZtGqNGjeLll1/Gw8MDBwcH/P39OXz4MIaGhsyZM4exY8fy0ksvYWFhgZ+fHwBTpkzhwIEDDB48mPXr1zN79mwA3njjDYqKihg8eDDBwcEsWrTo0VYphBDigQwURamZ/pZqJN1E1Ufq02/1vT6t1oLMzNzabsYjoXfdREIIIeo/CQMhhBASBkIIISQMhBBCILewFkLUAFvbdly7dq1K897vRooPYmlpSUrK+Sqt7+9KwkAI8chdu3atSt8Kquo3bqoSIH930k0khBBCwkAIIYSEgRBCCCQMhBBCIGEghBACCQMhhBBIGAghhEDCQAghBBIGQgghkDAQQghBBW9HERcXxxdffEFJSQlvvvkmI0aMKDP+6NGjzJ49m+LiYlq2bMnixYuxsLDAx8eH0tJSAG7cuEFaWhq7d+/m5s2beHh4qM9AbtGiBatWrarm0oQQQlRUuWGQkZFBWFgYkZGRmJiYMHz4cJycnOjcubM6TXBwMAEBAbi6uvLJJ5+watUqpk2bRmRkpDrN//3f/zFkyBBatGjB1q1b8fT0ZM6cOY+mKiGEEJVSbjdRYmIizs7OWFpa0rhxY9zc3EhISCgzjU6no6CgAIDCwkIaNmxYZnxSUhLHjx/H398fgMOHD5OSkoK3tzejRo3ixIkT1VWPEEKIKig3DDIzM9FoNOqwVqslIyOjzDSBgYHMnDmTvn37kpiYyPDhw8uMX7p0KdOmTcPIyAgAU1NTvLy8iIqKws/Pj4kTJ1JUVFQd9QghhKiCcruJdDodBgYG6rCiKGWGb9y4QVBQEBERETg4OPD1118zffp0VqxYAcDJkye5evUq/fv3V+eZPHmy+m9XV1dCQ0M5c+YM9vb2FWp0eQ92rm4ajXmNrq+mSX36TR/qi5/8AnkrRld6vqo+Lj5+8gt68XepS20sNwysra3Zv3+/OpyVlYVWq1WHU1JSMDU1xcHBAYBhw4axZMkSdfz27dtxd3cvs8y1a9fi4eFB06ZNgVsB06BBxR+tcOVKPjqdUuHpH0ZV76euL6Q+/aYv9Q1atqNGn2cwSGtB5qy6/XepyW1naGhQ7kF0ud1Effr0ISkpiZycHAoLC9m2bRsuLi7qeBsbG9LT0zlz5gwAO3bsoFu3bur4AwcO4OjoWGaZ+/bt44cffgBg79696HQ6OnbsWPHKhBBCVKtyD8etrKyYNm0ao0aNori4GF9fXxwcHPD39ycgIIBu3bqxYMECpk6diqIoNG/enPnz56vzp6WlYWVlVWaZQUFBBAYGEhMTg6mpKaGhoRgayk8ehBCithgoilIz/S3VSLqJqo/Up9/0pT6t1qLGH3tZlfXVJL3rJhJCCFH/SRgIIYSQMBBCCCFhIIQQAgkDIYQQSBgIIYRAwkAIIQQSBkIIIZAwEEIIgYSBEEIIJAyEEEIgYSCEEAIJAyGEEEgYCCGEQMJACCEEEgZCCCGoYBjExcXh7u7OwIEDWbdu3V3jjx49yiuvvIKXlxfjx48nN/fWQyX27t2Lk5MT3t7eeHt7M2PGDAByc3MZN24cgwYNYsSIEWRlZVVjSUIIISqr3DDIyMggLCyM9evXEx0dzYYNGzh16lSZaYKDgwkICCA2NpYOHTqwatUqAI4cOcKYMWOIiYkhJiaGBQsWABAeHo6joyPx8fEMHTqU4ODgR1CaEEKIiir3GciJiYk4OztjaWkJgJubGwkJCUyaNEmdRqfTUVBQAEBhYSGPPfYYAIcPHyY7O5uffvqJ1q1b8+GHH9KyZUt27typnmF4eHgwZ84ciouLMTY2rvYChRB1g1ZrUWPruv15JSqu3DDIzMxEo9Gow1qtlkOHDpWZJjAwkDFjxjB//nwaNWrExo0bATA3N2fQoEEMHDiQ7777jmnTpvH999+XWWaDBg0wMzMjJycHKyur6qxNCFFHVPV5xPrwLOP6otww0Ol0GBgYqMOKopQZvnHjBkFBQURERODg4MDXX3/N9OnTWbFiBXPmzFGne+211wgNDSUv7+4HQCuKgqFhxa9ll/dg5+qm0ZjX6PpqmtSn36Q+/VWXais3DKytrdm/f786nJWVhVarVYdTUlIwNTXFwcEBgGHDhrFkyRJ0Oh3Lly9n3LhxGBkZqdMbGRmh1WrJzs7G2tqakpISCgoKKnVad+VKPjqdUuHpH4ZGY05W1t0BVl9IffqtvtcH1Nv6anLbGRoalHsQXe7heJ8+fUhKSiInJ4fCwkK2bduGi4uLOt7Gxob09HTOnDkDwI4dO+jWrRuGhob8/PPPbN26FYDo6Gi6d+9O48aNcXV1JTo6GoAtW7bg6Ogo1wuEEKIWGSiKUu4hdlxcHMuXL6e4uBhfX1/8/f3x9/cnICCAbt26sWvXLkJDQ1EUhebNmzN37lzatm3LyZMnmTVrFnl5eTRr1oxFixbRsmVLrl27RmBgIGlpaZibmxMSEkKbNm0q3Gg5M6g+Up9+q+/11edrBnXtzKBCYVDXSBhUH6lPv9X3+iQMqke1dBMJIYSo/yQMhBBCSBgIIYSQMBBCCIGEgRBCCCQMhBBCIGEghBACCQMhhBBIGAghhEDCQAghBBIGQgghkDAQQgiBhIEQQggkDIQQQiBhIIQQAgkDIYQQVDAM4uLicHd3Z+DAgaxbt+6u8UePHuWVV17By8uL8ePHk5t762EUp0+fZsSIEXh7ezNs2DCOHTsGwMWLF+nZsyfe3t54e3vj5+dXjSUJIYSorHLDICMjg7CwMNavX090dDQbNmzg1KlTZaYJDg4mICCA2NhYOnTowKpVqwCYOXMm/v7+xMTEMHXqVKZPnw7AkSNH8PT0JCYmhpiYGHV6IYQQtaPcMEhMTMTZ2RlLS0saN26Mm5sbCQkJZabR6XQUFBQAUFhYSMOGDQEYOnQo/fr1A8DOzo7Lly8DcPjwYVJSUvD29mbUqFGcOHGiWosSQghROeWGQWZmJhqNRh3WarVkZGSUmSYwMJCZM2fSt29fEhMTGT58OAA+Pj4YGRkBsHTpUgYMGACAqakpXl5eREVF4efnx8SJEykqKqq2ooQQQlROg/Im0Ol0GBgYqMOKopQZvnHjBkFBQURERODg4MDXX3/N9OnTWbFihTr9okWLOHjwIGvWrAFg8uTJ6vyurq6EhoZy5swZ7O3tK9To8h7sXN00GvMaXV9Nk/r0m9Snv+pSbeWGgbW1Nfv371eHs7Ky0Gq16nBKSgqmpqY4ODgAMGzYMJYsWQJASUkJ06dPJyMjgzVr1mBufqvwtWvX4uHhQdOmTYFbgdGgQblNUV25ko9Op1R4+oeh0ZiTlZVXI+uqDVKffqvv9QH1tr6a3HaGhgblHkSX203Up08fkpKSyMnJobCwkG3btuHi4qKOt7GxIT09nTNnzgCwY8cOunXrBsDChQvJz89n9erVahAA7Nu3jx9++AGAvXv3otPp6NixY+UrFEIIUS0MFEUp9xA7Li6O5cuXU1xcjK+vL/7+/vj7+xMQEEC3bt3YtWsXoaGhKIpC8+bNmTt3Lk2aNKFv3760adOGRo0aqcuKiYkhIyODwMBAsrKyMDU1JTg4uMJdRCBnBtVJ6tNv9b0+rdaCzMzc2m7GI1HXzgwqFAZ1jYRB9ZH69Ft9r0/CoHpUSzeREEKI+k/CQAghhISBEEIICQMhhBBIGAghhEDCQAghBBIGQgghkDAQQgiBhIEQQggkDIQQQiBhIIQQAgkDIYQQSBgIIYRAwkAIIQQSBkLopcjITbi4OGFkZISLixORkZtqu0lV5uLihFZrcc//gPuOc3FxquWW1y8Vf9akEKJOiIzcxPz5cwkP/wwPj4H89NM2pk6dBICPz9Babl3l7d6dfN9x9f15DXVJhc4M4uLicHd3Z+DAgaxbt+6u8UePHuWVV17By8uL8ePHk5t762EUubm5jBs3jkGDBjFixAiysrIAKCoq4v3332fQoEEMGTKE06dPV2NJQtRv4eEhhId/Rt++LhgbG9O3rwvh4Z8RHh5S200TeqzcMMjIyCAsLIz169cTHR3Nhg0bOHXqVJlpgoODCQgIIDY2lg4dOrBq1SoAwsPDcXR0JD4+nqFDhxIcHAzA2rVradSoEfHx8XzwwQfMmDHjEZQmRP2UknICJ6feZV5zcupNSsqJWmqRqA/KDYPExEScnZ2xtLSkcePGuLm5kZCQUGYanU5HQUEBAIWFhTRs2BCAnTt34unpCYCHhwe7d++muLiYnTt34uXlBcDTTz9NTk4Oly5dqtbChKivbG3tSE5OKvNacnIStrZ2tdQiUR+UGwaZmZloNBp1WKvVkpGRUWaawMBAZs6cSd++fUlMTGT48OF3zdugQQPMzMzIycm5a5kajYb09PRqKUiI+m7q1PeYOnUSe/bcOrjas2c3U6dOYurU92q7aUKPlXsBWafTYWBgoA4rilJm+MaNGwQFBREREYGDgwNff/0106dPZ8WKFXctS1EUDA0N71rG7dcrqrwHO1c3jca8RtdX06Q+/TJ+/BgsLBoxa9Z0fH296NKlC598Mp/XXnuttpv2SNS37XenulRbuWFgbW3N/v371eGsrCy0Wq06nJKSgqmpKQ4ODgAMGzaMJUuWALfOIrKzs7G2tqakpISCggIsLS2xsrIiMzOTdu3aAZCdnV1mmeW5ciUfnU6p8PQPo75/m0Hq008DBngwYIBHmfrqY531dftBzdZmaGhQ7kF0uYfjffr0ISkpiZycHAoLC9m2bRsuLi7qeBsbG9LT0zlz5gwAO3bsoFu3bgC4uroSHR0NwJYtW3B0dMTY2BhXV1diYmIA2L9/P6amprRq1apqVQohhHho5Z4ZWFlZMW3aNEaNGkVxcTG+vr44ODjg7+9PQEAA3bp1Y8GCBUydOhVFUWjevDnz588HYMqUKQQGBjJ48GDMzc0JCbn11bc33niD2bNnM3jwYExMTFi0aNGjrVIIIcQDGSiKUjP9LdVIuomqj9Sn36Q+/aV33URCCCHqP728HYWhoUH5E+nx+mqa1KffpD79VVO1VWQ9etlNJIQQonpJN5EQQggJAyGEEBIGQgghkDAQQgiBhIEQQggkDIQQQiBhIIQQAgkDIYQQSBgIIUS9V5HfFuvl7Sj0TU5ODidPnuTgwYPk5uby2muv0bp169puVrW582FFOp2OhIQEfvvtN2bNmlXLLXv0cnJySElJwdnZubab8shcvXqVU6dOceTIEa5evcrAgQPp2rXrXQ+pqmvy8/M5fPgwKSkpHDlyBJ1Ox8SJE+nYsWNtN+2RuXObFBYWsnv3bgwNDXnxxRfR6XQPfIiYnBk8Ijqdjri4OJ599lnefvttVq9eTWZmJtevX2fy5Mm13byHkpOTQ1JSEuvXr+fYsWMYGBioRx43b95kw4YN7Nu3j82bN9dySx/OX4+mUlNTiYiIoKSkRH3t5MmTTJ8+ndLS0ppuXrW7efMmKSkp/Pnnn+Tl5TFnzhxefvllXn75Zd544w0KCwuxs7Ojbdu2AHUyCM6fP8977916/GdISAj/93//R0ZGBn369OG5555j9OjRtdvAanLnvllUVMR//vMfNm3ahIGBgbovXr58mSlTphAREVHhhYoqKC0tVbZv364oiqJERkYqERERd00TFRWlTJ48ucxrhw8fVnr27KlcunSpRtpZXUpLS5XY2FilT58+yrBhw5Tx48crs2bNUkaPHq1ERkaq0xUVFSmurq7Kli1bFE9PTyU3N7cWW115V69eVRRFUXQ6naIoinLmzBnFw8NDKSkpUXbu3KnY2dkpM2bMUKf/888/lYEDByrZ2dm10t7Kul2XoihKZmamMnfuXOWdd95Rhg0bpri7uyseHh7KjBkzlMuXLyunTp1Sbt68qZw9e1bp3bt3Lbb6f+583+3cuVPx9vZWMjMz1fHp6enKM888o9y8eVNZvXq18tVXX6njioqKFHd3dyU9Pb3G210diouLFUX53zY8d+6cMmTIECU1NVXx8vJS7OzslD/++EOdPjU1VXnppZcUDw+PCi1fzgwqSafTqc9snjhxIleuXCE3N5e0tDTy8/OB/6W2VqulcePGHDt2jEOHDhEZGcmmTZt47733aNGiRW2WcRedTseOHTsAiIqK4ptvvikz3tDQkNLSUnr16sX333/Pl19+yZw5cxgyZAifffYZaWlpAJw+fRpra2ucnJywsLBg3bp1FBYW1ng9FXX9+nU++ugjjhw5wrFjx3B2dubHH39Uj3rbtm1LQUEBmZmZ2NnZYWVlxfHjxwkLC+PGjRtYWFhgYGDAhQsXarmSspQ7jhwVReHXX39l7ty5GBgYoNPpgFvdKN9++y3Dhg1j0aJFbN68mR9//JHNmzdz/vx5OnXqhImJCe3atePq1asUFRXVVjl3ve8yMzMxMjLi1KlTTJgwQd3/mjVrhrW1NWlpabRu3ZqjR4/y2WefMX/+fFxcXOjZs2elnrdeU+7cXjdu3ODo0aNs376doqIiCgsLeffdd/n666+B/52RFRQUALeeNtm9e3f69evH6tWr1e2UmJjI66+/TkFBARkZGeW2oe79VWqJoiikpqZSXFwMcN/TfkNDQ3VjtGzZkrNnz9K5c2eKi4u5cuUKgPpma9GiBRcvXmT8+PF89tlnLF26lAMHDuDk5ISxsXGFLuo8apUJNysrK0xNTUlNTQVunZ56eXnRpEkTNUh2795Nz549adasGQEBASQnJ7Nr1y51XbXhQUG3Zs0asrOz0Wg0WFlZqU/q27FjB6WlpTRo0AAzMzPOnj1LixYtaN68OVOmTGHv3r2sXr0agPbt23PixIkar0v5S1fB0aNHiYqKIiEhoUwXjqIofPfdd/zrX/8iNjZW/TDs0KEDDRo04JlnnqFdu3YoisLJkyfp2rUrVlZWAJSUlGBoaIiFhQUnT568a73VUUNl33etWrXi9OnTtGzZEh8fH4qLi/n88885duwYxsbGaDQajh49SufOnbl06RJHjx7liSeeICwsjNLSUoKCgtT3am2/B3U6Hdu3b1drW716NcOGDWPhwoXEx8czduxYCgsL8fX15ccffyQ7O1ud99dff+W5554DwNzcnD59+nDmzBn++OMPAA4cOICNjQ0dOnTg2LFjwIPr/duGgaIolJaWqjvfhQsXmD59OgcPHgTAyMgIgNzcXK5du6bO99133/HCCy/wwQcfkJ+fz5EjR2jTpg3FxcVkZmaWWYeFhQW2trZMmjSJFStWsG7dOrp160ZISAhFRUVl+tofRX3VFW6322hpaUmDBg24dOlSmWV07dqVo0ePApCZmUlcXByzZs0iODiYpKQkoqOjuXnzZo0fkZUXdCdOnODnn38mICAAKysrzM3N6dy5M6+++irffvut+pzudu3akZKSQoMGDbC0tKSoqIjPPvuM3bt3891339GuXTs1IGuqrjs/QLKzswkICGDRokX8/vvv/Pzzz7zzzjvqEaKhoSEnTpzgH//4B99++22ZsxitVsuoUaN444036NevH2+//TYjR47ExsamzDo7dOjAgQMH1PVX1f3ed4cOHQLu/75bv349L7zwAkFBQeTn53P06FG0Wi0GBgaMGzeOli1bEhoaSkZGBs888wyHDh2iWbNm2NnZ8dZbb/Hyyy/j7OzMa6+9hk6n47///S9Qe9c97tw3J02axOXLl0lLS2Pv3r2sXLmSNWvWEBoair29PZcvX6Z37960bt2auLg49SAtOTlZDe2mTZuiKAp9+/Zl7969ZGVl0aZNGwoLC3n88cc5cuQI8DcPg7/ufLcZGBhgZGSk7nzm5uZ07NhRPd3cvHkzPj4+vPnmmyxfvpxDhw6RmZlJcnIyn376KfPnz6d///7s3buXli1bYmRkRFZWFvC/D97GjRvTrFkzLl68CEDr1q15++23uXr1KkFBQWo7HkWdlQ2322+ye4Xb7R3osccew8zMTP0b3da5c2dyc3OBW0eobdu25amnniIkJISoqChyc3NZvHgxUD1nBw8TdJ06daKoqIj09HTmzp2Lr68vtra26HQ6jI2NsbCw4Mknn2Ts2LFs3LiRXbt24ezsrB7529ra8p///IfmzZsTEBDA0aNHiY+PV7f9o/TXD5Db+9WkSZNwdXXlm2++Ye7cucyYMYOOHTuqYX7hwgWaNm3K008/jUaj4bvvvlP3AVtbWywtLVm5ciXh4eE8/vjjnDp1iuvXr6vrBOjRo4caBhU5gKns++78+fPA/d93e/fu5dNPPyU4OJj+/fuzb98+zM3NMTU15dq1a0yaNAlbW1sCAwNp0qQJZ8+exdLSEhMTkzIHL82bN+fcuXN07ty5ytuhvLqrcqaTlpbGlStXKCgoICUlhUuXLnHy5EmmTJnCk08+CYC/vz8pKSn88ssvAGg0GoyNjYFbByxpaWm4urpiZmZGSEgIGo2Gzp0706ZNG06fPl1u2+tNGJw7d46NGzcye/Zsli5dqu4Af9354NYHVnJyMkuWLCEwMJDk5GQsLS1p3rw5aWlp5OXlsW/fPlauXElUVBRGRkZ88sknXL16lWPHjtG9e3cAxo0bx7FjxzAxMaFhw4bqB4KJiQlw68ygRYsW5OTkqOtu3bo1n3zyCdu2bSMyMrLSdT6qcAsODua55557YLiZmZlhaWnJ5cuX1Tp1Oh379u3DxcVF/bt9+OGHDBkyhMcff5wuXbrw9ttvq9ujKmcH1Rl0rVu3xsjIiA8//JDU1FTy8vLKfOXOzs6OX375hWeffZZhw4axceNG9u/fz9WrVwGwt7fn3LlzAPTp0wc/Pz8URSEhIYGbN29Wurbb9VXlAyQ1NZXz589jYmKi7pM3btygRYsW+Pv7q9elkpOT6dSpE+3bt2fSpEmcOnWKhIQE4FZ/s4mJCaampjg6OjJlyhSSkpL45JNP1HUCODg4kJSUdFebKvu+27t3L0uWLGHGjBn8+uuv93zfffXVV0RFRWFsbHzP952/v78azi1atCA1NRVDQ0OmTJmCjY0Ny5cvV7eXqakpW7duZcmSJUyZMoV//OMf+Pr60rJlyyptq7+qrh6GP/74gx49euDk5MTkyZN54403WLx4Me+88476bSBnZ2cef/xxduzYQWpqKjqdTv2bdOjQgfz8fJo0aUK/fv3417/+RWJiIp06dcLa2pr9+/eXW0u9+J3BhQsX+PzzzzE2NubJJ5/k/PnzzJkzh1mzZmFubs4PP/zAxYsXsbe3Z+jQoeoH/aBBgzAzM2PJkiWEh4djY2PDb7/9xvXr1/n+++85deoUBQUFPPbYYzz//PM0bdq0TFdQs2bN1J2uefPm5Ofnc/HiRU6cOMG+fft45plnaNiwIYWFheTl5WFubo5Op6N9+/b8+9//plmzZvet6dy5cyQnJ3PkyBFatGiBr68vrVq1Ut9kdyoqKuLAgQMkJSWRnp6Ot7c3zs7O93yTNW/enE8//ZQFCxbw0UcflXmTjR8/Hn9//weGm0aj4dChQ6SlpXHs2DF+//13mjRpwpAhQ9DpdJibm9OmTRvgf9957tevH/369St3OyqKoh6J3lnjX2u+M+gcHR3ZvHkzq1atQlEUnJ2dGTRoENbW1mrQde/eneLiYn799VdGjx7NqVOn0Ol0LFy4kC+//JLz588zb9484NaR8u0jLy8vLzQaDRMmTKBp06Zcv34dGxsbDh8+rLalQ4cOrF27lo0bN1b4rOevdd7+AHn//fdxdHQs8wGi0+mwtLQEbn2ArFy5EmdnZ/Ly8jh79izGxsY0atRI/dBu2LAhpaWlmJqaquvLy8vj559/xszMjAMHDnDw4EH+/PNPhg8fzrPPPktwcDBwK4QcHBx49913mTRpEk2bNmXatGkAODo60qtXLwAaNLj1sfGw77ulS5fSsWPHKr3vbn+wtmzZkjNnzlBSUoKpqSkffvghV65cYceOHeTn59O1a1cyMzNp2LAhQ4cOpVOnTlUKgke5b+p0OhITExk/fjwTJ05kzJgxXL58mRMnTnD48GF27NhBmzZtGDBgAAMGDODcuXMkJCRw5swZOnXqpP4d8vLyOHfuHJ6ennh4eKjXFxwdHfnoo4+ABx+M1YswSEpK4tKlS6xduxa4dWr722+/oSgKq1atIj09nS5duvDrr78C0KRJE15//XV69epFfHw8Z86cIT09ndatW/PTTz/RuHFjGjVqRGhoqNont2XLFrRaLV27diUiIgIfHx8SExO5ceMGFy9eRKPRsGDBAr7++mtsbGywsbGhefPmPPPMM3h5ealHdLc3xoOCoCbeZC+88EKVwq1Ro0b88ssvpKSkqOt49dVXady4MQAbN25Ul3e/LrDaCLpx48bh7+8PwOOPP461tTXPPvssbdu2ZfLkyXz00Ud89NFHdOrUST26NTIyom/fvoSEhDB9+nSuXLmCnZ0ds2bNorS0VG1rhw4dmD59+l11PsoPkNtH2f379+fmzZtluulMTEzIyclh1apVBAQEcPXqVbRaLdbW1kybNo1mzZoxe/ZsFixYwMSJE9Uzndv7Zq9evfjkk0/KdAdptVqWLFlSpr7aft+lpaWp++mVK1ewsrLCwMCAjz/+mPfffx8zMzMGDRrEoEGD7rkf3ktt7Jv+/v6MoVkHogAABXlJREFUGTMGgLi4OBwdHenUqROdOnXC3d2dkSNHqgdk7dq1Y8iQIfzjH//g2WefVdtiZmbGSy+9hJ2dHQBBQUFqaFtaWvLCCy+UW3u9CAOtVktmZiarV69GURSsrKzo06cPRkZGrFu3juTkZIyMjMjOzsbCwgITExO2bt3KmjVraN++PR07diQzM5M2bdqoR7MdOnRgx44d+Pr6kpiYyFdffUWvXr1YtGgRy5YtY+jQobz44ouMHDkSQ0NDBg8ejJubG2ZmZg9dT22/yR4Ubs7OzgwePLjKF4PrQtBptVpu3rxJSUkJ7f6/nfsJafqP4zj+nH+QRIhpaqLOpbVDisqgiYJ4EGMXLULssLJNQg8iJDhQxA4dPOhFMZweFEkkCALxUEPI/INESeHC0kEwSBAVEobCgmTrEH7Rn/b7pT9t8vH9OO009uH7fr9f3w+f774GA48fP8bhcOB0OmlqasJkMvHt2zcSEhIAsFqtWK1W7btu3LixZ03h2sXV1taSmprKxYsXmZ6exmKxaENjcnKSt2/fEhMTw8zMDM3NzXuGx4MHD+jp6SEuLo5Hjx6xvb2tDQ+AkpKS/7yW4e67qKgobceyeycUHx//rzdbvxOu2tTr9fj9fgAWFxdxu900NjayurqKx+PBYDBgNpuBX7MgPz+f+vp6CgsL9/z+mzdvap93X8s/pUQYlJSU4PP5ePnyJYmJifz48YOHDx/icDi4dOkSfr+f+Ph4Lly4QCAQ4Pnz57jdbhoaGsjNzeXJkyd4vV5MJhM6nY6lpSW6urpwuVw8ffqU6OhonE6nNkjb2to4d+7cia0n3E123OG2W7iDbmVlhfPnz+P1etnc3ESv15Oeno7L5SImJobk5GQGBgb+eD3hDLeds6i6ujpsNhsZGRmkpKTg9Xp59+4dLS0tBINBgsGg9gqGnfORgoICCgoKAKiqqjrStVSt78Jdmz6fD6fTyeDgIM3Nzej1ejIzM7Hb7VofRkREEAqFsNlsx75+JcIAwG63Y7fb2djYwO/38/nzZyYmJlheXubTp08UFxfj8XiYnJxkZWWFjIwMPnz4gNvtJhAIsL6+jt/v5/r164RCIQwGA62trcTGxu67Cz7JggT1mmy3cAddZGQkFRUVREVFaXeToVCIK1euHGk94R4gX7584fLlywwPDzM8PMyrV68wGo3cuXMHs9mMTqdjdHRUW+dxP96rUt+dhtqMiIigpqaG+/fv//Z3ntTjsLpQuP91cQzW1tYYGxvDbDZrh1xzc3OMj4+zsbFBZGQkSUlJeDwerFYrNpuNkZER5ufnsVgs5OXlYTQatS32afLPJpudnaWzs3Nfk+0c3K6vr/P69WssFguVlZV8/PgRk8nEtWvX2NraOrDJ/rahoaE9QTc3N4fD4WB6epr+/n5tix8IBHjx4sWeoLt16xalpaWUl5fT1tZGQ0MDSUlJuFwuFhYWiI6OpqmpiaKiIu07TnKITE1N0d7ezu3bt/cNkNLS0t8OkMXFRYxGI8+ePaOmpoa0tDTa29vp7e2lurqayspKbYB0d3fT19dHMBikp6eH9+/fU1ZWxvfv37l37572rqC/TcW+U6k2D0uJMNje3qajo4OvX7+Sk5ODz+djeXmZu3fvUl5ezps3b7RHt/Lz87XDztNKxSY7iCpBd1YHiGp9t5sqtXkYSoQB/DqYm5iYYH5+nqtXr5KdnU1WVtapf83uQVRuMpWD7iwOEJX6TuXa/BPKhIFqVGqy3VQLurM+QFSiWm0eloSB+OtUCrqzPkBUo1JtHpaEgRD/01keIEIdEgZCCCHUeVGdEEKIo5MwEEIIIWEghBBCwkAIIQQSBkIIIZAwEEIIgYSBEEIIJAyEEEIAPwHV8rtFuJzsPwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compare Algorithms\n",
    "fig = plt.figure()\n",
    "fig.suptitle('Scaled Algorithm Comparison') \n",
    "ax = fig.add_subplot(111) \n",
    "plt.boxplot(results) \n",
    "ax.set_xticklabels(names, rotation = '10')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "# improve single model results with parameter tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.993563 using {'n_neighbors': 5}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled KNN\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "neighbors = [1,3,5,7,9,11]\n",
    "param_grid = dict(n_neighbors=neighbors)\n",
    "model = KNeighborsClassifier()\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.991197 (0.006029) with: {'n_neighbors': 1}\n",
      "0.992745 (0.006280) with: {'n_neighbors': 3}\n",
      "0.993563 (0.006797) with: {'n_neighbors': 5}\n",
      "0.992954 (0.005756) with: {'n_neighbors': 7}\n",
      "0.990890 (0.005186) with: {'n_neighbors': 9}\n",
      "0.990041 (0.005529) with: {'n_neighbors': 11}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_['mean_test_score'] \n",
    "stds = grid_result.cv_results_['std_test_score'] \n",
    "params = grid_result.cv_results_['params']\n",
    "\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "                     metric_params=None, n_jobs=None, n_neighbors=5, p=2,\n",
       "                     weights='uniform')"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = KNeighborsClassifier(n_neighbors = 5)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.8998831775700935\n",
      "\n",
      "Accuracy\n",
      "0.9273504273504274\n",
      "\n",
      "Confusion Matrix\n",
      "[[205   9]\n",
      " [  8  12]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.96      0.96      0.96       214\n",
      "         1.0       0.57      0.60      0.59        20\n",
      "\n",
      "    accuracy                           0.93       234\n",
      "   macro avg       0.77      0.78      0.77       234\n",
      "weighted avg       0.93      0.93      0.93       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd1xT1/sH8E8SSCAMWQFUhhsVGa7iwD1oFW1VqtZZrVpbrf3an9ZdtNbZWq31W3e1y6ptVfRrRdx1VdzWVUREAdk7gez7+wMNXFlhZD/v16uvknszHk7pfXLOuc85HIZhGBBCCCFa4ho6AEIIIaaFEgchhJAaocRBCCGkRihxEEIIqRFKHIQQQmqEEgchhJAasTJ0AITU1fjx4xEbG8s6xuPx4OjoiPbt22POnDlo3rw563xeXh527NiBEydO4Pnz52jQoAHatWuHiRMnomvXrhV+zsmTJ/Hrr7/i/v37kEql8PX1xdtvv42RI0fC2tpaZ78fIcaGQ3UcxNSNHz8eSqUS8+bN0xyTy+V4+PAh/vvf/8LW1hbHjx+HQCAAACQmJmLSpElQKpWYNGkS/P39kZeXh0OHDuH06dOYOXMmPvroI9ZnLFu2DHv37sVbb72Ffv36QSgUIjY2Ft9//z169eqFDRs2gMfj6fX3JsRgGEJM3Lhx45hp06ZVeG7fvn1Mq1atmLNnzzIMwzBKpZIJDw9nBgwYwGRnZ5d7/oYNG5hWrVoxp06d0hw7ePAg06pVK2bv3r3lnn/06FGmVatWzMGDB+vptyHE+NEcBzFr9vb2rMdnzpxBXFwc5syZAxcXl3LPnzlzJnx8fLBlyxbNsZ07d8LPzw+jRo0q9/xBgwZh8uTJcHZ2rjKOffv2YfDgwQgMDMTrr7+O/fv3a8717dsXn3/+Oev5K1asQN++fTWP/fz8sGXLFgwePBghISGIjo6Gn58fbt68yXrdzz//jKCgIEgkEgDA3bt3MXHiRAQFBaFLly5Yvnw5iouLq4yVkOrQHAcxCwzDQKlUah7LZDLcvXsX69evR6NGjdCpUycAwMWLF8HlchEaGlrh+/B4PPTr1w+7du1CTk4OlEol4uLiMG3atEo/u+wQWUV27dqFNWvW4N1330XPnj0RGxuLJUuWQCgUIjw8XOvfcdOmTVi4cCGcnZ3RqVMneHp6Ijo6Gu3bt9c859ixY+jTpw/s7OwQHx+PcePGITg4GBs2bEB2djbWrVuH5ORkbN26VevPJeRVlDiIWTh37hz8/f1Zx2xsbNC1a1csWLAAdnZ2AICUlBQ4OztDKBRW+l5eXl4AgNTUVKhUKgBAo0aNahWXWq3Gli1bMHz4cMyfPx8A0K1bNyQlJeH69es1Shzdu3fHmDFjNI8HDRqEY8eOYf78+eBwOMjIyMCNGzfw7bffAgC+++47uLq6Ytu2beDz+QCAJk2aYOzYsbh69So6d+5cq9+JEEocxCx07NgRCxYsAAA8evQIa9asQdeuXbF27VrNRRMo6ZlUN4ld9vzLn9Vqda3ievLkCfLy8ljDTgCwbt26Gr/Xq3eGhYeH4/vvv8ft27cRHByM6Oho2Nvbo2fPngCAK1euoF+/fuByuZreWHBwMOzt7XH58mVKHKTWKHEQs+Dg4ICAgAAAQEBAABo2bIhJkyaBz+dj7dq1muc1btwYly9fhkwm09xl9aqUlBQAQMOGDTXHUlNTK/3sjIwMuLm5gcstP2WYl5cHABXOp9SUq6sr67G/vz+aNm2K6OhoBAcH49ixYxgwYIAmUebl5WHfvn3Yt29fuffKzMysczzEctHkODFLXbt2RUREBKKionD69GnN8T59+kChUODMmTMVvo5hGJw+fRoBAQFwcXGBi4sL2rZti/Pnz1f6WZMmTcKkSZMqPOfg4AAAyMnJYR1/8uQJbty4oXn8ao+mqKio6l/whfDwcMTExCA9PR03b95kDX3Z29tjxIgR+P3338v98/7772v1/oRUhBIHMVuffPIJHBwcsHr1asjlcgBAaGgoAgICsHbt2gq/dW/duhWPHz9mTYZPnDgRDx8+xG+//Vbu+VFRUYiPj8eQIUMqjKFZs2ZwcnLC2bNnWce/+eYbTU/I3t4eGRkZmnNqtbrc3VKVCQ8PR0pKCjZv3gw3NzeEhIRoznXs2BEJCQlo164dAgICND2xdevW4dGjR1q9PyEV4S1dunSpoYMgpC4OHjwIa2vrchdvW1tbcDgcHDt2DA4ODujQoQO4XC66dOmCI0eO4OeffwaXy4VcLseDBw+wceNG/PTTT5g+fTrGjRuneR8/Pz88fvwYO3bsQHp6OrhcLp4+fYpffvkF33zzDcLCwjB79mxwOJxysXG5XPB4POzYsQMymQwAcODAAezduxeLFy9Gs2bNkJKSgkOHDsHR0RFFRUWaCzufz8fEiRMBlNxR1b17d3To0IH1/i+T0rlz5zBs2DD07t1bc87X1xffffcd/v33XwiFQjx69AifffYZEhMTMWvWrHK3KhOiLaocJyZv/PjxEAqFFd5iKpfL8frrr6OgoAAxMTGauYb8/Hz8+OOPiI6ORnJyMuzs7BAYGIgJEyagW7du5d5HrVZj//79OHDgAJ4+fQq5XI6mTZti5MiRiIiIgJVV1dOFv/zyC3744QekpqaiSZMm+OCDDzBo0CAAgFgsxvLly3HixAlYWVlh6NChaNSoEX7++WfNMJufnx8+/fRTvPfee+Xee/fu3Vi1ahX27duH4OBg1rnr169jw4YNuHPnDgQCATp06ID/+7//Q8uWLbVrXEIqQImDEEJIjdAcByGEkBqhxEEIIaRGKHEQQgipEb0lDrFYjPDwcCQnJ5c79+DBAwwfPhxhYWFYtGgRa80hQgghxkUvieP27dt45513kJiYWOH5uXPn4rPPPsPx48fBMAxr5VBCCCHGRS9Ljuzfvx+RkZH49NNPy51LSUmBVCrV3EY4fPhwbNy4kbWYmzZycyVQqxm4utojO1tcL3GbMmqHEtQOpagtSlhCOzAMgyKZCnnFcuQXvfinWI78IgXyi+XgcrmYN7x99W9UCb0kjhUrVlR6LiMjAyKRSPNYJBIhPT29xp+hVjNQqxnNz4Ta4SVqh1LUFiVMuR2UKjVyJXLkSmTIefHv3CI5ciSykmNiOfKKZFCoKv8dPRrY1ikGgy9yqFarWRW3DMNUWIFbHVfX0ipYkcihXmIzddQOJagdSlFblDDGdmAYBmKpEtmFUmQVSkv+XVD+5/wiuaFDNXzi8PT0ZK0ZlJWVBXd39xq/T3a2GGo1A5HIAZmZhfUZokmidihB7VCK2qKEIdpBpWaQVyRDruRlz6C0x5Ajlml6DXJl7Zbvr4iNNQ/OQmuAw0FGgRSqMr0sTycT73E0btwYAoEA169fR8eOHREVFaXZT4AQQoxdkVxZJhGUJoWXSSJHIkdBkRz1NTjGAdBAyIezHR8udgI42/Hh/OLfLnYCzbH8IgV2nIvDw9R81uv7tPHEuO4t6hSDwRLH1KlTMWvWLAQEBOCrr77C4sWLIRaL4e/vjwkTJhgqLEIIAVAyD5JfLC+dR5CUmUcoc0yqUNXbZwqsuJok4GwngIvm36XHnIR88LiVD+crVWr8eScZB689Zc1zeDja4L1erdC2sRO4VbxeG2azVhUNVbFRO5SgdihFbVFCJHJA0vO8V5IBu8eQI5Ehv0iO+pxDd7S1ZiWD0h7Cy8cCCPm8Ws3xvvQksxA7zsbhabZEc4zLAQYFeWN4Jx/wrUp2tORyOax54Zoy+FAVIYTUFzXDoLBYUW4eoWxSyCuSQyKrvyJjax7nlV5C+eTgJOTDiqf7srlD15+xkkYTN3tM6d0KTdzqdwl9ShyEEJMgV6pf9BJkFQwflfw7r0jOmgSuK3sbq5JhIuGLpGAvKDe3YC+wqlMvoT5NCG2Beyl5UKkZjOjsizcCvaoc1qotShyEEIN6eRvqq0NGZZNDrkQGcT32EnhcTpXJwMWODyehAHwr413OTyJTwJrH1Qw/AYCrvQAf9muNRs5CeNaxVqMqlDgIITpTabGauOzQUdXFajUl5FuVDBXZlx8ycrYToKWPC2QSGbhG0kuojdiETPxw4TF6+nlgVEhT1rkOTVx1/vmUOAghNcYwDIrkSvaQkbj87agFUkW9fSaXAziV7SUIX+0llPwssOZV+T5OdgJkGkERXW3kSmT44UI8rj3JBgAcvZWELs1F8K3nOYzqUOIghLAoVWrkF8tL6xDE8vJzCzooVnOxF8BFyK/wdlRnOz4a2PLrfBupqWIYBmcfpOHXvxNQJC+9/dfRlg+xrP6Ss7YocRBiQcoVq4lfTC4XlQ4n1WuxGgdwsn11uKj8Y1s+XYoqk5ZfjJ3n4vDgeflCvtFdmsFOoP+2o/9ahJiBV4vVKrodNa9IjmJ5/Rerla1DKNtD0KZYjVROpWbw5+1kHLj2FApVae/Os4EtJvdsibaNnQwWGyUOQoycVKF6ZQmL8sta1GexGgelxWoV3W1UX8VqpHK5Ehm+OnYPT7NKl3/ncoDBwd4Y1tGHdSeVIVDiIMRAXi1Wq2g5i1yJjDWmXVfWPG4VyUC/xWqkco62fJRNyboq5KstShyE6IBcqWKvgFpBL6G+i9UcbKwrTQbOdny09HWFtFBKvQQTwONyMLV3KyyPuo23OvrorJCvtihxEFIDZYvVKppH0EWxmhWXU2ZCufLhI+tqegmOtnzIxLJ6i4vUD4lMidP3UzE42ItVW+LrZo+N40IgNMDkd3WMLyJCDIRdrFbxshb1XaxmJ7CqtJfwMjnY21ibdLEaqdzVhCzsvhCP/CI5bPk89PdvxDpvjEkDoMRBLICmWE1cOo/w6qqo9V2sxuNyXhSrvUgGL4rVXi5v8fJYdcVqxDy9WsgHAHv/foKQZiI42FobMDLtUOIgJk2pUiOvSF7BkNGLmgSZAlkF0notVrPl80p6CK8Wq5VZ4sLRlnoJpLzKCvkaCPl4N7SFSSQNgBIHMWJFciVyxRUMGemzWM2+/LIWNtRLILVQWSFf79aeeKerYQr5ast0IiVmQ61mXvQSZMgpYk8q63pnNZcyvQNNMnjRc2hAxWpEByor5HN3tMGUFzvymRpKHKReSRWqKucR9FGsVraH0NzbGZApYUvFasRAjt5Kwv7YRM3jkh35vDC8k6/BC/lqixIH0YqaYVBQrKhwHkEfxWqVLWtRXbEabZdKDG1Au0Y4dT8V2WIZfF3tMKV3KzQVORg6rDqhxEEgV6oqvPVUH8VqLhXWJZT8bGdEO6sRoi21mmGt4mvLt8Lkni3xLFuCNwIbm0VVPiUOM8YwDAqlinLJQF/FapUta6FNsRohpkYiU+LXywmQK1X4sH8b1rkgHxcE+bgYKLL6R4nDRClU6konlXMkMhRIS25DVdZjL8FOYFXlGkfOdnw42FhTL4FYnKtPsvDD+XjkvdggqmtLd7T31f1OfIZCicPIMAwDiUzJWvSuoh5DoQ6K1SrbK4GK1QipWK5Ehh8vPMbVJ1ms43ee5VLiIPWjumI1XeysJnxRrFbRsthUrEZI7TAMg3MP07Dn8hMUyUuHel8W8nVu5mbA6HSPEkc9YBgGxXJVhUNGuUUlezHnSmQoKFbUb7FaFb2E5l4ugFxJxWqE1LO0/GJ8fy4O9yss5GsKO4FpVH/XBSWOOmAYBj9dfIy//k2v12I1G2tepZPK2haridzs6TZUQuqRmikp5PvjavlCvvd6tYR/Y2cDRqdflDjq4G5yHmLuPtf6+RwAjkJ+aTIQVrA8tr0AQtp/mRCjwwEQl1agSRovC/mGdfS1uPk/ukLVQfQ/KZqfrXnc0uUsXplHeLm8RQNb2lmNEFPF4XDwbmgLPHieB3cHG7Mo5KstShy1lJpXhNvPcgCUfBNZ9XZHeDrZGjYoQki9+Tc1H01F9qxlQVzsBVg0NAhezkKL/hJoub95HcX8UzpEFezrQkmDEDMhkSmx81wclkfdxoFrz8qdb+Jmb9FJA6AeR61IZEr89W+a5nFYQGMDRkMIqS+vFvL9eTsJIc3dLHZIqjKUOGrh3MM0yF7UWni5COFvgssiE0JK5RXJ8cOFeFxNYBfytfd1RQNbvoGiMl6UOGpIpWYQU2ZSPCygMS2xQYiJqrSQz9YaE18U8tH/3+VR4qihG4nZyBLLAAD2Nlbo3tLdwBERQmojPb8YO/96hPspeazjvVp74J0uzWBvY/6FfLWll8Rx5MgRbN68GUqlEhMnTsTYsWNZ5+/du4fPPvsMCoUCDRs2xJdffglHR0d9hFZj0f8ka37u26ahyW7EQoglS8goxBeHb7OW93F3tMHkni3RzstyCvlqS+e3BqSnp2P9+vXYs2cPDh06hH379iE+Pp71nBUrVmDWrFk4fPgwmjZtip07d+o6rFp5klmIf1MLAJQsDNjfv5GBIyKE1Iavmz0aOwsBlCzfMyjIC6ve7khJQ0s6TxyXLl1Cly5d4OTkBKFQiLCwMERHR7Oeo1arIZFIAADFxcWwsbHRdVi1crzM3EZIMze42AsMGA0hpLZ4XA6m9GqFpiJ7LBvWHmO6NrO46u+60PlQVUZGBkQikeaxu7s77ty5w3rO/PnzMXnyZKxcuRK2trbYv39/jT/H1dVe87NIB7fO5Yil+Ds+U/N4VM9WOvmc+mTs8ekLtUMpS2yLO0+zEX0zCXPeDNKsAi0SOUAkckDH1p40+V0LOk8carWa9R+GYRjWY6lUikWLFmH37t0IDAzErl27MG/ePGzbtq1Gn5OdLYZazehsj+k/riZqNkVq6eEIVz7PqBcRpL22S1A7lLK0tpDIlNj7dwLOPCipufJqYIsB7RpZXDtUhMvlsL5s1/j19RhLhTw9PZGZWfpNPTMzE+7upXcixcXFQSAQIDAwEAAwatQoxMbG6jqsGlGo1Dh1P1XzOCyA5jYIMWbXnmRh3r5rmqQBAEduPoNSVX973VgynSeObt264fLly8jJyUFxcTFiYmLQs2dPzXlfX1+kpaUhISEBAHDq1CkEBAToOqwauRyfgYLikh33XOz46NTUvDdpIcRU5RXJsTHmPjYcv6+p/gaAjk1csXRYe4tfKqS+6HyoysPDA7Nnz8aECROgUCgQERGBwMBATJ06FbNmzUJAQABWrVqF//znP2AYBq6urli5cqWuw9IawzA4fqd0UnxAu8b0x0eIkWEYBn/9m45fLiVQIZ8ecBiGqa9N6QxKV3McD5/n4YvDJZP5fCsuNo4LMYnCIBrHLUHtUMpc26KmhXzm2g41Udc5Dqocr0bZPTdCW3mYRNIgxJKcvPeclTSokE/3KHFUIaOgGNefZGseh7WjSXFCjM2Izk0Qm5CFHIkMbwR6YUQny9uRT98ocVThxN3neDmOF+DljMYudgaNhxBLJ1eqoVCpYCco7fnbWPPwQV8/8K14aOZueXUqhkCJoxLFciXOPiy9le/1QNpzgxBDevg8DzvOPYKvmz0+GtCGda51I9raQJ8ocVQiNiELxXIVAMCzgS0CvGm8lBBDKJIpsffKE5x+UUuVll+Mbi3d0bGJq4Ejs1yUOCrxKK1A83PP1h6apQoIIfpz/UkWdp+PR26Zmgwbax6kZW65JfpHiaMST7PFmp+b07gpIXqV/2JHvthXduTr0MQV74a2oAVGDYwSRwWUKjWSsiWax751uN+ZEKK9l4V8ey4nQCIr7VU4vijke40K+YwCJY4KpOQWaRY0FDkIqHaDED2QKVRYH30Pd18p5Ovh54GxXWlHPmNCiaMCiVmlw1S+btTbIEQf+FZcVv2FyMEG7/WiQj5jRImjAmUTRxNKHIToBYfDwbs9WuDf1Hz08PPAiM5NYEOFfEapxolDLpeDz+frIhajkZhZNnHQxDgh9U2uVCPmnxQMaNeI1ctwthNg3ZjXYCeg77TGTKtlXpVKJTZt2oQePXogJCQESUlJGDduHGufDXOhVjN4ll12qIqqxQmpTw9T87Ho9+vYe+UJ/rj2tNx5ShrGT6vEsW7dOsTGxmLdunWwsrKCSCSCp6cnIiMjdR2f3qXlF0OmLNnspYGQD2c7uu2PkPpQJFdi11+P8EXUbaTmFQMAjt1JRlKOpJpXEmOjVWo/evQooqKi4OzsDA6HAxsbG3z++efo06ePruPTO9b8hiv1NgipDzcSs7Hr/CPkStiFfKO7NEVjZ6EBIyO1oXWfkMdjT1KpVCoIBOb3bZyVOEQ0MU5IXeQXyfHjxce48pg9rN3e1wXv9mgJVyrkM0laJY6BAwdi9uzZmDt3LhiGQXJyMr766iv069dP1/HpHfuOKpoYJ6Q2GIbB+biSHfleLeSb0L0FQppTIZ8p02qOY+7cufDx8cHo0aNRWFiIQYMGwc7ODnPmzNF1fHrFMAyeZtHEOCF1FZuQhW1n4lhJo4efB9aM6oQuLUSUNEycVj0OqVSKyMhIREZGIicnB05OTuByuUhKSoKdnflcXLMKZZo/dCHfCiIHGwNHRIhp6tzUDc1EDkjILITIQYDJPVvRCtNmRKseR9lJcBcXF3C5XCiVSrz11ls6C8wQErPLDlPZ0bciQrSkVjOsx1wuB1N6t8QbgY2xamQnShpmptIeR3JyMkaPHg2VSoWioiJ07dqVdV4mk6FFixY6D1CfWIV/IprfIKQ6cqUah64/xb9p+Vg0JAhcbumXLR9Xe4ztRjeYmKNKE4eXlxe2b9+OgoICTJs2Dd988w3rPJ/Ph5+fn84D1CdaaoQQ7T1MzcfOc3GamowT954jLIB2yrQEVc5xtGlTsj3jmTNn4OLiUu68Wq3WTVQG8pQWNySkWkVyJfb9/QSnXuzI99LtZzkY2K4RDfFaAK0mx/Pz87FmzRqkp6drkoVSqcTTp09x8eJFnQaoL3lFcuS92GVMYMVFwwa2Bo6IEONTVSFf37YNKWlYCK0Sx8KFC+Hs7IxGjRohJSUFr732Gvbu3YsxY8boOj69Kdvb8HG1Z43VEmLpqJCPlKXVXVUPHjzA119/jcmTJwMAZsyYgW+//RanTp3SaXD6xF4Rl4apCHnpQlw6Pt13jZU0HG2sMbN/a3zyuj8lDQukVY/DyckJAoEAPj4+iI+PBwAEBwcjKSlJp8HpEy01QkjFEjPF7EK+Vh4Y060ZHGhHPoulVY+jTZs2+Prrr8EwDNzd3XHixAmcP38etrbmMw9Au/4RUrGI15pA5CCAyEGAeYPb4f2+fpQ0LJxWPY5FixZh8eLFyMvLw5w5c/DRRx9BLpdj6dKlOg5PPyQyBTILpQAAHpcDL1qtk1iopGwJHGyt4SQs3azNxpqHOW+0g6uDDe3IRwBomThu3ryJLVu2wMbGBh4eHrhy5QoUCgWEQvO4wD7NKt0PwNvFDlY8rTpihJgNuVKNqBvP8L9bSejYxBWzBrZlnW/sYj5LC5G60+oKuXz5clhZleYYa2trs0kaAA1TEcv2b2o+Fv9+HVE3nkGlZhCbkIVrT7IMHRYxYlr1OPr164etW7diyJAhcHNjL4dsDvMciZmFmp/pjipiKYrkSuy/8gQn77EL+Vp5OqKhk/l8MST1T6vEceLECYjFYmzatElzjGEYcDgcPHjwQGfB6UtidulQFSUOYgluPs3Grr8eIeeVQr5RIU3Rz78huFTIR6qgVeKIiorSdRwGI1WokJpbBADgcABv2i6WmLH8Yjl+uvgYf8ezC/mCfVwwqWcLuNrTVgKkeloljsaN67Zw2ZEjR7B582YolUpMnDgRY8eOZZ1PSEhAZGQk8vPzIRKJ8PXXX6NBgwZ1+kxtJWVL8HJB6EZOQrprhJitrAIp5u29BnHZHflsrDE+tDm6NKfNlYj2tN5zvLbS09Oxfv16HDhwAHw+H6NHj0ZISIhmSXaGYfDBBx9g0aJF6NmzJ7766its27YNc+fOrZfPZxgGp+6nIj69oMLzGQVSzc80MU7MmZujDdo0dsLVhJKJ79BW7hjbrTnVZJAa03niuHTpErp06QInJycAQFhYGKKjozFz5kwAwL179yAUCtGzZ08AwPTp01FQUPFFvjauJWZj9/l4rZ5L8xvE3E3o3hwZBVKMCmmCQO/yK14Tog2dJ46MjAyIRCLNY3d3d9y5c0fz+NmzZ3Bzc8PChQvx4MEDNGvWDEuWLKnx57i6ll70RWU2YbqtZdKw5nHRv4MPRGZ0v7qINqMCYJnt8CS9AL+cj8ecoYGw4Zf+b96qiRu2Tu9p8cNSlvg3UZ+0ThyxsbHYt28fMjMzsX79evzyyy+YMWMGeLyq5wTUajXrj/Tl3VgvKZVKxMbG4ueff0ZAQAA2bNiA1atXY/Xq1TX6RbKzxVCrGYhEDsh8cXutWs3ganyG5jmjuzSFo235bjkHHLTydARfpda81tSVbQdLZmntoFCVFPIduZkElZqBA5+HMV2bAbC8tqgMtUPJ1r5lv2zXlFaJ48CBA9iwYQNGjhyJs2fPgsPh4MSJE5BIJFiwYEGVr/X09MS1a9c0jzMzM+Hu7q55LBKJ4Ovri4CAAABAeHg4Zs2aVZvfpZz4jALN4mzOQj4GB3lZ/DctYr7iUvOx41wcnr/YkQ8ATt57jsHBXmhgy6/ilYTUjFaV41u3bsX27dsxc+ZMcLlcuLi4YPv27Th69Gi1r+3WrRsuX76MnJwcFBcXIyYmRjOfAQDt27dHTk4OHj58CAA4ffo0/P39a/nrsN16mqP5OcjHhZIGMUvFciV+OB+P5VG3WUmjlacjlo/oQEmD1Dutehx5eXmau6BeXnzd3NygUCiqfa2Hhwdmz56NCRMmQKFQICIiAoGBgZg6dSpmzZqFgIAA/Pe//8XixYtRXFwMT09PrF27tg6/Uqnbz8okDl+aCCTmhwr5iCFolTg6dOiAjRs3Yvbs2ZpjP/zwA4KDg7X6kCFDhmDIkCGsY9u3b9f8HBQUhN9//12r99JWjliGpy8qwnlcDto1dqrX9yfEkApeFPJdpkI+YgBaJY7PPvsM06dPx969eyEWi9G3b1/Y2Nhg69atuo6v1m4nlfY2WjdsAFu+zm8gI0Rv/n6cyUoaDjbWGN+9Obq2oEI+ontaXU0bNmyIgwcP4s6dO0hNTYVIJEJwcDBrxVxjwxqm8qFhKmJe+rdthItxGXicUVhSyLZB3KIAACAASURBVNe1ORwquGOQEF3Q6so/evRovPnmm3jjjTe0Hp4yJIVKjbvJeZrHwTS/QUyYWs2gUKZgTXJzuRxM7d0K2WIZfTEieqfVXVURERGIiYlB7969MWPGDMTExGg1MW4o/6bmQ6pQAQDcHW3QsIHpL/1OLFNStgTLDt3C18fuQa1mWOe8XOwoaRCD0Dpx7Nq1CydOnMBrr72G77//HqGhoYiMjNR1fLVSdl2qQG9nGvMlJkehUuP3q4lY/McNPM4oxOOMQsTcTTF0WIQAqOGSIyKRCJ07d0ZWVhbS0tJw69YtXcVVJzKlWvOzs53AgJEQUnMVFfJZcTlQvtLjIMRQtEocjx8/xtGjR3H06FFIJBKEh4dj8+bNaNOmja7jqxWlqjRxWNP+4cREFMuV2H8lESfvPUfZFNHSwxFTerWkfb+J0dAqcYwYMQJ9+/bFwoUL0aNHD3C5xn0xVqpK/7ez4tEwFTF+VMhHTIlWiePChQuwtzedJccVZXocVkae5IhlYxgGW07/i4uPMljHg3xcMKlHC7g5UCEfMT5VJo7x48fjp59+wsSJEyudYK7viu/6oFSXSRzU4yBGjMPhsFZspkI+YgqqTBwjR44EAIwbN04vwdQXRZmhKprjIMZuROcmuJ6YjZYejhjXjQr5iPGrMnG8XF9KIpFUmDy2bNmim6jqqOzkuBUlDmIk1GoGJ+8/R+embqy7/WyseVg+ogPsBMa7EgMhZVX6l5qdnY27d+8CAL766it4e3uzzovFYmzduhXTp0/XbYS1wEocXOruE8NLzpFg+9k4PM4oxIOUfHwc1pZ1npIGMSWV/rXa2dlh48aNyM3NhUwmw7Jly1jn+Xy+USYNAFCoaaiKGAeFSo3DN57h8Isd+QDg6pMs3HqaQ0vhEJNVaeKwsbHBH3/8AQCYOXMmNm3apLeg6oqGqogxiEvLx45zj/A8t0hzjMfl4M0OPmjnRcv8E9NVZf84ISEBzZo1w3/+8x/Ex8dX+JyXGzwZk7J1HNY0VEX0rFiuxP7YRJy8S4V8xDxVmTgiIiJw48YNhIeHV3iew+HgwYMHOgmsLti341KPg+jPrac52HX+EbLFMs0xG2seRoY0QX//RlTIR8xClYnjxo0bAKDZD9xUKGioihhAQkYhvjp2l3UsyNsZk3q2pEI+Yla0vqpevnwZAJCTk4OlS5fim2++gVQq1VlgdcFeq4q+4RH9aObugNeauQEoKeT7sF9rzBnUjpIGMTta3QP45Zdf4s8//8SZM2ewaNEiiMViWFlZYenSpVi9erWuY6wx1lpVtOQI0RG1mgH3lTm0CaEtYCewwsjXmlIhHzFbWiWOEydOYO/evSgsLMRff/2F6OhouLi4oG/fvrqOr1YUtOQI0SG1msGJe89x5kEqlg5rDxtrnuack5CP93q1MmB0hOieVl/H8/Ly4OHhgYsXL8LLywve3t6wtrYGwxjn/gDs1XGpx0HqT3KOBJ9H3cJPFx8jOacIv8cmGjokQvROqx5HmzZtsHLlSly9ehX9+/dHXl4evvzySwQFBek6vlpR0H4cpJ5VVMgHAHdTciFXqsG3or8zYjm0+mtfvXo18vLy0KFDB3z00Ud4+vQpsrOz8cUXX+g6vhpjGIb1PzYtOULq6lFaARb/fgMHrz/T/G3xuBwM6+iD5SM6UNIgFkerHkfDhg2xdu1ayGQyZGRkoHnz5ka7wKHilXWqaGlqUlvFciV+i03EiVcK+Vp4OGBKr1bwokI+YqG0ShxisRiRkZE4duwY1Go1eDweevfujTVr1hjdBk9Uw0Hqw52kHOw8xy7kE1hxMTKkKQb4Nyp3NxUhlkTroSqJRIKjR4/izp07OHz4MFQqFVauXKnr+GpMoaQ7qkjdpeUVs5JGoLcz1ozqhLCAxpQ0iMXTqsdx9uxZHDt2DA4ODgCA5s2bY82aNRg4cKBOg6sN9jpV1OMgtdPfvxEux2ciNb8I47o1R/eW7jTsScgLWm8CoFKpWI/VajWsrY2vwImGqkhNZRVKIVep0chJqDnG5XLwQT8/2Fjz4GjLN2B0hBgfra6sAwYMwOzZs/HgwQPk5eXh3r17+OSTTzBgwABdx1djCmVpgqOhKlIVtZpBzD8pmLfvGrac+hdqNbsuyd3RlpIGIRXQKnHMnTsXLi4uePvtt9GlSxe88847aNy4MebOnavr+GqMajiINl4W8v148TFkSjUSMgtx/J8UQ4dFiEnQaqhKKBRi3bp1WLlyJQoKCuDm5ma0473s23EpcRC2ygr5GjkL0cLD0YCREWI6tEoccrkcmzZtwv/+9z9kZWWhYcOGGDZsGKZNmwaukV2cFazlRowzuRHDeJRWgB3n4pDyyo58Q9t7Y2gHH+qhEqIlrRLHF198gfv372PhwoXw9PRESkoKtm7dCrFYjDlz5lT7+iNHjmDz5s1QKpWYOHEixo4dW+Hzzp49i88//xynT5+u2W9RBvt2XLoQEECqUGH/lSdUyEdIPdEqcRw7dgzHjx+Hi4sLAKBdu3Zo3749hg4dWm3iSE9Px/r163HgwAHw+XyMHj0aISEh5baczcrKwpo1a2r5a5RSlLn7i7aNJTKFCgt/u46MgtK9Y6iQj5C60eorub29PeRyOeuYlZWVpq6jKpcuXUKXLl3g5OQEoVCIsLAwREdHl3ve4sWLMXPmTC3Drhz1OEhZAmseOjd10zymQj5C6k6rHseIESPw3nvv4cMPP4S3tzcyMjKwfft2dO3aFefOndM8r1evXuVem5GRAZFIpHns7u6OO3fusJ7z448/om3btvWy2i6rAJASBwEwvJMvHqbmY2C7RuhGhXyE1JlWiePAgQMAgHXr1rGOZ2Zm4sKFCwAADoeDU6dOlXutWq1m/Y/KMAzrcVxcHGJiYrB7926kpaXV/Dd4wdW1ZM0sxfN8zTE7IR8iUfW9InNlab97Rn4xtp98gOkD28K1zHatXo2c8N9pPShhwPL+JipD7VA3WiWOukxWe3p64tq1a5rHmZmZcHd31zyOjo5GZmYmRowYAYVCgYyMDIwZMwZ79uyp0edkZ4uhVjOs23FVChUyMwtrHbspE4kcLOZ3VzMMTt57jv1XEiFVqCApkuM/YW3B4XAsqh2qQ21RgtqhZGWEl1+2a/X6eoylQt26dcPly5eRk5OD4uJixMTEoGfPnprzs2bNwvHjxxEVFYVt27bB3d29xkmjrLKV4zRUZf5SciRYfug2frzwGFJFyX/7G4nZeJYtMXBkhJgvrdeqqi0PDw/Mnj0bEyZMgEKhQEREBAIDAzF16lTMmjULAQEB9fp5VMdhGZQqNY7cTELUjWdQvlLIN6VXS/i6Gddy/4SYE50nDgAYMmQIhgwZwjq2ffv2cs/z8vKq07AYQIscWoL49ALsOBuHZCrkI8Qgapw45HI5+HzjXfiNdTsu3W5pVqQKFX6LTUTMPymsQr7m7g6Y0rsVvKmQjxC90OqrmVKpxKZNm9CjRw+EhIQgKSkJ48aNQ2Zmpq7jqzFa5NB8/Zuaj+NlkobAiotx3Zoj8q1gShqE6JFWV9Z169YhNjYW69atg5WVFUQiETw9PREZGanr+GpMSUNVZivIxwVdmpfUBAV4OWP1yE54PZAK+QjRN62Gqo4ePYqoqCg4OzuDw+HAxsYGn3/+Ofr06aPr+GqMPcdBFxRTxTAM8osVcBKyh0XHhzZHsK8L7chHiAFp/ZWcx+OxHqtUKggEgnoPqK7KznHQ1rGmKVssxbpj9xB54KbmFtuXGtjyEdrKg5IGIQak1ZV14MCBmD17Nh4+fAiGYZCcnIwlS5agX79+uo6vxuiuKtOlZhicuPsc8/Zdx61nOcgWy7D/yhNDh0UIeYXWOwD6+Phg9OjRKCwsxKBBgyAUCrVaUl3f5GV7HDRUZTJScouw/NBt/HAhvlwvg2GYSl5FCDEEreY4BAIBIiMjERkZiZycHDg5ORndBk4vlZ0c51GPw+gpVWocuZWEqOuvFPI52WJKr1Zo1bCBAaMjhFREq8SxadOmSs/Vx1Lo9Yl1O66RJjdSIj69ZEe+5Bx2Id+Q9t4Y2t4HfCv670eIMdIqccTFxbEe5+Xl4datW+WqwY0B3VVlGn6LTcThG8/KF/L1agVvV6rJIMSYaZU4Nm7cWO7YxYsX67QYoa6w7qqioSqj5STkswr53n6tCQa2o5oMQkxBrdeqCgkJwUcffVSfsdQLVgEgXYSMVj//hrj0KAM21jxM7tkSIkeb6l9ECDEKWiWO+Ph41mOFQoEjR46gcePGOgmqLuh2XOPCMAz+fpyJRk5C1oq1XA4Hcwa1g5DPo5oMQkyMVokjPDwcHA5Hc1skl8tFkyZNsGTJEp0GVxs0VGU8ssVS7PorHree5aCpyB5Lh7UHr0wv0E6gl8WZCSH1TKv/c2NiYuDj46PrWOoFTY4bnpphcOpeKvZdeaKpyXiSKcap+88xsJ3x9VIJITWj1VfyUaNGQSwW6zqWekGr4xpWZYV8A/wbIbSVhwEjI4TUF616HN7e3rh37x5CQkJ0HU+dsXocVMehN1TIR4jl0CpxWFtbY9KkSRCJRHBzc2NNZv7+++86C642WBs50VCVXlRVyPcm7chHiNnRKnFEREQgIiJC17HUCxqq0q9ciQxfRN1m9TKokI8Q86ZV4pBIJBg3bly541u2bKn3gOpCzTBQlbmA8aiOQ+ec7QR4PdAL/7uVRIV8hFiIShNHdnY27t69CwD46quv4O3tzTovFouxdetWTJ8+XbcR1oBSVZo0rLgcqg/QAbWaKZcUhnfyQaFUgTc7eMPd0dZAkRFC9KXSxGFnZ4eNGzciNzcXMpkMy5YtY53n8/lGlTQA2jZWlxiGwZXHmfgtNhELhwbB1b50Ey++FQ9Te7cyYHSEEH2qNHHY2Njgjz/+AFCyAm5VK+QaC4Wa5jd0IVssw+7zj3DzaQ4AYPf5R/jkdX/q0RFioeq8rLoxUVLxX72qqJAPABKzxBXuB04IsQxmteYDe46Dehx18Ty3CDvOxSEurYB1vF/bhhjVpSmEfLP60yGE1IBZ/d+vVNG2sXWlVKnxv1tJOPRKIV/DF4V8flTIR4jFM6vEoShzoaPJ8Zp7klmIbWfikJQj0RzjcTkID/bCmx18aUc+QggAM0scSir+q5O8IjkraTQTOWBK75bwcbWv4lWEEEtjtomDiv9qrr2vK7q2EOF6YjYiOjfB6wFUyEcIKc+sEoeizOQ49TiqJpYqkFkoRVORA+v4+O7N8fZrTaiQjxBSKbNKHEo1FQBWp6SQLws/XoyHNY+L1SM7wrbMHVKOtnxQziCEVMWsEoeC7qqq0quFfACwPzYRE0NbGDAqQoipMavEQXUcFVMzDM7cT8Wvf7ML+Zzt+Gjn5WzAyAghpsjMEgdVjr/qeV4Rdp6Lw7+pFRTyhTSFkPb9JoTUkF6uGkeOHMHmzZuhVCoxceJEjB07lnX+5MmT+Pbbb8EwDLy8vLBq1So0aFDzQjNaq6qUUqVG1I1nOHT9KeumAc8GtpjSuxVaUyEfIaSWdH51TU9Px/r167Fnzx4cOnQI+/btQ3x8vOa8WCzG0qVLsW3bNhw+fBh+fn749ttva/VZrKEqC04cDMNg3s9X8FtsoiZp8LgcvNnBGyvf7khJgxBSJzq/ul66dAldunSBk5MThEIhwsLCEB0drTmvUCgQGRkJDw8PAICfnx9SU1Nr9VmsoSoLrj/gcDjo7d9I87ipyB6fD2+Pt19rStXfhJA60/lQVUZGBkQikeaxu7s77ty5o3ns7OyMAQMGAACkUim2bduG8ePH1/hzXF3twbex1jx2dLCB6JUaBUsy2M0el+PS0bGZG956rQl4FnyzgCX/HbyK2qIEtUPd6DxxqNVq1r4NDMNUuI9DYWEhZsyYgdatW2PYsGE1/pzsbDHyC6WaxwqZEpmZhbUL2oSIpQrsuZyAgQGN0cStdGkQkcgB/+nfGhwOBznZkirewbyJRA4W8XegDWqLEtQOAJfLgWsdlhLS+ddQT09PZGZmah5nZmbC3d2d9ZyMjAyMGTMGfn5+WLFiRa0/y5LuqmIYBn/HZ+LTfdfw17/p2HE2jrXfOgDaaIkQohM6TxzdunXD5cuXkZOTg+LiYsTExKBnz56a8yqVCtOnT8cbb7yBRYsW1eliZyl1HNliGb6OvodNJx+goFgBoGRzpZtPsw0cGSHEEuh8qMrDwwOzZ8/GhAkToFAoEBERgcDAQEydOhWzZs1CWloa7t+/D5VKhePHjwMA2rVrV6ueh8LMV8ettJBPyMe7PVuiYxNXA0ZHCLEUeqnjGDJkCIYMGcI6tn37dgBAQEAAHj58WC+fw16ryryGaaiQjxBiLMzqamOOq+MqVWocvZ1ccSFfr5Zo3cjJgNERQiyRWSUOc6zjSM0vxh9XE/Fy3pvH5WBwkBfe6kg78hFCDMO8EocZbh3r7WKHwcHeOHIzCU1F9pjSqxV83WhHPkKI4ZhV4jCHyfG8IjmchHzWsWEdfeHmIEDv1g1pZ0NCiMGZ5tW1EqZcxyGWKrDtzL+Y8+tVZIulrHN8Ky76tW1ESYMQYhTMLHGYXh1HyY58mZj3opBPqlBh9/l4MAxT/YsJIcQAzGqoytS2js0Ry/DDhXhcT2QX7lnzuFCo1OBb8QwUGSGEVM6sEoepbB2rZhiceZCKvX8/QbH8lUK+Hi3QsambAaMjhJCqmVXiMIX9OFLzirDz3CM8TM1nHe/btiFGUyEfIcQEmNVVitXjMMI5jtP3U/HTxfhyhXzv9WqJNlTIRwgxEWaVONh1HMY3VOVqL9AkDS4HGBzsjWEdfWguw4wVF0sgFudBpVIaOhRkZHChLjMPaKkspR14PCvY2zvB1tau3t/bvBKHyrgnx4N8XNCtpTtS84qokM8CFBdLUFiYCycnEayt+QZf5t7Kigul0vwvmNWxhHZgGAYKhRx5eSVbWtR38jDbxGHoAsC7ybkAgHZezqzjk3q0AN+KRzUZFkAszoOTkwh8vsDQoRALw+FwwOcL4OQkQn5+FiWOqiiMYK0qiaxkR75zD9PhYsfH6lGdIOSXNrMt36yanFRBpVLC2ppf/RMJ0RFra75OhkmNbzynltRqBmU3wNP3N3qGYRD7OBOf7r2Gcw/TAQA5EjkOXHuq1ziIcTH08BSxbLr6+zObr78qpuyS6hy9/g+bK5Fh9/nyhXydm7lhcJCX3uIghBB9MJvEYYgaDjXD4OyDNPz6dwKrkM9JyMfEHi3QmQr5iIkIDe0EgUAA7ovb2BmGgZubCOPGTUR4+Fua56WkJGPHji24evUKZDIZPD09MWjQUIwc+Q54vNK7A7OyMrFz5zb8/fdFSCQSuLu7Y9iwCIwYMapGcS1fvgSnTp3A778fgZubSHP8zz+P4I8/9mPnzp9Yz79x4xqWLJmHo0dPaY79/fcl/PrrT3j0KA4Mw6Bt27aYOvVDtG7dtkaxVKSgoACrVn2OGzeuws7OHpMnT2W1V1mnTsVg586tyMrKQps2/pgzZz68vX0AAKmpz7F69XLcv38Prq5u+Oij2ejevUe1n7FmzQoMGBCGDh061fl3qQmzGapS6rmGIzWvCCsP38H3fz1iJY0+bTyxZlQnShrE5Gzf/gNOnDiPEyfO4/jxc5g8eRrWrl2JxMQnAICnTxMxdepEuLmJsHv3HsTEnMOiRctw+nQMFi+ep3mfzMwMTJ48Do6Ojti1aw+OHz+LBQsisWfPT/j++21ax1NQUIBLly6iT5/+OHToj1r9TocPH8TKlcswcuQYHD58HIcOHcNrr3XFrFkfICHhca3es6y1a7+AUGiLw4dj8MUXa/Ddd98iPv5RuefdvfsPVqxYig8//Bh//nkK/foNwOzZMyCTlSxoumTJfLRp449jx07j44//D8uWLUZubm61nzFlyvtYv34tFApFnX+XmjCfxFHmvmyejms45Eo1voi6zar+9nC0wcIhgXivVyvYUfU3MXE8Hg8DB74BodAOT56UXGA3bvwa/fsPxIwZH8PNTQQOh4PWrdvgq6824tatG/jrr7MAgO3bNyMwMAgffPARnJycwOFw4O/fDvPnL0F2dpbWMURHH0VwcHsMH/42Dh8+WOOLY3FxMTZt2oB58xaje/cesLKygkAgwNix4zFsWASePn1S7jW3b9/EgAE9yv0zbtzIcs8tKirC+fPnMHny+xAIBGjbth0GDAjDkSMHyz33r7/OoEePXggN7QkrKysMHToMNjY2uHo1FomJT5CQEI9Jk6bAysoKXbt2R3BwB0RHH632M1xd3dCsWXMcP360Rm1TV2ZzhStb/KfrW3H5VlwM7+yLXX/Fg8sBBgV5Y3gnKuQj1fvzdjIOXHsKqUJV/ZPryMaah+GdfDGoFvNsCoUCBw/+BqVSAX//AMhkMly/HosJEzaXe26DBk7o3r0Hzp8/i549e+PKlcuYMeM/5Z7XuXMIOncO0TqGI0cO4v33ZyAgIAjOzs44c+YkBg58Q+vX//PPbahUSoSEdC137oMPPqrwNUFB7XHixHmt3j85+Rl4PCs0blzavj4+vjh37ky556rVaggENqxjHA4HycnPoFKp4OnZkHXex8cXT5481uozevfuh3379lQ6RKYLZpM4VDqc41CrGXBfuUurT5uGSMqWoHebhmhChXxES3/eTtZL0gAAqUKFP28na504pk9/DxwONN/sQ0K64ptvtsDd3QNZWZlQKpVwcXGt8LWurm549CgOAFBQkA9n57otofPPP7chFovRtWsoAODNN0fgjz/21yhx5OfnwcHBEVZWurnMFRdLIRCwa3QEAhtIpdJyzw0N7YlPP52NN94IR0BAEI4f/xPPnj2FXC6HVFoMGxt2UrGxsUFmZrpWn9G6dVs8eHAPMpm0XHLSFbNJHAp12TmO+huqupeSi93n4/Fhv9ZoKnLQHOdyOHi3R8t6+xxiGQYFeem1x1GT3saWLTvRrFkLPH+egoUL56JBAye0besPAHB0bAAej4eMjHTNhG5ZaWmpcHUtSSouLq7Iyckp9xyVSgWJRAxHxwbVxnL48EHk5+dh2LBBL16rRH5+Ph4+fIDWrdvA2toaKlX5NlSpVJraGVdXNxQUFECpVJZLHgUFBRAKheWO3759C/Pmle8teXh44ocf9rKO2djYQCaTsY7JZFLY2grLvT44uAM+/vj/sHbtChQWFiIsbBBCQrrC3t6hwveRSkveR5vPcHV1g0qlQnZ2Nho1alzus3XBbBJHffc4Sgr5nuDcwzQAwI6zcVg2vL1RLmVCTMegIK9aDR3pU6NGjbF69Tq8++4YNGzYCBMnvgc+n48uXbrh2LH/oWPHzqzn5+bm4MqVy1iwYAmAkp7KuXNnEBY2iPW8y5cvYNmyxYiKioZQWHkls1gsxpkzJ7Fhw3esIZpvvlmHP/7Yh0WLlkIkckdGRjoYhmHdev/8eQpEIncAgL9/AKytrfD33xcRGtqL9RmrVy+HUCjE4sXLWMeDgoIRHX1Wq3by9vaGSqVEWloaPD09AQDPnj1F06ZNyz03Pz8PAQFB+PXXAwBKElxExBBMmDAZ9vYOSE1NhVwuB5/P17xPhw4dtfqMl+tucbn6Gyo3m6sgexOnuvU4YhMy8em+65qkAQCZhTKk5BbV6X0JMRWeng0xa9Yn2LVru+YOno8++gSXLl3Apk0bkJWVBaVSifv372LOnI8RGBiMnj37AADefXcKbt++ga1b/4uCgnyoVCrcuHENX365CmPGTKgyaQDA8eNH0bixNwIDg+Hq6qb5Jzz8TZw6FYO8vDy0bdsOQqEQmzd/i6KiIqjVajx8eB979/6sGc4SCAR4//2ZWLt2JS5dugClUomiIgl27tyGa9diMWbM+Dq1kVBoh9DQnti6dROkUikePLiHEyeOY8CA8sNpiYlPMHPmNKSmPodMJsWOHVvg5OSEtm3boUmTpmjSpCl27NgMuVyOy5cv4ubN6+jTp79Wn5GVlQkej6fp8ekDhzGTPUov3U3GsoO3AQDtGjth/pDAGr9HrqRkR75rT8oX8k3o3hzOdqaz5pBI5IDMzEJDh2FwhmyHtLSn8PT0NchnV6Sqxf1CQzvhxx/3olmzFqzjs2fPQEFBAbZt2w0ej4fU1OfYsWMLrl27gqKiInh6NsQbb4Rj5MgxrGGfpKRn2LbtO9y6dQNSqRSenp4YPnwkhg2LqDbOiRPfQf/+YRg//l3WcbVajREjwjF8+NsYP34SkpKe4bvvvsGdO7cgk8kgErnjrbdGYOTIMaxeSExMNH77bQ+SkpLA5XLh798OU6Z8AD+/1jVovYoVFOTjyy9X4dq1WNja2mLy5GkID38TAPDjj9/j9u1bWLduIwBgz56fsH//HkilxQgKao85cxZoekdpaalYu3YF7t37By4urpg5s2wdR+WfUfL7HcP//heFjRu3VBhjRX+HXC4Hrq61n5s1m8Rx7nYyVh4uSRzBPi6YM6id1q81x0I+ShwlKHGUsoRVYbVhbu2wePGn6NKlOyuZlKWLxGE+cxy1HKpKzy/G9rNx5Xbk69PGE6O7NKOaDEKI0crMzEBCwmMsXbpSr59rNldF9iZO2k/dyJQqPEov0Dz2cLTBe71aoW1j2pGPEF347be92Lbtv5We/+mn3zQTwaRqO3duxSefzNPZLceVMZ/EUcslR3xc7REe7IUjN5OokI8QPXj77dF4++3Rhg7DLMyfv8Qgn2s2iUOboSqZQoWnWWK0asi+j/zNDr54rZmIduQjhBAtmE3iKFtPVdFQ1b2UXOw89wgFxQqsHtkRbg6lFZZ8Ky4lDaIDHDCMGhyO2dz1TkwMw6gB1P/afWbzF122x1F2rSqJTIHtZ+Ow6sg/yCiQQqpQYddfj2AmN5MRI8bn2yAvLwtKpYL+3oheMQwDpVKBvLws8Pn1vwyJ2fQ4WAWAL5YciU3IxA8XHiO/SK45J+TzcY5hfwAADfJJREFU0KmZ6d1eS0yPs7MIYnE+cnLSoVbrZ32qqnC5XE2VsSWzlHbgcnmwtbWHvX31S7zUlPkkjjJLjihVamw4fq98IV9TN0wINa1CPmK6OBwOHByc4OBgHHfoUW1PCWqHutPLUNWRI0cwaNAgDBw4EL/88ku58w8ePMDw4cMRFhaGRYsWQams+ebqZe+qOnHvOStpNBDy8fHAtvg4rC0lDUIIqSOdJ4709HSsX78ee/bswaFDh7Bv3z7Ex8eznjN37lx89tlnOH78OBiGwf79+2v8OWXrOBRleh+9W3ti7ahO6EzDU4QQUi90njguXbqELl26wMnJCUKhEGFhYYiOjtacT0lJgVQqRXBwMABg+PDhrPPaenXu0f3FjnxTetOOfIQQUp90fkXNyMiASFS6yby7uzvu3LlT6XmRSIT09PQaf05Iaw8cvJYIAHijvQ/eH9gWAmvLLuQTldk/xJJRO5SitihB7VA3Ok8carWatVLlq+vnV3deWz4NbPHthC7wEDlAXiSHuKAY4rqFbtJcXe2RnW3JLVCC2qEUtUUJaoeSRQ6dnate3r4qOk8cnp6euHbtmuZxZmYm3N3dWeczMzM1j7OysljnteXsbFfaELb82gdsRuqy+qU5oXYoRW1RgtqhbnQ+x9GtWzdcvnwZOTk5KC4uRkxMDHr27Kk537hxYwgEAly/fh0AEBUVxTpPCCHEuOhlP44jR45g69atUCgUiIiIwNSpUzF16lTMmjULAQEBePjwIRYvXgyxWAx/f3+sWrVKs4UiIYQQ42I2GzkRQgjRD7NZq4oQQoh+UOIghBBSI5Q4CCGE1AglDkIIITVCiYMQQkiNUOIghBBSIyabOPSxVLspqK4dTp48iTfffBNDhw7Fhx9+iPz8fANEqXvVtcNLZ8+eRd++ffUYmf5V1xYJCQkYP348hg4divfee89i/ybu3buHESNGYOjQoXj//fdRUFBggCj1QywWIzw8HMnJyeXO1epayZigtLQ0pk+fPkxubi4jkUiYIUOGMI8ePWI9Z/DgwczNmzcZhmGYBQsWML/88oshQtWp6tqhsLCQ6d69O5OWlsYwDMNs2LCBWb58uaHC1Rlt/h4YhmEyMzOZ119/nenTp48BotSP6tpCrVYzAwcOZM6dO8cwDMN8+eWXzNq1aw0Vrs5o8zfxzjvvMGfPnmUYhmFWrVrFfP3114YIVedu3brFhIeHM/7+/kxSUlK587W5Vppkj0NfS7Ubu+raQaFQIDIyEh4eHgAAPz8/pKamGipcnamuHV5avHgxZs6caYAI9ae6trh37x6EQqFmWZ/p06dj7NixhgpXZ7T5m1Cr1ZBIJACA4uJi2NjU/97cxmD//v2IjIyscA3A2l4rTTJxVLRUe9ml2OtrqXZjV107ODs7Y8CAAQAAqVSKbdu2oX///nqPU9eqawcA+PHHH9G2bVsEBQXpOzy9qq4tnj17Bjc3NyxcuBDDhg1DZGQkhEKhIULVKW3+JubPn4/FixcjNDQUly5dwujRo/Udpl6sWLECnTp1qvBcba+VJpk49LVUu7HT9vcsLCzEtGnT0Lp1awwbNkyfIepFde0QFxeHmJgYfPjhh4YIT6+qawulUonY2Fi88847OHjwILy9vbF69WpDhKpT1bWDVCrFokWLsHv3bly4cAFjxozBvHnzDBGqQdX2WmmSiePVpdh1tVS7sauuHYCSbxRjxoyBn58fVqxYoe8Q9aK6doiOjkZmZiZGjBiBadOmadrEHFXXFiKRCL6+vggICAAAhIeHszZWMxfVtUNcXBwEAgECAwMBAKNGjUJsbKze4zS02l4rTTJx0FLtJaprB5VKhenTp+ONN97AokWLzLLXBVTfDrNmzcLx48cRFRWFbdu2wd3dHXv27DFgxLpTXVu0b98eOTk5ePjwIQDg9OnT8Pf3N1S4OlNdO/j6+iItLQ0JCQkAgFOnTmmSqSWp9bWy/ubu9evw4cPM4MGDmYEDBzLbtm1jGIZhpkyZwty5c4dhGIZ58OABM2LECCYsLIz55JNPGJlMZshwdaaqdoiJiWH8/PyYoUOHav5ZuHChgSPWjer+Hl5KSkoy67uqGKb6trh16xYzYsQIZtCgQczkyZOZrKwsQ4arM9W1w9mzZ5khQ4Yw4eHhzMSJE5lnz54ZMlyd69Onj+auqrpeK2lZdUIIITVikkNVhBBCDIcSByGEkBqhxEEIIaRGKHEQQgipEUochBBCaoQSByF6VlxcjKysLEOHoZGUlGToEIiJocRBjNLOnTvRsWNHdO/eHQqFotLnXblyBSEhIXqMrHbat2+Px48fAwDGjh2Lf/75BwBw+PBhgy4yeP/+fbzzzjsG+3ximqwMHQAhFdm3bx8WLFiAiIgIQ4dSL27evKn5OS8vT/Pz0KFDMXToUEOEBKBkHbOqEjMhFaEeBzE6YWFhePbsGT7//HN8/vnnkEqlWLp0KQYMGIDg4GAMHDgQJ0+eLPc6uVyOBQsWICQkBKGhoZg1axZyc3MBlCxq98UXX6BHjx4IDQ3FmjVrIJfLK/z8vn37YtOmTejduzc6duyIJUuWQCaTAQAkEgmWLVuG7t27o3v37li0aBEKCwsBAM+fP8eECRPQqVMn9O/fH2vXrsXL+lo/Pz/ExcVhxowZeP78OT7++GP8+OOPOHDgAIYPHw6xWIygoCA8evRIE8fvv/+OkSNHat57+vTpCAkJwcCBA/HHH39UGHtycjI6duyI+fPno1OnToiKikJSUhKmT5+OXr16ITAwEKNHj8bjx4+RnZ2NqVOnIi8vD+3bt0dubm6N2olYLkocxOgcP34cjRo1wjfffIPPPvsMO3fuxOPHj3HgwAFcv34dw4cPx/Lly8u9LioqCo8fP8aZM2dw4sQJ/H97dxfS9BoHcPyby6Vb6v5zJDVGUWbSRUFCSo1WI8qQ2eom6UXqIkJ6I8ws1NWksGw0WlljdRUJUWChLduFJAS1VCxKgm5GC3K11iQbS7dk5yIanaOe2rk5gs/n6s//5Xn5XTy//V/2PNFolBs3bgBw7tw5fD4f7e3ttLe3MzAwgNPpnLQNbreb1tZWPB4Pr1+/5tKlSwBYLBZ8Ph8dHR08ePCAUCiExWIBwG63U1BQQE9PDzdv3sTtdvP06dO/ldvS0pLsW2VlZXL/7NmzMRqNuN3u5L779++zefPm5Jxjixcv5vHjxzgcDux2O16vd8K2RyIRtFotT548YcOGDdTX17Nw4UK6urrwer1IkoTT6SQ3N5dr166hUql4/vw5kiSlHCdhehKJQ5jyduzYgcPhQKFQEAgEUCqVE64ZkJWVhd/v5+7duwwNDeFyuTh8+DCJRIK2tjaOHj2KJEmo1WoOHjzI7du3J62zqqoKrVaLRqOhqqoKt9vNyMgIHo+Hmpoa1Go1OTk51NbW0tnZybdv38jKyqK3txePx4NCoeDRo0esWrXqj/tpNpvp7OwEfszm2t/fz6ZNm3j16hWBQIAjR44gl8spLCykoqKCO3fuTFqWyWRCLpeTmZnJ2bNnOXToEGNjYwwODqJSqSaM33+JkzA9iXccwpT39etXrFYrL1++RKfTodPpmGiKtdLSUsLhMG1tbZw5c4aCggIaGxvRarWMjIywa9eu5AzBiUSCeDzO6Ogos2bNGlfW/Pnzk9t5eXl8+vSJ4eFh4vE48+bNSx7TarUkEgmCwSA1NTU4HA4uXLhAdXU1a9as4fTp02g0mj/qp16vJxKJMDAwQF9fH6tXr0atVuP1eolEIqxcuTJ57tjY2L/OavtrnT6fj/Pnz/Px40fy8/OZMWPGhPELh8Mpx0mYnkTiEKa8kydPsmjRIpxOJzNnzqS3tzf5y/xXb9++paSkhO3btzM0NERLSwvHjh3D7XaTnp7OvXv30Ol0AESjUUKh0KSDYTAYTG4PDg4yd+5cNBoNcrmc9+/fo1argR/vFNLS0pAkiTdv3rB3715qa2t59+4ddXV1OBwOGhsb/6ifMpmMsrIyHj58SF9fH7t37wZ+rF6Xl5dHd3d38txQKDTh4P/Tz4E/Fotx4MABmpqaKC0tBeDy5cs8e/Zs3DUqlSrlOAnTk3hUJUx5kUiEjIwMZDIZgUCAixcvAoz7Gqirq4vq6mpCoRA5OTkolUpUKhUymQyTyYTNZmN4eJhoNIrFYuH48eOT1ulyufj8+TPBYBCn04nZbCYtLY3y8nJsNhvhcJgvX77Q3NyMwWAgOzubq1evYrPZGB0dJTc3F5lMhiRJ48pOT08nEolMWK/ZbKajowOfz4fRaARg+fLlZGRkcP36deLxOB8+fGDPnj20trb+NnY/7xYyMzMBePHiBbdu3UrGTi6XE4vFiMVi/ylOwvQkEocw5Z04cYLu7m5WrFjBzp07MRgMKBSK5P8ifqqsrGTZsmWYTCaKioro7++nqakJgLq6OiRJoqysDIPBQCQSwW63T1pnYWEh27Ztw2QyUVJSwr59+5JtWbBgAeXl5axfvx5Jkmhubgbg1KlTBINB9Ho9a9euZc6cOcnrfrVlyxYaGhq4cuXKuGNLly4lOzubjRs3IpfLgR+JxuVy0dPTg16vZ+vWrRQXF7N///7fxk6pVGK1Wqmvr6eoqAir1UpFRQV+v5/v37+zZMkS8vPzKS4uxu/3pxwnYXoS63EIwj8YjUYaGhpYt27d/90UQZiSxB2HIAiCkBKROARBEISUiEdVgiAIQkrEHYcgCIKQEpE4BEEQhJSIxCEIgiCkRCQOQRAEISUicQiCIAgp+QsIhy9IdMKPxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.998027 using {'C': 5, 'kernel': 'rbf'}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled SVM\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "c_values = [3,4,5,6]\n",
    "kernel_values = ['linear', 'poly', 'rbf', 'sigmoid']\n",
    "param_grid = dict(C=c_values, kernel=kernel_values)\n",
    "model = SVC()\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.944695 (0.018570) with: {'C': 3, 'kernel': 'linear'}\n",
      "0.991301 (0.010200) with: {'C': 3, 'kernel': 'poly'}\n",
      "0.997765 (0.003786) with: {'C': 3, 'kernel': 'rbf'}\n",
      "0.784240 (0.047821) with: {'C': 3, 'kernel': 'sigmoid'}\n",
      "0.946029 (0.018101) with: {'C': 4, 'kernel': 'linear'}\n",
      "0.991741 (0.010219) with: {'C': 4, 'kernel': 'poly'}\n",
      "0.997917 (0.003807) with: {'C': 4, 'kernel': 'rbf'}\n",
      "0.758505 (0.033167) with: {'C': 4, 'kernel': 'sigmoid'}\n",
      "0.947033 (0.017692) with: {'C': 5, 'kernel': 'linear'}\n",
      "0.991576 (0.010402) with: {'C': 5, 'kernel': 'poly'}\n",
      "0.998027 (0.003678) with: {'C': 5, 'kernel': 'rbf'}\n",
      "0.760038 (0.047344) with: {'C': 5, 'kernel': 'sigmoid'}\n",
      "0.947378 (0.017744) with: {'C': 6, 'kernel': 'linear'}\n",
      "0.991671 (0.010308) with: {'C': 6, 'kernel': 'poly'}\n",
      "0.998013 (0.003773) with: {'C': 6, 'kernel': 'rbf'}\n",
      "0.764321 (0.048646) with: {'C': 6, 'kernel': 'sigmoid'}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_['mean_test_score']\n",
    "stds = grid_result.cv_results_['std_test_score']\n",
    "params = grid_result.cv_results_['params']\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=5, cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "    kernel='rbf', max_iter=-1, probability=True, random_state=7, shrinking=True,\n",
       "    tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = SVC(random_state=seed, probability = True, C=5, kernel='rbf')\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9200934579439252\n",
      "\n",
      "Accuracy\n",
      "0.9529914529914529\n",
      "\n",
      "Confusion Matrix\n",
      "[[212   2]\n",
      " [  9  11]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.96      0.99      0.97       214\n",
      "         1.0       0.85      0.55      0.67        20\n",
      "\n",
      "    accuracy                           0.95       234\n",
      "   macro avg       0.90      0.77      0.82       234\n",
      "weighted avg       0.95      0.95      0.95       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3iT9RbA8W/SvaArpYxS9i4bWWULKBaUISJTULyoiBcviAIyVYYgQ5TtQEVAZIqUIgKijLKRZYEyChS66Eh3mvf+UQmEFpoW0rTN+TyPz9N3n/wM78k7zu+nUhRFQQghhDCR2tIBCCGEKF4kcQghhMgXSRxCCCHyRRKHEEKIfJHEIYQQIl8kcQghhMgXW0sHIMTjGjRoEKGhoUbzbGxsKFWqFI0aNWLMmDFUrVrVaHl8fDwrVqxg586d3Lx5k9KlS1OvXj2GDBlCy5Ytcz3Ob7/9xo8//sjZs2dJS0vD39+fF198kb59+2JnZ2e2zydEUaOSOg5R3A0aNAidTse4ceMM8zIyMjh//jxffPEFTk5O7NixAwcHBwCuXLnC0KFD0el0DB06lLp16xIfH8+mTZv4/fffGTlyJG+//bbRMaZOncqaNWt44YUX6NSpE87OzoSGhvLVV1/Rrl075s+fj42NTaF+biEsRhGimBs4cKDy+uuv57ps7dq1So0aNZQ9e/YoiqIoOp1OCQoKUjp37qzExsbmWH/+/PlKjRo1lF27dhnmbdy4UalRo4ayZs2aHOtv27ZNqVGjhrJx48Yn9GmEKPrkGYco0VxdXY2md+/eTVhYGGPGjMHT0zPH+iNHjqRixYosWbLEMG/lypXUrFmTl156Kcf63bp1Y9iwYXh4eDwyjrVr1/Lcc89Rv359nnnmGdatW2dY1rFjR6ZNm2a0/scff0zHjh0N0zVr1mTJkiU899xzNG/enODgYGrWrMnx48eNtvv+++9p0KABycnJAJw+fZohQ4bQoEEDWrRowfTp00lNTX1krELkRZ5xiBJBURR0Op1hOj09ndOnTzNv3jzKlStH06ZNAfjrr79Qq9UEBgbmuh8bGxs6derE119/TVxcHDqdjrCwMF5//fWHHvv+W2S5+frrr5k1axavvPIKbdu2JTQ0lA8//BBnZ2eCgoJM/oyLFi1i/PjxeHh40LRpU3x9fQkODqZRo0aGdbZv306HDh1wcXHh4sWLDBw4kIYNGzJ//nxiY2OZO3cu169fZ+nSpSYfV4gHSeIQJcLevXupW7eu0TxHR0datmzJBx98gIuLCwA3btzAw8MDZ2fnh+6rQoUKAERGRpKVlQVAuXLlChSXXq9nyZIl9OrVi/fffx+AVq1aERERwdGjR/OVOFq3bk3//v0N0926dWP79u28//77qFQqoqKiOHbsGJ9//jkAX375JV5eXixbtgx7e3sAKlWqxIABAzh8+DDNmjUr0GcSQhKHKBGaNGnCBx98AMCFCxeYNWsWLVu2ZPbs2YaTJmRfmeT1EPv+5Xf/1uv1BYrr8uXLxMfHG912Apg7d26+9/Xgm2FBQUF89dVXnDx5koYNGxIcHIyrqytt27YF4NChQ3Tq1Am1Wm24GmvYsCGurq4cOHBAEocoMEkcokRwc3MjICAAgICAAMqWLcvQoUOxt7dn9uzZhvXKly/PgQMHSE9PN7xl9aAbN24AULZsWcO8yMjIhx47KioKb29v1Oqcjwzj4+MBcn2ekl9eXl5G03Xr1qVy5coEBwfTsGFDtm/fTufOnQ2JMj4+nrVr17J27doc+4qOjn7seIT1kofjokRq2bIlffr0YfPmzfz++++G+R06dCAzM5Pdu3fnup2iKPz+++8EBATg6emJp6cnderUYd++fQ891tChQxk6dGiuy9zc3ACIi4szmn/58mWOHTtmmH7wiiYlJeXRH/BfQUFBhISEcPv2bY4fP25068vV1ZXevXuzfv36HP/95z//MWn/QuRGEocosd59913c3NyYOXMmGRkZAAQGBhIQEMDs2bNz/dW9dOlSLl26ZPQwfMiQIZw/f56ffvopx/qbN2/m4sWLdO/ePdcYqlSpgru7O3v27DGav2DBAsOVkKurK1FRUYZler0+x9tSDxMUFMSNGzdYvHgx3t7eNG/e3LCsSZMmhIeHU69ePQICAgxXYnPnzuXChQsm7V+I3NhMmTJliqWDEOJxbNy4ETs7uxwnbycnJ1QqFdu3b8fNzY3GjRujVqtp0aIFW7du5fvvv0etVpORkcG5c+dYuHAh3333HSNGjGDgwIGG/dSsWZNLly6xYsUKbt++jVqt5urVq/zwww8sWLCArl27Mnr0aFQqVY7Y1Go1NjY2rFixgvT0dAA2bNjAmjVrmDhxIlWqVOHGjRts2rSJUqVKkZKSYjix29vbM2TIECD7jarWrVvTuHFjo/3fTUp79+6lZ8+etG/f3rDM39+fL7/8kn/++QdnZ2cuXLjApEmTuHLlCqNGjcrxqrIQppLKcVHsDRo0CGdn51xfMc3IyOCZZ54hMTGRkJAQw7OGhIQEVq1aRXBwMNevX8fFxYX69eszePBgWrVqlWM/er2edevWsWHDBq5evUpGRgaVK1emb9++9OnTB1vbRz8u/OGHH/j222+JjIykUqVKvPHGG3Tr1g0ArVbL9OnT2blzJ7a2tvTo0YNy5crx/fffG26z1axZk/fee49XX301x76/+eYbZsyYwdq1a2nYsKHRsqNHjzJ//nxOnTqFg4MDjRs35n//+x/Vq1c3rXGFyIUkDiGEEPkizziEEELkiyQOIYQQ+SKJQwghRL4UWuLQarUEBQVx/fr1HMvOnTtHr1696Nq1KxMmTDDqc0gIIUTRUiiJ4+TJk7z88stcuXIl1+Vjx45l0qRJ7NixA0VRjHoOFUIIUbQUSpcj69atY/Lkybz33ns5lt24cYO0tDTDa4S9evVi4cKFRp25meLOnWT0egUvL1diY7VPJO7iTNohm7TDPdIW2aQdQK1W4eHhUuDtCyVxfPzxxw9dFhUVhUajMUxrNBpu376d72Po9Qp6vWL4W0g73CXtcI+0RTa9XuHXk9fZcOQqaZlZlg6n0JUp7cSqUR3zXvEhLN7JoV6vN6q4VRQl1wrcvHh53auC1WjcnkhsxZ20QzZph3ukLbJpNG5sPGqdSeNJsHji8PX1NeozKCYmBh8fn3zvJzZWi16voNG4ER2d9CRDLJakHbJJO9wjbZHtbjukZkjSKCiLJ47y5cvj4ODA0aNHadKkCZs3bzaMJyCEEIXh+xEl75xzKz6VFXvDOB+ZYDS/Q21fBrau9lj7tljiGD58OKNGjSIgIIA5c+YwceJEtFotdevWZfDgwZYKSwghijVdlp5fT11n45GrZGbde6ZVppQjr7arQZ3y7qjV+X8ccL9CTRz3j4uwfPlyw9+1atVi/fr1hRmKEEKUOJejk1ixJ4yrscmGeWoVdGvgR6+mFbG3ffTol6ay+K0qIYQQT8amo9eMkkYlb1dea1+DSt5Ptgt9SRxCCFFCDA6sxpkb8WTpFXo38+fZ+hWweczbUrmRxCGEEMVQcnomdjZqo9tPXq4OvNmpFuU8nPEt7WS2Y0snh0IIUcyEhkfz3tqjbDx6LceyxpW8zJo0QK44hLAoa65eFvl3Jzmdb/+8yJHLsQBsOxFBi6oa/J/wM4y8SOIQwoIkaVieo92TedPInBRFYc+5W/x4MJyU+woXSznZo03PLPR4JHEIYUGSNCzL0c6GXk39LR3GI91KSGXl3jDO3cxZyNevRRVcHAr/NC6JQ4giwtzVy9LlSLbi0g5Z93XEmJmlN8z3Le3EsLbVqVPe3WKxSeIQQogi5k5yOnO2n+FqzL3u39UqeK6hHz2bPLlCvoKSxCGEEEVMKSd77q++MFchX0HJ67hCCFHE2KhVDG9fA0c7G/q1qMzUXo2KTNIAueIQQgiLSk7X8fvZSJ5rWAH1fWMR+Xu7snBgc5wt8PA7L0UvIiGEsBKHw2P45s+LJKRk4GRvw9N1yxktL4pJAyRxCCFEoXuwkA9gzcHLNK+iwc3JzoKRmUYShxBCFJKHFfKVdrbnlcBqxSJpgCSOfJMuIoQQBfGwQr72tXx5uaVlCvkKqvhEWkRI0hDmUBy6vRAF87BCPp9Sjrz274h8xY0kjnySpCGetOLQ7YUouG0nIlgXesUwnT0iXwV6NfW3eCFfQUnieAxFeYD74tKtgrlJOwhL61yvHLvORhKrTcffy4XX2tegssbN0mE9FkkcQgjxBOn1Cur7Rt1zsrdlWNvqXItN5tn65bG1Kf5115I4hBDiCUhO1/HjgXAydFm8+XRto2UNKnrSoKKnhSJ78iRxCCHEYzp8OYZv910kPiUDgJbVfWjk72XhqMxHEocQQhTQneR0Vv15icOXY4zmn7p2RxKHEEKIexRFYe/5W6w+cJmUDJ1h/t1CvmZVvC0YnflJ4hBCiHy4lZDKV3vDOJtrIV9lXByKR/X345DEIYQQJtAr2YV8Px/OWcj3arvq1C3vYcHoCpckDhPdrfwUQlgnFRB2K9GQNO4W8vVs4o+DlVX+S+Iw0YNdjUgXEUJYF5VKxSuB1Th3Mx4fN8cSUchXUJI4TPRg0pAuIoQo2f6JTKCyxtWoWxBPVwcm9GhABQ/nElHIV1CSOApgxautLR2CEMJMktN1rDkYzu5ztwhq6Ee/FpWNlhelIVwtRRKHEEL868FCvl9PRtC8qrfV3pJ6GEkcQgirF5+Swbd/XuRwuHEhXyN/L0o72VsoqqJLEocQwmo9tJDPyY4h/xbyqVSqR+zBOkniEEJYpdsJqaz84wJnb8QbzW9Xqwwvt6iCq2PJL+QrqEJJHFu3bmXx4sXodDqGDBnCgAEDjJafOXOGSZMmkZmZSdmyZfn0008pVapUYYQmhLBC4VFJfLTlJBk640K+YW2rU6+C9RTyFZTZ3ye7ffs28+bNY/Xq1WzatIm1a9dy8eJFo3U+/vhjRo0axZYtW6hcuTIrV640d1hCCCvm7+1KeQ9nAFT/FvLNeLGJJA0Tmf2KY//+/bRo0QJ39+xxdbt27UpwcDAjR440rKPX60lOTgYgNTWV0qVLmzusHO5WhsvQsEKUfDZqFa+1q8GKvWEMbVOdKj7y1lR+mD1xREVFodFoDNM+Pj6cOnXKaJ3333+fYcOG8cknn+Dk5MS6devyfRwvr3vvVmsK8OrcxqOmJQ0ne5sC7d8Sikuc5ibtcI81tsWpq7EEH49gzPMNUP/7oFujcUOjcaNJLV95+F0AZk8cer3e6H+MoihG02lpaUyYMIFvvvmG+vXr8/XXXzNu3DiWLVuWr+PExmrR65UCjzGdmpF30nC0s6FnE/9iMYa1jLWdTdrhHmtri/sL+QAqlHaic71yVtcOuVGrVUY/tvPL7InD19eXI0eOGKajo6Px8fExTIeFheHg4ED9+vUBeOmll1iwYIG5w3qk70e0tejxhRCP58jlGL65r5APYOvxa3So7WvBqEoOsz8cb9WqFQcOHCAuLo7U1FRCQkJo2/beidnf359bt24RHh4OwK5duwgICDB3WEKIEig+JYOFIWeZv+OsUdJoUsmLKT0bWXX/Uk+S2a84ypQpw+jRoxk8eDCZmZn06dOH+vXrM3z4cEaNGkVAQAAzZszgv//9L4qi4OXlxSeffGLusIQQJYiiKPzxz21+2B8uhXyFoFDqOLp370737t2N5i1fvtzwd7t27WjXrl1hhCKEKGGkkK/wSeW4EKJY++3MTaOkIYV85ieJQwhRrPVuVonQ8BjiktN5tn4Feje1vhH5CpskDiFEsZGh05OZlYWLw73bT452NrzRsSb2tjZSyFdIJHEIIYqF8zfjWbH3Av7errzdubbRslrl3C0UlXUqsYlDuhARomRISdex5tBlfj8bCcCthFRaVfehSSUvC0dmvUps4iho0nCUe6NCFBlH/y3ku3NfTYajnQ1p971yKwpfiU0cBU0avZr6myEaIUR+JPw7Il/oAyPyNa7kxSuB1fB0dbBQZAJKcOK4n3QhIkTxcLeQb/WBcJLT711VlPq3kO8pKeQrEqwicQghir70zCzmBZ/h9AOFfG1qlmFASynkK0okcQghigR7W7VR/YXGzZFX20khX1EkiUMIUSSoVCpeaVONfyITaFOzDL2bVZKXVYqofCeOjIwM7O3tzRGLEMJKZOj0hPx9g871yhldZXi4ODC3/1O4OMhv2qLMpD6GdTodixYtok2bNjRv3pyIiAgGDhxIdHS0ueMTQpQw5yMTmLD+KGsOXebnI1dzLJekUfSZlDjmzp1LaGgoc+fOxdbWFo1Gg6+vL5MnTzZ3fEKIEiIlQ8fXf1zgo80niYxPBWD7qetExCVbODKRXyal9m3btrF582Y8PDxQqVQ4Ojoybdo0OnToYO74hBAlwLErsXy97wJ3ko0L+fq1qEx5D2cLRiYKwuRrQhsb44dUWVlZODhIEY4Q4uESUjJY9dclDl0yvq3dyN+TV9pUx0sK+YolkxJHly5dGD16NGPHjkVRFK5fv86cOXPo1KmTueMTQhRDiqKwLyx7RL4HC/kGt65G86pSyFecmfSMY+zYsVSsWJF+/fqRlJREt27dcHFxYcyYMeaOTwhRDIWGx7Bsd5hR0mhTswyzXmpKi2oaSRrFnElXHGlpaUyePJnJkycTFxeHu7s7arWaiIgIXFxczB2jEKKYaVbZmyoaN8Kjk9C4OTCsbQ0C/KSQr6Qw6Yrj/ofgnp6eqNVqdDodL7zwgtkCE0IUH3q9YjStVqt4rX11nq1fnhl9m0rSKGEeesVx/fp1+vXrR1ZWFikpKbRs2dJoeXp6OtWqVTN7gEKIoitDp2fT0av8cyuBCd0boFbfuwVV0cuVAa1cLRidMJeHJo4KFSqwfPlyEhMTef3111mwYIHRcnt7e2rWrGn2AIUQRdP5yARW7g0z1GTsPHOTrgHlLRyVKAyPfMZRu3b28Iy7d+/G09Mzx3K9Xm+eqIQQRVZKho61By+z698R+e46eS2OLvXKyYNvK2DSw/GEhARmzZrF7du3DclCp9Nx9epV/vrrL7MGKIQoOh5VyNexTllJGlbCpMQxfvx4PDw8KFeuHDdu3OCpp55izZo19O/f39zxCSGKACnkE/cz6a2qc+fO8dlnnzFs2DAA3nrrLT7//HN27dpl1uCEEJb3Z9ht3lt7xChplHK0Y+TTtXj3mbqSNKyQSVcc7u7uODg4ULFiRS5evAhAw4YNiYiIMGtwQgjLuxKtNS7kq1GG/q2q4CYj8lktk644ateuzWeffYaiKPj4+LBz50727duHk5OTueMTQlhYn6cqoXFzQOPmwLjn6vGfjjUlaVg5k644JkyYwMSJE4mPj2fMmDG8/fbbZGRkMGXKFDOHJ4QoTBGxybg52eHufG+wNkc7G8Y8Ww8vN0cZkU8AJiaO48ePs2TJEhwdHSlTpgyHDh0iMzMTZ2fpDlmIkiBDp2fzsWv8ciKCJpW8GNWljtHy8p7StZC4x6RbVdOnT8fW9l6OsbOzk6QhRAnxT2QCE9cfZfOxa2TpFULDYzhyOcbSYYkizKQrjk6dOrF06VK6d++Ot7dxd8jynEOI4iklQ8e6Q5f57YxxIV8N31KUdZcfhuLhTEocO3fuRKvVsmjRIsM8RVFQqVScO3fObMEJIczj+NVYvv7jAnEPFPK91LwyneqWRS2FfOIRTEocmzdvNnccQohCkJCawXd/XeLgReNCvoYVPRnathpero4WikwUJyYljvLlH6/jsq1bt7J48WJ0Oh1DhgxhwIABRsvDw8OZPHkyCQkJaDQaPvvsM0qXLv1YxxRCGItJTGPcmiNo7x+Rz9GOQYFVaVFVBlcSpjPp4fjjuH37NvPmzWP16tVs2rSJtWvXGooIIfuW1xtvvMHw4cPZsmULtWvXZtmyZeYOSwir413Kkdrl3Q3TgTV8mNWvKS2r+UjSEPli0hXH49i/fz8tWrTA3T37C9u1a1eCg4MZOXIkAGfOnMHZ2Zm2bdsCMGLECBITE80dlhBWaXDrqkQlpvFS80rU98vZ47UQpjB74oiKikKj0RimfXx8OHXqlGH62rVreHt7M378eM6dO0eVKlX48MMP830cL697A8ZoNG5Gyx6cthbW+rkfZI3tcPl2Ij/su8iYHvVxtL/3z7xGJW+Wjmhr9VcY1videJJMThyhoaGsXbuW6Oho5s2bxw8//MBbb72Fjc2jK0n1er3Rl/Tu21h36XQ6QkND+f777wkICGD+/PnMnDmTmTNn5uuDxMZq0esVNBo3oqOTjJY9OG0NcmsHa2Rt7ZCZlV3It/V4BFl6BTd7G/q3rAJYX1s8jLRD9tC+9//Yzvf2pqy0YcMGxowZQ+XKlTlz5gwqlYqdO3cye/bsPLf19fUlOvreGxzR0dH4+PgYpjUaDf7+/gQEBAAQFBRkdEUihDBNWGQCE346yqaj2YV8AL+duUlCakYeWwqRPyYljqVLl7J8+XJGjhyJWq3G09OT5cuXs23btjy3bdWqFQcOHCAuLo7U1FRCQkIMzzMAGjVqRFxcHOfPnwfg999/p27dugX8OEJYn9QMHd/uu8j0zSe5+e8wrpBdyDe9d2NKO9k/Ymsh8s+kW1Xx8fFUq1YNwHCbydvbm8zMzDy3LVOmDKNHj2bw4MFkZmbSp08f6tevz/Dhwxk1ahQBAQF88cUXTJw4kdTUVHx9fU26khFCSCGfsAyTEkfjxo1ZuHAho0ePNsz79ttvadiwoUkH6d69O927dzeat3z5csPfDRo0YP369SbtSwgBif8W8h2QQj5hASYljkmTJjFixAjWrFmDVqulY8eOODo6snTpUnPHJ4TIxcFL0UZJw83RjkGtq9KymhTyCfMzKXGULVuWjRs3curUKSIjI9FoNDRs2NCox1whROF5uk45/gqL4lJUEoE1fBjQsipuTjK4kigcJp35+/Xrx/PPP8+zzz5r8u0pIcSTodcrJKVnGj3kVqtVDG9fg1htOg0qSiGfKFwmvVXVp08fQkJCaN++PW+99RYhISEmPRgXQjyeiNhkpm46wWfbz6D/9xXbuyp4ukjSEBZhcuL4+uuv2blzJ0899RRfffUVgYGBTJ482dzxCWGVMrP0rD98hYk/H+NSVBKXopIIOX3D0mEJAeSzyxGNRkOzZs2IiYnh1q1bnDhxwlxxCWG1wiITWLE3zKgmw1atQvfAFYcQlmJS4rh06RLbtm1j27ZtJCcnExQUxOLFi6ldu7a54xPCaqRm6Fh36Aq/nbnJ/SmieplSvNauuoz7LYoMkxJH79696dixI+PHj6dNmzao1WbvjV0IqyKFfKI4MSlx/Pnnn7i6FrxDLCFE7hRFYcnv//DXhSij+Q0qejK0TTW83aSQTxQ9j0wcgwYN4rvvvmPIkCEPLSqSim8hCk6lUlHqvvoLKeQTxcEjE0ffvn0BGDhwYKEEI4Q16t2sEkevxFK9TCkGtpJCPlH0PTJx3O1fKjk5OdfksWTJEvNEJUQJpNcr/Hb2Js0qe+Ph4mCY72hnw/TejXFxkJ4YRPHw0G9qbGwsp0+fBmDOnDn4+fkZLddqtSxdupQRI0aYN0IhSoDrccks3xPGpagkzt1I4J2udYyWS9IQxclDv60uLi4sXLiQO3fukJ6eztSpU42W29vbS9IQIg+ZWXq2HLvGln9H5AM4fDmGE1fjaOgvVd+ieHpo4nB0dOTnn38GYOTIkSxatKjQgnoc6w+E893eMEuHIQRhtxJYsfcCN++kGObZqFU837gi9Sq4WzAyIR7PI6+Pw8PDqVKlCv/973+5ePFiruvcHeCpqPj+jzDSMrMM0452jx4TXYgnLTVDx7rQK/x2Wgr5RMn0yMTRp08fjh07RlBQUK7LVSoV586dM0tgBZWaYZw0ejX1t2A0wtqcuBrH1/suEKtNN8xztLOhb/NKPF23nBTyiRLhkYnj2LFjAIbxwIubFa+2tnQIwoqERyUxZ/tpo3kN/DwY2ra6FPKJEsXkvkMOHDgAQFxcHFOmTGHBggWkpaWZLTAhipsqPm48VcUbyC7ke7NTLcZ0qydJQ5Q4Jr0D+Omnn/Lrr7+ye/duJkyYgFarxdbWlilTpjBz5kxzxyhEkaTXK6jVxreeBgdWw8XBlr5PVZZCPlFimZQ4du7cyZo1a0hKSuKPP/4gODgYT09POnbsaO74hChy9HqFnWdusvtcJFN6NjJ6AcPd2Z5X29WwYHRCmJ9Jt6ri4+MpU6YMf/31FxUqVMDPzw87OzsURcYHENblelwy0zaf4Lu/LnE9LoX1oVcsHZIQhc6kK47atWvzySefcPjwYZ5++mni4+P59NNPadCggbnjE6JIyK2QD+D0jTtk6PTY28pQA8J6mPRtnzlzJvHx8TRu3Ji3336bq1evEhsby0cffWTu+ISwuAu3Epm4/hgbj14zJA0btYqeTSoyvXdjSRrC6ph0xVG2bFlmz55Neno6UVFRVK1aVTo4FCVeaoaOn0KvsPOBQr5qZdx4rV0NKkghn7BSJiUOrVbL5MmT2b59O3q9HhsbG9q3b8+sWbNkgCdRIp2KiGPlXuNCPgdbNX2bV6Zz3XI53qYSwpqYfKsqOTmZbdu2cerUKbZs2UJWVhaffPKJueMTwiJuxacaJY36fh7MeqkpXQPKS9IQVs+kK449e/awfft23NzcAKhatSqzZs2iS5cuZg1OCEt5um45DlyMJjIhhYGtqtK6uo+MyCfEv0weBCArK8toWq/XY2cnBU6i+ItJSiMjS085d2fDPLVaxRudauJoZ0MpJ3sLRidE0WPSrarOnTszevRozp07R3x8PGfOnOHdd9+lc+fO5o5PCLPR6xVC/r7BuLVHWLLrH/R647okn1JOkjSEyIVJiWPs2LF4enry4osv0qJFC15++WXKly/P2LFjzR2fEGZxt5Bv1V+XSNfpCY9OYsffNywdlhDFgkm3qpydnZk7dy6ffPIJiYmJeHt7y/1eUSw9rJCvnIcz1cqUsmBkQhQfJiWOjIwMFi1axC+//EJMTAxly5alZ8+evP7666jVUvwkiocLtxJZsTeMGw+MyNejkR89GlfEzka+y0KYwqTE8dFHH3H27FnGj/609q4AACAASURBVB+Pr68vN27cYOnSpWi1WsaMGZPn9lu3bmXx4sXodDqGDBnCgAEDcl1vz549TJs2jd9//z1/n0KIR0jLzGLdoctSyCfEE2JS4ti+fTs7duzA09MTgHr16tGoUSN69OiRZ+K4ffs28+bNY8OGDdjb29OvXz+aN2+eY8jZmJgYZs2aVcCPIUTu0jOzGP/TUaIS740dI4V8Qjwek67NXV1dycjIMJpna2trqOt4lP3799OiRQvc3d1xdnama9euBAcH51hv4sSJjBw50sSwhTCNg50NzSp7G6alkE+Ix2fSFUfv3r159dVXefPNN/Hz8yMqKorly5fTsmVL9u7da1ivXbt2ObaNiopCo9EYpn18fDh16pTROqtWraJOnTrS264wi15N/TkfmUCXeuVoJYV8Qjw2kxLHhg0bAJg7d67R/OjoaP78808AVCoVu3btyrGtXq83+oeqKIrRdFhYGCEhIXzzzTfcunUr/5/gX15eOfvM0mjyviIqyazt80clpLL8t3OM6FIHr/uGa61Qzp0vXm8jCQPr+048jLTD4zEpcTzOw2pfX1+OHDlimI6OjsbHx8cwHRwcTHR0NL179yYzM5OoqCj69+/P6tWr83Wc2FhtjgKu6OikAsdd3Gk0blbz+fWKwm9nbrLu0BXSMrNITsngv13roFKprKod8iJtkU3aIbtnhNx+bJu8/ROMJVetWrXiwIEDxMXFkZqaSkhICG3btjUsHzVqFDt27GDz5s0sW7YMHx+ffCcNYb1uxCUzfdNJVv15ibTM7G5xjl2J5VpssoUjE6LkMrmvqoIqU6YMo0ePZvDgwWRmZtKnTx/q16/P8OHDGTVqFAEBAeYOQZRAuiw9W49HsPnYNXQPFPK91q46/t7S3b8Q5mL2xAHQvXt3unfvbjRv+fLlOdarUKGC1HCIPF28nciKPWFcl0I+ISwi34kjIyMDe3vp+E0UvrTMLH4KvULI3zeMCvmq+rjxWvsa+EkhnxCFwqSfZjqdjkWLFtGmTRuaN29OREQEAwcOJDo62tzxCWHwT2QCO+5LGg62aga2qsrkFxpK0hCiEJmUOObOnUtoaChz587F1tYWjUaDr68vkydPNnd8Qhg0qOhJi6rZNUEBFTyY2bcpz9SXQj4hCptJt6q2bdvG5s2b8fDwQKVS4ejoyLRp0+jQoYO54xNWSlEUElIzcXc2vi06KLAqDf09ZUQ+ISzI5KeINjY2RtNZWVk4ODg88YCEiNWmMXf7GSZvOG54xfau0k72BNYoI0lDCAsyKXF06dKF0aNHc/78eRRF4fr163z44Yd06tTJ3PEJK6JXFHaevsm4tUc5cS2OWG066w5dtnRYQogHmDwCYMWKFenXrx9JSUl069YNZ2dnk7pUF8IUN+6kMH3TSb7982KOqwxFUR6ylRDCEkx6xuHg4MDkyZOZPHkycXFxuLu7ywBO4onQZenZeiKCzUcfKORzd+K1djWoUba0BaMTQuTGpMSxaNGihy6TrtBFQV28nT0i3/U440K+7o386NGoIva28uNEiKLIpMQRFhZmNB0fH8+JEydyVIMLYaqfQq+w5di1nIV87Wrg5yU1GUIUZSYljoULF+aY99dff0lnhKLA3J3tjQr5XnyqEl3qSU2GEMVBgfuqat68OW+//faTjEVYkU51y7L/QhSOdjYMa1sdTSnHvDcSQhQJJiWOixcvGk1nZmaydetWypcvb5agRMmhKAoHL0VTzt3ZqMdatUrFmG71cLa3kZoMIYoZkxJHUFAQKpXK8FqkWq2mUqVKfPjhh2YNThRvsdo0vv7jIieuxVFZ48qUno2wue9WlItDoXTOLIR4wkz6lxsSEkLFihXNHYsoIfSKwq4zkaw9dNlQk3E5WsuuszfpUk+uUoUo7kx63/Gll15Cq9WaOxZRAjyskK9z3XIE1ihjwciEEE+KSVccfn5+nDlzhubNm5s7HlFMSSGfENbDpMRhZ2fH0KFD0Wg0eHt7Gz3MXL9+vdmCE8XDowr5npcR+YQocUxKHH369KFPnz7mjkUUQ3eS0/lo80mjqwwp5BOiZDMpcSQnJzNw4MAc85csWfLEAxLFi4eLA8/Ur8AvJyKkkE8IK/HQxBEbG8vp06cBmDNnDn5+fkbLtVotS5cuZcSIEeaNUBQper2SIyn0alqRpLRMnm/sh08pJwtFJoQoLA9NHC4uLixcuJA7d+6Qnp7O1KlTjZbb29tL0rAiiqJw6FI0P4VeYXyPBni53hvEy97WhuHta1gwOiFEYXpo4nB0dOTnn38GsnvAfVQPuaJki9Wm882+Cxy/GgfAN/su8O4zdaXiWwgr9djdqouSK7dCPoArMdpcxwMXQlgH6fNB5OrmnRRW7A0j7Fai0fxOdcryUovKONvLV0cIayX/+oURXZaeX05EsOmBQr6y/xby1ZRCPiGsniQOYXA5Oollu8OIiEs2zLNRqwhqWIHnG/vLiHxCCEASh7hPfEqGUdKoonHjtfbVqejl+oithBDWRhKHMGjk70XLahqOXomlT7NKPBMghXxCiJwkcVgpbVom0UlpVNa4Gc0f1LoqLz5VSQr5hBAPJYnDymQX8sWw6q+L2Nmomdm3CU73vSFVyskeyRlCiEeRxGFFHizkA1gXeoUhgdUsGJUQoriRxGEF9IrC7rOR/HjQuJDPw8WeehU8LBiZEKI4ksRRwt2MT2Hl3jD+icylkK95ZZxl3G8hRD4Vyllj69atLF68GJ1Ox5AhQxgwYIDR8t9++43PP/8cRVGoUKECM2bMoHRpKTR7HLosPZuPXWPT0atkZt0r5PMt7cRr7WtQSwr5hBAFZPaKrtu3bzNv3jxWr17Npk2bWLt2LRcvXjQs12q1TJkyhWXLlrFlyxZq1qzJ559/bu6wSjRFURj3/SF+Cr1iSBo2ahXPN/bjkxebSNIQQjwWsyeO/fv306JFC9zd3XF2dqZr164EBwcblmdmZjJ58mTKlCkDQM2aNYmMjCzQsX49ef2JxFzcqVQq2tctZ5iurHFlWq9GvPhUZan+FkI8NrPfqoqKikKj0RimfXx8OHXqlGHaw8ODzp07A5CWlsayZcsYNGhQvo/j5eXKxqNXDdNO9jZoHqhRsCbPebtyIOw2Tap488JTlbBRW2/CsObvwYOkLbJJOzwesycOvV5vNG6Doii5juOQlJTEW2+9Ra1atejZs2e+jxMbqyU1494bQz2b+BMdnVSwoIsRbVomqw+E0yWgPJW873UNotG48d+na6FSqYiLTX7EHko2jcbNKr4HppC2yCbtAGq1Cq/H6ErI7D9DfX19iY6ONkxHR0fj4+NjtE5UVBT9+/enZs2afPzxx0/kuN0aVHgi+ymqFEXh4MVo3lt7hD/+uc2KPWFk3debLSADLQkhzMLsiaNVq1YcOHCAuLg4UlNTCQkJoW3btoblWVlZjBgxgmeffZYJEybIyc4Esdp0Pgs+w6LfzpGYmglkD650/GqshSMTQlgDs9+qKlOmDKNHj2bw4MFkZmbSp08f6tevz/Dhwxk1ahS3bt3i7NmzZGVlsWPHDgDq1av3xK48SpKHFvI52/NK2+o0qeRlweiEENaiUOo4unfvTvfu3Y3mLV++HICAgADOnz9fGGEUa1LIJ4QoKuRsU8TpsvRsO3k990K+dtWpVc7dgtEJIayRJI4iLjIhlZ8PX+Huc28btYrnGlTghSYyIp8QwjIkcRRxfp4uPNfQj63HI6isceW1djXw95YR+YQQliOJo4iJT8nA3dneaF7PJv54uznQvlZZbGREPiGEhcm9jiJCm5bJst3/MObHw8Rq04yW2duq6VSnnCQNIUSRIInDwrJH5Itm3L+FfGmZWXyz7yKKouS9sRBCWIDcqrKgOG063/55kaNXjAv37GzUZGbpsbe1sVBkQgjxcJI4LECvKOw+F8mag5eN+tfycLbnlTbVaFLZ24LRCSHEo0niKGSR8Sms3HuB85EJRvM71ilLPynkE0IUA3KWKkS/n43ku78u5ijke7VddWpLIZ8QopiQxFGIvFwdDElDrYLnGvrRs0lFeZZRgqWmJqPVxpOVpbN0KERFqdHr9ZYOw+KspR1sbGxxdXXHycnlie9bEkchalDRk1bVfYiMT5FCPiuQmppMUtId3N012NnZW7znZ1tbNTpdyT9h5sUa2kFRFDIzM4iPzx7S4kknD0kcZnL6+h0A6lXwMJo/tE017G1tpCbDCmi18bi7a7C3d7B0KMLKqFQq7O0dcHfXkJAQI4mjqEtOzx6Rb+/523i62DPzpaY4299rZid7aXJrkZWlw87OPu8VhTATOzt7s9wmlQLAJ0RRFEIvRfPemiPsPX8bgLjkDDYcuZrHlqIks/TtKWHdzPX9k5+/T8Cd5HS+2ZezkK9ZFW+eK+FD2AohrI8kjsegVxT2nLvFjwfDjQr53J3tGdKmGs2kkE8UE4GBTXFwcECtzr4JoSgK3t4aBg4cQlDQC4b1bty4zooVSzh8+BDp6en4+vrSrVsP+vZ9GRube28HxsREs3LlMg4e/Ivk5GR8fHzo2bMPvXu/lK+4pk//kF27drJ+/Va8vTWG+b/+upWff17HypXfGa1/7NgRPvxwHNu27TLMO3hwPz/++B0XLoShKAp16tRh+PA3qVWrTr5iyU1iYiIzZkzj2LHDuLi4MmzYcKP2ut+uXSGsXLmUmJgYateuy5gx7+PnVxGA8PBLzJ//KWFh53F2dqF79xd45ZXXUKlUZGRkMHfuTP74Yw+2trb06fMSQ4a8CsC3365Eo/GhW7fuuR7TXORWVQFFxqfwyZZTfPXHBaOk0aG2L7NeaipJQxQ7y5d/y86d+9i5cx87duxl2LDXmT37E65cuQzA1atXGD58CN7eGr75ZjUhIXuZMGEqv/8ewsSJ4wz7iY6OYtiwgZQqVYqvv17Njh17+OCDyaxe/R1ffbXM5HgSExPZv/8vOnR4mk2bfi7QZ9qyZSOffDKVvn37s2XLDjZt2s5TT7Vk1Kg3CA+/VKB93m/27I9wdnZiy5YQPvpoFl9++TkXL17Isd7p03/z8cdTePPNd/j111106tSZ0aPfIj09Db1ez/vvv0uzZi345Zff+PzzpWzf/gu//LIZgGXLvuTWrVv89NMWvvxyBb/8spk///wDgH79BrJ69XfcuXPnsT9LfkjiKIAMnZ6PNp80qv4uU8qR8d3r82q7GrhI9bco5mxsbOjS5VmcnV24fDn7BLtw4Wc8/XQX3nrrHby9NahUKmrVqs2cOQs5ceIYf/yxB4DlyxdTv34D3njjbdzd3VGpVNStW4/33/+Q2NgYk2MIDt5Gw4aN6NXrRbZs2UhmZma+PkNqaiqLFs1n3LiJtG7dBltbWxwcHBgwYBA9e/bh6tXLObY5efI4nTu3yfHfwIF9c6ybkpLCvn17GTbsPzg4OFCnTj06d+7K1q0bc6z7xx+7adOmHYGBbbG1taVHj544Ojpy+HAocXGxVKzoz4ABg7G1taV8+Qq0adOev/8+CUBIyHYGDx6Kq6srfn4V6dWrL1u2ZB/DwcGB1q3bsH79mny1zeOSM1wB2Nuq6dXMn6//uIhaBd0a+NGrqRTyibz9evI6G45cJS0zK++VH5OjnQ29mvrTrQDP2TIzM9m48Sd0ukzq1g0gPT2do0dDGTx4cY51S5d2p3XrNuzbt4e2bdtz6NAB3nrrvznWa9asOc2aNTc5hq1bN/Kf/7xFQEADPDw82L37N7p0edbk7f/++yRZWTqaN2+ZY9kbb7yd6zYNGjRi5859Ju3/+vVr2Nhkn+jvqljRn717d+dYV6/X4+DgaDRPpVJx/fo1AgPbMmfOQsP8zMxMDh06QI8ePUlMTCQuLpZKlSobHWPDhp8M0x06dGLcuHcZPvwNk+J+EiRxmECvV1A/UHfRoXZZImKTaV+7LJWkkE+Y6NeT1wslaQCkZWbx68nrJieOESNeRaXC8Mu+efOWLFiwBB+fMsTERKPT6fD09Mp1Wy8vby5cCAMgMTEBD4/H60Ln779PotVqadkyEIDnn+/Nzz+vy1fiSEiIx82tFLa25jnNpaam4eBgXKPj4OBIWlpajnUDA9vy3nujefbZIAICGrBjx69cu3aVjIwMo/UyMzOZMmU8dna2PP98TxISsu9qODreSzqOjo6kp987RrVqNUhIiCci4prhmYm5SeLIw5kbd/hm30Xe7FSLyho3w3y1SsUrbapbMDJRHHVrUKFQrzjyc7WxZMlKqlSpxs2bNxg/fiylS7tTp05dAEqVKo2NjQ1RUbdzPTnduhWJl1d2UvH09CIuLi7HOllZWSQnaylVqnSesWzZspGEhHh69uz277Y6EhISOH/+HLVq1cbOzo6srJxtmJWVZaid8fLyJjExEZ1OlyN5JCYm4uzsnGP+yZMnGDcu59VSmTK+fPut8e2g7BN4utG89PQ0nJycc2zfsGFj3nnnf8ye/TFJSUl07dqN5s1b4up675ySkBDP+PFj0el0zJ//JQ4Ojjg6pv+733RcXLJ/oKalpeHk5GTYztbWltKlSxMdHSWJw9KyC/kus/f8LQBW7Aljaq9G2NrIYyFRcN0aVCjQraPCVK5ceWbOnMsrr/SnbNlyDBnyKvb29rRo0Yrt23+hSZNmRuvfuRPHoUMH+OCDD4HsK5W9e3fTtWs3o/UOHPiTqVMnsnlzMM7OD69k1mq17N79G/Pnf2l0G2jBgrn8/PNaJkyYgkbjQ1TUbRRFMapVuHnzBhqNDwB16wZgZ2fLwYN/ERjYzugYM2dOx9nZmYkTpxrNb9CgIcHBe0xqJz8/P7KydNy6dQtfX18Arl27SuXKlXOsm5AQT0BAA378cQOQneD69OnO4MHDAIiMvMl//5v9ptf48ZMNVzKlSpXGw8OTa9euGq72rl27anTr6u7+7r4RVxjkLJiL0PBo3lt71JA0AKKT0rlxJ8WCUQlReHx9yzJq1Lt8/fVyw1tCb7/9Lvv3/8miRfOJiYlBp9Nx9uxpxox5h/r1G9K2bQcAXnnlNU6ePMbSpV+QmJhAVlYWx44d4dNPZ9C//+BHJg2AHTu2Ub68H/XrN8TLy9vwX1DQ8+zaFUJ8fDx16tTD2dmZxYs/JyUlBb1ez/nzZ1mz5nvD7SwHBwf+85+RzJ79Cfv3/4lOpyMlJZmVK5dx5Ego/fsPeqw2cnZ2ITCwLUuXLiItLY1z586wc+cOOnfOeTvtypXLjBz5OpGRN0lPT2PFiiW4u7tTp0490tPT+N//3qZZsxZMnfpJjttfXbo8y1dfLSMxMYGIiGts2LCOrl2fMyzPyMggKSnJkDALg0opIWOUxsZq6f/lXsP09yPa5nsfd5KzR+Q7cjlnId/g1lXxcCk+fQ5pNG5ERydZOgyLs2Q73Lp1FV9ff4scOzeP6twvMLApq1atoUqVakbzR49+i8TERJYt+wYbGxsiI2+yYsUSjhw5REpKCr6+ZXn22SD69u1vdNsnIuIay5Z9yYkTx0hLS8PX15devfrSs2efPOMcMuRlnn66K4MGvWI0X6/X07t3EL16vcigQUOJiLjGl18u4NSpE6Snp6PR+PDCC73p27e/0VVISEgwP/20moiICNRqNXXr1uO1196gZs1a+Wi93CUmJvDppzM4ciQUJycnhg17naCg5wFYteorTp48wdy52Q++V6/+jnXrVpOWlkqDBo0YM+YDNBofQkKCmTZtIo6OjkZxt23bng8/nE56ehoLF37G3r27UalUvPhiP8OVCsCpUyeYPn0yP/20OdcYc/seqtUqvLwK/mxWEgcls5BPEkc2SRz3WEOvsKYoae2waNF8HB0dee21EbkuN0fisPpnHLcTUlm+JyzHiHwdavvSr0UVqckQQhRZaWlp7Nu3h+XLvy3U41r9WTFdl8WF24mG6TKlHHm1XQ3qlJcR+YQwh59+WsOyZV88dPl33/1keNgsHu3HH79j0KChJr2p9iRZfeKo6OVKUMMKbD0eIYV8QhSCF1/sx4sv9rN0GCXC0KHDLXJcq0oc6ZlZXI3RUqOscXZ+vrE/T1XRyIh8QghhAqtJHGdu3GHl3gskpmYys28TvN3uVWLa26olaQgzUKEoelQqeetdWIai6IEnPyZHif9GJ6dnsnxPGDO2/k1UYhppmVl8/ccFSsjLZKIIs7d3JD4+Bp0uU75volApioJOl0l8fAz29o55b5BPJfqKIzQ8mm//vERCyr3+YJztbWhapfi9XiuKHw8PDVptAnFxt9HrC6d/qkdRq9Xo9SXnNdSCspZ2UKttcHJyxdX1yT84L7GJY/6OMzkL+Sp7MziweBXyieJLpVLh5uaOm1vReENPanuySTs8vkK5VbV161a6detGly5d+OGHH3IsP3fuHL169aJr165MmDABne7xB1e/P2mUdrbnnS51eKdrHUkaQgjxmMyeOG7fvs28efNYvXo1mzZtYu3atVy8eNFonbFjxzJp0iR27NiBoiisW7fuiR2/fS1fZr/UlGZye0oIIZ4IsyeO/fv306JFC9zd3XF2dqZr164EBwcblt+4cYO0tDQaNmwIQK9evYyWF5TPvyPyvdZeRuQTQognyexn1KioKDSae4PM+/j4cOrUqYcu12g03L59O9/H8fJyxcnehtSMLGzVKla82R4HO+su5NPcN36INZN2uEfaIpu0w+Mxe+LQ6/VGPT4+2H9+XstNdedOMgNbV2Pv+Uja1SqLNjEV7eOFXqx5ebkSG2vNLZBN2uEeaYts0g7ZnRx6eDy6e/tHMXvi8PX15ciRI4bp6OhofHx8jJZHR0cbpmNiYoyWm8rDw4U+gdXoE1gt75WtxOP0flmSSDvcI22RTdrh8Zj9GUerVq04cOAAcXFxpKamEhISQtu297o8L1++PA4ODhw9ehSAzZs3Gy0XQghRtBTKeBxbt25l6dKlZGZm0qdPH4YPH87w4cMZNWoUAQEBnD9/nokTJ6LVaqlbty4zZszA3t7e3GEJIYQogBIzkJMQQojCUeL7qhJCCPFkSeIQQgiRL5I4hBBC5IskDiGEEPkiiUMIIUS+SOIQQgiRL8U2cViiq/aiKK92+O2333j++efp0aMHb775JgkJCRaI0vzyaoe79uzZQ8eOHQsxssKXV1uEh4czaNAgevTowauvvmq134kzZ87Qu3dvevTowX/+8x8SExMtEGXh0Gq1BAUFcf369RzLCnSuVIqhW7duKR06dFDu3LmjJCcnK927d1cuXLhgtM5zzz2nHD9+XFEURfnggw+UH374wRKhmlVe7ZCUlKS0bt1auXXrlqIoijJ//nxl+vTplgrXbEz5PiiKokRHRyvPPPOM0qFDBwtEWTjyagu9Xq906dJF2bt3r6IoivLpp58qs2fPtlS4ZmPKd+Lll19W9uzZoyiKosyYMUP57LPPLBGq2Z04cUIJCgpS6tatq0RERORYXpBzZbG84rBUV+1FTV7tkJmZyeTJkylTpgwANWvWJDIy0lLhmk1e7XDXxIkTGTlypAUiLDx5tcWZM2dwdnY2dOszYsQIBgwYYKlwzcaU74Reryc5ORmA1NRUHB2f/NjcRcG6deuYPHlyrn0AFvRcWSwTR25dtd/fFfuT6qq9qMurHTw8POjcuTMAaWlpLFu2jKeffrrQ4zS3vNoBYNWqVdSpU4cGDRoUdniFKq+2uHbtGt7e3owfP56ePXsyefJknJ2dLRGqWZnynXj//feZOHEigYGB7N+/n379+hV2mIXi448/pmnTprkuK+i5slgmjsLqqr2oM/VzJiUl8frrr1OrVi169uxZmCEWirzaISwsjJCQEN58801LhFeo8moLnU5HaGgoL7/8Mhs3bsTPz4+ZM2daIlSzyqsd0tLSmDBhAt988w1//vkn/fv3Z9y4cZYI1aIKeq4slonjwa7YzdVVe1GXVztA9i+K/v37U7NmTT7++OPCDrFQ5NUOwcHBREdH07t3b15//XVDm5REebWFRqPB39+fgIAAAIKCgowGVisp8mqHsLAwHBwcqF+/PgAvvfQSoaGhhR6npRX0XFksE4d01Z4tr3bIyspixIgRPPvss0yYMKFEXnVB3u0watQoduzYwebNm1m2bBk+Pj6sXr3aghGbT15t0ahRI+Li4jh//jwAv//+O3Xr1rVUuGaTVzv4+/tz69YtwsPDAdi1a5chmVqTAp8rn9yz+8K1ZcsW5bnnnlO6dOmiLFu2TFEURXnttdeUU6dOKYqiKOfOnVN69+6tdO3aVXn33XeV9PR0S4ZrNo9qh5CQEKVmzZpKjx49DP+NHz/ewhGbR17fh7siIiJK9FtVipJ3W5w4cULp3bu30q1bN2XYsGFKTEyMJcM1m7zaYc+ePUr37t2VoKAgZciQIcq1a9csGa7ZdejQwfBW1eOeK6VbdSGEEPlSLG9VCSGEsBxJHEIIIfJFEocQQoh8kcQhhBAiXyRxCCGEyBdJHEIUstTUVGJiYiwdhkFERISlQxDFjCQOUSStXLmSJk2a0Lp1azIzMx+63qFDh2jevHkhRlYwjRo14tKlSwAMGDCAv//+G4AtW7ZYtJPBs2fP8vLLL1vs+KJ4srV0AELkZu3atXzwwQf06dPH0qE8EcePHzf8HR8fb/i7R48e9OjRwxIhAdn9mD0qMQuRG7niEEVO165duXbtGtOmTWPatGmkpaUxZcoUOnfuTMOGDenSpQu//fZbju0yMjL44IMPaN68OYGBgYwaNYo7d+4A2Z3affTRR7Rp04bAwEBmzZpFRkZGrsfv2LEjixYton379jRp0oQPP/yQ9PR0AJKTk5k6dSqtW7emdevWTJgwgaSkJABu3rzJ4MGDadq0KU8//TSzZ8/mbn1tzZo1CQsL46233uLmzZu88847rFq1ig0bNtCrVy+0Wi0NGjTgwoULhjjWr19P3759DfseMWIEzZs3p0uXLvz888+5xn79+nWaNGnCPn/lawAABfBJREFU+++/T9OmTdm8eTMRERGMGDGCdu3aUb9+ffr168elS5eIjY1l+PDhxMfH06hRI+7cuZOvdhLWSxKHKHJ27NhBuXLlWLBgAZMmTWLlypVcunSJDRs2cPToUXr16sX06dNzbLd582YuXbrE7t272blzJykpKaxatQqAWbNmER4ezpYtW9iyZQunT59myZIlD41h27Zt/PDDD+zYsYOzZ8/y+eefAzBp0iTCw8PZunUrv/76KzExMUyaNAmAefPmUaNGDUJDQ/n+++/Ztm0bBw4cMNrvF198YfhsgwcPNsx3dXWlY8eObNu2zTDvl19+4fnnnzf0OVa9enX27dvHwoULmTdvHgcPHsw1dq1WS/ny5dm/fz9dunRh4sSJVKlShV27dnHw4EE8PDxYsmQJXl5eLF++HHd3d44fP46Hh0e+20lYJ0kcosgbMGAACxcuxNnZmcjISFxcXHIdM8DNzY2rV6+yceNG7ty5w7Jly3jnnXdQFIUNGzYwZswYPDw88PT05O2332bdunUPPeYbb7xB+fLl8fb25o033mDbtm2kpaWxY8cOxo4di6enJ6VLl2bcuHFs376d1NRU3NzcOHz4MDt27MDZ2Zndu3fTqlUrkz/nCy+8wPbt24Hs3lyPHTvGs88+y99//01kZCSjR4/G3t6eWrVq0a9fP3766aeH7qt79+7Y29vj5OTEzJkzGTVqFFlZWdy8eRN3d/dc268g7SSskzzjEEVeUlISU6dO5dSpU/j5+eHn50duXaw988wzxMXFsWHDBj7++GNq1KjBtGnTKF++PGlpaQwaNMjQQ7CiKGRmZpKeno6Dg0OOffn7+xv+LlOmDNHR0SQmJpKZmUm5cuUMy8qXL4+iKERFRTF27FgWLlzIZ599xv/+9z/atm3LRx99hLe3t0mfMzAwEK1Wy+nTpzly5AitW7fG09OTgwcPotVqeeqppwzrZmVlPbJX2/uPGR4ezqeffsrt27epVq0aKpUq1/aLi4vLdzsJ6ySJQxR5kydPpmrVqixZsgRbW1sOHz5s+GV+vytXrtCiRQv69+/PnTt3+OKLL3jvvffYtm0bdnZ2bNq0CT8/PwBSUlKIiYl56MkwKirK8PfNmzcpW7Ys3t7e2Nvbc+PGDTw9PYHsZwpqtRoPDw/++ecfhg8fzrhx47h27RoTJkxg4cKFTJs2zaTPaWNjw3PPPUdwcDBHjhzhlVdeAbJHrytTpgx79uwxrBsTE5Pryf+uuyf+jIwMRo4cyYwZM3jmmWcAWLRoEYcOHcqxjbu7e77bSVgnuVUlijytVoujoyM2NjZERkayYMECgBxvA+3atYv//e9/xMTEULp0aVxcXHB3d8fGxobu3bszZ84cEhMTSUlJYdKkSbz//vsPPeayZcuIjY0lKiqKJUuW8MILL6BWq+nRowdz5swhLi6OhIQEZs+eTbt27ShVqhSLFy9mzpw5pKen4+XlhY2NDR4eHjn2bWdnh1arzfW4L7zwAlu3biU8PJyOHTsC0KBBAxwdHVmxYgWZmZncunWLoUOH8sMPP+TZdnevFpycnAA4ceIEa9asMbSdvb09GRkZZGRkFKidhHWSxCGKvA8++IA9e/bQuHFjBg4cSLt27XD+f3v3y6pIFMZx/JsMVxCmmA2CYrBMGMOAfxAMMoIWTVaDL0EHnCRYbAZfksH3MNVsNNywrFzusiwn7cJ+P/VwOA9P+fEcOJyPj/e7iJ/W6zXdbpcsy4jjmPv9zvF4BGC32xFFEdPplH6/z/P55Hw+//bMdrvNcrkkyzJ6vR6bzeZdS6PRYDabMR6PiaKI0+kEwOFw4PF4kKYpg8GAer3+3vfVfD4nz3Mul8sva51Oh1qtxmQyoVKpAD+C5nq9crvdSNOUxWJBkiRst9s/9q5arVIUBfv9njiOKYqC1WpFWZa8Xi9arRbNZpMkSSjLMrhP+j/5H4f0zWg0Is9zhsPh3y5F+ic5cUiSghgckqQgXlVJkoI4cUiSghgckqQgBockKYjBIUkKYnBIkoJ8AkxUz1wJoIXhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Fitting Ensemble Models\n",
    "\n",
    "The goal of ensemble is to combine predictions of base estimators to improve final model performance. \n",
    "\n",
    "In boosting methods, base estimators are built sequentially and one tries to reduce the bias of the combined estimator. We deliberatley choose AdaBoost and Gradient Boosting from this school of thought to fit. \n",
    "\n",
    "In averaging methods, the driving principle is to build several estimators independently and then to average their predictions. We pick Random Forest and Extremley Randomized Trees to fit. \n",
    "\n",
    "Initial predictions, with only default hyperparameter settings, return reasonable results across the four ensembles. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AB: 0.991720 (0.015759)\n",
      "GBM: 0.996465 (0.009254)\n",
      "RF: 0.993593 (0.011777)\n",
      "ET: 0.997149 (0.005914)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import ExtraTreesClassifier\n",
    "\n",
    "ensembles = []\n",
    "ensembles.append(('AB', AdaBoostClassifier())) \n",
    "ensembles.append(('GBM', GradientBoostingClassifier())) \n",
    "ensembles.append(('RF', RandomForestClassifier())) \n",
    "ensembles.append(('ET', ExtraTreesClassifier())) \n",
    "\n",
    "results = []\n",
    "names = []\n",
    "for name, model in ensembles:\n",
    "  kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "  cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "  results.append(cv_results)\n",
    "  names.append(name)\n",
    "  msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "  print(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEYCAYAAABfgk2GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfVhU1d4+8HuAMRMl1AaRsjR9BgrBdwHlmR7UI4igSKKVP/EcFbUygnw3MMNEKRUi09Q0O548hziOKKmImG8k4suxUB8VxTDQGF7UEA4mMuv3Rw/7OKEOyuAw7PtzXVy19prZ811rz9yzWbMdFEIIASIiavaszF0AERE9Hgx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQa+mQ0aNAjOzs73/ImPjzd3eXVkZ2fD2dkZlZWVj9RfX1euXIGLiwsmTZpUp0+r1cLDw6NB+3+QP47h3LlzOHLkiNTv7OyMffv2Ndrj10pOTsbo0aPRq1cvDBw4EO+88w7y8vIa/XFNqbGPFT0cG3MXQMC7776L4ODgOttbtWplhmqahu3bt+P555/H4cOH8csvv6Bjx46P7bF79eqFzMxMaf7ffPNNTJw4EZ6eno+thtmzZ+PIkSOIjIxE7969UV5ejtWrV+P111/H3//+d7zwwguPrZaG8Pf3x8svv2zuMuj/8Ay/CbC1tYVKparzY2tra+7SzCY1NRVjxoxBhw4doNVqH+tjt2jRAiqVCgqF4rE+bq2dO3di586d+OqrrzBq1Cg8//zzcHNzQ2JiIhwcHPDJJ5+Ypa5H0bJlS7Rv397cZdD/YeBbgE8//RTTp09HXFwc+vfvD29vb3z44YeoqakBAOh0OkybNg19+vRB3759ER4ejrKyMun+Bw4cwMiRI+Hu7o7hw4djy5YtUp9Wq0VwcDA2btwILy8v9OnTB8uWLcPFixcxduxY9OjRA6+++ioKCgoMatJqtfD29kavXr2wYMEC3Lp16561V1RUIDo6Gv3794eHhwfCw8Oh0+keON7Tp08jLy8Pnp6eGDJkCLZu3YoH/YPwq1evYtKkSejZsyeGDh2KpKQkODs7S/3FxcWYNWsWvLy80Lt3b7zzzjsoLi4GABQWFsLZ2RmrVq2Ch4cHwsLCDJZ0xo8fjytXrmDRokUYP368QY2jR4+Gm5sbAgICcPToUalv0KBB+Pvf/47XX38dbm5uGDFiBHJzc/Hpp5/Cw8MDnp6eWLdu3X3Hs2XLFvzpT39Cly5dDLbb2NhgxYoVmDlzprQtMzMTISEh6NmzJ3x8fPDFF19Ic/WwxzY7Oxt9+/aVjm3fvn0RHR1tcGwPHTqEMWPGwN3dHT169EBoaKi0zJSdnQ0PDw/ExcWhT58+WLBgQZ0lnc8++wwvv/wy3NzcEBQUhAMHDtTrOAG/L6XVjqlHjx4ICQnByZMn7zuPdA+CzMrHx0ds2rTpgbdJTEwUrq6uYv78+SIvL08kJSUJFxcXkZaWJoQQYtq0aeKNN94Qly5dEmfPnhXBwcFi5syZQgghcnNzhbu7u9i8ebO4fPmy2LFjh+jXr5/49ttvhRBCbNmyRbi6uoq3335b5OXlic2bNwu1Wi2GDBki9u3bJ86cOSN8fX3FrFmzhBBCHDlyRKjVahEQECBOnjwpjh07JoYMGSLee+89g/6KigohhBCRkZFi3Lhx4scffxTnz58X4eHhIiAgQFRXV993vIsXLxb/8z//Y7C/rKwsqX/Lli2if//+QgghqqurxfDhw8XkyZPF+fPnxd69e4WXl5dQq9VCCCFu374thg0bJsaNGydycnJETk6OGDt2rBg9erTQ6/WioKBAqNVqERISIi5duiRyc3MNxnD9+nWh0WjE559/Lq5fvy6EEEKtVgsvLy+xf/9+8dNPP4mpU6cKb29vodfrpWPav39/sWfPHnHx4kURFBQk+vXrJ+bMmSMuXrwoPv/8c+Hs7Czy8/PvOX4vLy+xfv36Bz4nhBDi6NGj4sUXXxSfffaZuHTpkti2bZvo1auX+Nvf/vbIx/bFF18U/v7+4sSJE+Lo0aNi0KBBYu7cuUIIIQoLC4Wrq6tYv369+Pnnn8XJkyfFiBEjxNSpUw2O1bRp08Tly5fFTz/9ZHCs0tPTRf/+/UV2drYoKCgQK1asED179hQ3b940epxq5/2///u/xf79+8XZs2fFa6+9JgICAozOE/0HA9/MfHx8hKurq+jZs2ednytXrgghfg/83r17i99++026X1BQkFi+fLkQQojAwEAxZ84cqT8/P1+cOXNGCCHE7NmzpTCutXr1ajFq1CghxO+hoFarRWlpqdTfo0cPsWLFCqmdmJgo3b72RX3ixAmpf8+ePcLV1VVUVlYahOXPP/8s1Gq1KCoqkm7722+/iZ49e4rvvvvunvNx584dMXDgQPHhhx9KbU9PT+kNrLbm2hA5dOiQcHV1FWVlZVJ/bbAJIcR3330nXF1dDWr45ZdfxEsvvSQyMzOlwK99A7x7jLVvWn98U1ar1WLDhg1S+9ixYwZz6OPjIxYuXCj1r127Vri6uoqqqippTC4uLmLv3r33nANXV1fxzTff3LPvbm+//baYMmWKwbY1a9YIb29vaZ4e5dgeO3ZM6q89tjdv3hSXL18Wf/3rXw0eb/369WLw4MEG9z958qTUf/ex+vLLL8XAgQNFQUGBEOL358KhQ4dEVVWV0eMkxO/zvmbNGqk/IyNDqNVqg9cFPRg/tG0Cpk6dihEjRtTZ7uDgIP2/k5MTWrRoIbVbt26N6upqAMA777yDyMhIpKenw8vLC3/6058QEBAAALhw4QJyc3OxY8cO6b537tyBjY2Nwb7uXmd94okn0KlTJ4P27du3pbaVlRXc3d2ltpubG6qrq3H58mWD+i9evAgA8PPzM9heVVWFn376CT4+PnXGfPjwYZSUlGDo0KEAAGtrawwePBipqalYsGAB2rRpY3D78+fP45lnnkG7du2kbb169ZL+/8KFC3ByckKHDh2kbY6OjnjmmWdw4cIFPP/88wBgMN76uPv2dnZ2AGCw9FG7XwB48skn8fTTT6Nly5bSmGxsbAzm9G729vb49ddfjdZw4cKFOs+bPn36YPny5SgvLwfw8MfW2toaPXv2lNq1x/bSpUtwd3fH0KFDsXbtWly8eBGXLl3CuXPnDJ6nAPDcc8/ds94RI0Zg69atGDJkCFxdXTFo0CC88soraNmypdHjNHDgQABA586dpf7WrVsD+P35fPdrg+6Pgd8EtG3b1iAg7kWpVN63b/Dgwdi/fz/27t2LQ4cOYeHChdi+fTs2bNiAmpoajB8/Hq+++up97393+Neysrr/xzsKhQLW1tZSW/zfmvEfa6ypqYFSqcTWrVvrfAD61FNP3XPf27dvBwBMmDDBYP96vR7ffvstXnvttTq1iwes79eG7B+J33+7NXq7+7l7/Hfv8+667vag+fyjHj164NSpU/fsS0lJwffff4+4uLh71lxbg16vv2cdxmqxsrIy6K/dn7W1NXJzczF27FgMHDgQ/fr1Q0hICH788Uds3rzZYB9PPPHEPffdrl07aLVaZGVlYf/+/di6dSu++uor/O1vf6v3cbrX6+BBx58M8UNbCyeEQFxcHIqLixESEoLExESsXLkS33//PcrKytC1a1dcvnwZzz//vPRz9OjROi/Sh1FTU4MLFy5I7R9++KHOmSMAvPDCC6iurkZVVZX02CqVCh999BHy8/Pr7LeqqgoZGRkYM2YMUlJSpJ9t27bh2WefNfiwuZZarcbVq1dx7do1advdYdm1a1dcvXrV4MM/nU6Hq1evNtlLG0eNGoW9e/fi0qVLBttv376NjRs3oqqqClZWVnjhhRfwww8/GNzm5MmTaN++/X3fUI2prq5Gbm6u1M7JycETTzyBLl26YOvWrXjppZewcuVKTJgwAf369UNhYWG9Azc9PR3/+Mc/4O3tjaioKKSlpcHW1hYHDhywyONkiRj4TUBlZSVKSkrq/Ny4ccPofRUKBfLy8hATE4P//d//xeXLl7Fjxw4888wzaNu2LSZOnIj9+/fj888/x+XLl7F7927ExsY26FI5hUKBefPmIScnB0eOHMFHH32E0NDQOmd2L7zwAgYNGoTZs2fj+PHjyMvLw5w5c/Djjz/e80W8Z88e/Pvf/8Zf/vIXqNVqg59x48bh1KlTBmEEAJ6enujatSvmzZuH3NxcHDp0yOCyxQEDBsDZ2RnvvvsuTp8+jdOnT2PGjBno3LkzvLy86jVeW1tb5OXlGVz51JiGDBmCQYMGYcKECUhJSUFBQQGOHz+ON954A7/88gtmzZoFAJg8eTIOHTqEVatWIT8/Hzt37sTatWsxfvz4Bl1SGh0djTNnziA7OxtLly5FSEgIWrVqhbZt2+LSpUs4fvw4CgoKsHHjRiQnJ993aeqPampq8PHHH2P37t24cuUK0tPTUVJSgu7du5vkOJFxXNJpAlasWIEVK1bU2d6/f39s2rTJ6P0XL16MRYsW4S9/+Qtu3bqFXr16Ye3atbCyskL37t2RmJgonfmrVCpMmTIFYWFhj1zvk08+ieDgYEyZMgV37txBUFAQwsPD73nbuLg4LFmyBG+99RZu376NHj16YMOGDdK6991SU1PRv3//e74ZvPLKK0hMTMSWLVsMLrlUKBRYuXIloqKi8Morr8DJyQkhISHSZY8KhQKrVq3C4sWLMX78eFhbW0Oj0SA+Pr7e676hoaFYsmQJTp48iZSUlHrdp6FWrFiBjRs34osvvsDChQvRunVr9OvXD0lJSdLy34svvohPP/0Un3zyCVatWgVHR0e8+eab+POf/9ygxx4+fDgmTZoEIQSCg4Px7rvvAgDGjx+P8+fPY9q0aVAoFHjppZewcOFCREdHo6ioyOh+hw0bhqtXr+Kjjz6CTqeDk5MTFixYIAV6Q48TGacQXAAjC1ZWVoacnByDD4B37dqFZcuWYe/evWaszPJkZ2cjNDQU//rXv2T9j/6aMy7pkEVTKBQIDw/Hhg0bUFhYiBMnTmDlypUYPny4uUsjanK4pEMWrV27dkhMTERCQgISEhLw1FNPYeTIkXj77bfNXRpRk8MlHSIimeCSDhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMtGk/wDK9euV0Oub9tf1t2/fGmVlFeYuo9ngfJoW59N0LGEurawUaNv2/n+eskkHvl4vmnzgA7CIGi0J59O0OJ+mY+lzySUdIiKZYOATEckEA5+ISCbqHfgVFRUICAhAYWFhnb6zZ88iODgYvr6+eO+993Dnzh0AwNWrVzFu3Dj4+fnhjTfeQGVlpekqJyKih1KvwP/xxx/x2muvIT8//579s2bNwoIFC7B7924IIfDNN98AAD744AO8/vrrSEtLQ/fu3bFq1SqTFU5ERA+nXoH/zTff4P3334eDg0OdvitXruDWrVvo2bMnACA4OBhpaWmorq7GsWPH4Ovra7CdiIjMo16XZS5evPi+fcXFxVCpVFJbpVJBp9Ph+vXraN26NWxsbAy2ExGReTT4Ony9Xg+FQiG1hRBQKBTSf+/2x7Yx7du3bmh5DdK9e3ecOXPGJPtydXXF6dOnTbKvpmZPpC+6Pm2aY3XTJHv5XV5pBf4Uv9uEe2w6+Nysv3bt2uH69evmLsNA27Ztce3atcf+uA0OfEdHR5SUlEjt0tJSODg4oF27drh58yZqampgbW2NkpKSey4JPUhZWUWj/EMHtfo53Lhxw+T7fZAzZ84YfcOzt7dHbu7Pj6ki0xmakI7i4nKT7EulaoOSEtPE/lAHOxTPN+VbSNOxb19WvW7n4GBXr2Njqjlvijb/v95Qd3jK3GUYyNX92ihzbmWleOCJcoMD/5lnnsETTzyBEydOoE+fPti2bRs0Gg2USiX69u2LnTt3IjAwECkpKdBoNA19OJO4ceNGkwwoBwc7k+yHiP5j2Kd7TfJ6N+VrfZiDHYqjTbKrh/LIgR8WFobw8HC4ublh2bJliIqKQkVFBVxdXREaGgoAeP/99zF37lysXr0aHTt2xIoVK0xWOBFRfTW1kyl7e3uzPK5CCNFkvxyisZZ06vtrbn2Y+gzfVHU9TpxP0zLHkmN9WOqSo6mY8rnZWBp9SYcIaHpnUID5zqIaikuO1FgY+NRg9Q0njcYD586dbfDjubi8iIMHsxu8HyK5YeDTY1OfkLaEX5uJLBW/PI2ISCYY+EREMiHLJZ1dbw/GzbV/Nsm+TLn4sOvtwSbcG1kqPj+psfCyzAbiZYSmxTV8Pj+bKkt4bvKyTCIL1BQvgbTUy1zpPxj4RE2MKc+iLeGslB4ffmhLRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0T0AFptMjQaD1hbW0Oj8YBWm2zukh4Z/6UtEdF9aLXJiI1dhISElQgIGIpvv01HRMR0AEBwcIiZq3t4PMMnIrqPhIRlSEhYCW9vDZRKJby9NUhIWImEhGXmLu2RMPCJmqHmtAxhTrm55+Hh4WWwzcPDC7m5581UUcPIdkmH30ZIzVVzW4YwJ7XaGdnZWfD21kjbsrOzoFY7m7GqRyfL78OvL1P90W2Af3i7vvjtjg2n0XggNvZjeHtrpPnMzDyI+fNn8Tn4kO735jl/fnSTfPM09n34DPwGYkCZFuez4Rwd7VFQUAKlUinNZ3V1NTp1UqGo6Ia5y7M4Wm0yEhKWITf3PNRqZ0REzGySYQ/wD6AQyU5zW4Ywt+DgEAQHhzSLkxF+aEvUzEREzERExHRkZh5EdXU1MjMPIiJiOiIiZpq7NDIznuETNTO1yw3z58/C6NEjoFY7N9k1Z3q86rWGn5qaitWrV+POnTuYMGECxo0bZ9B/4MABLFv2+3WparUaMTExsLW1RX5+PqKiovDrr7/C3t4eMTEx6NKlS72L4xq+/HA+TYvzaTqWMJfG1vCNLunodDrEx8dj8+bNSElJQVJSEi5evCj1l5eXY+7cuYiPj0dqaipcXFwQHx8PAJg3bx6Cg4ORmpqKGTNmICIiwgRDIiKiR2E08A8fPgxPT0/Y29ujVatW8PX1RVpamtSfn58PJycndOvWDQDg4+ODjIwMAMDZs2fh5+cHAOjZsyeKi4tRUFDQGOMgIiIjjAZ+cXExVCqV1HZwcIBOp5PanTt3RlFREc6dOwcA2LVrF0pLSwEAL730Enbs2AEAyMrKwo0bN1BSUmLSARARUf0Y/dBWr9dDoVBIbSGEQdvOzg5xcXGIjo6GXq/HmDFjoFQqAQBLly7FokWLsGnTJmg0Gri4uEh99fGgtaimRKVqY+4SmhXOp2lxPk3H0ufSaOA7Ojri+PHjUrukpAQODg5Su6amBo6OjkhO/v27OnJyctCpUycAwJ07d/DZZ5+hRYsWqK6uRlJSEp599tl6F8cPbeWH82lanE/TsYS5bPCHtgMGDEBWVhauXbuGqqoqpKenQ6P5zz/oUCgUmDhxInQ6HYQQ2LhxI/z9/QEA8fHx2Lt3LwDgn//8J9zc3NC2bduGjomIiB6B0cDv0KEDIiMjERoaiqCgIAQEBMDd3R1hYWE4deoUrKysEBMTg8mTJ8PPzw92dnaYNGkSAGDmzJn46quvMHz4cOzZswdLlixp9AEREdG98bt0GsgSfs2zJJxP0+J8mo4lzGWDl3SIiKh5YOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+I6AG02mRoNB6wtraGRuMBrTbZ3CU9snoFfmpqKvz9/TF06FB8/fXXdfoPHDiAwMBABAYGYsaMGaisrAQA/PrrrwgLC8OIESMwevRonD171rTVExE1Iq02GbGxixAb+zFu3bqF2NiPERu7yGJD32jg63Q6xMfHY/PmzUhJSUFSUhIuXrwo9ZeXl2Pu3LmIj49HamoqXFxcEB8fDwD48ssvoVarsX37drz55puIiYlpvJEQEZlYQsIyJCSshLe3BkqlEt7eGiQkrERCwjJzl/ZIjAb+4cOH4enpCXt7e7Rq1Qq+vr5IS0uT+vPz8+Hk5IRu3boBAHx8fJCRkQEA0Ov10tl+VVUVWrZs2RhjICJqFLm55+Hh4WWwzcPDC7m5581UUcMYDfzi4mKoVCqp7eDgAJ1OJ7U7d+6MoqIinDt3DgCwa9culJaWAgAmTpyIrKwseHt7IyoqCuHh4aaun4io0ajVzsjOzjLYlp2dBbXa2UwVNYyNsRvo9XooFAqpLYQwaNvZ2SEuLg7R0dHQ6/UYM2YMlEolAGDRokUYN24cQkNDcfLkSURGRmLHjh2wtbWtV3Ht27d+2PGYhUrVxtwlNCucT9PifD66BQuiMWPG21i/fj28vb1x+vRxzJjxNhYvXmyR82o08B0dHXH8+HGpXVJSAgcHB6ldU1MDR0dHJCf//iFGTk4OOnXqBADYu3evtG7fq1cvtG/fHnl5eXB3d69XcWVlFdDrRf1HYwYqVRuUlNw0dxnNBufTtDifDTNkSADKy6vw5ptvITf3PNRqZ8yZE4UhQwKa5LxaWSkeeKJsdElnwIAByMrKwrVr11BVVYX09HRoNBqpX6FQYOLEidDpdBBCYOPGjfD39wcAuLi4SOv5+fn5KC4uRpcuXRo6JiKixyY4OAQHD2ajpqYGBw9mIzg4xNwlPTKjZ/gdOnRAZGQkQkNDUV1djdGjR8Pd3R1hYWEIDw+Hm5sbYmJiMHnyZNy+fRteXl6YNGkSAGDp0qVYsGAB1q1bhxYtWiAuLg5t2ljer0FERM2BQgjRZNdMuKQjP5xP0+J8mo4lzGWDl3SIiKh5YOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCZs6nOj1NRUrF69Gnfu3MGECRMwbtw4g/4DBw5g2bJlAAC1Wo2YmBjY2toiODgYNTU1AIBbt26hoKAABw8exNNPP23iYRARkTFGA1+n0yE+Ph5arRYtWrTAq6++Cg8PD3Tr1g0AUF5ejrlz52LTpk3o1q0b1q1bh/j4eERFRUGr1Ur7mT17NkaNGsWwJyIyE6NLOocPH4anpyfs7e3RqlUr+Pr6Ii0tTerPz8+Hk5OT9Abg4+ODjIwMg31kZWXh3LlzCAsLM3H5RERUX0YDv7i4GCqVSmo7ODhAp9NJ7c6dO6OoqAjnzp0DAOzatQulpaUG+0hMTERkZCSsra1NVTcRET0ko0s6er0eCoVCagshDNp2dnaIi4tDdHQ09Ho9xowZA6VSKfVfuHAB169fh4+Pz0MX175964e+jzmoVG3MXUKzwvk0Lc6n6Vj6XBoNfEdHRxw/flxql5SUwMHBQWrX1NTA0dERycnJAICcnBx06tRJ6s/IyIC/v/8jFVdWVgG9XjzSfR8XlaoNSkpumrsMi6fVJiMhYRlyc89DrXZGRMRMBAeHmLssi8fnp+lYwlxaWSkeeKJsdElnwIAByMrKwrVr11BVVYX09HRoNBqpX6FQYOLEidDpdBBCYOPGjQYB/8MPP6Bv374NHAY1Z1ptMmJjFyE29mPcunULsbEfIzZ2EbTaZHOXRtSsGA38Dh06IDIyEqGhoQgKCkJAQADc3d0RFhaGU6dOwcrKCjExMZg8eTL8/PxgZ2eHSZMmSfcvKChAhw4dGnUQZNkSEpYhIWElvL01UCqV8PbWICFhJRISlpm7NKJmRSGEaLJrJlzSkQdHR3sUFJRAqVRK81ldXY1OnVQoKrph7vIsGp+fpmMJc9ngJR2ixqZWOyM7O8tgW3Z2FtRqZzNVRNQ8MfDJ7CIiZiIiYjoyMw+iuroamZkHERExHRERM81dGlGzUq+vViBqTLVX48yfPwujR4+AWu2M+fOjeZUOkYlxDb+BLGFdz5JwPk2L82k6ljCXXMMnIiIADHwiItlg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTNQr8FNTU+Hv74+hQ4fi66+/rtN/4MABBAYGIjAwEDNmzEBlZSUAoKKiAjNmzEBQUBCCgoJw5swZ01ZPRET1ZjTwdTod4uPjsXnzZqSkpCApKQkXL16U+svLyzF37lzEx8cjNTUVLi4uiI+PBwAsWbIEHTt2REpKCt59910sXLiw0QZCREQPZjTwDx8+DE9PT9jb26NVq1bw9fVFWlqa1J+fnw8nJyd069YNAODj44OMjAwIIZCeno4pU6YAADQaDWJjYxtpGEREZIzRwC8uLoZKpZLaDg4O0Ol0Urtz584oKirCuXPnAAC7du1CaWkpysrK0KJFC2zevBljx45FaGgoampqGmEIRERUHzbGbqDX66FQKKS2EMKgbWdnh7i4OERHR0Ov12PMmDFQKpWoqalBaWkp2rRpg6SkJHz//fd46623sHfv3noX175964ccjnmoVG3MXUKzwvk0Lc6n6Vj6XBoNfEdHRxw/flxql5SUwMHBQWrX1NTA0dERycnJAICcnBx06tQJbdu2hY2NDQICAgAAAwcOxL///W+UlZWhffv29SqurKwCer14qAE9bipVG5SU3DR3Gc0G59M0tNpkJCQsQ27ueajVzoiImIng4BBzl2XRLOG5aWWleOCJstElnQEDBiArKwvXrl1DVVUV0tPTodFopH6FQoGJEydCp9NBCIGNGzfC398fLVq0wIABA7Bjxw4AwA8//PBQ45EAAApASURBVIAnn3wSbdu2NcGwiOh+tNpkxMYuQmzsx7h16xZiYz9GbOwiaLXJ5i6NzEwhhDB6Cp2amoo1a9aguroao0ePRlhYGMLCwhAeHg43Nzfs378fy5cvx+3bt+Hl5YX33nsPSqUSxcXFWLBgAQoLC2FjY4MPPvgAPXr0qHdxPMOXH85nw2k0HoiN/Rje3hppPjMzD2L+/Fk4eDDb3OVZLEt4bho7w69X4JsLA19+OJ8N5+hoj4KCEiiVSmk+q6ur0amTCkVFN8xdnsWyhOdmg5d0iMiyqNXOyM7OMtiWnZ0FtdrZTBVRU8HAJ2pmIiJmIiJiOjIzD6K6uhqZmQcRETEdEREzzV0amZnRq3SIyLLUXo0zf/4sjB49Amq1M+bPj+ZVOsQ1/IayhHU9S8L5NC3Op+lYwlxyDZ+IiAAw8ImIZIOBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQyUa/AT01Nhb+/P4YOHYqvv/66Tv+BAwcQGBiIwMBAzJgxA5WVlQCAo0ePwsPDAyNHjsTIkSMxb94801ZPRET1ZmPsBjqdDvHx8dBqtWjRogVeffVVeHh4oFu3bgCA8vJyzJ07F5s2bUK3bt2wbt06xMfHIyoqCqdPn8bEiRMxderURh8IERE9mNEz/MOHD8PT0xP29vZo1aoVfH19kZaWJvXn5+fDyclJegPw8fFBRkYGAODUqVPIzMxEYGAgpk2bhl9++aWRhkFERMYYDfzi4mKoVCqp7eDgAJ1OJ7U7d+6MoqIinDt3DgCwa9culJaWAgDatGmD8ePHIzU1FS+//DIiIyNNXT8REdWT0SUdvV4PhUIhtYUQBm07OzvExcUhOjoaer0eY8aMgVKpBADExMRIt3vttdewfPly3Lx5E23atKlXce3bt673QMxJparfeKh+OJ+mxfk0HUufS6OB7+joiOPHj0vtkpISODg4SO2amho4OjoiOTkZAJCTk4NOnTpBr9djzZo1mDJlCqytraXb3/3/xpSVVUCvF/W+vTmoVG1QUnLT3GU0G5xP0+J8mo4lzKWVleKBJ8pGl3QGDBiArKwsXLt2DVVVVUhPT4dGo5H6FQoFJk6cCJ1OByEENm7cCH9/f1hZWWHPnj3YvXs3ACAlJQU9evRAq1atTDAsIiJ6WEYDv0OHDoiMjERoaCiCgoIQEBAAd3d3hIWF4dSpU7CyskJMTAwmT54MPz8/2NnZYdKkSQCAuLg4/PWvf8Xw4cOxZcsWfPjhh40+ICIiujeFEKLJrplwSUd+OJ+mxfk0HUuYywYv6RARUfPAwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcB/RFptMjQaD1hbW0Oj8YBWm2zukoiIHsjot2VSXVptMmJjFyEhYSUCAobi22/TERExHQAQHBxi5uqIiO6NZ/iPICFhGRISVsLbWwOlUglvbw0SElYiIWGZuUsjIrovBv4jyM09Dw8PL4NtHh5eyM09b6aKiIiMY+A/ArXaGdnZWQbbsrOzoFY7m6kiIiLjGPiPICJiJiIipiMz8yCqq6uRmXkQERHTEREx09ylERHdFz+0fQS1H8zOnz8Lo0ePgFrtjPnzo/mBLRE1afwDKA1kCX8UwZJwPk2L82k6ljCX/AMoREQEgIFPRCQbDHwiIplg4BMRyUSTvkrHykph7hLqxVLqtBScT9PifJpOU59LY/U16at0iIjIdLikQ0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBNN+qsVmqLc3FwEBgYiMTERvr6+AIBBgwahZcuWUCqVuHPnDrp06YLFixfjqaeeMnO1TcOdO3ewbt06bN++HQqFAjU1NRg1ahSmTp2KefPm4ciRI3jqqaeg1+vRsmVLLFq0CC4uLigsLMTgwYMxduxYxMTESPs7e/YsgoKCsGTJEgQHB5txZE1DYWEh/Pz80LVrVwCAXq9HZWUlgoKCEBwcbNBX6/PPP0fHjh3NUW6T98f5rDVmzBh88803AIC8vDw899xzUCqV6N27N95//31zlPrQGPgPacuWLfDz80NSUpIU+ACwdu1aPPvsswCAxYsXY82aNZg9e7a5ymxSPvjgA5SWliIpKQl2dnaoqKjAW2+9hTZt2gAAwsPDpeDOyMhAVFQU/vnPfwIA7O3tcejQIdTU1MDa2hoAsHPnTrRr1848g2miHBwcsG3bNqmt0+ng6+uL4cOH1+kj4+43Z+PGjQPw+0ne3a95S8ElnYdQXV2N1NRURERE4MyZM/j555/r3Kb27Orpp582Q4VNT1FREbZv346lS5fCzs4OANC6dWssWLDgnnN08+ZNg+22trZ48cUXcezYMWnb999/jwEDBjR+8RaspKQEQghcv37d3KVQE8Iz/Idw4MABODk5oUuXLhgyZAiSkpIwa9YsAMCUKVOgVCpRVlYGa2trTJ8+3czVNg05OTno2rVrneWtrl27omvXrti3bx8SExPx1VdfoaqqClevXsXq1asNbjts2DDs3r0bnp6eyMnJgbOzM/idf4aKi4sxcuRI/Pbbb7h+/Trc3NywcuVKODo6Sn21AgMDMXnyZDNW2/T9cc4A4KOPPoKzs7OZKjINBv5D2LJlCwICAgAA/v7+mDlzJt555x0Ahks6GzZswKRJk7Bz504oFE3761Qfh7vnIC0tDatXr4Zer0eLFi3wX//1XwZLOv/6178wefJkbNu2TbrfoEGDkJCQAL1ej127dmHYsGHYuXOnWcbSVNUuQej1eixduhR5eXkYOHAgrl69yiWdR9Bc54xLOvVUVlaGQ4cOYcOGDRg0aBCioqJQXl6OPXv21LltSEgILl26xF+nAXTv3h15eXmoqKgAAPj5+WHbtm1YvXr1Peend+/eeO6553DmzBlpm62tLVxcXHDixAkcOXKEyzkPYGVlhdmzZ0On02H9+vXmLoeaGAZ+PW3btg2enp44ePAgvvvuO+zbtw/Tpk3DP/7xjzq3zcrKQseOHfnBIgAnJyeMGDECc+bMQXl5OYDfr9rZv38/rKzqPv2uXLmCwsJCuLi4GGwfNmwYli9fju7du8PGhr+YPoiNjQ1mz56NVatWobS01NzlUBPCV049bd26FZGRkQbbxo0bhy+++AKtW7eW1vCtrKxgbW2NFStWmKnSpmfhwoX48ssvERoaipqaGlRWVsLDwwPr1q3DmjVrpDV8a2tr/Pbbb5gzZw46d+6MwsJCaR8+Pj547733pCU0ejCNRoNevXrhk08+MXcpFulea/j9+vVDVFSUmSoyDf7FKyIimeCSDhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpKJ/w+er/awzJeA/wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compare Algorithms\n",
    "\n",
    "fig = plt.figure()\n",
    "fig.suptitle('Ensemble Algorithm Comparison') \n",
    "ax = fig.add_subplot(111) \n",
    "plt.boxplot(results) \n",
    "ax.set_xticklabels(names)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 6-way comparison, adding in XGBoost and LightGBM per reviewers' comments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AB: 0.991720 (0.015759)\n",
      "GBM: 0.996452 (0.009252)\n",
      "RF: 0.990161 (0.016996)\n",
      "ET: 0.998357 (0.003925)\n",
      "XGB: 0.997167 (0.007187)\n",
      "LGBM: 0.998314 (0.003246)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import ExtraTreesClassifier\n",
    "from xgboost import XGBClassifier\n",
    "from lightgbm import LGBMClassifier\n",
    "\n",
    "ensembles = []\n",
    "ensembles.append(('AB', AdaBoostClassifier())) \n",
    "ensembles.append(('GBM', GradientBoostingClassifier())) \n",
    "ensembles.append(('RF', RandomForestClassifier())) \n",
    "ensembles.append(('ET', ExtraTreesClassifier())) \n",
    "ensembles.append(('XGB', XGBClassifier())) \n",
    "ensembles.append(('LGBM', LGBMClassifier())) \n",
    "\n",
    "results = []\n",
    "names = []\n",
    "for name, model in ensembles:\n",
    "  kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "  cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "  results.append(cv_results)\n",
    "  names.append(name)\n",
    "  msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "  print(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEYCAYAAABfgk2GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfVhUZf4G8HtAyHwhlUDEKEt3wBB8F1B2WsxEEZRItHKlTUGtVYPwlcRcTJIyZ1LS0jLLcjNWRElF1PItEaVU1J+IYBhqDC9aCKEi8/z+cDnrCDKAwDCc+3NdXvWc55w5z3fOzM3hmcMZhRBCgIiIWjwzYw+AiIiaBgOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFvZEOHDoWjo2O1/9RqtbGHV0VqaiocHR1RWlpar/7aunz5MpycnDB58uQqffHx8XBzc3ugx6/JvTVkZGTgyJEjUr+joyN++OGHRtt/pbi4OIwdOxZ9+/bFkCFD8MYbbyA7O7vR99uQGvtYUd20MvYACHjzzTcREBBQZXmbNm2MMJrmYdu2bXjiiSdw+PBh/Pbbb+jSpUuT7btv3744dOiQ9Py//vrrmDRpEtzd3ZtsDHPmzMGRI0cQFhaGfv36obi4GKtXr8bLL7+Mf//733jqqaeabCwPwsfHB88884yxh0H/xTP8ZqBt27awsbGp8q9t27bGHprRJCYmYty4cejcuTPi4+ObdN+WlpawsbGBQqFo0v1W2rFjB3bs2IEvvvgCzz//PJ544gm4uLhgxYoVsLW1xYcffmiUcdVH69atYW1tbexh0H8x8E3AypUrMX36dMTExGDQoEHw9PTEO++8g4qKCgCAVqvFtGnT0L9/fwwYMAAzZ85EUVGRtP3+/fsxZswYuLq6YtSoUdi8ebPUFx8fj4CAAKxfvx4eHh7o378/li1bhqysLIwfPx69e/fGiy++iNzcXL0xxcfHw9PTE3379sXChQtx48aNasdeUlKCyMhIDBo0CG5ubpg5cya0Wm2N9Z4+fRrZ2dlwd3fHsGHDsGXLFtT0B+FXrlzB5MmT0adPHwwfPhybNm2Co6Oj1J+fn4/Zs2fDw8MD/fr1wxtvvIH8/HwAwKVLl+Do6IhVq1bBzc0NISEhelM6EydOxOXLl7F48WJMnDhRb4xjx46Fi4sLfH19cfToUalv6NCh+Pe//42XX34ZLi4uGD16NDIzM7Fy5Uq4ubnB3d0da9euvW89mzdvxnPPPYcnn3xSb3mrVq2wfPlyzJo1S1p26NAhBAYGok+fPvDy8sKnn34qPVd1PbapqakYMGCAdGwHDBiAyMhIvWN78OBBjBs3Dq6urujduzeCgoKkaabU1FS4ubkhJiYG/fv3x8KFC6tM6Xz00Ud45pln4OLiAn9/f+zfv79Wxwm4M5VWWVPv3r0RGBiI48eP3/d5pGoIMiovLy+xYcOGGtdZsWKFcHZ2FhERESI7O1ts2rRJODk5iaSkJCGEENOmTROvvfaauHDhgjh79qwICAgQs2bNEkIIkZmZKVxdXcXGjRvFxYsXxfbt28XAgQPFd999J4QQYvPmzcLZ2VnMmDFDZGdni40bNwqlUimGDRsmfvjhB3HmzBnh7e0tZs+eLYQQ4siRI0KpVApfX19x/PhxcezYMTFs2DDx1ltv6fWXlJQIIYQICwsTEyZMECdPnhTnzp0TM2fOFL6+vqK8vPy+9S5ZskT87W9/03u8lJQUqX/z5s1i0KBBQgghysvLxahRo0RwcLA4d+6c2Lt3r/Dw8BBKpVIIIcStW7fEyJEjxYQJE0R6erpIT08X48ePF2PHjhU6nU7k5uYKpVIpAgMDxYULF0RmZqZeDdeuXRMqlUp8/PHH4tq1a0IIIZRKpfDw8BD79u0Tv/zyi5g6darw9PQUOp1OOqaDBg0Su3fvFllZWcLf318MHDhQzJ07V2RlZYmPP/5YODo6ipycnGrr9/DwEJ999lmNrwkhhDh69Kjo2bOn+Oijj8SFCxfE1q1bRd++fcVXX31V72Pbs2dP4ePjI3766Sdx9OhRMXToUDFv3jwhhBCXLl0Szs7O4rPPPhO//vqrOH78uBg9erSYOnWq3rGaNm2auHjxovjll1/0jlVycrIYNGiQSE1NFbm5uWL58uWiT58+4vr16waPU+Xz/te//lXs27dPnD17Vrz00kvC19fX4PNE/8PANzIvLy/h7Ows+vTpU+Xf5cuXhRB3Ar9fv37i5s2b0nb+/v7igw8+EEII4efnJ+bOnSv15+TkiDNnzgghhJgzZ44UxpVWr14tnn/+eSHEnVBQKpWisLBQ6u/du7dYvny51F6xYoW0fuWb+qeffpL6d+/eLZydnUVpaaleWP76669CqVSKvLw8ad2bN2+KPn36iO+//77a5+P27dtiyJAh4p133pHa7u7u0g+wyjFXhsjBgweFs7OzKCoqkvorg00IIb7//nvh7OysN4bffvtNPP300+LQoUNS4Ff+ALy7xsofWvf+UFYqlWLdunVS+9ixY3rPoZeXl1i0aJHUv2bNGuHs7CzKysqkmpycnMTevXurfQ6cnZ3Ft99+W23f3WbMmCGmTJmit+yTTz4Rnp6e0vNUn2N77Ngxqb/y2F6/fl1cvHhRfPnll3r7++yzz8Szzz6rt/3x48el/ruP1eeffy6GDBkicnNzhRB3XgsHDx4UZWVlBo+TEHee908++UTq37Nnj1AqlXrvC6oZP7RtBqZOnYrRo0dXWW5rayv9v729PSwtLaV2u3btUF5eDgB44403EBYWhuTkZHh4eOC5556Dr68vAOD8+fPIzMzE9u3bpW1v376NVq1a6T3W3fOsDz30EBwcHPTat27dktpmZmZwdXWV2i4uLigvL8fFixf1xp+VlQUAGDFihN7ysrIy/PLLL/Dy8qpS8+HDh1FQUIDhw4cDAMzNzfHss88iMTERCxcuRPv27fXWP3fuHLp27YpOnTpJy/r27Sv9//nz52Fvb4/OnTtLy+zs7NC1a1ecP38eTzzxBADo1Vsbd69vZWUFAHpTH5WPCwAPP/wwHn30UbRu3VqqqVWrVnrP6d06dOiAP/74w+AYzp8/X+V1079/f3zwwQcoLi4GUPdja25ujj59+kjtymN74cIFuLq6Yvjw4VizZg2ysrJw4cIFZGRk6L1OAeDxxx+vdryjR4/Gli1bMGzYMDg7O2Po0KF44YUX0Lp1a4PHaciQIQCAbt26Sf3t2rUDcOf1fPd7g+6Pgd8MdOzYUS8gqmNhYXHfvmeffRb79u3D3r17cfDgQSxatAjbtm3DunXrUFFRgYkTJ+LFF1+87/Z3h38lM7P7f7yjUChgbm4utcV/54zvHWNFRQUsLCywZcuWKh+APvLII9U+9rZt2wAAr7zyit7j63Q6fPfdd3jppZeqjF3UML9fGbL3End+uzW43v3cXf/dj3n3uO5W0/N5r969e+PUqVPV9iUkJODHH39ETExMtWOuHINOp6t2HIbGYmZmptdf+Xjm5ubIzMzE+PHjMWTIEAwcOBCBgYE4efIkNm7cqPcYDz30ULWP3alTJ8THxyMlJQX79u3Dli1b8MUXX+Crr76q9XGq7n1Q0/EnffzQ1sQJIRATE4P8/HwEBgZixYoViI2NxY8//oiioiJ0794dFy9exBNPPCH9O3r0aJU3aV1UVFTg/PnzUvvEiRNVzhwB4KmnnkJ5eTnKysqkfdvY2OC9995DTk5OlcctKyvDnj17MG7cOCQkJEj/tm7discee0zvw+ZKSqUSV65cwdWrV6Vld4dl9+7dceXKFb0P/7RaLa5cudJsL218/vnnsXfvXly4cEFv+a1bt7B+/XqUlZXBzMwMTz31FE6cOKG3zvHjx2FtbX3fH6iGlJeXIzMzU2qnp6fjoYcewpNPPoktW7bg6aefRmxsLF555RUMHDgQly5dqnXgJicn45tvvoGnpycWLFiApKQktG3bFvv37zfJ42SKGPjNQGlpKQoKCqr8+/333w1uq1AokJ2djaioKPzf//0fLl68iO3bt6Nr167o2LEjJk2ahH379uHjjz/GxYsXsWvXLkRHRz/QpXIKhQLz589Heno6jhw5gvfeew9BQUFVzuyeeuopDB06FHPmzEFaWhqys7Mxd+5cnDx5sto38e7du/Hnn3/i1VdfhVKp1Ps3YcIEnDp1Si+MAMDd3R3du3fH/PnzkZmZiYMHD+pdtjh48GA4OjrizTffxOnTp3H69GmEh4ejW7du8PDwqFW9bdu2RXZ2tt6VT41p2LBhGDp0KF555RUkJCQgNzcXaWlpeO211/Dbb79h9uzZAIDg4GAcPHgQq1atQk5ODnbs2IE1a9Zg4sSJD3RJaWRkJM6cOYPU1FQsXboUgYGBaNOmDTp27IgLFy4gLS0Nubm5WL9+PeLi4u47NXWviooKvP/++9i1axcuX76M5ORkFBQUoFevXg1ynMgwTuk0A8uXL8fy5curLB80aBA2bNhgcPslS5Zg8eLFePXVV3Hjxg307dsXa9asgZmZGXr16oUVK1ZIZ/42NjaYMmUKQkJC6j3ehx9+GAEBAZgyZQpu374Nf39/zJw5s9p1Y2Ji8O677+Kf//wnbt26hd69e2PdunXSvPfdEhMTMWjQoGp/GLzwwgtYsWIFNm/erHfJpUKhQGxsLBYsWIAXXngB9vb2CAwMlC57VCgUWLVqFZYsWYKJEyfC3NwcKpUKarW61vO+QUFBePfdd3H8+HEkJCTUapsHtXz5cqxfvx6ffvopFi1ahHbt2mHgwIHYtGmTNP3Xs2dPrFy5Eh9++CFWrVoFOzs7vP766/jHP/7xQPseNWoUJk+eDCEEAgIC8OabbwIAJk6ciHPnzmHatGlQKBR4+umnsWjRIkRGRiIvL8/g444cORJXrlzBe++9B61WC3t7eyxcuFAK9Ac9TmSYQnACjExYUVER0tPT9T4A3rlzJ5YtW4a9e/cacWSmJzU1FUFBQfj5559l/Ud/LRmndMikKRQKzJw5E+vWrcOlS5fw008/ITY2FqNGjTL20IiaHU7pkEnr1KkTVqxYAY1GA41Gg0ceeQRjxozBjBkzjD00omaHUzpERDLBKR0iIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSiWX8ByrVrpdDpmuZ2/dbW7VBUVNIk+zIG1mfaWJ/pasrazMwU6Njx/l9P2awDX6cTTRb4lftryVifaWN9pqu51MYpHSIimWDgExHJBAOfiEgmah34JSUl8PX1xaVLl6r0nT17FgEBAfD29sZbb72F27dvAwCuXLmCCRMmYMSIEXjttddQWlracCMnIqI6qVXgnzx5Ei+99BJycnKq7Z89ezYWLlyIXbt2QQiBb7/9FgDwr3/9Cy+//DKSkpLQq1cvrFq1qsEGTkREdVOrwP/222/x9ttvw9bWtkrf5cuXcePGDfTp0wcAEBAQgKSkJJSXl+PYsWPw9vbWW05ERMZRq8sylyxZct++/Px82NjYSG0bGxtotVpcu3YN7dq1Q6tWrfSWExGRcTzwdfg6nQ4KhUJqCyGgUCik/97t3rYh1tbtHnR4enr16oUzZ87UeTtnZ2ecPn26QcdSX7vDvNH90bo/L9frub/swhI8p95Vz60bVks4fp06dcK1a9eabH8dO3bE1atXm2x/Lfn12RJqe+DAt7OzQ0FBgdQuLCyEra0tOnXqhOvXr6OiogLm5uYoKCiodkqoJkVFJXX+gwWl8nH8/vvvddrGkDNnztz3h1WHDh2Qmflrg+6vJsM1ycjPL67zdjY27VFQUPeX3nBbK+RH1Pcl27B++CHlvn22tlY1Pi/1qb0xbPx7Pyg7P9Jk+8sqLGnS2lvy69MUajMzU9R4ovzAgd+1a1c89NBD+Omnn9C/f39s3boVKpUKFhYWGDBgAHbs2AE/Pz8kJCRApVI96O4M+v3335v0oNjaWtV5G5Kv/pFb6rVdfV+ffeu1N2qp6h34ISEhmDlzJlxcXLBs2TIsWLAAJSUlcHZ2RlBQEADg7bffxrx587B69Wp06dIFy5cvb7CBU8v0IL+h1eeHb1P/hlZX8fFx0GiWITPzHJRKR4SGzkJAQKCxh0Umqk6B//3330v/v3btWun/nZyc8J///KfK+l27dsWGDRseYHhUnab8raJDhw5Nti+Av6HdLT4+DtHRi6HRxMLXdzi++y4ZoaHTAaBZh35Lfn2aumZ98zSqqj5hCBie46bmR6NZBo0mFp6ed6ZIPT1V0GhiERExu9kGfn1fY/X9gU11w8BvQVQqN2RknL1v//3OvJyceuLAgdTGGhbVU2bmObi5eegtc3PzQGbmOSONiEz9txcGfgtSU2jzDMr0KJWOSE1Ngafn/y52SE1NgVLpaMRRyVdNv70YOtm6n6Y+2WLgEzVToaGzEBo6XZrDP3ToAEJDpyMiItLYQ6N7mMrJVosL/J0znsX1Nf+o83b1PRw7Zzxbzy2pOjx+/1M5Tx8RMRtjx46GUumIiIjIZjt/T82fQgjRPL6KpRr1+cOr+n44+SBXeZjCh6HN6SyjJjx+1TOV41dfLbm+pqzN0B9e8X74REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMt7vbIgOl/Kw0RUWNocYHfEr6VhoioMbS4wK+JqXwrDRFRY+AcPhGRTDDwiYhkolaBn5iYCB8fHwwfPhxff/11lf79+/fDz88Pfn5+CA8PR2lpKQAgJycHf//73+Hn54eJEyfil19+adjRExFRrRkMfK1WC7VajY0bNyIhIQGbNm1CVlaW1F9cXIx58+ZBrVYjMTERTk5OUKvVAID58+cjICAAiYmJCA8PR2hoaONVQkRENTIY+IcPH4a7uzs6dOiANm3awNvbG0lJSVJ/Tk4O7O3t0aNHDwCAl5cX9uzZAwA4e/YsRowYAQDo06cP8vPzkZub2xh1EBGRAQav0snPz4eNjY3UtrW1RXp6utTu1q0b8vLykJGRAScnJ+zcuROFhYUAgKeffhrbt29HYGAgUlJS8Pvvv6OgoAAODg6NUAq1FPw7CqLGYTDwdTodFAqF1BZC6LWtrKwQExODyMhI6HQ6jBs3DhYWFgCApUuXYvHixdiwYQNUKhWcnJykvtqwtm5Xl1oemI1N+ybdX1MzhfqEEPXaTqFQ1HtbU2EKx+9BtOT6mkttBgPfzs4OaWlpUrugoAC2trZSu6KiAnZ2doiLiwMApKenS2fwt2/fxkcffQRLS0uUl5dj06ZNeOyxx2o9uKKiEuh0TfMmbunX4bf0+gC06Ppa+vFrifXFx8dBo1mGzMxzUCodERo6CwEBgY26TzMzRY0nygbn8AcPHoyUlBRcvXoVZWVlSE5OhkqlkvoVCgUmTZoErVYLIQTWr18PHx8fAIBarcbevXsBAP/5z3/g4uKCjh07PmhNRETNWnx8HKKjFyM6+n3cuHED0dHvIzp6MeLj44w6LoOB37lzZ4SFhSEoKAj+/v7w9fWFq6srQkJCcOrUKZiZmSEqKgrBwcEYMWIErKysMHnyZADArFmz8MUXX2DUqFHYvXs33n333UYviIjI2DSaZdBoYuHpqYKFhQU8PVXQaGKh0Swz6rgUohlPfHJKp+G09Ppsba1qvI+SqWvpx6+l1Wdn1wG5uQWwsLCQaisvL4eDgw3y8n5vtP0amtKR1b10yLQZuvnd/a7u4c3vqKkplY5ITU2Bp+f/pr9TU1OgVDoacVQMfDIhvPkdmYrQ0FkIDZ0OjSYWvr7DcejQAYSGTkdERKRRx8XAJyJqYJVX40REzMbYsaOhVDoiIiKy0a/SMYQ3TyNqxuLj46BSucHc3BwqlZvRr/Kg2gsICMSBA6moqKjAgQOpRg97gGf4RM1W5aV9ldMC332XjNDQ6QDQLMKDTA/P8ImaqeZ6aR+ZLgY+UTOVmXkObm4eesvc3DyQmXnOSCMiU8fAJ2qmKi/tu1tzuLSPTBcDn6iZqry079ChAygvL5cu7QsNnWXsoZGJ4oe2RM1Uc720j0wXA5+oGQsICERAQCD/sIwaBKd0iIhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCZqFfiJiYnw8fHB8OHD8fXXX1fp379/P/z8/ODn54fw8HCUlpYCAP744w+EhIRg9OjRGDt2LM6ePduwoyciolozGPharRZqtRobN25EQkICNm3ahKysLKm/uLgY8+bNg1qtRmJiIpycnKBWqwEAn3/+OZRKJbZt24bXX38dUVFRjVcJERHVyGDgHz58GO7u7ujQoQPatGkDb29vJCUlSf05OTmwt7dHjx49AABeXl7Ys2cPAECn00ln+2VlZWjdunVj1EBERLVg8AtQ8vPzYWNjI7VtbW2Rnp4utbt164a8vDxkZGTAyckJO3fuRGFhIQBg0qRJGD9+PDw9PVFaWop169Y1QglERFQbBgNfp9NBoVBIbSGEXtvKygoxMTGIjIyETqfDuHHjYGFhAQBYvHgxJkyYgKCgIBw/fhxhYWHYvn072rZtW6vBWVu3q2s9D8TGpn2T7q+psT7TxvpMV3OpzWDg29nZIS0tTWoXFBTA1tZWaldUVMDOzg5xcXEAgPT0dDg4OAAA9u7dK83b9+3bF9bW1sjOzoarq2utBldUVAKdTtS+mgfQ0r9CjvWZNtZnupqyNjMzRY0nygbn8AcPHoyUlBRcvXoVZWVlSE5OhkqlkvoVCgUmTZoErVYLIQTWr18PHx8fAICTk5M0n5+Tk4P8/Hw8+eSTD1oTERHVg8Ez/M6dOyMsLAxBQUEoLy/H2LFj4erqipCQEMycORMuLi6IiopCcHAwbt26BQ8PD0yePBkAsHTpUixcuBBr166FpaUlYmJi0L598/jVhohIbhRCiKaZM6kHTuk0HNZn2lif6TKpKR0iImoZGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BOR0cTHx0GlcoO5uTlUKjfEx8cZe0gtmsG/tCUiagzx8XGIjl4MjSYWvr7D8d13yQgNnQ4ACAgINPLoWiae4RORUWg0y6DRxMLTUwULCwt4eqqg0cRCo1lm7KG1WAx8IjKKzMxzcHPz0Fvm5uaBzMxzRhpRy8fAJyKjUCodkZqaorcsNTUFSqWjkUbU8jHwicgoQkNnITR0Og4dOoDy8nIcOnQAoaHTERo6y9hDa7H4oS0RGUXlB7MREbMxduxoKJWOiIiI5Ae2jYiBT0RGExAQiICAwBZ9e+TmhFM6REQywcAnIpIJBj4RkUww8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSiVrdWiExMRGrV6/G7du38corr2DChAl6/fv378eyZXfuYa1UKhEVFYW2bdsiICAAFRUVAIAbN24gNzcXBw4cwKOPPtrAZRARkSEGA1+r1UKtViM+Ph6WlpZ48cUX4ebmhh49egAAiouLMW/ePGzYsAE9evTA2rVroVarsWDBAsTHx0uPM2fOHDz//PMMeyIiIzE4pXP48GG4u7ujQ4cOaNOmDby9vZGUlCT15+TkwN7eXvoB4OXlhT179ug9RkpKCjIyMhASEtLAwyciotoyGPj5+fmwsbGR2ra2ttBqtVK7W7duyMvLQ0ZGBgBg586dKCws1HuMFStWICwsDObm5g01biIiqiODUzo6nQ4KhUJqCyH02lZWVoiJiUFkZCR0Oh3GjRsHCwsLqf/8+fO4du0avLy86jw4a+t2dd7mQdjYtG/S/TU11mfaWJ/pai61GQx8Ozs7pKWlSe2CggLY2tpK7YqKCtjZ2SEuLg4AkJ6eDgcHB6l/z5498PHxqdfgiopKoNOJem1bVy31ftzx8XHQaJYhM/MclEpHhIbOapFfMNFSj18l1me6mrI2MzNFjSfKBqd0Bg8ejJSUFFy9ehVlZWVITk6GSqWS+hUKBSZNmgStVgshBNavX68X8CdOnMCAAQMesAyqj/j4OERHL0Z09Pu4ceMGoqPfR3T0YsTHxxl7aERkBAYDv3PnzggLC0NQUBD8/f3h6+sLV1dXhISE4NSpUzAzM0NUVBSCg4MxYsQIWFlZYfLkydL2ubm56Ny5c6MWQdXTaJZBo4mFp6cKFhYW8PRUQaOJhUazzNhDIyIjUAghmmbOpB44pfNg7Ow6IDe3ABYWFlJ95eXlcHCwQV7e78YeXoNqicfvbqzPdJnUlA6ZLqXSEampKXrLUlNToFQ6GmlERGRMDPwWLDR0FkJDp+PQoQMoLy/HoUMHEBo6HaGhs4w9NCIyglrdWoFMU+XVOBERszF27GgolY6IiIhskVfpEJFhDPwWLiAgEAEBgS16jpSIaodTOkREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpIJBj4RkUww8ImIZKJWgZ+YmAgfHx8MHz4cX3/9dZX+/fv3w8/PD35+fggPD0dpaSkAoKSkBOHh4fD394e/vz/OnDnTsKMnIqJaMxj4Wq0WarUaGzduREJCAjZt2oSsrCypv7i4GPPmzYNarUZiYiKcnJygVqsBAO+++y66dOmChIQEvPnmm1i0aFGjFUJERDUzGPiHDx+Gu7s7OnTogDZt2sDb2xtJSUlSf05ODuzt7dGjRw8AgJeXF/bs2QMhBJKTkzFlyhQAgEqlQnR0dCOVQUREhhgM/Pz8fNjY2EhtW1tbaLVaqd2tWzfk5eUhIyMDALBz504UFhaiqKgIlpaW2LhxI8aPH4+goCBUVFQ0QglERFQbrQytoNPpoFAopLYQQq9tZWWFmJgYREZGQqfTYdy4cbCwsEBFRQUKCwvRvn17bNq0CT/++CP++c9/Yu/evbUenLV1uzqW82BsbNo36fKjF7UAAA9HSURBVP6aGuszbazPdDWX2gwGvp2dHdLS0qR2QUEBbG1tpXZFRQXs7OwQFxcHAEhPT4eDgwM6duyIVq1awdfXFwAwZMgQ/PnnnygqKoK1tXWtBldUVAKdTtSpoPqysWmPgoLrTbIvY2B9po31ma6mrM3MTFHjibLBKZ3BgwcjJSUFV69eRVlZGZKTk6FSqaR+hUKBSZMmQavVQgiB9evXw8fHB5aWlhg8eDC2b98OADhx4gQefvhhdOzYsQHKIiKiujJ4ht+5c2eEhYUhKCgI5eXlGDt2LFxdXRESEoKZM2fCxcUFUVFRCA4Oxq1bt+Dh4YHJkycDAJYsWYKFCxdi48aNaNWqFdRqNczMeOk/EZExKIQQTTNnUg+c0mk4rM+0sT7TZVJTOkRE1DIw8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCQY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQTDHwiIplg4BMRyQQDn4hIJhj4REQywcAnIpKJWgV+YmIifHx8MHz4cHz99ddV+vfv3w8/Pz/4+fkhPDwcpaWlAICjR4/Czc0NY8aMwZgxYzB//vyGHT0REdVaK0MraLVaqNVqxMfHw9LSEi+++CLc3NzQo0cPAEBxcTHmzZuHDRs2oEePHli7di3UajUWLFiA06dPY9KkSZg6dWqjF0JERDUzeIZ/+PBhuLu7o0OHDmjTpg28vb2RlJQk9efk5MDe3l76AeDl5YU9e/YAAE6dOoVDhw7Bz88P06ZNw2+//dZIZRARkSEGAz8/Px82NjZS29bWFlqtVmp369YNeXl5yMjIAADs3LkThYWFAID27dtj4sSJSExMxDPPPIOwsLCGHj8REdWSwSkdnU4HhUIhtYUQem0rKyvExMQgMjISOp0O48aNg4WFBQAgKipKWu+ll17CBx98gOvXr6N9+/a1Gpy1dbtaF9IQbGxqNy5TxfpMG+szXc2lNoOBb2dnh7S0NKldUFAAW1tbqV1RUQE7OzvExcUBANLT0+Hg4ACdTodPPvkEU6ZMgbm5ubT+3f9vSFFRCXQ6Uev1H4SNTXsUFFxvkn0ZA+szbazPdDVlbWZmihpPlA1O6QwePBgpKSm4evUqysrKkJycDJVKJfUrFApMmjQJWq0WQgisX78ePj4+MDMzw+7du7Fr1y4AQEJCAnr37o02bdo0QFlERFRXBgO/c+fOCAsLQ1BQEPz9/eHr6wtXV1eEhITg1KlTMDMzQ1RUFIKDgzFixAhYWVlh8uTJAICYmBh8+eWXGDVqFDZv3ox33nmn0QsiIqLqKYQQTTNnUg+c0mk4rM+0sT7TZVJTOkRE1DIw8ImIZIKBT0QkEwx8IiKZYOATEckEA5+ISCYY+EREMsHAJyKSCdkHfnx8HFQqN5ibm0OlckN8fJyxh0RE1CgM3jytJYuPj0N09GJoNLHw9R2O775LRmjodABAQECgkUdHRNSwZH2Gr9Esg0YTC09PFSwsLODpqYJGEwuNZpmxh0ZE1OBkHfiZmefg5uaht8zNzQOZmeeMNCIiosYj68BXKh2Rmpqityw1NQVKpaORRkRE1HhkHfihobMQGjodhw4dQHl5OQ4dOoDQ0OkIDZ1l7KERETU4WX9oW/nBbETEbIwdOxpKpSMiIiL5gS0RtUiyDnzgTugHBAS26PtxExEBMp/SISKSEwY+EZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhk0nh7a6Laq1XgJyYmwsfHB8OHD8fXX39dpX///v3w8/ODn58fwsPDUVpaqtefl5eHQYMG4dKlSw0zaiL87/bW0dHv48aNG4iOfh/R0YsZ+kT3YTDwtVot1Go1Nm7ciISEBGzatAlZWVlSf3FxMebNmwe1Wo3ExEQ4OTlBrVZL/TqdDm+99RbKy8sbpwKSLd7emqhuDAb+4cOH4e7ujg4dOqBNmzbw9vZGUlKS1J+TkwN7e3v06NEDAODl5YU9e/ZI/Z9++ikGDx6Mjh07NsLwSc54e2uiujEY+Pn5+bCxsZHatra20Gq1Urtbt27Iy8tDRkYGAGDnzp0oLCwEAJw+fRpHjhzBq6++2tDjJuLtrYnqyODN03Q6HRQKhdQWQui1raysEBMTg8jISOh0OowbNw4WFhYoKyvDv/71L3z44YcwM6vfZ8PW1u3qtV192di0b9L9NbWWVt/ChZEID5+Bzz77DJ6enjh9Og3h4TOwZMmSFlcr0PKO371acn3NpTaDgW9nZ4e0tDSpXVBQAFtbW6ldUVEBOzs7xMXd+aAsPT0dDg4OSEtLQ1FREV577TUAd35TmDJlCmJjY/HUU0/VanBFRSXQ6USdCqqvln63zJZY37BhviguLsPrr/8TmZnnoFQ6Yu7cBRg2zLfF1doSj9/dWnJ9TVmbmZmixhNlg4E/ePBgrFy5ElevXsXDDz+M5ORkLF68WOpXKBSYNGkS4uLiYGtri/Xr18PHxwd//etf8f3330vrDR06FGvWrMFjjz32gCUR/Q9vb01UewbnWjp37oywsDAEBQXB398fvr6+cHV1RUhICE6dOgUzMzNERUUhODgYI0aMgJWVFSZPntwUYyciojpQCCGaZs6kHjil03BYn2ljfaarOU3p8C9tiYhkgoFPRCQTzfo7bc3MFIZXMuH9NTXWZ9pYn+lqqtoM7adZz+ETEVHD4ZQOEZFMMPCJiGSCgU9EJBMMfCIimWDgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDTrWys0pszMTPj5+WHFihXw9vYGcOee/a1bt4aFhQVu376NJ598EkuWLMEjjzxi5NFW7/bt21i7di22bdsGhUKBiooKPP/885g6dSrmz5+PI0eO4JFHHoFOp0Pr1q2xePFiODk54dKlS3j22Wcxfvx4REVFSY939uxZ+Pv7491330VAQIARK7u/S5cuYcSIEejevTuAO9/IVlpaCn9/fwQEBOj1Vfr444/RpUsXYwy3zu6tr9K4cePw7bffAgCys7Px+OOPw8LCAv369cPbb79tjKHWKDU1FeHh4di6dSusra0B3Pl+65MnT2LlypVISUnBRx99hIKCAuh0OvTs2RMRERGws7NDamoqpk2bhscffxxCCJSXl+PFF1/EK6+8YtR6YmNjsWHDBr3lNb0HFQoF5s2b17zeh0KmoqOjxcyZM8Wrr74qLfPy8hK5ublS+5133hExMTHGGF6tLFiwQEybNk388ccfQgghrl+/LoKCgsRXX30l5s6dKzZv3iytu3v3bvHCCy8IIYTIzc0VgwYNEn/729/E7du3pXWWLVsm3N3d9bZrbnJzc4WXl5fesry8PNG7d2+RlZVVpc/UVFffve59nTZXS5cuFdOmTRNCCPHzzz+L5557Tvzxxx/i2LFjYsiQIeL48ePSul999ZUICAgQQghx5MgR8fe//13qu379ulCpVOL8+fNNW8Bd7h1TpZreg0KIZvc+lOWUTnl5ORITExEaGoozZ87g119/rbJO5Znjo48+aoQRGpaXl4dt27Zh6dKlsLKyAgC0a9cOCxcurHbM169f11vetm1b9OzZE8eOHZOW/fjjjxg8eHDjD76BFRQUQAiBa9euGXsodJewsDBcuXIFX375JebOnYuYmBhYWVlh1apVeO2119CnTx9p3QkTJsDHxwe3bt2q8jg3b96Eubk52rdvHt8LW6mu70HA+O9DWU7p7N+/H/b29njyyScxbNgwbNq0CbNnzwYATJkyBRYWFigqKoK5uTmmT59u5NFWLz09Hd27d68y3dS9e3d0794dP/zwA1asWIEvvvgCZWVluHLlClavXq237siRI7Fr1y64u7sjPT0djo6OECZwL738/HyMGTMGN2/exLVr1+Di4oLY2FjY2dlJfZX8/PwQHBxsxNHW3b01AMB7770HR0dHI42ofiwtLbFs2TKMGTMGU6ZMQd++fQEAJ06cwLx586qsf/c35Z0+fRpjxoyBTqfDr7/+ipEjR+p9l3ZzYOg9WKk5vQ9lGfibN2+Gr68vAMDHxwezZs3CG2+8AQB637u7bt06TJ48GTt27IBC0fxu3Xr3mJKSkrB69WrodDpYWlriL3/5C2bOnCnNAf78888IDg7G1q1bpe2GDh0KjUYDnU6HnTt3YuTIkdixY4dRaqkLW1tbbN26FTqdDkuXLkV2djaGDBmCK1euSH2mrCXUUOnnn39Gx44dkZKSgunTp6NVqzuRU/kavHXrFgIDAwEAf/zxB5YvXw4A6NWrlzRfXlJSguDgYKxZswZTp041QhX3V9N7cPPmzQDQrN6HspvSKSoqwsGDB7Fu3ToMHToUCxYsQHFxMXbv3l1l3cDAQFy4cKFZThX06tUL2dnZKCkpAQCMGDECW7duxerVq6sdb79+/fD444/jzJkz0rK2bdvCyckJP/30E44cOWJy0zlmZmaYM2cOtFotPvvsM2MPh+6RlZWFlStX4ptvvoGlpaV0Zuvi4oKff/4ZwJ3fArZu3YqtW7fCwcEB5eXlVR6nXbt2GDlypLRNc1HX9yBg/Peh7AJ/69atcHd3x4EDB/D999/jhx9+wLRp0/DNN99UWTclJQVdunRBp06djDDSmtnb22P06NGYO3cuiouLAdy5YmDfvn0wM6t6WC9fvoxLly7ByclJb/nIkSPxwQcfoFevXtLZlylp1aoV5syZg1WrVqGwsNDYw6H/unnzJsLCwjB79mw4ODhg6dKl+Oqrr3DixAnMmDEDH330EU6ePCmtn5GRgdzcXJibm1d5rIqKChw9ehRPP/10U5ZgUF3fg4Dx34em9w5/QFu2bEFYWJjesgkTJuDTTz9Fu3btpDl8MzMzmJubS79iNkeLFi3C559/jqCgIFRUVKC0tBRubm5Yu3YtPvnkE2nu0NzcHDdv3sTcuXPRrVs3XLp0SXoMLy8vvPXWW9KUlilSqVTo27cvPvzwQ2MPpUFUN4c/cOBALFiwwEgjqrvo6Gj85S9/kero2rUr5s+fj9mzZyMhIQFqtRoajQaFhYX4888/0aVLF8ydOxcDBgxAamqqNIevUChw+/ZtODo6IiQkxKg1paWlSZ9DAHc+H6rpPVipOb0P+Y1XREQyIbspHSIiuWLgExHJBAOfiEgmGPhERDLBwCcikgkGPhGRTDDwiYhkgoFPRCQT/w/tJM0YpsC8EgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compare Algorithms\n",
    "\n",
    "fig = plt.figure()\n",
    "fig.suptitle('Ensemble Algorithm Comparison') \n",
    "ax = fig.add_subplot(111) \n",
    "plt.boxplot(results) \n",
    "ax.set_xticklabels(names)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Tuning Hyperparameters\n",
    "- For ensmeble models there are always hyperparameters we can fine tune to yield better model performance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.992714 using {'learning_rate': 0.1, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled ABM\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([300, 400, 500, 600]), learning_rate = np.array([1,0.1, 0.01]))\n",
    "model = AdaBoostClassifier(random_state=seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.991378 (0.014802) with: {'learning_rate': 1.0, 'n_estimators': 300}\n",
      "0.992221 (0.015169) with: {'learning_rate': 1.0, 'n_estimators': 400}\n",
      "0.991803 (0.017905) with: {'learning_rate': 1.0, 'n_estimators': 500}\n",
      "0.991897 (0.017266) with: {'learning_rate': 1.0, 'n_estimators': 600}\n",
      "0.992466 (0.014103) with: {'learning_rate': 0.1, 'n_estimators': 300}\n",
      "0.992550 (0.014541) with: {'learning_rate': 0.1, 'n_estimators': 400}\n",
      "0.992182 (0.015725) with: {'learning_rate': 0.1, 'n_estimators': 500}\n",
      "0.992714 (0.014159) with: {'learning_rate': 0.1, 'n_estimators': 600}\n",
      "0.973190 (0.014813) with: {'learning_rate': 0.01, 'n_estimators': 300}\n",
      "0.981683 (0.014084) with: {'learning_rate': 0.01, 'n_estimators': 400}\n",
      "0.984168 (0.015045) with: {'learning_rate': 0.01, 'n_estimators': 500}\n",
      "0.986146 (0.015300) with: {'learning_rate': 0.01, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaBoostClassifier(algorithm='SAMME.R', base_estimator=None, learning_rate=0.1,\n",
       "                   n_estimators=600, random_state=7)"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = AdaBoostClassifier(random_state=seed, learning_rate = 0.1, n_estimators=600)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9836448598130841\n",
      "\n",
      "Accuracy\n",
      "0.9743589743589743\n",
      "\n",
      "Confusion Matrix\n",
      "[[214   0]\n",
      " [  6  14]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.97      1.00      0.99       214\n",
      "         1.0       1.00      0.70      0.82        20\n",
      "\n",
      "    accuracy                           0.97       234\n",
      "   macro avg       0.99      0.85      0.90       234\n",
      "weighted avg       0.98      0.97      0.97       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3iT5dfA8W/SvaArpYyyN5SylFW2gEKLMgRkCoiiIr74A1FAUBwMQYYoW5wIiEyRUmSJsrcsC5RN6aQj3Wme949KILTQtJCmbc7nuryuPvvkNjwnzzj3rVIURUEIIYQwkdrSAQghhCheJHEIIYTIF0kcQggh8kUShxBCiHyRxCGEECJfJHEIIYTIF1tLByDE4xo0aBCHDh0ymmdjY0OpUqVo1KgRY8eOpVq1akbL4+PjWbZsGdu3b+fWrVuULl2a+vXrM2TIEFq0aJHrcf744w9+/vlnzp49S1paGpUqVeLFF1+kT58+2NnZme3zCVHUqKSOQxR3gwYNQqfTMX78eMO8jIwMzp8/z1dffYWTkxPbtm3DwcEBgCtXrjB06FB0Oh1Dhw6lXr16xMfHs2HDBnbu3MmoUaN46623jI7x0UcfsWrVKl544QU6duyIs7Mzhw4d4ptvvqFt27bMnTsXGxubQv3cQliMIkQxN3DgQOXVV1/Nddnq1auVmjVrKrt371YURVF0Op0SFBSkdOrUSYmNjc2x/ty5c5WaNWsqO3bsMMxbv369UrNmTWXVqlU51t+yZYtSs2ZNZf369U/o0whR9MkzDlGiubq6Gk3v2rWLsLAwxo4di6enZ471R40aRcWKFVm0aJFh3vLly6lVqxZ9+/bNsX7Xrl0ZNmwYHh4ej4xj9erVdOvWjQYNGvDss8+yZs0aw7IOHTowdepUo/U//fRTOnToYJiuVasWixYtolu3bjRr1oyQkBBq1arF8ePHjbb78ccfCQgIIDk5GYDTp08zZMgQAgICaN68OR9//DGpqamPjFWIvMgzDlEiKIqCTqczTKenp3P69GnmzJlDuXLlaNq0KQB///03arWawMDAXPdjY2NDx44dWbFiBXFxceh0OsLCwnj11Vcfeuz7b5HlZsWKFcyYMYOXX36ZNm3acOjQIT744AOcnZ0JCgoy+TMuWLCACRMm4OHhQdOmTfH19SUkJIRGjRoZ1tm6dSvt27fHxcWFixcvMnDgQBo2bMjcuXOJjY1l9uzZ3Lhxg8WLF5t8XCEeJIlDlAh79uyhXr16RvMcHR1p0aIF77//Pi4uLgDcvHkTDw8PnJ2dH7qvChUqABAREUFWVhYA5cqVK1Bcer2eRYsW0bNnT9577z0AWrZsyfXr1zl69Gi+EkerVq3o37+/Ybpr165s3bqV9957D5VKRVRUFMeOHePLL78E4Ouvv8bLy4slS5Zgb28PQOXKlRkwYACHDx/mqaeeKtBnEkIShygRmjRpwvvvvw/AhQsXmDFjBi1atGDmzJmGkyZkX5nk9RD7/uV3/9br9QWK6/Lly8THxxvddgKYPXt2vvf14JthQUFBfPPNN5w8eZKGDRsSEhKCq6srbdq0AeDgwYN07NgRtVptuBpr2LAhrq6u7N+/XxKHKDBJHKJEcHNzw9/fHwB/f3/Kli3L0KFDsbe3Z+bMmYb1ypcvz/79+0lPTze8ZfWgmzdvAlC2bFnDvIiIiIceOyoqCm9vb9TqnI8M4+PjAXJ9npJfXl5eRtP16tWjSpUqhISE0LBhQ7Zu3UqnTp0MiTI+Pp7Vq1ezevXqHPuKjo5+7HiE9ZKH46JEatGiBb1792bjxo3s3LnTML99+/ZkZmaya9euXLdTFIWdO3fi7++Pp6cnnp6e1K1bl7179z70WEOHDmXo0KG5LnNzcwMgLi7OaP7ly5c5duyYYfrBK5qUlJRHf8D/BAUFERoaSmRkJMePHze69eXq6kqvXr1Yu3Ztjv9ee+01k/YvRG4kcYgS65133sHNzY3p06eTkZEBQGBgIP7+/sycOTPXX92LFy/m0qVLRg/DhwwZwvnz5/nll19yrL9x40YuXrxIcHBwrjFUrVoVd3d3du/ebTR/3rx5hishV1dXoqKiDMv0en2Ot6UeJigoiJs3b7Jw4UK8vb1p1qyZYVmTJk0IDw+nfv36+Pv7G67EZs+ezYULF0zavxC5sfnwww8/tHQQQjyO9evXY2dnl+Pk7eTkhEqlYuvWrbi5udG4cWPUajXNmzdn8+bN/Pjjj6jVajIyMjh37hzz58/nhx9+YOTIkQwcONCwn1q1anHp0iWWLVtGZGQkarWaq1ev8tNPPzFv3jy6dOnCmDFjUKlUOWJTq9XY2NiwbNky0tPTAVi3bh2rVq1i0qRJVK1alZs3b7JhwwZKlSpFSkqK4cRub2/PkCFDgOw3qlq1akXjxo2N9n83Ke3Zs4cePXrQrl07w7JKlSrx9ddf8++//+Ls7MyFCxeYPHkyV65cYfTo0TleVRbCVFI5Loq9QYMG4ezsnOsrphkZGTz77LMkJiYSGhpqeNaQkJDA999/T0hICDdu3MDFxYUGDRowePBgWrZsmWM/er2eNWvWsG7dOq5evUpGRgZVqlShT58+9O7dG1vbRz8u/Omnn/juu++IiIigcuXKvP7663Tt2hUArVbLxx9/zPbt27G1taV79+6UK1eOH3/80XCbrVatWrz77rsMHz48x76//fZbpk2bxurVq2nYsKHRsqNHjzJ37lxOnTqFg4MDjRs35n//+x81atQwrXGFyIUkDiGEEPkizziEEELkiyQOIYQQ+SKJQwghRL4UWuLQarUEBQVx48aNHMvOnTtHz5496dKlCxMnTjTqc0gIIUTRUiiJ4+TJk7z00ktcuXIl1+Xjxo1j8uTJbNu2DUVRjHoOFUIIUbQUSpcja9asYcqUKbz77rs5lt28eZO0tDTDa4Q9e/Zk/vz5Rp25meLOnWT0egUvL1diY7VPJO7iTNohm7TDPdIW2aQdQK1W4eHhUuDtCyVxfPrppw9dFhUVhUajMUxrNBoiIyPzfQy9XkGvVwx/F3W/n7zBuiNXScvMsnQoQggrU6a0E9+P7pD3ig9h8U4O9Xq9UcWtoii5VuDmxcvrXhWsRuP2RGIzp/VHJWkIIYoniycOX19foz6DYmJi8PHxyfd+YmO16PUKGo0b0dFJTzJEs0jNkKQhhCieLJ44ypcvj4ODA0ePHqVJkyZs3LjRMJ6Atfhx5JP/vMUlgZqbtMM90hbZrKUdbsensmxPGOcjEozmt6/jy8BW1R9r3xZLHCNGjGD06NH4+/sza9YsJk2ahFarpV69egwePNhSYQkhRLGmy9Lz+6kbrD9ylcyse897y5RyZHjbmtQt745anf/HAfcr1MRx/7gIS5cuNfxdu3Zt1q5dW5ihCCFEiXM5Oollu8O4GptsmKdWQdcAP3o2rYi97aNHvzSVxW9VCSGEeDI2HL1mlDQqe7vySruaVPZ+sl3oS+IQQogSYnBgdc7cjCdLr9DrqUo816ACNo95Wyo3kjiEEKIYSk7PxM5GbXT7ycvVgTc61qachzO+pZ3Mdmzp5FAIIYqZQ+HRvLv6KOuPXsuxrHFlL7MmDZArjnyTim8hhKXcSU7nu78ucuRyLABbTlyneTUNlZ7wM4y8SOLIpyedNBztnsxbDkKIkktRFHafu83PB8JJua94uJSTPdr0zEKPRxJHPj3ppNGzaaUntj8hRMlzOyGV5XvCOHcrZyFfv+ZVcXEo/NO4JI7HYI6KbyGEAMjSK4Zb45lZesN839JODGtTg7rl3S0WmyQOIYQoYu4kpzNr6xmuxtzr/l2tgm4N/ejR5MkV8hWUJA4hhChiSjnZc3/1hbkK+QpKXscVQogixkatYkS7mjja2dCveRU+6tmoyCQNkCsOIYSwqOR0HTvPRtCtYQXU941FVMnblfkDm+FsgYffeSl6EQkhhJU4HB7Dt39dJCElAyd7G56pV85oeVFMGiCJQwghCt2DhXwAqw5cpllVDW5OdhaMzDSSOIQQopA8rJCvtLM9LwdWLxZJAyRxmOzu+9RCCFEQDyvka1fbl5daWKaQr6CKT6QW9mBXI9JViBDCFA8r5PMp5cgr/43IV9xI4jDRg0lDugoRQphiy4nrrDl0xTCdPSJfBXo2rWTxQr6CksRRAMuGt7J0CEKIYqJT/XLsOBtBrDadSl4uvNKuJlU0bpYO67FI4hBCiCdIr1dQ3zfqnpO9LcPa1OBabDLPNSiPrU3xr7uWxCGEEE9AcrqOn/eHk6HL4o1n6hgtC6joSUBFTwtF9uRJ4hBCiMd0+HIM3+29SHxKBgAtavjQqJKXhaMyH0kcQghRQHeS0/n+r0scvhxjNP/UtTuSOIQQQtyjKAp7zt9m5f7LpGToDPPvFvI9VdXbgtGZnyQOIYTIh9sJqXyzJ4yzuRbyVcHFoXhUfz8OSRxCCGECvZJdyPfr4ZyFfMPb1qBeeQ8LRle4rDpx3K3mfJLjiAshSiYVEHY70ZA07hby9WhSCQcr60nCqhNHQZKGdDUihHVSqVS8HFidc7fi8XFzLBGFfAVl1YmjIElDuhoRwjr8G5FAFY2rUbcgnq4OTOweQAUP5xJRyFdQVp047vfjyDaWDkEIUQQkp+tYdSCcXeduE9TQj37NqxgtL0pDuFqKJA4hhPjPg4V8v5+8TrNq3lZ7S+phJHEIIaxefEoG3/11kcPhxoV8jSp5UdrJ3kJRFV2SOIQQVuuhhXxOdgz5r5BPpVI9Yg/WSRKHEMIqRSaksvzPC5y9GW80v23tMrzUvCqujiW/kK+gCiVxbN68mYULF6LT6RgyZAgDBgwwWn7mzBkmT55MZmYmZcuW5fPPP6dUqVKFEZoQwgqFRyXxyaaTZOiMC/mGtalB/QrWU8hXUGZ/nywyMpI5c+awcuVKNmzYwOrVq7l48aLROp9++imjR49m06ZNVKlSheXLl5s7LCGEFavk7Up5D2cAVP8V8k17sYkkDROZ/Ypj3759NG/eHHf37HF1u3TpQkhICKNGjTKso9frSU5OBiA1NZXSpUs/9nGlKlwI8TA2ahWvtK3Jsj1hDG1dg6o+8tZUfpg9cURFRaHRaAzTPj4+nDp1ymid9957j2HDhvHZZ5/h5OTEmjVr8n0cL69771ZrNG6sP2p60nCyt0FTAl+3K4mfqSCkHe6xxrY4dTWWkOPXGft8AOr/HnRrNG5oNG40qe0rD78LwOyJQ6/XG/2PURTFaDotLY2JEyfy7bff0qBBA1asWMH48eNZsmRJvo4TG6tFr1fQaNyIjk4iNcO0pOFoZ0OPJpWIjk7K1/GKurvtYO2kHe6xtra4v5APoEJpJzrVL2d17ZAbtVpl9GM7v8yeOHx9fTly5IhhOjo6Gh8fH8N0WFgYDg4ONGjQAIC+ffsyb968JxqDVIULYV2OXI7h2/sK+QA2H79G+zq+Foyq5DD7w/GWLVuyf/9+4uLiSE1NJTQ0lDZt7p3IK1WqxO3btwkPDwdgx44d+Pv7mzssIUQJFJ+SwfzQs8zddtYoaTSp7MWHPRpZdf9ST5LZrzjKlCnDmDFjGDx4MJmZmfTu3ZsGDRowYsQIRo8ejb+/P9OmTeP//u//UBQFLy8vPvvsM3OHJYQoQRRF4c9/I/lpX7gU8hWCQqnjCA4OJjg42Gje0qVLDX+3bduWtm3bFkYoQogSRgr5Cp9UjgshirU/ztwyShpSyGd+kjiEEMVar6cqcyg8hrjkdJ5rUIFeTa1vRL7CJolDCFFsZOj0ZGZl4eJw7/aTo50Nr3eohb2tjRTyFRJJHEKIYuH8rXiW7blAJW9X3upUx2hZ7XLuForKOpW4xLF2fzg/7AmzdBhCiCckJV3HqoOX2Xk2AoDbCam0rOFDk8peFo7MepW4xPHjn2FGXY04yr1OIYqto/8V8t25rybD0c6GtPteuRWFr8Qljvu7GnG0s6Fn00oWjEYIURAJ/43Id+iBEfkaV/bi5cDqeLo6WCgyASUwcdxv2fBWlg5BCJEPdwv5Vu4PJzn93lVFqf8K+Z6WQr4ioUQnDiFE8ZGemcWckDOcfqCQr3WtMgxoIYV8RYkkDiFEkWBvqzaqv9C4OTK8rRTyFUWSOIQQRYJKpeLl1tX5NyKB1rXK0OupyvJySxGV78SRkZGBvb29OWIRQliJDJ2e0H9u0ql+OaOrDA8XB2b3fxoXB/lNW5SZ1MewTqdjwYIFtG7dmmbNmnH9+nUGDhxIdHS0ueMTQpQw5yMSmLj2KKsOXubXI1dzLJekUfSZlDhmz57NoUOHmD17Nra2tmg0Gnx9fZkyZYq54xNClBApGTpW/HmBTzaeJCI+FYCtp25wPS7ZwpGJ/DIptW/ZsoWNGzfi4eGBSqXC0dGRqVOn0r59e3PHJ4QoAY5diWXF3gvcSTYu5OvXvArlPZwtGJkoCJOvCW1sjB9SZWVl4eBQtIpwfj95w9IhCCHuk5CSwfd/X+LgJePb2o0qefJy6xp4SSFfsWRS4ujcuTNjxoxh3LhxKIrCjRs3mDVrFh07djR3fPmy7r77pfI2hhCWoygKe8OyR+R7sJBvcKvqNKsmhXzFmUnPOMaNG0fFihXp168fSUlJdO3aFRcXF8aOHWvu+PLl/j6qpKsRISznUHgMS3aFGSWN1rXKMKNvU5pX10jSKOZMuuJIS0tjypQpTJkyhbi4ONzd3VGr1Vy/fh0XFxdzx1ggXQMqWDoEIazWU1W8qapxIzw6CY2bA8Pa1MTfTwr5SgqTrjjufwju6emJWq1Gp9PxwgsvmC0wIUTxodcrRtNqtYpX2tXguQblmdanqSSNEuahVxw3btygX79+ZGVlkZKSQosWLYyWp6enU716dbMHKIQoujJ0ejYcvcq/txOYGByAWn3vFlRFL1cGtHS1YHTCXB6aOCpUqMDSpUtJTEzk1VdfZd68eUbL7e3tqVWrltkDFEIUTecjEli+J8xQk7H9zC26+Je3cFSiMDzyGUedOtnDM+7atQtPT88cy/V6vXmiEkIUWSkZOlYfuMyO/0bku+vktTg61y8nD76tgEkPxxMSEpgxYwaRkZGGZKHT6bh69Sp///23WQMUQhQdjyrk61C3rCQNK2FS4pgwYQIeHh6UK1eOmzdv8vTTT7Nq1Sr69+9v7viEEEWAFPKJ+5n0VtW5c+f44osvGDZsGABvvvkmX375JTt27DBrcEIIy/srLJJ3Vx8xShqlHO0Y9Uxt3nm2niQNK2TSFYe7uzsODg5UrFiRixcvAtCwYUOuX79u1uCEEJZ3JVprXMhXswz9W1bFTUbks1omXXHUqVOHL774AkVR8PHxYfv27ezduxcnJydzxyeEsLDeT1dG4+aAxs2B8d3q81qHWpI0rJxJVxwTJ05k0qRJxMfHM3bsWN566y0yMjL48MMPzRyeEKIwXY9Nxs3JDnfne4O1OdrZMPa5+ni5OUofcAIwMXEcP36cRYsW4ejoSJkyZTh48CCZmZk4O0t3yEKUBBk6PRuPXeO3E9dpUtmL0Z3rGi0v71k0uxYSlmHSraqPP/4YW9t7OcbOzk6ShhAlxL8RCUxae5SNx66RpVc4FB7Dkcsxlg5LFGEmXXF07NiRxYsXExwcjLe3cXfI8pxDiOIpJUPHmoOX+eOMcSFfTd9SlHWXH4bi4UxKHNu3b0er1bJgwQLDPEVRUKlUnDt3zmzBCSHM4/jVWFb8eYG4Bwr5+jarQsd6ZVFLIZ94BJMSx8aNG80dhxCiECSkZvDD35c4cNG4kK9hRU+GtqmOl6ujhSITxYlJiaN8+cfruGzz5s0sXLgQnU7HkCFDGDBggNHy8PBwpkyZQkJCAhqNhi+++ILSpUs/1jGFEMZiEtMYv+oI2vtH5HO0Y1BgNZpXk8GVhOlMejj+OCIjI5kzZw4rV65kw4YNrF692lBECNm3vF5//XVGjBjBpk2bqFOnDkuWLDF3WEJYHe9SjtQp726YDqzpw4x+TWlR3UeShsgXk644Hse+ffto3rw57u7ZX9guXboQEhLCqFGjADhz5gzOzs60adMGgJEjR5KYmGjusISwSoNbVSMqMY2+zSrTwC9nj9dCmMLsiSMqKgqNRmOY9vHx4dSpU4bpa9eu4e3tzYQJEzh37hxVq1blgw8+yPdxvLyMB4zRaNwKHnQJIW2QzRrb4XJkIj/tvcjY7g1wtL/3z7xmZW8Wj2xj9VcY1videJJMThyHDh1i9erVREdHM2fOHH766SfefPNNbGweXUmq1+uNvqR338a6S6fTcejQIX788Uf8/f2ZO3cu06dPZ/r06fn6ILGxWqPp6OikfG1f0mg0blbfBmB97ZCZlV3It/n4dbL0Cm72NvRvURWwvrZ4GGmH7KF9H/yxna/tTVlp3bp1jB07lipVqnDmzBlUKhXbt29n5syZeW7r6+tLdPS9Nziio6Px8fExTGs0GipVqoS/vz8AQUFBRlckQgjThEUkMPGXo2w4ml3IB/DHmVskpGbksaUQ+WNS4li8eDFLly5l1KhRqNVqPD09Wbp0KVu2bMlz25YtW7J//37i4uJITU0lNDTU8DwDoFGjRsTFxXH+/HkAdu7cSb169Qr4cYSwPqkZOr7be5GPN57k1n/DuEJ2Id/HvRpT2sn+EVsLkX8m3aqKj4+nevXqAIbbTN7e3mRmZua5bZkyZRgzZgyDBw8mMzOT3r1706BBA0aMGMHo0aPx9/fnq6++YtKkSaSmpuLr62vSlYwQQgr5hGWYlDgaN27M/PnzGTNmjGHed999R8OGDU06SHBwMMHBwUbzli5davg7ICCAtWvXmrQvIQQk/lfIt18K+YQFmJQ4Jk+ezMiRI1m1ahVarZYOHTrg6OjI4sWLzR2fECIXBy5FGyUNN0c7BrWqRovqUsgnzM+kxFG2bFnWr1/PqVOniIiIQKPR0LBhQ6Mec4UQheeZuuX4OyyKS1FJBNb0YUCLarg5yeBKonCYdObv168fzz//PM8995zJt6eEEE+GXq+QlJ5p9JBbrVYxol1NYrXpBFSUQj5RuEx6q6p3796EhobSrl073nzzTUJDQ016MC6EeDzXY5P5aMMJvth6Bv1/r9jeVcHTRZKGsAiTE8eKFSvYvn07Tz/9NN988w2BgYFMmTLF3PEJYZUys/SsPXyFSb8e41JUEpeikgg9fdPSYQkB5LPLEY1Gw1NPPUVMTAy3b9/mxIkT5opLCKsVFpHAsj1hRjUZtmoVugeuOISwFJMSx6VLl9iyZQtbtmwhOTmZoKAgFi5cSJ06dcwdnxBWIzVDx5qDV/jjzC3uTxE1ypTilbY1ZNxvUWSYlDh69epFhw4dmDBhAq1bt0atNntv7EJYFSnkE8WJSYnjr7/+wtW14B1iCSFypygKi3b+y98XoozmB1T0ZGjr6ni7SSGfKHoemTgGDRrEDz/8wJAhQx5aVCQV30IUnEqlotR99RdSyCeKg0cmjj59+gAwcODAQglGCGvU66nKHL0SS40ypRjYUgr5RNH3yMRxt3+p5OTkXJPHokWLzBOVECWQXq/wx9lbPFXFGw8XB8N8RzsbPu7VGBcH6YlBFA8P/abGxsZy+vRpAGbNmoWfn5/Rcq1Wy+LFixk5cqR5IxSiBLgRl8zS3WFcikri3M0E3u5S12i5JA1RnDz02+ri4sL8+fO5c+cO6enpfPTRR0bL7e3tJWkIkYfMLD2bjl1j038j8gEcvhzDiatxNKwkVd+ieHpo4nB0dOTXX38FYNSoUSxYsKDQghKiJAi7ncCyPRe4dSfFMM9GreL5xhWpX8HdgpEJ8XgeeX0cHh5O1apV+b//+z8uXryY6zp3B3gSQmRLzdCx5tAV/jgthXyiZHpk4ujduzfHjh0jKCgo1+UqlYpz586ZJTAhiqMTV+NYsfcCsdp0wzxHOxv6NKvMM/XKSSGfKBEemTiOHTsGYBgPXAjxcOFRSczaetpoXoCfB0Pb1JBCPlGimNx3yP79+wGIi4vjww8/ZN68eaSlpZktMCGKm6o+bjxd1RvILuR7o2NtxnatL0lDlDgmvQP4+eef8/vvv7Nr1y4mTpyIVqvF1taWDz/8kOnTp5s7RiGKJL1eQa02vvU0OLA6Lg629Hm6ihTyiRLLpMSxfft2Vq1aRVJSEn/++SchISF4enrSoUMHc8cnRJGj1ytsP3OLXeci+LBHIxztbAzL3J3tGd62pgWjE8L8TLpVFR8fT5kyZfj777+pUKECfn5+2NnZoSgyPoCwLjfikpm68QQ//H2JG3EprD10xdIhCVHoTLriqFOnDp999hmHDx/mmWeeIT4+ns8//5yAgABzxydEkZBbIR/A6Zt3yNDpsbeVoQaE9TDp2z59+nTi4+Np3Lgxb731FlevXiU2NpZPPvnE3PEJYXEXbicyae0x1h+9ZkgaNmoVPZpU5ONejSVpCKtj0hVH2bJlmTlzJunp6URFRVGtWjXp4FCUeKkZOn45dIXtDxTyVS/jxitta1JBCvmElTIpcWi1WqZMmcLWrVvR6/XY2NjQrl07ZsyYIQM8iRLp1PU4lu8xLuRzsFXTp1kVOtUrl+NtKiGsicm3qpKTk9myZQunTp1i06ZNZGVl8dlnn5k7PiEs4nZ8qlHSaODnwYy+TeniX16ShrB6Jl1x7N69m61bt+Lm5gZAtWrVmDFjBp07dzZrcEJYyjP1yrH/YjQRCSkMbFmNVjV8ZEQ+If5j8iAAWVlZRtN6vR47OylwEsVfTFIaGVl6yrk7G+ap1Spe71gLRzsbSjnZWzA6IYoek25VderUiTFjxnDu3Dni4+M5c+YM77zzDp06dTJ3fEKYjV6vEPrPTcavPsKiHf+i1xvXJfmUcpKkIUQuTEoc48aNw9PTkxdffJHmzZvz0ksvUb58ecaNG2fu+IQwi7uFfN//fYl0nZ7w6CS2/XPT0mEJUSyYdKvK2dmZ2bNn89lnn5GYmIi3t7fc7xXF0sMK+cp5OFO9TCkLRiZE8WFS4sjIyGDBggX89ttvxMTEULZsWXr06MGrr76KWi3FT6J4uHA7kWV7wrj5wIh83Rv50b1xRexs5LsshClMShyffPIJZ8+eZcKECfj6+nLz5k0WL16MVqtl7NixeW6/efNmFi5ciE6nY77YEYkAACAASURBVMiQIQwYMCDX9Xbv3s3UqVPZuXNn/j6FEI+QlpnFmoOXpZBPiCfEpMSxdetWtm3bhqenJwD169enUaNGdO/ePc/EERkZyZw5c1i3bh329vb069ePZs2a5RhyNiYmhhkzZhTwYwiRu/TMLCb8cpSoxHtjx0ghnxCPx6Rrc1dXVzIyMozm2draGuo6HmXfvn00b94cd3d3nJ2d6dKlCyEhITnWmzRpEqNGjTIxbCFM42Bnw1NVvA3TUsgnxOMz6YqjV69eDB8+nDfeeAM/Pz+ioqJYunQpLVq0YM+ePYb12rZtm2PbqKgoNBqNYdrHx4dTp04ZrfP9999Tt25d6W1XmEXPppU4H5FA5/rlaCmFfEI8NpMSx7p16wCYPXu20fzo6Gj++usvAFQqFTt27MixrV6vN/qHqiiK0XRYWBihoaF8++233L59O/+f4D9eXsZ9Zmk0eV8NlXTW1gZRCaks/eMcIzvXxeu+4VorlHPnq1dbS8LA+r4TDyPt8HhMShyP87Da19eXI0eOGKajo6Px8fExTIeEhBAdHU2vXr3IzMwkKiqK/v37s3LlynwdJzZWazQdHZ1U4JhLAo3GzWraQK8o/HHmFmsOXiEtM4vklAz+r0tdVCqVVbVDXqQtskk7ZPeM8OCP7Xxt/wRjyVXLli3Zv38/cXFxpKamEhoaSps2bQzLR48ezbZt29i4cSNLlizBx8cn30lDWK+bccl8vOEk3/91ibTM7G5xjl2J5VpssoUjE6LkMrmvqoIqU6YMY8aMYfDgwWRmZtK7d28aNGjAiBEjGD16NP7+/uYOQZRAuiw9m49fZ+Oxa+geKOR7pW0NKnlLd/9CmIvZEwdAcHAwwcHBRvOWLl2aY70KFSpIDYfI08XIRJbtDuOGFPIJYRH5ThwZGRnY20vHb6LwpWVm8cuhK4T+c9OokK+ajxuvtKuJnxTyCVEoTPppptPpWLBgAa1bt6ZZs2Zcv36dgQMHEh0dbe74hDD4NyKBbfclDQdbNQNbVmPKCw0laQhRiExKHLNnz+bQoUPMnj0bW1tbNBoNvr6+TJkyxdzxCWEQUNGT5tWya4L8K3gwvU9Tnm0ghXxCFDaTblVt2bKFjRs34uHhgUqlwtHRkalTp9K+fXtzxyeslKIoJKRm4u5sfFt0UGA1GlbylBH5hLAgk58i2tjYGE1nZWXh4ODwxAMSIlabxuytZ5iy7rjhFdu7SjvZE1izjCQNISzIpMTRuXNnxowZw/nz51EUhRs3bvDBBx/QsWNHc8cnrIheUdh++hbjVx/lxLU4YrXprDl42dJhCSEeYPIIgBUrVqRfv34kJSXRtWtXnJ2dTepSXQhT3LyTwscbTvLdXxdzXGUoivKQrYQQlmDSMw4HBwemTJnClClTiIuLw93dXQZwEk+ELkvP5hPX2Xj0gUI+dydeaVuTmmVLWzA6IURuTEocCxYseOgy6QpdFNTFyOwR+W7EGRfyBTfyo3ujitjbyo8TIYoikxJHWFiY0XR8fDwnTpzIUQ0uhKl+OXSFTceu5Szka1sTPy+pyRCiKDMpccyfPz/HvL///ls6IxQF5u5sb1TI9+LTlelcX2oyhCgOCtxXVbNmzXjrrbeeZCzCinSsV5Z9F6JwtLNhWJsaaEo55r2REKJIMClxXLx40Wg6MzOTzZs3U758ebMEJUoORVE4cCmacu7ORj3WqlUqxnatj7O9jdRkCFHMmJQ4goKCUKlUhtci1Wo1lStX5oMPPjBrcKJ4i9WmseLPi5y4FkcVjSsf9miEzX23olwcCqVzZiHEE2bSv9zQ0FAqVqxo7lhECaFXFHaciWD1wcuGmozL0Vp2nL1F5/pylSpEcWfS+459+/ZFq9XmvaKweg8r5OtUrxyBNctYMDIhxJNi0hWHn58fZ86coVmzZuaORxRTUsgnhPUwKXHY2dkxdOhQNBoN3t7eRg8z165da7bgRPHwqEK+52VEPiFKHJMSR+/evendu7e5YxHF0J3kdD7ZeNLoKkMK+YQo2UxKHMnJyQwcODDH/EWLFj3xgETx4uHiwLMNKvDbietSyCeElXho4oiNjeX06dMAzJo1Cz8/P6PlWq2WxYsXM3LkSPNGKIoUvV7JkRR6Nq1IUlomzzf2w6eUk4UiE0IUlocmDhcXF+bPn8+dO3dIT0/no48+Mlpub28vScOKKIrCwUvR/HLoChO6B+Dlem8QL3tbG0a0q2nB6IQQhemhicPR0ZFff/0VyO4B91E95IqSLVabzrd7L3D8ahwA3+69wDvP1pOKbyGs1GN3qy5KrtwK+QCuxGhzHQ9cCGEdpM8Hkatbd1JYtieMsNuJRvM71i1L3+ZVcLaXr44Q1kr+9Qsjuiw9v524zoYHCvnK/lfIV0sK+YSwepI4hMHl6CSW7ArjelyyYZ6NWkVQwwo837iSjMgnhAAkcYj7xKdkGCWNqho3XmlXg4pero/YSghhbSRxCINGlbxoUV3D0Sux9H6qMs/6SyGfECInSRxWSpuWSXRSGlU0bkbzB7WqxotPV5ZCPiHEQ0nisDLZhXwxfP/3Rexs1Ezv0wSn+96QKuVkj+QMIcSjSOKwIg8W8gGsOXSFIYHVLRiVEKK4kcRhBfSKwq6zEfx8wLiQz8PFnvoVPCwYmRCiOJLEUcLdik9h+Z4w/o3IpZCvWRWcZdxvIUQ+FcpZY/PmzSxcuBCdTseQIUMYMGCA0fI//viDL7/8EkVRqFChAtOmTaN0aSk0exy6LD0bj11jw9GrZGbdK+TzLe3EK+1qUlsK+YQQBWT2iq7IyEjmzJnDypUr2bBhA6tXr+bixYuG5Vqtlg8//JAlS5awadMmatWqxZdffmnusEo0RVEY/+NBfjl0xZA0bNQqnm/sx2cvNpGkIYR4LGZPHPv27aN58+a4u7vj7OxMly5dCAkJMSzPzMxkypQplClTBoBatWoRERFh7rBKNJVKRbt65QzTVTSuTO3ZiBefriLV30KIx2b2W1VRUVFoNBrDtI+PD6dOnTJMe3h40KlTJwDS0tJYsmQJgwYNyvdxvB6obtY8UJ9gbbp5u7I/LJImVb154enK2KitN2FY+3fhftIW2aQdHo/ZE4derzcat0FRlFzHcUhKSuLNN9+kdu3a9OjRI9/HiY3VGk1HRyflP9hiSJuWycr94XT2L09l73vJU6Nx4/+eqY1KpSIuNvkReyjZNBo3q/ku5EXaIpu0A6jVqhw/tvO1/ROMJVe+vr5ER0cbpqOjo/Hx8TFaJyoqiv79+1OrVi0+/fRTc4dUIiiKwoGL0by7+gh//hvJst1hZN3Xmy0gAy0JIczC7ImjZcuW7N+/n7i4OFJTUwkNDaVNmzaG5VlZWYwcOZLnnnuOiRMnysnOBLHadL4IOcOCP86RmJoJZA+udPxqrIUjE0JYA7PfqipTpgxjxoxh8ODBZGZm0rt3bxo0aMCIESMYPXo0t2/f5uzZs2RlZbFt2zYA6tevL1ceuXhoIZ+zPS+3qUGTyl4WjE4IYS0KpY4jODiY4OBgo3lLly4FwN/fn/PnzxdGGMWaFPIJIYoKOdsUcbosPVtO3si9kK9tDWqXc7dgdEIIaySJo4iLSEjl18NXuPvc20atoltABV5oIiPyCSEsQxJHEefn6UK3hn5sPn6dKhpXXmlbk0reMiKfEMJyJHEUMfEpGbg72xvN69GkEt5uDrSrXRYbGZFPCGFhcq+jiNCmZbJk17+M/fkwsdo0o2X2tmo61i0nSUMIUSRI4rCw7BH5ohn/XyFfWmYW3+69iKIoeW8shBAWILeqLChOm853f13k6BXjwj07GzWZWXrsbW0sFJkQQjycJA4L0CsKu85FsOrAZVIzHijka12dJlW8LRidEEI8miSOQhYRn8LyPRc4H5FgNL9D3bL0k0I+IUQxIGepQrTzbAQ//H0xRyHf8LY1qCOFfEKIYkISRyHycnUwJA21Cro19KNHk4ryLKMES01NRquNJytLZ+lQiIpSo9frLR2GxVlLO9jY2OLq6o6Tk8sT37ckjkIUUNGTljV8iIhPkUI+K5CamkxS0h3c3TXY2dlbvOdnW1s1Ol3JP2HmxRraQVEUMjMziI/PHtLiSScPSRxmcvrGHQDqV/Awmj+0dXXsbW2kJsMKaLXxuLtrsLd3sHQowsqoVCrs7R1wd9eQkBAjiaOoS07PHpFvz/lIPF3smd63Kc7295rZyV6a3FpkZemws7PPe0UhzMTOzt4st0mlAPAJURSFQ5eieXfVEfacjwQgLjmDdUeuWjgyYUmWvj0lrJu5vn/y8/cJuJOczrd7cxbyPVXVm24BFSwUlRBCmIckjsegVxR2n7vNzwfCjQr53J3tGdK6Ok9JIZ8oJgIDm+Lg4IBanX0TQlEUvL01DBw4hKCgFwzr3bx5g2XLFnH48EHS09Px9fWla9fu9OnzEjY2994OjImJZvnyJRw48DfJycn4+PjQo0dvevXqm6+4Pv74A3bs2M7atZvx9tYY5v/++2Z+/XUNy5f/YLT+sWNH+OCD8WzZssMw78CBffz88w9cuBCGoijUrVuXESPeoHbtuvmKJTeJiYlMmzaVY8cO4+LiyrBhI4za6347doSyfPliYmJiqFOnHmPHvoefX0UArly5zKxZ07h4MQwXF1f69u1Pnz79c2mPyaSnp/HJJzMB+O675Wg0PnTtGpxjXXOSW1UFFBGfwmebTvHNnxeMkkb7Or7M6NtUkoYodpYu/Y7t2/eyfftetm3bw7BhrzJz5mdcuXIZgKtXrzBixBC8vTV8++1KQkP3MHHiR+zcGcqkSeMN+4mOjmLYsIGUKlWKFStWsm3bbt5/fworV/7AN98sMTmexMRE9u37m/btn2HDhl8L9Jk2bVrPZ599RJ8+/dm0aRsbNmzl6adbMHr064SHXyrQPu83c+YnODs7sWlTKJ98MoOvv/6Sixcv5Fjv9Ol/+PTTD3njjbf5/fcddOzYiTFj3iQ9PbtD06lTPyAwsA1bt+5i1qz5fPPNEk6cOGa0j127/mD79hCjef36DWTlyh+4c+fOY3+W/JDEUQAZOj2fbDxpVP1dppQjE4IbMLxtTVyk+lsUczY2NnTu/BzOzi5cvpx9gp0//wueeaYzb775Nt7eGlQqFbVr12HWrPmcOHGMP//cDcDSpQtp0CCA119/C3d3d1QqFfXq1ee99z4gNjbG5BhCQrbQsGEjevZ8kU2b1pOZmZmvz5CamsqCBXMZP34SrVq1xtbWFgcHBwYMGESPHr25evVyjm1OnjxOp06tc/w3cGCfHOumpKSwd+8ehg17DQcHB+rWrU+nTl3YvHl9jnX//HMXrVu3JTCwDba2tnTv3gNHR0cOHz4EwPXr19DpdIb6ErXaxujFipiYaBYv/opu3bob7dfBwYFWrVqzdu2qfLXN45IzXAHY26rp+VQlVvx5EbUKugb40bOpFPKJvP1+8gbrjlwlLTMr75Ufk6OdDT2bVqJrAZ6zZWZmsn79L+h0mdSr5096ejpHjx5i8OCFOdYtXdqdVq1as3fvbtq0acfBg/t5883/y7HeU08146mnmpkcw+bN63nttTfx9w/Aw8ODXbv+oHPn50ze/p9/TpKVpaNZsxY5lr3++lu5bhMQ0Ijt2/eatP8bN65hY2NL+fL32rdixUrs2bMrx7p6vR4HB0ejeSqVihs3rgEwePAwli5dyNKlC8nKymLo0BHUq1cfyL5t+NlnHzF8+Gtcu3aVpKREo/20b9+R8ePfYcSI102K+0mQKw4T6PU5uzhvX6csz9Qry9RejenXvIokDWGS30/eKJSkAZCWmcXvJ2+YvP7IkcPp0qUtHTq0pEuXthw/fpR58xbh41OGpKREdDodnp5euW7r5eVNbGz2yyGJiQl4eDxeFzr//HMSrVZLixaBADz/fC9+/XVNvvaRkBCPm1spbG3N8/s4NTUNBwfjGh0HB0fS0tJyrBsY2Ibdu3dy/PhRdDodW7Zs4tq1q2RkZACgVqt4++2xbN++l0WLvmH9+l/Yv/9vANauXY2rqxudOj2baxzVq9ckISGe69evPeFP+HByxZGHMzfv8O3ei7zRsTZVNG6G+WqVipdb17BgZKI46hpQoVCvOPJztbFo0XKqVq3OrVs3mTBhHKVLu1O3bj0ASpUqjY2NDVFRkYYHuve7fTsCL6/spOLp6UVcXFyOdbKyskhO1lKqVOk8Y9m0aT0JCfH06NH1v211JCQkcP78OWrXroOdnR1ZWTnbMCsry3CLx8vLm8TE7IT3YPJITEzE2dk5x/yTJ08wfnzOq6UyZXz57jvj20GOjo6kp6cbzUtPT8PJyTnH9g0bNubtt//HzJmfkpSURJcuXWnWrAWurm6cP3+WX39dw7p1WwCoX78BwcE9+O23DZQtW45ffvmZZcu+f2hb2draUrp0aaKjo3L9f2MOkjgeIruQ7zJ7zt8GYNnuMD7q2QhbG7lIEwXXNaBCgW4dFaZy5cozffpsXn65P2XLlmPIkOHY29vTvHlLtm79jSZNnjJa/86dOA4e3M/7738AQLNmLdizZxddunQ1Wm///r/46KNJbNwYgrPzwyuZtVotu3b9wdy5XxvdBpo3bza//rqaiRM/RKPxISoqEkVRjGoVbt26iUbjA0C9ev7Y2dly4MDfBAa2NTrG9Okf4+zszKRJHxnNDwhoSEjIbpPayc/Pj6wsHbdv38bX1xeAa9euUqVKlRzrJiTE4+8fwM8/rwOyE1zv3sEMHjyMyMjbOZ7f2NraYmtry549O7lzJ44+fZ4HICMjA71ez5Ah/YwSWVZWluGNuMIgZ8FcHAqP5t3VRw1JAyA6KZ2bd1IsGJUQhcfXtyyjR7/DihVLDW8JvfXWO+zb9xcLFswlJiYGnU7H2bOnGTv2bRo0aEibNu0BePnlVzh58hiLF39FYmICWVlZHDt2hM8/n0b//oMfmTQAtm3bQvnyfjRo0BAvL2/Df0FBz7NjRyjx8fHUrVsfZ2dnFi78kpSUFPR6PefPn2XVqh8Nz0EcHBx47bVRzJz5Gfv2/YVOpyMlJZnly5dw5Mgh+vcf9Fht5OzsQmBgGxYvXkBaWhrnzp1h+/ZtdOqU8znMlSuXGTXqVSIibpGensayZYtwd3enbt36+PsHkJmZwbffLiMrK4uLFy+wadM6OnTozJAhw9m+fS8hIbsJCdnNgAFDCAxsY5Q0MjIySEpKMiTMwiBXHPe5k5w9It+RyzkL+Qa3qoaHi/Q5JKxH167BbN8ewrRpU1my5FsqVPBj+fIfWLZsEcOHDyAlJQVf37I891wQffr0N/zy9/Epw6JFK1iy5GsGDHiRtLQ0fH19efnlV+jRo3eex920aQPPPNMlx/ymTZ+mdGl3Nm9ez6BBQ5kz5yu+/noeL74YTHp6OhqNDy+80Iveve/VivTs+SKurm6sWLGEqVM/QK1WU69efb78cjFVq1Z/7DYaP34Sn38+jR49uuLk5MQbb4w2PNT+/vtvOHnyBLNnzycgoBH9+g3k9deHk5aWSkBAI2bOnItarcbT04vPP5/HV1/N4+eff8Dd3YOhQ0fQtm17k2I4f/4sPj6+Rldn5qZSSsjg1rGxWvp/vccw/ePINiZvWxIL+TQaN6KjkywdhsVZsh1u376Kr28lixw7N9bQK6wpSlo7LFgwF0dHR155ZWSuy3P7HqrVKry8Ct47t9VfcUQmpLJ0d1iOEfna1/GlX/OqUpMhhCiy0tLS2Lt3N0uXfleox7X6s2K6LosLkffeiy5TypHhbWtSt7yMyCeEOfzyyyqWLPnqoct/+OEXw8Nm8Wg///wDgwYNNelNtSfJ6hNHRS9XghpWYPPx61LIJ0QhePHFfrz4Yj9Lh1EiDB06wiLHtarEkZ6ZxdUYLTXLGmfn5xtX4umqGhmRTwghTGA1iePMzTss33OBxNRMpvdpgrfbvfJ/e1u1JA1hBioURY9KJW+9C8tQFD3w5MfkKPHf6OT0TJbuDmPa5n+ISkwjLTOLFX9eoIS8TCaKMHt7R+LjY9DpMuX7JgqVoijodJnEx8dgb++Y9wb5VKKvOA6FR/PdX5dISMkwzHO2t6Fp1eL3eq0ofjw8NGi1CcTFRaLXF07/VI+iVqsNva9aM2tpB7XaBicnV1xdn/yD8xKbOOZuO5OzkK+KN4MDpZBPFA6VSoWbmztubkXjDT2p7ckm7fD4CuVW1ebNm+natSudO3fmp59+yrH83Llz9OzZky5dujBx4kR0uscfXP3+pFHa2Z63O9fl7S51JWkIIcRjMnviiIyMZM6cOaxcuZINGzawevVqLl68aLTOuHHjmDx5Mtu2bUNRFNasyV/3yY/SrrYvM/s25Sm5PSWEEE+E2RPHvn37aN68Oe7u7jg7O9OlSxdCQu4Nf3jz5k3S0tJo2LAhAD179jRaXlA+/43I90o7GZFPCCGeJLOfUaOiotBo7g0y7+Pjw6lTpx66XKPREBkZme/jeHm54mRvQ2pGFrZqFcveaIeDnXUX8mnuGz/Emkk73CNtkU3a4fGYPXHo9Xqj/vIf7D8/r+WmunMnmYGtqrPnfARta5dFm5iK9vFCL9a8vFyJjbXmFsgm7XCPtEU2aYfsTg49PB7dvf2jmD1x+Pr6cuTIEcN0dHQ0Pj4+Rsujo6MN0zExMUbLTeXh4ULvwOr0Dnz8rpJLisfp/bIkkXa4R9oim7TD4zH7M46WLVuyf/9+4uLiSE1NJTQ0lDZt7nV5Xr58eRwcHDh69CgAGzduNFouhBCiaCmU8Tg2b97M4sWLyczMpHfv3owYMYIRI0YwevRo/P39OX/+PJMmTUKr1VKvXj2mTZuGvb29ucMSQghRACVmICchhBCFo8T3VSWEEOLJksQhhBAiXyRxCCGEyBdJHEIIIfJFEocQQoh8kcQhhBAiX4pt4rBEV+1FUV7t8Mcff/D888/TvXt33njjDRISEiwQpfnl1Q537d69mw4dOhRiZIUvr7YIDw9n0KBBdO/eneHDh1vtd+LMmTP06tWL7t2789prr5GYmGiBKAuHVqslKCiIGzdu5FhWoHOlUgzdvn1bad++vXLnzh0lOTlZCQ4OVi5cuGC0Trdu3ZTjx48riqIo77//vvLTTz9ZIlSzyqsdkpKSlFatWim3b99WFEVR5s6dq3z88ceWCtdsTPk+KIqiREdHK88++6zSvn17C0RZOPJqC71er3Tu3FnZs2ePoiiK8vnnnyszZ860VLhmY8p34qWXXlJ2796tKIqiTJs2Tfniiy8sEarZnThxQgkKClLq1aunXL9+Pcfygpwri+UVh6W6ai9q8mqHzMxMpkyZQpkyZQCoVasWERERlgrXbPJqh7smTZrEqFGjLBBh4cmrLc6cOYOzs7OhW5+RI0cyYMAAS4VrNqZ8J/R6PcnJyQCkpqbi6Pjkx+YuCtasWcOUKVNy7QOwoOfKYpk4cuuq/f6u2J9UV+1FXV7t4OHhQadOnQBIS0tjyZIlPPPMM4Uep7nl1Q4A33//PXXr1iUgIKCwwytUebXFtWvX8Pb2ZsKECfTo0YMpU6bg7OxsiVDNypTvxHvvvcekSZMIDAxk37599OvXr7DDLBSffvopTZs2zXVZQc+VxTJxFFZX7UWdqZ8zKSmJV199ldq1a9OjR4/CDLFQ5NUOYWFhhIaG8sYbb1givEKVV1vodDoOHTrESy+9xPr16/Hz82P69OmWCNWs8mqHtLQ0Jk6cyLfffstff/1F//79GT9+vCVCtaiCniuLZeJ4sCt2c3XVXtTl1Q6Q/Yuif//+1KpVi08//bSwQywUebVDSEgI0dHR9OrVi1dffdXQJiVRXm2h0WioVKkS/v7+AAQFBRkNrFZS5NUOYWFhODg40KBBAwD69u3LoUOHCj1OSyvoubJYJg7pqj1bXu2QlZXFyJEjee6555g4cWKJvOqCvNth9OjRbNu2jY0bN7JkyRJ8fHxYuXKlBSM2n7zaolGjRsTFxXH+/HkAdu7cSb169SwVrtnk1Q6VKlXi9u3bhIeHA7Bjxw5DMrUmBT5XPrln94Vr06ZNSrdu3ZTOnTsrS5YsURRFUV555RXl1KlTiqIoyrlz55RevXopXbp0Ud555x0lPT3dkuGazaPaITQ0VKlVq5bSvXt3w38TJkywcMTmkdf34a7r16+X6LeqFCXvtjhx4oTSq1cvpWvXrsqwYcOUmJgYS4ZrNnm1w+7du5Xg4GAlKChIGTJkiHLt2jVLhmt27du3N7xV9bjnSulWXQghRL4Uy1tVQgghLEcShxBCiHyRxCGEECJfJHEIIYTIF0kcQggh8kUShxCFLDU1lZiYGEuHYXD9+nVLhyCKGUkcokhavnw5TZo0oVWrVmRmZj50vYMHD9KsWbNCjKxgGjVqxKVLlwAYMGAA//zzDwCbNm2yaCeDZ8+e5aWXXrLY8UXxZGvpAITIzerVq3n//ffp3bu3pUN5Io4fP274Oz4+3vB39+7d6d69uyVCArL7MXtUYhYiN3LFIYqcLl26cO3aNaZOncrUqVNJS0vjww8/pFOnTjRs2JDOnTvzxx9/5NguIyOD999/n2bNmhEYGMjo0aO5c+cOkN2p3SeffELr1q0JDAxkxowZZGRk5Hr8Dh06sGDBAtq1a0eTJk344IMPSE9PByA5OZmPPvqIVq1a0apVKyZOnEhSUhIAt27dYvDgwTRt2pRnnnmGmTNncre+tlatWoSFhfHmm29y69Yt3n77bb7//nvWrVtHz5490Wq1BAQEcOHCBUMca9eupU+fPoZ9jxw5kmbNmtG5c2d+/fXXXGO/ceMGTZo04b333qNp06Zs3LiR69evM3LkSNq2bUuDBg3o168fly5dIjY2lhEjRhAfH0+jRo24c+dOvtpJWC9JHKLI2bZtG+XKlWPevHlMnjyZ5cuXc+nSJdatW8fRo0fp2bMnH3/8cY7tNm7cyKVLl9i1axfbt28n7otKKgAABXpJREFUJSWF77//HoAZM2YQHh7Opk2b2LRpE6dPn2bRokUPjWHLli389NNPbNu2jbNnz/Lll18CMHnyZMLDw9m8eTO///47MTExTJ48GYA5c+ZQs2ZNDh06xI8//siWLVvYv3+/0X6/+uorw2cbPHiwYb6rqysdOnRgy5Ythnm//fYbzz//vKHPsRo1arB3717mz5/PnDlzOHDgQK6xa7Vaypcvz759++jcuTOTJk2iatWq7NixgwMHDuDh4cGiRYvw8vJi6dKluLu7c/z4cTw8PPLdTsI6SeIQRd6AAQOYP38+zs7ORERE4OLikuuYAW5ubly9epX169dz584dlixZwttvv42iKKxbt46xY8fi4eGBp6cnb731FmvWrHnoMV9//XXKly+Pt7c3r7/+Olu2bCEtLY1t27Yxbtw4PD09KV26NOPHj2fr1q2kpqbi5ubG4cOH2bZtG87OzuzatYuWLVua/DlfeOEFtm7dCmT35nrs2DGee+45/vnnHyIiIhgzZgz29vbUrl2bfv368csvvzx0X8HBwdjb2+Pk5MT06dMZPXo0WVlZ3Lp1C3d391zbryDtJKyTPOMQRV5SUhIfffQRp06dws/PDz8/P3LrYu3ZZ58lLi6OdevW8emnn1KzZk2mTp1K+fLlSUtLY9CgQYYeghVFITMzk/T0dBwcHHLsq1KlSoa/y5QpQ3R0NImJiWRmZlKuXDnDsvLly6MoClFRUYwbN4758+fzxRdf8L///Y82bdrwySef4O3tbdLnDAwMRKvVcvr0aY4cOUKrVq3w9PTkwIEDaLVann76acO6WVlZj+zV9v5jhoeH8/nnnxMZGUn16tVRqVS5tl9cXFy+20lYJ0kcosibMmUK1apVY9GiRdja2nL48GHDL/P7XblyhebNm9O/f3/u3LnDV199xbvvvsuWLVuws7Njw4YN+Pn5AZCSkkJMTMxDT4ZRUVGGv2/dukXZsmXx9vbG3t6emzdv4unpCWQ/U1Cr1Xh4ePDvv/8yYsQIxo8fz7Vr15g4cSLz589n6tSpJn1OGxsbunXrRkhICEeOHOHll18GskevK1OmDLt37zasGxMTk+vJ/667J/6MjAxGjRrFtGnTePbZZwFYsGABBw8ezLGNu7t7vttJWCe5VSWKPK1Wi6OjIzY2NkRERDBv3jyAHG8D7dixg//973/ExMRQunRpXFxccHd3x8bGhuDgYGbNmkViYiIpKSlMnjyZ995776HHXLJkCbGxsURFRbFo0SJeeOEF1Go13bt3Z9asWcTFxZGQkMDMmTNp27YtpUqVYuHChcyaNYv09HS8vLywsbHBw8Mjx77t7OzQarW5HveFF15g8+bNhIeH06FDBwACAgJwdHRk2bJlZGZmcvv2bYYOHcpPP/2UZ9vdvVpwcnIC4MSJE6xatcrQdvb29mRkZJCRkVGgdhLWSRKHKPLef/99du/eTePGjRk4cCBt27bF2dnZUBdx1+DBg2nQoAHBwcE0adKEY8eOMW3aNAAmTpyIh4cH3bp1o23btmi1WubMmfPQY9auXZu+ffsSHBxM8+bN/7+9O0TREIrCMPwliyBYzAZBsBosBhXBJmhyBwa3oKDVBRhckru41TUY/jDwM8yE4aYZmPephwOHUz7OLVfjOL5nCcNQbduqrmv5vq993yVJ67rqvm/lea6iKBQEwbvvs67rtCyLjuP4VkuSRJ7nqWkaOY4j6SNozvPUdV3K81x93yvLMk3T9OPuXNfVtm2a51lpmmrbNg3DIGOMnudRHMeKokhZlskYY70n/E/8xwF8UVWVlmVRWZa/PQrwJ3FxAACsEBwAACs8VQEArHBxAACsEBwAACsEBwDACsEBALBCcAAArLwAmHBvNUoySAcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a232af860>"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAD7CAYAAAAywXBqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdeVhV9dr/8TfCFhIHcEolzdQjwvFUmENoiiIqDohsHtQwpMGf5jHrWIo4g0MOZRQ4HYTEkAJREBGJDj5ZHgdQG8BsUMkTYIIoOCHT3uv3B4/7SMqkTHt3v66r64LFGr73fjrP3Xft7/osI0VRFIQQQgg90ayxByCEEELUhjQuIYQQekUalxBCCL0ijUsIIYRekcYlhBBCr0jjEkIIoVekcQkhhNArJo09AEORn38brdbwHolr164lV6/eauxh1AtDrc1Q6wKpTR9VVlezZkZYWpo/1DkfunF5eXkRHh4OwKJFi3jjjTewsrIiNjaWjRs30q5dOwCGDx/OvHnzHvYy1Tp//jxLly6lsLCQNm3asG7dOqysrO7bz9ramp9//rnexqHVKgbZuACDrQsMtzZDrQukNn1U13UZPWxyxr2NwNHRkU8++YQnnniCVatWYWdnx4QJE+p0oJXx8vLi//2//8ewYcP47LPPOHXqFBs3bqxyvEIIIR5dUXEZN2/cqXKfDh1aceXKzfu2N2tmRLt2LR/qutXOuMrKyvDz8+PcuXPk5eVhbW1N27ZtAfDw8GDUqFHk5uYyc+ZMIiIiSE9P5+LFi/zzn//E2tqaZcuW0aZNm0rP7+XlRZ8+fTh16hTFxcUsXryYF154AV9fXwoKCvjPf/7DggULaN++PWvXrqWoqAhLS0v8/f3p2rUrO3bswMTEBK1Wy6VLl2jdujUAWVlZLFiwgMLCQp555hnd9XJycli8eDE3b94kNzcXNzc33nrrLTw9PZkzZw5DhgxBURTGjBlDeHg4jz/+eI0+yNdWf0FuftX/BxRCCEMSv9GV+1tS/at2cca3336LSqUiKiqKf/3rX9y8eZOhQ4cCEB0dzcyZM+nYsSPBwcFYWlrSoUMH/v73v7N//346d+7MypUrqx3ErVu3dLcYfX19KSkpAcDCwoLExEReeOEFli5dysaNG4mNjeWVV15h2bJlAJiYmHDjxg3djGvy5MkArFq1CrVaTVxcHP369dNd68CBA0yYMIHdu3cTHx/Pzp07uXbtGu7u7sTFxQFw6tQpunXrVuOmJYQQf1YdOrSq8p/K9nnY2RbUYMY1YMAALCwsiIiIICMjg4sXL1JYWFjp/ps3b9b9PGPGDEaNGlXtIO42GxsbGzp06KC7pff0008DcPHiRTIzM5k9e7bumFu3/vtlX+vWrfn3v//N119/zezZszl06BCpqam6W4YTJ05k6dKlALz22mucOHGC0NBQzp07R2lpKXfu3GHs2LEEBARQWFhIbGwsarW62nELIcSf3YNuA96rPm4VVjvjOnToEPPnz8fMzAy1Ws2AAQOo7GuxmzdvEhYWpvtdURSMjY2rHcS9+2i1WkxMyvupmZmZbtsTTzxBXFwccXFxxMTE8OmnnwJw8OBB3XiGDRtGUVER169f110fwMjIiGbNyktdt24d4eHhdOnShdmzZ2NpaYmiKLRo0YJhw4aRlJTEiRMnGDlyZLXjFkII0fCqnXEdP36csWPH4u7uTmZmJikpKdjb22NsbExZWRkmJiYYGxuj0Who0aIFISEh2NnZ8cwzz7Br164azbgOHjzI008/TXp6Ojdu3KB3794V/t6jRw+uX7/OqVOn6N+/P3v37iU+Pp7w8HA+/vhjTExMGD16NCdOnMDS0pK2bdsyePBg9u/fz7Rp0/jiiy8oLi4G4OjRo/j7+9OvXz8OHz5MTk4OWq0WAHd3d+bNm8eIESMwNTWt1QcZunR0rfYXQgh9V1Rc1ijXrbZxeXh4MH/+fBISElCpVPTr14+srCxGjhyJq6srMTExDB8+nJkzZxISEsKHH36In58fRUVFdO/enQ0bNlQ7iMzMTNzc3AAICAi4b5bWvHlzPvroI9asWUNxcTEtW7Zk/fr1QPkMatmyZWzevJlWrVoRGBgIwPLly1mwYAFRUVH07dsXc/Py5wVmzZqFj48PZmZmdOrUib59+5KVlUW3bt147rnnMDIywt3dvXafInD16i2DXMpa2TTfEBhqbYZaF0htotxDL4evK15eXrzxxhsMGjSoMYeBoij88ssvLFy4kH379tX6eGlc+sdQazPUukBq00eNshy+LrzzzjucP3/+vu2Ojo4Ncfka2blzJyEhIXz00UeNPRQhhBBVaPQZl6GQGZf+MdTaDLUukNr0UX3MuB66cVUW+XT69GnWrl1LaWkpFhYWvPvuuw+MYKorubm5+Pj4cO3aNUxNTVm5ciU2Njb37XdvuocQQjQ1JaUarhdU/qiRvmpStwpTU1N1P6ekpDBnzhwAFixYwJYtW+jTpw979uxh9erVbN269WEvU62AgADGjBnDiy++yNdff42/vz+RkZH1dr3KSHKGEOJRxG90bewh6I06jXz65JNPeOutt+jTpw9Qng+4a9euKs/v6+uLqakp6enp3L59m9mzZzNp0iSCgoL47rvv+P3333nppZcYMmQIfn5+FBQUYGZmxrJly7C1tWXNmjW6c2VlZekinwoKCliwYAGXL1+mZ8+euuXwt27dYvHixeTk5JCbm4u9vT1r1qzBx8eHAQMG6B6G9vLyYv78+RXiooQQQjS+ahvXvZFPWq0Wb29vXF1diY6OJjo6GoDIyEiCg4Np3749rq7l/9Wg1WrZtGkTTk5O1Q4iMzOTqKgorl69ilqtZsiQIQCUlJRw8OBBAKZOncry5cuxtbXl/PnzzJkzh6SkJN2Dxc7OzmRnZ7NlyxYAAgMDsbW1Zfv27Zw8eZLExEQADh8+jI2NDYGBgZSUlDB+/Hh++OEH3N3dCQoKYvLkyWRnZ3Pt2jVpWkKIBnU3IsnQ1HVddR75BOUNx9fXl7KyMmbNmlXtINRqNSqVik6dOtGvXz9Onz4N/Dfy6fbt25w5c4ZFixbpjiksLCQ/Px9LS0sAPv/8c3788UdeffVVEhMTK0Q+DRgwgK5duwIwYcIE0tLSCAsLIyMjg4KCAgoLCxk0aBDLli0jKyuLuLg4XQMWQoiGIoszaqbaxnXo0CECAwOZPn06arWa/Pz8SiOfAN3tPgsLC7Zu3YpKpap2EDWJfGrevLkuBBfg8uXLWFhYcPjwYQYMGIC5uTk2NjZ06dKFzMxMjIyMKozz7jXCw8NJSkpi8uTJDB48mF9++QVFUTAyMmLSpEkkJCSQmJhIaGhoteMWQgjR8Oo08gnKF2c8+eST+Pv7627jVScxMRFnZ2cuXbpEWloaa9as4ccff9T9vVWrVnTv3l03Ezp69CjLly8nOTmZ2NhYfv31V1555RXOnz9PXl4ePXr0wN7enri4OPr06UNaWhq//fYbUB75NGXKFFxcXEhPT+enn37SRT6p1Wo8PT3p1atXrZPhJfJJCPEoSko1jT0EvVHtcviff/6Z+fPnA6BSqbCysqJHjx5kZGSQkZFBTEwM77//Pl9//TUBAQG4ubnRq1cv3aypY8eObN++vdLz+/r6cu3aNfLy8igpKeHtt9/G0dGRoKAgAObOnQvAhQsXdIszVCoVfn5+PP3007r3a125cgVTU1MWLlxI//79uXXrFr6+vvz666/06NGDn3/+mY8//pjMzEz8/Pxo3rw5LVu2xMzMjHHjxuHh4QGAp6cnL730EuPGjavVBynPcekfQ63NUOsCqU0fNannuOqKr68vAwcObPTXiCiKQm5uLl5eXhw4cIDmzZvX6nhpXPrHUGsz1LpAatNHTeo5rtpYv349x44du2973759G+LyNZKUlISfn59uNiaEEKJpeujGVZvkjIULF9bZgCtz9OhRgoOD2blz5wP/Xl1yhrOzM87OzvU5RCGEEHVA75MztFotYWFh/POf/7zvPV4N6VFeQ93UGeqzJWC4tRlqXVB9bUXFZdy8ISk2hkzvkzMuXLjAhQsXWLVqlW4GCA2fnCGRT0I0DfEbXTG8b4rEvapdr35vcsa//vUvbt68ydChQwGIjo5m5syZdOzYsU6SM3bu3MmGDRu4cuUK8N/kDE9PTxYuXMiCBQuIjY1l1apVzJs3D4C//OUvrFmzhjZt2lQ4593kjPj4eKZNm0ZeXh7w3+SMqKgokpKSOHnypC454+5zYpKcIYQQTZfBJGf8kSRnCPHnpa+3SvV13NVp8Minpp6cURlJzhDiz0sfl5XLcviaq/ZW4b3JGa1btyYlJQWNRqNLzgAemJzx4Ycf1nhZeWJiIoqikJ2dTVpaGs8991yFv9+bnAHlKwinTZtW5TnvJmcAD0zOmDhxIsXFxfclZ0RGRtK5c+daJ2cIIYRoGNXOuDw8PJg/fz4JCQmoVCr69etHVlYWI0eOxNXVlZiYGIYPH87MmTMJCAjg0KFD9OrVCzc3N6D65AyAoqIi3N3dKSkpYeXKlQ+8/ffee+/h5+dHSEgIKpWKgIAAjIyMKj3nm2++ia+vL+PHj6dHjx66W4Xe3t74+fkRHBxMy5YtsbOzIysrC4DOnTvTuXNn3dhrQyKfhGgaiorLGnsIop5Jcsb/keSMBzPU2xdguLUZal0gtekjSc6oR5KcIYQQ+qHRZ1yGQmZc+sdQazPUukBq00eNsjijMl5eXrqfFy1aRHZ2NgCnTp1CrVbj4uLC66+/zvXr1x/2ErVy9OhRvL29K/27o6Oj7rssIYQQ+qvOI58WLVrE1q1b6dWrF++//z6hoaG8/fbbjz7SSkjkU/0z1GdLwHBrq6u6JD5JNEV1GvkUERHBwYMHUalUlJaWkpOTg7W1dZXnl8gnIZouiU8STVGdRj5ZWlqiUqn4+eefcXBwICUlhfHjx1c7CIl8EkIIUVP1EvlkbW3NsWPHiIyMZN68eURGRla5v0Q+CdF0NbXbqU1tPHXJUGtr0pFPxcXFHDlyRBesO3HiRNavX1/tICTySYimqymtdDPUlXdguLU1+cgnExMT/P39OXPmDFAe5dSvX79qByGRT0IIIWqqTiOfQkJCCAgIYPny5Wg0Gh5//HHWrFlT7SAk8kmIpknik0RT1OgPIEvkU9NmqLcvwHBrM9S6QGrTRxL5VI8k8kkIIfRDo8+4DIXMuPSPodZmqHWB1KaPmnzk09WrV3F1ddX94+joiJ2d3cNeokZyc3N57bXXcHV1xc3NjePHjz9wP4l8EkIIw1CnkU/t2rXTreTTarV4e3vrHhSuLxs2bMDR0ZFp06aRkZGBl5cXX3/9dYUl9g1BIp/0U0PVJtFJQtSdOo98ursicO/evTz22GO4uLhUef5HjXwaNWoUzz//PABPPvkkxcXFFBYWotFoJPJJNBkSnSRE3am2cd0b+XR3FuXq6kp0dDTR0dEAREZG6iKfADQaDdu2bWPLli01GsTdyKerV6+iVqsZMmQI8N/IJ4CpU6eyfPlybG1tOX/+PHPmzCEpKYkxY8bozhMaGoqNjQ2tWrVi5cqV2Nrasn37dk6ePEliYiLw38inwMBASkpKGD9+vC7yKSgoiMmTJ0vkkxBCNGH1Evl05MgRunfvXm3A7l11EfkUFhZGVFQUu3btAiTySTQ9DXnLVW7v6idDra1JRz7dlZyczLhx42o8iEeNfNqwYQNfffUVERERdOrUCZDIJ9H0NNSKMUNdnQZSmz5q8pFPd3333Xf079+/xoN4lMinsLAwUlJS+Oyzz3RNCyTySQghDFWdRz517dqVzMzMCk2kOg8b+QSwefNmWrZsWWF5fnBwsEQ+iSZFopOEqDuN/gCyRD41bYZ6+wIMtzZDrQukNn0kkU/1SCKfhBBCPzT6jMtQyIxL/xhqbYZaF0ht+qhJRT41Rbt372bo0KE1ennlH/3v//4varWasWPHsnr16noYnRBCiLrQILcKG8qBAwdYu3YtL7zwQq2Oy8zMZMWKFURHR9OuXTu8vb356quvcHBwqPE5JPKpYUh0khBCbxpXSkoK27ZtQ6VSkZWVhaOjIy1atCA5ORkoD9FNT0/H39+fJUuWsG3bNiIjIwGIiYnh+++/x9/f/4Hn/te//sW4ceN0KyEDAgIwNTWt1fgk8qlhSHSSEEKvbhXebT579+4lIiKCtm3bEhMTg7W1Na1ataJv376sXr0aBwcHrly5ont2a9++fVWuWvzPf/6DRqPh9ddfx9XVlU8//ZQ2bdo0VFlCCCFqQW9mXAC9e/emc+fOAFhaWmJvbw9Aly5duHHjhm4/IyMj3Nzc2L9/P2q1mqtXr1aZO6jRaDh16hTh4eG0aNGC2bNnExsb2+hL9MWD1eWty6Z0G7QuGWpdILXpowaPfGpKVCpVhd+renWJm5sbM2bMoHnz5tXmDrZv3x57e3td6r2TkxNpaWnSuJqoulp59WdbxWUIpDb9o7fPcTUGKysrOnXqRGRkJJ999lmV+44YMYKFCxdy48YNzM3NOXLkCCNHjqzV9SQ5o2FIAoUQwmAbF8C4ceP44osvqs0dfOaZZ5gxYwaenp6UlpYyZMgQ3N3da3UteY5LCCEahsE+gFxWVoaPjw/Ozs6MHl3/syFpXPrHUGsz1LpAatNHcquwhhRFYejQoQwePBgnJycATp06xapVqx64f3BwsKTBCyGEnjDIxmVkZMTx48crbOvfv3+F93kJIYTQTw/9HNe9rxFZtGgR2dnZQPnMRq1W4+Liwuuvv87169cffZRVyM3N5bXXXsPV1RU3N7f7GtZdjo6OuteXCCGE0F8PPeNKTU3V/ZySksKcOXOA8ia2detWevXqxfvvv09oaChvv/32o4+0Ehs2bMDR0ZFp06aRkZGBl5cXX3/9dZVL5euDRD41DIl8EkJU27jKysrw8/Pj3Llz5OXlYW1trXveycPDg1GjRpGbm8vMmTOJiIjg4MGDqFQqSktLycnJwdrausrz+/r6YmpqSnp6Ordv32b27NlMmjSJoKAgvvvuO37//XdeeuklhgwZgp+fHwUFBZiZmbFs2TJsbW0ZNWoUzz//PABPPvkkxcXFFBYWotFoWLBgAZcvX6Znz54UFxcDcOvWLRYvXkxOTg65ubnY29uzZs0afHx8GDBgAJMnTwbKZ5Tz58+v8sHle0nkU8OQyCchRLWN69tvv0WlUhEVFYVWq8Xb2xtXV1eio6OJjo4GIDIykuDgYN2bi3/++WdeeeUVTExMajTbyszMJCoqiqtXr6JWqxkyZAgAJSUlHDx4EICpU6eyfPlybG1tOX/+PHPmzCEpKYkxY8bozhMaGoqNjQ2tWrVi5cqV2Nrasn37dk6ePEliYiIAhw8fxsbGhsDAQEpKShg/fjw//PAD7u7uBAUFMXnyZLKzs7l27VqNm5YQQoiGU23jGjBgABYWFkRERJCRkcHFixcpLCys8hhra2uOHTtGZGQk8+bN04XdVkatVqNSqejUqRP9+vXj9OnTADz99NMA3L59mzNnzrBo0SLdMYWFheTn5+uaZVhYGFFRUezatQsov5W5ceNGXQ1du3YFYMKECaSlpREWFkZGRgYFBQUUFhYyaNAgli1bRlZWFnFxcdWmbYjGI5FP1TPUukBq00cNHvl06NAhAgMDmT59Omq1mvz8fCp79Ku4uJgjR47olqBPnDixRu/Guvf7KK1Wi4lJ+bDMzMx025o3b15hVeDly5exsLAAyr/n+uqrr4iIiNAlvBsZGVUY591rhIeHk5SUxOTJkxk8eDC//PILiqJgZGTEpEmTSEhIIDExkdDQ0GrHLRqHRD5VzVDrAqlNHzXKc1zHjx9n7NixuLu7k5mZSUpKCvb29hgbG1NWVoaJiQnGxsZoNBpMTEzw9/enU6dO9O3bl8TERPr161ftIBITE3F2dubSpUukpaWxZs0afvzxR93fW7VqRffu3XUzoaNHj7J8+XKSk5MJCwsjJSWFzz77jNatW+uOsbe3Jy4ujj59+pCWlqZLij969ChTpkzBxcWF9PR0fvrpJ7RaLVA+8/P09KRXr161fq5LIp8ahkQ+CSGqbVweHh7Mnz+fhIQEVCoV/fr1Iysri5EjR+Lq6kpMTAzDhw9n5syZhISEEBAQwPLly9FoNDz++OOsWbOm2kEUFRXh7u5OSUkJK1eu1N3+u9d7772Hn58fISEhqFQqAgICANi8eTMtW7assDw/ODiYN998E19fX8aPH0+PHj10twq9vb3x8/MjODiYli1bYmdnp1sm37lzZzp37oybm1vNPr17SHKGEEI0jEaPfPL19WXgwIGNnsSuKAq5ubl4eXlx4MABmjdvXqvjpXHpH0OtzVDrAqlNH+lt5NP69es5duzYfdv79u3bEJevkaSkJPz8/PDz86t10xJCCNFwGn3GZShkxqV/DLU2Q60LpDZ9VB8zroeOfBJCCCEag16H7N79fiwrK4uYmBi8vb155ZVXHupcsbGxbNy4kXbt2gEwfPhw5s2bV+PjJfKpYUjkkxBCrxvXXXFxcezYsYOnnnrqoc9x5swZfH19mTBhwkMdL5FPDUMin4QQevUdl6IorFu3jsOHD9OxY0c0Gg3p6ekoikK3bt3o1q0bw4YNw9PTk6ioKMLCwkhMTKS0tBQnJyeSk5NRqVQPPPfkyZNp1aoVubm5WFtbs2zZMtq0aVPjsUnjahjxGyXRRIg/O72acSUlJXH27FkOHDjAzZs3mThxIv7+/mzatIng4GAyMjLYs2cPnp6enDhxguvXr5OXl8f58+exs7OrtGkBdOjQgVdffZV+/frxwQcfsHLlSl1klGhaJDmjaoZaF0ht+uhPvzgjNTWV0aNHo1KpaNu2LcOGDavw90GDBvH999+j0WjIyMhg3LhxnDx5kq+//prhw4dXee7Nmzfz3HPPYWRkxIwZMzhy5Eg9ViKEEOJh6dWM64/5g3czDe8yNTXFxsaG+Ph4evTowaBBgzh+/DinT59mxowZlZ735s2b7N27l5dffhkovyVZ2/d5SeRTw5DIJyGEXjUue3t7QkNDmTp1Knfu3OHIkSM8++yzFfZxcHBg8+bNvPrqqwwcOJDVq1fTpUsX3TvEHqRFixaEhIRgZ2fHM888w65duxg1alStxibPcQkhRMPQq8bl5OREeno6EyZMoH379vTs2fO+fYYPH46fnx8DBw6kTZs2tGvXrtrbhMbGxnz44Yf4+flRVFRE9+7d2bBhQz1VIYQQ4lHo1arCpkxmXPrHUGsz1LpAatNHeptV2FR4eXlx48aN+7ZPnTqVF198sRFGJIQQorb+VI0rPDy8sYcghBDiEel146rLyKfTp0+zdu1aSktLsbCw4N1338XKyqrGx0vkU8OQyCchhF43rrvqIvJpwYIFbNmyhT59+rBnzx5Wr17N1q1ba3y8JGc0DIl8EkLoVeN6UORTQkICiqIwZ86ch458Kikp4a233qJPnz4AWFtbs2vXroYuT9RQXc4Am9Jssi4Zal0gtemjuq5LrxpXfUU+NW/eHFfX8gw8rVbLpk2bcHJyasjSRC1I5FPVDLUukNr0kUQ+1WPkE5TPvObPn09ZWRmzZs2qpyqEEEI8Cr2acdVX5BPA7du3mT17NhYWFmzdurXKQN4HkcinhiGRT0IIvWpc9RX5BOWLM5588kn8/f1p1qz2E1F5AFkIIRqGXjWu+op8Onv2LIcOHaJXr164ubkB0LFjR7Zv314fZQghhHgEEvlUR2TGpX8MtTZDrQukNn0kkU//Z/fu3QQFBTFhwgQWLlxY4+PuRj5ptVouXrxI165dUalUTJ06lSeffJK1a9dSXFzM2LFjmTdvXj1WIIQQ4mHpZeM6cOAAa9eu5YUXXqjVceHh4Xz//fcsXboURVEIDg7miSeeoKioCGdnZ8LDw+ncuTOzZs3iq6++wsHBocbn1sfkDEmhEELooybXuFJSUti2bRsqlYqsrCwcHR1p0aIFycnJADg6OpKeno6/vz9Llixh27ZtREZGAhATE8P333+Pv79/peffvXs3K1aswMfHR7ctLS2NJ598kq5duwLg4uLC559/XqvGpY/JGZJCIYTQR03yOa67zWfv3r1ERETQtm1bYmJisLa2plWrVvTt25fVq1fj4ODAlStX+O233wDYt28farW6ynOvWbOG/v37V9iWm5tLhw4ddL937NiRnJycui9MCCHEI2tyMy6A3r1707lzZwAsLS2xt7cHoEuXLhVeS2JkZISbmxv79+9HrVZz9epVnnnmmVpfT6vVYmRkpPtdUZQKvxuymkSxGGoMDRhubYZaF0ht+uhPEfn0x4d/jY2NK93Xzc2NGTNmVIhtqq1OnTpx5coV3e9XrlyhY8eOD3UufVPdKiZDXekEhlubodYFUps++tNHPj2IlZUVnTp1IjIy8qEb1zPPPMOvv/7Kf/7zHzQaDQcOHLgvTkoIIUTT0CRnXLU1btw4vvjiCx5//PGHOt7U1JR169Yxd+5ciouLcXBwwNnZuVbn0MfIJ4lPEkLoI71/ALmsrAwfHx+cnZ0ZPbrxmoc8gKx/DLU2Q60LpDZ9JA8g/4GiKAwdOpTBgwfrXkNy6tQpVq1a9cD9g4ODH3pWJoQQomnQ68ZlZGTE8ePHK2zr378/cXFxjTQiIYQQ9U3vF2fca/fu3QwdOpT169fX+tjg4GDGjBmDi4sLW7durYfRCSGEqAt6PeP6o4eNgjp27Bjx8fHs3buXxx57jDlz5vDFF1/U6jsziXwSQoiGoTeNqz6joM6ePcsLL7xAy5blzWfo0KEkJyfXqnFJ5JMQQjQMvbpVWF9RUH/961/597//TUFBAcXFxfzv//4veXl5DVWWEEKIWtCbGRfUXxSUvb09arUaLy8vLCwssLe35/vvv6/fYpoIiXwyzNoMtS6Q2vTRnyLyqTL1FQV169YtRo8ezSuvvAJASEiILine0Enkk+HVZqh1gdSmjyTyqRZqEwWVlZXF3//+d8rKyrh58yZ79uxh7NixDTRSIYQQtaFXM67aqmkUVJ8+fRg9ejQTJ05Eo9Hw8ssv89xzz9XqWhL5JIQQDUPvI58q09BRUBL5pH8MtTZDrQukNlJlv48AACAASURBVH0kkU81JFFQQghhuAyycUkUlBBCGK6HXpzh5eWl+3nRokVkZ2cD5Qsdpk2bhqurK15eXrrt9eX27du89dZbuLi44OLiQkJCQqXjTUlJqdexCCGEqH8PPeNKTU3V/ZySksKcOXMA+Oijjxg/fjyenp6Eh4cTEBDA+++//+gjrURwcDBdunTho48+4urVq7i6ujJo0CDat29fb9d8EIl8EkKIhlFt4yorK8PPz49z586Rl5eHtbU1bdu2BcDDw4NRo0aRm5vLzJkziYiIQKvVcuvWLQDu3LmDmZlZlecPCgri0qVLXLhwgfz8fKZMmcKMGTOIiYkhNjaWgoICRowYwfTp01m+fDmXL1/GyMiId955h8GDBzNw4ECeeuopANq1a4eFhQV5eXm0bt2aJUuWcObMGaysrMjPz6+0ng8++IBt27ahKArz5s0DwNfXl2HDhjFu3LgafZAS+SSEEA2j2sb17bffolKpiIqKQqvV4u3tjaurK9HR0URHRwMQGRlJcHAwlpaWvPXWW0ydOpXw8HBKS0uJioqqdhBnzpwhMjISrVaLWq3WJWLk5ORw8OBBTExMmDdvHu7u7owcOZLc3Fw8PT3Zt28fQ4YM0Z3n4MGDlJSU0KtXL3bu3AlAYmIiFy9eZOLEiZXW89VXX+Hu7o63tzf/+Mc/KCoq4sSJE5VmGwohhGg81TauAQMGYGFhQUREBBkZGVy8eJHCwsJK91+4cCErV67EycmJpKQk3njjDfbv34+RkVGlx0yYMAFzc3OgPCz3xIkTWFpaYmtri4lJ+RCPHTtGRkYGgYGBQPnMKTMzExsbG6C8Qb377ruEhIRgYmJCamoqU6ZMAaB79+7Y2dlVWU/Xrl2xsrLi5MmTXLp0CQcHB0xNTWvyGeo1iXwyzNoMtS6Q2vRRg0c+HTp0iMDAQKZPn45arSY/P5/KHv26du0aGRkZuiXoY8aMYcWKFeTn5+tuLz7IvdFNWq1W9/u9txm1Wi07d+7EwsICgNzcXNq1awdAeHg4oaGhhIaGYm1tDZSvLLx3nHcbYFX1uLu7c+DAAS5dusTcuXOr+2gMgkQ+GV5thloXSG36qFEin44fP87YsWNxd3endevWpKSkoNFoMDY2pqysPHnB2NgYjUaDpaUlpqamnDp1CoDTp09jbm5eZdMCSE5OpqSkhOvXr/Pll18+8H1azz//PJ9++ikA58+fx8XFhTt37pCcnExYWBifffaZrmlBeXBufHw8Wq2W7OxsvvnmmyrrAXB2dub48ePk5eVVGcorhBCi8VQ74/Lw8GD+/PkkJCSgUqno168fWVlZjBw5EldXV2JiYhg+fDgzZ84kJCSETZs2sWrVKoqKijA3NycoKKjaQZiamuLp6cmtW7eYNWsWvXr1Ii0trcI+S5cuZfny5bi4uACwYcMGWrZsSWBgIMXFxbz++uu6fVevXo2npyfnzp1j7NixWFlZ0bt37yrrgfIZ3rPPPqvbtzYk8kkIIRpGo0c+3W1sjX1rTlEUbt++zZQpUwgLC6NDhw61Ol4in/SPodZmqHWB1KaP9DbyKSwsjNjY2Pu2d+zYkaeffrohhlCt9PR0ZsyYwZw5c2rdtIQQQjScRp9xGQqZcekfQ63NUOsCqU0fyfu4hBBC/Onpdciur68vAwcOJCsri5iYGLy9vXVvMa6t3Nxcli5dSm5uLmZmZrz//vs88cQTNT5eIp+EEKJh6HXjuisuLo4dO3boop8eho+PD2PGjOHFF1/ks88+4/333+fDDz+s8fES+SSEEA1DrxqXoiisW7eOw4cP07FjRzQaDQkJCSiKwpw5c+jWrRvDhg3D09OTqKgowsLCSExMpLS0FCcnJ5KTk1GpVPed99q1a/z000/s2LEDKH8Q+W7slBBCiKZFrxpXUlISZ8+e5cCBA9y8eZOJEyfi7+/Ppk2bCA4OJiMjgz179uDp6cmJEye4fv06eXl5nD9/Hjs7uwc2LYDMzEy6dOnCunXrOHXqFB06dGDZsmUNXF3jkMgnw6zNUOsCqU0fNXjkU1OSmprK6NGjUalUtG3blmHDhlX4+6BBg1i2bBkajYaMjAzGjRvHyZMnSU9PZ/jw4ZWet6ysjLNnzzJ37lwWLVpEdHQ0vr6+hIeH13NFjU8inwyvNkOtC6Q2faS3z3HVlcryB+8yNTXFxsaG+Ph4evTowaBBgzh+/DinT59mxowZlZ63Q4cOmJubM2LECKA89Hf16tW1GpskZwghRMPQq8Zlb29PaGgoU6dO5c6dOxw5coRnn322wj4ODg5s3ryZV199lYEDB7J69Wq6dOlSZV5it27d6NSpE1999RUODg58+eWX/PWvf63V2Az1OS4hhGhq9KpxOTk5kZ6ezoQJE2jfvj09e/a8b5/hw4fj5+fHwIEDadOmDe3atavyNuFdQUFBrFixgvfee4+WLVuybt26eqhACCHEo5LkjDpiqDMuQ73vDoZbm6HWBVKbPvrTf8f1qLy8vLhx48Z926dOncqLL77YCCMSQghRW3oZ+bR7926GDh3K+vXra3VceHg4cXFxREREoNFo2Lx5M3FxcbqmVVpaire3NykpKfUxbCGEEHVAL2dcBw4cYO3atQ984WR1vv/+e5YuXcrFixcrbM/IyGDx4sWcPXu2jkYphBCiPjS5xpWSksK2bdtQqVRkZWXh6OhIixYtSE5OBsDR0ZH09HT8/f1ZsmQJ27ZtIzIyEoCYmBi+//57/P39Kz3/7t27WbFiBT4+PhW279mzhxkzZrBz5876K04IIcQja5K3Cu82n7179xIREUHbtm2JiYnB2tqaVq1a0bdvX1avXo2DgwNXrlzht99+A2Dfvn2o1eoqz71mzRr69+9/33YfHx+cnJzqpR4hhBB1p8nNuAB69+5N586dAbC0tNTlBnbp0qXC4gojIyPc3NzYv38/arWaq1ev8swzzzTKmPUxHb6mDDWGBgy3NkOtC6Q2ffSniHz6Y6agsbFxpfu6ubkxY8YMmjdvjqura30PrVKyHF7/GGpthloXSG36SF4k+QBWVlZ06tSJyMjIRm1cQgghGobeNy6AcePG0bNnTx5//PHGHooQQoh6pvfJGWVlZfj4+ODs7Mzo0Y0XdCu3CvWPodZmqHWB1KaPJDnjDxRFYejQoQwePFi3IvDUqVOsWrXqgfsHBwfLrEwIIfScXjcuIyMjjh8/XmFb//79iYuLa6QRCSGEqG8P/R2Xl5eX7udFixaRnZ3N1atXcXV11f3j6OiInZ1dnQy0Mrm5ubz88stMnDgRDw8Pfvzxxwfu5+joSFZWVr2ORQghRP176MaVmpqq+zklJQVFUWjXrh1xcXHExcURGxuLlZUVK1eurJOBViYgIIAxY8awf/9+5s6dW2VqhhBCCP1X7a3CsrIy/Pz8OHfuHHl5eVhbW+teyujh4cGoUaPIzc1l5syZREREYGlpCcDevXt57LHHcHFxqfL8vr6+mJqakp6ezu3bt5k9ezaTJk0iKCiI7777jt9//52XXnqJIUOG4OfnR0FBAWZmZixbtgxbW1vWrFmjO1dWVhatW7cGoKCggAULFnD58mV69uxJcXExALdu3WLx4sXk5OSQm5uLvb09a9aswcfHhwEDBjB58mSgfEY5f/78RnugWQghxINV27i+/fZbVCoVUVFRaLVavL29cXV1JTo6mujoaAAiIyMJDg7WNS2NRsO2bdvYsmVLjQaRmZlJVFQUV69eRa1WM2TIEABKSko4ePAgUP7qkeXLl2Nra8v58+eZM2cOSUlJNGtWPml0dnYmOztbd83AwEBsbW3Zvn07J0+eJDExEYDDhw9jY2NDYGAgJSUljB8/nh9++AF3d3eCgoKYPHky2dnZXLt2TZqWEEI0QdU2rgEDBmBhYUFERAQZGRlcvHiRwsLCKo85cuQI3bt3x9raukaDUKvVqFQqOnXqRL9+/Th9+jQATz/9NAC3b9/mzJkzLFq0SHdMYWEh+fn5umb5+eef8+OPP/Lqq6+SmJhIamoqGzdu1NXQtWtXACZMmEBaWhphYWFkZGRQUFBAYWEhgwYNYtmyZWRlZREXF1frh5kl8kk/GWpthloXSG36qMEjnw4dOkRgYCDTp09HrVaTn59PdY9+JScnM27cuBoP4t5IJ61Wi4lJ+bDMzMx025o3b15hteDly5exsLDg8OHDDBgwAHNzc2xsbOjSpQuZmZkYGRlVGOfda4SHh5OUlMTkyZMZPHgwv/zyC4qiYGRkxKRJk0hISCAxMZHQ0NAajx/kOS59ZKi1GWpdILXpo0aJfDp+/Dhjx47F3d2d1q1bk5KSgkajwdjYmLKyMqC8KWg0Gt0x33333QMT2CuTmJiIoihkZ2eTlpbGc889V+HvrVq1onv37rrGdfToUaZNmwZAbGwsu3fvBuD8+fPk5eXRo0cP7O3tdfunpaXpEuSPHj3KlClTmDhxIsXFxfz0009otVqgfOYXGRlJ586d5XkvIYRooqqdcXl4eDB//nwSEhJQqVT069ePrKwsRo4ciaurKzExMQwfPpyZM2cSEhJC165dyczMpFOnTjUeRFFREe7u7pSUlLBy5Urd7b97vffee/j5+RESEoJKpSIgIAAjIyMWL17M4sWLiY2NxdTUlI0bN2Jubs6bb76Jr68v48ePp0ePHrpbhd7e3vj5+REcHEzLli2xs7PTLZPv3LkznTt3xs3NrcZjF0II0bAaPfLJ19eXgQMHVvserfqmKAq5ubl4eXlx4MABmjdvXqvj5Vah/jHU2gy1LpDa9JHeRj6tX7+eY8eO3be9b9++DXH5GklKSsLPzw8/P79aNy0hhBANp9FnXIZCZlz6x1BrM9S6QGrTR03qfVwPinwCdA8jT5o0ialTpzZYzNLRo0fx9vau9O8S+SSEEIahTiOfAHx8fBgxYgT79u3D1dWV999//9FHWQWtVsvHH3/M22+/rVsdKIQQwnDVaeTTJ598wk8//cSOHTsAcHd3x97evsrzP2rk04ULF7hw4QKrVq0iPDxcd16JfBJCCAOlVCM1NVXx8/NTFEVRNBqN8tJLLymff/650rt3b90+I0aMUDIzM5XvvvtOcXNzU959911FrVYrs2bNUrKysqo8/8KFC5VXXnlFKSkpUX7//XfF3t5eyc3NVQIDA5WXXnpJt9+UKVOUH374QVEURTl37pwyevToCuc5ceJEhf39/f2VDz74QFdD7969lczMTCU+Pl7ZsmWLoiiKUlxcrDg5OSnp6enK8ePHFU9PT0VRFCUrK0sZN25cdR+NEEKIRlCnkU9lZWWcPXuWuXPnsmjRIqKjo/H19a0wE3qQuoh8+qOGjnySxRn6x1BrM9S6QGrTR42yHL42kU8dOnTA3NycESNGAOVNYvXq1dUO4lEinyrT0JFPQgghGkadRj5169aNTp068dVXXwHw5Zdf8te//rXaQTxK5FNlJPJJCCEMU51HPgUFBbFixQree+89WrZsybp166odxKNEPlVGIp+EEMIwNfoDyBL51LQZ6n13MNzaDLUukNr0kUQ+1SOJfBJCCP3Q6DMuQyEzLv1jqLUZal0gtekjvYh8uuvs2bMNMqO6ffs2b731Fi4uLri4uJCQkPDA/by8vEhJSan38QghhKhfdR75BHDnzh1WrVpFaWnpo42uBoKDg+nSpQvx8fGEhYWxdu1a8vLy6v26QgghGkedRj5FRERgaWnJunXr8Pb25ptvvql2AEFBQVy6dIkLFy6Qn5/PlClTmDFjBjExMcTGxlJQUMCIESOYPn06y5cv5/LlyxgZGfHOO+8wePBgBg4cyFNPPQVAu3btsLCwIC8vj9atW7NkyRLOnDmDlZUV+fn5ldbzwQcfsG3bNhRFYd68eUD5opFhw4Yxbty4h/5whRBC1IPqojVqE/mkKIqSnJys+Pj4KIqiVNinMoGBgcqECROUW7duKTdu3FCcnJyUM2fOKHv37lVGjRqllJaWKoqiKP/4xz+U5ORkRVEUJScnRxk5cqRy8+bNCudKSEjQHRMSEqLMnz9fURRF+fXXX5W//e1vyokTJyqt57ffflNGjBihaLVapbCwUHFwcFCKioqqHb8QQoiGVaeRT1euXGHr1q2EhYXVqnlOmDABc3NzoPz1IydOnMDS0hJbW1tdisaxY8fIyMggMDAQKJ85ZWZmYmNjA5Q/xPzuu+8SEhKCiYkJqampTJkyBYDu3btjZ2dXZT1du3bFysqKkydPcunSJRwcHDA1Na1xDbI4Q/8Yam2GWhdIbfqoyUc+HT58mIKCggqpFq6urkRERNCyZeUD/GPk093f70Y+3d2+c+dOXcxTbm4u7dq1A8pjnEJDQwkNDcXa2hq4P/LpbgOsqh53d3cOHDjApUuXmDt3bnUfjRBCiEZQp5FPHh4eJCcnExcXp4tbiouLq7JpASQnJ1NSUsL169f58ssveeGFF+7b5/nnn+fTTz8F4Pz587i4uHDnzh2Sk5MJCwvjs88+0zUtKI98io+PR6vVkp2drfu+rbJ6AJydnTl+/Dh5eXnyOhMhhGii6jzy6W60Um2Ympri6enJrVu3mDVrFr169SItLa3CPkuXLmX58uW4uLgAsGHDBlq2bElgYCDFxcW8/vrrun1Xr16Np6cn586dY+zYsVhZWdG7d+8q64HyGd6zzz6r21cIIUTT0+gPIAcFBQE0+q05RVG4ffs2U6ZMISwsjA4dOtTqePmOS/8Yam2GWhdIbfpIbyOfwsLCiI2NvW97x44dde/camzp6enMmDGDOXPm1LppCSGEaDiNPuMyFDLj0j+GWpuh1gVSmz5qUpFPTYGvry8xMTEEBgYyfPhwduzY8cjnbKioKiGEEA+nQW4V1re4uDh27NihS9B4WA0ZVSWEEOLh6NWMS1EU1q5dy5gxY/Dy8uK3335jxYoV5OTkMGfOHF5//XXdkvmoqCjGjh0LQGlpKQ4ODtU2pLtRVUIIIZouvZpxJSUlcfbsWQ4cOMDNmzeZOHEi/v7+bNq0ieDgYDIyMtizZw+enp6cOHGC69evk5eXx/nz57Gzs0OlUlV67kOHDlFUVISzs/NDje1h79Xqgw4dWjX2EOqNodZmqHWB1KaP6rouvWpcqampjB49GpVKRdu2bRk2bFiFvw8aNIhly5ah0WjIyMhg3LhxnDx5kvT0dIYPH17peR82qupesjhD/xhqbYZaF0ht+uhPvzijshinu0xNTbGxsSE+Pp4ePXowaNAgTp48ydGjR+9rcve6N6rK1dUVKI+qunXrVv0UIoQQ4qHp1YzL3t6e0NBQpk6dyp07dzhy5AjPPvtshX0cHBzYvHkzr776KgMHDmT16tV06dJF9yqWB/Hw8MDDw0P3u7W1tS6y6lFoNGXk51+hrKzkkc/VWHJzm6HVaht7GDVmYtIcS8sOGBvr1b/aQoha0Kv/dTs5OZGens6ECRNo3749PXv2vG+f4cOH4+fnx8CBA2nTpg3t2rWr8jZhfcrPv4KZWQvMzTthZGTUKGN4VCYmzSgr04/GVZ5+coP8/Cu0b9+5sYcjhKgn8gByHXnQd1yXL/+Hxx/vprdNC/SrcUF588rJ+Y1OnZ6sdt8/23cKhkBq0z96G/nUVHh5eXHjxo37tk+dOpUXX3yxXq6pz01LH8nnLYThe+jG5eXlRXh4OACLFi3ijTfewMrKilOnTvHuu+9SWlqKlZUV69evp02bNnU24D/Kzc3Fx8eHa9euYWpqysqVK3Uvl7yXo6Mjn3zyCU888US9jUUIIUT9e+hVhampqbqfU1JSdKv9Fi1axIYNG4iPj6dXr16EhoY++iirEBAQwJgxY9i/fz9z587F39+/Xq/3KFq1fowOHVrV+T+tWj9Wo+v/9NNZ1q1bVc9Vljt79gxbtgQ2yLWEEH8u1c64ysrK8PPz49y5c+Tl5WFtba1boefh4cGoUaPIzc1l5syZREREcPDgQVQqFaWlpeTk5FR4ueOD+Pr6YmpqSnp6Ordv32b27NlMmjSJoKAgvvvuO37//XdeeuklhgwZgp+fHwUFBZiZmbFs2TJsbW1Zs2aN7lxZWVm0bt0agIKCAhYsWMDly5fp2bMnxcXFANy6dYvFixeTk5NDbm4u9vb2rFmzBh8fHwYMGMDkyZOB8hnl/Pnz6/SFkmamJri88+irFf8ofqMrNbkz3qePLb6+tnV+/Qe5ePFX8vOvNci1hBB/LtU2rm+//RaVSkVUVBRarRZvb29cXV2Jjo4mOjoagMjISIKDg7G0tATg559/5pVXXsHExIS333672kFkZmYSFRXF1atXUavVDBkyBICSkhIOHjwIlH8PtXz5cmxtbTl//jxz5swhKSmJZs3KJ43Ozs5kZ2ezZcsWAAIDA7G1tWX79u2cPHmSxMREoPyZLRsbGwIDAykpKWH8+PH88MMPuLu7ExQUxOTJk8nOzubatWsG9xbkb745xccfBwNgbd2HtLTvKCkp4fXX5xIdHcnFixlMmeLJlCnTCA39Jzk5l/nPf36loKAAV1c1np7T0Wq1BAZu5NSpkxgZwZgx43jppZf55ptTbN0aiEajpWPHjvzyy8/cuXOHnTtD+Z//mcLatau4ciWXvLwr9O8/EF/fZXz77WnCw3dgZmbGxYu/0rNnL1asWPN//75FsG/fXoyNjRk8eCh///ubXLt2lffee5ecnByaNWvGrFlzGDBgUCN/qkKIhlZt4xowYAAWFhZERESQkZHBxYsXKSwsrPIYa2trjh07RmRkJPPmzSMyMrLK/dVqNSqVik6dOtGvXz9Onz4NoHtX1+3btzlz5gyLFi3SHVNYWEh+fr6uWX7++ef8+OOPvPrqqyQmJpKamsrGjRt1Ndx9M/OECRNIS0sjLCyMjIwMCgoKKCws1KVuZGVlERcXp3sQuaYetDomN7cZJiYN84x3Ta5jbNxMt3jByAh27NhFSMg/+eij99i1K4r8/HymT3+RadO8aNbMiJ9//pHt28PQajW8/PI0Bg4cxJkz6Vy5kktERBSlpSX8/e8z+ctf/oKZ2WNkZv7Gvn0JtGzZigMH9vPNN6d57bX/xxdffI61tTXr1r1HaWkpL77ozoULP2Ns3IwzZ9KIioqhffsOzJjhzalTJ2jXrj2xsXsIC9uFmdlj/OMfb3D+/E/s2vUJLi6TGDbMgby8K8ya9RqffPIZ5ubmFeps1qxZjSNmJGJH/0ht+qfBI58OHTpEYGAg06dPR61Wk5+fT2Ur6IuLizly5AhOTk4ATJw4kfXr11c7CGNjY93PWq1Wl4hhZmam29a8efMKDwVfvnwZCwsLDh8+zIABAzA3N8fGxoYuXbqQmZl5X8rG3WuEh4eTlJTE5MmTGTx4ML/88guKomBkZMSkSZNISEggMTGx1t/NPWg5vFarbbCl5DW5jkaj1X0mAwcOpqxMS8eOnbC1/RsmJqZ06NCJmzdvUlamRatVcHIaQ4sWLSgr0zJkyDBSU1M5ezYdZ+fxKIoRJiamODk5k5qawpAhw+ja9UnMzMx1xyuKQlmZFkfH0Zw9e4ZPP93FxYu/UlBwnZs3b6MoCk891ZO2bTug1UK3bt0pKLjOr79eZMiQoZiZlTekDz8sn0WfPJnCxYsXCQ7eCpSHJ//222/85S8Vb0drtdoaLSv+sy0/NgRSm/5plMin48ePM3bsWNzd3WndujUpKSloNBqMjY0pKysDypuCRqPBxMQEf39/zpw5A0BiYiL9+vWrdhCJiYkoikJ2djZpaWk899xzFf7eqlUrunfvrmtcR48eZdq0aQDExsaye/duAM6fP09eXh49evTA3t5et39aWhq//fab7tgpU6YwceJEiouL+emnn3TJEGq1msjISDp37szjjz9e/aenx+6Ny7r3PxzuVfE/KBRMTIwfkMeooNFogPLIrQfZsyeSLVsCsbCw5H/+ZwpPPfWUroE2b95ct9/d/9goH9t/l7Xn5V3h5s2baDRaAgO3Ehb2KWFhnxIcvJMePXrVpmwhhAGotnF5eHiQkJCAi4sLb731Fv369SMrK4uRI0fi6upKcXExw4cPZ+bMmVy6dImAgACWL1+Oq6srSUlJFRZPVKaoqAh3d3dmzZrFypUrdbf/7vXee++xZ88eXFxc2LhxIwEBARgZGbF48WL+/e9/M3HiRBYtWsTGjRsxNzfnzTffJDMzk/Hjx7N9+3bdrUJvb282bdqEi4sL7777LnZ2dmRlZQHQuXNnOnfujJubW20/R4P09deHKSkp4caNGxw9+jUDBjzPc8/1JzExAY1GQ1FREV988Tl2dv3vO/buf8xA+Uxp4kQ1o0ePpaSkhHPnfqkyRuqZZ+w4ceIohYWF/7c4aAk//XSW557rT0xM+feqv/6awfTpUyguLqqf4oUQTVa1twqtra2Jj4+vcp8lS5awZMkSALp27UpMTEytBuHs7Ixara6wbe7cuRV+79mzp+65sXs9/vjjD7yt17JlSzZt2nTf9ieeeIKkpKT7tiuKQm5uLnl5ebpbnXWtqLiM+I21++6spuetD6amprz++mvcunULL69XeOqpHnTt2o3MzN94+eUXKSsrY/TosTg4jOCbb05VONbG5q98/HEwW7cGMXmyJ++/v5Zdu3Zgbt6Svn2f5vffL2Fl9eBn6qyt+6BWT+b1119Bq1VwcBjBgAGDeOqpHmzYsAZv76koisKyZStp0cL8gecQQhiuBol8Wr9+PceOHbtve9++fdFoNAwcOPC+xtXQPv/8c/z8/PDz83uod3JVFvlUk+ihpig09J8AzJo1W68in6Dmn/uf7TsFQyC16R+9jXxauHBhQ1zmkTg7Oz/0SySFEEI0nIdeq+3l5aX7edGiRWRnZwPlDwHffa+Vl5eXbnt9uX37NnPnzsXFxYVJkyY9cGZ3d7wpKSn1OhZD8tprs3jttVmNPQwhhLhPnUc+ffTRR4wfP564uDhGjx5NQEDAo4+yZFOUegAACh5JREFUCjt27ODJJ58kPj6ejRs34uPjU6/XE0II0bjqPPJJq9Xq3hx8584d3bNYlQkKCuLSpUtcuHCB/Px8pkyZwowZM4iJiSE2NpaCggJGjBjB9OnTWb58OZcvX8bIyIh33nmHwYMH88Ybb+iW5WdlZekCfUtKSliyZAlnzpzBysqK/Pz8Suv54IMP2LZtG4qiMG/ePKA8imrYsGGMGzfuIT/acnefERMNQ97SI4Thq/PIp7feeoupU6cSHh5OaWkpUVFR1Q7izJkzREZGotVqUavV2NvbA5CTk8PBgwcxMTFh3rx5uLu7M3LkSHJzc/H09GTfvn20bNkSExMTXnvtNY4fP87KlSsBdCsQExMTuXjxIhMnTqy0nq+++gp3d3e8vb35xz/+QVFRESdOnHjkwF4Tk+bcvn0Dc/PW0rwawN0XSZqYNK9+ZyGE3qrzyKeFCxeycuVKnJycSEpK4o033mD//v1V/j/uCRMm6GJ7HB0dOXHiBJaWltja2uoelD127BgZGRkEBpYnjpeVlZGZmal7hUloaCjZ2dlMnToVOzs7UlNTmTJlCgDdu3fHzs6uynq6du2KlZUVJ0+e5NKlSzg4OFT6QO2DPGh1jIWFGZmZmVy5klXj84hH89hjZvTq9RQqlapG+0vEjv6R2vRPk458unbtGhkZGbrnoMaMGcOKFSvIz8/X3V58kD9GPt39/d7bjFqtlp07d2JhYQGUv4erXbt2pKam0r17dzp27IiVlRV2dnacO3fuvsinuw2wqnrc3d05cOAAly5duu85suo8aDk8QKtWHWilx/8u6uMS3YKCIqD6B5P1sbaaMNS6QGrTR00+8snS0hJTU1NOnSp/GPX06dOYm5tX2bQAkpOTKSkp4fr163z55Ze88MIL9+3z/PPP8+mnnwLl0U4uLi7cuXOHw4cPExxcnniem5vLmTNn+Nvf/oa9vT3x8fFotVqys7P55ptvqqwHypfEHz9+nLy8PINLhhdCCENR7YzLw8OD+fPnk5CQgEqlui/yKSYmRhf5FBISwqZNm1i1ahVFRUX8//buLaTp/4/j+KvyQDVBE0fgTUV0uAkDiVi58CLXnHNrGhNXQcM2TCg63DglkCLQLorVTRIigUFXug4ikp0oDSoyLXYhVtDJzc2Vba3w8P5fhN/cbP7+7rfvd/+v//cDhLbvvrGnW593O/D9rly5EpcuXfrHO5Geno7KykoEg0HY7XasX78eAwMDEbepr6/H6dOnodfrAQBNTU1QKBQ4cuQI6urqoNfrsWzZMjgcDuTm5qKyshJDQ0PQarXIzc3Fhg0b5u0Bfr/Cy8vLE27LGGPsf48kR86Yz8xgW+hbc4n2+4P9EMxmM1pbW5GTk7Og/QOB0F/fKpS77GwF/P5gsu+GKBZr22LtArhNjmJ1LV26BFlZ8R2yTZIjZ7S2tqK9vX3O9UqlUjjnVrINDg6iqqoKNTU1Cx5aAOJ+AOQg3veh5WCxti3WLoDb5CjRXUl/xcUYY4wthDSn52WMMcYShAcXY4wxWeHBxRhjTFZ4cDHGGJMVHlyMMcZkhQcXY4wxWeHBxRhjTFZ4cDHGGJMVHlx/cevWLRQXF6OoqAhtbW1ztrvdbphMJmg0GtTV1QkHG/78+TMsFgv27NmD6upqhEIhAMD4+DhsNhu0Wi0sFgtGR0cl7Zkt0W3Dw8OwWCwwGAwwm81wu92S9sxIdNeMkZERbNu2TTieZTIkui0YDOLkyZMwGo0wGo148+aNpD2zJbrt27dvOHz4MEpLS1FeXi675+OMixcvRhzndTGsITOi2+JaQ4hFGBkZocLCQgoEAhQKhUiv19PQ0FDEbXQ6Hb18+ZKIiGpra6mtrY2IiGw2G92+fZuIiC5fvkxNTU1ERNTQ0EBXrlwhIqL29nY6duyYVDkRxGirqKig+/fvExFRb28v6fV6iWr+EKOLiGhqaoqsVivl5eXRhw8fJKqJJEabw+Gg8+fPExHRw4cPqby8XKqcCGK0XbhwQfhzT08PVVRUSJUj+Ddd4+PjVFtbS1u2bCGn0yncfjGsIbHa4llD+BVXlN7eXmzfvh2ZmZlYsWIFNBoNurq6hO2fPn3Cz58/kZeXBwAwmUzo6urCxMQEnj17Bo1GE3E9ADx48EA4qn1JSQkePXqEiYkJicvEadu3bx8KCgoAABs3bsSXL18krhKnCwCuXr0KlUqFrKwsaYNmSXQbEaG7uxs2mw0AoFarce7cOenDIM7jNj09Lbz6CofDEef0k0q8XcDv8wWuWbMGhw4divg75b6GALHb4llDeHBF8Xq9EQfZVSqV8Hg8Mbfn5OTA4/EgEAhAoVAIJ6ycuT56n5SUFCgUCoyNjUmRE0GMNpPJJJz40+l0CicRlZIYXa9fv8bTp0/n/COTWqLb/H4/0tLScP36dZjNZhw8eFA4H53UxHjcrFYr+vr6sHPnTtTX1+Po0aMS1fwRbxcAGI1G2Gy2iJPrRu8jxzUEiN0WzxrCgyvK9PQ0lixZIlwmoojLsbZH3w7AnMuz91m6VPpfvVhtRITGxka8evUKDodDxIK/S3RXOBxGQ0MDzp49m5THabZEt01NTcHn8yEjIwM3btyA3W5HTU2N+CF/Icbz8cyZM7BYLHj8+DFaWlpw/PjxOZ9bii3eroWQ2xryTxa6hvDgirJ69eqIDz5HR0ehVCpjbvf5fFAqlVi1ahW+f/8u/O919n5KpRI+nw8AMDk5iVAohMzMTClyIojRNjk5iVOnTmFwcBDXrl1DRkaGRDV/JLrr+fPn8Pv9qK6uhsFggNfrhc1mw9u3b6WLinHf/21bVlYWUlJSUFJSAgDYsWMHfvz4Ab/fL1HRH2I8H3t6elBWVgYA2Lp1K7KzszE8PCxFTsz7/d92zUfua8h84llDeHBFUalU6Ovrw9jYGMLhMLq7u6FWq4Xtubm5SE9Px4sXLwAALpcLarUaqampyM/PR2dnJwCgo6ND2G/Xrl3o6OgAAHR2diI/Px+pqakSl4nT1tjYiGAwiJaWlqQMLSDxXQUFBbh37x5cLhdcLheUSiWam5uxbt062belpaVBpVLhzp07AID+/n4sX748KZ/jifF83LRpE+7evQsAeP/+PbxeL9auXSuLrvnIfQ2ZT1xryAK/VPJ/4ebNm6TT6aioqIiam5uJiKiqqooGBgaIiMjtdlNZWRlpNBo6ceIE/fr1i4iIPn78SPv37yetVktWq5W+fv1KRESBQIDsdjsVFxeT2WxO2jfUiBLb5vf7afPmzbR7924qLS0VfuTeFa2wsHDRPGZERB6Ph+x2O+l0OjIYDNTf35+cMEp827t37+jAgQOk0+lo79699OTJE1l1zXA6nRHfvFsMa8iM2W3xriF8IknGGGOywm8VMsYYkxUeXIwxxmSFBxdjjDFZ4cHFGGNMVnhwMcYYkxUeXIwxxmSFBxdjjDFZ4cHFGGNMVv4DsidpUETEpsMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate feature importance plot \n",
    "\n",
    "feature_importance = pd.DataFrame(model.feature_importances_, \n",
    "                                  index = df.columns[:-1], \n",
    "                                  columns = ['importance']).sort_values('importance', ascending = True)\n",
    "feature_importance[:20].plot.barh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.993757 using {'max_depth': 10, 'n_estimators': 200}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled GBM\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([200, 300, 400]), max_depth = np.array([10, 20, 30]))\n",
    "model = GradientBoostingClassifier(random_state=seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.993757 (0.015823) with: {'max_depth': 10, 'n_estimators': 200}\n",
      "0.993757 (0.015823) with: {'max_depth': 10, 'n_estimators': 300}\n",
      "0.993757 (0.015823) with: {'max_depth': 10, 'n_estimators': 400}\n",
      "0.993050 (0.015204) with: {'max_depth': 20, 'n_estimators': 200}\n",
      "0.993050 (0.015204) with: {'max_depth': 20, 'n_estimators': 300}\n",
      "0.993050 (0.015204) with: {'max_depth': 20, 'n_estimators': 400}\n",
      "0.993050 (0.015204) with: {'max_depth': 30, 'n_estimators': 200}\n",
      "0.993050 (0.015204) with: {'max_depth': 30, 'n_estimators': 300}\n",
      "0.993050 (0.015204) with: {'max_depth': 30, 'n_estimators': 400}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "                           learning_rate=0.1, loss='deviance', max_depth=10,\n",
       "                           max_features=None, max_leaf_nodes=None,\n",
       "                           min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                           min_samples_leaf=1, min_samples_split=2,\n",
       "                           min_weight_fraction_leaf=0.0, n_estimators=200,\n",
       "                           n_iter_no_change=None, presort='auto',\n",
       "                           random_state=7, subsample=1.0, tol=0.0001,\n",
       "                           validation_fraction=0.1, verbose=0,\n",
       "                           warm_start=False)"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = GradientBoostingClassifier(random_state=seed, max_depth = 10, n_estimators=200)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9822429906542056\n",
      "\n",
      "Accuracy\n",
      "0.9572649572649573\n",
      "\n",
      "Confusion Matrix\n",
      "[[212   2]\n",
      " [  8  12]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.96      0.99      0.98       214\n",
      "         1.0       0.86      0.60      0.71        20\n",
      "\n",
      "    accuracy                           0.96       234\n",
      "   macro avg       0.91      0.80      0.84       234\n",
      "weighted avg       0.95      0.96      0.95       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix \n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3iTZffA8W/SvaArpcyyd9nKKltAsaAMEZmCoqiIP94XXpQtqAxBhijbLQIqU6SULbLKRpYFyihQOulId5rn90c1EFpo2pKmbc7nuryuPvs8tyEnzzj3rVIURUEIIYQwkdrSAQghhChZJHEIIYTIF0kcQggh8kUShxBCiHyRxCGEECJfJHEIIYTIF1tLByBEYQ0ZMoSQkBCjeTY2NpQpU4amTZsybtw4atSoYbQ8Pj6eVatWsXPnTu7cuUPZsmVp2LAhw4YNo3Xr1rkeZ9euXfz0009cuHCBtLQ0/Pz8eOmll+jfvz92dnZmOz8hihuV1HGIkm7IkCHodDomTJhgmJeRkcGlS5f44osvcHJyYseOHTg4OABw/fp1hg8fjk6nY/jw4TRo0ID4+Hg2bdrEnj17GD16NO+++67RMT788EPWrl3Liy++SJcuXXB2diYkJISvvvqKDh06sHDhQmxsbIr0vIWwGEWIEm7w4MHKG2+8keuydevWKbVr11b27dunKIqi6HQ6JTAwUOnatasSGxubY/2FCxcqtWvXVnbv3m2Yt3HjRqV27drK2rVrc6y/bds2pXbt2srGjRuf0NkIUfzJMw5Rqrm6uhpN7927l9DQUMaNG4enp2eO9UePHk2VKlVYtmyZYd7q1aupU6cOL7/8co71e/TowYgRI/Dw8HhsHOvWreP555+nUaNGPPvss6xfv96wrHPnzsyYMcNo/Y8//pjOnTsbpuvUqcOyZct4/vnnadmyJUFBQdSpU4dTp04ZbffDDz/QuHFjkpOTATh37hzDhg2jcePGtGrVipkzZ5KamvrYWIXIizzjEKWCoijodDrDdHp6OufOnWPBggVUqFCBFi1aAHDw4EHUajUBAQG57sfGxoYuXbrw9ddfExcXh06nIzQ0lDfeeOORx37wFlluvv76a+bMmcOrr75K+/btCQkJYcqUKTg7OxMYGGjyOS5ZsoSJEyfi4eFBixYt8PX1JSgoiKZNmxrW2b59O506dcLFxYUrV64wePBgmjRpwsKFC4mNjWX+/PncunWL5cuXm3xcIR4miUOUCvv376dBgwZG8xwdHWndujUffPABLi4uANy+fRsPDw+cnZ0fua9KlSoBEBERQVZWFgAVKlQoUFx6vZ5ly5bRp08f3n//fQDatGlDeHg4J06cyFfiaNu2LQMHDjRM9+jRg+3bt/P++++jUqmIiori5MmTfP755wB8+eWXeHl5sWLFCuzt7QGoWrUqgwYN4tixYzz11FMFOichJHGIUqF58+Z88MEHAFy+fJk5c+bQunVr5s6da/jShOwrk7weYj+4/N+/9Xp9geK6du0a8fHxRredAObPn5/vfT38ZlhgYCBfffUVZ86coUmTJgQFBeHq6kr79u0BOHr0KF26dEGtVhuuxpo0aYKrqyuHDx+WxCEKTBKHKBXc3Nzw9/cHwN/fn/LlyzN8+HDs7e2ZO3euYb2KFSty+PBh0tPTDW9ZPez27dsAlC9f3jAvIiLikceOiorC29sbtTrnI8P4+HiAXJ+n5JeXl5fRdIMGDahWrRpBQUE0adKE7du307VrV0OijI+PZ926daxbty7HvqKjowsdj7Be8nBclEqtW7emX79+bN68mT179hjmd+rUiczMTPbu3ZvrdoqisGfPHvz9/fH09MTT05P69etz4MCBRx5r+PDhDB8+PNdlbm5uAMTFxRnNv3btGidPnjRMP3xFk5KS8vgT/EdgYCDBwcFERkZy6tQpo1tfrq6u9O3bl19++SXHf2+++aZJ+xciN5I4RKn1n//8Bzc3N2bPnk1GRgYAAQEB+Pv7M3fu3Fx/dS9fvpyrV68aPQwfNmwYly5d4ueff86x/ubNm7ly5Qo9e/bMNYbq1avj7u7Ovn37jOYvWrTIcCXk6upKVFSUYZler8/xttSjBAYGcvv2bZYuXYq3tzctW7Y0LGvevDlhYWE0bNgQf39/w5XY/PnzuXz5skn7FyI3NtOnT59u6SCEKIyNGzdiZ2eX48vbyckJlUrF9u3bcXNzo1mzZqjValq1asXWrVv54YcfUKvVZGRkcPHiRRYvXsz333/PqFGjGDx4sGE/derU4erVq6xatYrIyEjUajU3btzgxx9/ZNGiRXTv3p2xY8eiUqlyxKZWq7GxsWHVqlWkp6cDsGHDBtauXcvkyZOpXr06t2/fZtOmTZQpU4aUlBTDF7u9vT3Dhg0Dst+oatu2Lc2aNTPa/79Jaf/+/fTu3ZuOHTsalvn5+fHll1/y999/4+zszOXLl5k6dSrXr19nzJgxOV5VFsJUUjkuSrwhQ4bg7Oyc6yumGRkZPPvssyQmJhIcHGx41pCQkMB3331HUFAQt27dwsXFhUaNGjF06FDatGmTYz96vZ7169ezYcMGbty4QUZGBtWqVaN///7069cPW9vHPy788ccf+fbbb4mIiKBq1aq89dZb9OjRAwCtVsvMmTPZuXMntra29OrViwoVKvDDDz8YbrPVqVOH//3vf7z22ms59v3NN98wa9Ys1q1bR5MmTYyWnThxgoULF3L27FkcHBxo1qwZ//3vf6lVq5ZpjStELiRxCCGEyBd5xiGEECJfJHEIIYTIF0kcQggh8qXIEodWqyUwMJBbt27lWHbx4kX69OlD9+7dmTRpklGfQ0IIIYqXIkkcZ86c4ZVXXuH69eu5Lh8/fjxTp05lx44dKIpi1HOoEEKI4qVIuhxZv34906ZN43//+1+OZbdv3yYtLc3wGmGfPn1YvHixUWduprh3Lxm9XsHLy5XYWO0Tibskk3bIJu1wn7RFNmkHUKtVeHi4FHj7IkkcH3/88SOXRUVFodFoDNMajYbIyMh8H0OvV9DrFcPflvD7mVtsOH6DtMwsixxfCCFMUa6sE9+N6Zz3io9g8U4O9Xq9UcWtoii5VuDmxcvrfhWsRuP2RGLLr40nJGkIIUo/iycOX19foz6DYmJi8PHxyfd+YmO16PUKGo0b0dFJTzJEk6VmSNIQQpR+Fk8cFStWxMHBgRMnTtC8eXM2b95sGE+gJPthlGXPwZIJtDiRdrhP2iKbtbTD3fhUVu0P5VJEgtH8TvV8Gdy2ZqH2bbHEMXLkSMaMGYO/vz/z5s1j8uTJaLVaGjRowNChQy0VlhBClGi6LD2/n73FxuM3yMy6/7y3XBlHXutQm/oV3VGr8/844EFFmjgeHBdh5cqVhr/r1q3LL7/8UpShCCFEqXMtOolV+0K5EZtsmKdWQY/GlenTogr2to8f/dJUFr9VJYQQ4snYdOKmUdKo6u3K6x1rU9X7yXahL4lDCCFKiaEBNTl/O54svULfp/x4rlElbAp5Wyo3kjiEEKIESk7PxM5GbXT7ycvVgbe71KWChzO+ZZ3Mdmzp5FAIIUqYkLBo/rfuBBtP3MyxrFlVL7MmDZArjlxJBbgQoji6l5zOt39e4fi1WAC2nQ6nVQ0Nfk/4GUZeJHHkorBJw9Huyby5IIQQkN2jxr6Ld/npSBgpDxQal3GyR5ueWeTxSOLIRWGTRp8Wfk8wGiGENbubkMrq/aFcvJOzkG9Aq+q4OBT917gkjjxYugJcCGGdsvSK4bZ5ZpbeMN+3rBMj2teifkV3i8UmiUMIIYqZe8npzNt+nhsx97t/V6vg+SaV6d38yRXyFZQkDiGEKGbKONnzYPWFuQr5CkpexxVCiGLGRq1iZMfaONrZMKBVNT7s07TYJA2QKw4hhLCo5HQdey5E8HyTSqgfGIvIz9uVxYNb4myBh995KX4RCSGElTgWFsM3f14hISUDJ3sbnmlQwWh5cUwaIIlDCCGK3MOFfABrj1yjZXUNbk52FozMNJI4hBCiiDyqkK+ssz2vBtQsEUkDJHHk8PuZW5YOQQhRCj2qkK9jXV9eaW2ZQr6CKjmRFpENx28Y/pauQ4QQhfWoQj6fMo68/s+IfCWNJI6HPNjdiHQdIoQorG2nw1kfct0wnT0iXyX6tPCzeCFfQUnieIwejStZOgQhRAnXtWEFdl+IIFabjp+XC693rE01jZulwyoUSRxCCPEE6fUK6gdG3XOyt2VE+1rcjE3muUYVsbUp+XXXkjiEEOIJSE7X8dPhMDJ0Wbz9TD2jZY2reNK4iqeFInvyJHEIIUQhHbsWw7cHrhCfkgFA61o+NPXzsnBU5iOJQwghCuhecjrf/XmVY9dijOafvXlPEocQQoj7FEVh/6W7rDl8jZQMnWH+v4V8T1X3tmB05ieJQwgh8uFuQipf7Q/lQq6FfNVwcSgZ1d+FIYlDCCFMoFeyC/l+PZazkO+1DrVoUNHDgtEVLatLHP9WcBZmXHEhhPVRAaF3Ew1J499Cvt7N/XCwsl4mrC5xmJo0pLsRIcSDVCoVrwbU5OKdeHzcHEtFIV9BWV3iMDVpSHcjQli3vyMSqKZxNeoWxNPVgUm9GlPJw7lUFPIVlNUljgf9MKq9pUMQQhQzyek61h4JY+/FuwQ2qcyAVtWMlhenIVwtxaoThxBCPOjhQr7fz4TTsoa31d6SehRJHEIIqxefksG3f17hWJhxIV9TPy/KOtlbKKriSxKHEMJqPbKQz8mOYf8U8qlUqsfswTpJ4hBCWKXIhFRW/3GZC7fjjeZ3qFuOV1pVx9Wx9BfyFVSRJI6tW7eydOlSdDodw4YNY9CgQUbLz58/z9SpU8nMzKR8+fJ8+umnlClTpihCE0JYobCoJD7acoYMnXEh34j2tWhYyXoK+QrK7O+TRUZGsmDBAtasWcOmTZtYt24dV65cMVrn448/ZsyYMWzZsoVq1aqxevVqc4clhLBift6uVPRwBkD1TyHfrJeaS9IwkdkTx6FDh2jVqhXu7u44OzvTvXt3goKCjNbR6/UkJycDkJqaiqOjo7nDEkJYMRu1itc71KaaxpUPezdlYOvqVlf9XRhmv1UVFRWFRqMxTPv4+HD27Fmjdd5//31GjBjBJ598gpOTE+vXr8/3cby87r9brTHx1TlT1yupSvv5mUra4T5rbIuzN2IJOhXOuBcao/7nQbdG44ZG40bzur7y8LsAzJ449Hq90f8YRVGMptPS0pg0aRLffPMNjRo14uuvv2bChAmsWLEiX8eJjdWi1ytoNG5ERyeZtI2p65VE+WmH0kza4T5ra4sHC/kAKpV1omvDClbXDrlRq1VGP7bzvf0TjCVXvr6+REdHG6ajo6Px8fExTIeGhuLg4ECjRo0AePnllwkJCTF3WEKIUuz4tRgmrDtuSBoAW0/dRPdAr7ai4MyeONq0acPhw4eJi4sjNTWV4OBg2re/39WHn58fd+/eJSwsDIDdu3fj7+9v7rCEEKVQfEoGi4MvsHDHBUP1N0Dzql5M793UqvuXepLMfquqXLlyjB07lqFDh5KZmUm/fv1o1KgRI0eOZMyYMfj7+zNr1iz+7//+D0VR8PLy4pNPPjF3WEKIUkRRFP74O5IfD4VJIV8RKJI6jp49e9KzZ0+jeStXrjT83aFDBzp06FAUoQghShkp5Ct6UjkuhCjRdp2/Y5Q0pJDP/CRxCCFKtL5PVSUkLIa45HSea1SJvi2sb0S+oiaJQwhRYmTo9GRmZeHicP/2k6OdDW91roO9rQ3VfayvTsUSJHEIIUqES3fiWbX/Mn7errzbtZ7RsroV3C0UlXWSxCGEKNZS0nWsPXqNPRciALibkEqbWj40r+pl4cislyQOIUSxdeJaDN8cuMK9B2oyHO1sSHvglVtR9CRxCCGKnYR/RuQLeWhEvmZVvXg1oCaerg4WikyAJA4hRDHybyHfmsNhJKffv6oo808h39NSyFcsSOIQQhQL6ZlZLAg6z7mHCvna1SnHoNZSyFecSOIQQhQL9rZqo/oLjZsjr3WQQr7iSBKHEKJYUKlUvNquJn9HJNCuTjn6PlUVRynkK5bynTgyMjKwt7c3RyxCCCuRodMT/NdtujasYHSV4eHiwPyBT+PiIL9pizOT+hjW6XQsWbKEdu3a0bJlS8LDwxk8eLDROBtCCGGKSxEJTPrlBGuPXuPX4zdyLJekUfyZlDjmz59PSEgI8+fPx9bWFo1Gg6+vL9OmTTN3fEKIUiIlQ8fXf1zmo81niIhPBWD72VuExyVbODKRXyal9m3btrF582Y8PDxQqVQ4OjoyY8YMOnXqZO74hBClwMnrsXx94DL3ko0L+Qa0qkZFD2cLRiYKwuRrQhsb44dUWVlZODgU/yKc38/cYsPxG6RlZlk6FCGsTkJKBt8dvMrRq8a3tZv6efJqu1p4SSFfiWRS4ujWrRtjx45l/PjxKIrCrVu3mDdvHl26dDF3fIX2qKQhb2sIYT6KonAgNHtEvocL+Ya2rUnLGlLIV5KZ9Ixj/PjxVKlShQEDBpCUlESPHj1wcXFh3Lhx5o6v0B6VNPq08LNANEJYh5CwGFbsDTVKGu3qlGPOyy1oVVMjSaOEM+mKIy0tjWnTpjFt2jTi4uJwd3dHrVYTHh6Oi4uLuWN8Yn4Y1d7SIQhhFZ6q5k11jRth0Ulo3BwY0b42/pWlkK+0MOmK48GH4J6enqjVanQ6HS+++KLZAhNClBx6vWI0rVareL1jLZ5rVJFZ/VtI0ihlHnnFcevWLQYMGEBWVhYpKSm0bt3aaHl6ejo1a9Y0e4BCiOIrQ6dn04kb/H03gUk9G6NW378FVcXLlUFtXC0YnTCXRyaOSpUqsXLlShITE3njjTdYtGiR0XJ7e3vq1Klj9gCFEMXTpYgEVu8PNdRk7Dx/h+7+FS0clSgKj33GUa9e9vCMe/fuxdPTM8dyvV5vnqiEEMVWSoaOdUeusfufEfn+deZmHN0aVpAH31bApIfjCQkJzJkzh8jISEOy0Ol03Lhxg4MHD5o1QCFE8fG4Qr7O9ctL0rASJiWOiRMn4uHhQYUKFbh9+zZPP/00a9euZeDAgeaOTwhRDEghn3iQSYnj4sWLHDlyhFu3bjFz5kzeeecd2rZty4wZM3jnnXfMHWOB/FsxLoQonD9DI/n+4FXjQj5HO4YG1KBlDanJsEYmJQ53d3ccHByoUqUKV65cAaBJkyaEh4ebNbjCeLhiXCrFhSiY69Fa40K+2uUY2KY6bjIin9UyqY6jXr16fPbZZyiKgo+PDzt37uTAgQM4OTmZO74CezhpSKW4EAXT7+mqaNwc0Lg5MOH5hrzZuY4kDStn0hXHpEmTmDx5MvHx8YwbN453332XjIwMpk+fbubwnoxVr7W1dAhClAjhscm4Odnh7nx/sDZHOxvGPdcQLzdHuXIXgImJ49SpUyxbtgxHR0fKlSvH0aNHyczMxNlZukMWojTI0OnZfPImv50Op3lVL8Z0q2+0vKJnyelaSJifSbeqZs6cia3t/RxjZ2cnSUOIUuLviAQm/3KCzSdvkqVXCAmL4fi1GEuHJYoxk644unTpwvLly+nZsyfe3sbdIRfn5xxCiEdLydCx/ug1dp03LuSr7VuG8u7yw1A8mkmJY+fOnWi1WpYsWWKYpygKKpWKixcvmi04IYR5nLoRy9d/XCbuoUK+l1tWo0uD8qjlFVvxGCYljs2bN5s7DiFEEUhIzeD7g1c5csW4kK9JFU+Gt6+Jl6ujhSITJYlJiaNixcJ1XLZ161aWLl2KTqdj2LBhDBo0yGh5WFgY06ZNIyEhAY1Gw2effUbZsmULdUwhhLGYxDQmrD2O9qFCviEBNWglhXwiH0x6OF4YkZGRLFiwgDVr1rBp0ybWrVtnKCKE7Fteb731FiNHjmTLli3Uq1ePFStWmDssIayOdxlH6lV0N0wH1PZhzoAWtK7pI0lD5ItJVxyFcejQIVq1aoW7e/YHtnv37gQFBTF69GgAzp8/j7OzM+3bZ4/ON2rUKBITE80dlhBWaWjbGkQlpvFyy6o0qpyzx2shTGH2xBEVFYVGozFM+/j4cPbsWcP0zZs38fb2ZuLEiVy8eJHq1aszZcqUfB/Hy+v+gDEajZvRsoenrYW1nvfDrLEdrkUm8uOBK4zr1QhH+/v/zGtX9Wb5qPZWf4VhjZ+JJ8nkxBESEsK6deuIjo5mwYIF/Pjjj7zzzjvY2Dy+klSv1xt9SP99G+tfOp2OkJAQfvjhB/z9/Vm4cCGzZ89m9uzZ+TqR2Fgter2CRuNGdHSS0bKHp61Bbu1gjaytHTKzsgv5tp4KJ0uv4GZvw8DW1QHra4tHkXbIHtr3wR/b+d7elJU2bNjAuHHjqFatGufPn0elUrFz507mzp2b57a+vr5ER99/gyM6OhofHx/DtEajwc/PD39/fwACAwONrkiEEKYJjUhg0s8n2HQiu5APYNf5OySkZuSxpRD5Y1LiWL58OStXrmT06NGo1Wo8PT1ZuXIl27Zty3PbNm3acPjwYeLi4khNTSU4ONjwPAOgadOmxMXFcenSJQD27NlDgwYNCng6Qlif1Awd3x64wszNZ7jzzzCukF3IN7NvM8o62T9mayHyz6RbVfHx8dSsWRPAcJvJ29ubzMzMPLctV64cY8eOZejQoWRmZtKvXz8aNWrEyJEjGTNmDP7+/nzxxRdMnjyZ1NRUfH19TbqSEUJIIZ+wDJMSR7NmzVi8eDFjx441zPv2229p0qSJSQfp2bMnPXv2NJq3cuVKw9+NGzfml19+MWlfQghI/KeQ77AU8gkLMClxTJ06lVGjRrF27Vq0Wi2dO3fG0dGR5cuXmzs+IUQujlyNNkoabo52DGlbg9Y1pZBPmJ9JiaN8+fJs3LiRs2fPEhERgUajoUmTJkY95gohis4z9StwMDSKq1FJBNT2YVDrGrg5yeBKomiY9M0/YMAAXnjhBZ577jmTb08JIZ4MvV4hKT3T6CG3Wq1iZMfaxGrTaVxFCvlE0TLprap+/foRHBxMx44deeeddwgODjbpwbgQonDCY5P5cNNpPtt+Hv0/r9j+q5KniyQNYREmJ46vv/6anTt38vTTT/PVV18REBDAtGnTzB2fEFYpM0vPL8euM/nXk1yNSuJqVBLB525bOiwhgHx2OaLRaHjqqaeIiYnh7t27nD592lxxCWG1QiMSWLU/1Kgmw1atQvfQFYcQlmJS4rh69Srbtm1j27ZtJCcnExgYyNKlS6lXr5654xPCaqRm6Fh/9Dq7zt/hwRRRq1wZXu9QS8b9FsWGSYmjb9++dO7cmYkTJ9KuXTvUarP3xi6EVZFCPlGSmJQ4/vzzT1xdC94hlhAid4qisGzP3xy8HGU0v3EVT4a3q4m3mxTyieLnsYljyJAhfP/99wwbNuyRRUVS8S1EwalUKso8UH8hhXyiJHhs4ujfvz8AgwcPLpJghLBGfZ+qyonrsdQqV4bBbaSQTxR/j00c//YvlZycnGvyWLZsmXmiEqIU0usVdl24w1PVvPFwcTDMd7SzYWbfZrg4SE8MomR45Cc1NjaWc+fOATBv3jwqV65stFyr1bJ8+XJGjRpl3giFKAVuxSWzcl8oV6OSuHg7gfe61zdaLklDlCSP/LS6uLiwePFi7t27R3p6Oh9++KHRcnt7e0kaQuQhM0vPlpM32fLPiHwAx67FcPpGHE38pOpblEyPTByOjo78+uuvAIwePZolS5YUWVBClAahdxNYtf8yd+6lGObZqFW80KwKDSu5WzAyIQrnsdfHYWFhVK9enf/7v//jypUrua7z7wBPQohsqRk61odcZ9c5KeQTpdNjE0e/fv04efIkgYGBuS5XqVRcvHjRLIEJURKdvhHH1wcuE6tNN8xztLOhf8uqPNOgghTyiVLhsYnj5MmTAIbxwIUQjxYWlcS87eeM5jWu7MHw9rWkkE+UKib3HXL48GEA4uLimD59OosWLSItLc1sgQlR0lT3cePp6t5AdiHf213qMq5HQ0kaotQx6R3ATz/9lN9//529e/cyadIktFottra2TJ8+ndmzZ5s7RiGKJb1eQa02vvU0NKAmLg629H+6mhTyiVLLpMSxc+dO1q5dS1JSEn/88QdBQUF4enrSuXNnc8cnRLGj1yvsPH+HvRcjmN67KY52NoZl7s72vNahtgWjE8L8TLpVFR8fT7ly5Th48CCVKlWicuXK2NnZoSgyPoCwLrfikpmx+TTfH7zKrbgUfgm5bumQhChyJl1x1KtXj08++YRjx47xzDPPEB8fz6effkrjxo3NHZ8QxUJuhXwA527fI0Onx95WhhoQ1sOkT/vs2bOJj4+nWbNmvPvuu9y4cYPY2Fg++ugjc8cnhMVdvpvI5F9OsvHETUPSsFGr6N28CjP7NpOkIayOSVcc5cuXZ+7cuaSnpxMVFUWNGjWkg0NR6qVm6Pg55Do7Hyrkq1nOjdc71KaSFPIJK2VS4tBqtUybNo3t27ej1+uxsbGhY8eOzJkzRwZ4EqXS2fA4Vu83LuRzsFXTv2U1ujaokONtKiGsicm3qpKTk9m2bRtnz55ly5YtZGVl8cknn5g7PiEs4m58qlHSaFTZgzkvt6C7f0VJGsLqmXTFsW/fPrZv346bmxsANWrUYM6cOXTr1s2swQlhKc80qMDhK9FEJKQwuE0N2tbykRH5hPiHyYMAZGVlGU3r9Xrs7KTASZR8MUlpZGTpqeDubJinVqt4q0sdHO1sKONkb8HohCh+TLpV1bVrV8aOHcvFixeJj4/n/Pnz/Oc//6Fr167mjk8Is9HrFYL/us2EdcdZtvtv9HrjuiSfMk6SNITIhUmJY/z48Xh6evLSSy/RqlUrXnnlFSpWrMj48ePNHZ8QZvFvId93B6+SrtMTFp3Ejr9uWzosIUoEk25VOTs7M3/+fD755BMSExPx9vaW+72iRHpUIV8FD2dqlitjwciEKDlMShwZGRksWbKE3377jZiYGMqXL0/v3r154403UKul+EmUDJfvJrJqfyi3HxqRr1fTyvRqVgU7G/ksC2EKkxLHRx99xIULF5g4cSK+vr7cvn2b5cuXo9VqGTduXJ7bb926laVLl6LT6Rg2bBiDBg3Kdb19+/YxY8YM9sRh8dcAACAASURBVOzZk7+zEOIx0jKzWH/0mhTyCfGEmJQ4tm/fzo4dO/D09ASgYcOGNG3alF69euWZOCIjI1mwYAEbNmzA3t6eAQMG0LJlyxxDzsbExDBnzpwCnoYQuUvPzGLizyeISrw/dowU8glROCZdm7u6upKRkWE0z9bW1lDX8TiHDh2iVatWuLu74+zsTPfu3QkKCsqx3uTJkxk9erSJYQthGgc7G56q5m2YlkI+IQrPpCuOvn378tprr/H2229TuXJloqKiWLlyJa1bt2b//v2G9Tp06JBj26ioKDQajWHax8eHs2fPGq3z3XffUb9+/SfS2+4vh8P4fn9oofcjSo8+Lfy4FJFAt4YVaCOFfEIUmkmJY8OGDQDMnz/faH50dDR//vknACqVit27d+fYVq/XG/1DVRTFaDo0NJTg4GC++eYb7t69m/8z+IeXV3afWT98dZC0zPvFik72Nmg0eV8ZlUbWdt5RCams3HWRUd3q4/XAcK2VKrjzxRvtJGFgfZ+JR5F2KByTEkdhHlb7+vpy/Phxw3R0dDQ+Pj6G6aCgIKKjo+nbty+ZmZlERUUxcOBA1qxZk6/jxMZq0esVUjPuJw1HOxt6N/cjOjqpwPGXVBqNm9Wct15R2HX+DuuPXictM4vklAz+r3t9VCqVVbVDXqQtskk7ZPeM8O+P7YIwucuRgmrTpg2ff/45cXFxODk5ERwczMyZMw3Lx4wZw5gxYwC4desWQ4cOzXfSeJRVr7V9IvsRxdftuGRW7b/M5chEw7yT12O5GZuMn7f03CyEOZg9cZQrV46xY8cydOhQMjMz6devH40aNWLkyJGMGTMGf39/c4cgSiFdlp6tp8LZfPImuocK+V7vUEuShhBmZPbEAdCzZ0969uxpNG/lypU51qtUqZLUcIg8XYlMZNW+UG5JIZ8QFpHvxJGRkYG9vXT8JopeWmYWP4dcJ/iv20aFfDV83Hi9Y20qSyGfEEXCpJ9mOp2OJUuW0K5dO1q2bEl4eDiDBw8mOjra3PEJYfB3RAI7HkgaDrZqBrepwbQXm0jSEKIImZQ45s+fT0hICPPnz8fW1haNRoOvry/Tpk0zd3xCGDSu4kmrGtk1Qf6VPJjdvwXPNpJCPiGKmkm3qrZt28bmzZvx8PBApVLh6OjIjBkz6NSpk7njE1ZKURQSUjNxdza+LTokoAZN/DxlRD4hLMjkp4g2NjZG01lZWTg4ODzxgArj9zO3LB2CeAJitWnM336eaRtOGRVzApR1siegdjlJGkJYkEmJo1u3bowdO5ZLly6hKAq3bt1iypQpdOnSxdzx5cuG4zcMfzva2TxmTVEc6RWFnefuMGHdCU7fjCNWm876o9csHZYQ4iEmjwBYpUoVBgwYQFJSEj169MDZ2dmkLtWL0oO/Tvu08LNgJCK/bt9LYeamM3z755UcVxmKojxiKyGEJZj0jMPBwYFp06Yxbdo04uLicHd3L/YDOPVoXMnSIQgT6LL0bD0dzuYTDxXyuTvxeofa1C5f1oLRCSFyY1LiWLJkySOXSVfooqCuRGaPyHcrzriQr2fTyvRqWgV72+L940QIa2VS4ggNNe6mPD4+ntOnT+eoBhfCVD+HXGfLyZs5C/k61Kayl9RkCFGcmZQ4Fi9enGPewYMHn1hnhML6uDvbGxXyvfR0Vbo1lJoMIUqCAvdV1bJlS959990nGYuwIl0alOfQ5Sgc7WwY0b4WmjKOeW8khCgWTEocV65cMZrOzMxk69atVKxY0SxBidJDURSOXI2mgruzUY+1apWKcT0a4mxvIzUZQpQwJiWOwMBAVCqV4bVItVpN1apVmTJlilmDEyVbrDaNr/+4wumbcVTTuDK9d1NsHrgV5eJQJJ0zCyGeMJP+5QYHB1OlShVzxyJKCb2isPt8BOuOXjPUZFyL1rL7wh26NZSrVCFKOpPed3z55ZfRarXmjkWUAo8q5OvaoAIBtctZMDIhxJNi0hVH5cqVOX/+PC1btjR3PKKEkkI+IayHSYnDzs6O4cOHo9Fo8Pb2NnqY+csvv5gtOFEyPK6Q7wUZkU+IUsekxNGvXz/69etn7lhECXQvOZ2PNp8xusqQQj4hSjeTEkdycjKDBw/OMX/ZsmVPPCBRsni4OPBso0r8djpcCvmEsBKPTByxsbGcO3cOgHnz5lG5cmWj5VqtluXLlzNq1CjzRiiKFb1eyZEU+rSoQlJaJi80q4xPGScLRSaEKCqPTBwuLi4sXryYe/fukZ6ezocffmi03N7eXpKGFVEUhaNXo/k55DoTezXGy/X+IF72tjaM7FjbgtEJIYrSIxOHo6Mjv/76K5DdA+7jesgVpVusNp1vDlzm1I04AL45cJn/PNtAKr6FsFKF7lZdlF65FfIBXI/R5joeuBDCOkifDyJXd+6lsGp/KKF3E43md6lfnpdbVcPZXj46Qlgr+dcvjOiy9Px2OpxNDxXylf+nkK+OFPIJYfUkcQiDa9FJrNgbSnhcsmGejVpFYJNKvNDMT0bkE0IAkjjEA+JTMoySRnWNG693rEUVL9fHbCWEsDaSOIRBUz8vWtfUcOJ6LP2eqsqz/lLIJ4TISRKHldKmZRKdlEY1jZvR/CFta/DS01WlkE8I8UiSOKxMdiFfDN8dvIKdjZrZ/Zvj9MAbUmWc7JGcIYR4HEkcVuThQj6A9SHXGRZQ04JRCSFKGkkcVkCvKOy9EMFPR4wL+Txc7GlYycOCkQkhSiJJHKXcnfgUVu8P5e+IXAr5WlbDWcb9FkLkU5F8a2zdupWlS5ei0+kYNmwYgwYNMlq+a9cuPv/8cxRFoVKlSsyaNYuyZaXQrDB0WXo2n7zJphM3yMy6X8jnW9aJ1zvWpq4U8gkhCsjsFV2RkZEsWLCANWvWsGnTJtatW8eVK1cMy7VaLdOnT2fFihVs2bKFOnXq8Pnnn5s7rFJNURQm/HCUn0OuG5KGjVrFC80q88lLzSVpCCEKxeyJ49ChQ7Rq1Qp3d3ecnZ3p3r07QUFBhuWZmZlMmzaNcuXKAVCnTh0iIiLMHVapplKp6NiggmG6msaVGX2a8tLT1aT6WwhRaGa/VRUVFYVGozFM+/j4cPbsWcO0h4cHXbt2BSAtLY0VK1YwZMiQfB/H66HqZs1D9QnW5nlvVw6HRtK8ujcvPl0VG7X1Jgxr/yw8SNoim7RD4Zg9cej1eqNxGxRFyXUch6SkJN555x3q1q1L7969832c2Fit0XR0dFL+gy2BtGmZrDkcRjf/ilT1vp88NRo3/u+ZuqhUKuJikx+zh9JNo3Gzms9CXqQtskk7gFqtyvFjO1/bP8FYcuXr60t0dLRhOjo6Gh8fH6N1oqKiGDhwIHXq1OHjjz82d0ilgqIoHLkSzf/WHeePvyNZtS+UrAd6swVkoCUhhFmYPXG0adOGw4cPExcXR2pqKsHBwbRv396wPCsri1GjRvHcc88xadIk+bIzQaw2nc+CzrNk10USUzOB7MGVTt2ItXBkQghrYPZbVeXKlWPs2LEMHTqUzMxM+vXrR6NGjRg5ciRjxozh7t27XLhwgaysLHbs2AFAw4YN5cojF48s5HO259X2tWhe1cuC0QkhrEWR1HH07NmTnj17Gs1buXIlAP7+/ly6dKkowijRpJBPCFFcyLdNMafL0rPtzK3cC/k61KJuBXcLRieEsEaSOIq5iIRUfj12nX+fe9uoVTzfuBIvNpcR+YQQliGJo5ir7OnC800qs/VUONU0rrzeoTZ+3jIinxDCciRxFDPxKRm4O9sbzevd3A9vNwc61i2PjYzIJ4SwMLnXUUxo0zJZsfdvxv10jFhtmtEye1s1XepXkKQhhCgWJHFYWPaIfNFM+KeQLy0zi28OXEFRlLw3FkIIC5BbVRYUp03n2z+vcOK6ceGenY2azCw99rY2FopMCCEeTRKHBegVhb0XI1h75BqpGQ8V8rWrSfNq3haMTgghHk8SRxGLiE9h9f7LXIpIMJrfuX55BkghnxCiBJBvqSK050IE3x+8kqOQ77UOtagnhXxCiBJCEkcR8nJ1MCQNtQqeb1KZ3s2ryLOMUiw1NRmtNp6sLJ2lQyEqSo1er7d0GBZnLe1gY2OLq6s7Tk4uT3zfkjiKUOMqnrSp5UNEfIoU8lmB1NRkkpLu4e6uwc7O3uI9P9vaqtHpSv8XZl6soR0URSEzM4P4+OwhLZ508pDEYSbnbt0DoGElD6P5w9vVxN7WRmoyrIBWG4+7uwZ7ewdLhyKsjEqlwt7eAXd3DQkJMZI4irvk9OwR+fZfisTTxZ7ZL7fA2f5+MzvZS5Nbi6wsHXZ29nmvKISZ2NnZm+U2qRQAPiGKohByNZr/rT3O/kuRAMQlZ7Dh+A0LRyYsydK3p4R1M9fnT37+PgH3ktP55kDOQr6nqnvzfONKFopKCCHMQxJHIegVhX0X7/LTkTCjQj53Z3uGtavJU1LIJ0qIgIAWODg4oFZn34RQFAVvbw2DBw8jMPBFw3q3b99i1aplHDt2lPT0dHx9fenRoxf9+7+Cjc39twNjYqJZvXoFR44cJDk5GR8fH3r37kffvi/nK66ZM6ewe/dOfvllK97eGsP833/fyq+/rmf16u+N1j958jhTpkxg27bdhnlHjhzip5++5/LlUBRFoX79+owc+TZ169bPVyy5SUxMZNasGZw8eQwXF1dGjBhp1F4P2r07mNWrlxMTE0O9eg0YN+59KleuAsD169eYN28WV66E4uLiyssvD6R//4EAJCUlsWjRPI4ePYyi6GnZsjXvvTeeMmXKEBy8nfDwm7z22puFPpf8kFtVBRQRn8InW87y1R+XjZJGp3q+zHm5hSQNUeKsXPktO3ceYOfOA+zYsZ8RI95g7txPuH79GgA3blxn5MhheHtr+OabNQQH72fSpA/ZsyeYyZMnGPYTHR3FiBGDKVOmDF9/vYYdO/bxwQfTWLPme776aoXJ8SQmJnLo0EE6dXqGTZt+LdA5bdmykU8++ZD+/QeyZcsONm3aztNPt2bMmLcIC7taoH0+aO7cj3B2dmLLlmA++mgOX375OVeuXM6x3rlzf/Hxx9N5++33+P333XTp0pWxY98hPT27Q9MZM6YQENCe7dv3Mm/eYr76agWnT58EYPHi+aSkpLB27QbWrt2EVqtl4cJPAeja9VmOHj3MtWthhT6X/JDEUQAZOj0fbT5jVP1drowjE3s24rUOtXGR6m9RwtnY2NCt23M4O7tw7Vr2F+zixZ/xzDPdeOed9/D21qBSqahbtx7z5i3m9OmT/PHHPgBWrlxKo0aNeeutd3F3d0elUtGgQUPef38KsbExJscQFLSNJk2a0qfPS2zZspHMzMx8nUNqaipLlixkwoTJtG3bDltbWxwcHBg0aAi9e/fjxo1rObY5c+YUXbu2y/Hf4MH9c6ybkpLCgQP7GTHiTRwcHKhfvyFdu3Zn69aNOdb944+9tGvXgYCA9tja2tKrV28cHR05diwEgPDwm+h0OkN9iVptY3ixQq/PYvjw13FxccXV1ZWePXvz119ngexnGD16BPLDD1/nq20KS77hCsDeVk2fp/z4+o8rqFXQo3Fl+rSQQj7xeL+fucWG4zdIy8zKe+VCcrSzoU8LP3oU8BlbZmYmGzf+jE6XSYMG/qSnp3PiRAhDhy7NsW7Zsu60bduOAwf20b59R44ePcw77/xfjvWeeqolTz3V0uQYtm7dyJtvvoO/f2M8PDzYu3cX3bo9Z/L2f/11hqwsHS1bts6x7K233s11m8aNm7Jz5wGT9n/r1k1sbGypWPF+G1ep4sf+/XtzrKvX63FwcDSap1KpuHXrJgBDh45g5cqlrFy5lKysLIYPH0mDBg0BmDJlptF2Bw/+Qc2atQzTHTp0YdGi+fz3vx/g7OxsUuyFJVccJtDrc3Zx3qleeZ5pUJ4ZfZsxoFU1SRoiT7+fuVUkSQMgLTOL38/cytc2o0a9RvfuHejcuQ3du3fg1KkTLFq0DB+fciQlJaLT6fD09Mp1Wy8vb2Jjs18OSUxMwMOjcF3o/PXXGbRaLa1bBwDwwgt9+fXX9fnaR0JCPG5uZbC1Nc/v49TUNBwcjGt0HBwcSUtLy7FuQEB79u3bw6lTJ9DpdGzbtoWbN2+QkZEBgFqt4r33xrFz5wGWLfuKjRt/5vDhgzn289NPP7B37y5GjRptmOfh4YGXlzfnz599wmf4aHLFkYfzt+/xzYErvN2lLtU0bob5apWKV9vVesyWQhjr0bhSkV5x5PdqY9my1VSvXpM7d24zceJ4ypZ1p379BgCUKVMWGxsboqIiDQ90H3T3bgReXtlJxdPTi7i4uBzrZGVlkZyspUyZsnnGsmXLRhIS4undu8c/2+pISEjg0qWL1K1bDzs7O7KycrZjVlaW4RaPl5c3iYnZCe/h5JGYmIizs3OO+WfOnGbChJxXS+XK+fLtt2uN5jk6OpKenm40Lz09DSennL/6mzRpxnvv/Ze5cz8mKSmJ7t170LJla1xd3bh06QK//rqeDRu2AdCwYSN69uzNb79tonXrtobz+vzzz9izZxcLFy7Fz6+q0f69vLyJiorKcVxzkcTxCNmFfNfYf+kuAKv2hfJhn6bY2shFmiiYHo0rFfjWUVGqUKEis2fP59VXB1K+fAWGDXsNe3t7WrVqw/btv9G8+VNG69+7F8fRo4f54IMpALRs2Zr9+/fSvXsPo/UOH/6TDz+czObNQTg7P7qSWavVsnfvLhYu/NLoNtCiRfP59dd1TJo0HY3Gh6ioSBRFMapVuHPnNhqNDwANGvhjZ2fLkSMHCQjoYHSM2bNn4uzszOTJHxrNb9y4CUFB+0xqp8qVK5OVpePu3bv4+voCcPPmDapVq5Zj3YSEePz9G/PTTxuA7ETQr19Phg4dQWTk3RzPb2xtbQ1JLT09nSlTJhAZGcmKFd/g61s+x/6zsrIMb8QVBfkWzEVIWDT/W3fCkDQAopPSuX0vxYJRCVF0fH3LM2bMf/j665WGt4Teffc/HDr0J0uWLCQmJgadTseFC+cYN+49GjVqQvv2nQB49dXXOXPmJMuXf0FiYgJZWVmcPHmcTz+dxcCBQx+bNAB27NhGxYqVadSoCV5e3ob/AgNfYPfuYOLj46lfvyHOzs4sXfo5KSkp6PV6Ll26wNq1Pxiegzg4OPDmm6OZO/cTDh36E51OR0pKMqtXr+D48RAGDhxSqDZydnYhIKA9y5cvIS0tjYsXz7Nz5w66ds35HOb69WuMHv0GERF3SE9PY9WqZbi7u1O/fkP8/RuTmZnBN9+sIisriytXLrNlywY6d+4GwKeffsK9e3EsXboq16QBEBsbg49PuUKdT36olFIyRmlsrJaBX+43TP8wqn2+93EvOXtEvuPXchbyDW1bAw+XktPnkEbjRnR0kqXDsDhLtsPduzfw9fWzyLFz87jO/QICWvDdd2upXr2m0fyxY98hMTGRFSu+wcbGhoiIO6xatYzjx4+SkpKCr295nnsukP79Bxrd9gkPv8mKFV9y+vRJ0tLS8PX1pU+f/vTu3S/POIcNe4VnnunOkCGvGs3X6/X07RtInz4vMWTIcMLDb/Lll4s4e/Y06enpaDQ+vPhiX/r3H2h0FRIcHMTPP68hPDwctVpNgwYNef31t6hTp24+Wi93iYkJfPrpLI4fD8HJyYkRI94gMPAFAL777ivOnDnN/PmLAViz5nvWr19DWloqjRs3Zdy4DwxXR3/9dYYvvljEtWtXcXf34JVXBvPii/2IiYnmxRefw97e3qhOpmxZd375ZSuQ/frzK6/0Ydu2XTkewEPun0O1WoWXV8E7WZXEQeks5JPEkU0Sx33W0CusKUpbO/z66zouXDjPlCkzcl1ujsRh9c84IhNSWbkvNMeIfJ3q+TKgVXWpyRBCFFuKovDbb5uZPv2TIj2u1X8rpuuyuByZaJguV8aR1zrUpn5FGZFPCHP4+ee1rFjxxSOXf//9z4aHzeLxgoK20bZt+xxvWZmb1SeOKl6uBDapxNZT4VLIJ0QReOmlAbz00gBLh1EqPPdcoEWOa1WJIz0zixsxWmqXN36P/IVmfjxdXSMj8gkhhAmsJnGcv32P1fsvk5iayez+zfF2u//2gb2tWpKGMAMViqJHpZK33oVlKIoeePJjcpT6T3RyeiYr94Uya+tfRCWmkZaZxdd/XKaUvEwmijF7e0fi42PQ6TLl8yaKlKIo6HSZxMfHYG+f8xXdwirVVxwhYdF8++dVElIyDPOc7W1oUb3kvV4rSh4PDw1abQJxcZHo9UXTR9XjqNVqQ++r1sxa2kGttsHJyRVX17y7eMmvUps4Fu44n7OQr5o3QwNKViGfKLlUKhVubu64uRWPN/SktiebtEPhFcmtqq1bt9KjRw+6devGjz/+mGP5xYsX6dOnD927d2fSpEnodIUfXP3BpFHW2Z73utXnve71JWkIIUQhmT1xREZGsmDBAtasWcOmTZtYt24dV65cMVpn/PjxTJ06lR07dqAoCuvX56/75MfpWNeXuS+34Cm5PSWEEE+E2RPHoUOHaNWqFe7u7jg7O9O9e3eCgoIMy2/fvk1aWhpNmjQBoE+fPkbLC8rnnxH5Xu8oI/IJIcSTZPZv1KioKDSa+4PM+/j4cPbs2Ucu12g0REZG5vs4Xl6uONnbkJqRha1axaq3O+JgZ92FfJoHxg+xZtIO90lbZJN2KByzJw69Xm/UU+XD/efntdxU9+4lM7htTfZfiqBD3fJoE1PRFi70Es3Ly5XYWGtugWzSDvdJW2STdsju5NDD4/Hd2z+O2ROHr68vx48fN0xHR0fj4+NjtDw6OtowHRMTY7TcVB4eLvQLqEm/gJp5r2wlCtP7ZWki7XCftEU2aYfCMfszjjZt2nD48GHi4uJITU0lODiY9u3vd3lesWJFHBwcOHHiBACbN282Wi6EEKJ4KZLxOLZu3cry5cvJzMykX79+jBw5kpEjRzJmzBj8/f25dOkSkydPRqvV0qBBA2bNmoW9vb25wxJCCFEApWYgJyGEEEWj1PdVJYQQ4smSxCGEECJfJHEIIYTIF0kcQggh8kUShxBCiHyRxCGEECJfSmzisERX7cVRXu2wa9cuXnjhBXr16sXbb79NQkKCBaI0v7za4V/79u2jc+fORRhZ0curLcLCwhgyZAi9evXitddes9rPxPnz5+nbty+9evXizTffJDEx0QJRFg2tVktgYCC3bt3KsaxA35VKCXT37l2lU6dOyr1795Tk5GSlZ8+eyuXLl43Wef7555VTp04piqIoH3zwgfLjjz9aIlSzyqsdkpKSlLZt2yp3795VFEVRFi5cqMycOdNS4ZqNKZ8HRVGU6Oho5dlnn1U6depkgSiLRl5todfrlW7duin79+9XFEVRPv30U2Xu3LmWCtdsTPlMvPLKK8q+ffsURVGUWbNmKZ999pklQjW706dPK4GBgUqDBg2U8PDwHMsL8l1ZIq84LNVVe3GTVztkZmYybdo0ypUrB0CdOnWIiIiwVLhmk1c7/Gvy5MmMHj3aAhEWnbza4vz58zg7Oxu69Rk1ahSDBg2yVLhmY8pnQq/Xk5ycDEBqaiqOjk9+bO7iYP369UybNi3XPgAL+l1ZIhNHbl21P9gV+5Pqqr24y6sdPDw86Nq1KwBpaWmsWLGCZ555psjjNLe82gHgu+++o379+jRu3LiowytSebXFzZs38fb2ZuLEifTu3Ztp06bh7OxsiVDNypTPxPvvv8/kyZMJCAjg0KFDDBgwoKjDLBIff/wxLVq0yHVZQb8rS2TiKKqu2os7U88zKSmJN954g7p169K7d++iDLFI5NUOoaGhBAcH8/bbb1sivCKVV1vodDpCQkJ45ZVX2LhxI5UrV2b27NmWCNWs8mqHtLQ0Jk2axDfffMOff/7JwIEDmTBhgiVCtaiCfleWyMTxcFfs5uqqvbjLqx0g+xfFwIEDqVOnDh9//HFRh1gk8mqHoKAgoqOj6du3L2+88YahTUqjvNpCo9Hg5+eHv78/AIGBgUYDq5UWebVDaGgoDg4ONGrUCICXX36ZkJCQIo/T0gr6XVkiE4d01Z4tr3bIyspi1KhRPPfcc0yaNKlUXnVB3u0wZswYduzYwebNm1mxYgU+Pj6sWbPGghGbT15t0bRpU+Li4rh06RIAe/bsoUGDBpYK12zyagc/Pz/u3r1LWFgYALt37zYkU2tS4O/KJ/fsvmht2bJFef7555Vu3bopK1asUBRFUV5//XXl7NmziqIoysWLF5W+ffsq3bt3V/7zn/8o6enplgzXbB7XDsHBwUqdOnWUXr16Gf6bOHGihSM2j7w+D/8KDw8v1W9VKUrebXH69Gmlb9++So8ePZQRI0YoMTExlgzXbPJqh3379ik9e/ZUAgMDlWHDhik3b960ZLhm16lTJ8NbVYX9rpRu1YUQQuRLibxVJYQQwnIkcQghhMgXSRxCCCHyRRKHEEKIfJHEIYQQIl8kcQhRxFJTU4mJibF0GAbh4eGWDkGUMJI4RLG0evVqmjdvTtu2bcnMzHzkekePHqVly5ZFGFnBNG3alKtXrwIwaNAg/vrrLwC2bNli0U4GL1y4wCuvvGKx44uSydbSAQiRm3Xr1vHBBx/Qr18/S4fyRJw6dcrwd3x8vOHvXr160atXL0uEBGT3Y/a4xCxEbuSKQxQ73bt35+bNm8yYMYMZM2aQlpbG9OnT6dq1K02aNKFbt27s2rUrx3YZGRl88MEHtGzZkoCAAMaMGcO9e/eA7E7tPvroI9q1a0dAQABz5swhIyMj1+N37tyZJUuW0LFjR5o3b86UKVNIT08HIDk5mQ8//JC2bdvStm1bJk2aRFJSEgB37txh6NChtGjRgmeeeYa5c+fyb31tnTp1CA0N5Z133uHOnTu89957fPfdd2zYsIE+ffqg1Wpp3Lgxly9fNsTxyy+/0L9/f8O+R40aRcuWLenWrRu//vprrrHfqGaCQAAABfdJREFUunWL5s2b8/7779OiRQs2b95MeHg4o0aNokOHDjRq1IgBAwZw9epVYmNjGTlyJPHx8TRt2pR79+7lq52E9ZLEIYqdHTt2UKFCBRYtWsTUqVNZvXo1V69eZcOGDZw4cYI+ffowc+bMHNtt3ryZq1evsnfvXnbu3ElKSgrfffcdAHPmzCEsLIwtW7awZcsWzp07x7Jlyx4Zw7Zt2/jxxx/ZsWMHFy5c4PPPPwdg6tSphIWFsXXrVn7//XdiYmKYOnUqAAsWLKB27dqEhITwww8/sG3bNg4fPmy03y+++MJwbkOHDjXMd3V1pXPnzmzbts0w77fffuOFF14w9DlWq1YtDhw4wOLFi1mwYAFHjhzJNXatVkvFihU5dOgQ3bp1Y/LkyVSvXp3du3dz5MgRPDw8WLZsGV5eXqxcuRJ3d3dOnTqFh4dHvttJWCdJHKLYGzRoEIsXL8bZ2ZmIiAhcXFxyHTPAzc2NGzdusHHjRu7du8eKFSt47733UBSFDRs2MG7cODw8PPD09OTdd99l/fr1jzzmW2+9RcWKFfH29uatt95i27ZtpKWlsWPHDsaPH4+npydly5ZlwoQJbN++ndTUVNzc3Dh27Bg7duzA2dmZvXv30qZNG5PP88UXX2T79u1Adm+uJ0+e5LnnnuOvv/4iIiKCsWPHYm9vT926dRkwYAA///zzI/fVs2dP7O3tcXJyYvbs2YwZM4asrCzu3LmDu7t7ru1XkHYS1kmecYhiLykpiQ8//JCzZ89SuXJlKleuTG5drD377LPExcWxYcMGPv74Y2rXrs2MGTOoWLEiaWlpDBkyxNBDsKIoZGZmkp6ejoODQ459+fn5Gf4uV64c0dHRJCYmkpmZSYUKFQzLKlasiKIoREVFMX78eBYvXsxnn33Gf//7X9q3b89HH32Et7e3SecZEBCAVqvl3LlzHD9+nLZt2+Lp6cmRI0fQarU8/fTThnWzsrIe26vtg8cMCwvj008/JTIykpo1a6JSqXJtv7i4uHy3k7BOkjhEsTdt2jRq1KjBsmXLsLW15dixY4Zf5g+6fv06rVq1YuDAgdy7d48vvviC//3vf2zbtg07Ozs2bdpE5cqVAUhJSSEmJuaRX4ZRUVGGv+/cuUP58uXx9vbG3t6e27dv4+npCWQ/U1Cr1Xh4ePD3338zcuRIJkyYwM2bN5k0aRKLFy9mxowZJp2njY0Nzz//PEFBQRw/fpxXX30VyB69rly5cuzbt8+wbkxMTK5f/v/694s/IyOD0aNHM2vWLJ599lkAlixZwtGjR3Ns4+7unu92EtZJblWJYk+r1eLo6IiNjQ0REREsWrQIIMfbQLt37+a///0vMTExlC1bFhcXF9zd3bGxsaFnz57MmzePxMREUlJSmDp1Ku+///4jj7lixQpiY2OJiopi2bJlvPjii6jVanr16sW8efOIi4sjISGBuXPn0qFDB8qUKcPSpUuZN28e6enpeHl5YWNjg4eHR45929nZodVqcz3uiy++yNatWwkLC6Nz584ANG7cGEdHR1atWkVmZiZ3795l+PDh/Pjjj3m23b9XC05OTv/f3h2qKBBFYRz/pwkKwhSzQVAMBieMYWBUBIOMoEXDYDX4BKIDThIsNoNP4GOYDeIrTDWbBsOGZYdll2W5W1bw+9XL5R5O+TgXLheA6/XK8XjMemdZFmmakqbpn/okr0nBIU9vsVhwOp1oNBqEYYjv++RyuexdxIfpdEq9XicIAhzH4XK5sNlsAFgul9i2Tb/fx/d97vc7u93uxzOr1Srj8ZggCGg2m8xms6yWUqnEYDCg2+1i2zbb7RaA9XrN7XbD8zxarRbFYjHb99lwOCSKIvb7/be1Wq1GoVCg1+thWRbwHjSHw4Hz+YzneYxGI1zXZT6f/9q7fD5PHMesViscxyGOYyaTCUmS8Hg8qFQqlMtlXNclSRLjPslr0n8cIl90Oh2iKKLdbv93KSJPSROHiIgYUXCIiIgRXVWJiIgRTRwiImJEwSEiIkYUHCIiYkTBISIiRhQcIiJi5A0g8Lna6D2nygAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a23be3a58>"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAD7CAYAAADO1FugAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deViU9f7/8SfCAAmYYChollsR5ik1N7TEyBRFQOGApiFWHs3IOvVVQkwDl9zyawcsPaRHykhwQckF6egvrVxQrATTcuFYgAug4IKyzv37g8N8JVnlHpjB9+O6vC4ZZu55zX3O1dv7nvvzuk0URVEQQgghDEiLpg4ghBBC/JkMJyGEEAZHhpMQQgiDI8NJCCGEwZHhJIQQwuDIcBJCCGFwZDgJIYQwOGZNHaC5yMsrQKs17CVjbdpYc+XKzaaOUSvJqS7JqS7JqY4WLUywtbWq9vf3PJwCAgJYv349ALNmzeLNN9+kQ4cOZGdn8/7775OdnY2lpSUfffQRDz/88L2+TZ0dOHCAqKgoPv/88yp/7+bmxhdffKG3LFqtYvDDCTCKjCA51SY51SU59e+eh9ORI0d0f09OTiYoKAiA4OBghg8fzksvvcSGDRv46KOP+PjjjxuetBparZbo6Gj++c9/8vjjj+vtfWrTpo11k713fdjb2zR1hDqRnOqSnOqSnOUKi0q5cf22XrZd63AqLS0lLCyMM2fOkJubi5OTE3Z2dgD4+fnx4osvkp2dzZQpU/jiiy/49ddfWbduHQC+vr64uLjUuP2QkBAsLCxIS0ujoKCAadOmMXr0aCIjI/n555+5ePEiL7/8MoMGDSIsLIz8/HwsLS2ZM2cO3bt359y5c5w7d4758+frjuQA8vPzmTlzJpcuXaJr164UFRUBcPPmTUJDQ7l8+TLZ2dm4uLiwcOFCgoOD6du3L/7+/kD5keGMGTN4+umn67QjX1vwDdl5+vkfSQghDNH25d7c0NO2ax1OP/30ExqNhri4OLRaLYGBgXh7e7Np0yY2bdoEQGxsLFFRUWRlZdG+fXsWL15MSkoK9vb2zJkzp9YQGRkZxMXFceXKFXx8fBg0aBAAxcXF7Nq1C4Bx48Yxd+5cunfvztmzZwkKCiIpKYnHHnuMhQsXkpycXGmbERERdO/enc8++4yjR4+SmJgIwL59+3B2diYiIoLi4mI8PDz45Zdf8PX1JTIyEn9/f7Kysrh69WqdB5MQQtyv9HV0Vutw6tu3L61btyYmJob09HTOnz/PrVu3qnxuaWkpJ0+eZPr06cyaNYtNmzYREhJS6YimKj4+Pmg0GhwcHOjduzfHjh0D4KmnngKgoKCAEydOMGvWLN1rbt26RV5eHra2tlVu88iRIyxfvlz3GTp27AjAqFGjSE1NJTo6mvT0dPLz87l16xb9+/dnzpw5ZGZmkpCQgLe3d227Rggh7ns5Ofd27NSihUmNX4fUOpz27t1LREQEEydOxMfHh7y8PKorMre3t8fKyornn38eKB8ECxYsqDWkqamp7u9arRYzs/JYlpaWusfMzc1JSEjQPe/SpUu0bt262m2amJhUylnxHuvXrycpKQl/f38GDhzI6dOnURQFExMTRo8ezc6dO0lMTGTt2rW15hZCCKEftQ6nQ4cOMWLECHx9fcnIyCA5ORkXFxdMTU0pLS3FzMwMU1NTysrKePTRR3FwcGD//v24urry7bff8uSTT9YaIjExEXd3dy5cuEBqaioLFy7k1KlTut/b2NjQqVMn3RHNgQMHmDt3Lnv27Kl2my4uLiQkJPDEE0+QmprKH3/8AZRf1Td27Fg8PT1JS0vj119/RavVAuVHcOPHj6dbt260a9eu1tx3Wvv+sHo9XwghjF1hUanetl3rcPLz82PGjBns3LkTjUZD7969yczM5IUXXsDb25v4+HiGDBnClClTWLNmDZGRkXzwwQcsW7YMa2trFi9eXGuIwsJCfH19KS4uZt68eVWeqlu2bBlhYWGsWbMGjUbDihUrMDExqXabb731FiEhIXh4eNClSxfdab3AwEDCwsKIiorC2tqaXr16kZmZCYCjoyOOjo6MGTOm1sx/duXKTYO/bNPe3uaeD8Ebk+RUl+RUl+RsHCZNfbPBkJAQ+vXrh4+PT1PGQFEUsrOzCQgIYMeOHZibm9fr9TKc1CM51SU51SU51dHg75zUsGTJEg4ePHjX4z169GiMt6+TpKQkwsLCCAsLq/dgEkIIoa5GGU7vvfdeY7xNg7i7u+Pu7t7UMYQQQmAAp/UaouKUYGZmJvHx8QQGBvLKK680aJsnT57E39+fEydOqJRSCCH026ZQFTmtZwASEhJYt24dnTt3btB2bt++zfz58ykpKan3a6UhQghRE322KTRHRnXLDEVRWLRoEcOHDycgIIA//viDDz74gMuXLxMUFMTrr7/OV199BUBcXBwjRowAoKSkBFdX11qHzuLFiwkMDNT75xBCCFEzozpySkpK4uTJk+zYsYMbN27g5eVFeHg4K1euJCoqivT0dDZv3sz48eM5fPgw165dIzc3l7Nnz9KrVy80Gk212967dy+FhYXyvZMQQm8auzDWWApqq2JUw+nIkSMMGzYMjUaDnZ0dgwcPrvT7igqisrIy0tPTGTlyJEePHiUtLY0hQ4ZUu92cnBxWrVpFdHS0fj+AEOK+1pjfARn7d05GdVrvz5VEFTVHFSwsLHB2dmb79u106dKF/v37c/ToUQ4cOHDXILvTvn37yM/PZ8KECbpOPW9vb27eNNwbdQkhRHNmVEdOLi4urF27lnHjxnH79m2+//57evbsWek5rq6ufPLJJ7z66qv069ePBQsW0L59e91tPqri5+eHn5+f7mcnJ6dKPX51IfVFQoia6LPqpzkyquE0dOhQ0tLSGDVqFA899BBdu3a96zlDhgwhLCyMfv368eCDD9KmTZsaT+mpRRoi1CM51SU51WUsOY2dUa9zMiQynNQjOdUlOdUlOdVxX6xzqquAgACuX79+1+Pjxo3jpZdeaoJEQgghqmKUw2njxo1ERkYyatSoelUjVdz08ObNm4wbN47Vq1fz8MMPA3Dw4EEWLVpEUVERI0aM4J133tFLdiGEELUzyuG0Y8cOFi1axLPPPlvv1x4/fpz333+f8+fP6x4rLCwkNDSU9evX4+joyNSpU3X3pKqrmg5PDYmxrHuQnOqSnOqyt7dp9Dqi+43BDafk5GRWr16NRqMhMzMTNzc3WrZsqbuxoJubG2lpaYSHhzN79mxWr15NbGwsAPHx8Rw/fpzw8PBqt79x40Y++OADgoODdY+lpqby6KOP6u755Onpye7du+s1nKS+SIj7i9QR6ZdBrnOqGDBbtmwhJiYGOzs74uPjcXJywsbGhh49erBgwQJcXV3JycnR3eV227Zttd4XauHChfTp06fSY9nZ2djb2+t+btu2LZcvX1b/gwkhhKgTgztyAnj88cdxdHQEwNbWFhcXFwDat29f6YIGExMTxowZw9dff42Pjw9Xrlzh6aefrvf7abXaSnfVVRSlxrvsCiEEGP5pSEPPVxODHE5/7sAzNTWt9rljxoxh8uTJmJub69od6svBwYGcnBzdzzk5ObRt2/aetiWEuH8Y8qXaxn4puUGe1quPDh064ODgQGxs7D0Pp6effpr//Oc//P7775SVlbFjx44a646EEELol0EeOdXXyJEj+eabb2jXrt09vd7CwoLFixczffp0ioqKcHV1rXc7udQXCXF/kToi/TL6hojS0lKCg4Nxd3dn2LCmGxDSEKEeyakuyakuyamOZt0QoSgKzz33HAMHDmTo0KEApKSkMH/+/CqfHxUVdc9HV0IIIRqPUQ8nExMTDh06VOmxPn361LtRXAghhGEx+gsi7rRx40aee+45lixZUu/Xrly5Eg8PDzw8PFi6dKke0gkhhKgroz5y+rN7rTU6ePAgP/zwA1u3bsXExITJkyfz73//mxdffLHO25D6InVJTnVJTvUUl5Q1dYT7gtEMJ33WGtnb2xMSEoK5uTkAXbt25cKFC/XKJ/VFQtwfti+/tyUron6M6rSevmqNHnvsMd0ddc+fP09iYmK9evWEEEKoy2iOnED/tUZnzpxh6tSpBAcH06lTJ718BiGE8TOG049gPDmrYlTDSZ+1RseOHeOtt94iNDQUDw+PBmcVQjRfhrx+qIKsczJQd9YabdiwocbnXrx4kaCgIFasWKE7GhNCCNF0mu1wgrrXGq1du5aioiIWL16se6y+t26X+iIh7g9ytV7jMPr6ouo0dq2R1BepR3KqS3KqS3Kq4748rSe1RkIIYdya5XCSWiMhhDBu97zOKSAgQPf3WbNmkZWVVen3H3/8MZGRkfeerI6ys7OZNGkSXl5e+Pn5cerUqSqf5+bmRmZmpt7zCCGEaLh7PnI6cuSI7u/JyckEBQUBcOPGDRYtWsTOnTuZPHlywxPWYsWKFQwfPpyXXnqJ7777jvDwcF0zRGOS+iJ1SU516TtnYVEpN65LQ4pQT63DqbS0lLCwMM6cOUNubi5OTk7Y2dkB4Ofnx4svvkh2djZTpkwhJiaG/fv306lTJ1555ZU6BQgJCcHCwoK0tDQKCgqYNm0ao0ePJjIykp9//pmLFy/y8ssvM2jQIMLCwsjPz8fS0pI5c+bQvXt3Fi5cqNtWZmYmrVq1AiA/P5+ZM2dy6dIlunbtSlFREQA3b94kNDSUy5cvk52djYuLCwsXLiQ4OJi+ffvi7+8PlB8Zzpgxo06Ld0Hqi8T9bftybwz3q3dhjGodTj/99BMajYa4uDi0Wi2BgYF4e3uzadMmNm3aBEBsbCxRUVHY2toyevRogHqd0svIyCAuLo4rV67g4+PDoEGDACguLmbXrl1A+aXdc+fOpXv37pw9e5agoCCSkpJo0aL8zKS7uztZWVl8+umnAERERNC9e3c+++wzjh49SmJiIgD79u3D2dmZiIgIiouL8fDw4JdffsHX15fIyEj8/f3Jysri6tWrdR5MQggh1FXrcOrbty+tW7cmJiaG9PR0zp8/z61bt1QN4ePjg0ajwcHBgd69e3Ps2DEAnnrqKQAKCgo4ceIEs2bN0r3m1q1b5OXlYWtrC8Du3bs5deoUr776KomJiRw5coTly5frPkPHjh0BGDVqFKmpqURHR5Oenk5+fj63bt2if//+zJkzh8zMTBISEurUKiGE+D9qnDqU06TqMpacVal1OO3du5eIiAgmTpyIj48PeXl5qL006s4aIq1Wi5lZeSxLS0vdY+bm5pWutrt06RKtW7dm37599O3bFysrK5ydnWnfvj0ZGRmYmJhUylnxHuvXrycpKQl/f38GDhzI6dOnURQFExMTRo8ezc6dO0lMTGTt2rWqfkYhmruGrqkx9HU5FSSnOmpb51Tr1XqHDh1ixIgR+Pr60qpVK5KTkykrK8PU1JTS0lKg/D/8ZWX3vmo6MTERRVHIysoiNTWVZ555ptLvbWxs6NSpk244HThwgAkTJgCwdetWNm7cCMDZs2fJzc2lS5cuuLi46J6fmpqqayg/cOAAY8eOxcvLi6KiIn799Ve0Wi1QfgQXGxuLo6OjrHsSQogmVOuRk5+fHzNmzGDnzp1oNBp69+5NZmYmL7zwAt7e3sTHxzNkyBCmTJnCmjVrdKfP6qOwsBBfX1+Ki4uZN2+e7lTdnZYtW0ZYWBhr1qxBo9GwYsUKTExMCA0NJTQ0lK1bt2JhYcHy5cuxsrLirbfeIiQkBA8PD7p06aLLFRgYSFhYGFFRUVhbW9OrVy/dJeaOjo44OjoyZsyYen8GqS8S97PCotKmjiCamSavLwoJCaFfv3413m+pMSiKQnZ2NgEBAezYsUN348G6kvoi9UhOdUlOdUlOdRhEfdGSJUs4ePDgXY/36NGjMd6+TpKSkggLCyMsLKzeg0kIIYS6GmU4vffee43xNg3i7u6Ou7t7U8cQQgiBkd2mXQghxP3BqItfK76vyszMJD4+nsDAwDo3U/xZdnY277//PtnZ2VhaWvLRRx/x8MMP1/n1Ul+kLsmpLn3mlOoioQ9GPZwqJCQksG7dOjp37nzP2wgODtZ19G3YsIGPPvqIjz/+uM6vl/oicb+S6iKhD0Y1nBRFYfHixezbt4+2bdtSVlbGzp07URSFoKAgHnnkEQYPHsz48eOJi4sjOjqaxMRESkpKGDp0KHv27EGj0dy13atXr/Lrr7+ybt06AHx9feV27UII0YSMajglJSVx8uRJduzYwY0bN/Dy8iI8PJyVK1cSFRVFeno6mzdvZvz48Rw+fJhr166Rm5vL2bNn6dWrV5WDCcq7/dq3b8/ixYtJSUnB3t6eOXPmNPKnE8J4qXXaUE6TqstYclbFqIbTkSNHGDZsGBqNBjs7OwYPHlzp9xX9eGVlZaSnpzNy5EiOHj1KWloaQ4YMqXa7paWlnDx5kunTpzNr1iw2bdpESEgI69ev1/MnEqJ5UGM9jaGvy6kgOdVhEOuc1PLnvryKDr4KFhYWODs7s337drp06UL//v05dOgQx44dq/HeUvb29lhZWfH8888D5eWwCxYsqFc2aYgQ9ytphxD6YFTDycXFhbVr1zJu3Dhu377N999/T8+ePSs9x9XVlU8++YRXX32Vfv36sWDBAtq3b6+7B1VVHnnkERwcHNi/fz+urq58++23PPnkk/XKJg0R6pGc6jKWnELcyaiG09ChQ0lLS2PUqFE89NBDdO3a9a7nDBkyhLCwMPr168eDDz5ImzZtajylVyEyMpIPPviAZcuWYW1tzeLFi/XwCYQQQtRFk3frNRdy5KQeyakuyakuyamOZvWdU0MFBARw/fr1ux4fN24cL730UhMkEkIIUZVmNZw2btxIZGQko0aNqrLPr7qr7zZt2lTpzreZmZl4e3szd+5cvWUVQghRvWZ1Wm/ixIlMmTKFZ5999p63cebMGYKCgoiNja3xIgrR/NS3hsfQT5tUkJzqkpzqaDan9ZKTk1m9ejUajYbMzEzc3Nxo2bIle/bsAcDNzY20tDTCw8OZPXs2q1evJjY2FoD4+HiOHz9OeHh4re8TFhbGO++8U+/BJPVFxk9qeIQwHEbVSl4xYLZs2UJMTAx2dnbEx8fj5OSEjY0NPXr0YMGCBbi6upKTk6O7Nfu2bdvqdDPDgwcPUlhYyIgRI/T9UYQQQtTAaI6cAB5//HEcHR0BsLW11fXftW/fvtKFDiYmJowZM4avv/4aHx8frly5wtNPP13r9mNjY++51Vw0D/WtezGWehjJqS7JqX9GNZz+3I1nampa7XPHjBnD5MmTMTc3r3SxQ3WKi4s5evSorG+6z9XnHL2hn9OvIDnVJTnV0Wy+c6qvDh064ODgQGxsLBs2bKj1+b/99hudOnWiZcuW9/R+Ul9k/KSGRwjD0WyHE8DIkSP55ptvaNeuXa3PzcjIwMHB4Z7fSxbhqsdYcgoh9KdZXUp+p9LSUoKDg3F3d2fYMP0f1chwUo/kVJfkVJfkVMd9eVpPURSee+45Bg4cyNChQwFISUlh/vz5VT4/KiqqTkdXQgghGkezHE4mJiYcOnSo0mN9+vQhISGhiRIJIYSoD6Na5ySEEOL+0KAjp5CQEPr161enBa4VKq6cU6to9ZVXXmHChAm603dLliwhNjaW5ORkzM3NAXj22WeJjY3l4YcfBuDjjz/G1NSU6dOn6z7H4cOHefDBB3XbHTJkCO+8806dc9R07tSQGMu6BzVz1reWSAjR9Br9tJ7a7d8DBgzg2LFjuuF08OBBevbsybFjx3BxceH333+nZcuWPPzww9y4cYNFixaxc+fOu+6M+9Zbb9VryP6Z1BcZLqklEsL41Gs4KYrC4sWL2bdvH23btqWsrIx+/fqxbds2Pv/8c7RaLU8++SQffPABcXFx/P7778yZMweAxYsX4+DgwI0b5f+ZmD59Otu3b2fVqlWYmJjwl7/8hfnz51NcXMy8efM4c+YMZWVl/O1vf2PUqFHVZnJxceHDDz8E4PLly5ibmzN8+HB++OEHXFxcSElJYdCgQQDs3buXTp06SQvEfUhfR4z345GoPklOdRlLzqrUazglJSVx8uRJduzYwY0bN/Dy8uL27dvs3LmT2NhYLCwsWL58ue5W6qNHjyY0NJQWLVrwzTffEBcXpytjvXz5MosWLSI+Ph4HBwdmzpzJ/v37+fnnn3nyySdZsmQJN2/eZNy4cTz99NN07NixykxPPvkkf/zxB0VFRfzwww8MGjSIQYMG8eabbzJz5kxSUlJ44YUXABg9ejRQftfbP4uIiODzzz/X/RwTE4O1tXGcqhO108cltYZ+qW4FyakuyakOVS8lP3LkCMOGDUOj0WBnZ8fgwYNRFIXff/8df39/AEpKSujevTt2dnY88cQTJCcno9Fo6Ny5M/b29rpt/fTTT/Tu3Vu38HXZsmUAfPrppxQWFrJlyxYAbt26xZkzZ6odTqampjz99NOkpaXxww8/MGHCBDp27EhhYSHXrl3jp59+Yvbs2bV+toae1hNCCKGeeg0nExMT7lyza2ZmRllZGSNGjOD9998HoKCggLKyMgC8vb3ZtWsXGo0GT0/Pym9sZoaJiYnu56tXrwKg1WpZtmwZTz75JAC5ubmVLlSoyoABA/jxxx9JTU3VDTkXFxf27t2Lra1toxwBSX2R4ZJaIiGMT72Gk4uLi+6U3e3bt/n+++/p2rUr//73v5k2bRp2dnaEhYXxyCOPMH36dF544QUiIiJQFIUZM2ZU2tZf/vIXwsPDycnJwd7eng8//JD+/fszYMAANmzYwIIFC8jOzmb06NHExsbyyCOP1JjrnXfe4fHHH8fMrPwjDRo0iIiICIYPH34Pu6X+pCFCPcaSUwihP/UaTkOHDiUtLY1Ro0bx0EMP0bVrV2xsbHjzzTcJDAxEq9Xi7OzMlClTALC0tKR3794UFxdjZWVVaVvt2rVj9uzZvPbaa2i1Wnr27ImPjw+3b98mLCyMUaNGUVZWxsyZM2scTFB+K438/HzGjx+ve2zAgAH8/e9/Z+DAgfX5iEIIIQxAs+3Wa2xy5KQeyakuyakuyamOZtGtJ714QghxfzGK4SS9eEIIcX8xiuFUnYr6pMzMTOLj4wkMDLznBbYpKSl8+OGHlJSU0KFDB5YsWVLrVYJ3kvoidamVU6qLhDBORj2cKiQkJLBu3To6d+58z9uYNWsWq1atolu3bnz00UesXbuWd999t86vl/oiwyTVRUIYJ6MaTlXVJ+3cuRNFUQgKCuKRRx5h8ODBjB8/nri4OKKjo0lMTKSkpIShQ4eyZ88eNBpNlduuWI9VUlLC5cuXcXJyauRPJ/RFn0eL99uRqL5JTnUZS86qGNVwqqo+KTw8nJUrVxIVFUV6ejqbN29m/PjxHD58mGvXrpGbm8vZs2fp1atXtYMJQKPR8Ntvv/HKK69gZmZWr6MmYdj0dcWSoV8NVUFyqktyqqO2q/WM6n5OVdUn3al///4cP36csrIy0tPTGTlyJEePHuW7775jyJAhtW7fycmJgwcP8sYbb9TrdhlCCCHUZVRHTlXVJ93JwsICZ2dntm/fTpcuXejfvz+HDh3i2LFjd90i405FRUV8//33uttueHl5sWTJknplk/oiwyTVRUIYJ6MaTlXVJ/Xs2bPSc1xdXfnkk0949dVX6devHwsWLKB9+/bY2dlVu10zMzPCw8NxcHCgR48eJCYm0rt373plk0W46jGWnEII/TGq4VRVfdKfDRkyhLCwMPr168eDDz5ImzZtaj2lZ2pqyooVK5g7dy5lZWW0a9eOhQsX6ulTCCGEqI3UF6lEjpzUIznVJTnVJTnV0Szqi+pq48aNREZGMmrUKN577727fh8QEMD169fvenzcuHHcuHGDLVu2YG5uzsiRI5k2bVpjRBZCCFGFZjWcduzYwaJFi3j22Wer/P369eurfPzgwYMsWrSILVu28MADDxAUFMQ333zDsGF1v8hBGiLU1ZCc0gohhPEzmuGUnJzM6tWr0Wg0ZGZm4ubmRsuWLdmzZw8Abm5upKWlER4ezuzZs1m9erXulvDx8fEcP36c8PDwKrd98uRJnn32Wd1NCZ977jn27NlTr+EkDRGGQ1ohhDB+RrXOqWLAbNmyhZiYGOzs7IiPj8fJyQkbGxt69OjBggULcHV1JScnhz/++AOAbdu21XgL9ieffJIffviB/Px8ioqK+H//7/+Rm5vbWB9LCCHEnxjNkROU31TQ0dERAFtbW1xcXABo3759pe+STExMGDNmDF9//TU+Pj5cuXKFp59+utrturi44OPjQ0BAAK1bt8bFxYXjx4/r98MIvWqs05f3w2nSxiQ51WUsOatiVMPpz/VDpqam1T53zJgxTJ48GXNzc7y9vWvc7s2bNxk2bJiu0XzNmjV07Nix4YFFk2mMq5QM/WqoCpJTXZJTHc2qvqg+OnTogIODA7GxsbUOp8zMTN544w1KS0u5ceMGmzdvZsSIEY2UVAghxJ8Z1ZFTfY0cOZJvvvmm1jvlPvHEEwwbNgwvLy/KysqYNGkSzzzzTL3eS+qLDIdUFglh/JrtItzS0lKCg4Nxd3ev11V390oW4apHcqpLcqpLcqrjvlqEW0FRFJ577jkGDhyoK3NNSUlh/vz5VT4/Kiqq1qMrIYQQjadZDicTExMOHTpU6bE+ffqQkJDQRImEEELUxz1fEBEQEKD7+6xZs8jKyuLKlSt4e3vr/ri5udGrVy9VglYnOzubSZMm4eXlhZ+fH6dOnaryeW5ubmRmZuo1ixBCCHXc85HTkSNHdH9PTk4mKCiINm3a6I5OtFotgYGBer9p34oVKxg+fDgvvfQS3333HeHh4bpmiMYk9UUNJ7VDQogKtQ6n0tJSwsLCOHPmDLm5uTg5OenujeTn58eLL75IdnY2U6ZMISYmBltbWwBdT52np2eN2w8JCcHCwoK0tDQKCgqYNm0ao0ePJjIykp9//pmLFy/y8ssvM2jQIMLCwsjPz8fS0pI5c+bQvXv3Sre2yMzMpFWrVgDk5+czc+ZMLl26RNeuXSkqKgLK1zSFhoZy+fJlsrOzcXFxYeHChQQHB9O3b1/8/f2B8iPDGTNm1Lh4905SX9RwUjskhKhQ60Y2F+UAACAASURBVHD66aef0Gg0xMXF6Y6GvL292bRpE5s2bQIgNjaWqKgo3WAqKytj9erVfPrpp3UKkZGRQVxcHFeuXMHHx4dBgwYBUFxczK5du4Dy5vC5c+fSvXt3zp49S1BQEElJSbRoUX5m0t3dnaysLN17RkRE0L17dz777DOOHj1KYmIiAPv27cPZ2ZmIiAiKi4vx8PDgl19+wdfXl8jISPz9/cnKyuLq1at1HkxCCCHUVetw6tu3L61btyYmJob09HTOnz/PrVu3anzN999/T6dOnXBycqpTCB8fHzQaDQ4ODvTu3Ztjx44B8NRTTwFQUFDAiRMnmDVrlu41t27dIi8vTzcQd+/ezalTp3j11VdJTEzkyJEjLF++XPcZKhofRo0aRWpqKtHR0aSnp5Ofn8+tW7fo378/c+bMITMzk4SEhFoX7gr9qDjtaMinH+8kOdUlOdVlLDmrUutw2rt3LxEREUycOBEfHx/y8vKobWnUnj17GDlyZJ1D3FlDpNVqMTMrj2Vpaal7zNzcvNLVdpcuXaJ169bs27ePvn37YmVlhbOzM+3btycjIwMTE5NKOSveY/369SQlJeHv78/AgQM5ffo0iqJgYmLC6NGj2blzJ4mJiaxdu7bO+YV6cnJuGPz6jAqSU12SU12GnrPB9UWHDh1ixIgR+Pr60qpVK5KTkykrK8PU1JTS0vKV+KamppSVlele8/PPP9OnT586h0xMTERRFLKyskhNTb2rncHGxoZOnTrphtOBAweYMGECAFu3bmXjxo0AnD17ltzcXLp06YKLi4vu+ampqbqG8gMHDjB27Fi8vLwoKiri119/RavVAuVHcLGxsTg6Osq6JyGEaEK1Hjn5+fkxY8YMdu7ciUajoXfv3mRmZvLCCy/g7e1NfHw8Q4YMYcqUKbrC1IyMDBwcHOocorCwEF9fX4qLi5k3b57uVN2dli1bRlhYGGvWrEGj0bBixQpMTEwIDQ0lNDSUrVu3YmFhwfLly7GysuKtt94iJCQEDw8PunTpojutFxgYSFhYGFFRUVhbW9OrVy/dJeaOjo44OjoyZsyYOmevIPVFDSe1Q0KICk1eXxQSEkK/fv1qvN9SY1AUhezsbAICAtixYwfm5ub1er3UF6lHcqpLcqpLcqrDIOqLlixZwsGDB+96vEePHo3x9nWSlJREWFgYYWFh9R5MQggh1NUow+m9995rjLdpEHd3d9zd3Zs6hhBCCFSuLwI4duwYf/3rX/H29iYwMFD3uL5IfZEQQjQ/qtYXAcycOZNPP/2UJ554gs2bN7NgwQJWrVrV8KTVkPqi+mnougepGBJCNAZV64u++OIL3n77bZ544gkAnJyc+PLLL2vcvtQXGRepGBJCNAZV64seeughXbOCVqtl5cqVuvsp1UTqi4QQQtxJL/VFxcXFhISEUFpaytSpU2sNIfVFxqUxKlGMpXZFcqpLcqrLWHJWRfX6oopTc61bt2bVqlVoNJpaQ0h9kXHR99oJQ1+fUUFyqktyqsvQczZ6fdHMmTN59NFH+fjjj+u8Xkjqi4QQQtxJ1fqiFStWsHfvXrp166arAGrbti2fffZZje8h9UXGQyqGhBCNQeqL/kvqiwyH5FSX5FSX5FSH1BfVkdQXCSGE4ZD6ov+S+iIhhDAcqtcXVTh58mSjHBkVFBQwffp0PD09GT16dJVHaFCeNzk5We95hBBCNJzq9UUAt2/fZv78+ZSUlDQsXR2sW7eORx99lMjISM6dO0dgYCA//PCD3t/3z5pzfZFUFgkhGpuq9UUxMTHY2tqyePFiAgMD+fHHH2sNEBkZyYULFzh37hx5eXmMHTuWyZMnEx8fz9atW8nPz+f5559n4sSJzJ07l0uXLmFiYsL//M//MHDgQN58803dJe2ZmZk8+OCDQPlC4NmzZ3PixAk6dOhAXl5etZ/nf//3f1m9ejWKovDOO+8A5RdqDB48uM63m2/O9UVSWSSEaGyq1hfZ2tqyd+9eCgsL6/X9zYkTJ4iNjUWr1eLj44OLiwsAly9fZteuXZiZmfHOO+/g6+vLCy+8QHZ2NuPHj2fbtm1YW1tjZmbGa6+9xqFDh5g3bx5QvtgWytdQnT9/Hi8vr2o/z/79+/H19SUwMJC///3vFBYWcvjwYcLDw+u3N4UQQqhC1fqinJwcVq1aRXR0dL1CjBo1CisrK6D81haHDx/G1taW7t2769oiDh48SHp6OhEREUD5EVBGRgbOzs4ArF27lqysLMaNG0evXr04cuQIY8eOBaBTp0706tWrxs/TsWNHOnTowNGjR7lw4QKurq5YWFjU63M0Z41dg2IstSuSU12SU13GkrMqqtYX7du3j/z8fF17A4C3tzcxMTFYW1f/ncyf64sqfq6oL6p4/PPPP6d169ZA+X2c2rRpw5EjR+jUqRNt27alQ4cO9OrVizNnztxVX1Qx5Gr6PL6+vuzYsYMLFy4wffr02nbNfaUx10sY+vqMCpJTXZJTXYaes8HrnO6sL8rIyCA5ORkXFxddfZGZmZmuvsjPzw8/Pz/da52cnCr14VVnz549TJw4kdu3b/Ptt9+yevVqUlNTKz1nwIABfPXVV7zxxhucPXuWCRMmsHfvXvbt20dxcTHvv/8+2dnZnDhxgvfee4/Lly+zfft2nn/+eS5evKj7/qu6zwPll5N/8sknWFlZ1buRvDk3REgrhBCisalaX7RmzRpdTVB9WFhYMH78eG7evMnUqVPp1q3bXcPp/fffZ+7cuXh6egKwdOlSrK2teeONN5g9ezaenp6YmpoSGhpKhw4dGD9+PGfOnGHEiBF06NCBxx9/vMbPA+VHaj179tQ9tz6kIUIIIdTT5PVFkZGRAE1+Gk1RFAoKChg7dizR0dHY29vX6/UynNQjOdUlOdUlOdVhEPVF0dHRbN269a7H27Ztq7tnU1NLS0tj8uTJBAUF1XswCSGEUFejDKdJkyYxadKkxnire/bUU09VWlgshBCi6aheX5SZmcmECRPw9vYmICDgrlojtUl9kRBCND+q1xf94x//wMPDg/Hjx7N+/XpWrFjBRx991PCk1ZD6ovqpat2D1BMJIQyN6vVFWq2WmzdvAuUde3euVaqK1Bc1PaknEkIYGtXri95++23GjRvH+vXrKSkpIS4urtYQUl8khBDiTqrWF0H5vZvmzZvH0KFDSUpK4s033+Trr7/GxMSk2tdIfVHTM7SaE0PLUx3JqS7JqS5jyVkVVeuLrl69Snp6OkOHDgVg+PDhfPDBB+Tl5elOBVZF6ouaniGthzD09RkVJKe6JKe6DD1no9YX2draYmFhQUpKCn369OHYsWNYWVnVOJhA6ouamtQTCSEMjer1RStXrmT+/PkUFhZiZWWla4CoidQXNQ5D/5eUEEJUkPqi/5L6IsMhOdUlOdUlOdUh9UV1JPVFQghhOKS+6L+kvkgIIQxHs6gvevvtt/H09MTT05OdO3dWm1fqi4QQwjgYfX1RVFQU7du35x//+AdXrlzB29ub/v3789BDD+ntPatiTPVFUlckhDB0Rl9f1K9fPzp37gxAmzZtaN26Nbm5ubRq1Urqi6ohdUVCCENn9PVFgwYN0m1n165dFBcX061bNz7//HNA6ouEEMIYNZv6osTERD788EPWrFmDmZmZ1BfVwtBrTQw9XwXJqS7JqS5jyVkVo68vgvKS17Vr17J27VqcnJwApL6oFoa8/sHQ12dUkJzqkpzqMvSczb6+6PDhw0RHR7NhwwYcHR11z3dxcZH6ompIXZEQwtAZfX1RREQERUVFvP7667rnLliwQOqLqmDo/5ISQogKUl/0X1JfZDgkp7okp7okpzqkvqiOpL5ICCEMh9QX/ZfUFwkhhOFQtb6ooqGh4o+bm5vuEm59yc7O5rXXXsPb25sxY8Zw6NChKp/n5uam+25JCCGEYVO1vqhNmzYkJCQA6Ba4VjQu6MvSpUtxc3NjwoQJpKenExAQwHfffVfp8vTG0Nj1RVJBJIRozlSvL7K1tQVgy5YtPPDAA7qr66oTEhKChYUFaWlpFBQUMG3aNEaPHk1kZCQ///wzFy9e5OWXX2bQoEGEhYWRn5+PpaUlc+bMoXv37rz44osMGDAAgEcffZSioiJu3bpFWVkZM2fO5NKlS3Tt2pWioiIAbt68SWhoKJcvXyY7OxsXFxcWLlxIcHAwffv2xd/fHyg/MpwxY0adLylv7PoiqSASQjRnqtcXAZSVlbF69Wo+/fTTOoXIyMggLi6OK1eu4OPjo6skKi4uZteuXQCMGzeOuXPn0r17d86ePUtQUBBJSUkMHz5ct521a9fi7OyMjY0N8+bNo3v37nz22WccPXqUxMREAPbt24ezszMREREUFxfj4eHBL7/8gq+vL5GRkfj7+5OVlcXVq1frvdapsd3L6m9jWTEuOdUlOdUlOfVP9foigO+//55OnTrp2hpq4+Pjg0ajwcHBgd69e3Ps2DEA3ZV8BQUFnDhxglmzZulec+vWLfLy8nQDMTo6mri4OL788kug/LTj8uXLdZ+hY8eOQHlVUmpqKtHR0aSnp5Ofn8+tW7fo378/c+bMITMzk4SEBLy9veuUvSnV9zJRQ7+0tILkVJfkVJfkVEeDLyWvT31RhT179tS5zRvuri+qqBqqqC/SarWYm5vrvs8CuHTpkq7KaOnSpezfv5+YmBgcHByAu+uLKt5j/fr1JCUl4e/vz8CBAzl9+jSKomBiYsLo0aPZuXMniYmJrF27ts75hRBCqEvV+qIKP//8M3/729/qHCIxMRF3d3cuXLhAamoqCxcu5NSpU7rf29jY0KlTJ90RzYEDB5g7dy579uwhOjqa5ORkNmzYQKtWrXSvcXFxISEhgSeeeILU1FT++OMPAA4cOMDYsWPx9PQkLS2NX3/9Fa1WC5QfwY0fP55u3brRrl27OueHxq8vkgoiIURzpnp9UceOHcnIyNAdwdRFYWEhvr6+FBcXM2/ePN2pujstW7aMsLAw1qxZg0ajYcWKFQB88sknWFtbV7q0PSoqirfeeouQkBA8PDzo0qWL7rReYGAgYWFhREVFYW1tTa9evXSXmDs6OuLo6MiYMWPqnL2CMTRECCGEsWjy+qKQkBD69euHj49PU8ZAURSys7MJCAhgx44dmJub1+v1xjCcDP0cdAXJqS7JqS7JqQ6DqC9asmQJBw8evOvxHj16NMbb10lSUhJhYWGEhYXVezAJIYRQV6MMp/fee68x3qZB3N3dcXd3b+oYQgghULm+CCAlJQUfHx88PT15/fXXuXbtWsNT1iA7O5tJkybh5eWFn59fpQsp7iT1RUIIYTzueTj9ub6o4qurWbNmsXTpUrZv3063bt30fkn2ihUrGD58OF9//TXTp08nPDxcr+9XnTZtrLG3t9H9sWn1QJPkEEKI5kD1+qJdu3ah0WgoKSnh8uXLtS7EbWh90cKFC3XbyszM1F1Onp+f36T1RVIvJIQQ904v9UW//fYbr7zyCmZmZrz77ru1hmhIfVGLFuUHf+7u7mRlZekqkyIiIpq8vsgQq0MMMVNVJKe6JKe6JKf+6aW+yMnJiYMHDxIbG8s777xDbGxsjc9Xo75o9+7dnDp1ildffZXExESDqC8ytMs4Df3S0gqSU12SU12SUx21XUpe63dOe/fuZcaMGVhaWuLj40Pfvn2rrS8qKipiz549up+9vLz47bffag1Zn/qiij+bNm2idevW7Nu3j4KCAgCcnZ1p3749GRkZNdYXLV26FDs7O15++WW6du1aZX2RMXTrCSFEc1XrcLqzvqhVq1YkJydTVlamqy8CdPVFZmZmhIeHc+LECaC8lqh37961hkhMTERRFLKyskhNTeWZZ56p9Ps764ugvIJowoQJAGzdupWNGzcCcPbsWXJzc+nSpYuuvgiosr7Iy8uLoqKiu+qLYmNjcXR0vKf6ou3LvXV/pF5ICCHuner1RStWrGDu3LmUlZXRrl27ShcsVOde64tMTEwIDQ0lNDSUrVu3YmFhwfLly7GyspL6IiGEMGJSX/RfUl9kOCSnuiSnuiSnOqS+qI6kvkgIIQzHPQ+ngIAA1q9fD5QvvH3zzTfp0KEDKSkpfPjhh5SUlNChQweWLFmi1/qi7OxsZs2aRW5uLi1atCA4OBgXF5e7nufm5sYXX3zBww8/XOV2pL5ICCEMh9E3RCxduhQ3NzcSEhJYvnw5M2bMqHRvKSGEEMbH6BsiXnzxRQYMGADAo48+SlFREbdu3aKsrKxRGyKEEEKox+gbIoYPH67bztq1a3F2dsbGxoZ58+Y1eUOEEEKIe9NsGiKio6OJi4vjyy+/BGj0hoiarjoxJMZSZyI51SU51SU59a/W4bR3714iIiKYOHEiPj4+5OXl1dgQ8f333zN06FCgvCFiyZIltYaoT0NEhUuXLtG6dWug/Hun/fv3ExMTo7s9fE0NEUlJSfj7+zNw4EBOnz5dZUNEfb8rk0vJ1SM51SU51SU51dHg+iJDb4iIjo4mOTmZDRs26AYT0OgNEUIIIdRj1A0RAJ988gnW1taVbn4YFRXV6A0RVSkrKyUvL4fS0mJVttdQ2dktdIPYkDUkp5mZOba29piaNsoSPiGEnkhDxH/poyEiN/cilpYtsbJqhYmJiZpx74mZWQtKSw1/ON1rTkVRKCi4TmHhLR56yFEPySoz9NMmFSSnuiSnOqQhoo700RBRWlqMlZWDQQym+4GJiQlWVq24eTO/qaMIIRqoUYaTPhsi1KKvhggZTI1L9rcQzcM9N0Tc+R3PrFmzyMrKAiAlJQUfHx88PT15/fXXuXbtWsNT1sGBAwcIDAys9vdubm6675aEEEIYNqOvL9JqtfzrX//i3XffNfgv+21aPYC9vY3qf2xaPVCn9z916iSLF8/X86csd/LkCT79NKJR3ksI0fwYfX3RuXPnOHfuHPPnz9cV0QLk5+cbXH2RpYUZnv+TUPsT62n7cm/q8rWns3N3QkLmqP7+VTl//j/k5V1tlPcSQjQ/Rl9f9Nhjj7Fw4UKSk5MrbTMiIkLqi/7k2LEUPvtsNQBOTk+QmvozxcXFvP76dDZtiuX8+XTGjh3P2LETWLv2n1y+fInz5//DtWv5eHv7MH78RLRaLRERy0lJOYqJCQwfPpKXX57Ejz+msGpVBGVlWtq2bcvp079x+/ZtPv98LX/961gWLZpPTk42ubk59OnTj5CQOfz00zHWr1+HpaUl58//h65du/HBBwsxM7MgLi6Gbdu2YGpqysCBz/HGG29x9eoVli37kMuXL9OiRQumTg2ib9/+TbxXhRD60Gzqi/7MEOqLsrNbYGZ2z2dO66Wu71NxwYCJCaxb9yVr1vyTf/xjGV9+GUdeXh4TJ77EhAkBtGhhwm+/neKzz6LRasuYNGkC/fr158SJNHJysomJiaOkpJg33pjCY489hqXlA2Rk/MG2bTuxtrZhx46v+fHHY7z22t/45pvdODk5sXjxMkpKSnjpJV/OnfsNU9MWnDiRSlxcPA89ZM/kyYGkpBymTZuH2Lp1M9HRX2Jp+QB///ubnD37K19++QWenqMZPNiV3Nwcpk59jS++2ICVlVWlz9iiRYtGq20xlnoYyakuyal/zaK+qCqGUF+k1WobbV1RXd+nYp/06zeQ0lItbds60L37XzAzs8De3oEbN25QWqpFq1UYOnQ45ubl/xsMGjSYI0eOcPJkGu7uHiiKCWZmFgwd6s6RI8kMGjSYjh0fxdLSSvd6RVEoLdXi5jaMkydP8NVXX3L+/H/Iz7/GjRsFKIpC585dsbOzR6uFRx7pRH7+Nf7443cGDXoOS8vyofPxx58CcPRoMufPnycqahUAJSUl/PHHHzz2WOVTx1qttlHWdxj6OpIKklNdklMdzb6+qDpSX1Szin8AQOV/HNyp8j8aFMzMTKvoD1R098+ysLCocjubN8fy6acRtG5ty1//OpbOnTvrhuSda8oq/kFRnu3/LgnPzc3hxo0blJVpiYhYRXT0V0RHf0VU1Od06dKtPh9bCGEkah1Ofn5+7Ny5E09PT95+++276ouKiop09UUXLlzQ1Rd5e3uTlJRUr/qiqVOn1lhftHnzZjw9PVm+fDkrVqyocU3LW2+9RUZGBh4eHnz22WeV6otWrlyJp6cnH374oV7ri4zdd9/to7i4mOvXr3PgwHf07TuAZ57pQ2LiTsrKyigsLOSbb3bTq1efu15b8Q8WKD/i8fLyYdiwERQXF3PmzOkar6x8+uleHD58gFu3bv33gpzZ/PrrSZ55pg/x8eXfc/7nP+lMnDiWoqJC/Xx4IUSTqvW0npOTE9u3b6/xObNnz2b27NkAdOzYkfj4+HqFcHd3v6u+aPr06ZV+7tq1a6Wr8f6sf//+9O//f1+OW1tbs3Llyrue9/DDD5OUlHTX4xX1Rbm5ubrTkmorLCpl+/L6fZdV1+3qg4WFBUFBkykoKCAg4BU6d+5Cx46PkJHxB5MmvURpaSnDho3A1fV5fvwxpdJrnZ2f5F//imLVqkj8/cfz0UeL+PLLdVhZWdOjx1NcvHiBDh0ervJ9n3jCGR8ff15//RW0WgVX1+fp27c/nTt3YenShQQGjkNRFObMmUfLllZVbkMIYdwapVuvpvqisrIyg+jW2717t66+6F6aIqr6zunSpd9xcHhUrYgNVp/OurVr/wnAa69N1WekKjW0A7Cx9ruhn9OvIDnVJTnVYRDdevdzfZEQQoj6a5zrnFW2ceNGnnvuuTpdCViVmzdvMmrUqLvqjEpKSggMDLxrzdT96LXXpjbJUZMQQkAjHTmpbceOHSxatIhnn3223q89fvw477//PufPn6/0eHp6OqGhoZw8eVKllEIIIe6VwQ2n5ORkVq9ejUajITMzEzc3N1q2bMmePXuA8gLXtLQ0wsPDmT17NqtXr9Yt8o2Pj+f48eOEh4dXu/2NGzfywQcfEBwcXOnxzZs3M3nyZD7//HNVP0/FGirROJr49mRCCJUY5Gm9igGzZcsWYmJisLOzIz4+HicnJ2xsbOjRowcLFizA1dWVnJwc3Rqmbdu21XphxcKFC+nT5+5Ln4ODg1W/Ss/MzJyCguvyH8xGUnGzQTMzde7HJYRoOgZ35ATw+OOP4+hYfidTW1tbXFxcAGjfvj3Xr1/XPc/ExIQxY8bw9ddf4+Pjw5UrV5qsD6+qq05at7YkIyODnBy5VUdjeeABS7p164xGo2mU9zOWehjJqS7JqX8GOZz+/B+W6hoMAMaMGcPkyZMxNzevdx+emqq6lBzAxsYeGwP5/4ehX1paoaE58/MLAf0vzr1f9mdjkZzqMvScDa4vMnQdOnTAwcGB2NjYJh1OQggh1GP0wwlg5MiRdO3atdn34QkhxP2iURoi9Km0tJTg4GDc3d0ZNmxYk+XIyyuo8rSeIWnTxporV242dYxaSU51SU51SU51tGhhgq1t9fVjRj2cFEVh4MCBDBw4kGXLltGiRQtSUlKYP7/qW5FHRUXJ0ZUQQhgBox5OQgghmqdm8Z2TEEKI5kWGkxBCCIMjw0kIIYTBkeEkhBDC4MhwEkIIYXBkOAkhhDA4MpyEEEIYHBlOQgghDI4Mp//avn07I0eOZNiwYcTExNz1+1OnTuHj48Pw4cOZPXs2paWlAFy4cIEJEybg7u7OtGnTKCgoAOD69etMmTKFESNGMGHCBHJycgAoLi5m5syZjBgxgjFjxnDu3DmDzJmVlUWvXr3w9vbG29ub1157rUlzVti0aRMhISG6nw1tf1aX09D257lz55gwYQLe3t6MHTuWU6dOAYa3P6vLaWj78+zZs4wbNw4vLy8CAgLIysoCDG9/VpezoftTLxShXLp0SXn++eeVvLw8paCgQPH09FTOnDlT6TkeHh7KTz/9pCiKosyaNUuJiYlRFEVRpkyZouzYsUNRFEVZuXKlsnTpUkVRFCU8PFz55z//qSiKomzdulV5++23FUVRlDVr1ihz5sxRFEVRjhw5ovj5+Rlkzt27d+ty1pc+chYWFirLli1Tevbsqbz33nu67Rja/qwup6Htz3HjxinffvutoiiKcvDgQcXT01NRFMPbn9XlNLT9+fLLLyv79+9XFEVRvvrqK+Xdd99VFMXw9md1ORuyP/VFjpyAgwcPMmDAAFq3bk3Lli0ZPnw4u3fv1v0+KyuLwsJCevbsCYCPjw+7d++mpKSEo0ePMnz48EqPA+zbtw9PT08ARo0axXfffUdJSQn79u3Dy8sLgL59+3L16lUuXLhgcDnT0tI4ffo03t7eTJw4kd9++61J9+fRo0fRarXMnDmz0nsZ2v6sLqeh7U8/Pz+ee+45AJycnLh48aJB7s/qchra/ly3bh2DBw9Gq9Vy4cIFWrVqZZD7s7qcDdmf+iLDCcjOzsbe3l73c9u2bbl8+XK1v7e3t+fy5cvk5eVhbW2NmZlZpcf//BozMzOsra25evVqldu6dOmSweW0sLDAy8uLrVu38tprrxEUFERxcXGT5Xz22WcJDg7G0tKyxvdq6v1ZXU5D258+Pj66m3hGREQwdOjQarfVlPuzupyGtj/NzMy4fv06gwcPZsOGDfj7+1e7rabcn9XlbMj+1BcZToBWq8XExET3s6IolX6u7vd/fh5w1893vqZFixZ3vabicUPLOX36dMaPH0+LFi1wdXWlZcuWpKenG0zO6rZtaPuzgiHuT0VRWLJkCcePHyc0NLTKbRvC/qwqpyHuz1atWvHDDz/wv//7v0ybNo2ysjKD3J9V5WzI/tQXGU6Ag4OD7kIAgJycHNq2bVvt73Nzc2nbti12dnbcuHGDsrKyu17Xtm1bcnNzgfJ7ThUUFNC6dWvatWtHdnb2XdsytJzr168nLy9Pty1FUXT/EmuKnNUxtP1ZHUPbn6WlpcyYMYO0tDS++OILbGxsAMPbn9XlNLT9uWvXLpT/3uBh8ODBFBYWcu3aNYPbn9XlbMj+1BcZWwqehAAAAdhJREFUTsDAgQM5dOgQV69e5fbt23zzzTcMHjxY9/sOHTpgYWHBsWPHAEhISGDw4MFoNBr69OnDrl27ANi2bZvuda6urmzbtg0o/z9Enz590Gg0uLq6kpCQAEBKSgoWFha0b9/e4HIePXqUzZs3A3DkyBG0Wi1dunRpspzVMbT9WR1D259Llizh5s2b/Otf/9L9Bx8Mb39Wl9PQ9ue//vUv/v3vfwNw+PBhbG1tsbOzM7j9WV3OhuxPvdHXlRbG5uuvv1Y8PDyUYcOGKVFRUYqiKMrkyZOV1NRURVEU5dSpU4qvr68yfPhw5d1331WKiooURVGUzMxM5eWXX1ZGjBihvPrqq0p+fr6iKIqSl5enTJ06VRk5cqQyduxYJSMjQ1GU8qu5goODlZEjRyqjR49WTpw4YZA5L126pEyaNEnx8PBQfHx8lFOnTjVpzgpbtmypdBWcoe3P6nIa0v68cuWK4uzsrLz44ouKl5eX7o+h7c+achrS/lQURTlz5owybtw4xcvLS5kwYYJy+vRpg9ufNeVs6P7UB7nZoBBCCIMjp/WEEEIYHBlOQgghDI4MJyGEEAZHhpMQQgiDI8NJCCGEwZHhJIQQwuDIcBJCCGFwZDgJIYQwOP8fF819c+pyZdkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate feature importance plot\n",
    "\n",
    "feature_importance = pd.DataFrame(model.feature_importances_, \n",
    "                                  index = df.columns[:-1], \n",
    "                                  columns = ['importance']).sort_values('importance', ascending = True)\n",
    "feature_importance[:20].plot.barh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.997042 using {'n_estimators': 700}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled RF\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([400, 500, 600, 700]))\n",
    "model = RandomForestClassifier(random_state=seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.996986 (0.007230) with: {'n_estimators': 400}\n",
      "0.996993 (0.007299) with: {'n_estimators': 500}\n",
      "0.997014 (0.007396) with: {'n_estimators': 600}\n",
      "0.997042 (0.007313) with: {'n_estimators': 700}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "                       max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=700,\n",
       "                       n_jobs=None, oob_score=False, random_state=7, verbose=0,\n",
       "                       warm_start=False)"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = RandomForestClassifier(random_state=seed, n_estimators=700)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9804906542056075\n",
      "\n",
      "Accuracy\n",
      "0.9572649572649573\n",
      "\n",
      "Confusion Matrix\n",
      "[[213   1]\n",
      " [  9  11]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.96      1.00      0.98       214\n",
      "         1.0       0.92      0.55      0.69        20\n",
      "\n",
      "    accuracy                           0.96       234\n",
      "   macro avg       0.94      0.77      0.83       234\n",
      "weighted avg       0.96      0.96      0.95       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix \n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3iTZffA8W/SvaArpYyyN5SNZZQtIFhQhohMQfEFQXzxB6KAoKAyFEHEl+0WWTJFoIiAyCobWZYyC3SX7pnk+f1RDYQWmhbStM35XJfX1Wef3Ibn5BnnvlWKoigIIYQQJlJbOgAhhBAliyQOIYQQBSKJQwghRIFI4hBCCFEgkjiEEEIUiCQOIYQQBWJr6QCEeFxDhw4lJCTEaJ6NjQ1lypShadOmTJw4kRo1ahgtT0hIYOXKlezevZs7d+5QtmxZGjZsyPDhw2ndunWex/ntt9/46aefuHDhAhkZGVSpUoUXXniBAQMGYGdnZ7bPJ0Rxo5I6DlHSDR06FK1Wy+TJkw3zsrKyuHTpEl9++SVOTk7s2rULBwcHAK5fv86IESPQarWMGDGCBg0akJCQwObNm/n9998ZN24cb7zxhtExPvjgA9asWcPzzz9Ply5dcHZ2JiQkhK+++ooOHTqwcOFCbGxsivRzC2ExihAl3JAhQ5TXXnstz2Vr165Vateurezbt09RFEXRarVKUFCQ0rVrVyUuLi7X+gsXLlRq166t7NmzxzBv06ZNSu3atZU1a9bkWn/79u1K7dq1lU2bNj2hTyNE8SfPOESp5urqajS9d+9eQkNDmThxIp6enrnWHzduHJUrV2bp0qWGeatWraJOnTq8+OKLudbv2bMnI0eOxMPD45FxrF27lmeffZZGjRrxzDPPsG7dOsOyzp07M3PmTKP1P/roIzp37myYrlOnDkuXLuXZZ58lICCAnTt3UqdOHU6dOmW03Q8//EDjxo1JTU0F4Ny5cwwfPpzGjRvTqlUrZs2aRXp6+iNjFSI/8oxDlAqKoqDVag3TmZmZnDt3jgULFlChQgVatGgBwMGDB1Gr1QQGBua5HxsbG7p06cLXX39NfHw8Wq2W0NBQXnvttYce+/5bZHn5+uuvmTt3Li+//DLt27cnJCSE9957D2dnZ4KCgkz+jIsXL2bKlCl4eHjQokULfH192blzJ02bNjWss2PHDjp16oSLiwthYWEMGTKEJk2asHDhQuLi4pg/fz63bt1i2bJlJh9XiAdJ4hClwv79+2nQoIHRPEdHR1q3bs27776Li4sLALdv38bDwwNnZ+eH7qtSpUoAREREoNPpAKhQoUKh4tLr9SxdupS+ffvyzjvvANCmTRvCw8M5ceJEgRJH27ZtGTRokGG6Z8+e7Nixg3feeQeVSkV0dDQnT57kiy++AOB///sfXl5eLF++HHt7ewCqVq3K4MGDOXbsGC1btizUZxJCEocoFZo3b867774LwOXLl5k7dy6tW7dm3rx5hpMm5FyZ5PcQ+/7l//6t1+sLFde1a9dISEgwuu0EMH/+/ALv68E3w4KCgvjqq684c+YMTZo0YefOnbi6utK+fXsAjh49SpcuXVCr1YarsSZNmuDq6srhw4clcYhCk8QhSgU3Nzf8/f0B8Pf3p3z58owYMQJ7e3vmzZtnWK9ixYocPnyYzMxMw1tWD7p9+zYA5cuXN8yLiIh46LGjo6Px9vZGrc79yDAhIQEgz+cpBeXl5WU03aBBA6pVq8bOnTtp0qQJO3bsoGvXroZEmZCQwNq1a1m7dm2ufcXExDx2PMJ6ycNxUSq1bt2a/v37s2XLFn7//XfD/E6dOpGdnc3evXvz3E5RFH7//Xf8/f3x9PTE09OT+vXrc+DAgYcea8SIEYwYMSLPZW5ubgDEx8cbzb927RonT540TD94RZOWlvboD/iPoKAggoODiYqK4tSpU0a3vlxdXenXrx8bNmzI9d9//vMfk/YvRF4kcYhS66233sLNzY05c+aQlZUFQGBgIP7+/sybNy/PX93Lli3jypUrRg/Dhw8fzqVLl1i/fn2u9bds2UJYWBi9evXKM4bq1avj7u7Ovn37jOZ//vnnhishV1dXoqOjDcv0en2ut6UeJigoiNu3b7NkyRK8vb0JCAgwLGvevDlXr16lYcOG+Pv7G67E5s+fz+XLl03avxB5sXn//ffft3QQQjyOTZs2YWdnl+vk7eTkhEqlYseOHbi5udGsWTPUajWtWrVi27Zt/PDDD6jVarKysrh48SKLFi3i+++/Z/To0QwZMsSwnzp16nDlyhVWrlxJVFQUarWaGzdu8OOPP/L555/TvXt3JkyYgEqlyhWbWq3GxsaGlStXkpmZCcDGjRtZs2YN06ZNo3r16ty+fZvNmzdTpkwZ0tLSDCd2e3t7hg8fDuS8UdW2bVuaNWtmtP9/k9L+/fvp06cPHTt2NCyrUqUK//vf//j7779xdnbm8uXLTJ8+nevXrzN+/PhcryoLYSqpHBcl3tChQ3F2ds7zFdOsrCyeeeYZkpKSCA4ONjxrSExM5LvvvmPnzp3cunULFxcXGjVqxLBhw2jTpk2u/ej1etatW8fGjRu5ceMGWVlZVKtWjQEDBtC/f39sbR/9uPDHH3/k22+/JSIigqpVqzJmzBh69uwJQEpKCrNmzWL37t3Y2trSu3dvKlSowA8//GC4zVanTh3efvttXnnllVz7/uabb5g9ezZr166lSZMmRstOnDjBwoULOXv2LA4ODjRr1oz/+7//o1atWqY1rhB5kMQhhBCiQOQZhxBCiAKRxCGEEKJAJHEIIYQokCJLHCkpKQQFBXHr1q1cyy5evEjfvn3p3r07U6dONepzSAghRPFSJInjzJkzvPTSS1y/fj3P5ZMmTWL69Ons2rULRVGMeg4VQghRvBRJlyPr1q1jxowZvP3227mW3b59m4yMDMNrhH379mXRokVGnbmZ4u7dVPR6BS8vV+LiUp5I3KaKS8lgzdHrRCcWn+6qVWoVil5emJN2uEfaIoe0A3i6OjDzpacKvX2RJI6PPvroocuio6PRaDSGaY1GQ1RUVIGPodcr6P/5MuiL+Eux48xtDv5d8JiFEMISypV1eqztLd7JoV6vN6q4VRQlzwrc/Hh53auC1WjcnkhsptIVPFwhhCixLJ44fH19jfoMio2NxcfHp8D7iYtLQa9X0GjciIlJfpIh5isjI9vw96DW1WldU/OItYuGJW7ZFUfSDvdIW+SwpnbQKwqHL8ew+eRNMrJ1hvmVPB8+Ho0pLJ44KlasiIODAydOnKB58+Zs2bLFMJ5ASeRsb4OHS97ddRclLzdH9PclNGsl7XCPtEUOa2mHyIR0Vu4P5VJEotH8TvV8GdK25mPt22KJY9SoUYwfPx5/f38+/fRTpk2bRkpKCg0aNGDYsGGWCksIIUo0rU7Pr2dvsen4DbJ19573livjyCsdalO/ojtq9ePdXy/SxHH/uAgrVqww/F23bl02bNhQlKEIIUSpcy0mmZX7QrkRl2qYp1ZBz8Z+9G1RGXvbR49+aSqL36oSQgjxZGw+cdMoaVT1duXVjrWp6v1ku9CXxCGEEKXEsMCanL+dgE6v0K9lFXo0qoTNY96WyoskDiGEKIFSM7Oxs1Eb3X7ycnXg9S51qeDhjO9j1mo8iiSOPCiKwo24VKPX1x4lIa30v6EhhCg+Qq7G8O2fV2hfpxwvBlQzWtasqpfZjy+JIw9f/naJI1dyj0cthBCWdDc1k2//DOP4tTgAtp8Op1UNDVWe8DOM/EjieIBeUTh6tfBJw8vV8jUcQojSRVEU9l2M5KcjV0nLuncnpIyTPSmZRX/HQxJHHu4fTLe2bxmTt2tQ0Z0GlTzMEJEQwlpFJqazan8oF+/kLuQb2Ko6Lg5FfxqXxPEIKmD6800sHYYQwgrp9Aq/nrnFxuM3yNbpDfN9yzoxsn0t6ld0t1hskjiEEKKYuZuayac7znMj9l6fWmoVPNvEjz7Nn1whX2FJ4hBCiGKmjJM991dfmKuQr7BkzHEhhChmbNQqRnWsjaOdDQNbVeODvk2LTdIAueIQQgiLSs3U8vuFCJ5tUgn1fWMRVfF2ZdGQAJwt8PA7P8UvIiGEsBLHrsbyzZ9hJKZl4WRvw9MNKhgtL45JAyRxCCFEkXuwkA9gzZFrBFTX4OZkZ8HITCOJQwghisjDCvnKOtvzcmDNEpE0QBJHbkr+qwghREE9rJCvY11fXmptmUK+wio5kRaRu2mZhr9dHKV5hBCP52GFfD5lHHn1nxH5Sho5Mz4gMjHd8Lc5uyUWQliH7afDWRdy3TCdMyJfJfq2qGLxQr7CksTxAEkcQognqWvDCuy5EEFcSiZVvFx4tWNtqmncLB3WY5HE8YCoxAzD3+UkcQghCkivV1DfN+qek70tI9vX4mZcKj0aVcTWpuTXXUvieIBccQghCiM1U8tPh6+SpdXx+tP1jJY1ruxJ48qeForsyZPE8QBJHEKIgjp2LZZvD4SRkJYFQOtaPjStYv6R+CxFEsd99HqF6PsSh9yqEkI8yt3UTL778wrHrsUazT97864kDmsRl5KJVp9TyFHWyQ5ne2keIURuiqKw/1Ikqw9fIy1La5j/byFfy+reFozO/OTMeJ9IudoQQuQjMjGdr/aHciHPQr5quDiUjOrvx2H1ieN2fCor9ocSmZiOVnevbFyebwgh7qdXcgr5fj6Wu5DvlQ61aFDReoaNtvrEEXzuDmFRybnml3d3tkA0QojiSgWERiYZksa/hXx9mlfBwa5kFvIVltUnjtT77k/+y6eMI+3rlLNANEKI4kqlUvFyYE0u3knAx82xVBTyFZbVJ477jWhfk6eqaXBxtDUaUEUIYX3+jkikmsbVqFsQT1cHpvZuTCUP51JRyFdY1vvJ8+BkZ4ubk50kDSGsWGqmllX7Q5m15Qwbj9/Mtbyqt6tVJw2QKw4hhDB4sJDv1zPhBNTwttpbUg8jiUMIYfUS0rL49s8wjl01LuRrWsWLsk72Foqq+JLEIYSwWg8t5HOyY/g/hXwquXWdiyQOIYRVikpMZ9Ufl7lwO8Fofoe65XipVXVcHUt/IV9hFUni2LZtG0uWLEGr1TJ8+HAGDx5stPz8+fNMnz6d7OxsypcvzyeffEKZMmWKIjQhhBW6Gp3Mh1vPkKU1LuQb2b4WDStZTyFfYZn91YCoqCgWLFjA6tWr2bx5M2vXriUsLMxonY8++ojx48ezdetWqlWrxqpVq8wdlhDCilXxdqWiR06Rr+qfQr7ZLzSXpGEisyeOQ4cO0apVK9zd3XF2dqZ79+7s3LnTaB29Xk9qaioA6enpODo6mjssIYQVs1GreLVDbappXPmgT1MGta5uddXfj8Pst6qio6PRaDSGaR8fH86ePWu0zjvvvMPIkSP5+OOPcXJyYt26dQU+jpeXq+FvTQFenXO8r0OyMmUcC7RtcVeaPsvjkHa4xxrb4uyNOHaeCmfic40NNVoajRsajRvN6/rKw+9CMHvi0Ov1Rv9jFEUxms7IyGDq1Kl88803NGrUiK+//prJkyezfPnyAh0nLi4FvV5Bo3EjJiZ331MPk5GZbfg7KSmjQNsWZwVth9JK2uEea2uL1Ewta45cZe/FSAAqlXWia8MKVtcOeVGrVUY/tgu8/ROMJU++vr7ExMQYpmNiYvDx8TFMh4aG4uDgQKNGjQB48cUXCQkJMXdYQohS7Pi1WCavPW5IGgDbTt1Ee1+vtqLwzJ442rRpw+HDh4mPjyc9PZ3g4GDat29vWF6lShUiIyO5evUqAHv27MHf39/cYQkhSqGEtCwWBV9g4a4LhupvgOZVvXi/T1Or7yrkSTH7rapy5coxYcIEhg0bRnZ2Nv3796dRo0aMGjWK8ePH4+/vz+zZs/nvf/+Loih4eXnx8ccfmzssIUQpoigKf/wdxY+HrkohXxEokjqOXr160atXL6N5K1asMPzdoUMHOnToUBShCCFKGSnkK3pSOS6EKNF+O3/HKGlIIZ/5SeIQQpRo/VpWJeRqLPGpmfRoVIl+LaxvRL6iJolDCFFiZGn1ZOt0uNxXf+VoZ8OYznWwt7Whuo/11alYglUnjgu3EzgSFpP/ikIIi7t0J4GV+y9TxduVN7rWM1pWt4K7haKyTlabOJIzsvnk178sHYYQIh9pmVrWHL3G7xciAIhMTKdNLR+aV/WycGTWy2oTR0xSBtk6xTCtVkGd8mUtGJEQ4kEnrsXyzYEw7t5Xk+FoZ0PGfa/ciqJntYnjQZ++1BIvVwdLhyGEABL/GZEv5IER+ZpV9eLlwJp4yr9Vi5LEAVTTuOJTxsnSYQhh9f4t5Ft9+CqpmfeuKsr8U8j3lBTyFQuSOIQQxUJmto4FO89z7oFCvnZ1yjG4tRTyFSeSOIQQxYK9rdqo/kLj5sgrHaSQrziSxCGEKBZUKhUvt6vJ3xGJtKtTjn4tq+IohXzFUoETR1ZWFvb29uaIRQhhJbK0eoL/uk3XhhWMrjI8XByYP+gpXBzkN21xZlIfw1qtlsWLF9OuXTsCAgIIDw9nyJAhRuNsCCGEKS5FJDJ1wwnWHL3Gz8dv5FouSaP4MylxzJ8/n5CQEObPn4+trS0ajQZfX19mzJhh7viEEKVEWpaWr/+4zIdbzhCRkA7AjrO3CI9PtXBkoqBMSu3bt29ny5YteHh4oFKpcHR0ZObMmXTq1Mnc8ZnF3xGJzNpyxtJhCGE1Tl6P4+sDl7mbalzIN7BVNSp6OFswMlEYJl8T2tgYP6TS6XQ4OJTMIpxfz9wymra3lVHBhDCHxLQsvjt4haNXjG9rN63iycvtaknRbQllUuLo1q0bEyZMYNKkSSiKwq1bt/j000/p0qWLueMzi/sLiwC6+1e0UCRClE6KonAgNGdEvgcL+Ya1rUlADSnkK8lM+qk9adIkKleuzMCBA0lOTqZnz564uLgwceJEc8dndpOfbchT1TWWDkOIUiXkaizL94YaJY12dcox98UWtKqpkaRRwpl0xZGRkcGMGTOYMWMG8fHxuLu7o1arCQ8Px8XFxdwxmpWNWm5TCfGktazmTXWNG1djktG4OTCyfW38/aSQr7Qw6ax5/0NwT09P1Go1Wq2W559/3myBCSFKDr1eMZpWq1W82rEWPRpVZPaAFpI0SpmHXnHcunWLgQMHotPpSEtLo3Xr1kbLMzMzqVmzptkDFEIUX1laPZtP3ODvyESm9mqMWn3vFlRlL1cGt3G1YHTCXB6aOCpVqsSKFStISkritdde4/PPPzdabm9vT506dcweoBCieLoUkciq/aGGmozd5+/IiyZW4pHPOOrVyxmece/evXh6euZartfrzROVEKLYSsvSsvbINfb8MyLfv87cjKdbwwry4NsKmPRwPDExkblz5xIVFWVIFlqtlhs3bnDw4EGzBiiEKD4eVcjXuX55SRpWwqTEMWXKFDw8PKhQoQK3b9/mqaeeYs2aNQwaNMjc8RWYTq/n4p0EsrQPvxpKycwuwoiEKPmkkE/cz6TEcfHiRY4cOcKtW7eYNWsWY8eOpW3btsycOZOxY8eaO8YCmbr6GKeuxea/ohDCJH+GRvH9wSvGhXyOdgwLrEFADanJsEYmJQ53d3ccHByoXLkyYWFhADRp0oTw8HCzBldQGdm6AicNbzf5pSTEo1yPSTEu5KtdjkFtquMmI/JZLZMSR7169fjss88YN24cPj4+7N69G0dHR5yciu843SoV+D9i5DC1SsVT1b1lrHEh8tH/qaqcuJ7zg2xk+1r4++V+UUZYF5MSx9SpU5k2bRoJCQlMnDiRN954g6ysLN5//30zh1d49jZq3n7W39JhCFGihMel4uZkh7vzvcHaHO1smNijIV5ujjIinwBMTBynTp1i6dKlODo6Uq5cOY4ePUp2djbOztIdshClQZZWz5aTN/nldDjNq3oxvlt9o+UVPUt210LiyTKpy5FZs2Zha3svx9jZ2UnSEKKU+DsikWkbTrDl5E10eoWQq7EclxdMxCOYdMXRpUsXli1bRq9evfD2Nu4OuTg/5xBCPFxalpZ1R6/x23njQr7avmUo7y4/DMXDmZQ4du/eTUpKCosXLzbMUxQFlUrFxYsXzRacEMI8Tt2I4+s/LhP/QCHfiwHV6NKgPGp5xVY8gkmJY8uWLeaOQwhRBBLTs/j+4BWOhBkX8jWp7MmI9jXxcnW0UGSiJDEpcVSs+Hgdl23bto0lS5ag1WoZPnw4gwcPNlp+9epVZsyYQWJiIhqNhs8++4yyZcs+1jGFEMZikzKYvOY4KQ8U8g0NrEErKeQTBWD2UYyioqJYsGABq1evZvPmzaxdu9ZQRAg5t7zGjBnDqFGj2Lp1K/Xq1WP58uXmDksIq+NdxpF6Fd0N04G1fZg7sAWta/pI0hAFYtIVx+M4dOgQrVq1wt095wvbvXt3du7cybhx4wA4f/48zs7OtG/fHoDRo0eTlJRk7rCEsErD2tYgOimDFwOq0kgK+UQhmT1xREdHo9HcG9Pbx8eHs2fPGqZv3ryJt7c3U6ZM4eLFi1SvXp333nuvwMfx8nLFOeveJbhKpUKjcXu84Es4a//8/7LGdrgWlcSPB8KY2LsRjvb3/pnXrurNstHtrf4Kwxq/E0+SyYkjJCSEtWvXEhMTw4IFC/jxxx8ZO3YsNjaPriTV6/VGX9J/38b6l1arJSQkhB9++AF/f38WLlzInDlzmDNnToE+SFxcCmn33btVFIWYmOQC7aM00WjcrPrz/8va2iFbl1PIt+1UODq9gpu9DYNaVwesry0eRtohZ2hfL6/Cj85o0jOOjRs3MnHiRKpVq8b58+dRqVTs3r2befPm5butr68vMTH33uCIiYnBx8fHMK3RaKhSpQr+/jndgwQFBRldkQghTBMakcjU9SfYfCKnkA/gt/N3SEzPymdLIQrGpMSxbNkyVqxYwbhx41Cr1Xh6erJixQq2b9+e77Zt2rTh8OHDxMfHk56eTnBwsOF5BkDTpk2Jj4/n0qVLAPz+++80aNCgkB9HCOuTnqXl2wNhzNpyhjv/DOMKOYV8s/o1o6yT/SO2FqLgTLpVlZCQQM2aNQEMt5m8vb3Jzs5/QKRy5coxYcIEhg0bRnZ2Nv3796dRo0aMGjWK8ePH4+/vz5dffsm0adNIT0/H19fXpCsZIYQU8gnLMClxNGvWjEWLFjFhwgTDvG+//ZYmTZqYdJBevXrRq1cvo3krVqww/N24cWM2bNhg0r6EEJD0TyHfYSnkExZgUuKYPn06o0ePZs2aNaSkpNC5c2ccHR1ZtmyZueMTQuThyJUYo6Th5mjH0LY1aF1TCvmE+ZmUOMqXL8+mTZs4e/YsERERaDQamjRpYtRjrhCi6DxdvwIHQ6O5Ep1MYG0fBreugZuTjMgnioZJZ/6BAwfy3HPP0aNHD5NvTwkhngy9XiE5M9voIbdarWJUx9rEpWTSuLIU8omiZdJbVf379yc4OJiOHTsyduxYgoODTXowLoR4POFxqXyw+TSf7TiP/p9XbP9VydNFkoawCJMTx9dff83u3bt56qmn+OqrrwgMDGTGjBnmjk8Iq5St07Ph2HWm/XySK9HJXIlOJvjcbUuHJQRQwC5HNBoNLVu2JDY2lsjISE6fPm2uuISwWqERiazcH2pUk2GrVqF94IpDCEsxKXFcuXKF7du3s337dlJTUwkKCmLJkiXUq1fP3PEJYTXSs7SsO3qd387f4f4UUatcGV7tUEvG/RbFhkmJo1+/fnTu3JkpU6bQrl071Gqz98YuhFWRQj5RkpiUOP78809cXQvfIZYQIm+KorD09785eDnaaH7jyp6MaFcTbzcp5BPFzyMTx9ChQ/n+++8ZPnz4Q4uKpOJbiMJTqVSUua/+Qgr5REnwyMQxYMAAAIYMGVIkwQhhjfq1rMqJ63HUKleGIW2kkE8Uf49MHP/2L5Wamppn8li6dKl5ohKiFNLrFX67cIeW1bzxcHEwzHe0s2FWv2a4OEhPDKJkeOg3NS4ujnPnzgHw6aef4ufnZ7Q8JSWFZcuWMXr0aPNGKEQpcCs+lRX7QrkSnczF24m82b2+0XJJGqIkeei31cXFhUWLFnH37l0yMzP54IMPjJbb29tL0hAiH9k6PVtP3mTrPyPyARy7FsvpG/E0qSJV36JkemjicHR05OeffwZg3LhxLF68uMiCEqI0CI1MZOX+y9y5m2aYZ6NW8VyzyjSs5G7ByIR4PI+8Pr569SrVq1fnv//9L2FhYXmu8+8AT0KIHOlZWtaFXOe3c1LIJ0qnRyaO/v37c/LkSYKCgvJcrlKpuHjxolkCE6IkOn0jnq8PXCYuJdMwz9HOhgEBVXm6QQUp5BOlwiMTx8mTJwEM44ELIR7uanQyn+44ZzSvsZ8HI9rXkkI+UaqY3HfI4cOHAYiPj+f999/n888/JyMjw2yBCVHSVPdx46nq3kBOId/rXeoysWdDSRqi1DHpHcBPPvmEX3/9lb179zJ16lRSUlKwtbXl/fffZ86cOeaOUYhiSa9XUKuNbz0NC6yJi4MtA56qJoV8otQyKXHs3r2bNWvWkJyczB9//MHOnTvx9PSkc+fO5o5PiGJHr1fYff4Oey9G8H6fpjja2RiWuTvb80qH2haMTgjzM+lWVUJCAuXKlePgwYNUqlQJPz8/7OzsUBQZH0BYl1vxqczccprvD17hVnwaG0KuWzokIYqcSVcc9erV4+OPP+bYsWM8/fTTJCQk8Mknn9C4cWNzxydEsZBXIR/Audt3ydLqsbeVoQaE9TDp2z5nzhwSEhJo1qwZb7zxBjdu3CAuLo4PP/zQ3PEJYXGXI5OYtuEkm07cNCQNG7WKPs0rM6tfM0kawuqYdMVRvnx55s2bR2ZmJtHR0dSoUUM6OBSlXnqWlvUh19n9QCFfzXJuvNqhNpWkkE9YKZMSR0pKCjNmzGDHjh3o9XpsbGzo2LEjc+fOlQGeRKl0NjyeVfuNC/kcbNUMCKhG1wYVcr1NJYQ1MflWVWpqKtu3b+fs2bNs3boVnU7Hxx9/bO74hLCIyIR0o6TRyM+DuS+2oLt/RUkawuqZdMWxb98+duzYgZubGwA1atRg7ty5dOvWzazBCWEpTzeowOGwGCIS0xjSpgZta/nIiHxC/PYIOu8AACAASURBVMPkQQB0Op3RtF6vx85OCpxEyRebnEGWTk8Fd2fDPLVaxZgudXC0s6GMk70FoxOi+DHpVlXXrl2ZMGECFy9eJCEhgfPnz/PWW2/RtWtXc8cnhNno9QrBf91m8trjLN3zN3q9cV2STxknSRpC5MGkxDFp0iQ8PT154YUXaNWqFS+99BIVK1Zk0qRJ5o5PCLP4t5Dvu4NXyNTquRqTzK6/bls6LCFKBJNuVTk7OzN//nw+/vhjkpKS8Pb2lvu9okR6WCFfBQ9napYrY8HIhCg5TEocWVlZLF68mF9++YXY2FjKly9Pnz59eO2111CrpfhJlAyXI5NYuT+U2w+MyNe7qR+9m1XGzka+y0KYwqTE8eGHH3LhwgWmTJmCr68vt2/fZtmyZaSkpDBx4sR8t9+2bRtLlixBq9UyfPhwBg8enOd6+/btY+bMmfz+++8F+xRCPEJGto51R69JIZ8QT4hJiWPHjh3s2rULT09PABo2bEjTpk3p3bt3vokjKiqKBQsWsHHjRuzt7Rk4cCABAQG5hpyNjY1l7ty5hfwYQuQtM1vHlPUniE66N3aMFPIJ8XhMujZ3dXUlKyvLaJ6tra2hruNRDh06RKtWrXB3d8fZ2Znu3buzc+fOXOtNmzaNcePGmRi2EKZxsLOhZTVvw7QU8gnx+Ey64ujXrx+vvPIKr7/+On5+fkRHR7NixQpat27N/v37Det16NAh17bR0dFoNBrDtI+PD2fPnjVa57vvvqN+/frS264wi74tqnApIpFuDSvQRgr5hHhsJiWOjRs3AjB//nyj+TExMfz5558AqFQq9uzZk2tbvV5v9A9VURSj6dDQUIKDg/nmm2+IjIws+Cf4h5eXK85ZWsO0SqVCo8n/iqg0s7bPH52YzorfLjK6W3287huutVIFd758rZ0kDKzvO/Ew0g6Px6TE8TgPq319fTl+/LhhOiYmBh8fH8P0zp07iYmJoV+/fmRnZxMdHc2gQYNYvXp1gY4TF5dCWua9xKEoCjExyYWOu6TTaNys5vPrFYXfzt9h3dHrZGTrSE3L4r/d6xt+PFhLO+RH2iKHtENOzwheXoXvoNbs7x+2adOGw4cPEx8fT3p6OsHBwbRv396wfPz48ezatYstW7awfPlyfHx8Cpw0hPW6HZ/KrM1n+O7PK2Rk53SLc/J6HDfjUi0cmRCll8l9VRVWuXLlmDBhAsOGDSM7O5v+/fvTqFEjRo0axfjx4/H39zd3CKIU0ur0bDsVzpaTN9E+UMj3aodaVPGW7v6FMBezJw6AXr160atXL6N5K1asyLVepUqVpIZD5CssKomV+0K5JYV8QlhEgRNHVlYW9vbS8ZsoehnZOtaHXCf4r9tGhXw1fNx4tWNt/KSQT4giYdJPM61Wy+LFi2nXrh0BAQGEh4czZMgQYmJizB2fEAZ/RySy676k4WCrZkibGsx4vokkDSGKkEmJY/78+YSEhDB//nxsbW3RaDT4+voyY8YMc8cnhEHjyp60qpFTE+RfyYM5A1rwTCMp5BOiqJl0q2r79u1s2bIFDw8PVCoVjo6OzJw5k06dOpk7PmGlFEUhMT0bd2fj26JDA2vQpIqnjMgnhAWZ/BTRxsbGaFqn0+Hg4PDEAxIiLiWD+TvOM2PjKcMrtv8q62RPYO1ykjSEsCCTEke3bt2YMGECly5dQlEUbt26xXvvvUeXLl3MHZ+wInpFYfe5O0xee4LTN+OJS8lk3dFrlg5LCPEAk0cArFy5MgMHDiQ5OZmePXvi7OxsUpfqQpji9t00Zm0+w7d/huW6ylAU5SFbCSEswaRnHA4ODsyYMYMZM2YQHx+Pu7u7DOAkngitTs+20+FsOfFAIZ+7E692qE3t8mUtGJ0QIi8mJY7Fixc/dJl0hS4KKywqZ0S+W/HGhXy9mvrRu2ll7G3lx4kQxZFJiSM0NNRoOiEhgdOnT+eqBhfCVOtDrrP15M3chXwdauPnJTUZQhRnJiWORYsW5Zp38OBB6YxQFJq7s71RId8LT1WlW0OpyRCiJCh0X1UBAQG88cYbTzIWYUW6NCjPocvRONrZMLJ9LTRlHPPfSAhRLJiUOMLCwoyms7Oz2bZtGxUrVjRLUKL0UBSFI1diqODubNRjrVqlYmLPhjjb20hNhhAljEmJIygoCJVKZXgtUq1WU7VqVd577z2zBidKtriUDL7+I4zTN+OppnHl/T5NsbnvVpSLQ5F0ziyEeMJM+pcbHBxM5cqVzR3LY0nP1rHh2HVLhyHIKeTbcz6CtUevGWoyrsWksOfCHbo1lKtUIUo6k953fPHFF0lJSTF3LI/l8OVodp69bZiW2x+W8bBCvq4NKhBYu5wFIxNCPCkmXXH4+flx/vx5AgICzB1PocWmZBpNN/LzsFAk1kkK+YSwHiYlDjs7O0aMGIFGo8Hb29vo1/yGDRvMFlxhtazuzbin61k6DKvxqEK+52REPiFKHZMSR//+/enfv7+5Y3liGlZ0l3qAInI3NZMPt5wxusqQQj4hSjeTEkdqaipDhgzJNX/p0qVPPCBRsni4OPBMo0r8cjpcCvmEsBIPTRxxcXGcO3cOgE8//RQ/Pz+j5SkpKSxbtozRo0ebN0JRrOj1Sq6k0LdFZZIzsnmumR8+ZZwsFJkQoqg8NHG4uLiwaNEi7t69S2ZmJh988IHRcnt7e0kaVkRRFI5eiWF9yHWm9G6Ml+u9QbzsbW0Y1bG2BaMTQhSlhyYOR0dHfv75ZyCnB9xH9ZArSre4lEy+OXCZUzfiAfjmwGXeeqaBvPIshJV67G7VRemVVyEfwPXYlDzHAxdCWAfp80Hk6c7dNFbuDyU0Mslofpf65XmxVTWc7eWrI4S1kn/9wohWp+eX0+FsfqCQr/w/hXx1pJBPCKsniUMYXItJZvneUMLjUw3zbNQqgppU4rlmVWREPiEEIIlD3CchLcsoaVTXuPFqx1pU9nJ9xFZCCGsjiUMYNK3iReuaGk5cj6N/y6o84y+FfEKI3CRxWKmUjGxikjOopnEzmj+0bQ1eeKqqFPIJIR5KEoeVySnki+W7g2HY2aiZM6A5Tve9IVXGyR7JGUKIR5HEYUUeLOQDWBdyneGBNS0YlRCipJHEYQX0isLeCxH8dMS4kM/DxZ6GlWTcEiFEwUjiKOXuJKSxan8of0fkUcgXUA1nGfdbCFFARXLW2LZtG0uWLEGr1TJ8+HAGDx5stPy3337jiy++QFEUKlWqxOzZsylbVgrNHodWp2fLyZtsPnGDbN29Qj7fsk682rE2daWQTwhRSGav6IqKimLBggWsXr2azZs3s3btWsLCwgzLU1JSeP/991m+fDlbt26lTp06fPHFF+YOq1RTFIXJPxxlfch1Q9KwUat4rpkfH7/QXJKGEOKxmD1xHDp0iFatWuHu7o6zszPdu3dn586dhuXZ2dnMmDGDcuXKAVCnTh0iIiLMHVapplKp6NiggmG6msaVmX2b8sJT1aT6Wwjx2Mx+qyo6OhqNRmOY9vHx4ezZs4ZpDw8PunbtCkBGRgbLly9n6NChBT6Ow3336l3dHNE8UJ9gbZ71duVwaBTNq3vz/FNVsVFbb8Kw9u/C/aQtckg7PB6zJw69Xm80boOiKHmO45CcnMzYsWOpW7cuffr0KfBxMjO1hr9TkjOIiUkuXMAlTEpGNqsPX6Wbf0Wqet/rGkSjceO/T9dFpVIRH5f6iD2UbhqNm9V8F/IjbZFD2gHUahVej9GVkNl/hvr6+hITE2OYjomJwcfHx2id6OhoBg0aRJ06dfjoo4/MHVKpoCgKR8JieHvtcf74O4qV+0LR3debLSADLQkhzMLsiaNNmzYcPnyY+Ph40tPTCQ4Opn379oblOp2O0aNH06NHD6ZOnSonOxPEpWTy2c7zLP7tIknp2UDO4EqnbsRZODIhhDUw+62qcuXKMWHCBIYNG0Z2djb9+/enUaNGjBo1ivHjxxMZGcmFCxfQ6XTs2rULgIYNG8qVRx4eWsjnbM/L7WvRvKqXBaMTQliLIqnj6NWrF7169TKat2LFCgD8/f25dOlSUYRRokkhnxCiuJCzTTGn1enZfuZW3oV8HWpRt4K7BaMTQlgjSRzFXERiOj8fu86/z71t1CqebVyJ55vLiHxCCMuQxFHM+Xm68GwTP7adCqeaxpVXO9SmireMyCeEsBxJHMVMQloW7s72RvP6NK+Ct5sDHeuWx0ZG5BNCWJjc6ygmUjKyWb73byb+dIy4lAyjZfa2arrUryBJQwhRLEjisLCcEflimPxPIV9Gto5vDoShKEr+GwshhAXIrSoLik/J5Ns/wzhx3bhwz85GTbZOj72tjYUiE0KIh5PEYQF6RWHvxQjWHLlGetYDhXztatK8mrcFoxNCiEeTxFHEIhLSWLX/MpciEo3md65fnoFSyCeEKAHkLFWEfr8QwfcHw3IV8r3SoRb1pJBPCFFCSOIoQl6uDoakoVbBs0386NO8sjzLKMXS01NJSUlAp9Pmv7KZRUer0ev1lg7D4qylHWxsbHF1dcfJyeWJ71sSRxFqXNmTNrV8iEhIk0I+K5Cenkpy8l3c3TXY2dlbvOdnW1s1Wm3pP2HmxxraQVEUsrOzSEjIGdLiSScPSRxmcu7WXQAaVvIwmj+iXU3sbW2kJsMKpKQk4O6uwd7ewdKhCCujUqmwt3fA3V1DYmKsJI7iLjUzZ0S+/Zei8HSxZ86LLXC2v9fMTvbS5NZCp9NiZ2ef/4pCmImdnb1ZbpNKAeAToigKIVdieHvNcfZfigIgPjWLjcdvWDgyYUmWvj0lrJu5vn/y8/cJuJuayTcHchfytazuzbONK1koKiGEMA9JHI9BryjsuxjJT0euGhXyuTvbM7xdTVpKIZ8oIQIDW+Dg4IBanXMTQlEUvL01DBkynKCg5w3r3b59i5Url3Ls2FEyMzPx9fWlZ8/eDBjwEjY2994OjI2NYdWq5Rw5cpDU1FR8fHzo06c//fq9WKC4Zs16jz17drNhwza8vTWG+b/+uo2ff17HqlXfG61/8uRx3ntvMtu37zHMO3LkED/99D2XL4eiKAr169dn1KjXqVu3foFiyUtSUhKzZ8/k5MljuLi4MnLkKKP2ut+ePcGsWrWM2NhY6tVrwMSJ7+DnVxmA69ev8emnswkLC8XFxZUXXxzEgAGDAMjKymL+/Dn88cc+bG1t6d//RYYPfwWAb79dhUbjQ8+evfI8prlI4iikhxXydarny8BW1XGRQj5RwqxY8S3Vq9cEQKfTsWdPMB9+OIOGDRtTtWo1bty4zpgxr/Dss7355pvVeHl58/ffl5g/fzZnz55m9uxPAYiJieaVV4bSo0cQX3+9mrJly3LhwnmmT3+HxMRERo58zaR4kpKSOHToIJ06Pc3mzT/z6qujC/yZtm7dxMqVS5k8eRoBAa3R6XRs3ryB8ePHsHTpV1SvXqPA+7zfvHkf4uzsxNatwVy5cpmJE9+kbt0G1KxZy2i9c+f+4qOP3mfmzDm0atWGX3/dxoQJY/nxx/U4ODgyc+Z7dOv2DF98sYzr168xZsxIateuS5MmzVi+/H9ERkayfv1W7t6N5623xlGjRi0CA9szcOAQXnllKK1bB+Lh4fGQKJ88ecZRCFlaPR9uOWOUNMqVcWRKr0a80qG2JA1R4tnY2NCtWw+cnV24du0KAIsWfcbTT3dj7Ng38fbWoFKpqFu3Hp9+uojTp0/yxx/7AFixYgmNGjVmzJg3cHd3R6VS0aBBQ9555z3i4mJNjmHnzu00adKUvn1fYOvWTWRnZxfoM6Snp7N48UImT55G27btsLW1xcHBgcGDh9KnT39u3LiWa5szZ07RtWu7XP8NGTIg17ppaWkcOLCfkSP/g4ODA/XrN6Rr1+5s27Yp17p//LGXdu06EBjYHltbW3r37oOjoyPHjoUAEB5+E61Wa6gvUattDC9WBAfvYNiwEbi6uuLnV5m+fQewdWvOMRwcHGjbth0bNqwpUNs8LjnDFYK9rZq+Lavw9R9hqFXQs7EffVtIIZ/I369nbrHx+A0ysnX5r/yYHO1s6NuiCj0L8ZwtOzubTZvWo9Vm06CBP5mZmZw4EcKwYUtyrVu2rDtt27bjwIF9tG/fkaNHDzN27H9zrdeyZQAtWwaYHMO2bZv4z3/G4u/fGA8PD/bu/Y1u3XqYvP1ff51Bp9MSENA617IxY97Ic5vGjZuye/cBk/Z/69ZNbGxsqVjxXvtWrlyF/fv35lpXr9fj4OBoNE+lUnHr1k0Ahg0byYoVS1ixYgk6nY4RI0bRoEFDkpKSiI+Po2rVakbH2LhxvWG6U6cuTJ78FqNGjTEp7idBEocJ9HoF9QN1F53qlSc8LpWO9cpTVQr5hIl+PXOrSJIGQEa2jl/P3DI5cYwe/QoqFYZf9gEBrfn886X4+JQjNjYGrVaLp6dXntt6eXlz+XIoAElJiXh4PF4XOn/9dYaUlBRatw4E4Lnn+vHzz+sKlDgSExNwcyuDra15TnPp6Rk4OBjX6Dg4OJKRkZFr3cDA9rz99gR69AjC378xu3b9ys2bN8jKygJArVbx5psTCQp6jtDQS7zzzlvUr9+QGjVybh06Ot5LOo6OjmRm3jtGzZq1SUxMIDz8puGZiblJ4sjH+dt3+eZAGK93qUs1jZthvlql4uV2tR6xpRC59WxcqUivOApytbF06SqqV6/JnTu3mTJlEmXLulO/fgMAypQpi42NDdHRUXmenCIjI/Dyykkqnp5exMfH51pHp9ORmppCmTJl841l69ZNJCYm0KdPz3+21ZKYmMilSxepW7cednZ26HS521Cn0xlu8Xh5eZOUlIRWq82VPJKSknB2ds41/8yZ00yenPtqqVw5X7791vh2UM4JPNNoXmZmBk5Ozrm2b9KkGW+++X/Mm/cRycnJdO/ek4CA1ri6unHp0gV+/nkdGzduB6Bhw0b06tWHX37ZzOTJ0/7ZbyYuLjk/UDMyMnBycjLs29bWlrJlyxITEy2Jw9JyCvmusf9SJAAr94XyQd+m2NrIYyFReD0bVyrUraOiVKFCRebMmc/LLw+ifPkKDB/+Cvb29rRq1YYdO36hefOWRuvfvRvP0aOHeffd94CcK5X9+/fSvXtPo/UOH/6TDz6YxpYtO3F2fnglc0pKCnv3/sbChf8zug30+efz+fnntUyd+j4ajQ/R0VEoimJUq3Dnzm00Gh8AGjTwx87OliNHDhIY2MHoGHPmzMLZ2Zlp0z4wmt+4cRN27txnUjv5+fmh02mJjIzE19cXgJs3b1CtWrVc6yYmJuDv35ifftoI5CS4/v17MWzYSKKiInM9v7G1tcXW1pYyZcri4eHJzZs3DFd7N2/eMLp19e/+/n0jrijIWTAPIVdjeHvtCUPSAIhJzuT23TQLRiVE0fH1Lc/48W/x9dcrCAu7DMAbb7zFoUN/snjxQmJjY9FqtVy4cI6JE9+kUaMmtG/fCYCXX36VM2dOsmzZlyQlJaLT6Th58jiffDKbQYOGPTJpAOzatZ2KFf1o1KgJXl7ehv+Cgp5jz55gEhISqF+/Ic7OzixZ8gVpaWno9XouXbrAmjU/GG5nOTg48J//jGPevI85dOhPtFotaWmprFq1nOPHQxg0aOhjtZGzswuBge1ZtmwxGRkZXLx4nt27d9G1a+7badevX2PcuNeIiLhDZmYGK1cuxd3dnfr1G+Lv35js7Cy++WYlOp2OsLDLbN26kc6duwHQrVsPvvpqOUlJiYSH32TjxnV07/6sYd9ZWVkkJycbEmZRUCmlZIzSJTvOsfmfKu0R7WrSpUGFAu/jbmrOiHzHr+Uu5BvWtgYeLiWnzyGNxo2YmGRLh2FxlmyHyMgb+PpWscix8/Kozv0CA1vw3XdrDK/j/mvChLEkJSWxfPk32NjYEBFxh5Url3L8+FHS0tLw9S1Pjx5BDBgwyOi2T3j4TZYv/x+nT58kIyMDX19f+vYdQJ8+/fONc/jwl3j66e4MHfqy0Xy9Xk+/fkH07fsCQ4eOIDz8Jv/73+ecPXuazMxMNBofnn++HwMGDDK6CgkO3sn69asJDw9HrVbToEFDXn11DHXq1C1A6+UtKSmRTz6ZzfHjITg5OTFy5GsEBT0HwHfffcWZM6eZP38RAKtXf8+6davJyEinceOmTJz4ruFk/9dfZ/jyy8+5du0K7u4evPTSEJ5/PqetMjMzWLToM/bv34tKpeKFFwYybNhIQwxnz55m1qwZrF+/Jc8Y8/oeqtUqvLwK/2xWEgels5BPEkcOSRz3WEOvsKYobe2wePFCHB0dH1rnYo7EYfXPOKIS01mxL1QK+YQQJU5GRgYHDuxjxYpvi/S4Vn9WzNTquByVZJguV8aRVzrUpn5FGZFPCHNYv34Ny5d/+dDl33+/3vCwWTzaTz99z9ChI0x6U+1JsvrEUdnLlaAmldh2KlwK+YQoAi+8MJAXXhho6TBKhREjRlnkuFaVODKzddyITaF2eePs/FyzKjxVXSMj8gkhhAmsJnGcv32XVfsvk5SezZwBzfF2u1eJaW+rlqQhzECFouhRqeStd2EZiqIHnvyYHKX+G52amc2KfaHM3vYX0UkZZGTr+PqPy5SSl8lEMWZv70hCQixabbZ830SRUhQFrTabhIRY7O0d89+ggEr1FUfI1Ri+/fMKiWlZhnnO9ja0qF7yXq8VJY+Hh4aUlETi46PQ64umf6pHUavVht5XrZm1tINabYOTkyuurk/+wXmpTBxpmVoW7jqfu5CvmjfDAktWIZ8ouVQqFW5u7ri5FY839KS2J4e0w+MrkltV27Zto2fPnnTr1o0ff/wx1/KLFy/St29funfvztSpU9FqH29w9U0nbholjbLO9rzZrT5vdq8vSUMIIR6T2RNHVFQUCxYsYPXq1WzevJm1a9cSFhZmtM6kSZOYPn06u3btQlEU1q1b91jHzNLduwztWNeXeS+2oKXcnhJCiCfC7Inj0KFDtGrVCnd3d5ydnenevTs7d+40LL99+zYZGRk0adIEgL59+xotN9WD7w34/DMi36sdZUQ+IYR4ksx+Ro2OjkajuTfIvI+PD2fPnn3oco1GQ1RUVIGP07qeL5uP30Clgh5NK/OfbvVxsLPuQj7NfeOHWDNph3ukLXJIOzwesycOvV5v1FPlg/3n57fcVJXLOrF4eCt8NK5kp2WTkpROyuOFXqJ5ebkSF2fNLZBD2uEeaYsc0g45nRx6eDy6e/tHMXvi8PX15fjx44bpmJgYfHx8jJbHxMQYpmNjY42Wm8rDw+VeQzjJA3DgsXq/LE2kHe6Rtsgh7fB4zP6Mo02bNhw+fJj4+HjS09MJDg6mffv2huUVK1bEwcGBEydOALBlyxaj5UIIIYqXIhmPY9u2bSxbtozs7Gz69+/PqFGjGDVqFOPHj8ff359Lly4xbdo0UlJSaNCgAbNnz8be3t7cYQkhhCiEUjOQkxBCiKJR6vuqEkII8WRJ4hBCCFEgkjiEEEIUiCQOIYQQBSKJQwghRIFI4hBCCFEgJTZxFHVX7cVVfu3w22+/8dxzz9G7d29ef/11EhMTLRCl+eXXDv/at28fnTt3LsLIil5+bXH16lWGDh1K7969eeWVV6z2O3H+/Hn69etH7969+c9//kNSUpIFoiwaKSkpBAUFcevWrVzLCnWuVEqgyMhIpVOnTsrdu3eV1NRUpVevXsrly5eN1nn22WeVU6dOKYqiKO+++67y448/WiJUs8qvHZKTk5W2bdsqkZGRiqIoysKFC5VZs2ZZKlyzMeX7oCiKEhMTozzzzDNKp06dLBBl0civLfR6vdKtWzdl//79iqIoyieffKLMmzfPUuGajSnfiZdeeknZt2+foiiKMnv2bOWzzz6zRKhmd/r0aSUoKEhp0KCBEh4enmt5Yc6VJfKKo6i6ai/u8muH7OxsZsyYQbly5QCoU6cOERERlgrXbPJrh39NmzaNcePGWSDCopNfW5w/fx5nZ2dDtz6jR49m8ODBlgrXbEz5Tuj1elJTUwFIT0/H0fHJj81dHKxbt44ZM2bk2QdgYc+VJTJx5NVV+/1dsT+prtqLu/zawcPDg65duwKQkZHB8uXLefrpp4s8TnPLrx0AvvvuO+rXr0/jxo2LOrwilV9b3Lx5E29vb6ZMmUKfPn2YMWMGzs7OlgjVrEz5TrzzzjtMmzaNwMBADh06xMCBA4s6zCLx0Ucf0aJFizyXFfZcWSITR1F11V7cmfo5k5OTee2116hbty59+vQpyhCLRH7tEBoaSnBwMK+//rolwitS+bWFVqslJCSEl156iU2bNuHn58ecOXMsEapZ5dcOGRkZTJ06lW+++YY///yTQYMGMXnyZEuEalGFPVeWyMTxYFfs5uqqvbjLrx0g5xfFoEGDqFOnDh999FFRh1gk8muHnTt3EhMTQ79+/XjttdcMbVIa5dcWGo2GKlWq4O/vD0BQUJDRwGqlRX7tEBoaioODA40aNQLgxRdfJCQkpMjjtLTCnitLZOKQrtpz5NcOOp2O0aNH06NHD6ZOnVoqr7og/3YYP348u3btYsuWLSxfvhwfHx9Wr15twYjNJ7+2aNq0KfHx8Vy6dAmA33//nQYNGlgqXLPJrx2qVKlCZGQkV69eBWDPnj2GZGpNCn2ufHLP7ovW1q1blWeffVbp1q2bsnz5ckVRFOXVV19Vzp49qyiKoly8eFHp16+f0r17d+Wtt95SMjMzLRmu2TyqHYKDg5U6deoovXv3Nvw3ZcoUC0dsHvl9H/4VHh5eqt+qUpT82+L06dNKv379lJ49eyojR45UYmNjLRmu2eTXDvv27VN69eqlBAUFKcOHD1du3rxpyXDNrlOnToa3qh73XCndqgshhCiQEnmrSgghhOVI4hBCCFEgkjiEEEIUiCQOIYQQBSKJQwghRIFI4hCiiKWnpxMbG2vpMAzCw8MtHYIoYSRxiGJp1apVNG/enLZt25Kdnf3Quw6wxgAABzlJREFU9Y4ePUpAQEARRlY4TZs25cqVKwAMHjyYv/76C4CtW7datJPBCxcu8NJLL1ns+KJksrV0AELkZe3atbz77rv079/f0qE8EadOnTL8nZCQYPi7d+/e9O7d2xIhATn9mD0qMQuRF7niEMVO9+7duXnzJjNnzmTmzJlkZGTw/vvv07VrV5o0aUK3bt347bffcm2XlZXFu+++S0BAAIGBgYwfP567d+8COZ3affjhh7Rr147AwEDmzp1LVlZWnsfv3LkzixcvpmPHjjRv3pz33nuPzMxMAFJTU/nggw9o27Ytbdu2ZerUqSQnJwNw584dhg0bRosWLXj66aeZN28e/9bX1qlTh9DQUMaOHcudO3d48803+e6779i4cSN9+/YlJSWFxo0bc/nyZUMcGzZsYMCAAYZ9jx49moCAALp168bPP/+cZ+y3bt2iefPmvPPOO7Ro0YItW7YQHh7O6NGj6dChA40aNWLgwIFcuXKFuLg4Ro0aRUJCAk2bNuXu3bsFaidhvSRxiGJn165dVKhQgc8//5zp06ezatUqrly5wsaNGzlx4gR9+/Zl1qxZubbbsmULV65cYe/evezevZu0tDS+++47AObOncvVq1fZunUrW7du5dy5cyxduvShMWzfvp0ff/yRXbt2ceHCBb744gsApk+fztWrV9m2bRu//vorsbGxTJ8+HYAFCxZQu3ZtQkJC+OGHH9i+fTuHDx822u+XX35p+GzDhg0zzHd1daVz585s377dMO+XX37hueeeM/Q5VqtWLQ4cOMCiRYtYsGABR44cyTP2lJQUKlasyKFDh+jWrRvTpk2jevXq7NmzhyNHjuDh4cHSpUvx8vJixYoVuLu7c+rUKTw8PArcTsI6SeIQxd7gwYNZtGgRzs7ORERE4OLikueYAW5ubty4cYNNmzZx9+5dli9fzptvvomiKGzcuJGJEyfi4eGBp6cnb7zxBuvWrXvoMceMGUPFihXx9vZmzJgxbN++nYyMDHbt2sWkSZPw9PSkbNmyTJ48mR07dpCeno6bmxvHjh1j165dODs7s3fvXtq0aWPy53z++efZsWMHkNOb68mTJ+nRowd//fUXERERTJgwAXt7e+rWrcvAgQNZv379Q/fVq1cv7O3tcXJyYs6cOYwfPx6dTsedO3dwd3fPs/0K007COskzDlHsJScn88EHH3D27Fn8/Pzw8/Mjry7WnnnmGeLj49m4cSMfffQRtWvXZubMmVSsWJGMjAyGDh1q6CFYURSys7PJzMzEwcEh176qVKli+LtcuXLExMSQlJREdnY2FSpUMCyrWLEiiqIQHR3NpEmTWLRoEZ999hn/93//R/v27fnwww/x9vY26XMGBgaSkpLCuXPnOH78OG3btsXT05MjR46QkpLCU089ZVhXp9M9slfb+4959epVPvnkE6KioqhZsyYqlSrP9ouPjy9wOwnrJIlDFHszZsygRo0aLF26FFtbW44dO2b4ZX6/69ev06pVKwYNGsTdu3f58ssvefvtt9m+fTt2dnZs3rwZPz8/ANLS0oiNjX3oyTA6Otrw9507dyhfvjze3t7Y29tz+/ZtPD09gZxnCmq1Gg8PD/7++29GjRrF5MmT+f/27tileSAO4/gXg6VaqA2Crg6C4qBghzoUokVQKCnVRQcRnRx0c1FqxboUSkAUUSmugruLi9BVEfEvEOqilCooQTA6vIO0vLwqvnmXV/D5rMnlLr/hHu5IuOvrazKZDFtbW6yvr//VexqGQTKZ5Pj4mPPzc2ZmZoC30+va29splUr1e6vV6oeTf01t4vc8j4WFBfL5PKOjowBsb29zenr6rk0kEvFdJ/mZtFUl357rugSDQQzD4Obmhs3NTYB3XwOdnJywuLhItVqlpaWFUChEJBLBMAxs28ZxHB4fH3l6emJ1dZWlpaVP+ywWi9zd3VGpVNjb2yOdTtPQ0EAqlcJxHO7v73l4eKBQKGBZFuFwmN3dXRzH4fn5mdbWVgzDwDTNd89ubGzEdd0P+02n0xwdHXF1dUUikQCgr6+PYDDI/v4+Ly8v3N7eMjs7y8HBwZe1q60WmpqaALi8vOTw8LBeu0AggOd5eJ73T3WSn0nBId/e8vIypVKJ/v5+pqamsCyL5ubm+n8RNdPT0/T29mLbNtFolIuLC/L5PACZTAbTNEkmk1iWheu6bGxsfNpnd3c3ExMT2LbNwMAAc3Nz9bF0dHSQSqUYHh7GNE0KhQIAa2trVCoV4vE4g4ODtLW11dv9bmxsjGw2y87OzrtrPT09hMNhRkZGCAQCwFvQFItFzs7OiMfjjI+PE4vFmJ+f/7J2oVCIXC7HysoK0WiUXC7H5OQk5XKZ19dXurq66OzsJBaLUS6XfddJfiadxyHyh0QiQTabZWho6H8PReRb0opDRER8UXCIiIgv2qoSERFftOIQERFfFBwiIuKLgkNERHxRcIiIiC8KDhER8eUXT8sMW8PJICAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a28040160>"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAD7CAYAAADO1FugAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deViU9f7/8SfCAKmUUiroyUxN3CojN3DBTAVFBCH3iCwjjfR866ihpoKKa/7sYKWRnCwjIRMlRcK0tELDJRVMLZVMsWQTTEVZZu7fHxzmiIIwMDfM4PtxXV2XzHLPa6jp7X3P537dFoqiKAghhBAmpEFdBxBCCCFuJ8NJCCGEyZHhJIQQwuTIcBJCCGFyZDgJIYQwOTKchBBCmBwZTkIIIUyOVV0HqC9yc6+j05n+KWMPPtiYnJxrdR2jSiSreswpr2RVR11nbdDAgqZNG1V4vwwnI9HpFLMYToDZ5ATJqiZzyitZ1WHKWS2M2RARHBxMz5498fX1rfJzNm7cCMC4ceOMkuHQoUMsXLiwzG2//fYbS5cuxdvbG4Bff/2VN998k/j4eACSk5OZPHkyrVu3LvO82NhYLC0tjZJLCCHqm5sFxVz9+0a1ntuggQUPPti4wvvrfM/JWEOpVPfu3YmLi9P/vH79erZs2YKHhwcAW7duZeXKlWg0mjLP69q1Kxs2bKj26768aCeZudX7lySEEOZo20pvrqq07RoNJ0VRWLp0KXv27KF58+ZotVp69uzJ1q1b+eSTT9DpdHTp0oX58+cTExPDH3/8wdy5cwFYunQpDg4OXL1a8tamTp3Ktm3bWLNmDRYWFjz++OMsXLiQwsJCFixYwOnTp9FqtbzyyisMHz68SvkOHTrEmjVr2LRpEzY2Nly9epXdu3fz//7f/+Ott96qyVsXQggBNGtmp8p2azScEhMTOXHiBNu3b+fq1auMGDGCGzduEB8fT3R0NDY2NqxcuZLIyEjGjh2Lj48Ps2fPpkGDBuzcuZOYmBiio6MByMjIYMmSJcTGxuLg4MCMGTPYu3cvR48epUuXLixbtoxr164xduxYnnzySR5++OG7ZsvJyeHNN99k0aJF+sN1dnZ2rF69mvT09Dsef/z4cf1hP4CXX36ZESNG1OTXI4QQ9V5WVvX2nVQ9rHfgwAGGDBmCRqPB3t6e/v37oygKf/zxB6NHjwagqKiIzp07Y29vT8eOHUlOTkaj0fDoo4/SrFkz/baOHDmCs7MzDg4OAKxYsQKADz74gJs3b7J582YA8vPzOX369F2Hk06nY/r06Xh6ejJ48OAqvZeaHtYTQghhPDUaThYWFty6nsLKygqtVsvQoUN5++23Abh+/TparRYAb29vduzYgUajwcvLq2wQKyssLCz0P1++fBkoGTQrVqygS5cuAGRnZ/PAAw/cNdd7771HYWEh//rXv2ry9gwS+faQWnstIYQwBTcLilXbdo2Gk4uLi/6Q3Y0bN/jhhx9o164d33zzDVOmTMHe3p6QkBBat27N1KlTefbZZwkPD0dRFKZPn15mW48//jihoaFkZWXRrFkzFi9eTK9evejduzcbN25k0aJFZGZm4uPjQ3R09B0r60olJSWxadMmNm/ejJVV7a33yMm5ZtLLMks1a2ZX7d3w2iZZ1WNOeSWrOkw9a43+7z1o0CBSU1MZPnw4Dz30EO3atcPOzo7XX3+dgIAAdDodnTp1IjAwEABbW1ucnZ0pLCykUaOyJ1+1aNGCOXPm8PLLL6PT6ejWrRu+vr7cuHGDkJAQhg8fjlarZcaMGRUOJoAPP/xQv3DiVmPHjjX6ykAhhBDqMOp5Tvcy2XMyPsmqHnPKK1nVUddZTf48p+oo70TbUhEREbRo0aKWEwkhhDCmau85+fv761e3zZo1i9dff51WrVqRmZnJ22+/TWZmJra2trzzzjv84x//MGro8iQlJREREcEnn3xS7v0DBw7k008/rZUsQghRH9SkAaIyqu05HThwQP/n5ORkgoKCAJg5cybu7u6MGzeOjRs38s477/Duu+9W92UqpdPpWL9+PR9++CEdOnRQ7XUqIw0RQoj6Rs0GiMpUOpyKi4sJCQnh9OnTZGdn4+TkhL29PQCjRo1i8ODBZGZmEhgYyKeffsqpU6f4+OOPAfDz88PFxeWu2w8ODsbGxobU1FSuX7/OlClT8PHxYfXq1Rw9epS//vqL559/nj59+hASEkJeXh62trbMnTuXzp07c/bsWc6ePcvChQvLnKeUl5fHjBkzuHTpEu3ataOgoACAa9euMXv2bDIyMsjMzMTFxYWwsDBmzpxJjx499Odn+fv7M336dJ588snq/WaFEEJUW6XD6ciRI2g0GmJiYtDpdAQEBODt7c2mTZvYtGkTANHR0URERHDx4kVatmzJ0qVLOXToEM2aNdPXFd3NhQsXiImJIScnB19fX/r06QNAYWEhO3bsAEpW282bN4/OnTtz5swZgoKCSExM5LHHHiMsLIzk5OQy2wwPD6dz58589NFHHDx4kISEBAD27NlDp06dCA8Pp7CwEE9PT3755Rf8/PxYvXo1o0eP5uLFi1y+fFkGkxDinqdWPVFlKh1OPXr0oEmTJkRFRZGWlsa5c+fIz88v97HFxcWcOHGCqVOnMmvWLDZt2kRwcHClzQu+vr5oNBocHBxwdnbm8OHDADzxxBNAyYm8x48fZ9asWfrn5Ofnk5ubS9OmTcvd5oEDB1i5cqX+PZQ2SgwfPpyUlBTWr19PWloaeXl55Ofn06tXL+bOnUt6ejpxcXFlqoyEEOJepdaKvhp/57R7927Cw8N54YUX8PX1JTc3l4rWUDRr1oxGjRrxzDPPACWDYNGiRZWGvPWyFDqdTn/yrK2trf42a2vrMm3jly5dokmTJhVu8/b2itLX2LBhA4mJiYwePRpXV1d+++03FEXBwsICHx8f4uPjSUhIIDIystLcQggh1FHpcNq/fz9Dhw7Fz8+PCxcukJycjIuLC5aWlhQXF2NlZYWlpSVarZZHHnkEBwcH9u7di5ubG999952+duhuEhIS8PDw4M8//yQlJYWwsDBOnjypv9/Ozo42bdro92iSkpKYN28eu3btqnCbLi4uxMXF0bFjR1JSUjh//jxQsqpvzJgxeHl5kZqayqlTp9DpdEDJHtz48eNp3769wcvRpb5ICFHfqFlPVJlKh9OoUaOYPn068fHxaDQanJ2dSU9P59lnn8Xb25vY2FgGDBhAYGAg69atY/Xq1cyfP58VK1bQuHFjli5dWmmImzdv4ufnp788RnmH6lasWEFISAjr1q1Do9GwatWqMl18t5s2bRrBwcF4enrStm1b/WG9gIAAQkJCiIiIoHHjxjz11FP6lnJHR0ccHR0ZOXJkpZlvJyfhGp9kVY855ZWs6jD1rHXeEFGdq+eqQVEUMjMz8ff3Z/v27VhbWxv0fBlOxidZ1WNOeSWrOuo6q0k0RCxbtox9+/bdcXvXrl1r4+WrJDExkZCQEEJCQgweTEIIIYyr2sOpooaIUu+++y6WlpZMnTpV1avOZmZmMmvWLLKzs2nQoAEzZ84s99yqyhoiPDw89JdyF0IIUbeM3hBx9epVlixZQnx8PJMmTap5wkosX76cgQMHMmHCBNLS0vD39+f7778vswKwNtxt99TU1NV5C9UhWdVjTnklqzpKs6pZU1RdRm2IiIqKYu/evbRp04aJEydWKUBNGyIGDx5M7969AXjkkUcoKCggPz9ff3mN2mqIkPoiIYS5qsuaoooYtSGiadOm+Pj4ALB69eoqh6hJQ4S7u7t+O5GRkXTq1Ak7OzsWLFggDRFCCGGmjNoQUV3GaIhYv349MTExfPbZZ4A0RAghhCFM7XCkURsiqqumDRHLly9n7969REVF4eDgAEhDhBBCGKK2l5VXtpS8QWUbuLUh4v777yc5ORmtVqtviAD0DRHVlZCQgKIoXLx4kZSUFJ5++uky99/aEAElLQ8TJkwASvaYkpOT2bhxo34wwf8aIoByGyJGjBhBQUHBHQ0R0dHRODo6ygULhRCiDhm9IaL08JkhqtsQAfD+++/TuHFj/P399Y+NiIio9YYIqS8SQpiruqwpqog0RPyXNESYHsmqHnPKK1nVUddZpSGiiqQhQgghTEetDCc1GyKMRRoihBDCdFS6IMKcfPHFF/Tr149ly5YZ/Nz33nsPT09PPD09Wb58uQrphBBCVFWt7DnVlu3bt7NkyRL69u1r0PP27dvHjz/+yJYtW7CwsGDSpEl88803DB48uMrbkPoidUhW9ZhTXsl6d6ZYP1RTZjOckpOTWbt2LRqNhvT0dAYOHEjDhg31FxwcOHAgqamphIaGMmfOHNauXUt0dDQAsbGxHDt2jNDQ0HK33axZM4KDg/XfNbVr144///zToHxSXySEqCumWD9UU2Z1WK90wGzevJmoqCjs7e2JjY3FyckJOzs7unbtyqJFi3BzcyMrK0t/btPWrVvvuhrwscceo1u3bgCcO3eOhIQE3NzcauU9CSGEuJPZ7DkBdOjQAUdHRwCaNm2qvzRGy5Yt+fvvv/WPs7CwYOTIkXz11Vf4+vqSk5NTpZ6806dP8+qrrzJz5kzatGmjynsQQgg1VOdwoikfLjWr4aTRaMr8fLfLYowcOZJJkyZhbW1dpZ68w4cPM23aNGbPno2np2eNswohRG0y9JwlOc+pjrRq1QoHBweio6PZuHHjXR/7119/ERQUxKpVq8q9UKEQQojaVW+HE8CwYcPYuXNnpT15kZGRFBQUsHTpUv1tY8eOZdy4cVV+LakvEkLUFVOsH6qpOq8vUktxcTEzZ87Ew8ODIUPUHxxSX2R8klU95pRXsqqjrrPek4f1FEWhX79+uLq6MmjQIAAOHTrEwoULy318RESEtJALIYQJqZfDycLCgv3795e5rXv37mWuByWEEMJ0Vfs8p1svUTFr1iwuXrxY5v53333XoEu111RSUhIBAQEV3j9w4ED9pTGEEEKYtmrvOR04cED/5+TkZIKCggC4evUqS5YsIT4+nkmTJtU8YSV0Oh3r16/nww8/pEOHDqq/XkWkvkgdklU95pT3Xs9aH+uJKlPpcCouLiYkJITTp0+TnZ2Nk5MT9vb2QMmFCAcPHkxmZiaBgYFERUWxd+9e2rRpw8SJE6sUIDg4GBsbG1JTU7l+/TpTpkzBx8eH1atXc/ToUf766y+ef/55+vTpQ0hICHl5edja2jJ37lw6d+7M2bNnOXv2LAsXLmTDhg367ebl5TFjxgwuXbpEu3btKCgoAODatWvMnj2bjIwMMjMzcXFxISwsjJkzZ9KjRw9Gjx4NlOwZTp8+vUon74LUFwkh1FMf64kqU+lhvSNHjqDRaIiJieGbb77h6tWr9OvXD4BNmzYRGBhI8+bNiYiIoGnTpvj4+BAYGHjXE2Rvd+HCBWJiYvjkk09Yvnw5WVlZABQWFrJjxw7Gjx/PW2+9xYwZM9iyZQsLFy7kjTfeAEqqh8LCwnjggQfKbDM8PJzOnTuzbds2JkyYQHZ2NgB79uyhU6dOxMTEkJiYyMGDB/nll1/w8/PTfyd18eJFLl++XOXBJIQQwrgq3XPq0aMHTZo0ISoqirS0NM6dO0d+fr5RQ/j6+qLRaHBwcMDZ2ZnDhw8D8MQTTwBw/fp1jh8/zqxZs/TPyc/PJzc3t9xLukPJYceVK1fq30PpZdqHDx9OSkoK69evJy0tjby8PPLz8+nVqxdz584lPT2duLi4KrVKCCFEbVHjcKEpHy6tdDjt3r2b8PBwXnjhBXx9fcnNzcXYp0bdupel0+mwsiqJZWtrq7/N2tq6zGq7S5cu0aRJkwq3aWFhUSZn6Wts2LCBxMRERo8ejaurK7/99huKomBhYYGPjw/x8fEkJCQQGRlp1PcohBA1Yexzkkz9PKdKD+vt37+foUOH4ufnx/33309ycjJarRZLS0uKi0vOSra0tESr1VY7ZEJCAoqicPHiRVJSUnj66afL3G9nZ0ebNm30wykpKYkJEybcdZsuLi76x6ekpOgbypOSkhgzZgwjRoygoKCAU6dOodPpgJI9uOjoaBwdHeW8JyGEqEOV7jmNGjWK6dOnEx8fj0ajwdnZmfT0dJ599lm8vb2JjY1lwIABBAYGsm7dOv3hM0PcvHkTPz8/CgsLWbBgQbmH6lasWEFISAjr1q1Do9GwatUqLCwsKtzmtGnTCA4OxtPTk7Zt2+pzBQQEEBISQkREBI0bN+app57SLzF3dHTE0dGRkSNHGvwepL5ICKGW+lhPVJk6ry8KDg6mZ8+ed73eUm1QFIXMzEz8/f3Zvn27/sKDVSX1RcYnWdVjTnklqzrqOqtJ1BctW7aMffv23XF7165da+PlqyQxMZGQkBBCQkIMHkxCCCGMq1aG01tvvVUbL1MjHh4eeHh41HUMIYQQ1IP6oszMTF588UVGjBjBqFGjOHnyZLmPk/oiIYQwH2ZfX7Rq1Src3d0ZN24c33//PaGhoURHR6v+ureT+iJ1SFb1mFNec896L9YP1ZTZ1xeFhYXpt5Wens79998PSH2REMJ03Iv1QzVV6XC6tb5Ip9MREBCAt7c3mzZtYtOmTQBER0eXqS8CDDqkV1pflJOTg6+vL3369AH+V18EJVemnTdvHp07d+bMmTMEBQWRmJhIgwYlRyY9PDy4ePEiH3zwAfC/+qKPPvqIgwcPkpCQAPyvvig8PJzCwkI8PT319UWrV69m9OjRUl8khBB1rN7UF3399decPHmSl156iYSEBKkvEkKYFFM8NGmKmUqZfX3Rnj176NGjB40aNaJTp060bNmSCxcuSH2REMKkmNr5T2Z/ntOt9UUXLlwgOTkZFxcXfX2RlZWVUeqLPDw8+PPPP0lJSSEsLKzMqrtb64u8vb1JSkpi3rx57Nq1iy1btvD7778zceJEzpw5Q3Z2Nm3bttXXF3Xs2LHc+iIvLy9SU1PvqC8aP3487du3N7i+SBoihBAVuRcbHmrK7OuLZs+ezezZs9myZQs2NjasXLmSRo0a1Xp9kTREGJ9kVY855ZWs9yapL/ovqS8yPZJVPeaUV7Kqo66zSn1RFUl9kRBCmA6pL/ovqS8SQgjTUe36IiGEEEItdf6dU02Ufl+Vnp5ObGwsAQEBVW6muN3hw4dZsmQJRUVFNGnShMWLF9OqVSsjJxZC1ISp1wDV9fc4hqjrrCbxnZPa4uLi+Pjjj3n00UervY0ZM2bwwQcf0LFjR7788ksWLVrEmjVrqvx8qS8SQn1SA3TvMKvDeoqisGTJEtzd3fH39+f8+fPMnz+fjIwMgoKCmDx5Mp9//jkAMTExDB06FICioiLc3NwoKioqd7uFhYX885//pGPHjgA4OTnx119/1c6bEkIIcQez2nNKTEzkxIkTbN++natXrzJixAhCQ0N57733iIiIIC0tjS+//JLx48fz008/ceXKFbKzszlz5gxPPfUUGo2m3O1aW1vr64p0Oh3vvfcegwYNqs23JoSoIlOu3AHTz3crU85qVsPpwIEDDBkyBI1Gg729Pf379y9zf2k/nlarJS0tjWHDhnHw4EFSU1MZMGBApdsvLCwkODiY4uJiXn31VZXehRCiJkz5O526/h7HEHWdtV5953R7X15pB18pGxsbOnXqxLZt22jbti29evVi//79HD58uNJrS5VerqNJkyasWbOmwr2sikh9kRDqkxqge4dZDScXFxciIyMZO3YsN27c4IcffqBbt25lHuPm5sb777/PSy+9RM+ePVm0aBEtW7bUX4OqIjNmzOCRRx4hNDRUfxkOQ0hDhPFJVvWYU15zyiqMx6yG06BBg0hNTWX48OE89NBDtGvX7o7HDBgwgJCQEHr27MkDDzzAgw8+WOkhvRMnTrB7927at2+v79Vr3rw5H330kRpvQwghRCXM+jwnUyJ7TsYnWdVjTnklqzrqOmu9+s6ppvz9/fn777/vuH3s2LGMGzeuDhIJIYQozz01nDZs2FDXEYQQQlSBWQ8nY9YXHTp0iMWLF1NUVESrVq1YtmwZDzzwQJWff7fdU1Njyuc23E6yqscU8pp6HZGoO2Y9nEoZo75o1qxZrFmzhvbt2/POO+8QGRnJm2++WeXnS32REIaTOiJREbMaToqisHTpUvbs2UPz5s3RarXEx8ejKApBQUG0bt2a/v37M378eGJiYli/fj0JCQkUFRUxaNAgdu3aVeH5Szt27ECj0VBUVERGRgZOTk61/O6EuDdVZQ/OFPbyqkqyGodZDSe16osANBoNv/76KxMnTsTKysqgvSYhRPVVtmKsrleVGUKyVl1lq/XMqvi1KvVFx44du6O+6Pvvv69SfZGTkxP79u3jtdde44033lDpXQghhKiMWe05qVVfVFBQwA8//KAvex0xYgTLli0zKJvUFwlhOKkjEhUxq+GkVn2RlZUVoaGhODg40LVrVxISEnB2djYom5yEa3ySVT3mllfce8xqOKlVX2RpacmqVauYN28eWq2WFi1aEBYWptK7EEIIURmpLzIS2XMyPsmqHnPKK1nVUddZpb7oFlJfJIQQ5uGeGk5SXySEEOahRsOptD7I19e3ys/ZuHEjgNH2VCZOnMiECRP0K+2WLVtGdHQ0ycnJWFtbA9C3b1+io6P5xz/+AcC7776LpaUlU6dO1b+Pn376qUxd0YABAwxaTi71ReqQrOoxdl6pIhLGVOt7TsY+fNa7d28OHz6sH0779u2jW7duHD58GBcXF/744w8aNmzIP/7xD65evcqSJUuIj4+/Y2n5tGnTDBqyt5P6InGvkyoiYUwGDafy6oN69uzJ1q1b+eSTT9DpdHTp0oX58+cTExPDH3/8wdy5cwFYunQpDg4OXL1a8p/v1KlT2bZtG2vWrMHCwoLHH3+chQsXUlhYyIIFCzh9+jRarZZXXnmF4cOHV5jJxcWFxYsXA5CRkYG1tTXu7u78+OOPuLi4cOjQIfr06QPA7t27adOmTbXLYYUQd6fW3qM57ZVKVuMwaDiVVx9048YN4uPjiY6OxsbGhpUrV+rPRfLx8WH27Nk0aNCAnTt3EhMTQ3R0NFAySJYsWUJsbCwODg7MmDGDvXv3cvToUbp06cKyZcu4du0aY8eO5cknn+Thhx8uN1OXLl04f/48BQUF/Pjjj/Tp04c+ffrw+uuvM2PGDA4dOsSzzz4LgI+PDwCrV6++Yzvh4eF88skn+p+joqJo3Nh8DtUJYQrUWP1V16vKDCFZq86oq/XKqw9SFIU//viD0aNHA1BUVETnzp2xt7enY8eOJCcno9FoePTRR2nWrJl+W0eOHMHZ2RkHBwcAVqxYAcAHH3zAzZs32bx5MwD5+fmcPn26wuFkaWnJk08+SWpqKj/++CMTJkzg4Ycf5ubNm1y5coUjR44wZ86cSt9bTQ/rCSGEMB6DhlN59UFarZahQ4fy9ttvA3D9+nW0Wi0A3t7e+rZvLy+vsi9sZYWFhYX+58uXLwOg0+lYsWIFXbp0ASA7O7vS6yr17t2bn3/+mZSUFP2Qc3FxYffu3TRt2rRW9oCkvkjc66SKSBiTQcOpvPqgdu3a8c033zBlyhTs7e0JCQmhdevWTJ06lWeffZbw8HAURWH69OlltvX4448TGhpKVlYWzZo1Y/HixfTq1YvevXuzceNGFi1aRGZmJj4+PkRHR9O6deu75nrjjTfo0KGDvm+vT58+hIeH4+7uXo1fi+HkJFzjk6zqMbe84t5j0HAqrz7Izs6O119/nYCAAHQ6HZ06dSIwMBAAW1tbnJ2dKSwspFGjRmW21aJFC+bMmcPLL7+MTqejW7du+Pr6cuPGDUJCQhg+fDharZYZM2bcdTABdOjQgby8PMaPH6+/rXfv3vzf//0frq6uhrxFIYQQJkDqi4xE9pyMT7Kqx5zySlZ11HXWelFfdOjQIRYuXFjufREREbRo0QKAL774gtWrVzN8+HDeeustg17j3//+N4mJiVhYWPDcc8/JcnMhhKhDZjGcunfvTlxcXKWP2759O0uWLKFv374Gbf/AgQP89NNPfPXVVxQXFzNs2DDc3Nxo27ZtlbchDRHqkKzqKS+vtDwIU2EWwwkgOTmZtWvXotFoSE9PZ+DAgTRs2JBdu3YBMHDgQFJTUwkNDWXOnDmsXbtWf05VbGwsx44dIzQ0tNxt9+zZk08//RQrKysyMjLQarU0bNjQoHzSECHqA2l5EKbCrC7TXjpgNm/eTFRUFPb29sTGxuLk5ISdnR1du3Zl0aJFuLm5kZWVxfnz5wHYunVrpecwaTQawsPD8fT0xMXFRX+oUAghRO0zmz0nKFmV5+joCEDTpk1xcXEBoGXLlmUuhWFhYcHIkSP56quv8PX1JScnhyeffLLS7U+bNo1XXnmFyZMn88UXXzBmzBh13ogQJswUD0+aYqaKSFbjMKvhpNFoyvxsaWlZ4WNHjhzJpEmTsLa2xtvb+67bPXv2LIWFhXTq1In77ruPIUOG8OuvvxolsxDmxtRWm9X1qjJDSNaqq2y1nlkd1jNEq1atcHBwIDo6utLhlJ6ezttvv01hYSGFhYXs3r2bp59+upaSCiGEuJ1Z7TkZatiwYezcubPS74/c3NxISUnBx8cHS0tLhgwZgqenp0GvJfVFoj6QCiJhKurtSbjFxcXMnDkTDw8PhgxRf3DISbjGJ1nVY055Jas66jprvTgJ11CKotCvXz9cXV31FyGs6om8Qggh6l69HE4WFhbs37+/zG1VPZFXCCFE3atXCyK++OIL+vXrx7Jlywx+blxcHJ6ennh6elbr+UIIIYynXu05Vbe+6MaNG4SFhfH1119z//33M27cOPbt22dQo7nUF6lDsladVA+J+sRshpOa9UVarRadTseNGzdo2LAhxcXF2NjYGJRP6otEXZPqIVGfmNVhPbXqixo3bsw///lPhg4dipubG61atcLZ2bm23pYQQojbmM2eE6hXX3Tq1Ck2b97Md999h52dHdOnTycyMpJJkyap+4aEMDJDDi3W9WFIQ0hWdZhyVrMaTmrVF/3444+4uLjw4IMPAuDr68vnn38uw0mYnaqet1LX57gYQrKqo66zSn1RFeqLOnbsyL59+8jPz0dRFL799lsef/zxWkoqhObKRN4AACAASURBVBDidma152SoqtYX9e3blxMnTuDr64tGo+Hxxx8nMDDQoNeS+iJR16R6SNQnUl9kJFJfZHySVT3mlFeyqqOus0p9kdQXCSGE2amXw0nqi4QQwryZ5YKImtQUAVy7do3hw4eTnp6uv23fvn14eXkxZMgQVq1aZayoQgghqsEs95yqW1MEJSfyvv3225w7d05/282bN5k9ezYbNmzA0dGRV199lb179+Lm5lbl7Up9kTpMOavUBQmhHpMbTmrWFEHJXtf8+fOZOXOm/raUlBQeeeQRHn74YQC8vLz4+uuvDRpOUl9075G6ICHUY5KH9dSqKQIICwuje/fuZW7LzMykWbNm+p+bN29ORkaG8d+YEEKIKjG5PSdQr6aoIjqdDgsLC/3PiqKU+VmIitx62NGUD0GWx5zySlZ1mHJWkxxOatUUVcTBwYGsrCz9z1lZWTRv3rxa2xL3ltLzROr6nBFDmVNeyaqOus5a7+uLDKkpqsiTTz7J77//zh9//IFWq2X79u3079/fyEmFEEJUlUnuORmqqjVFFbGxsWHp0qVMnTqVgoIC3Nzc8PDwMGgbUl9075G6ICHUY/b1RbVdU1QRqS8yPsmqHnPKK1nVUddZ63V9kdQUCSFE/WTWw0lqioQQon6q9oIIf39//Z9nzZrFxYsXgZI9F19fX7y8vJg8eTJXrlypecq7yMzM5MUXX2TEiBGMGjWKkydPlvu4gQMHlqkrEkIIYbqqved04MAB/Z+Tk5MJCgoCSgbVmjVraN++Pe+88w6RkZG8+eabNU9agVWrVuHu7s64ceP4/vvvCQ0N1TdG1CapLzK+wiJtXUcQQtSRSodTcXExISEhnD59muzsbJycnLC3twdg1KhRDB48mMzMTAIDA4mKimLHjh1oNBqKiorIyMjAycnprtsPDg7GxsaG1NRUrl+/zpQpU/Dx8WH16tUcPXqUv/76i+eff54+ffoQEhJCXl4etra2zJ07l86dOxMWFqbfVnp6Ovfffz8AeXl5zJgxg0uXLtGuXTsKCgqAktLX2bNnk5GRQWZmJi4uLoSFhTFz5kx69OjB6NGjgZI9w+nTp1f5pF6pLzK+bSurd2qAEML8VTqcjhw5gkajISYmBp1OR0BAAN7e3mzatIlNmzYBEB0dTUREBE2bNgXg119/ZeLEiVhZWVVpr+nChQvExMSQk5ODr68vffr0AaCwsJAdO3YAMHbsWObNm0fnzp05c+YMQUFBJCYm0qBByZFJDw8PLl68yAcffABAeHg4nTt35qOPPuLgwYMkJCQAsGfPHjp16kR4eDiFhYV4enryyy+/4Ofnx+rVqxk9ejQXL17k8uXL1WqbEEIIUXOVDqcePXrQpEkToqKiSEtL49y5c+Tn59/1OU5OTuzbt4/o6GjeeOONSg+zlV4e3cHBAWdnZw4fPgzAE088AcD169c5fvw4s2bN0j8nPz+f3Nxc/UD8+uuvOXnyJC+99BIJCQkcOHCAlStX6t9Daanr8OHDSUlJYf369aSlpZGXl0d+fj69evVi7ty5pKenExcXV+0TeoVxmcshSDCvrGBeeSWrOkw5a6XDaffu3YSHh/PCCy/g6+tLbm4uFZ0aVVBQwA8//KBf1j1ixIgqXXPp1noinU6HlVVJLFtbW/1t1tbWZVbhXbp0iSZNmrBnzx569OhBo0aN6NSpEy1btuTChQtYWFiUyVn6Ghs2bCAxMZHRo0fj6urKb7/9pu/S8/HxIT4+noSEBCIjIyvNLdQn54yow5zySlZ11HXWGp/ntH//foYOHYqfnx8XLlwgOTkZFxcXLC0tKS4uxsrKCktLS7RaLVZWVoSGhuLg4EDXrl1JSEjA2dm50pAJCQl4eHjw559/kpKSQlhYWJlVd3Z2drRp00a/R5OUlMS8efPYtWsXW7Zs4ffff2fixImcOXOG7Oxs2rZti4uLC3FxcXTs2JGUlBR9c3lSUhJjxozBy8uL1NRUTp06hU6nA0r24MaPH0/79u0NPh9KGiKMTxZECHHvqnQ4jRo1iunTpxMfH49Go8HZ2Zn09HSeffZZvL29iY2NZcCAAQQGBrJu3TpWrVrFvHnz0Gq1tGjRosyChYrcvHkTPz8/CgsLWbBggf5Q3a1WrFhBSEgI69atQ6PRsGrVKiwsLJg9ezazZ89my5Yt2NjYsHLlSho1asS0adMIDg7G09OTtm3b6g/rBQQEEBISQkREBI0bN+app57SLzF3dHTE0dGRkSNHGvp7lIYIFZjyIQchhLrqvL4oODiYnj17VnodJrUpikJmZib+/v5s374da2trg54vw8n4JKt6zCmvZFVHXWc1ifqiZcuWsW/fvjtu79q1a228fJUkJiYSEhJCSEiIwYNJCCGEcdXKcHrrrbdq42VqxMPDw+AmciGEEOqoF/VFL7/8Mt7e3owcOfKOrr1SUl8khBDmw+zri5YvX87AgQOZMGECaWlp+Pv78/3339/16rlqkPoi45JrJQlxbzP7+qLBgwfTu3dvAB555BEKCgrIz89Hq9VKfZEZk+oiIe5tZl9f5O7urt9OZGQknTp1ws7OjgULFkh9kRBCmKl6U1+0fv16YmJi+OyzzwCkvqieMIdDkKXMKSuYV17Jqg5Tzmr29UVQ8r3T3r17iYqKwsHBAUDqi+oJOWdEHeaUV7Kqo66z1vv6ovXr15OcnMzGjRv1l8sApL7IzN0sKMbWxqwv1CyEqAGzri8CeP/992ncuHGZpe0RERFSX1SBuv7bkiFsTfiQgxBCXVJf9F9SX2R6JKt6zCmvZFVHXWeV+qIqkvoiIYQwHVJf9F9SXySEEKaj2vVFpuiLL76gX79+VVohWJFly5YRHBxsxFRCCCEMVa+WQ23fvp0lS5bQt2/faj1///79bNmyhQEDBhj8XKkvMp6bBcVc/VvaNoS4l5nNcEpOTmbt2rVoNBrS09MZOHAgDRs2ZNeuXUBJsWtqaiqhoaHMmTOHtWvX6k/+jY2N5dixY4SGhla4/by8PFatWsXkyZM5deqUwfmkvsh4tq30xjy+UhZCqMWsDuuVDpjNmzcTFRWFvb09sbGxODk5YWdnR9euXVm0aBFubm5kZWXpz23aunVrpasB582bxxtvvFHmXCkhhBB1w2z2nAA6dOiAo6MjAE2bNsXFxQWAli1b8vfff+sfZ2FhwciRI/nqq6/w9fUlJyfnrj15mzZtwtHRERcXF2JjY9V9E6JKSg89mvohyFuZU1Ywr7ySVR2mnNWshpNGoynz890uizFy5EgmTZqEtbV1pT15O3bsICsrC29vb65cuUJ+fj6LFy9m9uzZRsktDJeVdbXOz8MwhDllBfPKK1nVUddZTeI8p7rQqlUrHBwciI6OZuPGjXd97Mcff6z/c2xsLAcOHDB4MEl9kfHItZyEEPV2OAEMGzaMnTt3GtyTVx3SECGEEMZT5/VFaikuLmbmzJl4eHgwZIj6ezUynIxPsqrHnPJKVnXUddZ78rCeoij069cPV1dX/eU7Dh06xMKFC8t9fERERK3sXQkhhKiaejmcLCws2L9/f5nbunfvXuZ6UEIIIUxXtc9zuvUSFbNmzeLixYtAyR6Kr68vXl5eTJ48mStXrtQ8ZRUkJSUREBBQ4f0DBw7UXxpDCCGEaav2ntOBAwf0f05OTiYoKAgoGVRr1qyhffv2vPPOO0RGRvLmm2/WPGkFdDod69ev58MPP6RDhw6qvU5lpL7IOKS6SAgBVRhOxcXFhISEcPr0abKzs3FycsLe3h4ouRDh4MGDyczMJDAwkKioKHbs2IFGo6GoqIiMjAycnJzuuv3g4GBsbGxITU3l+vXrTJkyBR8fH1avXs3Ro0f566+/eP755+nTpw8hISHk5eVha2vL3Llz6dy5M2fPnuXs2bMsXLiQDRs26Lebl5fHjBkzuHTpEu3ataOgoACAa9euMXv2bDIyMsjMzMTFxYWwsDBmzpxJjx49GD16NFCyZzh9+vS7nrx7K6kvMg6pLhJCQBWG05EjR9BoNMTExKDT6QgICMDb25tNmzaxadMmAKKjo4mIiNBfwfbXX39l4sSJWFlZVWmv6cKFC8TExJCTk4Ovry99+vQBoLCwkB07dgAwduxY5s2bR+fOnTlz5gxBQUEkJiby2GOPERYWRnJycplthoeH07lzZz766CMOHjxIQkICAHv27KFTp06Eh4dTWFiIp6cnv/zyC35+fqxevZrRo0dz8eJFLl++XOXBJIzr1j07U97Lu505ZQXzyitZ1WHKWSsdTj169KBJkyZERUWRlpbGuXPnyM/Pv+tznJyc2LdvH9HR0bzxxhv6AtaK+Pr6otFocHBwwNnZmcOHDwPwxBNPAHD9+nWOHz/OrFmz9M/Jz88nNze33Eu6Q8lhx5UrV+rfQ+ll2ocPH05KSgrr168nLS2NvLw88vPz6dWrF3PnziU9PZ24uLhKWyWEekqXt9b1UldDmFNWMK+8klUddZ21xkvJd+/eTXh4OC+88AK+vr7k5uZS0alRBQUF/PDDD/rl2yNGjKjStZVurSHS6XRYWZXEsrW11d9mbW1dZrXdpUuXaNKkSYXbtLCwKJOz9DU2bNhAYmIio0ePxtXVld9++w1FUbCwsMDHx4f4+HgSEhKIjIysNLcQQgh1VDqc9u/fz9ChQ/Hz8+PChQskJyfj4uKCpaUlxcXFWFlZYWlpiVarxcrKitDQUBwcHOjatSsJCQk4OztXGiIhIQEPDw/+/PNPUlJSCAsL4+TJk/r77ezsaNOmjX6PJikpiXnz5ukvl1EeFxcX4uLi6NixIykpKfqG8qSkJMaMGYOXlxepqamcOnUKnU4HlOzBjR8/nvbt2xt83pPUFxmHVBcJIaAKw2nUqFFMnz6d+Ph4NBoNzs7OpKen8+yzz+Lt7U1sbCwDBgwgMDCQdevWsWrVKubNm4dWq6VFixaEhYVVGuLmzZv4+flRWFjIggULyj1Ut2LFCkJCQli3bh0ajYZVq1ZhYWFR4TanTZtGcHAwnp6etG3bVn9YLyAggJCQECIiImjcuDFPPfWUfom5o6Mjjo6OjBw5stLMt5OGCCGEMJ46ry8KDg6mZ8+elV5vSW2KopCZmYm/vz/bt2/H2traoOfLcDI+yaoec8orWdVR11lNor5o2bJl7Nu3747bu3btWhsvXyWJiYmEhIQQEhJi8GASQghhXLUynN56663aeJka8fDwwMPDo65jCCGEwMwu03674OBgYmNjCQ8PZ8CAAWWuy2So0hOJfXx8GDt2rFQdCSFEHaoXxa9xcXF8/PHHPProo9XexsyZM3F3d2fcuHFs3LiRd955h3fffbfKz5f6ouqRuiIhRHnMajgpisLSpUvZs2cPzZs3R6vVEh8fj6IoBAUF0bp1a/r378/48eOJiYlh/fr1JCQkUFRUxKBBg9i1a9cdl3oHuHz5MqdOndLvefn5+eHi4mJQNqkvqh6pKxJClMeshlNiYiInTpxg+/btXL16lREjRhAaGsp7771HREQEaWlpfPnll4wfP56ffvqJK1eukJ2dzZkzZ3jqqafKHUxQUp/UsmVLli5dyqFDh2jWrBlz586t5Xd377rbnpwp7eVVxpyygnnllazqMOWsZjWcDhw4wJAhQ9BoNNjb29O/f/8y95dWEGm1WtLS0hg2bBgHDx4kNTWVAQMGVLjd4uJiTpw4wdSpU5k1axabNm0iODi4TJGsUE9Fy1nreqmrIcwpK5hXXsmqjrrOWtlScrNaEHF7JVFpzVEpGxsbOnXqxLZt22jbti29evXi4MGDJCUl3THIbtWsWTMaNWrEM888A/yvf08IIUTdMKs9JxcXFyIjIxk7diw3btzghx9+oFu3bmUe4+bmxvvvv89LL71Ez549WbRoES1bttRf5qM8rVu3xsHBgb179+Lm5sZ3331Hly5dDMom9UXVI3VFQojymNVwGjRoEKmpqQwfPpyHHnqIdu3a3fGYAQMGEBISQs+ePXnggQd48MEH73pIr9Tq1auZP38+K1asoHHjxixdutSgbNIQIYQQxlPn9UX1hQwn45Os6jGnvJJVHXWd1STqi0yFv78/f//99x23jx07lnHjxtVBIiGEEOW5p4aTrL4TQgjzYLbDqbTNPD09ndjYWAICApg4cWKNtnnixAlGjx7N8ePHDX6uNERUnbRCCCEqY7bDqZQxqosAbty4wcKFCykqKqrW86UhouqkFUIIURmzOc9JURSWLFmCu7s7/v7+nD9/nvnz55ORkUFQUBCTJ0/m888/ByAmJoahQ4cCUFRUhJubW6VDZ+nSpQQEBKj+PoQQQlTObPac1KouAti9ezc3b96US2bUoqoeWqzrQ5CGMKesYF55Jas6TDmr2QwntaqLsrKyWLNmDevXr1f3DYgyqrKEta6XuhrCnLKCeeWVrOqo66z1pr5IreqiPXv2kJeXx4QJE/D29gbA29uba9euqfNGhBBCVMps9pzUqi4aNWoUo0aN0v/s5OREXFycwfmkvqjqpLJI1BWttpjc3CyKiwtV2X5mZgN0Op0q2za22sraoIEl993XmMaNH8DCwqLKzzOb4aRmdZExSEOEEKYvNzcLW9uGNGrkYND/KKvKyqoBxcXmMZxqI6uiKGi1xVy9mkdubhb29s2r/FypLzISGU7GJ1nVY055jZn10qU/aNGitSqDCWQ4VURRdGRkpOPg0Fp/m9QX/ZdUFwkhANUGk6iYhUUDwLC/vJvNgohbffHFF/Tr149ly5ZV+TkbNmwgLi6OuLg4oqKi0Gq1vP/++2UGU1FREQEBASQnJ6sRWwghRBWZ5Z7T9u3bWbJkCX379jX4uceOHePtt9/m3LlzZW5PS0tj9uzZnDhxolqZ6kN9kdQKiXuR3f33YWtj/P8VVuXzdOrUCbZu3Uxw8Fyjv/7tTpw4zp493/Laa9NUfy1jMLnhlJyczNq1a9FoNKSnpzNw4EAaNmzIrl27ABg4cCCpqamEhoYyZ84c1q5dS3R0NACxsbEcO3aM0NDQCrf/xRdfMH/+fGbOnFnm9i+//JJJkybxySefVCt3fagvklohcS+ytbHC61+Gr9CtTFU+Tx07diY4uLPRX7s85879Tm7u5Vp5LWMwueEEJXs38fHxNGnSBFdXV9566y1iY2OZNWsWdnZ2dO3alddff52ePXuycOFCzp8/T+vWrdm6dSv/+te/7rrtsLCwcm8vHVbVHU5CCGGon38+xH/+EwGAk1NHUlKOUlhYyOTJU9m0KZpz59IYM2Y8Y8ZMIDLyQzIyLnHu3O9cuZKHt7cv48e/gE6nIzx8JYcOHcTCAtzdh/H88y/y88+HWLMmHK1WR/Pmzfntt1+5ceMGn3wSyXPPjWHZskVkZGSQnZ1F9+49CQ6ey5Ejh9mw4WNsbW05d+532rVrz/z5YWg0GmJioti6dTOWlpa4uvbjtdemcflyDitWLCYjI4MGDRrw6qtB9OjRyyi/G5McTh06dMDR0RGApk2b4uLiAkDLli3LLGqwsLBg5MiRfPXVV/j6+pKTk8OTTz5ZJ5nrC1OrMzG1PHdjTlnBvPIaK2tmZgOsrGrvq/bKXsvSsoF+gYaFBXz88WesW/ch//73Cj77LIbc3FxeeGEcEyb406CBBb/+epKPPlqPTqflxRcn0LNnL44fTyUrK5OoqBiKigp57bVAHnvsMWxt7+PChfNs3RpP48Z2bN/+FT//fJiXX36FnTu/5rHHOrB48XKKiooYN86Ps2d/xdKyAcePpxATE8tDDzVj0qQADh36iQcffIgtW75k/frPsLW9j//7v9c5c+YUn332KV5ePvTv70Z2dhavvvoyn366kUaNGt3xXhs0aGDQv0eTHE639+BZWlpW+NiRI0cyadIkrK2t9Q0PovpMaXnxvbrcuTaYU15jZtXpdLW61Luy19Jqdfrmm549XSku1tG8uQOdOz+OlZUNzZo5cPXqVYqLdeh0CoMGuWNtbQtAnz79OXDgACdOpOLh4YmiWGBlZcOgQR4cOJBMnz79efjhR7C1baR/vqIoFBfrGDhwCL/+eoLPP/+Mc+d+Jy/vClevXkdRFB59tB329s3Q6aB16zbk5V3h99/P0adPP2xtS4bOu+9+AMDBg8mcO3eOiIg1QMmisvPnz/PYY053vFedTlfm32O9X0reqlUrHBwciI6OZuPGjXUdRwghquXWSraK/kJ+6+06nYKVlWU551cqaLVaoKTWrTxffhnN3r3f4uU1kuee68nvv5/VD0lra2v940pr40qy/W8JfnZ2FjY2tmi1OsLD13D//Q/89/ZsmjZtWuX3fDdmP5wAhg0bxs6dO2nRokWdZagP9UVSKySEafv++z0899xYbt68SVLS9yxbtgpLS0sSEuJxde1HUVERO3d+jb//nRdetbS01A+tgweT8fHx49ln3Tl16gSnT/+GTqejQYPyD0M++eRTLFjwNi+//CrW1taEhMwhIOBlnn66O7Gxm3jxxUn8/nsaQUGv8OWXX9Gw4Z2H9QxlcsOpV69e9Or1vy/Uvv32W/2fp06desfji4uL2b9/f5l+vKq4dbu3qu6l3KUhQgjzdLOgmG0rjf+VgBp/2bOxsSEoaBLXr1/H338ijz7alocfbs2FC+d58cVxFBcXM2TIUNzcnuHnnw+VeW6nTl34z38iWLNmNaNHj2flyqV88sl/aNSoMV27PsFff/1Jq1b/KPd1nZw64us7msmTJ6LTKbi5PUOPHr149NG2LF8eRkDAWBRFYe7cBUYZTGDm9UWKouDq6oqrqysrVqygQYMGHDp0iIULF5b7+IiICNX2rmQ4GZ9kVY855TV2fZGDwyNG2VZ51KwEioz8EICXX37VKNur7aql23/39fo7JwsLC/bv31/mtu7du1erVVwIIYTpMMv6oopUp9aoVEREBO7u7nh5ebFmzRoV0gkhRPW9/PKrRttrMgdmved0u+rWGu3bt49t27axefNm7rvvPoKCgti5cydDhlR9kYOx64ukSkgIcS8zm+GkZq3RiRMn6Nu3L40blwyYfv36sWvXLoOGk7Hri6RKSAh1KIoizeS1TFF03LoUvSrM6rBe6YDZvHkzUVFR2NvbExsbi5OTk77WaNGiRbi5uZGVlcX58+cB2Lp1K76+vhVut0uXLvz444/k5eVRUFDAt99+S3Z2dm29LSFELbGysub69b8x43VgZqXkpN8i8vKy9ScPV5XZ7DmBerVGLi4u+Pr64u/vT5MmTXBxceHYsWPqvpkqUKte5l6srakN5pQVzCuvsbI2aWLLhQsXyMpKN8r2ROWsrCxp2rQpDz30UIXnUZX7PBUzGZ1atUbXrl1jyJAhTJxYcuLaunXrePjhh2seuIbUWOp7ry4hVps5ZQXzymvsrHZ2zbBTaS7fy7/XyuTkXC/zc2VLyc3qsJ4hbq01qmw4paen89prr1FcXMzVq1f58ssvGTp0aC0lFUIIcTuz2nMyVFVrjTp27MiQIUMYMWIEWq2WF198kaefftqg1zJ2fZFUCQkh7mVm3RBxN8XFxcycORMPDw+DVt1VV27udbNoiHjwwcbk5Fyr6xhVIlnVY055Jas66jprgwYWNG1acdVRvRxOplRrJIQQwnD1cjgJIYQwb/V2QYQQQgjzJcNJCCGEyZHhJIQQwuTIcBJCCGFyZDgJIYQwOTKchBBCmBwZTkIIIUyODCchhBAmR4ZTObZt28awYcMYMmQIUVFRd9x/8uRJfH19cXd3Z86cORQXl/Tg/fnnn0yYMAEPDw+mTJnC9eslLbx///03gYGBDB06lAkTJpCVlWWyWc+ePcuECRPw9vZmzJgxnDx50mSzlrp06RI9e/YkPd14l0EwdtZr167xr3/9Cx8fH3x8fPjll19MNuuVK1d45ZVXGDFiBM8995xJ/DdQ6t1332X16tX6n03xs1VRVlP8bFWUtZQan60qU0QZly5dUp555hklNzdXuX79uuLl5aWcPn26zGM8PT2VI0eOKIqiKLNmzVKioqIURVGUwMBAZfv27YqiKMp7772nLF++XFEURQkNDVU+/PBDRVEUZcuWLco///lPk806duxY5bvvvlMURVH27duneHl5mWxWRVEUrVarvPTSS0q3bt2UCxcumGzW2bNnKytWrFAURVH27t2rPPfccyabddWqVfo/7969Wxk7dmydZ/3777+VWbNmKU888YQSHh6uf7wpfrYqymqKn62KsiqKOp8tQ8ie02327dtH7969adKkCQ0bNsTd3Z2vv/5af//Fixe5efMm3bp1A8DX15evv/6aoqIiDh48iLu7e5nbAfbs2YOXlxcAw4cP5/vvv6eoqMgks44aNYp+/foB4OTkxF9//VXjnGplhZJrb7m6utK0aVOj5FQjq6Io7Ny5k8DAQAD69+/P4sWLTTIrgE6n0+9F3bhxA1tbw65gauysALt376ZNmzb6a66VMrXP1t2ymtpn625ZQZ3PliFkON0mMzOTZs2a6X9u3rw5GRkZFd7frFkzMjIyyM3NpXHjxlhZWZW5/fbnWFlZ0bhxYy5fvmySWX19ffUXcQwPD2fQoEE1zqlW1uPHj/PTTz+V+8Eypaw5OTlYW1vz+eefM2bMGF544QW0Wq1JZgV46aWX2L9/P3379uXtt99m2rRpdZoVwMfHh8DAwDsuMGpqn627ZTW1z9bdsqr12TJEvb6eU3XodDosLCz0PyuKUubniu6//XHAHT/f+hxDLldc21kVRWH58uUcO3aMTz/9tMY51ch648YNQkND+fe//22U36WaWbVaLdnZ2djZ2RETE0NSUhJBQUHs3r3b5LICLFy4kAkTJvDCCy9w5MgR3njjDeLj42nUqOLLG6iZ1RB1/dmqSj5T+WxVRM3PliFkz+k2Dg4OZb5UzcrKonnz5hXen52dTfPmzbG3t+fq1av6vxHf+rzmzZuTnZ0NlFxn6vr16zRp0sQksxYXlXAXCwAAAjhJREFUFzN9+nRSU1P59NNPsTPS9ayNnfXQoUPk5OQwZcoUvL29yczMJDAwkLS0NJPL2rRpU6ysrBg+fDgAffr0IT8/n5ycHJPLCiWHevz8/AB46qmnePDBBzl79mydZb0bU/ts3Y2pfbYqouZnyxAynG7j6urK/v37uXz5Mjdu3GDnzp30799ff3+rVq2wsbHh8OHDAMTFxdG/f380Gg3du3dnx44dAGzdulX/PDc3N7Zu3QrAjh076N69OxqNxiSzLlu2jGvXrvGf//zHaB8eNbL269ePb7/9lri4OOLi4mjevDkRERG0bdvW5LJaW1vj6upKfHw8AEePHuW+++4zyrF8Nf4b6NixI7t27QLg3LlzZGZm8uijj9ZZ1rsxtc/W3ZjaZ6sian62DFJbKy/MyVdffaV4enoqQ4YMUSIiIhRFUZRJkyYpKSkpiqIoysmTJxU/Pz/F3d1defPNN5WCggJFURQlPT1def7555WhQ4cqL730kpKXl6coiqLk5uYqr776qjJs2DBlzJgxRl35YsysOTk5SqdOnZTBgwcrI0aM0P9jillv98wzz5js71VRFCUjI0N59dVXFU9PT8Xb21s5evSoyWb9/fffFX9/f8XT01MZOXKkkpSUVOdZS4WHh5dZVWaKn63ysprqZ6u8rLcz9merquRig0IIIUyOHNYTQghhcmQ4CSGEMDkynIQQQpgcGU5CCCFMjgwnIYQQJkeGkxBCCJMjw0kIIYTJkeEkhBDC5Px/Cxq57VJfDOQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate feature importance plot\n",
    "\n",
    "feature_importance = pd.DataFrame(model.feature_importances_, \n",
    "                                  index = df.columns[:-1], \n",
    "                                  columns = ['importance']).sort_values('importance', ascending = True)\n",
    "feature_importance[:20].plot.barh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.999576 using {'max_depth': 30, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled ET\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([400, 500, 600]), max_depth=[30,40,50])\n",
    "model = ExtraTreesClassifier(random_state=seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.999522 (0.001255) with: {'max_depth': 30, 'n_estimators': 400}\n",
      "0.999549 (0.001174) with: {'max_depth': 30, 'n_estimators': 500}\n",
      "0.999576 (0.001094) with: {'max_depth': 30, 'n_estimators': 600}\n",
      "0.999501 (0.001316) with: {'max_depth': 40, 'n_estimators': 400}\n",
      "0.999563 (0.001134) with: {'max_depth': 40, 'n_estimators': 500}\n",
      "0.999576 (0.001094) with: {'max_depth': 40, 'n_estimators': 600}\n",
      "0.999508 (0.001316) with: {'max_depth': 50, 'n_estimators': 400}\n",
      "0.999556 (0.001154) with: {'max_depth': 50, 'n_estimators': 500}\n",
      "0.999576 (0.001094) with: {'max_depth': 50, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ExtraTreesClassifier(bootstrap=False, class_weight=None, criterion='gini',\n",
       "                     max_depth=30, max_features='auto', max_leaf_nodes=None,\n",
       "                     min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                     min_samples_leaf=1, min_samples_split=2,\n",
       "                     min_weight_fraction_leaf=0.0, n_estimators=600,\n",
       "                     n_jobs=None, oob_score=False, random_state=7, verbose=0,\n",
       "                     warm_start=False)"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = ExtraTreesClassifier(random_state=seed, max_depth= 30, n_estimators=600)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9955607476635514\n",
      "\n",
      "Accuracy\n",
      "0.9786324786324786\n",
      "\n",
      "Confusion Matrix\n",
      "[[213   1]\n",
      " [  4  16]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.98      1.00      0.99       214\n",
      "         1.0       0.94      0.80      0.86        20\n",
      "\n",
      "    accuracy                           0.98       234\n",
      "   macro avg       0.96      0.90      0.93       234\n",
      "weighted avg       0.98      0.98      0.98       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEdCAYAAAAb9oCRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3iT5dfA8W/TvaArpYxS9i5lyipbQLGgDBGZgqKoiC/+QGQICCpDEUVUpigqAirTSilblFE2sixQRoHSSUe60zzvH5VAaKFpIU3bnM91eV199slteE6ece7bSlEUBSGEEMJIKnMHIIQQonSRxCGEEKJQJHEIIYQoFEkcQgghCkUShxBCiEKRxCGEEKJQbMwdgBCPaujQoYSFhRnMs7a2ply5cjRt2pTx48dTs2ZNg+WJiYksX76c7du3c/PmTcqXL0+jRo0YPnw4bdq0yfc4O3bs4Oeff+bs2bNkZGTg5+fH888/z4ABA7C1tTXZ5xOipLGSOg5R2g0dOhStVsvEiRP187Kysjh//jxfffUVjo6ObNu2DXt7ewCuXLnCiBEj0Gq1jBgxgoYNG5KYmMjGjRvZtWsXY8aM4a233jI4xgcffMCaNWt47rnn6Nq1K05OToSFhfHtt9/SsWNHPv/8c6ytrYv1cwthNooQpdyQIUOUV199Nd9la9euVerUqaPs2bNHURRF0Wq1SlBQkNKtWzclPj4+z/qff/65UqdOHWXnzp36eRs2bFDq1KmjrFmzJs/6wcHBSp06dZQNGzY8pk8jRMknzzhEmebi4mIwvXv3bsLDwxk/fjweHh551h8zZgxVq1Zl8eLF+nkrVqygbt26vPDCC3nW79mzJyNHjsTd3f2hcaxdu5ZnnnmGxo0b89RTT7Fu3Tr9si5dujBz5kyD9T/66CO6dOmin65bty6LFy/mmWeeoVWrVoSEhFC3bl2OHz9usN2PP/5IQEAAqampAJw+fZrhw4cTEBBA69atmTVrFunp6Q+NVYiCyDMOUSYoioJWq9VPZ2Zmcvr0aRYsWEClSpVo0aIFAH///TcqlYrAwMB892NtbU3Xrl1ZuXIlCQkJaLVawsPDefXVVx947HtvkeVn5cqVzJ07l5deeokOHToQFhbG+++/j5OTE0FBQUZ/xkWLFjF58mTc3d1p0aIFPj4+hISE0LRpU/06W7dupXPnzjg7O3Px4kWGDBlCkyZN+Pzzz4mPj2f+/Plcv36dJUuWGH1cIe4niUOUCXv37qVhw4YG8xwcHGjTpg2TJk3C2dkZgBs3buDu7o6Tk9MD91WlShUAoqKiyMnJAaBSpUpFikun07F48WL69u3Le++9B0Dbtm2JjIzk6NGjhUoc7dq1Y9CgQfrpnj17snXrVt577z2srKyIiYnh2LFjfPnllwB8/fXXeHp6snTpUuzs7ACoVq0agwcP5vDhw7Rs2bJIn0kISRyiTGjevDmTJk0C4MKFC8ydO5c2bdowb948/UkTcq9MCnqIfe/yO3/rdLoixXX58mUSExMNbjsBzJ8/v9D7uv/NsKCgIL799ltOnjxJkyZNCAkJwcXFhQ4dOgBw6NAhunbtikql0l+NNWnSBBcXFw4cOCCJQxSZJA5RJri6uuLv7w+Av78/FStWZMSIEdjZ2TFv3jz9epUrV+bAgQNkZmbq37K6340bNwCoWLGifl5UVNQDjx0TE4OXlxcqVd5HhomJiQD5Pk8pLE9PT4Pphg0bUr16dUJCQmjSpAlbt26lW7du+kSZmJjI2rVrWbt2bZ59xcbGPnI8wnLJw3FRJrVp04b+/fuzadMmdu3apZ/fuXNnsrOz2b17d77bKYrCrl278Pf3x8PDAw8PDxo0aMC+ffseeKwRI0YwYsSIfJe5uroCkJCQYDD/8uXLHDt2TD99/xVNWlrawz/gf4KCgggNDSU6Oprjx48b3PpycXGhX79+/Prrr3n+e+2114zavxD5kcQhyqx33nkHV1dX5syZQ1ZWFgCBgYH4+/szb968fH91L1myhEuXLhk8DB8+fDjnz5/nl19+ybP+pk2buHjxIr169co3hho1auDm5saePXsM5n/xxRf6KyEXFxdiYmL0y3Q6XZ63pR4kKCiIGzdu8M033+Dl5UWrVq30y5o3b05ERASNGjXC399ffyU2f/58Lly4YNT+hciP9YwZM2aYOwghHsWGDRuwtbXNc/J2dHTEysqKrVu34urqSrNmzVCpVLRu3ZotW7bw448/olKpyMrK4ty5cyxcuJAffviB0aNHM2TIEP1+6taty6VLl1i+fDnR0dGoVCquXr3KTz/9xBdffEGPHj0YN24cVlZWeWJTqVRYW1uzfPlyMjMzAVi/fj1r1qxh6tSp1KhRgxs3brBx40bKlStHWlqa/sRuZ2fH8OHDgdw3qtq1a0ezZs0M9n8nKe3du5c+ffrQqVMn/TI/Pz++/vpr/v33X5ycnLhw4QLTpk3jypUrjB07Ns+rykIYSyrHRak3dOhQnJyc8n3FNCsri6eeeork5GRCQ0P1zxqSkpJYtWoVISEhXL9+HWdnZxo3bsywYcNo27Ztnv3odDrWrVvH+vXruXr1KllZWVSvXp0BAwbQv39/bGwe/rjwp59+4vvvvycqKopq1arx+uuv07NnTwA0Gg2zZs1i+/bt2NjY0Lt3bypVqsSPP/6ov81Wt25d3n33XV5++eU8+/7uu++YPXs2a9eupUmTJgbLjh49yueff86pU6ewt7enWbNm/O9//6N27drGNa4Q+ZDEIYQQolDkGYcQQohCkcQhhBCiUCRxCCGEKJRiSxwajYagoCCuX7+eZ9m5c+fo27cvPXr0YMqUKQZ9DgkhhChZiiVxnDx5khdffJErV67ku3zChAlMmzaNbdu2oSiKQc+hQgghSpZi6XJk3bp1TJ8+nXfffTfPshs3bpCRkaF/jbBv374sXLjQoDM3Y9y+nYpOp+Dp6UJ8vOaxxF2aSTvkkna4S9oil7QDqFRWuLs7F3n7YkkcH3300QOXxcTEoFar9dNqtZro6OhCH0OnU9DpFP3fhfHHyeusP3KVjOycQh9XCCFKmwrlHVk1tkvBKz6A2Ts51Ol0BhW3iqLkW4FbEE/Pu1WwarVrobbdcFSShhBCGMvsicPHx8egz6C4uDi8vb0LvZ/4eA06nYJa7UpsbEqhtk3PkqQhhBDGMnviqFy5Mvb29hw9epTmzZuzadMm/XgC5vDjaPMd+3EqSgIti6Qd7pK2yGUp7XArMZ3le8M5H5VkML9zfR+GtKv1SPs2W+IYNWoUY8eOxd/fn08//ZSpU6ei0Who2LAhw4YNM1dYQghRqmlzdPxx6jobjlwlO+fu894K5Rx4uWMdGlR2Q6Uq/OOAexVr4rh3XIRly5bp/65Xrx6//vprcYYihBBlzuXYFJbvCedqfKp+nsoKegb40rdFVexsHj76pbHMfqtKCCHE47Hx6DWDpFHNy4VXOtWhmtfj7UJfEocQQpQRwwJrceZGIjk6hX4t/Xi6cRWsH/G2VH4kcQghRCmUmpmNrbXK4PaTp4s9b3StRyV3J3zKO5rs2NLJoRBClDJhEbG8u/YoG45ey7OsWTVPkyYNsLArDqkQF0KUZrdTM/n+r4scuRwPQPCJSFrXVOP3mJ9hFMSiEkdBScPB9vG8cSCEEI+ToijsOXeLnw9GkHZPwXI5Rzs0mdnFHo9FJY6CkkbfFn7FGI0QQhTsVlI6K/aGc+5m3kK+ga1r4Gxf/Kdxi0oc9yorFeJCiLIpR6fob69n5+j0833KOzKyQ20aVHYzW2wWmziEEKKkup2ayadbz3A17m737yoreKaJL32aP75CvqKSxCGEECVMOUc77q2+MFUhX1HJ67hCCFHCWKusGNWpDg621gxsXZ0P+jYtMUkD5IpDCCHMKjVTy66zUTzTpAqqe8Yi8vNyYeGQVjiZ4eF3QUpeREIIYSEOR8Tx3V8XSUrLwtHOmicbVjJYXhKTBkjiEEKIYnd/IR/AmoOXaVVDjaujrRkjM44kDiGEKCYPKuQr72THS4G1SkXSAAtIHNLNiBCiJHhQIV+nej682MY8hXxFVXoiLaL8koZ0LSKEKC4PKuTzLufAK/+NyFfalPnEkV/SkK5FhBDFJfhEJOvCruinc0fkq0LfFn5mL+QrqjKfOO4l3YwIIYpbt0aV2Hk2inhNJn6ezrzSqQ7V1a7mDuuRWFTiEEIIU9PpFFT3jLrnaGfDyA61uRafytONK2NjXfrrriVxCCHEY5CaqeXnAxFkaXN448n6BssCqnoQUNXDTJE9fpI4hBDiER2+HMf3+y6SmJYFQJva3jT18zRzVKYjiUMIIYrodmomq/66xOHLcQbzT127LYlDCCHEXYqisPf8LVYfuExallY//04hX8saXmaMzvQkcQghRCHcSkrn273hnM23kK86zvalo/r7UUjiEEIII+iU3EK+3w7nLeR7uWNtGlZ2N2N0xavMJY5fD0Tww95w6WJECPFYWQHht5L1SeNOIV+f5n7YW1hvFGUucfz4Z/5JQ7oZEUI8CisrK14KrMW5m4l4uzqUiUK+oipziSM9K/+kId2MCCEK49+oJKqrXQy6BfFwsWdK7wCquDuViUK+oipzieNe0sWIEKKwUjO1rDkYwe5ztwhq4svA1tUNlpekIVzNpUwnDiGEKIz7C/n+OBlJq5peFntL6kEkcQghLF5iWhbf/3WRwxGGhXxN/Twp72hnpqhKLkkcQgiL9cBCPkdbhv9XyGdlZfWQPVgmSRxCCIsUnZTOij8vcPZGosH8jvUq8GLrGrg4lP1CvqIqlsSxZcsWvvnmG7RaLcOHD2fw4MEGy8+cOcO0adPIzs6mYsWKfPLJJ5QrV644QhNCWKCImBQ+3HySLK1hId/IDrVpVMVyCvmKyuTvk0VHR7NgwQJWr17Nxo0bWbt2LRcvXjRY56OPPmLs2LFs3ryZ6tWrs2LFClOHJYSwYH5eLlR2dwLA6r9CvtnPN5ekYSSTJ479+/fTunVr3NzccHJyokePHoSEhBiso9PpSE1NBSA9PR0HBwdThyWEsGDWKite6ViH6moXPujTlEFtalhc9fejMPmtqpiYGNRqtX7a29ubU6dOGazz3nvvMXLkSD7++GMcHR1Zt25doY/j6Zn33Wq1hb9CZ+mf/w5ph7sssS1OXY0n5Hgk458NQPXfg2612hW12pXm9Xzk4XcRmDxx6HQ6g/8xiqIYTGdkZDBlyhS+++47GjduzMqVK5k4cSJLly4t1HHi4zXodIrBvNjYlEcLvhRTq10t+vPfIe1wl6W1xb2FfABVyjvSrVEli2uH/KhUVvn+2DZ6+8cYS758fHyIjY3VT8fGxuLt7a2fDg8Px97ensaNGwPwwgsvEBYWZuqwhBBl2JHLcUxce0SfNAC2HL+G9p5ebUXRmTxxtG3blgMHDpCQkEB6ejqhoaF06HC3KxA/Pz9u3bpFREQEADt37sTf39/UYQkhyqDEtCwWhp7l821n9dXfAM2reTKjT1OL7l/qcTL5raoKFSowbtw4hg0bRnZ2Nv3796dx48aMGjWKsWPH4u/vz+zZs/m///s/FEXB09OTjz/+2NRhCSHKEEVR+PPfaH7aHyGFfMXASlEUpeDVSr47zziGLP5TP8+SOzmU+7i5pB3uKqttUdhCvrLaDoXxqM84pHJcCFGq7Thz0yBpSCGf6UniEEKUav1aViMsIo6E1EyeblyFfi0sb0S+4iaJQwhRamRpdWTn5OBsf/f2k4OtNa93qYudjTU1vC2vTsUcJHEIIUqF8zcTWb73An5eLrzVrb7BsnqV3MwUlWUqU4njj5PXzR2CEOIxS8vUsubQZXadjQLgVlI6bWt707yap5kjs1xlKnGsP3JV/7eD3OMUotQ7ejmO7/Zd5PY9NRkOttZk3PPKrSh+ZSpxZGTn6P/u28LPjJEIIR5F0n8j8oXdNyJfs2qevBRYCw8XezNFJqCMJY579QyoYu4QhBCFdKeQb/WBCFIz715VlPuvkO8JKeQrEcps4hBClC6Z2TksCDnD6fsK+drXrcDgNjIiX0kiiUMIUSLY2agM6i/Urg683FEK+UoiSRxCiBLBysqKl9rX4t+oJNrXrUC/ltXkJZcSqtCJIysrCzs7O1PEIoSwEFlaHaH/3KBbo0oGVxnuzvbMH/QEzvbym7YkM6qPYa1Wy6JFi2jfvj2tWrUiMjKSIUOGGIyzIYQQxjgflcSUX4+y5tBlfrvnFfo7JGmUfEYljvnz5xMWFsb8+fOxsbFBrVbj4+PD9OnTTR2fEKKMSMvSsvLPC3y46SRRiekAbD11nciEVDNHJgrLqNQeHBzMpk2bcHd3x8rKCgcHB2bOnEnnzp1NHZ8Qogw4diWelfsucDvVsJBvYOvqVHZ3MmNkoiiMvia0tjZ8SJWTk4O9vRThCCEeLCkti1V/X+LQJcPb2k39PHipfW08pZCvVDIqcXTv3p1x48YxYcIEFEXh+vXrfPrpp3Tt2tXU8QkhSiFFUdgXnjsi3/2FfMPa1aJVTSnkK82MesYxYcIEqlatysCBA0lJSaFnz544Ozszfvx4U8cnhCiFwiLiWLo73CBptK9bgbkvtKB1LbUkjVLOqCuOjIwMpk+fzvTp00lISMDNzQ2VSkVkZCTOzs6mjlEIUcq0rO5FDbUrEbEpqF3tGdmhDv6+UshXVhh1xXHvQ3APDw9UKhVarZbnnnvOZIEJIUoPnU4xmFaprHilU22eblyZ2QNaSNIoYx54xXH9+nUGDhxITk4OaWlptGnTxmB5ZmYmtWrVMnmAQoiSK0urY+PRq/x7K4kpvQJQqe7egqrq6cLgti5mjE6YygMTR5UqVVi2bBnJycm8+uqrfPHFFwbL7ezsqFu3rskDFEKUTOejklixN1xfk7H9zE16+Fc2c1SiODz0GUf9+rnDM+7evRsPD488y3U6nWmiEkKUWGlZWtYevMzO/0bku+PktQS6N6okD74tgFEPx5OSkpg7dy7R0dH6ZKHVarl69Sp///23SQMUQpQcDyvk69KgoiQNC2FU4pg8eTLu7u5UqlSJGzdu8MQTT7BmzRoGDRpk6viEECWAFPKJexn1VtW5c+f47LPPGDlyJABvvvkmX375JTt37jRpcEII8/srPJp31x4xSBrlHGwZ82Q93nmqoSQNC2TUFYebmxv29vZUrVqVixcvAtCkSRMiIyNNGpwQwvyuxGoMC/nqVGBQ2xq4yoh8FsuoK4769evz2WefoSgK3t7ebN++nX379uHo6Gjq+IQQZtb/iWqoXe1Ru9oz8ZlGvNalriQNC2fUFceUKVOYOnUqiYmJjB8/nrfeeousrCxmzJhh4vCEEMUpMj4VV0db3JzuDtbmYGvN+Kcb4enqICPyCcDIxHH8+HEWL16Mg4MDFSpU4NChQ2RnZ+PkJN0hC1EWZGl1bDp2jd9PRNK8midjuzcwWF7ZQ7oWEncZdatq1qxZ2NjczTG2traSNIQoI/6NSmLqr0fZdOwaOTqFsIg4jlyOM3dYogQz6oqja9euLFmyhF69euHlZdgdsjznEKJ0SsvSsu7QZXacMSzkq+NTjopu8sNQPJhRiWP79u1oNBoWLVqkn6coClZWVpw7d85kwQkhTOP41XhW/nmBhPsK+V5oVZ2uDSuikkI+8RBGJY5NmzaZOg4hRDFISs/ih78vcfCiYSFfk6oejOhQC08XBzNFJkoToxJH5cqP1nHZli1b+Oabb9BqtQwfPpzBgwcbLI+IiGD69OkkJSWhVqv57LPPKF++/CMdUwhhKC45g4lrjqC5d0Q+B1uGBtakdU0ZXEkYz6iH448iOjqaBQsWsHr1ajZu3MjatWv1RYSQe8vr9ddfZ9SoUWzevJn69euzdOlSU4clhMXxKudA/cpu+unAOt7MHdiCNrW8JWmIQjHqiuNR7N+/n9atW+PmlvuF7dGjByEhIYwZMwaAM2fO4OTkRIcOHQAYPXo0ycnJpg5LCIs0rF1NYpIzeKFVNRr75u3xWghjmDxxxMTEoFar9dPe3t6cOnVKP33t2jW8vLyYPHky586do0aNGrz//vuFPo6np+GAMWq1a9GDLiOkDXJZYjtcjk7mp30XGd+7MQ52d/+Z16nmxZLRHSz+CsMSvxOPk9GJIywsjLVr1xIbG8uCBQv46aefePPNN7G2fnglqU6nM/iS3nkb6w6tVktYWBg//vgj/v7+fP7558yZM4c5c+YU6oPEx2sMpmNjUwq1fVmjVrtafBuA5bVDdk5uId+W45Hk6BRc7awZ1KYGYHlt8SDSDrlD+97/Y7tQ2xuz0vr16xk/fjzVq1fnzJkzWFlZsX37dubNm1fgtj4+PsTG3n2DIzY2Fm9vb/20Wq3Gz88Pf39/AIKCggyuSIQQxgmPSmLKL0fZeDS3kA9gx5mbJKVnFbClEIVjVOJYsmQJy5YtY8yYMahUKjw8PFi2bBnBwcEFbtu2bVsOHDhAQkIC6enphIaG6p9nADRt2pSEhATOnz8PwK5du2jYsGERP44Qlic9S8v3+y4ya9NJbv43jCvkFvLN6teM8o52D9laiMIz6lZVYmIitWrVAtDfZvLy8iI7O7vAbStUqMC4ceMYNmwY2dnZ9O/fn8aNGzNq1CjGjh2Lv78/X331FVOnTiU9PR0fHx+jrmSEEFLIJ8zDqMTRrFkzFi5cyLhx4/Tzvv/+e5o0aWLUQXr16kWvXr0M5i1btkz/d0BAAL/++qtR+xJCQPJ/hXwHpJBPmIFRiWPatGmMHj2aNWvWoNFo6NKlCw4ODixZssTU8Qkh8nHwUqxB0nB1sGVou5q0qSWFfML0jEocFStWZMOGDZw6dYqoqCjUajVNmjQx6DFXCFF8nmxQib/DY7gUk0JgHW8Gt6mJq6MMriSKh1Fn/oEDB/Lss8/y9NNPG317SgjxeOh0CimZ2QYPuVUqK0Z1qkO8JpOAqlLIJ4qXUW9V9e/fn9DQUDp16sSbb75JaGioUQ/GhRCPJjI+lQ82nuCzrWfQ/feK7R1VPJwlaQizMDpxrFy5ku3bt/PEE0/w7bffEhgYyPTp000dnxAWKTtHx6+HrzD1t2NciknhUkwKoadvmDssIYBCdjmiVqtp2bIlcXFx3Lp1ixMnTpgqLiEsVnhUEsv3hhvUZNiorNDed8UhhLkYlTguXbpEcHAwwcHBpKamEhQUxDfffEP9+vVNHZ8QFiM9S8u6Q1fYceYm96aI2hXK8UrH2jLutygxjEoc/fr1o0uXLkyePJn27dujUpm8N3YhLIoU8onSxKjE8ddff+HiUvQOsYQQ+VMUhcW7/uXvCzEG8wOqejCifS28XKWQT5Q8D00cQ4cO5YcffmD48OEPLCqSim8his7Kyopy99RfSCGfKA0emjgGDBgAwJAhQ4olGCEsUb+W1Th6JZ7aFcoxpK0U8omS76GJ407/Uqmpqfkmj8WLF5smKiHKIJ1OYcfZm7Ss7oW7s71+voOtNbP6NcPZXnpiEKXDA7+p8fHxnD59GoBPP/0UX19fg+UajYYlS5YwevRo00YoRBlwPSGVZXvCuRSTwrkbSbzdo4HBckkaojR54LfV2dmZhQsXcvv2bTIzM/nggw8MltvZ2UnSEKIA2Tk6Nh+7xub/RuQDOHw5jhNXE2jiJ1XfonR6YOJwcHDgt99+A2DMmDEsWrSo2IISoiwIv5XE8r0XuHk7TT/PWmXFs82q0qiKmxkjE+LRPPT6OCIigho1avB///d/XLx4Md917gzwJITIlZ6lZV3YFXaclkI+UTY9NHH079+fY8eOERQUlO9yKysrzp07Z5LAhCiNTlxNYOW+C8RrMvXzHGytGdCqGk82rCSFfKJMeGjiOHbsGIB+PHAhxINFxKTw6dbTBvMCfN0Z0aG2FPKJMsXovkMOHDgAQEJCAjNmzOCLL74gIyPDZIEJUdrU8HbliRpeQG4h3xtd6zG+ZyNJGqLMMeodwE8++YQ//viD3bt3M2XKFDQaDTY2NsyYMYM5c+aYOkYhSiSdTkGlMrz1NCywFs72Ngx4oroU8okyy6jEsX37dtasWUNKSgp//vknISEheHh40KVLF1PHJ0SJo9MpbD9zk93nopjRpykOttb6ZW5OdrzcsY4ZoxPC9Iy6VZWYmEiFChX4+++/qVKlCr6+vtja2qIoMj6AsCzXE1KZuekEP/x9iesJafwadsXcIQlR7Iy64qhfvz4ff/wxhw8f5sknnyQxMZFPPvmEgIAAU8cnRImQXyEfwOkbt8nS6rCzkaEGhOUw6ts+Z84cEhMTadasGW+99RZXr14lPj6eDz/80NTxCWF2F24lM/XXY2w4ek2fNKxVVvRpXpVZ/ZpJ0hAWx6grjooVKzJv3jwyMzOJiYmhZs2a0sGhKPPSs7T8EnaF7fcV8tWq4MorHetQRQr5hIUyKnFoNBqmT5/O1q1b0el0WFtb06lTJ+bOnSsDPIky6VRkAiv2Ghby2duoGNCqOt0aVsrzNpUQlsToW1WpqakEBwdz6tQpNm/eTE5ODh9//LGp4xPCLG4lphskjca+7sx9oQU9/CtL0hAWz6grjj179rB161ZcXV0BqFmzJnPnzqV79+4mDU4Ic3myYSUOXIwlKimNIW1r0q62t4zIJ8R/jB4EICcnx2Bap9NhaysFTqL0i0vJICtHRyU3J/08lcqK17vWxcHWmnKOdmaMToiSx6hbVd26dWPcuHGcO3eOxMREzpw5wzvvvEO3bt1MHZ8QJqPTKYT+c4OJa4+weOe/6HSGdUne5RwlaQiRD6MSx4QJE/Dw8OD555+ndevWvPjii1SuXJkJEyaYOj4hTOJOId+qvy+RqdUREZvCtn9umDssIUoFo25VOTk5MX/+fD7++GOSk5Px8vKS+72iVHpQIV8ldydqVShnxsiEKD2MShxZWVksWrSI33//nbi4OCpWrEifPn149dVXUamk+EmUDhduJbN8bzg37huRr3dTX3o3q4qttXyXhTCGUYnjww8/5OzZs0yePBkfHx9u3LjBkiVL0Gg0jB8/vsDtt2zZwjfffINWq2X48OEMHjw43/X27NnDzJkz2bVrV+E+hRAPkZGdw7pDl6WQT4jHxKjEsXXrVrZt2/MWtAMAACAASURBVIaHhwcAjRo1omnTpvTu3bvAxBEdHc2CBQtYv349dnZ2DBw4kFatWuUZcjYuLo65c+cW8WMIkb/M7Bwm/3KUmOS7Y8dIIZ8Qj8aoa3MXFxeysrIM5tnY2OjrOh5m//79tG7dGjc3N5ycnOjRowchISF51ps6dSpjxowxMmwhjGNva03L6l76aSnkE+LRGXXF0a9fP15++WXeeOMNfH19iYmJYdmyZbRp04a9e/fq1+vYsWOebWNiYlCr1fppb29vTp06ZbDOqlWraNCggfS2K0yibws/zkcl0b1RJdpKIZ8Qj8yoxLF+/XoA5s+fbzA/NjaWv/76CwArKyt27tyZZ1udTmfwD1VRFIPp8PBwQkND+e6777h161bhP8F/PD0N+8xSqwu+GirrLK0NYpLSWbbjHKO7N8DznuFaq1Ry46tX20vCwPK+Ew8i7fBojEocj/Kw2sfHhyNHjuinY2Nj8fb21k+HhIQQGxtLv379yM7OJiYmhkGDBrF69epCHSc+XmMwHRubUuSYywK12tVi2kCnKOw4c5N1h66QkZ1DaloW/9ejAVZWVhbVDgWRtsgl7ZDbM8L9P7YLtf1jjCVfbdu25cCBAyQkJJCenk5oaCgdOnTQLx87dizbtm1j06ZNLF26FG9v70InDWG5biSkMmvjSVb9dYmM7NxucY5diedafKqZIxOi7DK6r6qiqlChAuPGjWPYsGFkZ2fTv39/GjduzKhRoxg7diz+/v6mDkGUQdocHVuOR7Lp2DW09xXyvdKxNn5e0t2/EKZi8sQB0KtXL3r16mUwb9myZXnWq1KlitRwiAJdjE5m+Z5wrkshnxBmUejEkZWVhZ2ddPwmil9Gdg6/hF0h9J8bBoV8Nb1deaVTHXylkE+IYmHUTzOtVsuiRYto3749rVq1IjIykiFDhhAbG2vq+ITQ+zcqiW33JA17GxVD2tZk+nNNJGkIUYyMShzz588nLCyM+fPnY2Njg1qtxsfHh+nTp5s6PiH0Aqp60Lpmbk2QfxV35gxowVONpZBPiOJm1K2q4OBgNm3ahLu7O1ZWVjg4ODBz5kw6d+5s6viEhVIUhaT0bNycDG+LDg2sSRM/DxmRTwgzMvoporW1tcF0Tk4O9vb2jz0gIeI1Gczfeobp64/rX7G9o7yjHYF1KkjSEMKMjEoc3bt3Z9y4cZw/fx5FUbh+/Trvv/8+Xbt2NXV8woLoFIXtp28yce1RTlxLIF6TybpDl80dlhDiPkaPAFi1alUGDhxISkoKPXv2xMnJyagu1YUwxo3baczaeJLv/7qY5ypDUZQHbCWEMAejnnHY29szffp0pk+fTkJCAm5ubjKAk3gstDk6tpyIZNPR+wr53Bx5pWMd6lQsb8bohBD5MSpxLFq06IHLpCt0UVQXo3NH5LueYFjI16upL72bVsXORn6cCFESGZU4wsPDDaYTExM5ceJEnmpwIYz1S9gVNh+7lreQr2MdfD2lJkOIksyoxLFw4cI88/7++2/pjFAUmZuTnUEh3/NPVKN7I6nJEKI0KHJfVa1ateKtt956nLEIC9K1YUX2X4jBwdaakR1qoy7nUPBGQogSwajEcfHiRYPp7OxstmzZQuXKlU0SlCg7FEXh4KVYKrk5GfRYq7KyYnzPRjjZWUtNhhCljFGJIygoCCsrK/1rkSqVimrVqvH++++bNDhRusVrMlj550VOXEugutqFGX2aYn3PrShn+2LpnFkI8ZgZ9S83NDSUqlWrmjoWUUboFIWdZ6JYe+iyvibjcqyGnWdv0r2RXKUKUdoZ9b7jCy+8gEajKXhFYfEeVMjXrWElAutUMGNkQojHxagrDl9fX86cOUOrVq1MHY8opaSQTwjLYVTisLW1ZcSIEajVary8vAweZv76668mC06UDg8r5HtWRuQToswxKnH079+f/v37mzoWUQrdTs3kw00nDa4ypJBPiLLNqMSRmprKkCFD8sxfvHjxYw9IlC7uzvY81bgKv5+IlEI+ISzEAxNHfHw8p0+fBuDTTz/F19fXYLlGo2HJkiWMHj3atBGKEkWnU/Ikhb4tqpKSkc2zzXzxLudopsiEEMXlgYnD2dmZhQsXcvv2bTIzM/nggw8MltvZ2UnSsCCKonDoUiy/hF1hcu8APF3uDuJlZ2PNqE51zBidEKI4PTBxODg48NtvvwG5PeA+rIdcUbbFazL5bt8Fjl9NAOC7fRd456mGUvEthIV65G7VRdmVXyEfwJU4Tb7jgQshLIP0+SDydfN2Gsv3hhN+K9lgftcGFXmhdXWc7OSrI4Slkn/9woA2R8fvJyLZeF8hX8X/CvnqSiGfEBZPEofQuxybwtLd4UQmpOrnWausCGpShWeb+cmIfEIIQBKHuEdiWpZB0qihduWVTrWp6unykK2EEJZGEofQa+rnSZtaao5eiad/y2o85S+FfEKIvCRxWChNRjaxKRlUV7sazB/aribPP1FNCvmEEA8kicPC5BbyxbHq74vYWquYM6A5jve8IVXO0Q7JGUKIh5HEYUHuL+QDWBd2heGBtcwYlRCitJHEYQF0isLus1H8fNCwkM/d2Y5GVdzNGJkQojSSxFHG3UxMY8XecP6NyqeQr1V1nGTcbyFEIRXLWWPLli188803aLVahg8fzuDBgw2W79ixgy+//BJFUahSpQqzZ8+mfHkpNHsU2hwdm45dY+PRq2Tn3C3k8ynvyCud6lBPCvmEEEVk8oqu6OhoFixYwOrVq9m4cSNr167l4sWL+uUajYYZM2awdOlSNm/eTN26dfnyyy9NHVaZpigKE388xC9hV/RJw1plxbPNfPn4+eaSNIQQj8TkiWP//v20bt0aNzc3nJyc6NGjByEhIfrl2dnZTJ8+nQoVKgBQt25doqKiTB1WmWZlZUWnhpX009XVLszs25Tnn6gu1d9CiEdm8ltVMTExqNVq/bS3tzenTp3ST7u7u9OtWzcAMjIyWLp0KUOHDi30cTzvq25W31efYGme8XLhQHg0zWt48dwT1bBWWW7CsPTvwr2kLXJJOzwakycOnU5nMG6Doij5juOQkpLCm2++Sb169ejTp0+hjxMfrzGYjo1NKXywpZAmI5vVByLo7l+Zal53k6da7cr/PVkPKysrEuJTH7KHsk2tdrWY70JBpC1ySTuASmWV58d2obZ/jLHky8fHh9jYWP10bGws3t7eBuvExMQwaNAg6taty0cffWTqkMoERVE4eDGWd9ce4c9/o1m+J5yce3qzBWSgJSGESZg8cbRt25YDBw6QkJBAeno6oaGhdOjQQb88JyeH0aNH8/TTTzNlyhQ52RkhXpPJZyFnWLTjHMnp2UDu4ErHr8abOTIhhCUw+a2qChUqMG7cOIYNG0Z2djb9+/encePGjBo1irFjx3Lr1i3Onj1LTk4O27ZtA6BRo0Zy5ZGPBxbyOdnxUofaNK/macbohBCWoljqOHr16kWvXr0M5i1btgwAf39/zp8/XxxhlGpSyCeEKCnkbFPCaXN0BJ+8nn8hX8fa1KvkZsbohBCWSBJHCReVlM5vh69w57m3tcqKZwKq8FxzGZFPCGEekjhKOF8PZ55p4suW45FUV7vwSsc6+HnJiHxCCPORxFHCJKZl4eZkZzCvT3M/vFzt6VSvItYyIp8QwszkXkcJocnIZunufxn/82HiNRkGy+xsVHRtUEmShhCiRJDEYWa5I/LFMvG/Qr6M7By+23cRRVEK3lgIIcxAblWZUYImk+//usjRK4aFe7bWKrJzdNjZWJspMiGEeDBJHGagUxR2n4tizcHLpGfdV8jXvhbNq3uZMTohhHg4SRzFLCoxjRV7L3A+KslgfpcGFRkohXxCiFJAzlLFaNfZKH74+2KeQr6XO9amvhTyCSFKCUkcxcjTxV6fNFRW8EwTX/o0ryrPMsqw9PRUNJpEcnK05g6FmBgVOp3O3GGYnaW0g7W1DS4ubjg6Oj/2fUviKEYBVT1oW9ubqMQ0KeSzAOnpqaSk3MbNTY2trZ3Ze362sVGh1Zb9E2ZBLKEdFEUhOzuLxMTcIS0ed/KQxGEip6/fBqBRFXeD+SPa18LOxlpqMiyARpOIm5saOzt7c4ciLIyVlRV2dva4ualJSoqTxFHSpWbmjsi393w0Hs52zHmhBU52d5vZ0U6a3FLk5GixtbUreEUhTMTW1s4kt0mlAPAxURSFsEuxvLvmCHvPRwOQkJrF+iNXzRyZMCdz354Sls1U3z/5+fsY3E7N5Lt9eQv5Wtbw4pmAKmaKSgghTEMSxyPQKQp7zt3i54MRBoV8bk52DG9fi5ZSyCdKicDAFtjb26NS5d6EUBQFLy81Q4YMJyjoOf16N25cZ/nyxRw+fIjMzEx8fHzo2bM3Awa8iLX13bcD4+JiWbFiKQcP/k1qaire3t706dOffv1eKFRcs2a9z86d2/n11y14ean18//4Ywu//baOFSt+MFj/2LEjvP/+RIKDd+rnHTy4n59//oELF8JRFIUGDRowatQb1KvXoFCx5Cc5OZnZs2dy7NhhnJ1dGDlylEF73WvnzlBWrFhCXFwc9es3ZPz49/D1rQpAePh5Fiz4hEuXLlKxYkVGjx5DmzaBQO7/i5Url7Fp029kZmbSrFlLJk2ahqurK6GhW4mMvMbLL7/2yJ+lMORWVRFFJabx8eZTfPvnBYOk0bm+D3NfaCFJQ5Q6y5Z9z/bt+9i+fR/btu1l5MhXmTfvY65cuQzA1atXGDVqOF5ear77bjWhoXuZMuUDdu0KZerUifr9xMbGMHLkEMqVK8fKlavZtm0PkyZNZ/XqH/j226VGx5OcnMz+/X/TufOTbNz4W5E+0+bNG/j44w8YMGAQmzdvY+PGrTzxRBvGjn2diIhLRdrnvebN+xAnJ0c2bw7lww/n8vXXX3Lx4oU8650+/Q8ffTSDN954mz/+2EnXrt0YN+5NMjMzSE3VMGHC2zRv3pLg4B1MnDiVWbOm6/fz229r2b17B8uWrWLjxhAUReHrrxcC0K3bUxw6dIDLlyMe+bMUhiSOIsjS6vhw00mD6u8K5RyY3KsxL3esg7NUf4tSztramu7dn8bJyZnLl3NPsAsXfsaTT3bnzTffxstLjZWVFfXq1efTTxdy4sQx/vxzDwDLln1D48YBvP76W7i5uWFlZUXDho147733iY+PMzqGkJBgmjRpSt++z7N58ways7ML9RnS09NZtOhzJk6cSrt27bGxscHe3p7Bg4fSp09/rl69nGebkyeP061b+zz/DRkyIM+6aWlp7Nu3l5EjX8Pe3p4GDRrRrVsPtmzZkGfdP//cTfv2HQkM7ICNjQ29e/fBwcGBw4fDOHXqJAAvv/watra2NGjQiC5durF16+8ArF//C2+8MRZv7wo4ODjw3ntTefHFIUDuM4yePYP48ceVhWqbRyVnuCKws1HRt6UfK/+8iMoKegb40reFFPKJh/vj5HXWH7lKRnZOwSs/Igdba/q28KNnEZ+xZWdns2HDL2i12TRs6E9mZiZHj4YxbNg3edYtX96Ndu3as2/fHjp06MShQwd4883/y7Ney5ataNmyldExbNmygddeexN//wDc3d3ZvXsH3bs/bfT2//xzkpwcLa1atcmz7PXX38p3m4CApmzfvs+o/V+/fg1raxsqV77bxlWr+rF37+486+p0OuztHQzmWVlZcf36NXx9/bCzczB4kK1S5S5LT0/n2rWrxMfHMXToAJKSkmjdui3/93/j9et27NiVL76Yz//+NwknJyejYn9UcsVhBJ0ubxfnnetX5MmGFZnZrxkDW1eXpCEK9MfJ68WSNAAysnP44+T1Qm0zevTL9OjRkS5d2tKjR0eOHz/KF18sxtu7AikpyWi1Wjw8PPPd1tPTi/j43JdDkpOTcHd/tC50/vnnJBqNRn+f/9ln+/Hbb+sKtY+kpERcXcthY2Oa38fp6RnY2xvW6NjbO5CRkZFn3cDADuzZs4vjx4+i1WoJDt7MtWtXycrKwt8/gLQ0DevW/Ux2djbnzp1h164dZGVlkZKSDMDWrcEsWPA1P/ywlpiYaBYu/Ey/b3d3dzw9vThz5pRJPmd+5IqjAGdu3Oa7fRd5o2s9qqtd9fNVVla81L62GSMTpU3PgCrFesVR2KuNxYtXUKNGLW7evMHkyRMoX96NBg0aAlCuXHmsra2JiYnWP9C9161bUXh65iYVDw9PEhIS8qyTk5NDaqqGcuXKFxjL5s0bSEpKpE+fnv9tqyUpKYnz589Rr159bG1tycnJ2445OTn62hlPTy+Sk3MT3v3JIzk5GScnpzzzT548wcSJea+WKlTw4fvv1xjMc3BwIDMz02BeZmYGjo55f/U3adKMt9/+H/PmfURKSgo9evSkVas2uLi4Uq5cOebO/Zwvv/yM775bTqNGjenV6zmuX4/Uf5YhQ4bj5ZX73HT48JeZOvVd3nvvff3+PT29iImJyb8xTUASxwPkFvJdZu/5WwAs3xPOB32bYmMtF2miaHoGVCnyraPiVKlSZebMmc9LLw2iYsVKDB/+MnZ2drRu3ZatW3+nefOWBuvfvp3AoUMHmDQp90TWqlUb9u7dTY8ePQ3WO3DgLz74YCqbNoXg5PTgSmaNRsPu3Tv4/POvDW4DffHFfH77bS1TpsxArfYmJiYaRVEMbvHcvHkDtdobgIYN/bG1teHgwb8JDOxocIw5c2bh5OTE1KkfGMwPCGhCSMgeo9rJ19eXnBwtt27dwsfHB4Br165SvXr1POsmJSXi7x/Azz+vB3ITXP/+vRg2bCRZWVnY2tqyZMnd5xTTp0+iTp26uLu7U65ceVJSNPplOp2O+8d5y8nJ0b8RVxzkLJiPsIhY3l17VJ80AGJTMrlxO82MUQlRfHx8KjJ27DusXLlM/3bPW2+9w/79f7Fo0efExcWh1Wo5e/Y048e/TePGTejQoTMAL730CidPHmPJkq9ITk4iJyeHY8eO8Mknsxk0aNhDkwbAtm3BVK7sS+PGTfD09NL/FxT0LDt3hpKYmEiDBo1wcnLim2++JC0tDZ1Ox/nzZ1mz5kf9cxB7e3tee20M8+Z9zP79f6HVaklLS2XFiqUcORLGoEFDH6mNnJycCQzswJIli8jIyODcuTNs376Nbt3yPoe5cuUyY8a8SlTUTTIzM1i+fDFubm40aNAInU7HW2+9xoEDf5GTk8Pevbs4dOgATz31DABPPx3EqlUriIuLJTk5me+/X0GXLk8a7D8+Pg5v7wqP9HkKw0opI2OUxsdrGPT1Xv30j6M7FHoft1NzR+Q7cjlvId+wdjVxdy49fQ6p1a7ExqaYOwyzM2c73Lp1FR8fP7McOz8P69wvMLAFq1atoUaNWgbzx417k+TkZJYu/Q5ra2uiom6yfPlijhw5RFpaGj4+FXn66SAGDBhkcNsnMvIaS5d+zYkTx8jIyMDHx4e+fQfQp0//AuMcPvxFnnyyB0OHvmQwX6fT0a9fEH37Ps/QoSOIjLzG119/walTJ8jMzESt9ua55/oxYMAgg6uQ0NAQfvllNZGRkahUKho2bMQrr7xO3br1CtF6+UtOTuKTT2Zz5EgYjo6OjBz5KkFBzwKwatW3nDx5gvnzc1+dXb36B9atW01GRjoBAU0ZP36S/uro0KEDfPnlAqKjb+HnV4233hpHQEBTIPdFhRUrlhAaupW0tFQCAzvyzjsT9Q/CY2NjePHFvgQH78jzAB7y/x6qVFZ4eha9k1VJHJTNQj5JHLkkcdxlCb3CGqOstcNvv63l7NkzvP/+zHyXmyJxWPwzjuikdJbtCc8zIl/n+j4MbF1DajKEECWWoij8/vsmZsz4uFiPa/FnxUxtDheik/XTFco58HLHOjSoLCPyCWEKv/yyhqVLv3rg8h9++EX/sFk8XEhIMO3adcDPr1qxHtfiE0dVTxeCmlRhy/FIKeQTohg8//xAnn9+oLnDKBOefjrILMe1qMSRmZ3D1TgNdSoavkf+bDM/nqihlhH5hBDCCBaTOM7cuM2KvRdITs9mzoDmeLneffvAzkYlSUOYgBWKosPKSt56F+ahKDrg8Y/JUea/0amZ2SzbE87sLf8Qk5xBRnYOK/+8QBl5mUyUYHZ2DiQmxqHVZsv3TRQrRVHQarNJTIzDzi7vK7qPqkxfcYRFxPL9X5dISsvSz3Oys6ZFjdL3eq0ofdzd1Wg0SSQkRKPTFU8fVQ+jUqnQ6crOa6hFZSntoFJZ4+jogotLwV28FFaZTRyfbzuTt5CvuhfDAktXIZ8ovaysrHB1dcPVtWS8oSe1PbmkHR5dsdyq2rJlCz179qR79+789NNPeZafO3eOvn370qNHD6ZMmYJW++iDq9+bNMo72fF29wa83aOBJA0hhHhEJk8c0dHRLFiwgNWrV7Nx40bWrl3LxYsXDdaZMGEC06ZNY9u2bSiKwrp1hes++WE61fNh3gstaCm3p4QQ4rEweeLYv38/rVu3xs3NDScnJ3r06EFISIh++Y0bN8jIyKBJkyYA9O3b12B5UXn/NyLfK51kRD4hhHicTH5GjYmJQa2+O8i8t7c3p06deuBytVpNdHR0oY/j6emCo5016Vk52KisWP5GJ+xtLbuQT33P+CGWTNrhLmmLXNIOj8bkiUOn0xn0VHl///kFLTfW7dupDGlXi73no+hYryKa5HQ0BW9WZnl6uhAfb8ktkEva4S5pi1zSDrmdHLq7P7x7+4cxeeLw8fHhyJEj+unY2Fi8vb0NlsfGxuqn4+LiDJYby93dmf6BtegfWKvglS3Eo/R+WZZIO9wlbZFL2uHRmPwZR9u2bTlw4AAJCQmkp6cTGhpKhw53uzyvXLky9vb2HD16FIBNmzYZLBdCCFGyFMt4HFu2bGHJkiVkZ2fTv39/Ro0axahRoxg7diz+/v6cP3+eqVOnotFoaNiwIbNnz8bOzs7UYQkhhCiCMjOQkxBCiOJR5vuqEkII8XhJ4hBCCFEokjiEEEIUiiQOIYQQhSKJQwghRKFI4hBCCFEopTZxmKOr9pKooHbYsWMHzz77LL179+aNN94gKSnJDFGaXkHtcMeePXvo0qVLMUZW/Apqi4iICIYOHUrv3r15+eWXLfY7cebMGfr160fv3r157bXXSE5ONkOUxUOj0RAUFMT169fzLCvSuVIphW7duqV07txZuX37tpKamqr06tVLuXDhgsE6zzzzjHL8+HFFURRl0qRJyk8//WSOUE2qoHZISUlR2rVrp9y6dUtRFEX5/PPPlVmzZpkrXJMx5vugKIoSGxurPPXUU0rnzp3NEGXxKKgtdDqd0r17d2Xv3r2KoijKJ598osybN89c4ZqMMd+JF198UdmzZ4+iKIoye/Zs5bPPPjNHqCZ34sQJJSgoSGnYsKESGRmZZ3lRzpWl8orDXF21lzQFtUN2djbTp0+nQoUKANStW5eoqChzhWsyBbXDHVOnTmXMmDFmiLD4FNQWZ86cwcnJSd+tz+jRoxk8eLC5wjUZY74TOp2O1NRUANLT03FwePxjc5cE69atY/r06fn2AVjUc2WpTBz5ddV+b1fsj6ur9pKuoHZwd3enW7duAGRkZLB06VKefPLJYo/T1ApqB4BVq1bRoEEDAgICiju8YlVQW1y7dg0vLy8mT55Mnz59mD59Ok5OTuYI1aSM+U689957TJ06lcDAQPbv38/AgQOLO8xi8dFHH9GiRYt8lxX1XFkqE0dxddVe0hn7OVNSUnj11VepV68effr0Kc4Qi0VB7RAeHk5oaChvvPGGOcIrVgW1hVarJSwsjBdffJENGzbg6+vLnDlzzBGqSRXUDhkZGUyZMoXvvvuOv/76i0GDBjFx4kRzhGpWRT1XlsrEcX9X7Kbqqr2kK6gdIPcXxaBBg6hbty4fffRRcYdYLApqh5CQEGJjY+nXrx+vvvqqvk3KooLaQq1W4+fnh7+/PwBBQUEGA6uVFQW1Q3h4OPb29jRu3BiAF154gbCwsGKP09yKeq4slYlDumrPVVA75OTkMHr0aJ5++mmmTJlSJq+6oOB2GDt2LNu2bWPTpk0sXboUb29vVq9ebcaITaegtmjatCkJCQmcP38egF27dtGwYUNzhWsyBbWDn58ft27dIiIiAoCdO3fqk6klKfK58vE9uy9emzdvVp555hmle/fuytKlSxVFUZRXXnlFOXXqlKIoinLu3DmlX79+So8ePZR33nlHyczMNGe4JvOwdggNDVXq1q2r9O7dW//f5MmTzRyxaRT0fbgjMjKyTL9VpSgFt8WJEyeUfv36KT179lRGjhypxMXFmTNckymoHfbs2aP06tVLCQoKUoYPH65cu3bNnOGaXOfOnfVvVT3quVK6VRdCCFEopfJWlRBCCPORxCGEEKJQJHEIIYQoFEkcQgghCkUShxBCiEKRxCFEMUtPTycuLs7cYehFRkaaOwRRykjiECXSihUraN68Oe3atSM7O/uB6x06dIhWrVoVY2RF07RpUy5dugTA4MGD+eeffwDYvHmzWTsZPHv2LC+++KLZji9KJxtzByBEftauXcukSZPo37+/uUN5LI4fP67/OzExUf9379696d27tzlCAnL7MXtYYhYiP3LFIUqcHj16cO3aNWbOnMnMmTPJyMhgxowZdOvWjSZNmtC9e3d27NiRZ7usrCwmTZpEq1atCAwMZOzYsdy+fRvI7dTuww8/pH379gQGBjJ37lyysrLyPX6XLl1YtGgRnTp1onnz5rz//vtkZmYCkJqaygcffEC7du1o164dU6ZMISUlBYCbN28ybNgwWrRowZNPPsm8efO4U19bt25dwsPDefPNN7l58yZvv/02q1atYv369fTt2xeNRkNAQAAXLlzQx/Hrr78yYMAA/b5Hjx5Nq1at6N69O7/99lu+sV+/fp3mzZvz3nvv0aJFCzZt2kRkZCSjR4+mY8eONG7cmIEDB3Lp0iXi4+MZNWoUiYmJNG3alNu3bxeqnYTlSnJymgAABbZJREFUksQhSpxt27ZRqVIlvvjiC6ZNm8aKFSu4dOkS69ev5+jRo/Tt25dZs2bl2W7Tpk1cunSJ3bt3s337dtLS0li1ahUAc+fOJSIigs2bN7N582ZOnz7N4sWLHxhDcHAwP/30E9u2bePs2bN8+eWXAEybNo2IiAi2bNnCH3/8QVxcHNOmTQNgwYIF1KlTh7CwMH788UeCg4M5cOCAwX6/+uor/WcbNmyYfr6LiwtdunQhODhYP+/333/n2Wef1fc5Vrt2bfbt28fChQtZsGABBw8ezDd2jUZD5cqV2b9/P927d2fq1KnUqFGDnTt3cvDgQdzd3Vm8eDGenp4sW7YMNzc3jh8/jru7e6HbSVgmSRyixBs8eDALFy7EycmJqKgonJ2d8x0zwNXVlatXr7JhwwZu377N0qVLefvtt1EUhfXr1zN+/Hjc3d3x8PDgrbfeYt26dQ885uuvv07lypXx8vLi9ddfJzg4mIyMDLZt28aECRPw8PCgfPnyTJw4ka1bt5Keno6rqyuHDx9m27ZtODk5sXv3btq2bWv053zuuefYunUrkNub67Fjx3j66af5559/iIqKYty4cdjZ2VGvXj0GDhzIL7/88sB99erVCzs7OxwdHZkzZw5jx44lJyeHmzdv4ubmlm/7FaWdhGWSZxyixEtJSeGDDz7g1KlT+Pr64uvrS35drD311FMkJCSwfv16PvroI+rUqcPMmTOpXLkyGRkZDB06VN9DsKIoZGdnk5mZib29fZ59+fn56f+uUKECsbGxJCcnk52dTaVKlfTLKleujKIoxMTEMGHCBBYuXMhnn33G//73Pzp06MCHH36Il5eXUZ8zMDAQjUbD6dOnOXLkCO3atcPDw4ODBw+i0Wh44okn9Ovm5OQ8tFfbe48ZERHBJ598QnR0NLVq1cLKyirf9ktISCh0OwnLJIlDlHjTp0+nZs2aLF68GBsbGw4fPqz/ZX6vK1eu0Lp1awYNGsTt27f56quvePfddwkODsbW1paNGzfi6+sLQFpaGnFxcQ88GcbExOj/vnnzJhUrVsTLyws7Oztu3LiBh4cHkPtMQaVS4e7uzr///suoUaOYOHEi165dY8qUKSxcuJCZM2ca9Tmtra155plnCAkJ4ciRI7z00ktA7uh1FSpUYM+ePfp14+Li8j3533HnxJ+VlcWYMWOYPXs2Tz31FACLFi3i0KFDebZxc3MrdDsJyyS3qkSJp9FocHBwwNramqioKL744guAPG8D7dy5k//973/ExcVRvnx5nJ2dcXNzw9raml69evHpp5+SnJxMWloa06ZN47333nvgMZcuXUp8fDwxMTEsXryY5557DpVKRe/evfn0009JSEggKSmJefPm0bFjR8qVK8c333zDp59+SmZmJp6enlhbW+Pu7p5n37a2tmg0mnyP+9xzz7FlyxYiIiLo0qULAAEBATg4OLB8+XKys7O5desWI0aM4Keffiqw7e5cLTg6OgJw4sQJ1qxZo287Ozs7srKyyMrKKlI7CcskiUOUeJMmTWLPnj00a9aMIUOG0LFjR5ycnPR1EXcMGzaM/2/vflUUiKI4jn+TRRCmmA2CYrBMGMOAfxAMMoIWfQODj6ADThIsNoOPZPAdppqNhg0LsuyyLDftwn4/9XK5h1N+nFNut9slyzLiOOZ2u3E4HADYbrdEUcR0OqXf7/N4PDidTt++2W63WS6XZFlGr9djvV6/amk0GsxmM8bjMVEUcTweAdjv99zvd9I0ZTAYUK/XX/c+ms/n5HnO+Xz+ctbpdKjVakwmEyqVCvAeNJfLhev1SpqmLBYLkiRhs9n82LtqtUpRFOx2O+I4pigKVqsVZVnyfD5ptVo0m02SJKEsy+A+6X/yPw7pk9FoRJ7nDIfD3y5F+pOcOCRJQQwOSVIQV1WSpCBOHJKkIAaHJCmIwSFJCmJwSJKCGBySpCBvGo+0TZB/SQ4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate AUC\n",
    "\n",
    "fpr, tpr, threshold = roc_curve(Y_validation, predictions_proba[:,1])\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "plt.figure()\n",
    "plt.xlim([-0.01, 1.00])\n",
    "plt.ylim([-0.01, 1.01])\n",
    "plt.plot(fpr, tpr, color = 'steelblue', lw = 3, label = 'ROC_AUC = {:0.3f})'.format(roc_auc))\n",
    "plt.xlabel('false positive rate', fontsize = 13)\n",
    "plt.ylabel('true positive rate', fontsize = 13)\n",
    "plt.title('ROC curve', fontsize = 16)\n",
    "plt.legend(loc = 'lower right', fontsize = 13)\n",
    "plt.plot([0, 1], [0, 1], color = 'steelblue', lw = 3, linestyle = '--')\n",
    "# plt.axes().set_aspect('equal')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a24769198>"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAD7CAYAAAAGuLF2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzde1RVdf7/8SfCASZRQdNA1AydEMaxtNTwHnlBRVH44gVDLP1hhTbj5AW8HkTzUn4t6EqQmJEQCaIiUTqjzlcR1EpwTFORETBBFAxFruf8/mA4Iyly6cBh0/uxVmvBYZ+9XxtXfvzssz+vbaTVarUIIYQQrUQbQwcQQggh9EkGNiGEEK2KDGxCCCFaFRnYhBBCtCoysAkhhGhVZGATQgjRqsjAJoQQolUxMXSA1qKg4A4aTcteEtipkwU3btw2dIw6KSGnEjKCMnIqISNITn3q1MmCgoI7WFm1bZL9y8CmJxqNtsUPbIAiMoIyciohIygjpxIyguTUp6bMaKTk5hF/f38GDRpEdnY2sbGx+Pj48NJLLzVqX6dOnWLDhg2Ul5djaWnJm2++ia2trZ4TCyFE61FSWkHRL3cb/L7Ondtx48ZtOnWyaIJUrWTGFh8fz7Zt23jiiScavY8lS5bwwQcf0KdPH7766ivWrVvHhx9+WO/3z133DXkFDf8DFkIIpdq7xY0iQ4d4AEUNbFqtlo0bN3Lo0CG6dOlCZWUlCQkJaLVa/Pz86NGjByNGjMDLy4vo6GgiIiJITEykvLyc0aNHc+DAAVQq1X37LSsr4y9/+Qt9+vQBwN7ens8//7y5T08IIRSnc+d2jXpfU83WQGEDW1JSEmfPnmXfvn0UFRUxefJkAgMDee+99wgNDSUjI4OvvvoKLy8vjh8/zq1bt8jPz+fixYv079//gYMagKmpKW5ubgBoNBree+89Ro8e3ZynJoQQinT9esPnbE19KVJRt/unpqYyduxYVCoVHTt2ZMSIETV+PnjwYE6fPk1lZSUZGRlMmDCBEydOcOTIEUaNGlXn/svKyli8eDEVFRXMnz+/ic5CCCFEU1LUjM3IyIh773UxMakZ38zMDAcHB/bu3YudnR2DBw8mOTmZU6dOMW/evIfu+86dO7z66qtYWlry4Ycf1jq7q034yrEN2l4IIZSupLTC0BEeSFEDm5OTE+Hh4cyYMYO7d+/yz3/+k6effrrGNiNHjuT999/n5ZdfZtCgQaxbt46uXbvSsWPHh+57yZIlPP744wQGBtKmTcMnsjdu3G7xt9h27tyuUZcNmpsSciohIygjpxIyguRUEkUNbKNHjyY9PR1XV1ceffRRevXqdd82o0aNQq1WM2jQIDp06ECnTp3qvAx59uxZDh48SO/evZk6dSoAXbp04ZNPPmmK0xBCCNGEFL2OrSWRGZv+KCGnEjKCMnIqISNITn2SdWx65O3tzS+//HLf6zNmzGDmzJkGSCSEEELfGj1j8/b2ZseOHQAEBASwYMGCGk0d77zzDsbGxixcuFA/SWuRl5dHQEAA+fn5tGnThqVLl+Lk5HTfds7Oznz22Wd069atSfMIIcTvQWNbR6AFz9hSU1N1X6ekpODn5wdAUVERGzZsICEhoc47EfVh8+bNODs7M2vWLDIyMvD29ubIkSMYGxs3+bHvJc0jQojfk5baOgL1GNgqKipQq9VcuHCB/Px87O3tdXcYenp6MmbMGPLy8vD19SUyMpLDhw/Ts2fPenc2+vv7Y2ZmRnp6uu6W+ylTphASEsIPP/zAzz//zIsvvsjQoUNRq9UUFhZibm7OqlWrcHR0ZMyYMTz33HMAPP7445SWllJcXExlZSVLlizh2rVr9OrVi9LSUgBu377N8uXLyc3NJS8vDycnJ9avX8/SpUsZOHAg06ZNA6pmpIsXL+app55q1C9WCCGEYdQ5sH3//feoVCqio6PRaDT4+Pjg5uZGTEwMMTExAERFRREaGoqVlRVTpkwBICQkpN4hsrKyiI6O5saNG7i7uzN06FCgasH0/v37garPwVavXo2joyMXL17Ez8+PpKQkxo0bp9tPeHg4Dg4OtGvXjrVr1+Lo6Mgnn3zCiRMnSExMBODQoUM4ODgQHBxMWVkZEydO5F//+hceHh6EhIQwbdo0cnJyuHnzpgxqQgjxEI2t0wIDV2oNHDgQS0tLIiMjycjIIDMzk+LiYr2GcHd3R6VSYW1tzYABAzh16hQA/fr1A6oWT585c4aAgADde4qLiykoKMDKygqAiIgIoqOjdR2PqampbNmyRXcO3bt3B8DV1ZW0tDQiIiLIyMigsLCQ4uJiBg8ezKpVq8jOziY+Pl5XsSWEEOLBGnv3pcE/Yzt48CDBwcHMnj0bd3d3CgoK0PcKgXs/D9NoNLpGEXNzc91rpqamxMfH67a7du0alpaWQNXnbIcPHyYyMhJra2vg/paS6mPs2LGDpKQkpk2bxpAhQ/jpp5/QarUYGRkxZcoUEhISSExMJDw8XK/nKIQQonnUObAlJyczfvx4PDw8yMrKIiUlBScnJ4yNjamoqMDExARjY2MqKysbHSIxMREXFxeuXr1KWloa69ev58cff9T9vF27dvTs2VM3kzp69CirV6/mwIEDREREkJKSws6dO2nfvr3uPU5OTsTHx9OnTx/S0tK4cuUKAEePHmX69OlMmjSJ9PR0zp07h0ajAapmjl5eXvTu3ZvHHnusQecglVpCiN+TllqnBfUY2Dw9PVm8eDEJCQmoVCoGDBhAdnY2L7zwAm5ubsTGxjJq1Ch8fX0JCwvTXfJriJKSEjw8PCgrK2Pt2rW6y4v3euutt1Cr1YSFhaFSqdi6dSsA77//PhYWFnh7e+u2DQ0N5fXXX8ff35+JEydiZ2eny+Xj44NarSY0NBQLCwv69+9PdnY2ADY2NtjY2OjaRxpCFmjrjxJyKiEjKCOnEjKC5FQSgzePVD8F293d3ZAx0Gq15OXl4e3tzb59+zA1NW3Q+2Vg0x8l5FRCRlBGTiVkBMmpTwb/jE0fNm3axLFjx+57vW/fvs1x+HpJSkpCrVajVqsbPKgJIYRoOZplYFu2bFlzHOY3cXFxwcXFxdAxhBBC/EYGvxT5W1RfxszOziY2NhYfH596LwyvzdmzZ5k2bRpnzpzRU0ohhGgdfkuN1r1axaXIphYfH8+2bdt44oknftN+7t69S1BQEOXl5Q1+r1RqCSFau5Zco3Wvhj9R04C0Wi0bNmxg3LhxeHt7c+XKFdasWUNubi5+fn688sorfPHFFwBER0czfvx4AMrLyxk5cmSdA9bGjRvx8fFp8vMQQgjRdBQ1Y0tKSuLs2bPs27ePoqIiJk+eTGBgIO+99x6hoaFkZGTw1Vdf4eXlxfHjx7l16xb5+flcvHiR/v37o1Kpat33wYMHKSkpkc/ZhBDiIX5Ljda9DFqp1ZKkpqYyduxYVCoVHTt2ZMSIETV+Xl2LVVlZSUZGBhMmTODEiROkp6c/9Cna169f58MPPyQiIqJpT0AIIRROH0sJmvozNkVdivx1TVZ19VY1MzMzHBwc2Lt3L3Z2dgwePJgTJ05w9OjR+wbBex06dIjCwkJmzZql64h0c3Pj9u3bTXMiQgghmoyiZmxOTk6Eh4czY8YM7t69yz//+U+efvrpGtuMHDmS999/n5dffplBgwaxbt06unbtqnvUzoN4enri6emp+97e3r5GL2V9SKWWEKK1a8k1WvdS1MA2evRo0tPTcXV15dFHH6VXr173bTNq1CjUajWDBg2iQ4cOdOrU6aGXIfVFmkf0Rwk5lZARlJFTCRlBciqJotextSQysOmPEnIqISMoI6cSMoLk1CdZx6ZH3t7e/PLLL/e9PmPGDGbOnGmAREIIIfSt0QObt7c3O3bsACAgIIAFCxZga2tLXFwcW7ZsoVOnTkDVpcFFixbpJ+1DxMTEcOrUKTZu3PjAn9vb23P+/PkmzyGEEMKwGj2wpaam6r5OSUnBz88PgDNnzuDv74+rq+tvT1cPpaWlhISEEBkZybhx45rlmA/SlGsy9Elfa1CamhJyKiEjKCOnEjJC68ypr5qslqTOga2iogK1Ws2FCxfIz8/H3t5ed4ehp6cnY8aMIS8vD19fXyIjI0lPTyczM5OPP/4Ye3t7Vq1aRYcOHWrdv7e3N3369OHkyZOUlpayfPlyhg0bhr+/P4WFhfz73/9myZIlPProo2zYsIGSkhKsrKwIDAyke/funDhxAo1Gw5IlS0hLS9PtNzs7myVLllBcXMxTTz2lez03N5fly5dTVFREXl4eU6dO5S9/+QteXl74+fkxdOhQtFot48aNY8eOHfV+4KhUagkhlEgpNVkNUec6tu+//x6VSkV0dDTffvstRUVFDB8+HKi6/Ofr60uXLl0IDQ3FysqKzp0789prr7Fnzx5sbGxYu3ZtnSFu376tu4Tp7+9PWVkZAJaWliQmJjJs2DBWrlzJli1biIuL46WXXmLVqlUADBs2jKVLl2Jubl5jn0FBQbi7uxMfH8+AAQN0r+/btw9XV1e+/PJL9u7dy/bt27l58yYeHh66W/xPnjxJjx49GvwUbSGEEIZX54xt4MCBWFpaEhkZSUZGBpmZmRQXF9e6/fvvv6/7et68eYwZM6bOENOmTQPAwcGBzp076z4L69evHwCZmZlkZWXx6quv6t5T1+Lp1NRUtmzZAsDkyZNZuXIlAHPnzuX48eOEh4dz4cIFysvLuXv3LuPHj2fr1q0UFxcTFxdn8AefCiFEczHEJVaDVmodPHiQ4OBgZs+ejbu7OwUFBdS2QqCoqIhdu3YxZ84coKq02NjYuM4Q926j0Wh0jSLVszCNRkO3bt10M6rKykry8/Pr3G91TiMjI9q0qZqcbty4kaysLFxdXRk9ejTHjh1Dq9XyyCOPMGLECJKSkjh+/Dhr1qypc/9CCNEaNPfyAINXaiUnJzN+/Hg8PDxo3749KSkpVFZWYmxsTEVF1Sp0Y2NjKisreeSRRwgLC+P06dMAfP755/Wase3fvx+A9PR0fvnlF5588skaP7ezs+PWrVucPHkSgF27drF48eKH7nPIkCHs2bMHgG+++YbS0lIAjh49yty5cxk/fjyXL18mNzcXjUYDgIeHB1u3bmX48OGYmZnVmVsIIUTLU+eMzdPTk8WLF5OQkIBKpWLAgAFkZ2fzwgsv4ObmRmxsLKNGjcLX15ewsDDeeecd1Go1JSUl9OzZk82bN9cZIisri6lTpwKwdevW+2Z5pqamvPvuu6xfv57S0lIsLCzYtGnTQ/e5evVqlixZQnR0NH379qVt27YAzJ8/X/eZnLW1NX379iU7O5sePXrwzDPPYGRkhIeHR52Zf00qtYQQSqSUmqyGMHjziLe3NwsWLGDw4MGGjIFWq+Wnn35i2bJl7N69u8Hvl+YR/VFCTiVkBGXkVEJGkJz61CqaR9544w0uXrx43+vOzs7Ncfh62b59O2FhYbz77ruGjiKEEOI3aJaBrfruxJZszpw5uptehBBCKFejn8fm7e2t+zogIICcnJwaPz979ix9+/ZtfLJ6unPnDn/5y1+YNGkSkyZNIiEh4YHbeXt7k5KS0uR5hBBCGJbeK7UA7t69S1BQEOXl5b8tXT2EhobStWtX3n33XW7cuIGbmxuDBw/m0UcfbfJj30sqtfRLCTmVkBGUkVMJGaF15GyNFVq/pvdKLSsrKzZu3IiPjw/fffddnQFCQkK4evUqly5doqCggOnTpzNv3jxiY2OJi4ujsLCQ559/ntmzZ7N69WquXbuGkZERb7zxBkOGDGHQoEE88cQTAHTq1AlLS0vy8/Np3749K1as4MyZM9ja2lJQUFDr+fzv//4vH330EVqtVlfY7O/vz4gRI5gwYUK9fpFSqSWEUILWWKH1a3UObPdWamk0Gnx8fHBzcyMmJoaYmBgAoqKidJVaBw8epKSkBBcXl3qHOHPmDFFRUWg0Gtzd3XFycgKqeh3379+PiYkJixYtwsPDgxdeeIG8vDy8vLzYvXs3Q4cO1e1n//79lJWV0bt3b7Zv3w5AYmIimZmZTJ48udbzOXz4MB4eHvj4+PDXv/6VkpISjh8/TmBgYP1/k0IIIVoEvVZqXb9+nQ8//JCIiIgGhXB1ddWtM3N2dub48eNYWVnh6OioayE5duwYGRkZBAcHA1Uzr6ysLBwcHICqAezNN98kLCwMExMTUlNTmT59OgA9e/akf//+Dz2f7t27Y2try4kTJ7h69SojR46URdpCiFapJVxSVUyl1qFDhygsLGTWrFm619zc3IiMjMTCovaT+HWlVvX39xYbazQatm/fjqWlJQB5eXm6Z77t2LGD8PBwwsPDsbe3B6pqtO7NWT1APux8PDw82LdvH1evXmXhwoV1/WqEEEKRDL3OTVGVWp6enhw4cID4+Hhdr2N8fPxDBzWAAwcOUFZWxq1bt/jHP/7BsGHD7tvmueee44svvgDg4sWLTJo0ibt373LgwAEiIiLYuXOnblADcHJyYu/evWg0GnJycnSf99V2PgAuLi4kJyeTn59f41E3QgghlEPvlVrdu3dvcAgzMzO8vLy4ffs28+fPp3fv3jWerQawcuVKVq9ezaRJkwDYvHkzFhYWBAcHU1payiuvvKLbdt26dXh5eXHhwgXGjx+Pra2trn+ytvOBqhni008/fV9XZX1IpZYQQglaY4XWrxm8UiskJATA4Jf+tFotd+7cYfr06URERNC5c+cGvV8qtfRHCTmVkBGUkVMJGUFy6lOrqNSKiIggLi7uvte7dOmie+aaoaWnpzNv3jz8/PwaPKgJIYRoOZplYFNCXVW/fv1qLDoXQgihTI2u1GqJvvzyS4YPH17nI20eZtOmTfj7++sxlRBCiObULDO25rJv3z42bNjwwLsq6yM5OZm4uDhGjRrV4PdKpZZ+KSGnEjKCMnIqISMoO+fvoUqrmmIGtpSUFD766CNUKhXZ2dk4OzvzyCOPcODAAaBqYXd6ejqBgYGsWLGCjz76iKioKABiY2M5ffr0Q5tECgsL2bp1K6+88grnzp1rcD6p1BJCtGS/hyqtaoq6FFk9OO3atYvIyEg6duxIbGws9vb2tGvXjr59+7Ju3TpGjhzJ9evXuXLlCgC7d+/G3d39oftevXo1ixYton379s1xKkIIIZqIYmZsAE8++SQ2NjYAWFlZ6Tolu3btyi+//KLbzsjIiKlTp7Jnzx7c3d25cePGQxdcx8TEYGNjg5OTE7GxsU17EkIIYSAt6VKqQSu1WhKVSlXj+3uruH5t6tSpzJs3D1NTU9zc3B663/3793P9+nXc3Ny4desWxcXFvPnmmyxfvlwvuYUQoiVoKevbWsU6NkOwtbXF2tqaqKgodu7c+dBtt23bpvs6NjaW1NTUBg9q0jwihGjJfg+NI9Va7cAGMGHCBL755hsee+yxJj+WNI/ojxJyKiEjKCOnEjKC5FQSg1dqNZWKigqWLl2Ki4sLY8c2/WxKBjb9UUJOJWQEZeRUQkaQnPoklyIbQavVMnz4cIYMGcLo0aMBOHnyJEFBQQ/cPjQ0tFlmdUIIIZpeqxzYjIyMSE5OrvHas88+q3uUjhBCiNar0evYvL29dV8HBASQk5MDQFxcHMOGDcPNzQ03Nze2bt3621M+xKVLl5g1axZubm5Mnz6dH3/88YHb3fusNiGEEK1Xo2ds9xYGp6Sk4OfnB8CZM2fw9/fH1dX1t6erh5UrVzJ//nxGjRpFcnIyy5YtY8+ePc1y7HtJpZZ+KSGnEjKCMnIqISMoI2dZeaWhIxhcnQNbRUUFarWaCxcukJ+fj729PR07dgSqHto5ZswY8vLy8PX1JTIykvT0dDIzM/n444+xt7dn1apVdOjQodb9e3t706dPH06ePElpaSnLly9n2LBh+Pv7U1hYyL///W+WLFnCo48+yoYNGygpKcHKyorAwEC6d++Op6cnw4cPB6pmZT///DMA2dnZLFmyhOLi4hqLs3Nzc1m+fDlFRUXk5eUxdepU/vKXv+Dl5YWfnx9Dhw5Fq9Uybtw4duzYUe/P3qRSSwjREuzd8vB1u78HdV6K/P7771GpVERHR/Ptt99SVFSkG0hiYmLw9fWlS5cuhIaGYmVlRefOnXnttdfYs2cPNjY2rF27ts4Qt2/fJi4uji1btuDv709ZWRkAlpaWJCYmMmzYMFauXMmWLVuIi4vjpZdeYtWqVQC4u7vrFmoHBwfrbhYJCgrC3d2d+Ph4BgwYoDvWvn37cHV15csvv2Tv3r1s376dmzdv4uHhofsM7uTJk/To0UNuKBFCCAWqc8Y2cOBALC0tiYyMJCMjg8zMTIqLi2vd/v3339d9PW/ePMaMGVNniGnTpgHg4OBA586dOX/+PIDuIaSZmZlkZWXx6quv6t5z+/Zt3ddarZbNmzdz+vRpPvvsM6DqUumWLVsAmDx5MitXrgRg7ty5HD9+nPDwcC5cuEB5eTl3795l/PjxbN26leLiYuLi4urslhRCiJZKCZdMDVqpdfDgQYKDg5k9ezbu7u4UFBRQ29K3oqIidu3apXuoqFarfWjtVbV7t9FoNJiYVMUyNzfXvdatWzfdjKqyspL8/Hyg6lLpsmXLyM3N5bPPPqNdu//+gVbnNDIyok2bqsnpxo0bycrKwtXVldGjR3Ps2DG0Wi2PPPIII0aMICkpiePHj7NmzZo6cwshREsk69jqkJyczPjx4/Hw8CArK4uUlBScnJwwNjamoqICExMTjI2Nqays5JFHHiEsLIz+/fvz1FNP8fnnn9drxrZ//3769etHeno6v/zyC08++WSNn9vZ2XHr1i1OnjzJs88+y65du9i7dy87duxg06ZN3L59m08//RRTU1Pde4YMGcKePXuYNWsW33zzDaWlpQAcPXqUwMBABgwYwKFDh8jNzUWj0QDg4eHBokWLeP755zEzM2vQL1IqtYQQLYHcPFKPgc3T05PFixeTkJCASqViwIABZGdn88ILL+Dm5kZsbCyjRo3C19eXsLAw3nnnHdRqNSUlJfTs2ZPNmzfXGSIrK4upU6cCsHXr1vtmeaamprz77rusX7+e0tJSLCws2LRpEzdv3iQyMpJu3brh6emp2z4+Pp7Vq1ezZMkSoqOj6du3L23btgVg/vz5LF26FHNzc6ytrenbty/Z2dn06NGDZ555BiMjIzw8PBr0SwRpHtEnJeRUQkZQRk4lZARl5fy9M3illre3NwsWLGDw4MGGjIFWq+Wnn35i2bJl7N69u8Hvl4FNf5SQUwkZQRk5lZARJKc+GfxSpD688cYbXLx48b7XnZ2dm+Pw9bJ9+3bCwsJ49913DR1FCCHEb9AsA1v13Ykt2Zw5c3Q3vQghhFCuRldqGdKXX37J8OHD2bRpU6Pef/v2bVxdXcnOzta9duzYMSZNmsTYsWObvAZMCCFE01FkCfK+ffvYsGEDw4YNa/B7T58+zcqVK8nMzNS9VlJSwvLly9mxYwc2NjbMnz+fw4cPM3LkyHrvVyq19EsJOZWQEZSRUwkZwTA5S0orKPpFWo0aosUNbCkpKXz00UeoVCqys7NxdnbmkUce4cCBA0DV53Lp6ekEBgayYsUKPvroI6KiooCqp1+fPn2awMDAWvf/5ZdfsmbNGpYuXap7LS0tjccff5zu3bsDMGnSJL7++usGDWxSqSWEaAp7t7jRsm8FaXla3MAGVbOqhIQELC0tGTJkCMuWLSM2NpaAgADatWtH3759WbBgAYMGDSIoKIgrV67Qo0cPdu/ezRtvvPHQfa9fv/6+1/Ly8ujcubPu+y5dupCbm6v38xJCiMZo6ExRCTNggzaPGMKTTz6JjY0NAFZWVjg5OQHQtWtXfvnlF912RkZGTJ06lT179uDu7s6NGzdqFB7Xl0ajwcjISPe9Vqut8b0QQhhSQ27fl9v9W+jAplKpanz/sFquqVOnMm/ePExNTXFza1yrtbW1NdevX9d9f/36dbp06dKofQkhhDCsFjmwNYStrS3W1tZERUWxc+fORu3jqaee4vLly/z73/+mW7du7Nu3r8HtI1KpJYRoCiWlFYaOoDiKH9gAJkyYwDfffNPox8yYmZmxceNGFi5cSGlpKSNHjsTFxaVB+5DmEf1RQk4lZARl5FRCRlBOTtECKrV+q4qKCpYuXYqLiwtjxxpu1iQDm/4oIacSMoIyciohI0hOffpdfsZWX1qtluHDhzNkyBDdA0ZPnjxJUFDQA7cPDQ2Vh4cKIUQrp+iBzcjIiOTk5BqvPfvss7rntgkhhPj9aXSllre3t+7rgIAAcnJygKo1Yb6+vkyZMoUZM2bUqK1qCnl5ecyZM4fJkyfj6enJjz/++MDtnJ2dmzyLEEIIw2v0jC01NVX3dUpKCn5+fgAsXbqUcePGMXPmTHbu3Mnbb7/NO++889uT1mLr1q264x05coTAwEBdE0lzkkot/VJCTiVkBGXkVEJGaLqcUpulX3UObBUVFajVai5cuEB+fj729vZ07NgRqHoI6ZgxY3SztM8++4xz586xbds2oOqJ1NWLq2vj7++PmZkZ6enp3Llzh1dffZUpU6YQEhLCDz/8wM8//8yLL77I0KFDUavVFBYWYm5uzqpVq3B0dKzRJJKdnU379u0BKCwsZMmSJVy7do1evXrpnqB9+/Ztli9fTm5uLnl5eTg5ObF+/XqWLl3KwIEDmTZtGlA1I128eHG9F3xLpZYQorGkNku/6hzYvv/+e1QqFdHR0Wg0Gnx8fHBzcyMmJoaYmBgAoqKiCA0NJScnh65du7Jx40ZOnjxJ586dWbVqVZ0hsrKyiI6O5saNG7i7uzN06FAAysrK2L9/PwAzZsxg9erVODo6cvHiRfz8/EhKSqJNm6qrqS4uLuTk5PDBBx8AEBwcjKOjI5988gknTpwgMTERgEOHDuHg4EBwcDBlZWVMnDiRf/3rX3h4eBASEsK0adPIycnh5s2bjWoxEUKIxtDnbFAJM2CDVmoNHDgQS0tLIiMjycjIIDMzk+Li4gduW1FRwdmzZ1m4cCEBAQHExMTg7+/Pjh07HnoMd3d3VCoV1tbWDBgwgFOnTgHQr18/AO7cucOZM2cICAjQvae4uJiCggKsrKwA+Prrr/nxxx95+eWXSUURyJEAACAASURBVExMJDU1VfccuIEDB+oKjl1dXUlLSyMiIoKMjAwKCwspLi5m8ODBrFq1iuzsbOLj4xvdYiKEEI2hr1v05Xb/egxsBw8eJDg4mNmzZ+Pu7k5BQQG1LX3r3Lkzbdu25fnnnweqBpF169bVGeLeyiyNRoOJSVUsc3Nz3WumpqY17na8du0alpaWHDp0iIEDB9K2bVscHBzo2rUrWVlZGBkZ1chZfYwdO3aQlJTEtGnTGDJkCD/99JOuG3LKlCkkJCSQmJhIeHh4nbmFEEK0PHUObMnJyYwfPx4PDw+ysrJISUnByckJY2NjKioqMDExwdjYmMrKSh5//HGsra11zzL7xz/+wZ/+9Kc6QyQmJuLi4sLVq1dJS0tj/fr1Ne5ubNeuHT179tTNpI4ePcrq1as5cOAAcXFxXL58mZdeeomLFy+Sn5+PnZ0dTk5OxMfH06dPH9LS0rhy5QoAR48eZfr06UyaNIn09HTOnTuHRqMBqmaOXl5e9O7du8Hr3aRSSwjRWFKbpV91No+cP3+exYsXA1XlxLa2ttjZ2ZGRkUFGRgaxsbG8/fbbHDlyhLCwMMrLy1mzZg0FBQVYWFiwceNGevbsWev+/f39uXnzJvn5+ZSVlfG3v/0NZ2dnQkJCAFi4cCEAly5d0t08olKpUKvV9OvXj9zcXJYvX87169cxMzNj2bJlPPvss9y+fRt/f38uX76MnZ0d58+f59NPPyUrKwu1Wo2pqSkWFhaYm5szYcIEPD09AfDy8uLFF19kwoQJDfpFSvOI/ighpxIygjJyKiEjSE59aupLkQav1PL392fQoEG4u7sbMgZarZa8vDy8vb3Zt28fpqamDXq/DGz6o4ScSsgIysiphIwgOfXJ4J+x6cOmTZs4duzYfa/37du3OQ5fL0lJSajVat1sTgghhDI1y8C2bNmy5jjMb+Li4tLgRn8hhBAtT6MvRXp7e+tu4w8ICGDBggXY2tqSl5fHypUrycvLw9zcnLfffptu3brpNfS98vLyCAgIID8/nzZt2rB06dIHLgp3dnbms88+a9IsQggBhm0SkUuRraBSa/PmzTg7OzNr1iwyMjLw9vbmyJEjD33qdlOQ5hEhRDVpEjEsxVdqjRkzhueeew6Axx9/nNLSUoqLi6msrGzWSi0hhBAtg+IrtcaNG6fbT3h4OA4ODrRr1461a9dKpZYQwmAMWWsllVp1UEqlVkREBNHR0Xz++ecAUqklhDAoQ33OJZ+xtYJKLaj6nO3w4cNERkZibW0NIJVaQgjxO6X4Sq2IiAhSUlLYuXOn7pE1gFRqCSEMRiqyDEvRlVp//vOfGTRoEBYWFjUGtdDQUNq2bSuVWr+ihEsUoIycSsgIysiphIwgOfVJKrWaiVRqtRxKyKmEjKCMnErICJJTnwz+GZs+SKWWEEKI5tLoga0hzSPNUal19OhRQkND2b59+wN/XlfziFRqCSFE66D45hGNRkNERAQff/wxTz75ZJMdpy5NuSZDn5SwvgWUkVMJGUEZOZWQEaBd+z8YrCpL1J/im0cuXbrEpUuXCAoKqrFerrCwsFmbR6RSS4jWT6qylKFNXRvc2zzy7bffUlRUxPDhwwGIiYnB19eXLl263Nc84uHhweuvv45KpaozRHXzyPbt29m8eTPXr18H/ts84uXlxbJly1iyZAlxcXEEBQWxaNEiAP74xz+yfv16OnToUGOfwcHBODo6snfvXmbNmkV+fj7w3+aR6OhokpKSOHHihK55pHqdnDSPCCGEcrWa5pFfk+YRIURTUMJlUyVkNGillhKaRx5EmkeEEE1BCbfSKyFjU97uX+elyHubR9q3b09KSgqVlZW65hFA1zzSo0cPXfMI0KDmEa1WS05ODmlpaTzzzDM1fn5v8whU3QE5a9ash+6zunkEeGDzyOTJkyktLb2veSQqKgobG5sGN48IIYRoGeqcsXl6erJ48WISEhJQqVQMGDCA7OxsXnjhBdzc3IiNjWXUqFH4+voSFhZGSEgIa9as4a233tI1j9SlpKQEDw8PysrKWLt27QMvL7711luo1WrCwsJQqVRs3boVIyOjWvf5+uuv4+/vz8SJE7Gzs9NdivTx8UGtVhMaGoqFhQX9+/cnOzsbABsbG2xsbJg6dWqdmX9NKrWEaP2kKksZpHnkP6R5pOVQQk4lZARl5FRCRpCc+iTNI81EmkeEEKJ1aJaBrTmaR34raR4RQojWoc6bR2rj7e2t+zogIICcnJwaPz979myzzMju3LnDwoULmTRpElOmTHngzBCq8qakpDR5HiGEEIal90otgLt37xIUFER5eflvS1cP27Zt4/HHHyckJIRLly7h4+PD//3f/zX5cX9NKrX0Swk5lZARlJGzuTOWlFZINVYrptdKrcjISKysrNi4cSM+Pj589913dQYICQnh6tWrXLp0iYKCAqZPn868efOIjY0lLi6OwsJCnn/+eWbPns3q1au5du0aRkZGvPHGGwwZMoQFCxbolh1kZ2frGkjKyspYsWIFZ86cwdbWloKCglrP53//93/56KOP0Gq1ukYTf39/RowYUe/nskmllhDKIdVYrVudA9u9lVoajQYfHx/c3NyIiYkhJiYGgKioKEJDQ7GysuLgwYOUlJQ06POqM2fOEBUVhUajwd3dXdcvmZuby/79+zExMWHRokV4eHjwwgsvkJeXh5eXF7t378bCwgITExPmzp1LcnIya9euBdC1nSQmJpKZmcnkyZNrPZ/Dhw/j4eGBj48Pf/3rXykpKeH48eMEBgY27LcphBDC4PRaqXX9+nU+/PBDIiIiGhTC1dWVtm3bAlWPlzl+/DhWVlY4OjrqWkiOHTtGRkYGwcHBQNXMKysrCwcHBwDCw8PJyclhxowZ9O/fn9TUVKZPnw5Az5496d+//0PPp3v37tja2nLixAmuXr3KyJEjMTMza9B5CCGUozGXP5VwWReUkVMxlVqHDh2isLCwRiuIm5sbkZGRWFjUfhK/rtSq/r66Uqv69e3bt+tqtPLy8ujUqROpqan07NmTLl26YGtrS//+/blw4cJ9lVrVA+TDzsfDw4N9+/Zx9epVFi5cWNevRgihYA1d66WE9WGgjJyKqtTy9PTkwIEDxMfH6+qs4uPjHzqoARw4cICysjJu3brFP/7xD4YNG3bfNs899xxffPEFABcvXmTSpEncvXuXQ4cOERoaClQNdmfOnOHPf/4zTk5O7N27F41GQ05Oju7zvtrOB6pu+U9OTiY/P1+a/YUQQqH0XqlVXV3VEGZmZnh5eXH79m3mz59P7969SUtLq7HNypUrWb16NZMmTQJg8+bNWFhY8Nprr7FixQomTZqEsbExy5cvx9bWFi8vLy5cuMD48eOxtbXVPYS0tvOBqhni008/3agHlkqllhDKIdVYrZvBK7VCQkIADH7pT6vVcufOHaZPn05ERASdO3du0PulUkt/lJBTCRlBGTmVkBEkpz61ikqtiIgI4uLi7nu9S5cuumeuGVp6ejrz5s3Dz8+vwYOaEEKIlqNZBrY5c+YwZ86c5jhUo/Xr16/GonMhhBDK1GSVWu+8847uMmNTysvLY86cOUyePBlPT09+/PHHB27n7Oys+yxNCCFE66X3Sq2ioiI2bNhAQkIC8+bN++0J67B161bGjRvHzJkzOXLkCIGBgURFRTX5cX9NKrX0Swk5lZAR9JtTqqiEEui9Uuvw4cP07NmTl156qV4B/P39MTMzIz09nTt37vDqq68yZcoUQkJC+OGHH/j555958cUXGTp0KGq1msLCQszNzVm1ahWOjo6sX79et6/s7Gzat28PQGFhIUuWLOHatWv06tWL0tJSAG7fvs3y5cvJzc0lLy8PJycn1q9fz9KlSxk4cCDTpk0Dqmakixcvrvdt/1KpJX4PpIpKKIHeK7WmTJkC0KDLkFlZWURHR3Pjxg3c3d0ZOnQoUNX3uH//fgBmzJjB6tWrcXR05OLFi/j5+ZGUlESbNlVXU11cXMjJyeGDDz4AIDg4GEdHRz755BNOnDhBYmIiULWI3MHBgeDgYMrKypg4cSL/+te/8PDwICQkhGnTppGTk8PNmzdlLZsQQiiQXiu1Gsvd3R2VSoW1tTUDBgzg1KlTALo7Ju/cucOZM2cICAjQvae4uJiCggKsrKwA+Prrr/nxxx95+eWXSUxMJDU1lS1btujOoXp9naurK2lpaURERJCRkUFhYSHFxcUMHjyYVatWkZ2dTXx8PG5ubno9RyFai6a4BPt7vKzblJSQUzGVWo3160qt6vqr6kotjUaDqamprs0E4Nq1a1haWnLo0CEGDhxI27ZtcXBwoGvXrmRlZd1XqVV9jB07dpCUlMS0adMYMmQIP/30E1qtFiMjI6ZMmUJCQgKJiYmEh4fr9RyFaC30vUZKCeuuQHLqk8HXsd1bQZWVlUVKSgpOTk66Si0TExNdpVZjJSYm4uLiwtWrV0lLS2P9+vU17m5s164dPXv21M2kjh49yurVqzlw4ABxcXFcvnyZl156iYsXL5Kfn4+dnR1OTk7Ex8fTp08f0tLSuHLlCgBHjx5l+vTpTJo0ifT0dM6dO4dGowGqZo5eXl707t2bxx57rEHnIM0j4vdAGjuEErSISq2SkhI8PDwoKytj7dq1usuL93rrrbdQq9WEhYWhUqnYunUrRkZGLF++nOXLlxMXF4eZmRlbtmyhbdu2vP766/j7+zNx4kTs7Ox0uXx8fFCr1YSGhmJhYUH//v11ywBsbGywsbFh6tSpDT4HaR7RHyXkVEJGUE5OIfTJ4JVa/v7+DBo0CHd3d0PGQKvVkpeXh7e3N/v27cPU1LRB75eBTX+UkFMJGUEZOZWQESSnPhn8UqQ+bNq0iWPHjt33et++fZvj8PWSlJSEWq1GrVY3eFATQgjRcjTLwLZs2bLmOMxv4uLi0qCnfgshhGiZFF+pVe3o0aP4+PjU+nOp1BJCiN8HxVdqaTQaIiIi+Pjjjxv1HDV9kUot/VJCzubMKFVWQtSf4iu1Ll26xKVLlwgKCmLHjh26/UqllmhNpMpKiPqr81LkvZVa3377LUVFRQwfPhyAmJgYfH196dKlS41KLV9f3xqLrutSXam1fft2Nm/ezPXr14H/Vmp5eXmxbNkylixZQlxcHEFBQSxatAiAP/7xj6xfv54OHTrU2Gd1pdbevXuZNWsW+fn5wH8rtaKjo0lKSuLEiRO6Sq3qBeBSqSWEEMrVaiq1fk0qtURr09hLn3JZV38kp/5IpdZDKrVqI5VaorVpzNokpaxpaukZQXLqk8HXsbX0Sq3aSKWWaE2kykqI+lN8pVZtpFLrfkr4lxwoI6cSMgrxeyWVWv8hlVothxJyKiEjKCOnEjKC5NQng1+K1Aep1BJCCNFcpFLrP6RSSwghWodGV2oJIYQQLVGzzNiaSvXnc9nZ2cTGxuLj41PvxpNfy8vLY+XKleTl5WFubs7bb79Nt27d6v1+qdTSr6bKKdVUQrR+ih7YqsXHx7Nt2zaeeOKJRu9j6dKljBs3jpkzZ7Jz507efvtt3nnnnXq/Xyq1lEGqqYRo/RQ1sGm1WjZu3MihQ4fo0qULlZWVJCQkoNVq8fPzo0ePHowYMQIvLy+io6OJiIggMTGR8vJyRo8ezYEDB1CpVPft9+bNm5w7d45t27YB4OHhgZOTU3OfnhBCCD1Q1MCWlJTE2bNn2bdvH0VFRUyePJnAwEDee+89QkNDycjI4KuvvsLLy4vjx49z69Yt8vPzuXjxIv3793/goAZVXZVdu3Zl48aNnDx5ks6dO7Nq1apmPjvRXPR1mfP3fllXn5SQESSnPhm0UqslSU1NZezYsahUKjp27MiIESNq/Ly677GyspKMjAwmTJjAiRMnSE9PZ9SoUbXut6KigrNnz7Jw4UICAgKIiYnB39+/xtMCROuhjzU+SlgrBMrIqYSMIDn1qVWsY9OXX/c/VndKVjMzM8PBwYG9e/diZ2fH4MGDSU5O5tSpUw99Nlznzp1p27Ytzz//PFBVlLxu3boGZZNKLWWQaiohWj9FDWxOTk6Eh4czY8YM7t69yz//+U+efvrpGtuMHDmS999/n5dffplBgwaxbt06unbtqnuG3IP06NEDa2trDh8+zMiRI/nHP/7Bn/70pwZlk+YR/VFKTiFEy6SogW306NGkp6fj6urKo48+Sq9eve7bZtSoUajVagYNGkSHDh3o1KnTQy9DVgsJCWHNmjW89dZbWFhYsHHjxiY4AyGEEE3N4F2RrYXM2PRHCTmVkBGUkVMJGUFy6pN8xqZH3t7e/PLLL/e9PmPGDGbOnGmAREIIIfStVQ1sX375JSEhIbi6uj6wn/JhdzmGhoaya9cuTE1NmTBhAq+++mpTRhVCCNFEWtXAtm/fPjZs2MCwYcMa9L5jx46xd+9edu3axR/+8Af8/Pz45ptvGDu2/nc6SqWWftUnp9RjCSEeRDEDW0pKCh999BEqlYrs7GycnZ155JFHdE/RdnZ2Jj09ncDAQFasWMFHH31EVFQUALGxsZw+fZrAwMAH7vvs2bMMGzYMC4uqwWn48OEcOHCgQQObVGo1P6nHEkI8iGIGNoDTp0+TkJCApaUlQ4YMYdmyZcTGxhIQEEC7du3o27cvCxYsYNCgQQQFBXHlyhV69OjB7t27eeONN2rd75/+9CfefPNN5s+fzx/+8Af+/ve/I/fUKIMhZ6CtafZraErICJJTn6R55D+efPJJbGxsALCystL1OXbt2rXGTSFGRkZMnTqVPXv24O7uzo0bN3jqqadq3a+TkxPu7u54e3tjaWmJk5MTp0+fbtqTEXphqLu/lHDnGSgjpxIyguTUJ7kr8h6/7no0NjauddupU6cyb948TE1NcXNze+h+b9++zdixY3WPvAkLC6N79+6/PbAQQohmp6iBrSFsbW2xtrYmKiqKnTt3PnTb7Oxsli1bxq5du7h79y5fffUV69evb9DxpFKr+Uk9lhDiQVrtwAYwYcIEvvnmGx577LGHbtenTx/Gjh3L5MmTqaysZM6cOTzzzDMNOpYs0NYfpeQUQrRMrbZ5pKKigqVLl+Li4tKguxsbSwY2/VFCTiVkBGXkVEJGkJz6JJ+xNYJWq2X48OEMGTKE0aNHA3Dy5EmCgoIeuH1oaGidszohhBDK0CoHNiMjI5KTk2u89uyzzxIfH2+gREIIIZpLG0MHaIwvv/yS4cOHs2nTpka9//bt27i6upKdnV3j9fLycnx8fEhJSdFHTCGEEAagyBlbY6uzoGqR98qVK8nMzKzxekZGBsuXL+fs2bONyiSVWlWk5koIYWgtbmBryuosqJrtrVmzhqVLl9Z4/auvvmLevHls3769UbmlUquK1FwJIQytxQ1s0HTVWUCt69OqB7rGDmziv/QxK1RCJZASMoIyciohI0hOffrdVWo1VXWWaB6/9VZjpdyu3NIzgjJyKiEjSE59+l3e7t9U1VlCCCFavxY5sDVEQ6qzmpJUalWRmishhKEpfmCD+ldnNSVpHhFCiJahxQ1sgwcPZvDgwbrv//73v+u+Xrhw4X3bV1RUkJycjKenZ4OOc+9+77Vjx44G7UcIIUTL0uIGtoaQ6iwhhBC/1ujmEW9vb93XAQEB5OTkAFUDi7u7O5MmTeKVV17h1q1bvz1lLYyMjIiPj+fGjRtMmTIFT09P2rZtS3x8/H3/FRUVUV5e3mRZhBBCtAyNnrGlpqbqvk5JScHPzw+oGuQ+/PBDevfuzdtvv014eDh/+9vffnvSWmzdupVx48Yxc+ZMjhw5QmBgoG7BdnNqic0j0gIihPg9qnNgq6ioQK1Wc+HCBfLz87G3t6djx44AeHp6MmbMGPLy8vD19SUyMpL9+/ejUqkoLy8nNzcXe3v7h+7f398fMzMz0tPTuXPnDq+++ipTpkwhJCSEH374gZ9//pkXX3yRoUOHolarKSwsxNzcnFWrVuHo6FhjwXV2djbt27cHoLCwkCVLlnDt2jV69epFaWkpUNUTuXz5cnJzc8nLy8PJyYn169ezdOlSBg4cyLRp04CqGenixYvrvS6uJTaPSAuIEOL3qM6B7fvvv0elUhEdHY1Go8HHxwc3NzdiYmKIiYkBICoqitDQUKysrAA4f/48L730EiYmJvWarWVlZREdHc2NGzdwd3dn6NChAJSVlbF//34AZsyYwerVq3F0dOTixYv4+fmRlJREmzZVV1NdXFzIycnhgw8+ACA4OBhHR0c++eQTTpw4QWJiIgCHDh3CwcGB4OBgysrKmDhxIv/617/w8PAgJCSEadOmkZOTw82bN2WxtxBCKFCdA9vAgQOxtLQkMjKSjIwMMjMzKS4ufuh77O3tOXbsGFFRUSxatKjOS4Pu7u6oVCqsra0ZMGAAp06dAqBfv34A3LlzhzNnzhAQEKB7T3FxMQUFBbrB9Ouvv+bHH3/k5ZdfJjExkdTUVLZs2aI7h+7duwPg6upKWloaERERZGRkUFhYSHFxMYMHD2bVqlVkZ2cTHx/fahZ7/7paRwlVO6CMnErICMrIqYSMIDn1yaCVWgcPHiQ4OJjZs2fj7u5OQUEBtT10u7S0lH/+85+6OxQnT55cr0fL3NssotFoMDGpimVubq57zdTUtMbz1K5du4alpSWHDh1i4MCBtG3bFgcHB7p27UpWVhZGRkY1clYfY8eOHSQlJTFt2jSGDBnCTz/9hFarxcjIiClTppCQkEBiYiLh4eF15laCe9etKWUdmxJyKiEjKCOnEjKC5NSnpq7UqvOuyOTkZMaPH4+Hhwft27cnJSWFyspKjI2NqaioapkwNjamsrISExMTAgMDOXPmDACJiYkMGDCgzhCJiYlotVpycnJIS0vjmWeeqfHzdu3a0bNnT93AdvToUWbNmgVAXFwcX375JQAXL14kPz8fOzs7nJycdNunpaVx5coV3XunT5/O5MmTKS0t5dy5c2g0GqBq5hgVFYWNjY0sCxBCCIUy0tY2/fqP8+fPs3jxYqCqw9HW1hY7OzsyMjLIyMggNjaWt99+myNHjhAWFkZubi5vvvkmlZWVPPbYY6xduxZra+ta9+/v78/NmzfJz8+nrKyMv/3tbzg7OxMSEgL8d1H2pUuXdDePqFQq1Go1/fr1Izc3l+XLl3P9+nXMzMxYtmwZzz77LLdv38bf35/Lly9jZ2fH+fPn+fTTT8nKykKtVmNqaoqFhQXm5uZMmDBBt8Dby8uLF198kQkTJujlF2xIv74rUgn/kgNl5FRCRlBGzubMWFlZQUHBdSoqyhr83jZt2uj+EdyStbScbdoY84c/WGBh0QEjIyOg6WdsdQ5sTc3f359Bgwbh7u5uyBhotVry8vLw9vZm3759mJqaNuj9UqmlP0rIqYSMoIyczZkxP/9nzM0foW3b9rq/ZOvLxKQNFRUtZ8CoTUvKqdVqqaysoKioEK1WS8eOXYBW0u6/adMmjh07dt/rffv2bY7D10tSUhJqtVo3mxNCtD4VFWW0bWvd4EFNNI6RkREmJiosLTuRm5vdbMdtloFt2bJlzXGY38TFxQUXFxdDxxBCNDEZ1JqfkVEboPmuaCm6UgsgLy+PuXPn4ubmxtSpU0lOTn7gds7OzmRnN9+/GIQQQhiG4iu1Nm/ejLOzM7NmzSIjIwNvb2+OHDny0IeTNgWp1BJCmdq1/wPmZvq/eFWf///OnTvL7t278Pdfpffj/9rZs2c4dOjvvPba601+LENTfKXWmDFjeO655wB4/PHHKS0tpbi4mMrKSqnUkkotIepkbmbCpDfi696wgerz/1+fPo74+zvq/dgPkpl5mYKCm81yLENTfKXWuHHjdPsJDw/HwcGBdu3asXbtWqnUEkK0aN99d5JPPw0FwN6+D2lpP1BWVsYrrywkJiaKzMwMpk/3Yvr0WYSHf0xu7jUyMy9z61Yhbm7ueHnNRqPREBy8hZMnT2BkBOPHu+LlNZvvvjvJhx8GU1mpoUuXLvz003nu3r3L9u3h/M//TGfDhiCuX88jP/86zz47CH//VXz//Sl27NiGubk5mZmX6dWrN2vWrP/PGBDJ7t27MDY2ZsiQ4bz22uvcvHmDt956k9zcXNq0acP8+X4MHDi4jrNueq2mUisiIoLo6Gg+//xzAKnU+g+p1Go6SsgIysjZXBnz8tpgYtLoWwsarK5jGRu30d3MYmQE27Z9TljYx7z77lt8/nk0BQUFzJ49k1mzvGnTxojz53/kk08i0GgqmTNnFoMGDebMmXSuX88jMjKa8vIyXnvNl169emFu/geysq6we3cCFhbt2LdvD999d4q5c/8f33zzNfb29mzc+Bbl5eXMnOnBpUvnMTZuw5kzaURHx/Loo52ZN8+HkyeP06nTo8TFfUVExOeYm/+Bv/51ARcvnuPzzz9j0qQpjBgxkvz868yfP5fPPttJ27Zt7zvXNm3a1Phzlkqth1RqQdXnbIcPHyYyMlK3GFwqtapIpVbTUEJGUEbO5syo0WiadY1XXceqrNTo/p4aNGgIFRUaunSxxtHxz5iYmNG5szVFRUVUVGjQaLSMHj0OU9OqvxeHDh1BamoqZ8+m4+IyEa3WCBMTM8aNG09qagpDh46ge/fHMTdvq3u/VqulokKDs/NYzp49wxdffE5m5mUKC29RVHQHrVbLE0/0omPHzmg00KNHTwoLb3H5ciZDhw7H3LxqwHrnnaqy+RMnUsjMzCQ09EMAysvLuXLlCn/84/0fQWk0Gt2fs1Rq8fBKrYiICFJSUti5c2eNhhOp1BJCKEn1P+iBWm9+qzkJ0GJiYnxfMUTVouhKAMzMzB64n6++iuKDD4KxtLTif/5nOk888YRugL13HW/1BKEq23+XSeTnX6eoqIjKSg3BwR8SEfEFERFfEBq6HTu73g078SZQ54zN09OTxYsXk5CQgEqlYsCAAWRnZ/PCCy/g5uZGbGwso0aNwtfXl7CwMLZupuJAvgAAC0NJREFU3crq1at1lVr3Pi+tNiUlJXh4eFBWVsbatWt1lxfv9dZbb6FWqwkLC0OlUrF161YA3n//fSwsLGosPwgNDeX111/H39+fiRMnYmdnp7sU6ePjg1qtJjQ0FAsLC/r3769bBmBjY4ONjQ1Tp06t32/vHuErxzb4PU2tpLTC0BGEEHp05Mgh/ud/ZlBSUsLRo0fYtGkrxsbGJCYmMGTIcMrLy0lKSuTFF1+6773VExCommlNnuzO2LEunDt3lgsXfkKj0egeA/ZrTz3Vn7VrVzJ37nxMTU1Rq1fg4zOXZ555ltjYGObMmcflyxn4+f0/vvpqD488cv+lyOZU58Bmb2/P3r17H7rNihUrWLFiBQDdu3cnNja2QSFcXFzuq9Sq7ois1qtXL3bs2HHfe0+cOFHrft977737XuvWrRtJSUn3vV5dqZWfn6+7lNoQSqjUEkLcr6S0gr1b9P+ZelP8w9LMzAw/v3ncuXMHb++XeOIJO7p370FW1hXmzJlJRUUFLi4TGDnyeb777mSN9zo4/IlPPw3lww9DmDbNi7ff3sDnn2+jbVsL+vbtx88/X8XWttsDj2tv3wd392m88spLaDRaRo58noEDB/PEE3Zs3rweH58ZaLVaVq1aa/BBDZqpK/JhlVqVlZUtoivy66+/1lVqNaaBRAkDmxI+bwFl5FRCRlBGzubMeO3av7G2frxR7zV0B2N4+McAzJ07/6HbGTpnbe793beKrkip1BJCCNFcFF+pVe3o0aP4+PjU+nOp1BJCKNncufPrnK2JKoqv1NJoNERERPDxxx/z5JNPNtlx6tJSKrWkRksI8Xun+EqtS5cucenSJYKCgmrcXFJYWPi7rNSSGi0hHq563apoPlqthnuXCzS1Oi9F3lup9e2331JUVMTw4cMBiImJwdfXly5duugqtVQqFefPn2fkyJGkpKQwceLEOkNUV2pt376dzZs3c/36deC/lVpeXl4sW7aMJUuWEBcXR1BQEIsWLQLgj3/8I+vXr6dDhw419hkcHIyjoyN79+5l1qxZ5OfnA/+t1IqOjiYpKYkTJ07oKrWq171JpZYQrZOJiSl37vxSa8mE0K+qBeHlFBbm6xaWN4dWU6n1a7/nSq2H1RMpoV4JlJFTCRlBGTmbK6OlpTlZWVlcvy6ftzcXExNjrKysePTRR2usk5NKrToqtR7k/7d3dyFN/WEcwL9YJoiC/8IVdVF00dtFLyAUXTi8yLXJnK3ESu1NUyNCDI0suqg7u5IlWFFdFF4VqCmiFRQRCpr0YrHypZvS1NSFNTJdfv8X++/Qsq36e45nW88HgjxnZ/vy7MHH5en3+5uX1Ap063Q43PoNhEfOcMgIhEfOuc4YH5+I+P8xR8OhlkDo5hwddSt/lyW1EHxJrUBkSS0hhPg7hfWSWsF+Afy3Lqkly2gJIf52c7LySDAnT54MiZVHfEtq5ebmorGx0W8h0N/hcrlDfuWRRYviMDr6We8YvxQOOcMhIxAeOcMhIyA51bRoURxcLjf++Ueb5bfmZOWRYEtqhYqWlhZlSa0/HWoANHuD1BYq/9/uV8IhZzhkBMIjZzhkBCSnmrT8nqn7JzYhhBBCTXO3lawQQggxB2SwCSGEiCgy2IQQQkQUGWxCCCEiigw2IYQQEUUGmxBCiIgig00IIUREkcEmhBAioshg+09DQwMsFgtSU1NRU1Mz47zT6YTdbofJZMLp06eVBaAHBgaQnZ2N7du348iRI3C7vStYj4+Po6CgAGazGdnZ2X57zJWVlcFsNmPHjh3o6+vTNWdfXx+ys7Nhs9mQlZUFp9MJwLsn3aZNm2Cz2WCz2ZCXl6dbxvb2dmzevFnJ4tu+KFCN9cppt9uVjCaTCevWrcPIyIgutfSprKzEhQsXlK9DrS8D5dSiL7XIqUVvqp1Ri76cTc7Ozk7s2rULNpsN+/fvR39/PwCVe5OCg4ODTElJocvlotvtptVqZU9Pj99j0tLS+OTJE5JkeXk5a2pqSJIFBQVsbGwkSVZVVfH8+fMkybNnz/LSpUskydraWhYXF5Mkr1y5wjNnzpAk29vbmZmZqWvO3bt38/79+yTJ1tZWWq1WkmRzc7OS809okfHq1au8ePHijNcKVGO9cn6vrKyM1dXVJPWp5fj4OMvLy7l+/Xo6HA7l8aHWl4Fyqt2XWuVUuze1yPg9NfpytjlTUlLodDpJkjdv3mRRURFJdXtTPrEBaG1txZYtW5CQkIDY2FiYTCY0Nzcr5/v7+zExMYGNGzcC8P4E1NzcjKmpKXR0dMBkMvkdB7w7dVutVgDezU0fPnyIqakpPHjwAOnp6QC8G6COjY1hYGBAt5yZmZnKjuirV6/G+/fvAQBdXV3o7u6GzWbDvn378Pr1a90ydnV14dGjR7BarSgqKlIyBqqxXjl92tra8OrVKxw+fFiXWgLefRRXrFiBgwcP+j1nKPVlsJxq96VWOdXuTS0y+qjVl7PJOTk5ieLiYqxZswaA/3urZm/KYAMwPDyMxMRE5WuDwYChoaGA5xMTEzE0NASXy4W4uDhlY1Tf8R+vmT9/PuLi4jA2NvbT5xocHNQtp91uVzZhdTgcyiaxMTExSE9PR21tLfLy8nD06FFMTk7qkjE+Ph65ubloaGiA0WhESUnJjOf6vsa/Q4ucPg6HAyUlJUpd57qWAJCRkYGCggK/TXx/vEbvvgyWU+2+1Cqn2r2pRUYftfpyNjkXLFgAm80GwLuBdFVVlfLeqtmbMtjgLfD3e7vxvx21f3X+x8cBCLhHHElERUXNuMZ3XM+cJFFRUYFnz57h1KlTAIBjx45h7969iIqKgtFoRGxsLN68eaNLxnPnziE11bvf3Z49e9Db24tPn2buEBwKtezp6YHL5UJKSopybK5r+Sf07MvfyaZWX2qVU+3e1KqWavalGjknJydRWloKj8eDwsLCn77GbHpTBhuAJUuW+P1y98OHDzAYDAHPj4yMwGAwYOHChfj06RO+ffs24zqDwYCRkREAgMfjgdvtRkJCAhYvXozh4eEZz6VXTo/Hg9LSUnR1deH69euIj48HANy4cQMul0t5LpLKp5S5zDg9PY3q6mrluM+8efMC1vh3aFFLALh37x4sFovfa811LYMJpb4MRu2+1CKnFr2pRS0Bdftytjndbjfy8/Ph8XhQXV2N6OhoAOr2pgw2AFu3bkVbWxvGxsbw5csX3LlzB8nJycr5ZcuWISYmBp2dnQCA+vp6JCcnIzo6GklJSWhqagIA1NXVKdcZjUbU1dUBAJqampCUlITo6GgYjUbU19cDAB4/foyYmBgsXbpUt5wVFRX4/Pkzrl27pnzzAICOjg7cunULgPfOr+npaaxcuXLOM0ZFReHu3btoaWlRjm/YsAGxsbEBa6xXLQHg6dOnSEpK8nutua5lMKHUl8Go3Zda5NSiN7WoJaBuX842Z1lZGZYvX47Kykq/vS9V7c1f3l7yl7h9+zbT0tKYmprKy5cvkyTz8/P5/PlzkqTT6eTOnTtpMpl4/Phxfv36lST57t075uTk0Gw289ChQ/z48SNJ0uVysbCwkBaLhVlZWXz79i1JcmJigidOnKDFYmFGRgZfvHihW87R0VGuXbuW27ZtY3p6uvKH9N71dODAAaalpdFutyt3MelRy+7ubmZlZdFisTAnJ4cDAwNBa6xXTpI0m83s7e31ex09aunjcDj87pALtb78WU6t+lLtnKQ2val2RlL9vvy/OV++fMlVq1bRYrEo72t+fj5JdXtTNhoVQggRUeSfIoUQQkQUGWxCCCEiigw2IYQQEUUGmxBCiIgig00IIUREkcEmhBAioshgE0IIEVFksAkhhIgo/wIeHlRHr81PhgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate feature importance plot\n",
    "\n",
    "feature_importance = pd.DataFrame(model.feature_importances_, \n",
    "                                  index = df.columns[:-1], \n",
    "                                  columns = ['importance']).sort_values('importance', ascending = True)\n",
    "feature_importance[:20].plot.barh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [],
   "source": [
    "# venture into XGBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.997693 using {'max_depth': 5, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled XGBoost\n",
    "\n",
    "from xgboost import XGBClassifier\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([400, 500, 600]), max_depth=[5, 10, 15])\n",
    "model = XGBClassifier(random_state = seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.997666 (0.006216) with: {'max_depth': 5, 'n_estimators': 400}\n",
      "0.997666 (0.006175) with: {'max_depth': 5, 'n_estimators': 500}\n",
      "0.997693 (0.006102) with: {'max_depth': 5, 'n_estimators': 600}\n",
      "0.997488 (0.006339) with: {'max_depth': 10, 'n_estimators': 400}\n",
      "0.997529 (0.006218) with: {'max_depth': 10, 'n_estimators': 500}\n",
      "0.997502 (0.006299) with: {'max_depth': 10, 'n_estimators': 600}\n",
      "0.997679 (0.005968) with: {'max_depth': 15, 'n_estimators': 400}\n",
      "0.997652 (0.005968) with: {'max_depth': 15, 'n_estimators': 500}\n",
      "0.997693 (0.005847) with: {'max_depth': 15, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "              colsample_bynode=1, colsample_bytree=1, gamma=0,\n",
       "              learning_rate=0.1, max_delta_step=0, max_depth=5,\n",
       "              min_child_weight=1, missing=None, n_estimators=600, n_jobs=1,\n",
       "              nthread=None, objective='binary:logistic', random_state=7,\n",
       "              reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,\n",
       "              silent=None, subsample=1, verbosity=1)"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = XGBClassifier(random_state=seed, max_depth= 5, n_estimators=600)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9878504672897197\n",
      "\n",
      "Accuracy\n",
      "0.9658119658119658\n",
      "\n",
      "Confusion Matrix\n",
      "[[214   0]\n",
      " [  8  12]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.96      1.00      0.98       214\n",
      "         1.0       1.00      0.60      0.75        20\n",
      "\n",
      "    accuracy                           0.97       234\n",
      "   macro avg       0.98      0.80      0.87       234\n",
      "weighted avg       0.97      0.97      0.96       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "# venture into LightGBM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.999180 using {'max_depth': 15, 'n_estimators': 400}\n"
     ]
    }
   ],
   "source": [
    "# Tune scaled LightGBM\n",
    "\n",
    "from lightgbm import LGBMClassifier\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=np.array([400, 500, 600]), max_depth=[5, 10, 15])\n",
    "model = LGBMClassifier(random_state = seed)\n",
    "kfold = StratifiedKFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.998744 (0.003216) with: {'max_depth': 5, 'n_estimators': 400}\n",
      "0.998757 (0.003175) with: {'max_depth': 5, 'n_estimators': 500}\n",
      "0.998812 (0.003016) with: {'max_depth': 5, 'n_estimators': 600}\n",
      "0.999097 (0.001870) with: {'max_depth': 10, 'n_estimators': 400}\n",
      "0.999111 (0.001915) with: {'max_depth': 10, 'n_estimators': 500}\n",
      "0.999125 (0.001875) with: {'max_depth': 10, 'n_estimators': 600}\n",
      "0.999180 (0.001598) with: {'max_depth': 15, 'n_estimators': 400}\n",
      "0.999152 (0.001665) with: {'max_depth': 15, 'n_estimators': 500}\n",
      "0.999166 (0.001641) with: {'max_depth': 15, 'n_estimators': 600}\n"
     ]
    }
   ],
   "source": [
    "means = grid_result.cv_results_[ \"mean_test_score\" ]\n",
    "stds = grid_result.cv_results_[ \"std_test_score\" ]\n",
    "params = grid_result.cv_results_[ \"params\" ]\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LGBMClassifier(boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,\n",
       "               importance_type='split', learning_rate=0.1, max_depth=15,\n",
       "               min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n",
       "               n_estimators=400, n_jobs=-1, num_leaves=31, objective=None,\n",
       "               random_state=7, reg_alpha=0.0, reg_lambda=0.0, silent=True,\n",
       "               subsample=1.0, subsample_for_bin=200000, subsample_freq=0)"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare the model\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = LGBMClassifier(random_state=seed, max_depth= 15, n_estimators=400)\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9873831775700934\n",
      "\n",
      "Accuracy\n",
      "0.9743589743589743\n",
      "\n",
      "Confusion Matrix\n",
      "[[214   0]\n",
      " [  6  14]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.97      1.00      0.99       214\n",
      "         1.0       1.00      0.70      0.82        20\n",
      "\n",
      "    accuracy                           0.97       234\n",
      "   macro avg       0.99      0.85      0.90       234\n",
      "weighted avg       0.98      0.97      0.97       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [],
   "source": [
    "# venture into Voting Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "VotingClassifier(estimators=[('model_AB',\n",
       "                              AdaBoostClassifier(algorithm='SAMME.R',\n",
       "                                                 base_estimator=None,\n",
       "                                                 learning_rate=0.1,\n",
       "                                                 n_estimators=600,\n",
       "                                                 random_state=7)),\n",
       "                             ('model_GBM',\n",
       "                              GradientBoostingClassifier(criterion='friedman_mse',\n",
       "                                                         init=None,\n",
       "                                                         learning_rate=0.1,\n",
       "                                                         loss='deviance',\n",
       "                                                         max_depth=10,\n",
       "                                                         max_features=None,\n",
       "                                                         max_leaf_nodes=None,\n",
       "                                                         min_impurity_decrease=0.0,\n",
       "                                                         min_impurity...\n",
       "                                             importance_type='split',\n",
       "                                             learning_rate=0.1, max_depth=15,\n",
       "                                             min_child_samples=20,\n",
       "                                             min_child_weight=0.001,\n",
       "                                             min_split_gain=0.0,\n",
       "                                             n_estimators=400, n_jobs=-1,\n",
       "                                             num_leaves=31, objective=None,\n",
       "                                             random_state=7, reg_alpha=0.0,\n",
       "                                             reg_lambda=0.0, silent=True,\n",
       "                                             subsample=1.0,\n",
       "                                             subsample_for_bin=200000,\n",
       "                                             subsample_freq=0))],\n",
       "                 flatten_transform=True, n_jobs=-1, voting='soft',\n",
       "                 weights=[1, 0, 0, 1, 0, 1])"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import VotingClassifier\n",
    "\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "\n",
    "model_AB = AdaBoostClassifier(random_state=seed, learning_rate = 0.1, n_estimators=600)\n",
    "model_GBM = GradientBoostingClassifier(random_state=seed, max_depth = 10, n_estimators=200)\n",
    "model_RF = RandomForestClassifier(random_state=seed, n_estimators=700)\n",
    "model_ET = ExtraTreesClassifier(random_state=seed, max_depth= 10, n_estimators=600)\n",
    "model_XGB = XGBClassifier(random_state=seed, max_depth= 5, n_estimators=600)\n",
    "model_LGBM = LGBMClassifier(random_state=seed, max_depth= 15, n_estimators=400)\n",
    "\n",
    "model = VotingClassifier(estimators=[('model_AB', model_AB), \n",
    "                                     ('model_GBM', model_GBM), \n",
    "                                     ('model_RF', model_RF), \n",
    "                                     ('model_ET', model_ET),\n",
    "                                     ('model_XGB', model_XGB),\n",
    "                                     ('model_LGBM', model_LGBM)], \n",
    "                         weights = [1, 0, 0, 1, 0, 1], \n",
    "                         voting='soft',\n",
    "                         n_jobs=-1)\n",
    "\n",
    "model.fit(rescaledX, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate prediction on validation dataset\n",
    "\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "predictions_proba = model.predict_proba(rescaledValidationX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC\n",
      "0.9934579439252337\n",
      "\n",
      "Accuracy\n",
      "0.9743589743589743\n",
      "\n",
      "Confusion Matrix\n",
      "[[214   0]\n",
      " [  6  14]]\n",
      "\n",
      "Classification Report\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.97      1.00      0.99       214\n",
      "         1.0       1.00      0.70      0.82        20\n",
      "\n",
      "    accuracy                           0.97       234\n",
      "   macro avg       0.99      0.85      0.90       234\n",
      "weighted avg       0.98      0.97      0.97       234\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# generate confusion matrix\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "\n",
    "print('AUC')\n",
    "print(roc_auc_score(Y_validation, predictions_proba[:,1]))\n",
    "print()\n",
    "\n",
    "print('Accuracy')\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Confusion Matrix')\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print()\n",
    "\n",
    "print('Classification Report')\n",
    "print(classification_report(Y_validation, predictions))\n",
    "print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Adjusting Prediction Cut-Off\n",
    "- Typical cut-off for prediction is 0.5. Predicted value greater than 0.5 is consider posiitve, and below negative. \n",
    "- Currently our voting model errors on the side of false negatives. By lowering the cut-off point, we may be able to flip a few cases to positives. But we want to calibrate the cut-off point carefully so as to increase true positives as much as possible, without incurring huge increase in false positives. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pin point datapoints our model fails to correctly classify "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1      0.0\n",
       "126    0.0\n",
       "144    0.0\n",
       "151    0.0\n",
       "159    0.0\n",
       "224    0.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(predictions)[pd.Series(predictions) != pd.Series(Y_validation)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.67663781 0.32336219]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[1])\n",
    "print(predictions_proba[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.6897754 0.3102246]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[126])\n",
    "print(predictions_proba[126])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.64486502 0.35513498]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[144])\n",
    "print(predictions_proba[144])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.73887716 0.26112284]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[151])\n",
    "print(predictions_proba[151])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.66033935 0.33966065]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[159])\n",
    "print(predictions_proba[159])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[0.6922889 0.3077111]\n"
     ]
    }
   ],
   "source": [
    "print(predictions[224])\n",
    "print(predictions_proba[224])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[212   2]\n",
      " [  2  18]]\n"
     ]
    }
   ],
   "source": [
    "custom_cut = np.where(predictions_proba > 0.31, 1, 0)\n",
    "print(confusion_matrix(Y_validation, custom_cut[:,1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "# persist preferred model to file\n",
    "\n",
    "import pickle\n",
    "\n",
    "f = open('20190516_model_et.save', 'wb')\n",
    "pickle.dump(model, f, protocol = pickle.HIGHEST_PROTOCOL)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "# retrieve preferred model from file\n",
    "\n",
    "# f = open('20190516_model_et.save', 'rb')\n",
    "# return_tree = pickle.load(f)\n",
    "# f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10. Other Possible Explorations\n",
    "\n",
    "Should our prediction fail to meet our expectation, there are still options we can explore. \n",
    "\n",
    "- Talk to domain expert, and understand the meaning of the 8 telemetry attributes. Knowing what those attributes mean, and why they are collected in the first place should give us better insight on how to construct the features to capture the device failure dynamics. \n",
    "- Collect more data, especially failure data points with diverse range across the measured attributes should enable us to train the model with more confidence, instead of relying on resampling or synthetic data points for the purpose.  \n",
    "- Try other resampling methods. We have not exhausted all possible options. We might get some mileage from both undersampling of majority class, and over-sampling of minority class at the same time to yield better predictions.  \n",
    "- We can experiment with more leading attribute readings, instead of just previous 1 to 3 days. \n",
    "- In model scoring, we treat the cost of making false positives and false negatives equal. If we know better the relative cost of making false positives vs. false negatives, we can factor this into optimization. The revised prediction should make more economic sense in a production setting. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Summary\n",
    "\n",
    "All in all, we successfully build a model to predict device failure reasonably well. We pull in most of the data science toolsets to arrive at the milestone. We relentlessly cleanse the data. We extract and build new features. We normalize the data to be comparable across devices with different length of historical information. We fit a series of models from naive baseline, to families of classification models including KNN and SVM, to ensembles of averaging and boosting schools. We even experiment with voting model taking in best of all ensembles. We go the extra mile adjusting the prediction cut-off point to further optimize. In the end, we use 1,167 device records to train our model, and correctly classify 18 out of the 20 actual failure cases from validation pool of 234. The model precision is 90%, and recall also 90%. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}