01_ML0103EN-Clas-K-Nearest-neighbors-CustCat__M.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"text-align:center\">\n",
    "    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2022-01-01\" target=\"_blank\">\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\">\n",
    "    </a>\n",
    "</p>\n",
    "\n",
    "# K-Nearest Neighbors\n",
    "\n",
    "Estimated time needed: **25** minutes\n",
    "\n",
    "## Objectives\n",
    "\n",
    "After completing this lab you will be able to:\n",
    "\n",
    "*   Use K Nearest neighbors to classify data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this Lab you will load a customer dataset, fit the data, and use K-Nearest Neighbors to predict a data point. But what is **K-Nearest Neighbors**?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**K-Nearest Neighbors** is a supervised learning algorithm. Where the data is 'trained' with data points corresponding to their classification. To predict the class of a given data point, it takes into account the classes of the 'K' nearest data points and chooses the class in which the majority of the 'K' nearest data points belong to as the predicted class.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Here's an visualization of the K-Nearest Neighbors algorithm.\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/images/KNN_Diagram.png\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, we have data points of Class A and B. We want to predict what the star (test data point) is. If we consider a k value of 3 (3 nearest data points), we will obtain a prediction of Class B. Yet if we consider a k value of 6, we will obtain a prediction of Class A.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this sense, it is important to consider the value of k. Hopefully from this diagram, you should get a sense of what the K-Nearest Neighbors algorithm is. It considers the 'K' Nearest Neighbors (data points) when it predicts the classification of the test point.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Table of contents</h1>\n",
    "\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "    <ol>\n",
    "        <li><a href=\"#About-the-dataset\">About the dataset</a></li>\n",
    "        <li><a href=\"#Data-Visualization-and-Analysis\">Data Visualization and Analysis</a></li>\n",
    "        <li><a href=\"#classification\">Classification</a></li>\n",
    "    </ol>\n",
    "</div>\n",
    "<br>\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load required libraries\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.11/site-packages (1.5.2)\n",
      "Requirement already satisfied: numpy>=1.19.5 in /opt/conda/lib/python3.11/site-packages (from scikit-learn) (2.1.3)\n",
      "Requirement already satisfied: scipy>=1.6.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn) (1.14.1)\n",
      "Requirement already satisfied: joblib>=1.2.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn) (1.4.2)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn) (3.5.0)\n",
      "Requirement already satisfied: matplotlib in /opt/conda/lib/python3.11/site-packages (3.9.2)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (1.3.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (4.55.0)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (1.4.7)\n",
      "Requirement already satisfied: numpy>=1.23 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (2.1.3)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (24.0)\n",
      "Requirement already satisfied: pillow>=8 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (11.0.0)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (3.2.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (2.9.0)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
      "Requirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (2.2.3)\n",
      "Requirement already satisfied: numpy>=1.23.2 in /opt/conda/lib/python3.11/site-packages (from pandas) (2.1.3)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas) (2.9.0)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas) (2024.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.11/site-packages (from pandas) (2024.2)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)\n",
      "Requirement already satisfied: numpy in /opt/conda/lib/python3.11/site-packages (2.1.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install scikit-learn\n",
    "!pip install matplotlib\n",
    "!pip install pandas \n",
    "!pip install numpy \n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn import preprocessing\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"about_dataset\">\n",
    "    <h2>About the dataset</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Imagine a telecommunications provider has segmented its customer base by service usage patterns, categorizing the customers into four groups. If demographic data can be used to predict group membership, the company can customize offers for individual prospective customers. It is a classification problem. That is, given the dataset,  with predefined labels, we need to build a model to be used to predict class of a new or unknown case.\n",
    "\n",
    "The example focuses on using demographic data, such as region, age, and marital, to predict usage patterns.\n",
    "\n",
    "The target field, called **custcat**, has four possible values that correspond to the four customer groups, as follows:\n",
    "1- Basic Service\n",
    "2- E-Service\n",
    "3- Plus Service\n",
    "4- Total Service\n",
    "\n",
    "Our objective is to build a classifier, to predict the class of unknown cases. We will use a specific type of classification called K nearest neighbour.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's read the data using pandas library and print the first five rows.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>tenure</th>\n",
       "      <th>age</th>\n",
       "      <th>marital</th>\n",
       "      <th>address</th>\n",
       "      <th>income</th>\n",
       "      <th>ed</th>\n",
       "      <th>employ</th>\n",
       "      <th>retire</th>\n",
       "      <th>gender</th>\n",
       "      <th>reside</th>\n",
       "      <th>custcat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>13</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>64.0</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>136.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>68</td>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>33</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>33.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>23</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>30.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   region  tenure  age  marital  address  income  ed  employ  retire  gender  \\\n",
       "0       2      13   44        1        9    64.0   4       5     0.0       0   \n",
       "1       3      11   33        1        7   136.0   5       5     0.0       0   \n",
       "2       3      68   52        1       24   116.0   1      29     0.0       1   \n",
       "3       2      33   33        0       12    33.0   2       0     0.0       1   \n",
       "4       2      23   30        1        9    30.0   1       2     0.0       0   \n",
       "\n",
       "   reside  custcat  \n",
       "0       2        1  \n",
       "1       6        4  \n",
       "2       2        3  \n",
       "3       1        1  \n",
       "4       4        3  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/data/teleCust1000t.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"visualization_analysis\">\n",
    "    <h2>Data Visualization and Analysis</h2> \n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let’s see how many of each class is in our data set\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "custcat\n",
       "3    281\n",
       "1    266\n",
       "4    236\n",
       "2    217\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['custcat'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 281 Plus Service, 266 Basic-service, 236 Total Service, and 217 E-Service customers\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can easily explore your data using visualization techniques:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[<Axes: title={'center': 'income'}>]], dtype=object)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGzCAYAAADNKAZOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAqhklEQVR4nO3df1iVdZ7/8RcoHEQ9ICogKajV+mNUVBrxbFmNImhsY+nOpOMalqs7LjYVjWvsNv5qdnR1rmzHIevay1+z5dTYpk1mKmpqjfiL0fJHcamr0aRAyQD+SDjC5/uHy/31FlMRBP3wfFzXueR87s/53J/7fX2gV/c597kDjDFGAAAAlgls7AkAAADcDIQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAV7Vs2TIFBATo+PHjjT0VAKgVQg4AALBSAPeuAnA1lZWV8vv98ng8CggIaOzpAMB1I+QAAAAr8XYVgKu6/DM5nTt31t/93d/p448/1oABAxQSEqKuXbvqd7/7XY3XlpSU6Nlnn1Xnzp3l8XjUsWNHPf744/rmm2+cPkVFRZowYYKioqIUEhKi+Ph4LV++3DXO8ePHFRAQoF//+tfKyspS165dFRoaquTkZH355ZcyxujFF19Ux44d1aJFC40YMULFxcU15vPBBx9o0KBBatmypVq3bq3U1FQdPHiwfgsG4JbRvLEnAOD2c+TIEf393/+9JkyYoLS0NC1ZskTjx49XQkKCvve970mSzpw5o0GDBumzzz7Tk08+qf79++ubb77RH//4R/3lL39Ru3bt9O233+rBBx/UkSNHNGXKFHXp0kUrV67U+PHjVVJSoqefftq13zfeeEMVFRV66qmnVFxcrHnz5unHP/6xBg8erC1btmjatGk6cuSIFi5cqJ///OdasmSJ89r//u//VlpamlJSUvQf//EfOnfunBYtWqT77rtPe/fuVefOnRuyhAAaggGAq1i6dKmRZI4dO2aMMSYuLs5IMtu2bXP6FBUVGY/HY5577jmnbfr06UaSeeedd2qMWVVVZYwx5uWXXzaSzOuvv+5sq6ioMD6fz7Rq1cqUlZUZY4w5duyYkWTat29vSkpKnL6ZmZlGkomPjzd+v99pHzNmjAkODjbnz583xhhz+vRpEx4ebiZOnOiaR0FBgQkLC6vRDsAOvF0FoNZ69uypQYMGOc/bt2+vbt266X//93+dtv/5n/9RfHy8Hn300Rqvr/4A89q1axUdHa0xY8Y424KCgvSzn/1MZ86c0datW12v+9GPfqSwsDDneWJioiTpH/7hH9S8eXNXe0VFhb766itJUnZ2tkpKSjRmzBh98803zqNZs2ZKTEzUhx9+WJdyALhF8XYVgFqLjY2t0damTRv99a9/dZ4fPXpUo0aNuuo4X3zxhe6++24FBrr/f6tHjx7O9qvttzrwdOrU6Yrt1fM5fPiwJGnw4MFXnIfX673qPAHcngg5AGqtWbNmV2w3N/lize/a77XmU1VVJeni53Kio6Nr9Lv0LBAAe/CbDeCmuPPOO3XgwIGr9omLi9Onn36qqqoq19mczz//3NleX3ORpMjISCUlJdXLmABufXwmB8BNMWrUKH3yySdatWpVjW3VZ1geeughFRQU6K233nK2XbhwQQsXLlSrVq30wAMP1MtcUlJS5PV69atf/Up+v7/G9q+//rpe9gPg1sKZHAA3xdSpU/X222/rRz/6kZ588kklJCSouLhYf/zjH/Xqq68qPj5ekyZN0muvvabx48crNzdXnTt31ttvv60//elPevnll9W6det6mYvX69WiRYs0btw49e/fX6NHj1b79u2Vn5+v999/X/fee69++9vf1su+ANw6CDkAbopWrVrpo48+0owZM7Rq1SotX75ckZGRGjJkiDp27ChJatGihbZs2aLnn39ey5cvV1lZmbp166alS5dq/Pjx9Tqfn/zkJ4qJidHcuXM1f/58lZeX64477tCgQYP0xBNP1Ou+ANwauK0DAACwEp/JAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACw0m35PTlVVVU6ceKEWrdu7dzNGAAA3NqMMTp9+rRiYmJq3Jj3ZrgtQ86JEydq3HUYAADcHr788kvnS0Fvptsy5FR/1fuXX34pr9db5/H8fr82bNig5ORkBQUF1Xm82x31cKMebtTDjXq4UQ836uFWXFysLl261NstW67ltgw51W9Reb3eegs5oaGh8nq9LEJRj8tRDzfq4UY93KiHG/Vwq75BbkN91IQPHgMAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYqXljT+BW1Pn596/Z5/jc1AaYCQAAuFGcyQEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWKlOIWfu3LkKCAjQM88847SdP39e6enpatu2rVq1aqVRo0apsLDQ9br8/HylpqYqNDRUkZGRmjp1qi5cuFCXqQAAALjccMjZvXu3XnvtNfXp08fV/uyzz+q9997TypUrtXXrVp04cUIjR450tldWVio1NVUVFRXavn27li9frmXLlmn69Ok3fhQAAACXuaGQc+bMGY0dO1b/9V//pTZt2jjtpaWlWrx4sV566SUNHjxYCQkJWrp0qbZv364dO3ZIkjZs2KBDhw7p9ddfV9++fTV8+HC9+OKLysrKUkVFRf0cFQAAaPKa38iL0tPTlZqaqqSkJP3yl7902nNzc+X3+5WUlOS0de/eXbGxscrJydHAgQOVk5Oj3r17KyoqyumTkpKiyZMn6+DBg+rXr1+N/ZWXl6u8vNx5XlZWJkny+/3y+/03cggu1WNU/+tpZq77NTa6vB5NHfVwox5u1MONerhRD7eGrkOtQ86bb76pP//5z9q9e3eNbQUFBQoODlZ4eLirPSoqSgUFBU6fSwNO9fbqbVcyZ84czZo1q0b7hg0bFBoaWttD+E7Z2dmSpHkDrt137dq19bbfW1V1PXAR9XCjHm7Uw416uFGPi86dO9eg+6tVyPnyyy/19NNPKzs7WyEhITdrTjVkZmYqIyPDeV5WVqZOnTopOTlZXq+3zuP7/X5lZ2dr6NChCgoKUq+Z66/5mgMzU+q831vV5fVo6qiHG/Vwox5u1MONeridOnWqQfdXq5CTm5uroqIi9e/f32mrrKzUtm3b9Nvf/lbr169XRUWFSkpKXGdzCgsLFR0dLUmKjo7Wrl27XONWX31V3edyHo9HHo+nRntQUFC9Lprq8corA66rr+3qu763O+rhRj3cqIcb9XCjHhc1dA1q9cHjIUOGaP/+/dq3b5/zuOeeezR27Fjn56CgIG3atMl5TV5envLz8+Xz+SRJPp9P+/fvV1FRkdMnOztbXq9XPXv2rKfDAgAATV2tzuS0bt1avXr1crW1bNlSbdu2ddonTJigjIwMRUREyOv16qmnnpLP59PAgQMlScnJyerZs6fGjRunefPmqaCgQC+88ILS09OveLYGAADgRtzQ1VVXs2DBAgUGBmrUqFEqLy9XSkqKXnnlFWd7s2bNtGbNGk2ePFk+n08tW7ZUWlqaZs+eXd9TAQAATVidQ86WLVtcz0NCQpSVlaWsrKzvfE1cXFyTuDoJAAA0Hu5dBQAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJVqFXIWLVqkPn36yOv1yuv1yufz6YMPPnC2nz9/Xunp6Wrbtq1atWqlUaNGqbCw0DVGfn6+UlNTFRoaqsjISE2dOlUXLlyon6MBAAD4P7UKOR07dtTcuXOVm5urPXv2aPDgwRoxYoQOHjwoSXr22Wf13nvvaeXKldq6datOnDihkSNHOq+vrKxUamqqKioqtH37di1fvlzLli3T9OnT6/eoAABAk9e8Np0ffvhh1/N///d/16JFi7Rjxw517NhRixcv1ooVKzR48GBJ0tKlS9WjRw/t2LFDAwcO1IYNG3To0CFt3LhRUVFR6tu3r1588UVNmzZNM2fOVHBwcP0dGQAAaNJqFXIuVVlZqZUrV+rs2bPy+XzKzc2V3+9XUlKS06d79+6KjY1VTk6OBg4cqJycHPXu3VtRUVFOn5SUFE2ePFkHDx5Uv379rriv8vJylZeXO8/LysokSX6/X36//0YPwVE9RvW/nmbmul9jo8vr0dRRDzfq4UY93KiHG/Vwa+g61Drk7N+/Xz6fT+fPn1erVq20atUq9ezZU/v27VNwcLDCw8Nd/aOiolRQUCBJKigocAWc6u3V277LnDlzNGvWrBrtGzZsUGhoaG0P4TtlZ2dLkuYNuHbftWvX1tt+b1XV9cBF1MONerhRDzfq4UY9Ljp37lyD7q/WIadbt27at2+fSktL9fbbbystLU1bt269GXNzZGZmKiMjw3leVlamTp06KTk5WV6vt87j+/1+ZWdna+jQoQoKClKvmeuv+ZoDM1PqvN9b1eX1aOqohxv1cKMebtTDjXq4nTp1qkH3V+uQExwcrLvuukuSlJCQoN27d+s///M/9dhjj6miokIlJSWuszmFhYWKjo6WJEVHR2vXrl2u8aqvvqrucyUej0cej6dGe1BQUL0umurxyisDrquv7eq7vrc76uFGPdyohxv1cKMeFzV0Der8PTlVVVUqLy9XQkKCgoKCtGnTJmdbXl6e8vPz5fP5JEk+n0/79+9XUVGR0yc7O1ter1c9e/as61QAAAActTqTk5mZqeHDhys2NlanT5/WihUrtGXLFq1fv15hYWGaMGGCMjIyFBERIa/Xq6eeeko+n08DBw6UJCUnJ6tnz54aN26c5s2bp4KCAr3wwgtKT0+/4pkaAACAG1WrkFNUVKTHH39cJ0+eVFhYmPr06aP169dr6NChkqQFCxYoMDBQo0aNUnl5uVJSUvTKK684r2/WrJnWrFmjyZMny+fzqWXLlkpLS9Ps2bPr96gAAECTV6uQs3jx4qtuDwkJUVZWlrKysr6zT1xcXJO4MgkAADQu7l0FAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKzVv7Ancrjo///41+xyfm9oAMwEAAFfCmRwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKtQo5c+bM0fe//321bt1akZGReuSRR5SXl+fqc/78eaWnp6tt27Zq1aqVRo0apcLCQlef/Px8paamKjQ0VJGRkZo6daouXLhQ96MBAAD4P7UKOVu3blV6erp27Nih7Oxs+f1+JScn6+zZs06fZ599Vu+9955WrlyprVu36sSJExo5cqSzvbKyUqmpqaqoqND27du1fPlyLVu2TNOnT6+/owIAAE1e89p0Xrdunev5smXLFBkZqdzcXN1///0qLS3V4sWLtWLFCg0ePFiStHTpUvXo0UM7duzQwIEDtWHDBh06dEgbN25UVFSU+vbtqxdffFHTpk3TzJkzFRwcXH9HBwAAmqxahZzLlZaWSpIiIiIkSbm5ufL7/UpKSnL6dO/eXbGxscrJydHAgQOVk5Oj3r17KyoqyumTkpKiyZMn6+DBg+rXr1+N/ZSXl6u8vNx5XlZWJkny+/3y+/11OQRnnEv/9TQzdR7z0vFuN5fXo6mjHm7Uw416uFEPN+rh1tB1uOGQU1VVpWeeeUb33nuvevXqJUkqKChQcHCwwsPDXX2joqJUUFDg9Lk04FRvr952JXPmzNGsWbNqtG/YsEGhoaE3egg1ZGdnS5LmDaif8dauXVs/AzWS6nrgIurhRj3cqIcb9XCjHhedO3euQfd3wyEnPT1dBw4c0Mcff1yf87mizMxMZWRkOM/LysrUqVMnJScny+v11nl8v9+v7OxsDR06VEFBQeo1c32dx5SkAzNT6mWchnZ5PZo66uFGPdyohxv1cKMebqdOnWrQ/d1QyJkyZYrWrFmjbdu2qWPHjk57dHS0KioqVFJS4jqbU1hYqOjoaKfPrl27XONVX31V3edyHo9HHo+nRntQUFC9Lprq8corA+ptvNtZfdf3dkc93KiHG/Vwox5u1OOihq5Bra6uMsZoypQpWrVqlTZv3qwuXbq4tickJCgoKEibNm1y2vLy8pSfny+fzydJ8vl82r9/v4qKipw+2dnZ8nq96tmzZ12OBQAAwFGrMznp6elasWKF3n33XbVu3dr5DE1YWJhatGihsLAwTZgwQRkZGYqIiJDX69VTTz0ln8+ngQMHSpKSk5PVs2dPjRs3TvPmzVNBQYFeeOEFpaenX/FsDQAAwI2oVchZtGiRJOnBBx90tS9dulTjx4+XJC1YsECBgYEaNWqUysvLlZKSoldeecXp26xZM61Zs0aTJ0+Wz+dTy5YtlZaWptmzZ9ftSAAAAC5Rq5BjzLUvrQ4JCVFWVpaysrK+s09cXNxtf+URAAC4tXHvKgAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACs1LyxJ2Czzs+/f80+x+emNsBMAABoejiTAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwUq1DzrZt2/Twww8rJiZGAQEBWr16tWu7MUbTp09Xhw4d1KJFCyUlJenw4cOuPsXFxRo7dqy8Xq/Cw8M1YcIEnTlzpk4HAgAAcKlah5yzZ88qPj5eWVlZV9w+b948/eY3v9Grr76qnTt3qmXLlkpJSdH58+edPmPHjtXBgweVnZ2tNWvWaNu2bZo0adKNHwUAAMBlav2Nx8OHD9fw4cOvuM0Yo5dfflkvvPCCRowYIUn63e9+p6ioKK1evVqjR4/WZ599pnXr1mn37t265557JEkLFy7UQw89pF//+teKiYmpw+EAAABcVK+3dTh27JgKCgqUlJTktIWFhSkxMVE5OTkaPXq0cnJyFB4e7gQcSUpKSlJgYKB27typRx99tMa45eXlKi8vd56XlZVJkvx+v/x+f53nXT1G9b+eZqbOY9Z237eSy+vR1FEPN+rhRj3cqIcb9XBr6DrUa8gpKCiQJEVFRbnao6KinG0FBQWKjIx0T6J5c0VERDh9LjdnzhzNmjWrRvuGDRsUGhpaH1OXJGVnZ0uS5g2otyGvae3atQ23s1qqrgcuoh5u1MONerhRDzfqcdG5c+cadH+3xQ06MzMzlZGR4TwvKytTp06dlJycLK/XW+fx/X6/srOzNXToUAUFBanXzPV1HvN6HZiZ0mD7ul6X16Opox5u1MONerhRDzfq4Xbq1KkG3V+9hpzo6GhJUmFhoTp06OC0FxYWqm/fvk6foqIi1+suXLig4uJi5/WX83g88ng8NdqDgoLqddFUj1deGVBvY17PPm9V9V3f2x31cKMebtTDjXq4UY+LGroG9fo9OV26dFF0dLQ2bdrktJWVlWnnzp3y+XySJJ/Pp5KSEuXm5jp9Nm/erKqqKiUmJtbndAAAQBNW6zM5Z86c0ZEjR5znx44d0759+xQREaHY2Fg988wz+uUvf6m7775bXbp00S9+8QvFxMTokUcekST16NFDw4YN08SJE/Xqq6/K7/drypQpGj16NFdWAQCAelPrkLNnzx794Ac/cJ5Xf1YmLS1Ny5Yt07/8y7/o7NmzmjRpkkpKSnTfffdp3bp1CgkJcV7zxhtvaMqUKRoyZIgCAwM1atQo/eY3v6mHwwEAALio1iHnwQcflDHffYl1QECAZs+erdmzZ39nn4iICK1YsaK2uwYAALhu3LsKAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlQg4AALASIQcAAFiJkAMAAKxEyAEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGCl5o09gaau8/PvX7PP8bmpDTATAADswpkcAABgJUIOAACwEiEHAABYiZADAACsRMgBAABWIuQAAAArEXIAAICVCDkAAMBKhBwAAGAlvvH4NsC3IgMAUHucyQEAAFYi5AAAACsRcgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJe5CbgnuVA4AgBtncgAAgJUIOQAAwEqEHAAAYCVCDgAAsBIfPIZL5+ffl6eZ0bwBUq+Z61VeGVCjDx9gBgDcDjiTAwAArETIAQAAVuLtqibker5LBwAAW3AmBwAAWImQAwAArETIAQAAViLkAAAAKzXqB4+zsrI0f/58FRQUKD4+XgsXLtSAAQMac0q4Dg35AWa+kwcAcKMa7UzOW2+9pYyMDM2YMUN//vOfFR8fr5SUFBUVFTXWlAAAgEUa7UzOSy+9pIkTJ+qJJ56QJL366qt6//33tWTJEj3//PONNS3chq7nzBJnhACg6WmUkFNRUaHc3FxlZmY6bYGBgUpKSlJOTk6N/uXl5SovL3eel5aWSpKKi4vl9/vrPB+/369z587p1KlTCgoKUvMLZ+s85u2seZXRuXNVau4PVGVVzds6NKS7fv6Ha/a5nkV8PePszBxyxfZL18d9v952w+NcKnHOpmv2uR6Nsa/Lf19uZQ1R58aox/Uc1/WsjZvhdlofDeFWrUdjraHi4mJJkjGm3se+ItMIvvrqKyPJbN++3dU+depUM2DAgBr9Z8yYYSTx4MGDBw8ePCx4HD16tEHyxm3xjceZmZnKyMhwnldVVam4uFht27ZVQEDdzzSUlZWpU6dO+vLLL+X1eus83u2OerhRDzfq4UY93KiHG/VwKy0tVWxsrCIiIhpkf40Sctq1a6dmzZqpsLDQ1V5YWKjo6Oga/T0ejzwej6stPDy83ufl9XpZhJegHm7Uw416uFEPN+rhRj3cAgMb5rqnRrm6Kjg4WAkJCdq06f+/J1hVVaVNmzbJ5/M1xpQAAIBlGu3tqoyMDKWlpemee+7RgAED9PLLL+vs2bPO1VYAAAB10Wgh57HHHtPXX3+t6dOnq6CgQH379tW6desUFRXV4HPxeDyaMWNGjbfEmirq4UY93KiHG/Vwox5u1MOtoesRYExDXccFAADQcLh3FQAAsBIhBwAAWImQAwAArETIAQAAViLkAAAAKxFyJGVlZalz584KCQlRYmKidu3a1dhTqndz5szR97//fbVu3VqRkZF65JFHlJeX5+rz4IMPKiAgwPX46U9/6uqTn5+v1NRUhYaGKjIyUlOnTtWFCxca8lDqxcyZM2sca/fu3Z3t58+fV3p6utq2batWrVpp1KhRNb6h25ZaSFLnzp1r1CMgIEDp6emS7F8b27Zt08MPP6yYmBgFBARo9erVru3GGE2fPl0dOnRQixYtlJSUpMOHD7v6FBcXa+zYsfJ6vQoPD9eECRN05swZV59PP/1UgwYNUkhIiDp16qR58+bd7EO7IVerh9/v17Rp09S7d2+1bNlSMTExevzxx3XixAnXGFdaU3PnznX1saEekjR+/Pgaxzps2DBXn6ayPiRd8W9JQECA5s+f7/RpsPXRIHfIuoW9+eabJjg42CxZssQcPHjQTJw40YSHh5vCwsLGnlq9SklJMUuXLjUHDhww+/btMw899JCJjY01Z86ccfo88MADZuLEiebkyZPOo7S01Nl+4cIF06tXL5OUlGT27t1r1q5da9q1a2cyMzMb45DqZMaMGeZ73/ue61i//vprZ/tPf/pT06lTJ7Np0yazZ88eM3DgQPO3f/u3znabamGMMUVFRa5aZGdnG0nmww8/NMbYvzbWrl1r/u3f/s288847RpJZtWqVa/vcuXNNWFiYWb16tfnkk0/MD3/4Q9OlSxfz7bffOn2GDRtm4uPjzY4dO8xHH31k7rrrLjNmzBhne2lpqYmKijJjx441Bw4cML///e9NixYtzGuvvdZQh3ndrlaPkpISk5SUZN566y3z+eefm5ycHDNgwACTkJDgGiMuLs7Mnj3btWYu/XtjSz2MMSYtLc0MGzbMdazFxcWuPk1lfRhjXHU4efKkWbJkiQkICHDdlLOh1keTDzkDBgww6enpzvPKykoTExNj5syZ04izuvmKioqMJLN161an7YEHHjBPP/30d75m7dq1JjAw0BQUFDhtixYtMl6v15SXl9/M6da7GTNmmPj4+CtuKykpMUFBQWblypVO22effWYkmZycHGOMXbW4kqefftrceeedpqqqyhjTtNbG5X+0q6qqTHR0tJk/f77TVlJSYjwej/n9739vjDHm0KFDRpLZvXu30+eDDz4wAQEB5quvvjLGGPPKK6+YNm3auOoxbdo0061bt5t8RHVzpf+IXW7Xrl1Gkvniiy+ctri4OLNgwYLvfI1N9UhLSzMjRoz4ztc09fUxYsQIM3jwYFdbQ62PJv12VUVFhXJzc5WUlOS0BQYGKikpSTk5OY04s5uvtLRUkmrcCfaNN95Qu3bt1KtXL2VmZurcuXPOtpycHPXu3dv1rdQpKSkqKyvTwYMHG2bi9ejw4cOKiYlR165dNXbsWOXn50uScnNz5ff7Xeuie/fuio2NddaFbbW4VEVFhV5//XU9+eSTCggIcNqb0tq41LFjx1RQUOBaD2FhYUpMTHSth/DwcN1zzz1On6SkJAUGBmrnzp1On/vvv1/BwcFOn5SUFOXl5emvf/1rAx3NzVFaWqqAgIAaN06eO3eu2rZtq379+mn+/Pmuty9tq8eWLVsUGRmpbt26afLkyTp16pSzrSmvj8LCQr3//vuaMGFCjW0NsT4a7bYOt4JvvvlGlZWVNW4lERUVpc8//7yRZnXzVVVV6ZlnntG9996rXr16Oe0/+clPFBcXp5iYGH366aeaNm2a8vLy9M4770iSCgoKrlir6m23k8TERC1btkzdunXTyZMnNWvWLA0aNEgHDhxQQUGBgoODa/zBjoqKco7TplpcbvXq1SopKdH48eOdtqa0Ni5XPf8rHd+l6yEyMtK1vXnz5oqIiHD16dKlS40xqre1adPmpsz/Zjt//rymTZumMWPGuO6y/bOf/Uz9+/dXRESEtm/frszMTJ08eVIvvfSSJLvqMWzYMI0cOVJdunTR0aNH9a//+q8aPny4cnJy1KxZsya9PpYvX67WrVtr5MiRrvaGWh9NOuQ0Venp6Tpw4IA+/vhjV/ukSZOcn3v37q0OHTpoyJAhOnr0qO68886GnuZNNXz4cOfnPn36KDExUXFxcfrDH/6gFi1aNOLMGt/ixYs1fPhwxcTEOG1NaW3g+vn9fv34xz+WMUaLFi1ybcvIyHB+7tOnj4KDg/VP//RPmjNnjnX3cRo9erTzc+/evdWnTx/deeed2rJli4YMGdKIM2t8S5Ys0dixYxUSEuJqb6j10aTfrmrXrp2aNWtW46qZwsJCRUdHN9Ksbq4pU6ZozZo1+vDDD9WxY8er9k1MTJQkHTlyRJIUHR19xVpVb7udhYeH62/+5m905MgRRUdHq6KiQiUlJa4+l64LW2vxxRdfaOPGjfrHf/zHq/ZrSmujev5X+zsRHR2toqIi1/YLFy6ouLjY2jVTHXC++OILZWdnu87iXEliYqIuXLig48ePS7KvHpfq2rWr2rVr5/r9aGrrQ5I++ugj5eXlXfPviXTz1keTDjnBwcFKSEjQpk2bnLaqqipt2rRJPp+vEWdW/4wxmjJlilatWqXNmzfXOA14Jfv27ZMkdejQQZLk8/m0f/9+1y9r9R+3nj173pR5N5QzZ87o6NGj6tChgxISEhQUFORaF3l5ecrPz3fWha21WLp0qSIjI5WamnrVfk1pbXTp0kXR0dGu9VBWVqadO3e61kNJSYlyc3OdPps3b1ZVVZUTCH0+n7Zt2ya/3+/0yc7OVrdu3W67tyKqA87hw4e1ceNGtW3b9pqv2bdvnwIDA523bWyqx+X+8pe/6NSpU67fj6a0PqotXrxYCQkJio+Pv2bfm7Y+avUxZQu9+eabxuPxmGXLlplDhw6ZSZMmmfDwcNdVIjaYPHmyCQsLM1u2bHFdsnfu3DljjDFHjhwxs2fPNnv27DHHjh0z7777runatau5//77nTGqLxNOTk42+/btM+vWrTPt27e/bS4TvtRzzz1ntmzZYo4dO2b+9Kc/maSkJNOuXTtTVFRkjLl4CXlsbKzZvHmz2bNnj/H5fMbn8zmvt6kW1SorK01sbKyZNm2aq70prI3Tp0+bvXv3mr179xpJ5qWXXjJ79+51rhaaO3euCQ8PN++++6759NNPzYgRI654CXm/fv3Mzp07zccff2zuvvtu1yXCJSUlJioqyowbN84cOHDAvPnmmyY0NPSWvET4avWoqKgwP/zhD03Hjh3Nvn37XH9Pqq+E2b59u1mwYIHZt2+fOXr0qHn99ddN+/btzeOPP+7sw5Z6nD592vz85z83OTk55tixY2bjxo2mf//+5u677zbnz593xmgq66NaaWmpCQ0NNYsWLarx+oZcH00+5BhjzMKFC01sbKwJDg42AwYMMDt27GjsKdU7SVd8LF261BhjTH5+vrn//vtNRESE8Xg85q677jJTp051fReKMcYcP37cDB8+3LRo0cK0a9fOPPfcc8bv9zfCEdXNY489Zjp06GCCg4PNHXfcYR577DFz5MgRZ/u3335r/vmf/9m0adPGhIaGmkcffdScPHnSNYYttai2fv16I8nk5eW52pvC2vjwww+v+PuRlpZmjLl4GfkvfvELExUVZTwejxkyZEiNOp06dcqMGTPGtGrVyni9XvPEE0+Y06dPu/p88skn5r777jMej8fccccdZu7cuQ11iLVytXocO3bsO/+eVH+vUm5urklMTDRhYWEmJCTE9OjRw/zqV79y/UffGDvqce7cOZOcnGzat29vgoKCTFxcnJk4cWKN/1FuKuuj2muvvWZatGhhSkpKary+IddHgDHGXP95HwAAgNtDk/5MDgAAsBchBwAAWImQAwAArETIAQAAViLkAAAAKxFyAACAlQg5AADASoQcAABgJUIOAACwEiEHAABYiZADAACs9P8A4oZnoRCa64sAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.hist(column='income', bins=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature set\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define feature sets, X:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['region', 'tenure', 'age', 'marital', 'address', 'income', 'ed',\n",
       "       'employ', 'retire', 'gender', 'reside', 'custcat'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use scikit-learn library, we have to convert the Pandas data frame to a Numpy array:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  2.,  13.,  44.,   1.,   9.,  64.,   4.,   5.,   0.,   0.,   2.],\n",
       "       [  3.,  11.,  33.,   1.,   7., 136.,   5.,   5.,   0.,   0.,   6.],\n",
       "       [  3.,  68.,  52.,   1.,  24., 116.,   1.,  29.,   0.,   1.,   2.],\n",
       "       [  2.,  33.,  33.,   0.,  12.,  33.,   2.,   0.,   0.,   1.,   1.],\n",
       "       [  2.,  23.,  30.,   1.,   9.,  30.,   1.,   2.,   0.,   0.,   4.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = df[['region', 'tenure','age', 'marital', 'address', 'income', 'ed', 'employ','retire', 'gender', 'reside']] .values  #.astype(float)\n",
    "X[0:5]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What are our labels?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 4, 3, 1, 3])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['custcat'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize Data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data Standardization gives the data zero mean and unit variance, it is good practice, especially for algorithms such as KNN which is based on the distance of data points:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.02696767, -1.055125  ,  0.18450456,  1.0100505 , -0.25303431,\n",
       "        -0.12650641,  1.0877526 , -0.5941226 , -0.22207644, -1.03459817,\n",
       "        -0.23065004],\n",
       "       [ 1.19883553, -1.14880563, -0.69181243,  1.0100505 , -0.4514148 ,\n",
       "         0.54644972,  1.9062271 , -0.5941226 , -0.22207644, -1.03459817,\n",
       "         2.55666158],\n",
       "       [ 1.19883553,  1.52109247,  0.82182601,  1.0100505 ,  1.23481934,\n",
       "         0.35951747, -1.36767088,  1.78752803, -0.22207644,  0.96655883,\n",
       "        -0.23065004],\n",
       "       [-0.02696767, -0.11831864, -0.69181243, -0.9900495 ,  0.04453642,\n",
       "        -0.41625141, -0.54919639, -1.09029981, -0.22207644,  0.96655883,\n",
       "        -0.92747794],\n",
       "       [-0.02696767, -0.58672182, -0.93080797,  1.0100505 , -0.25303431,\n",
       "        -0.44429125, -1.36767088, -0.89182893, -0.22207644, -1.03459817,\n",
       "         1.16300577]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = preprocessing.StandardScaler().fit(X).transform(X.astype(float))\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Test Split\n",
    "\n",
    "Out of Sample Accuracy is the percentage of correct predictions that the model makes on data that the model has NOT been trained on. Doing a train and test on the same dataset will most likely have low out-of-sample accuracy, due to the likelihood of our model overfitting.\n",
    "\n",
    "It is important that our models have a high, out-of-sample accuracy, because the purpose of any model, of course, is to make correct predictions on unknown data. So how can we improve out-of-sample accuracy? One way is to use an evaluation approach called Train/Test Split.\n",
    "Train/Test Split involves splitting the dataset into training and testing sets respectively, which are mutually exclusive. After which, you train with the training set and test with the testing set.\n",
    "\n",
    "This will provide a more accurate evaluation on out-of-sample accuracy because the testing dataset is not part of the dataset that has been used to train the model. It is more realistic for the real world problems.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (800, 11) (800,)\n",
      "Test set: (200, 11) (200,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"classification\">\n",
    "    <h2>Classification</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>K nearest neighbor (KNN)</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import library\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Classifier implementing the k-nearest neighbors vote.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "Let's start the algorithm with k=4 for now:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>KNeighborsClassifier(n_neighbors=4)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;KNeighborsClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\">?<span>Documentation for KNeighborsClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>KNeighborsClassifier(n_neighbors=4)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "KNeighborsClassifier(n_neighbors=4)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = 4\n",
    "#Train Model and Predict  \n",
    "neigh = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n",
    "neigh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting\n",
    "\n",
    "We can use the model to make predictions on the test set:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 3, 2, 4])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat = neigh.predict(X_test)\n",
    "yhat[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accuracy evaluation\n",
    "\n",
    "In multilabel classification, **accuracy classification score** is a function that computes subset accuracy. This function is equal to the jaccard_score function. Essentially, it calculates how closely the actual labels and predicted labels are matched in the test set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set Accuracy:  0.5475\n",
      "Test set Accuracy:  0.32\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\n",
    "print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice\n",
    "\n",
    "Can you build the model again, but this time with k=6?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set Accuracy:  0.51625\n",
      "Test set Accuracy:  0.31\n"
     ]
    }
   ],
   "source": [
    "# write your code here\n",
    "\n",
    "k = 6\n",
    "neigh6 = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n",
    "yhat6 = neigh6.predict(X_test)\n",
    "print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh6.predict(X_train)))\n",
    "print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for the solution</summary>\n",
    "\n",
    "```python\n",
    "k = 6\n",
    "neigh6 = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n",
    "yhat6 = neigh6.predict(X_test)\n",
    "print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh6.predict(X_train)))\n",
    "print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat6))\n",
    "\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What about other K?\n",
    "\n",
    "K in KNN, is the number of nearest neighbors to examine. It is supposed to be specified by the user. So, how can we choose right value for K?\n",
    "The general solution is to reserve a part of your data for testing the accuracy of the model. Then choose k =1, use the training part for modeling, and calculate the accuracy of prediction using all samples in your test set. Repeat this process, increasing the k, and see which k is the best for your model.\n",
    "\n",
    "We can calculate the accuracy of KNN for different values of k.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.3  , 0.29 , 0.315, 0.32 , 0.315, 0.31 , 0.335, 0.325, 0.34 ])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Ks = 10\n",
    "mean_acc = np.zeros((Ks-1))\n",
    "std_acc = np.zeros((Ks-1))\n",
    "\n",
    "for n in range(1,Ks):\n",
    "    \n",
    "    #Train Model and Predict  \n",
    "    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n",
    "    yhat=neigh.predict(X_test)\n",
    "    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n",
    "\n",
    "    \n",
    "    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n",
    "\n",
    "mean_acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot the model accuracy for a different number of neighbors.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHVCAYAAAB8NLYkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAACCc0lEQVR4nO3deXwU5f0H8M/M7Jk7IeSCEE4BuW8R8YyCIoLaitaWS+tPi1fxAq2AogJWLbUoVqrgURFtBTwQDxQ8iohAQEWRIBCuBBJy7b078/z+2GTIkgSSsMlmdz9vX3nJzs7OPrMQ8uE5vo8khBAgIiIiorAnh7oBRERERBQcDHZEREREEYLBjoiIiChCMNgRERERRQgGOyIiIqIIwWBHREREFCEY7IiIiIgiBIMdERERUYRgsCMiIiKKEIZQN6A10jQNhw8fRnx8PCRJCnVziIiIKIoJIVBZWYmsrCzI8mn65ESILVq0SOTk5Aiz2SyGDh0qNm3a1KDXLV++XAAQ48aNCzg+adIkASDga9SoUY1q04EDB2pdg1/84he/+MUvfvErlF8HDhw4bYYJaY/dihUrMH36dLzwwgsYNmwYFi5ciFGjRmHXrl1IS0ur93X79u3Dvffei5EjR9b5/OjRo7F06VL9sdlsblS74uPjAQAHDhxAQkJCo15LREREFEwVFRXIzs7W88mphDTYPfPMM/jjH/+IKVOmAABeeOEFfPDBB3j55ZcxY8aMOl+jqipuvPFGPPLII/jyyy9RVlZW6xyz2YyMjIwmt6t6+DUhIYHBjoiIiFqFhkwPC9niCY/Hgy1btiA3N/dEY2QZubm52LhxY72ve/TRR5GWloabbrqp3nPWr1+PtLQ0dO/eHbfddhtKSkpO2Ra3242KioqALyIiIqJwE7JgV1xcDFVVkZ6eHnA8PT0dhYWFdb7mq6++wksvvYQlS5bUe93Ro0fj1Vdfxbp167BgwQJs2LABl19+OVRVrfc18+bNQ2Jiov6VnZ3dtJsiIiIiCqGwWRVbWVmJP/zhD1iyZAlSU1PrPe/666/Xf92nTx/07dsXXbp0wfr163HJJZfU+ZqZM2di+vTp+uPqsWwiIiKicBKyYJeamgpFUVBUVBRwvKioqM75cXv27MG+ffswduxY/ZimaQAAg8GAXbt2oUuXLrVe17lzZ6SmpiI/P7/eYGc2mxu9wEIIAZ/Pd8qeQAoPiqLAYDCwtA0REYW9kAU7k8mEQYMGYd26dRg/fjwAf1Bbt24dbr/99lrn9+jRA99//33Asb/85S+orKzE3//+93p72A4ePIiSkhJkZmYGre0ejwdHjhyBw+EI2jUptGJiYpCZmQmTyRTqphARETVZSIdip0+fjkmTJmHw4MEYOnQoFi5cCLvdrq+SnThxItq1a4d58+bBYrGgd+/eAa9PSkoCAP24zWbDI488gmuvvRYZGRnYs2cP7r//fnTt2hWjRo0KSps1TcPevXuhKAqysrJgMpnY0xPGhBDweDw4duwY9u7di27dup2++CMREVErFdJgN2HCBBw7dgyzZs1CYWEh+vfvj7Vr1+oLKgoKChr1Q1ZRFOzYsQOvvPIKysrKkJWVhcsuuwxz585t9FBrfTweDzRNQ3Z2NmJiYoJyTQotq9UKo9GI/fv3w+PxwGKxhLpJRERETSIJIUSoG9HaVFRUIDExEeXl5bXq2LlcLuzduxedOnViAIgg/H0lIqLW6lS55GQccyIiIiKKEAx2RERERBGCwY6IiIgoQjDYRZmNGzdCURSMGTMm1E0hIiKiIGOwizIvvfQS7rjjDnzxxRc4fPhwSNvi8XhC+v5ERESRhsHuDAkhYPfYQ/LV2AXNNpsNK1aswG233YYxY8Zg2bJltc557733MGTIEFgsFqSmpuLqq6/Wn3O73XjggQeQnZ0Ns9mMrl274qWXXgIALFu2TK8rWG3VqlUBNf7mzJmD/v3741//+lfA6tO1a9fivPPOQ1JSEtq0aYMrr7wSe/bsCbjWwYMHccMNNyAlJQWxsbEYPHgwNm3ahH379kGWZXz33XcB5y9cuBA5OTn67iRERETRIGz2im2tHF4H4ubFheS9bTNtiDXFNvj8t956Cz169ED37t3x+9//HnfffTdmzpyph68PPvgAV199NR566CG8+uqr8Hg8WLNmjf76iRMnYuPGjXj22WfRr18/7N27F8XFxY1qc35+Pv773//inXfegaIoAAC73Y7p06ejb9++sNlsmDVrFq6++mrk5eVBlmXYbDZccMEFaNeuHd59911kZGRg69at0DQNHTt2RG5uLpYuXYrBgwfr77N06VJMnjyZxYaJiKjZCSFazWYFDHZR5KWXXsLvf/97AMDo0aNRXl6ODRs24MILLwQAPP7447j++uvxyCOP6K/p168fAOCXX37BW2+9hU8++QS5ubkA/PvwNpbH48Grr76Ktm3b6seuvfbagHNefvlltG3bFjt37kTv3r3xxhtv4NixY9i8eTNSUlIAAF27dtXPv/nmm3HrrbfimWeegdlsxtatW/H9999j9erVjW4fERFRQ2hCg9PrhM1jg1t1o118OyiyEupmMdidqRhjDGwzbSF774batWsXvv32W6xcuRIAYDAYMGHCBLz00kt6sMvLy8Mf//jHOl+fl5cHRVFwwQUXnFGbc3JyAkIdAOzevRuzZs3Cpk2bUFxcrA+fFhQUoHfv3sjLy8OAAQP0UHey8ePHY9q0aVi5ciWuv/56LFu2DBdddBE6dux4Rm0lIiI6mdvnhsPrQLmrHHavHQICJtkETWhQwGAX9iRJatRwaKi89NJL8Pl8yMrK0o8JIWA2m7Fo0SIkJibCarXW+/pTPQcAsizXmvPn9XprnRcbW/uzGjt2LHJycrBkyRJkZWVB0zT07t1bX1xxuvc2mUyYOHEili5dimuuuQZvvPEG/v73v5/yNURERA2laiocXgcq3ZWweWzwaB6YFTMSLYlQNRVunzvUTdRxAlIU8Pl8ePXVV/H0008jLy9P/9q+fTuysrKwfPlyAEDfvn2xbt26Oq/Rp08faJqGDRs21Pl827ZtUVlZCbvdrh/Ly8s7bdtKSkqwa9cu/OUvf8Ell1yCnj17orS0NOCcvn37Ii8vD8ePH6/3OjfffDM+/fRTPP/88/D5fLjmmmtO+95ERET1EULA6XWi2F6MfWX7UFBegHJ3OUwGE1KsKYg1xUKWWl+Man0toqB7//33UVpaiptuugm9e/cO+Lr22mv1la2zZ8/G8uXLMXv2bPz000/4/vvvsWDBAgBAx44dMWnSJEydOhWrVq3C3r17sX79erz11lsAgGHDhiEmJgYPPvgg9uzZgzfeeKPOVbcnS05ORps2bfDiiy8iPz8fn332GaZPnx5wzg033ICMjAyMHz8eX3/9NX799Vf897//xcaNG/VzevbsiXPOOQcPPPAAbrjhhtP28hEREdXFq3pR7irHwYqD2F+2H0X2IgBAkiUJiZZEmBRTiFt4agx2UeCll15Cbm4uEhMTaz137bXX4rvvvsOOHTtw4YUX4u2338a7776L/v374+KLL8a3336rn7t48WL85je/wZ/+9Cf06NEDf/zjH/UeupSUFLz++utYs2YN+vTpg+XLl2POnDmnbZssy3jzzTexZcsW9O7dG3/+85/x17/+NeAck8mEjz/+GGlpabjiiivQp08fzJ8/X19VW+2mm26Cx+PB1KlTm/ApERFRtNKEBrvHjiJbEfaX7ceBigNweB2IMcUg2ZoMq9Haala9no4kGlsMLQpUVFQgMTER5eXlSEhICHjO5XJh7969AXXYqHWYO3cu3n77bezYsaPRr+XvKxFR9Km5EMLhdQAALEYLzIq5wUHOq3rh9rnRKbkTjIqxWdp5qlxyMi6eoLBns9mwb98+LFq0CI899liom0NEUa411TSj2upbCBFvjm8V5UrOFIMdhb3bb78dy5cvx/jx4zkMS0QhowkN5a5ylLpKYVbMiDXFwqSYYFbMEREYwpkQAi6fC3aPHeXucrh8LsiSDKvRijglNJsMNBcGOwp7y5Yta9BCDSKi5uL0OlHsKEa5uxxmxQybakOpqxQyZJgNZliNVsQYY2BWzDAbzK1yNWUk8qpeOLwOVLgrYPfYoQoVFoMFSZakiO1VZbAjIiJqIlVTUeYqQ4mjBKpQkWRJCghtmtDgUT2ocFfguPM4FEmBSTEhxhjjD3oGM0yKiUEviGruCFHproRLdcEoGxFjioFBjvzYE/l3SERE1AzsHjuKHcWo9FQixhiDOEPtIT1ZkmExWGAx+BdlqZoKj+rxh0FniR704kxxsBqtMCv+oBepvUnNqb6FEMnG5Kj6PBnsiIiIGsGn+VDqLEWJowQColYv3akosgKrbIXV6K+1WR30Spwl0BwaDJIBJkNV0DNYYVJMDHqnoGoqnD4nKt2VqHRXRtxCiKZgsCMiImoAIQTsXjuO2Y/B7rUj1hgLs8F8Rtc8Oej5NB88qgfFjmJoQoNRNsKsmBFnioPZ4J+fZ5SNUR30hBBwq27YPXaUucoieiFEUzDYERERnYZX9eK48zhKHCVQZAXJluYZ3jPIBhhkA2KMMfr7ejUvjtqPQkDAKBv1odvqemvNVTuttfFpPn2oNVoWQjQFgx0REVE9hBCo9FTimP0YnD4n4k3xLRqkjIoRRsUYEPQ8qkff5sooG2ExWvw9elUrbiNpgYAQAk6ffyFEhasi6hZCNAU/lSApqnC16PulJ3B3hFOZM2cOVq1ahby8vFA3hYjClNvnxnHncZS6SmGUjc3WS9cY1UEvFrEQQsCreeHyuVDprgQAmBQTLAZ/0DMpprANeh7VA4fXgTJnWVQvhGgKrq+mejmdTsTGxiI/P7/Br7nzzjsxaNAgmM1m9O/fP+htkiQJq1atCvp1iYiqaUJDmasMB8oPoMRZgjhTHGJNsa0uUEiSpA/LJluTkWRJgkE2wOF14HDlYewr24d9pftwqOIQylz+gKRqaqibXS9VU2Hz2HCk8gj2le7DwfKD8GgexJvjkWRNgsVgaXW/B61R+MV4CooLL7wQkydPxuTJk+s955NPPkFOTg66du3aqGtPnToVmzZtatKerUREoeTyuVDiKEGpqxQWgwUp1pRQN6nBJEnSF1gAJxYZ2Dytt1hyXQshJEmC1WBFnJkLIZqCPXZUr9WrV+Oqq65q1GueffZZTJs2DZ07d67z+alTp6Jv375wu90AAI/HgwEDBmDixIn649tvvx2ZmZmwWCzIycnBvHnzAAAdO3YEAFx99dWQJEl/DADz589Heno64uPjcdNNN8HlatmhcSIKb5rQcNxxHAVlBSh3lSPRnKjPawtXkiTBYrAg3hyPFGsKEiwJkCQJFe4KHKw4iH1l+7C3dC+OVB5Bucu/zZYmtBZpm0/znWhH6T4csR2BgECiJRFJlqQzXm0czRjsqE6apuH999/HuHHjgnrdZ599Fna7HTNmzAAAPPTQQygrK8OiRYv0599991289dZb2LVrF/7973/rAW7z5s0AgKVLl+LIkSP647feegtz5szBE088ge+++w6ZmZl4/vnng9puIopcDq8DhyoO4bDtMBRZQZI1KSJroFUXS04wJyDFmoJ4czwAoMxVhoMVB7G3dC/2lu5Fka0IFe4KuHwuCCGC9v5CCDi8Dhy1H8W+0n0oKC+A3WuH1WhFijUFMcYY7sARBByKpTp98803AIBhw4YF9bpxcXF4/fXXccEFFyA+Ph4LFy7E559/joSEBABAQUEBunXrhvPOOw+SJCEnJ0d/bdu2bQEASUlJyMjI0I8vXLgQN910E2666SYAwGOPPYZPP/2UvXZEdEqqpvoLDTtLoAmtUYWGI0F17beGFEu2GCxN3hXj5IUQAgIWg6VVLEaJRNHzJzjKPfHEE4iLi9O/vvzyS9x6660BxwoKCvTzV69ejSuvvBKyLKOgoCDgvCeeeOKM2jJ8+HDce++9mDt3Lu655x6cd955+nOTJ09GXl4eunfvjjvvvBMff/zxaa/3008/1Qqgw4cPP6M2ElFks3lsOFB+AIW2QpgUExItiVEV6uqiyAqsRiuSLEn+HjRTDDShodhRjILyAuwr24f9ZftRbC/27/Kgeurt0TvVQohkazKsRitDXTNhj12UuPXWW3Hdddfpj2+88UZce+21uOaaa/RjWVlZ+q/fffddzJ8/Xz9es2xISsqZTSbWNA1ff/01FEWpteJ24MCB2Lt3Lz788EN8+umnuO6665Cbm4v//Oc/Z/SeRESAvw5cqcu/HZgkSUi2steoPk0plqwKlQshQozBLkqkpKQEBDKr1Yq0tLQ6V7zu3r0b+/fvx6WXXgoAMBgMjV4Zeyp//etf8fPPP2PDhg0YNWoUli5diilTpujPJyQkYMKECZgwYQJ+85vfYPTo0Th+/DhSUlJgNBqhqoHL9Xv27IlNmzbpCzCAE0PJRESAf36XzWPDMccxOLwOvc4bNVxDiiVrQoNP+GAxWNgLGiIMdlTL6tWrkZubi5iYxq8Iy8/Ph81mQ2FhIZxOp97Td/bZZ8NkMmHbtm2YNWsW/vOf/2DEiBF45plncNddd+GCCy5A586d8cwzzyAzMxMDBgyALMt4++23kZGRgaSkJAD+lbHr1q3DiBEjYDabkZycjLvuuguTJ0/G4MGDMWLECPz73//Gjz/+WO/KXCKKLh7Vo5cwMcgGzu0KkrqKJUuQomaLs9aKwS5IImkniNWrV2PSpElNeu3NN9+MDRs26I8HDBgAANi7dy8yMjLw+9//HpMnT8bYsWMBALfccgs++OAD/OEPf8AXX3yB+Ph4PPnkk9i9ezcURcGQIUOwZs0ayLL/X31PP/00pk+fjiVLlqBdu3bYt28fJkyYgD179uD++++Hy+XCtddei9tuuw0fffTRGX4SRBTOhBCocFeg2FEMl8+FeHN8WO7CEA6qiyVT6EkimGuZI0RFRQUSExNRXl6ur9as5nK5sHfvXnTq1AkWS+SEuWrFxcXIzMzEwYMHkZ6eHurmtJhI/30lijZunxvFjmKUucpgNpjDviYdtV5e1Qu3z41OyZ2arbfyVLnkZPynCwU4fvw4nnnmmagKdUQUOTShodxVjmJHMTyqBwnmhIisSUdUHwY7CnDWWWfhrLPOCnUziIgazel1othRjHJ3OawGK5KtyaFuElGLY7AjIqKwpmoqylxlKHYUQxVq1BUaJqqJwY6IiMKW3WNHsaMYlZ5KxBhjEG+ID3WTiEKKwY6IgkIIAVWo8Gk+qFrV/4UKj88Dt+qGJEkwK2YYFSMMsgGKpPj/LytQJIXlJ6hRfJrPvx2YowSQwF46oioMdkTUINXBTdXUgADn1fwrwjyqB5rQ9OerVQc3wN+7ogkNAgISJCiS4n9eVmCSTTAZTDDKxoDAV/1rIuBEoeFiRzHsXjsLDROdhMGOiHTVoaxmj5tP9cGtuuFW3dA0TT8OAALiRDiTFBgVIywGS4ODWPX7aEKDw+dAhaciYO/J6p49RVZgNphhkk0wKIaA4wbZwJ6aKOFVvTjuPI4SRwkUWWGhYaI6MNgRRZGaPWp6j5vqhUfzwOPzBBwX8AcsSZICes5MBlPQhk6re+vqcnIPYaW7Ej7Npz8vQQro2TMbzDAbzLWGeA2ygT/8w5wQApWeShyzH4PT50S8KZ67GxDVg8EuSFRNhSa0Fns/WZI5PEW1nDzPrfrXHp8HHs0Dr+o90SsnVEjwB56aPV8Wg6VVhCFJkmCQDPXuFFAzpHo1L1xuFzSnBg0aJEiQJVm/J6NihFkxw6SYag3xcn5f6+b2ufXtwEyKCSnWlNO/iCiKMdgFgaqpOFB+AB7V02LvaVJMyE7MZrirx+TJk1FWVoZVq1aFuilBV3OYtPrXp5vnVjPkmA1mKLIS9sOXsiRDVmQYUXfPjSY0PeC6fK7Tzu8zG/wLO2oO8Vb/mlqeJjT/dmD2YrhUFxLMCdwOjKgB+F0SBJrQ4FE9+g+D5ubTfPoPcAXN90PH6XQiNTUV27dvR9euXU97fklJCW688Ubs2LEDJSUlSEtLw7hx4/DEE0+cdguUhti3bx86deqEbdu2oX///md8vdaqZiCpb55bdU9cNUmS9CDS2HlukUqWZP+k+no+hvrm9wkIyJADevZMBhNMsskf/E7q8Qv3gNwauXwufTswq8HKXjqiRmCwCyKDbGixeR+qpp7+pFO48MILMXnyZEyePLnecz755BPk5OQ0KNQBgCzLGDduHB577DG0bdsW+fn5mDZtGo4fP4433njjjNobSeqa56YKFW6fG26fO6AnTlT9J0tyrR4mDiGemYbM7/NpPviED263Ww/S1cGvOtgZZIN/mLdq7mHN4/w9apzqQsMljhL4NB8SzYlR/w8UosZisKN6rV69GldddVWDz09OTsZtt92mP87JycGf/vQn/PWvfwXg/2F56aWXQlEUrF27FpIk4fjx4+jbty+mTp2KRx99FKWlpbj99tvx8ccfw2azoX379njwwQcxZcoUdOrUCQAwYMAAAMAFF1yA9evXQ1VV3HfffXj55ZehKApuuummgJWVrYXNY0OpsxQe1aOHuYChwaqeoOoVoLFyLENBiDR0fl9177nT54TqrHvo26RUDfPKxoDAV/08f4/9HF4Hih3FqHBXwGqwIs4cF+omEYUlBjuqk6ZpeP/9989ojtrhw4fxzjvv4IILLgDg/2H5yiuvoE+fPnj22Wdx11134dZbb0W7du0wa9YsAMDDDz+MnTt34sMPP0Rqairy8/PhdDoBAN9++y2GDh2KTz/9FL169YLJ5K9d9fTTT2PZsmV4+eWX0bNnTzz99NNYuXIlLr744jP7EILEp/n0Eg2Af36kyWBimY4wps/vq6eHvmbZGKfPCZvHFrhYpTrYyTLMihlmxQyDUrtoczT0Vvk0H8qc/u3ABAQLDROdIQY7qtM333wDABg2bFijX3vDDTdg9erVcDqdGDt2LP71r3/pz7Vr1w7//Oc/MXHiRBQWFmLNmjXYtm0bDAb/H8WCggIMGDAAgwcPBgB07NhRf23btm0BAG3atEFGRoZ+fOHChZg5cyauueYaAMALL7yAjz76qNHtDjYWUo1eiqz457/WkctqlnHRhAabx4ZyUa73Mlf33lb37JkMJv+cyToWdYR7ALJ5bCi2F8PmsSHWFAuzwRzqJhGFvfD+W4Ea7IknnkBcXJz+9eWXX+LWW28NOFZQUKCfv3r1alx55ZWQZRkFBQUB5z3xxBOnfK+//e1v2Lp1K1avXo09e/Zg+vTpAc//9re/xdVXX4358+fjqaeeQrdu3fTnbrvtNrz55pvo378/7r//fvzvf/875XuVl5fjyJEjAQHUYDDowTBUvKoXR+1H9dXSyZZkhjoCcGKhi9lghtVoRbw5HkmWJCRbk5FsTUaiJREWgwWSJMEnfKhwV+BI5REcqDiA/WX7sbd0r/5VUF6AwspCHHceR4W7AnaPHW6ffz5ga5yOUM2relFkK8KB8gNwqS4kW5MZ6oiChD12UeLWW2/Fddddpz++8cYbce211+q9XACQlZWl//rdd9/F/Pnz9eN5eXn6cykpp16hlpGRgYyMDPTo0QMpKSkYOXIkHn74YWRmZgIAHA4HtmzZAkVRsHv37oDXXn755di/fz/WrFmDTz75BJdccgmmTZuGp556qsn33pJYSJXOlCRJMCrGBpVxcfvcsGt2vYamgIBBMpxym7ZQlnE5+fuDvdhEwcdgFyVSUlICApnVakVaWlqdK153796N/fv349JLLwXg7wFr6MrYk2ma/weO2+3Wj91zzz2QZRkffvghrrjiCowZMyZgPlzbtm0xadIkTJo0CSNHjsR9992Hp556Sp9Tp6onJqknJiYiMzMTmzZtwvnnnw8A8Pl82LJlCwYOHNikNjeVR/XohVSNspHbHVGzaGoZl2o1F29Ul3FpiW3aan5/GGQDvz+ImgmDXRDVrCsWzu+zevVq5ObmIiYmplGvW7NmDYqKijBkyBDExcXhxx9/xH333YcRI0boc+U++OADvPzyy9i4cSMGDhyI++67D5MmTcKOHTuQnJyMWbNmYdCgQejVqxfcbjfef/999OzZEwCQlpYGq9WKtWvXon379rBYLEhMTMRdd92F+fPno1u3bujRoweeeeYZlJWVBflTqZ8Qwl9I1VEMp8/JQqoUUg3dpu3kMi5A4DZtwSrjUvP7w+VzId4cz+8PombE764gqP4XdHUZi5ZgUkzNNnF69erVmDRpUqNfZ7VasWTJEvz5z3+G2+1GdnY2rrnmGsyYMQMAcOzYMdx0002YM2eO3pv2yCOP4OOPP8att96KFStWwGQyYebMmdi3bx+sVitGjhyJN998E4C/5/DZZ5/Fo48+ilmzZmHkyJFYv3497rnnHhw5cgSTJk2CLMuYOnUqrr76apSXlwfvQ6mH2+fWC6maDWYWUqVWrTHbtJ2ujEt1IeyTy7jU3KbN5XOhxFGif38kW5Nb6laJopYkWvMM2xCpqKhAYmIiysvLa+2Y4HK5sHfvXnTq1AkWi0U/Hil7xRYXFyMzMxMHDx5Eenp60K/fWtX3+1qf6u2OjtmPwaN6kGBOiIrSFBTdapZxqbmyt9rJZVwcXge/PyjieVX/lo6dkjs125zqU+WSk7HHLkj08gZh7vjx43jmmWeiKtQ1VvV2R+WuclgMFvZCUNRobBkXg2zg9wdRC2OwowBnnXUWzjrrrFA3o1XShIZSZ6m+3RF7IYhOON0wLxG1DH4HEjWAw+tAiaME5e5ybndEREStFoMd0SmomurvpXOWQBUqtzsiIqJWjcGuibjmJLLU9ftp99hR7ChGpacSMcYYxBviQ9AyIiKihmOwaySj0b/ixeFwwGq1hrg1FCwOhwOA//fXp/n0uXSQwEKqREQUNhjsGklRFCQlJeHo0aMAgJiYGP7QD2NCCDgcDhw9ehRJSUlwqk7/puReG2KN3JSciIjCC4NdE2RkZACAHu4o/MUnxEOKk1BQVgBFVthLR0REYSnkwe65557DX//6VxQWFqJfv374xz/+gaFDh572dW+++SZuuOEGjBs3DqtWrdKPCyEwe/ZsLFmyBGVlZRgxYgQWL16Mbt26Ba3NkiQhMzMTaWlp8Hq9QbsutTwhBDzCg2JXMcqcZdyUnIiIwlpIg92KFSswffp0vPDCCxg2bBgWLlyIUaNGYdeuXUhLS6v3dfv27cO9996LkSNH1nruySefxLPPPotXXnkFnTp1wsMPP4xRo0Zh586dDdpRoDEURYGisI5ZuPKoHhx3HkeJowRGxcheOiIiCnsh3VJs2LBhGDJkCBYtWgQA0DQN2dnZuOOOO/T9RU+mqirOP/98TJ06FV9++SXKysr0HjshBLKysnDPPffg3nvvBQCUl5cjPT0dy5Ytw/XXX1/nNd1uN9xut/64oqIC2dnZDdq6g8KPEAKVnkocsx+D0+dEvCm+2baBISKiyNbathQLWUEuj8eDLVu2IDc390RjZBm5ubnYuHFjva979NFHkZaWhptuuqnWc3v37kVhYWHANRMTEzFs2LBTXnPevHlITEzUv7Kzs5t4V9TauX1uHKk8ggPlB6AJDSnWFIY6IiKKGCELdsXFxVBVtdaepOnp6SgsLKzzNV999RVeeuklLFmypM7nq1/XmGsCwMyZM1FeXq5/HThwoDG3QmFAExrKXGU4UH4Ax13HEW+OR6wpNtTNIiIiCqqQL55oqMrKSvzhD3/AkiVLkJqaGtRrm81mmM0saxGpXD4XShwlKHWVwmKwIMWaEuomERERNYuQBbvU1FQoioKioqKA40VFRXo5kZr27NmDffv2YezYsfoxTdMAAAaDAbt27dJfV1RUhMzMzIBr9u/fvxnuglozTWgod5Wj2FEMj+pBojkRiszFLkREFLlCNhRrMpkwaNAgrFu3Tj+maRrWrVuH4cOH1zq/R48e+P7775GXl6d/XXXVVbjooouQl5eH7OxsdOrUCRkZGQHXrKiowKZNm+q8ZigJIWDz2OD2ubk9WTNwep04VHEIhyoPQZZkJFuTGeqIiCjihXQodvr06Zg0aRIGDx6MoUOHYuHChbDb7ZgyZQoAYOLEiWjXrh3mzZsHi8WC3r17B7w+KSkJAAKO33333XjsscfQrVs3vdxJVlYWxo8f31K31SAe1YMjlUegairMBjPiTHGwGCwwG8wwykaW3WgiVVNR5ipDiaMEqlCRZEmCLIXs3y9EREQtKqTBbsKECTh27BhmzZqFwsJC9O/fH2vXrtUXPxQUFECWG/dD+f7774fdbsctt9yCsrIynHfeeVi7dm3Qa9idKQGhhzpVqDhq9+9iYZSNMCkmf9AzWmBWzFy12UAOrwPFjmJUuCsQY4xBnCEu1E0iIiJqUSGtY9daNaZeTFO5fC7sK92HWFNswBChV/XCo3rgUT0AAJNigsVg0XdEMBvMMMhhs+alRaiailJnKYodxRAQiDfHs5eOiIhaRGurY8eE0MoYFSOMihGxiPVvd6V64PQ5UeGuAACYFTOsRitiTbEwK2aYFFNUzx2ze+w4Zj8Gm8fm/0wMXN1MRETRi8GuFZMkCWaDGWb4w4oQAm7VDZvHhjJXGST4n48xxiDGGAOzwR/0oqG3yqf59O3AJElCspXbgRERETHYhRFJkmAxWGAx+OcLakKD2+f2LxZwlsAgGWBUjIgzxcFqtOo9epEUeKpXExc7imH32vUhaiIiImKwC2uyJMNqtMJqtALwzzXzqB6UOEugOTQYZSPMin/Frdlg1nv0wpVX9eq9dIqsINnCXjoiIqKaGOwiiCIrsMongp5P88GjenDUfhQaNJhkU0BpFZNiCosVt0IIVHoqccx+DE6fE/Gm+LBoNxERUUtjsItgBtkAg2xAjDEGAPTVtoU2/765NVfcmg1mmBVzq1uI4VE9+nZgBtnAXjoiIqJTYLCLIibFBJNiClhx6/A66l1xazaYQ7YQQwiBCncFih3FcPlciDfHs8wLERHRafAnZZTSV9xWlQfRhAaP6kGlpxKlrlIokgKTYgpYcWtWzC3SW+b2ufVeOrPBjGRrcrO/JxERUSRgsCMA/oUYNVfcVi/EqLni1mTw74hhNVj13r9gBj1NaKhwV+CY/Rg8qgcJ5oRWNzRMRETUmjHYUZ3qW4hR7CiGJgJX3AZj6zOXz4ViRzHKXGWwGqzspSMiImoCBjtqkJMXYlRvfXbUfhQCAkbZCIuxaiFG1fy8hsyJ04SGMmcZih3F8Gk+JJoT2UtHRETURAx21CTVW58B/oUOXs0Ll8+FCncFZEmGSfavuK3e5quuFbdOrxPFjmKUu8thNVgRZ44Lxa0QERFFDAY7OmOSJOlz7gDoK27tXjvK3GWQIQdsfWZSTLB77ChxlkAVKpIsSVGxDRoREVFzY7CjoKtvxW25uxwlzhIokgJVqIgxxiDeEB/i1hIREUUOBjtqdnWtuJUlmYWGiYiIgozBjlocF0cQERE1D05sIiIiIooQDHZEREREEYLBjoiIiChCMNgRERERRQgGOyIiIqIIwWBHREREFCEY7IiIiIgiBIMdERERUYRgsCMiIiKKEAx2RERERBGCwY6IiIgoQjDYEREREUUIBjsiIiKiCMFgR0RERBQhGOyIiIiIIgSDHREREVGEYLAjIiIiaiRNE3B6fCh1eELdlACGUDeAiIiIKBwIIeD2aXD7VHh9GgQAn6YBkELdNB2DHREREdEpeFUNLq8Kt0+FEKFuzakx2BERERGdRNUE3F4VLp8KVWvlaa4GBjsiIiIi+IdaXd6qoVZVC3VzmoTBjoiIiKKa26fC7dXg8akIn765ujHYERERUdTx6fPmNGitfeJcIzDYERERBYmmCahCwKiwmlhrpGkCbp8Kpze85s01BoMdERFREDg9Ptg9PggBSBJgVGQYZRlGg8ygF0I1S5R4fOE5b64xGOyIiIjOgFfVUOnyBvQACQF4fBo80ACPv8qZQZFhUmQYFBlGRYIktZ7aZ5HI4/MPtXrU1l+iJJgY7IiIiJpACAGb2weXVz39ufAHwOqVltVBz6h/MegFg6oJvd5cpA61ng6DHRERUSO5vCpsbm+Te4LqCnqK3qMnwaTIDHoN5J83p8HlU+EL0xIlwcRgR0RE1EA+VYPN7Qt6jTNRdW3fSUHPKEswGmQGvTq4vP45c5FQoiSYGOyIiIhOQwgBh0eF0+NrkRBxIugBzqqhXkWW9Dl6JkWGLEdf0Aunrb1ChcGOiIjoFNw+FXa3L+RztlRNwKmpQI2gVz1HL5KDXrhu7RUqDHZERER1UDUBm9vbaktkqJqAqqn64o2aQc+oyFDCOOhVlyhxecN3a69QYbAjIiI6icPtg6OFhl2Dpa6gZ5CrevQM4RH0Imlrr1BhsCMiIqri8Wmwub0RMeRXHfTcPhVwA7IkBZRXMbSSosmRurVXqDDYERFR1NM0f006t+/0NenClSb822lV3+OJoOf/f0sGvWjY2itUGOyIiCiqOT0q7J6m16QLVyeCnv9x9TZoJ3bHCG7Qq7m1l9encai1mTDYERFRVPKpGirdPha1raJvg1a1WCRY+916qsIcS5S0DAY7IiKKKkII2N0+vT4c1e1M9rvl1l6hw2BHRERRw+X116TjJP3GO9V+twZFglGWubVXK8BgR0REEa+116QLRycHPWodGOyIiChiCSHg9KhhV5OOqKkY7IiIKCK1lq3AiFoSgx0REUWUaKhJR1QfBjsiIooYTo8Pdo+PZTUoajHYERFR2POqGipdkbEVGNGZYLAjIqKwJYR/2NXFmnREABjsiKgZ+TchF9CEgCxJMMgSZLnugqZEjeXyqrC5o28rMKJTabkdf+vx3HPPoWPHjrBYLBg2bBi+/fbbes995513MHjwYCQlJSE2Nhb9+/fHa6+9FnDO5MmTIUlSwNfo0aOb+zaIopKqCXh8mn+vTbcPFU4vyhwelNjcKK504bjdjXKnB5UuL8qdHpTYTxwvc3hQ4fTCUdXb4vFpHEajBvGpGsoc/j9XDHVEgULaY7dixQpMnz4dL7zwAoYNG4aFCxdi1KhR2LVrF9LS0mqdn5KSgoceegg9evSAyWTC+++/jylTpiAtLQ2jRo3Szxs9ejSWLl2qPzabzS1yP0SRRAh/b5sqBDTNv2H4iceiyZX7Bap68qqqirlPel4CIEkSFFmCXOP/sgwYZBkKe/yilhACDo8KJ2vSEdVLEiJ0/94ZNmwYhgwZgkWLFgEANE1DdnY27rjjDsyYMaNB1xg4cCDGjBmDuXPnAvD32JWVlWHVqlVNbldFRQUSExNRXl6OhISEJl/nVFw+F/aV7kOsKRaKrDTLexCdijgpqPmHTHHGwa0l+IOeBOWk4Ff9uL79Kyl8sSYdtVZezYsEi4ROyZ1gVIzN8h6NySUh67HzeDzYsmULZs6cqR+TZRm5ubnYuHHjaV8vhMBnn32GXbt2YcGCBQHPrV+/HmlpaUhOTsbFF1+Mxx57DG3atKn3Wm63G273iX6DioqKJtwRUeuiVYU2VRMBIa56zlsrzm2npQkBTRXw1fO8JAGKLEORJMgS9LBnkBn8wg23AiNqnJAFu+LiYqiqivT09IDj6enp+Pnnn+t9XXl5Odq1awe32w1FUfD888/j0ksv1Z8fPXo0rrnmGnTq1Al79uzBgw8+iMsvvxwbN26EotTdMzZv3jw88sgjwbkxohZSM7hV/1qLkOB2poTwz8M6VfCrHuat2ctX/ZgLPFoHh9vHrcCIGinsVsXGx8cjLy8PNpsN69atw/Tp09G5c2dceOGFAIDrr79eP7dPnz7o27cvunTpgvXr1+OSSy6p85ozZ87E9OnT9ccVFRXIzs5u1vsgOp2aK0q1k3rbNE3wh90ZEFVDzvUN69UMfrJ0Iuwx+LUMj0+Dzc2adERNEbJgl5qaCkVRUFRUFHC8qKgIGRkZ9b5OlmV07doVANC/f3/89NNPmDdvnh7sTta5c2ekpqYiPz+/3mBnNpu5wIJaXM3gFhDiGNxC7rTBD4BcY3FHddjTHzP4NQm3AiM6cyELdiaTCYMGDcK6deswfvx4AP7FE+vWrcPtt9/e4OtomhYwP+5kBw8eRElJCTIzM8+0yURnxOnxwV1V0qM1L0yg06u5sreuurg1g1/NsMfgVz+nR4Xdw/IlRGcqpEOx06dPx6RJkzB48GAMHToUCxcuhN1ux5QpUwAAEydORLt27TBv3jwA/rlwgwcPRpcuXeB2u7FmzRq89tprWLx4MQDAZrPhkUcewbXXXouMjAzs2bMH999/P7p27RpQDoWoJXGro+hzckmXunBlr59P1VDp9sGncnEEUTCENNhNmDABx44dw6xZs1BYWIj+/ftj7dq1+oKKgoICyPKJGsp2ux1/+tOfcPDgQVitVvTo0QOvv/46JkyYAABQFAU7duzAK6+8grKyMmRlZeGyyy7D3LlzOdRKLY5bHdGpnG5lb83gJ1et8o2k4CeEgN3tg5PfH0RBFdI6dq0V69jRmXJ5/TW3OORKzSWcS7rw+4MiCevYEUUwn6rB5vbBy2ElamaNKelSvU9vqEu6sCYdUfNjsCMKAm51RK1NQ1f2tkRJFyEEnB6VNemIWgCDHdEZ4lZHFI70BR7NXNKF3x9ELYvBjqiJWHOLItmZlnQBADu/P4haHIMdURM4PT7YPT7W3KKo1ZCSLkTU8hjsiBqBNemIiKg1k09/ChEJIVDp8qLM4WGoIyIinSY0HCg/EOpm6NhjR3QarLlFREQn+6XkZ3yQvwof5r8LAQ0Fdxc0Wx27xmCwI6oHa9IREVFNhbbD+DD/PXyQvwr5pb/oxxPMCfjl+C/om943hK3zY7AjOglr0hERUbUKdwU+3fshPshfja1HvoWo+slglE04v8NFGNXlSlze7UJ0T+0e4pb6MdgR1cCaW0RE5FHd+KpgPT7IX40vCj6HV/Pozw3KHIYxXccht9NoJJgT4dW8MCmtZws/Bjsi+Ms2sOYWEVH00oSGbYXf4YP8Vfjk1w9R6anQn+uS3A1Xdr0ao7uORWZcVghbeXoMdhT1nB4f7G4OuxIRRaP8479gTf5qfLjnXRyxHdaPp8Vm4PIuY3FF13E4K6UHJKn19MqdCoMdRS3WpCMiik5F9kKs3fMe1uS/i10lO/XjccY45Ha6HFd0vQqDModBkZUQtrJpGOwo6miagN3jg6uufZKIiJrIp/nwYf67KHOXoX/6IPRIPRtGOfTlL8jP5qnEun0fYc3u1fj28EZ9EYRBNuK87AtwRddxOL/DxbAYLCFu6ZlpdLDr2LEjpk6dismTJ6NDhw7N0SaiZuPyqrC5vdwKjIiC6rsjmzD/6zkBJTAsBiv6pQ3AgIwhGJgxGH3SB8BqsIawldHHq3rwv4Nf4oP8Vdiwfx3cqlt/bkD6YFzRdRwu7Xw5kizJIWxlcDU62N19991YtmwZHn30UVx00UW46aabcPXVV8NsNjdH+4iCgjXpiKg5HLUX4ZlN87B2z3sAgERzEvqmDcD2o1tR4S7HpsP/w6bD/wMAGCQDerbtjYEZQzAwYwj6pw9CoiUphK2PTEIIbD+6FR/sXo2Pf/0A5e4y/blOSV1wRddxuKLLVWiXkB26RjYjSYim9V1s3boVy5Ytw/Lly6GqKn73u99h6tSpGDhwYLDb2OIqKiqQmJiI8vJyJCQkNMt7uHwu7Cvdh1hTbFiO4YcL1qQjoubg1bx444dl+OfWf8DhtUOChN/0vAHTBk9HkiUZmtDwa+lubC38DlsLN2Nb4WYU2QtrXadr8ll6j97AjCFIj8sMwd1Ehr1le7AmfzXW5L+LQ5UntvhKtbbF6C5jMabbOPRo0yvoiyC8mhcJFgmdkjs1284TjcklTQ521bxeL55//nk88MAD8Hq96NOnD+68805MmTIlbFaQnIzBLjKwJh0RNYdNh77Ggv89il/L8gEAfdL6Y+a5c3B22z71vkYIgcO2Q9h65NuqoPcd9pX/Wuu8rLj2/pCXORQDM4YgJ7FT2P4sbQnFjmNYu+d9rMlfhZ3FP+jHY4yxuLjjZRjTdTyGZg1v1p+zERPsvF4vVq5ciaVLl+KTTz7BOeecg5tuugkHDx7Ec889h4svvhhvvPFGk24g1Foi2Dm9Tuw8ugdJlnhYjAZ+4wYRa9IRUXMotB3G09/Mwyd71wAAki0puGvoA7jqrGsgS3Kjr3fcWYxthVv0Hr2fS3ZCE4HTRVKsbTAgfbDeq3dWm54wyNG97tHhteOzfR/jg92rsenw1/pnZpAMGN5+JMZ0G4cLcnJbbD5j2Ae7rVu3YunSpVi+fDlkWcbEiRNx8803o0ePHvo5P/zwA4YMGQKn09m0Owixlgh2Do8TWw//ghhDDAyyAoMiw6jIMCgSTIrMoNdErElHRMHmUd14/fuleHHbIrh8TsiSjOt63og/Df4zEsyJQXsfm6cSO4q26T163x/Lg0f1BJwTa4xD37QBGJjpn6fXu20/mA2RP8fdq3nxzcGv8EH+aqzf/ylcvhP5om/aAFzRdRwu63wFUqxtQtK21hTsGh37hwwZgksvvRSLFy/G+PHjYTTWvolOnTrh+uuvb+ylo5aAv6Za9cR+CYCiyDAx6DUYa9IRUXP434EvsGDjo9hfvheAfyXljBFz0L1Nz6C/V5wpHudmn49zs88H4A+UPx77Htuq5unlFW2BzVOJjYe+xMZDXwLw71faq20fDMwYggEZg9E/YzDiTfFBb1soCCHww7Ht+GD3Knz06wcodR3Xn+uQ0BFjuo3H5V3GokNix9A1shVqdI/d/v37kZOT01ztaRVausfudGP/1UHPKEswGmQGvRpYk46ImsPhykN46pvH8Nm+jwEAbayp+POwGRjTdXzI/v5VNRX5pb9UzdP7DtsKN6PYeSzgHAkSuqX0CJinlxrTNiTtbaqC8n1Yk/8uPshfhQMV+/XjyZYUjK7aCaJ3276t5udga+uxa3Sw27x5MzRNw7BhwwKOb9q0CYqiYPDgwY1vcSvT2oJdXRRZqurR8wc9WW4df8BbEmvSEVGwuX1uvLpjCV7KWwyX6oIiKbi+10TcOuiuVtcTJoTAgYr9esjbWrg5IAhVy07IqerR88/Ty07IaTWhqNpxZwk+/vUDfJC/Gt8fzdOPWwxWXNzxMlzR9SoMazeiVRZ8bm3BrtFDsdOmTcP9999fK9gdOnQICxYswKZNmxp7SWoCVRNwaipQ1VMVTUGPNemIqDl8UfA5/rrxURyoKAAADMochpnnzkbXlO4hblndJElCh8SO6JDYEeO7/wYAcMxx1D90W9Wrt/v4zzhQsR8HKvZj9S//AeAv/6GXWMkciq7JZ4WkOoPT68D6/Z/ig/zV2HjwS6jC//NMlmQMb3cerug6Dhd1vBQxxtgWb1s4a3SPXVxcHHbs2IHOnTsHHN+7dy/69u2LysrKoDYwFMKhx+50FFmCsWpBRqQEPdakI6LmcLCiAH/d+Bg2FKwDALSNScf0YTMwusvYVtez1VgV7gpsL9qiz9P78dj38GqBCzLiTPHonz5In6fXq20fmJTmWZDh03z49vBGrNm9Cuv2fQynz6E/16ttX1zR9SqM6nxlWA0fh32PndlsRlFRUa1gd+TIERgM0b0EuzVRNQFVU/W5Z+Ee9FiTjoiCzeVzYen2f2Lp9hfgUT0wSAbc2GcKbhlwO2JNcaFuXlAkmBMwssNFGNnhIgD+e/7x2A5sLdyMrUc2Y/vRrbB5KvHVgfX46sB6AIBZMaN3234YUFU0uV/6wDP6PIQQ2Fn8Pdbkv4u1e95DibNYf659fAdc0fUqXNF1HDomdT7FVaihGt1jd8MNN+DIkSNYvXo1EhP9y7zLysowfvx4pKWl4a233mqWhrakSOixOx1FlmCQZZgM/rCntNKgx5p0RBRsQghsKFiHv258TN+hYFjWuXjg3NnonNw1xK1rWT7Nh19KfgqYp1dz9SngHxrt0ebsquHbIRiQMQgp1tTTXvtgRQE+zH8XH+SvDijGnGROxmVdxmBM13HomzYg7HtFW1uPXaOD3aFDh3D++eejpKQEAwYMAADk5eUhPT0dn3zyCbKzw3/vtWgIdierDnpGxR/2WkPQY006Igq2gvJ9WLDxUXx9YAMAID02A/ec8xAu7XR52AeMYBBCYH/5Xn+PXlWv3mHbwVrndUzsrPfoDcwciqy4dpAkCWWuUnz86xp8kL8K24u26uebFTMuzMnFmK7jMbz9eTAqppa8rWYV9sEOAOx2O/79739j+/btsFqt6Nu3L2644YY6a9qFo2gMdieTpRNDt0ZFgkFpfFX1pvL4NNjcrElHRMHj9Dnx0rbn8cqOf8GreWCQjZjY92b8sf+fYDXGhLp5rVqR7UhV0PP36uWX/lLrnPTYDGQn5CCvcAt8wgfAX3plaLtzMabrOFzc8TLEtbJVxcESEcEu0jHY1dYSQY816Ygo2IQQWLfvIzz9zeM4YjsMADi3/Ug8MHw2cpI6hbh14ancVYa8oi36Dhk7j32vhzkA6NGmF8Z0HYdRXa5EWmx6CFvaMlpbsGvyaoedO3eioKAAHk/g6pqrrrqqqZekVkwTAm6fqs91kyTApCgwKlLVVmhnFvRYk46Igm1f2a+Y/79H8M2hrwAAmXFZuO+cv+Cijpdx2PUMJFqScEHOJbgg5xIA/rIl3x/Nw77yvRiUORRdkruFuIXRrdHB7tdff8XVV1+N77//HpIkobrDr/qbRFXZ2xINhEBV0PM/liToK26r971tCNakI6Jgc3jtWLLtObz2/cvwaV4YZRMm9/sjpva/rcU2ho8mVmMMhrY7F0PbnRvqphCARnez3HXXXejUqROOHj2KmJgY/Pjjj/jiiy8wePBgrF+/vhmaSOFAiOq5cT6UOTwotrlQ7vTAUU9oE8K/2rXM4WGoI6KgEELgoz3vY/zbl2Hp9n/Cp3kxMvsi/Pc3H2La4OkMdRQVGt1jt3HjRnz22WdITU2FLMuQZRnnnXce5s2bhzvvvBPbtm1rjnZSmKkOeh5ogOdEj55R9tfQc3hYk46IgmdP6W4s+N8j+PbwRgBAu/hs3D/8YX24kChaNDrYqaqK+Hj/ypbU1FQcPnwY3bt3R05ODnbt2hX0BlJkCAh6RERBYvNU4p9b/4HlP7wCn/DBrJgxtf9tmNT3j7AYLKFuHlGLa3Sw6927N7Zv345OnTph2LBhePLJJ2EymfDiiy/W2o2CiKjSU4ky13GkWtuyrAQFjRACa/a8i4Wb5uOY4ygA4MKcXNx3zl/QLiH866kSNVWjg91f/vIX2O12AMCjjz6KK6+8EiNHjkSbNm2wYsWKoDeQiFovTWgodhzDEduhqq/DOGI7jELbYf2xzXNi/+gkczIy4rKQGZeFjLgsZMW18z+Ob4fM2CykWNtwtSKd1u7juzD/6znYUvgtACA7IQcPnDsL52VfGNqGEbUCjQ52o0aN0n/dtWtX/Pzzzzh+/DiSk5P5FzJRhHH73Ci0HcZh26GAsHbEdgiFtiMotB+BT/Oe9joWxQKX6kKZuxRl7lL8XPJjneeZFTPSYzORGd/uROjTv9ohPTYjoirWU+NUeiqxeMtCrPjxNahChUWx4OYB0zCx703Ntmk9UbhpVLDzer2wWq3Iy8tD79699eMpKSlBbxgRNS8hBMrdZbXCWs3Hx50lp72OIilIi83Qw1dGXCYy49oFPI4xxqLSU1kVDqsCYuUhFNoP43DlIRTaj+CYvQhu1Y2Cin0oqNhX53tJkJAakxYQ9k7u+YuP0Or20UwTGt7fvRILv12g/5nM7TQa95zzEDLjskLcOqLWpVHBzmg0okOHDqxVRxQGfJoPR+1FdfS0HdaHTJ0+x2mvYzXE+INafBYyY/3hqWaYahuTBoN8+r9K4k3xiE/pjm4p3et83qt6cNReVEfv4Il2u1U3jjmKcMxRhB1H616BH2eM87exqq0nh8C2MWmQpZbbIo/OzM8lOzHv69n6vqMdEzvjgXNnY3j780LcMqLWqdFDsQ899BAefPBBvPbaa+ypIwohh9ceEHoOV89tq/TPdzvqKIImTr8KuY01tUbPV1bV8OeJHrcEc2KLTLMwKia0S8iud+K7EAKlrpITYa/yEI7YD+s9f0cqD6PMXQqb14bdx3dh9/G6V+kbZCPSq3oYqwNqzZ6/9LhMrqZsBSrc5Xjuu7/h7Z/+DU1osBpicMvA2/H73lM4HE90Co0OdosWLUJ+fj6ysrKQk5OD2NjYgOe3bt0atMYRRSshBI47S+qc21a9OKHcXXba6xhkIzLjMvXeq4Bhy7gsZMRmwWwIj7lJkiQhxZqKFGsqerXtW+c5Tq8j4HM6eXj5qL0QPs2LQ5UHcKjyQL3vlWJtg4zYLGTpPX8neisz49oh0ZzEOcXNRBMaVu/6D57d/FeUuo4DAEZ1HoPpw2YiPS4zxK0jav0aHezGjx/fDM0gii5e1YNC+5EaCxOO6D1tR2yHUWg/DI/qOe114kzxAT1OJw87psa0japhR6sxBp2Tu6Jzctc6n/dpPhxzHA1cEKL3/PmDoNPnwHFnCY47S7Cz+Pu638cQU9Xjl1l7nl9cO6TFpjdoeJoC/XhsB+b/7xF8fzQPANA5qRtmjpiNIVnDQ9swojAiCcFt109WUVGBxMRElJeXIyEhoVnew+FxYuvhXxBjiIEiK83yHtR6HHcWY8m25/HjsR04YjuEYscxCJz6W0+ChLax6ciIzURWfM3w1k4fRuRCgeASQqDCXV5rTqI+zG07hBJn8WmvI0sy0mLST/yeVZVzyYw/8fsXY4w97XWiRZmrFIs2P43//vwmBARijXG4deCduL73RBhlY6ibR3RKXs2LBIuETsmdYFSa589rY3IJ/0lJ1Ix8mg9v7/w3ntvyt4B6boC/tEfgfLbAVZ5psemcS9TCJElCoiUJiZYk9EjtVec5bp/bP6evKuwF9PzZDuslYArt/nIwKKr7vRLNSbV6WWv+/kdDTT9VU7Fy1wr8Y/PT+tSCMV3H4+5hD6BtTFpoG0cUphod7GRZPuVfNlwxS+S3rfA7zPt6Nn45/jMAoGdqL0zq+0dkJ+QgMy4LyZbI/8EdicwGM3ISOyEnsVOdz1cXbT55ZW/148O2Q7B5KlHuLkO5uwy7SnbWeR2TYvLP76sR+DPjTwz5ZsRmhnXw//5oHuZ9PRs7i38AAHRL6Y6Z5z6CgZlDQtwyovDW6GC3cuXKgMderxfbtm3DK6+8gkceeSRoDSMKV8WOY1i4aT7ez18FAEgwJ+L2wffg2h7Xc9g9CsiSjLTYdKTFpqNv+oA6zzm5pt/JIfCYvQge1dOAmn5tT6od2C4gCCaYm2cqyZk47izBPzY/hZW73gLgL0/zp8F/xnVn/57zEomCIGhz7N544w2sWLECq1evDsblQopz7KgpvJoXK358HS9sWQib1wYJEq7ucR3uGHIvki0sDUQN19CafqcTZ4w7MdxfY55f9bFUa9sW+/tH1VS8/dMbeO67Z1DpqQAAXHXWtbhryP1oE5PaIm0gag4RO8funHPOwS233BKsyxGFle+ObML8r+cgv/QXAECvtn0x49w56JPWL8Qto3DU6Jp+J/X8FdqOoNR1HDavDfmlv+h/Lk9WV02/kxfoBKOmX17RFsz7eo4+7NyjTS/MGDEb/dMHnfG1iShQUIKd0+nEs88+i3bt2gXjckRh46i9CM9smoe1e94D4N/k/o6h9+Lq7tdFVZkRalmNqekXuNfvicfBqOmXEZeFJHP9+4SXOIqx8NsFeG/3OwCAeFMCbh9yD37T4waOVBA1k0YHu+TkwG9iIQQqKysRExOD119/PaiNI2qtvKoH//5hGV7ctggOrx0SJPym5+9w++DpSLQkhbp5RKet6adqKo45imr1+NUMgQ6v/bQ1/SwGa1VPX2BNv2LnMSzZukifljC++29xx5B7kWJt05y3TRT1Gh3s/va3vwUEO1mW0bZtWwwbNgzJyclBbRxRa7Tp0NeY/79HsLdsDwCgb9oAzBwxBz1Te4e4ZUQNp8iKf3VtXBbqWuIhhEClpwKHbYf827bpc/2O6CGw2HkMLp8Tv5bl49ey/Drf5+zUPpg54hFOSyBqIY0OdpMnT26GZhC1foW2w3j6m3n4ZO8aAECyJQV3DX0AV511DYddKeJIkoQEcyISzIno0ebsOs9x+9wosh8JCHzVoc/utePq7tfh6u7XcdiVqAU1OtgtXboUcXFx+O1vfxtw/O2334bD4cCkSZOC1jii1sCjuvHa9y9jybbn4PI5IUsyJpz9e9w26M+tspwEUUsxG8zokNgRHRI7hropRFSl0d0M8+bNQ2pq7aXpaWlpeOKJJ4LSKKLW4n8HvsBv/nMF/rH5Kbh8TgxIH4zlV7+LB86dzVBHREStTqN77AoKCtCpU+2K6zk5OSgoKAhKo4hC7XDlITz1zWP4bN/HAIA21lT8edgMjOk6nrtFEBFRq9XoYJeWloYdO3agY8eOAce3b9+ONm242onCm9vnxqs7luClvMVwqS4okoLre03ErYPuQrwpPtTNIyIiOqVGB7sbbrgBd955J+Lj43H++ecDADZs2IC77roL119/fdAbSNRSvij4HH/d+CgOVPh7ngdlDsPMc2eja0r3ELeMiIhaG1mSYDbIiJUlaMIb6uboGh3s5s6di3379uGSSy6BweB/uaZpmDhxIufYUVg6WFGAv258DBsK1gEA2sakY/qwGRjdZSyHXYmISCdJgNmgwGxQYDL4lyl4VS/cvhA3rIZGBzuTyYQVK1bgscceQ15eHqxWK/r06YOcnJzmaB9Rs3H5XHh5+wtYtv2f8KgeGCQDbuwzBbcMuB2xprhQN4+IiFoBCYCpKshZjK2/dE+TtxTr1q0bunXrFsy2ELUIIQTW7/8Uf934GA7bDgIAhmWdiwfOnV1vlX4iIooeEgCjQa7qnZPDavSm0cHu2muvxdChQ/HAAw8EHH/yySexefNmvP3220FrHFGw7S/fiyc3zsXXBzYAANJjM3DPOQ/h0k6Xh9U3LhERBZ9RORHmZDk8fyY0Oth98cUXmDNnTq3jl19+OZ5++ulgtIko6JxeB/6V9zxe3fESvJoHBtmIiX1vxh/7/wlWY0yom0dERCFiUGRYqnrnwjXM1dToAsU2mw0mk6nWcaPRiIqKikY34LnnnkPHjh1hsVgwbNgwfPvtt/We+84772Dw4MFISkpCbGws+vfvj9deey3gHCEEZs2ahczMTFitVuTm5mL37t2NbhdFBiEEPt27Fle/fRleylsMr+bBue1H4r/Xfog7h9zLUEdEFIUUWUKsyYCUWDOSY0ywmgwREeqAJgS7Pn36YMWKFbWOv/nmmzj77Lr3E6zPihUrMH36dMyePRtbt25Fv379MGrUKBw9erTO81NSUvDQQw9h48aN2LFjB6ZMmYIpU6bgo48+0s958skn8eyzz+KFF17Apk2bEBsbi1GjRsHlcjXuRins7Sv7Fbd9OBn3fjoNhfYjyIzLwjO5z+O50UuRk1S7yDYR0ZmyGBXEmQ3+3h9O72hVZEmC1aggKcaElFgzYswGKBES5mqShBCiMS947733cM011+B3v/sdLr74YgDAunXrsHz5crz99tsYP358g681bNgwDBkyBIsWLQLgL5uSnZ2NO+64AzNmzGjQNQYOHIgxY8Zg7ty5EEIgKysL99xzD+69914AQHl5OdLT07Fs2bIG19mrqKhAYmIiysvLkZDQPNtGLdq0CBYlBX3TBsBsMDfLe0Qrh9eOJduew2vfvwyf5oVRNmFyvz9iav/bYDVYQ908IopARkVGnNkAgxLYX6JqAl5V079UrVE/cukM1VWeJNj85U7c6JTcCUbF2Czv0Zhc0ug5dmPHjsWqVavwxBNP4D//+Q+sViv69u2LTz/9FBdccEGDr+PxeLBlyxbMnDlTPybLMnJzc7Fx48bTvl4Igc8++wy7du3CggULAAB79+5FYWEhcnNz9fMSExMxbNgwbNy4sd5g53a74Xa79cdNGVJujAPlB3DH2jsAADHGWPRLG4iBGYMxIHMIerftB4vB0qzvH6mEEPj41w/w9KZ5OGovBACMzL4I9w3/CzcpJ6JmIUsSYs2GestgKLIERVb05zVNwKtp8Po0eDUBVdXAqBdckgSYFAVmo3/eXLRpUrmTMWPGYMyYMbWO//DDD+jdu3eDrlFcXAxVVZGenh5wPD09HT///HO9rysvL0e7du3gdruhKAqef/55XHrppQCAwsJC/RonX7P6ubrMmzcPjzzySIPaHQw2jw2ju4zGVwe+hs1TiY2HvsTGQ18CAIyyCWe37Y2BGUMwIGMI+qcP4mbzDbCndDcW/O8RfHvY/4+CdvHZuH/4w7gg55IQt4yIIpEEwGoyIMakNGpFvSxLMMuKHjiEEPCqAj5V8wc+VUPjxtEICKw1F27lSYKtyXXsqlVWVmL58uX417/+hS1btkBV1WC0q17x8fHIy8uDzWbDunXrMH36dHTu3BkXXnhhk685c+ZMTJ8+XX9cUVGB7OzsILS2bj3b9sR/r3sH3x36GUcqDyKvaCu2FW7G1sLvcMxRhO1FW7G9aCuWbv8nJEjoltLD36OXMQQDM4egbUxas7Ut3Ng8lfjn1n9g+Q+vwCd8MCtmTO1/Gyb1/SN7PomoWZgNCmKDND9LkiSYDFLAMKFP1eBV/UO4Po3Dt6diVPxFg6M9zNXU5GD3xRdf4F//+hfeeecdZGVl4ZprrsFzzz3X4NenpqZCURQUFRUFHC8qKkJGRka9r5NlGV27+ovI9u/fHz/99BPmzZuHCy+8UH9dUVERMjMzA67Zv3//eq9pNpthNrf8PDdZktEtpQd6pPbC9b3+ACEEDlYWYGvhZmw78h22Fm5GQcU+/HL8J/xy/Ce8udO/Ajg7oYM/5GUMwYCMweiQ0DHq/kALIbBmz7tYuGk+jjn8i20uzMnFfef8Be0Smi+UE1H0UmQJcWZjs83VqmZQZBgUwIoTw7ceVavq1ePwbaSVJwm2RgW7wsJCLFu2DC+99BIqKipw3XXXwe12Y9WqVY1eEWsymTBo0CCsW7dOX3ChaRrWrVuH22+/vcHX0TRNnx/XqVMnZGRkYN26dXqQq6iowKZNm3Dbbbc1qn2hIEkSshNykJ2Qg3Fn/QYAUOw4hm2F/pC3tXAzfin5CQcqCnCgogDv/vJfAECqtW1V0PPP0+uW3B2KHLnzCnYf34X5X8/BlkJ/aZzshBw8cO4snJd9YWgbRkQRSZKAWJMRVlNo/l6VZQkWWQGM0Tt8q8gSLAYFZqMSkStZg6nBwW7s2LH44osvMGbMGCxcuBCjR4+Goih44YUXmvzm06dPx6RJkzB48GAMHToUCxcuhN1ux5QpUwAAEydORLt27TBv3jwA/rlwgwcPRpcuXeB2u7FmzRq89tprWLx4MQB/MLr77rvx2GOPoVu3bujUqRMefvhhZGVlNWq1bmuSGtMWl3a+HJd2vhwAUOmpxPaiLdha+B22HdmMH47tQLHzGD7Zuwaf7F0DAIgzxqFfxiAMrOrV69W2D0xK+K+8rXBXYPGWhXhr5+tQhQqLYsFNA/6ESX1vjoj7I6LWRYK/fEms2dCqRkXqH749MYSrRUDSU2RJ3wXi5NXGVL8GB7sPP/wQd955J2677bag7RE7YcIEHDt2DLNmzUJhYSH69++PtWvX6osfCgoKIMsnfjPtdjv+9Kc/4eDBg7BarejRowdef/11TJgwQT/n/vvvh91uxy233IKysjKcd955WLt2LSyWyJhvFW+Kx3nZF+q9U26fGz8e21HVo/ctthdtg81rw9cHNujbZpkUE/q07Y8BGYMxMGMI+qYPQJwpPoR30Tia0PD+7pVY+O0CHHeWAAByO43G9GEPIiu+XYhbR0SRyGSQEWuqXb6ktfIP38qoLugUrmVWZEmC2SDDbFRgDJPPvrVpcB27b775Bi+99BJWrFiBnj174g9/+AOuv/56ZGZmYvv27Y0eim3NWqKOncPjxNbDvyDGEBPUYVOf5sPu4z/75+lVDeFWh6FqsiSje5ue+jy9gRmDkWJNDVobgunnkp2Y9/VsbC/aCgDomNgZ9587C+e2HxnilhFRJFJkCTGm+suXhCshqufpVS3KaEXz9MK9PElrq2PX6ALFdrsdK1aswMsvv4xvv/0WqqrimWeewdSpUxEfHz69QKcSzsHuZEIIFJTvw5bCb/Wwd6jyQK3zchI76UO3AzOGICu+fUiHHirc5Xjuu7/h7Z/+DU1osBpicMuA2/H7PlNgVGpvaUdEdCYkADEmA6yNLF8SzqoDXiiGbyOpPEnYB7uadu3ahZdeegmvvfYaysrKcOmll+Ldd99t6uVajUgKdnUpshfq5VW2FW5G/vFfIE76t1tabMaJEisZQ9AluRtkqfm7xTWhYfWu/+DZzX9Fqes4AGBU5zGYPmwm0uMyT/NqIqLGMxv824BF+wrL5h6+lQAYq1azhnuYqymigl01VVXx3nvv4eWXX2awa6BQBruTVbjLkVe4RV95u/PY9/AJX8A5CeZE9E8/sSCjZ2qvoPec/XhsB+b/7xF8fzQPANA5qRtmnDsLQ9udG9T3ISIC/PPS4swGzuWqR7DKrBiVE2EuEsNzRAa7SBNtwe5kTp8TPxzNw9aqOXo7irbB6XMEnGNRLOiT1h8DMv1Br1/aAFiNMU16vzJXKRZtfhr//flNCAjEGuNw68A7cX3viTDKzfNNQkTR63TbgFHdGlNmJZrKk7S2YHfGO09Q5LEarBiSNRxDsoYDALyaF7tKfsLWI/55enmFW1DmLsXmI99g85FvAACKpKBHai+9R29AxmAkWZJP+T6qpmLlrhX4x+anUe4uAwBc0XUc/jxsBnfXIKKga+o2YOR3ul0yVCFgUmSWJwkx9tjVIdp77E5HExr2lu0JmKd3xHa41nmdk7rpRZMHZgxBZlyW/tz3R/Mw7+vZ2Fn8AwCgW0p3zDh3DgZlDm2x+yCi6GEyyIgzGyO+94haXmvrsWOwqwODXeMdrjxUFfT8K29/LcuvdU5mXBYGZAyBBAkf5K8C4C+mfNvguzHh7D/AILMDmYiCS5H9w67hWEaDwkNrC3b8SUpBkRXfDlnx7TCm23gAQKnrOLYVfqfX0vu5+EccsR3GkfzV+mvGdrsGdw29H6kxbUPUaiKKVP5twAywmvhjjqIL/8RTs0i2pODijpfh4o6XAQAcXju2F23DtsLNOGovwrjuv8GAjMEhbiURRSKLUUGsieVLKDox2FGLiDHGYnj78zC8/XmhbgoRRSijIiOW5UsoyjHYERFRWGP5EqITGOyIiCgssXwJUW0MdkQUFJLk7zlRZMn/f0mCXPVrWQI0AWhCQNMEVCGgCQFV8/+fa/OpscwGBbFmA8uXEJ2EwY6ITksCaoS0E+FNlgGl6vGZ9JiIqpCnCgFNQ0Doqw6DzH4E+MuXxJmNAUVyiegEBjsiqgppVb1sEvSgpsgnet6akyRJMCjSKf9Cqu7pUzWhB0FNoCoM+gMgRS5/+RIjrCbOoyM6FQY7ogh3qiFSRZbCZihLliXIkFDf/HghqoKeFjjMq/f8sdcvLEnwly+JYfkSogZhsCMKY809RBpOJEmCUtXbWJ/T9foJwfDXmhgVGXFmA/cdJWoEBjuiVkySAEWWQzZEGmlO1+sH+Hv8avbycaFHy1NkCTEmli8hagoGO6IQOVVvm0GWIUuImt621uR0w9Nc6NF8JAAxJgOsLF9C1GQMdkQtxGxQYDLI7G0Lc1zo0TzMBgVxZs6jIzpTDHZEzcxQNU+I2xxFDw75Nhy/P4iCi8GOqJlwmyM6lWgf8uX3B1HzYLAjCjJuc0TB0Ngh3+ohXrWV9/pVly+JNRv4/UHUDBjsiIKI2xxRS2pIbb/W1OtnMsiIMxv5/UHUjBjsiIKA2xxRa9RaFnoosn/Y1WzgsCtRc2OwIzoD3OaIwl1zLvTwf38YYDXxRw1RS+F3G1ETcJsjiiZNWegBAFajwu8PohbGYEfUSNzmiChQQ4Z8iahl8PuQqIG4zREREbV2DHZEpyFJQIyR2xwREVHrx2BHdArc5oiIiMIJgx1RHbjNERERhSMGO6IauM0RERGFMwY7InAbMCIiigwMdhT1uA0YERFFCgY7ilrcBoyIiCINgx1FHW4DRkREkYrBjqIGtwEjIqJIx2BHUcFkkBFr4jZgREQU2RjsKKIpsr98idnAYVciIop8DHYUkbgNGBERRSMGuxCRZQmJFiNMsgEaJHhVDUKEulWRgduAERFRtGKwCyGjIiPGZIAi+4cJfaoGryrgVTX4NA2qxqTXGNwGjIiIoh2DXStiUGQYFMAKf9DTNAGPqvkDnyagqhoY9WrjNmBERER+DHatmCxLsMgKUBVYhBDwqqIq6GlRP3zLbcCIiIgCMdiFEUmSYDJIATslROvwLbcBIyIiqo3BLsxF2/AttwEjIiKqH4NdhInU4VtuA0ZERHR6DHYRLtyHb6u3AYs1GziPjoiI6DQY7KJQuAzfchswIiKixmGwo1Y3fMttwIiIiJqGwY5qCdXwrX8enQFWE/9YEhERNQV/glKDNPfwrcWoINbEbcCIiIjOBIMdNUmwhm+5DRgREVHwMNhRUJxu+NaratBqJD1uA0ZERBR8DHbUbOobvhVCwGLkNmBERETBxmBHLUYfviUiIqJmwYlNRERERBGCwY6IiIgoQjDYEREREUUIBjsiIiKiCMFgR0RERBQhGOyIiIiIIkTIg91zzz2Hjh07wmKxYNiwYfj222/rPXfJkiUYOXIkkpOTkZycjNzc3FrnT548GZIkBXyNHj26uW+DiIiIKORCGuxWrFiB6dOnY/bs2di6dSv69euHUaNG4ejRo3Wev379etxwww34/PPPsXHjRmRnZ+Oyyy7DoUOHAs4bPXo0jhw5on8tX768JW6HiIiIKKQkIRqyo2fzGDZsGIYMGYJFixYBADRNQ3Z2Nu644w7MmDHjtK9XVRXJyclYtGgRJk6cCMDfY1dWVoZVq1Y1uV0VFRVITExEeXk5EhISmnydU3H5XNhXug+xplgoLNpLREQUlryqF26fG52SO8GoGJvlPRqTS0LWY+fxeLBlyxbk5uaeaIwsIzc3Fxs3bmzQNRwOB7xeL1JSUgKOr1+/HmlpaejevTtuu+02lJSUnPI6brcbFRUVAV9ERERE4SZkwa64uBiqqiI9PT3geHp6OgoLCxt0jQceeABZWVkB4XD06NF49dVXsW7dOixYsAAbNmzA5ZdfDlVV673OvHnzkJiYqH9lZ2c37aaIiIiIQihs94qdP38+3nzzTaxfvx4Wi0U/fv311+u/7tOnD/r27YsuXbpg/fr1uOSSS+q81syZMzF9+nT9cUVFBcMdERERhZ2Q9dilpqZCURQUFRUFHC8qKkJGRsYpX/vUU09h/vz5+Pjjj9G3b99Tntu5c2ekpqYiPz+/3nPMZjMSEhICvoiIiIjCTciCnclkwqBBg7Bu3Tr9mKZpWLduHYYPH17v65588knMnTsXa9euxeDBg0/7PgcPHkRJSQkyMzOD0m4iIiKi1iqk5U6mT5+OJUuW4JVXXsFPP/2E2267DXa7HVOmTAEATJw4ETNnztTPX7BgAR5++GG8/PLL6NixIwoLC1FYWAibzQYAsNlsuO+++/DNN99g3759WLduHcaNG4euXbti1KhRIblHIiIiopYS0jl2EyZMwLFjxzBr1iwUFhaif//+WLt2rb6goqCgALJ8InsuXrwYHo8Hv/nNbwKuM3v2bMyZMweKomDHjh145ZVXUFZWhqysLFx22WWYO3cuzGZzi94bERERUUsLaR271op17IiIiKghWMeOiIiIiJoFgx0RERFRhGCwIyIiIooQDHZEREREEYLBjoiIiChCMNgRERERRQgGOyIiIqIIwWBHREREFCEY7IiIiIgiREi3FCMiIiIKRx7VA5fPBZ/mQ4wxBpIkhbpJABjsiIiIiBqkZpgzySbEm+IRZ4qD1WiFQW4dkap1tIKIiIioFXL73HCr7jrDnEkxhbp5tTDYEREREdVQV5iLN8fDYrC0yjBXE4MdERERRT23zw2XzwVVqDAr5rAKczUx2BEREVHUEULoc+aqw1yCOQHx5nhYDVYYFWOom9gkDHZERERB5FE9MMgGyBIrirU2dYW5RHMi4sxxYR3mamKwIyIiCgKXzwWH1wGTbIJDc0CDBkVSYFSMMMrGiAgN4ahmmNOEBpNiQpIlCbGm2IgJczUx2BEREZ0Br+qFzWODUTYiPTYdCeYEaELTw4Tda4fL50KlpxISJBhkA4yKESbFxF69ZiKEgFt1w+1zB4S5OFMcLAZLxIW5mhjsiIiImkDVVD2stYlpgyRLEiwGi/681WhFIhIhhIBX88KjeuBRPbB77HD73Kj0VUITGiRJgkkxwSgbYZANrabQbbg5OcyZFXPUhLmaGOyIiIgaQRMa7B47VKEi0ZyIZGsyYowx9Z5fHdyqV1amWFOgaqoe9NyqWw97Ns0GAaEP3bJX79Sqw1z1MKtFsehhrjUVDW5J0XfHRERETSCEgMPrgFt1I94UjxRrCuJMcU3qYVNkBVbZCqvR6r92zIlePa/qhcPr8A/fuiuhChWyJMOkmPzDuLIxqnv16gpzyZbkqA5zNUX33RMRETWA0+uE0+uE1WhFdkI24s3xQe1JO7lXL9marPfqeTWvf2GGxwGPzwO7ZgcAGGQDDLIBJsUERVaC1pbWqK4wl2JJ8S+AYJgLwE+CiIioHm6fG3avHRbFgsz4TCRaElssROi9erAiwZwAESPg03z6EG51r57NY4MmNADwz9WrWoUb7r16NcOcEAJmxYw21jaIMcYwzJ0CPxUiIqKT+DQfbG4bFFlBWmwakixJId99QJIkf2hTjIhFLJKtyfrqW4/q8YdQjx0enwcO4YAmNH3oNlx69YQQcPlccKvugDAXa4qFxWBhmGsAfkJERERVVE2FzWODEALJ1mQkWZL0eXCtkSzJsBgs/tW4ZqBtbFt41RMrcJ0+/xCy3WOHT/ggQWp1vXonhzmLwcIwdwb4aRERUdQTQsDutcOrepFgTkCKNQUxxphWEXwaK6BXDyd69byqVx9ari67IiD0Xj2jYmyxEFUd5lw+FwAwzAURPzkiIopaQgg4fU64fC7EGeOQEZeBOFNcRJUYqdmrF2+ORypS4VW9+ipcp9cJh9cBh8cBVaj+IsrKiSHcYIVbvWfO54aAv2cuNSZV3wEiHIaKwwGDHRERRaXqLcCsBivaJ7RHvCk+asJFda9ejDEGSZYkaELTh3DdPjccPgfcPjecXicEhF5upbG9enWGudhU/wIIhrlmwWBHRERRxaN6YPPYYFbMyIjNQKIlMWp2JaiPLMkwG8wwG8yIN8cDQMAKXJfXvzWa0+uET/MBQMAeuDV7ODWhwe3z7wABCTArZoa5FsRgR0REUcGn+WD32CFBQtuYtkiyJMFsMIe6Wa1WdZ28GGMMYPH3vlUHPY/q8c/V8/mHcjVoUCQFQoiAMBdr9M+ZY5hrOQx2REQU0TShweaxQdVUJFmSTrsFGNVNkiS9Vw8A2qANfJpPH8J1+VyQJRkxxhiGuRBisCMiooh08hZgbWLaINYYG5YrXVur6l49q9GKRCSGujkEBjsiIopATq8TTp8TscZYpMWmBX0LMKLWisGOiIgiRs0twNrFt0O8OZ410Siq8E87ERGFPa/qhc1jg1E2Ij02HYmWxJBvAUYUCgx2REQUtlRNRaWnEhIkpFhTkGxN9m+vRRSlGOyIKKiEEFCFClVToQkNqvD/HwAUSYEsyZAlGYqsQJEUTmSnJtGEBrvHDlWoSDAnINmSjFhTbKibRRRyDHZE1GCa0PxhrUZoq/61gIBU9V/N0GYxWGCS/UNiHs2/X6VP88Htc+uvq6bI/uBXHQAZ/uhkJ28B1iamDeJMcfwzQlSFwY6IAEAPbNU9bDUDHAD/tkI4EbZkWYZVserbDCmSoj9XM6Cd/AO3Zo9e9XtV/9rj88CjeeDTfFA1FR7VA1VTIar+kyAFBL7qX1f3AlJkc3qdcHqdsBqjbwswooZisCOKAieHtupfC3Git6xmT5ksy7AoFpgMJhhlY0CYqhmqmtJLIkkSDJLhlCsVhRABPYI1/+9Vvf6CqJqnVvgLuJcaga9m2KTwU3MLsMz4TCRaErnSlage/M4gCnN19bJVh7aAXq6qcGOQDLCarPo+j3X1soW6F0SSJH87oAD1NKW+8KcJDV7Nq/f+aZoGn/AFzPULeI+T7pvhr/XwaT7Y3DYossItwIgaiMGOqJU6Objowe2knraaQ5IGyYAYcwxMsskf2E7qZYuk4NKQ8Aegzp5KVVPh1bx6758qVLhVt36ePl/wpPDXWoJvpNOEhkp3JYQQ+hZgVqM11M0iCgsMdkQhUFdoq/51dagAEBAqjLIRJoMJJtkEg2KotcAgkkJbMClyVfg7hZNDn/5/TYVH88Dj8/jnAKoe/feqWnX4O3m1b1OHqqOZEAJ2rx1e1etf6WpN5hZgRI3EYEfUQqq3OKortJkNZhhl/9BodWg7ObAxtDUfWZIhKzKMMNZ7zsnhrzrg+VRfQPjzql64NJce/qqDel0LPrji9wSH1wGXz4VYYywy4jIQZ4rjn3miJmCwI2pmLp8LDq8DFsWCrPisWvPa+MM9PDQ0/NXV++dTff6h35rDv1XP1RSN5V6qvz+sBivaxbdDgjmBQ91EZ4DBjqiZVK/kM8kmZMRmINGSCKNSfyig8Ffds3q6Fb91LfiItnIv/P4gah4MdkRBVr2ST5ZkruSjWqK93ItP88HmsUGGjNSYVCRbkvn9QRREDHZEQaIJDTaPDZqmIdGSiGRrMmKMMaFuFoWhppZ7qR4GrlnuRQhRa8UvgFqBr2YPYHPQvz+EhkQzvz+ImguDHdEZ4ko+CoXGlHupq/fPp/ngUT21yr2oQtUX+ARjxa8QAg6vAx7VgziTfwswfn8QNR8GO6ImOnnPSq7ko9aoutxLY1b8VvfuNXXFb/WvXT4XnD4nrIaqLcDM8fz+IGpmDHZETVBdukT/gcU9KymMBXvFb3X4MytmZMZxCzCilsTvNKJGcPvcsHvt+g+sBHMCV/JRVGjKil+DbIBJMbVgK4mIwY6oAbyqFzaPDQbZwJWuRPVoyIpfImpe/O4jOgVVU2Hz2AAAKdYUJFmSuGclERG1Wgx2RHXQhAa7xw5VqIg3xSPFmoIYYwxX8hERUavGYEdUQ3VpBrfqRpzRX5ohzhTHQEdERGGBwY6oSs2VrtkJ2SzNQEREYYfBjqJe9UpXi2JhaQYiIgpr/OlFUat6patRNiI9Nh2JlkSWZiAiorDGYEdRp+Ym5G1i2iDJkgSLwRLqZhEREZ0xBjuKGtyEnIiIIh2DHUU8IQTsXjs8qgfxpnhuQk5ERBGLwY4imsPrgMvnQqwxFhlxGYgzxXGlKxERRSwGuxBThQoF3Dw+2Fw+FxxeB6wGK9rFt0OCOQGKzM+ZiIgiW8i7Lp577jl07NgRFosFw4YNw7ffflvvuUuWLMHIkSORnJyM5ORk5Obm1jpfCIFZs2YhMzMTVqsVubm52L17d3PfRqMpkgKr0Qq3z43jzuModZai0l0Jl88FTWihbl7Y8qgelDpLoWoqMmIz0CGxA5KtyQx1REQUFUIa7FasWIHp06dj9uzZ2Lp1K/r164dRo0bh6NGjdZ6/fv163HDDDfj888+xceNGZGdn47LLLsOhQ4f0c5588kk8++yzeOGFF7Bp0ybExsZi1KhRcLlcLXVbDWJUjOiQ2AEdkzqiY1JHZMZnIsYYA03TUOmuxHHncZS5yuDwOuBRPRBChLrJrZpP86HMWQa3z43UmFR0SOyA1NhUGBVjqJtGRETUYiQRwsQwbNgwDBkyBIsWLQIAaJqG7Oxs3HHHHZgxY8ZpX6+qKpKTk7Fo0SJMnDgRQghkZWXhnnvuwb333gsAKC8vR3p6OpYtW4brr7++Qe2qqKhAYmIiysvLkZCQ0PQbbAJVU+FRPfBqXv9woscf7HzCB01oMMgGGGUjjIqRRXTh/7xsHhuEEEiyJCHZmgyr0RrqZhEREQVNY3JJyJKBx+PBli1bMHPmTP2YLMvIzc3Fxo0bG3QNh8MBr9eLlJQUAMDevXtRWFiI3Nxc/ZzExEQMGzYMGzdurDfYud1uuN1u/XFFRUVTbikoFFmBVbbCCisSzAkQMQI+zQeP6oFH9ehzxxweB1ShAvD3/lWHvWhZGKAJDXaPHT7NhwRzAlKsKYgxxnClKxERRbWQBbvi4mKoqor09PSA4+np6fj5558bdI0HHngAWVlZepArLCzUr3HyNaufq8u8efPwyCOPNKb5LUaSJH9wU4yIRSwAf6jxql54VA/cPjccPgc8Pg+cXicEBCRJgkkxwSj7e/UiKewIIeD0OeHyuRBnjENmfCZXuhIREVUJ27G8+fPn480338T69ethsZzZrgEzZ87E9OnT9ccVFRXIzs4+0yY2G1mSYTaYYTaYEW+OB+CfY1Yd9qp79dw+N2yaDQBgkA0wyAaYFFPYLiRwep1wep2wGq1on9Ae8ab4sL0XIiKi5hCyYJeamgpFUVBUVBRwvKioCBkZGad87VNPPYX58+fj008/Rd++ffXj1a8rKipCZmZmwDX79+9f7/XMZjPMZnMT7qL1qA5uVqMViUiEEAJezauHvep6bnaPHT7hgwR/r1512GvNvXpunxt2rx0WxYLM+EwkWhI5v5CIiKgOIRu/MplMGDRoENatW6cf0zQN69atw/Dhw+t93ZNPPom5c+di7dq1GDx4cMBznTp1QkZGRsA1KyoqsGnTplNeMxJVD8fGmmKRbE1Gu4R26JTcCR2TO6JDYgekx6bDpJjg03wod5XjuPM4yl3lcHqd8KreUDcfAOBVvSh1lsKjepAWm4YOSR3QJqYNQx0REVE9QvoTcvr06Zg0aRIGDx6MoUOHYuHChbDb7ZgyZQoAYOLEiWjXrh3mzZsHAFiwYAFmzZqFN954Ax07dtTnzcXFxSEuLg6SJOHuu+/GY489hm7duqFTp054+OGHkZWVhfHjx4fqNlsNWZJhMVj8G96bgVSkBi7M8Lrg8Pl79io9lZAg+VfhVi3OaKlhT1VT9fdPsaYg2ZrsbzMRERGdUkiD3YQJE3Ds2DHMmjULhYWF6N+/P9auXasvfigoKIAsn+hUXLx4MTweD37zm98EXGf27NmYM2cOAOD++++H3W7HLbfcgrKyMpx33nlYu3btGc/Di1TVQ7gxxhjAAn0ItzrsOTwOuFU3bB4bVKFClmR9Ba5RNgZ1CLd6pasqVCSYE5BsSUasKTZo1yciIop0Ia1j11qFso5da6Rqqh723D437B67/lhAnHFtPSGEXog5zhSHFGsK4kxxrXreHxERUUsJizp2FD4UWYEiKyeGcGNODOF6NS+cXiccXgecXid8mg8A9JBnUkynLEXi9Drh9DkRY4xB+9j2iDfHs3QJERFREzHYUaPVrK0HAEmWpIDaetWrcN0+Nyq8FXXW1nOrbji8DlgUC9rFt0O8OZ6LIoiIiM4Qf5JSUNSsrQcAbdBG3x7No3r8Qc7jD3uVWiVMsgkZsRlItCRyP1ciIqIgYbCjZqNvj1a1d6uIOVFbzyAb9BBIREREwcFgRy2mejjWpJhC3RQiIqKIxFnqRERERBGCwY6IiIgoQjDYEREREUUIBjsiIiKiCMFgR0RERBQhGOyIiIiIIgSDHREREVGEYLAjIiIiihAMdkREREQRgsGOiIiIKEIw2BERERFFCAY7IiIiogjBYEdEREQUIRjsiIiIiCIEgx0RERFRhDCEugGtkRACAFBRURHilhAREVG0q84j1fnkVBjs6lBZWQkAyM7ODnFLiIiIiPwqKyuRmJh4ynMk0ZD4F2U0TcPhw4cRHx8PSZKa5T0qKiqQnZ2NAwcOICEhoVneozWK1vsGovfeo/W+gei992i9byB67z1a7xtomXsXQqCyshJZWVmQ5VPPomOPXR1kWUb79u1b5L0SEhKi7psAiN77BqL33qP1voHovfdovW8geu89Wu8baP57P11PXTUuniAiIiKKEAx2RERERBGCwS5EzGYzZs+eDbPZHOqmtKhovW8geu89Wu8biN57j9b7BqL33qP1voHWd+9cPEFEREQUIdhjR0RERBQhGOyIiIiIIgSDHREREVGEYLAjIiIiihAMdi3siy++wNixY5GVlQVJkrBq1apQN6lFzJs3D0OGDEF8fDzS0tIwfvx47Nq1K9TNahGLFy9G37599eKVw4cPx4cffhjqZrW4+fPnQ5Ik3H333aFuSrObM2cOJEkK+OrRo0eom9UiDh06hN///vdo06YNrFYr+vTpg++++y7UzWp2HTt2rPV7LkkSpk2bFuqmNStVVfHwww+jU6dOsFqt6NKlC+bOndugPU3DXWVlJe6++27k5OTAarXi3HPPxebNm0PdLO480dLsdjv69euHqVOn4pprrgl1c1rMhg0bMG3aNAwZMgQ+nw8PPvggLrvsMuzcuROxsbGhbl6zat++PebPn49u3bpBCIFXXnkF48aNw7Zt29CrV69QN69FbN68Gf/85z/Rt2/fUDelxfTq1Quffvqp/thgiPy/bktLSzFixAhcdNFF+PDDD9G2bVvs3r0bycnJoW5as9u8eTNUVdUf//DDD7j00kvx29/+NoStan4LFizA4sWL8corr6BXr1747rvvMGXKFCQmJuLOO+8MdfOa1c0334wffvgBr732GrKysvD6668jNzcXO3fuRLt27ULXMEEhA0CsXLky1M0IiaNHjwoAYsOGDaFuSkgkJyeLf/3rX6FuRouorKwU3bp1E5988om44IILxF133RXqJjW72bNni379+oW6GS3ugQceEOedd16om9Eq3HXXXaJLly5C07RQN6VZjRkzRkydOjXg2DXXXCNuvPHGELWoZTgcDqEoinj//fcDjg8cOFA89NBDIWqVH4diKSTKy8sBACkpKSFuSctSVRVvvvkm7HY7hg8fHurmtIhp06ZhzJgxyM3NDXVTWtTu3buRlZWFzp0748Ybb0RBQUGom9Ts3n33XQwePBi//e1vkZaWhgEDBmDJkiWhblaL83g8eP311zF16lRIkhTq5jSrc889F+vWrcMvv/wCANi+fTu++uorXH755SFuWfPy+XxQVRUWiyXguNVqxVdffRWiVvlF/tgAtTqapuHuu+/GiBEj0Lt371A3p0V8//33GD58OFwuF+Li4rBy5UqcffbZoW5Ws3vzzTexdevWVjHvpCUNGzYMy5YtQ/fu3XHkyBE88sgjGDlyJH744QfEx8eHunnN5tdff8XixYsxffp0PPjgg9i8eTPuvPNOmEwmTJo0KdTNazGrVq1CWVkZJk+eHOqmNLsZM2agoqICPXr0gKIoUFUVjz/+OG688cZQN61ZxcfHY/jw4Zg7dy569uyJ9PR0LF++HBs3bkTXrl1D27iQ9hdGOUTpUOytt94qcnJyxIEDB0LdlBbjdrvF7t27xXfffSdmzJghUlNTxY8//hjqZjWrgoICkZaWJrZv364fi5ah2JOVlpaKhISEiB9+NxqNYvjw4QHH7rjjDnHOOeeEqEWhcdlll4krr7wy1M1oEcuXLxft27cXy5cvFzt27BCvvvqqSElJEcuWLQt105pdfn6+OP/88wUAoSiKGDJkiLjxxhtFjx49Qtou9thRi7r99tvx/vvv44svvkD79u1D3ZwWYzKZ9H/FDRo0CJs3b8bf//53/POf/wxxy5rPli1bcPToUQwcOFA/pqoqvvjiCyxatAhutxuKooSwhS0nKSkJZ511FvLz80PdlGaVmZlZqye6Z8+e+O9//xuiFrW8/fv349NPP8U777wT6qa0iPvuuw8zZszA9ddfDwDo06cP9u/fj3nz5kV8L22XLl2wYcMG2O12VFRUIDMzExMmTEDnzp1D2i7OsaMWIYTA7bffjpUrV+Kzzz5Dp06dQt2kkNI0DW63O9TNaFaXXHIJvv/+e+Tl5elfgwcPxo033oi8vLyoCXUAYLPZsGfPHmRmZoa6Kc1qxIgRtcoY/fLLL8jJyQlRi1re0qVLkZaWhjFjxoS6KS3C4XBAlgOjhKIo0DQtRC1qebGxscjMzERpaSk++ugjjBs3LqTtYY9dC7PZbAH/at+7dy/y8vKQkpKCDh06hLBlzWvatGl44403sHr1asTHx6OwsBAAkJiYCKvVGuLWNa+ZM2fi8ssvR4cOHVBZWYk33ngD69evx0cffRTqpjWr+Pj4WnMoY2Nj0aZNm4ifW3nvvfdi7NixyMnJweHDhzF79mwoioIbbrgh1E1rVn/+859x7rnn4oknnsB1112Hb7/9Fi+++CJefPHFUDetRWiahqVLl2LSpElRUd4GAMaOHYvHH38cHTp0QK9evbBt2zY888wzmDp1aqib1uw++ugjCCHQvXt35Ofn47777kOPHj0wZcqU0DYspAPBUejzzz8XAGp9TZo0KdRNa1Z13TMAsXTp0lA3rdlNnTpV5OTkCJPJJNq2bSsuueQS8fHHH4e6WSERLXPsJkyYIDIzM4XJZBLt2rUTEyZMEPn5+aFuVot47733RO/evYXZbBY9evQQL774Yqib1GI++ugjAUDs2rUr1E1pMRUVFeKuu+4SHTp0EBaLRXTu3Fk89NBDwu12h7ppzW7FihWic+fOwmQyiYyMDDFt2jRRVlYW6mYJSYgoKA9NREREFAU4x46IiIgoQjDYEREREUUIBjsiIiKiCMFgR0RERBQhGOyIiIiIIgSDHREREVGEYLAjIiIiihAMdkREREQRgsGOiFqNffv2QZIk5OXlhbopup9//hnnnHMOLBYL+vfv3+zv17FjRyxcuLDB5zfkM1u2bBmSkpLOuG3BUlJSgrS0NOzbt6/Br9m5cyfat28Pu93efA0jigAMdkSkmzx5MiRJwvz58wOOr1q1CpIkhahVoTV79mzExsZi165dWLduXZ3nBPNz27x5M2655ZYmtzccPP744xg3bhw6duwIoO5wWllZiYsuughnn302Dh48iLPPPhvnnHMOnnnmmdA0mihMMNgRUQCLxYIFCxagtLQ01E0JGo/H0+TX7tmzB+eddx5ycnLQpk2bes8L1ufWtm1bxMTEnNE1WorX6230axwOB1566SXcdNNN9Z5z7NgxXHTRRbDb7fjyyy/Rvn17AMCUKVOwePFi+Hy+JreZKNIx2BFRgNzcXGRkZGDevHn1njNnzpxaw5ILFy7Ue2AAfy/W+PHj8cQTTyA9PR1JSUl49NFH4fP5cN999yElJQXt27fH0qVLa13/559/xrnnnguLxYLevXtjw4YNAc//8MMPuPzyyxEXF4f09HT84Q9/QHFxsf78hRdeiNtvvx133303UlNTMWrUqDrvQ9M0PProo2jfvj3MZjP69++PtWvX6s9LkoQtW7bg0UcfhSRJmDNnzhl9bgDw1VdfYeTIkbBarcjOzsadd94ZMLx48lDszz//jPPOOw8WiwVnn302Pv30U0iShFWrVgVc99dff8VFF12EmJgY9OvXDxs3bqz13qtWrUK3bt1gsVgwatQoHDhwIOD5xYsXo0uXLjCZTOjevTtee+21gOclScLixYtx1VVXITY2Fo8//jhKS0tx4403om3btrBarejWrVudv6fV1qxZA7PZjHPOOafO5w8cOICRI0ciMTERn332WUCYvvTSS3H8+PFafx6I6AQGOyIKoCgKnnjiCfzjH//AwYMHz+han332GQ4fPowvvvgCzzzzDGbPno0rr7wSycnJ2LRpE2699Vb83//9X633ue+++3DPPfdg27ZtGD58OMaOHYuSkhIAQFlZGS6++GIMGDAA3333HdauXYuioiJcd911Add45ZVXYDKZ8PXXX+OFF16os31///vf8fTTT+Opp57Cjh07MGrUKFx11VXYvXs3AODIkSPo1asX7rnnHhw5cgT33ntvvffakM9tz549GD16NK699lrs2LEDK1aswFdffYXbb7+9zvNVVcX48eMRExODTZs24cUXX8RDDz1U57kPPfQQ7r33XuTl5eGss87CDTfcENCz5XA48Pjjj+PVV1/F119/jbKyMlx//fX68ytXrsRdd92Fe+65Bz/88AP+7//+D1OmTMHnn38e8D5z5szB1Vdfje+//x5Tp07Fww8/jJ07d+LDDz/ETz/9hMWLFyM1NbXez+nLL7/EoEGD6nxu165dGDFiBM4++2ysWbMGcXFxAc+bTCb0798fX375Zb3XJ4p6goioyqRJk8S4ceOEEEKcc845YurUqUIIIVauXClq/nUxe/Zs0a9fv4DX/u1vfxM5OTkB18rJyRGqqurHunfvLkaOHKk/9vl8IjY2VixfvlwIIcTevXsFADF//nz9HK/XK9q3by8WLFgghBBi7ty54rLLLgt47wMHDggAYteuXUIIIS644AIxYMCA095vVlaWePzxxwOODRkyRPzpT3/SH/fr10/Mnj37lNdp6Od20003iVtuuSXgtV9++aWQZVk4nU4hhBA5OTnib3/7mxBCiA8//FAYDAZx5MgR/fxPPvlEABArV64UQpz4zP71r3/p5/z4448CgPjpp5+EEEIsXbpUABDffPONfs5PP/0kAIhNmzYJIYQ499xzxR//+MeAtv32t78VV1xxhf4YgLj77rsDzhk7dqyYMmXKKT+fmsaNG6d/PtWq78FkMomLLrpI+Hy+el9/9dVXi8mTJzf4/YiiDXvsiKhOCxYswCuvvIKffvqpydfo1asXZPnEXzPp6eno06eP/lhRFLRp0wZHjx4NeN3w4cP1XxsMBgwePFhvx/bt2/H5558jLi5O/+rRowcAf49Ytfp6hapVVFTg8OHDGDFiRMDxESNGnNE9n+pz2759O5YtWxbQ9lGjRkHTNOzdu7fW+bt27UJ2djYyMjL0Y0OHDq3zffv27av/OjMzEwACPleDwYAhQ4boj3v06IGkpCS9nT/99FODPovBgwcHPL7tttvw5ptvon///rj//vvxv//9r872VXM6nbBYLHU+d9VVV+HLL7/EO++8U+/rrVYrHA7HKd+DKJoZQt0AImqdzj//fIwaNQozZ87E5MmTA56TZRlCiIBjdU2kNxqNAY8lSarzmKZpDW6XzWbD2LFjsWDBglrPVQcaAIiNjW3wNYPpVJ+bzWbD//3f/+HOO++s9boOHTqc0fvW/FyrV+I25nNtqJM/18svvxz79+/HmjVr8Mknn+CSSy7BtGnT8NRTT9X5+tTU1HoXmDz00EPo27cvfve730EIUWt4HQCOHz+OLl26nPmNEEUo9tgRUb3mz5+P9957r9ZE/LZt26KwsDAg3AWz9tw333yj/9rn82HLli3o2bMnAGDgwIH48ccf0bFjR3Tt2jXgqzFhLiEhAVlZWfj6668Djn/99dc4++yzz6j99X1uAwcOxM6dO2u1u2vXrjCZTLWu0717dxw4cABFRUX6sc2bNzepTT6fD999953+eNeuXSgrK9M/1549ezb5s2jbti0mTZqE119/HQsXLsSLL75Y77kDBgzAzp07633+4Ycfxpw5c3DjjTdixYoVtZ7/4YcfMGDAgNO2iShaMdgRUb369OmDG2+8Ec8++2zA8QsvvBDHjh3Dk08+iT179uC5557Dhx9+GLT3fe6557By5Ur8/PPPmDZtGkpLSzF16lQAwLRp03D8+HHccMMN2Lx5M/bs2YOPPvoIU6ZMgaqqjXqf++67DwsWLMCKFSuwa9cuzJgxA3l5ebjrrrvOqP31fW4PPPAA/ve//+H2229HXl4edu/ejdWrV9e7eOLSSy9Fly5dMGnSJOzYsQNff/01/vKXvwBAo+vjGY1G3HHHHdi0aRO2bNmCyZMn45xzztGHdu+77z4sW7YMixcvxu7du/HMM8/gnXfeOeWCEQCYNWsWVq9ejfz8fPz44494//339bBYl1GjRuHHH388ZVmYhx56CHPnzsWNN96I5cuX68f37duHQ4cOITc3t1H3ThRNGOyI6JQeffTRWkN6PXv2xPPPP4/nnnsO/fr1w7fffnvaANAY8+fPx/z589GvXz989dVXePfdd/WVltW9bKqq4rLLLkOfPn1w9913IykpKWA+X0PceeedmD59Ou655x706dMHa9euxbvvvotu3bqd8T3U9bn17dsXGzZswC+//IKRI0diwIABmDVrFrKysuq8hqIoWLVqFWw2G4YMGYKbb75ZXxVb3zy1+sTExOCBBx7A7373O4wYMQJxcXEBPWLjx4/H3//+dzz11FPo1asX/vnPf2Lp0qW48MILT3ldk8mEmTNnom/fvjj//POhKArefPPNes/v06cPBg4ciLfeeuuU150xYwaeeOIJ/OEPf8Abb7wBAFi+fDkuu+wy5OTkNPzGiaKMJE6eKENERK3W119/jfPOOw/5+flhO9fsgw8+wH333YcffvihwWHc4/GgW7dueOONN2ot8iCiE7h4goioFVu5ciXi4uLQrVs35Ofn46677sKIESPCNtQBwJgxY7B7924cOnQI2dnZDXpNQUEBHnzwQYY6otNgjx0RUSv26quv4rHHHkNBQQFSU1ORm5uLp59++pTbmxFR9GKwIyIiIooQXDxBREREFCEY7IiIiIgiBIMdERERUYRgsCMiIiKKEAx2RERERBGCwY6IiIgoQjDYEREREUUIBjsiIiKiCPH/+0JKEFQi8FQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,Ks),mean_acc,'g')\n",
    "plt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\n",
    "plt.fill_between(range(1,Ks),mean_acc - 3 * std_acc,mean_acc + 3 * std_acc, alpha=0.10,color=\"green\")\n",
    "plt.legend(('Accuracy ', '+/- 1xstd','+/- 3xstd'))\n",
    "plt.ylabel('Accuracy ')\n",
    "plt.xlabel('Number of Neighbors (K)')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best accuracy was with 0.34 with k= 9\n"
     ]
    }
   ],
   "source": [
    "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Congratulations on completing this lab!\n",
    "\n",
    "## Author\n",
    "\n",
    "Saeed Aghabozorgi\n",
    "\n",
    "### Other Contributors\n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2022-01-01\" target=\"_blank\">Joseph Santarcangelo</a>\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation. All rights reserved. <h3/>\n",
    "\n",
    "<!--\n",
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                 |\n",
    "| ----------------- | ------- | ---------- | ---------------------------------- |\n",
    "| 2021-01-21        | 2.4     | Lakshmi    | Updated sklearn library            |\n",
    "| 2020-11-20        | 2.3     | Lakshmi    | Removed unused imports             |\n",
    "| 2020-11-17        | 2.2     | Lakshmi    | Changed plot function of KNN       |\n",
    "| 2020-11-03        | 2.1     | Lakshmi    | Changed URL of csv                 |\n",
    "| 2020-08-27        | 2.0     | Lavanya    | Moved lab to course repo in GitLab |\n",
    "|                   |         |            |                                    |\n",
    "|                   |         |            |                                    |\n",
    "--!>\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "prev_pub_hash": "3dd33d6d5e2262aa8f7b4af66defe86b20edadaefb82d87f7f74aa7b1d46c23c"
 },
 "nbformat": 4,
 "nbformat_minor": 4
}