cnn_trainer_2_0.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/bmeyboom/robot_cnn_trainer/blob/main/cnn_trainer_2_0.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jnHfnr13ltBf"
      },
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dMmNU8A57SrO"
      },
      "source": [
        "# Notes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YdKQNzwWNb8q"
      },
      "source": [
        "General Notes:\n",
        "- check that data with no driving labels is not included in training\n",
        "- figure out how many of each type of command we have (turn right, left, straight), and make sure we use equal amounts of each\n",
        "- need ~2000 images!! Miti said train to just complete outer loop, and then separately inner loop\n",
        "- Check images and associated commands to make sure the associated commands make sense, potential for lag between command and image recordings\n",
        "  - to do this assess ~20 of each command/image, RANDOMLY (important)\n",
        "\n",
        "\n",
        "  Notes Monday, March 29th:\n",
        "  - I think we have our left and right turns mixed up\n",
        "  - disproportional amounts of left vs right vs straight turns\n",
        "  - images associated with turns maybe kinda wrong? don't know...\n",
        "  - need more data!\n",
        "  \n",
        "  Wednesday notes:\n",
        "  - had to modify a bunch of things to avoid exceeding RAM limits\n",
        "  - getting 20% training accuracy... need to fix. maybe has to do with the activation functions?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eOB3JWrp7A69"
      },
      "source": [
        "# Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xatsBWTUq5Sp"
      },
      "source": [
        "#download from gdrive or take from folder?\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6n0Fm_nV3SO1"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mGZYb9Svpu1J"
      },
      "source": [
        "img_dir = \"/content/drive/MyDrive/test_data_enph353/img\"\n",
        "csv_file = \"/content/drive/MyDrive/test_data_enph353/data.csv\"\n",
        "pickle_f = \"/content/drive/MyDrive/test_data_enph353/pickled_data.pickle\"\n",
        "pickle_f2 = \"/content/drive/MyDrive/test_data_enph353/img3.pickle\"\n",
        "checkpoint_filepath = \"/content/drive/MyDrive/test_data_enph353/\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I-6p-T0ZRnW2"
      },
      "source": [
        "import csv\n",
        "import math\n",
        "import numpy as np\n",
        "import re\n",
        "import cv2\n",
        "\n",
        "from collections import Counter\n",
        "from matplotlib import pyplot as plt\n",
        "from PIL import Image\n",
        "\n",
        "from google.colab.patches import cv2_imshow # for image display\n",
        "\n",
        "#Data\n",
        "import pickle as pkl\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# CNN\n",
        "%tensorflow_version 1.14.0\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten\n",
        "from tensorflow.keras.layers import Conv1D, Conv2D, MaxPooling2D, MaxPooling1D\n",
        "from tensorflow.keras import models\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras import optimizers\n",
        "\n",
        "from tensorflow.keras.utils import plot_model\n",
        "from tensorflow.keras import backend\n",
        "\n",
        "# Evaluating network\n",
        "from sklearn.metrics import ConfusionMatrixDisplay, multilabel_confusion_matrix, plot_confusion_matrix\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jj3nGfonRB7m"
      },
      "source": [
        "## For one hot encoding y data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MPHsEmkXnLYw"
      },
      "source": [
        "def one_hot_encoding(data, n = 3):\n",
        "# 0 for no turn\n",
        "# 1 for right turn (+ angular)\n",
        "# 2 for left turn (- angular)\n",
        "  data_float = float(data)\n",
        "  y = [0 if (data_float == 0.0) else 1 if (data_float > 0) else 2] # > 0 is turning right, = 0 is straight, < 0 is turning left\n",
        "  encoding = np.zeros(n)\n",
        "  encoding[y] = 1\n",
        "  return encoding"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S2p8vo7zRICC"
      },
      "source": [
        "##Loading raw data from CSV"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ARmoGNARxeyb"
      },
      "source": [
        "# TRYING NEW DATA PARSING\n",
        "#start with angular first\n",
        "\n",
        "# getting rid images without driving labels?\n",
        "# display images and associated labels\n",
        "# in batches (random <-)\n",
        "\n",
        "data_list = []\n",
        "\n",
        "def load_csv(csv_file, data_list):\n",
        "  with open(csv_file, 'r', newline = '') as f:\n",
        "    reader = csv.reader(f, delimiter = ',')\n",
        "    # r[0] = linear, r[1] = angular, r[2] = image\n",
        "    for row in reader:\n",
        "      if 'png' not in row[0] and not row[0] == row[1]:\n",
        "        if not row[0] == \"\" or not row [1] == \"\": # MAKE SURE NO EMPTY COMMANDS!\n",
        "          data_list.append(row)\n",
        "\n",
        "  tr = [] # turn right\n",
        "  tl = [] # turn left\n",
        "  nt = [] # go straight (no turn)\n",
        "  for d in data_list:\n",
        "    fl = float(d[1])\n",
        "    if fl > 0:\n",
        "      tr.append(d)\n",
        "    elif fl < 0:\n",
        "      tl.append(d)\n",
        "    else:\n",
        "      nt.append(d)\n",
        "\n",
        "  print(len(tr))\n",
        "  print(len(tl))\n",
        "  print(len(nt))\n",
        "  # not an equal amount of each command!!\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RqI3EpWiP9vK"
      },
      "source": [
        "# Parsing and Pickling methods"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2yBsW3Rj7aly"
      },
      "source": [
        "###Load pickle method"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I6KUNmhRnIsa"
      },
      "source": [
        "def load_pkl(obj, pickle_f, python_version_int):\n",
        "  with open(pickle_f,\"rb\") as fileOpener:\n",
        "    while True:\n",
        "        try:\n",
        "            if python_version_int == 2: # for python 2 pickling\n",
        "              obj.append(pkl.load(fileOpener, encoding='latin1'))\n",
        "            else:\n",
        "              obj.append(pkl.load(fileOpener))\n",
        "        except EOFError:\n",
        "            print('file ended')\n",
        "            break"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "piAUVvBb7dBL"
      },
      "source": [
        "###Pickle method"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QR_x4-nxwNzw"
      },
      "source": [
        "def pickle(data_list, pickle_f):\n",
        "  # creating new pickle file with grayscale images as matrices\n",
        "  outfile = open(pickle_f, 'wb')\n",
        "\n",
        "  for i in range(len(data_list)):\n",
        "  # get file path\n",
        "    path = data_list[i][2]\n",
        "    # parse path to get image number\n",
        "    if 'img/' in path:\n",
        "      start_index = path.index('img/') + 3\n",
        "    else:\n",
        "      continue\n",
        "    number = path[start_index:]\n",
        "    path = img_dir + '/' + number\n",
        "    # # # load image\n",
        "    try:\n",
        "      im = cv2.imread(path)\n",
        "      gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)\n",
        "      sml = cv2.resize(gray, (int(gray.shape[1]/6), int(gray.shape[0]/6)))\n",
        "      data = [data_list[i][0], one_hot_encoding(data_list[i][1]), np.array(sml)]\n",
        "      pkl.dump(data, outfile)\n",
        "    except cv2.error as e:\n",
        "      print('file not found')\n",
        "\n",
        "  outfile.close()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w_AJbtqLSWqw"
      },
      "source": [
        "##Load and format data for learning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e-19ilaJ7oeE"
      },
      "source": [
        "###Extracting pickled data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "if4DhoJNpZqv"
      },
      "source": [
        "# # getting rid images without driving labels?\n",
        "# # display images and associated labels\n",
        "# # in batches (random <-)\n",
        "\n",
        "# takes the unpickled object with [linear speed, one-hot-encoded angular velocity,\n",
        "# grey-scale+reduced size image matrix], x list, y list\n",
        "def parse_pickle_data(pkl_o):\n",
        "  X = []\n",
        "  y = []\n",
        "  print('Shuffling')\n",
        "  np.random.shuffle(pkl_o)\n",
        "  for o in pkl_o:\n",
        "    y.append(o[1])\n",
        "    X.append((o[2]/255))\n",
        "  # normalize and make array-like\n",
        "  print('Converting to arrays')\n",
        "  # X = np.asarray(X)\n",
        "  # y = np.asarray(y)\n",
        "  # new_X = X/255.0\n",
        "  # X = new_X\n",
        "  return X, y\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Uf9etD-WBml"
      },
      "source": [
        "# load pickle file data\n",
        "unpickled_data = []\n",
        "load_pkl(unpickled_data, pickle_f, 3)\n",
        "load_pkl(unpickled_data, pickle_f2, 2)\n",
        "print(len(unpickled_data))\n",
        "print(unpickled_data[0])\n",
        "print(unpickled_data[-1])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pWNLk84S8iry"
      },
      "source": [
        "#Prepping data for network"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SRiM_U4h8v1f"
      },
      "source": [
        "st=[] # straight\n",
        "lt=[] # left turns\n",
        "rt=[] # right turns\n",
        "for o in unpickled_data:\n",
        "  if o[1][0] == 1:\n",
        "    st.append(o)\n",
        "  elif o[1][1] == 1:\n",
        "    lt.append(o)\n",
        "  elif o[1][2] == 1:\n",
        "    rt.append(o)\n",
        "  else:\n",
        "    print('problem...')\n",
        "print(len(st))\n",
        "print(len(lt))\n",
        "print(len(rt))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iOm1wV9K83jT"
      },
      "source": [
        "def train_test_split(set, test_size):\n",
        "  train_size = int(len(set)*(1-test_size))\n",
        "  np.random.shuffle(set)\n",
        "  train_set = set[0:train_size]\n",
        "  test_set = set[train_size:]\n",
        "  print(len(test_set)/len(train_set))\n",
        "  return train_set, test_set\n",
        "\n",
        "# even out the amounts of data in each\n",
        "np.random.shuffle(st)\n",
        "np.random.shuffle(lt)\n",
        "np.random.shuffle(rt)\n",
        "\n",
        "test_size = 0.1\n",
        "st_train_set, st_test_set = train_test_split(st, test_size)\n",
        "lt_train_set, lt_test_set = train_test_split(lt, test_size)\n",
        "rt_train_set, rt_test_set = train_test_split(rt, test_size+0.01)\n",
        "print(len(rt_test_set))\n",
        "xlt_train, ylt_train = parse_pickle_data(lt_train_set)\n",
        "xlt_test, ylt_test = parse_pickle_data(lt_test_set)\n",
        "xrt_train, yrt_train = parse_pickle_data(rt_train_set)\n",
        "xrt_test, yrt_test = parse_pickle_data(rt_test_set)\n",
        "xst_train, yst_train = parse_pickle_data(st_train_set)\n",
        "xst_test, yst_test = parse_pickle_data(st_test_set)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IpdsNMhXn-Bf"
      },
      "source": [
        "print(len(xrt_test))\n",
        "print(len(xrt_train))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-na5Y06p_Iiu"
      },
      "source": [
        "# Create test data with good amounts of each turn\n",
        "st_vs_lt  =1.5\n",
        "X_test = xst_test[0:int(len(xlt_test)*st_vs_lt)]\n",
        "y_test = yst_test[0:int(len(xlt_test)*st_vs_lt)]\n",
        "\n",
        "X_test.extend(xlt_test)\n",
        "y_test.extend(ylt_test)\n",
        "\n",
        "X_test.extend(xrt_test)\n",
        "y_test.extend(yrt_test)\n",
        "\n",
        "X_test = np.asarray(X_test)\n",
        "y_test = np.asarray(y_test)\n",
        "\n",
        "# Create training set with good amounts of each turn\n",
        "X_train= xst_train[0:int(len(xlt_train)*st_vs_lt)]\n",
        "y_train = yst_train[0:int(len(xlt_train)*st_vs_lt)]\n",
        "\n",
        "X_train.extend(xlt_train)\n",
        "y_train.extend(ylt_train)\n",
        "\n",
        "X_train.extend(xrt_train)\n",
        "y_train.extend(yrt_train)\n",
        "\n",
        "X_train = np.asarray(X_train)\n",
        "y_train = np.asarray(y_train)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JhkCYeESLuC0"
      },
      "source": [
        "print(len(X_train))\n",
        "print(len(y_train))\n",
        "print(len(X_test))\n",
        "print(len(y_test))\n",
        "print(X_test.shape)\n",
        "print(X_train.shape)\n",
        "print(y_test.shape)\n",
        "print(y_train.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3QrijNF-UuqA"
      },
      "source": [
        "#To load raw images from CSV"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u600Wn0gUsV2"
      },
      "source": [
        "# np.random.shuffle(data_list)\n",
        "# # X is the images, y is the associated cmd_vel values\n",
        "# X_og = []\n",
        "# y_og = []\n",
        "\n",
        "def parse_raw_image_data(data_list, X_og, y_og):\n",
        "  for i in range(len(data_list)):\n",
        "\n",
        "    #for my sanity\n",
        "    if i == int(len(data_list)/4):\n",
        "      print('quarter way there')\n",
        "    if i == int(len(data_list)/2):\n",
        "      print('halfway there')\n",
        "    if i == int(len(data_list)*3/4):\n",
        "      print('3/4 way there!')\n",
        "\n",
        "    # get file path\n",
        "    path = data_list[i][2]\n",
        "\n",
        "    # parse path to get image number to find in google drive\n",
        "    if 'img/' in path:\n",
        "      start_index = path.index('img/') + 3\n",
        "    elif 'image/' in path:\n",
        "      start_index = path.index('image/') + 5\n",
        "    else:\n",
        "      continue\n",
        "    number = path[start_index:]\n",
        "    path = img_dir + '/' + number\n",
        "    # load image\n",
        "    try:\n",
        "      im = cv2.imread(path)\n",
        "      gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)\n",
        "      cv2_imshow(sml)\n",
        "      # add image to X data as matrix\n",
        "      X_og.append(np.array(gray))\n",
        "      # print(data_list[i][1])\n",
        "      # # get associated turning data\n",
        "      y_og.append(one_hot_encoding(data_list[i][1]))\n",
        "    except cv2.error as e:\n",
        "      print('file not found')\n",
        "      # print(path)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gM3xoTVMveas"
      },
      "source": [
        "# parse_raw_image_data(data_list, X_og, y_og)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hzEJzgpQRwgY"
      },
      "source": [
        "###Visual data check\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cjA_Tj_x2cz2"
      },
      "source": [
        "# # LEFT TURNS\n",
        "\n",
        "for i in range(20):\n",
        "  # get file path\n",
        "  path = tl[i][2]\n",
        "  print(tl[i][2])\n",
        "  if 'img/' in path:\n",
        "    start_index = path.index('img/') + 3\n",
        "  elif 'image/' in path:\n",
        "    start_index = path.index('image/') + 5\n",
        "  else:\n",
        "    continue\n",
        "  number = path[start_index:]\n",
        "\n",
        "  path = img_dir + '/' + number\n",
        "  # cv2 option\n",
        "  im = cv2.imread(path)\n",
        "  im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)\n",
        "  cv2_imshow(im)\n",
        "  sml = cv2.resize(im, (300, 300))\n",
        "  # Displaying the image\n",
        "  cv2_imshow(im)\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "om9L_JXA0R0a"
      },
      "source": [
        "# # to visually check the data\n",
        "\n",
        "# # RIGHT TURNS\n",
        "np.random.shuffle(tr)\n",
        "np.random.shuffle(tl)\n",
        "np.random.shuffle(nt)\n",
        "\n",
        "for i in range(20):\n",
        "  # get file path\n",
        "  path = tr[i][2]\n",
        "  if 'img/' in path:\n",
        "    start_index = path.index('img/') + 3\n",
        "  elif 'image/' in path:\n",
        "    start_index = path.index('image/') + 5\n",
        "  else:\n",
        "    continue\n",
        "  number = path[start_index:]\n",
        "\n",
        "  path = img_dir + '/' + number\n",
        "  # cv2 option\n",
        "  im = cv2.imread(path)\n",
        "  im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)\n",
        "\n",
        "  # Displaying the image\n",
        "  cv2_imshow(im)\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7KMd-NWb2hMb"
      },
      "source": [
        "# # STRAIGHT\n",
        "\n",
        "for i in range(20):\n",
        "  # get file path\n",
        "  path = nt[i][2]\n",
        "  if 'img/' in path:\n",
        "    start_index = path.index('img/') + 3\n",
        "  elif 'image/' in path:\n",
        "    start_index = path.index('image/') + 5\n",
        "  else:\n",
        "    continue\n",
        "  number = path[start_index:]\n",
        "\n",
        "  path = img_dir + '/' + number\n",
        "  # cv2 option\n",
        "  im = cv2.imread(path)\n",
        "  im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)\n",
        "  # Displaying the image\n",
        "  cv2_imshow(im)\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uTtab5XQTjU5"
      },
      "source": [
        "# Network stuff"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RAmb427EpJ9U"
      },
      "source": [
        "# From lab 5 sample code\n",
        "def reset_weights(model):\n",
        "    print('Resetting weights')\n",
        "    session = backend.get_session()\n",
        "    for layer in model.layers:\n",
        "        if hasattr(layer, 'kernel_initializer'):\n",
        "            layer.kernel.initializer.run(session=session)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I_XuI1nYM_UX"
      },
      "source": [
        "print(X_train.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "neLvhm2Ia64O"
      },
      "source": [
        "def make_marikondo_model(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X.shape[1:]))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dense(256, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_yHvn1uWpcB5"
      },
      "source": [
        "def make_model(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X_train.shape[1:]))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  model.add(Conv1D(128, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(254, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  # model.add(Dense(2, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5AaC5H2AAuHo"
      },
      "source": [
        "def make_model_extraDense(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X.shape[1:]))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  model.add(Conv1D(128, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(254, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(256, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "55OT4bwFw7yI"
      },
      "source": [
        "def make_model_extraExtraDense_extraDropout(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X.shape[1:]))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  model.add(Conv1D(128, 3, activation='relu'))\n",
        "  model.add(Dropout(0.1))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(254, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dense(256, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ya7AA6Hn9bty"
      },
      "source": [
        "def make_model_extraExtraDense(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X.shape[1:]))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  model.add(Conv1D(128, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "  model.add(Conv1D(254, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides = 2))\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dense(256, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RNUzsDJJST-4"
      },
      "source": [
        "# Final Model\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iETl8hBgQb-q"
      },
      "source": [
        "def make_model_extraDense_lessConv(model, X):\n",
        "  # For grayscale images we need to use 1D Conv and MaxPooling layers!\n",
        "  print('Making model')\n",
        "  # one dimensional convolution for grayscale image\n",
        "  model.add(Conv1D(32, 3, activation='relu', input_shape=X.shape[1:]))\n",
        "  model.add(MaxPooling1D(pool_size=(2), strides=2))\n",
        "  model.add(Conv1D(64, 3, activation='relu'))\n",
        "  model.add(MaxPooling1D(pool_size=(2)))\n",
        "\n",
        "  # flattens so that Dense can be used\n",
        "  model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
        "  # Trying linear activation (regressive) vs classification (softmax)\n",
        "  model.add(Dense(512, activation='relu'))\n",
        "  model.add(Dropout(0.5))\n",
        "  model.add(Dense(256, activation='relu'))\n",
        "  model.add(Dense(3, activation='softmax'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "muBBUVfkhXes"
      },
      "source": [
        "# Test and Train methods"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ltIJfJK_oIv-"
      },
      "source": [
        "def train(model, X, y, batch, epoch):\n",
        "  print('In train, fitting model.')\n",
        "  history_conv = model.fit(X, y, batch_size=batch, epochs=epoch, validation_data=(np.asarray(X_test), y_test))\n",
        "  print('Plotting loss')\n",
        "  plt.plot(history_conv.history['loss'])\n",
        "  plt.plot(history_conv.history['val_loss'])\n",
        "  plt.title('model loss')\n",
        "  plt.ylabel('loss')\n",
        "  plt.xlabel('epoch')\n",
        "  plt.legend(['train loss', 'val loss'], loc='upper left')\n",
        "  plt.show()\n",
        "\n",
        "  print('Plotting Accuracy')\n",
        "  plt.plot(history_conv.history['acc'])\n",
        "  plt.plot(history_conv.history['val_acc'])\n",
        "  plt.title('model accuracy')\n",
        "  plt.ylabel('accuracy (%)')\n",
        "  plt.xlabel('epoch')\n",
        "  plt.legend(['train accuracy', 'val accuracy'], loc='upper left')\n",
        "  plt.show()\n",
        "\n",
        "  return history_conv\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9HM_gagWt-F_"
      },
      "source": [
        "def plot_confusion_matrix(cm, classes,\n",
        "                        normalize=False,\n",
        "                        title='Confusion matrix',\n",
        "                        cmap=plt.cm.Blues):\n",
        "    \"\"\"\n",
        "    This function prints and plots the confusion matrix.\n",
        "    Normalization can be applied by setting `normalize=True`.\n",
        "    \"\"\"\n",
        "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
        "    plt.title(title)\n",
        "    plt.colorbar()\n",
        "    tick_marks = np.arange(len(classes))\n",
        "    plt.xticks(tick_marks, classes, rotation=45)\n",
        "    plt.yticks(tick_marks, classes)\n",
        "\n",
        "    if normalize:\n",
        "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
        "        print(\"Normalized confusion matrix\")\n",
        "    else:\n",
        "        print('Confusion matrix, without normalization')\n",
        "\n",
        "    print(cm)\n",
        "\n",
        "    thresh = cm.max() / 2.\n",
        "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
        "        plt.text(j, i, cm[i, j],\n",
        "            horizontalalignment=\"center\",\n",
        "            color=\"white\" if cm[i, j] > thresh else \"black\")\n",
        "\n",
        "    plt.tight_layout()\n",
        "    plt.ylabel('True label')\n",
        "    plt.xlabel('Predicted label')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lvK2LXzcwF3C"
      },
      "source": [
        "import sklearn.metrics as skm\n",
        "from sklearn.metrics import confusion_matrix\n",
        "import itertools\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "def model_summary(m, X_test, y_true):\n",
        "  model.summary()\n",
        "  print('Generating predictions')\n",
        "  y_true, y_pred = y_test, model.predict(np.asarray(X_test))\n",
        "  print('Creating confusion matrix')\n",
        "  cm=confusion_matrix(np.argmax(y_test, axis=1), np.argmax(y_pred, axis=1))\n",
        "  print(cm)\n",
        "  cm_plot_labels = ['Straight', 'Left Turn', 'Right Turn']\n",
        "  plot_confusion_matrix(cm, cm_plot_labels)\n",
        "  print(skm.classification_report(np.argmax(y_test, axis=1), np.argmax(y_pred, axis=1)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bIJ2ur2w7J5p"
      },
      "source": [
        "#Testing various things"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "udOSM8wzKMpd"
      },
      "source": [
        "# Round 1"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YscWoIleoRSk"
      },
      "source": [
        "batch = 5\n",
        "epoch = 15\n",
        "i=3\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, int(batch*i), int(epoch*i))\n",
        "\n",
        "# m = {}\n",
        "# for i in range(7):\n",
        "#   model = Sequential()\n",
        "#   make_model(model, X_train)\n",
        "#   reset_weights(model)\n",
        "#   # compile\n",
        "#   LEARNING_RATE = 1e-5\n",
        "#   model.compile(loss='categorical_crossentropy',\n",
        "#                 optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "#               metrics=['accuracy'])\n",
        "\n",
        "#   history_conv = train(model, X_train, y_train, int(batch*i), int(epoch*i))\n",
        "#   model_summary(model)\n",
        "#   m['model{0}'.format(i)] = model"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7yKGhykfCg6m"
      },
      "source": [
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YcztMODBKJrs"
      },
      "source": [
        "# Round 2\n",
        "fewer epochs -- not as effective"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bopckm3woO9U"
      },
      "source": [
        "batch = 5\n",
        "epoch = 15\n",
        "i=2\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, int(batch*i), int(epoch*i))\n",
        "model_summary(model)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s8hBrakkDHNE"
      },
      "source": [
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MefjiNDsKVrl"
      },
      "source": [
        "#Round 3\n",
        "Reduced amount of data corresponding to straight cmd_vel\n",
        "Effect: Better left turn prediction"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OluL8wRCtvfd"
      },
      "source": [
        "even_data = st[0:int(len(lt)*1.5)]\n",
        "print('trimmed')\n",
        "print(len(even_data))\n",
        "even_data.extend(lt)\n",
        "print(len(even_data))\n",
        "\n",
        "even_data.extend(rt[0:])\n",
        "print(len(even_data))\n",
        "\n",
        "np.random.shuffle(even_data)\n",
        "\n",
        "print(len(even_data))\n",
        "\n",
        "X2 = []\n",
        "y2 = []\n",
        "\n",
        "parse_pickle_data(even_data, X2, y2)\n",
        "\n",
        "# to get a visual\n",
        "print(len(X))\n",
        "print(len(y))\n",
        "print(X[50])\n",
        "print(y[50])\n",
        "\n",
        "batch = 5\n",
        "epoch = 15\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u7yq6CeNKjif"
      },
      "source": [
        "#Round 4\n",
        "- Increased learning rate\n",
        "- Effect: Lower val loss"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OYeKLyu0DBIN"
      },
      "source": [
        "# larger learning rate\n",
        "\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "# modified learning rate\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)\n",
        "# performs much better!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ut6BR1h8K2Ft"
      },
      "source": [
        "# Round 5\n",
        "Increased learning rate further\n",
        "- Effect: no change in val_acc"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UABCrW7fF-r3"
      },
      "source": [
        "# increased learning rate further\n",
        "epoch = 30\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "# modified learning rate\n",
        "LEARNING_RATE = 1e-3\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)\n",
        "# starts with higher accuracy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ANQIFQFEKAo7"
      },
      "source": [
        "Increased number of epochs\n",
        "Effect:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OokW8WMOLAoX"
      },
      "source": [
        "# Round 6\n",
        "- Increased number of epochs\n",
        "- Effect: Reduced validation loss, improved accuracy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n3pqxsRxItp5"
      },
      "source": [
        "# increased number of epochs\n",
        "epoch = 60\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "# modified learning rate\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)# reduces validation loss"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2_KzrOxQyqwd"
      },
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MczX5q9rLXOE"
      },
      "source": [
        "#Round 7\n",
        "- Added additional dense layer but now the val_acc does not change\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EieFqs3SI9gu"
      },
      "source": [
        "# added a layer\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "# modified learning rate\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_pEc7TJcNLY5"
      },
      "source": [
        "# Round 8\n",
        "- reduce epochs\n",
        "- add additional dense layer and removed shape thing from after the first layer\n",
        "- batches of 8\n",
        "- learning rate reduced a bit\n",
        "- increasing epochs does good stuff"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "13tr1BavNKzD"
      },
      "source": [
        "epoch = 64\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DJ-WCVcVGMtF"
      },
      "source": [
        "epoch = 48\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "apI6SHvWGVrN"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVI_wab8A1Px"
      },
      "source": [
        "# Round 9\n",
        "- added an extra dense layer\n",
        "- results in no change in val_acc and nothing other than straight cmd_vels?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m-nURX7SAqOW"
      },
      "source": [
        "epoch = 30\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2TMqLaYJG-PW"
      },
      "source": [
        "epoch = 48\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EIJYSiw_7ZyS"
      },
      "source": [
        "#Round 10\n",
        "- 2xDense layers + dropout!\n",
        "- good stuff!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jGkkpodg15Rt"
      },
      "source": [
        "- tried with only some of straight data and all of the straight data\n",
        "- conclusion:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LXejH-SXHAcW"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1dpBGPu-0G5d"
      },
      "source": [
        "- trying this with less straight cmd_vel data (1.4 vs 1.5 times the left turn data)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NDfzLZLszXIk"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YIkXk37dwGWv"
      },
      "source": [
        "- worse with 200 eopchs vs 128"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6jf3HJ7s9KcO"
      },
      "source": [
        "epoch = 200\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraExtraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s7ECs1F7x0Hs"
      },
      "source": [
        "not good\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GTYfm-A5Ha9J"
      },
      "source": [
        "epoch = 128\n",
        "batch = 5\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-3\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "olMLSdzexmfE"
      },
      "source": [
        "- lower learning rate better - this performed more poorly than a learning rate of 1e-5"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "skMdqLo0Hgxj"
      },
      "source": [
        "epoch = 100\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-4\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gozjiTxsHgjn"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Un3312hsXCAm"
      },
      "source": [
        "# Round 11\n",
        "- trying extra dropouts"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dry1Zrt0J5W5"
      },
      "source": [
        "GOLD MEDAL FOR THIS ONE vvv\n",
        "- fewer epochs and smaller batch size = beauty\n",
        "[[222  22   0]\n",
        " [ 52 123   0]\n",
        " [ 24   6   0]]\n",
        "Confusion matrix, without normalization\n",
        "[[222  22   0]\n",
        " [ 52 123   0]\n",
        " [ 24   6   0]]\n",
        "              precision    recall  f1-score   support\n",
        "\n",
        "           0       0.74      0.91      0.82       244\n",
        "           1       0.81      0.70      0.75       175\n",
        "           2       0.00      0.00      0.00        30\n",
        "\n",
        "    accuracy                           0.77       449\n",
        "   macro avg       0.52      0.54      0.52       449\n",
        "weighted avg       0.72      0.77      0.74       449\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAgAElEQVR4Ae2dB5gV1fnGX1tca1QsAWkLLL2jsURNzN9eolGjsSUajV1jTVQsiKjRaGJviSXYxYoYbIgVFUV67yAiSFWKisn5P+/lnOXseHfP3d07c+/eec/zfM+ZOTNzZs67e39z5jsNUJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFikCBTQC8BGA5gAH1eJ7jAbxWj+uL6dI9AUwupgfSs0gBKRCPAscB+ATACgDzAQwGsEcebnUigOEANsxDXg0hCwOgTUN4UD2jFJAC8SpwIYCFAI4AsBmAjQAcCuBvebjtFQAezUM+DSWLEFjT8oJpKH8vPacUiEWBH9ta6m9qyH1jALcC+Nwat5nG8AsAnwG4yMKZtd2T7bFrAHwHYI29xykA+kRA2xIAYeSAcxKAGQC+BjATAF0ADEx/z24z2h3Ax9bFwJj7LrwF4FoA79t86ELY1h2MxO75/+w9/+EADgIwBcASAJd71/wUwAcAltma/Z0AfmSPv2PLstKW9xhPn78A+ALAI14aL2tt79HT5tEEwJf2HO+22pQCUqAhKXAAgO89sGV79r4APgSwPYDtAAyz4OK5BBOv5zms6RJIqwBsbTOKgjS674OVteWvALSz1zYG0Mlu+2DdBsBSAHQzEMjH2v1G9lyCdTqAtgDo4+X+X+2xaOSe/yr7/H+0YHscwBb2/qsBlNsLewHY1d6Xzz4RwPleptEaq8v/Rvsy4vM4mLvLeM8JADYF8CqAm90BxVJACjRMBVgjZE2qpkBIEZgu7A9glt0hJAgeV+NkMt0KhA9DFKTR/ShYWRM80gLRZpGJfLA6v61/nLVInsNAkNIF4cJZAF5xO5HYPf8GNp0wJRx38c4bAYC12GyBUH3eO5ANrKy1l3nnRMHKQwMBjAUwxvsa8C7RphSQAg1JgVxqrASnqzmybO3tJz63s0GC0N3HihAFaXTfBysvIbRft5/aL9t7Md0HKz+roz0MngTQ296TYD3Vbkev9ZIzm9Hn5wuCcORzuUAXxAl2h7XgQfZlxNo1a+fvuhPttX7jFfOf5x3nZvSeTKNPm/dl7VVBCkiBBq4Afaz0CR5VQzmiNdb9IjVW+lj9UBNYLwHwnHcya7YEil/j5WF+Mt/iQcsHa7YaK90Tfo01LrAOsZ/qrNkysMbq+36z1Vij+kTBurl1XfzLQpiuDgUpIAUauAJseFpgP3fp56Ov9EAAN9ly9bN+VfpX2QhEkDCNIQoJptUE1n0BLALQHACh/qIH1h0AHGZ7JqwPgI1fb6+9TZUaK32pdBmwixiBzEYi7rsGqjhrrOw6Rn/serY2zf6oPljpVuGLx4Vs+kTTHgDwlL3gfgBPu4sVSwEp0LAVoK+V/VhZeyUc+BnuWtrpH7zdtoKz1Z/bzmcYhQRVqAmsPH6XBeE0++nraqxsrCJIOZiAoCQgO1pZ/Rork9jHlr5PnsvY73MbJ1j3AjDJtvrTBcBGOx+sZ1id+PxHV/Pi8TXji4SuAldLZe2VurjeELb4iqSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUiBhBTYoM+ttsp1sk+1Ml+49ZVaDnj17GVkv06hRI/b2yGtYf4vmOf3eahhpl9fnUWYxKEColnU/W9b9bDN3yTcyq8HqNcbITOblku+fXa6/Odt1MN+3V35JKJDrHzkN8BVY171YBNW1LxbW2vP9O1xv0+1NWc/zgiaw5lv5BPMTWNfV1gVWgTX6QokNrL3ON2UBE1gTBGG+byWwCqzZXihRwKR1Pzaw7nSBKQuYwJpv2iWYn8AqsAqs1fuT4wHrDqZs54uCJrAmCMJ830pgFVgF1oTButkOpmyXPwdNYM037RLMT2AVWAXWpMH6E1O261+CJrAmCMJ830pgFVgF1gKAdbfLTFnABNZ80y7B/ARWgVVgLQBYd7/clAVMYE0QhPm+lcAqsAqsCYN188ambI8rgyaw5pt2CeYnsAqsAmsBwLrn1aYsYAJrgiDM960EVoFVYE0arE1M2V7XBE1gzTftEsxPYBVYBdaEwbpFE1O2d7+gCawJgjDftxJYBVaBtQBg/eV1pixgAmu+aZdgfgKrwCqwJg3WHU3Z/90QNIE1QRDm+1YCq8AqsBYArPv81ZQFTGDNN+0SzE9gFVgF1oTBumVTU7bf34IWAGszAEMBTAAwHsCfLDa4BPnrAKbaeGubvp5d8p3Lko8B0DNBzKTvVgKrwCqwFgCs+99iygIWAGtjD45bAJgCoCOAmwBcaknG+Ea7fRCAwQAI2F0BfJQ+2iVYYoFVYBVYCwDWA/5uygIWAGuUEi8C2BfAZACELgNj7jPcB+BYu83IP89L1mZeFBBYBVaBNWmwNjNlB90WNACzLFw/sfFp1fzoWwKYA2BLAMu8c1g7dfuDAOzhHRsCYCdvX5v5VEBgFVgF1oTB+uNmpuzgO4KWY411cwAjABxhueBA6jCx1G4IrE6RJGKBVWAVWJMGa3NTdshdQcsBrBsBeBXAhR4r/E98uQI8YRLdFFgFVoG1AGA99G5TFrAAWPmZ3x/ArRFg/C3SeMXGLIaDI41Xw226ojgUEFgFVoG1AGA97D5TFrAAWOkv5Qqy7Do1yhpb/hsBoP+U3a3eAMDuVwwE8V0ApgMYK/+qVSWuSGAVWAXWhMG6VQtTdvg/gxYAa1xIUL75UEBgFVgF1gKA9df/MmUBE1jzQbgC5ZEEWNvs39u8NXyymTD9czN+2ufmohsHmLLuZ5u/P/y6mTRjvhkz+TPz4pBRZoc9Ls6kH3T6HWbE+Nlm7JR5mXj/P96WSec1cVo2wMSZ9tGYqWa3PfYyFW3bm7btOpg+199seL/Tz7nAtK5oa9p37Gz2P/hXZtzMLzLpcT5LNO9CLnf94qDBpqJtW9OqdWvT97obTCGfJZZVWrdqYTY54oGgCawFgmI+bpsEWFvuc5nZ9bc3ZKC47e4XmimzFpjuR1xrDj7jDrNZr3Mz6Tc/+JqhEZy7HHODKd/38sx2zyP7mXkLlma244Qq847CJe79TybMNP8Z+kHmvhNnf2nKW7cxQ4aNNI8+85KZuXBFJv3M8y4ytLifJZp/oWC24pvvTXmrVmbC5Olm+cpvTZcuXc2no8cXDK5xgHX9rVuYTY96MGgCaz4IV6A8kgBrFIgDh442rJX66UdfcJ954uXhVdLc8cXLVpgtd/5T1mPunHzEUbgkvb/vgYeYx54dVAWi9/d/yhx+1DFV0pJ4rkKBdeg7w8w+++5XCdK+/a43tEI9TzxgbWk2+81DQRNYCwTFfNw2abC2PfBKM+fzxWa7n11UBZSD3hpjTrr84SpphOWxF//TDPlw4g/S8wHSaB5JAKu6ewwbNck02bGpmTBrYRWI7rP/Qea2ex+sklZdHvlMLxTIHntygDnp5FMqQfrAQ/3N6WeeXbmf9HPFBdbNj37YhExgXUe43namGdcFYhcA5wPYdN0pOW/1BbBP4Ow+AC7Ocs5WAM7Kkv6DpCTB2mi3CzI+02MuvL8KKK+6Y2DGxxoFXY8jrjXT5yw0HQ65usr50fPytZ9PMNUmr0lzFpku3XqY+//9ZBWAXtK7T8bHOmfx6irptcm7rucmDTB3v7SAdcvf9jchE1jX4mo3AB8A2NjSa1sATex4X25nCxtkS6xFWnVg5djhcbnkkxRYN9/pXPPa+xPMn29+tgokT72yv/lw9Ayz9a7nV0lvvV/vjC9279/fUiU9XxDNlk9dIVSf62Ys+Nrstfc+5sprb6wCz1vuvN/03GkXM+WzJVXS63Ov2lzrQJd0nApXwDYtzZbH9g+awLqWYByr+1IEZucB+M52yOXciQwrANwCYLSdFOEqAB9bEN5vO/LyvIcBHLX2ErDj7yQ7Jvh2ABz3y0CwPgjgLQAzAPB+DE8CWG07DXMkRrUhKbA++tKH5o5H36wCyUPPujPTU6Dp3n+pks7eAaMnzzXRmm02GOYzrTbgyce5rIkeecxx5pTTz6kCz/5PD8z0FBg1ZW6V9HzcM9c8kgaqu9/Xq9eYluXlZuKUGZWNVyNGjSspV8AG25SbHx/3SNAE1rXY4mQIHAHBeRHvBvBzSzPOUOPXWDla4miPdG5kBJMeAXCoPebAWgZgLoBym/5EBKzDbC2Z91gMgOOHQzVWzpKzdsacjTavArV8gsrl9cuTbjEM7FY1atLcjB12zl1m2uyFZu78JZVp9w94N/MsV9850KxY9U1lOq9pFoGvyzufca7Qydd5z/5nCP8XMt2qOnbuamj/fuoF06K8lWncZMfMPtNOOOnUxAHrQFeI+PmBL5s2FRWZ3gF9+vYrGFRZ9jh8rATrVsc/GjSBdR0l+Wn/CwDXAPgCwElZXAHfA/BdAEfaSWc5zGyeN87XgbU7gLfX3QK/ioCVfl0XJgJomgNY3flIqsaaTwDGlVe+gFkK+RQCqMV4z7jAus2Jj5uQCayVmKqywc94ugaiNVa6AlxgbXQBAC6zwMBPexpDrmD1G6/oV2VtNVRjtbeAwOoNOigFIOarDMUIuUI8UyxgbVRutvnd40ETWNdiqh2AikpiAf0A3Gn9q+4znod9sLL1nmDdBABdCQRjFKw8RlcAYcnwWKTGmg2snIhhtj2/xkg11nWjufIFpVLIpxAQK8Z7xgXWRr9/woRMYF2Lrl4A6O/k4l7sbvWc9a2ea5dB8BuvfNgRwJxx5n0AD2UBK8+l39U1Xt1r4cp0QjgbWHnscQvqomi8iuvzPZ/5lgIQ81WGYoRcIZ4pDrBu2KiV2e7kp4ImsPqYjGebtVkGTv3FhrEL7H69I9VYVWPNBuNCQKwY7xkbWP/wlNkuYAJrvfEWzIAgZY8D1obpCqjLgIOsNxFYBVaBNdnZrTbctpXZ/pSng5YDWNnVcmGkz/pTlhXkBdt3GDPQlei6YDKNX74KcSkgsAqsAmvyYN3h1AEmZDmAdS+7BHZ1g4HYX5795BlybtC25yuqjwICq8AqsCYL1o22bW0an/Zs0HIAa03ApNuQjd6uQV1grQ8oa3utwCqwCqwFAOvpz5rGAbOf8m7pa8bZlr+uDpiszfIaF3jeSgAjbb/4Pd0BxTEoILAKrAJrwmDdrrVpcsZzQYuAsbpff3VgvQfARd5FnMOE3TAZ2IOJtdkt7b6ifCsgsAqsAmvyYN3xzOdNyOoB1g1t/3iOwqwucH6Rnao7qPR6KiCwCqwCa/JgbXr2CyZk9QDrAZFh8KTEdt5Q+lZ2+Lw/T0k9SaLLqyggsAqsAmvyYG12zosmZDmAlRMyzQewBsBnAE6xP24Ohz+jyg8d4Jwk4233q0+9yZ4ip2k3LwoIrAKrwJosWH+0fRvT/NyBQcsBrHlhgDKJQQGBVWAVWJMHa8s/DTIhE1hjAF5SWQqsAqvAWgCwnj/ItAyYwJoUBWO4j8AqsAqsyYO1/IKXTcgE1hiAl1SWAqvAKrAmDNYd2phWF/4naAJrUhSM4T4Cq8AqsCYL1o13qDBtLh4cNIE1BuAllaXAKrAKrMmDteKSwSZkAmtSFIzhPgKrwCqwJg/Wtn9+xYRMYI0BeEllKbAKrAJrsmAt+0mF6XDZq0ETWJOiYAz3EVgFVoE1ebB2vPxVEzKBNQbgJZWlwCqwCqzJg7VT79dMyATWpCgYw30EVoFVYE0YrI0rTOcrXguawBoD8JLKUmAVWAXWZMG6SeO2putVbwRNYE2KgjHcR2AVWAXW5MHa7eo3TMgE1hiAl1SWAqvAKrAmD9bufYaYkOUA1myrtPaxc61yJVbaQR5LLgMwDcBkAPt76drMtwICq8AqsCYM1iZtTY9rhgQtB7BmW6WVYL04Cyc6AhgNgEu0lAOY7k18neV0JdVLAYFVYBVYkwXrpk3aml7Xvhm0HMDK3350zavqwMraKs2FVwHs5nYU51kBgVVgFViTB+tO/YaakNVxlVaCdRaAMQDoKtjaIuNOACd4+HgAwFHevjbzqYDAKrAKrMmDdefrhpqQ1bHGuoP9xF8fwHUWrkSGwJpPcIbyat+5u/lo+jLZ9GXm2P4jZVaD1Wuqh02ajvXs2cuEfkO1Pb7Zjm3NLje8FbQ6gtV/HN9NIFeAr0zc2wLrupeKwLruxZImeNZU1njA2s7s+te3g1ZHsDb2mHEBgCftfqdI49UMNV55SuV7U2AVWLO9UGqCTZqOxQXW3W5824QsB7BmW6X1EQBjrY91IAAftL1tbwB2tzow3yxRfp4CAqvAKrBW7/aIC6y73/SOCVkOYPV+ydosKgUEVoFVYE0WrJs3bWf2uPndoAmsRYXK2j2MwCqwCqzJg3XPW94zIRNYa8eyojpbYBVYBdbkwbrX398zIRNYiwqVtXsYgVVgFVgTBmuzduYXt74fNIG1diwrqrMFVoFVYE0WrFs0a2/2vm1Y0ATWokJl7R5GYBVYBdbkwfrL24eZkAmstWNZUZ0tsAqsAmvyYN3njg9MyATWokJl7R5GYBVYBdZkwbpl8/Zmv7s+DFoxg/UOALfXYLWjUAmeLbAKrAJr8mA94O4PTciKGay/B1CTlSAqa1ckgVVgFViTB+uB93xkQlbMYI1SZtNoQtr3BVaBVWBNHqwH3fuRCVlDACtnyp4AYI4FaTcAd6cdqiy/wCqwCqzJgvXHzdubQ+8bHrSGANaPADQDMNKD6ThvO7WbAqvAKrAmDNYWHcyv7v84aA0FrISnD1YurJX6ILAKrAJr8mA97J8fm5A1BLA+A2B3AJ8C2MiuYugmgE01XAVWgVVgTRasW7XoYI544JOg5QDWbMtf/w3AJDsf6/MAtrKA42oCq+2S2FwW+958gG9bAI8BWADgSwCPAmiUj4wbeh4Cq8AqsCYP1iMfHGFClgNYsy1/vR+ADS2XbgRAY/CXabFJimJTQGAVWAXWZMG6dcsO5qiHRgQtB7CGgPlrW6EMnVdnvrQC8JKtrS4E8CIApqU+CKwCq8CaPFiPfvhTE7I6Ln/tM43Mc0tes8a60rYzvQ1gT//Eum5/COBEW0VmNZk3Y0+B1AeBVWAVWJMH62//PdKErJ41Vq5vRR/rehZyG3vuz14A5gLYsr4AHJMlA/UKUD/WKst+ZwNMWtPStGBgTWWNY82rbVp2MMf1Hxm0eoD1JAAfAKhpQNRbAHbKwsWckrYBQKMD91LrwG0B4M8AbsgphxI/STVW1VizvTxqgk2ajsUC1vKO5vhHRgWtjmA9wA6G2i6CLu5vYNPoBp1n2Rg5LbfdmQC4fjbjqDE99UFgFVgF1mRdAduUdzQnPDoqaDmANdvy19PsZz67VPndqo4EMN6msdvpoamHX5wCFAKsjXdsZlq37WgqOnQ27v7Hn3quadGqwrRp19H8fN+DzRsjZ1X5TP9o+joAxrWdDTD5Ths6dZFZtvo7M2fpKuPyfmncAjNv2Woze8kqM3z2UnPKE2Myx654ebKZuXhVxmYtXmVuGTqj8hp3bVxxIWulLw4abCratjWtWrc2fa+7wRTyWeKosTYq72h+//jooOUA1jjRkHPenQEcDeB3nuV8came6MAWF6yy5Uuwvvrx9CrgvO3h58z7kxdl0k487U+Glu3aONPigpSf7zWvTDGXDZpUBazXvz7NHP/IyAw0Xxz7haHxmt8/xs/FtelnDhibAbLb9/OMY7tQMFvxzfemvFUrM2HydLN85bemS5eu5tPR4wsG17jAetITY0zIGgJYrwYw1A4QeAjAFwA4Giv1oVjA6gPzxrsfMfv/6jclCVZC8Nxnx1cBqw9G1krfm7H4BzXT854bb5at+q4StP41cWwXCqxD3xlm9tl3v0qQ9u13vaEV6nniAuvJT4wxIWsIYB0LYH0ArifADgBeTz1VC9QroHHT5qZdp66mXadu5tJ+t/4AoHv8cn/T55b7fpDuwzeO7TgAlS3PmsA6Yu4yc+e7syrBesV/Jpu5S1eZ1d99nwpXwGNPDjAnnXxKJUgfeKi/Of3Msyv3kwZsHGDdtlVHc+pTY4PWEMA63EJ0hO27xb5dHE9b37CiFhmwVY59ZzkRDDvnnpXlWg6zdU5n1qrZcuf2f5Tl/HonFaLG+tL7EzLQHPzRVNOmfSdz7xMvV0L0jAuvyPhYP5y2tDItDohmyzMbBONIqw6sT376ecbHmu2eF70wwUz7coX53aOjKqGb7bx8pSUNMHe/tID1j0+PMyFrCGDl3KucjOAMAFMt3OgSqG+oDVh/C+Bf9oa5jNvtYyeLyfUZ3fjgXM/PnFcIsPpQO/W8v5hzL702A9Erb7zLdO6xs3l73OeJQ5XPlC8ohfLJBtZ73pttpixckfGrVnf9uPlfmcsHTUrkOR3oko7T4ApgjfW0AeOC1hDA6sOGUOvqJ9RjOxtYWwN4BQBrx+/yaxtAdzvJNieAYQ30KW+mGc5Gky34YH0YwFHeSe6+v7D3GAhgCgDus+Mv/ceskXPiGTfywrt83WbSYH1r7Dzz5ui5GXByu0uPn5pbH3wmYy3btDOvDJ9WEKgWEqw3vDHNzF262pz21NreAA6s9Ku6xqpznx1nlqz87gfnuHPzHScNVHe/r1evMS3Ly83EKTMqG69GjBpXUq6A7Vp1Mmc8Mz5oxQzWngBqsnWEqduWA5x/9RAAFTZhFwBv2m2OhrjTbte2xloTWDn+t9zmS7AuB9DU+pQ5+mIPeyxrlDRYnxs6KvP5TxdAeUV7w09/Qq1p83Kz/U92zHTBYjesXx97cuKAzTecsuX3/owlGUCu+e//zKIV35r7hs0285d/k9l2Xaten/xlplZ617uzMv5Vps9YtDIVPlYC9vmBL5s2FRWZ3gF9+vYrGFT5LHH4WLdr3cmc9dyEoBUzWNkToDpzwMsKnBwTo2Dd3KuJOt/oRJtXXGBl+VwgWP1GuXu8SRjcOYxPs3+0T37SpFniAPNdAcW0nQ2EaU1zNci0x3GB9ZznJpiQFTNYfZjEsR0FKyc1mF/NjeoDVvpm2QeXgb0bvrPbBOkgu80ous8aMu9bbUi6xlpMII0+S1ohmq3caQeqK38cYN2+dSdz7vMTgyawVsXWMAC/sUn0b3LhQgYfrGz9n23Tq4t8H+sV3oS1hwMw9qIoSKP7AmstRmxlA0xa0xxY0h7HBdYLXpxoQpZmsP4PwGeeXWj9nWy8Yp9Zrgx7lYWgD1YmPQ6ACxrm0njFfrec+pB5ckIZV1OOgjS6L7AKrHXqRZB2oLryxwHWHdp0MhcOnBS0NIO1utpmg0mXK2DdHARprZ1mK7cDS9rjuMB68UuTTMgaAlj5Sc7JrV3tsTmAnzYY+sX4oAKrwCqwJju7FWuslwyaFLSGAFa2jt8FwLXQbw3g4xh51WCyFlgFVoE1WbD+pE0nc9nLk4OWA1izrdLK+afZM4gDoRiTdQysXN4OgNMKcuJ/dkOtd+D8gwwcTuqCmzfA7acyFlgFVoE1ebBe/p/JJmQ5gDXbKq032Un9yTNO7u9WaT0IwGAL2F3ztTQVx+hz9mwHWI7b9yGbSqiy0AKrwCqwJgvWxhWdzRWDpwQtB7DyJxwdbDQZQGMLNMbcZ7gPwLF2m5F/npdcu83jAXDYJ1vwr7OZui5RtcupxM4WWAVWgTV5sF75yhQTsjqCdZmHKH7+u332d/dHYXKEaJ3XvPLukRmzfzaAcwB08A+keVtgFVgF1mTB2qSis7nmtalBq+Py1w6kDmtL7UYsYGUvgGzmbp7aWGAVWAXW5MHa9/WpJmR1rLH6n/ixuwI40TVbwhiztex7u7BWaoHqCi6wCqwCa8JgbdvZ9HtjatDqCFYOOGKjFQNjNmYxHBxpvHJzVNvD+YnY1cDNjZqfHBtoLgKrwCqwJgvWHdt2NjcMmRa0HMCabZVWDpen/5QVyDe8Ja7pb2WX0+m2gpkX/2o27LH2mvogsAqsAmvyYL3xzWkmZDmAteD84hh+ZxfbcfqvFvypiuABBFaBVWBNHqw3DZ1uQtYQwMpVWp31BsDuV2VFwLWCP4LAKrAKrMmCtWnbzubmt6YHrdjByoEBNxecYEX6AAKrwCqwJgzWdp3NP96ZEbRiBqtbYI9LlChkUUBgFVgF1mTB2qxdZ3PbuzOCVsxgdUNYOQkLR16dCOAIz7KgJl1JAqvAKrAmD9bb35thQtYQwMqlrp1xRhhuM059EFgFVoE1WbA2b9fF3PX+zKAVM1g5NwB7A1zkxdx2+wJr5+5aTNCuMpANMGlNS/sE1678cUx03bx9F3PPsJlBK2awcmE/Tm7tegT4sZv0OtVwVY1VNdZsLw8HlrTHcYH1vg9mmZAVM1idjzXV8Kyp8AKrwCqwJusKaNG+i7n/w1lBK2awas7Vmqiq+ViruEGyASataWmvqbryx1FjJVgfGD47aMUMVi5ToFCDAqqxqsaa7eXhwJL2OA6wtmzfxTw0fHbQihmsNSBFh6iAwCqwCqzJugJaduhiHv54TtAE1gbMaIFVYBVYkwfrvz+eY0ImsAqsVXyVH9nuSw0tzgaYtKal3QXgyh+HK6C8Qxfz6CdzgyawCqwCa/+RppQg7MCS9jgusD42Yq4JWQCs7QCM8uwrAOcD6ANgnpfO1VkVklZArgC5ArK9ENIOVFf+eMDa1Tz+6WdBC4DVRwUnmvoCQAsLVk6NqlBIBQRWgVVgTdbH2qpDV/PUyHlBqwVY9wPwvuUIa6wCayGhynsLrAKrwJo8WJ8eOc+ELMdVWvkz5rwnXH2agWCdZdf4Y/rWNl1Rkgp06dbTzF78jWzxN+ar1f+VWQ3cp3Da4zhcAa06djUDRn0etBxrrD8CsAjADpYbjOkaWB/AdZpsKkmaevcSWNe9VATWdS+WtAPVlT8OsLbu2NU8O+rzoOUI1sMAvOb9pP3NlgDG+QnaTkgBgVVgzfZCcWBJexwXWJ8fPd+ELEewPgngZA8XjcPkrl0AABlTSURBVL3tCwDwuELSCgisAqvAmqyPlTXWF8bMD1oOYN0MwGIAP/a48Yhd3nqMndzfB613mjZjVUBgFVgF1mTB2qZjVzNwzBdBywGssbJBmddDAYFVYBVYkwZrNzNo7IKgCaz1AFuhLxVYBVaBNXmwvjxugQmZwFpoOtbj/gKrwCqwJgvWio7dzOBxC4MmsNYDbIW+VGAVWAXWhMHaqZt5ZfzCoAmshaZjPe4vsAqsAmvyYH1twpcmZAJrPcBW6EsFVoFVYE0WrG07dTNvTPwyaAJroelYj/sLrAKrwJo8WIdMXGRCJrDWA2yFvlRgFVgF1uTB+uakRSZkAmuh6ViP+wusAqvAmixY23XqZt6evDhoAms9wFboSwVWgVVgTRqs3c07k5cETWAtNB3rcX+BVWAVWBMGa+fu5t0pS4ImsNYDbIW+VGAVWAXWZMHKyeWHTV0aNIG10HSsx/0FVoFVYE0erB9MXWpCJrDWA2yFvlRgFVgF1gKAddpS80HAcgArl2AZa1dk/cSyZBsArwOYamMtzVIIyAqsAqvAmjxYP5y2zIQsR7BuG+HGTQAutWmMb4wc124SCgisAqvAmjxYh09fZkJWR7BOBuAmt2bMfYWkFRBYBVaBtQBgnbHMDA9YDmCdCeBTACMAnGbZscxjyHoA/H3vkDZjVUBgFVgF1mTB2qFzd/PxjOVBy2H56x0tHLYHMBrAXllAujRWgCjz7AoIrAKrwJowWLt0NyNmLg9aDjVW/0fdB8DF9tNfrgBfmUJsC6wCq8BaALDOWm5GBCwAVi4kuIVlBreHATgAwN8ijVdszFJIWgGBVWAVWJMGaw/z6ayvghYAayv7+U8XwHgAvS07GgEYYrtbvQGA3a8UklYgabB+MGaq2XWPvUybtu1NRbsO5urrbzazF6+DW+++fzUAzMgpn1VJ98+JazsbYJJMmzN/sTns8CNMRdt2pm279ub1oe+ZJO/v32v1muphE/exFwcNNhVt25pWrVubvtfdYOK+X0359+zZi/+PeQ0duvQwI2d/FbQAWPP6TMoszwokDdbh42eaQW9+kIHm+FlfmvLWbczr74/M7BO6e+29j9mxabNUgvXY4080d9x9Xwami5avNgStD7skt2uCTZzHVnzzvSlv1cpMmDzdLF/5renSpav5dPT4gsE1DrB27NLDjJrzVdAE1jzDLsnskgZrtLa57wGHmEefGZQB64GH/toMfnu4adqseerAOveLJaZFi5Zm+arvCwZTH9xxwrOmvIe+M8zss+9+lSDt2+96Q6vpmjiPxQXW0XO+NiETWJMkYZ7vVUiwvjdykmmyY1MzbuZC889HBpiTTzs7A9g0gvW9D0eYnr12Nsed8DvTtVt387uT/mDmL/qqYJCNE1Y15f3YkwPMSSefUgnSBx7qb04/8+zK/ZqujeNYXGAdM/drE7JSBut/7TjccQBeArCV5VoTAM/kwLgV1ZxzOICOWY7RAT3Kmrs398/Lcm5ekgoF1gmzF5nO3XqYex9+0kyau8R077mzGTtzQWrB+tZ7H5kNNtjADHl7WAamZ5x1rrnk0t4Ca4mCdezcr03IShmsPhj/7bW85Qo1/3r/mocBHOUnZNmu7tosp2aSNqzuQE3phQDrtC++zvhSr7j2xgxIX333E9No2+0yLgDWVgkY1mQ/njArczzqPohr3/8UTnp76sx5pnnzFpUgfeX1t8x+BxxYuZ/088RR+8slz1S4Arr2MOM+WxG0tID1DAB3W0i1BMBaLMOmAJ4GMAHA8wA+ArCTPUY4Xme7RXwIYAcAuwNYAoDD0VgbbW3PjUYOrP69eA47AbMzMMNbAG61f4CL7D4nXRgOYAqAPe151UZJg3XWotXmiKOPM384/ZxqoZlGVwDBudvue5gRoydkYHpp76vMeedflDqwfr16jWlZXm4mTplR2Xg1YtS4knIFdOraw4yftyJoaQDrBgAG2A64hJQPO4LuPkuuzgC+98DKrhqH2mPsqHuF3a5NjdW/Fy+PgtXBnscI2lvsPQ4CwH5s2QLHFXOasU/YAh9XDTBbvs+8PCTTnap9x86mY+euGXvoyReqPENawUo/a4+evUynzl3MwYf8ysz+fFHqwMpa7fMDXzZtKioyvQP69O1XMKjyWeLwsRKsE+atCFopg9X5Ob8E8A4AApbBh90LAPa26Yw4KYKrsX4LgBMhMBwD4F92O59g/bnNkxHB+jO7z9rxNO9Y1s2ka6zZYFssaUl/bhfz/XL5bE/DOXGBdeLnK03IShms7nOcn/vveo1IuYLVXU+o0adKoDLUBqxNrZvBXpqp9fquAAdxHidY3T7naeREuDUGgXXd4IRiBl3Sz5YGaOZSxrjAOmn+ShOyNICVcOoBYDYANhL5YL0EwD2WXmzpX+PBrTqw3gHg5BqJB7hrNwKwCACHqm0MgL5agdUbrZWvGm/S8Crm++UCnTScExdYJ89faUKWFrCSg+xydWIErJwAgV2v2Hj1nG2QqrDQdHDkrl9j5ec6zx+ZQ+MVr2V3q+nWHcHarsAqsMbqf00DNHMpY2xg/WKlmRywUgar5WONEf2uZfYMtvCztf9HNV5RRAflCpArIFvNORfopOGcOMDK/ttTF6wKWtrByqm72MLOWWbGADiwiLgZfBSBVWAVWKufcCYusE5bsMqELO1gDcKrmE8QWAVWgbUAYF24ykwLmMBazOQMPJvAKrAKrEmDtaeZvnB10AJgbQZgqG2r4Xysf7I/dba/zPOGxrM/u0LSCgisAqvAmixY+Zub+eXqoAXAyuVXelpe0B3JkZbslUSwchCRQiEVEFgFVoG1AGBdtNrMDFgArFFsvAhgX4E1KkuB9gVWgVVgTR6ssxZ9Y0KWwyqtjhrsVz8HwJYWrBwYxIb0BwFs7U5SnKACAqvAKrAmD9Y5i78xIcuxxro5gBEAjrDY4FB2dgFd304ARbgqJK2AwCqwCqwJg7V7TzN3yTdBywGsHJX5KoALq+GGP0K0mlOUHIsCAqvAKrAWAqzfmrlLarYAWDm5U387bajPBjZquXABgCfdjuIEFRBYBVaBNVmwdu3e03y29NugBcC6B1cztr5Ut+oIu1Y9AmCsTR8IwAdtgmRJ+a0EVoFVYE0erJ8v+9aELADWlJOryIsvsAqsAmvyYJ2/7DsTMoG1yOFZ0+MJrAKrwFoAsC7/zswPmMBaE7mK/JjAKrAKrMmCtVuPnmbBV98FTWAtcnjW9HgCq8AqsCYP1oVfrTEhE1hrIleRHxNYBVaBtQBg/XqNWRgwgbXI4VnT4wmsAqvAmjxYv/x6jQmZwFoTuYr8mMAqsAqsSYO1l1m84vugCaxFDs+aHk9gFVgF1mTB2r1HL7Nk5fdBE1hrIleRHxNYBVaBVWAtckw1vMcTWAVWgTV5sC5d9b0JmWqsDY+nlU8ssAqsAmvCYO3Zyyxf/d+gCayVmGp4GwKrwCqwJgvWHj17mWyaR9ME1obH08onFlgF1ugPmvur11QPmzQdi2P56wxYv/mv+SpgAmslphrehsAqsAqs1b9E4gLrim//Z0ImsDY8nlY+scAqsAqsyYN15bf/MyHLAawHAJgMYBqASyt/1NoovAICq8AqsBYArN/9z6wMWACsXNdqOoBWAH4EYLRd/rrwUNETAAKrwCqwJg/WVd8ZE7IAWHez6105jF0GgKZQJAp8af+AnxQw5nK9hbx/Md1bWqz7XygGLfj7yHd4Jcf/93GR807zHuQoAP/y9k8EcKe3r00pkIGqZFirACGvsFYBaVH9f4LAWr02OmIV0A9o3b+CtJAW6xSofkuugOq10RGrgGCy7l9BWkiLdQpUv7UhgBkAyr3Gq07Vn64jaVTA9x2lsfx+maXFOjWkxTotsm1xuesptndA72wnKE0KSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACSSiwXuQm0f3IYe1KASkgBaoqsAmAQ21SVwCHVT2cuj0fotRjSwB+WuoE8QrsdCgDsJGXrk0pIAUiCnCCiasBjLTWNnI8rbtnA5gAYCiAEwA0TqsQkXLzxfuyHe65d+SYdqWAFPAU2B3AFwDe9dLYQTpNwdXGWOYjADwOYFMAxwC4B8DpAH6SJkGylLUdgDcBcLjnHwFMBbBflvOUJAVSq4ADCWO6AzoDuA/AQDvShMKkBSROC5a5CYDHALzt/WcQtPcC+BOAHbz0NG32AvAMgOu8Qv/WdqA/0EvTphRIpQLbAGhjS86RJQRGX7u/GYCHAAwCsBeAZwHsmCKVzgfwlC37ewD80TbHArgVwFYp0sMV9RcALrTa/AdAC8/v/DsAcwFs66W56xRLgVQowEYHQvRaAL8GMMp+6nL8PKdI43FO8MtP3w8AHJIKVdYW8le2tr69LfMeAJ4EcLmnwRbedlo26XN/HUBTW+BHAdwNoLkngPzPnhjaTKcC+wC4EcADAC6yErABi7XUf3puANZAGPzPZJtUEpFfLr5Q7gCw2Kuh0z3yMwCsoV1sS+xfUxIiBApBYH5ov1zYO4JhYwAPW2PNVUEKpFqB9b3SdwDwoK2h0bfKwO4zbAF/pIRhynJG4Xj42uLDuUGeAOBqrYTrrh5s7aklHTl93Oc+e0QMA/BLC1UWnnBl45773ylpQVQ4KVCdAu7Hwh/C0wC2s37Wf9hPXYKWgT0BdrLbpRr92BbMacKeEKyVMWwN4BYA/05Rw50tegaWbpsvFtbg/2JfRH8A8Jr1PROqDE4/u6tICqRLAfcD2B/A/QC4HAVrpWzdJlAJEvpc0zDXJP2o71go+I1QL9l0/mcwnb0jqBVr+U6/Uv6voR/1KvtideX8DYC/A2CDHsPJAN4HwIasNGhii61IClRVwB8Zw5rqZwA4Kzp7ArABizVX+lE5uuh2r6dA1VxKZ48vFjbW9bSfsKyNud4RLCUbaOgKYWDN1bkDbFLJRuybOgLA0fblyoERLtBNwv+Ns2wC+67+1B1ULAXSpgAbHU6y/VNZ9p1tLdXp0No2VrHmSoiwJ0ApB3Zi56J19CuztrWLfbGc58GVrd7f2p4ApayFXzZCdYGtmTKdflQu+XyGdxJfQFz6+QIvTZtSIJUKVNjPfPpSubREIwCTLGydIKy1sgGCw1npN/Mbt9w5pRAfDGAMgEsAXGOXMGa3KdbU+WLhp25LAHQT8HPYr8WWQvmrKwOh+rHtZrcIAHuLMPwcwKdeLZVfOezb6/zw9jRFUiCdCrBFm40w/WxDDH84HOPNfpnux8Ma210lLg8bXva0ZWTN9TbbKOPgyvK/YNc3Sss8CfzfYMMlO/czcATVVx5cOTjkcztYhOs+7WvPUyQFUqlAtFGhi62R8HOOjVPdAbChhp28uwHgJBps7XV9FNMgGl8wHEFFTdhDgLV1ji5L0wgz/p3d0Fz3PxOFaysAv7d++TT8X6iMUiCrAu4HwlZbfvKydsZJRPhpy2Gql9puVryY5xKqnL2Jn8SlGJwermz+PuHK3hB97PBMd04aYl8Hltfv+UC4cpAE3ScM0XNtsiIpkC4FOBnGeNv4wE7df7NDEeln5fDMKy1s2V+VP6JS9Cf6te/oUEsfFNSKE4rQ/5yGwAZKV34Xu3Jz36WdCGCV1aVU/e6u3IqlQFABtmqz1ZufcKyRsbGKnbxZM2PXIaaXau3UicMJZk61U/wdZz/56U/0gwMI01ijT0PgkF36mTlbGbvbcUIZXwdq4MM1+kJKg0YqoxT4gQL8/KexGxVbez+yfVTZCDETwM12gpUfXFhCCZyA+U77AmGZFwLY3JYv2p0sCpUSkiFrUehHZu8HNl6yMYqNl9mC08XVVN1+tnOVJgVKWgE2UL0IoIctJScO4WTEDDzGFu/2dr9UIwL0LQDHA+CsVJxPdbjtYlWqZc61XA6O/B+YDGCA7YKXtonMc9VL50mBzJj/NXaaP8rhahrsnzgEwLQUTf3Hz1uOnGKDHD996RrhEEw25DGwlsYXTZqCG31HvzP/NzjVHxvs2CuCk8sw0MccrdHbQ4qkQDoU4AxMHEXFwFEy7DLDIYirI2P9WRvhcEQO30xL4EizJbYrGTv8M3S0o4b4CcyXTVqmuaM/3XWn4v8B59ZlQx1975wiktNG0j3ELmccPJEWXey/hSIpsE4BNijwc5/zpnLEFCdTcbUOjipiN5k0TePmPnGdQuyHyg7+9CVyEhUuJcLA+RA472wp9oCwRfxBxN4fK+x8sv3tYogc788BI1yvijVZruHFCc3TvirvD8RTQnoUoH9sou2szckyltoeAFTAff5zLPd3KWj5j/7Vz7XdyjhxN+FKF8BltiGLwzHTGugCIVw5TJeBPUOOtBNUuxFX7n8n+pKylyiSAqWrAD9rWTs9xRaRfVI5wxAnxvhzpNj8HE7TqplnAnjD9gTg8t2ciYmBNVcO5WVXM/pb0wIOV06WmYH+1JWeW4D+VPZfZuMeX0AKUiC1CnAeTI5vZ+APh8NSCU+O+Z9vgcvO7my4cT8o9wOzl5VsxNoY+67yU59+VPoPafzUJTjS0vnf/W8w5sxdBKebNOWvdspIrj7LQNdIWlbftUVWJAV+qABbs9m6zflEOQCAS4fQLcDGCK4kyhob12XiZ14agntpMOYCiOyzS5+z6z50jvUfpkELV0anCf9HnrN9l7lAJPs0M3Ai8+V2aW+bpEgKpFsBjg5igwwnaea67hxBwzlU2eLPIaussbnZ8N0PrNQU+z8A/Ox3wfkGOT3iHG9RRLpC2N0qLbNU+SPHqAVdRm50HUfdccIdB9ebbE8Sp6FiKSAFLEB9ITjSit1oXLca/1ipbbOL2X8BnOYVzPXRZE8J+pvZeMWaPbtYpSGwbyrnSXUvVfYaed7OLevKP9gOlHBuAKaX6svXlVmxFKiTAgQKx3sTJm4Wojpl1MAuYvcp9oZws9uz1urgysY81tzdQoENrGh1flzClDVVugAYuCw1e43wi4aB/x/8P+HIPAUpIAWqUYAg4ZBV+lUPreacUk7mCrKEq1uDiWWlT/UVDyalXH6WLTr/Ad0fs61riC8f+tvpf2fjJof1srbP9c3UYFXq/xkqX70UIFzdjySNn3UOrmysO8Q21LCnRBoC+zNzqDIb7a7waugnABhrJy9ndzx2/ueENJzMnBPxsNbqarFp0ElllAJSoA4KEK7/s/OGusaaOmTToC6h75i9QDgtIpebITg5XNUFzvbPvrycwNwFDmGdksI5Elz5FUsBKVBLBdhP07V41/LSBnc6++Zy9dSB9sn5pXIxgN523/maCVz2imBfVQaOREvTUF5bbEVSQApIgdwUoK+Uc8w6/zKhynlnOS0k1zDj8F2uyut6ibguabnlrrOkgBSQAilVwPmX2a3qHTspD5ft5uiq120PgZRKo2JLASkgBequAPvtsmfE1TYLVzN1n/91z1lXSgEpIAVSrAC7VdEtwBnNXEhjDxFXdsVSQApIgbwowMlWvrarAgiqeZFUmUgBKSAFAH+pb+khBaSAFJACeVDA1VRdnIcslYUUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFSlsBTlDN1Q84u/0AAP6s97UtOecb5fLNDJzhqaZJrTkR+O723NpEs7yx9v511aX753BV1NoELvbHMf8KUkAKSIFaKeDDhovcXRi52q1VFUnOuuuDNesJXmJdoVUdQKtL926ZWW7a3w9t1/UZQ/nquBSQAiWugA9Wzv5/NwDWJt+1szVxWrsN7PpdHwMY4y0GyG5EnB5vsrdooquxvgWAY+cZDgDwqZ13lPOVcsnwLwDMs7VlrhPGCUqeBcB70DhhOANXcn0NwHhbC+ak0dmGh/pg5cQnI+w1/nIxLOs/bDqfg/dkaG0n4+Y1LDfnVWUQWK0QiqSAFKidAg6srJlyiRAuCkiwcr17TsrMQDhx4maGjQFwhVEeO8JOLkLwct2mZZ4rwIGV8Jrr5cUlWhii0OJKrnvYY83tSrfcvR0Al9Fm4FImJgewuntsYl0cbpltXnu8zYt58qXAQMhyCRUGjph6025Hn9EmK5ICUkAK1KyA87HSz8qVQzm3KME61LuMq8+y5spzaJwObz8At9pJnd2pXNo5WmPlEjV0MURDFFocW+/yZ8zaLJc54XYr7+IlOYCVeXNWfhqXld7VXs+yOtcG82TevMfqyL25fDlD9BltsiIpIAWkQM0KuBqrfxbBOshL4Ce6WxDPS84rWBcBKPMzt9u1BSufnTP5u0Y41pyZxhAFK2f057DT+fZ4NBJYo4poXwpIgZwUyAWsdAXQb+lmwm8LYDPrCnjV+mC5Iimny4vWWKtzBVwE4BrvCekKuMTbd+tk0RXg3BAH5uAKOAzASzYf+kq/8cBKV8Bv7THmyRo6wzAAv7Hb9BtzLSoGgdUKoUgKSIHaKZALWDmn6PV2QTx2y6KbgEtZ+41XnMiZK5BGwcqnIRBZO+SnOc9jIJzZEMYaKRuv2CD1lE3j8iX32vP8xqt/2hVPa2q8og94sPXR8mXg11hZ1r9bvyv9qK7xiv5iriTL5+O9nU9XYLV/BEVSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACiSjwP8Dcf1W08Qz7A8AAAAASUVORK5CYII=)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xg-0Y2D6x7nC"
      },
      "source": [
        "# epoch = 128\n",
        "epoch = 25\n",
        "batch = 1\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraExtraDense_extraDropout(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AJh504dMas3k"
      },
      "source": [
        "epoch = 128\n",
        "batch = 2\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraExtraDense_extraDropout(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O5qN1gaD0ZKF"
      },
      "source": [
        "- removed last two 0.1 dropout layers\n",
        "- not as good"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6fNNkXThP1rh"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model2 = Sequential()\n",
        "make_model_extraExtraDense_extraDropout(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model2.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model2, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "chXVQJMie-gU"
      },
      "source": [
        "              precision    recall  f1-score   support\n",
        "\n",
        "           0       0.76      0.82      0.79       177\n",
        "           1       0.73      0.75      0.74       127\n",
        "           2       0.00      0.00      0.00        18\n",
        "\n",
        "    accuracy                           0.75       322\n",
        "   macro avg       0.50      0.52      0.51       322\n",
        "weighted avg       0.70      0.75      0.72       322\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAgAElEQVR4Ae2dCbwe0/nHf22oa6ki9iWy73u0RUO1tbeqVVSrlPJPLVW70nQhoqkoRe21Ve2pncZOhCARksgiq0SQRSSRRRD6/D+/65zkZLz3nvfevDPv+975nc/nycycmTkz88t9v/PMczZASQpIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSoAAXWB/AQgA8ADFmL+zkcwONrcX4lnbobgMmVdEO6FykgBdJR4OcAXgGwDMAcAEMB9C3BpY4AMBLAOiUoqxqKMABtq+FGdY9SQAqkq8BpAOYDOAjAhgDWBXAAgItKcNk/ALi1BOVUSxExsOblBVMt/1+6TymQigJfc17qIfWUvh6ASwG864zrzGPaA8DbAE53cKa3e7Tbdx6ATwCsdNc4BsC5CdC2BEAYeeAcBWAGgKUA3gTAEAAT859361zsCmCUCzFwyW2fngVwPoAXXDkMIWzudyaW/v7PCu7/RwD2BzAFwEIAvw/O+QaAFwEsdp79FQC+4vY/555luXvenwb6/A7AXAD/DvJ4Wht3jd6ujG0BvOeOCS6rVSkgBapJgX0BfBqArdC9DwDwEoAtAWwBYIQDF48lmHg+j6GnSyB9CGBTV1ASpMntEKz0lpcA6ODO3QZAF7cegnUzAIsAMMxAIP/MbTd3xxKs0wG0B8AYL7f/6vYlF/7+/+Tu//8c2G4H8FV3/RUAWrkT+wDY2V2X9z4JwClBoUmP1Zd/oXsZ8X48zP1pvOZEABsAeAzA3/wOLaWAFKhOBegR0pOqLxFSBKZP+wCY6TYICYLHe5zMZliB8GFKgjS5nQQrPcGfOCC6ImoXIVh93DbcTy+SxzARpAxB+HQCgEf9RmLp77+ZyydMCcdvBseNBkAvtlAiVO8LdhQCK732muCYJFi560EArwMYF3wNBKdoVQpIgWpSoBiPleD0niOfraP7xOd6IUgQuns6EZIgTW6HYOUphPYT7lP7EXct5odg5Wd1soXBnQD6u2sSrMe69eS5QXbtavL++YIgHHlfPjEE8Qu3QS/4YfcyondN73y4P9CdG1Zesfx3gv1cTV6TeYxp87r0XpWkgBSocgUYY2VM8OB6niPpse6d8FgZYw1TfWA9E8C9wcH0bAmU0OPlbn4yXxxAKwRrIY+V4YnQY00LrE+5T3V6tkz0WMPYbyGPNalPEqwbudDF9Q7CDHUoSQEpUOUKsOJpnvvcZZyPsdL9AAx2zzXQxVUZX2UlEEHCPKYkJJhXH1j3ArAAQAsAhPoDAVi3AnCga5nwZQCs/Br2+WXW8FgZS2XIgE3ECGRWEnHbV1Cl6bGy6RjjsV9y3jTbo4ZgZViFLx6fCumTzLsBwF3uhOsA3O1P1lIKSIHqVoCxVrZjpfdKOPAz3Ne0Mz54uasFZ60/133MMAkJqlAfWLn/SgfCae7T13usrKwiSNmZgKAkIDs7WUOPlVlsY8vYJ4/lMmxzmyZYdwfwhqv1ZwiAlXYhWI9zOvH+D63jxRNqxhcJQwXeS6X3Sl18awj3+FpIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaRAxgo0q7Evrb+FbP0trFPXnjKnQe/efUzWx5o3b87WHiVNX/5qi6J+b/X0tCvp/aiwFBQgVGt6nijreaKNmrFY5jRYsdJMZrUvl1L/7Ir9zbmmg6W+vMrLQoFi/5PzAF+BdfWLRVD9/MVCr73Uv8MvbbCl1fT+bdQE1lIrn2F5Autqb11gFViTL5TUwNrnFKuJmMCaIQhLfSmBVWAt9EJJAiav26mBdadTrSZiRYD1RjcK2/gCXGAXb3rbvos0uy+ztyF7xHH0MT92boFTlbXWCgisAqvAWnc8OR2wbmU1Xz89akWAlV2TCcgkWHdwY+LOCsDKITM5VREBy0GDXl5reKiAuhUQWAVWgTVjsG64ldV886yoFQFW/rA5NGQSrP8B0MONd+E91mvdYOoeBhx4h+NaKKWhgMAqsAqsWYN1a6vZ+XdRc2DkoELe+hVgQBKsHBDnMnccBxLyYOX4u+EgPxw6cqcC5SmrFAoIrAKrwFoGsO5yjtVErBEeK4fL5Cc+h7JkElidEJkvBFaBVWAtA1h3/b3VRKwRYO3mKrMIVBrncXsLwNYAFArIkq4Cq8AqsGYM1o22sZq+f4xaI8CaREfosX4/UXnFwc6V0lJAYBVYBdYygHW3P1tNxIoA6x1uQHJOzc6pczg1e5hCsLI1AAdn5xRFnPBR8dVQqVKvC6wCq8CaNVi3tZrdz4taEWAtNQ5UXqkUEFgFVoE1Y7B+dVur+c7AqAmspaJcGcoRWAVWgbUMYP3uBVYTMYG1DEAs1SUFVoFVYM0arNtZzfcGRU1gLRXlylCOwCqwCqxlAOuef7WaiAmsZQBiqS4psAqsAmvGYN14e6vZ+6KoCaylolwZyhFYBVaBtQxg3ediq4mYwFoGIJbqkgKrwCqwlgGs+15iNRETWEtFuTKUI7AKrAJr1mDdwWr2vyxqAmsZgFiqSwqsAqvAmjFYv7aD1Xz/H1ETWEtFuTKUI7AKrAJr1mBtYTU/uDJqAmsZgFiqSwqsAqvAWgawHnCV1URMYC0V5cpQjsAqsAqsZQDrgddaTcQE1jIAsVSXFFgFVoE1Y7BusqPV/OifURNYS0W5MpQjsAqsAmsZwPrj660mYgJrGYBYqktmAdab7xth895fYuOnvmM1PVeDjOu/u/geY9puj7Nq9+11zKW2eMmHNuaN2bV23lUPfeGcZBml2i4EmDTznp801zp3723tOnaxVu062rEnnWW83j4/PMRatGprrdt3sgMOPtxenPxebX6a95Isu5zTXT/w8FBr1769tW7TxgZcMMjKeS+pzNK6yY62/kE3RE1gLRXlylBOFmD93q8usZ0PG/QFsLbdp789/sJEm/Xu+2uA9ZFhr2cG0xDKSbikvT1y+iIb9vrbtdAkPLv06GM33vOEXXrD3cZ9tL1/8BP73YCLcwPWZR99aq1at7aJk6fbB8s/tm7duturYyeUDa5pgPXLm+5oGxx8Y9QE1jIAsVSXzAKshFf7/f74BbDe+8Sr9vVD/2Iz31mQS7CG4B4+4V3r0KW73XTvk2tA9NT+F9jRJ5y+Rl54Xlrr5fISn3luhO25196rQDpg4F+MVq77SQesLW3DQ26KmsBaKsqVoZxygfXgU66xK257utYzTYJ1waJlNnbybHv0+fHW66DzM/Ne04JUfeW+NPV9a9epq62/wYZ2ZL+T1wAovVjC9ro7/7tGfn3llWpfuUB2251D7Kijj1kF0htuusV+ffyJq7azvq+0wLrRoTdbzATW1UDsD2ACgHEAxgD4JoBTAHBa2oamAQD2jJx0LoAzChyzCYATCuR/IascYN1051Ns5Lg3bcu+p38BrFt863RrvsuptfkH/uZKmzprXpMGqwfh02NmWp+d+9odQ0esguiBPz3SDjvquFXb/tgsllkDzF8vL2Dd+LBbLGYC6+e42gXAiwDWc/TaHMC2ibm9k2Brlsxo4HZdYG0JYHwxZZUDrH0OvqC2MoueKm3lyk/trXfftx2/d/YXIBp6s2E8NI31LIBV3zX6nXKO/fac82tB2u/ks+3be+1vL09bmCuw5iIUsFlL2/hnt0StCLDe6Ka7Dn/rFwF4wzl39wGgk+XTOQCmAZgMYB+fWenLgwA8lLjJ3wL4xM2K+IzbtwzAxQDGAugL4E8ARjkQXgeAsyky3QzgYLe+vxNrNIDLATzs8glWivssgBkAeD2mOwGscF4zha4zlQOsSSiG8Azh2vfwwbXATR6f1nZ90Etj3+Ojphk9VZY9fOIc67nTLnbJP++0/oMut269v1Gbl8Z1iynTe5BZL5euWGktW7WySVNmrKq8Gj1mfJMKBTTbrJV97ef/jloRYN0dQO+EE7U3gHXcD/5CADSmzo45dPxaudla19axc0Wnu9jIgWwKgKsAfNtdLpyCllkG4NDgVjYL1v8N4AC37cFaA2C2E4O7OOVtCNYRzkumh/w+gHUBxDzWfu4/7RWsu9EXvMRSg+uuoaPs3fmL7ZNPPrW35y60X5976xrXDMF6yqC7bMK0d2tjrC+PnWF7/PJvaxxb6nsLyysGOKU85vZHnrf2nbtZ2w6da5tW0WNl+c2aNbPtWrSsjb0y/urzS3ntWFlZAzW83n0PPmJt27WrbR1w7oCBZYMq7ymNGCvBusnht0atCLCSB/X91n8M4DbHE3qrNJ8eA8Cv7KpIfAPsAeA8AHMBHFUgFPApgPBN8RMALzuv9h0AZ7sn9WDtCWBY8PQ/TICVcV2fJgHYPiK2P7Z2mZXHGgKsUtdjsMnT/hB0eV5PC6ybHXG7xcyx45VVThBAhyiZ6gMrv6B/4U64Ilhn1g3BF3GyzIre5mc8HyzpsTIU4BO90XkAdnAZ/LSnMRUL1rDyirEWCl2f2K74zxcC6+oOC3kCZ+xZ8wzT8NlTAWvzVrbZkbdHbS09VjpcjLH60GLVgrUDgHYBtQYC4MO8HnzGc3cIVgaWCdb1ATCUQDAmwcp9DAUQlkx07cNQQCGwNgcwyx1f70JgFVgLQTaES57X0wJr81/eYTFbC7DyS5kV6WFrpKoNBfQBwHjnRFcjdy8Axj1PcrVwYeVVCDsCeDqAFwDcVACsPJZxV9b0sfLqmiBuQggXAivPud2BuiIqryr18z+8r0KAyWtenmEaPnsaYF2neWvb4ui7otZIsO7rGLRFCBkAXRKVV6zsDkOSicPzsUlvloluPSvGTnXba72QxyqPtdDLI4RLntdTA+uv7rItIlYEWFmRPQfASgBvAzjGNafiFy7b0dPoiPnE8ACdODa32s9n5nlJkFIkesMMBYQu/lrpIrAKrAJrtqNbrbN5a9vymLujVgRY1+q3r5NTVEBgFVgF1uzButWxQyxmAmuK4Eu7aIFVYBVYswXrupu3sW363RM1gTVt+qVYvsAqsAqsZQDrr++xbSImsKYIvrSLFlgFVoE1Y7Bu0ca2Pe7eqAmsadMvxfIFVoFVYM0erNsdf5/FTGBNEXxpFy2wCqwCa/Zg3f7E+y1mAmva9EuxfIFVYBVYswfrDr95wGImsKYIvrSLFlgFVoE1W7B+Zcu21uKkB6MmsKZNvxTLF1gFVoE1e7C2PPlhi5nAmiL40i5aYBVYBdYygPWUh61lxATWtOmXYvkCq8AqsGYP1lanPmIxE1hTBF/aRQusAqvAmjFYt2prrU/7b9QE1rTpl2L5AqvAKrBmC9b1tmpnbc8YGjWBNUXwpV20wCqwCqzZg7XdmUMtZgJr2vRLsXyBVWAVWLMHa/uzHrWYCawpgi/togVWgVVgzRasNVu3s07nPBY1gTVt+qVYvsAqsAqs2YO18+8fs5gJrCmCL+2iBVaBVWDNHqxd+j9uMRNY06ZfiuULrAKrwJoxWLdpZ13/8HjUigDrjQDmu0lDPSU2A/AEgKluuanbwfnyLndzYo0D0NufoGUKCgisAqvAmi1Y19+mvXX/05NRKwKsuztAjg/QMBjA2W6bywvd+v4AhroJSXcG8HJwjlZLrYDAKrAKrNmDtcefn7SYFQFW4qBlwmPlDKzbOE5wyW2mawH8zK1zER4XZGu1JAoIrAKrwJo9WHue+5TFrJFgXRyAgZ//fvthAH2DfU8B2CnY1mopFRBYBVaBNWOwbtveep33VNQAzHRwfcUt+xX47Sc9Vg9Sf+gityKwekWyWAqsAqvAmi1YN9i2vfU5/+moNdJjDT/xFQrIAqKFriGwCqwCa/Zg3WngMxazRoL1okTlFSuzmL6fqLwa6fK1SEMBgVVgFVizB+vXL3jGYlYEWO8AMAfASgBvAzgGQHMAjJ+yudWTANj8ionx1isBTAfwuuKrTpW0Fp269rRCP6w85u06eJjJPtdgxcq6YZOnfb1797FS//Y23K69fXPQs1ErAqylvjWVVyoFBNbFq14sgurqF0ue4Fnfs6YD1g6281+HRU1gLRXlylCOwCqwFnqh1AebPO1LC6y7XDjMYiawlgGIpbqkwCqwCqx1hz3SAuuug5+zmAmspaJcGcoRWAVWgTVbsG60fQfr+7fhURNYywDEUl1SYBVYBdbswbrbxc9bzATWUlGuDOUIrAKrwJo9WHe/5HmLmcBaBiCW6pICq8AqsGYM1h062B6XvhA1gbVUlCtDOQKrwCqwZgvWr+7Q0b5z2YioCaxlAGKpLimwCqwCa/Zg/e7lIyxmAmupKFeGcgRWgVVgzR6se/7jRYuZwFoGIJbqkgKrwCqwZgvWjVt0tL2vfClqlQzWf7h5XjjXSyErFZ+qthyBVWAVWLMH675XvWQxq2Sw/hJAfVa1QCzVjQusAqvAmj1Y97v6ZYtZJYM1yZ8Nkhl53xZYBVaBNXuw7n/NyxazagDrLgAmAnjLgbQHgKvyDlU+v8AqsAqs2YL1ay062gHXjoxaNYCVU73uAOC1AKbhlLFBdr5WBVaBVWDNGKw7drIfXjcqatUCVhIzBOvYfCG08NMKrAKrwJo9WA/85yiLWTWA9T8AdgXwKoB1AZwB4M7CqMlXrsAqsAqs2YJ1kx072UE3vBK1agDr5gBuAzAPwHsAbnVzw+SLogWeVmAVWAXW7MH6kxtHW8yqAawFkKIsKiCwCqwCa7Zg3bRlJzv4ptFRKwKspwKYAID1RZxYsAZAKwCsU5oG4C4AX0mTdK0BPOS81fkAHgDAvNwngVVgFVizB+uhN79qMYuAdTsAbwJY30HsbgBHAeDyMJd3DYDj04TcSwCOALCOs184qqd5zaooW2AVWAXW7MF62L9es5gVAdbZbnprcu1hAPsAWOAYR/6wmeljaYJoXIHC1SpAoYBVM7Ryuu9CgMlrXp4mDKzvWdOY82qzlp3s57e8FjUAMx1cX3HLfgmOnQxgmfsSZx0S65IYAvCJTUxTaVa6mSP6hQDOBtASwI4AzgIwyF89z0t5rPJYC7086oNNnvalAtZWne3wf4+JWsRj3RTA0wC2cC2d7gfAL/FMwMoYxAwXi+B6aMzPfRJYBVaBNdtQwGatOtsvbh0TtQhYDwFwQwCwIwFcnXUoILi+VkMFsgbr85PmWufuva1dxy7Wql1HO/aks2o/x/f54SHWolVba92+kx1w8OH24uT31vhM56d62lYIMGnm/f3JqTZ9/jKb8d4yu/SpabWhiOufn2nzl3xkU+YtrbXTh4wrS4iinF7pAw8PtXbt21vrNm1swAWDrJz3kobH2rxVZ/vl7WOjFgHrN12LAI5/8iUA/wJwEoAhicqrE8LfexrrXQEcCoBk95bGdaqqzKzBOnL6Ihv2+tu1kCQ8u/ToYzfe84RdesPdxn20vX/wE/vdgItTB2kS1GlCNFn24TeOqoXqdy4ZbrtdNMxGzlxoh1z3shGs/3hmellgGt5juWC27KNPrVXr1jZx8nT7YPnH1q1bd3t17ISywTUtsB51xziLWQSs5Mx5AN5wcdR/A1jPtXYa6UIChCzzUkt/BvCM6yBwE4C5ANgbK/cpa7CGMBs+4V3r0KW73XTvk2tA9NT+F9jRJ5y+Rl54XlrrIVjSXu9//wR7cOycVQC9bvibdsWz03MP1meeG2F77rX3KpAOGPgXo5UL9GmB9eg7xlnMigBr2fn1OoAvA/AtAbYC8ETZ76oCbqAcYH1p6vvWrlNXW3+DDe3IfievAVB6sYTtdXf+d438tGAalps2TMPyf3b9SJv1/nLb9/IXjF7r6+98YENGv10L1ncXr7Cp85faQ+Pm2D6XvbAKvuH5aa+XC2S33TnEjjr6mFUgveGmW+zXx5+4ajvr+0oDrJu37mzH3vV61KoBrHSNmUYD2NjFJOhCr21iU4diE2vv2COCA8HsBqBQ7KM5gDHO6FW/E2yn0oOiHGD1MHt6zEzrs3Nfu2PoiFUQPfCnR9phRx23atsfm8UybVgly//L0Mk2ac4Se+2txXbva+/YXaNm2/evGGF9Lxpm3xo8zG4eMasWrsnzstjOGmD+enkB6//dPd5iVg1g5dirmwA4DsBUBzeGBNY2NQSs7A1xvbsgm33F2ped6waLKfYe2Ui4wamcYCUs+51yjv32nPNrQdrv5LPt23vtby9PW5gLsIaA/NeLs+yix6es4Z0edM1LtXHY8Lis1j3osl7mIRRAj7XfkPFRqwawhsAh1LqHGWuxXgisbQA86rzj4QA6AujpBtnmADD0SNmHd4Vbv6iO64dgvRnAwcFx/rp7AOA1HgQwBQC3n3XxY3rkbDTMGsM6U9ZgfXzUNKOnSqgOnzjHeu60i13yzzut/6DLrVvvb9TmZeGdFrpGVtDy19n/ihG1IP3x1S/ZzAXLbe/LnrcDrnxxFVwve2qaPTFx3qptf14Wy6yB6q+3dMVKa9mqlU2aMmNV5dXoMeObVChgi9Zd7Lj/TIhaJYO1N4D6rE7gFLnDAy48/CkA7VwGm0SwES8T+/Fe4dYb6rHWB9blbuAFFk2wfgBgexdTfhFAX3fNgouswXr7I89b+87drG2HzrVNq+ixEnLNmjWz7Vq0rI29Mv7q8wsBMK28LIAVXmPM7MW1Ta3YtOqkO8fWAnTo+Lk2bf6y2hjr8KkL1gBteG7a6x505Vje9+Aj1rZdu9rWAecOGFg2qPLZ04ixbtGmi51w78SoVTJY2RKgLvPAKwicIjOTYN0o8ER9rHSSKystsPL5fCJYw0o5Nhhmb4xkYte42m5yW2+7Q1k+u9OC49qUmzasqqn8cgC1Eq+ZFlh/c+9Ei1klgzUJlFJvJ8HKirE5dVxkbcDK2Czb4DKxdcMnbp0g5eALPiW36SHzunWmrD3WtQFf2udWE/jSvtdKhFw57ikNsG7ZpouddN+kqAmsa2JrBAB2N2NifJMTFzKFYGXt/yyXX9cijLH+AQDHO2D6EQBz60mQJrcF1gb02EobVtVUfjkgVonXTAuspz4wyWKWZ7D+D8DbgZ3m4p2svGKbWc4M+ycHwRCszLrdtQwopvKK7W459CHLJGC9p5wEaXJbYBVYG1X5VYmQK8c9pQHWrdp2sdMefCNqeQarY2b1LhQKWD0GQTV5lGnfazkgVonXTAusZzz0hsWsGsDKT3JW4njvsQWAb1QvDkt35wKrwFoI0pUIuXLcU1pgPfPhNyxm1QBW1o5fCcDX0HMsw1Glw1P1liSwCqwCa7bDBm7dtoud88jkqFUDWDntNRO7k/rkxw3w27lcCqwCq8CaPVh//9/JFrNqACv76DcD4AHLfvshZHMJVT60wCqwCqzZgnWbdl3tD0OnRK0awHq46/bJGvwLAEwOmkTlFqoC62qoso1sIcDkNa8c8cxKvGYaMVaC9Y+PTolaNYCVDGGf/RMB/IaOWq5pGjy8PNbVcM0rRAs9dyVCrhz3lAZYt23X1c57fGrUqgGsbAVQyALE5HNVYBVYBdZsQwEE64AnpkatGsDKga45BTaXHDbwUzdfTD5pGjy1wCqwCqwZg7V9Vxv45NSoVQNYA5TUrnLEKz82anJfrrYFVoFVYM0WrNu172qDnpoWtWoEK+FJ7zX3SWAVWAXW7MF64dPTLGZFgJWD93PuPo69zDb6uwDYzI1wxy9zjnTHNvupJfbh93aG66f/WGpXq6KCBVaBVWDNHqyDn5luMSsCrJzy+liHG07dRNAOBnC2y+PSD96UCpU4S6u3/gDY/KomlStVWaECq8AqsGYL1u3bd7W/PTs9ahGwfg3AmwVmCGFT0m0chrjkdiqJHQP+lkrJTaBQgVVgFVgzBmuHrvb352ZEDcBMB9faQekBcIB6nzjdEydJ5ewi7OzEOqMNASz2BzjohtvBrrVb9RPscYoSpQIKCKwCq8CaLVh36NDVLhs+I2oRj3Un17qJ0z8xXQbg/ARYmb/o892l/dd3YeUgLJxw7wgABwVW2qtVYWkCq8AqsGYP1sufn2Exi4B1a+fReursBuAR9+mfeijAg5VTXXu70a1zmfsksAqsAmu2YG3RoZtd+cKbUYuAleziDM0dHMQ44wgHzaeFlVeszCp54tgAbA1werDkut8u+QWrrUCBVWAVWDMGa8dudvWIN6NWBFgZZ2X8lZ2f7ndNqzjtE2eKZnOrJ13zq5JjiRP7cXBr3yIgXPpBr0t+0WoqUGAVWAXW7MF67YszLWZFgLVsqPGhgLLdQKVfWGAVWAXWbMG6Y8dudt1LM6NWyWDVmKsRsgusAqvAmj1Ybxg5y2JWyWBl9y6lehQQWAVWgTVbsLbs2M1uGjkrapUM1nqQol1UQGAVWAXWjMHaqZvdPOqtqAmsVcxogVVgFVizB+u/Rr1lMRNYBVbj1CbVboUAk9e8cozWX4nXTGMGgVadutmtr8yOmsAqsFY9VDXn1bA15vyqRMiV457SAutto2dbzARWgVVgHbwmmKrd0y0HxCrxmumAtbvd/urbURNYBVaBVWC1SgTj2t5TGmBt3am73fXaO1ETWAVWgVVgFViL5ADBevdr70RNYC1S0Eo8rFuP3vbW+x/J3v/Iln70mcxpsLaeXlM5PxWPtXN3GzLm3agJrJVIzCLvSWBd/VIRWFe/WJoKGNf2OdIAa5vO3e2eMe9GTWAtEmKVeJjAKrAWeqGsLZCayvlpgfW+sXMsZgJrJRKzyHsSWAVWgTXbDgL0WO8fNydqAmuREKvEwwRWgVVgzRasbTt3twfHzY2awFqJxCzyngRWgVVgzRqsPezh1+dFTWAtEmKVeJjAKrAKrNmD9ZHx8yxmAmslErPIexJYBVaBNVuwtuvcw4aOnx81gbVIiFXiYQKrwCqwZgzWLj3s0Qnzo1YkWJsB4ID+Dzu+tALwMoBpAO4C8JVK5E6TvyeBVWAVWLMH6+MT37OYFQlWTpZ6ewDWuwEc5sB1DYDjmzzEKvEBBVaBVQhxFOgAABhgSURBVGDNFqztu/SwJye9F7UiwLq9m5H1uw6sXwKwAMA6jjW7AHisErnT5O9JYBVYBdbswfrUpAUWsyLA+h8AfQDs4cC6uQsBeG7tAGC839AyQwUEVoFVYM0erE+/scBiBmCmg+srbtkvQMMPAFzltgXWQJiKWBVYBVaBNVuwdujSw4ZNfj9qEY91EIC3HXznAvgQwG0KBVQEVgGBVWAVWLMGa097bvLCqEXAGhLEe6zMG5KovDohPFDrGSkgsAqsAmvGYO3a04ZPWRi1RoK1NYCRLtZKyK6XEUp0mVABgVVgFVizBWvHrj1txNRFUWsAWMOftNYrQQGBVWAVWLMH64tTF1nMBNZKIGQj70FgFVgF1jKAddoiezFiAmsjoVYJpwmsAqvAmj1YX5q22GImsFYCIRt5DwKrwCqwZg/WkdMXW8wE1kZCrRJOE1gFVoG1DGCdsdhGRkxgrQRCNvIeBFaBVWDNFqyduva0UTM+iJrA2kioVcJpAqvAKrBmDNZuPW30mx9ETWCtBEI28h4EVoFVYC0DWGd+YKMjJrA2EmqVcJrAKrAKrFmDtZe9OnNJ1ATWSiBkI+8ha7Ae+vMjrfnmW1j7jp3trfc/h9opZ/W3rbbe1jp37V5rN995/6p9/pgsloUAk1Xee4uXW5+dvm5du3W3jp0629m//6Nlde1C11mxsm7YpL3vgYeHWrv27a11mzY24IJBlvb16iu/d+8+1sifVp2nderWy16btSRqAmudElb+jqzBOuShJ+yRp1/8Alj7nzeoLDANgV0IMFnlLVnxqc1Z8EEtTBcu/ch2+vo37KlhL5QNrvXBJs19yz761Fq1bm0TJ0+3D5Z/bN26dbdXx04oG1zTAGvnbr1szFtLoiawVj4/67zDrMFKkL3w2hsC60ef1QnNeQuXWo+evezp50bUeUzawE8TnvWV/cxzI2zPvfZeBdIBA/9itPrOSXNfWmAd+9ZSi5nAWie2Kn9HpYB1+x1aWMfOXY2hgnHT55TFe00bVrHyFy//xLp172EbbrihnXL6mWWDKu8zTVjVV/Ztdw6xo44+ZtX1b7jpFvv18Seu2q7v3DT2pQXWcbOXWsyaMlg/AzDGTW/wEIBNHCq3BcBpEWJpWR0H/AhA5wL7+rvr8Zr+2lz/bYFjS5JVCWAdPWmWvTl/uc1870P7zaln1cI1/ETPaj0Gvqz2z577vu22+x728uixZYNrGpAqpsy8gPX12UstZk0ZrCEY/wWA4GtICs8Pz7sZwMFhRoH1us4tcGhtlp9ArK79BfMrAawhOJNhgnBf2utZgbOY6/T/47k2cNDg3IE1F6GA7r1s/NvLopYXsB4XzDHTMpikawMAnHJ2IoD73JzeOzmKEY4XABgL4CUAWwHYFcBCAG8677RNQeIBHqzhtXjoGQDOdec8C+BS9x9wOgBuX+gGup0CYLc6yl6VXQlgHTXhzVWf/n8aONgO+PEhq7bThmlYfjHAS+uYGbPnGj1Vlj9/0TLbZde+dve9D+QOrEtXrLSWrVrZpCkzVlVejR4zvkmFArp072UT3lkWtTyAtZmb8mBfR6QQdgTdtS6/K4BPAXiwsqnGAW7fYAB/cOsN8VjDa/H0JFj9hGLcR7Be7K6xP4An3XpywUnJaico2277HTKF2A8POsS22GprW2eddWzrbbazwZddbT8+5GfWoVOX2hjrnvt+30LQhuBLez0taBZT7oujXrPuPXpal67drFPnLkaPtZjz0jqmmM/2tI6578FHrG27drWtA84dMLBsUOXzpRFjJVgnvrMsak0ZrD7O+R6A5wAQsEwh7O4H8B2Xz8WrAVg/BsC5vpl+CuB6t15KsH7blckFwfott03veFqwr+BqOTzWtAHZ2PLTglQ1lpsWNKut3LTAOund5RazpgxW/znOz/3hQSVSsWD15xNqjKkSqEwNAev2LszgTq31esNQgPeOuZ9g9ducY5zT59abBFb1vCoE/moDYFr3mxZY35iz3GKWB7ASTr0AzALASqIQrGcCuNrRizX9KwO41QXWfwA4ul7irY6xruumtG3uJgZjrFZgdb22GuudFjqvEGDympcWqKqt3LTAOnnOcotZBKw7AHjGOV0TAJzseLIZgCcATHXLTSOcKcvuEIy8ATa5OiIB1g1d0ytWXt3rKqTaubsNzw89Vn6u8/jXAMQqr1gUm1tNd+EIersCq8Caagy22gCY1v2mBta5y21yxCJg3QZAb8eZrwJgZTUdO9blnO3yuWRldlUmxl1r3J0Tkqzt/0q1PIlCAQoFFPLK0wJVtZWbBli79uhlU+d9GLUIWJOIeQDAXgAmAyB0mbjkdlUmvi1Yw84mVeMA7FdNTyGwCqwCa90DzqQF1mnzPrSYuTqS2tY7jjFszVMoMTT5FoCNASwODmDFebgd7NJqqgoIrAKrwFoGsM7/0KZFrEiPdSMAowEc5ECRBOmiVAGiwgsrILAKrAJr1mDtbdPnr4haEWBlxfZjAE4Lft1NJhQQPFP1rQqsAqvAmi1Y+Zt7870VUYuAlZ/5t7ielyF4LkpUXrEySylrBQRWgVVgLQNYF6ywNyMWAWtfAOzZyXodDtREY29LNst8yjW3Ys9LNr9SyloBgVVgFVizB+vMBR9ZzCJgzRoVul5DFBBYBVaBNXuwFuqgkswTWBtCsgo7VmAVWAXWjMHas7fNXvhR1ATWCoNlQ25HYBVYBdZygPVjm72wfhNYG0KyCjtWYBVYBdZswdq9Z297e9HHURNYKwyWDbkdgVVgFVizB+u7iz+2mAmsDSFZhR0rsAqsAmv2YJ2z+BOLmcBaYbBsyO0IrAKrwFoGsH7wic2JmMDaEJJV2LECq8AqsGYL1h69etu8JZ9ETWCtMFg25HYEVoFVYM0erPOXrLSYCawNIVmFHSuwCqwCaxnAunSlzY+YwFphsGzI7QisAqvAmj1Y31u60mImsDaEZBV2rMAqsAqsWYO1j72/7NOoCawVBsuG3I7AKrAKrNmCtWevPrZw+adRE1gbQrIKO1ZgFVgFVoG1wrBU/bcjsAqsAmv2YF304acWM3msVcxXgVVgFVgzBmvvPvbBis+iJrAKrJYcS7IatwsBJq951TZNdVr3m8Ysrb1697ElKz6LmsAqsAqsH31mTQnCaYGq2spNDawffWZLIiawCqwCq8Bq1QbNYu43LbAu+/h/FjOBVWAVWAVWgbVIDjAUsPzj/0WtCLDuC4DTXU8LZmYt8i50WKoKqPJKlVeFQhjFeHN5OCYtj3X5J/+zmEXA2gzAdACtAXwFwFgAnVOFhQovXgGBVWAVWLNtFUCP9cNPLGoRsO4C4LHgl34OAJpShSjwnvsPfKWMy5llvHY5n7vQtaUF4HWpBC34+yh1erTIv/fxieP6BTdyMIDrg+0jAFwRbGtVCtT+kCTD5woQKkqfKyAt6v5LEFjr1kZ7nAL6Aa3+U5AW0mK1AnWvKRRQtzba4xQQTFb/KUgLabFagbrX1gEwA0CroPKqS92Ha08eFQhjR3l8/vCZpcVqNaTFai0Kre0PYIprHdC/0AHKkwJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpEAWCnwpcZHkdmK3NqWAFJACayqwPoADXFZ3AAeuuTt3WyFEqcfGAMK83AkSPLDXoQbAukG+VqWAFEgowAEm/gzgNWftE/vzunkigIkAngHwCwDb5FWIxHPzxfuI6+75ncQ+bUoBKRAosCuAuQCGB3lsIJ2n5L0xPvNBAG4HsAGAnwK4GsCvAWydJ0EKPGsHAE8DYHfP/wMwFcDeBY5TlhTIrQIeJFwyHNAVwLUAHnQ9TShMXkDiteAzbwvgNgDDgr8MgvYaACcD2CrIz9NqHwD/AXBB8NCHuQb0+wV5WpUCuVRgMwBt3ZOzZwmBMcBtbwjgJgAPA9gdwD0AtsuRSqcAuMs9+/MAwt42PwNwKYBNcqSHf9Q9AJzmtPkvgB2DuPORAGYD2DzI8+dpKQVyoQArHQjR8wH8GMAY96nL/vMcIo37OcAvP31fBPCDXKjy+UP+0HnrW7pn7gvgTgC/DzT4arCel1XG3J8AsL174FsBXAWgRSCA4s+BGFrNpwJ7ArgQwA0ATncSsAKLXuo/gzAAPRCm8DPZZTWJRfhcfKH8A8D7gYfO8Mi3ANBDO8M9cXhOkxAh8hAE5kvuy4WtI5jWA3CzM3quSlIg1wp8OXj6TgBudB4aY6tMbD7DGvB/N2GY8jmTcPzR548PHwa5A4D3WgnXnQPYukOb9MLr4z/32SJiBIDvOqjy4QlXVu75v50mLYgeTgrUpYD/sfCHcDeALVyc9e/uU5egZWJLgJ3celNdfM09mNeELSHolTFtCuBiAP/KUcWde/RaWPp1vljowf/OvYh+BeBxF3smVJm8fm5TCymQLwX8D2AfANcB4HQU9EpZu02gEiSMueZhrEnGUZ9zUAgroR5y+fzLYD5bR1Arevlev6b8V8M46p/ci9U/5yEALgHACj2mowG8AIAVWXnQxD22FlJgTQXCnjH0VN8GwFHR2RKAFVj0XBlHZe+iy4OWAmuW0nS2+GJhZV1v9wlLb8y3juBTsoKGoRAmeq4+HOCymuyCbVNHAzjUvVzZMcInhkn4t3GCy2Db1W/4nVpKgbwpwEqHo1z7VD77152X6nVo4yqr6LkSImwJ0JQTG7Fz0jrGleltfdO9WH4bwJW13h+7lgBNWYvw2QjVec4zZT7jqJzy+bjgIL6AOPXzqUGeVqVALhVo5z7zGUvl1BLNAbzhYOsFodfKCgh2Z2XcLKzc8sc0heX3AYwDcCaA89wUxmw2RU+dLxZ+6rYEwDABP4dDL7YpPH9dz0CojnLN7BYAYGsRpm8DeDXwUvmVw7a9Pg7vDtNCCuRTAdZosxJmoKuI4Q+HfbzZLtP/eOixXdnE5WHFy27uGem5XuYqZTxc+fz3u/mN8jJOAv82WHHJxv1M7EG1JIArO4e86zqLcN6nvdxxWkiBXCqQrFTo5jwSfs6xcqonAFbUsJF3DwAcRIO1vb6NYh5E4wuGPaioCVsI0Ftn77I89TDj/7Pvmuv/ZpJwbQ3gly4un4e/Cz2jFCiogP+BsNaWn7z0zjiICD9t2U31bNfMiifzWEKVozfxk7gpJq+Hf7Zwm3Bla4hzXfdMf0welqEOfN6w5QPhyk4SDJ8wJY912VpIgXwpwMEwJrjKBzbqvsh1RWScld0z/+hgy/aq/BE1xXhi6H0nu1qGoKBWHFCE8ec8JFZQ+uf3S//c3PZ5RwD40OnSVOPu/rm1lAJRBVirzVpvfsLRI2NlFRt50zNj0yHmN1Xv1IvDAWaOdUP8/dx98jOeGCYPEObRo89DYpddxpk5Whmb23FAmVAHahDCNflCyoNGekYp8AUF+PlPYzMq1va+7NqoshLiTQB/cwOsfOHEJpTBAZivcC8QPvN8ABu550s2J0tCpQnJUPBRGEdm6wdWXrIyipWXhZLXxXuqfrvQscqTAk1aAVZQPQCgl3tKDhzCwYiZuI813h3ddlNdEKDPAjgcAEel4niqI10Tq6b6zMU+l4cj/wYmAxjimuDlbSDzYvXScVKgts//SjfMH+XwngbbJz4FYFqOhv7j5y17TrFCjp++DI2wCyYr8pjopfFFk6fke98x7sy/DQ71xwo7torg4DJMjDEnPXq3SwspkA8FOAITe1ExsZcMm8ywC+KKRF9/eiPsjsjum3lJ7Gm20DUlY4N/ps6u1xA/gfmyycswd4yn++ZU/Dvg2LqsqGPsnUNEcthIhofY5IydJ/Kii/uz0EIKrFaAFQr83Oe4qewxxcFUvNfBXkVsJpOnYdz8J65XiO1Q2cCfsUQOosKpRJg4HgLHnW2KLSDcI35hwdYfy9x4sre4yRDZ358dRjhfFT1ZzuHFAc3zPivvF8RTRn4UYHxskmuszcEyFrkWAFTAf/6zL/cnOaj5T/6vn+SalXHgbsKVIYBzXEUWu2PmNTEEQriymy4TW4b8xA1Q7Xtc+b+d5EvKnaKFFGi6CvCzlt7pMe4R2SaVIwxxYIyzEo/Nz+E8zZp5PIAnXUsATt/NkZiY6LmyKy+bmjHemhdw+OfkMzMxnro8CAswnsr2y6zc4wtISQrkVgGOg8n+7Uz84bBbKuHJPv9zHHDZ2J0VN/4H5X9g7rQmu6A3xrar/NRnHJXxQxo/dQmOvDT+938bXHLkLoLTD5ryVzdkJGefZWJoJC+z77pH1kIKfFEB1mazdpvjibIDAKcOYViAlRGcSZQeG+dl4mdeHpJ/aXDJCRDZZpcxZ9986DcufpgHLfwzek34N3Kva7vMCSLZppmJA5l/4Kb2dllaSIF8K8DeQayQ4SDNnNedPWg4hipr/NlllR6bHw3f/8CammLfA8DPfp98bJDDI74VTIrIUAibW+VllKqw5xi1YMjI965jrzsOuOPhOti1JPEaaikFpIADaCgEe1qxGY1vVhPua2rrbGL2GYB+wYP5NppsKcF4Myuv6NmziVUeEtumcpxU/1Jlq5H73Niy/vmHuo4SPgzA/Kb68vXPrKUUaJQCBAr7exMmfhSiRhVUZSex+RRbQ/jR7em1eriyMo+eu58osMoerdG3S5jSU2UIgInTUrPVCL9omPj3wb8T9sxTkgJSoA4FCBJ2WWVc9YA6jmnK2ZxBlnD1czDxWRlTfTSASVN+fj5bcvwDhj9mudAQXz6MtzP+zspNduult8/5zVRh1dT/MvR8a6UA4ep/JHn8rPNwZWXdD1xFDVtK5CGxPTO7KrPS7g+Bh/4LAK+7wcvZHI+N/zkgDQcz50A89Fq9F5sHnfSMUkAKNEIBwvV/btxQX1nTiGKq6hTGjtkKhMMicroZgpPdVX3iaP9sy8sBzH1iF9YpORwjwT+/llJACjRQAbbT9DXeDTy16g5n21zOnvqgu3N+qZwBoL/b9rFmApetIthWlYk90fLUldc9thZSQApIgeIUYKyUY8z6+DKhynFnOSwk5zBj913OyutbifgmacWVrqOkgBSQAjlVwMeX2azqOTcoD6ftZu+qJ1wLgZxKo8eWAlJACjReAbbbZcuIP7sivGfqP/8bX7LOlAJSQArkWAE2q2JYgCOa+ZTHFiL+2bWUAlJACpREAQ62stTNCiColkRSFSIFpIAUAMKpvqWHFJACUkAKlEAB76n6ZQmKVBFSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUaNoKcIBqzn7A0e2HAAhHvW/ok3O8UU7fzMQRnuob1JoDge/qjm3IYmbQ1z48r6788BjOitqQxMn+2OdfSQpIASnQIAVC2HCSu9MSZ/u5qhLZBTdDsBY8IMhsLLTqAmhd+cEla6ebDrdj6429x1i52i8FpEATVyAEK0f/vwoAvcnhbrQmDmvXzM3fNQrAuGAyQDYj4vB4k4NJE73H+iwA9p1n2hfAq27cUY5XyinD5wJ4x3nLnCeMA5TcA4DXoHHAcCbO5Po4gAnOC+ag0YW6h4Zg5cAno9054XQxfNa/u3zeB6/J1MYNxs1z+NwcV5VJYHVCaCEFpEDDFPBgpWfKKUI4KSDByvnuOSgzE+HEgZuZ1gPAGUa57yA3uAjBy3mbFgehAA9Wwmt2UBanaGFKQoszufZ1+1q4mW65eTkATqPNxKlMrAiw+mus70Icfpptnnu4K4tl8qXARMhyChUm9ph62q0n79FlayEFpIAUqF8BH2NlnJUzh3JsUYL1meA0zj5Lz5XH0Dgc3t4ALnWDOvtDObVz0mPlFDUMMSRTElrsW+/L55LeLKc54Xrr4OSFRYCVZXNUfhqnld7Znc9n9aENlsmyeY0ViWtz+nKm5D26bC2kgBSQAvUr4D3W8CiC9eEgg5/ofkK8ILukYF0AoCYs3K03FKy8d47k7yvh6DkzjykJVo7oz26nc9z+5EJgTSqibSkgBYpSoBiwMhTAuKUfCb89gA1dKOAxF4PljKQcLi/psdYVCjgdwHnBHTIUcGaw7efJYijAhyH2KyIUcCCAh1w5jJV+FICVoYDD3D6WSQ+daQSAQ9w648aci4pJYHVCaCEFpEDDFCgGrBxT9C9uQjw2y2KYgFNZh5VXHMiZM5Amwcq7IRDpHfLTnMcxEc6sCKNHysorVkjd5fI4fck17riw8uqfbsbT+iqvGAMe6mK0fBmEHiuf9RIXd2Uc1VdeMV7MmWR5f7y2j+kKrO4/QQspIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBbJR4P8BQ+W0fz6bDCUAAAAASUVORK5CYII=)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7guV_oMi0YqA"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraExtraDense_extraDropout(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zhFh9dblI0Ap"
      },
      "source": [
        "- re-added drop layers, took away one dense layer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SIYhvH2PIw1k"
      },
      "source": [
        "epoch = 128\n",
        "batch = 8\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraExtraDense_extraDropout(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "history_conv = train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPWuA7qNa4VP"
      },
      "source": [
        "# Round 12\n",
        "- downsizing\n",
        "\n",
        "              precision    recall  f1-score   support\n",
        "\n",
        "           0       0.75      0.89      0.81       244\n",
        "           1       0.79      0.72      0.75       175\n",
        "           2       0.00      0.00      0.00        30\n",
        "\n",
        "    accuracy                           0.76       449\n",
        "   macro avg       0.51      0.54      0.52       449\n",
        "weighted avg       0.72      0.76      0.74       449\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAgAElEQVR4Ae2dB7gVxfnG31jitcSoiEqVeundJGrUlD/2GA0aYywJxkSNxsSa2AvB2BOj0dixiz0gBhuCDREEQXpv0jtSRDTzf97jzGXueu6dw71n95x79p3n+Zjd2d2Z3Zd7fjv7TQMUpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSIEiUGBHAC8BWAPg2VrczykAXqvF9cV06cEAphbTDelepIAUiEeBkwF8CGAdgEUABgM4KA9FnQZgJIDt8pBXXcjCAGhVF25U9ygFpEC8ClwIYCmAXgB2BrA9gGMA3JKHYq8E8Hge8qkrWYTAmpYXTF35/9J9SoFYFPi2raX+vJrcdwBwO4CF1rjNNIYfAvgEwEUWzqztnm6PXQfgcwCbbRlnALg2AtpmAAgjB5zeAGYB+BTAbAB0ATAw/V27zehAAKOsi4Ex910YBuCvAN6z+dCFsKc7GInd/f/Zu//jABwFYBqAlQAu9675LoD3Aay2Nft/AfimPf62fZb19nl/4enzFwCLATzmpfGylraM7jaPhgCW2XO8YrUpBaRAXVLgCABfeGDLdu99AIwAsBeA+gCGW3DxXIKJ1/Mc1nQJpA0AdrcZRUEa3ffBytryWgBt7LUNAHSw2z5Y9wCwCgDdDATyL+1+PXsuwToTQDkA+ni5f6M9Fo3c/V9t7/93FmxPAviWLX8jgOb2wh4A9rfl8t4nAzjfyzRaY3X532RfRrwfB3N3GcucBGAnAK8CuNUdUCwFpEDdVIA1QtakqguEFIHpwuEA5tgdQoLgcTVOJtOtQPgwREEa3Y+ClTXB4y0QbRaZyAer89v6x1mL5DkMBCldEC6cA+AVtxOJ3f1va9MJU8Lxe955owGwFpstEKovegeygZW19jLvnChYeWgggPEAPva+BrxLtCkFpEBdUiCXGivB6WqOfLa29hOf29kgQej2tCJEQRrd98HKSwjt1+2n9su2LKb7YOVndbSHQX8AV9gyCdbf2u3otV5yZjN6/3xBEI68LxfogjjV7rAWPMi+jFi7Zu38HXeivdZvvGL+C7zj3IyWyTT6tFkua68KUkAK1HEF6GOlT/CEap4jWmM9LFJjpY/VD9WB9RIAL3gns2ZLoPg1Xh7mJ/NtHrR8sGarsdI94ddY4wLrEPupzpotA2usvu83W401qk8UrLtY18UDFsJ0dShIASlQxxVgw9MS+7lLPx99pUcCuNk+V1/rV6V/lY1ABAnTGKKQYFp1YD0UwHIATQEQ6gM8sO4N4FjbM2EbAGz8euurYirVWOlLpcuAXcQIZDYScd81UMVZY2XXMfpjv2Fr0+yP6oOVbhW+eFzIpk807UEAT9sL7gPwjLtYsRSQAnVbAfpa2Y+VtVfCgZ/hrqWd/sE7bCs4W/257XyGUUhQherAyuN3WRDOsJ++rsbKxiqClIMJCEoCsr2V1a+xMol9bOn75LmM/T63cYL1EABTbKs/XQBstPPBerbVifd/YhUvHl8zvkjoKnC1VNZeqYvrDWEfX5EUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQAokrMC2ZeYbO9aX7VjfdOzSXWY16N69h5H1MPXq1WNvj7yGbb7VNKffWzUj7fJ6P8osBgUI1bKu58q6nmtmLdsosxps3GyMzGReLvn+2eX6m7NdB/NdvPJLQoFc/5PTAF+BdcuLRVD96sXCWnu+f4ff2GkvU9b9j0ETWPOtfIL5CaxbausCq8AafaHEBtYe55uygAmsCYIw30UJrAJrthdKFDBp3Y8NrPtdYMoCJrDmm3YJ5iewCqwCa9X+5HjAurcp+85FQRNYEwRhvosSWAVWgTVhsO68tyn73p+DJrDmm3YJ5iewCqwCa9Jg3ceU7f+XoAmsCYIw30UJrAKrwFoAsB5wmSkLmMCab9olmJ/AKrAKrAUA64GXm7KACawJgjDfRQmsAqvAmjBYd2lgyg66KmgCa75pl2B+AqvAKrAWAKwHX2PKAiawJgjCfBclsAqsAmvSYG1oyg65LmgCa75pl2B+AqvAKrAmDNZvNTRlP+obtABYmwAYCmASgIkA/mSxweVxuELwdBvvbtO5vhmXI+KSOVyevHuCmElfUQKrwCqwFgCsP77elAUsAFaurebgyNV2p9m11bh45qWWZIxvsttHARhsF5DkqsIfpI92CT6xwCqwCqxJg7WRKfu/G4IWAGuUElwpmKsHc8VdQpeBMfcZ7gXwS7vNyD/PS9ZmXhQQWAVWgbUAYO15oykLmF0pmKsOOzuzih99MwDzAOxqVwZ2p/HznyvtMgyKrAI8BMB+9piifCsgsAqsAmvCYN21sSk77Jag5Vhj5VLjXDq9l2WDA6lDxSq7IbA6RZKIBVaBVWAtAFgPv82UBSwHsG4P4FUAF3qs8D/x5QrwhEl0U2AVWAXWAoD1iL+bsoAFwMrP/EcB3B4Bxi2Rxis2ZjEcHWm8GmnTFcWhgMAqsAqsSYO1iSk76p9BC4D1IABc3YBdp8ZaY8t/PQD0n7K71RsA2P2KgSC+C8BMAOPlX7WqxBUJrAKrwJowWL/dxJQdfWfQAmCNCwnKNx8KCKwCq8CaNFibmrKf3BU0gTUfhCtQHgKrwCqwFgCsx9xtygImsBYIivkoVmAVWAXWAoD12HtNWcAE1nwQrkB5CKwCq8CaMFh329eUHXd/0ATWAkExH8UKrAKrwFoAsP7sAVMWMIE1H4QrUB5JgLXV4VeYYSOnmkkzF5qJMxaai2561pR1PdecfPEDmf0vv/zSHHjyTZk0pve+rJ8ZO2V+hfH4d0/8W8VxnhOHZQNMnGnvjp1m9v/+IaZVeVvTuk07c1XfWwzLO/rY4027Dp0z1qhJ00wc531ky7uQy10PGDTYtC4vNy1atjR9rr/BFPJeYlmldbd9zY69HgyawFogKOaj2CTA2qznZWb/k27IwHDPAy800+YsMV17/dV0+Vkf0+nY68xbo6ZVAqsPzR4nXG9mzlsaC0j9cridDTBxpo0YP8sMfGN4ptyPZy81zVq0Mq++O6bSfZzx+z+a8/9yVaW0OO/J5V0omK377AvTvEULM2nqTLNm/SbTqVNnM2bcxILBNQ6wbrP7vmanEx4KmsCaD8IVKI8kwBoF2MCh48xRZ91ZAcvqwHrTA6+YG+9/peLcaF753HdQKVTc84ifmEeeHVQB0ZlLN5gGDRuZISPGV6QldW+FAuvQt4ebnoceVgHSPn3/ZmiFup94wNrM7PzzfkETWAsExXwUmzRYy4+8ysxbuMLU//5FFbCsDqysrXY/vm/FufkEaTSvpKCVrZy3R08xDRo1NuNmLamAaP8Br5uOXbpX7Ge7Lq60QoHsif7Pmt6nn1EB0gf7PWrO+v25FftJ31dcYN3lxIdNyATWLYS7ws7m7YaZfQ/A+QB22nJKzlt9APQMnH0tgIuznLMbgHOypH8tKUmw1jvgAjN64lzziwvvqwTKqsB68Kk3m/HTFlQ6NwrDfO7HBalQvuNnLzMdO3czd/d7qhJET+79O3P5dTdUSgvlla/jSQPMlZcWsO560qMmZALrV7g6AMD7AHaw9NoTQEM7pyK3s4VtsyVuRVpVYOX8jBNyyScpsO6y33nmtfcmmT/f+vzXQFkVWO94bIi56o4BXzs/nzD188oXlLYmn6kL15qDf9jTXN7nxkoAnbboU1Ov/l7mvXHTK6VvTd61OdeBLuk4Fa6APZqZXX/5aNAE1q8IxvkQX4rA7I8APreTHnB9GoZ1AG4DMM5OPHs1gFEWhPfZyRJ43sMATvjqEnByhSl23kWuW8O5FRkI1ocADAMwCwDLY+gPYKOdmIGz3VQZkgLr4y+NMHc+/mZWSGYD647d/mAWLFll2h59ddZrfCDma7s2IKrJtfSh/uzEk03vM7/ecNav/wDz3QMOKghU+SxJA9WV9+nGzaZZ8+Zm8rRZFY1Xo8dOKNj9xOEK2HaP5ubbJz8WNIH1K2xxwlnOMsO1Z+4G8ANLszkA/BorZ6Q50SOdm32GSY8BOMYec2AtAzAfQHOb/lQErMNtLZllrADAORpDNVbORP7VrOTb7xI7uH7c+zbD8PHUTyq6UB37h7vMiRfcaz5ZvNJ8tulzs3j5mkyN1kHy0DNuNx+MmxX7vbnyGNcEjrW55umX3uDfgmnTvmNF96oHn3wxcx/H/+JU89db7kj8ntzzONAVIn5x4MumVevWmd4B1/bpWzCo8tnjAutupzxuQiawbqEkP+1/COA6AIsB9M7iCvgCgO8CON4u7MWpvBZ4cyk6sHYF8NaWIvDTCFjp13VhMoDGOYDVnY+kaqw+wIp120FFceFqrIUAeXVlxgXWPU570oRMYK3AVKUNfsbTNRCtsdIV4AJro0sAcClbBn7a0xhyBavfeEW/KmuroRqrLQICqzfYQEDdWFFDrg42aToWC1jrNTd7/OrJoOUAVroBl0baU5725mcle/gVzUAmOPcg0+6x6UUftQHQ2rvLvgD+Zf2r7jOeh32wsvWeYN0RAF0JBGMUrDxGVwCFYXgiUmPNBlZOdjvXnl9tpBrrllFcAqvAGn1pxAXWer9+yoQsB7AeYpfArqqhmm05bMNhyLmyZc8vmqgHAPo7J9lZvV+wvtXz7FKzfuOVf9MEMGf1fg9Avyxg5bn0u7rGK75pCFcGQjgbWHnsSQvqomi8KtbPf/++BFaBNQmwblevhal/+tNBywGs/J1XBUyuGMAKmavsVXVeBiRp/Ye1WQaKxYaxC+x+rSPVWFVjzfZCiQImrftx1FgzYP3N06Z+wHJc/roqYLI2ywZqF3jeegAf2Tabg92BNMcEKf0irA2ztlqTAQdZ9RNYBVaBNdnZrbbbs4XZ64xnghYBY9bfbzU11n8DuMi7iP3r6SJk4Nc1a7O72n1F+VZAYBVYBdbkwbr3b581IasFWLezbTfsIVRVYN/3/ao6qPRaKiCwCqwCa7Jg3X7PlqbBmc8HrRZgPSLSRZOUqO9182xhu3b6fehrSRJdXkkBgVVgFVgLANaznjcNApYDWDlYaBGAzQA+AXCG/XGzq+bZlX7oAPvLT7QuxTHeQKTIadrNiwICq8AqsCYM1votTcOzXwhaDmDNCwOUSQwKCKwCq8CaPFgb/f5FEzKBNQbgJZWlwCqwCqzJg7Xxuf8xIRNYk6JgDOUIrAKrwJo8WJv8YYAJmcAaA/CSylJgFVgF1mTB+s29Wpmm5w0MmsCaFAVjKEdgFVgF1uTB2uxPg0zIBNYYgJdUlgKrwCqwFgCs5w8yzQImsCZFwRjKEVgFVoE1ebA2v+BlEzKBNQbgJZWlwCqwCqwJg3XvVqbFhf8NmsCaFAVjKEdgFVgF1mTBusPerU2riwcHTWCNAXhJZSmwCqwCa/JgbX3JYBMygTUpCsZQjsAqsAqsyYO1/M+vmJAJrDEAL6ksBVaBVWBNFqxl+7Q27S57NWgCa1IUjKEcgVVgFViTB2v7y181IRNYYwBeUlkKrAKrwJo8WDtc8ZoJmcCaFAVjKEdgFVgF1oTB2qC16Xjla0HLAazZlr/mAqMLvCWwj/KwcRmAGXZx08O9dG3mWwGBVWAVWJMF644Nyk3nq98IWg5gzbb8dXTlZoeM9gDGAeDaV83tytDbuoOK86yAwCqwCqzJg7XLNW+YkOUAVtIgukprVWBlbZXmwqsADnA7ivOsgMAqsAqsyYO167VDTMhquPw1wToHwMcA6CrY3SLjXwBO9fDxIIATvH1t5lMBgVVgFVgTBmvDctPtuiFBq2GNdW+7aOA2AK63cCUyBNZ8gjOUl8AqsAqsyYJ1p4blpsdf3wxaDcHq/+R9N4FcAb4ycW8LrAKrwJo8WPfrO9SErIZgbeAx4wIA/e1+h0jj1SxvOWzvEm3mRQGBVWAVWJMH63euH2pClgNYsy1//RiA8dbHOhCAD9orbG+AqQCOzAtAlEl2Bdp17GpGzVotm7XaHHf/KJnVYOPmqmGTpmPdu/cw2X85NU/duVG5+d4Nw4KWA1hrfhO6Ml4FBNYtLxWBdcuLJU3wrO5Z4wFrG7P/jW8FTWCNl32x5i6wCqzZXijVwSZNx+IC6wE3vWVCJrDGir54MxdYBVaBtWq3R1xgPfDmt03IBNZ42Rdr7gKrwCqwJgvWXRq3MQfd+k7QBNZY0Rdv5gKrwCqwJg/Wg29714RMYI2XfbHmLrAKrAJr8mA95O/vmpAJrLGiL97MBVaBVWBNGKxN2pgf3v5e0ATWeNkXa+4Cq8AqsCYL1m81aWt+9M/hQRNYY0VfvJkLrAKrwJo8WH98x3ATMoE1XvbFmrvAKrAKrMmDteed75uQCayxoi/ezAVWgVVgTRasuzZtaw67a0TQihmsdwK4oxqLl1p1IHeBVWAVWJMH6xF3jzAhK2aw/hpAdVYH0BfvLQqsAqvAmjxYj/z3ByZkxQzWKJV2iiakfV9gFVgF1uTBetQ9H5iQ1QWwctGsSQDmWZB2AXB32qHK5xdYBVaBNVmwfrtpW3PMvSODlgNYsy1/fQuAKXY+1hcB7GY5x9UENnrLYt+TD/59AKAJgI+8zCZ426ndFFgFVoE1YbDu28789L5RQcsBrNmWvz4MwHYWaDcBoDH4y7TYpNpHBCuDD1ausZ36ILAKrAJr8mA99v5RJmQ5gJX8qg6YPwPwhIVcdefVmIPPATgQwBgA2wO42FsLpsaZlsKFAqvAKrAmC9bd9m1nej34YdBquPy1j6WXvCWvCdb1tnL5FoCD/RNrur2nJfcSAMsAPA6gXk0zK6XrBFaBVWBNHqzHPzTahKyWNVaub0Uf6zcsr3bwmNcDwHwAu5YSy4rqWQRWgVVgTRasuzdrZ07oNzpotQBrbwDvA6iuF9QwAPvVFkYtALBazNrqUgADADAt9UFgFVgF1uTBeuLDY0zIagjWI2wPqPoRuHF/W5tG9i0AsEfknK3eHQHgNNtaxhazUwG4Bq2tzqyULhBYBVaBNXmwnvTIRyZkOYA12/LXM+xn/ljbtcp1qzoewESbxramY/LBsY+zZKJeAerHWmnZ72yASWtamhYMrO5Z41jzao9m7czJj34UtBzAmgVrySSxqktjX65LbdeEfQH8GcANydxCcZeiGqtqrNleHtXBJk3HYgFr8/bmlMfGBq2YwTobwCwAjKPG9NQHgVVgFViTdQXs0by9OfXxsUErZrCmHpwhAQoB1hHTV5jy9p3MQT86PPMpfvfjA0ybDp1Ni/J25uheJ5n3py2v9Ik+atYW+MW5nQ0w+U57Y8oys3rD52buig3G5f3iuEVm/qoNZvaK9eb92SvNyY+MqTj2p+cnmCmLPzVzV24wc1asNz9/6MOKY+76OOJC1koHDBpsWpeXmxYtW5o+199gCnkvcdRY6zVvb3795Lig1RWwdgRwIoBfeRbiTskfLwRYz7+8rzn8mBMyYP1gxkqzV4NG5rk3PszA9IzzLjFX3nBHyYL18pcmmwtfmFgJrNf8d6rp9cCoDDCfH7vQ0AhLphG25z8/IbN/2qNjKs6LA6Z+noWC2brPvjDNW7Qwk6bONGvWbzKdOnU2Y8ZNLBhc4wJr76c+NiGrC2C9BsBQABwg0A/AYgAcjZX6kDRYB7030ex3wCGGtVTWWF8bNcM0atqsAqT39f+vOfAHh1bsx1lDjebtgyXO7d89Na4SWP2ybnhtuhk2fXkGpH0GT6vY9s9JYrtQYB369nDT89DDKkDap+/fDK1Q9xMXWE9/6mMTsroA1vEAtgHgegLsDeD11FO1AL0CfnzET82jA4aZe558KQPWkTNXmX0aNjaP/GdoBqa/6H2WaVnePrVgHTl3lfnH0JkZsD4wfK4ZOm25GTN/tZmxbJ15+IP5mfRSBusT/Z81vU8/owKkD/Z71Jz1+3Mr9pMGbBxg3bNFe/Pbp8cHrS6AdaSF6Gg7jIvDvDi1Vm3Duq3IgB102XeWE8FwnO45Wa7lMFvX/4y1anbidfvfzHJ+rZOSrLH+/f7+5vhTzshA04GVtcYHn3vNdN3vANO+c3fzm3MvNq3bdUwlWB8f9UnGx+rA2W/EPLN47WfmtEc/Mic+NDrja73q5SmJwDVpgLny0gLW3z0zwYSsLoCVc69yXsKzAUy3cKNLoLZha8B6EoAHbIG5zDRzrZ0sJtd7dFOF5Xp+5rwkwfrrsy8we+3T0DRo1MTssedeZoeyHc0Rx55YCaJ3PvKC6XnUcZXSop/sce07oMUdZ3MF/HPYrAw4CVBX/q1DZpg3py6r2H9i1Cfm4RHzKvbdeXHEDnRJx2lwBbDGeuazE4JWF8Dqw4ZQ6+wn1GI7G1hbAngFAGvH7wBoC6CrnWSbQ2pZA33am3SWE9NmCz5YHwZwgneSK/eHtoyBAKYB4D7HANN/zBo5pwxzkzB4l2/ZTBKsPhD9GuurI6dnQPre5CUV/lf/3KS24wBUtjyjYL1u8FQzb+UG86vHPqoEzVMeGZNxARC2bMga+8ka89dXplU6J1v++UhLGqiuvE83bjbNmjc3k6fNqmi8Gj12Qkm5Auq36GDOfm5i0IoZrN0BVGdbCFOzLQc4/+ohAFrbhO8BeNNuc2KEf9ntra2xVgdWTgXW3OZLsK4B0Nj6lDkRw0H2WNaoGMB66u/OM81alpumzVuZC678W0Fqq4R3PoAUyuPtGcvNivWbzOYvvzTL1m0yd7412yxcvdEs+3STmbV8fcZembSk4l7ob2VXK3bPemHcoor0UDm1Pe5AV4j4xYEvm1atW2d6B1zbp2/BoMpnj8PHWr9lB3POC5OCVsxgZU+AqswBLytwckyMgnUXrybqfKOTbV5xgZXP5wLB6jfK/dubj9Gdw/hM+5/24T4NmxQMZEnVRHMtp7YwKqXrCwHUYiwzLrD+4YVJJmTFDFYfJnFsR8HK+Q0XVVFQbcBK3yz74DKwd8PndpsgHWS3GUX3WUNmuVWGQtVYc4VdkueVEhhr+yzFCLlC3FMcYN2rZQdz3ouTgyawVsbWcAA/t0n0b3LhQgYfrGz9n2vTq4p8H+uV3to1xwEw9qIoSKP7AutWjNqqLYxK6fpCQKwYy4wLrBcMmGxClmaw/g/AJ55daP2dbLxin1muDHu1haAPViY9CYALGubSeMV+t5z6kHlyQhlXU46CNLovsAqsNfLLFiPkCnFPcYB171YdzIUDpwQtzWCtqrZZZ9LlCtgyD0Ep1Thr+yyFgFgxlhkXWC9+aYoJWQ5gzbb8NWfzYzsLu5Uy3t3CiF/PdwDgfK2cRpWN+rUOzJSTW7vaY1MA3611riWQgcAqsGaDcDFCrhD3FBdYLxk0xYQsB7BmW/76ZjtFKunEqVLd8tdHARhsu1/un6+J/tk6fhcA10JPio8qAS7W+hEEVoFVYE122sB9WnUwl708NWg5gJW//2jXzakAGlgwMOY+w70Afmm3Gfnneclbt8mlCBg4nNQFN2+A209lLLAKrAJr8mC9/L9TTchquPz1ag9k/FJ3++w95PdpZ3/7Wi8myDH6XEjLAZbj9n3IeveSrk2BVWAVWJMFa4PWHc2Vg6cFrYY1VgdSB7JVdiMWsJ4CgMM+2YJ/va0Guy5R7gZSGQusAqvAmjxYr3plmglZDcHqf+LH7gogNDlm/1wAf+BseamkaJaHFlgFVoE1WbA2bN3RXPfa9KDVEKzsvslGKwbGbMxiODrSeOVm/LOHaxaxF0A2q1luJXSVwCqwCqzJg7XP69NNyHIAa7blrzn4iP5Tdrd6wy6mSmLR38oG/JkAOD91rf2rzJQZse8WYxb4hV1ju4QQWbNHEVgFVoE1YbCWdzR935getBzAWrMffYxXsXOsmxs1xmKKP2uBVWAVWJMFa6PyjuaGITOCVhfBSuKx9pr6ILAKrAJr8mC96c0ZJmR1Aawcw+/sYjtO/9XUU7UAa14lOVvV1paVDTBpTSvEKKdiLDOOkVessd48dGbQ6gJYuUqrsysAsPtVmcAKqMaqGmu2l0cxQq4Q9xQHWBuXdzS3DpsZtGIHKwcG3CqIZldAYBVYBdZkXQGN23Q0/3h7VtCKGaxugT0uUaKQRQGBVWAVWJMFa5M2Hc0/35kVtGIGqxvCyklYOPLqNAC9PMuCmnQlCawCq8CaPFjveHeWCVldACuXunbGOQy5zTj1QWAVWAXWZMHatE0nc9d7s4NWzGDl3ADsDXCRF3Pb7QusHbtqMUG7ykA2wKQ1rRANRcVYZhyNV03bdjL/Hj47aMUMVi7sx8mtXY8AP3aTXqcarqqxqsaa7eVRjJArxD3FBdZ7359jQlbMYHU+1lTDs7qHF1gFVoE1WVfAvm07mftGzAlaMYNVc65WR1UNEKjkBskGmLSmFaJ2WIxlxlFjJVgfHDk3aMUMVi6spVCNAqqxqsaa7eVRjJArxD3FAdZmbTuZfiPnBq2YwVoNUnSICgisAqvAmqwroFm7TubhUfOCJrDWYUYLrAKrwJo8WB8ZNc+ELADWNgDGerYWwPkArgWwwEvn6qwKSSsgsAqsAmuyYG3erpN5/MP5QQuA1UcFh+0vBrCvBSsnmlIopAICq8AqsCYP1idGzzch2wqwHgbgPcsR1lgF1kJClWULrAKrwJo0WDubJ8d8ErStACtHkXItPwaCdY5dMYXpu9t0RUkqILAKrAJrsmBt0a6zefqjBUGzgPzQApbxmVnY8E0AywHsbY8xpmtgG7sitYbuZxEt9iSBVWAVWJMH6zMfLTAhy7HGeiyA16oARTMAE6o4puQ4FejUpbuZu3yjbPlGs2bjlzKrQSH6jBZjmXH0Y23RvrN5duzCoOUI1v4ATvcY0cDbvgAAjyskrYDAuuWlIrBuebEUI+QKcU9xgLVl+87m+bELg5YDWHcGsALAtz1uPOatSs2pUn3QeqdpM1YFBFaBNdsLpRAQKzeA7ZgAABk+SURBVMYy4wLri+MWmZDlANZY2aDMa6GAwCqwCqzJ+lhZY/3Px4uCJrDWAmyFvlRgFVgF1mTB2qp9ZzPw48VBE1gLTcdalC+wCqwCa9Jg7WIGjV8SNIG1FmAr9KUCq8AqsCYP1pcnLDEhE1gLTcdalC+wCqwCa7Jgbd2+ixk8YWnQBNZagK3QlwqsAqvAmjBYO3Qxr0xcGjSBtdB0rEX5AqvAKrAmD9bXJi0zIRNYawG2Ql8qsAqsAmuyYC3v0MW8MXlZ0ATWQtOxFuULrAKrwJo8WIdMXm5CJrDWAmyFvlRgFVgF1uTB+uaU5SZkAmuh6ViL8gVWgVVgTRasbTp0MW9NXRE0gbUWYCv0pQKrwCqwJg3WrubtqSuDJrAWmo61KF9gFVgF1oTB2rGreWfayqAJrLUAW6EvFVgFVoE1WbC27djVDJ++KmgCa6HpWIvyBVaBVWBNHqzvT19lQpYDWLm21Xi71DWXbmHYA8DrAKbbWGteWWESjQRWgVVgLQBYZ6wy7wcsR7DuGQHGzQAutWmMb4oc124SCgisAqvAmjxYR8xYbUJWQ7BO9VYN4OoB3FdIWgGBVWAVWJMH68iZq03IcgDrbABjAIz2VnBd7THkGwD8fe+QNmNVQGAVWAXWAoB11mozMmA5LH/dyMJhLwDjABySBaSrYgWIMs+ugMAqsAqsyYL1qyXn15hRs6q3HGqs/o/6WgAX209/t4CgXAG+QkluC6wCq8CaMFg7dTWjZ68JWgCsXKH1W5YV3B4O4AgAt0Qar9iYpZC0AgKrwCqwFgCsc9aY0QELgLWF/fynC2AigCssO+oBGGK7W71hu18ljRWVJ7AKrAJr0mDtZsbMWRu0AFgFr2JWIGmwvj9umtn/+4eYVuVtTes27cw1199i5i7fArcrrrvBADAfTZ1fKd0/J67tbIBJMm3uohXmp8f1Mq3L25jyNm3N60PfNUmW75e1cXPVsIn72IBBg03r8nLTomVL0+f6G0zc5VWXf/fuPfj3mNfQrlM389HctUETWPMqe7KZJQ3WkRNmmUFDhmegOXH2UtO8RSvz+ntjMvuE7iE/6mkaNW6SSrD+8pTTzB1335uB6bI1Gw1B68Muye3qYBPnsXWffWGat2hhJk2dadas32Q6depsxoybWDC4xgHW9p26mbHz1gZNYE2WhXktLWmwRmubhx7xE/P4c4MyYD3ymOPM4GEfmMZNmqYOrPMWrzRN921mVm/4omAw9cEdJzyry3vo28NNz0MPqwBpn75/M7TqronzWFxgHTfvUxMygTWvqEs2s0KC9d0xU0zDRo3NhNlLzP2PPmNOP/OcDGDTCNZ3Row23Xt8x5x86q9Mpy5dzWm9f2MWLl9bMMjGCavq8n6i/7Om9+lnVID0wX6PmrN+f27FfnXXxnEsLrB+PP9TE7JSBuuXdoKDCQBeArCbxV5DAM/lgMB1VZxzHID2WY6xZW+sNVc29/+Y5dy8JBUKrJPmLDMdO3cz9zz8lJkyb4Xp2n0/M37W4tSCdei7H5htt93WDHlreAamZ59znrnk0isE1hIF6/j5n5qQlTJYfTA+4nVpyBVq/vX+NQ8DOMFPyLJd1bVZTs0kbVfVgerSCwHWGYvWZnypV/a5MQPSV98eZertWT/jAmBtlYBhTXbUxNmZ41H3QVz7/qdw0tvTZi8wTZvuWwHSwa8PM4cdcWTFftL3E0ftL5c8U+EK6NzNTPhkXdDSAtazAdxtIdUMAGuxDDsBeAbAJAAvAvgAwH72GOF4ve1vNgLA3gAOBLASAMf5sjba0p4bjRxY/bJ4DkdXcJQFwzAAt9v/gIvsPmezGQlgGoCD7XlVRkmDdc6yDabXiSeb35x1bpXQTKMrgOA84MCDzIfjJmVgeukVV5vzzr8odWD9dONm06x5czN52qyKxqvRYyeUlCugQ+duZuKCdUFLA1i3BfCsHdlASPmwI+juteTqCOALD6zsqnGMPcYREFfa7a2psfpl8fIoWB3seYygvc2WcRQAdhDOFs60/2kfsgU+rhpgtnyfG/RGpjtV2/YdTfuOnTPW76kXK91DWsFKP2vX7j1Mh46dzNE/+amZs3B56sDKWu2LA182rVq3zvQOuLZP34JBlfcSh4+VYJ20YF3QShmszs+5DMDbAAhYBh92/wHwI5vOiLPNuBrrJgCcYYbhFwAesNv5BOsPbJ6MCNbv233Wjmd4x7JuJl1jzQbbYklL+nO7mMvL5bM9DefEBdbJC9ebkJUyWN3nOD/33/EakXIFq7ueUKNPlUBl2BqwNrZuBntpptbruwIcxHmcYHX7nACXM4xXGwTWLYMTihl0Sd9bGqCZyzPGBdYpi9abkKUBrIRTNwBzAbCRyAfrJQD+benFlv7NHtyqAuudAE6vlniAu3Z7AMsBcAzwDgDoqxVYvdFa+arxJg2vYi4vF+ik4Zy4wDp10XoTsrSAlRxkl6vTImDlzDLsesXGqxdsg1RrC00HR+76NVZ+rvP8j3JovOK17G4107ojWNsVWAXWWP2vaYBmLs8YG1gXrzdTA1bKYLV8rDai37XMnsEWfrb2f7PaK4rooFwBcgVkqznnAp00nBMHWDt26WamL9kQtLSDlXMicoVETt/1MYAji4ibwVsRWAVWgbXqCWfiAuuMJRtMyNIO1iC8ivkEgVVgFVgLANalG8yMgAmsxUzOwL0JrAKrwJo0WLubmUs3Bi0A1iYAhtq2Gk50/Sf7U2f7ywJvaDz7syskrYDAKrAKrMmClb+52cs2Bi0AVq5n1d3ygu5IjrRkrySClYOIFAqpgMAqsAqsBQDr8o1mdsACYI1iYwCAQwXWqCwF2hdYBVaBNXmwzln+mQlZDstfO2qwX/08ALtasHJgEBvSHwKwuztJcYIKCKwCq8CaPFjnrfjMhCzHGusuAEYD6GWxwaHs7AK6jZ0AinBVSFoBgVVgFVgTBmvX7mb+ys+ClgNYOSrzVQAXVsENf4RoFacoORYFBFaBVWAtBFg3mfkrq7cAWDm506N22lCfDWzUcuECAP3djuIEFRBYBVaBNVmwdu7a3XyyalPQAmA9iKsZW1+qW3WEXaseAzDepg8E4IM2QbKkvCiBVWAVWJMH68LVm0zIAmBNObmK/PEFVoFVYE0erItWf25CJrAWOTyruz2BVWAVWAsA1jWfm0UBE1irI1eRHxNYBVaBNVmwdunW3SxZ+3nQBNYih2d1tyewCqwCa/JgXbp2swmZwFoduYr8mMAqsAqsBQDrp5vN0oAJrEUOz+puT2AVWAXW5MG67NPNJmQCa3XkKvJjAqvAKrAmDdYeZsW6L4ImsBY5PKu7PYFVYBVYkwVr1249zMr1XwRNYK2OXEV+TGAVWAVWgbXIMVX3bk9gFVgF1uTBumrDFyZkqrHWPZ5W3LHAKrAKrAmDtXuPnJYuF1grMFX3NgRWgVVgTRas3br3MGs3fhk0gbXu8bTijgVWgVVgLQBYP/vSrA2YwFqBqbq3IbAKrAJr8mBdt+l/JmQCa93jacUdC6wCq8CaPFjXb/qfCVkOYD0CwFQAMwBcWvGj1kbhFRBYBVaBtQBg/fx/Zn3AAmDlulYzAbQA8E0A4+zy14WHiu4AEFgFVoE1ebBu+NyYkAXAeoBd78ph7DIANIUiUWCZ/Q/8sIAxl+stZPnFVLa02PK3UAxa8PeR7/BKjn/vEyLnnendyAkAHvD2TwPwL29fm1IgA1XJ8JUChLzCVwpIi6r/EgTWqrXREauAfkBb/hSkhbTYokDVW3IFVK2NjlgFBJMtfwrSQlpsUaDqre0AzALQ3Gu86lD16TqSRgV831Ean99/ZmmxRQ1psUWLbFtc7nqa7R1wRbYTlCYFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkiBJBT4RqSQ6H7ksHalgBSQApUV2BHAMTapM4BjKx9O3Z4PUeqxKwA/LXWCeA/sdCgDsL2Xrk0pIAUiCnCCiWsAfGStPHI8rbvnApgEYCiAUwE0SKsQkefmi/dlO9zzR5Fj2pUCUsBT4EAAiwG846Wxg3SagquN8Zl7AXgSwE4AfgHg3wDOArBPmgTJ8qxtALwJgMM9fwdgOoDDspynJCmQWgUcSBjTHdARwL0ABtqRJhQmLSBxWvCZGwJ4AsBb3l8GQXsPgD8B2NtLT9NmDwDPAbjee+iTbAf6I700bUqBVCqwB4BW9sk5soTA6GP3dwbQD8AgAIcAeB5AoxSpdD6Ap+2zvwvAH23zSwC3A9gtRXq4R/0hgAutNv8FsK/nd/4VgPkA9vTS3HWKpUAqFGCjAyH6VwA/AzDWfupy/DynSONxTvDLT9/3AfwkFap89ZA/tbX1vewzHwSgP4DLPQ2+5W2nZZM+99cBNLYP/DiAuwE09QSQ/9kTQ5vpVKAngJsAPAjgIisBG7BYS73fcwOwBsLgfybbpJKI/OfiC+VOACu8GjrdI98HwBraxfaJ/WtKQoTAQxCYI+yXC3tHMOwA4GFrrLkqSIFUK7CN9/TtADxka2j0rTKw+wxbwB8rYZjyOaNwPO6rx4dzgzwFwNVaCdf9PdjaU0s6cvq4z332iBgO4McWqnx4wpWNe+5vp6QF0cNJgaoUcD8W/hCeAVDf+ln/YT91CVoG9gTYz26XavRt+2BOE/aEYK2MYXcAtwF4JEUNd/bRM7B023yxsAb/F/si+g2A16zvmVBlcPrZXUVSIF0KuB/A4QDuA8DlKFgrZes2gUqQ0Oeahrkm6Ud920LBb4R6yabzL4Pp7B1BrVjLd/qV8l8N/ahX2xere86fA/g7ADboMZwO4D0AbMhKgyb2sRVJgcoK+CNjWFP9BABnRWdPADZgseZKPypHF93h9RSonEvp7PHFwsa67vYTlrUx1zuCT8kGGrpCGFhzde4Am1SyEfumjgZwon25cmCEC3ST8G/jHJvAvqvfdQcVS4G0KcBGh962fyqf/Tu2lup0aGkbq1hzJUTYE6CUAzuxc9E6+pVZ2/qefbH80YMrW7032Z4ApayF/2yE6hJbM2U6/ahc8vls7yS+gLj08wVemjalQCoVaG0/8+lL5dIS9QBMsbB1grDWygYIDmel38xv3HLnlEJ8NICPAVwC4Dq7hDG7TbGmzhcLP3WbAaCbgJ/Dfi22FJ6/qmcgVEfZbnbLAbC3CMMPAIzxaqn8ymHfXueHt6cpkgLpVIAt2myE6WsbYvjD4Rhv9st0Px7W2O4qcXnY8HKwfUbWXP9pG2UcXPn8/7HrG6VlngT+bbDhkp37GTiCaq0HVw4OWWgHi3Ddp0PteYqkQCoViDYqdLI1En7OsXGqKwA21LCTdxcAnESDrb2uj2IaROMLhiOoqAl7CLC2ztFlaRphxv9nNzTX/c1E4doCwK+tXz4Nfxd6RimQVQH3A2GrLT95WTvjJCL8tOUw1UttNytezHMJVc7exE/iUgxOD/ds/j7hyt4Q19rhme6cNMS+Dnxev+cD4cpBEnSfMETPtcmKpEC6FOBkGBNt4wM7dd9ihyLSz8rhmVdZ2LK/Kn9EpehP9Gvf0aGWPiioFScUof85DYENlO75Xeyem/su7TQAG6wupep3d8+tWAoEFWCrNlu9+QnHGhkbq9jJmzUzdh1ieqnWTp04nGDmt3aKv5PtJz/9iX5wAGEaa/RpCByySz8zZytjdztOKOPrQA18uEZfSGnQSM8oBb6mAD//aexGxdbeD2wfVTZCzAZwq51g5WsXllACJ2D+l32B8JmXAtjFPl+0O1kUKiUkQ9ZHoR+ZvR/YeMnGKDZeZgtOF1dTdfvZzlWaFChpBdhANQBAN/uUnDiEkxEz8BhbvNva/VKNCNBhAE4BwFmpOJ/qSNvFqlSfOdfncnDk38BUAM/aLnhpm8g8V710nhTIjPnfbKf5oxyupsH+iUMAzEjR1H/8vOXIKTbI8dOXrhEOwWRDHgNraXzRpCm40Xf0O/Nvg1P9scGOvSI4uQwDfczRGr09pEgKpEMBzsDEUVQMHCXDLjMcgrgxMtaftREOR+TwzbQEjjRbabuSscM/Q3s7aoifwHzZpGWaO/rTXXcq/h1wbl021NH3zikiOW0k3UPscsbBE2nRxf5ZKJICWxRggwI/9zlvKkdMcTIVV+vgqCJ2k0nTNG7uE9cpxH6o7OBPXyInUeFSIgycD4HzzpZiDwj7iF+L2PtjnZ1P9lG7GCLH+3PACNerYk2Wa3hxQvO0r8r7NfGUkB4F6B+bbDtrc7KMVbYHABVwn/8cy/15Clr+o//r59luZZy4m3ClC+Ay25DF4ZhpDXSBEK4cpsvAniHH2wmq3Ygr97cTfUnZSxRJgdJVgJ+1rJ2eYR+RfVI5wxAnxvhz5LH5OZymVTN/D+AN2xOAy3dzJiYG1lw5lJddzehvTQs43HPymRnoT13vuQXoT2X/ZTbu8QWkIAVSqwDnweT4dgb+cDgslfDkmP9FFrjs7M6GG/eDcj8we1nJRqyNse8qP/XpR6X/kMZPXYIjLZ3/3d8GY87cRXC6SVNutFNGcvVZBrpG0rL6rn1kRVLg6wqwNZut25xPlAMAuHQI3QJsjOBKoqyxcV0mfualIbiXBmMugMg+u/Q5u+5Df7D+wzRo4Z7RacK/kRds32UuEMk+zQycyHyNXdrbJimSAulWgKOD2CDDSZq5rjtH0HAOVbb4c8gqa2xuNnz3Ays1xf4PAD/7XXC+QU6POM9bFJGuEHa3SsssVf7IMWpBl5EbXcdRd5xwx8H1ZtuTxGmoWApIAQtQXwiOtGI3Gtetxj9WatvsYvYlgDO9B3N9NNlTgv5mNl6xZs8uVmkI7JvKeVLdS5W9Rl60c8u65x9sB0o4NwDTS/Xl655ZsRSokQIECsd7EyZuFqIaZVTHLmL3KfaGcLPbs9bq4MrGPNbc3UKBdezRany7hClrqnQBMHBZavYa4RcNA/8++HfCkXkKUkAKVKEAQcIhq/SrHlPFOaWczBVkCVe3BhOflT7VVzyYlPLz89mi8x/Q/THXuob48qG/nf53Nm5yWC9r+1zfTA1Wpf6XoeerlQKEq/uRpPGzzsGVjXU/sQ017CmRhsD+zByqzEa7K70a+qkAxtvJy9kdj53/OSENJzPnRDystbpabBp00jNKASlQAwUI1//ZeUNdY00NsqlTl9B3zF4gnBaRy80QnByu6gJn+2dfXk5g7gKHsE5L4RwJ7vkVSwEpsJUKsJ+ma/Heykvr3Onsm8vVUwfaO+eXysUArrD7ztdM4LJXBPuqMnAkWpqG8trHViQFpIAUyE0B+ko5x6zzLxOqnHeW00JyDTMO3+WqvK6XiOuSllvuOksKSAEpkFIFnH+Z3aretpPycNlujq563fYQSKk0emwpIAWkQM0VYL9d9oy4xmbhaqbu87/mOetKKSAFpECKFWC3KroFOKOZC2nsIeKeXbEUkAJSIC8KcLKVT+2qAIJqXiRVJlJACkgBwF/qW3pIASkgBaRAHhRwNVUX5yFLZSEFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkiB0laAE1Rz9QPObv8sAH/W+619cs43yuWbGTjDU3WTWnMi8APtuVsTzfHG2vvXVZXun8NVUbcmcLE/jvlXkAJSQApslQI+bLjI3YWRq91aVZHkrLs+WLOe4CXWFFpVAbSqdK/IzHLT/n5ou6b3GMpXx6WAFChxBXywcvb/uwGwNvmOna2J09pta9fvGgXgY28xQHYj4vR4U71FE12NdRgAjp1nOALAGDvvKOcr5ZLhiwEssLVlrhPGCUqeB8AyaJwwnIErub4GYKKtBXPS6GzDQ32wcuKT0fYaf7kYPus/bDrvg2UytLSTcfMaPjfnVWUQWK0QiqSAFNg6BRxYWTPlEiFcFJBg5Xr3nJSZgXDixM0MOwDgCqM81stOLkLwct2m1Z4rwIGV8Jrv5cUlWhii0OJKrgfZY03tSrfcvQMAl9Fm4FImJgewujJ2tC4Ot8w2rz3F5sU8+VJgIGS5hAoDR0y9abej92iTFUkBKSAFqlfA+VjpZ+XKoZxblGAd6l3G1WdZc+U5NE6HdxiA2+2kzu5ULu0crbFyiRq6GKIhCi2OrXf5M2ZtlsuccLuFd/HKHMDKvDkrP43LSu9vr+ezOtcG82TeLGNjpGwuX84QvUebrEgKSAEpUL0Crsbqn0WwDvIS+InuFsTzkvMK1uUAyvzM7fbWgpX3zpn8XSMca85MY4iClTP6c9jpIns8GgmsUUW0LwWkQE4K5AJWugLot3Qz4ZcD2Nm6Al61PliuSMrp8qI11qpcARcBuM67Q7oCLvH23TpZdAU4N8SRObgCjgXwks2HvtLPPLDSFXCSPcY8WUNnGA7g53abfmOuRcUgsFohFEkBKbB1CuQCVs4p+je7IB67ZdFNwKWs/cYrTuTMFUijYOXdEIisHfLTnOcxEM5sCGONlI1XbJB62qZx+ZJ77Hl+49X9dsXT6hqv6AMebH20fBn4NVY+69+t35V+VNd4RX8xV5Ll/bFs59MVWO1/giIpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBZJR4P8B5DhXU1+nRmMAAAAASUVORK5CYII=)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JqM3S3SASfl4"
      },
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bskTvE4c2_K1"
      },
      "source": [
        "- Additional data:\n",
        "              precision    recall  f1-score   support\n",
        "\n",
        "           0       0.77      0.91      0.84       348\n",
        "           1       0.84      0.76      0.80       249\n",
        "           2       0.00      0.00      0.00        36\n",
        "\n",
        "    accuracy                           0.80       633\n",
        "   macro avg       0.54      0.56      0.54       633\n",
        "weighted avg       0.76      0.80      0.77       633\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAgAElEQVR4Ae2dB7gdVbmGP4zIoZdQpIV00itIuYDgBQSUIh0UhYtSpHcQVOAGkeZFEBClg7QokBAEQgm9pEAS0nuhpZAEkpCEoP99vpO1TlbGfc7aO9l79pwz33qeP2tmTf9y9jv//KsBSlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaRABhRYF8DTAD4H0HcN7ufHAAauwfFZOnRPAOOzdEO6FykgBSqjwPEAhgJYBOATAM8C2KMMlzoBwGAA3yzDuRrDKQxA28Zwo7pHKSAFKqvA+QBmAzgcwPoA1gZwMIAbynDZKwA8VIbzNJZTxMCalxdMY/n/0n1KgYoosLHzUo9q4OzrALgZwMfOuMwypr0BfAjgAgdnersnuW1XAfgKwHJ3jZMBXJkAbUsAhJEHzokApgBYCGAqAIYAmFj+hltmtjuAIS7EwJzrPr0C4H8BvOnOwxDC5n5jIvf3f3Fw/4cBOAjABADzAPwqOOY7AN4GsMB59n8C8C23/TX3LIvd8x4T6HMJgE8BPBiU8bA27hq93Dm2ATDH7RNcVotSQAo0JgUOAPB1ALZC9341gHcAbAlgCwBvOXBxX4KJx3MferoE0pcANnUnSoI0uR6Cld7yFwB2dMduDaCzWw7BuhmA+QAYZiCQj3Przd2+BOtkAO0BMMbL9d+7bcnM3/9v3P3/woHtYQAbuusvAdDKHdgbwK7uurz3sQDODU6a9Fj9+a9zLyPej4e5P4zXHANgPQDPA7jRb1AuBaRA41SAHiE9qYYSIUVg+vR9ANPcCiFB8HiPk8UMKxA+TEmQJteTYKUneIQDojtFbRaC1cdtw+30IrkPE0HKEIRPvwTwnF9J5P7+m7lywpRw3CXYbxgAerGFEqH6ZLChEFjptdcE+yTByk39AXwAYGTwNRAcokUpIAUakwLFeKwEp/cc+Wwd3Cc+lwtBgtDd14mQBGlyPQQrDyG0X3Cf2s+4a7E8BCs/q5MtDB4FcLm7JsH6c7ecPDYorl1M3j9fEIQj78snhiB+4lboBQ9wLyN61/TOX/c7umPDyiue/6NgOxeT12QZY9q8Lr1XJSkgBRq5AoyxMiZ4ZAPPkfRY9094rIyxhqkhsF4E4IlgZ3q2BEro8XIzP5lvCqAVgrWQx8rwROixVgqsL7lPdXq2TPRYw9hvIY81qU8SrBu40MVdDsIMdShJASnQyBVgxdMs97nLOB9jpQcCuN49Vx8XV2V8lZVABAnLmJKQYFlDYN0PwFwALQAQ6v0CsG4F4FDXMuEbAFj59eqKy6zisTKWypABm4gRyKwk4rqvoKqkx8qmY4zHruW8abZHDcHKsApfPD4V0idZdjeAx9wBfwHwuD9YuRSQAo1bAcZa2Y6V3ivhwM9wX9PO+OAtrhactf5c9jHDJCSoQkNg5fbbHAgnuU9f77GysoogZWcCgpKA7ORkDT1WFrGNLWOf3Jd52Oa2kmDdC8A4V+vPEAAr7UKwnuZ04v0fXc+LJ9SMLxKGCryXSu+VuvjWEO7xlUkBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpEDKCjSrsbXW3UK27hbWtUcvmdOgV6/eJuttzZs3Z2uPsqZvbNiiqN9bAz3tyno/OlkFFCBUa3qcIetxhs2ct1TmNFiy3ExmtS+Xcv/siv3NuaaD5b68zpeGAsX+J+cBvgLryheLoLrixUKvvdy/w7XW29Jqep0dNYG13MqneD6BdaW3LrAKrMkXSsXA2vtcq4mYwJoiCMt9KYFVYC30QkkCJq/rFQPrTudZTcQE1nLTLsXzCawCq8Bafzy5MmDdymp2viBqAmuKICz3pQRWgVVgTRms629lNbtcHDWBtdy0S/F8AqvAKrCmDdZvW82ul0QtAlYOIMRRykYAGO1GUSM5OEvEu24QHI425qfc4TREXOfgONwejtWbInFycimBVWAVWKsA1t0us5qIRcDKIR85ihgTh8gkLDkuMIdrPNaV/xnA6W6ZM1BwnYnb/RCPrkhZWRUQWAVWgbUKYN39V1YTsQhYQw5w/OH33FQ8HCfYD7i+m5t/jPtyHjKuM3E79yOclSqhgMAqsAqsKYN1g62tZo9fR60IsHKOs+FuXF1O9MhB0vmp79P2AEa5Febb+Q1utgc/qHpQrMWyKCCwCqwCaxXAuudvrSZibrB1Dtzu7ZR6fvSbABjkBkoXWOsRKdVigVVgFVjTBus2VrPXVVErwmMNWcGpdjgXm0IBoSrVWhZYBVaBNWWwbriN1ezTJ2oRsHLuNnqqTJy0klPs/NDN/BtWXrHSiumMROWV5iRzwlQkE1gFVoG1CmD93jVWE7EIWLsBeB/ASBdHpcfK1No1w2JIgNOrs5kVE5tncZ3lbKbF/ZQqpYDAKrAKrGmDdVur+e9roxYBa6WQoPOWQwGBVWAVWKsA1n1/bzURE1jLQbgqnUNgFVgF1pTButF2VrP/DVETWKsExXJcVmAVWAXWKoD1+zdZTcQE1nIQrkrnEFgFVoG1CmA94A9WEzGBtUpQLMdlBVaBVWBNG6zbW81Bf4yawFoOwlXpHAKrwCqwpgzWjbe3mh/cGjWBtUpQLMdlBVaBVWBNG6wtrOaHt0VNYC0H4ap0DoFVYBVYqwDWg2+3mogJrFWCYjkuK7AKrAJrFcB66J1WEzGBtRyEq9I5BFaBVWBNGayb7GA1h/01agJrlaBYjssKrAKrwFoFsP7oLquJmMBaDsJV6RxpgHXj75xjQz6YaiPGz7TRkz62Pn9+xmp6nGHnXvuYTZo+25i23fvi2jKW0/Y7+WYbPm7F/q8NnbDKNr9PufNCgKlk2cSPF1j3XjtZx85drf2OHe3ci35lvN5hRx5jrdu2s/YdOtnRx//UpsxaWFteyXtJnrua0133G/CstWvf3lq3aWNXX3OtVfNeKjJL6yY72LqH3x01gbVKUCzHZdMAKwHYfLfzauG4wU5n2eCRU22vE26wXY651tof+Gub9tHcVcC61R4X2pjJH1u7A66oPWb7fS5pkmCd8dkSGzdjbi00Cc8evXa2fs+/avc/9pRxG+2Qw4+ya268JTdgXbT0a2vVurWNGT/ZPl+8zLp27WbvjRhdNbhWAqzf2HQHW+/Ie6ImsJaDcFU6R1pg9d7lpruea++NmWF7/uT6OlgmwXr2NY/atX95tm67P7bSedJrS3N9wofzrEu3HtZ/4GurQPQ3fa63M8+/eJWyNO6rWl7ioNfesn33278OpFf3+Z3RqnU/lQFrS1v/qHujJrBWCYrluGxaYF2v15m1n/YLFy+1G+8ZuAo0k2C99aGX7Y5HX7VXh0ywYaOn2/9ccf8q+1cKsGkAK3mNaXMWW6cu3Wy99de308++YBWA0oslbP/+zIurlCfPUYn1aoHsb4/2tRNPOrkOpHff+4CdevoZdetp31elwLrB0fdZzATWlYS73M3xzcFnOcnXLgDOBcBZFEtNVwPYN3LQlQAuLLAPRxb3I4cX2LyyKC2wehjyM/+VweOt1xF96mCZBOsdj7xi746YYpvtel5tiGDi9FnW5ZCr6vb35yp3XglAFXvOUVM/td322MteeGNYHUSPO+EkO/nUM+vWiz1XOfZLG2D+enkB60bHPmAxE1hXcIpTy74djNjNGRC3cROC1TcbImdZXJNUH1hbBrMzNnj+tMFKGF51+9N26U1P1IEyCdYr/vhUXQUX97/3yTft+Avvqtu/3ED15ysHkNbkHBdc+mu7/Kpra0F6/iVX2P4HHWzT536ZK7DmIhSwWUvb6LgHoiawrkDX4QCeTlDsbABfAfjAzaLIzYsA3ARghJtVkVMqDHEg/Esw1/d9AI505zsIwDgAwwDcAmCAKydY7wHwCoApAHg9pkcBLHFe8w2urGCWBli32+cSo6dKgG2yy7n2xrCJ9qOz7qgDZRKs3X90tb38zjhbv/dZxpjsqIkfreLhehCWO18TKK7OscMnzDR6qjx2wkfzbeddd7d7H3nCrr/5duu98661Zatz3nIc4z3ItPOFS5Zby1atbOyEKXWVV8OGj2pSoYBmm7WyjY9/MGoC6wpkbeBANgHA7QC+60g2zc337cFmAI72KwA2C5YfBHCwW/dg5Vw1MwG0cuWPJMD6lvOS6RV/BmBtADGPldPorphSd+0N6gBXblD58+101DX2/tgZNnL8h7WQpMfKbedf97h9+Ok8W778a/t49gK754k36+7lsj88UdsygFC98Pq+deX+nJXIywGkUs4x8PUh1rlrd+vQqUtt0yp6rDy+WbNm1qJlq9rYK+OvvryUc6/pvmkDNbzek/2fsbbt2tW2Drjy6j5VgyrvqRIxVoJ1kx8/FDWBdSUZ+Wm/N4CrAHwK4MQCoYCvAYQhgCMAvOu82o8AXJoAaw8Ar668BA5JgJVxXZ/GAtiuCLD6/ZGGx1oJCFbinGsKo6Z0fAi6PC9XCqybnfCwxUxgrcPUKgv8jGdoIOmxMhTgE73RWQC2dwX8tKcxeY81Btaw8mqUg2rMY3WXgMDqOiQQ1E0JjGv6LHmGafjsFQFr81a22U8fjprAugJTOwJoV0csoA+APzlP1H/Gc3MIVtbeE6ycF5yhBIIxCVZuYyiAsGT6W8JjLQTW5gCmu/0bzOSxqktrIQiHcMnzcqXA2vxnj1jMBNYV6OoNgPHOMW6u7ydcbPUsAOMTlVch7AjgyQDeBHBvAbByX8ZdfeXVnx1cWU4IFwIrtz3sQF31yqtKfLZX4pyFAJPXsjzDNHz2SoD1m81b2xYnPRY1gTXEZGWW6c0yreUqxs5z62ucyWOVx1ro5RHCJc/LFQPr/zxmW0QsAlaGDwc5R240gHMcDOhssa6G7ehpbFHk02UAJjlH7/u+MM85QUqR6A0zFLA6HQ4K6iewCqwCa7qjW31z89a25cmPRy0C1q0B9HI/6g0BsDVSpwJfsf53z21s4rmOa2HEr+SwEt3vp7wcCgisAqvAmj5Yt/p5X4tZBKzJn38/APs1AFZ6qzSfngfATk1KlVBAYBVYBdZ0wbr25m1s61P+ETXXomhFe/MV7c7Z/rxQYsX2DAAbObCyJRK71bPz0KbuAFak/yQ4+O6gA1JQrMWyKCCwCqwCaxXAeuo/bOuIFemxsv6FPTLZ85NpK/eJ/w0A1zi4slxgXaFPOv8KrAKrwJoyWLdoY9uc9kTUigAre1nyk/78emgRtmdXKKAekSpSLLAKrAJr+mDd9vQnLWYRsLKF0AMAbk6AgZVaPrHSm+OGMHVOVF5xbBFVXjlxyp4JrAKrwJo+WLc74ymLWQSsewDguCN+iFLftIrjjXDQJ5b3BxCClt3f2RqA7eoPLDtMdMKVCgisAqvAmj5Ytz+zn8UsAtaVP2ItZU8BgVVgFVjTBeu3tmxrLc7qHzWBNXu8LPqOBFaBVWBNH6wtzxlgMRNYi8ZY9nYUWAVWgbUKYD13gLWMmMCaPV4WfUcCq8AqsKYP1lbnPWMxE1iLxlj2dhRYBVaBNWWwbtXWWp//z6gJrNnjZdF3JLAKrAJrumBdZ6t21vbCZ6MmsBaNseztKLAKrAJr+mBtd9GzFjOBNXu8LPqOBFaBVWBNH6ztL37OYiawFo2x7O0osAqsAmu6YK35djvreNnzURNYs8fLou9IYBVYBdb0wdrpV89bzATWojGWvR0FVoFVYE0frJ0vH2gxE1izx8ui70hgFVgF1pTBunU763LFwKgJrEVjLHs7CqwCq8CaLljX3bq9dfvNi1ETWLPHy6LvSGAVWAXW9MHa/bcvWswE1qIxlr0dBVaBVWBNH6w9rnzJYiawZo+XRd+RwCqwCqwpg3Wb9tbzqpeiJrAWjbHs7SiwCqwCa7pgXW+b9tb7f1+OmsCaPV4WfUcCq8AqsKYP1p36DLKYCaxFYyx7OwqsAqvAmj5Yd75mkMVMYM0eL4u+o05de9qIGV/IZnxhB93xrsxpsGR5/bDJ07ZevXpzwr6ypvW3bW+7XPtK1ATWssqe7skE1pUvFYF15YslT/Bs6FkrA9Ydbdffvxo1gTVdFpb1agKrwFrohdIQbPK0rVJg3e26Vy1mAmtZUZfuyQRWgVVgrT/sUSmw7n79axYzgTVdFpb1agKrwCqwpgvWDbbb0fa48fWoRcC6PYBBAMYAGA3gHAeGzQC8AGCiyzd15WsBuAXAJAAjAfQqK0h0slUVEFgFVoE1fbDuedMbFrMIWLcO4LghgAkAOgG4HsCl7lfO/Dq3fBCAZwEQsLsCeHdVEmitrAoIrAKrwJo+WPf6wxsWswhYkxzoB2A/AOMBELpMzLnOdCeA49wys3C/oFiLZVFAYBVYBdaUwbr9jrb3zW9GDcA0B9ehLj+lnh99SwAzAGwEYEGwD71Tvz4AwB7BtpcA7BSsa7GcCgisAqvAmi5YN9y+g+3zx7eiVqTHugGAYQAOd1zwIPWYmO8WBFavSBq5wCqwCqzpg/V7t7xlMSsCrGsDeB7A+QErwk98hQICYVJdFFgFVoE1fbDue+vbFrMIWPmZ/wCAmxPAuCFRecXKLKYfJCqvBrtyZZVQQGAVWAXWdMG6UYsOtv9t70QtAlbGS9ndlk2nhjtjzX9zAIyfsrnViwDY/IqJIL4NwGQAH6xpfPVW13aL7bcKmbtmfjOBVWAVWNMH6wG3v2Mxi4C1qtD6GYCGrKo3l4WLC6wCq8CaPlgPvONdi1mWwZpk13rJgryvC6wCq8CaPlgP+vO7FrPGANbdXNcvtvVi6g7gdrec60xgFVgF1nTBunGLDnbwnYOj1hjAyu5b7Fv7fkDRUcFybhcFVoFVYE0ZrDt0tEP+MiRqjQWshGcI1hG5pWnw4AKrwCqwpg/WQ/86xGLWGMD6dwC7A3gPABvVXgjg0YAvuV0UWAVWgTVdsG6yQ0c7/O6hUWsMYN0cwN8AzAIwB8BDrr1XboHqH1xgFVgF1vTBesQ9wyxmjQGsniPKEwoIrAKrwJouWDdt2dGOvHdY1BoDWFsDeNp5q7MBcIgtluU+CawCq8CaPliPvu89i1ljAOs7AE4A8E1nP9FAryveKQKrwCqwpg/WY+9/32LWGMDK/rTJpFYBHHJc01/XTf1dCDB5LcvThIENPWsl5rzarGVHO/6B96OWZbByAAIapyfgNAUcEHYHABcDuDZJ2jyuC6zyWAu9PBqCTZ62VQSsrTrZjx8cHrUsg3UqgCkAmCeN5blPAqvAKrCmGwrYrFUn+8lDw6OWZbDmHpwxAaoB1tc/mGH7HniotWzTzlq1bW8PPPmCvTZymu26xz7WomXr2vz1kdPrPtFHzFgJv0ouFwJMucueHzvb5n/5lU39bLH5c5/5+Ac29tOFNnnOIpswa5Gd+49RtduOvnuovTnlM5syd7GNm7XQTn90ZN0x/thK5dX0SvsNeNbatW9vrdu0sauvudaqeS+V8Fibt+pkP3t4RNQaC1i7ADgawE8Di3GnyW+vBlgPPuI4++11t9aCc+ikuUbQnnjqOXb2JVfWljE/8bRzmyRYL3pqjJ3V94NVwDpsxgL79YBxtdD8zTPjbMRHn9cu//39j+3BwTNrl095ZIS9P3NBkwfroqVfW6vWrW3M+Mn2+eJl1rVrN3tvxOiqwbVSYD3xkZEWs8YA1t+6ObjZQeBeAJ8CYG+s3Ke0wfrGqJm2zfY72PDpn68Czh1at7UXh0yoLWPO9Up6p4XOXSnvL3neEx96fxWwDp2xwK4dOLEWmte9MNEGTZhbuzx42ny76MnRdTD9eMESO/6+YXXryfOWc71aXuKg196yfffbvw6kV/f5ndGqdT+VAutJj4y0mDUGsHLE7G8A8C0BtgLwQu6pWoVWAY89+4Z17t7LDjnyeNuxczf70bE/tbfHfWIbbrRxHUgJ3XC9EAQrUVZOMDV0riRYT31khM36YqnNXrjU5ixaZj978P1aeD467CN7csQntcsMD3z9r3/b2X0/aNJg/dujfe3Ek06uA+nd9z5gp55+Rt162oCtBFg3b93Jfv7YB1FrDGD187twNkNOEctpCsaVAayLSjjHFq7tLAeC2RPALwscy2kV/DQL9Ko/Cta/VWD/NS5K22N9+OlXrFmzZvZgv5dqQXr8SafZL86+6D9AuuFGm9SBthIQLXTOhmBYzm1JsPYb+Yn1eW5CLTB/N3Bi3Sf/EXcNsYFjZ9fGXl8aP8fGz1pojMeW817qO1faAPPXywtYf/H4KItZYwArx17dBMBpbi4Ywo0hgTVNpYD1WAB3uQuy2Vds2MIr3WAxxd4jOz+UnNIG60tDJ9o227Wog+a9fZ+zPb+3f+2nf15DAYuWLl8FlouXfb3Kuoffp18sNcLWr1cy96BLO89DKIAe6yl9R0WtMYA1BA6h1i0sWIPlQmBtA+A5N9f36wA6AOgBgINscwAYeqSPAVjiljmrYqEUgvU+AEcGO/nr7g2A1+gPYAIArr/i4sf0yDnwDL3zelPaYKWn2HPn3azfoGG1cD3t3EvtZ6eeXWurVF6dek4dfAt5l5UoqySownMnPdYZ8760S/qNqQXmZf3H2sTZi2qXj7p7qB1y5+Da5T++MsVeHDcnFajyXtMGqr/ewiXLrWWrVjZ2wpS6yqthw0dV7X4qEQrYonVnO+3vo6OWZbD2AtCQ1QucIjd4wIW7c4bEdq5gFwAvu+UTAfzJLZfqsTYE1sUAWrnzEqyfA9jOxZTfBsDZHOtN1QAr46y8brsOnW2f/X9gbFr16oip9p3dv1vb3GqX/9q7tvlVJeDZ0DlD+FVq+ZUJc+2zRcts+df/sjkLl9nNgybbhU+OroXp5DmLbdynC+viqOc/Mdo+nL/EZs7/0t6c/Jmx+VWl7it5Xg+6auRP9n/G2rZrV9s64Mqr+1QNqnz2ioC1TWf75RNjopZlsA5yrQEK5R549UKniA1JsG4QeKI+VjrWnadSYOWz+USwhpVydwDguAjJdIr7Txu69bbbp+4ZNgS3am5LwiXP69UAahavWSmwnvnEGItZlsGaBEq515NgZcXYJ/VcZE3Aytgs2+AysXXDV26ZIB3glpkl1+kh87r1pmp4rNWEZ0PXzjNIk8+eRchV454qAdYt23S2s54cG7UiwHoPAI7WF9bXMIQYVnofFPz4LwMwCcB4AN8PyjO3mAQrb/AtAEe5O2V8kxMXMoVgZe3/dFdeXxbGWK9w4x1w38MAmDsoCdLkusBaQq+tJFzyvF4NiGXxmpUC63n9xlrMigDrXi7UmQQrZ0hJpk6uuek6LnQ4GUCz5E5ZWf83gA8DO9/dNCuv2GZ2DIDfuJsNwcqih92bppjKK7a75dCHPCcHlPFAT4I0uS6wCqyrFZfNIuSqcU+VAOtWbTvb+f3HRa0IsJIjyfqa0CELOUlvlebT8wA4e7VSJRRQKGDlOAR59lCTz14NiGXxmpUC64VPj7OYrQFYpwHgUKkMFWzquEEnK6xvuTvR0mi18MJPcp7Ue48tAHxntc7UxA4SWAXWJFS5nkXIVeOeKgXWiwaMs5gBICCHBsZK52RKeqz8uuUnPutirnFw5TEVAStrx28D4GvoSfEhyTvM47rAKrAKrOkOG/jttp3tsmfGR201PdYQYyF0KxIK4LTXTOxx5ZMfN8Cv5zIXWAVWgTV9sP7qn+MtZqsJ1q0DkJ0H4FG33jlRecXxqNe48upddxIPWPbbDyEb3Eu+FgVWgVVgTResW7frYlc8OyFqRYD1Ede8c7mrQD8ZwIMAOOgUY6zskRmC9nIAbA3A5lYHloN0P3YXYQ0+4w48sW8SVY7zN9pzCKwCq8CaPlh//dwEi1kRYM0Ed9hn/wwAZwLomIk7ysBNCKwCq8CaLli3adfFrho4MWqNAaxsBVDIMoC26t6CwCqwCqzpg/XqFyZazBoDWH3MgflEAF8DGF1dpGXj6gKrwCqwpgzW9l2sz4sTo9YYwJqkGEe88mOjJrflal1gFVgF1nTBum37LnbtS5Oi1hjBSnjSe819ElgFVoE1fbBe9/Iki1ljACv78HvjAAXsp8++srlPAqvAKrCmD9brB022mDUGsHKWVm9sy8XmVzW5p2oVJhNsaNi+am8rBJi8llWj+2gWr1mJLq3bte9iN74yOWpZByt7F9woiBZWQB6rPNZCL48sQq4a91QRsO7Yxf7vtSlRyzJY/QR7nKJEqYACAqvAKrCmGwrYfscu9sfXp0Qty2D1XVg5CAu7d50A4PDACqAmX0UCq8AqsKYP1lvemGIxawxg5VTX3jhGIZeZ5z4JrAKrwJouWFvs2NVue3Nq1LIMVo4NwNYAFwQ5l/26wNq1pyYTdLMMFAJMXsuqEc/M4jUrEWNt0aGr3fHW1KhlGayc2I+DW/sWAWHuB73ONVzlscpjLfTyyCLkqnFPlQLrnW9Ps5hlGaw+xppreDb08AKrwCqwphsK2KFDV/vLO9OilmWwaszVhqiqdqyrhEEKASavZdXwDrN4zUp4rATr3YOnRy3LYN0swpXcb5bHKo+10Msji5Crxj1VAqwtO3S1ewdPj1qWwZp7cMYEEFgFVoE13VBAy45d7b4hM6ImsMboleHtAqvAKrCmD9b7h8ywmAmsGQZn7NYEVoFVYE0XrK06drWHhs6MmsAao1eGtwusAqvAmj5Y/zZspsVMYM0wOGO3JrAKrAJr2mDtZg+/92HUBNYYvTK8XWAVWAXWdMHaumM3e+z9j6ImsGYYnLFbE1gFVoE1fbA+/v5HFjOBNUavDG/v2qOXzZy3VDZvqS1a9m+Z06AabUazeM1KtGNt3amb9R3+cdSKACsHkpoNYFSAGLbdf8FNmsp8U7dtLQC3AJgEYCQAzvunVCkFBNaVLxWBdeWLJYuQq8Y9VQKsbTp1s38M/zhqRYB1LwfIEKzXA7jU8YL5dW75IADPAiBgdwXwbqWYovz/SzEAABlJSURBVPMCEFgF1kIvlGpALIvXrBRYnxzxicWsCLCSYS0THut4AFs7uDHnOtOdAI5zy8zC/YJiLZZFAYFVYBVY042x0mN9auQnUQMwzcF1qMtPKfCjT4J1QbAPvVO/PgDAHsG2lwDsFKxrsZwKCKwCq8CaLljbdupm/Ud+GrXV9Fg9SD0m5rsFgdUrkkYusAqsAmvaYO1uAz6YFbXVBGv4ia9QQBoQLXQNgVVgFVjTB+szo2ZZzFYTrDckKq9YmcX0g0Tl1WBXrqwSCgisAqvAmi5Y23Xqbs+Omh21IsD6CADOkrIcAKehOhlAcwCMn04E8CIAP3Qq4623AZgM4APFVytB0+CcAqvAKrCmDNbO3e250bOjVgRYg1+yFjOlgMAqsAqs6YN14Jg5FjOBNVOoLO1mBFaBVWBNF6ztO3e3F8fOiZrAWhrLMrW3wCqwCqzpg/WlsXMtZgJrplBZ2s0IrAKrwJo+WF8eN9diJrCWxrJM7S2wCqwCa7pg3bFzd3t1/GdRE1gzhcrSbkZgFVgF1rTB2sNeGz8vagJraSzL1N4Cq8AqsKYM1i497PUJ86ImsGYKlaXdjMAqsAqs6YK1Q5ce9tbE+VETWEtjWab2FlgFVoE1fbC+PXG+xUxgzRQqS7sZgVVgFVirANZJ8+3tiAmspbEsU3sLrAKrwJo+WN+ZtMBiJrBmCpWl3YzAKrAKrOmDdfDkBRYzgbU0lmVqb4FVYBVYqwDWKQtscMQE1kyhsrSbEVgFVoE1XbB27NLDhkz5PGoCa2ksy9TeAqvAKrCmDNauPWzY1M+jJrBmCpWl3YzAKrAKrFUA67TPbVjEBNbSWJapvQVWgVVgTRusPe29aV9ETWDNFCpLu5m0wTrx4wXWvddO1rFzV2u/Y0c796Jf2cx5S+2wI4+x1m3bWfsOnezo439qU2YtrC3ntrSsEGDSLPv8y+XWrXsPO+DAH1ia1y10rSXL64dNpbf1G/CstWvf3lq3aWNXX3OtVfp6DZ2/V6/eVtovKr53x6497f3pX0RNYI1rmdk90gbrjM+W2LgZc2thSXj26LWz9Xv+Vbv/saeM22iHHH6UXXPjLakB1YO7EGDSLLv2uhvtqGOOyzVYFy392lq1bm1jxk+2zxcvs65du9l7I0ZXDa6VAGunrj1t+IwvoiawZhab8RtLG6weYswnfDjPunTrYf0HvrYKRH/T53o78/yLVykLj6vUcpoQTV5r/OQZ9t29v2cDnnsx12Ad9Npbtu9++9eB9Oo+vzNaQ15lJbdVCqwjZiy0mAmscX5ldo9qgHXanMXWqUs3W2/99e30sy9YBaD0Ygnbvz/z4irllYJpeN4k7NJcP/RHR9jrbw+xfw58Oddg/dujfe3Ek06uA+nd9z5gp55+Rt16JSFa6NyVAuvImQstZk0ZrP8CMBzAKABPA9jEEXIbAH8vgpaL6tnnMACdCmy73F2P1/TX5vLZBfYtS1E1wOphNmrqp7bbHnvZC28Mq4PocSecZCefembdut83jTxNkIbX6vtEf/v5KafVxlUF1nyA9YOZCy1mTRmsIRjvB0DwlZLC48Pj7gNwZFhQYLm+YwvsWlv0zfo2NFReTbASlhdc+mu7/Kpra0F6/iVX2P4HHWzT536ZK7BecNElts2221qLHXawLbfaytZdd1075rgfV7UCq5D3lkZZLkIB3XraqA8XRS0vYD0NwO0OUi2dF8vV9QA8DmAMgCcBvAtgJ7cf4XgNgBEA3gGwFYDdAcwDMNV5p23qAZ8Ha3gt7nohgCvdMa8AuNn9B1wAgOvXARgMYAKAPes5d11x2mAdPmGm0VMlVCd8NN923nV3u/eRJ+z6m2+33jvvWluWhnda6BqhF1mt5bx7rAuXLLeWrVrZ2AlT6iqvhg0f1aRCAZ279bTRHy2KWhFgnQbgA8eRoe5HvRmAFwBMdPmmdT/2DC14uDUD0BfAAe7eQtgRdHe68i4Avg7AyqYaB7tt1wO4wi2X4rGG1+LhSbB62HMbwXqTu8ZBAF50y8nsFPefNnTb7bZP1Tsc+PoQ69y1u3Xo1KW2aRU9VkKuWbNm1qJlq9rYK+OvvrwQACtVVi2YhtfNO1jpFT/Z/xlr265dbeuAK6/uUzWo8l4qEWMlWMd8tChqRYJ188SPm5y51JUxp6OVueTjnHMAvAaAgGUKYfcUgH1cObP3ArAuA7CW23YMgLvccjnB+t3g2gTrf7l1eseTgm0FF9P2WCsFxXKcNwRc3pfT+OxvDNeoFFjHfrzYYraaYB0PYGv3Y2fO9cwl77Hyc//1oBKpWLD64/lgjKkSqEylgHU7F2Zwh9Z6vWEowIcduJ1g9et8k/FTocEksK7sgJB3mIbP3xigl8Y9Vgqs4z5ZbDFzv19+4nvjl2aYGE6kIzcMgN+2INiBTl24Hmyq7mIIxp4ApgNgJVEI1osA3OFukzX9ywO4hceHYL0VwEmRR/PHrg1gLoDmANZxsVqBtQI9skKw5H05DWg1hmtUCqzjP1lsMSvCY93WMWRLV4+zVwGQzo9wpiqbPdz8xdnk6oQEWNd3Ta9YefWECyS3cweEx4dg5ec6938fQKzyiqdic6vJLhxBb1dgFVgr2mqgMUAvjXusGFg/XWzjI1YEWD2XmJMJrH9pFKGA8MbrW2bctcZtJCTpnn+rvp2zVq5QgEIBhbzzNKDVGK5RCbB26d7TJs76MmoRsNKh29DxhMtvucr1GxKVV6zMapSJD8cYCJtUjQRwYGN6CoFVYBVY6x9wplJgnTTrS4tZBKytHXPIndFBG3uGDF9yza3YKojNr5TSVkBgFVgF1iqAdfaXNiliEbCmjQpdrxQFBFaBVWBNG6y9bPLsJVETWEshWcb2FVgFVoE1XbB27d7Lps5ZEjWBNWOwLOV2BFaBVWCtAljnLrGpERNYSyFZxvYVWAVWgTV9sE6bu9RiJrBmDJal3I7AKrAKrOmDdcZnSy1mAmspJMvYvgKrwCqwpgzWHr2KGvhIYM0YLEu5HYFVYBVYqwHWZTZzXsMmsJZCsoztK7AKrAJrumDt1qOXfTh/WdQE1ozBspTbEVgFVoE1fbB+vGCZxUxgLYVkGdtXYBVYBdb0wfrJgq8sZgJrxmBZyu0IrAKrwFoFsH7+lX0SMYG1FJJlbF+BVWAVWNMFa/eevWzWF19FTWDNGCxLuR2BVWAVWNMH6+wvllvMBNZSSJaxfQVWgVVgrQJYFy632RETWDMGy1JuR2AVWAXW9ME6Z+Fyi5nAWgrJMravwCqwCqxpg7W3fbbo66gJrBmDZSm3I7AKrAJrumDt0bO3zVv8ddQE1lJIlrF9BVaBVWAVWDOGpcZ/OwKrwCqwpg/W+V9+bTGTx9qI+SqwCqwCa8pg7dXbPl/yr6gJrAJrUcOgzZy3EmJZXC4EmLyWNYapqdO4x0rM0tqzV2/7Ysm/oiawCqwC67J/W1OCcBrQagzXqBhYl/7LvoiYwCqwCqwCqzUGUJZ6j5UCazEvYYFVYBVYBVaBtUgOMBSweNm/o1YEWA8AMB7AJACXFnl57ZaGAqq8Whn3LcaLyMs+pXp2TXX/Snmsi7/6t8UsAtZmACYDaA3gWwBGAOiUBjN0jSIUEFgF1kIvi6YKylKfq1Jg/fIrs5hFwLobgOeDn/hlAGhKGVFgjvsPHFrFfFoVr13N5y50bWkBeF2yoAV/H+VOzxX59z4qsd8pwY0cCeCuYP0EAH8K1rUoBWp/SJJhhQKEitIKBaRF/X8JAmv92miLU0A/oJV/CtJCWqxUoP4lhQLq10ZbnAKCyco/BWkhLVYqUP/SNwFMAdAqqLzqXP/u2pJHBcLYUR6fP3xmabFSDWmxUotCSwcBmOBaB1xeaAeVSQEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUiANBdZKXCS5ntisVSkgBaTAqgqsC+BgV9QNwKGrbs7dWghR6rERgLAsd4IED+x1qAGwdlCuRSkgBRIKcICJ3wJ431n7xPa8rp4BYAyAQQB+AmDrvAqReG6+eJ9x3T33SWzTqhSQAoECuwP4FMDrQRkbSOcpeW+Mz3w4gIcBrAfgGAB3ADgVwLfzJEiBZ90RwMsA2N3zFwAmAti/wH4qkgK5VcCDhDnDAV0A3Amgv+tpQmHyAhKvBZ95GwB/A/Bq8JdB0P4ZwDkAtgrK87TYG8DfAVwTPPSxrgH9gUGZFqVALhXYDEBb9+TsWUJgXO3W1wdwL4ABAPYC8A8A2+ZIpXMBPOae/Q0AYW+b4wDcDGCTHOnhH3VvAOc7bf4JYIcg7vxTADMBbB6U+eOUS4FcKMBKB0L0fwH8CMBw96nL/vMcIo3bOcAvP33fBvDDXKiy4iEPcd76lu6Z9wDwKIBfBRpsGCznZZEx9xcAbOce+CEAtwNoEQig+HMghhbzqcC+AK4DcDeAC5wErMCil/rXIAxAD4Qp/Ex2RU0iC5+LL5RbAXwWeOgMj/wXAHpoF7onDo9pEiJEHoLAfMd9ubB1BNM6AO5zRs9VSQrkWoFvBE/fEcA9zkNjbJWJzWdYA/5gE4YpnzMJx8NWPD58GOQRAN5rJVx3DWDrdm3SmdfHf+6zRcRbAL7noMqHJ1xZuef/dpq0IHo4KVCfAv7Hwh/C4wC2cHHW/3OfugQtE1sC7OSWm2q2sXswrwlbQtArY9oUwE0A7s9RxZ179FpY+mW+WOjBX+JeRP8DYKCLPROqTF4/t6pMCuRLAf8D+D6AvwDgdBT0Slm7TaASJIy55mGsScZRX3NQCCuhnnbl/MtgOVtHUCt6+V6/pvxXwzjqb9yL1T/nUQD+AIAVekwnAXgTACuy8qCJe2xlUmBVBcKeMfRUPwTAUdHZEoAVWPRcGUdl76JbgpYCq56l6azxxcLKul7uE5bemG8dwadkBQ1DIUz0XH04wBU12YxtU4cBONq9XNkxwieGSfi38UtXwLar3/EblUuBvCnASocTXftUPvvOzkv1OrRxlVX0XAkRtgRoyomN2DlpHePK9LZ2cS+WswO4stZ7mWsJ0JS1CJ+NUJ3lPFOWM47KKZ9PC3biC4hTP58XlGlRCuRSgXbuM5+xVE4t0RzAOAdbLwi9VlZAsDsr42Zh5ZbfpynkPwAwEsBFAK5yUxiz2RQ9db5Y+KnbEgDDBPwcDr3YpvD89T0DoTrENbObC4CtRZi+C+C9wEvlVw7b9vo4vNtNmRTIpwKs0WYlTB9XEcMfDvt4s12m//HQY7uticvDipc93TPSc/2jq5TxcOXzP+XmN8rLOAn822DFJRv3M7EH1RcBXNk55GPXWYTzPu3n9lMmBXKpQLJSoavzSPg5x8qpHgBYUcNG3t0BcBAN1vb6Nop5EI0vGPagoiZsIUBvnb3L8tTDjP/Pvmuu/5tJwrU1gJ+5uHwe/i70jFKgoAL+B8JaW37y0jvjICL8tGU31UtdMysezH0JVY7exE/ippi8Hv7ZwnXCla0hrnTdM/0+echDHfi8YcsHwpWdJBg+YUru64qVSYF8KcDBMEa7ygc26r7BdUVknJXdM3/tYMv2qvwRNcV4Yuh9J7tahqCgVhxQhPHnPCRWUPrn97l/bq77shMAfOl0aapxd//cyqVAVAHWarPWm59w9MhYWcVG3vTM2HSI5U3VO/XicICZn7sh/o53n/yMJ4bJA4Rl9OjzkNhll3FmjlbG5nYcUCbUgRqEcE2+kPKgkZ5RCvyHAvz8p7EZFWt733VtVFkJMRXAjW6Alf84sAkVcADmP7kXCJ95NoAN3PMlm5MlodKEZCj4KIwjs/UDKy9ZGcXKy0LJ6+I9Vb9eaF+VSYEmrQArqPoB6OmekgOHcDBiJm5jjXcHt95UMwL0FQA/BsBRqTie6mDXxKqpPnOxz+XhyL+B8QD6uiZ4eRvIvFi9tJ8UqO3zv9wN80c5vKfB9okvAZiUo6H/+HnLnlOskOOnL0Mj7ILJijwmeml80eQp+d53jDvzb4ND/bHCjq0iOLgME2PMSY/ebVImBfKhAEdgYi8qJvaSYZMZdkFckujrT2+E3RHZfTMviT3N5rmmZGzwz9TJ9RriJzBfNnkZ5o7xdN+cin8HHFuXFXWMvXOISA4byfAQm5yx80RedHF/FsqkwEoFWKHAz32Om8oeUxxMxXsd7FXEZjJ5GsbNf+J6hdgOlQ38GUvkICqcSoSJ4yFw3Nmm2ALCPeJ/ZGz9sciNJ/uAmwyR/f3ZYYTzVdGT5RxeHNA877Py/od4KsiPAoyPjXWNtTlYxnzXAoAK+M9/9uX+Kgc1/8n/9bNcszIO3E24MgRwmavIYnfMvCaGQAhXdtNlYsuQI9wA1b7Hlf/bSb6k3CHKpEDTVYCftfROT3aPyDapHGGIA2NcnHhsfg7nadbM0wG86FoCcPpujsTERM+VXXnZ1Izx1ryAwz8nn5mJ8dTFQViA8VS2X2blHl9ASlIgtwpwHEz2b2fiD4fdUglP9vn/xAGXjd1ZceN/UP4H5g5rshm9MbZd5ac+46iMH9L4qUtw5KXxv//bYM6RuwhOP2jK792QkZx9lomhkbzMvuseWZkU+E8FWJvN2m2OJ8oOAJw6hGEBVkZwJlF6bJyXiZ95eUj+pcGcEyCyzS5jzr750JkufpgHLfwzek34N/KEa7vMCSLZppmJA5l/7qb2dkXKpEC+FWDvIFbIcJBmzuvOHjQcQ5U1/uyySo/Nj4bvf2BNTbH/BsDPfp98bJDDI84IJkVkKITNrfIySlXYc4xaMGTke9ex1x0H3PFwvd61JPEaKpcCUsABNBSCPa3YjMY3qwm3NbVlNjH7F4BTggfzbTTZUoLxZlZe0bNnE6s8JLZN5Tip/qXKViNPurFl/fM/6zpK+DAAy5vqy9c/s3IpsFoKECjs702Y+FGIVutEjewgNp9iawg/uj29Vg9XVubRc/cTBTayR1vt2yVM6akyBMDEaanZaoRfNEz8++DfCXvmKUkBKVCPAgQJu6wyrnpwPfs05WLOIEu4+jmY+KyMqT4XwKQpPz+fLTn+AcMf011oiC8fxtsZf2flJrv10tvn/GaqsGrqfxl6vjVSgHD1P5I8ftZ5uLKy7oeuooYtJfKQ2J6ZXZVZaXdF4KH/BMAHbvByNsdj438OSMPBzDkQD71W78XmQSc9oxSQAquhAOH6bzduqK+sWY3TNKpDGDtmKxAOi8jpZghOdlf1iaP9sy0vBzD3iV1YJ+RwjAT//MqlgBQoUQG20/Q13iUe2uh2Z9tczp7a3905v1QuBHC5W/exZgKXrSLYVpWJPdHy1JXXPbYyKSAFpEBxCjBWyjFmfXyZUOW4sxwWknOYsfsuZ+X1rUR8k7Tizq69pIAUkAI5VcDHl9ms6jU3KA+n7WbvqhdcC4GcSqPHlgJSQAqsvgJst8uWEb91p/Ceqf/8X/0z60gpIAWkQI4VYLMqhgU4oplPeWwh4p9duRSQAlKgLApwsJWFblYAQbUskuokUkAKSAEgnOpbekgBKSAFpEAZFPCeqs/LcEqdQgpIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJNWwEOUM3ZDzi6fV8A4aj3pT45xxvl9M1MHOGpoUGtORD47m7fUrJpQV/78Lj6ysN9OCtqKYmT/bHPv5IUkAJSoCQFQthwkrvzE0f7uaoSxQVXQ7AW3CEoXF1o1QfQ+sqDS9ZONx2ux5ZX9x5j59V2KSAFmrgCIVg5+v/tAOhNvu5Ga+Kwds3c/F1DAIwMJgNkMyIOjzc+mDTRe6yvAGDfeaYDALznxh3leKWcMvxTAB85b5nzhHGAkn8A4DVoHDCciTO5DgQw2nnBHDS6UPfQEKwc+GSYOyacLobP+n+unPfBazK1cYNx8xg+N8dVZRJYnRDKpIAUKE0BD1Z6ppwihJMCEqyc756DMjMRThy4mWkdAJxhlNsOd4OLELyct2lBEArwYCW8Zgbn4hQtTElocSbXPdy2Fm6mW67eAoDTaDNxKhMrAqz+Guu6EIefZpvH/tidi+fkS4GJkOUUKkzsMfWyW07eoytWJgWkgBRoWAEfY2WclTOHcmxRgnVQcBhnn6Xnyn1oHA5vfwA3u0Gd/a6c2jnpsXKKGoYYkikJLfat9+dnTm+W05xwuXVw8LwiwMpzc1R+GqeV3tUdz2f1oQ2ek+fmNZYkrs3py5mS9+iKlUkBKSAFGlbAe6zhXgTrgKCAn+h+QryguKxgnQugJjy5Wy4VrLx3juTvK+HoObOMKQlWjujPbqefuO3JTGBNKqJ1KSAFilKgGLAyFMC4pR8Jvz2A9V0o4HkXg+WMpBwuL+mx1hcKuADAVcEdMhRwUbDu58liKMCHIQ4sIhRwKICn3XkYK10agJWhgGPdNp6THjrTWwCOcsuMG3MuKiaB1QmhTApIgdIUKAasHFP0d25CPDbLYpiAU1mHlVccyJkzkCbByrshEOkd8tOc+zERzqwIo0fKyitWSD3myjh9yZ/dfmHl1V/djKcNVV4xBvysi9HyZRB6rHzWP7i4K+OovvKK8WLOJMv747V9TFdgdf8JyqSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUSEeB/wcrx5IXYIHQIwAAAABJRU5ErkJggg==)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oIuJbP8WbTOJ"
      },
      "source": [
        "# epoch = 128\n",
        "epoch = 25\n",
        "batch = 2\n",
        "\n",
        "model = Sequential()\n",
        "make_marikondo_model(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NVTIfmPsQWdi"
      },
      "source": [
        "# Round 13: back to square 1\n",
        "- simpler model with no extra dropouts works just as well\n",
        "\n",
        "- With dropout before dense layers\n",
        "              precision    recall  f1-score   support\n",
        "\n",
        "           0       0.77      0.93      0.84       373\n",
        "           1       0.86      0.72      0.78       249\n",
        "           2       0.00      0.00      0.00        36\n",
        "\n",
        "    accuracy                           0.80       658\n",
        "   macro avg       0.54      0.55      0.54       658\n",
        "weighted avg       0.76      0.80      0.77       658\n",
        "\n",
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAgAElEQVR4Ae2dB7gdRfnGXwXkQughID2F9J4gTaoCCooFAZGiINJEkN6FEIMICH9EiqgU6RCKCaGFEmqAhPTeO4EkpJEeYP7PezJzM3c59869N2fnnHP3nef5Mruzu7Ozb+757ew3DVCQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBUpAgc0APA9gCYDeG1CekwD034DrS+nSAwFMKKUCqSxSQAqko8CJAD4CsAzAXAAvATigALc6BcAgABsXIK9yyMIA2LMcCqoySgEpkK4CFwGYB+AYAI0AbALgaAC3FOC21wB4pAD5lEsWIbBm5QVTLv9fKqcUSEWBrW0t9bgact8UwO0APrbGbaYxHAJgNoCLLZxZ2z3NHrsewBoAa+09TgfQIwHapgAIIwecUwFMBfA5gGkA6AJgYPq7dpvR/gAGWxcDY+678CaAPwN4z+ZDF8L27mAiduW/zCv/zwAcBWAigIUArvKu2RvA+wAW25r9nQC+ZY+/bZ9luX3eX3r6XA7gEwAPe2m8rIW9Rzebx84A5ttzvNtqUwpIgXJS4IcAvvDAlq/sPQF8AGAHAE0ADLTg4rkEE6/nOazpEkgrAGxrM0qCNLnvg5W15aUAWttrdwLQ3m77YN0OwCIAdDMQyL+y+43tuQTrFACtANDHy/2/2mPJyJX/Wlv+MyzYHgOwpb3/SgDN7IXdAexr78uyjwNwgZdpssbq8r/JvoxYHgdzdxnvORbA5gBeAfA3d0CxFJAC5akAa4SsSdUUCCkC04UfAJhudwgJgsfVOJlMtwLhw5AEaXI/CVbWBH9hgWizyEU+WJ3f1j/OWiTPYSBI6YJw4fcAXnY7idiVfyObTpgSjvt45w0BwFpsvkCoPucdyAdW1torvHOSYOWhvgBGARjpfQ14l2hTCkiBclKgNjVWgtPVHPlsbewnPrfzQYLQPcyKkARpct8HKy8htF+1n9ov2Hsx3QcrP6uTPQyeAHC1vSfB+ju7nbzWS85tJsvPFwThyHK5QBfEyXaHteB+9mXE2jVr5++4E+21fuMV85/jHedm8p5Mo0+b92XtVUEKSIEyV4A+VvoEj63hOZI11iMSNVb6WP1QE1gvBfCsdzJrtgSKX+PlYX4y3+pBywdrvhor3RN+jTUtsL5uP9VZs2VgjdX3/earsSb1SYJ1C+u6+I+FMF0dClJACpS5Amx4+tR+7tLPR1/pkQButs/Vy/pV6V9lIxBBwjSGJCSYVhNYDwewAMDuAAj1Ph5YdwTwU9sz4ZsA2Pj11rrbVKmx0pdKlwG7iBHIbCTivmugSrPGyq5j9Md+w9am2R/VByvdKnzxuJBPn2TafQCetBf8C8BT7mLFUkAKlLcC9LWyHytrr4QDP8NdSzv9g3fYVnC2+nPb+QyTkKAKNYGVx++yIJxsP31djZWNVQQpBxMQlARkOyurX2NlEvvY0vfJcxn7fW7TBOtBAMbbVn+6ANho54P1bKsTy398NS8eXzO+SOgqcLVU1l6pi+sNYR9fkRSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkRWYKMK843Nmsg2a2I6dekmsxp069bdyLqbxo0bs7dHQcM3t9y9Vr+3GkbaFbQ8yiwFBQjVii7nyrqcaz5ZskZmNVi51hiZyb1cCv2zq+1vznYdLPTtlV8MBWr7n5wF+Aqs618sguq6Fwtr7YX+HX5j8x1MRbfzgyawFlr5iPkJrOtr6wKrwJp8oaQG1u4XmIqACawRQVjoWwmsAmu+F0oSMFndTw2se11oKgImsBaadhHzE1gFVoG1en9yOmDd0VR85+KgCawRQVjoWwmsAqvAGhmsjXY0FftcFjSBtdC0i5ifwCqwCqyxwfptU7Hv5UETWCOCsNC3ElgFVoG1CGDd70pTETCBtdC0i5ifwCqwCqxFAOv+V5mKgAmsEUFY6FsJrAKrwBoZrFvsZCoO+FPQBNZC0y5ifgKrwCqwFgGsB15nKgImsEYEYaFvJbAKrAJrbLDubCoOuj5oAmuhaRcxP4FVYBVYI4N1y51NxaG9giawRgRhoW8lsAqsAmsRwPq9G0xFwATWQtMuYn4Cq8AqsMYG6y6m4vs3Bk1gjQjCQt9KYBVYBdYigPWwv5qKgAmshaZdxPwEVoFVYI0M1q12NRVH3BI0gTUiCAt9K4FVYBVYiwDWH9xqKgImsBaadhHzE1gFVoG1CGD94W2mImACa0QQFvpWAqvAKrDGButupuKovwdNYC007SLmJ7AKrAJrZLBuvZup+NE/giawRgRhoW8lsAqsAmtssO5uKn58V9AE1kLTLmJ+AqvAKrAWAaxH320qAiawRgRhoW8lsAqsAmsRwPrTe01FwAJgrQAwCMAIAGMAXG/Z0AzAhwAmA3gSwLds+qZ2n+k83rTQLFF+ngICq8AqsEYG6zZ7mIqf/TtoAbB+A8AW9qe8iYXlvgCeAnCCTf8ngHPs9u8BcJ+BxwldhbQUEFgFVoG1CGD9+X9MRcACYPWRsDmAoQD2AbAAwMb24H4AXrHbjLnPwOM8j3BWSEOBGGDdeu8/msGjppkRE2aZMZM/Nr3++YKp6LIeaHc9NsB8vnxVlbSTLv2PGTvl49z5T7w4uMox/9pCbucDTJppQ8ZMMfsfeLBp2bqNadWmrfnzX281vN9Pfn6sad+xU8523X2PXJxmOfLlXczlrvv0e8m0bNXKNG/RwvS84UZTzLKkskrrNnuYzY65L2gAplu4fmTjMxMM2AjAcADLANwEYHvrAnCn7QZgtN1hvKs7AGCKPd9L0mbBFIgBVsKv8X4X5uC4xV7nmUEjp5mDTrklt7//iTeZR5//sApY2/+khxk2bqb59oGX5M7Z7dDLGyRYR0yYYfq/9WEOppNnf2aat9jTvPXh8Ny+g91Zf7jAXHrVtVXS3LE042LBbNmqL0yz5s3N2AlTzJLlq03Hjp3M0BFjigbXNMD6zW33MJsfe3/Q6lBj3QbAAAAHCKwFQ+OGZRQLrK5mue2+F5ihY2eaA0++2Wze7Q/mzUETTNPDrqwC1lsf6G/Ovv6RKDB15WKcJqhqk/cPjvqxefK5FyvLMXfxarPzLruagUPHVKbVJp9CnFMssA54e6A57PAjKkHas9dfDK1Y5UkHrE1No+MeCFodwEoIXAvgUrkCNoyHBbs6FlgJ0eHjZ+UA+rf7++egefFNvc2ltzyd2/ZdAX3fGG4I14HDJpsPR0w1R//+ziiQLQSQ6pvHoJETzS677mYmzVpQCdHnXnzddOrSrXK/vnnX57pigezRJ3qbU087vRKk9z3wkDnrnHMr92OXKy2wbnH8gyZkAbA2AcCaKsNmAN4B8GMAvRONV2y0Yjg30XjFRq6yCVfbrg8jre+DzuQLANC5XNfQE8BhgYt6ALgkzzkU3Ama5/D6pFhgdTXDHQ+4JFdL/f5vbzPvDZ1sGnU/72tgfeGtUabP68MN3Qatj7rWzJq70PA6l0dacX0AVIhrpsxZaDp17mrue/jJKhD99W/PNNf1uqlKWiHuV5s8YgPM3S8rYN3qhIdMyAJg7QRgGACyhv5T1lgZmttuWOxWRciymxUDu2dxn+nspsXzyiKwxe1970HoSN7ZOqC5nS/Q+bwhoTqwso+ac1rXmH9ssBKK19/9vOl5dz8zd/4SM33Ogpx9+eWXZvKMeTl4/qv3O+aMax+uBOkbH4w33z3xpsr9hgTWWQuWm0O+d7jpccPNVQA6+7MVZvsmO5ihY6dWSa8NFAtxjgNd7DgTroDtmpqtfvVQ0AJgrfF33ZAOHgPg+cQDnQ9gDYBR1rnMw2zBu9V27KWzmW+awRaE//K6QDwI4Fib31EAxgMYAuAOAP1sOsF6P4A3AUwFwPsxPAFgpa0132LT8kYxwLrroZdX1ji32ecC8+6QSebn591TBZS+K4Cf/g/3/SB3fJdDLsvVWHc++LIq56cB10IAqS550Id67AknmTPOOe9r8HzsmefNvt898Gvpdcl/Q86NDVR3v89XrjVNmzUz4yZOrWy8GjJ8dINyBWy0XTOz9YkPB01gXYcsdtZl14eJAO4GcLAlGbtM+DVWA+B4j3LbedsPAzja7juwsgo/CwBHVDA8ngDrQFtL5j0+A8DOwqEaK7ttrOvCsckWqQNrr+NuyLXyj5ww24yeNCdXY02C0Qcrj/394ddz3a1GTZxjTrn8/tTLyHtuCIjqc22flwfwb8G0bd+hsnvVI7375Mpx/ImnmJtuuzN6mdxzONAVI36u7wtmz5Ytc70DevTsVTSo8tnT8LESrNuc9EjQBFZLPAD8tD/EDi/7BMCpeVwBX9jz3FW/sKMmWKudA+AKe8CBtQuAt9zJAH6SACv9ui6Ms33VQmB15yNGjTUJ0VLdd1BRvKaoMCsGzKu7Z1pg3e6Ux0zIBNZKTFXZ4Gc8XQPJGitdAS6wNvopAHbkZeCnPY2htmD1G6/oVyVUBVZvAEJtQS6grqmsIVcHmqylpwLWxs3Mdr9+LGgC6zoQtgbQct1m7t9eAO60/lX3Gc8DPljZek+wsrsEXQkEYxKsPEZXAGHJ8GiixpoPrI0BzLDn1xipxrp+BJjAKrAmXxxpgbXxbx43IRNY16GrOwD6O8faLhDPWt/qeQAmJBqvfNgRwBxe9h6AB/KAlefS7+oarziRAuHKQAjnAyuPPWZBXfTGq9rWGIt9nsAqsMYA68aNm5smpz0ZNIF1HeTS/NfNYsNJE9gwdmGhbqYaq2qs+V4oScBkdT+NGmsOrL990jQJmMBaKMpVnw9Byh4HrA2ztlqfAQd5cxdYBVaBNe7sVhtv39zscPpTQRNY8yKrPBIFVoFVYI0P1h1/19uETGAtD4bmLaXAKrAKrHHBusn2LcxOZz4TNIE1L7LKI1FgFVgF1iKA9axnzE4BE1jLg6F5SymwCqwCa2SwNmlhdj772aAJrHmRVR6JAqvAKrDGB+su5zxnQiawlgdD85ZSYBVYBdb4YN313P+ZkAmseZFVHokCq8AqsMYH625/6GNCJrCWB0PzllJgFVgF1rhg/dYOe5rdz+sbNIE1L7LKI1FgFVgF1vhgbfrHfiZkAmt5MDRvKQVWgVVgLQJYL+hnmgZMYM2LrPJIFFgFVoE1PlibXfiCCZnAWh4MzVtKgVVgFVgjg3XHPU3zi14MmsCaF1nlkSiwCqwCa1ywbrpjS7PnJS8FLQBWTo4/wE7MNAbAHy1xOJUoVyLhpE00rpfnwpV2lVZOY/oDl6g4BQUEVoFVYI0P1paXvmRCFgDrTgC6WSRsadfaa5dnjmZHDR4bYdfH48T7nAN6Q1eJdnkrTiogsAqsAmt8sLa67GUTsgBYkz/lPgAOrwGsrK3SXHgFwH5uR3GBFRBYBVaBNS5YK77d0rS98pWg2fXy1q2mvG5VZa6unC9w2aaZALayYOU6eyMB3A9gW3sBl4k62bv4PgBcl08hDQUEVoFVYI0P1nZXvWJCVssaK1cXGQLgGMuHHe0n/jcB3GDhykMCaxoArS5PgVVgFVjjg7X91f1NyGoB1k0A8JP+omp+3/5qzXIFVCNSKskCq8AqsEYG604tTYdr+gctAFauf/cQgNsTYGCjlgtc0ukJu9M+0Xg1VY1XTqYUYoFVYBVY44J1s51amU7Xvha0AFgPAGCsL9XvWvUwgFE2vS8AH7RX294A7G51ZAo4UZZOAYFVYBVY44O183WvmZAFwOp+wopLUQGBVWAVWOODtUuP103IBNZSJGYtyySwCqwCa2Sw7tzKdL3+9aAJrLWEWCmeJrAKrAJrXLBuvnMr0/3PbwRNYC1FYtayTAKrwCqwxgfrXr0GmJAJrLWEWCmeJrAKrAJrfLB+54YBJmQCaykSs5Zlatuxqxk2Y6lsxlJzxlOjZFaDlWurh02WjnXr1p1dmgoaGu3Syuxz45tBE1gLKnvczATW9S8VgXX9iyVL8KzpWdMBa2uz71/fCprAGpeFBb2bwCqw5nuh1ASbLB1LC6z73fSWCZnAWlDUxc1MYBVYBdbq3R5pgXX/m982IRNY47KwoHcTWAVWgTUuWLfYtbU54G/vBE1gLSjq4mYmsAqsAmt8sB5467smZAJrXBYW9G4Cq8AqsMYH60G3vWtCJrAWFHVxMxNYBVaBNTJYd2ttDrn9vaAJrHFZWNC7CawCq8AaF6xb7tbGHPr3gUETWAuKuriZCawCq8AaH6zfu2OgCZnAGpeFBb2bwCqwCqzxwXrYP943IRNYC4q6uJkJrAKrwBoXrFvt3sYccdcHQStlsP4DwB01WFyKleDdBFaBVWCND9Yf3v2BCVkpg/U3AGqyEkRd3CIJrAKrwBofrEfe86EJWSmDNUmpzZMJWd8XWAVWgTU+WI/654cmZOUA1v0AjAUw04K0M4C7sw5VPr/AKrAKrHHBuvXubczR9w4KWgCsuwEYYLk2BsAfLc+2A/AqgEk23tamc7lsukUn2xVcuxWCfx8CYEGGeZmN9rYzuymwCqwCa2Sw7tHW/ORfg4MWACuXtXZw3BLARADtANwM4AoLNMY32e2jALwEgIDdFwCZuMHBZeKDdcQG59oAMhBYBVaBNT5Yf/rvwSZkAbAm6dMHwOEAJgAgdBkYc5/hXgC/stuM/PO85LptPg1gfwBDAWwC4BIAT9Qti4Z5tsAqsAqsccG6zR5tzTH3fRQ0ANMtXD+y8ZnVUKipdXNuBWCxdw5rp26/H4ADvGOvA9jL26/X5vYAHgXwKYD5AB4B0LheOTWwiwRWgVVgjQ/WX9w/xISsljXWLQAMAXCMRZMDqSPVIruRCljdTRQnFBBYBVaBNS5Yt23a1hz7wJCg1QKs/Pp+BcBF3s/a/8RP3RXQHMDztrY6DwD9EUzLfBBYBVaBNT5Yj39wqAlZAKz8zH8IwO0JiN2SaLxiYxbDjxKNV4Ns+gZFHwA4BcDG1k4uVKvYBpWqBC4WWAVWgTU+WE/47zATsgBY6S/lCrIjAQy3xpZ/ujjpP2V3q9cAsPsVA0F8F4ApAEYVwr/KTHnzZFCvAPVjrbLsdz7AZDUtSwsG1vSsaax5tV3TtubEh4YFLQDWJM+i7pPWNPblYp8utp7tAeAyADdGLUmJ3kw1VtVY8708aoJNlo6lAtZm7cxJDw8PWimDdRqAqQAYJ43pmQ8Cq8AqsMZ1BWzXrJ05+ZHhQStlsGYenCEBYoP1uTeGmFbtOlZaoy22NJdce6N5c8R0s88Bh5rdmjbPxW+NnFHlM33YjPUATGs7H2AKnfbu1IVmycq1ZvbilcblPWjmYjNz0YqczV+2Ohfz2Nm9Rxuez3N5/JYBUyuvcdemFRezVtqn30umZatWpnmLFqbnDTeaYpYljRpr42btzG8eGxG0cgFrBwDHA/i1ZyHuNPjjscHqQ/GjqYtM4yY7mBfeG21+c9YfzXmX98jBlPGpZ1/QIMF68xtTzJ/7T6oCVh+Or4yfb/436pMcQB8dMicHVh6/qM9YM33hCnPmU6OiwLVYMFu26gvTrHlzM3bCFLNk+WrTsWMnM3TEmKLBNS2wnvr4SBOycgDrdXbCAg4QeADAJwA4GivzoZhgvfvh50zn7vvkALpH8z1N/0ETc9uMue9DOMa2D7g0t6/oN75asH62fI25+sUJOXi+MWmBue+DWZUgHfvJ5+aGVydX7qdZxmKBdcDbA81hhx9RCdKevf5iaMUqT1pgPe3xkSZk5QBWdi/4JgDXE2BHO/OLwNqxa3SAOUj+9LiTzeXX35K7/xZbbV1ZjqHTlxh/352fdpwmqPy8qwMra7PTPltRCc6HBs82H81cbM7qPcrwmuWrvzD3vDej8rifZ6G3iwWyR5/obU497fRKkN73wEPmrHPOrdyPXa40wLp983bmd0+OClo5gNV1huXQL46nZZ+u8QWg6rI65NHE9p3lRDAHAvh9nmvZB831SWOteo63/608529wUrFqrIMmLTDbbLudee2jyV8DKwG65VbbVII2baC6/AsNp+ryqw6sAyYvME8N/7gSnATqqxPm5/yrw2YvMSM/Xmruend65fHq8i9EemyAuftlBaxnPDXahKwcwMq5V7cBcLbtOEu40SWwoaEuYD0BwH/sDdntKzRtYQ87WUxty8jBD3UOxQLrbf9+3Ox74KGV8My6K4AQZaPWpc+Pqxack+cvN9e+NLHa44UAqsvDgS52nAVXAGusZ/YeHbRyAKsPHEKtk5+wAdv5wNoCwMt2YoR3ALQB0MXOPsMJYFgjfRLASrvNIWj5gg/WBwEc653k7nsIAN6jr52PkftvWv8xa+SceIa182pDscB6xNHHmB633F0J1l+feX6Vxis2ZrmaZKzYQSXtOF+N9fa3ppkJ85ZVgea5z4w2NJbntjenfu14muWMDVR3v89XrjVNmzUz4yZOrWy8GjJ8dINyBTRp3t6c/fSYoJUyWDkRbE1WLXBqecABzj+dw8la2oR9ALxht08FcKfdrmuNtSawLgfQzOZLsC4BsKv1Kb+fmCrMnrY+KgZYB46ba7beZlvz9qhZlfAcMHya2Xv/g3Pdrfb+7iG57lexgOrukyaoXN4fzlhkFq9YY7748iuzcPka8+Cg2TlwvjdtoXn4o3Xb7lwCeO7SVebjJSsNG64u7ze+CnjdeWnEDnTFiJ/r+4LZs2XLXO+AHj17FQ2qfPY0fKxNWrQ3v392bNBKGaxcuqA6c8BbT5m6byXByim8XE3U+UrH2WzTAiufzwWClcsyuHAPAM6LkAyc1zE3x+NOu+xWCTcHmKzGaQCqXPMsBlBL8Z5pgfUPz441IStlsCaBUuj9JFjZMDa3mptsCFjpm2UfXAb2blhjtwlSzrPoQnKfNWTet9pQjBprqYK7XCGYRrlLEXLFKFMaYN2hRXtz3nPjgiawVsXWQADH2ST6N7lwIYMPVrb+z7Dp1UW+j/Uab+2an9lZbXhdEqTJfYG1DiO20gBUueZZDIiV4j3TAuuFfcaZkGUZrF8BmO0ZJ5ylv5ONV+wzy5Vhr7Xk9MHKpMdsz4DaNF6x3y2nPmSenFDG1ZSTIE3uC6wCa738sqUIuWKUKQ2w7rhne3NR3/FByzJYq6ttlk26XAHr5yAo19plGuUuBsRK8Z5pgfWS58ebkJUDWPlJzkYcV3vcHcDeZUO/FAsqsAqs+cBcipArRpnSAuul/cabkJUDWNk6ztmzXQv9tgAGp8irsslaYBVYBda40wZ+e8/25soXJgStHMDKZa8ZOOLKBTdvgNvPZCywCqwCa3ywXvXiBBOycgDrhwA2AuAAy3H7PmQzCVU+tMAqsAqsccG6U8sO5pqXJgatHMB6kh32yRb8GwBwiVjXJSqzUBVY10OVfWvzASaracXwZ5biPdPwsRKsf3p5YtDKAaxkCMfsnwvgD6yoZZqm3sOrxroerlmFaL7nLkXIFaNMaYB155YdzPX9JwWtFmC9H8C8xIRO7P/uz4rHlVtduBLAZFux/IFL3JCYvQDy2Ybk2SCuFVgFVoE1riuAYO356qSg1QKsB9m5UPyZ8vyBRT6j2tl+8JvavvZcBpvu0Q0KnOiaS2Az5nrbXwAYs0E5NpCLBVaBVWCNDNZWHUyv1yYFrRZgJYWSEzpVB1bWVmkuvAJgP7dTqJgzXrm5UQuVZ1nmI7AKrAJrXLDu0qqDufH1yUEDMN3CNTdhEgBOnpQM+cDK61iRpKuAXUsZOArTn5DpvsRUpPa0DY9Ye818EFgFVoE1PlhvemOyCVk9a6wc/s5PfE7WxIZ6wpUhFbByDL+zS+w4fVaFMx8EVoFVYI0P1psHTDEhqydYfab5tdlUXAFcpdXZ1QDY/arCL0FWtwVWgVVgjQvWXVt1MH97c0rQ6gnWnTyWXQjgCbvfPtF4NXVDG69YLf6bdzNtegoIrAKrwBoZrK07mP97e2rQagHWx+38z2vtDHunA3jYNtLTx8olm3zQslLJ3gDsx3+kh4E6b7oF9rhEiUIeBQRWgVVgjQvW3Vp3MH9/Z2rQagHWPL/oOEluCCsnYSG9TwFwjGdxSlHCdxFYBVaBNT5Y73h3qglZOYCVS107YysZt11rWQljL/2iCawCq8AaF6y7t+5o7npvWtBKGaycG4C9AS72Ym67/fTJVeJ3EFgFVoE1MljbdDT3DJwWtFIGKxf24+TWrkeAH7tJr0scfekWT2AVWAXW+GC99/3pJmSlDFbnY02XTmWcu8AqsAqsccG6R5uO5l8fTA9aKYNVc64GoC+wCqwCa3yw3jdohglZKYN1uwBXMn9YYBVYBda4YG3apqN5YNCMoJUyWDMPzpAAAqvAKrBGBmvbjubBwTODJrCG6FXCxwVWgVVgjQ/W/w6eaUImsJYwOENFE1gFVoE1Llibte1oHvloVtAE1hC9Svi4wCqwCqzxwfrokFkmZAJrCYMzVDSBVWAVWGODtZN5bOjsoAmsIXqV8HGBVWAVWOOCtXnbTubJYXOCJrCWMDhDRRNYBVaBNT5Ynxo2x4RMYA3Rq4SPd+rSzcxeuEq2cJVZvvormdWgGEtNl+I901j+unm7Tqb38I+DJrCWMDhDRRNY179UBNb1L5ZShFwxypQGWFu062SeGf5x0ATWEL1K+LjAKrDme6EUA2KleM+0wPrciLkmZAJrCYMzVDSBVWAVWOP6WFlj/d/IuUETWEP0KuHjAqvAKrDGBeue7TqZviM/CZrAWsLgDBVNYBVYBdbYYO1s+o36NGgCa4heJXxcYBVYBdb4YH1h9KcmZLUAK5eXmgdgtIcYzuj3KoBJNt7WHvsGgDsATAbAFVy7eddos9AKCKwCq8AaF6wt23U2L42eF7RagPUgC0gfrDcDuMJygvFNdvsoAC8BIGD3BfBhoVmi/DwFBFaBVWCNDNb2nc3LY+YFrRZg5S+5aaLGOgHATu10rMgAABjoSURBVPYnzpj7DPcC+JXdZuSf5yVrsyAKCKwCq8AaH6z9x843IQMw3cL1IxufmedHnwTrYu8c1k7dfj8AB3jHXgewl7evzUIqILAKrAJrXLC2at/ZvDZuftDqWWN1IHWYWGQ3BFanSIxYYBVYBdb4YH193AITsnqC1f/ElysgBkTz3UNgFVgF1vhgfWP8AhOyeoL1lkTjFRuzGH6UaLwaZNMVpaGAwCqwCqxxwdq6fWfz1oTPglYLsD4OYC6AtQBmAzgdQGMA9J+yu9VrANyCqvS33gVgCoBR8q+mQVMvT4FVYBVYY4O1i3l7wsKg1QKs3i9ZmyWlgMAqsAqskcHaoYt5Z+LCoAmsJYXKuhVGYBVYBda4YG3ToYsZOGlR0ATWurGspM4WWAVWgTU+WN+ftMiETGAtKVTWrTACq8AqsBYBrJMXmfcDJrDWjWUldbbAKrAKrPHB+sHkxSZkAmtJobJuhRFYBVaBNT5YB01ZbEImsNaNZSV1tsAqsAqsRQDr1MVmUMAE1pJCZd0KI7AKrAJrXLC27dDFDJ66JGgCa91YVlJnC6wCq8AaGawdu5gh05YETWAtKVTWrTACq8AqsBYBrNOXmCEBE1jrxrKSOltgFVgF1thg7WqGTl8aNIG1pFBZt8LEBuvkjxebzt32Mm3bdzStWrc1F1x6lZm9cJX55Um/yaW1adfBHHX0z82EmQty6TwWy/IBJlbaZ0tWmO57fcd06NjJtG3bzlx59Z9MrHvnu8/KtdXDJu1jffq9ZFq2amWat2hhet5wo0n7fjXl361bd1O3X1T47LYdu5phM5YGTWANa1myZ8QG66zPVlZCc9qnn5su3b5j+rzylhk3fV4lQM8453xzxbV/rtzPAliXrfrSfPrZ0hxMFy9bbfb6zt5mwNsDiwbXmmCT5rFlq74wzZo3N2MnTDFLlq82HTt2MkNHjCkaXNMAa7uOXc3wmUuDJrCWLDbDBYsNVh+Sk2YvNB06dTF9+79dCVGC95TTzjBXXterMs2/Js3tfDW3YqTNX7TMdO7S1bz5zvuZAytfJocdfkQlSHv2+ouhpQnzmvJOC6wjZn5uQiawhvlVsmcUA6wz5i837Tp0Mps3amTOOf/iSoAe/6tTzPZNdjD7HXCQIXTThGi+vIsBUf+eS1esNR07dTaNGjUyF158adGgyjLVBJs0jz36RG9z6mmnV97/vgceMmedc27lfpr3zpd3WmAdOetzE7KGDNYvAQy3qyA+D2AbS8idATxdC1ouq+acnwFol+fY1fZ+vKe7N7fPz3NuQZKKAVYHtTHTPslB9LV3h1RClND99W/PNLf+497KNHd+2rEPuWJuz/l0oTno4EPMoKEjiwbXfJCJkZYVsI6a9bkJWUMGqw/G/wIg+OoS/Ov96x4EcKyfkGe7umvznJpL2ri6AzWlFxOsBOXFV/zJXHP9jVUg+nS/V833jziySlraUGX+xYRp8t7XXNvD3HDjzUUrUwyI5rtHJlwBnbqa0bOXBS0rYD0bwN0WUv6Ss5sDeArAWADPAfjQW/aAcLwBwAgAHwDYEcD+ABYCmGZrpy2qAZ8Dq38vnnoJgB72mjcB3G7/Ay4GwP2bAHA9m4kADqwm78rk2GAdMXGWYU2VIJs0Z5HZe9/9zQOPPWPe+WhMLo0+1rPOvSBnMWDq3yMJt5j702d/alhT5T0XLF5u9v/uAebpZ/tmDqyfr1xrmjZrZsZNnFrZeDVk+OgG5Qpo36mrGTNnWdCyANaNAPQG8ENLJB92BN29Nr0DgC88sLKrxtH2GBf1usZu16XG6t+LlyfB6mDPYwTrrfYeR9k1b+xulYjrk+fWKt9l192i1gz7vzPYtO/Y2bBbVes27XI11pkLVpi99t7XtG7bPpf2s2N/WaWXgA+/NLdjgjR5rw8+Gm46de5i2nfoaNq2a29YY02eE3M/X20yVtpzfV8we7Zsmesd0KNnr6JBlc+bho+VYB07Z1nQGjJYnZ9zPoC3ARCwDD7s/gfgUJvOaKgH1tUAuIgXwy8B/MduFxKsB9s8GRGs37X7rB1P9o7l3YxdY00TjBuad0xwlfq9YkG01O+TFljHfbzchKwhg9V9jvNz/x2vEam2YHXXE2r0qRKoDHUB667WzWAvzdV6fVfAXu6ABavb3x7AdO9Y3k2Bdf0AhFKHXczylTrwYpUvLbCOn7vchKwWYOXvmyuusoGbX6AMXJX1VbtKK+NtbXpJRT4YuwKYAYCNRD5YLwVwjy01W/q5FK2Dm3+9D9Z/ADgt8KTu2k0ALLDL2m5qfbUCawojsmKCq9TvFQtcpX6ftMA6Ye5yEzIPltWhgmBlBcoPdDleYRMYs82l5IKDmysYu1ydkgBrI9v1io1Xz9q3R0t7gX+9D1Z+rvP8YQBCjVfMit2tuB443RGs7QqsAmuqPthSB16s8qUG1k+WmwkBqydYJwDYyfKHMffLMtDvWmFLTkiytf9b5fIkcgXIFZCv9hwLXKV+nzTA2qFzVzPp0xVBs668XCOzhSwbnf1A1rBNZwgAd2yxdwLbd/x971Dpb25pH5pdqkYCOLL0i7y+hAKrwCqwVj/hTFpgnfzpChOyWtRYd7G/5B1sl86D8oB00fpfu7aiKSCwCqwCaxHAOm+FmRywWoDV5wTdg+yK2WBcAf7Dld22wCqwCqyxwdrNTJm3MmgBsLJth1/LDNweaPvZ35JovGJjlkJsBQRWgVVgjQvWjp27mWnzVwYtANbm9vOfLsgx3nD7xgBet92tXrPdr2JjRfcTWAVWgbUIYF2w0kwLWACsglcpKyCwCqwCa3ywTl+wyoRMYC1lcgbKJrAKrAJrfLDO/GyVCZnAGoBXKR8WWAVWgTUyWLt0M7MWrgqawFrK5AyUTWAVWAXWYoB1tZm1sGYTWAPwKuXDAqvAKrDGBWvuN7dotZkdMIG1lMkZKJvAKrAKrPHB+vHi1SZkAmsAXqV8WGAVWAXW+GCdu3iNCZnAWsrkDJRNYBVYBdYigHXJGjM3YAJrAF6lfFhgFVgF1rhg7dy1m/l06ZqgCaylTM5A2QRWgVVgjQ/WeUvXmpAJrAF4lfJhgVVgFViLANbP15p5ARNYS5mcgbIJrAKrwBofrPM/X2tCJrAG4FXKhwVWgVVgjQ3W7uazZV8ETWAtZXIGyiawCqwCa1ywduna3Sxc/kXQBNYAvEr5sMAqsAqsAmspM6osyyawCqwCa3ywLlrxhQmZaqxlidR1hRZYBVaBNTJYu3U3S1Z+GTSBVWA1sxeuB1S5bucDTFbTSn1Z6ljlS2OV1q7dupulK78MmsAqsAqsq78yDQnCscBV6vdJDayrvjRLAyawCqwCq8BqSh2S9SlfWmBdtvorEzKBVWAVWAVWgbWWHKAroDZfN7UA6w8BTAAw2Vvyupal0GmpKqDGq/W+4dr8sWflnPrU7hriNWnVWJev+cqELADWjQBMAcBlsL9ll8JulyoslHntFRBYBdZ8L4uGCMn6PFNaYF2xxpiQBcC6H4BXvF/6lQBoCiWiwHz7H/hREePpRbx3MZ87372lBeB0KQUt+PsodHi5ln/voxPnnekV5FgA//H2TwFwp7evTSmQ+yFJhnUKECoK6xSQFtX/JQis1WujI1YB/YDW/ylIC2mxXoHqt+QKqF4bHbEKCCbr/xSkhbRYr0D1WxsDmAqgmdd41b7603Ukiwr4vqMsPr//zNJivRrSYr0W+baOAjDR9g64Ot8JSpMCUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaRADAW+kbhJcj9xWLtSQApIgaoKbAbgaJvUCcBPqx7O3J4PUeqxFQA/LXOCeA/sdKgAsImXrk0pIAUSCnCCiesADLPWKnE8q7vnAhgLYACAkwHslFUhEs/NF+8LdrjnoYlj2pUCUsBTYH8AnwB4x0tjB+ksBVcb4zMfA+AxAJsD+CWAewCcBeDbWRIkz7O2BvAGAA73PAPAJABH5DlPSVIgswo4kDCmO6ADgHsB9LUjTShMVkDitOAz7wzgUQBveX8ZBO0/AfwRwI5eepY2uwN4GsAN3kOfYDvQH+mlaVMKZFKB7QDsaZ+cI0sIjJ52vxGABwD0A3AQgGcA7JIhlS4A8KR99ncB+KNtfgXgdgDbZEgP96iHALjIavMigD08v/OvAcwCsL2X5q5TLAUyoQAbHQjRPwP4OYDh9lOX4+c5RRqPc4Jffvq+D+DHmVBl3UP+xNbWd7DPfACAJwBc5WmwpbedlU363F8FsKt94EcA3A1gd08A+Z89MbSZTQUOA3ATgPsAXGwlYAMWa6n/9twArIEw+J/JNqlBRP5z8YXyDwCfeTV0uke+C4A1tEvsE/vXNAgRAg9BYH5gv1zYO4JhUwAPWmPNVUEKZFqBb3pP3xbA/baGRt8qA7vPsAX84QYMUz5nEo4/W/f4cG6QxwG4Wivhuq8HW3tqg46cPu5znz0iBgL4noUqH55wZeOe+9tp0ILo4aRAdQq4Hwt/CE8BaGL9rP9nP3UJWgb2BNjLbjfUaGv7YE4T9oRgrYxhWwC3Avhvhhru7KPnYOm2+WJhDf5y+yL6LYD+1vdMqDI4/eyuIimQLQXcD+AHAP4FgMtRsFbK1m0ClSChzzULc03Sj/q2hYLfCPW8TedfBtPZO4JasZbv9GvIfzX0o15rX6zuOY8DcBsANugxnAbgPQBsyMqCJvaxFUmBqgr4I2NYU50NgLOisycAG7BYc6UflaOL7vB6ClTNpeHs8cXCxrpu9hOWtTHXO4JPyQYaukIYWHN17gCb1GAj9k0dAuB4+3LlwAgX6Cbh38bvbQL7ru7tDiqWAllTgI0Op9r+qXz279haqtOhhW2sYs2VEGFPgIYc2Imdi9bRr8za1j72xXK+B1e2eq+2PQEashb+sxGqn9qaKdPpR+WSz2d7J/EFNALAhV6aNqVAJhVoaT/z6Uvl0hKNAYy3sHWCsNbKBggOZ6XfzG/ccuc0hPhHAEYCuBTA9XYJY3abYk2dLxZ+6jYFQDcBP4f9WmxDeP7qnoFQHWy72S0AwN4iDAcDGOrVUvmVw769zg9vT1MkBbKpAFu02QjTyzbE8IfDMd7sl+l+PKyx3dXA5WHDy4H2GVlz/bttlHFw5fP/z65vlJV5Evi3wYZLdu5n4AiqpR5cOTjkYztYhOs+HW7PUyQFMqlAslGho62R8HOOjVNdALChhp28OwPgJBps7XV9FLMgGl8wHEFFTdhDgLV1ji7L0ggz/j+7obnubyYJ1+YAfmP98ln4u9AzSoG8CrgfCFtt+cnL2hknEeGnLYepXmG7WfFinkuocvYmfhI3xOD0cM/m7xOu7A3Rww7PdOdkIfZ14PP6PR8IVw6SoPuEIXmuTVYkBbKlACfDGGMbH9ip+xY7FJF+Vg7P/JOFLfur8kfUEP2Jfu07OdTSBwW14oQi9D9nIbCB0j2/i91zc9+lnQJghdWlofrd3XMrlgJBBdiqzVZvfsKxRsbGKnbyZs2MXYeY3lBrp04cTjDzOzvF34n2k5/+RD84gDCNNfosBA7ZpZ+Zs5Wxux0nlPF1oAY+XJMvpCxopGeUAl9TgJ//NHajYmvvh7aPKhshpgH4m51g5WsXNqAETsB8p32B8JnnAdjCPl+yO1kSKg1IhryPQj8yez+w8ZKNUWy8zBecLq6m6vbznas0KdCgFWADVR8AXe1TcuIQTkbMwGNs8W5j9xtqRIC+CeAkAJyVivOpDrJdrBrqM9f2uRwc+TcwAUBv2wUvaxOZ11YvnScFcmP+19pp/iiHq2mwf+LrACZnaOo/ft5y5BQb5PjpS9cIh2CyIY+BtTS+aLIU3Og7+p35t8Gp/thgx14RnFyGgT7mZI3eHlIkBbKhAGdg4igqBo6SYZcZDkFcmRjrz9oIhyNy+GZWAkeaLbRdydjhn6GdHTXET2C+bLIyzR396a47Ff8OOLcuG+roe+cUkZw2ku4hdjnj4Ims6GL/LBRJgfUKsEGBn/ucN5UjpjiZiqt1cFQRu8lkaRo394nrFGI/VHbwpy+Rk6hwKREGzofAeWcbYg8I+4hfi9j7Y5mdT/Yhuxgix/tzwAjXq2JNlmt4cULzrK/K+zXxlJAdBegfG2c7a3OyjEW2BwAVcJ//HMu9JgMt/8n/9fNstzJO3E240gVwpW3I4nDMrAa6QAhXDtNlYM+QX9gJqt2IK/e3k3xJ2UsUSYGGqwA/a1k7Pd0+IvukcoYhToxxWeKx+TmcpVUzzwHwmu0JwOW7ORMTA2uuHMrLrmb0t2YFHO45+cwM9Kcu99wC9Key/zIb9/gCUpACmVWA82ByfDsDfzgclkp4csz/XAtcdnZnw437QbkfmL2swUasjbHvKj/16Uel/5DGT12CIyud/93fBmPO3EVwuklT/mqnjOTqswx0jWRl9V37yIqkwNcVYGs2W7c5nygHAHDpELoF2BjBlURZY+O6TPzMy0JwLw3GXACRfXbpc3bdh/5g/YdZ0MI9o9OEfyPP2r7LXCCSfZoZOJH5Eru0t01SJAWyrQBHB7FBhpM0c113jqDhHKps8eeQVdbY3Gz47gfW0BT7PgB+9rvgfIOcHnGmtygiXSHsbpWVWar8kWPUgi4jN7qOo+444Y6D6822J4nTULEUkAIWoL4QHGnFbjSuW41/rKFts4vZlwDO9B7M9dFkTwn6m9l4xZo9u1hlIbBvKudJdS9V9hp5zs4t657/JTtQwrkBmN5QX77umRVLgXopQKBwvDdh4mYhqldGZXYRu0+xN4Sb3Z61VgdXNuax5u4WCiyzR6t3cQlT1lTpAmDgstTsNcIvGgb+ffDvhCPzFKSAFKhGAYKEQ1bpVz26mnMacjJXkCVc3RpMfFb6VF/2YNKQn5/Plpz/gO6PGdY1xJcP/e30v7Nxk8N6Wdvn+mZqsGrofxl6vg1SgHB1P5IsftY5uLKx7se2oYY9JbIQ2J+ZQ5XZaHeNV0M/GcAoO3k5u+Ox8z8npOFk5pyIh7VWV4vNgk56RikgBeqhAOH6lZ031DXW1CObsrqEvmP2AuG0iFxuhuDkcFUXONs/+/JyAnMXOIR1YgbnSHDPr1gKSIE6KsB+mq7Fu46Xlt3p7JvL1VP72pLzS+USAFfbfedrJnDZK4J9VRk4Ei1LQ3ntYyuSAlJACtROAfpKOces8y8Tqpx3ltNCcg0zDt/lqryul4jrkla73HWWFJACUiCjCjj/MrtVvW0n5eGy3Rxd9artIZBRafTYUkAKSIH6K8B+u+wZcZ3NwtVM3ed//XPWlVJACkiBDCvAblV0C3BGMxey2EPEPbtiKSAFpEBBFOBkK5/bVQEE1YJIqkykgBSQAoC/1Lf0kAJSQApIgQIo4GqqLi5AlspCCkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAg1bAU5QzdUPOLt9bwD+rPd1fXLON8rlmxk4w1NNk1pzIvD97bl1iaZ7Y+3966pL98/hqqh1CVzsj2P+FaSAFJACdVLAhw0XubsocbVbqyqRnHfXB2veE7zE+kKrOoBWl+7dMrfctL8f2q5vGUP56rgUkAINXAEfrJz9/24ArE2+Y2dr4rR2G9n1uwYDGOktBshuRJweb4K3aKKrsb4JgGPnGX4IYKidd5TzlXLJ8E8AzLG1Za4TxglKngHAe9A4YTgDV3LtD2CMrQVz0uh8w0N9sHLikyH2Gn+5GD7r/9l0loP3ZGhhJ+PmNXxuzqvKILBaIRRJASlQNwUcWFkz5RIhXBSQYOV695yUmYFw4sTNDJsC4AqjPHaMnVyE4OW6TYs9V4ADK+E1y8uLS7QwJKHFlVwPsMd2tyvdcvcOAFxGm4FLmZhagNXdYzPr4nDLbPPak2xezJMvBQZClkuoMHDE1Bt2O1lGm6xICkgBKVCzAs7HSj8rVw7l3KIE6wDvMq4+y5orz6FxOrwjANxuJ3V2p3Jp52SNlUvU0MWQDElocWy9y58xa7Nc5oTbzb2LF9YCrMybs/LTuKz0vvZ6PqtzbTBP5s17rEzcm8uXMyTLaJMVSQEpIAVqVsDVWP2zCNZ+XgI/0d2CeF5yQcG6AECFn7ndritYWXbO5O8a4VhzZhpDEqyc0Z/DTufa48lIYE0qon0pIAVqpUBtwEpXAP2Wbib8VgAaWVfAK9YHyxVJOV1essZanSvgYgDXeyWkK+BSb9+tk0VXgHNDHFkLV8BPATxv86GvdJUHVroCTrDHmCdr6AwDARxnt+k35lpUDAKrFUKRFJACdVOgNmDlnKJ/sQvisVsW3QRcytpvvOJEzlyBNAlWloZAZO2Qn+Y8j4FwZkMYa6RsvGKD1JM2jcuX/NOe5zde/duueFpT4xV9wC9ZHy1fBn6Nlc96m/W70o/qGq/oL+ZKsiwf7+18ugKr/U9QJAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaSAFJACUkAKSAEpIAWkgBSQAlJACkgBKSAFpIAUkAJSQApIASkgBaRAHAX+H8naEwhrWGZeAAAAAElFTkSuQmCC)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ogVAfgW3gxgW"
      },
      "source": [
        "# THE WINNER"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m6EwAiIHDE67"
      },
      "source": [
        "- reduced images to most recent\n",
        "- 84 and 82% accuracy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vo7Lpip9QWOA"
      },
      "source": [
        "epoch = 50\n",
        "batch = 3\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense_lessConv(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dAStTgstC6Tb"
      },
      "source": [
        "epoch = 50\n",
        "batch = 3\n",
        "\n",
        "model = Sequential()\n",
        "make_model_extraDense_lessConv(model, X_train)\n",
        "reset_weights(model)\n",
        "# compile\n",
        "LEARNING_RATE = 1e-5\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=optimizers.Adam(lr=LEARNING_RATE),\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "train(model, X_train, y_train, batch, epoch)\n",
        "model_summary(model, X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wCviVZuFcsvh"
      },
      "source": [
        "# What I learned today\n",
        "- add 0.1 dropout after conv layers (except last one) is good\n",
        "- fewer epochs and tiny batches (1 or 2) works well\n",
        "- model predictions way too slow in simulation -- how can I minimize that?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dKrVe34MEBY2"
      },
      "source": [
        "# Questions\n",
        "- why do you need so many units for the second last dense layer?\n",
        "- loss is TERRIBLE\n",
        "- not predicting right turns at all... can't really generate data to do it though...\n",
        "- in simulation there is a huge delay between having the model predict, and when the prediction gets to the robot - how to fix? less complex model?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jgKgIG55ADpC"
      },
      "source": [
        "#Save model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5yVJrvu331nc"
      },
      "source": [
        "#save the model here\n",
        "#we'll have to download from drive to our local\n",
        "model.save('/content/drive/MyDrive/test_data_enph353/comp_day_test_model.h5')\n",
        "model_path = 'comp_day_test_model.h5'\n",
        "model.save(model_path)"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}