univariate_linear_regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "univariate_linear_regression.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyPw7HxUmyPkpt4J5LSQuCoG",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/rares985/machine-learning/blob/master/univariate_linear_regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tuf3kngOoqks",
        "colab_type": "text"
      },
      "source": [
        "### Univariate linear regression\n",
        "\n",
        "This document showcases the application of gradient descent on univariate linear regression.\n",
        "\n",
        "First off, let's generate some random points in 2D.\n",
        "The points will be stored in an $ m \\times 1 $ matrix, where $ m $ represents the number of training samples or, in our case, the number of generated points.\n",
        "Visually, $ X $ looks like this:\n",
        "\n",
        "$$ \n",
        "X = \n",
        "\\begin{bmatrix}\n",
        "    x_0 \\\\\n",
        "    x_1 \\\\\n",
        "    x_2 \\\\\n",
        "    \\vdots \\\\\n",
        "    x_m\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "We are also adding a column of ones as the first column in the matrix, also known as the *intercept* term. Therefore, $ X $ will look like:\n",
        "\n",
        "$$\n",
        "X = \n",
        "\\begin{bmatrix}\n",
        "    1 & x_0 \\\\\n",
        "    1 & x_1 \\\\\n",
        "    1 & x_2 \\\\\n",
        "    \\vdots & \\vdots \\\\\n",
        "    1 & x_m\n",
        "\\end{bmatrix}\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hBa_ygBpo8TR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8fe81af2-5750-4a9c-a132-54cbacd4f1e5"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "low = 0\n",
        "high = 100\n",
        "how_many = 25\n",
        "\n",
        "a = 3\n",
        "b = 4\n",
        "noise_lo = 0\n",
        "noise_hi = 60\n",
        "\n",
        "X = np.random.uniform(low, high, how_many).reshape(-1, 1)\n",
        "\n",
        "noise = np.random.uniform(noise_lo, noise_hi, how_many).reshape(-1, 1)\n",
        "y = a * X + b + noise # This is to better see how the line is fitted so it does not overlap with the points\n",
        "\n",
        "X_intercept = np.hstack((np.ones_like(X), X))\n",
        "\n",
        "print(X_intercept.shape)\n",
        "\n",
        "\n",
        "m = X.shape[0] # Number of training samples"
      ],
      "execution_count": 306,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(25, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ij6VgaZHqadj",
        "colab_type": "text"
      },
      "source": [
        "Now let's initialize our regressor. $ \\theta $ will be a matrix of two elements of size $ 2 \\times 1 $, like this:\n",
        "\n",
        "$$\n",
        "\\theta = \n",
        "\\begin{bmatrix}\n",
        "    \\theta_0 \\\\\n",
        "    \\theta_1 \n",
        "\\end{bmatrix}\n",
        "$$\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p7a5Fg_-uiv9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "efb15678-46dc-4875-a6e5-53c5a8811b1a"
      },
      "source": [
        "import tqdm\n",
        "\n",
        "theta = np.array([0.5, 0.5]).reshape(-1, 1)\n",
        "\n",
        "J_hist = []\n",
        "\n",
        "# Hyper parameters\n",
        "epochs = 100\n",
        "learning_rate = 1e-4\n",
        "\n",
        "for n_ep in tqdm.tqdm(range(epochs)):\n",
        "    \n",
        "    # Compute outputs\n",
        "\n",
        "    y_hat = X_intercept @ theta\n",
        "\n",
        "    # Compute additional helper value - delta\n",
        "    delta = y_hat - y\n",
        "\n",
        "    # Cost function\n",
        "    J = (delta.T @ delta) / (2 * m)\n",
        "\n",
        "    J_hist.append(J)\n",
        "\n",
        "    # Derivative\n",
        "    djdw = (1 / m) * (X_intercept.T @ delta)\n",
        "\n",
        "    # Update weights\n",
        "    theta -= learning_rate * djdw\n",
        "\n",
        "print('\\n')\n",
        "print('Final theta is: {} {}'.format(theta[0], theta[1]))\n",
        "print('Real theta is: {} {}'.format(b, a))"
      ],
      "execution_count": 326,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "100%|██████████| 100/100 [00:00<00:00, 41750.99it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "Final theta is: [0.61096473] [3.62346657]\n",
            "Real theta is: 4 3\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T-qDbiiFNLmb",
        "colab_type": "code",
        "outputId": "2013c63e-4733-4a39-ef55-f6ad41231d1d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 268
        }
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "sns.set()\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "\n",
        "# Plot the points first\n",
        "ax.scatter(y, X)\n",
        "\n",
        "xs = np.arange(low, high, 0.1).reshape(-1, 1)\n",
        "xs = np.hstack((np.ones_like(xs), xs))\n",
        "\n",
        "ys = xs @ theta\n",
        "\n",
        "ax.plot(ys, xs[:, 1], color='red')\n",
        "\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": 325,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD7CAYAAACL+TRnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deUCUdeLH8ffMIMMxIIdoaG4WpbG22yGbWZmmtmlL6rrb6rod27Ftm5W3oiV4K2iamVZ2allulqtJKVamHWt39gvJLNKyxItDDmGAmef3h8mGAQEzwwwzn9dfOsfzfHyYPn35zvN8H5NhGAYiIuLXzN4OICIinqeyFxEJACp7EZEAoLIXEQkAKnsRkQCgshcRCQAqexGRABDk7QANKSwsw+ls2mUAsbE28vNLPZTIfVpLTmg9WZXTvZTTvVoip9lsIjo6vM7nfLrsnU6jyWV/8n2tQWvJCa0nq3K6l3K6lzdzahpHRCQAqOxFRAKAyl5EJAD8Ytmnp6fTr18/unXrxp49e2oe37t3L8OHD+fqq69m+PDh7Nu3r1HPiYhIy/vFsu/fvz+rV6+mU6dOtR5PS0tj5MiRZGVlMXLkSFJTUxv1nIiItLxfLPukpCTi4+NrPZafn09OTg7JyckAJCcnk5OTQ0FBQYPPiYiIdzRrzj4vL48OHTpgsVgAsFgstG/fnry8vAafExGRupmOHiXiX7cR9fs+4IHbjPj0efaxsbZmvS8uLsLNSTyjteSE1pNVOd1LOd2rzpyGAWvWwD33wLFjMH8+ce0j3b7vZpV9fHw8hw4dwuFwYLFYcDgcHD58mPj4eAzDqPe5psrPL23yRQhxcREcOVLS5H21tNaSE1pPVuV0L+V0r7pymg/8gG3SWKxbNlPVI4mSxctwnJsIzfz3mM2megfJzZrGiY2NJTExkczMTAAyMzNJTEwkJiamwedERARwOglZ9RTRvXsS/PZ2SmfOpSjztRNF7yGmX7oH7ezZs9myZQtHjx4lOjqaqKgoXnnlFXJzc0lJSaG4uJjIyEjS09M566yzABp8rik0svcNrSWrcrqXcrrXyZzmb3KJGH8Pwe++TWXvPpTc/yDOLme6ZR8Njex/sey9SWXvG1pLVuV0L+V0r7joUEpnpxOePhujTTBlM+ZQ8bcbwWRy2z4aKnuf/oJWRMQfWHJ2wcR7sH34IfaB11CavghnfMcWzaCyFxHxFLudsAcWErbkfoiOpnjFU9iHDHPraL6xVPYiIh4Q9PGHRIy9i6DdX1Dx5+GEPPwQdsPqtTxaCE1ExJ3KygifNoWoawZgKi7m2HNrKVn+GLRr59VYGtmLiLhJm7e2ETHuHizf7aP877dSNm0GRoT7L5BqDpW9iIiLTMeKCJ8xjdBnV1J9VgJFGzZR1esyb8eqRWUvIj5jx66DrNueS36xndhIK8P6JNCr+2nejtWg4E2vYJs0FvORwxy/awxlE6dAaKi3Y/2Myl5EfMKOXQdZuWk3ldVOAPKL7azctBvAJwvfdOQItnsnErJ+HdW/Po+iZ9ZQfcFF3o5VL31BKyI+Yd323JqiP6my2sm67bleSlQPw8C6dg0xlydhfTWTsinTKHxtu08XPWhkLyI+Ir/Y3qTHvcH8w/fYJo7B+voWqpIupuSBZTi6dvN2rEbRyF5EfEJsZN3noNf3eItyOgl56nGiL7+Y4P++Q+mcdIo2ZrWaogeVvYj4iGF9EggOql1JwUFmhvVJ8FKiEyy5X9F26DVETB5HdY/fUbD9Pcr/8S/48QZNrYWmcUTEJ5z8EtZnzsaprib04YcIXzAXwxpC8ZLl2Ef8zStLHbiDyl5EfEav7qf5xJk3luzPiRgzijb/txP7NddSmn4/zg7ez+UKlb2IyEl2O2GLMwh7cDFGVDTHnlhFZfKQVjua/ymVvYh4TGu6SCrog/eJGHcXQXu+pGL4SEpnzsWI9p877KnsRcQjWs1FUqWlhM+bSejjj+LsdDpFa9ZR1W+At1O5nc7GERGPaA0XSbXZtpWYvr0Ie+wRKm75B4VvveeXRQ8a2YuIh/jyRVKmokLC0+4l9PlnqT77HApfzqL6kl7ejuVRGtmLiEf46kVSwa9sJPryiwl54XmOjx5P4dZ3/b7oQWUvIh7iaxdJmQ4dIvLWG2l7899wtu9A0ZZtlN2bBiEhXsnT0jSNIyIe4TMXSRkG1n8/hy11CqbyckrvTaP8znugTZuWzeFlKnsR8RhvXyRl3v8dERNGE/zmG1RdfAklix/CcU5Xr+XxJpW9iPgfp5OQpx7DNms6ACXzFlBx8z/AHLgz1yp7EfErlq+/OrHUwQfvUXllf0oWLsHZ+VfejuV1KnsR8Q9VVYQuf5DwhfMxQkMpfvBh7MNH+sVSB+6gsheRVi/o88+wjbmLNp9/hv3aoZTMXYDRoYO3Y/kUlb2ItF4VFTB1HlEZGThj23HsyWepTB7s7VQ+SWUvIq1S0Hs7iBg7CnK/puKv11M2Yw5GVLS3Y/kslb2ItCqm0hLCZ08n9MnHcPzqDNiyhdILLvF2LJ/n8nlIb775JkOHDmXIkCEMHjyYLVu2ALB3716GDx/O1VdfzfDhw9m3b5+ruxKRANdm6+tEX3EJIU89zvF/3EHBth1w1VXejtUquDSyNwyDSZMmsXr1arp27cru3bv561//yoABA0hLS2PkyJEMGTKEDRs2kJqayqpVq9yVW0QCiKmwANu0KYS88DzV53SlaOMWqi/u6e1YrYrLI3uz2UxJSQkAJSUltG/fnsLCQnJyckhOTgYgOTmZnJwcCgoKXN2diAQSwyB443piLvsd1nVrKRs3kcI33lHRN4NLI3uTycQDDzzAnXfeSVhYGGVlZaxYsYK8vDw6dOiA5ce7r1ssFtq3b09eXh4xMf5z5xcR8RzzoYPYJo/H+upGirqex/1/ns5OOhH75Mc+fccrX+VS2VdXV/Poo4+yfPlyevTowccff8yYMWPIyMhwS7jYWFuz3hcXF+GW/Xtaa8kJrSercrqXV3IaBjz9NIwbBxUV5N49hSkhPSl3nHg6v9jOqs1fEhkRQt8enb2Xsxm8mdOlsv/iiy84fPgwPXr0AKBHjx6EhoZitVo5dOgQDocDi8WCw+Hg8OHDxMfHN2n7+fmlOJ1Gk94TFxfBkSMlTXqPN7SWnNB6siqne3kjp/nbfUSMH03wW29SecmllC5eyqysw5SfcsMTe5WDpzN30f1XUTqeP2E2m+odJLs0Z3/aaadx8OBBvvnmGwByc3PJz8/njDPOIDExkczMTAAyMzNJTEzUFI6I1M3hIPSxh4npcwlBH39ISfoijq1/FUfCOT59x6vWxKWRfVxcHNOnT2f06NGYflx/Yu7cuURFRTF9+nRSUlJYvnw5kZGRpKenuyWwiPgXy54vTyxc9tEH2PtfRemCB3Ce3rnm+dhIa53F7u07XrU2Ll9UNXjwYAYP/vnlyQkJCaxdu9bVzYuID9mx66D7bkZSVUXY0sWELcrAsNkoXrYC+5+H/2zhsmF9Eli5aXetm5d7845XrZWuoBWRRtmx62Ct0s0vtrNy026AJhd+0GefEjF6FEE52VQMHUbpnAUYcXF1vtZn7njVyqnsRaRR1m3PrTW6BqisdrJue27ji7e8nPAF8whd/iD26HYsG5HK6x0vInbtHob1cdS7HW/f8cofqOxFpFFc/aK0zY53sY29i6Bvcvn2D9dxX8IfKQoKq9lGc39LkMYJ3Ht0iUiT1PeF6C99UWoqKcY2aSxRQwZhqnZQ9OLLpP3ulpqiP+nkbwniGSp7EWmUYX0SCA6qXRm/9EVp8OtZJxYuW/kkx/85ioLtO6i6oq9Op/QCTeOISKM05YtSU34+tmkphLz4b6q7nUvRK69RnXRxzfM6nbLlqexFpNF+8YtSw8C6YR22qRMxFRVRNiGF46PHg7V2iet0ypanshcRtzAfzMM2aSzWza9SdcGFlLy4Ecevu9f5Wp1O2fJU9iLiGsMgZPUqwqffh6nSTun0OZTf/i8IarhedDply1LZi0iT/PQq2nOrC5j09goiPn2Pyksvp2TRUpxnaSrGF6nsRaTRTl5FW11ZxZBPX+GGd5/FYbbw2biZdJx0D5h1gp+vUtmLSKOt257LaQf3cveWZZx7cA8fnJXE8v53QGQnFqjofZrKXkQap7KSq7JW8pf3X+S4NYwF14zjrW69TyxcpvPjfZ7KXkR+UdCnHxMxZhR/+yKHbedewWN9b6U4rG3N87ZQVYmv0+9dIlK/48cJT7uXqEH9MRUV8f6cR1hy7fhaRQ9QXlHNjl0HvRRSGkNlLyJ127aNmL69CHt4KRXX/53Ct9/nrH+MxNrm57XhMNC6Nj5OZS8itZiKj2EbPxquvJKKSgfzb5rPX9oNZsKz2ezYdZCyCked79O6Nr5NE20iUiM4axO2SWMxHzrIdzf8k8ntBlBqagOcKPPHNubU+16ta+PbNLIXEUxHjxJxxy20vWE4RlQ0RZveYHrXYTVF/0u0ro3vU9mLBDLDwPrSC8RcnoR14wbKJk2l8LXtVF/Yg6OF5Y3aRGyklZsGnaulD3ycpnFEApT5wA8nFi7bspmqHkmULF6G49zEmufbRYdypBGFv+DOyzwZU9xEI3uRQON0ErLySaIvv5jgt7dTOnMuRZmv1Sp6gBsHJf7sZiWn0jx966GRvUgAMX+TS8T4ewh+920qe/eh5P4HcXY5s87X9u3RmeKSippFz06lefrWRWUvEgiqqwl9dDnh6bMx2gRTsmgpFX+78cRSBw346TLEP13tUuvPtz4qexE/Z9mVTcTYUbTZ+Sn2gddQmr4IZ3zHJm9H68+3bip7EX9ltxP2wELCltyPERVF8WNPYx/8x18czYt/UtmL+KGgjz4gYuxdBH25m4o/D6d09nyMmFhvxxIvUtmL+JOyMsLnzyZ0xXKc8R059txaKgdc7e1U4gNU9iJ+os1b24gYdw+W7/ZR/vdbKZs2AyMi0tuxxEeo7EVaOdOxIsJnTCP02ZVUn5VA0YZNVPXShU5Sm8pepBUL3vTKiYXLjh7h+N1jKZuQAqGh3o4lPsjlsrfb7cydO5cdO3ZgtVq54IILmDVrFnv37iUlJYWioiKioqJIT0+nS5cubogsIqbDh7HdO4mQDeuo7v4bip79N9XnX+jtWOLDXC77BQsWYLVaycrKwmQycfToUQDS0tIYOXIkQ4YMYcOGDaSmprJq1SqXA4sENMPA+uK/sd03GVNZGWVTpnH8rjHQpnGrU0rgcmltnLKyMtavX8/o0aMx/Xjubrt27cjPzycnJ4fk5GQAkpOTycnJoaCgwPXEIgHK/P1+Ikf+mchRt+NIOIfCre9yfOxEFb00iksj+/379xMVFcVDDz3E+++/T3h4OKNHjyYkJIQOHTpgsVgAsFgstG/fnry8PGJiYtwSXCRgOJ2EPP0E4bPSMBlOSuekU37L7fDjf18ijeFS2TscDvbv38+vf/1rJk+ezGeffcYdd9zBkiVL3BIuNtbWrPfFxUW4Zf+e1lpyQuvJ6nc59+yB226Dt9+GAQNgxQpsZ55J8/7LaDq/O55e5s2cLpV9fHw8QUFBNdM1559/PtHR0YSEhHDo0CEcDgcWiwWHw8Hhw4eJj49v0vbz80txOo0mvScuLoIjR0qa9B5vaC05ofVkbcmcriwK1qic1dWELl9K+IK5GCGhlC5Zjn3E304sddBC/0b93N2rJXKazaZ6B8kuzdnHxMTQs2dP3n33XQD27t1Lfn4+Xbp0ITExkczMTAAyMzNJTEzUFI74hR27DrJy0+6aZX/zi+2s3LSbHbsOumX7luzPiRrYD9vsNCr7/57Cdz7A/tfrtaaNuMTls3FmzJjB1KlTSU9PJygoiIyMDCIjI5k+fTopKSksX76cyMhI0tPT3ZFXxOvWbc+lstpZ67HKaifrtue6tipkRQVhizMIW/oARnQMx554hsprh7iYVuQEl8u+c+fOPPPMMz97PCEhgbVr17q6eRGfU9eNPBp6vDGCPnifiLGjCPpqD++cP4Bll96EdX87hu06qGWFxS10Ba1IE8VGWuss9mbdoq+0lPB5Mwl9/FHK28cz77rpfND5ghNP/Tg9BKjwxWUqe5EmGtYngZWbdteayqnvFn11fZE7uO+JMzLavPkGERNGY9n/HeW33s649lfzg7326ZRNmR7SnaSkISp7kSY6WaC/VKwnv8g9+T+Fk1/kxhgVJK1IJ2TNaqrPPofCl7OovqQXP8zfWuf+GjM9VN++fppXApvKXqQZGnOLvrq+yO3xxbuct/zvWMuPcXz0eMrGT4aQEMC16SGPfWksfkNlL+IhPy3uqLJC7ti6gsu+2sE3cWdibNhA9W/Or/X6pkwPNbSvxjwugUdlL+IhsZFW8o9V0C/nTW7b9iTWajsrL7+edwb8lbmnFD00fnqo3n2560tj8UsqexEP+Vs3K3FTpnLBvk/J6ZjI0t+P4nD7X3F38m/qfU9jpofq4spvBRIYVPYi7uZ0EvLkCgbMnoHDafDMoH+x9tyriGkbyk19Eujbo7PbL5t35bcCCQwqexE3sny1h4ixd9Hmg/eovLI/JQuXMLDzrxjYAvtu7m8FEhhU9iLuUFVF2LIlhC2cjxEWRvHSR7D/5a9az0Z8hspexEVBn3+GbfQo2mT/H/Zrh1IybyFG+/bejiVSi8pepLkqKghfOJ/QZUtwxrbj2JPPUpk8uFmb0tWv4mkqe5FmCHpvx4mFy3K/pvyv11M2Yw5GVHSztqWrX6UluLSevUigMZWWYEsZT/TgqzFVVVH0wnpKlyxvdtFDw1e/iriLRvYiDfjp9ErfQ5/zr9cfJuRwHsdv/xdlKdPA5voNAnX1q7QElb1IPU5Or1hLihiz/Sn657zJ/tjO7Fu6hnP/Msht+9HVr9ISVPYi9Vi37WuSct7hjq2PElFRypqe1/Hvnn+h7VEbC9y4H139Ki1BZS9SB/Ohg9y+eiaXfv0eX3VIIPVP09kXdybg/ukVXf0qLUFlL/JThkHI888SnnYvSWXHear3jazvMQSn+X83FfHE9IqufhVPU9mL/Mj87T4ixo8m+K03qbzkUt6+M43MbDtOTa+IH9CplyIOB6ErlhPT5xKCPvmIkvRFHFv/Kr8d2IubBp1bM5KPjbRy06BzNQKXVkkjewloli93n1i47KMPsPe/itKFS3B2Or3meU2viL9Q2UtgqqoibOliwhZlYNhsFC9/DPuf/qKFy8Rvqewl4AR99ikRo0cRlJNNxdBhlM5ZgBEX5+1YIh6lspfAUV5O+IJ5hC5/EGdce46tfJ7KQX/42cu0KJn4I5W9BIQ2/30H29i7CNr7DeXX30RZ2iyMtlE/e50WJRN/pbNxxK+ZSoqxTRxL1NBrMDmdFL34MqWLltZZ9KBFycR/aWQvfiv49SxsE8ZgPpjH8TvuomzyvRAe3uB7tCiZ+CuVvfgdU34+tvsmE/LSC1R3O5eiJ1ZR3eN39b7+p3P0ZhM4jZ+/RouSSWunshf/YRhY17+EbepETEVFlE1I4fjo8WCtv6hPnaOvq+h11az4A5W9+AVz3gG4bRKRL79M1QUXUvLiRhy/7v6L76trjh6oGeHrbBzxF277gvahhx6iW7du7NmzB4CdO3cyePBgrr76am655Rby8/PdtSuR/zEMQp55mujLL4YtWyidPoeiV99oVNFD/XPxTgOeTOnHgjsvU9GLX3BL2e/atYudO3fSqVMnAJxOJxMnTiQ1NZWsrCySkpJYuHChO3YlUsO89xva/ulaIsbfQ/Vvz4fPP6f8zrshqPG/sNY3F685evE3Lpd9ZWUlM2fOZPr06TWPZWdnY7VaSUpKAmDEiBFs3rzZ1V2JnOBwEPrwQ8T07UXQZzspWbiEYy9thLPPbvKmhvVJIDio9n8GmqMXf+TynP2SJUsYPHgwp5/+v8Wj8vLy6NixY83fY2JicDqdFBUVERVV9/nNdYmNbd79PePiIpr1vpbWWnKCe7Nu+3g/qzZ9wdHCctpFh3LjoET69ujcuDdnZ8Ott8IHH0ByMqaHHybi9NM5ma6pOQf3jSAyIqT5eZqptfzsldO9vJnTpbL/9NNPyc7OZsKECe7KU0t+finOuk6PaEBcXARHjpR4JI87tZac4N6sp579cqSwnKUv7KS4pKLhufHKSsKW3E/YAwsxIiMpfeQJ7H/884mFy37M1tyc3X8VRfo/e9V6zJM/m9bys1dO92qJnGazqd5BskvTOB9++CG5ubn079+ffv36cfDgQW699Va+/fZbDhw4UPO6goICzGZzk0b14p+ac4Vq0CcfEX3VFYQvmIf92qEUvP0h9mHXaYVKkSZwaWR/++23c/vtt9f8vV+/fjzyyCOcffbZvPDCC3z00UckJSWxZs0aBg4c6HJYaf2adIXq8eOEp88h9NFlODucxrFn/k3l1YM8kkuLn4m/88h59mazmYyMDNLS0rDb7XTq1IkFCxZ4YlfSysRGWuss9lPPfmnzzltEjL0Ly7f7KL/xFspSZ2BEtvVIJi1+JoHArWW/devWmj9fdNFFbNy40Z2bFz8wrE9CrWKF2me/mIqPET4jldBnnsLR5UyK/vMKVZf19mimhqaWVPbiL3QFrbSok+VZ15RJcNYmbBPHYD58iON33kPZpKkQFubxTFr8TAKByl5a3Kn3dTUdPYrtjlsIWfci1YndKVr5HNUX9mixPI2dWhJpzbSevXiPYWB96QViLk/CunEDZZOmUvja9hYtetCFVRIYNLIXrzD/8D22SWOxvpZFVY8kShYvw3FuoleyNDS1JOIvVPbSspxOQp55mvAZ0zA5HZTOmkf5bXeAxdKkzdR1quTgvs2/OvHUqSURf6OylxZj+eZrbOPuIfi/71DZuy8l9y/B2eXMJm+nvlMlIyNC6P4rXbgnUhfN2YvnVVcTuuxBovteSlD255QsfohjL25oVtFD/adKrtr0hTvSivgljezFoyy7sokYO4o2Oz/FPvAPlGYswnlavEvbrO+UyKOF5S5tV8SfaWQvnmG3EzZ/NtFXXYHl+/0UP/Y0xSufc7noof5TIttFh7q8bRF/pbIXtwv66AOiB/QmfFEG9qF/ouCdD7EPGea2hcvqO1XyxkHeOZtHpDXQNI64T1kZ4fNnEbriYZwdO3HsubVUDrja7bup71TJvj06t4qlbkW8QWUvbtHmrW1EjLsHy3f7KL/5Nsrum44REemx/elUSZGmUdmLS0zHigiffh+hq1dRfVYCRRs2UdXrMm/HEpFTqOyl2YJfzcQ2eRzmo0c4fvdYyiakQGjLfUl66oVVf0/urvPsReqhL2ilyUyHDxNx2020/ftIjHZxFG3eStm0GS1e9Cs37a45DTO/2M5Daz9jx66DLZZBpDVR2UvjGQbWF54npvfvsG5+hbIp0yjcso3q8y9s8Sh1XVhlr3I0eHtDkUCmaRxpFPP3+7FNHIP1jdeoSrqYkgeW4ejazWt5tAa9SNNoZC8Nczph+XKie/ckeMd/KZmbQdHGLK8WPdR/YZXWoBepm8pe6mXJ/Yq2Q6+BUaOoTvodBW+9R0UzVqj0hLourLK2sWgNepF6aBpHfq66mtDlSwlfMBcjJBSeeopj17jvClh3qOvCKp2NI1I/lb3UYvn8/4gYexdt/m8n9muupTT9fmLPOwd88MrUUy+siouL0BW0IvVQ2csJFRWELcogbOlijJhYjj3xDJXXDvF2KhFxE5W9EPTB+0SMHUXQV3uoGD6S0plzMaJjvB1LRNxIZR/ISksJnzuD0CdW4Ox0OkVr1lHVb4C3U4mIB6jsA1SbN98gYsJozN/vp+KWf1B2bxqGrfn3cBUR36ayDzCmwgJsafcSsmY11WefQ9GGzVRf0svbsUTEw1T2ASR44wYiUsZjKsjn+OjxlI2fDCEh3o4lIi1AZR8ATIcOETFlAtbMDVSd91tK17xE9W/O93YsEWlBKnt/ZhhY//0cttQpmMrLKb1vOuX/uhvatPF2MhFpYSp7P2X+7lsiJowmeNtWqnr2omTxQzjOPsfbsUTES1wq+8LCQiZNmsR3331HcHAwZ5xxBjNnziQmJoadO3eSmpqK3W6nU6dOLFiwgNjYWHflDnin3rhjWJ+EE1eTOp2EPLkC2+wZGCYTJfMWUnHzbWDWMkgigcylBjCZTNx2221kZWWxceNGOnfuzMKFC3E6nUycOJHU1FSysrJISkpi4cKF7soc8Oq6ccfKTbv5fNO7RA0eSMTUSVT1vITCt96j4tbbVfQi4lrZR0VF0bNnz5q/X3DBBRw4cIDs7GysVitJSUkAjBgxgs2bN7uWVGqceuMOi6OaIe/+m963XItlz26Klz7CsTXrcHb+lRdTiogvcducvdPp5Pnnn6dfv37k5eXRsWPHmudiYmJwOp0UFRURFaVVCV310xt0JBzK5e4tD5FwZC/vnHMp5/5nFUb79l5MJyK+yG1lP2vWLMLCwrj++ut57bXX3LLN2Fhbs94XF9c6rgRtbs646FCOHS5ixHsvMOyj/3AsrC1zrk0h9+IrebK7Z9Zzb2zWbR/vZ9WmLzhaWE676FBuHJRI3x6dPZKpLv7+s29pyule3szplrJPT0/n22+/5ZFHHsFsNhMfH8+BAwdqni8oKMBsNjd5VJ+fX4rTaTTpPa1lmVtXct7SNp8zF02kU+EPbOnenyf73EyVLZKbLj/TI//2xmY9+V3CySmmI4XlLH1hJ8UlFbWWIvaUQPjZtyTldK+WyGk2m+odJLtc9osWLSI7O5sVK1YQHBwMwHnnnUdFRQUfffQRSUlJrFmzhoEDB7q6q4BnKi0hfPZ0Ln/yMcpOO52MG+bydtyviY20MvLk2TgNqPcMHjep6ybgldVO1m3PbZGyF5H6uVT2X331FY8++ihdunRhxIgRAJx++uksW7aMjIwM0tLSap16Kc0X/MYWbBPGYD7wA8dv/xfHU6Zxs83GzY18/6mj7pNn8ABuK2LdBFzEd7lU9ueccw5ffvllnc9ddNFFbNy40ZXNC2AqyMc2bQoha9dQ3bUbRZlbqP5dz19+4ylaYtQdG2mts9h1E3AR79MJ2L7KMAh++T/EXH4x1v+8SNm4iRS+8U6zih5aZo5E0aUAAAsFSURBVNRd103Ag4PMugm4iA/Qcgk+yHzoILZJ47BuyqTq/AspeWE9jvN+49I2W2LUXddNwN39vYCINI/K3pcYBiHPP0t46lRMlXZKU2dRfscoCHL9xzSsT0KtOXvwzKj71JuAi4hvUNn7CPO+vURMGEPwW29S2esyShc9iCPBfQuXadQtEthU9t7mcBD6+COEz5uFYbZQkrGYihtv9sh6Nhp1iwQulb0XWb7cTcSYUbT5+EPs/a+idOESnJ1O93YsEfFDOhvHGyorYdYsovtfjmVvLsXLH6P4uRdV9CLiMRrZt7CgnZ8QMeYuyMnG/sc/UTo7AyMuztuxRMTPqexbSnk54RlzCX14Kc72HWDDBkp6XentVCISIDSN0wLa/Pcdovv2ImzZEir+diOFb78Pgwd7O5aIBBCN7D3IVFJM+Mw0Qlc+geOMLhS9tJGq3n28HUtEApDK3kOCX9uMbeJYzAfzOH7HXZRNvhfCw70dS0QClMrezUz5+djum0zISy9Q3e1cip5YRXWP33k7logEOJW9uxgG1vUvYZs6EVNxMWUTUjg+ejxYteKjiHifyt4NzHkHsE0eh3Xzq1RdeBEli5fh+HV3b8cSEamhsneFYRDy7ErCp9+HqbqK0ulzKP/nnWCxeDuZiEgtKvtmMu/9hojx9xD8zltUXtabkvsfxHmW1m0XEd+ksm8qh4PQFQ8TPn8WRlAbShYuoeL6mzyycJmIiLuo7JvA8kUOEWNH0eaTj7H/fiClGYtxduzk7VgiIr9IZd8YlZWELbmfsAcWYkRGUvzok9iH/glMJm8nExFpFJV9PXbsOsi67bnEfPk5Y99YTtzhfVQMu47SORkYsbHejici0iQq+zrs2HWQ51/+jL+89QyDP8mkMDyaucOmcd7dN9BLRS8irZDKvg45T69j0frFxB87xKbfXs3TvW/kuDWcPdtzdacnEWmVVPY/YSo+RviMaaQ88zQH2p7GlOtmkd35NzXP5xfbvZhORKT5VPY/Cs7ahG3iGMyHD7Hp0j/xRI+/YG9Te6mD2EgtfSAirVPAnxxuOnqUiH/eTNsbhmNEx1C06Q2q5szDCA2t9brgIDPD+uiiKRFpnQJ3ZG8YWNetxXbvJEwlJZRNvpfjd4+F4GB6/fiSddtzyS+2ExtpZVifBM3Xi0irFZBlb/7he2yTxmJ9LYuqHkknFi47N7HWa3p1P03lLiJ+I7DK3ukkZNVThM9MxeR0UDprHuW33aGFy0TE7wVM2Vu++RrbuHsI/u87VPbuS8n9S3B2OdPbsUREWoT/l311NaGPLCM8Yw5GsJWSxQ9RMfIGLXUgIgHFo2W/d+9eUlJSKCoqIioqivT0dLp06eLJXdZi2ZV9YuGynZ9iH/gHSjMW4TwtvsX2LyLiKzx66mVaWhojR44kKyuLkSNHkpqa6snd/Y/dTtj82URfdQWW77/n2OMrKV75nIpeRAKWx8o+Pz+fnJwckpOTAUhOTiYnJ4eCggJP7fKETz4hekBvwhdlYP/jnyl45wMqB/9R0zYiEtA8No2Tl5dHhw4dsPx4povFYqF9+/bk5eURExPTqG3ExtqavuN/zCToeBm8+iohgwYR0vQttJi4uAhvR2i01pJVOd1LOd3Lmzl9+gva/PxSnE6jSe+Je/55jhQch+BgOFLioWSui4uL4IgP5/up1pJVOd1LOd2rJXKazaZ6B8kem8aJj4/n0KFDOBwOABwOB4cPHyY+3sPz5qGhJ4peRERqeKzsY2NjSUxMJDMzE4DMzEwSExMbPYUjIiLu49FpnOnTp5OSksLy5cuJjIwkPT3dk7sTEZF6eLTsExISWLt2rSd3ISIijRDwSxyLiAQClb2ISABQ2YuIBACfPs/ebG7eVa/NfV9Lay05ofVkVU73Uk738nTOhrZvMgyjaVctiYhIq6NpHBGRAKCyFxEJACp7EZEAoLIXEQkAKnsRkQCgshcRCQAqexGRAKCyFxEJACp7EZEA4NPLJTTV3r17SUlJoaioiKioKNLT0+nSpYu3Y9GvXz+Cg4OxWq0ATJgwgd69e7Nz505SU1Ox2+106tSJBQsWEBsb22K50tPTycrK4ocffmDjxo107doVaPg4euMY15ezvuMKeOXYFhYWMmnSJL777juCg4M544wzmDlzJjExMQ3maemsDeXs1q0bXbt2xWw+MQ7MyMigW7duAGzdupWMjAwcDgfdu3dn3rx5hIaGeiwnwJ133sn333+P2WwmLCyMadOmkZiY6HOf0fpy+tRn1PAjN9xwg7F+/XrDMAxj/fr1xg033ODlRCdceeWVxpdfflnrMYfDYQwYMMD48MMPDcMwjGXLlhkpKSktmuvDDz80Dhw48LN8DR1Hbxzj+nLWdVwNw3vHtrCw0Hjvvfdq/j5//nxjypQpDebxRtb6chqGYXTt2tUoLS392XtKS0uNSy+91Ni7d69hGIYxdepUY+nSpR7NaRiGUVxcXPPn1157zRg6dKhhGL73Ga0vpy99Rv1mGic/P5+cnBySk5MBSE5OJicnh4KCAi8nq1t2djZWq5WkpCQARowYwebNm1s0Q1JS0s/uCdzQcfTWMa4rZ0O8dWyjoqLo2bNnzd8vuOACDhw40GAeb2StL2dD3nrrLc4777yaEfKIESPYtGmTJ2MCEBERUfPn0tJSTCaTT35G68rZEG/83P1mGicvL48OHTpgsVgAsFgstG/fnry8PJ+47+2ECRMwDIMePXowbtw48vLy6NixY83zMTExOJ3Oml89vaWh42gYhs8d41OPa2RkpE8cW6fTyfPPP0+/fv0azOPtrD/NedINN9yAw+Hgiiuu4O677yY4OPhnOTt27EheXp7H8wHce++9vPvuuxiGweOPP+6zn9FTc57kK59RvxnZ+7LVq1fz8ssv89JLL2EYBjNnzvR2JL/gy8d11qxZhIWFcf3113s7SoNOzblt2zbWrVvH6tWr+frrr1m2bJmXE8KcOXPYtm0bY8eOJSMjw9tx6lVXTl/6jPpN2cfHx3Po0CEcDgcADoeDw4cPN+nXf085mSE4OJiRI0fyySefEB8fX+tX54KCAsxms1dH9dDwcfS1Y1zXcT35uDePbXp6Ot9++y0PPPAAZrO5wTzezHpqTvjfMbXZbFx33XX1HtMDBw60+M996NChvP/++5x22mk+/Rk9mbOwsNCnPqN+U/axsbEkJiaSmZkJQGZmJomJiV6fwjl+/DglJSUAGIbBq6++SmJiIueddx4VFRV89NFHAKxZs4aBAwd6MyrQ8HH0pWNc33EFvHpsFy1aRHZ2NsuWLSM4OPgX83gra105jx07RkVFBQDV1dVkZWXVHNPevXvz+eefs2/fvpqcgwYN8mjGsrKyWlNFW7dupW3btj73Ga0vp9Vq9anPqF/dvCQ3N5eUlBSKi4uJjIwkPT2ds846y6uZ9u/fz913343D4cDpdJKQkMB9991H+/bt+eSTT0hLS6t16lW7du1aLNvs2bPZsmULR48eJTo6mqioKF555ZUGj6M3jnFdOR955JF6jyvglWP71VdfkZycTJcuXQgJCQHg9NNPZ9myZQ3maems9eW87bbbSE1NxWQyUV1dzYUXXsjUqVMJDw8H4PXXX2fBggU4nU4SExOZP38+YWFhHst59OhR7rzzTsrLyzGbzbRt25bJkyfTvXt3n/qM1pczMjLSpz6jflX2IiJSN7+ZxhERkfqp7EVEAoDKXkQkAKjsRUQCgMpeRCQAqOxFRAKAyl5EJACo7EVEAsD/A8NGXC+pBe4sAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LxPvthDvxkmW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}