Session #2 Homework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "53a28aed",
   "metadata": {},
   "source": [
    "## Homework\n",
    "\n",
    "### Dataset\n",
    "\n",
    "In this homework, we will use the California Housing Prices from [Kaggle](https://www.kaggle.com/datasets/camnugent/california-housing-prices).\n",
    "\n",
    "Here's a wget-able [link](https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv):\n",
    "\n",
    "```bash\n",
    "wget https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv\n",
    "```\n",
    "\n",
    "The goal of this homework is to create a regression model for predicting housing prices (column `'median_house_value'`)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50493524",
   "metadata": {},
   "source": [
    "### EDA\n",
    "\n",
    "* Load the data.\n",
    "* Look at the `median_house_value` variable. Does it have a long tail? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b0044cd",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\"wget\" не является внутренней или внешней\n",
      "командой, исполняемой программой или пакетным файлом.\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "data = 'https://raw.githubusercontent.com/alexeygrigorev/datasets/master/housing.csv'\n",
    "\n",
    "!wget $data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "109a4963",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-122.23</td>\n",
       "      <td>37.88</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>452600.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-122.22</td>\n",
       "      <td>37.86</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>358500.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-122.24</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>352100.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>341300.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>342200.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20635</th>\n",
       "      <td>-121.09</td>\n",
       "      <td>39.48</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1665.0</td>\n",
       "      <td>374.0</td>\n",
       "      <td>845.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>1.5603</td>\n",
       "      <td>78100.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20636</th>\n",
       "      <td>-121.21</td>\n",
       "      <td>39.49</td>\n",
       "      <td>18.0</td>\n",
       "      <td>697.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>356.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>2.5568</td>\n",
       "      <td>77100.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20637</th>\n",
       "      <td>-121.22</td>\n",
       "      <td>39.43</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2254.0</td>\n",
       "      <td>485.0</td>\n",
       "      <td>1007.0</td>\n",
       "      <td>433.0</td>\n",
       "      <td>1.7000</td>\n",
       "      <td>92300.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20638</th>\n",
       "      <td>-121.32</td>\n",
       "      <td>39.43</td>\n",
       "      <td>18.0</td>\n",
       "      <td>1860.0</td>\n",
       "      <td>409.0</td>\n",
       "      <td>741.0</td>\n",
       "      <td>349.0</td>\n",
       "      <td>1.8672</td>\n",
       "      <td>84700.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20639</th>\n",
       "      <td>-121.24</td>\n",
       "      <td>39.37</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2785.0</td>\n",
       "      <td>616.0</td>\n",
       "      <td>1387.0</td>\n",
       "      <td>530.0</td>\n",
       "      <td>2.3886</td>\n",
       "      <td>89400.0</td>\n",
       "      <td>INLAND</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20640 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0        -122.23     37.88                41.0        880.0           129.0   \n",
       "1        -122.22     37.86                21.0       7099.0          1106.0   \n",
       "2        -122.24     37.85                52.0       1467.0           190.0   \n",
       "3        -122.25     37.85                52.0       1274.0           235.0   \n",
       "4        -122.25     37.85                52.0       1627.0           280.0   \n",
       "...          ...       ...                 ...          ...             ...   \n",
       "20635    -121.09     39.48                25.0       1665.0           374.0   \n",
       "20636    -121.21     39.49                18.0        697.0           150.0   \n",
       "20637    -121.22     39.43                17.0       2254.0           485.0   \n",
       "20638    -121.32     39.43                18.0       1860.0           409.0   \n",
       "20639    -121.24     39.37                16.0       2785.0           616.0   \n",
       "\n",
       "       population  households  median_income  median_house_value  \\\n",
       "0           322.0       126.0         8.3252            452600.0   \n",
       "1          2401.0      1138.0         8.3014            358500.0   \n",
       "2           496.0       177.0         7.2574            352100.0   \n",
       "3           558.0       219.0         5.6431            341300.0   \n",
       "4           565.0       259.0         3.8462            342200.0   \n",
       "...           ...         ...            ...                 ...   \n",
       "20635       845.0       330.0         1.5603             78100.0   \n",
       "20636       356.0       114.0         2.5568             77100.0   \n",
       "20637      1007.0       433.0         1.7000             92300.0   \n",
       "20638       741.0       349.0         1.8672             84700.0   \n",
       "20639      1387.0       530.0         2.3886             89400.0   \n",
       "\n",
       "      ocean_proximity  \n",
       "0            NEAR BAY  \n",
       "1            NEAR BAY  \n",
       "2            NEAR BAY  \n",
       "3            NEAR BAY  \n",
       "4            NEAR BAY  \n",
       "...               ...  \n",
       "20635          INLAND  \n",
       "20636          INLAND  \n",
       "20637          INLAND  \n",
       "20638          INLAND  \n",
       "20639          INLAND  \n",
       "\n",
       "[20640 rows x 10 columns]"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "e4d0a6f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "842bdc0e",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='median_house_value', ylabel='Count'>"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEHCAYAAABWecpSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYNklEQVR4nO3dfbBlVX3m8e9DI6CCSIeGammwsWQSgfK1xRcsiwSNjMMEDaJoDI0yoZLg+0QFrYqZGqkh0XLUGI2MbzhRoQkSiDEqoujEINAIAg22NK+2TeiOZKKOMyj4mz/2In26+57bdzf3nnPvPd9P1a2zz9p7n7PWreY+7LX2XitVhSRJfew27gpIkhYew0OS1JvhIUnqzfCQJPVmeEiSett93BWYK/vvv3+tXLly3NWQpAXl2muv/eeqWraz4xZteKxcuZK1a9eOuxqStKAkuWsmx9ltJUnqzfCQJPVmeEiSejM8JEm9GR6SpN4MD0lSb4aHJKk3w0OS1JvhIUnqbdE+Ya7558RXncKmLfdNue9xy5Zy0Wc/PeIaSdpVhodGZtOW+3j8Se+cct9dF5494tpIejjstpIk9WZ4SJJ6MzwkSb0ZHpKk3gwPSVJvhockqTfDQ5LUm+EhSerN8JAk9WZ4SJJ6MzwkSb0ZHpKk3gwPSVJvhockqTenZJekBWjY+jijWhvH8JCkBWjY+jijWhvHbitJUm9zFh5JPpFkc5KbBsqWJrksya3tdb+BfWcl2ZBkfZIXDZQ/I8mNbd8Hk2Su6ixJmpm5vPL4FHDcdmVnApdX1WHA5e09SQ4HTgaOaOd8OMmSds5HgNOBw9rP9p8pSRqxOQuPqvomsP1ozgnAeW37POAlA+XnV9X9VXUHsAE4Ksly4DFVdWVVFfDpgXMkSWMy6jGPA6vqHoD2ekArPwj4wcBxG1vZQW17+/IpJTk9ydoka7ds2TKrFZckbTVfBsynGseoacqnVFXnVtWqqlq1bNmyWaucJGlbow6Pe1tXFO11cyvfCBw8cNwKYFMrXzFFuSRpjEYdHpcCq9v2auCSgfKTk+yZ5FC6gfGrW9fWT5I8u91ldcrAOZKkMZmzhwSTfA44Btg/yUbgXcA5wJokpwF3AycBVNW6JGuAm4EHgDOq6sH2UX9Ad+fWI4G/bz+SpDGas/CoqlcO2XXskOPPBnZ4NLKq1gJHzmLVJEkPk9OTLCLD5rqB0c13I2kyGB6LyLC5bmB0891Imgzz5VZdSdICYnhIknozPCRJvRkekqTeDA9JUm+GhySpN8NDktSb4SFJ6s2HBLVLhj3N7pPs0mQwPLRLhj3N7pPs0mQwPLRgefUjjY/hoQXLqx9pfAyPCedMvJJ2heEx4WZ7Jt5bv7+e57zw+Cn3bbjtdh7f8zwDTJqfDA/Nqgdqt6FhdMu7T+19nl1Q0vzkcx6SpN4MD0lSb4aHJKk3w0OS1JsD5prXdvXuLUlzy/DQvLard29Jmlt2W0mSejM8JEm9GR6SpN4MD0lSb4aHJKm3sYRHkjcnWZfkpiSfS7JXkqVJLktya3vdb+D4s5JsSLI+yYvGUWdJ0lYjD48kBwFvAFZV1ZHAEuBk4Ezg8qo6DLi8vSfJ4W3/EcBxwIeTLBl1vSVJW42r22p34JFJdgceBWwCTgDOa/vPA17Stk8Azq+q+6vqDmADcNRoqytJGjTy8KiqHwLvBe4G7gH+taq+AhxYVfe0Y+4BDminHAT8YOAjNrayHSQ5PcnaJGu3bNkyV02QpIk3jm6r/eiuJg4FHgc8OsmrpztlirKa6sCqOreqVlXVqmXLlj38ykqSpjSObqsXAHdU1Zaq+gXweeC5wL1JlgO0183t+I3AwQPnr6Dr5pIkjck4wuNu4NlJHpUkwLHALcClwOp2zGrgkrZ9KXBykj2THAocBlw94jpLkgaMfGLEqroqyV8D3wEeAK4DzgX2BtYkOY0uYE5qx69Lsga4uR1/RlU9OOp6zycnvuoUNm25b4dyZ5mVNCpjmVW3qt4FvGu74vvprkKmOv5swMWsm01b7ptypllnmZU0Kj5hLknqzfU8JsSwRZWm6+pyISZJwxgeE2LYokrTdXW5EJOkYey2kiT1ZnhIknozPCRJvTnmITXDnp953LKlXPTZT4+hRtL8ZXhIzbDnZ+660EeMpO3ZbSVJ6s3wkCT1ZnhIknozPCRJvRkekqTeDA9JUm+GhySpN8NDktSb4SFJ6s3wkCT1ZnhIknozPCRJvTkx4jw1bIZXcAlYSeM3o/BIcnRVfWtnZZo9w2Z4BZeAlTR+M+22+vMZlkmSJsC0Vx5JngM8F1iW5C0Dux4DLJnLikmS5q+ddVvtAezdjttnoPzHwMvmqlKTwnENSQvVtOFRVd8AvpHkU1V114jqNDEc11gYbv3+ep7zwuOn3OcStZpUM73bas8k5wIrB8+pqt+Yi0pJ88kDtdvQkHeJWk2qmYbHhcBfAh8DHpy76kiSFoKZhscDVfWR2frSJI+lC6IjgQJeC6wHLqC7urkTeHlV/Us7/izgNLrgekNVfXm26iJJ6m+m4fG3Sf4QuBi4/6HCqpp6tHfnPgB8qapelmQP4FHAO4DLq+qcJGcCZwJvT3I4cDJwBPA44KtJ/l1VeQWkKTlGIc29mYbH6vb61oGyAp7Q9wuTPAZ4PnAqQFX9HPh5khOAY9ph5wFXAG8HTgDOr6r7gTuSbACOAq7s+92aDNONUXz17FOGBot3uEkzN6PwqKpDZ/E7nwBsAT6Z5CnAtcAbgQOr6p72ffckOaAdfxDw7YHzN7ayHSQ5HTgd4JBDDpnFKmuxmC5YvMNNmrmZTk9yylTlVbUr1/+7A08HXl9VVyX5AF0X1dCvn+qrh9TnXOBcgFWrVk15jCTp4Ztpt9UzB7b3Ao4FvgPsSnhsBDZW1VXt/V/Thce9SZa3q47lwOaB4w8eOH8FsGkXvleSNEtm2m31+sH3SfYF/ueufGFV/VOSHyT51apaTxdEN7ef1cA57fWSdsqlwGeTvI9uwPww4Opd+W5J0uzY1SnZf0b3R3xXvR74TLvT6nbgNXSTNK5JchpwN3ASQFWtS7KGLlweAM7wTitJGq+Zjnn8LVvHGZYATwLW7OqXVtX1wKopdh075PizAR/llaR5YqZXHu8d2H4AuKuqNs5BfaQFZdgzJT5PosVupmMe30hyIFsHzm+duypJC8ewW3+d80qL3YwWg0rycrpB6pOAlwNXJXFKdkmaUDPttnon8Myq2gyQZBnwVbrbbCVJE2amy9Du9lBwND/qca4kaZGZ6ZXHl5J8Gfhce/8K4ItzUyVJ0ny3szXMn0g359Rbk/w28Dy66UKuBD4zgvpJkuahnXU9vR/4CUBVfb6q3lJVb6a76nj/3FZNkjRf7Sw8VlbVDdsXVtVaukWbJEkTaGfhsdc0+x45mxWRJC0cOwuPa5L83vaFbf6pa+emSpKk+W5nd1u9Cbg4ye+wNSxWAXsAL53DekmS5rFpw6Oq7gWem+TXgSNb8d9V1dfmvGaSpHlrpnNbfR34+hzXRZK0QPiUuCSpN8NDktTbrq4kKGkXnfiqU9i05b4p92288w5WrDx0h3LXB9F8Y3hII7Zpy31TrgECcMu7T3V9EC0Ihoc0B4atMAiw4bbbefyI6yPNNsNDmgPDVhiE7upCWugcMJck9WZ4SJJ6MzwkSb0ZHpKk3gwPSVJvhockqTfDQ5LUm+EhSeptbOGRZEmS65J8ob1fmuSyJLe21/0Gjj0ryYYk65O8aFx1liR1xnnl8UbgloH3ZwKXV9VhwOXtPUkOB04GjgCOAz6cZMmI6ypJGjCW6UmSrAD+A3A28JZWfAJwTNs+D7gCeHsrP7+q7gfuSLIBOAq4coRVfliGzaLqHEeSFqpxzW31fuBtwD4DZQdW1T0AVXVPkgNa+UHAtweO29jKdpDkdOB0gEMOOWSWq7zrhs2i6hxHkhaqkXdbJTke2FxV1870lCnKaqoDq+rcqlpVVauWLVu2y3WUJE1vHFceRwO/leTFwF7AY5L8FXBvkuXtqmM5sLkdvxE4eOD8FcCmkdZYkrSNkV95VNVZVbWiqlbSDYR/rapeDVwKrG6HrQYuaduXAicn2TPJocBhwNUjrrYkacB8Ws/jHGBNktOAu4GTAKpqXZI1wM3AA8AZVfXg+KopSRpreFTVFXR3VVFVPwKOHXLc2XR3ZkmS5oH5dOUhaYjplrV93LKlXPTZT4+4Rpp0hoe0AEy3rO1dF3pRrtFzbitJUm+GhySpN8NDktSb4SFJ6s3wkCT1ZnhIknozPCRJvRkekqTeDA9JUm8+YS7pYRu2WiY4fcpiZXhIetiGrZYJTp+yWNltJUnqzSsPaYFzxl2Ng+EhLXDOuKtxMDwkzch0g+Ibbrudx4+4Phovw0PSjEw3KH7Lu08dbWU0dg6YS5J688pD0jaGdU/ZNaVBhoekbQzrnrJrSoPstpIk9eaVhzSBvHNKD5fhIS1iwx4g3HDb7Rx75senPMfuKc2E4SEtYsMeIDQg9HA55iFJ6s0rD0lzaljXmfNuLWyGh6Q5NazrzHm3FraRd1slOTjJ15PckmRdkje28qVJLktya3vdb+Ccs5JsSLI+yYtGXWdJ0rbGMebxAPCfq+pJwLOBM5IcDpwJXF5VhwGXt/e0fScDRwDHAR9OsmQM9ZYkNSMPj6q6p6q+07Z/AtwCHAScAJzXDjsPeEnbPgE4v6rur6o7gA3AUSOttCRpG2O92yrJSuBpwFXAgVV1D3QBAxzQDjsI+MHAaRtb2VSfd3qStUnWbtmyZc7qLUmTbmzhkWRv4CLgTVX14+kOnaKspjqwqs6tqlVVtWrZsmWzUU1J0hTGEh5JHkEXHJ+pqs+34nuTLG/7lwObW/lG4OCB01cAm0ZVV0nSjsZxt1WAjwO3VNX7BnZdCqxu26uBSwbKT06yZ5JDgcOAq0dVX0nSjsbxnMfRwO8CNya5vpW9AzgHWJPkNOBu4CSAqlqXZA1wM92dWmdU1YMjr7Uk6d+MPDyq6h+YehwD4Ngh55wN+ESRtIgMe/IcfPp8IfAJc0ljMezJc/Dp84XA8JA07zgf1vxneEiad5wPa/5zSnZJUm+GhySpN8NDktSbYx6SFgxv750/DA9JC8Z0t/d+9exTDJYRMjwkLQo+NzJajnlIknrzykPSoudDh7PP8JC06A3r0nKcZNcZHpIm1myPk5z4qlPYtOW+KfcttjAyPCRpCrtyW/CmLfdNzKC94SFJU9iV24I33HY7j5/ris0Thscsme5ydZL+QUmTYFiw3PLuU0dfmTExPGbJdJerk/QPStJk8DkPSVJvhockqTfDQ5LUm2MekjRGC/XZEMNDkkZg2HMjG267nWPP/PiU58znZ0MMD0kagV25vXe6BxXH/QiA4SFJ89R0DyqO+xEAB8wlSb0ZHpKk3gwPSVJvhockqbcFM2Ce5DjgA8AS4GNVdc446jHsnuxx3/kgSaO0IMIjyRLgL4AXAhuBa5JcWlU3z8X37WyG3KnuyR73nQ+SNEoLIjyAo4ANVXU7QJLzgROAOQkPZ8iVpOmlqsZdh51K8jLguKr6T+397wLPqqrXbXfc6cDp7e2vAuun+Lj9gX+ew+rOV5PabrDttn2yPNx2P76qlu3soIVy5ZEpynZIvao6Fzh32g9K1lbVqtmq2EIxqe0G227bJ8uo2r1Q7rbaCBw88H4FsGlMdZGkibdQwuMa4LAkhybZAzgZuHTMdZKkibUguq2q6oEkrwO+THer7ieqat0ufty03VqL2KS2G2z7pJrUto+k3QtiwFySNL8slG4rSdI8YnhIknqbmPBIclyS9Uk2JDlz3PWZqSSfSLI5yU0DZUuTXJbk1va638C+s1ob1yd50UD5M5Lc2PZ9MEla+Z5JLmjlVyVZOXDO6vYdtyZZPaIm/5skByf5epJbkqxL8sZWvqjbn2SvJFcn+W5r93+ZhHYPSrIkyXVJvtDeT0Tbk9zZ6nx9krWtbH62vaoW/Q/dIPttwBOAPYDvAoePu14zrPvzgacDNw2U/RlwZts+E/jTtn14a9uewKGtzUvavquB59A9M/P3wL9v5X8I/GXbPhm4oG0vBW5vr/u17f1G3PblwNPb9j7A91sbF3X7Wx33btuPAK4Cnr3Y273d7+AtwGeBL0zYv/k7gf23K5uXbR/pP4hx/bRf4pcH3p8FnDXuevWo/0q2DY/1wPK2vRxYP1W76O5Oe0475nsD5a8EPjp4TNvene7J1Awe0/Z9FHjlmH8Pl9DNbzYx7QceBXwHeNaktJvuOa7Lgd9ga3hMStvvZMfwmJdtn5Ruq4OAHwy839jKFqoDq+oegPZ6QCsf1s6D2vb25ducU1UPAP8K/Mo0nzUW7fL6aXT/F77o29+6ba4HNgOXVdVEtLt5P/A24JcDZZPS9gK+kuTadNMtwTxt+4J4zmMWzGh6k0VgWDuna/+unDNSSfYGLgLeVFU/bt23Ux46RdmCbH9VPQg8NcljgYuTHDnN4Yum3UmOBzZX1bVJjpnJKVOULci2N0dX1aYkBwCXJfneNMeOte2TcuWx2KY3uTfJcoD2urmVD2vnxra9ffk25yTZHdgXuG+azxqpJI+gC47PVNXnW/HEtL+q/jdwBXAck9Huo4HfSnIncD7wG0n+isloO1W1qb1uBi6mm1F8frZ9lP154/qhu8K6nW5Q6aEB8yPGXa8e9V/JtmMe72HbAbQ/a9tHsO0A2u1sHUC7hm7Q9aEBtBe38jPYdgBtTdteCtxBN3i2X9teOuJ2B/g08P7tyhd1+4FlwGPb9iOB/wUcv9jbPcXv4Ri2jnks+rYDjwb2Gdj+R7r/aZiXbR/5P4hx/QAvprtb5zbgneOuT496fw64B/gF3f8dnEbXR3k5cGt7XTpw/DtbG9fT7rBo5auAm9q+D7F1doG9gAuBDXR3aDxh4JzXtvINwGvG0Pbn0V063wBc335evNjbDzwZuK61+ybgj1v5om73FL+HY9gaHou+7XR3g363/ayj/Z2ar213ehJJUm+TMuYhSZpFhockqTfDQ5LUm+EhSerN8JAk9WZ4SJJ6Mzw0UZJckWRV2/5im/5jtj77U0leNlufN0pJTk3yoXHXQwvHpMxtJe2gql487jpIC5VXHpr3kqxM8r0kH0tyU5LPJHlBkm+1hWuOSvLodAtnXdMWETqhnfvIJOcnuSHJBXTTfTz0uXcm2b9t/02byXTdwGymJPlpkrPTLcz07SQH7qS6z0/yj0luf+gqJJ33tLrfmOQVrfyYtMWO2vsPJTm1bZ+T5OZW7/e2smVJLmptvCbJ0UN+X7u1tj12oGxDkgOT/Me2CNB1Sb46VXu2v4JK8tOB7be2774hbZEqTSbDQwvFE4EP0E3d8WvAq+imL/kj4B100zR8raqeCfw68J4kjwb+APhZVT0ZOBt4xpDPf21VPYNuWoc3JPmVVv5o4NtV9RTgm8Dv7aSey1u9jgfOaWW/DTwVeArwgla35cM+IMlS4KV08689GXh32/UB4L+3Np4IfGyq86vql3Rrn7y0fd6zgDur6l7gH4BnV9XT6CYefNtO2jNYr98EDqObrO+pwDOSPH+m52txsdtKC8UdVXUjQJJ1wOVVVUlupJs4cgXdbKx/1I7fCziEbiXGDwJU1Q1Jbhjy+W9I8tK2fTDdH8kfAT8HHro6uJZuMarp/E37433zwP/VPw/4XHXTrN+b5BvAM4EfD/mMHwP/D/hYkr8b+P4XAIcPTEn/mCT7VNVPpviMC4A/Bj5JWzGula8ALmjhtQfdBHgz9Zvt57r2fm+639M3e3yGFgnDQwvF/QPbvxx4/0u6f8cPAidW1frBk9of2mkncGvrRryAboW1nyW5gi58AH5RWyeAe5Cd/zczWM9s97q9B9j26n8v6BbpSXIUcCzdH/7X0a2qt1ur4//dSR0ArgSemGQZ8BK2Xr38OfC+qrq0tftPpqtXul/gHgPt+G9V9dEZfL8WObuttFh8GXh9+2NHkqe18m8Cv9PKjqTr9trevsC/tOD4NbqprGfTN4FXpFsdcBnd1dDVwF10VxJ7JtmXLiweWvxq36r6IvAmui4igK/QBQntuIfKd9AC72LgfcAtVfWjtmtf4Idte/WQ0+9ka/feCXTrqEP3O35tqx9JDkq3aJEmkFceWiz+K93ypTe0ALmTbtzhI8AnW3fV9XR/tLf3JeD32zHrgW/Pct0upltb+rt0V0Fvq6p/Akiyhm7q9VvZ2h20D3BJkr3o/m//za38DcBftHruThdKvz/N915At67DqQNlfwJcmOSHdO08dIrz/kf7/qvppgD/PwBV9ZUkTwKubBn9U+DVbF2cSBPEKdklSb3ZbSVJ6s1uK6mnJO8ETtqu+MKqOnsMdXkN8Mbtir9VVWeMui6aLHZbSZJ6s9tKktSb4SFJ6s3wkCT1ZnhIknr7/6Z6j5cqGGRfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.histplot(df.median_house_value) #, bins=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a45bab7",
   "metadata": {},
   "source": [
    "Yes, distribution of median_house_value variable has a long tail."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c2ae452",
   "metadata": {},
   "source": [
    "### Features\n",
    "\n",
    "For the rest of the homework, you'll need to use only these columns:\n",
    "\n",
    "* `'latitude'`,\n",
    "* `'longitude'`,\n",
    "* `'housing_median_age'`,\n",
    "* `'total_rooms'`,\n",
    "* `'total_bedrooms'`,\n",
    "* `'population'`,\n",
    "* `'households'`,\n",
    "* `'median_income'`,\n",
    "* `'median_house_value'`\n",
    "\n",
    "Select only them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "b5eb8a66",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>37.88</td>\n",
       "      <td>-122.23</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>452600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>37.86</td>\n",
       "      <td>-122.22</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>358500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37.85</td>\n",
       "      <td>-122.24</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>352100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>37.85</td>\n",
       "      <td>-122.25</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>341300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>37.85</td>\n",
       "      <td>-122.25</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>342200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20635</th>\n",
       "      <td>39.48</td>\n",
       "      <td>-121.09</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1665.0</td>\n",
       "      <td>374.0</td>\n",
       "      <td>845.0</td>\n",
       "      <td>330.0</td>\n",
       "      <td>1.5603</td>\n",
       "      <td>78100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20636</th>\n",
       "      <td>39.49</td>\n",
       "      <td>-121.21</td>\n",
       "      <td>18.0</td>\n",
       "      <td>697.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>356.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>2.5568</td>\n",
       "      <td>77100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20637</th>\n",
       "      <td>39.43</td>\n",
       "      <td>-121.22</td>\n",
       "      <td>17.0</td>\n",
       "      <td>2254.0</td>\n",
       "      <td>485.0</td>\n",
       "      <td>1007.0</td>\n",
       "      <td>433.0</td>\n",
       "      <td>1.7000</td>\n",
       "      <td>92300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20638</th>\n",
       "      <td>39.43</td>\n",
       "      <td>-121.32</td>\n",
       "      <td>18.0</td>\n",
       "      <td>1860.0</td>\n",
       "      <td>409.0</td>\n",
       "      <td>741.0</td>\n",
       "      <td>349.0</td>\n",
       "      <td>1.8672</td>\n",
       "      <td>84700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20639</th>\n",
       "      <td>39.37</td>\n",
       "      <td>-121.24</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2785.0</td>\n",
       "      <td>616.0</td>\n",
       "      <td>1387.0</td>\n",
       "      <td>530.0</td>\n",
       "      <td>2.3886</td>\n",
       "      <td>89400.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20640 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0         37.88    -122.23                41.0        880.0           129.0   \n",
       "1         37.86    -122.22                21.0       7099.0          1106.0   \n",
       "2         37.85    -122.24                52.0       1467.0           190.0   \n",
       "3         37.85    -122.25                52.0       1274.0           235.0   \n",
       "4         37.85    -122.25                52.0       1627.0           280.0   \n",
       "...         ...        ...                 ...          ...             ...   \n",
       "20635     39.48    -121.09                25.0       1665.0           374.0   \n",
       "20636     39.49    -121.21                18.0        697.0           150.0   \n",
       "20637     39.43    -121.22                17.0       2254.0           485.0   \n",
       "20638     39.43    -121.32                18.0       1860.0           409.0   \n",
       "20639     39.37    -121.24                16.0       2785.0           616.0   \n",
       "\n",
       "       population  households  median_income  median_house_value  \n",
       "0           322.0       126.0         8.3252            452600.0  \n",
       "1          2401.0      1138.0         8.3014            358500.0  \n",
       "2           496.0       177.0         7.2574            352100.0  \n",
       "3           558.0       219.0         5.6431            341300.0  \n",
       "4           565.0       259.0         3.8462            342200.0  \n",
       "...           ...         ...            ...                 ...  \n",
       "20635       845.0       330.0         1.5603             78100.0  \n",
       "20636       356.0       114.0         2.5568             77100.0  \n",
       "20637      1007.0       433.0         1.7000             92300.0  \n",
       "20638       741.0       349.0         1.8672             84700.0  \n",
       "20639      1387.0       530.0         2.3886             89400.0  \n",
       "\n",
       "[20640 rows x 9 columns]"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = [\n",
    "    'latitude',\n",
    "    'longitude',\n",
    "    'housing_median_age',\n",
    "    'total_rooms',\n",
    "    'total_bedrooms',\n",
    "    'population',\n",
    "    'households',\n",
    "    'median_income',\n",
    "    'median_house_value'\n",
    "           ]\n",
    "\n",
    "df = df[features]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82eda40a",
   "metadata": {},
   "source": [
    "### Question 1\n",
    "\n",
    "Find a feature with missing values. How many missing values does it have?\n",
    "- **207**\n",
    "- 307\n",
    "- 408\n",
    "- 508"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "72f5a138",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "latitude                0\n",
       "longitude               0\n",
       "housing_median_age      0\n",
       "total_rooms             0\n",
       "total_bedrooms        207\n",
       "population              0\n",
       "households              0\n",
       "median_income           0\n",
       "median_house_value      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5821c8c5",
   "metadata": {},
   "source": [
    "### Question 2\n",
    "\n",
    "What's the median (50% percentile) for variable 'population'?\n",
    "- 1133\n",
    "- 1122\n",
    "- **1166**\n",
    "- 1188"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "32b60888",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    20640.000000\n",
       "mean      1425.476744\n",
       "std       1132.462122\n",
       "min          3.000000\n",
       "25%        787.000000\n",
       "50%       1166.000000\n",
       "75%       1725.000000\n",
       "max      35682.000000\n",
       "Name: population, dtype: float64"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['population'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "274aaa27",
   "metadata": {},
   "source": [
    "### Split the data\n",
    "\n",
    "* Shuffle the initial dataset, use seed `42`.\n",
    "* Split your data in train/val/test sets, with 60%/20%/20% distribution.\n",
    "* Make sure that the target value ('median_house_value') is not in your dataframe.\n",
    "* Apply the log transformation to the median_house_value variable using the `np.log1p()` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c0cb339c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20640, 4128, 4128, 12384)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = len(df)\n",
    "\n",
    "n_val = int(n * 0.2)\n",
    "n_test = int(n * 0.2)\n",
    "n_train = n - n_val - n_test\n",
    "\n",
    "n, n_val, n_test, n_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e5355e7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "idx = np.arange(n)\n",
    "np.random.seed(42)\n",
    "np.random.shuffle(idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9b49b479",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(12384, 4128, 4128)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train = df.iloc[idx[:n_train]]\n",
    "df_val = df.iloc[idx[n_train:n_train + n_val]]\n",
    "df_test = df.iloc[idx[n_train + n_val:]]\n",
    "len(df_train), len(df_val), len(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "719dbb3b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20046</th>\n",
       "      <td>36.06</td>\n",
       "      <td>-119.01</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1505.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1392.0</td>\n",
       "      <td>359.0</td>\n",
       "      <td>1.6812</td>\n",
       "      <td>47700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3024</th>\n",
       "      <td>35.14</td>\n",
       "      <td>-119.46</td>\n",
       "      <td>30.0</td>\n",
       "      <td>2943.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1565.0</td>\n",
       "      <td>584.0</td>\n",
       "      <td>2.5313</td>\n",
       "      <td>45800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15663</th>\n",
       "      <td>37.80</td>\n",
       "      <td>-122.44</td>\n",
       "      <td>52.0</td>\n",
       "      <td>3830.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1310.0</td>\n",
       "      <td>963.0</td>\n",
       "      <td>3.4801</td>\n",
       "      <td>500001.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20484</th>\n",
       "      <td>34.28</td>\n",
       "      <td>-118.72</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3051.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1705.0</td>\n",
       "      <td>495.0</td>\n",
       "      <td>5.7376</td>\n",
       "      <td>218600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9814</th>\n",
       "      <td>36.62</td>\n",
       "      <td>-121.93</td>\n",
       "      <td>34.0</td>\n",
       "      <td>2351.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>428.0</td>\n",
       "      <td>3.7250</td>\n",
       "      <td>278000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "20046     36.06    -119.01                25.0       1505.0             NaN   \n",
       "3024      35.14    -119.46                30.0       2943.0             NaN   \n",
       "15663     37.80    -122.44                52.0       3830.0             NaN   \n",
       "20484     34.28    -118.72                17.0       3051.0             NaN   \n",
       "9814      36.62    -121.93                34.0       2351.0             NaN   \n",
       "\n",
       "       population  households  median_income  median_house_value  \n",
       "20046      1392.0       359.0         1.6812             47700.0  \n",
       "3024       1565.0       584.0         2.5313             45800.0  \n",
       "15663      1310.0       963.0         3.4801            500001.0  \n",
       "20484      1705.0       495.0         5.7376            218600.0  \n",
       "9814       1063.0       428.0         3.7250            278000.0  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "37678f76",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = df_train.reset_index(drop=True)\n",
    "df_val = df_val.reset_index(drop=True)\n",
    "df_test = df_test.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9b9986db",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = np.log1p(df_train.median_house_value.values)\n",
    "y_val = np.log1p(df_val.median_house_value.values)\n",
    "y_test = np.log1p(df_test.median_house_value.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f84debba",
   "metadata": {},
   "outputs": [],
   "source": [
    "del df_train['median_house_value']\n",
    "del df_val['median_house_value']\n",
    "del df_test['median_house_value']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d75b544c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>36.06</td>\n",
       "      <td>-119.01</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1505.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1392.0</td>\n",
       "      <td>359.0</td>\n",
       "      <td>1.6812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>35.14</td>\n",
       "      <td>-119.46</td>\n",
       "      <td>30.0</td>\n",
       "      <td>2943.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1565.0</td>\n",
       "      <td>584.0</td>\n",
       "      <td>2.5313</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37.80</td>\n",
       "      <td>-122.44</td>\n",
       "      <td>52.0</td>\n",
       "      <td>3830.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1310.0</td>\n",
       "      <td>963.0</td>\n",
       "      <td>3.4801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>34.28</td>\n",
       "      <td>-118.72</td>\n",
       "      <td>17.0</td>\n",
       "      <td>3051.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1705.0</td>\n",
       "      <td>495.0</td>\n",
       "      <td>5.7376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36.62</td>\n",
       "      <td>-121.93</td>\n",
       "      <td>34.0</td>\n",
       "      <td>2351.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>428.0</td>\n",
       "      <td>3.7250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0     36.06    -119.01                25.0       1505.0             NaN   \n",
       "1     35.14    -119.46                30.0       2943.0             NaN   \n",
       "2     37.80    -122.44                52.0       3830.0             NaN   \n",
       "3     34.28    -118.72                17.0       3051.0             NaN   \n",
       "4     36.62    -121.93                34.0       2351.0             NaN   \n",
       "\n",
       "   population  households  median_income  \n",
       "0      1392.0       359.0         1.6812  \n",
       "1      1565.0       584.0         2.5313  \n",
       "2      1310.0       963.0         3.4801  \n",
       "3      1705.0       495.0         5.7376  \n",
       "4      1063.0       428.0         3.7250  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b5fa04d",
   "metadata": {},
   "source": [
    "\n",
    "### Question 3\n",
    "\n",
    "* We need to deal with missing values for the column from Q1.\n",
    "* We have two options: fill it with 0 or with the mean of this variable.\n",
    "* Try both options. For each, train a linear regression model without regularization using the code from the lessons.\n",
    "* For computing the mean, use the training only!\n",
    "* Use the validation dataset to evaluate the models and compare the RMSE of each option.\n",
    "* Round the RMSE scores to 2 decimal digits using `round(score, 2)`\n",
    "* Which option gives better RMSE?\n",
    "\n",
    "Options:\n",
    "- With 0\n",
    "- With mean\n",
    "- **Both are equally good**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "8bf79ab1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_linear_regression(X, y):\n",
    "    ones = np.ones(X.shape[0])\n",
    "    X = np.column_stack([ones, X])\n",
    "\n",
    "    XTX = X.T.dot(X)\n",
    "    XTX_inv = np.linalg.inv(XTX)\n",
    "    w_full = XTX_inv.dot(X.T).dot(y)\n",
    "    \n",
    "    return w_full[0], w_full[1:]\n",
    "\n",
    "def rmse(y, y_pred):\n",
    "    se = (y - y_pred) ** 2\n",
    "    mse = se.mean()\n",
    "    return round(np.sqrt(mse), 2)\n",
    "\n",
    "def prepare_X_zero(df):\n",
    "    df_num = df\n",
    "    df_num = df_num.fillna(0)\n",
    "    X = df_num.values\n",
    "    return X\n",
    "\n",
    "def prepare_X_mean(df):\n",
    "    df_num = df\n",
    "    df_num['total_bedrooms'] = df_num['total_bedrooms'].fillna(df_num['total_bedrooms'].median())\n",
    "    X = df_num.values\n",
    "    return X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "fb874422",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-11.686975241610243,\n",
       " array([-2.82087184e-01, -2.76255120e-01,  2.91329621e-03, -8.64531514e-06,\n",
       "         1.50811922e-04, -2.08205659e-04,  5.69546573e-04,  1.71308139e-01]))"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_zero = prepare_X_zero(df_train)\n",
    "\n",
    "w0, w = train_linear_regression(X_train_zero, y_train)\n",
    "\n",
    "y_pred = w0 + X_train_zero.dot(w)\n",
    "\n",
    "w0, w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "200c840d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.34"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD4CAYAAAAdIcpQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZ1UlEQVR4nO3df3DU933n8ecbMApWbCMMS2SQI3MhDsa0qSy4kjQ3btw2vtAat41zdJyGUjimOadx766pgcw0f1xhPJNMGl+uTYeTg/HViUPTFNNe09imcRLPYBtYo8oYKxBsg2DNCswvS7J+8e4f+5VYrb7iuxLa/e6P12Mms7uf/e7q/c0Iv/T9fD7fz8fcHRERkSuZEncBIiJS+hQWIiISSWEhIiKRFBYiIhJJYSEiIpGmxV1AocyePdsbGxvjLkNEpKzs37//tLvPyW2v2LBobGxk3759cZchIlJWzOzNsHZ1Q4mISCSFhYiIRFJYiIhIJIWFiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRCpYWJjZt8wsbWavZLXNMrNnzOxw8FiX9d5GMztiZu1m9oms9jvMrC1473+bmRWqZhERCVfIO7gfA/4P8HhW2wZgt7s/bGYbgtcPmdltwCpgMXAT8KyZfdDdB4FvAuuBF4B/Bu4GflDAukVKwoNr1tCVTo9qr00keGTbthgqkmpWsLBw95+YWWNO80rgzuD5duA54KGg/Ul37wVeN7MjwDIzewO43t33AJjZ48C9KCykCnSl07QsXTqqfd3evTFUI9Wu2GMWc909BRA8JoL2ecDxrOM6grZ5wfPc9lBmtt7M9pnZvs7OzkktXESkmpXKQoJh4xB+hfZQ7r4V2ArQ3NyszcWl5KhrScpVscPilJnVu3vKzOqBoX81HUBD1nHzgZNB+/yQdpGyNFbXUnNLC+tWrBjRdrC1FUKOFYlDscNiF7AaeDh4fCqr/dtm9jUyA9wLgZfcfdDMLprZLwMvAp8FvlHkmkUKbkpf36gQWaaxCSkhBQsLM/sOmcHs2WbWAXyZTEjsMLO1wDHgPgB3P2hmO4BXgQHggWAmFMDnyMysmkFmYFuD2yIiRVbI2VC/N8Zbd41x/GZgc0j7PuD2SSxNRETGSXdwi4hIJIWFiIhEUliIiEgkhYWIiERSWIiISKRSuYNbpKKMdae2brSTcqWwECmAse7U1o12Uq7UDSUiIpF0ZSFSZg60to5aRwq0GKEUlsJCpMyErSMF2udCCkvdUCIiEklhISIikRQWIiISSWEhIiKRFBYiIhJJs6FEKkTYlFpNp5XJorAQqRBhU2o1nVYmi7qhREQkksJCREQiKSxERCSSwkJERCIpLEREJJJmQ4lcpbCNjrTJkVQahYXIVQrb6EibHEmlUTeUiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRFJYiIhIJIWFiIhEUliIiEikWMLCzP67mR00s1fM7Dtm9h4zm2Vmz5jZ4eCxLuv4jWZ2xMzazewTcdQsIlLNih4WZjYP+ALQ7O63A1OBVcAGYLe7LwR2B68xs9uC9xcDdwN/bWZTi123iEg1i6sbahoww8ymAdcCJ4GVwPbg/e3AvcHzlcCT7t7r7q8DR4BlxS1XRKS6FT0s3P0E8FXgGJACzrv708Bcd08Fx6SARPCRecDxrK/oCNpGMbP1ZrbPzPZ1dnYW6hRERKpO0RcSDMYiVgK3AOeAvzOzz1zpIyFtHnagu28FtgI0NzeHHiMyUWGry4JWmJXqEMeqs78GvO7unQBm9n3gI8ApM6t395SZ1QND/yo7gIasz88n020lUlRhq8uCVpiV6hDHmMUx4JfN7FozM+Au4BCwC1gdHLMaeCp4vgtYZWY1ZnYLsBB4qcg1i4hUtaJfWbj7i2b2PSAJDAAvk+k6ei+ww8zWkgmU+4LjD5rZDuDV4PgH3H2w2HWLiFSzWDY/cvcvA1/Oae4lc5URdvxmYHOh6xIRkXC6g1tERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCSSwkJERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCRSLAsJilSL3oEBkqnU8Ot3+vvpHRigZpr+6Ul50W+sSAElUym2/HQmidoFAJzsPkcylWJ5Q0PEJyfHgdZW1q1YMaq9NpHgkW3bilKDVAaFhUiBJWoX0HDDYgCusUNAT9F+9pS+vtDd/dZpdz8ZJ41ZiIhIJIWFiIhEUjeUiMSit7eXZDI5qr2pqYmampoYKpIrUViISCySySRbtrSRSCwZbkun29i0CZYvXx5jZRJGYSEisUkkltDQoGAoBxqzEBGRSAoLERGJpLAQEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpJvyRHI8uGYNXen0qPaDra0QsoKrSDVQWIjk6EqnQ5f1XlakZb2HNkx6p7+fPcePD7c31ddr0ySJjX7zRGLWOzAwIhja0ml2vraQzp77aUkuAiDddZRNHyvepkkiuRQWIjFLplKc7P44LckmANpPP0/djLlMnzpleNOk4Mh4ChQhprAws5lAC3A74MAfAu3Ad4FG4A3g0+5+Njh+I7AWGAS+4O4/LHrRIgV0jdUPB0O662jk8bl7e4P295bCiuu36hHgX9z9U2Y2HbgW2ATsdveHzWwDsAF4yMxuA1YBi4GbgGfN7IPuPhhT7SKxy93bG4q/v7dUl6KHhZldD/wn4A8A3L0P6DOzlcCdwWHbgeeAh4CVwJPu3gu8bmZHgGXAnqIWLlJisvf2huLv7y3VJY77LBYAncA2M3vZzFrMrBaY6+4pgOAxERw/Dzie9fmOoG0UM1tvZvvMbF9nZ2fhzkBEpMrEERbTgCbgm+7+S0AXmS6nsVhIm4cd6O5b3b3Z3ZvnzJlz9ZWKiAiQZ1iY2UfzactTB9Dh7i8Gr79HJjxOmVl98N31QDrr+OxO2PnAyQn+bBERmYB8ryy+kWdbJHd/CzhuZrcGTXcBrwK7gNVB22rgqeD5LmCVmdWY2S3AQuClifxsERGZmCsOcJvZcuAjwBwz+x9Zb10PTL2Kn/vHwBPBTKijwBoywbXDzNYCx4D7ANz9oJntIBMoA8ADmgkl5cp9kLacpUTa0mncb46pIpH8RM2Gmg68Nzjuuqz2C8CnJvpD3f0A0Bzy1l1jHL8Z2DzRnydSKga8k+0HGvjQ7MuzmNpPP8+AX4qxKpFoVwwLd/8x8GMze8zd3yxSTSIVbdaM94+Y8pq5Ca87voJE8pDvfRY1ZraVzN3Vw59x948XoigRGb+wLq5LHjpxUGTc8g2LvwP+hswSHRovEClBuV1c6a6jdA8MxFyVVIp8w2LA3b9Z0EpE5KrldnH9XPemyiTJd+rsP5rZfzOzejObNfS/glYmIiIlI98ri6H7H76Y1eZklu4QEZEKl1dYuPsthS5ERERKV15hYWafDWt398cntxwRESlF+XZDZW9I/B4yN88lAYWFiEgVyLcb6o+zX5vZDcD/K0hFIiJScia6+VE3mQX9RKQMHWhtZd2KFaPaaxMJHtm2LYaKpNTlO2bxj1zeQ2IqsAjYUaiiRKSwpvT10bJ06aj2dXv3xlCNlIN8ryy+mvV8AHjT3TsKUI9I2egdGCCZSvFOfz97jmc2c2yqr6dmWlxb24sUTr5jFj82s7lcHug+XLiSRMpDMpViy09n0tlzPy3JRaQu/ox7F7WyJJEYDpC2dJpBrSgrFSDfbqhPA18BniOzzek3zOyL7v69AtYmUvIStQuYPrWbhhsWk+46Orw2U2fPDFqSi2g//Tx1M7ppnBl3pRm6GpKJyvc35EvAUndPA5jZHOBZMluiipStB9esoStnpdaDra0Q0p+fj6G1mbIDpJTkXg2lu46y6WMpljc0RH9Yqlq+YTFlKCgCZ8h/XSmRktWVTo8a6F1W4YO82VdDGclY65HykG9Y/IuZ/RD4TvD6vwD/XJiSRESk1ETtwf0BYK67f9HMfgf4FTJjFnuAJ4pQn4iIlICoK4uvA5sA3P37wPcBzKw5eO+3ClibiAQGL/UP74L3Tn9/XrOsBi/10z0wMDyQDWh2lkxYVFg0uvu/5Ta6+z4zayxMSSKS60zP8REzrR4/cCZyltWZnuOceffXaUk2DbcNzc7SkKOMV1RYvOcK782YzEJE5MqyZ1rVzajJ6zNTbe6InfNKbXaWlI+oPy/2mtl/zW00s7XA/sKUJCIipSbqyuJPgH8ws/u5HA7NwHTgtwtYl4iIlJArhoW7nwI+Yma/CtweNP9/d//XglcmIiIlI9+1oX4E/KjAtYiISInSlAgREYmksBARkUgKCxERiaSwEBGRSAoLERGJpB1PpOoNbQg0ZGhjIG0KJHJZbFcWZjbVzF42s38KXs8ys2fM7HDwWJd17EYzO2Jm7Wb2ibhqlso0tCFQS7KJlmQTnT33s+WnM0cEiEi1i7Mb6kHgUNbrDcBud18I7A5eY2a3AauAxcDdwF+b2dQi1yoVLlG7gIYbFgdrL72fRO2CuEsSKSmxhIWZzQdWAC1ZzSuB7cHz7cC9We1Punuvu78OHAGWFalUEREhvjGLrwN/BlyX1TbX3VMA7p4ys0TQPg94Ieu4jqBtFDNbD6wHuPnmmye5ZClnYXttQ7Df9vveF0NFIuWl6GFhZr8JpN19v5ndmc9HQto87EB33wpsBWhubg49RqpLb28vyWSSo+3tbFq0aLh9aPA63/22ewcGhge+h2gjIakmcVxZfBS4x8w+SWa/jOvN7G+BU2ZWH1xV1ANDfwZ2AA1Zn58PnCxqxVK2kskkW7a08bNjv0FLzxwgs6fDpo+lWN7QEPHprO9JpTjZ/fGK30joQGsr61asGNFWc+ONfOZznxvR1tTURE1NfntqSGUoeli4+0ZgI0BwZfGn7v4ZM/sKsBp4OHh8KvjILuDbZvY14CZgIfBSkcuWMpZILOHEjFoabsjuvUyO+3uusfqK30hoSl8fLUuXjmj7raefZsuWNhKJJQCk021s2gTLly+Po0SJSSlNIn8Y2BFsrHQMuA/A3Q+a2Q7gVWAAeMDdB+MrU6pB9p7XkOlycq/ecbBEYgkNDQqHahZrWLj7c8BzwfMzwF1jHLcZ2Fy0wqTqZe95DZkup4EKHJ/IDcV3+vvpHRjQzYgyin4jRMYwtOc1DHU5dcdbUAHkhuLJ7nMkU+Mbz5HqoLAQqXLZoXiNHQJ64i1ISlLlTOMQEZGCUViIiEgkhYWIiERSWIiISCSFhYiIRNJsKKk62fcWvNPfrzWeRPKgsJCqk31vQWfPDB4/cIa6Gd00zoy7svi5D464SQ/g7MWL3Dg4EFNFUioUFlJRhlaZHdLW1sbg4KJRxw3dWzB9ajd1M7Qg3pAB7xxxkx7A4Y40dY1naGyMry6Jn8JCKsrQKrNDi961t/+curpExKckW/ZNegA108a/6KJUHoWFVJzsRe/S6baYqxGpDJoNJSIikXRlIRXlL//iL2htXUrtjMxeE6fPvYjbXPzd2TAvdDdeEcmDwkIqSs/Zs/zKnDnDGx3tt5nADSRPaAsUkauhbigREYmkKwsRGZfBwT7a2tpHtWtf7sqmsBCRcTlzpp3t29/mQx+63KZ9uSufwkJExm3WrA9pT+4qozELERGJpLAQEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpPssROSKzl+8wIEXX+LEa13A5cUZ08fhP997b7zFSdEoLKRsPbhmDV05W4C+fvgwifkxFVSh7JLz4Zk3cMdNIxdnPNHVFWtdUlwKCylbF1Ip1ucsO/7su+8y6JdiqkikcikspCTk7p0N0QvTnb5wgS1HF5OoXXC57d2fc6a7m8aZhapUpDopLKQk5O6dne/CdInaBSP2i55mPyponSLVSmEhJSN772wRKS1FDwszawAeB94HXAK2uvsjZjYL+C7QCLwBfNrdzwaf2QisBQaBL7j7D4tdtxSX9kwQKS1xXFkMAP/T3ZNmdh2w38yeAf4A2O3uD5vZBmAD8JCZ3QasAhYDNwHPmtkH3V37ZFYw7ZlQXsLCXcFeWYoeFu6eAlLB84tmdgiYB6wE7gwO2w48BzwUtD/p7r3A62Z2BFgG7Clu5VJs2jOhfOSGu4K98sQ6ZmFmjcAvAS8Cc4Mgwd1TZpYIDpsHvJD1sY6gLez71gPrAW6++eYCVS0iYRTulS225T7M7L3A3wN/4u4XrnRoSJuHHejuW9292d2b58yZMxlliogIMYWFmV1DJiiecPfvB82nzKw+eL8eGLo1twNoyPr4fOBksWoVEZEYwsLMDHgUOOTuX8t6axewOni+Gngqq32VmdWY2S3AQuClYtUrIiLxjFl8FPh9oM3MDgRtm4CHgR1mthY4BtwH4O4HzWwH8CqZmVQPaCaUSGnT1OfKE8dsqOcJH4cAuGuMz2wGNhesKBGZVJr6XHl0B7eUjdy/Vs9evMiNU7VoYKnS7KjKorCQspH71+rhjkbq5mjRwHKRG/ZtbW0MDi6KsSIZD4WFlJXsv1ZrpiUijpZSkhv27e0/p64uQWNjrGVJnhQWIlI02WGfTrfFXI2Mh8JCRCbkrVOn2PXEE8DlrVZPvNbFNbW1JBoiPixlR2EhRRe20ZH6r8uPDQ5yz7yRW63ecdM8dp04EWtdUhgKCym63I2O4Mr91z/YuZP+rq4Rf70CXLh4AeqKVLRIlVNYSCxyNzq6Uv91f1cX98ybN+KvV4Cvvxm6RJiIFIDCQgout9tJXU4i5UdhIZMuLBx27hykvv7DgKZMipQjhYVMutwxiUw4LNeUySrx1qlTnD53cMT4kmZIlT+FhRRE9piEwqG62OAgv1g3k+zxJc2QKn+xbX4kIiLlQ2EhIiKRFBYiIhJJYSEiIpEUFiIiEklhISIikTR1Vq7KZC4K+MrLB4bn5WevA5U+dQqCBetEJB4KC7kq410U8EoG+vq45wOjVzH9vydPTlq9IjIxCgu5auNZFFCqU9hd3afPvcjUmgbuuOPeeIuTvCgsRKTgwu7q3m8zOXCuL9a6JH8KCxGJzYXz54d32wPo6jlM+tDTLLj1Vh7Zti3GyiSXwkJEYmN+aXi3PYDj58+xbtEiHn3rrRirkjCaOisiIpF0ZSGxyJ4mC5nBzncuvCfGikTkShQWMi6Ttetd9jRZyAx2/vR896TUKOXvQGsr61asGNVem0hoLCMmCgsZl/CNjbTrnUyuKX19tCxdOqp93d69o9oeXLOGrnR6VLuCZXIpLGRMY92dfeONi/Le2OgHO3eSOvHiqPn16nKSiQi74jjY2sqedetGHRsWLDJxCgsZ02Tcnd3f1RU6v15dTjIRYVccy8YIBXVlTS6FhVxRvndn/2DnTvq7wtd1mve+opQqMsJ4urIkmsJChl3N4HV/V9fwfHmt6yRSecomLMzsbuARYCrQ4u4Px1xSWRtrPGLnzkHq6z8MjN3lFLY6rL87WyvDSlkI6556pb2d22+9ddSxxe6yKuXB+rIICzObCvwV8OtAB7DXzHa5+6vxVpa/sP84AzQ1NVFTUzOp35PPMWOPRyyPHLwOWx02eWIw73MQidNY4x5hXVbNLS2h4x5h4TLewAkLhlIerC+LsACWAUfc/SiAmT0JrAQKEhZ79uyZ9O9sa2vj0UdfYebMxuG2c+feYO3aNpYsWTL2ByO+59ArbbzbdZIPNpyk7rrrADh78SJH3no/v9j08VE/69mnnqLn7FnOXrzIsVPLmDH9JG9fOM+s62/gXNerQIqfvXwAgJOnn+faGTcNvwY417Wf7q7pHD+fCYuzPSdxeugbnMLx89eOaDt+/r30Db7J2Z4zw6+H3h/0Xo6fPzj8vbltud9x/Py1I9oABvw0b/e8Ofw6rG0i9Y2n5uzvnmjNY537eGoe69zHU3Mp/H+a7jpKKRlr3CMsXMYbOGHBMN7B+rCAKtRViLn7pH/pZDOzTwF3u/u64PXvA//R3T+fc9x6YH3w8lagvaiF5m82cDruImKk89f56/xL1/vdfU5uY7lcWVhI26iUc/etwNbCl3N1zGyfuzfHXUdcdP46f51/+Z1/uSwk2AE0ZL2eD2iajYhIkZRLWOwFFprZLWY2HVgF7Iq5JhGRqlEW3VDuPmBmnwd+SGbq7Lfc/WDEx0pZyXeVFZjOv7rp/MtQWQxwi4hIvMqlG0pERGKksBARkUgKiwIzs2+ZWdrMXslqm2Vmz5jZ4eCxLs4aC2mM87/PzA6a2SUzK7sphOMxxvl/xcxeM7N/M7N/MLOZMZZYUGOc//8Kzv2AmT1tZjfFWWMhhZ1/1nt/amZuZrPjqG28FBaF9xhwd07bBmC3uy8EdgevK9VjjD7/V4DfAX5S9GqK7zFGn/8zwO3u/gvAz4CNxS6qiB5j9Pl/xd1/wd0/DPwT8OfFLqqIHmP0+WNmDWSWLzpW7IImSmFRYO7+E+DtnOaVwPbg+Xbg3mLWVExh5+/uh9y9VO+un1RjnP/T7j4QvHyBzH1DFWmM87+Q9bKWkBtsK8UY//4B/hL4M8ro3Mti6mwFmuvuKQB3T5lZIu6CJDZ/CHw37iKKzcw2A58FzgO/GnM5RWVm9wAn3L3VLGxxitKkKwuRmJjZl4AB4Im4ayk2d/+SuzeQOffPRx1fKczsWuBLlGHXm8IiHqfMrB4geBy9gL1UNDNbDfwmcL9X981O3wZ+N+4iiug/ALcArWb2BpkuyKSZlfx+kgqLeOwCVgfPVwNPxViLFFmwkddDwD3uXnWbkZvZwqyX9wCvxVVLsbl7m7sn3L3R3RvJrHvX5O5vxVxaJN3BXWBm9h3gTjLLEp8CvgzsBHYAN5OZDXGfu4cNgpW9Mc7/beAbwBzgHHDA3T8RU4kFNcb5bwRqgDPBYS+4+x/FUmCBjXH+nySzhcAl4E3gj9z9RFw1FlLY+bv7o1nvvwE0u3spL1kOKCxERCQP6oYSEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpLAQEZFI/w5wXnJrzWsf9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.histplot(y_pred, color='red', alpha=0.5, bins=50)\n",
    "sns.histplot(y_train, color='blue', alpha=0.5, bins=50)\n",
    "rmse(y_train, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "0ea77f14",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_val_zero = prepare_X_zero(df_val)\n",
    "y_pred = w0 + X_val_zero.dot(w)\n",
    "rmse(y_val, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "466bddda",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-11.753065907839352,\n",
       " array([-2.82166266e-01, -2.76757635e-01,  2.93028421e-03, -1.45062254e-05,\n",
       "         2.41468074e-04, -2.04002173e-04,  4.92851655e-04,  1.72979157e-01]))"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_mean = prepare_X_mean(df_train)\n",
    "\n",
    "w0, w = train_linear_regression(X_train_mean, y_train)\n",
    "\n",
    "y_pred = w0 + X_train_mean.dot(w)\n",
    "\n",
    "w0, w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "34a71b04",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.34"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD4CAYAAAAdIcpQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZ1UlEQVR4nO3df3DU933n8ecbMApWbCMMS2SQI3MhDsa0qSy4kjQ3btw2vtAat41zdJyGUjimOadx766pgcw0f1xhPJNMGl+uTYeTg/HViUPTFNNe09imcRLPYBtYo8oYKxBsg2DNCswvS7J+8e4f+5VYrb7iuxLa/e6P12Mms7uf/e7q/c0Iv/T9fD7fz8fcHRERkSuZEncBIiJS+hQWIiISSWEhIiKRFBYiIhJJYSEiIpGmxV1AocyePdsbGxvjLkNEpKzs37//tLvPyW2v2LBobGxk3759cZchIlJWzOzNsHZ1Q4mISCSFhYiIRFJYiIhIJIWFiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRCpYWJjZt8wsbWavZLXNMrNnzOxw8FiX9d5GMztiZu1m9oms9jvMrC1473+bmRWqZhERCVfIO7gfA/4P8HhW2wZgt7s/bGYbgtcPmdltwCpgMXAT8KyZfdDdB4FvAuuBF4B/Bu4GflDAukVKwoNr1tCVTo9qr00keGTbthgqkmpWsLBw95+YWWNO80rgzuD5duA54KGg/Ul37wVeN7MjwDIzewO43t33AJjZ48C9KCykCnSl07QsXTqqfd3evTFUI9Wu2GMWc909BRA8JoL2ecDxrOM6grZ5wfPc9lBmtt7M9pnZvs7OzkktXESkmpXKQoJh4xB+hfZQ7r4V2ArQ3NyszcWl5KhrScpVscPilJnVu3vKzOqBoX81HUBD1nHzgZNB+/yQdpGyNFbXUnNLC+tWrBjRdrC1FUKOFYlDscNiF7AaeDh4fCqr/dtm9jUyA9wLgZfcfdDMLprZLwMvAp8FvlHkmkUKbkpf36gQWaaxCSkhBQsLM/sOmcHs2WbWAXyZTEjsMLO1wDHgPgB3P2hmO4BXgQHggWAmFMDnyMysmkFmYFuD2yIiRVbI2VC/N8Zbd41x/GZgc0j7PuD2SSxNRETGSXdwi4hIJIWFiIhEUliIiEgkhYWIiERSWIiISKRSuYNbpKKMdae2brSTcqWwECmAse7U1o12Uq7UDSUiIpF0ZSFSZg60to5aRwq0GKEUlsJCpMyErSMF2udCCkvdUCIiEklhISIikRQWIiISSWEhIiKRFBYiIhJJs6FEKkTYlFpNp5XJorAQqRBhU2o1nVYmi7qhREQkksJCREQiKSxERCSSwkJERCIpLEREJJJmQ4lcpbCNjrTJkVQahYXIVQrb6EibHEmlUTeUiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRFJYiIhIJIWFiIhEUliIiEikWMLCzP67mR00s1fM7Dtm9h4zm2Vmz5jZ4eCxLuv4jWZ2xMzazewTcdQsIlLNih4WZjYP+ALQ7O63A1OBVcAGYLe7LwR2B68xs9uC9xcDdwN/bWZTi123iEg1i6sbahoww8ymAdcCJ4GVwPbg/e3AvcHzlcCT7t7r7q8DR4BlxS1XRKS6FT0s3P0E8FXgGJACzrv708Bcd08Fx6SARPCRecDxrK/oCNpGMbP1ZrbPzPZ1dnYW6hRERKpO0RcSDMYiVgK3AOeAvzOzz1zpIyFtHnagu28FtgI0NzeHHiMyUWGry4JWmJXqEMeqs78GvO7unQBm9n3gI8ApM6t395SZ1QND/yo7gIasz88n020lUlRhq8uCVpiV6hDHmMUx4JfN7FozM+Au4BCwC1gdHLMaeCp4vgtYZWY1ZnYLsBB4qcg1i4hUtaJfWbj7i2b2PSAJDAAvk+k6ei+ww8zWkgmU+4LjD5rZDuDV4PgH3H2w2HWLiFSzWDY/cvcvA1/Oae4lc5URdvxmYHOh6xIRkXC6g1tERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCSSwkJERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCRSLAsJilSL3oEBkqnU8Ot3+vvpHRigZpr+6Ul50W+sSAElUym2/HQmidoFAJzsPkcylWJ5Q0PEJyfHgdZW1q1YMaq9NpHgkW3bilKDVAaFhUiBJWoX0HDDYgCusUNAT9F+9pS+vtDd/dZpdz8ZJ41ZiIhIJIWFiIhEUjeUiMSit7eXZDI5qr2pqYmampoYKpIrUViISCySySRbtrSRSCwZbkun29i0CZYvXx5jZRJGYSEisUkkltDQoGAoBxqzEBGRSAoLERGJpLAQEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpJvyRHI8uGYNXen0qPaDra0QsoKrSDVQWIjk6EqnQ5f1XlakZb2HNkx6p7+fPcePD7c31ddr0ySJjX7zRGLWOzAwIhja0ml2vraQzp77aUkuAiDddZRNHyvepkkiuRQWIjFLplKc7P44LckmANpPP0/djLlMnzpleNOk4Mh4ChQhprAws5lAC3A74MAfAu3Ad4FG4A3g0+5+Njh+I7AWGAS+4O4/LHrRIgV0jdUPB0O662jk8bl7e4P295bCiuu36hHgX9z9U2Y2HbgW2ATsdveHzWwDsAF4yMxuA1YBi4GbgGfN7IPuPhhT7SKxy93bG4q/v7dUl6KHhZldD/wn4A8A3L0P6DOzlcCdwWHbgeeAh4CVwJPu3gu8bmZHgGXAnqIWLlJisvf2huLv7y3VJY77LBYAncA2M3vZzFrMrBaY6+4pgOAxERw/Dzie9fmOoG0UM1tvZvvMbF9nZ2fhzkBEpMrEERbTgCbgm+7+S0AXmS6nsVhIm4cd6O5b3b3Z3ZvnzJlz9ZWKiAiQZ1iY2UfzactTB9Dh7i8Gr79HJjxOmVl98N31QDrr+OxO2PnAyQn+bBERmYB8ryy+kWdbJHd/CzhuZrcGTXcBrwK7gNVB22rgqeD5LmCVmdWY2S3AQuClifxsERGZmCsOcJvZcuAjwBwz+x9Zb10PTL2Kn/vHwBPBTKijwBoywbXDzNYCx4D7ANz9oJntIBMoA8ADmgkl5cp9kLacpUTa0mncb46pIpH8RM2Gmg68Nzjuuqz2C8CnJvpD3f0A0Bzy1l1jHL8Z2DzRnydSKga8k+0HGvjQ7MuzmNpPP8+AX4qxKpFoVwwLd/8x8GMze8zd3yxSTSIVbdaM94+Y8pq5Ca87voJE8pDvfRY1ZraVzN3Vw59x948XoigRGb+wLq5LHjpxUGTc8g2LvwP+hswSHRovEClBuV1c6a6jdA8MxFyVVIp8w2LA3b9Z0EpE5KrldnH9XPemyiTJd+rsP5rZfzOzejObNfS/glYmIiIlI98ri6H7H76Y1eZklu4QEZEKl1dYuPsthS5ERERKV15hYWafDWt398cntxwRESlF+XZDZW9I/B4yN88lAYWFiEgVyLcb6o+zX5vZDcD/K0hFIiJScia6+VE3mQX9RKQMHWhtZd2KFaPaaxMJHtm2LYaKpNTlO2bxj1zeQ2IqsAjYUaiiRKSwpvT10bJ06aj2dXv3xlCNlIN8ryy+mvV8AHjT3TsKUI9I2egdGCCZSvFOfz97jmc2c2yqr6dmWlxb24sUTr5jFj82s7lcHug+XLiSRMpDMpViy09n0tlzPy3JRaQu/ox7F7WyJJEYDpC2dJpBrSgrFSDfbqhPA18BniOzzek3zOyL7v69AtYmUvIStQuYPrWbhhsWk+46Orw2U2fPDFqSi2g//Tx1M7ppnBl3pRm6GpKJyvc35EvAUndPA5jZHOBZMluiipStB9esoStnpdaDra0Q0p+fj6G1mbIDpJTkXg2lu46y6WMpljc0RH9Yqlq+YTFlKCgCZ8h/XSmRktWVTo8a6F1W4YO82VdDGclY65HykG9Y/IuZ/RD4TvD6vwD/XJiSRESk1ETtwf0BYK67f9HMfgf4FTJjFnuAJ4pQn4iIlICoK4uvA5sA3P37wPcBzKw5eO+3ClibiAQGL/UP74L3Tn9/XrOsBi/10z0wMDyQDWh2lkxYVFg0uvu/5Ta6+z4zayxMSSKS60zP8REzrR4/cCZyltWZnuOceffXaUk2DbcNzc7SkKOMV1RYvOcK782YzEJE5MqyZ1rVzajJ6zNTbe6InfNKbXaWlI+oPy/2mtl/zW00s7XA/sKUJCIipSbqyuJPgH8ws/u5HA7NwHTgtwtYl4iIlJArhoW7nwI+Yma/CtweNP9/d//XglcmIiIlI9+1oX4E/KjAtYiISInSlAgREYmksBARkUgKCxERiaSwEBGRSAoLERGJpB1PpOoNbQg0ZGhjIG0KJHJZbFcWZjbVzF42s38KXs8ys2fM7HDwWJd17EYzO2Jm7Wb2ibhqlso0tCFQS7KJlmQTnT33s+WnM0cEiEi1i7Mb6kHgUNbrDcBud18I7A5eY2a3AauAxcDdwF+b2dQi1yoVLlG7gIYbFgdrL72fRO2CuEsSKSmxhIWZzQdWAC1ZzSuB7cHz7cC9We1Punuvu78OHAGWFalUEREhvjGLrwN/BlyX1TbX3VMA7p4ys0TQPg94Ieu4jqBtFDNbD6wHuPnmmye5ZClnYXttQ7Df9vveF0NFIuWl6GFhZr8JpN19v5ndmc9HQto87EB33wpsBWhubg49RqpLb28vyWSSo+3tbFq0aLh9aPA63/22ewcGhge+h2gjIakmcVxZfBS4x8w+SWa/jOvN7G+BU2ZWH1xV1ANDfwZ2AA1Zn58PnCxqxVK2kskkW7a08bNjv0FLzxwgs6fDpo+lWN7QEPHprO9JpTjZ/fGK30joQGsr61asGNFWc+ONfOZznxvR1tTURE1NfntqSGUoeli4+0ZgI0BwZfGn7v4ZM/sKsBp4OHh8KvjILuDbZvY14CZgIfBSkcuWMpZILOHEjFoabsjuvUyO+3uusfqK30hoSl8fLUuXjmj7raefZsuWNhKJJQCk021s2gTLly+Po0SJSSlNIn8Y2BFsrHQMuA/A3Q+a2Q7gVWAAeMDdB+MrU6pB9p7XkOlycq/ecbBEYgkNDQqHahZrWLj7c8BzwfMzwF1jHLcZ2Fy0wqTqZe95DZkup4EKHJ/IDcV3+vvpHRjQzYgyin4jRMYwtOc1DHU5dcdbUAHkhuLJ7nMkU+Mbz5HqoLAQqXLZoXiNHQJ64i1ISlLlTOMQEZGCUViIiEgkhYWIiERSWIiISCSFhYiIRNJsKKk62fcWvNPfrzWeRPKgsJCqk31vQWfPDB4/cIa6Gd00zoy7svi5D464SQ/g7MWL3Dg4EFNFUioUFlJRhlaZHdLW1sbg4KJRxw3dWzB9ajd1M7Qg3pAB7xxxkx7A4Y40dY1naGyMry6Jn8JCKsrQKrNDi961t/+curpExKckW/ZNegA108a/6KJUHoWFVJzsRe/S6baYqxGpDJoNJSIikXRlIRXlL//iL2htXUrtjMxeE6fPvYjbXPzd2TAvdDdeEcmDwkIqSs/Zs/zKnDnDGx3tt5nADSRPaAsUkauhbigREYmkKwsRGZfBwT7a2tpHtWtf7sqmsBCRcTlzpp3t29/mQx+63KZ9uSufwkJExm3WrA9pT+4qozELERGJpLAQEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpPssROSKzl+8wIEXX+LEa13A5cUZ08fhP997b7zFSdEoLKRsPbhmDV05W4C+fvgwifkxFVSh7JLz4Zk3cMdNIxdnPNHVFWtdUlwKCylbF1Ip1ucsO/7su+8y6JdiqkikcikspCTk7p0N0QvTnb5wgS1HF5OoXXC57d2fc6a7m8aZhapUpDopLKQk5O6dne/CdInaBSP2i55mPyponSLVSmEhJSN772wRKS1FDwszawAeB94HXAK2uvsjZjYL+C7QCLwBfNrdzwaf2QisBQaBL7j7D4tdtxSX9kwQKS1xXFkMAP/T3ZNmdh2w38yeAf4A2O3uD5vZBmAD8JCZ3QasAhYDNwHPmtkH3V37ZFYw7ZlQXsLCXcFeWYoeFu6eAlLB84tmdgiYB6wE7gwO2w48BzwUtD/p7r3A62Z2BFgG7Clu5VJs2jOhfOSGu4K98sQ6ZmFmjcAvAS8Cc4Mgwd1TZpYIDpsHvJD1sY6gLez71gPrAW6++eYCVS0iYRTulS225T7M7L3A3wN/4u4XrnRoSJuHHejuW9292d2b58yZMxlliogIMYWFmV1DJiiecPfvB82nzKw+eL8eGLo1twNoyPr4fOBksWoVEZEYwsLMDHgUOOTuX8t6axewOni+Gngqq32VmdWY2S3AQuClYtUrIiLxjFl8FPh9oM3MDgRtm4CHgR1mthY4BtwH4O4HzWwH8CqZmVQPaCaUSGnT1OfKE8dsqOcJH4cAuGuMz2wGNhesKBGZVJr6XHl0B7eUjdy/Vs9evMiNU7VoYKnS7KjKorCQspH71+rhjkbq5mjRwHKRG/ZtbW0MDi6KsSIZD4WFlJXsv1ZrpiUijpZSkhv27e0/p64uQWNjrGVJnhQWIlI02WGfTrfFXI2Mh8JCRCbkrVOn2PXEE8DlrVZPvNbFNbW1JBoiPixlR2EhRRe20ZH6r8uPDQ5yz7yRW63ecdM8dp04EWtdUhgKCym63I2O4Mr91z/YuZP+rq4Rf70CXLh4AeqKVLRIlVNYSCxyNzq6Uv91f1cX98ybN+KvV4Cvvxm6RJiIFIDCQgout9tJXU4i5UdhIZMuLBx27hykvv7DgKZMipQjhYVMutwxiUw4LNeUySrx1qlTnD53cMT4kmZIlT+FhRRE9piEwqG62OAgv1g3k+zxJc2QKn+xbX4kIiLlQ2EhIiKRFBYiIhJJYSEiIpEUFiIiEklhISIikTR1Vq7KZC4K+MrLB4bn5WevA5U+dQqCBetEJB4KC7kq410U8EoG+vq45wOjVzH9vydPTlq9IjIxCgu5auNZFFCqU9hd3afPvcjUmgbuuOPeeIuTvCgsRKTgwu7q3m8zOXCuL9a6JH8KCxGJzYXz54d32wPo6jlM+tDTLLj1Vh7Zti3GyiSXwkJEYmN+aXi3PYDj58+xbtEiHn3rrRirkjCaOisiIpF0ZSGxyJ4mC5nBzncuvCfGikTkShQWMi6Ttetd9jRZyAx2/vR896TUKOXvQGsr61asGNVem0hoLCMmCgsZl/CNjbTrnUyuKX19tCxdOqp93d69o9oeXLOGrnR6VLuCZXIpLGRMY92dfeONi/Le2OgHO3eSOvHiqPn16nKSiQi74jjY2sqedetGHRsWLDJxCgsZ02Tcnd3f1RU6v15dTjIRYVccy8YIBXVlTS6FhVxRvndn/2DnTvq7wtd1mve+opQqMsJ4urIkmsJChl3N4HV/V9fwfHmt6yRSecomLMzsbuARYCrQ4u4Px1xSWRtrPGLnzkHq6z8MjN3lFLY6rL87WyvDSlkI6556pb2d22+9ddSxxe6yKuXB+rIICzObCvwV8OtAB7DXzHa5+6vxVpa/sP84AzQ1NVFTUzOp35PPMWOPRyyPHLwOWx02eWIw73MQidNY4x5hXVbNLS2h4x5h4TLewAkLhlIerC+LsACWAUfc/SiAmT0JrAQKEhZ79uyZ9O9sa2vj0UdfYebMxuG2c+feYO3aNpYsWTL2ByO+59ArbbzbdZIPNpyk7rrrADh78SJH3no/v9j08VE/69mnnqLn7FnOXrzIsVPLmDH9JG9fOM+s62/gXNerQIqfvXwAgJOnn+faGTcNvwY417Wf7q7pHD+fCYuzPSdxeugbnMLx89eOaDt+/r30Db7J2Z4zw6+H3h/0Xo6fPzj8vbltud9x/Py1I9oABvw0b/e8Ofw6rG0i9Y2n5uzvnmjNY537eGoe69zHU3Mp/H+a7jpKKRlr3CMsXMYbOGHBMN7B+rCAKtRViLn7pH/pZDOzTwF3u/u64PXvA//R3T+fc9x6YH3w8lagvaiF5m82cDruImKk89f56/xL1/vdfU5uY7lcWVhI26iUc/etwNbCl3N1zGyfuzfHXUdcdP46f51/+Z1/uSwk2AE0ZL2eD2iajYhIkZRLWOwFFprZLWY2HVgF7Iq5JhGRqlEW3VDuPmBmnwd+SGbq7Lfc/WDEx0pZyXeVFZjOv7rp/MtQWQxwi4hIvMqlG0pERGKksBARkUgKiwIzs2+ZWdrMXslqm2Vmz5jZ4eCxLs4aC2mM87/PzA6a2SUzK7sphOMxxvl/xcxeM7N/M7N/MLOZMZZYUGOc//8Kzv2AmT1tZjfFWWMhhZ1/1nt/amZuZrPjqG28FBaF9xhwd07bBmC3uy8EdgevK9VjjD7/V4DfAX5S9GqK7zFGn/8zwO3u/gvAz4CNxS6qiB5j9Pl/xd1/wd0/DPwT8OfFLqqIHmP0+WNmDWSWLzpW7IImSmFRYO7+E+DtnOaVwPbg+Xbg3mLWVExh5+/uh9y9VO+un1RjnP/T7j4QvHyBzH1DFWmM87+Q9bKWkBtsK8UY//4B/hL4M8ro3Mti6mwFmuvuKQB3T5lZIu6CJDZ/CHw37iKKzcw2A58FzgO/GnM5RWVm9wAn3L3VLGxxitKkKwuRmJjZl4AB4Im4ayk2d/+SuzeQOffPRx1fKczsWuBLlGHXm8IiHqfMrB4geBy9gL1UNDNbDfwmcL9X981O3wZ+N+4iiug/ALcArWb2BpkuyKSZlfx+kgqLeOwCVgfPVwNPxViLFFmwkddDwD3uXnWbkZvZwqyX9wCvxVVLsbl7m7sn3L3R3RvJrHvX5O5vxVxaJN3BXWBm9h3gTjLLEp8CvgzsBHYAN5OZDXGfu4cNgpW9Mc7/beAbwBzgHHDA3T8RU4kFNcb5bwRqgDPBYS+4+x/FUmCBjXH+nySzhcAl4E3gj9z9RFw1FlLY+bv7o1nvvwE0u3spL1kOKCxERCQP6oYSEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpLAQEZFI/w5wXnJrzWsf9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.histplot(y_pred, color='red', alpha=0.5, bins=50)\n",
    "sns.histplot(y_train, color='blue', alpha=0.5, bins=50)\n",
    "rmse(y_train, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "6af67d5c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_val_mean = prepare_X_mean(df_val)\n",
    "y_pred = w0 + X_val_mean.dot(w)\n",
    "rmse(y_val, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7fcbee0",
   "metadata": {},
   "source": [
    "Both methods are equally good according to RMSE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbe4f91e",
   "metadata": {},
   "source": [
    "### Question 4\n",
    "\n",
    "* Now let's train a regularized linear regression.\n",
    "* For this question, fill the NAs with 0. \n",
    "* Try different values of `r` from this list: `[0, 0.000001, 0.0001, 0.001, 0.01, 0.1, 1, 5, 10]`.\n",
    "* Use RMSE to evaluate the model on the validation dataset.\n",
    "* Round the RMSE scores to 2 decimal digits.\n",
    "* Which `r` gives the best RMSE?\n",
    "\n",
    "If there are multiple options, select the smallest `r`.\n",
    "\n",
    "Options:\n",
    "- **0**\n",
    "- 0.000001\n",
    "- 0.001\n",
    "- 0.0001\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "0ffcae4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_linear_regression_reg(X, y, r=0.001):\n",
    "    ones = np.ones(X.shape[0])\n",
    "    X = np.column_stack([ones, X])\n",
    "\n",
    "    XTX = X.T.dot(X)\n",
    "    XTX = XTX + r * np.eye(XTX.shape[0])\n",
    "\n",
    "    XTX_inv = np.linalg.inv(XTX)\n",
    "    w_full = XTX_inv.dot(X.T).dot(y)\n",
    "    \n",
    "    return w_full[0], w_full[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "d8082f85",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = prepare_X_zero(df_train)\n",
    "w0, w = train_linear_regression_reg(X_train, y_train, r=0.01)\n",
    "\n",
    "X_val = prepare_X_zero(df_val)\n",
    "y_pred = w0 + X_val.dot(w)\n",
    "rmse(y_val, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "d45926fb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "r = 0           w0 = -11.753065907839352        score = 0.33\n",
      "r = 1e-06       w0 = -11.75304972716687         score = 0.33\n",
      "r = 0.0001      w0 = -11.7514480655525          score = 0.33\n",
      "r = 0.001       w0 = -11.736907498383292        score = 0.33\n",
      "r = 0.01        w0 = -11.593456233857026        score = 0.33\n",
      "r = 0.1         w0 = -10.330762645069981        score = 0.33\n",
      "r = 1           w0 = -4.94409128150871          score = 0.33\n",
      "r = 5           w0 = -1.4885725360387094        score = 0.34\n",
      "r = 10          w0 = -0.7933211979144272        score = 0.34\n"
     ]
    }
   ],
   "source": [
    "r_list = [0, 0.000001, 0.0001, 0.001, 0.01, 0.1, 1, 5, 10]\n",
    "\n",
    "\n",
    "for r in r_list:\n",
    "    # min_rmse = 1000\n",
    "    \n",
    "    X_train = prepare_X_zero(df_train)\n",
    "    w0, w = train_linear_regression_reg(X_train, y_train, r=r)\n",
    "\n",
    "    X_val = prepare_X_zero(df_val)\n",
    "    y_pred = w0 + X_val.dot(w)\n",
    "    rmse(y_val, y_pred)\n",
    "\n",
    "    score = rmse(y_val, y_pred)\n",
    "    \n",
    "    # min_rmse = score if score < min_rmse else min_rmse\n",
    "    \n",
    "    print(\"r =\", r, f'{(10 - len(str(r))) * \" \"}',\"w0 =\", w0, f'{(25 - len(str(w0))) * \" \"}', \"score =\", score)\n",
    "#print(\"min_rmse\", min_rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0e99989",
   "metadata": {},
   "source": [
    "### Question 5 \n",
    "\n",
    "* We used seed 42 for splitting the data. Let's find out how selecting the seed influences our score.\n",
    "* Try different seed values: `[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]`.\n",
    "* For each seed, do the train/validation/test split with 60%/20%/20% distribution.\n",
    "* Fill the missing values with 0 and train a model without regularization.\n",
    "* For each seed, evaluate the model on the validation dataset and collect the RMSE scores. \n",
    "* What's the standard deviation of all the scores? To compute the standard deviation, use `np.std`.\n",
    "* Round the result to 3 decimal digits (`round(std, 3)`)\n",
    "\n",
    "> Note: Standard deviation shows how different the values are.\n",
    "> If it's low, then all values are approximately the same.\n",
    "> If it's high, the values are different. \n",
    "> If standard deviation of scores is low, then our model is *stable*.\n",
    "\n",
    "Options:\n",
    "- 0.16\n",
    "- 0.00005\n",
    "- **0.005**\n",
    "- 0.15555\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "c90ae0e5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "for seed = 0 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     32.58    -117.05                22.0       2101.0           399.0   \n",
      "1     33.92    -117.97                32.0       2620.0           398.0   \n",
      "2     38.65    -121.84                29.0       3167.0           548.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1551.0       371.0         4.1518  \n",
      "1      1296.0       429.0         5.7796  \n",
      "2      1554.0       534.0         4.3487  \n",
      "for seed = 1 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     34.06    -118.36                39.0       2810.0           670.0   \n",
      "1     36.78    -119.78                37.0       2185.0           455.0   \n",
      "2     37.73    -122.42                46.0       1819.0           411.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1109.0       624.0         3.2500  \n",
      "1      1143.0       438.0         1.9784  \n",
      "2      1534.0       406.0         4.0132  \n",
      "for seed = 2 score = 0.33 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     33.60    -117.65                15.0       5736.0             NaN   \n",
      "1     38.62    -120.91                12.0       4545.0           748.0   \n",
      "2     33.93    -118.23                35.0       1149.0           277.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      2529.0       762.0         6.4114  \n",
      "1      2033.0       718.0         4.1843  \n",
      "2       909.0       214.0         1.7411  \n",
      "for seed = 3 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     39.21    -123.19                22.0       1542.0           291.0   \n",
      "1     39.96    -120.93                15.0       1666.0           351.0   \n",
      "2     33.79    -117.99                21.0       2695.0           707.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0       821.0       285.0         3.5917  \n",
      "1       816.0       316.0         2.9559  \n",
      "2      1888.0       683.0         3.2857  \n",
      "for seed = 4 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     33.99    -118.10                36.0       1529.0           290.0   \n",
      "1     33.83    -118.13                44.0       1710.0           333.0   \n",
      "2     38.09    -122.23                26.0       4397.0           997.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1271.0       287.0         3.6875  \n",
      "1       786.0       344.0         4.2917  \n",
      "2      2539.0       965.0         2.4875  \n",
      "for seed = 5 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     32.57    -117.07                14.0       1527.0           357.0   \n",
      "1     33.92    -117.95                18.0       2825.0           660.0   \n",
      "2     38.19    -120.03                26.0       7005.0          1358.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1224.0       363.0         2.7361  \n",
      "1      1590.0       643.0         3.6106  \n",
      "2       416.0       189.0         2.1250  \n",
      "for seed = 6 score = 0.35 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     33.86    -117.60                23.0       2949.0           473.0   \n",
      "1     34.00    -118.34                44.0       3183.0           513.0   \n",
      "2     34.08    -117.86                31.0       2524.0           349.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1671.0       477.0         5.1950  \n",
      "1      1183.0       473.0         5.0407  \n",
      "2      1003.0       343.0         7.5196  \n",
      "for seed = 7 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     34.06    -118.31                31.0       2827.0          1084.0   \n",
      "1     33.81    -118.31                30.0       1773.0           356.0   \n",
      "2     37.81    -122.24                52.0       2513.0           502.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      3107.0       993.0         2.0278  \n",
      "1       905.0       352.0         4.3056  \n",
      "2      1048.0       518.0         3.6750  \n",
      "for seed = 8 score = 0.35 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     37.78    -122.46                47.0       1682.0           379.0   \n",
      "1     39.14    -121.61                44.0       2035.0           476.0   \n",
      "2     33.97    -118.04                29.0       2376.0           700.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0       837.0       375.0         5.2806  \n",
      "1      1030.0       453.0         1.4661  \n",
      "2      1968.0       680.0         2.6082  \n",
      "for seed = 9 score = 0.34 \n",
      "\n",
      "   latitude  longitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
      "0     38.67    -121.80                10.0       2086.0           380.0   \n",
      "1     33.66    -117.97                22.0       3914.0           600.0   \n",
      "2     40.28    -124.25                32.0       1430.0           419.0   \n",
      "\n",
      "   population  households  median_income  \n",
      "0      1073.0       378.0         4.5526  \n",
      "1      1871.0       607.0         5.8541  \n",
      "2       434.0       187.0         1.9417  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.34, 0.34, 0.33, 0.34, 0.34, 0.34, 0.35, 0.34, 0.35, 0.34]"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seed_values = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n",
    "\n",
    "def split_dataset(df, seed):\n",
    "    n = len(df)\n",
    "\n",
    "    # split parameters\n",
    "    n_val = int(n * 0.2)\n",
    "    n_test = int(n * 0.2)\n",
    "    n_train = n - n_val - n_test\n",
    "\n",
    "    idx = np.arange(n)\n",
    "    np.random.seed(seed)\n",
    "    np.random.shuffle(idx)\n",
    "    \n",
    "    # split dataset into 3 parts\n",
    "    df_train = df.iloc[idx[:n_train]]\n",
    "    df_val = df.iloc[idx[n_train:n_train + n_val]]\n",
    "    df_test = df.iloc[idx[n_train + n_val:]]\n",
    "    \n",
    "    # reset indexes\n",
    "    df_train = df_train.reset_index(drop=True)\n",
    "    df_val = df_val.reset_index(drop=True)\n",
    "    df_test = df_test.reset_index(drop=True)\n",
    "    \n",
    "    # normalize target variable\n",
    "    y_train = np.log1p(df_train.median_house_value.values)\n",
    "    y_val = np.log1p(df_val.median_house_value.values)\n",
    "    y_test = np.log1p(df_test.median_house_value.values)\n",
    "    \n",
    "    # drop target variable from the dataset\n",
    "    del df_train['median_house_value']\n",
    "    del df_val['median_house_value']\n",
    "    del df_test['median_house_value']\n",
    "    return df_train, df_val, df_test, y_train, y_val, y_test\n",
    "    \n",
    "scores = []\n",
    "\n",
    "for seed_value in seed_values:\n",
    "\n",
    "    df_train, df_val, df_test, y_train, y_val, y_test = split_dataset(df, seed_value)\n",
    "    \n",
    "    X_train_zero = prepare_X_zero(df_train)\n",
    "    w0, w = train_linear_regression(X_train_zero, y_train)\n",
    "    y_pred = w0 + X_train_zero.dot(w)\n",
    "\n",
    "    X_val_zero = prepare_X_zero(df_val)\n",
    "    y_pred = w0 + X_val_zero.dot(w)\n",
    "    score = rmse(y_val, y_pred)\n",
    "    \n",
    "    scores.append(score)\n",
    "    \n",
    "    print('for seed =', seed_value, 'score =', score, '\\n')\n",
    "    print(df_train.head(n=3))\n",
    "    \n",
    "scores\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "56880952",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.005"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(np.std(scores), 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab3a3d45",
   "metadata": {},
   "source": [
    "### Question 6\n",
    "\n",
    "* Split the dataset like previously, use seed 9.\n",
    "* Combine train and validation datasets.\n",
    "* Fill the missing values with 0 and train a model with `r=0.001`. \n",
    "* What's the RMSE on the test dataset?\n",
    "\n",
    "Options:\n",
    "- **0.35**\n",
    "- 0.135\n",
    "- 0.450\n",
    "- 0.245\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "aaa65382",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.8250e+01, -1.2204e+02,  3.8000e+01, ...,  6.3200e+02,\n",
       "         2.5400e+02,  2.8438e+00],\n",
       "       [ 3.7770e+01, -1.2086e+02,  2.8000e+01, ...,  5.3500e+02,\n",
       "         2.3200e+02,  2.3523e+00],\n",
       "       [ 3.4070e+01, -1.1828e+02,  1.4000e+01, ...,  2.2260e+03,\n",
       "         7.9200e+02,  2.2552e+00],\n",
       "       ...,\n",
       "       [ 3.7970e+01, -1.2120e+02,  3.9000e+01, ...,  2.7000e+02,\n",
       "         9.7000e+01,  6.0582e+00],\n",
       "       [ 3.7780e+01, -1.2241e+02,  5.2000e+01, ...,  1.5200e+03,\n",
       "         6.1400e+02,  1.4554e+00],\n",
       "       [ 3.8050e+01, -1.2130e+02,  5.2000e+01, ...,  6.2000e+01,\n",
       "         2.5000e+01,  1.1500e+00]])"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seed_values = [9]\n",
    "df_train, df_val, df_test, y_train, y_val, y_test = split_dataset(df, seed_values)\n",
    "\n",
    "df_full_train = pd.concat([df_train, df_val])\n",
    "\n",
    "df_full_train = df_full_train.reset_index(drop=True)\n",
    "\n",
    "X_full_train = prepare_X_zero(df_full_train)\n",
    "X_full_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "adbbf60d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.34"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_full_train = np.concatenate([y_train, y_val])\n",
    "\n",
    "w0, w = train_linear_regression_reg(X_full_train, y_full_train, r=0.001)\n",
    "\n",
    "X_test = prepare_X_zero(df_test)\n",
    "y_pred = w0 + X_test.dot(w)\n",
    "score = rmse(y_test, y_pred)\n",
    "score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b69591f",
   "metadata": {},
   "source": [
    "## Submit the results\n",
    "- Submit your results here: https://forms.gle/WCVb4KMNsgbkuJtr6\n",
    "- If your answer doesn't match options exactly, select the closest one.\n",
    "- You can submit your solution multiple times. In this case, only the last submission will be used\n",
    "\n",
    "## Deadline\n",
    "\n",
    "The deadline for submitting is, 19 September 2022, 23:00 CET. After that, the form will be closed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63bc5de7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}