lab_1_perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DVL7_bgmIAPR"
   },
   "source": [
    "# Perceptron Lab\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "6ZbYjZZZ_yLV"
   },
   "outputs": [],
   "source": [
    "from sklearn.base import BaseEstimator, ClassifierMixin\n",
    "from sklearn.linear_model import Perceptron\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sCcEPx5VIORj"
   },
   "source": [
    "## 1. (40%) Correctly implement and submit your own code for the perceptron learning algorithm. \n",
    "\n",
    "### Code requirements\n",
    "- Shuffle the data each epoch.\n",
    "- A way to create a random train/test split. Write your own. In the future you can use the scikit-learn version if you want.\n",
    "- Use Stochastic/On-line training updates: Iterate and update weights after each training instance (i.e. do not attempt batch updates)\n",
    "- Implement a stopping criteria: when your model has trained for a number of epochs with no significant improvement in accuracy, stop training. Note that the weights/accuracy do not usually change monotonically.\n",
    "- Use your perceptron to solve the Debug data. We provide you with several parameters, and you should be able to replicate our results every time. When you are confident it is correct, run your perceptron on the Evaluation data with the same parameters, and print your final weights and accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "_a2KSZ_7AN0G"
   },
   "outputs": [],
   "source": [
    "class PerceptronClassifier(BaseEstimator,ClassifierMixin):\n",
    "\n",
    "    def __init__(self, lr=.1, shuffle=True):\n",
    "        \"\"\" \n",
    "            Initialize class with chosen hyperparameters.\n",
    "        Args:\n",
    "            lr (float): A learning rate / step size.\n",
    "            shuffle: Whether to shuffle the training data each epoch. DO NOT \n",
    "            SHUFFLE for evaluation / debug datasets.\n",
    "        \"\"\"\n",
    "        self.lr = lr\n",
    "        self.shuffle = shuffle\n",
    "\n",
    "    def fit(self, X, y, initial_weights=None):\n",
    "        \"\"\" \n",
    "            Fit the data; run the algorithm and adjust the weights to find a \n",
    "            good solution\n",
    "        Args:\n",
    "            X (array-like): A 2D numpy array with the training data, excluding\n",
    "            targets\n",
    "            y (array-like): A 2D numpy array with the training targets\n",
    "            initial_weights (array-like): allows the user to provide initial \n",
    "            weights\n",
    "        Returns:\n",
    "            self: this allows this to be chained, e.g. model.fit(X,y).predict(X_test)\n",
    "        \"\"\"\n",
    "        self.weights = self.initialize_weights() if not initial_weights else initial_weights\n",
    "\n",
    "        return self\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\" \n",
    "            Predict all classes for a dataset X\n",
    "        Args:\n",
    "            X (array-like): A 2D numpy array with the training data, excluding \n",
    "            targets\n",
    "        Returns:\n",
    "            array, shape (n_samples,)\n",
    "                Predicted target values per element in X.\n",
    "        \"\"\"\n",
    "        pass\n",
    "\n",
    "    def initialize_weights(self):\n",
    "        \"\"\" Initialize weights for perceptron. Don't forget the bias!\n",
    "        Returns:\n",
    "        \"\"\"\n",
    "\n",
    "        return [0]\n",
    "\n",
    "    def score(self, X, y):\n",
    "        \"\"\" \n",
    "            Return accuracy of model on a given dataset. Must implement own \n",
    "            score function.\n",
    "        Args:\n",
    "            X (array-like): A 2D numpy array with data, excluding targets\n",
    "            y (array-like): A 2D numpy array with targets\n",
    "        Returns:\n",
    "            score : float\n",
    "                Mean accuracy of self.predict(X) wrt. y.\n",
    "        \"\"\"\n",
    "\n",
    "        return 0\n",
    "\n",
    "    def _shuffle_data(self, X, y):\n",
    "        \"\"\" \n",
    "            Shuffle the data! This _ prefix suggests that this method should \n",
    "            only be called internally.\n",
    "            It might be easier to concatenate X & y and shuffle a single 2D \n",
    "            array, rather than shuffling X and y exactly the same way, \n",
    "            independently.\n",
    "        \"\"\"\n",
    "        pass\n",
    "\n",
    "    ### Not required by sk-learn but required by us for grading. Returns the weights.\n",
    "    def get_weights(self):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KibCIXIThpbE"
   },
   "source": [
    "## 1.1 Debug and Evaluation\n",
    "\n",
    "Debug and Evaluate your model using the parameters below\n",
    "\n",
    "Learning Rate = 0.1 \\ \n",
    "Deterministic = 10 [This means run it 10 epochs and should be the same everytime you run it] \\ \n",
    "Shuffle = False \\ \n",
    "Initial Weights = All zeros\n",
    "\n",
    "---\n",
    "\n",
    "### 1.1.1 Debug\n",
    "\n",
    "Debug your model by running it on the [debug dataset](https://raw.githubusercontent.com/cs472ta/CS472/master/datasets/linsep2nonorigin.arff)\n",
    "\n",
    "Expected Results:\n",
    "\n",
    "Accuracy = [0.88]\\\n",
    "Final Weights = [-0.23  0.18 -0.1 ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "KgAyy82gixIF"
   },
   "outputs": [],
   "source": [
    "# Load debug data\n",
    "\n",
    "# Train on debug data\n",
    "\n",
    "# Print accuracy and weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kY3VNB1ui03N"
   },
   "source": [
    "### 1.1.2 Evaluation\n",
    "\n",
    "We will evaluate your model based on it's performance on the [evaluation dataset](https://raw.githubusercontent.com/cs472ta/CS472/master/datasets/data_banknote_authentication.arff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "2yAxA78QjDh2"
   },
   "outputs": [],
   "source": [
    "# Load evaluation data\n",
    "\n",
    "# Train on evaluation data\n",
    "\n",
    "# Print accuracy and weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9vWiTdlbR2Xh"
   },
   "source": [
    "## 2. (30%) Classifying on linearly separable and non-linearly separable data\n",
    "\n",
    "### 2.1 Create 2 datasets\n",
    "\n",
    "- Both with 8 instances using 2 real valued inputs (ranging between -1 and 1) with 4 instances from each class. \n",
    "- One data set should be linearly separable and the other not.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "4SSoasDQSKXb"
   },
   "outputs": [],
   "source": [
    "# Create 2 datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SIRG42TgSR4x"
   },
   "source": [
    "### 2.2 Train on both sets with your perceptron code (with LR=.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "KBGUn43ASiXW"
   },
   "outputs": [],
   "source": [
    "# Train on each dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v19fpixqTe-7"
   },
   "source": [
    "### 2.3 Graph the datasets and their corresponding decision line\n",
    " \n",
    " - Graph each dataset\n",
    " - Use your trained perceptrons above to determine each dataset's decision line\n",
    " - For all graphs always label the axes!\n",
    " \n",
    "![Linearly Separable Data](https://raw.githubusercontent.com/rmorain/CS472-1/master/images/perceptron/linearly_separable.png)\n",
    "\n",
    "![Not Linearly Separable](https://raw.githubusercontent.com/rmorain/CS472-1/master/images/perceptron/not_linearly_separable.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "ZCPFUAGTS2sX"
   },
   "outputs": [],
   "source": [
    "# Graph datasets and decision lines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Discuss why the perceptron won’t converge on non-linearly separable data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Discussion goes here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0ySvhd1lUGSe"
   },
   "source": [
    "## 3. (20%) Use your perceptron code to learn this version of the [voting data set](https://raw.githubusercontent.com/cs472ta/CS472/master/datasets/voting-dataset.arff).\n",
    "\n",
    "This particular task is an edited version of the standard voting set, where we have replaced all the “don’t know” values with the most common value for the particular attribute.  \n",
    "\n",
    "### 3.1 Create a table that reports the final training and test set accuracy and the number of epochs for each trial.\n",
    "\n",
    "- Try it five times with different random 70/30 splits. \n",
    "- Use your own code to randomize and make splits. \n",
    "- Report the 5 trials and the average across the 5 trials in a table.  \n",
    "\n",
    "| Trial | Training Accuracy | Test accuracy | Number of epochs |\n",
    "| --- | --- | --- | --- |\n",
    "| 1 | .95 | .55 | 4 |\n",
    "| 2 | .85 | .45 | 6 |\n",
    "| Average | .9 | .5 | 5 | \n",
    "\n",
    "*- As a rough sanity check, typical Perceptron test accuracies for the voting data set are 90%-98%.*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Romyl4G8Trki"
   },
   "outputs": [],
   "source": [
    "# Create the table any way you like"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cm4rbWkzVeoR"
   },
   "source": [
    "### 3.2 By looking at the weights, explain what the model has learned and how the individual input features affect the result. Which specific features are most critical for the voting task, and which are least critical? \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g3F0Qp-BVi1R"
   },
   "source": [
    "*Explanation goes here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NyUmJ4yWVsQ7"
   },
   "source": [
    "### 3.3 Make a graph of the average misclassification rate vs epochs (0th – final epoch).\n",
    "\n",
    "- Average the misclassification rate for the training set across your 5 trials (not across 5 epochs).\n",
    "\n",
    "\n",
    "![Average Misclassification Rate](https://raw.githubusercontent.com/rmorain/CS472-1/master/images/perceptron/avg_misclassification_rate_vs_epochs.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "2_ZRPWWVVeFM"
   },
   "outputs": [],
   "source": [
    "# Graph here\n",
    "    \n",
    "def plot_misclassification(avg_misclassification_rate):\n",
    "    \"\"\"\n",
    "        Plots the average misclassification rate\n",
    "    Args:\n",
    "        avg_misclassification_rate (array-like): A 1D array or list\n",
    "    \"\"\"\n",
    "    plt.plot(np.arange(len(avg_misclassification_rate)), avg_misclassification_rate)\n",
    "    plt.title(\"Average Misclassification Rate vs. Epoch\")\n",
    "    plt.xlabel(\"Epoch\")\n",
    "    plt.ylabel(\"Misclassification Rate\")\n",
    "    plt.show()\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oBBmeNQ7jvcQ"
   },
   "source": [
    "## 4.1 (5%) Use the perceptron algorithm from the [scikit-learn toolkit](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html) to learn the voting task above.\n",
    "- Report and compare your results with your own perceptron code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "OFQv70W2VyqJ"
   },
   "outputs": [],
   "source": [
    "# Load sklearn perceptron\n",
    "\n",
    "# Train on voting dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Report your comparison*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 (5%) Use the perceptron algorithm from the [scikit-learn toolkit](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html) to learn one other data set of your choice.\n",
    "- Try out some of the hyper-parameters that scikit-learn makes available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load sklearn perceptron\n",
    "\n",
    "# Train on your dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BqSFAXwlk3Ms"
   },
   "source": [
    "*Report what hyperparameters you experimented with & what performed well*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cTlK-kijk8Mg"
   },
   "source": [
    "## 5. (Optional 5% extra credit) Use the perceptron rule to learn the [iris task](https://raw.githubusercontent.com/cs472ta/CS472/master/datasets/iris.arff) or some other task with more than two possible output values. \n",
    "\n",
    "Note that the [iris data](https://raw.githubusercontent.com/cs472ta/CS472/master/datasets/iris.arff) set has 3 output classes, and a perceptron node only has two possible outputs.  You could implement either of the two most common ways to deal with this. For testing you just execute the novel instance on each model and combine the overall results to see which output class wins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5iY77P7gk1Nh"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "lab 1 - perceptron",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}