PyTorch_cvlassification_exercises.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "machine_shape": "hm",
      "gpuType": "A100",
      "authorship_tag": "ABX9TyMAfOKTu9tsVOW+/6YvJSTE",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/christophergaughan/PyTorch/blob/main/PyTorch_cvlassification_exercises.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Steps to Build and Fit the Model\n",
        "1. **Understand the Data**\n",
        "\n",
        "* The data from question 1 needs to be recalled or recreated. For this example, I'll assume it's a simple 2D dataset (e.g., using scikit-learn's `make_moons`, `make_circles`, or `make_regression`).\n",
        "\n",
        "2. Define the Model\n",
        "\n",
        "* Subclassing `nn.Module` allows you to create a custom model by defining layers and the forward pass.\n",
        "* Include non-linear activation functions (like `ReLU`, `Sigmoid`, or `Tanh`) to ensure the model can fit complex, non-linear data.\n",
        "* Use a sequential or layer-by-layer structure for clarity.\n",
        "\n",
        "3. Train the Model\n",
        "\n",
        "* Define a loss function (e.g., MSELoss for regression, BCEWithLogitsLoss for binary classification).\n",
        "* Choose an optimizer (e.g., Adam for adaptive learning rates).\n",
        "* Use a training loop with forward passes, loss computation, backpropagation, and weight updates.\n"
      ],
      "metadata": {
        "id": "yweolXIBANM2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Build a model by subclassing nn.Module that incorporates non-linear activation functions and is capable of fitting the data you created in 1.\n"
      ],
      "metadata": {
        "id": "z_pwZhG8xN_d"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Device agnostic code\n",
        "\n"
      ],
      "metadata": {
        "id": "lXxwniOR5J8P"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Working with NumPy Data in PyTorch: Key Points to Remember- at least for myself\n",
        "\n",
        "1. **Data Type Conversion:**\n",
        "   - Data imported from scikit-learn (or other libraries) is typically in the **NumPy data format**, which needs to be converted into PyTorch tensors for compatibility.\n",
        "   - Use `torch.tensor(data, dtype=torch.float)` to ensure the data is in the correct format.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     import torch\n",
        "     import numpy as np\n",
        "\n",
        "     numpy_data = np.array([[1.0, 2.0], [3.0, 4.0]])\n",
        "     torch_data = torch.tensor(numpy_data, dtype=torch.float)\n",
        "     ```\n",
        "\n",
        "2. **Device Compatibility:**\n",
        "   - Ensure that any tensor you create is moved to the appropriate device (`cpu` or `cuda`) before computation.\n",
        "   - A common error arises when tensors remain on the CPU but are used with a model or operation on the GPU.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "     torch_data = torch.tensor(numpy_data, dtype=torch.float).to(device)\n",
        "     ```\n",
        "\n",
        "3. **Ensure Matching Data Types:**\n",
        "   - PyTorch models and operations often require `torch.float` for numerical computations, but integers (`torch.int`) may be required for labels or indices.\n",
        "   - Carefully manage data types when converting from NumPy, as NumPy's `int32` or `float64` might cause issues in PyTorch operations.\n",
        "\n",
        "4. **Gradient Computation:**\n",
        "   - If you're using tensors for model inputs that require gradient computation, ensure `requires_grad=True` is set during tensor creation.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     torch_data = torch.tensor(numpy_data, dtype=torch.float, requires_grad=True).to(device)\n",
        "     ```\n",
        "\n",
        "5. **Keep an Eye on CPU/GPU Transfers:**\n",
        "   - Data generated during computation might end up on the CPU (e.g., results from NumPy or after detaching gradients).\n",
        "   - Always check the `.device` property of tensors, especially when combining operations.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     # Check where the tensor is located\n",
        "     print(torch_data.device)\n",
        "\n",
        "     # Move to device if necessary\n",
        "     torch_data = torch_data.to(device)\n",
        "     ```\n",
        "\n",
        "6. **Avoid Implicit Data Conversion:**\n",
        "   - Operations between NumPy arrays and PyTorch tensors may lead to implicit conversions that can cause errors.\n",
        "   - Convert NumPy arrays to PyTorch tensors explicitly before performing operations.\n",
        "\n",
        "7. **Random Seed Consistency:**\n",
        "   - Ensure reproducibility by setting random seeds for both NumPy and PyTorch, especially when generating data or initializing models.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     import numpy as np\n",
        "     import torch\n",
        "\n",
        "     np.random.seed(42)\n",
        "     torch.manual_seed(42)\n",
        "     ```\n",
        "\n",
        "8. **Batch Dimension Awareness:**\n",
        "   - Ensure that data intended for model inputs includes a batch dimension, even for single samples. Use `.unsqueeze(0)` to add the batch dimension if necessary.\n",
        "   - **Example:**\n",
        "     ```python\n",
        "     single_sample = torch.tensor([1.0, 2.0], dtype=torch.float).to(device)\n",
        "     single_sample = single_sample.unsqueeze(0)  # Shape becomes [1, 2]\n",
        "     ```\n",
        "\n",
        "---\n",
        "\n",
        "### Common Errors to Watch Out For\n",
        "\n",
        "1. **Device Mismatch:**\n",
        "   - **Example Error:** `RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0`.\n",
        "   - **Solution:** Ensure all tensors and models are on the same device.\n",
        "\n",
        "2. **Data Type Incompatibility:**\n",
        "   - **Example Error:** `RuntimeError: Expected scalar type Float but found Double`.\n",
        "   - **Solution:** Convert NumPy data to the appropriate PyTorch data type (`torch.float` for numerical tensors).\n",
        "\n",
        "3. **Missing Batch Dimension:**\n",
        "   - **Example Error:** `RuntimeError: Expected 4-dimensional input for 4-dimensional weight`.\n",
        "   - **Solution:** Add a batch dimension using `.unsqueeze()` or reshape the data appropriately.\n",
        "\n",
        "---\n",
        "\n",
        "### Key Takeaways\n",
        "\n",
        "- Always **convert NumPy data to PyTorch tensors** with the correct data type (`torch.float` or `torch.int`).\n",
        "- Pay attention to **device placement** (`cpu` or `cuda`) and ensure consistency across operations.\n",
        "- Set random seeds for reproducibility.\n",
        "- Handle batch dimensions explicitly to avoid runtime errors.\n"
      ],
      "metadata": {
        "id": "8pL_yVcMs0X0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "fSFWdepmgxQS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Convert to PyTorch tensors\n",
        "\n",
        "### Why Use .unsqueeze(1)?\n",
        "1. Shape of `y_train` and `y_test`:\n",
        "\n",
        "* The labels (`y_train` and `y_test`) generated by `make_moons` are 1D arrays with shape (`n_samples,`) (e.g., `[0, 1, 0, 1, ...]`).\n",
        "* PyTorch expects labels for binary classification to have a shape matching the model's output logits when using loss functions like `BCEWithLogitsLoss`. Typically, the logits are shaped (`n_samples, 1`).\n",
        "2. What `.unsqueeze(1)` Does:\n",
        "\n",
        "* `.unsqueeze(1)` *adds* an extra dimension, changing the shape from (`n_samples,`) to (`n_samples, 1`).\n",
        "This ensures that the labels align with the model's output logits, which are usually shaped as (`n_samples, 1`) for binary classification.\n",
        "3. Why It’s Needed:\n",
        "\n",
        "Without `.unsqueeze(1)`, you’ll likely encounter a shape mismatch error when calculating the loss, such as:"
      ],
      "metadata": {
        "id": "dn-wRslBCKuf"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lFCJJULU4jgH"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "from torch import nn\n",
        "\n",
        "# Setup **device agnostic code**\n",
        "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "device"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi"
      ],
      "metadata": {
        "id": "mIeXxE0A5OcE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a dataset with Scikit-Learn's make_moons()\n",
        "import sklearn\n",
        "from sklearn.datasets import make_moons\n",
        "# Make 1500 circles\n",
        "n_samples = 1000\n",
        "\n",
        "# Create circles\n",
        "X, y = make_moons(n_samples,\n",
        "                    noise = 0.1,# we'll increase the noise in this data set\n",
        "                    random_state=42)\n"
      ],
      "metadata": {
        "id": "rM-iAKOX5cYT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Turn data into a DataFrame\n",
        "import pandas as pd\n",
        "moons = pd.DataFrame({'X1': X[:, 0],\n",
        "                        'X2': X[:, 1],\n",
        "                        'label': y})\n",
        "moons.head()"
      ],
      "metadata": {
        "id": "ZmJUne5I5nFa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.scatter(x=X[:, 0],\n",
        "            y=X[:, 1],\n",
        "            c=y,\n",
        "            cmap=plt.cm.RdYlBu);"
      ],
      "metadata": {
        "id": "hsuG6qTN6AZe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "# Normalize features\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n"
      ],
      "metadata": {
        "id": "qLpuC7228-PM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# The data is in numpy arrays, we need to turn into pytorch tensors\n",
        "import torch\n",
        "X = torch.from_numpy(X).type(torch.float)\n",
        "y = torch.from_numpy(y).type(torch.float)"
      ],
      "metadata": {
        "id": "Tvs57FDU-XTF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Split data randomly\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X,\n",
        "                                                    y,\n",
        "                                                    test_size=0.2,\n",
        "                                                    random_state=42)\n"
      ],
      "metadata": {
        "id": "gjCls5Pf-ZOf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Define the neural network model for binary classification\n",
        "class MoonModelV2(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        # Input layer to the first hidden layer (2 input features, 64 hidden units)\n",
        "        self.layer_1 = nn.Linear(2, 64)\n",
        "        self.relu_1 = nn.ReLU()  # Apply ReLU activation for non-linearity\n",
        "\n",
        "        # Second hidden layer (64 hidden units)\n",
        "        self.layer_2 = nn.Linear(64, 64)\n",
        "        self.relu_2 = nn.ReLU()  # Apply ReLU activation for non-linearity\n",
        "\n",
        "        # Output layer (1 unit for binary classification logits)\n",
        "        self.layer_3 = nn.Linear(64, 1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        # Pass the input through the layers with activations\n",
        "        x = self.relu_1(self.layer_1(x))\n",
        "        x = self.relu_2(self.layer_2(x))\n",
        "        return self.layer_3(x)  # Return raw logits for use with BCEWithLogitsLoss\n",
        "\n"
      ],
      "metadata": {
        "id": "v1xWiZNs_BQp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Instantiate the model and move it to the appropriate device (GPU)\n",
        "model_1a = MoonModelV2().to(device)\n",
        "\n",
        "# Ensure training and testing data are also on the same device\n",
        "X_train, y_train = X_train.to(device), y_train.to(device)\n",
        "X_test, y_test = X_test.to(device), y_test.to(device)\n"
      ],
      "metadata": {
        "id": "AaCllt_fBSJV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### If you're running a jupyter notebook and re-running cells, this initializing weights could come in handy"
      ],
      "metadata": {
        "id": "6J7ljLpUTruw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to initialize weights for the linear layers\n",
        "def initialize_weights(m):\n",
        "    if isinstance(m, nn.Linear):\n",
        "        # Xavier initialization for weights (good for layers with ReLU activations)\n",
        "        nn.init.xavier_uniform_(m.weight)\n",
        "        # Set biases to zero\n",
        "        nn.init.zeros_(m.bias)\n",
        "\n",
        "# Apply the weight initialization to all layers of the model\n",
        "model_1a.apply(initialize_weights)\n"
      ],
      "metadata": {
        "id": "W77ADFruBrBn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Define the Binary Cross-Entropy Loss with Logits\n",
        "# This loss function is designed for binary classification and expects raw logits\n",
        "loss_fn = nn.BCEWithLogitsLoss()\n",
        "\n",
        "# Use the Adam optimizer with a learning rate of 0.01 for efficient training\n",
        "# Adam dynamically adjusts learning rates for each parameter\n",
        "optimizer = torch.optim.Adam(params=model_1a.parameters(), lr=0.01)\n"
      ],
      "metadata": {
        "id": "UzsFqlOBB6a0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Set manual seeds for reproducibility\n",
        "torch.manual_seed(42)\n",
        "torch.cuda.manual_seed(42)\n",
        "\n",
        "# Define the number of epochs for training- this is an easy model so it doesn't require much computing power\n",
        "epochs = 100\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    # Set the model to training mode\n",
        "    model_1a.train()\n",
        "\n",
        "    # Perform a forward pass to calculate logits\n",
        "    y_logits = model_1a(X_train).squeeze()  # Squeeze to ensure dimensions match\n",
        "\n",
        "    # Calculate the loss using BCEWithLogitsLoss\n",
        "    loss = loss_fn(y_logits, y_train.squeeze())\n",
        "\n",
        "    # Zero gradients to prevent accumulation\n",
        "    optimizer.zero_grad()\n",
        "\n",
        "    # Backpropagate the loss to compute gradients\n",
        "    loss.backward()\n",
        "\n",
        "    # Update model weights using the optimizer\n",
        "    optimizer.step()\n",
        "\n",
        "    # Set the model to evaluation mode for testing\n",
        "    model_1a.eval()\n",
        "    with torch.no_grad():  # Disable gradient computation for efficiency\n",
        "        # Forward pass for the test data\n",
        "        test_logits = model_1a(X_test).squeeze()  # Logits for test data\n",
        "\n",
        "        # Calculate test loss\n",
        "        test_loss = loss_fn(test_logits, y_test.squeeze())\n",
        "\n",
        "        # Convert logits to probabilities and round to binary predictions\n",
        "        test_pred = torch.round(torch.sigmoid(test_logits))\n",
        "\n",
        "        # Calculate test accuracy\n",
        "        test_acc = (test_pred == y_test.squeeze()).float().mean().item() * 100\n",
        "\n",
        "    # Print results every 100 epochs\n",
        "    if epoch % 10 == 0:\n",
        "        print(f\"Epoch {epoch}: Loss = {loss:.4f}, Test Loss = {test_loss:.4f}, Test Acc = {test_acc:.2f}%\")"
      ],
      "metadata": {
        "id": "Tl_SRQtECDo4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_1a.eval()\n",
        "with torch.inference_mode():\n",
        "    y_preds = torch.round(torch.sigmoid(model_1a(X_test))).squeeze()\n",
        "y_preds[:10], y_test[:10]"
      ],
      "metadata": {
        "id": "o9GRRbF4DLzG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import requests\n",
        "from pathlib import Path\n",
        "\n",
        "# 1. (Optional) Remove the existing (likely invalid) helper_functions.py\n",
        "# !rm helper_functions.py\n",
        "\n",
        "# 2. Use the *raw* GitHub URL\n",
        "url_to_download = \"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/helper_functions.py\"\n",
        "\n",
        "if Path(\"helper_functions.py\").is_file():\n",
        "    print(\"helper_functions.py already exists, skipping download\")\n",
        "else:\n",
        "    print(\"Downloading helper_functions.py\")\n",
        "    request = requests.get(url_to_download)\n",
        "    with open(\"helper_functions.py\", \"wb\") as f:\n",
        "        f.write(request.content)\n"
      ],
      "metadata": {
        "id": "qBoOemzhDrm7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from helper_functions import plot_predictions, plot_decision_boundary\n"
      ],
      "metadata": {
        "id": "X1fJXr_NDtuR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# plot decision Boundaries\n",
        "plt.figure(figsize=(12, 6))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.title(\"Train\")\n",
        "plot_decision_boundary(model_1a, X_train, y_train)\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.title(\"Test\")\n",
        "plot_decision_boundary(model_1a, X_test, y_test)"
      ],
      "metadata": {
        "id": "qEXgam_DCMJg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n"
      ],
      "metadata": {
        "id": "I51pATPcR4I9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Assuming 'device' is defined (e.g., device = 'cuda' or 'cpu')\n",
        "\n",
        "# Install torchmetrics if not installed\n",
        "!pip install torchmetrics\n",
        "\n",
        "# Import necessary modules\n",
        "import torch\n",
        "from torchmetrics.classification import BinaryAccuracy\n",
        "\n",
        "# Assuming y_test and y_preds are already defined\n",
        "\n",
        "# Move tensors to the appropriate device (e.g., CUDA or CPU)\n",
        "target = y_test.to(device)  # Ground truth labels\n",
        "preds = y_preds.to(device)  # Model predictions (logits)\n",
        "\n",
        "# Apply sigmoid if necessary (if your preds are logits)\n",
        "preds = torch.sigmoid(preds)\n",
        "\n",
        "# Initialize the BinaryAccuracy metric and move it to the same device as preds and target\n",
        "metric = BinaryAccuracy().to(device)\n",
        "\n",
        "# Update the metric with predictions and targets\n",
        "accuracy = metric(preds, target)\n",
        "\n",
        "# Print the accuracy\n",
        "print(f\"Binary Accuracy: {accuracy.item():.4f}\")\n"
      ],
      "metadata": {
        "id": "bQefqyxhR_cw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Replicate the Tanh (hyperbolic tangent) activation function in pure PyTorch.\n",
        "Feel free to reference the ML cheatsheet website for the formula.\n"
      ],
      "metadata": {
        "id": "LsyourGohQH-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn.functional as F\n",
        "\n",
        "def custom_tanh_v2(x):\n",
        "    return F.tanh(x)\n",
        "\n",
        "# Testing the function\n",
        "x = torch.tensor([0.0, 1.0, -1.0, 2.0])  # Example input tensor\n",
        "output = custom_tanh_v2(x)\n",
        "\n",
        "print(\"Input:\", x)\n",
        "print(\"Tanh Output:\", output)\n"
      ],
      "metadata": {
        "id": "eo8OAsc1TEBR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Lets make the graph"
      ],
      "metadata": {
        "id": "y-ZbBVWqkUgA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor\n",
        "A = torch.arange(-10, 10, 1, dtype = torch.float32)\n",
        "A.dtype\n"
      ],
      "metadata": {
        "id": "FdfKZ0FwWvZf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def tanh(A):\n",
        "\treturn (torch.exp(A) - torch.exp(-A)) / (torch.exp(A) + torch.exp(-A))"
      ],
      "metadata": {
        "id": "Ab0j4aMAkZoO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "plt.plot(tanh(A));"
      ],
      "metadata": {
        "id": "afBcnqZIlBXh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Create a multi-class dataset using the spirals data creation function from CS231n (see below for the code).\n",
        "* Construct a model capable of fitting the data (you may need a combination of linear and non-linear layers).\n",
        "* Build a loss function and optimizer capable of handling multi-class data (optional extension: use the Adam optimizer instead of SGD, you may have to experiment with different values of the learning rate to get it working).\n",
        "* Make a training and testing loop for the multi-class data and train a model on it to reach over 95% testing accuracy (you can use any accuracy measuring function here that you like).\n",
        "* Plot the decision boundaries on the spirals dataset from your model predictions, the plot_decision_boundary() function should work for this dataset too."
      ],
      "metadata": {
        "id": "m1nqn4Dplvyq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Code for creating a spiral dataset from CS231n\n",
        "import numpy as np\n",
        "N = 100 # number of points per class\n",
        "D = 2 # dimensionality\n",
        "K = 3 # number of classes\n",
        "X = np.zeros((N*K,D)) # data matrix (each row = single example)\n",
        "y = np.zeros(N*K, dtype='uint8') # class labels\n",
        "for j in range(K):\n",
        "  ix = range(N*j,N*(j+1))\n",
        "  r = np.linspace(0.0,1,N) # radius\n",
        "  t = np.linspace(j*4,(j+1)*4,N) + np.random.randn(N)*0.2 # theta\n",
        "  X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]\n",
        "  y[ix] = j\n",
        "# lets visualize the data\n",
        "plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)\n",
        "plt.show();"
      ],
      "metadata": {
        "id": "uJqU8F_ylLEx"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Using the spiral dataset\n",
        "In the case of the spiral dataset, you're starting with NumPy arrays (X and y) and want to split the dataset before converting it into tensors.\n",
        "\n",
        "### Here’s the main difference:\n",
        "\n",
        "* In the make_blobs example, the data was already in the format you needed, so you just directly converted it to PyTorch tensors before splitting.\n",
        "* In the spiral dataset example, since you're using train_test_split from sklearn, the data has to be in a NumPy array format for it to work. So, you convert the data to tensors after splitting (since train_test_split works with NumPy arrays).\n",
        "### Proper syntax for the spiral dataset:\n",
        "Given that you want to split the dataset into training and test sets before converting the data into PyTorch tensors, here's the updated version:"
      ],
      "metadata": {
        "id": "VyxL32AlsMID"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "qWBQl7jf55uv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Split into training and test using NumPy arrays\n",
        "X_train, X_test, y_train, y_test = train_test_split(\n",
        "    X,  # X is a NumPy array at this point\n",
        "    y,  # y is a NumPy array at this point\n",
        "    test_size=0.2,\n",
        "    random_state=42\n",
        ")\n",
        "\n",
        "# Now convert the training and test sets into PyTorch tensors\n",
        "X_train = torch.tensor(X_train, dtype=torch.float32)\n",
        "X_test = torch.tensor(X_test, dtype=torch.float32)\n",
        "y_train = torch.tensor(y_train, dtype=torch.long)  # For classification, use torch.long for labels\n",
        "y_test = torch.tensor(y_test, dtype=torch.long)\n"
      ],
      "metadata": {
        "id": "j3Tfkffco4dA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create device agnostic code\n",
        "\n",
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "device"
      ],
      "metadata": {
        "id": "gyqYo-WIsIJz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "class ImprovedSpiralModel(nn.Module):\n",
        "    def __init__(self, input_features, output_features, hidden_units=8):  # Default hidden_units to 8\n",
        "        super().__init__()\n",
        "        self.linear_layer_stack = nn.Sequential(\n",
        "            nn.Linear(input_features, hidden_units),\n",
        "            nn.BatchNorm1d(hidden_units),  # Match BatchNorm to hidden_units\n",
        "            nn.LeakyReLU(),\n",
        "            nn.Dropout(0.2),  # Dropout for regularization\n",
        "            nn.Linear(hidden_units, hidden_units),\n",
        "            nn.BatchNorm1d(hidden_units),  # Match BatchNorm to hidden_units\n",
        "            nn.LeakyReLU(),\n",
        "            nn.Dropout(0.2),  # Dropout for regularization\n",
        "            nn.Linear(hidden_units, output_features)  # Output layer\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        return self.linear_layer_stack(x)\n",
        "\n",
        "# Create an instance of the updated model\n",
        "model_4a = ImprovedSpiralModel(\n",
        "    input_features=2,  # Match input features to your dataset\n",
        "    output_features=3,  # Number of classes\n",
        "    hidden_units=8      # Match hidden_units to avoid conflicts\n",
        ").to(device)\n",
        "model_4a"
      ],
      "metadata": {
        "id": "GdnHjCZ-tNt2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to initialize weights for the linear layers\n",
        "def initialize_weights(m):\n",
        "    if isinstance(m, nn.Linear):\n",
        "        # Xavier initialization for weights (good for layers with ReLU activations)\n",
        "        nn.init.xavier_uniform_(m.weight)\n",
        "        # Set biases to zero\n",
        "        nn.init.zeros_(m.bias)\n",
        "\n",
        "# Apply the weight initialization to all layers of the model\n",
        "model_4a.apply(initialize_weights)"
      ],
      "metadata": {
        "id": "K_rONDfr6B5R"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train.shape, y_train.shape, X_test.shape, y_test.shape"
      ],
      "metadata": {
        "id": "wrWIubhrt6th"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "torch.unique(y_train)"
      ],
      "metadata": {
        "id": "lxPm019OuCFZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Use Adam optimizer as per directions above"
      ],
      "metadata": {
        "id": "9hlYuCcY1E6h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create loss and optimizer- with multiclass we use cross entropy loss\n",
        "# Note: we have a balanced training set\n",
        "loss_fn = nn.CrossEntropyLoss() # loss function measures how wrong our model our model's predictions are\n",
        "optimizer = torch.optim.Adam(params=model_4a.parameters(), # optimizer updates our model parameter's to try to reduce the loss\n",
        "                            lr=0.1)"
      ],
      "metadata": {
        "id": "-igcAUgSu-vm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Problem Explanation\n",
        "* You are using `.to(device)` on `X_test`, which is intended to move the test data to a specific device (either CPU or GPU).\n",
        "* However, the model itself (`model_4a`) might not be on the same device as `X_test`. If the model is on the CPU and you're trying to send the data to the GPU (or vice versa), this will lead to a device mismatch, which PyTorch cannot handle when performing computations.\n",
        "### Fixing the Issue\n",
        "* To fix this, ensure that both the model and the input data are on the same device. Here's the corrected approach:\n",
        "\n",
        "* Move the model to the same device as the data (whether it's the CPU or GPU).\n",
        "Ensure that you move both `X_test` and the model to the same device before performing inference."
      ],
      "metadata": {
        "id": "u3KKWAqcxC3P"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(model_4a)\n"
      ],
      "metadata": {
        "id": "9hgW2jd2wX9F"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(X_test.shape)\n"
      ],
      "metadata": {
        "id": "ReLkeW_0yspY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "\n",
        "# Replace the first layer to accept 2 features instead of 100\n",
        "model_4a.linear_layer_stack[0] = nn.Linear(2, 8)  # Update the first layer\n",
        "model_4a.to(device)  # Move the model back to the correct device\n"
      ],
      "metadata": {
        "id": "vibZEOydy0ua"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_test = X_test.to(device)\n"
      ],
      "metadata": {
        "id": "SL6Kc3-NDZVL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Ensure model is on the correct device\n",
        "model_4a.to(device)\n",
        "\n",
        "# Ensure data tensors are on the same device\n",
        "X_train, y_train = X_train.to(device), y_train.to(device)\n",
        "X_test, y_test = X_test.to(device), y_test.to(device)\n"
      ],
      "metadata": {
        "id": "evYfWyAgDZRs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(next(model_4a.parameters()).device)\n"
      ],
      "metadata": {
        "id": "BHJI0hOyDZNz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"X_test device:\", X_test.device)      # Should print 'cuda:0'\n",
        "print(\"Model device:\", next(model_4a.parameters()).device)  # Should also print 'cuda:0'\n"
      ],
      "metadata": {
        "id": "PrKSKbt2EBrV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_test[:10]"
      ],
      "metadata": {
        "id": "9SabDlSjzWym"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Get logits directly from the model\n",
        "y_logits = model_4a(X_test)\n",
        "\n",
        "# Convert logits to probabilities\n",
        "y_pred_probs = torch.softmax(y_logits, dim=1)  # Calculate probabilities across classes\n",
        "print(\"Logits:\\n\", y_logits[:5])\n",
        "print(\"Probabilities:\\n\", y_pred_probs[:5])\n"
      ],
      "metadata": {
        "id": "lgtFdHgGEnAY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"y_logits shape:\", y_logits.shape)  # Should be (batch_size, num_classes)\n"
      ],
      "metadata": {
        "id": "dq1b_zdDErMc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert our models logit outputs to prediction probabilities\n",
        "y_pred_probs = torch.softmax(y_logits, dim=1) # we want them accross the first dimension\n",
        "print(y_logits[:5])\n",
        "print(y_pred_probs[:5])"
      ],
      "metadata": {
        "id": "GhTexOMN0ZHg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_pred_probs[0]"
      ],
      "metadata": {
        "id": "tlEgAt_N05Ph"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "torch.max(y_pred_probs[0])"
      ],
      "metadata": {
        "id": "If3qVw6j1TBl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### convert our models prediction probabilities to prediction labels - done using `argmax()`--> finds the index of this argmax"
      ],
      "metadata": {
        "id": "h1BG2s601-Qj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "y_preds = torch.argmax(y_pred_probs, dim=1)\n",
        "y_preds"
      ],
      "metadata": {
        "id": "e44mw2C61tz7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_test"
      ],
      "metadata": {
        "id": "UJ81gM7S13Ws"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Calculate accuracy- out of 100 examples what percentage does our model get right?\n",
        "def accuracy_fn(y_true, y_pred):\n",
        "    correct = torch.eq(y_true, y_pred).sum().item()\n",
        "    acc = (correct / len(y_pred)) * 100\n",
        "    return acc"
      ],
      "metadata": {
        "id": "yXyVQh0z36ty"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Learning Rate Scheduler\n",
        "A static learning rate can lead to slower convergence or overshooting. Use a learning rate scheduler to reduce the learning rate during training:"
      ],
      "metadata": {
        "id": "Hk22ohs39MNJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torch.optim.lr_scheduler import StepLR\n",
        "\n",
        "# Define the optimizer and scheduler\n",
        "optimizer = torch.optim.Adam(model_4a.parameters(), lr=0.01)  # Start with a higher learning rate\n",
        "scheduler = StepLR(optimizer, step_size=20, gamma=0.5)  # Reduce LR by 50% every 20 epochs\n"
      ],
      "metadata": {
        "id": "m70DDGgz8FiC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Incorporate the scheduler step in the training loop:"
      ],
      "metadata": {
        "id": "PJ9Dw68P9irf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for epoch in range(epochs):\n",
        "    model_4a.train()\n",
        "    y_logits = model_4a(X_train).squeeze()\n",
        "    loss = loss_fn(y_logits, y_train.long())\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    # Scheduler step\n",
        "    scheduler.step()\n"
      ],
      "metadata": {
        "id": "6PXJAk6-9dQ1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Regularization\n",
        "Add regularization techniques to improve generalization:\n",
        "\n",
        "Weight Decay: Apply L2 regularization using the `weight_decay` parameter in the optimizer"
      ],
      "metadata": {
        "id": "gPVFTUOL9skQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "optimizer = torch.optim.Adam(model_4a.parameters(), lr=0.01, weight_decay=1e-4)\n"
      ],
      "metadata": {
        "id": "WUT0-X3s8iGJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Dropout: Increase dropout in the model"
      ],
      "metadata": {
        "id": "vqVvLgFp91IR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "nn.Dropout(p=0.3)  # Increase dropout rate to 30%\n"
      ],
      "metadata": {
        "id": "8l73gobd8kAw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Batch Size\n",
        "Use a smaller batch size to improve gradient estimates, which can lead to better optimization. For example:"
      ],
      "metadata": {
        "id": "2-YEFfg_98q3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "batch_size = 16  # Experiment with batch sizes like 8, 16, or 32\n",
        "train_dataset = torch.utils.data.TensorDataset(X_train, y_train)\n",
        "train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    model_4a.train()\n",
        "    for X_batch, y_batch in train_loader:\n",
        "        X_batch, y_batch = X_batch.to(device), y_batch.to(device)\n",
        "        y_logits = model_4a(X_batch).squeeze()\n",
        "        loss = loss_fn(y_logits, y_batch.long())\n",
        "        optimizer.zero_grad()\n",
        "        loss.backward()\n",
        "        optimizer.step()\n"
      ],
      "metadata": {
        "id": "CGQOXdEe8qhS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Early Stopping\n",
        "Monitor the test accuracy and stop training early if it stops improving:"
      ],
      "metadata": {
        "id": "XuRMcNW8-I3v"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "best_test_acc = 0\n",
        "early_stop_count = 0\n",
        "patience = 10  # Stop if no improvement for 10 epochs\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    # Training and testing as before...\n",
        "    if test_acc > best_test_acc:\n",
        "        best_test_acc = test_acc\n",
        "        early_stop_count = 0\n",
        "    else:\n",
        "        early_stop_count += 1\n",
        "\n",
        "    if early_stop_count >= patience:\n",
        "        print(f\"Early stopping at epoch {epoch}\")\n",
        "        break\n"
      ],
      "metadata": {
        "id": "DFt18Urq8wjJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Optimize Data Preprocessing\n",
        "Ensure data is normalized to have zero mean and unit variance for optimal performance:"
      ],
      "metadata": {
        "id": "r0n7uoMK-Q3k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "scaler = StandardScaler()\n",
        "X_train = torch.tensor(scaler.fit_transform(X_train.cpu())).float().to(device)\n",
        "X_test = torch.tensor(scaler.transform(X_test.cpu())).float().to(device)\n"
      ],
      "metadata": {
        "id": "9BWAUzR481a-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for epoch in range(epochs):\n",
        "    model_4a.train()\n",
        "    y_logits = model_4a(X_train).squeeze()\n",
        "    loss = loss_fn(y_logits, y_train.long())\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    # Scheduler step\n",
        "    scheduler.step()\n"
      ],
      "metadata": {
        "id": "zkbjT3Rm8atx"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "torch.manual_seed(42)\n",
        "torch.cuda.manual_seed(42)\n",
        "\n",
        "# Optimizer and Scheduler\n",
        "optimizer = torch.optim.Adam(model_4a.parameters(), lr=0.01, weight_decay=1e-4)\n",
        "scheduler = StepLR(optimizer, step_size=20, gamma=0.5)\n",
        "\n",
        "# Early Stopping\n",
        "best_test_acc = 0\n",
        "early_stop_count = 0\n",
        "patience = 10\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    model_4a.train()\n",
        "    y_logits = model_4a(X_train).squeeze()\n",
        "    y_pred = torch.softmax(y_logits, dim=1).argmax(dim=1)\n",
        "    loss = loss_fn(y_logits, y_train.long())\n",
        "    acc = accuracy_fn(y_true=y_train, y_pred=y_pred)\n",
        "\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    # Scheduler step\n",
        "    scheduler.step()\n",
        "\n",
        "    # Testing\n",
        "    model_4a.eval()\n",
        "    with torch.inference_mode():\n",
        "        test_logits = model_4a(X_test)\n",
        "        test_pred = torch.softmax(test_logits, dim=1).argmax(dim=1)\n",
        "        test_loss = loss_fn(test_logits, y_test.long())\n",
        "        test_acc = accuracy_fn(y_true=y_test, y_pred=test_pred)\n",
        "\n",
        "    # Early stopping\n",
        "    if test_acc > best_test_acc:\n",
        "        best_test_acc = test_acc\n",
        "        early_stop_count = 0\n",
        "    else:\n",
        "        early_stop_count += 1\n",
        "\n",
        "    if early_stop_count >= patience:\n",
        "        print(f\"Early stopping at epoch {epoch}\")\n",
        "        break\n",
        "\n",
        "    if epoch % 10 == 0:\n",
        "        print(f\"Epoch: {epoch} | Loss: {loss:.4f}, Acc: {acc:.2f}% | Test Loss: {test_loss:.4f}, Test Acc: {test_acc:.2f}%\")\n"
      ],
      "metadata": {
        "id": "obqtdETk2Owe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Meets requirements"
      ],
      "metadata": {
        "id": "8RDo3jXR-buw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Helper functions"
      ],
      "metadata": {
        "id": "qrcZwBFp_yBS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import requests\n",
        "from pathlib import Path\n",
        "\n",
        "# 1. (Optional) Remove the existing (likely invalid) helper_functions.py\n",
        "# !rm helper_functions.py\n",
        "\n",
        "# 2. Use the *raw* GitHub URL\n",
        "url_to_download = \"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/helper_functions.py\"\n",
        "\n",
        "if Path(\"helper_functions.py\").is_file():\n",
        "    print(\"helper_functions.py already exists, skipping download\")\n",
        "else:\n",
        "    print(\"Downloading helper_functions.py\")\n",
        "    request = requests.get(url_to_download)\n",
        "    with open(\"helper_functions.py\", \"wb\") as f:\n",
        "        f.write(request.content)\n"
      ],
      "metadata": {
        "id": "LVXCLLln3_Ee"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from helper_functions import plot_predictions, plot_decision_boundary\n"
      ],
      "metadata": {
        "id": "oKqfwdtu-wWE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "plt.figure(figsize=(12, 6))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.title(\"Train\")\n",
        "plot_decision_boundary(model_4a, X_train, y_train)\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.title(\"Test\")\n",
        "plot_decision_boundary(model_4a, X_test, y_test)\n"
      ],
      "metadata": {
        "id": "osyAPAHI-0qt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## That's as well as I can do here given the synthetic data set- colors are hard to read which is how the problem was written so...."
      ],
      "metadata": {
        "id": "YDM59voM_b0B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torchmetrics import Accuracy\n",
        "\n",
        "# Setup metric\n",
        "torchmetric_accuracy = Accuracy(task=\"multiclass\", num_classes=3).to(device)\n",
        "\n",
        "# Pass raw logits (or predicted probabilities) instead of class indices\n",
        "torchmetric_result = torchmetric_accuracy(test_logits, y_test)\n",
        "print(f\"Accuracy (torchmetrics): {torchmetric_result:.4f}\")\n"
      ],
      "metadata": {
        "id": "wNt-Qvse_Upc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Using sklearn for Full Classification Report\n",
        "If you prefer sklearn, you can use the classification_report function. <b> *However, this requires moving data back to the CPU and converting tensors to NumPy arrays.*</b>"
      ],
      "metadata": {
        "id": "xxwpU0C4Bc2s"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import classification_report\n",
        "\n",
        "# Move tensors to CPU and convert to NumPy\n",
        "y_pred_classes = test_logits.argmax(dim=1).cpu().numpy()\n",
        "y_true = y_test.cpu().numpy()\n",
        "\n",
        "# Generate the classification report\n",
        "print(classification_report(y_true, y_pred_classes, target_names=[\"Class 0\", \"Class 1\", \"Class 2\"]))\n"
      ],
      "metadata": {
        "id": "LIL4LoYsA6gT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Overall, I say this assignment got tricky when the Spiral data was introduced and a 95% accuracy was required, and the assignment called for using a mix of linear and non-linear layers as in\n",
        "\n",
        "* Construct a model capable of fitting the data (you may need a combination of linear and non-linear layers)."
      ],
      "metadata": {
        "id": "7NGL-mBjCMzs"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "upQ-csOvBadX"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}