DDS_cudf_pandas_colab_demo.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/arshad831/Community_Workshops-/blob/main/DDS_cudf_pandas_colab_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kcF9ZWvjSybR"
      },
      "source": [
        "# 10 Minutes to RAPIDS cuDF's pandas accelerator mode (cudf.pandas)\n",
        "\n",
        "cuDF is a Python GPU DataFrame library (built on the Apache Arrow columnar memory format) for loading, joining, aggregating, filtering, and otherwise manipulating tabular data using a DataFrame style API in the style of pandas.\n",
        "\n",
        "cuDF now provides a pandas accelerator mode (`cudf.pandas`), allowing you to bring accelerated computing to your pandas workflows without requiring any code change.\n",
        "\n",
        "This notebook is a short introduction to `cudf.pandas`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SH_h6ci1Sx0u"
      },
      "source": [
        "# ⚠️ Verify your setup\n",
        "\n",
        "First, we'll verify that you are running with an NVIDIA GPU."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y2vPCtXcCvUR",
        "outputId": "aa23331e-ee45-41b2-cf51-0499f12e923b",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon May 20 11:18:32 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   38C    P8               9W /  70W |      3MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi  # this should display information about available GPUs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zhPt4Xj8THgo",
        "tags": []
      },
      "outputs": [],
      "source": [
        "import cudf  # this should work without any errors"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4zGUeWvcTbDs"
      },
      "source": [
        "# Download the data\n",
        "\n",
        "The data we'll be working with is the [Parking Violations Issued - Fiscal Year 2022](https://data.cityofnewyork.us/City-Government/Parking-Violations-Issued-Fiscal-Year-2022/7mxj-7a6y) dataset from NYC Open Data.\n",
        "\n",
        "We're downloading a copy of this dataset from an s3 bucket hosted by NVIDIA to provide faster download speeds. We'll start by downloading the data. This should take about 30 seconds.\n",
        "\n",
        "## Data License and Terms\n",
        "As this dataset originates from the NYC Open Data Portal, it's governed by their license and terms of use.\n",
        "\n",
        "### Are there restrictions on how I can use Open Data?\n",
        "\n",
        "> Open Data belongs to all New Yorkers. There are no restrictions on the use of Open Data. Refer to Terms of Use for more information.\n",
        "\n",
        "### [Terms of Use](https://opendata.cityofnewyork.us/overview/#termsofuse)\n",
        "\n",
        "> By accessing datasets and feeds available through NYC Open Data, the user agrees to all of the Terms of Use of NYC.gov as well as the Privacy Policy for NYC.gov. The user also agrees to any additional terms of use defined by the agencies, bureaus, and offices providing data. Public data sets made available on NYC Open Data are provided for informational purposes. The City does not warranty the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set made available on NYC Open Data, nor are any such warranties to be implied or inferred with respect to the public data sets furnished therein.\n",
        "\n",
        "> The City is not liable for any deficiencies in the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set, or application utilizing such data set, provided by any third party.\n",
        "\n",
        "> Submitting City Agencies are the authoritative source of data available on NYC Open Data. These entities are responsible for data quality and retain version control of data sets and feeds accessed on the Site. Data may be updated, corrected, or refreshed at any time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5EoQqNwsTqeP",
        "outputId": "8e50ff35-fd66-4e20-e1a8-0837b8a00cff",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-05-20 11:19:14--  https://data.rapids.ai/datasets/nyc_parking/nyc_parking_violations_2022.parquet\n",
            "Resolving data.rapids.ai (data.rapids.ai)... 99.86.38.57, 99.86.38.71, 99.86.38.102, ...\n",
            "Connecting to data.rapids.ai (data.rapids.ai)|99.86.38.57|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 474211285 (452M) [binary/octet-stream]\n",
            "Saving to: ‘/tmp/nyc_parking_violations_2022.parquet’\n",
            "\n",
            "/tmp/nyc_parking_vi 100%[===================>] 452.24M  65.3MB/s    in 4.7s    \n",
            "\n",
            "2024-05-20 11:19:20 (95.7 MB/s) - ‘/tmp/nyc_parking_violations_2022.parquet’ saved [474211285/474211285]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "!wget https://data.rapids.ai/datasets/nyc_parking/nyc_parking_violations_2022.parquet -O /tmp/nyc_parking_violations_2022.parquet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hAvNFbYKWwti"
      },
      "source": [
        "# Analysis using Standard Pandas\n",
        "\n",
        "First, let's use Pandas to read in some columns of the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SLRleX9xWxqX",
        "tags": []
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "OLatEi7rW0la",
        "outputId": "705b4360-09f3-4d6c-85af-755a826a6b5f",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "         Registration State           Violation Description Vehicle Body Type  \\\n",
              "15021263                 IN  84-Platform lifts in low posit              TRAC   \n",
              "7767088                  NY                    50-Crosswalk              DELV   \n",
              "7611893                  NY    21-No Parking (street clean)              SUBN   \n",
              "11398718                 FL                            <NA>               SDN   \n",
              "1461478                  NY  PHTO SCHOOL ZN SPEED VIOLATION              4DSD   \n",
              "14205062                 PA  PHTO SCHOOL ZN SPEED VIOLATION               SDN   \n",
              "2444493                  NY                  14-No Standing               VAN   \n",
              "3043615                  NY  38-Failure to Dsplay Meter Rec              SUBN   \n",
              "9709871                  NY  38-Failure to Dsplay Meter Rec              SUBN   \n",
              "6698668                  NY  PHTO SCHOOL ZN SPEED VIOLATION              SUBN   \n",
              "\n",
              "          Issue Date  Summons Number  \n",
              "15021263  06/01/2022      8854577224  \n",
              "7767088   11/26/2021      8953442448  \n",
              "7611893   12/13/2021      8893860016  \n",
              "11398718  04/03/2022      1485959135  \n",
              "1461478   08/05/2021      4743377389  \n",
              "14205062  06/14/2022      4782939220  \n",
              "2444493   08/02/2021      8919289141  \n",
              "3043615   09/18/2021      8686693623  \n",
              "9709871   01/28/2022      8857008885  \n",
              "6698668   11/30/2021      4758683074  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-21f078a9-0514-40f2-8ba5-f4ee4f0f4ffe\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Registration State</th>\n",
              "      <th>Violation Description</th>\n",
              "      <th>Vehicle Body Type</th>\n",
              "      <th>Issue Date</th>\n",
              "      <th>Summons Number</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>15021263</th>\n",
              "      <td>IN</td>\n",
              "      <td>84-Platform lifts in low posit</td>\n",
              "      <td>TRAC</td>\n",
              "      <td>06/01/2022</td>\n",
              "      <td>8854577224</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7767088</th>\n",
              "      <td>NY</td>\n",
              "      <td>50-Crosswalk</td>\n",
              "      <td>DELV</td>\n",
              "      <td>11/26/2021</td>\n",
              "      <td>8953442448</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7611893</th>\n",
              "      <td>NY</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>SUBN</td>\n",
              "      <td>12/13/2021</td>\n",
              "      <td>8893860016</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11398718</th>\n",
              "      <td>FL</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>SDN</td>\n",
              "      <td>04/03/2022</td>\n",
              "      <td>1485959135</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1461478</th>\n",
              "      <td>NY</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>4DSD</td>\n",
              "      <td>08/05/2021</td>\n",
              "      <td>4743377389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14205062</th>\n",
              "      <td>PA</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>SDN</td>\n",
              "      <td>06/14/2022</td>\n",
              "      <td>4782939220</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2444493</th>\n",
              "      <td>NY</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>VAN</td>\n",
              "      <td>08/02/2021</td>\n",
              "      <td>8919289141</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3043615</th>\n",
              "      <td>NY</td>\n",
              "      <td>38-Failure to Dsplay Meter Rec</td>\n",
              "      <td>SUBN</td>\n",
              "      <td>09/18/2021</td>\n",
              "      <td>8686693623</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9709871</th>\n",
              "      <td>NY</td>\n",
              "      <td>38-Failure to Dsplay Meter Rec</td>\n",
              "      <td>SUBN</td>\n",
              "      <td>01/28/2022</td>\n",
              "      <td>8857008885</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6698668</th>\n",
              "      <td>NY</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>SUBN</td>\n",
              "      <td>11/30/2021</td>\n",
              "      <td>4758683074</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-21f078a9-0514-40f2-8ba5-f4ee4f0f4ffe')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-21f078a9-0514-40f2-8ba5-f4ee4f0f4ffe button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-21f078a9-0514-40f2-8ba5-f4ee4f0f4ffe');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-fac1609b-2d46-49cc-8649-078964f88d9d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fac1609b-2d46-49cc-8649-078964f88d9d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-fac1609b-2d46-49cc-8649-078964f88d9d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "# read 5 columns data:\n",
        "df = pd.read_parquet(\n",
        "    \"/tmp/nyc_parking_violations_2022.parquet\",\n",
        "    columns=[\"Registration State\", \"Violation Description\", \"Vehicle Body Type\", \"Issue Date\", \"Summons Number\"]\n",
        ")\n",
        "\n",
        "# view a random sample of 10 rows:\n",
        "df.sample(10)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.info()"
      ],
      "metadata": {
        "id": "hyPbWAiNu3rw",
        "outputId": "7460d6e7-3bf7-4cc6-922e-c5c231d73733",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 15435607 entries, 0 to 15435606\n",
            "Data columns (total 5 columns):\n",
            " #   Column                 Dtype \n",
            "---  ------                 ----- \n",
            " 0   Registration State     object\n",
            " 1   Violation Description  object\n",
            " 2   Vehicle Body Type      object\n",
            " 3   Issue Date             object\n",
            " 4   Summons Number         int64 \n",
            "dtypes: int64(1), object(4)\n",
            "memory usage: 588.8+ MB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7qXNJU9W53D"
      },
      "source": [
        "Next, we'll try to answer a few questions using the data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VmkFv9ZUW37g"
      },
      "source": [
        "## Which parking violation is most commonly committed by vehicles from various U.S states?\n",
        "\n",
        "Each record in our dataset contains the state of registration of the offending vehicle, and the type of parking offence. Let's say we want to get the most common type of offence for vehicles registered in different states. We can do this in Pandas using a combination of [value_counts](https://pandas.pydata.org/docs/reference/api/pandas.Series.value_counts.html) and [GroupBy.head](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.head.html):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "bHXq-s_ZXOQN",
        "outputId": "e5fdd569-572c-47df-8496-ac782b182499"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Registration State           Violation Description  count\n",
              "0                  99                            <NA>  17550\n",
              "1                  AB                  14-No Standing     22\n",
              "2                  AK  PHTO SCHOOL ZN SPEED VIOLATION    125\n",
              "3                  AL  PHTO SCHOOL ZN SPEED VIOLATION   3668\n",
              "4                  AR  PHTO SCHOOL ZN SPEED VIOLATION    537\n",
              "..                ...                             ...    ...\n",
              "62                 VT  PHTO SCHOOL ZN SPEED VIOLATION   3024\n",
              "63                 WA    21-No Parking (street clean)   3732\n",
              "64                 WI                  14-No Standing   1639\n",
              "65                 WV  PHTO SCHOOL ZN SPEED VIOLATION   1185\n",
              "66                 WY    21-No Parking (street clean)    138\n",
              "\n",
              "[67 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8a588be8-0870-4914-be7a-4c66b70b05d5\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Registration State</th>\n",
              "      <th>Violation Description</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>99</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>17550</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>AB</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>22</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>AK</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>AL</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3668</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>AR</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>537</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>62</th>\n",
              "      <td>VT</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3024</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>63</th>\n",
              "      <td>WA</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>3732</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>64</th>\n",
              "      <td>WI</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>1639</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>65</th>\n",
              "      <td>WV</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>1185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>66</th>\n",
              "      <td>WY</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>138</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>67 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8a588be8-0870-4914-be7a-4c66b70b05d5')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8a588be8-0870-4914-be7a-4c66b70b05d5 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8a588be8-0870-4914-be7a-4c66b70b05d5');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-d7107137-bb67-456d-8204-8b8cbc1df0d0\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d7107137-bb67-456d-8204-8b8cbc1df0d0')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-d7107137-bb67-456d-8204-8b8cbc1df0d0 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \")\",\n  \"rows\": 67,\n  \"fields\": [\n    {\n      \"column\": \"Registration State\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 67,\n        \"samples\": [\n          \"NC\",\n          \"GV\",\n          \"AR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Violation Description\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"14-No Standing\",\n          \"21-No Parking (street clean)\",\n          \"<NA>\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 458031,\n        \"min\": 1,\n        \"max\": 3743378,\n        \"num_unique_values\": 67,\n        \"samples\": [\n          31102,\n          2451,\n          537\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "(df[[\"Registration State\", \"Violation Description\"]]  # get only these two columns\n",
        " .value_counts()  # get the count of offences per state and per type of offence\n",
        " .groupby(\"Registration State\")  # group by state\n",
        " .head(1)  # get the first row in each group (the type of offence with the largest count)\n",
        " .sort_index()  # sort by state name\n",
        " .reset_index()\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8lXF4v4SXRf3"
      },
      "source": [
        "The code above uses [method chaining](https://tomaugspurger.net/posts/method-chaining/) to combine a series of operations into a single statement. You might find it useful to break the code up into multiple statements and inspect each of the intermediate results!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H7_9EmGyXUJd"
      },
      "source": [
        "## Which vehicle body types are most frequently involved in parking violations?\n",
        "\n",
        "We can also investigate which vehicle body types most commonly appear in parking violations"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 455
        },
        "id": "d7Ax-u4TXZtp",
        "outputId": "e86ef19e-3b5c-4934-86c6-5715bdf78a8e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                     Count\n",
              "Vehicle Body Type         \n",
              "SUBN               6449007\n",
              "4DSD               4402991\n",
              "VAN                1317899\n",
              "DELV                436430\n",
              "PICK                429798\n",
              "...                    ...\n",
              "CARY                     1\n",
              "ISUZ                     1\n",
              "IXMR                     1\n",
              "BILB                     1\n",
              "ONEL                     1\n",
              "\n",
              "[967 rows x 1 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-488d7fad-abec-4ceb-ad5e-c9c2caed6b9d\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Count</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Vehicle Body Type</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SUBN</th>\n",
              "      <td>6449007</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4DSD</th>\n",
              "      <td>4402991</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>VAN</th>\n",
              "      <td>1317899</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>DELV</th>\n",
              "      <td>436430</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PICK</th>\n",
              "      <td>429798</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>CARY</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ISUZ</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>IXMR</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>BILB</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ONEL</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>967 rows × 1 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-488d7fad-abec-4ceb-ad5e-c9c2caed6b9d')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-488d7fad-abec-4ceb-ad5e-c9c2caed6b9d button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-488d7fad-abec-4ceb-ad5e-c9c2caed6b9d');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b145478d-eafe-4de6-8490-966da20058f3\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b145478d-eafe-4de6-8490-966da20058f3')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b145478d-eafe-4de6-8490-966da20058f3 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \")\",\n  \"rows\": 967,\n  \"fields\": [\n    {\n      \"column\": \"Vehicle Body Type\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 967,\n        \"samples\": [\n          \"FPM\",\n          \"ACCO\",\n          \"OEN\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 255877,\n        \"min\": 1,\n        \"max\": 6449007,\n        \"num_unique_values\": 187,\n        \"samples\": [\n          2,\n          757,\n          1900\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "(df\n",
        " .groupby([\"Vehicle Body Type\"])\n",
        " .agg({\"Summons Number\": \"count\"})\n",
        " .rename(columns={\"Summons Number\": \"Count\"})\n",
        " .sort_values([\"Count\"], ascending=False)\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VjFfQLZHXehM"
      },
      "source": [
        "## How do parking violations vary across days of the week?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s5_y9m_AXhIw",
        "outputId": "616c8abf-c8af-47bb-9cb7-d675df260a63"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "issue_weekday\n",
              "Sunday        462992\n",
              "Saturday     1108385\n",
              "Monday       2488563\n",
              "Wednesday    2760088\n",
              "Tuesday      2809949\n",
              "Friday       2891679\n",
              "Thursday     2913951\n",
              "Name: Summons Number, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "weekday_names = {\n",
        "    0: \"Monday\",\n",
        "    1: \"Tuesday\",\n",
        "    2: \"Wednesday\",\n",
        "    3: \"Thursday\",\n",
        "    4: \"Friday\",\n",
        "    5: \"Saturday\",\n",
        "    6: \"Sunday\",\n",
        "}\n",
        "\n",
        "df[\"Issue Date\"] = df[\"Issue Date\"].astype(\"datetime64[ms]\")\n",
        "df[\"issue_weekday\"] = df[\"Issue Date\"].dt.weekday.map(weekday_names)\n",
        "\n",
        "df.groupby([\"issue_weekday\"])[\"Summons Number\"].count().sort_values()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LDeYr6xkXiDc"
      },
      "source": [
        "It looks like there are fewer violations on weekends, which makes sense! During the week, more people are driving in New York City."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JKBQcT64XlMr"
      },
      "source": [
        "## Let's time it!\n",
        "\n",
        "Loading and processing this data took a little time. Let's measure how long these pipelines take in Pandas:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 460
        },
        "id": "mDpQhus-Xnfs",
        "outputId": "c7ca9ade-3477-40f7-9fda-aa9541cf4a59"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 6.44 s, sys: 2.11 s, total: 8.56 s\n",
            "Wall time: 7.63 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Registration State           Violation Description  count\n",
              "0                  99                            <NA>  17550\n",
              "1                  AB                  14-No Standing     22\n",
              "2                  AK  PHTO SCHOOL ZN SPEED VIOLATION    125\n",
              "3                  AL  PHTO SCHOOL ZN SPEED VIOLATION   3668\n",
              "4                  AR  PHTO SCHOOL ZN SPEED VIOLATION    537\n",
              "..                ...                             ...    ...\n",
              "62                 VT  PHTO SCHOOL ZN SPEED VIOLATION   3024\n",
              "63                 WA    21-No Parking (street clean)   3732\n",
              "64                 WI                  14-No Standing   1639\n",
              "65                 WV  PHTO SCHOOL ZN SPEED VIOLATION   1185\n",
              "66                 WY    21-No Parking (street clean)    138\n",
              "\n",
              "[67 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7ec89fd0-1a39-4442-8571-5fa34a3576a9\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Registration State</th>\n",
              "      <th>Violation Description</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>99</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>17550</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>AB</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>22</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>AK</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>AL</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3668</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>AR</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>537</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>62</th>\n",
              "      <td>VT</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3024</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>63</th>\n",
              "      <td>WA</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>3732</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>64</th>\n",
              "      <td>WI</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>1639</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>65</th>\n",
              "      <td>WV</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>1185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>66</th>\n",
              "      <td>WY</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>138</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>67 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7ec89fd0-1a39-4442-8571-5fa34a3576a9')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-7ec89fd0-1a39-4442-8571-5fa34a3576a9 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-7ec89fd0-1a39-4442-8571-5fa34a3576a9');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-32f23b45-f56a-4a86-a964-d12aa2f05e81\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-32f23b45-f56a-4a86-a964-d12aa2f05e81')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-32f23b45-f56a-4a86-a964-d12aa2f05e81 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"get_ipython()\",\n  \"rows\": 67,\n  \"fields\": [\n    {\n      \"column\": \"Registration State\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 67,\n        \"samples\": [\n          \"NC\",\n          \"GV\",\n          \"AR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Violation Description\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"14-No Standing\",\n          \"21-No Parking (street clean)\",\n          \"<NA>\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 458031,\n        \"min\": 1,\n        \"max\": 3743378,\n        \"num_unique_values\": 67,\n        \"samples\": [\n          31102,\n          2451,\n          537\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "df = pd.read_parquet(\n",
        "    \"/tmp/nyc_parking_violations_2022.parquet\",\n",
        "    columns=[\"Registration State\", \"Violation Description\", \"Vehicle Body Type\", \"Issue Date\", \"Summons Number\"]\n",
        ")\n",
        "\n",
        "(df[[\"Registration State\", \"Violation Description\"]]\n",
        " .value_counts()\n",
        " .groupby(\"Registration State\")\n",
        " .head(1)\n",
        " .sort_index()\n",
        " .reset_index()\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 491
        },
        "id": "9Gw5TWH2Xqgv",
        "outputId": "30bb3e94-31df-442f-dcf9-033ab70c53d0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 731 ms, sys: 213 ms, total: 944 ms\n",
            "Wall time: 940 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                     Count\n",
              "Vehicle Body Type         \n",
              "SUBN               6449007\n",
              "4DSD               4402991\n",
              "VAN                1317899\n",
              "DELV                436430\n",
              "PICK                429798\n",
              "...                    ...\n",
              "CARY                     1\n",
              "ISUZ                     1\n",
              "IXMR                     1\n",
              "BILB                     1\n",
              "ONEL                     1\n",
              "\n",
              "[967 rows x 1 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-2a54d9d4-b123-45db-a12e-9e9189bb4dc3\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Count</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Vehicle Body Type</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SUBN</th>\n",
              "      <td>6449007</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4DSD</th>\n",
              "      <td>4402991</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>VAN</th>\n",
              "      <td>1317899</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>DELV</th>\n",
              "      <td>436430</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PICK</th>\n",
              "      <td>429798</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>CARY</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ISUZ</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>IXMR</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>BILB</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ONEL</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>967 rows × 1 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2a54d9d4-b123-45db-a12e-9e9189bb4dc3')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-2a54d9d4-b123-45db-a12e-9e9189bb4dc3 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-2a54d9d4-b123-45db-a12e-9e9189bb4dc3');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-263725af-9c2c-4487-9008-9183180fe820\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-263725af-9c2c-4487-9008-9183180fe820')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-263725af-9c2c-4487-9008-9183180fe820 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"get_ipython()\",\n  \"rows\": 967,\n  \"fields\": [\n    {\n      \"column\": \"Vehicle Body Type\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 967,\n        \"samples\": [\n          \"FPM\",\n          \"ACCO\",\n          \"OEN\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 255877,\n        \"min\": 1,\n        \"max\": 6449007,\n        \"num_unique_values\": 187,\n        \"samples\": [\n          2,\n          757,\n          1900\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "(df\n",
        " .groupby([\"Vehicle Body Type\"])\n",
        " .agg({\"Summons Number\": \"count\"})\n",
        " .rename(columns={\"Summons Number\": \"Count\"})\n",
        " .sort_values([\"Count\"], ascending=False)\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BovQgNrpXr2l",
        "outputId": "35321a0a-6ff6-4a1b-f254-8045993dd406"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 3.48 s, sys: 552 ms, total: 4.03 s\n",
            "Wall time: 4.02 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "issue_weekday\n",
              "Sunday        462992\n",
              "Saturday     1108385\n",
              "Monday       2488563\n",
              "Wednesday    2760088\n",
              "Tuesday      2809949\n",
              "Friday       2891679\n",
              "Thursday     2913951\n",
              "Name: Summons Number, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "weekday_names = {\n",
        "    0: \"Monday\",\n",
        "    1: \"Tuesday\",\n",
        "    2: \"Wednesday\",\n",
        "    3: \"Thursday\",\n",
        "    4: \"Friday\",\n",
        "    5: \"Saturday\",\n",
        "    6: \"Sunday\",\n",
        "}\n",
        "\n",
        "df[\"Issue Date\"] = df[\"Issue Date\"].astype(\"datetime64[ms]\")\n",
        "df[\"issue_weekday\"] = df[\"Issue Date\"].dt.weekday.map(weekday_names)\n",
        "\n",
        "df.groupby([\"issue_weekday\"])[\"Summons Number\"].count().sort_values()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VgAWS0yXXtGj"
      },
      "source": [
        "# Using cudf.pandas\n",
        "\n",
        "Now, let's re-run the Pandas code above with the `cudf.pandas` extension loaded.\n",
        "\n",
        "Typically, you should load the `cudf.pandas` extension as the first step in your notebook, before importing any modules. Here, we explicitly restart the kernel to simulate that behavior."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hW5rUr2tXzUW",
        "outputId": "0c67d671-edde-405d-9ff1-180e7f2f0619"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'status': 'ok', 'restart': True}"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "get_ipython().kernel.do_shutdown(restart=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NjvPsTlGZrW7"
      },
      "outputs": [],
      "source": [
        "%load_ext cudf.pandas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XL_u4l5gZJte",
        "outputId": "14f4d4b1-8b1a-44ad-92e8-5f8e151ebf8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 460
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 655 ms, sys: 319 ms, total: 974 ms\n",
            "Wall time: 1.33 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Registration State           Violation Description  count\n",
              "0                  99                            <NA>  17550\n",
              "1                  AB                  14-No Standing     22\n",
              "2                  AK  PHTO SCHOOL ZN SPEED VIOLATION    125\n",
              "3                  AL  PHTO SCHOOL ZN SPEED VIOLATION   3668\n",
              "4                  AR  PHTO SCHOOL ZN SPEED VIOLATION    537\n",
              "..                ...                             ...    ...\n",
              "62                 VT  PHTO SCHOOL ZN SPEED VIOLATION   3024\n",
              "63                 WA    21-No Parking (street clean)   3732\n",
              "64                 WI                  14-No Standing   1639\n",
              "65                 WV  PHTO SCHOOL ZN SPEED VIOLATION   1185\n",
              "66                 WY    21-No Parking (street clean)    138\n",
              "\n",
              "[67 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-439097b2-0649-48f0-8dc5-03d8dd20b399\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Registration State</th>\n",
              "      <th>Violation Description</th>\n",
              "      <th>count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>99</td>\n",
              "      <td>&lt;NA&gt;</td>\n",
              "      <td>17550</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>AB</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>22</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>AK</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>AL</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3668</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>AR</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>537</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>62</th>\n",
              "      <td>VT</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>3024</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>63</th>\n",
              "      <td>WA</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>3732</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>64</th>\n",
              "      <td>WI</td>\n",
              "      <td>14-No Standing</td>\n",
              "      <td>1639</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>65</th>\n",
              "      <td>WV</td>\n",
              "      <td>PHTO SCHOOL ZN SPEED VIOLATION</td>\n",
              "      <td>1185</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>66</th>\n",
              "      <td>WY</td>\n",
              "      <td>21-No Parking (street clean)</td>\n",
              "      <td>138</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>67 rows × 3 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-439097b2-0649-48f0-8dc5-03d8dd20b399')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-439097b2-0649-48f0-8dc5-03d8dd20b399 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-439097b2-0649-48f0-8dc5-03d8dd20b399');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-0f95ce69-6d52-4e97-9448-4209229316b1\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0f95ce69-6d52-4e97-9448-4209229316b1')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-0f95ce69-6d52-4e97-9448-4209229316b1 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"get_ipython()\",\n  \"rows\": 67,\n  \"fields\": [\n    {\n      \"column\": \"Registration State\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 67,\n        \"samples\": [\n          \"PA\",\n          \"BC\",\n          \"NV\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Violation Description\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"40-Fire Hydrant\",\n          \"Detached Trailer\",\n          \"<NA>\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 458031,\n        \"min\": 1,\n        \"max\": 3743378,\n        \"num_unique_values\": 67,\n        \"samples\": [\n          149773,\n          13,\n          747\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "df = pd.read_parquet(\n",
        "    \"/tmp/nyc_parking_violations_2022.parquet\",\n",
        "    columns=[\"Registration State\", \"Violation Description\", \"Vehicle Body Type\", \"Issue Date\", \"Summons Number\"]\n",
        ")\n",
        "\n",
        "(df[[\"Registration State\", \"Violation Description\"]]\n",
        " .value_counts()\n",
        " .groupby(\"Registration State\")\n",
        " .head(1)\n",
        " .sort_index()\n",
        " .reset_index()\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BLWa8ed6d-pD",
        "outputId": "6ea923d9-f23e-4548-c438-3d6fa2ef7053",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 491
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 23.8 ms, sys: 7.4 ms, total: 31.2 ms\n",
            "Wall time: 58.5 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                     Count\n",
              "Vehicle Body Type         \n",
              "SUBN               6449007\n",
              "4DSD               4402991\n",
              "VAN                1317899\n",
              "DELV                436430\n",
              "PICK                429798\n",
              "...                    ...\n",
              "YANT                     1\n",
              "YBSD                     1\n",
              "YEL                      1\n",
              "YL                       1\n",
              "Z1UT                     1\n",
              "\n",
              "[967 rows x 1 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0101ae2c-f570-466c-9fb1-329ce2706fd6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Count</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Vehicle Body Type</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SUBN</th>\n",
              "      <td>6449007</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4DSD</th>\n",
              "      <td>4402991</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>VAN</th>\n",
              "      <td>1317899</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>DELV</th>\n",
              "      <td>436430</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PICK</th>\n",
              "      <td>429798</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>YANT</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>YBSD</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>YEL</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>YL</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Z1UT</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>967 rows × 1 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0101ae2c-f570-466c-9fb1-329ce2706fd6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0101ae2c-f570-466c-9fb1-329ce2706fd6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0101ae2c-f570-466c-9fb1-329ce2706fd6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-d1791cd7-5703-4910-9f09-18773d431da8\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d1791cd7-5703-4910-9f09-18773d431da8')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-d1791cd7-5703-4910-9f09-18773d431da8 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"get_ipython()\",\n  \"rows\": 967,\n  \"fields\": [\n    {\n      \"column\": \"Vehicle Body Type\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 967,\n        \"samples\": [\n          \"TRAI\",\n          \"SEAT\",\n          \"EBRP\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Count\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 255877,\n        \"min\": 1,\n        \"max\": 6449007,\n        \"num_unique_values\": 187,\n        \"samples\": [\n          701,\n          2217,\n          322814\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "(df\n",
        " .groupby([\"Vehicle Body Type\"])\n",
        " .agg({\"Summons Number\": \"count\"})\n",
        " .rename(columns={\"Summons Number\": \"Count\"})\n",
        " .sort_values([\"Count\"], ascending=False)\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X6ASy4mPd_-c",
        "outputId": "5af97128-36af-44f8-b2ad-c3dd736b6b8c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 249 ms, sys: 57.8 ms, total: 307 ms\n",
            "Wall time: 385 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "issue_weekday\n",
              "Sunday        462992\n",
              "Saturday     1108385\n",
              "Monday       2488563\n",
              "Wednesday    2760088\n",
              "Tuesday      2809949\n",
              "Friday       2891679\n",
              "Thursday     2913951\n",
              "Name: Summons Number, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "weekday_names = {\n",
        "    0: \"Monday\",\n",
        "    1: \"Tuesday\",\n",
        "    2: \"Wednesday\",\n",
        "    3: \"Thursday\",\n",
        "    4: \"Friday\",\n",
        "    5: \"Saturday\",\n",
        "    6: \"Sunday\",\n",
        "}\n",
        "\n",
        "df[\"Issue Date\"] = df[\"Issue Date\"].astype(\"datetime64[ms]\")\n",
        "df[\"issue_weekday\"] = df[\"Issue Date\"].dt.weekday.map(weekday_names)\n",
        "\n",
        "df.groupby([\"issue_weekday\"])[\"Summons Number\"].count().sort_values()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FMUrf6iMeBdM"
      },
      "source": [
        "Much faster! Operations that took 5-20 seconds can now potentially finish in just milliseconds without changing any code."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "00m6gUxqeGzk"
      },
      "source": [
        "# Understanding Performance\n",
        "\n",
        "`cudf.pandas` provides profiling utilities to help you better understand performance. With these tools, you can identify which parts of your code ran on the GPU and which parts ran on the CPU.\n",
        "\n",
        "They're accessible in the `cudf.pandas` namespace since the `cudf.pandas` extension was loaded above with `load_ext cudf.pandas`.\n",
        "\n",
        "#### Colab Note\n",
        "If you're running in Colab, the first time you run use the profiler it may take 10+ seconds due to Colab's debugger interacting with the built-in Python function [sys.settrace](https://docs.python.org/3/library/sys.html#sys.settrace) that we use for profiling. For demo purposes, this isn't an issue. Just run the cell again.\n",
        "\n",
        "## Profiling Functionality\n",
        "\n",
        "We can generate a per-function profile:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fcHVUlJoJXkK",
        "outputId": "bf111b0f-4f6f-4f94-a611-28ce84da5c91",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "15435607"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "len(df)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RFm22OWbeHF2",
        "outputId": "0fb368be-3b79-4e4b-bb12-f6b15985c826",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 672
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "PYDEV DEBUGGER WARNING:\n",
            "sys.settrace() should not be used when the debugger is being used.\n",
            "This may cause the debugger to stop working correctly.\n",
            "If this is needed, please check: \n",
            "http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html\n",
            "to see how to restore the debug tracing back correctly.\n",
            "Call Location:\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/cudf/pandas/profiler.py\", line 97, in __enter__\n",
            "    sys.settrace(self._tracefunc)\n",
            "\n",
            "\n",
            "PYDEV DEBUGGER WARNING:\n",
            "sys.settrace() should not be used when the debugger is being used.\n",
            "This may cause the debugger to stop working correctly.\n",
            "If this is needed, please check: \n",
            "http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html\n",
            "to see how to restore the debug tracing back correctly.\n",
            "Call Location:\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/cudf/pandas/profiler.py\", line 116, in __exit__\n",
            "    sys.settrace(self._oldtrace)\n",
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[3m                                                                                                         \u001b[0m\n",
              "\u001b[3m                                        Total time elapsed: 1.143 seconds                                \u001b[0m\n",
              "\u001b[3m                                      6 GPU function calls in 0.049 seconds                              \u001b[0m\n",
              "\u001b[3m                                      0 CPU function calls in 0.000 seconds                              \u001b[0m\n",
              "\u001b[3m                                                                                                         \u001b[0m\n",
              "\u001b[3m                                                      Stats                                              \u001b[0m\n",
              "\u001b[3m                                                                                                         \u001b[0m\n",
              "┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n",
              "┃\u001b[1m \u001b[0m\u001b[1mFunction           \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mGPU ncalls\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mGPU cumtime\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mGPU percall\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCPU ncalls\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCPU cumtime\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCPU percall\u001b[0m\u001b[1m \u001b[0m┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n",
              "│ DataFrame           │ 1          │ 0.006       │ 0.006       │ 0          │ 0.000       │ 0.000       │\n",
              "│ concat              │ 1          │ 0.013       │ 0.013       │ 0          │ 0.000       │ 0.000       │\n",
              "│ DataFrame.min       │ 2          │ 0.014       │ 0.007       │ 0          │ 0.000       │ 0.000       │\n",
              "│ DataFrame.groupby   │ 1          │ 0.001       │ 0.001       │ 0          │ 0.000       │ 0.000       │\n",
              "│ SeriesGroupBy.count │ 1          │ 0.015       │ 0.015       │ 0          │ 0.000       │ 0.000       │\n",
              "└─────────────────────┴────────────┴─────────────┴─────────────┴────────────┴─────────────┴─────────────┘\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">                                                                                                         </span>\n",
              "<span style=\"font-style: italic\">                                        Total time elapsed: 1.143 seconds                                </span>\n",
              "<span style=\"font-style: italic\">                                      6 GPU function calls in 0.049 seconds                              </span>\n",
              "<span style=\"font-style: italic\">                                      0 CPU function calls in 0.000 seconds                              </span>\n",
              "<span style=\"font-style: italic\">                                                                                                         </span>\n",
              "<span style=\"font-style: italic\">                                                      Stats                                              </span>\n",
              "<span style=\"font-style: italic\">                                                                                                         </span>\n",
              "┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n",
              "┃<span style=\"font-weight: bold\"> Function            </span>┃<span style=\"font-weight: bold\"> GPU ncalls </span>┃<span style=\"font-weight: bold\"> GPU cumtime </span>┃<span style=\"font-weight: bold\"> GPU percall </span>┃<span style=\"font-weight: bold\"> CPU ncalls </span>┃<span style=\"font-weight: bold\"> CPU cumtime </span>┃<span style=\"font-weight: bold\"> CPU percall </span>┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n",
              "│ DataFrame           │ 1          │ 0.006       │ 0.006       │ 0          │ 0.000       │ 0.000       │\n",
              "│ concat              │ 1          │ 0.013       │ 0.013       │ 0          │ 0.000       │ 0.000       │\n",
              "│ DataFrame.min       │ 2          │ 0.014       │ 0.007       │ 0          │ 0.000       │ 0.000       │\n",
              "│ DataFrame.groupby   │ 1          │ 0.001       │ 0.001       │ 0          │ 0.000       │ 0.000       │\n",
              "│ SeriesGroupBy.count │ 1          │ 0.015       │ 0.015       │ 0          │ 0.000       │ 0.000       │\n",
              "└─────────────────────┴────────────┴─────────────┴─────────────┴────────────┴─────────────┴─────────────┘\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "%%cudf.pandas.profile\n",
        "\n",
        "small_df = pd.DataFrame({'a': [\"0\", \"1\", \"2\"], 'b': [\"x\", \"y\", \"z\"]})\n",
        "small_df = pd.concat([small_df, small_df])\n",
        "\n",
        "axis = 0\n",
        "for i in range(0, 2):\n",
        "    small_df.min(axis=axis)\n",
        "    axis = i\n",
        "\n",
        "counts = small_df.groupby(\"a\").b.count()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 822
        },
        "id": "Syb-_vZweN2H",
        "outputId": "bd05e9aa-7e55-4a32-a813-e53e8d2ade27"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "PYDEV DEBUGGER WARNING:\n",
            "sys.settrace() should not be used when the debugger is being used.\n",
            "This may cause the debugger to stop working correctly.\n",
            "If this is needed, please check: \n",
            "http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html\n",
            "to see how to restore the debug tracing back correctly.\n",
            "Call Location:\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/cudf/pandas/profiler.py\", line 97, in __enter__\n",
            "    sys.settrace(self._tracefunc)\n",
            "\n",
            "\n",
            "PYDEV DEBUGGER WARNING:\n",
            "sys.settrace() should not be used when the debugger is being used.\n",
            "This may cause the debugger to stop working correctly.\n",
            "If this is needed, please check: \n",
            "http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html\n",
            "to see how to restore the debug tracing back correctly.\n",
            "Call Location:\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/cudf/pandas/profiler.py\", line 116, in __exit__\n",
            "    sys.settrace(self._oldtrace)\n",
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[3m                                                                                                                   \u001b[0m\n",
              "\u001b[3m                                             Total time elapsed: 2.646 seconds                                     \u001b[0m\n",
              "\u001b[3m                                                                                                                   \u001b[0m\n",
              "\u001b[3m                                                           Stats                                                   \u001b[0m\n",
              "\u001b[3m                                                                                                                   \u001b[0m\n",
              "┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n",
              "┃\u001b[1m \u001b[0m\u001b[1mLine no.\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mLine                                                                    \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mGPU TIME(s)\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCPU TIME(s)\u001b[0m\u001b[1m \u001b[0m┃\n",
              "┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n",
              "│ 2        │ \u001b[38;2;248;248;242;48;2;39;40;34m    \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mpd\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mDataFrame\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m{\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m'\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34ma\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m'\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m0\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m1\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m2\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m'\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mb\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m'\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mx\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34my\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mz\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m…\u001b[0m │ 0.004195585 │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 3        │ \u001b[38;2;248;248;242;48;2;39;40;34m    \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mpd\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mconcat\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m                          \u001b[0m │ 0.008323235 │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 5        │ \u001b[38;2;248;248;242;48;2;39;40;34m    \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34maxis\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[48;2;39;40;34m                                                            \u001b[0m │             │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 6        │ \u001b[38;2;248;248;242;48;2;39;40;34m    \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mfor\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34min\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mrange\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m2\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m                                               \u001b[0m │             │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 7        │ \u001b[38;2;248;248;242;48;2;39;40;34m        \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmin\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34maxis\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34maxis\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m                                         \u001b[0m │ 0.007513770 │ 0.033017991 │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 8        │ \u001b[38;2;248;248;242;48;2;39;40;34m        \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34maxis\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m                                                        \u001b[0m │             │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "│ 10       │ \u001b[38;2;248;248;242;48;2;39;40;34m    \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mcounts\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34msmall_df\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mgroupby\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34ma\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mb\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m.\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mcount\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m                            \u001b[0m │ 0.019637824 │             │\n",
              "│          │ \u001b[48;2;39;40;34m                                                                        \u001b[0m │             │             │\n",
              "└──────────┴──────────────────────────────────────────────────────────────────────────┴─────────────┴─────────────┘\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">                                                                                                                   </span>\n",
              "<span style=\"font-style: italic\">                                             Total time elapsed: 2.646 seconds                                     </span>\n",
              "<span style=\"font-style: italic\">                                                                                                                   </span>\n",
              "<span style=\"font-style: italic\">                                                           Stats                                                   </span>\n",
              "<span style=\"font-style: italic\">                                                                                                                   </span>\n",
              "┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n",
              "┃<span style=\"font-weight: bold\"> Line no. </span>┃<span style=\"font-weight: bold\"> Line                                                                     </span>┃<span style=\"font-weight: bold\"> GPU TIME(s) </span>┃<span style=\"font-weight: bold\"> CPU TIME(s) </span>┃\n",
              "┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n",
              "│ 2        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">    small_df </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> pd</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">DataFrame({</span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">'a'</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">: [</span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"0\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">, </span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"1\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">, </span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"2\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">], </span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">'b'</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">: [</span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"x\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">, </span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"y\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">, </span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"z\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">]…</span> │ 0.004195585 │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 3        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">    small_df </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> pd</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">concat([small_df, small_df])</span><span style=\"background-color: #272822\">                          </span> │ 0.008323235 │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 5        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">    axis </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> </span><span style=\"color: #ae81ff; text-decoration-color: #ae81ff; background-color: #272822\">0</span><span style=\"background-color: #272822\">                                                            </span> │             │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 6        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">    </span><span style=\"color: #66d9ef; text-decoration-color: #66d9ef; background-color: #272822\">for</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> i </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">in</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> range(</span><span style=\"color: #ae81ff; text-decoration-color: #ae81ff; background-color: #272822\">0</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">, </span><span style=\"color: #ae81ff; text-decoration-color: #ae81ff; background-color: #272822\">2</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">):</span><span style=\"background-color: #272822\">                                               </span> │             │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 7        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">        small_df</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">min(axis</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">axis)</span><span style=\"background-color: #272822\">                                         </span> │ 0.007513770 │ 0.033017991 │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 8        │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">        axis </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> </span><span style=\"color: #ae81ff; text-decoration-color: #ae81ff; background-color: #272822\">1</span><span style=\"background-color: #272822\">                                                        </span> │             │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "│ 10       │ <span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">    counts </span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">=</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\"> small_df</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">groupby(</span><span style=\"color: #e6db74; text-decoration-color: #e6db74; background-color: #272822\">\"a\"</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">)</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">b</span><span style=\"color: #ff4689; text-decoration-color: #ff4689; background-color: #272822\">.</span><span style=\"color: #f8f8f2; text-decoration-color: #f8f8f2; background-color: #272822\">count()</span><span style=\"background-color: #272822\">                            </span> │ 0.019637824 │             │\n",
              "│          │ <span style=\"background-color: #272822\">                                                                        </span> │             │             │\n",
              "└──────────┴──────────────────────────────────────────────────────────────────────────┴─────────────┴─────────────┘\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "%%cudf.pandas.line_profile\n",
        "\n",
        "small_df = pd.DataFrame({'a': [\"0\", \"1\", \"2\"], 'b': [\"x\", \"y\", \"z\"]})\n",
        "small_df = pd.concat([small_df, small_df])\n",
        "\n",
        "axis = 0\n",
        "for i in range(0, 2):\n",
        "    small_df.min(axis=axis)\n",
        "    axis = 1\n",
        "\n",
        "counts = small_df.groupby(\"a\").b.count()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCZ6BxwBpfjL"
      },
      "source": [
        "## Behind the scenes: What's going on here?\n",
        "\n",
        "When you load `cudf.pandas`, Pandas types like `Series` and `DataFrame` are replaced by proxy objects that dispatch operations to cuDF when possible. We can verify that `cudf.pandas` is active by looking at our `pd` variable:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jogk5UrgeTkS",
        "outputId": "83079e87-3988-4df6-90eb-68acd00ff038"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<module 'pandas' (ModuleAccelerator(fast=cudf, slow=pandas))>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "pd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vxh70rpDph3I"
      },
      "source": [
        "As a result, all pandas functions, methods, and created objects are proxies:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "id": "RYTCGl7spgjs",
        "outputId": "c1d915a3-29c2-46ec-db62-00dad9b5a89f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "cudf.pandas.fast_slow_proxy._FunctionProxy"
            ],
            "text/html": [
              "<div style=\"max-width:800px; border: 1px solid var(--colab-border-color);\"><style>\n",
              "      pre.function-repr-contents {\n",
              "        overflow-x: auto;\n",
              "        padding: 8px 12px;\n",
              "        max-height: 500px;\n",
              "      }\n",
              "\n",
              "      pre.function-repr-contents.function-repr-contents-collapsed {\n",
              "        cursor: pointer;\n",
              "        max-height: 100px;\n",
              "      }\n",
              "    </style>\n",
              "    <pre style=\"white-space: initial; background:\n",
              "         var(--colab-secondary-surface-color); padding: 8px 12px;\n",
              "         border-bottom: 1px solid var(--colab-border-color);\"><b>cudf.pandas.fast_slow_proxy._FunctionProxy</b><br/>def __call__(*args, **kwargs) -&gt; Any</pre><pre class=\"function-repr-contents function-repr-contents-collapsed\" style=\"\"><a class=\"filepath\" style=\"display:none\" href=\"#\">/usr/local/lib/python3.10/dist-packages/cudf/pandas/fast_slow_proxy.py</a>Proxy for a pair of fast and slow functions.</pre>\n",
              "      <script>\n",
              "      if (google.colab.kernel.accessAllowed && google.colab.files && google.colab.files.view) {\n",
              "        for (const element of document.querySelectorAll('.filepath')) {\n",
              "          element.style.display = 'block'\n",
              "          element.onclick = (event) => {\n",
              "            event.preventDefault();\n",
              "            event.stopPropagation();\n",
              "            google.colab.files.view(element.textContent, 851);\n",
              "          };\n",
              "        }\n",
              "      }\n",
              "      for (const element of document.querySelectorAll('.function-repr-contents')) {\n",
              "        element.onclick = (event) => {\n",
              "          event.preventDefault();\n",
              "          event.stopPropagation();\n",
              "          element.classList.toggle('function-repr-contents-collapsed');\n",
              "        };\n",
              "      }\n",
              "      </script>\n",
              "      </div>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "type(pd.read_csv)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-NvKu7XplmO"
      },
      "source": [
        "Operations supported by cuDF will be **very** fast:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MFvLJo4upnUG",
        "outputId": "f40387af-0e54-43a2-95d5-d0f403791e08"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 1.93 ms, sys: 0 ns, total: 1.93 ms\n",
            "Wall time: 1.94 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Registration State       15435607\n",
              "Violation Description    15117819\n",
              "Vehicle Body Type        15402365\n",
              "Issue Date               15435607\n",
              "Summons Number           15435607\n",
              "issue_weekday            15435607\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "%%time\n",
        "df.count(axis=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Np6VP-wSpomO"
      },
      "source": [
        "Operations not supported by cuDF will be slower, as they fall back to using Pandas (copying data between the CPU and GPU under the hood as needed). For example, cuDF does not currently support the `axis=` parameter to the `count` method. So this operation will run on the CPU and be noticeably slower than the previous one."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mThydJIYpuha",
        "outputId": "b803003f-adb3-4ef0-c152-0def5aa39a2f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 8.36 s, sys: 2.25 s, total: 10.6 s\n",
            "Wall time: 13.6 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0           5\n",
              "1           5\n",
              "2           5\n",
              "3           5\n",
              "4           5\n",
              "           ..\n",
              "15435602    6\n",
              "15435603    6\n",
              "15435604    6\n",
              "15435605    6\n",
              "15435606    6\n",
              "Length: 15435607, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "%%time\n",
        "df.count(axis=1) # This will use pandas, because cuDF doesn't support axis=1 for the .count() method"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tbDVvkP2pyra"
      },
      "source": [
        "But the story doesn't end here. We often need to mix our own code with third-party libraries that other people have written. Many of these libraries accept pandas objects as inputs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3yK3a-mIp0vr"
      },
      "source": [
        "# Using third-party libraries with cudf.pandas\n",
        "\n",
        "You can pass Pandas objects to third-party libraries when using `cudf.pandas`, just like you would when using regular Pandas.\n",
        "\n",
        "Below, we show an example of using [plotly-express](https://plotly.com/python/plotly-express/) to visualize the data we've been processing:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H0QwPQcAp2RV"
      },
      "source": [
        "## Visualizing which states have more pickup trucks relative to other vehicles?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "Ecs213eEqCd9",
        "outputId": "04c33fe3-c170-4078-a96f-e8adf1ca8336"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"3de1faf7-ccd4-46a4-8693-88cd613843bf\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"3de1faf7-ccd4-46a4-8693-88cd613843bf\")) {                    Plotly.newPlot(                        \"3de1faf7-ccd4-46a4-8693-88cd613843bf\",                        [{\"coloraxis\":\"coloraxis\",\"geo\":\"geo\",\"hovertemplate\":\"reg_state=%{location}\\u003cbr\\u003e% Pickup Trucks=%{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"locationmode\":\"USA-states\",\"locations\":[\"99\",\"AB\",\"AK\",\"AL\",\"AR\",\"AZ\",\"BC\",\"CA\",\"CO\",\"CT\",\"DC\",\"DE\",\"DP\",\"FL\",\"FO\",\"GA\",\"GV\",\"HI\",\"IA\",\"ID\",\"IL\",\"IN\",\"KS\",\"KY\",\"LA\",\"MA\",\"MD\",\"ME\",\"MI\",\"MN\",\"MO\",\"MS\",\"MT\",\"MX\",\"NB\",\"NC\",\"ND\",\"NE\",\"NH\",\"NJ\",\"NM\",\"NS\",\"NT\",\"NV\",\"NY\",\"OH\",\"OK\",\"ON\",\"OR\",\"PA\",\"PE\",\"PR\",\"QB\",\"RI\",\"SC\",\"SD\",\"SK\",\"TN\",\"TX\",\"UT\",\"VA\",\"VT\",\"WA\",\"WI\",\"WV\",\"WY\"],\"name\":\"\",\"z\":[0.13389536325316143,3.361344537815126,3.231017770597738,1.7287093010923744,2.394492666866208,1.9063117517145183,2.380952380952381,1.5011320889269222,2.5121819166215484,2.2371009577431846,0.8736415938632005,1.5900925005733506,0.10559662090813093,1.240151879359909,5.769230769230769,1.218891034221343,0.15390534821085033,2.035623409669211,2.0983832129342965,1.6982364467668192,2.222128698287109,0.19814151347766662,3.771551724137931,1.929158760278305,2.3114937031723257,1.806253890724513,1.6966827976518286,1.1657951399569364,2.4443999198557402,1.1358235909134113,2.980698753970193,2.541914548404543,8.7442218798151,null,4.2105263157894735,1.6118169981592063,3.4954407294832825,2.7896995708154506,2.7165297593313156,2.3414980998610484,3.4408602150537635,2.8368794326241136,null,2.0449212202480727,3.085109006789598,1.557976619301231,1.1781183061459877,5.109179563281747,0.8855435023245517,1.535894444602521,null,3.9877300613496933,0.6427110721589249,2.050473186119874,1.5910269355675026,1.0161662817551964,6.666666666666667,1.2353172194651718,1.598054542296335,2.064516129032258,1.541369928782908,3.519491628986924,2.6731533793680686,0.7976794778825236,2.094361334867664,7.054455445544554],\"type\":\"choropleth\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"geo\":{\"domain\":{\"x\":[0.0,1.0],\"y\":[0.0,1.0]},\"center\":{},\"scope\":\"usa\"},\"coloraxis\":{\"colorbar\":{\"title\":{\"text\":\"% Pickup Trucks\"}},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"legend\":{\"tracegroupgap\":0},\"margin\":{\"t\":60}},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('3de1faf7-ccd4-46a4-8693-88cd613843bf');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "import plotly.express as px\n",
        "\n",
        "df = df.rename(columns={\n",
        "    \"Registration State\": \"reg_state\",\n",
        "    \"Vehicle Body Type\": \"vehicle_type\",\n",
        "})\n",
        "\n",
        "# vehicle counts per state:\n",
        "counts = df.groupby(\"reg_state\").size().sort_index()\n",
        "# vehicles with type \"PICK\" (Pickup Truck)\n",
        "pickup_counts = df.where(df[\"vehicle_type\"] == \"PICK\").groupby(\"reg_state\").size()\n",
        "# percentage of pickup trucks by state:\n",
        "pickup_frac = ((pickup_counts / counts) * 100).rename(\"% Pickup Trucks\")\n",
        "del pickup_frac[\"MB\"]  # (Manitoba is a huge outlier!)\n",
        "\n",
        "# plot the results:\n",
        "pickup_frac = pickup_frac.reset_index()\n",
        "px.choropleth(pickup_frac, locations=\"reg_state\", color=\"% Pickup Trucks\", locationmode=\"USA-states\", scope=\"usa\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9bgMrWs5qDG_"
      },
      "source": [
        "## Beyond just passing data: **Accelerating** third-party code\n",
        "\n",
        "Being able to pass these proxy objects to libraries like Plotly is great, but the benefits don't end there.\n",
        "\n",
        "When you enable `cudf.pandas`, pandas operations running **inside the third-party library's functions** will also benefit from GPU acceleration where possible!\n",
        "\n",
        "Below, you can see an image illustrating how `cudf.pandas` can accelerate the pandas backend in Ibis, a library that provides a unified DataFrame API to various backends. We ran this example on a system with an NVIDIA H100 GPU and an Intel Xeon Platinum 8480CL CPU.\n",
        "\n",
        "\n",
        "By loading the `cudf.pandas` extension, pandas operations within Ibis can use the GPU with zero code change. It just works."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8JW2CQL6qEv3"
      },
      "source": [
        "![ibis](https://drive.google.com/uc?id=1uOJq2JtbgVb7tb8qw8a2gG3JRBo72t_H)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pyVNtGUhtFs5"
      },
      "source": [
        "# Conclusion\n",
        "\n",
        "With `cudf.pandas`, you can keep using pandas as your primary dataframe library. When things start to get a little slow, just load the `cudf.pandas` and run your existing code on a GPU!\n",
        "\n",
        "To learn more, we encourage you to visit [rapids.ai/cudf-pandas](https://rapids.ai/cudf-pandas)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XjELOIf3xykH"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.8"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}