01-intro-gpu.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "illegal-thumb",
   "metadata": {},
   "source": [
    "# Introduction to GPU Programming with Python\n",
    "## Intro to GPU computing\n",
    "Questions\n",
    "* How to program GPU with Python ?\n",
    "\n",
    "Objectives\n",
    "* Understand difference between CPU and GPU architectures\n",
    "* Understand the complexity of the problem\n",
    "* Choose a python library to access GPU instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "normal-cherry",
   "metadata": {},
   "source": [
    "### What is GPU\n",
    "GPU is a graphics processing unit (GPU), a computer chip that renders graphics and images by performing rapid mathematical calculations. \n",
    "![](images/what_is_gpu.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a8cc9d0-4b73-4da5-ad5e-720a9924ae4a",
   "metadata": {},
   "source": [
    "### Understand the complexity of the problem "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abe97ae8-660e-44fd-8785-787b9bc3eda5",
   "metadata": {},
   "source": [
    "If you work with neural networks then TF and Torch may suit you really well. However, you will be limited by what those high level libs can offer ? What if you have other kind of problem that you may want to accelerate with GPUs ? For this you will need to understand the difference between CPU and GPU:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25259dfe-0ab7-4f8f-a4ce-7bbec4a9c754",
   "metadata": {},
   "source": [
    "![](images/cpu_gpu.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3728a148-ef49-4c1c-96f6-9b4adc2168f3",
   "metadata": {},
   "source": [
    "### CPU vs GPU\n",
    "CPU :\n",
    "* Small number of cores\n",
    "* Low latency\n",
    "* CPU core is heavyweight\n",
    "* High precission\n",
    "* Perfect for serial codes\n",
    "* Parallel processing is mediocre (can't handle millions of identical ops)\n",
    "\n",
    "GPUs :\n",
    "* Huge number of cores\n",
    "* GPU core is lightweight\n",
    "* Very high compute and data throughput\n",
    "* Perfect for massive parallel processing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0993127c-03d4-4a88-b795-965e24d32be3",
   "metadata": {},
   "source": [
    "### Is my code well suited for the GPUs ? \n",
    "1. Does your algorithm or program has a lot of SIMD operations (Single Instruction Multiple Data operations) ? \n",
    "    - Examples: Matrix operations like matrix multiplication, addition, SVD, Eigen value computation, Rotation, Image operations, Machine learning operations, Particle simulations, FFT, convolution etc. are SIMD kind of operations.\n",
    "2.  Does your program need to manipulate Large amount of data at time? (If the amount of data > 2 time cache size of CPU, it can be considered large data) ?\n",
    "\n",
    "If you answered YES to both questions, then your program will achieve good speedup in GPU. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b8980ee-99cd-4afc-b011-4cfda4a8f610",
   "metadata": {},
   "source": [
    "### Can GPU be programmed with a high-level language like Python ? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfc3a112-2f59-41d5-9793-87fa4ec499a1",
   "metadata": {},
   "source": [
    "There are lots of libraries in Python, including neural network libraries like TF or Torch. But what if you need more general oriented ones, say for matrix multiplication ? Numba is an elegant solution that offers flexibility and can help with custom made kernels."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0003335f-836a-4543-8c1f-65b4fd295181",
   "metadata": {},
   "source": [
    "![](images/gpu_libs.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27dba612-69c6-40bd-88f0-d41f61ceebc9",
   "metadata": {},
   "source": [
    "With Numba you can approximate CUDA C using just Python !"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7cf44ed-9a94-4ecc-8fc4-f65e9b4ac69f",
   "metadata": {},
   "source": [
    "## Key points\n",
    "* **GPU** \n",
    "    * a device physically separated from the CPU\n",
    "    * has its own memory and scheduler\n",
    "    * but does not work without CPU\n",
    "    * lots of cores but lightweight\n",
    "* **Core** \n",
    "    * is a processing unit that reads instructions to perform specific actions\n",
    "    * CPU and GPU cores are different\n",
    "* **Parallelization**  \n",
    "    * a computing technique involving processing data in parallel (or simmultaneously)\n",
    "* **Python GPU libraries** \n",
    "    * Numba\n",
    "    * PyCuda\n",
    "    * CuPy\n",
    "    * PyTorch"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}