csi_camera.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"../images/DLI Header.png\" alt=\"Header\" style=\"width: 400px;\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hello Camera\n",
    "### CSI (Camera Serial Interface) Cameras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, you can test your camera to make sure it's working on the Jetson Nano as expected.  It should already be plugged into the MIPI CSI camera port.  Make sure there is no obstruction on the camera lens such as a film or cover."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"../images/csi_setup_sm.jpg\" width=600/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border:2px solid black; background-color:#e3ffb3; font-size:12px; padding:8px; margin-top: auto;\"><i>\n",
    "    <h4><i>Tip</i></h4>\n",
    "To execute the Python or system code in the code cells, select the cell and click the \"Run\" button at the top of the window.<br>Keyboard shortcut: <strong>[SHIFT][ENTER]</strong>\n",
    "    </i></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check to see if the device is available\n",
    "Execute the following system command to list all video devices on the Jetson Nano.  If your camera doesn't show up with a device id, check your connection.  You should get an output similar to \n",
    "```text\n",
    "crw-rw----+ 1 root video 81, 0 Jun  2 17:35 /dev/video0\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "crw-rw----+ 1 root video 81, 0 May 26 10:13 /dev/video0\r\n"
     ]
    }
   ],
   "source": [
    "!ls -ltrh /dev/video*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the camera object\n",
    "\n",
    "First, create a camera object by importing the `CSICamera` class from the library by executing the following Python code cell.  Please note, you can only create one `CSICamera` instance.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from jetcam.csi_camera import CSICamera\n",
    "\n",
    "camera = CSICamera(width=224, height=224)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then capture a frame from the camera with the `read` method.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(224, 224, 3)\n"
     ]
    }
   ],
   "source": [
    "image = camera.read()\n",
    "\n",
    "print(image.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling the `read` method for `camera` also updates the camera's internal `value`.  By looking at the value's `shape`, we see three numbers representing the pixel height, pixel width, and number of color channels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(224, 224, 3)\n"
     ]
    }
   ],
   "source": [
    "print(camera.value.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a widget to view the image stream\n",
    "We can create a \"widget\" to display this image in the notebook.  In order to see the image, convert it from its blue-green-red format (brg8) to a format the browser can display (jpeg).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1f29c81ff8654b2ead11debc62f3e055",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Image(value=b'\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\x01\\x01\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\xff\\xdb\\x00C\\x00\\x02\\x01\\x0…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ipywidgets\n",
    "from IPython.display import display\n",
    "from jetcam.utils import bgr8_to_jpeg\n",
    "\n",
    "image_widget = ipywidgets.Image(format='jpeg')\n",
    "\n",
    "image_widget.value = bgr8_to_jpeg(image)\n",
    "\n",
    "display(image_widget)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see an image from the camera if all is working correctly.  If there seems to be an image but it's fuzzy or a funny color, check to make sure there is no protective film or cap on the lens.  \n",
    "\n",
    "Now let's watch a live stream from the camera.  Set the `running` value of the camera to continuously update the value in background.  This allows you to attach \"callbacks\" to the camera value changes.\n",
    "\n",
    "The \"callback\" here is the function, `update_image`, which is attached by calling the `observe` method below.  `update_image` is executed whenever there is a new image available to process, which is then displayed in the widget."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.running = True\n",
    "\n",
    "def update_image(change):\n",
    "    image = change['new']\n",
    "    image_widget.value = bgr8_to_jpeg(image)\n",
    "    \n",
    "camera.observe(update_image, names='value')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you move something in front of the camera, you should now see the live video stream in the widget. To stop it, unattach the callback with the `unobserve` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.unobserve(update_image, names='value')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border:2px solid black; background-color:#e3ffb3; font-size:12px; padding:8px; margin-top: auto;\"><i>\n",
    "    <h4><i>Tip</i></h4>\n",
    "You can move the widgets (or any cell) to new window tabs in JupyterLab by right-clicking the cell and selecting \"Create New View for Output\". This way, you can continue to scroll down the JupyterLab notebook and still see the camera view!\n",
    "    </i></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Another way to view the image stream\n",
    "You can also use the traitlets `dlink` method to connect the camera to the widget, using a transform as one of the parameters.  This eliminates some steps in the process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import traitlets\n",
    "\n",
    "camera_link = traitlets.dlink((camera, 'value'), (image_widget, 'value'), transform=bgr8_to_jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can remove the camera/widget link with the `unlink` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera_link.unlink()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and reconnect it again with `link`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera_link.link()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shut down the kernel of this notebook to release the camera resource. \n",
    "Return to the DLI course pages for the next instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border:2px solid black; background-color:#e3ffb3; font-size:12px; padding:8px; margin-top: auto;\"><i>\n",
    "    <h4><i>Tip</i></h4>\n",
    "<p>There can only be one instance of CSICamera or USBCamera at a time.  If you want to create a new camera instance, you must first release the existing one. To do so, shut down the notebook's kernel from the JupyterLab pull-down menu: <strong>Kernel->Shutdown Kernel</strong>, then restart it with <strong>Kernel->Restart Kernel</strong>.</p>\n",
    "    <p> If the camera setup appears \"stuck\" or the images \"frozen\", follow these steps:\n",
    "    <ol><li>Shut down the notebook kernel as explained above</li>\n",
    "        <li>Open a terminal on the Jetson Nano by clicking the \"Terminal\" icon on the \"Launch\" page</li>\n",
    "        <li>Enter the following command in the terminal window:  <code>sudo systemctl restart nvargus-daemon</code> with password:<code>dlinano</code> </li>\n",
    "    </ol>\n",
    "    </i></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"../images/DLI Header.png\" alt=\"Header\" style=\"width: 400px;\"/></center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}