Example: finding camera lens intrinsics

This document is intended as a reference on how to calibrate a camera lens using sltk.

In this example, the desired data is the following intrinsic calibration parameters:

• 3x3 camera matrix
• 5x1 camera radial/tangential distortion coefficients
• 2x1 camera pixel resolution

Crash course on camera lens calibration

A sequence of black and white patterns are shown on a flat screen (e.g. LCD) and individually captured by a camera. These images are what's needed to calibrate a lens with structured light. slcrunch takes these images and calculates a list of pixels in the camera and screen that correspond to each other.

If everything goes well, the intermediate output will be data saying monitor pixel (102,600) was seen at camera pixel (231,402) for many many pairs of points, which will be passed to slcalibrate to calculate camera calibration (camera matrix, distortion,..). The flat screen monitor is the calibration object, instead of a printed chessboard pattern. If we know the pixel pitch of the display (the physical length of a single pixel edge), we can translate between 2D pixel addresses and 3D real world coordinates. For example, given:

• scale value: s = 0.2652 millimeters/pixel = 0.0002652 meters/pixel

• 2D display pixel: (Xd,Yd) = (20 pixels, 74 pixels)

• 2D camera pixel: (Xc,Yc) = (200 pixels, 300 pixels)

Let's calculate the real world coordinate of this pixel in relation to the 0px,0px corner (generally top left).

Xd*s = 20 pixels * 0.0002652 meters/pixel = 0.005304 meters

Yd*s = 74 pixels * 0.0002652 meters/pixel = 0.019625 meters

So, real world X = 0.005304 meters = 5.3 millimeters and real world Y = 0.019625 meters = 19.6 millimeters The pixel is located at real world coordinates (X,Y,Z) = (5.3mm, 19.6mm, 0mm). The last number Z is always zero because the display is flat. Basically all LCD panels are machined to be extremely flat so this condition is relatively easy to satisfy.

So the core data for camera calibration is a huge list of pairs like this:

• camera (200px, 300px) saw the point in space : (5.3mm, 19.6mm, 0mm)

Procedure

Programs used for camera calibration:

1. sldisp
2. mjpg-streamer
3. slcapture.py
4. slcontrol
5. slcrunch
6. slcalibrate

Procedure:

1. Launch sldisp, drag window to output display, and press 'f' to enable fullscreen output.
2. Launch mjpg-streamer, use live camera view in browser to focus camera on flatscreen monitor output of sldisp. Try to fill as much of the camera's field of view with the display as possible.
3. Run slcapture.py and verify image frames are coming in.
4. Run slcontrol GUI and press return to begin a single scan (~40 patterns/images) or click START_CAPTURE
5. Reposition the monitor relative to the camera, ideally avoiding adjustments to focus.
6. Repeat steps 3-4 until enough views have been captured (in practice, 4-5 with large variation in angle provide good results)
7. Run slcrunch on each sequence file list (cap_XX/list.yaml) to get correspondence output files (slscan_XXXX.yaml).
8. Run list_creator to generate a list (chesslist.yaml) of correspondence files to be used for calibration.
9. Run slcalibrate using the list of correspondence data (chesslist.yaml).
10. Save calibration_XXXX.yaml to desired location.

Notes:

When running slcrunch for camera calibration, it's best to use the -st <subsample_type> and -sf <subsample_factor> options to subsample the correspondence data. Calibration can take extremely long with no subsampling. Example arguments: "-st point -sf 32" or "-st grid -sf 64". The below images show visualizations (-vis option) of some correspondence data using grid subsampling:

Below are examples of captured images of the highest frequency x and y (respectively) patterns shown on a monitor. Note that at full zoom, each monitor pixel is observed by multiple camera pixels.