Geometric calibration of a camera

Introduction

By calibrating the parameters of the lens and image sensors, the distortion of an image can be calibrated to reconstruct the relative position of a camera and objects in the three-dimensional world. It has been widely used in the areas of remote sensing, measurement, etc. The MATLAB provides calibration tool based on the calibration method proposed by Zhang et al. In this project, the crema of a Samsung Galaxy A20 was calibrated by using MATLAB toolbox. The results showed high accuracy in both standard mode and fisheye mode.

Background

The camera calibration process is a transformation process from the 3D world coordinate to the image coordinate system. This process can be artificially divided into two steps: 1) from the 3-D world coordinate system to the 3-D camera’s coordinate system (extrinsic); 2) from the 3-D camera’s coordinate system to the image plane coordinate system (intrinsic).

Extrinsic parameter The extrinsic parameter is a 3D to 3D transformation, it describes the rotation (R) and translation (t) of the relative position of crema and 3D object. Intrinsic parameter The camera can be modeled as a pinhole crema as shown in figure 2. Hence, the intrinsic transformation is to transform from a 3D camera coordinate to 2D image plane coordinate, which can be modeled as K = [fx 0 0|s fy 0|cx cy 1], where fx and fy is the normalized focal length, cx and cy is the optical center of the image, and s is Skew coefficient for non-perpendicular axes case.

Therefore, the calibration process can be mathematically described as: P = [R|t]K.

Distortion The lens distortion in camera contains radical and tangential distortion, which cannot be described by the pinhole camera model. For the radical distortion, the further from the center, the larger the distortion we have, which can be corrected by using Taylor series expansion: xdistorted = x(1 + k1*r2 + k2*r4 + k3*r6); ydistorted= y(1 + k1*r2 + k2*r4 + k3*r6) On the other hand, the tangential distortion origins from the unparallel of the lens and camera sensor plane, which can be corrected by xdistorted = x + [2 * p1 * x * y + p2 * (r2 + 2 * x2)] and ydistorted = y + [p1 * (r2 + 2 *y2) + 2 * p2 * x * y].

Methods and Results

Pattern for calibration The checkerboard pattern is used for MATLAB Camera Calibrator. The pattern was printed on a letter size paper and stick on a hard board to prevent curvature on the pattern. The grid size of the checkerboard pattern is 21.2×21.2mm. The pattern was hold ~50 cm from the camera and was focus by the camera. The camera was not in auto focus mode and the zoom keep constant in the process. Totally 15 images were taken with different orientation relative to the camera. These pictures were shown in figure 2.

Calibration After the identification of the checkerboard, the standard mode with two coefficients were first tested. The mean reprojection error is shown in figure 3. While the three coefficients give a lower error (mean value 1.39 pixels), nevertheless, the two coefficient is sufficient for the standard pictures. The extrinsic parameters (the relative orientation of the checkerboard with respect to the camera) are then shown in the 3D diagram in figure 4. Figure 4a demonstrates a camera-centric schematic, while figure 4b is a pattern-centric view.

File:Yecunfig3.jpg

File:Yecunfig4.jpg

The intrinsic and extrinsic parameters were exported as: Intrinsics

---

Focal length (pixels): [ 3310.3793 +/- 10.4656 3309.6850 +/- 10.4052 ] Principal point (pixels):[ 2075.4137 +/- 2.2403 1538.1733 +/- 2.2774 ] Radial distortion: [ 0.3611 +/- 0.0107 -1.7308 +/- 0.1035 2.7642 +/- 0.2904 ]

Extrinsics

---

Rotation vectors:

                        [    0.0494 +/- 0.0019        0.1789 +/- 0.0027       -3.0998 +/- 0.0003  ]
                        [    0.1573 +/- 0.0019        0.2086 +/- 0.0024       -2.6450 +/- 0.0003  ]
                        [   -0.1917 +/- 0.0018        0.5245 +/- 0.0021       -3.0313 +/- 0.0004  ]
                        [   -0.1036 +/- 0.0019        0.3879 +/- 0.0022       -2.7345 +/- 0.0004  ]
                        [   -0.3314 +/- 0.0026       -0.1681 +/- 0.0029        2.9346 +/- 0.0004  ]
                        [    0.3260 +/- 0.0018        0.0409 +/- 0.0019       -3.0734 +/- 0.0003  ]
                        [    0.0438 +/- 0.0019       -0.2061 +/- 0.0015       -0.0696 +/- 0.0003  ]
                        [    0.0192 +/- 0.0017        0.1670 +/- 0.0016        0.1131 +/- 0.0003  ]
                        [   -0.0683 +/- 0.0016       -0.2486 +/- 0.0013        0.0169 +/- 0.0003  ]
                        [    0.1402 +/- 0.0022       -0.0764 +/- 0.0026       -2.5605 +/- 0.0003  ]
                        [    0.0375 +/- 0.0021        0.2135 +/- 0.0020       -1.2631 +/- 0.0003  ]
                        [    0.2354 +/- 0.0019       -0.0809 +/- 0.0020       -2.1068 +/- 0.0003  ]
                        [    0.2133 +/- 0.0017       -0.1764 +/- 0.0018       -2.3140 +/- 0.0003  ]
                        [    0.4288 +/- 0.0016        0.3009 +/- 0.0018       -2.0037 +/- 0.0004  ]
                        [    0.1148 +/- 0.0014       -0.4500 +/- 0.0016       -2.4415 +/- 0.0003  ]

Translation vectors (millimeters):

                        [   83.8853 +/- 0.1960       51.8912 +/- 0.2008      300.4381 +/- 0.9506  ]
                        [   56.7701 +/- 0.2154       83.4066 +/- 0.2193      333.9287 +/- 1.0382  ]
                        [   61.8608 +/- 0.2096       38.1369 +/- 0.2203      324.2352 +/- 0.9936  ]
                        [   96.5474 +/- 0.2207       97.1636 +/- 0.2237      340.3009 +/- 1.0644  ]
                        [   39.1373 +/- 0.2250       48.6800 +/- 0.2273      340.0936 +/- 1.0491  ]
                        [  108.7953 +/- 0.1930       41.8584 +/- 0.2046      306.2588 +/- 0.9483  ]
                        [ -113.7009 +/- 0.2074      -49.4557 +/- 0.1963      282.3532 +/- 0.9158  ]
                        [ -124.2760 +/- 0.2455      -75.8827 +/- 0.2545      376.1750 +/- 1.1748  ]
                        [  -80.5848 +/- 0.2133      -54.0782 +/- 0.2037      298.2958 +/- 0.9469  ]
                        [   60.1974 +/- 0.2277       75.5904 +/- 0.2332      342.6287 +/- 1.0913  ]
                        [    3.6265 +/- 0.2419       84.3545 +/- 0.2429      369.9493 +/- 1.1358  ]
                        [   10.9645 +/- 0.2281       99.2212 +/- 0.2271      338.6736 +/- 1.0737  ]
                        [   45.5693 +/- 0.2144       90.2729 +/- 0.2198      322.1885 +/- 1.0307  ]
                        [  -39.3711 +/- 0.2535       81.1931 +/- 0.2570      380.2468 +/- 1.0889  ]
                        [   18.1064 +/- 0.1943       71.1540 +/- 0.2000      284.4594 +/- 0.9456  ]

With the intrinsic parameters, we can calculate the focal length in mm scale, i.e., ~3.45 mm. While according to the information of the Samsung, the focal length is 27mm (35mm equivalent), which is actually 3.63mm, so the estimation error is ~4.9%.

An example of the original image and undistorted image are shown in figure 5.

File:Yecunfig5.jpg

Measurement of objects We then took the parameters of the camera to measure the size of a penny. Four coins were put on top of the checkerboard as shown in figure 6a, and the image were taken about 50 cm away from the checkerboard. The undistorted image can be obtained by using the parameters of the calibrated camera as P=[R|t]K, which is demonstrated in figure 6b. Since the penny is shiny and colorful than the background, so it is possible to detect it from the coins in by using the saturation component of the HSV representation of the image. The image was first converted to HSV color space, and then threshold the image to black and white as shown in figure 6c. The coins can be detected as the image in figure 6d.