Geometric calibration of a camera: Difference between revisions
imported>Student221 No edit summary |
imported>Student221 No edit summary |
||
| Line 5: | Line 5: | ||
The camera calibration process is a transformation process from the 3D world coordinate to the image coordinate system. This process can be artificially divided into two steps: 1) from the 3-D world coordinate system to the 3-D camera’s coordinate system (extrinsic); 2) from the 3-D camera’s coordinate system to the image plane coordinate system (intrinsic). | The camera calibration process is a transformation process from the 3D world coordinate to the image coordinate system. This process can be artificially divided into two steps: 1) from the 3-D world coordinate system to the 3-D camera’s coordinate system (extrinsic); 2) from the 3-D camera’s coordinate system to the image plane coordinate system (intrinsic). | ||
[[File:1.png]] | [[File:1.png|600px|center|thumb|'''Figure 1''': Checkerboard patterns for calibration.]] | ||
Extrinsic parameter | Extrinsic parameter | ||
Revision as of 05:30, 12 December 2019
Introduction
By calibrating the parameters of the lens and image sensors, the distortion of an image can be calibrated to reconstruct the relative position of a camera and objects in the three-dimensional world. It has been widely used in the areas of remote sensing, measurement, etc. The MATLAB provides calibration tool based on the calibration method proposed by Zhang et al. In this project, the crema of a Samsung Galaxy A20 was calibrated by using MATLAB toolbox. The results showed high accuracy in both standard mode and fisheye mode.
Background
The camera calibration process is a transformation process from the 3D world coordinate to the image coordinate system. This process can be artificially divided into two steps: 1) from the 3-D world coordinate system to the 3-D camera’s coordinate system (extrinsic); 2) from the 3-D camera’s coordinate system to the image plane coordinate system (intrinsic).
Extrinsic parameter The extrinsic parameter is a 3D to 3D transformation, it describes the rotation (R) and translation (t) of the relative position of crema and 3D object. Intrinsic parameter The camera can be modeled as a pinhole crema as shown in figure 2. Hence, the intrinsic transformation is to transform from a 3D camera coordinate to 2D image plane coordinate, which can be modeled as K = [fx 0 0|s fy 0|cx cy 1], where fx and fy is the normalized focal length, cx and cy is the optical center of the image, and s is Skew coefficient for non-perpendicular axes case.
Therefore, the calibration process can be mathematically described as: P = [R|t]K.
Distortion The lens distortion in camera contains radical and tangential distortion, which cannot be described by the pinhole camera model. For the radical distortion, the further from the center, the larger the distortion we have, which can be corrected by using Taylor series expansion: xdistorted = x(1 + k1*r2 + k2*r4 + k3*r6); ydistorted= y(1 + k1*r2 + k2*r4 + k3*r6) On the other hand, the tangential distortion origins from the unparallel of the lens and camera sensor plane, which can be corrected by xdistorted = x + [2 * p1 * x * y + p2 * (r2 + 2 * x2)] and ydistorted = y + [p1 * (r2 + 2 *y2) + 2 * p2 * x * y].
Methods and Results
Pattern for calibration The checkerboard pattern is used for MATLAB Camera Calibrator. The pattern was printed on a letter size paper and stick on a hard board to prevent curvature on the pattern. The grid size of the checkerboard pattern is 21.2×21.2mm. The pattern was hold ~50 cm from the camera and was focus by the camera. The camera was not in auto focus mode and the zoom keep constant in the process. Totally 15 images were taken with different orientation relative to the camera. These pictures were shown in figure 2.
Calibration After the identification of the checkerboard, the standard mode with two coefficients were first tested. The mean reprojection error is shown in figure 3. While the three coefficients give a lower error (mean value 1.39 pixels), nevertheless, the two coefficient is sufficient for the standard pictures. The extrinsic parameters (the relative orientation of the checkerboard with respect to the camera) are then shown in the 3D diagram in figure 4. Figure 4a demonstrates a camera-centric schematic, while figure 4b is a pattern-centric view.
The intrinsic and extrinsic parameters were exported as:
Intrinsics
Focal length (pixels): [ 3310.3793 +/- 10.4656 3309.6850 +/- 10.4052 ] Principal point (pixels):[ 2075.4137 +/- 2.2403 1538.1733 +/- 2.2774 ] Radial distortion: [ 0.3611 +/- 0.0107 -1.7308 +/- 0.1035 2.7642 +/- 0.2904 ]
Extrinsics
Rotation vectors:
[ 0.0494 +/- 0.0019 0.1789 +/- 0.0027 -3.0998 +/- 0.0003 ]
[ 0.1573 +/- 0.0019 0.2086 +/- 0.0024 -2.6450 +/- 0.0003 ]
[ -0.1917 +/- 0.0018 0.5245 +/- 0.0021 -3.0313 +/- 0.0004 ]
[ -0.1036 +/- 0.0019 0.3879 +/- 0.0022 -2.7345 +/- 0.0004 ]
[ -0.3314 +/- 0.0026 -0.1681 +/- 0.0029 2.9346 +/- 0.0004 ]
[ 0.3260 +/- 0.0018 0.0409 +/- 0.0019 -3.0734 +/- 0.0003 ]
[ 0.0438 +/- 0.0019 -0.2061 +/- 0.0015 -0.0696 +/- 0.0003 ]
[ 0.0192 +/- 0.0017 0.1670 +/- 0.0016 0.1131 +/- 0.0003 ]
[ -0.0683 +/- 0.0016 -0.2486 +/- 0.0013 0.0169 +/- 0.0003 ]
[ 0.1402 +/- 0.0022 -0.0764 +/- 0.0026 -2.5605 +/- 0.0003 ]
[ 0.0375 +/- 0.0021 0.2135 +/- 0.0020 -1.2631 +/- 0.0003 ]
[ 0.2354 +/- 0.0019 -0.0809 +/- 0.0020 -2.1068 +/- 0.0003 ]
[ 0.2133 +/- 0.0017 -0.1764 +/- 0.0018 -2.3140 +/- 0.0003 ]
[ 0.4288 +/- 0.0016 0.3009 +/- 0.0018 -2.0037 +/- 0.0004 ]
[ 0.1148 +/- 0.0014 -0.4500 +/- 0.0016 -2.4415 +/- 0.0003 ]
Translation vectors (millimeters):
[ 83.8853 +/- 0.1960 51.8912 +/- 0.2008 300.4381 +/- 0.9506 ]
[ 56.7701 +/- 0.2154 83.4066 +/- 0.2193 333.9287 +/- 1.0382 ]
[ 61.8608 +/- 0.2096 38.1369 +/- 0.2203 324.2352 +/- 0.9936 ]
[ 96.5474 +/- 0.2207 97.1636 +/- 0.2237 340.3009 +/- 1.0644 ]
[ 39.1373 +/- 0.2250 48.6800 +/- 0.2273 340.0936 +/- 1.0491 ]
[ 108.7953 +/- 0.1930 41.8584 +/- 0.2046 306.2588 +/- 0.9483 ]
[ -113.7009 +/- 0.2074 -49.4557 +/- 0.1963 282.3532 +/- 0.9158 ]
[ -124.2760 +/- 0.2455 -75.8827 +/- 0.2545 376.1750 +/- 1.1748 ]
[ -80.5848 +/- 0.2133 -54.0782 +/- 0.2037 298.2958 +/- 0.9469 ]
[ 60.1974 +/- 0.2277 75.5904 +/- 0.2332 342.6287 +/- 1.0913 ]
[ 3.6265 +/- 0.2419 84.3545 +/- 0.2429 369.9493 +/- 1.1358 ]
[ 10.9645 +/- 0.2281 99.2212 +/- 0.2271 338.6736 +/- 1.0737 ]
[ 45.5693 +/- 0.2144 90.2729 +/- 0.2198 322.1885 +/- 1.0307 ]
[ -39.3711 +/- 0.2535 81.1931 +/- 0.2570 380.2468 +/- 1.0889 ]
[ 18.1064 +/- 0.1943 71.1540 +/- 0.2000 284.4594 +/- 0.9456 ]
With the intrinsic parameters, we can calculate the focal length in mm scale, i.e., ~3.45 mm. While according to the information of the Samsung, the focal length is 27mm (35mm equivalent), which is actually 3.63mm, so the estimation error is ~4.9%.
An example of the original image and undistorted image are shown in figure 5.
Measurement of objects
We then took the parameters of the camera to measure the size of a penny. Four coins were put on top of the checkerboard as shown in figure 6a, and the image were taken about 50 cm away from the checkerboard. The undistorted image can be obtained by using the parameters of the calibrated camera as P=[R|t]K, which is demonstrated in figure 6b. Since the penny is shiny and colorful than the background, so it is possible to detect it from the coins in by using the saturation component of the HSV representation of the image. The image was first converted to HSV color space, and then threshold the image to black and white as shown in figure 6c. The coins can be detected as the image in figure 6d.
The image was then transformed into 3D world coordinate to measure the size of the penny. The diameter of the four pennies were shown in table 1 with a mean value of 19.19 mm and variance of 0.02. The actual size of a penny should be 19.05 mm, the test results give an error of 0.7%, indicating the high accuracy of the measurement.
[ # Diameter (mm) ]
[ 1 19.30 ]
[ 2 19.40 ]
[ 3 19.12 ]
[ 4 18.95 ]
Fisheye mode calibration The fisheye lens is an extremely wide-angle lens that the angle of view of is between 100 and 180 degrees. However, to achieve the ultra-wide-angle, the lines of perspective in the images are distorted. The pinhole camera model is no longer suitable for the fisheye lens because the alignment coefficients of the intrinsic parameters are related to the relative position of the sensor plane and the pixel plane. In this project, instead of using a fisheye lens, the fish eye image were taken from a software embedded in the Samsung Galaxy A20 camera as shown in figure 7.
By using the Matlab toolbox in fisheye mode, we can calibrate the fisheye image as shown in figure 8.
Conclusions
In summary, by using the MATLAB camera calibrator, both standard and fisheye mode image are calibrated. The measurements of real planner objects show high accuracy without distortion. The future work may concentrate on the calibration of dual camera model to measure the depth and movement of object in 3D space.