Depth Mapping Algorithm Performance Analysis

Introduction

We will implement various disparity estimation algorithms and compare their performance.

Background

Disparity and Depth

Depth information about a scene can be captured using a stereo camera (2 cameras that are separated horizontally but aligned vertically). The stereo image pair taken by the stereo camera contains this depth information in the horizontal differences (when comparing the stereo image pair, objects closer to the camera will be more horizontally displaced). These differences (also called disparities) can be used to determine the relative distance from the camera for different objects in the scene. In Figure 1, you can see such differences on the left where the red and blue don't match up.

Figure 1. Anaglyph of stereo image pair (left) and example disparity map computed from the same stereo image pair (right)

Disparity and depth can be related by the following equation (where x-x' is disparity, z is depth, f is the focal length, and B is the interocular distance). !!!!!!!!!!!!OSCAR WRITE STUFF HERE!!!!!!!!!!!!!!!!!!!!!!

$x-x^{\prime }=\frac{Bf}{z}$

Figure 2. Diagram to Calculate Disparity and Depth

Image Rectification

In order to extract depth information, the stereo image pair must first be rectified (i.e. the images must be transformed in some way such that the only differences that remain are horizontal differences corresponding to the distance of the object from the camera). Rectification can be accomplished both with and without camera calibration.

If the corresponding camera intrinsics and extrinsics are given for the stereo image pair, then calibration is not necessary. If they are not given, but photos of a checkerboard or some other calibration object are provided, then the calibration parameters can be calculated. !!!!!!!!!!!!!!!!!!!!TALK ABOUT CALIBRATED RECTIFICATION HERE??????!!!!!!!!!!!!!!1

If not enough camera parameters are given and there are no checkerboard images to be used for calibration, then the following method can be used to accomplish

• Detect SURF keypoints(scale and rotation invariant feature detector) in each stereo image and extract feature vectors
• Find matching keypoints using sum of absolute differences (SAD)
• Remove outliers (incorrect matches) using epipolar constraint
• Compute 2-D projective geometric transformation from distance between remaining matches
• Apply transformation to get rectified images

Figure 3. Epipolar

Results

Sum of Squared Differences

Sum of Absolute Difference

• Performance with default parameters

Reference image

Disparity Map without semi-global matching

Disparity Map with semi-global matching

• Effect of Block Size and Smoothing

Census Transformation

Disparity Map from Census Transformation

Appendix II

You can write math equations as follows: $y=x+5$

You can include images as follows (you will need to upload the image first using the toolbox on the left bar.):

caption