LuppescuShah

Methods

Hole Filling

The pixels in the depth image with zero values were considered to be holes.

Mean filter-based Hole Filling

We implemented a basic hole-filling algorithm using a mean filter. A mean filter of size NxN updates the central pixels with the mean of the NxN neighborhood around the central pixel. However, directly applying a mean filter to the entire image updates not only the value of the holes but also the value of the pixels with valid depth values. Hence, we first find the pixels that correspond to holes i.e. have value = 0 and apply the mean filter to update values of only the pixels that correspond to the holes. Also, the neighborhood of a hole may have holes as well. For faster convergence, it is useful not to include the holes in the neighborhood while calculating the mean. Thus, for each pixel corresponding to a hole, this can be written mathematically as -

The hole-filling pipeline is given below -

Median filter-based Hole Filling

Hole-filling can also be implemented using the algorithm mentioned by using a median filter instead of a mean filter. A median filter updates the value of a pixel to the median of the values in a neighborhood around the pixel. Again, to speed-up the convergence, one should only consider the non-zero values to calculate the median. Thus, for each pixel corresponding to a hole, this can be written mathematically as -

The hole-filling pipeline using the median filter is given below -