BellMoonVassiliev

Back to Psych 221 Projects 2013

Background

In many image processing applications, it is desirable to combine a series of images of the same scene in order to acheive a higher signal-to-noise ratio (SNR), higher resolution, or both. This can be done with a set of still frames or from a video sequence.

Work on super-resolution has focused primarily on grayscale data. There are several challenges to working with RGB data. The image channels should remain aligned, so that colors are not distorted. This is typically handled by splitting the image into luminance and chrominance channels, and performing the operations in that space [TODO: cite].

When these operations are performed after demosaicking, errors from the demosaicking process are treated as "signal" in the following steps. By performing demosaicking jointly on a set of images, we are able to reduce the color artifacts introduced by the Bayer pattern.

Methods

Our approach consists of an image alignment step, which finds the relative horizontal and vertical offsets between the LR images, and a demosaicking step which combines the data from LR images into a single HR image.

Image Alignment

• Used phase correlation
• Fit parabola to get subpixel resolution
• Do least-squares fit to find a globally-consistent set of offsets

Demosaicking

Principles from single-image demosaicing

Before directly working with multiple images for demosaicing, we first apply our demosaicing algorithms to a single image to figure out how effective it is within a single-image framework. Through this experiment, we expected to get some ideas of how to extend this algorithms to multi-frame images.

Our algorithm is executed in two steps.

1. Interpolate Green channel by using 2nd gradient information of Red and Blue channels.

2. Interpolate Red and Blue by using inter color correlation parameters, Red-to-Green and Blue-to-Green.

Since red and blue channels are down-sampled two times more than the green channel, it's reasonable to interpolate green channel first then red and blue channel with the interpolated green channel. The simplest approach would be to estimate the unknown pixel values by linear interpolation. However, this approach will ignore important information about the correlation between color channels and will cause severe color artifacts. Anther critical drawback of linear interpolation is that uncorrelated pixels across edges will be averaged and will result in blurred edges.

Conclusions

Here is where you say what your results mean.

References - Resources and related work

References

[1] S. Farsiu, M. Elad, and P. Milanfar, Multi-Frame Demosaicing and Super-Resolution of Color Images, IEEE Trans. on Image Processing, vol. 15, no. 1, pp. 141-159, January 2006.

[2] E. P. Bennett, M. Uyttendaele, C. L. ZIitnick, R. Szeliski, and S.B. Kang, Video and image Bayesian demosaicing with a two color image prior. In Proceedings of the European Conference on Computer Vision (ECCV’06). Lecture Notes in Computer Science, vol. 3951. 508–521, 2006

Software

Appendix I - Code and Data

Code

All of our code is available on GitHub: [1]

Data

TODO

Appendix II - Work partition

TODO