WanlingLiuQianDong

Introduction

The project is to study and find a robust machine learning method for blind motion deblurring. Motion blur happens when there is a relative motion between a camera and a scene during image exposure time. Reducing exposure time is a fairly straight-forward way to solve the problem, but the method can increase image noise significantly especially in dark scenes. Instead of doing it online, machine learning is able to learn and remove the blurring kernel offline without the kernel function information. We will apply different machine learning models, mainly focusing on CNN (Convolutional Neural Network) and GAN (Generative Adversarial Network), to solve the motion deblurring problem. By comparing their pros and cons, we will choose the best model with highest performance and robustness, and combine or modify the existing CNN and GAN models to better improve the quality of deblurred images. We calculated PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity) and MSE (Mean Square Error) to evaluate our results. We used the GOPRO dataset, which is widely used for training and testing the models for motion deblurring. The dataset can be found here.

Background

Image deblurring is an important task in the area of image processing and computer vision. The goal of image deblurring is to recover a sharp image for a blurry image caused by motions or out-of-focus. Different methods, including traditional methods and machine learning methods, have been proposed to solve the problem. Traditional methods need to model the characteristics of blur and often require setting up some constraints and some prior knowledge, which would be very complex and limited to simple situations. However, machine learning methods have a general structure and we do not need to explicitly specify information about the image. In this project, we applied machine learning models to deblur motion-blurred images.

There are two kinds of blurry images, one is blind images, knowing the sharp images while not knowing the source of blurring, the other one is non-blind, knowing both the sharp image and blurring source. We focus on the blind images in our project. To address the blurry image problem, CNN and GAN are commonly used. For example, this paper (Links to an external site.) used multi-layer perceptron (MLP) for non-blind images, this work (Links to an external site.) designed a deep multi-scale CNN for dynamic scene deblurring, and this repo (Links to an external site.) applied GAN for blind images.

Methods

GAN model

• cGAN structure

• Generator architecture

Image Quality Evaluation

• Metrics

The mean-square error (MSE) computes the pixel-based difference between two images which represents the cumulative squared error between the deblurred and the original image. A lower value of MSE represents a lower error and a high image quality.

$MSE=\frac{1}{mn}\sum _{i=0}^{m-1}\sum _{j=0}^{n-1}[I(i,j)-K(i,j){]}^2$

PSNR computes the peak signal-to-noise ratio between two images. The ratio representing a measure of the peak error can be used as a reference for image quality evaluation between the original and the deblurred image. An image with a higher PSNR has a higher image quality.

$PSNR=10{\log }_{10}(\frac{MA{X}_I^2}{MSE})$

SSIM is used for measuring the similarity between two images. SSIM is designed to improve on traditional methods such as peak signal-to-noise ratio (PSNR) and mean squared error (MSE).

$SSIM(x,y)=\frac{(2{\mu }_x{\mu }_y+C_1)+(2{\sigma }_{xy}+C_2)}{({\mu }_x^2+{\mu }_y^2+C_1)+({\sigma }_x^2+{\sigma }_y^2+C_2)}$

• Yolo object detection

Results

• GAN
• CNN

Appendix

You can write math equations as follows: $y=x+5$

You can include images as follows (you will need to upload the image first using the toolbox on the left bar, using the "Upload file" link).