Hsu

Back to Psych 221 Projects 2013

Background

While images files are usually represented in three color channels, digital cameras typically only have one image sensor. For the sensor to capture three different light wavelengths, a color filter array(CFA) is placed in front of the image sensor so that each pixel is only allowed to capture a narrow spectrum of light. The most common type of CFA is the Bayer filter (need figure), which is divided between red, green, and blue light. This work assumes that all test images employ the Bayer filter.

CFA Interpolation

After the image is captured by the sensor, the channels are interpolated into a full three-channel color images.

Discuss interpolation techniques here.

Effect of Tampering on CFA Interpolation

The section above shows that CFA interpolation introduces regular patterns into interpolated image. These regular patterns can be used to detect tampering of an image because the frequency spectrum of the interpolated pixels will be changed if a section of the image is altered.

Methods

The expectation-maximization algorithm calculated an estimation of the linear model used to interpolate the Bayer array and generated the per-pixel probability that each sample pixel belonged to the calculated linear interpolation filter (need a reference). The probability map was then analyzed for periodic frequencies to determine whether or not the image had been tampered.

Expectation-Maximization Algorithm

Expectation

The expectation step of the algorithm estimates the per-pixel probability that the pixel belongs to the estimated linear model, α. Each color channel is described as

$f(x,y)=\sum _{u,v=-N}^N{\alpha }_{u,v}f(x+u,y+v)+n(x,y)$

where $n(x,y)=f(x,y)-\sum _{u,v=-N}^N{\alpha }_{u,v}f(x+u,y+v)$ is the residual error between the actual image and the estimated image. The residual error is approximated using a Gaussian function with zero mean and $\alpha$ standard deviation. The probability that each pixel is taken from the estimated linear model is calculated using the probability distribution function of the residual errors:

$P(x,y)=\frac{1}{\sigma \sqrt{2\pi }}exp[-\frac{1}{2{\sigma }^2}{(f(x,y)-\sum _{u,v=-N}^N{\alpha }_{u,v}f(x+u,y+v))}^2]$

Using this probability, the per-pixel probability that the image is estimated by the proposed $\alpha$ is calculated using Bayes' rule. In these calculations, only the proposed linear model and an unspecified non-linear model were considered as possibilities. The non-linear model M_{2} was estimated to be one over 255;

$\mathrm{Pr}\{𝑓\mathit{(}𝑥,𝑦\mathit{)}\in M_1\mid 𝑓\mathit{(}𝑥,𝑦\mathit{)}\}=\frac{\mathrm{Pr}\{𝑓\mathit{(}𝑥,𝑦\mathit{)}\mid 𝑓\mathit{(}𝑥,𝑦\mathit{)}\in M_1\}\mathrm{Pr}\{𝑓\mathit{(}𝑥,𝑦\mathit{)}\in M_1\}}{\sum _{i=1}^2\mathrm{Pr}\{𝑓\mathit{(}𝑥,𝑦\mathit{)}\mid 𝑓\mathit{(}𝑥,𝑦\mathit{)}\in M_i\}\mathrm{Pr}\{𝑓\mathit{(}𝑥,𝑦\mathit{)}\in M_i\}}$

Maximization

The maximization step of the algorithm computes a revised α based on minimization of the residual error. The error function that is minimized is:

equation here

Minimization of the residual error is calculated by setting the partial derivative of the error function with respect to each element of α to zero. The resulting system of linear equations can be solved for a new α estimation. The algorithm stops when the difference between subsequent α arrays is sufficiently low.

Tamper Detection

Measure of Similarity

Tamper detection was implemented by calculating the cosine similarity between an ideal probability map and the estimated probability map. The similarity function is given by:

$M(p,s)=\sum _{w_x,w_y}P\times S$

where P} is the Fourier transform of the calculated probability map and S is the Fourier transform of the ideal probability map for each channel. The similarity between the ideal and calculated probability maps determines whether or not the image has been modified; high similarity indicates that the given CFA interpolation of the image can be approximated using a linear array since the frequency content of the probability map will be close to ideal. In contrast, tampering changes the frequency content of the probability map and the frequencies between the ideal and real case will result in a lower similarity measure.

Show FFTs of ideal maps

Windowing

Given that the tampered areas in the training images were small areas of the main image, a sliding window was used to calculate the measure of similarity for segmented blocks of the image. This prevented the slight spatial frequency differences in the probability map caused by tampering to be overshadowed by the regular frequency in the non-tampered parts of the image. The windows with the smallest measure of similarity are identified as tampered regions.

Threshold

Ideally, a minimum threshold for the measure of similarity can be determined to automatically detect tampered regions. Windows that have a measure of similarity below a certain threshold would be considered tampered images. Unfortunately, the threshold was difficult to determine due to:

1. Variation in frequency content between windows
2. Variation in frequency content between images
3. Low frequency content of uniform color patches in images

The results section will discuss alternatives to detection based on frequency spectrum.

Conclusions

- Buggy areas: places with large areas of pixels with the same pixel so that the probability map shows the same value for a large swath - Automation: normalization between different images - Use information about the CFA to determine tampering (i.e. threshold adjustment done based on CFA interpolation technique - Classification using frequency information between channels instead of processing them differently Could give better estimate in the case of CFA interpolators across multiple channels or get better estimate of frequency across different channels

References - Resources and related work

References

Software

Appendix I - Code and Data

Code

File:CodeFile.zip

Data

zip file with my data