NPorfilio
Back to Psych 204 Projects 2009
Background
Image quality is one of the most important considerations when deciding which camera to purchase. Image quality is affected by a camera’s internal mechanics but also by external factors.
With respect to a camera’s internal factors, image quality is affected by:
Lens: The lens transfers the visual information of the object to be depicted onto the plane of the image sensor.
Sensor: The sensor is an array of millions of tiny pixels that produce the final image. When you press your camera's shutter button and the exposure begins, each of these pixels has a photosite which is uncovered to collect and store photons in a cavity. Once the exposure finishes, the camera closes each of these photosites, and then tries to assess how many photons fell into each (Cambridge in Colour). Sharpness, distortion, vignetting, Lateral Chromatic Aberration, noise, and dynamic range are the principal factors that can be measured at this stage (Imatest).
Image processing pipeline: The image processing pipeline is a set of digital adjustments made to the pre-processed image taken from the lens and sensor, and then transferred into the pipeline. It typically includes demosaicing, color correction, white balance, application of gamma and tonal response curves, sharpening, and noise reduction. The output of the pipeline may be compared to the minimally-processed lens and sensor measurements. However, the effect of the pipeline on subjective image quality can be highly scene and application-dependent, making it difficult to assign objective rankings (Imatest).
Oftentimes, image quality is a function of camera price—many consumers expect a more expensive camera to deliver a higher quality image. But is this always the case? Does an expensive DSLR outperform a compact digital camera that is one-third its price? My project investigates two cameras on the market--the Nikon D5100 and the Sony Cyber-shot DSC-T700—using a subset of the factors discussed above and compares the quality of the image outputted from each.
Methods
I decided I needed both objective and subjective data to appropriately compare the two cameras. After all, even if all the quantitative measures say that one camera produces higher quality photos than another, it’s a moot point if the actual pictures do not look better to humans. Consequently, to get the quantitative data, I took a number of test shots with both cameras using the ISO 12233 and MacBeth ColorChecker and analyzed those in Imatest, a software package for testing digital image quality, on several different metrics discussed below. Then to get the subjective data, I took a couple of pictures of Memorial Church on Stanford University’s campus under a number of different lighting conditions. I then used all this data to compare the cameras and came up with my conclusions.
Modulation Transfer Function
The Modulation Transfer Function is the most important measure of device and image sharpness because it determines the amount of detail an image can convey. It is defined as the contrast at a given spatial frequency relative to low frequencies.
I measured this using the photo from each camera of the ISO 12233 chart and uploading the image to Imatest. Then I chose a region of interest to analyze. I choose a slanted edge in the middle of the chart as the region of interest. In order to compare MTF measurements across photos and cameras, Imatest uses the MTF50 metric, which is the spatial frequency where the MTF is 50% of the low frequency MTF measured in Line Widths/Picture Height. According to Imatest documentation, MTF50 is a good parameter for comparing the sharpness of different cameras and lenses for two reasons: (1) Image contrast is half its low frequency or peak values, hence detail is still quite visible. The eye is relatively insensitive to detail at spatial frequencies where MTF is low: 10% or less. (2) The response of most cameras falls off rapidly in the vicinity of MTF50.
To give this measurement context you divide it by some picture height of interest. For example, if you were interested to see if one camera produced a 5” x 7” photo with excellent sharpness, you would divide the calculated MTF50 by 5”. The resulting metric is then MTF50 measured in Line Widths/inch of the print and has the following breakpoints to determine the image sharpness:
150 and above – Excellent – “Extremely sharp at any viewing distance. About as sharp as most inkjet printers can print.”
110-150 – Very Good – “Large prints (A3 or 13×19 inch) look excellent, though they won’t look perfect under a magnifier. Small prints still look very good”
80-110 – Good – “Large prints look OK when viewed from normal distances, but somewhat soft when examined closely. Small prints look soft— adequate, perhaps, for the average consumer, but definitely not crisp."
Chromatic Aberration
Chromatic aberration (CA) is one of several aberrations that degrade lens performance. (Others include coma, astigmatism, and curvature of field.) It occurs because the index of refraction of glass varies with the wavelength of light, i.e., glass bends different colors by different amounts. This phenomenon is called dispersion. Minimizing chromatic aberration is one of the goals of lens design. It is accomplished by combining glass elements with different dispersion properties (Imatest). Lateral chromatic aberration also called “color fringing” is a lens aberration that causes colors to focus at different distances from the image center. It is most visible near corners of images.
The metric here is: CA (area) as a percentage of the distance to image center. This is the main chromatic aberration metric used in Imatest because it is relatively independent of the region of interest. This is important because CA tends to be proportional to the distance from the image center. Similar to the MTF, I used the ISO 12233 photos from each camera but this time chose a slanted edge away from the center as a region of interest.
CA (area) as a percentage of the distance to image center = 100 * (area between the channels with the highest and lowest CA levels) / (distance from center in pixels), corrected for the angle of the ROI.
The breakpoints are:
Under 0.04: Insignificant
0.04-0.08: Minor
0.08-0.15: Moderate
over 0.15: Serious
Delta-E
Delta-E is a represents the 'distance' between the color in a photo and the standardized CIELAB (L*a*b*) colors.
CIELAB standard color space is specified by the International Commission on Illumination. It describes all the colors visible to the human eye and was created to serve as a device independent model to be used as a reference. The three coordinates of CIELAB represent the lightness of the color (L* = 0 yields black and L* = 100 indicates diffuse white), its position between red/magenta and green (a*, negative values indicate green while positive values indicate magenta) and its position between yellow and blue (b*, negative values indicate blue and positive values indicate yellow).
I measured this using the photo from each camera of the Macbeth ColorCheck and uploading the image to Imatest. The greater away the calculated Delta-E is from 0, the more the color is away from the standard color.
Results - What you found
Modulation Transfer Function
The calculated MTF50:
Nikon: 1646
Sony: 1222
Chromatic Aberration
The calculated CA (area) as a percentage of distance to the center:
Nikon: 0.053% - Low
Sony: 0.031% - Insignificant
Delta-E
The mean Delta E:
Nikon 11.5
Sony: 12.1
Noise
Nikon has a higher noise level than the Sony at every exposure value.
Memorial Church
Daytime
Nighttime
Conclusions
Here is where you say what your results mean.
References - Resources and related work
[1] Dr. Karl Lenhardt, Bad Kreuznach. http://www.schneiderkreuznach.com/knowhow/opt_quali_e.htm
[2] Cambridge in Colour. http://www.cambridgeincolour.com/tutorials/camera-sensors.htm
[3] Wikipedia. http://en.wikipedia.org/wiki/Lab_color_space
Software
Imatest. http://www.imatest.com/
Appendix I - Code and Data
Presentation Slides
"Slides: An Analysis of Camera Image Quality: Nikon D5100 vs Sony Cyber-shot DSC-T700"