IOS app for programmable camera

Introduction

Programmable cameras

A programmable camera is a portable digital camera which can be controlled using a smartphone or any Wi-Fi enabled remote device. These cameras have a fixed image sensor, but can have different lenses mounted on it. Today there are a lot of APIs being made available for the developers to integrate smartphones (Android and iOS) with these Wi-Fi enabled Digital cameras. The objective of this project was to use a prototype programmable camera and build an application on an iPhone which can make it easier for users to create cool effects on photos they have taken.

Smartphone Application

Users can take great pictures with these digital cameras by changing properties like ISO, Exposure, F-Number, white balance, etc. Unfortunately, the users need prior knowledge of photography to operate these cameras properly. Besides this, there is no preview available, when the user changes the properties of the camera and these cameras are low on processing memory.

Flowchart showing the phone acting as a controller to the Programmable camera

Our team decided to therefore, leverage the processing capability of the smartphone and create an application that would provide post-processing capabilities. We decided to implement real-time filters for the camera so that the users can see the effect of the filters applied on the screen before capturing the picture. The picture above shows how the smartphone acts as a controlling interface for the camera.

Methodology

Histogram and histogram equalization

Image Histograms: Histograms are graphical representations of tonal and luminance distributions in an image. Inspecting histograms of live preview in a camera can help the photographers to adjust the exposure values of their cameras. Also, the RGB histograms can be used as a guide to selectively adjust exposure for different colors. You can make sure that you are not losing details by underexposing or overexposing certain colors. That's why we incorporated the live histogram feature into our app. The GPUImage library provides APIs by which histograms of images can be extracted. We used the APIs GPUImageHistogramFilter and GPUImageHistogramGenerator to extract a histogram and plot it. After extracting the histogram image, we blend it into the live preview image using GPU blend filter. The screenshots are attached.

Histogram equalization and histogram matching: Equalizing a histogram is a technique that is used to adjust pixel intensities to enhance contrast. It's a transformation by which the image pixel intensities (often ranging from 0 - 256) are normalized and the image is transformed to an image with a linear CDF of gray levels. Since CDF can be inverse transformed, the intensities can be got back from this linearized CDF.

In a grayscale image or a monotone image, this works very well and results in better exposure of underexposed parts. Both in live images and when applied to stored images, grayscale equalization is shown to result in improving the details of the image. We could not find histogram equalization APIs in GPUIMage library. So, we used the Accelerate framework from iOS to implement histogram equalization.

Equalizing the histogram on the live image preview is useful tool for the user to adjust exposure and lighting for the scene that he is shooting.

In a color image however, this equalization needs to be repeated for the three color channels R,G and B separately. However, applying the same levels in all bins for the three channels results in color distortion and the tonal distribution of the image is messed up. This is because rarely does any image have equal intensities in all the three color channels.

We show an example of a distorted color image when the R, G and B histogram is equalized.

The same images with their RGB histograms blended over the images are as shown.

There are many RGB equalization techniques in research and in practice. In future, we plan to incorporate one technique which is called the histogram matching technique in our camera app. This will allow the user to specify a reference photo and ask the camera to match the histogram of the current view to the specified image’s histogram. This will be useful for user’s to mimic the appearance of good photos.