Jim Best-Devereux - Revision history

imported>Student221: /* Appendix */

2020-11-21T04:08:09Z

Appendix

← Older revision		Revision as of 04:08, 21 November 2020
Line 77:		Line 77:

	Main Scripts Referenced for Work:		Main Scripts Referenced for Work:

	- s_sensorStackedPixels.m		- s_sensorStackedPixels.m

imported>Student221: /* Appendix */

2020-11-21T04:08:03Z

Appendix

← Older revision		Revision as of 04:08, 21 November 2020
Line 78:		Line 78:
	Main Scripts Referenced for Work:		Main Scripts Referenced for Work:
	- s_sensorStackedPixels.m		- s_sensorStackedPixels.m

	- t_sensorEstimation.m		- t_sensorEstimation.m

	[[File:Screen_Shot_2016-11-29_at_7.05.37_PM.png\|200px]]		[[File:Screen_Shot_2016-11-29_at_7.05.37_PM.png\|200px]]

imported>Student221 at 04:07, 21 November 2020

2020-11-21T04:07:46Z

@@ Line 61: / Line 61: @@
 == Conclusions ==
 While this project did not perfectly capture the filter spectra of the film types as hoped for, it does show the promise and the possibility of capturing the spectra that would output a matching image to what would have been captured using a color film. Additional constraints such as the solutions differentiability and continuity could be imposed onto the fitting in an attempt to force the solution to better fit the dynamics of the spectral-sensitivity of the film. In addition this work shows how once a filter spectra has been fit using training images of “film” photos how those can be applied to other scenes to attempt to create the capture those would have produced.
 == Appendix ==
 [[File:Screen_Shot_2016-11-29_at_7.05.37_PM.png|200px]]

imported>Student221: /* Results */

2020-11-21T04:02:26Z

Results

← Older revision		Revision as of 04:02, 21 November 2020
Line 40:		Line 40:
	Fitting with the normal equations we find spectra similar to those below. These are both noisy and chaotic, as well as falling outside of the acceptable bounds of spectral transmittances. While a simple solution at times can be the best approach, fitting from the input scene photons to the volts of the filtered image led to poor performance.		Fitting with the normal equations we find spectra similar to those below. These are both noisy and chaotic, as well as falling outside of the acceptable bounds of spectral transmittances. While a simple solution at times can be the best approach, fitting from the input scene photons to the volts of the filtered image led to poor performance.

	[[File:leastSquaresSpectra.png\|~~400px~~]]		[[File:leastSquaresSpectra.png\|700px]]

	An example of one of the rendered images where the normal equation filter spectra was used is as below, compared to the result hoped for.		An example of one of the rendered images where the normal equation filter spectra was used is as below, compared to the result hoped for.

imported>Student221 at 04:02, 21 November 2020

2020-11-21T04:02:11Z

← Older revision		Revision as of 04:02, 21 November 2020
Line 39:		Line 39:
	== Results ==		== Results ==
	Fitting with the normal equations we find spectra similar to those below. These are both noisy and chaotic, as well as falling outside of the acceptable bounds of spectral transmittances. While a simple solution at times can be the best approach, fitting from the input scene photons to the volts of the filtered image led to poor performance.		Fitting with the normal equations we find spectra similar to those below. These are both noisy and chaotic, as well as falling outside of the acceptable bounds of spectral transmittances. While a simple solution at times can be the best approach, fitting from the input scene photons to the volts of the filtered image led to poor performance.

			[[File:leastSquaresSpectra.png\|400px]]

	An example of one of the rendered images where the normal equation filter spectra was used is as below, compared to the result hoped for.		An example of one of the rendered images where the normal equation filter spectra was used is as below, compared to the result hoped for.

			[[File:leastSquaresRender.png\|400px]]

	Moving onto the non-linear least squares the filter spectra is at least now within the realm of what is allowed of filter spectra. By adding bounds to the predictions the filter spectra predictions will stay within the range [0,1] as it should, but the output spectra is still noisy and sporadic. One potential route for further work would be to add constraints to the differentiability of the solution so the output spectra has smoother curves that better match those of the film spectral-sensitivities seen above.		Moving onto the non-linear least squares the filter spectra is at least now within the realm of what is allowed of filter spectra. By adding bounds to the predictions the filter spectra predictions will stay within the range [0,1] as it should, but the output spectra is still noisy and sporadic. One potential route for further work would be to add constraints to the differentiability of the solution so the output spectra has smoother curves that better match those of the film spectral-sensitivities seen above.

			[[File:nonlinearSpectra.png\|400px]]

	While the computed volts from the sensor lacks some of the detail, in the rendered images it shows that the image is approaching that of the “film” image.		While the computed volts from the sensor lacks some of the detail, in the rendered images it shows that the image is approaching that of the “film” image.

			[[File:nonlinearRender.png\|400px]]


	When applied to another image from those downloaded from DxO Filmpack it is clear that there are still some large discrepancies. One note of interest is that it seems the red saturation in rendered image is too high. This red artifact seems to appear in just about all of the rendered images from the Foveon filter. It was questioned if this is a colorspace issue, but comparing outputs from both Foveon and Bayer arrays (with a Nikon D1 filter spectra) there does exist a 3x3 transform to perfectly convert one to the other so they do exist within the same colorspace and are only a linear transform apart.		When applied to another image from those downloaded from DxO Filmpack it is clear that there are still some large discrepancies. One note of interest is that it seems the red saturation in rendered image is too high. This red artifact seems to appear in just about all of the rendered images from the Foveon filter. It was questioned if this is a colorspace issue, but comparing outputs from both Foveon and Bayer arrays (with a Nikon D1 filter spectra) there does exist a 3x3 transform to perfectly convert one to the other so they do exist within the same colorspace and are only a linear transform apart.

			[[File:nonlinearTest.png\|400px]]

	== Conclusions ==		== Conclusions ==

imported>Student221: /* Results */

2020-11-21T04:00:17Z

Results

← Older revision		Revision as of 04:00, 21 November 2020
Line 38:		Line 38:
	After this a non-linear bounded least squares was used in an attempt to add more constraints on the values returned so they actually fell within the realm of possible solutions. Using this there was a noticeable improvement in performance in how well the two captures matched. While there was improvement it was still not a perfect match and the spectra is substantially noisier than what would be hoped for, but this shows the possibility in using more and more constrained fittings to try and produce the filter spectra desired.		After this a non-linear bounded least squares was used in an attempt to add more constraints on the values returned so they actually fell within the realm of possible solutions. Using this there was a noticeable improvement in performance in how well the two captures matched. While there was improvement it was still not a perfect match and the spectra is substantially noisier than what would be hoped for, but this shows the possibility in using more and more constrained fittings to try and produce the filter spectra desired.
	== Results ==		== Results ==
			Fitting with the normal equations we find spectra similar to those below. These are both noisy and chaotic, as well as falling outside of the acceptable bounds of spectral transmittances. While a simple solution at times can be the best approach, fitting from the input scene photons to the volts of the filtered image led to poor performance.

			An example of one of the rendered images where the normal equation filter spectra was used is as below, compared to the result hoped for.

			Moving onto the non-linear least squares the filter spectra is at least now within the realm of what is allowed of filter spectra. By adding bounds to the predictions the filter spectra predictions will stay within the range [0,1] as it should, but the output spectra is still noisy and sporadic. One potential route for further work would be to add constraints to the differentiability of the solution so the output spectra has smoother curves that better match those of the film spectral-sensitivities seen above.

			While the computed volts from the sensor lacks some of the detail, in the rendered images it shows that the image is approaching that of the “film” image.

			When applied to another image from those downloaded from DxO Filmpack it is clear that there are still some large discrepancies. One note of interest is that it seems the red saturation in rendered image is too high. This red artifact seems to appear in just about all of the rendered images from the Foveon filter. It was questioned if this is a colorspace issue, but comparing outputs from both Foveon and Bayer arrays (with a Nikon D1 filter spectra) there does exist a 3x3 transform to perfectly convert one to the other so they do exist within the same colorspace and are only a linear transform apart.

	== Conclusions ==		== Conclusions ==

imported>Student221 at 02:28, 21 November 2020

2020-11-21T02:28:43Z

← Older revision		Revision as of 02:28, 21 November 2020
Line 32:		Line 32:
	The images used for this are as below, with the original image on top and the two filtered versions of it below.		The images used for this are as below, with the original image on top and the two filtered versions of it below.

	[[File:~~fittingDiagram~~.png\|~~450px~~]]		[[File:orig&filmImages.png\|550px]]

	As a first pass the normal equation solution was employed to train and fit to the “film” spectra. While this is simple to implement there are several issues with it. Since least squares does not limit the range of possible values for the output the generated spectra does not obey the rules of being bound between [0,1]. In addition to this because of the structure of the problem and there being more input features in comparison to output features (there are more wavelengths of light measured in comparison to the three colors output for the RGB image) there are infinitely many solutions. Becuase of this the solution is unstable and can vary greatly between runs.		As a first pass the normal equation solution was employed to train and fit to the “film” spectra. While this is simple to implement there are several issues with it. Since least squares does not limit the range of possible values for the output the generated spectra does not obey the rules of being bound between [0,1]. In addition to this because of the structure of the problem and there being more input features in comparison to output features (there are more wavelengths of light measured in comparison to the three colors output for the RGB image) there are infinitely many solutions. Becuase of this the solution is unstable and can vary greatly between runs.

imported>Student221 at 02:28, 21 November 2020

2020-11-21T02:28:01Z

← Older revision		Revision as of 02:28, 21 November 2020
Line 27:		Line 27:

	In addition to this images were taken from DxO Filmpack to act as our baseline and comparison that the work aimed to fit. Using these images the following process was attempted to fit the new spectra that was needed to create the “feel” of film. Both original and film filtered images were loaded in as scenes, and from these the output volts that would be fed into the image processing pipeline were created. The output volts from the filtered image would be what we want to create from the transform of the original scene radiance and the “film” mimicking spectra. This is visualized as below.		In addition to this images were taken from DxO Filmpack to act as our baseline and comparison that the work aimed to fit. Using these images the following process was attempted to fit the new spectra that was needed to create the “feel” of film. Both original and film filtered images were loaded in as scenes, and from these the output volts that would be fed into the image processing pipeline were created. The output volts from the filtered image would be what we want to create from the transform of the original scene radiance and the “film” mimicking spectra. This is visualized as below.

			[[File:fittingDiagram.png\|450px]]

			The images used for this are as below, with the original image on top and the two filtered versions of it below.

	[[File:fittingDiagram.png\|450px]]		[[File:fittingDiagram.png\|450px]]

imported>Student221 at 02:26, 21 November 2020

2020-11-21T02:26:01Z

← Older revision		Revision as of 02:26, 21 November 2020
Line 15:		Line 15:
	Each kind of film has a specific set of characteristics much like the digital sensors covered in this class. Two examples of the curves for different types of film are as below. Kodak Portra 400 and Kodachrome 200 are very popular types of film that are still used extensively today.		Each kind of film has a specific set of characteristics much like the digital sensors covered in this class. Two examples of the curves for different types of film are as below. Kodak Portra 400 and Kodachrome 200 are very popular types of film that are still used extensively today.

	[[File:portra400.png\|~~399px~~]] [[File:kodachrome200.png\|~~399px~~]]		[[File:portra400.png\|500px]] [[File:kodachrome200.png\|500px]]

	As we can see the spectral-sensitivity curves are very different for the two types of film, and this leads to the difference in their color perception of the same scene. These curves are what we aim to simulate using our simulated Foveon sensor.		As we can see the spectral-sensitivity curves are very different for the two types of film, and this leads to the difference in their color perception of the same scene. These curves are what we aim to simulate using our simulated Foveon sensor.

imported>Student221 at 02:25, 21 November 2020

2020-11-21T02:25:41Z

← Older revision		Revision as of 02:25, 21 November 2020
Line 15:		Line 15:
	Each kind of film has a specific set of characteristics much like the digital sensors covered in this class. Two examples of the curves for different types of film are as below. Kodak Portra 400 and Kodachrome 200 are very popular types of film that are still used extensively today.		Each kind of film has a specific set of characteristics much like the digital sensors covered in this class. Two examples of the curves for different types of film are as below. Kodak Portra 400 and Kodachrome 200 are very popular types of film that are still used extensively today.

	[[File:portra400.png\|~~400px~~]] [[File:kodachrome200.png\|~~400px~~]]		[[File:portra400.png\|399px]] [[File:kodachrome200.png\|399px]]

	As we can see the spectral-sensitivity curves are very different for the two types of film, and this leads to the difference in their color perception of the same scene. These curves are what we aim to simulate using our simulated Foveon sensor.		As we can see the spectral-sensitivity curves are very different for the two types of film, and this leads to the difference in their color perception of the same scene. These curves are what we aim to simulate using our simulated Foveon sensor.