WenkeZhangChien-YiChang - Revision history

imported>Wandell: /* Lense Shading */

2020-12-07T22:24:20Z

Lense Shading

← Older revision		Revision as of 22:24, 7 December 2020
Line 17:		Line 17:
	=== Camera Modeling ===		=== Camera Modeling ===

	==== ~~Lense~~ Shading ====		==== Lens Shading ====

	Lens shading, also known as lens vignetting, refers to the phenomenon where the light falls off towards the edges of an image when captured by digital camera systems. Lens shading occurs because in the imaging system with an optical lens, the off-axis pixels often receive a smaller bundle of incident rays compared to on-axis pixels, resulting in relatively lower illumination towards the edges. This effect is ubiquitous and not surprisingly can be observed in the data we collected for the Cornell box lighting estimation.		Lens shading, also known as lens vignetting, refers to the phenomenon where the light falls off towards the edges of an image when captured by digital camera systems. Lens shading occurs because in the imaging system with an optical lens, the off-axis pixels often receive a smaller bundle of incident rays compared to on-axis pixels, resulting in relatively lower illumination towards the edges. This effect is ubiquitous and not surprisingly can be observed in the data we collected for the Cornell box lighting estimation.

imported>Student221: /* Conclusions */

2020-12-07T11:15:48Z

Conclusions

← Older revision		Revision as of 11:15, 7 December 2020
Line 165:		Line 165:
	In this project, we use image systems simulation software (specifically, ISET3D) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial model from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.		In this project, we use image systems simulation software (specifically, ISET3D) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial model from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.

	We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may ~~think about~~		We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may want to figure out,

	* Is there misuse of ISET softwares?		* Is there misuse of ISET softwares?

imported>Student221: /* Conclusions */

2020-12-07T11:15:04Z

Conclusions

← Older revision		Revision as of 11:15, 7 December 2020
Line 173:		Line 173:
	* Will a black box model works better than replicating physical scene?		* Will a black box model works better than replicating physical scene?

	* Depends on the goal of this project, is it possible to change the physical light source ~~to get convergence between the two~~?		* Depends on the goal of this project, is it possible to change the physical light source?

	== Reference ==		== Reference ==

imported>Student221 at 10:10, 7 December 2020

2020-12-07T10:10:01Z

← Older revision		Revision as of 10:10, 7 December 2020
Line 1:		Line 1:
	== Introduction ==		== Introduction ==
	The project is to use the graphics model of the Cornell box in ~~ISET3d~~ to simulate the illumination in the box. The project is to adjust the model parameters of the lighting so that the rendered image of the Cornell box matches the relative illumination map measured with the Google Pixel 4a camera. The project was done by Wenke Zhang and Chien-Yi Chang.		The project is to use the graphics model of the Cornell box in ISET3D to simulate the illumination in the box. The project is to adjust the model parameters of the lighting so that the rendered image of the Cornell box matches the relative illumination map measured with the Google Pixel 4a camera. The project was done by Wenke Zhang and Chien-Yi Chang.

	== Background ==		== Background ==

	In the mid-1980s, researchers at Cornell University compared photographs of a real physical box with computer graphics renderings of the simulated box. It has since been widely used by the computer vision and graphics community for calibrations and other research purposes. The goal of this project is to compare digital camera images of a real physical box with predicted camera images of the simulated box. In particular, we want to adjust the search for parameters of the light model in ~~ISET3d~~ so that the simulated illumination map matches the real image captured by the Google camera. Two steps are required to make the match. The first is to derive a relative illuminant map from the empirical data. We captured images with different exposure times and these can be combined into a single high dynamic range representation. The linear values from the camera, corrected for lens shading, are the target relative illumination map. We will have these images from a few different camera positions. We will use the 3D model of the Cornell box, the spectral radiance of the light, and the reflectance of the white surfaces, to simulate the relative illuminant map in ~~ISET3d~~. That code produces the spectral irradiance, and we can summarize the spectral irradiance with a single overall value by, say, simulating capture with a monochrome sensor. After that, we can study the effect of lighting parameter models and experiment with different conditions (e.g. type, position, properties) to make the relative illuminant map matches the real as close as possible.		In the mid-1980s, researchers at Cornell University compared photographs of a real physical box with computer graphics renderings of the simulated box. It has since been widely used by the computer vision and graphics community for calibrations and other research purposes. The goal of this project is to compare digital camera images of a real physical box with predicted camera images of the simulated box. In particular, we want to adjust the search for parameters of the light model in ISET3D so that the simulated illumination map matches the real image captured by the Google camera. Two steps are required to make the match. The first is to derive a relative illuminant map from the empirical data. We captured images with different exposure times and these can be combined into a single high dynamic range representation. The linear values from the camera, corrected for lens shading, are the target relative illumination map. We will have these images from a few different camera positions. We will use the 3D model of the Cornell box, the spectral radiance of the light, and the reflectance of the white surfaces, to simulate the relative illuminant map in ISET3D. That code produces the spectral irradiance, and we can summarize the spectral irradiance with a single overall value by, say, simulating capture with a monochrome sensor. After that, we can study the effect of lighting parameter models and experiment with different conditions (e.g. type, position, properties) to make the relative illuminant map matches the real as close as possible.

	== Methods ==		== Methods ==
Line 10:		Line 10:
	=== Overview ===		=== Overview ===

	Before moving into details, we first discuss the necessary data to create the model using ~~ISET3d~~. The inputs are i) a 3D model in PBRT of the Cornell box. ii)A mat-file SPD data for light source spectral radiance. iii) The reflectance of the white surface at the interior of the box.		Before moving into details, we first discuss the necessary data to create the model using ISET3D. The inputs are i) a 3D model in PBRT of the Cornell box. ii)A mat-file SPD data for light source spectral radiance. iii) The reflectance of the white surface at the interior of the box.
	Note that these empirical data are provided to estimate the box illumination in Files/Data/Lighting estimation and Files/Data/Lighting estimation (PRNU_DSNU.zip file for lens shading estimation). In addition to these, high dynamic range sensor images (DNG) are obtained using exposure bracketing of the Cornell box with the lighting and white interior. The images will be obtained from different camera positions. We are also provided with camera lens shading data that can be used to calculate the lens shading effect which will be discussed in the following sections. The DNG sensor images are stored in 5 .zip files from 5 different views. They are 'view1.zip' - 'view5.zip'. Each zip file contains a fileinfo.xlsx (Excel) file which summarizes the ISO speed and exposure duration. lightImg.zip file contains several DNG sensor images of the light from the view right below the light window. This file may be helpful for light shape determination. Light spectrum and surface reflectance data are stored in the spectralInfo.zip file.		Note that these empirical data are provided to estimate the box illumination in Files/Data/Lighting estimation and Files/Data/Lighting estimation (PRNU_DSNU.zip file for lens shading estimation). In addition to these, high dynamic range sensor images (DNG) are obtained using exposure bracketing of the Cornell box with the lighting and white interior. The images will be obtained from different camera positions. We are also provided with camera lens shading data that can be used to calculate the lens shading effect which will be discussed in the following sections. The DNG sensor images are stored in 5 .zip files from 5 different views. They are 'view1.zip' - 'view5.zip'. Each zip file contains a fileinfo.xlsx (Excel) file which summarizes the ISO speed and exposure duration. lightImg.zip file contains several DNG sensor images of the light from the view right below the light window. This file may be helpful for light shape determination. Light spectrum and surface reflectance data are stored in the spectralInfo.zip file.

Line 135:		Line 135:
	==== Simulation ====		==== Simulation ====

	For 3D Scene, input includes: 3D model of the Cornell box exported from cinemas 4d that store the geometry information of the scene, measured the spectral radiance of the light and the reflectance of the white surfaces at the interior of the box. In the 3D model, it has the Cornell box and each camera position with the same parameters as how we taking images. On the top of the window, we test different ways to represent the window and light source, determined by the parameters measured from physical Cornell box and DNG sensor images of the light from the view right below the light window. We then import the PBRT file into ~~ISET3d~~, edit the recipe, including setting the light spectrum to Box Lamp SPD, apply the material with diffuse reflection color of the walls. Then fine tuning the properties of different light types: spot, area and point light. To simulate the relative illuminant map in ~~ISET3d~~, we render the scene from PBRT file, it produces the spectral irradiance, then summarize the spectral irradiance with a single overall value by simulating capture with a monochrome sensor, derive the relative illuminant map by subsample the pixels at the same position as the same color pattern in the DNG file.		For 3D Scene, input includes: 3D model of the Cornell box exported from cinemas 4d that store the geometry information of the scene, measured the spectral radiance of the light and the reflectance of the white surfaces at the interior of the box. In the 3D model, it has the Cornell box and each camera position with the same parameters as how we taking images. On the top of the window, we test different ways to represent the window and light source, determined by the parameters measured from physical Cornell box and DNG sensor images of the light from the view right below the light window. We then import the PBRT file into ISET3D, edit the recipe, including setting the light spectrum to Box Lamp SPD, apply the material with diffuse reflection color of the walls. Then fine tuning the properties of different light types: spot, area and point light. To simulate the relative illuminant map in ISET3D, we render the scene from PBRT file, it produces the spectral irradiance, then summarize the spectral irradiance with a single overall value by simulating capture with a monochrome sensor, derive the relative illuminant map by subsample the pixels at the same position as the same color pattern in the DNG file.

	In Figure 10. (a) shows 3D model to simulate the three LED lights as spot light on the top of the window and pointing in the direction of the surfaces. (b) shows the relative illuminant map, it resembles the highlight on the wall and floor.		In Figure 10. (a) shows 3D model to simulate the three LED lights as spot light on the top of the window and pointing in the direction of the surfaces. (b) shows the relative illuminant map, it resembles the highlight on the wall and floor.
Line 163:		Line 163:
	== Conclusions ==		== Conclusions ==

	In this project, we use image systems simulation software (specifically, ~~ISET3d~~) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial model from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.		In this project, we use image systems simulation software (specifically, ISET3D) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial model from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.

	We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may think about		We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may think about

imported>Student221 at 09:41, 7 December 2020

2020-12-07T09:41:33Z

← Older revision		Revision as of 09:41, 7 December 2020
Line 121:		Line 121:


	==== ~~Real Scene~~ ====		==== Empirical Data ====

	To derive relative illuminant map from the empirical data, we captured images with different exposure times and these can be combined into a single high dynamic range representation. The linear values from the camera, corrected for lens shading, are the target relative illumination map. We have these images from a few different camera positions.		To derive relative illuminant map from the empirical data, we captured images with different exposure times and these can be combined into a single high dynamic range representation. The linear values from the camera, corrected for lens shading, are the target relative illumination map. We have these images from a few different camera positions.
Line 133:		Line 133:
	[[File:hdr-views.png\|1200px\|center\|thumb\|'''Figure 9''': Relative illuminant map of HDR image corrected for lens shading.]]		[[File:hdr-views.png\|1200px\|center\|thumb\|'''Figure 9''': Relative illuminant map of HDR image corrected for lens shading.]]

	==== ~~3D Scene~~ ====		==== Simulation ====

	For 3D Scene, input includes: 3D model of the Cornell box exported from cinemas 4d that store the geometry information of the scene, measured the spectral radiance of the light and the reflectance of the white surfaces at the interior of the box. In the 3D model, it has the Cornell box and each camera position with the same parameters as how we taking images. On the top of the window, we test different ways to represent the window and light source, determined by the parameters measured from physical Cornell box and DNG sensor images of the light from the view right below the light window. We then import the PBRT file into ISET3d, edit the recipe, including setting the light spectrum to Box Lamp SPD, apply the material with diffuse reflection color of the walls. Then fine tuning the properties of different light types: spot, area and point light. To simulate the relative illuminant map in ISET3d, we render the scene from PBRT file, it produces the spectral irradiance, then summarize the spectral irradiance with a single overall value by simulating capture with a monochrome sensor, derive the relative illuminant map by subsample the pixels at the same position as the same color pattern in the DNG file.		For 3D Scene, input includes: 3D model of the Cornell box exported from cinemas 4d that store the geometry information of the scene, measured the spectral radiance of the light and the reflectance of the white surfaces at the interior of the box. In the 3D model, it has the Cornell box and each camera position with the same parameters as how we taking images. On the top of the window, we test different ways to represent the window and light source, determined by the parameters measured from physical Cornell box and DNG sensor images of the light from the view right below the light window. We then import the PBRT file into ISET3d, edit the recipe, including setting the light spectrum to Box Lamp SPD, apply the material with diffuse reflection color of the walls. Then fine tuning the properties of different light types: spot, area and point light. To simulate the relative illuminant map in ISET3d, we render the scene from PBRT file, it produces the spectral irradiance, then summarize the spectral irradiance with a single overall value by simulating capture with a monochrome sensor, derive the relative illuminant map by subsample the pixels at the same position as the same color pattern in the DNG file.

imported>Student221: /* Conclusions */

2020-12-07T09:40:52Z

Conclusions

← Older revision		Revision as of 09:40, 7 December 2020
Line 171:		Line 171:
	* Are there important physical properties missing from the current models? we didn't fully understand like objects like diffuser, which could affect light distribution. Would parameter search of lighting properties help?		* Are there important physical properties missing from the current models? we didn't fully understand like objects like diffuser, which could affect light distribution. Would parameter search of lighting properties help?

	* ~~Would~~ a black box model works better than replicating physical scene?		* Will a black box model works better than replicating physical scene?

	* Depends on the goal of this project, is it possible to change the physical light source to get convergence between the two?		* Depends on the goal of this project, is it possible to change the physical light source to get convergence between the two?

imported>Student221: /* Conclusions */

2020-12-07T09:40:13Z

Conclusions

← Older revision		Revision as of 09:40, 7 December 2020
Line 167:		Line 167:
	We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may think about		We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may think about

	Is there misuse of ISET softwares?		* Is there misuse of ISET softwares?

	Are there important physical properties missing from the ~~model~~? we didn't fully understand like objects like diffuser, which could affect light distribution.		* Are there important physical properties missing from the current models? we didn't fully understand like objects like diffuser, which could affect light distribution. Would parameter search of lighting properties help?

	Would ~~parameter search of lighting properties help~~?		* Would a black box model works better than replicating physical scene?

	~~Would a black box model works better than replicating physical scene?~~		* Depends on the goal of this project, is it possible to change the physical light source to get convergence between the two?

	Depends on the goal of this project, is it possible to change the physical light source to get convergence between the two?

	== Reference ==		== Reference ==

imported>Student221: /* Conclusions */

2020-12-07T09:38:03Z

Conclusions

← Older revision		Revision as of 09:38, 7 December 2020
Line 163:		Line 163:
	== Conclusions ==		== Conclusions ==

	In this project, we use image systems simulation software (specifically, ISET3d) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial ~~models~~ from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.		In this project, we use image systems simulation software (specifically, ISET3d) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial model from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.

	We experimented with 3D models derived from physical model, the simulation is related to the camera images. There are simplification we made in each case that could lead to the difference in relative illuminant map.		We experimented with 3D models derived from physical model, the simulation is related but not the same as the camera images. There are simplification we made in each case that could lead to the difference in the relative illuminant map. For the next step, we may think about

			Is there misuse of ISET softwares?

			Are there important physical properties missing from the model? we didn't fully understand like objects like diffuser, which could affect light distribution.

			Would parameter search of lighting properties help?

	Would a black box model works better than replicating physical scene?		Would a black box model works better than replicating physical scene?

	~~Are there important physical properties missing from the model? There are objects we didn't fully understand like diffuser, which could affect light distribution.~~		Depends on the goal of this project, is it possible to change the physical light source to get convergence between the two?

	Depends on the goal of this project, is it possible to change the physical light source ~~during~~ the ~~next iterations~~?

	== Reference ==		== Reference ==

imported>Student221: /* Conclusions */

2020-12-07T09:32:11Z

Conclusions

← Older revision		Revision as of 09:32, 7 December 2020
Line 162:		Line 162:

	== Conclusions ==		== Conclusions ==

			In this project, we use image systems simulation software (specifically, ISET3d) to create different models for the spatial distribution of light with the simulated Cornell Box. These models are used to predict the amount of light reflected from the walls of the box and from surfaces within the box. We build polynomial models from camera lens shading data to correct HDR images from the lens shading effect, and compare predicted spectral radiance with measurements of spectral radiance obtained.

			We experimented with 3D models derived from physical model, the simulation is related to the camera images. There are simplification we made in each case that could lead to the difference in relative illuminant map.

			Would a black box model works better than replicating physical scene?

			Are there important physical properties missing from the model? There are objects we didn't fully understand like diffuser, which could affect light distribution.

			Depends on the goal of this project, is it possible to change the physical light source during the next iterations?

	== Reference ==		== Reference ==

imported>Student221: /* Appendix */

2020-12-07T08:55:38Z

Appendix

← Older revision		Revision as of 08:55, 7 December 2020
Line 173:		Line 173:

	== Appendix ==		== Appendix ==

	~~[https://stanford.zoom.us/rec/play/3dXfZQc_LYT8ZurtlmswSCLWwSvuz2zmEfIha_kvKN4yYXWg54L4zuTYCDs3SJRSzQTeaxYIe6rDJMg.zcilxHmSXeaiRr4s Image systems simulation for camera design and evaluation]~~

	[https://canvas.stanford.edu/courses/124940/pages/cornell-box-lighting-estimation Psych221 Cornell Box Lighting Estimation project page]		[https://canvas.stanford.edu/courses/124940/pages/cornell-box-lighting-estimation Psych221 Cornell Box Lighting Estimation project page]