Ray Tracing with Neural Networks - Revision history

Kminseo: /* Dataset Generation using Zemax */

2024-12-13T03:33:03Z

Dataset Generation using Zemax

← Older revision		Revision as of 03:33, 13 December 2024
Line 9:		Line 9:
	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.

	[[File:~~Zemax~~.png\|500px\|center]]		[[File:Macro.png\|500px\|center]]

	===RTM vs. RTF===		===RTM vs. RTF===

Kminseo: /* Dataset Generation using Zemax */

2024-12-13T03:32:13Z

Dataset Generation using Zemax

← Older revision		Revision as of 03:32, 13 December 2024
Line 9:		Line 9:
	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.

	[[File:Zemax.png\|~~300px~~\|center]]		[[File:Zemax.png\|500px\|center]]

	===RTM vs. RTF===		===RTM vs. RTF===

Kminseo: /* Dataset Generation using Zemax */

2024-12-13T03:32:03Z

Dataset Generation using Zemax

← Older revision		Revision as of 03:32, 13 December 2024
Line 9:		Line 9:
	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.

	[[File:Zemax.png\|~~500px~~]]		[[File:Zemax.png\|300px\|center]]

	===RTM vs. RTF===		===RTM vs. RTF===

Kminseo at 03:29, 13 December 2024

2024-12-13T03:29:50Z

← Older revision		Revision as of 03:29, 13 December 2024
Line 8:		Line 8:

	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.

			[[File:Zemax.png\|500px]]

	===RTM vs. RTF===		===RTM vs. RTF===

Kminseo: /* Dataset Generation using Zemax */

2024-12-13T03:29:07Z

Dataset Generation using Zemax

← Older revision		Revision as of 03:29, 13 December 2024
Line 7:		Line 7:
	The lens is reversed in Zemax to enable tracing rays from the sensor toward the object space. A dataset of rays is generated using a Zemax macro that samples the pupil at various field heights. The macro ensures uniform sampling in the polar coordinate system by increasing the number of angular samples linearly with radial distance. The ray-pass function defines the input and output planes, which determines whether the input rays will pass through the lens or get vignetted. The function is based on the shape of the pupil at different field heights and can be represented using ellipses or circle intersections.		The lens is reversed in Zemax to enable tracing rays from the sensor toward the object space. A dataset of rays is generated using a Zemax macro that samples the pupil at various field heights. The macro ensures uniform sampling in the polar coordinate system by increasing the number of angular samples linearly with radial distance. The ray-pass function defines the input and output planes, which determines whether the input rays will pass through the lens or get vignetted. The function is based on the shape of the pupil at different field heights and can be represented using ellipses or circle intersections.

	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl. A screenshot of the Zemax macro script is shown below.

	===RTM vs. RTF===		===RTM vs. RTF===

Zonyul at 03:06, 13 December 2024

2024-12-13T03:06:46Z

← Older revision		Revision as of 03:06, 13 December 2024
Line 115:		Line 115:
	We will continuously investigate this problem.		We will continuously investigate this problem.
	== Conclusions ==		== Conclusions ==
	We implemented and evaluated various machine learning approaches to model the ray transfer function and classify whether a ray can successfully pass through the lens. For ray tracing, we found that incorporating positional encoding techniques helped improve the performance of these models. The machine learning models demonstrated enhanced performance compared to polynomial fitting employing second and fifth degree polynomial orders. However, they were unable to match the accuracy attained by the eighth degree polynomial fitting approach. For ray classification, our best model achieved an accuracy of approximately 94%, due to the very imbalanced data distribution. We will explore strategies to address this class imbalance and further improve the classification performance in future work.		We implemented and evaluated various machine learning approaches to model the ray transfer function and classify whether a ray can successfully pass through the lens.
			All data were simulated using Zemax under different wavelengths and optical properties (Double Gauss and Wide Angle).
			For ray tracing, we found that incorporating positional encoding techniques helped improve the performance of these models. The machine learning models demonstrated enhanced performance compared to polynomial fitting employing second and fifth degree polynomial orders. However, they were unable to match the accuracy attained by the eighth degree polynomial fitting approach. For ray classification, our best model achieved an accuracy of approximately 94%, due to the very imbalanced data distribution. We will explore strategies to address this class imbalance and further improve the classification performance in future work.
	== References ==		== References ==
	[1] Thomas Goossens, Zheng Lyu, Jamyuen Ko, Gordon C. Wan, Joyce Farrell, and Brian Wandell, "Ray-transfer functions for camera simulation of 3D scenes with hidden lens design," Opt. Express 30, 24031-24047 (2022).		[1] Thomas Goossens, Zheng Lyu, Jamyuen Ko, Gordon C. Wan, Joyce Farrell, and Brian Wandell, "Ray-transfer functions for camera simulation of 3D scenes with hidden lens design," Opt. Express 30, 24031-24047 (2022).

Zonyul at 03:04, 13 December 2024

2024-12-13T03:04:34Z

← Older revision		Revision as of 03:04, 13 December 2024
Line 110:		Line 110:

	===Remaining Issue: Imbalanced Dataset===		===Remaining Issue: Imbalanced Dataset===
			Given the imbalanced dataset, the best way to address this issue is to collect more data to create a more equally distributed dataset.
			However, within limited amount of time, we were unable to do so. Nevertheless, we believe a simple method (such as kernel SVM)
			can work well due to the decision boundary shown in the previous section.
			We will continuously investigate this problem.
	== Conclusions ==		== Conclusions ==
			We implemented and evaluated various machine learning approaches to model the ray transfer function and classify whether a ray can successfully pass through the lens. For ray tracing, we found that incorporating positional encoding techniques helped improve the performance of these models. The machine learning models demonstrated enhanced performance compared to polynomial fitting employing second and fifth degree polynomial orders. However, they were unable to match the accuracy attained by the eighth degree polynomial fitting approach. For ray classification, our best model achieved an accuracy of approximately 94%, due to the very imbalanced data distribution. We will explore strategies to address this class imbalance and further improve the classification performance in future work.
	== References ==		== References ==
	[1] Thomas Goossens, Zheng Lyu, Jamyuen Ko, Gordon C. Wan, Joyce Farrell, and Brian Wandell, "Ray-transfer functions for camera simulation of 3D scenes with hidden lens design," Opt. Express 30, 24031-24047 (2022).		[1] Thomas Goossens, Zheng Lyu, Jamyuen Ko, Gordon C. Wan, Joyce Farrell, and Brian Wandell, "Ray-transfer functions for camera simulation of 3D scenes with hidden lens design," Opt. Express 30, 24031-24047 (2022).

Kminseo: /* Polynomial Estimation of Ray-Transfer Functions */

2024-12-13T01:26:16Z

Polynomial Estimation of Ray-Transfer Functions

← Older revision		Revision as of 01:26, 13 December 2024
Line 36:		Line 36:
	The user must specify the order of polynomial (denoted by <math>k</math>), as reported in [1],		The user must specify the order of polynomial (denoted by <math>k</math>), as reported in [1],
	the fitting error varies as the polynomial degree changes (see the figure below).		the fitting error varies as the polynomial degree changes (see the figure below).

	[[File:PolyDegree.png\|500px]]		[[File:PolyDegree.png\|500px]]

	== Methods ==		== Methods ==
	===Network Architecture===		===Network Architecture===

Kminseo: /* Constructing the RTF */

2024-12-13T01:25:38Z

Constructing the RTF

← Older revision		Revision as of 01:25, 13 December 2024
Line 9:		Line 9:
	Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl		Additionally, to simplify the data collection process and reduce the amount of data needed, we exploit the rotational symmetry of most lenses. We sample rays radially along one axis and then rotate these rays around the optical axis to create a complete dataset. This means we only need to store data for rays with an x-coordinate of 0. All in all, the Zemax macro traces these sampled rays through the reversed lens and records input and output ray parameters. The output of the macro is a text file that contains the position and unit direction vectors of the input and output rays for each sample. Any input ray that fails to exit the lens is marked with 'NaN' values in the output which are discarded before the training process. Refer to this example Zemax macro script to generate training data for wide angle lens: https://github.com/ISET/mlp-tiny/blob/20241117/interface%20scripts/zemax_opticalsystemdata.zpl

	===~~Constructing the~~ RTF===		===RTM vs. RTF===

	[[File:Rtf.png\|500px]]		[[File:Rtf.png\|500px]]

Kminseo: /* Constructing the RTF */

2024-12-13T01:24:49Z

Constructing the RTF

← Older revision		Revision as of 01:24, 13 December 2024
Line 12:		Line 12:

	[[File:Rtf.png\|500px]]		[[File:Rtf.png\|500px]]

			The RTF extends the classic ABCD ray-transfer matrices (RTM) used for paraxial calculations. There are several advantages of using RTFs over RTMs.

			*While RTMs are suitable for paraxial ray tracing, they are limited in modeling aberrations and other non-linear optical phenomena. RTFs, with their polynomial or neural network representation, are better at handling these complexities.
			*RTFs can account for 3D effects like depth and occlusion, which are crucial for accurately simulating real-world scenes. Also, we can integrate the RTF into ray tracing software like PBRT to simulate complex 3D spectral scenes.

	===Polynomial Estimation of Ray-Transfer Functions===		===Polynomial Estimation of Ray-Transfer Functions===