Identifying Model Weaknesses Using Image System Degradations - Revision history

Skhan44: /* Pixel Size */

2024-12-13T20:09:27Z

Pixel Size

← Older revision		Revision as of 20:09, 13 December 2024
Line 229:		Line 229:
	=== Pixel Size ===		=== Pixel Size ===

	The relative ranking of results for pixel size closely mirrors the results of exposure time. However, the change in model confidence for increasing pixel size was much more drastic than for exposure times. ~~This can perhaps be~~ interpreted as the change in pixel size degradation being a composition of blurring and increased exposure time.		The relative ranking of results for pixel size closely mirrors the results of exposure time. However, the change in model confidence for increasing pixel size was much more drastic than for exposure times. We interpreted this as the change in pixel size degradation being a composition of blurring and increased exposure time.
	{\| class="wikitable"		{\| class="wikitable"
	\|+ Ranking of Classes by Mean Drop for Pixel Size 8 µm and Exposure Time 0.01 s		\|+ Ranking of Classes by Mean Drop for Pixel Size 8 µm and Exposure Time 0.01 s

Skhan44 at 20:05, 13 December 2024

2024-12-13T20:05:19Z

← Older revision		Revision as of 20:05, 13 December 2024
Line 226:		Line 226:

	Indeed, improving the ResNet architecture in low-light scenarios appears to be a significant area of study [https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement(3)].		Indeed, improving the ResNet architecture in low-light scenarios appears to be a significant area of study [https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement(3)].

			=== Pixel Size ===

			The relative ranking of results for pixel size closely mirrors the results of exposure time. However, the change in model confidence for increasing pixel size was much more drastic than for exposure times. This can perhaps be interpreted as the change in pixel size degradation being a composition of blurring and increased exposure time.
			{\| class="wikitable"
			\|+ Ranking of Classes by Mean Drop for Pixel Size 8 µm and Exposure Time 0.01 s
			! Rank !! Ground Truth !! Mean Drop (Pixel Size 8 µm) !! Mean Drop (Exposure Time 0.01 s) !! Rank (Pixel Size 8 µm) !! Rank (Exposure Time 0.01 s)
			\|-
			\| 1 \|\| 15 \|\| 0.808365 \|\| 0.566514 \|\| 1 \|\| 1
			\|-
			\| 2 \|\| 292 \|\| 0.731833 \|\| 0.535144 \|\| 2 \|\| 2
			\|-
			\| 3 \|\| 19 \|\| 0.673408 \|\| 0.462795 \|\| 3 \|\| 3
			\|-
			\| 4 \|\| 120 \|\| 0.653789 \|\| 0.415552 \|\| 4 \|\| 4
			\|}

	== Conclusion ==		== Conclusion ==

Le: /* Impact on Image Quality */

2024-12-13T20:00:12Z

Impact on Image Quality

← Older revision		Revision as of 20:00, 13 December 2024
Line 51:		Line 51:
	The following table summarizes the trade-offs observed as pixel size changes. Note that the changes in the table discuss the relative changes between the different pixel sizes chosen.		The following table summarizes the trade-offs observed as pixel size changes. Note that the changes in the table discuss the relative changes between the different pixel sizes chosen.

			<center>
	{\| class="wikitable"		{\| class="wikitable"
	\|+ Effect of Increasing Pixel Size on Various Parameters		\|+ Effect of Increasing Pixel Size on Various Parameters
Line 61:		Line 62:
	\| 11 µm \|\| Lower \|\| Higher SNR \|\| Lower MTF		\| 11 µm \|\| Lower \|\| Higher SNR \|\| Lower MTF
	\|}		\|}
			</center>

	====== Discussion of Exposure Time and Channel Clipping ======		====== Discussion of Exposure Time and Channel Clipping ======

Le: /* References */

2024-12-13T19:59:42Z

References

← Older revision		Revision as of 19:59, 13 December 2024
Line 243:		Line 243:
	[https://arxiv.org/abs/2310.18894 Li et al., Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity]		[https://arxiv.org/abs/2310.18894 Li et al., Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity]

	[chatgpt.com GPT4] for writing support		[https://chatgpt.com GPT4] for writing support

Le: /* References */

2024-12-13T19:59:20Z

References

← Older revision		Revision as of 19:59, 13 December 2024
Line 238:		Line 238:
	== References ==		== References ==
	[https://arxiv.org/abs/1811.12231 Geirhos et al., ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness]		[https://arxiv.org/abs/1811.12231 Geirhos et al., ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness]

	[https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement Xu et al., Swin transformer and ResNet based deep networks for low-light image enhancement]		[https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement Xu et al., Swin transformer and ResNet based deep networks for low-light image enhancement]

	[https://arxiv.org/abs/2310.18894 Li et al., Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity]		[https://arxiv.org/abs/2310.18894 Li et al., Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity]

	[chatgpt.com GPT4] for writing support		[chatgpt.com GPT4] for writing support

Le: /* References */

2024-12-13T19:59:04Z

References

← Older revision		Revision as of 19:59, 13 December 2024
Line 237:		Line 237:

	== References ==		== References ==
			[https://arxiv.org/abs/1811.12231 Geirhos et al., ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness]
			[https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement Xu et al., Swin transformer and ResNet based deep networks for low-light image enhancement]
			[https://arxiv.org/abs/2310.18894 Li et al., Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity]
			[chatgpt.com GPT4] for writing support

Skhan44 at 19:56, 13 December 2024

2024-12-13T19:56:07Z

← Older revision		Revision as of 19:56, 13 December 2024
Line 235:		Line 235:

	By offering a systematic, lightweight alternative to traditional benchmarking, our approach empowers researchers and practitioners to diagnose and address model vulnerabilities with greater efficiency. This work highlights the value of controlled degradations as a tool for understanding model perception and improving robustness, thereby advancing the development of vision systems capable of withstanding the diverse and unpredictable nature of real-world imagery.		By offering a systematic, lightweight alternative to traditional benchmarking, our approach empowers researchers and practitioners to diagnose and address model vulnerabilities with greater efficiency. This work highlights the value of controlled degradations as a tool for understanding model perception and improving robustness, thereby advancing the development of vision systems capable of withstanding the diverse and unpredictable nature of real-world imagery.

			== References ==

Le: /* Exposure Time */

2024-12-13T19:54:52Z

Exposure Time

← Older revision		Revision as of 19:54, 13 December 2024
Line 209:		Line 209:
	The effects of simulating a decreased exposure time on the images in our dataset suggest that the model is highly sensitive to noise.		The effects of simulating a decreased exposure time on the images in our dataset suggest that the model is highly sensitive to noise.

	To illustrate this point, we once again evaluated the worst-performing classes after applying the image degradation. The worst among them was the robin class with an exposure time of 5e-3. Once noise was applied to the images, ResNet-18 classified the robin as a platypus, hyena, and toaster. We believe the model's deterioration can be attributed to the injection of noise; it relies heavily on fine-grained details, and makes spurious predictions when its primary features are distorted. There is a possibility that the model is over-attentive to the colors of the image again, which is supported by the toaster and the platypus classification. The simple prescription based on these results would be to fine-tune ResNet-18 on more low-light conditions.		To illustrate this point, we once again evaluated the worst-performing classes after applying the image degradation. The worst among them was the robin class with an exposure time of <math> 5*10^{-3} </math> seconds. Once noise was applied to the images, ResNet-18 classified the robin as a platypus, hyena, and toaster. We believe the model's deterioration can be attributed to the injection of noise; it relies heavily on fine-grained details, and makes spurious predictions when its primary features are distorted. There is a possibility that the model is over-attentive to the colors of the image again, which is supported by the toaster and the platypus classification. The simple prescription based on these results would be to fine-tune ResNet-18 on more low-light conditions.

	[[File:Comp3.png\|800px\|center\|thumb]]		[[File:Comp3.png\|800px\|center\|thumb]]

Le: /* Exposure Time */

2024-12-13T19:54:25Z

Exposure Time

← Older revision		Revision as of 19:54, 13 December 2024
Line 209:		Line 209:
	The effects of simulating a decreased exposure time on the images in our dataset suggest that the model is highly sensitive to noise.		The effects of simulating a decreased exposure time on the images in our dataset suggest that the model is highly sensitive to noise.

	To illustrate this point, we once again evaluated the worst-performing classes after applying the image degradation. The worst among them was the robin class. Once noise was applied to the images, ResNet-18 classified the robin as a platypus, hyena, and toaster. We believe the model's deterioration can be attributed to the injection of noise; it relies heavily on fine-grained details, and makes spurious predictions when its primary features are distorted. There is a possibility that the model is over-attentive to the colors of the image again, which is supported by the toaster and the platypus classification. The simple prescription based on these results would be to fine-tune ResNet-18 on more low-light conditions.		To illustrate this point, we once again evaluated the worst-performing classes after applying the image degradation. The worst among them was the robin class with an exposure time of 5e-3. Once noise was applied to the images, ResNet-18 classified the robin as a platypus, hyena, and toaster. We believe the model's deterioration can be attributed to the injection of noise; it relies heavily on fine-grained details, and makes spurious predictions when its primary features are distorted. There is a possibility that the model is over-attentive to the colors of the image again, which is supported by the toaster and the platypus classification. The simple prescription based on these results would be to fine-tune ResNet-18 on more low-light conditions.

	[[File:Comp3.png\|800px\|center\|thumb]]		[[File:Comp3.png\|800px\|center\|thumb]]

Skhan44: /* Pixel Size */

2024-12-13T19:53:25Z

Pixel Size

← Older revision		Revision as of 19:53, 13 December 2024
Line 224:		Line 224:

	Indeed, improving the ResNet architecture in low-light scenarios appears to be a significant area of study [https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement(3)].		Indeed, improving the ResNet architecture in low-light scenarios appears to be a significant area of study [https://www.researchgate.net/publication/373599865_Swin_transformer_and_ResNet_based_deep_networks_for_low-light_image_enhancement(3)].

	~~===Pixel Size===~~
	TODO: table of exposure time top 5 and bottom 5 and pixel size top 5/bottom 5 to indicate the pixel size and exposure time classes correspond quite well to each other, which makes sense because pixel size big = exposure time long

	== Conclusion ==		== Conclusion ==