RomeroPrabalaWan - Revision history

imported>Student2016: /* Appendix I: Code, Data, and Results */

2016-12-16T10:38:34Z

Appendix I: Code, Data, and Results

← Older revision		Revision as of 10:38, 16 December 2016
Line 141:		Line 141:
	== Appendices ==		== Appendices ==
	=== Appendix I: Code, Data, and Results ===		=== Appendix I: Code, Data, and Results ===
	All of ~~our~~ code, data, RealSense SDK modifications, heatmaps for all images, and results can be found in the following Github repository: [https://github.com/rwan6/rs-materials/tree/master]		All of the code, data, RealSense SDK modifications, heatmaps for all images, and results can be found in the following Github repository: [https://github.com/rwan6/rs-materials/tree/master]

	=== Appendix II: Group Work Partition ===		=== Appendix II: Group Work Partition ===

imported>Student2016: /* Data Acquisition and Pre-processing Pipeline */

2016-12-16T08:34:54Z

Data Acquisition and Pre-processing Pipeline

← Older revision		Revision as of 08:34, 16 December 2016
Line 35:		Line 35:
	* The first step is to capture the RGB and depth images with the long-range camera, and the RGB, depth, and IR images with the short-range camera. This is done using the RealSense SDK's raw camera application, with a small modification to capture the streamed image. Figures 4, 5, and 6 shows sample RGB, depth, and infrared images from the dataset taken with the short-range camera, respectively.		* The first step is to capture the RGB and depth images with the long-range camera, and the RGB, depth, and IR images with the short-range camera. This is done using the RealSense SDK's raw camera application, with a small modification to capture the streamed image. Figures 4, 5, and 6 shows sample RGB, depth, and infrared images from the dataset taken with the short-range camera, respectively.
	* The second step is to resize all the images. The images need to have a one-to-one pixel mapping to each other, since they will be passed to the classifiers as matrices. We selected to downsize the images so as to not create artificial pixels/features. The new image size of 600x400 is slightly smaller than the raw size of the IR and depth images, and allows for easier tiling, which is explained in the next step.		* The second step is to resize all the images. The images need to have a one-to-one pixel mapping to each other, since they will be passed to the classifiers as matrices. We selected to downsize the images so as to not create artificial pixels/features. The new image size of 600x400 is slightly smaller than the raw size of the IR and depth images, and allows for easier tiling, which is explained in the next step.
	* The third step is to break the image into smaller "tiles". As shown in Figure 7, we selected to partition the image into 100 tiles to make classification and prediction on the images computationally feasible for a personal ~~laptop~~. However, the tiling can be made more fine-grain, which has the potential to improve the classifier's prediction ability. Partitioning the image into tiles also increased our dataset size.		* The third step is to break the image into smaller "tiles". As shown in Figure 7, we selected to partition the image into 100 tiles to make classification and prediction on the images computationally feasible for a personal computer. However, the tiling can be made more fine-grain, which has the potential to improve the classifier's prediction ability. Partitioning the image into tiles also increased our dataset size.
	* The fourth step is to label the tiled images as skin, not-skin, or unknown (denoted as a question mark in Figure 3). We manually labeled all of the tiles to be as consistent as possible. In particular, tiles that were on the border of skin or not-skin were marked as unknown and not included in our classifier, since it would lead to inconsistent predictions and results.		* The fourth step is to label the tiled images as skin, not-skin, or unknown (denoted as a question mark in Figure 3). We manually labeled all of the tiles to be as consistent as possible. In particular, tiles that were on the border of skin or not-skin were marked as unknown and not included in our classifier, since it would lead to inconsistent predictions and results.

imported>Student2016: /* Data Acquisition and Pre-processing Pipeline */

2016-12-16T08:33:44Z

Data Acquisition and Pre-processing Pipeline

← Older revision		Revision as of 08:33, 16 December 2016
Line 34:		Line 34:

	* The first step is to capture the RGB and depth images with the long-range camera, and the RGB, depth, and IR images with the short-range camera. This is done using the RealSense SDK's raw camera application, with a small modification to capture the streamed image. Figures 4, 5, and 6 shows sample RGB, depth, and infrared images from the dataset taken with the short-range camera, respectively.		* The first step is to capture the RGB and depth images with the long-range camera, and the RGB, depth, and IR images with the short-range camera. This is done using the RealSense SDK's raw camera application, with a small modification to capture the streamed image. Figures 4, 5, and 6 shows sample RGB, depth, and infrared images from the dataset taken with the short-range camera, respectively.
	* The second step is to resize all the images. The images need to have a one-to-one pixel mapping to each other, since they will be passed to the classifiers as matrices. We selected to downsize the images so as to not create artificial pixels/features. The new image size of 600x400 is slightly smaller than the raw size of the IR and depth images, and allows for easier tiling, which is explained in the next ~~bullet-point~~.		* The second step is to resize all the images. The images need to have a one-to-one pixel mapping to each other, since they will be passed to the classifiers as matrices. We selected to downsize the images so as to not create artificial pixels/features. The new image size of 600x400 is slightly smaller than the raw size of the IR and depth images, and allows for easier tiling, which is explained in the next step.
	* The third step is to break the image into smaller "tiles". As shown in Figure 7, we selected to partition the image into 100 tiles to make classification and prediction on the images computationally feasible for a personal laptop. However, the tiling can be made more fine-grain, which has the potential to improve the classifier's prediction ability. Partitioning the image into tiles also increased our dataset size.		* The third step is to break the image into smaller "tiles". As shown in Figure 7, we selected to partition the image into 100 tiles to make classification and prediction on the images computationally feasible for a personal laptop. However, the tiling can be made more fine-grain, which has the potential to improve the classifier's prediction ability. Partitioning the image into tiles also increased our dataset size.
	* The fourth step is to label the tiled images as skin, not-skin, or unknown (denoted as a question mark in Figure 3). We manually labeled all of the tiles to be as consistent as possible. In particular, tiles that were on the border of skin or not-skin were marked as unknown and not included in our classifier, since it would lead to inconsistent predictions and results.		* The fourth step is to label the tiled images as skin, not-skin, or unknown (denoted as a question mark in Figure 3). We manually labeled all of the tiles to be as consistent as possible. In particular, tiles that were on the border of skin or not-skin were marked as unknown and not included in our classifier, since it would lead to inconsistent predictions and results.

imported>Student2016: /* Background */

2016-12-16T08:30:13Z

Background

← Older revision		Revision as of 08:30, 16 December 2016
Line 10:		Line 10:
	Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. The long-range camera used in this project uses stereo infrared cameras to generate depth data.		Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. The long-range camera used in this project uses stereo infrared cameras to generate depth data.

	The short-range camera used in this project generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is ~~there is a~~ limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.		The short-range camera used in this project generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is the limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.

	The final relevant form of measuring depths of a scene is time of flight. This technique is most notably used by Microsoft's Kinect V2. This works by analyzing the time delay from the time light was sent onto the scene to the time that light was detected to be reflected back onto the scene. This measurement is done periodically, so in the case of scene objects that are farther away from the camera, the light might return to the sensor after the next period had already started. This can lead to ambiguity in the depth of parts of the scene. Time of flight was not used by any of the cameras in this project.		The final relevant form of measuring depths of a scene is time of flight. This technique is most notably used by Microsoft's Kinect V2. This works by analyzing the time delay from the time light was sent onto the scene to the time that light was detected to be reflected back onto the scene. This measurement is done periodically, so in the case of scene objects that are farther away from the camera, the light might return to the sensor after the next period had already started. This can lead to ambiguity in the depth of parts of the scene. Time of flight was not used by any of the cameras in this project.

imported>Student2016: /* Background */

2016-12-16T08:28:11Z

Background

← Older revision		Revision as of 08:28, 16 December 2016
Line 8:		Line 8:

	== Background ==		== Background ==
	Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. The ~~longe~~-range camera used in this project uses stereo infrared cameras to generate depth data.		Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. The long-range camera used in this project uses stereo infrared cameras to generate depth data.

	The short-range camera used in this project generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.		The short-range camera used in this project generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.

imported>Student2016: /* Background */

2016-12-16T08:28:04Z

Background

← Older revision		Revision as of 08:28, 16 December 2016
Line 8:		Line 8:

	== Background ==		== Background ==
	Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. ~~One of the cameras~~ used in this project uses stereo infrared cameras to generate depth data.		Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. The longe-range camera used in this project uses stereo infrared cameras to generate depth data.

	The ~~second~~ camera we used generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.		The short-range camera used in this project generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.

	The final relevant form of measuring depths of a scene is time of flight. This technique is most notably used by Microsoft's Kinect V2. This works by analyzing the time delay from the time light was sent onto the scene to the time that light was detected to be reflected back onto the scene. This measurement is done periodically, so in the case of scene objects that are farther away from the camera, the light might return to the sensor after the next period had already started. This can lead to ambiguity in the depth of parts of the scene. Time of flight was not used by any of the cameras in this project.		The final relevant form of measuring depths of a scene is time of flight. This technique is most notably used by Microsoft's Kinect V2. This works by analyzing the time delay from the time light was sent onto the scene to the time that light was detected to be reflected back onto the scene. This measurement is done periodically, so in the case of scene objects that are farther away from the camera, the light might return to the sensor after the next period had already started. This can lead to ambiguity in the depth of parts of the scene. Time of flight was not used by any of the cameras in this project.

imported>Student2016: /* Background */

2016-12-16T08:24:31Z

Background

← Older revision		Revision as of 08:24, 16 December 2016
Line 8:		Line 8:

	== Background ==		== Background ==
	Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo when there are repetitive elements in the images, as there can be many candidates a corresponding point. One of the cameras used in this project uses stereo infrared cameras to generate depth data.		Generating a 2D image with depth data can be done with various techniques that fall under range imaging. Initially, this was done by attempting to emulate the human visual system using stereo cameras and recreating a scene from the differences in the images. Reconstructing the scene involves heavy use of epipolar geometry. However, this technique has a downside of not being able to identify the depths of a uniform surface. On a uniform surface, finding corresponding points is nearly impossible. In addition, this technique struggles whenever there are occlusions present in the images, as one image will contain information not present in the other. Another weakness of stereo algorithms is when there are repetitive elements in the images, as there can be many candidates at a corresponding point. One of the cameras used in this project uses stereo infrared cameras to generate depth data.

	The second camera we used generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.		The second camera we used generates depth data through structured light patterns. This process works by projecting a carefully chosen infrared light pattern (or series of patterns) onto a scene. The depth information is calculated based on the distortion of the light pattern when it hits the scene object. The light pattern is chosen in a way to be able to uniquely identify each part of the light stripe. The downside to this approach is there is a limited resolution of depth data available. Depth is only calculable for the points illuminated by the light pattern, and the granularity of the light pattern determines how many distinct depths can be calculated. This means that interpolation is sometimes used to guess the depths for parts of the image [5]. Hence, this technique is only used for shorter range applications, where resolution is less likely to become an issue.

imported>Student2016: /* Introduction */

2016-12-16T08:20:14Z

Introduction

← Older revision		Revision as of 08:20, 16 December 2016
Line 1:		Line 1:
	== Introduction ==		== Introduction ==
	Material classification combines the fields of computer vision and machine learning in an attempt to emulate the inner workings of the human eye and brain. It attempts to answer the following question:		Material classification combines the fields of computer vision and machine learning in an attempt to emulate the inner workings of the human eye and brain. It attempts to answer the following question:
	given an image, what materials are present and can be identified.		given an image, what materials are present and can be identified?

	There have been efforts to answer this question relying solely on RGB images. This generally involves pre-processing a database of images to extract additional information about the material before using all of that information to train a classifier [1]. The pre-processing steps range from a simple high pass filter to computing a spectral histogram of the image [2]. The goal of evaluating pre-processing techniques is to select the most useful features for training a classifier. Although the choice of these techniques is a major contributor to the effectiveness of a material classifier, we looked at the effectiveness of depth and infrared data. More information about related work can be found in [3] and [4].		There have been efforts to answer this question relying solely on RGB images. This generally involves pre-processing a database of images to extract additional information about the material before using all of that information to train a classifier [1]. The pre-processing steps range from a simple high pass filter to computing a spectral histogram of the image [2]. The goal of evaluating pre-processing techniques is to select the most useful features for training a classifier. Although the choice of these techniques is a major contributor to the effectiveness of a material classifier, we looked at the effectiveness of depth and infrared data. More information about related work can be found in [3] and [4].

imported>Student2016: /* Introduction */

2016-12-16T08:19:29Z

Introduction

← Older revision		Revision as of 08:19, 16 December 2016
Line 1:		Line 1:
	== Introduction ==		== Introduction ==
	Material classification combines the fields of computer vision and machine learning in an attempt to emulate the inner workings of the human eye and brain. It attempts to answer the following question:		Material classification combines the fields of computer vision and machine learning in an attempt to emulate the inner workings of the human eye and brain. It attempts to answer the following question:
	given an image, what materials are present and can be identified?		given an image, what materials are present and can be identified.

	There have been efforts to answer this question relying solely on RGB images. This generally involves pre-processing a database of images to extract additional information about the material before using all of that information to train a classifier [1]. The pre-processing steps range from a simple high pass filter to computing a spectral histogram of the image [2]. The goal of evaluating pre-processing techniques is to select the most useful features for training a classifier. Although the choice of these techniques is a major contributor to the effectiveness of a material classifier, we looked at the effectiveness of depth and infrared data. More information about related work can be found in [3] and [4].		There have been efforts to answer this question relying solely on RGB images. This generally involves pre-processing a database of images to extract additional information about the material before using all of that information to train a classifier [1]. The pre-processing steps range from a simple high pass filter to computing a spectral histogram of the image [2]. The goal of evaluating pre-processing techniques is to select the most useful features for training a classifier. Although the choice of these techniques is a major contributor to the effectiveness of a material classifier, we looked at the effectiveness of depth and infrared data. More information about related work can be found in [3] and [4].

imported>Student2016: /* Data Acquisition and Pre-processing Pipeline */

2016-12-16T08:07:50Z

Data Acquisition and Pre-processing Pipeline

← Older revision		Revision as of 08:07, 16 December 2016
Line 44:		Line 44:
	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\| [[File:Pic_ir.png\|450px\|center\|thumb\|caption\|Figure 6: Example of infrared image taken with the F200 camera]]		\| [[File:Pic_ir.png\|450px\|center\|thumb\|caption\|Figure 6: Example of infrared image taken with the F200 camera]]
	\| [[File:Pic_tile.png\|450px\|center\|thumb\|caption\|Figure 7: ~~Images were~~ partitioned into 100 tiles]]		\| [[File:Pic_tile.png\|450px\|center\|thumb\|caption\|Figure 7: Image being partitioned into 100 tiles]]
	\|}		\|}