Merge pull request #3 from ifm/lm_new_documents

Parameters documentation

O3R/Parameters/AcquisitionSettings/README.md

@@ -0,0 +1,4 @@
+# README - Acquisition settings
+
+- [Modes](modes.md)
+- [Offset](offset.md)

O3R/Parameters/AcquisitionSettings/modes.md

@@ -0,0 +1,29 @@
+# Modes
+
+## Description
+
+The O3R has the specificity to provide several measurement ranges. A distance measurement is computed only for a subspace of the scene. Elements *fully* outside of the range are not taken into account and have no impact on the measurement (for instance by causing artifacts like [stray-light](../Filter/strayLight.md) or Multi-Path Interference(*coming soon*)).
+
+>Note: Objects *very* close to the beginning or end of the measurement range can still have an impact on the measurement.
+
+By default, the O3R provides two measurement ranges: two meters and four meters. These ranges use different frequencies to perform the Time-of-Flight (ToF) measurement and therefore show different characteristics, especially in how they are affected by artifacts. The four-meter mode shows higher noise levels than the two-meter mode (around twice as much).
+
+## Example
+
+Let’s look at a simple scene: three boxes are placed in front of the camera, one, two, and three meters away (see image below).
+![RGB view of the scene](resources/modes_scene.png)
+
+The table below shows the computed distance measurement in the distance image view and the point cloud view with the two modes available by default:
+
+| Mode| Distance Image| Point Cloud|
+|--|--|--|
+| 2 m| ![Distance image for the two meter mode](resources/2m_mode_distance.png)| ![Point cloud for the two meter mode](resources/2m_mode_cloud.png)|
+| 4 m| ![Distance image for the four meter mode](resources/4m_mode_distance.png)| ![Point cloud for the four meter mode](resources/4m_mode_cloud.png)|
+
+The third box, which is three meters away from the camera, is outside of the measurement range when using the two-meter mode; however it is visible when using the four-meter mode.
+
+> Note: Using the [offset](offset.md) parameter in combination with the mode is interesting and allows for a lot of flexibility in using the coded modulation ToF technology. We encourage you to investigate strategies using multiple modes in combination with offsets (see our application note (*coming soon*) on the topic).
+
+## Related Topics
++ [Offset](offset.md)
++ [Basic concepts](../basicConcepts.md)

O3R/Parameters/AcquisitionSettings/offset.md

@@ -0,0 +1,23 @@
+# Offset
+
+## Description
+The offset parameter shifts the beginning of the measurement range in space. For instance, when using the 2m mode with an offset of 1m, the O3R will compute distance data for a range between 1 and 3 m from the camera.
+
+Using the offset can allow you to collect distance measurements past the measurement range set by the [mode](modes.md) while taking advantage of the robust point cloud the O3R provides and the specificities of each mode.
+
+The offset can be set at negative values, which brings the end of the measurement range closer to the camera. This can be useful for mitigating MPI artifacts (*coming soon*)), for instance, or for avoiding artifacts caused by highly reflective objects (see [stray-light artifacts](../Filters/strayLight.md)), by removing the cause of the artifact from the FoV.
+
+## Example
+Let's look at the following scene. Three boxes are positioned in front of the camera at about one, two, and three meters away.
+![RGB view of the offset scene](resources/offset_scene.png)
+
+We are using the 2m [mode](modes.md), with all the other settings as default. The table below shows the point cloud for multiple values of the offset.
+
+| Offset (meters)| Point Cloud|
+|--|--|
+| -0.5| ![Point cloud with offset -0.5](resources/offset_-05_cloud.png)|
+| 0| ![Point cloud with offset 0](resources/offset_0_cloud.png)|
+| 1.5| ![Point cloud with offset 1.5](resources/offset_15_cloud.png)|
+| 2.5| ![Point cloud with offset 2.5](resources/offset_25_cloud.png)|
+
+> Note: In the last image where the offset is set to 2.5m, we can see that the noise is higher than in the other images. This is due to the distance to the camera, with which the noise increases, and to the fact that the most robust measurement is in the middle of the range, which is from around 3 to 4 m in the case of the last example. The ground in front of the box is outside of the robustness area.

O3R/Parameters/Filters/README.md

@@ -0,0 +1,11 @@
+# README - Filters
+- [Filtering process overview](filteringProcess.md)
+- [Maximum distance noise](maxDistNoise.md)
+- [Minimum amplitude](minAmplitude.md)
+- [Minimum reflectivity](minReflectivity.md)
+- [Adaptive noise bilateral filter](bilateralFilter.md)
+- [Median filter](median.md)
+- [Temporal filter](temporalFilter.md)
+- [Mixed pixel filter](mixedPixelFilter.md)
+- [Symmetry threshold](symmetryThreshold.md)
+- [Stray light filter](strayLight.md)

O3R/Parameters/Filters/bilateralFilter.md

@@ -0,0 +1,41 @@
+# Adaptive noise bilateral filter
+## Abstract
+The O3R software allows for filtering the distance measurement in the spatial domain. The spatial domain of a 3D image can be thought of as the local neighborhood of a pixel, that is, the neighboring pixels X-, Y-, and Z-coordinates. Radial distance information for a pixel is combined with its neighbors' information to form a new distance image with reduced noise.  
+
+The bilateral filter is the preferred spatial filter and is enabled by default. It can be applied with different filter mask sizes, which can be set via the parameter `anfFilterSizeDiv2`. Larger filter mask sizes allow for stronger noise reduction.
+
+## Description
+
+This distance bilateral filter is, in its concept, highly similar to a [bilateral filter applied to RGB 2D images](https://en.wikipedia.org/wiki/Bilateral_filter). A bilateral filter is a nonlinear edge-preserving smoothing filter. The idea is to replace the pixel value with a weighted average of the information from nearby pixels. The weighting is a combination of the spatial kernel and the range kernel. The O3R implementation additionally incorporates the distance noise estimation for calculating the filter weights.     
+
+> Note: The weighted average is computed by convolution over the spatial domain. The convolution of the original image and the filter mask returns an image reduced by half the filter size at each image border; that is, with a filter mask of 7x7 pixels, the image is stripped of 7 pixels vertically and horizontally (these pixels are marked as invalid).
+
+The bilateral filter is controlled by the parameter `anfFilterSizeDiv2` (turn it off with `anfFilterSizeDiv2 = 0`). `anfFilterSizeDiv2 = 3` sets the filter mask size to a local 7x7 pixel neighborhood.   
+
+> Note: The bilateral filter is preferred over the [median filter](median.md) because it preserves edge and corner information better (the median filter tends to *round* corners). It is also possible to apply the bilateral filter with larger filter masks (up to 7x7 pixel masks) compared to 5x5 pixel masks for the median filter. The size of the mask defines how many neighboring pixels are considered when computing a pixel’s value.   
+
+Invalid pixels will be ignored during the filtering process and therefore have no impact on the surrounding pixels. Invalid pixels will remain invalid after the filtering. 
+### Example
+
+The following pictures give an overview of the capabilities of spatial filtering with the bilateral filter for different filter mask sizes. All other filters ([temporal](temporalFilter.md) and [median](median.md)) are deactivated for the purpose of illustrating the bilateral filter's effect. The maximum allowed distance noise is set at 0.2 m for all images. Note that for maximum distance noise values below 0.2 m the point cloud becomes extremely sparse for smaller filter mask sizes (not shown in the following).  
+
+The scene shows a view of our lab, containing various typical objects including a black tote in the center of the room. It is a static scene, which makes it simpler to illustrate the filter's effect, but these settings (active bilateral filter and inactive temporal filter) are typical for scenes involving motion. Have a look specifically at the distance noise images in the following table. We can see that the distance noise greatly reduces as the filter mask size increases (the color red denotes negligible noise, whereas blue represents noise of around 1 cm and above).
+
+| Bilateral filter mask size| Point cloud| Distance (top left), amplitude (top right), distance noise (bottom left), and reflectivity images (bottom right)| |
+|--|--|--|--|
+| 0 (filter disabled)| ![anfFilterSizeDiv2_0_value](./resources/anfFilterSizeDiv2_0.png "3D point cloud without spatial filtering / bilateral filter switched off")| ![anfFilterSizeDiv2_0_value](./resources/anfFilterSizeDiv2_0_imgs.png "distance, amplitude, distance noise, and reflectivity images without bilateral filtering")| ![Color bar](resources/color_bar_noise.png)| 
+| 1 (3x3 mask size)| ![anfFilterSizeDiv2_1_value](./resources/anfFilterSizeDiv2_1.png "3D point cloud with spatial filtering: bilateral filter mask set to 3x3 pixel neighbourhood")| ![anfFilterSizeDiv2_1_value](./resources/anfFilterSizeDiv2_1_imgs.png "distance, amplitude, distance noise, and reflectivity images with bilateral filter mask set to 3x3 pixel neighbourhood")| |
+| 2 (5x5 mask size)| ![anfFilterSizeDiv2_2_value](./resources/anfFilterSizeDiv2_2.png "3D point cloud with spatial filtering: bilateral filter mask set to 5x5 pixel neighbourhood")| ![anfFilterSizeDiv2_2_value](./resources/anfFilterSizeDiv2_2_imgs.png "distance, amplitude, distance noise, and reflectivity images with bilateral filter mask set to 5x5 pixel neighbourhood")| |
+| 3 (7x7 mask size)| ![anfFilterSizeDiv2_3_value](./resources/anfFilterSizeDiv2_3.png "3D point cloud with spatial filtering: bilateral filter mask set to 7x7 pixel neighbourhood")| ![anfFilterSizeDiv2_3_value](./resources/anfFilterSizeDiv2_3_imgs.png "distance, amplitude, distance noise, and reflectivity images with bilateral filter mask set to 7x7 pixel neighbourhood")| |
+
+> Note: distance information for the black tote in the middle of the image remains extremely hard to compute even with a *strong* lateral filtering. For better handling of dark objects, have a look at the [minimum amplitude](minAmplitude.md), [maximum distance noise](maxDistNoise.md), and [temporal filter](temporalFilter.md)).
+
+### Scenes involving motion
+The spatial filtering can be performed in scenes where motion is present: only the parts of the images that are not affected by movement will be filtered. This differentiation is possible because the detection of motion is performed before the spatial filter in the processing pipeline. It is perfectly fine and encouraged to use large filter mask sizes.
+
+> Note: this is not true for [temporal filtering](temporalFilter.md), which is not best suited to in-motion cases.
+
+
+## Related topics
++ [Median filter](median.md)
++ [Temporal filter](temporalFilter.md)

O3R/Parameters/Filters/filteringProcess.md

@@ -0,0 +1,9 @@
+# Filtering Process
+
+This document details the filtering process applied to the O3R ToF data to produce distance measurements. Each filter listed below can be enabled, disabled or fine-tuned to better fit your application. Please read the detailed documentation for each filter (linked below) for more details.
+
+![Diagram of the filtering process](resources/algo_processing_chain.png)
+
+# Related Topics
+- Find an overview of the filters [here](parameters.md).
+- Find a description of the available images *coming soon*.

O3R/Parameters/Filters/maxDistNoise.md

@@ -0,0 +1,33 @@
+# Maximum Distance Noise
+## Abstract
+
+The O3R software estimates distance noise per pixel as well as the distance information per pixel. This distance noise parameter is an estimation of the standard deviation of the radial distance measurement, given in meters. It is based on a noise model built upon the acquired time of flight (ToF) measurements of a single frame. Pixels with a noise value above the threshold `maxDistNoise` are invalidated.
+
+
+## Description
+The O3R camera and software use the ifm ToF technology to measure the distance of objects per pixel. The result are a distance image as and a distance noise image. The distance noise deduction can be interpreted as a standard deviation of the distance measurement in a metric scale. The noise level is dependent on the received signal’s amplitude (lower amplitude means greater noise) and on the ambient light level (high ambient light level, especially sunlight, can lead to high noise level). 
+
+The distance noise image is processed in the same algorithmic pipeline as the distance image. Any filter applied to the distance image is applied to the distance noise image as well. For example, if filters are activated in the spatial domain (see the [bilateral filter](bilateralFilter.md)), then they also filter the distance noise image such that the adapted noise image reflects the lowered noise due to lateral filtering.
+
+The parameter `maxDistNoise` is used to invalidate pixels with high noise levels. Higher `maxDistNoise` values will allow noisier pixels to be valid pixels in the point cloud. The maximum allowed value is 1 meter, although we do not recommend using such a high value because the resulting distance measurement will be highly inaccurate in the noisy areas. 
+Low `maxDistNoise` values will result in more noisy pixels being marked as invalid. Values lower than 0.01 meters should always be validated against worst-case expected object and ambient light levels.    
+
+The minimum allowed `maxDistNoise` value is 0.00 meters. This will switch off the validation process based on the estimated distance noise image. The distance noise image is still computed and available to the user.
+
+We suggest you start your experiments with the default values and assess the point cloud quality for your specific use case. The current default value is set to 0.02 meters, which allows for a robust point cloud with negligible noise. For applications where a rich point cloud (i.e., a cloud with more valid pixels) is preferred over accuracy, increasing the noise threshold can be a good idea.
+
+### Example
+The following table shows measurements for the same scene with two different distance noise threshold values. The scene consists of a box positioned 1 meter away from the camera, outside in full sunlight. The amount of noise due to the ambient light is high, but it is apparent that we can still get distance values for many pixels by increasing the noise threshold.
+> Note: For demonstration purposes, we disabled the [temporal filter](temporalFilter.md) in these images.
+
+| Noise threshold [m]| Distance noise image| | Point cloud|
+|:--:|--|--|--|
+| 0.01| ![Low noise threshold - noise image](resources/noise_outside_noise.png)| ![Color bar](resources/color_bar_noise.png)| ![Low noise threshold in the point cloud](resources/low_noise_001_outside_cloud.png)|
+| 0.07 | | | ![Low noise threshold in the point cloud](resources/med_noise_007_outside_cloud.png)|
+
+> Note: The distance noise image is the same for both noise threshold values. The difference is viewed in the point cloud, where the noise filtering discards more or fewer pixels.
+
+## Related topics
++ [Bilateral filter](bilateralFilter.md) (see also the [median filter](median.md))
++ [Temporal filter](temporalFilter.md)
++ [Minimum amplitude](minAmplitude.md)

O3R/Parameters/Filters/median.md

@@ -0,0 +1,51 @@
+# (Spatial) Median Filter
+
+## Abstract
+The O3R software supports two spatial filters for improving the distance measurements: the median filter and the [bilateral filter](bilateralFilter.md).   
+**We recommend using the bilateral filter in most cases instead of the median filter because the median filter can have undesirable side effects.**
+
+## Description
+
+The median filter is conceptually very similar to a [median filter applied to RGB 2D images](https://en.wikipedia.org/wiki/Median_filter). A median filter is a nonlinear, edge-preserving smoothing filter. It can be thought of as a filter that replaces the value per pixel with the median value of neighboring pixels. The computation is achieved by sliding the filter mask in the spatial domain until it covers the whole image. 
+This filtering technique is robust (i.e., not affected by outliers) and reduces noise while keeping edge information intact. 
+The median filter is applied to the distance image. The distance noise is lowered to heuristically reflect the new noise in the distance image.
+
+
+
+The median filter is controlled by the parameter `medianSizeDiv2`; turn it off with 
+`medianSizeDiv2 = 0`. 
+`medianSizeDiv2 = 1` sets the filter mask size to a size of 3 x 3 pixels.
+`medianSizeDiv2 = 2` is the highest allowed value. It represents a filter mask size of 5 x 5 pixels.
+Using larger filter mask sizes combines more pixels’ distance measurements into the filterer value. The filter’ effect will be stronger, resulting in a *smoother* image.
+
+>Note: Invalid pixels are ignored during the filtering process and therefore have no impact on their surrounding pixels. Invalid pixels remain invalid after the filtering. 
+ 
+
+## Example
+Below are images of the same scene with different settings for the median filter. Look more specifically at the distance noise image that shows the amount of noise in the scene—the larger the filter mask size, the lower the noise level. The color red corresponds to negligible noise levels and blue to noise around 1 cm and above. See the [bilateral filter](bilateralFilter.md) example for comparison with the same scene.
+
+| Filter mask size `medianSizeDiv2`| Point cloud| Distance (top left), amplitude (top right), distance noise (bottom left), and reflectivity (bottom right) images| |
+|--|--|--|--|
+| 0 (filter deactivated)| ![medianSizeDiv2_0_value](./resources/medianSizeDiv2_0.png "3D point cloud without spatial filtering / median filter switched off")| ![medianSizeDiv2_0_value](./resources/medianSizeDiv2_0_imgs.png "distance, amplitude, distance noise, and reflectivity images without spatial filtering / median filter switched off")| ![color bar for noise image](resources/color_bar_noise.png)|
+| 1 (3 x 3 mask size)| ![medianSizeDiv2_1_value](./resources/medianSizeDiv2_1.png "3D point cloud with spatial filtering: median filter mask set to 3x3 pixel neighbourhood")| ![medianSizeDiv2_0_value](./resources/medianSizeDiv2_1_imgs.png "distance, amplitude, distance noise, and reflectivity images with spatial filtering: median filter mask set to 3x3 pixel neighbourhood")| |
+| 2 (5 x 5 mask size)| ![medianSizeDiv2_2_value](./resources/medianSizeDiv2_2.png "3D point cloud with spatial filtering: median filter mask set to 5x5 pixel neighbourhood")| ![medianSizeDiv2_0_value](./resources/medianSizeDiv2_2_imgs.png "distance, amplitude, distance noise, and reflectivity images with spatial filtering: median filter mask set to 5x5 pixel neighbourhood")| |
+
+
+## Bilateral vs. median filtering
+### Disadvantages of the median filter
+The median filter is not our spatial filter of choice for two reasons: it does not preserve corners of objects as well as the bilateral filter, and it uses a heuristic method for dealing with the distance noise image. Moreover, the median filter can introduce a bias in the distance image (locally) in some cases, an effect that is not present in the bilateral filter.
+We recommend using the [bilateral filter](bilateralFilter.md) in most cases.
+
+### Bilateral and median filters combined
+A combination of both spatial filters is rarely required, and we recommend increasing the filter mask size as a first step. However, if the filtering is not strong enough, then one can use both the bilateral and median filters at the same time. This will further reduce local noise levels but can result in bias in larger noise patterns. 
+
+To give you an idea, the image below shows the effect of combined bilateral (`anfFilterSizeDiv2=3`) and median (`medianFilterSizeDiv2=2`) filtering for the example scene.
+![bilateral3_median2](./resources/bilateral3_median2.png "3D point cloud with heavy spatial filtering: median filter mask set to 5x5 pixel neighbourhood, bilateral filter mask set to 7x7 pixel neighbourhood")   
+
+## Related topics
++ [Bilateral filter](bilateralFilter.md)
++ [Temporal filter](temporalFilter.md)
+
+
+
+