topic Utility for changing laser camera parameters (IVCAM v0.5) in Software Archive

Utility for changing laser camera parameters (IVCAM v0.5)

samontab — Fri, 02 Jan 2015 08:12:02 GMT

IVCAM v0.5 is a free utility that I made that allows you to do the following with your Intel RealSense F200 camera:

- Read and write camera and laser projector settings.
- Display depth, XYZ (color coded), or IR streams in real time while you are changing those settings.

Requirements are either:

a) Laptop with a built-in RealSense F200 camera

b) Computer with the following:
- 32 or 64 bit Windows 8.1 or Windows 10.
- DCM installed (to use the USB version of the F200 camera).

This is how it currently looks:

You can download it from here:
http://www.samontab.com/web/files/ivcam.zip

Thank you Sebastian! Great

Colleen_C_Intel — Fri, 02 Jan 2015 20:33:36 GMT

Thank you Sebastian! Great add to the community!

(In a few days I'll move it up to a post note to stay near the top).

The RealSense camera has a

Robert_Oschler — Fri, 16 Jan 2015 10:42:04 GMT

The RealSense camera has a laser in it? I thought the depth sense data stream was infrared using the time-of-flight algorithm?

Robert

OK, here is how this camera

samontab — Fri, 16 Jan 2015 13:44:31 GMT

OK, here is how this camera works:

The Intel RealSense camera has an IR laser projector that emits IR light into the scene. There is an IR camera that reads the emitted pattern. Based on the displacement of the pattern due to objetcs in the scene, it can calculate the distance of the objects from the camera. This method of calculating depth in general is know as structured light, and this is the way other 3D cameras, like the kinect, work.

In the following image you can see the IR light being emitted by the camera into a wall that was a few cms away. (I expected a pincushion distortion in both axes, but it seems that only the vertical axis is distorted).

Now, about the IVCAM settings:

The LaserPower argument basically allows controlling the intensity of this laser projection, from powered off at 0 up to 16 at maximum power. Usually you will be using this setting at 16.

MotionRangeTradeOff: The difference here is that the pattern used in this camera is not a fixed set of points as is the case with the kinect, but instead is a moving pattern, from top to bottom. It seems that the rate at how the patttern is projected is controlled by this setting. The longer the number, the longer the vertical pattern gets stretched, so less patterns per second are being emitted and detected. This allows to gather more light, and therefore to detect objects farther away, but when they move, you don't detect the pattern correctly so no depth info for fast moving objects far away. The reverse is true for smaller numbers. You can think of this parameter as exposure time in normal camera terms.

Here is an image of the pattern. Because the image was exposed for too long (normal camera rates, like 30fps for example), the pattern looks blurred in the vertical direction, but some information can be seen in the horizontal axis.:

If you decrease the exposure time, you can actually see the pattern that is being projected into the wall:

Note that the pattern is projected so that it is seen straight, even though the illuminated area is not.

Accuracy: This parameter changes between 3 types of projected patterns, from coarse up to fine grained.

FilterOption: I think this parameter is only for the post processing and creation of the depth map. Given the initial points, with this option you select the method to interpolate the rest of the frame. Here there are a few different option. Look at the SDK manual for more details on this.

I hope these parameters make a bit more sense now.

Samontab,

Jack_K_1 — Fri, 16 Jan 2015 14:44:50 GMT

Samontab,

My understanding is that the Real Sense camera uses time-of-flight not structured light. Here is a description of the camera www.eetasia.com posted on January 9th:

" It comprises a conventional HD RGB sensor, a laser that emits 16 different modulated signals and an IR sensor to detect the time-of-flight of the transmitted laser signals and compute distance. "

This article also mentions that ASUS and Lenovo are marketing laptops with the camera in the Netherlands.

However, a user comment to an article in PC Perspective (www.pcper.com) in October 2014, stated that Real Sense uses structured light depth sensing as did Kinect v1, whereas the first version used time-of-flight. Perhaps this is the source of confusion.

By the way, Kinect v2 now uses time-of-flight.

Maybe someone from Intel can clear up the debate.

Regards,

Jack

Thank you Sebastian, this has

simsam7 — Sun, 18 Jan 2015 00:55:15 GMT

Thank you Sebastian, this has helped me a lot.

Regards,

Sam

The previous "Perceptual"

Zsolt_E_ — Sun, 18 Jan 2015 09:27:49 GMT

The previous "Perceptual" camera was a rebranded SoftKinetic DepthSense device, which was Time of Flight camera.
I'd be really surprised if they would have totally changed the device between the beta and the gold release. The resolutions, range, etc. are mostly the same, so I'm 90% sure it's still Time of Flight.

samontab: thanks for this brilliant tool, can you extend it with SetDepthConfidenceThreshold? It's strangely missing.

Looking at Samontab's

Colleen_C_Intel — Mon, 19 Jan 2015 05:04:05 GMT

Looking at Samontab's excellent pictures of patterned (structured) light would seem to answer the question.
The Perceptual camera was totally different than the RealSense F200. (And similarly future Realsense cameras may use different tecnology than F200).

@samontab and others. Thank

Robert_Oschler — Mon, 19 Jan 2015 12:15:45 GMT

@samontab and others. Thank you for the information!

Robert

Very useful tool and topic.

oliveirafhm — Fri, 30 Jan 2015 17:34:29 GMT

Very useful tool and topic.

Thanks!

Hi guys,

samontab — Sun, 01 Feb 2015 04:47:13 GMT

Hi guys,

I just updated this tool:

IVCAM v0.2
- Updated camera frame access for better performance.
- Added DepthConfidenceThreshold to the controls.
- Added individual buttons to set a feature to its default value.
- Added tool tips explaining the feature values.
- Added IR Camera information details.

Here is a screenshot of it, showing a longer range capability by tweaking the parameters:

This version should work better in general than the previous one. So if you had any issues before, give it a try again.

You can still get it at the same place:

http://www.samontab.com/web/files/ivcam.zip

The longer range is

simsam7 — Fri, 06 Feb 2015 22:46:11 GMT

The longer range is definitely something I'm interested in, thanks!

Regards,

Sam

Nice post. Thank you all!

Ricardo_Braga — Tue, 10 Feb 2015 08:16:35 GMT

Nice post. Thank you all!

Another update to this tool:

samontab — Tue, 10 Feb 2015 09:15:16 GMT

Another update to this tool:

IVCAM v0.3
- Added XYZ camera stream (Lab color space coded image based on XYZ position of each pixel, X and Y for color, Z for lightness)
- Updated depth stream view to display the entire depth range correctly (You should now see more gray scale tones, not mostly just black and white as it was before)
- Minor bug fixes

You can still get it at the same place:

http://www.samontab.com/web/files/ivcam.zip

I wrote similar software for

Daniel_J_1 — Tue, 10 Feb 2015 18:35:43 GMT

I wrote similar software for Linux, and recorded a youtube demo of adjusting the controls. https://www.youtube.com/watch?v=Ht9PzVjWOgI

Nice to hear that Daniel!

samontab — Tue, 10 Feb 2015 23:05:34 GMT

Nice to hear that Daniel!

I'm also developing a Linux version of the SDK for the Intel RealSense camera, as I usually prefer to use Linux. It looks promising because as you could see, the OS detects the camera without an issue. Actually the app that I published here is cross platform, so once I finish the Linux access to the camera, this app will be able to run in Linux as well.

Regarding your depth pixel format mistery, the camera actually has 6 depth formats, as seen in the documentation:

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?pixelformat_pxcimage.html

Here are the 6 depth formats explained:

PIXEL_FORMAT_Y8

The 8-bit gray format. Also used for the 8-bit IR data. See fourcc.org for the description and memory layout.

PIXEL_FORMAT_Y8_IR_RELATIVE

The IVCAM specific 8-bit IR data containing the I1-I0 IR image. I1 is the IR image with the coded pattern on and I0 is the IR image with the coded pattern off.

PIXEL_FORMAT_Y16

The 16-bit gray format. Also used for the 16-bit IR data. See fourcc.org for the description and memory layout.

PIXEL_FORMAT_DEPTH

The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

PIXEL_FORMAT_DEPTH_RAW

The depth map data in 16-bit unsigned integer. The value precision is device specific. The application can get the device precision via the QueryDepthUnit function;

PIXEL_FORMAT_DEPTH_F32

The depth map data in 32-bit floating point. The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

I guess the mistery would be what does the YUV stream looks like, as it seems that it does not belong there.

Best of luck, and keep up the good work!

PS: I wish Intel could have some of those awesome hacking events here in Sydney, Australia...

p { margin-bottom: 0.25cm;

samontab — Wed, 11 Feb 2015 01:09:00 GMT

Hi Daniel, great work!, thanks for sharing it here.

I am also working on a Linux version of the SDK for the camera, as I prefer to use Linux. It seems promising because, as you saw, the camera seems to be working fine out of the box. Once I'm finished with that, this app would be cross platform.

Here is some help with your format mistery. The camera actually has 6 pixel depth formats documented:

PIXEL_FORMAT_Y8
The 8-bit gray format. Also used for the 8-bit IR data. See fourcc.org for the description and memory layout.

PIXEL_FORMAT_Y8_IR_RELATIVE
The IVCAM specific 8-bit IR data containing the I1-I0 IR image. I1 is the IR image with the coded pattern on and I0 is the IR image with the coded pattern off.

PIXEL_FORMAT_Y16
The 16-bit gray format. Also used for the 16-bit IR data. See fourcc.org for the description and memory layout.

PIXEL_FORMAT_DEPTH
The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

PIXEL_FORMAT_DEPTH_RAW
The depth map data in 16-bit unsigned integer. The value precision is device specific. The application can get the device precision via the QueryDepthUnit function;

PIXEL_FORMAT_DEPTH_F32
The depth map data in 32-bit floating point. The value precision is in millimeters.

A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.

The only mistery that I see here is the YUV pixel format. It doesn't really make much sense to be there. I wonder what that image looks like...

I hope it helps!

PS: I wish Intel would have those awesome hacking events here in Sydney as well!

I saw the pixel formats in

Daniel_J_1 — Wed, 11 Feb 2015 06:56:24 GMT

I saw the pixel formats in the both the documentation, and in the USB descriptors, but when I select anything I get 16 bit per pixel data. I looked for interleaved frames, and multiplanar formats, and didn't find anything there either. The YUV format shows up as effectively 16 bit little endian depth values too. I'm going to try to figure out how to get the IR stream tomorrow.

mmm, that sounds like maybe

samontab — Wed, 11 Feb 2015 07:15:49 GMT

mmm, that sounds like maybe the stream selection is not working. I am currently requesting the pixels with PIXEL_FORMAT_Y8, and using only 8 bits per pixel to create the depth image, and it is working fine. This means that I am not getting the 16 bit per pixel data stream that you are receiving regardless of the stream you use.

New SDK breaks this utility

Vlad_C_ — Tue, 24 Feb 2015 17:01:12 GMT

New SDK breaks this utility

	PIXEL_FORMAT_Y8	The 8-bit gray format. Also used for the 8-bit IR data. See fourcc.org for the description and memory layout.
	PIXEL_FORMAT_Y8_IR_RELATIVE	The IVCAM specific 8-bit IR data containing the I1-I0 IR image. I1 is the IR image with the coded pattern on and I0 is the IR image with the coded pattern off.
	PIXEL_FORMAT_Y16	The 16-bit gray format. Also used for the 16-bit IR data. See fourcc.org for the description and memory layout.
	PIXEL_FORMAT_DEPTH	The depth map data in 16-bit unsigned integer. The values indicate the distance from an object to the camera's XY plane or the Cartesian depth.The value precision is in millimeters. A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.
	PIXEL_FORMAT_DEPTH_RAW	The depth map data in 16-bit unsigned integer. The value precision is device specific. The application can get the device precision via the QueryDepthUnit function;
	PIXEL_FORMAT_DEPTH_F32	The depth map data in 32-bit floating point. The value precision is in millimeters. A special depth value is used to indicate low-confidence pixels. The application can get the value via the QueryDepthLowConfidenceValue function.