How to use get point cloud of d435 camera in C# ?

[HHa4]

Hi all,

I want to get point cloud of d435 camera in c# . How can I do that ?

[MartyG]

The RealSense SDK 2.0 comes with a point cloud sample program but it is intended for C++, as are the majority of samples. Looking at the code of the small number of C# samples though, their scripts have the # includes in the headers (something that is not mandatory in C# ). So it is possible that the C++ point cloud code may not be difficult to convert to a C# script.

So I would suggest trying the C++ code in a C# script and see how many errors, if any, you get, and work from that point onwards.

https://github.com/IntelRealSense/librealsense/tree/master/examples/pointcloud librealsense/examples/pointcloud at master · IntelRealSense/librealsense · GitHub

For comparison of C++ code versus C# in the RealSense SDK 2.0, you can see the two types of sample listed on this page:

https://github.com/IntelRealSense/librealsense/tree/master/examples librealsense/examples at master · IntelRealSense/librealsense · GitHub

[HHa4]

Thank you so much MartyG.

[MartyG]

I re-checked the RealSense SDK 2.0 information. Though support for "C" was added in the latest 2.9.1 release, I believe that this refers to the original C language, not C# . This would explain why the "C" scripts have # include headers. It may be that SDK 2.0 does not have C# support yet. I am sorry for the confusion.

[jb455]

It may be that SDK 2.0 does not have C# support yet.

In case anyone reads this and gets confused, https://github.com/IntelRealSense/librealsense/tree/master/wrappers/csharp C# is (mostly) supported currently.

[MartyG]

Thanks JB. The matter of C# came up on another message a little later after this post and it was at that point that I remembered that the terms C# and .NET are used together by some people (e.g "I wish there was a C# version of this script as I need .NET integration"). The documentation could do with better signposting about this, such as referring to .NET / C# support instead of just .NET.

[SChop3]

Thanks for this, I'm trying to do the same and was wondering whether I was missing a method somewhere. Quick question with regard to intrinsics. What instrics do you use? From what I've seen of custom calibrations, you calculate intrinsics for left and right IR cameras and the RGB camera. Is the depth stream given with respect to the left IR camera or is a virtual camera created?

Any help greatly appreciated,

Wiredchop

[jb455]

I'm using depth aligned to colour so I use the colour camera intrinsics. Are you aware that you can get the intrinsics from the API? So you don't have to calculate them yourself. Like this:

PipelineProfile profile = pipeline.Start(cfg);

var stream = (VideoStreamProfile)profile.GetStream(Stream.Color);

var colourIntrinsics = stream.GetIntrinsics();

So, I assume, doing the same for Stream.Depth will give you the intrinsics you require. I think I read somewhere that the depth is aligned to the left camera, or perhaps it was the right one. I'm pretty sure it's not a virtual viewpoint though.

FYI, the method used internally to calculate the point cloud is https://github.com/IntelRealSense/librealsense/blob/master/include/librealsense2/rsutil.h# L46 here. I ignored the distortion stuff but you may prefer to include it.

[SChop3]

Fantastic, thanks for your help

[jb455]

The easiest thing to do for now until the PointCloud class is wrapped up for C# is to obtain the camera intrinsics (for the depth camera if using the raw depth, or colour camera if using depth aligned to colour) and implement the https://threeconstants.wordpress.com/tag/pinhole-camera-model/ Pinhole Camera Model (which is what the PointCloud class does internally). From the intrinsics you get from the camera, ppx & ppy are camera_principle_point_x and y in the link and fx & fy are camera_focal_length_x and y.

You need to loop through each pixel in the depth image, obtain the depth value (most efficient is through the depth data pointer), multiply it by the depth scale, and apply the x_3D and y_3D formulae given in the link.