topic Re: d435 depth camera in Items with no label

d435 depth camera

EAlta — Tue, 31 Jul 2018 23:45:50 GMT

How can I get the depth (in meters for example) of each pixel of the depth image, given by the realsense d435 depth camera, obtained by the RealSense Viewer?

Re: d435 depth camera

MartyG — Wed, 01 Aug 2018 05:37:05 GMT

The discussion linked to below may be of use to you. It contains code references for getting pixel depth with just SDK 2.0, or by using SDK 2.0 in combination with the OpenCV vision software.

Re: d435 depth camera

EAlta — Wed, 01 Aug 2018 13:24:02 GMT

Im using matlab for the postprocessing, so Im obtaining the sensor´s data from the Real Sense Viewer or python and I need to make use of the distance in meters in matlab.

Re: d435 depth camera

MartyG — Wed, 01 Aug 2018 13:35:53 GMT

There is a discussion on using raw RealSense depth data in MATLAB. Does this help?

Edit: this one too:

Re: d435 depth camera

EAlta — Fri, 03 Aug 2018 04:06:34 GMT

Thanks for your answer MartyG.

Let me explain the issue I have in a better way.

I obtained a depth image, RGB image and Point cloud (.pyd) from the intel RealSense Viewer. Then I made a segmentation process in matlab, so I deleted some points of the original point cloud, but Im still having a .pyd file. Now I need to convert this .pyd file (the processed point cloud) into a depth image, either using matlab or the Intel´s SDK. How can I do it?

I really apreciate your help.

Re: d435 depth camera

MartyG — Fri, 03 Aug 2018 07:33:05 GMT

Getting a .pyd file to load into MATLAB without crashing does not seem to be easy. The most useful references I found were:

https://stackoverflow.com/questions/7811387/embedding-python-in-matlab Embedding Python in MATLAB - Stack Overflow

https://mail.scipy.org/pipermail/numpy-discussion/2009-November/046676.html [Numpy-discussion] bus error in embedded numpy

It sounds as though it would be much easier to process the point cloud in the RealSense Viewer as much as you can, export it as a ply and import it into MATLAB. There are a couple of ply tools available for MATLAB.

https://uk.mathworks.com/matlabcentral/fileexchange/54041-plytomat plyToMat - File Exchange - MATLAB Central

https://uk.mathworks.com/matlabcentral/fileexchange/55968-ply-file-utilities?s_tid=FX_rc2_behav PLY File Utilities - File Exchange - MATLAB Central

Re: d435 depth camera

EAlta — Fri, 03 Aug 2018 11:52:08 GMT

I made a mistake in my las message, when i wrote .pyd, y really meant .ply : ( I obtained a depth image, RGB image and Point cloud (.ply) from the intel RealSense Viewer. Then I made a segmentation process in matlab, so I deleted some points of the original point cloud, but Im still having a .ply file. )

I have alrready made some modifications to the original point cloud in matlab and I saved it as another point cloud (.ply). Now I need to convert this .ply file (the processed point cloud) into a depth image, either using matlab or the Intel´s SDK.

Original data obtained from Intel´s SDK Viewer:

a) Original depth image (I alrready have it in gray scale). b) Original point cloud

Processed pointcloud, in matlab (it is also a .ply file). It is the same as the original, but I deleted some points:

c) I need to convert this one into a depth image.

Re: d435 depth camera

MartyG — Fri, 03 Aug 2018 12:48:54 GMT

This link sounds like what you are trying to achieve:

https://uk.mathworks.com/matlabcentral/answers/228917-3d-point-cloud-to-2d-depth-image 3D Point Cloud to 2D depth image - MATLAB Answers - MATLAB Central

https://uk.mathworks.com/matlabcentral/fileexchange/55031-pointcloud2image-x-y-z-numr-numc pointcloud2image( x,y,z,numr,numc ) - File Exchange - MATLAB Central

Re: d435 depth camera

EAlta — Fri, 03 Aug 2018 14:43:38 GMT

Dou you think is possible to import the .ply file to python and use the SDK´s function "rs2_project_point_to_pixel" to make the convertion?

Re: d435 depth camera

MartyG — Fri, 03 Aug 2018 15:02:54 GMT

This week, Intel highlighted a point cloud system for Python called Pyntcloud that allows the loading in of ply files and the performing of a large number of different types of operation on it. It contains some example programs..

https://github.com/daavoo/pyntcloud GitHub - daavoo/pyntcloud: pyntcloud is a Python library for working with 3D point clouds.

One of these example programs allows the visualization of the point cloud as a color map.

Re: d435 depth camera

EAlta — Fri, 03 Aug 2018 18:07:42 GMT

But once having the .ply on python, I would have to use the function "https://github.com/IntelRealSense/librealsense/blob/5e73f7bb906a3cbec8ae43e888f182cc56c18692/include/librealsense2/rsutil.h# L15 rs2_project_point_to_pixel" that needs the intrinsic parameters.

Is there a way to obtain this intrinsic parameters from the Real Sense Viewer?

Are the intrinsic parameters always the same for the camera or they change accoding to the scene (like maximum distance measured in the scene)?

Thank you for your time.

Re: d435 depth camera

MartyG — Fri, 03 Aug 2018 22:04:32 GMT

Apologies for the delay in responding, I was considering how best to answer your questions.

Yes, intrinsics can change, especially when calibrating the camera. The sample program 'Sensor Control' provides a pre-made means to interface with the camera's details.

https://github.com/IntelRealSense/librealsense/tree/814541662529bf6a0d6178db95e1f948a85dbfa7/examples/sensor-control librealsense/examples/sensor-control at 814541662529bf6a0d6178db95e1f948a85dbfa7 · IntelRealSense/librealsense · GitHub

The API How To page also provides scripting for getting field of view and video stream intrinsics.

https://github.com/IntelRealSense/librealsense/wiki/API-How-To# get-video-stream-intrinsics API How To · IntelRealSense/librealsense Wiki · GitHub

Re: d435 depth camera

EAlta — Sat, 04 Aug 2018 03:34:19 GMT

Ok MartyG, I´m going to read the information and as soon as I have some results, I willl let you know.

Regards.

Re: d435 depth camera

EAlta — Sat, 04 Aug 2018 10:43:00 GMT

MartyG intel_admin intel_corp

You mentioned that intrinsics can change, especially when calibrating the camera, but can intrinsics change while acquiring data from the sensor?

Lets supose Im acquiring data from the sensor and I get 100 frames, and each frame has a pointcloud and depth image,

Is it possible that the camera has taken different intrinsic values while capturing the video, and therefore having different intrinsic values for each frame?

I am going to process each frame's pointcloud independently, so I want to know if it is necessary to obtain the intrinsic values for each frame while taking the video (this would be complicated) ?

I just want to be able to convert a processed pointcloud to a depth image.

Regards.

Re: d435 depth camera

MartyG — Sat, 04 Aug 2018 11:04:19 GMT

Unless the camera is absolutely motionless in position, there are very likely to be variations in the intrinsics. I have never heard of anyone having to manually update the intrinsics on every frame though. The RealSense SDK should automatically take care of those calculations as factors such as distance change.

Re: d435 depth camera

EAlta — Mon, 06 Aug 2018 05:31:09 GMT

Hi again MartyG

Do you know how to get the intrinsic values with python?

Re: d435 depth camera

MartyG — Mon, 06 Aug 2018 05:59:40 GMT

I tracked down the part of the SDK's Python wrapper that handles the creation and modification of intrnsics and extrinsics. It may be a starting point in working out how to access those functions in your own work.

https://github.com/IntelRealSense/librealsense/commit/658d672b97b64257afdbcf18cb60bcea38fe5f87# diff-063dde808a39e40b353ac48a477cbb5bR155 Merge pull request # 1138 from zivsha/register_extrin_python · IntelRealSense/librealsense@658d672 · GitHub

The green parts of the listing show the current version of the code, and the red parts usually indicate parts that have been removed from the script by the most recent changes. Once you find a block of green code relating to intrinsics and extrinsics, you can find it within the main python.cpp wrapper script by looking for the same script line numbers (e.g 156).

https://github.com/IntelRealSense/librealsense/blob/658d672b97b64257afdbcf18cb60bcea38fe5f87/wrappers/python/python.cpp librealsense/python.cpp at 658d672b97b64257afdbcf18cb60bcea38fe5f87 · IntelRealSense/librealsense · GitHub

It might be easier though just to use the original Python script that the SDK's intrinsics / extrinsics support is based on.

https://github.com/IntelRealSense/librealsense/pull/1138 [Python] Allow extrinsics and intrinsics creation and modification by zivsha · Pull Request # 1138 · IntelRealSense/libre…

Re: d435 depth camera

jb455 — Mon, 06 Aug 2018 08:37:25 GMT

Intrinsics don't fundamentally change, they're a measurement of the properties of each individual camera. However, it's hard to measure them directly, so they are estimated by a calibration process. This estimation may change slightly when calibration is done, depending on the quality of the calibration, but the actual values should be constant for the life of the camera, not dependent on scene or anything.

The current Intel calibration tool doesn't calibrate intrinsics though, just extrinsics between the depth cameras and possibly depth/colour (which can change if the camera is dropped or exposed to high temperatures/humidity etc), so unless you do your own intrinsic calibration process and write it to the camera (there is a tool for writing intrinsics/extrinsics to a camera included in the calibration API tools) they will remain constant.

Re: d435 depth camera

EAlta — Mon, 06 Aug 2018 14:44:13 GMT

Thankyou jb455

I'm trying to get the intrinsics from my D435 depth camera with python and this is what i get:

>>> import pyrealsense2 as rs2

>>> intrin = rs2.intrinsics()

>>> intrin

width: 0, height: 0, ppx: 0, ppy: 0, fx: 0, fy: 0, model: None, coeffs: [0, 0, 0, 0, 0]

Do you have an idea of what might be happening?

Re: d435 depth camera

jb455 — Mon, 06 Aug 2018 14:52:16 GMT

It looks like you're just creating a new Intrinsics object filled with default values (0).

I don't know what the syntax is in Python, but in C# , you can get the intrinsics of a stream like so:

PipelineProfile profile = pipeline.Start();

var stream = (VideoStreamProfile)profile.GetStream(Stream.Depth);

var intrinsics = stream.GetIntrinsics();

I assume it'll be similar in Python.