topic Re: MATLAB RGB point cloud in Items with no label

MATLAB RGB point cloud

JBos3 — Wed, 26 Sep 2018 17:52:32 GMT

I would like to acquire and plot RGB colored point clouds from a D435 in MATLAB. The desired output is analogous to the rs-pointcloud.cpp example.

I reviewed rs-pointcloud.cpp and the MATLAB wrappers, and I'm able to acquire a point cloud, however I can't find how to properly color and plot the point cloud. Below is my code, which I hope will be useful to others.

There are two options that I'm considering:

1) is there a way for MATLAB to consume the realsense.pointcloud() object directly?

2) Assuming the answer to 1) is no, can I access and use the vertices and textures to reformat the RS point cloud object into a MATLAB point cloud object?

points.get_vertices(); and points.get_texture_coordinates(); seem to get correct values, but I'm not sure how to use them in MATLAB.

Thanks!

MATLAB:

% Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% define point cloud object

pcl_obj = realsense.pointcloud();

% define colorizer to give point cloud color

colorizer = realsense.colorizer();

% Start streaming on an arbitrary camera with default settings

profile = pipe.start();

% Get frames. We discard the first couple to allow

% the camera time to settle

for i = 1:5

frames = pipe.wait_for_frames();

end

% Stop streaming

pipe.stop();

% Select depth frame

depth = frames.get_depth_frame();

% get point cloud points without color

points = pcl_obj.calculate(depth);

% get texture mapping

color = frames.get_color_frame();

% map point cloud to color

pcl_obj.map_to(color);

% get vertices (nx3)

vertices = points.get_vertices();

% get texture coordinates (nx2)

tex_coords = points.get_texture_coordinates();

Re: MATLAB RGB point cloud

idata — Thu, 27 Sep 2018 17:33:51 GMT

Hello jjbos,

Thank you for our interest in the Intel RealSense D435 camera.

Please make sure that you have the latest firmware on your camera and librealsense version 2.16.1.

We have modified your code (it follow the export_to_ply python examples).

function pc_example

% Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% define point cloud object

pcl_obj = realsense.pointcloud();

% Start streaming on an arbitrary camera with default settings

pipe.start();

% Get frames. We discard the first couple to allow

% the camera time to settle

for i = 1:5

frames = pipe.wait_for_frames();

end

% Select depth frame

depth = frames.get_depth_frame();

% get color frame

color = frames.get_color_frame();

% map point cloud to color

pcl_obj.map_to(color);

% get point cloud points without color

pnts = pcl_obj.calculate(depth);

%export ply file

pnts.export_to_ply('pc_example.ply',color);

pipe.stop();

end

Please let us know if this works with you as well!

Best regards,

Eliza

Re: MATLAB RGB point cloud

JBos3 — Fri, 28 Sep 2018 18:54:39 GMT

Hi Eliza,

Thank you. This is helpful, and *almost* what I need.

I would like to visualize the colored point cloud live stream, just like the rs-pointcloud.cpp example... but in MATLAB.

Is there a way to convert the RealSense point cloud object to something MATLAB can manipulate without having to write and then read an intermediate .ply file?

Thanks,

-Joe

Re: MATLAB RGB point cloud

idata — Mon, 01 Oct 2018 16:27:27 GMT

Hello jjbos,

You check out this post that I believe can help you with your project: https://www.mathworks.com/matlabcentral/answers/7202-processing-on-a-live-webcam-streaming-video https://www.mathworks.com/matlabcentral/answers/7202-processing-on-a-live-webcam-streaming-video. Please note that you might need the Image Acquisition Toolbox.

Thank you,

Eliza

Re: MATLAB RGB point cloud

JBos3 — Mon, 01 Oct 2018 17:25:00 GMT

Hi Eliza,

Thanks for looking but this does not address the issue. The link you sent relates to processing 2D images. Please correct me if you saw something related to point clouds.

Again, I need to convert the RealSense point cloud object and other data (e.g. pc.vertices, pc.texture, color image, depth image) into a MATLAB point cloud without going through an intermediate .ply file. MATLAB requires point cloud vertices and RGB values for each vertex.

It appears I can get the correct vertices from the point cloud object but I don't know how to get the corresponding RGB values to color the point cloud. Can you help with this?

Thanks,

-Joe

Here is how I get the point cloud vertices (x,y,z values), but again I need the corresponding RGB values:

% Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% define point cloud object

pcl_obj = realsense.pointcloud();

% define colorizer to give point cloud color

colorizer = realsense.colorizer();

% Start streaming on an arbitrary camera with default settings

profile = pipe.start();

% Get frames. Discard the first several to allow

% the camera time to settle

for i = 1:5

frames = pipe.wait_for_frames();

end

% Select depth frame

depth = frames.get_depth_frame();

% get texture mapping

color = frames.get_color_frame();

% map point cloud to color

pcl_obj.map_to(color);

% get point cloud points without color

points = pcl_obj.calculate(depth);

% get vertices (nx3)

vertices = points.get_vertices();

% get texture coordinates (nx2)

tex_coords = points.get_texture_coordinates();

Re: MATLAB RGB point cloud

idata — Wed, 03 Oct 2018 15:26:56 GMT

Hello jjbos,

The RealSense team is working on an example to demonstrate how to do this but I don't have a timeline for when it will be completed. Meanwhile, you can check out this code: https://communities.intel.com/community/tech/realsense/blog/2018/10/03/unofficial-pointcloud-stream-viewer-in-matlab https://communities.intel.com/community/tech/realsense/blog/2018/10/03/unofficial-pointcloud-stream-viewer-in-matlab.

We need to figure out how to use the pcshow command and to use the same algorithm in the align.cpp example to make the depth and color frames the same size.

Regards,

Jesus G.

Intel Customer Support

Re: MATLAB RGB point cloud

JBos3 — Wed, 03 Oct 2018 20:20:03 GMT

Thank you. I very much look forward to the colored point cloud example in MATLAB.

-Joe

Re: MATLAB RGB point cloud

VHess — Fri, 21 Dec 2018 07:43:53 GMT

I made this code that I thinks solve this problem:

% Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% Make Colorizer object to prettify depth output

colorizer = realsense.colorizer();

% define point cloud object

pcl_obj = realsense.pointcloud();

% Start streaming with default settings

profile = pipe.start();

align_to = realsense.stream.color;

alignedFs = realsense.align(align_to);

% Get streaming device's name

dev = profile.get_device();

name = dev.get_info(realsense.camera_info.name);

% Get frames. We discard the first couple to allow

% the camera time to settle

for i = 1:5

fs = pipe.wait_for_frames();

end

%create a point cloud player

player1 = pcplayer([-1 1],[-1 1],[-1 1]);

frameCount =0;

while isOpen(player1) && frameCount < 2000

frameCount = frameCount+1;

fs = pipe.wait_for_frames();

%align the depth frames to the color stream

aligned_frames = alignedFs.process(fs);

depth = aligned_frames.get_depth_frame();

color = fs.get_color_frame();

%get the points cloud based on the aligned depth stream

pnts = pcl_obj.calculate(depth);

pcl_obj.map_to(color);

colordata = color.get_data();

colordatavector = [colordata(1:3:end)',colordata(2:3:end)',colordata(3:3:end)'];

vertices = pnts.get_vertices();

view(player1,vertices,colordatavector)

end

% Stop streaming

pipe.stop();

Re: MATLAB RGB point cloud

SVerg — Thu, 03 Jan 2019 22:22:13 GMT

Hello! I have a problem with the code, the function

alignedFs = realsense.align (align_to);

(Error using realsense.align Error: File: align.m Line: 9 Column: 20 A constructor call to superclass realsense.filter appears after the object is used, or after a return)

Apparently there is an error in the code of align.m, could you help me with this matter? I would greatly appreciate it.

Regards,

S.V.I

Re: MATLAB RGB point cloud

VHess — Tue, 08 Jan 2019 00:48:18 GMT

@SVerg I had to comment out validateattributes on line 17 in align.m to get it to work, but I'm not sure if that is related to your issue.

Re: MATLAB RGB point cloud

JBos3 — Fri, 11 Jan 2019 06:17:59 GMT

Thanks VHess! Works like a charm. Cheers.

Re: MATLAB RGB point cloud

SonuMathur — Sat, 29 Aug 2020 06:09:15 GMT

In which for you save the point cloud...?

Re: MATLAB RGB point cloud

Pcmehdi — Thu, 24 Feb 2022 07:27:28 GMT

Hello VHess,

I run your codes but I have faced an error as below:

Error using realsense.align
Error: File: align.m Line: 4 Column: 1
Illegal use of reserved keyword "classdef".
Error in realsense.VHess (line 21)
alignedFs = realsense.align(align_to);

I am using Matlab2020b!!

Do you have any idea what the issue might be?