- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would like to acquire and plot RGB colored point clouds from a D435 in MATLAB. The desired output is analogous to the rs-pointcloud.cpp example.
I reviewed rs-pointcloud.cpp and the MATLAB wrappers, and I'm able to acquire a point cloud, however I can't find how to properly color and plot the point cloud. Below is my code, which I hope will be useful to others.
There are two options that I'm considering:
1) is there a way for MATLAB to consume the realsense.pointcloud() object directly?
2) Assuming the answer to 1) is no, can I access and use the vertices and textures to reformat the RS point cloud object into a MATLAB point cloud object?
points.get_vertices(); and points.get_texture_coordinates(); seem to get correct values, but I'm not sure how to use them in MATLAB.
Thanks!
MATLAB:
% Make Pipeline object to manage streaming
pipe = realsense.pipeline();
% define point cloud object
pcl_obj = realsense.pointcloud();
% define colorizer to give point cloud color
colorizer = realsense.colorizer();
% Start streaming on an arbitrary camera with default settings
profile = pipe.start();
% Get frames. We discard the first couple to allow
% the camera time to settle
for i = 1:5
frames = pipe.wait_for_frames();
end
% Stop streaming
pipe.stop();
% Select depth frame
depth = frames.get_depth_frame();
% get point cloud points without color
points = pcl_obj.calculate(depth);
% get texture mapping
color = frames.get_color_frame();
% map point cloud to color
pcl_obj.map_to(color);
% get vertices (nx3)
vertices = points.get_vertices();
% get texture coordinates (nx2)
tex_coords = points.get_texture_coordinates();
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I made this code that I thinks solve this problem:
% Make Pipeline object to manage streaming
pipe = realsense.pipeline();
% Make Colorizer object to prettify depth output
colorizer = realsense.colorizer();
% define point cloud object
pcl_obj = realsense.pointcloud();
% Start streaming with default settings
profile = pipe.start();
align_to = realsense.stream.color;
alignedFs = realsense.align(align_to);
% Get streaming device's name
dev = profile.get_device();
name = dev.get_info(realsense.camera_info.name);
% Get frames. We discard the first couple to allow
% the camera time to settle
for i = 1:5
fs = pipe.wait_for_frames();
end
%create a point cloud player
player1 = pcplayer([-1 1],[-1 1],[-1 1]);
frameCount =0;
while isOpen(player1) && frameCount < 2000
frameCount = frameCount+1;
fs = pipe.wait_for_frames();
%align the depth frames to the color stream
aligned_frames = alignedFs.process(fs);
depth = aligned_frames.get_depth_frame();
color = fs.get_color_frame();
%get the points cloud based on the aligned depth stream
pnts = pcl_obj.calculate(depth);
pcl_obj.map_to(color);
colordata = color.get_data();
colordatavector = [colordata(1:3:end)',colordata(2:3:end)',colordata(3:3:end)'];
vertices = pnts.get_vertices();
view(player1,vertices,colordatavector)
end
% Stop streaming
pipe.stop();
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello jjbos,
Thank you for our interest in the Intel RealSense D435 camera.
Please make sure that you have the latest firmware on your camera and librealsense version 2.16.1.
We have modified your code (it follow the export_to_ply python examples).
function pc_example
% Make Pipeline object to manage streaming
pipe = realsense.pipeline();
% define point cloud object
pcl_obj = realsense.pointcloud();
% Start streaming on an arbitrary camera with default settings
pipe.start();
% Get frames. We discard the first couple to allow
% the camera time to settle
for i = 1:5
frames = pipe.wait_for_frames();
end
% Select depth frame
depth = frames.get_depth_frame();
% get color frame
color = frames.get_color_frame();
% map point cloud to color
pcl_obj.map_to(color);
% get point cloud points without color
pnts = pcl_obj.calculate(depth);
%export ply file
pnts.export_to_ply('pc_example.ply',color);
pipe.stop();
end
Please let us know if this works with you as well!
Best regards,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Eliza,
Thank you. This is helpful, and *almost* what I need.
I would like to visualize the colored point cloud live stream, just like the rs-pointcloud.cpp example... but in MATLAB.
Is there a way to convert the RealSense point cloud object to something MATLAB can manipulate without having to write and then read an intermediate .ply file?
Thanks,
-Joe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello jjbos,
You check out this post that I believe can help you with your project: https://www.mathworks.com/matlabcentral/answers/7202-processing-on-a-live-webcam-streaming-video https://www.mathworks.com/matlabcentral/answers/7202-processing-on-a-live-webcam-streaming-video. Please note that you might need the Image Acquisition Toolbox.
Thank you,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Eliza,
Thanks for looking but this does not address the issue. The link you sent relates to processing 2D images. Please correct me if you saw something related to point clouds.
Again, I need to convert the RealSense point cloud object and other data (e.g. pc.vertices, pc.texture, color image, depth image) into a MATLAB point cloud without going through an intermediate .ply file. MATLAB requires point cloud vertices and RGB values for each vertex.
It appears I can get the correct vertices from the point cloud object but I don't know how to get the corresponding RGB values to color the point cloud. Can you help with this?
Thanks,
-Joe
Here is how I get the point cloud vertices (x,y,z values), but again I need the corresponding RGB values:
% Make Pipeline object to manage streaming
pipe = realsense.pipeline();
% define point cloud object
pcl_obj = realsense.pointcloud();
% define colorizer to give point cloud color
colorizer = realsense.colorizer();
% Start streaming on an arbitrary camera with default settings
profile = pipe.start();
% Get frames. Discard the first several to allow
% the camera time to settle
for i = 1:5
frames = pipe.wait_for_frames();
end
% Select depth frame
depth = frames.get_depth_frame();
% get texture mapping
color = frames.get_color_frame();
% map point cloud to color
pcl_obj.map_to(color);
% get point cloud points without color
points = pcl_obj.calculate(depth);
% get vertices (nx3)
vertices = points.get_vertices();
% get texture coordinates (nx2)
tex_coords = points.get_texture_coordinates();
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello jjbos,
The RealSense team is working on an example to demonstrate how to do this but I don't have a timeline for when it will be completed. Meanwhile, you can check out this code: https://communities.intel.com/community/tech/realsense/blog/2018/10/03/unofficial-pointcloud-stream-viewer-in-matlab https://communities.intel.com/community/tech/realsense/blog/2018/10/03/unofficial-pointcloud-stream-viewer-in-matlab.
We need to figure out how to use the pcshow command and to use the same algorithm in the align.cpp example to make the depth and color frames the same size.
Regards,
Jesus G.
Intel Customer Support
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you. I very much look forward to the colored point cloud example in MATLAB.
-Joe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I made this code that I thinks solve this problem:
% Make Pipeline object to manage streaming
pipe = realsense.pipeline();
% Make Colorizer object to prettify depth output
colorizer = realsense.colorizer();
% define point cloud object
pcl_obj = realsense.pointcloud();
% Start streaming with default settings
profile = pipe.start();
align_to = realsense.stream.color;
alignedFs = realsense.align(align_to);
% Get streaming device's name
dev = profile.get_device();
name = dev.get_info(realsense.camera_info.name);
% Get frames. We discard the first couple to allow
% the camera time to settle
for i = 1:5
fs = pipe.wait_for_frames();
end
%create a point cloud player
player1 = pcplayer([-1 1],[-1 1],[-1 1]);
frameCount =0;
while isOpen(player1) && frameCount < 2000
frameCount = frameCount+1;
fs = pipe.wait_for_frames();
%align the depth frames to the color stream
aligned_frames = alignedFs.process(fs);
depth = aligned_frames.get_depth_frame();
color = fs.get_color_frame();
%get the points cloud based on the aligned depth stream
pnts = pcl_obj.calculate(depth);
pcl_obj.map_to(color);
colordata = color.get_data();
colordatavector = [colordata(1:3:end)',colordata(2:3:end)',colordata(3:3:end)'];
vertices = pnts.get_vertices();
view(player1,vertices,colordatavector)
end
% Stop streaming
pipe.stop();
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello VHess,
I run your codes but I have faced an error as below:
Error using realsense.align
Error: File: align.m Line: 4 Column: 1
Illegal use of reserved keyword "classdef".
Error in realsense.VHess (line 21)
alignedFs = realsense.align(align_to);
I am using Matlab2020b!!
Do you have any idea what the issue might be?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello! I have a problem with the code, the function
alignedFs = realsense.align (align_to);
(Error using realsense.align Error: File: align.m Line: 9 Column: 20 A constructor call to superclass realsense.filter appears after the object is used, or after a return)
Apparently there is an error in the code of align.m, could you help me with this matter? I would greatly appreciate it.
Regards,
S.V.I
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@SVerg I had to comment out validateattributes on line 17 in align.m to get it to work, but I'm not sure if that is related to your issue.
![](/skins/images/D2683F18326913BBA0436CB7114DD569/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page