Converting RealSense D415 bag file to PCD

[AMahm3]

I've received my RealSense D415 10 days ago (I'm not a developer) and I was trying to use it for room scanning. I've recorded a *.bag file through the viewer and the Unity wrapper and I'm trying to convert the .bag file to a point cloud; I've tried using ROS:

rosrun pcl_ros bag_to_pcd

But the output directory is always empty, so is there a way that I can use to get a point cloud (then create a mesh) from the D415?

BTW: I've tried the *.PLY in the viewer and that's not a 3D scan it's simply a depth snap/shot of what is directly in-front of the camera not a real 3D scan. RecFusion can be used but it has limitations on the size of the object you want to scan.

Thanks in advance.

[GRedd]

Hi GamesMechanic.

To be honest I am confused as to what they are targeting. Surely it is more accurate point cloud data at a higher sample frequency? If it is low accuracy at low frequency....well we already had that with old technology.

So assuming that it is more accurate point cloud data at a higher sample frequency (which to my logic it can only be), why is it so hard to get poingtcloud data with a timestamp? This should be fundamental and easy to extract. Yet there are dozens of threads on this forum of people struggling to extract such data.

I could understand if this was a 3 person startup trying to get off the ground. But this is Intel, a billion dollar corporation previously associated with excellence. Why the mediocrity? It's all very srange to me.

[GRedd]

Hi Marty, just allow us to extract all point cloud data with a timestamp in the viewer for all frames captured in a session via a single export from a captured sequence with the ability to specify file format and you will solve so many people problems on this forum. Intel has the resources to do this. So, do it.

[MartyG]

The SDK has a tool called Data Collect that is a console application that can export frame statistics (but not the content) to a .csv file. CSV is a format that stores data in a tabular format suitable for databases and spreadsheets.

https://github.com/IntelRealSense/librealsense/tree/master/tools/data-collect librealsense/tools/data-collect at master · IntelRealSense/librealsense · GitHub

[AMahm3]

Hello MartyG,

I went through the resources and that's regular photogrammetry we can already make that using a mobile phone camera and a software like photoscan or 3DF, you don't need a depth sensing camera for that, I was hoping to use the D400 to improve the work flow of our photogrammetry not to make it far more complex. Thanks but as I've said a tool to generate the model in the viewer would be great or at least a tutorial on how to use ROS to do so, I've tried with Unity, librealsense and ROS but couldn't achieve anything. I've two cameras (D415) with some potential alas couldn't extract the data I need from them.

[GRedd]

Hi Marty, thanks again. Unfortunately, as specified by the tools documentation 'The tool does not record the content of the frames'

So again in order to collect and extract all point cloud data with a timestamp, we would need to piece things together using multiple 'tools'.

And once again, unfortunately it seems nobody has been able to do with this. In all sincerity I would again suggest as above that Intel just allow us to extract all point cloud data with a timestamp in the viewer for all frames captured in a session via a single export from a captured sequence with the ability to specify file format and you will solve so many peoples problems on this forum.

If a goal of this technology is to allow is to extract more accurate point cloud data with a timestamp at higher frequency than previously possible, as it is touted as being capable of doing then lets provide an easy way to do that. Let's target this as a a minimum starting point.

I think it will be great for everyone, and certainly much appreciated.

[MartyG]

The official location for submitting feature requests is the "Issues" page of the RealSense GitHub site, where the RealSense engineers and developers reside. Please leave a message here detailing the features that you would like to see. You can post your request by clicking on the 'New issue' button on the page.

https://github.com/IntelRealSense/librealsense/issues Issues · IntelRealSense/librealsense · GitHub

[AMahm3]

Hello MartyG,

Thanks for your clarifications, I agree with gpred that a simple way to export the generated point cloud would be great instead of jumping through hoops. I'd like to say that this hardware has a huge potential and I can see many industrial applications for it, unfortunately instead of me focusing on trying to find a pipeline for these applications, I've to struggle with getting a simple export. I think you should consider that developers aren't just the people who can create software but are those who can develop applications for hardware that wasn't there before.

[MartyG]

As a developer myself, I do understand your frustrations. I am not an Intel employee though and have no influence over RealSense development. The RealSense GitHub is the best place for users to have direct dialog with the RealSense developers and engineers.

[GRedd]

I'll submit this feature request but will leave it here for a day, incase anyone wants to add to it,

Viewer feature requests,

- Playback slider to show milliseconds

- Hovering the mouse over image in 2D and 3D mode should show x, x and z coordinates of closest point to 4 decimal places

When playing back a recorded .bag file,

- Ability to export all frames to a specified folder (or a specified time range)

- This export option should allow the specification to choose what to export, eg. depth images, .ply, .pcd etc- The timestamp of all frames should be exported to the same folder, either appended to each file name or in a separate readable file which identifies the timestamp of each output file- An output file which records all the camera settings that were used during the capture- The ability to skip forward and back one frame (this is currently disabled)

[AMahm3]

I've found that you can use the LabVIEW wrapper to get the time stamp you were asking for, you can find it here in this link:

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/labview librealsense/wrappers/labview at master · IntelRealSense/librealsense · GitHub

Have you found any solution for constructing large scans, intel doesn't seem to be interested in even providing a simple tutorial covering using ROS nodelet.

[GRedd]

Thanks for that. I only just posted the feature request now. I suspect it will be ignored, but worth a go.

Will be back to working on this in a couple of weeks, so will post if I have any breakthroughs.

Microsoft is releasing Project Kinect for Azure soon.

https://azure.microsoft.com/en-us/campaigns/kinect/ Project Kinect for Azure | Microsoft Azure

Will be looking into using this as an alternative to realsense. It seems Microsoft might have a better appreciation of the research and business applications of this tech, rather than it just being a consumer camera in a laptop. There sdk is also likely to be more professional and better documented than Intel. So fingers crossed. In the meantime I'll persist with realsense, as despite the poor support it is still promising tech.

[AMahm3]

Thanks for the information.

[VT000]

Hello,

I have a similar problem.

How can I extract the point cloud and RGB, XYZ arrays from "bag" file with Matlab, when I record a scene using RealSense Viewer in Win10? I am using RealSense D415&D435 and the data are stored in "bag" file.

I tried with matlab functions: rosbag, select, readMessages, but I can't extract the point cloud and the XYZ array of the scene/frame. I only got the RGB or the INFRA image data tih mentioned functions (the dimensions should be 640x480x3).

Or, how can I directly store the captured data as pointcloud1 message with XYZ and RGB arrays (307200x3 dimsnesions)?

I am using Win10 and Matlab. I need the point cloud data for image processing.

Can you help me? Also, I have the Matlab wrapper, but I can't extract the point cloud with the pointcloud_example function.

Thank you in advance.

[AMahm3]

Hi, I've converted the bag file to ply through the above function, for my application we're currently using RecFusion and a software from Hexagon. Hope this helps.

[VT000]

Hello,

I tried, but I have the same error in align.m file.

Can you provide me please a syntax in command window? Maybe I did something wrong.

I only need the point cloud with XYZ and RGB arrays. I can not believe that this is impossible with the wrapper and Viewer.

Thank you in advance.

[AMahm3]

Hello, I've created an m file pasted this code; ptCloud = pcread('D:\Flash\1.ply'); pcshow(ptCloud); pcwrite(ptCloud,'D:\testpcd.pcd','Encoding','ascii'); change the directories to direct to your directories and what I get is this; hope this helps.

[VT000]

Hello,

Ok, you have converted a rosbag file to ply file? After the conversion, the resolution was preserved as in the rosbag file? When I export the scene as a ply file, the resolution is always lower then 640x480x3. I need this 640x480x3 resolution (307200x3 XYZ and RGB arrays). Is this possible?

I changed the pointcloud_example function from RealSense Matlab wrapper to pull the stream from the bag file. I have got some variables and objects, but I can't access to 'poincloud' object.

[AMahm3]

I can't help with the resolution, I've shared what I know, I think you should take a look at OpenNi the your application, that's all I can help with.