- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
I am using ZR300 on Ubuntu 16.04. I have written a driver for a system which gives images(colors, depth, params) from driver and converts to my format. And in the system I am using Person tracking module to track the person. Below is the procedure I am following.
- Read Image from device and convert images (RGB, Depth) and params into my format.
- Convert images and params back to realsense format to use the SDK for persontracking module.
I am able to read rgb, depth images well but the problem is when I use them with person tracking module. Below code might make my problem clear.
Intel::RealSense::PersonTracking::PersonTrackingData::PointCombined centerMass = personTrackingData->QueryCenterMass();after this call "image" coordinates are correct but "world" coordinates are always zero.
struct PointCombined
{
PointInfo image; // These points valid and correct
PointInfo world; // Always getting zero
int32_t reserved[10];
};
...
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are you the only person being tracked, or are there multiple people in the camera's vieew? The 'Person Detection' mode in the RealSense SDK For Linux can track all people, but the 'Person Tracking' mode can only track one person. Also, the person should be at least 0.5 m away from the camera, up to a maximum distance of 5 m. Bear in mind too that Center of Mass will stop working beyond 3 m distance or when less than 0.5 m from the camera.
https://software.intel.com/sites/products/realsense/person/developer_guide.html# 2.2_Person_Tracking_ Intel® RealSense™ for Linux - Person Library: Developer's Guide
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi MartyG,
Thanks for quick reply. I am tracking all the persons at the moment. I have run the examples(librealsense)and its working fine. But problem is with my system where I am converting realsense to my system back and forth the images and params. I guess some info is missed which is required for the algorithm. I have checked intrinsic and extrinsic params and they are fine. Can you suggest why the problem is only with world coordinates of center of mass. Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I wonder if the problem is in the conversion. When converting RealSense to MATLAB .mat format, for example, the image is commonly converted from RealSense's native 16-bit resolution to 8-bit resolution. The user then finds that there are differences in color and scale between the original and converted images, and cannot do a reverse conversion from .mat to RealSense. Does your system convert image data to 8-bit?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am using 8 bit unsigned char as RGB format. I will check my changing it to 16 bit format. Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For 8-bit I am using rgb8 format. What is the format for 16 bit rgb for realsense ?. Also I have found that in examples of ROS is also RGB8 format is used. So I guess problem is not with 8 bit.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
According to the table linked to below, the format for 16-bit grayscale is Y16.
https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?pixelformat_pxcimage.html Intel® RealSense™ SDK 2016 R2 Documentation
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I guess problem is not due to RGB8 format because same is used in ros application. Y16 is for Grayscale and person tracking uses color and depth image. Can you suggest which formats are 16 bit colour. Below are the aviable formats.
/// \brief Formats: defines how each stream can be encoded./// \c rs_format specifies how a frame is represented in memory (similar to the V4L pixel format).
enum class format : int32_t
{
any , /**<</span> When passed to enable stream, librealsense will try to provide best suited format */
z16 , /**<</span> 16 bit linear depth values. The depth is meters is equal to depth scale * pixel <span style="col...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I deeply researched this issue but was only able to find support for an RGB8 mode.
One of the leaders of Librealsense had this advice about getting depth from a 16 bit image:
https://github.com/IntelRealSense/librealsense/issues/509 How to get the real distance from 16-bit depth image pixel value? · Issue # 509 · IntelRealSense/librealsense · GitHub
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am able to read rgb, depth images well but the problem is when I use them with person tracking module. Below code might make my problem clear.
Intel::RealSense::PersonTracking::PersonTrackingData::PointCombined centerMass = personTrackingData->QueryCenterMass();after this call "image" coordinates are correct but "world" coordinates are always zero.
struct PointCombined
{
PointInfo image; // These points valid and correct
PointInfo world; // Always getting zero
int32_t reserved[10];
};
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The code that you quoted seems to be correct.
I note that in the Person Tracking module's code, the above struct is preceded earlier in the script by an enum instruction.
enum TrackingState
{
TRACKING_STATE_TRACKING = 0,
TRACKING_STATE_DETECTING
};
https://software.intel.com/sites/products/realsense/person/PersonTrackingData_8h_source.html Intel® RealSense™ for Linux - Person Library: /Users/mcurfman/ws/Linux_SDK/realsense_documentation/realsense_person/sdk/…
I apologize that my knowledge about programming Person Tracking in the Intel RealSense SDK For Linux is limited.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No problem, I have added some debug info about stream param, it might be useful.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks a lot. This level of depth on extrinsics is outside of my field of knowledge, but hopefully someone else on this forum who has worked with these values can add some useful commentary.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Finally I solved the problem. Problem is due to converting the depth image as array of floats and making cv::Mat with CV_32FC1. By using uint16_t and CV_16UC1 and evrerything is working fine.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That's great news! Thank you very much for sharing the knowledge with the rest of the community.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page