Community
cancel
Showing results for 
Search instead for 
Did you mean: 
RKura
New Contributor I
2,543 Views

Issue parsing output from rs_get_frame_data()

Hello,

I am having a slight issue parsing the data from realsense library function rs_get_frame_data()

What I need to do is copy the output from rs_get_frame_data() into a local memory buffer and pass that buffer on to GStreamer. Here is my code:

struct rgb *rgb = ( struct rgb *) rs_get_frame_data( camera, RS_STREAM_COLOR, &e ); // get single frame from realsense camera

char *buffer = malloc(640*480*3); // allocates buffer large enough to hold a single frame coming from realsense

memcpy (buffer,rgb, sizeof(rgb));

Then pass buffer on to GStreamer:

GstBuffer *gstbuffer = gst_buffer_new_wrapped_full( ( GstMemoryFlags )0, buffer, SIZE, 0, SIZE, NULL, NULL ); // GStreamer does not seem to like this. SIZE = 640*480*3 for a single frame 

struct rgb is a 3 character struct defined as

@struct rgb {

unsigned char r;

unsigned char g;

unsigned char b;

};

What am I doing wrong here ?

Any help is much appreciated.

best

Rohan

0 Kudos
13 Replies
RKura
New Contributor I
85 Views

MartyG

If you can help answer the above question that would be great.

Thanks

Rohan

MartyG
Honored Contributor III
85 Views

I sadly have no knowledge of programming GStreamer. I wonder if this link might provide useful insight though

https://developer.gnome.org/gstreamer/stable/gstreamer-GstBuffer.html GStreamer 1.0 Core Reference Manual: GstBuffer

Looking at the formatting of your code though:

GstBuffer *gstbuffer = gst_buffer_new_wrapped_full( ( GstMemoryFlags )0

You may have a misplaced bracket, as a couple of others using a similar line wrote it as

GstBuffer *gstbuffer = gst_buffer_new_wrapped_full( ( GstMemoryFlags(0)

I'm guessing you got your code from this article?

https://software.intel.com/en-us/articles/intel-aero-realsense-streaming-rgb-depth-data Code Sample: Broadcast RGB and Depth Data from an Intel® Aero Compute Board | Intel® Software

RKura
New Contributor I
85 Views

Marty,

That was a typo on my part. The code I have is what you have there. So thats good. Do you see anything going wrong with the type conversion ? i.e when I memcpy the struct rgb data into char * buffer ?

best

Rohan

MartyG
Honored Contributor III
85 Views

Another thing that got my attention was that you are trying to use a frames per second (FPS) speed of 3. I can see the possibility that this may cause an issue with the R200 camera, as the R200 has a minimum FPS of 30 and tries to force the speed to be 30 if you set it lower than that in your code. Ideally, you should only be using '30' or '60' (for RealSense camera resolutions that support 60 FPS - most of them are 30 FPS).

RKura
New Contributor I
85 Views

Hi Marty,

Where do you see a fps = 3 ? I have fps = 30 set in my rs_enable_stream command. The malloc for 640*480*3 is for HEIGHT = 640, WIDTH = 480 and 3 bytes per pixel.

Thanks

Rohan

MartyG
Honored Contributor III
85 Views

I'm sorry if I misunderstood. But in RealSense SDK code, the 3rd number in a resolution bracket always refers to FPS (width, height, fps). So in RealSense code, if you want a 640x480 resolution at 30 FPS then you write it as (640, 480, 30).

RKura
New Contributor I
85 Views

I am bit confused now

I assume you are referring to the code here:

struct rgb *rgb = ( struct rgb *) rs_get_frame_data( camera, RS_STREAM_COLOR, &e ); // get single frame from realsense camera

char *buffer = malloc(640*480*3); // allocates buffer large enough to hold a single frame coming from realsense

memcpy (buffer,rgb, sizeof(rgb));

The 2nd line doing the malloc gets memory for a single frame which I would think is 640*480*3 bytes for reasons mentioned above. Then when we get the next frame (this code is executed in a loop while we do a rs_wait_for_frames), we malloc a new buffer for the next frame. Are you saying the malloc is incorrect ?

Thanks

Rohan

MartyG
Honored Contributor III
85 Views

I am at a disadvantage because I do not have knowledge of GStreamer programming. However, when trying to look at a single frame of a live analysis, the usual procedure is to pause the analysis on a particular frame and then get the data of that frame, rather than try to grab a frame whilst the stream is moving.

RKura
New Contributor I
85 Views

Thanks Marty. I will take a look at the link you provided.

My question for the following line of code

struct rgb *rgb = ( struct rgb* ) rs_get_frame_data( camera, RS_STREAM_COLOR, NULL );

Does this point the structure pointer rgb to a single frame data (640*480 pixel stream) or does it point to a single pixel (3 byte rgb) in a frame ? I assume this points of a stream of pixels information in one frame that is returned by rs_get_frame_data. Just want to be sure.

best

Rohan

MartyG
Honored Contributor III
85 Views

My guess would be that this line of code refers to a frame rather than a pixel, since a query about a frame is what is contained in the instruction. From my research of GStreamer instructions, I think that if it was referring only to a pixel then the code would reference 'pixels' instead of 'frame'. This is entirely guesswork on my part though.

RKura
New Contributor I
85 Views

I agree. So thats not what it is.

RKura
New Contributor I
85 Views

MartyG

I wanted to let you know that I think I figured out what was going on here w.r.t the output from rs_get_frame_data(). Thanks for your help. Turned out something in my application was not receiving the buffers properly. Still facing the mavlink error on QGC thou which I have a separate thread for.

best

Rohan

MartyG
Honored Contributor III
85 Views

I'm pleased you made progress on one of your issues. A little encouragement can go a long way.

Reply