Frame lost with D415 (solved)

[cvest]

Hi, I need to acquire IR, depth and color images with a frame rate of 30fps. I have no trouble with IR and depth, but I lost some images when using the 3 streams. 

I don't care if I lost some images, but I need the correct images for my processing. I use pipe.wait_for_frames() to wait for all images but it does not prevent from lost frame. Is there a way to insure all images are correct?

Next is a code test, I flip image to vizualize frame lost.

I only use 1 realsense, on a windows 64, RS 1.19.2, and USB3 port

edit1: I tested with wait_for_frames() and poll_for_frames() (https://github.com/IntelRealSense/librealsense/wiki/API-How-To#wait-for-coherent-set-of-frames) and have same trouble

Edit2: it was misunderstanding of the exemple, see my comment

Thanks

// License: Apache 2.0. See LICENSE file in root directory.   // Copyright(c) 2017 Intel Corporation. All Rights Reserved.       // https://github.com/IntelRealSense/librealsense/tree/master/include   #include <librealsense2/rs.hpp> // Include RealSense Cross Platform API   #include <librealsense2/rs_advanced_mode.hpp> // extra functionalities   #include <opencv2/opencv.hpp> // Include OpenCV API       using namespace std;   using namespace cv;           // acquisition parameters   int w = 1280;   int h = 720;   int framerate = 30;   rs2_stream camera_for_detect = RS2_STREAM_INFRARED; // RS2_STREAM_COLOR or RS2_STREAM_INFRARED   rs2_stream camera_col = RS2_STREAM_COLOR; // for dump           int main(int argc, char * argv[]) try   {   // Declare depth colorizer for pretty visualization of depth data   rs2::colorizer color_map;       // Declare RealSense pipeline, encapsulating the actual device and sensors   rs2::pipeline pipe;       //Create a configuration for configuring the pipeline with a non default profile   rs2::config cfg;       //Add desired streams to configuration   cfg.enable_stream(camera_for_detect, w, h, RS2_FORMAT_BGR8, framerate);   cfg.enable_stream(camera_col, w, h, RS2_FORMAT_BGR8, framerate);   cfg.enable_stream(RS2_STREAM_DEPTH, w, h, RS2_FORMAT_Z16, framerate);       // Start streaming with default recommended configuration   cout << "Start streaming" << endl;   rs2::pipeline_profile profile = pipe.start(cfg);   cout << "Start streaming OK" << endl;   rs2::device dev = profile.get_device();   cout << "GetDevice OK" << endl;           while (waitKey(1) < 0)   {   // Camera warmup - dropping several first frames to let auto-exposure stabilize   rs2::frameset data = pipe.wait_for_frames();   rs2::frame ir_frame = data.first(camera_for_detect);   rs2::frame depth = data.get_depth_frame();   rs2::frame depthCol = data.get_depth_frame().apply_filter(color_map);       // Creating OpenCV matrix from IR image   Mat ir(Size(w, h), CV_8UC3, (void*)ir_frame.get_data(), Mat::AUTO_STEP);       // test frame lost   flip(ir, ir, +1);   imshow("FrameLostTest", ir);   }       cout << "Exit()" << endl;       return EXIT_SUCCESS;   }   catch (const rs2::error & e)   {   std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n " << e.what() << std::endl;   return EXIT_FAILURE;   }   catch (const std::exception& e)   {   std::cerr << e.what() << std::endl;   return EXIT_FAILURE;   }          

[MartyG]

The system overhead and software stack may affect the stream fps and record.  For users who want to use multiple streams simultaneously whilst avoiding dropped frames, Dorodnic the RealSense SDK Manager offers advice in the link below.

https://github.com/IntelRealSense/librealsense/issues/2563#issuecomment-431145365

[cvest · Accepted solution]

Thanks MartyG,

i found that there is no frame drop but a misunderstanding of the exemples.

I though data was not changed (frame drop) when displayed image were switching between the normal and the fliped views, while in fact it is due that cv::Mat point to frame data which is rewriten between the flip and the draw.

When I made a copy of ir image, then flip, then draw I have no problem.

There was no problem at all.

[MartyG]

Great news, thanks for letting us know. :)