Frame lost with D415 (solved)

cvest — Mon, 15 Apr 2019 14:51:07 GMT

Hi, I need to acquire IR, depth and color images with a frame rate of 30fps. I have no trouble with IR and depth, but I lost some images when using the 3 streams.

I don't care if I lost some images, but I need the correct images for my processing. I use pipe.wait_for_frames() to wait for all images but it does not prevent from lost frame. Is there a way to insure all images are correct?

Next is a code test, I flip image to vizualize frame lost.

I only use 1 realsense, on a windows 64, RS 1.19.2, and USB3 port

edit1: I tested with wait_for_frames() and poll_for_frames() (https://github.com/IntelRealSense/librealsense/wiki/API-How-To#wait-for-coherent-set-of-frames) and have same trouble

Edit2: it was misunderstanding of the exemple, see my comment

Thanks

// License: Apache 2.0. See LICENSE file in root directory.
 
// Copyright(c) 2017 Intel Corporation. All Rights Reserved.
 
 
 
// https://github.com/IntelRealSense/librealsense/tree/master/include
 
#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
 
#include <librealsense2/rs_advanced_mode.hpp> // extra functionalities
 
#include <opencv2/opencv.hpp>  // Include OpenCV API
 
 
 
using namespace std;
 
using namespace cv;
 
 
 
 
 
// acquisition parameters
 
int w = 1280;
 
int h = 720;
 
int framerate = 30;
 
rs2_stream camera_for_detect = RS2_STREAM_INFRARED; // RS2_STREAM_COLOR or RS2_STREAM_INFRARED
 
rs2_stream camera_col = RS2_STREAM_COLOR; // for dump
 
 
 
 
 
int main(int argc, char * argv[]) try
 
{
 
  // Declare depth colorizer for pretty visualization of depth data
 
  rs2::colorizer color_map;
 
 
 
  // Declare RealSense pipeline, encapsulating the actual device and sensors
 
  rs2::pipeline pipe;
 
 
 
  //Create a configuration for configuring the pipeline with a non default profile
 
  rs2::config cfg;
 
 
 
  //Add desired streams to configuration
 
  cfg.enable_stream(camera_for_detect, w, h, RS2_FORMAT_BGR8, framerate);
 
  cfg.enable_stream(camera_col, w, h, RS2_FORMAT_BGR8, framerate);
 
  cfg.enable_stream(RS2_STREAM_DEPTH, w, h, RS2_FORMAT_Z16, framerate);
 
 
 
  // Start streaming with default recommended configuration
 
  cout << "Start streaming" << endl;
 
  rs2::pipeline_profile profile = pipe.start(cfg);
 
  cout << "Start streaming OK" << endl;
 
  rs2::device dev = profile.get_device();
 
  cout << "GetDevice OK" << endl;
 
 
 
 
 
  while (waitKey(1) < 0)
 
  {
 
    // Camera warmup - dropping several first frames to let auto-exposure stabilize
 
    rs2::frameset data = pipe.wait_for_frames();
 
    rs2::frame ir_frame = data.first(camera_for_detect);
 
    rs2::frame depth = data.get_depth_frame();
 
    rs2::frame depthCol = data.get_depth_frame().apply_filter(color_map);
 
 
 
    // Creating OpenCV matrix from IR image
 
    Mat ir(Size(w, h), CV_8UC3, (void*)ir_frame.get_data(), Mat::AUTO_STEP);
 
 
 
    // test frame lost
 
    flip(ir, ir, +1);
 
    imshow("FrameLostTest", ir);
 
  }
 
 
 
  cout << "Exit()" << endl;
 
 
 
  return EXIT_SUCCESS;
 
}
 
catch (const rs2::error & e)
 
{
 
  std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n  " << e.what() << std::endl;
 
  return EXIT_FAILURE;
 
}
 
catch (const std::exception& e)
 
{
 
  std::cerr << e.what() << std::endl;
 
  return EXIT_FAILURE;
 
}

Re: Frame lost with D415 (solved)

MartyG — Mon, 15 Apr 2019 15:42:06 GMT

The system overhead and software stack may affect the stream fps and record. For users who want to use multiple streams simultaneously whilst avoiding dropped frames, Dorodnic the RealSense SDK Manager offers advice in the link below.

https://github.com/IntelRealSense/librealsense/issues/2563#issuecomment-431145365

Re: Frame lost with D415 (solved)

cvest — Mon, 15 Apr 2019 17:11:38 GMT

Thanks MartyG,

i found that there is no frame drop but a misunderstanding of the exemples.

I though data was not changed (frame drop) when displayed image were switching between the normal and the fliped views, while in fact it is due that cv::Mat point to frame data which is rewriten between the flip and the draw.

When I made a copy of ir image, then flip, then draw I have no problem.

There was no problem at all.

Re: Frame lost with D415 (solved)

MartyG — Mon, 15 Apr 2019 17:17:43 GMT

Great news, thanks for letting us know. :)

topic Frame lost with D415 (solved) in Items with no label

Frame lost with D415 (solved)

Re: Frame lost with D415 (solved)

Re: Frame lost with D415 (solved)

Re: Frame lost with D415 (solved)