question about the rs-measure example

[zting1]

I have some question about the example of rs-measure. mainly in the code below：

std::thread video_processing_thread([&]() { // In order to generate new composite frames, we have to wrap the processing // code in a lambda rs2::processing_block frame_processor( [&](rs2::frameset data, // Input frameset (from the pipeline) rs2::frame_source& source) // Frame pool that can allocate new frames { // First make the frames spatially aligned data = data.apply_filter(align_to);   // Decimation will reduce the resultion of the depth image, // closing small holes and speeding-up the algorithm data = data.apply_filter(dec);   // To make sure far-away objects are filtered proportionally // we try to switch to disparity domain data = data.apply_filter(depth2disparity);   // Apply spatial filtering data = data.apply_filter(spat);   // Apply temporal filtering data = data.apply_filter(temp);   // If we are in disparity domain, switch back to depth data = data.apply_filter(disparity2depth);   // Send the post-processed depth for path-finding pathfinding_queue.enqueue(data.get_depth_frame());   //Apply color map for visualization of depth data = data.apply_filter(color_map);   // Send the composite frame for rendering source.frame_ready(data); }); // Indicate that we want the results of frame_processor // to be pushed into postprocessed_frames queue frame_processor >> postprocessed_frames;   while (alive) { // Fetch frames from the pipeline and send them for processing rs2::frameset fs; if (pipe.poll_for_frames(&fs)) frame_processor.invoke(fs); } });

I can understand the central idea​ is that:

1.when receive a frame , frame processor invoke the processing function,that is the lambda function:

[&](rs2::frameset data, // Input frameset (from the pipeline) rs2::frame_source& source) // Frame pool that can allocate new frames { …… }

confusion 1. if fs is the first parameter(rs2::frameset data

) of the lambda function, what is the second parameter(rs2::frame_source& source

), I can't find any variable Incoming the lambda function as the second parameter.

confusio2. frame_processor >> postprocessed_frames; the operator >> is a callback function to notice the frame_ready. Am I right? Can I see it as frame_processor.operator>>(postprocessed_frames)? if so, I can't find any operation that can push back the frame processed to postprocessed_frames. Or what does the operator >> push back to the postprocessed_frames?

confusion 3: if the postprocessed_frames is the parameter of on_frame, and the operator >> also return on_frame, which variable receive the on_frame?

template<class S> S& operator>>(S& on_frame) { start(on_frame); return on_frame; }

Can you trouble explain? Thanks a lot!

[Eliza_D_Intel]

Hello zting1, Thank you for your interest in the Intel RealSense Technology. Here are the answers to your questions: Confusion 1: The "source" is not an input. It is an output. This code in rs_measure.cpp: rs2::processing_block frame_processor( [&](rs2::frameset data, // Input frameset (from the pipeline) rs2::frame_source& source) // Frame pool that can allocate new frames Calls this template below, where "processing function" = (rs2::frameset data, rs2::frame_source& source). You can see this if you debug the rs-measure.cpp code with a breakpoint at the above function call and step into the call. template<class S> processing_block(S processing_function) { rs2_error* e = nullptr; _block = std::shared_ptr<rs2_processing_block>( rs2_create_processing_block(new frame_processor_callback<S>(processing_function), &e), rs2_delete_processing_block); options::operator=(_block); error::handle(e); } The "frame_processor_callback" from the above function, calls this template below. The "on_frame_function" returns the output - src, which is a reference to the input - frm. class frame_processor_callback : public rs2_frame_processor_callback { T on_frame_function; public: explicit frame_processor_callback(T on_frame) : on_frame_function(on_frame) {} void on_frame(rs2_frame* f, rs2_source * source) override { frame_source src(source); frame frm(f); on_frame_function(std::move(frm), src); } Confusion 2 and 3: This command frame_processor >> postprocessed_frames; "frame_processor" is the input, and "postprocessed_frames" is the output. The "on_frame" returns to "postprocessed_frames." The description in bold tells you what it does * Does the same thing as "start" function * * \param[in] on_frame address of callback function for noticing the frame to be processed is ready. * \return address of callback function. */ template<class S> S& operator>>(S& on_frame) { start(on_frame); return on_frame; } Thank you and best regards, Eliza

[zting1]

Hi, ElizaD_Intel, Thanks for your explanation.

I'm confused about the "on_frame" returns to "postprocessed_frames."

​"on_frame" is a function, and "postprocessed_frames" is a queue of frame. How can the "on_frame" return to "postprocessed_frames"?

I'm looking forward to your further reply.

[Eliza_D_Intel]

Hello zting1, According to this comment in rs-measure.cpp, the results of the frame_processor get pushed into postprocessed_frames. Somehow all the function templates, callbacks, and references make it all work. // Indicate that we want the results of frame_processor // to be pushed into postprocessed_frames queue frame_processor >> postprocessed_frames; Thank you and best regards, Eliza

[zting1]

Hello ElizaD,

I understood the comment. But I just encounter some problem when use the code block, so I want to know this code block better.

Thanks for your reply!

[Eliza_D_Intel]

Hello zting1, Could you please let us know what are the problems you have encountered? Thank you and best regards, Eliza

[zting1]

Hello ElizaD

I wrote a simple program to get distance from the depth frame based on the example of rs-measure, but the code crash when it run to 

auto udist = depth.get_distance(u, v);

and there is no useful message output. I don't know how to fix it. Can you give me some advice. Thanks a lot!

The complete code is below, it is just a process thread and a main thread to visit the depth.

// License: Apache 2.0. See LICENSE file in root directory. // Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.   #include <librealsense2/rs.hpp> // Include RealSense Cross Platform API #include <librealsense2/rsutil.h> #include "example.hpp" // Include short list of convenience functions for rendering // This example will require several standard data-structures and algorithms: #define _USE_MATH_DEFINES #include <math.h> #include <queue> #include <algorithm> // std::min, std::max #include <thread> #include <atomic> #include <opencv2/opencv.hpp> // Include OpenCV API #include <mutex> //PCL #include <pcl/common/common_headers.h> #include <pcl/visualization/pcl_visualizer.h> #include <pcl/io/pcd_io.h> #include <pcl/filters/passthrough.h> typedef pcl::PointXYZRGB RGB_Cloud; typedef pcl::PointCloud<RGB_Cloud> point_cloud; typedef point_cloud::Ptr cloud_pointer; typedef point_cloud::Ptr prevCloud; using pixel = std::pair<int, int>; //Global variable int i = 1; // Index for incremental file name int width = 1280; int height = 720; using namespace std;   std::shared_ptr<pcl::visualization::PCLVisualizer> createRGBVisualizer(pcl::PointCloud<pcl::PointXYZRGB>::ConstPtr cloud); cloud_pointer PCL_Conversion_rect(const rs2::depth_frame& frame, const rs2::video_frame &color, const vector<cv::Point_<u_int32_t>> &vertex,const rs2_intrinsics intr_);   int main(int argc, char * argv[]) try { rs2::align align_to(RS2_STREAM_COLOR); // Decimation filter reduces the amount of data (while preserving best samples) rs2::decimation_filter dec; // If the demo is too slow, make sure you run in Release (-DCMAKE_BUILD_TYPE=Release) // but you can also increase the following parameter to decimate depth more (reducing quality) dec.set_option(RS2_OPTION_FILTER_MAGNITUDE, 2); // Define transformations from and to Disparity domain rs2::disparity_transform depth2disparity; rs2::disparity_transform disparity2depth(false); // Define spatial filter (edge-preserving) rs2::spatial_filter spat; // Enable hole-filling // Hole filling is an agressive heuristic and it gets the depth wrong many times spat.set_option(RS2_OPTION_HOLES_FILL, 5); // 5 = fill all the zero pixels // Define temporal filter rs2::temporal_filter temp;   // After initial post-processing, frames will flow into this queue: rs2::frame_queue postprocessed_frames;   // Declare RealSense pipeline, encapsulating the actual device and sensors rs2::pipeline pipe;   rs2::config cfg; cfg.enable_stream(RS2_STREAM_DEPTH,width,height,RS2_FORMAT_Z16,30); // For the color stream, set format to RGBA // To allow blending of the color frame on top of the depth frame cfg.enable_stream(RS2_STREAM_COLOR, width,height,RS2_FORMAT_BGR8,30); // Start streaming with default recommended configuration auto profile = pipe.start(cfg); auto stream = profile.get_stream(RS2_STREAM_COLOR).as<rs2::video_stream_profile>(); auto intr =stream.get_intrinsics();   cv::Point_<u_int32_t> left_up(540,240); cv::Point_<u_int32_t> right_up(740,240); cv::Point_<u_int32_t> right_down(740,420); cv::Point_<u_int32_t> left_down(540,420);   // Alive boolean will signal the worker threads to finish-up std::atomic_bool alive{ true }; // Video-processing thread will fetch frames from the camera, // apply post-processing and send the result to the main thread for rendering // It recieves synchronized (but not spatially aligned) pairs // and outputs synchronized and aligned pairs std::thread video_processing_thread([&]() { // In order to generate new composite frames, we have to wrap the processing // code in a lambda rs2::processing_block frame_processor( [&](rs2::frameset data, // Input frameset (from the pipeline) rs2::frame_source& source) // Frame pool that can allocate new frames { // First make the frames spatially aligned data = data.apply_filter(align_to);   // Decimation will reduce the resultion of the depth image, // closing small holes and speeding-up the algorithm data = data.apply_filter(dec);   // To make sure far-away objects are filtered proportionally // we try to switch to disparity domain data = data.apply_filter(depth2disparity);   // Apply spatial filtering data = data.apply_filter(spat);   // Apply temporal filtering data = data.apply_filter(temp);   // If we are in disparity domain, switch back to depth data = data.apply_filter(disparity2depth);   // Send the composite frame for rendering source.frame_ready(data); }); // Indicate that we want the results of frame_processor // to be pushed into postprocessed_frames queue frame_processor >> postprocessed_frames;   while (alive) { // Fetch frames from the pipeline and send them for processing rs2::frameset fs; if (pipe.poll_for_frames(&fs)) frame_processor.invoke(fs); } });   //main thread while (alive) { static rs2::frameset current_frameset; postprocessed_frames.poll_for_frame(&current_frameset); if(current_frameset) { auto depth = current_frameset.get_depth_frame(); auto color = current_frameset.get_color_frame(); int cloud_width = right_up.x - left_up.x; int cloud_height = left_down.y - left_up.y; for (int i = 0; i < cloud_height; i++) { for (int j = 0; j < cloud_width; ++j) { int index_pcl = i * cloud_width +j; int v = left_up.y + i ; int u = left_up.x + j; cout<<"u="<<u<<",v="<<v<<endl; auto udist = depth.get_distance(u, v); cout<<"u="<<u<<",v="<<v<<endl; } alive = false; video_processing_thread.join(); return EXIT_SUCCESS; } catch (const rs2::error & e) { std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n " << e.what() << std::endl; return EXIT_FAILURE; } catch (const std::exception & e) { std::cerr << e.what() << std::endl; return EXIT_FAILURE; }

[Eliza_D_Intel]

Hello zting1, Another RealSense user posted a similar issue on GitHub, and by debugging the code he found that the issue was not related to the function. Here is the link: https://github.com/IntelRealSense/librealsense/issues/3365 Thank you, Eliza

[zting1]

Hello ElizaD,

Oh no, The user is me. There was another problem that time. But this time I simplify the code as above, I really don't know what cause the crash. Can you help me?

[Eliza_D_Intel]

Hello zting1, Could you please let us know what is the error you are receiving when running this code? what is the message received by your compiler? Thank you, Eliza

[zting1]

Hello Eliza,

The message is :Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

I just can't get anything from this message.

[zting1]

I point out which line of code cause the exiting, The code in this line is below:

// Decimation will reduce the resultion of the depth image, // closing small holes and speeding-up the algorithm data = data.apply_filter(dec);

I'm trying to figure out why could this line cause the exiting. If you have some idea, please let me know. Thanks a lot!

[zting1 · Accepted solution]

Hello Eliza

The problem has been solved in github, Thanks for your patient answer.