d435 opencv glitch

PManc1 — Fri, 31 May 2019 15:04:46 GMT

Hi, I'm using a d435 and I'm trying to flip the depth image with opencv, but I often get a glitch (the image looks flipped for a dozen of frames and not flipped for a few frames).

If I use recorded bitmap depth the glitch doesn't occur.

This is the code I'm using:

rs2::colorizer c;
rs2::context  ctx;            // Create librealsense context for managing devices
rs2::colorizer colorizer;      // Utility class to convert depth data RGB colorspace
std::vector<rs2::pipeline>  pipelines;
for (auto&& dev : ctx.query_devices())
{
	const char* cameraserial = dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER);
	rs2::pipeline pipe = rs2::pipeline(ctx);
	rs2::config cfg;
	cfg.enable_device(cameraserial);
	cfg.enable_stream(RS2_STREAM_DEPTH, COLS, ROWS, RS2_FORMAT_Z16, 30);
	pipe.start(cfg);
 
	pipelines.emplace_back(pipe);
 
	auto profile = pipe.get_active_profile();
	std::ifstream t("test.json");
	std::string str((std::istreambuf_iterator<char>(t)),
	std::istreambuf_iterator<char>());
	rs400::advanced_mode dev = profile.get_device();
	dev.load_json(str);
}
std::map<int, rs2::frame> depthframes;
while (true)
{
		cv::Mat depth;
 
		std::vector<rs2::frame> new_frames;
		for (auto &&pipe : pipelines)
		{
			rs2::frameset fs;
			if (pipe.poll_for_frames(&fs))
			{
				for (const rs2::frame& f : fs)
				new_frames.emplace_back(f);
			}
		}
		for (const auto& frame : new_frames)
		{
			c.set_option(rs2_option::RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 0);
			rs2::frame v = c.colorize(frame);
			depthframes[frame.get_profile().unique_id()] = v;
 
		}
		cv::Mat depth = cv::Mat(cv::Size(COLS, ROWS), CV_8UC3, (void*)((depthframes.begin())->second));
		
		cv::flip(depth, depth, -1);
		
		cv::namedWindow("depth");
		cv::imshow("depth", depth);
		cv::waitKey(1);
 
	}

Can someone help me?

Thank you very much.

Re: d435 opencv glitch

MartyG — Fri, 31 May 2019 16:14:05 GMT

There is a phenomenon where the camera can have the same frame number and timestamp for a few frames. This can occur when a frame has been dropped, and the camera tries to compensate by returning the last known frame. Dorodnic the RealSense SDK Manager has advised that in such cases, "If you are building the SDK from source, please make sure to build with -DCMAKE_BUILD_TYPE=Release. You can also try reducing the total resolution".

If your unflipped frames have the same frame number and timestamp, that may indicate that there is a hiccup and recovery occurring due to dropped frames.

topic Re: d435 opencv glitch in Items with no label

d435 opencv glitch

Re: d435 opencv glitch