topic Re: RGB image glitches D415 in Items with no label

RGB image glitches D415

RVC — Thu, 13 Sep 2018 07:25:44 GMT

Hi all,

as said in a previous thread, we are using multiple D415's for 360° (outward) capturing. Updating firmware solved some of the inconsistencies.

However, now we are having trouble with the colour (pink & green) and image itself (cut and misaligned pixels?) for the RGB-streams:

This also shows up in the coloured point clouds.

Is this shutter-related (and too fast movement, notice there is no blur though)? Software or hardware? Who has experienced similar errors?

Thanks!

Re: RGB image glitches D415

MartyG — Thu, 13 Sep 2018 08:14:01 GMT

A similar coloring issue once occurred on a four-camera SR300 setup. In that case the problem was with the depth stream, which was pink and blue. They were using Linux, and fixed it by re-doing the Video4Linux backend preparation to apply the kernel patches. Are you using Linux, please?

Re: RGB image glitches D415

RVC — Thu, 13 Sep 2018 09:57:10 GMT

Hi MartyG, yes, Linux. We will try and let you know. Thanks!

Re: RGB image glitches D415

RVC — Mon, 17 Sep 2018 09:40:02 GMT

Update: tried the Video4Linux trick but did not help -still having the glitch (whichever the resolution or framerate).

What else could be checked?

Re: RGB image glitches D415

MartyG — Mon, 17 Sep 2018 10:16:33 GMT

Are you using hardware sync cables with your cameras? And if so, have you built ESD protection into those cables? The multiple camera white paper says that if there is not protection from ESD events (e.g static electricity discharge) then the RGB imager can freeze up when streaming both depth and RGB (depth streaming only is not so badly affected by the counter resets that occur in an ESD event). The D415 model is affected by this more than the D435.

Pages 3 and 4 of the multiple camera paper discuss how to build ESD protection into the cable.

https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf

Re: RGB image glitches D415

RVC — Mon, 17 Sep 2018 10:19:18 GMT

No hardware sync, just USB cables on hubs...

Re: RGB image glitches D415

MartyG — Mon, 17 Sep 2018 10:37:48 GMT

Are all of the D415 cameras experiencing the same issue?

What length of USB cable are you using, please?

Re: RGB image glitches D415

RVC — Mon, 17 Sep 2018 10:42:28 GMT

No, only some of them. We will try to figure out which ones 1 by 1, without hub inbetween, today.

The USB cable length is 15cm -> USBhub -> 5cm, so about 20cm in total.

Re: RGB image glitches D415

MartyG — Mon, 17 Sep 2018 10:46:57 GMT

I ask whether only some of the cameras are affected because in a similar case of coloration like this, the problem was eventually decided to be an issue with the hardware. So the user returned the affected camera for a replacement and the replacement camera worked fine.

Re: RGB image glitches D415

RVC — Tue, 18 Sep 2018 14:11:28 GMT

It seems we get less problems without the hubs -annoying since we are already using pretty expensive ones.

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 14:41:17 GMT

In Intel's tests with a hub in their multiple camera paper, they used a cheap AmazonBasics hub. The key point seems to be that it is a mains-powered hub, rather than "passive" hubs that are plugged into the PC's USB port and powered by the computer. Are your own hubs mains-powered ones?

The AmazonBasics model they used is linked to below.

https://www.amazon.com/dp/B00DQFGH80/ref=asc_df_B00DQFGH805315309/ Amazon.com: AmazonBasics 4 Port USB 3.0 Hub with 5V/2.5A power adapter: Computers & Accessories

Re: RGB image glitches D415

RVC — Tue, 18 Sep 2018 14:52:12 GMT

No, they are externally powered. See https://www.club-3d.com/en/detail/2383/usb_3.0_hub_4_port_with_power_adapter/ https://www.club-3d.com/en/detail/2383/usb_3.0_hub_4_port_with_power_adapter/

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 14:56:53 GMT

They are mains powered then, because I mean that they plug into a wall socket (mains socket) with a power adapter like the one that comes with your hub.

Re: RGB image glitches D415

RVC — Tue, 18 Sep 2018 14:58:08 GMT

I meant yes, they are ;-)

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 15:17:58 GMT

USB hubs can be 'daisy chained' - plugging one hub into another hub. Apparently you can chain up to 5 hubs in this way. I wonder, if the cameras work in some hub ports but not others, then you might be able to get all the cameras working by chaining a couple of hubs together and attaching your cameras to them. If they are 4-port hubs and you have 8 ports available in total with 2 hubs joined together, one would think you ought to be able to get 4 fully working ports out of that.

Both hubs should be plugged into the mains.

Re: RGB image glitches D415

RVC — Tue, 18 Sep 2018 15:32:27 GMT

I'm afraid for 11 cameras that would take too many hubs.. But good point on checking if it's linked to certain ports. Will try that. Thanks!

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 15:37:47 GMT

The USB standard supports 127 USB devices on a single computer. Each hub claims a handful of these addresses, meaning that you cannot actually plug 127 devices in. So if the limit of 5 hubs just applies to the number of hubs, not the number of ports, then you may be able to fit 11 cameras into 5 ports if you use hubs with more ports on, such as 6 or 8 ports.

Anyways, good luck with the port check!

Edit: I did some more research. The limit is apparently in the USB specification, which lets a "stack" of hubs be '5 hubs deep' from the root port.

Re: RGB image glitches D415

RVC — Tue, 18 Sep 2018 15:44:20 GMT

Correct, bet then again, won't we have issues with bandwidth? This is already the case after a while of streaming.

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 16:02:54 GMT

In an Intel online seminar about multiple cameras last week, the RealSense CTO said that as more cameras are added, a higher spec of PC hardware will be needed to cope with the strain (for example, a machine with an Intel i7 processor). In the same seminar, they said that they have developed a new 'x3' data compression algorithm that will allow more cameras to be attached. They did not give a date for when that will be switched on, but they are continuously working on squeezing greater performance out of the cameras.

They have also developed a 'Modified Huffman' algorithm that will give a greater number of channels in multiple camera configurations.

Below is a chart from the camera paper about bandwidth use on a non-synced hub of up to 6 cameras. Left-click on the image to see it in full size.1

You can alternatively use multiple PCs and collectively join the cameras together using hardware sync cabling. This was the approach Intel took with a multiple-camera demo at the Sundance Festival in January. They had 4 cameras, each one connected to an i7 PC, and transferred the data automatically to a 5th PC to sync it all together. The process is described in the article below, in which they talk about using 8 or more cameras.

https://realsense.intel.com/intel-realsense-volumetric-capture/ Volumetric Capture @ Sundance using Intel RealSense Depth Cameras

Re: RGB image glitches D415

MartyG — Tue, 18 Sep 2018 16:26:04 GMT

I would also add that if there is not a need for the cameras to all be operating at exactly the same time, you can have more cameras on the same machine and avoid excessive bandwidth consumption by turning them on and off in batches of 4. You take a data snapshot with the first batch, then turn them off and activate the next 4. Take a snapshot and activate the next batch, until the cycle gets back to the first set of cameras and begins again. In the old RealSense camera models, this process of activating devices by ID number was known as Enumeration.