RGB image glitches D415

[RVC]

Hi all,

as said in a previous thread, we are using multiple D415's for 360° (outward) capturing. Updating firmware solved some of the inconsistencies.

However, now we are having trouble with the colour (pink & green) and image itself (cut and misaligned pixels?) for the RGB-streams:

This also shows up in the coloured point clouds.

Is this shutter-related (and too fast movement, notice there is no blur though)? Software or hardware? Who has experienced similar errors?

Thanks!

[MartyG]

A similar coloring issue once occurred on a four-camera SR300 setup. In that case the problem was with the depth stream, which was pink and blue. They were using Linux, and fixed it by re-doing the Video4Linux backend preparation to apply the kernel patches. Are you using Linux, please?

[RVC]

Hi MartyG, yes, Linux. We will try and let you know. Thanks!

[RVC]

Update: tried the Video4Linux trick but did not help -still having the glitch (whichever the resolution or framerate).

What else could be checked?

[MartyG]

Are you using hardware sync cables with your cameras? And if so, have you built ESD protection into those cables? The multiple camera white paper says that if there is not protection from ESD events (e.g static electricity discharge) then the RGB imager can freeze up when streaming both depth and RGB (depth streaming only is not so badly affected by the counter resets that occur in an ESD event). The D415 model is affected by this more than the D435.

Pages 3 and 4 of the multiple camera paper discuss how to build ESD protection into the cable.

https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf

[RVC]

No hardware sync, just USB cables on hubs...

[MartyG]

Are all of the D415 cameras experiencing the same issue?

What length of USB cable are you using, please?

[RVC]

No, only some of them. We will try to figure out which ones 1 by 1, without hub inbetween, today.

The USB cable length is 15cm -> USBhub -> 5cm, so about 20cm in total.

[MartyG]

I ask whether only some of the cameras are affected because in a similar case of coloration like this, the problem was eventually decided to be an issue with the hardware. So the user returned the affected camera for a replacement and the replacement camera worked fine.

[RVC]

It seems we get less problems without the hubs -annoying since we are already using pretty expensive ones.

[MartyG]

In Intel's tests with a hub in their multiple camera paper, they used a cheap AmazonBasics hub. The key point seems to be that it is a mains-powered hub, rather than "passive" hubs that are plugged into the PC's USB port and powered by the computer. Are your own hubs mains-powered ones?

The AmazonBasics model they used is linked to below.

https://www.amazon.com/dp/B00DQFGH80/ref=asc_df_B00DQFGH805315309/ Amazon.com: AmazonBasics 4 Port USB 3.0 Hub with 5V/2.5A power adapter: Computers & Accessories

[RVC]

No, they are externally powered. See https://www.club-3d.com/en/detail/2383/usb_3.0_hub_4_port_with_power_adapter/ https://www.club-3d.com/en/detail/2383/usb_3.0_hub_4_port_with_power_adapter/

[MartyG]

They are mains powered then, because I mean that they plug into a wall socket (mains socket) with a power adapter like the one that comes with your hub.

[RVC]

I meant yes, they are ;-)

[MartyG]

USB hubs can be 'daisy chained' - plugging one hub into another hub. Apparently you can chain up to 5 hubs in this way. I wonder, if the cameras work in some hub ports but not others, then you might be able to get all the cameras working by chaining a couple of hubs together and attaching your cameras to them. If they are 4-port hubs and you have 8 ports available in total with 2 hubs joined together, one would think you ought to be able to get 4 fully working ports out of that.

Both hubs should be plugged into the mains.

[RVC]

I'm afraid for 11 cameras that would take too many hubs.. But good point on checking if it's linked to certain ports. Will try that. Thanks!

[MartyG]

The USB standard supports 127 USB devices on a single computer. Each hub claims a handful of these addresses, meaning that you cannot actually plug 127 devices in. So if the limit of 5 hubs just applies to the number of hubs, not the number of ports, then you may be able to fit 11 cameras into 5 ports if you use hubs with more ports on, such as 6 or 8 ports.

Anyways, good luck with the port check!

Edit: I did some more research. The limit is apparently in the USB specification, which lets a "stack" of hubs be '5 hubs deep' from the root port.

[RVC]

Correct, bet then again, won't we have issues with bandwidth? This is already the case after a while of streaming.

[MartyG]

In an Intel online seminar about multiple cameras last week, the RealSense CTO said that as more cameras are added, a higher spec of PC hardware will be needed to cope with the strain (for example, a machine with an Intel i7 processor). In the same seminar, they said that they have developed a new 'x3' data compression algorithm that will allow more cameras to be attached. They did not give a date for when that will be switched on, but they are continuously working on squeezing greater performance out of the cameras.

They have also developed a 'Modified Huffman' algorithm that will give a greater number of channels in multiple camera configurations.

Below is a chart from the camera paper about bandwidth use on a non-synced hub of up to 6 cameras. Left-click on the image to see it in full size.1

You can alternatively use multiple PCs and collectively join the cameras together using hardware sync cabling. This was the approach Intel took with a multiple-camera demo at the Sundance Festival in January. They had 4 cameras, each one connected to an i7 PC, and transferred the data automatically to a 5th PC to sync it all together. The process is described in the article below, in which they talk about using 8 or more cameras.

https://realsense.intel.com/intel-realsense-volumetric-capture/ Volumetric Capture @ Sundance using Intel RealSense Depth Cameras

[MartyG]

I would also add that if there is not a need for the cameras to all be operating at exactly the same time, you can have more cameras on the same machine and avoid excessive bandwidth consumption by turning them on and off in batches of 4. You take a data snapshot with the first batch, then turn them off and activate the next 4. Take a snapshot and activate the next batch, until the cycle gets back to the first set of cameras and begins again. In the old RealSense camera models, this process of activating devices by ID number was known as Enumeration.

[RVC]

Hi MartyG,

the cameras are already being used intermittently (at the lowest frame-rate, being 6 fps) using the ID. I am afraid it's a hardware problem but lack of space limits the number of computers and hubs.

Looking forward to potential solutions offered by Intel.