What technology does the R200 sensor module use?

[Joe_R_]

Hi,

Please can someone confirm the technology used by the R200 sensor module to collect depth data. Does it use structured light, stereoscopic cameras, time of flight or a combination of these?

Thanks

[samontab]

A combination of structured light, similar to the Kinect 1, and passive infrared stereo.

It does not use time of flight as the Kinect 2 for example.

[Joe_R_]

OK thanks, does anyone know how susceptable the sensor is to sunlight?

[Eddie_Offermann]

OK thanks, does anyone know how susceptable the sensor is to sunlight?

Level of susceptibility and to what degree it will impede you will vary enormously, but the sensors are somewhat unfriendly to bright sunlight.

When we were doing R&D for this project: https://www.youtube.com/watch?v=ZoiazKxU3ek we did a lot of testing under varying light conditions, with different fabrics, against different backgrounds, etc. Since you're dealing mainly with infrared, the impact of visible light is negligible, but IR sources (which can be a little harder to visualize) can have a profound impact on sensor behavior.

We were concerned that the rear projection we'd envisioned might have posed a problem - it turns out that the IR component was negligible. Some fabrics, however, were nearly invisible to the cameras - and it was related to material, not color. One solid black item under identical lighting conditions would be invisible, another would fluoresce white under IR. Naturally, tracking behavior of these different materials would be strikingly different. Only experimentation would answer our concerns.

What I'm saying is that your mileage may vary in daylight conditions - and it won't be entirely because of the ambient light levels. You'll need to do tests. If your application involves pre-construction surveying, for instance, where the area you're probing is an empty field or undeveloped woodland, the behavior will be quite different than if the objective is to provide location awareness in an office courtyard with a combination of grass, glass, and concrete. If the R200 may capture the sun, expect lens artifacts to interfere with tracking and possible creation of spurious geometry in your application.

Daylight is not your friend with RealSense (or any other structure/depth sensing cameras that I've seen)  but it's not always your enemy either. Do some quick tests with a laptop and an R200 in the environment you expect to encounter. Test early or late in the day and around noon, and test under bright skies and on a cloudy day and get a range of anticipated behaviors so you know what to expect. It's really the only way to be sure.

[samontab]

Joe R. wrote:

OK thanks, does anyone know how susceptable the sensor is to sunlight?

Of course it will be very susceptible to sunlight, as sunlight contains a lot of IR light, and is very powerful. It is a difficult problem to solve. Think of it as trying to get a picture at night time, when no light is around, it is very hard. The solution there would be to put more light. The solution to your problem would be to remove the sunlight.

Interestingly, the R200 will be affected in two ways, the IR pattern will be absolutely covered by the sun, so you will not be able to see any pattern in the IR cameras. This leads to no depth reconstruction using structure light. However, having a good amount of IR light will benefit the passive IR stereo depth reconstruction part. You just need to make sure that the cameras are exposing correctly (you don't want to have completely white images because of over exposure).

Now, if you are more interested in an active sensor that would work in sunlight, you are asking for trouble, because the sun is extremely powerful compared to any type of light emitted by these sensors. There are some solutions out there that work better than others, specially time-of-flight sensors, like the absolutely amazing technology of the Kinect 2, or some other (very expensive) Lidars.

To be honest, it is better to use passive technologies when dealing with the sun, and even then it is tricky as the light changes all the time (I've worked processing data from smart security cameras, and I know how much those images can change).

[Joe_R_]

Thanks both of you, that is very useful to know and we will be testing for our application over the next 2 weeks.

In terms of the passive depth sensing, is this in the SDK or has anyone had success using the left and right IR images to perform depth sensing? We will probably write our own algorithms that make use of structured light in low light conditions and passive stereoscopic in bright light conditions. Is this possible? Is there anything else we should be considering?

Cheers

[Ning_R_]

I'm not sure if, we can remove the IR filter attached to the cameras R200 using now. If so, we can modify it into the depth camera with both IR and ambient light, by making our own IR-filter glasses.  When the sunlight was weak, we put the glasses on and the R200 just working as it is now; when outdoor, we take off the glasses, then R200 working as traditional stereovision camera. 

But I don't know R200 using OV4682 RGB-IR module, like Project Tango, if so, theorotically we can make this "softly".

[Mario_M_1]

Hey, Columbia University helmet startup company looking to to develop a head measurement app for fitted helmets using depth sensor technology. Any assistance/guidance would be greatly appreciated. Thanks. Mario