Person tracking lost track easily

[tlian5]

Hi, I'm working on a camera based person following bot using R200, please see the video below is what I got so far:

https://youtu.be/L-9M0H3LjAQ Camera based person following robot - YouTube

The problem is that the R200 camera randomly lost track of the person it is tracking while moving, is there anyway to make it more robust and reliable? I've been thinking if I add another camera will help with the situation?

I'm using SDK 2016R2 in C# .

Thanks.

[MartyG]

I did some research on the idea of widening the R200's field of view (FOV). RealSense robotics expert McCool - who I have linked into this discussion so he can hopefully contribute - suggested linking three R200s together to provide a 150 degree FOV in a horizontal camera arrangement or 130 degrees in a vertical camera arrangement.

Skip down to the third paragraph of the above post that begins with 'What WILL work'.

[tlian5]

Thank you sir, linking three R200s together to provide a 150 degree FOV sounds greate, is there any more details about how to connect them together and how to use 3 camers in the SDK? I'm using a windows laptop.

Thank you very much.

[MartyG]

McCool documented the building of his three-camera bot in this post.

[tlian5]

Thank you very much Marty! That Turtlebot 3 looks really good!

From the article I can see that three R200 are connected to a USB3.0 hub, and the hub is connected to Joule. In my case I can connected three R200 to my PC(I have three USB 3.0 port), but what should I do with the SDK software, how can I calibrate these three R200?

[MartyG]

Since the three R200s are probably just looking straight ahead like they normally do, then you could likely just use the normal R200 Camera Calibrator tool on your PC to calibrate each one individually.

https://downloadcenter.intel.com/download/24958/Intel-RealSense-Camera-Calibrator-for-Windows- Download Intel® RealSense™ Camera Calibrator for Windows*

[tlian5]

Thanks Marty, I just downloaded this Camera Calibrator, but when I run it, I got an error,

I wonder if you seen this before:

[MartyG]

This is a very rare error. So far it has only been seen on the Wacom MobileStudio Pro and Acer Aspire Switch 12S. What PC device are you using, please?

[tlian5]

Thanks Marty.

I'm using a Dell Alienware 15 laptop, using windows 10 Pro with latest updates.

The camera itself is working perfectly fine.

[tlian5]

Same error happened to my other laptop Dell Inspiron 15 as well. I tried both 32bit and 64bit Calibrator , same result.

[tlian5]

Also, I wonder how this Calibrator tool works? Can this Calibrator tool merge data from three R200 and combine them into one so that in the SDK I can use those three cameras as one camera instance?

[MartyG]

Ok, thank you very much for the report of the problem on your various laptops. It is possible that the problem affects more machines than just the Wacom and Acer, but people have not reported it because they have not tried using the calibrator program and encountered the error. Since Intel's development for the R200 has ended, there will not be an updated calibrator for the R200 released.

Calibration can also be done with scripting if you know how to compile and build a script in Visual Studio.

Here's an example:

https://software.intel.com/en-us/forums/realsense/topic/597711 R200 Calibration Data

I am sure the Calibrator tool was only designed for one R200 camera at a time.

[tlian5]

OK, After some more research, I think the multiple cameras is not supported in the official SDK, the only thing I can find is in librealsense there is an example 'cpp-multicam'.

[MartyG]

In the RealSense SDK for Windows, you can have multiple cameras attached to the PC but only one can be active at a time, so you have to cycle through them in sequence, deactivating one and activating the next one until you get back to the first camera in the sequence and the cycle begins again (1, 2, 3, 1).

Librealsense does support multiple cameras.

[tlian5]

Thank you very much Marty for your patience, I see now that could be the limit of the SDK I'm using. Cycle through the cameras may not be what I need...I think what I really want is a "Virtual camera" which combine the view of all three R200.

[MartyG]

You're very welcome. McCool can likely give advice about how he programmed his own three-camera setup.

Here's a discussion about combining camera views into one program.

[tlian5]

Thanks Marty,

It will be great if there are some sample code in c# .

BTW I figured out why the I fixed the camera calibrator error, what I did:

Uninstall this R200 RGB from device manager, unplug the R200, plug R200 back in. And the camera calibrator is working now.

[MartyG]

Glad you got the camera working with the Calibrator!

On the Wacom and Acer devices that had the DS_ARG_INVALID error, the solution was to uninstall the entire Depth Camera Manager, as an incompatibility with the DCM on those devices was apparently causing the Calibrator to not work. Since you said that your camera was otherwise working totally fine, I was hesitant to recommend that course of action in case it broke something on your laptop that wasn't broken! So I'm very pleased that the method you tried worked out for you.

I remember from previous cases that C# calibration script code is very hard to find. There was one place where I located some though.

[tlian5]

Thanks Marty,

I will hold the thought, since I couldn't find an easy way to increase the field of view by using multi R200 cameras..

The D435 camera looks like has a much wider view range, I hope the SDK coming with it has good support for windows and c# .

[MartyG]

The D-cameras should be an excellent fit for your project, since they were designed to be suited for robotics and autonomous vehicles.

https://www.youtube.com/watch?v=pvXJSn22ujU&t=2s Intel RealSense 400 for self-driving cars, autonomous drones, Robots, for indoor and outdoor use - YouTube