Items with no label
3335 Discussions

Person tracking lost track easily

tlian5
Beginner
2,783 Views

Hi, I'm working on a camera based person following bot using R200, please see the video below is what I got so far:

https://youtu.be/L-9M0H3LjAQ Camera based person following robot - YouTube

The problem is that the R200 camera randomly lost track of the person it is tracking while moving, is there anyway to make it more robust and reliable? I've been thinking if I add another camera will help with the situation?

I'm using SDK 2016R2 in C# .

Thanks.

0 Kudos
19 Replies
MartyG
Honored Contributor III
1,600 Views

I did some research on the idea of widening the R200's field of view (FOV). RealSense robotics expert McCool - who I have linked into this discussion so he can hopefully contribute - suggested linking three R200s together to provide a 150 degree FOV in a horizontal camera arrangement or 130 degrees in a vertical camera arrangement.

Skip down to the third paragraph of the above post that begins with 'What WILL work'.

0 Kudos
tlian5
Beginner
1,600 Views

Thank you sir, linking three R200s together to provide a 150 degree FOV sounds greate, is there any more details about how to connect them together and how to use 3 camers in the SDK? I'm using a windows laptop.

Thank you very much.

0 Kudos
MartyG
Honored Contributor III
1,600 Views

McCool documented the building of his three-camera bot in this post.

0 Kudos
tlian5
Beginner
1,600 Views

Thank you very much Marty! That Turtlebot 3 looks really good!

From the article I can see that three R200 are connected to a USB3.0 hub, and the hub is connected to Joule. In my case I can connected three R200 to my PC(I have three USB 3.0 port), but what should I do with the SDK software, how can I calibrate these three R200?

0 Kudos
MartyG
Honored Contributor III
1,600 Views

Since the three R200s are probably just looking straight ahead like they normally do, then you could likely just use the normal R200 Camera Calibrator tool on your PC to calibrate each one individually.

https://downloadcenter.intel.com/download/24958/Intel-RealSense-Camera-Calibrator-for-Windows- Download Intel® RealSense™ Camera Calibrator for Windows*

0 Kudos
tlian5
Beginner
1,600 Views

Thanks Marty, I just downloaded this Camera Calibrator, but when I run it, I got an error,

I wonder if you seen this before:

0 Kudos
MartyG
Honored Contributor III
1,600 Views

This is a very rare error. So far it has only been seen on the Wacom MobileStudio Pro and Acer Aspire Switch 12S. What PC device are you using, please?

0 Kudos
tlian5
Beginner
1,600 Views

Thanks Marty.

I'm using a Dell Alienware 15 laptop, using windows 10 Pro with latest updates.

The camera itself is working perfectly fine.

0 Kudos
tlian5
Beginner
1,600 Views

Same error happened to my other laptop Dell Inspiron 15 as well. I tried both 32bit and 64bit Calibrator , same result.

0 Kudos
tlian5
Beginner
1,600 Views

Also, I wonder how this Calibrator tool works? Can this Calibrator tool merge data from three R200 and combine them into one so that in the SDK I can use those three cameras as one camera instance?

0 Kudos
MartyG
Honored Contributor III
1,600 Views

Ok, thank you very much for the report of the problem on your various laptops. It is possible that the problem affects more machines than just the Wacom and Acer, but people have not reported it because they have not tried using the calibrator program and encountered the error. Since Intel's development for the R200 has ended, there will not be an updated calibrator for the R200 released.

Calibration can also be done with scripting if you know how to compile and build a script in Visual Studio.

Here's an example:

https://software.intel.com/en-us/forums/realsense/topic/597711 R200 Calibration Data

I am sure the Calibrator tool was only designed for one R200 camera at a time.

0 Kudos
tlian5
Beginner
1,600 Views

OK, After some more research, I think the multiple cameras is not supported in the official SDK, the only thing I can find is in librealsense there is an example 'cpp-multicam'.

0 Kudos
MartyG
Honored Contributor III
1,600 Views

In the RealSense SDK for Windows, you can have multiple cameras attached to the PC but only one can be active at a time, so you have to cycle through them in sequence, deactivating one and activating the next one until you get back to the first camera in the sequence and the cycle begins again (1, 2, 3, 1).

Librealsense does support multiple cameras.

0 Kudos
tlian5
Beginner
1,600 Views

Thank you very much Marty for your patience, I see now that could be the limit of the SDK I'm using. Cycle through the cameras may not be what I need...I think what I really want is a "Virtual camera" which combine the view of all three R200.

0 Kudos
MartyG
Honored Contributor III
1,600 Views

You're very welcome. McCool can likely give advice about how he programmed his own three-camera setup.

Here's a discussion about combining camera views into one program.

0 Kudos
tlian5
Beginner
1,600 Views

Thanks Marty,

It will be great if there are some sample code in c# .

BTW I figured out why the I fixed the camera calibrator error, what I did:

Uninstall this R200 RGB from device manager, unplug the R200, plug R200 back in. And the camera calibrator is working now.

0 Kudos
MartyG
Honored Contributor III
1,600 Views

Glad you got the camera working with the Calibrator!

On the Wacom and Acer devices that had the DS_ARG_INVALID error, the solution was to uninstall the entire Depth Camera Manager, as an incompatibility with the DCM on those devices was apparently causing the Calibrator to not work. Since you said that your camera was otherwise working totally fine, I was hesitant to recommend that course of action in case it broke something on your laptop that wasn't broken! So I'm very pleased that the method you tried worked out for you.

I remember from previous cases that C# calibration script code is very hard to find. There was one place where I located some though.

0 Kudos
tlian5
Beginner
1,600 Views

Thanks Marty,

I will hold the thought, since I couldn't find an easy way to increase the field of view by using multi R200 cameras..

The D435 camera looks like has a much wider view range, I hope the SDK coming with it has good support for windows and c# .

0 Kudos
MartyG
Honored Contributor III
1,600 Views

The D-cameras should be an excellent fit for your project, since they were designed to be suited for robotics and autonomous vehicles.

https://www.youtube.com/watch?v=pvXJSn22ujU&t=2s Intel RealSense 400 for self-driving cars, autonomous drones, Robots, for indoor and outdoor use - YouTube

0 Kudos
Reply