Why is my accuracy on our T265 so poor?

[JOdom1]

We have mounted a T265 on our robot. It's mounted about 1 meter above the ground, positioned facing forward, and connected to a Jetson running the T265 localization example code. We are getting poor accuracy to the point that I think we must be doing something wrong or we must have a faulty unit. At the zero point, we get accuracy +- 2 cm, which is fine. When we move to 3 m, 5 m and 7 m, we see increasingly poor accuracy, with up to a meter or more of error. When we bring the robot back to zero, we see that it is accurate again, but only at zero. Some important notes:

• We have tried this on our own Python code and on the Python example, with the same results.
• We have connected the T265 directy to a Windows laptop running the RealSense viewer and walked alongside the robot, seeing the same poor results.
• We have tried adding objects to the physical environment to make the environment more feature rich.
• We have tried taking the camera to different rooms with different lighting conditions, and even took it outside, seeing similar.
• We angled the camera and remounted the camera on the side of the robot, but the accuracy was still poor.

Note that we see the same poor results whether we plug the camera into a Jetson running the example code, or into a Windows laptop with the RealSense viewer. Is inaccuracy of one meter or more typical?

[MartyG]

Your results are consistent with those of a user in the discussion linked to below, where they had 0.5 m drift over a 7 m distance.

https://github.com/IntelRealSense/librealsense/issues/3970

The discussion offers a number of suggestions for improving the accuracy and confidence level of the tracking. These include:

- Using wheel odometry

- Giving the robot enough initial motion to generate a high confidence and then reducing to normal speed.

If your robot travels in a circle, you may gain additional accuracy by disabling the 'pose jumping' feature.

https://github.com/IntelRealSense/librealsense/blob/development/doc/t265.md#are-there-any-t265-specific-options

[JOdom1]

Thank you for your helpful answer. We have experimented with wheel odometry, but I'm not confident that we're doing it right. We'll do more homework and get back to you on that. In the meantime...

Today in the robot lab we ran a linear path at high speed (starting off at 2 - 3 m/s, no wheel odometry). We output the tracker confidence and see it jump from 2 to 3 (high confidence) after the initial high speed start. Despite that, the RealSense reports almost 8 meters when we stop at 7 meters. We've measured several times to make sure we're not doing anything silly. Is one meter of inaccuracy expected when the T265 reports high confidence?

[JOdom1]

To give you more data and to see if we're doing something obviously silly, here is a video of us running the robot at high speed:

https://youtu.be/e8otdJadevU

And here is a picture of the mount (near the top of the robot facing forward):

[MartyG]

In the post about drift that I linked to earlier, it was reported by a user that at 7 meters travel, they sometimes experienced 0.5 meters overshoot (7.5 m) as well as 0.5 m undershoot (6.5 m) when at high confidence, So that would suggest that it is at least possible to achieve better results than the 1 meter margin of error that you are experiencing. Use of wheel odometry to improve accuracy is certainly highly recommended.

In yesterday's release of SDK version 2.31.0, which adds a huge number of updates, Intel added a new section about wheel odometry to the T265 documentation page that contains a basic wheeled odometry description, sample drawings and a json.

https://github.com/IntelRealSense/librealsense/blob/master/doc/t265.md#extrinsic-calibration-for-wheeled-odometry-examples

You can also read in the link below everything that was added in SDK 2.31.0:

https://github.com/IntelRealSense/librealsense/wiki/Release-Notes#release-2310

Thanks too for sharing the video and static image of your setup. I would recommend checking to see if vibration is being transmitted through the robot's frame to the T265, as it is sensitive to vibration.

https://github.com/IntelRealSense/librealsense/issues/4176

[JOdom1]

Thank you for the additional suggestions. We are studying them all now to run this problem to the ground. We tried isolating the T265 using a rubber band between the robot and the T265 to minimize vibration (see photo). We have tried wheel odometry with some interesting results, but we'll need to finish all of our homework before we send that to you because I want to make sure we're doing it right. One of our use cases is on a robot that has poor odometry due to wheel slippage, so that is why I am so keen on benchmarking the product before adding odometry. Two questions. If our wheel odometry becomes unreliable during the course of running the robot, will that make the issue worse, or is the T265 robust against odometry errors / wheel slippage? What is the expected error ellipse of the T265 when it reports high confidence?

[MartyG]

The link below has carefully constructed accuracy tests that somebody did with T265.

https://jaspereb.github.io/TestingRealsenseT265/

Generally, wheel odometry should be used with ground robots to help to ensure robust tracking accuracy. I suspect that if you cannot rely on the wheels for accuracy then you treat the robot as though it is a flying drone, and the T265 has been used successfully in flight drones. The article below provides useful guidance for drone principles with the T265.

https://diydrones.com/profiles/blogs/autonomous-flying-using-the-realsense-t265-tracking-camera-in-gps

For deep technical questions on T265 subjects such as confidence, I recommend posting a question at the RealSense GitHub for expert specialist advice about T265 from Intel RealSense team members. You can do so at the link below by clicking the 'New Issue' button.

https://github.com/IntelRealSense/librealsense/issues

[JOdom1 · Accepted solution]

We have run the T265 to the ground to understand its capabilities. We have added wheel odometry, we have carefully verified that our wheel odometry is correct, we have checked and rechecked the calibration parameters. We have experimented with the noise variance on the wheel odometry calibration. We have experimented with T265 options such as pose jumping, relocalization and mapping. We have double checked that we are setting those options correctly by fetching them from the device. Whe have tried rebooting the T265 between runs and spent hours trying to get the thing to be accurate. We have experimented with environment and mounting as I discuss above. The thing isn't any good. The closed loop accuracy is brilliant. It always knows when it is back at zero. Everything else is far worse than just wheel odometry and an IMU. It is typically 10% to 20% off on a 10 meter course, and doesn't have reproducible results. The thing just doesn't work. We are giving up.

[HDiet]

What is the minimum velocity the T265 can be used under ideal conditions. Is it possible to track velocity of about 15cm/min? I think this will be hard, because the the smaller the velocity the more the noise of the imu is present or am I wrong?

[MartyG]

Ideally, you should give the T265 enough initial motion to generate a "High" tracking confidence and understand its local position. You can then drop the speed down to your desired rate after that once the High confidence has been achieved. The link below discusses the subject in more detail.

https://github.com/IntelRealSense/librealsense/issues/3970

If you have further T265 questions, please post them at the RealSense GitHub forum and include T265 in the title of the message, as that is where T265 questions are now being handled. You can post a question by visiting the link below and clicking on the 'New Issue' button. Thanks!

https://github.com/IntelRealSense/librealsense/issues