D435i "motion module force pause" when turning on IMU

[FZhao8]

I'm on Raspbian Buster, Raspberry Pi 4, librealsense 2.28.0

I actually just finished updating the firmware to 5.11.11.100 , reconnected the camera. I launch realsense-viewer and there was a red dialog that said "motion module force pause" when I clicked on it, and it says the same thing in the log

Attempting to turn on IMU data resulted in realsense-viewer crashing, console message says

WARNING [2918077504] (types.cpp:48) hwmon command 0x4f failed. Error type: No data to return (-21).

Plugged it back into my Windows 10 computer and IMU data shows up fine.

What's wrong? The camera is fine since it works on Windows, so I'm not super worried. (and I'm moving it to a Jetson Nano later)

[MartyG]

The D435i's IMU is not calibrated in the factory, unlike the imaging sensors which are factory-calibrated. The IMU on the D435i therefore has to be calibrated by the user. Intel's support advice on the hwmon error you encountered is that it means the IMU is uncalibrated (which is its default setting). It is advised that you calibrate the IMU using Intel's guide document if you need to make use of the IMU.

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_Depth_D435i_IMU_Calib.pdf

[FZhao8]

Thanks! I will try this after I finish 3D printing the whole project box. It's weird that it crashes the GUI...

[FZhao8]

I spent the day installing realsense-ros , and then when launching it, I get the following error:

[ INFO] [1568342288.983922734]: Expected frequency for infra2 = 30.00000 [ INFO] [1568342288.991215568]: Expected frequency for color = 30.00000 [ INFO] [1568342289.018412228]: setupStreams... [ INFO] [1568342289.138761438]: insert Depth to Stereo Module [ INFO] [1568342289.138910921]: insert Color to RGB Camera [ INFO] [1568342289.138987329]: insert Infrared to Stereo Module [ INFO] [1568342289.139036756]: insert Infrared to Stereo Module [ INFO] [1568342289.139099164]: insert Gyro to Motion Module [ INFO] [1568342289.139184109]: insert Accel to Motion Module 12/09 19:38:09,163 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b 12/09 19:38:09,176 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b 12/09 19:38:09,186 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b 12/09 19:38:09,197 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b 12/09 19:38:09,208 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b 12/09 19:38:09,218 WARNING [2768233488] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 86, error: Resource temporarily unavailable, number: b [camera/realsense2_camera_manager-2] process has died [pid 1090, exit code -7, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/pi/.ros/log/822bb8ae-d5cf-11e9-aaba-dca632195a3e/camera-realsense2_camera_manager-2.log]. log file: /home/pi/.ros/log/822bb8ae-d5cf-11e9-aaba-dca632195a3e/camera-realsense2_camera_manager-2*.log

Since it's kind of cryptic and happened after the gyro and accel stuff, I thought it might have to do with the calibration

I attempted to perform calibration while on the Raspberry Pi, I was met with a very simple "Bus Error" in the Python console before the process crashes

I successfully completed the calibration on my Windows 10 computer and saved it to EEPROM.

Now, moving back to the Raspberry Pi, roslaunch still gives the same error as above

realsense-viewer now gives a different error when I try to get IMU data (RGB and depth still works fine)

12/09 21:17:15,996 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 12/09 21:17:16,007 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 12/09 21:17:16,018 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 12/09 21:17:16,029 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 12/09 21:17:16,039 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 12/09 21:17:16,060 WARNING [2816472128] (messenger-libusb.cpp:94) bulk_transfer returned error, endpoint: 134, error: Resource temporarily unavailable, number: 11 Bus error

Research on the internet yields no good solution, running sudo is not possible, and also, the log file mentioned in the error message doesn't actually exist

Please help

edit: tried FW update again, no help, also tried using the USB 2.0 port instead, same error, plus it complained about the IR stream not being supported

edit: I disabled gyro and accel in the .launch file and was met with no errors

[MartyG]

Do you get the same problem if you plug the camera in after booting instead of booting up with the camera plugged in, please?

[FZhao8]

Yes, it happens in both cases. I do need it to work without unplugging, as this project is going to be inside a totally enclosed box. (don't worry it'll be cooled with a 40mm fan)

Once the error happens (the "bulk_transfer returned error"), the whole camera must be unplugged and replugged-in before anything else works.

I got some dmesg

[20391.538943] usb 2-2: new SuperSpeed Gen 1 USB device number 3 using xhci_hcd [20391.569776] usb 2-2: New USB device found, idVendor=8086, idProduct=0b3a, bcd Device=50.bb [20391.569792] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [20391.569805] usb 2-2: Product: Intel(R) RealSense(TM) Depth Camera 435i [20391.569817] usb 2-2: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435i [20391.569829] usb 2-2: SerialNumber: 908323050034 [20391.573663] uvcvideo: Unknown video format 00000050-0000-0010-8000-00aa00389b 71 [20391.573815] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Came ra 435i (8086:0b3a) [20391.577973] uvcvideo: Unable to create debugfs 2-3 directory. [20391.578304] uvcvideo 2-2:1.0: Entity type for entity Intel(R) RealSense(TM) D epth Ca was not initialized! [20391.578320] uvcvideo 2-2:1.0: Entity type for entity Processing 2 was not ini tialized! [20391.578334] uvcvideo 2-2:1.0: Entity type for entity Camera 1 was not initial ized! [20391.578733] input: Intel(R) RealSense(TM) Depth Ca as /devices/platform/scb/f d500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/input/input8 [20391.581523] uvcvideo: Unknown video format 36315752-1a66-a242-9065-d01814a8ef 8a [20391.581547] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Came ra 435i (8086:0b3a) [20391.584527] uvcvideo: Unable to create debugfs 2-3 directory. [20391.584922] uvcvideo 2-2:1.3: Entity type for entity Processing 7 was not ini tialized! [20391.584937] uvcvideo 2-2:1.3: Entity type for entity Extension 8 was not init ialized! [20391.584951] uvcvideo 2-2:1.3: Entity type for entity Camera 6 was not initial ized! [20391.593503] hid-generic 0003:8086:0B3A.0005: hiddev96,hidraw0: USB HID v1.10 Device [Intel(R) RealSense(TM) Depth Camera 435i Intel(R) RealSense(TM) Depth Ca mera 435i] on usb-0000:01:00.0-2/input5 [20399.931285] usb 2-2: usbfs: process 11919 (nodelet) did not claim interface 4 before use [20400.006829] Alignment trap: not handling instruction 9d928a00 at [<add468c8>] [20400.006835] Unhandled fault: alignment exception (0x221) at 0xa56fecce [20400.013510] pgd = b9566968 [20400.016253] [a56fecce] *pgd=1824e003, *pmd=f93ee003[20399.931285] usb 2-2: usbfs: process 11919 (nodelet) did not claim interface 4 before use [20400.006829] Alignment trap: not handling instruction 9d928a00 at [<add468c8>] [20400.006835] Unhandled fault: alignment exception (0x221) at 0xa56fecce [20400.013510] pgd = b9566968 [20400.016253] [a56fecce] *pgd=1824e003, *pmd=f93ee003 [20485.388793] Alignment trap: not handling instruction 9d928a00 at [<adc468c8>] [20485.388801] Unhandled fault: alignment exception (0x221) at 0xa56fecce [20485.388972] usb 2-2: usbfs: process 12474 (nodelet) did not claim interface 4 before use [20485.395481] pgd = e461d59e [20485.398228] [a56fecce] *pgd=178b0003, *pmd=f7df8003

With gyro and accel disabled, it all works fine, none of those errors, and I can actually get visual data into rviz (I am working on rtabmap_ros, but that's a totally different story, but I am going to ignore those uvcvideo errors). So at this point, I actually suspect the code for the ROS nodelet is actually missing a libusb call to claim the interface. I've coded enough with libusb to know how confusing that call is... depending on what platform your code is running on, you don't know if you need to claim it or not, and sometimes without a good way to check beforehand.

Is this something simple to fix? I have all the source code already, everything (ROS, librealsense, realsense2_camera) is built and installed from source.

rtabmap_ros is giving me errors too but probably unrelated, I'll ask those questions on the ROS forum

[MartyG]

The RealSense ROS GitHub may indeed be the best place to ask about this ROS question. First though, you could try patching uvcvideo.

[FZhao8]

I patched uvcvideo, it was not easy. The patch file from Intel's github is behind the kernel for Raspbian Buster. I had to manually write in the new video formats myself.

What did that accomplish? Cheese now works with the camera... No more errors on dmesg

But IMU data is still not coming through due to "bus error", I still get libusb errors for the bulk transfer, and rtabmap has a bunch of its own problems.

[ 74.637055] usb 2-2: usbfs: process 1059 (rtabmap) did not claim interface 4 before use [ 74.740640] usb 2-2: usbfs: process 1059 (rtabmap) did not claim interface 1 before use [ 74.825613] Alignment trap: not handling instruction 9d928a00 at [<b423c8c8>] [ 74.825621] Unhandled fault: alignment exception (0x221) at 0x8641a96e [ 74.832653] pgd = 8fb96e2e [ 74.835410] [8641a96e] *pgd=19820003, *pmd=f7e9c003

The good news is that I ordered a D415 and it should be here next week...

[MartyG]

The interface claim on Linux seems to be something to do with permissions.

On Raspian there are some installation quirks, I believe, that merit it having its own install guide for Librealsense. As a next step, as it is a libusb issue now apparently, you could try updating the udev rules.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_raspbian.md#update-udev-rule

[FZhao8]

I did that before, even realized that "sudo udevadm control --reload-rules && udevadm trigger" needs to become "sudo udevadm control --reload-rules && sudo udevadm trigger", but it still doesn't help. No changes in the error messages I get, either in terminal or in dmesg

[MartyG]

Apologies for the delay in responding further, I was carefully researching your latest information.

It may be getting to the point where posting a question on GitHub is the best course of action in order to get specialist ROS advice.

Are you launching the ROS session with the camera.launch instruction, please?

roslaunch realsense2_camera rs_camera.launch

[FZhao8]

I'm way past that now, I have a variation of the launch file with the gyro and accel turned off to avoid the crash, rviz can subscribe to the images, the problem is with rtabmap, which I'm hypothesizing that is trying to contact IMU and then crashing

I am currently rebuilding librealsense back to 2.25.0 instead of 2.28.0 , and backing off rtabmap to 0.18.0 from 0.19.3

I noticed something... when I run cmake for rtabmap, it says

-- -------------------------------------------- -- Info : -- Version : 0.18.0 -- CMAKE_INSTALL_PREFIX = /usr/local -- CMAKE_BUILD_TYPE = Release -- CMAKE_INSTALL_LIBDIR = lib -- BUILD_APP = ON -- BUILD_TOOLS = ON -- BUILD_EXAMPLES = ON -- BUILD_SHARED_LIBS = ON -- CMAKE_CXX_FLAGS = -fmessage-length=0 -fopenmp -- PCL_DEFINITIONS = -DDISABLE_OPENNI2;-DDISABLE_PCAP;-DDISABLE_PNG;-DDISABLE_LIBUSB_1_0

Why is there a -DDISABLE_LIBUSB_1_0? I did some basic poking around and found no way to get rid of that definition...

There are other concerning cmake warnings that I ignored the first time around, and it started to make sense why I was getting non-fatal errors in rtabmap... I think the Raspbian distributions of some packages are minimal and missing required files, I need to rebuild everything from source, starting from VTK, then PCL, then RTAB-Map... This will take a few hours or even another few days...

Thanks for all your help so far, I work fast but not in a hurry, have a good weekend

(there's gotta be a easy-to-use cross compiling framework out there... is there something I can give my Raspberry Pi's SSH credentials to, and it does cross compiling for me with my Intel i7 instead of the ARM?)

[MartyG]

Librealsense does have a non-patching method for installing it in cases where it is not working using the regular installation methods. It is called libuvc-backend

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md