Are there any examples or pointers for experimenting with realsense on the joule? I note that /usr/lib contains librealsense.so and /usr/include includes a librealsense directory. But any getting started information would be good, even if it is: "yes it works go to this link for some examples" or "no it doesn't work quite yet".
Thank you for contacting us, we'll be glad to help you.
Right now, all available examples are in this website https://software.intel.com/en-us/node/672542 https://software.intel.com/en-us/node/672542. I believe there are no Realsense examples so far, however, there will be more documentation in the near future. You can also check this thread https://software.intel.com/en-us/forums/realsense/topic/637894 https://software.intel.com/en-us/forums/realsense/topic/637894, there are some examples that you can try to make work with the Joule board.
Note that the Intel RealSense R200 is not officially supported on the Joule. The main focus for the Joule is actually the newer and more capable ZR300. If you are impatient the R200 can be made to work, however are a few issues that you have to work around. The workarounds I want to share here are not perfect or fully tested, there are no guarantees these workarounds will be acceptable to everyone, nor that the issues will ever be fully resolved. Given those grains of salt, and the understanding that the following is basically a collection of useful hacks and not any kind of official support (even though I am an Intel employee...), I would like to share my personal experience so far with using the R200 on Joule. It's also possible others will find better workarounds
So with those provisos out of the way...
First, install Ubuntu using this thread:
This will allow you to install a regular Ubuntu ISO. Note however that this installation approach does not yet have drivers for the pins afaik nor is it officially supported. But the USB3.0 port works, which is what matters for using the RealSense camera. I also installed ROS, since I'm building a robot with this setup, something I will comment about in more detail elsewhere.
You MAY have to also use a powered USB hub. I've heard reports that the R200 also works without a hub, and may even work better, but it may also work worse, depending on your hub, cables, etc. I haven't tried all the variants; in my application, I needed extra USB ports, so I used a hub. You also may or may not need a powered hub, it depends on what else you have plugged in. Anyway, I used this hub, which also has the advantage that it can be driven from the same 12V power supply as the carrier board in the Joule development kit (using a Y adapter and the same barrel power connector). I also lets you use your own micro-B-to-A USB3.0 cable so you can pick a length that works for you: https://www.amazon.com/Transcend-Information-SuperSpeed-USB-TS-HUB3K/dp/B005D69QD8 https://www.amazon.com/Transcend-Information-SuperSpeed-USB-TS-HUB3K/dp/B005D69QD8
After I installed 64-bit Ubuntu 16.04, I then installed librealsense. You can install the librealsense library following the normal install instructions. Make sure you use the "Ubuntu 16.04" V4L setup script when you load the kernel module. I also found I did not have enough space to do this on the main eMMC with other things I had already installed (it downloads 1.4GB of stuff before it can build the patch...), so I formatted the included 16GB SD card in ext4 (which you need apparently for the patch to work) and ran the install there. Note though that if you leave the 16GB SD card in when you boot Ubuntu it will hang (I'm still trying to find a workaround for this issue; some boot order thing, perhaps), so eject it before rebooting.
Unfortunately if you then run the standard test samples in librealsense/bin (eg cpp-capture) with an R200 it will hang almost immediately (actually, the color stream keeps going, but the depth stream hangs). There is a workaround however: if you lower the framerate to 30Hz, and/or decrease the depth resolution to 320x240, the connection appears to be stable. You can make these the defaults by editing the presets in src/ds-device.cpp before doing the RealSense install. You can also edit the source code for the examples to avoid the higher frame rates and resolutions. Of course in your own applications (i.e. using ROS) you should pick the modes that work.
The issue does not seem to be the performance of the Joule (which is excellent), but some kind of USB3.0 driver, kernel, or timing issue. It may also be something else going on completely; I haven't tried other OS variants or kernel versions in particular. The Intel RealSense Robotic Development Kit uses a CherryTrail (which actually has lower performance) but is stable with the R200. I should mention though I that I use active cooling on my Joule and run it with the "performance" governor enabled which lets me maintain a consistently high clock rate. I haven't tested it with the active cooling off yet
At any rate, avoiding "stressing" the USB3.0 connection by lowering the data rate seem to improve stability. HOWEVER I have not run the R200 yet for a long time to really confirm that it is acceptably stable with these changes, and I'm still not sure what combinations of modes works best.
For the applications I am looking at, robotic navigation (SLAM), the lower frame rate and resolution are acceptable, though. In fact the minimum useful range is reduced (eg improved) at the lower depth resolution.