RealSense SDK R4 Released

MartyG · ‎08-08-2015

Hi everyone,

I just received an email from the Intel Software Developments mailing list stating that the R4 version of the RealSense SDK is now available for download.

I checked the main RealSense SDK download page and whilst the version number doesn't say R4, the release notes PDF that accompanies it has "VR4" in its URL address, so I assume that the download link at this page is the new SDK.

https://software.intel.com/en-us/intel-realsense-sdk/download

If you have problems with that link, here's the one from the Intel email's 'download' button.

https://registrationcenter.intel.com/download.aspx?ProductID=2508

This version of the download will take you to this page:

Edit: Intel staffer Colleen has posted a sticky at the top of the forum about the R4 release.

https://software.intel.com/en-us/forums/topic/565153

MartyG · ‎08-08-2015

I have now installed the latest DCM and SDK and tested it with Unity 5.1, and thought I would provide feedback on what happened.

1. I uninstalled the previous version of the DCM, as Intel recommend doing before installing a new version of it (the main SDK does not require such a removal when updating it, as it removes the older version automatically). Then I installed the new DCM.

2. I clicked the download button on the Windows SDK download web-page, filled in the registration form and used the web downloader to download the R4 SDK. It automatically removed R3 and then downloaded and installed R4 without any problems.

3. I opened Unity 5.1 and ran the Unity Toolkit application in the RSSDK/framework/Unity folder of the SDK, then clicked the Import button to overwrite the R3 version of the toolkit files in my project.

At this point, I got an error about duplicate files. What I should have done was delete the contents of the Plugins and Plugins.Managed folders in the project before running the Toolkit.

4. I deleted the contents of Plugins and Plugins.Managed and ran the Toolkit's import again. There was no error this time when I ran my project. And since the camera lit up, it became apparent that Intel have now fixed the problem the older Toolkit had, where you had to import the 64-bit library files into your project manually because the Toolkit only loaded in the 32-bit libraries. They achieved this by changing the file import so that the 32-bit and 64-bit library folders are both copied into the Plugins folder.

5. Although the project started up when run and the green light of the camera came on, there was a slightly worrying moment when Unity stated that the project now contained files with obsolete code in, and asked for permission to convert the scripts into the latest Unity 5 versions of the instructions.

Fortunately, I made sure to backup my project before installing the new SDK, so I had no fear about letting it go ahead with the conversion since I could just load in the project backup if something went terribly wrong.

Fortunately, there was no problems resulting from the conversion and when the project started up, I was able to control my full-body avatar with the new R4 SDK with no trouble at all. Yay! And as with previous SDK updates, the project seemed to be running faster than before.

When I tried to do a full-screen Build and Run though, a familiar problem from when Unity 5 first launched reared its head again. The project refused to go into Build and Run mode, bringing the build window up instead.

Unity built the project and exported it as a standalone, but when that standalone was run outside of Unity the camera refused to light up. This was the exact same problem at the Unity 5 launch, when the camera wouldn't start because it was trying to use the 32-bit libraries instead of the 64-bit ones.

So I tried the old trick of deleting the contents of Plugins and Plugins.Managed and hand-importing the DLLs from the bin/x64 folder, but this technique no longer worked. Instead, when doing Build and Run, a red error message saying "Plugins colliding with each other" kept appearing.

So I can continue development of my project in the editor but not do Build and Run tests right now. Hopefully a solution will soon be found!

Edit: I found the solution to get Build and Run working with the new Toolkit!

STEP ONE

Delete the DLLs in Plugins and Plugins.Managed and manually copy in the files from the SDK folder.

STEP TWO

Run the Unity Toolkit application. BUT do not just click Import. Un-tick the 'Plugins' and 'Plugins.Managed' options from the install list so that they won't be imported (which doesn't matter, as you've just hand-copied the new DLLs into your project).

Then Build and Run ... runs!

MartyG · ‎08-09-2015

I tested the new alpha support for building web-browser versions of applications this morning in Unity 5.1, the day after R4's launch. It is marked as being an Alpha (very early) feature for a reason. If you are going to try it, absolutely make sure that you have a backup of your project first.

When I selected 'Web Player' from the Build Settings window and did a Build and Run, Unity deleted every single Action script from the project's objects (the Action script folder itself in the Assets panel was not deleted) and then hung up halfway through creating the build. I did this twice, with the exact same result each time.

Edit: after the web browser test, I tested the release notes' assertion that face tracking was much improved in the R4 release. This did seem to very much be the case, especially with eyebrow tracking (which used to be one of the hardest landmarks for the camera to detect motion in). The eyebrows of my avatar now flexed up and down easily when the RL brows were flexed, and the head and the eye pupils moved very smoothly in response to nose-tracking.

After testing the face tracking I tested the hand tracking, which also claims improvements in the release notes. This was a bit harder to measure in my avatar, since the hand-tracked parts such as the arms and hands are always in motion during tracking. My overall impression was that there was indeed significant improvement.

The arms could now easily do intricate poses that had been a bit more difficult to achieve in the previous R3 SDK, and occurrences of "joint flipping" that would break the arm poses til you reset the tracking recognition were significantly reduced, going from happening 1 in 3 times in R3 to hardly ever happening in R4. On the very rare occasions that the joints did go a bit wonky, they instantly reset themselves without having to stop tracking, providing the player with a much smoother and professional user experience.

Next, I turned off the avatar upper and lower arm movement so I could analyze the finger bending movement without the hands moving around. The release notes claimed improved finger joint tracking and this too was true, as the failure rate of finger bending and unbending on the avatar's virtual hands had reduced significantly and the thumbs were easier to move precisely where you wanted to them to go.

It was a bit of a disappointment that my greatest wish-list feature for the past year - being able to disable recognition of wrist side-turning by ticking a box - still had not been implemented, as it makes it very hard to use virtual hands to reliably grip an object in the fingers when the arm keeps getting pulled away from the object when you turn the hand. Overall though, the R4 release meets many of the other things on my wish-list, so congrats to Intel for that!

Xintian_W_Intel · ‎08-18-2015

The Unity samples are written in Unity 4, as the SDK must support both Unity 4 & 5. When you launch in Unity 5, you will see the conversion window. Unity3d does a good job of conversion.

Here is what we found as a known limitation in the Scene Perception sample:

MartyG · ‎08-18-2015

Thanks for the info, Xintian!