Software Archive
Read-only legacy content
17061 Discussions

Using RealSense Applications on Xbox One as "Universal Apps"?

MartyG
Honored Contributor III
867 Views

Hi everyone,

I was just reading that the Xbox One console has a USB 3 port.  This led me along the following chain of thinking.

1.  "Universal" apps developed for Windows 10 will be able to run on Windows cellphones and Xbox One too without having to change the app (as the control interface for the app will adapt automatically to suit the device it is running on.)

2.  The RealSense SDK works awesomely well on Windows 10 (I use it myself on my RealSense dev machine.)

3.  Microsoft are updating the Xbox One OS soon from Windows 8 to Windows 10 to make this cross-play possible.

Taking all this into account, I wonder how hard it would be to plug the RealSense camera into the Xbox One's USB 3 port and and use RealSense applications on it that are created on Windows 10 and packaged as Universal Apps.

I guess the library files to run the camera would be packaged within the Universal App files so that Xbox One users would not have to install RealSense drivers on their console, much like the end-users of PC RealSense apps presumably don't have to download the SDK to be able to use RealSense applications made by other people.

What do the rest of you think?

Edit: the above train of thought is made more interesting by the news that Microsoft has developed a tool that converts Android mobiles into Windows 10 ones.  It opens the question of whether such a converted device that had the RealSense rear-facing camera in it would then be able to use RealSense apps packaged as Windows 10 Universal Apps, if the drivers for the camera were packaged within the app.  Assuming of course that anyone owning a RealSense Android would actually want to convert their device in this way!

http://techcrunch.com/2015/03/17/microsoft-android-rom/

0 Kudos
10 Replies
Vidyasagar_MSC
Innovator
867 Views

That will be a good try. With Windows 10, Microsoft is truly going universal.There will be only one OS running on all the devices including mobile phones. RealSense will take Windows 10 + Xbox One experience to the next level.

Even i tried installing Realsense SDK R2 on windows 10 and it works seamlessly. 

0 Kudos
MartyG
Honored Contributor III
867 Views

It would seem like a natural fit for Intel for the 2015 RealSense App Challenge - having Windows 8.1 as the minimum OS and Windows 10 / Xbox One as an additional option (especially as the upgrade to 10 will be free for Win 7 and 8.1 user's in the first 12 months after launch).  

It may not be a good idea to make 10 mandatory in the way that 8.1 was mandatory in the last contest, given how much trouble that caused for people who didn't want to upgrade or couldn't.  It's not impossible that the 2015 contest may adopt 10 as the recommended OS if it offers clear performance advantages over 8.1 though.  We don't know what shape the next camera will take either (IF there is a new camera for the 3rd year running) and whether it might have even more powerful features that would benefit from 10.  

I suspect though that they'll stick with the current cam for the next year and improve it through SDK updates instead.  The present cam still hasn't been used to its fullest power potential yet, and it has a lot still to give.  They'll also likely want to hold back on a new cam to give time for the mobile Android cam to develop in popularity.

The Windows 10 / Windows 10 Phone / Xbox One "develop on PC, use on all three" approach will certainly be extremely compelling for developers.  It also offers a rapid route to a RealSense mobile powered by Windows 10 too to complement the Android version, as it wouldn't necessarily need a new dedicated SDK - manufacturers could offer the same RealSense-powered device with either Android OS or Windows 10 installed on it. 

0 Kudos
Vidyasagar_MSC
Innovator
867 Views

Agreed!!! But windows 10 upgrade is free only for a year. So supporting both Win 8.1/ Win 10 atleast for few months would be a good idea. You are right, a new camera will also be a overhead.Improvements through software is the best option going forward. Even Microsoft and google are going in that direction.

0 Kudos
Hugo_Z_Intel
Employee
867 Views

I think the issue here will be the drivers for the Xbox One.

0 Kudos
MartyG
Honored Contributor III
867 Views

The drivers are certainly the unknown quantity.  There's no way for someone to test it until Microsoft do the update of Xbox One's OS to Win 10.  There would be more certainty if the camera drivers were stored in the camera (perhaps in the existing storage space where the firmware is held) so that they automatically ran in Xbox One's version of Win 10 when the camera was plugged into the USB 3 port of the Xbox instead of needing to manually install drivers via a separate download. 

Microsoft have released the app SDK in the past week, but it of course the apps created in it can't be tested with the Xbox or Windows phone yet.

http://www.extremetech.com/computing/201844-now-you-can-create-windows-10-apps-for-pcs-smartphones-and-the-xbox-one

0 Kudos
Vidyasagar_MSC
Innovator
867 Views

You can even connect to your Xbox one from your Windows 10 PC/laptop. Which is pretty cool. With the release of windows 10 developer tool preview, we can give a try. Here are steps to setup windows 10 Dev tools to play with Realsense.

https://onestopdotnet.wordpress.com/2015/03/25/windows-10-start-building-universal-apps-getting-started/ ;

 

0 Kudos
samontab
Valued Contributor II
867 Views

Although it is definitely something interesting, I reckon it makes more sense to use the Kinect v2 for the Xbox one....

0 Kudos
MartyG
Honored Contributor III
867 Views

There are certain advantages to using a RealSense instead of a Kinect 2 on Xbox One, such as greater capability in areas such as finger tracking and power in general.  However, you are right that it is also natural that people might prefer to use a Kinect 2 because (a) they already own one, and (b) it is currently the better-known device.  

The kind of scenario where an Xbox One owner might want to use RealSense is where they already own the rear-facing camera for their PC and so plugging it into their Xbox is no great problem for them.  For the time being, it may be best to try to support both RealSense and Kinect 2 in an Xbox One Universal App, offering a choice of alternative control systems in a similar way to how a racing game player can choose between a joypad and steering wheel.

This Trojan Horse approach would expose more Xbox One owners to RealSense who might otherwise not hear about it or be interested in it.  Come for Kinect, upgrade for RealSense.

On the other hand, if one creates a RealSense-only "killer app" that is extremely well reviewed and people are desperate to play, they may shell out for a RealSense just so they are able to play it, similar to the huge controller needed to play the old original-Xbox game Steel Batallion.

https://thegamersabstract.files.wordpress.com/2011/02/steel-battalion-controller.jpg

0 Kudos
Vidyasagar_MSC
Innovator
867 Views

I completely agree with Marty. Also the emotion recognition is one good feature in realsense, which is a big differentiator and game changer. I also love the voice recognition with localization - supporting local languages.

0 Kudos
MartyG
Honored Contributor III
867 Views

Microsoft have released some new info about Kinect 2 and Windows.  They aren't producing the special Kinect For Windows PC version of the camera any more and are instead offering developers the standard Xbox One camera and an adaptor to use it with Windows.

http://venturebeat.com/2015/04/02/microsoft-stops-producing-kinect-for-windows-v2-sensor-will-focus-on-kinect-for-xbox-one-and-windows-apps/

0 Kudos
Reply