Community
cancel
Showing results for 
Search instead for 
Did you mean: 
RKura
New Contributor I
3,291 Views

Help with Intel Aero drone

Hello,

I bought the Intel Aero compute board a month ago and started working on it and port my application to it. I was able to get it on my home WiFi network and have my application run on it.

I then went on to purchase the Intel Aero RTF last week. This is what I want to do with it as starters -

1. Connect it to my home WiFi network and have it ping my home laptop

2. Port my application to the drone

3. Have my application process video streaming taken by the cameras on the drone --> Has anyone done this and if so, how should I proceed ? I am reading thru some information that I found online but any help and directions would be much appreciated.

best

Rohan

0 Kudos
18 Replies
RKura
New Contributor I
65 Views

For the RealSense camera, I have gone thru the following article - https://software.intel.com/en-us/articles/intel-aero-realsense-streaming-rgb-depth-data# _Toc467619798 Intel® Aero Compute Board and Intel® RealSense™ Technology for Wi-Fi* Streaming of RGB and Depth Data | Intel® Software

I have an application that I plan to port and run on the RTF drone. I want to start using data (live video streaming) from the realsense camera and feed it to my application. Can I, from within my application code, do what this article is doing ? i.e detect the camera and enable the camera and when there is video streaming, feed it to my application buffers ? Would that work ?

Rohan

idata
Community Manager
65 Views

Hello rohanK23,

 

 

Thank you for your interest in the Intel® Aero Ready to Fly Drone.

 

 

You mention that you were able to connect your Aero Compute Board to your home WiFi network, and also to run your application on it. Both processes are done in the same way in the Aero Ready to Fly.

 

 

Regarding the processing of streaming video inquiry, I'm curious to see what information you have been reading. A good place to start is to check the librealsense examples found here https://github.com/intel-aero/meta-intel-aero/wiki/RealSense. Also, you may want to check the WiFi Streaming section https://github.com/intel-aero/meta-intel-aero/wiki/WiFi-Streaming. Applications can be developed using the librealsense and GStreamer libraries to process the data capture by the camera (just keep in mind that GStreamer is out of the support of this technical support team).

 

 

I haven't personally tested something like what you are trying to accomplish, but I don't see why it shouldn't work (especially if you adapt the code from the article to suit your needs). If you try it, please share your experience with the community.

 

 

If you have any other question, don't hesitate to contact us.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

Hi Andres,

I have been looking at this : https://software.intel.com/en-us/articles/intel-aero-realsense-streaming-rgb-depth-data%23_Toc467619... Intel® Aero Compute Board and Intel® RealSense™ Technology for Wi-Fi* Streaming of RGB and Depth Data | Intel® Software

I do not see any librealsense examples in /usr/share/... on my compute board. I do not have a camera attached to the board but I do not think that should matter.

The wiki says:

Intel Aero SW comes with librealsense sample examples located at /usr/share/librealsense/examples/

These examples can be invoked as follows: /usr/share/librealsense/examples/cpp-capture

Does the camera need to be attached to the board to see the examples ? Or are the examples only on the RTF drone ? Also, is GStreamer installed on the board or RTF drone ?

I have connected my laptop to the Aero Wifi Access point. I then ssh into it and ran the gst-launch command but got the following error:

think@penguin:~/t25$ ssh root@192.168.8.1

Last login: Thu May 18 15:38:36 2017 from 192.168.8.2

root@intel-aero:~# gst-launch-1.0 v4l2src device=/dev/video13 do-timestamp=true ! video/x-raw, format=YUY2, width=640, height=480, framerate=15/1 ! autovideoconvert ! vaapih264enc ! rtph264pay ! udpsink host=192.168.8.2 port=5600

Setting pipeline to PAUSED ...

error: XDG_RUNTIME_DIR not set in the environment.

libva info: VA-API version 0.39.0

libva info: va_getDriverName() returns 0

libva info: Trying to open /usr/lib/dri/i965_drv_video.so

libva info: Found init function __vaDriverInit_0_39

libva info: va_openDriver() returns 0

ERROR: Pipeline doesn't want to pause.

Got context from element 'vaapiencodeh264-0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL;

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Cannot identify device '/dev/video13'.

Additional debug info:

../../../gst-plugins-good-1.8.2/sys/v4l2/v4l2_calls.c(606): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:

system error: No such file or directory

Setting pipeline to NULL ...

Freeing pipeline ...

root@intel-aero:~#

Let me know what I am doing wrong. I am following the link you provided - https://github.com/intel-aero/meta-intel-aero/wiki/WiFi-Streaming WiFi Streaming · intel-aero/meta-intel-aero Wiki · GitHub

I have QGC running on my ubuntu laptop and from what I read, it uses GStreamer as well. I also have GStreamer installed on my ubuntu terminal. I did that using the following commands:

sudo apt-get update

sudo apt-get install ubuntu-restricted-extras

sudo apt-get install gstreamer1.0-libav

sudo apt-get install libgstreamer-plugins-base1.0-dev

Do I need to run any commands to install GStreamer on the Aero board ? Note: I do not have the visionary kit attached to the board at the moment (but I do not think it should matter or maybe it does).

I noticed this in the article https://software.intel.com/en-us/articles/intel-aero-realsense-streaming-rgb-depth-data# _Toc467619801 Intel® Aero Compute Board and Intel® RealSense™ Technology for Wi-Fi* Streaming of RGB and Depth Data | Intel® Software

" This article shows you how to send a video stream including RGB and Depth data from the Intel® Aero Compute Board that has an Intel® RealSense™ R200 camera attached to it. " --> Does the camera (I do not have the visionary kit at the moment) need to be attached to get any of this working ? I would imagine NOT, I wont see any video thats all on QGC, correct ?

Thanks

Rohan

idata
Community Manager
65 Views

Hello rohanK23,

 

 

The available documentation is under constant change, and some thinks need to be updated, one of those things is the location of the librealsense examples. If you are using the latetest OS version, the examples are located in /usr/bin/, so the path to run the "capture" example is /usr/bin/cpp-capture.

 

 

You don't need to attach the camera to see the files associated with the examples, but obviously you need a camera to check them out.

 

 

Regarding your last question, both the librealsense and the GStreamer libraries are installed in the Compute Board.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

Thanks Andres.

Can you also reply to the following comment:

I have connected my laptop to the Aero Wifi Access point. I then ssh into it and ran the gst-launch command but got the following error:

think@penguin:~/t25$ ssh root@192.168.8.1

Last login: Thu May 18 15:38:36 2017 from 192.168.8.2

root@intel-aero:~# gst-launch-1.0 v4l2src device=/dev/video13 do-timestamp=true ! video/x-raw, format=YUY2, width=640, height=480, framerate=15/1 ! autovideoconvert ! vaapih264enc ! rtph264pay ! udpsink host=192.168.8.2 port=5600

Setting pipeline to PAUSED ...

error: XDG_RUNTIME_DIR not set in the environment.

libva info: VA-API version 0.39.0

libva info: va_getDriverName() returns 0

libva info: Trying to open /usr/lib/dri/i965_drv_video.so

libva info: Found init function __vaDriverInit_0_39

libva info: va_openDriver() returns 0

ERROR: Pipeline doesn't want to pause.

Got context from element 'vaapiencodeh264-0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL;

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Cannot identify device '/dev/video13'.

Additional debug info:

../../../gst-plugins-good-1.8.2/sys/v4l2/v4l2_calls.c(606): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:

system error: No such file or directory

Setting pipeline to NULL ...

Freeing pipeline ...

root@intel-aero:~#

Let me know what I am doing wrong. I am following the link you provided - https://github.com/intel-aero/meta-intel-aero/wiki/WiFi-Streaming WiFi Streaming · intel-aero/meta-intel-aero Wiki · GitHub

I have QGC running on my ubuntu laptop and from what I read, it uses GStreamer as well. I also have GStreamer installed on my ubuntu terminal. I did that using the following commands:

sudo apt-get update

sudo apt-get install ubuntu-restricted-extras

sudo apt-get install gstreamer1.0-libav

sudo apt-get install libgstreamer-plugins-base1.0-dev

Do I need to run any commands to install GStreamer on the Aero board ? Note: I do not have the visionary kit attached to the board at the moment (but I do not think it should matter or maybe it does).

I noticed this in the article https://software.intel.com/en-us/articles/intel-aero-realsense-streaming-rgb-depth-data%23_Toc467619... Intel® Aero Compute Board and Intel® RealSense™ Technology for Wi-Fi* Streaming of RGB and Depth Data | Intel® Software

" This article shows you how to send a video stream including RGB and Depth data from the Intel® Aero Compute Board that has an Intel® RealSense™ R200 camera attached to it. " --> Does the camera (I do not have the visionary kit at the moment) need to be attached to get any of this working ? I would imagine NOT, I wont see any video thats all on QGC, correct ?

Thanks

Rohan

idata
Community Manager
65 Views

Hello rohanK23,

 

 

I ran the commands with the Vision Accessory Kit connected and got the expected video output. Then I ran the same commands without the Vision Accessory Kit and got the same error message that you showed in your post.

 

 

Do I need to run any commands to install GStreamer on the Aero board?

 

No, you don't, the necessary GStreamer files are already installed in the board. I ran the commands after flashing the board.

 

 

You need the Vision Accessory Kit to run the code in beside that, you'll need a flight controller compatible with QGC.

 

 

Please let me know if you find this information useful.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

Hi Andres,

Thanks for your reply.

>> beside that, you'll need a flight controller compatible with QGC. --> I have downloaded QGC on my ubuntu laptop and will be connecting the board with the laptop and using QGC on my laptop to view the video. I do not have the visionary kit at the moment but I do have a RTF drone which I will use instead of the board soon.

Can you please expand on this from the Wiki ?

I have run sudo apt-get install libusb-1.0-0-dev pkg-config and sudo apt-get install libglfw3-dev. Sometimes I get an okay but today the below failed on a different machine. I copied the realsense libraries over from my old machine but I would like to understand how the below works. When you say "Navigate to librealsense root directory" , what do you mean ? I am on my ubuntu machine at this point.

Library Build Process

 

librealsense employs CMake as a cross-platform build and project management system. Navigate to librealsense root directory and runmkdir build && cd build

 

The default build is set to produce the core shared object and unit-tests binariescmake ../

Thanks

Rohan

idata
Community Manager
65 Views

Hello rohanK23,

 

 

The librealsense root directory is the main folder that houses all of the librealsense files, perhaps this article may help you get a better understanding of the subject: https://www.lifewire.com/what-is-a-root-folder-or-root-directory-2625989.

 

 

If you have any other update or question, don't hesitate to contact us.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

HI Andres,

I know what a root directory is.

I meant after running the following 2 commands on my ubuntu laptop :

Install the packages required for librealsense build:

libusb-1.0 and pkg-config

sudo apt-get install libusb-1.0-0-dev pkg-config

sudo apt-get install libglfw3-dev

where are the librealsense files installed so I can go there and run the rest of the commands to complete the installation. ?

Thanks

Rohan

SJha3
Beginner
65 Views

I also have the RTF drone and need data from the cameras on ubuntu 16.04. The librealsense and the ubuntu xenial patch installation were successful, but I see no /dev/video files. It says in the documentation that the data transmission is done using USB 3.0, so why is it not getting detected when I have all the required installations complete?

RKura
New Contributor I
65 Views

Hi Siddhaarth,

I am not sure what you meant by "data transmission is done using USB 3.0". Can you please expand on that or paste the link to the Wiki that says that ? USB 3.0 is used to connect a mouse or keyboard to the drone. I usually just ssh to the drone from my ubuntu (after connecting to the Wifi access point provided by the drone). When I do that , I can see the camera devices under /dev/video*

root@intel-aero:~# ls /dev/video*

/dev/video0 /dev/video11 /dev/video2 /dev/video5 /dev/video8

/dev/video1 /dev/video12 /dev/video3 /dev/video6 /dev/video9

/dev/video10 /dev/video13 /dev/video4 /dev/video7

Have you made sure that the drone was flashed and updated with the latest OS and BIOS ?

best

Rohan

SJha3
Beginner
65 Views

Hello Rohan,

What I meant by the USB 3.0 connection was from the datasheet https://software.intel.com/sites/default/files/managed/d7/a9/realsense-camera-r200-product-datasheet... https://software.intel.com/sites/default/files/managed/d7/a9/realsense-camera-r200-product-datasheet..., if you look at the last paragraph on Page 17.

I also see these devices when I boot Yocto on the drone. But I also have installed Ubuntu on the drone, installed librealsense and the required patch and even then I can't see any devices like those.

I want Ubuntu on the drone because I want catkin and ROS support on the drone, which is quite limited when you look at meta-ros.

Thanks

 

Siddharth
RKura
New Contributor I
65 Views

Hi Sidharth,

I do not have ubuntu installed on the drone. I compile and make on ubuntu laptop and scp the binaries over to the drone which seems to be binary compatible so far. I was thinking about installing ubuntu on the drone but I came across this which I am not sure if its related to your problem or not

"Many developers may prefer prototyping with typical linux distributions like https://www.ubuntu.com/ Ubuntu. Intel is not supporting nor shipping Ubuntu on Intel Aero, but the community may publish recipes on how to install Ubuntu manually. Ubuntu and other Linux distros may be installed on Intel Aero, but some functionality may or may not work, notably the the cameras from the Computer Vision kit. "

best

Rohan

idata
Community Manager
65 Views

Hello Rohan,

 

 

I noticed in this thread https://communities.intel.com/thread/114987 https://communities.intel.com/thread/114987, that Jose gave you the corresponding answer regarding the location of the directory.

 

 

If you have any other update or question, don't hesitate to contact us.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
idata
Community Manager
65 Views

Hello Siddharth,

 

 

Thank you for your interest in the Intel® Aero Ready to Fly Drone.

 

 

As Rohan kindly commented, please keep in mind the following statement from the official documentation while installing Ubuntu in your RTF drone:

 

 

Many developers may prefer prototyping with typical linux distributions like Ubuntu. Intel is not supporting nor shipping Ubuntu on Intel Aero, but the community may publish recipes on how to install Ubuntu manually. Ubuntu and other Linux distros may be installed on Intel Aero, but some functionality may or may not work, notably the cameras from the Computer Vision kit.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

Hi Siddharth,

Can you please share your steps to install librealsense on ubuntu ? I had some trouble doing that on my laptop. I ended up just copying the so's from the drone over. I would appreciate if you can share your steps. I folowed the Wiki but no luck.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md librealsense/installation.md at master · IntelRealSense/librealsense · GitHub

Thanks

Rohan

idata
Community Manager
65 Views

Hello Rohan,

 

 

I suggest you to continue posting your questions regarding the installation of librealsense on Ubuntu on the thread that you created for said purpose: https://communities.intel.com/thread/114987 https://communities.intel.com/thread/114987.

 

 

Have a nice day.

 

 

Regards,

 

Andres V.
RKura
New Contributor I
65 Views

Hi Andres,

I did post a question there today. Thanks.

best

Rohan

Reply