Community
cancel
Showing results for 
Search instead for 
Did you mean: 
DMarc8
Beginner
1,623 Views

ISSUE WITH OPTICAL FLOW

Hello,

I'm able to get images from the bottom facing camera and publish on a ROS topic following the suggestions in . However my images have an issue on the right and bottom edges.

Is this issue caused by the hardware or software and is there any way to solve it?

Can I apply the optical flow algorithm correctly although I have the issue on the edges?

Thank you in advance,

Daniele

0 Kudos
14 Replies
idata
Community Manager
115 Views

Hello daniele.marchetti,

 

 

Thank you for your interest in the Intel Aero Ready to Fly drone.

 

 

Could you please let us know if you have already done the calibration mentioned by Coline on the thread you have shared?

 

Calibration Tool: http://wiki.ros.org/camera_calibration

 

Instructions: http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration

 

rosrun image_view image_view image:=/usb_cam/image_raw

 

 

Also, could you let us know what sensor do you have? LeddarOne, Lidar Lite?

 

 

Thank you in advance,

 

Eliza
DMarc8
Beginner
115 Views

Hi Eliza,

thank you for your support. Yes, I have done the calibration as explained in that thread. As a result, the images are slightly rectified but the issue on the edges is still.

In addition, I have LeddarOne installed on the drone. I would like to know if the optical flow algorithm can be correctly applied although the images from the camera have the issue on the edge.

Thank you,

Daniele

idata
Community Manager
115 Views

Hello daniele.marchetti,

 

 

Thank you for the information!

 

 

Could you please provide us the information on how are you capturing the frames?

 

Also, could you please check the video stream for the bottom facing camera and see if this is percent. https://github.com/intel-aero/meta-intel-aero/wiki/06-Cameras-and-Video# rtsp-streaming-with-the-camera-streaming-daemon https://github.com/intel-aero/meta-intel-aero/wiki/06-Cameras-and-Video# rtsp-streaming-with-the-camera-streaming-daemon

 

 

Can you please also confirm that the camera is firmly attached to the drone?

 

 

Thank you in advance,

 

Eliza
DMarc8
Beginner
115 Views

Hi,

I use the usb_cam node to acquire frames and the node image_view to show the frames. The result has an issue on the right and lower edges as shown in the previous attachment.

The video stream from the bottom camera does not work. Only the video stream from the Intel Real Sense works.

Finally, I confirm that the bottom camera is firmly attached to the drone.

PS:

I have Ubuntu 16.04 installed on the compute board and on the development ground station.

Thank you,

Daniele

DMarc8
Beginner
115 Views

Hi,

I tried to apply the optical flow algorithm.ta I checked from QGroundControl stion that the quality is between 50 and 200 depending on the floor. Moving the drone by hand with motor disarmed the optical flow seems to work. For a better check I should compare the velocity measured by the optical flow with an external sensor measurement. However, at the moment an issue has come out: using QGroundControl-widgets-analyze I checked that the drone velocity from the optical flow algorithm (M1:C100:OPTICAL_FLOW_RAD.integrated_x and M1:C100:OPTICAL_FLOW_RAD.integrated_x) is expressed in a frame 180 deg rotated respect to the vehicle frame.

Am I the only one that exhibit this issue?

Moreover, if I set EKF_AID_MASK = 2, the local_position_x and y are constant zero also if I move the drone by hand, thus I cannot switch to position flight mode.

Thank you,

Daniele

idata
Community Manager
115 Views

Hi Daniele,

 

 

Which PX4 firmware are you using? Setting EKF2_AID_MASK to 2 should allow you to use position control. What is the output of systemctl status aero-optical-flow?

 

 

Regards,

 

Jesus
DMarc8
Beginner
115 Views

Hi,

This is the output of systemctl status aero-optical-flow

● aero-optical-flow.service - Aero optical flow

Loaded: loaded (/lib/systemd/system/aero-optical-flow.service; disabled; vendor preset: enabled)

Active: active (running) since lun 2018-09-03 10:22:36 CEST; 13min ago

Main PID: 2254 (aero-optical-fl)

CGroup: /system.slice/aero-optical-flow.service

└─2254 /usr/bin/aero-optical-flow

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

set 03 10:34:39 aerocomputeboard-CherryTrail aero-optical-flow[2254]: Number of good matches: 1, desired: 20

Optical flow is active as I can see in QGroundControl-Widgets-analyze and the quality is above 100 but LOCAL_POSITION_NED_ doesn't exhibit any output. In analyze tool I see three fields of optical flow:

M1:C1:OPTICAL_FLOW_RAD.integrated_x is positive when I move the drone backward

M1:C1:OPTICAL_FLOW_RAD.integrated_y is positive when I move the drone the right

M1:C100:OPTICAL_FLOW_RAD.integrated_x, and M1:C100:OPTICAL_FLOW_RAD.integrated_y work similarly.

M1:OPTICAL_FLOW_RAD_integrated_x and M1:OPTICAL_FLOW_RAD_integrated_y don't work

Thank you in advance,

Daniele

idata
Community Manager
115 Views

Hi Daniele,

 

 

Could you try with the px4 firmware attached. Please note that instead of setting SENSE_EN_LEDDAR1 to 1, you need to set AEROFC_TELE_BPS to 0. Make sure to do a frame reset and calibrate all the components. Also set EKF2_AID_MASK to 2 and EKF2_HGT_MODE = 2 (Range Sensor).

 

 

Regards,

 

Jesus
DMarc8
Beginner
115 Views

Hi Jesus,

thank you for your support. I uploaded the drone with your px4 firmware attached. When I connect to the drone from QGroundControl, it shows the following message:

You are currently connected to a vehicle but it did not return the full parameter list. As a result, the full set of vehicle setup option are not avaliable.

Thus I cannot calibrate my vehicle and reset the airframe.

However I can set AEROFC_TELE_BPS = 0, EKF2_AID_MASK = 2 and EKF2_HGT_MODE = 2. With this setup, the optical flow works and I get consistent values from LOCAL_POSITION_X and Y, so it could be the correct way.

Thank you,

Daniele

GWest4
Beginner
115 Views

Hi,

With interest I have read this post, as we are experiencing the same problem.

We are trying to get optical flow working, using the down facing camera and LeddarOne. So far no result, because the LOCAL_POSITION_NED.x and .y stay 0.

However after flashing aerofc-v1_default_leddar_one.px4 it's working. The only thing is that fusion of the VISION_POSITION_ESTIMATE has stopped working in this build, which was working in release AeroFC firmware version 1.8.0.

Is there any plan on commiting the change/fix in this image to the sources in github, so we can build it ourself and run with optical flow?

Thanks and regards,

Gertjan

Florinco BV

GWest4
Beginner
115 Views

Since our project depends on optical flow in combination with vision pose estimate (coming from our own indoor GPS), we are very curious about any updates on the issue.

DMarc8
Beginner
115 Views

Finally I decided to buy PX4flow. I will implement optical flow with it because it is the recommended method.

idata
Community Manager
115 Views

The source code to the flight controller firmware I provided (aerofc-v1_default_leddar_one.px4) can be found here:

 

https://github.com/zehortigoza/PX4Firmware/tree/stable-leddar_one

 

 

I would recommend opening a ticket on the PX4 Github Issues section to investigate the issue with the latest PX4 release.

 

 

Regards,

 

Jesus
DNich1
Beginner
115 Views

Hi Jesus,

I am curious, what is changed in zehortigoza's leddarone firmware - should I still be using it? I noticed it shows version 1.5.0 . .

Reason I ask is that distance_sensor/leddar_one is included in the latest px4 release (1.8) without any modification. We are using this firmware and are able to get DISTANCE_SENSOR.current_distance and LOCAL_POSITION_NED.z estimate from leddarone.

However, altitude hold doesn't work, the drone lands slowly when enabled.

Reply