D-435 Framerate way off Saving to Disk using C#

[BEick2]

Requesting 30fps and saving to disk but only receiving about 16fps in my application for COLOR & DEPTH frames.

I also isolated my test to the Sample C# Wrapper code and getting same.

I am using the C# Wrapper Tutorial 2 - Capture, with the only code change as follows:

        //**********MODIFIED CODE***********

        //cfg.EnableStream(Stream.Depth, 640, 480);

        //cfg.EnableStream(Stream.Color, Format.Rgb8);

        cfg.EnableStream(Intel.RealSense.Stream.Color, 640, 360, Format.Rgb8, 30);

        cfg.EnableStream(Intel.RealSense.Stream.Depth, 640, 360, Format.Z16, 30);

        cfg.EnableRecordToFile(@"C:\test\ros.bag");

        //**********END MODIFIED CODE***********

Using rs-converter command:

rs-convert -i ros.bag -v csv/test -p png/test

How can I BEST get support?

• This forum?
• Github issue?
• Paid support ticket?

It is very important that I am able to get 30 frames per second, as close as possible.

Please help, Thanks in advance

[MartyG]

The link below has advice on stabilizing the frame rate, including a method to enforce a constant FPS rate.by disabling "Auto Exposure Priority". I hope that it will be helpful to you.

https://github.com/IntelRealSense/librealsense/issues/1957

Regarding the most suitable places to receive support, this forum is potentially the least useful as it is the old RealSense forum and is no longer replied to by Intel RealSense support staff (though I answer the cases on here that I am able to and refer people to the other forums for those that I cannot help with). The new forum is here:

https://support.intelrealsense.com/hc/en-us/community/topics

For highly technical questions, especially about RealSense programming, the RealSense GitHub is better in my opinion.

You do not have to pay any money to receive RealSense support. :)

[BEick2]

Hi Marty,

• Thanks for the fast and Awesome response!
• i was aware of the new forum and mistakenly used the old one, my bad :(
• Helpful link, thanks! First, I was glad to hear that disabling auto-exposure should help, but...
• We are collecting data in different hospitals, subject to differing lighting conditions so auto-exposure might be necessary. We can drop FPS to 15, which is adequate, as long as it is consistent.

Will I’ll do more tests. Hopefully, auto- exposure under low light conditions doesn’t drop below 15 FPS. I’m apprehensive because my tests so far were under medium light conditions.

Couple of questions:

• I have been able to collect depth data in near complete darkeness, why are those frames dropped to 16 FPS when asking for 30? To match the color frames?
• iIf I have a problem getting 15 FPS color frames consistently in low light, what else can I do? Would it help to reduce resolution further (currently 640 x 360)

Fyi, we are using deep learning with multiple modalities to recognize movement epochs including 1) color (Optical Flow), 2) depth, 3) IMU (accelerometers), against 4) manual video rating (control, classifier)

thanks!

brent

Question

[MartyG]

I'm glad I could be of help!

• You can disable the Auto Exposure Priority control and still have auto-exposure enabled.

• Dorodnic the RealSense SDK Manager has said that "another thing that can affect FPS is auto-exposure of RGB if the scene is dark. You can disable it in the Viewer, just to make sure". In the RealSense Viewer, there are separate auto-exposure controls for depth and RGB. The depth one is in the 'Stereo' section of the side-panel options and the other for color is in the 'RGB' options section.

• If you are going to be operating the camera in hospitals, bear in mind that fluorescent lights such as ceiling strip-lights can create noise on the image, because the gas inside them flickers at a rate hard to see with the human eye. Setting the FPS near to the operating frequency of the particular lights in that location may help. For some lights it is 30 FPS and for others it is 60 FPS (matching 30 / 60 Hz frequency of the lights).

• If you are using machine learning, you may be interested in the Intel Neural Compute Stick 2. It is a stick that you just plug into a USB port on the computing device you are using with the camera, and it can accelerate the computation of machine-learning applications. Intel sell it in their online RealSense store.

https://store.intelrealsense.com/buy-intel-neural-compute-stick-2.html

• Given the variable light conditions that you will encounter, it may be a good idea to apply outdoor drone methodology to your project. In the past week, Intel released a paper about improving depth maps for drones.

https://www.intelrealsense.com/depth-cameras-on-drones/

• Another option may be to consider trying the RealSense SR305 camera model, which uses a different camera technology (Coded Light, instead of the Stereo sensing of the 400 Series cameras). Coded light was also used by the original Microsoft Kinect V1 camera. The SR305's depth resolution is limited to 640 x 480 maximum, though it sounds as though that would be okay for you if you were willing to reduce resolution below that.

The SR305's store listing says "As with other structured and coded light depth cameras, it functions best indoors or in a controlled lighting situation. It can also function well at lower light levels down to 0 Lux, since infrared light is being projected onto the scene from the camera".

https://store.intelrealsense.com/buy-intel-realsense-depth-camera-sr305.html

The SR305's depth sensing range is limited to around 1.5 meters though, unlike the longer range of the 400 Series cameras.

[BEick2]

Thanks Marty, for the various suggestions.

Regarding the Compute Stick, I'm not involved in the Deep Learning, I just need to do good data acquisition and capture COLOR and DEPTH data, preferably at a constant and common framerate.

It seemed odd to me that I could only get 16fps (not 30fps) in a well lit office so I tried various resolutions and lighting conditions.

30fps 640x360 Office lighting -> 17fps

30fps 640x360 Low lighting ->8fps

30fps 424x240 Office lighting -> 32fps

30fps 424x240 Low lighting -> 28fps

15fps 424x240 Office lighting -> 15fps

15fps 424x240 low lighting -> 16fps

I can live with 15fps but I need it constant.

I want to purse the suggestion you made about disabling "Auto Exposure Priority" and still have auto-exposure enabled.

Can you point me to any C# code samples that modify camera settings?

I'm not using the Viewer, my application is based on the C# Wrapper, Code Samle #2 'Capture'

Thanks!

Brent

[BEick2]

Marty. per your suggestion, I'll move this over to the new forum, thanks!

Brent