topic Re: D435 can't reach 60FPS with PyRealsense2 in Items with no label

D435 can't reach 60FPS with PyRealsense2

SHuan76 — Wed, 10 Apr 2019 18:09:13 GMT

Sorry for my poor English.

I want to record only RGB video with PyRealsense2 and opencv-python, the config is below:

config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 60)

But when I replayed the video I've recorded, I found that the number of frames is smaller than 60*the_time_I_spent_recording_in_seconds, in other words, the FPS is smaller than 60 (just 30 to 33).

My USB port is 3.2 and I prefer recording video with Python rather than Realsense Viewer.

Thnak you in advance for your apply.

Re: D435 can't reach 60FPS with PyRealsense2

Eliza_D_Intel — Wed, 10 Apr 2019 22:06:46 GMT

Hello SHuan76, Thank you for your interest in the Intel RealSense D435 camera. Could you please record a video with only 30 FPS and let us know what will be the number of frames received? Thank you, Eliza

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Thu, 11 Apr 2019 21:06:56 GMT

Hello, ElizaD_Intel,

I recorded a video with 30 FPS.

It received 1800 frames in 60.38 seconds (29.81FPS), so I think 30FPS works well.

Re: D435 can't reach 60FPS with PyRealsense2

Eliza_D_Intel — Thu, 11 Apr 2019 21:24:25 GMT

Hello SHuan76, Great news! If you want a higher number of FPS, you can use Intel RealSense Viewer to record the bag file. Thank you, Eliza

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Fri, 12 Apr 2019 16:00:33 GMT

Thanks for your reply, but I don't want to use Realsense Viewer cause the size of bag file is too big.

I prefer using Python code to record 60FPS video with Pyrealsense2 and opencv.

Re: D435 can't reach 60FPS with PyRealsense2

Eliza_D_Intel — Fri, 12 Apr 2019 20:19:58 GMT

Hello SHuan76, You can check out this article, where it is explained how to increase the number of FPS recorded with Python and OpenCV. https://www.pyimagesearch.com/2015/12/21/increasing-webcam-fps-with-python-and-opencv/ Thank you, Eliza

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Fri, 12 Apr 2019 22:00:20 GMT

Hello, ElizaD_Intel,

I tried that approach a few days ago, the drawback of this approach is that you can increase the FPS but can't control the FPS to be stable.

According to the datasheet of D435 (https://www.mouser.com/pdfdocs/Intel_D400_Series_Datasheet.pdf), D435 should support 60FPS well on resolution 640*480.

Thank you,

SHuan.

Re: D435 can't reach 60FPS with PyRealsense2

Sahira_Intel — Sat, 20 Apr 2019 03:56:27 GMT

Hi SHuan, I apologize for the delay in responding. Are you just modifying the Python opencv_viewer_example to stream only RBG at 60 fps? When you stream at 6 or 15 fps, are you still getting an incorrect number of frames? I'll reproduce this in the lab and get back to you with my findings. Best Regards, Sahira

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Mon, 22 Apr 2019 20:51:07 GMT

Hello, sahira,

I use Pyrealsense2 to get rgb frames and use opencv-python to write these frames into a video.

30 fps works well, but I fail to get rgb frames in 60 fps.

Re: D435 can't reach 60FPS with PyRealsense2

Sahira_Intel — Fri, 26 Apr 2019 01:40:00 GMT

Hi SHuan, Can you please provide your code? I'd like to run your exact program on my system. What FPS are you getting when using the RealSense Viewer while doing regular streaming without recording and when recording a bag file? Regards, Sahira

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Mon, 29 Apr 2019 09:15:40 GMT

Here's my code.

I run it on Jupyter Notebook. After running the code, you can see the RGB stream, press 'R' will start recording, and the stream will freeze, and press 'Q' will store the video.

**********************

import pyrealsense2 as rs

import numpy as np

import cv2

import datetime as dt

import threading as th

import multiprocessing as mp

import time

import os

class RGBRecordingClass(th.Thread):

def __init__(self, out, queue):

super(RGBRecordingClass, self).__init__()

self.stop_flag = th.Event()

self.stop_flag.set()

self.out = out

self.queue = queue

def run(self):

while self.stop_flag.isSet():

if not self.queue.empty():

self.out.write(self.queue.get())

print('RGB Thread Exitted')

def stop(self):

self.stop_flag.clear()

pipeline = rs.pipeline()

config = rs.config()

config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

pipeline.start(config)

directory_path = '0429'

if not os.path.exists(directory_path):

os.mkdir(directory_path)

rgb_queue = mp.Queue()

frame_count=0

start_record = False

rgb_video_name = directory_path+'/rgb.avi'

fourcc = cv2.VideoWriter_fourcc(*'XVID')

out = cv2.VideoWriter(rgb_video_name, fourcc, 30, (640, 480))

rgb_thd = RGBRecordingClass(out, rgb_queue)

rgb_thd.start()

try:

while True:

# Wait for a coherent pair of frames: depth and color

frames = pipeline.wait_for_frames()

color_frame = frames.get_color_frame()

if not color_frame:

print('not ready')

# Convert images to numpy arrays

color_image = np.asanyarray(color_frame.get_data())

if start_record:

rgb_queue.put(color_image)

frame_count+=1

# Show images

if not start_record:

cv2.imshow('frame', color_image)

if cv2.waitKey(10) & 0xFF == ord('r'):

start_record = True

print('start recording...')

print(str(dt.datetime.now()))

if cv2.waitKey(10) & 0xFF == ord('q'):

break

finally:

# Stop streaming

print('end')

print('file saved')

print(frame_count)

print(str(dt.datetime.now()))

cv2.destroyAllWindows()

out.release()

pipeline.stop()

rgb_thd.stop()

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Mon, 29 Apr 2019 09:33:48 GMT

When I use RealSense Viewer to do streaming, 30 and 60 FPS both work well on 640*480 resolution, whether it's recording a bag file or not.

But the bag file is really too big, so it's excluded from my consideration.

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Mon, 29 Apr 2019 09:39:51 GMT

Supplement for the code above:

frame_count is the number of frames recorded, and it will show the time starting recording and finishing recording.

I calculate FPS by frame_count/(end time-start time)

Re: D435 can't reach 60FPS with PyRealsense2

JesusG_Intel — Tue, 30 Apr 2019 05:40:18 GMT

Hello SirNight, Try setting the RS2_OPTION_AUTO_EXPOSURE_PRIORITY option to 0 to maintain constant FPS. If RS2_OPTION_AUTO_EXPOSURE_PRIORITY is set to 1, then FPS will fluctuate. Let us know if this solves your problem. You can test with a live stream on RealSense Viewer. Regards, Jesus Intel Customer Suppor

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Fri, 03 May 2019 17:25:10 GMT

It's strange that when I enable auto exposure, the fps can reach 60 fps.

When I turn off auto exposure, the fps will drop to 32 fps.

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Fri, 03 May 2019 17:26:15 GMT

Besides, may you teach me how to set auto exposure with Pyrealsense2 (Python code)?

Re: D435 can't reach 60FPS with PyRealsense2

JesusG_Intel — Sat, 04 May 2019 00:53:11 GMT

You want to set AUTO_EXPOSURE_PRIORITY, not AUTO_EXPOSURE. In your current settings, you may want to check your Exposure value. If it's high, the FPS will drop. Here is sample code for setting sensor options in Python: import pyrealsense2 as rs pipeline = rs.pipeline() config = rs.config() profile = pipeline.start(config) # Start streaming sensor_dep = profile.get_device().first_depth_sensor() print "Trying to set auto_exposure_priority" exp = sensor_dep.get_option(rs.option.auto_exposure_priority) print "exposure = %d" % exp print "Setting AUTO_EXPOSURE_PRIORITY to new value" exp = sensor_dep.set_option( rs.option.auto_exposure_priority, 0) exp = sensor_dep.get_option(rs.option.auto_exposure_priority) print "New AUTO_EXPOSURE_PRIORITY = %d" % exp profile = pipeline.stop You can find the full list of options in rs_option.h file. The Python wrapper exposes all these options like this: RS2_OPTION_EXPOSURE -> rs.option.exposure RS2_OPTION_ENABLE_AUTO_EXPOSURE -> rs.option.enable_auto_exposure RS2_OPTION_AUTO_EXPOSURE_PRIORITY -> rs.option.auto_exposure_priority In RealSense Viewer, you will find the control for Auto Exposure Priority under the RGB Controls menu. Regards, Jesus Intel Customer Support

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Tue, 07 May 2019 20:31:11 GMT

Hi Jesus G.,

I've tried both opening and closing auto exposure,

but the fps still stuck at about 32.

Also, whether I set rs.option.auto_exposure_priority to 0 or 1, the fps still can't rise to 60.

I really want to record a video in 60fps.

Re: D435 can't reach 60FPS with PyRealsense2

JesusG_Intel — Wed, 08 May 2019 23:59:28 GMT

Have you re-tested with RealSense Viewer? If you record with Python then replay in RS Viewer, what FPS do you get? You can see the FPS by clicking the "i" icon on the top of the stream window.

Re: D435 can't reach 60FPS with PyRealsense2

SHuan76 — Thu, 09 May 2019 08:57:55 GMT

The file format I record the RGB video is .avi, so it can't be replayed with RS viewer.

The way how I count average FPS is (The number of frames catched)/(Total recording time).

I pasted the code 10 days ago above, replying to sahira.rizvi.