D435 can't reach 60FPS with PyRealsense2

[SHuan76]

Sorry for my poor English.

I want to record only RGB video with PyRealsense2 and opencv-python, the config is below:

config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 60)

But when I replayed the video I've recorded, I found that the number of frames is smaller than 60*the_time_I_spent_recording_in_seconds, in other words, the FPS is smaller than 60 (just 30 to 33).

My USB port is 3.2 and I prefer recording video with Python rather than Realsense Viewer.

Thnak you in advance for your apply.

[SHuan76]

Here's my code.

I run it on Jupyter Notebook. After running the code, you can see the RGB stream, press 'R' will start recording, and the stream will freeze, and press 'Q' will store the video.

**********************

import pyrealsense2 as rs

import numpy as np

import cv2

import datetime as dt

import threading as th

import multiprocessing as mp

import time

import os

class RGBRecordingClass(th.Thread):

  def __init__(self, out, queue):

    super(RGBRecordingClass, self).__init__()

    self.stop_flag = th.Event()

    self.stop_flag.set()

    self.out = out

    self.queue = queue

  def run(self):

    while self.stop_flag.isSet():

      if not self.queue.empty():

        self.out.write(self.queue.get())

    print('RGB Thread Exitted')

  def stop(self):

    self.stop_flag.clear()

pipeline = rs.pipeline()

config = rs.config()

config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

pipeline.start(config)

directory_path = '0429'

if not os.path.exists(directory_path):

  os.mkdir(directory_path)

rgb_queue = mp.Queue()

frame_count=0

start_record = False

rgb_video_name = directory_path+'/rgb.avi'

fourcc = cv2.VideoWriter_fourcc(*'XVID')

out = cv2.VideoWriter(rgb_video_name, fourcc, 30, (640, 480))

rgb_thd = RGBRecordingClass(out, rgb_queue)

rgb_thd.start()

try:

  while True:

    # Wait for frames: color

    frames = pipeline.wait_for_frames()

    color_frame = frames.get_color_frame()

    if not color_frame:

      print('not ready')

    # Convert images to numpy arrays

    color_image = np.asanyarray(color_frame.get_data())

    if start_record:

      rgb_queue.put(color_image)

      frame_count+=1

    # Show images

    if not start_record:

      cv2.imshow('frame', color_image)

    if cv2.waitKey(10) & 0xFF == ord('r'):

      start_record = True

      print('start recording...')

      print('START TIME: '+str(dt.datetime.now()))

    if cv2.waitKey(10) & 0xFF == ord('q'):

      break

finally:

  # Stop streaming

  print('end')

  print('file saved')

  print(frame_count)

  print('END TIME: '+str(dt.datetime.now()))

  cv2.destroyAllWindows()

  out.release()

  pipeline.stop()

  rgb_thd.stop()

[SHuan76]

frame_count will save the total number of frames received during recording video.

And I print the time of starting recording and finishing recording, so the average fps can be calculated.

[JesusG_Intel]

Hello Mr. Huang,

I was able to reproduce your problem and play with your code to try different scenarios. I noticed that when I did not display the CV window output and just calculated fps, then I was able to reach 58 - 59 fps. It seems that displaying the stream causes delays in frames rendering that does not allow for the full fps to be counted in the software. Attached is a simple program that demonstrates. The frame count and FPS is printed onto the screen and stops at 500 frames.

[SHuan76]

Hello Jesus G.,

Sorry for my late reply,

I can reach 58-59 fps too.

I will focus on how to record video in 60 fps in the future,

but it seems to be another issue.

If I encounter another problem, I'll create a new post.

Thank you for your kind support.