D435 VSYNC Pin

[JWare]

Is it true that external synchronization mode is only supported by the D415? We noticed that there is a 1.8V pulse that appears on the VSYNC pin at the camera framerate. Can we rely on this as a synchronization mechanism for external devices for frames captured on the D435?

If the pulse on the VSYNC pin is tied to the image capture in a reliable and repeatable way, when does it occur relative to image capture? For example, is it at the beginning of the frame capture, at the end, or something else?

Does changing the RS2_OPTION_INTER_CAM_SYNC_MODE option affect the behavior of the VSYNC pin in default vs master mode?

Regards,

-Jake

[MartyG]

I don't have much knowledge of the pin waveforms, unfortunately.

The slave camera listens for a sync pulse from the master camera. If it does not detect one after a certain amount of time, it gives up and triggers itself without sync. A slave camera in a multi can setup continues to generate frames even when a sync pulse is not active.

[Wgao6]

Marty,

Could you ask your co-worker at Intel about the waveforms?

Thanks

[MartyG]

I am not an Intel employee. The best place to ask will be at the RealSense GitHub, where the RealSense developers and engineers are. Please visit the link below and click the 'New Issue' button to post your request. Thanks!

https://github.com/IntelRealSense/librealsense/issues Issues · IntelRealSense/librealsense · GitHub

[idata]

This question was answered at https://github.com/IntelRealSense/librealsense/issues/2610:

PIN4 is not implemented at this moment (just leave it).

PIN5 is the Sync trigger for both Master (output) and Slave (input). The Master Cam will generate a pulse (50us, 1.8V) on PIN5 every frame, and drive PIN5 on Slave Cams.

Regards,Jesus G.Intel Customer Support

[idata]

There are two laser diodes in D435 module. Each of them is driven by PWM0 and PWM1 respectively and working alternately. See attached waveform.

PIN4 by default is NOT connected with PWM.Actually by default, the PIN6=PWM0, and PIN7=PWM1. These can be used for users.Assume you're using FW5.10.9 and RS Viewer 2.16.x, PIN5 will output the Sync trigger while:1) Check box "Output Trigger Enabled";2) Or in slide bar of "Inter Cam Sync Mode", move the bar to "1.000" (Master Mode)You will see the same waveform as attached.Regards,Jesus G.Intel Customer Support

[idata]

Second waveform attached.

[Wgao6]

Jesus,

Thanks.

The Intel realsense D400 datasheet shows that pin 6 and 7 are I2C connection.

so actually they are used as PWM0 (pin 6) and PWM1 (pin7)?

[MartyG]

I believe i2c functions are primarily related to the IMU functions on the tracking connector. Intel discontinued the T150 tracking module IMU this connector was designed for, so these pins may currently be redundant.

Intel are currently testing a new D435i model with an IMU (likely the T265 Tracking Module mentioned in the current SDK's downloads page). If Intel decide to proceed with releasing D435i for public sale, I would speculate that the i2c functions may come back into use on the D435i model, but not the standard D435.

[idata]

Hi,

The datasheet needs to be updated to reflect that Laser_PWM0 is tied to Pin 6 and Laser_PWM1 is tied to Pin 7 by default. A new, updated datasheet will be released soon.Regards,Jesus G.Intel Customer Support

[Sahira_Intel]

Hi jakeware,

Were you able to get your questions answered, or did you need further assistance?

Please let us know!

Regards,

Intel Customer Support

[JWare]

srizvi, thanks for following up on this thread. My questions has not yet been answered. For clarity, I will restate the the most important and relevant subset of my questions below.

When you enable the output trigger on the D435, there is a 1.8V pulse that appears at the camera framerate on pin 5 (VSYNC) of the External Sensor Sync Connector pin. Is this pulse on the VSYNC pin tied to image capture in a reliable and repeatable way? If so, when does it occur relative to image capture? For example, is it at the beginning of the frame capture, at the end, or something else?

Thanks again for your help,

-Jake

[idata]

Hi Jake,

Thank you for being patient with us while we researched your question. The output sync pulse can be recognized as the VSYNC pulse of the depth sensor. It outputs 7 lines after the end of exposure. The latest development firmware, v 5.10.3 with support for multi-camera sync is now available for download here! https://downloadcenter.intel.com/download/28076/Latest-Firmware-for-Intel-RealSense-D400-Product-Family?v=tRegards,SahiraIntel Customer Support

[MartyG]

Thanks a lot for the heads-up about the firmware update, Sahira - I'll highlight it in a forum message and update it with the release details when the firmware documentation is updated.

[JWare]

srizvi, thanks again for your feedback and no worries on the delay.

Can you clarify what you mean by "7 lines after the end of exposure"? Are you referring to lines of code or processed lines of the image? Is it a relatively constant amount of time? Do you have an estimate on how much time passes during those "7 lines"?

Also, my understanding was that 5.9.14 already supported multi-camera synchronization for the D415. Does this new version support it on the D435 explicitly? Can you just say a little bit more about how this new firmware version addresses the questions raised in this thread and the D435 specifically?

Thanks again,

-Jake

[idata]

Hi jake,

7 lines of exposure time is equal to about 20 microseconds. The 5.10.3 firmware adds multi camera sync support to the D435 and maintains multi camera sync support for the D415. More information about multi-camera sync is available on the whitepapers here: https://simplecore.intel.com/realsensehub/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper04.pdfRegards,SahiraIntel Customer Support

[JWare]

srizvi, that new firmware is exactly what we needed and seems to have done the trick for us. We tested the new realsense firmware and our external IMU with https://github.com/ethz-asl/kalibr kalibr, and got sub-ms synchronization once we accounted for half the exposure time reported in the metadata. Thanks to you and everyone on the RealSense team for working on that and being responsive to threads like this.

Having done all that, we have one more question. We noticed that there is some latency associated with the realsense outputs. At 30 Hz, it seems to be about a framerate. That number also seems to change with the camera framerate. Can you guys speak to how much latency we should expect relative to the strobe on the VSYNC pin and how it varies with respect to framerate? Is it always about 1 frame capture period?

Many thanks,

-Jake

[idata]

Hi Jake,

There is latency associated with RealSense ASIC output. While the RealSense camera does not have an internal full-frame butter, most latency comes from system level buffering done in the drivers or software. So, the amount of latency you can expect would be different for each system. More information on latency can be found in the multi camera whitepapers on page 8. Regards,Sahira

[idata]

Multi camera whitepaper:

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_Multiple_Camera_WhitePaper.pdf

[MAbde11]

Hi Jake,

I am trying to use D435 in a VIO system with ROS.

Could you please share some details on how you setup the realsense ROS package to work with D435 in external trigger mode. Also, if you made some synchronization between an IMU and the D435, can you mention how is that done?

Thanks.

Mohamed

[idata]

Hi mzahana,

We are asking the ROS team about it and we will get back to you.Regarding your second question, we are planning to do this for a new model of the D435. If it is released, it will be identical to the D435 except for the addition of an IMU (which will be hardware-timestamped and synced), and D435s and D435i will be able to be synched together.https://communities.intel.com/thread/128935 https://communities.intel.com/thread/128935Regards,Alexandra

[MAbde11]

Thanks Alexandra for your reply.

It is unfortunate that the depth HW synched and not color, even for the anticipated future D435 product. This becomes less useful for VIO application.

Mohamed.