Transceiver data passthrough without jitter (Cyclone IV)

[Altera_Forum]

Hello, 

I need to transfer data through some number of FPGAs (Cyclone IV, 2.5GHz, 16-bit words at 125MHz) without jitter at 125 MHz in order to have 1 clock domain in the fpgas connected together in line. Is there are any possible ways to use RX recovered clock from the receiver as TX clock for transievers? The compiler does not allow to do it (I cannot connect RX clock to the inclk). I cannot use cross clock domains inside FPGA (FIFOs, etc.) as the overall jitter becomes very big by the end point in he net.

[Altera_Forum]

I think the only way you can do this is if you design your board so that the recovered clock can be routed to an external jitter cleaner, and the reference clock from that jitter cleaner is another REFCLK input. I'm pretty sure I read this idea in the SyncE paper on the SiLabs web  

 https://www.silabs.com/documents/public/application-notes/an420.pdf  I haven't ever had a chance to test that it works though :)  The alternative (and more standard way) is to distribute a common REFCLK and then all the links are coherent (phase shifted, but all effectively phase-locked).  Why do you need them to be synchronous though? If the payload rate is lower than the data rate, then so long as you have some way of extracting the data (timestamps, headers, etc) you do not need coherent clocks.  Cheers, Dave

[Altera_Forum]

Thank you, 

I also have found the link from Altera -  https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/wp/wp-01257-synchronous-ethernet-solutions-with-altera-fpgas-and-jitter-attenuating-plls.pdf This is very helpful for me.  I need to guarantee synchronous conrol (with minimal jitter) of up to 10 devces working together. To do that we transfer 10-12 control signals at 125 MHz and only 2 of them are used for packet transfering (we have no problems here, just the speed of the "data" channels is slower). We use the tranciever 16-bit data exchange and split it to 16 parallel channels. If we have jitter on a signle receiver, than by the end point of the net we will have much bigger jitter. So, we can't guarantee the synchronous conrol of the devices without solving the problem of jitter.  I also thought about Reverse Serial Loopback (Pre-CDR mode). But this is a question - this is a test mode but if we can we use it for other purposes, for example to connect 10 devices configured with this option to send the same control data to the devices with munimal jitter.

[Altera_Forum]

--- Quote Start ---   I need to guarantee synchronous conrol (with minimal jitter) of up to 10 devces working together.    --- Quote End ---   So route a reference clock to all 10 devices. It possible to design systems with thousands of FPGAs synchronized. Here's some slides ...  https://www.ovro.caltech.edu/~dwh/correlator/pdf/hawkins_jpl_2014.pdf  Slide 16; the system on the left has 600+ FPGAs, the one on the right 1000+. They're all synchronous.  It really comes down to how your system needs to be interconnected. These boards use an external reference clock, and then phase-locked loops to keep all the clocks coherent. The ADCs in the system operate at 1GHz sample rate, sampling signals from 500MHz to 1GHz, and have an ENOB close to optimal for the 8-bit ADCs, so the jitter is sufficiently small (sub-picosecond).  The black cables in the photos carry LVDS data at 125MHz (source synchronous, i.e., a clock and data is carried on the cables, FIFOs are used to cross the phase-locked, but phase offset, clock domains). The data traffic over all the cables is equivalent to 250 million phone calls (the data is noise and is cross-correlated in real time, the output rate is much much lower than the input/sampled data rate).    --- Quote Start ---   To do that we transfer 10-12 control signals at 125 MHz and only 2 of them are used for packet transfering (we have no problems here, just the speed of the "data" channels is slower). We use the tranciever 16-bit data exchange and split it to 16 parallel channels. If we have jitter on a signle receiver, than by the end point of the net we will have much bigger jitter. So, we can't guarantee the synchronous conrol of the devices without solving the problem of jitter.   --- Quote End ---   Generally when you have a synchronous system involving transceivers, when you generate data, eg., at an ADC, you generate a packet with some form of header, eg., a timestamp created from say a 1pps and a count of 125MHz reference clock periods. When that packet travels over the network and arrives at some other FPGA along with packets that have traveled a different route, you buffer the data, read the timestamps in the headers, align the packets (eg., wait until timestamps match, or use the timestamp to load a digital delay line or digital filter).  Try and describe what your system is doing a bit more. If you don't want to describe it on the list, just email me (my forum user name is an email address).  Cheers, Dave