Data Clock Race conditions

[Altera_Forum]

Hello everyone, 

 I am trying to connect two Altera Stratix II boards together through a custom 2-bit serial connection. Right now I send the clock (along with the data) from board 1 to board 2 so that the data can be sampled at the same clock remotely. Then I perform a clock domain translation on board 2 using a two-flop synchronizer.  The problem is that the data bits sent out from board 1 (as seen on the Logic Analyzer) are different from the data bits received on board 2 (as seen on LA). I am using the same clock to run both LAs so I assume the data bits should appear to be the same.   Since they are not, so I thought there must a race condition occurring between the clock and the data so I started sampling on the neg edge of the clock on board2. It has improved, but still sometimes the data bits are skipped or received twice. Entire system operates on a standard 50 MHz clock... I have tried many different things but its still not stable. Any suggestions/advice would be highly appreciated.  Btw, in the info after compilation the following msg is shown where sys_clk is the board 1 clock and sndr_clk_UR_LR_A is the signal that carries the clock to the remote board:  "Info: Longest tpd from source pin "sys_clk" to destination pin "sndr_clk_UR_LR_A" is 11.467 ns  Info: 1: + IC(0.000 ns) + CELL(1.063 ns) = 1.063 ns; Loc. = PIN_AF15; Fanout = 2; CLK Node = 'sys_clk'  Info: 2: + IC(8.021 ns) + CELL(2.383 ns) = 11.467 ns; Loc. = PIN_C21; Fanout = 0; PIN Node = 'sndr_clk_UR_LR_A'  Info: Total cell delay = 3.446 ns ( 30.05 % )  Info: Total interconnect delay = 8.021 ns ( 69.95 % )"  Thanks in advance !!!

[Altera_Forum]

Minor correction: 

 Data bits are sometimes received twice when sent from Board 1 to Board 2, and are skipped some times on the reverse path.  Please ask if I missed any related detail. Thanks !

[Altera_Forum]

I will first make sure my serialiser/deserialiser functionality is correct. 

If so then it will point to a timing problem. Make sure you are using fast i/o output registers and possibly a dedicated clk output pin. A practical test is to send your clk from a pll and rotate the phase to optimise the timing window.

[Altera_Forum]

I am not sure why you use two stage synchroniser since your case is that of source synchronous transfer.

[Altera_Forum]

Thanks for the reply ! 

 I have verified the functionality many times and if used on the same board using the same clock, there are no issues !  "Make sure you are using fast i/o output registers and possibly a dedicated clk output pin. A practical test is to send your clk from a pll and rotate the phase to optimise the timing window."  That I am not doing ! How do I use the fast I/O registers ??

[Altera_Forum]

The network right now is mesochronous but the intention is for the design to be portable to completely asynchronous networks with minor changes. Thus once the serial data is converted to a full packet, it is passed through a two-flop synchronizer to translate the clock domain without entering in to any of the metastability issues. That part seems to be working ok, the problem is that the bits do not appear to be the same as the ones sent !!

[Altera_Forum]

Quartus doesn't use io registers by default. 

to use i/o registers go to assignment editor, select your nodes then enable fast output registers on your data and clk and and enable fast output_enable register if you are using this. For the other board enable the fast input register for data.    For PLL, instantiate one, and either manually change phase per build or you can go for the dynamic phase feature...   Make sure you are testing your data before any clk domain crossings in the Rx board.

[Altera_Forum]

One more thing, 

add Tco constraint to your outputs(data and clk) otherwise the fitter will swing around from build to build.

[Altera_Forum]

I am using Quartus 7.1 - After I select the nodes in the assignment editor I cant seem to find the option that is used to set the fast I/O registers -- how does one do that? 

 Also can you please be a bit more specific in terms on where to find the option for setting the Tco constraint ! Thanks !

[Altera_Forum]

Xinjia, 

 How long is that cable between the boards? If it is not properly terminated, you may see bits and clocks bouncing back and forth between rx an tx. The longer the cable, the more corruption that causes.  If you have a fast oscilloscope, see the analog waveforms near the rx.

[Altera_Forum]

Once you select a node you get many options in the form of spreadsheet style. It will be named "fast input register" and the next cell choose "enable" 

or it could be "fast output register" and so on... Tco is also one of the options. remember Tco and PLL rotation can be conflicting to the fitter. You will need to see what actual figures are finally accomplished by the fitter.   your signals must be registered for this option.   Alternative to assignment editor, use commands(there are several example around, can't remeber from top of my head now).

[Altera_Forum]

Found them, thanks for the details Kaz ! 

 For Progemies: The cable is around 6-7 inches, would be hard to reduce its length any further, though the max try would reduce it to 5 inches I think !

[Altera_Forum]

--- Quote Start ---   For Progemies: The cable is around 6-7 inches, would be hard to reduce its length any further, though the max try would reduce it to 5 inches I think !  --- Quote End ---    Xinjia, Do you have a chance to slow down the data clock? If you get errors even with 1MHz clock, then it's a race condition. Bus if the comm is fine at 1MHz and you'll start to get errors when dialling up the clock frequency closer to 50MHz, it must be a signal integrity issue.  The cable should have a good ground return path and the clock and data should be separated with a ground to reduce cross talk. If possible, try two coaxial cables for the signals.

[Altera_Forum]

Thanks Progemies ! 

Kaz's ideas helped a lot and the communication from brd 1 to brd 2 has stablized, but the reverse path is still faulty at times with data being skipped in sampling (bits are skipped but are not corrupted). Thanks for the clock idea, will check it out !

[Altera_Forum]

Thanks a lot for the help guys ! 

 The problem has finally been resolved. I tried all the optimization measures but still the tpd was higher than what was required for the smooth system operation - now I am operating the communication modules at half the originate clock rate and it runs A ok :)   Cheers :cool:

[Altera_Forum]

--- Quote Start ---  Minor correction:   Data bits are sometimes received twice when sent from Board 1 to Board 2, and are skipped some times on the reverse path.   Please ask if I missed any related detail. Thanks !  --- Quote End ---     Hi,   Maybe your problem has been solved. If so, ignore this. From the description above it sounds very much like you are trying to send data from one system to another with nominally the same clock frequency, but physically different oscillators. One oscillator is slightly faster than the other, which means that in one direction you occasionally duplicate a bit (if sent from the slower clok to the faster clock) and in the other direction you occasionally loose a bit (if sent from the faster clock to the slower clock). If your whole system runs from one physical clock source, then this should not not happen and the problem has to lie elsewhere.   If your intention is to have the two systems run from completely asynchronous clocks, then the safest way to cross the clock domains is with a dual clock FIFO. The simple two FF synchroniser is good for passing a single control signal, but when passing multiple signals (like data bytes), the FIFO is the safest.   Regards, Niki

[Altera_Forum]

Hi Niki,  

Thanks for the reply.    --- Quote Start ---  Hi, If your whole system runs from one physical clock source, then this should not not happen and the problem has to lie elsewhere.   --- Quote End ---    The transfer is being done source synchronous (i.e. the clock travels with the data so it is the same clock, but somewhat delayed of course. I am able to run the brd 2 brd comm system correctly at 48MHz, but not at 50MHz. Would have loved it if it had worked at 50 though.    --- Quote Start ---  Hi, If your intention is to have the two systems run from completely asynchronous clocks, then the safest way to cross the clock domains is with a dual clock FIFO. The simple two FF synchroniser is good for passing a single control signal, but when passing multiple signals (like data bytes), the FIFO is the safest.   --- Quote End ---    I see your point there. A bi-sync FIFO is definitely the most suitable way. But the simple two FF synch is also good enough to do the job. The single control signal that is passed using the 2 FF synch, is used as a flag by the rx side to know exactly when there is stable data on the data lines. Delay of one cycle placed in by the 2 FFs ensures that we never enter metastability.