Problem with Data Integrity & Clock between 2 FPGAs

[Altera_Forum]

Hi 

I have a problem with data integriry & clock between 2 FPGAs. A scheme of my design is inserted at the end. My design consists of 2 Cyclone III FPGAs. Each FPGA has several clock domains, each feeding their own logic. In FPGA-2 there is a PLL which generates two 50 MHz clocks with 180-degree phase shift; i.e. clk2 = NOT clk1. The clk1 feeds FPGA-2 logic. The clk2 signal goes out from FPGA-2 using a normal IO pin and feeds FPGA-1 logic through a dedicated clock pin. FPGA-1 generates a 16-bit data using clk2 and sends it to FPGA-2.  Both FPGAs use Fast Input/Output Register on this data bus pins in order to minimize t_co & t_su. FPGA-2 processes that data using clk1.  I used two clocks with 180 degree phase shift and I hoped that half of clock cycle is sufficient for all delays, including t_co, t_su, PCB delay, etc.  Now, using SignalTap it is evident that FPGA-1 has generated data correctly, but in FPGA-2 received data is corrupted. What could cause this problem?  

• sending out a clock from a normal IO with Fast Output Register option enabled? Is this a good design practice? 
• Not setting IO delay constraints? Keep in mind that 50 MHz is not so high and PCB tracks are short, though with unequal lengths (less that 0.5 inch) 
• Something else ??!!! 

  I'm using Quartus-II v.11 My design scheme: http://www.alteraforum.com/forum/attachment.php?attachmentid=9651&stc=1   Thanks in advance.

[Altera_Forum]

--- Quote Start ---   I hoped that half of clock cycle is sufficient for all delays   --- Quote End ---   Why hope when you can "design"?  Did you create a TimeQuest .SDC file? If not, its time to learn.  The synthesis tool will make no attempt to meet external timing if you do not provide it the information it needs to understand what the external timing requirements are.  Cheers, Dave

[Altera_Forum]

Thanks Dave 

Well, actually I have the .SDC constraint file, but I constrained only clocks & clock uncertainties, not input & output delay constraints (using set_input_delay & set_output_delay). Since I used Fast Input/Output Registers in my design, The t_co & t_su should be minimal (how much minimal? I don't know how to utilize TimeQuest for this parameter, but according to an_366 I don't think it will be more than 1 ns). PCB delays should be about 166*0.5=83 ps at most. Summing up all delays I get at most 2 ns delay. Comparing with 10 ns (half of 50 MHz clock period) it is negligible. So I think input & output delay is not required. Am I right? Did I illustrate my point?

[Altera_Forum]

--- Quote Start ---   Did I illustrate my point?   --- Quote End ---   Yes. However, you should never "assume" anything. The tool *can* analyze the interface timing, so you *should*.  My recommendation is to write an SDC file that includes the FPGA I/O timing requirements. Here's an outline for you ...  # Timing parameters set data_tsu 1.0 set data_th 1.0 set data_tco_min 0.0 set data_tco_max 6.0 # Timing constraints set data_input_delay_min $data_tco_min set data_input_delay_max $data_tco_max set data_output_delay_min -$data_th set data_output_delay_max $data_tsu # Bidirectional buses set data_io }] # Inputs set data_i }] # Outputs set data_o }] # Bidirectional buses set_output_delay -clock vclk_50MHz -min $data_output_delay_min $data_io set_output_delay -clock vclk_50MHz -max $data_output_delay_max $data_io set_input_delay -clock vclk_50MHz -min $data_input_delay_min $data_io set_input_delay -clock vclk_50MHz -max $data_input_delay_max $data_io # Inputs set_input_delay -clock vclk_50MHz -min $data_input_delay_min $data_i set_input_delay -clock vclk_50MHz -max $data_input_delay_max $data_i # Outputs set_output_delay -clock vclk_50MHz -min $data_output_delay_min $data_o set_output_delay -clock vclk_50MHz -max $data_output_delay_max $data_o  You'll need to edit this to define data_i, data_o, and data_io correctly. You'll also need to define your clocks correctly, eg., the output clock you generated. Take a look at this DE0-nano SDRAM example  http://www.alteraforum.com/forum/showthread.php?t=45927  There's an SDC file in that project that defines the SDRAM clock, and SDRAM I/O timing parameters. Since an FPGA data sheet does not really have a fixed set of timing parameters, use the ones I've given you above to start with, and then iterate as needed.  The other thing you can do is to look at the signals with a scope.  NOTE: You will need two different SDC files, one for each FPGA project.  Cheers, Dave

[Altera_Forum]

Thanks a lot. 

I will do so and get back (hopefully to tell my problem is resolved!).

[Altera_Forum]

I applied input_delay & output_delay constraints, thanks to your SDC guidelines. 

In Cyclone III spreadsheets, t_su is negative. I inserted its value as a positive value. t_h is copied from spreadsheet. t_co is calculated according to Method-2 of AN-366.  After synthesis & programming FPGAs, still I have problem with data corruption. What could cause problem? Is sending out clock from a normal IO pin a good design practice? One more question: Isn't it better that I feed FPGA-1 with clk1, instead of clk2? i.e. without 180 degree phase shift? is this phase shift in compliance with our constraints? What about output currents? is there a guideline to set output current? specially for clocks?  Thanks

[Altera_Forum]

--- Quote Start ---  I applied input_delay & output_delay constraints, thanks to your SDC guidelines. In Cyclone III spreadsheets, t_su is negative. I inserted its value as a positive value. t_h is copied from spreadsheet. t_co is calculated according to Method-2 of AN-366.   --- Quote End ---   No, you've misunderstood the procedure. FPGA I/O cells have programmable delay elements, so there is no data sheet values for tsu/th/tco. The data sheets have "micro" parameters uTsu/uTh/uTco. Rather than try to calculate the delays by hand, you enter reasonable constraints, as I have above, and let Quartus/TimeQuest try to meet those constraints. For example, if the tco constraint above was too small, then TimeQuest would fail to meet that requirement. In that case, you would enter a slightly larger value (based on the failed result) and iterate. The above values were determined from a Stratix II FPGA design.  If you have to increase the tco from both Cyclone IIIs to meet the output timing, then you would edit the SDCs for both projects. The TimeQuest analysis of the inputs would then change, reflecting the increase in the delay on the other FPGA output. If you re-synthesize the design, then the programmable input delays will be adjusted, centering the setup/hold margin.     --- Quote Start ---   After synthesis & programming FPGAs, still I have problem with data corruption. What could cause problem?   --- Quote End ---   Poor PCB layout. Did you look at the clock waveform using an oscilloscope?    --- Quote Start ---   Is sending out clock from a normal IO pin a good design practice?   --- Quote End ---   No, in general you should try to use a dedicated PLL output pin. Quartus will warn you that your use of a general I/O will result in excessive jitter. At 50MHz you should be fine though.    --- Quote Start ---   One more question: Isn't it better that I feed FPGA-1 with clk1, instead of clk2? i.e. without 180 degree phase shift? is this phase shift in compliance with our constraints?   --- Quote End ---   TimeQuest analysis will answer this. Try with a common clock, and if timing cannot be met, see if the phase of the clock needs to be adjusted to meet timing. If that is the case, then use a PLL to adjust the phase. The SDRAM design I referred to has an example of this.    --- Quote Start ---   What about output currents? is there a guideline to set output current? specially for clocks?   --- Quote End ---   It depends on your board design. Look at the clock waveform with a scope and try a few different settings to see if the signal integrity gets better or worse.  Cheers, Dave

[Altera_Forum]

Thanks again. 

First, my problem was resolved using IO constraints based on your SDRAM Nano Board template; Thanks. Probably because my clock was not too high, I got answered though my IO constraints were set without an exact understanding of the problem. Nevertheless your guidelines helped me get out of the trouble. However for understanding the problem, another question raised in my mind: You said I should enter reasonable constraints for IO delay constraint. What do you mean by reasonable here? In my example, assume I want to use clk1 (no phase shift) to feed FPGA-1. How should I set my first trial of IO constraint? More specifically, what are reasonable t_co_min, t_co_max, t_su & t_h values in the SDC file? What are the starting point values?

[Altera_Forum]

--- Quote Start ---   However for understanding the problem, another question raised in my mind: You said I should enter reasonable constraints for IO delay constraint. What do you mean by reasonable here? In my example, assume I want to use clk1 (no phase shift) to feed FPGA-1. How should I set my first trial of IO constraint? More specifically, what are reasonable t_co_min, t_co_max, t_su & t_h values in the SDC file? What are the starting point values?  --- Quote End ---    You need to learn how to use TimeQuest. The TimeQuest GUI shows you the timing waveforms for your given set of constraints, including by how much you fail. So you could start off with 0ns constraints and see by how much you fail. However, after you've done that once for an FPGA family, you'll know what "reasonable" means, so from that point on, you use reasonable constraints. Take a look at the figures in this document - they show screen captures from TimeQuest  https://www.ovro.caltech.edu/~dwh/correlator/pdf/timequest_quad_spi_flash.pdf https://www.ovro.caltech.edu/~dwh/correlator/pdf/timequest_quad_spi_flash.zip  Cheers, Dave

[Altera_Forum]

normally, in case fpga outputs to a an ASIC (such as DAC) you will find out tSU/tH of DAC device and include board delays to set output delay constraints. 

In your case fpga feeds fpga and so timing responsibility can be shared as both are configurable at io.  thus max output offset of fpga1 + tSU at fpga2 can equal clock period  However in practice you can first set input delays of fpga2 at some convenient values and see report(datasheet) to find out tSU/tH achieved at input pins. Then go to fpga1 and set output delays accordingly taking board delays into account

[Altera_Forum]

If this is your first time with TimeQuest, Altera has some free training to get you started. Here's the introductory one: 

 http://www.altera.com/education/training/courses/odsw1115

[Altera_Forum]

Thanks. I got what I needed.

[Altera_Forum]

When I run timequest and report datasheet, I get negative values for t_h; i.e. Hold Times. Is this reasonable? 

For some synthesis, t_h is positive, when I change settings t_h becomes negative. Does it mean something is wrong?

[Altera_Forum]

--- Quote Start ---  When I run timequest and report datasheet, I get negative values for t_h; i.e. Hold Times. Is this reasonable? For some synthesis, t_h is positive, when I change settings t_h becomes negative. Does it mean something is wrong?  --- Quote End ---    negative slack = failure  You should not just play with settings to get a pass but rather set it to your io correctly. Your design is really too complicated for a beginner to get i right because your clk is opposite data direction and you have inserted a PLL with phase shift. I suggest you first remove the pll and compute your settings then if necessary add pll

[Altera_Forum]

Thanks kaz. 

I did not mean my slack is negative. All my slacks are positive. When I generate reports using TimeQuest, DataSheet Report can also be generated. The first section of Data Sheet report is Set Up Times and the second section is Hold Times. Keep in mind that they are NOT slack, they are real setup times & hold times (If I'm wrong, please tell me).  This is the case even for a simple counter. When I synthesize a simple counter & run TimeQuest, I get negative values for Hold Times. When I set Fast Input Registers & Fast Output Registers options ON, I get negative Hold Time for some ports, & positive for others. This is the case for Setup Times. Please forget about my design, this is a fundamental question. I insert the two reports here for comparison. I have attached a snapshot of my TimeQuest report as well. I would be grateful If you have a look at them & tell me what goes wrong. Synthesis 1:  +---------------------------------------------------------------------------+ ; Setup Times ; +---------------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +---------------+------------+-------+-------+------------+-----------------+ ; i_clk_enable ; clk_in ; 1.993 ; 2.406 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.703 ; 2.096 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.272 ; 1.674 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.703 ; 2.096 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.357 ; 1.747 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.327 ; 1.715 ; Rise ; clk_in ; ; i_enable ; clk_in ; 2.051 ; 2.435 ; Rise ; clk_in ; ; i_load ; clk_in ; 2.657 ; 3.138 ; Rise ; clk_in ; +---------------+------------+-------+-------+------------+-----------------+ +-----------------------------------------------------------------------------+ ; Hold Times ; +---------------+------------+--------+--------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +---------------+------------+--------+--------+------------+-----------------+ ; i_clk_enable ; clk_in ; -1.680 ; -2.067 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.886 ; -1.288 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.886 ; -1.288 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -1.321 ; -1.705 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.992 ; -1.372 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.962 ; -1.340 ; Rise ; clk_in ; ; i_enable ; clk_in ; -1.768 ; -2.140 ; Rise ; clk_in ; ; i_load ; clk_in ; -1.289 ; -1.755 ; Rise ; clk_in ; +---------------+------------+--------+--------+------------+-----------------+ +-----------------------------------------------------------------------+ ; Clock to Output Times ; +-----------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +-----------+------------+-------+-------+------------+-----------------+ ; o_done ; clk_in ; 7.083 ; 7.151 ; Rise ; clk_in ; +-----------+------------+-------+-------+------------+-----------------+ +-----------------------------------------------------------------------+ ; Minimum Clock to Output Times ; +-----------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +-----------+------------+-------+-------+------------+-----------------+ ; o_done ; clk_in ; 5.684 ; 5.668 ; Rise ; clk_in ; +-----------+------------+-------+-------+------------+-----------------+   Synthesis 2:  +---------------------------------------------------------------------------+ ; Setup Times ; +---------------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +---------------+------------+-------+-------+------------+-----------------+ ; i_clk_enable ; clk_in ; 2.126 ; 2.583 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.746 ; 2.141 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.160 ; 1.557 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.585 ; 1.990 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.746 ; 2.141 ; Rise ; clk_in ; ; i_data_in ; clk_in ; 1.735 ; 2.140 ; Rise ; clk_in ; ; i_enable ; clk_in ; 2.065 ; 2.499 ; Rise ; clk_in ; ; i_load ; clk_in ; 2.077 ; 2.526 ; Rise ; clk_in ; +---------------+------------+-------+-------+------------+-----------------+ +-----------------------------------------------------------------------------+ ; Hold Times ; +---------------+------------+--------+--------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +---------------+------------+--------+--------+------------+-----------------+ ; i_clk_enable ; clk_in ; -1.757 ; -2.177 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.781 ; -1.177 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -0.781 ; -1.177 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -1.209 ; -1.604 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -1.361 ; -1.747 ; Rise ; clk_in ; ; i_data_in ; clk_in ; -1.351 ; -1.747 ; Rise ; clk_in ; ; i_enable ; clk_in ; -1.781 ; -2.199 ; Rise ; clk_in ; ; i_load ; clk_in ; -0.948 ; -1.363 ; Rise ; clk_in ; +---------------+------------+--------+--------+------------+-----------------+ +-----------------------------------------------------------------------+ ; Clock to Output Times ; +-----------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +-----------+------------+-------+-------+------------+-----------------+ ; o_done ; clk_in ; 7.100 ; 7.159 ; Rise ; clk_in ; +-----------+------------+-------+-------+------------+-----------------+ +-----------------------------------------------------------------------+ ; Minimum Clock to Output Times ; +-----------+------------+-------+-------+------------+-----------------+ ; Data Port ; Clock Port ; Rise ; Fall ; Clock Edge ; Clock Reference ; +-----------+------------+-------+-------+------------+-----------------+ ; o_done ; clk_in ; 5.683 ; 5.653 ; Rise ; clk_in ; +-----------+------------+-------+-------+------------+-----------------+

[Altera_Forum]

Those tables are not final slack except that you have thought of them as slack. 

They are to do with arrival times (neg for hold, pos for setup relevant to clk edge) and you don't need them now. look at slack report => setup or hold summary or top failing paths. You can select these from TQ GUI directly without need for commands. Normally if timing fails you get red coloured text as a warning and quartus reports timing failure at end of messages.

[Altera_Forum]

Thanks for your prompt response. 

As you advised me in the previous posts, I want to know setup times & hold times of my IO:   --- Quote Start ---  However in practice you can first set input delays of fpga2 at some convenient values and see report(datasheet) to find out tSU/tH achieved at input pins. Then go to fpga1 and set output delays accordingly taking board delays into account  --- Quote End ---   The only report that I found was Datasheet Report/Setup Times & Hold Times. Setup Summary & Hold Summary only show setup & hold times of my clock, not IO. To summarize my problem: i want to know setup time & hold time of my ios. How can I have them? Timequest datasheet report? Or some other way? If TimeQuest Datasheet Report is that, then is there any problem with negative t_su & t_h values? (These parameters are not slack)

[Altera_Forum]

If design has one fpga outputting to a DAC(for example) then you get DAC tSU/tH and enter constraints for FPGA outputs accordingly. 

 in your case you have two FPGAs. Now you have set input delays for FPGA2 on an arbitrary basis and got above tables for arrival times(the worst values to be taken as tSU/tH of FPGA2).  Now go to FPGA1 and set output delays based on FPGA2 figures for tSU/tH i.e. max output delay of tSU and min output delay as -tH but if clk and data were in same direction(source synchronous).  In your case clk is opposite data and there is additionally pll phase shift so you need to work that out

[Altera_Forum]

--- Quote Start ---  you have set input delays for FPGA2 on an arbitrary basis and got above tables for arrival times(the worst values to be taken as tSU/tH of FPGA2).  --- Quote End ---    And What if the tSU/tH become negative? Does it mean that something goes wrong?

[Altera_Forum]

--- Quote Start ---  And What if the tSU/tH become negative? Does it mean that something goes wrong?  --- Quote End ---    it doesn't matter. Ideally and right "silicon" deep level of a register both are positive by definition. However, the tool then gives the figures from pins perspective of clk and data as this is more relevant to the designer. This results in a shift of window(of setup + hold) relative to clk edge due to different delays inserted on clk path/data path. a positive setup means it is still before edge and a negative hold means its before edge as well i.e. the window has shifted forward. and vice versa

[Altera_Forum]

Thanks a lot.