Configuring 2 Cyclone III's from one EPCS16 Using 2.5V

[Altera_Forum]

Hello, 

 My FPGA configuration signals an error by pulling the nStatus line down at the end of configuration and starting over (and over and over).  1. I am attempting to configure two EP3C40F484C8N's (Cyclone III) from a single EPCS16.  2. Each FPGA has a different pinout and different logic.  3. "Generate compressed bit streams" is selected for both projects in the Device Configuration Settings and "Compression" is selected for both of the project "SOF's" under properties in the "Convert Programming Files" utility. PMC_FPGA.sof is set as the AS configuration master of the two and IO_FPGA.sof is set as the PS configured slave FPGA. The PMC_FPGA.sof is listed above the IO_FPGA.sof under "SOF Data" within the "Convert Programming Files" utility.  4. I believe I have followed the schematic layout for this multichip configuration with all of the pull-ups, pull-downs and buffers and data/dclk series terminations. All EXCEPT the bits described below.  5. The two FPGA use LVDS IO in banks 1&2 and 5&6 and hence VCCIO in these banks is connected to 2.5V.  6. The EPCS16 device is a 3.3V part and decided to use a FET voltage converter (SN74CB3T1G125DBVR) on the signals between the EPCS16 and the Master FPGA and the Buffer Drivers to the Slave FPGA.  7. The signals that through the translator/voltage are DATA_0, DCLK, nCONFIG, nCE.  8. I have not analyzed the full serial data stream during configuration, but all of the signals appear to be behaving in an expected manner. The Data and clock signals are nice and clean.   9. The ASDO signals into the EPCS16 have a high logic level of 2.2V. The EPCS16 has a 1.98V V high in minimum. Seems OK.  10. MSEL lines on the AS device are msel(3..0) = 0011, on the PS device msel(3..0) = 0000.  11. The entire circuit (both FPGA's and EPCS etc.) are layed out in a 2" x 2" space.   Q1. There was a previous reply to a post that warned about signal propagation delay through voltage translators. Do you know what this warning may have been?  Q2. What else can I check?

[Altera_Forum]

--- Quote Start ---   AS configuration master of the two and IO_FPGA.sof is set as the PS configured slave FPGA   --- Quote End ---   So do you have the nCE of the second device driven by the nCEO of the first device?  I performed a timing analysis of this method, and technically it can fail to meeting timing, eg., see the attached.  The SN74CB3T1G125DBVR is a bus-switch, so you have selected the appropriate type of part to minimize the buffer delay.  You should measure the signals with a scope and compare them to the setup/hold time requirements in the attached.  If its possible to separate the nSTATUS signal and CONF_DONE signals for the two FPGAs, you could try configuring them individually. You could also try configuring them with JTAG to check that they do actually configure correctly. JTAG will only work if you configure them in sequence, so that nCEO from the first enables the second.  Cheers, Dave

[Altera_Forum]

MORE INFO FROM POSTER: 

 12. The Config_Done line does not even twitch off of zero Volts.  13. The nStatus line has a rise time (10% to 90%) of approx. 700 ns, and a fall time of about 1.5 ns.  14. The INIT_DONE line will go high 448 ns after the nStatus line goes low, and goes low 115 us later, 60 us after nStatus goes high again.  15. nStatus goes low 800 ns before nCEO of the Master FPGA goes back high.  16. From estimates made by my PCB CAD program, the delay of DCLK from FPGA to EPCS16 is 0.56 ns plus the translator delay. The delay of DATA_0 from the EPCS16 to the FPGA is 0.47 ns plus the translator delay. the translator delay from the datasheet is 0.15 ns max., but could actually be seen to be 1.5 ns. Round trip clock to data back is then about 1.5 + 1.5 + 0.56 + 0.47 = 4.03 ns.

[Altera_Forum]

--- Quote Start ---   12. The Config_Done line does not even twitch off of zero Volts.   --- Quote End ---   If it did, we probably would not be having this conversation :)    --- Quote Start ---   13. The nStatus line has a rise time (10% to 90%) of approx. 700 ns, and a fall time of about 1.5 ns.   --- Quote End ---   To be expected; its driven low, but tri-stated and pulled-up when going high.    --- Quote Start ---   14. The INIT_DONE line will go high 448 ns after the nStatus line goes low, and goes low 115 us later, 60 us after nStatus goes high again.   --- Quote End ---   INIT_DONE is an optional pin, it only has meaning near the end of configuration.   Have a look at this doc, it has some INIT_DONE timing examples  http://www.ovro.caltech.edu/~dwh/carma_board/fpga_configuration.pdf    --- Quote Start ---   15. nStatus goes low 800 ns before nCEO of the Master FPGA goes back high.   --- Quote End ---   Could you clarify; did nCEO transition from high-to-low, implying that the first FPGA configured ok, and its the second that causes the failure?    --- Quote Start ---   16. From estimates made by my PCB CAD program, the delay of DCLK from FPGA to EPCS16 is 0.56 ns plus the translator delay. The delay of DATA_0 from the EPCS16 to the FPGA is 0.47 ns plus the translator delay. the translator delay from the datasheet is 0.15 ns max., but could actually be seen to be 1.5 ns. Round trip clock to data back is then about 1.5 + 1.5 + 0.56 + 0.47 = 4.03 ns.  --- Quote End ---   So do the math - is there any timing margin for the PS device?  Cheers, Dave

[Altera_Forum]

Thank you for your reply. 

 I do have nCE of the second device driven by the nCEO of the first device.  I believe you helped me pick the bus-switch when I was doing the schematics for the board. I see the issue in your diagram, that the PS timing of the second FPGA has little or no margin. I did not get the impact of this the last time I saw this.  I did measure my signals and have included images with this reply. The clock was measured with a nice differential probe, but the data was measured with a standard probe with a long ground lead.  I really have very little margin on the PS as you suspected.   Q3: Do you know off the top of your head whether I can simply invert the clock to the second FPGA to improve the PS Timing? I could use a NAND gate buffer instead of an AND.  I did notice that DCLK does not pause at the point where nCEO goes from inactive to active. DATA  however does not transition for an interval before and after the switch (nCEO going active). I  expect that there is a "start bit" in the DATA and the timing of the clock with respect to nCEO does not matter. What do you think?  Much to my dismay the nStatus and Config_Done signals between the two FPGA run completely on inner signal layers from one BGA via to the other BGA via.  Thanks again

[Altera_Forum]

"Could you clarify; did nCEO transition from high-to-low, implying that the first FPGA configured ok, and its the second that causes the failure?" 

 Sorry, Good catch. nStatus goes low when nCEO goes inactive, low-to-high.  Q4: Would nStatus go low earlier if the first FPGA had detected failure?

[Altera_Forum]

--- Quote Start ---   I do have nCE of the second device driven by the nCEO of the first device.   --- Quote End ---   Ok.    --- Quote Start ---   I believe you helped me pick the bus-switch when I was doing the schematics for the board.   --- Quote End ---   I recalled helping someone, but was not sure it was you :)    --- Quote Start ---   I see the issue in your diagram, that the PS timing of the second FPGA has little or no margin. I did not get the impact of this the last time I saw this.   --- Quote End ---   Right, but that margin is only in the worst-case condition of the internal oscillator having the minimum period. If your scope traces show a slower DCLK period, then you might be ok.    --- Quote Start ---   I did measure my signals and have included images with this reply. The clock was measured with a nice differential probe, but the data was measured with a standard probe with a long ground lead.   --- Quote End ---   I don't see the images - try attaching them again (use the "Go Advanced" button to get to the page that allows you to add attachments).    --- Quote Start ---   I really have very little margin on the PS as you suspected.    --- Quote End ---   If you have margin, then it should work.    --- Quote Start ---   Q3: Do you know off the top of your head whether I can simply invert the clock to the second FPGA to improve the PS Timing? I could use a NAND gate buffer instead of an AND.   --- Quote End ---   Try it and see. The problem I suspect you will trying to change DCLK to the FPGA, is that you have a common DCLK to both the EPCS device and second FPGA, so you will need to remove your bus switch and create two unique DCLK paths.    --- Quote Start ---   I did notice that DCLK does not pause at the point where nCEO goes from inactive to active. DATA  however does not transition for an interval before and after the switch (nCEO going active). I  expect that there is a "start bit" in the DATA and the timing of the clock with respect to nCEO does not matter. What do you think?   --- Quote End ---   If you look at the .pof files, there's a bunch of FFs at the start. There is likely something in the bit-sequence that acts as the start indicator.    --- Quote Start ---   Much to my dismay the nStatus and Config_Done signals between the two FPGA run completely on inner signal layers from one BGA via to the other BGA via.   --- Quote End ---   You live and learn.  There lesson here is to make all configuration signals route through 0-ohm resistors so that you can disconnect your programming chain when debugging. This goes for JTAG chains too. I've had to jumper over bad devices before, and its very convenient to simply remove a 0-ohm resistor on the TDO->TDI paths, and jumper the working TDO to a working TDI.  Cheers, Dave

[Altera_Forum]

--- Quote Start ---  "Could you clarify; did nCEO transition from high-to-low, implying that the first FPGA configured ok, and its the second that causes the failure?"  Sorry, Good catch. nStatus goes low when nCEO goes inactive, low-to-high.  Q4: Would nStatus go low earlier if the first FPGA had detected failure?  --- Quote End ---    If there was something wrong with the configuration data, then I would expect the first device to "complain" earlier, i.e., drop nSTATUS low.  Do you have JTAG connected on this board? Does it work?  If it does, you could ...  1. Confirm that both devices can be configured using JTAG. Probe nCEO, nSTATUS, and CONF_DONE.  2. Load the EPCS with only the configuration data for the first device.   Pulse nCONFIG, and the EPCS should configure only the first device. That first device should assert nCEO to enable the second device.  2. Use JTAG to load the second device.  I don't see any reason why that should not work - but I'm not sure if its totally valid - try it and see.  Cheers, Dave

[Altera_Forum]

Yes I will give the JTAG a go, but I cannot isolate the nStatus's and Config_Done's. If one of the devices feel that it has not been configurated I will not get Config_Done going high. The Quartus programming tool will report this as an error. I do not know what nStatus will do? I will try this.

[Altera_Forum]

I cannot program the two FPGA's separately since I cannot decouple the Config_Done's. However, if I repeatedly trigger 

off of nCEO and monitor Config_Done at 10 mV per division and lots of averaging I can tell that the first FPGA must have released its Config_Done right at the time that nCEO goes active (low). Config_Done shows ringing and a 3 mV change in its level.  This looks like evidence that it is the second FPGA that is failing its configuration, yes ??

[Altera_Forum]

--- Quote Start ---  I cannot program the two FPGA's separately since I cannot decouple the Config_Done's.   --- Quote End ---   You misunderstand. The first FPGA *can* be configured by the EPCS device, but it will not enter user mode, since the other device is holding CONF_DONE low.  If you use JTAG to program the second device, then it will release CONF_DONE, and both FPGAs should enter user mode.  Yeah, its subtle, but it should work.    --- Quote Start ---    However, if I repeatedly trigger off of nCEO and monitor Config_Done at 10 mV per division and lots of averaging I can tell that the first FPGA must have released its Config_Done right at the time that nCEO goes active (low). Config_Done shows ringing and a 3 mV change in its level.  This looks like evidence that it is the second FPGA that is failing its configuration, yes ??  --- Quote End ---    Yeah, that sounds like a reasonable assumption.  You did not comment yet whether you can program these devices with JTAG.  Cheers, Dave

[Altera_Forum]

--- Quote Start ---  Yes I will give the JTAG a go, but I cannot isolate the nStatus's and Config_Done's.   --- Quote End ---   That should not matter for this test.    --- Quote Start ---   If one of the devices feel that it has not been configurated I will not get Config_Done going high. The Quartus programming tool will report this as an error. I do not know what nStatus will do? I will try this.  --- Quote End ---   If the first device is configured from EPCS, and the EPCS is setup to ignore CONF_DONE assertion, then it should configure the first device, and be done. CONF_DONE will not assert, but the first FPGA will be configured, and just "stalled" waiting for CONF_DONE to be released.  If you then program the second device via JTAG - while bypassing the first device - then the second device should configure and release CONF_DONE, and both devices will enter user mode.  Cheers, Dave

[Altera_Forum]

Ok, I am back. Going home and fixing dinner for the kids comes earlier here in Illinois. Thank you for your suggestions. I will be trying out your suggestions as soon as I go get some coffee.  

 Also, over the weekend I was reading about the advanced functions in the "Convert Programming Files" utility. It appears that I can disable the Config Done error check, modify the "Program Length Count", and pad bytes onto the bitstream (http://www.altera.com/literature/hb/qts/qts_qii53022.pdf). Any experience with these adjustments ?  Coffee first.

[Altera_Forum]

Ok, I did the following to load my two FPGA's separately, and got both FPGA's doing their thing. 

 1. Used "Convert Programming File" (CPF) utility to combine the two *.sof files into a *.pof for loading into the EPCS16, for programming the two FPGA's.  1a. In the Advanced menu of the CPF I selected the "Disable AS/AP mode CONF_DONE pin error check" check box. It had previously been checked but grayed out. I am not sure what being grayed out means (?).  2. Loaded the EPCS16 with this *.pof. Powered down. Powered back up. And ...    a) No blinking lights, FPGA logic not running.  b) nStatus High and NOT pulsing low. Not re-configuring.  c) nCEO Low and not toggling.   d) Config_Done (OR'ed from both FPGA's) Low, not happy.  e) Init_Done from first FPGA Low, not happy.  3. Attached USB Blaster to JTAG port for second FPGA and loaded second FPGA with its own *.sof.   a) Blinking lights from both FPGA's. FPGA logic running in both FPGA's.  b) nStatus High and NOT pulsing low. Not re-configuring.  c) nCEO Low and not toggling.   d) Config_Done (OR'ed from both FPGA's) High, happy.  e) Init_Done from first FPGA High, happy.   OK, mystery fans, what is my problem loading both FPGA's from the EPCS16 and having them work ?  My Speculations:  1. When I selected the "AS/AP mode CONF_DONE pin error check" I got the FPGA's to relax about the error in the configuration (wherever it is) and this allowed me to program the second FPGA through the JTAG.  2. Signs indicate trouble with the configuration of the second FPGA from the EPCS16.  Also Noticed:  When loading the EPCS16 using the USB Blaster and the Quartus II Programmer utility the process goes through the following. I noted that the "Programming device 1" step quits and moves on to verification when the "Progress Bar" in the top corner of the window has only reach 35%.  Info: Device 1 silicon ID is 0x14 Info: Erasing ASP configuration device(s) Info: Blank-checking device 1 Info: Programming device 1 <<<<<<<<<<<<<<<< Quits when progress bar has only reached 35% Info: Performing verification on device 1

[Altera_Forum]

--- Quote Start ---   Ok, I did the following to load my two FPGA's separately, and got both FPGA's doing their thing.   --- Quote End ---   Yah, progress!    --- Quote Start ---   OK, mystery fans, what is my problem loading both FPGA's from the EPCS16 and having them work ?   --- Quote End ---   Not sure about this one. I've always used PS/FPP to configure chains of FPGAs.  Hopefully someone with a similar setup to yours can comment.  You should also try filing a Service Request with Altera. You now have a good description that shows how to get the chain to work vs fail, so Altera may be able to suggest a few tests.  Cheers, Dave

[Altera_Forum]

--- Quote Start ---   Also Noticed:  When loading the EPCS16 using the USB Blaster and the Quartus II Programmer utility the process goes through the following. I noted that the "Programming device 1" step quits and moves on to verification when the "Progress Bar" in the top corner of the window has only reach 35%.  Info: Device 1 silicon ID is 0x14 Info: Erasing ASP configuration device(s) Info: Blank-checking device 1 Info: Programming device 1 <<<<<<<<<<<<<<<< Quits when progress bar has only reached 35% Info: Performing verification on device 1  --- Quote End ---    Read the contents of the EPCS device and see whether they match what you expect.  EPCS is just SPI flash, so its easy to read it.  Do you have an AS header on the board, or are you using SFL to access the EPCS device?  I probably have some example code I can send you.  Cheers, Dave

[Altera_Forum]

I performed another variation on the programming of the FPGA's. This time I UN-Checked the "Disable AS/AP mode CONF_DONE pin error check" check box. 

in the Advanced menu of the "Convert Programming File" (CPF) utility.  Also, I disabled the "Auto-restart the configuration after error" check box in the Devices Settings for the FPGA's and then recompiled them.  This is what I got  4. Loaded the EPCS16 with this *.pof. Powered down. Powered back up. And ...  a) No blinking lights, FPGA logic not running. b) nStatus LOW and pulsing. Not re-configuring. <<<< this time the Config Done error was detected and indicated by nStatus. c) nCEO Low and not toggling.  d) Config_Done (OR'ed from both FPGA's) Low, not happy. e) Init_Done from first FPGA High, happy (?). <<<< Init_Done seems to indicate things are well but the FPGA logic is not running (?)   5. Attached USB Blaster to JTAG port for second FPGA and loaded second FPGA with its own *.sof.  Info: Device 1 contains JTAG ID code 0x020F40DD Info: Configuration succeeded -- 1 device(s) configured Info: Successfully performed operation(s) Info: Ended Programmer operation at Mon Nov 04 09:57:52 2013  nStatus still Low. No change in signals from step (trial) 4. above.  6. Repeated the steps with the "Disable AS/AP mode CONF_DONE pin error check" check box checked and got the same results as before for this case.  =================================================================================  My Speculations:  3. Init_Done is not valid unless nStatus is High.  4. A Config Done Error is probably being signaled by the second FPGA and by checking the "Disable AS/AP mode CONF_DONE pin error check" check box, the nStatus will not see this and go High at the end of the configuration cycle.  5. Programming via JTAG will not get things going if nStatus is indicating an error

[Altera_Forum]

I am using the standard 10 pin AS Header for programming the EPCS.

[Altera_Forum]

--- Quote Start ---  I am using the standard 10 pin AS Header for programming the EPCS.  --- Quote End ---   Send me an email (to my forum name) and I'll send you an application that can read/write the EPCS.  Cheers, Dave

[Altera_Forum]

--- Quote Start ---   3. Init_Done is not valid unless nStatus is High.   --- Quote End ---   I believe I have already mentioned to you that this is an optional output, and as such it is meaningless until near the end of configuration.    --- Quote Start ---   4. A Config Done Error is probably being signaled by the second FPGA and by checking the "Disable AS/AP mode CONF_DONE pin error check" check box, the nStatus will not see this and go High at the end of the configuration cycle.   --- Quote End ---   CONF_DONE does not signal errors, nSTATUS does. CONF_DONE is held low by the first device until it configures, at which point it remains low, because the second device is not configured.    --- Quote Start ---   5. Programming via JTAG will not get things going if nStatus is indicating an error  --- Quote End ---   Can JTAG be used to program both devices at power-on? If so, then the issue is likely related to the EPCS configuration file contents.  Cheers, Dave