PC doesn't boot when RXM_BAR4 connects ext DDR3 cntrlr in AVMM DMA PCIe Qsys design

[Altera_Forum]

Hi, 

 I am stumped with this one. I have modified the Altera stratix V PCIe AVMM DMA reference design to include a UniPhy based external DDR3 controller to the Qsys. The target fpga is sitting on a PCIe development board. It is a PCIe x4 gen 3. No CvP. The qsys has both external memory controller and on-chip memory of 64KB. I have verified that the address regions do not overlap between the external memory and on-chip memory.  A couple of stumbling blocks that I cannot explain -  1. When I connect RXM_Bar4 to the DDR3 controller's AVL interface, the PC doesn't boot. With signaltap I see that the DDR3 controller has calibrated. If remove the RXM_Bar4 connection to the DDR3 controller. The PC boots and I can access the DDR3 while maintaining data integrity. So what gives?  2. If I remove the RXM_Bar4 connection to the DDR3 controller, then the BAR4's address region is automatically adjusted down from 4GB (32bits) to 64KB(16bits).   Thank you in advance for helping me figure this out. Best regards, Sanjay

[Altera_Forum]

Sounds like you are trying to create a larger BAR than the BIOS can allocate resource to. 

A 4GB BAR would have to be 64bit address - and recognised by the BIOS as such, I'm not sure PC BIOS support them. I'd guess that all the memory BAR (for all cards) have to fit into either 512M or 1G at the top of the 32bit address space. You might manage a 256M window into the bottom of the DDR (feed the avalon signals through a conduit to force the high address bits to zero).

[Altera_Forum]

Thank you DSL. That may explain it. But I'm still confused why by connecting RXM_Bar4 to the external DDR3 controller the PC doesn't boot. I see that the PCIe doesn't link train in this case as the link leds on the board do not come up. Whereas if I increase the range of the card memory, the pc doesn't boot, but the link leds indicate that the card did link train and enter L0 state. 

 Best regards, Sanjay

[Altera_Forum]

trilion99, 

Thank you for sending over the pictures and sharing your experience. This sounds like the way to go.