100G E-Tile transceiver adaptation

[alexforencich]

I am having trouble getting the RX side of the link working with the E-Tile in 100G mode on an Agilex F 014 part (DE10-Agilex board).  I previously had a similar issue with the E-Tile in 10G and 25G mode, and was directed to manually trigger the PMA configuration load following the E-Tile PHY user guide (https://community.intel.com/t5/FPGA-SoC-And-CPLD-Boards-And/E-tile-automatic-adaptation-issues/m-p/1279308#M19854).  Unfortunately, this same technique doesn't seem to work for the core in 100G mode. 

The problem that I am running in to is that the status register at address 0x40144 always reads as 0x00.  According to the documentation, it should read as 0x04 while the module is loading the configuration, then it should read as 0x01 when it is done.  So it seems like maybe the configuration load soft IP is not getting enabled and included in the design, despite the check box being checked in the parameter editor. 

Screen shot of the transceiver reconfig AVMM interface, which shows the write of 0x80 to 0x40143, followed by a read of 0x40144 which returns 0x00, on all four channels:

Has anyone else run in to this issue?  Is there a particular way I need to configure the E-Tile in the parameter editor so that the transceivers will work correctly?

[alexforencich]

Minor update to this, just to rule out some possible settings combinations:

• "Adaptation load soft IP" off, xcvr_reconfig tied off (no reconfig logic): no link, need to adjust adaptation settings in system console to get an RX link
• "Adaptation load soft IP" on and NRZ_28Gbps_LR selected and settings saved in config 0, xcvr_reconfig tied off (no reconfig logic): no link, need to adjust adaptation settings in system console to get an RX link
• "Adaptation load soft IP" on and NRZ_28Gbps_LR selected and settings saved in config 0, xcvr_reconfig connected to state machine: no link, need to adjust adaptation settings in system console to get an RX link (and state machine is stuck trying to trigger the PMA configuration streamer, which never appears to start)

Hardware config:

FPGA (DE10-Agilex rev B with AGFB014R24A2E2VR0) E-Tile -> QSFP-DD cage -> QSFP28 DAC -> Mellanox ConnectX-5 NIC

In this case, the ConnectX-5 reports that the link is up at 100 Gbps, but rx_pma_ready on the E-Tile is low.  If take the settings that the transceivers get when in 25G mode and apply them via the system console, then the link does come up.  Question is how to do that without involving the system console. 

Edit: and I just tested with a 30M AOC, and I was able to get a link.  So, apparently the default parameters on the E-Tile transceivers will work with AOCs with integrated CDRs, but not DACs.  However, the same exact DAC works perfectly with Xilinx UltraScale+ devices at 100G, and with the E-Tile at 25G (with the adaptation load soft IP working to configure the transceivers), so it seems like the E-Tile should be able to work in this situation. 

[Kshitij_Intel]

Hi Alex,

Can you please share which OS and Quartus version you are using.

Also, Please confirm your target device AGFB014R24B2E2V.

It would be good if you can share your project .qar.

Thank you

Kshitij Goel

[alexforencich]

OS is Linux.  I am currently using Quartus Prime Pro 22.1, as 22.2 is broken (see https://community.intel.com/t5/Intel-Quartus-Prime-Software/PCIe-HIP-clocking-regression-on-Stratix-10-MX-in-Quartus-Prime/m-p/1394323#M74234). You guys sent me a DE10-Agilex rev B instead of a rev C, so I have an ES part (AGFB014R24A2E2VR0) instead of a production part (AGFB014R24B2E2V). 

The code is here: https://github.com/alexforencich/corundum/tree/master/fpga/mqnic/DE10_Agilex/fpga_100g

I'll generate a QAR soon.

[alexforencich]

QAR is attached

[Kshitij_Intel]

Hi Alex,

It seems you are loading the configuration but PMA Adaption needs to be enabled manually which not happening in your case.

Please follow one of flow for PMA Bring up to start with follow the STF Link bring up flow(LHS)

https://www.intel.com/content/www/us/en/docs/programmable/683723/current/pma-bring-up-flow.html

Thank you

Kshitij Goel

[Kshitij_Intel]

Hi Alex,

Hope the problem is resolved. If not, please let me know.

Thank you

Kshitij Goel