100G E-tile error on Stratix 10 DX

[alexforencich]

I am attempting to build a design with an E-tile configured for 100G operation on a Stratix 10 DX device, and I am running in to some strange build issues.  I am building designs for both Stratix 10 DX and Agilex F, and the Agilex design so far has been fine (although getting the link to work is a different story), but the Stratix 10 design so far does not want to build. 

The core is in the "100GE or 1 to 4 10GE/25GE Channels with optional RS-FEC and PTP", in 100G mode, with PTP and RS-FEC enabled.  When I build the design, I get the following error during synthesis:

Error(18806): WYSIWYG primitive "qsfp1_mac_inst|mac_inst|alt_ehipc3_0|TX_ADAPTER_100G.TX_ADAPTER_SYNC.PTP_TX_ADAPTER_AND_FIFO_PR.ptp_tx_ff|wdata_reg_2[0]" connected to WYSIWYG primitive "qsfp1_mac_inst|mac_inst|alt_ehipc3_0|TX_ADAPTER_100G.TX_ADAPTER_SYNC.PTP_TX_ADAPTER_AND_FIFO_PR.ptp_tx_ff|ml[0].lrm" must be a simple Hyperflex-friendly register with only D, CLK, and Q port connected.

This seems to be from somewhere deep inside the core, so I'm not sure if I'm doing something wrong here, or if there is some sort of a bug in the core itself.  Has anyone else run in to a similar issue with the E-tile?

[wchiah]

Hi,

Welcome to Intel Forum Support.

Could you share both the Quartus QAR for design with VQM and HDL source file for investigation?

Regards,

Wincent_C_Intel

[alexforencich]

Sure, I'll get that put together soon.

[alexforencich]

QAR is attached

[wchiah]

Hi Alex,

Apologize for late reply, as I am ooo for urgent.

I try to run your design, I am getting the same error.

I lay down some suggestion so that we can narrow down the issue.

1. Do you try the design in Quartus version 22.2 or 21.0 ?
2. Can I know how you design the code? from scrap? or any base example you are referring?
3. Can I know which design IP is this? if I understand correctly it shall be Ethernet right ? Please correct me if I am wrong.
4. Do you try another similar function example with the same IP in our FPGA Design Store ?
5. https://www.intel.com/content/www/us/en/support/programmable/support-resources/design-examples/design-store.html?s=Newest&f:guidetm83741EA404664A899395C861EDA3D38B=%5BIntel%C2%AE%20Stratix%C2%AE%5D

As the architecture of Stratix 10 and Agilex F is quite different from hardware to software, we cannot guarantee the same design can completely work within both of them as a lot of factors need to be considered such as pin assignment, logic cells, I/O elements, IP, and IP cannot be migrated directly between different devices.

Regards,

Wincent_Intel

[alexforencich]

I'm using 22.1.  I have not been using 22.2 due to this issue: https://community.intel.com/t5/Intel-Quartus-Prime-Software/PCIe-HIP-clocking-regression-on-Stratix-10-MX-in-Quartus-Prime/m-p/1394323#M74234

The code that interfaces with the E-tile was developed from scratch based on the documentation for the Agilex F series part, and is used with minimal modification on the Stratix 10 DX.  The core logic that does not interface with the E-tile was also written from scratch, but runs and builds correctly on many different FPGAs, including the Stratix 10 DX.  

The problem is from the E-tile 100G Ethernet IP core.

I have not tried to build the example design, but I will give it a shot and see if I can replicate the issue.  I suspect that maybe it has to do with some of the unused PTP signals being tied off, instead of being driven by registers. 

Sure, the Stratix 10 DX and Agilex F are somewhat different, but the E-tile is ostensibly exactly the same on both.  And the core logic already works fine on the Stratix 10 DX and 10G and 25G, only difference between 10G/25G/100G in terms of the core logic is clock frequency and datapath width.  And the core logic runs correctly on many other FPGAs without any device-specific modifications (Xilinx Virtex 7, Xilinx UltraScale, Xilinx UltraScale+, Intel Stratix 10 MX, Intel Stratix 10 DX, Intel Agilex F). 

[wchiah]

Hi Alex,

I am the same support engineer who works with you on the issue that happens in 22.2.

The escalation case was just assigned to debug engineer. Will continue to follow up on that.

You can try the suggested method of trying to replicate the issue.

As you mention that the design was built from scrap,

the possibility of missing some of the design elements might happen sometime/somewhere

Where it is quite hard to pinpoint, as the FPGA design somehow can be really complicated.

Meanwhile, I do suggest you try out the existing E-tite Ethernet Design for Stratix 10 Instead of modified form Agilex design,

Yes, you are correct by right there shall be the same for both as for E-tile
But, as the device is different, I do afraid there is a compatible issue with software, driver, system, kernel and other etc.... 

https://www.intel.com/content/dam/support/us/en/programmable/support-resources/bulk-container/pdfs/literature/ug/archives/ug-s10-etile-hip-ethernet-20-2.pdf

It is tested on our team and shall work fine.

Hope this help,

Regards,

Wincent_Intel

[wchiah]

Hi Alex,

For https://community.intel.com/t5/Intel-Quartus-Prime-Software/PCIe-HIP-clocking-regression-on-Stratix-...

The clock will be obtainable in the new release of Quartus version 22.3.

Please try it in the newest version of the Quartus (once available).

This thread will be transitioned to community support.

If you have a new question, feel free to open a new thread to get support from Intel experts.

Otherwise, the community users will continue to help you on this thread. Thank you

Best regards,

Wincent_Intel

[wchiah]

Hi Alex,

Just a quick follow up since there is no reply.

Did you try out the provided E-tile example ?

Regards,

Wincent_Intel

[alexforencich]

So I did some digging around, and the problem seems to be with how i_ptp_fp was being driven.  Apparently something about the soft logic wrapper around the E-Tile ends up requiring that all of the PTP-related input signals are driven by a "hyperflex-friendly register". 

For reference, the error is associated with the wdata_reg input of

        alt_ehipc3_mlab #(
            .WIDTH      (TX_PTP_WIDTH),
            .ADDR_WIDTH (4),
            .SIM_EMULATE(0)
        ) ptp_tx_ff (
            .wclk       (int_tx_clk),
            .wdata_reg  (usr_tx_ptp_ff_in),
            .wena       (tx_ts_ff_write),
            .waddr_reg  (ptp_tx_ff_waddr),
            .raddr      (ptp_tx_ff_raddr),
            .rdata      (usr_tx_ptp_ff_out_wire)
        );

This is driven by

        assign usr_tx_ptp_ff_in = {i_ptp_ins_ets,       // 1-bit
                                   i_ptp_ins_cf,        // 1-bit
                                   i_ptp_zero_csum,     // 1-bit
                                   i_ptp_update_eb,     // 1-bit
                                   i_ptp_ts_format,     // 1-bit
                                   i_ptp_ts_offset,     // 16-bit
                                   i_ptp_cf_offset,     // 16-bit
                                   i_ptp_csum_offset,   // 16-bit
                                   i_ptp_eb_offset,     // 16-bit
                                   i_ptp_tx_its,        // 96-bit
                                   i_ptp_ts_req,        // 1-bit
                                   i_ptp_fp             // 8-bit
                                  };

Apparently whatever directly drives these signals must be a simple flip flop with no reset input.  Not an MLAB, not an M20K, not even a flip flop with a reset, preset, etc. input.

Personally, I consider this to be a bug in the soft logic wrapper.  If this MLAB instance needs to be driven by "hyperflex-friendly registers", then such registers should exist in the soft logic wrapper to isolate the upstream logic from the MLAB instances to avoid this kind of difficult-to-debug DRC error.  It's unclear why this only seems to be a problem on Stratix 10 devices, but not on Agilex.

Anyway, what I ended up doing was adding a register slice across all of the input signals on the transmit side, both the AVST interface and i_ptp_fp.  Presumably this will end up being a permanent workaround for this issue.

[wchiah]

Hi Alex,

Thousand appreciate for sharing with me the solution,

My best action will be to submit an enhancement ticket to the development team.

Hope that this Issue will resolve in the next release version of the software.

Do you have any further question ?

Else I would like to have your permission to close this case.

If you feel your support experience was less than a 9 or 10,

please allow me to correct it before closing or please let me know the cause so that I may improve your future support experience.

Regards,

Wincent_Intel