Help with A10 Backplane Auto Negotiation

[JShel4]

Hello,

I need help/pointers for debug of Backplane Auto Negotiation.

My System:

• Arria10 FPGA <=> Copper Backplane <=> AMD SoC
• Both partners are capable of 10G and 1G modes
• I have matched the Pause ability on both side -- i.e. only enabled C0

Observation:

• FPGA does not believe it's getting valid DME packets. SignalTap shows :
• an_rx_idle goes high and stay HIGH indefinitely
• an_pg_received is stuck at LOW
• the dme_in bus is showing a constant value of 10100009C0100009C (hex)

Any pointers on how to debug will be very helpful. Thank You in advance!

[CheePin_C_Intel]

Hi Jaydev, As I understand it, you have some inquiries related to the backplane auto-negotiation. To ensure we are on the same page, just would like to check with you if you are using any specific Ethernet IP in A10 device? Please feel free to let me know to facilitate further debugging. Thank you.

[JShel4]

Hello,

Yes, I am using the 1G/10G Ethernet IP in Arria10.

My observation:

• FPGA believes to be sending the right DME -- I can see it in SignalTap
• But, I don't see the AMD registering that information -- I see this by reading the LP registers using the MDIO interface
• The same happens in reverse direction:
  • i.e. AMD believes it is sending information (again, I read the advertised/tech registers using MDIO interface) ...
  • but, FPGA is not registering these values in the LP registers ... actually, I don't even see the page_received signal go high.

Please share any tips/tricks/information you may have with Arria10 experience. Thank You very much in advance!

[JShel4]

Attaching the two STP files showing information at the Sequencer and AN. Thank You in advance for any help/guidance you can provide.

(since I cannot upload .stp files, I have put the two files in a ZIP file)

[CheePin_C_Intel]

Hi Jaydev, Thanks for your clarification. Just would like to double check with you if you are using 10GBASE-KR PHY IP Core or 1-Gigabit/10-Gigabit Ethernet (GbE) PHY IP Core as described in the A10 XCVR PHY user guide? If not, please feel free to refer me to the specific web link or documentation reference for it. This would be helpful for me to debug into the right IP. Sorry for the inconvenience. The reason I am asking this is because I am unable to find an_rx_idle or an_pg_received status signals at the above PHY IP core. Thank you. Best regards, Chee Pin

[JShel4]

Hi Chee Pin,

In the XCVR PHY User Guide: 2.6.3. 10GBASE-KR PHY IP Core This is the IP I am using for 10g Backplane interface (1-lane).

Those signals are inside the IP -- not the signals/interfaces available at top level to a user

If you go down to the level where Sequencer is instantiated, you will see those signals.

Thanks!

[CheePin_C_Intel]

Hi Jaydev, Thanks for your clarification that you are using the 10GBaseKR PHY IP in A10 devices. To facilitate further debugging, just would like to check with you on the following: 1. Just wonder if you have had a chance to try perform a loopback from A10 TX back to A10 RX to see if there is any issue with the Auto-Negotiation. Just to help isolating the 3rd party devices from our debugging. 2. If you are unable to perform serial loopback at the copper backplane, you may try to enable the internal serial loopback by writing to the phy_serial_loopback register at 0x461. 3. Just wonder if you have had a chance to try perform a Modelsim simulation to see if similar AN issue can be replicated in the simulation? This would be helpful to isolate any functional issue prior to hardware testing. 4. Mind share with me what is the top level signal that you observe anomaly before you signaltap the internal signals? For example, you are observing led_an not gettting asserted? 5. Please help to share with me the status signal at the IP top level so that I could have better understanding of the IP during issue occurence. 6. Please share with me you IP .qsys file so that I can have better understanding of your configuration. Please let me know if there is any concern. Thank you. Best regards, Chee Pin

[JShel4]

Hi Chee Pin, Answer to your questions:

I cannot perform the loopback on the copper backplane. I tried internal serial loopback. The Link stays in AN mode (does not exit). For some reason, the rx_parallel_data_navtive[63:0] is stuck at 0100009C0100009C0h

I have not yet done Simulation. I will try that in next couple of days.

At top level I see: pcs_mode[5:0] = 00001 | all LED signals are LOW | rx_data_ready=0 | rx_block_lock=0 | rx_hi_ber=0

I will attach the IP .qsys file with this update. Please rename the file to .qsys (I had to change extension because this website wont allow me to upload the file as-is).

I did a experiment: I forced 10G mode on the FPGA and AMD SoC. I am able to pass 10g traffic when both are forced to 10g mode. This tell me that the two PHYs and the physical connection are doing what they are supposed to do.

Please let me know what you think. Thank You in advance!

[CheePin_C_Intel]

Hi Jaydev, Thanks for your update. As I understand it, when you enabled the internal serial loopback, the AN seems not working as well. This could help to isolate out 3rd party modules and allow us to focus in A10 device itself. For your information, as I search through the web, I found some A10 10G example designs as following: https://fpgawiki.intel.com/wiki/Arria_10_Transceiver_PHY_Design_Examples#1G.2F10GbE_and_10GBASE-KR_PHY_Design_Example Since you have narrow down to A10 device with serial loopback, it would be great if you could further look into these designs to see if you can find anything useful. These designs seems to be for older Quartus version but you can try to refer to see if it helps. I believe the first milestone to achieve would be to get a one channel 10GBaseKR PHY to simulate successfully and out from AN. Please let me know if there is any concern. Thank you. Best regards, Chee Pin

[CheePin_C_Intel]

Hi Jaydev, Just would like to follow up with you on this and if you are able to replicate similar observation in simulation. Thank you.