Arria 10 HPS EMAC with KSZ9131 PHY

[Zach2]

We've developed a custom Arria 10 SX board that pairs a KSZ9131 Ethernet PHY with the HPS EMAC, and using Linux 5.4 we see inconsistent Ethernet behavior depending on whether the link has gone down:

• The first time a link is established (whether an Ethernet cable was plugged in during boot or afterward), we're able to ping both directions between a PC and the Arria 10, with no dropped packets.
• Subsequently, if the link goes down and then comes back up (either because the cable was unplugged or by using ifdown/ifup), we can't ping to or from the Arria 10.

If we disable EEE (Energy Efficient Ethernet), the problem seems to disappear: we can take the link down and up without subsequent traffic being lost.

As far as we can tell, the PHY, EMAC, and relevant Linux drivers all support EEE, so we're trying to understand what might be going wrong. Any help would be much appreciated.

Additional details

We've previously done successful Arria 10 designs with the KSZ9031 PHY, which is very similar and shares almost all of the same Linux driver code and device tree bindings; we've never seen this problem with those designs.

We've run tshark on the Arria 10 and Wireshark on a PC while in the "bad" state trying to ping from the Arria 10:

• The PC receives some of the packets sent by the Arria 10, but not many; the PC responds to the packets it does receive.
• The Arria 10 receives none of the packets sent by the PC.

Testing setup:

• Direct Ethernet connection between Arria 10 and PC
• 1000 Mbps full duplex link
• Static IP addresses for both Arria 10 and PC

[ZiYing_Intel]

Hi,

May I know that you use which IP for this testing?

Best regards,

Zi Ying

[ZiYing_Intel]

Hi,

• Subsequently, if the link goes down and then comes back up (either because the cable was unplugged or by using ifdown/ifup), we can't ping to or from the Arria 10.

For this statement, just wonder that are you running external loopback under the condition cable was unplugged?

Best regards,

Zi Ying

[Zach2]

Hello,

We weren't using any loopback modes. The only actual traffic we were testing was pings.

Regarding your previous question, we're using the Synopsys DesignWare hard IP block for the MAC.

Thanks,

Zach

[ZiYing_Intel]

Hi Zach,

I haven't try this Synopsys DesignWare hard IP before. I would suggest you go to the Synopsys website to get more information about this IP.

Best regards,

Zi Ying

[gschuell]

Hi Zach,

I'm assuming from your initial post that you are using the HPS EMAC (from Synopsys).  Is this correct?

We are running into a very similar problem with our custom Arria 10 SX board and the KSZ9131 PHY.  We seem to be able to transmit properly but never receive successfully.  We are running our ping tests in U-Boot rather than Linux for now.

Have you had any luck in resolving your problem?

Thanks,

Greg

[Zach2]

Hi Greg,

---

I'm assuming from your initial post that you are using the HPS EMAC (from Synopsys).  Is this correct?

That's correct.

We've been able to work around the problem in Linux by adding the following properties to the PHY device tree node:

eee-broken-100tx;
eee-broken-1000t;

These disable advertisement of EEE in the Arria 10's PHY so that even if the link partner advertises EEE, it doesn't get enabled.

Since both the the KSZ9131 PHY and HPS EMAC claim to support EEE, we're still not sure why this is necessary.

I'm not sure if U-Boot has the same (or equivalent) device tree properties to disable EEE.

Zach

[gschuell]

Hi Zach,

That seems to have done the trick.  I appreciate your help on this.

Thanks,

Greg