D-1559 10G issue

[Chanman]

Hi,

We used Xeon D-1548 in original design. 

We replaced D-1548(8-core) with D-1559 (12-core) in new revision.

All the interfaces are working fine on new D-1559 board except 10G interface.

We are using  X557-AT2 10GBASE-T PHY on D-1559 10GBASE-KR interface.

As per the data sheet D-1559 supports 10GBASE-KR.

There is no layout change in 10G section from previous revision.

Linux is reporting link down whereas PHY is performing auto-negotiation. link and speed LED's are ON.

Tried to debug using Intel lanconf tool in UEFI shell. lanconf tool shows link is down. cable diagnosis passes. 

I think lanconf tool uses sideband(MDC/MDIO) for checking the cable

Seems problem with MAC to PHY interface.

I am able to program PHY NVM through MDC/MDIO interface and can able to read PHY registers in lanconf.

Basic test in lanconf always passes.

I suspect there is problem with 10GBASE-KR interface. Our board uses AMI BIOS released in 2017.

Is there any change in straps from D-1548 to D-1559?

What may cause this issue?

Regards,

Suresh

[CarlosAM_INTEL]

Hello, @Chanman:

Thank you for contacting Intel Embedded Community.

You need to use the Broadwell-DE-NS [BDX-DE-NS V2 and Y0 Stepping] 10G LAN Enabling Kit [LEK] document # 573095 that may solve the cited inconvenience. You can find it when you are logged into your Resource & Design Center (RDC) privileged account on the following website:

http://www.intel.com/cd/edesign/library/asmo-na/eng/573095.htm

The RDC Account Support form is the channel to process your account update request or report any inconveniences with the provided site. It can be found at:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best regards,

@CarlosAM_INTEL.

[Chanman]

Thanks @CarlosAM_INTEL ,

Is D-1559 a Broadwell-DE-NS family?

We are currently using

 Content ID: 561475  : "Broadwell-DE [BDX-DE V2 and Y0 Stepping] 10G LAN Enabling Kit [LEK], 19-November-2018"

10G works fine on  D-1548 and does not work on D-1559.

As per your recommendation, we tried programming Content ID: 573095, Broadwell-DE-NS [BDX-DE-NS V2 and Y0 Stepping] - file: BDXNS_10GBASET_NO_MNG_LAN0_2p00_800008D1.bin on D-1559. It does not help.

Did we missed any thing? Does Intel have tool to check KR interface between MAC and PHY?

PHY loopback fails in lanconfig tool.

Regards,

Suresh

[Chanman]

Hi @CarlosAM_INTEL 

I am awaiting  your response.

Regards,

Suresh

[CarlosAM_INTEL]

Hello, @Chanman:

Thanks for your updates.

We need to address the following questions to help you:

Could you please let us know if you have tested this issue on a Customer Reference Board (CRB) using Intel BIOS?

Could you please inform the microcode version of the reported situation? Please note that the D-1559 is a Y0 stepping SoC, while the D-1548 is a V2 stepping SoC.

For D-1548 which corresponds to CPUID 50663 customer should be using microcode 0x7000019.

For D-1559 which corresponds to a CPUID 50664 customer should be using microcode update 0xF000017.

Please check  the GitHub page that has microcode updates and confirm your system is up to the latest taking into consideration the SoC:

https://github.com/platomav/CPUMicrocodes/tree/master/Intel cpu50663_plat10_ver07000019_2019-06-17_PRD_C5E77EC6.bin

Updated to MCE DB r122

cpu50664_plat10_ver0F000017_2019-06-17_PRD_43C2AD6B.bin

Updated to MCE DB r12

Could you please clarify if the affected design has been verified through an Intel Layout and Schematic Review?

We are waiting for your answer to these questions.

Best regards,

@CarlosAM_INTEL.

[Chanman]

Thanks @CarlosAM_INTEL 

We didn't test D-1559 on Intel CRB.

Our board AMI BIOS supports D-1559  CPUID 50664 with microcode verison F00000A.

cpu50664_plat10_ver0F00000A_2016-06-02_PRD_746156A3.bin

How ever, Linux is loading latest version of microcode F000017 cpu50664_plat10_ver0F000017_2019-06-17_PRD_43C2AD6B.bin

We have 10G link working once in 20 times.

Are there any things do you want us to try?

Do we need to update BIOS or FSP?

Design has been verified. 10G is working fine with D-1548 without issues.

Regards,

Suresh

[CarlosAM_INTEL]

Hello, @Chanman :

Thanks for your reply.

We need to address the following questions to understand the reported situation:

Could you please clarify that this condition is related to FSP?

Could you please integrate the Microcode version F000017 into the AMI BIOS? Because the microcode should be included in the BIOS. 

Best regards,

@CarlosAM_INTEL.

[Chanman]

Thanks @CarlosAM_INTEL 

Are there any updates in Intel Reference Code or Best Known Configuration(BKC) for Xeon-D SOC?

If there are changes we will contact our BIOS vendor. Our BIOS developed in 2017.

Regards,

Suresh

[CarlosAM_INTEL · Accepted solution]

Hello, @Chanman:

Thanks for your reply.

You should review the microcode information stated in table 3 of the Intel® Xeon® D-1500/D-1500 NS and Intel® Xeon® D-1600 NS Processor Families NDA Specification Update document # 557970. You can find it when you are logged into your Resource & Design Center (RDC) privileged account on the following website:

http://www.intel.com/cd/edesign/library/asmo-na/eng/557970.htm

You should have the system update using the latest microcode version.

The cited specification update contains a list of errata, which are fixed some of them updating the microcode to the latest revision.

The RDC Account Support form is the channel to process your account update request or report any inconveniences with the provided site. It can be found at:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best regards,

@CarlosAM_INTEL.