Intel WGi210IT packet loss at temperatures >= 65C

[Jean-Francois]

Hello intel,

I am facing an issue with packet losses on the i210IT (WGI210IT S LJXS, procured from Mouser) on several PCBa, only when operating at a link speed of 1Gbps. I have tested operation @ 100Mbps using an external 100Mbps switch and I do not face this issue.

Here is our current configuration:

- External crystal (ECS-250-18-33-JGN-TR), ±30PPM frequency stability, ±20PPM frequency tolerance, with 27pF load capacitance and rated -40/+85C, ESR of max 40R, drive level of 100uW. The load capacitors are C0G/NP0

- Using the i210 internal power rails for +1.5Vcc and +0.9Vcc. Considering the DC bias of the capacitors used, we have ~41.2uF effective on +1.5Vcc and ~ 47.1uF effective on +0.9Vcc

- Using a dedicated magnetic (S558-5999-P3-F) rated for -40/+85C and validated already on other designs. Based on the intel reference design guide, the center-tap on the i210 side are tied together, with 0.1uF per center-tap and 1uF global.

- 100R differential impedance on the PCB layout without any disruptions / traces routed over a plane split. The MDI pairs have a solid GND reference (on L2) and the impedance control has been verified with the PCB manufacturer.

- The stackup is 4-Layer, 1oz external copper thickness, 1oz internal copper thickness

The issue:

Either in a PING configuration with a switch outside the thermal chamber or by doing MAC-to-MAC transaction, we observe a gradually increasing quantity of packet being lost when exceeding 65C, with packet losses ~ 70% at 85C.

Here are the steps taken, to date, to try and diagnose the issue:

1) Our design has no heatsinking solution due to our operating environment (industrial, must have 0 moving parts). Thus we designed and implemented a custom heatsink, and we measured the IC temperature using a thermocouple following Intel recommendation in the i210 datasheet: at 85C ambient temperature, the IC stabilizes to 97C, which is below the maximum junction-case temperature of 105C. I was expecting this to solve our issue, but it did not, we still see an unusual high amount of packets being lost at >= 65C.

2) We have used the lanconf tool to re-perform IEEE compliance test at ambient temperature, and all tests passed successfully at all speeds (10Mbps, 100Mbps, 1Gbps).

3) We have used the lanconf tool with our ethernet card installed in thermal chamber to setup the i210 in internal MAC loopback, internal PHY loopback, and external loopback. These 3 tests were executed successfully at 85C for 100M+ packets, there were no "Data verify failures"

4) I have tried to change the crystal to a precision oscillator (±1.5PPM) without luck, still seeing the same packet loss issue @ 1Gbps at >= 65C

5) I have tried reworking our PCBa to power the i210IT using external regulator instead of the internal ones, without luck, still seeing the same packet loss issue @ 1Gbps at >= 65C

I am out of ideas, can you please help us investigate this issue? I have a hard time figuring out where the bug might be and felt like I have tried every possible ways.

Thank you!

[Sazirah]

Hi Jean-Francois,

Thank you for contacting Intel Community Forum.

Upon checking, we have our dedicated team Intel Embedded team that is expertise in this product issue that you reported. We will transfer this forum to Embedded team, kindly give them some time to get back to you.

Regards,

Sazzy_Intel

[CarlosAM_INTEL]

Hello,  @Sazirah :

Thank you for contacting Intel Embedded Community.

We need to clarify that this forum is not related to an Intel Embedded Platform.

Due to this, the customer request is out of our scope of support.

Please consider this information when you provide the previous suggestion, or transfer any forums to the channel that you mentioned in your previous communication.

Best regards,

@CarlosAM_INTEL.

[Jean-Francois]

@Sazirah  What would be the appropriate forum to be redirected to based on my ticket description ??

Unfortunately this ticket has not progressed yet since last week.

[Jean-Francois]

Hello @CarlosAM_INTEL , can you please let me know if I need to redirect my question to another forum or if Intel will automatically transfer the post to the appropriate support forum?

Thank you.

[Jean-Francois]

Some updates (the issue is not resolved yet).

Here are details / additional steps performed in the last 2 weeks:

- Dedicated magnetic is S558-5999-AT-F and not S558-5999-P3-F

- We performed the ethernet compliance tests at 85C using Tektronix suite + lanconf IEEE tests, everything is a PASS.

- We have been able to reproduce the packet loss issue on commercial off-the-shelf SOM from other vendors that are also using the i210-IT.

- We have measured the T(case) of the i210 and the T(amb) near the i210 with thermocouples on our PCBa, following recommendations from i210 datasheet, using our custom heatsink assembly:
1) At 65C thermal chamber ambient temperature, the i210 T(case) reaches 85.2DegC and the i210 T(amb) reaches 72.3DegC. No packets were lost.
2) At 70C thermal chamber ambient temperature, the i210 T(case) reaches 92.3DegC and the i210 T(amb) reaches 77.5DegC.
There was 0.21% and 0.31% packets loss on the 2 ethernet ports (510 lost out of 240784 on P1, 744 lost out of 239505 on P2).
The test ran for 60 hours.
The i210 T(case) and T(amb) were well within the limits of the IC, T(case-max) being 105C for IT version and T(amb-max) being 85C for IT version.

See below pictures for both the T(case) and T(amb) thermocouple location

[Sazirah]

Hi Jean-Francois,

Apologies for taking some time on checking on this case and thank you for your patience. Unfortunately, this issue is beyond my scope expertise and I shall reroute you to the correct team. To ensure you get proper assistance and support related to this issue, please post the question under embedded community: https://community.intel.com/t5/Embedded-Connectivity/bd-p/embedded-networking-connectivity and/or open an IPS case at premiersupport.intel.com.

After registration, you will be able to open a case on the Intel® Premier Support (IPS) platform and your request will be handled by one of our engineers as soon as possible.  

You shall be getting the proper support after you posted in the embedded forum and open IPS case. Since this is beyond our expertise and no further assistance that I can offer other than suggestion above, we will proceed with closing this forum ticket.

My pleasure to assist you and thank you for your patience and assistance.

Regards,

Sazzy_Intel

[CarlosAM_INTEL]

Hello,  @Sazirah :

Thanks for your reply.

However, as we mentioned in our previous communication on this thread, this forum is not related to an Intel Embedded Platform.

Due to this, the customer request is out of our scope of support through the embedded community.

Please again consider this information when you provide the previous suggestion, or transfer any forums to the channel that you mentioned in your previous communication.

Best regards,

@CarlosAM_INTEL.

[Jean-Francois]

Hello @CarlosAM_INTEL and @Sazirah 

I need guidance on what to do next because your statements are contradictory.

We have a thermal problem on the i210IT on one of our own design (we have designed the schematics / PCB).

The i210IT is an Intel ethernet product, thus why I originally posted on the ethernet product forum.

There seems to be i210IT related posts on the embedded connectivity forum too.

I need to speak with an engineer about the thermal metrics and environment in which the i210IT can operate without losing packets.

The tests we have conducted in our thermal chamber reveals that the i210IT start losing packets at an ambient temperature of ~ 70C and case temperature of ~ 85C, even if the IC purchased is an industrial IC supposed to operate at T(amb) max of 85C and T(case) max of 105C without issue (WGI210IT S LJXS).

In the "Online service center -> support request" the i210IT cannot be found under "Ethernet products", there is only the "Ethernet server adapter i210-T1", which I believe is an add-on board.

I want to ask a question that is directly related to the design of the IC of the i210IT.

Please help me get the appropriate support for this matter by either redirecting me to the appropriate forum page where I can ask this question.

Thank you