i211 cause "Detected Tx Unit Hang" on new design with ubuntu16.04 4.4.0-31-generic

[GOman1]

Hi Intel,

we have created a new PCB that integrates two WGI211AT S LJXY. These chips were flashed using the files from

• 348742-quartzville-tools-516020.zip
• 513655-i210-i211-production-images.zip

and the command

./eeupdate64e /NIC=1 /FILE=../../../images/Flash\ Images/iNVM\ \&\ I211/I210_Invm_Copper_NoAPM_v0.6.txt /MAC=0a0000000002 /INVMUPDATE

That seemed to work fine, the chips are now showing up in linux and use the igb driver. However, as soon as an actual ethernet connection is made, dmesg throws errors like this:

[ 35.810158] igb 0000:04:00.0 ethsensor0: Reset adapter [ 35.811943] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ 35.826200] igb 0000:01:00.0 ethsensor2: Reset adapter [ 35.885568] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Down [ 38.286657] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Up 10 Mbps Half Duplex, Flow Control: None [ 38.538215] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ 164.974006] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <ffff8308> next_to_watch <ffff880072817010> jiffies <ffff85be> desc.status <420200> [ 166.820026] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <ffff8308> next_to_watch <ffff880072817010> jiffies <ffff87b2> desc.status <420200> [ 168.666628] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <ffff8308> next_to_watch <ffff880072817010> jiffies <ffff89a6> desc.status <420200> [ 168.673318] igb 0000:01:00.0 ethsensor2: Reset adapter [ 168.673390] igb 0000:04:00.0 ethsensor0: Reset adapter [ 171.048492] asix 1-2.4.1:1.0 eth1: link up, 100Mbps, full-duplex, lpa 0xDDE1 [ 171.162553] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Up 10 Mbps Half Duplex, Flow Control: None [ 171.443100] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ 324.657439] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100002c82> next_to_watch <ffff880072817010> jiffies <100002eaf> desc.status <420200> [ 326.496691] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100002c82> next_to_watch <ffff880072817010> jiffies <1000030a1> desc.status <420200> [ 328.342821] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100002c82> next_to_watch <ffff880072817010> jiffies <100003295> desc.status <420200> [ 328.353373] igb 0000:01:00.0 ethsensor2: Reset adapter [ 328.353452] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Up 10 Mbps Half Duplex, Flow Control: None [ 328.353820] igb 0000:04:00.0 ethsensor0: Reset adapter [ 328.424125] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Down [ 331.004587] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Up 10 Mbps Half Duplex, Flow Control: None [ 331.219324] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ 547.117378] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100011832> next_to_watch <ffff880072817010> jiffies <100011a0a> desc.status <420200> [ 548.963416] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100011832> next_to_watch <ffff880072817010> jiffies <100011bfe> desc.status <420200> [ 550.809687] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100011832> next_to_watch <ffff880072817010> jiffies <100011df2> desc.status <420200> [ 552.655566] igb 0000:04:00.0: Detected Tx Unit Hang Tx Queue <2> TDH <0> TDT <2> next_to_use <2> next_to_clean <0> buffer_info[next_to_clean] time_stamp <100011832> next_to_watch <ffff880072817010> jiffies <100011fe6> desc.status <420200> [ 552.662230] igb 0000:01:00.0 ethsensor2: Reset adapter [ 552.662601] igb 0000:04:00.0 ethsensor0: Reset adapter [ 555.162521] igb 0000:01:00.0 ethsensor2: igb: ethsensor2 NIC Link is Up 10 Mbps Half Duplex, Flow Control: None [ 555.443050] igb 0000:04:00.0 ethsensor0: igb: ethsensor0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

The two i211 chips are named ethsensor0 and ethsensor2 in linux.

What we tried:

• verify voltages (1.5 and 0.9 volts)
• make sure design matches the i211 reference design (RSVD7_PD is pulled up!)
• make sure RSVD7_PD is pulled to ground via 10k resistor. Doesn't change anything.
• make sure RSVD7_PD is pulled to ground directly. Makes the chip disappear from PCI.
• ethtool -K ${DEVICE} gro off gso off tso off sg off tx off

Questions:

• We plan on using SDP0...3 for time synchronization, and have them hooked up to pulse generators that aren't currently active. What is a safe state for these pins while debugging above problem. Can we just leave them floating? Pull HI? LO?
• Can we submit the schematics for review? How?
• What else could cause these problems?

Thanks for your help!

[Caguicla_Intel]

Hello GOman1, Thank you for posting in Intel Ethernet Communities. Your query will be best answered by our Embedded Support team, we will help you to move this post to the designated team. Please feel free to contact us if you need assistance form Ethernet support team. Best regards, Crisselle C Intel Customer Support Under Contract to Intel Corporation

[CarlosAM_INTEL]

​Hello, @GOman1​:

Thank you for contacting Intel Embedded Community.

In order to review your design, feel free to use the Design Review Service following the procedure stated in the following website:

https://edc.intel.com/Tools/Design-Review/Default.aspx?language=en&r=1016014051

Best regards,

@Mæcenas_INTEL​.

[GOman1]

Hi Maecenas,

thank you, I am submitting the schematics for review.

However, the schematics match your reference schematics quite closely. As mentioned above, we have also tried different variants, which didn't help either.

How can we debug this from the software side? I installed kernel 4.20 and enabled the igb debug=16 option. Attached is an output of dmesg with a lot of text.

Can you please have an engineer look over this and give us some ideas?

Did I use the right file with eeupdate? Should I have made some changes to the customer-changeable default values?

Thanks!

[CarlosAM_INTEL]

​Hello, @GOman1​:

Thanks for your update.

Could you please verify that the driver is fulfilling its propose? It is detecting a hang and reset the part.

Waiting for your answer.

Best regards,

@Mæcenas_INTEL​.