Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
103 Views

Monitoring tools on Windows for Intel i350 to find cause for event 27

Dear Community,

I'm looking for help concerning tools which might be useful to track down an issue we've been having with multiple i350 T4 v2 network cards on a single mainboard.

The affected ports are connected to GigE cameras (utilizing about 40 % of the bandwidth) and run without a problem for a few hours or even days before we are seeing simultaneous disconnects on all ports with event ID 27 (link disconnected). I have read that it might be an issue with the cables used, but it seems to systematic that the problem is with the cables. We have four cards in total built in and the card, without any cameras connected to it, is not affected by these simultaneous disconnects.

Our system is based on Windows 7 (I know it's not support, but never fix a running system) and I would like to find the cause of the disconnects. We have tested this with the current and older drivers. We have disabled EEE and other power saving options in Windows without any effect. We have also tried disabling interrupt throttling to no avail.

There seems to be something 'overflowing' or 'peaking' at some point and I would want to monitor some of the lower level values/states. As far as I can judge the Window Performance Monitor partly gathers useful information, but not all affected (internal) values.

Can anybody recommend tools/utilities which might be useful to continuously monitor the ports/cards to see correlation between the values and the disconnects?

Thank you in advance!

0 Kudos
10 Replies
Highlighted
93 Views

Hello CVBln,


Thank you for posting in Intel Ethernet Communities. 


We understand that you are looking for monitoring tools for Intel Ethernet adapter and we will be glad checking this for you. Before we proceed with that,kindly provide the following information that would help us to further check your request.

1.) You mentioned that you have tested this with the current and older drivers, can you please share the driver version tested and where was the driver downloaded from? 

2.) Please share the PBA and serial number of the adapter. You may refer to the link below on where to find the PBA number. Providing photos of the adapter focusing on the markings (white sticker) found on the physical card will be highly appreciated for us to double check on it. The PBA consists 6-3 digit number located at the last part of the serial number.

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products...

3.) Is there a specific application running when the error occurred and how often does the issue occurs?

4.) Can you share where was Event ID 27 seen? 

5.) You mentioned 'We have four cards in total built in and the card, without any cameras connected to it, is not affected by these simultaneous disconnects.

Can you share the model of the other cards? Have you tried connecting your GigE cameras to other cards and check if same issue persists? 

6.) Looking at the Supported Operating Systems for Retail Intel® Ethernet Adapters (link below), I can confirm that Windows 7 is supported on an Intel® Ethernet Server Adapter I350-T4V2

https://www.intel.com/content/www/us/en/support/articles/000025890/network-and-i-o/ethernet-products...

Can you share more information about your statement 'Our system is based on Windows 7 (I know it's not support, but never fix a running system)?


Looking forward to your reply.


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Highlighted
Beginner
88 Views

Hello,

thank you for your quick response!

1.) We downloaded and tried at least the versions 22.1 and 25.0 from the Intel download page. We also received and tested packages with two versions (22.2 and 25.0) from the motherboard manufacturer (SuperMicro). I do not have the exact versions, but we also tried versions between 22.1 and 25.0 without progress.

2.) Since the systems are off-site I'll get this information to you a.s.a.p.

3.) Our own software (which acquires images from the cameras, etc.) is running when this error occurs, but I have seen the resets occuring without our software being started. The software has previously been stable without problems with the i350 cards for multiple years.

4.) Yes, sorry that I was not explicit. We are seeing them in the Windows Event Viewer, which contains the following message (e.g.).

 

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="e1rexpress" /> 
<EventID Qualifiers="40964">27</EventID> 
<Level>3</Level> 
<Task>0</Task> 
<Keywords>0x80000000000000</Keywords> 
<TimeCreated SystemTime="2020-08-27T16:23:28.764684000Z" /> 
<EventRecordID>44216</EventRecordID> 
<Channel>System</Channel> 
<Computer>H-350</Computer> 
<Security /> 
</System>
<EventData>
<Data /> 
<Data>Intel(R) I350 Gigabit Network Connection #11</Data> 
<Binary>0000040002003000000000001B0004A00000000000000000000000000000000000000000000000001B0004A0</Binary> 
</EventData>
</Event>

 

5.) All four cards are the same with the i350 controller. It's the fourth card which is not connected to any cameras and as such does not have a high resource demand.

6.) Summarized: I opened an issue with Intel support first and I got a reply that due to Windows 7 running the legacy drivers and that the legacy driver will not be further updated, that they cannot help me. Furthermore, since supposedly Intel does not test the network cards/controllers with other manufacturers' mainboards, they cannot guarantee compatibility. That is why I am trying to track down the issue myself, because it's possible even if we upgrade to Windows 10, that we might face the same problem(s).

Thank you and kind regards!

0 Kudos
Highlighted
66 Views

Hello CVBln,


Thank you for the update and we will check these details.


And also, when can we expect the markings that requested for us to check the cards?


If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days.


Thank you.


Best regards,

Michael L.

Intel Customer Support Technicians


0 Kudos
Highlighted
51 Views

Hello CVBln,


Good day!


We'd like to follow up the PBA and serial number of the adapter for us to continue checking on your request.


We also need some clarifications on item 3 and 6, we hope you don't mind giving us additional information.

Item 3

Our own software (which acquires images from the cameras, etc.) is running when this error occurs, but I have seen the resets occuring without our software being started. The software has previously been stable without problems with the i350 cards for multiple years.


Is there any software or hardware changes made prior to the issue?

 

Item 6

Summarized: I opened an issue with Intel support first and I got a reply that due to Windows 7 running the legacy drivers and that the legacy driver will not be further updated, that they cannot help me. Furthermore, since supposedly Intel does not test the network cards/controllers with other manufacturers' mainboards, they cannot guarantee compatibility. That is why I am trying to track down the issue myself, because it's possible even if we upgrade to Windows 10, that we might face the same problem(s).


Can you share the link or ticket number you had with us? This is for our reference only and we will also try to check on it. 


Awaiting to your reply


Should there be no reply, we will follow up after 3 business days.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Highlighted
Beginner
42 Views

Hello,

sorry, that it took so long, but I received the information just today. Some of the affected network cards:

Serialnr. 1:  H47822-001

Serialnr. 2:  H47822-001

Serialnr. 3: H47822-001

 

3.) We did indeed also have a software change, but have tested it with both the new and old software versions and both exhibit errors only on the new hardware. There was no other hardware change before these issues arose.

6.) The support request number is 04789822.

 

Can you recommend any software tools to track down the cause of the problems?

0 Kudos
Highlighted
35 Views

Hello CVBln,

 

No problem at all, we appreciate your effort in sending the requested information.

 

Just to clarify, 'We did indeed also have a software change, but have tested it with both the new and old software versions and both exhibit errors only on the new hardware. There was no other hardware change before these issues arose.' Does this mean that the issue only occurs to these 3 new hardware? 

 

Can you share how many adapters are working without any issues and provide the PBA as well? 

 

I'm afraid to say that I didn't find any Intel monitoring software tools for Intel Ethernet adapter. But don't worry as I will check this with our engineers to further investigate the issue and confirm with them if there is a tool they can recommend.

 

Looking forward to hear from you.

 

We will follow up after 3 business days in case we don't receive a reply. 

 

Best regards,

Crisselle C

Intel® Customer Support

 

0 Kudos
Highlighted
Beginner
32 Views

Hello Crisselle C,

thank you for your quick reply!

There are more affected adapters. We've swapped PCs and at least two other systems with the same hardware (i.e. new mainboard and i350 T4v2 cards) exhibited the same errors.

In contrast there are hundreds of the previous i350 T4 and also i350 T4v2 adapters in other systems, which run without a problem on the previous mainboard.

Since these systems are off-site and a technician would need to remove the PCs from enclosures and open the PCs, it is difficult to get the information on all adapters. Furthermore, active systems cannot be taken offline without scheduled maintenance.

Thank you for your effort!

Kind regards,

CVBln

0 Kudos
Highlighted
23 Views

Hello CVBln,


Thank you for the reply. 


We fully understand that the information requested is difficult to get. We highly appreciate your time and effort on this matter.


Please allow us to continue checking on your request. We will get back to you as soon as possible but no later than 2-3 business days.


Hoping for your kind patience.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Highlighted
13 Views

Hello CVBln,


Good day!


While we are checking on this, will you be able to provide System Support Utility from your system? This will allow us to check your Adapter details and configuration. Kindly refer to the steps below.

a- Go to https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91...

b- Open SSU.exe

c- Mark the box "Everything" and then click "Scan".

d- When finished scanning, click "Next".

e- Click on "Save" and attach the file to a post.


By the way, please be informed that we removed the Serial Numbers provided but left the PBA on this thread as this is considered confidential information. 


Looking forward to your reply.


Should there be no response, I’ll make sure to reach out after 3 business days.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Highlighted
Beginner
8 Views

Hello Crisselle,

I will receive an affected system in the next few days and can provide the PBAs (for now I only have the MAC addresses, which are probably part of the confidential information).

When I have the system I'll run SSU and post the saved information.

Kind regards,

CVBln

0 Kudos