Re: x552/x557 to HPE OfficeConnect 1950 24G 2SFP+ 2XGT Switch(JG960A)

진JinS · ‎08-23-2016

Hi, I connected server 10GBase-T port to switch 10GBase-T port with Cat.7 STP cable(length 1m).

Two interfaces are up at 1Gbps speed not at 10Gbps.

= Motherboard: Supermicro X10SDV-4C-TLN4F

- CPU: Intel Xeon D-1518

- LAN: 2x 1GbE via i350-AM2

2x 10GbE via SoC(x552/x557 AT?)

https://www.supermicro.com.tw/products/motherboard/Xeon/D/X10SDV-4C-TLN4F.cfm Supermicro | Products | Motherboards | Xeon® Boards | X10SDV-4C-TLN4F

- OS: FreeBSD 11.0-RC1, CentOS x86_64 1511

- Driver: ix-3.1.13k(FreeBSD), ix-3.1.14(CentOS)

I tried ix-3.1.14 driver installation under FreeBSD but failed to compile.

= Switch: HPE OfficeConnect 1950 24G 2SFP+ 2XGT Switch(JG960A)

- 24x 1GbE port(1-24)

- 2x 10GbE port(27, 28)

http://www8.hp.com/us/en/products/networking-switches/product-detail.html?oid=6887535 http://www8.hp.com/us/en/products/networking-switches/product-detail.html?oid=6887535

After connecting two 10GbE port of server mother board with Cat.7 STP cable, both inferfaces are up at 10Gbps.

Also connecting two switch 10GbE port with same cable, both interfaces are up at 10Gbps.

But connecting server port and switch port, all are up at 1Gbps.

It seems auto-negotiation not work well.

idata · ‎08-24-2016

Hi JinSooSuh,

Thank you for the post. Have you tried using the Intel Linux-based driver, we do not have driver for FreeBSD and Centos, but only Linux based driver below

https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Ethernet-Network-Connections-under-Linux-

Are the drivers used came from Supermicro? If not, have you also check with Supermicro* for customized driver as these are onboard NICs embedded on their system which is part of the system. Changes might have made on the ethernet controller during integration which is beyond our control. The system vendor are in better position provide the driver.

rgds

wb

진JinS · ‎08-24-2016

I tried ixgbe-4.3.15 from Intel.

I contacted Supermicro technical support and waiting for response.

진JinS · ‎08-25-2016

From Supermicro tech. support:

The X10SDV-TLN4F's 10GbE controller is part of the SoC (D-1540/D-1541).

It is categorized as X552/X557 ixgbe device under Linux and driver support is limited.

Source code can be found on Sourceforge from the following link:

https://sourceforge.net/projects/e1000/files/ixgbe%20stable/

Under CentOS-7.2: ethtool was used to set link speed 10Gbps.

# ethtool -i eno3

driver: ixgbe

version: 4.4.6

firmware-version: 0x800003e7

bus-info: 0000:03:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

# ethtool -s eno3 advertise 0x1000

# ethtool eno3

Settings for eno3:

Supported ports: [ TP ]

Supported link modes: 1000baseT/Full

10000baseT/Full

Supported pause frame use: No

Supports auto-negotiation: Yes

Advertised link modes: 10000baseT/Full

Advertised pause frame use: Symmetric

Advertised auto-negotiation: Yes

Speed: 10000Mb/s

Duplex: Full

Port: Twisted Pair

PHYAD: 0

Transceiver: external

Auto-negotiation: on

MDI-X: Unknown

Supports Wake-on: umbg

Wake-on: g

Current message level: 0x00000007 (7)

drv probe link

Link detected: yes

Rebooting or "# ethtool -s eno3 advertise 0x1020" make link speed 1Gbps.

I installed CentOS for testing purpose and will use FreeBSD as production service.

ixgbe driver 3.1.13-k version of FreeBSD-11.0-RC2 and Intel provided ix-3.1.14 under FreeBSD-10.3 do not auto-negotiated to 10Gbps.

Intel provided ix-3.1.14 under FreeBSD-11.0-RC2 make load failed:

cc -O2 -pipe -DSMP -DINET -DINET6 -DIXGBE_STANDALONE_BUILD -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I. -I/usr/src/sys -fno-common -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -MD -MF.depen

d.if_ix.o -MTif_ix.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmiss

ing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__

-Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -mno-aes -mno-avx -std=iso9899:1999 -c if_ix.c -o if_ix.o

if_ix.c:4393:3: error: incompatible pointer types initializing 'int *' with an expression of type 'uint64_t *' (aka 'unsigned long *') [-Werror,-Wincompatible-pointer-types]

SYSCTL_ADD_INT(ctx, queue_list, OID_AUTO, "lro_queued",

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/usr/src/sys/sys/sysctl.h:523:7: note: expanded from macro 'SYSCTL_ADD_INT'

int *__ptr = (ptr); \

^ ~~~~~

if_ix.c:4396:3: error: incompatible pointer types initializing 'int *' with an expression of type 'uint64_t *' (aka 'unsigned long *') [-Werror,-Wincompatible-pointer-types]

SYSCTL_ADD_INT(ctx, queue_list, OID_AUTO, "lro_flushed",

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/usr/src/sys/sys/sysctl.h:523:7: note: expanded from macro 'SYSCTL_ADD_INT'

int *__ptr = (ptr); \

^ ~~~~~

2 errors generated.

*** Error code 1

Stop.

make: stopped in /root/ix-3.1.14/src

Is there any tool in FreeBSD like linux ethtool?

# ifconfig ix0 media 10000baseT

ifconfig: unknown media subtype: 10000baseT

Please help me.

idata · ‎08-25-2016

Hi Jinsoo.suh

Thank you for the info. I don;t have the information about Freebsd as this is third party OS. I managed to search on web and found information below, you may want to check if this is applicable to you (please note this is a third party website for your reference only, we are not endorising this web content and have on control over the suggestion given).

http://www.cyberciti.biz/faq/howto-configure-freebsd-full-half-duplex-speed/

You may want to check with Freebsd support if there is equivalent ethtool.

thanks,

wb

진JinS · ‎08-25-2016

Intel provide ixgbe driver for FreeBSD.

https://downloadcenter.intel.com/download/14688/Network-Adapter-Driver-for-PCI-E-10-Gigabit-Network-Connections-under-FreeBSD- Download Network Adapter Driver for PCI-E* 10 Gigabit Network Connections under FreeBSD*

# ifconfig ix0 media 10000baseT

ifconfig: unknown media subtype: 10000baseT

What is media subtype for 10Gbps?

idata · ‎08-25-2016

Hi Jinsoosuh,

I will check on this, but have you tried also the driver provided by Supermicro* on the sourceforge site?

https://sourceforge.net/projects/e1000/files/ixgbe%2520stable/ https://sourceforge.net/projects/e1000/files/ixgbe%20stable/

As mentioned the driver support is limited.

rgds,

wb

진JinS · ‎08-25-2016

Hi wb,

The driver on the sourceforge site is for Linux not for FreeBSD.

idata · ‎08-28-2016

HI Jinsoosuh,

Thank you for the information. The Freebsd 3.1.14 you downloaded from Intel website does not support X552/557 series.

https://downloadcenter.intel.com/download/14688/Network-Adapter-Driver-for-PCI-E-10-Gigabit-Network-Connections-under-FreeBSD-?product=59787 Download Network Adapter Driver for PCI-E* 10 Gigabit Network Connections under FreeBSD*

rgds,

wb

idata · ‎09-06-2016

Hi JinSooSuh,

Good day. Please try to connect the NIC to other switch brand to isolate the issue.

Do let me know the test result.

Thanks,

wb

진JinS · ‎09-07-2016

Dear wb,

The Supermicro tech. support team informed me that they tested on their https://www.supermicro.com.tw/products/accessories/Networking/SSE-X3348S.cfm SSE-X3348SR switch.

I will try your suggestion. But at this moment, I couldn't locate other switch brand.

idata · ‎09-07-2016

Hi JinSoo.Suh,

Thank you for the update. Please do let me know the test results of testing on other switch when you or Supermicros had done the test.

Thanks,

wb

진JinS · ‎09-08-2016

Dear wb,

In my Linux case with Supermicro X10SDV-4C-TLN4F and HPE 1950 24G 2SFP+ 2XGT Switch,

following two methods make both intefaces are up with 10Gbps.

= Increase the interface MTU value to 9216(HP 1950 series default and maximum value)

# ifconfig eno3 mtu 9216

# ifconfig eno4 mtu 9216

= Set the interface to advertise only 10000baseT Full in autonegotiation.

# ethtool -s eno3 advertise 0x1000

# ethtool -s eno4 advertise 0x1000

But in FreeBSD, increasing only mtu values did not make intefaces are up with 10Gbps.

Do you Intel have plan for updating FreeBSD driver for X552/X557?

idata · ‎09-08-2016

Hi JinSooSub,

Thank you for the update. I do not have information about the driver yet. Let me check for you.

rgds,

wb

진JinS · ‎09-12-2016

Dear wb,

Extreme Summit X670V-48t was temporarily available and I connected it to X10SDV-4C-TLN4F.

They autonegotiated to 10Gbps as expected.

idata · ‎09-12-2016

Hi JinSoo.Suh,

Thank you for sharing the information. Good to know it work on the Extreme Summit X670V switch

rgds,

wb

진JinS · ‎09-21-2016

Hi wb,

I also tested Netgear models.

Here are test results.

Model: port used, link speed

HPE http://www8.hp.com/us/en/products/networking-switches/product-detail.html?oid=6887535 1950 24G 2XFP+ 2XGT (JG960A): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/smart/S3300-28X.aspx# tab-techspecs S3300-28X (GS728TX): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/smart/S3300-52X.aspx# tab-techspecs S3300-52X (GS752TX): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/managed/M4300-28G.aspx# tab-techspecs M4300-28G (GSM4228S): 10GBase-T uplink port, 10Gbps

Extreme Networks Summit http://www.extremenetworks.com/product/x670-series/ X670V-48t: 10GBase-T port, 10Gbps

Netgear M4300 series are more expensive than Netgear S3300 series and HPE 1950 series.

It seems Intel 10Gbase-T NICs are not compatible to low price switch models.

rgds,

ssoo

idata · ‎09-22-2016

Hi Jin soo. Suh,

Thank you very much for taking time to test and sharing the test result. The is no issue when connecting the X550 to Netgear M4300 or Netgear XS712T. We will further check on this.

thanks,

wb

idata · ‎10-03-2016

Hi JinSoo.Suh,

Good day. X550/X552 does supports Freebsd driver version 3.1.14 and is available at

https://downloadcenter.intel.com/download/14688/Network-Adapter-Driver-for-PCI-E-10-Gigabit-Network-Connections-under-FreeBSD-

However, the network adapter is still not listed on the website.

Thanks,

wb

진JinS · ‎10-05-2016

Hi wb,

I omitted OS & Driver version information from the test results.

This is not a OS or driver problem but hardware problem.

= Motherboard: Supermicro https://www.supermicro.com.tw/products/motherboard/Xeon/D/X10SDV-4C-TLN4F.cfm X10SDV-4C-TNL4F

- CPU: Intel Xeon D-1518

- LAN: 2x 1GBase-T via Intel i350-AM2, 2x 10GBase-T via SoC(Xeon D-1518)

= OS Version, Driver Version

CentOS 7.2, ixgbe-4.3.15

FreeBSD 10.3, ix-3.1.14

FreeBSD 11.0-RC3, ix-3.1.13-k

= Switch Model: port used, link speed

HPE http://www8.hp.com/us/en/products/networking-switches/product-detail.html%3Foid%3D6887535 1950 24G 2XFP+ 2XGT (JG960A): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/smart/S3300-28X.aspx%23tab-techspecs S3300-28X (GS728TX): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/smart/S3300-52X.aspx%23tab-techspecs S3300-52X (GS752TX): 10GBase-T uplink port, sometimes 1Gbps, sometime 10Gbps, mostly 1Gbps, rebooting or cable unplug/plug

Netgear http://www.netgear.com/business/products/switches/managed/M4300-28G.aspx%23tab-techspecs M4300-28G (GSM4228S): 10GBase-T uplink port, 10Gbps

Extreme Networks Summit http://www.extremenetworks.com/product/x670-series/ X670V-48t: 10GBase-T port, 10Gbps

Thanks,

ssoo

idata · ‎10-05-2016

Hi JinSoo.Suh,

Thank you for the additional test done. Have you tried using back-to-back connection to test the speed of our Intel adapter adapter just to isolate if this might be compatibility issue on the switch.

rgds,

wb