- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi!
I am trying to use one of the dual port XL710 adapters to connect with a 10Gb port. For that, I am trying to use an adapter from Mellanox: MAM1Q00A-QSA Cable Module ETH 10GbE 40Gb/s to 10Gb/s QSFP to SFP+. Unfortunately, just plugging it into XL710 and connecting the resulting SFP+ does not lead to an established link. So, I wonder, whether XL710 supports such configuration? My setup:
OS: Ubuntu 14.04
lspci -v:
83:00.0 Ethernet controller: Intel Corporation Ethernet Controller LX710 for 40GbE QSFP+ (rev 01)
Subsystem: Super Micro Computer Inc Device 0000
Physical Slot: 6
ethtool -i eth4
driver: i40e
version: 1.2.37
firmware-version: f4.22.26225 a1.2 n4.24 e12f3
ethtool eth4
Settings for eth4:
Supported ports: [ ]
Supported link modes: 40000baseCR4/Full
40000baseSR4/Full
40000baseLR4/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Advertised link modes: 40000baseCR4/Full
40000baseSR4/Full
40000baseLR4/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: Unknown!
Duplex: Unknown! (255)
Port: Other
PHYAD: 0
Transceiver: external
Auto-negotiation: off
Supports Wake-on: g
Wake-on: d
Current message level: 0x0000000f (15)
drv probe link timer
Link detected: no
Thank you
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
We will check on this.
thanks,
wb
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
The Mellanox MAM1Q00A-QSA Cable is not the tested cable. You may refer to the list of supported cable at http://www.intel.ph/content/dam/www/public/us/en/documents/product-briefs/ethernet-xl710-brief.pdf http://www.intel.ph/content/dam/www/public/us/en/documents/product-briefs/ethernet-xl710-brief.pdf
Further just to double check based on your information, are you connecting one port of XL710 to another "network adapter" instead of a switch?
Thanks,
wb
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi WB:
True, this adapter is not listed (yet?). In the end, I would need to connect it to Intel 10GbE card (can't tell you exact model this time). However, the link was not established neither with the card nor 10GbE switch.
It seems to me that I miss something very simple, such as autonegotiation of the speed of the link. How can I enable 10000baseXXXX link mode on a port of XL710 card in Linux? As you could see above in ethtool printout, this mode is not supported.
Best,
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
Thanks for the clarification. Let me check on this.
rgds,
wb
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi WB:
since it's probably not easy to get your hands on the adapter, let me ask a simpler question:
* does XL710 support 10Gb data rates? It seems that it does support it according to the "Data Brief" of the adapter (
Data rate supported per port: • Optical: 10 GbE/40 GbE, • Direct Attach: 40 GbE).
If it does support 10Gb, how do I enable it in Linux?
Best,
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
You may follow the instruction in the guide below to enable the adapter to configure Port Speed for XL710.
http://www.intel.com/content/dam/www/public/us/en/documents/guides/qsfp-configuration-utility-quick-usage-guide.pdf http://www.intel.com/content/dam/www/public/us/en/documents/guides/qsfp-configuration-utility-quick-usage-guide.pdf
Should see 4 x 10Gb devices with driver loaded
Hope this helps.
rgds,
wb
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi WB:
thank you for instructions. However, I do have a problem. Namely, it seems that QCU is not working for us as it should (see below). In addition, in our server, the official name of the card is Supermicro AOC-S40G-i2Q which uses XL710 controller. I wounder if that the cause for the problems.
Here is mu session with QCU:
# ./qcu64e
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
NIC B:D Ven-Dev Mode Adapter Name
=== ===== ========= ====== ====================================================
1) 83:00 8086-1583 N/A Intel(R) Ethernet Converged Network Adapter XL710-Q2
Warning: No adapter selected.
# ./qcu64e /nic=1 /info
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
QSFP+ Configuration modification is not supported by this adapter.
# echo $?
5
# ./qcu64e /exitcodes
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
Exit Codes:
0x0 := Requested action executed successfully.
0x1 := Insufficient privileges to run the utility.
0x2 := Invalid adapter selected.
0x3 := Bad command line parameter.
0x4 := Unsupported link mode selected.
0x5 := Adapter does not support link mode selection.
0x6 := No adapter selected.
0x7 := Memory allocation error.
0x8 := No base driver available.
0x9 := Unsupported base driver version detected.
0xA := QSFP+ configuration for selected adapter not supported by this utility version.
Here are the driver/firmware info:
ethtool -i eth4
driver: i40e
version: 1.2.37
firmware-version: f4.22.26225 a1.2 n4.24 e12f3
Best,
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@M_V
For the QCU utility to work correctly, you need to update the firmware and have the latest Linux driver (v1.2.37) loaded. It looks like you have the latest Linux Driver.
https://downloadcenter.intel.com/download/24769/NVM-Update-Utility-for-Intel-Ethernet-Converged-Network-Adapter-XL710-X710-Series NVM-Update-Utility-for-Intel-Ethernet-Converged-Network-Adapter-XL710-X710-Series
Be sure to reboot after running NVM Update before running QCU for it to work properly.
My XL710 Card:
driver: i40e
firmware-version: f4.33.31377 a1.2 n4.85 e1930
- David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi David:
strange, I get from NVM Update utility (run in linux) that firmware is up-to-date:
# ./nvmupdate64e
Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.24.33.08
Copyright (C) 2013 - 2015 Intel Corporation.
WARNING: TO AVOID DAMAGE TO YOUR DEVICE, DO NOT EXIT OR REBOOT OR POWER OFF THE SYSTEM DURING THIS UPDATE
Inventory in progress. Please wait [.......-**]
Num Description Device-Id B:D Adapter Status
=== ====================================== ========= ===== ====================
01) Intel(R) 82599 10 Gigabit Network Conn 8086-1557 03:00 Update not available
02) Intel(R) I350 Gigabit Network Connecti 8086-1521 04:00 Update not available
03) Intel(R) Ethernet Converged Network Ad 8086-1583 131:00 Up to date
Tool execution completed with the following status: All operations completed successfully
ethtool -i eth3
driver: i40e
version: 1.2.37
firmware-version: f4.33.31377 a1.2 n4.42 e18a3
Do I have to unplug the server from mains? Or is the reboot sufficient?
Best,
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@M_V,
Looking at your post above:
ethtool -i eth4
driver: i40e
version: 1.2.37
firmware-version: f4.22.26225 a1.2 n4.24 e12f3
And your current post:
ethtool -i eth3
driver: i40e
version: 1.2.37
firmware-version: f4.33.31377 a1.2 n4.42 e18a3
Shows the Firmware to have been updated between each post.
I upgraded the firmware on (8) XL710 cards without powering off, a reboot was sufficient. Same with running the QCU utility.
- David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I guess I haven't rebooted after the update. However, even now, I still get 'no support' message from QCU. I do wonder if its due to the fact that its Supermicro-branded version?
# ./qcu64e /nic=1 /info
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
QSFP+ Configuration modification is not supported by this adapter.
# echo $?
5
# ethtool -i eth4
driver: i40e
version: 1.2.37
firmware-version: f4.33.31377 a1.2 n4.42 e18a3
bus-info: 0000:83:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What does './qcu64e' show after the reboot? You're looking for something in the Mode to look like (1x40 or 4x10, etc).
Your Card:
NIC B:D Ven-Dev Mode Adapter Name
=== ===== ========= ====== ====================================================
1) 83:00 8086-1583 N/A Intel(R) Ethernet Converged Network Adapter XL710-Q2
My Card:
NIC B:D Ven-Dev Mode Adapter Name
=== ===== ========= ====== ====================================================
1) 03:00 8086-1584 4x10 Intel(R) Ethernet Converged Network Adapter XL710-Q1
Not sure about the Supermicro OEM vs Intel Branded.
- David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It still shows N/A:
# ./qcu64e
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
NIC B:D Ven-Dev Mode Adapter Name
=== ===== ========= ====== ====================================================
1) 83:00 8086-1583 N/A Intel(R) Ethernet Converged Network Adapter XL710-Q2
I'll try to power-circle it on Monday. Thank you very much for help!
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately, no change after power-circle. QCU and ethtool show the same info:
ethtool -i eth3
driver: i40e
version: 1.2.37
firmware-version: f4.33.31377 a1.2 n4.42 e18a3
./qcu64e /nic=1 /info
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
QSFP+ Configuration modification is not supported by this adapter.
./qcu64e
Intel(R) QSFP+ Configuration Utility
QCU version: v1.24.35.01
Copyright(C) 2015 by Intel Corporation.
Software released under Intel Proprietary License.
NIC B:D Ven-Dev Mode Adapter Name
=== ===== ========= ====== ====================================================
1) 83:00 8086-1583 N/A Intel(R) Ethernet Converged Network Adapter XL710-Q2
Warning: No adapter selected.
Maybe we managed to get an adapter that does not support 4 x 10Gb?
Best
M_V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
Thank you for taking time to further test. let me further check on this.
rgds,
wb
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi M_V,
Can you please check with SuperMicro if there is similar utility to configure the ports?
Thanks,
wb
![](/skins/images/7B0AB6865064EAE32B30EC9A4E94B48A/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page