Very different ISCSI performance on Windows 2008R2 and Linux (Redhat & Debian)

[GRein2]

Hi,

for some time now, we try to figure out what might be the problem/difference between Windows Server 2008R2 and Redhat & Debian regarding the ISCSI speed we notice.

The Server ist a dual core intel Core 2with 2.4 GHz, 4GB Memory, an 8x PCIe slot, an Intel x520-D and a Qsan P600Q-D316 10GbE ISCSI Raid.

We tested the setup with two different Intel x520-D and different twinaxial sfp+ cables from Intel. The support of Qsan was helping and doing some tests, so did the distributor we got the storage from.

From that we can be 99% sure, the hardware is not faulty.

What did we do and check:

We installed Windows Server 2008R2, redhat and debian for each test on a local server disk, connected to the storage point to point, so no switch is involved. The Server ISCSI Port and the Storage ISCSI Port are in the same VLAN.

On Windows we installed the most recent driver from Intel.

On Redhat we used the default and later the most recent one. V 3.9.15-k and 3.18.7

On debian we just tested the default one. version: 3.6.7-k

Than we run different test like IOmeter, IO zone, dd to the device and dd to the filesystem.

The conclusion after a couple of days and a lot of research in readmies, presentations and tuning guides by redhat, intel, NASA, CERN etc etc is:

• The Windows system performs the best, and that by default: We get read put to 600MB/s and writ 450 MB/s

That is +- that what could be expected from the storage and Raidlevel setup we use. So that is the goal.

• Redhat by default has a retransmit problem, what could be solved by e.g. the tuning guides and we went from aprox. 50 MB/s r/w up to 350 MB/s +-.
• Debian is close to that also after some tuning.

But the big question still is:

Why are we (is linux) still so far behind? As mentioned we use the same hardware, all tests are done one after an other on the same server.

We applied and checked tuning tips from:

http://www.intelcloudbuilders.com/docs/IBC_RA_Unified%20Networking_iSCSI_Linux.pdf http://www.intelcloudbuilders.com/docs/IBC_RA_Unified%20Networking_iSCSI_Linux.pdf

http://www.redhat.com/promo/summit/2008/downloads/pdf/Thursday/Mark_Wagner.pdf http://www.redhat.com/promo/summit/2008/downloads/pdf/Thursday/Mark_Wagner.pdf

http://fasterdata.es.net/host-tuning/linux/ http://fasterdata.es.net/host-tuning/linux/

https://access.redhat.com/site/documentation/de-DE/Red_Hat_Enterprise_Linux/6/html/Power_Management_... 2.5.2. Tuned-adm - Red Hat Customer Portal

We and the storage manufacturer still don't have any clue what might be the problem. May be the iscsi implementation in linux is that worse? But from the intelcloudbuilders it looks like possible.

Any hint, help and suggestion regarding that setup is welcome.

Rergards . Götz