Rapid Storage Technology
Intel® RST, RAID
2047 Discussions

Slow write performance with RAID5 on Z170 and Z270 chipsets

TGołę
Novice
11,575 Views

Hello community, in short story:

I had fully functional RAID1, built with 2 x ST4000DM004 and Vertex 3 60GB SSD as SSD cache. Z170 chipset, 3.815.448MB capacity.

I've migrated to RAID5 to expand RAID array. Now it's 3 x ST4000DM004 without SSD cache, 7.630.891MB capacity. And now problem occured.

 

Write performance is awful, 5-10MB/s during sequentional transfers. Read performance sometimes good, sometimes bad. I generaly have large files on volume.

With SSD cache ON, I had several data loss incidents. So I've disabled it.

With newer version of BIOS and Intel RST, I had several data loss incidents, including FULL-RESTORE-FROM-BACKUP my system NVMe drive (which is not a part of RAID array by the way). So I've rolled back BIOS and fully restored OS.

Currently, only Intel RST read-only cache is enabled. No buffer flush enabled. And I'm running following versions:

Intel RST software 15.8.1.1007 (while latest available is: 16.8.0.1000, or 15.9.x for Win8)

MSI Z170 Gaming M5, BIOS ver. 7977v1G (while latest available is: 7977v1I)

3 x ST4000DM004, all perfectly fine condition (tested separately under Linux, no errors, no performance drops), FW ver: 0001

Windows 8.1 64-bit

 

The problem rises when newer version of RST or/and BIOS is installed. Data loss occurs even on single system disk (this is really worrying), when it's perfecty fine NVMe device. The system hungs randomly.

I belive, this has something to do with timeouts on devices because they start to appear in system log, like: device has been reset, NTFS flush error or something similar. High response times occurs during accessing volume.

 

To figure this out is it my OS or motherboard problem I've tested such config just by installing new, clean Win10 64-bit on Z270 motherboard and moving RAID 5 members around. The degraded write peformance exists. No error logged (NTFS flush, device reset), but high response times exists.

 

There is no such problems when disks work as standalone or RAID1 devices, either Z170 or Z270.

 

I remember from the past that Intel chipsets have no problems on earlier generations with RAID5, but now it is. Why?

 

1 Solution
TGołę
Novice
8,198 Views

After long struggling with the isse it turned to be SMR drives.

ST4000DM004 are SMRs and manufacturer (Seagate) has never mentioned that they're SMRs.

View solution in original post

0 Kudos
34 Replies
TGołę
Novice
6,598 Views

This is how it looks like on Z170:

RST-3x4TB-RAID5-cache-off.pngRST-3x4TB-RAID5-cache-read.pngRST-3x4TB-RAID5-cache-rorw.png

should be similar to:

 

RST-2x4TB-RAID1-cache-off.pngRST-2x4TB-RAID1-SSD-cache.png

 

and Z270 chipsets.

 

Z270-RST-3x4TB-RAID5-cache-ro.pngZ270-RST-3x4TB-RAID5-cache-off.png

Z270-RST-3x4TB-RAID5-cache-rorw.png

 

 

 

 

0 Kudos
Wanner_G_Intel
Moderator
6,598 Views
Hello TGołę, Thank you for joining this Intel Community. First, we would like to review your system configuration. Please attach the following reports: A. Intel® RST report: To save a system report, follow these steps: 1.Open the Intel Rapid Storage Technology user interface (Start > Programs > Intel > Intel® Rapid Storage Technology). 2.Click the Help icon. 3.Click the System Report icon. 4.Click Save. B. Run the Intel® System Support Utility and attach the report to this thread. 1.Download the Intel® System Support Utility and save the application to your system. https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91600 2.Open the application and click Scan to see system and device information. The Intel® System Support Utility defaults to the Summary View on the output screen following the scan. Click the menu where it says summary to change to Detailed View. 3.To save your scan, click Next and click Save. You can save the file to any accessible location on your computer. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TGołę
Novice
6,598 Views

Hello Wanner,

 

there is initialization task ongoing since I started from scratch and rebuild RAID5 array on the same disks.

I think my report will not be accurate at this moment.

 

This is to check if performance increases after that. The RST driver has been updated BEFORE volume has been deleted and recreated. I'm on 15.9.0.1015.

Stripe size is the same - 128kB. Full capacity assigned to single RAID5 volume. It took around 24hrs to get 13% progress.

 

There are 2 additional disks which I'm using as temporary media until whole process is completed.

 

 

I noticed on both controllers Z170 and Z270 strange behaviour that when wolume is accessed latency times increases in Task Manager -> Performance on this volume. I measured 500-3000ms while reading and up to (!!!) 15000-19000ms while writing.

I think this is root cause why data loss occurs and NTFS errors appears. It always happen for RAID5 (even with SSD cache!), never happen for RAID0 or RAID1.

 

ROM versions I've tested:

Z170 has 15.2.0.2754

Z270 has 15.5.0.2875

0 Kudos
TGołę
Novice
6,598 Views

It looks like this.

read1.PNGwrite1.PNGwrite2.PNG

 

 

0 Kudos
Wanner_G_Intel
Moderator
6,598 Views
Hello TGołę, Thank you for your response. We will review this information. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TGołę
Novice
6,598 Views

SSU report regarding RAID volume (censored ver).

Initialization status 25-27% after 48h. Around 6 days left. Way too slow..

 

0 Kudos
Wanner_G_Intel
Moderator
6,598 Views
Hello TGołę, Thank you for attaching this information. We will get back to you soon. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TGołę
Novice
6,598 Views

aligment-ok.PNG

 

 

In the meantime, I've checked aligment of RAID volume via fsutil and msinfo32 tools, quick calc reveals that should be no problem there.

My Initialization task progress is 33% which seems to be around 4 TIMES slower than normal (typicaly 48hrs = 2 days for synchronization of 4TB disks, my is going to be completed in total 7,5-8 DAYS). And this is for IDLE volume (there is no data on it, only empty NTFS partition - I'm waiting until it finishes).

The only strange thing in this issue is that only RAID5 level seems to be affected, I haven't tested RAID10 however it does not parity calculations, I assume will also work normally.

My CPU is more-than-enough to handle such tasks...

 

CALC source:

https://www.techpowerup.com/articles/other/157

 

 Is there any way to disable NCQ only for RAID member via reg keys like HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\iaStorA

 ?

0 Kudos
TGołę
Novice
6,598 Views

Hi again,

 

I'm sorry to tell this, but.. I've stopped volume initialization.

Parity calculations speed for time 7,5 to 8 days for 7630891MB volume is something like 11MB/s to 11,78MB/s, way too slow, when EACH of these disks alone can perform following:

ST4000DM004-alone-ONE.pngST4000DM004-alone-TWO.pngST4000DM004-alone-THREE.png

 

and imagine that RAID1 sync takes only few hours, not DAYS..

 

I don't know why RAID5 on Z170 and Z270 is useless, but IT IS.

I've even checked all BIOS settings if any could affect this, found only Hot-plug feature which was Enabled on one of the ports, so I've Disabled for all, no change at all.

0 Kudos
Amy_C_Intel1
Employee
6,598 Views
The IRST does not allow to disable/enable this feature, since this is hw/sw RAID controller, once you enabled RAID in BIOS this is enabled by default. We have seen initialization cases taking more than a week with 4TB of data. We do not think there is an issue with with your computer. Amy C Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TGołę
Novice
6,598 Views

I've ended up configuring RAID10 with one additional drive, full sync time around 8h (hours, not days!).

Of course, performance is outstanding compared to RAID5.

 

What's I'm trying to tell You that THIS IS NOT NORMAL. I know many RAID5/RAID6 solutions, both SW and HW, I use them widely, this should be not so slow. Any modern CPU can do easily parity calculations like this.

For 3 drive RAID5 array, typical sync time should be between 24h and 48h. And write performance should be better.

 

Clearly there is problem with PARITY CALCULATIONS in there.

 

EDIT: in addition, I've done another RAID5 test on Z270, but under Linux mdadm and only available spare drives, which were 2TB Hitachi 7200 RPMs, rebuild was normal (around 8-10hrs), seq ro/rw speeds also normal (250-300MB/s read, up to 100MB/s seq, and NO slowdowns) so it clearly shows Intel RST issue there.

 

 

 

0 Kudos
Wanner_G_Intel
Moderator
6,598 Views
Hello TGołę, Are you able to post on this thread? Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
Wanner_G_Intel
Moderator
6,599 Views
Hello TGołę, I am writing to let you know that you should be able to post on this thread. Please let us know if you have any further issues. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TKavv
Beginner
6,600 Views

Hi,

 

I have similar problems with my newly built PC and RAID 5.

 

Write speeds are around 35-40MB/s with Windows Cache on and with Write-Back Cache enabled on Intel RST write speeds actually drop to around 25-35MB/s. Array has been initialized. I got 12-15MB/s before initialization, so something happened there. The array holds around 160-180MB/s for the first ~10-12 sec before cache or something slows it down.

 

Changing between different cache modes have no effect on writing speed.

 

I followed these steps when setting up the array: http://www.tomshardware.co.uk/forum/id-2033069/intel-raid-poor-write-performance-fixed-mine.html

 

Before using the on-board Intel controller for RAID, I tried setting the array up via Win 10. At that time I got better write performance, around 50-60MB/s. which, at the time, felt very slow. I really hoped to get better performance after the array had been initialized.

 

Attached detailed system report.

 

EDIT: Also weird is that read speed reaches around 650-700MB/s. So I guess no fault on the disks themselves.

0 Kudos
TKavv
Beginner
6,600 Views

And RST report.

0 Kudos
Wanner_G_Intel
Moderator
6,600 Views
Hello TKavv, We will review the information posted and get back to you soon. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TKavv
Beginner
6,600 Views

Hi,

 

I have the disk reset, performing initialization at the moment. 6% done, so it'll be maybe ready on Sunday for bench-marking.

0 Kudos
Wanner_G_Intel
Moderator
6,600 Views
Hello TKavv, As soon as you have any update, please let us know. Wanner G. Intel Customer Support Technician Under Contract to Intel Corporation
0 Kudos
TKavv
Beginner
6,600 Views

Hi!

 

Initialization done. Speeds remain low.

 

Here's new reports.

0 Kudos
TKavv
Beginner
5,637 Views

Write speeds are now, depending on flushing setting, between 41-48MB/s with CrystalDisk, in real world, 27-37MB/s.

0 Kudos
Reply