Copying processes freezes and continues after a while

[Mike_M]

I have here a new Shuttle SW580R8 with i9-11, 4x Seagate X18 18TB as RAID 5. The PC also has 2x Samsung 990 Pro SSDs. It is equipped with 128GB. It also has an NVIDIA 4060, so the device should have plenty of speed.

Everything works so far, but when copying to the RAID, there is always a drop to 0 kb/s. Then a short pause of 3-5 seconds, after which it continues as if nothing had happened.

This happens regardless of whether I use the HDDs in RAID 5, RAID0 or individually. The stripe set with 128KB or 64KB also makes no difference. Even if I switch the different cache combinations on and off, nothing changes.

Booting in safe mode or with a live Windows 11 from USB does not change anything either.

The HDDs have the latest firmware (SN05) and are new. It also doesn't matter which of the 4 I address, it happens on every single one.

All available Windows updates have been installed. Driver updates via Intel Driver Updater as well.
There is a note in the event log when this occurs:

Protokollname: Microsoft-Windows-Ntfs/Operational
Quelle: Microsoft-Windows-Ntfs
Datum: 02.04.2024 02:24:08
Ereignis-ID: 147
Aufgabenkategorie:Keine
Ebene: Warnung
Schlüsselwörter:Antwortzeit,Statistik
Benutzer: SYSTEM
Computer: SERVER-NEW
Beschreibung:
Das Abschließen eines E/A hat länger als 30000 ms gedauert:

Prozess-ID: 6336
Prozessname: explorer.exe
Dateiname: \Test - Laufwerk X\TEST (Videos, Textfiles, Ordner, 57GB)\Runde 1\C\2\2 (14).mp4
Dateioffset: 157286400
E-/A-Typ: Write: NonPaging, Cached, Sync
E-/A-Größe: 1048576 Bytes
64 Cluster beginnend bei Cluster: 4015307
Wartezeit: 42032 ms

Volumen-ID: {b22bb233-ee9d-11ee-a89b-80ee73f6bd84}
Volumenname: X:
Is-Startvolume:false

Geräte-GUID: {00000000-0000-0000-0000-000000000000}
Gerätehersteller:
Gerätemodell:
Geräterevision:
Geräteseriennummer:
Bustyp: <unbekannt>

Adapterseriennummer:

Ereignis-XML:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Ntfs" Guid="{3ff37a1c-a68d-4d6e-8c9b-f79e8b16c482}" />
<EventID>147</EventID>
<Version>4</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x4001000000200000</Keywords>
<TimeCreated SystemTime="2024-04-02T00:24:08.2434117Z" />
<EventRecordID>1660</EventRecordID>
<Correlation />
<Execution ProcessID="4" ThreadID="10872" />
<Channel>Microsoft-Windows-Ntfs/Operational</Channel>
<Computer>SERVER-NEW</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="VolumeCorrelationId">{b22bb233-ee9d-11ee-a89b-80ee73f6bd84}</Data>
<Data Name="VolumeNameLength">2</Data>
<Data Name="VolumeName">X:</Data>
<Data Name="IsBootVolume">false</Data>
<Data Name="MaxLatencyMs">30000</Data>
<Data Name="ProcessId">6336</Data>
<Data Name="ProcessName">explorer.exe</Data>
<Data Name="FileNameLength">80</Data>
<Data Name="FileName">\Test - Laufwerk X\TEST (Videos, Textfiles, Ordner, 57GB)\Runde 1\C\2\2 (14).mp4</Data>
<Data Name="FileIdHigh">0x0</Data>
<Data Name="FileIdLow">0x10000000345e1</Data>
<Data Name="IoType">5</Data>
<Data Name="IoTypeStr">Write: NonPaging, Cached, Sync</Data>
<Data Name="IoSize">1048576</Data>
<Data Name="FileOffset">157286400</Data>
<Data Name="LatencyMs">42032</Data>
<Data Name="StartingLcn">4015307</Data>
<Data Name="ClustersCount">64</Data>
<Data Name="DeviceGuid">{00000000-0000-0000-0000-000000000000}</Data>
<Data Name="VendorIdLength">0</Data>
<Data Name="VendorId">
</Data>
<Data Name="ProductIdLength">0</Data>
<Data Name="ProductId">
</Data>
<Data Name="ProductRevisionLength">0</Data>
<Data Name="ProductRevision">
</Data>
<Data Name="DeviceSerialNumberLength">0</Data>
<Data Name="DeviceSerialNumber">
</Data>
<Data Name="BusType">0</Data>
<Data Name="AdapterSerialNumberLength">0</Data>
<Data Name="AdapterSerialNumber">
</Data>
</EventData>
</Event>

See attached Video to see the issue.
Does anyone else have any ideas?

[Mike_Intel]

Hello Mike_M,

Thank you for posting in Intel community Forum.

For me to further check this issue. please help provide the following details.

1. When did you built this system?
2. Is it working fine before?
3. When did you start noticing this issue?
4. What was the last change in the system before you noticed the issue?
5. Please help generate the SSU logs of your system, kindly download the utility here:

https://www.intel.com/content/www/us/en/download/18377/intel-system-support-utility-for-windows.html

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hi Michael,
thank you for your answer!

About your questions:
1. Last Week
2. It´s working fine, except this issue.
3. Last Saturday when I started copying my data to the system.
4. As the system has been fresh built, I can't answer that. What I can explain, is what I did allready to check about the reasons:
I have operated the RAID on 0 and on 5. I also connected the drives individually both as RAID and as AHCI. I also booted from the HDD or test booted from a Windows 11 live system from USB. Unfortunately, all experiments came to a standstill.

As I want to build a second identical system of this type, I already have more HDDs of this series here. The same problem occurs with these. A total of 8 drives (all new, 2x4 as RAID 0 and 5 and all 8 individually) have been tested. Always with the same problem. So it is probably not due to a single defective disk.

I was able to find out a few things:

When I copy from drive C (SSD) to X (RAID), it does not occur. Nor from D (SSD) to X. But from X to X always.

And always after a few seconds. In the Resource Manager I can then see that reading is taking place first, then it stops and writing is taking place according to the Resource Manager. However, 0 bytes are read! Then after a few seconds only reading and writing is done at the same time and the stalling stops. It stays like this until the end of the copy process.

I have attached the log files, but please consider the following:

Since I have installed and uninstalled a lot of different drivers in the meantime, the system is no longer very stable. Initially it was very stable and there were virtually no errors in the event log. This is now different. So please do not be surprised that there are many errors. However, these errors should have nothing to do with the problem, as apart from those described above, no other errors have actually occurred. I will probably have to reinstall Windows 11 after further error diagnosis.

But thank you very much for your support!

[Mike_Intel]

Hello Mike_M,

Thank you the quick reply.

Let me ask more questions about the issue.

1. Can you provide the complete brand and model of your motherboard? In SSU, I can only see SW580.
2. What is the chipset of your board?
3. Can you share a screenshot of your RST console and also the version?
4. Since I cannot get the model of your board in SSU, let me ask if you tried updating all the drivers and BIOS of your board?

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hi Michael,

about your questions:
1. It´s a Shuttle XPC SW580R8
here is the specification of it:
https://www.shuttle.eu/de/products/cube/sw580r8/spec

2. I honestly don´t know and I don´t know how to determine it. Somewhere inside the Device Manager there is appearing an Entry named "Intel PEG60", but I am not sure, if that´s a kind of Chipset!?

3. Yes, I attached the needed screenshot.

4. Yes sure. I updated the Bios (most actual version from Shuttle Computers) each and every driver. First round I used just the standard-updates using the windows update and Intel Driver Update. As this didn´t solve the thing, I tried separate tools like "Snappy Driver Origin" and "WinGetui" to find more and different drivers. On my way I also tried drivers like "18.6.1.1016" This way it either did not solve the issue. I also tried to start from Live-ISO another Windows 11 which ended up in same Problem.
Today I deleted the whole system and installed it two times from the scratch again.
1. First I tried Windows 10 Pro. with the same drivers, which I used for Windows 11. Guess what? The Issues where gone!
I guessed that this could happen, because I own another (older) System from Shuttle (called SZ270) which is running on Windows 10 and without this issues as well. However I bought the new one to go further to Windows 11 as 10 is getting old and in future out of Support.
2. Then I was motivated again and gave Windows 11 Pro. a try again. And again with the same drivers. Guess what? The issues are back again... And again only on HDDs connected to Intel RST. I even did not install
So good news is, that I am now sure that it´s not a hardware Problem. It´s kind of Windows 11 Problem and/or belonging hardware setting and/or Driver Issue which is appearing in Windows 11 but not in 10.

So I am finally still don´t have any ideas what I could do now.

Thanks for the support!

Kind regards,
Mike

[Mike_Intel]

Hello Mike_M,

Thank you the quick reply.

If the same set up is working fine on Windows 10, that means there is no issue with the hardware.

There is a big chance that this is a driver issue or how the system was validated with Windows 11.

I am looking at the link that you provided and it says that It supports Windows 10/11.

Since you are having issues with Windows 11, you may need to contact the manufacturer of the board/system for further diagnosis of the issue. It is also best to ask them the recommended Intel Rapid Storage Technology driver for Windows 11 in your system since they may have altered/modify the controller.

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hi Michael,

That's exactly why I'm here;

it is an Intel RST controller. It is now quite clear that it is causing the problems under Windows 11 and that it works under Windows 10. It is therefore correct that it is probably not a hardware defect, but a driver problem under Windows 11. However, this driver is provided by Intel and not by Shuttle. Shuttle only offers the original one from Intel.

The same applies to support for the RST controller. Shuttle only knows what to say in general terms, but not about such very specific problems. There seems to be a lack of diagnostics.

Unfortunately, the manufacturer of the device does not know more about this either, apart from referring to you.

So what else can I do?

With best regards,

Michael Muhlack

[Mike_Intel]

Hello Mike_M,

Thank you for the update.

We will do further research on this matter and post the response on this thread once it is available.

Have a fantastic day, and thank you very much for your patience and understanding!

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_Intel]

Hello Mike_M,

I hope this message finds you well.

We are still checking this issue and while doing that, please help provide or clarify the following detils:

1) Where did you get the Intel RST driver that you installed on your system? Did you ever use the Intel RST driver from Shuttle website since this it is an OEM product (OEM might have done some customization vs generic Intel drivers).

Shuttle Global || SW580R8 - Download

RST_RAID_Intel_W580 - 18.1.5.1038 - https://download.shuttle.eu/Mirror/MiniPC&amp;amp;AIO/cube/SW580R8/Driver/RST_RAID_INTEL_W580_W11-64_W10-64.ZIP

2) When you said copying from X to X - so it means from RAID to same RAID array?

3) When you said pauses at 3 - 5 seconds, does it mean that it will still proceed with copying?

4) Did you try a different file set to test this just to make sure that there is no corruption with the original file set used? The 0 kb/s transfer happens on file 2 (16).mp4 which may have been corrupted. Or does this happen on any file or always on that file?

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hello Michael,

thanks for the answer!

About your questions:

1) Yes, I have downloaded the original driver from Shuttle. I did this before the first installation and then packed it onto a USB stick so that I could install the missing drivers directly after the installation. But unfortunately this did not lead to the desired result. I then tried the drivers from Intel and drivers that I downloaded via "Snappy Driver Origin". The results were all the same.

2) Yes, "X" actually stands for "X:\" The problems only occur in the combination RAID to RAID. SSD to RAID is not a problem. And RAID to SSD just as little. SSD to SSD is also not a problem.

3) This is a little more difficult to describe. The pause in the copy process is not a "real" pause. Under normal load (i.e. only copying) on the system, the copying process starts and drops off after approx. 3-5, maximum approx. 20 seconds and, according to the copying display, the average drops rapidly to 0KB/s at the end. The bar then stops and there is visibly no more progress during copying. This state can last up to a minute. Sometimes, however, it may only last a few seconds.

If you then look at the copying process in the Resource Manager, you can clearly see that initially only reading (not writing!) is taking place. Then the copy process switches to write mode (no more reading!) and at that moment the stall occurs. Then the copying process switches to a mode where reading and writing take place simultaneously and the stocker stops. This only occurs once and reproducibly at the beginning. But only under Windows 11. I could not cause this under Windows 10 and Ubuntu. It runs as it should; quickly and without stuttering. Of course, the KB/S display also fluctuates greatly during the copy process under Windows 10, but I know this from other systems under Windows 11 and consider it normal.

4) This occurs no matter which folders and files I test with. If I copy large or many files, it is easier to see, as the duration then goes over the 3-20 second limit. For testing purposes, I created a folder at the end into which I copied a collection of small and very large files. I named this "/1". Then I also copied these contents into a folder "/2" and did the same up to "/9". Then I put all the folders into a folder "/Round1" and a folder "/Round2" and this in turn into a folder "/Test" so that in the end a structure like "/Test/Round1/1/testfile.mp4" was created. The folder is about 50GB in size, so it takes a while to copy everything, but I got a typical mix of different file types and folders.

I hope I was able to describe the problem in a reasonably understandable way.

Best regards,
Michael Muhlack

[Mike_Intel]

Hello Mike_M,

Thank you for the update.

I will have this checked again and post a feedback on this thread once it is available.

Have a fantastic day, and thank you very much for your patience and understanding!

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_Intel]

Hello Mike_M,

I hope this message finds you well.

We just want to gather additional details while we are checking this issue. 

At the Intel Optane Memory and Storage Management page - "Manage" tab, can you send screenshots of the drives (1 SATA drive selected then nVME drive selected). These should show additional information, so at that is at least 2 screenshots.

Lastly, what is the role of the 2 Samsung 990 Pro SSDs in the system and where is the OS installed at?

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hi Michael,

thanks for your answer.

The two drives are for the OS and a Database.

However I don´t think, that they have anything to do with the issue. To localize the cause, I also started from a Windows 11 boot CD and removed all unnecessary hardware (graphics card and SSDs) from the system beforehand. The problem remained the same. However, Windows 10 was installed on the smaller SSD (which is also the installation disk for Windows 11). The error did not occur under Windows 10. I still suspect an error in the Intel RST driver under Windows 11.

Kind regards,

Michael

[Mike_Intel]

Hello Mike_M,

Thank you for this information.

I will have this checked again and post a feedback on this thread once it is available.

Have a fantastic day, and thank you very much for your patience and understanding!

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_Intel]

Hello Mike_M,

I hope you are having a good day.

Please check our update here:

1) Please update Intel RST driver using v18.7.6.1010.3 from Intel® Rapid Storage Technology Driver Installation Software with Intel® Optane™ Memory (10th and 11th Gen Platforms).

2) If you are willing and depending on your RAID array setup, please measure CDMv8 (Crystal Disk Mark) performance with or without changing the stripe size. For CDMv8, please make sure that you select default settings for consistency and best with screenshots for each. Also check if copying is better to/from the RAID array after.

Here are the RAID stripe size to try:

• RAID 0 - stripe size 2 MB
• RAID 5 - stripe size 256 KB

Here's where to download CDMv8 - CrystalDiskMark - Crystal Dew World [en] (crystalmark.info) - 3rd party application.

If you have questions, please let us know. Thank you.

Best regards,

Michael L.

Intel Customer Support Technician

[Mike_M]

Hi Michael,
I am sorry to say, but I have given up (for now).
I´ve sent the whole two systems back to amazon, before the time is over to do so and receive my money back.

But, after some holidays I´ll take now for some weeks, I´ll give a try to two new systems I ordered now. I´ll do so by mid of next month.
It´ll have the same RST-controller, but it´s an ASUS Board (ASUS PRO WS W680M-ACE SE) then, because I don´t have any more trust in this shuttle systems.
So for all which may have the same issues and just searching for a solution:
A simple one is just use Windows 10 or if possible Linux. If not possible, you may can try the driver above (which I did not).

So I´ll be back for sure, just to say, if the ASUS Boards do not have the problem as well, or if they have the same issue, then for sure, I´ll try your last Ideas and will report the results here as well.

Thanks until now for your assistance.

I´ll be back.

Kind regards,
Mike

[Mike_Intel]

Hello Mike_M,

Thank you for the update and I understand.

Since you sent back the 2 systems, I need to close this inquiry. 

If you need further assistance, please post a new question as this thread will no longer be monitored. 

Thank you and have a great day

Best regards,

Michael L.

Intel Customer Support Technician