Intel Tech Support - Please Help with SSD Corruption with RST

[Michael4000]

I have multiple computers with Western Digital SN850 SSDs and Samsung Evo 990 SSDs that are being corrupted by Intel RST.  Another person on this forum has the same issue.

Can we please get some help from Intel?

[Earl_Intel]

Hi Michael4000,

Thank you for posting in the communities!

I'm sorry to hear that you are having problems with your system. In order to help you more effectively and figure out the best solution, could you kindly supply the information below:

• Can you describe the issue more to us?
• May I know what you mean by corrupting the SSDs? does the physical device being corrupted or the files inside the device?

Kindly run this on your system so that I can also gauge the following peripherals and the driver versions that you have:

Intel® System Support Utility for Windows*

Once finished downloading, open the tool, click on scan (leave the default items checked), click next, then save the system information to a .txt file. Please provide us the .txt file.

Best regards,

Earl E.

Intel Customer Support Technician

[Michael4000]

Thanks, Earl.

Here is the situation.  We have been building HP Z2 G9 workstations for years, and use Intel RST for RAID 1 mirroring.  Recently, the RAID has been failing.  This started about six months ago when we started using the new SSDs from Western Digital and Samsung.  They are the WD Black SN850, and the Samsung Evo 990.  I have made sure the SSD firmware, and Intel Rapid Storage Technology drivers from HP are up-to-date.  Here is an example of what it looks like when it fails.

Michael4000_0-1737694846041.png

When I say "corrupted", the WD SSDs get SMART errors.  Samsung SSDs don't typically get SMART errors, but they just aren't recognized anymore by Intel RST.  We did not have this issue prior to using these latest, faster SSD drives.  Every HP workstation we have built with the latest WD and Samsung drives has had this problem.  So the problem is very repeatable (and concerning),  None of the HP workstations running the older drives, such as the Samsung Evo 970 has had this problem.

Other symptoms include:

1.  Blue screen errors.

2.  Boot device not found errors.

I have uploaded the Intel System Support Utility output txt file.  I appreciate your help!

Michael

[Earl_Intel]

Hi Michael4000,

Thank you for sharing this useful information with us.

No worries, I will further check on this internally and offer an update on this thread as soon as feasible.

Best regards,

Earl E.

Intel Customer Support Technician

[Earl_Intel]

Hi Michael4000,

Thank you for patiently waiting.

To further troubleshoot the issue, can you provide the following information below:

• Did you set up Windows on top of the Intel RST RAID 1 array or is Windows on a separate drive and not in the RAID 1 array?
• Can you explain the exact steps that leads to the BSODs or Boot device not found errors? What specific SMART errors are you getting?
• Can you confirm if this is the exact model of the PC?
  • HP Z2 G9 Tower Workstation - Customizable - HP® Store
• Did you used the latest Intel RST driver from HP? (Intel Rapid Storage Technology VMD Driver 20.2.3.1018.1 Rev.H - dated January 9, 2025)
  • HP Z2 Tower G9 Workstation Desktop PC Software and Driver Downloads | HP® Support
• In relation to this, my I know if you have the latest BIOS installed?
• Did use Intel RST setup guide from HP to properly setup RST RAID 1 array?
  • 4AA8-3337ENW.pdf

Best regards,

Earl E.

Intel Customer Support Technician

[Michael4000]

Earl,

Here are the answers to your questions.

• Did you set up Windows on top of the Intel RST RAID 1 array or is Windows on a separate drive and not in the RAID 1 array?

I installed Windows on top of the Intel RST RAID 1.

• Can you explain the exact steps that leads to the BSODs or Boot device not found errors? What specific SMART errors are you getting?

The BSOD is intermittent.  It just appears on the screen while the user was using it.

The Boot device not found error occurs after the computer has been sitting overnight.

• Can you confirm if this is the exact model of the PC?
  • HP Z2 G9 Tower Workstation - Customizable - HP® Store

Yes, this is the computer.

• Did you used the latest Intel RST driver from HP? (Intel Rapid Storage Technology VMD Driver 20.2.3.1018.1 Rev.H - dated January 9, 2025)
  • HP Z2 Tower G9 Workstation Desktop PC Software and Driver Downloads | HP® Support

We use HP Image Assistant to update drivers.  I know I have also manually downloaded and installed the Intel RST driver, but I'll have to check to be sure it was this one.  I'll install it on a couple Z2 G9s and get back to you.  We had another computer today that said "Unrecognized drive".  Power cycling did not fix the issue on this one (it does on some).  I didn't want to install this driver for fear that the entire computer might go dead.

• In relation to this, my I know if you have the latest BIOS installed?

Yes, we have the latest BIOS installed.

• Did use Intel RST setup guide from HP to properly setup RST RAID 1 array?
  • 4AA8-3337ENW.pdf

I didn't use this setup guide per se, but I am following the same instructions as "Create RAID Arrays" #1.

Thanks for you help!

Michael

[Earl_Intel]

Hello Michael4000,

Thank you for providing this information with us.

Noted on this, I will check on this internally and provide an update on this thread as soon as possible.

Best regards,

Earl E.

Intel Customer Support Technician

[Michael4000]

Hi Earl,

We installed the latest HP BIOS 03.03.11 Rev. A and Intel RST VMD Driver 20.2.3.1018 on three HP Z2 G9 computers running RAID 1.  Two were updated yesterday, and one today.  We rebooted them, and rebuild the RAID 1 on both before installing the updates.

I checked the first computer today that we did yesterday.  Bad news.  The RAID is already broken.

Michael4000_0-1738287394664.png

You can it has the latest RST version.

Michael4000_1-1738287438091.png

The BIOS shows it is the latest.

Michael4000_2-1738287577487.png

Today on the second computer that was updated yesterday with new BIOS and Intel RST, Intel RST won't launch.

Michael4000_3-1738288061798.png

I rebooted the computer, and the following came up.

Michael4000_4-1738288884282.png

In conclusion, unfortunately, the BIOS and Intel RST updates did not have the desired outcome.

I appreciate any help.

Michael

[Earl_Intel]

Hello Michael4000,

Thank you for sharing this update with us, I will further check on this internally, on the meantime, can you supply us with additional information below:

• Were you able to confirm with HP Support that the 2 drives WD Black SN850 and Samsung Evo 990 are compatible with the HP Z2 G9 workstation PCs?
• May I know if you setup Windows in either the nVMEs as stand-alone drives (not in RST RAID 1 config), will you still get the SMART errors, BSODs and Boot Device Not Found errors?
• And on same topic on "unrecognized drive", have you check in the BIOS, will it still show up that the RAID 1 array is still setup in RST?

Best regards,

Earl E.

Intel Customer Support Technician.

[Earl_Intel]

Hello Michael4000,

Just checking in if you had a chance to look over the questions I posted.

In order to decide on the appropriate course of action to address this issue, kindly inform me as soon as possible.

Best regards,

Earl E.

Intel Customer Support Technician

[Earl_Intel]

Hi Michael4000,

I have not heard back from you so I will close this inquiry now. If you need further assistance, please submit a new question as this thread will no longer be monitored.

Best regards,

Earl E.

Intel Customer Support Technician

[Michael4000]

It takes time to collect this information.  I was on the phone with HP for 2.75 hours.  When I went back to give you the information, you closed the case.  I don't think that's right.

Intel really needs to look into this.

[fishowl]

@Michael4000 Did you solve the problem? Any news from HP/Intel or Samsung/WD?

[Michael4000]

After all of this time, here is what I got back from Intel.

"Thank you for your response. I would like to confirm that our team confirmed that the SSD's you mentioned are compatible and works properly. 

As much as I would like to assist you further, it is already out of my scope of support and HP should be the one to assist you to resolve this issue. 

I will now be closing this case again."

If they are working properly, why are other people having the same problem from different motherboard manufacturers?  I am not convinced this was ever tested with Intel RST RAID 1.

So I contacted HP.  I even sent one of our computer to them to evaluate.

They never tested it with the drives I supplied.  Instead they put in Samsung Evo 970s, slower drives, and called it good.  He never replicated the failure, so the test was not a validation.  Here's what I got back,

"After reviewing log files, the drives themselves are in good health, they lack certain RAID critical features like Power Loss Protection (PLP). I checked all the drives you sent me in the list and none of them have PLP. Overtime, especially during power cycles, system sleep, or shutdowns, this can lead to desynchronization between the drives, causing corruption or instability in the RAID volume. One key example is power loss protection. Enterprise SSDs have built in capacitors that flush in flight data to nonvolatile storage during an unexpected shutdown. Consumer drives generally do not. This is important because your system has logged 44 abnormal shutdowns over a 91 power on count. Each one of these could pose a risk of partial or corrupt writes, especially in RAID 1 setups where both drives are expected to stay in sync. Without power loss protection, these shutdowns can silently damage file systems or destabilize the RAID overtime.

Even though third-party consumer NVME drives may physically install and operate, they’re not officially supported or tested in this environment. That means if a failure or compatibility issue arises, it’s difficult to isolate or support at the system level and HP will not be able to provide direct assistance when non-qualified drives are involved."

Problem is, HP's drives for their workstations are not Enterprise grade either.  He blamed the failures on power failures, yet the computer had a battery backup. 

Are there any actual engineers anymore?  I've worked as a degreed test engineer.  This is nonsense.

[loganhooker02]

Hello Michael, have you ever found any remedy? Have been having same issue start to pop up with HP Z2 G9's as well and have read through all your threads here and the one you made on spiceworks, very frustrating there seems to be no answer anywhere.

[Michael4000]

Unfortunately, no solution was found.  I shipped a Z2 G9 computer to HP, and that was a waste of time.  They didn't do any real analysis on it.  I got a call from Intel Support one day.  When I called back, they offered no help.  I'm not sure why they even called me.  Their response was it was HP's problem.  I have been diligent on updating drivers, but none of the updates has helped.

here is what Microsoft 356 Copilot says:

"From the evidence I've seen, including your own documentation, I actually suspect the root cause is Intel VMD/RST interacting badly with modern consumer PCIe Gen4 SSD power management, rather than Samsung 990 or WD SN850 hardware defects alone. The fact that both brands exhibit nearly identical failures points toward the RAID stack or VMD layer rather than a specific SSD controller.

What Intel could do

1. Make RST More Tolerant

Intel RST appears to remove a drive from the mirror relatively quickly when communications are interrupted.

A better approach would be:

• Retry communication multiple times
• Wait longer before marking a member failed
• Distinguish temporary NVMe timeouts from actual drive failures

2. Improve VMD Diagnostics

Instead of:

"Drive Unidentified"

RST should report:

• NVMe timeout
• Link reset
• Power-state transition failure
• Firmware communication loss

That would dramatically simplify troubleshooting.

3. Publish a Certified SSD List

Intel should maintain a live list of:

• Tested SSDs
• Tested firmware revisions
• Known bad combinations

Much like enterprise storage vendors do."

This issue has been going on for quite some time.  I just wish it would get addressed by the manufacturers involved.