Intel Optane Memory and Storage Management fails to remap NVMe SSDs to VMD controller

[mps1]

I am trying to set up bootable raid arrays and not having any luck. 

I have an OWC 8M2 with 8 1tb SK Hynix Platinum P41 NVMe gen 4 ssds. It is installed on an Asus Q670 motherboard with an Intel i9-12900ks processor. It is connected to the sole PCIe Gen 4 x16 slot. The two M.2 PCIe slots are unused.

I have followed the instructions here using both the Intel provided and the Asus provided drivers:  How to Configure RAID or Intel® Optane™ Memory with...

I have tried every possible permutation i can in the BIOS. I have tried this both managed by my org in intune and without the intune management. I have tried using only local admin accounts and using connected microsoft and work.school accounts.

i preinstall the driver on the booth disk during windows install after wiping the disk. I boot into windows and install the Intel Optane memory and storage management gui. I restart. Then upon restart when I reopen the Intel Optane Memory and Storage Management program, I am prompted to do a one time remap of my PCIe NVMes, which show as devices on the device list in the program. The VMD driver shows in the device management console in windows. However, after restarting, the drives are not remapped. Nor do the drives show as having the VMD driver. They show as having the standard windows driver. There is also a driver for the broadcom controller that is in the OWC 8M2. I have attached a screenshot of the device management console. When i try and manually install the VMD driver in place of these default controllers, I get bluescreen of death and am required to reinstall windows.

It seems no matter what I try, I cannot get any of these drives to be configurable in RST.

ASUS support has me rollback the bios, flash the CMOS and reinstall the drivers. Which I did to appease them, although I knew this wasn't the issue.

I attached screenshots showing the state of the RST Gui as well and the relevant device manager drivers.

Any suggestions?

[mps1]

Here is the SSU output.

[AndrewG_Intel]

Hello @mps1

Thank you for posting on the Intel® communities.

In order to review this further, could you please confirm the Intel® VMD driver version that you are trying to install, as well as the version of the Intel Optane™ Memory and Storage Management app running on the system?

Also, do all the PCIe SSDs show the same Controller/Port number association in the Intel Optane™ Memory and Storage Management app?

Finally, what is the Blue Screen of Death (BSOD) error code or error message that you get when trying to manually install the Intel® VMD driver?

Best regards,

Andrew G.

Intel Customer Support Technician

[mps1]

Hi Andrew

The VMD driver version is Intel RST VMD Controller 467F v 19.5.0.1037.

The Intel Optane Memory and Storage Management app version is 19.0.1.1034.0.

In the Intel Optane Memory and Storage Management App, all of the drives show the same Controller/Port number association of Controller 0 and Port 0. 

I will get the error code from the BSOD screen shortly.

Thanks

-Mike Sevarino

[mps1]

The BSOD error was something like "iaStorVD.sys is not less than or equal to." Or it might have been not greater than or equal to. Let me know if this is specific enough or if I should recreate the problem to get the exact error. I don't remember the specific error code.

Edit: I think this is the BSOD error I get actually. 

DRIVER IRQL NOT LESS OR EQUAL

And then it will say what failed: iaStorVD.sys." which is the driver i tried to manually install.

This seems consistent with the remap failure. 

[Jean_Intel]

Hello mps1,

Thank you for your response.

Based on the information provided, we noticed that the Intel® Rapid Storage Technology Driver Install on your system is the one provided by Asus, we always recommend using the software provided by the Original Equipment Manufacturer (OEM); however, there are Intel’s generic drivers which are intended for testing. We would recommend installing our Intel® Rapid Storage Technology (Intel® RST) Driver 19.2.0.1003.

Also, regarding the information you shared by the BSOD, we found information from a Microsoft article and forum with recommendations about this specific issue.

Best regards, 

Jean O.  

Intel Customer Support Technician

[mps1]

Unfortunately, I have tried with the intel driver as well with the same result.

Re the IRQL error, the only option I am given each time is a full restore to correct the issue. After the restore, the drivers are set back to the default and not to the Intel VMD, and remapping does not correct this. I am just in a constant loop of being prompted to do a one time remap, selecting all drives for remap, restarting, and then being prompted again to remap, and the remap never occurring.

[mps1]

So i have found that i get different results depending on whether I install the Intel Optane™ Memory and Storage Management app from the Microsoft store or from the Setup installer, both with the Intel driver and the Asus OEM driver. When I install from the Microsoft store, the Create RAID Volume menu isn't greyed out, although all the options with in it are. Also, I get different error messages in the Optane memory blade, although its my understanding that the 12th and 13th gen chips no longer support Optane Memory.  The drives don't show at all, even the boot drive that definitely has the drivers installed.

When I install from the Setup app immediately or when i upgrade from the version in the Microsoft store, I am in the VDM remap loop again, with the Create a Raid option greyed out. In both situations, it appears that the device configurations in device manager are identical. I have attached screen shots and the SSU from that configuration.

[AndrewG_Intel]

Hello mps1

Thank you for your response and all the details. Please allow us to take a look into this and we will be posting back in the thread as soon as possible.

Best regards,

Andrew G.

Intel Customer Support Technician

[mps1]

Thanks I appreciate your help.

I found a setting in my Intune blocking DMA access for PCI downstream hotpluggable devices. I suspect this might be blocking the VDM remap. I am going to wipe the disk with this config disabled and a fresh reinstall and see if that works. If it doesn't I will roll back the BIOS and drivers to the previous version and then just do a series of drive wipes/clean installs with both ASUS OEM and Intel drivers through the most recent and do SSU scans for each configuration.

Hopefully the DMA setting will solve the problem.

[mps1]

Unfortunately, I was unsuccessful. Turning of DMA and Kernal DMA didn't change anything.

I did a BIOS secure erase of the NVMes before each clean install.

I do have a couple questions if anyone can answer them.

Do either VT-x or VT-d need to be enabled for RST to work on the host machine. Not worrying about passing raid arrays through to VMs yet.

If so, does TXT support need to be enabled?

 If VT-d needs to be enabled, there is a setting in my BIOS called "Control pre-boot IOMMU behavior," with options, "Enable pre-boot IOMMU," and "Disable IOMMU." Would this need to be enabled for RST to run? If so, this is a problem because whenever I enable pre-boot IOMMU, I get a bootload failure with both the windows bootloader and the UEFI install media.

Would publishing HII resources make any difference to whether RST functions?

I noticed in services I do not have anything tha would be Intel RST, I just have something called an Intel Storage Middleware Service. I get a dialogue saying to upgrade to the latest version on the Optane MEmory blade, when it says No Controller Found." Although this could be because of the dropped support for Optane memory.

Going from the 2020 first edition of the user manual, which is the most recent I can find: https://www.intel.com/content/www/us/en/support/articles/000057208/technologies/intel-rapid-storage-technology-intel-rst.html

it shows the option to clear meta data from a drive. However, I noticed this option isn't available to me in my RST gui.

The manual also mentions that at least one SATA drive is required. It seemed like this requirement went away in some of the  release notes from different upgrades, but I could be wrong. The most recent read me also seems to mention the SATA requirement. Will RST not function if the SATA controller is turned off? I have tried with it both turned off and turned on, and it didn't seem to make a difference. I don't have any SATA drives. Is one required for RST to function?

Any chance that Broadcom controller is causing a problem? 

Finally, the only other thing I can think of is does it matter if I am logged on with an Azure AD Work account or a local admin account?

The root of it seems to be the failure to remap the drives to the RST driver. 

Thanks for your help with this.

[AndrewG_Intel]

Hello mps1

Thank you for the additional information. We are checking your inquiries and we will post back in the thread once more information is available.

Best regards,

Andrew G.

Intel Customer Support Technician

[mps1]

I noticed something on my motherboard and it prompted me to ask Asus about it and they confirmed that IRST is not supported on the PCIE 4.0x16 slot. I circled the legends that seem to indicate as such. 

Turns out, although the Q670 spec from intel supports IRST for PCIe lanes both to the CPU and the PCH, the only PCIe lanes supported to the CPU on this particular asus motherboard are from the one M.2 PCIe that is connected to the CPU. The documentation states that in addition to the two M.2 slots for NVMe and the SATA controller, only PCIe NVMe devices that are connected to the PCH are supported. Ergo, not the PCIe 4.0 x 16 since that is connected to the CPU.

Theoretically I suppose it would be possible to add M.2 PCIe NVMe expansion cards to the PCIe 3.0 x 1 slots, which are connected to the PCH, to add two more drives to the M.2 slots, or I suppose any number of SATA to M.2 NVMe adapters, but an expansion card through the PCIe 4.0 x 16 is a no go with this motherboard if using IRST.

With that said,  I have two NVMe SSDs installed in the M.2 slots and they are running as expected in a Raid0 config.

Thanks for your help with this.

Also, for what its worth, the IRSTe and IRST Premium and IRST with optane and all of these very similar naming conventions for very different products makes navigating the limited documentation that is out there for IRST premium difficult. It isn't readily apparent that IRSTe, or I guess VROC now, is only compatible with xeon chips for example and is an entirely different platform that IRST Premium.

In any case, again thanks for the help, but it turns out that I was chasing the impossible.

No reason to keep this discussion open if the mods want to close it out. 

[AndrewG_Intel]

Hello mps1

Thank you for your response and for the details. Hopefully, this may help other community users experiencing similar behavior.

We were working on investigating this matter; however, we understand that you have contacted ASUS* who confirmed that the PCIe 4.0 x16 slot does not support Intel® RST. Also, since two NVMe SSDs installed in the M.2 slots are running as expected in a RAID 0 configuration, it seems that the Intel® RST feature is working fine.

Also, we do appreciate your feedback and comments regarding Intel® RST documentation and naming conventions.

Since you have found the root cause of the behavior after working with ASUS* and as per your suggestion that the thread can be closed, we will proceed to close it now.

If you need any additional information, please submit a new question as this thread will no longer be monitored.

It has been a pleasure to assist you.

Best regards,

Andrew G.

Intel Customer Support Technician