VROC boot problem

[KOstr]

Hello all,

I am having constant issues with Windows 10 booting from my VROC RAID0 volume. In about 30% of cases Windows boot hangs with the balls rolling indefinitely. The only help is to use hard reset. I believe I have a configuration that should be fully supported according to Intel documentation.

Here are the most relevant details of my configuration:

Intel Xeon W-2155

ASUS WS C422 Pro/SE (latest BIOS version 702)

Intel VROCPREMMOD Key

ASUS Hyper M.2 X16

Samsung SM961 MZVKW1T0HMLH (four units)

no other storage devices connected to either PCH or VMD

Broadcom MegaRAID 9460-16i controller (with non-bootable RAID6 using mechanical disks)

I have one VROC array configured with two volumes: RAID0 boot volume (1TB), and RAID5 data volume (2TB). Windows was freshly installed with the help of F6 drivers version https://downloadcenter.intel.com/download/27331/Intel-Virtual-RAID-on-CPU-Intel-VROC-and-Intel-Rapid-Storage-Technology-enterprise-Intel-RSTe-Driver-for-Windows- 5.3.0.1413.

Best regards,

Karol

[idata]

Hi joe_e_e,

Thank you for posting in the Intel communities.The issue you are describing is not common. I'd recommend you to create the RAID arrays again and check if the issue is not related to one of the drives that could not be working properly.On the other hand, there is a newer driver available which is the version 5.4.0.1.1463. I recommend you update and install this version.Let me know if you are still having the issue with the latest driver and after creating again the RAID arrays.Have a nice day.Regards,Diego V.

[KOstr]

Hello Diego,

thank you for the prompt response. I have been in contact with ASUS previously, and from my problem description they did not think it could be caused by a single drive issue. With one faulty drive the RAID0 array would probably be marked as failed, and the Windows boot loader would not start. However, they replaced the ASUS Hyper M.2 X16 card as a more likely problem source. So, I have already recreated the RAID array with the new adapter card without any improvement.

In the Intel RSTe GUI and in the BIOS/UEFI, all the four drives are show with normal working status.

As for the new version of the drivers: yes, I have noticed that they are available, but my understanding from the documentation is that they need the associated UEFI update to work to full extent. I have contacted ASUS about this, and I am awaiting their reply. Should I still install the drivers, even on an older ASUS BIOS/UEFI?

I should add that the boot problem appears irrespective of the start type: cold start or restart. For example, it occurred several times after regular Windows update that required a restart.

Best regards,

Karol

[idata]

Hi Karol,

Thank you for the information.You are correct about the BIOS/UEFI update, so let's wait for the ASUS* response.On the other hand, I'll investigate about this and let you know any additional information I can find.Please keep me informed about the ASUS* response.Have a nice day.Regards,Diego V.

[idata]

Hi Karol,

I'd like to know if you have updates about this.Have a nice day.Regards,Diego V.

[KOstr]

Hello Diego,

an ASUS technician replied that he has notified their BIOS developers about the new VROC version. However, they gave no dates or any specific commitment to when they will release a new BIOS with VROC 5.4 included. Is there any chance for Intel to talk to ASUS about it? It cannot be much work to integrate an updated UEFI sub-system.

Otherwise, I started to keep more detailed statistics about the starts and it is about 40% starts that it hangs.

I don't know if this plays any role, but Windows disk management reports the MegaRAID RAID array as physical disk 0, and my bootable VROC RAID 0 as physical disk 1 (when it starts successfully).

Best regards,

Karol

[idata]

Hi Karol,

Thank you for the information.

Regarding to the latest VROC drive support on ASUS* BIOS, that depends on them if they want to release an update or not.

On the other hand, I'll try to find additional information that may help you to resolve the issue.

However, I'm still thinking that maybe the issue could be related to the drives. I understand that you have changed the ASUS* Hyper M. X16 card, and the drives are shown with normal working status, but just to be sure and discard possible causes, have you tried with different drives?

I'll post my updates as soon as possible.

Regards,

Diego V.

[idata]

Hi Karol,

Could you please provide the log files from the RSTe application? They should be under C:\Users\\Intel\LogsRegards,Diego V.

[KOstr]

Hello Diego,

there is nothing in that location. I searched for Intel under C:\Users\ and there is nothing. Nothing under C:\Users\\AppData either. Do I need to turn on logging in some specific way?

Best regards,

Karol

[idata]

Hi Karol,

Thank you for letting me know it. Could you please try to generate the report from the RSTe application? You can use the Email option on the RSTe GUI to generate the report file.Regards,Diego V.

[KOstr]

Hello Diego,

I have attached the report from the RSTe application.

Best regards,

Karol

[idata]

Hi Karol,

Thank you. I'll post any updates as soon as possible.Have a nice day.Regards,Diego V.

[idata]

Hi Karol,

We've been working on the issue you described, however we haven't found any possible issues that could cause this behavior. We consider that this may be due to an integration problem with the motherboard, so our advice is to contact ASUS* for additional support.

Regards,

Diego V.

[KOstr]

Hello Diego,

I have been in contact with Asus previously, and as a first step they exchanged the Hyper M.2 x16 board. This did not improve the situation. I have also notified them of the new VROC UEFI version, but they could not commit to any delivery date for a new UEFI BIOS. Do you have any technical details or recommendations that I can pass on to Asus in order to get additional help from them?

Additionally, the ASUS UEFI BIOS correctly recognises the VROC RAID arrays. There does not seems to be a problem in the ASUS BIOS in this respect. But could this issue be related to how the motherboard enumerates various storage devices (VROC, MegaRAID, etc.) and how Windows boot process then tries to load appropriate drivers? Is there a way that I can get logs from the Windows boot process to see where the issue really arises?

2018-09-28 edit: there seems to be a strange development: in the last two days, during cold start two DIMMs out of four were not recognised by the BIOS (first two positions/channels A1 and B1 even when swapping DIMMs around). Is this something that can be contributed to a possible motherboard or a CPU issue?

Best regards,

Karol

[idata]

Hi Karol,

I'm sorry for the delay in my response, but I was checking with the Engineering Team to see if there was anything that can be useful for you, however further debugging from our side will not help at this point since the driver version is customized around the ASUS* platform specifics which we do not have access to.You should continue working with ASUS*. If they conduct their analysis and conclude that the issue is with the RSTe/VROC driver, ASUS* will reach out to us for help on a fix.Regards,Diego V.

[KOstr]

Hello Diego,

thank you for the reply. I am in active contact with ASUS technical support, and we have made some progress. However, there is one big hurdle to overcome: on the platform I am running, ASUS has verified all the Samsung SM961 drives up to 512GB. I am running 1TB drives, and ASUS is trying file my request under "not supported configuration", even thought according to Intel documentation all SM961 are supported. For example, ASUS recommended that I try to test with verified drives. But I have no options to procure 4 other Samsung SM961 SSD just for a test.

Personally, I don't see why there should be anything different with the 1TB version of SM961. The controller is the same, so I would expect that the external (NVMe) behaviour of these drives should be the same as all the smaller drives. Power requirements could be different, and if my motherboard has issues with power delivery then this might have some impact. But this is hard for me to verify.

Is there any relevant information that the Intel Engineering Team might have with respect to any potential differences between the various capacity variants of Samsung SM961 drives? It might help me with tests and perhaps provide more details for ASUS technical support.

Best regards,

Karol

[idata]

Hi Karol,

Unfortunately, there is nothing from our side that I can share with you as the Samsung* SM961 model is indeed a supported drive. We are not aware of any difference due to the drive capacity, however there might indeed be some differences between drives that are the same model but different capacities. As you already said, one difference might be the power consumption, but there might be also differences in the way the data is accessed. These differences may not be relevant in regular or simple configurations, but they may affect other kind of configurations like RAID arrays, most likely because of the power requirements.If the drives indeed require more power, the motherboard may not be able to power them correctly and that would explain why sometimes everything works, but sometimes it doesn't. Having said this, it may be good idea to contact Samsung* as well as these drives are manufactured by them. If there is indeed any difference between the capacity versions, they may be able to explain it to you.Have a nice day.Regards,Diego V.

[KOstr]

Hello Diego,

thank you for the update. I will await the result of ASUS analysis, which is leaning towards a motherboard replacement, and update you later.

Best regards,

Karol

[KOstr]

Hello Diego,

I have one small update that might point the problem back to the Intel drivers or Windows: my start statistics from the last few days is 50% failed starts. After each two failed start attempts, Windows goes into a self-repair mode. There, I was able to enter command line, and I have noticed that my two RAID volumes were mounted in the "opposite" order: the first RAID0 volume containing Windows was mounted as drive letter D and the second RAID5 volume containing data was mounted as drive letter C. If this is used for booting Windows then I can understand that it cannot boot from C. The RSTe GUI and BIOS show both volumes as bootable, and there is no option to disable bootability of the RAID5 volume.

Could this mixup of my two volumes be the issue leading to boot failures?

Best regards,

Karol

[idata]

Hi Karol,

I cannot say that this may be the root cause of the issue, and I cannot replicate it either. As it was mentioned before, the driver version is customized around the ASUS* platform specifics which we do not have access to. You can report this to ASUS* and if their conclusion is that the issue is because of the RSTe/VROC driver, then ASUS* should reach out to us for further revision.Having said this, I don't have any suggestion for you besides what we have discussed so far in the thread.Regards,Diego V.

[KOstr]

Hello Diego,

I can report that ASUS supplied me with a beta BIOS for my motherboard that includes a solution to my issue. It also includes the latest VROC version 5.4, and I upgraded my Windows drivers. So far, after one week of usage I have had zero boot failures.

Best regards,

Karol