VROC Raid 5 journaling drive selection for Lenovo ST650v2

SteveP2 · ‎09-27-2022

Good afternoon,

Need help finding the best option for the ST650v2 server to deal with Raid 5 Write hole issue.

We have a quote for the ST650v2 with qty(3) U.2 P5620 6.4TB mixed use NVME PCIe 4.0x4 HS SSD. We have VROC premium handling the qty(3) drives as a RAID 5 volume.

The system has qty(2) NVMe Bay Backplanes so my understanding is we have 5 open slots for more PCIe 4.0x4 drives.

Would another 6.4TB P5620 be the best option for a journaling drive?, or would you suggest some other Intel Optane drive option instead ? If Intel Optane is the best option what size / model ?

BrusC_Intel · ‎09-28-2022

Hello, SteveP2.

Thank you for posting on the Intel Community Support forums.

I received your case regarding the R5WH, and I will be assisting you with this.

Intel VROC provides two options to take care of the Write Hole, one that requires the additional "Journaling" drive, or the "Distributed" mode that uses the existing RAID members (and is also the recommended option).

There is no particular "best drive" for Journaling mode, because using the an SSD similar in capacity to the RAID members should be fine, and the Optane SSDs are also a good option because of its endurance.

Something to keep into consideration if you are going to add a disk is the port, as it must also be VMD enabled/capable.

Additional details can be found in the following article:

- https://www.intel.com/content/www/us/en/support/articles/000057368/memory-and-storage.html

If you have questions, just let me know, and I will follow up on September 30th just in case.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎09-30-2022

Hello, SteveP2.

I wanted to follow up on this community thread to check if you had any questions regarding my previous response.

If I can help you with this or anything else, please let me know, and I will follow up again on October 5th.

Regards,

Bruce C.

Intel Customer Support Technician

SteveP2 · ‎10-01-2022

Thank you - I read the link sent to me going over the R5WH issue but still have a few questions.

Based on the article/Link the three methods listed :

1. Distributed - default method - preferred option to choose - cheapest method , only drawback- slower write performance - How much slower than the journal drive method ? 10 percent, 20 percent ?

2. Journal drive using a same exact drive used to make the raid 5 array dedicated to only this function - can't be used as hot spare -etc

No questions on this option - self explanatory

3. Journal drive using Intel Optane Drive which has a much higher DWPD rating of 30 to 100 -what is the minimum size I can use for

a. qty (3) 6.4TB ssd drives rated at 3 DWPD setup raid 5 ?

b. qty(3) 3.2 TB ssd drives rated at 3 DWPD setup raid 5 ? --- This raid 5 array will be increased in size within a year with an extra 3.2 drive ( there are qty(10) slots free in the server .

The cost is so high on the optane drives - if I need anything bigger than 500gb , then the cost exceeds simply going option 2 route .

I'm not getting a lot of help from any of the Server Vendors.

Lenovo- Unfortunately, my sales rep at Lenovo is still confusing the Journal drive for the hot spare drive .....He's the only rep they have to quote servers for the US market from what everyone at Lenovo has told me when I ask for someone else to help me purchase a server.

ThinkMate - My sales rep and his tech support specialist have never heard of the term "journal/journaling drive" for VROC.

ThinkMate also told me they stopped selling any SuperMicro based servers using VROC, and now using mostly Gigabyte/ASUS to support VROC because all the issues they've had with SuperMicro using VROC.

Dell - Normally we would purchase a Dell server , however they don't have any PowerEdge Servers or Precision Workstations that support VROC- per our latest Dell Sales Rep and his senior server sales tech specialist - the sales tech specialist told me that VROC just really never took off like it was supposed to in his opinion , and he speculated that is why they don't offer it.

All the laptops we purchase today have an SSD hard drive using the faster PCIe NVME interface -I really don't want to settle on spending a large amount of money for SSDs in a Server limited to an HBA controller, and the drives using the much slower SATA 6gb or the SAS 12GB interface - there is almost no cost difference between the U.2 drives using PCiE, SATA 6gb , or the 12GB SAS interface.

Thank you again for your help

BrusC_Intel · ‎10-03-2022

Hello, SteveP2.

Good day,

Allow us to review your inquiries and I will get back to you as soon as possible.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎10-04-2022

Hello, SteveP2.

Thank you for your patience.

For question one, both methods present a performance penalty during write-intensive workloads. We cannot tell a specific percentage or expected difference because it depends on the drive performance, and the penalty is typically less with the Journaling method because there is one extra element (SSD) taking care of the "journal writings", while in Distributed mode, since the RWH journal is written in the same RAID members, there is an additional task being performed on all drives, hence the slight performance impact during write-intensive workloads.

Regarding question two, there is no specific size ratio for the journaling drive or exact SSD recommendations we can provide. As a general rule, the journaling drive should be at least as big as the smallest drive in the RAID volume, but the reason behind that is because of the amount of writings being done to the journalist drive. The Optane SSDs option is mentioned because of their higher endurance compared to NAND SSDs. That makes the Optane SSDs a very good candidate to work as the journalist drive. In that scenario, the journaling drive could be of less size capacity, but that is compensated by the much higher endurance of the drive.

Something important to highlight is that this refers to Data Center Optane SSDs which are the ones with the higher endurance ratings. not the consumer Optane SSDs.

If you have other questions, just let me know, and I will follow up on October 7th.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎10-05-2022

Hello, Steve.

Good day,

I wanted to follow up on this ticket today because I was informed about another support ticket under your name.

Based on the case details and what the agent shared with me, you still have some concerns regarding the information we shared, so I would like to share the following.

Regarding the Journaling drive, the size requirement is taken into consideration due to endurance concerns, since this drive requires to be performing "write" operations constantly, with an Optane DC SSD the endurance is not a concern since these will generally have a higher endurance than any of the RAID members, but this can be verified in the Intel ARK website.

For example, the Intel DC SSD P4510 4.0TB has an endurance of 6.3PBW, while an Optane DC P4801X 100GB has 10.9PBW, but again, this becomes feasible depending on availability and current market price, which we do not handle directly from the support side, and you need to review this with a distributor; Distributed gets rid of these concerns and the extra cost.

Regarding the performance impact in exact percentage, it entirely depends on your system configuration and the workload being applied to the drives, so the only way to determine the impact would require you to run the tests personally with and without R5WH, and to do it in both both modes (Journal and Distributed) applying the same workload. Any numbers we share here will not be replicated truthfully in your configuration due to the many variables involved.

I also submitted the feedback you shared regarding the support received from different system integrators while looking for information regarding Intel VROC and these particular questions.

If I can help with any questions, just let me know, and I will follow up on October 10th just in case.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎10-10-2022

Hello, SteveP2.

I wanted to follow up on this thread to check if you had any other questions.

I will keep the ticket open and will follow up one last time on October 13th before closing the thread.

Regards,

Bruce C.

Intel Customer Support Technician

SteveP2 · ‎10-11-2022

Good afternoon

Do you have any contacts at Lenovo tech support ?

I’ve given up on the journaling drive and simply going back with the original Lenovo tower model ST650v2 - the whole journaling drive is too overwhelming for the sales team - they keep confusing it with the hot spare drive -
So now all I’ve asked for is some verification from Lenovo sales that at least the “distributed” protection is enabled to protect the RAID 5 array from any R5WH event - their answer -“they think so “ but nothing in any written form - I’ve searched the internet and can’t find any information showing the setup of a VROC raid 5 array on this series of Lenovo servers and from all the PDFs and You Tube videos - nothing shows bios setup settings for Raid 5 write hole protection with VROC - not even a simple check box for enabled - and definitely no further settings that allows you to choose between “distributed” or “journaling drive protection” - I’ve opened a call to Lenovo tech support and was able to get a tech to try and send this question to their level 2 support group - he said no guarantees they will answer it since we haven’t purchased the system yet.
Maybe this is why Dell doesn’t offer any servers or workstations that support VROC ?
Do you know of any other server manufacturers that fully support VROC and have all the settings to protect the raid 5 array ?

BrusC_Intel · ‎10-12-2022

Hello, SteveP2.

We do not have any direct Lenovo contacts we can share, I'm really sorry to hear about your experience with them.

I can tell you for sure there are options in the UEFI HII and also the Windows GUI to enable the RWH using Distributed or Journaling mode.

Checking the Lenovo documentation for VROC, I see the option is present in their system BIOS, even if it is not explained in detail, so it should be possible for you to configure it. See "Figure 11" on page 11, "RWH policy" setting is present:

- https://lenovopress.lenovo.com/lp1466.pdf

Unfortunately, I do not have access to a Lenovo system, but I can show you what it looks like on a local Intel platform. Please see the screenshot below.

Intel Server M50CYP platform:

As you can see in the top-right corner, to select the journaling drive, it is necessary to mark it as such in the SSD list menu before creating the RAID 5 and enabling RWH, and this RWH configuration is most likely the same on their platform; there should be no reason for this to be removed or not implemented.

Regarding other system recommendations, I can share with you the list of official Intel distributors and you can contact them directly for guidance on how to acquire specific systems based on your needs, I just don't know if they also offer assistance with OEM platforms:

- https://marketplace.intel.com/s/pmp-partner-program/a723b0000008PICAA2/distributor?language=en_US

I really hope this helps, if I can help you with anything else, please let me know, and I will follow up on October 17th just in case.

Regards,

Bruce C.

Intel Customer Support Technician

SteveP2 · ‎10-12-2022

Thank you - apparently nobody at Lenovo I’m working with knows this document exist.

Based on Lenovo's limited knowledge of VROC and all the other unknowns and possible headaches- I think it’s time to give up on RAID 5, and just use RAID 10

Please tell me if you agree- If we go raid 10 instead of raid 5 , there is no need to setup a journal drive whether it be a same exact drive in the Raid 5 array , or some special/expensive optane drive and NO need to enable the RWH setting since that only applies to RAID 5 arrays ?

Am I reading the published VROC RAID 5 performance results correctly ? - All the specs I see comparing raid 0/1/5/10 show raid 5 has the worse performance - almost 1/5 on some of the specs ? - also I can’t tell if the specs presented in the lab tests are for RAID 5 with the RWH policy even enabled , and set to the performance hit setting of “distributed “ so they could actually be worse? - so on top of this definite documented performance hit, nobody can answer how much worse the performance will be using distributed over journal drive method for the RWH setting.

I did get another responce from Lenovo today. Instead of my Lenovo sales tech emailing me the correct document you found with the R5WH answer , he told me he couldn’t find anything, and then went on to try and sway me away from VROC altogether back to HBA hardware raid. It makes no sense why you found the correct pdf but he couldn’t ?
Here is what he had to say about the VROC technology-Quote1-“ VROC can be useful for some use cases but it also comes with performance hits to the CPUs as well as no protection from data corruption in the event of a power or system failure.” And goes on further stating latency time is the only benefit to going with VROC - quote2 -“ Thus, unless your workload *really* needs that improvement in latency I still recommend going with some SAS SSDs instead and a Hardware RAID controller with Flash-backed Cache”
Further disparaging VROC raid as NOT an “Enterprise level” technology- Quote3 - “ Using VROC for RAID 1 is safer as there’s no write delay due to Parity calculation, thus why it is more well documented, but for a proper enterprise grade solution we recommend a Hardware RAID configuration.”
Are any of the three quotes correct ? From what I see 1 and 2 are completely wrong - and he has nothing to backup quote 3 especially if you agree his first two quotes aren’t correct .

BrusC_Intel · ‎10-13-2022

Hello, SteveP2.

Good day,

Regarding RAID 10, that is correct, there is no need to to have an "additional" drive in standby, except if you want to set a spare drive to replace one of the RAID members right after the failure of one of the members.

Regarding the performance, I'm not sure what document you are referring to, but you can use the following article and its included documents for reference:

- https://www.intel.com/content/www/us/en/support/articles/000032376/memory-and-storage/ssd-management-tools.html

- Indeed, there are no details on the R5WH as this is not a concern during performance testing.

I don't really agree with the comments, but they may be based on internal testing with their own platforms and VROC implementation, so I can't say.

For example, I'm not aware of those "CPU performance hits" mentioned, as any I/O operation will require the involvement of the CPU some way or another, and it is not like CPU cores are destined or locked for VROC. Also, VROC is an enterprise RAID solution and can really help due to its simplicity, upgradeability and cost efficiency, compared to the cost of a RAID controller card, of course, it all depends on what the client needs. Let's say, in scenarios where you would like to use RAID configurations other than 0/1/5/10, or if the required HW implementation is not yet validated for VROC usage (like a new OS, distribution, version or custom hardware).

There is a document with a comparison between hardware and VROC RAID that may be used for reference:

- https://www.intel.com/content/dam/www/central-libraries/us/en/documents/vroc-vs-hba-performance-comparison.pdf

I will follow up on October 18th just in case.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎10-17-2022

Hello, SteveP2.

I wanted to follow up on your thread to check if you had any other questions.

I will follow up again on October 20th.

Regards,

Bruce C.

Intel Customer Support Technician

BrusC_Intel · ‎10-20-2022

Hello, SteveP2.

This post is to inform you about the closure of this support thread regarding Intel VROC.

We previously discussed different details regarding VROC RAID configurations, specially RAID5, but there was no confirmation about further assistance being required after trying to follow up.

This thread will be closed right now, and no longer monitored by Intel support, but if you require any type of assistance from Intel in the future, please open a new thread, or contact us using any of the available support methods:

- https://www.intel.com/content/www/us/en/support/contact-intel.html

Best regards,

Bruce C.

Intel Customer Support Technician