Bootable VROC RAID of 3rd pary M2 SSDs on x299 MB/ i9-7980XE set up and key options

[AAbra7]

I have carefully read threads on the VROC on x299 topic but am still struggling with the answers to these questions and am hoping you can help me:

I want build a system with a bootable VROC RAID 0 of 2 Samsung SM961(M.2) on the Asus Rampage VI Extreme X299 MB with an i9-7980XE CPU. (MB have a VROC key slot.) I want to run windows 10 64 bit pro.

Is this possible?

If so what key do I need? I understand there are three different keys:

VROCISSDMOD (intel SSDs only so I assume no)

VROCSTANMOD raid 0,1,10

VROCPREMMOD raid 0,1,5,10

Can I use a raid card, i.e. ASUS hyper M.2 x16 PCIe card with one of the 3 VROC keys above and create a bootable VROC RAID 0,1, 5,10 of 4 Samsung SM961 on the same MB configuration (Asus Rampage VI Extreme X299 MB with a i9-7980XE CPU).

Thanks for your help!

IntelBu1ld3r

[CvanR]

VROC on X299 will only work with Intel Data or Professional series SSD's on X299 - don't waste your money on this. I spent money on 4 x 760P's and a Asus Hyper M.2 card and it DOES NOT WORK. Nice trick of confusing customers for as long as possible and boost sales of sub-par Intel SSD's. Don't waste your money on this - buy AMD or use software Raid.

[dmerr6]

I have pretty much given up on the VROC. I have an asus x299 board with their hyper m.2 x 16 card. when I tried intel drives (600p) with the vroc intel ssd key, the performance was about 30% less that using samsung sm961 drives with the hyper card in pass-through mode ( no vroc key) configured as a regular stripped volume in winodws. I still may try stripping with windows storage space, but, in windows 10, you have to use windows server powershell command for it to strip across drives. very disappointing. also having problem with pcie lanes due to the way asus does their pcie buss configuration. may be looking for another system board. this vroc is definitely not ready for prime time. plus, the system board vendors have nothing in place to supply the correct vroc key and their documentation does not explain which one you need

[MLam3]

Hi insight2663,

May I ask when you say 30% less performance using 600p with VROC vs the Samsung drives, what performance metrics are you referring to? Did you bench your 600p RAID array for their 4K random performance? What sort of levels were you hitting if you did bench it?

appreciate your thoughts here as I am determined to get to the bottom of this.

many thanks

michael

[dmerr6]

actually, turns out the vroc key I received was premium. i tried retesting with the 600p drive and it was totally unstable. don't know why it worked the first time. I was able to confirm tha the only "supported" vroc with x299 chipset is ntel ssd key using intel ssd. I will have that key in the next couple of day. just for kicks, i installed two Samsung 960 pro in the motherboard m.2 slots and three sm961 in the asus hyper card. created one big stripped volume in windows and here are the stats from crystal dis mark. will redo with intel 600p in the next week or so. This whole vroc thing is not ready for prime time. very poorly documented and supported by intel and other motherboard manufacturers ( asus)

CrystalDiskMark 6.0.0 x64 (C) 2007-2017 hiyohiyo

Crystal Dew World : https://crystalmark.info/-----------------------------------------------------------------------* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 7348.278 MB/s

Sequential Write (Q= 32,T= 1) : 5992.115 MB/s Random Read 4KiB (Q= 8,T= 8) : 1507.521 MB/s [ 368047.1 IOPS] Random Write 4KiB (Q= 8,T= 8) : 1359.872 MB/s [ 332000.0 IOPS] Random Read 4KiB (Q= 32,T= 1) : 399.803 MB/s [ 97608.2 IOPS] Random Write 4KiB (Q= 32,T= 1) : 358.517 MB/s [ 87528.6 IOPS] Random Read 4KiB (Q= 1,T= 1) : 38.173 MB/s [ 9319.6 IOPS] Random Write 4KiB (Q= 1,T= 1) : 153.987 MB/s [ 37594.5 IOPS]

Test : 1024 MiB [Z: 12.1% (578.0/4768.7 GiB)] (x5) [Interval=5 sec]

Date : 2017/12/20 22:57:16 OS : Windows 10 Professional [10.0 Build 16299] (x64)

[dmerr6]

So, here are some more performance stats that I did. basically, on the x299 chipset you only have two options for Vroc. Use intel drive with the vroc key that is specifically for intel drives , or, use any drive you want and vroc acts as a passthorugh and you are really running through the nvme controllers on the m.2 drives.

test 1- samsung sm 961 on asus hyper m.2 card configured as pass through. using stripped volume in windows

Sequential Read (Q= 32,T= 1) : 7348.278 MB/s

Sequential Write (Q= 32,T= 1) : 5992.115 MB/s

Random Read 4KiB (Q= 8,T= 8) : 1507.521 MB/s [ 368047.1 IOPS]

Random Write 4KiB (Q= 8,T= 8) : 1359.872 MB/s [ 332000.0 IOPS]

Random Read 4KiB (Q= 32,T= 1) : 399.803 MB/s [ 97608.2 IOPS]

Random Write 4KiB (Q= 32,T= 1) : 358.517 MB/s [ 87528.6 IOPS]

Random Read 4KiB (Q= 1,T= 1) : 38.173 MB/s [ 9319.6 IOPS]

Random Write 4KiB (Q= 1,T= 1) : 153.987 MB/s [ 37594.5 IOPS]

Test : 1024 MiB [Z: 12.1% (578.0/4768.7 GiB)] (x5) [Interval=5 sec]

Date : 2017/12/20 22:57:16

OS : Windows 10 Professional [10.0 Build 16299] (x64)

test 2 - intel 600p ON ASUS Hyper card using vroc as raid 0 ****WRITE PERFORMANCE IS TERRIBLE*****

Sequential Read (Q= 32,T= 1) : 5524.837 MB/s

Sequential Write (Q= 32,T= 1) : 1667.870 MB/s

Random Read 4KiB (Q= 8,T= 8) : 1009.655 MB/s [ 246497.8 IOPS]

Random Write 4KiB (Q= 8,T= 8) : 1021.727 MB/s [ 249445.1 IOPS]

Random Read 4KiB (Q= 32,T= 1) : 478.627 MB/s [ 116852.3 IOPS]

Random Write 4KiB (Q= 32,T= 1) : 441.089 MB/s [ 107687.7 IOPS]

Random Read 4KiB (Q= 1,T= 1) : 26.009 MB/s [ 6349.9 IOPS]

Random Write 4KiB (Q= 1,T= 1) : 86.746 MB/s [ 21178.2 IOPS]

TEST 3 Intel 600p on Asus Hyper card in passthrough ***** not a lot of performance gain by using vroc. *****

Sequential Read (Q= 32,T= 1) : 5667.678 MB/s

Sequential Write (Q= 32,T= 1) : 1659.496 MB/s

Random Read 4KiB (Q= 8,T= 8) : 797.353 MB/s [ 194666.3 IOPS]

Random Write 4KiB (Q= 8,T= 8) : 984.298 MB/s [ 240307.1 IOPS]

Random Read 4KiB (Q= 32,T= 1) : 430.886 MB/s [ 105196.8 IOPS]

Random Write 4KiB (Q= 32,T= 1) : 325.184 MB/s [ 79390.6 IOPS]

Random Read 4KiB (Q= 1,T= 1) : 34.351 MB/s [ 8386.5 IOPS]

Random Write 4KiB (Q= 1,T= 1) : 149.533 MB/s [ 36507.1 IOPS]

So, I will be using the sm 961 on the Asus Hyper card in pass through . It is disappointing that the vroc does not see to give a lot of performance gain. and, since the intel drive are significantly slower than samsung, there is definable no advantage to using VROC

***NOTE- if you are using windows 10 storage spaces, the caching will decrease disk performance. power shell to disable is

Set-StoragePool -FriendlyName -IsPowerProtected $True

[MRobi7]

Thank you for taking time to analyse the performance.

I particularly like that you have done low Q depth tests. People keep pumping out high Q depths which look great but are terribly unrealistic for the average home workstation.

It does look rather terrible for VROC. The concept sounds great but something is causing it to be a rubbish performer when your OS is doing a better job of managing it.

I wonder how a full hardware raid card would go in this scenario?

What performance does a single drive in passthrough give in comparison to your 4 drive Samsung array?

I am unsurprised the performance of the Intel SSD's neutered the outcome as well.

[dmerr6]

still working out some pcie issues with asus, but, I agree. the whole VROC thing is not ready for prime time. I have no way to test the XEON chipsets, so, don't know if it is better there. don't have anything at work either. with the significant decrease in physical servers, I really don't understand why Intel would not be putting more efforts into the high end workstation market. anyway, hjere is the benchmark I did for a single Samsung 960 pro. The sm961 and pm961 are the OEM equivalents, so, perffomance should be similar. I am also thinking of looking at the High point hardware m.2 raid card, but, would have to sneak that past the wife.

-----------------------------------------------------------------------

CrystalDiskMark 6.0.0 x64 (C) 2007-2017 hiyohiyo Crystal Dew World : https://crystalmark.info/-----------------------------------------------------------------------* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 3145.422 MB/s

Sequential Write (Q= 32,T= 1) : 2177.067 MB/s Random Read 4KiB (Q= 8,T= 8) : 1374.100 MB/s [ 335473.6 IOPS] Random Write 4KiB (Q= 8,T= 8) : 1174.635 MB/s [ 286776.1 IOPS] Random Read 4KiB (Q= 32,T= 1) : 406.492 MB/s [ 99241.2 IOPS] Random Write 4KiB (Q= 32,T= 1) : 365.901 MB/s [ 89331.3 IOPS] Random Read 4KiB (Q= 1,T= 1) : 50.707 MB/s [ 12379.6 IOPS] Random Write 4KiB (Q= 1,T= 1) : 165.868 MB/s [ 40495.1 IOPS]

[MRobi7]

At this point looking at the difference in performance I have to think that RAID is not doing a whole lot.

These are only showing 4k r/w to be sure but the improvements for workstation activity are not great. I don't think many workstation tasks get above q depth of 8.

Any thoughts?

Also, I contacted this site to see if they can do a direct comparison of the Asus card with the highpoint: http://www.thessdreview.com/daily-news/latest-buzz/highpoint-ssd7101a-1-nvme-raid-controller-review-samsung-toshiba-m-2-ssds-tested/ HighPoint SSD7101A-1 NVMe RAID Controller Review – Samsung and Toshiba M.2 SSDs Tested | The SSD Review

[dmerr6]

correct about the queue depth. I use VMware a lot. that does really hammer disk. that's the main reason I'm looking for the best disk performance

[MRobi7]

Sounds like the entire setup is rather half-arsed.

I can see the issue from the motherboard side as they are required to accommodate all of the CPU's in the line up .... and that means different number of PCIe lanes. Unfortunately companies like Asus are so busy talking up and hyping their product that they bury any useful information regarding the actual plausible configurations.

The EVGA board and one of the Asrock ones looked pretty decent to me.

"pass-through" just means you are using CPU lanes and VROC is doing nothing? YOu have a windows based array?

In this situation do you just use an RST driver to inject into the installation if required (eg in WinPE or the OS install) to have the array recognised? I've only done hardware raid on NAS devices

I think if I go this path it will be based on a single SSD (perhaps small optane) as an OS drive, and a windows based array off a hyper m.2 or Highpoint card with 2-4 drives.

I need 4 lanes for a single 10GBe port on my X540-T2 (although intel specify 8 for both 1 and 2 port card), 8 for GPU and then 8-16 for the SSD's in RAID. Possibly 4 for the OS SSD as well. THat means 20-24 lanes so the 6 or 8 core i9 is a fair option for me whereas the z370 with only 16 lanes on CPUs is not.

Zen+ is going to hit Q1 2018. Maybe that will be more sensible than x299.

[dmerr6]

I did one last performance test. 4 samsung sm961 ( same thing as 960 pro) vs. intel 600p. I am abandoning the VROC untill Intel decides to stop being stupid. Using slower intel drives with vroc is just not as fast as samsung drives. as it stands now, it just a way for intel to try and sell slower drives. don't get me wrong, even the slowest NVME drive blows away SATA, but, intel can do much better.

intel 600p VROC raid 0

-----------------------------------------------------------------------

CrystalDiskMark 6.0.0 x64 (C) 2007-2017 hiyohiyo

Crystal Dew World : https://crystalmark.info/

-----------------------------------------------------------------------

* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 5575.639 MB/s

Sequential Write (Q= 32,T= 1) : 1660.886 MB/s

Random Read 4KiB (Q= 8,T= 8) : 1042.851 MB/s [ 254602.3 IOPS]

Random Write 4KiB (Q= 8,T= 8) : 1098.540 MB/s [ 268198.2 IOPS]

Random Read 4KiB (Q= 32,T= 1) : 572.573 MB/s [ 139788.3 IOPS]

Random Write 4KiB (Q= 32,T= 1) : 522.322 MB/s [ 127520.0 IOPS]

Random Read 4KiB (Q= 1,T= 1) : 26.372 MB/s [ 6438.5 IOPS]

Random Write 4KiB (Q= 1,T= 1) : 77.421 MB/s [ 18901.6 IOPS]

samsung sm691 in windows striped volume

-----------------------------------------------------------------------

CrystalDiskMark 6.0.0 x64 (C) 2007-2017 hiyohiyo

Crystal Dew World : https://crystalmark.info/

-----------------------------------------------------------------------

* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

* KB = 1000 bytes, KiB = 1024 bytes

Sequential Read (Q= 32,T= 1) : 7049.617 MB/s

Sequential Write (Q= 32,T= 1) : 5071.261 MB/s

Random Read 4KiB (Q= 8,T= 8) : 1378.910 MB/s [ 336647.9 IOPS]

Random Write 4KiB (Q= 8,T= 8) : 1196.641 MB/s [ 292148.7 IOPS]

Random Read 4KiB (Q= 32,T= 1) : 599.672 MB/s [ 146404.3 IOPS]

Random Write 4KiB (Q= 32,T= 1) : 439.998 MB/s [ 107421.4 IOPS]

Random Read 4KiB (Q= 1,T= 1) : 51.363 MB/s [ 12539.8 IOPS]

Random Write 4KiB (Q= 1,T= 1) : 196.865 MB/s [ 48062.7 IOPS]

[SEric]

Well, the VROCISSDMOD key is now OBSOLETE by Intel.

What are users supposed to do now?

Is there a replacement key for the functions it provided?

Is Intel abandoning the VROC function of the HEDT platform?

[dmerr6]

I've pretty much abandoned the whole Vroc thing.don't know if it works better on Xeon chipset, but, on 299, i don't think it is worth it. I am currently using the asus hyper card in pass through and getting excellent performance with Samsung 960 pro. Unless intel opens up vroc to non intel drive on 299 chipset, no way to test. I'm done for now.

[MRobi7]

YOu have to be kidding? Is there no replacement part superseding it? Perhaps a bios update that removes the requirement to use it at all? Worth checking out to see what the plan is - and if there is none I'd be contacting Linus, Gamers Nexus and any of the major streamers who will hapilly jump all over that type of crap. THis platform isn't even 6 months old and deserves better support.

From my reading a lot of boards are still needing bios updates, and the clarity around VROC is an utter joke. Same same regarding mainboards and their split of lanes for various CPU configs vs what is on the PCH. Some list it clearly but overall it's really poor. Intel has done a shit job and is just sweeping the x299 turd along the floor rather than cleaning it up.

My rant aside, I am considering dropping a x299 option and going z370 due the performance gap that the 8700k achieves over the 7800x - especially in overclocking headroom. I don't have enough focus on multicore usage to gain from the 7920x 8 core as a performance improvement.

I wanted the extra PCIe lanes but can tolerate the 16 on the 8700k.

My plan is to have the following installed:

1. GPU (1080TI) at x8

2. Asus Hyper m2 x16 with 960Evo x2 in RAID 0 at x8

3. Intel x550 10GBe NIC on x4 connector via PCH.

THe Asus ROG Maximus X Apex would be one option for this but I'm not hearing great things about the Asus z370 boards and their bios implementation. The EVGA z370 Classified K could be a choice for me - it also has soundblaster Core 3d audio not the realtek which has been causing serious problems on many z370 boards.

My final option (less desirable due to cost) is to drop the RAID idea and just buy a 240GB Optane drive - the intel 900p. I would likely use this with the ASROCK z370 Taichi, which can split its Lanes off the CPU at x8/x4/x4 so all 3 of my cards will have direct CPU access. It's $100 less than the Asus board too, so works out close to even with the RAID 0 option usng the Hero and Hyper m.2.

Any thoughts on RAID 0 Samsung 2 drive setup in passthrough vs 900p Optane configuration?

I look at reviews but too many are citing high queue counts like 32 which are completely unrealistic for desktop/workstation workloads. I'm trying to compare and evaluate queue depths up to about 8.

[SEric]

You might want to consider the Asrock X299 Extreme 4.

Then grab an i7 7800X from Silicon Lottery.

The board is 200 at Newegg and the CPU delidded and binned for 4.9GHz. is 379,

which is near what a non binned lidded cpu goes for.

Plus 28 lanes!

I have not heard any negative things about the Asrock X299 boards,

but other brands have been having bios issues.

Also Asrock just released their version of the Hyper M.2 card at CES.

The Asrock "Ultra quad" is on youtube. (Do a search it should come right up)

Video just posted today. 1/15/18

Allegedly with 900P drives, no key is needed for RAID0,

although I do have quite a few VROCISSDMOD keys here.

I will be using 4 280GB u.2-M.2 900P drives on a TAICHI XE board.

Nearing completion now.

[MRobi7]

That's good food for thought.... I'll need to look at the numbers though. I am in Australia - which means I need to look at the pricing of the silicon lottery chips vs cost here (I have ben interested in their offerings and looked over them already as a means of risk mitigation in just buying and seeing what I get).

I think the most cost effective solution for me is - z370 board (Asrock Taichi), 240GB 900p and 8700k (nearly all of which will hit at least 5GHz and I run a full custom loop and can delid if desired).

I'll compare that as a base price against the x299 options. I just rechecked some benches on the 900p and at QD1-8 it is a beast, especially in the 4k block workload. Most of my workloads are in 4k-16k with some up to 128k so I suspect in terms of latency, and responsiveness it will be best. I can use my Evo 960 in part as a scratch partition for my 10GBe transfers and work anyway which should still saturate the NIC

It will be a decent step up from my current Z79 setup (Maximus VII Hero) even though the 8700k is not a "required" upgrade from my 4790k @4.6GHz.

[dmerr6]

turns out i had a bad asus x299 deluxe. replaced it and all is working great. I may revisit VROC sometime in the future. I have the asus HyperCard with one drive. works great. may add more and test vroc if intel ever decides to put some effort into the x299

[SEric]

The X299 will initially give you an additional 12 lanes, and an upgrade path to 16 more.

Z370 is somewhat limited.

16 lanes period no matter what.

The board I mentioned has 1 less PCIe slot than most,

and the Taichi XE has onboard wifi and BT 4.2, along with 3 onboard M.2 slots,

but it is $130 US more.

Tough call.

[GDiau]

VROC Raid 0 (not so expensive Intel 760p M.2 X 4)

Bootable

Not use as bootable

Motherboard: Asus ROG Strix X299-E Gaming Bios 1102

CPU: Intel Core i9 7900X

RAM: HyperX Predator DDR4-3600 32G (8G X 4)

VROC: Hyper M.2 X16 Card: Intel 760p M.2 (256G X 4)

VROC driver: RSTe_5.3.0.1413_F6-drivers (RSTe_f6_iaVROC_win8_64)

Video Card: Nvidia P4000

OS: Windows 10 x64 Build 1790

[SEric]

Random reads are pretty low.

I am pretty sure it's a driver issue.

I have run into the situation and am actually working on it right now.

When I figure it out, I'll post results

[SEric]

I built a system with 4 900P Optane drives last year.

No key needed.

Performance is tremendous

(modded Hyp

They MUST be Intel drives, no exceptions.

My understanding is that the 760P WILL work

with the latest RSTe driver.

The PCI slot used must be set to x4x4x4x4 in the UEFI,

and you will need a CPU with enough PCIe lanes.

I7 7800X has only 28, whereas the i9 7900X has 44,

as do the higher core count CPU's.

The Skylake X "refresh" chips ALL have 44 lanes.