p4608 detected as two devices, not displayed in vmware if using vroc

[EVers]

i have just recently purchased a super micro server;

SUPERMICRO 1U LGA3647 SP 24DIMM 10X2.5" RPSU 750W

SKL-SP 4110 8C/16T 2.1G 11M 9.6GT UPI

256GB DDR4-2666 2Rx4 LP ECC REG DIMM

Intel® SSD DC P4608 Series (6.4TB; 1/2 Height PCIe 3.1 x8; 3D1; TLC) Generic Single Pack (this is meant to be the primary drive to run VMWARE and store running VM Data)

6 x Seagate 3.5" 4TB SATA 6Gb/s 7.2K RPM 128MB Makara BP (512N) (this is the secondary drive raided in raid 10, used to store backups on)

ConnectX®-4 EN network interface card, 40/56GbE dual-port QSFP28, PCIe3.0 x16, tall bracket, ROHS R6

as stated above the Intel P4608 drive was purchased because of its speed and reliability based on the stats on intels website.

this must be the primary drive

when i got the server in the bios this drive displays as two controllers and two 3TB drives;

[windows server 2012 recognizes this as two controllers and two 3TB drives]

[Centos 7 recognizes this as two controllers and two 3TB drives]

esxi 6.5 update 1 however recognizes two controllers but only a single 3TB drive

my problem is two fold, first why is this single device displaying as two and how do i get it to display as a single device?

second why does esxi 6.5 only pick up one drive when every other OS displays it as two?

as a side note i discovered vroc settings on the server and was able to raid the device together, however esxi 6.5 does not pick up anything then.

please assist

[idata]

Hello Eugene.Verster,

Thanks for contacting the Intel® Virtual RAID on CPU (Intel® VROC) Support, in regards to your question on why the Intel® SSD DC P4608 6.4Gb is showing as 2 Controllers and 2 Drives it's because it consists of a Dual x4 NVMe Controller in one PCIe card; in order to show the SSD as a single Drive you'll need to create a RAID0 with both drives. About the drive not showing up when using Intel® VROC, we'll like to know which VROC Key you're currently using and which driver version you've installed, since this technology requires the use of VMD (Volume Management Device) enabled drivers and the use of a Physical VROC key. By checking on the VMware* Website I've found that the P4608 Drive it's supported on your VMware* version, please find the compatibility information, drivers and current information from your OS manufacturer here:http://https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=43965 https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=43965NOTE: Any links provided for third party tools or sites are offered for your convenience and should not be viewed as an endorsement by Intel® of the content, products, or services offered there. We do not offer support for any third party tool mentioned here.Additionally, please remember that if you require more assistance or want to add additional details, we'll be glad to assist youBest regards,Chris

[EVers]

sorry commented bellow.

[EVers]

Hi Chris,

thank you very much for you response.

late last night i figured i could load the following drivers https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI65-INTEL-INTEL-NVME-12115&productId=614 Download VMware vSphere , the link you sent did not go anywhere, so i'm not sure if i'm doing something stupid.

the drivers i loaded allowed me to see both drives (when vroc is not configured) in vmware, although i now have a new issue where the drive is performing at half the advertised speed.

as per https://ark.intel.com/products/96958/Intel-SSD-DC-P4608-Series-6_4TB-12-Height-PCIe-3_1-x8-3D1-TLC Intel® SSD DC P4608 Series (6.4TB, 1/2 Height PCIe 3.1 x8, 3D1, TLC) Product Specifications i should expect read speeds of about 6GB /sec and write of 3GB / sec

the benchmark tool i used i get read speeds of 3GB /sec and write speeds at 1.8GB /sec

i think that may be a driver issue as the driver i used is not precise for my drive type, however i am not sure.

if it is a driver issue please can you check the link above and resend the correct one, i would appreciate it.

with the VROC issue if i check the settings in my bios, under chipset configuration / IIO Configuration / Intel VMD Technoogy/ Intel VMD for Volume Management Device on CPU / i enable the slot that the P4608 is plugged into and also enable two NVME devices.

after that the NVME Devices no longer display in the root of the advanced option but display under Intel Virtual Raid on CPU, under there i am able to add new virtual raid drive. however VMWare does not see this as a single device.

my new question would be would purchasing a Vroc key make a difference;

i.e. would VMWARE (6.5) see the new single drive as a single drive,

would there be an increase in performance (higher than advertised not higher than what i am receiving currently)

what would your advice be?

i am desperate here as i recommended this drive to my boss based on the specs.

[idata]

Hello Eugene.Verster,

Thanks for your reply and in regards to the link, it seems that according to the VMware* website the link that you've posted it's not for the most recent drivers for your drive (1.2.1.15); please copy and paste the following one that corresponds to the latest version for your platform (1.4.0.1016) : https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESX65-INTEL-INTEL-NVME-VMD-1401016&productId=614NOTE: Any links provided for third party tools or sites are offered for your convenience and should not be viewed as an endorsement by Intel® of the content, products, or services offered there. We do not offer support for any third party tool mentioned here.Please check if your drive it's currently configured as a RAID0, since it requires that kind of configuration to work at full speed and full capacity. In regards to the Key, actually it's required to Officially Support Intel® Virtual RAID on CPU (Intel® VROC) on the supported platforms so it'll enable all the features that it offers. Additionally, please remember that if you require more assistance or want to add additional details, we'll be glad to assist youBest regards,Chris

[EVers]

Hi Chris, i will check the driver out when i reload the server tomorrow,

currently i have turned off vroc because vmware does not recognize the raid 0 drive.

if i load these drivers would vmware see the raided drive?

are you suggesting that i purchase a vroc key?

if i do, are you saying that vmware will recognize the raided volume and that it will perform at the advertised speeds?

sorry, just to clarify

[idata]

Hello Eugene.Verster,

Thanks for your reply and because this drivers aren't provided by Intel you'll need to refer to your OS/Drivers Manufacturer (In this case VMware*) so they can provide you additional information about them and if you'll be able to see the RAID within the OS. About the Intel® Virtual RAID on CPU (Intel® VROC) Key, since the hardware key is the mechanism to obtain a license to the Intel® VROC software required to Officially Support it and also the Performance varies depending on system configuration as stated on our Frequently Asked Questions about Intel® Virtual RAID on CPU (Intel® VROC) available here: https://www.intel.com/content/www/us/en/support/articles/000024550/memory-and-storage.html https://www.intel.com/content/www/us/en/support/articles/000024550/memory-and-storage.htmlWe strongly recommend you to contact your OS Manufacturer and test with the latest drivers to ensure proper compatibility; and please feel free to ask more questions and to add additional details to your case if you need, we'll be glad to assist youBest regards,Chris

[EVers]

Hi Chris,

thanks for the response, some feedback.

i raided the volumes using virtual raid0, this presented fine in Centos7, and i installed KVM.

if i run a virtual in KVM my disk tests are max read 9GB/sec but my write is only max 2GB/sec (intel advertises 3GB/sec.

i am still waiting for a response from the OS manufacturer for the ESXi not seeing the virtual raid, however on everything i have read i would suspect that VROC is not supported on ESXi.

regards

'Eugene

[idata]

Hello Eugene.Verster,

Thanks for your reply and in regards to your performance situation with the Intel® SSD DC P4608, I'll like that you please provide us the Motherboard Model of your equipment; also we want to remind you that in order to use your P4608 at the advertised speeds you'll need to connect it into a PCIe 3.1 x8 port into your motherboard. In regards to the VROC compatibility please keep us informed on this situation so we can further assist you. We hope you find this information useful for your situation and please remember that if you require more assistance or want to add additional details, we'll be glad to assist youBest regards,Chris

[EVers]

Hi Chris,

thanks for the reply,

if i look in the bios the motherboard identifies as a supermicro X11DPU

the slot that the P4608 is inserted in as AOC-2UR68-m2TS Slot2 PCI-E 3.0 X16 OPROM

i am assuming that the 3.0 / 3.1 is a version number and could be achieved from a bios upgrade?

regards

Eugene

[idata]

Hello Eugene.Verster,

Thanks for your reply and in regards to your situation actually PCIe 3.0 and 3.1 are different Electrical Specifications that can't be upgraded; please to refer to your manufacturer for information about the specifics on Performance on this system.Please remember that if you require more assistance or want to add additional details, we'll be glad to assist youBest regards,Chris

[EVers]

Hi Chris,

i just wanted to thank you for the assistance on this, we have deceded to not go the vroc route because vmware currently does not support it.

for other readers that are having the same issue, loading the drivers above solves vmware seeing both volumes.

regards

Eugene

[idata]

Hello Eugene.Verster,

Thanks for your time and we're sending you this message as a follow-up to your question and to see if the information that we've provided you was useful for your situation. Also, please remember that if you require more assistance, want to add additional details, or if all your questions have been solved you can also ask us to mark this thread as solved; we'll be glad to assist you.Best regards,Chris

[idata]

Hello Eugene.Verster,

Thanks for your time and we're sending you this message as a follow-up to your question and to see if the information that we've provided you was useful for your situation. At this point we'll mark this question as solved; however, if you want to ask more questions please fell free to contact us, we'll be glad to assist youBest regards,Chris