topic 4K format on Optane SSD P1600X in Intel® Optane™ Solid State Drives

4K format on Optane SSD P1600X

Vald — Sun, 16 Apr 2023 16:47:48 GMT

Hi,

I need to format the SSD (SSDPEK1A118GA, fw U5110550) to 4096 byte blocks.

I try to follow instructions by "Retired Principal Engineer - Intel Corporation " here: https://www.linkedin.com/pulse/linux-nvme-cli-cheat-sheet-frank-ober/

When issue "nvme format /dev/nvme3n1 --lbaf=1 --reset", the command freezes and the device becomes unresponsive until reboot.

I tried

* to give the command some time (hours) to finish

* I tried to format just one namespace as well as entire controller

* I used nvme-cli Version : 2.0+30.g86f82c58cb97

* I also tried intelmas-2.2.18 tool

intelmas start -intelssd 3 -nvmeformat LBAFormat=1

and intelmas start -intelssd 3 -nvmeformat -namespace 1 LBAFormat=1

The result is always the same - the drive freezes until system reset.

How can I format Optane SSD P1600X to 4K blocks?

Thanks!

Re: 4K format on Optane SSD P1600X

Vald — Sun, 16 Apr 2023 21:20:08 GMT

Machine:

i9-13900 on W680 chipset motherboard

OS:

Linux kernel 5.14.21 and 6.1

Re:4K format on Optane SSD P1600X

BrusC_Intel — Mon, 17 Apr 2023 22:03:43 GMT

Hello, Vald.

Good day,

I received your ticket regarding the sector size of the P1600X, and I will be reviewing this with you.

If the command does not work, it is possible this unit does not support the modification of this parameter, and I can help confirming this for you.

In the meantime, please share the output of the following command to check the SSD details:

- IntelMAS show -a -intelssd X

You may also attempt to use the following:

- intelmas start -intelssd X -nvmeformat LBAFormat=3 (3 is generally the one you need to use)

- intelmas start -intelssd X -nvmeformat LBAFormat=2 (in some other Intel Optane SSDs)

The only sector sizes supported in Intel NVMe drives are 512B and 4096B which corresponds to indexes 0 and 3 respectively.

I will follow up on April 19th just in case.

Regards,

Bruce C.

Intel Customer Support Technician

Re: Re:4K format on Optane SSD P1600X

Vald — Mon, 17 Apr 2023 22:14:13 GMT

Hi Bruce,

host:~ # intelmas show -intelssd 3

- 3 Intel SSD P1600X Series PHOC202100UJ118B -

Bootloader : UFCB5004
Capacity : 110.28 GB (118,410,444,800 bytes)
DevicePath : /dev/nvme3n1
DeviceStatus : Healthy
Firmware : U5110550
FirmwareUpdateAvailable : The selected drive contains current firmware as of this tool release.
Index : 3
MaximumLBA : 231270399
ModelNumber : INTEL SSDPEK1A118GA
NamespaceId : 1
PercentOverProvisioned : 100.00
ProductFamily : Intel SSD P1600X Series
SMARTEnabled : True
SectorDataSize : 512
SerialNumber : XXXXXXXXXXXX118B

host:~ # intelmas start -intelssd 3 -nvmeformat LBAFormat=3
WARNING! You have selected to format the drive!
Proceed with the format? (Y|N): Y
Formatting...(This can take several minutes to complete)

- Intel SSD P1600X Series XXXXXXXXXXXX118B -

Status : Invalid LBA format.

I get same error with LBAFormat=2

Regards,

Vald

Re:4K format on Optane SSD P1600X

BrusC_Intel — Tue, 18 Apr 2023 21:19:51 GMT

Hello, Vald.

Thank you for the information.

Please allow me to confirm if there are any additional details we can provide, and I will contact you back as soon as possible.

Regards,

Bruce C.

Intel Customer Support Technician

Re: Re:4K format on Optane SSD P1600X

justintocci — Wed, 19 Apr 2023 19:47:46 GMT

My experience was slightly different. I was unable to use the Freebsd nvmecontrol application to alter my P1600X. So I put it in a Windows 10 box and downloaded the Intel® Memory and Storage (Intel® MAS) Tool for Windows. I'm pretty sure I downloaded the CLI package, installed it, realized there was a GUI tool, downloaded the GUI version, ran second install, it detected the GUI was already installed and then I cancelled the second install.

I went into the GUI tool and despite the tool showing my P1600X, I clicked and right clicked all around but was unable to change anything at all. So I went to CMD, right clicked on the command prompt window and clicked "run as administrator".

I ran

>intelmas show -intelssd

That showed my P1600X as index zero. So then I ran:

C:\Windows\system32>intelmas start -intelssd 0 -nvmeformat LBAFormat=1
WARNING! You have selected to format the drive!
Proceed with the format? (Y|N): y
Formatting...(This can take several minutes to complete)

- Intel SSD P1600X Series BTOC14120WET118B -

Status : This feature cannot be run with the current driver.

C:\Windows\system32>

I also tried LBA 2 and LBA 3 with the same result. I did not get "Status : Invalid LBA format" like Vlad above but the end result is the same. Any help you can give to Vlad and I would be greatly appreciated.

Re: 4K format on Optane SSD P1600X

justintocci — Wed, 19 Apr 2023 20:01:57 GMT

My setup:

C:\Windows\system32>intelmas version
- Version Information -
Name: Intel(R) Memory and Storage Tool
Version: 2.2.18
Description: Interact and configure Intel SSDs.

C:\Windows\system32>intelmas show -a -intelssd 0

- 0 Intel SSD P1600X Series BTOC14120WET118B -

AggregationThreshold : 0
AggregationTime : 0
ArbitrationBurst : 0
Bootloader : UFCB5004
BusType : 17
Capacity : 110.28 GB (118,410,444,800 bytes)
CoalescingDisable : 1
ControllerCompatibleIDs : PCI\VEN_8086&DEV_2525&REV_00PCI\VEN_8086&DEV_2525PCI\VEN_8086&CC_010802PCI\VEN_8086&CC_0108PCI\VEN_8086PCI\CC_010802PCI\CC_0108
ControllerDescription : @stornvme.inf,%pci\cc_010802.devicedesc%;Standard NVM Express Controller
ControllerID : PCI\VEN_8086&DEV_2525&SUBSYS_380A8086&REV_00\4&1D02AF65&0&00E4
ControllerIDEMode : False
ControllerManufacturer : @stornvme.inf,%ms-nvme%;Standard NVM Express Controller
ControllerService : stornvme
CurrentPercent : 100.00
DevicePath : \\.\PHYSICALDRIVE0
DeviceStatus : Healthy
DirectivesSupported : False
DisableThermalThrottle : The selected drive does not support this feature.
DownloadMicrocodePossible : True
DriverDescription : Standard NVM Express Controller
DriverMajorVersion : 10
DriverManufacturer : Standard NVM Express Controller
DriverMinorVersion : 0
DriverProvider : Microsoft
DriverVersion : 10.0.19041.2075
DynamicNamespaceGranularity : 1073741824
DynamicNamespaceSupported : False
EUI64 : 474229047087708
EndToEndDataProtCapabilities : 0
EnduranceAnalyzer : Media Workload Indicators have reset values. Run 60+ minute workload prior to running the endurance analyzer.
ErrorString : Value not found
Firmware : U5110550
FirmwareUpdateAvailable : The selected drive contains current firmware as of this tool release.
FormatNVMCryptoEraseSupported : True
FormatNVMSupported : True
HighPriorityWeightArbitration : 0
IOCompletionQueuesRequested : The request could not be performed because of an I/O device error.
IOSubmissionQueuesRequested : The request could not be performed because of an I/O device error.
Index : 0
Intel : True
IntelGen3SATA : False
IntelNVMe : True
InterruptVector : 0
IsCommandPathNVMeMSFT : True
IsDualPort : False
LBAFormat : 0
LatencyTrackingEnabled : The request could not be performed because of an I/O device error.
LowPriorityWeightArbitration : 0
Lun : 0
MaximumDataTransferSize : 5
MaximumLBA : 231270399
MediumPriorityWeightArbitration : 0
MetadataSetting : 0
MetadataSize : 0
ModelNumber : INTEL SSDPEK1A118GA
NVMe1Point1OrGreater : True
NVMeControllerID : 0
NVMeMajorVersion : 1
NVMeMinorVersion : 1
NVMePowerState : 0
NVMeTertiaryVersion : 0
NamespaceId : 1
NamespaceManagementSupported : False
NativeMaxLBA : 231270399
NumErrorLogPageEntries : 63
NumLBAFormats : 1
NumberOfNamespacesSupported : 1
OEM : Generic
OptionalNVMCompareSupported : False
OptionalNVMDatasetMgmtSupported : True
OptionalNVMReservationsSupported : False
OptionalNVMSaveFieldSupported : True
OptionalNVMTimestampSupported : True
OptionalNVMVerifySupported : False
OptionalNVMWriteUncorrectableSupported : True
OptionalNVMWriteZeroesSupported : False
PCILinkGenSpeed : 3
PCILinkWidth : 4
PNPString : SCSI\DISK&VEN_NVME&PROD_INTEL_SSDPEK1A11\5&E720B86&0&000000
PathID : 0
PercentOverProvisioned : 100.00
PhysicalSize : 118410444800
PowerGovernorMode : 31 Unknown
Product : Ox Creek
ProductFamily : Intel SSD P1600X Series
ProductProtocol : NVME
ProtectionInformation : 0
ProtectionInformationLocation : 0
SCSIPortNumber : 1
SMARTEnabled : True
SMARTHealthCriticalWarningsConfiguration : 31
SMBusAddress : The request could not be performed because of an I/O device error.
SMI : False
SectorDataSize : 512
SectorSize : 512
SelfTestSupported : True
SerialNumber : BTOC14120WET118B
SupportCommandEffectsLog : True
TCGSecurityState : Unsupported
TCGSupported : False
TargetID : 0
TelemetryLogSupported : False
TempThreshold : 85
TimeLimitedErrorRecovery : 0
TrimSupported : True
Vendor : NVMe
VolatileWriteCacheEnabled : The request could not be performed because of an I/O device error.
WriteAtomicityDisableNormal : 0

Re:4K format on Optane SSD P1600X

BrusC_Intel — Wed, 19 Apr 2023 21:35:18 GMT

Hello, Vald.

Thank you very much for your patience.

I just received confirmation that P1600X does not support variable sector size, so it only supports 512.

If you have other questions or comments, please let me know, but if that is not the case I will proceed to close this community thread on April 21st.

Regards,

Bruce C.

Intel Customer Support Technician

Re: 4K format on Optane SSD P1600X

justintocci — Wed, 19 Apr 2023 22:56:31 GMT

I see now. if i run:

# smartctl -a /dev/nvme2

I get the following:

... for P1600X ...

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         0

...

I had not noticed before that ID 1 shows a performance of zero. The plus sign indicates the current format. Usually it would show:

... for an SSD that supports 4k block sizes ...

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

...

That would indicate that Rel_perf is better on ID 1 because the Rel_perf number is lower. Then running the format command to select ID 1 would cause the plus sign to move to ID 1.

Thank you very much for your prompt response!

Re: 4K format on Optane SSD P1600X

Vald — Wed, 19 Apr 2023 23:23:45 GMT

Hi,

The problem is with firmware then. Because the drive reports (see below)

1. 4K blocks as being supported,

2. 2K blocks providing best perfomance vs 512b blocks

Can we expect firmware update which fixes this (either allow to switch to 4K, or stop reporting it as supported)?

host:~ # intelmas show -intelssd 3 -identify
...

- Byte 128-131 -

LBA Format 0 Support : 0x02090000
Bit 31:26 - Reserved : 0x00
Bit 25:24 - Relative Performance : 0x2 (Good performance)
Bit 23:16 - LBA Data Size : 0x09
Bit 15:0 - Metadata Size : 0x0000

- Byte 132-135 -

LBA Format 1 Support : 0x000C0000
Bit 31:26 - Reserved : 0x00
Bit 25:24 - Relative Performance : 0x0 (Best performance)
Bit 23:16 - LBA Data Size : 0x0C
Bit 15:0 - Metadata Size : 0x0000
...

Re: 4K format on Optane SSD P1600X

Vald — Wed, 19 Apr 2023 23:38:05 GMT

Correction:

The problem is with firmware then. Because the drive reports (see below)

1. 4K blocks as being supported,

2. 4K blocks providing best perfomance vs 512b blocks

Can we expect firmware update which fixes this (either allow to switch to 4K, or stop reporting it as supported)?

host:~ # intelmas show -intelssd 3 -identify
...

Re: 4K format on Optane SSD P1600X

justintocci — Thu, 20 Apr 2023 00:46:22 GMT

Vlad,

I had to put this drive to use in a freebsd box so I don't have it where I can use intelmas right now. However, i do recall seeing the "best performance" somewhere. Another reason I kept going despite the many roadblocks.

However, not to be a debbie downer, but the (hexadecimal?) number for Relative Performance is still reporting zero (0x0). I would expect that 0x1 would be 'best performance' as I see in the information that other SSD devices report.

I suspect that the phrase "best performance" is hard coded into the tool for the lowest number. Perhaps the tool programmer expected an unsupported LBA to simply not be reported. Then, in the case of this SSD, I suspect it was decided to report 4k anyway so that any program that expected a second LBA line got an answer. This may not have been the only 'right' answer but I don't think it was the only 'wrong' answer either. Six of one, half a dozen of the other.

I am setting up my P1600X with a ZFS SLOG partition and a swap partition. Both use cases should perform very well despite the small block size.

In the SLOG, larger block sizes would make a difference if I expected my database sync writes to be large, but that would be unlikely. Our databases are mostly accounting apps. The average data update is probably less than 100 characters at a time.

The swap will be used for various things. A big example would be for the instant reservation of space for programs that claim they will need it but then may never actually use it. Also, actual swapping will usually be optimized by the system in such a way that the data actually needed is put back into RAM. The swap is supposed to be for sleeping programs and such.

As long as i get the great latency that I've seen reported then I am happy despite the small block size, but your point is well taken. I'm sure that the throughput for larger writes would have been even faster with larger blocks.

Re:4K format on Optane SSD P1600X

BrusC_Intel — Thu, 20 Apr 2023 21:46:19 GMT

Hello, justintocci and Vald.

Thank you very much for all the information and valuable feedback.

I can certainly submit this information to the engineering so they can determine if any actions are required.

I will close this community thread for now, and it will no longer be monitored by Intel support, but if you require any type of assistance from Intel in the future, just open a new thread, or contact us using any of the available support methods:

- https://www.intel.com/content/www/us/en/support/contact-intel.html

Best regards,

Bruce C.

Intel Customer Support Technician