Internal disk cache policy in battery backed raid array (RSP3MD088F)

[Julian1234]

Hello,

I'm setting up a RAID disk array on a RSP3MD088F RAID controller with Battery Backup.

I'm enabling the controllers WriteBack cache, but I'm not sure about the disks' internal cache. 

Is there any guaratee from the RAID controller that the information on the controller's cache, sent to the disks, is mantained until it receives actual confirmation from the drive?

In other words, is there any race condicion on which a power failure after the information is sent to the disks, but before the data is actually written in the disk platters, could result in data loss?

The command to configure: sudo storcli64 /c0 /v0 set pdcache=off/on

Thanks,
Julian.

[IntelSupport]

Hello Julian1234,

Thank you for posting your question on this Intel® Community.

The write back cache policy for Intel® RAID Controllers can be enabled on a logical drive basis when a RAID controller cache battery backup module (BBU) is installed. The BBU will hold data in cache for a period of time in the case of a power interruption and then complete the data write to hard disk once power is returned. The amount of time the battery will hold data in cache is dependent on the controller memory and battery configuration.

For Intel® RAID Adapter RSP3MD088F, you can use the Intel® RAID Maintenance Free Backup AXXRMFBU7 (RMFBU). It is used to save RAID Cache in event of unexpected power loss. The RMFBU can constantly maintain the RAID volume in write back mode even when doing learn cycles.

If you are using this component, you can find additional information in section 7.1 Intel® RAID Maintenance Free Backup Unit AXXRMFBU7 (Page 40) of the Intel® RAID Adapters RSP3TD160F, RSP3MD088F, RSP3DD080F and RSP3WD080E Hardware User Guide.

Link: https://www.intel.com/content/dam/support/us/en/documents/server-products/raid-products/TriMode_Adapter_Hardware_User_Guide.pdf

Intel® RAID Adapters provide reliability, high performance, and fault-tolerant disk subsystem management. A complete fault-tolerant strategy requires protection of all data, including the unwritten cached data in the RAID card’s RAM cache. If power is lost, the data in the RAM cache is lost. To avoid losing this data, a RAID Maintenance Free Backup Unit (RMFBU) can be added to the configuration.

During normal system operation, the RMFBU monitors the voltage level of the DRAM of the Intel® RAID Adapter or Intel® RAID Adapter. If the voltage drops below a predefined level due to an AC power failure or brief power outage, the RMFBU protects the integrity of the cached data by providing sufficient back-up power to offload the data from the RAID RAM to the NAND flash. When the voltage level returns to unacceptable level, the RAID RAM is recovered from flash, and all pending writes to storage devices are completed without losing any data.

The cache memory available on Intel® RAID Adapters and Intel® RAID Adapters can improve the overall system performance. Writing data to the adapter’s cache memory is much faster than writing data to a storage device. Write operations appear to complete very quickly at the software application level. The Intel® RAID Adapter writes the cached data to the storage device when system activity is low or when the cache is getting full. The risk of using write-back cache is that the cached data can be lost if the AC power fails before it is written to the storage device. This risk factor is eliminated when the Intel® RAID Adapter has an RMFBU installed. In addition, the RMFBU provides an alternative to using an Uninterruptible Power Supply (UPS) or can act as a second level of fault tolerance when used with a UPS. Furthermore, it eliminates the need for lithium ion (Li-ion) batteries traditionally used to protect DRAM cache memory on PCI RAID adapters. Therefore this is a greener and lower total cost cache protection solution.

I hope you find this information helpful. If you have further questions, please let us know.

Wanner G.

Intel Customer Support Technician

[Julian1234]

Hi, 

Thanks for the reply and the information, but my original question remains unanswered.

I know that the CONTROLLER's cache is backed up by the battery/capacitor, and that in case of power loss it will write the controller's RAM contents to the flash memory.

My question is related to the internal cache of disk Drivers. I'm using 12TB WD disks with 64Mb of internal cache. What happens when the controller writes data to the disk, which ends up in the disk's write cache, later writte to the disk platters. What happens if there's a powerloss in this time window? Does the controller think the data has been actually written, thus potentially not providing protection for that information? Or does it have knoledge that the data, although sent to the disks, has not been actually written?

Disabling the disk's internal cache reduces perfoemance (i've tested IOPS, and there is quite a difference, and may reduce lifespan of the disks, specially SSDs.

I've been searching for some time for the information, but nothing pops up. The only reference I got was that newer 3ware raid controllers have a propietary method for assuring the disks internal write cache, but nothing on Intel/LSI raid controllers.

1) The OS writes something to a RAID volume
2) The controller caches it, and it's backed up by the BBU
3) The controller writes data to the disks when appropiate
4) The data arrives to the internal disk write cache
5) There's a power loss before it's actually written to persistent media
6) After powering on again, is the data lost at 5 still in the controller's cache vault?

Thanks!
Julian.

[IntelSupport]

Hello Julian1234,

In the presence of a battery module, and although I/O performance may be lower, hard disk drive write-back cache is disabled by default because data can potentially be lost if a power outage occurs.

As you have pointed out, enabling the HDD write-back cache may improve performance. However, when enabled, it is recommended to use an uninterruptible power supply (UPS) device to prevent data loss during power outages. In other words, enabling the disk cache in Write-back mode increases the risk of data loss due to power failure.

In this scenario, RAID controller firmware automatically checks for the presence of the battery module. If found, it allows the write back cache option.

In addition, adapters that support the battery module include a “dirty cache” LED. When power is lost to the system and data remains in the cache memory that has not been written to disk, the LED signals that this operation needs to be completed. Upon reboot, the data in memory can then write to the hard disk drive.

Please let me know if this information addresses your inquiry.

Wanner G.

Intel Customer Support Technician

[IntelSupport]

Hello Julian1234,

I hope the information provided addressed your questions. 

If you need further assistance, please let me know.

Wanner G.

Intel Customer Support Technician

[IntelSupport]

Hello Julian1234,

Since I have not heard back from you, I will proceed to close this thread.

I hope you found the details provided helpful. 

Regards,

Wanner G.

Intel Customer Support Technician