Community
cancel
Showing results for 
Search instead for 
Did you mean: 
SDoni
Beginner
3,388 Views

High write latency on Intel Pro 1500 SSD

Hello!

I recently upgraded my laptop with a brand new Intel Pro 1500 SSD (180 GB). While trying out various measurements using Linux tools - I'm a performance addict - I discovered something interesting.

 

I've recorded some IO operations (mostly write sync) with latency of about 170 ms ! I couldn't believe it: an SSD 1 OOM slower than an HDD?

Workload: normal working sessions, no stress at all. OS is Ubuntu 14.04 (trim enabled by default via a crontab weekly job), application running were Chromium browser and a Win 7 KVM VM with MS office open. Filling status:

$ df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda5 101G 71G 25G 74% /

none 4,0K 0 4,0K 0% /sys/fs/cgroup

udev 3,8G 4,0K 3,8G 1% /dev

tmpfs 768M 1,3M 767M 1% /run

none 5,0M 0 5,0M 0% /run/lock

none 3,8G 38M 3,8G 1% /run/shm

none 100M 68K 100M 1% /run/user

/dev/sda2 57G 33G 24G 58% /media/ste/Windows7_OS

Here is an example of the latency data, which I recorded and confirmed with multiple tools: Brendan Gregg ftrace scripts (https://github.com/brendangregg/perf-tools brendangregg/perf-tools · GitHub), blktrace/blkparse/btt, iostat -x 1.

Example:

sudo ./iolatency:

>=(ms) .. <(ms) : I/O |Distribution |

0 -> 1 : 0 | |

1 -> 2 : 0 | |

2 -> 4 : 0 | |

4 -> 8 : 1 |# |

8 -> 16 : 1 |# |

16 -> 32 : 0 | |

32 -> 64 : 0 | |

64 -> 128 : 0 | |

128 -> 256 : 32 |# |

sudo ./iosnoop:

Tracing block I/O. Ctrl-C to end.

COMM PID TYPE DEV BLOCK BYTES LATms

jbd2/sda5-166 166 WS 8,0 227094344 53248 177.84

0 WS 8,0 227094448 4096 2.02

kworker/u16:0 20402 W 8,0 178377528 4096 175.48

kworker/u16:0 20402 W 8,0 178377472 4096 175.46

kworker/u16:0 20402 W 8,0 230578432 4096 175.46

kworker/u16:0 20402 W 8,0 178377696 4096 175.50

kworker/u16:0 20402 W 8,0 230431976 4096 176.35

kworker/u16:0 20402 W 8,0 230409184 4096 176.36

kworker/u16:0 20402 W 8,0 121944064 4096 178.21

kworker/u16:0 20402 WM 8,0 121944088 4096 178.24

kworker/u16:0 20402 WM 8,0 121944112 4096 178.24

kworker/u16:0 20402 WM 8,0 159693064 4096 178.79

kworker/u16:0 20402 WM 8,0 205830272 4096 178.81

kworker/u16:0 20402 WM 8,0 226801776 4096 179.03

kworker/u16:0 20402 WM 8,0 285522744 4096 179.04

kworker/u16:0 20402 WM 8,0 289716352 4096 179.30

kworker/u16:0 20402 WM 8,0 289716480 4096 179.32

kworker/u16:0 20402 WM 8,0 289717120 4096 179.56

kworker/u16:0 20402 WM 8,0 289782040 4096 179.58

kworker/u16:0 20402 WM 8,0 302301320 4096 179.59

kworker/u16:0 20402 WM 8,0 306493488 4096 179.59

I have tried the same measurements on a older and cheaper SSD (with the same OS and applications) and did not see such latency peaks.

Am I missing something? What could be the cause of this?

Thank you!

0 Kudos
15 Replies
Jose_H_Intel1
Employee
112 Views

Hello stef3a, the following information may be helpful:

SDoni
Beginner
112 Views

Hello Joe,

Thanks for the reply.

I have already checked the OS-level knobs, no further tuning seems to be needed on a stock Ubuntu 14.04. I have checked for:

- partition alighment

- TRIM use and actual functionality

- ext4

- noatime on fstab

As regards the benchmarks, I think you missed my point. What I have provided is a trace of IO latency, which is basically a measure of IO latency during real-life workloads. My point was not related to getting low benchmark scores, which doens't seem the case by the way. I have tested the SSD with fio on random read and write workloads on 8 GB datasets, as per Intel procedures, and got results in line with the specs.

Despite this, during actual use (and light load!), the device is showing horrible latency peaks.

Any hint about that?

SDoni
Beginner
112 Views

Here are some iostat -x 1 statistics:

Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util

sda 0,00 0,00 1,00 0,00 4,00 0,00 8,00 0,18 176,00 176,00 0,00 176,00 17,60

sda 0,00 0,00 0,00 1,00 0,00 12,00 24,00 0,18 176,00 0,00 176,00 176,00 17,60

sda 0,00 10,00 0,00 33,00 0,00 172,00 10,42 5,79 175,39 0,00 175,39 5,45 18,00

sda 0,00 0,00 0,00 3,00 0,00 16,00 10,67 0,52 173,33 0,00 173,33 58,67 17,60

sda 0,00 0,00 0,00 3,00 0,00 32,00 21,33 0,52 172,00 0,00 172,00 57,33 17,20

sda 0,00 0,00 0,00 2,00 0,00 16,00 16,00 0,34 172,00 0,00 172,00 86,00 17,20

sda 0,00 1,00 0,00 23,00 0,00 108,00 9,39 3,94 171,13 0,00 171,13 7,83 18,00

sda 0,00 0,00 0,00 14,00 0,00 60,00 8,57 2,34 166,86 0,00 166,86 16,00 22,40

sda 0,00 0,00 1,00 22,00 4,00 148,00 13,22 3,68 159,83 0,00 167,09 7,65 17,60

sda 0,00 12,00 0,00 8,00 0,00 236,00 59,00 1,10 138,00 0,00 138,00 27,50 22,00

As you can see, with a very low write IOPS (even 1 per second!), the response time is constantly above 170 ms.

Jose_H_Intel1
Employee
112 Views

Please allow us more time to investigate about this.

SDoni
Beginner
112 Views

Hello Joe,

Any update on this one?

Jose_H_Intel1
Employee
112 Views

What results do you get with no programs running, including the Windows* 7 VM?

SDoni
Beginner
112 Views

Hello Joe,

Same results even with no VM,

Jose_H_Intel1
Employee
112 Views

Please allow us more time to investigate.

SDoni
Beginner
112 Views

Hi Joe,

Any news on this one?

ASouz7
Honored Contributor II
112 Views

Hello Stef3a,

We are checking with our engineers and as soon as we have an answer we will let you know.

ASouz7
Honored Contributor II
112 Views

Hello stef3a,

We have tested the same drive and did not found the sames latencies. In this particular case we recommend that you contact your nearest Intel Contact Center at your earliest convenience for extra support and other options. Here is the link: http://www.intel.com/p/en_US/support/contactsupport Contact Support - In you are located in The United States, you may call this number: 916 377 7000 Mon to Fri from 7:00 A.M to 5:00 P.M. (PST)

KPodl
Beginner
112 Views

This is still not solved , I did manage to get in W540 new PRO 1500 180GB, and until I secure erased it I had write level as low as 2MB/s ... after secure erasing, I did formatted it to only 135GB (25% OP) and now all is back as it should be, qestion is , FW is LSTi (CVDA407600YX1802GN) - so I know that this is SF-2281, and it has his "ups-and-downs" (not so PRO choice as a controller IMHO) but .. does it have new FW? Will it handle as a drive in USB case (how good GC process is?) ?

ASouz7
Honored Contributor II
112 Views

Hello kpodlaski,

The drive will work just fine connected via USB, only the performance will be impacted by the USB connection. As for the firmware, the LSTi is the latest version for the time being.

KPodl
Beginner
112 Views

That's great to know, I already see limitations of cheap USB3.0 enclosures - so that is what I'm aware. More what I'm curious is how well Garbage Collection will help (having +25% space available). Also ... is it worth having Thunderbolt case (as I've learned lately, Thunderbolt should show drive as SATA device , and therefore allow to pass Trim command to the drive from Intel SSD Tool box) was it tested?

ASouz7
Honored Contributor II
112 Views

Hello kpodlaski,

We can't guarantee it as we have not tested these devices. In theory it should work. If you are willing to try, please know that there is no guarantee that work.

Reply