- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are running IOR benchmarking on a Lustre filesystem and we see performance degradation when upgrading from cpu v3 to cpu v4;
We are seeing cpus overload under IOR benchmark, this is occurring after cpu v3 to v4 upgrade
As a result, we see performance degradation after the upgrade..
can someone take a look at the syslog to see if this is a known issue:
Syslog messages and snapshot of the cpus utilizations are below:
kernel:NMI watchdog: BUG: soft lockup - CPU# 54 stuck for 22s! [ll_ost_io01_107:24645]
Message from syslogd@OSS1-A212 at Aug 21 15:53:49 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 3 stuck for 23s! [ptlrpcd_00_13:7854]
Message from syslogd@OSS1-A212 at Aug 21 15:53:49 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 10 stuck for 23s! [ptlrpcd_00_07:7848]
Message from syslogd@OSS1-A212 at Aug 21 15:53:49 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 46 stuck for 22s! [ll_ost_io01_047:49948]
Message from syslogd@OSS1-A212 at Aug 21 15:53:49 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 48 stuck for 22s! [ll_ost_io01_092:24578]
Message from syslogd@OSS1-A212 at Aug 21 15:53:53 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 4 stuck for 22s! [ptlrpcd_00_19:7860]
Message from syslogd@OSS1-A212 at Aug 21 15:53:53 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 27 stuck for 23s! [ll_ost_io01_086:22338]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 0 stuck for 22s! [ptlrpcd_00_08:7849]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 12 stuck for 22s! [ptlrpcd_00_14:7855]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 15 stuck for 22s! [ll_ost_io01_008:50182]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 17 stuck for 22s! [kiblnd_sd_01_02:7835]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 18 stuck for 22s! [kiblnd_sd_01_01:7834]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 20 stuck for 22s! [ll_ost_io01_016:44925]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 21 stuck for 22s! [ll_ost_io01_076:22069]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 22 stuck for 22s! [kiblnd_sd_01_00:7833]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 23 stuck for 22s! [ll_ost_io01_094:24592]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 24 stuck for 22s! [kiblnd_sd_01_03:7836]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 25 stuck for 22s! [ll_ost_io01_106:24643]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 26 stuck for 22s! [ll_ost_io01_064:33777]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 36 stuck for 22s! [kiblnd_sd_00_00:7829]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 43 stuck for 22s! [ll_ost_io01_031:42636]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 44 stuck for 22s! [ll_ost_io01_109:24689]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 49 stuck for 22s! [ll_ost_io01_046:50178]
Message from syslogd@OSS1-A212 at Aug 21 15:53:57 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 53 stuck for 22s! [ll_ost_io01_029:37763]
Message from syslogd@OSS1-A212 at Aug 21 15:54:01 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 8 stuck for 22s! [ptlrpcd_00_09:7850]
Message from syslogd@OSS1-A212 at Aug 21 15:54:01 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 16 stuck for 22s! [ll_ost_io01_098:24613]
Message from syslogd@OSS1-A212 at Aug 21 15:54:01 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 32 stuck for 22s! [ptlrpcd_00_12:7853]
Message from syslogd@OSS1-A212 at Aug 21 15:54:01 ...
kernel:NMI watchdog: BUG: soft lockup - CPU# 42 stuck for 22s! [ll_ost_io01_024:39085]
Message from syslogd@OSS...
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What are the processor model numbers?
What motherboard and BIOS are you using?
What operating system are you using?
Doc
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Doc,
Operating System: Centos 7.3
cpu info:
ndor_id : GenuineIntelcpu family : 6model : 79model name : Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHzMotheboard and Bios info:
description: Rack Mount Chassis
product: X10DRH (085915D9)
vendor: Supermicro
version: 123456789
serial: 123456789
width: 64 bits
capabilities: smbios-3.0 dmi-3.0 smp vsyscall32
configuration: boot=normal chassis=rackmount family=SMC X10 sku=085915D9 uuid=00000000-0000-0000-0000-002590F9F7AE
*-core
description: Motherboard
product: X10DRH-IT
vendor: Supermicro
physical id: 0
version: 1.00
serial: OM14BS004942
slot: Default string
*-firmware
description: BIOS
vendor: American Megatrends Inc.
physical id: 0
version: 2.0
date: 12/17/2015
size: 64KiB
capacity: 8128KiB
capabilities: pci upgrade shadowing cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer acpi usb biosbootspecification uefi
thanks,
Abe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello aasraoui,
I know that you are expecting support here but please be aware that your best contact support is with your system manufacturer (Supermicro). I know that this is an Intel processor but your system manufacturer is more familiar with your PC features, specifications and how all the components interact with each other. At this point for you to get the proper support you can contact here:
http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRH-iT.cfm http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRH-iT.cfm
NOTE: This link is being offered for your convenience and should not be viewed as an endorsement by Intel of the content, products, or services offered there.
Regards,
Ivan U.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Abe
Were you able to find a root cause or resolution? we are experiencing the same issue with the v4 procs. Our environment has over 50 FC630's running with the v3 proc's with no issues, we added 30 FC630's running with the v4 proc's into the same environment and most of them are having CPU Softlocks issues when running under a heavy load.
Operating System: Ubuntu 14.04
Hardware Info: Dell PowerEdge FC630
CPU Info:
vendor_id : GenuineIntel
cpu family : 6
model : 79
model name : Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
stepping : 1
microcode : 0xb00001f
BIOS:
BIOS Information
Vendor: Dell Inc.
Version: 2.4.2
Release Date: 01/09/2017
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 16384 kB
Characteristics:
ISA is supported
PCI is supported
PNP is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
EDD is supported
Japanese floppy for Toshiba 1.2 MB is supported (int 13h)
5.25"/360 kB floppy services are supported (int 13h)
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
CGA/mono video services are supported (int 10h)
ACPI is supported
USB legacy is supported
BIOS boot specification is supported
Function key-initiated network boot is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 2.4
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page