Why the cores of Intel E5-2643v4 CPUs are numbered non-continuously in CentOS 7.2?

[idata]

Yesterday, we upgraded the CPUs in two 1U servers (http://www.aicipc.com/ProductDetail.aspx?ref=SB122A-PH AIC SB122A-PH) running CentOS 7.2 to 2xhttp://ark.intel.com/products/92989/Intel-Xeon-Processor-E5-2643-v4-20M-Cache-3_40-GHz E5-2643v4 3.4Ghz CPUs (Broadwell)/server. Soon afterwards, we spotted that the cores of the 4 new CPUs are numbered non-continuously. Instead of 0-5, now there is a gap. Please see below for /usr/bin/sensors output.

Specifically, we have four such servers. fs00 and fs01 still use the existing E5-2643v3 (i.e. Haswell) 3.4Ghz CPUs. Their sensors output look fine. fs10 and fs11 sport the Broadwell E5-2643v4 3.4Ghz CPUs, and their cores are numbered from 0-3, then jumped to 6-7, skipping 4. This is puzzling and causes our monitoring dashboard to malfunction. Can anyone provide us a resolution or get around to this issue?

OS:

[root@fs10 ~]# uname -a

Linux fs10 3.10.0-327.36.1.el7.x86_64 # 1 SMP Sun Sep 18 13:04:29 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

sensors version:

[root@fs10 ~]# rpm -qf /usr/bin/sensors

lm_sensors-3.3.4-11.el7.x86_64

$ ansible tf2:bf2 -m shell -a "/usr/bin/sensors|grep Core" -u root

sc1u+fs00 | SUCCESS | rc=0 >>

Core 0: +44.0 C (high = +80.0 C, crit = +98.0 C)

Core 1: +46.0 C (high = +80.0 C, crit = +98.0 C)

Core 2: +46.0 C (high = +80.0 C, crit = +98.0 C)

Core 3: +47.0 C (high = +80.0 C, crit = +98.0 C)

Core 4: +44.0 C (high = +80.0 C, crit = +98.0 C)

Core 5: +44.0 C (high = +80.0 C, crit = +98.0 C)

Core 0: +47.0 C (high = +80.0 C, crit = +98.0 C)

Core 1: +51.0 C (high = +80.0 C, crit = +98.0 C)

Core 2: +48.0 C (high = +80.0 C, crit = +98.0 C)

Core 3: +47.0 C (high = +80.0 C, crit = +98.0 C)

Core 4: +48.0 C (high = +80.0 C, crit = +98.0 C)

Core 5: +49.0 C (high = +80.0 C, crit = +98.0 C)

sc1u+fs01 | SUCCESS | rc=0 >>

Core 0: +38.0 C (high = +80.0 C, crit = +98.0 C)

Core 1: +40.0 C (high = +80.0 C, crit = +98.0 C)

Core 2: +38.0 C (high = +80.0 C, crit = +98.0 C)

Core 3: +37.0 C (high = +80.0 C, crit = +98.0 C)

Core 4: +42.0 C (high = +80.0 C, crit = +98.0 C)

Core 5: +41.0 C (high = +80.0 C, crit = +98.0 C)

Core 0: +41.0 C (high = +80.0 C, crit = +98.0 C)

Core 1: +46.0 C (high = +80.0 C, crit = +98.0 C)

Core 2: +42.0 C (high = +80.0 C, crit = +98.0 C)

<span style="color: # 274e13; font-family: monospace, monospace;"...

[idata]

Some additional info. To ensure that it's not a bug in the /usr/bin/sensors command, I also used the following:

[root@fs11 ~]# lstopo -v --of png > /tmp/fs11_v4_numa_layout.png

As can be seen below, the CPU P# skipped 4

[root@fs11 ~]# rpm -qi hwloc-gui

Name : hwloc-gui

Version : 1.7

Release : 5.el7

Architecture: x86_64

Install Date: Tue 19 Jul 2016 02:36:34 PM PDT

Group : Development/Libraries

Size : 112209

License : BSD

Signature : RSA/SHA256, Wed 25 Nov 2015 06:41:38 AM PST, Key ID 24c6a8a7f4a80eb5

Source RPM : hwloc-1.7-5.el7.src.rpm

Build Date : Fri 20 Nov 2015 04:44:36 AM PST

Build Host : worker1.bsys.centos.org

Relocations : (not relocatable)

Packager : CentOS BuildSystem <</span>http://bugs.centos.org http://bugs.centos.org>

Vendor : CentOS

URL : http://www.open-mpi.org/projects/hwloc/ http://www.open-mpi.org/projects/hwloc/

Summary : The gui-based hwloc program(s)

Description :

GUI-based tool for displaying system topology information.

The above is also confirmed via cat /proc/cpuinfo:

[root@fs11 ~]# cat /proc/cpuinfo |grep -i 'core id'

core id : 0

core id : 1

core id : 2

core id : 3

core id : 6

core id : 7

core id : 0

core id : 1

core id : 2

core id : 3

core id : 6

core id : 7

core id : 0

core id : 1

core id : 2

core id : 3

core id : 6

core id : 7

core id : 0

core id : 1

core id : 2

core id : 3

core id : 6

core id : 7

[idata]

Hello zperry,

I am sorry to hear you are having issues with your monitoring dashboard for the count core reflected in CentOS 7.2

Let me further investigate this matter and get back to you with the outcome of the investigation all the information provided will certainly help with the investigation.

As soon as I get input for you, will let you know.

Regards,

Esteban C

[idata]

Hi Esteban,

As far as I know, Linux kernel reads CORE ID from MSR, which is the register that CORE ID is put in, and it is a READ ONLY register to BIOS. But if somehow there is a safe way for "knowledgeable" end user (e.g. a seasoned sysadmin) to change it, I am willing to try. Hints and pointers to such info would be appreciated!

-- Zack

[Ronny_G_Intel]

Hello zperry,

I have confirmation that this is not an issue on a Windows Server like Operating System so we beleive it has to be something with the Operating System and it might be related to the LM_Sensors.

Would you please make sure that you are running the latest package, latest version: 3.4.0

See the following information below:

http://www.linuxfromscratch.org/blfs/view/svn/general/lm_sensors.html lm_sensors-3.4.0

Please check on that and let me know the results

Regards,

Ronny G

[idata]

Hello zperry,

I would like to know if you were able to read the last answer provided by rguevara, please let me know.

Regards,

Esteban C

[idata]

Hi zperry,

Please let me know if you require any further support for this inquiry.

Regards,

Esteban C