What hardware is making my consecutive loop times slow?

Franks__Brandon · ‎12-14-2017

I have two Dell Precision Tower 3620's that have different internal hardware. They are both running Ubuntu 16.04 LTS, and have kernel version 4.6.2-rt5 (a real time patched kernel). I am 100% certain both kernels were configured the exact same way when I was setting them up. I use the towers to do the same thing: I run a real-time priority C program that is isolated to one processor to read and write from the exact same PCIe board, and I gather timing statistics related to how long it takes for the computer to do consecutive reads. I am finding one tower to have a longer read to read time than the other tower, and I am 100% certain the loop to loop time difference isn't a software issue.

My first question is is this the correct forum for this question? My actual question is what hardware could be making one tower run faster than the other? Which hardware is related to the loop time of a C code that is interacting with a PCIe board?

Slower Tower Average Loop to Loop time: 0.065 ms

Faster Tower Average Loop to Loop time: 0.055 ms

Slower Tower has Intel® Core™ i7-7700K CPU @ 4.20GHz × 8

Faster Tower has Intel® Xeon(R) CPU E3-1241 v3 @ 3.50GHz × 8

Slower Tower lspci ouput:

user@towerI:~$ sudo lspci

00:00.0 Host bridge: Intel Corporation Device 591f (rev 05)

00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16) (rev 05)

00:02.0 VGA compatible controller: Intel Corporation Device 5912 (rev 04)

00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31)

00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H Thermal subsystem (rev 31)

00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #1 (rev 31)

00:17.0 SATA controller: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] (rev 31)

00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #1 (rev f1)

00:1d.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #9 (rev f1)

00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller (rev 31)

00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31)

00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)

00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31)

02:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge

04:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)

05:00.0 Unassigned class [ffff]: Measurement Computing PCIe-DAS1602/16

Faster Tower lspci output:

user@towerII:~$ sudo lspci

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller (rev 06)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)

00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 04)

00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)

00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 04)

00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 04)

00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 04)

00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d4)

00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d4)

00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 (rev d4)

00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 04)

00:1f.0 ISA bridge: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller (rev 04)

00:1f.2 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 04)

00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)

01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [NVS 310] (rev a1)

01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)

03:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge

05:00.0 PCI bridge: Intel Corporation Device 156b

06:00.0 PCI bridge: Intel Corporation Device 156b

06:03.0 PCI bridge: Intel Corporation Device 156b

06:04.0 PCI bridge: Intel Corporation Device 156b

07:00.0 System peripheral: Intel Corporation Device 156a

08:00.0 Unassigned class [ffff]: Measurement Computing PCIe-DAS1602/16

Travis_D_ · ‎12-18-2017

It is not at all surprising to me that two completely different pieces of hardware, from different generations and with different PCIe controllers would have an ~18% performance difference for some piece of code. I'll admit that it's weird that the apparently older/slower machine has the faster number, but I don't think you'll be able to point to some line in the lspci output and say "that's it!".