6349P Not Running As Fast As It Could Be

[Hank9000]

I am replacing a rack server which has an E-2388G with a newer model that has a 6349P. I was expecting a nice uplift in performance but the benchmarks are underwhelming.

In some of the tests the 6349P is about 10-20% faster than the E-2388G, in other tests they are on par, but in a few others (mostly single threaded apps) the 6349P is slower (in some cases much slower). Overall the results are disappointing and not what I was expecting, especially when I saw graphics that said this series was 1.57x faster than the E-2388G (I know - specific tests would prove that, real world would be much less).

Why Slower?

I believe that the manufacturer of the new rack server deliberately turn down the turbo speeds in order to keep power level and temperatures down. I would like to reverse, and unleash the true potential of the 6349P, but the manufacturer will not assist with this, or admit that they hobble the chip to prevent it from running as fast as it could.

There are no issues with temps! There are no issues with power draw. There is no issue with the BIOS settings (tried all combinations) or the O/S (Rocky Linux and Windows 11). In all cases the max turbo speed for single core is locked at 5.2Ghz and nothing can be done to let the chip fly at it's intended 5.7GHz.

Linux utility turbostat indicates that the cores max turbos are this:

cpu0: MSR_TURBO_RATIO_LIMIT: 0x3535353536363334
53 * 100.0 = 5300.0 MHz max turbo 8 active cores
53 * 100.0 = 5300.0 MHz max turbo 7 active cores
53 * 100.0 = 5300.0 MHz max turbo 6 active cores
53 * 100.0 = 5300.0 MHz max turbo 5 active cores
54 * 100.0 = 5400.0 MHz max turbo 4 active cores
54 * 100.0 = 5400.0 MHz max turbo 3 active cores
51 * 100.0 = 5100.0 MHz max turbo 2 active cores
52 * 100.0 = 5200.0 MHz max turbo 1 active cores

(note I don't know if this a bug with turbostat - possibly the version I have is not aware of the 6400 series, but it is showing 8 cores instead of 6).

That to me indicates that the manufacturer will not allow 5.7GHz to be hit under any circumstances. At best 5.4GHz could be achieved but with all the testing I have done I have only ever managed to hit 5.2Ghz.

The Temps are Fine - The Watts Are Low!

The ambient temperature is 20', the core temp is around 50-70' for idling and running a benchmark. The Package Watt idles at 5W and goes upto 90W.  If I use

cpupower idle-set --disable-by-latency 0

The Package Watt is a constant 50-70W on idle, and 70-95W when benching. Suffice to say this is no where near the max ratings of the 6349P.

What's The Issue?
The issue is I have bought a rack server that is not doing what it is supposed to do. It has been hobbled to prevent it from running at or near it's best performance. The performance is similar to one of a chip that is four years old.  Windows benchmarks (CPU-Z, Geekbench 6) state that this Xeon 6349P is similar to an i7-12700K, or an  i5-13600K and that can't be right at all.

So does anyone have a solution on how I can unleash the true potential of this 6349P with this manufacturers motherboard?

Is anyone aware of any tool that I can use to change the turbo profile? Maybe something like msr-tools?

Would I be better off just ditching the server, buying a SuperMicro board and putting the 6349P in that board instead?

[Fikri_Intel]

Hi Hank9000,

Thank you for contacting Intel.

We appreciate you reaching out to us regarding the reported issue. To assist you more effectively and provide an accurate solution, we kindly request the following details:

1- What system model are you currently using? Brand/Model?

2- Does the CPU come with the motherboard or purchased separately?

3- Have you checked the CPU and motherboard compatibility?

Once we have this information, we will be able to guide you better and provide the appropriate support.

Looking forward to your response.

Regards,

Fikri_Intel

[Hank9000]

Thank you for your prompt reply Fikri,

This is an off-the-shelf complete 1U rack server. It's an R360 as shown here:

https://www.dell.com/en-uk/shop/enterprise-products/servers/spd/poweredge-r360/emea_per3601a

The CPU is:

Intel® Xeon® 6 Performance 6349P 3.6G, 6C/12T, 18M Cache, Turbo, (95W) DDR5-4800

Signature: Type 9, Family 6, Model 183, Stepping 1

Voltage: 1.8 V

External Clock: 100 MHz

Max Speed: 5700 MHz

Current Speed: 3600 MHz

Status: Populated, Enabled

Upgrade: Socket LGA1700

L1 Cache Handle: 0x0700

L2 Cache Handle: 0x0701

L3 Cache Handle: 0x0702

Core Count: 6

Thread Count: 12

[Amina_Sadiya]

Hi Hank9000,

Thank you for sharing the details. We need a few more details to proceed further, Kindly help us with the below details.

> Does the CPU come with the motherboard or purchased separately?

> Have you checked the CPU and motherboard compatibility?

Best regards,

Amina

Intel Customer Support Technician

[Hank9000]

Hi, 

> Does the CPU come with the motherboard or purchased separately?

> Have you checked the CPU and motherboard compatibility?

Yes - the CPU came with the motherboard.

Yes - the CPU and the motherboard are compatible.

[Amina_Sadiya]

Hello Hank9000,

Thank you for your response. As you have mentioned that the processor is pre-installed on the board, you will need to contact the board manufacturer or the place of purchase for further assistance. 

Best regards,

Amina

Intel Customer Support Technician

[Hank9000]

The manufacturer is refusing to provide anwsers. All they say is "go find someone to tune it".

That's why I am asking Intel.

It's a very simple question:

Is there anything I can do (via a BIOS hack, or msr-tools, or some other app) in order to make this 6349P run at it's proper speed (5.7GHz). 

Why can't Intel help with that?

[pujeeth]

Hi Hank9000,

Greetings!!

Thank you for the response, kindly allow us sometime we will get back to you with an update as soon as possible.

Regards

Pujeeth_Intel

[pujeeth]

Hi Hank9000,

Greetings!!

Thank you for your patience,

We understand your concern regarding the turbo frequency behavior of your Intel® Xeon® 6349P processor.

Since this is a tray processor, Intel does not control or validate the power limits, turbo policies, or BIOS-level performance settings on OEM/server platforms. These parameters are fully defined by the motherboard manufacturer or system OEM, and they may set power or thermal limits that differ from Intel’s default recommendations. For this reason, we recommend contacting your system or motherboard manufacturer for further assistance, as they control the turbo and power configuration on their platform.

As a best-effort, please find below some information and steps that may help:

Any processor will have a base frequency. Depending upon the tasks that are running, it will raise its frequency to maximum. When Turbo Boost is enabled, it gives Processor clocks option to overclock and exceed the base frequency whenever demanding task(s) are running. However there is NO way, even by making changes in BIOS that we can always keep all the processor cores at Maximum Turbo frequency always without having them changed dynamically as per workload. On top of turbo boost being enabled, what can be done on a system or server if to ensure High performance mode is enabled in OS and any power saving features to be disabled.

Below are some steps that can be tried. The shared steps may vary depending on the BIOS of the Motherboard. you should cross check with Motherboard Vendor/ OEM regarding below BIOS settings/ steps shared:

BIOS Settings:

1. Enable Intel Turbo Boost Technology: Make sure this is enabled to allow the CPU to operate at higher frequencies.

2. Set the Power and Performance Policy to Performance: This setting will prioritize performance over power saving.

3. Set the CPU P-State Control to Performance: This will ensure that the CPU runs at higher performance states.

4. Disable Energy Efficient Turbo: This prevents the CPU from entering lower power states when utilization is high.

5. Set Boot Performance Mode to Max Performance: This ensures the system boots in performance mode.

Operating System Settings:

1. Use the Intel P-State Driver: Ensure that the Intel P-State driver is being used for frequency scaling.

2. Set the Frequency Governor to Performance: This can be done using the "cpupower" command:

cpupower frequency-set -g performance

3. Disable Turbo Mode if necessary: To achieve a constant maximum non-turbo frequency:

echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo

 Other Considerations:

1. Disable C-States: This can be done in the BIOS to prevent the CPU from entering lower power states.

2. Adjust Minimum and Maximum Performance Percentages: You can set the minimum and maximum performance percentages to 100% to keep the CPU at maximum frequency:

echo 100 > /sys/devices/system/cpu/intel_pstate/min_perf_pct

echo 100 > /sys/devices/system/cpu/intel_pstate/max_perf_pct

NOTE: Please note that running the CPU at maximum frequency at all times may lead to increased power consumption and higher temperatures, which could affect system stability and longevity.

You may also use TurboStat to check if Processor cores are at Maximum/Turbo frequency. Below are the key columns to look at in the turbostat output:

1. CPU: The CPU core number.

2. Bzy_MHz: The average frequency in MHz while the CPU is busy.

3. %Busy: The percentage of time the CPU is busy.

4. TSC_MHz: The actual frequency of the Time Stamp Counter.

5. PkgWatt: The power consumption of the CPU package.

See below example:

1. %c0: Indicates the percentage of time the CPU is in the C0 (active) state.

2. GHz: The actual operating frequency in GHz.

3. TSC GHz: The Time Stamp Counter frequency, which should be close to the nominal frequency.

4. PkgWatt: The power consumption of the CPU package.

Check the Frequencies:

1. Ensure that the GHz column values are close to or at the turbo frequency of your CPU. This indicates that the CPU is running at or near its maximum turbo frequency.

2. You can also look at the Bzy_MHz column, which shows the average busy frequency in MHz.

Use Other Tools:

1.lscpu: Provides information about the CPU architecture and current frequency.

  lscpu | grep "MHz"

2. watch command: To continuously monitor frequency.

  watch "grep 'cpu MHz' /proc/cpuinfo"

Please be aware that Intel AVX workloads reduce turbo frequency by design due to higher power consumption. This is expected behavior across all Intel processors.

https://www.intel.com/content/www/us/en/support/articles/000100192/processors/intel-xeon-processors.html

By monitoring these values, you can determine if your CPU is consistently operating at its turbo frequency.

Remember that turbo frequency operation depends on factors like workload, thermal headroom, and power limits, so ensure those conditions are met for sustained turbo frequencies.

Reminder: These are just basic steps that can be tried out but as stated earlier, for any OEM System/Motherboard Vendor, it is best to check with OEM Support.

Regards

Pujeeth_Intel

[Hank9000]

Thank you for your detailed reply. I can assure you have done all that apart from the AVX disabling. Unfortunately that will not work because the O/S then reports that the hardware is not compatible.

The issue is not so much the all core turbo speed. The issue is the single core turbo. If I run a single threaded app the speed never goes above 5.2GHz and thus the Xeon is not running at it's full potential. The temps and PkgWatt are well below the maximums, and all combinations of tuning have been performed to no avail.

I appreciate there is nothing much Intel can do in this situation. I was just hoping someone knows of a hack that can remove Dell's restrictive limits.

Thank you for your time with this.

[TejasMohan]

Hello Hank9000,

Greetings from Intel!

Thank you for writing back.

We would like to inform you that we are closing this request as there are no pending actions from Intel.

Please don’t hesitate to reach out with any further questions in the future. Feel free to start a new conversation, as this thread will no longer be monitored.

Regards,

Tejas

Intel Customer Support Technician