Intel and Canonical have been collaborating to better tune your Ubuntu servers for your power and performance needs, as server platforms have unique considerations when you tune them. Depending on the workload, tuning can allow you to improve both performance and energy savings at the same time. Tuning for lower power does not always result in the most energy savings, due to the fact that energy consumption is a result of power consumption over time. Though it may seem counter-intuitive, it may be beneficial to operate at higher performance over a shorter time and then allow a system to go into a deeper sleep state when it is not needed.
Power and performance tuning is necessary to ensure that any device is configured to produce the best performance and energy efficiency for a given workload. Generally speaking, the default values for tuning parameters in the operating system are set to an optimal value for an average set of workloads, and are generally good to avoid getting the worst performance. However, gains can be substantial if you tune based on your workload. This is especially true for CPU intensive workloads, or ones that have performance that is far from the mean.
Performance and energy efficiency aren’t always a trade-off
When tuning to optimize energy efficiency of a system, it’s important to consider that the total power consumption of a system can be broken down into consumption from the processor and consumption from other system components. Fans, storage, memory, and other components can consume significant power even when the CPU is idle. The processor itself can be broken into dynamic and static consumption. While we may tune the processor, we should do so in a way that optimizes the system as a whole.
Reducing dynamic consumption with P-States
When we reduce the operating frequency of a processor, we run it slower than its maximum designed speed. We are, linearly, trading off performance for power. We also reduce the minimum voltage the processor requires to operate, and this can have a quadratic impact on power without sacrificing additional performance. This is called Dynamic Voltage and Frequency Scaling. On Intel and other x86 processors, this is provided to the operating system in the form of Performance States (P-States), which are a set of voltage-frequency pairs that the processor can run. Linux exposes this via the intel_pstate driver, which controls the processor’s frequency and lets the hardware determine the correct operating voltage.
Reducing static consumption with C-States
The static power consumption of a computer processor refers to the amount of electrical energy that the processor consumes when it’s not actively processing data. This includes keeping time, managing interrupts, and performing housekeeping operations. Even when the processor is idle or in sleep mode, it still draws a small but non-zero amount of power to keep itself ready for immediate wake-up. In general, the static power consumption is relatively low compared to active power consumption during processing but can add up over time and contribute to overall system energy efficiency.
To reduce static consumption without penalizing functionality, modern processors implement a set of features called C-states, which allow parts of the CPU to be put to sleep and woken up on demand.
Source: Intel - https://www.intel.com/content/dam/doc/white-paper/energy-efficient-platforms-2011-white-paper.pdf Page 15. Used with permission.
These sleep states are gradual and, with each increasing number, power off more and more parts of the CPU. However, powering them on and off does not occur instantaneously: there is an associated latency with entering/exiting those states. Furthermore, the latency increases the deeper the sleep is, so there is a tradeoff between how much we sleep and the amount of time spent transitioning between states. If we sleep too frequently, we lose performance going in and out of sleep. Therefore, the threshold at which we should sleep is not constant and varies per task, as we do not know beforehand how long it will take for it to wake up again.
Finally, running the CPU at an efficient – but slow – operating point might be counterproductive when we are dealing with hardware with high static consumption, such as servers. When your chassis is pulling over 100 Watts just to keep fans, BMCs, hard drives, and SSDs running, it is crucial that the CPU performs well enough to justify that fixed energy cost. Therefore, depending on your setup it might actually be more energy-efficient to run faster, but be able to perform more work and offset that static consumption, than being shortsighted and running the CPU at an efficient point, but wasting a lot of energy in static consumption.
Using EPP to optimize energy consumption
The Linux kernel provides several CPU governors, which allows a user to configure CPU cores to operate in different modes: performance, balance_performance, and powersave. Also, Intel has introduced a tunable setting called Energy Performance Preference, or EPP, that allows end-users to specify how aggressively the processor should leverage the latest integrated power management techniques. The CPU governors set the EPP value, ranging from 0 to 255, as shown in the diagram below. Intel provides default EPP values in the kernel for each governor and processor generation. The EPP acts as a hint to the processor for how to set the P-States and C-States. By default, Ubuntu utilizes the balance_performance setting.
Whenever the intel_pstate driver is active, the CPU governors will work differently than with other drivers. For instance, the performance governor will not necessarily pin the CPU frequency to the maximum – it will only set the scaling to be as aggressive as possible. Contrarily, the powersave governor will not force the frequency to be at its lowest, it will only enable more performance-costly power-saving techniques.
Profiling results show performance gains and power savings
In collaboration with Intel, we profiled a series of workloads using the Phoronix Test Suite on a 2-socket Intel(R) Xeon(R) Platinum 8592+ (codename: Emerald Rapids) reference server. By utilizing a Yokogawa power meter, we were then able to derive energy consumption data for the whole system.
Based on the analysis, we updated the balance_performance governor for Emerald Rapids to an EPP value of 32, which yields these improvements and savings:
- OpenSSL operating 12% faster, within 1% of energy consumption
- Building the Linux Kernel was 29% faster, and used 18% less energy
- The POV-Ray benchmark ran 32% faster, with 12% less energy
These results demonstrate that while we traditionally think of power and performance as trade-offs, this isn’t always the case for energy and performance. Given the sophistication of the power and performance management in modern Intel processors, there can be an opportunity to improve both performance and energy efficiency (see figures below). Other workloads will have different performance and energy characteristics, which underscores the importance of tuning EPP for your specific workload.
You can also learn more about how Canonical and Intel work together to optimize Ubuntu performance on Intel silicon ranging from servers to laptops, desktops, and IoT devices.
