@radekmieBy Radosław Miernik · Published on · Comment on Reddit
If you have ever worked with a profiler, you should be aware that basically any process on your computer may affect the results. It’s simple – CPUs are complex beasts, and your OS is doing its best to keep the system responsive. The only thing you have to know is that it’s actually good at it. Like, really good.
As all resources of our computers are strictly limited, it’s often the case that some of the processes (tasks) will not be given as much of them as needed. Or, more likely and even worse, they’ll receive a different amount each time.
But what’s the deal with “cold keyboard”? Well, I work on a laptop with some Linux on it1. What is important for the story (and the title!), I love its built-in keyboard, and I’m not using an external one. That means, every time the CPU gets hot, I can actually feel it. And like most people (I guess?), I honestly hate it.
To keep my keyboard cold, I could stop pushing the CPU that hard. However, it’s not possible during profiling, as you want to run your program and another one in parallel to measure the performance2. What else can we do?
My laptop uses an Intel CPU (8th generation). As most (all?) modern Intel CPUs, it has an Intel Turbo Boost feature that allows the CPU to increase its performance for a short period. It’s a significant increase – my CPU is over two times faster while boosted! (As far as I know, other CPU brands have similar features, often referred to as auto overclocking or dynamic overclocking.)
Of course, it has its drawbacks. I’m not going to focus on energy consumption or noise here, but it’s obvious that such an enormous improvement comes with a cost. The two things that matter for me people who are profiling are that it’s only a temporary improvement, and that it generates much more heat.
For these two reasons, I strongly recommend turning off the boosting while profiling to make sure that it won’t interfere with the measurements. The fact that such a boost may kick in while the profiled program is running means that the data before and after this point are not comparable. The same goes for a situation when it suddenly stops. (Remember, it’s temporary!)
The other, more general problem is the heat. CPUs manufacturers are aware of the fact that cooling systems may not be enough to keep them safe. Whatever the source of the heat is, the CPU can try to prevent itself from overheating. What happens is the CPU downclocks (slows) itself to reduce the generated heat. Such behavior is called dynamic frequency scaling or thermal throttling. When the overheating prevention starts, the CPU may slow down significantly, leading to problems similar to the boost suddenly stopped.
What can we do to make the CPU perform more or less equally at all times? Disabling boost helps, but what about the heat? Ideally, we’d like to configure it in a way that no workload would make it “hot enough” for thermal throttling.
Luckily, we can do that: a CPU can be downclocked (or underclocked). The CPU, while forced to be slower, will generate less heat, hopefully maintaining the same performance every time. (I assume that no other factors are interfering.)
It may be the case that you don’t have to do that at all. I used this technique in the past with my previous laptop. The one I have now is just fine with boosting disabled – it never overheats.
The last thing you can do with your CPU to keep getting consistent results is to choose a correct CPU frequency governor (“power scheme”). I won’t get into details here, but the point is that the CPU can be told to be focused either on lasting longer on the battery or delivering the best performance it can.
From my experience, all power-saving or on-demand governors tend to be my default ones, delivering good performance on my day-to-day work. However, while profiling I do switch to the performance governor. The difference is that it always keeps the CPU at high frequency, reducing the problem of varying performance between the measurements.
Keep in mind that your laptop may choose a different governor based on its battery status. By default, most of them will stay in power-saving mode when unplugged and switch back to a performance-oriented one while charging3.
The metaphor of a cold keyboard is not really applicable to the vast world of non-laptop PCs; neither to the users of external keyboards. But even if you won’t feel the heat under your fingers, you may hear it. It turns out that a cold keyboard and a quiet fan are not that different after all.
And finally, remember that everything your CPU does in parallel to the profiler may interfere with its results. For example, a video player load depends on the content. It’s better not to run anything else while profiling, or at least try to minimize the number of unneeded background tasks.
Profiling is hard.
No, I won’t tell you which Linux exactly. No more “holy wars” for me.
Profilers as “sidecar programs” are not computationally heavy on their own, but the instrumentation needed to make them work is. In simple words, the profiled program is going an extra mile to tell the profiler what is happening.
I do use a USB-C cable to connect an external monitor and as it has a power delivery function, my laptop is constantly charging. That means I have to force the power-saving mode all the time.