Scott Hanselman

The New Turbo Button - Balancing Power Management and Performance on Windows Servers

September 17, '13 Comments [20] Posted in Musings
Sponsored By

TurboButtonDo you remember the Turbo Button? I actually thought of it is the "be slow button" because we always kept it on Turbo. Why wouldn't you want a fast computer all the time? The Turbo Button was actually an "underclock" button. When it was off, you were setting your 286 or 386 to XT speeds so older DOS games would work at their designed speed.

Power Management, both software and hardware, seems to be the new Turbo Button. My laptops get way faster when I plug it in - like very noticeably faster to the point where I just don't like using them on battery. For typing documents, it's fine, but for development, compiling, running VMs, it's unacceptable to me. I'll end up spending more power to get more performance. 

It's important to remember that Power Management affects servers as well.

Recently Mike Harder, a development manager, noticed that stuff he does every day was taking longer on the "Balanced" power option than the "High Performance" option. He said:

My naïve belief was that “Balanced” is supposed to save power when your machine is idle, but give full power when needed, so the overall perf hit should be small.

Here's a very basic benchmark Mike did:

Hardware/OS
Hardware: HP z420, Intel Xeon E5 1650 @ 3.2GHz, 32GB RAM, SSD
OS: Windows Server 2012 Standard

(in seconds) High Performance Balanced Delta
7-Zip, LZMA, 2 Threads 55 115 109%
7-Zip, LZMA2, 12 Threads 28 49 75%
Build Source Tree, 48 Threads 37 55 49%

This started a fascinating thread on power management and the balance between getting good performance from a system (desktop or laptop or server) and wasting power and heat. Here's the best parts of that internal thread here for all of our education.

Bruce Worthington said:

Depends on the workload.  The full performance of the system is available, but (for example) if the workload is very bursty you will take an initial hit at the beginning of each burst as the power management algorithms determine that more resources need to be brought on line.  Or if it is a low-utilization steady state workload, you will run at a lower CPU frequency throughout.

There is no free lunch, so there is always a tradeoff that is being made.

There is also an excellent thread on this at ServerFault. Jeff Atwood asks:

Our 8-cpu database server has a ton of traffic, but extremely low CPU utilization (just due to the nature of our SQL queries -- lots of them, but really simple queries). It's usually sitting at 10% or less. So I expect it was downclocking even more than the above screenshot. Anyway, when I turned power management to "high performance" I saw my simple SQL query benchmark improve by about 20%, and become very consistent from run to run.

imageThis makes sense to me. The CPU isn't working hard enough for long enough for the power management algorithms to put full power to the CPU. But, if Jeff sets power management to High Performance he's effectively saying "full speed ahead...always."

In the last half-decade power management in servers has become more of an issue. With high power comes heating and cooling as well as power costs. Windows Server 2008's default power is "Balanced."

Bruce again in an excellent explanation with emphasis mine:

I'll try to give a quick perspective below as to why we use Balanced mode as our default and how we arrive at the tunings for that mode.

As of Windows Server 2008, the default setting of the OS was switched from High Performance to Balanced.  Energy efficiency was becoming a larger factor in the real world, and our ability to balance between the oft-opposing poles of Power and Perf was improving.  That being said, there will always be environments where our recommendation is that the power policy should be switched back to High Performance.  Anything super latency sensitive will clearly fall into that bucket, such as banking, stock markets, etc.

OEMs have the flexibility to add custom tunings onto their factory settings if they want to put in the additional effort to find a balance that works better for their specific customers.  System administrators also have that flexibility. But tuning the power/perf knobs in the OS is a very tricky business, not for the faint of heart. 

<snip…>


Some of us on the Windows "power" teams were performance analysts before we become power analysts, so we are very sensitive to the tradeoffs that are being made and don’t like seeing any perf lost at all.  But there is no free lunch to be had, and there are big electric bills being paid (and polar bears falling into the water) that can be helped through sacrificing some level of performance in many environments.

<snip>

We will continue to provide multiple power policies because one size clearly does not fit all servers.

Another great point made for why have "Balanced"  be the default, from Sean McGrane:

[We're] looking at an industry landscape where servers in data centers are very underutilized, typically somewhere below 20% utilization. By going with balanced mode we saved a lot of energy and cost and improved their carbon footprint more or less for free. There was very strong support from customers to do this.

Virtualization has helped raise the utilization levels and most cloud DCs now operate at higher levels of utilization. However the majority of servers deployed are still running a single workload and that will be the case for a while.

This get to the point of measuring. Are your servers working hard now? Perhaps they'll perform better on High Performance. Are they often idle or at lower levels of utilization? Then Balanced is likely fine and will save power. Test and see.

As with all things in software development, it's a series of trade offs. If you blindly switch your servers' power options to High Performance because you read it on a random blog on the Internet, you're of course missing the point.

Change a variable, then measure.

Consider your workloads, how your workloads cause your CPUs to idle and how hard they work the CPU when pushed. Are you doing single threaded low CPU work, or massively parallel CPU intensive work?

I'm now going to pay more attention to power management profiles when developing, putting machines into production, stress testing and benchmarking. It's nice to have a Turbo Button.


Sponsor: Thank you to RedGate for sponsoring the feed this week! Easy release management: Deploy your .NET apps, services and SQL Server databases in a single, repeatable process with Red Gate’s Deployment Manager. There’s a free Starter edition, so get started now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Tuesday, September 17, 2013 10:40:32 PM UTC
Hi Mr Scott,

Another option (what I do), during the peak window (business hours/trading hours), we press the turbo button aka High Performance.

To save money/environment/make the world a happy place, off peak we switch to balanced.

Easy to do with task scheduler and a power shell script.


Jimbob McGoo
Wednesday, September 18, 2013 12:12:26 AM UTC
Made me look, and the power mode in an Azure VM is set to Balanced, with no option to change.
Michael Teper
Wednesday, September 18, 2013 12:14:01 AM UTC
I haven't noticed lower performance on battery, maybe because I work in VM's on a Mac? I've never thought to look at how Parallels interferes. SSD's have really removed the biggest performance bottleneck for me.
Wednesday, September 18, 2013 1:20:47 AM UTC
Jimbob - Fantastic tip!

Michael - Don't worry, the VM setting doesn't matter, it's the HOST setting that matters. They tune that stuff themselves.

Jeff - Again, it's very likely that setting is overridden by VMWare/Parallels.
Wednesday, September 18, 2013 2:17:34 AM UTC
This is a fairly big issue with database servers, especially with OLTP workloads. Changing the Power Plan from Balanced to High Performance often increases query performance by 20-25%.

It is also very important to use a tool like CPU-Z to verify that the Windows Power Plan is not being overridden by hardware-level power management(controlled in the BIOS).
Wednesday, September 18, 2013 4:51:10 AM UTC
They really need to improve the fine tuning capabilities of the windows power management utility.(or if its already there, make it easier and more obvious)

For instance, change power settings automatically based on time. A FTP server in an office doesn't need to run at full speed at night. And active directory should have the ability to change the power states of various servers based on the number of logged in users.
Sihan
Wednesday, September 18, 2013 9:25:37 AM UTC
Hey, I remember this Turbo button. I was a child and did not know what this magic button meant ;) Let me agree with Sihan - tuning capabilities are really not very obvious and sometimes confusing.
Wednesday, September 18, 2013 12:42:28 PM UTC
I remember the "Turbo" button on our school's three PCs, which would switch between 8 and 12 MHz. Both of which seemed blisteringly fast compared to the ZX Spectrum, whose clock speed was so low the manual had to list it to 12 decimal places just so it looked better!
Richard
Wednesday, September 18, 2013 1:06:28 PM UTC
(off-topic.. but I'm hopeful!)

Hi, I always struggle configuring Hyper-V's Virtual Switch on Windows 8. What are the configuration steps so the host and guest have intranet and both can access internet? Whatever configuration I try, I end up Internet working on both or none. Why don't Microsoft make it easy like VirtualBox... just start the box and you will get aforementioned configuration by default!

Please post a blog about it on blogs.windows.com, so more people know how to configure it for basic usage: like we have Win 8.1 RTM ISOs from BizSpark partner account, but we can't fully utilize it on Hyper-V due to complexity of Virtual Switch settings. Most people ended up using VirtualBox on Win8 to run 8.1 RTM and it worked like a charm..without altering network settings.

Please also cover this scenario: "Using Hyper-V with VirtualBox". Or support Hyper-V plugin development for Vagrant (an opensource Ruby implementation of abstraction of Virtual Machines for developers), so we don't need Hyper-V at all.. github.com/mitchellh/vagrant/issues/1244. Here is the proposed bounty www.bountysource.com/fundraisers/429-basic-vagrant-hyperv-support. If Microsoft chip-in some bucks, we can probably achieve the target.

Thanks.
David
Wednesday, September 18, 2013 2:45:54 PM UTC
I run server as a desktop OS; huzzah for more better performance. Thanks for bringing up the issue.
Jason
Wednesday, September 18, 2013 4:19:47 PM UTC
And another note: fine-tunning performance in production machines can (should?) go beyond changing the default profile.
You can create custom profiles, and do what's best for your use case (what that is can be triky though :-)
Example: if you run a database that is slow-changing, but fast-reading, you might put it all in-memory, push the CPU to max, but allow the hard-disks to go down, maybe even go to sleep.
Mihai
Wednesday, September 18, 2013 5:31:47 PM UTC
@Jimbob McGoo It's such a shame people have to go to lengths like this. A thoughtfully designed Power Management Console would give fine-tuning options like this out of the box.
Sam
Wednesday, September 18, 2013 6:08:28 PM UTC
Jason - Note this applies equally to Windows Client and Windows Server.
Mike Harder
Thursday, September 19, 2013 7:09:02 AM UTC
For One ASP.net, can you guys please bring option to select from multiple layout frameworks besides Twitter Bootstrap? One of the most popular is Zurb Foundation. It would be great if we have choice to select from these popular frameworks when creating new ASP.NET project.

http://aspnet.uservoice.com/forums/41199-general-asp-net/suggestions/4449961-zurb-foundation-as-a-template-option-when-creatin

Please vote it up if you like the idea.

Thanks
Tommy
Thursday, September 19, 2013 8:31:02 AM UTC
Great article Scott and, for me, appropriate and timely. I'm performance optimizing a large (~1.2 MM SLOC) 4-tier MS-stack app, and among a lot of other things noticed that the WS2008 R2 servers (dual hex-core hyperthreaded = 12 physical/24 total cores) were in "balanced" power mode. I profiled one of the problematic business tier operations -- which is compute-bound -- and found that in balanced mode it took 288 sec, while in "high performance" mode it clocked in at 217 sec. That's an instant 25% improvement, which is no joke. (With some code optimizations I got it down to ~77 sec, but that's another story ;-).)

But to the point of your article it certainly is an aspect of performance engineering that folks should be aware of!
Thursday, September 19, 2013 5:53:06 PM UTC
What do you recommend for those of us running VMs in a workplace? For example, we have 3 physical servers, on which run 12 virtual servers. Two of those are VERY busy database servers. Does setting the power mode in the virtual servers have any impact, or is this completely overridden?
Matt
Thursday, September 19, 2013 7:37:31 PM UTC
Matt - Based on my tests the power setting in the Guest OS has no impact. It is only the power setting in the Host OS. And setting "High Performance" in the Host OS does make things faster in the Guest OS.
Mike Harder
Friday, September 20, 2013 12:06:30 PM UTC
I always knew to do this on my own machines but kind of assumed that the power plan for Windows Server would be high performance by default. Quick look on our SQL servers and its set to balanced, fixed now but its another thing to remember.
Pete
Monday, September 23, 2013 11:56:43 AM UTC
Also, note that besides the Windows-supplied power plans, OEMs can add their own power plans as well. On several notebooks, I found that the OEM had created (and set as default) their own power plan. That plan had a maximum of 50% CPU speed while in battery power (and some of those were with Atom CPUs, which aren't known for their high speed at 100%).
Daniel Rose
Monday, September 23, 2013 2:29:02 PM UTC
I guess given the whole emphasis on processor TPC performance, I'm not surprised... though I probably wouldn't have anticipated it on my own equipment.

That said, one of the first things I do with a new laptop is create/configure my own power profile... never goes to sleep, never auto hibernates, closing laptop lid does nothing (I like to close it while I move between rooms, which only takes a min or so), CPU/HDD/LAN/WiFi are always at 100% performance...

While I love my profile, I'll gladly switch away when scenarios dictate (i'm on "balanced" right now), I just like having my profile as a default, or just a click or two away.

More than anything though, I'd be curious if there are any long term impacts on the battery, since my usage tends to encourage either AC power all day, or quick discharge and frequent recharge.
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.