Scott Hanselman

Just a reminder: Release builds are not Debug builds. Seems obvious, but it's worth saying again. Release builds are optimized for speed and debug builds are optimized for, well, debug-ability. However, most of the optimizations in managed code are done by the JIT compiler rather than the language compiler.

However, sometimes in our zealous attempts to make Release builds easier to debug we can inadvertently (or totally "vertently") remove a lot of the optimizations that make Release builds faster.

Introduction

I was talking to Jeremy and he had come upon some code that was inserting a try{} catch() { throw; } block in every method. The rationale was that "they wanted the complete stack trace in their logging after an exception was thrown." That's a noble goal, but their technique of adding these guards is what's called an "anti-pattern" or as Egon would say "it would be bad." An anti-pattern is "a pattern that tells how to go from a problem to a bad solution."

First, let's start by exploring why they aren't getting "a complete stack trace" before we tackle why they believe they want/need a complete one.

Here's a simple program:

using System;

class NormalProgram
{
    static void Main(string[] args)
    {
        try
        {
            methodA();
        }
        catch (System.Exception e)
        {
            Console.WriteLine(e.ToString());
        }
    }
    static void methodA() { methodB(); }
    static void methodB() { methodC(); }
    static void methodC() { badMethod(); }
    static void badMethod() { throw new ApplicationException("generic bad thing"); }
}

In this program, Main calls A, then A calls B, which calls C, which calls badMethod which throws an exception. That exception is caught up at Main. So far, makes sense right?

We'll compile two versions, one in Debug mode and one in Release and run them on a regular 32-bit machine. Notice we're saying /o for enable optimizations for the Release build. We are making debug symbols for Release via /debug:pdbonly, but it doesn't affect performance in a significant way.

"%FXROOT%\csc.exe" /t:exe /out:NormalRelease.exe /debug:pdbonly /o NormalProgram.cs
"%FXROOT%\csc.exe" /t:exe /out:NormalDebug.exe /debug NormalProgram.cs

We run Debug on 32-bit and we see:

System.ApplicationException: generic bad thing
   at NormalProgram.badMethod() in NormalProgram.cs:line 24
   at NormalProgram.methodC() in NormalProgram.cs:line 23
   at NormalProgram.methodB() in NormalProgram.cs:line 21
   at NormalProgram.methodA() in NormalProgram.cs:line 19
   at NormalProgram.Main(String[] args) in NormalProgram.cs:line 11

Looks good, makes sense. Now we run Release on 32-bit:

System.ApplicationException: generic bad thing
   at NormalProgram.badMethod() in NormalProgram.cs:line 24
   at NormalProgram.Main(String[] args) in NormalProgram.cs:line 11

Our call stack has been "collapsed"? Does it make it harder to debug? Not really, because we knew where we ended up and the path between the two is clear (otherwise this "inlining" wouldn't have happened.)

However, it is confusing and there's a perception that "information is missing." It's important to note that information isn't missing but rather this stack trace is showing the runtime reality. What you write (the Programmer's Intent) isn't exactly what runs, especially when things are optimized. As programmers we are doing our best to tell the machine what to do and it is doing its best to do that fast and correct - let it do what it does best. When we see a stack like this our first reaction is that information is there, but has been hidden. In fact, the code we wrote ran, but not every method got their own local stack frame as they were optimized into fewer methods.

Inlining

Disclaimer: I'm showing you this for informational purposes. Doing this without thinking or having a good reason is usually  a bad idea and could confuse other issues. The only reason you might want to do this would be if you were doing some kind of funky thing that made your method require its own stack space. DON'T ADD THIS ATTRIBUTE WILLY-NILLY. Don't Program By Coincidence. Remember the point of this post is that Release and Debug are different. Use them differently.

What if I tell the runtime JIT-ter to not inline? I'll add one line and one using:

using System;
using System.Runtime.CompilerServices;

class NormalProgram
{
    static void Main(string[] args)
    {
        try
        {
            methodA();
        }
        catch (System.Exception e)
        {
            Console.WriteLine(e.ToString());
        }
    }
    [MethodImpl(MethodImplOptions.NoInlining)]
    static void methodA() { methodB(); }
    static void methodB() { methodC(); }
    static void methodC() { badMethod(); }
    static void badMethod() { throw new ApplicationException("generic bad thing"); }
}

And run on 32-bit:

System.ApplicationException: generic bad thing
   at NormalProgram.badMethod() in NormalProgram.cs:line 24
   at NormalProgram.methodA() in NormalProgram.cs:line 19
   at NormalProgram.Main(String[] args) in NormalProgram.cs:line 11

Well, that stopped the inlining of method A, but of course B and C are still inlined. You can see how this is going to get icky.

ASIDE: We as programmers have all reached this moment. This is the moment when we decide whether or not to slap an attribute on every method in our program. (This attribute is just an example, but you know this moment...there's still time to actually stop and try to understand the problem...)

...but we have to ship. So, let's try this:

using System;
using System.Runtime.CompilerServices;

class NormalProgram
{
    [MethodImpl(MethodImplOptions.NoInlining)] 
    static void Main(string[] args)
    {
        try
        {
            methodA();
        }
        catch (System.Exception e)
        {
            Console.WriteLine(e.ToString());
        }
    }
    [MethodImpl(MethodImplOptions.NoInlining)]
    static void methodA() { methodB(); }
    [MethodImpl(MethodImplOptions.NoInlining)] 
    static void methodB() { methodC(); }
    [MethodImpl(MethodImplOptions.NoInlining)] 
    static void methodC() { badMethod(); }
    static void badMethod() { throw new ApplicationException("generic bad thing"); }
}

And compile and run as Release on 32-bit...

Now we're back where we were before, right? Well, not really. We've got the full stack again, but we've neutered the build such that the JITter can't really do it's job.

Plus, we haven't tried 64-bit yet. Let's run the Release build on 64-bit:

System.ApplicationException: generic bad thing
   at NormalProgram.methodC() in NormalProgram.cs:line 23
   at NormalProgram.Main(String[] args) in NormalProgram.cs:line 11

Weird. One conclusion to draw would be that 64-bit CLR ignores method inlining attributes. That's not necessarily the case. Another would be that 64-CLR is more aggressive about inlining. The deal is that there are MANY optimizations that the JITter can to. MANY. They wouldn't give me a number because it's Special Sauce, but le's just say it's not one or two. It's more than that. ;)

Inlining isn't the only thing that can "collapse" a call stack like this. This also isn't exactly true. In this case, remember that x64 is a whole new processor architecture while x86 has been around for, let's say, a while. Turns out that x64 really likes to optimize and this is an example of a "Tail Call Optimization" rather than explicit Method Inlining.

Be sure to read David Broman's detailed blog post on exactly x64 tail-call optimizes and when it doesn't.

A good explanation from The Original Wiki at the Portland Pattern Wiki:

"Tail-call optimization (or tail-call merging or tail-call elimination) is a generalization of TailRecursion: If the last thing a routine does before it returns is call another routine, rather than doing a jump-and-add-stack-frame immediately followed by a pop-stack-frame-and-return-to-caller, it should be safe to simply jump to the start of the second routine, letting it re-use the first routine's stack frame (environment)...."

"However, TailCallOptimization has some drawbacks. The runtime environment will have a confusing stack during execution of tail-called routines, which can make debugging difficult ("How did I get here? baz() never even calls foo()!")."

The wiki explanation was written with C in mind, but the concept is universal and it's the "confusing stack during execution of tail-called routines" that's got me writing this blog post today.

There's some controversy on whether Tail Call Optimizations are a subset of Method Inlining. That may never be resolved, but for now they are discrete and different. Eric Gunnerson has a good article on some of the heuristics that the JIT uses in its inlining decision making process.

For some hardcore details, be sure to read David Notario's excellent series on x86 JIT inlining from 2004, both Part 1 and Part 2. Perhaps start by reading his "The CLR x86 JIT, an overview" as he explains seven basic stages the x86JIT.

An Anti-Pattern

Now, back to our customer from the beginning who was inserting a try{} catch() { throw; } block in every method. Let's try that (don't do this at home):

using System;

class UglyProgram
{
    static void Main(string[] args)
    {
        try
        {
            methodA();
        }
        catch (Exception e)
        {
            Console.WriteLine(e);                
        }
    }
    static void methodA() { try { methodB(); } catch { throw; } }
    static void methodB() { try { methodC(); } catch { throw; } }
    static void methodC() { try { badMethod(); } catch { throw; } }
    static void badMethod(){throw new ApplicationException("generic bad thing");}
}

...and compile both Debug and Release and this produces the same output on both 32-bit and 64-bit:

System.ApplicationException: generic bad thing
   at UglyProgram.badMethod() in UglyProgram.cs:line 20
   at UglyProgram.methodC() in UglyProgram.cs:line 19
   at UglyProgram.methodB() in UglyProgram.cs:line 18
   at UglyProgram.methodA() in UglyProgram.cs:line 17
   at UglyProgram.Main(String[] args) in UglyProgram.cs:line 10

Why? Because by setting up the try's you're introducing the potential for other ways out of each method so the JIT'ter can't optimize anything that would collapse the call stack like we saw earlier. This isn't a good pattern as it doesn't deal with the underlying issue in a simple way. The customer wants to be able to debug Release builds easier.

Options

Here's some options we could try:

• Try/catch every method - Not a good idea. Not only are you cluttering your code, but you're paying a cost all the time when you only need to debug in Production/Release occasionally.
• Put [MethodImpl(MethodImplOptions.NoInlining)] on every method - Also not a good idea, not only because of the operative "every method" violating the DRY Principle but also because it only deals with one specific optimization out of many.
• Turn off Optimizations in the compiler - This particular kind of inlining appears to be a JITter thing, not a language compiler thing so turns out that doesn't work in this instance.

Ok, so the customer wants a fuller call stack. I would propose that they in fact don't want this. One of my old bosses used to say:

"Customer walks in with a cell phone and says 'this thing needs a bigger antenna.' We have to ask ourselves does he want a bigger antenna or better cell phone reception" - Mark Klein

This is a great pithy analogy. Everyone comes in with both Problems and Solutions. This customer wants to be able to debug Release builds. They surely don't want full call stacks in Release Mode if it means turning off all optimizations.

Best Solution

One thing the customer could do that would give them the best of both worlds is something Mark Pearce told me about a while back, the [.NET Framework Debugging Control] section of an .ini file you've probably never used:

This JIT configuration has two aspects:

• You can request the JIT-compiler to generate tracking information. This makes it possible for the debugger to match up a chain of MSIL with its machine code counterpart, and to track where local variables and function arguments are stored.
• You can request the JIT-compiler to not optimize the resulting machine code.

If you have a file Foo.exe you can create a Foo.ini with these contents:

[.NET Framework Debugging Control]
GenerateTrackingInfo=1
AllowOptimize=0

Implementation Detail: Why is it an INI file? Well, the .NET runtime hasn't started up yet, so we don't have all that XML-parsey goodness lying around. Instead they need to use the Win32 method GetPrivateProfileString() in order to retrieve those.

This solution assumes you compiled with /debug:pdbonly in Release mode in order to generate the full stack when needed.

DICSLAIMER: This isn't something you want to put into production! You'd only use this for debugging. It's something you'd enable on a case-by-case basis.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

2010 UPDATE: This is cool and interesting to read, but you should just go get this Free Outlook 2007/2010 No Reply All AddIn to do this work for you.

UPDATE: To be clear. This technique does NOT send macros in your email. It only flips a metadata bit in the message and that metadata is only transmitted within Exchange (within your company). It is not propagated to outside email addresses. It's better than BCC because it doesn't break local Outlook Rules. It's harmless.

I really hate it when I say "please send replies directly to me" and a Reply To All happens. It's not only a bummer for the person who sent it but also for everyone on the distribution list.

Trivia: At Microsoft they/we say "little-r" me when we want a reply directly to us or "Big-R" when we mean Reply To All.

I was thinking it'd be cool to have a button on my Outlook Message Form that prevented folks from Reply'ing to All or Forwarding the message. I poked around a bit trying to write an Outlook Macro and realized that I've completely overwritten all the brain cells that had previously held information about VBA macro programming. Seriously. I worked in VB3-6 for years. I was a ninja. Now I'm just an old fat guy with a Black Belt that used to fit. Anyway.

I asked for help and KC Lemson pointed me to Bill Jacob, both actual ninjas, and he took my Rube Goldberg-ian plan and turned it into two lines of code. Doh.

Here's "How to easily disable Reply To All and Forward in Outlook":

Go Tools|Macro|Macros...

In the next dialog, type something like NoReplyAll and click Create.

At this point, even when running Vista 64, you'll be magically transported back to 1996, completely with owner-draw non-standard gray toolbars and other bits of gray that will leak in from the past.

Add these lines to your new subroutine:

ActiveInspector.CurrentItem.Actions("Reply to All").Enabled = False
ActiveInspector.CurrentItem.Actions("Forward").Enabled = False

Then close this window.

At this point you've got a macro that prevents Replying to All and Forwarding (at least within Outlook world. This won't prevent folks running other mail systems from Replying to All, but we're mostly focused on internal work with this tip.)

Now, open up a new Outlook Message and right click at the VERY top (assuming Outlook 2007).

Click More Commands...now from this dialog select "Macros" from the dropdown, select your new Macro and click Add>>.

If you like, click on your Macro on the right and select the Modify button and pick a nice icon for it and a Display Name. I used a "halting hand" icon:

Click OK and look at your Quick Access Toolbar...you've got a nice little icon there.

Now, CLICK that button then send an email to yourself or a coworker...

Cool, mission accomplished. One less thing to worry about. Thanks Bill!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

It's Carl as a one-man-band playing piano, drums, bass and guitar, mixed together seamlessly into one amazing Hi-Def 720p video. Remember to double-click for fullscreen once you're in the player.

He explains the technical details on his blog. How he recorded it, which cameras, what programs and the final squishing with Expression Media Encoder. Here's the link to his original test and to

You wanna talk multi-talented? I need to learn the piano. Great job, Carl, truly amazing.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

Hello Dear Reader. I have been working with IIS7 for a while and I'm convinced that it's the cat's pajamas^[1]

I started playing with IIS7 and noticed that it had FastCGI support. This means I could plug in PHP or maybe Perl or even Ruby into IIS7. This would be nice because I could host my ASP.NET blog, but also drop in some of the nice open source PHP applications that are available for maybe a photo gallery or something, all hosted on the same IIS machine.

What I did was take Bill Staples blog post on FastCGI as a guideline and got IIS7, FastCGI and PHP running on my machine. I did three load tests, one with CGI, one with FastCGI and one with Kernal Output Caching (new IIS7 feature) turned on. This video shows a number of tools and how to configure IIS7 step by step.

You can watch the video/screencast on the new Hanselman Silverlight Player (thanks Tim!) or download the WMV directly. If you guys want more IIS7 videos in detail, give me feedback in the comments and I'll see what we can do about putting together a series over on http://www.iis.net.

FastCGI GoLive on IIS6

If you're running IIS5.1 or IIS6, there is a GoLive release available to put your PHP apps into production on IIS and FastCGI. If you've previously downloaded the FastCGI TP2 release, be sure to get the latest GoLive version for IIS5.1/6.

There's also an actively maintained FastCGI IIS Forum with members of the team and MVPs helping out.

FastCGI for IIS7

If you're running IIS7 on a non-SP1 Vista, you can get FastCGI as a download for x86 and x64 as well. However, if you're running Vista SP1 Beta, as I am in the video, or Windows 2008 RC0, then you've already got FastCGI. One less step, eh? Check it out in the video.

WCat (Web Capacity Analysis Tool) 6.3

In this video I use a tool call WCat that you can download in x86 and x64 flavors. It is very lightweight and can simulate thousands of concurrent users on even a laptop. I pushed my local IIS7 with caching to over 2000 requests a second. It's free, easy with a basic scripting language. It's a great way to beat on your development servers and do some powerful profiling. I loves me some free tools.

Screencast Survey

I have been thinking about doing a series of IIS7 screencasts to augment the very good articles on http://www.iis.net. If you haven't been over there, I recommend you check it out.

Anyway, if you've seen me speak on stage, you know I'm a visibility/accessibility wonk and I really like to think about how folks learn, etc. I've been working with Camtasia for a while now and doing some video editing in Sony Vegas. In the recent ALT.NET Videos I put myself in PIP (Picture in Picture) and the response (even though the video was very rough) was very positive. I think that PIP really adds a lot to a screencast, but only if combined with appropriate editing, callouts, zooming and moving/sizing of the PIP window to make sure nothing is obscured. I wonder if you agree?

I'm interested in both your thoughts and opinions on the FastCGI stuff but also on Screencasts in general:

• I'd also like to try creating screencasts that look great at 640x480 but also would be viewable on a 320x240 screen using Zoom and Pan or a Viewfinder technique. Do you have any interest in that?
• Are screencasts a big part of your learning process?
• Does Picture in Picture add value to you?
• Do you prefer a fairly casual screencast with PIP like I've done here, or a more formal greenscreen/floating head with a nice suit reading a script. This screencast was fairly "organic" on purpose because it's real. There's no fakery.

Thanks, Dear Reader.

^[1] Cat's Pajamas - An adjective used by hipsters of the 1920's to describe a person who is the best at what they do. lso used to describe another person who is genial and fun to be with.
"Martin sure knows how to dance, he's the cats pajamas, man!""

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

UPDATE: Here's a Bit.ly Bundled Link of the complete "Wiring your house for Gigabit Ethernet 5 PART SERIES."

Looks like folks ARE interested in the Home Wiring topic. I'm not an expert, but I do read a lot. Here's some answers to some great questions in the comments.

Q: "Wooden walls... huh... wooden walls. Umm, right, whatever. But why don't you use tubes inside the wall that hold the wires?
Right now when you close the wall, you can't add new cables, right?
Over here in The Netherlands/Europe we embed tubes in the concrete or bricks so we are able to pull new wires through it." - Rutger

A: Sure, that's sometimes used. Usually instead we just pull WAY more cable that we'd need. Wooden walls "can" be opened (with trouble) and additional pulls can be "fished" through, but your point is well taken, this is a hassle. We've done a combination by pulling lots of wires, but also by including a 2" conduit (in orange in the picture) leading between floors as well as to the main entertainment center (TV).

Q: "what about sound isolation? Seems to me pretty noisy." - Rutger

A: The closet will just have the Home Server, Media Server, the switch and the wireless router. The servers are fairly quiet, but certainly not silent. I haven't given much thought to sound isolation in the wiring closet, instead focusing on the room I record Hanselminutes in. If it becomes a problem, first I'd add a sealing strip of rubber to the bottom of the door, and then I'd look at "blowing in" extra insulation in that one wall, and finally I'd line the one inward-facing wall with sound dampening material.

Q: "Let me guess - you handle the tech, while your wife is in charge of picking out the color scheme, window treatments, appliances, granite countertops, carpeting, landscaping, shower curtains, decor, and new furniture?" - Frank

A: Uh, gulp. Actually, if you knew some things about me you'd know I'd kind of a Nate Berkus about this stuff. I came up with most of the colors, the window treatments, all the appliances. Mo and I did the countertops (ceramic, not granite) and carpets together. I'll do most of the furniture and decor, accents and landscaping, fencing, etc, and she'll do art, photos, pictures, knick-knacks etc. I actually just spent my lunch hour at Bed, Bath, and Beyond.

Q: "I was a little surprised to see you getting a Netgear Switch. Netgear is a good consumer brand but they really don't have the chipsets to compete with the big boys such as Dell, HP, and Cisco. See through put is all dependent on how much the chipset can handle. Really what you want to look at is the switching capacity because it is totally different than bandwidth." - Nick

A: Hm. I picked up a nice Netgear GS724TS for literally nothing on Ebay. I can add more switches as I need to, and it has a switching capacity of 20Gbps. Even if I had every port full (I won't) with everything running at 1Gbps (I don't) I'd still be "ok." Sure I could have got an HP with 48Gbps capacity, but it seems overkill and was more money. The Dell you references only had an 8Gbps capacity, so I think I did well for very little money. Of course, the switch is just mounted into the closet, so it can always be popped out for the future, but I think 20Gbps is pretty reasonable.

Q: Also, designate where you electrical taps are and ensure your office has at least 3 dedicated circuits if you're going to have a reasonable amount of gear/UPS. I took the rule of a jack box a foot away from each power drop so that I didn't end up with wires all over the floor. For instance, I have three separate jack plates in my office alone, each with 2+ CAT5e jacks. The one by the server has 6 CAT5e jacks so that everything can have a dedicated drop to the switch to avoid overloading a single cable. - Chris

A: Totally agree. I've got two dedicated 20 amp circuits in my office and another 20 amp in the wiring closet.

Q: What model of switch do you have? You said it can do RJ11 ports, which sounds like a cool feature. - Tony

A: It's not the switch that does the RJ11, basically it's the punchdown block. There's a separate "hub" thing for RJ11, and if I want to change a Data Run into a Phone Run, I just move the run from the switch into the phone hub. Then can put RJ11 wires into an RJ45 in the room and I'm set. I'm moving off of Vonage now that they are imploding and going back to Verizon for phone service, but we'll be using Skype exclusively for our many overseas calls.

Again, I'm just stumbling through this, it's only the second time I've put together this kind of setup and the first time was only 8-ports and 100mbps, so I DO appreciate all your comments and (constructive) criticisms!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By