Scott Hanselman

Should one go to Production with a DEBUG Build?

February 11, '04 Comments [15] Posted in ASP.NET | Bugs
Sponsored By

Here's a question that I'd really like to stir up some talk/comments/feedback. 

Should one go live with a .NET (in this case ASP.NET) app compiled as DEBUG?  Some folks have claimed that whatever speed you might gain is pale compared to the usefulness of the occasional late night production debugging session.  A RELEASE build strips out Debug.Write calls, etc., but we use Log4net exclusively, so that doesn't affect us.

Thoughts and comments, oh wise bloggers?  Is this clever, controversial, or stupid?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Wednesday, February 11, 2004 5:34:06 AM UTC
I think there are too many gotchas involved to place it in the clever or controversial categories. Since you're using a logging framework, you should be able to get the same info you'd get from Debug.Write by setting the logging level appropriately. (I haven't used Log4Net, but I imagine it has an Debug endpoint, so if your guys are accustomed to writing to the logging framework instead of Debug.Write, you wind up with Debug output and development time and something entirely different in production.)

Someone enumerated the several "bad things" about debug builds in production; unfortunately I don't remember who. It was either Rico[1] or someone at PDC. If it was Rico, you can check his blog and if it was PDC you can check out the presentations online[2]. (Minor gripe: We paid something like $200 or $500 for those conference DVDs, the content of which is available online for free and also available for free with the Longhorn Migration kit. What's up with that?)

[1] http://weblogs.asp.net/ricom/
[2] http://microsoft.sitestream.com/PDC2003/Default.htm
Wednesday, February 11, 2004 5:54:36 AM UTC
Ah, one very interesting thing I do remember is that the runtime isn't as aggressive about cleaning up dead references with debug code. Example:

1 void SomeMethod() {
2 SomeClass someVariable = GetSomeHugeObjectGraph();
3 if (ImportantClass.ShouldIKeepDoingSomething) {
4 SomeMethod()
5 }
6 }

Code compiled in release mode may release someVariable's reference to the huge object graph at line 3 since it isn't used again. In debug mode the references are kept since you may want to inspect the variable later. Since our code is recursive, this could have pretty significant ramifications. I thought this was only while the debugger was attached, but Doug Reilly set me straight[1].

[1] http://weblogs.asp.net/jkey/archive/2003/12/19/44810.aspx
Wednesday, February 11, 2004 6:21:00 AM UTC
I wanted to ask why you were using log4net in production? I hadn't heard of it until your blog entry, but after looking at the FAQ on sourceforge, I quickly got the opinion that it shouldn't be used in production?

Have you thought about EIF for this same purpose?
Wednesday, February 11, 2004 8:52:30 AM UTC
I did look at the EIF, but I was mostly wanting potentially verbose LOGGING (Trace). I felt the EIF was overkill and a little more complex than needed. Now, that being said, I did abstract the Log4Net system away into Corillian.Logging...I could certainly change the framework and maintain the interfaces if I find the EIF is better.

Seriously, though, Log4Net has been a CHAMP. I LOVE IT.

What about the FAQ gave you the impression it wasn't useful in Production? I plan to turn the loglevel to FATAL and use it.
Wednesday, February 11, 2004 2:40:49 PM UTC
I usually go into production with the RELEASE build. We've taken a very similar approach regarding wrapping log4net, so if need be we can swap in another provider. Since the log4net logs give me everything I need (usually) to diagnose the problem there isn't that much benefit from going into production with a DEBUG build.
Wednesday, February 11, 2004 2:52:51 PM UTC
How about going into production with *both*? You can have side by side assemblies. Make the debug assemblies have a different strong name. You can then use version policy to have fusion load the appropriate one. Then you can use a debug version of one assembly (the one causing the problems) and release of the rest.

Please note I have never done this, but it seems feasible.
Wednesday, February 11, 2004 2:59:29 PM UTC
I should have added that you would likely need to use shadow copy on your appdomain so that you get the different version when you change the policy.
Wednesday, February 11, 2004 4:35:18 PM UTC
You won't receive line numbers in stack trace with release build. With line number it's 10x easier to locate bug (e.g. unhandled exception).
Stack trace is written to log4net log, not shown to user, of course.
Dejan Grujic
Wednesday, February 11, 2004 4:39:59 PM UTC
I was going to mention the same thing as Dejan - having line numbers is huge and in my opinion outweighs any negatives. In our applications any exception is logged and then sent directly into our bug tracking software. All the stack trace and any additional information captured about the exception is put in the ticket. Exceptions can usually be tracked down and fixed within minutes of it happening.
Paolo
Wednesday, February 11, 2004 5:59:02 PM UTC
Just curious, how does Log4Net improve on the Tracing framework that's part of System.Diagnostics? I've been using the Tracing framework and am curious about Log4Net.
Wednesday, February 11, 2004 7:14:58 PM UTC
Well, the second topic in the FAQ http://sourceforge.net/docman/display_doc.php?docid=10328&group_id=31983#reliable says


"Is log4net a reliable logging system?

No. log4net is not reliable. It is a best-effort and fail-stop logging system"

to be sure, the FAQ was written in 2002. I guess it just made me look twice.

Wednesday, February 11, 2004 7:29:51 PM UTC
Ah...by reliable they don't mean "is it reliable" in the literal sense. In the Computer Science Sense of RELIABLE as in, can you guarantee delivery (also Queuing or WS-ReliableMessaging)...in this definition, SQL Server isn't reliable, nor is NTFS. ;)
Scott Hanselman
Thursday, February 12, 2004 7:11:10 AM UTC
Ahhh. I gotcha. I probably should've read further. After your post here, we actually replace a production site build with a DEBUG build because we off too much trouble on the hosting party's servers. I'd really love to get EIF going so we could've just changed the .config file to get what we needed.
Friday, February 13, 2004 12:16:05 AM UTC
Hi Scott,

You can have the best of both worlds with a rather neat trick. The major differences between the default debug build and default release build are that when doing a default release build, optimization is turned on and debug symbols are not emitted. So:

Step 1: Change your release config to emit debug symbols. This has virtually no effect on the performance of your app, and is very useful if (when?) you need to debug a release build of your app.

Step 2: Compile using your new release build config, i.e. *with* debug symbols and *with* optimization. Note that 99% of code optimization is done by the JIT compiler, not the language compiler, so read on...

Step 3: Create a text file in your app's folder called xxxx.exe.ini (or dll or whatever), where xxxx is the name of your executable. This text file should initially look like:

[.NET Framework Debugging Control]
GenerateTrackingInfo=0
AllowOptimize=1

Step 4: With these settings, your app runs at full speed. When you want to debug your app by turning on debug tracking and possibly turning off (CIL) code optimization, just use the following settings:

[.NET Framework Debugging Control]
GenerateTrackingInfo=1
AllowOptimize=0

Hope this helps,

Mark
Monday, May 09, 2005 11:47:16 AM UTC
We made heavy use of EIF on a project. Here’s my analysis.

EIF Shortcomings:
The log file viewer is very limited:
Doesn’t sort correctly (because everything is treated as a string; this ruins timestamp sorting)
No filters
Doesn’t remember UI layout
No way to segment data (i. e., a new log every day)
Finicky configuration
Must be separately installed on deployment platform
Event sources must be pre-created using EventLog.CreateEventSource

Good things about EIF:
Handles large amount of data and high volumes well
Stable once configured correctly
David Douglass
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.