Well here's an interesting thing, considering that this is a technical blog.

Last week I wrote a post on how I was offended at CNN's posting of a picture of a white face on their home page when ~20,000 non-whites were dead (>120,000 now, likely 150,000 soon and one third were children). I felt, and still feel, that it was an ethnocentric decision by CNN.

Since, for whatever reason, CNN has posted more sensitive pictures, but not before they did two days of coverage on a Sports Illustrated Swimsuit Model who survived and a whole story on celebrities affected.  Certainly I wasn't the only one who noticed this, and a number of people mentioned it to me, both publically and privately. The Progressive Magazine did an article in the same vein as my post called "NYT says Tsunami Kills White People Too."

A U.S. government site has quote from President Bush about relief aid.

[For example], in the year 2004, our government provided $2.4 billion in food, in cash, in humanitarian relief to cover the disasters for last year. That's $2.4 billion. That's 40 percent of all the relief aid given in the world last year, was provided by the United States government. No, we're a very generous, kindhearted nation.

The interesting things are the $2.4B the government gave this year to humanitarian relief (it doesn't say if it include the Tsunami $350M), and the statement that it was 40% of all relief give in the world. It's good we gave more that any country in the world, considering that our GDP is nearly 11 MILLION MILLION dollars, almost double our nearest competitor, China, and almost greater than the entire rest of the top ten.

The government aid was originally $4M, then $35M on the third day (the same amount pledged by the Pzifer corporation alone) and today was upp'ed the amount of government-pledged aid to $350M. This number will likely change as the death toll rises. This is in comparison to the $13.6 Billion that has been pledged for Florida Hurricane relief. In another comparison, Bill Gates and his wife gave $168M last year to Malaria research alone and another $42M this year.

Anyway, I received this email today in criticism to my earlier post on CNN's ethnocentric news focus.

Shame on you. Americans pay attention to other Americans killed in disasters partly because it hits close to home, and reminds us of how those thousands of people over there feel. America is a very empathetic country.
The Americans have given $350 million dollars to this disaster. How dare anyone criticize them. As for media, anyone with an IQ over 50 knows never to trust the opinions of the media.
For some reason, America is seemingly the most criticized nation as far as it's moral standards and supposed "self-centeredness" goes. However, America gives 40% of the world's disaster relief funds every year. America rushes to help, and immediately orchestrates organizations for rescue and help whenever something goes wrong...
When 9/11 happened, who rushed to their side? Not really anyone, at least no other country in a heroic way. Instead, the Americans themselves took charge, and they didn't give anyone else any grief at all for not giving enough. America isn't perfect, but they do a lot of good for this world.
America has good intentions, and have proven time and time again their selflessness. Anyone who continues to criticize despite this has issues, and needs a reality check.
Either way, is your bitching about the wrong doings of American media helping this situation at all? No, but the American Red Cross is...
You're a flippin' hypocrite- and I'm not even American. Lay off, you come off as an arrogant swine.
[Heather Erickson-Sander]
I was bummed to get such negativity in my inbox so close to the end of the year. This email was so vituperous it triggered my spam filters, and I might not have seen it if I didn't fish it out.
 
A few things stood out in this response other than "flippin' hypocrite" and "arrogant swine" as I get those all the time. :)
 
"How dare anyone criticize [The Americans]." Personally, I think that there are few things on this earth that are beyond reasoned criticism. Certainly that's why I posted this critical email on my blog. Like it or not, American is big and worthy of both praise and criticism.
 
"When 9/11 happened, who rushed to their side?" Invoking 9/11, a tragic, but non-natural disaster that killed 1/50th of the people this tsunami did and that U.S. citizenry and government was capable of handling, doesn't seem like a reasonable parallel. Remember that over 1.5 BILLION was raised after 9/11, much of it by citizens domestically, more that could be distributed.
 
"America has good intentions, and have proven time and time again their selflessness. Anyone who continues to criticize despite this has issues, and needs a reality check." To be clear, I was criticizing CNN's editorial staff, but I can see how a media outlet can be confused as the face of a country.
 
Interestingly, she is cut-and-pasting variations of this criticism on other blogs under different names. I would encourage her to channel her enthusiasm towards more useful ends.
 
Greg Hughes and I have done what little we can by pressuring Bloggers and Google to donate their AdSense revenue. This has worked very well, and it looks like our little movement has taken off. Many other bloggers in other niches had the same idea simultaneously and we've received word that Google has noticed the movement and is exploring options.
 
I post this as the New Year fast approaches, and it's good to end the year on an introspective note. The human condition is overwhelming. Who are we to be blessed so? 150,000 people tragically dead today that weren't on Christmas, and expected to see the New Year themselves. 5 million people homeless today that weren't last week, but I sit here warm in my house. There but for the grace of God go I.
 
Acute and instant tragedies like this inspire people to give. Hearing that 150,000 died in a short period of time is overwhelming. However, as you enter the new year and plan your giving, remember that nearly 3,000,000 people die of Malaria every year, and 3,000 children every DAY. That's 250,000 people a month and causes 50% of the deaths of African children. Roughly 2% of Africans in Africa have AIDS, but 25% have Malaria each year. Give often and always. Give to those causes that you feel need help, but have a healthy perspective during this difficult time.
 
In conclusion, I'd like to point you to another useful thing.  India Today has an interesting currency neutral chart that suggests an appropriate level of giving based on your yearly and daily salary. I think it's more than reasonable to expect everyone who is able to give a few days or a week's pay to help assist in the worst natural disaster that, God willing, Insha Allah the likes of which we will never see again.
 
Give. I'll see you next year. I'll try to dial up the technical content and dial down the editiorials, thanks for your patience.



There's an amazing essay at The Spurious Pundit on "Picture Hanging." It's an allegory that explores how simple requirements in software aren't that obvious to folks who may not have context. The writing is wonderful, do check it out, it's worth your time. Subscribed.

A highlight:

You tell him to hang the photo of your pet dog, and he comes back a week later, asking if you could "just double-check" his design for a drywall saw.

    "Why are you designing a drywall saw?"
    "Well, the wood saw in the office toolbox isn't good for cutting drywall."
    "What, you think you're the first person on earth to try and cut drywall? You can buy a saw for that at Home Depot."
    "Okay, cool, I'll go get one."
    "Wait, why are you cutting drywall in the first place?"
    "Well, I wasn't sure what the best practices for hanging pictures were, so I went online and found a newsgroup for gallery designers. And they said that the right way to do it was to cut through the wall, and build the frame into it. That way, you put the picture in from the back, and you can make the glass much more secure since you don't have to move it. It's a much more elegant solution than that whole nail thing."
    "..."

This metaphor may be starting to sound particularly fuzzy, but trust me - there are very real parallels to draw here. If you haven't seen them yet in your professional life, you will. [Spurious Pundit]



Apple.com has taken mind-boggling severity of the Indian Ocean Tsunami and updated their corporate homepage to drive drive to donation sites.

It's a definite statement when a company chooses to remove products from their home page.

The power of the Net with this disaster will be realized if it's made fantastically easy to donate money to legitimate charities.

Here's some photos that show you the before, recoil and push foward of the tsunami in Sri Lanka. There is an excellent PDF analysis here.

Typical Shot (2004-Jan)

Srilanka_kalutara_beforeflood_jan1_2004_dg

Ocean pulls back 400 meters...(2004-Dec-26)

Srilanka_kalutara_beach2_dec26_2004_dg

Tsunami arrives...(2004-Dec-26)

Srilanka_kalutara_flood_dec26_2004_dg

 



Greg Hughes and I were talking about this idea. The power of blogging isn't citizen journalism, it's the power to start a movement.

Nick Bradbury is donating his profits to the Red Cross. Kudos Nick. Let's ALL take our passive Google Adsense Revenue for the year and donate it directly to earthquake relief. Mine so far is US$-omittedduetogooglepolicy- since I started ads in June. I'm sure hundreds of thousands, even millions could be raised quickly in this manner.

To that end, let's pressure Google into allowing us to automatically donate our revenue from their side! Spread the word and trackback this link.

To me, spreading an idea like this is the power of blogging, more than citizen journalism.



Certainly Suzanne Cook is the definitive source for details on Assembly.Load and Binding Contexts, a black art tantamount to voodoo that few people understand. Patrick and I have been hip-deep in it lately, and have discovered/uncovered/madecleartoourselves how some of this voodoo works. Here's a (annotated) writeup from Patrick that was sent out internally. Another great resource is Mike Gunderloy's article on Binding Policy in .NET.

  • Assemblies will only EVER be loaded from the GAC based on a full bind (name, version, and public key token).  A partial bind with name and public key token only WON’T load from the GAC. 
    • If you reference an assembly with VS.NET you're asking for a full bind. If you say Assembly.Load("foo") you're asking for a partial bind.
  • However, the way this usually works is…
    • You do a partial bind on assembly name, or name and public key token with Assembly.Load
    • Fusion (the code name for the Assembly Loader/Binder) starts walking the probing path looking for an assembly that matches the partial bind.
    • Counter Intuititive: If it finds one while probing (the first one) it will then attempt to use the strong name of the one it found to do a full bind against the GAC.
    • If it’s in the GAC, that’s the one that gets loaded.
    • Any of that loaded assemblies will try to load from the GAC first without going to the probing path, since the embedded references constitute a full bind.
    • If they aren’t found in the GAC, then it will start probing.
    • It’ll grab the first one it finds in the probing path.  If the versions don’t match, Fusion fails.  If they do match, Fusion loads that one.
    • So, if you specify a partial name, and the file is in the GAC, but not the probing path, the load fails, since there’s no way to do a full bind.  

All this is mostly an issue for plugins that we load dynamically.  It shouldn’t be an issue for compile-time dependencies, since they use full binds.  One way to make sure you get what you expect is to specify a full bind in your config files via an Assembly Qualified Name (QN) like:  "Foo.Bar, Version=2.0.205.0, Culture=neutral, PublicKeyToken=59388ae2d2746794" and doing something like this:

  100 public static object CreateInstance(string assemblyandtype)
101 {
102 Type type = Type.GetType(assemblyandtype);
103 object instance = null;
104 instance = type.InvokeMember(String.Empty,BindingFlags.CreateInstance, null, null, null);
105 return instance;
106 }

Thanks to Patrick for the writeup!



earthquake1.jpg

This is a technical blog, so note that I don't do this all the time so forgive me ahead of time if you don't like my rants.

I'm looking at the home page of CNN.com.  There are currently over 22,000 brown people dead, and apparently this white guy. I feel for his family, but I feel more for the countless hundreds of thousands of displaced and suffering others.

I am tired of the American Media (last night it was ABC News) who spend 5 minutes on a massive natural disasters, and then gloss over it when we are told "and no Americans were injured." Notice the text of this CNN blurb - 22,000 dead and 27 Western People, which details the counts of British, French and Italians.

This is unspeakably ethnocentric and it makes me a little ill.  I'm not trying to be P.C. here, but these are humans, and whether it was a hundred Somali Fisherman, or this guy from Illinois, I expect more from a leading news organization. This is like a home-town newspaper concerned about its native son away on mission.

We must never forget that tommorow isn't promised to us. One day there will be an earthquake off the coast of Oregon. No doubt that will get media coverage.

God help us all, but thanks for the time I've had.



CR_Documentor_smTravis has rev'ed CR_Documentor to 1.1.0.1220 including these new features:

  • Has been updated for NDoc 1.3 tags
  • Provides the option for what level of "tag compatibility" to follow (Microsoft tags only or NDoc 1.3)
  • Provides the option for how to handle "unrecognized" tags
  • Has updated styles to match NDoc 1.3
  • Has been updated to work with CodeRush 1.1.6 / DXCore 1.1.8 or higher

If you're not familiar with this project, Travis talked to Lutz and informally "took over" the applet originally know as Documentor that let a developer see what the compiled results of XML Documentation comment would look like.  Travis has since extended documented and made it into a CodeRush plugin that runs in a toolwindow within Visual Studio. It will let you see a preview in realtime as you type of your comments.

If you are trying to do extensive C# doc or if you use NDoc, this is a great tool for you. Thanks Travis!



ASP.NET Performance Tuning - Making dasBlog faster...

Posted 2004-12-18 01:39 AM in ASP.NET | DasBlog | XML | Bugs | Tools.

Greg Hughes and I stayed up late last night tuning and installing a custom build of dasBlog. If you remember, dasBlog is Clemens' rewrite/imagining of Chris Anderson's BlogX codebase that has been moved over to GotDotNet and is under the supervision of Omar Shahine.

ORCSWeb, my awesome ISP, and Scott Forsyth (ASP.NET MVP who works there) had complained to me that as my traffic increased, my website instance was being a poor citizen on the shared server. My site is on a server with something like a dozen other sites. While I'd survived slashdotting, my traffic lately has been getting big enough to bother the server.

ScottF had noticed that my blog had these unfortunate characteristics (remember these are bad):

  • CPU Threads that were taking minutes to complete their work
  • Caused 9,538,000 disk reads during a time period while another site on the same server with twice as many visitors had 47,000 reads.
  • My process was tied for CPU time with "system."
  • I used 2 hours, 20 seconds of CPU time one day. My nearest competitor had used only 20 seconds.
  • I was 2nd for disk reads, and 11th for disk writes (the writes weren't bad)
  • In a day, I surpassed even the backup process which was running for a WEEK.

These bullets, of course, are quite BAD. So, during my recent burst of creativity when I added a number of features to dasBlog including a comment spam solution, a referral spam solution, and an IP address blacklist, I did some formal performance work.

If you're familiar with dasBlog, I yanked the need for entryCache.xml, categoryCache.xml and blogData.xml, which were older BlogX hold overs, and move them into thread-safe in memory storage. I change the EntryIDCache and other internal caches, and added outputcaching for RSS, Atom, and Permalinks.

According to ScottF and the folks at Orcsweb and their initial measurements "from what I can tell today, this *is* 250% better. CPU used only 20 minutes [as opposed to nearly 2.5 hours] of time by the end of the day and disk IO was much less than normal." This is early, but we'll see if these numbers hold.

I seem to have a few other bugs to work out, so hollar at me if the site's goofed, but otherwise I hope to get Omar to integrate these changes into his own great new stuff coming in dasBlog 1.7.

During this perf test, I used Perfmon, CLR Profiler and other tools, but mostly I thought. Just literally say down and thought about it. I tried to understand the full call stack of a single request. Once you really know what's going on, and can visualize it, you're in a much better position to profile.

Since you are a technical group, here's a few tidbits I found during this process.

  • If some condition can allow you to avoid accesses to expensive resources and bail early, do. For this blog, if an entry isn't found (based on the GUID in the URL) in my cache, now I won't even look in the XML database. Additionally, I'll send a 404, use Response.SupressContent and End the Response.
        1 if (WeblogEntryId.Length == 0) //example condition

    2 {
    3 Response.StatusCode = 404;
    4 Response.SuppressContent = true;
    5 Response.End();
    6 return null; //save us all the time
    7 }
  • Lock things only as long as needed and be smart about threading/locking.
  • If you're serving content, caching even for a few seconds or a minute can save you time. Not caching is just wasting time. Certainly if I update a post, I can wait 60 seconds or so before it's seen updated on the site. However, if a post is hit hard, either by slashdot'ing or a DoS attack, caching for a minute will save mucho CPU.
    <%@ OutputCache Duration="60" VaryByParam="*" %>
    at the top of one of my pages will cache the page for a minute using all combinations of URL parameters. To be thorough, but use more memory, one would add VaryByHeader for Accept-Languages and Accept-Encoding, but this is handled in my base page.

 

 



If you want to use the XmlSerilializer, you'll (ASPNET user) need write access to the Windows/Temp folder. Otherwise you may see this as the temporary assembly fails to be saved:

File or assembly name zmp0husw.dll, or one of its dependencies, was
not found.
Description: An unhandled exception occurred during the execution of
the current web request. Please review the stack trace for more
information about the error and where it originated in the code.

Exception Details: System.IO.FileNotFoundException: File or assembly
name zmp0husw.dll, or one of its dependencies, was not found.

Internally, there are Security Demands, to see if you have the "right" from a Code Access Security point of view, but noone actually CHECKS to see if you have the ACL rights.

So, here's a brute force way to find out, once per AppDomain, to check if you have access. I reflectored into XmlSerializer to find out what they were doing to find our what path to write to. They were using GetTempPath from kernel32.dll, so we could PInvoke as well. (Update: However, Kevin Dente points out that Path.GetTempPath() will do the PInvoke for you. I was mirroring XmlSerializer's code, but as long as we do the same thing in essense, we're OK. Edits below, line numbers changed. Thanks Kevin!)

    1 public sealed class SomeUtilityThingieClass
2 {
3 const string ErrorMessage = "We neede write access to: {0}";
6
7 private static bool tempFileAccess = false;
8 private static object tempFileAccessLock = new object();
9
10 public static bool EnsureTempFileAccess()
11 {
12 if (tempFileAccess == true)
13 {
14 return true;
15 }
16
17 if(tempFileAccess == false)
18 {
19 lock(tempFileAccessLock)
20 {
21 if(tempFileAccess == false)
22 {
29 string tempFile = Path.Combine(Path.GetTempPath(),"WriteTest.txt");
30 try
31 {
32 using(StreamWriter file = File.CreateText(tempFile))
33 {
34 file.Write("This is a test to see if we can write
to this TEMP folder and consequently make XmlSerializer
assemblies without trouble.");
35 }
36 }
37 catch(System.IO.IOException ex)
38 {
39 throw new System.IO.IOException(
string.Format(ErrorMessage,tempFullPath),ex);
40 }
41 catch(UnauthorizedAccessException ex)
42 {
43 throw new UnauthorizedAccessException(
string.Format(ErrorMessage,tempFullPath),ex);
44 }
45
46 if (File.Exists(tempFile))
47 {
48 File.Delete(tempFile);
49 }
50 tempFileAccess = true;
51 }
52 }
53 }
54 return tempFileAccess;
55 }
56 }

Once the write has worked once, we catch the success in a static and return that. If multiple threads get in here together, only one will make it past the lock. The others will wait. After we learn of success or failure from the first thread, the threads that were waiting for the lock will check the (now change) tempFileAccess boolean, and find it changed. The file write will happen only once per AppDomain. Rather than calling "throw;" or not catching the exceptions at all, I add a little extra polite message and wrap and throw. They won't know the line number that went wrong, but they WILL get a nice message.

The most interesting stuff to the beginner is the classic check/lock/doublecheck threadsafety. Note also that we have an explicit object tempFileAccessLock that exists ONLY for the purposes of locking.

 

 



In ASP.NET you can yank a value out of the QueryString like this, where QueryString is of type NameValueCollection but is internally an HttpValueCollection that includes some extra helper methods.

string foo = Request.QueryString["foo"];

But you can also go like this:

string foo = Request["foo"];

And folks know (passed through myth and legend) that the line above will search through the QueryString, Form, Cookies, and ServerVariables collections. However, it's important (for performance) to know what order the collections are searched. Here's the code from Reflector:  

    1 public string get_Item(string key)
    2 {
    3     string text1 = this.QueryString[key];
    4     if (text1 != null)
    5     {
    6         return text1;
    7     }
    8     text1 = this.Form[key];
    9     if (text1 != null)
   10     {
   11         return text1;
   12     }
   13     HttpCookie cookie1 = this.Cookies[key];
   14     if (cookie1 != null)
   15     {
   16         return cookie1.Value;
   17     }
   18     text1 = this.ServerVariables[key];
   19     if (text1 != null)
   20     {
   21         return text1;
   22     }
   23     return null;
   24 }

So you can see what order things are searched in. However, personally, I don't like this default Item indexer. I prefer to be more explicit. I'd hate to accidentally retrieve a Cookie because a QueryString variable was missing. It's always better to be explicit and ask for what you want.

Interestingly, there is ANOTHER collection of QueryString, Form, Cookies, and ServerVariables, but rather than a "pseudo-collection" as we see above, this is an actual combined collection.

  432 public NameValueCollection Params
433 {
434 get
435 {
436 InternalSecurityPermissions.AspNetHostingPermissionLevelLow.Demand();
437 if (this._params == null)
438 {
439 this._params = new HttpValueCollection();
440 this.FillInParamsCollection();
441 this._params.MakeReadOnly();
442 }
443 return this._params;
444 }
445 }
446
447 private void FillInParamsCollection()
448 {
449 this._params.Add(this.QueryString);
450 this._params.Add(this.Form);
451 this._params.Add(this.Cookies);
452 this._params.Add(this.ServerVariables);
453 }
454

The internal collection "_params" inside is a special derived NameValueCollection of type HttpValueCollection, and is exposed as NameValueCollection.

Important Note: The constructor for HttpRequest will parse the actual string QueryString and UrlDecode the values for you. Be careful not to DOUBLE DECODE. Know what's encoded, when, and who does the decoding.  Likely it's not you that needs to do anything. If you double decode you can get into some weird situations. Ben Suter reminded me that if you pass in /somepage.aspx?someParam=A%2bB you expect to get "A+B" as that param is the equivalent of HttpUtility.UrlEncode("A+B"). But, if you make a mistake and do HttpUtility.UrlDecode(Request.Params("someParam")), you'll get "A B" as the + was double-decoded as a space.

Here's the trick though. If you have BOTH a QueryString parameter "Foo=Bar1" AND a Forms item called "Foo=Bar2" if you say string foo = Request.Params["Foo"]; you'll get a string back "Bar1,Bar2"! It's a collection, not a HashTable. So, never make assumptions when you use HttpRequest.Params, or you will get in trouble. If there's a chance you could get multiple values back, you need to consider using an explicit collection or be smart about your string.Split() code.



Hack? Brilliance? Sloppy? Simple? Here's when I know something is good:

  • It's easily explained.
  • It's not ambiguous.
  • There aren't many files.
  • Self-containment.
  • It's simple

Seeing a pattern? Loren has come up with an interesting and elegant hack called XmlPreprocess to deal with the deployment of config files to multiple environments without the need for multiple copies. It would be EASILY integrated into a Continuous Integration environment like ours.

The idea is bone simple:

<configuration>
<system.web>
<!-- ifdef ${production} -->
<!-- <compilation defaultLanguage="c#" debug="false"/> -->
<!-- else -->
<compilation defaultLanguage="c#" debug="true"/>
<!-- endif -->
</system.web>
</configuration>

gets loaded into "XmlPreprocess.exe /d production" and you get:

<configuration>
<system.web>
<compilation defaultLanguage="c#" debug="false"/>
</system.web>
</configuration>

Cool, eh? Now, some may thing preprocessor/if statements "tunneled" in XML Comments are gross, but to the nay-sayers I say show me one man's extensibility hack and I'll show you xs:annotation. Would you feel better if the "language" was hidden in other XML elements? Or another namespace? Sure, it's possible, but the goal isn't a new language (if statement = new language) it's #ifdef for XML for a specific purpose.

Reasons it's cool:

  • The files are well-formed and legit XML before and after the process. No processing is needed to use them as found in Source Control.
  • The files are self-describing. Just look at them, it's clear what's up.
  • As Loren says, Get and Go.

Charlie says in comments on Jon Galloway's blog that it's a hack that this is "definitely not the way most natural way of dealing with XML" and he'd prefer XSLT. He does point out that XSLT isn't for newbies. However, for this small task I say, Wow, no way.

Using XSLT to change a config file in such a small way seems like hammering a screw. It's possible, but overkill that WILL get messy. Look at the previous requirements and apply XSLT. XSLT is great for transforming Infosets into other totally different Infosets, and while I'm deep into XSLT, I'm not a fan of using XSLT for small "pruning" changes.

I think XmlPreprocess is clever thing that's good for exactly what it's good for. Check it out!



I'm sure this has been done before, but it was faster to write it than to google for it. There's some IP Addresses that have been bothering me and I don't have access to a firewall or IIS at my ISP, so...

I can upload a text file called blockedips.txt to my site and the changes happen immediately.

    9 namespace YourModuleNameHere
   10 {
   11     public class IPBlackList : IHttpModule
   12     {
   13         private EventHandler onBeginRequest;
   14 
   15         public IPBlackList()
   16         {
   17             onBeginRequest = new EventHandler(this.HandleBeginRequest);
   18         }
   19 
   20         void IHttpModule.Dispose()
   21         {
   22         }
   23 
   24         void IHttpModule.Init(HttpApplication context)
   25         {
   26             context.BeginRequest += onBeginRequest;
   27         }
   28 
   29         const string BLOCKEDIPSKEY = "blockedips";
   30         const string BLOCKEDIPSFILE = "SiteConfig/blockedips.config";
   31 
   32         public static StringDictionary GetBlockedIPs(HttpContext context)
   33         {
   34             StringDictionary ips = (StringDictionary)context.Cache[BLOCKEDIPSKEY ];
   35             if (ips == null)
   36             {
   37                 ips = GetBlockedIPs(GetBlockedIPsFilePathFromCurrentContext(context));
   38                 context.Cache.Insert(BLOCKEDIPSKEY , ips, new CacheDependency(GetBlockedIPsFilePathFromCurrentContext(context)));
   39             }
   40             return ips;
   41         }
   42 
   43         private static string BlockedIPFileName = null;
   44         private static object blockedIPFileNameObject = new object();
   45         public static string GetBlockedIPsFilePathFromCurrentContext(HttpContext context)
   46         {
   47             if (BlockedIPFileName != null)
   48                 return BlockedIPFileName;
   49             lock(blockedIPFileNameObject)
   50             {
   51                 if (BlockedIPFileName == null)
   52                 {
   53                     BlockedIPFileName = context.Server.MapPath(BLOCKEDIPSFILE);
   54                 }
   55             }
   56             return BlockedIPFileName;
   57         }
   58 
   59         public static StringDictionary GetBlockedIPs(string configPath)
   60         {
   61             StringDictionary retval = new StringDictionary();
   62             using (StreamReader sr = new StreamReader(configPath))
   63             {
   64                 String line;
   65                 while ((line = sr.ReadLine()) != null)
   66                 {
   67                     line = line.Trim();
   68                     if (line.Length != 0)
   69                     {
   70                         retval.Add(line, null);
   71                     }
   72                 }
   73             }
   74             return retval;
   75         }
   76 
   77         private void HandleBeginRequest( object sender, EventArgs evargs )
   78         {
   79             HttpApplication app = sender as HttpApplication;
   80 
   81             if ( app != null )
   82             {
   83                 string IPAddr = app.Context.Request.ServerVariables["REMOTE_ADDR"];
   84                 if (IPAddr == null || IPAddr.Length == 0)
   85                 {
   86                     return;
   87                 }
   88 
   89                 StringDictionary badIPs = GetBlockedIPs(app.Context);
   90                 if (badIPs != null && badIPs.ContainsKey(IPAddr))
   91                 {
   92                     app.Context.Response.StatusCode = 404;
   93                     app.Context.Response.SuppressContent = true;
   94                     app.Context.Response.End();
   95                     return;
   96                 }
   97             }
   98         }
   99     }
  100 }

And in your web.config:

   42 <system.web>
   43    <httpModules>
   44         <add type="YourModuleNameHere.IPBlackList, YourAssemblyNameHere"
   45             name="IPBlackList" />
   46    </httpModules>
   47 </system.web>

No warrenty, express or implied. If it sucks or has bugs/security holes, let me know as it's 9 minutes work.



I googled around for info on the ServiceProcessInstaller, hoping to change the HelpText. It's a getter, so you have derive and override, rather than setting it. No biggie, but I was surprised. I'm used to googling for .NET documentation and finding MASSIVE amounts of stuff...lot's of articles, etc. Apparently the ServiceProcessInstaller just isn't sexy enough to write about.

This overloaded getter will show help text when someone calls "InstallUtil.exe myserviceassembly.exe /?" on my service exe.

Anyway, I did this. It's not rocket science, but I'd have perferred a setter. Maybe I'm old fashioned.

   88 internal class MySpecialServiceProcessInstaller : System.ServiceProcess.ServiceProcessInstaller
   89 {
   90     public override string HelpText
   91     {
   92         get
   93         {
   94             return @"Enter the Username and Password for the Whatever account in the form \\domain\username. For machines not in a domain, enter hostname\username.";
   95         }
   96     }
   97 }

Then I change the InitializeComponent section - you know, the section you're never supposed to change?

   64 private void InitializeComponent()
   65 {
   66     this.serviceProcessInstaller1 = new MySpecialServiceProcessInstaller();
   67     this.serviceInstaller1 = new System.ServiceProcess.ServiceInstaller();

Maybe when I google for it again, I'll find my own site. If I can touch the life of just one child who's also using System.ServiceProcess.ServiceProcessInstaller...



PerfmonsearchWell, I'm deep into the MSN Desktop Search. It's apparently more appropriate to refer to it as the "MSN Toolbar Suite."

Sriram Krishnan has even more technical forensic details on how Google Desktop Search works, but here's what I've gleaned about MSN Desktop Search.

  • The footprint is about 5 megs installed - this changes after you index,
  • The search is per user and everything is in "C:\documents and settings\[username]\local settings\application data\msn toolbar suite\"
    • There's lots of temp files with "Microsoft Search Gatherer Transaction Log. Format Version 4.9" inside them.
  • There's an interesting video with the team up on Channel 9.
  • They don't index your whole harddrive by default - they start with the "My*" files and Outlook* Applications.
  • They are using Microsoft Indexing IFilters - This makes me wonder if I should go download a bunch of new IFilters to get better searching.
  • They index MP3s and Music...
  • When you search, the UI lives in Explorer or Internet Explorer. They've got their own namespace.
  • There are a CRAPLOAD of Perfmon Counters added under "RS Search*" in Perfmon.

MSN Desktop Search vs. Google Desktop Search

MSN: Toolbar pointing you to msn.com
Google: Toolbar pointing you to google.com

MSN: Interface is very shiny and Windowsy and lives in Explorer
Google: Interface is very plain and HTMLy and lives in any browser

MSN: Indexes any IFilter-able thing ala Index Server (this has potential)
Google: Indexes Outlook and a number of text files. (has a hacked .NET plugin API)

MSN: Type-ahead autocomplete ala X1.com
Google: Nope, but I suspect it's coming very soon...

Winner (for now): The user, as now two large companies realize it's hard to find files on your hard drive!



Certainly comparisions will be made to Google Desktop Search, which I'm already a fan of, but here's YATB (yet another toolbar) for IE. Rather than building it into the operating system or XP SP2, it's a viral MSN toolbar. Here it is, Microsoft's Desktop Search.

I've installed it, and I'm currently digging into how it works. Doesn't support Firefox, though. ;)

It has an X1-style type-ahead feature and rather than browser integration it's an Explorer Toolbar (I needed more things in my tray! Woohoo!)

Currently Indexing...if it indexes .CS files, I'm all in.

UPDATE: It DOES index source and XML! Yum.



Changes to dasBlog - Watch for Weirdness

Posted 2004-12-12 02:13 AM in ASP.NET | DasBlog.

I've just made some changes to my personal branch of dasBlog including:

  • Referral Blacklist - This should cleanup my referrers and make my life easier.
  • Fully integrated CAPTCHA - You won't lose your post if you goof the CAPTCHA. MaxS will feel better.
  • Performance re-work of RSS/Atom - I've added internal caching to the syndication service. You the end-user won't see a change, but my ISP and I will see big change in CPU. DasBlog continues to honor the HTTP Headers ETag, If-Modified-Since and returns 304s as appropriate.
  • A number of small caching and perf improvements - Meant to please my ISP who has noticed my CPU and bandwidth characteristics were degrading as the amount of content I've got archived and hits have grown.

I say again, ORCSWeb.com kicks bootay. I highly recommend them, their service is without equal.

Let me know if you see any weird stuff due to these changes. Omar will likely put these changes into dasBlog 1.7 along with his many improvements. Clemens may also test them out. We shall see.



We need a good ASP.NET programmer for a three month contract, with an option to hire at the end.

If you feel comfortable answering some of these questions, that's a ++ for you. You likely live in the Portland/Metro area.

More importantly, I want:

  • Passion and enthusiasm
    • You can't sleep until the solution "feels" right
  • Ruthless competence
    • "I read about it once" isn't enough. Did you do it? Your old job didn't require you to do it? Did you go home and do it anyway?
  • Good troubleshooting skills
    • How do Assemblies get loaded, shadow copied, etc. What is VS.NET doing that?
  • Familiar with Objects as much as DataSets
    • Being good at ASP.NET isn't just hooking DataSets up to DataGrids. We do more than that, and we want you to do more as well.
  • Self Starter
    • If you're chillin', you must be done. :)
  • I don't care if you're MC*.*
    • I do care if you've got a resume that shows a proven track record of successful ASP.NET work.

Email me your resume - scott/atnospam/corillian.com.



I loves me the right-click. I right-click in my sleep. Any utility that lets me right-click to do something that is a hassle, is my new best friend. Oh, and not crashing Explorer.exe is a requirement.

Here's my three favorites today (there are others for later):

  • RunAsLimitedUser - Jonas Blunck does it again. The only guy who writes more stuff is Jeff Key. Anyway, RunAsLimitedUser will launch an application using an account that has a limited set of permissions on my box. It's different than the shift-right-click RunAs...which lets you run as any user. If you're doing development in a restricted or very restricted environment, it's the tool. Slick.
  • ClipPath - Adds a context-menu to put the current path from Explorer in your clipboard. Even better, you can chosse from \'s or /'s, as well as Outlook friendly <file:///http://www.hanselman.com/blog/content/binary/whatever>. Why wasn't this in Windows already?
  • Clean Sources - "This Application does one thing. It adds an explorer shell menu to folders that when selected will recursively delete the contents of the bin, obj and setup folders. If you have a .NET project that you wish to share with some one, this is useful to remove the unnecessary stuff from the folder before you zip it up and send it off."



If I had a nickel for every time someone sent an email followed by an attempt of "Fred would (desperately) like to recall this message."

Sorry Fred. I already saw your "I'd like one" or "Me too" or "Joe is a jerk" email that you Big-R'ed rather than Little-R'ed.

  • Stop recalling your messages. It just looks bad and chances are it's already been delivered to your boss's BlackBerry.
  • Set a "Delay" Rule in Outlook that holds your email in the outbox for 1 minute. This saves me weekly.
  • "Little-R me" means Reply. "Big-R" means Reply to All.  Don't Big-R if you've been BCC'ed. It kind of shatters the mystique of it all.
  • Check twice if you are thinking of attaching something. Then check again, especially if it's your resume.
  • Use punctuation. And Capitalization. im glad youre h4pi to be lazi but pls don't inflict your inability to use the "Shift" key on me. And don't blame your carpal tunnel. Just Shift.
  • The P4-3.2Ghz1G can also check your spelling. Look into it.
  • Subjects in emails are nice. "Dude...check this out" isn't a subject.
  • If it's a throw-away comment, chat me instead. Ignoring you is easier than deleting.
  • Think about a utility like SSW LookOut. Not the LookOut search engine, this is a tool that catches your swear words, complains if you use the word "attach" then attach nothing, and generally enforces Netiquette.

Thank you.



BignewlaptopGasp. I returned from lunch only to find my computer in the middle of a blue screen.

Let me tell you folks, this is the computerguy-equivalent of finding your wife in bed with your brother. All the same emotions run through your mind (I would imagine):

  • Oh, you dirty bitch.
  • How could you do this to me?
  • And now, during the holidays!?

I rebooted, and was welcomed by and immediately disconcerting clank and the ominous evil that followed:

Missing Operating System

Seriously people, drink that in. Really, wallow in the pitiful blackness that is the failed BIOS POST, accompanied by the omnipresent but almost subconcious whispering "f*ck you...f*ck you..." clack of what remained of your drive heads.

Here's the catch. I backed it all up (for the first time in months) THE DAY BEFORE. I backed it all up (the data) to my Iomega REV Drive.

I'm digging this drive more and more.  Some poo-poo'ed the purchase, saying it was too expensive, and instead opting for other mini-harddrives (USB or Firewire) to copy their data to. I prefer a removable solution, one that I can put in a safety deposit box or fire safe.

Anyway, I'm about 40% done installing my programs as you can see by my Start Menu.  I'll know I'm done when the Start Menu completely fills my 1400x1050 screen.

Back up people. Do it today. Do it now.



I'm not sure why I wanted to know this, but I was looking for where the User Tasks (not the auto-generated Comment Token Tasks) were stored from Visual Studio.NET.

They are stored in the Solution file's (.SLNs) parallel Solution User Options (.SUO) file. This file appears to be OLE Structured Storage, even in Whidbey.

Now I know.

P.S. We, as a policy, don't check either *.SUO files or *.CSPROJ.USER files into source control (we've added them to .CSVIgnore) as they are totally user-specific options and would not only clutter and confused, but potentially share settings and tasks we don't want to share.



I'm surprised I'm just now noticing this. Jon Galloway hooked up the apparently unused System.Web.Handler.BatchHandler to an httpHandler and was able to precompile all his .ASPX pages. This could be useful during deployment to catch any goofs in ASPX code. Certainly not something you want on in production lest you be DoS'ed with compilation, but a helpful thing regardless.

To set this up at the machine level, add the following line to the <httpHandlers> section of %windir%\Microsoft.NET\Framework\v1.1.4322\CONFIG\machine.config:

<add verb="*" path="precompile.axd" type="System.Web.Handlers.BatchHandler"/>

To set this up at the application level, add the you'll need to create an httpHandlers section like so: [JonGalloway]

<configuration>
  <system.web>
    <httpHandlers>
      <add verb="*" path="precompile.axd" type="System.Web.Handlers.BatchHandler"/>
    </httpHandlers>
  </system.web>
</configuration>

Now playing: Stevie Wonder - As



Target: Referral Spam in dasBlog

Posted 2004-12-03 11:37 PM in ASP.NET | DasBlog | XML.

I've pretty much solved the comment-Spam problem (only one person has voiced their distaste so far) but a recently perusal of my logs and older posts indicated a ridiculous amount of referral spam. 

This is when someone hits a post on your site and has changed/hacked the HTTP Referrer Header to indicate where they came from. If your blog adds this referrer to the page, as most to, you've just linked to Hot Gay Sex (not that there's anything wrong with Hot Sex between consenting adults : ) ) or whatever by their actions.

The story goes when Google comes around, they see that you've linked to them, and they get Google Juice via the Page Rank System.

Not only is this potentially offensive to my readers, it also obscures the posts and comments when they are filled with referrals.

Potential Solutions:

  • Stop printing out referrals on my pages.
    • Personally, I like to see them, and I think they provide value to the reader so they can see other places with information of interest. It also promotes cross-linking between my peer blogs.
  • Modify dasBlog to NOT add icky referrals.
    • This would be idea. However, it will likely be in version 1.7 in some way, either via James Snape's whitelist solution (I think a whitelist removes the point of referrals, and I'll greatly prefer a keyword-based black list) or some other technique.
    • I've avoided running a "private build" of dasBlog so far (as evidenced by my care in creating the CAPTCHA solution without recompiling) and I'd to continue as such
  • Clean the .xml files occasionally with a process
    • This is quick, easy, can be automated, and will work in the short term for me as I await dasBlog 1.7.

So, here was an opportunity to use the only dev stuff I have on my home machine, Visual Studio C# 2005 Express

Here's what I did. Use at your own risk, back up your /content directory, and know that this will only have to run on your "*.dayextra.xml" files from dasBlog. No error handling, no warrenty, but it worked for me. Enjoy.

Usage: TrackingFilter "c:\yourdasblogcontentdirectory"

File Attachment: TrackingFilter.zip (9 KB) (for VS.NET 2005, I don't know if it works in 2003)

WARNING: The words I put in the .config file are ; delimited and are unquestionably offensive. Not only do they include most of George Carlin's words but they also include "bloglines" and "artima" because they don't provide a value in my referral list.



I spend a lot of time with the XmlSerializer (I personally dig it immensely, and I think too many people complain about it, but anyway) and while I put up an article on how to debug directly into the generated assemblies, I noticed that Mathew Nolton has a GUI Front-End to Chris's XmlSerializerPreCompiler.

The tool will check to see if a type can be serialized by the XmlSerializer and shows any compiler errors that happen behind the scenes. +1 for Useful, thanks Chris and Mathew.



A fellow emailed me wanting to screen scrape, er, ah, harvest a page that only displays the data he wants with a postback.

Remember what an HTTP GET looks like under the covers:

GET /whatever/page.aspx?param1=value&param2=value

Note that the GET includes no HTTP Body. That's important. With a POST the 'DATA' moves from the QueryString into the HTTP Body, but you can still have stuff in the QueryString.

POST /whatever/page.aspx?optionalotherparam1=value
Content-Type: application/x-www-form-urlencoded
Content-Length: 25
param1=value&param2=value

Note the Content-Type header and the Content-Length, those are important.

A POST is just the verb for when you have an HTTP document. A GET implies you got nothing.

So, in C#, here's a GET:

public static string HttpGet(string URI)
{
   System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
   req.Proxy = new System.Net.WebProxy(ProxyString, true); //true means no proxy
   System.Net.WebResponse resp = req.GetResponse();
   System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
   return sr.ReadToEnd().Trim();
}

Here's a POST:

public static string HttpPost(string URI, string Parameters)
{
   System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
   req.Proxy = new System.Net.WebProxy(ProxyString, true);
   //Add these, as we're doing a POST
   req.ContentType = "application/x-www-form-urlencoded";
   req.Method = "POST";
   //We need to count how many bytes we're sending. Post'ed Faked Forms should be name=value&
   byte [] bytes = System.Text.Encoding.ASCII.GetBytes(Parameters);
   req.ContentLength = bytes.Length;
   System.IO.Stream os = req.GetRequestStream ();
   os.Write (bytes, 0, bytes.Length); //Push it out there
   os.Close ();
   System.Net.WebResponse resp = req.GetResponse();
   if (resp== null) return null;
   System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
   return sr.ReadToEnd().Trim();
}

I could and should have put in more 'using' statements, but you get the gist. And, there are other ways to have done this with the BCL, but this is one.

Now, how would you fake an HTTP PostBack? Use a tool like ieHttpHeaders to watch what a real postback looks like, and well, fake it. :) Just hope they don't require unique/encrypted ViewState (via ViewStateUserKey or EnableViewStateMac) for that page, or you're out of luck.



I've not been one to work the newsgroups, answering questions. I probably should. I'm more of a one on one person, and I tend to go the extra mile when folks (largely strangers) ask me technical questions. I've had email threads 10-deep with total strangers on technical questions, and only at the end do I say, "Um, do I know you?"

I haven't done what Scott Mitchell wisely did and setup a "Getting Help" policy, but I'm quickly getting there. I'll happily answer your question for $75 too, satisfaction guaranteed, and I'll blog the answer. I've done this hundreds of times for free. :)

Anyway, the point of this post was this: People, for crying out loud, debug a few things before you ask for help. If you don't know how to debug, learn or ask someone to teach you.

So, I present:

The Hanselman List of .NET Debugging Dos and Don'ts

Don't - Say "Hey, I got a NullReferenceException," what's the problem?
Do - Provide a Stack Trace/Dump with the line number it likely happened on.

Don't - Get deep into your complicated program, find a bug and insist it's BillGs fault.
Do - Reproduce the bug in some simple test program and tell the world. Remember, 9/10 times it's you.

Don't - Decide there's a problem if you don't know the preferred behavior.
Do - Always Assert your assumptions. If null can happen, check for it. BUT, if null must never happen it's time for a Debug.Assert

Don't - Move code around blindly, somehow fix your bug, ignore it and keep coding. Programming by Coincidence!
Do - Understand your program fully. Remember what Andy and Dave say about lucky folks who step into minefields and don't die. Just because you didn't die, doesn't mean there aren't mines!

Don't - Reformat or "pave" something because you don't know what's wrong. If you get a spot on your carpet, fix the spot. Don't lay new carpet.
Do - Know enough about your environment to know what your program's dependencies are. If your registry settings can get boogered, Debug.Assert that you are getting good values from the registry.

Don't - Get overly frustrated with Assembly loading/versioning/policy. At least the Assembly Loader follows clear, set, rules.
Do - Make a folder called C:\FusionLogs, then go to the registry in HKLM:\Software\Microsoft\Fusion and make a DWORD value LogFailures=1 and string value LogPath=C:\FusionLogs. Every AppDomain that has a binding failure or weird redirect will get logged. Know: What assembly you want, what they looked for, what you got. Know where Assemblies are searched for.

Don't - Avoid debugging. Debugging in .NET is easier than ever before. Remote debugging and AttachToProcess are gifts. Don't stop at a point in the call stack if you can keep going by finding PDBs.
Do - Keep your Source and PDBs in the same location. We keep ZIPs of every build's PDBs. Just today we dug up 9-month old PDBs and source (from CVS) to debug into some confusion. Not saving those PDBs would have screwed us. Create a Symbol Server.

Don't - Limit yourself to the QuickWatch. Learn what VS.NET has to offer.
Do - Use the Immediate Window to test theories. Remember that you can perform Casts in the Watch Window. Remember that you can drag and drop variables into the Watch Window. Remember you have 4 Watch Windows, Autos, Locals, not to mention. Learn how to use Conditional Breakpoints!

Don't - not debug something just because you can't figure out how to launch the process from the VS.NET Project Properties.
Do - Debug|Processes|Attach to attach to processes that have your DLL loaded. Use ProcExp from SysInternals as a better Task Manager to see .NET processes, as well as a system-wide DLL search. Who's got you loaded?



FireBlogging - The View From My House

Posted 2004-12-02 01:10 AM in Musings.

It's 1:09am on Thursday, December 2nd 2004, and here's the view from my bedroom window. The house next door is burning and we share a wooden fence. Pretty exciting stuff! Fortunately, I'm not too worried, I'm in a family of fire-fighters.

CIMG2539 (Small) CIMG2546 (Small)

CIMG2547 (Small) CIMG2540 (Small)

P.S. For those of you not in the U.S., most, if not all, residential housing (especially in the Suburban Western U.S.) is made of wood and quite flammable. My wife's still not used to this fact, and her family isn't impressed that our family fights fires. :)



Here's a great little free util that Omar has found.  I used to move the plugin's manually to speed things up, but PDF Speedup makes me NOT DREAD opening a PDF anymore. BTW, does Acrobat 6 suck LOTS more than Acrobat 5? I HATE the new Find Dialog.

I have previously written about how darn slow Adobe Acrobat 6 is when launching. I don't understand why Acrobat is so darn annoying. Here are some things I don't care for:

* Don't create a “My eBooks“ folder in My Documents when I have nothing to put there.
* Don't load 500 plugins when none of them are necessary to view a PDF
* Don't place shortcuts for some lame Internet Printing thingy in my Start Menu (I loathe Start Menu Advertising)
* Do install a PDF IFilter so that indexing products like Lookout can index PDFs w/o installing it seperatley
* Don't load PDFs in IE because it is god awful slow
* Don't make copy and paste so freaking hard
* Don't ask me to install other Adobe software when I boot Acrobat
* Don't create an updater (6.0.2) that creates an additional entry in my add/remove programs

If you want to fix most of these things, PDF SpeedUp is a free application that should come bundled with Acrobat. It's a must have piece of software to make Acrobat behave (as much as you can anyway).
[shahine.com/omar/]