Scott Hanselman's Computer Zen

Page 1 of 4 in the IIS category Next Page

These are the little bugs that lead to madness

Posted 2009-06-23 01:27 PM in ASP.NET | IIS | Musings.

I received an interesting email today where a fellow was trying to make sure that all browsers could successfully download his company's MSI installer. He had found a blog post that I wrote SIX YEARS AGO on the Content-Disposition header and some trouble I'd had with Check Images. Just in case you're not clear, 6 years is like a century years on the internet.

Here's a little snippet from my incredibly old blog post:

HTTP Headers are name values pairs, so they are easily added with the Response object in ASP or ASP.NET You use it like this (the HTTP Headers):

HTTP/1.1 200 OK
<snip>
Content-Disposition: filename=checkimage.jpg
Content-Length: 76127
Content-Type: image/JPEG

Or, if you want to immediately prompt the user with a File Download Box:

HTTP/1.1 200 OK
<snip>
Content-Disposition: attachment; filename=checkimage.jpg
Content-Length: 76127
Content-Type: image/JPEG

However, Internet Explorer has never really got it right.

Here's a list of gotchas, starting with my own:

On IE 6.0, things mostly work, but if you ALSO setup Cache-Control: no-cache, your suggested filename (and type!) will be IGNORED. A bummer if you have to choose between security and convienence. Of course, security wins.

On IE 4, the attachment option is flaky, see Q182315

On IE 5.5, the attachment option is REALLY flaky, see Q267991 and Q279667 and Q281119

On IE 5.0, the filename suggested can mangle your filenames, see Q262042

On nearly all versions of IE, including 6.0, sometimes the browser will use the filename in the address bar instead of the Content-Disposition Header, and with IE5.5SP2 you're expected to change the UseCDFileName registry key, see Q303750. This was fixed with IE6.0SP1.

IE's not the only browser with past trouble around this header, but it's been the worst historically. Last year, IE8 made a good move forward when it proposed (during the beta cycle) an "authoritative=true" addition to the Content-Type HTTP header. This would be a way for your server to basically insist that the Content-Type it offered was the correct one. Seems reasonable, like it should have always been that way, eh?

Here's an example on how we'd (under this OLD proposal) force an HTML page to be delivered and rendered as plaintext. Sam Ruby thought it was a good idea as well as sniffing, while inside the HTML5 spec, is generally considered a bad idea.

HTTP/1.1 200 OK
Content-Length: 108
Date: Thu, 26 Jun 2008 22:06:28 GMT
Content-Type: text/plain; authoritative=true;

<html>
<body bgcolor="#AA0000">
This page renders as HTML source code (text) in IE8.
</body>
</html>

Unfortunately this blog post was never updated. EricL (author of Fiddler and very nice person) wrote it, and he'll know I'm not picking on him personally, as this is a huge problem on all blogs, mine included. It's really hard to update old posts when they are obsolete. It's a manual process and all we as bloggers can do is our best to update our old posts with pointers to new information.

Two months later, this post came out and the final design that was agreed on with community feedback looked like this:

Over the past two months, we’ve received significant community feedback that using a new attribute on the Content-Type header would create a deployment headache for server operators. To that end, we have converted this option into a full-fledged HTTP response header. Sending the new X-Content-Type-Options response header with the value nosniff will prevent Internet Explorer from MIME-sniffing a response away from the declared content-type.

For example, given the following HTTP-response:

HTTP/1.1 200 OK 
Content-Length: 108 
Date: Thu, 26 Jun 2008 22:06:28 GMT 
Content-Type: text/plain; 
X-Content-Type-Options: nosniff

<html> 
<body bgcolor="#AA0000"> 
This page renders as HTML source code (text) in IE8. 
</body> 
</html>

I'd like this post to serve as a reminder to all of us who are blogging technical content to update our posts if and when appropriate, and certainly when a reader points out errata. As the gent who emailed me so wisely put it:

"These are the little bugs that lead to madness."

Thoughts?

Microsoft Web Platform, Web Application Gallery, Web Platform Installer (and DasBlog)

It's so fun to help "announce" things, especially things that are a long time coming and that a lot of people worked on to make things "just work." (You can follow them on Twitter @mswebplatform.)

This is one of those fantastic things. Today, you can install free applications direct to IIS using the Web Platform Installer 2.0 and the Web Application Gallery. The WebPI will also check for application dependencies, and if you like, automatically download and install them. It'll install SQL Express, PHP, whatever you need to get your apps running.

For example, you can now install DasBlog just by clicking this button:

There's a bunch of cool apps available in the gallery immediately, both ASP.NET and PHP (remember that IIS kicks ass at hosting PHP):

This gallery sits on top of some technologies that have been slowly but surely sneaking out of the most-awesome IIS team lately.

How's it work?

Here's some technical details. You can install free apps a few ways, and they are all easy.

You can visit the Web App Gallery, find an app and click install. If you've got the Web Platform Installer it'll launch out of the browser. (Don't worry, it's not a browser plugin, it's more like iTunes. It'll "leap out of the browser" from a link.)
You can open IIS manager on your machine after installing WebPI, and click "Install Application from Gallery." This feature is awesome enough to require a screenshot to drink in:
Or, just run the Web Platform Installer and browse around the catalog. The list of apps comes down as an Atom Feed.

Hey, isn't that dasBlog in the list there? Why, yes! ;)

Packaging Your App - Technical Details - DasBlog

You can upload YOUR free app to the gallery. Start by downloading the Application Developer's Kit. We packaged up DasBlog 2.3 like this. You install DasBlog

The package is a zip file. Inside at the root are two files, manifest.xml and parameters.xml. You start by "exporting" your app already working in IIS. Then, if your app has specific needs you can edit the files manually.

For example, here's our manifest.xml, which is an MSDeploy file:

<MSDeploy.iisApp>

    <!-- Copies content from path into the destination specified in parameters.xml and marks as an app in IIS 5.1, 6.0 and 7.x -->
    <iisapp path="dasblogce" />

    <!-- Set NTFS ACLs on the destination specified in parameters.xml -->
    <setAcl path="dasblogce/content" setAclAccess="Read,Write,Delete" />
    <setAcl path="dasblogce/logs" setAclAccess="Read,Write,Delete" />
    <setAcl path="dasblogce/siteconfig" setAclAccess="Read,Write,Delete" />

</MSDeploy.iisApp>

Remember that DasBlog doesn't use a database, but rather stores all data in XML files in the content folder. Our MSDeploy file sets ACLs on a few directories to allow "Read,Write,Delete." This file encapsulates a potentially tricky setup step for DasBlog.

The parameters.xml is a little more sophisticated and drives a dynamic UI form inside of the Web Platform Installer 2.0.

For example, the blog owner's email is a parameter, and it's entered in two files. Since DasBlog uses XML files, we can use XPath to describe where to poke the values in.

<!-- Prompts for admin email -->
<parameter name="Email Address" description="Enter the blog owner's email address." defaultValue="dasblog@example.com">
    <parameterEntry type="XmlFile" scope="dasblogce\\siteconfig\\site.config" match="//MSDeploy:Contact/text()" />
    <parameterEntry type="XmlFile" scope="dasblogce\\siteconfig\\siteSecurity.config" match="//EmailAddress/text()" />
</parameter>

Here's what the generated UI looks like:

You can read about this format in the Application Developer's Kit. You can parameterize your SQL files as well, it's a very flexible format.

Then we zipped up the deployment and uploaded it to CodePlex. The CodePlex site is smart enough to notice when the Platform Installer is requesting a release (it sniffs the User-Agent) and serves up the file rather than the web site, so I can use CodePlex to host the zip. (It would have sucked to have to host somewhere else.)

MSDeploy -> WebDeploy -> Web Platform Installer 2.0 -> Web App Gallery

Just to take a second, and make sure this is clear.

There's the command-line MSDeploy engine, there's WebDeploy that integrates with IIS and has a nice UI, and there's the Web Application Gallery where you can get a bunch of free applications. When you're in the App Gallery, the Web Platform Installer will do the installation.

Here's how I see it. Of course, you just have to click a button, but I always like to see what's underneath.

What's cool about this is that even if you don't want to put your app up on the Web Application Gallery, you can still use MSDeploy/WebDeploy to deploy your apps. They are REALLY powerful tools that can help you deploy, migrate between IIS6 and IIS7, and synchronize content and apps between machines in a web farm. If you deal with IIS apps all day, host them, develop them, etc, take some time and spend it over at http://www.iis.net.

Now, if you want to have your app included in the gallery, here's the principles your app should follow:

Web Application Gallery Development Principles

1. Be Current: The application you provide a link to must be the latest, stable final release version available, hosted on a publically available Web URL

2. Be Free of Charge: The application for which you submit a link to the Microsoft Web App Gallery must be provided free of charge and fully functional without time restrictions. You are welcome to charge for professional support or consulting services and/or provide an enhanced, enterprise version of the application for purchase on your site. We will be happy to provide a link to your commercial products and services from your page in the Microsoft Web Application Gallery.

3. Be Compatible: The application to which you provide a link must run on Windows Server 2003, Windows Server 2008, Windows XP & Windows Vista using best practices on running ASP.NET applications and PHP applications on IIS.

4. Be Deployable: The application to which you provide a link must integrate with Microsoft Web Deploy as described in the Microsoft Web Application Packaging Guide, and run with the Microsoft Web Platform Installer v2.

5. Be Supported: You must provide a publicly available Web site where end users can download your application, find documentation and/or get free on a best effort basis support through a forum.

6. Be Hostable: The application to which you provide a link must run well in a shared hosted environment as well as when the user has administrative rights for the computer.

7. Be Inclusive: If your link is included in the Gallery, you should include a link on your application community’s Web site to your Application entry on the Microsoft Web Application Gallery.

8. Be Safe: The application to which you provide a link must not harm customers or be malicious, dishonest, destructive, invasive, or act in any manner restricted by the Web Gallery Application Submission agreement.

Congrats to the IIS and /Web teams for this release, and thanks for all the guys on dasBlog for helping. Big thanks to Bill Staples for letting dasBlog 2.3 in on the launch.

Enjoy!

Related Links

Web Application Gallery and Web Platform Installer
MSDeploy Team Blog
Learn the Basics of Web Deploy (Quick Guides)
1. Web Deployment Tool Overview (overview of the features)
2. Installing the Web Deployment Tool (installation and remote service options)
Learn How to Package and Deploy an App (Quick Guides)

Create a Package (creating a package including content, databases and more)
Install a Package (deploying the package to a local or remote machine)
Configure the Web Deployment Handler (enable deployment to an IIS 7.0 server)

Bill’s Web Platform Release
Mai-lant’s on IIS at MIX
Mai-lan’s Web App Gallery and community engagement

UrlScan and ADO.NET Data Services (Astoria)

Posted 2009-02-09 05:44 PM in ASP.NET | IIS | Web Services.

In an ongoing series on different reasons that I suck, I wanted to share another interesting quasi-debugging story about this public ADO.NET Data Services (Astoria) service I'm trying to help get live at a large company.

I'd mentioned how I had some trouble with UrlScan and Astoria Services in my post on "RTFLF - Read the Expletive Log File."

UrlScan 3.1 is a security tool that restricts the types of HTTP requests that IIS will process. By blocking specific HTTP requests, the UrlScan 3.1 security tool helps to prevent potentially harmful requests from reaching applications on the server. UrlScan 3.1 is an update to UrlScan 2.5 supports IIS 5.1, IIS 6.0 and IIS 7.0 on Windows Vista and Windows Server 2008.

UrlScan is a really valuable tool and a really powerful way to slap down evil and attacks on your Web Server. It is VERY configurable and very powerful and takes a clear mind to make sure that all your application's URLs work well while still preventing evil. Learn how to setup UrlScan here.

When we moved the service onto a production machine, it turned out that their production UrlScan.ini was locked down more than development. They were disallowing single quotes in QueryStrings as well as Dots in Url Paths.

The trick was that they haven't partitioned their sites up by subdomain, or website, just AppDomain. This means that they have 40+ "logical sites" running under one instances of IIS6, which means that they can't setup a separate IIS site with their own configuration and instance of the UrlScan ISAPI Filter. They also didn't feel like making a separate domain or subdomain just for our one little service.

OK, blah blah blah, background, right? So where did things go wrong?

URLScan has a section in the UrlScan.ini file called [AlwaysAllowedUrls] that does just that. It allows access to any URL in that section. Adding a URL there will make UrlScan completely ignore it.

I added our service to this section like this:

[AlwaysAllowedUrls]
/foo/MyService.svc

And I thought this would work. It DID allow us to request that Url. However, no other Astoria queries worked.

When you traverse Entities in ADO.NET Data Services (Astoria), your URLs look like this, for example:

/foo/MyService.svc/Cars()
/foo/MyService.svc/Cars()$filter=((Visible%20eq%20true)

And I didn't initially see why these were being blocked.

The issue, pointed out to me by the IIS Team's Wade Hilmo (check out his blog) was obvious (hence: I suck). The URL isn't the file, it's everything before the ?. It's everything before the QueryString. This is one of those "Duh" moments, and was my bad. I know the difference, of course, and understand Urls fully, but I completely brain-farted this obvious point when looking at ADO.NET Data Service queries and had been mentally parsing them wrong for hours. Once Wade set me straight, configuring UrlScan.ini was trivial:

[AlwaysAllowedUrls]
/foo/MyService.svc/Cars()
/foo/MyService.svc/Houses()
/foo/MyService.svc/Whatevers()

In the ADO.NET Data Services parlance, the Entities exist after the .svc filename, but before the ? delimiter. Once we added those to our UrlScan.ini file, we were able to maintain the existing state of "super locked-down-ed-ness" while still allowing our services to run.

It is not possible to have a SQL Injection attack on a ADO.NET Data Service, due to the complete query rewrite that happens server side when a URL query comes into the system. The analogy is to think of in the same way that a LINQ query goes through a complete query re-write or a translation layer before it actually gets executed as SQL on the server. This re-write or translation is the layer of protection against SQL Injection attacks.

I was happy to learn that I could have my ~~Cake~~ ADO.NET Data Services and my UrlScan too.

RTFLF - Read the Expletive Log File

A buddy of mine and I had a nice slap in the face yesterday. I was helping him deploy an ADO.NET Data Service to a large company's staging server and we were seeing REALLY odd behavior.

We'd request something like /myservice.svc and get a 404. But we could request /myservice.svc/Stuff or /myservice.svc/?metadata.

We settled in to debug this. We thought we were "getting down to basics." You know, you've done this. The conversation goes something like:

"Ok, people, what's the definition of insanity? Trying the same thing and expecting a different result."

"Right...let's challenge all our assumptions. Let's start from scratch. Can get Hello World working?"

"What's the ACLs on that file? Is the .svc extension registered? Are we sure we have the right version of .NET?"

We were both tired and we wasted a couple of hours basically dicking around, hitting Refresh and hoping for another solution. I wanted to plugin procexp and filemon and get down to some serious CSI: IIS-type debugging, but here's the rub: We didn't have access to the machine. Only "large company's" guys were allow to touch anything. We could make suggestions, watch a SharedView session, but the human latency of the whole process was slowing us by a factor of at least five.

But I can't blame it all on the process. In retrospect, it was my fault. I'm a good debugger. I know this and I'm happy to say it. However, I can recognize a ninja when I see one. Well, if you can see the ninja, maybe they aren't a ninja, but still. I reached out to a real debugging ninja. What did he do that I was missing?

I ignored a basic tenet of debugging. It wasn't that I didn't RTFM. I didn't RTFLF.

My debugger-ninja-friend started out by simply asking us to Start|Run and type "LogFiles."

At this point I realized that this process was going to make me look and feel like an idiot. My internal lights went on and I realized my buddy and I hadn't bothered to check any log files. We'd been treating IIS like it was a black box. It's not. It logs the hell out of everything that goes in and out if you want it to.

We were trying to debug a 404 on this .svc. We opened the log in Notepad, went to the bottom, searched up for ".svc" and there it was:

This Page was blocked by Microsofts URL Scan 3.0 Reason=Dot-in-path-detected

You could have knocked me over with a feather. I've said myself, UrlScan is step 0. If you're debugging a weird 404, UrlScan is the first and most obvious place to look and it was all there, in the log files. You remember, the log files I never looked at. ;)

Did my ninja friend know or care? No, because he RTFLF. A painful reminder to me as I wasted a bit of a ninja's time. Everyone knows, don't piss off a ninja. He was cool about it though.

Today's Lesson: Whatever it was, it was probably logged. Try there first.

* Pic from Dr. McNinja.

Web Platform Installer now supports XP - And the Master Plan continues

Posted 2008-11-24 12:10 PM in ASP.NET | ASP.NET MVC | IIS.

Remind me to tell you some time why the IIS team is so ~~evil~~ clever. It' cool to see the first steps of a master plan to make things awesome start to come to fruition.

I've said over and over that IIS7 is rocking sweet and once you start using it, you'll never want to go back. It's all part of the whole /web platform and it's always been ridiculously tedious to install. It's getting WAY easier, all as part of a super-secret master plan I can't tell you about or I'd have to kill you.

Seriously, the Web Platform Installer, or Web-PI, hit Release Candidate last night. The #1 complaint on my last post about this was that it didn't support XP. Bam:

Supported Operating Systems are: Windows Vista RTM, Windows Vista SP1, Windows XP, Windows Server 2003, Windows Server 2008

How ya like me now, son? ;) Also works for Server 2003, so that's cool.

Web-PI's got a number of cool things, but one subtle one you might take note of is the Web Deployment Tool Beta (MSDeploy) that I mentioned before. This tool makes it very easy to package up all the configuration schmutz of your application and (re)deploy it elsewhere. As such, it supports synchronizing your sites and applications across server farms, which was a huge problem for me when I was in banking. It integrates nicely with IIS Manager too.

Also of note is the inclusion of ASP.NET MVC Beta as an option, as well as URL Scan 3.1 and IIS 7.0 manager which lets you manage IIS7 from non-IIS7 machines like, ahem, XP machines. Basically all the things that were hard to find, now, less so.

Now, my first worry when I saw this was, "well, some small team is doing this applet and it's neat and all but soon it will fall into disrepair and fade away, like calc.exe." Well, I just got a personal call from Bill Staples who manages all this stuff and he told me

"oh no, we've got a master plan."

He then proceeded to tell me the master plan and it was good™. I can't tell you, but I'm working on ways to tell you without Bill firing me. Regardless, there's some cool stuff coming in the /web space. It's very tidy, very open, very clean, very sourcey, very appy, very nice.

So, go get the Web Platform Installer and the Web Application Installer and ponder on why there's a "A - F | G - L | M - R | S - T | U - Z" navigation thing on a page that only has 7 products for download.

Web Platform Installer Announcement

View SlideShare presentation or Upload your own. (tags: microsoft web)

Discuss!

Related Posts