Scott Hanselman

Blocking Image Hotlinking, Leeching and Evil Sploggers with IIS Url Rewrite

June 3, '13 Comments [60] Posted in IIS
Sponsored By

I recently discovered that a blog called (seriously) "Google Chrome Browser" was reblogging my site. (It of course has NO relationship to Google or the lovely folks on the Chrome team.)

This is a splog or "spam blog." It's less of a blog and more of a 'suck your feed in and reblog it.' Basically every post is duplicated or sucked in via RSS from somewhere else.  I get this many times a week and have for years.

However, this particular site started showing up ahead of mine in searches and that's not cool.

You evil bastards.

Worse yet, they have almost 25k followers on Twitter. I've asked them a few times to stop doing this, but this time I got tired of it.

They're even 'hotlinking' my images, which means that all my PNGs are still hosted on my site. When you visit their site, the text is from my RSS but I pay for the images bandwidth. The irony of this is thick. Not to mention my copyright notice is intact on their site. ;)

When an image is linked to from another domain the HTTP_REFERER header is populated with the location that the image is linked from. That means when my web server gets a request for 'foo.png' from the Google Chrome Browser blog I can see the page that asked for that image.

For example:

Request URL:http://www.hanselman.com/blog/content/binary/Windows-Live-Writer/How-to-run-a-Virtual-Conference-for-10_E53C/image_5.png
Request Method:GET
Referer:http://google-chrome-browser.com/penny-pinching-cloud-how-run-two-day-virtual-conference-10

Because this differentiates the GET request that means I can do something about it. This brings up a few important things to remember in general about the web that I feel a lot of programmers forget about:

That said, I want to detect these requests and serve a different image.

If I was using Apache and had an .htaccess file, I might do this:

RewriteCond %{HTTP:Referer} ^.*http://(?:www\.)?computersblogsexample.info.*$
RewriteHeader Referer: .* damn\.spammers

RewriteCond %{HTTP:Referer} ^.*http://(?:www\.)?google-chrome-browser.*$
RewriteHeader Referer: .* damn\.spammers

#make more of these for each evil spammer

RewriteCond %{HTTP:Referer} ^.*damn\.spammers.*$
RewriteRule ^.*\.(?:gif|jpg|png)$ /images/splog.png [NC,L]

Since I'm using IIS, I'll do similar rewrites in my web.config. I could do a whitelist where I only allow hotlinking from a few places, or a blacklist where I only block a few folks. Here's a blacklist.

<system.webServer>
<rewrite>
<rules>
<rule name="Blacklist block" stopProcessing="true">
<match url="(?:jpg|jpeg|png|gif|bmp)$" />
<conditions>
<add input="{HTTP_REFERER}" pattern="^https?://(.+?)/.*$" />
<add input="{DomainsBlackList:{C:1}}" pattern="^block$" />
<add input="{REQUEST_FILENAME}" pattern="splog.png" negate="true" />
</conditions>
<action type="Redirect" url="http://www.hanselman.com/images/splog.png" appendQueryString="false" redirectType="Temporary"/>
</rule>
</rules>
<rewriteMaps>
<rewriteMap name="DomainsBlackList" defaultValue="allow">
<add key="google-chrome-browser.com" value="block" />
<add key="www.verybadguy.com" value="block" />
<add key="www.superbadguy.com" value="block" />
</rewriteMap>
</rewriteMaps>
</rewrite>
</system.webServer>

I could have just made a single rule and put this bad domain in it but it would have only worked for one domain, so instead my buddy Ruslan suggested that I make a rewritemap and refer to it from the rule. This way I can add more domains to block as the evil spreads.

It was important to exclude the splog.png file that I am going to redirect the bad guy to, otherwise I'll get into a redirect loop where I redirect requests for the splog.png back to itself!

The result is effective. If you visit their site, I'll issue an HTTP 307 (Moved Temporarily) and then you'll see my splog.png image everywhere that they've hotlinked my image.

Not cool, splogger, not cool.

If you wanted to change the blacklist to a white list, you'd reverse the values of allow and block in the rewrite map:

<rewriteMaps>
<rewriteMap name="DomainsBlackList" defaultValue="block">
<add key="google-chrome-browser.com" value="allow" />
<add key="www.verybadguy.com" value="allow" />
<add key="www.superbadguy.com" value="allow" />
</rewriteMap>
</rewriteMaps>

Nice, simple and clean. I don't plan on playing "whac a mole" with sploggers as it's a losing game, but I will bring down the ban-hammer on particularly obnoxious examples of content theft, especially when they mess with my Google Juice.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Redirecting ASP.NET Legacy URLs to Extensionless with the IIS Rewrite Module

May 10, '13 Comments [19] Posted in ASP.NET | IIS
Sponsored By

ASP.NET has included support for "friendly URLs" for a while now. ASP.NET MVC has always supported friendly URLs and more recently, so has Web Forms. That means if you don't want to have the .aspx extension, you certainly don't have to.

However, there's a LOT of existing legacy apps out there as well as apps that you may not have full control over. For example, there's a site that I want to influence but it's got dozens (hundreds) of links to foo.html and bar.html existing pages.

Legacy (n): A super-fancy way of saying "already exists."

What I want to do is kind of sloppy and I'm doing it for aesthetic reasons. I'll hopefully get around to updating the site's links later and know that future links will be extensionless. But look at me, I'm justifying why I'm doing this, Dear Reader. You of all people know that sometimes you just gotta do something just because ya gotta Get It Done™©.

I want to:

  • redirect existing GETs to a /foo.html to /foo
    • Redirects are external
  • but, keep rewriting /foo to the underlying /foo.html so it handles the request
    • Rewrites are internal

I can do all this within my web.config using the IIS Url Rewrite Module.  I can do this with ANY file type that IIS can handle, meaning this isn't an ASP.NET-specific thing. This all happens well before your application gets involved. You'll note I did a similar thing with a PHP app running under IIS just last month.

Here's what my web.config looks like. Note that since I have Azure (or in just IIS 7+ and the Rewrite module) I just added this file. There was no configuration needed. The same would apply to any existing site. Be aware that sometimes super-"greedy" rewrite or redirect rules can end up grabbing ahold of your CSS or JS so you'll want to be aware if something odd happens.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="extensionless" stopProcessing="true">
<match url="(.*)\.html$" />
<action type="Redirect" url="{R:1}" redirectType="Permanent" />
</rule>
<rule name="removeextension" enabled="true">
<match url=".*" negate="false" />
<conditions>
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
<add input="{URL}" pattern="(.*)\.(.*)" negate="true" />
</conditions>
<action type="Rewrite" url="{R:0}.html" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>

This stuff is hard to write, though.

SIDE NOTE: My RegEx expert Ruslan points out that the final Action could be simplified slightly like this:

<add input="{URL}" pattern="\." negate="true" />

This stuff is also hard to test. The IIS Rewrite module has a great UI for IIS that will write a lot of these rules for you and let you test them interactively:

URL Rewrite module in IIS

Hope this helps. I wrote this post (and bookmarked it) for myself because I am always googling around for this particular rule to remind myself. Now I'll search my own blog. ;) Reason #64 to blog, friends.


SPONSOR: Big thanks to the feed sponsor this past week, Ext.NET (seriously, check out their demos, really amazing stuff!) - Quickly build modern WebForm and MVC (including RAZOR) Apps for ASP.NET. Free pancake breakfast with all purchases!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Changing ASP.NET web.config inheritance when mixing versions of child applications

March 26, '13 Comments [19] Posted in ASP.NET | Bugs | IIS
Sponsored By

Mixed Application Pools

My blog and all the sites in and around it are a mix of .NET 2.0, 3.5 and 4. This blog engine is currently .NET 3.5 and runs at http://hanselman.com/blog, but the application at http://hanselman.com/ (the root) is .NET 4.

You can happily mix and match applications across .NET versions on a single IIS instance. You can see how mixed my system is in the screenshot at right there.

However, things got messy when I changed the parent / application to .NET 4, but kept the child /blog as .NET 3.5 (the 2.0 CLR). I got lots of errors like

  • Unrecognized attribute ‘targetFramework’. Note that attribute names are case-sensitive. The targetFramework attribute was inherited from the root .NET 4 Web.config file in the Default Web Site root using ASP.NET configuration inheritance and confused the /blog .NET 2 application.

I didn't want to change the /blog applications' web.config. I just wanted to stop it from inheriting the settings from the parent application. Turns out you can wrap whole sections in a location tag, and then tell that scoped tag to prevent child applications from inheriting.

What you do is change the parent .NET 4 app's web.config to indicate its settings shouldn't flow down to the children, like the .NET 2/3.5 /blog app.

<location path="." inheritInChildApplications="false">
<system.web>
...your system.web stuff goes here
</system.web>
</location>

You can actually read about this in detail in the ASP.NET 4 "breaking changes" documentation. Of course YOU read those closely, don't you? ;)

I chose to change this settings for all of System.Web, but you could do it on a per-section basis if you preferred.

Hope this helps you!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Moving old apps from IIS6 to IIS8 and why Classic Mode exists

March 20, '13 Comments [12] Posted in ASP.NET | IIS
Sponsored By

I had an interesting emailed question today. Here's a paraphrased one sentence version of the question:

Why does an ASP.NET Runtime issue surface in IIS 8.0 Integrated Pool for an application we have run successfully on previous versions of IIS and classic mode in IIS 8.0 ?

It's less interesting that they've moved from IIS6 to IIS8 and more significant that they've moved from Classic Mode to the Integrated Mode pipeline.

So, the short answer. These pipelines are different and if you write an app expecting the behavior of one pipeline and the move it, your observed behavior will likely be different when you move it to the new pipeline.

IIS6 was released in Windows Server 2003 over 10 years ago. Apps used the ISAPI pipeline. By its nature everything in IIS6 is what we think of today as "classic mode," rather than the newer Integrated Pipeline in IIS7.

IIS6 with ASP.NET had two pipelines - the IIS unmanaged one and the managed ASP.NET one. Here's a simplistic but mostly accurate diagram (as is the case with diagrams):

IIS Classic Pipeline is two pipelines. One for IIS and one for ASP.NET

IIS7 and 8 were re-architected with the superior and faster Integrated Mode pipeline but retain "Classic" mode for compatibility.

IIS7 and up is one integrated pipeline

"Doctor, it hurts when I do that."

"Don't do that."

If you're moving an older app from IIS6 (which by definition was only "classic" mode) to IIS 8, the best near-term decision is to run in Classic Mode on IIS8.  Classic Mode is fully supported so you aren’t doing anything wrong by running in classic mode.  It's a mode that is there for a reason.

Think about your reasons and make an educated decision.

Remember that there can be incompatibilities and edge-case behavioral issues when ASP.NET apps attempt to move from the older ISAPI hosting model circa IIS 6 to the newer integrated mode that came in IIS7 without thinking about the ramifications.

I hope this helps.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

New Tools and New Content - ASP.NET, Visual Studio 11 Web and .NET 4.5 Developer Preview (with commentary)

September 16, '11 Comments [48] Posted in ASP.NET | ASP.NET Ajax | ASP.NET MVC | IIS | Mobile
Sponsored By

ASPNET_vNextWhile all of you Build attendees are making me feel bad because you have a fancy Tablet and I don't (sell me yours!) the folks over here in the "Angle Brackets Team" (I'm trying out some new names. One will stick.) have been busy.

Here is a summary of all the items from our team that have been announced and shown at Build this week, all of this will be publicly available today (9/16) along with a some narrative and asides from yours truly.

They are listening

One of the things I am am personally enjoying in working daily with this build of Visual Studio is that there's dozens (hundreds) of little "mental speed bumps" that are smoothed over. A lot of thought was put into workflow and common scenarios in order to, well, get out of a developer's way.

Another constant source of happiness on my part is the team's realization that not everyone can upgrade their Visual Studio, so you can round trip your projects and solutions. I can use Visual Studio 11 Developer Preview and open a .NET 4 application, save the project, and then open it in Visual Studio 2010 and it'll just work. As it always should have. Teams can mix and match and the beta guy can  be wacky without checking in solution files that will break the other folks.

By the way, have you seen our new org chart?

Our new org chart

Big Aside: Changing how we release software

Scott Aside: It's also worth noting that while there is a bunch of stuff in the next version of Visual Studio, there's an equally compelling amount of stuff being released from the Azure/Web Team on NuGet. This gets to my LEGO analogy where we developers have more choice of what we snap together. Most importantly many things will only live on NuGet. I'm looking at the possibility of  NuGet Feed of Supported Microsoft Products. Folks sometimes assume that NuGet is a place to throw binaries up because it's convenient, but we also need a way to know if a NuGet package is actually a support product, kind of like "Verified" is on Twitter. I'm interested in your thoughts on this in the comments, or even better, in the Discussion Tab on the NuGet project site.

Shipping via NuGet is also significant because it helps level the playing field with Open Source libraries. Microsoft's open source libraries will "compete" for our developer-attention by all being in NuGet, and folks should use the ones that make them happy. If you want to use Microsoft's Ajax Minifier or any of a half dozen others, then just get the one you want from NuGet. Microsoft's won't ship out of the box. It'll be on NuGet.

Visual Studio 11 Developer Preview

A new Solution ExplorerTuesday the "Windows 8 Express" version of Visual Studio was released with the Windows 8 Developer Preview release which supports Windows 8 development. Wednesday during the keynote with Satya Nadella,  Scott Guthrie, Jason Zander  Visual Studio 11 Developer Preview and .NET 4.5 were announced. The bits are available today to the public.

ASP.NET 4.5 Developer Preview includes new core functionality like WebSockets, anti-XSS encoding, granular request validation. It also includes Web Forms improvements like model binding, support for HTML5 and unobtrusive JavaScript. And there have many performance improvements in ASP.NET when combined with Windows 8 Server can decrease startup time by 30% and reduce the memory footprint by 30%.

Scott Aside: I continue to put gentle pressure on all the teams about semantic versioning. In this case, .NET 4.5 is the right version number. It's using the .NET 4 CLR and is an additive (with bug fixes) release to .NET 4. Because it does that, it's very compatible. Your existing .NET 4 stuff won't (shouldn't) break. Check out the Target framework dropdown in this version of Visual Studio. See how you can target from 2.0 to 4.5 with the same IDE? That will be useful for corporate customers who need to develop 2.0 apps but still want the IDE improvements.
All the frameworks from 2 to 4.5 living together in a single dropdown. I never thought I'd see the day.

Visual Studio 11 Developer Preview includes new lots of new functionality including full editor support for CSS 3, full editor support for HTML 5, full editor support for JS, and many niceties like how Web Forms events can be wired up in source view, Web Forms designers can be launched directly from source view, how Live Inspector (code named "Eureka") makes editing the HTML of your document live much easier by automatically finding the HTML in your document as you select it in the Browser and much more.

Scott Aside: They've leveled the playing field across all the editors so things like regions, collapsing hierarchies, commenting, formatting, etc are all the same between CSS, JS, HTML and code. One more reason to just turn off all your foolbars and embrace the text editor and code.

You can read more about all the new features that are enabled in this release by reading the What's new in ASP.NET 4.5 and Visual Studio Web Developer whitepaper here: http://www.asp.net/vnext/whats-new.

NuGet 1.5 w/Dev11 Support Released

We just published an update to NuGet 1.5 that supports Visual Studio 11 Developer Preview. This allows developers on the Preview builds to continue to use NuGet with the preview releases of Visual Studio.

Note: NuGet does not support the "Windows 8 Express" release that was released yesterday. Support was just recently added to that SKU of Visual Studio and will appear in the next public preview release of "Windows 8 Express".

You can download the new release here: http://www.nuget.org/

ASP.NET 4.5

There's a lot of new improvements in ASP.NET and WebForms in particular in this coming release. ScottGu is doing a series and has done these posts already:

Model Binding will be familiar to MVC folks ,as is Routing. Both features now existing in both WebForms and MVC. So much time is spent in WebForms pulling data out of the request, via Forms, or QueryStrings, or Controls. Sometimes it seems like 30% of my WebForms code is just moving data from the Request into a variable. Now I can do this:

public IQueryable<Product> GetProducts(
[QueryString("q")]string keyword,
[Control("categories")]int? categoryId)
{
IQueryable<Product> query = _db.Products;

if (!String.IsNullOrWhiteSpace(keyword))
{
query = query.Where(p => p.ProductName.Contains(keyword));
}

if (categoryId.HasValue && categoryId > 0)
{
query = query.Where(p => p.CategoryID == categoryId);
}

return query;
}

This method supports a GridView that can sort, filter by category, and search via URL. Note the use of a nullable int as well. It makes WebForms code a lot cleaner, and it's just one example.

Scott Aside: ASP.NET 4.5 is additive to ASP.NET 4, so your existing apps will run, of course, but you can just start using these new features once you're using 4.5. I've been able to "refactor via subtraction" and yank out a bunch of tedious monkey code.

One other thing I wanted to say. It's becoming clearer to me that it's less about WebForms or MVC or WCF WebAPI Services or OData or IHttpHandlers or SingalR and more about ASP.NET. I used to call mix-and-match apps "hybrid" applications, but now I'm realizing they are just ASP.NET applications. I'll use whatever LEGO piece(s) I want and you should also.

MVC 4 and Web Pages 2 Developer Preview Released

The new responsive design of the MVC4 HTML5 default templateASP.NET MVC 4 Developer Preview is the latest release of our MVC framework. This release includes built-in support for mobile sites, new, fresh HTML5 project templates as well as jQuery Mobile. It has enhanced support for asynchronous methods, and custom code generation.

Also included are Web Pages 2 (like MVC with the V and C in the same file) a lightweight framework for creating dynamic, data-driven websites. The latest version expands support for mobile devices and for integrating client scripts, and adds helpers for tasks like mapping. You can download the new release for Visual Studio 2010 and Visual Studio 11 here: www.asp.net/mvc/mvc4

Scott Aside: Web Pages is what Rob and I used to make http://thisdeveloperslife.com and I'll be doing http://hanselminutes.com over again in Web Pages in the next week or so. If you love Razor just want a simple site, write code, hit F5, and deploy, WebMatrix and Web Pages is a nice change of pace. Also, the fact that they were able to integrate node.js into WebMatrix is not only cool but it brings up interesting questions about WebMatrix and what is can be used for. You can do PHP on it too, you know. Hm.

WCF 4.5 Developer preview AND "Web API" shipping out of band on .NET 4 and up!

Introduced Wednesday as part of .NET 4.5, includes new core WCF functionality like WebSockets, UDP multicast, improved streaming and better async support leveraging  Tasks and C# async improvements. We continued our commitment to simplicity with a number of configuration improvements (read "reductions"), making WCF throttles and quotas work for developers by default, better manageability. The WCF Client programming model is included in .NET Core profile available to Windows-tailored app developer.

Scott Aside: I've used WCF for years, begrudgingly. It is powerful, no one would disagree, but it's notoriously complex and scary. I'm impressed with what they've done in this release because they've effectively used the infinite configurability of WCF and pluged in a re-imagination of what WCF works like in a world of REST and JSON. When it's time for WS-* and the Enterprise, that part of WCF is still there, but when you move to the RESTful open web of 2011-2012, the new Web API is clean and light, nicely integrated with ASP.NET and able to create simple and lightweight web services using things like JSON and conventions that make sense.

RIA with HTML5/JavaScript

Dinesh announced the developer preview for RIA for JS/HTML5 in his BUILD talk. It's a set of jQuery plugins that let you easily work with server data on the client. By bridging the different tiers, RIA/JS lets you quickly build rich forms-over-data applications with HTML/JQuery. The libraries are available via RIAServices.jQuery NuGet package as well as codeplex. The WCF RIA Services for Silverlight were also updated via WCF RIA Services V1.0 SP2 and WCF RIA Services Toolkit Aug 2011 update.

Scott Aside: RIA was another project that I had checked out very early on and said, meh. I'm zero for two on judging their projects because the WCF team has embraced jQuery, JSON and a more open web in a big way. I'm impressed with their efforts and open attitude. Take a look at their "BigShelf" starter project and check out how they are using jQuery to retrieve and bind, filter and sort data on the client side. Again, fewer black boxes since it's all HTML and JavaScript. Be sure to scroll down on that page as it's a complete walkthrough.

Here's a rollup of other related links for you, Dear Reader, with downloads at the very bottom.

Announcement Links

Download Links

Enjoy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Page 1 of 9 in the IIS category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.