Scott Hanselman

Blocking Image Hotlinking, Leeching and Evil Sploggers with IIS Url Rewrite

June 3, '13 Comments [60] Posted in IIS
Sponsored By

I recently discovered that a blog called (seriously) "Google Chrome Browser" was reblogging my site. (It of course has NO relationship to Google or the lovely folks on the Chrome team.)

This is a splog or "spam blog." It's less of a blog and more of a 'suck your feed in and reblog it.' Basically every post is duplicated or sucked in via RSS from somewhere else.  I get this many times a week and have for years.

However, this particular site started showing up ahead of mine in searches and that's not cool.

You evil bastards.

Worse yet, they have almost 25k followers on Twitter. I've asked them a few times to stop doing this, but this time I got tired of it.

They're even 'hotlinking' my images, which means that all my PNGs are still hosted on my site. When you visit their site, the text is from my RSS but I pay for the images bandwidth. The irony of this is thick. Not to mention my copyright notice is intact on their site. ;)

When an image is linked to from another domain the HTTP_REFERER header is populated with the location that the image is linked from. That means when my web server gets a request for 'foo.png' from the Google Chrome Browser blog I can see the page that asked for that image.

For example:

Request URL:http://www.hanselman.com/blog/content/binary/Windows-Live-Writer/How-to-run-a-Virtual-Conference-for-10_E53C/image_5.png
Request Method:GET
Referer:http://google-chrome-browser.com/penny-pinching-cloud-how-run-two-day-virtual-conference-10

Because this differentiates the GET request that means I can do something about it. This brings up a few important things to remember in general about the web that I feel a lot of programmers forget about:

That said, I want to detect these requests and serve a different image.

If I was using Apache and had an .htaccess file, I might do this:

RewriteCond %{HTTP:Referer} ^.*http://(?:www\.)?computersblogsexample.info.*$
RewriteHeader Referer: .* damn\.spammers

RewriteCond %{HTTP:Referer} ^.*http://(?:www\.)?google-chrome-browser.*$
RewriteHeader Referer: .* damn\.spammers

#make more of these for each evil spammer

RewriteCond %{HTTP:Referer} ^.*damn\.spammers.*$
RewriteRule ^.*\.(?:gif|jpg|png)$ /images/splog.png [NC,L]

Since I'm using IIS, I'll do similar rewrites in my web.config. I could do a whitelist where I only allow hotlinking from a few places, or a blacklist where I only block a few folks. Here's a blacklist.

<system.webServer>
<rewrite>
<rules>
<rule name="Blacklist block" stopProcessing="true">
<match url="(?:jpg|jpeg|png|gif|bmp)$" />
<conditions>
<add input="{HTTP_REFERER}" pattern="^https?://(.+?)/.*$" />
<add input="{DomainsBlackList:{C:1}}" pattern="^block$" />
<add input="{REQUEST_FILENAME}" pattern="splog.png" negate="true" />
</conditions>
<action type="Redirect" url="http://www.hanselman.com/images/splog.png" appendQueryString="false" redirectType="Temporary"/>
</rule>
</rules>
<rewriteMaps>
<rewriteMap name="DomainsBlackList" defaultValue="allow">
<add key="google-chrome-browser.com" value="block" />
<add key="www.verybadguy.com" value="block" />
<add key="www.superbadguy.com" value="block" />
</rewriteMap>
</rewriteMaps>
</rewrite>
</system.webServer>

I could have just made a single rule and put this bad domain in it but it would have only worked for one domain, so instead my buddy Ruslan suggested that I make a rewritemap and refer to it from the rule. This way I can add more domains to block as the evil spreads.

It was important to exclude the splog.png file that I am going to redirect the bad guy to, otherwise I'll get into a redirect loop where I redirect requests for the splog.png back to itself!

The result is effective. If you visit their site, I'll issue an HTTP 307 (Moved Temporarily) and then you'll see my splog.png image everywhere that they've hotlinked my image.

Not cool, splogger, not cool.

If you wanted to change the blacklist to a white list, you'd reverse the values of allow and block in the rewrite map:

<rewriteMaps>
<rewriteMap name="DomainsBlackList" defaultValue="block">
<add key="google-chrome-browser.com" value="allow" />
<add key="www.verybadguy.com" value="allow" />
<add key="www.superbadguy.com" value="allow" />
</rewriteMap>
</rewriteMaps>

Nice, simple and clean. I don't plan on playing "whac a mole" with sploggers as it's a losing game, but I will bring down the ban-hammer on particularly obnoxious examples of content theft, especially when they mess with my Google Juice.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Monday, June 03, 2013 7:30:21 PM UTC
Brilliant - It'd be great if we had much more of this, and just had 1 source of truth for a lot of information. I hate going to other sites just to find a badly formatted copy of stackoverflow.com and the such. Keep up the good work!
Monday, June 03, 2013 7:38:20 PM UTC
That's great. If only you could insert a link to your blog through the image. Content takes time to create and it's too bad that the owner of that site is only concerned with how many hits they get.
Monday, June 03, 2013 7:38:38 PM UTC
This is the sort of thing that search engines should be doing more to address. Ranking and relevance are among the main reasons we rely on search engines, and this is a case where they are clearly getting it wrong.
Leo Bushkin
Monday, June 03, 2013 7:41:11 PM UTC
Now that is the way to show them Scott.

But I would have been no so nice in that image with your message.

Tell them what you really think!! LOL

--Ron
Ron Garlit
Monday, June 03, 2013 7:48:33 PM UTC
Amazing use of wits!
Monday, June 03, 2013 7:50:36 PM UTC
Yeeaah Boyeeee! Well done Scott. Hope this deters them once and for all.
KR
pdimarco
Monday, June 03, 2013 7:54:19 PM UTC
What you need is a totally mental, trance inducing animated gif. Featuring cats
Dave Forster
Monday, June 03, 2013 8:00:51 PM UTC
Ahh! Thanks for this Scott! I've had years of people embedding my flash files on their sites and stealing my bandwidth (let alone copyright infringement), so I'll put this to good use with swf too.
Monday, June 03, 2013 8:10:16 PM UTC
Nice tip Scott.

I had the same problem several years ago, but I used an HttpModule with a wildcard (IIS6).
Monday, June 03, 2013 8:17:04 PM UTC
Nicely done! However it doesn't change the fact that visitors will use your bandwidth to download the splog.png image... perhaps you should host it on imgur.com instead ;)
Monday, June 03, 2013 8:18:01 PM UTC
If you are taking the time to url-rewrite your images for this site, why not just take the time to block that site from reading your RSS feed?
Davin
Monday, June 03, 2013 8:18:19 PM UTC
Well done, well done.
Although, you should put some funny animated gif
Monday, June 03, 2013 8:20:15 PM UTC
Checkmate! Who's in control now? Haha
Mike
Monday, June 03, 2013 8:21:45 PM UTC
Awesome article.
Thanks for the write-up, Scott.
Brett Jackson
Monday, June 03, 2013 8:44:17 PM UTC
@Davin

"Block that..." doesn't punish the offender in any way, shape, or form - and runs the risk of ruining the experience of people who actually visit your site.
K
Monday, June 03, 2013 9:00:21 PM UTC
IMO, keep a white list and a black list.

Here's the issue. Anyone can link to your content. There is no way to prevent that. But you can display a message saying "hey, don't do this."

But -- the thing is, they also can go to no-ip and come out with a new DNS. On Blogger or similar platforms, they can sign up for a new blog in minutes.

So I'd go with a white list, instead of a black list.
Dave Bacher
Monday, June 03, 2013 9:08:48 PM UTC
We used to do that, but found that many browsers block the referrer being transmitted. Also, many web bots/spiders don't send one either. Be careful.
Anthony Wieser
Monday, June 03, 2013 9:45:31 PM UTC
Dave - There are HUNDREDS of obscure Google Reader clones and last time I did that I ended up opening up LOTS domains and angering many.
Monday, June 03, 2013 9:50:01 PM UTC
You should update your substitute image to include your *real* blog's URL.

Maybe you can convert a few of the honest folks who stumble on the pirated material if you make it easy for them to find you?
Eddie
Monday, June 03, 2013 9:51:46 PM UTC
I saw a guy online that changed all the pictures to ones he could hotlink off porn sites. It was a site about high-end race cars and a very official French racing site was hotlinking his images that he took with his own camera that he was selling on his site.

Needless to say the racing organization was very embarrassed, but he almost got arrested on hacking charges (in the US, he most likely would have)!
PRMan
Monday, June 03, 2013 10:35:26 PM UTC
Google should provide a well-known image link you could insert instead of your custom message. Then when Google spiders the site, they could detect their image and know that a splogger lives there.
Monday, June 03, 2013 11:03:20 PM UTC
Heh - a few years back, one site I knew had the same problems - he started feeding the old image (pre site sale) from goat.se (NOT safe for work)
KG2V
Tuesday, June 04, 2013 12:25:28 AM UTC
White list is a bad idea. Referer can be disabled by browser settings (by privacy-minded people), most proxies strip it, and https:// won't pass it on to http:// links.
Anonymous
Tuesday, June 04, 2013 1:01:22 AM UTC
You should find someone with the Newman animated graphic from Jurassic Park movie and redirect the requests there. Then they won't be hijacking your bandwidth too :)

http://jurassicpark.wikia.com/wiki/File:YouDidn'tSayTheMagicWord.gif
Jay
Tuesday, June 04, 2013 3:16:22 AM UTC
Great job! Yeah, this is a horrible problem and one that unfortunately will continue on. Luckily if you're on your own server you can take control over this. More blog services (weblogs.asp.net hint hint) need the ability to enable users to do this.
Tuesday, June 04, 2013 4:03:46 AM UTC
No animated gif?
Tuesday, June 04, 2013 6:49:24 AM UTC
very nice article scott.
ramesh bolla
Tuesday, June 04, 2013 7:17:56 AM UTC
Another fun idea: https://github.com/kitcambridge/evil.js + https://www.owasp.org/index.php/Script_in_IMG_tags
Tuesday, June 04, 2013 7:22:24 AM UTC
Sadly, blocked by all modern browsers. Boo.
Tuesday, June 04, 2013 9:42:43 AM UTC
I wanted to see for myself, went to the site found lots of guff on there, but no Hanselman post..

So i searched for:

"ya buddy"

I got three hits - all yours ;).. and your web.config is working.
Gordon
Tuesday, June 04, 2013 10:25:56 AM UTC
Nice - I'm sure you (the real you) blogged about something like this a while ago with regard to IIS Url rewrite rules, or maybe it was just the canonical domain ones etc? I'm glad you found a good way to stop the spam anyhow - damn MOFOS
Tuesday, June 04, 2013 10:54:44 AM UTC
Brilliant!

I am wondering, would it also make sense to cause a delay when serving the "splog.png" image to make browsing experience on these spam blog sites even more sucky?

Tuesday, June 04, 2013 10:59:27 AM UTC
Thanks for this great post, Scott. Learning a lot about Internet, the way it works and how to stay safe.
Tuesday, June 04, 2013 12:50:27 PM UTC
As always, thanks for taking the time and extra effort not to punish those of us using aggregators to read your stuff. With the imminent demise of Google Reader, many of us have had to move to alternates, making your task much harder, unfortunately.
Frank Hagen
Tuesday, June 04, 2013 2:42:13 PM UTC
Everyone suggesting evil images are forgetting that the people who see the images aren't the people stealing the content.
Tuesday, June 04, 2013 2:54:04 PM UTC
Have you sent a takedown notice to their ISP, or is that more work than it's worth?
Tuesday, June 04, 2013 3:12:48 PM UTC
I hope they will reblog this post...

For the irony !!
Xas
Tuesday, June 04, 2013 4:33:33 PM UTC
waiting for the irony to unfold when they re-blog this post
ryan
Tuesday, June 04, 2013 6:01:46 PM UTC
I've done something similar in the past, although my replacement image wasn't so nice.

Back before the days of image-sharing sites like Imgur, I discovered that a user of mine was uploading images to my forum and then using it as a host for the images in other, much higher traffic forums. This was eating a good chunk of my bandwidth and resources.

My solution was a similar trick to serve up a replacement image for requests that originated from one of those forums (as a blacklist), and since I happened to have a picture of the user, the replacement image was the goatse picture censored with his face, as well as a note about how it's not nice to steal someone else's bandwidth.

Meanwhile, users of my own forum (and elsewhere) still got to see his original images.
Mike C
Tuesday, June 04, 2013 6:16:12 PM UTC
Nick: I agree with you on why not to use evil images.

In my case above, the person doing the hotlinking was somewhat known to his audience, since it was a forum environment. My use of a semi-evil image was more of a vindictive move to tarnish his reputation on those forums as punishment for using my server as an image host without my consent (to the point where legitimate requests to my server were seeing noticeable delays).
Mike C
Tuesday, June 04, 2013 6:29:15 PM UTC
Scott Nice trick to trick the spammers
Kamran Shahid
Tuesday, June 04, 2013 6:49:27 PM UTC
Of course you could also try contacting namecheap support, where the domain is registered or hostnine where the site's dns is pointing or "A Small Orange" where the site seems to be hosted. Maybe one of them would be interested in the abuse case.
Jonathan
Tuesday, June 04, 2013 6:54:26 PM UTC
Nice article...
Your message is clear but Do you think owner of that google chrome browser com will bother/mind/aware with this kind of stuff.
Sumit Kapadia
Tuesday, June 04, 2013 6:55:01 PM UTC
You're too nice, Scott :3
Mathias
Tuesday, June 04, 2013 8:15:31 PM UTC
Mike C: I agree your situation is different, even though your solution is the same. :)
Tuesday, June 04, 2013 8:24:48 PM UTC
Not only an informative article, but a great example of pique-induced ingenuity!
Joel
Tuesday, June 04, 2013 11:29:43 PM UTC
All is nice and dandy, Scott, until they stop hotlinking the images and copy them locally.

What are you going to do then? I am thinking legal actions.
Wednesday, June 05, 2013 3:07:24 AM UTC
Good article. Empty http_referer still gets through, but this is as good as it gets. As another reader pointed out a white list might be more effective in that it automatically rejects every one other than empty and your domain.
Wednesday, June 05, 2013 4:37:49 AM UTC
Hahaha! That's a real Nose-Break! I loved your approach Scott Hanselman. great one. Hope the Sploggers or Snailers are gonna get a damn who're faking to get traffic. Great Job! I appreciate! :-)
Wednesday, June 05, 2013 7:30:49 AM UTC
Scott as usual interesting article....So my query here is you were making use of rewrite and i think this feature is present only in IIS7...So in case of previous versions one way of handling these type of things is

Serving such files using handlers or module and in the handler we can check for Referer and serve with right image.

Do we have any simple process other than this.

Thanks,
Pavan
Wednesday, June 05, 2013 5:57:10 PM UTC
@Andrei Rinea:
I think Scott is one of few people remaining who is not obsessed with copyrights issues , we become crazy about any one copy a one word from us and we get screaming "take legal action, call laywer" , even if no real harm causing to us.
no thing require legal action in the end this website is market his work as they are not publishing it under another name , so why would he bother, I usually get happy if other websites publish my blog .
Sam
Wednesday, June 05, 2013 6:59:31 PM UTC
It is interesting that they appeared higher than you in search results because you have the rel author tag set in your google+ link so you would have hoped that google would have taken some notice of that!
Wednesday, June 05, 2013 8:20:46 PM UTC
Nicely done!!!
George
Thursday, June 06, 2013 10:24:47 AM UTC
Brilliant!

But I wouldn't be so kind w.r.t. the image... Some HD offensive porn is appropriate in this case.
Thursday, June 06, 2013 1:43:06 PM UTC
What about inserting a link to your feed articles, I've seen other people do it and it would prevent them from messing with your "Google juice", while it wouldn't be annoying for people reading your RSS.
Marc
Thursday, June 06, 2013 4:14:22 PM UTC
Hope they'll reblog this article as well! :)
Niclas
Friday, June 07, 2013 4:37:19 PM UTC
Scott, it's been a few days now. You should have some metrics. Time to post an update with the results of your efforts!
Matthew Noonan
Thursday, June 13, 2013 2:42:23 AM UTC
It wouldnt be such an issue if the search engines policed this themselves. But they wont, because they are the ones making money on the ad space....
Dave
Monday, September 30, 2013 10:39:17 PM UTC
Scott,

Thank you for the post. I was able to use it for my e-commerce class regarding search engine spam and splogging. I recalled your effective methods for combating such methods. Great posts, keep up the great work.

Thanks,

Dave A.
Dave A
Monday, November 25, 2013 4:44:03 AM UTC
Scott,

Nice. Congrats. One question.

I have a case where I don't want to serve my files from my website.

Lets say I have this.

media.domain.com

I want to be able to serve only domain.com (Referer would be domain.com).

Since I am requesting the file from media.domain.com it doesn't have a Referer. So, that situation bypass your code.

I added (^$|^https?://(.+?)/.*$) whether is empty or has something in the referer.

the thing is that C:1 is either null or empty.

I added to my black list a key="" value="block" but it doesn't seem to get it.

How could I solve this issue?

Thanks
Tony
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.