Scott Hanselman

Back to Basics: Dynamic Image Generation, ASP.NET Controllers, Routing, IHttpHandlers, and runAllManagedModulesForAllRequests

April 7, '12 Comments [33] Posted in ASP.NET | ASP.NET MVC | Back to Basics | Learning .NET
Sponsored By

Warning, this is long but full of info. Read it all.

Often folks want to dynamically generate stuff with ASP.NET. The want to dynamically generate PDFs, GIFs, PNGs, CSVs, and lots more. It's easy to do this, but there's a few things to be aware of if you want to keep things as simple and scalable as possible.

You need to think about the whole pipeline as any HTTP request comes in. The goal is to have just the minimum number of things run to do the job effectively and securely, but you also need to think about "who sees the URL and when."

A timeline representation of the ASP.NET pipeline

 

This diagram isn't meant to be exhaustive, but rather give a general sense of when things happen.

Modules can see any request if they are plugged into the pipeline. There are native modules written in C++ and managed modules written in .NET. Managed modules are run anytime a URL ends up being processed by ASP.NET or if "RAMMFAR" is turned on.

RAMMFAR means "runAllManagedModulesForAllRequests" and refers to this optional setting in your web.config.

<system.webServer>
<modules runAllManagedModulesForAllRequests="true" />
</system.webServer>

You want to avoid having this option turned on if your configuration and architecture can handle it. This does exactly what it says. All managed modules will run for all requests. That means *.* folks. PNGs, PDFs, everything including static files ends up getting seen by ASP.NET and the full pipeline. If you can let IIS handle a request before ASP.NET sees it, that's better.

Remember that the key to scaling is to do as little as possible. You can certainly make a foo.aspx in ASP.NET Web Forms page and have it dynamically generate a graphic, but there's some non-zero amount of overhead involved in the creation of the page and its lifecycle. You can make a MyImageController in ASP.NET MVC but there's some overhead in the Routing that chopped up the URL and decided to route it to the Controller. You can create just an HttpHandler or ashx. The result in all these cases is that an image gets generated but if you can get in and get out as fast as you can it'll be better for everyone. You can route the HttpHandler with ASP.NET Routing or plug it into web.config directly.

Works But...Dynamic Images with RAMMFAR and ASP.NET MVC

A customer wrote me who was using ASP.NET Routing (which is an HttpModule) and a custom routing handler to generate images like this:

routes.Add(new Route("images/mvcproducts/{ProductName}/default.png", 
new CustomPNGRouteHandler()));

Then they have a IRouteHandler that just delegates to an HttpHandler anyway:

public class CustomPNGRouteHandler : IRouteHandler
{
public System.Web.IHttpHandler GetHttpHandler(RequestContext requestContext)
{
return new CustomPNGHandler(requestContext);
}
}

Note the {ProductName} route data in the route there. The customer wants to be able to put anything in that bit. if I visit http://localhost:9999/images/mvcproducts/myproductname/default.png I see this image...

A dynamically generated PNG from ASP.NET Routing, routed to an IHttpHandler

Generated from this simple HttpHandler:

public class CustomPNGHandler : IHttpHandler
{
public bool IsReusable { get { return false; } }
protected RequestContext RequestContext { get; set; }

public CustomPNGHandler():base(){}

public CustomPNGHandler(RequestContext requestContext)
{
this.RequestContext = requestContext;
}

public void ProcessRequest(HttpContext context)
{
using (var rectangleFont = new Font("Arial", 14, FontStyle.Bold))
using (var bitmap = new Bitmap(320, 110, PixelFormat.Format24bppRgb))
using (var g = Graphics.FromImage(bitmap))
{
g.SmoothingMode = SmoothingMode.AntiAlias;
var backgroundColor = Color.Bisque;
g.Clear(backgroundColor);
g.DrawString("This PNG was totally generated", rectangleFont, SystemBrushes.WindowText, new PointF(10, 40));
context.Response.ContentType = "image/png";
bitmap.Save(context.Response.OutputStream, ImageFormat.Png);
}
}
}

The benefits of using MVC is that handler is integrated into your routing table. The bad thing is that doing this simple thing requires RAMMFAR to be on. Every module sees every request now so you can generate your graphic. Did you want that side effect? The bold is to make you pay attention, not scare you. But you do need to know what changes you're making that might affect the whole application pipeline.

(As an aside, if you're a big site doing dynamic images, you really should have your images on their own cookieless subdomain in the cloud somewhere with lots of caching, but that's another article).

So routing to an HttpHandler (or an MVC Controller) is an OK solution but it's worth exploring to see if there's an easier way that would involve fewer moving parts. In this case the they really want the file to have the extension *.png rather than *.aspx (page) or *.ashx (handler) as it they believe it affects their image's SEO in Google Image search.

Better: Custom HttpHandlers

Remember that HttpHandlers are targeted to a specific path, file or wildcard and HttpModules are always watching. Why not use an HttpHandler directly and plug it in at the web.config level and set runAllManagedModulesForAllRequests="false"?

<system.webServer>
<handlers>
<add name="pngs" verb="*" path="images/handlerproducts/*/default.png"
type="DynamicPNGs.CustomPNGHandler, DynamicPNGs" preCondition="managedHandler"/>
</handlers>
<modules runAllManagedModulesForAllRequests="false" />
</system.webServer>

Note how I have a * there in part of the URL? Let's try hitting http://localhost:37865/images/handlerproducts/myproductname/default.png. It still works.

A dynamically generated PNG from an ASP.NET IHttpHandler

This lets us not only completely bypass the managed ASP.NET Routing system but also remove RAMMFAR so fewer modules are involved for other requests. By default, managed modules will only run for requests that ended up mapped to the managed pipeline and that's almost always requests with an extension. You may need to be aware of routing if you have a "greedy route" that might try to get ahold of your URL. You might want an IgnoreRoute. You also need to be aware of modules earlier in the process that have a greedy BeginRequest.

The customer could setup ASP.NET and IIS to route request for *.png to ASP.NET, but why not be as specific as possible so that the minimum number of requests is routed through the managed pipeline? Don't do more work than you need to.

What about extensionless URLs?

Getting extensionless URLs working on IIS6 was tricky before and lots of been written on it. Early on in IIS6 and ASP.NET MVC you'd map everything *.* to managed code. ASP.NET Routing used to require RAMFARR set to true until the Extensionless URL feature was created.

Extentionless URLs support was added in this KB http://support.microsoft.com/kb/980368 and ships with ASP.NET MVC 4. If you have ASP.NET MVC 4, you have Extentionless URLs on your development machine. But your server may not. You may need to install this hotfix, or turn on RAMMFAR. I would rather you install the update than turn on RAMMFAR if you can avoid it. The Run All Modules options is really a wildcard mapping.

Extensionless URLs exists so you can have URLs like /home/about and not /home/about.aspx. It exists to get URLs without extensions to be seen be the managed pipelines while URLs with extensions are not seen any differently. The performance benefits of Extensionless URLs over RAMMFAR are significant.

If you have static files like CSS, JS and PNG files you really want those to be handled by IIS (and HTTP.SYS) for speed. Don't let your static files get mapped to ASP.NET if you can avoid it.

Conclusion

When you're considering any solution within the ASP.NET stack (or "One ASP.NET" as I like to call it)...

The complete ASP.NET stack with MVC, Web Pages, Web Forms and more called out in a stack of boxes

...remember that it's things like IHttpHandler that sit at the bottom and serve one request (everything comes from IHttpHandler) while it's IHttpModule that's always watching and can see every request.

In other words, and HttpHandler sees the ExecuteRequestHandler event which is just one event in the pipeline, while HttpModules can see every event they subscribe to.

HttpHandlers and Modules are at the bottom of the stack

I hope this helps!


Sponsor: Thank you to my friends at Axosoft for sponsoring the Hanselman feed this week. Do check out their product! Imagine agile project management software that is brilliantly easy to use, blazingly fast, totally customizable, and just $7 per user. With OnTime Scrum, you won't have to imagine. Get started free.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Saturday, April 07, 2012 1:55:55 AM UTC
I ended up ditching modules and handlers entirely in a small project that I recently ported, I suppose out of desire to not have stuff "lost" in web.config. Instead, I'm using MVC actions for handlers and global action filters as modules.

Of course, as you said, that might be doing more than I should on some things, and the actions and global filters actually become related. So for example, I have a global filter that does some work about what users can see, but it checks to see if the action has a custom ignore action filter on it first (those actions return images).

Is this doing more than I have to? Absolutely. Does it create a scaling problem? In this case, not even close. It's an optimization that will never see the traffic to justify it. In my case, I'll take the easy development discovery of the actions and filters over extra web.config entries or .ashx files.
Saturday, April 07, 2012 2:21:13 AM UTC
Great article, Scott! This topic has been on my 'to-blog' list for 3 years, thanks for relieving me of the duty :)

I'm going to be linking to this on my "Why ActionResult < HttpModule for disk cached responses" KB article.

I'd like to add a few notes:

1. If you're caching more than a few dozen/hundred images, you're going to want some kind of disk caching instead of using the built-in memory cache. The performance characteristics of the ASP.NET Cache are particularly bad when it comes to images.

2. IIS is way more efficient than ASP.NET at serving static files from disk.

3. PostAuthorizeRequest is where you have to call RewritePath to let IIS do the serving. Any later, and it won't work. Any earlier, and you won't get authorization or authentication support (which may or may not be important to you).

4. An HttpHandler (or ActionResult) gets involved too late to be able to deleagte work back to IIS in an efficient (I.e. RewritePath, not WriteFile) manner. It also prevents you from leveraging any existing authentication or authorization frameworks.

Conclusion - an HttpModule offers far more flexibility and room for optimization than an IHttpHandler or ActionResult does. For most tasks it doesn't matter, but with image processing or generation, you may need to sacrifice a (small) amount of code simplicity and consistency to gain performance and scalability.


Further notes

A perfect implementation of disk caching is Extremely Hard to do. That's coming from someone whose two favorite coding specialties are compiler design and image processing. Dealing with proper file access coordination between IIS and ASP.NET, web farm/web garden support, and designing efficient background cleanup algorithms makes for a difficult job even if you and System.Threading are BFFs.

My implementation for the imageresizing.net library does NOT implement ICacheProvider - it invalidates based on modified dates, not callbacks, and uses a sharded hybrid LRU/LRC cleanup algorithm that minimizes disk I/O during high-traffic periods. It supports VirtualPathProviders and the more easily implemented IVirtualImageProvider.

I've done a ground up rewrite of the disk caching system 5 times in the last 5 years. It went from handling 200 images to handling 5K, then to 20K, then to 80K, then to 2 million with cleanup off, then to 2 million with cleanup working.

Getting past 2 million cached images and/or 20TB of source images is very difficult. Even with a fresh NTFS partition of 4TB dedicated to a caching, System.File.Exists becomes pretty slow. Even a binary search is slow with enough elements. If you suspect you are going to have more than 100K 'live and active' images, it's probably time to get yourself a CDN that supports custom origin servers, like Amazon CloudFront and let it handle the caching.

To you DB junkies, SQL is NOT an appropriate place for caching or even storing images. Despite the strides it's made in recent years, it's still officially recommended against by MS. Reason 1 of 20: the whole file waits in RAM while the client downloads it.

Side note regarding example: Unless your OutputStream is somehow seek-able, you shouldn't be able to directly encode a PNG to it.

On .NET 2, this won't work: bitmap.Save(context.Response.OutputStream, ImageFormat.Png);
Saturday, April 07, 2012 2:26:45 AM UTC
One more thing - if you're generating images and already using the imageresizing.net library, you can hook into it's disk caching easily by implementing IVirtualImageProvider. An implementation of a gradient generator weighs in at 70 lines excluding using statements.
Saturday, April 07, 2012 3:44:40 AM UTC
Amazing, I was looking for a simple graphic to explain ASP.Net pipelining before having to do it myself...and you just did the job...as usual, thanks Scott!
Saturday, April 07, 2012 10:05:58 AM UTC
Wouldn't it be great if just by defining routes the runtime could tell the web server about the paths we're interested in (including wildcards) and the server would (transparently) add 'modules' instead of us using RAMMFAR?

E.g.: register rout images/{product}/logo.png should translate into the path images/*/logo.png, then a transparent 'module' registration would make sure that path, and only that path, is mapped.

Pit of success and all...
Mike
Saturday, April 07, 2012 12:04:12 PM UTC
@Nathanael: Don't overlook proper implementation of 304 status and the appropriate headers. If you're serving non-static images, there's more to win there than with caching.
Saturday, April 07, 2012 1:02:25 PM UTC
Excellent article. Thanks Scott!

One question: Is there a way in the handler to get the name of the handler that is defined in the web.config?

Thanks again
Saturday, April 07, 2012 1:42:07 PM UTC
@Jeff - That's true only if 1/N or more server resources are used to transmit the static file vs. generate it, where N is the number of requests for the same file by the same browser. Unlikely in this scenario, considering avg. RAM and CPU use.

Also, by letting IIS take control of serving the cached file, we not only get proprer 304 responses and Last-Modified support, but accept-bytes, and support for about 30 other HTTP features that ASP.NET is scared of. You get the best of both worlds this way.

Also, in IIS7, we get to set the Expires header programmatically as well.
Saturday, April 07, 2012 4:42:58 PM UTC
I was about to make the case for using HttpModules with cached generated images, but I see Nathanael beat me to it.

We've taken that approach - using RewritePath in an HttpModule to get IIS to serve the file directly - in DynamicImage, and in the tests I've done it's more performant than using HttpHandler and WriteFile (or equivalent).
Saturday, April 07, 2012 5:37:31 PM UTC
Really interesting post Scott. Thanks for putting this together! Here's a question though. It seems that ELMAH has become almost a standard plugin for web apps. Every example configuration I see has runAllManagedModulesForAllRequests="true" I'm wondering if that is really necessary. What are the effects of actually setting that to false? I Other words, why would I ever want it set to true? Pardon my ignorance here.
Sunday, April 08, 2012 4:40:35 AM UTC
@ Nathanael: I think you're limiting the scope and arguing something different. There's nothing unlikely about the scenario to me. For example, I store user avatars in my forums in the database, and it's pretty typical that the same five people post stuff on every page. Sure, I could write images to the file system, and let IIS do what it does, but then I'd have thousands of files to backup. No thanks!

It's like anything else... there is no cure-all. I'm also not Facebook, and using a tiny fraction of the resources my overkill servers have. Implementing 304's is not hard, and you do it once, and you never need to do it again.
Sunday, April 08, 2012 6:24:01 PM UTC
@Nathanael, that's great but what about situations where you don't have a persistent disk store and want to cache files long term? A database of some form is basically the only option in that situation - be it SQL, nosql or whatever.

Richard Price
Sunday, April 08, 2012 8:27:28 PM UTC
Using an external datastore means you're doubling network traffic, and increasing RAM usage from a 8-128K buffer to storing the entire file in ram. We're also occupying triple the I/O threads, using the thread pool much more, and adding additional CPU. When you destroy 5/8ths of the benefit of caching, you might be better off with a non-persistent disk cache, especially if your reboot cycle averages more than a week.

I strongly suggest CloudFront (or another reverse caching proxy) when local disk storage isn't available. The DB is a really bad solution for public-facing sites.
Monday, April 09, 2012 2:35:26 PM UTC
Again, you're making a generalization. Most applications aren't Facebook. Heck, most apps aren't even Hanselman's blog!
Monday, April 09, 2012 5:08:56 PM UTC
If you've been applying security patches regularly you may already have the Extensionless URLs enhancement. It not only came out as a standalone patch (KB980368) but also as part of MS10-040 (which on Windows Server 2008 SP2 corresponds to KB982666).
Monday, April 09, 2012 5:14:19 PM UTC
@Rob, Elmah never required RAMMFAR, check pre-MVC Elmah setup guides. Early versions of MVC required RAMMFAR, that's why post-MVC-release examples of Elmah Web.Config had that set. And then it got copy/pasted all over the Web, so people think it's Elmah that needs it. In fact, Elmah work perfectly fine with RAMMFAR disabled, although it won't be able to catch static file 404s anymore.
Monday, April 09, 2012 9:50:36 PM UTC
Hi,

I'm wondering why you want to address the generated picture as a png. The sample: https://captchamvc.codeplex.com/ seems to embed the image by it's controller-name. Wouldn't it thus work without enableing RAMMFAR ?

Berst,
Bernhard
Monday, April 09, 2012 10:29:39 PM UTC
I'm drawing a blank here, but I seem to remember some common library/package/plugin/framework that required RAMMFAR to be true - ninject, elmah, glimpse, jquery mobile, modernizr?

i'm almost positive one of them required it, but it could have just been the sample configs they distributed with this switch on prior to the extensionless URL patch.

Anything you can think of?
Tuesday, April 10, 2012 9:38:29 AM UTC
I've used that for watermark creation.
Tuesday, April 10, 2012 1:41:10 PM UTC
Why is this even relevant? Switch to MVC people.
Rob
Tuesday, April 10, 2012 3:16:50 PM UTC
@Rob, @Jeff, MVC is great for XML, XHTML, and logic. It's terrible for static (or statically cached files). We're not talking about a 2 or 5X difference here, we're talking about up to two orders of magnitude. Static files can be served that much more efficiently than dynamic content, because you don't necessarily need a thread to do it - thus the nginx/G-WAN craze. Forcing static files to go through 2-4 native/managed boundary switches and potentially keep a copy in RAM *per browser request* (if it's not a FileResult) is pretty expensive, and MVC doesn't implement much more than HTTP 200 and 500.

If you have less than 1K viewers per day, it may not matter. But if you have a decent volume of static files, you really need to think about this stuff.
Tuesday, April 10, 2012 5:12:44 PM UTC
Rob - It's hugely relevant to performance. Do less and you will scale more. MVC is lovely, but as Nathanael points out, it's not the answer to everything. I realize MVC is fun but it's not the hammer for all nails.
Tuesday, April 10, 2012 5:52:16 PM UTC
Thanks for the article, Scott. I would like to see more articles in this same vein.

I think a lot of developers (myself included) who jumped on the ASP.NET bandwagon after ASP.NET MVC are confused by the lower level inner workings of ASP.NET (as demonstrated by the 'just use MVC for everything!' comments).

I've been a professional ASP.NET developer since the MVC beta days, but I still find it hard to find relevant, up-to-date information about things like HttpModules, the ASP.NET application life cycle, what the events in Global.asax should actually be used for, etc.
Ryan
Wednesday, April 11, 2012 10:47:24 AM UTC
Hello sir,


I have make one mcq online test application which is web based. I found one big problem and I have no more time in deadline in this project. Any one please give me guidance in my beloved problem.

I have developed above app. in webbased, with .NET 3.5 with C# and SQL Server 2008. I found one big problem, that I could not use the Maths Equations which are made by MathType Software
in my Editor control. Even I couldn't use Ms-Word Equations in my Editor control in asp.net
application please guide me. I have use Telerik Editor control as well as Ajax Editor Control.

Please some one help me in this matter...

Please ...
Thanx in Adv.. if any one get solution let me know...


Regards,

AMIT
Amit
Wednesday, April 11, 2012 4:07:00 PM UTC
MVC is lovely, but as Nathanael points out, it's not the answer to everything. I realize MVC is fun but it's not the hammer for all nails.


MVC + Azure CDN to point all of my static-ish "files" to is currently one giant sledge hammer that destroys all nails :)
Friday, April 20, 2012 10:35:35 AM UTC
Hi,

If i have hundred of image in a page, which will resize and display the image on the fly using method above, will it be any side effect?
Sunday, July 01, 2012 7:17:52 PM UTC
Months late to the game here, but lets say you have shared hosting and so disabling this makes it so there is no "default document" per se and the website no longer loads (unless I manually enter in the URL to a controller and action). How would one go about turning RAMMFAR off in this case or is this one of those instances I have to use it? I will continue attempting some Google-foo to track down an answer, but if you have one handy that'd be dandy.
Jon Seeley
Friday, September 21, 2012 12:06:16 PM UTC
Hello Sir,This is great post

What if i want create custom url ,I dont understand how HTTP handler work in ASP.NET MVC.
Harshil Gandhi
Wednesday, October 03, 2012 8:15:22 AM UTC
Hi Scott
I have a web application that works locally on my workstation as well as deployed to a local test server running IIS7.5 and Server 2008 R2 with the RAMMFAR setting turned off. The performance improvement with this off was significant.

<modules runAllManagedModulesForAllRequests="false" />

It is an MVC3 application.

However when we deploy this same application to Azure we get 404 and 403 errors when the web config has this turned off, am assuming something relating to the way the routing is handled in Azure - though it alludes me as to what would be different in a standard web role..?

In Azure when it is turned on the app runs ok - but naturally we need to be turned off. I have read in forum posts that there are issues with this in Azure. Can you detail a work around for this in Azure.?

Thanks so much.

Really enjoyed this post and also the podcast - very informative.
Sunday, June 23, 2013 8:01:18 PM UTC
At this moment I am going away to do my breakfast, afterward having my breakfast coming yet again to
read other news.
Wednesday, July 10, 2013 6:36:09 PM UTC
Hi Scott,
Excellent article! Thanks for putting that together!
Steve
derSteve
Thursday, August 08, 2013 4:24:27 PM UTC
Hi Scott,

In registering the handler, you still have preCondition="managedHandler", which I think isn't an attribute when adding handlers. Not to be nitpicking, but it's kinda confusing in this context of this article.

-Freek
Monday, August 19, 2013 7:23:51 AM UTC
This is my sample data from database,

ID Name ImageUrl

1 a http://notous.blob.core.windows.net/images/1-9.jpg

2 b http://notous.blob.core.windows.net/images/10_discount-150x150.jpg

3 c http://notous.blob.core.windows.net/images/FB-button-341x341.png

I want display this data not for same This ImageURL display as image in My view Using ASP.NET MVC,

And also At the time of insert also after uploading the image save in database Like above(its possible???)

please help me,I am new to this MVC
anil
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.