Scott Hanselman

A Back To Basics Case Study: Implementing HTTP File Upload with ASP.NET MVC including Tests and Mocks

June 28, '08 Comments [23] Posted in ASP.NET | ASP.NET MVC | Back to Basics
Sponsored By

A number of folks have asked me how to "implement the ASP.NET File Upload Control" except using ASP.NET MVC. This is a really interesting question for a number of reasons and a great opportunity to explore some fundamentals.

First, ASP.NET MVC is different since we don't get to use ASP.NET Server Controls as we're used to them. There's no "server controls" in the way that we're used to them.

Second, it'd be important to write Unit Tests for something like File Upload, and since ASP.NET MVC tries to be Unit Test friendly, it's an interesting problem to do tests. Why is it interesting? Well, ASP.NET MVC sits on top of ASP.NET. That means ASP.NET MVC didn't do any special work for File Upload support. It uses whatever stuff is built into ASP.NET itself. This may or not be helpful or interesting or even easy to test.

It seems then, that this is a good exercise in understanding a number of things:

  • HTTP and How File Upload works via HTTP
  • What ASP.NET offers for to catch File Uploads
  • How to Mock things that aren't really Mock Friendly
  • And ultimately, How to do File Upload with ASP.NET MVC

Here we go.

HTTP and How File Upload works via HTTP

It's always better, for me, to understand WHY and HOW something is happening. If you say "just because" or "whatever, you just add that, and it works" then I think that's sad. For some reason while many folks understand FORM POSTs and generally how form data is passed up to the server, when a file is transferred many just conclude it's magic.

Why do we have to add enctype="multipart/form=data" on our forms that include file uploads? Because the form will now be POSTed in multiple parts.

If you have a form like this:

<form action="/home/uploadfiles" method="post" enctype="multipart/form-data">
<label for="file">Filename:</label>
<input type="file" name="file" id="file" /><br />
<input type="submit" name="submit" value="Submit" />
</form>

The resulting Form POST will look like this (slightly simplified):

POST /home/uploadfiles HTTP/1.1
Content-Type: multipart/form-data; boundary=---------------------------7d81b516112482
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64)
Content-Length: 324

-----------------------------7d81b516112482
Content-Disposition: form-data; name="file"; filename="\\SERVER\Users\Scott\Documents\test.txt"
Content-Type: text/plain

foo
-----------------------------7d81b516112482
Content-Disposition: form-data; name="submit"

Submit
-----------------------------7d81b516112482--

Notice a few things about this POST. First, notice the content-type and boundary="" and how the boundary is used later, as exactly that, a boundary between the multiple parts. See how the first part shows that I uploaded a single file, of type text/plain. You can interpolate from this how you'd expect multiple files to show up if they were all POSTed at once.

And of course, look at how different this would look if it were just a basic form POST without the enctype="multipart/form=data" included:

POST /home/uploadfiles HTTP/1.1
Content-Type: application/x-www-form-urlencoded
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64)
Content-Length: 13

submit=Submit

See how the content type is different? This is a regular, typical form POST. Perhaps atypical in that it includes only a Submit button!

The point is, when folks add a ASP.NET FileUpload Control to their designer, it's useful to remember that you're buying into an abstraction over something. In this case, you're using a control that promises to hide the whole multipart MIME way of looking at things, and that's totally cool.

Back To Basics Tip
Know what your library is hiding from you and why you chose it.

As an aside, if you looked at an email of yours with multiple attached files, it would look VERY similar to the body of the first HTTP message as multipart MIME encoding is found everywhere, as is common with most good ideas.

What ASP.NET offers for to catch File Uploads

The FileUpload control is just a control that sits on top of a bunch of support for FileUploads in ASP.NET, starting with the classes Request.Files and HttpPostedFile. Those are the things that actually do the hold on to the parsed Files from an HTTP Request. You can use them to get a hold of a stream (a bunch of bytes in memory that are the file) or just save the file.

Since we can't use ASP.NET Server Controls in ASP.NET MVC, we'll use these classes instead. Here's how you usually grab all the files from an upload and save them:

foreach (string file in Request.Files)
{
HttpPostedFile hpf = Request.Files[file] as HttpPostedFile;
if (hpf.ContentLength == 0)
continue;
string savedFileName = Path.Combine(
AppDomain.CurrentDomain.BaseDirectory,
Path.GetFileName(hpf.FileName));
hpf.SaveAs(savedFileName);
}

Of course, you might want to change the directory and filename, maybe check the mimeType to allow only certain kinds of files, or check the length to limit your uploads, but this is the general idea.

Note that Request.Files has been around since 1.x and isn't a strongly typed collection of anything, so the GetEnumerator() of .Files that we're using in the foreach returns strings that are then used as keys into the Files[] indexer. It's a little wonky as it's old.

However, don't let me get ahead of myself, let's write the tests first!

How to Mock things that aren't really Mock Friendly

After creating a new ASP.NET MVC Project and making sure to select a test framework, I'll drop into a Controller Test and make a new TestMethod that kind of looks like I expect my method to be used.

[TestMethod]
public void FakeUploadFiles()
{
HomeController controller = new HomeController();

ViewResult result = controller.UploadFiles() as ViewResult;
var uploadedResult = result.ViewData.Model as List<ViewDataUploadFilesResult>;
Assert.AreEqual(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "foo.doc"), uploadedResult[0].Name);
Assert.AreEqual(8192, uploadedResult[0].Length);
}

This is incomplete, though, as I'm writing the tests before I the implementation exists. I need to think about how this should be implemented, and as I learn what should be mocked, I need to go back and forth between the tests and the implementation.

If we tried to compile this test, it won't, until I add a few types and methods. Once it actually compiles, but before I write the method itself, I'll want to see it FAIL. If you get a test to PASS on the first try, you don't really know yet if it CAN fail. Making it fail first proves that it's broken. Then you get to fix it.

Back To Basics Tip
Remember, in TDD, if it ain't broke, you don't get to fix it.

image

There's a bit of a chicken and the egg because it's unclear what will need to be mocked out until I start the implementation. However, this draft method above generally says what I want to do. I want to my controller to have a method called UploadFiles() that will grab the uploaded files from Request.Files, save them, then put a type in the ViewData saying which files were saved and how large they were.

Ok, take a breath. The following code may look freaky, but it's really cool actually. You can use any Mock Framework you like, but I like Moq for it's fluency.

We're having to "mock" things because we need to lie to our controller, who's expecting an HTTP Post, remember? It's going to go and spin through Request.Files and try to save each file. Since we want to test this without the web server or web browser, we'll want to tell the Moq framework about our expectations.

Back To Basics Tip
Be careful to mock context and assert outputs but don't mock away the whole test!

I've commented the code to explain...

[TestMethod]
public void FakeUploadFiles()
{
//We'll need mocks (fake) of Context, Request and a fake PostedFile
var request = new Mock<HttpRequestBase>();
var context = new Mock<HttpContextBase>();
var postedfile = new Mock<HttpPostedFileBase>();

//Someone is going to ask for Request.File and we'll need a mock (fake) of that.
var postedfilesKeyCollection = new Mock<HttpFileCollectionBase>();
var fakeFileKeys = new List<string>() { "file" };

//OK, Mock Framework! Expect if someone asks for .Request, you should return the Mock!
context.Expect(ctx => ctx.Request).Returns(request.Object);
//OK, Mock Framework! Expect if someone asks for .Files, you should return the Mock with fake keys!
request.Expect(req => req.Files).Returns(postedfilesKeyCollection.Object);

//OK, Mock Framework! Expect if someone starts foreach'ing their way over .Files, give them the fake strings instead!
postedfilesKeyCollection.Expect(keys => keys.GetEnumerator()).Returns(fakeFileKeys.GetEnumerator());

//OK, Mock Framework! Expect if someone asks for file you give them the fake!
postedfilesKeyCollection.Expect(keys => keys["file"]).Returns(postedfile.Object);

//OK, Mock Framework! Give back these values when asked, and I will want to Verify that these things happened
postedfile.Expect(f => f.ContentLength).Returns(8192).Verifiable();
postedfile.Expect(f => f.FileName).Returns("foo.doc").Verifiable();

//OK, Mock Framework! Someone is going to call SaveAs, but only once!
postedfile.Expect(f => f.SaveAs(It.IsAny<string>())).AtMostOnce().Verifiable();

HomeController controller = new HomeController();
//Set the controller's context to the mock! (fake)
controller.ControllerContext = new ControllerContext(context.Object, new RouteData(), controller);

//DO IT!
ViewResult result = controller.UploadFiles() as ViewResult;

//Now, go make sure that the Controller did its job
var uploadedResult = result.ViewData.Model as List<ViewDataUploadFilesResult>;
Assert.AreEqual(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "foo.doc"), uploadedResult[0].Name);
Assert.AreEqual(8192, uploadedResult[0].Length);

postedfile.Verify();
}

How to do File Upload with ASP.NET MVC

Now, what is the least amount of code in our Controller do we need to write to make this test pass? Here we get to use the Request.Files method that ASP.NET (not ASP.NET MVC) has had for years, and use it as advertised. It works in the tests and it works in production.

Important Note: We have to use the HttpPostedFileBase class, rather than the HttpPostedFile because every Request, Response, HttpContext and all related ASP.NET intrinsic abstractions are one layer farther way in ASP.NET MVC. If you get an HttpRequest in ASP.NET, then in ASP.NET MVC at runtime...

  • you'll get an HttpRequestWrapper while running under a Webserver
  • you'll get a dynamically generated derived Mock of an HttpRequestBase while running outside a Webserver (like inside a test) when you've made your own ControllerContext.

In each case, the instances you'll get are both (ultimately) of type HttpRequestBase, but it's this extra layer of abstraction that makes ASP.NET MVC easy to test and ASP.NET WebForms less so. I hope these Wrappers will be included in a future release of WebForms. The fact that they live in the System.Web.Abstractions.dll and not System.Web.Mvc.Abstractions.dll tells me someone has their eye on that particular ball.

At any rate, here's the Controller that takes File Upload requests:

public class ViewDataUploadFilesResult
{
public string Name { get; set; }
public int Length { get; set; }
}

public class HomeController : Controller
{
public ActionResult UploadFiles()
{
var r = new List<ViewDataUploadFilesResult>();

foreach (string file in Request.Files)
{
HttpPostedFileBase hpf = Request.Files[file] as HttpPostedFileBase;
if (hpf.ContentLength == 0)
continue;
string savedFileName = Path.Combine(
AppDomain.CurrentDomain.BaseDirectory,
Path.GetFileName(hpf.FileName));
hpf.SaveAs(savedFileName);

r.Add(new ViewDataUploadFilesResult()
{ Name = savedFileName,
Length = hpf.ContentLength });
}
return View("UploadedFiles",r);
}
}

At the bottom where I ask for the "UploadedFiles" view, and I pass in my list of ViewDataUploadFilesResults. This will appea in the ViewData.Model property. The View then displays them, and that's ALL the View does.

<ul>
<% foreach (ViewDataUploadFilesResult v in this.ViewData.Model) { %>
<%=String.Format("<li>Uploaded: {0} totalling {1} bytes.</li>",v.Name,v.Length) %>
<% } %>
</ul>

Conclusion

I always encourage people to take the little bit of time to use Fiddler or SysInternals or look at your call stack or just to take a breath and remind oneself, "so how is this supposed to work?" Otherwise, one is just cargo-cult programming.

This post was a long answer to the question "How do I do FileUpload with ASP.NET MVC?" but I feel better having written in this way.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Saturday, June 28, 2008 6:05:57 AM UTC
I think in the first sentence you mean "except using ASP.NET MVC" (note the missing MVC).
Brian Yarger
Saturday, June 28, 2008 8:33:32 AM UTC
Thanks, I fixed it!
Saturday, June 28, 2008 9:21:01 AM UTC
Hi Scott, Great post,

Of late what i have been coding is a Http module where by you can grab the input stream by bytes of the file upload. I.e. doing all the leg work youerself. There are commercial products out there but it really interested me how it was done. One of the trickiest parts was extracting the file data out of the multipart form data. What i did was, very roughly, port a class from the apache library called MultiPartStream and that did the trick.

A snippet from the code, and where you get the raw request is below.

HttpWorkerRequest request = (HttpWorkerRequest)application.Context.GetType().GetProperty("WorkerRequest", (BindingFlags)36).GetValue(application.Context, null);

// Only trigger if the request is of type 'multipart/form-data'
if (application.Context.Request.ContentType.IndexOf("multipart/form-data") > -1)
{

Another good one was figuring out how to identify boundaries inside the stream. I saw a lot of attempts on the net where people were convert the stream to string first, major performance hit. Anyway, great post. :-)

I will publish all my source on Andrew rea in the next week for this, as it will hopefully help out others who want to do this and may not be able to afford the commercial ones. :-)

Cheers,

Andrew
Saturday, June 28, 2008 11:15:40 AM UTC
Hi Scott - great post - any chance of a solution? (The mocking demo is very cool.)
Saturday, June 28, 2008 12:37:34 PM UTC
Great post.
I couldn't agree more on the "Back to basics" thing. Too many WebForms people knows nothing about HTTP and very little about HTML. It's like saying that using LinqToSql/Entities/NHibernate mean that one does not need to *know* SQL.

Anyway - I've posted a bit about how in MonoRail, a sent file can be easily bound to a HttpPostedFile action parameter, and how to use a custom databinder to allow testability
http://www.kenegozi.com/Blog/2008/06/28/filebinderattribute-to-ease-fileupload-in-monorail.aspx
Saturday, June 28, 2008 1:03:17 PM UTC
That's a good point Ken. HTTP is pretty important. In fact there is a W3C working group that is re-publishing RFC2616 to include errata, editorial improvements and a separation of the spec into parts - which should make the spec clearer and more accessible.

http://www.ietf.org/html.charters/httpbis-charter.html

Saturday, June 28, 2008 2:16:43 PM UTC
Scott,
Great post, as usual. And +1 on the necessity of knowing what your tools are doing for you. I love magic, but only when I know how the trick works...

But I'm really commenting in reply to your tweet on the Seattle Nerd Dinner on July 22nd. I plan to be there -- assuming "green badges" are also welcome. :) (I work for MSNBC.com in NYC and will be in Seattle that week to get some face time with my team.)

See you then.
-JB
Saturday, June 28, 2008 8:42:54 PM UTC
@Andrew - Great to hear - can't wait to see more on your custom module.

If you make it open besides being more affordable it will be something the community can build on and improve over time. Please keep us posted.

@Scott - good post. It was very helpful, although I can't help note the irony that a lot of us are waiting to move to a new architecture that gives us less functionality.

Not your fault and not the fault of the mvc design, just one of those things that requires up front investment. And yes I'm one of the people waiting for it :).

Sunday, June 29, 2008 3:03:01 AM UTC
Great post - thanks for being so detailed.
Sunday, June 29, 2008 11:28:04 AM UTC
I think you may also like to have a look at:
http://weblogs.asp.net/meligy/archive/2008/02/18/unit-test-friendly-file-upload-handling-in-n-tier-applications.aspx
Sunday, June 29, 2008 11:32:44 AM UTC
I was thinking about how to do file uploads in an ASP.net MVC website without using the upload control that ships with visual studio for sometime now. Thanks for your solution Scott.
Sunday, June 29, 2008 2:52:02 PM UTC
How about
public ActionResult UploadFiles([FileBinder]HttpPostedFile file)
{
}

or

public ActionResult UploadFiles([FileBinder]HttpPostedFile[] files)
{

}
Combines the usual way with MVC style.
I think MVC preview 3 doesn't support this but MvcContrib supports, so no problem!

and here is the way to do it in MvcContrib

public class HttpFileBinderAttribute : AbstractParameterBinderAttribute
{
public HttpFileBinderAttribute(string prefix):base(prefix)
{
}
public HttpFileBinderAttribute()
: base(null)
{
}

public override object Bind(Type targetType, string paramName, System.Web.Mvc.ControllerContext context)
{

if (targetType == typeof(HttpPostedFileBase))
{
return context.HttpContext.Request.Files[string.IsNullOrEmpty(this.Prefix) ? paramName : this.Prefix + paramName];
}
else
return null;
}
}
Tuna Toksoz
Monday, June 30, 2008 6:00:39 AM UTC
Is it possible to do a file upload with MVC (as show above) INCLUDING a nice UI that has the percentage bar, etc...

there's a number of those 'controls' available (I can drop some links if required) that do it .. but of course they are the old ways ... and we want to do it in the MVC way!

... so .. is it possible?

-PK-
PK
Monday, June 30, 2008 9:40:54 AM UTC
Thanks for this, Scott! I've already made a fileupload in MVC, but I didn't know how to write tests for it. Thanks for sharing!
Tuesday, July 01, 2008 8:23:02 AM UTC
Dugg for this snippet:

Path.Combine(AppDomain.CurrentDomain.BaseDirectory, Path.GetFileName(hpf.FileName));

I'll stop using String.Format() for this kind of thing now. Thanks!
Mike
Tuesday, July 01, 2008 9:19:17 PM UTC
Interesting - doesn't seem to work though. Testing fails due to an issue with GetEnumerator() in the foreach.
Brian
Wednesday, July 02, 2008 10:04:46 AM UTC
Thanks for the example. I changed to (IEnumerable) this.ViewData.Model in UploadedFiles or one can change in the codebehind. I am new to mvc so I have questions : I cannot make multiple selections of files while browsing, is it not possible ? Or is it missing a step where one browse, select, submit several times and save the filenames for later showing in uploaded files view ?
How to choose a directory and make a files listing ? It looks like the user has to select a file for the program to get hold of the directory?
Help wanted !
Lela
Thursday, July 03, 2008 7:31:17 AM UTC
Hey,

This is a bit off-topic but it kind of relates to code I see everywhere. People abusing the 'as' keyword. Would you mind taking a look at this post. And comment, becuase I can't really see why people are using the 'as' keyword all the time. It's not meant for code just anywhere.

Regards



Monday, July 07, 2008 2:59:47 PM UTC
You can check this long ago but relevant code post I used to also include name/value pair in the file upload; such as a user id/encrypted password or any other type of form field data. I have a newer version that does multiple files as well if anyone is interested.

http://www.c-sharpcorner.com/UploadFile/gregoryprentice/DotNetBugs12062005230632PM/DotNetBugs.aspx
Tuesday, July 08, 2008 12:36:58 PM UTC
Hey Scott,
you're using .Verifiable() on expectations, but you're later calling .VerifyAll(). That makes the former redundant, as it will verify everything, regardless of whether the expectations are verifiable or not.

If you want to verify only verifiable expectations, use .Verify() instead.

Cheers!
Tuesday, July 08, 2008 3:46:59 PM UTC
Daniel, thanks for setting me straight!
Friday, August 15, 2008 9:25:12 PM UTC
thanks again! We now have a beautiful, testable, object o file upload with concise, standard html. Love it!
Monday, August 25, 2008 1:40:57 PM UTC
pingback from Samiq Bits : Model-View-Controller: a reference to its basics


...

Just like Scott Hanselman wrote earlier on his blog,

It's always better, for me, to understand WHY and HOW something is happening. If you say "just because" or "whatever, you just add that, and it works" then I think that's sad

...

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.