Scott Hanselman

Proper benchmarking to diagnose and solve a .NET serialization bottleneck

February 20, '15 Comments [25] Posted in ASP.NET | Learning .NET | Open Source
Sponsored By

From's a few comments and disclaimers to start with. First, benchmarks are challenging. They are challenging to measure, but the real issue is that often we forget WHY we are benchmarking something. We'll take a complex multi-machine financial system and suddenly we're hyper-focused on a bunch of serialization code that we're convinced is THE problem. "If I can fix this serialization by writing a 10,000 iteration for loop and getting it down to x milliseconds, it'll be SMOOOOOOTH sailing."

Second, this isn't a benchmarking blog post. Don't point this blog post and say "see! Library X is better than library Y! And .NET is better than Java!" Instead, consider this a cautionary tale, and a series of general guidelines. I'm just using this anecdote to illustrate these points.

  • Are you 100% sure what you're measuring?
  • Have you run a profiler like the Visual Studio profiler, ANTS, or dotTrace?
  • Are you considering warm-up time? Throwing out outliers? Are your results statistically significant?
  • Are the libraries you're using optimized for your use case? Are you sure what your use case is?

A bad benchmark

A reader sent me a email recently with concern of serialization in .NET. They had read some very old blog posts from 2009 about perf that included charts and graphs and did some tests of their own. They were seeing serialization times (of tens of thousands of items) over 700ms and sizes nearly 2 megs. The tests included serialization of their typical data structures in both C# and Java across a number of different serialization libraries and techniques. Techniques included their company's custom serialization, .NET binary DataContract serialization, as well as JSON.NET. One serialization format was small (1.8Ms for a large structure) and one was fast (94ms) but there was no clear winner. This reader was at their wit's end and had decided, more or less, that .NET must not be up for the task.

To me, this benchmark didn't smell right. It wasn't clear what was being measured. It wasn't clear if it was being accurately measured, but more specifically, the overarching conclusion of ".NET is slow" wasn't reasonable given the data.

Hm. So .NET can't serialize a few tens of thousands of data items quickly? I know it can.

Related Links: Create benchmarks and results that have value and Responsible benchmarking by @Kellabyte

I am no expert, but I poked around at this code.

First: Are we measuring correctly?

The tests were using DateTime.UtcNow which isn't advisable.

startTime = DateTime.UtcNow;
resultData = TestSerialization(foo);
endTime = DateTime.UtcNow;

Do not use DateTime.Now or DateTime.Utc for measuring things where any kind of precision matters. DateTime doesn't have enough precision and is said to be accurate only to 30ms.

DateTime represents a date and a time. It's not a high-precision timer or Stopwatch.

As Eric Lippert says:

In short, "what time is it?" and "how long did that take?" are completely different questions; don't use a tool designed to answer one question to answer the other.

And as Raymond Chen says:

"Precision is not the same as accuracy. Accuracy is how close you are to the correct answer; precision is how much resolution you have for that answer."

So, we will use a Stopwatch when you need a stopwatch. In fact, before I switch the sample to Stopwatch I was getting numbers in milliseconds like 90,106,103,165,94, and after Stopwatch the results were 99,94,95,95,94. There's much less jitter.

Stopwatch sw = new Stopwatch();

// stuff


You might also want to pin your process to a single CPU core if you're trying to get an accurate throughput measurement. While it shouldn't matter and Stopwatch is using the Win32 QueryPerformanceCounter internally (the source for the .NET Stopwatch Class is here) there were some issues on old systems when you'd start on one proc and stop on another.

// One Core
var p = Process.GetCurrentProcess();
p.ProcessorAffinity = (IntPtr)1;

If you don't use Stopwatch, look for a  simple and well-thought-of benchmarking library.

Second: Doing the math

In the code sample I was given, about 10 lines of code were the thing being measured, and 735 lines were the "harness" to collect and display the data from the benchmark. Perhaps you've seen things like this before? It's fair to say that the benchmark can get lost in the harness.

Have a listen to my recent podcast with Matt Warren on "Performance as a Feature" and consider Matt's performance blog and be sure to pick up Ben Watson's recent Book called "Writing High Performance .NET Code".

Also note that Matt is currently exploring creating a mini-benchmark harness on GitHub. Matt's system is rather promising and would have a [Benchmark] attribute within a test.

Considering using an existing harness for small benchmarks. One is SimpleSpeedTester from Yan Cui. It makes nice tables and does a lot of the tedious work for you. Here's a screenshot I stole borrowed from Yan's blog.


Something a bit more advanced to explore is HdrHistogram, a library "designed for recoding histograms of value measurements in latency and performance sensitive applications." It's also on GitHub and includes Java, C, and C# implementations.


And seriously. Use a profiler.

Third: Have you run a profiler?

Use the Visual Studio Profiler, or get a trial of the Redgate ANTS Performance Profiler or the JetBrains dotTrace profiler.

Where is our application spending its time? Surprise I think we've all seen people write complex benchmarks and poke at a black box rather than simply running a profiler.

Visual Studio Profiler

Aside: Are there newer/better/understood ways to solve this?

This is my opinion, but I think it's a decent one and there's numbers to back it up. Some of the .NET serialization code is pretty old, written in 2003 or 2005 and may not be taking advantage of new techniques or knowledge. Plus, it's rather flexible "make it work for everyone" code, as opposed to very narrowly purposed code.

People have different serialization needs. You can't serialize something as XML and expect it to be small and tight. You likely can't serialize a structure as JSON and expect it to be as fast as a packed binary serializer.

Measure your code, consider your requirements, and step back and consider all options.

Fourth: Newer .NET Serializers to Consider

Now that I have a sense of what's happening and how to measure the timing, it was clear these serializers didn't meet this reader's goals. Some of are old, as I mentioned, so what other newer more sophisticated options exist?

There's two really nice specialized serializers to watch. They are Jil from Kevin Montrose, and protobuf-net from Marc Gravell. Both are extraordinary libraries, and protobuf-net's breadth of target framework scope and build system are a joy to behold. There are also other impressive serializers in including support for not only JSON, but also JSV and CSV in ServiceStack.NET.

Protobuf-net - protocol buffers for .NET

Protocol buffers are a data structure format from Google, and protobuf-net is a high performance .NET implementation of protocol buffers. Think if it like XML but smaller and faster. It also can serialize cross language. From their site:

Protocol buffers have many advantages over XML for serializing structured data. Protocol buffers:

  • are simpler
  • are 3 to 10 times smaller
  • are 20 to 100 times faster
  • are less ambiguous
  • generate data access classes that are easier to use programmatically

It was easy to add. There's lots of options and ways to decorate your data structures but in essence:

var r = ProtoBuf.Serializer.Deserialize<List<DataItem>>(memInStream);

The numbers I got with protobuf-net were exceptional and in this case packed the data tightly and quickly, taking just 49ms.

JIL - Json Serializer for .NET using Sigil

Jil s a Json serializer that is less flexible than and makes those small sacrifices in the name of raw speed. From their site:

Flexibility and "nice to have" features are explicitly discounted in the pursuit of speed.

It's also worth pointing out that some serializers work over the whole string in memory, while others like Json.NET and DataContractSerializer work over a stream. That means you'll want to consider the size of what you're serializing when choosing a library.

Jil is impressive in a number of ways but particularly in that it dynamically emits a custom serializer (much like the XmlSerializers of old)

Jil is trivial to use. It just worked. I plugged it in to this sample and it took my basic serialization times to 84ms.

result = Jil.JSON.Deserialize<Foo>(jsonData);

Conclusion: There's the thing about benchmarks. It depends.

What are you measuring? Why are you measuring it? Does the technique you're using handle your use case? Are you serializing one large object or thousands of small ones?

James Newton-King made this excellent point to me:

"[There's a] meta-problem around benchmarking. Micro-optimization and caring about performance when it doesn't matter is something devs are guilty of. Documentation, developer productivity, and flexibility are more important than a 100th of a millisecond."

In fact, James pointed out this old (but recently fixed) ASP.NET bug on Twitter. It's a performance bug that is significant, but was totally overshadowed by the time spent on the network.

Thanks to Marc Gravell and James Newton-King for their time helping with this post.

What are your benchmarking tips and tricks? Sound off in the comments!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

NuGet Package of the Week: A different take on ASP.NET MVC Forms with ChameleonForms

February 17, '15 Comments [32] Posted in ASP.NET MVC | NuGet | NuGetPOW | Open Source
Sponsored By

One of the nice things about any modular system (like ASP.NET) is the ability to swap out the parts you don't like. As the authors of ChameleonForms state, HTML forms is a pain. It's repetitive, it's repetitive, and it's boring. While ASP.NET MVC's Form Helpers help a lot, they felt that helper methods like Html.EditorForModel didn't go far enough or give you enough flexibility. ChameleonForms adds its own templating model and attempts to be as DRY as possible. It also takes a number of issues head on like better handling for drop-down lists and lists of radio buttons, and it even supports Twitter Bootstrap 3 to you can bang out HTML forms ASAP.

ChameleonForms also is a nice example of a tidy and well-run small open source project. They've got a public Trello backlog board, excellent documentation, a continuous integration build, a good example project, and of course, they're on NuGet. Check out the other projects that the folks in the "MRCollective" work on as well, as they've got their own GitHub organization.

NuGet Install ChameleonForms

Often ChameleonForms tries to use C# for the whole form, rather than switching back and forth from Div to Html Helper. For example:

@using (var f = Html.BeginChameleonForm()) {
using (var s = f.BeginSection("Signup for an account")) {
@s.FieldFor(m => m.FirstName)
@s.FieldFor(m => m.LastName)
@s.FieldFor(m => m.Mobile).Placeholder("04XX XXX XXX")
@s.FieldFor(m => m.LicenseAgreement).InlineLabel("I agree to the terms and conditions")
using (var n = f.BeginNavigation()) {

This is the whole form using usings for scoping, and it's nice and clean.  How about a comparison example? Here's standard ASP.NET MVC:

@using (Html.BeginForm())
<legend>A form</legend>
<dt>@Html.LabelFor(m => m.RequiredString, "Some string")</dt>
<dd>@Html.TextBoxFor(m => m.RequiredString) @Html.ValidationMessageFor(m => m.RequiredString)</dd>
<dt>@Html.LabelFor(m => m.SomeEnum)</dt>
<dd>@Html.DropDownListFor(m => m.SomeEnum, Enum.GetNames(typeof(SomeEnum)).Select(x => new SelectListItem {Text = ((SomeEnum)Enum.Parse(typeof(SomeEnum), x)).Humanize(), Value = x})) @Html.ValidationMessageFor(m => m.SomeEnum)</dd>
<dt>@Html.LabelFor(m => m.SomeCheckbox)</dt>
<dd>@Html.CheckBoxFor(m => m.SomeCheckbox) @Html.LabelFor(m => m.SomeCheckbox, "Are you sure?") @Html.ValidationMessageFor(m => m.SomeCheckbox)</dd>
<div class="form_navigation">
<input type="submit" value="Submit" />

And here is the same form with ChameleonForms.

@using (var f = Html.BeginChameleonForm()) {
using (var s = f.BeginSection("A form")) {
@s.FieldFor(m => m.RequiredString).Label("Some string")
@s.FieldFor(m => m.SomeEnum)
@s.FieldFor(m => m.SomeCheckbox).InlineLabel("Are you sure?")
using (var n = f.BeginNavigation()) {

But these are basic. How about something more complex? This one has a bunch of variety, a number overloads and customizations, as well as a FileUpload (note that the form is a Multipart form):

@using (var f = Html.BeginChameleonForm(method: FormMethod.Post, enctype: EncType.Multipart))
<p>@f.LabelFor(m => m.SomeCheckbox).Label("Are you ready for: ") @f.FieldElementFor(m => m.SomeCheckbox) @f.ValidationMessageFor(m => m.SomeCheckbox)</p>
<p>@f.FieldElementFor(m => m.RequiredStringField).TabIndex(4)</p>
using (var s = f.BeginSection("My Section!", InstructionalText(), new{@class = "aClass"}.ToHtmlAttributes()))
using (var ff = s.BeginFieldFor(m => m.RequiredStringField, Field.Configure().Attr("data-some-attr", "value").TabIndex(3)))
@ff.FieldFor(m => m.NestedField).Attr("data-attr1", "value").TabIndex(2)
@ff.FieldFor(m => m.SomeEnum).Attr("data-attr1", "value")
@ff.FieldFor(m => m.SomeEnum).Exclude(SomeEnum.SomeOtherValue)
@s.FieldFor(m => m.SomeCheckbox).AsDropDown()
using (var ss = s.BeginSection("Nested section"))
@ss.FieldFor(m => m.FileUpload).Attr("data-attr1", "value")
@s.FieldFor(m => m.RequiredStringField).OverrideFieldHtml(new MvcHtmlString("Custom html <b>she-yeah</b>!"))
@s.FieldFor(m => m.TextAreaField).Cols(60).Rows(5).Label("Some Label").AutoFocus().TabIndex(1)
@s.FieldFor(m => m.SomeCheckbox).InlineLabel("Some label").WithHint("Format: XXX")
@s.FieldFor(m => m.SomeCheckbox).AsRadioList().WithTrueAs("True").WithFalseAs("False")
@s.FieldFor(m => m.ListId)
@s.FieldFor(m => m.ListId).AsRadioList()
@s.FieldFor(m => m.SomeEnums)
@s.FieldFor(m => m.SomeEnumsList).AsRadioList()
@s.FieldFor(m => m.Decimal)
@s.FieldFor(m => m.Int).AsInputGroup().Append(".00").Prepend("$")
@s.FieldFor(m => m.DecimalWithFormatStringAttribute)
@s.FieldFor(m => m.NullableInt)
@s.FieldFor(m => m.Child.ChildField)
@s.FieldFor(m => m.Child.SomeEnum).AsRadioList()
@s.FieldFor(m => m.RequiredStringField).Disabled()
@s.FieldFor(m => m.RequiredStringField).Readonly()
using (var n = f.BeginNavigation())

ChameleonForms also has a special NuGet package if you're using TwitterBootstrap that changes how forms with the BeginChameleonForm method render.

ChameleonForms also has some convenient extra abilities, like being able to automatically infer/create a [DisplayName] so you don't have to. If you're doing Forms in English and your preferred Display Name will end up just being your variable name this can be a useful time saver (although you may have opinions about its purity.)

So instead of the tedium of:

[DisplayName("Email address")]
public string EmailAddress { get; set; }

[DisplayName("First name")]
public string FirstName { get; set; }

You can just say this once, picking just one...this is an example where they use HumanizedLabels.

HumanizedLabels.Register(LetterCasing.AllCaps) => "EMAIL ADDRESS"
HumanizedLabels.Register(LetterCasing.LowerCase) => "email address"
HumanizedLabels.Register(LetterCasing.Sentence) => "Email address"
HumanizedLabels.Register(LetterCasing.Title) => "Email Address"

If you've got a lot of Forms to create and they're just no fun anymore, you should definitely give ChameleonForms a try. If you're a Twitter Bootstrap shop, doubly so, as that's where ChameleonForms really shines.

I'll do a few other posts exploring different ways to for Forms in ASP.NET MVC in the coming weeks. Be sure to explore the NuGet Package of the Week Archives as well!

PLUG: Did you know I have a YouTube channel? Subscribe over here. I've got tutorials on how to effectively use Windows 8 and 8.1, Build to Build walkthroughs of the latest versions of Windows 10, and I just started a new series I'm sure you'll want to share with your family called "How to REALLY use Microsoft Office." Help me out and spread the word!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

JavaScript Has Won: Run Flash with Mozilla Shumway and Develop Silverlight in JS with Fayde

February 13, '15 Comments [54] Posted in Open Source | Silverlight
Sponsored By

Well this is some amazing news that has been a long time coming. You can take your Flash apps and run them without Flash, and take your Silverlight apps and (almost) run them without Silverlight.

If you still don't think JavaScript has won the web, please read on.

Shumway - Flash in JavaScript and HTML 5

The Mozilla Project "Shumway" has been a long time coming (as long ago as 2012, but it's now ready to be looked at more deeply).

Chris Peterson, a Program Manager on the Mozilla Shumway says:

"Shumway is in a race to stay relevant as Flash fades from the web, but there will always be a long-tail of Flash content that would/will be lost when Adobe or browsers stop supporting the Flash plugin."

Think about that. We've all largely got "Evergreen Browsers" now that update themselves as often as weekly, but sometimes it feels like Adobe Flash is being attacked daily, so we're told to update that as well. Flash itself has more than fallen from grace, as Chris points out, it's fading from the web itself. Fast forward a year or so when there is no more Flash installed, but there's still Flash on the web. Enter Shumway - it's a renderer for SWF (Flash files) without native code! Shumway literally.

Why is it called Shumway? Again, Chris:

"The name "Shumway" is derived from "Gordon Shumway", the actual name of the TV character ALF: Flash -> Flash Gordon -> Gordon Shumway -> Shumway."

That's awesome. What else is awesome? "Shumway is written in TypeScript. It has an ActionScript interpreter and a JIT that generates JavaScript, compiled using eval()."

Shumway makes Flash into HTML5

So Shumway is an HTML experiment that uses TypeScript (a modern typed JavaScript compiler/transpiler) to read ActionScript and resources and JIT the result into evaluated JavaScript. Fantastic. It's also open source and on GitHub. Even better, the Firefox Nightly is using Shumway for Flash videos on This is the beginning of their test, I presume, to sunset Flash in Firefox.

Fayde - Silverlight in JavaScript and HTML5 Canvas

At the same time, there's The Fayde project. Also Open Source, Fayde also compiles to JavaScript. But Fayde transforms Silverlight into HTML5 Canvas and JavaScript! It's an implementation of a XAML engine in JS. Here's a near-Silverlight implementation of the classic Todo application, expressed on the web without plugins. Not enough? Here's a complex Fantasy Football app written in a Silverlight-like environment but running in your browser, again, without Silverlight.

Fayde - Silverlight in HTML5

To be clear, there are significant architectural differences between these two projects. Shumway reads the binary SWF format and tries to Be Flash, while Fayde is reimagining, if you will, that takes the Silverlight concepts of ViewModels and Views in XAML and adds TypeScript (a comfortable language for the C#-familiar) with the result rendered on a Canvas. It's not a Silverlight Emulator, it's a Silverlight-like implementation and app development pattern for HTML5. It's open source, and they are looking for participation, so head over and get involved! Fayde may be the Silverlight migration strategy you've been looking for.

Aside: From my perspective, it's not unreasonable to imagine taking something like JSIL (listen to my podcast on this amazing project) or a similar IL->JS system and combine it with Fayde to somehow run XAPs as well.

I love how crazy JavaScript is and what people have been able to do with it. Now, go run this PC Emulator (~virtual machine) written by Fabrice Bellard in JavaScript. It's Linux, on a 486, in your browser, in JavaScript. *brain explodes*

Have a lovely day and enjoy today's Open Web.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Announcing: Running Ruby on Rails on IIS8 (or anything else, really) with the new HttpPlatformHandler

February 9, '15 Comments [22] Posted in IIS | Ruby
Sponsored By

For years there's been numerous hacks and ways to get Ruby on Rails to run on IIS. There's also ways to get Java via Tomcat or Jetty, Go, and other languages and environments to run as well. There's ways to get Node.JS running on IIS using iisnode but that's been node-specific. The blog posts you do find say things like "get Rails to run on IIS in 10 steps" and I'm like JUST TEN?!? Why not 13? Others say "You can deploy Rails under IIS, it's just very difficult and there's not a lot of documentation. You'll need a special Fast-CGI implementation...WELCOME TO HELL."

No longer.

Azure Websites has supported node AND Java (again, Tomcat or Jetty) for a while now, in production and it's very nice and it runs under IIS. How? They've now brought that support to Windows running IIS 8+ with the release of the HttpPlatformHandler. Here's their example on how to get IIS 8+ running Java, easily.

Let's see if this works well with Ruby on Rails as well!

Why is HttpPlatformHandler interesting? Check this from the means IIS can host anything that runs on Windows now, easily. These things were possible before, but with all kinds of hacks and FastCGI this and that. What's great about HttpPlatformHandler is that it isn't about Rails. It's about any process that's listening on a port. You get all the value of IIS *and* total control of your self-hosting scenario.

The HttpPlatformHandler is an IIS Module, for IIS 8+, which does the following two things:

  1. Process management of http listeners - this could be any process that can listen on a port for http requests.  For example - Tomcat, Jetty, Node.exe, Ruby etc;
  2. Proxy requests to the process that it manages.

To be clear, you can work with Ruby on Rails on Windows and have it host itself with WEBrick locally, but if you're going to go production on Windows you'll want to have IIS and more likely, jRuby in a Tomcat container, similar to using Nginx on linux. What value does IIS provide in a scenario like this? Static file hosting, Reverse Proxy, complex auth that can span multiple apps, languages and frameworks, it monitors and manages your process looking at memory and CPU, crashes, etc.

Running Ruby on Rails on IIS 8 with the HttpPlatformHandler

First make sure you have Ruby on Rails. If you do, skip forward.

I use the for Windows and go. You'll get Ruby, Rails, Bundler, Sqlite, and TinyTDS. Even SQL Server support. Very kind of them. Another good Rails on Windows on is RailsFTW.

I go to Turn Windows Features On and Off to make sure I have IIS installed as well.

Turn on IIS in Windows Features

Then get the HttpPlatformHandler. You can get it with the Web Platform Installer, or just install it from here: x86/x64

I make a folder for the app I'm going to make. I put it in c:\inetpub\wwwroot\rails but you can move it around if you like.

Make sure your Rails app is an application in IIS

I right-click my folder in IIS Manager and "Convert to Application."

My Rails application in IIS

I run "gem install rails" to make sure I have Rails in the first place. ;) You will if you installed with the RailsInstaller. If you installed with the RubyInstaller, then this will get you Rails.

NOTE: If you have issues with SSL running gem on Windows, you'll need manually to update gem to 2.2.3 as of the time of this writing. I'm not sure why this isn't already done by the installer. The symptom I saw was weird errors on 'bundle install' that was fixed by this.

Installing Rails

Then, from inside c:\inetpub\wwwroot\rails, I ran "rails new helloworld." I ended up moving this folder up. I should have just made the app first, then converted the folder to an app in IIS. Order of operations and all that, eh?

OK, now I'll "rails server" from within c:\inetpub\wwwroot\rails, just to make sure Rails can run under the local WEBrick server. And it does:

Rails on Windows

Now, let's do it under IIS.

I need to make sure there's a web.config file in the same root folder as my Rails app. WHAT?!? Web.config is for ASP.NET, right? Well, no. It's config for any IIS application. You'll need this for Go, Java, PHP, Rails, node, ASP.NET, whatever. IIS can host basically anything.

Lemme add a hello world controller and edit its view. I'll "rails generate controller welcome index" then edit app\views\welcome\index.html.erb for good measure.

I put my Rails app under http://localhost/rails rather than at the root http://localhost so I did need to tell Rails 4 about the fact it's running in a subdirectory with a change to /, otherwise my routes wouldn't line up.

Rails.application.config.relative_url_root = '/rails'

map Rails.application.config.relative_url_root || "/" do
  run Rails.application

Make special note of the paths below AND the encoded &quot; there in the arguments. That's important, because it's a quoted argument passed into the ruby.exe process that IIS will kick off. Note also the %HTTP_PLATFORM_PORT% environment variable reference. That is passed in by IIS and will be a localhost-bound high port.

I also put in foo and bar for theoretical environment variables you might want your Rails app to know about. For example, I might add:

<environmentVariable name="RAILS_ENV" value="production"/>

...when it's time. I put in some standard debug logging there with the "stdout" but you can remove that if you don't want the clutter. Make sure your IISR_ users have write access to the folders if you want to see any logs.

WARNING/DISCLAIMER: This first example is just showing you what's possible. You DON'T want to go to production with the little built-in Ruby WEBrick web server. As Fabio Akita very kindly points out in the comments, and I'll pull his comment out here:

"One thing to be careful with the example using Rails. When running Ruby for Windows, when you run "rails server" it's going to spawn a single process that's mono-threaded, meaning that it can only respond to 1 request at a time. If IIS starts receiving too many requests simultaneously and each request is slow, it's going to generate a long queue until they start timing out.
In Linux we put NGINX to reverse-proxy HTTP requests to Unicorn, or Puma, or Rainbows, or Passenger. They all coordinate multiple Ruby processes (which in Unix, is very cheap as each process reuses memory from it's parent process upon forking - copy-on-write memory). So we can handle simultaneous requests."

You'd want to use JRuby and Tomcat with Puma under IIS for production on Windows.

KEEP SCROLLING FOR EXAMPLES USING JRuby, Trinidad/Tomcat, and Puma! They are farther down the page.

Also, on slower machines when running in development, you might need to up your startupTimeLimit if you are seeing IIS stop your Ruby processes that take to long to startup.

<?xml version="1.0" encoding="UTF-8"?>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
<httpPlatform stdoutLogEnabled="true" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="c:\RailsInstaller\Ruby2.1.0\bin\ruby.exe"
arguments="&quot;C:\RailsInstaller\Ruby2.1.0\bin\rails&quot; server -p %HTTP_PLATFORM_PORT% -b">
<environmentVariable name="foo" value="bar"/>

Here's a screenshot of Ruby within the SysInternals Process Explorer application. I wanted to show you this so you could see the Process Tree and see who started which process. You can see w3wp (that's IIS) which is a Service, and it's hosting Ruby, running Rails. Make note of the command line arguments as well.


And here it is. Ruby on Rails 4 running under IIS8 on my Windows 8 machine.

Ruby on Rails on IIS on Windows

Big thanks to Ranjith Ramachandra (@ranjithtweets) and Andrew Westgarth (@apwestgarth) at Microsoft for the help with the web.config values!


So, basically, to give you the TL;DR version, except at the end. When you have IIS, install HttpPlatformHandler and add a web.config as appropriate and you're all set. Run what you like, passing in the port that IIS will proxy to.

UPDATE: Puma and Trinidad (with Tomcat) on IIS

As pointed out in the comments, it's silly to use WEBrick in Production. Don't'.

I'm told JRuby is the way to go for prod. I was able to install JRuby and both Trinidad (with Tomcat) and Puma and get my HelloWorld running under IIS in an hour.

Here's Trinidad (I'm told Trinidad is out of vogue, however). I did a "JRuby -S gem install trinidad" and was on my way.

Note the JAVA_HOME environment variable setting. I also had to update some security policy files due to a "Illegal key size" error which was a Javaism. Otherwise, it just worked once the paths lined up. If you see problem, it'll be path related or you'll be loading the wrong Java version. Also, experiment but you don't really need to mess with processesPerApplications. If you create a lot of individual processes, Java can take up a lot of memory (300megs or so) per process and you can thrash your system.y

You'll get 10x the perf when running under production, so note I'm explicitly running in the prod environment.

<?xml version="1.0" encoding="UTF-8"?>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
<httpPlatform stdoutLogEnabled="false" processesPerApplication="1" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="C:\jruby-1.7.19\bin\jruby.exe"
arguments="-S trinidad --context /jRubyonRails --env production --dir C:\inetpub\wwwroot\jRubyonRails -p %HTTP_PLATFORM_PORT% ">
<environmentVariable name="JAVA_HOME" value="C:\Program Files\Java\jre1.8.0_31"/>

And it works. I was easily able to get 1600+ req/sec on my laptop with minimal effort. I'm sure I could do better with some tuning and a better machine.

JRuby on Rails using Trinidad behind IIS8 with the HttpPlatformHandler

OK, let's swap out Trindad/Tomcat for Puma. I put Puma in the Gemfile and did "bundle exec puma" to test. That worked. Now I need to hook it into IIS. I also had to add gem 'jruby-openssl', :require => false to  my Gemfile to avoid some weird errors.

Basically it was the same thing, as IIS is the reverse proxy and process manager in this case.

<?xml version="1.0" encoding="UTF-8"?>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
<httpPlatform stdoutLogEnabled="false" processesPerApplication="1" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="C:\jruby-1.7.19\bin\jruby.exe"
arguments="-S puma --env production --dir C:\inetpub\wwwroot\jRubyonRails -p %HTTP_PLATFORM_PORT% ">
<environmentVariable name="JAVA_HOME" value="C:\Program Files\Java\jre1.8.0_31"/>

I'm using just 1 processPerApplication and I'm getting 1500 req/s easily. I don't know anything about Puma or JRuby but I assume it can do better if I knew how to tune it.

Load testing JRuby on Windows under IIS

I tested without IIS against Puma itself and saw essentially the same results. IIS has minimal overhead that I can't measure in this case and you get its process management and other benefits. If you're having issues with perf doing this kind of stuff, it's unlikely it will be IIS.

JRuby on Rails using Puma with the HttpPlatformHandler

All in all, the HttpPlatformHandler just maybe the reverse proxy you've always wanted for Windows!

Related Links

Sponsor: Big thanks to Infragistics for sponsoring the feed this week! Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid – Download for free now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

3D Printer Shootout - $600 Printrbot vs. $20,000 uPrint SE Plus

February 6, '15 Comments [35] Posted in 3D Printing
Sponsored By

First, let's level set with a disclaimer. I am a smart enough person but a total beginner at 3D printing. I've been 3D printing for about two weeks using a Printrbot Simple Metal from Amazon that I got for US$599. Other than Amazon affiliate links (buy from my links and you buy me tacos!) I don't have ANY relationship with Printrbot or anyone in the 3D Printing space. I'm unaffiliated. This "shootout" is an interesting experience. I'm sure I did something wrong, or perhaps my partner in crime on this experiment, Brandon Potter, missed a step. Who knows? But this is how it went down.

Brandon saw that I was doing some 3D printing and mentioned they had a <$22,000 3D printer at his work and what did I think about doing a comparison? I had been having trouble printing with success, ruining at least 2 out of 3 prints, but recently got my Printrbot dialed in nicely. '

Here's my investment so far both in time and money.

So I'm into it for $722 and my time. I did NOT use a Heated Bed for my Printrbot Simple Metal but I am told it's a great upgrade.

Here's Brandon's company's outlay in his words:

  • uPrint SE Pro Printer and Dissolving Bath – about $22,000
  • 1 Spool of Model Material (Black) – $205.00 (produces 42 cubic inches of printing)
  • 1 Spool of Support Material – $200.00 (42 cubic inches worth)
  • Box of Build Plates – $125.00 for 24 (you need one for each print, so it costs about $5.20/each)
  • Soluble Concentrate – $149.00 for 12 bottles (dissolves support material, aka fancy drain-O)
  • Warranty Support – $2,000/year – because it does break from time to time.
  • Brandon says - Add a little bit of shipping, and for a mere $25K you’re ready to print your very own coffee cup.

We decided to print a coffee cup. I don't plan on drinking from it because who knows if the plastic will leech off, but it seemed like an interesting and common object. As a 3D model it has some nice curves, the handle overhang is a small challenge and it's something we can easily compare.

I printed one on my Printrbot and Brandon printed one on their Stratasys uPrint SE Pro. Then Brandon mailed me his cup and I'm sitting here holding them both in my hands.

Here is a timelapse video of my cup printing. My 0.2mm print took about 7 hours and this video is 90 seconds. I used Octoprint and the Raspberry Pi to create the video.

I downloaded the Coffee Cup model from Thingiverse as an STL 3D model. I ran the STL model through the Microsoft 3D Printing NetFabb online service to make sure the model was watertight. Then I loaded the result directly into Repetier and used Cura to slice the model into individual instructions for the printer. I made one adjustment to the slicing settings, telling it to make a "support structure" when it sees an overhang over 30 degrees. I was concerned about the cup's handle. Other than that one settings change which resulted in the support being added (which is easily pulled off after the print) it was a "printing as is." Brandon's model also used a support structure to print the cup and keep the handle stable so we decided this was pretty typical. He also printed on his pro printer with no changes.

My 3D model with support

Here is a picture of my setup. There's my printer on the right with some stuff I've made and my box of tools/crap on the left. You can see the spool, the printer (the red lipstick looking thing is where the hot plastic comes out), the Raspberry Pi and the tape that I print on. I can print objects up to 6 inches on a side, so a big 6 inch cube. If I paid more money I could get the 10 inch model, but so far for my hobbies I'm happy with that size. I can break things up and assemble them with acetone or bolts if I want to make larger models. One fellow 3D-printed a whole full-sized motorcycle with a small printer and assembled the parts.

My 3D Printing setup with Octoprint and a Printrbot Simple Metal

Here is Brandon's printer. Again, $600 vs. $25000.


By the way, is this a silly comparison? Perhaps. But no more than a Tesla vs. a Challenger Hellcat. Actually, I think my printer is a Geo Metro. But I get great mileage!

Ok, here's the prints. Feel free to click to zoom in to make your decision. Which cup came from which printer?

Coffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the otherCoffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the other

Coffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the otherCoffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the other

Here's slightly larger pics. Note I am not/have not moved the cups side to side. Left/right is consistent for this photo series.

Coffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the other

Coffee Cup 3D Print from a Printrbot Simple Metal on one side and a Stratasys uPrint SE Pro on the other

Which is which?

The $599 Printrbot Simple Model printing at 0.2mm is on the left. The $20k uPrint SE Plus printing at 0.1mm is on the right.

The uPrint's base was messed up somehow. A bad start perhaps, but unfortunate as it's supposed to just work since Brandon used the one-time-use custom bases that the uPrint comes with. He also used the dissolving bath to take the white support structure off. You can see some of the white still there, perhaps it wasn't in long enough. My print had supports but I just pulled them off with my hands. Of course, if you have an intricate print with small parts that's not always possible. This is a very simple object, to be clear. It's not interlocking gears or something.

Here's the uPrint cup. Brandon said it took 8 hours and 22 minutes.

Coffee Cup 3D Print from a Stratasys uPrint SE Pro

Here's the Printrbot's cup. Note the "Z scar" (an artifact from the printer moving up the Z-Axis) on the print? On my print it ended up by the handle, but on Brandon's it ended up on the side of the cup, marring the print, in my opinion.

Coffee Cup 3D Print from a Printrbot Simple Metal

What's the takeaway? If we assume that I have a totally dialed-in well calibrated super cheap consumer/hobbyist 3D Printer and that Brandon has a $20k professional 3D Printer that's maybe got some calibration issues, they seem very comparable.

However, in real one-time costs my cup cost me 21.02 meters of filament, costing me perhaps $2 maybe a little more if you count the few pieces of tape. For Brandon and his Pro printer, in direct costs, he used $23.62 in model material, $2.06 in support material, and $5.20 build plate, for a total of $30.88 for this cup.

Naysayers will say that this isn't a great model to have chosen. Clearly my little Printrbot Simple Metal can only make things of a certain size, but it's clearly fair to say that it's a surprisingly competent printer when it's calibrated well. Additionally, if I were a 3D modeling shop with a Pro printer, I think I would definitely pick up a few sub-$1000 printers for basic stuff as the big printers may be costing you money. If I was a small business I would really do my homework before buying a pro printer that claims to be problem- and maintenance-free. Go read Brandon Potter's account of his "pro print" and compare notes!

NOTE: This post is intended in no way to talk up Printrbot nor to talk down Stratasys. It's a simple one time anecdote as it happened, using a simple model.

What do you think, Dear Reader?

Sponsor: Big thanks to Infragistics for sponsoring the feed this week! Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid – Download for free now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.