Scott Hanselman

Running the Ruby Middleman Static Site Generator on Microsoft Azure

February 24, '15 Comments [16] Posted in Azure | Open Source | Ruby
Sponsored By

Middleman is "a static site generator using all the shortcuts and tools in modern web development." With any static site generator you can run it all locally and then push/FTP/whatever the resulting HTML to any host. However, static site generators are even more fun when you can host the source code in Git and have your static site build and deploy in the cloud.

Middleman uses Ruby for its build system and views, and some of the Gems it uses are native gems. That means if you are a Windows user, your system will need not just Ruby, but the Ruby DevKit so you can build those native gems. The DevKit is a lovely set of tools that "makes it easy to build and use native C/C++ extensions such as RDiscount and RedCloth for Ruby on Windows."

Azure Websites supports not just ASP.NET today, but also node.js, PHP, Python, and Java, all built in. But not Ruby, directly, yet. Also, Azure Websites doesn't know formally about the idea of a static site generator. You might be thinking, oh, this'll be hard, I'll need to use a VM and do this myself.

However, even though Azure Websites are totally "platform as a service" there's still a Windows Virtual Machine underneath, and you can use the disk space however you like. You've got a LOT of control and can even get a hold of a console where you can run commands and install stuff. The Azure Portal lets you open a command line from your website.

The New Azure Portal

Check me out, here in the new Azure Portal. This is where I did my practice work to see if I could programmatically download and install Ruby via a script. I tried a number of different commands, all from the browser, and explored a number of ideas. When I got it working, I put together a batch file called GetRuby. I could have also used a shell script or PowerShell, but Batch was easy given what I was doing.

ASIDE: You may recognize that console from this video I did about the "Super Secret Debug Console" in Azure. It's not so secret now, it's a feature.  There is still a cool debug "sidecar" website for every Azure site, it's at http://YOURSITENAME.scm.azurewebsites.net/DebugConsole but now a version of the console is in the portal as well.

Azure Websites uses an open source project called Kudu to deploy from locations with source like Github. Kudu supports custom deployment scripts where you can jump in and do whatever you like (within the limits of the security sandbox)

Basically I needed to do these things before running Middleman on my source:

  • Ensure Ruby is installed and in the path.
  • Ensure the DevKit (which includes native compilers, etc) is installed
  • Initialize and setup DevKit for builds
  • Update RubyGems to 2.2.3 until the Windows version of Ruby has this included
  • Install eventmachine 1.0.7, a problematic gem on Windows
  • Run the Ruby Bundler's update
  • Install Middleman

And then, every deployment run the Middleman static site generator.

  • Middleman build

The first part is a one time thing for a new website. I just need to make sure Ruby is around and in the path. The second part is what runs every time a source file for my static site generator is checked in. It runs middleman build. Then at the very end, Kudu takes the results from the /build folder and moves them to /wwwroot, which makes the changes live.

Here's an annotated part of the first bit, but the actual file is on GitHub. Note that I'm putting stuff in %temp% for speed. Turns out %temp% a local drive, so it's a few times faster than using the main drive, which makes this deployment faster. However, it's cleared out often, so if I wanted things to be persistent but slower to deploy, I'd put them in D:\deployments\tools. As it is, the deploy is fast (less than a minute) when Ruby is there, and just about 3 minutes to get and setup Ruby when it's not. The exists check handles the case when a deploy happens but %temp% has been cleared so it'll get Ruby again.

NOTE: If this seems confusing or complex, it's because I like to give folks LOTS of detail. But just look at my repository. All we have is a standard "Middleman init" site plus the Azure-generator deploy.cmd and my getruby.cmd. That's all you need, plus a Basic Azure Website. The getruby.cmd is my automating what you'd have to any way on a Windows machine without Ruby.

REM Note that D:\local\temp is a LOCAL drive on Azure, and very fast
SET PATH=%PATH%;D:\local\temp\r\ruby-2.1.5-x64-mingw32\bin

pushd %temp%
REM If you need things to be persistent, then put them elsewhere, not in TEMP
if not exist r md r
cd r
if exist ruby-2.1.5-x64-mingw32 goto end

echo No Ruby, need to get it!

REM Get 64-bit Ruby
curl -o ruby215.zip http://dl.bintray.com/oneclick/rubyinstaller/ruby-2.1.5-x64-mingw32.7z?direct
ECHO START Unzipping Ruby. 7Zip is already on Azure Websites
REM Note Azure deployments run faster with 7Zip not spewing so much. Redirect to a file.
d:\7zip\7za x -y ruby215.zip > out

REM Get DevKit to build Ruby native gems
REM If you don't need DevKit for your Gems, rem this out.
curl -o DevKit.zip http://cdn.rubyinstaller.org/archives/devkits/DevKit-mingw64-64-4.7.2-20130224-1432-sfx.exe
ECHO START Unzipping DevKit
d:\7zip\7za x -y -oDevKit DevKit.zip > out
ECHO DONE Unzipping DevKit

ruby DevKit\dk.rb init

REM Tell DevKit where Ruby is
echo --- > config.yml
echo - d:/local/temp/r/ruby-2.1.5-x64-mingw32 >> config.yml

REM Setup DevKit
ruby DevKit\dk.rb install

REM Update Gem223 until someone fixes the Ruby Windows installer https://github.com/oneclick/rubyinstaller/issues/261
curl -L -o update.gem https://github.com/rubygems/rubygems/releases/download/v2.2.3/rubygems-update-2.2.3.gem
call gem install --local update.gem
call update_rubygems --no-ri --no-rdoc > updaterubygemsout
ECHO What's our new Rubygems version?
call gem --version
call gem uninstall rubygems-update -x

REM This is needed on Windows, why is this gem such a problem?
ECHO Install eventmachine 1.0.7
call gem install eventmachine -v '1.0.7' --no-ri --no-rdoc > updateventmachineout

call bundle update

ECHO Install middleman...the whole point!
call gem install middleman --no-ri --no-rdoc

:end
popd

call middleman build

REM KuduSync and actual /build to /wwwroot is after this in deploy.cmd!

And in the Deploy.cmd all I needed to change was this under SETUP. This is where YOU can do whatever you like. Note since I'm using Batch, I need to put CALL in front of other Batch files (and Ruby uses them also!) otherwise my script will just end early.

ECHO CALLING GET RUBY

call getruby.cmd

ECHO WE MADE IT

Then later, still in Deploy.cmd, I just added \build to the source directory name.

call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_SOURCE%\build" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"

And that's it.  Now whenever I updated my views or other things in my Middleman source on GitHub, it automatically deploys to my live site.

Yes, again, to be clear, I realize it's a static site generator that I could run locally and FTP the results, but I'm working in a small team and this is a great way for us to collaborate on our static site. Plus, when it's done, it's all done and I don't have to mess with it again.

Middleman Static Site Generator on Azure

Debugging Custom Azure Website Deployments

I thought debugging my GetRuby batch file was going to be a nightmare. However, it turns out that the Azure cross-platform command line (the Azure x-plat CLI, open source, and written in nodejs, BTW) can connect to Azure's log streaming service. "Azure Site Log Tail" lets me see the LIVE console output as the deploy happens!

Azure Site Log Tail

Now, note that the need for this whole "getruby.bat" file totally goes away if the Azure Websites folks start including Ruby and DevKit in the Azure Websites VM image by default. That would make Ruby, Rails, Gems, DevKit, etc. available to everyone. Do you want Ruby on Azure? Do you care? Sound off in the comments!

HELP: The batch file could use more testing, especially for robustness as well as Ruby-correctness as I'm likely confused about a few things, but it works for me and it's a great start. Sometimes different native gems don't build, though, or Gems complains about conflicting versions and asks me to run Bundler. I have no idea why. Running things twice clears it. It's either my bug or someone else's. :)

I'm just happy that Azure Websites is as flexible as it is that I was able to console into it from the browser, look around, add my own custom deployment hook, and do something I initially didn't think was possible!

Give Azure Websites a try FOR FREE, no signup, no credit card for an hour in a sandbox with PHP, Node, ASP.NET, or Java at http://try.azurewebsites.net. (Full Disclosure, I helped a little with this site, so I'm a fan.)

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Proper benchmarking to diagnose and solve a .NET serialization bottleneck

February 20, '15 Comments [23] Posted in ASP.NET | Learning .NET | Open Source
Sponsored By

From http://adrianotto.com/2010/08/dev-null-unlimited-scale/Here's a few comments and disclaimers to start with. First, benchmarks are challenging. They are challenging to measure, but the real issue is that often we forget WHY we are benchmarking something. We'll take a complex multi-machine financial system and suddenly we're hyper-focused on a bunch of serialization code that we're convinced is THE problem. "If I can fix this serialization by writing a 10,000 iteration for loop and getting it down to x milliseconds, it'll be SMOOOOOOTH sailing."

Second, this isn't a benchmarking blog post. Don't point this blog post and say "see! Library X is better than library Y! And .NET is better than Java!" Instead, consider this a cautionary tale, and a series of general guidelines. I'm just using this anecdote to illustrate these points.

  • Are you 100% sure what you're measuring?
  • Have you run a profiler like the Visual Studio profiler or DotTrace?
  • Are you considering warm-up time? Throwing out outliers? Are your results statistically significant?
  • Are the libraries you're using optimized for your use case? Are you sure what your use case is?

A bad benchmark

A reader sent me a email recently with concern of serialization in .NET. They had read some very old blog posts from 2009 about perf that included charts and graphs and did some tests of their own. They were seeing serialization names (of tens of thousands of items) over 700ms and sizes nearly 2 megs. The tests included serialization of their typical data structures in both C# and Java across a number of different serialization libraries and techniques. Techniques included their company's custom serialization, .NET binary DataContract serialization, as well as JSON.NET. One serialization format was small (1.8Ms for a large structure) and one was fast (94ms) but there was no clear winner. This reader was at their wit's end and had decided, more or less, that .NET must not be up for the task.

To me, this benchmark didn't smell right. It wasn't clear what was being measured. It wasn't clear if it was being accurately measured, but more specifically, the overarching conclusion of ".NET is slow" wasn't reasonable given the data.

Hm. So .NET can't serialize a few tens of thousands of data items quickly? I know it can.

Related Links: Create benchmarks and results that have value and Responsible benchmarking by @Kellabyte

I am no expert, but I poked around at this code.

First: Are we measuring correctly?

The tests were using DateTime.UtcNow which isn't advisable.

startTime = DateTime.UtcNow;
resultData = TestSerialization(foo);
endTime = DateTime.UtcNow;

Do not use DateTime.Now or DateTime.Utc for measuring things where any kind of precision matters. DateTime doesn't have enough precision and is said to be accurate only to 30ms.

DateTime represents a date and a time. It's not a high-precision timer or Stopwatch.

As Eric Lippert says:

In short, "what time is it?" and "how long did that take?" are completely different questions; don't use a tool designed to answer one question to answer the other.

And as Raymond Chen says:

"Precision is not the same as accuracy. Accuracy is how close you are to the correct answer; precision is how much resolution you have for that answer."

So, we will use a Stopwatch when you need a stopwatch. In fact, before I switch the sample to Stopwatch I was getting numbers in milliseconds like 90,106,103,165,94, and after Stopwatch the results were 99,94,95,95,94. There's much less jitter.

Stopwatch sw = new Stopwatch();
sw.Start();

// stuff

sw.Stop();

You might also want to pin your process to a single CPU core if you're trying to get an accurate throughput measurement. While it shouldn't matter and Stopwatch is using the Win32 QueryPerformanceCounter internally (the source for the .NET Stopwatch Class is here) there were some issues on old systems when you'd start on one proc and stop on another.

// One Core
var p = Process.GetCurrentProcess();
p.ProcessorAffinity = (IntPtr)1;

If you don't use Stopwatch, look for a  simple and well-thought-of benchmarking library.

Second: Doing the math

In the code sample I was given, about 10 lines of code were the thing being measured, and 735 lines were the "harness" to collect and display the data from the benchmark. Perhaps you've seen things like this before? It's fair to say that the benchmark can get lost in the harness.

Have a listen to my recent podcast with Matt Warren on "Performance as a Feature" and consider Matt's performance blog and be sure to pick up Ben Watson's recent Book called "Writing High Performance .NET Code".

Also note that Matt is currently exploring creating a mini-benchmark harness on GitHub. Matt's system is rather promising and would have a [Benchmark] attribute within a test.

Considering using an existing harness for small benchmarks. One is SimpleSpeedTester from Yan Cui. It makes nice tables and does a lot of the tedious work for you. Here's a screenshot I stole borrowed from Yan's blog.

image11

Something a bit more advanced to explore is HdrHistogram, a library "designed for recoding histograms of value measurements in latency and performance sensitive applications." It's also on GitHub and includes Java, C, and C# implementations.

PercentileHistogramExample

And seriously. Use a profiler.

Third: Have you run a profiler?

Use the Visual Studio Profiler, or get a trial of the Redgate ANTS Performance Profiler or the JetBrains dotTrace profiler.

Where is our application spending its time? Surprise I think we've all seen people write complex benchmarks and poke at a black box rather than simply running a profiler.

Visual Studio Profiler

Aside: Are there newer/better/understood ways to solve this?

This is my opinion, but I think it's a decent one and there's numbers to back it up. Some of the .NET serialization code is pretty old, written in 2003 or 2005 and may not be taking advantage of new techniques or knowledge. Plus, it's rather flexible "make it work for everyone" code, as opposed to very narrowly purposed code.

People have different serialization needs. You can't serialize something as XML and expect it to be small and tight. You likely can't serialize a structure as JSON and expect it to be as fast as a packed binary serializer.

Measure your code, consider your requirements, and step back and consider all options.

Fourth: Newer .NET Serializers to Consider

Now that I have a sense of what's happening and how to measure the timing, it was clear these serializers didn't meet this reader's goals. Some of are old, as I mentioned, so what other newer more sophisticated options exist?

There's two really nice specialized serializers to watch. They are Jil from Kevin Montrose, and protobuf-net from Marc Gravell. Both are extraordinary libraries, and protobuf-net's breadth of target framework scope and build system are a joy to behold. There are also other impressive serializers in including support for not only JSON, but also JSV and CSV in ServiceStack.NET.

Protobuf-net - protocol buffers for .NET

Protocol buffers are a data structure format from Google, and protobuf-net is a high performance .NET implementation of protocol buffers. Think if it like XML but smaller and faster. It also can serialize cross language. From their site:

Protocol buffers have many advantages over XML for serializing structured data. Protocol buffers:

  • are simpler
  • are 3 to 10 times smaller
  • are 20 to 100 times faster
  • are less ambiguous
  • generate data access classes that are easier to use programmatically

It was easy to add. There's lots of options and ways to decorate your data structures but in essence:

var r = ProtoBuf.Serializer.Deserialize<List<DataItem>>(memInStream);

The numbers I got with protobuf-net were exceptional and in this case packed the data tightly and quickly, taking just 49ms.

JIL - Json Serializer for .NET using Sigil

Jil s a Json serializer that is less flexible than Json.net and makes those small sacrifices in the name of raw speed. From their site:

Flexibility and "nice to have" features are explicitly discounted in the pursuit of speed.

It's also worth pointing out that some serializers work over the whole string in memory, while others like Json.NET and DataContractSerializer work over a stream. That means you'll want to consider the size of what you're serializing when choosing a library.

Jil is impressive in a number of ways but particularly in that it dynamically emits a custom serializer (much like the XmlSerializers of old)

Jil is trivial to use. It just worked. I plugged it in to this sample and it took my basic serialization times to 84ms.

result = Jil.JSON.Deserialize<Foo>(jsonData);

Conclusion: There's the thing about benchmarks. It depends.

What are you measuring? Why are you measuring it? Does the technique you're suing handle your use case? Are you serializing one large object or thousands of small ones?

James Newton-King made this excellent point to me:

"[There's a] meta-problem around benchmarking. Micro-optimization and caring about performance when it doesn't matter is something devs are guilty of. Documentation, developer productivity, and flexibility are more important than a 100th of a millisecond."

In fact, James pointed out this old (but recently fixed) ASP.NET bug on Twitter. It's a performance bug that is significant, but was totally overshadowed by the time spent on the network.

Thanks to Marc Gravell and James Newton-King for their time helping with this post.

What are your benchmarking tips and tricks? Sound off in the comments!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

NuGet Package of the Week: A different take on ASP.NET MVC Forms with ChameleonForms

February 17, '15 Comments [32] Posted in ASP.NET MVC | NuGet | NuGetPOW | Open Source
Sponsored By

One of the nice things about any modular system (like ASP.NET) is the ability to swap out the parts you don't like. As the authors of ChameleonForms state, HTML forms is a pain. It's repetitive, it's repetitive, and it's boring. While ASP.NET MVC's Form Helpers help a lot, they felt that helper methods like Html.EditorForModel didn't go far enough or give you enough flexibility. ChameleonForms adds its own templating model and attempts to be as DRY as possible. It also takes a number of issues head on like better handling for drop-down lists and lists of radio buttons, and it even supports Twitter Bootstrap 3 to you can bang out HTML forms ASAP.

ChameleonForms also is a nice example of a tidy and well-run small open source project. They've got a public Trello backlog board, excellent documentation, a continuous integration build, a good example project, and of course, they're on NuGet. Check out the other projects that the folks in the "MRCollective" work on as well, as they've got their own GitHub organization.

NuGet Install ChameleonForms

Often ChameleonForms tries to use C# for the whole form, rather than switching back and forth from Div to Html Helper. For example:

@using (var f = Html.BeginChameleonForm()) {
using (var s = f.BeginSection("Signup for an account")) {
@s.FieldFor(m => m.FirstName)
@s.FieldFor(m => m.LastName)
@s.FieldFor(m => m.Mobile).Placeholder("04XX XXX XXX")
@s.FieldFor(m => m.LicenseAgreement).InlineLabel("I agree to the terms and conditions")
}
using (var n = f.BeginNavigation()) {
@n.Submit("Create")
}
}

This is the whole form using usings for scoping, and it's nice and clean.  How about a comparison example? Here's standard ASP.NET MVC:

@using (Html.BeginForm())
{
<fieldset>
<legend>A form</legend>
<dl>
<dt>@Html.LabelFor(m => m.RequiredString, "Some string")</dt>
<dd>@Html.TextBoxFor(m => m.RequiredString) @Html.ValidationMessageFor(m => m.RequiredString)</dd>
<dt>@Html.LabelFor(m => m.SomeEnum)</dt>
<dd>@Html.DropDownListFor(m => m.SomeEnum, Enum.GetNames(typeof(SomeEnum)).Select(x => new SelectListItem {Text = ((SomeEnum)Enum.Parse(typeof(SomeEnum), x)).Humanize(), Value = x})) @Html.ValidationMessageFor(m => m.SomeEnum)</dd>
<dt>@Html.LabelFor(m => m.SomeCheckbox)</dt>
<dd>@Html.CheckBoxFor(m => m.SomeCheckbox) @Html.LabelFor(m => m.SomeCheckbox, "Are you sure?") @Html.ValidationMessageFor(m => m.SomeCheckbox)</dd>
</dl>
</fieldset>
<div class="form_navigation">
<input type="submit" value="Submit" />
</div>
}

And here is the same form with ChameleonForms.

@using (var f = Html.BeginChameleonForm()) {
using (var s = f.BeginSection("A form")) {
@s.FieldFor(m => m.RequiredString).Label("Some string")
@s.FieldFor(m => m.SomeEnum)
@s.FieldFor(m => m.SomeCheckbox).InlineLabel("Are you sure?")
}
using (var n = f.BeginNavigation()) {
@n.Submit("Submit")
}
}

But these are basic. How about something more complex? This one has a bunch of variety, a number overloads and customizations, as well as a FileUpload (note that the form is a Multipart form):

@using (var f = Html.BeginChameleonForm(method: FormMethod.Post, enctype: EncType.Multipart))
{
<p>@f.LabelFor(m => m.SomeCheckbox).Label("Are you ready for: ") @f.FieldElementFor(m => m.SomeCheckbox) @f.ValidationMessageFor(m => m.SomeCheckbox)</p>
<p>@f.FieldElementFor(m => m.RequiredStringField).TabIndex(4)</p>
using (var s = f.BeginSection("My Section!", InstructionalText(), new{@class = "aClass"}.ToHtmlAttributes()))
{
using (var ff = s.BeginFieldFor(m => m.RequiredStringField, Field.Configure().Attr("data-some-attr", "value").TabIndex(3)))
{
@ff.FieldFor(m => m.NestedField).Attr("data-attr1", "value").TabIndex(2)
@ff.FieldFor(m => m.SomeEnum).Attr("data-attr1", "value")
@ff.FieldFor(m => m.SomeEnum).Exclude(SomeEnum.SomeOtherValue)
}
@s.FieldFor(m => m.SomeCheckbox).AsDropDown()
using (var ss = s.BeginSection("Nested section"))
{
@ss.FieldFor(m => m.FileUpload).Attr("data-attr1", "value")
}
@s.FieldFor(m => m.RequiredStringField).OverrideFieldHtml(new MvcHtmlString("Custom html <b>she-yeah</b>!"))
@s.FieldFor(m => m.TextAreaField).Cols(60).Rows(5).Label("Some Label").AutoFocus().TabIndex(1)
@s.FieldFor(m => m.SomeCheckbox).InlineLabel("Some label").WithHint("Format: XXX")
@s.FieldFor(m => m.SomeCheckbox).AsRadioList().WithTrueAs("True").WithFalseAs("False")
@s.FieldFor(m => m.ListId)
@s.FieldFor(m => m.ListId).AsRadioList()
@s.FieldFor(m => m.SomeEnums)
@s.FieldFor(m => m.SomeEnumsList).AsRadioList()
@s.FieldFor(m => m.Decimal)
@s.FieldFor(m => m.Int).AsInputGroup().Append(".00").Prepend("$")
@s.FieldFor(m => m.DecimalWithFormatStringAttribute)
@s.FieldFor(m => m.NullableInt)
@s.FieldFor(m => m.Child.ChildField)
@s.FieldFor(m => m.Child.SomeEnum).AsRadioList()
@s.FieldFor(m => m.RequiredStringField).Disabled()
@s.FieldFor(m => m.RequiredStringField).Readonly()
}
using (var n = f.BeginNavigation())
{
@n.Submit("Submit")
@n.Reset("Reset")
}
}

ChameleonForms also has a special NuGet package if you're using TwitterBootstrap that changes how forms with the BeginChameleonForm method render.

ChameleonForms also has some convenient extra abilities, like being able to automatically infer/create a [DisplayName] so you don't have to. If you're doing Forms in English and your preferred Display Name will end up just being your variable name this can be a useful time saver (although you may have opinions about its purity.)

So instead of the tedium of:

[DisplayName("Email address")]
public string EmailAddress { get; set; }

[DisplayName("First name")]
public string FirstName { get; set; }

You can just say this once, picking just one...this is an example where they use HumanizedLabels.

HumanizedLabels.Register(LetterCasing.AllCaps) => "EMAIL ADDRESS"
HumanizedLabels.Register(LetterCasing.LowerCase) => "email address"
HumanizedLabels.Register(LetterCasing.Sentence) => "Email address"
HumanizedLabels.Register(LetterCasing.Title) => "Email Address"

If you've got a lot of Forms to create and they're just no fun anymore, you should definitely give ChameleonForms a try. If you're a Twitter Bootstrap shop, doubly so, as that's where ChameleonForms really shines.

I'll do a few other posts exploring different ways to for Forms in ASP.NET MVC in the coming weeks. Be sure to explore the NuGet Package of the Week Archives as well!


PLUG: Did you know I have a YouTube channel? Subscribe over here. I've got tutorials on how to effectively use Windows 8 and 8.1, Build to Build walkthroughs of the latest versions of Windows 10, and I just started a new series I'm sure you'll want to share with your family called "How to REALLY use Microsoft Office." Help me out and spread the word!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

JavaScript Has Won: Run Flash with Mozilla Shumway and Develop Silverlight in JS with Fayde

February 13, '15 Comments [50] Posted in Open Source | Silverlight
Sponsored By

Well this is some amazing news that has been a long time coming. You can take your Flash apps and run them without Flash, and take your Silverlight apps and (almost) run them without Silverlight.

If you still don't think JavaScript has won the web, please read on.

Shumway - Flash in JavaScript and HTML 5

The Mozilla Project "Shumway" has been a long time coming (as long ago as 2012, but it's now ready to be looked at more deeply).

Chris Peterson, a Program Manager on the Mozilla Shumway says:

"Shumway is in a race to stay relevant as Flash fades from the web, but there will always be a long-tail of Flash content that would/will be lost when Adobe or browsers stop supporting the Flash plugin."

Think about that. We've all largely got "Evergreen Browsers" now that update themselves as often as weekly, but sometimes it feels like Adobe Flash is being attacked daily, so we're told to update that as well. Flash itself has more than fallen from grace, as Chris points out, it's fading from the web itself. Fast forward a year or so when there is no more Flash installed, but there's still Flash on the web. Enter Shumway - it's a renderer for SWF (Flash files) without native code! Shumway literally.

Why is it called Shumway? Again, Chris:

"The name "Shumway" is derived from "Gordon Shumway", the actual name of the TV character ALF: Flash -> Flash Gordon -> Gordon Shumway -> Shumway."

That's awesome. What else is awesome? "Shumway is written in TypeScript. It has an ActionScript interpreter and a JIT that generates JavaScript, compiled using eval()."

Shumway makes Flash into HTML5

So Shumway is an HTML experiment that uses TypeScript (a modern typed JavaScript compiler/transpiler) to read ActionScript and resources and JIT the result into evaluated JavaScript. Fantastic. It's also open source and on GitHub. Even better, the Firefox Nightly is using Shumway for Flash videos on Amazon.com. This is the beginning of their test, I presume, to sunset Flash in Firefox.

Fayde - Silverlight in JavaScript and HTML5 Canvas

At the same time, there's The Fayde project. Also Open Source, Fayde also compiles to JavaScript. But Fayde transforms Silverlight into HTML5 Canvas and JavaScript! It's an implementation of a XAML engine in JS. Here's a near-Silverlight implementation of the classic Todo application, expressed on the web without plugins. Not enough? Here's a complex Fantasy Football app written in a Silverlight-like environment but running in your browser, again, without Silverlight.

Fayde - Silverlight in HTML5

To be clear, there are significant architectural differences between these two projects. Shumway reads the binary SWF format and tries to Be Flash, while Fayde is reimagining, if you will, that takes the Silverlight concepts of ViewModels and Views in XAML and adds TypeScript (a comfortable language for the C#-familiar) with the result rendered on a Canvas. It's not a Silverlight Emulator, it's a Silverlight-like implementation and app development pattern for HTML5. It's open source, and they are looking for participation, so head over and get involved! Fayde may be the Silverlight migration strategy you've been looking for.

Aside: From my perspective, it's not unreasonable to imagine taking something like JSIL (listen to my podcast on this amazing project) or a similar IL->JS system and combine it with Fayde to somehow run XAPs as well.

I love how crazy JavaScript is and what people have been able to do with it. Now, go run this PC Emulator (~virtual machine) written by Fabrice Bellard in JavaScript. It's Linux, on a 486, in your browser, in JavaScript. *brain explodes*

Have a lovely day and enjoy today's Open Web.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Announcing: Running Ruby on Rails on IIS8 (or anything else, really) with the new HttpPlatformHandler

February 9, '15 Comments [19] Posted in IIS | Ruby
Sponsored By

For years there's been numerous hacks and ways to get Ruby on Rails to run on IIS. There's also ways to get Java via Tomcat or Jetty, Go, and other languages and environments to run as well. There's ways to get Node.JS running on IIS using iisnode but that's been node-specific. The blog posts you do find say things like "get Rails to run on IIS in 10 steps" and I'm like JUST TEN?!? Why not 13? Others say "You can deploy Rails under IIS, it's just very difficult and there's not a lot of documentation. You'll need a special Fast-CGI implementation...WELCOME TO HELL."

No longer.

Azure Websites has supported node AND Java (again, Tomcat or Jetty) for a while now, in production and it's very nice and it runs under IIS. How? They've now brought that support to Windows running IIS 8+ with the release of the HttpPlatformHandler. Here's their example on how to get IIS 8+ running Java, easily.

Let's see if this works well with Ruby on Rails as well!

Why is HttpPlatformHandler interesting? Check this from the docs...it means IIS can host anything that runs on Windows now, easily. These things were possible before, but with all kinds of hacks and FastCGI this and that. What's great about HttpPlatformHandler is that it isn't about Rails. It's about any process that's listening on a port. You get all the value of IIS *and* total control of your self-hosting scenario.

The HttpPlatformHandler is an IIS Module, for IIS 8+, which does the following two things:

  1. Process management of http listeners - this could be any process that can listen on a port for http requests.  For example - Tomcat, Jetty, Node.exe, Ruby etc;
  2. Proxy requests to the process that it manages.

To be clear, you can work with Ruby on Rails on Windows and have it host itself with WEBrick locally, but if you're going to go production on Windows you'll want to have IIS and more likely, jRuby in a Tomcat container, similar to using Nginx on linux. What value does IIS provide in a scenario like this? Static file hosting, Reverse Proxy, complex auth that can span multiple apps, languages and frameworks, it monitors and manages your process looking at memory and CPU, crashes, etc.

Running Ruby on Rails on IIS 8 with the HttpPlatformHandler

First make sure you have Ruby on Rails. If you do, skip forward.

I use the http://railsinstaller.org for Windows and go. You'll get Ruby, Rails, Bundler, Sqlite, and TinyTDS. Even SQL Server support. Very kind of them. Another good Rails on Windows on is RailsFTW.

I go to Turn Windows Features On and Off to make sure I have IIS installed as well.

Turn on IIS in Windows Features

Then get the HttpPlatformHandler. You can get it with the Web Platform Installer, or just install it from here: x86/x64

I make a folder for the app I'm going to make. I put it in c:\inetpub\wwwroot\rails but you can move it around if you like.

Make sure your Rails app is an application in IIS

I right-click my folder in IIS Manager and "Convert to Application."

My Rails application in IIS

I run "gem install rails" to make sure I have Rails in the first place. ;) You will if you installed with the RailsInstaller. If you installed with the RubyInstaller, then this will get you Rails.

NOTE: If you have issues with SSL running gem on Windows, you'll need manually to update gem to 2.2.3 as of the time of this writing. I'm not sure why this isn't already done by the installer. The symptom I saw was weird errors on 'bundle install' that was fixed by this.

Installing Rails

Then, from inside c:\inetpub\wwwroot\rails, I ran "rails new helloworld." I ended up moving this folder up. I should have just made the app first, then converted the folder to an app in IIS. Order of operations and all that, eh?

OK, now I'll "rails server" from within c:\inetpub\wwwroot\rails, just to make sure Rails can run under the local WEBrick server. And it does:

Rails on Windows

Now, let's do it under IIS.

I need to make sure there's a web.config file in the same root folder as my Rails app. WHAT?!? Web.config is for ASP.NET, right? Well, no. It's config for any IIS application. You'll need this for Go, Java, PHP, Rails, node, ASP.NET, whatever. IIS can host basically anything.

Lemme add a hello world controller and edit its view. I'll "rails generate controller welcome index" then edit app\views\welcome\index.html.erb for good measure.

I put my Rails app under http://localhost/rails rather than at the root http://localhost so I did need to tell Rails 4 about the fact it's running in a subdirectory with a change to /config.ru, otherwise my routes wouldn't line up.

Rails.application.config.relative_url_root = '/rails'

map Rails.application.config.relative_url_root || "/" do
  run Rails.application
end

Make special note of the paths below AND the encoded &quot; there in the arguments. That's important, because it's a quoted argument passed into the ruby.exe process that IIS will kick off. Note also the %HTTP_PLATFORM_PORT% environment variable reference. That is passed in by IIS and will be a localhost-bound high port.

I also put in foo and bar for theoretical environment variables you might want your Rails app to know about. For example, I might add:

<environmentVariable name="RAILS_ENV" value="production"/>

...when it's time. I put in some standard debug logging there with the "stdout" but you can remove that if you don't want the clutter. Make sure your IISR_ users have write access to the folders if you want to see any logs.

WARNING/DISCLAIMER: This first example is just showing you what's possible. You DON'T want to go to production with the little built-in Ruby WEBrick web server. As Fabio Akita very kindly points out in the comments, and I'll pull his comment out here:

"One thing to be careful with the example using Rails. When running Ruby for Windows, when you run "rails server" it's going to spawn a single process that's mono-threaded, meaning that it can only respond to 1 request at a time. If IIS starts receiving too many requests simultaneously and each request is slow, it's going to generate a long queue until they start timing out.
In Linux we put NGINX to reverse-proxy HTTP requests to Unicorn, or Puma, or Rainbows, or Passenger. They all coordinate multiple Ruby processes (which in Unix, is very cheap as each process reuses memory from it's parent process upon forking - copy-on-write memory). So we can handle simultaneous requests."

You'd want to use JRuby and Tomcat with Puma under IIS for production on Windows.

KEEP SCROLLING FOR EXAMPLES USING JRuby, Trinidad/Tomcat, and Puma! They are farther down the page.

Also, on slower machines when running in development, you might need to up your startupTimeLimit if you are seeing IIS stop your Ruby processes that take to long to startup.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<handlers>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
</handlers>
<httpPlatform stdoutLogEnabled="true" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="c:\RailsInstaller\Ruby2.1.0\bin\ruby.exe"
arguments="&quot;C:\RailsInstaller\Ruby2.1.0\bin\rails&quot; server -p %HTTP_PLATFORM_PORT% -b 127.0.0.1">
<environmentVariables>
<environmentVariable name="foo" value="bar"/>
</environmentVariables>
</httpPlatform>
</system.webServer>
</configuration>

Here's a screenshot of Ruby within the SysInternals Process Explorer application. I wanted to show you this so you could see the Process Tree and see who started which process. You can see w3wp (that's IIS) which is a Service, and it's hosting Ruby, running Rails. Make note of the command line arguments as well.

image

And here it is. Ruby on Rails 4 running under IIS8 on my Windows 8 machine.

Ruby on Rails on IIS on Windows

Big thanks to Ranjith Ramachandra (@ranjithtweets) and Andrew Westgarth (@apwestgarth) at Microsoft for the help with the web.config values!

TL;DR

So, basically, to give you the TL;DR version, except at the end. When you have IIS, install HttpPlatformHandler and add a web.config as appropriate and you're all set. Run what you like, passing in the port that IIS will proxy to.

UPDATE: Puma and Trinidad (with Tomcat) on IIS

As pointed out in the comments, it's silly to use WEBrick in Production. Don't'.

I'm told JRuby is the way to go for prod. I was able to install JRuby and both Trinidad (with Tomcat) and Puma and get my HelloWorld running under IIS in an hour.

Here's Trinidad (I'm told Trinidad is out of vogue, however). I did a "JRuby -S gem install trinidad" and was on my way.

Note the JAVA_HOME environment variable setting. I also had to update some security policy files due to a "Illegal key size" error which was a Javaism. Otherwise, it just worked once the paths lined up. If you see problem, it'll be path related or you'll be loading the wrong Java version. Also, experiment but you don't really need to mess with processesPerApplications. If you create a lot of individual processes, Java can take up a lot of memory (300megs or so) per process and you can thrash your system.y

You'll get 10x the perf when running under production, so note I'm explicitly running in the prod environment.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<handlers>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
</handlers>
<httpPlatform stdoutLogEnabled="false" processesPerApplication="1" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="C:\jruby-1.7.19\bin\jruby.exe"
arguments="-S trinidad --context /jRubyonRails --env production --dir C:\inetpub\wwwroot\jRubyonRails -p %HTTP_PLATFORM_PORT% ">
<environmentVariables>
<environmentVariable name="JAVA_HOME" value="C:\Program Files\Java\jre1.8.0_31"/>
</environmentVariables>
</httpPlatform>
</system.webServer>
</configuration>

And it works. I was easily able to get 1600+ req/sec on my laptop with minimal effort. I'm sure I could do better with some tuning and a better machine.

JRuby on Rails using Trinidad behind IIS8 with the HttpPlatformHandler

OK, let's swap out Trindad/Tomcat for Puma. I put Puma in the Gemfile and did "bundle exec puma" to test. That worked. Now I need to hook it into IIS. I also had to add gem 'jruby-openssl', :require => false to  my Gemfile to avoid some weird errors.

Basically it was the same thing, as IIS is the reverse proxy and process manager in this case.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<handlers>
<add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" requireAccess="Script" />
</handlers>
<httpPlatform stdoutLogEnabled="false" processesPerApplication="1" stdoutLogFile="rails.log" startupTimeLimit="20" processPath="C:\jruby-1.7.19\bin\jruby.exe"
arguments="-S puma --env production --dir C:\inetpub\wwwroot\jRubyonRails -p %HTTP_PLATFORM_PORT% ">
<environmentVariables>
<environmentVariable name="JAVA_HOME" value="C:\Program Files\Java\jre1.8.0_31"/>
</environmentVariables>
</httpPlatform>
</system.webServer>
</configuration>

I'm using just 1 processPerApplication and I'm getting 1500 req/s easily. I don't know anything about Puma or JRuby but I assume it can do better if I knew how to tune it.

Load testing JRuby on Windows under IIS

I tested without IIS against Puma itself and saw essentially the same results. IIS has minimal overhead that I can't measure in this case and you get its process management and other benefits. If you're having issues with perf doing this kind of stuff, it's unlikely it will be IIS.

JRuby on Rails using Puma with the HttpPlatformHandler

All in all, the HttpPlatformHandler just maybe the reverse proxy you've always wanted for Windows!

Related Links


Sponsor: Big thanks to Infragistics for sponsoring the feed this week! Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid – Download for free now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.