Scott Hanselman

How to run Background Tasks in ASP.NET

August 26, '14 Comments [41] Posted in ASP.NET | NuGet | NuGetPOW | Open Source
Sponsored By

A few years back Phil Haack wrote a great article on the dangers of recurring background tasks in ASP.NET. In it he points out a few gotchas that are SO common when folks try to do work in the background. Read it, but here's a summary from his post.

  • An unhandled exception in a thread not associated with a request will take down the process.
  • If you run your site in a Web Farm, you could end up with multiple instances of your app that all attempt to run the same task at the same time.
  • The AppDomain your site runs in can go down for a number of reasons and take down your background task with it.

If you think you can just write a background task yourself, it's likely you'll get it wrong. I'm not impugning your skills, I'm just saying it's subtle. Plus, why should you have to?

There's LOT of great ways for you to do things in the background and a lot of libraries and choices available.

Some ASP.NET apps will be hosted in IIS in your data center and others will be hosted in the Azure cloud. The spectrum of usage is roughly this, in my opinion:

  • General: Hangfire (or similar similar open source libraries)
    • used for writing background tasks in your ASP.NET website
  • Cloud: Azure WebJobs 
    • A formal Azure feature used for offloading running of background tasks outside of your Website and scale the workload
  • Advanced: Azure Worker Role in a Cloud Service
    • scale the background processing workload independently of your Website and you need control over the machine

There's lots of great articles and videos on how to use Azure WebJobs, and lots of documentation on how Worker Roles in scalable Azure Cloud Services work, but not a lot about how your hosted ASP.NET application and easily have a background service. Here's a few.

WebBackgrounder

As it says "WebBackgrounder is a proof-of-concept of a web-farm friendly background task manager meant to just work with a vanilla ASP.NET web application." Its code hasn't been touched in years, BUT the WebBackgrounder NuGet package has been downloaded almost a half-million times.

The goal of this project is to handle one task only, manage a recurring task on an interval in the background for a web app.

If your ASP.NET application just needs one background task to runs an a basic scheduled interval, than perhaps you just need the basics of WebBackgrounder.

using System;
using System.Threading;
using System.Threading.Tasks;

namespace WebBackgrounder.DemoWeb
{
public class SampleJob : Job
{
public SampleJob(TimeSpan interval, TimeSpan timeout)
: base("Sample Job", interval, timeout)
{
}

public override Task Execute()
{
return new Task(() => Thread.Sleep(3000));
}
}
}

Built in: QueueBackgroundWorkItem - Added in .NET 4.5.2

Somewhat in response to the need for WebBackgrounder, .NET 4.5.2 added QueueBackgroundWorkItem as a new API. It's not just a "Task.Run," it tries to be more:

QBWI schedules a task which can run in the background, independent of any request. This differs from a normal ThreadPool work item in that ASP.NET automatically keeps track of how many work items registered through this API are currently running, and the ASP.NET runtime will try to delay AppDomain shutdown until these work items have finished executing.

It can try to delay an AppDomain for as long as 90 seconds in order to allow your task to complete. If you can't finish in 90 seconds, then you'll need a different (and more robust, meaning, out of process) technique.

The API is pretty straightforward, taking  Func<CancellationToken, Task>. Here's an example that kicks of a background work item from an MVC action:

public ActionResult SendEmail([Bind(Include = "Name,Email")] User user)
{
if (ModelState.IsValid)
{
HostingEnvironment.QueueBackgroundWorkItem(ct => SendMailAsync(user.Email));
return RedirectToAction("Index", "Home");
}

return View(user);
}

FluentScheduler

FluentScheduler is a more sophisticated and complex scheduler that features a (you guessed it) fluent interface. You have really explicit control over when your tasks run.

using FluentScheduler;

public class MyRegistry : Registry
{
public MyRegistry()
{
// Schedule an ITask to run at an interval
Schedule<MyTask>().ToRunNow().AndEvery(2).Seconds();

// Schedule a simple task to run at a specific time
Schedule(() => Console.WriteLine("Timed Task - Will run every day at 9:15pm: " + DateTime.Now)).ToRunEvery(1).Days().At(21, 15);

// Schedule a more complex action to run immediately and on an monthly interval
Schedule(() =>
{
Console.WriteLine("Complex Action Task Starts: " + DateTime.Now);
Thread.Sleep(1000);
Console.WriteLine("Complex Action Task Ends: " + DateTime.Now);
}).ToRunNow().AndEvery(1).Months().OnTheFirst(DayOfWeek.Monday).At(3, 0);
}
}

FluentScheduler also embraces IoC and can easily plug into your favorite Dependency Injection tool of choice by just implementing their ITaskFactory interface.

Quartz.NET

Quartz.NET is a .NET port of the popular Java job scheduling framework of the (almost) same name. It's very actively developed. Quartz has an IJob interface with just one method, Execute, to implement.

using Quartz;
using Quartz.Impl;
using System;

namespace ScheduledTaskExample.ScheduledTasks
{
public class JobScheduler
{
public static void Start()
{
IScheduler scheduler = StdSchedulerFactory.GetDefaultScheduler();
scheduler.Start();

IJobDetail job = JobBuilder.Create<MyJob>().Build();

ITrigger trigger = TriggerBuilder.Create()
.WithDailyTimeIntervalSchedule
(s =>
s.WithIntervalInHours(24)
.OnEveryDay()
.StartingDailyAt(TimeOfDay.HourAndMinuteOfDay(0, 0))
)
.Build();

scheduler.ScheduleJob(job, trigger);
}
}
}

Then, inside your Application_Start, you call JobScheduler.Start(). There's a great getting started article on Quartz at Mikesdotnetting you should check out.

Hangfire

And last but definitely not least, the most polished (IMHO) of the group, Hangfire by @odinserj. It's a fantastic framework for background jobs in ASP.NET. It's even optionally backed by Redis, SQL Server, SQL Azure, MSMQ, or RabbitMQ for reliability.

The Hangfire documentation is amazing, really. Every open source project's document should be this polished. Heck, ASP.NET's documentation should be this good.

The best feature from Hangfire is its built in /hangfire dashboard that shows you all your scheduled, processing, succeeded and failed jobs. It's really a nice polished addition.

image

You can enqueue "fire and forget" jobs easily and they are backed by persistent queues:

BackgroundJob.Enqueue(() => Console.WriteLine("Fire-and-forget"));

You can delay them...

BackgroundJob.Schedule(() => Console.WriteLine("Delayed"), TimeSpan.FromDays(1));

Or great very sophisticated CRON style recurrent tasks:

RecurringJob.AddOrUpdate(() => Console.Write("Recurring"), Cron.Daily);

Hangfire is just a joy.

Check out the Hangfire Highlighter Tutorial for a sophisticated but easy to follow real-world example.

There's a rich ecosystem out there ready to help you with your background tasks. All these libraries are excellent, are open source, and are available as NuGet Packages.

Did I miss your favorite? Sound off in the comments!


Sponsor: Many thanks to my friends at Raygun for sponsoring the feed this week. I *love* Raygun and use it myself. It's amazing. Get notified of your software’s bugs as they happen! Raygun.io has error tracking solutions for every major programming language and platform - Start a free trial in under a minute!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

NuGet Package of the Week: MarkdownLog makes log files much prettier

July 30, '14 Comments [6] Posted in NuGet | NuGetPOW
Sponsored By

While I'm not 100% gaga over Markdown as I know many of you are, I definitely appreciate it's usefulness and it's clarity. Some folks would say I shouldn't rest until every binary document on my hard drive has been converted to Markdown. I say, nay nay. That said, I totally dig MarkdownPad and you should use it every day. It's lovely.

If you don't want to install anything, check out http://dillinger.io and quickly edit some Markdown in the browser.

image

But, I digress.

Note: Be sure to check out all the NuGet packages of the week! There's more!

The general idea behind Markdown is that HTML is way too complex for 90% of what you need, so rather than <ul> and <li> and all that, why not just express bullets with asterisks? We can all get behind that. There are many other simple Markdown syntaxes that you can learn in a few minutes. Once you've created some Markdown, you can easily generate PDFs, HTML, Word Documents, whatever you like.

Recently I stumbled upon Stuart Wheelwright's article on Using Markdown for Effective Logging over at CodeProject. Here's the first image borrowed from his excellent article where he clearly shows why Markdown is simpler than HTML:

html-vs-md

Stuart has taken Markdown and created a wonderful little library called MarkdownLog. It's a brilliant little idea that one of us should have come up with first! Kudos to Stuart. ;)

[MarkdownLog] is designed to make it trivial to produce Markdown formatted text from an application's data structures. Using just one line of code, a collection of .NET objects can be output as a table or list. And, because the output is Markdown, it can later be converted to HTML for publishing, if needed.

Even better, MarkdownLog is a Portable Class Library (PCL) and can be used with any .NET platform, including iOS with Xamarin. In fact, it's an iPhone app written with Xamarin that compelled Stuart to write MarkdownLog.

Here's one of his first examples. This C# code:

var myStrings = new[] { "John", "Paul", "Ringo", "George" };
Console.Write(myStrings.ToMarkdownBulletedList());

Gives you this simple Markdown.

   * John
* Paul
* Ringo
* George

At this point, you are likely unimpressed. But wait! There's more! There's a whole series of nice extension methods that make it easy to create templates from objects of any shape. Here's another example of his:

var data = new[]
{
new{Name = "Meryl Streep", Nominations = 18, Awards=3},
new{Name = "Katharine Hepburn", Nominations = 12, Awards=4},
new{Name = "Jack Nicholson", Nominations = 12, Awards=3}
};

Console.Write(data.ToMarkdownTable());

This gives you a Markdown table, of course. This looks nothing like an HTML table, but remember, Markdown is source code that can be translated into other formats like HTML and PDF.

 Name              | Nominations | Awards
----------------- | -----------:| ------:
Meryl Streep | 18 | 3
Katharine Hepburn | 12 | 4
Jack Nicholson | 12 | 3

Be sure to explore the full CodeProject article and the home page for MarkdownLog. Think about how you could add this to your existing logging framework and create better logs for tests, builds, anything. Take a look at the HTML render of one of Stuarts's Test Suite Runs and tell me that you want immediately want to get to work updating your old .LOG files.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

NuGet Package of the Week: ASP.NET Web API Caching with CacheCow and CacheOutput

June 27, '14 Comments [8] Posted in ASP.NET Web API | NuGet | NuGetPOW
Sponsored By

You can see other cool NuGet Packages I've mentioned on the blog here. Today's NuGet package is CacheCow, which has possibly the coolest Open Source Library name since Lawnchair.js.

image

"CacheCow is a library for implementing HTTP caching on both client and server in ASP.NET Web API. It uses message handlers on both client and server to intercept request and response and apply caching logic and rules."

CacheCow was started by Ali Kheyrollahi with help from Tugberk Ugurlu and the community, and is a fantastically useful piece of work. I wouldn't be surprised to see this library start showing in more places one day.

As an aside, Ali, this would be a great candidate for setting up a free AppVeyor Continuous Integration build along with a badge showing that the project is building and healthy!

CacheCow on the server can manage the cache in a number of ways. You can store it in SQL Server with the EntityTagStore, or implement your own storage handler. You can keep the cache in memcached, Redis, etc.

Consider using a library like CacheCow if you're putting together a Web API and haven't given sufficient thought to caching yet, or if you're already sprinkling cache code throughout your business logic. You might already suspect that is going to litter your code but perhaps haven't gotten around to tidying up. Now is a good time to unify your caching.

As a very simple example, here's the HTTP Headers from an HTTP GET to a Web API:

Cache-Control: no-cache
Content-Length: 19
Content-Type: application/json; charset=utf-8
Date: Fri, 27 Jun 2014 23:22:10 GMT
Expires: -1
Pragma: no-cache

Here's the same thing after adding the most basic caching to my ASP.NET applications config:

GlobalConfiguration.Configuration.MessageHandlers.Add(new CachingHandler(GlobalConfiguration.Configuration));

The HTTP Headers with the same GET with CacheCow enabled:

Cache-Control: no-transform, must-revalidate, max-age=0, private
Content-Length: 19
Content-Type: application/json; charset=utf-8
Date: Fri, 27 Jun 2014 23:24:16 GMT
ETag: W/"e1c5ab4f818f4cde9426c6b0824afe5b"
Last-Modified: Fri, 27 Jun 2014 23:24:16 GMT

Notice the Cache-Control header, the Last-Modified, and the ETag. The ETag is weak as indicted by "W/" which means that this response is semantically equivalent to the last response. If I was caching persistently, I could get a strong ETag indicating that the cached response was byte-for-byte identical. Also, if the client was smart about caching and added If-Modified-Since or If-None-Match for ETags, the response might be a 304 Not Modified, rather than a 200 OK. If you're going to add caching to your Web API server, you'll want to make sure your clients respect those headers fully!

From ALI's blog, you can still use HttpClient in your clients, but you use WebRequestHandler as the message handler:

HttpClient client = new HttpClient(new WebRequestHandler()
{
CachePolicy = new RequestCachePolicy(RequestCacheLevel.Default)
});
var httpResponseMessage = await client.GetAsync(http://superpoopy);

Really don't want a resource cached? Remember, this is HTTP so, Cache-Control: no-cache from the client!

Of course, one of the most important aspects of caching anything is "when do I invalidate the cache?" CacheCow gives you a lot control over this, but you really need to be aware of what your actual goal is or you'll find things cache you don't want, or things not cached that you do.

  • Are you looking for time-based caching? Cache for 5 min after a DB access?
  • Are you looking for smart caching that invalidates when it sees what could be a modification? Invalidate a collection after a POST/PUT/DELETE?

Given that you're likely using REST, you'll want to make sure that the semantics of these caching headers and their intent is reflected in your behavior. Last-Modified should reflect realty when possible.

From the CacheCow Wiki, there's great features for both the Server-side and Client-side. Here's CacheCow.Server features

  • Managing ETag, Last Modified, Expires and other cache related headers
  • Implementing returning Not-Modified 304 and precondition failed 412 responses for conditional calls
  • Invalidating cache in case of PUT, POST, PATCH and DELETE
  • Flexible resource organization. Rules can be defined so invalidation of a resource can invalidate linked resources

and the CacheCow.Client features

  • Caching GET responses according to their caching headers
  • Verifying cached items for their staleness
  • Validating cached items if must-revalidate parameter of Cache-Control header is set to true. It will use ETag or Expires whichever exists
  • Making conditional PUT for resources that are cached based on their ETag or expires header, whichever exists

Another good ASP.NET caching library to explore is ASP.NET Web API "CacheOutput" by Filip Wojcieszyn. While it doesn't have an fun to say name ;) it's got some great features and is super easy to get started with. You can find CacheOutput with NuGet at

Install-Package Strathweb.CacheOutput.WebApi2

And you'll configure your caching options using the intuitive CacheOutput attributes like those you may have seen in ASP.NET MVC:

[CacheOutput(ClientTimeSpan = 100, ServerTimeSpan = 100)]
public IEnumerable<string> Get()
{
return new string[] { "value1", "value2" };
}

ASP.NET Web API CacheOutput has great getting started docs and clear easy to ready code.

So, you've got options. Go explore!

You can also pickup the Pro ASP.NET Web API book at Amazon. Go explore CacheCow or CacheOutput and support open source! If you find issues or feel there's work to be done in the documentation, why not do it and submit a pull request? I'm sure any project would appreciate some help with updated samples, quickstarts, or better docs.


Sponsor: Many thanks to our friends at Octopus Deploy for sponsoring the feed this week. Did you know that NuGet.org deploys with Octopus? Using NuGet and powerful conventions, Octopus Deploy makes it easy to automate releases of ASP.NET applications and Windows Services. Say goodbye to remote desktop and start automating today!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

NuGet Package of the Week: ImageProcessor - lightweight image manipulation in C#

May 21, '14 Comments [38] Posted in NuGet | NuGetPOW
Sponsored By

I really enjoy image manipulation in code. Sure, resizing photos is fun in Photoshop, but there's something viscerally enjoyable when you change images with your own code.

I've talked about image resizing libraries like ImageResizer before, but there's certainly room for more than one. Today I want to showcase ImageProcessor, an open source "collection of lightweight libraries written in C# that allows you to manipulate images on-the-fly using .NET 4+." ImageProcessor is available on GitHub.

ImageProcessor

ImageProcessor methods include; Resize, Rotate, Rounded Corners, Flip, Crop, Watermark, Filter, Saturation, Brightness, Contrast, Quality, Format, Vignette, Gaussian Blur, Gaussian Sharpen, and Transparency.

ImageProcessor has a notable number of configuration options for web apps, and a supporting ImageProcessor.Web package as well. It's an impressive body of work.

I like this simple example of loading, resizing, and saving an image with their fluent API:

// Read a file and resize it.
byte[] photoBytes = File.ReadAllBytes(file);
int quality = 70;
ImageFormat format = ImageFormat.Jpeg;
Size size = new Size(150, 0)

using (MemoryStream inStream = new MemoryStream(photoBytes))
{
using (MemoryStream outStream = new MemoryStream())
{
using (ImageFactory imageFactory = new ImageFactory())
{
// Load, resize, set the format and quality and save an image.
imageFactory.Load(inStream)
.Resize(size)
.Format(format)
.Quality(quality)
.Save(outStream);
}

// Do something with the stream.
}
}

You can easily chain functions with the API, like tinting and constraning:

imageFactory.Load(inStream)
.Constrain(size)
.Tint(Color.FromArgb(255, 106, 166, 204))
.Format(format)
.Save(outStream);

When you add ImageProcessor.Web it adds caching that takes pressure off your web servers. You can easily add HttpHandlers to watermark an image, for example, and cache the result.

This is a library that has as a lot of potential. Since it's open source, I'm sure they'd appreciate help from the community! Personally, I think they could use more Unit Tests and more examples.

Head over to https://github.com/JimBobSquarePants/ImageProcessor and star this project! Get involved, file issues, and contribute! http://imageprocessor.org/

Related Links


Sponsor: Many thanks to Izenda for sponsoring the blog feed this week. Please do check out their Intuitive Ad Hoc Reporting with Stunning Visualizations - Embed real time dashboards into your ASP.NET applications for easy, custom reports across all devices. Download a FREE TRIAL of Izenda Today!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

NuGet Package of the Week: Canopy Web Testing Framework with F#

March 25, '14 Comments [18] Posted in NuGet | NuGetPOW | Open Source
Sponsored By

I've been exploring Automated Browser Testing recently, and also checking out F# for unrelated reasons. However, when you combine the two you end up with "canopy." Canopy is a "f#rictionless web testing" framework that combines the flexibility of Selenium with the clean look of the F# language. F# is much terser (more elegant, even) than C#, and is garnering the interest of a lot of the .NET Open Source community. Folks are creating cool domain specific languages of their own using F# as the base.

You already have F# and perhaps didn't realize you did! If you don't, there's lots of ways to get F# for free. You can use F# for free with VS2013 Desktop Express plus Visual F# Tools 3.1.1.

F# is open source and cross platform, running on Linux, Mac OS X, Android, iOS, Windows as well as HTML5 and GPUs. F# is free to use and has an OSI-approved open source license.

Even if you don't feel like installing anything, you can learn and play with F# in your browser now! Check out http://www.tryfsharp.org 

Also check out FunScript, which is F# to JavaScript! Don't believe them? Try Pacman using F# and JavaScript with source!

image

Anyway, back to Canopy. Make a new Console app and NuGet in the canopy package:

image

The NuGet package will bring in Selenium as a dependency.

Then, try out their "Hello World" web testing sample, that I've also pasted here.

//these are similar to C# using statements
open canopy
open runner
open System

//start an instance of the firefox browser
start firefox

//this is how you define a test
"taking canopy for a spin" &&& fun _ ->
//this is an F# function body, it's whitespace enforced

//go to url
url "http://lefthandedgoat.github.io/canopy/testpages/"

//assert that the element with an id of 'welcome' has
//the text 'Welcome'
"#welcome" == "Welcome"

//assert that the element with an id of 'firstName' has the value 'John'
"#firstName" == "John"

//change the value of element with
//an id of 'firstName' to 'Something Else'
"#firstName" << "Something Else"

//verify another element's value, click a button,
//verify the element is updated
"#button_clicked" == "button not clicked"
click "#button"
"#button_clicked" == "button clicked"

//run all tests
run()

System.Console.WriteLine("press [enter] to exit")
System.Console.ReadLine() |> ignore

quit()

And boom, it just works. You can run this .NET application just like any other. .NET apps are .NET apps, as they say. It doesn't matter what language it's written in. When (if) you distribute this application you'd just include the contents of your Debug folder. No need to "install" F# or anything on the target machine.

image

You can do all sorts of Selenium testing with canopy, like:

//start a bunch of browsers and switch around
start firefox
let mainBrowser = browser
start chrome
let secondBrowser = browser
//switch back to mainBrowser after opening secondBrowser
switchTo mainBrowser

//take screenshots
let path = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\canopy\"
let filename = DateTime.Now.ToString("MMM-d_HH-mm-ss-fff")
screenshot path filename

//get an element
element "#firstName" |> someParent

//press buttons
press tab
press enter
press down
press up
press left
press right

//check and click things
check "#yes"
click "#login"

//or even drag things!
drag ".todo" ".inprogress"

Oh, and by the way, the canopy library builds itself using FAKE, the F# Build System we talked about last week! Go check these projects out and offer to help or support them. There's a lot of interesting open source happening in the .NET space lately that may have been flying under your radar.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Page 1 of 7 in the NuGet category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.