First time here? Check out the site's "greatest hits" or read a post from the archives. Feel free to leave a comment or ask a question, and consider subscribing to the latest posts via RSS or e-mail. Thanks for visiting!

My one-hundred-and-tenth podcast is up. This episode was recorded at the ALT.NET Open Spaces Conference in Seattle a few weeks back. I got to sit down with two gentlemen from Microsoft Research, Mike and Rustan, and talk about Spec# after their presentation at the conference.

What's Spec#? Currently it's a new language that is essentially extensions to C# 2.0. For example:

class ArrayList { public virtual void Insert(int index , object value)
requires 0 <= index && index <= Count;
requires !IsReadOnly && !IsFixedSize;
{ . . . }

See all that new stuff BEFORE the first curly brace? That's where you specify preconditions that you want enforced. You can also specify postconditions:

ensures Count == old(Count) + 1;
ensures value == this[index ];
ensures Forall{int i in 0 : index ; old(this[i]) == this[i]};
ensures Forall{int i in index : old(Count); old(this[i]) == this[i + 1]};

There's LOTS more, including a pretty extraordinary static analysis that runs in the background, kind of like the grammar checker in Word. Word has the spelling checker that turns errors red with squiggly underlines, and the grammar checker that turns grammatical errors green with squiggly underlines. Spec# will turn static errors into squiggles while you type.

They've even added specifications for the Base Class Library (BCL) that have previously only existed in documentation.

UseMscorlibContracts - Microsoft Visual Studio

BTW, if you want to see horribly shaky cellphone video of their presentation, you can see it at Greg Young's blog. He's posted recently about Spec# as well. If you don't like videos or podcasts, you can also read the Spec# Research Paper (PDF).

How can you get involved? We need to convince that folks on the C# team that Spec# is a valuable thing and that we'd like it sooner than later. Go over to Charlie Calvert's blog and email him that you want Spec# and why!

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

If you have trouble downloading, or your download is slow, do try the torrent with µtorrent or another BitTorrent Downloader.

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Check out their UI Suite of controls for ASP.NET. It's very hardcore stuff. One of the things I appreciate about Telerik is their commitment to completeness. For example, they have a page about their Right-to-Left support while some vendors have zero support, or don't bother testing. They also are committed to XHTML compliance and publish their roadmap. It's nice when your controls vendor is very transparent.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?



I'm thoroughly enjoying Twitter (follow me!) Apparently others are enjoying it also as there's a funny estimate that it's costing $14 billion in lost productivity (via a back of envelope calculation). There is about a million people on Twitter and maybe 60k adding each a month. Certainly it'll be blocked by most corporate firewalls soon for just this reason.

Twitter is also down ALL the time and it's been having weekly (daily?) scale problems for a YEAR, culminating in rumors that the development team is leaving Ruby on Rails that have been denied by Evan Williams (via a "Tweet," of course). However, that doesn't change the fact that Twitter is down so often there is a site dedicated to "Twitter Down Art" showcasing all the pictures that Twitter puts up when their sites are down. Google changes their art monthly and during holidays, but never goes down. Twitter goes down so often that they use that as an opportunity to change their art! So far, in this era of transparency, no one has explained in technical (or any) terms what the problem with Twitter is and folks are getting impatient. I think that it would be a great PR and Karmic move to just start a Twitter Technical Blog and share the crazy IT problem of the day. Surely someone is sleeping on a cot next to the TweetBoxes and has a story to tell.

There's another rumor that Twitter is worth about $150M. Seems to me that the second bubble hasn't popped. I'm certainly no business man, but we can build this better ourselves.

Twitter is centralized, which is insane (unless you're Google or Amazon or Live.com, etc, which are actually distributed in their own special way).

Twitter is IRC, it's IM, it's chatrooms, but ultimately it's just microblogging with a multi-format API. Here's the XML for my Twitter Account but it looks nicer as Atom.

But I already have a blog! Why use a service when I already host a blog? I hooked up Microsummaries to DasBlog two years ago. There's clearly precedent.

Two folks I know already have Twitter Backup Feeds. Both LazyCoder (backup twitstream) and Dave Winer (twitstream) have Backup Twitter Feeds. Dave takes it even farther and makes his twistream discoverable via a link in his blog:

<link rel="twitstream" href="http://twitter.scripting.com/daveRss.xml" />

With backups like these, who needs Twitter? There is already a very responsive group of Twitter client applications some of which, like twhirl cross-post to Pownce and Jaiku, two Twitter competitors. Russell Beattie has an idea for an Open Twitter Server called Peep using Jabber.

I propose that we distribute Twitter into generalized spec for microblogging and just use RSS or Jabber for the transport. I think the very-fast-rev'ing Twitter clients would support it. 

Thoughts? Should Twitter be a service or should Microblogging be an Open Thing?



OpenID Logo We spent a lot of time at Corillian (my last job) thinking about Identity, and a few months before I left I started getting into Cardspace and OpenID. This was a little over a year ago. We did a podcast on OpenID as well.

At that time, I tried to take the only .NET implementation at the time of OpenID which was written in in Boo written originally by Grant Monroe and port it to C# causing me to go through the Programmer Phases of Grief. Andrew Arnott and Jason Alexander took the reins and we spend a number of late nights trying to get a good OpenID library working. I gave up, but they soldiered on and created dotnetopenid (downloads), including a client and server as well as Andrew's excellent ASP.NET controls .

Fast-forward to now. My new friend Aaron Hockley decided to a stands and promote OpenID. He said:

Effective immediately, I will no longer comment on tech blogs that don’t support OpenID for comment authentication.

He's just one guy, but his heart is in the right place. He points out that:

Google offers it as a Blogger option. It’s available as a super-easy-to-install WordPress plugin. Movable Type has it as a built-in feature.

OpenID is a good thing and it's Growing. You may already have an OpenID if, for example, you have a Yahoo! account. More on this soon.

How to turn your blog into an OpenID

Simon Willison wrote How to turn your blog into an OpenID and it's very easy.

STEP 1: Get an OpenID. There a lots of servers and services out there you can use. I use http://www.myopenid.com for two reasons. One, I know the CEO (they're in Portland), and two, they support optionally using CardSpace to authenticate yourself (as well as the standard way with password).

STEP 2: Add these two lines to your blog's main template in-between the <HEAD></HEAD> tags at the top of your template. Most all blog engines support editing your template so this should be an easy and very possible thing to do.

Example:

<link rel="openid.server" href="http://www.myopenid.com/server" />
<link rel="openid.delegate" href=
http://YOURUSERNAME.myopenid.com/ />

This will let you use your domain/blog  as your OpenID. Now, I can log in with "http://www.hanselman.com" when I see an OpenID Login option - and so can you! Go do it now!

Making OpenID Logins Easier

If you have a blog or site with OpenID support, you should go get this little snippet of JavaScript and install an OpenID ID Selector on your blog from http://www.idselector.com/.

virgin-3

One of the things that is slowing OpenID adoption is that many people don't realize that they may already have one. That's what this little Javascript is trying to do by showing folks sites that they recognize. This way my Dad could login using Yahoo and it would make sense to him. It's a little busy, but it's a start. I've added an http://www.idselector.com/ to my blog for comments.

Adding OpenID Support to DasBlog

A year ago, I originally tried to port the Boo code to C# in an attempt to enable OpenID in DasBlog but eventually gave up. However, last night, I re-familiarized myself with the OpenID spec (it's on 2.0 now) and started reading the source for http://code.google.com/p/dotnetopenid/.

In a word, it's a joy. I was able to get OpenID running OK in two hours and working well and up on my blog in two more. I have to give credit to the fantastic work that Andrew Arnott and Jason Alexander and team are doing. It's come far and you should know about it, Dear Reader.

I had two scenarios in DasBlog (again, in case you didn't know, it's the C# and XML-based blog that runs this site and others) to handle.

First, I wanted to support OpenID for Comments which wouldn't actually "log a user in" in the stateful FormsAuthentication sense. I think this isn't a very common scenario, and I'd describe it as One-Time Occasional Authentication. In this case, I used the dotnetopenid classes directly in a moderately complex scenario.

Second, I wanted to support OpenID to login as the Administrator for my site. This would, in fact, log me in via FormsAuthentication. This would be a common scenario that you'd probably care about as it's very typical. In this case, I used the dotnetopenid ASP.NET Controls, which were about as easy as falling off a log. (That's pretty easy.)

Here's the first, harder, scenario.

If you've entered your OpenID and hit Submit Comment then we'll store the current entry and the comment you're submitting. We'll be redirecting away to get authenticated and we'll need them when we get back. If you're running in a WebFarm, you'll want to store these temporary variables in a database or somewhere that doesn't have node-affinity.

Session["pendingComment"] = comment.Text;
Session["pendingEntryId"] = ViewState["entryId"] as string;
OpenIdRelyingParty openid = new OpenIdRelyingParty();
IAuthenticationRequest req = openid.CreateRequest(openid_identifier.Text);
ClaimsRequest fetch = new ClaimsRequest();
fetch.Email = DemandLevel.Require;
fetch.Nickname = DemandLevel.Require;
req.AddExtension(fetch);
SaveCookies();
req.RedirectToProvider();
return;

What I think of as an "OpenID Client" is called a "Relying Party" or "RP" in the parlance of the OpenID folks. In this code we create an AuthenticationRequest and add some additional claims. There's a nice interface-based extension model in this lower-level library that lets you Request or Require information from the user's profile. For comments on the blog, I just need your email for your Gravatar and your Nickname for Display.

I then call RedirectToProvider, and that's if for the request side. Remember I said this was the hard scenario! Not so hard. ;)

Next, we're redirected to an OpenIDProvider, we authenticate (or not) and are redirected BACK with additional information encoded on the GET. On the way back in, in our Page_Load (or an HttpHandler if you like) we check the Response status.  If we're Authenticated, we grab the info we requested and add the comment. Bam. Sprinkle in a little error handling and we're all set.

OpenIdRelyingParty openid = new OpenIdRelyingParty();
if (openid.Response != null)
{
// Stage 3: OpenID Provider sending assertion response
switch (openid.Response.Status)
{
case AuthenticationStatus.Authenticated:
ClaimsResponse fetch = openid.Response.GetExtension(typeof(ClaimsResponse)) as ClaimsResponse;
string nick = fetch.Nickname;
string homepage = openid.Response.ClaimedIdentifier;
string email = fetch.Email;
string comment = Session["pendingComment"] as string;
string entryId = Session["pendingEntryId"] as string;
if (String.IsNullOrEmpty(comment) == false && String.IsNullOrEmpty(entryId) == false)
{
AddNewComment(nick, email, homepage, comment, entryId, true);
}
break;
}
}

Here's the second scenario where we'll log in as the Administrator of the blog. I just register the DotNetOpenId assembly in my ASPX page and put an <openidlogin> control on the page. Notice that even the claims I created in the manual scenario above are just properties on this control. There's also events like OnLoggedIn to handle the results.

<%@ Register Assembly="DotNetOpenId" Namespace="DotNetOpenId.RelyingParty" TagPrefix="cc1" %>
<cc1:openidlogin id="OpenIdLogin1"
RequestEmail="Require" RequestNickname="Request" RegisterVisible="false"
RememberMeVisible="True" PolicyUrl="~/PrivacyPolicy.aspx" TabIndex="1"
OnLoggedIn="OpenIdLogin1_LoggedIn"/></cc1:openidlogin>

This controls renders nicely as seen in the screenshot below.

image

In the OnLoggedIn event, I call my existing security APIs (Thanks to Tony Bunce and Anthony Bouch) and set the AuthCookie from FormsAuthentication.

protected void OpenIdLogin1_LoggedIn(object sender, OpenIdEventArgs e)
{
UserToken token = SiteSecurity.Login(e.Response);
if (token != null)
{
FormsAuthentication.SetAuthCookie(userName, rememberCheckbox.Checked);
Response.Redirect(SiteUtilities.GetAdminPageUrl(), true);
}
}

Poof. I love using well designed libraries and just work. At this point all that was left was adding some CSS and tidying up.

OpenID and ASP.NET WebForms and MVC

The dotnetopenid source includes source for sample sites. It actually includes three samples, two WebForms and one ASP.NET MVC.

The MVC implementation is very clean, even though (or because?) it doesn't use controls. Here's the Authenticate Controller Action:

public void Authenticate() {
var openid = new OpenIdRelyingParty();
if (openid.Response == null) {
// Stage 2: user submitting Identifier
openid.CreateRequest(Request.Form["openid_identifier"]).RedirectToProvider();
} else {
// Stage 3: OpenID Provider sending assertion response
switch (openid.Response.Status) {
case AuthenticationStatus.Authenticated:
FormsAuthentication.RedirectFromLoginPage(openid.Response.ClaimedIdentifier, false);
break;
case AuthenticationStatus.Canceled:
ViewData["Message"] = "Canceled at provider";
RenderView("Login");
break;
case AuthenticationStatus.Failed:
ViewData["Message"] = openid.Response.Exception.Message;
RenderView("Login");
break;
}
}
}

What about CardSpace?

Infocard LogoOpenID is a spec for a protocol that "eliminates the need for multiple usernames across different websites, simplifying your online experience."  What's cool is that it's open, so you (the consumer) gets to pick your Provider. It's not owned by anyone, so it's ours to screw up (or succeed with).

CardSpace is built into Vista and installed on XP when you put .NET 3.0 on your system. There are also Identity Selectors for Safari and Firefox in the works. It's different than OpenID in that it's concerned with strong authentication. Therefore, they are very complimentary.

Here's my CardSpace login as I'm getting ready to log into this blog...

image

...because my chosen OpenID provider at http://www.myopenid.com (it's free) also supports both InfoCards and SSL Certificates for authentication as well as strong passwords.

Notice the "Sign into Information Card" icon below next to the IconCard purple icon.

image

An OpenID provider can choose to use anything available with which to authenticate you. Here's a video of a Belgian using an eID to authenticate against an OpenID provider at http://openid.trustbearer.com/ that supports biometric devices, USB keys, and smart cards.

So What?

Get involved and give it a try! Here's some things you can do.

  1. Sign up for a Free OpenID at MyOpenID or one of the many public OpenID providers out there.
  2. Go use your new OpenID at one of the many sites that supports OpenID.
    • Come back to this post and leave your first comment using OpenID!
  3. Watch Simon Willison talk about the case for OpenID (video)

And, if you're a developer, get an OpenID library like dotnetopenid and consider enabling your app. Consider using the Javascript ID Selector to make for a nicer User Experience.

Technorati Tags: ,,


Three years ago I postulated about a ToString implementation for C# that seemed useful to me and a few days later I threw it out on the blog. We used in at my old company for a number of things.

Just now I realized that it'd be even more useful (or more evil) with Extension Methods, so I opened up the old project, threw in some MbUnit tests and changed my implementation to use extension methods.

So, like bad take-out food, here it is again, updated as ToString() overloads:

[Test]
public void MakeSimplePersonFormattedStringWithDoubleFormatted()
{
Person p = new Person();
string foo = p.ToString("{Money:C} {LastName}, {ScottName} {BirthDate}");
Assert.AreEqual("$3.43 Hanselman, {ScottName} 1/22/1974 12:00:00 AM", foo);
}

[Test]
public void MakeSimplePersonFormattedStringWithDoubleFormattedInHongKong()
{
Person p = new Person();
string foo = p.ToString("{Money:C} {LastName}, {ScottName} {BirthDate}",new System.Globalization.CultureInfo("zh-hk"));
Assert.AreEqual("HK$3.43 Hanselman, {ScottName} 1/22/1974 12:00:00 AM", foo);
}

It's moderately well covered for all of the hour it took to write it, and I find it useful.

image

Here's all it is.

public static class FormattableObject 
{
public static string ToString(this object anObject, string aFormat)
{
return FormattableObject.ToString(anObject, aFormat, null);
}

public static string ToString(this object anObject, string aFormat, IFormatProvider formatProvider)
{
StringBuilder sb = new StringBuilder();
Type type = anObject.GetType();
Regex reg = new Regex(@"({)([^}]+)(})",RegexOptions.IgnoreCase);
MatchCollection mc = reg.Matches(aFormat);
int startIndex = 0;
foreach(Match m in mc)
{
Group g = m.Groups[2]; //it's second in the match between { and }
int length = g.Index - startIndex -1;
sb.Append(aFormat.Substring(startIndex,length));

string toGet = String.Empty;
string toFormat = String.Empty;
int formatIndex = g.Value.IndexOf(":"); //formatting would be to the right of a :
if (formatIndex == -1) //no formatting, no worries
{
toGet = g.Value;
}
else //pickup the formatting
{
toGet = g.Value.Substring(0,formatIndex);
toFormat = g.Value.Substring(formatIndex+1);
}

//first try properties
PropertyInfo retrievedProperty = type.GetProperty(toGet);
Type retrievedType = null;
object retrievedObject = null;
if(retrievedProperty != null)
{
retrievedType = retrievedProperty.PropertyType;
retrievedObject = retrievedProperty.GetValue(anObject,null);
}
else //try fields
{
FieldInfo retrievedField = type.GetField(toGet);
if (retrievedField != null)
{
retrievedType = retrievedField.FieldType;
retrievedObject = retrievedField.GetValue(anObject);
}
}

if (retrievedType != null ) //Cool, we found something
{
string result = String.Empty;
if(toFormat == String.Empty) //no format info
{
result = retrievedType.InvokeMember("ToString",
BindingFlags.Public | BindingFlags.NonPublic |
BindingFlags.Instance | BindingFlags.InvokeMethod | BindingFlags.IgnoreCase
,null,retrievedObject,null) as string;
}
else //format info
{
result = retrievedType.InvokeMember("ToString",
BindingFlags.Public | BindingFlags.NonPublic |
BindingFlags.Instance | BindingFlags.InvokeMethod | BindingFlags.IgnoreCase
,null,retrievedObject,new object[]{toFormat,formatProvider}) as string;
}
sb.Append(result);
}
else //didn't find a property with that name, so be gracious and put it back
{
sb.Append("{");
sb.Append(g.Value);
sb.Append("}");
}
startIndex = g.Index + g.Length +1 ;
}
if (startIndex < aFormat.Length) //include the rest (end) of the string
{
sb.Append(aFormat.Substring(startIndex));
}
return sb.ToString();
}
}

Enjoy.



Greatest Hits I've officially been blogging for six years this week. Yikes! It didn't all suck, but it was pretty rough there early on and I've finally found my blog's heartbeat.

Here's a little about me if you're interested, but more importantly, here's some posts that are the most often-read according to the last several year's web server logs. If you enjoy them, consider subscribing via RSS or email. I also do a weekly podcast that you might like. It's available on iTunes and Zune and you can get the complete MP3 here. I publish an Ultimate Tools List each year at http://www.hanselman.com/tools that you might like.

Blogging

Programming

.NET

Gadgets

Africa, Travel and Language

Books and Music

Diabetes

Babies and Children

Family

General Coolness

Staying Organized and Home Office

Reviews



There were a number of interesting comments from my post on Faster Builds with MSBuild using Parallel Builds and Multicore CPUs.

Here's a few good ones, and some answers. First,

Is the developer that sits inside of VS all day debugging able to take advantage of the same build performance improvements?

Yes, I think. This post is about my own hack to make it work.

@Scott, there are still people using VS2005 and .Net 2.0 (our employers are either too cheap to upgrade or we work in non-profit[my case]) So it would be nice for posts like this to clearly denote that these are things only available in the 3.0/3.5+ world. Or should I just assume that .Net 2.0 is dead?

Certainly 2.0 is not dead, considering that 3.5 is still built on the the 2.0 CLR. Additionally, if you're a 2.0 shop, there's really no reason not to upgrade to VS2008 and get all the IDE improvements while still building for .NET 2.0SP1 as MSBUild supports multitargeting.

However, the underlying question here is, "why are you teasing me?" as this gentleman clearly wants faster builds on .NET 2.0. Well, this trick under VS2008 still works when building 2.0 stuff.

If you're using a Solution File (SLN) it's automatic, and if you're using custom MSBuild files, you can build .NET 2.0 stuff with the 3.5 MSBuild by putting

<TargetFrameworkVersion>v2.0</TargetFrameworkVersion>

under a <PropertyGroup> in your C# project. Then you can use /m and get multiproc builds. Not much changes though, because either way your build output is running on CLR2.0. (.NET 3.0 and .NET 3.5 use CLR2.0 underneath.) You will get some warnings if you try to use v3.0/v3.5 types.

It's true that parallel managed code compilation lags behind unmanaged.

Welcome, msbuild, to the 21st century. ;-) GNU make has the -j switch (the equivalent to /m here) _at least_ since the early-to-mid 1990's. We use its mingw port to drive Visual Studio 2003's commandline interface to build vcproj files and to handle dependencies, and since we got multi-core build machines, our build times have improved tremendously (we're talking about roughly 2,5 million SLOCs).

There's a number of reasons for that. For example, Visual Basic is compiling in the background all the time while you're typing in VS using a non-reentrant in-proc compiler. The in-proc C# compiler is the same way.

The Multicore MSBuild with VS Hack

This trick is unsupported because it hasn’t been tested. Off the top of our collective head, I can’t think of why it wouldn’t work OK, at least if your solution is all VB/C#. Having VC projects in your solution, especially if there are project references with the VB/C# projects, would probably not work so well. That’s because VC projects are not native MSBuild format in VS2008.

To start with, this is a totally obvious hack. We'll add an External Tool to Visual Studio to call MSBuild, then add some Toolbar buttons.

First, Tools | External Tools. We'll add MSBuild, pointing it to the v3.5 .NET Framework folder. Then, we'll add /m to the arguments for multi-proc and /v:m (where the v is verbosity and the m is minimal) to tone down MSBuild's chattiness. Finally, we'll check the Use Output Window box and set the initial directory to the current Solution's Directory.

Adding MSBuild to External Tools

Next, we'll right click on the Toolbar, and hit Customize.

External Tools in Visual Studio

Then, from the Commands tab, drag the number of the newly added External Tool (mine is #3) to the Toolbar. I also added a little Lode Runner icon to the button. Additionally, I added a Clean Solution option, just because I like it handy.

Parallel MSBuild Button in Visual Studio

You can also re-map Ctrl-Shift-B if it makes you happy, but I used Ctrl-Shift-M, mapping it to the Tools.ExternalCommand, again, in my case it is #3.

image

At this point, I can now run MSBuild out-of-proc from inside Visual Studio.

What DOESN'T Work

The only thing that doesn't work for me so far is that the errors reported by MSBuild don't appear neatly parsed in the Error List. For some, this may be a deal breaker. However, the good news is that if you double click on the errors in the Output Window you WILL get taken automatically to the right file and line number which is all I personally care about. Still, sucks.

image 

Gotchas for this Hack

If you're building against a SLN file you can use Project References or manually specified project dependencies otherwise MSBuild has no way to figure out the dependencies and you might get "file locked" or other weird concurrency errors.

Also, even if Project to Project references are set up, sometimes project authors are a bit sloppy (I've done this) and inadvertently access the same file in multiple projects, which can cause spurious file locking/access errors. If you see this behavior, turn off multiproc.

Note also that while you can do builds with MSBuild like this, if you get F5 and start a debug session the VS in-process compilers will do a quick build. It WILL figure out that it doesn't need to do anything, but it will check.

Hey, it's NOT faster for me?

If you have a small solution (<10 projects) and/or your solution is designed such that each project "lines up" in a dependency chain rather than a dependency tree, then there will be less parallelism possible.

If you're using a small number of projects, or you're using Visual Basic on small projects, which is compiling in the background anyway, you might not get big changes. Additionally, remember that we're shelling out n new instances of MSBuild, while VS compiles using the in-process versions of the languages compilers. With a small number of projects, in-proc single-core might be faster.

I've got 900 Projects and Visual Studio is Slow, what's the deal?

"Doctor, doctor, it's hurts what I do that!"

"Then don't do that"

Seriously, yes, there are limits. Personally, if you have more than 50 or so projects, it's time to break them out into sub-system SLN files, and keep the über-solution just for the build server.

The team is aware that if you have hundreds of projects in a solution VS totally breaks down. I, and many others, have made suggestions of things like a "tree of projects" where you could build from one point down, and there's ways you can simulate this with multiple-configurations, but it's kinda lame right now if you have a buttload of projects.

Here's a post I did three years ago called How do you organize your code? and I think it still mostly works today. I was using NAnt at the time, but the same concepts apply to MSBuild. Forgive me now, as I quote myself:

Every 'subsystem' has a folder, and usually (not always) a Subsystem is a bigger namespace. In this picture there's "Common" and it might have 5 projects under it. There's a peer Common.Test. Under CodeGeneration, for example, there's 14 projects. Seven are sub-projects within the CodeGeneration subsystem, and seven are the NUnit tests for each. The reason that you see Common and Common.Test at this stuff is that they are the 'highest." But the pattern follows as you traverse the depths. Each subsystem has it's own NAnt build file, and these are used by the primary default.build.

This is not only a nicer way to lay things out for the project, but also for the people on the project, most of whom are not making cross-subsystem changes.

Action Item for you, Dear Reader

If you have feedback then email the MSBuild team at msbuild@ (you know the company name) and they will listen. I warned them you might be emailing, so be nice. If you DEEPLY want multi-proc builds in the next version of Visual Studio, flood them with your (kind) requests. They are aware of ALL these issues and really want to make things better.

It's interesting as I've been at MSFT about 7 months now to realize that for something like multi-proc builds to work it involves (all) the Compiler Teams, the VS Tooling Teams, and the MSBuild folks. Each has to agree it's important and work towards the goal. Sometimes this is obvious and easy, and other times, less so.

Thanks for Dan Moseley, Cliff Hudson and Cullen Waters for their help!

Related Links



My one-hundred-and-ninth podcast is up. Recorded at the Microsoft MVP Summit, this show is about the new ASP.NET Dynamic Data subsystem that's being added to ASP.NET. I sit down with Scott Hunter, a Program Manager for ASP.NET (everyone in the division is required to be named Scott, BTW ;) ) and dig deeper into Dynamic Data.

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

If you have trouble downloading, or your download is slow, do try the torrent with µtorrent or another BitTorrent Downloader.

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Check out their UI Suite of controls for ASP.NET. It's very hardcore stuff. One of the things I appreciate about Telerik is their commitment to completeness. For example, they have a page about their Right-to-Left support while some vendors have zero support, or don't bother testing. They also are committed to XHTML compliance and publish their roadmap. It's nice when your controls vendor is very transparent.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?



UPDATE: I've written an UPDATE on how to get MSBuild building using multiple cores from within Visual Studio. You might check that out when you're done here.

Jeff asked the question "Should All Developers Have Manycore CPUs?" this week. There are number of things in his post I disagree with.

First, "dual-core CPUs protect you from badly written software" is just a specious statement as it's the OS's job to protect you, it shouldn't matter how many cores there are.

Second, "In my opinion, quad-core CPUs are still a waste of electricity unless you're putting them in a server" is also silly. Considering that modern CPUs slow down when not being used, and use minimal electricity when compared to your desk lamp and monitors, I can't see not buying the best (and most) processors) that I can afford. The same goes with memory. Buy as much as you can comfortably afford. No one ever regretted having more memory, a faster CPU and a large hard drive.

Third he says,"there are only a handful of applications that can truly benefit from more than 2 CPU cores" but of course, if you're running a handful of applications, you can benefit even if they are not multi-threaded. Just yesterday I was rendering a DVD, watching Arrested Development, compiling an application, reading email while my system was being backed up by Home Server. This isn't an unreasonable amount of multitasking, IMHO, and this is why I have a quad-proc machine.

That said, the limits to which a machine can multi-task are often limited to the bottleneck that sits between the chair and keyboard. Jeff, of course, must realize this, so I'm just taking issue with his phrasing more than anything.

He does add the disclaimer, which is totally valid: "All I wanted to do here is encourage people to make an informed decision in selecting a CPU" and that can't be a bad thing.

MSBuild

Now, enough picking on Jeff, let's talk about my reality as a .NET Developer and a concrete reason I care about multi-core CPUs. Jeff compiled SharpDevelop using 2 cores and said "I see nothing here that indicates any kind of possible managed code compilation time performance improvement from moving to more than 2 cores."

When I compiled SharpDevelop via "MSBuild SharpDevelop.sln" (which uses one core) it took 11 seconds:

TotalMilliseconds : 11207.7979

Adding the /m:2 parameter to MSBuild yielded a 35% speed up:

TotalMilliseconds : 7190.3041

And adding /m:4 yielded (from 1 core) a a 59% speed up:

TotalMilliseconds : 4581.4157

Certainly when doing a command line build, why WOULDN'T I want to use all my CPUs? I can detect how many there are using an Environment Variable that is set automatically:

C:>echo %NUMBER_OF_PROCESSORS%
4

But if I just say /m to MSBuild like

MSBuild /m

It will automatically use all the cores on the system to create that many MSBuild processes in a pool as seen in this Task Manager screenshot:

Four MSBuild Processes

The MSBuild team calls these "nodes" because they are cooperating and act as a pool, building projects as fast as they can to the point of being I/O bound. You'll notice that their PIDs (Process IDs) won't change while they are living in memory. This means they are recycled, saving startup time over running MSBuild over and over (which you wouldn't want to do, but I've seen in the wild.)

You might wonder, why do we not just use one multithreaded process for MSBuild? Because each building project wants its own current directory (and potentially custom tasks expect this) and each PROCESS can only have one current directory, no matter how many threads exist.

When you run MSBuild on a SLN (Solution File) (which is NOT an MSBuild file) then MSBuild will create a "sln.cache" file that IS an MSBuild file.

Some folks like to custom craft their MSBuild files and others like to get the auto-generate one. Regardless, when you're calling an MSBuild task, one of the options that gets set is (from an auto-generated file):

<MSBuild Condition="@(BuildLevel1) != ''" Projects="@(BuildLevel1)" Properties="Configuration=%(Configuration); Platform=%(Platform); ...snip... BuildInParallel="true" UnloadProjectsOnCompletion="$(UnloadProjectsOnCompletion)" UseResultsCache="$(UseResultsCache)"> ...

When you indicate BuildInParallel you're asking for parallelism in building your Projects. It doesn't cause Task-level parallelism as that would require a task dependency tree and you could get some tricky problems as copies, etc, happened simultaneously.

However, Projects DO often have dependency on each other and the SLN file captures that. If you're using a Visual Studio Solution and you've used Project References, you've already given the system enough information to know which projects to build first, and which to wait on.

More Granularity (if needed)

If you are custom-crafting your MSBuild files, you could turn off parallelism on just certain MSBuild tasks by adding:

BuildInParallel=$(BuildInParallel)

to specific MSBuild Tasks and then just those sub-projects wouldn't build in parallel if you passed in a property from the command line:

MSBuild /m:4 /p:BuildInParallel=false

But this an edge case as far as I'm concerned.

How does BuildInParallel relate to the MSBuild /m Switch?

Certainly, if you've got a lot of projects that are mostly independent of each other, you'll get more speed up than if your solution's dependency graph is just a queue of one project depending on another all the way down the line.

In conclusion, BuildInParallel allows the MSBuild task to process the list of projects which were passed to it in a parallel fashion, while /m tells MSBuild how many processes it is allowed to start.

If you have multiple cores, you should be using this feature on big builds from the command line and on your build servers.

Thanks to Chris Mann and Dan Mosley for their help and corrections.

Related Links

Technorati Tags: ,,


Have you ever looked at a word you use and see every day, like the word "What," and suddenly start doubting yourself and wonder if it's spelled correctly? There you go. What. Is THAT how it's spelled? That doesn't look right. Why would such a simple thing suddenly be called into question?

Anyway, there's a glut of technical books out there, and I'm starting to realize that I take their style and their existence for granted. They usually describe what some product is and what it does.

I really enjoy books/blogs and writing that spend less time on the What and more time on the So What? and Now What? and What For? I'd like to see more books that put technologies into a larger context and less on just What.

It'd be interesting to take any book title and add one of these phrases to the end, like "Professional ASP.NET 2.0, So What?" or "Ruby on Rails, Now What?" or "SQL Server, What For?"

What For?

The Ruby Way does a good job explaining What Ruby is Good For. It answers the Why, to me at least, very well. It's the best book I've found that doesn't just teach Ruby syntax, but also Ruby idioms; it's mastering idiomatic structures that brings fluency in any language, human or computer.

Programming Pearls - I used this book while teaching my C# Class at OIT even though it had nothing to do with C# because it's just that good. It's a series of collected articles from Communications of the ACM. It helped me understand what a number of things were for. I better understand computer problem solving, what Math is for (its relationship to programing, which isn't always clear to everyone)

How to be a Programmer (free) by Robert Read - This is a fun, short read that is general enough that it makes sense in most languages. If you want a CS101 (or 201) practical refresher, this is a great book that answers mostly "how to" questions, but ultimately explains what a number of technique are for.

So What?

Dissecting a C# Application: Inside SharpDevelop is a fantastic book and one of my favorites because it's a story that's full of So What? answers. They talk about what they tried, what failed, what didn't and how they moved forward. The Reviews on Amazon aren't all favorable as one reviewer says this is "an extremely difficult book from which to learn" but if you learn, as I do, from reading source with explanations of design decisions, then this (slightly old, but still useful) book is worth your time.

Programming WPF by Chris Sells and Ian Griffiths (his blog is a technical joy) is great because the authors are always trying to answer these "so what" questions. Chris is like this in real life. "So What?" is, I think, his number one question as he takes little for granted. If you read Ian's blog, you know his knowledge is deep to the point of being obscene (that's a good thing) and when you put these two guys together you get a great book that answers, for me, WPF, So What?

Now What?

OK, after I've learned a technology, Now What? What's the next step in my learning and what should I do?

The Pragmatic Programmer by Andrew Hunt and Dave Thomas is a book that answers a lot of Now What questions for me. It shows a complete, pragmatic attitude towards the software development lifecycle. I wouldn't say it's complete in an Enterprise sense, but good for medium-size projects, and most of the general kinds of things you're going to bump into.

In May, my friend and long-time compatriot Patrick Cauldwell will release Code Leader: Using People, Tools, and Processes to Build Successful Software which I read, enjoyed and happily wrote a foreword for. This is a book based on Patrick's "This I believe: The Developer Edition" post of last year. He takes many different tools and processes and answers the Now What? question.

Making Things Happen by Scott Berkun is an update to The Art of Project Management and I now have both editions. Scott answers lots of the Now What questions in a comfortable, conversational tone with a deep focus on getting things done in software development. He helps with planning and execution.

Your Turn

This is a woefully short list. Perhaps I'm missing something, but other than actually doing it (and failing and trying again) there aren't a lot of books that describe how to build large, complete applications that support the entire software development lifecycle.

What books have you read that answer these So What, Now What, and What For questions?

Related Links



I was at the ALT.NET Open Spaces Conference in Seattle last week. I LOVE Open Spaces conferences, my first was Foo Camp, and I've been to two ALT.NET Conferences. When Open Spaces is done right, it's a brilliant self-organizing thing of beauty. If you want to hire an Open Spaces Facilitator for your own conference, you might consider Steven "Doc" List. He did a fantastic job keeping us on track and keeping the energy level both high and positive.

These are the four Open Spaces principles:

  • Whoever comes are the right people
  • Whatever happens is the only thing that could have
  • Whenever it starts is the right time
  • When it's over, it's over

if it sounds a little meta-physical, it is, but it works. Anyone can convene a space/talk and anyone can participate. It's less about personality and power and more about people and discussion and it matches the ALT.NET way of thinking perfectly.

Dave Laribee coined the term ALT.NET last year. I did a podcast on ALT.NET with Dave last month. A few weeks before this year's conference even began, Jeremy Miller blogged about the need for ALT.NET. In a nutshell he said:

Specifically, I’d like to see ALT.NET start to fill what I see as a void for information and leadership on:

  • OOP fundamentals. I think we as a community would achieve far more benefits on our projects from a stronger base of OOP design knowledge than we do for learning new API’s like WCF or LINQ. Learning the fundamentals will make us far better able to choose and use these new API’s.
  • Development processes and practices. Microsoft builds and sells tools, and they naturally see tools as the solution to most problems. We the community should be completely in charge of determining our best practices because we are the ones out there building software. Go to a .Net centric development conference. Outside of some token Agile track, you will not see very much about configuration management, project management, requirements, testing, or the all important people issues. A lot of Linq to XYZ talks, but not much about running successful software projects.
  • Alternative tools. Some of the tools Microsoft makes are great, but some are inferior to non-Microsoft options. Some of these alternative tools open up entirely new possibilities in how we can work.  I’d like to see the entire .Net community to simply broaden its horizons to be more accepting of tools originating from outside of Redmond.

I think he’s right on. Not everything MSFT does is the “last word,” Microsoft can do better on prescriptive guidance and process, and many .NET programmers could do to refresh their knowledge of Computer Science 101 Fundamentals, myself included.

Also, ALT.NET isn't about Microsoft vs. The World, as Ayende pointed out last year:

Saying things like "An ALT.NET developer would be using Castle Windsor before Enterprise Libraries ObjectBuilder.", or "An ALT.NET developer was using NHibernate before the Entity Framework." is giving the wrong impression. It gives the impression of you are either with us (good) and against us (bad). And you must follow Our (notice the royality speak) way and no other.

The other objection is that it is focusing on tools and not on a mind set. The way I see it, this is much more about keeping your head open to new approach and ideas, regardless of where they come from. In short, I really like the ideas and concepts that Dave presents, I don't want the idea to turn into "A .NET developers that seeks to use non Microsoft technologies." I would much rather it be "A developer that seeks to find the best tools and practices, and judge them on merit."

I think some folks at Microsoft perceive the ALT.NET crowd as being loud, small, divisive, or all of the above. Deep passion and belief can sometimes be perceived as loud or argumentative. I think a better way to put it would be "pragmatic." More and more MSFTies, ScottGu included, get this. I enjoy being a part of this group.

ALT.NET is about picking processes and tools that work for you and make you happy, picking the best one or a number of them, and using them all together, so I took a moment this afternoon and whipped up an idea that a bunch of us at the conference had late one night: The ALT.NET Geek Code.

Here's mine (note the DIV tag and CLASS in the source. Any designers want to whip up a nice CSS box, with a small ALT.NET Logo? Thanks to Simone for the nice CSS and layout and Adam Kinney for the Silverlight Badge!)

Get Microsoft Silverlight

What's it mean? Well, it's a code that describes the processes and tools that I like to use to develop (as of this moment...these things tend to evolve.) You can get your own ALT.NET Geek Code and post it on your blog if you like. Perhaps we'll be able to use Google to scrape the Interwebs and do a distributed ALT.NET Tools Survey someday. ;)

Should you care about ALT.NET? Even if you aren't into the cult of personality, you can definitely learn something from getting involved in the conversation.

UPDATE: I also want to point out that I'm intending to use the term "Guys" in this post's title in the gender-neutral usage. I am acutely aware of the need for more women in this field, so much so that I convened a talk at this year's ALT.NET Open Spaces Conference called "What's with all these White Guys" which was attended by the both the women who came to the conference this year.

Related Links

Technorati Tags: ,