Scott Hanselman

The Weekly Source Code 19 - LINQ and More What, Less How

March 13, '08 Comments [19] Posted in Learning .NET | LINQ | Microsoft | Programming | Source Code
Sponsored By

Dear Reader, I present to you nineteenth in a infinite number of posts of "The Weekly Source Code."

At Mix, Clint Rutkas and I were messing around writing a plugin model for one of his apps. We were prototyping and I typed up this typical-looking plugin style code:

string[] filesToTest = Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "plugins"), "*.dll");
foreach (string file in filesToTest)
{
    Assembly a = Assembly.LoadFrom(file);
    foreach (Type t in a.GetTypes())
    {
        if (!t.IsAbstract && t.BaseType == typeof(MyBaseClass))
        {
            myListOfInstances.Add((MyBaseClass)Activator.CreateInstance(t, credentials));
        }
    }
}

It's pretty straightforward and has little (read: no) error handling. It spins through the DLLs in the /plugins folder, loads them each (this should be more explicit rather than a blanket load of all dlls in a folder), then looks for any types that are derived from MyBaseClass that aren't abstract, then instantiates them and puts them in a list.

I'm going to quote Jeff Moser's very good blog post "What does it take to be a grandmaster" in both this post and the next one I write. It's that good. Seriously, go read it now, I'll wait here.

Jeff invokes Anders Heilsberg (Father of C# and LINQ) and:

"Anders' recent statement that future versions of C# will be about getting the programmer to declare "more of the what, and less of the how" with respect to how a result gets computed. A side effect of this is that your "chunks" tend to be more efficient for the runtime, and more importantly, your brain."

I'm going to repeat part I bolded. I like Programming Languages that allow me to declare "more of the what, and less of the how."  This how we tried to design the system at my last job and I automatically mentally migrate towards systems that encourage this kind of thinking.

So, back to the foreach loops above. My good friend (and fellow baby sign language fan) Kzu came by while Clint and I were working and suggested we turn this increasingly complex procedure into a LINQ query.

After a few tries, here's what we came up with.

myListOfInstances =
    (from file in Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "plugins"), "*.dll")
     let a = Assembly.LoadFrom(file)
     from t in a.GetTypes()
     where !t.IsAbstract && t.BaseType == typeof(MyBaseClass) 
     select (MyBaseClass)Activator.CreateInstance(t, credentials))
    .ToList();

This is all one line, with some whitespace for readability. It says the same thing as the first chunk of code, but for some minds, it might be easier to read. This isn't a very good example of LINQ shining, from my point of view even though I like reading it. Certainly there's nowhere (I can see) for me to put error handling code that catches a bad load or bad cast in this example. There are, however, a LOT of places within that single line that one could set a breakpoint.

A better example would be like the one Jeff uses:

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = 0;
for (int i = 0; i < primes.Length; i++)
{
primeSum += primes[i];
}

...or even...

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = 0;
foreach (int i in primes)
{
  primeSum += i;
}

...could be more easily written and read like:

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = primes.Sum();

where .Sum() is part of LINQ as Arrays are IEnumerable.

I totally recommend you go get the free, no-installer LINQPad that comes with 200 examples from C# in a Nutshell.

Here are a few examples that I think really typify pretty code; certainly an improvement over the standard procedural code I'd usually write:

var names = new[] { "Tom", "Dick", "Harry" }.Where(n => n.Length >= 4);

Here's onen that does quite a few things to a list in a fairly clean syntax:

string[] names = { "Tom", "Dick", "Harry", "Mary", "Jay" };

var results =
     (
       from n in names
       where n.Contains("a")  // Filter elements
       orderby n.Length       // Sort elements
       select n.ToUpper()     // Translate each element (project)
     ).ToList();       	

What LINQ-style queries have you written that feel more what than how?

Related, ahem, LINQS

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Thursday, March 13, 2008 7:47:36 PM UTC
You are turning more and more functional with each blog post. I thought the first bit of code was very intuitive and nicely written, while the LINQ way was just sad. It reminded me of the C style versus Basic/Pascal talks when the C programmers were doing amazing stuff in only one line. Incidentally, C won the battle, but how many of us really do use one liners when three or ten lines are easier to read?
Thursday, March 13, 2008 7:47:52 PM UTC
Scott,

Here is a bit of both the How and then the What:

public static IEnumerable<Control> Flatten(this ControlCollection controls)
{
foreach (Control control in controls)
{
yield return control;
control.Controls.Flatten();
}
}

Now for the What:

LinkButton linkButton = cell.Controls.Flatten().FirstOrDefault(c => c is LinkButton) as LinkButton;
Thursday, March 13, 2008 7:53:42 PM UTC
Hi Scott,
Loading assemblies in current application domain to check that all are abstract or derive from specified class makes a mess. What if non of assemblies are not valid plugins? All will be stay loaded but you can't unload them. But... :) My resolution is described on my blog in polish but there is code for that article. :)

Best regards
Thursday, March 13, 2008 9:18:40 PM UTC
A really LINQable version of this would hide the mess in an IQueryable so that you could write something like this:

var myInstances=from p in Plugins.Find("plugins/") select p;


Thursday, March 13, 2008 9:25:45 PM UTC
'What' not 'How':
The 'what' instead of 'how' mindset is becoming more wide spread (good and bad). Good for coding, but potentially bad because even more developers will exist that know less about how stuff works. That aside, I do feel the movement to intent-based (or 'what') development is overdue.

Examples of intent-based movements:
- LINQ (of course)
- Workflow Foundation (on its way)
- WCF (a huge step in the right direction)
- PFX - Parallel Framework Extensions (just awesome)

The intent-based development starts to deteriorate in technologies such as Workflow and WCF when you start trying to host them, because there isn't a declaratively configurable host for the runtimes (especially one that is free). I hope that BizTalk or the Oslo initiative in general will rectify this soon?
-
Add-In:
As for the AddIn space, check out System.AddIn. It is a contract based add-in framework native to the .NET framework that handles the AppDomains. There is also a lot of guidance for implementing and understanding the AddIn framework (down to the effects of using NGEN and working with domain neutrality ). Add-In Team Blog
James F. Hanauer
Thursday, March 13, 2008 9:49:17 PM UTC
I didn't exactly write the LINQ queries - I ported them from Python - originally written by Peter Norvig at Google:

Code:
http://aspadvice.com/blogs/rbirkby/attachment/34077.ashx

Discussion and pointers to the original code:
http://aspadvice.com/blogs/rbirkby/archive/2007/08/23/Silverlight-Sudoku-with-LINQ.aspx
RichB
Thursday, March 13, 2008 10:26:00 PM UTC
You have to be very careful when you start advocating intent-based programming, because almost invariably assumptions have to be made about what the programmer intended, and we all know what assumptions do. That said, I think there is certainly plenty of room for today's "typical" development to become more intent-based. I disagree with Siderite, in that I think the LINQ example is more readable than the iterative example; however, as has been pointed out, it leaves no room for error handling or AppDomain transitions. This is a problem with LINQ in general; in trying to make everything very compact, it leaves too little room to maneuver.

Also, why does everyone keep insisting that the Sum and Where extension methods for IEnumerable<T> are part of LINQ? They may happen to implement standard query operators that LINQ defines, but IIRC, LINQ stands for Language-INtegrated Queries. What is language-integrated about calling a Sum method? The IEnumerable<T> extensions should not have been defined in the System.Linq namespace for this very reason; they are extraordinarily useful even if you never use the C# or VB LINQ syntax.
Thursday, March 13, 2008 11:13:08 PM UTC
I did find as well that the first piece of code is ok and it really only needs some kind of error handling, but the second example is just horrible.
It reminds me those 300 char long one liners written in a custom language developed for customization by our business system provider. now we have to upgrade the system, understand and change these kind of statement and it is just the worse nightmare I ever had.
I am sure that there is nicer form to put out these kind of statements that does not drive crazy the poor soul who needs to touch the code couple of years later.
Thursday, March 13, 2008 11:56:26 PM UTC
Has anyone else been having serious display quirks when they try to use LINQPad? We were playing around with it at work this afternoon and everyone who tried it lost their computer for a bit to a wonderful little display error. The start bar would shrink and move to the top, all your open windows would flicker and fade, the screen would be covered with grey then flash and then eventually you'd get your PC back so you could kill LINQPad which was still hung doing something....

You didn't run into that at all Scott?
Friday, March 14, 2008 1:33:59 AM UTC
DavidNelson - Sorry, my loose definition of LINQ usually includes the "fun sugar" that makes LINQ interesting and considerably more useful. I recognize that System.Core adds a lot of MixIns that aren't LINQ Proper.

I think you nailed it here though:

This is a problem with LINQ in general; in trying to make everything very compact, it leaves too little room to maneuver.

I do find LINQ to be right at the intersection of readable, compact and expressive more often than not, but I wanted to show the good and the bad.

George - Nope, I've never seen a problem with painting in LINQPad.
Friday, March 14, 2008 10:03:06 AM UTC
var names = new[] { "Tom", "Dick", "Harry" }.Where(n => n.Length >= 4);


Is it me, or do you have to read that a couple of times to spot that the => and >= aren't the same operator?

Although I've not come across new[] {...} before - like that a lot. Type inference saves typing - always good.
Friday, March 14, 2008 2:50:53 PM UTC
@Andrew Robinson:

That flatten method won't work. You're only returning the controls in the top-level control collection, since the return list from "control.Controls.Flatten();" is ignored.
Friday, March 14, 2008 4:09:05 PM UTC
@Scott

You're not the only one who tends to dump a lot of the new features and framework components from the latest releases of the .NET stack into the "LINQ" pool; many of the bloggers that I read (including many MSFTies) do the same thing. I point it out only because, with the "radical" new features in the latest version of the language (radical to C# anyway), the mental load being put on developers trying to keep up with the language is already severe; there is a tendency to treat things like LINQ as "magic", rather than spend the mental energy trying to understand how these advanced features work. When major players in the community don't differentiate between the various independent components which enable these advanced features, but refer to them all under one feature umbrella, it makes the feature that much more opaque to many who are just trying to keep up. I think that we best serve the community by choosing our words carefully and calling things what they are.
Friday, March 14, 2008 6:07:00 PM UTC
John, You are correct. My bad, but this works. Wonder if it can be simplified beyond this though?

public static IEnumerable<Control> Flatten(this ControlCollection controls)
{
List<Control> list = new List<Control>();
controls.Traverse(c => list.Add(c));
return list as IEnumerable<Control>;
}

public static void Traverse(this ControlCollection controls, Action<Control> action)
{
foreach (Control control in controls)
{
action(control);
control.Controls.Traverse(action);
}
}
Sunday, March 16, 2008 5:37:42 PM UTC
Well, regarding to plugin issue, I'd do it with plinq-even though it is still not in production. It will load much more faster.

This is how I do it in my blog application

Parallel.ForEach(indexContributors,
(contributor) => Parallel.ForEach(contributor.GetIndexableItems(),
(indexableItem) => indexBuilder.AddContent(indexableItem)));

Tuna Toksoz
Monday, March 17, 2008 6:35:11 PM UTC
I have to say, I love the "intent-based" programming method, and I love the Enumerable/IEnumerable extension methods that support LINQ. I love chaining them together to produce a single-pass walk instead of writing sequential foreach loops that walk redundantly.

What I am not, so far, particularly fond of, is LINQ itself. My beef with LINQ is that it is superficially similar to pure SQL, but seems hobbled by rigidity and verbosity in odd places. This has repeatedly led me to try to do things that I know I can do quickly and easily in pure SQL, but that are either bulky or impossible in LINQ without breaking it up and throwing in ugly procedural code in the middle. I guess what I'm saying is that the O/R impedance mismatch is still there when you get beyond relatively simple stuff.

So as a SQL "power user", I'd prefer to work with the straight query itself in all but the simplest cases, if I'm interacting with the DB. And for LINQ to objects, I similarly prefer the chained IEnumerable functions when things start to get more complex.
Monday, March 17, 2008 8:11:23 PM UTC
I like the LINQ syntax in the second example. It reads more like English rather than typical C# code. I'm a big fan of making code read more like English rather than programming code. In this particular context, the LINQ syntax reads much easier than the C# code, and certainly lends itself to a "what" style of programming as opposed to "how".
TheWookie
Tuesday, March 18, 2008 6:41:46 PM UTC
Speaking of designing a .NET application to handle add-ins, one of the things that was added to .NET Framework 3.5 is a System.AddIn namespace designed specifically for adding extensibility to add-ins. Check out this MSDN Magazine article - http://msdn2.microsoft.com/en-us/magazine/cc163476.aspx, it would be cool to see you and Clint refactor the application to use the AddIn Framework :)

Cheers,
-Dan
Wednesday, April 02, 2008 5:20:09 PM UTC
This blog post is about the time, we are designing plugins API for our app. Going to use some tips, thanks!
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.