Scott Hanselman

How do Extension Methods work and why was a new CLR not required?

April 4, '08 Comments [23] Posted in Learning .NET | Programming
Sponsored By

Someone said recently that they thought extensions methods required a new CLR.

Extension methods are a new feature in .NET 3.5 (C#3/VB9) that let you appear to "spot weld" new methods on to existing classes. If you think that the "string" object needs a new method, you can just add it and call it on instance variables.

Here's an example. Note that the IntHelper35 class below defines a new method for integers called DoubleThenAdd. Now, I can do things like 2.DoubleThenAdd(2). See how the method directly "hangs off" of the integer 2? It's the "this" keyword appearing before the first parameter that makes this magic work. But is it really magic? Did it require a change to the CLR, or just a really smart compiler?

Let's do some experimenting and see if we can figure it out for ourselves.

using System;
namespace Foo
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine(2.DoubleThenAdd(3));
Console.WriteLine(IntHelper20.DoubleThenAdd(2, 3));
Console.ReadLine();
}
}

public static class IntHelper20
{
public static int DoubleThenAdd(int myInt, int x)
{
return myInt + (2 * x);
}
}

public static class IntHelper35
{
public static int DoubleThenAdd(this int myInt, int x)
{
return myInt + (2 * x);
}
}
}

I've also added an IntHelper20 class with an identical method but WITHOUT the "this" keyboard. It's a standard static method, and I call it in the standard way. Now, let's compile it, then disassemble it with Reflector and take a look at the IL (Intermediate Language).

.method private hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
L_0000: nop
L_0001: ldc.i4.2
L_0002: ldc.i4.3
L_0003: call int32 ConsoleApplication8.IntHelper35::DoubleThenAdd(int32, int32)
L_0008: call void [mscorlib]System.Console::WriteLine(int32)
L_000d: nop
L_000e: ldc.i4.2
L_000f: ldc.i4.3
L_0010: call int32 ConsoleApplication8.IntHelper20::DoubleThenAdd(int32, int32)
L_0015: call void [mscorlib]System.Console::WriteLine(int32)
L_001a: nop
L_001b: call string [mscorlib]System.Console::ReadLine()
L_0020: pop
L_0021: ret
}

Interestingly, both method calls look the same. They look like static method calls with two integer parameters. From looking at this part of the IL, you can't actually tell which one is an extension method. We know the first one, IntHelper35, is, but from this snippet of IL, we can't tell.

Can Reflector tell the difference if we ask it to decompile to C# or VB (rather than IL)?

private static void Main(string[] args)
{
Console.WriteLine(2.DoubleThenAdd(3));
Console.WriteLine(IntHelper20.DoubleThenAdd(2, 3));
Console.ReadLine();
}

Interestingly, it knows the difference. How? Here's the decompilation of the IntHelper35 class itself:

.method public hidebysig static int32 DoubleThenAdd(int32 myInt, int32 x) cil managed
{
.custom instance void [System.Core]System.Runtime.CompilerServices.ExtensionAttribute::.ctor()
.maxstack 3
.locals init (
[0] int32 CS$1$0000)
L_0000: nop
L_0001: ldarg.0
L_0002: ldc.i4.2
L_0003: ldarg.1
L_0004: mul
L_0005: add
L_0006: stloc.0
L_0007: br.s L_0009
L_0009: ldloc.0
L_000a: ret
}

The only difference between the two methods is the CompilerServices.ExtensionAttribute. This attribute is added because of the "this" keyword, and it looks like it's what Reflector is using to correctly identify the extension method.

Extension methods are a really nice syntactic sugar. They're not really added to the class, as we can see, but the compiler makes it feel like they are.

Slightly Related Aside about Object Oriented "C"

This reminded me of what we called "object oriented C" in college. I found a great example on Phil Bolthole's site.

Basically you make a struct to represent your member variables, and then you create a number of methods where the first parameter is the struct. For example:

#include "FooOBJ.h" 
void diddle(){
FooOBJ fobj;

fobj=newFooOBJ(); /* create a new object of type "FooOBJ" */

/* Perform member functions on FooOBJ.
* If you try these functions on a different type of object,
* you will get a compile-time error
*/
setFooNumber(fobj, 1);
setFooString(fobj, "somestring");
dumpFooState(fobj);

deleteFooOBJ(fobj);
}

int main(){
diddle();
return 0;
}

In this C example, if you mentally move the first parameter to the left side and add a ".", like fobj.dumpFooState() it's almost like C++. Then, you ask yourself, "gosh, wouldn't it be nice if a compiler did this for me?"

Technorati Tags: ,,

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Friday, April 04, 2008 8:04:54 AM UTC
Hi Scott,

Great article as always. I find it very sad that so many developers still can't understand the difference between the CLR version and the .Net Framework version.

Thanks again!

StevenMcD
Friday, April 04, 2008 8:10:25 AM UTC
A lot of the new C# 3.0 features can actually be used while only targeting .NET 2.0 framework. You can even use LINQ to Objects: http://rogeralsing.com/2008/03/31/linq-to-objects-for-net-2-available/

Friday, April 04, 2008 8:28:32 AM UTC
Any chance you could sort out your code rendering in BlogLines? This post is virtually unreadable.
RichB
Friday, April 04, 2008 8:31:24 AM UTC
Hm...OK, Rich. I'll take a look. The thing I don't understand is that it's just text in a pre tag. Seriously. I think it might be Bloglines, but I'll dig into it.
Friday, April 04, 2008 8:47:05 AM UTC
To prove that from the opposite direction, I tried to replace "this" by [System.Runtime.CompilerServices.Extension], however the C#3 compiler threw error CS1112 at me (which basically means "dont be a smartass, use this instead").

But I guess it should be possible to define extension methods precisely that way with a YourFavouriteOtherNetLanguage compiler and then call them from C# or (shudder) VB.
MillKa
Friday, April 04, 2008 8:51:09 AM UTC
For what it's worth, the code is also unreadable in the Outlook 2007 RSS feed.
Friday, April 04, 2008 8:58:13 AM UTC
MillKa - yes, I think you could. It'd be interesting to try that in Boo or something else.

Domenic - Ok, then the RSS feed is being changed on the way out of this blog and through FeedBurner. Sounds like enough people are seeing this, so I'll dig in.
Friday, April 04, 2008 9:00:34 AM UTC
It seems that the CompilerServices.ExtensionAttribute is the only thing preventing us from using this new language feature when targeting .NET Framework 2.0.

I am already using automatic properties (just writing "get; set;" in properties and getting a field and some code for free) in my old projects targeting 2.0, and it just seemed natural for extension methods to be used just using the new compiler and targeting the old framework, because they are "syntactic sugar" too.
The compile just does his best, but it is not possible: "Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll?" :D

Ok, they *should* have annotated the method in order to be recognized by VS (and other tools) as an extension method, but they *could* have used some existing attribute (DesignerCategory, etc. Who gives a damn?) and made the code generated by the compiler more compatible with previous frameworks.
Why I must have the full 3.5 installed (or some ugly hack to copy System.Core.dll separately, is it possible?) just to use this tiny little feature?
Bahador
Friday, April 04, 2008 10:57:59 AM UTC
So we have monkey-patching in .NET? Or a near-enough approximation thereof? Cool.

Oh, and the code looks OK in Google Reader, btw. A tad over-spaced, but perfectly readable.
Mike Woodhouse
Friday, April 04, 2008 11:45:23 AM UTC
Bahador,

you don't need 3.5 to enjoy the extension methods, nor many other new C# 3.0 features. Ultimately, it all* compiles down to CLR 2.0.

I'm not speaking theoretically here - we have a product running on 2.0 using extension methods, auto-properties, anonymous classes, lambdas, linq to objects (we had to "manually" implement the extension methods we needed, mostly for IEnumerable).

*if you want to use linq to db, you'll probably want to avoid reinventing most of the stuff in the System.Core.*, thus you'll end up dependent on 3.5.
Friday, April 04, 2008 12:57:09 PM UTC
Very interesting. If anyone is interested, I did a 5-post serie on extension methods that explain good practices, design issues and testability: http://www.nventive.net/blog
Friday, April 04, 2008 1:10:07 PM UTC
Thanks for writing this. It was really helpful. I thought I had read somewhere that Extension Methods required a new CLR. I might have also just assumed that this .NET version of monkey patching was like monkey patching in Ruby, and therefore actually opens the class again. It is really amazing what can be done with a clever compiler.
Aaron Davis
Friday, April 04, 2008 2:07:20 PM UTC
Almost like having an open class a la Ruby. The resulting syntactic sweetness is nice, but the recipe feels awkward to me. It reminds me of adding binary operators to a C++ namespace:


// Append a string to a Path with a path separator, creating a new path.
Path operator/(const Path& a_path, const std::string& new_dir);


I'm not a C# user, but I expected something a bit smoother for the developer.

Regarding "object-oriented C", we did something similar while I was working for my brother on a chip testing harness. We extended your example even more by adding function pointers to the structs [so calls would be ptr->foo(ptr, arg);]. I also seem to recall some unions in there as well to lend a bit of pseudo-polymorphism.

And for what it's worth, the code formatting is messed up in Google Reader too.
Matt McC
Friday, April 04, 2008 3:13:16 PM UTC
Excellent post. Now, what I've been trying to figure out is: can you have an extension method that adds a static method to another class? For instance, I want to add FormatSql() to string (so I can say string.FormatSql("...", ...)) - is this possible?
David Mullin
Friday, April 04, 2008 3:41:27 PM UTC
You can use extension methods with the "this" argument in .NET 2.0 projects too.
It's just a matter of creating your own System.Runtime.CompilerServices.ExtensionAttribute.

See Daniel Moth's blog post.

Unbelievable that it works :)
Friday, April 04, 2008 3:41:47 PM UTC
Drazen,
I think C# compiler requires CompilerServices.ExtensionAttribute in order to compile extension methods, which is located in of System.Core (3.5). It compiles down to CLR 2.0 but requires a type from a 3.5 library which makes them dependent to whole 3.5 framework.

Automatic properties, lambdas, and var keyword all work fine, but there exists a problem with extension methods.
Bahador
Friday, April 04, 2008 8:03:47 PM UTC
Bahador - it looks like you posted around the same time as Daniel, so you missed his last comment. Yes, the compiler needs to be able to resolve a class named System.CompilerServices.ExtensionAttribute. However, it does not have to be the same class from System.Core.dll. It just has to be a class with that name, which you are free to create.
Friday, April 04, 2008 8:43:00 PM UTC
Daniel and Joshua,

Amazing! :D
Thanks.
Bahador
Monday, April 07, 2008 6:27:07 PM UTC
Hi Scott! Thanks for a clear explanation of how extension methods work internally.
Don't you think that from ideological point of view, extension methods provide the same functionality as JavaScript's prototypes? Wasn't the whole idea taken from JS prototypes?
Monday, April 07, 2008 10:08:29 PM UTC
I was a little dissapointed when i learnt how method extentions work in C# 3..
However on a side note , when i was at LANG.NET symposium 2008.. I saw a demo of a really cool project called re-motion mixins . lang.net's side has their presentation on line, and i think its definately worth watching for a pattern to be able to do this stuff more dynamically.

-Karl
Wednesday, April 09, 2008 3:40:07 PM UTC
This is infact what the Boo programming language did to support extension methods in .NET 2.0. With the release of the 3.5 framework, the implementation has been adjusted to support the CLR exposed extension attribute, but the initial Boo implementation similarly just added an attribute to inform the compiler that it should be considered an extension.
Thursday, April 10, 2008 8:59:44 AM UTC
I love these extension methods. I'd love them even more if I could provide an extension method for the default constructor of selected types. Is that possible? I'm thinking about AOP without having to instanciate objects through a factory.
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.