Scott Hanselman

How do Extension Methods work and why was a new CLR not required?

April 04, 2008 Comment on this post [23] Posted in Learning .NET | Programming
Sponsored By

Someone said recently that they thought extensions methods required a new CLR.

Extension methods are a new feature in .NET 3.5 (C#3/VB9) that let you appear to "spot weld" new methods on to existing classes. If you think that the "string" object needs a new method, you can just add it and call it on instance variables.

Here's an example. Note that the IntHelper35 class below defines a new method for integers called DoubleThenAdd. Now, I can do things like 2.DoubleThenAdd(2). See how the method directly "hangs off" of the integer 2? It's the "this" keyword appearing before the first parameter that makes this magic work. But is it really magic? Did it require a change to the CLR, or just a really smart compiler?

Let's do some experimenting and see if we can figure it out for ourselves.

using System;
namespace Foo
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine(2.DoubleThenAdd(3));
Console.WriteLine(IntHelper20.DoubleThenAdd(2, 3));
Console.ReadLine();
}
}

public static class IntHelper20
{
public static int DoubleThenAdd(int myInt, int x)
{
return myInt + (2 * x);
}
}

public static class IntHelper35
{
public static int DoubleThenAdd(this int myInt, int x)
{
return myInt + (2 * x);
}
}
}

I've also added an IntHelper20 class with an identical method but WITHOUT the "this" keyboard. It's a standard static method, and I call it in the standard way. Now, let's compile it, then disassemble it with Reflector and take a look at the IL (Intermediate Language).

.method private hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
L_0000: nop
L_0001: ldc.i4.2
L_0002: ldc.i4.3
L_0003: call int32 ConsoleApplication8.IntHelper35::DoubleThenAdd(int32, int32)
L_0008: call void [mscorlib]System.Console::WriteLine(int32)
L_000d: nop
L_000e: ldc.i4.2
L_000f: ldc.i4.3
L_0010: call int32 ConsoleApplication8.IntHelper20::DoubleThenAdd(int32, int32)
L_0015: call void [mscorlib]System.Console::WriteLine(int32)
L_001a: nop
L_001b: call string [mscorlib]System.Console::ReadLine()
L_0020: pop
L_0021: ret
}

Interestingly, both method calls look the same. They look like static method calls with two integer parameters. From looking at this part of the IL, you can't actually tell which one is an extension method. We know the first one, IntHelper35, is, but from this snippet of IL, we can't tell.

Can Reflector tell the difference if we ask it to decompile to C# or VB (rather than IL)?

private static void Main(string[] args)
{
Console.WriteLine(2.DoubleThenAdd(3));
Console.WriteLine(IntHelper20.DoubleThenAdd(2, 3));
Console.ReadLine();
}

Interestingly, it knows the difference. How? Here's the decompilation of the IntHelper35 class itself:

.method public hidebysig static int32 DoubleThenAdd(int32 myInt, int32 x) cil managed
{
.custom instance void [System.Core]System.Runtime.CompilerServices.ExtensionAttribute::.ctor()
.maxstack 3
.locals init (
[0] int32 CS$1$0000)
L_0000: nop
L_0001: ldarg.0
L_0002: ldc.i4.2
L_0003: ldarg.1
L_0004: mul
L_0005: add
L_0006: stloc.0
L_0007: br.s L_0009
L_0009: ldloc.0
L_000a: ret
}

The only difference between the two methods is the CompilerServices.ExtensionAttribute. This attribute is added because of the "this" keyword, and it looks like it's what Reflector is using to correctly identify the extension method.

Extension methods are a really nice syntactic sugar. They're not really added to the class, as we can see, but the compiler makes it feel like they are.

Slightly Related Aside about Object Oriented "C"

This reminded me of what we called "object oriented C" in college. I found a great example on Phil Bolthole's site.

Basically you make a struct to represent your member variables, and then you create a number of methods where the first parameter is the struct. For example:

#include "FooOBJ.h" 
void diddle(){
FooOBJ fobj;

fobj=newFooOBJ(); /* create a new object of type "FooOBJ" */

/* Perform member functions on FooOBJ.
* If you try these functions on a different type of object,
* you will get a compile-time error
*/
setFooNumber(fobj, 1);
setFooString(fobj, "somestring");
dumpFooState(fobj);

deleteFooOBJ(fobj);
}

int main(){
diddle();
return 0;
}

In this C example, if you mentally move the first parameter to the left side and add a ".", like fobj.dumpFooState() it's almost like C++. Then, you ask yourself, "gosh, wouldn't it be nice if a compiler did this for me?"

Technorati Tags: ,,

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
April 04, 2008 12:04
Hi Scott,

Great article as always. I find it very sad that so many developers still can't understand the difference between the CLR version and the .Net Framework version.

Thanks again!

StevenMcD
April 04, 2008 12:10
A lot of the new C# 3.0 features can actually be used while only targeting .NET 2.0 framework. You can even use LINQ to Objects: http://rogeralsing.com/2008/03/31/linq-to-objects-for-net-2-available/

April 04, 2008 12:28
Any chance you could sort out your code rendering in BlogLines? This post is virtually unreadable.
April 04, 2008 12:31
Hm...OK, Rich. I'll take a look. The thing I don't understand is that it's just text in a pre tag. Seriously. I think it might be Bloglines, but I'll dig into it.
April 04, 2008 12:47
To prove that from the opposite direction, I tried to replace "this" by [System.Runtime.CompilerServices.Extension], however the C#3 compiler threw error CS1112 at me (which basically means "dont be a smartass, use this instead").

But I guess it should be possible to define extension methods precisely that way with a YourFavouriteOtherNetLanguage compiler and then call them from C# or (shudder) VB.
April 04, 2008 12:51
For what it's worth, the code is also unreadable in the Outlook 2007 RSS feed.
April 04, 2008 12:58
MillKa - yes, I think you could. It'd be interesting to try that in Boo or something else.

Domenic - Ok, then the RSS feed is being changed on the way out of this blog and through FeedBurner. Sounds like enough people are seeing this, so I'll dig in.
April 04, 2008 13:00
It seems that the CompilerServices.ExtensionAttribute is the only thing preventing us from using this new language feature when targeting .NET Framework 2.0.

I am already using automatic properties (just writing "get; set;" in properties and getting a field and some code for free) in my old projects targeting 2.0, and it just seemed natural for extension methods to be used just using the new compiler and targeting the old framework, because they are "syntactic sugar" too.
The compile just does his best, but it is not possible: "Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll?" :D

Ok, they *should* have annotated the method in order to be recognized by VS (and other tools) as an extension method, but they *could* have used some existing attribute (DesignerCategory, etc. Who gives a damn?) and made the code generated by the compiler more compatible with previous frameworks.
Why I must have the full 3.5 installed (or some ugly hack to copy System.Core.dll separately, is it possible?) just to use this tiny little feature?
April 04, 2008 14:57
So we have monkey-patching in .NET? Or a near-enough approximation thereof? Cool.

Oh, and the code looks OK in Google Reader, btw. A tad over-spaced, but perfectly readable.
April 04, 2008 15:45
Bahador,

you don't need 3.5 to enjoy the extension methods, nor many other new C# 3.0 features. Ultimately, it all* compiles down to CLR 2.0.

I'm not speaking theoretically here - we have a product running on 2.0 using extension methods, auto-properties, anonymous classes, lambdas, linq to objects (we had to "manually" implement the extension methods we needed, mostly for IEnumerable).

*if you want to use linq to db, you'll probably want to avoid reinventing most of the stuff in the System.Core.*, thus you'll end up dependent on 3.5.
April 04, 2008 16:57
Very interesting. If anyone is interested, I did a 5-post serie on extension methods that explain good practices, design issues and testability: http://www.nventive.net/blog
April 04, 2008 17:10
Thanks for writing this. It was really helpful. I thought I had read somewhere that Extension Methods required a new CLR. I might have also just assumed that this .NET version of monkey patching was like monkey patching in Ruby, and therefore actually opens the class again. It is really amazing what can be done with a clever compiler.
April 04, 2008 18:07
Almost like having an open class a la Ruby. The resulting syntactic sweetness is nice, but the recipe feels awkward to me. It reminds me of adding binary operators to a C++ namespace:


// Append a string to a Path with a path separator, creating a new path.
Path operator/(const Path& a_path, const std::string& new_dir);


I'm not a C# user, but I expected something a bit smoother for the developer.

Regarding "object-oriented C", we did something similar while I was working for my brother on a chip testing harness. We extended your example even more by adding function pointers to the structs [so calls would be ptr->foo(ptr, arg);]. I also seem to recall some unions in there as well to lend a bit of pseudo-polymorphism.

And for what it's worth, the code formatting is messed up in Google Reader too.
April 04, 2008 19:13
Excellent post. Now, what I've been trying to figure out is: can you have an extension method that adds a static method to another class? For instance, I want to add FormatSql() to string (so I can say string.FormatSql("...", ...)) - is this possible?
April 04, 2008 19:41
You can use extension methods with the "this" argument in .NET 2.0 projects too.
It's just a matter of creating your own System.Runtime.CompilerServices.ExtensionAttribute.

See Daniel Moth's blog post.

Unbelievable that it works :)
April 04, 2008 19:41
Drazen,
I think C# compiler requires CompilerServices.ExtensionAttribute in order to compile extension methods, which is located in of System.Core (3.5). It compiles down to CLR 2.0 but requires a type from a 3.5 library which makes them dependent to whole 3.5 framework.

Automatic properties, lambdas, and var keyword all work fine, but there exists a problem with extension methods.
April 05, 2008 0:03
Bahador - it looks like you posted around the same time as Daniel, so you missed his last comment. Yes, the compiler needs to be able to resolve a class named System.CompilerServices.ExtensionAttribute. However, it does not have to be the same class from System.Core.dll. It just has to be a class with that name, which you are free to create.
April 05, 2008 0:43
Daniel and Joshua,

Amazing! :D
Thanks.
April 07, 2008 22:27
Hi Scott! Thanks for a clear explanation of how extension methods work internally.
Don't you think that from ideological point of view, extension methods provide the same functionality as JavaScript's prototypes? Wasn't the whole idea taken from JS prototypes?
April 08, 2008 2:08
I was a little dissapointed when i learnt how method extentions work in C# 3..
However on a side note , when i was at LANG.NET symposium 2008.. I saw a demo of a really cool project called re-motion mixins . lang.net's side has their presentation on line, and i think its definately worth watching for a pattern to be able to do this stuff more dynamically.

-Karl
April 09, 2008 19:40
This is infact what the Boo programming language did to support extension methods in .NET 2.0. With the release of the 3.5 framework, the implementation has been adjusted to support the CLR exposed extension attribute, but the initial Boo implementation similarly just added an attribute to inform the compiler that it should be considered an extension.
April 10, 2008 12:59
I love these extension methods. I'd love them even more if I could provide an extension method for the default constructor of selected types. Is that possible? I'm thinking about AOP without having to instanciate objects through a factory.
oo

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.