Scott Hanselman

The Weekly Source Code 27 - Suck Less Libraries

May 21, 2008 Comment on this post [10] Posted in Programming | Source Code
Sponsored By
I've been getting more and more interested in how folks extend their applications using plugins and things. In my new ongoing quest to read source code to be a better developer, Dear Reader, I present to you twenty-sixth (half a year!) in a infinite number of posts of "The Weekly Source Code."

Now that I've been working with the .NET Base Class Library (BCL) for just about six years, I've got a pretty good sense of the classes I like, the ones that work well, and the ones that don't "feel" right. I'm learning LINQ and the various extension methods that can make code feel more elegant. I've got ideas on how I might fix/patch/improve the framework, but really, who has the time? Well apparently these guys. This week's Weekly Source Code is dedicated to the people who create libraries with the goal of making other libraries suck less.

Umbrella

The folks at http://www.nventive.net/ have created a project called Umbrella that is a series of helpers over the .NET BCL that aims to "reduce friction and increase predictability of the API." They plan to launch additional accelerators over Unity, the Entity Framework, WCF and the Enterprise Library.

It is exceedingly broad and includes literally hundreds of new methods and helpers. However, I don't think you're expected to "learn" Umbrella as they hope you'll stumble into it...in a good way. It's like the concept of The Pit of Success. There's little easier than just falling, and the idea is that if you've designed an API correctly folks will just fall into success. 

Do check out the project, it's really fun reading. Here's some cool examples:

Here's a few extensions to ICollection.

namespace nVentive.Umbrella.Extensions
{
public static class CollectionExtensions
{
public static void AddRange<T>(this ICollection<T> collection, IEnumerable<T> items)
{
items.ForEach(collection.Add);
}

public static void Remove<T>(this ICollection<T> collection, Func<T, bool> predicate)
{
collection.Where(predicate).ToArray().ForEach(item => collection.Remove(item));
}

public static ICollection<U> Adapt<T, U>(this ICollection<T> collection)
{
return Adapt<T, U>(collection, Funcs<U, T>.Convert, Funcs<T, U>.Convert);
}

public static ICollection<U> Adapt<T, U>(this ICollection<T> collection, Func<U, T> from, Func<T, U> to)
{
return new CollectionAdapter<T, U>(collection, from, to);
}

public static void ReplaceWith<T>(this ICollection<T> collection, IEnumerable<T> items)
{
collection.Clear();
AddRange<T>(collection, items);
}

public static IDisposable Subscribe<T>(this ICollection<T> collection, T item)
{
collection.Add(item);

return Actions.Create(() => collection.Remove(item)).ToDisposable();
}

public static void AddDistinct<T>(this ICollection<T> collection, T item)
{
AddDistinct<T>(collection, item, Predicates<T>.Equal);
}

public static void AddDistinct<T>(this ICollection<T> collection, T item, IEqualityComparer<T> comparer)
{
AddDistinct<T>(collection, item, comparer.Equals);
}

public static void AddDistinct<T>(this ICollection<T> collection, T item, Func<T, T, bool> predicate)
{
if (collection.None(collectionItem => predicate(collectionItem, item)))
{
collection.Add(item);
}
}
}
}

Ones like AddRange() are simple enough to make you wonder why there weren't there before, but ones like Remove() that includes a predicate indicating which items to remove are clever and clean, as in this example that removes numbers that are evenly divisible by two.

[Fact]
public void Remove()
{
collection.AddRange(new int[] { 1, 2, 3, 4 });

collection.Remove(item => item % 2 == 0);

Assert.Equal(2, collection.Count);
Assert.Equal(1, collection.ElementAt(0));
Assert.Equal(3, collection.ElementAt(1));
}

Another fun example is their ReflectionExtensions that make Reflection a little cleaner. Here's a test:

[Fact]
public void Instance()
{
Foo foo = new Foo();

IReflectionExtensionPoint fooReflection = foo.Reflection();

Assert.Equal(foo.I, fooReflection.Get("i"));

fooReflection.Set("i", 2);

Assert.Equal(2, foo.I);

Assert.Equal(2, fooReflection.Get("I"));

fooReflection.Set("I", 3);

Assert.Equal(3, foo.I);

Assert.Equal(3, fooReflection.Get("GetI"));

fooReflection.Set("SetI", 4);

Assert.Equal(4, foo.I);
}

Again, this just scratches the surface of Umbrella. It's huge.

Ukadc.Diagnostics

Even the team knows it as they say "Great logging, terrible name."  The Ukadc.Diagnostics library, also hosted on Codeplex, is a library of extensions to System.Diagnostics. Why? From Josh Twist:

Colleagues from Microsoft's UK Application Development Consulting (UKADC) team. The project was born out of a conversation with Morgan Skinner when I mentioned that I tended to use log4net, nlog or Enterprise Library logging over the stuff that ships out of the box (OOB) with .NET 2+.

I am the same way as I was a log4net guy for years. Check out their Reference Example or the Complete Source. The reference shows the smtpTraceListener and the sqlTraceListener. It includes a nice example of how to use the Trace.CorrelationManager to do context tracing through an entire HTTP request lifecycle.

The library includes a better base class for TraceListener, CustomTracelistener, that makes creating your own trace listeners easy with just two methods to override. I encourage you to check it out.

However, the gem I found while reading the source wasn't actually directly related to the library's purpose. Rather, it's the implementation of a Fast Property Getter. What's that you say? Well, sometimes when you're using Reflection to get properties from an object, you'll realize that asking for, then using the PropertyInfo object directly can be slow. Most folks will at least cache the PropertyInfo, which will save you from the initial hit but you'll still pay a little when using it to access the object's value.

This DynamicPropertyReader takes a CLR type as a string and a property name as a string. Rather than using Reflection each time, which would be slower, it uses this FastPropertyGetter. We did this at Corillian (my last job) in a number of places where we wanted the flexibility of Reflection but near-native speeds.

public DynamicPropertyReader(string sourceType, string propertyName)
{
if (string.IsNullOrEmpty(sourceType))
throw new ArgumentNullException("sourceType");
if (string.IsNullOrEmpty(propertyName))
throw new ArgumentNullException("propertyName");

// Attmpt to get the type, and throw a TypeLoadException if not found
Type type = Type.GetType(sourceType, true);

_fastPropertyGetter = new FastPropertyGetter(propertyName, type);

if (_fastPropertyGetter.PropertyType == typeof (string))
{
_comparator = StringComparator.Instance;
}
else if (typeof (IConvertible).IsAssignableFrom(_fastPropertyGetter.PropertyType) &&
typeof (IComparable).IsAssignableFrom(_fastPropertyGetter.PropertyType))
{
_comparator = NumericComparator.Instance;
}
else
{
throw new InvalidOperationException(string.Format(CultureInfo.CurrentCulture,
Resources.DoesNotImplementRightInterfaces,
propertyName, sourceType));
}
}

What does it do? It's long, but clever. I think it could actually be tightened up a bit, but basically it generates a method at runtime that is tailored to retrieve a property from an object. That getter is then cached and called to retrieve the property. One of these DynamicMethods is generated per property. Remember that DynamicMethod is one of the magical things that enables IronPython and the DLR. I heard that this kind of technique could be WAY cleaner using the DLR in the near future. Perhaps Harry can expand on that. There's a good article on CodeProject about this kind of specialized code generation for speeding up reflection.

public class FastPropertyGetter
{
private delegate PropertyResult GetData(object data);

private GetData GDDelegate;

public Type PropertyType { get; private set; }

public FastPropertyGetter(string propertyName, Type type)
{
if (string.IsNullOrEmpty(propertyName))
throw new ArgumentNullException("propertyName");
if (null == type)
throw new ArgumentNullException("type");

PropertyInfo propertyInfo = type.GetProperty(propertyName);

if (propertyInfo == null || !propertyInfo.CanRead)
throw new ArgumentException(
string.Format(CultureInfo.CurrentCulture, Resources.NoReadableProperty, propertyName, type));

MethodInfo methodInfo = propertyInfo.GetGetMethod();
PropertyType = methodInfo.ReturnType;

DynamicMethod dm = new DynamicMethod("FastGetProperty",
typeof (PropertyResult),
new Type[] {typeof (object)}, typeof (FastPropertyGetter));

ILGenerator il = dm.GetILGenerator();

il.DeclareLocal(typeof (PropertyResult));
LocalBuilder lb = il.DeclareLocal(type);

il.Emit(OpCodes.Newobj, typeof (PropertyResult).GetConstructor(new Type[0] {}));
il.Emit(OpCodes.Stloc_0);
il.Emit(OpCodes.Ldarg_0);

il.Emit(OpCodes.Isinst, type);

// branch if IS an instance of type
Label isInstance = il.DefineLabel();
il.Emit(OpCodes.Brtrue_S, isInstance);

{
// load the GetterResult
il.Emit(OpCodes.Ldloc_0);

// Create a 'false' bool
il.Emit(OpCodes.Ldc_I4_0);

// Set result code to GetterResultCode.ObjectNotExpectedType
il.Emit(OpCodes.Callvirt, typeof (PropertyResult).GetProperty("ObjectMatched").GetSetMethod());

// reload the GetterResult and return it
il.Emit(OpCodes.Ldloc_0);
il.Emit(OpCodes.Ret);
}

// isInstance:
// object IS expected instance
il.MarkLabel(isInstance);

{
// load the GetterResult
il.Emit(OpCodes.Ldloc_0);

// load the passed object
il.Emit(OpCodes.Ldarg_0);

if (type.IsValueType)
{
il.Emit(OpCodes.Unbox_Any, type);
il.Emit(OpCodes.Stloc_1);
il.Emit(OpCodes.Ldloca_S, lb.LocalIndex);
}

if (methodInfo.IsFinal)
{
// Call the property get method
il.Emit(OpCodes.Call, methodInfo);
}
else
{
// Call the property get method
il.Emit(OpCodes.Callvirt, methodInfo);
}

// box if necessary
if (methodInfo.ReturnType.IsValueType)
{
il.Emit(OpCodes.Box, methodInfo.ReturnType);
}

// Set data to result of query
il.Emit(OpCodes.Callvirt, typeof (PropertyResult).GetProperty("Data").GetSetMethod());

// reload the GetterResult and return it
il.Emit(OpCodes.Ldloc_0);
il.Emit(OpCodes.Ret);
}
GDDelegate = (GetData) dm.CreateDelegate(typeof (GetData), null);
}

public PropertyResult GetValue(object data)
{
return GDDelegate(data);
}
}

IIS7 Hostable Web Core

CarlosAg has written a nice wrapper around IIS7's core hwebcore.dll library that lets you host your own IIS7 instance INSIDE your process. This isn't HttpListener, nor is it out-of-process. This is IIS7, running in your process. This is a really powerful thing that is as simple as:

internal class Program { 
private static void Main(string[] args) {
int port = 54321;
int siteId = 1;

WebServer server = new WebServer(@"d:\Site", port, siteId);
server.Start();
Console.WriteLine("Server Started!... Press Enter to Shutdown");

Console.ReadLine();

Console.WriteLine("Shutting down");
server.Stop();
}
}

This is a nice alternative to HttpListener and it's more powerful than Visual Web Developer/Cassini. Carlos also muses about this possibility:

"Another scenario might include something like a "Demo/Trial CD" where you can package your application in a CD/DVD that users then can insert in their machine and suddenly get a running/live demo of your Web Application without requiring them to install anything or define new applications in their real Web Server."

It's also very very simple:

namespace HWCServer {
internal class WebServer : IDisposable {

private string _appHostConfigPath;
private string _rootWebConfigPath;

public WebServer(string physicalPath, int port, int siteId) {
string appPoolName = "AppPool" + port.ToString();
_appHostConfigPath = Path.Combine(Path.GetTempPath(), Path.GetTempFileName() + ".config");
_rootWebConfigPath = Environment.ExpandEnvironmentVariables(@"%windir%\Microsoft.Net\Framework\v2.0.50727\config\web.config");

string appHostContent = Resources.AppHostAspNet;
//string appHostContent = Resources.AppHostStaticFiles;

File.WriteAllText(_appHostConfigPath,
String.Format(appHostContent,
port,
physicalPath,
@"%windir%\Microsoft.NET\Framework\v2.0.50727",
siteId,
appPoolName));
}

~WebServer() {
Dispose(false);
}

public void Dispose() {
Dispose(true);
}

private void Dispose(bool disposing) {
if (disposing) {
GC.SuppressFinalize(this);
}

Stop();
}

public void Start() {
if (!HostableWebCore.IsActivated) {
HostableWebCore.Activate(_appHostConfigPath, _rootWebConfigPath, Guid.NewGuid().ToString());
}
}

public void Stop() {
if (HostableWebCore.IsActivated) {
HostableWebCore.Shutdown(false);
}
}

#region Hostable WebCore
internal static class HostableWebCore {

private static bool _isActivated;


private delegate int FnWebCoreActivate([In, MarshalAs(UnmanagedType.LPWStr)]string appHostConfig, [In, MarshalAs(UnmanagedType.LPWStr)]string rootWebConfig, [In, MarshalAs(UnmanagedType.LPWStr)]string instanceName);
private delegate int FnWebCoreShutdown(bool immediate);

private static FnWebCoreActivate WebCoreActivate;
private static FnWebCoreShutdown WebCoreShutdown;

static HostableWebCore() {
// Load the library and get the function pointers for the WebCore entry points
const string HWCPath = @"%windir%\system32\inetsrv\hwebcore.dll";
IntPtr hwc = NativeMethods.LoadLibrary(Environment.ExpandEnvironmentVariables(HWCPath));

IntPtr procaddr = NativeMethods.GetProcAddress(hwc, "WebCoreActivate");
WebCoreActivate = (FnWebCoreActivate)Marshal.GetDelegateForFunctionPointer(procaddr, typeof(FnWebCoreActivate));

procaddr = NativeMethods.GetProcAddress(hwc, "WebCoreShutdown");
WebCoreShutdown = (FnWebCoreShutdown)Marshal.GetDelegateForFunctionPointer(procaddr, typeof(FnWebCoreShutdown));
}

public static bool IsActivated {
get {
return _isActivated;
}
}

public static void Activate(string appHostConfig, string rootWebConfig, string instanceName) {
int result = WebCoreActivate(appHostConfig, rootWebConfig, instanceName);
if (result != 0) {
Marshal.ThrowExceptionForHR(result);
}

_isActivated = true;
}

public static void Shutdown(bool immediate) {
if (_isActivated) {
WebCoreShutdown(immediate);
_isActivated = false;
}
}

private static class NativeMethods {
[DllImport("kernel32.dll")]
internal static extern IntPtr LoadLibrary(String dllname);

[DllImport("kernel32.dll")]
internal static extern IntPtr GetProcAddress(IntPtr hModule, String procname);
}
}
}
}

What a clean wrapper class! Have fun, and keep reading source to be a better coder!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
May 21, 2008 3:04
Scott,

I'm folowing your Dynamic Data screencasts and have fallen at the first hurdle. I'm using the VS2008 SP1 Beta bits. After creating a L2S datacontext and placing it's type in global.asax I run the project. First page shows the grid of tables as in your 'cast. Clicking on the Products table (or any others for that matter) throws an exception:

[InvalidCastException: Unable to cast object of type 'WebApplication1.NorthWindDataContext' to type 'System.Data.Objects.ObjectContext'.]
System.Web.UI.WebControls.EntityDataSourceView.ConstructContext() +391
System.Web.UI.WebControls.EntityDataSourceView.ExecuteSelect(DataSourceSelectArguments arguments) +269
System.Web.UI.DataSourceView.Select(DataSourceSelectArguments arguments, DataSourceViewSelectCallback callback) +19
System.Web.UI.WebControls.DataBoundControl.PerformSelect() +142
System.Web.UI.WebControls.BaseDataBoundControl.DataBind() +73
System.Web.UI.WebControls.GridView.DataBind() +4
System.Web.UI.WebControls.BaseDataBoundControl.EnsureDataBound() +82
System.Web.UI.WebControls.CompositeDataBoundControl.CreateChildControls() +72
System.Web.UI.Control.EnsureChildControls() +87
System.Web.UI.Control.PreRenderRecursiveInternal() +44
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Control.PreRenderRecursiveInternal() +171
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +842

The global.asax.cs contains:

model.RegisterContext(typeof(NorthWindDataContext), new ContextConfiguration() { ScaffoldAllTables = true });

Is this a breaking change in the SP1 drop of Dynamic data, or am I being dumb?

Dave



May 21, 2008 4:31
Awesome stuff, Scott. Thanks!
May 21, 2008 8:52
Dave - A little off topic, ;) but: This looks like the using the wrong data model in the wrong project type. Note you have a L2S datacontext but the errors come from the EntityDataSource. There are two project templates for Dynamic Data: Dynamic Data and Dynamic Data Entites. For doing Linq2Sql models you need to use the Dynamic Data project. For doing EntityFramework models you need to use the Dynamic Data Entities. We have two different project types because the templates and pages are a little different based on which model type is used.
May 21, 2008 9:33
Cool trick with the FastPropertyGetter. Takes me back to the days when we used to stuff NOP (0x90) instructions into an algorithm at runtime to short-circuit a branching operation at once the branch had been computed. It felt wrong then but it made for some screamin' code. Still sort of feels wrong but you can't argue with the results. All 3 of these tips will help. Thanks, Scott.
May 21, 2008 9:53
You might want to check out my Dynamic library on CodePlex. It does some really nice reflection-based stuff, and it's faster than native C# :) here
May 21, 2008 10:57
The reflection based trick is nice but the new released Enterprise Library 4 has also its own share of tricks implemented to make logging damn fast (not reflected properties but who knows what we can do with Unity and build plans).
The new improvements in template based formatting (http://geekswithblogs.net/akraus1/archive/2008/05/20/122277.aspx) have speeded up the formatting over a factor 10. That should help to have great flexibility with very good logging throughput.

Yours,
Alois Kraus
May 21, 2008 17:45
Hi Scott,

Thanks for talking about the project. Funnily enough I was planning on posting about how you could re-write the FastPropertyGetter using Expression Trees in .NET 3.5:

A new FastPropertyGetter - with Lambda and Expression Trees

Josh
May 21, 2008 19:34
Heh - I was thinking along the same lines as Josh. Here is an alternative using lambdas compiled at runtime. This is obviously much easier to maintain than using DynamicMethod directly, but I'd be curious to see a performance comparison between the two approaches.

There are 3 versions below depending on how many types you know at compile-time.


using System;
using System.Linq.Expressions;

namespace LambdaExample
{
public class Example
{
public static void Main(params string[] args)
{
string s = "asdf";

var fpg0 = new FastPropertyGetter(typeof(string), "Length");
int stringLength0 = (int)fpg0.GetValue(s);

var fpg1 = new FastPropertyGetter<string>("Length");
int stringLength1 = (int)fpg1.GetValue(s);

var fpg2 = new FastPropertyGetter<string, int>("Length");
int stringLength2 = fpg2.GetValue(s);
}
}


// no types known at compile time
public class FastPropertyGetter
{
private Delegate _fn;

public FastPropertyGetter(Type objectType, string propertyName)
{
ParameterExpression pe = Expression.Parameter(objectType, "o");
LambdaExpression expr = Expression.Lambda(Expression.Property(pe, propertyName),pe);
_fn = expr.Compile();
}

public object GetValue(object data)
{
return _fn.DynamicInvoke(data);
}
}

// object-type known at compile time
public class FastPropertyGetter<TObject>
{
private Func<TObject, object> _fn;

public FastPropertyGetter(string propertyName)
{
ParameterExpression pe = Expression.Parameter(typeof(TObject), "o");
Expression<Func<TObject, object>> expr =
Expression.Lambda<Func<TObject, object>>(
Expression.Convert(Expression.Property(pe,propertyName), typeof(object)), pe);

_fn = expr.Compile();
}

public object GetValue(TObject data)
{
return _fn(data);
}
}

// both object-type and property-type known at compile time
public class FastPropertyGetter<TObject,TProperty>
{
private Func<TObject, TProperty> _fn;

public FastPropertyGetter(string propertyName)
{
ParameterExpression pe = Expression.Parameter(typeof(TObject), "o");
Expression<Func<TObject, TProperty>> expr =
Expression.Lambda<Func<TObject, TProperty>>(
Expression.Property(pe, propertyName), pe);

_fn = expr.Compile();
}

public TProperty GetValue(TObject data)
{
return _fn(data);
}
}
}
May 21, 2008 22:01
Nice work Scott,
How would you use this in a class that inherits System.IComparable so that you could simplify the code instead of using a long switch/case statement.

switch (SortExpression)
case "Id": ...

to something more like: return MyGenericType(SortExpression).CompareTo....

Do you know what I mean?
June 04, 2008 18:45
Dave's post of 5/20 with the error: Unable to cast object of type 'xxx' to type 'System.Data.Objects.ObjectContext' was the same problem I was struggling with.

Thanks for the solution (wrong project type). I certainly didn't see that as the issue.

You saved me!!

Thanks!!

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.