Scott Hanselman's ComputerZen.com

In October of 2008 I took an informal survey on Twitter. I wanted to get an idea of what features of the .NET Framework people were using.

Also, here's the disclaimer. I did this on a whim, it's not scientific, so the margin of error is +/-101%. That said, the results feel intuitively right to me, personally.

I put the poll out again last week, adding only Silverlight to the end as an option. I realize I could have added many other subsystems and choices, but I felt it would have made this new poll too different from the original. There's certainly many ways that it could be improved as a survey, but it's best to think of it more as a "which direction is the wind blowing" question, than a survey per se.

I also didn't push/promote this survey very hard, so it got only about 1250 responses, vs. the nearly 5000 from last year, but I've kept the same color and attempted to keep the scale so one could extrapolate trends visually.

Here's the original survey:

It's also worth noting that 'NHibernate' was written into the "other" option 24 times. The poll was taken with TwtPoll.

Here's my conclusions.

• WinForms remains popular but WPF is closing the gap.
• ASP.NET MVC is nearly as popular as ASP.NET WebForms. Remember, however, that my readership 'skews Alpha' so might be more likely to be using MVC.
• ADO.NET Data Services is starting to get some of the appreciation it deserves, but the existence of ADO Datasets persists.
• Lots of folks use Silverlight, in this example set, even more than WPF.

What are your conclusions and analysis?

Sometimes, in my job, I go onsite at partners and work with them, sometimes architecturally, sometimes doing proofs of concepts to make sure they're comfortable with things working together.

This week I’m onsite at a large enterprise and one of the things they wanted to see, amongst many, was .NET interoperating with an existing Web Service. It's not important what platform their Web Service is running on, but it's not Windows and .NET. What was important was that they had WSDL and XSDs for the service, which put them above 99% of the Web Services I come upon in the enterprise.

The team here said that this particular web service used WS-Security and was a compliant web service. I figured, and told them, no problem. That's something .NET is good at. Moving angle-brackets around is something both I, and .NET do pretty well. I figured we had a number of options.

In this scenario was I going to be the Client, I could use:

• WCF - svcutil.exe - good
• System.Web.Services - wsdl.exe - pretty good
• WebClient/XDocument/XmlDocument - not so good, but workable.

You get the idea. There were a few things wrong, though.

Bad-ish WSDL

They gave me the WSDL and when I ran svcutil.exe on it, I got this error (the elements have been changed to protect the innocent.)

C:\Users\Scott\Desktop\foo>svcutil foo.Wsdl foo.xsd /config:app.config
Microsoft (R) Service Model Metadata Tool
[Microsoft (R) Windows (R) Communication Foundation, Version 3.0.4506.2152]
Copyright (c) Microsoft Corporation.  All rights reserved.

Error: Cannot import wsdl:binding
Detail: The WSDL binding named FooBinding is not valid because no match for
operation GetFooDetails was found in the corresponding portType definition.
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:foo:v1']/wsdl:
binding[@name='FooBinding']

Error: Cannot import wsdl:port
Detail: There was an error importing a wsdl:binding that the wsdl:port is dependent on.
XPath to wsdl:binding: //wsdl:definitions[@targetNamespace='urn:foo:v1']
/wsdl:binding[@name='FooBinding']
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:foo:v1']
/wsdl:service[@name='FooService']/wsdl:port[@name='FooPort']

I googled binged around for this to no avail. After staring at the file long enough, I realized that while this is a lousy error message (to be clear) it was telling me (obscurely) what was up all the while.

Here's a snippet of what I was looking at:

    <Type name="FooType">
        <operation name="FooSearch">
            <input  message="tns:FooSearchRequest"></input>
            <output message="tns:FooSearchResponse"></output>
            <fault name="FooFault" message="tns:FooFault"></fault>
        </operation>
    </Type>

    <binding name="FooBinding" type="tns:FooType">

        <soap:binding style="document" trans="http://schemas.xmlsoap.org/soap/http"></soap:binding>

        <operation name="FooSearch">
            <soap:operation soapAction=""></soap:operation>
            <input name="FooSearchRequest">
                <soap:body use="literal"></soap:body>
            </input>
            <output name="FooSearchResponse">
                <soap:body use="literal"></soap:body>
            </output>
            <fault name="FooFault">
                <soap:fault name="FooFault" use="literal"></soap:fault>
            </fault>
        </operation>
...

The key was that their WSDL didn't have the name="" attribute on the input and output elements of the operation. The name needs to line up to the operation name in the binding.

<Type name="FooType">
    <operation name="FooSearch">
        <input   name="FooSearchRequest" message="tns:FooSearchRequest"></input>
        <output  name="FooSearchResponse" message="tns:FooSearchResponse"></output>
        <fault name="FooFault" message="tns:FooFault"></fault>
    </operation>
</Type>

Once these new name="" attributes were added, I was able to generate my client-side stubs. I had to edit their WSDL, which sucks. However, you might argue svcutil.exe could chill out. Either way, a speed bump.

Claiming Compliance

I was told the Web Service would use WS-Security and a usernameToken. However, the actual message seemed like it was missing something.

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/12/secext" xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility" xmlns:tns="urn:foo:v1" xsi:schemaLocation="http://www.w3.org/2003/05/soap-envelope http://www.w3.org/2003/05/soap-envelope/soap-envelope.xsd urn:foo:v1 com.foo.messages.v1.xsd">
<soapenv:Header>
   <wsse:Security>
      <wsse:UsernameToken>
         <wsse:Username>secret</wsse:Username>
      </wsse:UsernameToken>
   </wsse:Security>
</soapenv:Header>
<soapenv:Body>
   <tns:FooRequest>
...

It's been a while (about 18 months) since I did any WCF and WS-Security, but UsernameToken really needs to have a Password element also. Additionally, when you're using WS-Security, you typically get WS-Addressing, etc along for the ride. There's other headers I'd expect to see.

I trudged on, built up the message and tried to send it off. First problem was that the endpoint URI I had was http, not https. It's not possible to send a UsernameToken in plain-text - the system explicitly forbids it. However, their system was setup to default to basic HTTP. Some gnashing of teeth and I found an SSL endpoint I could use. However, it's a hassle to debug SSL traffic. I usually use ProxyTrace or TCPTrace but with SSL, not so much.

Sniffing SSL Traffic with a Proxy

I ended up using Charles, an HTTP Proxy that can act as a man-in-the middle, issue an SSL cert, then decrypt the traffic, and forward it along to the real endpoint. However, the SSL Cert Charles issues isn't from a certificate authority, so I had to make a Policy to blindly (temporarily) accept all certificates:

internal class AcceptAllCertificatePolicy : ICertificatePolicy
{
    public AcceptAllCertificatePolicy(){}

    public bool CheckValidationResult(ServicePoint sPoint,
        X509Certificate cert, WebRequest wRequest, int certProb)
    {
        return true; //Always accept            
    }
}

Then I apply it in this (obsolete, but easy) way:

ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

Now I can run all my traffic through my local man-in-the-middle. I can set the proxy in my config file:

<basicHttpBinding>
    <binding name="FooBinding" 
      ... 
      proxyAddress="http://BigAssLaptop:8888"
      useDefaultWebProxy="false">

or in my own binding:

WSHttpBinding oldBinding = new WSHttpBinding();
oldBinding.ProxyAddress = new Uri("http://BIGASSLAPTOP:8888");

FooPortTypeClient svc = new FooPortTypeClient(oldBinding, new EndpointAddress("https://example.com/foo/v1"));

This let me see the outgoing request. I noticed immediately that my WCF client was sending a LOT more stuff that I needed.

Breaking the Rules

It was hard for the client to hear, but here's the deal. They were using the usernameToken element, alone, in the WS-Security namespace in the style of an apiKey. You often see these kinds of APIs in the Web 2.0 world, when intense security isn't needed. You get a key that's unique to you, basically a GUID, and it also acts as a tracker for the provider.

However, this isn't how WS-Security usernameTokens work, or are supposed to work. Perhaps a better way would have been for them to use a custom soap:header, rather than trying to tunnel "apikey" semantics into an existing token.

At this point, regardless of relative-wrongness, I still need to get the WCF client to talk to this unusual endpoint. I could use one of the other XML mechanism available, or, gasp, a StringBuilder, but since I wasn't having trouble with the body of the message, just the envelope.

This essentially means that I wanted WCF to do something incorrect, on purpose. After a call to Steve Maine and team, along with some general freaking out, I was able to get WCF to spit out JUST a usernameToken, like this.

WSHttpBinding oldBinding = new WSHttpBinding();
oldBinding.Security.Mode = SecurityMode.TransportWithMessageCredential;
//Just the username
oldBinding.Security.Message.ClientCredentialType = MessageCredentialType.UserName;
//And basically nothing else
oldBinding.Security.Message.NegotiateServiceCredential = false;
oldBinding.Security.Message.EstablishSecurityContext = false;

//oldBinding.ProxyAddress = new Uri("http://BIGASSLAPTOP:8888");
//oldBinding.UseDefaultWebProxy = false;

//remove the timestamp
BindingElementCollection elements = oldBinding.CreateBindingElements();
elements.Find<SecurityBindingElement>().IncludeTimestamp = false;

//sets the content type to application/soap+xml
elements.Find<TextMessageEncodingBindingElement>().MessageVersion = MessageVersion.Soap12;
CustomBinding newBinding = new CustomBinding(elements);
FooPortTypeClient svc = new FooPortTypeClient(newBinding, new EndpointAddress("https://example.com/foo/v1"));
FooRequest req = new FooRequest();
//...etc...now it's just request and response.

Unfortunate, but I'll put this configuration of a custom binding, and hopefully when they fix it, it'll be a configuration change. This at least got us to a point where I can reliably call their web services.

Long day, but interesting stuff.

As web programmers, we use a lot of strings to move data around the web. Often we’ll use a string to represent a date or an integer or a boolean. Basically "1" in instead of 1, or "April 1, 2009" rather than a proper ISO-8601 formatted culture-invariant date.

While these strings are flying around via HTTP it's not a huge deal, but sometimes this loose, even sloppy, use of strings can leak into our own code. We might find ourselves leaving the data times as strings longer and longer, or not even bothering to convert them to their proper type at all. This problem is made worse by the proliferation of JSON, and schema-less/namespace-less XML (that I've often called "angle-bracket delimited files" as they're no more useful than CSVs in that point.

.NET 4.0 is pretty much locked down, but version 4.1 still has some really cool "Futures" features that are being argued about. If we don't know the type of a string, or we want to leave the string, as a string, longer than usual, what if we had an class that could be both a string and another type, essentially deferring the decision until the variable is observed. For example:

StringOr<int> userInput= GetUserInput("Quantity"); 
string szUserInput=userInput.StringValue; 
int intUserInput=userInput.OtherValue; 

Sometimes you just don't know, or can't know.

This reminds me of a similar, but orthogonal physics concept, that of the Heisenberg Uncertainty Principle. Sometimes you know that an object is a string, and sometimes you know how long it is, but you can’t know both at the same time.

One of my favorite jokes goes:

Heisenberg gets pulled over by the police. The officer asks, “Do you know how fast you were going?” Heisenberg answers, “No, but I know exactly where I am!”

This library doesn't solve THAT issue, with respect to strings, but we’ve got folks in DevDiv working on this and many other metaphysical - and physical - problems as they apply to computer science.

Big thanks to Eilon, who's working hard to get this pushed into the .NET 4.1 Base Class Library. Visit Eilon's blog for more details on this new library, more code, graphics and details on how Intellisense will handle this new case.

Hopefully, someone is working to make this important new library Open Source.

Your thoughts, Dear Reader?

Related Posts

• Rob Conery on StringOr<T> and Duck Typing and the Probably() extension
• Eilon Lipton's RFC and Analysis with Pictures and Code Samples
• Jonathan Carter on the ramifications to String-based User Input

In an ongoing series on different reasons that I suck, I wanted to share another interesting quasi-debugging story about this public ADO.NET Data Services (Astoria) service I'm trying to help get live at a large company.

I'd mentioned how I had some trouble with UrlScan and Astoria Services in my post on "RTFLF - Read the Expletive Log File."

UrlScan 3.1 is a security tool that restricts the types of HTTP requests that IIS will process. By blocking specific HTTP requests, the UrlScan 3.1 security tool helps to prevent potentially harmful requests from reaching applications on the server. UrlScan 3.1 is an update to UrlScan 2.5 supports IIS 5.1, IIS 6.0 and IIS 7.0 on Windows Vista and Windows Server 2008.

UrlScan is a really valuable tool and a really powerful way to slap down evil and attacks on your Web Server. It is VERY configurable and very powerful and takes a clear mind to make sure that all your application's URLs work well while still preventing evil. Learn how to setup UrlScan here.

When we moved the service onto a production machine, it turned out that their production UrlScan.ini was locked down more than development. They were disallowing single quotes in QueryStrings as well as Dots in Url Paths.

The trick was that they haven't partitioned their sites up by subdomain, or website, just AppDomain. This means that they have 40+ "logical sites" running under one instances of IIS6, which means that they can't setup a separate IIS site with their own configuration and instance of the UrlScan ISAPI Filter. They also didn't feel like making a separate domain or subdomain just for our one little service.

OK, blah blah blah, background, right? So where did things go wrong?

URLScan has a section in the UrlScan.ini file called [AlwaysAllowedUrls] that does just that. It allows access to any URL in that section. Adding a URL there will make UrlScan completely ignore it.

I added our service to this section like this:

[AlwaysAllowedUrls]
/foo/MyService.svc

And I thought this would work. It DID allow us to request that Url. However, no other Astoria queries worked.

When you traverse Entities in ADO.NET Data Services (Astoria), your URLs look like this, for example:

/foo/MyService.svc/Cars()
/foo/MyService.svc/Cars()$filter=((Visible%20eq%20true)

And I didn't initially see why these were being blocked.

The issue, pointed out to me by the IIS Team's Wade Hilmo (check out his blog) was obvious (hence: I suck). The URL isn't the file, it's everything before the ?. It's everything before the QueryString. This is one of those "Duh" moments, and was my bad. I know the difference, of course, and understand Urls fully, but I completely brain-farted this obvious point when looking at ADO.NET Data Service queries and had been mentally parsing them wrong for hours. Once Wade set me straight, configuring UrlScan.ini was trivial:

In the ADO.NET Data Services parlance, the Entities exist after the .svc filename, but before the ? delimiter. Once we added those to our UrlScan.ini file, we were able to maintain the existing state of "super locked-down-ed-ness" while still allowing our services to run.

It is not possible to have a SQL Injection attack on a ADO.NET Data Service, due to the complete query rewrite that happens server side when a URL query comes into the system. The analogy is to think of in the same way that a LINQ query goes through a complete query re-write or a translation layer before it actually gets executed as SQL on the server. This re-write or translation is the layer of protection against SQL Injection attacks.

I was happy to learn that I could have my Cake ADO.NET Data Services and my UrlScan too.

A buddy of mine and I had a nice slap in the face yesterday. I was helping him deploy an ADO.NET Data Service to a large company's staging server  and we were seeing REALLY odd behavior.

We'd request something like /myservice.svc and get a 404. But we could request /myservice.svc/Stuff or /myservice.svc/?metadata.

We settled in to debug this. We thought we were "getting down to basics." You know, you've done this. The conversation goes something like:

"Ok, people, what's the definition of insanity? Trying the same thing and expecting a different result."

"Right...let's challenge all our assumptions. Let's start from scratch. Can get Hello World working?"

"What's the ACLs on that file? Is the .svc extension registered? Are we sure we have the right version of .NET?"

We were both tired and we wasted a couple of hours basically dicking around, hitting Refresh and hoping for another solution. I wanted to plugin procexp and filemon and get down to some serious CSI: IIS-type debugging, but here's the rub: We didn't have access to the machine. Only "large company's" guys were allow to touch anything. We could make suggestions, watch a SharedView session, but the human latency of the whole process was slowing us by a factor of at least five.

But I can't blame it all on the process. In retrospect, it was my fault. I'm a good debugger. I know this and I'm happy to say it. However, I can recognize a ninja when I see one. Well, if you can see the ninja, maybe they aren't a ninja, but still. I reached out to a real debugging ninja. What did he do that I was missing?

I ignored a basic tenet of debugging. It wasn't that I didn't RTFM. I didn't RTFLF.

My debugger-ninja-friend started out by simply asking us to Start|Run and type "LogFiles."

At this point I realized that this process was going to make me look and feel like an idiot. My internal lights went on and I realized my buddy and I hadn't bothered to check any log files. We'd been treating IIS like it was a black box. It's not. It logs the hell out of everything that goes in and out if you want it to.

We were trying to debug a 404 on this .svc. We opened the log in Notepad, went to the bottom, searched up for ".svc" and there it was:

This Page was blocked by Microsofts URL Scan 3.0 Reason=Dot-in-path-detected

You could have knocked me over with a feather. I've said myself, UrlScan is step 0. If you're debugging a weird 404, UrlScan is the first and most obvious place to look and it was all there, in the log files. You remember, the log files I never looked at. ;)

Did my ninja friend know or care? No, because he RTFLF. A painful reminder to me as I wasted a bit of a ninja's time. Everyone knows, don't piss off a ninja. He was cool about it though.

Today's Lesson: Whatever it was, it was probably logged. Try there first.

* Pic from Dr. McNinja.