Scott Hanselman

Breaking All The Rules with WCF

June 10, '09 Comments [35] Posted in Web Services | XML | XmlSerializer
Sponsored By

Sometimes, in my job, I go onsite at partners and work with them, sometimes architecturally, sometimes doing proofs of concepts to make sure they're comfortable with things working together.

This week I’m onsite at a large enterprise and one of the things they wanted to see, amongst many, was .NET interoperating with an existing Web Service. It's not important what platform their Web Service is running on, but it's not Windows and .NET. What was important was that they had WSDL and XSDs for the service, which put them above 99% of the Web Services I come upon in the enterprise.

The team here said that this particular web service used WS-Security and was a compliant web service. I figured, and told them, no problem. That's something .NET is good at. Moving angle-brackets around is something both I, and .NET do pretty well. I figured we had a number of options.

In this scenario was I going to be the Client, I could use:

  • WCF - svcutil.exe - good
  • System.Web.Services - wsdl.exe - pretty good
  • WebClient/XDocument/XmlDocument - not so good, but workable.

You get the idea. There were a few things wrong, though.

Bad-ish WSDL

They gave me the WSDL and when I ran svcutil.exe on it, I got this error (the elements have been changed to protect the innocent.)

C:\Users\Scott\Desktop\foo>svcutil foo.Wsdl foo.xsd /config:app.config
Microsoft (R) Service Model Metadata Tool
[Microsoft (R) Windows (R) Communication Foundation, Version 3.0.4506.2152]
Copyright (c) Microsoft Corporation. All rights reserved.

Error: Cannot import wsdl:binding
Detail: The WSDL binding named FooBinding is not valid because no match for
operation GetFooDetails was found in the corresponding portType definition.
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:foo:v1']/wsdl:
binding[@name='FooBinding']

Error: Cannot import wsdl:port
Detail: There was an error importing a wsdl:binding that the wsdl:port is dependent on.
XPath to wsdl:binding: //wsdl:definitions[@targetNamespace='urn:foo:v1']
/wsdl:binding[@name='FooBinding']
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:foo:v1']
/wsdl:service[@name='FooService']/wsdl:port[@name='FooPort']

I googled binged around for this to no avail. After staring at the file long enough, I realized that while this is a lousy error message (to be clear) it was telling me (obscurely) what was up all the while.

Here's a snippet of what I was looking at:

    <Type name="FooType">
<operation name="FooSearch">
<input message="tns:FooSearchRequest"></input>
<output message="tns:FooSearchResponse"></output>
<fault name="FooFault" message="tns:FooFault"></fault>
</operation>
</Type>

<binding name="FooBinding" type="tns:FooType">

<soap:binding style="document" trans="http://schemas.xmlsoap.org/soap/http"></soap:binding>

<operation name="FooSearch">
<soap:operation soapAction=""></soap:operation>
<input name="FooSearchRequest">
<soap:body use="literal"></soap:body>
</input>
<output name="FooSearchResponse">
<soap:body use="literal"></soap:body>
</output>
<fault name="FooFault">
<soap:fault name="FooFault" use="literal"></soap:fault>
</fault>
</operation>
...

The key was that their WSDL didn't have the name="" attribute on the input and output elements of the operation. The name needs to line up to the operation name in the binding.

<Type name="FooType">
<operation name="FooSearch">
<input name="FooSearchRequest" message="tns:FooSearchRequest"></input>
<output name="FooSearchResponse" message="tns:FooSearchResponse"></output>
<fault name="FooFault" message="tns:FooFault"></fault>
</operation>
</Type>

Once these new name="" attributes were added, I was able to generate my client-side stubs. I had to edit their WSDL, which sucks. However, you might argue svcutil.exe could chill out. Either way, a speed bump.

Claiming Compliance

I was told the Web Service would use WS-Security and a usernameToken. However, the actual message seemed like it was missing something.

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/12/secext" xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility" xmlns:tns="urn:foo:v1" xsi:schemaLocation="http://www.w3.org/2003/05/soap-envelope http://www.w3.org/2003/05/soap-envelope/soap-envelope.xsd urn:foo:v1 com.foo.messages.v1.xsd">
<soapenv:Header>
<wsse:Security>
<wsse:UsernameToken>
<wsse:Username>secret</wsse:Username>
</wsse:UsernameToken>
</wsse:Security>
</soapenv:Header>
<soapenv:Body>
<tns:FooRequest>
...

It's been a while (about 18 months) since I did any WCF and WS-Security, but UsernameToken really needs to have a Password element also. Additionally, when you're using WS-Security, you typically get WS-Addressing, etc along for the ride. There's other headers I'd expect to see.

I trudged on, built up the message and tried to send it off. First problem was that the endpoint URI I had was http, not https. It's not possible to send a UsernameToken in plain-text - the system explicitly forbids it. However, their system was setup to default to basic HTTP. Some gnashing of teeth and I found an SSL endpoint I could use. However, it's a hassle to debug SSL traffic. I usually use ProxyTrace or TCPTrace but with SSL, not so much.

Sniffing SSL Traffic with a Proxy

I ended up using Charles, an HTTP Proxy that can act as a man-in-the middle, issue an SSL cert, then decrypt the traffic, and forward it along to the real endpoint. However, the SSL Cert Charles issues isn't from a certificate authority, so I had to make a Policy to blindly (temporarily) accept all certificates:

internal class AcceptAllCertificatePolicy : ICertificatePolicy
{
public AcceptAllCertificatePolicy(){}

public bool CheckValidationResult(ServicePoint sPoint,
X509Certificate cert, WebRequest wRequest, int certProb)
{
return true; //Always accept
}
}

Then I apply it in this (obsolete, but easy) way:

ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

Now I can run all my traffic through my local man-in-the-middle. I can set the proxy in my config file:

<basicHttpBinding>
<binding name="FooBinding"
...
proxyAddress="http://BigAssLaptop:8888"
useDefaultWebProxy="false">

or in my own binding:

WSHttpBinding oldBinding = new WSHttpBinding();
oldBinding.ProxyAddress = new Uri("http://BIGASSLAPTOP:8888");

FooPortTypeClient svc = new FooPortTypeClient(oldBinding, new EndpointAddress("https://example.com/foo/v1"));

This let me see the outgoing request. I noticed immediately that my WCF client was sending a LOT more stuff that I needed.

Breaking the Rules

It was hard for the client to hear, but here's the deal. They were using the usernameToken element, alone, in the WS-Security namespace in the style of an apiKey. You often see these kinds of APIs in the Web 2.0 world, when intense security isn't needed. You get a key that's unique to you, basically a GUID, and it also acts as a tracker for the provider.

However, this isn't how WS-Security usernameTokens work, or are supposed to work. Perhaps a better way would have been for them to use a custom soap:header, rather than trying to tunnel "apikey" semantics into an existing token.

At this point, regardless of relative-wrongness, I still need to get the WCF client to talk to this unusual endpoint. I could use one of the other XML mechanism available, or, gasp, a StringBuilder, but since I wasn't having trouble with the body of the message, just the envelope.

This essentially means that I wanted WCF to do something incorrect, on purpose. After a call to Steve Maine and team, along with some general freaking out, I was able to get WCF to spit out JUST a usernameToken, like this.

WSHttpBinding oldBinding = new WSHttpBinding();
oldBinding.Security.Mode = SecurityMode.TransportWithMessageCredential;
//Just the username
oldBinding.Security.Message.ClientCredentialType = MessageCredentialType.UserName;
//And basically nothing else
oldBinding.Security.Message.NegotiateServiceCredential = false;
oldBinding.Security.Message.EstablishSecurityContext = false;

//oldBinding.ProxyAddress = new Uri("http://BIGASSLAPTOP:8888");
//oldBinding.UseDefaultWebProxy = false;

//remove the timestamp
BindingElementCollection elements = oldBinding.CreateBindingElements();
elements.Find<SecurityBindingElement>().IncludeTimestamp = false;

//sets the content type to application/soap+xml
elements.Find<TextMessageEncodingBindingElement>().MessageVersion = MessageVersion.Soap12;
CustomBinding newBinding = new CustomBinding(elements);
FooPortTypeClient svc = new FooPortTypeClient(newBinding, new EndpointAddress("https://example.com/foo/v1"));
FooRequest req = new FooRequest();
//...etc...now it's just request and response.

Unfortunate, but I'll put this configuration of a custom binding, and hopefully when they fix it, it'll be a configuration change. This at least got us to a point where I can reliably call their web services.

Long day, but interesting stuff.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Wednesday, June 10, 2009 9:33:52 AM UTC
It sounds so familiar :) Web Services interoperability is really just a myth, since modern WS standards are too complicated, too easy to interpret differently and are changing too fast for an open standard.
Konstantin
Wednesday, June 10, 2009 10:01:50 AM UTC
True...I will say this, though, and I say it as someone who was on the WS-I basic profile WG and someone who spent 5 years doing this stuff *before* I came to Microsoft...when I see a Web Service running on Java and Apache, I am afraid, because I *KNOW* it's going to be a hassle and through HTTP 500s for any minor transgression.
Scott Hanselman
Wednesday, June 10, 2009 11:50:30 AM UTC
"while this is a lousy error message (to be clear) it was telling me (obscurely) what was up all the while."

This IS my problem with WCF : esoteric error messages - I think some better tooling and troubleshooting mechanisms out of the box would be good to see.





Wednesday, June 10, 2009 12:19:45 PM UTC
I'm finding situations with getting WCF to bend at your will is kinda common. I think a lot of it is a misunderstanding or no knowledge of SOAP and specs that support it. Unfortunately, WCF will complain and you have to break out hacks or program low level items such as custom token serializers. I was even in a situation where I had to program a WCF service that only accepted a username for a large client because they didn't want to maintain passwords or some
nonsense like that. Btw, hope you aren't using my service :).

Peace, Steve
Steve
Wednesday, June 10, 2009 1:48:46 PM UTC
//oldBinding.ProxyAddress = new Uri("http://BIGASSLAPTOP:8888");

Very nice - we had desktop support guy that kept naming computers "Jackass" while he was building them - until one time he forgot to change the name and sent it out - Quite funny (not for him)
HB
Wednesday, June 10, 2009 2:29:25 PM UTC
I'd use Fiddler 2 to debug the HTTPS traffic...
Richard
Wednesday, June 10, 2009 2:49:11 PM UTC
I don't mean to be negative, but whenever I use WCF I *frequently* end up dealing with problems like this - while WCFs infrastructure is very thoroughly designed, it is also *very* unhelpful with configuration and deployment. Getting WCF to work with anything more than the template that VS spits out by default is usually an uphill battle.
Wednesday, June 10, 2009 3:22:05 PM UTC
I sympathize, and this applies to clients and services. I've had Java developers force me to back down from WS-* bindings in a service in favor of BasicHttp, and even then needed help with passing credentials and required changes to some naming conventions that broke their wsdl.exe equivalent... Similarly, I've had non-standard services that I had to hack WCF a little to get working. I'm not sure if it's the developers or the platform, but in practice clean interop with non-.NET services and clients is a myth.

I will say WCF has enough extensibility points that I've yet to run into a situation that was impossible. Once you get your head around it, it's cool to be able to swap out all this 'goo' around the same service code.
Wednesday, June 10, 2009 4:02:59 PM UTC
We use Username-token in the same way on my project, but at least we do not claim to be compliant :-). As far as I remember we needed to hack around a piece of infrastructure that expected the message to be signed when username was used without a password. We don't have user's passwords available as we have single sign-on based on NTLM, so this was our first step in using WS-Security.

Our plan is to move to signed SAML tokens in the next release and tick off one more compliance issue.
Tore
Wednesday, June 10, 2009 6:51:59 PM UTC
Agreed Paul. For whatever reason, what should be simple is overly complicated in WCF once you leave non-trivial "Hello World" type applications. A big part of that is the completely unhelpful exceptions which often cause people with little WCF experience to go down the wrong path in trying to solve issues. Once you've been doing it for a while you learn to just ignore the exception messages, which is never a good thing. Also, don't get me started on the horrible MSDN documentation, might as well not even have it considering a good half of it doesn't even work.

Overall, WCF is long on potential, but it just isn't there from a practical stand point. Hopefully 4.0 will fix a lot of those issues, because right now I cringe everytime I try and setup a new enviornment because I honestly have no idea if it's going to work or not.
Andrew
Thursday, June 11, 2009 9:36:27 AM UTC
Well been there done that.. we wanted to create a web service using WCF that used the ws-security username token for authentication over ssl. Problem here is that we wanted to be as much interoperable as possible for platform like .net, java, php, flash and silverlight. Bottomline is that we now do basis binding over SSL with no security enabled and have a username, password as arguments.. :-( it doesn't feel right, is isn't right but it just works on most platforms. As for most platforms it is too difficult to supply a custom soap header if the platform does not support it.
Thursday, June 11, 2009 1:01:35 PM UTC
I ended up using Microsft WSE instead of WCF for just these reasons - the documentation is poor for WCF, exceptions are unhelpful and WSE just worked without horrendous config changes. It would help if you could get WSE integration to work in Visual Studio 2008, but it looks like it's only 2005 for ever more.
Dave R.
Thursday, June 11, 2009 2:04:10 PM UTC
Hi Scott,
I'm fairly new to the .Net world but I do work with Web Services in .Net frequently as a consumer. I was curious as to why you didn't go the easy just add a web reference route for this? The attributes of name are not required in the porttype definition of the WSDL and it doesn't ever seem to be a problem when using the web reference. Was to just demonstrate the new services features in WCF? Or was it the extra complexity of the ws-security stuff? Just curious if it were kind of a best practice/technical reason versus a demo of new features type of thing.

markg
Thursday, June 11, 2009 2:08:02 PM UTC
You should give us Steve Maine's number, it could be useful :D

Seriously, i worked on a project involving JEE Web services (Server) : it worked perfectly well on first run (interoperability) except with their custom in-house proxy ... I don't remember which Linux distrib it was but we could not do anything in HTTP because of an authentification problem: when the proxy was requiring credentials, we got a timeout. And the same code just worked in HTTPS ?! Strange ...
MS support just answered that they should use ISA Server :)
Thursday, June 11, 2009 3:06:07 PM UTC
>>This let me see the outgoing request. I noticed immediately that my WCF client was sending a LOT more stuff that I needed.

Did not WCF's built-in diagnostic trace facilities allow you to do this as well? I am wondering the additional information the proxy shows that Windows Communication Foundation (WCF) Message Logging does not.

Phil Bolduc, Vancouver, BC
Phil Bolduc
Thursday, June 11, 2009 3:53:28 PM UTC
.NET-based SOAP client only plays well when it comes with asmx/svc end-points. Axis (or any other Technology rather than .NET SOAP server) does not work well with .NET proxy classes generator (AKA wsdl.exe) the are plenty of issues confirmed by MS themselves , one in the BizTalk http://support.microsoft.com/kb/891386 , the other on wsdl.exe generation : http://support.microsoft.com/kb/326790/en-us . add to that WSE which may not work with certian routers and firewalls !
Thursday, June 11, 2009 8:10:24 PM UTC
WCF sucks! And, for the record, so does WF and SSIS! And the emperor has no clothes! Thanks - I feel much better now that I let that off my chest.

Mark Brackett
Friday, June 12, 2009 3:55:06 PM UTC
Trying to get VBScript/ASP to talk to WCF made me cry.
Harvey
Sunday, June 14, 2009 4:14:53 PM UTC
Harvey, I also found consuming a WCF service from VB/ASP to be challenging. Since the code was already working well with ASP.NET Web Services, I tried following the MSDN documentation for ASPX/WCF interop, with some success (I agree with Andrew, MSDN docs on WCF need some work).

I eventually decided to lose the beyond-beyond-end-of-lifed SOAP Toolkit 3.0 and replace it with actual .NET WCF clients exposed to VB via COM Interop (this works assuming you can deploy .NET 3.5 to the server running ASP, which may not be the case in hosted environments).

p.s. - Scott, as engineers we never "break" rules; we "bend" them. :)
Monday, June 15, 2009 9:09:10 AM UTC
I think the problem is one of the leaky abstraction. We use tools that protect us from the details of the underlying system...so well in fact that I can bet that the average person developing web services haven't a lick of experience hand-coding WSDL. That's like asking a doctor who has never stitched a wound to perform open heart surgery.

Case in point. I'm writing some software to match a spec put forth by a standards body. The spec doesn't even have a standardized WSDL document...oh wait it does...The type for the operation's parameter xs:Any. Although believe you me there is a very strong type expected by the standard. It was like pulliing teeth to figure out what that was. I could tell you more about it...but I've probably revealed too much as is.

Yes the true test of Web services is when you talk cross platform interop. It's never as easy as it sounds.
Thursday, June 18, 2009 2:05:40 AM UTC
We have been working on communicating between Delphi and a WCF service the last few days. It's been a lot of trouble. Our application is done in Delphi 5 and uses a very primitive interface to communicate to web services, similar to using XMLHTTP. A big help to us lately has been using the tool soapUI, which is freely at http://www.soapui.org/. The tool takes a WSDL as input and generates sample requests, showing which parameters are optional. It also will generate a test suite or a mock service. This allows you to take sample requests and communicate with your clients service at a very low level and quickly determine that yes it really is there service that doesn't work.
Mike Vermilion
Thursday, June 18, 2009 11:13:07 PM UTC
That must be an interesting job you’re into now. Honestly, I can relate well to some terms and code you are sharing. Good thing, I have my good coach beside. LOL

Friday, June 19, 2009 2:59:13 AM UTC
I've never had so much trouble dealing with webservices. Maybe you walked into the most anti-.net shop on the planet? Our company has such a heterogenious mix of software our java webservices just had to comply with all the .net programmers TOO. To me your experience is what happens when you limit your company around one technology for solutions (either direction, open, microsoft, other...) they end up getting some tech inbreeding going on and then nothing else works with it.

My feeling that is if they only had a few types of client software they'd be better served just fixing their wsdls and schemas to be more easily interoperable. It's always tough though when you have the "Well, we're not going to change it just to suite your microsoft stuff" attitude.
Saturday, June 20, 2009 10:29:50 AM UTC
Sending user/pass without ssl is possible in WCF:

ClearUsernameBinding
Sunday, June 21, 2009 12:42:10 PM UTC
I'm trying to figure out this "googled binged" thing. So when you run into a problem, you binge on Google or does this have to do with Google not wanting it's name used as a verb? Or do you binge on an alcoholic beverage when you can't find the answer... Maybe I'm not up to the times in new terminology.
JT
Monday, June 22, 2009 4:38:34 AM UTC
The problem is not with WCF ( or Axis) but with people who choose overly complicated scenarios. WCF is far superior to Axis though because Services framework allow things people use them.

eg
- Security 95% of the time no transport security/encryption is sufficient.
- Use custom headers dont fudge it like the article , this method is very common .
- Use complicated types , inheritcance , interfaces , collections etc .. KISS and use the DTO pattern. ( DOnt pollute your business domain with the need to tranport it)

The big advantage REST has is you cant do the fancy things but everything you can do with REST you can do with WCF and keep it or even more simple.

Ben
Monday, June 22, 2009 5:42:33 PM UTC
I feel your pain. I had to connect to a WebSphere WSS. What I've got was what the request should look like and how the response would be. Two XMLs basically... I had to manually set the configuration to fit the request. Somehow they just didn't know their system and I never work with WebSphere Application Server. I did it but it took a long long time!
Toni Pihlar
Tuesday, June 23, 2009 4:37:47 PM UTC
I don't know if JT is trolling (:D), but bing.com is the MS answer to google.

Speaking on the subject of interop, I have had years of experience with consuming from both .NET and JEE various web services hosted on various platforms (from IIS to SAP to Domino to PHP to Java). Sadly, the only time I have a trouble free experience is when I am consuming a Domino web service regardless of the consuming platform. If the consumer is in Java and the Service is in .NET or if the consumer is in .NET and the Service is in Java, a fair amount of time, I have no trouble, but I have had trouble often enough to feel confident in justifying padding my estimate by at least a factor of 1.5.

Oh, and if the Web Service is hosted on SAP, you will be rewriting the WSDL by hand.
Wednesday, June 24, 2009 7:02:39 PM UTC
my two cents, anyone else have a problem with the following microsoft error "The application has encountered an unknown error and now needs to close." <----Someone needs to tell the dev to fix there lazy message.
Martin Nelson
Thursday, June 25, 2009 11:51:19 AM UTC
I have a strange problem to handle here.
At one end MS Visual studio provides a utility function called as svcutil which takes WSDL file as a parameter and generates a proxy for client and a configuration file.WSDL file has the wcf services and not webservices.
On the other hand, I am now required to call the WCF services from a flash client.Can some one help me with any proxy generator for Flash which could process the WSDL file.
A prompt reply will be appreciated.
-Rajat
Friday, June 26, 2009 7:43:11 AM UTC
Nice article as always!

I just commented to help "JT" figure out the "googled binged" thing.

The Microsoft "Live" search is renamed as "Bing" search.

Try out yourself at http://www.bing.com/

Scott crossed google, because he is so close to Microsoft (He works there!), that he is almost forced to spread words around Microsoft's latest launches etc. (Though may not be best, and thats probably what he indicated by crossing google, since he may actually had used google to search, it simply is wayyy better than BING. BING has way to go!)
Abhinav
Tuesday, July 14, 2009 9:44:37 AM UTC
Hello All

I have the solution to the problem i stated in my previous post and wanted to share the way out with all of you.
The problem was:
"Can some one help me with any proxy generator for Flash which could process the WSDL file."
Solution:
There is an add on in FLash 8.0 called as webservice connector which is able to decode the WSDL file for the webservices easily and provide the methods exposed with the parameters to be sent as input to the webservice.
however since my problem was to decode a WSDL file for the WSF services,i had to create a consolidated merged WSDL file which could be read by the webservice connector component.The follwing blog helped me in that: : http://www.arquitecturadesoftware.org/blogs/antoniocruz/archive/2007/02/06/wsdl-merger.aspx.
The blog is in portuguese so u may have to take the help of google translator.
happy coding..
Tuesday, July 21, 2009 3:22:21 AM UTC
For creating WS-Security UsernameTokens which do not contain passwords, the free Vordel SOAPbox tool is useful. It's like SOAPui except it goes deep on security token creation and stress testing, and supports JMS and Email as well as HTTP/SSL. I've written a blog post about how to create a WS-Security UsernameToken without a password using SOAPbox. Hope it is useful.
Tuesday, July 21, 2009 3:59:23 AM UTC
Not sure who the client was but it must be nice to have you as their personal sales rep and code wrangler.
Thursday, August 20, 2009 9:22:48 PM UTC
tangential, but Fiddler2 now claims to have ssl-decode support... haven't used it much but that could be your general purpose proxy tool...
Mark
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.