Scott Hanselman

Subtle Behaviors in the XML Serializer can kill

May 25, '06 Comments [3] Posted in XmlSerializer | Bugs
Sponsored By

Dan Maharry is/was having a heck of a time with the XmlSerializer after upgrading an application from .NET 1.1 to .NET 2.0.

Given this XSD/schema:

<element name="epp" type="epp:eppType" />
<complexType name="eppType">
  <choice>
    <element name="hello" />
    <element name="greeting" type="epp:greetingType" />
  </choice>
</complexType>

The .NET 1.1 framework serializes a greeting element thusly (actually by incorrect and lucky behavior in the 1.x serializer):

<?xml version="1.0" encoding="utf-8"?>
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
  <greeting>
    <svID>Test</svID>
    <svDate>2006-05-04T11:01:58.1Z</svDate>
  </greeting>
</epp>

but although it seemed to be fine initially in .NET 2.0, he started getting this instead.

<?xml version="1.0" encoding="utf-8"?>
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
  <hello d2p1:type="greetingType" xmlns:d2p1="http://www.w3.org/2001/XMLSchema-instance">
    <SvID>Test</SvID>
    <svDate>2006-05-04T10:55:07.9Z</svDate>
  </hello>
</epp>

Dan worked with MS Support and filed a bug in the Product Feedback labs and attached an example if you'd like to download it.

Unfortunately, this isn't a bug. The problem is caused by the ordering of the elements in the original schema causing the XmlElement attributes to stack in the same order resulting in the wrong semantics:

   [System.Xml.Serialization.XmlTypeAttribute(Namespace = "urn:ietf:params:xml:ns:epp-1.0", TypeName = "eppType")]
   [System.Xml.Serialization.XmlRootAttribute("epp", Namespace = "urn:ietf:params:xml:ns:epp-1.0", IsNullable = false)]
   public class EppType
   {
      private object item;

      [System.Xml.Serialization.XmlElementAttribute("hello", typeof(object))]
      [System.Xml.Serialization.XmlElementAttribute("greeting", typeof(GreetingType))]
      public object Item
      {
         get
         {
            return this.item;
         }
         set
         {
            this.item = value;
         }
      }
   }

The problem is that the semantics of the schema and the resulting XmlSerializer attributes say "This object can be either an object or a GreetingType." Well, a GreetingType IS an object, so the 2.0 serializer happily obliges.

Reversing those two lines in the XSD and regening the CS file with XSD.EXE expresses the correct intent. "This object can be a GreetingType or any other kind of object." and the expected (original) output is achieved. If Dan can't change the original schema (which is likely wrong) then he'll have to change the generated code to get the semantics he wants. Not a bad thing, actually. I did the same thing with the code generated from the OFX schemas.

Using a previously published tip called HOW TO: Debug into a .NET XmlSerializer Generated Assembly I add an app.config with these lines:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.diagnostics>
    <switches>
      <add name="XmlSerialization.Compilation" value="1" />
    </switches>
  </system.diagnostics>
</configuration>

And check the contents of the Temp Directory by going Start|Run and typing in "%temp%" and pressing enter. I then sort by Date Modifed.

The contents of my temp folder

I run the test program twice, once the original way and once with the lines reversed (my "fix") and diff the geneated .cs files in BeyondCompare.

Debuggingserializer3

You can see from the picture above exactly where the difference is, in the middle of a series of if/elseifs that basically are saying "what kind of object is this?"

The XmlSerializer is glorious and wonderful until it's totally not. I know that's not going to make Dan or his team feel better, but hang in there, it gets better the more you use it.

UPDATE: Dan has an interesting update that points out that the order of the attributes generated isn't regular, nor is the order they come back via reflection. James weighs in as well. My solution worked because there were only two attributes. Nutshell - order matters, but it's not regular.

I'm not defending the XmlSerializer folks, although it may sound like I am. James says "it looks like a bug to me." Personally I think it's less a bug and more a complex and poorly documented edge case that highlights the funamental differences between the XML type system and the CLR type system. At the edges, it's dodgey at best.

I think where we're all getting nailed here is that that XSD Type System can represent things that the CLR Type System can't. Full stop.

In Schema, xs:choice is a complex thing, much like unions in C. The XmlSerializer chooses to present xs:choice as a Object that you have to downcast yourself. The mapping is uncomfortable at best. However, beyond this one uncomfortable type mapping, there are structures you can present in Schema that simply have no parallel in the CLR and the mappings won't ever been 100%. This is just what happens when translating between type systems. The same thing happened(s) for years with nullable DB columns as simple types got translated into the CLR and we leaned on IsDBNull. With the XmlSerializer they introduced the whole and parallel field with a "Specified" suffix.

In this instance, if it were me using this schema and dealing with these documents I'd switch over to implementing IXmlSerializable. IXmlSerializable provides coverage for the final few percent that the XmlSerializer doesn't provide.  It doesn't solve the problem of mapping between type-systems, but it at least puts YOU in control of the decisions being made.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

IE 7, Office 2007, RSS and the Feed Protocol

May 24, '06 Comments [2] Posted in DasBlog | Tools
Sponsored By

FeediconbigI've been having an interesting back and forth with Sean Lyndersay, a Senior Program Manager on the RSS Team at Microsoft.

I noticed that Office 2007 - specifically Outlook 2007 - registered itself as a system-wide handler (without asking) for the feed: psuedo-protocol. Omar and I added support for feed: in DasBlog and half the users loved it and half hated it. I've always liked the idea, and the user experience of single clicking on a feed icon/link and getting a "would you like to subscribe."

Feedhandleroutlook207The IE7 release will include the underpinnings for the new system-wide feed API. However, while Outlook 2007 Beta 2 includes support for RSS within PSTs, registers for feeds, and downloads feeds in the same dialog as mail, it doesn't appear to use this system wide RSS API or the Common Feed list.

Office 2007 Beta 2 doesn't appear to install the new RSS Platform at all!

UPDATE: There is an option in Tools|Options|Other|Advanced called "Sync RSS Feeds to the System Feed List" so that may mean that while Outlook stores RSS Feeds itself, it can update its internal OPML with the Common Feed List. Presumably 'System' in this case is the Common Feed Storage. Not sure if this means sync the feed URLs or the content itself.

Hopefully this will change before release, otherwise Outlook 2007 has just added an iffy RSS reader as a feature.

2007rssHere's Outlook having registered itself as a handler, after I clicked on my own feed from within Firefox. If I selected "Remember my choice" it'd be a one-click deal.

This is a reprinted part of my back and forth email with Sean and the team, posted with permission. Perhaps the RSS Team will post a little more about the integration between the new RSS Platform, IE7, Vista Beta 2 and Outlook 2007 Beta 2.


From: Scott Hanselman
To: Team RSS
Subject: Feed Protocol

Hi,
I was surprised while running Vista 5308 that feed:// wasn't (didn't appear to be) supported in IE7. I was going to blog about it, but I wanted to get the straight story. What's the deal?
Scott


From: Sean Lyndersay
To: Scott Hanselman

Scott,

This is an interesting question. The short answer is that we think that the
feed: "protocol" is an unnecessary feature, for a few reasons:

First, IE7 itself does not need a feed: link in order to decide whether a file is a feed, since we look at autodiscovery links in webpages, mime-types, and a few other things (see: section II of http://blogs.msdn.com/rssteam/articles/PublishersGuide.aspx) to make this decision.

Secondly, the existence of the feed: "protocol" implies the notion of a default feed reader -- a single client for consuming all feeds.

With the Windows RSS Platform that ships with IE7, and in particular the common feed list, we want to deprecate the notion of a single feed reader, and instead, promote the idea that feeds can and should be consumed from multiple different applications (often simultaneously). We have provided a rich API and event system for enabling applications to be notified whenever a user subscribes to a feed. They can then use that information to add the feed to their own feed lists, or they can use the common feed store to get the content of the feed directly (these different methods of integrating with the RSS platform are documented here:
http://blogs.msdn.com/rssteam/archive/2006/02/09/528195.aspx).

Finally, registering for feed: would then mean that we would be fighting over ownership of the feed: "protocol" with various aggregators on the machine, which is something we really don't want to do. We would rather that publishers and aggregators move away from using feed:// towards simply declaring appropriate mime-types for their feeds (or, if that isn't possible, making sure that their feeds are easily detectable from the first few bytes of the file).

Hope this helps explain why IE7 doesn't register for the feed: protocol. Let me know if you have any questions.

Thanks,
Sean


From: Scott Hanselman
To: Sean Lyndersay; Team RSS

Thanks for your reply. Do let me know if I may blog selected portions of this discussion.

I understand your point of view, but there are certainly a number of other non-protocol protocols, particularly in the media space (real, windows media, iTunes) that have associated apps registered as the default. The other obvious one is mailto: even though I use hotmail, gmail, outlook express, outlook and thunderbird. If I follow your logic then a "mail store" would be in order as a platform service and the mailto: pseudo-protocol would be an unnecessary feature.

Feed: is nice because it's explicitly saying "this is a feed." You say that IE7 uses a number of techniques to (my word) "glean" if a feed is a feed. If you (Windows Platform) implemented a "feed" handler ala Open|With that simply enumerated the installed readers and provided the user with the option. Precedent for this already exist with file extension/type and mime/type handlers. Why treat RSS any differently? The idea of an "open with/subscribe" with global (chained) handler not only solves the problem of "fighting aggregators" but it respects and explicitly recognizes that there are a number of aggregators out there. Additionally it respects the fact that there are a number of OSs out there that don't have common feed storage, Windows included, and won't until everyone upgrades to IE7. With the continuing popularity of FireFox, do you anticipate folks upgrading to IE7 *just* for the RSS Platform or will that be a separate redist?

I think that by eschewing feed: you might take a considerable hit in the blogosphere as the folks who came up with feed: did it because the community asked for it. Taking it away or ignoring it I think could cause controversy.

Just my thoughts...

Scott


From: Sean Lyndersay
To: Scott Hanselman

Thanks for the thoughtful response. I'll blog about this on our blog later this week, which should give you all you need point to and blog yourself. :)

Taking this out of the philosophical and into the practical, the first point I should make is that we haven't "taken away" the feed: protocol. It still exists, any aggregator can register for it, any publisher can use it, and it will operate just as it did with IE6 (or any other browser). We just choose not to register for it.

With IE7 and Firefox 2.0 (following Safari 2.0 and Opera 8.0), all major browsers will natively support viewing RSS feeds and all will provide entrypoints for aggregators to learn of the user's intent to subscribe to a feed. For browsers that don't render feeds natively, we recommend doing what feedburner does (e.g. view this page in a non-IE7 browser http://feeds.feedburner.com/BurnThisRSS2) -- which is to provide an XSL stylesheet that provides options, including feed:, for users who can't or won't upgrade.

The fact is that feed-protocol was created for a reason, and that reason is going away. It was a less-than-optimal solution even at the time (see, for example, the creator of Feeddemon said: http://nick.typepad.com/blog/2004/06/feeddemon_and_t.html), and it is now longer the right solution.

In my own experience, sites that use feed: protocol appear to be fairly rare in practice. Of course, now that I've said that, I'm sure you'll point me to a few dozen of them :) (Scott: Just Outlook 2007 Beta2)

There are several better ways for a publisher to say "this is a feed" that fit cleanly into the HTTP design. We have recently documented them all on our blog [1].Firefox uses the same rules. We don't need an additional one.

Hope this helps.
Sean

[1] http://blogs.msdn.com/rssteam/articles/PublishersGuide.aspx.


2007instantsearchI think that we both make a number of good points, and I appreciate that Sean acknowledges that the feed: protocol isn't going away, but IE7 isn't going to support it directly and I understand his point.

That said, I suspect that the first utility I will write after installing IE7 will be a system-wide feed: protocol handler that simply adds a feed to the RSS Platform Common Feed Store. That'll satisfy my one-click fix, making use of the clean protocol handler API and the clean RSS API. It's a win-win.

I recognize that when a new Platform API ships - like the RSS Platform API - that not everyone can turn the ship around - like Office 2007 - and get on board with a new dependency. However if the Windows Desktop Search folks got their hooks into the Outlook 12 Team and got them to coerce me into installing yet another Desktop Search Engine when my preferred engine is Google Desktop, the RSS Team should be able to whip them into shape and get them onto the RSS Platform API and get them behind the party line when it comes to the feed: protocol.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Office 2007 - Everything you know is Different?

May 24, '06 Comments [6] Posted in Musings | Tools
Sponsored By

2007hotkeysI downloaded and installed Office 2007 Beta 2 and I'm on freaking Mars. I'm totally lost. Everything's changed, it's like someone broke into my office and moved everything.

Good stuff:

  • 2007statusbarOffice 2003 Hot keys and Menu keys work - I typed Alt-T|O to get to Tools|Options and got a nice tool-top letting me know that I was on the right track. (see image)
  • One you accept that everything's be moved, you can find most stuff in two clicks. For example, I always want to select cells in Excel then "Zoom to Selection." Now I click View and there's a big "Zoom to Selection" button.
  • Hotkeys Galore - Just press Alt and you'll get subtle, but obvious overlaid hotkeys on all the actionable things. Now you can more easily string together Hotkey Streetfighter Combos like "Alt-V,Z,T" enter for "Zoom to Text Width" in Word. (see image)
  • Awesome tooltips - Giant help for nearly every thing you could hover over.
  • Live Preview - Finally something for my Pentium 4 with 2 gigs of RAM to do. This rocks.
  • 2007hotkeysgaloreNew Outlook - The new Outlook is better in a number of subtle ways.
    • Calendar - Much softer, easier to look at and better use of space.
    • The To-Do Bar is brilliant, showing me what's coming up no matter where I am.
    • Much better category support (see image)
  • RSS Support in Outlook ala Newsgator -

Weird stuff:

  • The installer said it would uninstall Office 2003, and it didn't. I ended up in a weird side-by-side place where I could run everything but Outlook 2003. So, I ran the Office 2003 uninstaller. Mistake #1. That boogered up Office 2007 - however, when I ran 2007 again, the MSI installer repaired all of Office 2007's stuff automatically (albeit slowly) and I was back in business.
  • The new Office Button - I'm used to double-clicking on the upper left corner of the window and closing the application.
  • Good luck with your Outlook Add-ins, nearly all mine totally freaked out and are now disabled.
  • Right click on the Status Bar in any application and tell me what the checkboxes on the left mean versus the "On|Off" on the right. Now ask your non-technical spouse. (see image)
  • 2007tagsWord Documents that have extensive existing styles don't get those styles respected in the "Quick Style Set" ribbon bar. Instead a default set of themed styles are suggested. This could cause one with a rich existing set of styles to shoot themselves in the foot. They could fill the "Quick Style Gallery" with lots of manual right-clicking though then save the "set." It'd have been nice to have a better way to specify existing styles en masse as a set.
  • The mini-toolbar that pops up as you select text is great for simple docs, but promotes bad habits (not using styles) in larger documents.

Very weird stuff:

  • Outlook 2007 Beta 2 registers itself as a system-wide handler for the feed:// protocol, a 'protocol' that the IE7/RSS Team disagrees with and has decided against supporting. I'll post about this oddity separately.

All in all stability of Beta 2 is iffy (I've crashed 6 times in 60 minutes and got a "not implemented" dialog four times) but I'm VERY excited about the possibilties of this new User Interface style and what it means for the future.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

XPaper - Digital Paper and the Logitech io2 Pen

May 24, '06 Comments [2] Posted in Coding4Fun
Sponsored By

XpaperscreenieI've blogged before about my love for the Logitech io2 pen and there's an article on their SDK on Coding4Fun.

The Logitech Software is slick and their SDK is fantastic, but I've always thought something was missing. Writing on blank paper, taking notes and such is neat, but what if...

Recently we sold our Condo and Mike, my very cool Real Estate agent, used his Tablet PC for all the paperwork. He showed up at the condo with no actual paper, just a Toshiba Tablet and a bunch of TIFs. All of us, buyer and seller, did all the work via email and tablet. (Mike's very progressive.)

How does this relate to the io2 pen?

XpaperThere's a company called Talario that has a product called XPaper. For the technical, here's how it works:

  • You want a form filled out, signed, etc, with ink. You also want to send PDFs around and archive the papers digitally without scanning.
  • You get some Anoto-enabled paper from Talario. This paper is uniquely numbered like a guid or ip address. Each page is unique to the stack. The paper includes the Anoto dot pattern that the io2 pen can see.
  • You print from any program to the "XPaper printer" on your PC. This is a 'fake printer' that will ask you via a dialog box what number your stack of paper started with. Then it just routes to your physical printer driver while saving a digital copy on the computer for later.
  • You get a print out on this special paper of your document.
  • You write on it with the io2 pen, whatever. Mike could have brought the pen and the paper to the meeting.
  • Later he docks the pen with his computer and the digital ink from his pen is reunited with the digital copy saved earlier and turned into a PDF that includes both the original document and the digital ink lined up perfectly.

You print, ink, then dock and you get digital ink versions of your printouts. It works very well and is written entirely in .NET.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Mo's Computer says System32/Config/Software is either missing, corrupt or has an error.

May 24, '06 Comments [4] Posted in Musings
Sponsored By

ScreenshotThumbnail-ERDCommander2005"A Repair Install is not foolproof and should not be considered the cure-all fix for non-boot situations." says an XP Repair FAQ. This is true.

If you've got a spot on your carpet, should you rip up the floor and lay new carpet?

I updated the wife's computer's network drivers yesterday and got this lovely error on reboot:

Blah blah Systemroot/system32/config/Software is either missing, corrupt or has an error.

You'd think they'd know if it were missing or not. ;) I thought seriously about doing reinstall/repair install, but then thought there might be a simpler, less drastic solution.

I held down F8 and tried "Last Known Good Configuration" and that didn't work. I also couldn't get into the system via Safe Mode, but I could get into the Recovery Console. The Last Known Good and Safe Mode not working (via my gut) told me that there was probably a disk corruption error around the area of the registry file, rather than a corrupt hive.

From within the console I changed directories down to system32/config and saw that SOFTWARE (the Registry Hive) was in fact there, and was of a reasonable size for this simple machine (about 26 megs). I ran chkdsk /p (it's /p, not /f, inside the Recovery Console) and it found and fixed errors. Rebooted and was were back in business.

(Had this not worked, the next step would have been a purchase of ERD Commander)

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.