Scott Hanselman

Porting WPF Applications to the Microsoft Surface

February 18, 2009 Comment on this post [5] Posted in BabySmash | Windows Client | WPF
Sponsored By

What do you need to get through the Great Depression 2.0? Why, a giant computing table, of course! Sadly, the Surface SDK isn't available to folks without a real Surface (which, I say, is a bummer) so I can only tell you that developing for it is awesome. Lame, yes, I know.

I can't get you the SDK (yet?) or a Surface (I'm broke) but I can tell you what developing for it was like.

Of course, if you DO have an extra US$15,000 lying around, why not let your baby sit on it? That's my thinking.

BabySmash (Code and DevCenter here) has been on hold over the holidays, although I have a lot of big plans once I get on the other side of the Mix conference.

One of the things I talked about at my PDC BabySmash talk (WMV) was porting BabySmash to the Surface. BabySmash is a simple WPF application that responds to keyboard and mouse input. Since there's no keyboard or mouse in the standard Surface configuration, I wanted to see how had it would be to port it.

I had to deal with these changes in thinking:

  • Orientation matters. People might be sitting on opposite sides of the Surface
  • No mouse or keyboard. Many people can touch it at once.
  • Multifinger resizing. Folks expect to be able to multitouch a Surface and resize items, throw them around, etc.
  • Different application launcher model. The Surface needs an XML manifest and special icons for its launcher.

Other than these changes, a Surface app is still .NET WPF app, so I wasn't too worried.

Surface Simulator

The first step was to install the Surface Simulator. It only works on 32-bit Vista, so I had to go make myself a 32-bit machine (I'm all 64-bitted up. FTW!) which was a tiny hassle. (Although there ARE some Vista 64 Surface Simulator hacks) The Simulator is exactly what it sounds like. It's not "emulating" however, as a Surface is still a Vista machine underneath. It's more of a frame and some mouse input management to handle multiple mice and make them look like multiple fingers. This works surprisingly well. In the PDC talk, about halfway in, I plugged in three mice, plus the built in touchpad and had multiple virtual "fingers" moving items around the screen.

You create your Surface app as you would a WPF app from File | New Project, hit START and it fires up the Simulator and frames up the WPF app.

Now in my case, I already had an existing WPF application, so I needed to change the type my main Window derived from from "Window" to "SurfaceWindow" and I added an assembly reference and a few namespaces:

using Microsoft.Surface;
using Microsoft.Surface.Presentation;
using Microsoft.Surface.Presentation.Controls;

This base class adds adds all the Surfacey goodness and events and properties that my Window will need.

"ContactDown rather than KeyDown and MouseDown

I added an event for "Contact_Down" rather than watching for keyboard events and mouse clicks that would never come.

private void SurfaceWindow_ContactDown(object sender, ContactEventArgs e)
{
controller.AddFigure(this, " ", e.GetPosition(mainGrid).X, e.GetPosition(mainGrid).Y);
}

BabySmash uses an MVC model, so I'm taking the input from the View and immediately delegating to a main controller object that handles everything.

Orientation

Your app can let the system decide which side of the Surface table the user is sitting on when it starts up. This isn't a huge deal with BabySmash, since it's just shapes, but still there's some text on the screen so I wanted the app to be right-side-up when appropriate.

There are a bunch of new properties on a SurfaceWindow, and one is:

AutoOrientsOnStartup="True"

This changed the orientation of my entire application, basically turning it upside down when need-be. This did mean that I couldn't use any low-level X and Y APIs, otherwise the coords would be "lost in translation." As long as I stuck with high-level stuff like "GetPosition" to pull my X and Y's out, the Surface SDK would handle translation and everything just worked.

Application Launcher Manifest

I needed to create a few icons for my app to live in the Surface Launcher and a manifest. There's other options I didn't use, as well as ways to create animated icons, or to have your application run as an "attract mode" app, to get folks wandering by to stop and touch the Surface.

<?xml version="1.0" encoding="utf-8" ?>

<ss:ApplicationInfo
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ss="http://schemas.microsoft.com/Surface/2007/ApplicationMetadata">
<Application>
<Title>BabySmash!</Title>
<Description>BabySmash! for Surface</Description>
<ExecutableFile>c:\BabySurface\BabySurface.exe</ExecutableFile>
<Arguments></Arguments>
<IconImageFile>c:\BabySurface\Resources\icon.png</IconImageFile>
<Preview>
<PreviewImageFile>c:\BabySurface\iconPreview.png</PreviewImageFile>
</Preview>
</Application>
</ss:ApplicationInfo>

ScatterView

In the original BabySmash app I had a canvas that would hold all the shapes and letters, but I wanted things to be more Surface-like in this (hacked-together) BabySurface version.

public void AddFigure(UserControl c, double x, double y)
{
this.figuresCanvas.Children.Add(c);
}

Fortunately there's a control called a ScatterView that makes throwing stuff on the Surface insanely easy. For example:

<s:SurfaceWindow
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:s="http://schemas.microsoft.com/surface/presentation">
<s:ScatterView>
<Image Source="Toco Toucan.jpg"/>
<Image Source="Green Sea Turtle.jpg"/>
<Image Source="Desert Landscape.jpg"/>
</s:ScatterView>
</s:SurfaceWindow>

From their blog (here's a video of the Surface SDK in action):

ScatterView is a custom ItemsControl in our SDK which apps can databind or populate with any type of content. Simply sticking some Image elements in it gives you a basic Photos-like app without writing any code. By baking common manipulations into WPF SDK controls like this, we’re able to free developers up to focus on things that are unique to their apps while designers use Blend to add some custom pizazz.

I changed my AddFigure method to put the new shapes in a ScatterView. I also hooked up a "ContactLeave" event to the new item so that the application would giggle when you lift your finger.

public void AddFigure(UserControl c, double x, double y)
{
var s = new ScatterViewItem();
s.Content = c;
s.ContactLeave += new ContactEventHandler(s_ContactLeave);
this.figureScatterView.Items.Add(s);
}

This was cool, but it had a few issues. First, the ScatterView puts everything in a "Polaroid"  border by default:

microsoft-surface

The result was that my BabySmash shapes were inside a Photo Border. Easily removed with a custom style that marked a few properties as {x:Null}:

<Style x:Key="ScatterViewItemBabySmashStyle" TargetType="{x:Type s:ScatterViewItem}">
<Setter Property="Background" Value="{x:Null}"/>
<Setter Property="BorderBrush" Value="{x:Null}"/>
<Setter Property="Foreground" Value="{x:Null}"/>
...

The second issue was that while my BabySmash shapes were draggable and scattered nicely, they weren't resizable. Well, they WERE, except when you resized them, you were resizing the container - that is, resizing the now-invisible Polaroid border. I needed a box to put the shapes in that would effectively lie to the inner control and scale/size them as the outside scaled and resized. Enter the ViewBox, including with WPF.

public void AddFigure(UserControl c, double x, double y)
{
var s = new ScatterViewItem();
s.Center = new Point(x, y);
s.Style = this.Resources["ScatterViewItemBabySmashStyle"] as Style;

Viewbox v = new Viewbox();
v.Child = c;

s.Content = v;

s.ContactLeave += new ContactEventHandler(s_ContactLeave);
this.figureScatterView.Items.Add(s);
}

Link to Video of Developing on Surface from PDC 2007I put the Control in the ViewBox, and put the ViewBox+Control in a ScatterViewItem. Bam, resizable, scattered BabySmash shapes.

I also got dragging and inertia (physics) for free so you, ahem, the baby, can spin and throw shapes around as well.

This development was all done on the simulator. I never saw a real Surface until I got to PDC last year. The app ran exactly as should. I was a little worried that the real Surface was somehow different, but not so. That was a huge relief.

If you have a very visual, shape oriented application that could benefit from touch and a big screen, porting it to the Surface isn't rocket science.

Definitely check out the PC17 "Developing for Microsoft Surface" talk from PDC. There's a lot of good stuff in the PPT as well with insights into how Multitouch is going to work in .NET 4.0 and Windows 7 and how work on Surface now is driving that future.

And really, who WOULDN'T want a Big Ass Table? I, for one, would like one for the living room. ;)

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
February 18, 2009 7:25
Lucky. I really hope they push that out pretty soon- I've been itching to give it a whirl, but can't quite convince any clients they need a Big Ass Table for their LOB apps...
February 18, 2009 12:03
Hacks? Hacks?! It's a finely crafted workaround thankyouverymuch! :-)

While I agree it's a bit of a shame that the SDK is not more freely available, I do think it's restricted for good reason. Having been lucky enough to work with the Surface I'd say that actually developing for it, from a code point of view, is a relatively minor step up if you already know WPF. The real challenge comes in thinking about the massively multi-touch, collaborative, 360degree nature of interacting with the unit itself, something you really can't "get" unless you have a unit there to try your code out on, and ideally have other people around to challenge your preconceptions about "how people will use it". The sim is great to allow you to make sure "stuff works", but I have my doubts whether you could create a good Surface app without an actual Surface.. certainly first time out at least.

One thing I do wish they'd do is ship a "developer unit", that's just a standard PC of the same spec as the Surface, for developers to run the simulator on. One of the problems we had is that we could throw effects, physics etc onto the sim and it would run just fine; but when it was moved over to the more modest hardware of the Surface unit it started to chug somewhat.

Still an awesome piece of kit though.. I'm still waiting for my table shaped giftwrapped package to turn up though :-(
February 19, 2009 2:35
Why can't the Surface run normal Windows apps? That really limits its usage, doesn't it?
February 19, 2009 2:58
It *can* run regular Windows Apps, but then you'd need a keyboard and mouse. It's just a Vista box.
March 03, 2009 7:54
Hello,
its very nice.

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.