Scott Hanselman

Deep Zoom (Seadragon) Silverlight 2 MultiScaleImages at Mix

March 6, '08 Comments [22] Posted in Mix | Silverlight
Sponsored By

outrun One of the cool things seen at Mix 08 today was the Deep Zoom technology. This was originally called Seadragon and came out of the Live Labs research.

You can see it live at the Hard Rock Cafe Memorabilia site. This Silverlight application includes a tiled montage of 512 many-megapixel images, some 10+megapixels, while some are stitched together to sizes upwards of 40-50 megapixels, it seems.

Remember the first 3D games like Sega's OutRun? They seemed 3D, but basically bitmaps were being scaled as you "drove forward" and each graphical asset, like a house, for example, would have a number of pre-rendered versions. Another example are .ico files that have 16x16, 32x32, etc, versions. As the viewer there is one image, but there's multiple resolutions depending on how far in you want to zoom. This is the basic idea behind "multi-scale images."

You can see examples of exceedingly large images, often called "giga-pixel" images all over the web. A great example of zooming using Flash is at Harlem 13-Gigapixels.com. The large Harlem image is 2,045 individual images of 12 megapixels each. The rendering when zooming in is a lot like Google Earth. You zoom into an image of a specific resolution then that first image transitions into a tile of a second image rendered another resolution. You can create these kinds of images with Panorama stitching software like AutoPano.net, and PanoramaFactory. I took a large 50+ megapixel panorama of Ngorongoro Crater in Tanzania that I think would make a good example for this kind of thing. I'll dig it up and make a Deep Zoom example project.

DeepZoom includes an optimized local Silverlight Renderer and XAML control that makes creating these fairly easy. Try it out over at the HardRock Memorabilia site and take specific notice of this guitar I've circled in the picture below.

Hardrock Memorabilia

Zoom in...

image

And more...into the screw knob in the lower left corner...

image

You can actually see a reflection of the bench and the camera and Scott Stanfield from Vertigo who's taking the shot of this guitar, reflected in the shiny knob. Vertigo wrote the Hardrock site and they also have a great Deep Zoom Gallery.

Project Name

You can go download the Preview of the Deep Zoom Composer now. You can see its heritage when you make a new Project and it puts it in a folder called "Seadragon Projects." Here's the Deep Zoom Composer User Guide.

You place a number of pictures (or a large one) on the surface and export the collection. There will be a scene graph XML file created indicating location, size, z-order and aspect ratio of the images.

SparseImageSceneGraph.xml - Notepad

Then an output directory will be created with folders for each image and, depending on the relative largeness of the source images, a series of tiles generated a number of zoom levels. Here's an example directory structure:

11

Silverlight Project Test Page - Windows Internet Explorer (2)See how that image at zoom level 11 has 20 different tiles? They've got to balance many factors to make it all seamless. They're considering how many tiles are needed for that resolution as it relates to a zoom level's need for memory, as well as considering a reasonable number of requests to the server. There's the extreme need for a good user-experience so you want the images to come fast and furious, be easily cached and quickly shown to give the perception of speed. You don't have to sweat any of this stuff, as the multi-resolution tiles stitched together automatically by the client rendered.

After you've exported a project into these files, you create a MultiScale image in XAML like this, referencing a bin file manifest that knows contains info about the directory structure and files:

<MultiScaleImage x:Name="deepZoomObject" Source="foo/info.bin"/>

This gets you the image rendered. To zoom a MultiScaleImage, you call ZoomAboutLogicalPoint:

public void ZoomAboutLogicalPoint(double zoomIncrementFactor, double zoomCenterLogicalX, double zoomCenterLogicalY)

But there's a decent amount of state management that needs to be done as well as deciding how you want things animated, whether or not your image contains sub-images, etc. In the morning I'll take a look at how to hook up JavaScript events like Mouse Wheel Scrolls and do some zooming and panning. Hopefully Vertigo will release their hard-won source code for the canonical example to the community. It's pretty slick.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Thursday, March 06, 2008 12:37:45 PM UTC
This seems like an impressive technology, but unfortunately none of the Deep Zoom demos I tried so far seems to work. The Hard Rock Memorabilia site produces a ManagedRuntimeError #4002 when trying to zoom in, and in the Vertigo Site Map demo the images remain fuzzy when zooming in.

Maybe it's because I've installed Vista SP1, but according to a thread on Channel 9 I'm not the only one having this problem.
Thursday, March 06, 2008 1:09:44 PM UTC
Did you come up with that example right after the announcement?? Wow, I'm truly impressed by your "groking" skills Scott!
Thursday, March 06, 2008 1:46:39 PM UTC
Scott, what happens if you don't have the images available a priori? Such as if you want to create a deep zoom where some of the images change or are added/removed over time? Can I do this?
Sean
Thursday, March 06, 2008 2:18:04 PM UTC
Here's a great video demo of this technology done last year (Mar 2007), at the TED conference. Really amazing.
Allan Nienhuis
Thursday, March 06, 2008 3:21:50 PM UTC
I so this is what Jeff Atwood (from CodingHorror) was talking about.
MLeo
Thursday, March 06, 2008 3:27:04 PM UTC
Looks exactly like this: http://www.zoomify.com/
which has been around for a few years.
Mike
Thursday, March 06, 2008 3:34:09 PM UTC
It looks like a bunch of people have got their images imported, exported into the right format - showing in SL2 but no-one has the code for mouse pan and zoom :(
I found all the methods and properties but no documentation at all. Anyone found a solution? Such a tease.
The best so far is using the keyboard here.
John.
Thursday, March 06, 2008 4:40:41 PM UTC
Quick! Someone patent "Levels of Detail...On The Internet"!
MarcT
Thursday, March 06, 2008 6:01:33 PM UTC
Why don't any of these zoom applications respond to +, - (zoom), arrow keys for scrolling, or PgUp, PgDown? Using shift or ctrl left click for zoomming in large increments doesn't give a fine enough granularity these applications deserve for laptops and others without a zoom wheel.
Jay
Thursday, March 06, 2008 7:04:04 PM UTC
Yes, it totally looks like Zoomify, I said the same thing when I saw it. I've not used either except as an amateur photographer. I did find this review of Zoomify, though that kind of summed up what I thought about it:

We've used Zoomify previously, courtesy of the free converter available on the Zoomify website, and it works fine. But, Zoomify's free converter is limited in several ways: there's no control over the degree of JPEG compression, it doesn't wrap up the resulting image pieces and Flash-based viewer in the html code necessary to post a Zoomified picture on the Web (instead, a code template and instructions for manually editing it are included) and the viewer itself is a bit clunky: the control bar overlays the picture and sometimes too easily blends into image content, it's possible to scroll past the edges of the picture and the loading of new picture segments, while fast, lacks the smoothness we're accustomed to from Flash content these days.

None of these things are deal-stoppers, but the Zoomify experience has been just clumsy enough to limit our desire to use it regularly. The paid-for Zoomify converter that removes some of these limitations is an add-on to a full-fledged Macromedia/Adobe Flash application like Flash Professional 8, which itself is more expensive than we've been able to justify to date.

Seems like there's a new gigapixel "image war" on the 'net. It'll be interesting what Flash comes up with (presumably very soon) around big images. I've seen at least 6 different flash gigapixel image viewers that I assume are custom code.
Thursday, March 06, 2008 7:54:47 PM UTC
wow.. seadragon is coming out of the labs now!

but what about photosynth? is it coming out any time soon

anyhow this is good news, I already have some uses for it ^^"
neko-mimi
Thursday, March 06, 2008 9:23:42 PM UTC
I see some big differences between this and zoomify, namely this is a control within silverlight, you can do what you want with it, bind whatever key / mouse events, apply all sorts of styles and effects. More importantly if you study the demo or play with the tool the seperate images in the collection you build are seperate, its not just one big image (it can be). If you take the hard rock application you can see you can filter, sort etc on these objects seemlessly. Now that is cool. We're not just talking about zooming into one really large image, it appears to support large collections of really large images.
So I now have all my photos rendering, the mouse working to pan nicely at all zoom levels but i'm yet to be able to hook up the mouse scroll wheel, only able to use +,- key currently. I think I need to capture the scroll event in javascript - anyone cracked it yet?
John.
Thursday, March 06, 2008 10:09:15 PM UTC
Been talking to Scott H. and MIX. Wanted to let those of you who have been playing with Silverlight 2 Beta 1, the most common error we have been seeing since the launch of the site yesterday has to do with installation. And the fix is easy: just restart your browser after installing SL 2. Pressing F5 doesn't work. MS has this down as an issue.

Regarding the mouse wheel stuff: we haven't decided when/if/how we're going to release it. We've been so focused on just getting this launched and the keynote, we haven't had time to think about it.


Thursday, March 06, 2008 10:15:08 PM UTC
I'm glad this technology has finally been brought out into the open.
We've used it a lot in our work with virtual microscopy, but unfortunately it has been kept primarily inhouse or at least in-market (mainly because of the subject matter)
A few years ago Aperio released something remarkably similar to the Harlem 13-Gigapixels.com interface for their online gallery (Example http://images2.aperio.com/Dev.%20Incisor.svs/view.apml ), but even that was only really seen by like-minded souls.
SeaDragon did well to see the wider uses, and did a good job in smoothing out the various issues with the techniques, and now it's been at MIX with downloadable tools? Well it's going to be everywhere :)
Merak
Friday, March 07, 2008 7:43:04 AM UTC
will i be able to upload a large image and cut it with deepzoom at runtime?
Friday, March 07, 2008 5:46:05 PM UTC
Very cool. I think this would be great for a lot of uses. Perhaps for those that use Birds Eye View in maps.live.com, it could switch to using deep zoom to make it a better experience. While they are at it, they might as well make the entire Live map site run in silverlight, so that it will actually work in safari.

Here is my attempt at humor.
It is also nice that they finally provide a tool where we can see both Scott's forehead and his facial features in the same shot without scrolling my browser or downloading then opening up paint.net to crop some of that thing out :)
Saturday, March 08, 2008 7:50:08 AM UTC
I've produced a full sample from all the bits floating around -
http://www.soulsolutions.com.au/Blog/tabid/73/EntryID/394/Default.aspx
It includes the mouse wheel, keyboard controls and catching a few bugs. Download the code and put in your images.
See the sample of high res images (54mb) from Thailand as an example (zoom in on a small monkey):
thailand.soulclients.com
John.
Monday, March 10, 2008 12:28:18 AM UTC
Just to add more info, SeaDragon came about as an acquisition of a company of the same name. It's extremely interesting to see a gamble paid off. Although, I would say that SeaDragon is probably still limited by how large the web server's pipe is, and works best as a client-side technology.

I'm sure MS has a VirtualEarth proof-of-concept using SeaDragon out there some where. Let's see it! :)
Minh
Tuesday, March 11, 2008 10:36:47 AM UTC
Hi,

I was thinking the same, it would suit a maps programme so well. I love the way the extra data seems to melt into view. I've made my own demo using a photo of the office of the design studio I work for. Couldn't have done it without help from this page. Thanks!
Wednesday, March 12, 2008 11:27:39 AM UTC

Just a few fact checking, mind you?

- SeaDragon is not a Microsoft research project, it's the result of the acquisition of an external company.

- What SeaDragon does technically speaking? it's a client viewer for a simple variant of JPEG 2000 LOD files. LOD = Level Of Details. This is a technique coming from Fourier/Wavelet transforms where you can interpolate a picture of size d, between two predefined pictures of size d1 and d2. What makes this technique special is that it tries to control the distortion and other artifacts resulting from the scaling. In a way, you can call it a better JPEG encoding. (hence JPEG 2000, which is patent encumbered unfortunately)

- This is actually pretty standard modern mathematics, nothing to write home about.

- No one should read your post thinking that this "breakthrough technology" creates picture details from nothing. Shannon rules of entropy explain why this is not possible, and that's pretty obvious anyway that you can't do that. So it's always a bit disingenuous, although sexy I admit, to zoom in a picture in a way that may make the audience believe that details are generated as it zooms in. A more genuine showcase is to zoom out of a picture, because that's what better explains what the technology does really, ie remove details under controlled distortion.

Stephane Rodriguez
Wednesday, March 12, 2008 8:38:38 PM UTC
I don't know about wavelet LOD compression which is pretty nice to use with MrSID etc on a local desktop, but this 'MultiScaleImage' element is the real killer here! I get smooth pan and zoom over a large image sure, but note that this is in a browser and the data is flowing in from a server not my local hardrive. All I need is a reference to the pyramid in the MultiScaleImage source and I have all this user experience with almost no effort. This I like! kudos to the microsoft reasearch labs for making this available.
Wednesday, March 19, 2008 3:37:24 AM UTC
i think we have the best flash ZUI, plus ours is a calendar too.
Social Helix - the future of event networking

*controls - use the scroll wheel, the zoom bar, or circle your mouse clockwise or counterclockwise to zoom in and out.

most of these site load the images very slowly. i was actually very impressed with the hard rock implementation. I can wait until we can afford our next speed boost.
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.