Scott Hanselman

Introducing Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen

July 3, '12 Comments [19] Posted in Lync | Open Source | Remote Work | WPF
Sponsored By

Logitech BCC950 Conference CamIf you check out my blog archives or check out the Remote Work category you'll see that I'm always tinkering with my remote work situation. I'm the most interested in high quality and pervasive video. I'm so interested in this that for a while I was running a 10 hour a day persistent "Portal" between Portland and Seattle. I still highly recommend this for co-located teams that have the bandwidth to burn. It's great to be able to turn one's head and see their teammate right there by their side - even though they are 200 miles away.

I recently picked up a pair of Logitech BCC950 Conference Cams as a possible replacement for the very expensive "RoundTables" that some rooms at Microsoft have. The RoundTables are lovely but they are becoming scarce at the office and the Logitech is literally one-tenth the price. I'll blog a full and detailed review of the BCC950 later on this week but for now my biggest issue was that the Video Kiosk software I was using was starting to show its age. It's flaky and unreliable and build on the Office Communicator 2007 interfaces while I've been on Lync 2010 for a while.

Additionally, the researchers that wrote the software are always adding features I don't need for hardware I don't have. My remote buddy Damian Edwards and I decided we needed to make a software break.

Features

We want these simple features to start:

  • Auto-answer of video calls - possibly with some whitelist for security
  • Auto-fullscreen of video calls on the remote machine - the single purpose kiosk in my Seattle office
  • Presence information and a simple UI for making calls - by simple I mean "my boss's boss" simple
  • Remote control of Pan Tilt Zoom (PTZ) features on the same - ideally using the standard "inbox UVC" drivers and no 3rd party software

Tonight I sat down and did the first three of these and put it on GitHub. I call it - wait for it - the Lync 2012 Super Simple Auto Answer Video Kiosk with Full Screen since the name "SmartGlass" was already taken. ;)

I searched ALL over to find out if there was SOME sample code out there that would just auto-answer a call from Lync and start video. I could find dozens of samples that made calls, that started chats, but none that would answer and auto-start video. You'd think this would be the FIRST thing that folks would want to do. I can only assume it's not a setting for security reasons.

Auto-Answering Lync Calls with Video

Now, it's late and there's likely problems so pull requests are welcome if I have done something stupid. Lync is complex and while you'd think there'd be an "AutoAnswer = true" it's actually a more complex API than that. I started from this MSDN article on "Responding to a Lync Conversation Invitation."

var lync = LyncClient.GetClient();
var conversationmgr = lync.ConversationManager;
conversationmgr.ConversationAdded += (_, cmea) =>
{
bool IncomingAV = false;
StringBuilder sb = new StringBuilder();

//Is this an audio/video invitation?
if (cmea.Conversation.Modalities[ModalityTypes.AudioVideo].State == ModalityState.Notified)
{
if (lync.DeviceManager.ActiveAudioDevice != null)
{
sb.Append("Incoming call from ");
IncomingAV = true;
}
else
{
cmea.Conversation.Modalities[ModalityTypes.AudioVideo].Reject(ModalityDisconnectReason.NotAcceptableHere);
}
}
if (cmea.Conversation.Modalities[ModalityTypes.InstantMessage].State == ModalityState.Connected)
{
sb.Append("Incoming IM from ");
}
sb.Append(String.Join(", ", cmea.Conversation.Participants.Select(i => i.Contact.Uri)));
Debug.WriteLine(sb.ToString());

if (IncomingAV == true && Properties.Settings.Default.autoAnswer == true) //I added that setting later on
{
InitiateAVStream(cmea.Conversation);
}
};

You watch for a Conversation to start and see if it's an Audio/Video on. If it is, then we call our InitiateAVStream method. You can't do all this stuff synchronously as Lync is full of async native COM APIs and events that you need to respond to. Here we accept the video which lets us see who called us but doesn't yet start OUR video. Remember "we" are the dumb Kiosk who is receiving a call from me.

private static void InitiateAVStream(Conversation pConversation)
{
if (pConversation.State == ConversationState.Terminated) { return; }

if (pConversation.Modalities[ModalityTypes.AudioVideo].CanInvoke(ModalityAction.Connect))
{
var video = (AVModality)pConversation.Modalities[ModalityTypes.AudioVideo];
video.Accept();

//Get ready to be connected, then WE can start OUR video
video.ModalityStateChanged += _AVModality_ModalityStateChanged;
}
}

See how they are chaining handlers? I think this code could be made cleaner with a series of nested closures like above in the ConversationAdded example, but then again, maybe not. It'll get four deep before we're done.

Now the call is being connected but perhaps not yet. When its state changes the VideoChannel has opened up and we watch for the video to be received.

static void _AVModality_ModalityStateChanged(object sender, ModalityStateChangedEventArgs e)
{
VideoChannel vc = null;
switch (e.NewState)
{
//we can't start video until it's connected
case ModalityState.Connected:
if (vc == null)
{
vc = ((AVModality)sender).VideoChannel;
vc.StateChanged += new EventHandler<ChannelStateChangedEventArgs>(_VideoChannel_StateChanged);
}
break;
}
}

As the video starts up, I can see if the system is ready for the Kiosk to start its video. If so, we call BeginStart (and the SDK says we HAVE to call EndStart, so watch out!).

static void _VideoChannel_StateChanged(object sender, ChannelStateChangedEventArgs e)
{
VideoChannel vc = (VideoChannel)sender;

//Are we receiving? Let's try to send!
if (e.NewState == ChannelState.Receive)
{
if (vc.CanInvoke(ChannelAction.Start))
{
vc.BeginStart(videoCallBack, vc);
}
else { Debug.WriteLine("CanInvoke said NO!"); }

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);

//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);
}
}


private static void videoCallBack(IAsyncResult ar)
{
((VideoChannel)ar.AsyncState).EndStart(ar);
}

I'm pretty frustrated as while this is SUPER powerful, it's also SUPER complex for basic scenarios in my opinion. I think there's an opportunity here for a small layer on top of Lync that handles the 80% cases like the small Lync Abstraction  layer introduced in this "ScreenPop" example application.

The Goodness - The WPF Controls

At this point in the code I had everything working in a Console Application. You can go cherry-pick that commit if you want just a Console app that Auto-Answers video calls from Lync.

Even though I NEED to stop as I've got it working in a Console and I should be sleeping I noticed this in Visual Studio and it was too epic to not try.

File | New Lync WPF Application

You know how it is. It's 2am, you're done with your goals. OF COURSE you're going to try to convert your Console App to a GUI before bed. Of course.

Turns out there's a MESS of visual controls that you can put into existing applications to make them Lync-ified in literally minutes.

All the Lync Controls like SendEmailButton and StartVideoCallButton

Ok, awesome. I took the basic UI and added a checkbox for "Auto Answer."

Perhaps the lamest UI ever. It's a head, a button and a checkbox

(ASIDE: You DO realize that the outline of the "unknown face" in Lync there looks an AWFUL lot like Bill Gates' legendary 1977 mug shot, right? I just noticed that.)

Lync 2010's default Person matches the outline of Bill Gates' 1977 Mug Shot

Anyway, then I made two settings, one for my "sip" address (that's in my app.config file as "sipEmailAddress" and one boolean for AutoAnswer. The complete XAML is just:

<Window x:Class="SuperSimpleLyncKiosk.Main"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:properties="clr-namespace:SuperSimpleLyncKiosk.Properties"
xmlns:controls="clr-namespace:Microsoft.Lync.Controls;assembly=Microsoft.Lync.Controls"
Title="The World's Simplest Lync Kiosk (with Auto Answer, too!)" Height="Auto" Width="Auto">
<Grid RenderTransformOrigin="0.5,0.5">
<Grid.RenderTransform>
<ScaleTransform ScaleX="3" ScaleY="3"/>
</Grid.RenderTransform>

<StackPanel Orientation="Vertical" HorizontalAlignment="Center" VerticalAlignment="Center">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center" VerticalAlignment="Center">
<!-- Show the presence indicator. Hover over the icon to see the contact card.
Set Source to a valid SIP URI in your organization. -->
<controls:PresenceIndicator
x:Name="Presence"
Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"
PhotoDisplayMode="Large"
/>
<!-- Use the DisplayName property from PresenceIndicator to show the user's name -->
<TextBlock
Text="{Binding DisplayName, ElementName=Presence}"
Margin="4,0,0,0"
VerticalAlignment="Center"
/>
<controls:StartVideoCallButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}" x:Name="startVideoCall" />
<controls:ShareDesktopButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"/>
</StackPanel>
<CheckBox IsChecked="{Binding Source={x:Static properties:Settings.Default}, Path=autoAnswer, Mode=TwoWay}" Content="Auto Answer Video Calls"/>
</StackPanel>
</Grid>

</Window>

I also added a 3x transform to scale ALL these default controls so they'd look good on the 42" TV that is sitting in my office. Because they are native WPF vector controls they just scale perfectly to high resolutions without raster jaggies.

Then I added a call to make the app Maximized by default:

this.WindowState = System.Windows.WindowState.Maximized;

And it looks like this when running:

The World's Simplest Lync Kiosk (with Auto Answer, too!)

And when I call it automatically answers. Looks like everyone's asleep and they've turned the lights out!

My Remote office in Seattle

Ah, I but I wish it was full screen so the people in Redmond don't need to do anything or touch anything...

The Badness

I can auto-answer calls but sometimes the window isn't in front and once it gets in front there's no programmatic way to tell Lync to go Fullscreen with video.

Two bad problems there. Both solved by breaking all the rules. I get the Window Class with a big assumption that the Kiosk only as one chat window open and then I "push" F5 which is the Lync hotkey for fullscreen video.

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);
//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);

Those last two, of course, are calls directly into Win32 from .NET:

public static class UnsafeNativeMethods
{
[DllImport("user32.dll")]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
}

But, it works! It's scandalous that this isn't built into the Lync SDK. Developers who are fans of Lync or who work on it all the time will say that my attempt at a "poor man's Kiosk" is silly and that I really want to use "UI Suppression in Lync" and just make an app that hosts Lync rather than automates Lync. They are likely right. However, frankly, it looked super-hard and I was tired. So, ya. If anyone wants to work on the Kiosk with me to make it simple answer and start video and do it all without showing Lync, that'd be awesome.

Thanks

I also want to make a special nod to the InputSimulator library. It's amazing and it just works. It's WAY WAY better than SendKeys which you should stop using NOW.

The Windows Input Simulator provides a simple .NET (C#) interface to simulate Keyboard or Mouse input using the Win32 SendInput method. All of the Interop is done for you and there's a simple programming model for sending multiple keystrokes.

Enjoy!

Lync Developer Resources

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Hanselminutes Podcast 241 - The MVVM Pattern with Laurent Bugnion

November 19, '10 Comments [2] Posted in Open Source | Podcast | Silverlight | WPF
Sponsored By

Scott talks to Laurent Bugnion about the often misunderstood Model-View-ViewModel (MVVM) pattern. What's the different between this pattern and MVC? Can I use this pattern for Silverlight, WPF and Windows Phone 7, and what Open Source projects can support this pattern?

Links from the Show

NOTE: If you want to download our complete archives as a feed - that's all 241 shows, subscribe to the Complete MP3 Feed here.

Also, please do take a moment and review the show on iTunes.

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

Download: MP3 Full Show

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Building quality software is never easy. It requires skills and imagination. We cannot promise to improve your skills, but when it comes to User Interface and developer tools, we can provide the building blocks to take your application a step closer to your imagination. Explore the leading UI suites for ASP.NET AJAX,MVC,Silverlight,Windows Forms and WPF. Enjoy developer tools like .NET Reporting, ORM, Automated Testing Tools, Agile Project Management Tools, and Content Management Solution. And now you can increase your productivity with JustCode, Telerik’s new productivity tool for code analysis and refactoring. Visit www.telerik.com.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

The Weekly Source Code 56 - Visual Studio 2010 and .NET Framework 4 Training Kit - Code Contracts, Parallel Framework and COM Interop

August 12, '10 Comments [11] Posted in ASP.NET | ASP.NET Ajax | ASP.NET Dynamic Data | ASP.NET MVC | BCL | Learning .NET | LINQ | OData | Open Source | Programming | Source Code | VB | Web Services | Win7 | Windows Client | WPF
Sponsored By

Do you like a big pile of source code? Well, there is an imperial buttload of source in the Visual Studio 2010 and .NET Framework 4 Training Kit. It's actually a 178 meg download, which is insane. Perhaps start your download now and get it in the morning when you get up. It's extremely well put together and I say Kudos to the folks that did it. They are better people than I.

I like to explore it while watching TV myself and found myself looking through tonight. I checked my blog and while I thought I'd shared this with you before, Dear Reader, I hadn't. My bad, because it's pure gold. With C# and VB, natch.

Here's an outline of what's inside. I've heard of folks setting up lunch-time study groups and going through each section.

C# 4 Visual Basic 10 
F# Parallel Extensions
Windows Communication Foundation Windows Workflow
Windows Presentation Foundation ASP.NET 4
Windows 7 Entity Framework
ADO.NET Data Services (OData) Managed Extensibility Framework
Visual Studio Team System RIA Services
Office Development  

I love using this kit in my talks, and used it a lot in my Lap Around .NET 4 talk.

There's Labs, Presentations, Demos, Labs and links to online Videos. It'll walk you step by step through loads of content and is a great starter if you're getting into what's new in .NET 4.

Here's a few of my favorite bits, and they aren't the parts you hear the marketing folks gabbing about.

Code Contracts

Remember the old coding adage to "Assert Your Expectations?" Well, sometimes Debug.Assert is either inappropriate or cumbersome and what you really need is a method contract. Methods have names and parameters, and those are contracts. Now they can have conditions like "don't even bother calling this method unless userId is greater than or equal to 0 and make sure the result isn't null!

Code Contracts continues to be revised, with a new version out just last month for both 2008 and 2010. The core types that you need are included in mscorlib with .NET 4.0, but you do need to download the tools to see them inside Visual Studio. If you have VS Pro, you'll get runtime checking and VS Ultimate gets that plus static checking. If I have static checking and the tools I'll see a nice new tab in Project Properties:

Code Contracts Properties Tab in Visual Studio

I can even get Blue Squigglies for Contract Violations as seen below.

A blue squigglie showing that a contract isn't satisfied

As a nice coincidence, you can go and download Chapter 15 of Jon Skeet's C# in Depth for free which happens to be on Code Contracts.

Here's a basic idea of what it looks like. If you have static analysis, you'll get squiggles on the lines I've highlighted as they are points where the Contract isn't being fulfilled. Otherwise you'll get a runtime ContractException. Code Contracts are a great tool when used in conjunction with Test Driven Development.

using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics.Contracts;

namespace ContractsDemo
{
[ContractVerification(true)]
class Program
{
static void Main(string[] args)
{
var password = GetPassword(-1);
Console.WriteLine(password.Length);
Console.ReadKey();
}

#region Header
/// <param name="userId">Should be greater than 0</param>
/// <returns>non-null string</returns>
#endregion
static string GetPassword(int userId)
{
Contract.Requires(userId >= 0, "UserId must be");
Contract.Ensures(Contract.Result<string>() != null);

if (userId == 0)
{
// Made some code to log behavior

// User doesn't exist
return null;
}
else if (userId > 0)
{
return "Password";
}

return null;
}
}
}

COM Interop sucks WAY less in .NET 4

I did a lot of COM Interop back in the day and it sucked. It wasn't fun and you always felt when you were leaving managed code and entering COM. You'd have to use Primary Interop Assemblies or PIAs and they were, well, PIAs. I talked about this a little bit last year in Beta 1, but it changed and got simpler in .NET 4 release.

Here's a nice little sample I use from the kit that gets the Processes on your system and then makes a list with LINQ of the big ones, makes a chart in Excel, then pastes the chart into Word.

If you've used Office Automation from managed code before, notice that you can say Range[] now, and not get_range(). You can call COM methods like ChartWizard with named parameters, and without including Type.Missing fifteen times. As an aside, notice also the default parameter value on the method.

static void GenerateChart(bool copyToWord = false)
{
var excel = new Excel.Application();
excel.Visible = true;
excel.Workbooks.Add();

excel.Range["A1"].Value2 = "Process Name";
excel.Range["B1"].Value2 = "Memory Usage";

var processes = Process.GetProcesses()
.OrderByDescending(p => p.WorkingSet64)
.Take(10);
int i = 2;
foreach (var p in processes)
{
excel.Range["A" + i].Value2 = p.ProcessName;
excel.Range["B" + i].Value2 = p.WorkingSet64;
i++;
}

Excel.Range range = excel.Range["A1"];
Excel.Chart chart = (Excel.Chart)excel.ActiveWorkbook.Charts.Add(
After: excel.ActiveSheet);

chart.ChartWizard(Source: range.CurrentRegion,
Title: "Memory Usage in " + Environment.MachineName);

chart.ChartStyle = 45;
chart.CopyPicture(Excel.XlPictureAppearance.xlScreen,
Excel.XlCopyPictureFormat.xlBitmap,
Excel.XlPictureAppearance.xlScreen);

if (copyToWord)
{
var word = new Word.Application();
word.Visible = true;
word.Documents.Add();

word.Selection.Paste();
}
}

You can also embed your PIAs in your assemblies rather than carrying them around and the runtime will use Type Equivalence to figure out that your embedded types are the same types it needs and it'll just work. One less thing to deploy.

Parallel Extensions

The #1 reason, IMHO, to look at .NET 4 is the parallelism. I say this not as a Microsoft Shill, but rather as a dude who owns a 6-core (12 with hyper-threading) processor. My most favorite app in the Training Kit is ContosoAutomotive. It's a little WPF app that loads a few hundred thousand cars into a grid. There's an interface, ICarQuery, that a bunch of plugins implement, and the app foreach's over the CarQueries.

This snippet here uses the new System.Threading.Task stuff and makes a background task. That's all one line there, from StartNew() all the way to the bottom. It says, "do this chunk in the background." and it's a wonderfully natural and fluent interface. It also keeps your UI thread painting so your app doesn't freeze up with that "curtain of not responding" that one sees all the time.

private void RunQueries()
{
this.DisableSearch();
Task.Factory.StartNew(() =>
{
this.BeginTiming();
foreach (var query in this.CarQueries)
{
if (this.searchOperation.Token.IsCancellationRequested)
{
return;
}

query.Run(this.cars, true);
};
this.EndSequentialTiming();
}, this.searchOperation.Token).ContinueWith(_ => this.EnableSearch());
}

StartNew() also has a cancellation token that we check, in case someone clicked Cancel midway through, and there's a ContinueWith at the end that re-enables or disabled Search button.

Here's my system with the queries running. This is all in memory, generating and querying random cars.12% CPU across 12 processors single threaded

And the app says it took 2.3 seconds. OK, what if I do this in parallel, using all the processors?

2.389 seconds serially

Here's the changed code. Now we have a Parallel.ForEach instead. Mostly looks the same.

private void RunQueriesInParallel()
{
this.DisableSearch();
Task.Factory.StartNew(() =>
{
try
{
this.BeginTiming();
var options = new ParallelOptions() { CancellationToken = this.searchOperation.Token };
Parallel.ForEach(this.CarQueries, options, (query) =>
{
query.Run(this.cars, true);
});
this.EndParallelTiming();
}
catch (OperationCanceledException) { /* Do nothing as we cancelled it */ }
}, this.searchOperation.Token).ContinueWith(_ => this.EnableSearch());
}

This code says "go do this in a background thread, and while you're there, parallelize this as you like." This loop is "embarrassingly parallel." It's a big for loop over 2 million cars in memory. No reason it can't be broken apart and made faster.

Here's the deal, though. It was SO fast, that Task Manager didn't update fast enough to show the work. The work was too easy. You can see it used more CPU and that there was a spike of load across 10 of the 12, but the work wasn't enough to peg the processors.

19% load across 12 processors 

Did it even make a difference? Seems it was 5x faster and went from 2.389s to 0.4699 seconds. That's embarrassingly parallel. The team likes to call that "delightfully parallel" but I prefer "you're-an-idiot-for-not-doing-this-in-parallel parallel," but that was rejected.

0.4699 seconds when run in parallel. A 5x speedup.

Let's try something harder. How about a large analysis of Baby Names. How many Roberts born in the state of Washington over a 40 year period from a 500MB database?

Here's the normal single-threaded foreach version in Task Manager:

One processor chilling.

Here's the parallel version using 96% CPU.

6 processes working hard!

And here's the timing. Looks like the difference between 20 seconds and under 4 seconds.

PLINQ Demo

You can try this yourself. Notice the processor slider bar there at the bottom.

ProcessorsToUse.Minimum = 1;
ProcessorsToUse.Maximum = Environment.ProcessorCount;
ProcessorsToUse.Value = Environment.ProcessorCount; // Use all processors.

This sample uses "Parallel LINQ" and here's the two queries. Notice the "WithDegreeofParallelism."

seqQuery = from n in names
where n.Name.Equals(queryInfo.Name, StringComparison.InvariantCultureIgnoreCase) &&
n.State == queryInfo.State &&
n.Year >= yearStart && n.Year <= yearEnd
orderby n.Year ascending
select n;

parQuery = from n in names.AsParallel().WithDegreeOfParallelism(ProcessorsToUse.Value)
where n.Name.Equals(queryInfo.Name, StringComparison.InvariantCultureIgnoreCase) &&
n.State == queryInfo.State &&
n.Year >= yearStart && n.Year <= yearEnd
orderby n.Year ascending
select n;

The .NET 4 Training Kit has Extensibility demos, and Office Demos and SharePoint Demos and Data Access Demos and on and on. It's great fun and it's a classroom in a box. I encourage you to go download it and use it as a teaching tool at your company or school. You could do brown bags, study groups, presentations (there's lots of PPTs), labs and more.

Hope you enjoy it as much as I do.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Dealing with Images with Bad Metadata - Corrupted Color Profiles in WPF

July 30, '10 Comments [4] Posted in Bugs | Windows Client | WPF
Sponsored By

Creating a Twitter client is a really interesting exercise in application development because, amongst many reasons, it's taking input from effectively an infinite number of people and places. Never trust user input, right? Input to your application comes not only in the form of text, but also images. Writing a Twitter client is effectively writing a web browser that only browses one website. Getting a browser stable is hard.

Long Zheng, Raphael Rivera and the MetroTwit team (MetroTwit is a lovely new Twitter client) have hit an extremely interesting crashing bug. The input comes in the form of a corrupted JPG image from the web.

Here's the bad image. Looks like a picture some folks speaking on a panel. However, even though this image looks fine, this specific binary version of it has a corrupted Color Profile.

Sometimes folks don't realize that image formats contain lots of metadata that you can't see. Your JPGs may show what camera you used, what lens, what settings, possibly even the geo-coordinates of where you took the picture!

You can view all this extended information (EXIF) with a number of tools. A great free one is ExifTool by Phil Harvey at the command line, or a non-command line one like ExifPro. Windows Live Photo Gallery lets you view the data also.

Here's a snippet of some of the info in this pic:

Device Mfg Desc                 : IEC http://www.iec.ch
Device Model Desc               : IEC 61966-2.1 Default RGB colour space - sRGB
Viewing Cond Desc               : Reference Viewing Condition in IEC61966-2.1
Viewing Cond Illuminant         : 19.6445 20.3718 16.8089
Viewing Cond Surround           : 3.92889 4.07439 3.36179
Viewing Cond Illuminant Type    : D50
Make                            : Leica Camera AG
Camera Model Name               : M8 Digital Camera
Software                        : Aperture 3.0.2
Shutter Speed Value             : 1/256
Exposure Compensation           : 0
Max Aperture Value              : 1.0
Metering Mode                   : Center-weighted average
Light Source                    : Flash
Focal Length                    : 0.0 mm

You can extract the image profile (ICC Profile) from an image like this with exiftool:

exiftool -icc_profile -b foo.jpg > profile.icc

If you're hardcore, you can get the Windows Imaging Component (WIC) Tools and run WICExplorer. WPF uses WIC to decode images. WICExplorer will report the error with this image as you load it.

Loading Images in WPF

When you're using WPF (Windows Presentation Foundation) to display an image on Windows, you might do something like this:

<Image Width="300" Height="300" ImageFailed="Image_ImageFailed">
<Image.Source>
<BitmapImage UriSource="http://hanselman.com/blog/images/JPGwithBadColorProfile.jpg"/>
</Image.Source>
</Image>

Except with this particular image, I'll get an exception the Color Profile (the image metadata) is corrupted. "ArgumentException: Value does not fall within the expected range." This is a corrupted file.

at System.Windows.Media.ColorContext.GetColorContextsHelper(GetColorContextsDelegate getColorContexts)
at System.Windows.Media.Imaging.BitmapFrameDecode.get_ColorContexts()
at System.Windows.Media.Imaging.BitmapImage.FinalizeCreation()
at System.Windows.Media.Imaging.BitmapImage.OnDownloadCompleted(Object sender, EventArgs e)
at System.Windows.Media.UniqueEventHelper.InvokeEvents(Object sender, EventArgs args)
at System.Windows.Media.Imaging.LateBoundBitmapDecoder.DownloadCallback(Object arg)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(Object source, Delegate method, Object args, Int32 numArgs, Delegate catchHandler)

If I get this exception, I can try to load the image again and ignore its color profile. Here's how I'd do that in XAML:

<Image Width="300" Height="300" ImageFailed="Image_ImageFailed"  >
<Image.Source>
<BitmapImage CreateOptions="IgnoreColorProfile" UriSource="http://hanselman.com/blog/images/JPGwithBadColorProfile.jpg"/>
</Image.Source>
</Image>

If you're loading from code, you can ignore color profile information by adding the BitmapCreateOptions.IgnoreColorProfile flag to CreateOptions.

As an aside, Andrew Eichacker has a nice post on how to read all the BitmapMetadata in WPF. There's lots in there!

Here's loading the Bitmap into an image Control called "Foo."

var bi = new BitmapImage();
bi.BeginInit();
bi.CreateOptions = BitmapCreateOptions.IgnoreColorProfile;
bi.UriSource = new Uri("http://hanselman.com/blog/images/JPGwithBadColorProfile.jpg");
bi.EndInit();

foo.Source = bi;

Knowing about possible corruption is important to be aware of, especially if you're loading arbitrary images from all over the place. If you don't care about color profiles, I'd just ignore them by default in your image loading code. If you are writing an image editor or you care about profiles, I'd catch the exception, let the user know, then load the image again without the profile.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Visual Studio 2010 Released

April 12, '10 Comments [114] Posted in ASP.NET | ASP.NET MVC | Learning .NET | Microsoft | MSDN | Spotlight | VS2010 | Windows Client | WPF
Sponsored By

It's a big day at Microsoft today as Visual Studio 2010 officially releases. There's a lot going on with this release and I thought I'd do a big rollup post with lots of details and context to help you find your way to the information and downloads you're looking for.

Visual Studio 2010 and .NET 4

Download Visual Studio 2010

First, if you want it, go download Visual Studio 2010 now. If you're an MSDN Subscriber or WebSiteSpark/BizSpark member, you can download the final release now. If not, you can download a free trial or one of the free Express editions.

I'm running the free Visual Web Developer 2010 Express on my netbook. You can install ASP.NET 4, ASP.NET MVC 2, and Visual Web Developer 2010 Express really quickly with the Web Platform Installer.

There's an excellent page on MSDN that's cherry-picked and categorized the best VS2010 content, but I've included my own list below.

What's new in Visual Studio 2010 and .NET 4?

Buttloads. Here's the things I'm digging most.

Also, there's a FREE e-Book called "Moving to Visual Studio 2010" that you might want to check out. It's an excerpt of a larger book that'll be coming from MSPress later this summer. It takes a interesting approach as it has three parts, moving from VS2003, moving from VS2005, and moving from VS2008. It's clever, actually. You start in the book on the version that you're currently on. If you're not familiar with versions like VS2008, you start at the beginning. Otherwise, you jump ahead. When you're done, you're ready to move to VS2010.

MSDN and Visual Studio 2010

When a new product launches, MSDN launches with updates and new features of its own. Here's a few things the folks at MSDN have been doing to support the launch.

  • Better MSDN Search - Most people likely use a search engine to search MSDN, but if you do search from within MSDN, there are a number of new improvements. You can refine by source, saying only search blogs, or only search the library. There's also an OpenSearch provider so you can search the MSDN Library directly from within Windows itself.
    XDocument - Search Results in MSDN
    MSDN Search also includes Metadata from the results to help you find right thing. For example, if a search turns up a CodePlex project, I can see type-specific details within search results:
    MSDN Search
  • MSDN Subscriber Downloads Improvements - There's been lots of UX improvements including as-you-type filtering as well as filtering by platform (x64, etc) and language. I will very likely not need to download Quechua Windows, so now I don't need to see it.
  • MSDN Library in Lightweight and ScriptFree - You can choose between three flavors of MSDN Library, Classic (the one with the treeview on the side), Lightweight (what I use) or ScriptFree. ScriptFree is great for mobile devices, and it's lightning fast anywhere. Lightweight is the new default and I like it because it features community annotations made to the library prominently on the left side as well as a tabbed interface for code sample languages. I blogged a preview of this work last year and included some charts and graphs showing the improvements in speed worldwide.
    XmlNode Class (System.Xml) - Windows Internet Explorer
  • Integration of all VS sites - There were too many developer "centers" on MSDN and folks were getting lost. Many centers have been conflated into a clearer, more logical layout. The Visual Studio, Team System, and VS Extensibility Centers were merged into the single Visual Studio Center. There's a lot more focus on discoverability in the Visual Studio Center.
  • Video Improvements - There's thousands of How Do I? videos on MSDN and they tell me they are improving the backend, the player and the metadata around them. The player is larger now, you can share videos from MSDN on your favorite social networking site, rate them, leave comments, and explore related videos.
  • Profile Activities - User Profiles are integrated between sites and you can see your activity and points as you move through the system. For example, here's Arnie Rowland's profile. You can see his activity in the forums and galleries as well as his ranking and points as a community contributor.

Other Cool Stuff Happening Today

I'd hate to have this little nugget get buried in the deluge of VS2010 goodness.

  • Microsoft Surface Logo The Surface Toolkit for Windows Touch BETA goes out today. Check out http://www.surface.com under Technical Resources and the Surface Blog for more details. This toolkit is a set of controls and sample code that let WPF developers create cool multi-touch enabled experiences with the cool "Surfacey" controls that the only folks with Big Ass Tables have been able to use. This is exceedingly cool because it not only makes it WPF devs can make better multi-touch apps for Windows Touch PCs but it acts as a jump-start for the next version of Microsoft Surface. It will integrate with Visual Studi0 2010 and give you new project and item templates and a dozen new controls like the ScatterView and SurfaceInkCanvas. This is a cool thing, so I'll be talking about it soon, as will Pete Brown.

Lots of great stuff going on today. Have fun!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Page 1 of 10 in the WPF category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.