Scott Hanselman

Introducing Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen

July 03, 2012 Comment on this post [25] Posted in Lync | Open Source | Remote Work | WPF
Sponsored By

Logitech BCC950 Conference CamIf you check out my blog archives or check out the Remote Work category you'll see that I'm always tinkering with my remote work situation. I'm the most interested in high quality and pervasive video. I'm so interested in this that for a while I was running a 10 hour a day persistent "Portal" between Portland and Seattle. I still highly recommend this for co-located teams that have the bandwidth to burn. It's great to be able to turn one's head and see their teammate right there by their side - even though they are 200 miles away.

I recently picked up a pair of Logitech BCC950 Conference Cams as a possible replacement for the very expensive "RoundTables" that some rooms at Microsoft have. The RoundTables are lovely but they are becoming scarce at the office and the Logitech is literally one-tenth the price. I'll blog a full and detailed review of the BCC950 later on this week but for now my biggest issue was that the Video Kiosk software I was using was starting to show its age. It's flaky and unreliable and build on the Office Communicator 2007 interfaces while I've been on Lync 2010 for a while.

Additionally, the researchers that wrote the software are always adding features I don't need for hardware I don't have. My remote buddy Damian Edwards and I decided we needed to make a software break.

Features

We want these simple features to start:

  • Auto-answer of video calls - possibly with some whitelist for security
  • Auto-fullscreen of video calls on the remote machine - the single purpose kiosk in my Seattle office
  • Presence information and a simple UI for making calls - by simple I mean "my boss's boss" simple
  • Remote control of Pan Tilt Zoom (PTZ) features on the same - ideally using the standard "inbox UVC" drivers and no 3rd party software

Tonight I sat down and did the first three of these and put it on GitHub. I call it - wait for it - the Lync 2012 Super Simple Auto Answer Video Kiosk with Full Screen since the name "SmartGlass" was already taken. ;)

I searched ALL over to find out if there was SOME sample code out there that would just auto-answer a call from Lync and start video. I could find dozens of samples that made calls, that started chats, but none that would answer and auto-start video. You'd think this would be the FIRST thing that folks would want to do. I can only assume it's not a setting for security reasons.

Auto-Answering Lync Calls with Video

Now, it's late and there's likely problems so pull requests are welcome if I have done something stupid. Lync is complex and while you'd think there'd be an "AutoAnswer = true" it's actually a more complex API than that. I started from this MSDN article on "Responding to a Lync Conversation Invitation."

var lync = LyncClient.GetClient();
var conversationmgr = lync.ConversationManager;
conversationmgr.ConversationAdded += (_, cmea) =>
{
bool IncomingAV = false;
StringBuilder sb = new StringBuilder();

//Is this an audio/video invitation?
if (cmea.Conversation.Modalities[ModalityTypes.AudioVideo].State == ModalityState.Notified)
{
if (lync.DeviceManager.ActiveAudioDevice != null)
{
sb.Append("Incoming call from ");
IncomingAV = true;
}
else
{
cmea.Conversation.Modalities[ModalityTypes.AudioVideo].Reject(ModalityDisconnectReason.NotAcceptableHere);
}
}
if (cmea.Conversation.Modalities[ModalityTypes.InstantMessage].State == ModalityState.Connected)
{
sb.Append("Incoming IM from ");
}
sb.Append(String.Join(", ", cmea.Conversation.Participants.Select(i => i.Contact.Uri)));
Debug.WriteLine(sb.ToString());

if (IncomingAV == true && Properties.Settings.Default.autoAnswer == true) //I added that setting later on
{
InitiateAVStream(cmea.Conversation);
}
};

You watch for a Conversation to start and see if it's an Audio/Video on. If it is, then we call our InitiateAVStream method. You can't do all this stuff synchronously as Lync is full of async native COM APIs and events that you need to respond to. Here we accept the video which lets us see who called us but doesn't yet start OUR video. Remember "we" are the dumb Kiosk who is receiving a call from me.

private static void InitiateAVStream(Conversation pConversation)
{
if (pConversation.State == ConversationState.Terminated) { return; }

if (pConversation.Modalities[ModalityTypes.AudioVideo].CanInvoke(ModalityAction.Connect))
{
var video = (AVModality)pConversation.Modalities[ModalityTypes.AudioVideo];
video.Accept();

//Get ready to be connected, then WE can start OUR video
video.ModalityStateChanged += _AVModality_ModalityStateChanged;
}
}

See how they are chaining handlers? I think this code could be made cleaner with a series of nested closures like above in the ConversationAdded example, but then again, maybe not. It'll get four deep before we're done.

Now the call is being connected but perhaps not yet. When its state changes the VideoChannel has opened up and we watch for the video to be received.

static void _AVModality_ModalityStateChanged(object sender, ModalityStateChangedEventArgs e)
{
VideoChannel vc = null;
switch (e.NewState)
{
//we can't start video until it's connected
case ModalityState.Connected:
if (vc == null)
{
vc = ((AVModality)sender).VideoChannel;
vc.StateChanged += new EventHandler<ChannelStateChangedEventArgs>(_VideoChannel_StateChanged);
}
break;
}
}

As the video starts up, I can see if the system is ready for the Kiosk to start its video. If so, we call BeginStart (and the SDK says we HAVE to call EndStart, so watch out!).

static void _VideoChannel_StateChanged(object sender, ChannelStateChangedEventArgs e)
{
VideoChannel vc = (VideoChannel)sender;

//Are we receiving? Let's try to send!
if (e.NewState == ChannelState.Receive)
{
if (vc.CanInvoke(ChannelAction.Start))
{
vc.BeginStart(videoCallBack, vc);
}
else { Debug.WriteLine("CanInvoke said NO!"); }

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);

//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);
}
}


private static void videoCallBack(IAsyncResult ar)
{
((VideoChannel)ar.AsyncState).EndStart(ar);
}

I'm pretty frustrated as while this is SUPER powerful, it's also SUPER complex for basic scenarios in my opinion. I think there's an opportunity here for a small layer on top of Lync that handles the 80% cases like the small Lync Abstraction  layer introduced in this "ScreenPop" example application.

The Goodness - The WPF Controls

At this point in the code I had everything working in a Console Application. You can go cherry-pick that commit if you want just a Console app that Auto-Answers video calls from Lync.

Even though I NEED to stop as I've got it working in a Console and I should be sleeping I noticed this in Visual Studio and it was too epic to not try.

File | New Lync WPF Application

You know how it is. It's 2am, you're done with your goals. OF COURSE you're going to try to convert your Console App to a GUI before bed. Of course.

Turns out there's a MESS of visual controls that you can put into existing applications to make them Lync-ified in literally minutes.

All the Lync Controls like SendEmailButton and StartVideoCallButton

Ok, awesome. I took the basic UI and added a checkbox for "Auto Answer."

Perhaps the lamest UI ever. It's a head, a button and a checkbox

(ASIDE: You DO realize that the outline of the "unknown face" in Lync there looks an AWFUL lot like Bill Gates' legendary 1977 mug shot, right? I just noticed that.)

Lync 2010's default Person matches the outline of Bill Gates' 1977 Mug Shot

Anyway, then I made two settings, one for my "sip" address (that's in my app.config file as "sipEmailAddress" and one boolean for AutoAnswer. The complete XAML is just:

<Window x:Class="SuperSimpleLyncKiosk.Main"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:properties="clr-namespace:SuperSimpleLyncKiosk.Properties"
xmlns:controls="clr-namespace:Microsoft.Lync.Controls;assembly=Microsoft.Lync.Controls"
Title="The World's Simplest Lync Kiosk (with Auto Answer, too!)" Height="Auto" Width="Auto">
<Grid RenderTransformOrigin="0.5,0.5">
<Grid.RenderTransform>
<ScaleTransform ScaleX="3" ScaleY="3"/>
</Grid.RenderTransform>

<StackPanel Orientation="Vertical" HorizontalAlignment="Center" VerticalAlignment="Center">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center" VerticalAlignment="Center">
<!-- Show the presence indicator. Hover over the icon to see the contact card.
Set Source to a valid SIP URI in your organization. -->
<controls:PresenceIndicator
x:Name="Presence"
Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"
PhotoDisplayMode="Large"
/>
<!-- Use the DisplayName property from PresenceIndicator to show the user's name -->
<TextBlock
Text="{Binding DisplayName, ElementName=Presence}"
Margin="4,0,0,0"
VerticalAlignment="Center"
/>
<controls:StartVideoCallButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}" x:Name="startVideoCall" />
<controls:ShareDesktopButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"/>
</StackPanel>
<CheckBox IsChecked="{Binding Source={x:Static properties:Settings.Default}, Path=autoAnswer, Mode=TwoWay}" Content="Auto Answer Video Calls"/>
</StackPanel>
</Grid>

</Window>

I also added a 3x transform to scale ALL these default controls so they'd look good on the 42" TV that is sitting in my office. Because they are native WPF vector controls they just scale perfectly to high resolutions without raster jaggies.

Then I added a call to make the app Maximized by default:

this.WindowState = System.Windows.WindowState.Maximized;

And it looks like this when running:

The World's Simplest Lync Kiosk (with Auto Answer, too!)

And when I call it automatically answers. Looks like everyone's asleep and they've turned the lights out!

My Remote office in Seattle

Ah, I but I wish it was full screen so the people in Redmond don't need to do anything or touch anything...

The Badness

I can auto-answer calls but sometimes the window isn't in front and once it gets in front there's no programmatic way to tell Lync to go Fullscreen with video.

Two bad problems there. Both solved by breaking all the rules. I get the Window Class with a big assumption that the Kiosk only as one chat window open and then I "push" F5 which is the Lync hotkey for fullscreen video.

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);
//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);

Those last two, of course, are calls directly into Win32 from .NET:

public static class UnsafeNativeMethods
{
[DllImport("user32.dll")]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
}

But, it works! It's scandalous that this isn't built into the Lync SDK. Developers who are fans of Lync or who work on it all the time will say that my attempt at a "poor man's Kiosk" is silly and that I really want to use "UI Suppression in Lync" and just make an app that hosts Lync rather than automates Lync. They are likely right. However, frankly, it looked super-hard and I was tired. So, ya. If anyone wants to work on the Kiosk with me to make it simple answer and start video and do it all without showing Lync, that'd be awesome.

Thanks

I also want to make a special nod to the InputSimulator library. It's amazing and it just works. It's WAY WAY better than SendKeys which you should stop using NOW.

The Windows Input Simulator provides a simple .NET (C#) interface to simulate Keyboard or Mouse input using the Win32 SendInput method. All of the Interop is done for you and there's a simple programming model for sending multiple keystrokes.

Enjoy!

Lync Developer Resources

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
July 03, 2012 15:03
Nice! Looking forward to an influx of inexpensive touchscreen monitors / tablet machines to run something like this on so they can just tap on your face to start a chat.

When the lights come on can you let us know how easy it is to read what someone is drawing on your whiteboard and also what the audio quality is like at either end? (i.e. is the BCC950 better audio than the old crisp rustling Cisco Umi)
July 03, 2012 20:48
On the remote machine where this app runs, do you have the machine auto-login with an account other than your account? I saw ESP Cart #7 in one of the screenshots so I'm assuming that is the name on an account you got created just for this purpose? Probably an account with zero permissions?
July 03, 2012 21:02
Bill Gates' legendary 1997 mug shot


He seems to have aged a lot in the 15 years since then ;-)

(1977 maybe?)
July 03, 2012 22:19
Brian - Yes, I have a Lync online account that doesn't have desktop logon permissions. There's a local account that auto-logins but doesn't have domain access.

Paul - Thanks!
July 04, 2012 15:08
The Amazon link you provided for the Logitech BCC950 Conference Cams isn't very anonymous, it shows your browsing history and suggested products
July 05, 2012 1:16
Phil - Mine? Or yours. It's a standard Amazon link. Are you logged in?
July 05, 2012 7:22
Either your GitHub project is incorrectly named or the blog article name is. One says Lync 2012 and the other says Lync 2010. Yep, I'm THAT guy apparently.
July 05, 2012 11:31
Scott, sorry, the link is fine. I wasn't logged in,
it didn't show any recent items, just top sellers.
July 05, 2012 14:49
Now, if we could only get a Lync Web access as well.. :)
July 11, 2012 18:24
Scott, This is neat. Can Auto answer be configured to accept multiple parties without a lobby, so you create a "Virtual Room" that anyone can go to (or on white list), chat to each other and look around at the same time ? I have been seeking a way to create this, just for regularly scheduled team meetings with a geographically dispersed group. The web cam could be dropped for this, or just be a conversation point on the weather from the roof, while waiting for the rest of the team.
July 12, 2012 17:57
Matthew - Sure, look at the code. You could make a whitelist and participants and check the list before you answer.
July 20, 2012 17:19
Hi Scott, loved the recent demo of this in Glasgow.
August 09, 2012 0:10
Scott,

I'd be happy to help. I don't know anything about coding Lync, but am willing to give it a try and fumble around learning it. I love to try new projects outside my realm (e.g. awhile ago I coded a web service that allowed FoxPro, don't ask why, to consume it and create Outlook task, emails, and appointments, and then update them/delete/etc. Never did Outlook programming before in C# and it was interesting to say the least. Let me know if you still want help.

David
October 16, 2012 22:53
Scott,

I remember reading one of your blog posts about a reference to a .net library which was on nuget to enable 2 way communication through firewall/proxy and it requred you to have part of it hosted outside the corporate and home network which you solved using Windows Azure.

Do you recall what this .net libaray was? :) I keep searching your posts but can't find it.

Mark
October 16, 2012 22:56
I found it on your site.. It was SignalR.

Thanks ;)

Mark
November 19, 2012 13:36
Hi Scott,

Very cool project.

I don't need anything nearly as involved as this - just want to log into a remote system then control the cam (connected to the remote system) from the remote desktop.

Your console tester is perfectly adequate for this.

Strange issue though - Up and Down only works if done at least once physically. Before that, the software has no effect.

Left, Right, Home and End work as expected.
Lee
May 01, 2013 8:42
Ensuring the conversation window is full screen should be as follows rather than interfacing with the Native Win32 API:
http://msdn.microsoft.com/en-us/library/lync/jj275437.aspx
May 01, 2013 10:14
Greg - That's new to 2013. This was written on 2010.
July 16, 2013 4:26
Does anyone know how to use the ShowFullScreen for a StartVideoCallButton control for WPF?
August 12, 2013 7:00
Can you share your complete source?? :)
August 12, 2013 10:10
David - It's ALL on GitHub.
August 12, 2013 15:31
I found it, thanks Scott :D
September 09, 2013 16:05
Hi Scott,

Now that Lync 2013 is available on Android. Do you know if it would be possible to create some form of autoanswer for that client? If so, then it will enable a super easy solution for "low budget video rooms" using the MK602 device from Rikomagic http://store.cloudsto.com/android-mini-pc-s/rikomagic-magic-box-mk602-detail.html that runs the Lync 2013 client with some AutoAnswer feature/service.

Yes, I know that Lync 2013 is not yet available on PAD version of Android, but the Phone edition APK works just fine also on PAD's (including the MK602 device).

best regards
Lidvar Kornberg
October 07, 2013 7:48
Hi Scott,
How can I auto accept "Unmute" event? It's mean:
In a conversation, when participant A select "Unmute" participant B, B will be display a notification "A prensenter wants you to unmute yourself."
So I want to auto accept this event without click anything.
Please help me.
Thank you very much.

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.