Scott Hanselman

Cloud-Controlled Remote Pan Tilt Zoom Camera API for a Logitech BCC950 Camera with Azure and SignalR

October 23, '12 Comments [33] Posted in Hardware | Lync | Open Source | Remote Work
Sponsored By

I like my giant blog post titles. Nyah.

Logitech BCC950 ConferenceCam, HD 1080p Video at 30 fps, 78deg. Field of View, USB 2.0 CompliantAs a remote worker for almost 5 years now, I live in video conferences. I feel really strongly about the power of seeing someone's face rather than just being a voice on a scratchy speaker phone. I've build an AutoAnswer Kiosk for Lync with some friends that you can get for free at http://lyncautoanswer.com (and read about the code here), I've got a BusyLight so the kids know I'm on a call, and the Holy Grail for the last few years has been a reliable Pan Tilt Zoom camera that I could control remotely.

Related Reading

A few years ago I super-glued a LifeCam camera to an Eagletron TrackerPod and build a web interface to it. I wanted to do this on the cheap as I can't afford (and my boss is into) a $1500 Panasonic IP Camera.

The Solution...er, the Problem

I have found my camera and built my solution. The Logitech BCC950 Conference Cam is the best balance between cost and quality and it's got Pan Tilt and (digital) Zoom functionality. The Zoom is less interesting to me than the motorized Pan Tilt.

Let's think about the constraints.

  • A Logitech BCC950 PTZ camera is installed on a Windows machine in my office in Seattle.
  • I'm anywhere. I'm usually in Portland but could be in a hotel.
    • I may or may not be VPN'ed into work. This means I want to be able to communicate with the camera across networks, traverse NATs and generally not worry about being able to connect.
  • I want to be able to control the camera in a number of ways, Web API, whatever, but ideally with cool buttons that are (or look) integrated with my corporate instant messaging system.

There's three interesting parts here, then.

  1. Can I even control the camera's PTZ functions programmatically?
  2. Can I relay messages across networks to the camera?
  3. Can I make a slick client interface easily?

Let's figure them out one at a time.

Can I even control the camera's PTZ functions programmatically?

I looked all over and googled my brains out trying to find an API to talk to the Logitech camera. I emailed the Logitech people and they folks me that the camera would respond to DirectShow APIs. This means I can control the camera without any drivers!

MSDN showed me PROPSETID_VIDCAP_CAMERACONTROL which has an enumeration that includes things like:

This lead me to this seven year old DirectShow .NET library that wraps the hardest parts of the DirectShow COM API. There's a little utility called GraphEdt.exe (GraphEdit) that you can get in the Windows SDK that lets you look at all the DirectShow-y things and devices and filters on your system.

GraphEdit

This utility let me control the camera's Zoom but Pan and Tilt were grayed out! Why?

GraphEdit showing Pan and Tilt grayed out

Turns out that this Logitech Camera supports only relative Pan and Tilt, not absolute. Whatever code that creates this Properties dialog was never updated to support a relative pan and tilt but the API supports it via KSPROPERTY_CAMERACONTROL_PAN_RELATIVE!

That means I can send a start message quickly followed by a stop message to pan. It's not super exact, but it should work.

Here's the C# code for my move() method. Note the scandalous Thread.Sleep call.

private void MoveInternal(KSProperties.CameraControlFeature axis, int value)
{
// Create and prepare data structures
var control = new KSProperties.KSPROPERTY_CAMERACONTROL_S();

IntPtr controlData = Marshal.AllocCoTaskMem(Marshal.SizeOf(control));
IntPtr instData = Marshal.AllocCoTaskMem(Marshal.SizeOf(control.Instance));

control.Instance.Value = value;

//TODO: Fix for Absolute
control.Instance.Flags = (int)CameraControlFlags.Relative;

Marshal.StructureToPtr(control, controlData, true);
Marshal.StructureToPtr(control.Instance, instData, true);
var hr2 = _ksPropertySet.Set(PROPSETID_VIDCAP_CAMERACONTROL, (int)axis,
instData, Marshal.SizeOf(control.Instance), controlData, Marshal.SizeOf(control));

//TODO: It's a DC motor, no better way?
Thread.Sleep(20);

control.Instance.Value = 0; //STOP!
control.Instance.Flags = (int)CameraControlFlags.Relative;

Marshal.StructureToPtr(control, controlData, true);
Marshal.StructureToPtr(control.Instance, instData, true);
var hr3 = _ksPropertySet.Set(PROPSETID_VIDCAP_CAMERACONTROL, (int)axis,
instData, Marshal.SizeOf(control.Instance), controlData, Marshal.SizeOf(control));

if (controlData != IntPtr.Zero) { Marshal.FreeCoTaskMem(controlData); }
if (instData != IntPtr.Zero) { Marshal.FreeCoTaskMem(instData); }
}

All the code for this PTZDevice wrapper is here. Once that library was working, creating a little console app to move the camera around with a keyboard was trivial.

var p = PTZDevice.GetDevice(ConfigurationManager.AppSettings["DeviceName"], PTZType.Relative);
while (true)
{
ConsoleKeyInfo info = Console.ReadKey();
if (info.Key == ConsoleKey.LeftArrow)
{
p.Move(-1, 0);
}
else if (info.Key == ConsoleKey.RightArrow)
{
p.Move(1, 0);
}
else if (info.Key == ConsoleKey.UpArrow)
{
p.Move(0, 1);
}
else if (info.Key == ConsoleKey.DownArrow)
{
p.Move(0, -1);
}
else if (info.Key == ConsoleKey.Home)
{
p.Zoom(1);
}
else if (info.Key == ConsoleKey.End)
{
p.Zoom(-1);
}
}

Also easy was a simple WebAPI. (I put the name of the camera to look for in a config file in both these cases.)

[HttpPost]
public void Move(int x, int y)
{
var p = PTZDevice.GetDevice(ConfigurationManager.AppSettings["DeviceName"], PTZType.Relative);
p.Move(x,y);
}

[HttpPost]
public void Zoom(int value)
{
var p = PTZDevice.GetDevice(ConfigurationManager.AppSettings["DeviceName"], PTZType.Relative);
p.Zoom(value);
}

At this point I've got the camera moving LOCALLY. Next, I mail it to Damian (my office buddy) in Seattle and he hooks it up to my office computer. But I need something to control it running on THAT machine...and talking to what?

Can I relay messages across networks to the camera?

Here's the architecture. Since I can't talk point to point via TCP between wherever I am and wherever the camera is, I need a relay. I could use a Service Bus Relay which would be great for something larger but I wanted to see if I could make something even simpler. I'd like to use HTTP since it's, well, it's HTTP.

A Diagram showing my laptop talksk via SignalR through Azure to the camera in Seattle

Since Azure lets me have 10 free websites and automatically supports SSL via a wildcard cert for sites at the *.azurewebsites.net domain, it was perfect for what I needed. I want to use SSL because it's the best way to guarantee that my traffic not be affected by corporate proxy servers.

There's three parts. Let's start in the middle. What's the Relay look like? I'm going to use SignalR because it will let me not only call methods easily and asynchronously but, more importantly, it will abstract away the connection details from me. I'm looking to relay messages over a psuedo-persistent connection. 

So what's the code look like for a complex relay system like this? ;)

using System;
using SignalR.Hubs;

namespace PTZSignalRRelay
{
public class RelayHub : Hub
{
public void Move(int x, int y, string groupName)
{
Clients[groupName].Move(x, y); //test
}

public void Zoom(int value, string groupName)
{
Clients[groupName].Zoom(value);
}

public void JoinRelay(string groupName)
{
Groups.Add(Context.ConnectionId, groupName);
}
}
}

Crazy, eh? That's it. Clients call JoinRelay with a name. The name is the name of the computer with the camera attached. (More on this later) This means that this single relay can handle effectively any number of clients. When a client calls to Relay with a message and group name, the relay then broadcasts to clients that have that group name.

Can I make a slick client interface easily?

I created a super basic WPF app that's just a transparent window with buttons. In fact, the background isn't white or black, it's transparent. It's a SolidColorBrush that is all but invisible. It's not totally transparent or I wouldn't be able to grab it with the mouse!

<SolidColorBrush x:Key="NotQuiteTransparent" Color="#01000000"></SolidColorBrush>

The buttons use the .NET SignalR library and call it like this.

HubConnection connection = null;
IHubProxy proxy = null;
string remoteGroup;
string url;

private void MainWindow_MouseDown(object sender, MouseButtonEventArgs e)
{
if (e.ChangedButton == MouseButton.Left)
this.DragMove();
}

private async void MoveClick(object sender, RoutedEventArgs e)
{
var ui = sender as Control;
Point p = Point.Parse(ui.Tag.ToString());
await proxy.Invoke("Move", p.X, p.Y, remoteGroup);
}

private async void ZoomClick(object sender, RoutedEventArgs e)
{
var ui = sender as Control;
int z = int.Parse(ui.Tag.ToString());
await proxy.Invoke("Zoom", z, remoteGroup);
}

private async void MainWindow_Loaded(object sender, RoutedEventArgs e)
{
url = ConfigurationManager.AppSettings["relayServerUrl"];
remoteGroup = ConfigurationManager.AppSettings["remoteGroup"];
connection = new HubConnection(url);
proxy = connection.CreateProxy("RelayHub");
await connection.Start();
await proxy.Invoke("JoinRelay", remoteGroup);
}

The client app just needs to know the name of the computer with the camera it wants to control. That's the "GroupName" or in this case, from the client side, the "RemoteGroup." Then it knows the Relay Server URL, like https://foofooserver.azurewebsites.net. The .NET client uses async and await to make the calls non-blocking so the UI remains responsive.

Here's a bunch of traffic going through the Relay while I was testing it this afternoon, as seen by the Azure Dashboard.

Traffic as shown in a graph on the Azure Dashboard

The client calls the Relay and the Relay broadcasts to connected clients. The Remote Camera Listener responds to the calls. We get the machine name, join the relay and setup two methods that will respond to Move and Zoom.

The only hard thing we ran into (Thanks David Fowler!) was that the calls to the DirectShow API actually have to be on a UI thread rather than a background thread, so we have to get the current SynchronizationContext and post our messages with it. This results in a little indirection but it's not too hard to read. Note the comments.

private async void MainWindow_Loaded(object sender, RoutedEventArgs e)
{
var deviceName = ConfigurationManager.AppSettings["DeviceName"];
device = PTZDevice.GetDevice(deviceName, PTZType.Relative);

url = ConfigurationManager.AppSettings["relayServerUrl"];
remoteGroup = Environment.MachineName; //They have to hardcode the group, but for us it's our machine name
connection = new HubConnection(url);
proxy = connection.CreateProxy("RelayHub");

//Can't do this here because DirectShow has to be on the UI thread!
// This would cause an obscure COM casting error with no clue what's up. So, um, ya.
//proxy.On<int, int>("Move",(x,y) => device.Move(x, y));
//proxy.On<int>("Zoom", (z) => device.Zoom(z));

magic = SynchronizationContext.Current;

proxy.On<int, int>("Move", (x, y) => {
//Toss this over the fence from this background thread to the UI thread
magic.Post((_) => {
Log(String.Format("Move({0},{1})", x,y));
device.Move(x, y);
}, null);
});

proxy.On<int>("Zoom", (z) => {
magic.Post((_) =>
{
Log(String.Format("Zoom({0})", z));
device.Zoom(z);
}, null);
});

try {
await connection.Start();
Log("After connection.Start()");
await proxy.Invoke("JoinRelay", remoteGroup);
Log("After JoinRelay");
}
catch (Exception pants) {
var foo = (WebException)pants.GetBaseException();
StreamReader r = new StreamReader(foo.Response.GetResponseStream());
string yousuck = r.ReadToEnd();
Log(yousuck);
throw;
}
}

It All Works Together

Now I've got all the parts. Buttons that call a Relay that then call back - through NAT and networks - to the Remote Camera Listener which uses the Camera library to move it.

It's ALIVE and it's awesome

It works like a champ. And, because the buttons are transparent, I can put them over the Lync window and pretend it's all integrated.

TODO: I'm hoping that someone who knows more about Windows Internals will volunteer to create some code that will automatically move the buttons as the Lync Window moves and position them over the video window in the corner. Ahem.

The buttons look integrated. But they aren't.The buttons look integrated. But they aren't.

You can set this up yourself, but I haven't gotten around to making an install or anything. If you have a Logitech BCC950 you are welcome to use my Relay until it costs me something. There's a preliminary download up here so you'd only need the Listener on one side and the Buttons on the other. No drivers are needed since we're using DirectShow itself.

This was great fun, and more importantly, I use this PanTiltZoom System ever day and it makes my life better. The best was that I was able to do the whole thing in C#. From client UI to cloud-based relay to device control to COM wrapper, it was all C#. It makes me feel very empowered as a .NET developer to be able to make systems like this with a minimal amount of code.

Lync Developer Resources

Related Links


Sponsor: Big thanks to this week's sponsor. Check them out, it's a great program, I've done it and spoken to actual live humans who help you get started writing an app! Begin your 30-day journey to creating a Windows Store app or game for Windows 8 or Windows Phone today. Your Idea. Your App. 30 Days.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Introducing LyncAutoAnswer.com - An open source remote worker's Auto Answer Kiosk with Lync 2010

August 15, '12 Comments [11] Posted in Lync | Remote Work
Sponsored By

The name badge on my door in Redmond, Washington says "Virtual Scott Hanselman" because I'm not there!Last month I did a blog post called "Introducing Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen" where I rewrote the original Embodied Social Proxy software that I'd borrowed from Microsoft Research and just made a better "auto answer client" for Lync. (Lync being the Microsoft Unified Communications software that includes phone calls, IMs and video chats. You get Lync with Office 365.)

This auto-answer software automated the Lync client. You'd still run the Lync client and see the Lync client show up in the Windows taskbar. However, sometimes you might want a simpler "kiosk" solution where the manager end user can't even see Lync or know that it's there. Lync calls this "UI Suppressed" mode. In this mode YOU are Lync and you have to automate the entire solution, paint the video, and manage events. You are using the Lync transport but providing all the UI. The benefits of UI Suppressed mode is that the user can't mess it up or fiddle with Lync. This can be useful in CRM applications where you might want chat and video embedded in some larger system where the user doesn't need to know you're using Lync.

In my case, I want people to come by my physical office in Redmond, Washington - where I am NOT - and sit down and have everything Just Work™. I decided I needed both a Kiosk and a non-Kiosk version. I started coding and realized not only that I was in over my head but also that the idea of a UI Suppression in Lync was in need of an abstraction. The ideas were high level and general but the Lync 2010 SDK code was, in my estimation, rather too low level to make it super easy. Why not wrap Lync and make a reusable library.

I reached out to Tom Morgan in the UK who knows Lync backwards and forwards and works for Modality Systems, a Lync and Unified Communications consultancy. I pitched him the idea - remotely, over Lync, in fact - and we got to pairing. I though it was a problem worth solving that could help not only the Lync community but also Remote Workers everywhere. Big thanks to Tom's bosses at Modality Systems for trusting Tom's judgment with this little weekend project! Feel free to thank them in Tom's blog post on the subject. ;)

Tom created a project on GitHub called Lync-UISuppression-VideoAutoAnswer that includes a WPF control and Lync wrapper abstraction to make UI Suppressed Lync applications easier. Then we updated my LyncAutoAnswer GitHub project to use Tom and Paul Nerney's new library! The project now includes BOTH a UI Suppressed AND non-UI Suppressed version. Tom and friends also redesigned the UI Suppressed version to be more "kiosky" and brighter so people could walk by and immediately know if I was busy or not.

http://lyncautoanswer.com

I am happy to announce (or re-announce) the LyncAutoAnswer client at LyncAutoAnswer.com as a collaboration by Tom Morgan from Thought Stuff and Paul Nearney from CodeLync and me.

Here's a few screenshots of the UI Suppressed version that you can download today to setup your own Lync based remote kiosk.

image

Here I'm away at a conference. Note that not only does the color change but my status message from Lync presence updated as well. The headshot comes directly from Lync and Active Directory.

image

The general idea is simple. You create a Lync account that is NOT you; it's an account for your cart and the one you will call. Mine is called "Cart #7." I made a Domain Account that does NOT have the ability to login to a desktop. It doesn't have file system or network share access, it can only use Lync. Then I setup my remote machine to auto logon when it boots up. I also disabled it's power management and screensaver as this is a dumb kiosk.

The video works!

Then you change the Lync Auto Answer settings like this:

<userSettings>
<SuperSimpleLyncKiosk.Properties.Settings>
<setting name="sipEmailAddress" serializeAs="String">
<value>sip:scottha@microsoft.com</value> <!-- THIS IS YOU -->
</setting>
<setting name="LyncAccountDomainUser" serializeAs="String">
<value>DOMAIN\thecart7</value>
</setting>
<setting name="LyncAccountEmail" serializeAs="String">
<value>thecart7@microsoft.com</value>
</setting>
<setting name="LyncAccountPassword" serializeAs="String">
<value><![CDATA[YOUR*CART*PASSWORDISHERE]]></value>
</setting>
</SuperSimpleLyncKiosk.Properties.Settings>
</userSettings>

And that's it! The kiosk does the rest. You can read more about the UI Suppressed version on Tom's blog post and the non-UI Suppressed version on my site. Also, note that UI Suppressed mode needs a registry key changed, but we include those .reg files for ON and OFF, 32- and 64-bit.

Again, Big thanks to Tom, Paul, their bosses, and the spirit of open source. http://lyncautoanswer.com. We look forward to redesigns, forks and pull requests as well as stories of remote work.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Is Daddy on a call? A BusyLight Presence indicator for Lync for my Home Office

July 19, '12 Comments [27] Posted in Lync | Remote Work
Sponsored By

Daddy is busyConsider this the first in a series of posts on using Lync effectively as a Remote Worker.

I've blogged about building and wiring my home office, working from home remotely, and even done a video with Chris Sells and Rory Blyth walking through the house ("Geek Developer Cribs"). Last week I put my Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen on GitHub. That software replaces the "Embodied Social Proxy" software I was running before. That solution was too complex.

I've got folks in Seattle who are interested in my "presence" and I've also got folks here at home who are interested. Specifically, it's a problem when my kids don't realize (or respect) that Daddy is on a conference call or that Daddy is recording a podcast. It's a problem even for my wife who has recently gotten in the habit of texting me from outside the door to see if I was busy.

I had an old radio On Air light that kind of worked but I never hooked it up to Lync, which is my primary source for presence and availability. Since Lync is also available on iPhone, iPad and Android, it really is the best way for anyone inside the company to know if I'm "available."

I'd consider building a system with LEDs and Netduino but the folks at BusyLight beat me to it. The BusyLight is a nice simple plug-it-in-and-forget-it light that integrates with Lync to provide a visual indicator of my presence. It installs itself as a USB HID (Human Interface Device) so it's likely possible to talk to it directly and make a presence system for other IM clients like GChat or Skype.

Lync availablity is broadcast to all clients

Any system I put in place needs to just work. I don't want to have to click anything more than I already do. I don't even bother to set Lync's status myself because it does a fine job of doing it on its own. It automatically marks me as busy when I'm in a meeting or presenting or on the phone. If I leave my computer for a while it turns yellow to indicate I'm away. When I do need to change it manually (for recording a podcast, for example) I can also just use my iPhone. Any change in my availability from my iPhone is automatically (and instantly...like 100ms instantly) broadcast to all my Lync devices including the BusyLight and my Polycom CX600 IP telephone.

I combined the USB BusyLight's already super-long USB cable with a 16-foot USB2 Active Repeater Cable in order to get all the way down the wall, over and up into the corner of my door. The BusyLight included a 3M adhesive and it's happily stuck up there and the door still closes easily.

The BusyLight changes color instantly as Lync presence changes Daddy is not busy

I will also be taking a BusyLight up to my remote office and hooking it up so that folks can see when I'm "in the office" or available for a call.

So far I'm thrilled with the results. I recorded a podcast today and I could hear the boys outside whispering to each other "ssh, it's red! Daddy's on the phone!" Ok, I could hear them, but at least they made an effort!

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Introducing Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen

July 3, '12 Comments [25] Posted in Lync | Open Source | Remote Work | WPF
Sponsored By

Logitech BCC950 Conference CamIf you check out my blog archives or check out the Remote Work category you'll see that I'm always tinkering with my remote work situation. I'm the most interested in high quality and pervasive video. I'm so interested in this that for a while I was running a 10 hour a day persistent "Portal" between Portland and Seattle. I still highly recommend this for co-located teams that have the bandwidth to burn. It's great to be able to turn one's head and see their teammate right there by their side - even though they are 200 miles away.

I recently picked up a pair of Logitech BCC950 Conference Cams as a possible replacement for the very expensive "RoundTables" that some rooms at Microsoft have. The RoundTables are lovely but they are becoming scarce at the office and the Logitech is literally one-tenth the price. I'll blog a full and detailed review of the BCC950 later on this week but for now my biggest issue was that the Video Kiosk software I was using was starting to show its age. It's flaky and unreliable and build on the Office Communicator 2007 interfaces while I've been on Lync 2010 for a while.

Additionally, the researchers that wrote the software are always adding features I don't need for hardware I don't have. My remote buddy Damian Edwards and I decided we needed to make a software break.

Features

We want these simple features to start:

  • Auto-answer of video calls - possibly with some whitelist for security
  • Auto-fullscreen of video calls on the remote machine - the single purpose kiosk in my Seattle office
  • Presence information and a simple UI for making calls - by simple I mean "my boss's boss" simple
  • Remote control of Pan Tilt Zoom (PTZ) features on the same - ideally using the standard "inbox UVC" drivers and no 3rd party software

Tonight I sat down and did the first three of these and put it on GitHub. I call it - wait for it - the Lync 2012 Super Simple Auto Answer Video Kiosk with Full Screen since the name "SmartGlass" was already taken. ;)

I searched ALL over to find out if there was SOME sample code out there that would just auto-answer a call from Lync and start video. I could find dozens of samples that made calls, that started chats, but none that would answer and auto-start video. You'd think this would be the FIRST thing that folks would want to do. I can only assume it's not a setting for security reasons.

Auto-Answering Lync Calls with Video

Now, it's late and there's likely problems so pull requests are welcome if I have done something stupid. Lync is complex and while you'd think there'd be an "AutoAnswer = true" it's actually a more complex API than that. I started from this MSDN article on "Responding to a Lync Conversation Invitation."

var lync = LyncClient.GetClient();
var conversationmgr = lync.ConversationManager;
conversationmgr.ConversationAdded += (_, cmea) =>
{
bool IncomingAV = false;
StringBuilder sb = new StringBuilder();

//Is this an audio/video invitation?
if (cmea.Conversation.Modalities[ModalityTypes.AudioVideo].State == ModalityState.Notified)
{
if (lync.DeviceManager.ActiveAudioDevice != null)
{
sb.Append("Incoming call from ");
IncomingAV = true;
}
else
{
cmea.Conversation.Modalities[ModalityTypes.AudioVideo].Reject(ModalityDisconnectReason.NotAcceptableHere);
}
}
if (cmea.Conversation.Modalities[ModalityTypes.InstantMessage].State == ModalityState.Connected)
{
sb.Append("Incoming IM from ");
}
sb.Append(String.Join(", ", cmea.Conversation.Participants.Select(i => i.Contact.Uri)));
Debug.WriteLine(sb.ToString());

if (IncomingAV == true && Properties.Settings.Default.autoAnswer == true) //I added that setting later on
{
InitiateAVStream(cmea.Conversation);
}
};

You watch for a Conversation to start and see if it's an Audio/Video on. If it is, then we call our InitiateAVStream method. You can't do all this stuff synchronously as Lync is full of async native COM APIs and events that you need to respond to. Here we accept the video which lets us see who called us but doesn't yet start OUR video. Remember "we" are the dumb Kiosk who is receiving a call from me.

private static void InitiateAVStream(Conversation pConversation)
{
if (pConversation.State == ConversationState.Terminated) { return; }

if (pConversation.Modalities[ModalityTypes.AudioVideo].CanInvoke(ModalityAction.Connect))
{
var video = (AVModality)pConversation.Modalities[ModalityTypes.AudioVideo];
video.Accept();

//Get ready to be connected, then WE can start OUR video
video.ModalityStateChanged += _AVModality_ModalityStateChanged;
}
}

See how they are chaining handlers? I think this code could be made cleaner with a series of nested closures like above in the ConversationAdded example, but then again, maybe not. It'll get four deep before we're done.

Now the call is being connected but perhaps not yet. When its state changes the VideoChannel has opened up and we watch for the video to be received.

static void _AVModality_ModalityStateChanged(object sender, ModalityStateChangedEventArgs e)
{
VideoChannel vc = null;
switch (e.NewState)
{
//we can't start video until it's connected
case ModalityState.Connected:
if (vc == null)
{
vc = ((AVModality)sender).VideoChannel;
vc.StateChanged += new EventHandler<ChannelStateChangedEventArgs>(_VideoChannel_StateChanged);
}
break;
}
}

As the video starts up, I can see if the system is ready for the Kiosk to start its video. If so, we call BeginStart (and the SDK says we HAVE to call EndStart, so watch out!).

static void _VideoChannel_StateChanged(object sender, ChannelStateChangedEventArgs e)
{
VideoChannel vc = (VideoChannel)sender;

//Are we receiving? Let's try to send!
if (e.NewState == ChannelState.Receive)
{
if (vc.CanInvoke(ChannelAction.Start))
{
vc.BeginStart(videoCallBack, vc);
}
else { Debug.WriteLine("CanInvoke said NO!"); }

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);

//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);
}
}


private static void videoCallBack(IAsyncResult ar)
{
((VideoChannel)ar.AsyncState).EndStart(ar);
}

I'm pretty frustrated as while this is SUPER powerful, it's also SUPER complex for basic scenarios in my opinion. I think there's an opportunity here for a small layer on top of Lync that handles the 80% cases like the small Lync Abstraction  layer introduced in this "ScreenPop" example application.

The Goodness - The WPF Controls

At this point in the code I had everything working in a Console Application. You can go cherry-pick that commit if you want just a Console app that Auto-Answers video calls from Lync.

Even though I NEED to stop as I've got it working in a Console and I should be sleeping I noticed this in Visual Studio and it was too epic to not try.

File | New Lync WPF Application

You know how it is. It's 2am, you're done with your goals. OF COURSE you're going to try to convert your Console App to a GUI before bed. Of course.

Turns out there's a MESS of visual controls that you can put into existing applications to make them Lync-ified in literally minutes.

All the Lync Controls like SendEmailButton and StartVideoCallButton

Ok, awesome. I took the basic UI and added a checkbox for "Auto Answer."

Perhaps the lamest UI ever. It's a head, a button and a checkbox

(ASIDE: You DO realize that the outline of the "unknown face" in Lync there looks an AWFUL lot like Bill Gates' legendary 1977 mug shot, right? I just noticed that.)

Lync 2010's default Person matches the outline of Bill Gates' 1977 Mug Shot

Anyway, then I made two settings, one for my "sip" address (that's in my app.config file as "sipEmailAddress" and one boolean for AutoAnswer. The complete XAML is just:

<Window x:Class="SuperSimpleLyncKiosk.Main"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:properties="clr-namespace:SuperSimpleLyncKiosk.Properties"
xmlns:controls="clr-namespace:Microsoft.Lync.Controls;assembly=Microsoft.Lync.Controls"
Title="The World's Simplest Lync Kiosk (with Auto Answer, too!)" Height="Auto" Width="Auto">
<Grid RenderTransformOrigin="0.5,0.5">
<Grid.RenderTransform>
<ScaleTransform ScaleX="3" ScaleY="3"/>
</Grid.RenderTransform>

<StackPanel Orientation="Vertical" HorizontalAlignment="Center" VerticalAlignment="Center">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center" VerticalAlignment="Center">
<!-- Show the presence indicator. Hover over the icon to see the contact card.
Set Source to a valid SIP URI in your organization. -->
<controls:PresenceIndicator
x:Name="Presence"
Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"
PhotoDisplayMode="Large"
/>
<!-- Use the DisplayName property from PresenceIndicator to show the user's name -->
<TextBlock
Text="{Binding DisplayName, ElementName=Presence}"
Margin="4,0,0,0"
VerticalAlignment="Center"
/>
<controls:StartVideoCallButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}" x:Name="startVideoCall" />
<controls:ShareDesktopButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"/>
</StackPanel>
<CheckBox IsChecked="{Binding Source={x:Static properties:Settings.Default}, Path=autoAnswer, Mode=TwoWay}" Content="Auto Answer Video Calls"/>
</StackPanel>
</Grid>

</Window>

I also added a 3x transform to scale ALL these default controls so they'd look good on the 42" TV that is sitting in my office. Because they are native WPF vector controls they just scale perfectly to high resolutions without raster jaggies.

Then I added a call to make the app Maximized by default:

this.WindowState = System.Windows.WindowState.Maximized;

And it looks like this when running:

The World's Simplest Lync Kiosk (with Auto Answer, too!)

And when I call it automatically answers. Looks like everyone's asleep and they've turned the lights out!

My Remote office in Seattle

Ah, I but I wish it was full screen so the people in Redmond don't need to do anything or touch anything...

The Badness

I can auto-answer calls but sometimes the window isn't in front and once it gets in front there's no programmatic way to tell Lync to go Fullscreen with video.

Two bad problems there. Both solved by breaking all the rules. I get the Window Class with a big assumption that the Kiosk only as one chat window open and then I "push" F5 which is the Lync hotkey for fullscreen video.

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);
//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);

Those last two, of course, are calls directly into Win32 from .NET:

public static class UnsafeNativeMethods
{
[DllImport("user32.dll")]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
}

But, it works! It's scandalous that this isn't built into the Lync SDK. Developers who are fans of Lync or who work on it all the time will say that my attempt at a "poor man's Kiosk" is silly and that I really want to use "UI Suppression in Lync" and just make an app that hosts Lync rather than automates Lync. They are likely right. However, frankly, it looked super-hard and I was tired. So, ya. If anyone wants to work on the Kiosk with me to make it simple answer and start video and do it all without showing Lync, that'd be awesome.

Thanks

I also want to make a special nod to the InputSimulator library. It's amazing and it just works. It's WAY WAY better than SendKeys which you should stop using NOW.

The Windows Input Simulator provides a simple .NET (C#) interface to simulate Keyboard or Mouse input using the Win32 SendInput method. All of the Interop is done for you and there's a simple programming model for sending multiple keystrokes.

Enjoy!

Lync Developer Resources

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Page 1 of 1 in the Lync category

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.