Scott Hanselman

Introducing Lync 2010 Super Simple Auto Answer Video Kiosk with Full Screen

July 3, '12 Comments [25] Posted in Lync | Open Source | Remote Work | WPF
Sponsored By

Logitech BCC950 Conference CamIf you check out my blog archives or check out the Remote Work category you'll see that I'm always tinkering with my remote work situation. I'm the most interested in high quality and pervasive video. I'm so interested in this that for a while I was running a 10 hour a day persistent "Portal" between Portland and Seattle. I still highly recommend this for co-located teams that have the bandwidth to burn. It's great to be able to turn one's head and see their teammate right there by their side - even though they are 200 miles away.

I recently picked up a pair of Logitech BCC950 Conference Cams as a possible replacement for the very expensive "RoundTables" that some rooms at Microsoft have. The RoundTables are lovely but they are becoming scarce at the office and the Logitech is literally one-tenth the price. I'll blog a full and detailed review of the BCC950 later on this week but for now my biggest issue was that the Video Kiosk software I was using was starting to show its age. It's flaky and unreliable and build on the Office Communicator 2007 interfaces while I've been on Lync 2010 for a while.

Additionally, the researchers that wrote the software are always adding features I don't need for hardware I don't have. My remote buddy Damian Edwards and I decided we needed to make a software break.

Features

We want these simple features to start:

  • Auto-answer of video calls - possibly with some whitelist for security
  • Auto-fullscreen of video calls on the remote machine - the single purpose kiosk in my Seattle office
  • Presence information and a simple UI for making calls - by simple I mean "my boss's boss" simple
  • Remote control of Pan Tilt Zoom (PTZ) features on the same - ideally using the standard "inbox UVC" drivers and no 3rd party software

Tonight I sat down and did the first three of these and put it on GitHub. I call it - wait for it - the Lync 2012 Super Simple Auto Answer Video Kiosk with Full Screen since the name "SmartGlass" was already taken. ;)

I searched ALL over to find out if there was SOME sample code out there that would just auto-answer a call from Lync and start video. I could find dozens of samples that made calls, that started chats, but none that would answer and auto-start video. You'd think this would be the FIRST thing that folks would want to do. I can only assume it's not a setting for security reasons.

Auto-Answering Lync Calls with Video

Now, it's late and there's likely problems so pull requests are welcome if I have done something stupid. Lync is complex and while you'd think there'd be an "AutoAnswer = true" it's actually a more complex API than that. I started from this MSDN article on "Responding to a Lync Conversation Invitation."

var lync = LyncClient.GetClient();
var conversationmgr = lync.ConversationManager;
conversationmgr.ConversationAdded += (_, cmea) =>
{
bool IncomingAV = false;
StringBuilder sb = new StringBuilder();

//Is this an audio/video invitation?
if (cmea.Conversation.Modalities[ModalityTypes.AudioVideo].State == ModalityState.Notified)
{
if (lync.DeviceManager.ActiveAudioDevice != null)
{
sb.Append("Incoming call from ");
IncomingAV = true;
}
else
{
cmea.Conversation.Modalities[ModalityTypes.AudioVideo].Reject(ModalityDisconnectReason.NotAcceptableHere);
}
}
if (cmea.Conversation.Modalities[ModalityTypes.InstantMessage].State == ModalityState.Connected)
{
sb.Append("Incoming IM from ");
}
sb.Append(String.Join(", ", cmea.Conversation.Participants.Select(i => i.Contact.Uri)));
Debug.WriteLine(sb.ToString());

if (IncomingAV == true && Properties.Settings.Default.autoAnswer == true) //I added that setting later on
{
InitiateAVStream(cmea.Conversation);
}
};

You watch for a Conversation to start and see if it's an Audio/Video on. If it is, then we call our InitiateAVStream method. You can't do all this stuff synchronously as Lync is full of async native COM APIs and events that you need to respond to. Here we accept the video which lets us see who called us but doesn't yet start OUR video. Remember "we" are the dumb Kiosk who is receiving a call from me.

private static void InitiateAVStream(Conversation pConversation)
{
if (pConversation.State == ConversationState.Terminated) { return; }

if (pConversation.Modalities[ModalityTypes.AudioVideo].CanInvoke(ModalityAction.Connect))
{
var video = (AVModality)pConversation.Modalities[ModalityTypes.AudioVideo];
video.Accept();

//Get ready to be connected, then WE can start OUR video
video.ModalityStateChanged += _AVModality_ModalityStateChanged;
}
}

See how they are chaining handlers? I think this code could be made cleaner with a series of nested closures like above in the ConversationAdded example, but then again, maybe not. It'll get four deep before we're done.

Now the call is being connected but perhaps not yet. When its state changes the VideoChannel has opened up and we watch for the video to be received.

static void _AVModality_ModalityStateChanged(object sender, ModalityStateChangedEventArgs e)
{
VideoChannel vc = null;
switch (e.NewState)
{
//we can't start video until it's connected
case ModalityState.Connected:
if (vc == null)
{
vc = ((AVModality)sender).VideoChannel;
vc.StateChanged += new EventHandler<ChannelStateChangedEventArgs>(_VideoChannel_StateChanged);
}
break;
}
}

As the video starts up, I can see if the system is ready for the Kiosk to start its video. If so, we call BeginStart (and the SDK says we HAVE to call EndStart, so watch out!).

static void _VideoChannel_StateChanged(object sender, ChannelStateChangedEventArgs e)
{
VideoChannel vc = (VideoChannel)sender;

//Are we receiving? Let's try to send!
if (e.NewState == ChannelState.Receive)
{
if (vc.CanInvoke(ChannelAction.Start))
{
vc.BeginStart(videoCallBack, vc);
}
else { Debug.WriteLine("CanInvoke said NO!"); }

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);

//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);
}
}


private static void videoCallBack(IAsyncResult ar)
{
((VideoChannel)ar.AsyncState).EndStart(ar);
}

I'm pretty frustrated as while this is SUPER powerful, it's also SUPER complex for basic scenarios in my opinion. I think there's an opportunity here for a small layer on top of Lync that handles the 80% cases like the small Lync Abstraction  layer introduced in this "ScreenPop" example application.

The Goodness - The WPF Controls

At this point in the code I had everything working in a Console Application. You can go cherry-pick that commit if you want just a Console app that Auto-Answers video calls from Lync.

Even though I NEED to stop as I've got it working in a Console and I should be sleeping I noticed this in Visual Studio and it was too epic to not try.

File | New Lync WPF Application

You know how it is. It's 2am, you're done with your goals. OF COURSE you're going to try to convert your Console App to a GUI before bed. Of course.

Turns out there's a MESS of visual controls that you can put into existing applications to make them Lync-ified in literally minutes.

All the Lync Controls like SendEmailButton and StartVideoCallButton

Ok, awesome. I took the basic UI and added a checkbox for "Auto Answer."

Perhaps the lamest UI ever. It's a head, a button and a checkbox

(ASIDE: You DO realize that the outline of the "unknown face" in Lync there looks an AWFUL lot like Bill Gates' legendary 1977 mug shot, right? I just noticed that.)

Lync 2010's default Person matches the outline of Bill Gates' 1977 Mug Shot

Anyway, then I made two settings, one for my "sip" address (that's in my app.config file as "sipEmailAddress" and one boolean for AutoAnswer. The complete XAML is just:

<Window x:Class="SuperSimpleLyncKiosk.Main"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:properties="clr-namespace:SuperSimpleLyncKiosk.Properties"
xmlns:controls="clr-namespace:Microsoft.Lync.Controls;assembly=Microsoft.Lync.Controls"
Title="The World's Simplest Lync Kiosk (with Auto Answer, too!)" Height="Auto" Width="Auto">
<Grid RenderTransformOrigin="0.5,0.5">
<Grid.RenderTransform>
<ScaleTransform ScaleX="3" ScaleY="3"/>
</Grid.RenderTransform>

<StackPanel Orientation="Vertical" HorizontalAlignment="Center" VerticalAlignment="Center">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center" VerticalAlignment="Center">
<!-- Show the presence indicator. Hover over the icon to see the contact card.
Set Source to a valid SIP URI in your organization. -->
<controls:PresenceIndicator
x:Name="Presence"
Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"
PhotoDisplayMode="Large"
/>
<!-- Use the DisplayName property from PresenceIndicator to show the user's name -->
<TextBlock
Text="{Binding DisplayName, ElementName=Presence}"
Margin="4,0,0,0"
VerticalAlignment="Center"
/>
<controls:StartVideoCallButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}" x:Name="startVideoCall" />
<controls:ShareDesktopButton Source="{Binding Source={x:Static properties:Settings.Default}, Path=sipEmailAddress, Mode=OneWay}"/>
</StackPanel>
<CheckBox IsChecked="{Binding Source={x:Static properties:Settings.Default}, Path=autoAnswer, Mode=TwoWay}" Content="Auto Answer Video Calls"/>
</StackPanel>
</Grid>

</Window>

I also added a 3x transform to scale ALL these default controls so they'd look good on the 42" TV that is sitting in my office. Because they are native WPF vector controls they just scale perfectly to high resolutions without raster jaggies.

Then I added a call to make the app Maximized by default:

this.WindowState = System.Windows.WindowState.Maximized;

And it looks like this when running:

The World's Simplest Lync Kiosk (with Auto Answer, too!)

And when I call it automatically answers. Looks like everyone's asleep and they've turned the lights out!

My Remote office in Seattle

Ah, I but I wish it was full screen so the people in Redmond don't need to do anything or touch anything...

The Badness

I can auto-answer calls but sometimes the window isn't in front and once it gets in front there's no programmatic way to tell Lync to go Fullscreen with video.

Two bad problems there. Both solved by breaking all the rules. I get the Window Class with a big assumption that the Kiosk only as one chat window open and then I "push" F5 which is the Lync hotkey for fullscreen video.

//Go looking around for the IM Window (there had better just be the one we just started)
// and force it to the foreground
IntPtr childHandle = UnsafeNativeMethods.FindWindowEx(IntPtr.Zero, IntPtr.Zero, "IMWindowClass", null);
UnsafeNativeMethods.SetForegroundWindow(childHandle);
//Try to get the video to go full screen by pressing F5
WindowsInput.InputSimulator.SimulateKeyPress(WindowsInput.VirtualKeyCode.F5);

Those last two, of course, are calls directly into Win32 from .NET:

public static class UnsafeNativeMethods
{
[DllImport("user32.dll")]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
}

But, it works! It's scandalous that this isn't built into the Lync SDK. Developers who are fans of Lync or who work on it all the time will say that my attempt at a "poor man's Kiosk" is silly and that I really want to use "UI Suppression in Lync" and just make an app that hosts Lync rather than automates Lync. They are likely right. However, frankly, it looked super-hard and I was tired. So, ya. If anyone wants to work on the Kiosk with me to make it simple answer and start video and do it all without showing Lync, that'd be awesome.

Thanks

I also want to make a special nod to the InputSimulator library. It's amazing and it just works. It's WAY WAY better than SendKeys which you should stop using NOW.

The Windows Input Simulator provides a simple .NET (C#) interface to simulate Keyboard or Mouse input using the Win32 SendInput method. All of the Interop is done for you and there's a simple programming model for sending multiple keystrokes.

Enjoy!

Lync Developer Resources

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

If malware authors ever learn how to spell we're all screwed - the coming HTML5 malware apocalypse

June 29, '12 Comments [84] Posted in HTML5 | Musings
Sponsored By

Forgive the lousy screenshot and transparency in the title bar, but I just got this fake virus popup while searching for an image. I admit for a single moment my heart jumped.

A very scary fake virus popup

Then I thought after a few seconds as a techie (and note that all these observations just happened all at once in my head in no order):

  • The dialog is perfectly centered in the browser. I'm not sure why this was my #1 tipoff, but for me, it was the first thing I noticed.
  • This "popup" was as a result of a browser navigation. If it were legit I'd expect it to happen a little more asynchronously.
  • The word "migth" misspelling in the popup.
  • The fonts in the column headers are anti-aliased with one technique and the rest of the text doesn't  use ClearType while my machine does.
  • Poorly phrased English: "You need to clean your computer immediately to prevent the system crash."
  • There's no option other than "Clean computer." No ignore, repair, quarantine.
  • The word "computer" at the end of the first line goes too far to the right of the grid's right margin. It should have wrapped to the next line. Yes, I'm a UI nerd.
  • Their Aero theme color is GRAY and mine is BLUE.
  • Ctrl-Scroll ZOOMs the image. ;)
  • The URL is obvious nonsense.
  • Adware.Win32.Fraud? Seriously?

It's scary just to look at floating in your webpage there isn't it?

A scary fake virus popup

How is my Mom supposed to defend against this? Windows OR Mac (or tablets) the bad guys are out there, and one day they will finally learn English and put a little work and attention to detail into these things.

One day these things won't be "selectable" to prove to us that they are HTML:

I selected the virus to make it invert its colors to prove it's fake

As we enable HTML5 with local storage, geolocation, possibly native code and  and other features the bad guys will start doing the same with their malware. If you can write Doom in HTML5 there's nothing (except the skill and the will) to keep you from writing adware/scareware/malware in JavaScript. Not just the standard CSRF/XSS type JS - which is bad, I know, I used to be in banking - but sophisticated duplicates of trusted software accurately recreated entirely in HTML5/CSS3 and today's modern JS.

Google Offline Mail and extensions run in the background in my browser now, what's to say some future malware won't? Should we digitally sign HTML5 apps? Do more Extended Validation SSL Certificates? How do you defend against this?

What do you think, Dear Reader?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

NuGet 2.0 (.NET Package Manager) released - GO UPGRADE NOW and here's why

June 27, '12 Comments [30] Posted in NuGet
Sponsored By

Before we get started, take a second and head over to http://nuget.org and click Install NuGet. Actually, just do it from here. I'll wait.

Install NuGet

It's a 2.5 meg VSIX file and will take just a minute to install. It'll work on Visual Studio 2010 SP1 as well as Visual Studio 2012 RC. If you have them all installed at the same time, NuGet will prompt you to install in all of them if you like.

Weird Issues you might hit

If you are an early adopter and are testing Visual Studio 2012 RC, first, thanks. If you see a dialog box on VS2012RC that says "Configuring Extensions" and seems to sit forever, we have a bug that was fixed for RTM that is starting to surface more frequently with the recent NuGet update. The bug is a race condition that occurs intermittently when a user updates his extensions in the RC release.

Workaround for hung at "Configuring Extensions" on Visual Studio 2012 RC

  1. Close all instances of VS
  2. Examine the contents of HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\11.0\ExtensionManager\PendingDeletions
  3. Delete the folders listed for each entry
  4. Delete HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\11.0\ExtensionManager\PendingDeletions

99.9% of upgrades or installs will work just fine. If you see any other issues, run Visual Studio once as Administrator, go to Tools | Extensions Manager and uninstall NuGet, then install again.

The most common installation issue is a certificate mismatch with Visual Studio 2010 SP1. Unfortunately this one isn't NuGet's bug but you can get a Visual Studio hotfix here to fix it once and for all - http://bit.ly/vsixcertfix This should be in Windows Update one day, I hope. Either installing that hotfix or uninstalling/reinstalling as admin will fix the issue for NuGet.

Fixed Issues and New Features

Here's a query to the complete list of the 80 issues that were fixed in Version 2.0 of NuGet. NuGet has seen over 14 MILLION package downloads and there are over 6,000 unique packages in the gallery. You can see the updated stats anytime at http://stats.nuget.org.

The best fix, and the one I have personally been pushing the most on was this Issue: NuGet PowerShell Tab Completion is SLOW over a slow connection. If you are on a slow collection (I'm talking to you, New Zealand) or just appreciate speed, this is reason enough to upgrade NuGet.

Before, typing Install-Package jQuery.[TAB] would cause an HTTP call to go to OData that would return more data than was required. I'm always pushing for folks who are not in the US on 35 megabit connections. Often because I'm over there to, sucking data through 3G.

With NuGet 2.0 typing Install-Package JQuery.[TAB] will make a quick JSON call like this:

GET /api/v2/package-ids?partialId=jQuery. HTTP/1.0

Which will return, in this case, 603 bytes of JSON, as it should. It's fast.

["jQuery.Ajax.Unobtrusive", 
"jQuery.ba-throttle-debounce",
...,
"jQuery.LiveQuery",
"jQuery.MaskedInput",
"jQuery.Meow"]

And you'll get nice Intellisense for packages.

Intellisense in NuGet 2.0

New Features

Not only is NuGet 2.0 faster, but there's some new features like dependency grouping by target framework. You can vary your dependences such that one package can service .NET 2 and .NET 4 but each target framework requires a different bunch of packages. Here's a example:

<dependencies> 
<group>
<dependency id="RouteMagic" version="1.1.0" />
</group>

<group targetFramework="net40">
<dependency id="jQuery" />
<dependency id="WebActivator" />
</group>

<group targetFramework="sl30">
</group>
</dependencies>

From the docs:

Note that a group can contain zero dependencies. In the example above, if the package is installed into a project that targets Silverlight 3.0 or later, no dependencies will be installed. If the package is installed into a project that targets .NET 4.0 or later, two dependencies, jQuery and WebActivator, will be installed. If the package is installed into a project that targets an early version of these 2 frameworks, or any other framework, RouteMagic 1.1.0 will be installed. There is no inheritance between groups. If a project's target framework matches the targetFramework attribute of a group, only the dependencies within that group will be installed.

Even better, you can now group your PowerShell scripts as well as your content files by target framework. Specific scripts can run depending on your versions and specific content files can be included. This uses the same directory hierarchy you are already using for dependencies only now it works for /content and /tools as well.

Turn on "Allow NuGet to download missing packages during build" to make your life easier

Finally, do be aware that you have to explicitly give the OK to "restore packages" at least once, in order to enable NuGet to fetch a bunch of dependencies for you. Often you'll get a large project that you want to compile and perhaps that project includes a packages.config but not the packages itself (you don't want check your binary packages into source control, for example) so NuGet will restore missing packages when it's time to build. You only need to do this once to satisfy the lawyers.

Turn on Package Consent in Package Manager | General

Sneak Peak of Feature UI Features (thanks Mads!)

You know the new Ctrl-Q "search all commands" feature in Visual Studio 2012? I've seen a daily build of a possible improvement to NuGet on Mads' computer that not only searched Visual Studio local commands, but also the Visual Studio Gallery AND NuGet Packages. Leave a comment if you like this feature and I'll put pressure on Mads. Or, you will. ;)

jQuery searched for in the main Visual Studio CTRL-Q Quick search box

The Great Package Rename

We've changed the names of a BUNCH of packages (and forwarded the old names) so there's a little more logic to the Microsoft ones, at least. For example, here's the autocomplete for Microsoft.AspNet...

Microsoft.AspNet. intellisense

With this RC release, all of the NuGet packages involved in the ASP.NET products were renamed. Internally we called it the "Big Package Rename of 2012." Here is a mapping of old package names to new package names. In this list, the old names refer to prior versions of the products, including the Beta releases that shipped with VS 11 Beta.

OLD PACKAGE NEW PACKAGE
AspNetMvc Microsoft.AspNet.Mvc
AspNetRazor.Core Microsoft.AspNet.Razor
AspNetWebApi Microsoft.AspNet.WebApi
AspNetWebApi.Core Microsoft.AspNet.WebApi.Core
AspNetWebApi.SelfHost Microsoft.AspNet.WebApi.SelfHost
AspNetWebPages.Core Microsoft.AspNet.WebPages
AspNetWebPages Microsoft.AspNet.WebPages.Administration
jQuery.Ajax.Unobtrusive Microsoft.jQuery.Unobtrusive.Ajax
jQuery.Validation.Unobtrusive Microsoft.jQuery.Unobtrusive.Validation
Microsoft.Web.Optimization Microsoft.AspNet.Web.Optimization
SqlServerCompact Microsoft.SqlServer.Compact
System.Net.Http Microsoft.Net.Http
System.Net.Http.Formatting Microsoft.AspNet.WebApi.Client
System.Web.Providers Microsoft.AspNet.Providers
System.Web.Providers.Core Microsoft.AspNet.Providers.Core
System.Web.Providers.LocalDb Microsoft.AspNet.Providers.LocalDb
System.Web.Providers.SqlCE Microsoft.AspNet.Providers.SqlCE

We are hoping the other companies (and others inside of Microsoft) will follow the same standard naming structure

Hosting your own Feeds (and other NuGet sightings in the community)

If you haven't noticed there's a bunch of cool NuGet-specific sites and applications showing up in the wild.

  • MyGet - Create and host your own NuGet feed in the cloud. Host for your company, add security, privileges and more. Great for companies as well as automated build systems.
  • SymbolSource - "SymbolSource is an integrated solution for hosting and browsing code releases - specifically, but not only, NuGet and OpenWrap packages. It's true power, however, comes from implementing the srcsrv protocol, which allows Visual Studio and other compatible software to download on-demand symbol (PDB) and source files from SymbolSource."
  • TeamCity - TeamCity 7 supports packing and publishing of Nuget packages via a NuGet plugin (thanks Eugene Petrenko!) that is now installed by default!
  • Sonatype Nexus - Supports the Java Maven repository and now NuGet.
    • Allows the customer to have a local copy of the entire NuGet repository
    • Allow the customer to select which license types of software they support and only show them matching NuGet’s
    • Allow the customer to see that the NuGet’s they are consuming do not contain code copied from other projects
  • NuGet Server written in Java - Eugene created a small NuGet server that you can run under Linux and Java 1.6. It's all part of the larger TeamCity NuGet support and on GitHub as well as NuGet itself (inception!)
  • WebMatrix 2 - WebMatrix not only supports NuGet but it includes the gallery as a toolbar button and uses NuGet to install additional functionality like iPhone and iPad simulators!
  • NuGetFeed - Create a personalized feed of packages you care about and never miss another update!
  • ProGet is an NuGet repository for the enterprise, includes LDAP-based permissions and scoped feeds for multiple teams. Host private NuGet packages, as well as cache and filter other repositories. Free edition available.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Back To Basics: You aren't smarter than the compiler. (plus fun with Microbenchmarks)

June 26, '12 Comments [36] Posted in Back to Basics
Sponsored By

Microbenchmarks are evil. Ya, I said it. Folks spend hours in tight loops measuring things trying to find out the "best way" to do something and forget that while they are changing 5ms between two techniques they've missed the 300ms Database Call or the looming N+1 selects issue that has their ORM quietly making even more database calls.

My friend Sam Saffron says we should "take global approach to optimizations." Sam cautions us to avoid trying to be too clever.

// You think you're slick. You're not.
// faster than .Count()? Stop being clever.
var count = (stuff as ICollection<int>).Count;

All that said, let's argue microbenchmark, shall we? ;)

I did a blog post a few months back called "Back to Basics: Moving beyond for, if and switch" and as with all blog posts where one makes a few declarative statement (or shows ANY code at all, for that matter) it inspired some spirited comments. The best of them was from Chris Rigter in defense of LINQ.

I started the post by showing this little bit of counting code:

var biggerThan10 = new List;
for (int i = 0; i < array.Length; i++){
if (array [i] > 10)
biggerThan10.Add (array[i]);
}

and then changed it into LINQ which can be either of these one liners

var a = from x in array where x > 10 select x; 
var b = array.Where(x => x > 10);

and a few questions came up like this one from Teusje:

"does rewriting code to one line make your code faster or slower or is it not worth talking about these nanoseconds?"

The short answer is, measure it. The longer answer is measure it yourself. You have the power to profile your code. If you don't know what's happening, profile it. There's some interesting discussion on benchmarking small code samples over on this StackOverflow question.

Now, with all kinds of code like this folks go and do microbenchmarks. This usually means doing something trivial a million times in a tight loop. That's lots of fun and I'm doing to do JUST that very thing right now with Chris's good work, but it's important to remember that your code is usually NOT doing something trivial a million times in a tight loop. Unless it is.

Knaģis says:

"Unfortunately LINQ has now created a whole generation of coders who completely ignores any perception of writing performant code. for/if are compiled into nice machine code, whereas .Where() creates instances of enumerator class and then iterates through that instance using MoveNext method...Please, please do not advocate for using LINQ to produce shorter, nicer to read etc. code unless it is accompanied by warning that it affects performance"

I think that LINQ above could probably be replaced with "datagrids" or "pants" or "google" or any number of conveniences but I get the point. Some code is shown in the comments where LINQ appears to be 10x slower. I can't reproduce his result.

Let's take Chris's comment and deconstruct it. First, taking an enumerable Range as an array and spinning through it.

var enumerable = Enumerable.Range(0, 9999999);
var sw = new Stopwatch();
int c = 0;

// approach 1

sw.Start();
var array = enumerable.ToArray();
for (int i = 0; i < array.Length; i++)
{
if (array[i] > 10)
c++;
}
sw.Stop();
Console.WriteLine("Enumerable.ToArray()");
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

The "ToArray()" part takes 123ms and the for loop takes 9ms on my system. Arrays are super fast.

Starting from the enumerable itself (not the array!) we can try the Count() one liner:

// approach 2
Console.WriteLine("Enumerable.Count()");
sw.Restart();
c = enumerable.Count(x => x > 10);
sw.Stop();
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

It takes 86ms.

I can try it easily in Parallel over 12 processors but it's not a large enough sample nor is it doing enough work to justify the overhead.

// approach 3
Console.WriteLine("Enumerable.AsParallel() (12 procs)");
sw.Restart();
c = enumerable.AsParallel().Where(x => x > 10).Count();
sw.Stop();
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

It adds overhead and takes 129ms. However, you see how easy it was to try a naïve parallel loop in this case. Now you know how to try it (and measure it!) in your own tests.

Next, let's do something stupid and tell LINQ that everything is an object so we are forced to do a bunch of extra work. You'd be surprised (or maybe you wouldn't) how often you find code like this in production. This is an example of coercing types back and forth and as you can see, you'll pay the price if you're not paying attention. It always seems like a good idea at the time, doesn't it?

//Approach 4 - Type Checking?
Console.WriteLine("Enumerable.OfType(object) ");
var objectEnum = enumerable.OfType<object>().Concat(new[] { "Hello" });
sw.Start();
var objectArray = objectEnum.ToArray();
for (int i = 0; i < objectArray.Length; i++)
{
int outVal;
var isInt = int.TryParse(objectArray[i].ToString(), out outVal);
if (isInt && Convert.ToInt32(objectArray[i]) > 10)
c++;
}
sw.Stop();
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

That whole thing cost over 4 seconds. 4146ms in fact. Avoid conversions. Tell the compiler as much as you can up front so it can be more efficient, right?

What if we enumerate over the types with a little hint of extra information?

// approach 5
Console.WriteLine("Enumerable.OfType(int) ");
sw.Restart();
c = enumerable.OfType<int>().Count(x => x > 10);
sw.Stop();
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

Nope, the type check wasn't necessarily in this case. It took 230ms and added overhead. What if this was parallel?

// approach 6
Console.WriteLine("Enumerable.AsParallel().OfType(int) ");
sw.Restart();
c = enumerable.AsParallel().OfType<int>().Where(x => x > 10).Count();
sw.Stop();
Console.WriteLine(c.Dump());
Console.WriteLine(sw.ElapsedMilliseconds.Dump());

That's 208ms, consistently. Slightly faster, but ultimately I shouldn't be doing unnecessary work.

In this simple example of looping over something simple, my best bet turned out to be either the Array (super fast if it was an Array to start) or a simple Count() with LINQ. I measured, so I would know what was happening, but in this case the simplest thing also performed the best.

What's the moral of this story? Measure and profile and make a good judgment. Microbenchmarks are fun and ALWAYS good for an argument but ultimately they exists only so you can know your options, try a few, and pick the one that does the least work. More often than not (not always, but usually) the compiler creators aren't idiots and more often than not the simplest syntax will the best one for you.

Network access, database access, unnecessary serializations, unneeded marshaling, boxing and unboxing, type coercion - these things all take up time. Avoid doing them and when do you do them, don't just know why you're doing them, but also that you are doing them.

Is it fair to say "LINQ is evil and makes things slow?" No, it's fair to say that code in general can be unintuitive if you don't know what's going on. There can be subtle side-effects whose time can get multiplied inside of a loop. This includes type checking, type conversion, boxing, threads and more.

The Rule of Scale: The less you do, the more you can do of it.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Managing the Cloud from the Command Line

June 20, '12 Comments [16] Posted in Azure | Open Source
Sponsored By

I blogged about the Windows Azure cloud a few weeks ago. I'm digging the new stuff and trying different scenarios on Macs, PCs and Linux (I prefer Ubuntu). As a long time PowerShell and Command Line fan I'm always looking for ways to do stuff "in text mode" as well as scripting site creations and deployments.

Turns out there are a mess of ways to access Azure from the command line - more than even I thought. There's a JSON-based Web API that these tools end up talking to. You could certainly call that API directly if you wanted, but the command line tools are damn fun.

You can install the Mac Azure SDK installer to get the tools and more on a Mac, or if you install node.js on Windows or Mac or Linux you can use the Node Package Manager (npm) to install Azure tools like this:

npm install azure-cli --global

You can also use apt-get or other repository commands. After this, you can just run "azure" which gives you these commands that you link together in a very intuitive way, "azure topic(noun)verb option" so "azure site list" or "azure vm disk create" and the like.

Azure command line format

There's even ASCII art, and who doesn't like that. ;)

Seriously, though, it's slick. Here's a sample interaction I did just now. I trimmed some boring stuff but this is starting from a fresh machine with no tools and ending with me interacting with my Windows Azure account.

scott@hanselmac:~$ npm install azure-cli
npm http GET https://registry.npmjs.org/azure
...bunch of GETS...
scott@hanselmac:~$ azure
info: _ _____ _ ___ ___
info: /_\ |_ / | | | _ \ __|
info: _ ___/ _ \__/ /| |_| | / _|___ _ _
info: (___ /_/ \_\/___|\___/|_|_\___| _____)
info: (_______ _ _) _ ______ _)_ _
info: (______________ _ ) (___ _ _)
info:
info: Windows Azure: Microsoft's Cloud Platform
info:
info: Tool version 0.6.0
...bunch of help stuff...

scott@hanselmac:~$ azure account download
info: Executing command account download
info: Launching browser to http://go.microsoft.com/fwlink/?LinkId=254432
help: Save the downloaded file, then execute the command
help: account import
info: account download command OK
scott@hanselmac:~$ cd ~
scott@hanselmac:~$ cd Downloads/
scott@hanselmac:~/Downloads$ ls
3-Month Free Trial.publishsettings
scott@hanselmac:~/Downloads$ azure account import 3-Month\ Free\ Trial.publishsettings
info: Executing command account import
info: Setting service endpoint to: management.core.windows.net
info: Setting service port to: 443
info: Found subscription: 3-Month Free Trial
info: Setting default subscription to: 3-Month Free Trial
warn: Remember to delete it now that it has been imported.
info: Account publish settings imported successfully
info: account import command OK
scott@hanselmac:~/Downloads$ azure site list
info: Executing command site list
+ Enumerating locations
+ Enumerating sites
data: Name State Host names
data: ----------------- ------- -------------------------------------------------------------
data: superawesome Running superawesome.azurewebsites.net,www.acustomdomain.com
info: site list command OK
scott@hanselmac:~/Downloads$ azure site list --json
[
{
"AdminEnabled": "true",
"AvailabilityState": "Normal",
"EnabledHostNames": [
"superawesome.azurewebsites.net",
"www.acustomdomain.com"
],
"HostNames": [
"ratchetandthegeek.azurewebsites.net",
"www.ratchetandthegeek.com"
"State": "Running",
"UsageState": "Normal",
"WebSpace": "eastuswebspace"
}
]

Here's how I can create and start a VM from the command line. First I'll list the available images I can start with, then I create it. I wait for it to get ready, then it's started and ready to remote (RDP, SSH, etc) into.

scott@hanselmac:~$ azure vm image list
info: Executing command vm image list
+ Fetching VM images
data: Name Category OS
data: -------------------------------------------------------------------------- --------- -------
data: CANONICAL__Canonical-Ubuntu-12-04-amd64-server-20120528.1.3-en-us-30GB.vhd Canonical Linux
data: MSFT__Windows-Server-2012-RC-June2012-en-us-30GB.vhd Microsoft Windows
data: MSFT__Sql-Server-11EVAL-11.0.2215.0-05152012-en-us-30GB.vhd Microsoft Windows
data: MSFT__Win2K8R2SP1-120514-1520-141205-01-en-us-30GB.vhd Microsoft Windows
data: OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd OpenLogic Linux
data: SUSE__openSUSE-12-1-20120603-en-us-30GB.vhd SUSE Linux
data: SUSE__SUSE-Linux-Enterprise-Server-11SP2-20120601-en-us-30GB.vhd SUSE Linux
info: vm image list command OK
scott@hanselmac:~$ azure vm create hanselvm MSFT__Windows-Server-2012-RC-June2012-en-us-30GB.vhd scott superpassword --location "West US"
info: Executing command vm create
+ Looking up image
+ Looking up cloud service
+ Creating cloud service
+ Retrieving storage accounts
+ Creating VM
info: vm create command OK
scott@hanselmac:~$ azure vm list
info: Executing command vm list
+ Fetching VMs
data: DNS Name VM Name Status
data: --------------------- -------- ------------
data: hanselvm.cloudapp.net hanselvm Provisioning
info: vm list command OK

That's the command line tool for Mac, Linux, and optionally Windows (if you install node and run "npm install azure --global") and there's PowerShell commands for the Windows admin. It's also worth noting that you can check out all the code for these as they are all open source and up on github at http://github.com/windowsazure. The whole command line app is written in JavaScript, in fact.

Just as the command line version of the management tools has a very specific and comfortable noun/verb/options style, the cmdlets are very "PowerShelly" and will feel comfortable folks who are used to PowerShell. The documentation and tools are in a Preview mode and are under ongoing development, so you'll find some holes in the documentation.

The PowerShell commands all work together and data is passed between them. Here a new Azure VM configuration is created while the VM Name is pull from the list, then the a provisioning config object is passed into New-AzureVM.

C:\PS>New-AzureVMConfig -Name "MySUSEVM2" -InstanceSize ExtraSmall -ImageName (Get-AzureVMImage)[7].ImageName 
` | Add-AzureProvisioningConfig –Linux –LinuxUser $lxUser -Password $adminPassword
` | New-AzureVM

Next, I want to figure out how I can spin up a whole farm of websites from the command line, deploy an app to the new web farm, configure the farm for traffic, then load test it hard, all from the command line. Such fun!


Sponsor: I want to thank the folks at DevExpress for sponsoring this week's feed. Check out their DXperience tools, they are amazing. You can create web-based iPad apps with ASP.NET and Web Forms. I was personally genuinely impressed. Introducing DXperience 12.1 by DevExpress - The technology landscape is changing and new platforms are emerging. New tools by DevExpress deliver next-generation user experiences on the desktop, on the Web or across a broad array of Touch-enabled mobile devices.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.