Scott Hanselman

Essential IFTTT (IfThisThenThat) - Programming Workflows for Humans using the Web's Social Glue

September 14, 2011 Comment on this post [15] Posted in Blogging | Tools
Sponsored By

There's hundreds of channels to plug into with IFTTTWow, every once in a while an app or website comes along that smacks me in the face. Not just a regular "Hey, that's cool" smack, but more of a "Duh, I should have made that and I would have had a license to print unlimited money and now those guys have already done it and no one will ever know my secret brilliance but they did it and I have no money but I should still tell everyone because they are awesome" smack in the face.

IfThisThenThat should be the next big thing on the social web. It's bloody brilliant.

Here's the dull description I made for nerds:

IFTTT is a cloud-based open-ended web workflow creator building on existing social APIs to create more sophisticated distributed aggregated tasks.

Here's my description for non-technical friend:

IfThisThenThat lets all your online stuff work together to do way more interesting stuff.

There are Channels, like Craigslist, Feeds, Twitter, etc. Those are places that data can come from or go to. There are triggers that a different for each kind of channel. One might be "the temperature goes over 100" or "I tweet something with a @mention" and then you can take action. You can combine them in any way you want.

Here's an example that I'll make right here as I blog. Let's say I want FourSquare checkins to show up in my Google Calendar. Didn't know you wanted to do that? ;) Oh, you do.

It's the magic of OAuth and the proliferation of nice, clean public APIs that makes this possible. As I add the FourSquare Channel to IFTTT, I authorize it to have access to my FourSquare data. I'm not giving them my FourSquare password, though, so I can always revoke access later if I like.

Any new checkin?

I'll select Any new check-in...then I'll select what to do from that same huge list of Channels:

What a huge list of channels!

I'll authorize Google Calendar in the same way, then the REALLY nice part shows up. Templated actions. Yes. I can add text from a dictionary of {{Addins}} (although I think that's a poor name from these {{Keywords}}, but whatever). This way I get complete control of what gets added.

Lots of Action Fields to choose from

I'm sure they'll include more documentation and details on what data is available to be passed between Channels.

Now, bask in the programming of it all. Bam.

If any new checkin on foursquare, add it to Google Calendar

Some examples are amazing in their elegant simplicity. Why is IFTT a great idea? Because it's an ideavirus. It fills a vacuum in your web experience you may not have even known existed. And now that you do, you won't be able to live without it.

If Scott blogs something, send the link to Instapaper to be read later

Hanselman to Instapaper IFTTT recipe

If MSFT stock goes above 30, email me.

Email me if our stock goes up

No, text me!

Text me if our stock goes up

That's better. Now, if I tweet something that is not a @Reply or RT, also post it to my Facebook Page:

Put new tweets on my Facebook Page

If I take any pictures and post them on Instagram, also save a copy in DropBox for me.

Download Instagram photos and save them in DropBox

Hey, you can do stuff with DropBox? Sometimes I find a file while on my phone but I can't download it. If I email you a link, save it to my DropBox, OK?

Save emailed links to DropBox

And, if I favorite something with a link, make sure to send a copy of that link to Instapaper.

Send twitter favorites to Instapaper

I know I tend to be hyperbolic, but this is cool like Yahoo Pipes was cool, except people without a PhD can use it. This is going to be huge.

There's a few interface quirks, like it's unclear how to make a new recipe (you make a task first, then click inspect task (an unlabeled arrow) there's an another unlabeled button that looks like a pharmacist's mortar and pestle (mixing UX metaphors between recipes and prescriptions) and you can create a recipe there.Still, these are nits. The workflows is there and I LOVE the giant iconography, clean typefaces, and above all, the sense of empowerment.

If you are on the interwebs and have a maze of social accounts, you need to seriously spend time time drinking in the wonder and the magic of IFTTT. It's gonna be massive. You heard it here. Now, go.

*hat tip to Rob Conery for turning me onto this!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook bluesky subscribe
About   Newsletter
Hosting By
Hosted on Linux using .NET in an Azure App Service

Xbox 360 won't Auto Login to Live - Fixed due to RSTP and Spanning Tree setting in a NetGear Managed GS724TP Switch

September 13, 2011 Comment on this post [15] Posted in Bugs | Tools
Sponsored By

Victory pose, Halo StyleYou change something months ago, then noticed an unrelated problem weeks later, then it takes a year to figure out that they were related.

"A butterfly flaps its wings in Brazil and my Xbox stops getting a DHCP lease reliably."

A few months back, as my wife recalls, the Xbox in the living room stopped automatically logging into Xbox live. It was irritating in that "First World Problem" kind of way. We use the Xbox for Netflix and Hulu and she uses Zumba and some other Kinect Fitness stuff so it was a daily reminder to me that "Daddy is a lousy IT guy."

I don't remember when it stopped working other than it did. I set static leases in the main router. Then I set static IP addresses in the router. The ONLY thing that would fix it was to get the Xbox to Refresh/Renew the IP Address (even when it was static!) by going into the System menu of My Xbox, then Network Settings, Configure Network, then Additional Settings, then Restore to Factory Defaults. <sarcasm>Let me say that the wife LOVED this.</sarcasm>

Unrelated to this issue, a few weeks back I lost some save games on my now dead PS3 so tonight I decided to go and backup all my Xbox save games onto a new USB stick. I started with the Xbox in my office, but it wouldn't log in. This set off a spark in my head. I've got literally 34 happy devices with IP Addresses in my house (don't ask) from various vendors but the two Xboxen won't login? OK, that can't be me.

All my nicely labeled network cables in a bundle.At this point, of course, I'm unable to login to Xbox live easily so I get irritated and decide that I will shift gears and finally get this thing fixed.

Even though there's no realistic way that I will bust out the IP Networking textbook from college, hook up WireShark and start sniffing traffic, I like to fool myself into believing that I am that good of a technologist (or that I care THAT much) but the thought brings me comfort.

I start following the wires around from the Xbox to the router, as they are all numbered and labeled in the wiring closet.

Aside: Here's a Bit.ly Bundled Link of the complete "Wiring your house for Gigabit Ethernet 5 PART SERIES" when I wired my house a few years back.

However, I get lost as there's just so darn many wires in the wiring closet. It's getting crowded.

My new wiring closet door, open My new wiring closet door, closed

I realize, however, that the wires aren't going directly into main Router that gives out IP addresses. They are actually plugged into a lovely Netgear 24-port Gigabit GS724TP Managed Switch. Because I wired the whole house for Gigabit and wanted basically hard drive speeds on the wire, I needed a switch with the chutzpah to push a lot of bandwidth, and this switch was the one. All the wires in the house go into it - including the Xboxen. But which ones?

I needed to figure out which cables were the two Xbox cables and plug them directly into the router. I know the MAC addresses of both Xboxes so I logged into the switch's admin console. After failing to remember the password (I set this up 4 years ago, you know) and starting to panic, I tried "password" out of frustration. It worked.

It shouldn't have.

Why was my password a default password?

Something had changed. Oh! I flashed the firmware on this GS724TP switch last year with version 5.0.0.15. It must have reset everything to the defaults.

Now that I was logged in, I checked out the list of all the MAC Addresses and their associated ports.

My two Xboxes were plugged into g14 and g22

OK, so the two consoles are plugged into Ports 14 and 22, so I found them and moved the cables to plug directly into my router, rather than the managed switch.A graphical view of my managed switch.

They worked. Suddenly a year of obnoxious just stopped. Why?

What would cause one device (or class of device) to not get an IP address consistently via DHCP?

There's an article on Windows 2000 called DHCP Renewal Failures on Switched Networks that sounds familiar.

A client connected to an Ethernet switch may not receive a renewed DHCP IP address during startup. Releasing and renewing the IP address manually works.

There's apparently a magical thing called STP (Spanning Tree Protocol) that is very useful but also totally confusing to me. I have read this Wikipedia twice and I have no idea what it does. I'm sure it's lovely, really.

The Spanning Tree Protocol (STP) is a network protocol that ensures a loop-free topology for anybridged Ethernet local area network.

Of COURSE it is.

O_O

That is my blank stare, right there, FYI.

Interestingly, if you Google with Bing or DuckDuckGo for "spanning tree DHCP" you'll get results with words like "failure," "blamed," "fixed, "slowed down," and "acting strange." Ya, spanning tree rocks.

There's also a version of Spanning Tree called RSTP or Rapid Spanning Tree Protocol. All of these protocols are meant to make things work better and recover faster after a network failure and prevent routing loops. However, they appear to cause some trouble in certain clients, like my Xbox and possibly my iPhone and IPad as well.

Here's another explanation:

The problem might be related to what type of network device the DHCP client is connected to. If the client is connected to a switch that performs a spanning tree calculation during port initialization, then the port the client is connected to may not be available for 10-15 seconds. This is long enough to prevent the DHCP client from renewing or obtaining IP addressing information, and thus preventing the TCP/IP stack from initializing.

This is not a problem on all switches that perform spanning tree calculations. Some devices will perform the calculations in the background, and thus the port becomes available immediately. Also, this type of functionality is only required on ports that are connected to other switching devices. Therefore you don't really need it enabled on ports that have clients directly connected to them

A buddy of mine says:

[Spanning Tree is] not bad, but it is there to prevent routing loops. If you are sure you won't ever connect that port to another switch and make a routing loop, an alternative to disabling spanning tree is to enable portfast and I bet that would fix it too. The problem is the normal startup procedure for a port running spanning tree is about 20-25 seconds of watching traffic on the port before enabling. Thus, when you turn your Xbox on, it only waits x seconds for a DHCP reply where x is definitely shorter than the time spanning tree takes to enable the port.

Also, deep in this Australian forum was a comment:

Just to clarify that RSTP has a faster up time than STP. I don't think this is going to directly cause the "limited or no connectivity" issue.

However for some devices you need to have the port active right away. On a cisco switch this is called portfast and it skips the process of checking for loops(RSTP, STP). On some devices (xbox 360) this can cause DHCP issues when they're first turned on.

PCs are generally immune to this as they'll poll for a DHCP service several times while your media device might only try once.

And thus, this is how I fixed the issue with my Xboxes not logging into Live on the first try. I'd flashed a network switch without even thinking and it turned Rapid Spanning Tree Protocol on, which Xboxes (and some other network devices) don't like.

My solution, check the router or switch and turn RSTP or "Spanning Tree" off.

As an aside, it turns out that I could have even turned off RSTP at the port level on this Netgear GS725TP switch. Amazing hardware, this switch. Love it. Except for when it reset settings to defaults and irritated me for a year.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook bluesky subscribe
About   Newsletter
Hosting By
Hosted on Linux using .NET in an Azure App Service

NuGet Package of the Week #10 - New Mobile View Engines for ASP.NET MVC 3, spec-compatible with ASP.NET MVC 4

September 06, 2011 Comment on this post [18] Posted in ASP.NET | ASP.NET MVC | Mobile | NuGet | NuGetPOW
Sponsored By

Desktop ASP.NET MVC Application next to the same application in a mobile browserI did some basic mobile view engine work for ASP.NET MVC for Mix in 2009 and then created what I thought was a better ASP.NET MVC Mobile ViewEngine in 2010. Unfortunately, the second one (the "better" one) had a caching bug that only showed itself in Release mode. This last month, Jon, John, Peter and I updated NerdDinner to MVC 3 with Razor and a pile of other new features. One of those new features was jQuery Mobile support and that meant we need to fix this bad Mobile View Engine. Additionally, ASP.NET MVC 4 will include actual supported Mobile Views support, so the pressure was on.

However, we wanted to make sure any new MVC 3 Mobile View sample was mostly compatible with whatever scheme ASP.NET MVC 4 uses. The original folder layout for my proposed ViewEngine was by folder but the final design was to use file names. That means instead of ~/Views/Home/Mobile/Index.cshtml, you'd have ~/Views/Home/Index.Mobile.cshtml. Of course, you can change this if you really want to yourself, but that's the default.

Alternate Views shown in Solution Explorer in subfolders Alternate Views shown in Solution Explorer separated by filename differences, not folders

Peter Mourfield jumped in and did the updated Mobile View Engines and we've put them on NuGet for you, Dear Reader.

Remember, these are for ASP.NET MVC 3. You don't need them when ASP.NET MVC 4 comes out, and the general idea will be that you will remove the Razor (or WebForms) ViewEngine and replace it with the mobile version which is a superset of functionality.

ViewEngines.Engines.Remove(ViewEngines.Engines.OfType<RazorViewEngine>().First());
ViewEngines.Engines.Add(new MobileCapableRazorViewEngine());
ViewEngines.Engines.Remove(ViewEngines.Engines.OfType<WebFormViewEngine>().First());
ViewEngines.Engines.Add(new MobileCapableWebFormViewEngine());

You can do this bit of work in Application_Start, or with the Web Activator like the MobileViewEngines.Razor.Samples does. The sample NuGet package includes both VB and C#, so you'll want to delete the one you won't use. You only need to use the ViewEngine you need, so if you aren't using WebForms, don't bother with those lines.

The whole ViewEngine that Peter made is only 81 lines of code so you can certainly change it to your taste. Peter and I put the source on BitBucket for changes, forks and fixes.

image

Just add the word Mobile in your views, like Index.Mobile.cshtml or Details.Mobile.aspx and those will be used when a mobile browser is detected. The detection  is using the standard Browser.IsMobileDevice call from ASP.NET, so consider using a browser database like http://51degrees.mobi (also on CodePlex, and NuGet).

Remember, this is a clean-room implementation (not derived from ASP.NET MVC  4) that has just basic mobile view overrides. I'm glad it doesn't have the release mode bug like my previous ones did, and we are using this implementation live on http://nerddinner.com. Modify the source if you need advanced support for multiple mobile views (like iPhone, BlackBerry, etc) other than just "mobile" like this one does. There are features that this basic ViewEngine doesn't have that a more sophisticated solution like ASP.NET MVC 4's or other folks' implementations could have like:

  • Browser Overrides: Forcing or "opting out" of mobile and using desktop
  • Device-specific custom layouts

Still, we've found it to be simple and useful on NerdDinner and we hope it's useful to you.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook bluesky subscribe
About   Newsletter
Hosting By
Hosted on Linux using .NET in an Azure App Service

A basic non-cloud-based personal backup strategy

September 05, 2011 Comment on this post [51] Posted in Tools
Sponsored By

A cloud with the international "no" symbol over itIn 2007 I posted about my Family's Backup Strategy and encouraged you to develop your own, or confirm the one you already have. At the time, my strategy was primary using Mozy.com as an online backup along with a Windows Home Server for local backup. After Windows Home Server recently removed Drive Extender technology (their version of RAID support in case a disk fails) I switched over to a Synology 1511+ and did a podcast on it.

Here's the summary if you feel the rest of this post is TL;DR and a ramble, which is likely because it is.

  • Use an imaging tool like Acronis True Image to create images of the machines you REALLY care about.
    • You can also use the System Image tool built into Windows, but I prefer the additional options from Acronis.
  • Encrypt your external drives.
  • Backup files (and disk images) to two external drives
  • Keep one external drive off-site
  • Test your backups by practicing a restore. The rule of thumb is that backups ALWAYS succeed. It's restores that fail!

I've been experimenting with cloud-based backup trying nearly everything out there from KeepVault to CrashPlan. I'm currently "between online backup strategies" right now, although I'm leaning towards CrashPlan. Most of these online backup companies are pretty confusing when you factor in someone with a server. For example, I have a main computer in my house but I also have a server. I have a KitchenPC and my wife's laptop but both of those don't matter as all the data is on server. However, I really want disk images for my main machine.

I need two things:

  • Continuous reliable image backup
  • My server's files backed up

The whole point here being if a natural disaster happens no family videos or photos will be lost.  For folks that keep all their files on their local PCs and not a server, any of these online services is great. However, as soon as I centralized my files, things got more complex. KeepVault is the best for Windows Home Server users as it has a custom Home Server client that integrates to the WHS control panel directly.

Now that I've moved over to a Linux-solution like Synology, other than using straight rsync or rsync to S3, the best solution appears to be CrashPlan Headless. While it's not directly supported on the Synology, there are LOTS of people who want to make this work so I wouldn't be totally alone, although the instructions are daunting and insane to say the least.

At this point, today, I've got a 4TB SAN with about 1.5TB on it and no cloud storage. And honestly, I wasn't looking forward to waiting two weeks (or longer) to upload 1.5TB to a new service, and I'm not sure what my ISP would say about it. I think that initial seed of large datasets is the Achilles Heel of online backup.

I really don't feel comfortable with my backups unless they are offsite. So I went and bought two Western Digital My Passport Essential SE 1 TB USB 3.0 Drives and labeled them Backup A and Backup B.

I recommend that you encrypt the whole disk. The last thing you want when you're copying your entire life onto one drive is to do it in the great wide open. There's two good ways to do it, BitLocker To Go or TrueCrypt

  • BitLocker To Go is in Windows 7 Enterprise or Ultimate. It's trivially easy to use and you CAN read BitLocker To Go'ed disks on any Windows7 Edition, or even on Windows XP or Windows Vista with BitLocker To Go Reader. It supports TPMs if your laptop has that feature and it's super secure.
  • TrueCrypt is open source and super hardcore. There's dozens of really amazing options for things like plausible deniability, honey pot secret partitions, and many choices of encryption. It's also crazy secure.

You really can't go wrong with either of these choices. For testing, I'm trying each of them, one on each drive. I like TrueCrypt, though because it's open source, but I like that BitLocker is built-in to Windows. We'll see. Point is, don't put your life on a disk unencrypted.

NOTE: Both of these encryption tools take forever (hours) to encrypt the whole disk. Be patient. It'll be a while.

I'm using Acronis  Image Home 2011 PC Backup and Recovery for my imaging solution. There are a lot of negative reviews of the 2011 version on Amazon but I haven't had any issues. Be aware. I'm always open to try new products if you have any recommendations, Dear Reader.

I use SyncBack to copy files on a schedule through my main machine from my server to the external drive. I could connect the drives directly to the Synology but I want to use my Windows machine for encryption.

My wife takes Drive A to the bank's safety deposit box in her monthly visit, then we just swap drives one a month with new backups. I'm not sure if I'll eventually get around to installing CrashPlan on the Synology (I hope it gets easier) but my current offsite "no cloud" backup strategy is working very nicely and it doesn't cost any bandwidth. In fact, I can transfer 1.5TB in just 10 minutes (of driving)!

* Cloud icon from The Noun Project, CC BY 3.0

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook bluesky subscribe
About   Newsletter
Hosting By
Hosted on Linux using .NET in an Azure App Service

Analyze your Web Server Data and be empowered with LogParser and Log Parser Lizard GUI

September 04, 2011 Comment on this post [8] Posted in IIS | NuGet | Tools
Sponsored By

The Log Parser Architecture Diagram showing all the inputs and outputs. There are a lot of choices on both sides.I've been using LogParser whenever I need to really dig into Web Server Logs since before 2005. It's an amazing tool. I love it. Jeff Atwood loves it, and you should to. It may not being something you use every day but when you need it, it's there and it's awesome. It's kind of like a really focused sed or awk. A low-level tool with a high-powered focus.

Log Parser hasn't changed that I know of since 2005. I've been working with some folks to try to get it to escape the big house, but we'll see how far we get. Until then, it works fabulously and unchanged after all these years. It's great because while my primary use of LogParser is with IIS Log files, it'll query anything you can plug into it like the File System, Event Logs, the Registry or just a CSV file. The diagram from their docs is at right.

I did a blog post 6 years ago before FeedBurner where I analyzed traffic to my RSS feed from Newsgator.  NewsGator was an RSS reader that would include statistics and information in its User-Agent HTTP Header. I was reminded of this post when I was talking to the NuGet team about how they are releasing new versions of NuGet every month or so but it's not clear how many people are upgrading. It'd also be interesting to find out what other ways folks are hitting the NuGet feed and what they are using to do it. I volunteered, so David Ebbo sent me a day's log file to "figure out."

Log Parser is wonderful because it effectively lets you run SQL queries against text files. Here's a few choice examples from Atwood's post a few years back:

Top 10 Slowest Items

SELECT TOP 10 cs-uri-stem AS Url, MIN(time-taken) as [Min], 
AVG(time-taken) AS [Avg], max(time-taken) AS [Max],
count(time-taken) AS Hits
FROM ex*.log
WHERE time-taken < 120000
GROUP BY Url
ORDER BY [Avg] DESC

HTTP Errors Per Hour

SELECT date, QUANTIZE(time, 3600) AS Hour, 
sc-status AS Status, COUNT(*) AS Errors
FROM ex*.log
WHERE (sc-status >= 400)
GROUP BY date, hour, sc-status
HAVING (Errors > 25)
ORDER BY Errors DESC

Given queries like these, I figured that LogParser would be perfect for me to explore the NuGet web service logs. (Of course, I realize that the service itself could be instrumented, but this is more flexible, and I plan to make these queries run on a schedule and show up on http://stats.nuget.org.)

There are a number of ways to access a NuGet packaging server. You can use the Add Package Dialog, the Command Line, the PowerShell Console within Visual Studio, or the NuGet Package Explorer. There's also some testing data and some "no user agent" stuff in there also. I filtered that out by just charting "NuGet" clients.

I started doing the initial work from the command line, but it was slow going. I was having trouble visualizing what I wanted and what was being returned. Here is one of my first command lines. It was pretty hairy and hard to build this at the command line.

C:\u_ex110831>LogParser.exe -i:IISW3C "SELECT DISTINCT cs(User-Agent) AS Client, 
count(1) AS NumberOfHits
FROM u_ex110831.log
WHERE Client
LIKE 'NuGet%'
GROUP BY Client
ORDER by count(1) DESC"

Client NumberOfHits
------------------------------------------------------------------------------------------- ------------
NuGet+Add+Package+Dialog/1.4.20701.9038+(Microsoft+Windows+NT+6.1.7601+Service+Pack+1) 38840
NuGet+Command+Line/1.5.20830.9001+(Microsoft+Windows+NT+6.1.7601+Service+Pack+1) 15591
NuGet+Add+Package+Dialog/1.4.20701.9038+(Microsoft+Windows+NT+6.1.7600.0) 13360
NuGet+Command+Line/1.4.20615.182+(Microsoft+Windows+NT+6.1.7600.0) 8562
NuGet+Add+Package+Dialog/1.4.20607.9007+(Microsoft+Windows+NT+6.1.7601+Service+Pack+1) 5531
NuGet+Package+Manager+Console/1.4.20701.9038+(Microsoft+Windows+NT+6.1.7601+Service+Pack+1) 5497
NuGet+Command+Line/1.4.20615.182+(Microsoft+Windows+NT+6.1.7601+Service+Pack+1) 3699
NuGet+Package+Manager+Console/1.4.20701.9038+(Microsoft+Windows+NT+6.1.7600.0) 3654
NuGet+Add+Package+Dialog/1.4.20701.9038+(Microsoft+Windows+NT+5.1.2600+Service+Pack+3) 3558
NuGet+Command+Line/1.4.20615.182+(Microsoft+Windows+NT+5.2.3790+Service+Pack+2) 2539
Press a key...

There were 40 unique User Agents in this file and they include the client and its version as well as the operating system. I wanted first to chop it up to find out what Types of NuGet clients were being used. I have broken the lines up to make it clearer in this snippet.

C:\u_ex110831>LogParser.exe 
-i:IISW3C "SELECT DISTINCT SUBSTR(cs(User-Agent),0, index_of(cs(User-Agent),'/')) AS Client,
count(1) AS NumberOfHits FROM u_ex110831.log
WHERE Client LIKE 'NuGet%'
GROUP BY Client
ORDER by count(1) DESC"

Client NumberOfHits
----------------------------- ------------
NuGet+Add+Package+Dialog 74761
NuGet+Command+Line 32284
NuGet+Package+Manager+Console 12637
NuGet+Package+Explorer 943
NuGet+Visual+Studio+Extension 49

Statistics:
-----------
Elements processed: 208235
Elements output: 5
Execution time: 0.79 seconds

Pretty amazing, though. A sub-second query over almost a quarter million line long log file with useful results and no database. Reminds me of working on Unix 20 years ago.

After some experimenting and installing the Office Web Components 2003 (discontinued) and was outputting a chart with this MONSTER command line:

C:\u_ex110831>LogParser.exe -i:IISW3C -o:CHART -chartType:PieExploded 
-categories:Off -values:On -view:on
-chartTitle:"NuGet Clients by User Agent"
"SELECT DISTINCT SUBSTR(cs(User-Agent),0,index_of(cs(User-Agent),'/')) AS Client,
count(1) AS NumberOfHits
INTO foo.png
FROM u_ex110831.log
WHERE Client
LIKE 'NuGet%'
GROUP BY Client
ORDER by count(1) DESC"

Which yields me this profoundly 2003-looking chart, but still allows me to cheer a tiny victory inside. I will be able to get this (or a prettier one) to run as on a schedule (AT or Chron job) and serve it to the interwebs. I t'll probably be better to output a CSV or XML file, then process that with the web server and create a proper interactive chart. Regardless, tiny cheer.

foo

Still, I'm thinking I'm too old for this crap. Where's my GUI? What's a brother got to do to drag a DataGrid around here? A little Binging with DuckDuckGo (yes, I'm trying DDG this month) and I find - wait for it - LogParser Lizard GUI.

LogParser Lizard GUI

What's this? Oh YES. It's intellisense and tooltips, baby!

Log Parser Lizard GUI Main Screen

I can't say how much faster this tool made me once I had figured out LogParser. It's funny how you have to suffer at the command line before you can really appreciate a good GUI. At this point I called Jon Galloway for some pair-SQLing and we pounded out a few more queries.

NuGet by Version

I filtered out NuGet Package Explorer because it has its own version scheme. However, I'm not sure about this query, as I wanted to get the Major.Minor versions. I noticed that by coincidence the third value z (of x.y.z) always started with .2 so I cheated with the SUB() below because I couldn't figure out how to just filter out the x.y values. Any thoughts are appreciated.

SELECT DISTINCT SUBSTR( cs(User-Agent), 
ADD(index_of(cs(User-Agent),'/'),1),
SUB(index_of(cs(User-Agent),'.2'),STRLEN(cs(User-Agent))))
AS Client, count(1) AS NumberOfHits
FROM u_ex110831.log
WHERE cs(User-Agent) NOT LIKE '%Explorer%' AND cs(User-Agent) LIKE '%NuGet%'
GROUP BY Client
ORDER by count(1) DESC
Client Hits
------ ----- 1.4 98097
1.5 18985
1.3 2524
1.6 69

So then I did the whole version:

SELECT SUBSTR( cs(User-Agent), 
ADD(index_of(cs(User-Agent),'/'),1),
SUB(index_of(cs(User-Agent),'+('),STRLEN(cs(User-Agent))))
AS Client, count(1) AS NumberOfHits
FROM u_ex110831.log
WHERE cs(User-Agent) NOT LIKE '%Explorer%' AND cs(User-Agent) LIKE '%NuGet%'
GROUP BY Client, cs(User-Agent)
ORDER by count(1) DESC

Client Hits
--------------- ------
1.4.20701.9038 38840
1.5.20830.9001 15591
1.4.20701.9038 13360
1.4.20615.182 8562
1.4.20607.9007 5531
1.4.20701.9037 5497
1.4.20615.182 3699
1.4.20701.9038 3654

I was extremely impressed with how quickly (about an hour) was able to get really substantive, interesting and targeted data out of these log files. The next step will be to get all the logs and run the command line tool create month over month line charts. The goal will be to figure out how many folks are successfully upgrading their NuGet installations as well as how they are using it. Are they using the right-click menu or are they using the console?

If you've got an application that makes HTTP calls to a service that you own, whether your application is a phone or a custom client, while you can certainly instrument your code on the server side to collect stats, there's a LOT of information in your IIS logs. You can use LogParser Lizard GUI to develop your queries and then schedule runs of the command line tool to generate reports that will really help you improve your product. This technique isn't as sophisticated as custom 3rd party analytics package but you can certainly get a surprising amount of information in a short amount of time with LogParser.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook bluesky subscribe
About   Newsletter
Hosting By
Hosted on Linux using .NET in an Azure App Service

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.