Scott Hanselman

A/B Testing and Testing In Production with Azure Web Apps

July 17, '15 Comments [6] Posted in Azure
Sponsored By

I've got a lot of production web sites running in Azure right now. Some are for small side projects and some are larger like the sites for the Hanselminutes Podcast and This Developer's Life. I like Web Apps/Sites (which is Platform as a Service) rather than Virtual Machines (Infrastructure as a Service) because I don't like thinking about the underlying operating system if I can avoid it. I like to be able to scale the site up (faster, bigger) or out (more machines in the farm) with a slider bar.

In fact, there's some other more advanced and useful features that Azure Web Apps have that keep me using Web Apps almost exclusively.

I'll use a little site I made called KeysLeft.com that tells you how many keystrokes are left in your hands before you die. Think of it as a productivity awareness tool.

First, I'll add a Deployment Slot to my existing Git-deployed Web App. The source for KeysLeft lives in GitHub here. When I check-in a change it's automatically deployed. But what if I wanted to have a staging branch and automatically deploy to a staging.keysleft.com first? If it works out, then move it to production by swapping sites. That'd be sweet.

Staging Slots for Azure Web Apps

You can see here my main KeysLeft web app has a Staging "side car" app that is totally separate but logically related/adjacent to production. Notice the "swap" button in the toolbar. Love it.

Adding Deployment Slots to an Azure Web App

This Web App has its configuration copied from the main one, and I can setup Continuous Deployment to pull from a different branch, like "staging" for example. The name of the deployment slot becomes a suffix, so keysleft-staging.azurewebsites.net unless you set up a custom CNAME like staging.keysleft.com. You can have up to 4 deployment slots in addition to production (so dev, test, staging, whatever, production) on Standard Web Apps.

A/B Testing for Azure Web Apps

Once I've got a slot or two set up and running a version of my app, I can do A/B testing if I'd like. I can set up a feature that was called "Testing in Production" and is now "Traffic Routing" and tell Azure what percentage of traffic goes to prod and what goes to staging. Of course, you have to be sure to write your application so such that authentication and session is managed however is appropriate, especially if you'd like the user to have a seamless experience.

Here I've got 10% of the traffic going to staging, seamlessly, and the other 90% is going to production. I can make a small change (background color for example) and then hit the main site over and over and see the occasional (10% of course) request being routed to the staging slot. You can configure this static routing however you'd like.

10% Traffic to Staging

Then I could hook up Application Insights or New Relic or some other event/diagnostics system and measure the difference in user reaction between features that changed.

Advanced Testing in Production

Made it this far? Then you're in for a treat. Static routing is cool, to be clear, but scripting a more dynamic experience is even more interesting. Galin Iliev, one of the developers of this feature, gave me this Powershell script to show off more powerful stuff.

First, you can use PowerShell to manage this stuff. You can change routing values and ramp up or ramp down. For example, here we start at 10% and change it by 5 every 10 minutes.

# Select-AzureSubscription YOURSGOESHERE

$siteName = "keysleft"
$rule1 = New-Object Microsoft.WindowsAzure.Commands.Utilities.Websites.Services.WebEntities.RampUpRule
$rule1.ActionHostName = "keysleft-staging.azurewebsites.net"
$rule1.ReroutePercentage = 10;
$rule1.Name = "staging"

$rule1.ChangeIntervalInMinutes = 10;
$rule1.ChangeStep = 5;
$rule1.MinReroutePercentage = 1;
$rule1.MaxReroutePercentage = 80;

Set-AzureWebsite $siteName -Slot Production -RoutingRules $rule1

But! What if you could write code to actually make the decision to continue or fall back dynamically? You can add a callback URL and a Site Extension called the "TiP Callback Extension."

$rule1.ChangeDecisionCallbackUrl = https://keysleft.scm.azurewebsites.net/TipCallback/api/routing

The Site Extension (and all Site Extensions for that matter) is just a little sidecar Web API. This callback gets a small POST when it's time to make a decision, and you decide what to do based on HTTP-related context that was passed in and then return a ChangeDirectionResult object as JSON. You can adjust traffic dynamically, you can adjust traffic when doing a deployment, do a slow, measured roll out, or back off if you detect issues.

NOTE: The ChangeDescisionCallbackUrl and this code below is totally optional (so don't stress) but it's super powerful. You can just do static routing, you can do basic scripted dynamic traffic routing, or you can have make a decision callback URL. So the choice is yours.

You can check out the code by visiting yoursite.scm.azurewebsites.net after installing the TiP callback site extension and look at the Site Extensions folder. That said, here is the general idea.

using System.Web.Http;
using TipCallback.Models;

namespace TipCallback.Controllers
{
public class RoutingController : ApiController
{
[HttpPost]
public ChangeDirectionResult GetRoutingDirection([FromBody] RerouteChangeRequest metrics)
{
// Use either Step or RoutingPercentage. If both returned RoutingPercentage takes precedence
return new ChangeDirectionResult
{
Step = (int)metrics.Metrics["self"].Requests,
RoutingPercentage = 10
};
}
}
}

Here's the object you return. It's just a class with two ints, but this is super-annotated.

/// <summary>
/// Return information how to change TiP ramp up percentage.
/// Use either Step or RoutingPercentage. If both returned RoutingPercentage takes precedence
/// Either way MinRoutingPercentage and MaxRoutingPercentage set in API rule are in force
/// </summary>
[DataContract]
public class ChangeDirectionResult
{
/// <summary>
/// Step to change the Routing percentage. Positive number will increase it routing.
/// Negative will decrease it.
/// </summary>
[DataMember(Name = "step")]
public int? Step { get; set; }

/// <summary>
/// Hard routing percentage to set regardless of step.
/// </summary>
[DataMember(Name = "routingPercentage")]
public int? RoutingPercentage { get; set; }
}

All this stuff is included in Standard Azure Web Apps so if you're using Standard apps (I have 19 websites running in my one Standard plan) then you already have this feature and it's included in the price. Pretty cool.

Related Links


Sponsor: Big thanks to Infragistics for sponsoring the feed this week. Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid - Download for free now!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Stop, think, research, debug

July 14, '15 Comments [19] Posted in Musings
Sponsored By

I got this great letter from a listener/reader recently. They listened to a recent show on Debugging Windows and it helped them debug a problem at work, but not in a specific technical way. Instead, it changed how they thought about their approach to the topic.

By the way, I've had some amazing guests on lately. If you haven't subscribed, it's a good time to join us. Explore the archives and check our our diverse topics and voices and subscribe. Also, BTW there is a new This Developer's Life out so check that out also.

I've been doing Hanselminutes: Fresh Air for Developers for almost 500 episodes over darn-near 10 years. Getting emails like this is so meaningful, especially when I think about taking breaks or stopping. Sometimes a few shows will go by with no comments and I'll wonder if anyone listens, and then I hear from a Real Live Human who was helped by all this free content I put out and it keeps me going. So first, thanks to all of you for this, if you've ever emailed or donated to fight diabetes.

US Navy Cryptanalytic Bombe by brewbooks, used under CC

Here's what this particular listener said, with emphasis mine.

Scott,
After listening to your podcast with Mario Hewardt earlier this week on Windows Debugging, I had some of the things you were talking about running through my head. Though I've always come away from your podcasts feeling enriched and excited to tackle new and interesting problems, this was the first time that it had a direct impact on my work so soon after listening.

I work at a big data company that does a lot of social network analysis. We use ElasticSearch in our stack, and we are consistently processing millions of documents using complicated, user generated queries. A release we put out late last week allowed for many, larger, even more complicated user queries, which in turn led to substantial slowdown of our product. Though the code only existed in our staging environment, we are on a deadline for release early this next week. As it became obvious that the application was spending a LOT of time in the code my team was responsible for, we were tasked with "fixing" it ASAP.

I took the first shift, and though my brain immediately started coming up with ways to improve our code, something about your podcast regarding "know the tools your tools are built on" was stuck in my head. Instead of jumping in and optimizing what I was already comfortable with, I spent an hour researching the internals of the ElasticSearch functionality we were relying on.

Not sure how familiar you are with ES, but it distinguishes between searches that simply return a set of documents that match a query, much the way that traditional SQL databases do, and searches that return how well documents match a query, for ranking purposes. As it turned out, we were inadvertently using one of the latter ones, meaning when we provided X giant queries in an OR block, even though it was an OR block, which we expected would short circuit as soon as it returned a TRUE condition, it processed all X queries to determine how well each document matched. My big O notation is a bit rusty, but suffice it to say, it was one of the bad ones.

Instead of a gigantic fire drill app optimization over a weekend, it turned out to be an hour of research followed by switching the word "bool" to the word "or". It's remarkable how the most efficient coding you can do is often stopping and thinking about the problem for awhile!

Anyway, thanks to both you and Mario for saving me and my team a bunch of time!

This was a great reminder to me as well. Research is hard. It's not as dynamic as interactive debugging but it can often save you many wasted hours. Truly successful debugging means doing whatever it takes to understand the problem domain and the code paths.

Do you have any tales of debugging where taking the time to really understand the problem domain saved you time? Or perhaps the opposite, where you just dove in and poked at some code until it worked? Don't be ashamed, I think we've all be on both sides.

Sound off in the comments!


Sponsor: Big thanks to Infragistics for sponsoring the feed this week. Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls.  Get super-charged performance with the world’s fastest HTML5 Grid - Download for free now!

* Photo - US Navy Cryptanalytic Bombe by brewbooks, used under CC

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Introducing Web Tiles for Microsoft Band - My diabetes data on a Band!

July 9, '15 Comments [27] Posted in Web Services
Sponsored By

Microsoft Bands in various colorsI love wearables. Check out my blog post from 11 years ago on my "Wrist.NET" Microsoft SPOT watch. This was a time before ubiquitous connectivity and it was an amazing device that provided tons of glanceable information.

Fast-forward to today and I've used a Fitbit, an Apple Watch, a few Pebble Watches, and a Microsoft Band. The thing that I wanted in 2004 - and the thing I want today - has always been an easy way to make an application for my wearable device. When the Microsoft Band (If you get one, print out this Sizing PDF first and measure your wrist) came out I immediately wanted to know what the SDK looked like! How easily could I make a new Tile on my Band?

Well, while the Band SDK is super powerful, just like the Apple Watch and most wearables, if I wanted to make a Band Tile I needed to make a mobile app first! That was a bummer for me. If I want to make a new simple Tile and share it with my friends I first need to make an app, and to have full coverage, I'll need three versions of the app (iPhone, Windows Phone, Android) as well?

The Band isn't a watch, and it's not just a pedometer. The Microsoft Band has ten sensors: an optical heart rate sensor, 3-axis accelerometer, gyrometer, GPS, light sensor, skin temp sensor, UV sensor, capacitive sensor, a microphone and one for skin response. I wanted a way to easily connect it to anything else I could think of. Lights, my glucose meter, my Nest, and on and on.

I went to the Band team and started pitching them an idea I called Web Tiles. Since every Band user already has the Microsoft Band (Health) app, why not let the existing app be a bridge and it would own new custom tiles! Web Tiles for glanceable data with a low barrier to entry, and the full Band SDK for rich interactivity. I figured we could write new Tiles with web tech. My personal use case was that I wanted a Web Tile to show my blood sugar from Nightscout, an open source app I use to manage my diabetes. Fortunately the Band Team were like-minded and we collaborated. Eventually they really started running and Web Tiles was born. You may have noticed that we gently introduced Web Tiles at the BUILD conference using my sugar data in the demo.

Web Tiles for Microsoft Band

Today the first preview version of Web Tiles is ready to go. You can make a custom Web Tile in just minutes for your Microsoft Band and install it now. You can put it on your OneDrive or blog, or even just email it to a friend.

If you're slightly technical, you can create Web Tiles with just the documentation, Notepad (or the VS Code editor) and a Zip utility. For the rest of us, you can use the online Web Tile Authoring Tool and it will generate the tile and give it to you for download.

Web Tiles are glanceable tiles that are feed by JSON, XML, or ATOM datasources. If you want to make one, feel free to use my Blood Sugar JSON datasource: http://hanselmanbanddata.azurewebsites.net and the Web Tile Authoring Tool.

The Web Tile Authoring Tool

Web Tile Authoring Tool

Otherwise, here's a little more detail. Be sure to check out the Band team's blog posts and web site!

More Technical Stuff

There is a new runtime inside the Microsoft Health app for iOS, Windows Phone, and Android to manage Web Tiles and keep them fresh. Web Tiles are a zip file with a manifest with image files and JSON inside. You can put Web Tiles anywhere on the web or in email attachments. They have a .webtile extension, but you can use the mshealth-webtile:// custom URL scheme to launch the app and download a webtile, like mshealth-webtile://?action=download-manifest&url=http://www.microsoft.com/mywebtile.webtile

A minimal Web Tile would look like this:

  • mytile.webtile (it's a renamed zip, and paths matter!)
    • /manifest.json // Contains web tile definition and references to other assets
    • /icons/*.png // PNG icons used in the web tile

Tiles can have multiple pages, in a master/detail style, binding to the data however you'd like.

image

Small Nightscout LogoTo make a Web Tile that shows my blood sugar from my Nightscout site, I created this 46x46 PNG of the Nightscout logo and pulled from the JSON feed that represents my own glucose values http://hanselmanbanddata.azurewebsites.net.

The JSON for my Diabetes Web Tile is here, as an example. You'll also find it in the How-To documentation for Web Tiles. The first part is obvious, just a manifest. Then the Tile Icon. I just have one. Then we have a single Tile with a Simple style and three lines. The format you see there "bgs[0].sgv" is a way of pulling from the JSON. Like foo.bar.baz[0] if the JSON were nested named objects. The resources are named, and then later bound in strings within pages.

You could create a Web Tile for anything you have that has a JSON endpoint. I'm going to make a Web Tile to monitor my 3D Printer using Octoprint's REST API for example.

{
"manifestVersion": 1,
"name": "Nightscout",
"description": "Nightscout Blood Sugar",
"version": 1,
"versionString": "1",
"author": "Scott Hanselman",
"organization": "Nightscout",
"contactEmail": "",
"tileIcon": {
"46": "icons/tileIcon.png"
},
"refreshIntervalMinutes": 15,
"resources": [
{
"url": "http://hanselmanbanddata.azurewebsites.net/api/Band",
"style": "Simple",
"content": {
"bgs0sgv": "bgs[0].sgv",
"bgs0bgdelta": "bgs[0].bgdelta",
"bgs0direction": "bgs[0].direction"
}
}
],
"pages": [
{
"layout": "MSBand_NoScrollingText",
"condition": "true",
"textBindings": [
{
"elementId": "1",
"value": "Sugar: {{bgs0sgv}}"
},
{
"elementId": "2",
"value": "Delta: {{bgs0bgdelta}}"
},
{
"elementId": "3",
"value": "Trend: {{bgs0direction}}"
}
]
}
]
}

I mail the Web Tile to myself and see this on my iPhone. (Again, it could be in Dropbox, OneDrive, etc)

Emailing a WebTile to myself

Now I "Open in Microsoft Health..."

Adding a Web Tile to my Band

Click Save...

Now I have 2 web tiles in my band

And I've got two new custom Web Tiles now!

IMG_2465

And here's my Band with my Web Tile installed! (Yes, at this moment in time my sugar is a little high, but I'm on it.)

In the future I'd like to see events, buttons, triggers, push notifications, inline images, charts/sparklines, and more. What do you want to use Web Ties for? Is this cool?

I'm sure the team is interested in the direction you'd like to see Web Tiles go. Interactions? Events? Real-time? More sensor support? Authentication? Sound off in the comments, vote on the Microsoft Band and Health UserVoice page and absolutely email them directly at healthms@microsoft.com.

DONATE: If you appreciate this blog and what I'm doing here, please donate to fight diabetes. Read about my story at http://hanselman.com/fightdiabetes, watch my Diabetes YouTube video, and make a tax-deductible donation here http://hanselman.com/fightdiabetes/donate

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

VS Refactoring Essentials (formerly) NR6Pack - Free analyzers and refactoring for Visual Studio 2015

July 8, '15 Comments [24] Posted in Open Source | VS2015
Sponsored By

There are some amazing free and open source C# and VB.NET analyzers and refactorings that you can download and use now in Visual Studio 2015. Formerly called "NR6Pack" they have now be renamed VS Refactoring Essentials, a nice nod to VSWebEssentials I must say.

The best part, of course, is that since the Visual Studio Community Edition is not only FREE but also now supports extensions, that any open source or indie developer can get pack some serious power into their Visual Studio installations.

There's some interesting history here as the project and its core technology has been around for a while. It's moved from SharpDevelop into NRefactory, then NRPack, and now with the major Roslyn refactor by Mike Krüger, you've got a nice Roslyn code-base and a free extension for all called VS Refactoring Essentials.

There's lots of great refactorings, too many to include screenshots for all of them, but here's a few favorites.

Sometimes you'll dig deep into a dictionary without being defensive. VS Refactoring Essentials will notice and check the dictionary key first. Note that you'll always get a preview of what it's going to change first!

CheckDictionaryKeyValueCodeRefactoring

Conditionals can sometimes get away from you. VS Refactoring Essentials will simplify common conditionals and make them easier to read.

SimplifyConditionalTernaryExpressionAnalyzer

When setting boolean flags you'll sometimes set it, check something, and update that same flag. This refactoring will notice that and do it all on one line for you.

ConvertIfToOrExpressionAnalyzer

I never get ?: and ?? correct. VS Refactoring Essentials will help you move between ternary operators and null coalescing.

ConvertConditionalTernaryToNullCoalescingAnalyzer

FYI - This is a purely open source project that is not affiliated with Microsoft. It's part of the SharpDevelop OSS project and is MIT-licensed. Big thanks to the SharpDevelop team!

NOTE: Visual Studio 2015 will launch on July 20th. Be sure to download Visual Studio 2015 Community on that date. Until then, the V2015RC of Community is here.

Be sure to follow @VSRefactoring on twitter and thanks for them for their hard work and community focus! Go download Refactoring Essentials here for VS2015. Report issues on their GitHub.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Windows 10 is coming...here's what to tell non-technical parent

July 1, '15 Comments [23] Posted in Screencasts | Win10
Sponsored By

Windows 10 is coming on July 29th! I've been doing Build to Build videos on my YouTube showing what's changing and how it will affect you.

I got a request to do a video showing Windows 7, Windows 8, and Windows 10 and how to reserve your upgrade, as well as a little demo of the Start Screen. Here's that video!

NOTE: I apologize for the mediocre audio. I had a microphone failure and ended up using the laptop microphone for the last part because I was excited to get the video out. It's not representative of the quality I'm known for, and it won't happen again.

Are you the IT manager for your extended family? Will you be upgrading non-technical parent to Windows 10 or letting them do it themselves? Sound off in the comments below.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.