Scott Hanselman

A buddy of mine was thinking to move some of his Linux-based website to Azure. If you're running a Web Site that this node.js, ASP, ASP.NET, or PHP, it's easiest to use Azure Web Sites. I showed how to do setup Azure Web Sites in minutes with Git in this post. They hide the underlying OS from you, are automatically updated, scale easily, and share disks.

However, he likes VMs and the control they give him, plus he can run whatever he wants, move things around and generally control his world.

I'll be using the open source (GitHub) cross platform CLI tools (command line interface) for Azure. If you have node package manager you can "npm install azure-cli --global" then import your subscription. You can also get the command line tools by downloading and installing from www.windowsazure.com.

Setting up a Linux VM

We'll create a the initial VM using the Portal (I'll show you how to do it from the command line in a minute). This virtual machine will be for setting up a template VM image. I'm going to create an Ubuntu 13.04 server, then add Apache and PHP. Then I'll capture a generic image of my now-set-up machine and use it to create copies that I'll add to a farm. This image will show up later in "My Images" in the Azure Portal.

From within the Azure Portal I'll go New | Virtual Machine | and select Ubuntu Server 13.04.

The name doesn't matter, but I'll setup a user name and password (or use a SSH key):

Note I'll create a stand-alone Virtual Machine:

TIP: If I didn't want to use the Portal at all to make this VM, I could even find a VM image programmatically, then create a VM instance, all from the command line. Using "azure vm list" would get me the list of VMs to choose from. The Ubuntu one I wanted is "Ubuntu-13_04-amd64-server-20130501-en-us-30GB" (with a guid in the name as well) so I'd just "azure vm create MyDNSName ImageName [options]" and then proceed from there.

Once this Linux VM has started up, I'll SSH into it. You can see that Azure has mapped a random high number public port to the VM's internal SSH port 22.

I SSH in. I'm gonna add Apache, PHP, restart apache, then add a test.php that will show PHP is working as well as output the current IP address so I can tell which machine served the request.

sudo apt-get install apache2
sudo apt-get install libapache2-mod-php5 php5

I'll add a test.php

sudo nano /var/www/test.php

and put in

<?php echo gethostbyname(trim(`hostname`)); ?>
<?php phpinfo(); ?>

Then I'll exit SSH. VMs are locked down by default, so to test this I need to add an endpoint. I can do that via the Portal but I'd like to see what I can do from the Azure command line.

Run vm endpoint create to map external 80 to internal 80.

azure vm endpoint create mylinuxtemplate 80 80

At this point I should be able to hit mylinuxtemplate.cloudapp.net:80/test.php and see it work.

Cool. So I've got my Linux VM template the way I want it. Now I want to "capture it" as an image so I can stamp out more of them. This "waagent" on Linux is like "sysprep" on Windows.

From within a SSH session, run waagent -deprovision.

~$ sudo waagent -deprovision
scott@mylinuxtemplate:~$ sudo waagent -deprovision
WARNING! The waagent service will be stopped.
WARNING! All SSH host key pairs will be deleted.
WARNING! Nameserver configuration in /etc/resolvconf/resolv.conf.d/{tail,originial} will be deleted.
WARNING! Cached DHCP leases will be deleted.
WARNING! root password will be disabled. You will not be able to login as root.
Do you want to proceed (y/n)? y

WINDOWS PEOPLE: If you're a Windows person, you can setup your Windows VM just as you like it, then run %windir%\system32\sysprep.exe on it, capture an image of the VM and do everything described in this post as well!

I could shut it down and capture an image from the Portal, but again, it's command line fun today. Note that shutting down can take a little while.

azure vm shutdown mylinuxtemplate
azure vm capture mylinuxtemplate hanselmanlinuxwebfarmimage --delete

The capture command will DELETE the Virtual Machine. Remember that it was just a template. However, how I have a reusable image! I can see the images available to me, both user images and gallery images with "azure vm list."

NOTE: When you delete Virtual Machines, you're just deleting the "configuration" of the VM. You're not deleting the disk that was associated with it, as it's possible you might want to start that VM up again. If you're really trying to remove things, make sure you delete the instance of the VM and then delete the disk, too.

Creating a Linux VM Farm from the command line

Now I have an image sitting in my storage account that I can use to make "n" VMs from. I'll make one VM to start. I can watch it startup with "azure site list" after making it. When it's ready, I can make more! Make sure you use the --ssh switch or you will NOT be able to SSH into the machine!

C:\> azure vm create hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott MyPassword123 --location "West US" --ssh
info:    Executing command vm create
+ Looking up image
+ Looking up cloud service
+ Creating cloud service
+ Creating VM
info:    vm create command OK
C:\> azure vm list
info:    Executing command vm list
+ Fetching VMs
data:    DNS Name                         VM Name             Status
data:    -------------------------------  ------------------  ----------
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm  CreatingVM
info:    vm list command OK
C:\> azure vm list
info:    Executing command vm list
+ Fetching VMs
data:    DNS Name                         VM Name             Status
data:    -------------------------------  ------------------  ---------
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm  ReadyRole
info:    vm list command OK

WEIRD: Azure has a concept called a "Cloud Service" which is a lousy name, IMHO. For us, let's consider it a "container" for our VMs. It's a logical container that will hold and associate all our VMs (and other cloudy stuff) together. When you have one VM you have one cloud service associated with it, but you can't see it in the Portal but because it doesn't really provide value...yet. When you have TWO VMs in the same container, then you'll see that cloud service "container" appear in the Portal.

I've made a hanselmanlinuxfarm VM now so there's a hanselmanlinuxfarm cloud service 'container.' Now, I'm going to make a few more VMs but I'll connect them to the first VM. There's two ways to do this. First, the --connect option from the command line. Note that you don't have control over the name of your VM this way, if you care. If you have hundreds, you likely don't.

The command will find the existing cloud service (again, 'container') then make a new VM. I'm going to run this command twice so I'll have three VMs total.

SO IMPORTANT: It's the --connect used on this second call that is the key. It makes the second (and then n+1) VM and adds it to the same cloud service "container." It seems the VMs associated with each other. The name of the next VM will be whatever-2, then -3, etc but they will also use the same external name, like hanselmanlinuxfarm.cloudapp.net.

I'll do this twice, each time using a different high SSH port number that will map to 22 internally. If I don't want SSH expose externally, I can delete the public endpoint later.

C:\> azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott MyPassword --ssh 12345
info:    Executing command vm create
+ Looking up image
+ Looking up cloud service
+ Getting cloud service properties
+ Looking up deployment
+ Creating VM
info:    vm create command OK

When creating a Linux VM you MUST add a --ssh flag to the command line or you'll not be able to SSH into the box. Make sure to add a high port number so you'll get a mapping from that port to 22, so maybe 12346 -> 22, etc. If you make these VMs from the Portal, it will pick a random port for you. When you do it from the command line, you need to choose.

At this point, azure vm list says I have three. Two are ready and the last is being created now. You can tell these VMs are running in the same Cloud Service "container" because the DNS name is the same. These VMs are officially a "farm."

C:\> azure vm list
info:    Executing command vm list
+ Fetching VMs
data:    DNS Name                         VM Name               Status
data:    -------------------------------  --------------------  ----------
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm    ReadyRole
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm-2  ReadyRole
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm-3  CreatingVM
info:    vm list command OK 

OK, now here's making a fourth VM from the Portal, just as an FYI.

Creating a Linux VM and adding to the Farm from the Portal

Just so you know, you can add VMs to your farm from the Portal also.

Give your new VM a name, then "connect it to an existing virtual machine." I don't like this phrasing, and I'm curious what you think. Basically it's "add this VM to this collection of VMs." It doesn't matter which one you select from this dropdown, as long as you pick one that's in the Cloud Service "container".

I won't click OK, but if I did, at this point I've would've a fourth VM, this one via the Portal.

Load Balancing my Linux VM Farm

I have three identical VMs running Apache and PHP and my test.php page.

C:\> azure vm list
info:    Executing command vm list
+ Fetching VMs
data:    DNS Name                         VM Name               Status
data:    -------------------------------  --------------------  ---------
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm    ReadyRole
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm-2  ReadyRole
data:    hanselmanlinuxfarm.cloudapp.net  hanselmanlinuxfarm-3  ReadyRole

info:    vm list command OK

Let me open up port 80 on all three.  Since I want them load balanced I can't do this, as this is how you map single non-load-balanced ports.

azure vm endpoint create hanselmanlinuxfarm   80 80
azure vm endpoint create hanselmanlinuxfarm-2 80 80
azure vm endpoint create hanselmanlinuxfarm-3 80 80

For load balanced ports I need to use "create-multiple." Not only may I want to open multiple ports all at once, but also since I want load-balancing, I also may want a probe set up. For example, since I'm using HTTP, if there's a result other than 200 returned from test.php then I'll want that VM pulled out of the farm. It will also pull the VM out of rotation if it doesn't hear back in 30 seconds.

Here I'm creating those 80 to 80 maps, but also watching test.php for anything other than an HTTP 200.

azure vm endpoint create-multiple hanselmanlinuxfarm   80:80:HttpTrafficIn:http:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-2 80:80:HttpTrafficIn:http:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-3 80:80:HttpTrafficIn:http:80:/test.php

In this case "HttpTrafficIn" is what I am calling the Load Balancing Set Name.

NOTE: I'm doing a pull request now to add the ability to see the ProbePath from the endpoint command but for now you can see it with "azure vm show" like this:

C:\>azure vm show hanselmanlinuxfarm
info:    Executing command vm show
+ Fetching VM
data:    DNSName "hanselmanlinuxfarm.cloudapp.net"
data:    VMName "hanselmanlinuxfarm"
data:    IPAddress "100.68.xx.xx"
data:    InstanceStatus "RoleStateUnknown"
data:    InstanceSize "Small"
data:    InstanceStateDetails ""
data:    OSVersion ""
data:    Image "hanselmanlinuxwebfarmimage"
data:    DataDisks ""
data:    Network Endpoints 0 LoadBalancedEndpointSetName "HttpTrafficIn"
data:    Network Endpoints 0 LocalPort "80"
data:    Network Endpoints 0 Name "endpname-80-80"
data:    Network Endpoints 0 Port "80"
data:    Network Endpoints 0 LoadBalancerProbe Path "/test.php"
data:    Network Endpoints 0 LoadBalancerProbe Port "80"
data:    Network Endpoints 0 LoadBalancerProbe Protocol "http"
data:    Network Endpoints 0 Protocol "tcp"
data:    Network Endpoints 0 Vip "137.135.xx.xx"
info:    vm show command OK

Cool, so now let's see if I have a load-balanced farm.

PERF NOTE: In order to get the best performance from your Azure VMs (or any cloud VM) considering putting things like MySQL/PostgreS databases on a separate disk with different caching semantics. You want the OS disk and the Data Disks to be separate. For example. I have a Windows VM running MySQL. The OS is on a standard 30 gig disk, but the MySQL Database is alone on a 5 gig disk that's attached. It keeps things separate and tidy, plus it performs better

Checking on my new Farm

If I log into the Portal, I can look at each individual VM or I can look at the farm as if it's one 'cloud service.' Get it?

Making my Farm more reliable

I want to make sure my new VMs are all on different racks in the Azure Data Center. I know they are in "West US" because I put them there, but I'm not sure if they are together on the same rack or near each other or what.  Since a rack is within a "fault domain" meaning that a Rack could, I dunno, spontaneously explode, then I want to tell Azure that all these VMs are part of an "availability set." This is a name I apply to the VMs that says "make these more available by keeping them apart."

From the Portal I'm going to pick the first VM and select Configure, then Create an Availability Set. I'll name it "hanselmanlinux" but it can be anything.

Adding an Availability Set can involve Azure moving my VM somewhere else within the Data Center and it may need to restart it if it does. Sometimes this is fast, other times it takes a minute or 10 as it's a big deal initially, so be aware. Once everyone's in the set, it's less of a big deal.

When it's done, head over to the other VMs and add them one at a time to the same availability set. The result looks like this in the Portal, and now I'm assured that my three VMs are all in different Fault Domains (and racks).

Hitting my Web Farm

Now I can hit hanselmanlinuxfarm.cloudapp.net/test.php and see the IP changing (as well as the CPU% changing in my portal!) or even watch HTOP over SSH and get a live view. Hey, I've got a little Linux farm!

Here's my SSH'ed into one of them, looking at htop (it's better than top!)

My Complete Script, Summarized

Here's my complete script. I used azure vm image list | find /I "13_04" to find an Ubuntu image. I could have done this with bash as well.

C:\>azure vm image list | find /i "13_04"
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130423-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130501-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130511-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130515-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130517-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130518-en-us-30GB
data:    b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130521-en-us-30GB

Once I've found an image, I create my first VM from the command line in a location of my choice. Again, it's Linux, don't forget the -ssh.

azure vm create mylinuxtemplate b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130501-en-us-30GB scott MyPassword --location "West US" --ssh

Get it how you like it. SSH in, set it up, run the waagent to prep it. Shut it down and capture it.

azure vm shutdown mylinuxtemplate
azure vm capture mylinuxtemplate mylinuxfarmimage --delete

Finally, here's a basic batch file to make 5 VMs. Note the first command is different from the Nth. Of course, with bash you could make a script like "spinup 5" and automate to your heart's content. The HTTP probe is optional on the endpoint creation.

azure vm create hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --location "West US" --ssh
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12345
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12346
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12347
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12348
azure vm endpoint create-multiple hanselmanlinuxfarm   80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-2 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-3 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-4 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-5 80:80:HttpTrafficInLhttp:80:/test.php

After it ran, I went in to the Portal and set up Availability Sets manually. That's only available in PowerShell today, but setting availability sets is coming soon to the cross-platform tools!

Next time, maybe I'll try"azure vm scale" to move these tiny VMs into 8 processor 56 gig monsters.

Sponsor: Big thanks to SoftFluent for sponsoring the feed this week! Less Plumbing, More Productivity! Generate rock-solid foundations for your .NET applications from Visual Studio and focus on what matters!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Sponsored By

Hosting By

I've been looking for ways to save money running my (now 12) websites in the cloud lately. Getting insights from logs has been helpful, but I really want more details as to what my app is doing so that I might do less of it. Remember the secret of scaling an application.

Have your app do as little as possible. If you do nothing, you can scale infinitely.

I like to use tools like Glimpse to profile my apps, check database calls, and explore what's really going on. If you like application insights you may be familiar with New Relic. They are a clever system that profiles apps of all kinds, including sites ASP.NET and Azure.

I tweeted this afternoon for folks to hit my site and help me test New Relic. My lovely friends on Twitter promptly DDOS'ed my site and I respect them all the more for it.

@shanselman Hey, you said "beat the crap out of", not "give it a little load". Two TOTALLY different things!

— Shane Milton (@Jaxidian) May 1, 2013

You can actually see my site getting loaded up in this chart, and see when I scaled up within Azure and switched from Shared to Reserved. The orange instance is shared, and the blue is a NEW instances when I switched to a Reserved VM. At this point, I had enough power to weather the storm. Thanks Twitter!

How did I setup New Relic on my Web Site? New Relic happens to have an office in Portland (where I live) so I went to visit today. Coincidentally (true story) their support for Azure Web Sites has sneaked out from a closed beta to an open one recently, so I decided to hide out and see if I could add New Relic performance and application monitoring to the Hanselminutes.com podcast site.

Adding New Relic to your Azure account

Login to the Azure Portal, click on New, then Store, then New Relic from within the Store.

Pick the free plan. You can have a free plan forever, they say. It doesn't include some "bells and whistles" and a storage of large amounts of historical data, but is otherwise quite functional. Plus you get a free 14-day trial (no credit card needed) of their Pro stuff.

Create the Add-On. They tell me the Data Center Location for New Relic doesn't matter, as you are just going to get a license key.

NOTE: If you already have a New Relic license key and existing billing relationship the you don't have to use the store or add an add-on. You can use your existing license key. However, I want my billing centralized, so any bill from New Relic will go through my Azure account. It's up to you.

Now, from within the Azure dashboard, click the new New Relic node. You can click Manage to automatically move over (and automatically single sign on) to the New Relic system dashboard. Note also the Connection Info button there. We'll need that in a minute.

If you click Manage and head over to the New Relic side you'll get a Welcome Message but you won't actually SEE anything interesting until your app has successfully made its first call to their system. You can check out their .NET docs if you like. They are in flux and not entirely accurate, but they'll get there.

We need to setup our Azure Website with some environment info, then add the New Relic NuGet package.

Staying with the Azure Portal, go to the Web Site you're going to instrument, and click Configuration and setup these Configuration Values. This hooks up the New Relic production profiler to the CLR. You can keep this running all the time, and it's easy to turn off.

Add these name/value pairs:

• COR_ENABLE_PROFILING - 1
• COR_PROFILER -{71DA0A04-7777-4EC6-9643-7D28B46A8A41}
• COR_PROFILER_PATH - C:\Home\site\wwwroot\newrelic\NewRelic.Profiler.dll
• NEWRELIC_HOME - C:\Home\site\wwwroot\newrelic

Your app settings will look like this in your website config within the Azure Portal:

Make sure you not only Save your config, but also (at least once) do a complete RESTART for New Relic to get a chance to hook in.

Add New Relic to your Web Site

Install the NewRelicWindowsAzure Nuget package using the NuGet Package Manager Console using this command:

Install-Package NewRelic.Azure.WebSites

The website for Hanselminutes.com is running on ASP.NET Web Pages and was written with WebMatrix so I used the NuGet GUI. You can tell that this package JUST got uploaded at the time of this writing as there's only 11 downloads!

Install this package and it'll lay down a few DLLS and your newrelic.config file. Go into the newrelic.config and copy in the license key from Connection Info (remember that? It's in the Azure Portal and pictured above) into the  config file. Also update your Application Name to some useful value as that's going to identify your site in the New Relic dashboard.

Here I am publishing my site up to Azure. Only the NewRelic agent dlls that I just NuGet'ed in to my app are being published (and a random jpg I forgot).

Once I deployed the site and hit it, I could see Hanselminutes appear within the New Relic system.

Here's some of the data I could access now! I can see an updated graph of where my time is being spent, both server side AND browser side.

I can see which pages load fast and which don't, and exactly why.

I can see SQL Queries and how long they took, what connections were opened...

And deep timelines showing not just where my time was spend in my application, but also where it was spent in the .NET Framework AND CLR itself!

And of course, I can see Stack Traces of problems with my code.

This just scratches the surface, really, but I'm stoked I was able to get the free New Relic tier setup on Azure Websites in just about 10 minutes. I found two spots where four SQL calls could possibly be collapsed into one. I also found a common (and dumb, on my part) ArgumentNullException that I'd been missing for weeks.

I'm pretty impressed with their offering. I think the Pro Tier is a little spendy for the small indie developer, but cheap for the pro dev. I'd like to see a $5 or $10 tier for small hobbyist sites but for now, Free is going to serve me very nicely.

Now I'm going to go and instrument my other sites!

Related Links

• Penny Pinching in the Cloud: How to run a two day Virtual Conference for $10
• Penny Pinching Video: Moving my Website's Images to the Azure CDN
• Penny Pinching Video: Moving an Azure Website between data centers
• Streaming Diagnostics Trace Logging from the Azure Command Line (plus Glimpse!)
• Pinching pennies when scaling in The Cloud

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Sponsored By

Hosting By

We've just finished Day One of "DotNetConf" our community-run free online conference for developers who love the .NET development platform and open source!

UPDATE: All the Videos from both days of DotNetConf are now online and available for on-demand viewing!

The Conference Platform

It seems funny to call the software our conference runs on a "platform" as that sounds very "enterprisey" and official. In the past we've done aspConf and mvcConf with sponsors who helped pay for things. We used Channel 9 and had a studio and streamed either from Seattle or using Live Meeting.

However, this year we wanted to do it on the cheap and more distributed. We wanted speakers from ALL over in all time zones. How cheap? About USD$10ish we figure. I'll get a complete bill later, but we basically wanted to scale up, do the talks and scale down.

Video Broadcasting and Screen-sharing

• This year we are using Google Hangouts with their "Hangouts On Air" feature. A "dotnetconf" Google Account invites the presenter to a Hang Out and check the "on air" box before start the hangout. Then we use the Hangout Toolbox to dynamically add on screen graphics and speaker labels. Everyone sets their resolution to 1280x768 and the live stream ends scaling down to 480p.
• Once you hit "Start Broadcast" you're given a YouTube link to the live stream. When you hit End Broadcast, the resulting video is ready to go on your YouTube page within minutes. The hangout owner (me or Javier) then clicks "Hide in Broadcast" and fades away. You can see I'm faded away in the below screenshot. I'm there, but only if I need to be. When there's only one active presenter the Hangout turns into a full screen affair, which is what we want.
• Important Note: Rather than an 8 hour Hangout, we started and stopped as each speaker did their talk. This means that our talks are already discrete on the YouTube page. The YouTube videos can have their start and end times trimmed so the start isn't so rough.

The Database

Surprise! There is no database. There is no need for one. We're running a two page site using ASP.NET Web Pages written in WebMatrix. It runs in the Azure cloud but since our dataset (speakers, schedule, the video stream location, etc) isn't changing a lot, we put all the data in XML files. It's data, sure, but it's a poor man's database. Why pay for more than we need?

How do we update the "database" during the talk? Get ready to have an opinion. The data is in Dropbox. (Yes, it could have been SkyDrive, or another URL, but we used DropBox)

Our Web App pulls the data from Dropbox URLs and caches it. Works pretty nice.

<appSettings>
   <add key="url.playerUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/VideoStreams.xml" />
   <add key="url.scheduleUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/Schedule.xml" />
   <add key="url.speakerUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/Speakers.xml" />
   <add key="Microsoft.ServiceBus.ConnectionString" value="Endpoint=sb://[your namespace].servicebus.windows.net;SharedSecretIssuer=owner;SharedSecretValue=[your secret]" />
</appSettings>

The code is simple, as code should be. Wanna show the schedule? And yes , it's a TABLE. It's a table of the schedule. Nyah.

@foreach(var session in schedule) {
    var confTime = session.Time;
    var pstZone = TimeZoneInfo.FindSystemTimeZoneById("Pacific Standard Time");
    var attendeeTime = TimeZoneInfo.ConvertTimeToUtc(confTime, pstZone);
<tr>
    <td>
        <p>@confTime.ToShortTimeString() (PDT)</p>
        <p>@attendeeTime.ToShortTimeString() (GMT)</p>
    </td>
    <td>
        <div class="speaker-info">
            <h4>@session.Title</h4>
            <br>
            <span class="company-name"><a class="speaker-website" href="/speakers.cshtml?speaker=@session.Twitter">@session.Name</a></span>
            <br>
            <p>@session.Abstract</p>
        </div>
    </td>
</tr>
}

Scaling Out

We've been on an extra small Azure Website and then switched to two large (and finally, two medium as large was totally overkill) web sites. 

We scale up (and hence, pay) only during the conference and turn it down to Small when we're done. No need to spend money if we don't need to.

Updating the Site in Real-time with SignalR

Because the YouTube link changes with each Hangout, we had the problem that attendees of the conference would have to hit refresh themselves to get the new URL. There's a number of solutions to this that I'm sure you're already thinking about. We could meta refresh, refresh on a timer, but these aren't on demand. We also wanted to show a few videos during the downtime. One of us preps the next speaker while the other queues up videos to watch.

We realized this was a problem at about 10pm PST last night. Javier and I got on Skype and came up with this late night hack.

What if everyone had SignalR running while their were watching the videos? Then we could push out the next YouTube video from an admin console.

So visualize this. There's the watcher (you), there's a admin (me) and there's the server (the Hub).

The watcher has this on their main page after including the /signalr/hub JavaScript:

$(function () {
    var youtube = $.connection.youTubeHub;
    $.connection.hub.logging = true;

    youtube.client.updateYouTube = function (message, password) {
        $("#youtube").attr("src", "http://www.youtube.com/embed/" + message + "?autoplay=1");
    };
    $.connection.hub.start();

    $.connection.hub.disconnected(function () {
        setTimeout(function () {
            $.connection.hub.start();
        }, 5000);
    });
});

The watcher is listening, er, watching, for a SignalR message from the server with the YouTube video short code. When we get it, we swap out the iFrame. Simple and it works.

Here's the admin console where we put in the next YouTube code (I'm using Razor in ASP.NET Web Pages in WebMatrix, so this is mixed HTML/JS):

<div id="container">
        <input type="text" id="videoId" name="videoId"><br/>
        <input type="text" id="password" name="passsword" placeholder="password"><br/>
        <button id="playerUpdate" name="playerUpdate">Update Player</button>
</div>

@section SignalR {
<script>    
    $(function () {
        var youtube = $.connection.youTubeHub;
        $.connection.hub.logging = true;

        $.connection.hub.start().done(function () {
            $('#playerUpdate').click(function () {
                youtube.server.update($('#videoId').val(), $('#password').val());
            });
        });
        $.connection.hub.disconnected(function() {
          setTimeout(function() {
              $.connection.hub.start();  
        }, 5000);
});
    });
</script>
}

We put in the short code, the password and update. All this must be complex eh? What's the powerful SignalR backend up running in the cloud backed by the power of Azure and Service Bus look like? Surely that code must be too complex to show on a simple blog, eh? Strap in, friends.

public class YouTubeHub : Microsoft.AspNet.SignalR.Hub
{
    public void update(string message, string password)
    {
        if (password.ToLowerInvariant() == "itisasecret")
        {
            Clients.All.updateYouTube(message);
            ConfContext.SetPlayerUrl(message);
        }
    }
}

This is either a purist's nightmare or a pragmatists dream. Either way, we've been running it all day and it works. Between talks we pushed in pre-recorded talks and messages, then finally when the live talk started we pushed that one as well.

We also updated the DropBox links with the current Video Stream so that new visitors showing up would get the latest video, as new site visitors wouldn't have been connected when the video "push" message went out.

What about scale out? We sometimes have two machines in the farm so we need the SignalR "push updated youtube video message" to travel across a scale-out backplane. That took another 10 minutes.

Scaling out with SignalR using the Azure Service Bus

We used the SignalR 1.1 Beta plus Azure Service Bus Topics for scale out and added an Azure Service Bus to our account. Our app startup changed, adding this call to UseServiceBus():

string poo = "Endpoint=sb://dotnetconf-live-bus.servicebus.windows.net/;SharedSecretIssuer=owner;SharedSecretValue=g57totalsecrets=";   
GlobalHost.DependencyResolver.UseServiceBus(poo,"dotnetconf");
RouteTable.Routes.MapHubs();

Now SignalR uses the Service Bus Topics for "Pub/Sub" to pass notifications between the two web servers. I can push a new video from Web 1 and it is sent to everyone on Web 1 and Web 2 (or Web N) via SignalR's realtime persistent connection.

We'll delete this Service Bus Topic as soon as we are done. I would hate for the bill to get up into the nickels. ;) Here's example pricing from the Azure site:

432,000 Service Bus messages cost 432,000/10,000 * $0.01 = 44 * $0.01 = $0.44 per day.

I'm not sure how many messages we've done, but I can rest assured it won't cost a pile, so that's a relief.

Thank you to the Community!

• Big thanks to designer Jin Yang who created the dotnetConf logo and design. Tweet @jzy and tell him you think he is special. Thanks to Dave Ward for converting Jzy's design into HTML!
• Kudos and thanks to Javier Lozano for his coding, his organizing, his brainstorming and his tireless hard work. It was also cool for him to sit with me for hours last night while we hacked on SignalR and the DotNetConf.net site.
• Thanks to David Fowler for saying "it'll just take 10 minutes to add Service Bus." 
• Thanks to Eric Hexter and Jon Galloway for their organizational abilities and generous gifts of time on all the *conf events!
• But mostly, thanks to the speakers who volunteered their time and presented and the community who showed up to watch, interact and play with us!

Sponsor: The Windows Azure Developer Challenge is on.  Complete 5 programming challenges for a chance at spot prizes, Stage prizes and the Grand Prize. Over $16,000 is up for grabs with 65 chances to win!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Sponsored By

Hosting By

I talked about Pinching pennies when scaling in The Cloud last week when I added jQuery lazy loading to my podcast's Website. Next, I moved my website to the same data center as my SQL Database (in fact, they should have always been together!). Now, I'm moving all my show images to the Azure CDN. There's been ~370 shows, and if someone visits the archives page and scrolls around it's about 8 megs of pics.

Additionally, I have a very international group who listen to my podcast, so by moving the images to the CDN I'll get load balancing and edge caching as well. Asians will get the images served from an Asian data center, etc.

Now, to be clear, Azure bandwidth is pretty cheap, with even 100GB costing around ~$11, but I also wanted to learn how to use Blob Storage and the CDN. I've only used it to store Virtual Machines and it's been hidden from me. Bandwidth from the Azure CDN is about the same price, but I get the geo-replication for free. Remember also that "ingress," that is, incoming traffic, is free. When you're estimating your bandwidth costs, you're only worried about outgoing traffic.

I also thought it would be cool to have a custom Hanselminutes subdomain, so I want the images to be served from http://images.hanselminutes.com, because it's cool. Plus, it means I could switch CDNs in the future and not change my URLs.

Here's the steps I used:

• I created a new storage account called 'hanselminutescdn,' then a container called 'images.'
• I went to 'Configure' and setup the images.hanselminutes.com CNAME to point to the new storage account. I went to my DNS provider (DNSimple.com) and added a CName from images.hanselman.com to point to hanselminutescdn.blob.core.windows.net, and then verified the domain per the Azure portable instructions. This involved adding another verification-specific CNAME.

• I downloaded the Azure SDK for .NET and used Visual Studio to upload the images.
   
• Later after the video, I got CloudBerry Explorer for Windows Azure and used their "sync" option to keep my local show images folder and Azure in sync. Very slick. Their app also lets me easily set HTTP Headers on my images if I want. I may pay for the Pro version.
  
• I changed my paths in my HTML to images.hanselminutes.com instead of just /images of the website's domain.
• I discovered that CDNs are case sensitive, then ran this PowerShell Script to make ALL my show images files all lowercase.

dir | Rename-Item { $_.Name.ToLowerInvariant() }

• I cleared out the files and reuploaded the new lower-case ones.
• PROFIT.

OK, I haven't figured out that last step, but soon...very soon. ;)

IMPORTANT UPDATE: Commenter Nate Jackson points out that while I've setup access to blob storage, I haven't yet actually enable the Edge Caching abilities of the CDN itself. For this, it seems I have to visit the old management portal (which explains why I missed it completely. It's not intuitive!).

You get to the old portal from your account dropdown:

There's instructions on the Azure docs site the say I need to create a "CDN endpoint" that associates the CDN with my now-public blob storage container.

I visit the Hosted Accounts + CDN button, then click New Endpoint.

The URL template looks like this.

http://<CDNNamespace>.vo.msecnd.net/<myPublicContainer>/<BlobName>

So after clicking new CDN endpoint and enabling the CDN, I'm told my CDN endpoint name is "az415467" per the old portal:

Given the URL template a URL from my container would be:

http://az415467.vo.msecnd.net/images/255.jpg

Then (if you want) click Add Custom Domain:

Then point the domain to the CDN, not to public blob storage:

At this point the old management portal looks like this:

And I can confirm it works when I visit

http://az415467.vo.msecnd.net/images/255.jpg  as well as http://images.hanselminutes.com/images/255.jpg

So the idea is that there's storage in general, there's blob storage made public (which is what I did) which can be geo-replicated, and then there's the formal geo-load-balanced Content Delivery Network, which I failed to configure!

CDN -> pulls from -> Public Blob Endpoint -> hosted in -> Your Storage Container itself

I will update the YouTube video soon. Big thanks to Nate Jackson for catching this oversight and educating me!

Related Links

• Pinching pennies when scaling in The Cloud
• Penny Pinching Video: Moving an Azure Website between data centers (YouTube)
• Streaming Diagnostics Trace Logging from the Azure Command Line (plus Glimpse!)
• How to keep your ASP.NET database connection strings secure when deploying to Azure from Source

Sponsor: The Windows Azure Developer Challenge is on.  Complete 5 programming challenges for a chance at spot prizes, Stage prizes and the Grand Prize. Over $16,000 is up for grabs with 65 chances to win!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Sponsored By

Hosting By

I talked about Pinching pennies when scaling in The Cloud last week when I added jQuery lazy loading to my podcast's Website. I wanted to avoid paying any unnecessary bandwidth costs. The result was great and I'll be under my bandwidth this month.

I'm continuing to look for ways to optimize and pinch pennies in the cloud. I realized recently that while my Website was running in the West US Azure datacenter, the database (managed by Carl Franklin's podcasting company) was running in North Central US. This means I was paying for the bandwidth of my database calls. Not to mention, it was slower, not the best idea, plus I was calling into a SQL Server over the open internet (although I had opened the firewall to do so).

This is unusual to have a website and SQL Database so far apart, of course, as you'll usually make your site and database at the same time in the same place. Azure also goes out of it's way to keep these linked resources together as you build them.

However, Carl had setup the database and original website a while back, and I only just redesigned it and moved it to Azure recently. Additionally, the administrative backend for the Hanselminutes podcast was in North Central, so we found ourselves in this position.

Azure Websites capacity opened up in the North Central datacenter, so I took lunch to move my site. You can't just click "move," but it's actually very easy to redeploy. The whole process including DNS changes took less than 15 minutes as you can see in the YouTube video above.

Here's the steps I used:

• I made a new site in the new Data Center
• I made it Shared so I could use a custom domain (or you can use Reserved)
• I took the domain names off the West US site, and moved them within the Portal to the North Central one
  • If this site was super important I would have had a load balancer and kept both sites up while I waited, but total downtime was like 5 min so I didn't sweat it for this.
• I ensured the database within North Central was a "Linked Resource" within my Website
• I made sure my new website had the right connection strings in configuration.
• I downloaded the new website's publish profile and imported it anew into WebMatrix (or Visual Studio, etc)
• Published the site using the new publish profile.
• Cleared DNS and visited the site and confirmed it worked.
• Deleted the old site.

It worked well and I'm happy with the result. My next penny pinching step (and a nice geo-load balanced optimization) will be to move all the images to the CDN so that folks overseas get edge caching...that means that Australians will get the images for the site served from a nearby datacenter. I'll get this extra benefit for less than I am paying for website bandwidth.

Related Links

• Pinching pennies when scaling in The Cloud
• Streaming Diagnostics Trace Logging from the Azure Command Line (plus Glimpse!)
• How to keep your ASP.NET database connection strings secure when deploying to Azure from Source

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Sponsored By

Hosting By