Scott Hanselman

Penny Pinching Video: Moving my Website's Images to the Azure CDN (and using a custom domain)

April 22, '13 Comments [36] Posted in Azure
Sponsored By

I talked about Pinching pennies when scaling in The Cloud last week when I added jQuery lazy loading to my podcast's Website. Next, I moved my website to the same data center as my SQL Database (in fact, they should have always been together!). Now, I'm moving all my show images to the Azure CDN. There's been ~370 shows, and if someone visits the archives page and scrolls around it's about 8 megs of pics.

CDN Metrics are looking greatAdditionally, I have a very international group who listen to my podcast, so by moving the images to the CDN I'll get load balancing and edge caching as well. Asians will get the images served from an Asian data center, etc.

Now, to be clear, Azure bandwidth is pretty cheap, with even 100GB costing around ~$11, but I also wanted to learn how to use Blob Storage and the CDN. I've only used it to store Virtual Machines and it's been hidden from me. Bandwidth from the Azure CDN is about the same price, but I get the geo-replication for free. Remember also that "ingress," that is, incoming traffic, is free. When you're estimating your bandwidth costs, you're only worried about outgoing traffic.

I also thought it would be cool to have a custom Hanselminutes subdomain, so I want the images to be served from http://images.hanselminutes.com, because it's cool. Plus, it means I could switch CDNs in the future and not change my URLs.

Here's the steps I used:

  • I created a new storage account called 'hanselminutescdn,' then a container called 'images.'
  • I went to 'Configure' and setup the images.hanselminutes.com CNAME to point to the new storage account. I went to my DNS provider (DNSimple.com) and added a CName from images.hanselman.com to point to hanselminutescdn.blob.core.windows.net, and then verified the domain per the Azure portable instructions. This involved adding another verification-specific CNAME.

A custom Azure CNAME for my Domain

  • I downloaded the Azure SDK for .NET and used Visual Studio to upload the images.
    image 
  • Later after the video, I got CloudBerry Explorer for Windows Azure and used their "sync" option to keep my local show images folder and Azure in sync. Very slick. Their app also lets me easily set HTTP Headers on my images if I want. I may pay for the Pro version.
    Cloud Berry Explorer
  • I changed my paths in my HTML to images.hanselminutes.com instead of just /images of the website's domain.
  • I discovered that CDNs are case sensitive, then ran this PowerShell Script to make ALL my show images files all lowercase.
dir | Rename-Item { $_.Name.ToLowerInvariant() }
  • I cleared out the files and reuploaded the new lower-case ones.
  • PROFIT.

OK, I haven't figured out that last step, but soon...very soon. ;)

IMPORTANT UPDATE: Commenter Nate Jackson points out that while I've setup access to blob storage, I haven't yet actually enable the Edge Caching abilities of the CDN itself. For this, it seems I have to visit the old management portal (which explains why I missed it completely. It's not intuitive!).

You get to the old portal from your account dropdown:

Switching over to the Previous Portal

There's instructions on the Azure docs site the say I need to create a "CDN endpoint" that associates the CDN with my now-public blob storage container.

I visit the Hosted Accounts + CDN button, then click New Endpoint.

Creating a CDN Endpoint

The URL template looks like this.

http://<CDNNamespace>.vo.msecnd.net/<myPublicContainer>/<BlobName>

So after clicking new CDN endpoint and enabling the CDN, I'm told my CDN endpoint name is "az415467" per the old portal:

Making a new CDN endpoint I'm given a unique URL

Given the URL template a URL from my container would be:

http://az415467.vo.msecnd.net/images/255.jpg

Then (if you want) click Add Custom Domain:

Associating my custom domain with my new CDN endpoint

Then point the domain to the CDN, not to public blob storage:

Pointing my custom domain to the CDN

At this point the old management portal looks like this:

Now my blob is associated with my custom domain name

And I can confirm it works when I visit

http://az415467.vo.msecnd.net/images/255.jpg  as well as http://images.hanselminutes.com/images/255.jpg

So the idea is that there's storage in general, there's blob storage made public (which is what I did) which can be geo-replicated, and then there's the formal geo-load-balanced Content Delivery Network, which I failed to configure!

CDN -> pulls from -> Public Blob Endpoint -> hosted in -> Your Storage Container itself

I will update the YouTube video soon. Big thanks to Nate Jackson for catching this oversight and educating me!

Related Links


Sponsor: The Windows Azure Developer Challenge is on.  Complete 5 programming challenges for a chance at spot prizes, Stage prizes and the Grand Prize. Over $16,000 is up for grabs with 65 chances to win!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Monday, April 22, 2013 5:19:37 PM UTC
hopefully , there is a central place that have all these Azure features and APIs .
Sam
Monday, April 22, 2013 5:20:09 PM UTC
I mean Central Documentation
Sam
Monday, April 22, 2013 5:28:31 PM UTC
This is cool! When I was at an Amazon sub we had TBs of content to ship out to AWS... we literally sent them physical hard drives via FedEx. LOL.
Aaron King
Monday, April 22, 2013 5:45:15 PM UTC
Another one I've used was Azure Storage Explorer on codeplex. Works pretty well and it's open source.
Stephen Hardie
Monday, April 22, 2013 6:12:30 PM UTC
I think this video only shows adding Azure BLOB storage to the mix, not the Azure CDN. You currently have to use the "old" Azure portal to enable CDN support for your BLOB container, and you'll have to do a similar DNS dance and URL swap in your app to point to the Azure CDN edge.
Nate Jackson
Monday, April 22, 2013 6:23:11 PM UTC
I'm struck by the irony that you are probably increasing traffic to your site by publishing these How-To's, thus increasing your bandwidth

:)
Marcus Swope
Monday, April 22, 2013 6:26:08 PM UTC
Nate Jackson - No, I never had to visit the old portal. Even the CNAME was configured in the video. Did you watch it? There was no slight of hand.
Monday, April 22, 2013 6:29:54 PM UTC
Nate,
Do you know how/if Azure CDN supports edge pull? I've seen vague mentions that it does, but never seen it implemented correctly.
Scott Scowden
Monday, April 22, 2013 6:35:25 PM UTC
I used the cdn just like you said, bu I had one problem with the cdn cache, it was very slow when the image in the cdn expired in the cache.

another alternative was to use the biggest max-age and then have a different folder for each version of the published site.

example:
images.hanselman.com/v20130422/ for the site published today.

what do you think of this approach?

Luiz Bicalho
Monday, April 22, 2013 6:39:51 PM UTC
Scott, I did watch the video. But, what I'm saying is that you didn't do anything with the Azure CDN. You created and configured your site to use Azure BLOB storage. That is not the CDN. To enable your BLOB on the CDN, you do have to open up the old Azure portal, and enabled it. Enabling it straight forward, and you can even do the custom domain thing on the CDN as you did with the BLOB storage.

Here is a cheat sheet: http://www.windowsazure.com/en-us/develop/net/common-tasks/cdn/

When you created your storage account, you enabled GEO replication, which just means that Azure will store your stuff in a couple places in case they have disaster. That has nothing to do with CDN.
Nate Jackson
Monday, April 22, 2013 7:04:45 PM UTC
Just a side note since it wasn't (specifically) mentioned: the HTTP 1.1 specification (RFC2616) mandates a two-connection limit, so using a CDN offers yet another way to 'speed up' the user experience.
Jason Harrell
Monday, April 22, 2013 7:17:51 PM UTC
Yes, as Nate said, this is exposing your images directly from blob storage, not the azure cdn which involves another few steps (and a visit to the old silverlight portal).

@Scott Scowden - believe all azure cdn is pull-based, either from blob storage or from a web role where you add content to a cdn subfolder. Note that it must be a web role - a windows azure web site will not work right now.
Paul H
Monday, April 22, 2013 7:24:39 PM UTC
Nate is 100% right on. I'm checking to see when this bit will be in the new Portal. I'm also updating this post now with his excellent instructions. I will also update the YouTube video.

Thanks everyone!
Monday, April 22, 2013 7:25:51 PM UTC
I followed the same instructions that @Nate did. It looks like you are just accessing the storage container directly instead of configuring a CDN to pull from the container. I followed the guide at http://www.windowsazure.com/en-us/manage/services/other/cdn/ which appears to have the same content as Nate's link. The CDN pulls from the storage account and picks up the cache control headers set there.

The only thing that is wrong on that page is how to get to the "previous portal", as the Management portal has changed since they wrote the instructions. You have to navigate to https://manage.windowsazure.com/ and click your account name (e-mail) in the upper right, then you can select Previous portal from the menu.

I did use CloudBerry and will be buying the Pro version for the sync support. They have some really great products for AWS, too.


Mark Larson
Monday, April 22, 2013 7:43:28 PM UTC
Mark - Thanks! I've updated the post (refresh) and I'm getting the docs updated now.
Monday, April 22, 2013 7:46:58 PM UTC
Scott - Listen to Nate. You actually never enabled CDN for your images. In the video you just moved them off your website and they are now served from Azure blob store.
Tim Skauge
Monday, April 22, 2013 7:50:07 PM UTC
Tim - Yep, see the updated post (Refresh), and new comments. Video is next!
Monday, April 22, 2013 7:57:18 PM UTC
My bad - didn't refresh the site before posting a comment.

Scott you need some SignarR on love on your comment section so I can see other peoples comments while I'm typing one :)

Looking forward to the updated version and video.
Tim Skauge
Monday, April 22, 2013 7:58:33 PM UTC
LOL. I hear you. I'll update the video today. Thanks everyone! I'm learning! :)
Monday, April 22, 2013 11:36:46 PM UTC
Azure bandwidth is not cheap. It's the reason I haven't moved to it. I have a pretty solid server at a host with data centers in Dallas and Seattle and pay $167 for the box and bandwidth... 2000 GB worth. Even if half the cost is for bandwidth, that's a little over $4 per 100 GB.

Like I said, it's the only thing keeping me from Azure.
Tuesday, April 23, 2013 2:21:38 AM UTC
The cheapest photo you can serve is one that doesn't actually hit your infrastructure! You should employ two techniques to maximize your investment in BLOBs and CDNs:

1) Once you publish a BLOB's URL, NEVER change the contents of the BLOB. If an edit or update is necessary, use a new BLOB with a new URL.
2) Ensure that your content is tagged with appropriate meta-data to allow the web's caching infrastructure to step in and serve the 2nd-n requests for a given resource. In Azure's case, use a tool that lets you manage the meta data that ultimately controls the HTTP Cache-Control headers. You mentioned Cloudberry Explorer; Cerebrata's Azure Management Studio is another option. Or, if you are writing code to post your BLOBs, take control of the expiration headers as shown here: http://msdn.microsoft.com/en-us/library/windowsazure/gg680306.aspx
Nate Jackson
Tuesday, April 23, 2013 12:22:23 PM UTC
I had done this with my own blog a while ago myself.
I used a script doing two things:

var maxage = 86400;
blob.Properties.CacheControl = String.Format("public, max-age={0}", maxage);


Because the CDN resends these headers, it helps to improve loadspeed as well.

I was following Google Pagespeed and even optimized the hell out of my PNGs using PNGPOUT ;)

(I also documented it here: Optimizing My Blog (WordPress) For Speed )
Tuesday, April 23, 2013 9:55:01 PM UTC
Fabian,
But isn't just a day too little?
instead of using just a day woudn't be better to use 2592000 and change the url when you need to change the image?

I think for me the problem is bigger because I'm in Brazil, very far from azure datacenter, and most of times with a very limited bandwith.



Luiz Bicalho
Thursday, April 25, 2013 7:06:14 AM UTC
Great video Scott. I look forward to seeing more of them.
Tim Murphy
Sunday, May 05, 2013 11:37:46 AM UTC
Fantastic video Scott.

I'm putting together a website to be hosted on azure and realised that I potentially need to host a great deal of images.

This video really pushed me in the right direction.
Tuesday, May 07, 2013 4:54:53 AM UTC
Hi Scott, I'm curious about the fact that you are now hosting your podcast images on the Azure CDN, but the MP3 files are stored on AWS S3. For your needs, how do you find these two services compare? I assume S3 is setup (or can be) as a CDN. Is S3 cheaper for large files?
Bart
Sunday, May 12, 2013 8:57:24 PM UTC
Bart - My producer, Carl, hosts the MP3s on S3 and has for years. I'd host them on LibSyn if I had my way. The pricing is similar but I think S3 is a little cheaper.
Tuesday, August 13, 2013 2:36:41 PM UTC
Hi Scott,

Someone has removed the CDN link from the old interface, and so I am unable to create a CDN. Could you let someone know this is broken?

Thanks,

Daniel
daniel
Wednesday, August 28, 2013 5:10:39 AM UTC
Hi Scott,

Ensure that your content is tagged with appropriate meta-data to allow the web's caching infrastructure to step in and serve the 2nd-n requests for a given resource.You created and configured your site to use Azure BLOB storage.

Thanks

John

Wednesday, August 28, 2013 3:30:45 PM UTC
Actually the link to enable the CDN is no longer there - there is no way to get back to the old portal.

Does anyone know how to enable the CDN? The newest version of the doucmentation says to

In the Windows Azure Management Portal, in the navigation pane, click CDN.


However, there is in fact no "CDN" link in the navigation pane. I looked through the source code of the page and there does seem to be a reference to the CDN in there, but it does not display. Any idea how to fix this?
Thursday, August 29, 2013 2:11:28 AM UTC
@Steve:

CDNs are now accessible from the new portal. If you want to create one, click the "Big Plus" at bottom left and they're under App Services > CDN.

If you have an existing CDN there is a button in the bottom half of the list of objects on the left hand side: https://manage.windowsazure.com/#Workspaces/Cdn
Thursday, August 29, 2013 3:17:06 PM UTC
@Porges - actually I don't have that option. All I have under Plus > App Services is Media Service, Active Directory and Service Bus.

I've got a paying account, and I pay MS a good chunk every month for it. Frustrating.
Friday, September 06, 2013 6:11:52 PM UTC
@Steve French - I just contacted Azure support regarding this and got this response:

"I inform you with regret that currently we’re in the process of building out our next generation Windows Azure Content Delivery Network (CDN) architecture, and during this time we are no longer accepting new CDN customers.

With the CDN functionality having recently transitioned to the new Azure management portal, we have enabled it automatically only for customers who were using CDN in the old management portal, so they can continue to manage their CDN configurations. For everyone else, the CDN feature is not enabled in the new portal.
We highly encourage you to wait until we’re ready building our next generation Azure CDN service."
GregP
Sunday, September 08, 2013 6:13:24 PM UTC
Azure's old management portal mostly does not work. For nearly an hour I gave up setting up CDN as first it would not load CDN section & when it did whole area was not responsive for mouse clicks.

Too bad that more new features are coming on to new management portal but not CDN. Finally our management switched to Amazon CloudFront. CloudFront custom domain SSL is very expensive, we hope Azure CDN on new portal will have some price benefits.
Sunday, September 29, 2013 1:25:17 PM UTC
JUst wondering , why can't these features all be automatcally installed - as a 58 year old stale left brainer- it ain't that easy .

Just so those engineers out there know !
Monday, September 30, 2013 6:05:16 PM UTC
Scott, I publish my azure website and move my images to blob storage, but sometimes, browser show 30% percent of my images. 70% return 304 http code and looks like empty. I enter some image URL (from other 70%) address bar and it is not loading. I see empty screen with ctrl+f5 and other key combinations. But when I change protocol to https - it load ok. It is correct to load blob data through httP ?
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.