Scott Hanselman

Real World Cloud Migrations: CDNs are an easy improvement to legacy apps

July 4, '19 Comments [5] Posted in Azure
Sponsored By

I'm doing a quiet backend migration/update to my family of sites. If I do it right, there will be minimal disruption. Even though I'm a one person show (plus Mandy my podcast editor) the Cloud lets me manage digital assets like I'm a whole company with an IT department. Sure, I could FTP files directly into production, but why do that when I've got free/cheap stuff like Azure DevOps.

As I'm one person, I do want to keep costs down whenever possible and I've said so in my "Penny Pinching in the Cloud" series. However, I do pay for value, so I'll try to keep costs down whenever possible, but I will pay for things I feel are useful or provide me with a quality product and/or save me time and hassle. For example, Azure Pipelines gives one free Microsoft-hosted CI/CD pipeline and 1 hosted job with 1800 minutes a month. This should be more than enough for my set up.

Additionally, sometimes I'll shift costs in order to both save money and improve performance as when I moved my Podcast's image hosting over to an Azure CDN in 2013. Fast forward 6 years and since I'm doing this larger migration, I wanted to see how easy it would be to migrate even more of my static assets to a CDN.

Make a Storage Account and ensure that its Access Level is Public if you intend folks to be able to read from it over HTTP GETs.

Public Access Level

With in the Azure Portal you can make a CDN Profile that uses Microsoft, Akamai, or Verizon for the actual Content Distribution Network. I picked Microsoft's because I have experience with Akamai and Verizon (they were solid) and I wanted to see how the new Microsoft one does. It claims to put content within 50ms of 60 countries.

You'll have a CDN endpoint host name that points to an Origin. That Origin is the Origin of your content. The Origin can be an existing place you have stuff so the CDN will basically check there first, cache things, then serve the content. Or it can be an existing WebApp or in my case, Storage.

Azure Storage Accounts for my blog

I didn't want to make things too complex, so I have a single Storage Account with a few containers. Then I mapped custom endpoints for both my blog AND my podcast, but they share the same storage. I also took advantage of the free SSL Certs so images.hanselman.com and images.hanselminutes.com are both SSL. Gotta get those "A" grades from https://securityheaders.com right?

Custom domains for my CDN

Then, just grab the Azure Storage Explorer. It's free and cross platform. In fact, you can get Storage Explorer as an app that runs locally, or you can check it out in the Azure  Portal/in-browser as well, here. I've uploaded all my main assets (images used in my CSS, blog, headers, etc).

I've settled on a basic scheme where anything that was "/images/foo.png" is now "https://images.hanselman.com/blog/foo.png" or /main/ or /podcast/ depending on who is using it. This made search and replaces reliable and easy.

It's truly a less-than-an-hour operation to enable a CDN on an existing site. While I've chose to use Azure Storage as my backing store, you can just use your existing site's /images folder and just change your markup to pull from the CDN's URL. I recommend you make a simple cdn.yourdomain.com or images.yourdomain.com. This can also easily be enabled with a CNAME in less than an hour. Having your images at another URL via a subdomain CNAME also allows for even more download parallelism from the browser.

This is all in my Staging so you won't see it quite yet until I flip the switch on the whole migration.

Exploring future possibilities for dynamic image content

I haven't yet moved all my blog content images (a few gigs, but many thousands of images) to a CDN as I don't want to change my existing publishing workflow with Open Live Writer. I'm considering a flow that keeps the images uploading to the Web App but then using the Custom Origin Path options in the Azure CDN that will have images get picked up Web App then get served by the CDN. In order to manage 17 years of images though, I'd need to catch URLs like this:

https://www.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

and redirect them to

https://images.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

As I see it, there's a few options to get images from a GET of /blog/content/binary to be served by a CDN:

  • Dynamically Rewrite the URLs on the way OUT (as the HTML is generated)
    • CPU expensive, but ya it's cached in my WebApp
  • Rewriting Middleware to redirect (301?) requests to the new images location
    • Easiest option, but costs everyone a 301 on all image GETs
  • Programmatically change the stored markup (basically forloop over my "database," search and replace, AND ensure future images use this new URL)
    • A hassle, but a one time hassle
    • Not sure about future images, I might have to change my publishing flow AND run the process on new posts occasionally.

What are your thoughts on if I should go all the way and manage EVERY blog image vs the low hanging fruit of shared static assets? Worth it, or overkill?

The learning process continues!


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Real World Cloud Migrations: Moving a 17 year old series of sites from bare metal to Azure

July 2, '19 Comments [7] Posted in ASP.NET | Azure | DasBlog
Sponsored By

Technical Debt has a way of sneaking up on you. While my podcast site and the other 16ish sites I run all live in Azure and have a nice CI/CD pipeline with Azure DevOps, my main "Hanselman.com" series of sites and mini-sites has lagged behind. I'm still happy with its responsive design, but the underlying tech has started to get more difficult to manage and build and I've decided it's time to make some updates.

Moving sites to Azure DevOps

I want to be able to make these updates and have a clean switch over so that you, the reader, don't notice a difference. There's a number of things to think about when doing any migration like this, realizing it'll take some weeks (or months if you're a bigger company that just me).

  • Continuous Deployment/Continuous Integration
    • I host my code on GitHub and Azure DevOps now lets you log in with GitHub and does a fine job of building AND deploying your code (while running tests AND allowing for manual quality gates) so I want to make sure my sites have a nice clean "check in and go live" process.
    • I'll also be using Azure App Services and Deployment Slots, so I'll have a dev/test/staging site and production, like a real professional. No more editing text files in production. Well, at least, I won't tell you when I'm editing text file in production.
  • Technology Update
    • Hanselman.com proper (not the blog) and the mini pages/sites underneath it run on ASP.NET 4.0 and WebForms. I was able to easily move the main site over to ASP.NET Razor Pages. Razor is just so elegant, as it's basically just HTML then you type @ and you're in C# (Razor). More on that below, but the upgrade was a day as the home page and minisites are largely readonly.
    • The Blog, hosted at /blog will be more challenging given I don't want to break two decades years of URLs, along with the fact that it's running DasBlog on a recently upgraded .NET 4.0. DasBlog was originally made in .NET 1, then upgraded to .NET 2, so this is 17 years of technical debt.
    • That said, the .NET Standard along with open source cross-platform .NET Core has allowed us - with the leadership of Mark Downie - to create DasBlog Core. DasBlog Core shares the core reliable (if crusty) engine of DasBlog along with an all new system of URL writing using ASP.NET Core middleware, as well as a complete re-do of the (well ahead of its time) DasBlog Theming Engine, now based on Razor Pages. It's brilliant. This is in active development.
  • Azure Front Door
    • Because I'm moving from a single machine running IIS to Azure, I'll want to split things apart to remove single points of failture. I'll use Azure Front Door to manage my URL structure and act as a front end cache as well as distribute traffic to multiple Azure App Services (Web Apps).
  • URL management
    • Are you changing your URLs and URL structure? Remember that URLs are UI and they matter. I've long wanted to remove the "aspx" extension from my URLs, as well as move the TitleCaseBlogPostThing to a more "modern" title-case-blog-post-thing style. I need to do this in a way that updates my google sitemap, breaks zero URLs, 301 redirects to the new style, and uses rel=canonical in a smart way.
  • Shared Assets/CDNs/Front Door
    • Since I run a family of sites, there's an opportunity to use a CDN as well and some clean CNAME DNS such that images.hanselman.com and images.hanselminutes.com can share assets. Since the Azure CDN is easy to setup and offers free SSL certs and pay-as-you go, I'll set both of those CNAMES up to point to the same Azure Storage where I'll keep images, show pics, CSS, and JS.

I'll be blogging the whole process. What do you want to hear/learn about?


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Git is case-sensitive and your filesystem may not be - Weird folder merging on Windows

June 27, '19 Comments [7] Posted in DasBlog | Open Source
Sponsored By

I was working on DasBlog Core (an .NET Core cross-platform update of the ASP.NET WebForms-based blogging software that runs this blog) with Mark Downie, the new project manager, and Shayne Boyer. This is part of a larger cloud re-architecture of hanselman.com and the systems that run this whole site.

Shayne was working on getting a DasBlog Core CI/CD (Continuous Integration/Continuous Development) running in Azure DevOps' build system. We wanted individual build pipelines to confirm that DasBlog Core was in fact, cross-platform, so we needed to build, test, and run it on Windows, Linux, and Mac.

The build was working great on Windows and Mac...but failing on Linux. Why?

Well, like all things, it's complex.

  • Windows has a case-insensitive file system.
  • By default, Mac uses a case-insensitive file system.

Since Git 1.5ish there's been a setting

git config --global core.ignorecase true

but you should always be aware of what a setting does before you just set it.

If you're not careful, you or someone on your team can create a case sensitive file path in your git index while you're using a case insensitive operating system like Windows or Mac. If you do this, you'll be able to end up with two separate entries from git's perspective. However Windows will silently merge them and see just one.

Here's our themes folder structure as seen on GitHub.com.

Case insenstive folder names

But when we clone it on Mac or Windows, we see just one folder.

DasBlog as a single folder in VS Code

Turns out that six months ago one of us introduced another folder with the name dasblog while the original was DasBlog. When we checked them on Mac or Windows the files ended up in merged into one folder, but on Linux they were/are two, so the build fails.

You can fix this in a few ways. You can rename the file in a case-sensitive way and commit the change:

git mv --cached name.txt NAME.TXT

Please take care and back up anything you don't understand.

If you're renaming a directory, you'll do a two stage rename with a temp name.

git mv foo foo2
git mv foo2 FOO
git commit -m "changed case of dir"

Be safe out there!


Sponsor: Looking for a tool for performance profiling, unit test coverage, and continuous testing that works cross-platform on Windows, macOS, and Linux? Check out the latest JetBrains Rider!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Adding Reaction Gifs for your Build System and the Windows Terminal

June 25, '19 Comments [6] Posted in Open Source | PowerShell | Win10
Sponsored By

So, first, I'm having entirely too much fun with the new open source Windows Terminal. If you've got the latest version of Windows (go run Windows Update and do whatever it takes) then you can download the Windows Terminal from the Microsoft Store! This is a preview release (think v0.2) but it'll automatically update, often, from the Windows Store if you have Windows 10 version 18362.0 or higher.

One of the most fun things is that you can have background images. Even animated gifs! You can add those images in your Settings/profile.json like this.

"backgroundImage": "c:/users/scott/desktop/doug.gif",
"backgroundImageOpacity": 0.7,
"backgroundImageStretchMode": "uniformToFill

The profile.json is just JSON and you can update it. I could even update it programmatically if I wanted to parse it and mess about.

BUT. Enterprising developer Chris Duck created a lovely PowerShell Module called MSTerminalSettings that lets you very easily make Profile changes with script.

For example, Mac developers who use iTerm often go to https://iterm2colorschemes.com/ and get new color schemes for their consoles. Now Windows folks can as well!

From his docs, this example downloads the Pandora color scheme from https://iterm2colorschemes.com/ and sets it as the color scheme for the PowerShell Core terminal profile.

Invoke-RestMethod -Uri 'https://raw.githubusercontent.com/mbadolato/iTerm2-Color-Schemes/master/schemes/Pandora.itermcolors' -OutFile .\Pandora.itermcolors
Import-Iterm2ColorScheme -Path .\Pandora.itermcolors -Name Pandora
Get-MSTerminalProfile -Name "PowerShell Core" | Set-MSTerminalProfile -ColorScheme Pandora

That's easy! Then I was talking to Tyler Leonhardt and suggested that we programmatically change the background using a folder full of Animated Gifs. I happen to have such a folder (with 2000 categorized gif classics) so we started coding and streamed the whole debacle on Tyler's Twitch!

The result is Windows Terminal Attract Mode and it's a hot mess and it is up on GitHub and all set up for PowerShell Core.

Remember that "Attract mode" is the mode an idle arcade cabinet goes into in order to attract passersby to play, so clearly the Terminal needs this also.

./AttractMode.ps1 -name "profile name" -path "c:\temp\trouble" -secs 5

It's a proof of concept for now, and it's missing background/runspace support, being wrapped up in a proper module, etc but the idea is solid, building on a solid base, with improvements to idiomatic PowerShell Core already incoming. Right now it'll run forever. Wrap it in Start-Job if you like as well and can stand it.

The next idea was to have reactions gifs to different developer situations. Break the build? Reaction Gif. Passing tests? Reaction Gif.

Here's a silly proof (not refactored) that aliases "dotnet build" to "db" with reactions.

#messing around with build reaction gifs

Function DotNetAlias {
dotnet build
if ($?) {
Start-job -ScriptBlock {
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\chrispratt.gif"
Start-Sleep 1.5
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\4003cn5.gif"
} | Out-Null
}
else {
Start-job -ScriptBlock {
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\idk-girl.gif"
Start-Sleep 1.5
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\4003cn5.gif"
} | Out-Null

}
}

Set-Alias -Name db -value DotNetAlias

I added the Start-job stuff so that the build finishes and the Terminal returns control to you while the gifs still are updating. Runspace support would be smart as well.

Some other ideas? Giphy support. Random mood gifs. Pick me ups. You get the idea.

Later, Brandon Olin jumped in with this gem. Why not get a reaction gif if anything goes wrong in your last command? ERRORLEVEL 1? Explode.

Why are we doing this? Because it sparks joy, y'all.


Sponsor: Looking for a tool for performance profiling, unit test coverage, and continuous testing that works cross-platform on Windows, macOS, and Linux? Check out the latest JetBrains Rider!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

You can now download the new Open Source Windows Terminal

June 20, '19 Comments [27] Posted in Open Source | Win10
Sponsored By

Last month Microsoft announced a new open source Windows Terminal! It's up at https://github.com/microsoft/Terminal and it's great, but for the last several weeks you've had to build it yourself as a Developer. It's been very v0.1 if you know what I mean.

Today you can download the Windows Terminal from the Microsoft Store! This is a preview release (think v0.2) but it'll automatically update, often, from the Windows Store if you have Windows 10 version 18362.0 or higher. Run "winver" to make sure.

Windows Terminal

If you don't see any tabs, hit Ctrl-T and note the + and the pull down menu at the top there. Under the menu go to Settings to open profiles.json. Here's mine on one machine.

Here's some Hot Windows Terminal Tips

You can do background images, even animated, with opacity (with useAcrylic off):

"backgroundImage": "c:/users/scott/desktop/doug.gif",
"backgroundImageOpacity": 0.7,
"backgroundImageStretchMode": "uniformToFill

You can edit the key bindings to your taste in the "key bindings" section. For now, be specific, so the * might be expressed as Ctrl+Shift+8, for example.

Try other things like cursor shape and color, history size, as well as different fonts for each tab.

 "cursorShape": "vintage"

If you're using WSL or WSL2, use the distro name like this in your new profile:

"wsl.exe -d Ubuntu-18.04"

If you like Font Ligatures or use Powerline, consider Fira Code as a potential new font.

I'd recommend you PIN terminal to your taskbar and start menu, but you can run windows terminal from the command "wt" from Windows R or from anotherc console. That's just "wt" and enter!

Try not just "Ctrl+Mouse Scroll" but also "Ctrl+Shift+Mouse Scroll" and get your your whole life!

Remember that the definition of a shell is someone fluid, so check out Azure Cloud Shell, in your terminal!

Windows Terminal menus

Also, let's start sharing nice color profiles! Share your new ones as a Gist in this format. Note the name.

{
"background" : "#2C001E",
"black" : "#4E9A06",
"blue" : "#3465A4",
"brightBlack" : "#555753",
"brightBlue" : "#729FCF",
"brightCyan" : "#34E2E2",
"brightGreen" : "#8AE234",
"brightPurple" : "#AD7FA8",
"brightRed" : "#EF2929",
"brightWhite" : "#EEEEEE",
"brightYellow" : "#FCE94F",
"cyan" : "#06989A",
"foreground" : "#EEEEEE",
"green" : "#300A24",
"name" : "UbuntuLegit",
"purple" : "#75507B",
"red" : "#CC0000",
"white" : "#D3D7CF",
"yellow" : "#C4A000"
}

Note also that this should be the beginning of a wonderful Windows Console ecosystem. This isn't the one terminal to end them all, it's the one to start them all. I've loved alternative consoles for YEARS, whether it be ConEmu or Console2 many years ago, I've long declared that Text Mode is a missed opportunity.

Remember also that Terminal !== Shell and that you can bring your shell of choice into your Terminal of choice! If you want the deep architectural dive, be sure to watch the BUILD 2019 technical talk with some of the developers or read about ConPTY and how to integrate with it!


Sponsor: Get the latest JetBrains Rider with WinForms designer, Edit & Continue, and an IL (Intermediate Language) viewer. Preliminary C# 8.0 support, rename refactoring for F#-defined symbols across your entire solution, and Custom Themes are all included.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.