Scott Hanselman

Dealing with Application Base URLs and Razor link generation while hosting ASP.NET web apps behind Reverse Proxies

July 9, '19 Comments [4] Posted in ASP.NET | Azure | DasBlog | Open Source
Sponsored By

Updating my site to run on AzureI'm quietly moving my Website from a physical machine to a number of Cloud Services hosted in Azure. This is an attempt to not just modernize the system - no reason to change things just to change them - but to take advantage of a number of benefits that a straight web host sometimes doesn't have. I want to have multiple microsites (the main page, the podcast, the blog, etc) with regular backups, CI/CD pipeline (check in code, go straight to staging), production swaps, a global CDN for content, etc.

I'm also moving from an ASP.NET 4 (was ASP.NET 2 until recently) site to ASP.NET Core 2.x LTS and changing my URL structure. I am aiming to save money but I'm not doing this as a "spend basically nothing" project. Yes, I could convert my site to a static HTML generated blog using any number of great static site generators, or even a Headless CMS. Yes I could host it in Azure Storage fronted by a CMS, or even as a series of Azure Functions. But I have 17 years of content in DasBlog, I like DasBlog, and it's being actively updated to .NET Core and it's a fun app. I also have custom Razor sites in the form of my podcast site and they work great with a great workflow. I want to find a balance of cost effectiveness, features, ease of use, and reliability.  What I have now is a sinking feeling like my site is gonna die tomorrow and I'm not ready to deal with it. So, there you go.

Currently my sites live on a real machine with real folders and it's fronted by IIS on a Windows Server. There's an app (an IIS Application, to be clear) leaving at \ so that means hanselman.com/ hits / which is likely c:\inetpub\wwwroot full stop.

For historical reasons, when you hit hanselman.com/blog/ you're hitting the /blog IIS Application which could be at d:\whatever but may be at c:\inetpub\wwwroot\blog or even at c:\blog. Who knows. The Application and ASP.NET within it knows that the site is at hanselman.com/blog.

That's important, since I may write a URL like ~/about when writing code. If I'm in the hanselman.com/blog app, then ~/about means hanselman.com/blog/about. If I write /about, that means hanselman.com/about. So the ~ is a shorthand for "starting at this App's base URL." This is great and useful and makes Link generation super easy, but it only works if your app knows what it's server-side base URL is.

To be clear, we are talking about the reality of the generated URL that's sent to and from the browser, not about any physical reality on the disk or server or app.

I've moved my world to three Azure App Services called hanselminutes, hanselman, and hanselmanblog. They have names like http://hanselman.azurewebsites.net for example.

ASIDE: You'll note that hitting hanselman.azurewebsites.com will hit an app that looks stopped. I don't want that site to serve traffic from there, I want it to be served from http://hanselman.com, right? Specifically only from Azure Front Door which I'll talk about in another post soon. So I'll use the Access Restrictions and Software Based Networking in Azure to deny all traffic to that site, except traffic from Azure - in this case, from the Azure Front Door Reverse Proxy I'll be using.

That looks like this in this Access Restrictions part of the Azure Portal.

Only allowing traffic from Azure

Since the hanselman.com app will point to hanselman.azurewebsites.net (or one of its staging slots) there's no issue with URL generation. If I say / I mean /, the root of the site. If I generate a URL like "~/about" I'll get hanselman.com/about, right?

But with http://hanselmanblog.azurewebsites.net it's different.

I want hanselman.com/blog/ to point to hanselmanblog.azurewebsites.net.

That means that the Azure Front Door will be receiving traffic, then forward it on to the Azure Web App. That means:

  • hanselman.com/blog/foo -> hanselmanblog.azurewebsites.net/foo
  • hanselman.com/blog/bar -> hanselmanblog.azurewebsites.net/foo
  • hanselman.com/blog/foo/bar/baz -> hanselmanblog.azurewebsites.net/foo/bar/baz

There's a few things to consider when dealing with reverse proxies like this.

Is part of the /path being removed or is a path being added?

In the case of DasBlog, we have a configuration setting so that the app knows where it LOOKS like it is, from the Browser URL's perspective.

My blog is at /blog so I add that in some middleware in my Startup.cs. Certainly YOU don't need to have this in config - do whatever works for you as long as context.Request.PathBase is set as the app should see it. I set this very early in my pipeline.

That if statement is there because most folks don't install their blog at /blog, so it doesn't add the middleware.

//if you've configured it at /blog or /whatever, set that pathbase so ~ will generate correctly
Uri rootUri = new Uri(dasBlogSettings.SiteConfiguration.Root);
string path = rootUri.AbsolutePath;

//Deal with path base and proxies that change the request path
if (path != "/")
{
app.Use((context, next) =>
{
context.Request.PathBase = new PathString(path);
return next.Invoke();
});
}

Sometimes you want the OPPOSITE of this. That would mean that I wanted, perhaps hanselman.com to point to hanselman.azurewebsites.net/blog/. In that case I'd do this in my Startup.cs's ConfigureServices:

app.UsePathBase("/blog");

Be aware that If you're hosting ASP.NET Core apps behind Nginx or Apache or really anything, you'll also want ASP.NET Core to respect  X-Forwarded-For and other X-Forwarded standard headers. You'll also likely want the app to refuse to speak to anyone who isn't a certain list of proxies or configured URLs.

I configure these in Startup.cs's ConfigureServices from a semicolon delimited list in my config, but you can do this in a number of ways.

services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders = ForwardedHeaders.All;
options.AllowedHosts = Configuration.GetValue<string>("AllowedHosts")?.Split(';').ToList<string>();
});

Since Azure Front Door adds these headers as it forwards traffic, from my app's point of view it "just works" once I've added that above and then this in Configure()

app.UseForwardedHeaders();

There seems to be some confusion on hosting behind a reverse proxy in a few GitHub Issues. I'd like to see my scenario ( /foo -> / ) be a single line of code, as we see that the other scenario ( / -> /foo ) is a single line.

Have you had any issues with URL generation when hosting your Apps behind a reverse proxy?


Sponsor: Develop Xamarin applications without difficulty with the latest JetBrains Rider: Xcode integration, JetBrains Xamarin SDK, and manage the required SDKs for Android development, all right from the IDE. Get it today

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Real World Cloud Migrations: CDNs are an easy improvement to legacy apps

July 4, '19 Comments [5] Posted in Azure
Sponsored By

I'm doing a quiet backend migration/update to my family of sites. If I do it right, there will be minimal disruption. Even though I'm a one person show (plus Mandy my podcast editor) the Cloud lets me manage digital assets like I'm a whole company with an IT department. Sure, I could FTP files directly into production, but why do that when I've got free/cheap stuff like Azure DevOps.

As I'm one person, I do want to keep costs down whenever possible and I've said so in my "Penny Pinching in the Cloud" series. However, I do pay for value, so I'll try to keep costs down whenever possible, but I will pay for things I feel are useful or provide me with a quality product and/or save me time and hassle. For example, Azure Pipelines gives one free Microsoft-hosted CI/CD pipeline and 1 hosted job with 1800 minutes a month. This should be more than enough for my set up.

Additionally, sometimes I'll shift costs in order to both save money and improve performance as when I moved my Podcast's image hosting over to an Azure CDN in 2013. Fast forward 6 years and since I'm doing this larger migration, I wanted to see how easy it would be to migrate even more of my static assets to a CDN.

Make a Storage Account and ensure that its Access Level is Public if you intend folks to be able to read from it over HTTP GETs.

Public Access Level

With in the Azure Portal you can make a CDN Profile that uses Microsoft, Akamai, or Verizon for the actual Content Distribution Network. I picked Microsoft's because I have experience with Akamai and Verizon (they were solid) and I wanted to see how the new Microsoft one does. It claims to put content within 50ms of 60 countries.

You'll have a CDN endpoint host name that points to an Origin. That Origin is the Origin of your content. The Origin can be an existing place you have stuff so the CDN will basically check there first, cache things, then serve the content. Or it can be an existing WebApp or in my case, Storage.

Azure Storage Accounts for my blog

I didn't want to make things too complex, so I have a single Storage Account with a few containers. Then I mapped custom endpoints for both my blog AND my podcast, but they share the same storage. I also took advantage of the free SSL Certs so images.hanselman.com and images.hanselminutes.com are both SSL. Gotta get those "A" grades from https://securityheaders.com right?

Custom domains for my CDN

Then, just grab the Azure Storage Explorer. It's free and cross platform. In fact, you can get Storage Explorer as an app that runs locally, or you can check it out in the Azure  Portal/in-browser as well, here. I've uploaded all my main assets (images used in my CSS, blog, headers, etc).

I've settled on a basic scheme where anything that was "/images/foo.png" is now "https://images.hanselman.com/blog/foo.png" or /main/ or /podcast/ depending on who is using it. This made search and replaces reliable and easy.

It's truly a less-than-an-hour operation to enable a CDN on an existing site. While I've chose to use Azure Storage as my backing store, you can just use your existing site's /images folder and just change your markup to pull from the CDN's URL. I recommend you make a simple cdn.yourdomain.com or images.yourdomain.com. This can also easily be enabled with a CNAME in less than an hour. Having your images at another URL via a subdomain CNAME also allows for even more download parallelism from the browser.

This is all in my Staging so you won't see it quite yet until I flip the switch on the whole migration.

Exploring future possibilities for dynamic image content

I haven't yet moved all my blog content images (a few gigs, but many thousands of images) to a CDN as I don't want to change my existing publishing workflow with Open Live Writer. I'm considering a flow that keeps the images uploading to the Web App but then using the Custom Origin Path options in the Azure CDN that will have images get picked up Web App then get served by the CDN. In order to manage 17 years of images though, I'd need to catch URLs like this:

https://www.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

and redirect them to

https://images.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

As I see it, there's a few options to get images from a GET of /blog/content/binary to be served by a CDN:

  • Dynamically Rewrite the URLs on the way OUT (as the HTML is generated)
    • CPU expensive, but ya it's cached in my WebApp
  • Rewriting Middleware to redirect (301?) requests to the new images location
    • Easiest option, but costs everyone a 301 on all image GETs
  • Programmatically change the stored markup (basically forloop over my "database," search and replace, AND ensure future images use this new URL)
    • A hassle, but a one time hassle
    • Not sure about future images, I might have to change my publishing flow AND run the process on new posts occasionally.

What are your thoughts on if I should go all the way and manage EVERY blog image vs the low hanging fruit of shared static assets? Worth it, or overkill?

The learning process continues!


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Real World Cloud Migrations: Moving a 17 year old series of sites from bare metal to Azure

July 2, '19 Comments [7] Posted in ASP.NET | Azure | DasBlog
Sponsored By

Technical Debt has a way of sneaking up on you. While my podcast site and the other 16ish sites I run all live in Azure and have a nice CI/CD pipeline with Azure DevOps, my main "Hanselman.com" series of sites and mini-sites has lagged behind. I'm still happy with its responsive design, but the underlying tech has started to get more difficult to manage and build and I've decided it's time to make some updates.

Moving sites to Azure DevOps

I want to be able to make these updates and have a clean switch over so that you, the reader, don't notice a difference. There's a number of things to think about when doing any migration like this, realizing it'll take some weeks (or months if you're a bigger company that just me).

  • Continuous Deployment/Continuous Integration
    • I host my code on GitHub and Azure DevOps now lets you log in with GitHub and does a fine job of building AND deploying your code (while running tests AND allowing for manual quality gates) so I want to make sure my sites have a nice clean "check in and go live" process.
    • I'll also be using Azure App Services and Deployment Slots, so I'll have a dev/test/staging site and production, like a real professional. No more editing text files in production. Well, at least, I won't tell you when I'm editing text file in production.
  • Technology Update
    • Hanselman.com proper (not the blog) and the mini pages/sites underneath it run on ASP.NET 4.0 and WebForms. I was able to easily move the main site over to ASP.NET Razor Pages. Razor is just so elegant, as it's basically just HTML then you type @ and you're in C# (Razor). More on that below, but the upgrade was a day as the home page and minisites are largely readonly.
    • The Blog, hosted at /blog will be more challenging given I don't want to break two decades years of URLs, along with the fact that it's running DasBlog on a recently upgraded .NET 4.0. DasBlog was originally made in .NET 1, then upgraded to .NET 2, so this is 17 years of technical debt.
    • That said, the .NET Standard along with open source cross-platform .NET Core has allowed us - with the leadership of Mark Downie - to create DasBlog Core. DasBlog Core shares the core reliable (if crusty) engine of DasBlog along with an all new system of URL writing using ASP.NET Core middleware, as well as a complete re-do of the (well ahead of its time) DasBlog Theming Engine, now based on Razor Pages. It's brilliant. This is in active development.
  • Azure Front Door
    • Because I'm moving from a single machine running IIS to Azure, I'll want to split things apart to remove single points of failture. I'll use Azure Front Door to manage my URL structure and act as a front end cache as well as distribute traffic to multiple Azure App Services (Web Apps).
  • URL management
    • Are you changing your URLs and URL structure? Remember that URLs are UI and they matter. I've long wanted to remove the "aspx" extension from my URLs, as well as move the TitleCaseBlogPostThing to a more "modern" title-case-blog-post-thing style. I need to do this in a way that updates my google sitemap, breaks zero URLs, 301 redirects to the new style, and uses rel=canonical in a smart way.
  • Shared Assets/CDNs/Front Door
    • Since I run a family of sites, there's an opportunity to use a CDN as well and some clean CNAME DNS such that images.hanselman.com and images.hanselminutes.com can share assets. Since the Azure CDN is easy to setup and offers free SSL certs and pay-as-you go, I'll set both of those CNAMES up to point to the same Azure Storage where I'll keep images, show pics, CSS, and JS.

I'll be blogging the whole process. What do you want to hear/learn about?


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Git is case-sensitive and your filesystem may not be - Weird folder merging on Windows

June 27, '19 Comments [7] Posted in DasBlog | Open Source
Sponsored By

I was working on DasBlog Core (an .NET Core cross-platform update of the ASP.NET WebForms-based blogging software that runs this blog) with Mark Downie, the new project manager, and Shayne Boyer. This is part of a larger cloud re-architecture of hanselman.com and the systems that run this whole site.

Shayne was working on getting a DasBlog Core CI/CD (Continuous Integration/Continuous Development) running in Azure DevOps' build system. We wanted individual build pipelines to confirm that DasBlog Core was in fact, cross-platform, so we needed to build, test, and run it on Windows, Linux, and Mac.

The build was working great on Windows and Mac...but failing on Linux. Why?

Well, like all things, it's complex.

  • Windows has a case-insensitive file system.
  • By default, Mac uses a case-insensitive file system.

Since Git 1.5ish there's been a setting

git config --global core.ignorecase true

but you should always be aware of what a setting does before you just set it.

If you're not careful, you or someone on your team can create a case sensitive file path in your git index while you're using a case insensitive operating system like Windows or Mac. If you do this, you'll be able to end up with two separate entries from git's perspective. However Windows will silently merge them and see just one.

Here's our themes folder structure as seen on GitHub.com.

Case insenstive folder names

But when we clone it on Mac or Windows, we see just one folder.

DasBlog as a single folder in VS Code

Turns out that six months ago one of us introduced another folder with the name dasblog while the original was DasBlog. When we checked them on Mac or Windows the files ended up in merged into one folder, but on Linux they were/are two, so the build fails.

You can fix this in a few ways. You can rename the file in a case-sensitive way and commit the change:

git mv --cached name.txt NAME.TXT

Please take care and back up anything you don't understand.

If you're renaming a directory, you'll do a two stage rename with a temp name.

git mv foo foo2
git mv foo2 FOO
git commit -m "changed case of dir"

Be safe out there!


Sponsor: Looking for a tool for performance profiling, unit test coverage, and continuous testing that works cross-platform on Windows, macOS, and Linux? Check out the latest JetBrains Rider!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Adding Reaction Gifs for your Build System and the Windows Terminal

June 25, '19 Comments [6] Posted in Open Source | PowerShell | Win10
Sponsored By

So, first, I'm having entirely too much fun with the new open source Windows Terminal. If you've got the latest version of Windows (go run Windows Update and do whatever it takes) then you can download the Windows Terminal from the Microsoft Store! This is a preview release (think v0.2) but it'll automatically update, often, from the Windows Store if you have Windows 10 version 18362.0 or higher.

One of the most fun things is that you can have background images. Even animated gifs! You can add those images in your Settings/profile.json like this.

"backgroundImage": "c:/users/scott/desktop/doug.gif",
"backgroundImageOpacity": 0.7,
"backgroundImageStretchMode": "uniformToFill

The profile.json is just JSON and you can update it. I could even update it programmatically if I wanted to parse it and mess about.

BUT. Enterprising developer Chris Duck created a lovely PowerShell Module called MSTerminalSettings that lets you very easily make Profile changes with script.

For example, Mac developers who use iTerm often go to https://iterm2colorschemes.com/ and get new color schemes for their consoles. Now Windows folks can as well!

From his docs, this example downloads the Pandora color scheme from https://iterm2colorschemes.com/ and sets it as the color scheme for the PowerShell Core terminal profile.

Invoke-RestMethod -Uri 'https://raw.githubusercontent.com/mbadolato/iTerm2-Color-Schemes/master/schemes/Pandora.itermcolors' -OutFile .\Pandora.itermcolors
Import-Iterm2ColorScheme -Path .\Pandora.itermcolors -Name Pandora
Get-MSTerminalProfile -Name "PowerShell Core" | Set-MSTerminalProfile -ColorScheme Pandora

That's easy! Then I was talking to Tyler Leonhardt and suggested that we programmatically change the background using a folder full of Animated Gifs. I happen to have such a folder (with 2000 categorized gif classics) so we started coding and streamed the whole debacle on Tyler's Twitch!

The result is Windows Terminal Attract Mode and it's a hot mess and it is up on GitHub and all set up for PowerShell Core.

Remember that "Attract mode" is the mode an idle arcade cabinet goes into in order to attract passersby to play, so clearly the Terminal needs this also.

./AttractMode.ps1 -name "profile name" -path "c:\temp\trouble" -secs 5

It's a proof of concept for now, and it's missing background/runspace support, being wrapped up in a proper module, etc but the idea is solid, building on a solid base, with improvements to idiomatic PowerShell Core already incoming. Right now it'll run forever. Wrap it in Start-Job if you like as well and can stand it.

The next idea was to have reactions gifs to different developer situations. Break the build? Reaction Gif. Passing tests? Reaction Gif.

Here's a silly proof (not refactored) that aliases "dotnet build" to "db" with reactions.

#messing around with build reaction gifs

Function DotNetAlias {
dotnet build
if ($?) {
Start-job -ScriptBlock {
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\chrispratt.gif"
Start-Sleep 1.5
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\4003cn5.gif"
} | Out-Null
}
else {
Start-job -ScriptBlock {
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\idk-girl.gif"
Start-Sleep 1.5
d:\github\TerminalAttractMode\SetMoodGif.ps1 "PowerShell Core" "D:\Dropbox\Reference\Animated Gifs\4003cn5.gif"
} | Out-Null

}
}

Set-Alias -Name db -value DotNetAlias

I added the Start-job stuff so that the build finishes and the Terminal returns control to you while the gifs still are updating. Runspace support would be smart as well.

Some other ideas? Giphy support. Random mood gifs. Pick me ups. You get the idea.

Later, Brandon Olin jumped in with this gem. Why not get a reaction gif if anything goes wrong in your last command? ERRORLEVEL 1? Explode.

Why are we doing this? Because it sparks joy, y'all.


Sponsor: Looking for a tool for performance profiling, unit test coverage, and continuous testing that works cross-platform on Windows, macOS, and Linux? Check out the latest JetBrains Rider!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.