Scott Hanselman

Dealing with Application Base URLs and Razor link generation while hosting ASP.NET web apps behind Reverse Proxies

July 9, '19 Comments [4] Posted in ASP.NET | Azure | DasBlog | Open Source
Sponsored By

Updating my site to run on AzureI'm quietly moving my Website from a physical machine to a number of Cloud Services hosted in Azure. This is an attempt to not just modernize the system - no reason to change things just to change them - but to take advantage of a number of benefits that a straight web host sometimes doesn't have. I want to have multiple microsites (the main page, the podcast, the blog, etc) with regular backups, CI/CD pipeline (check in code, go straight to staging), production swaps, a global CDN for content, etc.

I'm also moving from an ASP.NET 4 (was ASP.NET 2 until recently) site to ASP.NET Core 2.x LTS and changing my URL structure. I am aiming to save money but I'm not doing this as a "spend basically nothing" project. Yes, I could convert my site to a static HTML generated blog using any number of great static site generators, or even a Headless CMS. Yes I could host it in Azure Storage fronted by a CMS, or even as a series of Azure Functions. But I have 17 years of content in DasBlog, I like DasBlog, and it's being actively updated to .NET Core and it's a fun app. I also have custom Razor sites in the form of my podcast site and they work great with a great workflow. I want to find a balance of cost effectiveness, features, ease of use, and reliability.  What I have now is a sinking feeling like my site is gonna die tomorrow and I'm not ready to deal with it. So, there you go.

Currently my sites live on a real machine with real folders and it's fronted by IIS on a Windows Server. There's an app (an IIS Application, to be clear) leaving at \ so that means hanselman.com/ hits / which is likely c:\inetpub\wwwroot full stop.

For historical reasons, when you hit hanselman.com/blog/ you're hitting the /blog IIS Application which could be at d:\whatever but may be at c:\inetpub\wwwroot\blog or even at c:\blog. Who knows. The Application and ASP.NET within it knows that the site is at hanselman.com/blog.

That's important, since I may write a URL like ~/about when writing code. If I'm in the hanselman.com/blog app, then ~/about means hanselman.com/blog/about. If I write /about, that means hanselman.com/about. So the ~ is a shorthand for "starting at this App's base URL." This is great and useful and makes Link generation super easy, but it only works if your app knows what it's server-side base URL is.

To be clear, we are talking about the reality of the generated URL that's sent to and from the browser, not about any physical reality on the disk or server or app.

I've moved my world to three Azure App Services called hanselminutes, hanselman, and hanselmanblog. They have names like http://hanselman.azurewebsites.net for example.

ASIDE: You'll note that hitting hanselman.azurewebsites.com will hit an app that looks stopped. I don't want that site to serve traffic from there, I want it to be served from http://hanselman.com, right? Specifically only from Azure Front Door which I'll talk about in another post soon. So I'll use the Access Restrictions and Software Based Networking in Azure to deny all traffic to that site, except traffic from Azure - in this case, from the Azure Front Door Reverse Proxy I'll be using.

That looks like this in this Access Restrictions part of the Azure Portal.

Only allowing traffic from Azure

Since the hanselman.com app will point to hanselman.azurewebsites.net (or one of its staging slots) there's no issue with URL generation. If I say / I mean /, the root of the site. If I generate a URL like "~/about" I'll get hanselman.com/about, right?

But with http://hanselmanblog.azurewebsites.net it's different.

I want hanselman.com/blog/ to point to hanselmanblog.azurewebsites.net.

That means that the Azure Front Door will be receiving traffic, then forward it on to the Azure Web App. That means:

  • hanselman.com/blog/foo -> hanselmanblog.azurewebsites.net/foo
  • hanselman.com/blog/bar -> hanselmanblog.azurewebsites.net/foo
  • hanselman.com/blog/foo/bar/baz -> hanselmanblog.azurewebsites.net/foo/bar/baz

There's a few things to consider when dealing with reverse proxies like this.

Is part of the /path being removed or is a path being added?

In the case of DasBlog, we have a configuration setting so that the app knows where it LOOKS like it is, from the Browser URL's perspective.

My blog is at /blog so I add that in some middleware in my Startup.cs. Certainly YOU don't need to have this in config - do whatever works for you as long as context.Request.PathBase is set as the app should see it. I set this very early in my pipeline.

That if statement is there because most folks don't install their blog at /blog, so it doesn't add the middleware.

//if you've configured it at /blog or /whatever, set that pathbase so ~ will generate correctly
Uri rootUri = new Uri(dasBlogSettings.SiteConfiguration.Root);
string path = rootUri.AbsolutePath;

//Deal with path base and proxies that change the request path
if (path != "/")
{
app.Use((context, next) =>
{
context.Request.PathBase = new PathString(path);
return next.Invoke();
});
}

Sometimes you want the OPPOSITE of this. That would mean that I wanted, perhaps hanselman.com to point to hanselman.azurewebsites.net/blog/. In that case I'd do this in my Startup.cs's ConfigureServices:

app.UsePathBase("/blog");

Be aware that If you're hosting ASP.NET Core apps behind Nginx or Apache or really anything, you'll also want ASP.NET Core to respect  X-Forwarded-For and other X-Forwarded standard headers. You'll also likely want the app to refuse to speak to anyone who isn't a certain list of proxies or configured URLs.

I configure these in Startup.cs's ConfigureServices from a semicolon delimited list in my config, but you can do this in a number of ways.

services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders = ForwardedHeaders.All;
options.AllowedHosts = Configuration.GetValue<string>("AllowedHosts")?.Split(';').ToList<string>();
});

Since Azure Front Door adds these headers as it forwards traffic, from my app's point of view it "just works" once I've added that above and then this in Configure()

app.UseForwardedHeaders();

There seems to be some confusion on hosting behind a reverse proxy in a few GitHub Issues. I'd like to see my scenario ( /foo -> / ) be a single line of code, as we see that the other scenario ( / -> /foo ) is a single line.

Have you had any issues with URL generation when hosting your Apps behind a reverse proxy?


Sponsor: Develop Xamarin applications without difficulty with the latest JetBrains Rider: Xcode integration, JetBrains Xamarin SDK, and manage the required SDKs for Android development, all right from the IDE. Get it today

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Real World Cloud Migrations: CDNs are an easy improvement to legacy apps

July 4, '19 Comments [4] Posted in Azure
Sponsored By

I'm doing a quiet backend migration/update to my family of sites. If I do it right, there will be minimal disruption. Even though I'm a one person show (plus Mandy my podcast editor) the Cloud lets me manage digital assets like I'm a whole company with an IT department. Sure, I could FTP files directly into production, but why do that when I've got free/cheap stuff like Azure DevOps.

As I'm one person, I do want to keep costs down whenever possible and I've said so in my "Penny Pinching in the Cloud" series. However, I do pay for value, so I'll try to keep costs down whenever possible, but I will pay for things I feel are useful or provide me with a quality product and/or save me time and hassle. For example, Azure Pipelines gives one free Microsoft-hosted CI/CD pipeline and 1 hosted job with 1800 minutes a month. This should be more than enough for my set up.

Additionally, sometimes I'll shift costs in order to both save money and improve performance as when I moved my Podcast's image hosting over to an Azure CDN in 2013. Fast forward 6 years and since I'm doing this larger migration, I wanted to see how easy it would be to migrate even more of my static assets to a CDN.

Make a Storage Account and ensure that its Access Level is Public if you intend folks to be able to read from it over HTTP GETs.

Public Access Level

With in the Azure Portal you can make a CDN Profile that uses Microsoft, Akamai, or Verizon for the actual Content Distribution Network. I picked Microsoft's because I have experience with Akamai and Verizon (they were solid) and I wanted to see how the new Microsoft one does. It claims to put content within 50ms of 60 countries.

You'll have a CDN endpoint host name that points to an Origin. That Origin is the Origin of your content. The Origin can be an existing place you have stuff so the CDN will basically check there first, cache things, then serve the content. Or it can be an existing WebApp or in my case, Storage.

Azure Storage Accounts for my blog

I didn't want to make things too complex, so I have a single Storage Account with a few containers. Then I mapped custom endpoints for both my blog AND my podcast, but they share the same storage. I also took advantage of the free SSL Certs so images.hanselman.com and images.hanselminutes.com are both SSL. Gotta get those "A" grades from https://securityheaders.com right?

Custom domains for my CDN

Then, just grab the Azure Storage Explorer. It's free and cross platform. In fact, you can get Storage Explorer as an app that runs locally, or you can check it out in the Azure  Portal/in-browser as well, here. I've uploaded all my main assets (images used in my CSS, blog, headers, etc).

I've settled on a basic scheme where anything that was "/images/foo.png" is now "https://images.hanselman.com/blog/foo.png" or /main/ or /podcast/ depending on who is using it. This made search and replaces reliable and easy.

It's truly a less-than-an-hour operation to enable a CDN on an existing site. While I've chose to use Azure Storage as my backing store, you can just use your existing site's /images folder and just change your markup to pull from the CDN's URL. I recommend you make a simple cdn.yourdomain.com or images.yourdomain.com. This can also easily be enabled with a CNAME in less than an hour. Having your images at another URL via a subdomain CNAME also allows for even more download parallelism from the browser.

This is all in my Staging so you won't see it quite yet until I flip the switch on the whole migration.

Exploring future possibilities for dynamic image content

I haven't yet moved all my blog content images (a few gigs, but many thousands of images) to a CDN as I don't want to change my existing publishing workflow with Open Live Writer. I'm considering a flow that keeps the images uploading to the Web App but then using the Custom Origin Path options in the Azure CDN that will have images get picked up Web App then get served by the CDN. In order to manage 17 years of images though, I'd need to catch URLs like this:

https://www.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

and redirect them to

https://images.hanselman.com/blog/content/binary/Windows-Live-Writer/0409a9d5fae6_F552/image_9.png

As I see it, there's a few options to get images from a GET of /blog/content/binary to be served by a CDN:

  • Dynamically Rewrite the URLs on the way OUT (as the HTML is generated)
    • CPU expensive, but ya it's cached in my WebApp
  • Rewriting Middleware to redirect (301?) requests to the new images location
    • Easiest option, but costs everyone a 301 on all image GETs
  • Programmatically change the stored markup (basically forloop over my "database," search and replace, AND ensure future images use this new URL)
    • A hassle, but a one time hassle
    • Not sure about future images, I might have to change my publishing flow AND run the process on new posts occasionally.

What are your thoughts on if I should go all the way and manage EVERY blog image vs the low hanging fruit of shared static assets? Worth it, or overkill?

The learning process continues!


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Real World Cloud Migrations: Moving a 17 year old series of sites from bare metal to Azure

July 2, '19 Comments [6] Posted in ASP.NET | Azure | DasBlog
Sponsored By

Technical Debt has a way of sneaking up on you. While my podcast site and the other 16ish sites I run all live in Azure and have a nice CI/CD pipeline with Azure DevOps, my main "Hanselman.com" series of sites and mini-sites has lagged behind. I'm still happy with its responsive design, but the underlying tech has started to get more difficult to manage and build and I've decided it's time to make some updates.

Moving sites to Azure DevOps

I want to be able to make these updates and have a clean switch over so that you, the reader, don't notice a difference. There's a number of things to think about when doing any migration like this, realizing it'll take some weeks (or months if you're a bigger company that just me).

  • Continuous Deployment/Continuous Integration
    • I host my code on GitHub and Azure DevOps now lets you log in with GitHub and does a fine job of building AND deploying your code (while running tests AND allowing for manual quality gates) so I want to make sure my sites have a nice clean "check in and go live" process.
    • I'll also be using Azure App Services and Deployment Slots, so I'll have a dev/test/staging site and production, like a real professional. No more editing text files in production. Well, at least, I won't tell you when I'm editing text file in production.
  • Technology Update
    • Hanselman.com proper (not the blog) and the mini pages/sites underneath it run on ASP.NET 4.0 and WebForms. I was able to easily move the main site over to ASP.NET Razor Pages. Razor is just so elegant, as it's basically just HTML then you type @ and you're in C# (Razor). More on that below, but the upgrade was a day as the home page and minisites are largely readonly.
    • The Blog, hosted at /blog will be more challenging given I don't want to break two decades years of URLs, along with the fact that it's running DasBlog on a recently upgraded .NET 4.0. DasBlog was originally made in .NET 1, then upgraded to .NET 2, so this is 17 years of technical debt.
    • That said, the .NET Standard along with open source cross-platform .NET Core has allowed us - with the leadership of Mark Downie - to create DasBlog Core. DasBlog Core shares the core reliable (if crusty) engine of DasBlog along with an all new system of URL writing using ASP.NET Core middleware, as well as a complete re-do of the (well ahead of its time) DasBlog Theming Engine, now based on Razor Pages. It's brilliant. This is in active development.
  • Azure Front Door
    • Because I'm moving from a single machine running IIS to Azure, I'll want to split things apart to remove single points of failture. I'll use Azure Front Door to manage my URL structure and act as a front end cache as well as distribute traffic to multiple Azure App Services (Web Apps).
  • URL management
    • Are you changing your URLs and URL structure? Remember that URLs are UI and they matter. I've long wanted to remove the "aspx" extension from my URLs, as well as move the TitleCaseBlogPostThing to a more "modern" title-case-blog-post-thing style. I need to do this in a way that updates my google sitemap, breaks zero URLs, 301 redirects to the new style, and uses rel=canonical in a smart way.
  • Shared Assets/CDNs/Front Door
    • Since I run a family of sites, there's an opportunity to use a CDN as well and some clean CNAME DNS such that images.hanselman.com and images.hanselminutes.com can share assets. Since the Azure CDN is easy to setup and offers free SSL certs and pay-as-you go, I'll set both of those CNAMES up to point to the same Azure Storage where I'll keep images, show pics, CSS, and JS.

I'll be blogging the whole process. What do you want to hear/learn about?


Sponsor: Seq delivers the diagnostics, dashboarding, and alerting capabilities needed by modern development teams - all on your infrastructure. Download now.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

An experiment - The Azure Cloud Shell at the command line with AZ SHELL

May 7, '19 Comments [3] Posted in Azure | Open Source
Sponsored By

I've blogged before about the Azure Cloud Shell. It's super cool and you can get your own easily in any browse by hitting https://shell.azure.com. You can have either bash or powershell, and you get a shared "cloud drive" that is persisted between sessions.

If you have Visual Studio Code you can get an Azure Cloud Shell integrated within VSCode by just installing Visual Studio Code and adding the Azure Account Extension.

I recently got a build of the new open source Windows Terminal on my machine and I set up some profiles with tabs for DOS, PowerShell, VS2019, Ubuntu but something was missing. Why can't I get my Azure Cloud Shell?

Sure, I can fire up a VM and ssh into it. But Azure Cloud Shell spins up a free container with a persistent cloud drive AND has a bunch of developer tools like python, node, dotnet, and go already installed. I'd love to use it! But it's not a VM and the container isn't exposed with SSH. Instead, we'll want to spin the Azure Cloud Shell up the same way the https://shell.azure.com site does, with web calls and web sockets. So...why not do it?

image

I thought I was pretty clever when I had this idea so I started a C# implementation myself. Then I talked to Anders Liu from work about how to do it right, and over the weekend he beat me to it with his own VERY nice and clean implementation in Go that he put on his github at https://github.com/yangl900/azshell. We shared this on an internal alias and found out that Noel Bundick had the same great idea and put it in his Az CLI extensions pack (which has a ton of other cool stuff you should see). Anders' is standalone and Noel's is an Az CLI extension.

Either way, we all together think this idea has merit and maybe it should be an official thing! What do you think? Regardless maybe it doesn't need to be since you can try it today with these open source options.

Just put "azshell.exe" in your PATH and make sure you have the latest Azure CLI installed and you're logged in.

By the way, you can also get a Cloud Shell inside the Portal. In fact there's a button for it at the top that looks like >_ Personally I think with the addition of "az shell" (or in this case, azshell.exe) from the command line) it completes the circle in a really cool way.

image

Let me know what you think in the comments!


Sponsor: Manage GitHub Pull Requests right from the IDE with the latest JetBrains Rider. An integrated performance profiler on Windows comes to the rescue as well.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Did I leave the garage door open? A no-code project with Azure IoT Central and the MXChip DevKit

May 1, '19 Comments [10] Posted in Azure | Hardware
Sponsored By

Azure IoT DevKitFor whatever reason when a programmer tries something out for the first time, they write a "Hello World!" application. In the IoT (Internet of Things) world of devices, it's always fun to make an LED blink as a good getting started sample project.

When I'm trying out an IoT platform or tiny microcontroller I have my own "Hello World" project - I try to build a simple system that tells me "Did I leave the garage door open?"

I wanted to see how hard it would be to use an Azure IoT MXChip DevKit to build this little system. The DevKit is small and thin but includes Wifi, OLED display, headphone, microphone, sensors like temperature, humidity, motion, pressure sensors. The kit isn't super expensive given all it does and you can buy it most anywhere. The DevKit is also super easy to update and it's actively developed. In fact, I just updated mine to Firmware 1.6.2 yesterday and there is an Azure IoT Device Workbench Extension for VS Code. There is also a fantastic IoT DevKit Project Catalog you should check out.

I wanted to use this little Arduino friendly device and have it talk to Azure. My goal was to see how quickly and simply I could make a solution that would:

  • Detect if my garage door is open
  • If it's open for more than 4 minutes, text me
  • Later, perhaps I'll figure out how to reply to the Text or take an action to close the door remotely.

However, there is an Azure IoT Hub and there's Azure IoT Central and this was initially confusing to me. It seems that Azure IoT Hub is a individual Azure service but it's not an end-to-end IoT solution - it's a tool in the toolbox. Azure IoT Central, on the other hand, is an browser-based system with templates that is a SaaS (Software as a Service) and hides most of the underlying systems. With IoT Central no coding is needed!

Slick. I was fully prepared to write Arduino code to get this garage door sensor working but if I can do it with no code, rock on. I may finish this before lunch is over. I have an Azure account so I went to https://azureiotcentral.com and created a new Application. I chose Pay as You Go but it's free for the first 5 devices so, swag.

Create a New Azure IoT Central App

You should totally check this out even if you don't have an IoT DevKit because you likely DO have a Raspberry Pi and it totally has device templates for Pis or even Windows 10 IoT Core Devices.

Azure IoT Central

Updating the firmware for the IoT DevKit couldn't be easier. You plug it into a free USB port, it shows up as a disk drive, and you drag in the new (or alternate) firmware. If you're doing something in production you'll likely want to do OTA (Over-the-air) firmware updates with Azure IoT Hub automatic device management, so it's good to know that's also an option. The default DevKit firmware is fun to explore but I am connecting this device to Azure (and my Wifi) so I used the firmware and instructions from here which is firmware specific to Azure IoT Central.

The device reboots as a temporary hotspot (very clever) and then you can connect to it's wifi, and then it'll connect to yours over WPA2. Once you're connected to Wifi, you can add a new Real (or Simulated - you can actually do everything I'm doing where without a real device!) device using a Device ID that you'll pair with your Mxchip IoT DevKit. After it's connect you'll see tons of telemetry pour into Azure. You can, of course, choose what you want to send and send just the least amount your projects needs, but it's still a very cool first experience to see temp, humidity, and on and on from this little device.

MxChip in Azure

Here's a wonderful HIGH QUALITY diagram of my Garage door planned system. You only wish your specifications were this sophisticated. ;)

Basically the idea is that when the door is closed I'll have the IoT DevKit taped to the door with a battery, then when it open it'll rotate 90 degrees and the Z axis of the Accelerometer will change! If it stays there for more than 5 minutes then it should text me!

image

In Azure IoT central I made a Device Template with a Telemetry Rule that listens to the changes in the accelerometer Z and when the average is less than 900 (I figured this number out by moving it around and testing) then it fires an Action.

The "Action" is using an Azure Monitor action group that can either SMS or even call me voice!

In this chart when the accelerometer is above the line the garage door is closed and when it drops below the line it's open!

The gyroscope Z changing with time

Here's the Azure Monitoring alert that texts me when I leave the garage door open too long.

Azure Activity Monitor

And here's my alert SMS!

mxchip

I was very impressed I didn't have to write any code to pull this off. I'm going to try this same "Hello World" later with custom code using a AdaFruit Huzzah Feather and an ADXL345 Accelerometer. I'll write Arduino C code and still have it talk to Azure for Alerts.

It's amazing how clean and simple the building blocks are for projects like this today.


Sponsor: Manage GitHub Pull Requests right from the IDE with the latest JetBrains Rider. An integrated performance profiler on Windows comes to the rescue as well.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb
Previous Page Page 2 of 14 in the Azure category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.