Scott Hanselman

Azure DevOps Continuous Build/Deploy/Test with ASP.NET Core 2.2 Preview in One Hour

September 19, 2018 Comment on this post [10] Posted in Azure | Open Source
Sponsored By

Hanselminutes WebsiteI've been doing Continuous Integration and Deployment for well over 13 years. We used a lot of custom scripts and a lovely tool called CruiseControl.NET to check out, build, test, and deploy our code.

However, it's easy to get lulled into complacency. To get lazy. I don't set up Automated Continuous Integration and Deployment for all my little projects. But I should.

I was manually deploying a change to my podcast website this evening via a git deploy to Azure App Service. Pushing to Azure this way via Git uses "Kudu" to actually build the site. However, earlier this week I was also trying to update my site to .NET Core 2.2 which is in preview. Plus I have Unit Tests that aren't getting run during deploy.

So look at it this way. My simple little podcast website with a few tests and the desire to use a preview .NET Core SDK means I've outgrown a basic "git push to prod" for deploy.

I remembered that Azure DevOps (formerly VSTS) is out and offers free unlimited minutes for open source projects. I have no excuse for my sloppy builds and manual deploys. It also has unlimited free private repos, although I'm happy at GitHub and have no reason to move.

It usually takes me 5-10 minutes for a manual build/test/deploy, so I gave myself an hour to see if I could get this same process automated in Azure DevOps. I've never used this before and I wanted to see if I could do it quickly, and if it was intuitive.

Let's review my goals.

  • My source is in GitHub
  • Build my ASP.NET Core 2.2 Web Site
    • I want to build with .NET Core 2.2 which is currently in Preview.
  • Run my xUnit Unit Tests
    • I have some Selenium Unit Tests that can't run in the cloud (at least, I haven't figured it out yet) so I need them skipped.
  • Deploy the resulting site to product in my Azure App Service

Cool. So I make a project and point Azure DevOps at my GitHub.

Azure DevOps: Source code in GitHub

They have a number of starter templates, so I was pleasantly surprised I didn't need manually build my Build Configuration myself. I'll pick ASP.NET app. I could pick Azure Web App for ASP.NET but I wanted a little more control.

Select a template

Now I've got a basic build pipeline. You can see it will use NuGet, get the packages, build the app, test the assemblies (if there are tests...more on that later) and the publish (zip) the build artifacts.

Build Pipeline

I then clicked Save & Queue...and it failed. Why? It says that I'm targeting .NET Core 2.2 and it doesn't support anything over 2.1. Shoot.

Agent says it doesn't support .NET Core 2.2

Fortunately there's a pipeline element that I can add called ".NET Core Tool Installer" that will get specific versions of the .NET Core SDK.

NOTE: I've emailed the team that ".NET Tool Installer" is the wrong name. A .NET Tool is a totally different thing. This task should be called the ".NET Core SDK Installer." Because it wasn't, it took me a minute to find it and figure out what it does.

I'm using the SDK Agent version so I put that string into the properties.

Install the .NET Core SDK custom version

At this point it builds, but I get a test error.

There's two problems with the tests. When I look at the logs I can see that the "testadapter.dll" that comes with xunit is mistakenly being pulled into the test runner! Why? Because the "Test Files" spec includes a VERY greedy glob in the form of **\*test*.dll. Perhaps testadapter shouldn't include the word test, but then it wouldn't be well-named.


My test DLLs are all named with "tests" in the filename so I'll change the glob to "**\$(BuildConfiguration)\**\*tests*.dll" to cast a less-wide net.

Screenshot (45)

I have four Selenium Tests for my ASP.NET Core site but I don't want them to run when the tests are run in a Docker Container or, in this case, in the Cloud. (Until I figure out how)

I use SkippableFacts from XUnit and do this:

public static class AreWe
public static bool InDockerOrBuildServer {
get {
string retVal = Environment.GetEnvironmentVariable("DOTNET_RUNNING_IN_CONTAINER");
string retVal2 = Environment.GetEnvironmentVariable("AGENT_NAME");
return (
(String.Compare(retVal, Boolean.TrueString, ignoreCase: true) == 0)
(String.IsNullOrWhiteSpace(retVal2) == false));

Don't tease me. I like it. Now I can skip tests that I don't want running.

if (AreWe.InDockerOrBuildServer) return;

Now my tests run and I get a nice series of charts to show that fact.

22 tests, 4 skipped

I have it building and tests running.

I could add the Deployment Step to the Build but Azure DevOps Pipelines includes a better way. I make a Release Pipeline that is separate. It takes Artifacts as input and runs n number of Stages.

Creating a new Release Pipeline

I take the Artifact from the Build (the zipped up binaries) and pass them through the pipeline into the Azure App Service Deploy step.

Screenshot (49)

Here's the deployment in progress.

Manually Triggered Release

Cool! Now that it works and deploys, I can turn on Continuous Integration Build Triggers (via an automatic GitHub webhook) as well as Continuous Deployment triggers.

Continuous Deployment

Azure DevOps even includes badges that I can add to my so I always know by looking at GitHub if my site builds AND if it has successfully deployed.

4 releases, the final one succeeded

Now I can see each release as it happens and if it's successful or not.

Build Succeeded, Never Deployed

To top it all off, now that I have all this data and these pipelines, I even put together a nice little dashboard in about a minute to show Deployment Status and Test Trends.

My build and deployment dashboard

When I combine the DevOps Dashboard with my main Azure Dashboard I'm amazed at how much information I can get in so little effort. Consider that my podcast (my little business) is a one-person shop.

Azure Dashboard

And now I have a CI/CD pipeline with integrated testing gates that deploys worldwide. Many years ago this would have required a team and a lot of custom code.

Today it took an hour. Awesome.

I check into GitHub, kicks off a build, tests, emails me the results, and deploys the website if everything is cool. Of course, if I had another team member I could put in deployment gates or reviews, etc.

Sponsor: Copy: Rider 2018.2 is here! Publishing to IIS, Docker support in the debugger, built-in spell checking, MacBook Touch Bar support, full C# 7.3 support, advanced Unity support, and more.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
September 20, 2018 11:54
Hi Scott, thanks for your article. Would you mind adding an example of how the build pipeline should look like if you reference other projects that are not available via NuGet? Like pulling a referenced project first, and then build the referenced project before the final build? I have this setup for two different Git repositories A and B where A references B. I can't use Nuget (private build) and I don't want to reference push every local build of B into Nuget. So far, I've heard about sharing artifacts (like the built binaries of B after pushed somewhere and A retrieves these binaries before its build) but the available resources are usually for even simpler cases. I hope that this inspires you for another blog article :)
September 20, 2018 12:07
Nice work. I use a local build agent to debug all my builds. When you get an error you can go to the relative \agent\_work\ directory and run commands there to debug.

So now that you have all that working, Are you going to convert it to yaml?
September 20, 2018 18:38
@Matthias Couple of ways this could work, you could include B's repository as a git submodule/subtree or alternatively using Azure DevOps you can publish into a private feed and consume that from project A.

I'm working on an improvement to NuKeeper which will allow the build of A to be triggered automatically and presented as a pull request to the main project.
September 20, 2018 19:36
We're questioning ourselves when we should run our integration and API tests, so stuff that needs external/deployed infrastructure.

Would this be a deployment gate after deploying to integration (the first stage)?

In theory I would say it is damn too late to run these tests, otherwise you introduced release blockers to your branch which you are using for releases...

You would want to test these as early as possible and that is when creating the pull request. This is why I played around with pull request validation (build policy) which does pre-merging of the changes and runs a clone of the CI build (which does not trigger a release pipeline). But obviously... you would not want to deploy anything there either to run your integratio/API tests.

Is TestServer, in-memory database and "local" build server tests the solution? Don't think this would work out in big projects?

I am still thinking about this stuff, so if somebody has some links regarding these concept... it would be greatly appreciated. ;-)
September 21, 2018 7:03
Hey Scott,

I just went through the process of adding Selenium UI Tests to a Azure DevOps release recently. If you include the nuget package 'Selenium.WebDriver.ChromeDriver' it will copy the ChromeDriver into your build folder and you will be able to reference it in your tests.

browser = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location))

If you are only running dotnet core I found that the dotnet task with the test argument was faster than using the VsTest task but both will give you the results you are looking for. That should be all you need to run your Selenium UI Tests via Azure Pipelines. I used nunit for my tests but xunit should run essentially the same.
September 21, 2018 11:15
VSTS is a very cool, but terrible slow stuff.
So when you build, and create a publish package on local machine less then 1 minute,
then it will run more then 10 minutes on VSTS.
So be careful with those free minutes :D
September 21, 2018 17:45
Hi Scott, nice post.

I have set up something similiar for a large web app, the only issue I have is that the number of unit tests I have is >1000 so I get a warning during testing and I'm unable to view the test results in VSTS.

Just something for you and your readers to bear in mind.

September 24, 2018 16:09
Awesome stuff! We're just working on learning this, and even the very first cautious steps have been super. We have a build server inhouse pulling code from Azure DevOps and a publishing pipeline deploying to an internal test server - just like that. I've been a developer for long enough to appreciate the mountain of work that goes into this, and I truly appreciate the level of polish; this thing just works, for realsies!
September 25, 2018 23:16
Lajos Marton,

We appreciate the feedback on perf, we are doing a lot to speed up our cloud-hosted build agents. Also want to point out that open source projects have "unlimited build minutes".
September 26, 2018 20:21
Thanks for sharing. Really helpful.

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.