Scott Hanselman

Easier functional and integration testing of ASP.NET Core applications

March 30, 2018 Comment on this post [8] Posted in ASP.NET | DotNetCore
Sponsored By

.NET Test ExplorerIn ASP.NET 2.1 (now in preview) there's apparently a new package called Microsoft.AspNetCore.Mvc.Testing that's meant to help streamline in-memory end-to-end testing of applications that use the MVC pattern. I've been re-writing my podcast site at https://hanselminutes.com in ASP.NET Core 2.1 lately, and recently added some unit testing and automatic unit testing with code coverage. Here's a couple of basic tests. Note that these call the Razor Pages directly and call their OnGet() methods directly. This shows how ASP.NET Core is nicely factored for Unit Testing but it doesn't do a "real" HTTP GET or perform true end-to-end testing.

These tests are testing if visiting URLs like /620 will automatically redirect to the correct full canonical path as they should.

[Fact]
public async void ShowDetailsPageIncompleteTitleUrlTest()
{
// FAKE HTTP GET "/620"
IActionResult result = await pageModel.OnGetAsync(id:620, path:"");

RedirectResult r = Assert.IsType<RedirectResult>(result);
Assert.NotNull(r);
Assert.True(r.Permanent); //HTTP 301?
Assert.Equal("/620/jessica-rose-and-the-worst-advice-ever",r.Url);
}

[Fact]
public async void SuperOldShowTest()
{
// FAKE HTTP GET "/default.aspx?showId=18602"
IActionResult result = await pageModel.OnGetOldShowId(18602);

RedirectResult r = Assert.IsType<RedirectResult>(result);
Assert.NotNull(r);
Assert.True(r.Permanent); //HTTP 301?
Assert.StartsWith("/615/developing-on-not-for-a-nokia-feature",r.Url);
}

I wanted to see how quickly and easily I could do these same two tests, except "from the outside" with an HTTP GET, thereby testing more of the stack.

I added a reference to Microsoft.AspNetCore.Mvc.Testing in my testing assembly using the command-line equivalanet of "Right Click | Add NuGet Package" in Visual Studio. This CLI command does the same thing as the UI and adds the package to the csproj file.

dotnet add package Microsoft.AspNetCore.Mvc.Testing -v 2.1.0-preview1-final

It includes a new WebApplicationTestFixture that I point to my app's Startup class. Note that I can take store the HttpClient the TestFixture makes for me.

public class TestingMvcFunctionalTests : IClassFixture<WebApplicationTestFixture<Startup>>
{
public HttpClient Client { get; }

public TestingMvcFunctionalTests(WebApplicationTestFixture<Startup> fixture)
{
Client = fixture.Client;
}
}

No tests yet, just setup. I'm using SSL redirection so I'll make sure the client knows that, and add a test:

public TestingMvcFunctionalTests(WebApplicationTestFixture<Startup> fixture)
{
Client = fixture.Client;
Client.BaseAddress = new Uri("https://localhost");
}

[Fact]
public async Task GetHomePage()
{
// Arrange & Act
var response = await Client.GetAsync("/");

// Assert
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}

This will fail, in fact. Because I have an API Key that is needed to call out to my backend system, and I store it in .NET's User Secrets system. My test will get an InternalServerError instead of OK.

Starting test execution, please wait...
[xUnit.net 00:00:01.2110048] Discovering: hanselminutes.core.tests
[xUnit.net 00:00:01.2690390] Discovered: hanselminutes.core.tests
[xUnit.net 00:00:01.2749018] Starting: hanselminutes.core.tests
[xUnit.net 00:00:08.1088832] hanselminutes_core_tests.TestingMvcFunctionalTests.GetHomePage [FAIL]
[xUnit.net 00:00:08.1102884] Assert.Equal() Failure
[xUnit.net 00:00:08.1103719] Expected: OK
[xUnit.net 00:00:08.1104377] Actual: InternalServerError
[xUnit.net 00:00:08.1114432] Stack Trace:
[xUnit.net 00:00:08.1124268] D:\github\hanselminutes-core\hanselminutes.core.tests\FunctionalTests.cs(29,0): at hanselminutes_core_tests.TestingMvcFunctionalTests.<GetHomePage>d__4.MoveNext()
[xUnit.net 00:00:08.1126872] --- End of stack trace from previous location where exception was thrown ---
[xUnit.net 00:00:08.1158250] Finished: hanselminutes.core.tests
Failed hanselminutes_core_tests.TestingMvcFunctionalTests.GetHomePage
Error Message:
Assert.Equal() Failure
Expected: OK
Actual: InternalServerError

Where do these secrets come from? In Development they come from user secrets.

public Startup(IHostingEnvironment env)
{
this.env = env;
var builder = new ConfigurationBuilder();

if (env.IsDevelopment())
{
builder.AddUserSecrets<Startup>();
}
Configuration = builder.Build();
}

But in Production they come from the ENVIRONMENT. Are these tests Development or Production...I must ask myself.  They are Production unless told otherwise. I can override the Fixture and tell it to use another Environment, like "Development." Here is a way (given this preview) to make my own TestFixture by deriving and grabbing and override to change the Environment. I think it's too hard and should be easier.

Either way, the real question here is for me - do I want my tests to be integration tests in development or in "production." Likely I need to make a new environment for myself - "testing."

public class MyOwnTextFixture<TStartup> : WebApplicationTestFixture<Startup> where TStartup : class
{
public MyOwnTextFixture() { }

protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Development");
}
}

However, my User Secrets still aren't loading, and that's where the API Key is that I need.

BUG?: There is either a bug here, or I don't know what I'm doing. I'm loading User Secrets in builder.AddUserSecrets<Startup> and later injecting the IConfiguration instance from builder.Build() and going "_apiKey = config["SimpleCastAPIKey"];" but it's null. The config that's injected later in the app isn't the same one that's created in Startup.cs. It's empty. Not sure if this is an ASP.NE Core 2.0 thing or 2.1 thing but I'm going to bring it up with the team and update this blog post later. It might be a Razor Pages subtlety I'm missing.
For now, I'm going to put in a check and manually fix up my Config. However, when this is fixed (or I discover my error) this whole thing will be a pretty nice little set up for integration testing.

I will add another test, similar to the redirect Unit Test but a fuller integration test that actually uses HTTP and tests the result.

[Fact]
public async Task GetAShow()
{
// Arrange & Act
var response = await Client.GetAsync("/620");

// Assert
Assert.Equal(HttpStatusCode.MovedPermanently, response.StatusCode);
Assert.Equal("/620/jessica-rose-and-the-worst-advice-ever",response.Headers.Location.ToString());
}

There's another issue here that I don't understand. Because have to set Client.BaseAddress to https://localhost (because https) and the Client is passed into fixture.Client, I can't set the Base address twice or I'll get an exception, as the Test's Constructor runs twice, but the HttpClient that's passed in as a lifecycler that's longer. It's being reused, and it fails when setting its BaseAddress twice.

Error Message:
System.InvalidOperationException : This instance has already started one or more requests. Properties can only be modified before sending the first request.

BUG? So to work around it I check to see if I've done it before. Which is gross. I want to set the BaseAddress once, but I am not in charge of the creation of this HttpClient as it's passed in by the Fixture.

public TestingMvcFunctionalTests(MyOwnTextFixture<Startup> fixture)
{
Client = fixture.Client;
if (Client.BaseAddress.ToString().StartsWith("https://") == false)
Client.BaseAddress = new Uri("https://localhost");
}

Another option is that I create a new client every time, which is less efficient and perhaps a better idea as it avoids any side effects from other tests, but also feels weird that I should have to do this, as the new standard for ASP.NET Core sites is to be SSL/HTTPS by default..

public TestingMvcFunctionalTests(MyOwnTextFixture<Startup> fixture)
{
Client = fixture.CreateClient(new Uri(https://localhost));
}

I'm still learning about how it all fits together, but later I plan to add in Selenium tests to have a full, complete, test suite that includes the browser, CSS, JavaScript, end-to-end integration tests, and unit tests.

Let me know if you think I'm doing something wrong. This is preview stuff, so it's early days!


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
April 02, 2018 13:12
I've been using 'microsoft.aspnetcore.testhost' for hosting my apis in an integration test scenario. Is this conceptually the same but with nicer syntax and integration or am I missing something?
April 02, 2018 15:09
Great post, can't wait for this to land.
Your last code snippet looks like a copy and paste fail btw.
April 03, 2018 2:48
Great post. Do you know if v2.1 will provide the ability to invoke controllers without having to remember magic strings? My GitHub project https://github.com/tulde23/AspNetCore.IntegrationTesting/blob/master/README.md implements such a feature.
April 03, 2018 10:25
Good to see .net core + VS code maturing. Would like to see some MS effort into getting Fitnesse more usable for BDD and unit testing.
April 04, 2018 11:40
This looks like a very interesting development - especially as it looks like i'm finally going to be able to (for a while at least) divert my attention from legacy ASP.Net apps and look into Dot Net Core, for a new project.

One things that springs to mind about this new testing scheme is the question of how authentication/authorization is handled.
Given that a developer would want to be able to confirm that different user profiles cause different (and predictable) behaviour from the same URL, how does this testing framework support running test in different user contexts (if it supports this at all)?

April 04, 2018 14:39
Bjorn, I think the difference is that this will work with razor views
https://github.com/aspnet/Mvc/issues/3410
April 04, 2018 19:52
This does´t work at all on my computer. I´m using "VS 15.7.0 Preview 2 Community" and I get the first error when following your example.

And if I follow ASP.NET Core 2.1.0-preview1: Functional testing of MVC applications straight up with the command line creation I get error nr 2 here below.

  • WebApplication2.deps.json'. This file is required for functional tests to run properly. There should be a copy of the file on your source project bin folder. If that is not the case, make sure that the property PreserveCompilationContext is set to true on your project file. E.g '<PreserveCompilationContext>true</PreserveCompilationContext>'. For functional tests to work they need to either run from the build output folder or the WebApplication2.deps.json file from your application's output directory must be copied to the folder where the tests are running on. A common cause for this error is having shadow copying enabled when the tests run.) (The following constructor parameters did not have matching fixture data: WebApplicationTestFixture`1 fixture)

  • System.AggregateException : One or more errors occurred. (The content root 'C:\Users\me\source\repos\TestingMvc\TestingMvc' does not exist. Parameter name: contentRootPath) (The following constructor parameters did not have matching fixture data: WebApplicationTestFixture`1 fixture)

    I tried to do this few times without luck. Everything is up2 date I think. Sorry for drowning your comments in debug info but I so want this to work.

    Here is a GithubGist with my VS and dotnet --info if somebody can take a look to see if I´m missing something.
  • April 05, 2018 11:53
    I'm not sure I understand; how does this differ from using TestServer from the Microsoft.AspNetCore.TestHost package?

    Comments are closed.

    Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.