Scott Hanselman

Adding Resilience and Transient Fault handling to your .NET Core HttpClient with Polly

April 24, 2018 Comment on this post [9] Posted in ASP.NET | DotNetCore | Open Source
Sponsored By

b30f5128-181e-11e6-8780-bc9e5b17685eLast week while upgrading my podcast site to ASP.NET Core 2.1 and .NET. Core 2.1 I moved my Http Client instances over to be created by the new HttpClientFactory. Now I have a single central place where my HttpClient objects are created and managed, and I can set policies as I like on each named client.

It really can't be overstated how useful a resilience framework for .NET Core like Polly is.

Take some code like this that calls a backend REST API:

public class SimpleCastClient
{
private HttpClient _client;
private ILogger<SimpleCastClient> _logger;
private readonly string _apiKey;

public SimpleCastClient(HttpClient client, ILogger<SimpleCastClient> logger, IConfiguration config)
{
_client = client;
_client.BaseAddress = new Uri($"https://api.simplecast.com");
_logger = logger;
_apiKey = config["SimpleCastAPIKey"];
}

public async Task<List<Show>> GetShows()
{
var episodesUrl = new Uri($"/v1/podcasts/shownum/episodes.json?api_key={_apiKey}", UriKind.Relative);
var res = await _client.GetAsync(episodesUrl);
return await res.Content.ReadAsAsync<List<Show>>();
}
}

Now consider what it takes to add things like

  • Retry n times - maybe it's a network blip
  • Circuit-breaker - Try a few times but stop so you don't overload the system.
  • Timeout - Try, but give up after n seconds/minutes
  • Cache - You asked before!
    • I'm going to do a separate blog post on this because I wrote a WHOLE caching system and I may be able to "refactor via subtraction."

If I want features like Retry and Timeout, I could end up littering my code. OR, I could put it in a base class and build a series of HttpClient utilities. However, I don't think I should have to do those things because while they are behaviors, they are really cross-cutting policies. I'd like a central way to manage HttpClient policy!

Enter Polly. Polly is an OSS library with a lovely Microsoft.Extensions.Http.Polly package that you can use to combine the goodness of Polly with ASP.NET Core 2.1.

As Dylan from the Polly Project says:

HttpClientFactory in ASPNET Core 2.1 provides a way to pre-configure instances of HttpClient which apply Polly policies to every outgoing call.

I just went into my Startup.cs and changed this

services.AddHttpClient<SimpleCastClient>();

to this (after adding "using Polly;" as a namespace)

services.AddHttpClient<SimpleCastClient>().
AddTransientHttpErrorPolicy(policyBuilder => policyBuilder.RetryAsync(2));

and now I've got Retries. Change it to this:

services.AddHttpClient<SimpleCastClient>().
AddTransientHttpErrorPolicy(policyBuilder => policyBuilder.CircuitBreakerAsync(
handledEventsAllowedBeforeBreaking: 2,
durationOfBreak: TimeSpan.FromMinutes(1)
));

And now I've got CircuitBreaker where it backs off for a minute if it's broken (hit a handled fault) twice!

I like AddTransientHttpErrorPolicy because it automatically handles Http5xx's and Http408s as well as the occasional System.Net.Http.HttpRequestException. I can have as many named or typed HttpClients as I like and they can have all kinds of specific policies with VERY sophisticated behaviors. If those behaviors aren't actual Business Logic (tm) then why not get them out of your code?

Go read up on Polly at https://githaub.com/App-vNext/Polly and check out the extensive samples at https://github.com/App-vNext/Polly-Samples/tree/master/PollyTestClient/Samples.

Even though it works great with ASP.NET Core 2.1 (best, IMHO) you can use Polly with .NET 4, .NET 4.5, or anything that's compliant with .NET Standard 1.1.

Gotchas

A few things to remember. If you are POSTing to an endpoint and applying retries, you want that operation to be idempotent.

"From a RESTful service standpoint, for an operation (or service call) to be idempotent, clients can make that same call repeatedly while producing the same result."

But everyone's API is different. What would happen if you applied a Polly Retry Policy to an HttpClient and it POSTed twice? Is that backend behavior compatible with your policies? Know what the behavior you expect is and plan for it. You may want to have a GET policy and a post one and use different HttpClients. Just be conscious.

Next, think about Timeouts. HttpClient's have a Timeout which is "all tries overall timeout" while a TimeoutPolicy inside a Retry is "timeout per try." Again, be aware.

Thanks to Dylan Reisenberger for his help on this post, along with Joel Hulen! Also read more about HttpClientFactory on Steve Gordon's blog and learn more about HttpClientFactory and Polly on the Polly project site.


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
April 24, 2018 7:49
Wow, thanks for the great write-up, Scott! It's been an adventure working with the .NET Core team on this new feature, and seeing some great feedback from the community has been exciting. Huge props to Dylan for his amazing work on this and other features over the past few years.

I just added a link to this post to the Polly readme, where people can find this and other great posts about our little library :)
April 24, 2018 10:05
Polly is great library!
Last time in my .net core project I had to implement circuit breaker policy.
I spent two days for implement generic mechanism, which use all policies from Polly.

Now, each time, when I want to connect with third service - everything what i need to do is just use this mechanism ;)
April 24, 2018 13:24
Great post Scott, would be great to have one with DbConnection and Polly too!
April 24, 2018 18:48
Go read up on Polly at https://githaub.com/App-vNext/Polly and check out the extensive samples at https://github.com/App-vNext/Polly-Samples/tree/master/PollyTestClient/Samples.


Great stuff Scott! I didn't realize there was a German GitHub called GitHaub :)

April 24, 2018 19:27
I am using Polly in Windows Services. It allows me to code fallback logic during file parsing scenarios allowing me to "fail fast" and enhance logging so I have a better picture of what went wrong. I love being able to load up Polly's Context object and get access to it when I fallback.
April 25, 2018 6:03
Still trying to see ow this is any better than multithreaded app with blocking I/O calls and timeouts on those calls. Not much point in explicitly telling the compiler to generate a CPU yield than having it built into the API calls.
April 25, 2018 18:25
Will: When it's "built into the API calls", the thread is still blocked for the duration of the call, which if you want circuit-breaker-like functionality like what Scott pulls in towards the end includes individual breathing pauses on the order of minutes, but which even without that can easily add up to uncomfortably long times for something that should be effectively instantaneous. With enough threads blocked, the thread pool will try to battle thread starvation, just to take one consequence. In general, both the .NET thread pool, the CLR and the kernel prefer as few blocked threads as possible, as it lets them plan upcoming work better.

Not blocking is not a radical concept. It's well documented advice for good reason; Scott himself wrote a good summary in an old post. It may sometimes be simpler to implement and there are situations where it doesn't matter and/or you can get away with it, but you definitely trade it for less robust solutions and harder to debug problems.
May 01, 2018 11:30
Last time in my .net core project I had to implement circuit breaker policy.
I spent two days for implement generic mechanism, which use all policies from Polly.

Now, each time, when I want to connect with third service - everything what i need to do is just use this mechanism ;)

Regards
MJ

Best outdoor projectors
May 03, 2018 3:18
Somehow I'm not able to get Polly working when an error happens in typed HttpClient before sending the actual request. For example Polly's circuit breaker won't kick-in if you replace GetShows() content with
throw new HttpRequestException("Foo");
Raj

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.