Scott Hanselman

Using LazyCache for clean and simple .NET Core in-memory caching

May 12, 2018 Comment on this post [18] Posted in ASP.NET | DotNetCore | Open Source
Sponsored By

Tai Chi by Luisen Rodrigo - Used under CCI'm continuing to use .NET Core 2.1 to power my Podcast Site, and I've done a series of posts on some of the experiments I've been doing. I also upgraded to .NET Core 2.1 RC that came out this week. Here's some posts if you want to catch up:

Having a blast, if I may say so.

I've been trying a number of ways to cache locally. I have an expensive call to a backend (7-8 seconds or more, without deserialization) so I want to cache it locally for a few hours until it expires. I have a way that work very well using a SemaphoreSlim. There's some issues to be aware of but it has been rock solid. However, in the comments of the last caching post a number of people suggested I use "LazyCache."

Alastair from the LazyCache team said this in the comments:

LazyCache wraps your "build stuff I want to cache" func in a Lazy<> or an AsyncLazy<> before passing it into MemoryCache to ensure the delegate only gets executed once as you retrieve it from the cache. It also allows you to swap between sync and async for the same cached thing. It is just a very thin wrapper around MemoryCache to save you the hassle of doing the locking yourself. A netstandard 2 version is in pre-release.
Since you asked the implementation is in CachingService.cs#L119 and proof it works is in CachingServiceTests.cs#L343

Nice! Sounds like it's worth trying out. Most importantly, it'll allow me to "refactor via subtraction."

I want to have my "GetShows()" method go off and call the backend "database" which is a REST API over HTTP living at SimpleCast.com. That backend call is expensive and doesn't change often. I publish new shows every Thursday, so ideally SimpleCast would have a standard WebHook and I'd cache the result forever until they called me back. For now I will just cache it for 8 hours - a long but mostly arbitrary number. Really want that WebHook as that's the correct model, IMHO.

LazyCache was added on my Configure in Startup.cs:

services.AddLazyCache();

Kind of anticlimactic. ;)

Then I just make a method that knows how to populate my cache. That's just a "Func" that returns a Task of List of Shows as you can see below. Then I call IAppCache's "GetOrAddAsync" from LazyCache that either GETS the List of Shows out of the Cache OR it calls my Func, does the actual work, then returns the results. The results are cached for 8 hours. Compare this to my previous code and it's a lot cleaner.

public class ShowDatabase : IShowDatabase
{
    private readonly IAppCache _cache;
    private readonly ILogger _logger;
    private SimpleCastClient _client;

    public ShowDatabase(IAppCache appCache,
            ILogger<ShowDatabase> logger,
            SimpleCastClient client)
    {
        _client = client;
        _logger = logger;
        _cache = appCache;
    }

    public async Task<List<Show>> GetShows()
    {    
        Func<Task<List<Show>>> showObjectFactory = () => PopulateShowsCache();
        var retVal = await _cache.GetOrAddAsync("shows", showObjectFactory, DateTimeOffset.Now.AddHours(8));
        return retVal;
    }
 
    private async Task<List<Show>> PopulateShowsCache()
    {
        List<Show> shows = await _client.GetShows();
        _logger.LogInformation($"Loaded {shows.Count} shows");
        return shows.Where(c => c.PublishedAt < DateTime.UtcNow).ToList();
    }
}

It's always important to point out there's a dozen or more ways to do this. I'm not selling a prescription here or The One True Way, but rather exploring the options and edges and examining the trade-offs.

  • As mentioned before, me using "shows" as a magic string for the key here makes no guarantees that another co-worker isn't also using "shows" as the key.
    • Solution? Depends. I could have a function-specific unique key but that only ensures this function is fast twice. If someone else is calling the backend themselves I'm losing the benefits of a centralized (albeit process-local - not distributed like Redis) cache.
  • I'm also caching the full list and then doing a where/filter every time.
    • A little sloppiness on my part, but also because I'm still feeling this area out. Do I want to cache the whole thing and then let the callers filter? Or do I want to have GetShows() and GetActiveShows()? Dunno yet. But worth pointing out.
  • There's layers to caching. Do I cache the HttpResponse but not the deserialization? Here I'm caching the List<Shows>, complete. I like caching List<T> because a caller can query it, although I'm sending back just active shows (see above).
    • Another perspective is to use the <cache> TagHelper in Razor and cache Razor's resulting rendered HTML. There is value in caching the object graph, but I need to think about perhaps caching both List<T> AND the rendered HTML.
    • I'll explore this next.

I'm enjoying myself though. ;)

Go explore LazyCache! I'm using beta2 but there's a whole number of releases going back years and it's quite stable so far.

Lazy cache is a simple in-memory caching service. It has a developer friendly generics based API, and provides a thread safe cache implementation that guarantees to only execute your cachable delegates once (it's lazy!). Under the hood it leverages ObjectCache and Lazy to provide performance and reliability in heavy load scenarios.

For ASP.NET Core it's quick to experiment with LazyCache and get it set up. Give it a try, and share your favorite caching techniques in the comments.

Tai Chi photo by Luisen Rodrigo used under Creative Commons Attribution 2.0 Generic (CC BY 2.0), thanks!


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
May 13, 2018 11:42
Hi Scott,

nice article, as usual.

What about Microsoft.Extensions.Logging? Isn't supposed to replace ObjectCache?
May 13, 2018 11:45
May 13, 2018 19:44
What is wrong with using GetOrCreate from https://github.com/aspnet/Caching/blob/dev/src/Microsoft.Extensions.Caching.Abstractions/MemoryCacheExtensions.cs?
May 14, 2018 5:09
What's the benefit of using this over Microsoft.Extensions.Caching.Memory.MemoryCache? I typically use that in single-instance situations, and use the StackExchange.Redis client if I need something to span multiple instances (it's pretty easy to even swap them out in the same abstraction, if you think you need to).
May 14, 2018 5:51
Publish the new list of shows to both the back end server and your front end server at the same time. Always, get the values from the font end server for the list of shows.

Much easier than cache timeouts, and less complicated than using a third party caching mechanism.
Why
May 14, 2018 10:46
I'm also caching the full list and then doing a where/filter every time.


Can you point to where that is occurring? My naive reading of the code suggests the Where is prior to caching.
May 14, 2018 11:56
I understand that using the magic string "shows" can be a problem. But what about using the magic string "06a4fb7c-0bfd-47ea-ad9c-b8ca76c75db9" (randomly generated)? This can be extracted to a constant and it's under the hood of "ShowDatabase" class. (or shows_06a4fb7c-0bfd-47ea-ad9c-b8ca76c75db9)
May 14, 2018 16:29
I'm curious - did you look at interception? I use Windsor for that. The main reason is that keeps caching separate so a class doesn't "know" it's being cached.
May 14, 2018 18:37
Also curious about what John P asks. This looks similar to the built-in GetOrCreateAsync extension method:

var item = await cache.GetOrCreateAsync("mykeyfoo-" + id, async (c) =>{
c.SetAbsoluteExpiration(DateTimeOffset.Now.AddMinutes(1));
c.SetSlidingExpiration(TimeSpan.FromMinutes(1));
return await GetViewModelAsync(id, true);
});

Is there any reason to prefer LazyCache's implementation?
May 14, 2018 20:02
Due to the cost of the back-end request, I wonder if there's some way to set the cache so that once it first expires, it'll still return the cached values to the current request (and any subsequent requests until the cache updates), but also start a back-end process to get updated values for the cache?

That way, you won't have an unlucky user who has a 7-8 second wait every 8 hours. You could also reduce that 8 hour interval to one hour, or even 30 or 15 minutes.
May 14, 2018 21:07
To simplify it even further, you could switch this:

Func<Task<List<Show>>> showObjectFactory = () => PopulateShowsCache();
var retVal = await _cache.GetOrAddAsync("shows", showObjectFactory, DateTimeOffset.Now.AddHours(8));

To this:

var retVal = await _cache.GetOrAddAsync("shows", this.PopulateShowsCache, DateTimeOffset.Now.AddHours(8));
May 15, 2018 0:43
I'm sorry - I'm probably missing something basic here, but why the double assignment in row 25?

List<Show> shows = shows = await _client.GetShows();

Should it not rather be:

List<Show> shows = await _client.GetShows();
May 16, 2018 13:15
Lazycache has a developer friendly generics based API, and provides a thread safe cache implementation to only execute cachable delegates once. Ad it uses ObjectCache under the hood and by default it will use MemoryCache.Default. It holds all the expiring features of ObjectCache.
May 17, 2018 11:27
Lazycache really usefull for newbies like me. im trying learn something from your site. thanks mate.
May 17, 2018 11:40
I wouldn't cache Funcs... too easy to mess up... e.g. with EntityFramework & multi-threading scenarios (expect exceptions with invalid access to DbContext and other shizzle)
May 17, 2018 21:54
Unfortunately, due to a pretty debilitating bug in the Redis Object Cache package (reported https://github.com/justinfinch/Redis-Object-Cache/issues/9), I simply cannot use (and get the benefit of) the GetOrAdd methods of LazyCache.
May 20, 2018 8:09
Hi There!
Thanks for your post. very awesome
May 21, 2018 16:34
I think it's worth to mention that .net core's 2.0 MemoryCache GetOrCreate isn't entirely thread safe, in the sense that if two threads are calling it with the same key, and different values, each will get a different returned value, and you can't tell which was saved to the cache.

something to notice if LazyCache is using it behind the scenes..

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.