Scott Hanselman

Updating my ASP.NET podcast site to System.Text.Json from Newtonsoft.Json

January 10, '20 Comments [11] Posted in DotNetCore
Sponsored By

JSON LogoNow that .NET Core 3.1 is LTS (Long Term Support) and will be supported for 3 years, it's the right time for me to update all my .NET Core 2.x sites to 3.1. It hasn't take long at all and the piece of mind is worth it. It's nice to get all these sites (in the Hanselman ecosystem LOL) onto the .NET Core 3.1 mainline.

While most of my sites working and running just fine - the upgrade was easy - there was an opportunity with the podcast site to move off the venerable Newtonsoft.Json library and move (upgrade?) to System.Text.Json. It's blessed by (and worked on by) James Newton-King so I don't feel bad. It's only a good thing. Json.NET has a lot of history and existed before .NET Standard, Span<T>, and existed in a world where .NET thought more about XML than JSON.

Now that JSON is essential, it was time that JSON be built into .NET itself and System.Text.Json also allows ASP.NET Core to existed without any compatibility issues given its historical dependency on Json.NET. (Although for back-compat reasons you can add Json.NET back with one like using AddJsonOptions if you like).

Everyone's usage of JSON is different so your mileage will depend on how much of Json.NET you used, how much custom code you wrote, and how deep your solution goes. My podcast site uses it to access a number of JSON files I have stored in Azure Storage, as well as to access 3rd party RESTful APIs that return JSON. My podcast site's "in memory database" is effectively a de-serialized JSON file.

I start by bringing in two namespaces, and removing Json.NET's reference and seeing if it compiles! Just rip that Band-Aid off fast and see if it hurts.

using System.Text.Json;
using System.Text.Json.Serialization;

I use Json Serialization in Newtonsoft.Json and have talked before about how much I like C# Type Aliases. Since I used J as an alias for all my Attributes, that made this code easy to convert, and easy to read. Fortunately things like JsonIgnore didn't have their names changed so the namespace was all that was needed there.

NOTE: The commented out part in these snippets is the Newtonsoft bit so you can see Before and After

//using J = Newtonsoft.Json.JsonPropertyAttribute;
using J = System.Text.Json.Serialization.JsonPropertyNameAttribute;

/* SNIP */

public partial class Sponsor
{
[J("id")]
public int Id { get; set; }

[J("name")]
public string Name { get; set; }

[J("url")]
public Uri Url { get; set; }

[J("image")]
public Uri Image { get; set; }
}

I was using Newtonsoft's JsonConvert, so I changed that DeserializeObject call like this:

//public static v2ShowsAPIResult FromJson(string json) => JsonConvert.DeserializeObject<v2ShowsAPIResult>(json, Converter.Settings);
public static v2ShowsAPIResult FromJson(string json) => JsonSerializer.Deserialize<v2ShowsAPIResult>(json);

In other classes some of the changes weren't stylistically the way I'd like them (as an SDK designer) but these things are all arguable either way.

For example, ReadAsAsync<T> is a super useful extension method that has hung off of HttpContent for many years, and it's gone in .NET 3.x. It was an extension that came along for the write inside Microsoft.AspNet.WebApi.Client, but it would bring Newtonsoft.Json back along for the ride.

In short, this Before becomes this After which isn't super pretty.

return await JsonSerializer.DeserializeAsync<List<Sponsor>>(await res.Content.ReadAsStreamAsync());
//return await res.Content.ReadAsAsync<List<Sponsor>>();

But one way to fix this (if this kind of use of ReadAsAsync is spread all over your app) is to make your own extension class:

public static class HttpContentExtensions
{
public static async Task<T> ReadAsAsync<T>(this HttpContent content) =>
await JsonSerializer.DeserializeAsync<T>(await content.ReadAsStreamAsync());
}

My calls to JsonConvert.Serialize turned into JsonSerializer.Serialize:

//public static string ToJson(this List<Sponsor> self) => JsonConvert.SerializeObject(self);
public static string ToJson(this List<Sponsor> self) => JsonSerializer.Serialize(self);

And the reverse of course with JsonSerializer.Deserialize:

//public static Dictionary<string, Shows2Sponsor> FromJson(string json) => JsonConvert.DeserializeObject<Dictionary<string, Shows2Sponsor>>(json);
public static Dictionary<string, Shows2Sponsor> FromJson(string json) => JsonSerializer.Deserialize<Dictionary<string, Shows2Sponsor>>(json);

All in all, far easier than I thought. How have YOU found System.Text.Json to work in your apps?


Sponsor: When DevOps teams focus on fixing new flaws first, they can add to mounting security debt. Veracode’s 2019 SOSS X report spotlights how developers can reduce fix rate times by 72% with frequent scans.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb
Monday, January 13, 2020 7:45:48 AM UTC
I upgraded one of my private projects over the weekend and it was a Desaster. System.Text.Json is not a straight upgrade (faster, less external dependencies) but it's serving a different purpose! While newtonsoft is a very reliable and robust way to deserialize Jason from any webservice, System.Text.Json is very strict regarding types and throws exceptions unless the incoming json is formatted precisely.

You cannot map a quoted numeric value "527" to you model, because the parser would only map this to a string (double quotes). This is ok when you call well known (your own?) Services. I was calling OpenWeatherApi and parts of openstreetmap... Desaster at runtime.
Patrick
Monday, January 13, 2020 9:51:26 AM UTC
The problem I found using System.Text.Json is that it is unable to handle objects of type Dictionary<string, T>, so I ended up to switching back to Json.NET.
Farid
Monday, January 13, 2020 1:14:10 PM UTC
It did not go well for us either, bad enough we are explicitly standardizing on Json.NET moving forward on all new projects
Matt
Monday, January 13, 2020 1:58:03 PM UTC
Yeah, I agree basic stuff is pretty straight cutover, but for a project I am working on, we found a number of issues. Three off the top of my head:
- Cannot serialize/deserialize Dictionaries at all, but it will do key-value-pairs?
- Does not handle the deserialization properly unless properties are exact case, or have JsonPropertyName attributes
- Does not interpret numbers based on the type: ( MaxRange: 100 wouldn't deserialize to int, but Nullable<decimal> worked?
Chris
Monday, January 13, 2020 4:02:13 PM UTC
I think there's a semantical difference between the original ReadAsAsync and the new one. Original looks like it could process the incoming data stream piece by piece, and the new one has to read the whole thing into memory first.
Tuesday, January 14, 2020 1:02:31 AM UTC
I also don't find it as useful or helpful.
All of my "DTO" types are read-only and use [JsonConstructor] to set all properties, and I don't want to change it.

I saw that read-only classes (all properties are read-only) can be used a custom converter, but that is again an extra step that needs to be done.

BTW, the custom converter might work with the quoted integers and dictionaries as well.
Tuesday, January 14, 2020 6:40:28 AM UTC
I ran into a problem with list-typed properties where JSON.Net and System.Text.Json behaved differently, described here:
https://weblogs.asp.net/rweigelt/json-serialization-in-net-core-3-tiny-difference-big-consequences
Roland
Wednesday, January 15, 2020 4:11:53 AM UTC
I've run into all the problems that Chris mentioned above, and also enums don't work. The dictionary problem is most problematic. My team also recently found two more problems.

What they shipped is a good start, but it sure isn't "highly compatible."
Friday, January 17, 2020 12:22:55 PM UTC
System.Text.Json is only for Microsoft guys to do benchmark. Look ma its fast! Its a garbage, not compatible with lot of things with bloody NotSupportedException surprise everywhere. They want us to put custom converter while it was working nicely without any bloody converter. As long the business interest is different, showing fast performance on benchmark instead of usable api and framework then we will see this kind of library being thrown to us.
Andrew
Sunday, January 19, 2020 10:19:41 PM UTC
I found that a lot of things did not deserialize. Calls to Deserialize just returned null, where JSON.NET returned a fully deserialized object. With that level of unreliability, I'm surprised it made it out the door. But I definitely look forward to trying it more as it matures. Hopefully all those kinks will be ironed out.
Dave
Friday, January 24, 2020 4:09:35 PM UTC
I am having the same issues as many of you guys here. System.Text.Json is less than half baked and I am disappointed that Microsoft would have even released it. We have a fairly large Web API and depend on Newtonsoft.Json to serialize and deserialize objects being sent back and forth to the controller endpoints. I just spend the last week upgrading this application to 3.1.1. Which wasn't too bad, until I ran the Integration tests and every single POST and PUT test failed.

Now I am testing System.Text.Json and I find Github issues like these, which break many of the Commands and Queries (CQRS):

System.Text.Json cannot "Deserialization to reference types without a parameterless constructor isn't supported."
Issue - 38163
Issue - 38569

I trust Microsoft will fix these issues, but until then... we have to stay on Asp.net 2.2.

Sorry for my rant.
Aaron
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.