Support filtering property names on dictionary deserialization.

[terrajobst]

EDIT See #79059 (comment) for API proposal.

It seems there is currently no way to make JSON documents that reference a schema just deserializa.

Repro

{
    "$schema": "http://example.org/myschema.json",
    "Key1": {
        "V1": 10,
        "V2": "Ten"
    },
    "Key2": {
        "V1": 20,
        "V2": "Twenty"
    }
}

var content = "<json above>";
return JsonSerializer.Deserialize<Dictionary<string, CustomType>>(content);

class CustomType {
    public int V1 { get; set; }
    public string V2 { get; set; }
}

Expected Behavior

It just works. Or, based on JsonSerializerOptions, it could be configured to just work.

Actual Behavior

It causes an exception:

The JSON value could not be converted to CustomType. Path: $.$schema

In order to make it deserializable, the easiest option is to manually remove the schema node from the document:

if (JsonNode.Parse(content) is JsonObject jsonObject)
{
    if (jsonObject.Remove("$schema"))
        content = jsonObject.ToJsonString();
}

The more performant alternative would be to write a custom converter, but that seems like a lot of work. If the root type of the document weren't dictionary but a type I control, then presumably I could work this around by adding a property to my class and decorate it with [JsonPropertyName("$schema")] but that feels equally hacky.

Customer Evidence

It’s not entirely clear to me if this is worth prioritizing. On the one hand, it seems a decent amount of our customers might use JSON schemas (based on this super scientific poll below).

On the other hand, it would only impact customers who use dictionaries as the document type. That combination might be quite rare and may not be be worth handling directly. And based on James’s comments below, JSON.NET never handled that either. I suggest we leave it open for a bit and see what responses we get but my gut feel says cutting seems fine.

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

It seems there is currently no way to make JSON documents that reference a schema just deserializa.

Repro

{
    "$schema": "http://example.org/myschema.json",
    "Key1": "Value1",
    "Key2": "Value2"
}

var content = "<json above>";
return JsonSerializer.Deserialize<Dictionary<string, string>>(content);

Expected Behavior

It just works. Or, based on JsonSerializerOptions, it could be configured to just work.

Actual Behavior

It causes an exception:

The JSON value could not be converted to System.String. Path: $.$schema

In order to make it deserializable, the easiest option is to manually remove the schema node from the document:

if (JsonNode.Parse(content) is JsonObject jsonObject)
{
    if (jsonObject.Remove("$schema"))
        content = jsonObject.ToJsonString();
}

The more performant alternative would be to write a custom converter, but that seems like a lot of work. If the root type of the document weren't dictionary but a type I control, then presumably I could work this around by adding a property to my class and decorate it with [JsonPropertyName("$schema")] but that feels equally hacky.

Author:	terrajobst
Assignees:	-
Labels:	area-System.Text.Json
Milestone:	-

[layomia]

It seems reasonable to enable this scenario if we have substantial user asks for it. Feels like "future" for now wrt urgency.

I wonder if this problem is scoped to JSON schema support or whether a general solution is needed i.e a feature to express that a set of property names should be omitted from (de)serialization. I assume this would be scoped to just dictionaries since we already have that for POCO properties ([JsonIgnore] and related features).

[gregsdennis]

This isn't a JSON Schema problem (at least not properly). JSON Schema puts no requirements on JSON data.

Including $schema in an instance is a convention that some tools, like VS Code, uses to identify how to validate the data.

$schema inside of a JSON Schema carries additional and specific meaning.

[ShreyasJejurkar]

@terrajobst did you gave try with @JamesNK Newtonsoft one? Does it work there?

[JamesNK]

Newtonsoft.Json errors:

Newtonsoft.Json.JsonSerializationException: 'Error converting value "http://example.org/myschema.json" to type 'CustomType'. Path '$schema', line 2, position 49.'

I'd recommend people load the JSON into a JObject and remove the $schema property and then deserialize it.

[gregsdennis]

The System.Text.Json analog for that is JsonNode. @terrajobst has the proper workaround in the opening comment.

[eiriktsarpalis]

A few observations:

• POCO deserialization will generally skip any properties it fails to bind to so this shouldn't be an issue in that case. The same is true for non-generic dictionaries or IDictionary<string, object>, whose deserialization semantics closely follow those of JsonObject.
• This issue only concerns generic dictionary deserialization, and the fact that all values in the deserialized JSON object must conform to whatever contract TValue imposes in IDictionary<TKey, TValue>. Assuming we get union types eventually (where something like Dictionary<string, string | CustomType> becomes expressible) it would be possible to work around the issue using appropriate modelling.
• Assuming this is a request for JSON Schema support, this is being tracked in [System.Text.Json] Json Schemas Support #29887.
• Assuming it is a request for property deny lists for dictionary deserialization in general, it might be something we could consider. I don't believe it is possible to work around the issue currently using JsonIgnoreAttribute or JsonPropertyName attributes since dictionaries don't use them.

[terrajobst]

I added a customer evidence section with my thoughts.

[terrajobst]

@eiriktsarpalis

• Assuming this is a request for JSON Schema support

That’s a good point. I didn’t call it out explicitly but that wasn’t my intent. Rather, I think we could decide to never support JSON schemas (ie offer validation or an OM for reading one) while still making this scenario possible by simply ignoring the well-known property name during The deserialization. So in that sense, I think of them of as orthogonal issues.

[eiriktsarpalis]

Would we need to ignore other well-known metadata property names like $type or $id? There are dozens of schemes like that out there, I suspect we might want to expose it as deny list so that users can cherry-pick what they need to be skipped.