Fixing JSON parsing exception “value could not be converted” using a JsonConverter

by

I’ve been working on a tool lately to parse a bunch of work item data from Azure DevOps. Specifically, I needed to call the Work Item Updates REST service in order to get all the state value changes for a bunch of work items.

Everything was fine until I needed to convert the JSON result data from strings into objects. Then I started getting tons of exceptions saying things like “System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $.value[0].fields[‘System.Id’].newValue | LineNumber: 24 | BytePositionInLine: 25. —> System.InvalidOperationException: Cannot get the value of a token type ‘Number’ as a string.”

The ultimate answer was a custom implementation of JsonConverter<T>. (If you’re feeling impatient, here’s the sample code.)

Strange Data Formats

In the return value from that service, there’s a collection of changed field values that gives you the old value and the new value as those fields change. Well, in Azure DevOps, a field can be a handful of different data types including integer, string, boolean, date, or a person’s identity. Check out the JSON snippet below. The fields property is basically a Dictionary<string, object>. The keys are values like “System.Rev” or “System.State” but then the values for the “oldValue” and “newValue” properties can be completely different. For example, the data type for System.Rev is a number, the value for System.State is a string, System.ChangedDate is a datetime, and then Microsoft.VSTS.Common.ClosedBy is a really complex object that represents the person who performed the action.

"fields": {
  "System.Rev": {
    "oldValue": 5,
    "newValue": 6
  },
  "System.State": {
    "oldValue": "Active",
    "newValue": "Closed"
  },
  "System.ChangedDate": {
    "oldValue": "2022-08-19T12:39:33.07Z",
    "newValue": "2022-08-19T12:39:37.47Z"
  },
  "Microsoft.VSTS.Common.ClosedBy": {
    "newValue": {
      "displayName": "Benjamin Day",
      "url": "https://spsprodeus22.vssps.visualstudio.com/B5q33db6d-a202-4c3a-ae2c-be1d2a3875c3/_apis/Identities/B5q33db6d-7f35-4df7-b308-395bf4eaf8d9",
      "_links": {
        "avatar": {
          "href": "https://dev.azure.com/benday/_apis/GraphProfile/MemberAvatars/aad.mmmmmWFkZTYtOWI1ZS03NTZlLWExblatZTU2NDIbla3MzYw"
        }
      },
      "id": "B5q33db6d-7f35-4df7-b308-395bf4eaf8d9",
      "uniqueName": "benday@benday.com",
      "imageUrl": "https://dev.azure.com/benday/_apis/GraphProfile/MemberAvatars/aad.mmmmmWFkZTYtOWI1ZS03NTZlLWExblatZTU2NDIbla3MzYw",
      "descriptor": "aad.mmmmmWFkZTYtOWI1ZS03NTZlLWExblatZTU2NDIbla3MzYw"
    }
  }
}

System.Text.Json.JsonSerializer Gets Confused

In my application, I just needed to access the oldValue and newValue properties as strings. When the JsonSerializer was asked to parse that fields property as Dictionary<string, object>, everything was fine because the deserialization wasn’t especially worried about types. Can it be converted to an object? Yes. Good. Move on.

For example, if I tried to deserialize that fields data into a class like WorkItemRevisionInfoWithFieldsAsObjects (see below), it’s fine.

public class WorkItemRevisionInfoWithFieldsAsObjects
{
    [JsonPropertyName("fields")]
    public Dictionary<string, object> Fields { get; set; } = new();

    // ...       
} 

But when I tried to change that to use Dictionary<string, FieldRevision>, then it the problems started.

public class WorkItemRevisionInfoWithFieldsAsTypedValues
{
    // [JsonPropertyName("fields")]
    // public Dictionary<string, object> Fields { get; set; }

    [JsonPropertyName("fields")]
    public Dictionary<string, FieldRevision> Fields { get; set; } = new();

    // ...
}

The FieldRevision class wasn’t all that complex either — just two string properties.

public class FieldRevision
{
    [JsonPropertyName("oldValue")]
    public string OldValue { get; set; } = String.Empty;

    [JsonPropertyName("newValue")]
    public string NewValue { get; set; } = String.Empty;
}

But once I started trying to deserialize that JSON, there were errors all over the place.

Unhandled exception. System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $.value[0].fields['System.Id'].newValue | LineNumber: 24 | BytePositionInLine: 25.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'Number' as a string.
   at System.Text.Json.Utf8JsonReader.GetString()
   at System.Text.Json.Serialization.Converters.StringConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Program.<<Main>$>g__ParseUsingTypedValues|0_0(String json) in /Users/benday/code/temp/Benday.JsonConverterSample/Benday.JsonConverterSample/Program.cs:line 12
   at Program.<Main>$(String[] args) in /Users/benday/code/temp/Benday.JsonConverterSample/Benday.JsonConverterSample/Program.cs:line 8

JsonConverter<T> to the Rescue

The solution is to add an implementation of JsonConverter<T> to my project and then to instruct the JsonSerializer to use that converter to deserialize those properties.

Here’s my implementation:

public class EverythingToStringJsonConverter : JsonConverter<string>
{
    public override string Read(ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options)
    {

        if (reader.TokenType == JsonTokenType.String)
        {
            return reader.GetString() ?? String.Empty;
        }
        else if (reader.TokenType == JsonTokenType.Number)
        {
            var stringValue = reader.GetDouble();
            return stringValue.ToString();
        }
        else if (reader.TokenType == JsonTokenType.False ||
            reader.TokenType == JsonTokenType.True)
        {
            return reader.GetBoolean().ToString();
        }
        else if (reader.TokenType == JsonTokenType.StartObject)
        {
            reader.Skip();
            return "(not supported)";
        }
        else
        {
            Console.WriteLine($"Unsupported token type: {reader.TokenType}");

            throw new System.Text.Json.JsonException();
        }
    }

    public override void Write(Utf8JsonWriter writer, string value, JsonSerializerOptions options)
    {
        writer.WriteStringValue(value);
    }
}

Once I had that class in my project, I just had to tell the JsonSerializer where to use it. In this case, I needed to add a [JsonConverter] attribute to the properties on my FieldRevisions class.

public class FieldRevision
{
    [JsonPropertyName("oldValue")]
    [JsonConverter(typeof(EverythingToStringJsonConverter))]
    public string OldValue { get; set; } = String.Empty;

    [JsonPropertyName("newValue")]
    [JsonConverter(typeof(EverythingToStringJsonConverter))]
    public string NewValue { get; set; } = String.Empty;
}

After that, everything ran fine and I had the data I needed in the format that I wanted.

Summary

When you need to make custom adjustments when you serializing and deserializing your JSON data to or from objects, you’ll want to create an implementation of JsonConverter<T>. Create an implementation of JsonConverter and then add the JsonConverter attribute to you the properties and classes where you need that custom conversion.

I hope this helps.

-Ben

One Response to "Fixing JSON parsing exception “value could not be converted” using a JsonConverter"

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: