Encode nil value as null with JSONEncoder

Question

I'm using Swift 4's JSONEncoder. I have a Codable struct with an optional property, and I'd like this property to show up as null value in the produced JSON data when the value is nil. However, JSONEncoder discards the property and does not add it to the JSON output. Is there a way to configure JSONEncoder so that it preserves the key and sets it to null in this case?

Example

The code snippet below produces {"number":1}, but I'd rather like it to give me {"string":null,"number":1}:

struct Foo: Codable {
  var string: String? = nil
  var number: Int = 1
}

let encoder = JSONEncoder()
let data = try! encoder.encode(Foo())
print(String(data: data, encoding: .utf8)!)

Answer 1

Yes, but you'll have to write your own encode(to:) implementation, you can't use the auto-generated one.

struct Foo: Codable {
    var string: String? = nil
    var number: Int = 1

    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(number, forKey: .number)
        try container.encode(string, forKey: .string)
    }
}

Encoding an optional directly will encode a null, like you're looking for.

If this is an important use case for you, you may consider opening a defect at bugs.swift.org to ask for a new OptionalEncodingStrategy flag to be added on JSONEncoder to match the existing DateEncodingStrategy, etc. (See below why this is likely impossible to actually implement in Swift today, but getting into the tracking system is still useful as Swift evolves.)

---

Edit: To Paulo's questions below, this dispatches to the generic encode<T: Encodable> version because Optional conforms to Encodable. This is implemented in Codable.swift this way:

extension Optional : Encodable /* where Wrapped : Encodable */ {
    @_inlineable // FIXME(sil-serialize-all)
    public func encode(to encoder: Encoder) throws {
        assertTypeIsEncodable(Wrapped.self, in: type(of: self))

        var container = encoder.singleValueContainer()
        switch self {
        case .none: try container.encodeNil()
        case .some(let wrapped): try (wrapped as! Encodable).__encode(to: &container)
        }
    }
}

This wraps the call to encodeNil, and I think letting stdlib handle Optionals as just another Encodable is better than treating them as a special case in our own encoder and calling encodeNil ourselves.

Another obvious question is why it works this way in the first place. Since Optional is Encodable, and the generated Encodable conformance encodes all the properties, why does "encode all the properties by hand" work differently? The answer is that the conformance generator includes a special case for Optionals:

// Now need to generate `try container.encode(x, forKey: .x)` for all
// existing properties. Optional properties get `encodeIfPresent`.
...

if (varType->getAnyNominal() == C.getOptionalDecl() ||
    varType->getAnyNominal() == C.getImplicitlyUnwrappedOptionalDecl()) {
  methodName = C.Id_encodeIfPresent;
}

This means that changing this behavior would require changing the auto-generated conformance, not JSONEncoder (which also means it's probably really hard to make configurable in today's Swift....)

Answer 2

Here's an approach that uses a property wrapper (requires Swift v5.1):

@propertyWrapper
struct NullEncodable<T>: Encodable where T: Encodable {
    
    var wrappedValue: T?

    init(wrappedValue: T?) {
        self.wrappedValue = wrappedValue
    }
    
    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        switch wrappedValue {
        case .some(let value): try container.encode(value)
        case .none: try container.encodeNil()
        }
    }
}

Sample usage:

struct Tuplet: Encodable {
    let a: String
    let b: Int
    @NullEncodable var c: String? = nil
}

struct Test: Encodable {
    @NullEncodable var name: String? = nil
    @NullEncodable var description: String? = nil
    @NullEncodable var tuplet: Tuplet? = nil
}

var test = Test()
test.tuplet = Tuplet(a: "whee", b: 42)
test.description = "A test"

let data = try JSONEncoder().encode(test)
print(String(data: data, encoding: .utf8) ?? "")

Output:

{
  "name": null,
  "description": "A test",
  "tuplet": {
    "a": "whee",
    "b": 42,
    "c": null
  }
}

Full implementation here: https://github.com/g-mark/NullCodable

Answer 3

Here is an approach we have used in a project. Hope it helps.

struct CustomBody: Codable {
    let method: String
    let params: [Param]

    enum CodingKeys: String, CodingKey {
        case method = "method"
        case params = "params"
    }
}

enum Param: Codable {
    case bool(Bool)
    case integer(Int)
    case string(String)
    case stringArray([String])
    case valueNil
    case unsignedInteger(UInt)
    case optionalString(String?)

    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        if let x = try? container.decode(Bool.self) {
            self = .bool(x)
            return
        }
        if let x = try? container.decode(Int.self) {
            self = .integer(x)
            return
        }
        if let x = try? container.decode([String].self) {
              self = .stringArray(x)
              return
          }
        if let x = try? container.decode(String.self) {
            self = .string(x)
            return
        }
        if let x = try? container.decode(UInt.self) {
            self = .unsignedInteger(x)
            return
        }
        throw DecodingError.typeMismatch(Param.self, DecodingError.Context(codingPath: decoder.codingPath, debugDescription: "Wrong type for Param"))
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        switch self {
        case .bool(let x):
            try container.encode(x)
        case .integer(let x):
            try container.encode(x)
        case .string(let x):
            try container.encode(x)
        case .stringArray(let x):
            try container.encode(x)
        case .valueNil:
            try container.encodeNil()
        case .unsignedInteger(let x):
            try container.encode(x)
        case .optionalString(let x):
            x?.isEmpty == true ? try container.encodeNil() : try container.encode(x)
        }
    }
}

And the usage is something like this.

RequestBody.CustomBody(method: "WSDocMgmt.getDocumentsInContentCategoryBySearchSource", params: [.string(legacyToken), .string(shelfId), .bool(true), .valueNil, .stringArray(queryFrom(filters: filters ?? [])), .optionalString(sortMethodParameters()), .bool(sortMethodAscending()), .unsignedInteger(segment ?? 0), .unsignedInteger(segmentSize ?? 0), .string("NO_PATRON_STATUS")])

Answer 4

I am using this enum to control the behavior. It was required by our backend:

public enum Tristate<Wrapped> : ExpressibleByNilLiteral, Encodable {

/// Null
case none

/// The presence of a value, stored as `Wrapped`.
case some(Wrapped)

/// Pending value, not none, not some
case pending

/// Creates an instance initialized with .pending.
public init() {
    self = .pending
}

/// Creates an instance initialized with .none.
public init(nilLiteral: ()) {
    self = .none
}

/// Creates an instance that stores the given value.
public init(_ some: Wrapped) {
    self = .some(some)
}

public func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()
    switch self {
        case .none:
            try container.encodeNil()
        case .some(let wrapped):
            try (wrapped as! Encodable).encode(to: encoder)
        case .pending: break // do nothing
    }
}

}

typealias TriStateString = Tristate<String>
typealias TriStateInt = Tristate<Int>
typealias TriStateBool = Tristate<Bool>

/// Test

struct TestStruct: Encodable {
var variablePending: TriStateString?
var variableSome: TriStateString?
var variableNil: TriStateString?

}

    /// Structure with tristate strings:
    let testStruc = TestStruct(/*variablePending: TriStateString(),*/ // pending, unresolved
                               variableSome: TriStateString("test"), // some, resolved
                               variableNil: TriStateString(nil)) // nil, resolved

    /// Make the structure also tristate
    let tsStruct = Tristate<TestStruct>(testStruc)

    /// Make a json from the structure
    do {
        let jsonData = try JSONEncoder().encode(tsStruct)
        print( String(data: jsonData, encoding: .utf8)! )
    } catch(let e) {
        print(e)
    }

/// Output

{"variableNil":null,"variableSome":"test"}

// variablePending is missing, which is a correct behaviour

Answer 5

I ran into the same problem. Solved it by creating a dictionary from the struct without using JSONEncoder. You can do this in a relatively universal way. Here's my code:

struct MyStruct: Codable {
    let id: String
    let regionsID: Int?
    let created: Int
    let modified: Int
    let removed: Int?


    enum CodingKeys: String, CodingKey, CaseIterable {
        case id = "id"
        case regionsID = "regions_id"
        case created = "created"
        case modified = "modified"
        case removed = "removed"
    }

    var jsonDictionary: [String : Any] {
        let mirror = Mirror(reflecting: self)
        var dic = [String: Any]()
        var counter = 0
        for (name, value) in mirror.children {
            let key = CodingKeys.allCases[counter]
            dic[key.stringValue] = value
            counter += 1
        }
        return dic
    }
}

extension Array where Element == MyStruct {
    func jsonArray() -> [[String: Any]] {
        var array = [[String:Any]]()
        for element in self {
            array.append(element.jsonDictionary)
        }
        return array
    }
}

You can do this without the CodingKeys (if the table attribute names on server side are equal to your struct property names). In that case just use the 'name' from mirror.children.

If you need CodingKeys don't forget to add the CaseIterable protocol. That makes it possible to use the allCases variable.

Be careful with nested structs: E.g. if you have a property with a custom struct as type, you need to convert that to a dictionary too. You can do this in the for loop.

The Array extension is required if you want to create an array of MyStruct dictionaries.

Answer 6

As mentionned by @Peterdk, a bug report has been created on this issue :

https://bugs.swift.org/browse/SR-9232

Feel free to up-vote it, if you want to insist on how this feature should be part of the official API in future releases.

And, as mentioned (by Johan Nordberg) in this bug report, there is a library FineJson that can handle this issue without having to rewrite each encode(to:) implementation for all your codable structs ^^

Here is an example to show how I used this library to be able to encode a NULL value in a JSON payload requested by my application's backend :

import Foundation
import FineJSON

extension URLRequest {

    init<T: APIRequest>(apiRequest: T, settings: APISettings) {

        // early return in case of main conf failure
        guard let finalUrl = URL(string: apiRequest.path, relativeTo: settings.baseURL) else {
            fatalError("Bad resourceName: \(apiRequest.path)")
        }

        // call designated init
        self.init(url: finalUrl)

        var parametersData: Data? = nil
        if let postParams = apiRequest.postParams {
            do {
                // old code using standard JSONSerializer :/
                // parametersData = try JSONSerializer.encode(postParams)

                // new code using FineJSON Encoder
                let encoder = FineJSONEncoder.init()

                // with custom 'optionalEncodingStrategy' ^^
                encoder.optionalEncodingStrategy = .explicitNull

                parametersData = try encoder.encode(postParams)

                // set post params
                self.httpBody = parametersData

            } catch {
                fatalError("Encoding Error: \(error)")
            }
        }

        // set http method
        self.httpMethod = apiRequest.httpMethod.rawValue

        // set http headers if needed
        if let httpHeaders = settings.httpHeaders {
            for (key, value) in httpHeaders {
                self.setValue(value, forHTTPHeaderField: key)
            }
        }
    }
}

These are the only changes I had to perform to handle this issue.

Thank you Omochi for this great lib ;)

Hope that helps...