Swift: Decode JSON response and store nested JSON as String or JSON

Question

Given the following JSON from a network request; If you wanted to decode this into a Swift object that coforms to Codable, but you wanted to retain the nested JSON that is the value for the key configuration_payload, how could you do it?

{
    "registration": {
        "id": "0000-0000-0000-0000-000",
        "device_type": "device",
        "state": "provisioning",
        "thing_uuid": 999999999,
        "discovery_timeout": 10,
        "installation_timeout": 90,
        "configuration_payload":
            {
                "title": "Some Title",
                "url": "https://www.someurl.com/",
                "category": "test",
                "views": 9999
            }
      }
}

Using the following Swift struct, I want to be able to grab the configuration_payload as a String.

public struct Registration: Codable {
    public enum State: String, Codable {
        case provisioning, provisioned
    }
    
    public let id, deviceType: String
    public let state: State
    public let error: String?
    public let thingUUID: Int?
    public let discoveryTimeout, installationTimeout: Int
    public let configurationPayload: String?
}

As far as I can tell, the JSONDecoder in Swift, sees the value for configuration_payload as nested JSON and wants to decode it into it's own object. To add to confusion, configuration_payload is not always going to return the same JSON structure, it will vary, so I can not create a Swift struct that I can expect and simply JSON encode it again when needed. I need to be able to store the value as a String to account for variations in the JSON under the configuration_payload key.

Can the data structure of the payload be determined by one of the other values in the root object? If yes, use generics or an enum with associated values. — vadian, Jul 13 '21 at 18:22
@vadian hey thanks for the comment. No unfortunately not, the JSON could change to be anything and even have more nested JSON. Hence why I am keen to have it stored as a String — Dom Bryan, Jul 14 '21 at 17:04

gcharita · Answer 1 · 2021-07-13T18:16:55.450

You can achieve decoding of a JSON object to [String: Any] by using a third party library like AnyCodable.

Your Registration struct will look like this:

public struct Registration: Codable {
    public enum State: String, Codable {
        case provisioning, provisioned
    }
    
    public let id, deviceType: String
    public let state: State
    public let error: String?
    public let thingUUID: Int?
    public let discoveryTimeout, installationTimeout: Int
    public let configurationPayload: [String: AnyCodable]?
}

and then you can convert [String: AnyCodable] type to [String: Any] or even to String:

let jsonString = """
{
    "id": "0000-0000-0000-0000-000",
    "device_type": "device",
    "state": "provisioning",
    "thing_uuid": 999999999,
    "discovery_timeout": 10,
    "installation_timeout": 90,
    "configuration_payload":
        {
            "title": "Some Title",
            "url": "https://www.someurl.com/",
            "category": "test",
            "views": 9999
        }
}
"""

let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
do {
    let registration = try decoder.decode(Registration.self, from: Data(jsonString.utf8))
    
    // to [String: Any]
    let dictionary = registration.configurationPayload?.mapValues { $0.value }

    // to String
    if let configurationPayload = registration.configurationPayload {
        let encoder = JSONEncoder()
        encoder.keyEncodingStrategy = .convertToSnakeCase
        let data = try encoder.encode(configurationPayload)
        let string = String(decoding: data, as: UTF8.self)
        print(string)
    }
} catch {
    print(error)
}

You shouldn't use `JSONSerialization`, use JSONEncoder. Note that unknown keys have been converted from snake case and should be converted back to snake case. — Sulthan, Jul 13 '21 at 17:59
@gcharita Thanks for your answer. I am going to give thiis an upvote as it is a working solution. However, I am trynig to avoid using a third party library where possible, so I will be marking another response as the answer. — Dom Bryan, Jul 14 '21 at 17:19

score 3 · Accepted Answer · answered Jul 13 '21 at 17:54

As others have already said, you cannot just keep a part without decoding. However, decoding unknown data is trivial:

enum RawJsonValue {
    case boolean(Bool)
    case number(Double)
    case string(String)
    case array([RawJsonValue?])
    case object([String: RawJsonValue])
}

extension RawJsonValue: Codable {
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()

        if let boolValue = try? container.decode(Bool.self) {
            self = .boolean(boolValue)
        } else if let numberValue = try? container.decode(Double.self) {
            self = .number(numberValue)
        } else if let stringValue = try? container.decode(String.self) {
            self = .string(stringValue)
        } else if let arrayValue = try? container.decode([RawJsonValue?].self) {
            self = .array(arrayValue)
        } else {
            let objectValue = try container.decode([String: RawJsonValue].self)
            self = .object(objectValue)
        }
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()

        switch self {
        case .boolean(let boolValue):
            try container.encode(boolValue)
        case .number(let numberValue):
            try container.encode(numberValue)
        case .string(let stringValue):
            try container.encode(stringValue)
        case .array(let arrayValue):
            try container.encode(arrayValue)
        case .object(let objectValue):
            try container.encode(objectValue)
        }
    }
}

Now we can safely decode and convert to JSON string if needed:

struct Registration: Codable {
    public enum State: String, Codable {
        case provisioning, provisioned
    }

    let id, deviceType: String
    let state: State
    let error: String?
    let thingUUID: Int?
    let discoveryTimeout, installationTimeout: Int
    let configurationPayload: RawJsonValue?
}

let jsonData = """
{
    "id": "0000-0000-0000-0000-000",
    "device_type": "device",
    "state": "provisioning",
    "thing_uuid": 999999999,
    "discovery_timeout": 10,
    "installation_timeout": 90,
    "configuration_payload":
        {
            "title": "Some Title",
            "url": "https://www.someurl.com/",
            "category": "test",
            "views": 9999
        }
}
""".data(using: .utf8)!

let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
let registration = try! decoder.decode(Registration.self, from: jsonData)

let encoder = JSONEncoder()
encoder.keyEncodingStrategy = .convertToSnakeCase

let payloadString = String(data: try! encoder.encode(registration.configurationPayload), encoding: .utf8)!
print(payloadString) // {"title":"Some Title","views":9999,"url":"https:\/\/www.someurl.com\/","category":"test"}

The only problem I can see is potential loss of precision when decoding decimal numbers, which is a known problem with Foundation JSON decoder. Also, some null values could be also removed. This could be fixed by decoding object manually by iterating keys and having a special null type.

I like this answer. It doesn't require me to create a custom JSONDecoder that I feared I may have too. If you get a chance, I and others would greatly appreciate an example of the `null` implementation. However, that was not the scope for my question and I will award your response as the correct answer. — Dom Bryan, Jul 14 '21 at 17:22
@DomBryan This is actually pretty similar to the answer that recommends to use AnyCodable library. However, I don't like AnyCodable because I see no reason to remove types and AnyCodable seems a bit overcomplicated. — Sulthan, Jul 14 '21 at 22:02
Agreed @Sulthan, I did upvote @gcharita answer that is based on `AnyCodable` as it is a viable option. I preferred your answer for the reasons you've mentioned and my previous comments. — Dom Bryan, Jul 15 '21 at 13:01

score 1 · Answer 3 · answered Jul 05 '21 at 15:11

One (more limited than you probably want) way would be to make sure that Value part in configuration_payload JSON is a known Codable single type (String) instead of Any which can produce multiple types (String, Int, Double etc.).

I was trying to make it work with [String: Any] for the configuration_payload, the problem is Any does NOT conform to Codable.

Then I tried with [String: String] for configuration_payload and was able to make it work like following.

public struct Registration: Codable {
    public enum State: String, Codable {
        case provisioning, provisioned
    }
    public let id, deviceType: String
    public let state: State
    public let thingUUID: Int?
    public let discoveryTimeout, installationTimeout: Int
    public let configurationPayload: [String: String]? // NOT [String: Any]?
    
    enum CodingKeys: String, CodingKey {
        case id = "id"
        case deviceType = "device_type"
        case state = "state"
        case thingUUID = "thing_uuid"
        case discoveryTimeout = "discovery_timeout"
        case installationTimeout = "installation_timeout"
        case configurationPayload = "configuration_payload"
    }
    
    public init(from decoder: Decoder) throws {
        let values = try decoder.container(keyedBy: CodingKeys.self)
        
        id = try values.decodeIfPresent(String.self, forKey: .id) ?? ""
        deviceType = try values.decodeIfPresent(String.self, forKey: .deviceType) ?? ""
        
        let stateRaw = try values.decodeIfPresent(String.self, forKey: .state) ?? ""
        state = Registration.State(rawValue: stateRaw) ?? .provisioning
        thingUUID = try values.decodeIfPresent(Int.self, forKey: .thingUUID)
        
        discoveryTimeout = try values.decodeIfPresent(Int.self, forKey: .discoveryTimeout) ?? 0
        installationTimeout = try values.decodeIfPresent(Int.self, forKey: .installationTimeout) ?? 0
        
        configurationPayload = try values.decodeIfPresent([String: String].self, forKey: .configurationPayload)
    }
}

Tests

let json = Data("""
{
    "id": "0000-0000-0000-0000-000",
    "device_type": "device",
    "state": "provisioning",
    "thing_uuid": 999999999,
    "discovery_timeout": 10,
    "installation_timeout": 90,
    "configuration_payload": {
        "title": "Some Title",
        "url": "https://www.someurl.com/",
        "category": "test",
        "views": "9999"
    }
}
""".utf8
)

let decoded = try JSONDecoder().decode(Registration.self, from: json)
print(decoded)

let encoded = try JSONEncoder().encode(decoded)
print(String(data: encoded, encoding: .utf8))

Hi Tarun, I appreciate the answer. I too also looked at this solution. The problem here is that we are limited to configuration payload having all String keys and values. So in my case, `views` key had a `Int` value of 9999, and this would be missed in when using this `[String: String]?` solution. But thank you anyway — Dom Bryan, Jul 06 '21 at 09:36

SeaSpell · Answer 4 · 2021-07-10T19:41:21.110

This is not possible with the Codable protocol, because you do not know the type before hand. You'll have to either write your own method or have a different decoding strategy.

let json = """
            {
                 "id": "0000-0000-0000-0000-000",
                 "device_type": "device",
                 "state": "provisioning",
                 "thing_uuid": 999999999,
                 "discovery_timeout": 10,
                 "installation_timeout": 90,
                 "configuration_payload": {
                       "title": "Some Title",
                       "url": "https://www.someurl.com/",
                       "category": "test",
                       "views": 9999
                       }
                  }
            
            """.data(using: .utf8)
            
            do {
                let decoded = try? Registration.init(jsonData: json!)
                print(decoded)
            }catch {
                print(error)
            }


public struct Registration {
    public enum State: String, Codable {
        case provisioning, provisioned
    }
    
    public let id: String
    public let device_type: String
    public let state: State
    public let error: String?
    public let thing_uuid: Int?
    public let discovery_timeout, installation_timeout: Int
    public let configuration_payload: [String: Any]?

    public init(jsonData: Data) throws {
        
        let package = try JSONSerialization.jsonObject(with: jsonData, options: []) as! [String : Any]
        
        id = package["id"] as! String
        device_type = package["device_type"] as! String
        state = State(rawValue: package["state"] as! String)!
        error = package["error"] as? String
        thing_uuid = package["thing_uuid"] as? Int
        discovery_timeout = package["discovery_timeout"] as! Int
        installation_timeout = package["installation_timeout"] as! Int
        configuration_payload = package["configuration_payload"] as? [String: Any]
    }
}

This is one possible way to handle the different types. You could also create a struct containing keys and loop through them, I think this illustrates the basic idea though.

Edit:

 if let remaining = package["configuration_payload"] as? Data,
            let data = try? JSONSerialization.data(withJSONObject: remaining, options: []) as Data,
            let string = String(data: data, encoding: .utf8) {
            // store your string if you want it in string formatt
            print(string)
        }

Thanks for the answer. I appreciate that this can't be done easily with Codable or using Swifts' foundation JSONDecoder. Unfortunately we can't use `[String: Any]` as the type for the configuration payload, just as you pointed out, Swift can not infer that type and Codable will complain about using `Any`. I do like the option to loop through the keys, however I am going to avoid marking this as the answer, as I believe there is a way to do this with a custom JSONDecoder. — Dom Bryan, Jul 14 '21 at 17:10

froggomad · Answer 5 · 2021-07-13T17:34:39.953

If you have a list of possible keys, using optionals is another way you could employ Codable. You can mix keys this way - only the ones that are available will attempt to be encoded/decoded

import UIKit

public struct Registration: Codable {
    public enum State: String, Codable {
        case provisioning, provisioned
    }
    
    public let id, deviceType: String
    public let state: State
    public let error: String?
    public let thingUuid: Int?
    public let discoveryTimeout, installationTimeout: Int
    public var configurationPayload: ConfigurationPayload?
}

// nested json can be represented as a codable struct
public struct ConfigurationPayload: Codable {
    
    let title: String?
    let url: String?
    let category: String?
    let views: Int?
    let nonTitle: String?
    let anotherUrl: String?
    let someCategory: String?
    let someViews: Int?
    // computed properties aren't part of the coding strategy
    // TODO: avoid duplication in loop
    var jsonString: String {
        
        let mirror = Mirror(reflecting: self).children
        let parameters = mirror.compactMap({$0.label})
        let values = mirror.map({$0.value})
        
        let keyValueDict = zip(parameters, values)

        var returnString: String = "{\n"        
        for (key, value) in keyValueDict {
            if let value = value as? Int {
                returnString.append("\"\(key)\": \"\(value)\n")
            } else if let value = value as? String {
                returnString.append("\"\(key)\": \"\(value)\n")
            }
            
        }
        returnString.append("}")
    
        return returnString
    }
}

// your json has a preceding key of "registration", this is the type you will decode
public struct RegistrationParent: Codable {
    var registration: Registration
}

let jsonDataA =
"""
{
    "registration": {
        "id": "0000-0000-0000-0000-000",
        "device_type": "device",
        "state": "provisioning",
        "thing_uuid": 999999999,
        "discovery_timeout": 10,
        "installation_timeout": 90,
        "configuration_payload":
            {
                "title": "Some Title",
                "url": "https://www.someurl.com/",
                "category": "test",
                "views": 9999
            }
      }
}
""".data(using: .utf8)!

let jsonDataB =
"""
{
    "registration": {
        "id": "0000-0000-0000-0000-000",
        "device_type": "device",
        "state": "provisioning",
        "thing_uuid": 999999999,
        "discovery_timeout": 10,
        "installation_timeout": 90,
        "configuration_payload":
            {
                "non_title": "Some Title",
                "another_url": "https://www.someurl.com/",
                "some_category": "test",
                "some_views": 9999
            }
      }
}
""".data(using: .utf8)!


let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase

do {
    var registrationA = try decoder.decode(RegistrationParent.self, from: jsonDataA)
    print(registrationA.registration.configurationPayload?.jsonString ?? "{}")
    var registrationB = try decoder.decode(RegistrationParent.self, from: jsonDataB)
    print(registrationB.registration.configurationPayload?.jsonString ?? "{}")
} catch {
    print(error)
}

I see now you won't always know the values of `configurationPayload` so this method probably won't work for your scenario... — froggomad, Jul 12 '21 at 04:01
Thanks for the answer. Yes unfortunately I won't know what configuration payload will be everytime — Dom Bryan, Jul 12 '21 at 08:43
@DomBryan I updated my answer in case you have a list of keys that could be present. Though the jsonString won't match exactly, the values will — froggomad, Jul 13 '21 at 14:55
Just updated using Mirror to get the actual keys and values, with the caveat that it wraps the value with Optional() in `returnString`. Optional values that aren't present are also getting printed, so this can definitely be improved upon — froggomad, Jul 13 '21 at 17:07
Got it unwrapped. The values need to be downcast to their type due to mirror providing values as `Any` — froggomad, Jul 13 '21 at 17:21
Honestly, I don't understand why do you encode JSON manually into a String when you already support `Codable`. Also note that you have converted keys from snake case when decoding and you would have to change them back to snake case when encoding. Also, encoding JSON like this would fail pretty easily on special characters which have to be escaped in JSON. — Sulthan, Jul 13 '21 at 18:07
@Sulthan thanks I didn't think about using the encoder to create the String, and also didn't consider the snake case. As far as special characters, that's an edge case that could be handled using Coding Keys. Again, this strategy only works if keys are known beforehand — froggomad, Jul 14 '21 at 06:30
@froggomad Special characters in values cannot be handled using coding keys. The simplest example is when some string value contains `"`. Then your naive encoding fails. You should never attempt to compose JSON values manually from strings. Always use an encoder. — Sulthan, Jul 14 '21 at 10:13
I appreciate the updated response @froggomad. I have given this an upvote as it does pose one option, but I will avoid marking it as the answer as I believe a custom JSONDecoder might be the correct approach. I shall provide an answer once I have sused it out. — Dom Bryan, Jul 14 '21 at 17:13

jatin fl · Answer 6 · 2021-07-05T11:54:45.057

here is configurationPayload is dictionary so your Registration struct look like below

struct Registration : Codable {

    let configurationPayload : ConfigurationPayload?
    let deviceType : String?
    let discoveryTimeout : Int?
    let id : String?
    let installationTimeout : Int?
    let state : String?
    let thingUuid : Int?

    enum CodingKeys: String, CodingKey {
            case configurationPayload = "configuration_payload"
            case deviceType = "device_type"
            case discoveryTimeout = "discovery_timeout"
            case id = "id"
            case installationTimeout = "installation_timeout"
            case state = "state"
            case thingUuid = "thing_uuid"
    }

    init(from decoder: Decoder) throws {
            let values = try decoder.container(keyedBy: CodingKeys.self)
            configurationPayload = ConfigurationPayload(from: decoder)
            deviceType = try values.decodeIfPresent(String.self, forKey: .deviceType)
            discoveryTimeout = try values.decodeIfPresent(Int.self, forKey: .discoveryTimeout)
            id = try values.decodeIfPresent(String.self, forKey: .id)
            installationTimeout = try values.decodeIfPresent(Int.self, forKey: .installationTimeout)
            state = try values.decodeIfPresent(String.self, forKey: .state)
            thingUuid = try values.decodeIfPresent(Int.self, forKey: .thingUuid)
    }

}

and your ConfigurationPayload look like this

struct ConfigurationPayload : Codable {

        let category : String?
        let title : String?
        let url : String?
        let views : Int?

        enum CodingKeys: String, CodingKey {
                case category = "category"
                case title = "title"
                case url = "url"
                case views = "views"
        }
    
        init(from decoder: Decoder) throws {
                let values = try decoder.container(keyedBy: CodingKeys.self)
                category = try values.decodeIfPresent(String.self, forKey: .category)
                title = try values.decodeIfPresent(String.self, forKey: .title)
                url = try values.decodeIfPresent(String.self, forKey: .url)
                views = try values.decodeIfPresent(Int.self, forKey: .views)
        }

}

Great answer! just a misspelling of `ConfigurationPayload` (it says `onfigurationPayload`) — Ayrton CB, Jul 05 '21 at 10:57
Hi Jatin. As I mentioned in my question "configuration_payload is not always going to return the same JSON structure". So I said that I can't create a type (ConfigurationPayload struct in this case), to decode to. — Dom Bryan, Jul 05 '21 at 10:57
@DomBryan can you please share possibly structures of `ConfigurationPayload`. — jatin fl, Jul 05 '21 at 11:00
@jatinfl it could be anything, even 1000 key value pairs long. I don't want to decode this into an object, I need to retain it as a String. I can not guarantee what the value in configuration payload will be. — Dom Bryan, Jul 05 '21 at 11:02
so you might go with key values there is no except way with codable using different key value. before Rob napier try that but not get success yet. https://github.com/rnapier/RNJSON — jatin fl, Jul 05 '21 at 11:05
I think @jatinfl I will need to create a custom JSONDecoder, but I want to get some options first — Dom Bryan, Jul 05 '21 at 11:37
You don't need a custom JSONDecoder. I've been exploring that in RNJSON to add more flexibility and allowing better round-tripping (avoiding float rounding, maintaining key ordder). But for this you just need a JSON type. https://stackoverflow.com/questions/65901928/swift-jsonencoder-encoding-class-containing-a-nested-raw-json-object-literal/65902852#65902852 — Rob Napier, Jul 10 '21 at 19:55

Swift: Decode JSON response and store nested JSON as String or JSON

6 Answers6

Tests

Linked