How does the encoded data end up in the Encoder, when it has no mutating functions?

Question

I am trying to code a custom Coder and I am having trouble understanding something, particularly about the Encoder part.

The Encoder is only required to have three functions and two variables :

struct MyEncoder : Encoder {
    public var codingPath : [CodingKey]
    public var userInfo : [CodingUserInfoKey : Any]
    public func container<Key>(keyedBy type: Key.Type -> KeyedEncodingContainer<Key> where Key : CodingKey {}
    public func unkeyedContainer() -> UnkeyedEncodingContainer {}
    public func singleValueContainer() -> SingleValueEncodingContainer {}
}

You will notice, none of them are mutating. They all return something and that's it.

The containers themselves, to generalise, have sub functions to encode specific types into themselves :

struct MySingleValueContainer : SingleValueEncodingContainer {
    var codingPath: [CodingKey]
    mutating func encode(_ value : someType) throws {}
    /// etc with a lot of different types
}

If a user wants to customise how their class is encoded, they may do this :

func encode(to: Encoder) throws {}

And here is my problem ! The encoder persists after the call to encode(to: Encoder) but it does not mutate so how does the encoded data end up inside it ? Reading the JSON Encoder's source I can see that their containers have a variable containing an Encoder, which is passed down. But it should be copy-on-write as everything else in Swift, which means the data shouldn't be able to make its way back up the chain !

Is the Encoder somehow secretly passed as a reference instead of as a value ? That seems like the only logical conclusion but it seems very weird to me... If not, what is going on ?

I have also taken a look at this question and its answer, and although they have helped me they display the same behaviour of magically bringing data up the chain, except it passes down a custom type instead of the Encoder itself.

Let me ask you (just in case you have this misunderstanding), in the case of `JSONEncoder`, which concrete implementation of `Encoder` does it use? (Note that `JSONEncoder` does not conform to `Encoder`) — Sweeper, Mar 11 '22 at 14:38
@LeoDabus if you read the last paragraph, I have mentioned this question and explained why it doesn’t help for this particular problem — ice-wind, Mar 11 '22 at 15:00
@Sweeper the JSONEncoder uses JSONEncoderImpl which does conform to the protocol — ice-wind, Mar 11 '22 at 15:01
So what exactly are you confused about? `JSONEncoderImpl` is a class, so it is always pass by reference anyway...? — Sweeper, Mar 11 '22 at 15:03

Itai Ferber · Accepted Answer · 2022-03-11T14:43:18.887

Is the Encoder somehow secretly passed as a reference instead of as a value?

Close, but not quite. JSONEncoder and its underlying containers are relatively thin veneer interfaces for reference-based storage. In the swift-corelibs-foundation implementation used on non-Darwin platforms, this is done using RefArray and RefObject to store keyed objects; on Darwin platforms, the same is done using NSMutableArray and NSMutableDictionary. It's these reference objects that are passed around and shared, so while the individual Encoders and containers can be passed around, they're writing to shared storage.

But it should be copy-on-write as everything else in Swift, which means the data shouldn't be able to make its way back up the chain!

One more note on this: all of the relevant types internal to this implementation are all classes (e.g. JSONEncoderImpl in swift-corelibs-foundation, _JSONEncoder on Darwin), which means that they wouldn't be copy-on-write, but rather, passed by reference anyway.

Regarding your edit: this is the same approach that the linked answer uses with its fileprivate final class Data — it's that class which is shared between all of the internal types and passed around.

As an addition to this, you may find the [Flight School Guide to Swift Codable](https://flight.school/books/codable/) useful — it contains a pretty good walkthrough of how to write a basic `Encoder` and might help illuminate a few things. — Itai Ferber, Mar 11 '22 at 14:40
Thank you ! I had completely forgotten that classes are passed as reference. It completely makes sense now, I think I’m actually going to be able to finish my custom one. Definitely will take a look at what you linked also, resources about this topic are surprisingly rare. — ice-wind, Mar 11 '22 at 15:03

score 0 · Answer 2 · answered Mar 11 '22 at 15:00

JSONEncoder.encode uses JSONEncoderImpl as the concrete implementation of Encoder (source):

open func encode<T: Encodable>(_ value: T) throws -> Data {
    let value: JSONValue = try encodeAsJSONValue(value)
    ...
}

func encodeAsJSONValue<T: Encodable>(_ value: T) throws -> JSONValue {
    let encoder = JSONEncoderImpl(options: self.options, codingPath: [])
    guard let topLevel = try encoder.wrapEncodable(value, for: nil) else {
        ...
    }

    return topLevel
}

wrapEncodable would then eventually call encode(to: Encoder) on your Codable type with self as the argument.

Is the Encoder somehow secretly passed as a reference instead of as a value ?

Notice that JSONEncoderImpl is a class, so there is nothing "secret" about this at all. The encoder is passed as a reference in this particular case.

In general though, the Encoder implementation could also be a struct, and be passed by value. After all, all it needs to do is to provide mutable containers into which data can be encoded. As long as you have a class somewhere down the line, it is possible to "seemingly" mutate a struct without mutating or making the struct a var. Consider:

private class Bar {
    var magic: Int = -1
}

struct Foo: CustomStringConvertible {
    private let bar = Bar()
    
    var description: String { "\(bar.magic)" }
    
    func magicallyMutateMyself() {
        bar.magic = Int.random(in: 0..<100)
    }
}

let foo = Foo()
print(foo) // -1
foo.magicallyMutateMyself()
print(foo) // some other number

How does the encoded data end up in the Encoder, when it has no mutating functions?

2 Answers2