What is the modern canonical way to support converting ContiguousBytes into both Array and "not-Array"?

Question

For example, without this second overload, loading an Array will yield "UnsafeRawBufferPointer.load out of bounds". Is there a way to handle both cases without overloads?

let bytes: [UInt8] = [1, 0, 1, 0]
bytes.load() as Int32 // 0x1_00_01
bytes.load() as [Int16] // [1, 1]

public extension ContiguousBytes {
  func load<T>(_: T.Type = T.self) -> T {
    withUnsafeBytes { $0.load(as: T.self) }
  }

  func load<Element>(_: [Element].Type = [Element].self) -> [Element] {
    withUnsafeBytes { .init($0.bindMemory(to: Element.self)) }
  }
}

I think it would be better to make your generic methods more restricted as it would result in undefined behavior for other kind of types. I have already implemented [something similar](https://stackoverflow.com/a/63022536/2303865) to those methods but the point is to avoid its misuse — Leo Dabus, Mar 24 '22 at 18:08
Never mind I forgot you like to create generic types using the same name instead of a single character. Btw Why not simply `func load() -> [Element] {`? — Leo Dabus, Mar 24 '22 at 18:14
The arguments allow you to pass a metatype in, the way they do it for e.g. Codable. I don't tend to use metatypes that way, but it won't an API unless we ever get generic properties. — , Mar 24 '22 at 18:24
Yes I know that it allows you to pass the type. I just don't see any advantage over explicitly setting the resulting type or casting when calling that method as you did. — Leo Dabus, Mar 24 '22 at 18:27
I don't either, but there's clearly a different opinion going around—otherwise we wouldn't have the terrible requirement of using metatypes with Codable. So I always write these functions like this. I may be making a stupid choice. — , Mar 24 '22 at 19:11

Itai Ferber · Accepted Answer · 2022-03-24T19:12:31.313

In terms of having two overloads: you unfortunately can't avoid them. ContiguousBytes.withUnsafeBytes gives you access to the underlying storage of a type which implements ContiguousBytes, so the buffer given by, say, [UInt8].withUnsafeBytes will be the actual buffer the array instance is using to store data, not a pointer to itself in memory (e.g. its length, capacity, storage pointer, etc.).

The reason your first overload can't work is that you'd be calling the equivalent of:

withUnsafeBytes { rawBuffer in
    rawBuffer.load(as: [Int16].self)
}

where load(as:) is attempting to read [1, 0, 1, 0] as if those contents were the length, capacity, and buffer pointer of a single [Int16] instance, but that's not the case — it's only the contents of bytes.

You need the second overload in order to create a new array with the contents of that UnsafeRawBufferPointer.

As for the actual implementation of your array variant, as of SE-0333 Expand usability of withMemoryRebound coming in Swift 5.7, the correct way to do this would be to use withMemoryRebound(to:_:), as the unconditional bindMemory(to:) is unsafe if the buffer was already bound to a different type. withMemoryRebound(to:_:) is now always safe, even if the buffer is bound, so long as you meet alignment requirements:

/// A raw buffer may represent memory that has been bound to a type.
/// If that is the case, then T must be layout compatible with the
/// type to which the memory has been bound. This requirement does not
/// apply if the raw buffer represents memory that has not been bound
/// to any type.

This would look like

func load<Element>(_: [Element].Type = [Element].self) -> [Element] {
    withUnsafeBytes { rawBuffer in
        rawBuffer.withMemoryRebound(to: Element.self) { typedBuffer in
            Array(typedBuffer)
        }
    }
}

UnsafeRawBufferPointer.withMemoryRebound(to:_:) will do the math on binding the raw bytes to the right stride of Element, though keep in mind that this is only valid to do if the alignment of the raw buffer matches the alignment requirements of Element — otherwise you'll either crash or trigger undefined behavior.

It's not, though. I'll keep an eye out for it, and mark this as the answer if it ends up working. In the meantime, your answer is inaccurate without the square brackets from my signature—this overload only applies to array metatypes. I also assume you can use `Array.init` directly instead of making a new closure…? — , Mar 24 '22 at 19:10
Indeed, I was wrong — I believe the proposal was originally slated for Swift 5.6, but will only be coming to Xcode in Swift 5.7. I tried an online Swift playground with Swift 5.6 and it did have `withMemoryRebound` (so I wasn't totally making this up! ) but looks like it didn't make the cut into Xcode 13.3 itself. — Itai Ferber, Mar 24 '22 at 19:13
@ItaiFerber yes Xcode 13.3 is Swift 5.6 so we will need to wait the next release — Leo Dabus, Mar 24 '22 at 19:16
@Jessy Sorry about that typo, updated the answer to explicitly apply the Array metatype — Itai Ferber, Mar 24 '22 at 19:18
Tried your code in Swift 5.7 using the trunk development toolchain. — , Mar 27 '22 at 01:00

What is the modern canonical way to support converting ContiguousBytes into both Array and "not-Array"?

1 Answers1