Render UIView contents into MTLTexture

Question

I have an (animated) UIView-Hierarchy and I want to periodically render the UIView content into a MTLTexture for further processing.

What I have tried, is to subclass my parent UIView and

override public class var layerClass: Swift.AnyClass {
  return CAMetalLayer.self
}

but the texture from nextDrawable() is black and does not show the view content.

Any ideas how to get a MTLTexture containing the view content ?

I wrote an answer years ago to cover this scenario for OpenGL ES apps, though I suspect the bottom line is nearly identical, perhaps it'd be a useful starting point: https://stackoverflow.com/a/15777434/373944. Here is a more recent answer that may also be of use: https://stackoverflow.com/questions/33844130/take-a-snapshot-of-current-screen-with-metal-in-swift — ldoogy, May 11 '20 at 07:44

score 6 · Accepted Answer · answered May 18 '20 at 05:00

Thanks to Matthijs Hollemanns who pointed me into the right direction with some code, I came up with the following UIView extension, which does the job in about 12 ms per frame on an iPhone8plus for a full screen resolution.

extension UIView {

   func takeTextureSnapshot(device: MTLDevice) -> MTLTexture? {
      let width = Int(bounds.width)
      let height = Int(bounds.height)

      if let context = CGContext(data: nil,
                                 width: width,
                                 height: height,
                                 bitsPerComponent: 8,
                                 bytesPerRow: 0,
                                 space: CGColorSpaceCreateDeviceRGB(),
                                 bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue),
        let data = context.data {

        layer.render(in: context)

        let desc = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm,
                                                            width: width,
                                                            height: height,
                                                            mipmapped: false)
        if let texture = device.makeTexture(descriptor: desc) {
          texture.replace(region: MTLRegionMake2D(0, 0, width, height),
                          mipmapLevel: 0,
                          withBytes: data,
                          bytesPerRow: context.bytesPerRow)
          return texture
        }
      }
      return nil
    }
}

Render UIView contents into MTLTexture

1 Answers1

Linked