how to describe packed_float3 in Metal vertex shader MTLVertexAttributeDescriptor?

Question

I am passing an array of structs to my Metal shader vertex function. The struct looks like this:

struct Vertex {

  var x,y,z: Float     // position data
  var r,g,b,a: Float   // color data
  var s,t: Float       // texture coordinates
  var nX,nY,nZ: Float  // normal

  func floatBuffer() -> [Float] {
    return [x,y,z,r,g,b,a,s,t,nX,nY,nZ]
  }

};

The floatBuffer function is used to assemble the vertices into one big array of Floats. I am able to pass this into my shader function by using a struct definition which uses "packed" data types, like this:

struct VertexIn {
    packed_float3 position;
    packed_float4 color;
    packed_float2 texCoord;
    packed_float3 normal;
};


vertex VertexOut basic_vertex(
                          const device VertexIn* vertex_array [[ buffer(0) ]],
.
.
.

This works. However, I would like to know how to do the same thing using MTLVertexAttributeDescriptors and the associated syntax. Right now I am getting mangled polygons, presumably because of the byte alignment differences with float3 and packed_float3?

This is how I'm trying to define it now and getting the garbage polygons. I got an error that "packed_float3" is not valid for attributes, so I was trying to figure out how to make regular float3, float4, etc work.

struct VertexIn {
  float3 position [[attribute(RayVertexAttributePosition)]];
  float4 color [[attribute(RayVertexAttributeColor)]];
  float2 texCoord [[attribute(RayVertexAttributeTexCoord)]];
  float3 normal [[attribute(RayVertexAttributeNormal)]];
};

class func buildMetalVertexDescriptor() -> MTLVertexDescriptor {

    let mtlVertexDescriptor = MTLVertexDescriptor()
    var offset = 0

    mtlVertexDescriptor.attributes[RayVertexAttribute.position.rawValue].format = MTLVertexFormat.float3
    mtlVertexDescriptor.attributes[RayVertexAttribute.position.rawValue].offset = offset
    mtlVertexDescriptor.attributes[RayVertexAttribute.position.rawValue].bufferIndex = RayBufferIndex.positions.rawValue
    offset += 3*MemoryLayout<Float>.stride

    mtlVertexDescriptor.attributes[RayVertexAttribute.color.rawValue].format = MTLVertexFormat.float4
    mtlVertexDescriptor.attributes[RayVertexAttribute.color.rawValue].offset = offset
    mtlVertexDescriptor.attributes[RayVertexAttribute.color.rawValue].bufferIndex = RayBufferIndex.positions.rawValue
    offset += MemoryLayout<float4>.stride

    mtlVertexDescriptor.attributes[RayVertexAttribute.texCoord.rawValue].format = MTLVertexFormat.float2
    mtlVertexDescriptor.attributes[RayVertexAttribute.texCoord.rawValue].offset = offset
    mtlVertexDescriptor.attributes[RayVertexAttribute.texCoord.rawValue].bufferIndex = RayBufferIndex.positions.rawValue
    offset += MemoryLayout<float2>.stride

    mtlVertexDescriptor.attributes[RayVertexAttribute.normal.rawValue].format = MTLVertexFormat.float3
    mtlVertexDescriptor.attributes[RayVertexAttribute.normal.rawValue].offset = offset
    mtlVertexDescriptor.attributes[RayVertexAttribute.normal.rawValue].bufferIndex = RayBufferIndex.positions.rawValue
    offset += 3*MemoryLayout<Float>.stride

    print("stride \(offset)")
    mtlVertexDescriptor.layouts[RayBufferIndex.positions.rawValue].stride = offset
    mtlVertexDescriptor.layouts[RayBufferIndex.positions.rawValue].stepRate = 1
    mtlVertexDescriptor.layouts[RayBufferIndex.positions.rawValue].stepFunction = MTLVertexStepFunction.perVertex


    return mtlVertexDescriptor
}

Notice that I specify the first attribute as a float3, but I specify an offset of 3 floats instead of the 4 that a float3 would normally use. But it isn't enough, apparently. I'm wondering how to set up a MTLVertexDescriptor and the shader struct with attributes so that it handles the 'packed' data from my structs?

Thanks very much.

All of your vertex descriptor properties look correct to me. Have you triple-checked that the enum definitions agree between your Swift and Metal code? — warrenm, Nov 02 '17 at 20:59
Yes, pretty sure -- the swift and metal use the same enums, from a .h file which has #if switches for Metal/Swift. It seems to me the problem is getting it to recognize that the float3/4/etc are packed data. — bsabiston, Nov 03 '17 at 15:09
Have you tried using GPU Frame Capture to see how Metal thinks the vertex data is laid out? It should be apparent by inspection whether the issue is packing/alignment-related or something else. — warrenm, Nov 03 '17 at 18:28
@bsabiston: For what it's worth, the section of the [Metal Programming Guide](https://developer.apple.com/library/content/documentation/Miscellaneous/Conceptual/MetalProgrammingGuide/Render-Ctx/Render-Ctx.html#//apple_ref/doc/uid/TP40014221-CH7-SW44) that talks about vertex descriptors shows an example with "mis-aligned" `float4` data. The misalignment is due to coming packed after a `float2` but there's no reason that should be different from your case of a `float3`. So, as Warren says, it should work. — Ken Thomases, Nov 04 '17 at 15:18
@warrenm Thanks! I had forgotten I could look at the vertex data in the frame capture. At first I thought it looked correct, but when I compare with the working version, they are in fact different. It does indeed look like it is skipping over the first color float, thinking that it is the fourth unseen component of the float3. Ken Thomases -- it could be different from a float2 because a float2 really does have just two floats, right? Whereas a float3 has four floats. But surely Metal is capable of converting? — bsabiston, Nov 04 '17 at 17:11

score 1 · Answer 1 · answered Sep 22 '18 at 02:05

The key is in this part of your question: "Notice that I specify the first attribute as a float3, but I specify an offset of 3 floats instead of the 4 that a float3 would normally use".

The SIMD float3 type takes up 16 bytes, it has the same memory layout as the non-packed Metal float3 type. So when you set the offset to only 3*MemoryLayout.stride you are missing the last 4 bytes which are still present causing the next field to pull from those extra bytes and for the rest of the data to be offset.

To really use packed types to transfer data to Metal (or any graphics API) you either have to stick with what you were doing before and specify x, y, z in three separate Floats in an array, or you have to define your own struct like this:

struct Vector3 {
  var x: Float
  var y: Float
  var z: Float
}

Swift doesn't have any guarantees that this struct will be three Floats packed closely together, but for now and the foreseeable future it works and will be 12 bytes in size on most platforms.

If you want to be able to do vector operations on a struct like this then I would suggest looking for a library that defines types like these to save yourself some time as you will run into the same types of problems with 3x3 matrices also.

I ran into the same problems so I ended up rolling my own: https://github.com/jkolb/Swiftish

how to describe packed_float3 in Metal vertex shader MTLVertexAttributeDescriptor?

1 Answers1