MTLTexture from MTLBuffer issue

Question

I have results of compute shader stored in MTLBuffer. Each element of MTLBuffer is a UInt16. I am trying to pass the results to a fragment shader that displays the result by interpreting the elements of MTLBuffer as color intensity between 0 to 255. I create an MTLTexture out of this MTLBuffer using following code and pass the texture to shader. But I think something is not correct in the process as the output is not correct. I am not sure if it is an issue with pixel format and/or format conversions.

    let textureDescriptor = MTLTextureDescriptor()
    textureDescriptor.textureType = .type2D
    textureDescriptor.pixelFormat = .r16Uint
    textureDescriptor.width = bufferWidth
    textureDescriptor.height = 256
    textureDescriptor.usage = [.shaderRead, .shaderWrite]


    let texture = buffer?.makeTexture(descriptor: textureDescriptor, offset: 0, bytesPerRow: bufferWidth*MemoryLayout<UInt16>.stride)

and here is fragment shader code,

   fragment half4 fragmentShaderDisplay (MappedVertex in [[ stage_in ]],
                                              texture2d<ushort, access::sample> lumaTexture  [[ texture(0) ]]
                                              )
{
    constexpr sampler s(t_address::clamp_to_edge, r_address::clamp_to_edge, min_filter::linear, mag_filter::linear);

    float luma = lumaTexture.sample(s, in.textureCoordinate).r/255.0;

    luma = clamp(0.0, luma, 1.0);

   return half4(luma, luma, luma, 1.h);
}

score 1 · Answer 1 · answered May 13 '18 at 21:31

In what sense do you want to interpret the buffer values as between 0 and 255? Their inherent range is 0 to 65535 and the range for a floating-point component would typically be 0.0 to 1.0. Neither of those is 0 to 255.

If you simply divide by 65535.0 instead of 255.0, you'll get what you seem to want, a value between 0.0 and 1.0.

Also, your call to clamp() seems wrong. Given the order of arguments you've written, you're clamping the constant value 0.0 to be between luma and 1.0. I think you want to clamp luma to be between 0.0 and 1.0.

As it happens, the way you've written things will mostly work. If luma is <= 0.0, 0.0 will be between luma and 1.0 and will be returned unmodified. If 0.0 < luma <= 1.0, then 0.0 is below the range, so clamp() will return the lower bound of the range, which is luma. The problem comes if luma > 1.0. In that case, the results of clamp() are undefined, as per the documentation.

Now, values greater than 1.0 shouldn't happen if you're dividing by the proper divisor. Indeed, there's no need to clamp at all.

I want to interpret the numbers between 0 and 255, even if they are higher than 255, I wish to clamp them to 255. — Deepak Sharma, May 14 '18 at 09:11
clamp(x, minVal, maxVal ) = min(max(x, minVal), maxVal). In that case, does the order of x and minVal matter? — Deepak Sharma, May 14 '18 at 09:18
The documentation for `clamp()` specifically says that its result is undefined if `minVal` is greater than `maxVal` or, in the terms used in your code `luma` is greater than 1.0. — Ken Thomases, May 14 '18 at 20:59

MTLTexture from MTLBuffer issue

1 Answers1