0

I'm doing a color lookup using a texture to apply an effect to a picture. My lookup is a gradient map using the luminance of the fragment of the first texture, then looking that up on a second texture. The 2nd texture is 256x256 with gradients going horizontally and several different gradients top to bottom. So 32 horizontal stripes each 8 pixels tall. My lookup on the x is the luminance, on the y it's a gradient and I target the center of the stripe to avoid crossover.

My fragment shader looks like this:

 lowp vec4 source = texture2D(u_textureSampler, v_fragmentTexCoord0);
 float luminance = 1.0 - dot(source.rgb, W);
 lowp vec2 texPos;
 texPos.x = clamp(luminance, 0.0, 1.0);
 // the y value selects which gradient to use by supplying a T value
 // this would be more efficient in the vertex shader
 texPos.y = clamp(u_value4, 0.0, 1.0);

 lowp vec4 newColor1 = texture2D(u_textureSampler2, texPos);

It works good but I was getting distortion in the whitest parts of the whites and the blackest part of the blacks. Basically it looked like it grabbed that newColor from a completely different place on texture2, or possibly was just getting nothing for those fragments. I added the clamps in the shader to try to keep it from getting outside the edge of the lookup texture but that didn't help. Am I not using clamp correctly?

Finally I considered that it might have something to do with my source texture or the way it's loaded. I ended up fixing it by adding:

    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

So.. WHY?

It's a little annoying to have to clamp the textures because it means I have to write an exception in my code when I'm loading lookup tables..

If my textPos.x and .y are clamped to 0-1.. how is it pulling a sample beyond the edge?

Also.. do I have to use the above clamp call when creating the texture or can I call it when I'm about to use the texture?

badweasel
  • 2,349
  • 1
  • 19
  • 31

1 Answers1

4

This is correct behavior of texture sampler.

Let me explain this. When you use textures with GL_LINEAR sampling GPU will take an average color of pixel blended with nearby pixels (that's why you don't see pixelation as with GL_NEAREST mode - pixels are blurred instead). And with GL_REPEAT mode texture coordinates will wrap from 0 to 1 and vice versa, blending with nearby pixels (i.e. in extreme coordinates it will blend with opposite side of texture). GL_CLAMP_TO_EDGE prevents this wrapping behavior, and pixels won't blend with pixels from opposite side of texture.

Hope my explanation is clear.

keaukraine
  • 5,315
  • 29
  • 54
  • So it's impossible to clamp within the shader then if you're in GL_LINEAR? It has to be clamped with the opengl call? What about that glTexParameteri call? Can it be done when you bind the texture to use it or does it have to be done when you load the texture? – badweasel Sep 23 '13 at 21:13
  • @badweasel You ***can*** implement clamping in the shader, what `GL_CLAMP_TO_EDGE` *actually* does is restrict the coordinate range to [0+0.5/texSize,1-0.5/texSize]. This causes sampling at the ***exact*** texel center, so that interpolation using neighboring texels never occurs. – Andon M. Coleman May 09 '14 at 16:22
  • @badweasel This special guarantee of course only applies when sampling edge texels, hence the name. – Andon M. Coleman May 09 '14 at 16:26
  • @Andon I'll have to try that. My experiments showed that a textel of 1.0 with gl_repeat interpolated with the textel a on the opposite side. – badweasel May 09 '14 at 19:00
  • 3
    @badweasel: That is correct, and that is why you don't want to use **1.0**. A value of **1.0** actually lies on the exact edge of your texture. It is equidistant from the **center** of the first and last texel in your texture, so interpolation will weight both of them equally. `GL_CLAMP_TO_EDGE` moves the coordinate **1.0** so that it references the center of the the last texel, as such during interpolation the last texel has a weight of **1.0** so it's the only thing that shows up. – Andon M. Coleman May 09 '14 at 19:10