3

I'm using TWGL to create an NxN plane buffer which I then use as a mesh (twgl.primitives.createPlaneBufferInfo), with extrusion on the Z axis relative to lightness values in a video. It all looks pretty good, except that I am getting strange clipping artifacts in the (-X, -Y) and (X, Y) quadrants. See the attached image for an example...

enter image description here

I saw this post which raises similar questions, and suggests depth sorting but it's not obvious how I would even do this sorting given that relative depth is decided on the fly in the shader. There is also a suggestion to disable depth testing and enabling face culling, but that does not work. I assume there must be some sort of alternative since what I'm doing isn't particularly uncommon. Perhaps I need to generate the mesh differently? It's also not obvious to me why this would only happen in two of the four quadrants. Thanks in advance!

baadcafe
  • 63
  • 10
  • What happens if you increase the lowest brightness level of the video, so that it never is 0? For example, if the video brightness values for each pixel is between 0 and 255, you remap it to be between 1 and 255. I believe that the artifacts are due to the fact that the plane onto which the video is applied consists of many rectangles which again consist each of two triangles, and those two triangles are not in an exact plane (float rounding errors), when then the video brightness is applied, these rounding errors may cause some multiplication "errors" if multiplied by 0 (black=no height). – Daniel F Mar 27 '20 at 21:18
  • Hmm, I went for a super simple 'brightness' calculation (certainly I could convert to HSL but I was just trying to get this to work). I went ahead and added .01 to the Z value, so that it would never be 0. It didn’t seem to change anything. Is something like this what you had in mind? – baadcafe Mar 27 '20 at 23:48
  • gl_Position = perspectiveMatrixUniform * viewMatrixUniform * vec4((matrixUniform * vec3(position.xy, 1.0)).xy, (intensity.r + intensity.g + intensity.b)/3.0 + 0.01, 1.0); – baadcafe Mar 27 '20 at 23:50
  • Note that adding the .01 baseline Z value did not cause the issue to go away – baadcafe Mar 27 '20 at 23:50
  • It's also worth noting that the problem gets worse as I move the object further to the right or further down. It does not happen at all in the (-X, Y) or (X, -Y) quadrants... – baadcafe Mar 28 '20 at 16:25
  • What about adjusting the near and far clipping planes? Having them as close together as possible increases the depth resolution. – Daniel F Mar 28 '20 at 18:55
  • Hi, I tried making the far plane closer via m4.perspective. It didn't fix the problem. I dont think there is any way to adjust it globally for WebGL but I could be wrong. I am thinking this relates to the plane geometry itself... like an issue with the normals or something like that but I don't know where to look. – baadcafe Mar 29 '20 at 15:12
  • Max sure that when you get the webGL context you have not turned of depth testing, eg `gl = canavs.getContext("webgl", {depth: true})` The default is `true` so would only be a problem is it has been set to `false`. Check that you have set the correct depth function eg `gl.depthFunc(gl.LEQUAL);` Are you using `gl_fragDepth` in the fragment shader. If you are ensure that it gets the correct value. – Blindman67 Mar 29 '20 at 17:23
  • Hi, I'm using webgl, so I don't think gl_fragDepth is available in the shaders? After playing around with depth options, I tend to think it is not related to depth, unless there is something fundamental about my mesh and deformation logic that suppresses depth values in favor of some sort of ordering. – baadcafe Mar 29 '20 at 23:39
  • Today I watched a video on YT, and it reminded me of your issue. It's related to the z-buffer, maybe it can help you. https://youtu.be/x8TO-nrUtSI?t=335 the artifact on the T-Rex made me think of it. – Daniel F Apr 01 '20 at 13:10
  • Thanks for sharing. Yes, it definitly feels like something that would happen on janky old hardware. That said, the mystery is why the issue is so pronounced in the (-X,-Y) and (X,Y) quadrants, but not elsewhere. I suspect something is wrong with the mesh that TWGL produces. I'll be doing some experiments this weekend and will report back. – baadcafe Apr 04 '20 at 14:56

1 Answers1

1

After lots of experimentation, I finally found the issue. The perspective matrix that I was generating had a zero as the 16th array element (in other words position [4,4] in the X,Y,Z,W matrix) which was causing the depth values to be flattened.

I was generating my perspective matrix with twgl. Not sure why the implementation works this way (seems like a bug to me):

_public.perspectiveMatrix = twgl.m4.perspective(_public.POV, width/height, 0.0, 100);

This was an issue as I was setting the position in the vertex shader like so:

gl_Position = perspectiveMatrixUniform * viewMatrixUniform * vec4((matrixUniform * vec3(position.xy, 1.0)).xy, (intensity.r + intensity.g + intensity.b)/3.0 + 0.01, 1.0);

If I add the following line immediately after initializing the perspective matrix, everything works as expected:

_public.perspectiveMatrix[15] = 1.0;
baadcafe
  • 63
  • 10