handle large arrays/textures in fragment shader

Question

I am trying to pass a large amount of information to my fragment shader but I always reach a limit (too many textures binded, texture too large, etc., array too large, etc.). I use a ThreeJS custom shader.

I have a 256*256*256 rgba volume that I want to pass to my shader.

In my shader, I want to map the fragments's world position to a voxel in this 256*256*256 volume.

Is there a good strategy to deal with this amount of information? Which would be the best pratice? Is there any good workaround?

My current approach is to generate 4 different 2048x2048 rgba texture containing all the data I need.

To create each 2048x2048 texture, I just push every row of every slice sequencially to a big array and split this array in 2048x2048x4 chuncks, which are my textures:

  var _imageRGBA = new Uint8Array(_dims[2] *_dims[1] * _dims[0] * 4);
  for (_k = 0; _k < _dims[2]; _k++) {
    for (_j = 0; _j < _dims[1]; _j++) {
      for (_i = 0; _i < _dims[0]; _i++) {
      _imageRGBA[4*_i + 4*_dims[0]*_j + 4*_dims[1]*_dims[0]*_k] = _imageRGBA[4*_i + 1 + 4*_dims[0]*_j + 4*_dims[1]*_dims[0]*_k] = _imageRGBA[4*_i + 2 + 4*_dims[0]*_j + 4*_dims[1]*_dims[0]*_k] = _imageN[_k][_j][_i];//255 * i / (_dims[2] *_dims[1] * _dims[0]);
      _imageRGBA[4*_i + 3 + 4*_dims[0]*_j + 4*_dims[1]*_dims[0]*_k] = 255;
      }
    }
  }

Each texture looks something like that:

On the shader side, I try to map a fragment's worldposition to an actual color from the texture:

Vertex shader:

uniform mat4 rastoijk;

varying vec4 vPos;
varying vec2 vUv;

void main() {
    vPos = modelMatrix * vec4(position, 1.0 );
    vUv = uv;
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0 );

}
</script>

Fragment shader:

<script id="fragShader" type="shader">

vec4 getIJKValue( sampler2D tex0, sampler2D tex1, sampler2D tex2, sampler2D tex3, vec3 ijkCoordinates, vec3 ijkDimensions) {

    // IJK coord to texture
    float textureSize = 2048.0;
    float index = ijkCoordinates[0] + ijkCoordinates[1]*ijkDimensions[0] + ijkCoordinates[2]*ijkDimensions[0]*ijkDimensions[1];

    // map index to right 2048 x 2048 slice
    float sliceIndex = floor(index / (textureSize*textureSize));
    float inTextureIndex = mod(index, textureSize*textureSize);

    // get row in the texture
    float rowIndex = floor(inTextureIndex/textureSize);
    float colIndex = mod(inTextureIndex, textureSize);

    // map indices to u/v
    float u = colIndex/textureSize;
    float v =1.0 - rowIndex/textureSize;

    vec2 uv = vec2(u,v);
    vec4 ijkValue = vec4(0, 0, 0, 0);
    if(sliceIndex == float(0)){
      ijkValue = texture2D(tex0, uv);
    }
    else if(sliceIndex == float(1)){
      ijkValue = texture2D(tex1, uv);
    }
    else if(sliceIndex == float(2)){
      ijkValue = texture2D(tex2, uv);
    }
    else if(sliceIndex == float(3)){
      ijkValue = texture2D(tex3, uv);
    }

    return ijkValue;
  }

uniform mat4 rastoijk;
uniform sampler2D ijk00;
uniform sampler2D ijk01;
uniform sampler2D ijk02;
uniform sampler2D ijk03;
uniform vec3 ijkDimensions;

varying vec4 vPos;
varying vec2 vUv;

void main(void) {
    // get IJK coordinates of current element
    vec4 ijkPos = rastoijk * vPos;
    // show whole texture in the back...
    vec3 color = texture2D(ijk00, vUv).rgb;
    //convert IJK coordinates to texture coordinates
    if(int(floor(ijkPos[0])) > 0
      && int(floor(ijkPos[1])) > 0
      && int(floor(ijkPos[2])) > 0
      && int(floor(ijkPos[0])) < int(ijkDimensions[0])
      && int(floor(ijkPos[1])) < int(ijkDimensions[1])
      && int(floor(ijkPos[2])) < int(ijkDimensions[2])){

      // try to map IJK to value...
      vec3 ijkCoordinates = vec3(floor(ijkPos[0]), floor(ijkPos[1]), floor(ijkPos[2]));
      vec4 ijkValue = getIJKValue(ijk00, ijk01, ijk02, ijk03, ijkCoordinates, ijkDimensions);
      color = ijkValue.rgb;
    }

    gl_FragColor = vec4(color, 1.0);
    // or discard if not in IJK bounding box...
  }
</script>

That doesn't work well. I now get an image with weird artifacts (nyquist shannon effect?). As I zoom in, the image appears. (even though not perfect, some black dots)

Any help advices would be greatly appreciated. I also plan to do some raycasting for volume rendering using this approach (very needed in the medical field)

Best,

Answer 1

The approach to handle large arrays using multiple textures was fine.

The issue was how I was generating the texture with THREE.js.

The texture was generated using the default linear interpolation: http://threejs.org/docs/#Reference/Textures/DataTexture

What I needed was nearest neighboor interpolation. This was, the texture is still pixelated and we can access the real IJK value (not an interpolated value) Found it there: http://www.html5gamedevs.com/topic/8109-threejs-custom-shader-creates-weird-artifacts-space-between-faces/

texture = new THREE.DataTexture( textureData, tSize, tSize, THREE.RGBAFormat, THREE.UnsignedByteType, THREE.UVMapping,
      THREE.ClampToEdgeWrapping, THREE.ClampToEdgeWrapping, THREE.NearestFilter, THREE.NearestFilter );

Thanks