3

I'm probably missing something obvious, but i'm experimenting with gpu.js and getting some strange results. I just want to make sure i'm not doing something obviously stupid (which is likely).

Not sure if this is an issue with what i'm doing, or the way in which calculations are performed when done via gpu.js using WebGL.

I create a new GPU and new kernel:

const gpu = new GPU();
const test = gpu.createKernel(function () { 
    return 255 + 
        (255 * 256) + 
        (255 * 256 * 256) + 
        (255 * 256 * 256 * 256); 
}).setOutput([1]);

const res = test();

This gives me a result of 4294967296 (contained in a float32array).

If i run the same calculation from the console i get a result of 4294967295.

Nick Hingston
  • 8,724
  • 3
  • 50
  • 59
llihp
  • 195
  • 1
  • 2
  • 10

1 Answers1

5

An IEEE 754 single precision (32 bit) floating point value consists of 24 significant bits and 8 exponent bits.

4294967295 is 0xffffffff (integral) which can't be stored in a 32 bit float with the full accuracy, because it only has 24 significant bits.
4294967296 is 0x100000000 (integral) which can be stored in a 32bit float, because it is 0x4f800000 (floating point).

In compare, an IEEE 754 double precision (64 bit) floating point value consists of 53 significant bits and 11 exponent bits.
Therefore, a 64 bit floating point value, can store the value 4294967295 exactly (0x41efffffffe00000).

Rabbid76
  • 202,892
  • 27
  • 131
  • 174
  • Is there a way to use double precision float in gpu.js? Trying to do some cosmological simulation. – martian17 Jan 07 '19 at 19:09
  • @Y.Yoshii In GLSL it is possible to use `double`, since version 4.00 ore using the extension [`ARB_gpu_shader_fp64`](https://www.khronos.org/registry/OpenGL/extensions/ARB/ARB_gpu_shader_fp64.txt). See also [GLSL specification 4.6; Chapter 4.1. Basic Types; page 22](https://www.khronos.org/registry/OpenGL/specs/gl/GLSLangSpec.4.60.pdf). In GLSL ES and WebGL it is not possible so far. – Rabbid76 Jan 07 '19 at 19:29