3

So I'm very familiar with the good old

Math.floor(Math.random() * (max - min + 1)) + min;

and this works very nicely with small numbers, however when numbers get larger this quickly becomes biased and only returns numbers one zero below it (for ex. a random number between 0 and 1e100 will almost always (every time I've tested, so several billion times since I used a for loop to generate lots of numbers) return [x]e99). And yes I waited the long time for the program to generate that many numbers, twice. By this point, it would be safe to assume that the output is always [x]e99 for all practical uses.

So next I tried this

Math.floor(Math.pow(max - min + 1, Math.random())) + min;

and while that works perfectly for huge ranges it breaks for small ones. So my question is how can do both - be able to generate both small and large random numbers without any bias (or minimal bias to the point of not being noticeable)?

Note: I'm using Decimal.js to handle numbers in the range -1e2043 < x < 1e2043 but since it is the same algorithm I displayed the vanilla JavaScript forms above to prevent confusion. I can take a vanilla answer and convert it to Decimal.js without any trouble so feel free to answer with either.

Note #2: I want to even out the odds of getting large numbers. For example 1e33 should have the same odds as 1e90 in my 0-1e100 example. But at the same time I need to support smaller numbers and ranges.

ElJay
  • 347
  • 4
  • 17
  • See [How do I add 1 to a big integer represented as a string in JavaScript?](https://stackoverflow.com/q/43614407/) – guest271314 Aug 09 '17 at 02:12
  • a billion is nothing compared to `1e100`. I cannot reproduce the result always being `1e99`. Example result: `Math.floor(Math.random() * (1e100 + 1));`, `8.942283537027985e+98` – ASDFGerte Aug 09 '17 at 02:15
  • @guest271314 not sure how that has any relevance to generating random numbers? – ElJay Aug 09 '17 at 02:22
  • Generate the random numbers as in values less than cause the "e" artifact of JavaScript, that is for numbers greater than `Number.MAX_SAFE_INTEGER`, then create a concatenated string containing the N digits – guest271314 Aug 09 '17 at 02:25
  • @ASDFGerte Just tested with your code and still got only e99's. And yes a billion is nothing compared to 1e100 but the whole point of generating a random number in that range is that it won't be biased. A billion different ones with nothing under e99 is a huge problem. Even if I generated only two random numbers if they were consistently [x]e99 that would be a huge issue. – ElJay Aug 09 '17 at 02:25
  • Yes, i read the question too fast. However, the point remains, i am not getting `[x]e99` all the time but subjectively proper random output. If i loop often enough i even got as low as `[x]e94` ones, as expected. I am testing on FF and chrome, maybe a node issue? – ASDFGerte Aug 09 '17 at 02:31
  • @ASDFGerte Could be a node issue, but even `[x]e[94-99]` is a huge issue. So instead of trying to prove my question pointless can I get any suggestions on a solution? – ElJay Aug 09 '17 at 02:35
  • @guest271314 that is already what I am doing, not sure if you are understanding the point of this question? – ElJay Aug 09 '17 at 02:36
  • No, you are not using numbers less than `Number.MAX_SAFE_INTEGER` else you would not be getting the artifact "e" in your output – guest271314 Aug 09 '17 at 02:36
  • 90% of the results should be in `[x]e99`, 99% in `[x]e99` to `[x]e98`, and so on. It is not surprising simple tests with a few hundred thousand or million tests dont get anything below `[x]e94`. Used [an online nodejs tool](https://repl.it/languages/nodejs) and could not reproduce. – ASDFGerte Aug 09 '17 at 02:37
  • @guest271314 Looks like I was completely right, you don't have the slightest clue what this question is asking. Where do I say anywhere that I'm having trouble with the Decimal.js part? This question is about generating random numbers not using the Decimal.js library, I only mentioned that I was using it in the first place to try to give as much information as I could. You don't even seem to understand that numbers can be in e notation under the max safe integer. Please stop flooding the comments. – ElJay Aug 09 '17 at 02:42
  • @ASDFGerte Considering that the goal would be 1% of the results being `[x]e99`, 2% being `[x]e[98-99]`, etc. those statistics are horrible. – ElJay Aug 09 '17 at 02:44
  • Why would that be the goal, that is not random. – ASDFGerte Aug 09 '17 at 02:45
  • Just did re-read Question. Is requirement to return random numbers without duplicates? Or are you trying to distribute random numbers using a pattern to form a specific graph? – guest271314 Aug 09 '17 at 02:47
  • @ASDFGerte wait what? By that logic a random number generator between 1 and 100 should generate 98 or 99 99% of the time. That makes no sense? Each number would have a 1% chance to be picked ideally in this example. – ElJay Aug 09 '17 at 02:48
  • Your `e` is an exponent... To elaborate, there is e.g. 0-9 (10 different) one digit but 10-99 (90 different) two digit numbers. – ASDFGerte Aug 09 '17 at 02:48
  • Note that "e" notation within a number in JavaScript is not the number _`e`_ – guest271314 Aug 09 '17 at 02:51
  • @ASDFGerte Yes that's exactly what the problem is. Because of that it almost always returns `[x]e99` because it's weighed so heavily. My goal would be to unweigh it and return any `[x]e[y]` in the range with an equal chance. – ElJay Aug 09 '17 at 02:51
  • But it is not weighted as bad as you suggest (on average each tenth attempt should not be `[x]e99`. If you want a different distribution, you could e.g. use `Math.random() * Math.pow(10, 100 * Math.random());`. Note that i am currently neglecting a lot of more minor problems, e.g. some precision issues. – ASDFGerte Aug 09 '17 at 02:53
  • @LJTalbot It appears that you need to build the graph that you are expecting. Create an array of random numbers without duplicates from 0-N, then perform your algorithm. There is no native method to achieve what you are trying to accomplish - that is without recursively calling a function that either includes or excludes the number at that x, y or other coordinate on the graph. – guest271314 Aug 09 '17 at 02:55
  • @ASDFGerte That solution doesn't work well with small numbers from my testing. And the whole basis of the question is a random number generator that works with both large and small numbers. – ElJay Aug 09 '17 at 02:57
  • @guest271314 I don't mention a graph anywhere. Creating the random numbers is my problem. Any algorithms that use random numbers are outside the scope of this question. In fact, all your suggestions thus far have been outside the scope of the question. – ElJay Aug 09 '17 at 02:59
  • @LJTalbot Your Question is generating random number with specific criteria. That indicates a graph. You can generate a string of numbers then select a given `.length` of characters from the string using `String` methods, for example, `.slice()` to get a specific range of random numbers without bias. – guest271314 Aug 09 '17 at 03:01

3 Answers3

2

Your Problem is Precision. That's the reason you use Decimal.js in the first place. Like every other Number in JS, Math.random() supports only 53 bit of precision (Some browser even used to create only the upper 32bit of randomness). But your value 1e100 would need 333 bit of precision. So the lower 280 bit (~75 decimal places out of 100) are discarded in your formula.

But Decimal.js provides a random() method. Why don't you use that one?

function random(min, max){
    var delta = new Decimal(max).sub(min);
    return Decimal.random( +delta.log(10) ).mul(delta).add(min);
}

Another "problem" why you get so many values with e+99 is probability. For the range 0 .. 1e100 the probabilities to get some exponent are

e+99  => 90%, 
e+98  =>  9%,
e+97  =>  0.9%,
e+96  =>  0.09%,
e+95  =>  0.009%,
e+94  =>  0.0009%,
e+93  =>  0.00009%,
e+92  =>  0.000009%,
e+91  =>  0.0000009%,
e+90  =>  0.00000009%,
and so on

So if you generate ten billion numbers, statistically you'll get a single value up to 1e+90. That are the odds.

I want to even out those odds for large numbers. 1e33 should have the same odds as 1e90 for example

OK, then let's generate a 10random in the range min ... max.

function random2(min, max){
    var a = +Decimal.log10(min), 
        b = +Decimal.log10(max);
    //trying to deal with zero-values. 
    if(a === -Infinity && b === -Infinity) return 0;  //a random value between 0 and 0 ;)
    if(a === -Infinity) a = Math.min(0, b-53);
    if(b === -Infinity) b = Math.min(0, a-53);

    return Decimal.pow(10, Decimal.random(Math.abs(b-a)).mul(b-a).add(a) );
}

now the exponents are pretty much uniformly distributed, but the values are a bit skewed. Because 101 to 101.5 10 .. 33 has the same probability as 101.5 to 102 34 .. 100

Thomas
  • 11,958
  • 1
  • 14
  • 23
  • I am using the one that Decimal.js provides, although I wasn't setting the number of sig figs for it. I will try that. – ElJay Aug 09 '17 at 03:04
  • Also I guess I have probably worded my question badly but I want to even out those odds for large numbers. 1e33 should have the same odds as 1e90 for example. – ElJay Aug 09 '17 at 03:05
  • @LJ So you do not want uniform randomness? The formula you used gives numbers uniformly at random, but note that most numbers between 0 and 9e99 are larger than 1e99. – Paul Aug 09 '17 at 03:46
  • @LJTalbot Wanting 1e33 to be as common as 1e90 is like generating numbers between 0 and 999, but wanting to get numbers between `0` and `9` 1/3 of the time, numbers between `10` and `99` 1/3 of the time and numbers between `100` and `999` the other 1/3 (whereas if you generated numbers uniformly at random they would fall between `100` and `999` 90% of the time). There is nothing wrong with wanting to do that, as long as you are well aware that is not uniform randomness. – Paul Aug 09 '17 at 03:48
  • @LJTalbot I've added a snippet that generates random values with uniformly distributed exponents – Thomas Aug 09 '17 at 04:01
  • @Paulpro that is exactly my problem, I need uniform randomness for small numbers but for larger ones I need uniform randomness for the exponents. – ElJay Aug 09 '17 at 04:02
  • @LJTalbot then maybe a single function doesn't fit your needs, or you have to better formulate how exactly the function should behave for small numbers, for big numbers, for the middle range, where each of these ranges is located on the scale and how to transition. – Thomas Aug 09 '17 at 04:05
  • @LJTalbot Okay, there are a few ways to do that, but you would need to decide what you want to consider to be large. You can also decide if you want a continuously decreasing probability as the number increases, or if you want a step function where you choose the range for the number and then choose the number uniformly within that range. A continuously decreasing probability is probably the simplest implementation but requires more work to come up with and prove correctness. – Paul Aug 09 '17 at 04:06
  • @LJTalbot For a step function, you could use strings to build the number and then give the string to Decimal.js to be able to use it like a number. One algorithm would look something like this: 1) Choose the base10 length (exponent) of the number uniformly at random: `let exponent = Math.floor(Math.random() * (log10(max) - min + 1)) + min;`. 2) Choose a start digit for the number uniformly from `1 - 9`; 3) loop from` i = 1` up to `exponent` and choose each digit uniformly from `0-9` concatenating it to the string. 4) Give the string to Decimal.js constructor – Paul Aug 09 '17 at 04:11
  • I believe that would give you approximately what you want for large numbers. You would need to choose the breakpoint where you want it to behave differently for small numbers depending on your need. – Paul Aug 09 '17 at 04:12
1

The issue with Math.random() * Math.pow(10, Math.floor(Math.random() * 100)); at smaller numbers is that random ranges [0, 1), meaning that when calculating the exponent separately one needs to make sure the prefix ranges [1, 10). Otherwise you want to calculate a number in [1eX, 1eX+1) but have e.g. 0.1 as prefix and end up in 1eX-1. Here is an example, maxExp is not 100 but 10 for readability of the output but easily adjustable.

let maxExp = 10;

function differentDistributionRandom() {
  let exp = Math.floor(Math.random() * (maxExp + 1)) - 1;
  if (exp < 0) return Math.random();
  else return (Math.random() * 9 + 1) * Math.pow(10, exp);
}

let counts = new Array(maxExp + 1).fill(0).map(e => []);
for (let i = 0; i < (maxExp + 1) * 1000; i++) {
  let x = differentDistributionRandom();
  counts[Math.max(0, Math.floor(Math.log10(x)) + 1)].push(x);
}

counts.forEach((e, i) => {
  console.log(`E: ${i - 1 < 0 ? "<0" : i - 1}, amount: ${e.length}, example: ${Number.isNaN(e[0]) ? "none" : e[0]}`);
});

You might see the category <0 here which is hopefully what you wanted (the cutoff point is arbitrary, here [0, 1) has the same probability as [1, 10) as [10, 100) and so on, but [0.01, 0.1) is again less likely than [0.1, 1))

If you didn't insist on base 10 you could reinterpret the pseudorandom bits from two Math.random calls as Float64 which would give a similar distribution, base 2:

function exponentDistribution() {
  let bits = [Math.random(), Math.random()];
  let buffer = new ArrayBuffer(24);
  let view = new DataView(buffer);
  
  view.setFloat64(8, bits[0]);
  view.setFloat64(16, bits[1]);
  
  //alternatively all at once with setInt32
  for (let i = 0; i < 4; i++) {
    view.setInt8(i, view.getInt8(12 + i));
    view.setInt8(i + 4, view.getInt8(20 + i));
  }
  
  return Math.abs(view.getFloat64(0));
}

let counts = new Array(11).fill(0).map(e => []);

for (let i = 0; i < (1 << 11) * 100; i++) {
  let x = exponentDistribution();
  let exp = Math.floor(Math.log2(x));
  if (exp >= -5 && exp <= 5) {
    counts[exp + 5].push(x);
  }
}

counts.forEach((e, i) => {
  console.log(`E: ${i - 5}, amount: ${e.length}, example: ${Number.isNaN(e[0]) ? "none" : e[0]}`);
});

This one obviously is bounded by the precision ends of Float64, there are some uneven parts of the distribution due to some details of IEEE754, e.g. denorms/subnorms and i did not take care of special values like Infinity. It is rather to be seen as a fun extra, a reminder of the distribution of float values. Note that the loop does 1 << 11 (2048) times a number iterations, which is about the exponent range of Float64, 11 bit, [-1022, 1023]. That's why in the example each bucket gets approximately said number (100) hits.

ASDFGerte
  • 4,695
  • 6
  • 16
  • 33
0

You can create the number in increments less than Number.MAX_SAFE_INTEGER, then concatenate the generated numbers to a single string

const r = () => Math.floor(Math.random() * Number.MAX_SAFE_INTEGER);

let N = "";

for (let i = 0; i < 10; i++) N += r();

document.body.appendChild(document.createTextNode(N));

console.log(/e/.test(N));
guest271314
  • 1
  • 15
  • 104
  • 177
  • This has nothing to do with my question. – ElJay Aug 09 '17 at 03:03
  • @LJTalbot Yes, it does, you are presently just not able to "see" the algorithm and practical application. That is ok. Not every one needs to view a question or answer at the same perspective, else you would not have asked the question itself - you would be capable of viewing all perspectives simultaneously and thus be able to resolve your own inquiries. You can select any `.length` of string from a string of N `.length` to get result without bias, as there is no formal distribution at the original string of N length, where N is for example a string having `.length` of `100` or more – guest271314 Aug 09 '17 at 03:05
  • My question is how to generate large numbers, it has nothing to do with the max safe integer. Also, what do you mean by "see"? Are you saying that no matter what answer you give eventually I'll need it so it's acceptable? And your code throws the error `Cannot read property body of undefined`, as I expected it to. It seems that you didn't notice that this question is about Node.js. – ElJay Aug 09 '17 at 03:09
  • Also I need to generate one integer at a time, not multiple. – ElJay Aug 09 '17 at 03:10
  • @LJTalbot You can generate one integer at a time using approach at Answer - by first generating a large number as a string, then selecting random groups of indexes. Using a 0-9 based number system if you want to select a single digits you can select any single random index. If you want to select a number having fifty digits you can do so as well either at a single or multiple calls to `.slice()` then concatenating the result. Am not invested in whether you accept the Answer or not. The Answer is only a possible option. No error is thrown at stacksnippets. – guest271314 Aug 09 '17 at 03:12