0

I have a large, rather complicated procedural content generation lua project. One thing I want to be able to do, for debugging purposes, is use a random seed so that I can re-run the system & get the same results.

To the end, I print out the seed at the start of a run. The problem is, I still get completely different results each time I run it. Assuming the seed doesn't change anywhere else, this shouldn't be possible, right?

My question is, what other ways are there to influence the output of lua's math.random()? I've searched through all the code in the project, and there's only one place where I call math.randomseed(), and I do that before I do anything else. I don't use the time or date for any calculations, so that wouldn't be influencing the results... What else could I be missing?

Updated on 2/22/16 monkey patching math.random & math.randomseed has, oftentimes (but not always) output the same sequence of random numbers. But still not the same results – so I guess the real question is now: what behavior in lua is indeterminate, and could result in different output when the same code is run in sequence? Noting where it diverges, when it does, is helping me narrow it down, but I still haven't found it. (this code does NOT use coroutines, so I don't think it's a threading / race condition issue)

sinisterandroid
  • 23
  • 1
  • 9
  • 5
    Please create a [mcve]. – Colonel Thirty Two Feb 22 '16 at 01:36
  • I feel maybe an external program may be influencing it, but I don't have enough technical knowledge to back this up. – warspyking Feb 22 '16 at 02:23
  • math.random uses http://www.tutorialspoint.com/c_standard_library/c_function_rand.htm internally, perhaps there's some way this function is messed with, in another way besides your program. – warspyking Feb 22 '16 at 02:29
  • I would love to supply a minimal, complete & verifiable example, but it's a very complex system. So far my attempts to reproduce it in a smaller form have proved fruitless, which is why I'm looking for ideas on what to look for. – sinisterandroid Feb 22 '16 at 09:59
  • Have we ruled out some other code running in the middle of the process that also uses math.random? If some other piece of lua gets to run that uses math.random before the first one finishes they will conflict with each other to produce differing results. Have your math.random monkey also logging the caller to definitively rule out. – Brian Jack Nov 22 '22 at 04:25

2 Answers2

1

randomseed is using srandom/srand function, which "sets its argument as the seed for a new sequence of pseudo-random integers to be returned by random()".

I can offer several possible explanations:

  1. you think you call randomseed, but you do not (random will initialize the sequence for you in this case).
  2. you think you call randomseed once, but you call it multiple times (or some other part of the code calls randomseed as well, possibly at different times in your sequence).
  3. some other part of the code calls random (some number of times), which generates different results for your part of the code.
  4. there is nothing wrong with the generated sequence, but you are misinterpreting the results.
  5. your version of Lua has a bug in srandom/random processing.
  6. there is something wrong with srandom or random function in your system.

Having some information about your version of Lua and your system (in addition to the small example demonstrating the issue) would help in figuring out what's causing this.

Updated on 2016/2/22: It should be fairly easy to check; monkeypatch both math.randomseed and math.random and log all the calls and the values returned by the functions for two subsequent runs. Compare the results. If the results differ, you should be able to isolate why they differ and reproduce on a smaller example. You can also look at where the functions are called from using debug.traceback.

Paul Kulchenko
  • 25,884
  • 3
  • 38
  • 56
  • It's definitely not #1, and #2 seems unlikely. I'm not sure how #3 would produce different results if the first thing I do every run is to set the seed. It produces text output, which is very clearly different, so #4 seems impossible as well. I've tried with luajit 2.1.0 *and* lua 5.1.5 so #5 would require 2 different versions to have the same bug. And #6 just seems too unlikely. It's gotta be something *I'm* doing wrong, I would think. I'll continue to try to narrow it down to a reproducible example, though. – sinisterandroid Feb 22 '16 at 10:07
  • @sinisterandroid, updated the post with a suggestion. – Paul Kulchenko Feb 22 '16 at 22:36
  • thank you, this has been very useful! I still haven't found a solution, but your answer has gotten me much closer. – sinisterandroid Feb 23 '16 at 07:23
  • @sinisterandroid, it can't be the threading as the Lua doesn't use threads (and you said you don't use coroutines). If you really use threads, then you have multiple Lua states/VMs, which means that you have different random sequences per VM (and they can overlap in unpredictable ways). – Paul Kulchenko Feb 23 '16 at 08:02
0

Correct, as stated in the documentation, 'equal seeds produce equal sequences of numbers.'

Immediately after setting the seed to a known constant value, output a call to rand - if this varies across runs, you know something is seriously wrong (corrupt library download, whack install, gamma ray hit your drive, etc).

Assuming that the first value matches across runs, add another output midway through the code. From there, you can use a binary search to zero in on where things go wrong (I.E. first half or second half of the code block in question).

While you can & should use some intuition to find the error as you go, keep in mind that if intuition alone was enough, you would have already found it, thus a bit of systematic elimination is warranted.

Revision to cover comment regarding array order:

If possible, use debugging tools. This SO post on detecting when the value of a Lua variable changes might help.

In the absence of tools, here's one way to roll your own for this problem:

A full debugging dump of any sizable array quickly becomes a mess that makes it tough to spot changes. Instead, I'd use a few extra variables & a test function to keep things concise.

Make two deep copies of the array. Let's call them debug01 & debug02 & call the original array original. Next, deliberately swap the order of two elements in debug02.

Next, build a function to compare two arrays & test if their elements match up & return / print the index of the first mismatch if they do not. Immediately after initializing the arrays, test them to ensure:

  1. original & debug01 match
  2. original & debug02 do not match
  3. original & debug02 mismatch where you changed them

I cannot stress enough the insanity of using an unverified (and thus, potentially bugged) test function to track down bugs.

Once you've verified the function works, you can again use a binary search to zero in on where things go off the rails. As before, balance the use of a systematic search with your intuition.

Community
  • 1
  • 1
Pikalek
  • 926
  • 7
  • 21
  • Thanks! I'm doing this now. It's slow-going, but I think this approach will (eventually) get me there. – sinisterandroid Feb 23 '16 at 07:24
  • This approach eventually led me to the source of the issue – specifically, and I've no idea *how* this happened, but on subsequent runs the same array would find itself in a different order. So even though the random #s were the same, they were referring to different elements of this array, and that caused the runs to diverge. That said, I still haven't been able to reproduce it in a smaller test case. – sinisterandroid Feb 23 '16 at 13:52
  • This sounds like a different problem that lua tables do not guarantee their hash ordering in any way. Two different runs putting elements in a table via table[key]=value in the exact same manner can yield a different order when traversed via `for k,v in pairs(myTable) do ... end` – Brian Jack Nov 22 '22 at 04:36