random.choice gives different results on Python 2 and 3

Question

Background

I want to test my code which depends on random module.

The problematic PR is https://github.com/Axelrod-Python/Axelrod/pull/202 and code is here https://github.com/Axelrod-Python/Axelrod/blob/master/axelrod/strategies/qlearner.py

The problem

Since random module produces pseudo-random numbers, I always set random.seed(X) to known value X. This works for consecutive test runs. However, Python 3 seems to give different numbers than Python 2 when using random.choice([D, C])

Following snippet:

import random
random.seed(1)

for i in range(10):
    print(random.choice(['C', 'D']), end=', ')

gives different result for Python 2 and 3

$ python2 test.py                                                                                                                                                     
C, D, D, C, C, C, D, D, C, C

$ python3 test.py
C, C, D, C, D, D, D, D, C, C

However, random.random method works the same on 2.x and 3.x:

import random
random.seed(1)

for i in range(10):
    print(random.random())

$ python3 test.py
0.13436424411240122
0.8474337369372327
0.763774618976614
0.2550690257394217
0.49543508709194095
0.4494910647887381
0.651592972722763
0.7887233511355132
0.0938595867742349
0.02834747652200631

$ python2 test.py
0.134364244112
0.847433736937
0.763774618977
0.255069025739
0.495435087092
0.449491064789
0.651592972723
0.788723351136
0.0938595867742
0.028347476522

Workaround

I can mock the output of random.choice, which works well for simple test cases. However, for fairly complicated test cases, I'm not able to mock output, because I simply don't know how it should look like.

The question

Have I done something wrong when calling random.choice method?

I guess in the snippet it must be `for i in range(10):` and not `for i in random(10):` — Bhargav Rao, Jun 04 '15 at 16:53
Why do you expect the random module to produce the exact same results across versions of Python? — Martijn Pieters, Jun 04 '15 at 16:54
Maybe it would help if you provided a bit more context. What does the algorithm that uses random do? Maybe there is a better solution. — UloPe, Jun 04 '15 at 16:58
Have you read [the docs](https://docs.python.org/3/library/random.html#random.seed)? — jonrsharpe, Jun 04 '15 at 16:59
@MartijnPieters I expect, that random number generator should depend on seed rather on Python version. I need to run the same test suite for both python versions. — Tomáš Ehrlich, Jun 04 '15 at 17:00
Python 3 was intentionally not backward compatible with Python 2, so expecting exactly the same behavior between the two is probably not a good idea. See this answer about the backward compatibility: http://stackoverflow.com/a/9067012/514040 — neuronaut, Jun 04 '15 at 17:01
@jonrsharpe I've read it now and try it with random.seed(X, 1), but it doesn't work either. — Tomáš Ehrlich, Jun 04 '15 at 17:02
@UloPe Added more links to question (the "Background" section) — Tomáš Ehrlich, Jun 04 '15 at 17:04
Possible duplicate of [random.randint shows different output in Python 2.x and Python 3.x with same seed](https://stackoverflow.com/questions/55647936/random-randint-shows-different-output-in-python-2-x-and-python-3-x-with-same-see) — pppery, Sep 06 '19 at 23:45
See this answer: https://stackoverflow.com/questions/55647936/random-randint-shows-different-output-in-python-2-x-and-python-3-x-with-same-see In summary, due to this bug (https://bugs.python.org/issue9025) part of the implementation of the random library was changed on python 3.2 — Antonio Luna, Sep 06 '19 at 21:14

score 8 · Answer 1 · answered Jun 04 '15 at 17:00

There is a completely different implementation of random.choice in each version.

Python 2.7:

def choice(self, seq):
    """Choose a random element from a non-empty sequence."""
    return seq[int(self.random() * len(seq))]  # raises IndexError if seq is empty

https://hg.python.org/cpython/file/2.7/Lib/random.py

Python 3.4:

def choice(self, seq):
    """Choose a random element from a non-empty sequence."""
    try:
        i = self._randbelow(len(seq))
    except ValueError:
        raise IndexError('Cannot choose from an empty sequence')
    return seq[i]

https://hg.python.org/cpython/file/3.4/Lib/random.py

The _randbelow method may call random() more than once, or may call getrandbits which has a different underlying call to _urandom.

Thank you, that might be the problem! – Tomáš Ehrlich Jun 04 '15 at 17:05 — Tomáš Ehrlich, Jun 04 '15 at 17:05

Tom Wilson · Accepted Answer · 2015-06-04T17:06:57.307

According to https://docs.python.org/2/library/random.html, the RNG was changed in Python 2.4 and may use operating system resources. Based on this and the other answer to this question, it's not reasonable to expect Random to give the same result on two different versions of Python, two different operating systems, or even two different computers. For all any of us knows, the next version of Python could implement a Random function that uses the system's microphone to generate a random sequence.

Short version: you should never depend on a RNG to give a deterministic result. If you need a known sequence to satisfy a unit test, you need to either redesign your method or your unit test.

One way you might do this is to split your method into two parts: one part generates the random number. The second part consumes the value and acts on it. You would then write two unit tests: one to test coverage of the generated values, and a separate one to test the output of your method based on specific inputs.

Another way might be to change your method to output not just the result but the random number that created that result. You can modify your unit test to compare the two and pass or fail the test based on the expected output of known pairs.

Or perhaps your unit test can be modified to simply run the test n number of times and look for a spread that confirms some sort of randomness.

While true that using random in tests is rarely a good idea, the docs clearly state that `random` uses a deterministic Mersenne Twister implementation. — UloPe, Jun 04 '15 at 17:04
Alright, so probably the best solution is mock the function with known results and don't rely on `random.seed`, right? — Tomáš Ehrlich, Jun 04 '15 at 17:05
@UloPe relying on that is a bad idea, since the algorithm is not guaranteed to stay the same. Since OS resources are involved, we can't ever guarantee the same sequence in two different environments. — Tom Wilson, Jun 04 '15 at 17:08
@elvard Without knowing what you're trying to accomplish, it's hard to give a sound recommendation. But if you need a non-random list, then yes, you'd need to implement your own sequence generator. Random is supposed to be just that - random and unpredictable. — Tom Wilson, Jun 04 '15 at 17:09

score 0 · Answer 3 · answered Nov 30 '20 at 13:32

I've got the exact same problem and I'm disappointed at the number of responses that point to error on your part when, seeding the random function is expected to produce reliably consistent results across versions of Python, machine and operating system.

What seems to work is, painfully, to have your own random class and override the relevant methods with the logic from Python 2.7.

from random import Random

class MyRandom(Random):
    def sample(self, population, k):
        (code from Python 2.7.6 random module updated for Python 3 syntax)
    
    def choice...

my_random = MyRandom(0)
my_random.sample(['Apples', 'Bananas', 'Carrots'])

The randomness functions themselves are different so having the same seed result in the same numbers doesn't fix many random functions that refuse to return the same results. While there are reasons for the newer random functions, those reasons are moot on existing code bases already dependant on the older functions.

Anyway, I hope this can help anyone else fighting this issue.

random.choice gives different results on Python 2 and 3

Background

The problem

Workaround

The question

3 Answers3