1

I'm rather new to Python and I'm working on a multiprocessing example script. The code is ~100 lines long, so you can find it here - http://pastie.org/1813365

It's a simple game that occurs on square fields - they are represented as matrices(list containing x nested lists of x elements each). A separate game field is created for each player.

Each process that I create represents one player. Each turn and for each "player", two coordinates and a digit(0-9) are generated(randomly). Each "player" tries to place its' digit on his coordinates in the field; if there is no digit on another players' field in these coordinates, or the digit is less than the first players' one, or if the first player has the '0'(it is like a 'joker card') - the digit is (well, meant to be) placed on both fields and the player's score is increased. The game ends after a specified number of iterations.

All the data is stored in a single object of the data-containing class that is being transferred from the main thread to the "first player" thread and then is constantly being transferred from one thread to another, back and forth, until the game ends. 'JoinableQueue' is being used for that.

The problem about the code is that it seems that each "player" has its' own copies of both of the game fields. You can see this clearly if you output both game fields from both player each turn - they are identical for each player. For example, the commented line has no effect, since the another player's (that is being printed in his turn) field is never modified:

if x.data_PC[y2][x2] == 'X' or z2 == 0 or int(x.data_PC[y2][x2]) < z2:         
    x.data_PC2[y2][x2] = str(z2) # doesn't work
    x.data_PC[y2][x2] = str(z2)
    x.score_PC2 += 1

This is especially strange since all the remaining data in the object seem to be working just fine.

What causes such a behavior and how can I fix this?

havelock
  • 171
  • 1
  • 1
  • 10
  • Why are you using multiprocessing in the first place? An object-oriented approach would shorten your life a whole lot less. Not only that ... why do you even use multiprocessing to simulate a turn-based game that is consequent -- and not concurrent -- by definition?? – ktdrv Apr 19 '11 at 22:54
  • Because I need to create a program that uses multiprocessing (or multithreading at least). For academic purpouses, actually. These conditions (the turn-based game, "fighting" for resources in the game field and so on) were not invented by me. :) – havelock Apr 19 '11 at 22:57
  • I'd probably have the game data as shared state, and control the turns with the lock. That seems simpler than passing the game state back and forth in a queue. http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes – Thomas K Apr 19 '11 at 23:14
  • FWIW, I think the reason for what you see is that you're modifying a class variable, and that isn't getting passed across with the instance. The other numbers you assign to the instance, overriding the class variables. So you can "solve" it by attaching the game matrix to the `data` instance, via an `__init__` method, rather than directly on the class. But it's still an awkward design. – Thomas K Apr 19 '11 at 23:18
  • Indeed I tried to used shared state first (and the `Lock`-s I accidentally forgot to remove after that are the evidence), but I didn't really get it to work, and I didn't really like that design due to its' complexity. – havelock Apr 20 '11 at 18:48

1 Answers1

2

Ok, your problem doesn't have anything to do with multiprocessing. If you look at your data class definition(you should capitalize your class names, it's pythonic that way), all the variables you are creating are class variables. You want them to be in an __init__ method and defined as self.foo=1 so that they are instance variables and can be passed between the processes. Like so:

class Data(object):

    def __init__(self):

        self.SIZE = 8 # dimensions of the game field
        self.MAX_TURNS = 5 # amount of iterations for each of the players ("turns") before the end
        self.turns = 0 # count of turns done
        self.score_PC = self.score_PC2 = 0 # score values

        # create and init the data matrices
        self.data_PC = [['X' for j in xrange(self.SIZE)] for i in xrange(self.SIZE)]
        self.data_PC2 = [['X' for j in xrange(self.SIZE)] for i in xrange(self.SIZE)]

Now a few words about the multiprocessing aspect. You don't really need a lock if you are using the joinable queue. All you need to do is .get() the data, process it and .put() it back in the queue, and let it know that .task_done(). It will take care of the concurrency.

Here, I posted your code with the needed edits.

ktdrv
  • 3,602
  • 3
  • 30
  • 45
  • Thank you. It seems that I understand this now. The code without queue join-s work just fine, but there is one little issue - sometimes, randomly, after the script is done, `Exception in thread QueueFeederThread (most likely raised during interpreter shutdown):` pops out (CPython 2.7.1, Win7 x64). As far as I can remember, this was also happening with my original code unless I introduced queue joins. Locks were just remaining from the approach I tried before but had no success with - to store data, as mentioned above, in a shared state. I will vote up the answer when I have enough reputation:) – havelock Apr 20 '11 at 18:37
  • Yes, and one last thing - as far as I got it from documentation, it seems that `.task_done()` is used when you `.put()` the data in the queue, then `.get()` it, and process it. I mean, `.task_done()` is used after `.get()` , not after `.put() `, isn't it? – havelock Apr 20 '11 at 18:43
  • If you are interested, [this revision](http://pastie.org/1816530) works for me with no problems or error messages ever. Maybe I got a little bit paranoid with `q.join()`-s, but I like it in the safe way. – havelock Apr 20 '11 at 19:18
  • P.S. Just tried to run the script on VirtualBox'ed Ubuntu 10.04, and it still outputs the error pretty often, more detailed: `Exception in thread QueueFeederThread (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner File "/usr/lib/python2.6/threading.py", line 484, in run File "/usr/lib/python2.6/multiprocessing/queues.py", line 233, in _feed : 'NoneType' object is not callable` This seems to be a known issue: http://bugs.python.org/issue4106. – havelock Apr 20 '11 at 19:24
  • The way I understand it is that `.get()` should be followed by a corresponding `.task_done()` and that's all there is to it. So a `.put()` or any other call between the two should not matter. Either way, queues and multiple processes are not the best way here. I would go for threads and shared states. You seem to have tried that -- maybe you should give it another go :) – ktdrv Apr 20 '11 at 21:35