How to implement LIFO for multiprocessing.Queue in python?

Question

I understand the difference between a queue and stack. But if I spawn multiple processes and send messages between them put in multiprocessing.Queue how do I access the latest element put in the queue first?

Answer 1

You can use a multiprocessing manager to wrap a queue.LifoQueue to do what you want.

from multiprocessing import Process
from multiprocessing.managers import BaseManager
from time import sleep
from queue import LifoQueue


def run(lifo):
    """Wait for three messages and print them out"""
    num_msgs = 0
    while num_msgs < 3:
        # get next message or wait until one is available
        s = lifo.get()
        print(s)
        num_msgs += 1


# create manager that knows how to create and manage LifoQueues
class MyManager(BaseManager):
    pass
MyManager.register('LifoQueue', LifoQueue)


if __name__ == "__main__":

    manager = MyManager()
    manager.start()
    lifo = manager.LifoQueue()
    lifo.put("first")
    lifo.put("second")

    # expected order is "second", "first", "third"
    p = Process(target=run, args=[lifo])
    p.start()

    # wait for lifoqueue to be emptied
    sleep(0.25)
    lifo.put("third")

    p.join()

Answer 2

The multiprocessing.Queue is not a data type. It is a mean to communicate between two processes. It is not comparable to Stack

That's why there is no API to pop the last item off the queue.

I think what you have in mind is to make some messages to have a higher priority than others. When they are sent to the listening process, you want to dequeue them as soon as possible, bypassing existing messages in the queue.

You can actually achieve this effect by creating two multiprocessing.Queue: One for normal data payload and another for priority message. Then you do not need to worry about getting the last item. Simply segregate two different type of messages into two queues.