1

I'm attempting to poll a few sockets and a multiprocessing.Event
The documentation states:

A zmq.Socket or any Python object having a fileno() method that returns a valid file descriptor.

which means that I can't use my Event but I should be able to use a file(as returned from open(...) or an io object (anything from the io library), but I'm having no success:

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.3\helpers\pydev\pydevd.py", line 1683, in <module>
    main()
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.3\helpers\pydev\pydevd.py", line 1677, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.3\helpers\pydev\pydevd.py", line 1087, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.3\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:\work\polldamnyou.py", line 122, in <module>
    p = poller.poll(1000)
  File "C:\WinPython-64bit-3.6.3.0Qt5\python-3.6.3.amd64\Lib\site-packages\zmq\sugar\poll.py", line 99, in poll
    return zmq_poll(self.sockets, timeout=timeout)
  File "zmq\backend\cython\_poll.pyx", line 143, in zmq.backend.cython._poll.zmq_poll
  File "zmq\backend\cython\_poll.pyx", line 123, in zmq.backend.cython._poll.zmq_poll
  File "zmq\backend\cython\checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Unknown error

I have found the same question asked before but the solution was to use another socket which is sidestepping the problem. I am curious and want to see this working. Does any one have any clues what sort of object can be used in zmq.Poller other than a socket?

Edit: a few things I've tried

import traceback, os, zmq


def poll(poller):
    try:
        print('polled: ', poller.poll(3))
    except zmq.error.ZMQError as e:
        traceback.print_exc()


class Pollable:
    def __init__(self):
        self.fd = os.open('dump', os.O_RDWR | os.O_BINARY)
        self.FD = self.fd
        self.events = 0
        self.EVENTS = 0
        self.revents = 0
        self.REVENTS = 0

    def fileno(self):
        return self.fd

    def __getattribute__(self, item):
        if item != '__class__':
            print("requested: ", item)
        return super().__getattribute__(item)

ctx = zmq.Context()
sock = ctx.socket(zmq.SUB)
poller = zmq.Poller()
poller.register(sock, zmq.POLLIN)
poll(poller)  # works
file = open('dump', 'w+b')
print("fileno: ", file.fileno())
poller.register(file, zmq.POLLIN)
poll(poller)  # fails
file.events = 0
file.revents = 0
file.EVENTS = 0
file.REVENTS = 0
file.fd = file.fileno()
file.FD = file.fileno()
poll(poller)  # still fails
poller.unregister(file)
file.close()
poll(poller)  # works
fd = os.open('dump', os.O_RDWR|os.O_BINARY)
print("fd: ", fd)
dummy = Pollable()
poller.register(dummy, zmq.POLLIN)
poll(poller)  # fails

__getattribute__ shows that that fd and fileno are being accessed, but nothing else, so what is still wrong?!

Nullman
  • 4,179
  • 2
  • 14
  • 30

1 Answers1

0

In case one has never worked with ZeroMQ,
one may here enjoy to first look at "ZeroMQ Principles in less than Five Seconds"
before diving into further details



Q : "what sort of object can be used in zmq.Poller other than a socket?"

Welcome to the lovely lands of the Zen-of-Zero. Both the published API and the pyzmq ReadTheDocs are clear and sound on this :

The zmq_poll() function provides a mechanism for applications to multiplex input/output events in a level-triggered fashion over a set of sockets. Each member of the array pointed to by the items argument is a zmq_pollitem_t structure. The nitems argument specifies the number of items in the items array. The zmq_pollitem_t structure is defined as follows :

typedef struct
{
    void //*socket//;
    int //fd//;
    short //events//;
    short //revents//;
} zmq_pollitem_t;

For each zmq_pollitem_t item, zmq_poll() shall examine either the ØMQ socket referenced by socket or the standard socket specified by the file descriptor fd, for the event(s) specified in events. If both socket and fd are set in a single zmq_pollitem_t, the ØMQ socket referenced by socket shall take precedence and the value of fd shall be ignored.

For each zmq_pollitem_t item, zmq_poll() shall first clear the revents member, and then indicate any requested events that have occurred by setting the bit corresponding to the event condition in the revents member.

If none of the requested events have occurred on any zmq_pollitem_t item, zmq_poll() shall wait timeout milliseconds for an event to occur on any of the requested items. If the value of timeout is 0, zmq_poll() shall return immediately. If the value of timeout is -1, zmq_poll() shall block indefinitely until a requested event has occurred on at least one zmq_pollitem_t.


Last, but not least,
the API is explicit in warning about implementation :

The zmq_poll() function may be implemented or emulated using operating system interfaces other than poll(), and as such may be subject to the limits of those interfaces in ways not defined in this documentation.


Solution :

For any wished-to-have-poll()-ed object, that does not conform to the given mandatory property of having an ordinary fd-filedescriptor, implement a such mandatory property mediating-proxy, meeting the published API-specifications or do not use it with zmq_poll() for successful API calls.

There is no third option.

user3666197
  • 1
  • 6
  • 50
  • 92
  • I'm a little confused on the `fd` do objects returned by `open()` not have a `fd`? or is this a windows/linux thing(I'm on windows)? – Nullman Mar 23 '20 at 11:35
  • For details see *Doc.*: https://pyzmq.readthedocs.io/en/latest/api/zmq.html#zmq.Socket.fileno and API doc --- "*The returned file descriptor is intended for use with a `poll` or similar system call only. **Applications must never attempt to read or write data to it directly, neither should they try to close it**.*" ( On POSIX compliant O/S `.getsockopt( zmq.FD )` yields `int`, may yield a `SOCKET` on Windows ). – user3666197 Mar 23 '20 at 11:55
  • I still dont seem to understand, the gile object returned by `open` has the required `fileno` which returns an `fd`. I even tried using `os.open` which returns just an `fd` passing it straight or as part of an object as a `fileno` method or `fd` property and still nothing – Nullman Mar 23 '20 at 14:38
  • The API since version 2.0 until today **defines** *one and only one* **reason** for **`zmq_poll()`** to throw an Exception **`::= { ETERM | EFAULT | EINTR }`**. Write a **M**-inimum **C**-omplete **V**-erifiable **E**-example **code, that represents** your **problem** and either prove or disprove the root cause of the problem to be the `pyzmq`-wrapper `.poll()` method's API (in)compatibility - which is impossible to be done as you did not post a SLOC so far, or you might find an other root-cause of that *( in case the `.poll()`-method was not the one, who threw the above depicted Exception )* – user3666197 Mar 23 '20 at 15:57
  • any clues? i still cant seem to figure this out – Nullman Mar 26 '20 at 10:53