Re-broadcasting a UDP stream with python

Question

I am running a simple closing socket listener to ingest a UDP stream containing json directed to port 5001 (in this instance) on localhost:

import socket
import json

from contextlib import closing

def monitor_stream():
    with closing(socket.socket(socket.AF_INET, socket.SOCK_DGRAM)) as s:

        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        s.bind(('', 5001))

        while True:

            data, addr = s.recvfrom(4096)

            try:
                data = data.decode('ascii')
                data = json.loads(data)
            except:
                print 'Not valid JSON: ', data

I need to re-broadcast the stream to another local port (arbitrarily 5002 in this example) so that a second program can access the data in as near as real time. Low latency is crucial. Is the following (using socket.sendto() within the while) an acceptable method:

        while True:

            data, addr = s.recvfrom(4096)
           
            s.sendto(data, ('localhost', 5002))

If not, how else might I achieve the same result?

I have assumed that there is no way of multiple programs ingesting the original stream in a simultaneous fashion as they originate as unicast packets so only the first bound socket receives them.

Secondly, how might I cast the same stream to multiple ports (local or not)?

I am unable to change the incoming stream port / settings.

if you're using Linux then you can use [`SO_REUSEPORT`](https://lwn.net/Articles/542629/) otherwise you could consider using iptables (or similar) to do this. if you really want to do this in Python then your could try `send`ing to a second `connect`ed socket (to save resolver time) but that'll save microseconds at most — Sam Mason, Jul 17 '20 at 16:43
@SamMason Thanks for this. Would `SO_REUSEPORT` not result in a more load-balancing solution. I.e. packets split between listeners? I need both (or more) programs to receive the exact same feed at as near as real time. Is my `sendto` method above different to what you are suggesting? It works intermittently but also raises an _[Errno 10054] An existing connection was forcibly closed by the remote host_ error from time to time - suggesting all is not well. Any ideas? — Opticmountain, Jul 19 '20 at 12:48
oops, yes `REUSEPORT` isn't the right suggestion, sorry! AFAICT 10054 is related to MS Windows which I don't use if I can help it and hence can't help much. it looks like it could be due to the [receiving socket being closed](https://stackoverflow.com/a/2578794/1358308) but looks like MS hasn't been great about attributing it to the right place — Sam Mason, Jul 20 '20 at 09:20
@SamMason Thanks for the link. Yes I think you are right - incorrect attribution of the error. This in-turn helped me with a solution which I'll post below. — Opticmountain, Jul 20 '20 at 13:59

score 1 · Accepted Answer · answered Jul 20 '20 at 14:20

This was a simple misuse of socket.sendto() which can only be called on a non-connected socket.

I was trying to use the bound-listener to send-on the ingested stream.

I seems you need two socket objects to re-broadcast to a different address. The second socket is an un-bound echo client as the address is specified in .sendto(string, address)

Re-broadcasting a UDP stream with python

1 Answers1