7

Basically, I have 2 threads, receive and send. I want to be able to type a message, and whenever I get a new message it just gets 'printed above the line I am typing in'. first what I thought would work, and you can just paste this it will run:

import multiprocessing
import time
from reprint import output
import time
import random
import sys

def receiveThread(queue):
    i = 0
    while True:
        queue.put(i)
        i+=1
        time.sleep(0.5)

def sendThread(queue):
    while True:
        a = sys.stdin.read(1)
        if (a != ""):
            queue.put(a)
        


if __name__ == "__main__":
    send_queue = multiprocessing.Queue()
    receive_queue = multiprocessing.Queue()

    send_thread = multiprocessing.Process(target=sendThread, args=[send_queue],)
    receive_thread = multiprocessing.Process(target=receiveThread, args=[receive_queue],)
    receive_thread.start()
    send_thread.start()

    with output(initial_len=2, interval=0) as output_lines:
        while True:
            output_lines[0] = "Received:  {}".format(str(receive_queue.get()))
            output_lines[1] = "Last Sent: {}".format(str(send_queue.get()))

But what happens here is that i cannot send data. The input doesn't give me an EOF unlike when I put a = input(), but it overwrites whatever I put in that line, so how can I wait for the input in one thread while the other one works?

EXPECTED BEHAVOIR:

first line goes Received: 0, 1, 2, 3, 4...

second line goes [my input until I press enter, then my input]

ACTUAL BEHAVIOR if i don't check if input != ""

first line as expected only that the input overwrites the first couple of letters until it resets to Received

second line is always empty, maybe bc stdin only is filled for that one time i press enter and then always returns empty?

ACTUAL BEHAVIOR if i check if input != ""

first line stays: received = 0

second line is just like whatever i enter, if i press enter it goes into a new line where i then enter stuff

Konrad Rudolph
  • 530,221
  • 131
  • 937
  • 1,214
Maritn Ge
  • 997
  • 8
  • 35
  • 3
    What is `s` ? Is it a socket? What library/package are you using? – Gino Mempin Dec 07 '20 at 22:08
  • 1
    @GinoMempin yes, it is sockets, using sockets. I will edit the post real quick to show what happened since then – Maritn Ge Dec 07 '20 at 22:34
  • 2
    @MaritnGe Please read [ask] and provide a [MRE] – eyllanesc Dec 08 '20 at 00:53
  • 7
    Please [edit] your question to provide a [mcve]. Be aware you are not using threads, you are using processes. The later error also suggests your issue is reading from stdin, not threading/processes or sockets. – MisterMiyagi Dec 08 '20 at 14:12
  • 1
    Does this answer your question? [How can i get userinput in a thread without EOFError occuring in python?](https://stackoverflow.com/questions/65196869/how-can-i-get-userinput-in-a-thread-without-eoferror-occuring-in-python) – Troy Dec 08 '20 at 20:36
  • 2
    Please be aware the current iteration of this question has a syntax error. Even if it did not, it is not actually producing any output; both queues just infinitely accumulate data until memory runs out. – MisterMiyagi Dec 09 '20 at 13:45
  • 1
    @MisterMiyagi thanks a lot! i missed to add the 'output' part, and the : got lost, now it should work again, just tested it – Maritn Ge Dec 09 '20 at 13:48
  • 1
    I feel like there are several misunderstandings about concurrency and I/O here. First, be aware you usually do I/O with your shell/terminal, which then does I/O with your program. That means that usually your shell will line-buffer your input – merely printing it back to you as you type but only passing it on after a newline. ... – MisterMiyagi Dec 09 '20 at 13:55
  • 1
    ... Second, even if some parts of your program are concurrent that does not make all of them concurrent. In specific, ``Queue.get`` blocks until there is data, so the ``output_lines[1] = ...`` line will block the entire main thread and its ``while True:`` loop – the ``output_lines[0] = ...`` is not repeated until the former one has run. – MisterMiyagi Dec 09 '20 at 13:55
  • 1
    @MisterMiyagi so should i pass the output[0] and output[1] to the different processes and update them from within? – Maritn Ge Dec 09 '20 at 13:57
  • 1
    @MisterMiyagi I am sorry I lack a lot of knowledge about how python and I/O works (as you might have noticed) and this might be the danning krüger effect once more in action. I will probably have to read up on how that all works. could you maybe tell me what i should look for on google? I/O in python? – Maritn Ge Dec 09 '20 at 13:59
  • 3
    This is a general I/O thing, not specific to Python. Reading up on the [``stdin``/``stdout`` standard streams](https://en.wikipedia.org/wiki/Standard_streams) and working your way on from there might be helpful. It's not exactly the most beginner-friendly topic, to be honest. – MisterMiyagi Dec 09 '20 at 14:02
  • 1
    @MisterMiyagi i wouldn't call myself a beginner i've been working as a dev for about 3 years now, but usually always with guis or apps where i can just have an inputfield and a listener that gives me the data, i have no idea how to do this in command line only, especially all python i do is my in my free time having fun with it, ast only flutter – Maritn Ge Dec 09 '20 at 14:05
  • 1
    @MisterMiyagi but thank you a lot again! i will look into what you linked and if i ever solve this myself or otherwise ill just answer this question myself as good as i can – Maritn Ge Dec 09 '20 at 14:06

1 Answers1

8

Don't use the same socket for communicating with... itself. That may be possible to do, I'm not sure, but it certainly isn't normal. Instead make a socket pair, one for the sending thread, and one for the receiving thread, e.g. this works for me:

import socket;
import multiprocessing;

def receiveThread(sock):
    while True:
        msg = sock.recv(1024)
        print(msg.decode("utf-8"))

def sendThread(sock):
    while True:
        # msg=input("client: ")
        # input() is broken on my system :(
        msg="foo"
        sock.send(bytes(msg,"utf8"))

pair = socket.socketpair()
recieve_thread_socket = pair[0]
send_thread_socket = pair[1]

send_thread = multiprocessing.Process(target=sendThread, args=[recieve_thread_socket])
receive_thread = multiprocessing.Process(target=receiveThread,args=[send_thread_socket])
send_thread.start()
receive_thread.start()
Marcos Dimitrio
  • 6,651
  • 5
  • 38
  • 62
hanshenrik
  • 19,904
  • 4
  • 43
  • 89
  • 1
    the socket here is a tcp socket and not a threading socket, if you click on the link on the top you'll see how I actually did it with multiprocessing.Socket or is it better to use a socketpair? – Maritn Ge Dec 08 '20 at 21:52
  • 2
    @MaritnGe there's no such thing as a "threading socket", but socketpair() creates a unix socket on unix systems (because unix sockets are faster than tcp sockets), and creates a tcp socket on Windows (because Windows doesn't support unix sockets), and it doesn't really matter if you use a unix socket or a tcp socket, unless you plan to transfer large amounts of data, or need very high speeds, in which case it does matter.. anyway, i voted to close your question as `needs shortest code necessary to reproduce the problem` – hanshenrik Dec 08 '20 at 22:47
  • 1
    I'll look into it, thank you! and yes it is a bad question in retrospect i just didn't want to delete it so i made the other one completely rewritten. thanks for you advice both on python and SO – Maritn Ge Dec 08 '20 at 22:52
  • 1
    What do you mean communicating with itself? I am using this socket as a connection between me and my server, should I get two tcp connections for my server to lsiten and to send? – Maritn Ge Dec 09 '20 at 11:56
  • 1
    @MaritnGe seems your original code use 1 shared variable called `s` for both sending and receiving. that won't work, instead get 1 unique socket for your recieve thread, and another unique socket for your send thread, and those 2 sockets must be connected to each others. (socketpair() makes 2 unique sockets that are connected to each others) – hanshenrik Dec 09 '20 at 12:13
  • 1
    so i need 2 sockets/one socketpair to communicate between the remote server and the client I have? wow thanks! i literally am stupid in regards to networking i have no idea what to do – Maritn Ge Dec 09 '20 at 12:18