Sending short messages between python and java

Question

My java program calls a python script that sends it small JSON strings. I launch the python script using process builder, then wrap the inputstream in inputstreamreader and BufferedReader.

What I've noticed though, is that the program waits until a certain number of messages have been sent to transmit the data to my java application.

I traced this problem to the raw InputStream I get from the Process, because even if I use it alone without BufferedReader I still get this issue. When I increase the size of the messages they are transmitted without any problems. But I only need to transmit short messages every few seconds. This behavior makes me think that the inputstream has some internal buffer that it waits to fill before transmitting the data. Any help with this?

Q: have you tried "flush()"? – paulsm4 Dec 28 '18 at 18:57 — paulsm4, Dec 28 '18 at 18:57
what object exposes this method please? – Yazly Dec 28 '18 at 20:16 — Yazly, Dec 28 '18 at 20:16

paulsm4 · Accepted Answer · 2018-12-28T21:47:23.783

You haven't really given enough details to know exactly what's going on (an MCVE would be nice ;)), but:

You clearly have a problem, and
It sounds like it might indeed be related to "buffering".

Hence my suggestion about "flush()".

Q: If you "flush" a buffer - who needs to do it? The reader, or the writer?

A: The WRITER, of course :)

It sounds like your "writer" is Python. It also sounds like the Python app is writing to stdout.

This might help:

How often does python flush to a file?

For file operations, Python uses the operating system's default buffering unless you configure it do otherwise. You can specify a buffer size, unbuffered, or line buffered.

For example, the open function takes a buffer size argument.

http://docs.python.org/library/functions.html#open

"The optional buffering argument specifies the file’s desired buffer size:"
0 means unbuffered,
1 means line buffered,
any other positive value means use a buffer of (approximately) that size.

A negative buffering means to use the system default, which is usually 
line buffered for tty devices and fully buffered for other files.

If omitted, the system default is used.

Thanks man! That was it :) I just set the flush parameter in python's print command and it works well. Didn't think about it because when I run the script alone I can see the data on my terminal. Do you have any explanation to this? — Yazly, Dec 29 '18 at 18:10

Sending short messages between python and java

1 Answers1