3

I am working on a GUI version of a executable file. The file have a option as

exefile --output output.txt

My task is not save to output.txt, but can use some information in the file to plot a graph? I have some idea to plot the graph using pyqtgraph. The thing is how to get it as a text stream? Any thoughts?

Second question is similar but little more difficult. The same executable file can also output to a .wav

exefile --outwav outwav.wav

Instead of saving a .wav file, how to plot it out, in some of "real-time" way? Many thanks.

Note:

  1. My intention is to build a wrapper, and I don't have any control inside the executable file.

  2. Ideally the executable file should run continuously, while I plot output/wave during the time

cityzz
  • 301
  • 2
  • 12
  • Your question is unclear. Why not `output = open(path, 'w'); output.write(...)` or `print >> output, data`? – William Pursell Mar 04 '15 at 12:33
  • He's making a wrapper over an exe that saves to disk. He doesn't want to save to disk but to read the output to a variable so he can make a plot. – Ionut Hulub Mar 04 '15 at 12:34
  • Redirect the program output to a file can be done like this: `import sys sys.stdout = open('file', 'w')` –  Mar 04 '15 at 12:35
  • Also, he doesn't have controll over the executable, so he can't redirect the output like that. @cityzz your best bet is to run the exe and let it save to a file, then read the content of that file in your application and then delete the file. – Ionut Hulub Mar 04 '15 at 12:36
  • Yes, Ionut Hulub is right, sorry for this confusion. – cityzz Mar 04 '15 at 12:40
  • There is a convention honoured by many programs that specifying `-` as a file name means stdout (or stdin), e.g. `exefile --output -` _might_ send the output to stdout where it can be piped into your plotting script. It depends on the program though. – mhawke Mar 04 '15 at 12:52
  • What platform is this for? – loopbackbee Mar 04 '15 at 13:09
  • @goncalopp it is for linux. – cityzz Mar 04 '15 at 13:12
  • @mhawke I've try, the stdout did show anything for the text output option. For the --outwav option, it only output a sample_rate=16000, count=0 – cityzz Mar 04 '15 at 13:14
  • Probably `-` is not supported then. Did you get an error? What happens if you omit the `--output` option? – mhawke Mar 04 '15 at 13:17
  • @mhawke I didn't give anything, just hanging. If I omit the --output option, it indicates : no operation given – cityzz Mar 04 '15 at 13:21
  • Looks like a named pipe is the way to go then. It is seemingly a good fit if the executable is generating data continuously. – mhawke Mar 04 '15 at 13:22
  • When it comes to plotting of data in real time you might want to see the examples in pyqtgraph. More specifically: [PanningPlot](https://github.com/pyqtgraph/pyqtgraph/blob/develop/examples/PanningPlot.py), and [scrollingPlots](https://github.com/pyqtgraph/pyqtgraph/blob/develop/examples/scrollingPlots.py). – jojeck Mar 04 '15 at 13:27
  • @jojek Thanks, I am learning that as well :D – cityzz Mar 04 '15 at 13:31

2 Answers2

4

Assuming that:

  • you are on a Unix platform (Linux, Mac OSX)
  • the original exefile has no way to do what you want
  • you don't want / can't modify exefile

The best option would be using a named pipe.

This works just like a temporary file, but doesn't hit the hard drive - everything happens in memory. It has several advantages over a temporary file:

  • you're able to process several gigabytes per second, without trashing your disk I/O
  • you avoid having to synchronize reads, having to be extra careful with EOF, and not really knowing when exefile has finished writing.
  • you avoid filling up the filesystem.

You can use os.mkfifo to create a named pipe. Read from it just like you would read from a normal file.

Note that exefile will periodically I/O block if writing to the named pipe happens faster than your program reads. Similarly, your program will I/O block if reading faster than exefile writing.

loopbackbee
  • 21,962
  • 10
  • 62
  • 97
  • Seems that it's possible on Windows as well, it's just not as easy. http://jonathonreinhart.blogspot.se/2012/12/named-pipes-between-c-and-python.html – pss Mar 04 '15 at 13:26
  • @pss It's not. Note that a "named pipe" on windows is a very different beast. The most important difference here is that it isn't exposed (mounted) on the filesystem tree, so you can't provide it as an argument to `exefile` – loopbackbee Mar 04 '15 at 13:33
  • @goncalopp Any cross platform suggestions? I was told to not use "platform dependent" methods. :( – cityzz Mar 04 '15 at 17:42
  • @cityzz AFAIK there's no cross platform way of intercepting output to a file. I think you have no better option than using a temporary file. If you need Windows support, note that even reading a file that is being simultaneously written may be a though proposition, depending on how `exefile` is implemented. See [file locking on windows](https://en.wikipedia.org/wiki/File_locking#In_Microsoft_Windows) – loopbackbee Mar 04 '15 at 18:11
1

I think the best way would be to write to temporary file and then read it where needed. It's very easy to do in python:

from tempfile import NamedTemporaryFile
f = NamedTemporaryFile() # this is file object you will read from
f.name # and here you can get it's name

# here is simple way to run your command
import os
exit_code = os.system("execfile --output %s" % (f.name,))
if not exit_code:
    data = f.read()
else:
    print "Oooops. exit code = %s" % (exit_code,)

Check doc for more info.

pss
  • 736
  • 3
  • 7
  • 14
  • It might be, but for the continuous running executable, I would need to use some of the information inside the file. Would there be any problems for this file to be written and read at the same time? Can this file size been control? Many thanks. – cityzz Mar 04 '15 at 12:55
  • Probably your best bet is to try `-` as output file name and see if it works with your executable as @mhawke pointed in comment to your question. But if it doesn't you have no choice either than running program you are trying to wrap in background and pointing it's output to file that you are also gonna be reading. – pss Mar 04 '15 at 13:04
  • This may help you with figuring out how to read and write to the file simultaneously http://stackoverflow.com/questions/14271216/beginner-python-reading-and-writing-to-the-same-file – pss Mar 04 '15 at 13:10
  • And probably you will have to control file size by yourself. Maybe restart `execfile` when output file reaches some limit etc. – pss Mar 04 '15 at 13:11
  • Yes, I agree with you on the file limit part. When it was made, I think the author didn't consider to run it for a long time. However, I need to show the graph of its result continuously. – cityzz Mar 04 '15 at 13:19
  • Check the named pipe thing. it's seems to be a real dealio:) – pss Mar 04 '15 at 13:19