3

Standard streams are associated with a program. So, suppose there is a program already running in some way (I don't care how or in what way). The goal is to create pipes to the STDIN of the program from different processes (or programs) that run either locally or remotely and stream data into it asynchronously.

Available information is (1) the host address and (2) the pid of the program only. How does one implement both cases in Python in this case?

Edit: I should have mentioned this presupposition. The intended operating system is Linux with a (fairly) recent kernel.

OTZ
  • 3,003
  • 4
  • 29
  • 41
  • Nullpo. Seriously, I know every bit of subprocess module. It doesn't provide interface to get pipes of already running processes. – OTZ Sep 25 '10 at 01:33
  • Can you control how the remote programs are started? – Michael Mior Sep 25 '10 at 01:40
  • @OTZ I figured; that's why that is just a comment. – NullUserException Sep 25 '10 at 01:41
  • @Michael Ideally, remote programs should also be able to create pipes from hostanme and program pid. As long as it can be done remotely based on those info, I don't impose restrictions on how those programs are executed. – OTZ Sep 25 '10 at 01:43
  • Can you provide some context as to why you want to do this? It has a bit of an XY-problem smell to it. – Porculus Sep 25 '10 at 01:52
  • @Porculus I want to combine in an efficient manner all the output from multiple servers into a single file(-like) object. – OTZ Sep 25 '10 at 02:06

2 Answers2

2

This isn't portable, but on many Linux systems, you can write to

/proc/$PID/fd/0

I think this may be one of a very limited number of potentially complicated options if you don't have any other control over the remote process.

Michael Mior
  • 28,107
  • 9
  • 89
  • 113
  • Michael. Your solution scheme does work locally. But how do you do it remotely? Also, any python suggestions? – OTZ Sep 25 '10 at 01:57
  • Actually, I do know how one could do it remotely: use sshfs or some other virtual file system and mount the target file. But such methodology tends to be less robust and hold less integrity. So some other way is desired. – OTZ Sep 25 '10 at 02:01
  • You don't need to use sshfs. Just use SSH and pipe in the data. Something like `ssh $SERVER "cat - > /proc/$PID/fd/0"`. Or use Alex's suggestion and check out `paramiko`. – Michael Mior Sep 25 '10 at 02:43
  • Right. But in that case, we'd be relying on cat's stdin and stdout. So im not sure it'd be more efficient than sshfs. Either way, i think i have enough material to code further (thanks to you and Alex). – OTZ Sep 25 '10 at 04:31
  • No problem. You can use something like `netcat` if you're worried about the overhead of encryption via SSH. You can also try `ssh -c blowfish` to use the faster Blowfish cipher. – Michael Mior Sep 25 '10 at 05:18
  • Interesting, but this only works to Linux. Is there something similar on Unix or OSX? – Yuhao Nov 01 '15 at 21:17
1

In most platforms (i.e., operating systems), an existing process's existing file descriptors are inviolate -- the operating system, striving to guarantee process integrity, will be designed to not allow a separate, unrelated process to alter those file descriptors.

Nevertheless, if you do specify a very specific and well-identified platform (ideally including the exact version and release of the operating system in question, since security does tend to get tightened in successive releases compared with preceding ones), it's quite possible that there will be available tricks for your purposes. For example, you may be able to exploit some of the hooks which the operating system intends to be used for "remote debuggers" attaching themselves to existing processes -- if, that is, your very specific OS does offer such hooks (not all do!).

But, if you want a cross-platform solution, no way.

So, I recommend you edit your question, and in particular replace one of the tags with the name of the "one and only" OS you really need to support (in the Q's edited text, please be as specific as possible about the exact versions and releases you absolutely do need to support -- Python has very little indeed to do with the issue, as you need to operate at specific-OS levels, so there's no real need to similarly pinpoint the Python version).

Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • Edited. I should have mentioned it first. The intended OS for this question is Linux. Thanks for pointing it out. – OTZ Sep 25 '10 at 01:52
  • @OTZ, you're welcome. Then use @Michael's solution -- do it remotely via `ssh` of course (or directly with third-party Python extension package `paramiko`, which lets you speak "sshese" directly from your Python code, without needing to shell out to the `ssh` command-line client!-). Of course, you need credentials to authenticate yourself to the remote node's `sshd` (but then it's obvious that security _at least_ requires any remote process able to do such things to be safely authenticated with the node in question, right?-). – Alex Martelli Sep 25 '10 at 02:01
  • Still don't know how to get the file-descriptor given only pid and hostname remotely, but i'll look further. If there isn't a good way to do so in python, sshfs mounting I commented (to Michael's answer) may be a good compromise. – OTZ Sep 25 '10 at 02:10