Python: subprocess.stdin.write is not working properly

Question

I'm new to python but I can code and debug a bit. The following problem has been on my head for the past couple of days and looking out for an answer. Any help is much appreciated. Task: I wanted to do an interactive telnet (I know there is an telnet library, but we are not using it for various reasons). For this purpose, I use a subprocess.popen

p = subprocess.Popen(telnet_command,
                         stdin = subprocess.PIPE,
                         stdout = outputfileobj,
                         stderr = errorfileobj)

I do a poll() to see if the server has got connected to the session. Once I verified, I proceed to write to the stdin for interactive communication.

inputTxt = 'GET / HTTP/1.1\nHost: ' + hostheader + '\n\n'
p.stdin.write(inputTxt)
p.stdin.flush()

Here is where the problem has occurred. I get the http response (or atleast output) for 5/6 times but 1/6 time, I do not get a output and the subprocess gets terminated - which is not possible.

I ran a system trace for the failure case and please find below the same.

16129 write(7, "GET / HTTP/1.1\nHost: www.google."..., 37) = 37
16129 gettimeofday({1310538497, 134474}, NULL) = 0
16129 futex(0x81993a8, FUTEX_WAKE, 1)   = 0
16129 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16129 fstat64(4, {st_mode=S_IFREG|0644, st_size=520689, ...}) = 0
16129 _llseek(4, 520689, [520689], SEEK_SET) = 0
16129 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16129 write(4, "2011-07-13 06:28:17,134 : pconns"..., 170) = 170
16129 futex(0x81d3b30, FUTEX_WAKE, 1)   = 0
16129 waitpid(16198, 0xffa945e8, WNOHANG) = 0
16129 gettimeofday({1310538497, 135160}, NULL) = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf74b9000
16129 _llseek(6, 0, [0], SEEK_SET)      = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 _llseek(6, 0, [0], SEEK_CUR)      = 0
16129 read(6, "Trying 74.125.236.48...\nConnecte"..., 4096) = 81
16129 read(6, "", 4096)                 = 0
16129 close(6)                          = 0
16129 munmap(0xf74b9000, 4096)          = 0
16129 gettimeofday({1310538497, 135778}, NULL) = 0
16129 futex(0x81993a8, FUTEX_WAKE, 1)   = 0
16198 <... select resumed> )            = 1 (in [0])
16198 read(0, "GET / HTTP/1.1\nHost: www.google."..., 8191) = 37
16129 stat64("/etc/localtime",  <unfinished ...>
16198 ioctl(1, TCFLSH, 0)               = -1 ENOTTY (Inappropriate ioctl for device)
16129 <... stat64 resumed> {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16198 select(4, [0 3], [3], [3], {0, 0}) = 1 (out [3], left {0, 0})
16129 fstat64(4, {st_mode=S_IFREG|0644, st_size=520859, ...}) = 0
16198 send(3, "GET / HTTP/1.1\r\nHos\377\363\377\375\6", 24, 0 <unfinished ...>
16129 _llseek(4, 520859,  <unfinished ...>
16198 <... send resumed> )              = 24
16198 select(4, [0 3], [3], [3], {0, 0} <unfinished ...>
16129 <... _llseek resumed> [520859], SEEK_SET) = 0
16198 <... select resumed> )            = 1 (out [3], left {0, 0})
16198 send(3, ": www.google.com\r\n\r\n", 20, 0 <unfinished ...>
16129 stat64("/etc/localtime",  <unfinished ...>
16198 <... send resumed> )              = 20

If you closely look at the line 16198 send(3, "GET / HTTP/1.1\r\nHos\377\363\377\375\6", 24, 0 in the trace, the string "Host" is replaced by some "Hos\377\363\377\375\6". I'm not sure why this occurs once in a while and also this closes the telnet connection I established. Please let me know if you need more data.

Why are you making HTTP request to a telnet server?, or you are just testing connecting to google.com at 80? — cyraxjoe, Jul 13 '11 at 07:52
We are trying to check if a server support persistant connections(can post multiple requests after getting connected). — Velu Sankaran, Jul 13 '11 at 09:43

score 1 · Answer 1 · edited May 23 '17 at 10:31

1

Though communicate is better, if you really have to use stdin.write you can make the calls non-blocking using:

fcntl.fcntl(p.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)

for better understanding read this:

io non blocking

edited May 23 '17 at 10:31

Community

1
1

answered May 17 '17 at 10:01

Aditya Rohilla

327
2
5
15

score 1 · Answer 2 · edited Oct 20 '12 at 01:47

1

What if you try to use the communicate() method to send data to the subprocess instead of using stdin, as the latter is discouraged by Python's documentation:

Warning: Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Try this:

p.communicate(input='GET / HTTP/1.1\nHost: ' + hostheader + '\n\n')

edited Oct 20 '12 at 01:47

Sean Vieira

155,703
32
311
293

answered Jul 13 '11 at 07:36

Cédric Julien

78,516
15
127
132

1

Yes, I can try it but the problem is its not one command I wanted to execute in that opened process. Instead, I need to execute many in that session. Thats the first reason I decided to use poll() and stdin. – Velu Sankaran Jul 13 '11 at 08:38

score 0 · Answer 3 · answered Jun 30 '20 at 05:18

If there is no need to interact, I don't use communicate, but put the text into a pipe ahead of time. I've not found a more pythonic "text_to_stream" yet.

import os
import sys
import subprocess


def text_to_stream(text):
    p = os.pipe()
    os.write(p[1], text.encode('utf-8'))
    os.close(p[1])
    return os.fdopen(p[0], "r")


def run(cmd, stdin_text=None):
    stdin = text_to_stream(stdin_text) if stdin_text else sys.stdin
    result = subprocess.run(
        cmd, stdout=sys.stdout, stderr=sys.stderr, stdin=stdin)
    result.check_returncode()

Python: subprocess.stdin.write is not working properly

3 Answers3