2

I am trying to send the outputs of a Python CLI "SimpleHTTPServer" command to a pipe, in order to grep the results. But it seems that the first line of the output is not being passed to the pipe, and I can't figure out why.

Here is the typical output from running the command. The first line states that it has started a server, and the port it is using. Then it reports various network activities as they occur. So If I run python -m SimpleHTTPRequest and then load the index.html page twice, the output looks like this:

$ python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...
127.0.0.1 - - [07/May/2018 21:08:31] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [07/May/2018 21:08:36] "GET / HTTP/1.1" 200 -

If I redirect stderr to stdout, and then send to a pipe, I would expect that all lines would arrive in the stdin after the pipe. But this does not appear to be happening. When I use sed after the pipe and have it prefix all lines with "This is stdin:", the first line of the output does not appear.

$ python -m SimpleHTTPServer 2>&1 | sed "s/.*/This is stdin\:&/"
This is stdin:127.0.0.1 - - [07/May/2018 23:35:07] "GET / HTTP/1.1" 200 -
This is stdin:127.0.0.1 - - [07/May/2018 23:35:11] "GET / HTTP/1.1" 200 -

I tried different redirect options. I can redirect stdout to stderr using 1>&2, and this makes all three lines display without passing to the pipe, as expected. I tried redirecting from stdin to stdout with 0>$1, based on the discussion at this answer, but that did not send any of the lines to the pipe. I tried placing the python command in braces, based on this answer, but it did not change the result. It is clear that the network activity logs are being sent to to stderr, but it isn't clear where the first line is going. I also tried with grep after the pipe, with the same result. In all cases, the first line of the output is not arriving in the stdin after the pipe.

It appears that the python "SimpleHTTPServer" first line of output is not going to either stdout or stderr, so I can't pass it to a pipe. What is happening?

In case someone has a better way around this, here is what I'm trying to do. I want a one-line shell command that will (1) start a local server using python SimpleHTTPServer; (2) find out what port number it is using; then (3) open a browser at that localserver port. I know that SimpleHTTPServer's default port is 8000; but if that one is already in use, it will assign a different one so I need to know that. I also know that I can tell SimpleHTTPServer what port to use, but if that port is already in use then the command will fail. I know that within python you can ask what port is being used, but I need a shell script to run from a different application. So what I settled on was listening for the command's output when it starts the server, regexing the 4-digit port number, making that a variable, and then using that variable to open the right port in the browser. I am expecting the following to work, but it isn't.

$ port=$(python -m SimpleHTTPServer 2>&1 | grep -o '[0-9]\{4\}') ; open http://localhost:$port
BradA
  • 21
  • 2
  • Does [my reply](https://stackoverflow.com/a/70618255/17850902) answer your question? – jgru Jan 08 '22 at 08:13

1 Answers1

0

I'm referring to Python3 and the module http.server, since Python2 reached its end of life. However, the same situation occurs there as well. The behaviour observed by you is related to the way, that this HTTP-server is realizing concurrency, while it is not flushing stdout before calling the cuntion serve_forever(), which handles the clients' requests (See Lib/http/server.py). You could test this yourself by appending , flush=True to the function arguments in the call to print in line 1247 of Lib/http/server.py.

Since I assume, that you want to use Python3's HTTP-server anyways, you could use the following bash-script to achieve your task of spinning up an HTTP-server, finding its listening port and opening your system's default browser to open it:

#!/bin/bash

# Runs your server process
(python3 -m http.server &) >/dev/null 2>&1

# Adds delay, so that server started up
sleep 0.5

# Retrieves PID of the running server process
pid=$(pgrep -f "python3 -m http.server")

# Uses lsof to find the listening sockets
# -a is used to 'and' two filters
port=$(lsof -a -i -p $pid | grep -oP 'TCP.*:\K\d+')
echo "Server running on port ${port}"

# Opens the site
open "http://localhost:${port}"

Please note, that you would need to kill the server-process afterwards.

Hope, this helps!

jgru
  • 181
  • 5