1

Here is the reduced requirement of what I am trying to achieve.

# run.py
import requests
import time

from subprocess import Popen, PIPE

server = Popen("./app.py", stdout=PIPE, stderr=PIPE, shell=True)
time.sleep(1)
res = requests.get("http://localhost:1234/")
assert res.status_code == 200
server.kill()
server.terminate()
res = requests.get("http://localhost:1234/")
print res

And the actual server script.

#!/usr/bin/env python
from flask import Flask, make_response, request

app = Flask(__name__)
@app.route('/')
def view():
    return make_response("")

if __name__ == "__main__":
    app.run(host="localhost", port=1234)

On the command line I run python run.py. From shell:

(t)yeukhon@fubini:/tmp$ ps aux|grep app
yeukhon  21452  0.6  0.4  16416  9992 pts/2    S    03:50   0:00 python ./app.py
yeukhon  21471  0.0  0.0   4384   804 pts/2    S+   03:51   0:00 grep --color=auto app

So app.py is still hanging there. I have to kill it from the command line. In fact, the last line of run.py tells us the server was still alive (200 was returned).

I tried to kill with os.kill(server.pid, signal.SIGTERM) and os.kill(server.pid, signal.SIGKILL) but none works.

Normally kill will work, but I am really not sure why it can't receive the signal. I am sure somehow Flask is refusing to stop.

What options do I have?

strangely, my script above works perfectly fine on Mac OSX (I am on 10.8.5, Mountain Lion). So far I have tested on two Ubuntu 12.04 machines and they have the same behavior. I am running Python 2.7.3 on both Ubuntu machines and Python 2.7.2 on my Mac OSX.

correction: The only option I have is to use http://flask.pocoo.org/snippets/67/. But I prefer not to. And yes, I have to launch one using Popen.

CppLearner
  • 16,273
  • 32
  • 108
  • 163
  • Hi CppLearner. I encountered the same issue. However, it seems like I have to set shell=True for running the flask through popen. Otherwise, I will get error message like "raise child_exception OSError: [Errno 2] No such file or directory". Therefore, I wonder how you revised your code finally. When shell=True, the server started from popen just keep running. – Romaboy Jul 26 '16 at 00:16

3 Answers3

2

By specifying shell=True, the command is run by subshell.

server = Popen("./app.py", stdout=PIPE, stderr=PIPE, shell=True)

server.terminate will kill the subshell, but web server will not be killed.

How to verify this? Try print server.pid after the Popen call, and compare that with ps output.

falsetru
  • 357,413
  • 63
  • 732
  • 636
  • Yes. That works. I misunderstood the usage of ``shell``. But what do you mean with the verification line? Also, why can't a daemonized process be killed? Thanks a lot. – CppLearner Feb 17 '14 at 09:06
  • @CppLearner, output of `print server.pid` and `ps ..| grep ..` will be different. – falsetru Feb 17 '14 at 09:07
  • Thanks. Do you know of any good resource to help me understand this better? I don't mean to go through my OS book again. Sorry but appreciate your help. – CppLearner Feb 17 '14 at 09:09
  • @CppLearner, Sorry, non-daemonized process also does not be affected. (at least in Linux). removed that clause from the answer. – falsetru Feb 17 '14 at 09:13
  • @CppLearner, See http://stackoverflow.com/questions/269494/how-can-i-cause-a-child-process-to-exit-when-the-parent-does and http://stackoverflow.com/questions/395877/are-child-processes-created-with-fork-automatically-killed-when-the-parent-is – falsetru Feb 17 '14 at 09:23
2

Remove shell=True from your Popen. This will make the first request. Kill the process. And then throw an exception for the second attempt.

Jakob Bowyer
  • 33,878
  • 8
  • 76
  • 91
0

Problem quite old but I met similar when I was running flask server with subprocess.Popen in conftest.py with pytest in docker container on Mac OS. This piece of code is not working for me (in conftest.py)

flask_server_proc = subprocess.Popen([
  'python', 'code/flask_server/main.py',
  '--local'
])

flask_server_proc.kill()

Parent process was killed but server still was running and answering requests. So I did this:

import psutil
import subprocess

server = subprocess.Popen([
    'python', 'code/flask_server/main.py',
    '--mysql-user', 'root',
    '--mysql-password', 'pass',
    '--mysql-host', 'mysql',
    '--local-artifact-store'
])

# Here I check if server is up. For example with some health check
# url. Sleep mimics it.
import time
time.sleep(10)
print(server.pid)

# Somewhere here you will call 'yield server'. This is just snippet
# with potential solution of problem so no yielding it.
# The next part should be run after 'yield server'.

# This should kill server.
# server.kill()

# Actually it should be one such process like flask.
flask_to_kill: List[psutil.Process] = []
for process in psutil.process_iter():
    if 'code/flask_server/main.py' in process.cmdline():
        flask_to_kill.append(process)

def on_terminate(proc: psutil.Process):
    print("process {} terminated with exit code {}".format(proc, proc.returncode))

for f in flask_to_kill:
    f.terminate()

gone, alive = psutil.wait_procs(flask_to_kill, timeout=3, callback=on_terminate)

for to_stab in alive:
    to_stab.kill()

so looks like remedy is to use psutil package to kill server after tests. Happy hacking!

wiesiu_p
  • 558
  • 7
  • 6