Python script that can be externally terminated in a graceful manner

Question

My server generates every minute ~30 xml files, and I would like to do a sftp to transfer files from server to my machine. I would like to use Paramiko to do an sftp, I am getting what I want with the bellow script:

import paramiko
import os
import time
filename = "addresses.text"
localpath= "******"
serverpath= "*******"


while True:
    try:
        current_time = time.time()
        with open(filename) as f:
            data = f.readlines()
        for ipAddr in data:
            ssh = paramiko.SSHClient()
            ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            ssh.connect(ipAddr,username="root",password="root")
            sftp = ssh.open_sftp()
            for element in sftp.listdir(serverpath):
                if element.endswith(".xml"):
                    try:
                        print(os.stat(localpath+f))
                    except:
                        try:
                            creation_time = sftp.stat(serverpath+element).st_mtime
                            if (current_time+3400 - creation_time) / (3600) <= 1:
                                sftp.get(serverpath+element,localpath+element)
                        except:
                            print(Exception)
            sftp.close()
            ssh.close()
        for f in os.listdir(localpath):
            creation_time = os.path.getctime(localpath+f)
            print((3600+current_time - creation_time) / (3600))
            if (3600+current_time - creation_time) / (3600) >= 1.8:
                os.unlink(localpath+f)

    except OSError as e:
        print(OSError)

I would like to do something like start sftp.py and then run my script in the background. And when I want to stop the connection just run stop sftp.py

What's wrong with `while True:`? - Except that you do not need to reconnect in every iteration and that you should probably add some delay between iterations. — Martin Prikryl, Oct 24 '18 at 08:30
In fact, this script will be added to a jenkins job in the future to install a VM and then run this script to send files to an appropriate machine. The problem is with my script I can run it but if I would like to stop this process I should do an ps -edf I think. That's why I am looking for something easier to automated the process in the futur. — C.med, Oct 24 '18 at 08:52
So you question is only, how to implement a graceful stopping of a Python script - i.e. your question is not about SFTP or Paramiko, right? — Martin Prikryl, Oct 24 '18 at 09:00

Martin Prikryl · Accepted Answer · 2018-10-24T11:17:01.977

How this is usually implemented is that a running process stores its PID to a file.

And then you can implement another script (or a parameter to your existing script - like my example below does) that reads the PID from the file and terminates that process.

You can even make the termination graceful.

import signal
import time
import os
import sys

pidfile = "sftp.pid"

if (len(sys.argv) > 1) and (sys.argv[1] == "stop"):
    if os.path.isfile(pidfile):
        with open(pidfile, "r") as f:
            pid = int(f.read())
        print("stopping sftp process {0}".format(pid))
        os.kill(pid, signal.SIGTERM)
    else:
        print("sftp is not running")
    sys.exit()

class GracefulKiller:
    kill_now = False
    def __init__(self):
        signal.signal(signal.SIGINT, self.exit_gracefully)
        signal.signal(signal.SIGTERM, self.exit_gracefully)

    def exit_gracefully(self,signum, frame):
        self.kill_now = True

if __name__ == '__main__':
    pid = os.getpid()
    print("sftp is starting with pid {0}".format(str(pid)))
    with open(pidfile, "w") as f:
        f.write(str(pid))

    killer = GracefulKiller()
    while True:
        time.sleep(1)
        print("doing something in a loop ...")
        if killer.kill_now:
            break

    print "End of the program. I was killed gracefully :)"

    os.remove(pidfile)

Thank you @Martin, I used finally a shell script, but your solution is the best one ;) — C.med, Oct 24 '18 at 09:26

Python script that can be externally terminated in a graceful manner

1 Answers1