Python Script Hanging After Executing Jar File with Subprocess

Question

Can't seem to figure out what is causing the script to hang, I know for sure it is hanging as the code below the "p2" variable doesn't seem to get executed at all.

The jar file does what I need it to do which to summarize in a few words, downloads all the necessary files and folders from somewhere.

I need to kill the process before removing files related to it else there will be an error relating to it being occupied by another process which I am sure it is self explanatory.

Here is the full code with the paths changed a little for reasons:

#!/usr/bin/python3.8

import os
import shutil
import json
import time
import requests
import subprocess
import wget

from zipfile import ZipFile
from glob import glob


def post_to_slack(message):
    webhook_url = "someurlhere"

    encoded_data = json.dumps({'text': message}).encode('utf-8')
    response = requests.post(
        webhook_url,
        data=encoded_data
    )
    # print(str(response.status_code))
    

src = "/somepath/ATM6/"
dest = "/someotherpath/All the Mods 6/"
extension = ".zip"

files_in_dest = os.listdir(dest)
for file in files_in_dest:
    filepath = os.path.join(dest, file)
    try:
        if os.path.isfile(filepath) or os.path.islink(filepath):
            os.unlink(filepath)
        elif os.path.isdir(filepath):
            shutil.rmtree(filepath)

    except Exception as e:
        post_to_slack(f"Error occured in {os.path.basename(__file__)}!")

for item in os.listdir(src):
    abs_path = os.path.join(src, item)

    if item.endswith(extension):
        file_name = os.path.abspath(abs_path)
        zip_ref = ZipFile(file_name)       
        zip_ref.extractall(src)
        zip_ref.close()

time.sleep(20)

contents_in_dir = os.listdir(src)
folder = [content for content in contents_in_dir if content.startswith("SIMPLE") and not content.endswith(extension)]

os.chdir("{}{}".format(src, folder[0]))

wget.download("https://github.com/AllTheMods/alltheservers/releases/download/2.0.1/serverstarter-2.0.1.jar")

p1 = subprocess.run(["chmod", "+x", "serverstarter-2.0.1.jar"], stdout=subprocess.DEVNULL)

p2 = subprocess.Popen(["java", "-jar", "serverstarter-2.0.1.jar", "&"], stdout=subprocess.DEVNULL)
time.sleep(180)
p2.kill()    

try:
    log_files = glob(src + "**/*.log")
    for files in map(str, log_files):
        os.remove(files)

    zip_files = glob(src + "**/*.zip")
    for files in map(str, zip_files):
        os.remove(files)

    startserver_files = glob(src + "**/startserver.*")
    for files in map(str, startserver_files):
        os.remove(files)

    serverstarter_files = glob(src + "**/serverstarter*.*")
    for files in map(str, serverstarter_files):
        os.remove(files)

    files_to_move = glob(src + "**/*")
    for files in map(str, files_to_move):
        shutil.move(files, dest)

    time.sleep(20)

    forge_jar_file = glob(dest + "forge-*.jar")
    for files in map(str, forge_jar_file):
        print(files)
    os.rename(files, "{}{}".format(dest, "atm6.jar"))

except Exception as e:
        post_to_slack(f"Error occured in {os.path.basename(__file__)}! {e}")

quit()

PeptideWitch · Answer 1 · 2021-11-15T05:36:13.357

1

This line here: p2 = subprocess.Popen(["java", "-jar", "serverstarter-2.0.1.jar", "&"], stdout=subprocess.DEVNULL) creates a Popen object but it doesn't trigger the subprocess call. So your script jumps to the next line, which is a time.sleep(180) call; have you timed your 'hang'? It's probably 180 seconds in length. After that, the rest of your code should execute.

Take a look at this answer here; you might want to wrap your Popen inside of a with statement and deal with the execution using a .wait() command.

Something like:

with subprocess.Popen(["java", "-jar", "serverstarter-2.0.1.jar", "&"] as sub:
    try:
        sub.wait(timeout=timeout)
    except:
        sub.kill()

edited Nov 15 '21 at 05:36

answered Nov 15 '21 at 05:03

PeptideWitch

2,239
14
30

Seems like this did not resolve it. The jar file get's executed just fine like the code I showed above, but it seems like none of the code after that get's executed. Works fine on my local windows but this script will be running on an Ubuntu machine. – zeroerrors0945 Nov 15 '21 at 12:38
I think I may see a problem - could you try something? In the Popen command, you've got a `"&"` at the end, which means the subprocess terminal completes the jar code and then just sits there expecting a new command. I think this is unnecessary because the Popen subprocess is a process that should terminate upon completion of the `.wait()`, so we dont need to explicity tell the subprocess shell that we want parallel execution. Try and remove the `&`. – PeptideWitch Nov 15 '21 at 22:16
Seems odd, again the jar does what I need to do but everything below the same line does not get executed. Works perfectly fine locally so I am kind of wondering why it doesn't work on my linux machine. Once I've Ctrl+C and did a "ps aux | grep serverstarter" it seems it is still running so for sure the jar file did not get killed via the script. – zeroerrors0945 Nov 16 '21 at 10:44
As a sanity check, can you dial into your linux box and manually process the java `serverstarter-2.0.1.jar` using the command line and see if that executes + exits fine outside of the script? – PeptideWitch Nov 16 '21 at 22:12
It definitely does. One problem is the jar file is made to run without necessarily exiting with no explicit actions taken to kill the process of it, hence the purpose of the script. It seems like after the jar is executed via the script, it causes the rest of the script to hang. This confuses me as it works fine on windows apparently. So I am not entirely certain what exactly is happening. – zeroerrors0945 Nov 17 '21 at 10:26

Python Script Hanging After Executing Jar File with Subprocess

1 Answers1