2

I am looking for a way for my application to start its background logging application as a completely separate process.

The program is 90% user input so crashes happen, I need to see the logs but when using subprocess it spawns as a child process which means when the main application crashes so does the logger.

I have tried using os.system but that launches it as part of the current process stopping all execution until the process has completed.

The Logger accepts 3 arguments, a temp folder path, a port and a process id. Theses are required to enable the program to communicate to the main application.

Both applications are frozen and build on windows explicitly.

There is not much code I can show but here is how everything is generated.

if getattr(sys, 'frozen', False):
    debug = False
    exe_path = os.path.dirname(sys.executable)
elif __file__:
    debug = True
    exe_path = os.path.dirname(os.path.abspath(__file__))

def get_open_port():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('', 0))
    addr = s.getsockname()
    open_port = addr[1]
    s.close()
    return open_port
proc = [p.info for p in psutil.process_iter(attrs=['pid', 'name']) if str(os.getpid()) in str(p.info['pid'])][0]
proc = proc['name']

tmp = tempfile.mkdtemp()
tmp = str(tmp)+"\\"

port = get_open_port()
path = os.path.join(exe_path, "logger", "Logger.exe")
#os.startfile('''%s", "%s", "%s", "%s"''' %(path, tmp, proc, port)) #failes to get past here.
#subprocess.Popen([path, tmp, proc, port]) #creates as a child process which crashes with the main.
time.sleep(2) # waits for the application to start
Ondrej K.
  • 8,841
  • 11
  • 24
  • 39
Scott Paterson
  • 392
  • 2
  • 17
  • You could make 2 completely separate processes, log to stdout in one and pipe that into a logger which reads from stdin? Or could you run it in docker? That way if the process crashes, you can still check the container logs. – Dan Feb 17 '20 at 10:15
  • @Dan Running as 2 completely separate processes is what i am looking for, Do you know how i can achieve this ? – Scott Paterson Feb 17 '20 at 10:20
  • 1
    Child cannot outlive the parent. I mean... it can (and I've done so myself, but I had a specific case and I know exactly why I did this), but it's not a correct thing to do. Why don't you execute it the other way around (logger gets executed first, then program)? Or why don't you wrap the program call (`main()`?) in some catch-all `except` so that it isn't killed when it crashes? – h4z3 Feb 17 '20 at 10:21
  • @h4z3 Im trying to create it as a separate process and not a child. There are cases (in the dev env) where the logger is not needed and having it causes more problems as it takes 5-10 mins to get errors thought while developing so having the logger start the application would slow down development. I am using try/except on 90% of my code, but its live development and i do miss things from time to time so the logger is suppose to be versatile for those cases and does check the running state of the main program encase it crashes. – Scott Paterson Feb 17 '20 at 10:26
  • 1
    Create two separate applications and then call them like `my_app | my_logger`. – Dan Feb 17 '20 at 10:31
  • Why do you need a separate logger btw? Why don't you just log to a persistent file, then you can still check the logs after a crash. – Dan Feb 17 '20 at 10:32
  • @Dan I have 2 apps, the code in the question is the code from the first app that should launch the second app. The separate logger is for security. Working on medical stuff with client information so having the log saved as plain text with client data is a big security issue. I used a persistent log file 6 months ago and to this day still receive these despite telling everyone they are no longer used and there were over 150,000 log lines as they never got deleted. – Scott Paterson Feb 17 '20 at 10:39
  • 1
    Log to a secure DB? You don't have to persist in plain text. – Dan Feb 17 '20 at 10:41
  • @Dan that secure DB is actually 99% of the crashes i get :) The log is stored in RAM, written to a temp file sent as an email attachment and then deleted. If i try include the logger as part of the main application using a db then im back in the same issue of "application crashed, no log" and as a separate process im back to "how to create a completely separate process". – Scott Paterson Feb 17 '20 at 10:46
  • You know when you "Open File Location" on your browser after downloading a file... I want to do that... if you close the browser after then it does not close the explorer window. – Scott Paterson Feb 17 '20 at 10:48
  • 1
    `Im trying to create it as a separate process and not a child.` But you want it to start from your program. So it's a child. Every process (up to init) is some other process' child. You cannot start it from your program and NOT be a child. You can run both from the OS and they will be separate. – h4z3 Feb 17 '20 at 10:54
  • 1
    "that secure DB is actually 99% of the crashes i get" well that sounds like the problem you need to solve then. Not this. The DB should be completely independent from your app. Why should it crash when the app does? – Dan Feb 17 '20 at 10:56
  • 1
    Also "sent as an email attachment and then deleted" - encrypted first I hope. But these are logs, why email and delete them. They belong in a log store. – Dan Feb 17 '20 at 10:57
  • `I am using try/except on 90% of my code, but its live development and i do miss things from time to time` I meant, you can literally wrap `if __name__=='__main__': main()` in `except`. Then you will have 100% running code (and, like, 99% of object definitions - 1% for those created in global scope) checked for exceptions. That's what I do in my scripts, "unexpected exception" I call this case. - Then I have literally everything logged for sure. You can re-raise the exception if it needs to crash anyway after logging. – h4z3 Feb 17 '20 at 10:57
  • DB is most of the issue because of ", ', \ and everything else that is user input. using try/except in if __name__ == ... doesnt work in some cases, not sure why. debug not currently encrypted but will be when i get round to it, its not finished yet. delete them because this application is running on about 40-50 laptops which can be stolen or lost and im not willing to risk 5k clients data, there are a lot of checks and although im no security expert im doing the best i can. – Scott Paterson Feb 17 '20 at 10:58
  • `The separate logger is for security. Working on medical stuff with client information so having the log saved as plain text with client data is a big security issue` Oh, my... Hope you're not in EU. NEVER log anything that could compromise the program. If you have too much info, then you log wrong info (the program should never pass sensitive data to the logger) or log it wrong. – h4z3 Feb 17 '20 at 11:00
  • `DB is most of the issue because of ", ', \ and everything else that is user input.` ffs, sanitize your inputs! That's the basics! Encode this shit or just don't insert it directly by yourself - there are many libs that deal with such stuff and check the inputs already! – h4z3 Feb 17 '20 at 11:01
  • "sanitize your inputs!" I do, user input = random stuff. i also use IPA so i also encode/decode everything for unicode. "NEVER log anything that could compromise the program" It never saves anything for more than 2 seconds as the log is stored in RAM while the application is running. I have to log stuff because of issues that cause hours of work to not be deleted. again im planning on encrypting a lot of the log once i get these issues sorted. – Scott Paterson Feb 17 '20 at 11:06
  • Anyway this is going off topic. Im still reading some stuff and it looks like the option im looking for is not supported on windows "os.fork()", are there any alternatives to this ? I want both applications to act like the user double clicked the icons :) – Scott Paterson Feb 17 '20 at 11:10
  • 1
    An answer for Windows: https://stackoverflow.com/a/13593257/1319284 – kutschkem Feb 17 '20 at 13:08
  • Maybe python-daemon can help: https://pypi.org/project/python-daemon/ (I never used that myself) – Daniel Junglas Feb 17 '20 at 14:30
  • @kutschkem yes that was the exact solution i was looking for. Turns out "Detach" was the keyword i was missing. Thanks :) – Scott Paterson Feb 18 '20 at 16:45

0 Answers0