1

I have a python script to start some containers, wait for them to finish execution and then start a few others. I wanted to get the container logs and this bash command worked for me:

docker logs -f container-name &> tmp.log &

However when I try to add it to my python script using `subprocess.run like below, it doesn't create a new file.

subprocess.run(
     [
         "docker",
         "logs",
         "-f",
         "container-name",
         "&>",
         "tmp.log",
         "&"
     ],
     cwd=os.getcwd(),
     shell=False,
     stdout=subprocess.DEVNULL,
 )
Tony Stark
  • 511
  • 2
  • 15

1 Answers1

1

You are using shell redirection (&> and &) which are not recognized by subprocess.run() due to setting shell=False.

Although setting shell=True could be a solution to resolve this issue, it should be used with caution due to the potential security risks involved (See: Actual meaning of 'shell=True' in subprocess).

A safer approach would be to just create and/or open the file and redirect the output to it:

import subprocess
import os

log_file = "tmp.log"
if not os.path.exists(log_file):
    open(log_file, "w").close()

with open(log_file, "a") as outfile:
    subprocess.run(
        ["docker", "logs", "-f", "container-name"],
        cwd=os.getcwd(),
        shell=False,
        stdout=outfile,
    )
eDonkey
  • 606
  • 2
  • 7
  • 25
  • 1
    `shell=True` didn't work for me, I just tried with os.system and it worked. But I like your solution more and it works. Instead of opening and closing the file, and then appending, your answer with opening the file with write mode works good as well. Edit: I got it, you're trying to create the file if it doesn't exist, makes sense. I had this line before instead: `os.makedirs(os.path.dirname(log_file), exist_ok=True)` – Tony Stark Mar 03 '23 at 10:13
  • 1
    It didn't? Weird.. I didn't actually test `shell=True` but I will now. Note that when only opening the file in append mode, it will not create the file if it doesn't exit. – eDonkey Mar 03 '23 at 10:17
  • @TonyStark `os.makedirs(os.path.dirname(log_file), exist_ok=True` will create a *directory* (without raising an error if it already exists). It won't create a *file* - which, based on your question, seems to be the goal. – eDonkey Mar 08 '23 at 09:08