Python2.7 - Execute shell command with subprocess and use output for the other

Question

I'm trying to execute some command in a pod but it's not working. What could be your suggessing?

import subprocess

p = subprocess.Popen('kubectl get pods -o=custom-columns=NAME:.metadata.name | grep -i nottepod' , shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in p.stdout.readlines():
    print line,

#Here the line = exact name of the pod. After here not working. it's not login to the pods

p2 = subprocess.Popen("kubectl exec -it' +$(line)+ '-- bash'" , shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

p3 = subprocess.Popen("ls -lrth" , shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line2 in p3.stdout.readlines():
    print line2,

retval = p3.wait()

I expect to run a couple of command after get in the pod with p2 step.

What I really try is, first, get the pods name(p1), login to the pod with exec command(p2) that's why I need p1 output to use in it. After I'm login to the pod, I also need to run some commands (p3..etc) That's my main purpose. Below you can see what's happening when I run them manually.

[abc@master ~]$ kubectl get pods -o=custom-columns=NAME:.metadata.name | grep -i nottepod

nottepod-xyz

[abc@master ~]$

[abc@master ~]$ kubectl exec -it nottepod-xyz -- bash

Defaulted container "notte" out of: notte, key

[root@master -notte ]# ls -lrth

[root@master -notte ]# ls -lrth

total 216M

drwxr-xr-x 2 root root 6 Jul 6 2022 Cache

-rwxr-xr-x 3 1037 users 111 Dec 26 16:51 start.sh

-rwxr-xr-x 3 1037 users 291 Dec 26 16:51 rataten.sh

Please provide more information. What are you trying to do? What is a pod in this context? How do the commands look if you'd do them manually? From what I can see up to now: `line` may have a newline at the end (use `line.strip()` to get rid of them). p2 seems to run a `kubectl` command that opens an interactive bash shell (therefore never returning, since you don't send any `exit` equivalent on its stdin). Also, `$()` is not a python substitution format, but a bash substitution format, substituting the command `line` by its output, and the command `line` probably does not exist in bash. — orithena, Jan 09 '23 at 12:00
You do know that Python 2 went out of support several years ago? — tripleee, Jan 09 '23 at 13:11
@tripleeeii for sure I know but for some reason I have to use it. — GND, Jan 10 '23 at 08:36
Okay, with the new information, a quick hint: You need to combine p2 and p3 into one command that executes `ls` remotely _and returns_. Check out the `-c` argument to bash: instead of opening an interactive shell, the given command is executed and execution then returns. The p2 command line may then look like this: `kubectl exec -it nottepod-xyz -- bash -c "ls -lrth"`. Try it manually: does it output the desired data and return to the _local_ command line? Then you can use it. — orithena, Jan 10 '23 at 09:58

tripleee · Answer 1 · 2023-01-10T11:26:38.463

If you merely meant to interpolate the Python variable named line, you probably want

p2 = subprocess.check_output(
    ["kubectl", "exec", "-it", line, "--",
     "bash", "-c", "couple of commands; more commands"])

Notice how this also avoids the overhead and security implications of shell=True

Generally, avoid Popen always when you can.

If you really want the exec bash to accept commands interactively, you do need Popen, and need to submit those commands as standard input to the interactive shell. But this is probably best avoided if you can.

Here is a refactoring to use check_output throughout, and avoid shell=True everywhere.

import subprocess

pods = subprocess.check_output(['kubectl', 'get', 'pods', '-o=custom-columns=NAME:.metadata.name'])
nottepod = [pod for pod in pods.splitlines() if 'nottepod' in pod.lower()]
assert len(nottepod) == 1
nottepod = pod[0]
print(nottepod,)

lsout = subprocess.check_output(
    ['kubectl', 'exec', '-it', nottepod, '--',
     'bash', '-c', 'ls -lrth'])
for line in lsout.splitlines():
    print(line,)

Splitting the output into lines only then to print them each separately is obviously silly, but I imagine you might want to do something more useful with each output line ultimately. If not, simply print(lsout,) in one go for simplicity.

(The following was part of my original answer before you clarified your question. I have moved it down here in case there are future visitors with a different problem.)

The expression $(line) is wrong, but we can't really guess what you hope for it to mean.

If line is a command you want to run as a command substitution, the syntax is correct as such, but you will probably want to remove the single quotes and the plus marks around it, and maybe add double quotes. (This seems implausible, but I'll mention it anyway.)

# XXX FIXME: implausible but still possible
p2 = subprocess.check_output(
    'kubectl exec -it "$(line)" -- bash',
    shell=True)

If you wanted to use the value of the Python variable line as the command to run, that's

# XXX FIXME: also implausible but still possible
p2 = subprocess.check_output(
    'kubectl exec -it "$(' + shlex.quote(line) + ')" -- bash',
    shell=True)

It's not clear what you wanted to do with the output from the subprocess. You might want to use check_call if you don't care about the output, but then you probably want stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL if you don't want them displayed to the user.

"Below" where? Is the requirement to interact with the `exec` not something you can be flexible with? It will be much simpler to run multiple `exec` commands in sequence, but of course, if one invocation produces state which needs to be maintained over into the following one, you might need to force them to run in the same shell; but then, you probably can run them both with something like `bash -c "first command; second command"` and give up the interactive dialog with the shell. — tripleee, Jan 10 '23 at 08:45
What I really try is, first, get the pods name(p1), login to the pod with exec command(p2) that's why I need p1 output to use in it. After I'm login to the pod, I also need to run some commands (p3..etc) That's my main purpose. Below you can see what's happening when I run them manually. [abc@master ~]$ kubectl get pods | grep-i nottepod **nottepod** 2/2 Running 0 87m [abc@master ~]$ [abc@master ~]$ **kubectl exec -it nottepod -- bash** Defaulted container "notte" out of: notte, key [root@master -notte ]# ls -lrth — GND, Jan 10 '23 at 08:48
As you can see, posting commands in comments is pretty useless. Several users have asked you to clarify your question; please do that instead. — tripleee, Jan 10 '23 at 08:49

Python2.7 - Execute shell command with subprocess and use output for the other

1 Answers1