How to run a shell script in python from memory?

Question

The application I'm writing retrieves a shell script through HTTP from Network, I want to run this script in python however I don't want to physically save it to the hard drive because I have its content already in memory, and I would like to just execute it. I have tried something like this:

import subprocess

script = retrieve_script()
popen = subprocess.Popen(scrpit, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

Are there any alternatives to run a shell script from memory?

Is this maybe a duplicate of [How to run multiple commands synchronously from one subprocess.Popen command?](https://stackoverflow.com/questions/39721924/how-to-run-multiple-commands-synchronously-from-one-subprocess-popen-command/39722695)? — YellowShark, Oct 18 '18 at 14:18

unutbu · Answer 1 · 2018-10-19T11:45:48.333

Are you using a Unix-like OS? If so, you should be able to use a virtual filesystem to make an in-memory file-like object at which you could point subprocess.Popen:

import subprocess
import tempfile
import os
import stat

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

content = retrieve_script_content()
with tempfile.NamedTemporaryFile(mode='w', delete=False, dir='/dev/shm') as f:
    f.write(content)
    os.chmod(f.name, stat.S_IRUSR | stat.S_IXUSR)
    # print(f.name)
popen = subprocess.Popen(f.name, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                         shell=True)
stdOut, stdErr = popen.communicate()

print(stdOut.decode('ascii'))
# os.unlink(f.name)

prints

command1
command2
command3

Above I used /dev/shm as the virtual filesystem since Linux systems based on Glibc always have a tmpfs mounted on /dev/shm. If security is a concern you may wish to setup a ramfs.

One reason why you might want to use a virtual file instead of passing the script contents directly to subprocess.Popen is that the maximum size for a single string argument is limited to 131071 bytes.

Thanks, the code will be running on Raspberry Pi `Rasbian Jessie OS`. — Mykhailo Seniutovych, Oct 18 '18 at 15:15

that other guy · Accepted Answer · 2018-10-18T18:26:51.627

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

That is not the case. Instead, the reason why it doesn't work is:

The declaration of retrieve_script has to come before the call
You call it retrieve_script_content instead of retrieve_script
You misspelled script as scrpit

Just fix those and it's fine:

import subprocess

def retrieve_script():
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

script = retrieve_script()
popen = subprocess.Popen(script, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()
print(stdOut);

Result:

$ python foo.py
command1
command2
command3

However, note that this will ignore the shebang (if any) and run the script with the system's sh every time.

It was a typo, but I posted a test code sample here. I can confirm that running bash scripts like that indeed works on `Linux`. However when I ran similar code on `Windows` it did not work, so you will not be able to run `batch files` like that on `Windows`. — Mykhailo Seniutovych, Oct 24 '18 at 17:06

Serge · Answer 3 · 2018-10-24T01:44:50.650

0

You can execute multi command script with Popen. Popen only restricts you to one-command string when shell flag is False, yet it is possible to pass a list of commands. Popen's flag shell=True allows for multi-command scripts (it is considered unsecure, though what you are doing - executing scripts from the web - is already very risky).

edited Oct 24 '18 at 01:44

answered Oct 18 '18 at 14:16

Serge

3,387
3
16
34

We have a web service that provides our application with shell scripts so I know what I'm doing no worries :) – Mykhailo Seniutovych Oct 18 '18 at 14:18
The answer address general question of how one can execute a downloaded multiline script, though indeed does not goes to reviewing the sample code. – Serge Oct 19 '18 at 12:22

How to run a shell script in python from memory?

3 Answers3