0

So, I am trying to issue this command from a python script that collects cpu information across a predetermined number of nodes in a cluster. Here I use a fanout of 2 and only run it on nodes b127 through b129 for testing purposes.

    pdsh -f2 -w b[127-129] 'python -c "import multiprocessing
    num_cpu = multiprocessing.cpu_count()
    stat_fd = open('/proc/stat')
    stat_fd.close()"'

I printed the command and this is what it shows on the terminal. Thus, telling me that the quotes and commands are properly formatted. I get this string by executing the following code:

    python_command = "'python -c "\
                 + "\"import multiprocessing\n"\
                 + "num_cpu = multiprocessing.cpu_count()\n"\
                 + "stat_fd = open(\'/proc/stat\')\n"\
                 + "stat_fd.close()\"'"

    command = "pdsh -f2 -w b[127-129] " + python_command
    print command

Unfortunately, the line with open(\'/proc/stat\') seems to be the problem as that is the only line that causes the parser to fail due to the nested single quotes. I've tried numerous combinations of quoting and escaping in order to make it work to no avail. I've omitted some code between the opening and closing of the file to minimize the posted code.

I searched around and found this link, but found it was too simple of an example because I could replicate those commands. And yes, I know I can use bash commands to get what I want done and I may end up doing so, but this one has me beating my head on the wall. I also have scripts that gather data using top and ps so I don't need an explanation using them. I greatly appreciate the help in advance.

Community
  • 1
  • 1
Alex Ramos
  • 135
  • 2
  • 14
  • Using triple quotes is the way to go to strictly answer your question, however you might want to have a look at [ansible](http://www.ansible.com/) or [fabric](http://www.fabfile.org/) that bring the same parallel SSH capabilities as pdsh to Python directly, avoiding the need to over-engineer the quote pairs. – damienfrancois Aug 29 '14 at 12:07

1 Answers1

0

Try this:

python_command = """pdsh -f2 -w b[127-129] 'python -c "import multiprocessing
num_cpu = multiprocessing.cpu_count()
stat_fd = open(\\"/proc/stat\\")
stat_fd.close()"'"""

In Python, you can use triple quotes ("""...""" or '''...''') for strings containing new lines and single/double quotes.

The last level of quotes (on the open() line) will need to be escaped so that they don't conflict with outer quotes. You also need to escape the backslashes so they aren't immediately consumed by Python when interpreting the string: \\".

grc
  • 22,885
  • 5
  • 42
  • 63
  • Winner, thanks a lot. I've seen this before but completely forgot, much appreciated sir. I'll try it out when I get to work and check the performance vs some other methods I've been using. Sorry I can't vote up, this is my first Stack Overflow question so I don't have 15 rep points yet. – Alex Ramos Aug 29 '14 at 13:33