0

I am wracking my brain over here trying to do a simple function with python without using shell=true and am not getting the results I need. I am using python 2.7 on Linux.

I have tried multiple methods of doing this. It works fine if I use shell=true like so:

import subprocess as s

var1 = s.call("echo $HOME", shell=True)

and 

import subprocess as s

var1 = s.check_output("echo $HOME", shell=True)

both return

/home/myhost

... like it should but everything else I tried fails. In most cases it seems that it is passing my variable as a string instead of a command.

These are the results I have received with the various methods:

import subprocess as s

print var1

returns

myhost@me:~/Desktop$ python home_test.py 
  File "home_test.py", line 42, in <module>
    main()
  File "home_test.py", line 11, in main
    my_test()
  File "home_test.py", line 16, in check_auth
    var1 = s.check_output("echo $HOME")
  File "/usr/lib/python2.7/subprocess.py", line 567, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Then I tried this:

import subprocess as s

var1 = s.Popen(['echo', '$HOME'], stdout=s.PIPE) 

print var1

returns

myhost@me:~/Desktop$ python home_test.py 
<subprocess.Popen object at 0x7f297ebcbf90>
myhost@me:~/Desktop$echo: write error: Broken pipe

Then this:

import subprocess as s

cmd1 = '$HOME'
var1 = s.Popen(['/bin/echo', cmd1], stdout=s.PIPE)

print var1

returns

('$Home\n', None)

and this

import subprocess as s

cmd1 = 'HOME'
var1 = s.Popen(['/bin/echo', cmd1], stdout=s.PIPE)
print var1

returns

myhost@me:~/Desktop$ python home_test.py 
('Home\n', None)

found some doc that said to use os... which also failed, which isn't surprising after finding out this had been deprecated in 2.6.

import os

var1 = os.popen('echo $HOME')
print var1

returns

<open file 'echo $HOME', mode 'r' at 0x7f1c6b26f6f0>
sh: echo: I/O error

Then my final attempt (actually there were more... but we will just leave it at this)

import subprocess as s

var1= s.Popen(["echo", "$HOME"], stdout=s.PIPE).communicate()[0] 

print var1

returns:

myhost@me:~/Desktop$ python home_test.py 
$HOME

Can someone please point me in the right direction? I have spent an entire day fiddling with this and I need help please. Thank you in advance for any help given.

ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
shells belles
  • 1
  • 2
  • In `var1 = s.Popen(['echo', '$HOME'], stdout=s.PIPE)`, only the first array element is treated as a script. Subsequent array elements are arguments to that script. – Charles Duffy Oct 19 '16 at 17:25
  • ...btw, what's your real use case? In general, using `shell=True` is the Wrong Thing; it introduces security bugs if not used with sufficient (substantial!) caution, and does nothing you can't do by making your Python more verbose. – Charles Duffy Oct 19 '16 at 17:29

3 Answers3

3

Lets start with the answer

import os
home = os.environ['HOME']
print(home)

Now onto your attempts and results. Your main problem (besides a few random bugs) is that $HOME is replaced with the value of the environment variable by the shell. Without the shell argument expansion $HOME is just a string and that is what you get.

Mad Wombat
  • 14,490
  • 14
  • 73
  • 109
  • Thank you this worked. I don't know why I didn't think of this. Probably because I was beating my head on the wall trying to use subprocess. – shells belles Oct 19 '16 at 16:22
  • 1
    Your main problem was that you were trying to do shell variable expansion without using the shell :) – Mad Wombat Oct 19 '16 at 17:03
  • 1
    @shellsbelles: As a rule, the vast majority of the simple shell utilities have better (or at least lower overhead) equivalents in Python. Even in the cases where `subprocess`ing the shell utility appears faster (e.g. reading compressed data from a `.gz` file is superficially faster), it's usually only because `subprocess` does the parallelism "for free" (and in the `gzip` module's case, on some versions, it's because they didn't buffer by default, `io.BufferedReader` wrapping would fix it); if you threaded or parallel processed the Python equivalent code, it wouldn't lose out much. – ShadowRanger Oct 19 '16 at 17:06
  • A handful of examples: Reading the environment is best done with `os.environ`, moving and copying files with `shutil`, dealing with compressed data with the `gzip`/`bzip2`/`lzma` modules, `ls` without globbing becomes `os.listdir` (`os.scandir` on modern Python for better perf) or the `glob` module for globbing behavior, `find` becomes `os.walk`, etc. – ShadowRanger Oct 19 '16 at 17:08
1

It sounds like you really just want to get the value of an environment variable. os.getenv('HOME') should do that for you.

xulfir
  • 131
  • 3
  • Thanks, however this did not work, I put the results in the comments – shells belles Oct 19 '16 at 16:12
  • Nevermind, I had a typo. This also worked. Thanks to the both of you. – shells belles Oct 19 '16 at 16:22
  • 1
    Tangential warning: As a rule, it's better to use `os.environ` exclusively and ban the `getenv`/`putenv` for consistency. Specifically, it's a problem if you mutate the environment; calling `putenv` doesn't change `os.environ`, but mutating `os.environ` will also do `putenv` (if supported). Additionally, `getenv` and `putenv` aren't available on all OSes (though admittedly, it's only rare UNIX-like flavors that don't support it AFAICT), while `os.environ` is always supported. – ShadowRanger Oct 19 '16 at 17:15
1

The answer mentioning os.environ['HOME'] is correct, but it's probably not what you want for portability. There is no guarantee that HOME is set (it's by convention, not required; users could delete it, shells could choose not to set it, etc.). And if you eventually want portability to Windows, that's not its name. If you want the user's home directory, don't check the environment directly, instead, use os.path.expanduser to use smarter home directory lookup logic:

import os.path

home = os.path.expanduser('~')

This will work even if HOME is not defined, and generalizes to looking up other user's home directories (os.path.expanduser('~someotheruser')).

In Python 3.5+, you could also use pathlib.Path.home() to get a Path object representing the home directory.

Also, tangentially, if you need to expand a string with a bunch of environment variable replacements in general, you might want to look at os.path.expandvars, which does the environment lookup and string formatting as a single operation.

ShadowRanger
  • 143,180
  • 12
  • 188
  • 271