0

I have a python (2.7) package which depends on a library written in C (The brain imaging suite Freesurfer). In order to check for the library I wrote the following:

def crash_if_freesurfer_not_found():
  import os, subprocess
  with open(os.devnull) as devnull:
    p = subprocess.call(['which', 'mri_info'], stdout=devnull, stderr=devnull)
  if p!=0:
    print 'Useful error message'
    sys.exit(1)

If which finds the program mri_info in the path, then it has return code 0, otherwise it has return code 1. On the systems I developed this on, this worked.

I am on a system now, where this code is failing. But I was quite confused why, because `os.environ['PATH'] includes ~/freesurfer/bin, where this program is located.

In [3]: os.environ['PATH'].split(':')[0]
Out[3]: '/home/aestrivex/freesurfer/freesurfer/bin'

In [11]: ls /home/aestrivex/freesurfer/freesurfer/bin | grep mri_info
  mri_info*

So I dug deeper and I found this odd behavior which I don't understand:

In [10]: with open(os.devnull) as nil:
  p = subprocess.call(['which', 'mri_info'], stdout=nil, stderr=nil)
  ....:     

In [11]: p
Out[11]: 1

In [12]: with open(os.devnull) as nil:
  p = subprocess.call(['which', 'mri_info'], stdout=nil)
  ....:     
  sh: printf: I/O error

In [13]: with open(os.devnull) as nil:
  p = subprocess.call(['which', 'mri_info'])
  ....:     
  /home/aestrivex/freesurfer/freesurfer/bin/mri_info

aestrivex@apocrypha ~/gselu $ which which
  /usr/bin/which

So, which fails with an I/O error whenever the python subprocess redirects stdout to /dev/null, and otherwise operates normally.

But I get perfectly normal behavior from which otherwise than in a python subprocess

aestrivex@apocrypha ~/gselu $ which mri_info
/home/aestrivex/freesurfer/freesurfer/bin/mri_info
aestrivex@apocrypha ~/gselu $ which mri_info > /dev/null
aestrivex@apocrypha ~/gselu $ echo $?
0

There are several ways for me to fix this check, but my question is, what possible context could cause this bug to see which behave like this in a python subprocess?

10 Rep
  • 2,217
  • 7
  • 19
  • 33
aestrivex
  • 5,170
  • 2
  • 27
  • 44
  • Can you execute other commands from python and write it's output to a file opened via `open(os.devnull)`? Also, it seems weird to expect anything written to that "file" since you've opened it in the default mode (`'r'`) -- What happens if you open it for `'w'`? – mgilson Feb 10 '15 at 21:30
  • Thanks, it works with `'w'` passed to open. Which I should have caught, but I am confused by why it works without that additional flag on some other machines. I guess there is some magic somewhere which detects in some shell environments that you're trying to write to /dev/null, and doesn't, but for whatever reason in this case it actually wrote to /dev/null. – aestrivex Feb 10 '15 at 21:33
  • When dealing with pseudo-files, who knows what checks are actually implemented by the OS? I'll post as an answer. – mgilson Feb 10 '15 at 21:34
  • Heh, this means, my answer was technically correct, too, yet it missed the precise point. – ivan_pozdeev Feb 10 '15 at 21:39
  • related: [How to hide output of subprocess in Python 2.7](http://stackoverflow.com/q/11269575/4279) – jfs Feb 11 '15 at 02:18

1 Answers1

3

You are opening the "file" (/dev/null) in the default mode (which is 'r' [read]). However, subprocess is trying to write to the file -- So you probably want:

with open(os.devnull, 'w') as nil:
  p = subprocess.call(['which', 'program'], stdout=nil, stderr=nil)
ivan_pozdeev
  • 33,874
  • 19
  • 107
  • 152
mgilson
  • 300,191
  • 65
  • 633
  • 696