2

I have downloaded and compiled jsmin.c, and run it on a javascript file from the terminal, and it seems to work great. But when I call it from an os.system() or os.popen() call in a python script (on the same input file), the output file (the minified version of the js file) gets truncated, as though stdout were not getting flushed before the subprocess exits, or as if jsmin were terminating early, or as if maybe something were going on with disk buffering or something.

But none of those things seem to be the case. The exit value is whatever I return from main() even when the output is getting truncated, and adding a call to fflush(stdout) doesn't make any difference, and calling sync from within the same subshell after calling jsmin doesn't make any difference.

I tried replacing calls to putc() with calls to fputc(), and at first it seemed like that fixed the problem (for some unfathomable reason?), but then, inexplicably, the problem started happening again, and now happens reliably. Bizare?

I would say it's some problem with jsmin.c, but the program works fine on the same input file when run from the command line, so it has something to do with running it from a python subprocess.

Here's my subprocess call:

result = os.system('jsmin < ' + tpathname + ' > ' + tpathname + '.min')

(I have put jsmin in the path, and it is running, I get most of the expected results in the .min file.)

Can anyone imagine what might be causing this problem?

Shavais
  • 2,476
  • 1
  • 27
  • 25
  • Are you saying that the `jsmin jsfile.min` command works fine when run from directly from the shell? That's weird, because `os.system()` forks a shell so the execution environment for _jsmin_ should be almost identical in both cases. – C2H5OH Apr 19 '12 at 21:08
  • Yes, that's what I'm saying. I even tried creating a shell script that runs jsmin that begins with a shebang thingy. That shell script works from the terminal, but doesn't from python. Wtf? – Shavais Apr 19 '12 at 21:56
  • I tried writing the env to a file from the shell script to see if I could detect any env differences between running it from a python subprocess and running it from the shell. The only differences were SHLVL=4 instead of 3, and the PWD is different. That's it, everything else is exactly the same. – Shavais Apr 19 '12 at 22:08
  • I also tried running the python program using python -u, just in case there was some python buffering issue involved, but that didn't have any affect. – Shavais Apr 19 '12 at 22:27
  • 1
    Try changing `jsmin` with `cat` in your `os.system()` call, if the output file is identical to the input file then the culprit is your `jsmin` binary. Otherwise there is something very paranormal with your Python installation. – C2H5OH Apr 19 '12 at 22:34
  • In fact, I just replaced jsmin with a simple program that simply copies stdin to stdout using while (EOF != (c = getchar())) putchar(c); followed by fflush(stdout), and it's _still_ getting truncated! Somehow fflush(stdout) is not working in the python subprocess environment, or at least it looks that way. How in the world that is happening, I can't imagine. – Shavais Apr 19 '12 at 22:51
  • Oh sorry, I didn't see your suggestion. Ok I just tried that; yes, cat's output is truncated too. I'm using the python that comes with CentOS 6.2. I guess it says it's Python 2.6.6. – Shavais Apr 19 '12 at 22:58
  • Hmm, it's not happening from the python interactive shell, and it's not happening in a separate really simple python program, so it must have something to do with my python program, somehow.. – Shavais Apr 19 '12 at 23:08
  • Has this been solved? If so, please post an answer. – BoltClock Apr 22 '12 at 19:35

1 Answers1

0

Stack Overflow wouldn't let me answer my own question for 5 hours or something like that, so this 'answer' was originally added as an edit. (It also wouldn't let me chat, so the comments stretched on for a bit.)

I found the problem. The problem was with the python program that was creating the input file for jsmin and then calling jsmin on it. It was creating a file, failing to close it (yet), and then calling jsmin on it. So jsmin was not terminating early, nor was it having it's output truncated; rather, it was operating on an (as yet) incomplete input file. (Duh.)

I would have realized this a lot earlier than I did, except that the python program was eventually closing jsmin's input file (by exiting), so by the time I was examining it, it appeared complete. It's just that it was not complete by the time jsmin was processing it.

This very thing is one of the motivations for the 'with' idiom:

with open(targetpath, 'w+') as targetfile:
   ..code that writes to targetfile
Shavais
  • 2,476
  • 1
  • 27
  • 25