2

I want to run google closure compiler on many javascript files from python.
I use this:

subprocess.Popen('/usr/bin/java -jar /var/www/compiler.jar --js "%s" --js_output_file "%s"' % (fullname, dest_filename),shell=True);

But as I understand it creates java VM for each process.
So this eats all the RAM. I only can hear my hdd cracking as swap is being used. And system is almost hanged. Can I tell java somehow to use only 1 vm for all started processes?

May be I wrote something incorrectly. That is because I totally don't know java. Sorry for that

Sergey
  • 19,487
  • 13
  • 44
  • 68
  • Aside: this is precisely the sort of situation where it makes sense *not* to use `shell=True`, because then you don't need to worry about quoting filenames. – Thomas K May 25 '12 at 17:45
  • @ThomasK, thanks, just not to write the list of args in popen I used 1 string and Shell param – Sergey May 25 '12 at 18:31

1 Answers1

1

Possible answer 1:

The Google closure compiler does accept multiple input files with a syntax like this:

java -jar compiler.jar --js=in1.js --js=in2.js ... --js_output_file=out.js

This will produce only one output file that is the combination of all of the inputs. But this might not be what you want if you're trying to compile each file separately.


Possible answer 2:

It would not be hard to write a small wrapper script (using bash, python, or your favorite scripting language) that accepts pairs of parameters, e.g.

wrapper.sh in1.js out1.js in2.js out2.js ...

The code in wrapper.sh could loop over the (pairs of) parameters and call java -jar --js=xxx --js_output_file=yyy repeatedly, waiting for each to complete before beginning the next. This would have the benefit of not starting each process in parallel, so at least you wouldn't have (potentially) many JVMs running at the same time. Though you do have some inefficiency in having to restart the JVM for each run.


Possible answer 3:

If you really want just one JVM, then there is no way to do what you ask without writing a little bit of Java code (as far as I know). If you are familiar with Java, you could copy the source code of CommandLineRunner.java and modify it to suit your needs.

Or perhaps even easier, just write a small Java class whose main function simply invokes the CommandLineRunner main any number of times, passing in appropriate parameters to simulate a normal command line invocation. Here's something quick and dirty that would do the trick (hat tip to VonC)

import com.google.javascript.jscomp.CommandLineRunner;
import java.security.Permission;
public class MyRunner {
    public static void main(String [] args) {
        // Necessary since the closure compiler calls System.exit(...).
        System.setSecurityManager(new NoExitSecurityManager());
        for (int i=0; i<args.length; i+=2) {
            System.out.println("Compiling " + args[i] + " into " + args[i+1] + "...");
            try {
                CommandLineRunner.main(new String[] {
                    "--js=" + args[i],
                    "--js_output_file=" + args[i+1]
                });
            }
            catch (ExitException ee) {
                System.out.println("Finished with status: " + ee.getStatus());
            }
        }
    }

    private static class ExitException extends SecurityException {
        private int status;
        public ExitException(int status) { this.status = status; }
        public int getStatus() { return status; }
    }
    private static class NoExitSecurityManager extends SecurityManager {
        public void checkPermission(Permission p) { }
        public void checkPermission(Permission p, Object context) { }
        public void checkExit(int status) { throw new ExitException(status); }
    }
}

Compile it with with something like this:

javac -classpath compiler.jar MyRunner.java

Run it with something like this:

java -classpath .:compiler.jar MyRunner in1.js out1.js in2.js out2.js ...

And see output like this:

Compiling in1.js into out1.js...
Finished with status: 0
Compiling in2.js into out2.js...
Finished with status: 0
Community
  • 1
  • 1
csd
  • 1,724
  • 10
  • 18
  • Thanks, I think I'll try only 1,2 answers. Please say why has Google written this app in java?? It definitely should be written in c++ as it is a "compiler". It must be fast. – Sergey May 25 '12 at 18:56