1

I've got an application written in Java that needs to call some utilities that are binary executables. It runs on anything that runs Java. What it needs to call is not a fixed set, but can be chosen by the user.

When I first wrote this back in the late 1990s, there was much spilled blood before I figured out that calling the utilities directly was extraordinarily problematic. The shortest comment is that some programs require a "context" and some don't, so generalized solutions are harder.

I wanted it to run the same everywhere like the Java paradigm claims, and while not wanting to start any kind of war I should point out that it just doesn't. I sure wish the Java folks would bother to resolve the major platform differences for us, but they have chosen not to and say that's just platform specific, not their problem. (Hey, if I'm just ignorant about "modern developments" with Java since, oh, 1.4, please tell me!)

For the curious, briefly, I had code like this:

  if (ClientOS.indexOf("Windows") != -1)
  {
     if (ClientOS.equals("Windows 95"))
     {
        cmd = "command.com /C ";
     } else if (ClientOS.equals("Windows 98"))
     {
        cmd = "command.com /C ";
        //cmd = "cmd.exe /C ";
     } else if (ClientOS.equals("Windows NT"))
     {
        cmd = "cmd.exe /C ";
     } else if (ClientOS.equals("Windows 2000"))
     {
        cmd = "cmd.exe /C ";
     } else if (ClientOS.equals("Windows XP"))
     {
        cmd = "cmd.exe /C ";
     } else {
        cmd = "cmd.exe /C ";
     }
  } else {

So, for MANY years I've gotten by using Bash as a competent intermediary. I provided my app with a configurable "shell" variable which typically gets set on Linux to something like "/bin/bash". On Windows. to get some help, I've been using Cygwin, and a typical value there is "C:/cygwin/bin/bash -c", and I do away with the if-block seen above. Along the way I noticed that sometimes, on some versions, the -c gets in the way, and in other cases, it's required. I have no idea why; until recently, either adding the -c or taking it away was enough.

Recently, however, I upgraded a whole bunch of machines, both Windows (and cygwin) and Linux, and then started noticing that not everything was working any more. SOME Programs continue to run just fine, but others don't. After an embarrassing number of hours, I've come to realize I don't understand enough about how things are SUPPOSED to work.

At the moment, it's all broken on the versions I have installed now, and I've had two problems;

  1. It either can't find the executable at all, or;

  2. The executable runs and can't find any of its required arguments - that I did pass.

This MAY fall into two categories: What's up with Java calling programs anyway!? And how POSIX BASH is supposed to work. Because it's easy to do, I've down-graded Java a little with no improvements, so I'm more focused on the BASH side at the moment. To be clear, calling Bash from an established Bash context always works "as per specficication", however, calling from Java does not! What I really need to do is figure out how to get it to always work from Java.

Every built-in I've tried runs no matter, however, for non-built-ins, I've learned you need to provide the full path to the program. Beyond that, I THOUGHT I knew you sometimes had to quote the entire command. Some versions of BASH seem to take the double quotes fine, others want single quotes.

When I first noticed trouble, before doing anything about it, I got errors where it gave the executable's full path followed by a colon then space, repeated twice, then:

"cannot execute binary file"

When it runs the executable I'm now getting a complaint from the program (unique to each executable) that it can't find its' arguments. On some attempts to fix this with either adding or dropping the -c or changing the quoting, I sometimes get:

executable_name: -c: line 0: unexpected EOF while looking for matching `'' executable_name: -c: line 1: syntax error: unexpected end of file

BTW, I found this StackOverflow article that discusses things, but it doesn't help.

I can tell you that when I practice that kind of launch of the executable - that is, quoting the executable to be called in single quotes and leaving the arguments dangling behind, the executable DOES run, but does NOT get any arguments.

Any input appreciated.

UPDATE FOR THE JAVA HEROS:

Don't be silly like the first commenter who just had to presume I wasn't using the Java built in functions. OF COURSE I use the Java built-in Runtime library. The particular call looks like this:

Process p = Runtime.getRuntime().exec(cmd);

Before you get your panties all in a twist, you need to figure out that Java DOES NOT create a solid, reliable process context that is often expected by running programs in various platforms. I wouldn't be telling you Java has issues launching programs if it wasn't true.

Community
  • 1
  • 1
Richard T
  • 4,570
  • 5
  • 37
  • 49
  • why do you insist on not using Java's inbuilt Runtime functionality? which is available since JDK 1.0 – Uku Loskit Nov 20 '13 at 22:05
  • @UkuLoskit - What a silly comment! OF COURSE I AM USING THE JAVA BUILT IN RUN-TIME FUNCTIONALITY. The call looks like this: Process p = Runtime.getRuntime().exec(cmd); – Richard T Nov 20 '13 at 22:20
  • why do you need to specify the shell then? shells are useful if you have some input that needs to interpreted before passing it off to the system call. The correct way is to not use a shell, as it is unnecessary, and dependening on the situation, may be a security risk. Provide the full path to your executable and the arguments, no need for platform specific code. – Uku Loskit Nov 20 '13 at 22:29
  • @UkuLoskit - I explained that, but I guess you overlooked it. It's called - by operating system engineers - a "process context". It has to do with a whole host of details that may or may not be required by called programs, such as environment variables, paths, etc. Some programs "find their own way" and some don't. What I had working - and has now suffered some "bit rot" was a generalized solution that will run anything. I'm trying to fix it. – Richard T Nov 20 '13 at 23:41
  • @UkuLoskit - by the way, this statement of yours just isn't true: "Provide the full path to your executable and the arguments, no need for platform specific code." If it _were_ true, I would never have bothered to do anything else. – Richard T Nov 20 '13 at 23:45
  • this context that you refer to the can be passed along with program as well, "environmens and paths etc" are just environment variables variables. In fact this is how shells do it, they are nothing but regular programs as well. Refer to http://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html#exec(java.lang.String,%20java.lang.String[]) the second argument allows you to specify environment variables. – Uku Loskit Nov 20 '13 at 23:47
  • The last comment should point to the Runtime.exec which takes in a String array rather than a String. That version of the command should be able to avoid any tokenization issues that the plain String version could have. – Uku Loskit Nov 21 '13 at 00:14

1 Answers1

0

I know this isn't want you want to hear and that I'll probably write me some mocking comments as well, but Java's largely in the right on this one.

The problem you're seeing with "unexpected EOF" is because you run

Runtime.getRuntime().exec("bash -c " + commandToRun);

instead of 

Runtime.getRuntime().exec(new String[] { "bash", "-c", commandToRun });

To explain, there are two paradigms libraries use when executing external processes:

  • Insecure, unpredictable, deprecated system(3) style. This involves passing in a string that gets evaluating by the OS' command processor. This has a dozen pitfalls that open up for security exploits and incorrect behavior.

  • Secure, robust execve(2)/execlp(3) style. In this style, you pass in an executable and a series of logically separated flags. This maps well to OS process invocation, and is safe and solid by default.

Runtime.exec(String) actually falls inbetween these, and is a shortcut similar to Runtime.exec(command.split(" ")). It reduces the number of security pitfalls, but still leads to broken behavior when commands contain spaces, like what you're seeing.

Things that will break a seemingly reasonable pseudocode system("touch " + file) include:

Arbitrary code execution:

  1. file="$(rm -rf /)"
  2. file="foo; rm -rf /"
  3. file="`rm -rf /`"
  4. file="&rm -rf /"
  5. file="\nrm -rf /"

Broken behavior:

  1. file="foo bar"
  2. file="foo'bar"
  3. file="foo # bar"
  4. file="foo -bar"
  5. file="foo\\bar"

I could go on and on.

All in all, Java's Runtime.exec(String[]) actually does provide a "solid, reliable, process context", even if it's not what you believe you want.

If it executed a command like one of the OS shells would, it would be neither solid nor reliable.

I believe you're currently trying to execute a shell command because it seems like a simple, powerful and flexible way of doing it. This was the same mistake made by e.g. Python's "command" module (now deprecated in favor of "subprocess", which does it correctly).

If you really, truly need to repeat the mistakes of the past to learn, then you should use Runtime.exec(String[]), with the first two strings in the array being "bash" and "-c" as in the example at the start of this post, or "cmd.exe" and "/C" for windows.

If you want to do it right, let the user specify commands and arguments as logically separate strings. This generalizes the process, and still allows users to specify bash, -c, somecommand if they want to, but allows them to do it correctly as well.

that other guy
  • 116,971
  • 11
  • 170
  • 194