Executing /usr/bin/env bash -c "command" in Java

Question

If I run /usr/bin/env bash -c "docker info" in the terminal, I get the appropriate output. I try to replicate this in Java as below

ProcessBuilder pb = new ProcessBuilder("/usr/bin/env", "bash", "-c", "\"docker info\"");
pb.redirectErrorStream(true);
pb.start();

This fails as bash: docker info: command not found. I figured that it is treating it as single command and you can work around this by getting rid of those escaped quotes and get it to work. But if you have a command that is not a simple one like docker info but something like this

/usr/bin/env bash -c "grep docker -m1 /proc/self/cgroup|echo \$(read s;s=\${s##*/};s=\${s#*docker-};s=\${s%.scope};echo \$s)"

where without the quotes the command will not run on the terminal nor in the process builder I have above (for the same reason), but with quotes it works in the terminal but not in the above process builder because it is look for a file or directory of that quoted name.

That said, what you can also consider doing is trying to make `System.out.println(your_java_string)` exactly match the command, where `your_java_string` is the fourth argument to the `ProcessBuilder` initializer. Just doing that `println` should give you a pretty good idea of what you're doing wrong. — Charles Duffy, Mar 18 '20 at 00:13
@CharlesDuffy I already understand what you are saying. Which is why I gave the second command. It requires to be run with quotes. The command I have given is something that works. — Sam Thomas, Mar 18 '20 at 00:16
@CharlesDuffy I think you are wrong there, since in quoted commands IF you want the bash to parse what is given in -c then you need to escape the $ else it will be parsed by the current shell before sent to bash — Sam Thomas, Mar 18 '20 at 00:17
Pls delete your previous comments if they're not longer relevant so we won't be redirected to chat. I'm just asking, in ProcessBuilder the first arg should be the command, so having "bash" as second doesn't set it as a argument to ".../env"? https://stackoverflow.com/questions/11198678/processbuilder-environment-variable-in-java — itwasntme, Mar 18 '20 at 00:22
I thought of the single quote approach, but i have some commands that have awk in them and used singe quotes with it and i cant get around with it — Sam Thomas, Mar 18 '20 at 00:22
You can switch between multiple quoting styles within a single shell word. If you have a specific case where you're having trouble using single-quotes in conjunction with awk, I'm happy to assist. — Charles Duffy, Mar 18 '20 at 00:23
(though that said, manually writing code quoted to be included in a `bash -c` string is often a bad idea; bash itself can generate strings with shell functions serialized in an eval-safe manner, and it's typically best to let that built-in functionality do its job). — Charles Duffy, Mar 18 '20 at 00:24
BTW, to explain the "generally quite buggy" assertion I made above -- see the discussion in [BashFAQ #1](http://mywiki.wooledge.org/BashFAQ/001) of correct usage of `read`, and in [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo) re: `echo $s` having unintended side effects. — Charles Duffy, Mar 18 '20 at 00:26
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/209810/discussion-between-charles-duffy-and-sam-thomas). — Charles Duffy, Mar 18 '20 at 00:28

Charles Duffy · Answer 1 · 2020-03-18T01:58:59.300

Answering The Underlying Question

First, speaking to the real question, of trying to build Java code that invokes the shell script referenced in This script won't work through docker exec:

ProcessBuilder pb = new ProcessBuilder("/bin/sh", "-c", "shellQuoteWordsDef='shellQuoteWords() { sq=\"'\"'\"'\"; dq='\"'\"'\"'\"'\"'; for arg; do printf \"'\"'\"'%s'\"'\"' \" \"$(printf '\"'\"'%s\\n'\"'\"' \"$arg\" | sed -e \"s@${sq}@${sq}${dq}${sq}${dq}${sq}@g\")\"; done; printf '\"'\"'\\n'\"'\"'; }'; shellQuoteNullSeparatedStream() { xargs -0 sh -c \"${shellQuoteWordsDef};\"' shellQuoteWords \"$@\"' _; }; getProcessData() { systick=$(getconf CLK_TCK); for c in /proc/*/cmdline; do d=${c%/*}; pid=${d##*/}; name=$(awk '/^Name:/ { print $2 }' <\"$d\"/status); uid=$(awk '/^Uid:/ { print $2 }' <\"$d\"/status); pwent=$(getent passwd \"$uid\"); user=${pwent%%:*}; cmdline=$(shellQuoteNullSeparatedStream <\"$c\"); starttime=$(awk -v systick=\"$systick\" '{print int($22 / systick)}' \"$d\"/stat); uptime=$(awk '{print int($1)}' /proc/uptime); elapsed=$((uptime-starttime)); echo \"$pid $user $elapsed $cmdline\"; done; }; getProcessData");

This Java string was generated by the Clojure runtime. Taking the getProcessDataDef variable definition from that answer, I then ran:

$ getProcessDataDef="$getProcessDataDef" lein repl
nREPL server started on port 54512 on host 127.0.0.1 - nrepl://127.0.0.1:54512
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.8.0
OpenJDK 64-Bit Server VM 11.0.1+13-LTS
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (System/getenv "getProcessDataDef")
"shellQuoteWordsDef='shellQuoteWords() { sq=\"'\"'\"'\"; dq='\"'\"'\"'\"'\"'; for arg; do printf \"'\"'\"'%s'\"'\"' \" \"$(printf '\"'\"'%s\\n'\"'\"' \"$arg\" | sed -e \"s@${sq}@${sq}${dq}${sq}${dq}${sq}@g\")\"; done; printf '\"'\"'\\n'\"'\"'; }'; shellQuoteNullSeparatedStream() { xargs -0 sh -c \"${shellQuoteWordsDef};\"' shellQuoteWords \"$@\"' _; }; getProcessData() { systick=$(getconf CLK_TCK); for c in /proc/*/cmdline; do d=${c%/*}; pid=${d##*/}; name=$(awk '/^Name:/ { print $2 }' <\"$d\"/status); uid=$(awk '/^Uid:/ { print $2 }' <\"$d\"/status); pwent=$(getent passwd \"$uid\"); user=${pwent%%:*}; cmdline=$(shellQuoteNullSeparatedStream <\"$c\"); starttime=$(awk -v systick=\"$systick\" '{print int($22 / systick)}' \"$d\"/stat); uptime=$(awk '{print int($1)}' /proc/uptime); elapsed=$((uptime-starttime)); echo \"$pid $user $elapsed $cmdline\"; done; }; getProcessData"

Debugging The Problem Yourself

To print your literal strings, one per line:

printf '%s\n' /usr/bin/env bash -c "grep docker -m1 /proc/self/cgroup|echo \$(read s;s=\${s##*/};s=\${s#*docker-};s=\${s%.scope};echo \$s)"

...emits as output:

/usr/bin/env
bash
-c
grep docker -m1 /proc/self/cgroup|echo $(read s;s=${s##*/};s=${s#*docker-};s=${s%.scope};echo $s)

Each of those lines needs to be converted to a single literal Java string, by adding leading and trailing double quotes, and then adding backslashes for any characters which need to be escaped. Thus:

ProcessBuilder pb = new ProcessBuilder(
  "/usr/bin/env",
  "bash",
  "-c",
  "grep docker -m1 /proc/self/cgroup|echo $(read s;s=${s##*/};s=${s#*docker-};s=${s%.scope};echo $s)"
)

However, that code is generally quite buggy. I would strongly suggest replacing your original bash with the following:

s=$(grep docker -m1 /proc/self/cgroup)
s=${s##*/}
s=${s#*docker-}
s=${s%.scope}
printf '%s\n' "$s"

...as in:

ProcessBuilder pb = new ProcessBuilder(
  "/usr/bin/env",
  "bash",
  "-c",
  "s=$(grep docker -m1 /proc/self/cgroup); s=${s##*/}; s=${s#*docker-}; s=${s%.scope}; printf '%s\n' \"$s\""
)

score 1 · Answer 2 · answered Mar 18 '20 at 00:42

Your ProcessBuilder failed to start a process because the double-quotes don’t belong there. The quotes are needed by the command line, to indicate that docker info is a single argument and not two arguments. But when executing a process directly, without a command line, quotes have no special meaning. The argument is already a single argument, simply by your passing it as a single string.

I’d like to suggest an alternative. You don’t need bash and you don’t need grep. You have Java. Java supports regular expressions just fine.

So, here’s the same functionality without bash or grep:

Optional<String> matchingLine;
try (Stream<String> lines =
    Files.newBufferedReader(Paths.get("/proc/self/cgroup"),
        Charset.defaultCharset()).lines()) {

    matchingLine = lines.filter(l -> l.contains("docker")).findFirst();
}

if (matchingLine.isPresent()) {
    String line = matchingLine.get();
    line = line.replaceFirst("^.*/", "");
    line = line.replaceFirst("^.*docker-", "");
    line = line.replaceFirst("\\.scope$", "");

    // Do things with 'line' here
}

score 0 · Accepted Answer · answered Mar 18 '20 at 06:11

Well after Charles Duffy extensively worked on this, whose findings are detailed here as well as linked post within that answer, I realized that I don't have to escape embedded commands/scripts/substitutions like I would regularly for the same command executed through a script or a terminal.

The reason is that when executing through process builder, there is no processing done on the command itself (which would be always done by sh/bash the terminal was running). I fell into this rabbit hole, because I was making sure the command worked in the terminal first before testing it on Java.

*nod* -- or, to put the answer more generally, the same value needs to be escaped different ways to represent it as a literal string in different languages. — Charles Duffy, Mar 18 '20 at 13:45

Executing /usr/bin/env bash -c "command" in Java

3 Answers3

Answering The Underlying Question

Debugging The Problem Yourself