Running Unix Command in Java

Question

I am running the following code, and it stops at waitfor() function. What could be the reason and how can I solve it?

String line;
Process albumProcess;
try {
  albumProcess = Runtime.getRuntime().exec(
    "iconv -f UTF-16 -t UTF-8 /home/gozenem/"+ xmlFileName +
    ".xml | grep albumID");
  albumProcess.waitFor();
  BufferedReader in = new BufferedReader(
      new InputStreamReader(albumProcess.getInputStream()));
  ArrayList<String> lineList = new ArrayList<String>(); 
  while ((line = in.readLine()) != null) {  
    lineList.add(line);
  }
  result[0] = lineList.size();
  albumProcess.destroy();

} catch (Exception e) {}

score 2 · Answer 1 · edited May 23 '17 at 09:58

2

The | grep ... is not consuming the output from the command as you expect because getRuntime().exec does not understand piping symbols. The process gets bogged down waiting for something to consume its output and its getting passed bogus command line arguments "|", "grep", and "albumId".

A shell will understand | but execv will not, so you need to use bash -c instead to get a shell to do the piping (see java shell for executing/coordinating processes? do the piping yourself (see Pipe between java processes on command shell not reliable working). Java 7 has a new ProcessBuilder class that makes it easy to set up pipes so you can use those if you're only running on a bleeding edge JVM.

Once you've got grep running, if there's a bunch of lines that match, it may still fill up the buffer, so you need something sitting on the buffer consuming the process's output stream. Moving

albumProcess.waitFor();

after the while loop should do it.

edited May 23 '17 at 09:58

Community

1
1

answered Aug 11 '11 at 15:56

Mike Samuel

118,113
30
216
245

How can i use bash -e ? I am new about this and it would be great if you help me. thank you. – prgrmmr Aug 11 '11 at 16:04
@prgrmmr, `bash -c "foo"` starts a new shell that runs the command `"foo"` then exits. `bash -c "foo | bar"` includes piping. So just quote your command properly and put `bash -c` in front. From the manpage: "`-c string` If the `-c` option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0." – Mike Samuel Aug 11 '11 at 16:11
Processes don't typically hang around waiting for their output to be consumed, do they? Won't it just write to the stdout stream and exit? The process must be hanging for some other reason. – jches Aug 11 '11 at 16:44
1

@chesles, Yes they do. Writes to stdout are blocking once the buffer fills up. The process's stdout is not linked to the parent's process's stdout unless the process was spawned in a way to inherit file descriptors. With Java7 that's easy but no so easy with other versions. When you create a process in Java, you are responsible for providing what it needs on its stdin and closing that when nothing else is available *and* draining its stdout or closing it. – Mike Samuel Aug 11 '11 at 16:56
@chesles, One easy way to test this is by doing `Runtime.getRuntime().exec("yes")` and then `waitFor` it. – Mike Samuel Aug 11 '11 at 16:57
1

@Mike using `exec("yes")` is slightly misleading, since `yes` won't exit until killed, whether blocking or not. I did observe the blocking behavior using `cat` on a file larger than 65,536 bytes though. I assume that size would vary depending on the size of the buffer on different machines. – jches Aug 11 '11 at 19:01
@chesles, probably. I think [`ulimit -p`](http://ss64.com/bash/ulimit.html) lets you vary it within a shell. – Mike Samuel Aug 11 '11 at 19:18

score 1 · Answer 2 · answered Aug 11 '11 at 15:58

I think you should try to read the output from the process before waiting on it. Otherwise, if the command outputs to much then the buffer may get filled.

Have a look at this article which explains how to read from the process: http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4

Running Unix Command in Java

2 Answers2