Java executing curl command not working on some commands

Question

I am using Spring with Java 11.

I have a test curl command that I can call successfully via java.

final String cmdGetDocId = "curl -X POST https://postman-echo.com/post --data foo1=bar1&foo2=bar2";
Process process = Runtime.getRuntime().exec(cmdGetDocId);
InputStream inputStream = process.getInputStream();
JSONObject json = convertToJSON(inputStream);

The returned JSON is as expected.

If I use a different curl and execute it on the command line, it returns some JSON successfully as expected.

curl --location --request GET 'http://xxx.xx.xx.xx:8080/document/details/' --header 'Authorization: Basic xxxxx'

However, if I try to call the curl from my Java application, it fails.

String cmdGetDocId = "curl --location --request GET 'http://xxx.xx.xx.xx:8080/document/details/' --header 'Authorization: Basic xxxxx'";
Process process = Runtime.getRuntime().exec(cmdGetDocId);
InputStream inputStream = process.getInputStream();
JSONObject json = convertToJSON(inputStream);

The returned inputStream is empty.

Do you know what I am doing wrong? Why can the java method call the test curl but not the other GET curl?

try adding a ``-v`` to the curl command so you get more verbose output and see what comes back. There can be subtle differences between launching a command from the console/terminal and from within a program. — JohnXF, Dec 06 '21 at 11:19
Is it possible that you are hitting an internal redirect, because of http to https traffic? — Royal Bg, Dec 06 '21 at 11:20
@JohnXF, I tied adding the `-v`, but I don't have any difference, i.e. the inputStream is and empty String when calling `new String(inputStream.readAllBytes(), StandardCharsets.UTF_8)`. — Richard, Dec 06 '21 at 11:22
So maybe it gives an error - check the execution of the process - what's its exit code? — Royal Bg, Dec 06 '21 at 11:25
@RoyalBg, thanks for the reply. I am trying to call the 2nd curl with 'https' too, and it also returns a blank inputStream. Where from the command line using https, it returns an error `curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number` which i would expect to see in the java app too. — Richard, Dec 06 '21 at 11:27
There are three streams from a running command - input, output and error. It is likely your JSON will appear on the output stream and if there are errors they would be on the error stream. So check the result code from the command and then inspect the appropriate stream for more information. — JohnXF, Dec 06 '21 at 11:28
@RoyalBg, the exit code when it gets an error is `6` (the curl that gets no error has an exit code of `0`). — Richard, Dec 06 '21 at 11:30
You would be better using a HTTP library to connect directly to the URL and receive the HTTP response code and output rather than executing an external command and trying to capture the input from there. Use something like ``HttpURLConnection`` — JohnXF, Dec 06 '21 at 11:30
If there's exit code >0 then you can try to get the error message from the error stream — Royal Bg, Dec 06 '21 at 11:55

score 1 · Accepted Answer · answered Dec 06 '21 at 11:37

I made a minimal example to explain you a few things:

public class Playground {
    public static void main(String... args) throws IOException, InterruptedException {
        String cmdGetDocId = "curl -XGET 'https://google.com'";
        Process process = Runtime.getRuntime().exec(cmdGetDocId);
        InputStream inputStream = process.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
        System.out.println("output: ");
        Thread.sleep(2000);
        while(process.isAlive()) Thread.sleep(100);
        System.out.println("return value: " + process.exitValue());
        reader.lines().forEach(System.out::println);
        reader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
        reader.lines().forEach(System.err::println);
        System.out.println("---");
    }
}

On the one hand, you must make sure, the command is actually finished, when you try to output. In order to make sure, I have this dirty while-loop. Also, you want to have a look at the Error-Output, too.

Also you actually don't want to use cURL inside of your Java Program. There is a ton of beautiful Libraries or even the bare HttpURLConnection.

Thanks, I think I will try use the `HttpURLConnection` as both you and @JohnXF above suggests. — Richard, Dec 06 '21 at 11:42
There are good blog entries and tutorials for that. For example: https://www.journaldev.com/7148/java-httpurlconnection-example-java-http-request-get-post Good luck! — edean, Dec 06 '21 at 11:45
I have tried using `HttpURLConnection`, but it is not perfect, I must be doing something wrong. See: https://stackoverflow.com/questions/70245571/java-executing-curl-command-using-httpurlconnection-returns-a-http-no-content-co — Richard, Dec 06 '21 at 12:37

score 0 · Answer 2 · answered Dec 06 '21 at 12:06

Try this,

String command = "curl -X POST https://postman-echo.com/post --data foo1=bar1&foo2=bar2";
ProcessBuilder builder = new ProcessBuilder(command);
builder.redirectErrorStream(true);
Process process = builder.start();

InputStream response = process.getInputStream();

This will contain the error if any, Otherwise the reponse.

Java executing curl command not working on some commands

2 Answers2

Linked