3

Is it normal that any Amazon AWS Lambda Java function takes an unreasonable long time to finish when an uncaught exception is thrown? Please note this is a general question regarding Amazon Lambdas in Java, since I am testing this in a very generic way, with a very simple bare bones function.

For example, consider the function below, that validates a PIN. If the PIN is valid, it returns the text: PIN is OK: "A". Otherwise, it throws an IOException:

public class Hello implements RequestStreamHandler {    
    private static final int BUFFER_SIZE = 65_536;    
    private static final int MAX_SIZE = 262_144;    
    private static final String CHARSET_UTF8 = "UTF-8";    
    private static final byte[] buffer = new byte[BUFFER_SIZE];    
    private static final ByteArrayOutputStream baos = new ByteArrayOutputStream();

    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {

        String input = readInputStreamToString(inputStream);

        // PIN is valid.
        if (input.equals("\"A\"")) 
            writeStringToOutputStream(outputStream, "PIN is OK: " + input);

        // PIN is not valid.
        else 
            throw new IOException("PIN is wrong: " + input);
        }

    private String readInputStreamToString(InputStream inputStream) throws IOException {
        baos.reset();
        int length, total = 0;
        while ((length = inputStream.read(buffer)) != -1) {
            total += length;
            if (total > MAX_SIZE) throw new IllegalStateException("InputStream bigger than " + MAX_SIZE + ".");
            baos.write(buffer, 0, length);
            }
        return baos.toString(CHARSET_UTF8);
        }

    private void writeStringToOutputStream(OutputStream outputStream, String info) throws IOException {
        byte[] chars = info.getBytes(CHARSET_UTF8);
        outputStream.write(chars, 0, chars.length);
        }
    }

To test the above code:

  • For a valid PIN, use "A" as the test data.

  • For an invalid PIN, use any other input, for example: "B".

Memory size is 128 MB, and max memory used is 48 MB. When the PIN is valid, the function is very fast, and exits in less than 1 ms. However, when the PIN was not valid, the function was timing out in 3 secs, and I was getting this:

{
  "errorMessage": "2017-10-15T21:35:58.744Z *** Task timed out after 3.00 seconds",
  "errorType": "java.lang.RuntimeException"
}

I then increased the timeout to 10 secs, and now it actually finishes around 7.5 secs and gives me a stacktrace:

{
  "errorMessage": "PIN is wrong: \"B\"",
  "errorType": "java.io.IOException",
  "stackTrace": [ "example.Hello.handleRequest(Hello.java:83)" ]
}

My questions:

1) Is it normal that exceptions in lambda functions should take that much time for Amazon to process? Why? If not, why am I having this problem?

2) What is the recommended way of dealing with exceptions? Should I never let a function finish with an exception?

Marcelo Glasberg
  • 29,013
  • 23
  • 109
  • 133
  • 3
    Without any code I'm just guessing but why are you throwing an unchecked exception? The [Lambda docs](http://docs.aws.amazon.com/lambda/latest/dg/java-exceptions.html) are not clear but I'd try with a normal exception. – stdunbar Oct 15 '17 at 23:54
  • You may need to edit the question and add the lambda function code as well so that it is easy to understand and answer the invalid scenario. – Madhukar Mohanraju Oct 16 '17 at 07:24
  • @stdunbar: An unchecked exception is a normal exception. But OK, if you want me to test it with a checked exception, now I tried with an IOException (the one in the method signature). The problem remains the same. – Marcelo Glasberg Oct 16 '17 at 18:43
  • @MadhukarMohanraju I didn't provide the code because, as I said, the only difference is throwing an exception, and the code is very simple. But OK, I have updated the question to provide the code. – Marcelo Glasberg Oct 16 '17 at 18:45

2 Answers2

1

1) Never mind the max memory used. In my experience, 128 MB is very little for Java functions in general, not just with exceptions due to JVM overhead. You should increase it to at least 4x. But also, keep in mind that exceptions are not free:

See the previous questions:

How slow are Java exceptions?

Which part of throwing an Exception is expensive?

Note that increasing resources may not necessarily mean more costs for you, especially if your function is CPU-bound. You need to run experiments.

2) You could return specific HTTP status codes based on what exceptions your application may throw. But this flexiblity will add a bit of boilerplate to your code. See:

http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html?shortFooter=true#api-gateway-proxy-integration-lambda-function-java

  • Exceptions are not free but they also don't cost 7 seconds. The problem is something internal that Amazon Lambda does when it gets an exception. I tested with more memory, and it seems I have found the problem. Thanks. – Marcelo Glasberg Oct 18 '17 at 20:34
1

It seems I found out the problem. Amazon Lambda seems to do some kind of internal "exception initialization", only the first time it gets an exception in the container (to be clear, I mean an exception which is not caught by the user's handler, and is allowed to bubble up to the internal Amazon Lambda code).

So suppose you have some code that rarely issues an exception, and it runs, say, in 1 second, with a timeout of 3 secs. If this code throws an uncaught exception (because of a bug or by design), Lambda will initialize its internal exception handling, which takes about 7 secs for the lowest memory configuration of 128MB. Since the timeout is 3 secs it won't have time to finish, and the initialization will not complete. The next time it throws an exception, the initialization will start again and timeout again.

If you raise the memory, the exception initialization will run faster, and may finish before the timeout. The other possibility is raising the timeout limit to more than the time it takes for the exception initialization to finish. Once Lambda is able to finish this exception initialization, it will not have to initialize again (in this particular container). Subsequent exceptions will be very fast.

The implication of all this is that you must never allow exceptions to bubble to Amazon (probably by wrapping handle code in a try/catch), or else the timeout should be enough for the exception initialization to finish (add 7 seconds more than usually necessary, in 128MB).

Marcelo Glasberg
  • 29,013
  • 23
  • 109
  • 133