1

There is an app that needs to be migrated from the Java 8 runtime to Java 11 runtime in App Engine Standard environment. It was decided to take the simplest route and to migrate to Java 11 with bundled services, i.e. the deployment is still done using appengine-web.xml, and not app.yaml.

The problem is as follows: With Java 8, the App Engine instance was usually sitting at 75-80% of its available memory, and no memory errors ever occurred. Since migrating to Java 11 with bundled services, the instance is sitting at 90%+, and the error "Exceeded soft memory limit of ... ... after servicing X requests total" occurs very often even for very light requests.

Q1: What could be the causes of these memory issues only for the migrated version of the app?

Q2: Why is the migrated app not sending profiling information to Cloud Profiler?

My only assumption for the memory issues was that there are memory leaks. So I tried to profile the application following the guide from the documentation:

  1. Added the following to appengine-web.xml:
<env-variables>
    <env-var name="GAE_PROFILER_MODE" value="cpu,heap" />
</env-variables>
  1. Provided Cloud Profiler permissions to the service account.

However, the Cloud Profiler page still shows a welcome page with the message that there is no profiling agent configured.

Edit: Changing the instance class to a higher one is not an option.

reevesCFC
  • 13
  • 3

1 Answers1

0

For profiling in java 11/17 you have to do

<env-var name="JAVA_TOOL_OPTIONS" value="-agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true" />

I've also noticed higher memory usage after migrating and have been trying to figure out why. The profiler doesn't seem very helpful for finding memory leaks. A heap dump would be better but it doesn't seem possible in appengine yet.

The profiler does show total memory usage, and for my server its only about 100mb. This is a bit strange because the instance memory will increaes to over 400mb. Makes me think it could be the JVM itself using more memory.

I've tried tuning some parameters with moderate success. Currently using the serial garbage collector and have been experimenting with specifying the max heap size.

<env-var name="JAVA_OPTS" value="-XX:+UseSerialGC -Xms100M -Xmx100M" />

https://docs.oracle.com/en/java/javase/17/gctuning/available-collectors.html

paul
  • 795
  • 1
  • 7
  • 19