Google App Engine Java 11 - unclear errors com.google.apphosting.api.ApiProxy$CallNotFoundException

Question

We are currently migrating our App Engine from Java 8 to Java 11 using the bundled services (https://cloud.google.com/blog/products/serverless/support-for-app-engine-services-in-second-generation-runtimes).

We made it as far as that the application seems to be running as expected - BE, FE and Datastore all seem to work in our staging GCP project.

The following errors are showing up in our logs every 10 seconds, however:

Failed to query GCE metadata service

and then

java.io.IOException: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:70)
at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:609)
at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:488)
at com.google.devtools.cdbg.debuglets.java.GceMetadataQuery.readResponse(Unknown Source)
at com.google.devtools.cdbg.debuglets.java.GceMetadataQuery.queryMetadataAttribute(Unknown Source)
at com.google.devtools.cdbg.debuglets.java.GceMetadataQuery.getProjectId(Unknown Source)
at com.google.devtools.cdbg.debuglets.java.GcpHubClient.registerDebuggee(Unknown Source)
Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
at com.google.apphosting.api.ApiProxy$CallNotFoundException.foreignThread(ApiProxy.java:844)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:117)
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:54)
... 6 more

Edit: The Cloud Debugger says "The debugger could not find a debug target for the application". The versions still relying on Java 8 work though, so what extra step is missing for the migration to set up the debugger correctly? As we're using the App Engine standard environment, that should be enabled by default (https://cloud.google.com/debugger/docs/setup/java#gae-standard).

Edit 2: Looks like this could be a Google Cloud Debugger bug: https://github.com/GoogleCloudPlatform/cloud-debug-java/issues/18

Please, try reviewing the Stackdriver debugger, I think the problem can be related to it. — jccampanero, Oct 08 '21 at 21:42
Hi @vpriesner, did you have a chance to review your Stackdriver logs? — Farid Shumbar, Oct 11 '21 at 09:43

score 3 · Accepted Answer · answered Oct 22 '21 at 12:09

As per the App Engine documentation regarding issuing HTTP requests, if you use URL Fetch, it will cause requests to cloud client libraries (and that includes the Google Cloud Debugger Client for Java) to fail.

As wer are using the bundled services, we are still using an appengine-web.xml file.

We had the following line in there:

<url-stream-handler>urlfetch</url-stream-handler>

Removing that line fixed the issue for us.

Google App Engine Java 11 - unclear errors com.google.apphosting.api.ApiProxy$CallNotFoundException

1 Answers1