0

I wrote a service for one of my projects to create and start "Dataflow-Job-Templates" from "Google App Engine", combined with "Task Queues".

The Templates can only be started, if they were created successfully beforehand. I just realized, that sometimes, the following error is thrown. Is this a problem in gae or is there anything I can do to prevent this? I haven't seen this before and after some tries, the templates seem to run.

The stacktrace:

java.net.SocketException: Broken pipe (Write failed) at 
java.net.SocketOutputStream.socketWrite0(Native Method) at 
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) at 
java.net.SocketOutputStream.write(SocketOutputStream.java:155) at 
sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431) at 
sun.security.ssl.OutputRecord.write(OutputRecord.java:417) at 
sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:876) at 
sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:847) at 
sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:717) at 
sun.security.ssl.Handshaker.sendChangeCipherSpec(Handshaker.java:1124) at 
sun.security.ssl.ClientHandshaker.sendChangeCipherAndFinish(ClientHandshaker.java:1216) at 
sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:1128) at 
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:348) at 
sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) at 
sun.security.ssl.Handshaker.process_record(Handshaker.java:961) at 
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) at 
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) at 
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at 
sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316) at 
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291) at 
sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250) at 
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77) at 
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972) at 
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419) at 
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352) at 
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469) at 

--> Google doesn't like me

xy.myname.myproject.importhelp.dataflow.DataflowUtil.startTemplate(DataflowUtil.java:113) at 

--> start the template, which usually works (also most templates started while this error was thrown)

xy.myname.myproject.importhelp.webservices.Jobs.startStandardTemplate(Jobs.java:227) at 

---> No depending Jobs need to be checked

xy.myname.myproject.importhelp.webservices.Jobs.doPost(Jobs.java:123) at 

---> Here starts my code to receive template information

javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at 
javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at 
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:833) at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190) at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at 
org.eclipse.jetty.server.Server.handle(Server.java:564) at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317) at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) at 
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) at 
java.lang.Thread.run(Thread.java:748)
Malte
  • 589
  • 5
  • 24
  • Are you creating the templates with GAE or just launching them with GAE? I don't know if creating templates from GAE is supported, so you may need to create them outside of GAE and just launch the template from within GAE. – Ben Chambers Sep 26 '17 at 19:41
  • Yes I do create them in GAE and it usually works just fine. Like I said above, this error was occuring when starting the templates, after it was created in GAE. – Malte Sep 27 '17 at 08:09
  • It seems like you may need to retry these requests. Broken pipes are caused by the connection being reset. See for instance https://stackoverflow.com/questions/1181255/java-nio-what-does-ioexception-broken-pipe-mean. This is not likely to be caused by templates themselves, but some transient errors elsewhere. – Ben Chambers Sep 27 '17 at 17:06
  • Yeah my setup does not yet restart failed jobs automatically, as this might result in retrying jobs which could perhaps never succeed. Next steps will be to check the reason programatically why the job failed and retry those which could succeed when retrying. Thank you for you input Ben – Malte Sep 29 '17 at 09:54

0 Answers0