I have a program that loops on a set of products, and for each product, an XML is requested from an external application.
I have tried to use Apache HTTP client and the Java-11 HTTP client. Both will throw after a certain/fixed number of requests an IOException. This IOException repeats after that a number of times. And after that, the requests suddenly stop throwing IOExceptions anymore.
The weird part is that both pieces of code start throwing IOExceptions after exactly 104 times.
For the Apache way, the error is I/O exception (java.net.SocketException) caught when processing request to {}->http://somesite:80: Connection reset
For the Java11 way, it is HTTP/1.1 header parser received no bytes / connection reset by peer.
If I manually try to download the XML with an Id that is causing the problem, then the response is OK.
So, for me, I think that the external party closes the connection or something, but how to prevent this for happening? I tried to use connecttimeout/connectionrequesttimeouts/sockettimeout but that didn't solve the problem.
Who has any ideas to solve this problem or can explain what is happening here?
Code for the Apache variant looks like this, the methods get called for each product/Id :
public String ApacheHTTP_getIceCatSpecifications(Integer Id)
{
String list = new ArrayList<String>();
CredentialsProvider provider = new BasicCredentialsProvider();
provider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(env.getProperty("user"), env.getProperty("password")));
HttpGet httpGet = new HttpGet("http://somesite/" + Id + ".xml");
CloseableHttpClient client = HttpClientBuilder.create().setDefaultCredentialsProvider(provider).build();
ICECATInterface product = null;
try (CloseableHttpResponse response = client.execute(httpGet))
{
final HttpEntity entity = response.getEntity();
if (entity != null)
{
InputStream inputStream = entity.getContent()
//list = get from inputstream
}
} catch (ClientProtocolException e)
{
} catch (IOException e)
{
log.error("IOException {}");
}
return list;
}
And for the Java 11 variant:
public List<String> getSpecifications_JAVA11(Integer Id)
{
List<String> specificationList = new ArrayList<String>();
HttpClient client = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.authenticator(new Authenticator()
{
@Override
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication(
env.getProperty("user"),
env.getProperty("password").toCharArray());
}
})
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("http://somesite/" + Id + ".xml"))
.build();
HttpResponse<InputStream> response = null;
try
{
response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());
// specificationList = read from the response...
} catch (IOException e)
{
e.printStackTrace();
} catch (InterruptedException e)
{
e.printStackTrace();
}
return specificationList;
}
