9

While uploading a 50 mb size huge JSON string in ElasticSearch using this method - 

public static void postData(String json, String index, String type) {
    RestClient client = RestClient.builder(new HttpHost(testHostPreProd, 443, "https")).build();
    HttpEntity entity = new NStringEntity(json, ContentType.APPLICATION_JSON);

    Response response = client.performRequest("POST", "/" + index + "/" + type + "/_bulk",
            Collections.<String, String>emptyMap(), entity);

    client.close();
}

The error is 

Exception in thread "main" org.elasticsearch.client.ResponseException: POST https:someURL/indexHere/typeHere/_bulk: HTTP/1.1 413 Request Entity Too Large
{"Message":"Request size exceeded 10485760 bytes"}

Putting a counter in the code that works send 10k batches might work but I'm not sure how to do that.

Any suggestions on how to handle this issue?

  • What is your `http.max_content_length` set to? This indicates that the default is `100mb` so you should be able to index a `50mb` document: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html – hudsonb Sep 20 '18 at 17:08
  • 9
    If you are using hosted Elastic on AWS there is a network limit depending on instance size https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html – Most Wanted Jul 20 '20 at 19:59

0 Answers0