Unable to bulk insert docs into elastic search index: "Illegal Argument Exception: bulk request must be terminated by newline"

Question

I want to insert the docs contained my my_docs.json into my elasticsearch index. My json file looks like this:

{"_index":"twitter","_type":"_doc","_id":"v8XSJ2cB_TizemYYi5mW","_score":1,"_source":{"user":"me","c":"2018-11-18T17:16:08.953Z","content":"this is cool stuff"}}

{"_index":"twitter","_type":"_doc","_id":"x8XbJ2cB_TizemYYxZmf","_score":1,"_source":{"user":"you","c":"2018-11-18T17:26:13.634Z","content":"this is some other cool stuff"}}

I tried: curl -XPUT 'http://localhost:9200/twitter/_bulk?pretty&refresh' -H "Content-Type: application/json" -d @my_docs.json

where twitter is my index.

I got an error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "The bulk request must be terminated by a newline [\n]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "The bulk request must be terminated by a newline [\n]"
  },
  "status" : 400
}

I added a '\n' to the end of the file, but the problem persisted and I got the same error.

How do I fix this?

Have you seen https://stackoverflow.com/questions/48579980/bulk-request-throws-error-in-elasticsearch-6-1-1 ? — Matthew Pope, Nov 25 '18 at 16:54
It seems like you might need to specify the content type as data/binary. — Matthew Pope, Nov 25 '18 at 16:55

score 0 · Answer 1 · answered Nov 26 '18 at 00:26

curl -H "Content-Type: application/json" -XPOST "localhost:9200/{index}/{type}/_bulk?pretty&refresh" --data-binary "@my_docs.json"

Note: The json file should have content in following format for the above to work:

{"index":{"_index":"my_index","_type":"_doc","_id":"1"}}
{"field1":"field 1 data 1","field2":11}
{"index":{"_index":"my_index","_type":"_doc","_id":"2"}}
{"field1":"field 1 data 2","field2":21}`

Each document above is represent by two lines. First line indicates where to index and what is the document id. The next line is actual data of the doc.

Unable to bulk insert docs into elastic search index: "Illegal Argument Exception: bulk request must be terminated by newline"

1 Answers1