0

Is there any way to insert json data without metadata lines on ES or Kibana? or automatically creating index lines?

My data looks like this.

  {"time_slot":"2013-12-31T15:00:00.000Z","line_num":"1호선","line_num_en":"Line 1","station_name":"서울역","station_name_kr":"서울역","station_name_en":"Seoul Station","station_name_chc":"-","station_name_ch":"首?站","station_name_jp":"ソウルヨク","station_geo":{"lat":37.554648,"lon":126.972559},"people_in":91,"people_out":172}

I know Using bulk API, json data should be with metadata index lines

  {"index": {"_index" : "seoul-metro_2014-2","_type":"kr-subway","_id":"0"}}
  {"time_slot":"2013-12-31T15:00:00.000Z","line_num":"1호선","line_num_en":"Line 1","station_name":"서울역","station_name_kr":"서울역","station_name_en":"Seoul Station","station_name_chc":"-","station_name_ch":"首?站","station_name_jp":"ソウルヨク","station_geo":{"lat":37.554648,"lon":126.972559},"people_in":91,"people_out":172}

But real data doesn't have meta data lines,

  {"index": {"_index" : "seoul-metro_2014-2","_type":"kr-subway","_id":"0"}}

so I just wondering if I wanna use bulk API on ES,

Should I put every metadata lines to raw json data? But if data's huge, How can I insert to ES?

Yeonju Lee
  • 1
  • Are you going to insert all documents into the same index ? – Val Mar 27 '18 at 06:06
  • This answer should help: https://stackoverflow.com/questions/33980591/insert-multiple-documents-in-elasticsearch/33981143#33981143 though instead of parsing the file as JSON, you'll need to read it line by line, but the change is trivial (i.e. not call `json.loads` but iterate on the file line by line) – Val Mar 27 '18 at 06:07

0 Answers0