Store Date Format in elasticsearch

Question

I met a problem when I want to add one datetime string into Elasticsearch.

The document is below:

{"LastUpdate" : "2013/07/24 00:00:00"}

This document raised an error which is "NumberFormatException" [For input string: \"20130724 00:00:00\"]

I know that I can use the Date Format in Elasticsearch, but I don't know how to use even I read the document on the website.

{"LastUpdate": {
    "properties": {
        "type": "date", 
        "format": "yyyy-MM-dd"}
    }
}

and

{"LastUpdate": {
    "type": "date", 
    "format": "yyyy-MM-dd"
    }
}

are wrong.

How can I transfer the datetime string into date format in Elasticsearch?

How can I store the datetime string directly into Elasticsearch?

one can use double-pipe to support multiple format. e.g. "yyyMMdd||yyyy-MM-dd" — Mohan Kumar, Jan 08 '16 at 10:09

score 28 · Accepted Answer · edited Jun 06 '16 at 08:56

28

You are nearly there. Set your mapping like this:

{"LastUpdate": {
    "type" : "date",
    "format" : "yyyy/MM/dd HH:mm:ss"}
}

Read the docs on the date mapping and its options and the date format parameter (one of the options to the date mapping).

Good luck!

edited Jun 06 '16 at 08:56

backtrack

7,996
5
52
99

answered Jul 24 '13 at 11:34

ramseykhalaf

3,371
2
17
16

How "format" knows the datetime string is "2013/07/24 00:00:00" ? Or it will use the datetime now ? I change to "LastUpdate": {"type": "date", "format": "yyyy/MM/dd HH:mm:ss"}, it raised another error which is "ElasticSearchIllegalArgumentException[unknown property [type]]" – Jimmy Lin Jul 24 '13 at 12:54
That is the mapping I gave. The `"format": "yyyy/MM/dd HH:mm:ss"` tells elasticsearch to accept dates in that format. – ramseykhalaf Jul 24 '13 at 13:04
How can I push/let elasticsearch know the what data is ? – Jimmy Lin Jul 24 '13 at 13:13
1

Oh, I should define the data type before I index them, right ? – Jimmy Lin Jul 24 '13 at 13:21
Ah, you need to set up a mapping first. I made a [gist](https://gist.github.com/ramseykhalaf/6070512). – ramseykhalaf Jul 24 '13 at 13:21
Look at the elasticsearch docs [on put mapping api](http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping/) – ramseykhalaf Jul 24 '13 at 13:22
1

Think of mapping = schema in SQL world. Not really that accurate, but will help you understand what mapping means – ramseykhalaf Jul 24 '13 at 13:23
Done, one more question. Is id must be a number, I use sha1 of one string, and it's seems doesn't work. I found that your search is also brilliant. – Jimmy Lin Jul 24 '13 at 14:51
What's the different between {key : 1} and {"key" : 1} in elasticsearch ? You didn't use double quote, but still works. – Jimmy Lin Jul 24 '13 at 15:03
You should use double quotes, I was being lazy, and some places it works, but it is NOT the standard. `_id` is stored as a string. If you give a number as the `_id` it will be stored as a string representation of that number. – ramseykhalaf Jul 24 '13 at 15:11
Any reason why in my document the date appears as long even with the format as custom? – Maxrunner Nov 12 '14 at 19:03
1

I believe the date is parsed according to your format. It will then be stored as some kind of `datetime` internally, which is stored as a `Long` (in Java) representing the number of milliseconds since 1 Jan 1970. – ramseykhalaf Nov 13 '14 at 04:16
Oh ok thanks! Still having trouble with using the range on dates. – Maxrunner Nov 13 '14 at 10:31
Dead links in answer. Here's the updated urls. Core Types: http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html Mapping Date: http://www.elastic.co/guide/en/elasticsearch/hadoop/current/mapping.html#mapping-date – ocergynohtna Mar 23 '15 at 23:26
@ocergynohtna Thanks for the heads up. But actually the links you gave weren't exactly what I was linking to prior. Have updated answer. – ramseykhalaf Mar 27 '15 at 09:05

Store Date Format in elasticsearch

1 Answers1

Linked