MapperParsingException when re-indexing data

Question

I am trying to re-index data to correct date formats, however, I am encountering a MapperParsingException when parsing a date field in the format 'Thu Jan 01 02:00:00 SAST 1970'. I used the dateOptionalTime mapping, which I guess is wrong.

None of the built in formats on the Elasticsearch Date Format reference seems to fit the bill though. Is this something I must specifically customize or can a built in Date Format version be used?

EDIT 1: mappings

{
    "mappings": {
        "users": {
            "properties": {
                "creationdate": {
                    "type": "date",
                    "format": "dateOptionalTime"
                },
                "email": {
                    "type": "string"
                },
                "firstlogin": {
                    "type": "boolean"
                },
                "firstname": {
                    "type": "string"
                },
                "lastloggedin": {
                    "type": "date",
                    "format": "dateOptionalTime"
                },
                "lastname": {
                    "type": "string"
                },
                "lastprofileupdate": {
                    "type": "date",
                    "format": "dateOptionalTime"
                },
                "userid": {
                    "type": "string"
                },
                "username": {
                    "type": "string",
                    "fields": {
                        "raw": {
                            "type": "string",
                            "index": "not_analyzed"
                        }
                    },
                    "copy_to": [
                        "username.raw"
                    ]
                }
            }
        }
    }
}

Did you delete and re create indices after changing the mapping? Can you post your mapping? — Vamsi Krishna, Jul 07 '15 at 15:11
Yes, I deleted it prior to changing the mapping and re-creating the index. — emilio, Jul 07 '15 at 16:54
You should delete and recreate indices after you changed the mapping. — Vamsi Krishna, Jul 07 '15 at 16:58

score 1 · Answer 1 · edited May 23 '17 at 12:21

As you suspected you would need to use the custom date format.

Unfortunately though the above example uses timezone names i.e "z" which is not supported by JODA.

If it is possible for you to change the format of time zone to use zone id you could use custom format as follows :

Example Date: "Thu Jan 10 02:00:00 Africa/Johannesburg 1970"    
lastloggedin" : {
                "type": "date",
                "format" : "EEE MMM dd HH:mm:ss ZZZ y"              
            },

score 0 · Answer 2 · answered Jul 07 '15 at 17:06

Try this mapping instead

{
    "mappings": {
        "users": {
            "properties": {
                "creationdate": {
                    "type": "date",
                    "format": "E MMM d H:m:s z Y"
                },
                "email": {
                    "type": "string"
                },
                "firstlogin": {
                    "type": "boolean"
                },
                "firstname": {
                    "type": "string"
                },
                "lastloggedin": {
                    "type": "date",
                    "format": "E MMM d H:m:s z Y"
                },
                "lastname": {
                    "type": "string"
                },
                "lastprofileupdate": {
                    "type": "date",
                    "format": "E MMM d H:m:s z Y"
                },
                "userid": {
                    "type": "string"
                },
                "username": {
                    "type": "string",
                    "fields": {
                        "raw": {
                            "type": "string",
                            "index": "not_analyzed"
                        }
                    },
                    "copy_to": [
                        "username.raw"
                    ]
                }
            }
        }
    }
}

It's throwing an error as it can't handle the timezone (SAST..). It would probably be best to re-index from the source and ensure I import in a format that Joda can handle. — emilio, Jul 08 '15 at 08:34

MapperParsingException when re-indexing data

2 Answers2