11

I would like to know what the proper avro schema would be for some json to avro conversion that is in this format:

{"entryDate": "2018-01-26T12:00:40.930"}

My schema:

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : "long",
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

I keep getting 

`'Cannot convert field entryDate: Cannot resolve union: 
"2018-01-26T12:00:40.930" 
not in 
["null",{"type":"long","logicalType":"timestamp-millis"}]'`
fredrik
  • 13,282
  • 4
  • 35
  • 52
koala421
  • 786
  • 3
  • 11
  • 27

1 Answers1

12

It was a silly mistake...obviously I was storing the timestamp value as a string so the avro schema needed a string instead of long for type.

ie.

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"long"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

should be

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"string"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}

doh!

fredrik
  • 13,282
  • 4
  • 35
  • 52
koala421
  • 786
  • 3
  • 11
  • 27
  • Simple but makes totally sense. My most common use cases: When I am serializing from postgres db I have to change the type to "long". When I am serializing from json file I have to change the type to "string" – alexopoulos7 Apr 04 '18 at 13:13
  • 3
    It seems that logical types are restricted in the sense that they can only annotate certain basic types. The logical types for time and date should only annotate numeric types, `long` in the case of timestamp-micros (see [docs](https://avro.apache.org/docs/1.8.2/spec.html#Timestamp+%28microsecond+precision%29)). Did you run into problems using `string`? – malana Sep 11 '18 at 21:30
  • 3
    Same, anyone figure out String -> time/timestamp millis/micros? – Zack Bartel Mar 20 '19 at 20:52