2

Is there a way to read bytesArray via the AvroInputStream.binary api and somehow specify the writer version ? 

val in = new ByteArrayInputStream(bytesFromFile)
val input = AvroInputStream.binary[newClass](in)// old ?
val result = input.iterator.toSeq

in other words is avro4s support schema evolution with binary records ?

NetanelRabinowitz
  • 1,534
  • 2
  • 14
  • 26

1 Answers1

1

Does avro4s support schema evolution with binary records?

No. Avro in general supports Schema Evolution when you provide it a schema, it can match against the name of the properties and their types. When there's no schema, it relays on the order of declarations both the serializing and deserializing side. If you modify your classes, you break compatibility.

Yuval Itzchakov
  • 146,575
  • 32
  • 257
  • 321
  • Avro DatumReader also have an api that gets the writer schema ```GenericData.createDatumReader(Schema writer, Schema reader)``` so theoretically you can always supply a schema.. I'm looking for a similar capability in avro4s.. a way to give the inputStream (datumReader) the writer schema – NetanelRabinowitz Mar 23 '17 at 09:39
  • @NetanelRabinowitz You're right, `DatumReader` does have that API, but `avro4s` abstracts over it since it provides the Schema via compile time macros. Unfortunately that "lower level" API isn't supplied. Also, if your avro blob doesn't have a Schema, it doesn't matter if you provide `DatumReader` with one, it'll have nothing to match against. – Yuval Itzchakov Mar 23 '17 at 09:40
  • I guess I should open a feature request than.. :) thanks ! – NetanelRabinowitz Mar 23 '17 at 09:42
  • @NetanelRabinowitz Or even better, a pull request :). You welcome. – Yuval Itzchakov Mar 23 '17 at 09:42
  • end up with a really simple [pull request](https://github.com/sksamuel/avro4s/pull/114), but I guess there is a more elegant way of enabling this capability. I would like to hear your thoughts.. see if there is a better way :) – NetanelRabinowitz Mar 23 '17 at 20:55
  • 1
    @NetanelRabinowitz I'm on the phone right now, will take a look at it tomorrow. – Yuval Itzchakov Mar 23 '17 at 20:58