0

I want to store a uuid using avro. below is the schema i am using


  {
    "namespace": "somethingImportant",
    "type": "record",
    "name": "GoodObject",
    "fields": [
      {
        "name": "pk",
        "type": {"type": "string", "logicalType": "uuid"}
      }
    ]
  }

i am using maven plugin 1.10.2 plugin with the below config

<plugin>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro-maven-plugin</artifactId>
    <version>1.10.2</version>
    <executions>
        <execution>
            <id>schemas</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>schema</goal>
                <goal>protocol</goal>
                <goal>idl-protocol</goal>
            </goals>
            <configuration>
                <stringType>String</stringType>
                <customConversions>org.apache.avro.Conversions$UUIDConversion</customConversions>
                <enableDecimalLogicalType>true</enableDecimalLogicalType>
                <sourceDirectory>${project.basedir}/src/main/resources/</sourceDirectory>
                <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

the generated class the id field is generated as UUID, and i can do a get and set of uuid.

But uuid is stored as a string(36 bytes) which defeats the purpose. I came across another solution to define seomthing like


{
  "namespace": "somethingImportant",
  "type": "record",
  "name": "GoodObject",
  "fields": [
    {
      "name": "pk",
      "type": {"type": "string", "logicalType": "uuid", "CustomEncoding":"UUIDAsBytesEncoding"}
    }
  ]
}

but this does not do anything on its own. This stackoverflow answer talks about implementation, but i am not sure how we can make it work with maven plugin. This blog also talks about similar thing but i could not generate @AvroEncode(using = InstantAsStringAvroEncoding.class).

Has someone used uuid with avro so that

  1. we can set a uuid
  2. it gets stored as byte array
  3. The Get method returns a uuid

I don't want to do the marshelling and unmarshelling by myself obviously because that is a custom code to maintain

best wishes
  • 5,789
  • 1
  • 34
  • 59
  • UUIDs **are** strings, so why do you want them as bytes? And `@AvroEncode` is for Avro reflection on user-defined classes, not generated ones from schemas, but what exactly was the error when you tried to create a `CustomEncoding`? – OneCricketeer Jul 30 '21 at 05:45
  • @AvroEncode is for Avro reflection on user-defined classes, not generated ones from schemas, >> thanks for this. I am not sure how to make custom encoding work with plugin generated java class – best wishes Aug 01 '21 at 13:18
  • @OneCricketeer uuid are string, bit when stored as string they take 36bytes, but storing as bytes they will take 16 bytes. hence i want to store as bytes – best wishes Aug 02 '21 at 05:16
  • I have very limited experience with the custom encodings, but as long as they are on the classpath of the maven plugin, and on the classpath during runtime serialization, they'll work – OneCricketeer Aug 02 '21 at 14:33

0 Answers0