Apache Avro Maven Plugin generate UUID field instead of String

Question

Given the following Avro Schema:

{
    "namespace": "ro.dspr.coreentities",
    "type": "record",
    "name": "Organization",
    "fields": [
      {
        "name": "id",
        "type": "string",
        "logicalType": "uuid"
      },
      {
        "name": "name",
        "type": "string"
      },
      {
        "name": "description",
        "type": "string"
      }
    ]
}

Running avro-maven-plugin 1.9.0 goal schema, I get:

@org.apache.avro.specific.AvroGenerated
public class Organization extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {

  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"Organization\",\"namespace\":\"ro.dspr.coreentities\",\"fields\":[{\"name\":\"id\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"logicalType\":\"uuid\"},{\"name\":\"name\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"}},{\"name\":\"description\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"}}]}");

  // truncated 

  @Deprecated public java.lang.String id;
  @Deprecated public java.lang.String name;
  @Deprecated public java.lang.String description;

  // truncated
}

I want the generated POJO for Organization to have id UUID, not String (what I have now).

Links I looked at:

I do see the logical type def from Avro and there is the Conversion class I am actually trying to trigger, but I cannot connect the dots.

Other

Relevant Maven pom parts

<plugin>
  <groupId>org.apache.avro</groupId>
  <artifactId>avro-maven-plugin</artifactId>
  <version>${avro.version}</version>
  <configuration>
    <sourceDirectory>${avro-files-path}</sourceDirectory>
    <outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
    <stringType>String</stringType>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>schema</goal>
      </goals>
      <configuration>
        <sourceDirectory>${avro-files-path}</sourceDirectory>
        <outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

Extra info

I actually trying to use Avro to give my Kafka messages a structure. I am also using Confluent Schema Registry and Confluent Avro Kafka Serializer. Nevertheless, I thought I would only have the id as String, but if I try to send messages to Kafka as something non-UUID, it will fail later. However, I found out that there is actually no constraint, and I managed to send any String to Kafka. So, the "logicalType" in Avro is not enforced at all.

The Question

How can I generate Organization.class#id as UUID?
If there is no Avro-support in doing this, what would be the workaround (preferably reusing the org.apache.avro.Conversions.UUIDConversion) ?

Have you tried "logicalType": "uuid", from the docs it looks like it needs to be in lower case. — mwarren, Jun 20 '19 at 12:28
This was actually the first thing I've tried, but with no success. In the schema of the POJO, logicalType uuid is there (and it correctly recognised, because you cannot put anything there, otherwise it will fail) — Cristian Ciobotea, Jun 20 '19 at 12:31
Hi. No, but the truth be told, I haven't tried it since then. — Cristian Ciobotea, Dec 05 '19 at 21:02

score 4 · Answer 1 · answered Jan 21 '20 at 18:24

Here is a functional example:

Schema:

{
  "name": "pk",
  "type": {"type": "string", "logicalType": "uuid"}
},

Maven config:

            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>1.9.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/../avro/schemas/commands</sourceDirectory>
                            <outputDirectory>${project.build.directory}/generated-sources/java</outputDirectory>
                            <!-- TODO Enable this string type once we have 1.10.0 release, or 1.9.2-->
                            <!--<stringType>String</stringType>-->
                            <fieldVisibility>PRIVATE</fieldVisibility>
                            <customConversions>org.apache.avro.Conversions$UUIDConversion</customConversions>
                            <imports>
                                <import>${project.basedir}/../avro/schemas/common</import>
                                <import>${project.basedir}/../avro/schemas/commands/account</import>
                                <import>${project.basedir}/../avro/schemas/commands/rec</import>
                            </imports>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

The important thing to note is that there is a conflict between the UUIDConversion and String in the current release of the plugin, as such you either need to choose which is more important, or use 1.10.0-SNAPSHOT which has a fix merged, but not yet released.

Even with no stringType and the customConversions, the generated java class still expects `CharSequence` as argument to the setter on my end. — Timo Meijer, Apr 03 '20 at 15:04
Thanks a lot ! I've had a headache all day long trying to workaround the uuid issues — singe3, Jun 04 '20 at 15:56
@singe3 were you able to resolve, for me it still expects ``` private java.lang.CharSequence id;```. i am using 1.10.2 version of maven plugin. what is the type expected and how are you passing uuid? — best wishes, Jul 28 '21 at 11:43
@bestwishes I'm currently using avro-maven-pkugin version 1.10.2 but with a different schema. I don't use type "string" because it wastes space using UTF-8 encoding (36 bytes per string uuid). I use the "bytes" type (16 bytes per uuid) : {"name":"uuid","type":{"type":"bytes","logicalType":"uuid","CustomEncoding":"UUIDAsBytesEncoding"}} Finally for decoding I have a simple implementation of CustomEncoding (UUIDAsBytesEncoding) that translates from/to byte[] <> UUID and a simple implementation of Conversion to generate the proper UUID field in the generated class. Works for me. — singe3, Jul 28 '21 at 11:53
@bestwishes If you need more information for your specific case I suggest you write a question with all the details and I'll be happy to have a look at it. Comments here are to restricted to bring you a proper detailed answer — singe3, Jul 28 '21 at 11:58
@singe3 asked here https://stackoverflow.com/q/68569418/3892213. with M21BB's answer uuid get stored as string which is way less optimal than storing bytes — best wishes, Jul 29 '21 at 02:46

Apache Avro Maven Plugin generate UUID field instead of String

Links I looked at:

Other

Extra info

The Question

1 Answers1