Stack Overflow
Stack Overflow

Questions tagged [avsc]

Pure JavaScript implementation of the Avro specification.

Pure JavaScript implementation of the Avro specification (https://github.com/mtth/avsc).

26 questions
8
votes
4 answers

How to refer one avro schema inside another one

I need to refer to this Student schema from inside another schema: { "type": "record", "namespace": "data.add", "name": "Student", "fields": [ { "name": "Name", "type": "string" }, { "name": "Age", "type":…
jcrshankar
  • 1,165
  • 8
  • 25
  • 45
5
votes
0 answers

How to compile avro schema from an avsc that contains references using avrogen (C#)?

I have these two avsc files, the first makes a reference to the second but when trying to compile using avrogen an error is returned and the schema is not generated. Error: Exception occurred. Undefined name: AnalogProperty at…
Victor Silva
  • 51
  • 1
5
votes
1 answer

vscode json formatter: do not wrap json objects

I'm working with avro .avsc schemas now and I like formatting in samples: { "namespace": "example.avro", "type": "record", "name": "User", "fields": [ {"name": "name", "type": "string"}, {"name": "favorite_number", "type": ["int",…
Artsiom the Brave
  • 192
  • 2
  • 10
3
votes
0 answers

hive external table on avro alias column not working

I tried creating hive table on avro using avsc spec file and need to renam some of the columns . used alias but seems its not working. the columns are returned as null when i query the table SPARK DATAFRAME TO SAVE DATA val…
Ajith Kannan
  • 812
  • 1
  • 8
  • 30
3
votes
1 answer

Parsing Multiple AVRO (avsc files) which refer each other using python (fastavro)

I have a AVRO schema which is currently in single avsc file like below. Now I want to move address record to a different common avsc file which should be referenced from many other avsc file. So Customer and address will be separate avsc files. How…
Codegator
  • 459
  • 7
  • 28
3
votes
2 answers

How to use avro data with old and new namespace

I am facing a problem where I have updated the namespace in my avsc schema file. Since we were using common processor created in Java to parse the XML to avro and were using the avsc file. We have separated the interfaces and created 2 different…
AJ007
  • 125
  • 2
  • 11
1
vote
1 answer

Replace a field of enum value in AVRO schema to string

I need a little help in removing an ENUM and replacing it with String in an AVRO schema. I have an avro schema file which has something like this among other entries: { "name": "anonymizedLanguage", "type": [ …
Vishwa Bhat
  • 77
  • 11
1
vote
1 answer

How to deserialize avro file in react

Can anyone help me to deserialize the avro file in react? I tried with avsc npm package but I am now stuck on error. const avro = require('avsc') ...... avro.createFileDecoder('./abc.avro').on('data', (record) => { console.log(record) }) ...... but…
Speeder
  • 120
  • 3
  • 12
1
vote
1 answer

Avro schema getting undefined type name when using Record type

so im trying to parse an object with this avro schema. object is like: myInfo: {size: 'XL'} But Its behaving like the record type doesn't actually exist and im getting a undefined type name: data.platform_data.test_service.result.record at…
Strobe_
  • 495
  • 1
  • 13
  • 34
1
vote
1 answer

Avro - Add doc/description for each field

We are using avro for our schema definition. Is it possible to add field description for each of the fields in avro. I agree that we can add 'doc' at record level. we wanted to add description at a field level.
SunilS
  • 2,030
  • 5
  • 34
  • 62
1
vote
1 answer

How to read and manipulate Avro schema (avsc format)

I want to use the Avro schema and read it in python. Cannot read the schema. It is throwing an error import avro.schema from avro.datafile import DataFileReader, DataFileWriter from avro.io import DatumReader, DatumWriter schema =…
user9969893
1
vote
0 answers

Conversion from an .avsc file to a Java file using Maven

I created a .avsc file and I declared the following field: { "name": "event_time", "type": { "type" : "long", "logicalType": "timestamp-millis"}, "doc": "The timestamp when the event was registered." } When the package instruction from maven is…
dorinmoldovan
  • 89
  • 2
  • 7
1
vote
1 answer

Create Hive table from AVSC that contains reference to previous defined schema as a type

I am looking to find a way through Hive to take the following AVSC file content and externalize the nested schema "RENTALRECORDTYPE" for the purposes of schema reuse. { "type": "record", "name": "EMPLOYEE", "namespace": "", "doc":…
dtom
  • 11
  • 3
0
votes
1 answer

How to convert an .avsc Avro schema file into an .avdl Avro schema file as part of a Maven build?

The Confluent Schema Registry Maven Plugin can not register .avdl Avro schema files in a schema registry, only .avsc Avro schema files. The Apache Avro Maven plugin can not convert .avdl Avro schema files into .avsc Avro schema files. How can I…
Harold L. Brown
  • 8,423
  • 11
  • 57
  • 109
0
votes
1 answer

Unable to serialize Avro schema when we have an array with null objects

I have list of objects, out of which one object is null. When I am trying to serialize the below data, I am getting null of string of array of union of [object1, object2, object3, null, object4] Schema Definition { "name": "myList", "type":…
Kamal Chanda
  • 163
  • 2
  • 12
1
2