20

I am trying to use the Python Avro library (https://pypi.python.org/pypi/avro) to read a AVRO file generated by JAVA. Since the schema is already embedded in the avro file, why do I need to specify a schema file? Is there a way to extract it automatically?

Found another package called fastavro(https://pypi.python.org/pypi/fastavro) can extract avro schema. Is the manual specifying schema file in python arvo package by design? Thank you very much.

ljxue
  • 305
  • 1
  • 3
  • 7

3 Answers3

14

I use python 3.4 and Avro package 1.7.7

For schema file use:

reader = avro.datafile.DataFileReader(open('file_name.avro',"rb"),avro.io.DatumReader())
schema = reader.meta
print(schema)
Alon Hazan
  • 191
  • 1
  • 7
  • This worked well with Python 2.7 as well. My import statements are as follows (not sure how much you need): import avro.schema from avro.datafile import DataFileReader from avro.io import DatumReader – B. Griffiths Dec 01 '17 at 19:58
9

A direct examination of /usr/local/lib/python2.7/site-packages/avro/datafile.py reveals the answer:

reader = avro.datafile.DataFileReader(input,avro.io.DatumReader())
schema = reader.datum_reader.writers_schema
print schema

Curiously, in Java there is a special method for that: reader.getSchema().

sds
  • 58,617
  • 29
  • 161
  • 278
2

In my case in order to get the schema as a "consumable" python dictionary containing useful info such schema name and so on I did the following:

reader: DataFileReader = DataFileReader(open(avro_file, 'rb'), DatumReader())
schema: dict = json.loads(reader.meta.get('avro.schema').decode('utf-8'))

The reader.meta is a dictionary pretty useless "as is", since it contains 2 keys: avro.codec and avro.schema that are both bytes objects (so I had to parse it in order to access to properties).

daveoncode
  • 18,900
  • 15
  • 104
  • 159