3

I tried to use protocol buffers on my project and the problem I have is that when I use protoc to generate the python class. The file that's generated looks nothing like in the example provided by Google and cannot be imported in any file because there are some unresolved references.
So I followed the example from this page: https://developers.google.com/protocol-buffers/docs/pythontutorial

Preconditions

  1. Operating system macOS 12.6 on M1 Mac.
  2. I used Python 3.9.11 in a vrtualenv managed with pyenv and pyenv-virtualenv
  3. I downloaded the latest python package from https://github.com/protocolbuffers/protobuf/releases/tag/v21.7
  4. I installed protobuf with homebrew https://formulae.brew.sh/formula/protobuf
  5. I followed this instruction to install the package https://github.com/protocolbuffers/protobuf/tree/v21.7/python
  6. I also copiled the c++ protoc from the above protobuff package to see if it helps but it didn't

The packages I had in the end were:

$ python --version
$ Python 3.9.11
$ 
$ protoc --version
$ libprotoc 3.21.7
$
$ pip freeze | grep protobuf
$ protobuf==3.20.2

The code

First I try to generate the a python class from this tutorial .proto file:

syntax = "proto2";

package tutorial;

message Person {
  optional string name = 1;
  optional int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    optional string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phones = 4;
}

message AddressBook {
  repeated Person people = 1;
}

Then I use the command to generate the python class

protoc -I=. --python_out=. tutorial.proto

And the output file is:

# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: tutorial.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x0etutorial.proto\x12\x08tutorial\"\xd5\x01\n\x06Person\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12,\n\x06phones\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x1aG\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x01(\t\x12(\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06people\x18\x01 \x03(\x0b\x32\x10.tutorial.Person')

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'tutorial_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:

  DESCRIPTOR._options = None
  _PERSON._serialized_start=29
  _PERSON._serialized_end=242
  _PERSON_PHONENUMBER._serialized_start=126
  _PERSON_PHONENUMBER._serialized_end=197
  _PERSON_PHONETYPE._serialized_start=199
  _PERSON_PHONETYPE._serialized_end=242
  _ADDRESSBOOK._serialized_start=244
  _ADDRESSBOOK._serialized_end=291
# @@protoc_insertion_point(module_scope)

So as you can see there are no metaclasses created and all the constants below the line DESCRIPTOR.options=None are Unresolved references.

When I try to import that file later, runtime obviously crashes as this is not a valid Python file.

Any ideas?

Piotr BG
  • 333
  • 1
  • 6

2 Answers2

2

Turned out the example code works well and the problem was elsewhere in my code.
A few confusing things that made me question the code were:

  1. The example on the official page is outdated. Google has changed the way the protobuffs are generated and there are no metaclasses anymore.
  2. There are dynamic imports so an IDE will highlight descriptors like _PERSON as not defined but the code still works when launched.

So yeah I spent time debugging a not existing issue ;).

Piotr BG
  • 333
  • 1
  • 6
1

I had the same problem. It reported a bunch of errors in pb file,but it can be referenced normally in other files. In my opinion, it may just be a protocol update that causes the editor to fail to recognize this new format.

Fool The
  • 11
  • 1