protocol buffers in python: no classes generated

Question

My proto file is as follows:

syntax = "proto3";
option csharp_namespace = "Proto";

message FileListRequest {
    repeated File Files = 1;
}

message File {
    string Path = 1;
}

message ImageFile {
    File File = 1;
    Size Size = 2;
    bytes Content = 3;
}

message Size {
    int32 Width = 1;
    int32 Height = 2;
}

message SendNextFile {
    
}

I compile it with the following command:

protoc --proto_path=. -I . --python_out=..\..\python\Modules\PreloadingIteratorWrapper\ .\filelist.proto

This creates the following file:

# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: filelist.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x0e\x66ilelist.proto\"\'\n\x0f\x46ileListRequest\x12\x14\n\x05\x46iles\x18\x01 \x03(\x0b\x32\x05.File\"\x14\n\x04\x46ile\x12\x0c\n\x04Path\x18\x01 \x01(\t\"F\n\tImageFile\x12\x13\n\x04\x46ile\x18\x01 \x01(\x0b\x32\x05.File\x12\x13\n\x04Size\x18\x02 \x01(\x0b\x32\x05.Size\x12\x0f\n\x07\x43ontent\x18\x03 \x01(\x0c\"%\n\x04Size\x12\r\n\x05Width\x18\x01 \x01(\x05\x12\x0e\n\x06Height\x18\x02 \x01(\x05\"\x0e\n\x0cSendNextFileB\x08\xaa\x02\x05Protob\x06proto3')

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'filelist_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:

  DESCRIPTOR._options = None
  DESCRIPTOR._serialized_options = b'\252\002\005Proto'
  _FILELISTREQUEST._serialized_start=18
  _FILELISTREQUEST._serialized_end=57
  _FILE._serialized_start=59
  _FILE._serialized_end=79
  _IMAGEFILE._serialized_start=81
  _IMAGEFILE._serialized_end=151
  _SIZE._serialized_start=153
  _SIZE._serialized_end=190
  _SENDNEXTFILE._serialized_start=192
  _SENDNEXTFILE._serialized_end=206
# @@protoc_insertion_point(module_scope)

According to the documentation this file should contain a class for each message type, but it doesn't. Why?

Answer 1

I have the same issue. Experimenting with grpc-tools (I found answer in another thread suggesting usage of this tool) I finally found a solution.

Just add to protoc command this arg: --pyi_out like:

protoc --proto_path=. -I . --python_out=..\..\python\Modules\PreloadingIteratorWrapper\ --pyi_out=..\..\python\Modules\PreloadingIteratorWrapper\ .\filelist.proto

It will generate for you _bp2.py as well as corresponding _bp2.pyi (stub file). After that your IDE will see the class names, intellisense will work etc.

Answer 2

Documentation says:

Unlike when you generate Java and C++ protocol buffer code, the Python protocol buffer compiler doesn't generate your data access code for you directly.

It means, that your .proto files won't be converted into some familiar accessors (no classes, no methods and no properties defined)

Then, in docs, you see this:

Instead (as you'll see if you look at addressbook_pb2.py) it generates special descriptors for all your messages, enums, and fields, and some mysteriously empty classes, one for each message type

Which means:

• You'll see descriptors, parsed from serialized .proto file
• You'll see variables, named exactly the same as in .proto file

But these "variables" are nothing but GeneratedProtocolMessageType class, which is a metaclass, but for protocol messages

Then, you should look into docs for GeneratedProtocolMessageType from package google.protobuf.internal.python_message, where you should see this line:

Metaclass for protocol message classes created at runtime from Descriptors.

And this line means that you won't see any expected properties or methods while you code. Because these variables will become metaclasses for your protocol messages only at runtime! These metaclasses, at the time you look at the lines with their instantiation, are factories for your protocol messages

Moreover, this behavior is mentioned in the middle of the docs for that class:

The protocol compiler currently uses this metaclass to create protocol message classes at runtime.

It works only that way:

We add implementations for all methods described in the Message class. We also create properties to allow getting/setting all fields in the protocol message. Finally, we create slots to prevent users from accidentally "setting" nonexistent fields in the protocol message, which then wouldn't get serialized / deserialized properly.

This metaclass generates classes for your protocol messages, described in .proto files, using descriptors from generated files. Only at runtime (not at coding time)
And only at runtime you'll be able to use them as factories for your classes (those, which you expect to see in code) and, also, as a types for method parameters

So there is nothing wrong with Google Protobuf Documentation, it is not outdated