Dynamically creating python class from a protobuf file at run time?

Question

In Python, is it possible at run time to convert a Google Protocol Buffers .proto file into a python class that reads that data? Python is a very dynamic language. When you use protoc to convert a .proto file to python source code, the generated code makes a lot of use of python metaclasses, so it's already very dynamic.

Ideally, I'm thinking of something like this:

import whatever
module = whatever.load_from_file("myfile.proto")

Is this possible?

(I am new to protocol buffers, please let me know if my question makes no sense)

@AndriyIvaneyko Perhaps my question was unclear, I have updated it. I'm talking about Google's Protocol Buffers format — Amandasaurus, Feb 11 '16 at 15:02
If the `.proto` file itself is a python source code that you want to include as a module, you are probably looking for this: `import imp; module = imp.load_source('any_name', 'myfile.proto')` — Berci, Feb 11 '16 at 15:05
Anyway, just import the converted python code (perhaps use `sys.path.append(its_dir)` before). — Berci, Feb 11 '16 at 15:06

score 13 · Accepted Answer · answered Feb 13 '16 at 19:26

In theory, all the pieces exist to make this work. The Python protobuf implementation could call the C++ .proto parser library (libprotoc) as a C extension to get Descriptors, and then could feed those into the metaclasses.

However, as far as I know, no one has quite tied it altogether. (Disclaimer: My knowledge is a few years old, I may have missed a new development, but I don't see anything in the docs.)

Incidentally, Cap'n Proto's Python implementation does do what you describe, proving it is possible. But that doesn't help you if you need to work with Protobuf format.

(Disclosure: I was the author of most of Google's open source Protobuf code, and I am also the author of Cap'n Proto.)

Alas, I need to work with protobuf files. I'm not in a position to change that. However "No, I don't think anyone has done this" is an acceptable answer. — Amandasaurus, Feb 14 '16 at 12:28

score 1 · Answer 2 · answered Aug 30 '17 at 18:24

The most elegant solution to this problem is to use init.py. See: What is __init__.py for?

You can invoke your script to generate Python protobuf classes in init.py. Upon importing, that script will be automatically invoked to generate Python protobuf classes.

Dynamically creating python class from a protobuf file at run time?

2 Answers2