How to generate python code during a build and include those in a python wheel?

Question

We have a package that generates code by means of

$PYTHON -m grpc_tools.protoc -I="foo_proto" --python-out="$package/out" \
         --grpc_python_out="$package/out" ./path/to/file.proto

This was integrated (read hacked) into our setup.py building by means of:

from distutils.command.build_py import build_py

class BuildPyCommand(build_py):
    """
    Generate GRPC code before building the package.
    """
    def run(self):
        import subprocess
        subprocess.call(["./bin/generate_grpc.sh", sys.executable], shell=True)
        build_py.run(self)

setup(
      ....
      cmdclass={
        'build_py': BuildPyCommand
    },
)

Which ugly as is, seems to work when building with the legacy setup.py, but when the package is built with wheel it doesn't work at all.

How can I make this work when installing my package by means of wheel?

See if these threads help https://stackoverflow.com/questions/23585450/automate-compilation-of-protobuf-specs-into-python-classes-in-setup-py, https://stackoverflow.com/questions/27843481/python-project-using-protocol-buffers-deployment-issues, https://pypi.org/project/protobuf-distutils/ — Tarun Lalwani, Mar 29 '21 at 10:27
Can you provide some more information? Exceptions? The .sh file? — Can H. Tartanoglu, Apr 02 '21 at 19:12

score 2 · Accepted Answer · answered Apr 03 '21 at 08:35

You can override the wheel build process as well:

from wheel.bdist_wheel import bdist_wheel
from distutils.command.build_py import build_py
import subprocess


def generate_grpc():
    subprocess.call(["./bin/generate_grpc.sh", sys.executable], shell=True)


class BuildPyCommand(build_py):
    """
    Generate GRPC code before building the package.
    """
    def run(self):
        generate_grpc()
        build_py.run(self)


class BDistWheelCommand(bdist_wheel):
    """
    Generate GRPC code before building a wheel.
    """
    def run(self):
        generate_grpc()
        bdist_wheel.run(self)


setup(
      ....
      cmdclass={
        'build_py': BuildPyCommand,
        'bdist_wheel': BDistWheelCommand
    },
)

How to generate python code during a build and include those in a python wheel?

1 Answers1