VSCode Python autocomplete for generated code in separate directory

Question

I use pants to manage a Python project that uses protocol buffers. Pants places the generated _pb2.py and _pb2.pyi files under a separate dist/codegen tree. Is it possible to get VS Code autocomplete to work when using the _pb2 modules?

The file tree looks like this:

.
|-- dist/
|   `-- codegen/
|       `-- src/
|           `-- project/
|               |-- data_pb2.py
|               `-- data_pb2.pyi
`-- src/
    `-- project/
        |-- __init__.py
        |-- code.py
        `-- data.proto

And in code.py I have import statements like this:

from project import data_pb2

I've tried setting python.analysis.extraPaths to ["dist/codegen/src"] in settings.json. This makes pylance stop complaining that data_pb2 is missing. But autocomplete still does not work, and pylance has no type information for members of data_pb2.

score 0 · Answer 1 · answered Jun 08 '22 at 04:17

0

Replace your python.analysis.extraPaths with the following extent:

    "python.analysis.extraPaths": [
    "./dist/codegen/src"
     ],

And adding the following code to your code.py:

import sys
sys.path.append(".\dist\codegen\src")

answered Jun 08 '22 at 04:17

MingJie-MSFT

5,569
1
2
13

This doesn't seem to help. Still no autocomplete or typing information. Note that there's no `__init__.py` in `dist/codegen/src/project` so I wouldn't expect modifying `sys.path` or `PYTHONPATH` to help here. – satwell Jun 08 '22 at 05:00
This works for me. And it doesn't need to creat __init__.py in it. And how can pylance recognize the __pb2 if you don't tell it by modifying sys.path or PYTHONPATH. – MingJie-MSFT Jun 08 '22 at 05:17
I do get autocomplete and typing information if I put my proto in a separate directory. E.g., `src/project/code.py` and `src/data/data.proto` which generates `dist/codegen/src/data/data_pb2.py`. It seems like having the generated code at the same path causes issues. – satwell Jun 08 '22 at 14:10

score 0 · Accepted Answer · answered Jun 10 '22 at 14:47

You can use Python implicit namespace packages (PEP 420) to make this work. Namespace packages are allowed to have modules within the same package reside in different directories. Which allows pylance and other tools to work correctly when code is split between src and dist/codegen/src.

To use implicit namespace packages, you just need to remove src/package/__init__.py, and leave "python.analysis.extraPaths" set to ["dist/codegen/src"].

See also the GitHub issue microsoft/pylance-release#2855, which describes using implicit namespace packages to make pylance work correctly in a similar situation.

VSCode Python autocomplete for generated code in separate directory

2 Answers2