Python 3 import conflicts with internal "parser"

Question

.
├── gen.py
├── lexer
│   ├── engine.py
|   └── ...
└── parser
    ├── engine.py
    └── ...

I'm writing my compiler project, but now i'm stuck with python import conflicts. In gen.py, I want to import some function for code generation, like

import lexer.engine   # OK
import parser.engine  # ModuleNotFoundError: No module named 'parser.engine'; 'parser' is not a package

After some investigation, I've learned that "parser" is reserved for python internal parser. But I cannot change the directory name "parser", since it has been used everywhere.

How can I solve the problem?

Do you have an `__init__.py` file in the `parser` directory? — Neo, Jun 11 '18 at 10:09
you should be able to change your name easily in either an IDE (like pycharm or other. they should have some sort of function allowing renaming of folders in your project) or using `sed` in linux or a linux-like environment (on windows, if you download git, it comes with a simulated bash shell) — e.s., Jun 11 '18 at 10:23
@e.s. "parser" is used not only in C++ files, but also many hard-code python generator and even CMakeLists. And I hate the idea to change the folder name. — Mike Dog, Jun 12 '18 at 11:18
As @Neo says, create an empty `parser/__init__.py` file (that's two underscores before, and two underscores after `init`!). This way your `parser` will be recognized as a module, and `import parser` will be loading it. — Amadan, Jun 12 '18 at 12:03
@Amadan It works. Please add it as an answer here, then I can mark it accepted. — Mike Dog, Jun 15 '18 at 12:22
@Neo It works. Please add it as an answer here, then I can mark it accepted. — Mike Dog, Jun 15 '18 at 12:22

score 2 · Accepted Answer · answered Jun 15 '18 at 13:09

To be accepted as a Python package, a directory must have an __init__.py file.
Variables declared in this file can be access as if the package is a regular module. It can also be empty.

tl;dr: Add an empty __init__.py file to the directory and it should work.

Druta Ruslan · Answer 2 · 2018-06-12T11:34:10.310

1

python have a module parser , and you want to import engine that it doesn't have, if you wiil try to

import parser
print(dir(parser))
print(parser.__file__)

and you will see that parser module is not your parser module.

Simple rename your parser folder and every thing will be all right

Update

can you try to import with from for example:

from parser import engine

edited Jun 12 '18 at 11:34

answered Jun 11 '18 at 10:18

Druta Ruslan

7,171
2
28
38

1

Renaming is not an option for me, since it will break many dependencies. – Mike Dog Jun 12 '18 at 11:24
@MikeDog try to import with `from` i update my answer – Druta Ruslan Jun 12 '18 at 11:34
no use... you may try it before updating your answer. – Mike Dog Jun 12 '18 at 11:42
i did't it like in your example, and for me work `from parser import engine`, i create folders and files that you have – Druta Ruslan Jun 12 '18 at 11:45

e.s. · Answer 3 · 2018-06-12T16:27:59.023

Assuming your project is actually contained in a project directory like so, ...

my_package
| 
├── gen.py
├── lexer
│   ├── __init__.py
│   ├── engine.py
|   └── ...
├── parser
|   ├── __init__.py
|   ├── engine.py
|   └── ...
├── tests  
|   ├── test_thingy.py

in gen.py:

import my_package.lexer.engine
import my_package.parser.engine

in the parent directory of my_package,you can run python -m my_package.gen. This should run exactly as expected without the name conflicts. With similar import statements in your tests, if you run your test modules in the same way, it should work just fine.

I tested this with the following. In E/work/temp/ I have a directory called my_package. It has the following structure.

my_package
    | 
    ├── __init__.py  # needed in python 2, but not 3
    ├── import_test_b.py
    ├── parser
    |   ├── __init__.py
    |   └── import_test_a.py
    └── tests  
        ├── __init__.py  # needed in python 2 but not python 3
        └── test_imports.py

import_test_a:

def test(num):
    return num+3

import_test_b:

from my_package.parser.import_test_a import tst

print(tst(4))

test_imports.py:

from my_package.parser.import_test_a import tst

import unittest

class TestTst(unittest.TestCase):
    def test_one(self):
        self.assertEqual(tst(4), 7)

if __name__ == '__main__':
    unittest.main()

in E/work/temp:

run: python -m my_package.import_test_b - output = 7

run: python -m my_package.tests.test_imports output: .

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Python 3 import conflicts with internal "parser"

3 Answers3