1

I am creating a dependency diagram of all python modules on my Linux machine. I want the paths to be displayed so that I can correctly identify where in the system an import is.

I am using .rglob() to search through the machine starting at the root directory. The only problem with this is that I get a FileNotFoundError when the program gets to the /proc/ directory.

If there is a better way to search for the path of imported modules or if there is a way to avoid the /proc/ directory entirely please let me know below.

Here is my code:

def find_import_path(import_name):
    "Get the path of an import."
    big_globber = Path('/').rglob(f'{import_name}')
    matches = [path for path in big_globber]
    # One match
    if len(matches) == 1:
        return str(matches[0])
    # No matches..Should not be possible - scripts on this machine are importing the import_name.
    elif not matches:
        return ''
    # Multiple matches.
    else:
        return ''.join(f'{match}\n' for match in matches)

The error will occur on line 4.

I tried some exception handling already and had the program redundantly call back to find_import_path() which worked but ended up having an OSError pop from somewhere deeper within /proc/ again.

Traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/bushbak2/projects/software-dependency-diagram/depend.py", line 216, in <module>
    main()
  File "/home/bushbak2/projects/software-dependency-diagram/depend.py", line 20, in main
    generate_graphviz_diagram(parse_script(directory_of_execs))
  File "/home/bushbak2/projects/software-dependency-diagram/depend.py", line 129, in generate_graphviz_diagram
    imp_path = find_import_path(import_module)
  File "/home/bushbak2/projects/software-dependency-diagram/depend.py", line 202, in find_import_path
    matches = [path for path in big_globber]
  File "/home/bushbak2/projects/software-dependency-diagram/depend.py", line 202, in <listcomp>
    matches = [path for path in big_globber]
  File "/usr/local/lib/python3.9/pathlib.py", line 1179, in rglob
    for p in selector.select_from(self):
  File "/usr/local/lib/python3.9/pathlib.py", line 599, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/usr/local/lib/python3.9/pathlib.py", line 589, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/usr/local/lib/python3.9/pathlib.py", line 589, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/usr/local/lib/python3.9/pathlib.py", line 578, in _iterate_directories
    with scandir(parent_path) as scandir_it:
FileNotFoundError: [Errno 2] No such file or directory: '/proc/2812016'
Bushbaker27
  • 35
  • 9
  • 1
    The process with PID 2812016 existed when `rglob` was called but not when the returned iterator was consumed. It's not generally safe to expect that no processes will start/end between those two lines. You almost certainly don't want to search `/proc` at all, and for that matter `/dev`, `/root`, `/boot`, etc. If you're just trying to determine the path to a module, [python already has tools](https://stackoverflow.com/q/247770/11082165) to do that. – Brian61354270 Aug 05 '21 at 17:14
  • Thank you for the suggestion, but I plan on expanding this program to draw dependency diagrams of more languages than python. Will this method work on files that are in other languages? – Bushbaker27 Aug 05 '21 at 19:31

0 Answers0