2

I'm writing a pylint checker and I need to differentiate between an import that is a sibling import of a package and an import of a function or class.

Example of sibling import:

from . import sibling_package

Example of a function import:

from numpy import array

The latter example I want to flag, while the former I want to allow, so I need to be able to tell the difference between the two.

I'm currently using:

modspec = importlib.util.find_spec('numpy', 'array')

That returns a ModuleSpec, but I'm unclear how I can get to the goal of identifying the import array as a module vs. a function/class. In this example it is a function import, and thus should be flagged.

David Parks
  • 30,789
  • 47
  • 185
  • 328
  • This is not something you can easily detect without re-running the exact same import. That's because another module can do `import foo`, which adds the name `foo` to that module, and then `from bar import foo` would return that module. That doesn't make `foo` a sibling package however! – Martijn Pieters Aug 28 '18 at 19:00
  • As a module, I would accept it. My primary goal is to flag cases when a function or class is imported rather than the package that contains it. Unfortunately I don't see a way to use `ModuleSpec` to identify it as a module vs. something else (class/function/other?). – David Parks Aug 28 '18 at 19:03
  • Is this question a dupe of https://stackoverflow.com/questions/624926/how-do-i-detect-whether-a-python-variable-is-a-function ? Are you simply trying to see if `modspec` is a function? – Scott Mermelstein Aug 28 '18 at 19:12
  • 1
    You can’t detect the type from the import statement, no. You’ll have to actually execute the import, or maintain a list of modules yourself – Martijn Pieters Aug 28 '18 at 19:12
  • As far as I can see @Martijn is correct, I can use `importlib.import_module` in the same way as `find_spec`, and will have to import the module from the pylint checker (which will require some work to get the PYTHONPATH set up appropriately). That would be an appropriate answer to post. – David Parks Aug 28 '18 at 19:15
  • @Scott, not whether `ModSpec` is a function, whether the thing `ModSpec` is describing is a function. But that's still a useful reference link for the question. – David Parks Aug 28 '18 at 19:16
  • I take it back, this is still challenging. `importlib` doesn't seem to allow me to perform an import that could be either a module or a function/class. It seems like my only option might be to actually do an eval of the original text of the import statement, which is non-trivial because of all the ways an import statement could possibly be written. – David Parks Aug 28 '18 at 20:12

1 Answers1

2

This is not something you can easily detect just from the import line. Python is highly dynamic and not until runtime can you know what type of object the import resolves to. The module spec can't tell you this information, because an attribute on a module can resolve to anything (including another module).

The alternatives that I can see are:

  • to do the actual import, then test the object type.

    This is not without risk, imports can have side effects. Importing a module includes executing the top level statements. Those side effects can be mild, like replacing one object with another when a dependency is not met (try: from itertools import zip_longest, except ImportError: from itertools import izip_longest as ziplongest is a trivial Python 2 vs. Python 3 dependency check), but potentially an import could make filesystem changes!

    importing is also going to slow down the check. Importing a module like numpy or pandas can pull in å significant number of additional modules. You generally want to keep linting fast, or otherwise developers will tend not to bother and skip linting altogether.

  • Keep a list of known modules. For those modules you know about, complain if they are importing names from the module and not the module itself. This is fast, and will catch the majority of common cases. You can augment the list with what you can glean from the filesystem around the module being linted. Put differently, aim for fast and good enough and accept a few misses for new imports.

  • Only complain if imported names are called directly. Register all the names that are imported, and if the AST contains a Call node for that name then you know they imported a function or class. from foo import bar, then later on spam = bar('baz') is a clear indicator that bar is not a module.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343