9

There's a few of these question I've read through, but they all seem to link to methods exclusively for python 2.x and I'm working in python 3.x.

I have a python file, call it test.py, it has a number of package imports (eg. import numpy as np) but it also imports some other files, lets call them subTest1.py and subTest2.py. Each of the subTest files have their own imports (possibly both other files and packages). Is there an automated way to get a list of all packages required to run the route script (test.py)?

I'm aware that pip freeze shows me every packages installed, but that's not what I'm looking for - there could be a package installed that isn't required to run the test.py script.

Questions that are similar but python 2.x:

Return a list of imported Python modules used in a script?

Get all modules/packages used by a python project

Packages that seem to be specific to python 2.x:

snakefood

pipreqs

Where pipreqs fails:

Traceback (most recent call last):
  File "c:\anaconda36\envs\tensorflow\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\anaconda36\envs\tensorflow\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Anaconda36\envs\tensorflow\Scripts\pipreqs.exe\__main__.py", line 9, in <module>
  File "c:\anaconda36\envs\tensorflow\lib\site-packages\pipreqs\pipreqs.py", line 396, in main
    init(args)
  File "c:\anaconda36\envs\tensorflow\lib\site-packages\pipreqs\pipreqs.py", line 341, in init
    extra_ignore_dirs=extra_ignore_dirs)
  File "c:\anaconda36\envs\tensorflow\lib\site-packages\pipreqs\pipreqs.py", line 91, in get_all_imports
    raise exc
  File "c:\anaconda36\envs\tensorflow\lib\site-packages\pipreqs\pipreqs.py", line 77, in get_all_imports
    tree = ast.parse(contents)
  File "c:\anaconda36\envs\tensorflow\lib\ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 49
    print vsize
              ^
SyntaxError: Missing parentheses in call to 'print'
user6916458
  • 400
  • 2
  • 7
  • 19
  • if you are using PyCharm IDE, there is a plugin called requirements that does that per project scope – ElSheikh Nov 06 '19 at 20:08
  • @ElSheikh I wish I was. I'm using anaconda... that's a good point though, I'll see if anaconda has something similar – user6916458 Nov 06 '19 at 20:09

3 Answers3

20

You can use pipreqs package. (Docs)

To Install:

pip3 install pipreqs

Usage:

pipreqs ./your_script_directory

It produces a requirements.txt file in your script directory with all the dependencies used.

Rithin Chalumuri
  • 1,739
  • 7
  • 19
  • Thanks for the answer. Does pipreqs support python3? When trying to use it, it seems to fail – user6916458 Nov 06 '19 at 20:21
  • Yes it does. What do you mean fail? any errors or invalid modules in requirements.txt? – Rithin Chalumuri Nov 06 '19 at 20:22
  • It fails to run, with the error: SyntaxError: Missing parentheses in call to 'print'. I assumed that's due to python3 requiring print be used with parentheses and python2 not requiring them. I installed pipreqs using pip install, so I should have gotten the most recent version – user6916458 Nov 06 '19 at 20:23
  • @user6916458, You should use `pip3` instead of `pip`. You pip might still point to python2. – Rithin Chalumuri Nov 06 '19 at 20:25
  • I've added the entire output from pipreqs to the post – user6916458 Nov 06 '19 at 20:25
  • When using pip3 I get the same error. (I uninstalled pipreqs using pip uninstall..., and then reinstalled with pip3...) – user6916458 Nov 06 '19 at 20:26
  • @user6916458, Try creating a new conda environement like `conda create -n myenv python=3.8` and then running the above. – Rithin Chalumuri Nov 06 '19 at 20:29
  • I've tried making a new environment and running the script again and it has the same issue – user6916458 Nov 06 '19 at 20:34
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/201966/discussion-between-rithin-chalumuri-and-user6916458). – Rithin Chalumuri Nov 06 '19 at 20:36
  • The package is also available with `conda`: `conda install pipreqs` – Mathador May 20 '20 at 14:09
  • This doesn't appear to consider imported packages that were not originally installed through pip. Perhaps obvious from the name, but a problem for the use I was looking at. – Akh Aug 23 '22 at 00:02
  • `pigar` works better: https://stackoverflow.com/a/71445496/1599699 – Andrew Jul 21 '23 at 17:33
0

I have found this script in the google and it seems to be working,I have updated it for python 3,(just print function), I have tested it and it lists modules, but I believe it goes too deep under the modules but worth trying. 

from modulefinder import ModuleFinder
f = ModuleFinder()

# Run the main script
f.run_script('test.py')

# Get names of all the imported modules
names = list(f.modules.keys())

# Get a sorted list of the root modules imported
basemods = sorted(set([name.split('.')[0] for name in names]))
# Print it nicely
print("\n".join(basemods))

just change test.py in any filename you want to check. and then run the script from terminal

python module_finder_script.py

let me know if it helps.

man of knowledge
  • 1,077
  • 7
  • 13
-2

You can try this --

  1. Install gunicorn -- pip3 install gunicorn
  2. Inside your project directory, type -- pip3 freeze > requirements.txt

This (requirements.txt) will give you the list of all the packages used inside the project directory.

Blue Bird
  • 193
  • 3
  • 8