69

I have a python GUI application. And now I need to know what all libraries the application links to. So that I can check the license compatibility of all the libraries.

I have tried using strace, but strace seems to report all the packages even if they are not used by the application.

And, I tried python ModuleFinder but it just returns the modules that are inside python2.7 and not system level packages that are linked.

So is there any way I can get all the libraries that are linked from my application?

Vertexwahn
  • 7,709
  • 6
  • 64
  • 90
user2109788
  • 1,266
  • 2
  • 12
  • 29
  • 3
    pip freeze for all installed packages – wolendranh Mar 04 '16 at 13:11
  • 1
    pip install yolk, which gives you a list of all the eggs in your environment using: *yolk -l* – datashaman Mar 04 '16 at 13:15
  • @wolendranh, I am concerned with the use of `pip freeze` as I guess that pip freeze will simply list down all the packages present in that virtual env , but that might not be used by my application and I might have messed up my env by adding some unwanted packages.. how to solve that? I am more interested in cleaning up the dependencies list, sort of!! – eRaisedToX Jul 08 '21 at 18:08

3 Answers3

133

You can give a try to the library https://github.com/bndr/pipreqs found following the guide https://www.fullstackpython.com/application-dependencies.html

The library pipreqs is pip installable and automatically generates the file requirements.txt.

It contains all the imports libraries with versions you are using in the virtualenv or in the python correctly installed. Just type:

pip install pipreqs
pipreqs /home/project/location

It will print:

INFO: Successfully saved requirements file in /home/project/location/requirements.txt

In addition it is compatible with the pip install -r command: if you need to create a venv of your project, or update your current python version with compatible libraries, you just need to type:

pip install -r requirements.txt

I had the same problem and this library solved it for me. Not sure if it works for multiple layers of dependencies i.e. in case you have nested level of dependent libraries.

-- Edit 1:

If looking for a more sophisticated version manager, please consider as well pyvenv https://github.com/pyenv/pyenv. It wraps virtualenv producing some improvements over the version specification that is created by pipreqs.

-- Edit 2:

If, after creating the file with the dependency libraries of your module with pipreqs, you want to pin the whole dependency tree, take a look at pip-compile. It figures out a way to get the dependencies of your top level libraries, and it pins them in a new requirement files, indicating the dependency tree.

-- Edit 2:

If you want to split your dependency tree into different files (e.g. base, test, dev, docs) and have a way of managing the dependency tree, please take a look at pip-compile-multi.

SeF
  • 3,864
  • 2
  • 28
  • 41
  • seems that this install *latest* version of the packages and not the current version you are using. – enneppi Mar 02 '18 at 12:07
  • 1
    One of the best answers one could possible get – Venkata S S K M Chaitanya Mar 22 '18 at 14:00
  • @ennepi: does not look the case for me. Are you using it in a virtual-environment? – SeF Apr 16 '18 at 15:32
  • 2
    @ennepi, ok, got it. You may need to add the flag --use-local to get what you are looking for – SeF Apr 17 '18 at 14:04
  • 2
    Important! both pipreqs and other scanning packages fail on BOM character (UTF-8) on start of any .py file in your path. It fails ugly... So, if you get a bad character syntax error, remove bom character (using nodepad++ for example...) and you're good to go. – rubmz Jan 29 '19 at 21:48
  • I am interested in knowing if a module inside my env is actually needed by my application code or not... how to do that? thanks! – eRaisedToX Jul 08 '21 at 18:11
  • You can use pre-commit, if your application is versioned controlled, or simply a linter, like pylint. Pylint will indicate as warnings all the modules that are imported but not used. – SeF Jul 09 '21 at 09:29
  • pipreqs works, but I dont see how to use your other suggesion: pyenv. I have installed it, but what should I do to generate a requirements file of my python package? – Eelco van Vliet Jul 28 '22 at 05:07
  • Yep, I was indicating it as a possible package manager, after having found the packages with pipreqs. (I'll amend the answer to make it clear). – SeF Jul 29 '22 at 07:53
  • Any possibility of a package that scans a directory for all .py files, and scans those for all imports, and creates a list of packages that way? Also referring to host machine (or a named venv) for package versions would be nice. – MrChadMWood Jun 29 '23 at 15:08
  • 1
    @MrChadMWood as far as I know pipreqs does exactly this scan, but the version are the latest ones, not the one on the virtualenv. You can combine the requirements file with the result of pip freeze to get what you are looking for. – SeF Jul 21 '23 at 15:01
  • 1
    @SeF Thanks for mentioning that. I'm looking into it, and maybe it can do both! https://pypi.org/project/pipreqs/: `--use-local: Use ONLY local package info instead of querying PyPI` – MrChadMWood Jul 21 '23 at 20:43
2

Install yolk for python2 with:

pip install yolk

Or install yolk for python3 with:

pip install yolk3k

Call the following to get the list of eggs in your environment:

yolk -l

Alternatively, you can use snakefood for graphing your dependencies, as answered in this question.

You could try going into the site-packages folder where the unpacked eggs are stored, and running this:

ls -l */LICENSE*

That will give you a list of the licence files for each project (if they're stored in the root of the egg, which they usually are).

datashaman
  • 8,301
  • 3
  • 22
  • 29
-3

To get all the installed packages or modules. A very easy way is by going to your virtual environment directory on the terminal (The one with (venv) behind your computer's username) and run on the terminal one of these commands

pip freeze > requirements.txt

If you are using python3

pip3 freeze > requirements.txt

This would get all your installed packages from your virtual environment's library of packages used during the project, and store them in the 'requirements.txt' file that would automatically be created upon running the command.

Alienzzz
  • 1
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Sep 28 '22 at 20:47