1

I have a package with a list of many required packages that has grown over the lifetime and I do not know which are still relevant and which are obsolete.

How can I extract the relevant requirements i.e. the ones whose imports are used in the main package?

Adriaan
  • 17,741
  • 7
  • 42
  • 75
Cord Kaldemeyer
  • 6,405
  • 8
  • 51
  • 81
  • try brute force :D – sahasrara62 Dec 06 '22 at 19:34
  • The best answer. It should be #1 :D – Cord Kaldemeyer Dec 06 '22 at 19:37
  • 2
    Spontaneously, I'd also say brute force, i.e. starting on a clean environment and installing packages until "it" works. The problem with that is "it". "it" is easy if the package has good code-coverage. Also a `grep import` on the py-files can probably reveal the dependencies. I mean, this is not something you need to do every day and would require a fully automatic process. – Dr. V Dec 06 '22 at 20:48
  • 1
    if the package has unit tests you could: #0 create a venv without global packages access #1 `pip install bloater`. #2 pip uninstall . #3 run unit tests and #4 use the package. but if it's not your own package, you will still get re-whacked with all deps on any new `pip install bloater` so the overall gain is limited if you can't get those dependencies trimmed back down. step #1.5 do a pip freeze/pip list to see everything pulled in. – JL Peyret Dec 06 '22 at 21:16
  • 2
    Question already asked many times. Use [pipreqs](https://pypi.org/project/pipreqs/) and/or [pigar](https://pypi.org/project/pigar/). – sinoroc Dec 06 '22 at 22:25
  • 1
    The question is NOT already asked many times but different if you read carefully. Nonetheless, the package `pigar` lists the USED packages when calling it in the following way: `pigar gen --with-referenced-comments` Thanks! – Cord Kaldemeyer Dec 07 '22 at 12:59
  • Please do not add answers to the question body itself. Instead, you should add it as an answer. [Answering your own question is allowed and even encouraged](https://stackoverflow.com/help/self-answer). – Adriaan Dec 07 '22 at 13:04
  • Okay, but the linked answer does not exactly answer my question. Thus, I would remove the link and provide a proper answer here. – Cord Kaldemeyer Dec 07 '22 at 14:10

3 Answers3

0

Open the package in just about any IDE. The unused imports should be highlighted.

Alternatively, search through the package for any occurrences of the imported packages' names. The ones with no occurrences are useless.

FirTreeMan
  • 11
  • 1
  • 4
  • Of course, this is possible with a linter. But not with more than 20 dependencies over hundreds of code files which would take ages on every update of the package requirements. Thanks anyway. – Cord Kaldemeyer Dec 06 '22 at 19:40
  • @CordKaldemeyer I think you should write a script to parse through all your files and find any occurrences of your imports. Seems like the best solution in this case. – FirTreeMan Dec 06 '22 at 19:44
0

I would:

1. create a venv, empty, without global packages.

2. pip install bloater

3. pip freeze | tee alldeps.txt

remove bloater from alldeps.txt

4. grep who uses what using bash:

(I'd use ripgrep for that, easier recursion to understand than grep)

I stole the loop code from this answer (I don't think I've looped on file contents before).

touch imported.txt
while read package; do
   printf "found:$package\n" | tee -a imported.txt
   rg -t py -l $package | tee -a imported.txt
done <alldeps.txt

Note: rg -t py -l "$package.+import" and rg -t py -l "import.+$package", together, might be better at only seeing imports.

Anything without actual python files in imported.txt can be put into 2nd file, todelete.txt

5 loop deletion
touch uninstall.log
while read package; do
   pip uninstall $package | tee -a uninstall.log
done <todelete.txt

6 run unit tests and/or use package

One problem I see with this approach is the unclear relationship between package names and import names.

for example:

% pip list | egrep -i yaml
PyYAML                        6.0

but

from yaml import dump
JL Peyret
  • 10,917
  • 2
  • 54
  • 73
-1

Pip Check Command – Check Python Dependencies After Installation. Because pip doesn't currently address dependency issues on installation, the pip check command option can be used to verify that dependencies have been installed properly in your project. For example: $ pip check No broken requirements found.

Virat Mehar
  • 26
  • 2