1

I'm trying to iterate over a list of packages in a text file to get the version of each package. The list in the pkgs.txt file appears like this:

{
"package1",
"package2",
...
}

Here is my most recent code:

with open("pkgs.txt", "r") as pkgs:
    for line in pkgs:
        version = subprocess.check_call([sys.executable, '-m', 'pip', 'search', line])

with open("versions.txt", "w+") as versions:
    for ver in version:
        version.write(ver)

The error I'm getting is: CalledProcessError: Command '['/opt/conda/bin/python', '-m', 'pip, 'search', '{\n']' returned non-zero exit status 23.

Could the issue be that I first need to remove the quotation marks and commas before I can loop through this list?

sinoroc
  • 18,409
  • 2
  • 39
  • 70
Tgoyo
  • 13
  • 3
  • There is no `pip.main`, [_pip_ doesn't have an _API_](https://pip.pypa.io/en/stable/user_guide/#using-pip-from-your-program), one should do something like `subprocess.check_call([sys.executable, '-m', 'pip', 'search', line])`. – sinoroc Jul 26 '20 at 21:24
  • pip.main seems to work when I type in a package name manually. I'm just trying to find out how to get the names from an existing file instead. – Tgoyo Jul 26 '20 at 21:45
  • I see you might have an older version of _pip_ where `pip.main` still existed. The advice still stands, no matter what version of _pip_ you have. -- For the rest, I haven't looked closely, seems quite straightforward. What are you stuck on exactly? – sinoroc Jul 26 '20 at 22:11
  • Thank you, I've tried the subprocess code but I'm getting an error 'returned non-zero exit status 23'. Perhaps there is something wrong with my code but I'm not sure as I'm still quite new to python. – Tgoyo Jul 26 '20 at 22:42
  • Does this answer your question? [How to specify install order for python pip?](https://stackoverflow.com/questions/5394356/how-to-specify-install-order-for-python-pip) – phd Jul 26 '20 at 22:42
  • https://stackoverflow.com/search?q=%5Bpip%5D+install+packages+one+by+one – phd Jul 26 '20 at 22:43
  • Not quite, but thank you for the suggestion. I'll look through the other posts in the tag – Tgoyo Jul 26 '20 at 22:51
  • You haven't clarified which part is giving you trouble? Is it about reading a text file? Reading line by line? What are you stuck on? Which parts do you have under control? You have to give details about what you did, what you saw, what your current status is, etc. Does the code in the question show the current state of your research? – sinoroc Jul 27 '20 at 07:58
  • That's a much better question now after the edits, it's much easier to help you. I upvoted the question. Question about the input file: Is it a specific format? It doesn't seem to be JSON, CSV, or anything I can identify. – sinoroc Jul 27 '20 at 10:46
  • Thank you for your edits and upvote. It's a list in a text file (sorry, I don't have much knowledge beyond that). It's the output from a previous subprocess command that used wget to get the list of packages. – Tgoyo Jul 27 '20 at 10:57

1 Answers1

0

(There are too many aspects in this question to cover it all at once. I will try to cover the initial points here, the rest should be asked in a new question.)

First, pip doesn't have an API. The preferred way of using pip from your own program is to call it directly as a subprocess.

The input format is nothing recognizable (not CSV, JSON, maybe YAML could cover it). But coincidentally the input format is somewhat readable as a literal Python list of strings, so one could try parsing the content with ast.literal_eval.

pkgs.txt:

{
"package1",
"package2",
}

main.py

#!/usr/bin/env python3

import ast
import pathlib
import subprocess
import sys

def main():
    file_name = 'pkgs.txt'
    file_path = pathlib.Path(file_name)
    file_content = file_path.read_text()
    pkgs = ast.literal_eval(file_content)
    print("pkgs", pkgs)
    for pkg in pkgs:
        print("pkg", pkg)
        command = [
            sys.executable,
            '-m',
            'pip',
            'search',
            pkg,
        ]
        print("command", command)
        command_output = subprocess.check_output(command)
        print("command_output")
        print('"""')
        print(command_output.decode())
        print('"""')

if __name__ == '__main__':
    main()

Extracting the correct information out of command_result could be treated in a follow up question. (I suspect calling pip search is not the best choice here, it might be worth going with one of PyPI's APIs directly instead.)

command_output
"""
package2 (0.0.0)                      - 
101703383-python-package2 (0.0.3)     - This is my second package in python.
Vigenere-cipher-package2 (0.5)        - An example of Vigenere cipher
WuFeiLiuGuang-first-package2 (2.0.0)  - test pkg ignore

"""

References:

sinoroc
  • 18,409
  • 2
  • 39
  • 70
  • Thank you for this, I'll try it out. And apologies as this may be a basic question but what do I do with main.py? Do I need to create this file? – Tgoyo Jul 27 '20 at 12:13
  • Yes, create a `main.py` file in the same directory as `pkgs.txt` and run `python3 main.py`. – sinoroc Jul 27 '20 at 12:51