1

So I'm getting into Python and I'm writing a script to:

  1. Download an RPM using urllib.urlretrieve.
  2. Extract the files using rpm2cpio and cpio.
  3. Do some stuff with the files.
  4. Clean up using shutil.rmtree.

Functionally it's all good it does do all of that but since I put in the clean up code I'm getting the following output:

rpm2cpio: MyRPM.rpm: No such file or directory
cpio: premature end of archive

Here is the code:

#!/usr/bin/python

from contextlib import contextmanager
import os, subprocess, shutil

@contextmanager
def cd(directory):
    startingDirectory = os.getcwd()
    os.chdir(os.path.expanduser(directory))
    try:
        yield
    finally:
        os.chdir(startingDirectory)

# Extract the files from the RPM to the temp directory
with cd("/tempdir"):
    rpm2cpio = subprocess.Popen(["rpm2cpio", "MyRPM.rpm"], stdout=subprocess.PIPE)
    cpio = subprocess.Popen(["cpio", "-idm", "--quiet"], stdin=rpm2cpio.stdout, stdout=None)

# Do
# Some
# Things
# Involving
# Shenanigans

# Remove the temp directory and all it's contents
shutil.rmtree("/tempdir")

If you see some syntax issues with the code here (or missing imports or something) please ignore unless it actually pertains to the reason I'm getting the two messages. I tried to strip down the script to the releveant bits. All I'm looking for is an explanation as to why the above two messages are being printed. I'd have assumed that the script is executed from the top down but now I'm thinking I may be wrong in this case?

EDIT: It feels like the 'rpm2cpio' and 'cpio' commands are leaving something open as long as the script is running like something I need to explicitly close...? Does that make any sense? :)

Thanks! J

Jay
  • 456
  • 4
  • 11
  • 1) Is that an absolute path to a folder off the root? One will usually create temporary directories and files _inside_ of `/tmp`. 2) Did you create this temporary directory? At least what you've shown suggests it doesn't exist. 3) I don't see that you have placed `MyRPM.rpm` in the temporary directory. One or all of these very well might account for `rpm2cpio` not finding your RPM. – hunteke Nov 01 '17 at 14:58
  • @hunteke 1) It's an absolute path yes currently I'm just creating a directory beside the script and putting the RPM in there. I will move to /tmp eventually thanks for the advice 2) As I said functionally everything is working so yes I am creating the temp directory and successfully extracting the RPM files inside it 3) Same as 2, thanks – Jay Nov 01 '17 at 15:13
  • 1
    Is your script properly waiting until `rpm2cpio` and `cpio` finish their work? If your script removes the temporary directory before they've had a chance to do their work, that would account for those messages. – hunteke Nov 01 '17 at 15:21
  • @hunteke It does look like that yes but there is a substantial amount of code (represented here by "shenanigans") that's processing the extracted files without issue in between the cpio commands and the delete command. That's what I meant by my assumption that the script is run from the top down.. feels like it isn't! :) – Jay Nov 01 '17 at 15:29
  • Sorry mate, I'm not seeing the issue, without more context. Perhaps put an example up that showcases the issue so others can try it locally? – hunteke Nov 01 '17 at 15:31
  • @hunteke no problem I didn't include the bits where I create the temp directory and download the RPM into it because I don't think it's relevant. I mean everything works fine it's just this weird output. If I comment out the last line (shutil) I don't get that output (and of course the directory is not deleted). It feels like the 'rpm2cpio' and 'cpio' commands never actually return but then why would the subsequent code be getting executed? I'm wondering could there be something left open by those commands? – Jay Nov 01 '17 at 16:08
  • @hunteke You were right :) I put in a 30 second sleep before the call to the shutil.rmtree and I don't get the 2 messages from rpm2cpio and cpio. – Jay Nov 01 '17 at 16:25

1 Answers1

0

subprocess.Popen is non-blocking, so you basically have a race condition - between your calls to Popen and rmtree there is no guarantee that those processes can finish (or even start!) before rmtree runs.

I suggest you wait for the Popen objects to return with 

cpio.wait()
rpm2cpio.wait()

# Remove the temp directory and all it's contents
shutil.rmtree("/tempdir")

Using the blocking subprocess.call doesn't look like an option for you with how you're piping the commands.

datu-puti
  • 1,306
  • 14
  • 33
  • Awesome stuff thank you so much for your time and for the information. Also credit to @hunteke who helped me reach the conclusion that it was a timing issue. – Jay Nov 01 '17 at 16:56
  • 1
    Also just for further info it looks like cpio.wait() on it's own is working fine. – Jay Nov 01 '17 at 16:58