0

So this question is following on from this (Read comments aswell as that is the path I took). I just want the first call robocopy to finish executing before moving on to the rest of the code. As I want the second robocopy to just skip all of the files as they have already been copied over. However what is happening is that the rest of the script will run (ie starts the second robocopy) while the first robocopy is copying over the files. Below is the code:

call(["start", "cmd", "/K", "RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z", "/MT:8"], stdout=PIPE, shell=True) 
temp2 = Popen(["RoboCopy.exe", f"{self.srcEntry.get()}", f"{self.dstEntry.get()}", "*.*", "/E", "/Z"], stdout=PIPE, stdin=PIPE, shell=True)

EDIT 1:

Issue is noticeable when copying over large files. I'm thinking about putting in a sleep function which is dependent upon the total size of the files to be copied over. However this doesn't take into account upload/download speeds as the files will be transferred over a network.

Manib
  • 171
  • 15
  • You can use `subprocess.run` instead of Popen. [Link](https://stackoverflow.com/questions/39187886/what-is-the-difference-between-subprocess-popen-and-subprocess-run) – QuantumChris Feb 14 '20 at 10:29
  • Trying to implement the subprocess.run however when I'm running RoboCopy I give the following as source path in tkinter C:\Users\mnazir\Downloads\R-test and it produces this as the source path in the terminal C:\Users\mnazir\Desktop\Shortcuts 64bit\UsersmnazirDownloadsR-test\ – Manib Feb 14 '20 at 11:46
  • Figured the issue I had with the run command but it still doesn't resolve my issue. Once the last file has started to be copied over the second Robocopy seems to start. If I use the MT option in Robocopy then it pretty much starts the second Robocopy shortly after the first. – Manib Feb 14 '20 at 12:31
  • Does robocopy automatically return the terminal on use? That might trick your program into thinking the operation has completed when it hasn't – QuantumChris Feb 14 '20 at 13:00
  • Not too sure what you mean by "return the terminal". – Manib Feb 14 '20 at 14:20
  • Ok, so when I use subprocess.run it does continue on with the script although robocopy has not finished running. – Manib Feb 14 '20 at 14:23
  • I was thinking whether I should add an additional file into the directory to be copied over. Then I can make a function which checks that all of the files have been copied over into the destination folder before running the second robocopy (for some reason the last file will get copied over by the second robocopy as my program finds the file as it's being copied over by the first robocopy but the second robocopy doesn't). So if the last file is a small empty file then it should resolve the issue. What do you think? – Manib Feb 14 '20 at 14:36

3 Answers3

1

Try:

while temp2.poll() is not None:
    # ... do something else, sleep, etc

out, err = temp2.communicate()
Matt Shin
  • 424
  • 2
  • 7
0

I use the following function to launch my commands which waits until the action has finished, but has a timeout:

import os
import logging

logger = logging.getLogger()

def command_runner(command, valid_exit_codes=None, timeout=30, shell=False, decoder='utf-8'):
    """
    command_runner 2019103101
    Whenever we can, we need to avoid shell=True in order to preseve better security
    Runs system command, returns exit code and stdout/stderr output, and logs output on error
    valid_exit_codes is a list of codes that don't trigger an error
    """

    try:
        # universal_newlines=True makes netstat command fail under windows
        # timeout does not work under Python 2.7 with subprocess32 < 3.5
        # decoder may be unicode_escape for dos commands or utf-8 for powershell
        if sys.version_info >= (3, 0):
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             timeout=timeout, universal_newlines=False)
        else:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                             universal_newlines=False)
        output = output.decode(decoder, errors='backslashreplace')
    except subprocess.CalledProcessError as exc:
        exit_code = exc.returncode
        # noinspection PyBroadException
        try:
            output = exc.output
            try:
                output = output.decode(decoder, errors='backslashreplace')
            except Exception as subexc:
                logger.debug(subexc, exc_info=True)
                logger.debug('Cannot properly decode error. Text is %s' % output)
        except Exception:
            output = "command_runner: Could not obtain output from command."
        if exit_code in valid_exit_codes if valid_exit_codes is not None else [0]:
            logger.debug('Command [%s] returned with exit code [%s]. Command output was:' % (command, exit_code))
            if output:
                logger.debug(output)
            return exc.returncode, output
        else:
            logger.error('Command [%s] failed with exit code [%s]. Command output was:' %
                         (command, exc.returncode))
            logger.error(output)
            return exc.returncode, output
    # OSError if not a valid executable
    except OSError as exc:
        logger.error('Command [%s] faild because of OS [%s].' % (command, exc))
        return None, exc
    except subprocess.TimeoutExpired:
        logger.error('Timeout [%s seconds] expired for command [%s] execution.' % (timeout, command))
        return None, 'Timeout of %s seconds expired.' % timeout
    except Exception as exc:
        logger.error('Command [%s] failed for unknown reasons [%s].' % (command, exc))
        logger.debug('Error:', exc_info=True)
        return None, exc
    else:
        logger.debug('Command [%s] returned with exit code [0]. Command output was:' % command)
        if output:
            logger.debug(output)
        return 0, output


# YOUR CODE HERE

executable = os.path.join(os.environ['SYSTEMROOT'], 'system32', 'robocopy.exe')
mycommand = '"%s" "%s" "%s" "%s"' % (executable, source, dest, options)
result, output = command_runner(mycommand, shell=True)
Orsiris de Jong
  • 2,819
  • 1
  • 26
  • 48
  • I don't quite understand the code. which part of it is ensuring that the robocopy will finish running before the second robocopy starts? – Manib Feb 14 '20 at 11:31
  • That's subprocess.check_output which waits for the command to finish unless timeout is reached. – Orsiris de Jong Feb 20 '20 at 15:36
0

What I've found out: Thanks to QuantumChris. I've found out that robocopy returns from the terminal and back into my script although I've used subprocess.run which should have paused my script until it had finished running. I'm stalling the second robocopy from running by checking if the files have been copied over to the destination folder before progressing with the second robocopy. The issue is that if the last file is big then os.path.isfile() detects the file WHILE it is still being copied over. So it engages the second robocopy however the second robocopy doesn't detect that last file and so attempts to copy the file over but recognises that it can't access the file as it is already in use (by the first robocopy) so it waits 30 secs before trying again. After the 30 secs it is able to access the file and copies it over. What I would like to do now is to make my last file an empty dummy file which I don't care about it being copied twice as it is empty. Robocopy seems to copy over the files according to ASCII order. So I've named the file ~~~~~.txt :D

Manib
  • 171
  • 15