0

I want to run the following Bash command in Python:

import subprocess

command = 'tar -vcz -C /mnt/20TB_raid1  Illumina_Sequencing_Data | pigz | tee >(md5sum > "md5sum_Illumina_Sequencing_Data.txt") | split -b 50G - /mnt/4TB_SSD/Illumina_Sequencing_Data.tar.gz.part-'

subprocess.run(command.split(" "))

When I run this code I get the following error:

tar: 50G: Invalid blocking factor
Try 'tar --help' or 'tar --usage' for more information.

However, when I run this command in the shell directly it run without any issues. It looks like some code is ignored as it thinks the split -b flag is parsed to tar?

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
justinian482
  • 845
  • 2
  • 10
  • 18
  • 1
    Instead of running `tar`, try writing a separate Python script that just outputs its `sys.argv` values, and then try to use it in a similar way; the problem will immediately be apparent. *Everything* is passed to `tar` with this approach; for example, the `|` symbols become arguments by themselves, rather than representing redirections. See the linked duplicate for details. – Karl Knechtel Jul 24 '23 at 09:10

1 Answers1

2

It's the piping that causes the problem.

By default the run function runs a single command directly. It does not use a shell to run the command. And piping is a shell thing, it's handled all by the shell itself. Same with redirection and any kind of substitution.

Furthermore, since you split the command in Python, you will pass the pipe signs and all the piped commands as actual arguments to the tar command.

You need to invoke a shell if you want to use piping.

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621