2

I am using python subprocess to run a mysqlimport command. For tracking the uploading progress, I am using a named pipe and pipe_viewer. Here is another similar problem.

The subprocess Pipes are running without any error, but mysqlimport command is not uploading data to the required table.

import subprocess, os, sys
os.mkfifo('named_pipe')
pv = subprocess.Popen(
    ["pv", "-f", "file.csv", ">", "named_pipe"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)
pro = subprocess.Popen(
["mysqlimport", "--ignore-lines=1", "--fields-terminated-by=\',\'", 
"--local", "-u", "root", "-pRoot@123", "database_name", named_pipe],
stdin=pv.stdout,
)

pv.stdout.close()

This is the simple mysqlimport command which is running fine and uploading data but without pipe_viewer, the progress can not be tracked:

pro = subprocess.Popen(
["mysqlimport", "--ignore-lines=1", "--fields-terminated-by=\',\'",
 "--local", "-u", "root", "-pRoot@123", "database_name", "file.csv"], 
stdout=subprocess.PIPE
)

Here is the terminal command using pipe_viewer and mysqlimport which is working fine:

$ mkfifo named_pipe

$ pv -f file.csv > named_pipe | mysqlimport --ignore-lines=1 --fields-terminated-by=',' --local -u root -pRoot@123 database_name named_pipe
Rohit Thaply
  • 31
  • 2

1 Answers1

1

To get the ">" redirection to work correctly, you most likely need to set "shell=True" to the Popen function.

Daid Braam
  • 173
  • 4
  • Hi Daid, I set shell=True in the first subprocess but still no progress. Also, this [comment](https://stackoverflow.com/a/53982050/12045568) tells otherwise. Please see if you can find any other loophole in the code. – Rohit Thaply Sep 12 '19 at 13:51