We're having an issue kicking off a subprocess.Popen
within an airflow operator. We're using the following code to kick off sqlcl:
import subprocess
cmd = '/usr/local/bin/sqlcl -V'
p = subprocess.Popen(
cmd, shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
executable='/bin/bash')
for line in iter(p.stdout.readline, ''):
self.log.info('%s', line)
p.wait()
# we have also tried p.communicate() and p.poll() here
The snippet above works when run from ipython but hangs with no output when run from within airflow. Any suggestions?