3

When running an Extract/Load pipeline with Meltano, what's the best way (or ways) to kill a running job?

Generally these would be run via Airflow but would be nice to have a process that worked also with bare meltano elt and/or meltano run invocations from orphaned terminal sessions, which might not be able to be canceled simply by hitting Ctrl+C.

aaronsteers
  • 2,277
  • 2
  • 21
  • 38

1 Answers1

0

For future reference:

The workaround is to hit Ctrl+C while the job is running - you will see a message like Another 'dev:tap-postgres-to-target-snowflake:dev' pipeline is already running which started at 2023-03-20 18:34:49.615984. To ignore this check use the '--force' option.

You can wait for 5 minutes (stale detection time) before running again or you can run the command using the --force flag to omit the above message. The first meltano run after the stale detection time will automatically mark the job as failed.

josescuderoh
  • 65
  • 10