Watch Kubernetes job output until completion

Question

Is there a way to watch the job's output (STDOUT and STDERR) using kubectl? We need to wait the job completion watching its output and, if the job finishes with error, the entire process should be interrupted.

I'd like to redirect the job output (STDOUT and STDERR) to my current process STDOUT. I want to wait the job completion. In case it finished with error, the current process (which triggered the job via kubectl) should finish in error as well. I know kubectl wait but, as far as I know, it does not support listening to the jobs output.

Can you explain what you mean by "watching the job output"? What conditions on the output should it wait for? Why not just have the error condition fail the process so the job fails? In that case it wouldn't be different from https://stackoverflow.com/questions/44686568/tell-when-job-is-complete. — Andy Shinn, Jun 24 '19 at 21:12
I'd like to redirect the job output (STDOUT and STDERR) to my current process STDOUT. I want to wait the job completion. In case it finished with error, the current process (which triggered the job via kubectl) should finish in error as well. As I mention in the question, `kubectl wait` does not support redirecting the job output — viniciusjssouza, Jun 24 '19 at 21:35
OK, maybe edit the question to clarify that. It was not apparent to me that you wanted to pipe the output from a calling program. — Andy Shinn, Jun 24 '19 at 21:38

viniciusjssouza · Accepted Answer · 2021-02-23T02:13:56.743

13

We ended up using three commands to accomplish the task. First, we delete the old job (it might have ran before), create the new job, wait it for completion (with a timeout) and, after it has finished, we print the logs:

kubectl delete job my-job || true
kubectl apply -f ./jobs/my-job.yaml
kubectl wait --for=condition=complete job/my-job --timeout=60s
echo "Job output:"
kubectl logs job/my-job

edited Feb 23 '21 at 02:13

answered Jun 28 '19 at 12:43

viniciusjssouza

1,235
14
28

2

Two issues with this solution: 1. The job may not be complete in 60s and in that case the log printed is partial 2. Then you may think that just removing the timeout would solve the issue... however that is not true because a job may fail and in that case it is never `complete` (condition never triggered and process stuck) – collimarco Nov 10 '21 at 16:23

score 5 · Answer 2 · answered Jun 25 '19 at 09:22

There are few options you can try:

View the job logs in real time:
- kubectl logs --follow $POD

If you are having trouble locating the proper pod than first go kubectl describe job $JOB Pod name will be under the "Events"

There are few more kubectl commands that can be useful here:
- kubectl logs -f my-pod - stream pod logs (stdout)
- kubectl logs -f my-pod -c my-container - stream pod container logs (stdout, multi-container case)
- kubectl logs -f -l name=myLabel --all-containers - stream all pods logs with label name=myLabel (stdout)

More about that can be found here

Try again with kubectl wait but with proper flags like --for=condition=xxxxx A successful message will be printed to stdout indicating when the specified condition has been met.

Watch Kubernetes job output until completion

2 Answers2