3

So my problem is sometimes my Glue job takes a lot of time to run or even fails all together because of some inexplicable issues. Is there a way to get messages printed out to the output or error logs so that I can at least track where the job is right now or was at the end?

I tried the solution in How do I write messages to the output log on AWS Glue?

But that does not seem to be working for me. The Output log generates a ton of logs and even when I search or filter for keywords in my message, I am still not able to find them.

Right now I am considering writing messages to a S3 file to get around this issue. But it got me wondering if there isn't actually any other way to display messages that I can check while the job is running?

pallupz
  • 793
  • 3
  • 9
  • 25

1 Answers1

0

You can edit the job to use the Continuous logging feature and keep track of your loggers there

mintchip36
  • 171
  • 1
  • 1
  • 5