0

I have a KCL (2.0/Java) deployed on FARGATE reading multiple shards. Unfortunately I experience two issues right now I hope somebody can help me with:

  1. FARGATE does not allow graceful shutdown of JVM.
  2. Since 1. the checkpoint value in the LeaseTable on DynamoDB is not updated. A new instanced of the KCL application will reprocess all events in the given shard.

Regarding 1, I already found out that there are some logging issues after SIGTERM with FARGATE but since the shard checkpoints are not updated I assume there is general issue avoiding the shutdown routines to be run.

If, for 2, somebody know if there is a possibility to "manually" update, e..g every 5 min, the checkpoint value, that would at least reduce the number of repeatedly processed events.

GENERALIZATION: When does (except during shutdown) KCL update the lease table?

Thx, gapvision

gapvision
  • 999
  • 1
  • 10
  • 30

1 Answers1

0

So it turns out that

1) is in fact a docker "issue" and points to the ENTRYPOINT mode: SIGTERM not received by java process using 'docker stop' and the official java image

2) is a manual process, there is no internal triggering of the checkpoint update. One needs to explicitly call the checkpoint() method in order to update the pointer

gapvision
  • 999
  • 1
  • 10
  • 30