1

It is somewhat similar to this question: testing flink jobs with MiniCluster to trigger the timer using processing time

I have written a Flink jobs graph and I defined a timer in KeyedProcessStream. Now I am trying to write an end to end test and trying to find a way to trigger the onTimer function during end to end test

I have added some unit test with testHarness mentioned in here: https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/datastream/testing/

But I am having trouble to verify the timer behavior in end to end tests. It seems like MiniClusterWithClientResource should be something I can use. But I don't find there are any interface for advancing watermarks or timer.

In the flink doc, it mentioned "You can implement a custom parallel source function for emitting watermarks if your job uses event time timers." If anyone could help elaborate on this that would also be helpful!

Another approaches I think can be a work around, would be: if we could somehow manually trigger onTimer in the KeyedProcessFunction from outside, then I can manually call it to verify the behavior with different inputs. So the an alternative question would be how can we call onTimer with MiniClusterWithClientResource.

Edit1: As am I am exploring my options, I think the line "You can implement a custom parallel source function for emitting watermarks if your job uses event time timers." mentioned in Flink test doc, means I need to have a source that extends RichParallelSourceFunction and advance water marks in the function. Now I wonder how would I integrate the parallel source function to my existing graph? If someone happen to have a related example that would be very helpful!

user190245
  • 1,027
  • 1
  • 15
  • 31
Xuhui Ding
  • 11
  • 3

0 Answers0