It is somewhat similar to this question: testing flink jobs with MiniCluster to trigger the timer using processing time
I have written a Flink jobs graph and I defined a timer in KeyedProcessStream. Now I am trying to write an end to end test and trying to find a way to trigger the onTimer function during end to end test
I have added some unit test with testHarness mentioned in here: https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/datastream/testing/
But I am having trouble to verify the timer behavior in end to end tests. It seems like MiniClusterWithClientResource
should be something I can use. But I don't find there are any interface for advancing watermarks or timer.
In the flink doc, it mentioned "You can implement a custom parallel source function for emitting watermarks if your job uses event time timers." If anyone could help elaborate on this that would also be helpful!
Another approaches I think can be a work around, would be: if we could somehow manually trigger onTimer in the KeyedProcessFunction from outside, then I can manually call it to verify the behavior with different inputs. So the an alternative question would be how can we call onTimer with MiniClusterWithClientResource
.
Edit1: As am I am exploring my options, I think the line "You can implement a custom parallel source function for emitting watermarks if your job uses event time timers." mentioned in Flink test doc, means I need to have a source that extends RichParallelSourceFunction and advance water marks in the function. Now I wonder how would I integrate the parallel source function to my existing graph? If someone happen to have a related example that would be very helpful!