6

In the Dataflow streaming world.

My understanding when I say:

Window.into(FixedWindows.of(Duration.standardHours(1)))
  .triggering(AfterProcessingTime.pastFirstElementInPane()
      .plusDelayOf(Duration.standardMinutes(15))

is that for a fixed window of one hour, the trigger waits or batches the elements after it has seen the first element.

But when I say:

Window.into(FixedWindows.of(Duration.standardHours(1)))
  .triggering(AfterProcessingTime.pastFirstElementInPane()

Does it fire every time from the first time it sees the first element or does it implicitly batch elements? because firing on every element overloads the system.

Anil Muppalla
  • 416
  • 4
  • 14

1 Answers1

6

With both of those triggers, the window will be fired once, and any remaining elements will be discarded. You can use Repeatedly.forever(...) to trigger multiple times.

Regarding your specific question, there is a small amount of batching that happens if elements arrive around the same time.

Assuming you meant the following, then yes, the second one will trigger much more often, and may overload the system.

Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane()
    .plusDelayOf(Duration.standardMinutes(15)))

vs.

Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane())
Ben Chambers
  • 6,070
  • 11
  • 16
  • If it's going to fire once, what is the benefit of using .plusDelayOf() ? – Anil Muppalla May 10 '17 at 22:40
  • Also, I do not understand the answer. If both the triggers are going to be fired once, then how will the second one fire very frequently? am I missing something. – Anil Muppalla May 10 '17 at 22:55
  • `plusDelayOf()` will fire once after 15 minutes, while the other will fire once approximately as soon as possible. – Ben Chambers May 10 '17 at 23:32
  • Clarified the answer -- either they will both only trigger once (in which case neither will overload the system) or (if you meant to use repeatedly) then the second will trigger much more frequently, and may overload the system in some cases. – Ben Chambers May 10 '17 at 23:35