0

In the apache beam documentation, it is mentioned that the watermarks for the pcollections are determined by the source. Considering pubsub as source, what is the logic that pubsub uses to derive the watermark. Any documentation around this to understand better.

Murli Krishnan
  • 35
  • 5

1 Answers1

1

To define the watermark, we need to focus on the aspect of late data. "Refer the link for pubsub logic below."

What is the watermark heuristic for PubsubIO running on GCD?

Yelleti Priyanka
  • 132
  • 4
  • This is really interesting link and exactly what I had been looking for. Thanks a lot. Adding the link from the post provided - https://cdn.oreillystatic.com/en/assets/1/event/155/Watermarks_%20Time%20and%20progress%20in%20streaming%20dataflow%20and%20beyond%20Presentation.pdf – Murli Krishnan May 07 '21 at 12:23