1

My requirement is to create separate tables for each key in two different data streams and then join them. I have successfully created two separate tables from the data streams in Flink and performed the join using table api. However, I am facing difficulty in creating separate tables for each key.

Is there a way to create individual tables for each key in the data stream?

I have followed tutorial to create tables from data streams, but I'm not sure how to proceed with creating separate tables for each key. Any guidance or suggestions would be greatly appreciated.

Edit: I am also interested in receiving suggestions and learning about best practices for managing multi-tenancy in Flink.

Vinay Cheguri
  • 55
  • 7
  • Do you have a fixed set of keys in advance, or is the key space dynamic? How many distinct keys might there be? – David Anderson May 11 '23 at 18:12
  • Can you explain why you need separate, per-key tables? Perhaps there's a better way to achieve the ultimate goal. – David Anderson May 11 '23 at 18:13
  • key space is dynamic and it can be around a hundred. The use case is I'm doing some ETL processing by doing keyBy using the tenant. So, I need a separate table for each tenant. – Vinay Cheguri May 12 '23 at 01:58
  • Any suggestions on how to handle multi-tenancy in Flink while doing ETL processing? Which is suggested, running a separate job for each tenant or doing keyBy operation? – Vinay Cheguri May 12 '23 at 06:10
  • I don’t think there is one best answer to this question. There are lots of factors to consider. – David Anderson May 14 '23 at 19:03

0 Answers0