I am using amazon-kinesis-connectors to build a kinesis client side application. I am figuring out few things about it.
How KCL ensures load balancing and scaling. for example i have a stream with one shard and a lot of records/events are put for 15 mintues in whole day or for certain time. So how it will take care of such sudden traffic and load.
For load balancing, you need multiple shards. Kinesis attempts to balance the number of shards each worker takes on - if you only have one shard, you won't get any benefit from more than one worker. See my related answer here for a bit more information on how Kinesis load balancing works.
As for scaling - each shard has a limit to maximum amount of traffic it can take - from the kinesis limitation docs:
Each shard can support up to 1,000 records per second for writes, up to a maximum total data write rate of 1 MB per second
The way you scale up with kinesis is add more shards - more shards == more throughput. You can either have these shards pre-created, or you can add them as your traffic increases.
