0

I am trying to understand AWS Kinesis quotas and am looking to get some clarification on the following:

If I have a single Kinesis consumers(same appID) running in multiple containers/EC2 instances etc to try and improve data consumption via reading in parallel, how do the existing kinesis quotas listed below work?

https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html GetRecords can retrieve up to 10 MB of data per call from a single shard, and up to 10,000 records per call. Each call to GetRecords is counted as one read transaction.

Each shard can support up to five read transactions per second. Each read transaction can provide up to 10,000 records with an upper quota of 10 MB per transaction.

Each shard can support up to a maximum total data read rate of 2 MB per second via GetRecords. If a call to GetRecords returns 10 MB, subsequent calls made within the next 5 seconds throw an exception.

Question: Does this mean that even if we run multiple consumers(same AppID) instances, the number of reads are capped to 5 read transaction/sec/shard etc.

user1132593
  • 448
  • 1
  • 5
  • 8
  • https://stackoverflow.com/questions/63753139/how-are-shards-from-a-kinesis-stream-assigned-to-multiple-instances-of-a-kinesis has a good explanation. – user1132593 Feb 01 '23 at 21:28

0 Answers0