11

I'm using S3 select to query a object from a file in S3 bucket but wondering if I can do it with multiple files or if its limited to just one. I wrote one query from a file using aws lambda but when I tried writing another one for a different file it gave me a error message.

Josh Lee
  • 171,072
  • 38
  • 269
  • 275
runner05
  • 123
  • 2
  • 5
  • I am having similar issue. I am capturing incoming messages from a temperatures sensor through AWS IoT in to an s3 bucket. Each s3 object is a new temperature reading. Some thing like ```aws s3api select-object-content --bucket temperatures --key '$aws/things/sensor/shadow/update/accepted/2018-07-13/1531514034796' --expression 'S ELECT s.desired.temperature from S3Object s ' --expression-type SQL --input-serialization '{"JSON":{"Type": "document"}}' --output-serialization '{"CSV":{}}' test --profile myprofile``` works since I specify the one specific key. – geoaxis Jul 13 '18 at 21:43
  • Also see https://stackoverflow.com/questions/50721377/can-s3-select-search-multiple-objects – geoaxis Jul 13 '18 at 21:48
  • 1
    Possible duplicate of [Can S3 Select search multiple objects?](https://stackoverflow.com/questions/50721377/can-s3-select-search-multiple-objects) – moebius Aug 30 '18 at 23:46

1 Answers1

7

Amazon S3 Select operates on only one object.

To query multiple files at once on S3 with a SQL syntax you should use Amazon Athena.

xavierraffin
  • 189
  • 2
  • 8