0

We have data dumped into S3 buckets and we are using these data to pull some reports in Quicksight some directly accessing s3 as datasource and for other reports, we used Athena to query S3.

At which point, one need to use Redshift? Is there any advantage of using Redshift over S3+Athena?

Raviteja Gubba
  • 1,006
  • 6
  • 9
  • 1
    Are you asking about Athena vs Redshift Spectrum or Redshift with the data stored natively? If it's the former, there are some [good answers here](https://stackoverflow.com/questions/50250114/athena-vs-redshift-spectrum). FWIW, if you are happy with Athena on S3 for your usecase, then I don't think it would make any sense to introduce the overhead of an entire RDBMS. – JNevill Feb 21 '22 at 15:04

1 Answers1

1

No you might be perfectly fine with just QuickSight, Athena and S3 - it will be relatively cheaper as well if you keep Redshift out of the equation. Athena is based on PRESTO and is pretty comprehensive in terms of functionality for most SQL reporting needs.

You would need Redshift if you approach or hit the QuickSight's SPICE limits and would still like your reports to be snappy and load quickly. From a data engineering side, if you need to update existing records it is easier to micro batch and update records in RedShift. With athena/s3 you also need to take care of optimising the storage format (use orc/parquet file formats, use partitions, not use small files etc...) - it is not rocket science but some people prefer paying for RedShift and not having to worry about that at all. In the end, RedShift will probably scale better when your data grows very large (into the petabyte scale). However, my suggestion would be to keep using Athena and follow it's best practices and only use RedShift if you anticipate huge growth and want to be sure that you can scale the underlying engine on demand (and, of course, are willing to pay extra for it).

darcoli
  • 411
  • 3
  • 5