2

I want to read from AWS Athena with polars. Is this possible? Before I used pandas:

import pandas as pd

pd.read_sql(SQL_STATMENT, conn)

I found this User Guide: https://pola-rs.github.io/polars-book/user-guide/howcani/io/read_db.html where Athena is not yet supported.

Florian
  • 21
  • 1
  • Hey @Florian welocme For your first question and future questions will like to give a recommendation Your initial question is clear, but if you want to provide more context, you could add information about what you have tried so far to read from AWS Athena with Polars and any errors or issues you encountered. Additionally, you could provide information about the specific use case or scenario you are working on and any relevant details about the Athena setup or data you are trying to read. This could help provide more specific recommendations or solutions. – Shane Warne Apr 19 '23 at 09:56
  • Hey @Florian welocme For your first question and future questions will like to give a recommendation as below Your initial question is clear, but if you want to provide more context, you could add information about what you have tried so far to read from AWS Athena with Polars and any errors or issues you encountered. Additionally, you could provide information about the specific use case or scenario you are working on and any relevant details about the Athena setup or data you are trying to read. This could help provide more specific recommendations or solutions. – Shane Warne Apr 19 '23 at 09:56

1 Answers1

2

The good news is that the doc that you linked isn't the full list of databases that are supported.

polars uses two database connection libraries (or engines):

  1. connectorx

  2. Apache Arrow adbc

The bad news is that neither of those seems to support Athena. For the time being your best bet is probably to continue to use pandas for athena queries and then use pl.from_pandas(...)

Dean MacGregor
  • 11,847
  • 9
  • 34
  • 72