1

I'm running airflow on a docker container on my local machine. I was looking around but I could not find anyone that may have a similar issue. I'm trying to save some results to a CSV and then into my local machine, but for some reason i keep getting issues in my DAG.

def astra_connect():
    query = "SELECT * FROM iac689.data_2 limit 5;"
    df = pd.DataFrame(list(session.execute(query)))
    df = df.drop(columns=['ui'])
    return df.to_csv('localpath/output.csv')

Any ideas what I should be looking out for? Maybe permissions, a specific path, configuration of Airflow/docker itself?

v_him
  • 33
  • 7
  • What kind of issues are you getting? – user2314737 Feb 07 '22 at 21:44
  • its giving me permission issues that i cant write a file in the directory – v_him Feb 07 '22 at 21:47
  • Is it a directory in the Docker container or on the local machine? – user2314737 Feb 07 '22 at 21:49
  • im trying to write to export the csv out into my local machine – v_him Feb 07 '22 at 21:50
  • You will need to define some "volumes" and allow docker to write to them. Of course one wouldn't want to make it easy for docker to write to your filesystem, it could be an issue with security! See this question: [How to write data to host file system from Docker container](https://stackoverflow.com/questions/31448821/how-to-write-data-to-host-file-system-from-docker-container) – user2314737 Feb 07 '22 at 22:51
  • If you post the error message maybe you can get help .. – user2314737 Feb 09 '22 at 08:22

0 Answers0