0

I want to write a pandas dataframe to my s3 bucket with the following method:

df.to_parquet(s3_bucket_url)

However, this method will use my default AWS credentials. I have multiple profiles in my AWS credentials and I would like to know how to specify the one I want to use when I am calling the methode df.to_parquet.

Thanks

Bertrand
  • 103
  • 1
  • 2
  • 7
  • How are you generating s3_bucket_url? Is it just `s3://bucket/folder/file.gz`? See if the comments on [this answer](https://stackoverflow.com/a/53502445/271415) help. – jarmod Nov 17 '21 at 15:11
  • s3_bucket_url = 's3://bucketbbr/test' But I think that my problem comes from my jupyter notebook. I am using awsume to select the organization I am working in but then from jupyter notebook, df.to_parquet still want to use my default AWS profile. And by executing python script I don't have the problem any more. The question now is why do I have this problem form Jupyter notebook. For information I am working with Windows 10. – Bertrand Nov 19 '21 at 08:02

0 Answers0