0

I would like to upload and download data from AWS S3 bucket using python code. The files are present under a folder in AWS S3 location.

Example : Bucket - aws-xxx-use1-00-d-s3b-xxx-yyy-xxx Folder - Execution/Data/

Please share your thoughts on it. TIA

saurabh bhattacharya
  • 39
  • 4
  • 3
    Does this answer your question? [Download a folder from S3 using Boto3](https://stackoverflow.com/questions/49772151/download-a-folder-from-s3-using-boto3) also [upload a directory to s3 with boto](https://stackoverflow.com/questions/25380774/upload-a-directory-to-s3-with-boto) – Abdul Aziz Barkat Feb 14 '23 at 05:00
  • did you try reading boto docs? – Jatin Mehrotra Feb 14 '23 at 06:01

1 Answers1

0

I hope, it will give you a better understanding of uploading, downloading and getting an s3 object

# libraries import
import boto3
import pandas as pd

# configure aws on command line
# aws configure
# enter your access key, access secret and region in which you create your bucket
# You get your access key and access password from security credentials under the iam
# if its disabled delete the previous one and then create the new one

s3_obj = boto3.client("s3") # object for accessing s3

# Downloading file code
s3_obj.download_file(
    Filename="./local_file_name_which_you_download_from_s3.csv",
    Bucket="your_bucket_name",
    Key="file_in_s3_bucket.csv"
# )
df = pd.read_csv('./local_file_name_which_you_download_from_s3.csv')
df.head()

# Uploading file code
s3_obj.upload_file(
    Filename="./local_file_name_which_you_download_from_s3.csv",
    Bucket="your_bucket_name",
    Key="file_in_s3_bucket.csv"
)

s3_obj_file = s3_obj.get_object(Bucket='bucket_name', Key='file_in_s3_bucket')['Body'].read()
df_upload = pd.read_excel(s3_obj_file)
df_upload.head()
Muhammad Ali
  • 444
  • 7