2

I have a folder called myfolder containing multiple files with filename as follows,

ID001_2017-04-15.csv, ID002_2017-04-15.csv, ID001_2017-04-16.csv, ID002_2017-04-16.csv, 
ID001_2017-04-17.csv, ID002_2017-04-17.csv, ID001_2017-04-18.csv, ID002_2017-04-18.csv

The date on the filename is the date the file created. For example, file ID001_2017-04-17.csvis created on 2017-04-17. Here's how I uploaded all the files in a folder to Amazon S3, 

import boto3

def upload_files(path):
    session = boto3.Session(
              aws_access_key_id = 'this is my access key',
              aws_secret_access_key = 'this is my secret key',
              region_name = 'this is my region'
              )
    s3 = session.resource('s3')
    bucket = s3.Bucket('this is my bucket')

    for subdir, dirs, files in os.walk(path):
        for file in files:
            full_path = os.path.join(subdir, file)
            with open(full_path, 'rb') as data:
                bucket.put_object(Key = full_path[len(path) + 1:], Body = data)

if __name__ == "__main__":
    upload_files('path to myfolder') ## Replace this with your folder directory

My question is can I only upload files that are created today to Amazon S3?

Peggy
  • 143
  • 3
  • 14
  • Take a look at http://stackoverflow.com/questions/5141437/filtering-os-walk-dirs-and-files - and filter on todays date. – stdunbar Apr 25 '17 at 19:02
  • 1
    If your intention is to sync files from a local directory to S3, you could use the [AWS Command-Line Interface (CLI)](http://aws.amazon.com/cli/) which has a `aws s3 sync` command. Much easier than writing your own code. – John Rotenstein Apr 26 '17 at 00:27
  • @JohnRotenstein Thank you. Yes, I want to sync files from local directory to S3. Is it possible just to sync files generated by today to S3 using CLI? – Peggy Apr 26 '17 at 15:08
  • 1
    Why do you only want to sync "today's files"? Is it because you upload the files every day and just want to send the new files that weren't previously uploaded? If so, simply use `aws s3 sync` -- it only copies files that a new or changed since the last sync. – John Rotenstein Apr 26 '17 at 21:20
  • @JohnRotenstein Ok. I got it. Thank you very much! – Peggy Apr 27 '17 at 02:53

1 Answers1

1

This will check if a file was created today:

import os.path
import datetime.datetime

# Create a datetime object for right now:
now = datetime.datetime.now()
# Create a datetime object for the file timestamp:
ctime = os.path.getctime('example.txt')
filetime = datetime.datetime.fromtimestamp(ctime)

# Check if they're the same day:
if filetime.year == now.year and filetime.month == now.month and filetime.day = now.day:
    print('File was created today')

If you put something like that in your for file in files: loop, you should be able to isolate the files created today.

dogoncouch
  • 251
  • 3
  • 9