0

Please let me know if I am doing this part correctly. I am trying to grab files ONLY modified in the past 24 hours. However my output is ALL files in the directory regardless of modified time:

yesterday = date.today() - timedelta(days=1)
dayToStr = yesterday.strftime('%Y%m%d')

file_list_attr = sftp.listdir_attr()
for file in file_list_attr:
  if  file.st_mtime <= dayToStr:
    print file

Output

-rw-r--r-- 1 4012 60 3404961 09 Jan 18:32 2_YEAR_912828UD0_20130109.dat -rw-r--r-- 1 4012 60 10206411 09 Jan 18:32 3_YEAR_912828UG3_20130109.dat -rw-r--r-- 1 4012 60 68311760 09 Jan 18:34 5_YEAR_912828UE8_20130109.dat -rw-r--r-- 1 4012 60 54215712 09 Jan 18:35 7_YEAR_912828UF5_20130109.dat -rw-r--r-- 1 4012 60 88014103 09 Jan 18:37 10_YEAR_912828TY6_20130109.dat -rw-r--r-- 1 4012 60 53565072 09 Jan 18:38 30_YEAR_912810QY7_20130109.dat -rw-r--r-- 1 4012 60 8527412 04 Jan 18:31 2_YEAR_912828UD0_20130104.dat -rw-r--r-- 1 4012 60 21659138 04 Jan 18:31 3_YEAR_912828UC2_20130104.dat -rw-r--r-- 1 4012 60 91281894 04 Jan 18:34 5_YEAR_912828UE8_20130104.dat -rw-r--r-- 1 4012 60 80421507 04 Jan 18:36 7_YEAR_912828UF5_20130104.dat -rw-r--r-- 1 4012 60 108700356 04 Jan 18:38 10_YEAR_912828TY6_20130104.dat -rw-r--r-- 1 4012 60 50204292 04 Jan 18:39 30_YEAR_912810QY7_20130104.dat -rw-r--r-- 1 4012 60 2319656 07 Jan 18:24 2_YEAR_912828UD0_20130107.dat -rw-r--r-- 1 4012 60 6978760 07 Jan 18:24 3_YEAR_912828UC2_20130107.dat -rw-r--r-- 1 4012 60 53579177 07 Jan 18:25 5_YEAR_912828UE8_20130107.dat -rw-r--r-- 1 4012 60 46069381 07 Jan 18:26 7_YEAR_912828UF5_20130107.dat -rw-r--r-- 1 4012 60 70802355 07 Jan 18:28 10_YEAR_912828TY6_20130107.dat -rw-r--r-- 1 4012 60 43050822 07 Jan 18:29 30_YEAR_912810QY7_20130107.dat -rw-r--r-- 1 4012 60 2713906 08 Jan 18:31 2_YEAR_912828UD0_20130108.dat -rw-r--r-- 1 4012 60 8889264 08 Jan 18:31 3_YEAR_912828UC2_20130108.dat -rw-r--r-- 1 4012 60 63857903 08 Jan 18:32 5_YEAR_912828UE8_20130108.dat -rw-r--r-- 1 4012 60 55544096 08 Jan 18:34 7_YEAR_912828UF5_20130108.dat -rw-r--r-- 1 4012 60 89750161 08 Jan 18:36 10_YEAR_912828TY6_20130108.dat -rw-r--r-- 1 4012 60 59233399 08 Jan 18:37 30_YEAR_912810QY7_20130108.dat

Super_Py_Me
  • 169
  • 1
  • 4
  • 14
  • 1
    What is file.st_mtime? An integer? A string? I bet your comparing apples and oranges in terms of type. – John Szakmeister Jan 10 '13 at 20:08
  • In paramiko the following file attributes are available Representation of the attributes of a file (or proxied file) for SFTP in client or server mode. It attemps to mirror the object returned by os.stat as closely as possible, so it may have the following fields, with the same meanings as those returned by an os.stat object: st_size st_uid st_gid st_mode st_atime st_mtime – Super_Py_Me Jan 10 '13 at 20:31
  • @user1741381: Thank you for the detailed bug reports (below). They are helpful! However, it is frowned upon to put them in the answer section. Instead, please edit your original question to include the relevant bug details. – unutbu Jan 10 '13 at 21:05

4 Answers4

3

file.st_mtime is an integer timestamp. dayToStr is a string.

In Python2, integers always compare less than strings for the rather arbitrary reason that the i in int comes before the s in str alphabetically:

In [123]: 1234 < 'foobar'
Out[123]: True

In Python3, comparing an int to a str raises a TypeError:

>>> 1234 < 'foobar'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: int() < str()

Instead, compare datetime objects:

import datetime as DT
import os

yesterday = DT.datetime.now() - DT.timedelta(days=1)
# or, if you want 00:00 AM, yesterday:
# yesterday = DT.datetime.now().replace(hour = 0, minute = 0, second = 0, microsecond = 0) - DT.timedelta(days=1)

file_list_attr = sftp.listdir_attr()
for pfile in file_list_attr:
    if DT.datetime.fromtimestamp(pfile.st_mtime) > yesterday:
        print pfile

References:

  • datetime.fromtimestamp: This was used to convert the timestamp to a DT.datetime object.
  • datetime.replace: This was suggested for setting the hours, minutes, seconds (of yesterday) back to zero.
Community
  • 1
  • 1
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • @user1741381: In that case, you could use `DT.datetime.fromtimestamp` to convert the `file.st_mtime` timestamp into a datetime object. – unutbu Jan 10 '13 at 20:37
  • Above, I've changed `yesterday` to be a `DT.datetime` rather than a `DT.date` object so it can be compared with the `DT.datetime` object returned by `DT.datetime.fromtimestamp`. – unutbu Jan 10 '13 at 20:54
  • Thank you this worked flawlessly. I would like to better understand it though. I will certainly look it over – Super_Py_Me Jan 10 '13 at 21:10
0

Appears to fail when comparing to 'yesterday'

for pfile in file_list_attr:
    print DT.datetime.fromtimestamp(pfile.st_mtime)


2013-01-09 18:32:06
2013-01-09 18:32:22
2013-01-09 18:34:07
2013-01-09 18:35:27
2013-01-09 18:37:38

for pfile in file_list_attr:
    print DT.datetime.fromtimestamp(pfile.st_mtime) > yesterday



Traceback (most recent call last):
  File "<pyshell#41>", line 2, in <module>
    print DT.datetime.fromtimestamp(pfile.st_mtime) > yesterday
TypeError: can't compare datetime.datetime to datetime.date

Super_Py_Me
  • 169
  • 1
  • 4
  • 14
0

here's an example of how you can:

  1. list all the files in a directory

  2. Print all the files that meet the condition of been modified 24h ago

        # Task: grab files ONLY modified in the past 24 hours

    import os
    import datetime

    myPath = "/users/george/documents/"

    # Adding all the files found in myFolder in a collection
    fileCollection = os.listdir(myPath)

    # Iterating through the files, printing their last modified date
    for i in fileCollection:
        # Getting the timestamp in a variable
        fileModTimeStamp = os.path.getmtime(myPath + str(i))
        fileModDateTime = datetime.datetime.fromtimestamp(fileModTimeStamp)

        # Calculating the time delta
        currentTime = datetime.datetime.now()
        timeElapsed =  currentTime - fileModDateTime

        # 24h dimedelta
        twentyFourHours = datetime.datetime(1900, 1, 2, 0, 0, 0, 0) -  datetime.datetime(1900, 1, 1, 0, 0, 0, 0)

        # Print the files that meet the condition
        if timeElapsed <= twentyFourHours:
            print "The File: " + str(i) + " Was Last Modified At: " + str(fileModDateTime) + " ,Which was about: " \
              + str(timeElapsed) + " ago."
GeorgeZ
  • 51
  • 5
-1

I dont believe the os module will work as I am using paramiko to SFTP to the remote host and perform actions on the files in the directory

for filename in file_list_attr:
    mtime = os.path.getmtime(filename)
    print mtime



Traceback (most recent call last):
  File "<pyshell#22>", line 2, in <module>
    mtime = os.path.getmtime(filename)
  File "U:\ActivPy\lib\genericpath.py", line 54, in getmtime
    return os.stat(filename).st_mtime
TypeError: coercing to Unicode: need string or buffer, SFTPAttributes found
Super_Py_Me
  • 169
  • 1
  • 4
  • 14