Ignore (skip) corrupt files when logging metadata and proceed with the rest

Question

A metadata logging code gets stuck on a corrupt files and gives the error shown below.

How to skip (ignore) corrupt files and continue the program?

Code (lines 68-87 and 198-204):

#lines 68-87:
def createBasicInfoListFromDisk():

    global diskCompareListDetails, onlyFileNameOnDisk, driveLetter,walk_dir

    walk_dir = os.path.abspath(walk_dir)
    for root, subdirs, files in os.walk(walk_dir, topdown=True, onerror=None, followlinks=True ):
        for filename in files:
            file_path = os.path.join(root, filename)
            temp = file_path.split(":")
            driveLetter = temp[0]
            filePathWithoutDriveLetter = temp[1]
            fileSize = os.path.getsize(file_path)
            mod_on =  get_last_write_time(file_path)
            print('\t- file %s (full path: %s)' % (filename, file_path))
            print('FileName : {filename} is of size {size} and was modified on{mdt}'.format(filename=file_path,size=fileSize,mdt=mod_on ))

            diskCompareListDetails.append("\"" + filePathWithoutDriveLetter+"\",\""+str(fileSize) + "\",\"" + mod_on +'"')
            onlyFileNameOnDisk.append("\""+filePathWithoutDriveLetter+"\"")

    return


#lines 198-204:
    foundFile = 0
    foundFile=findAndReadCSVFile(csvfilewithPath)
    createBasicInfoListFromDisk()
    compareLogAndDiskLists()
    displayInfoForUserInput()
    processFiles(foundFile)
    writeCSVFile(csvfilewithPath)

Error:

FileName : T:\STBY\file1.txt is of size 1241 and was modified on2006-03-15 20:35:00
Traceback (most recent call last):
  File "c:\scripts\test.py", line 200, in <module>
    createBasicInfoListFromDisk()
  File "c:\scripts\test.py", line 79, in createBasicInfoListFromDisk
    fileSize = os.path.getsize(file_path)
  File "C:\Python\Python36\lib\genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'T:\\STBY\\file1.txt'

Modification - no error message, but 50% of files are skipped, files, that a powershell script recognizes as ok:

def createBasicInfoListFromDisk():
    try:
        global diskCompareListDetails, onlyFileNameOnDisk, driveLetter,walk_dir

        walk_dir = os.path.abspath(walk_dir)
        for root, subdirs, files in os.walk(walk_dir, topdown=True, onerror=None, followlinks=True ):
            for filename in files:
                file_path = os.path.join(root, filename)
                temp = file_path.split(":")
                driveLetter = temp[0]
                filePathWithoutDriveLetter = temp[1]
                fileSize = os.path.getsize(file_path)
                mod_on =  get_last_write_time(file_path)
                print('\t- file %s (full path: %s)' % (filename, file_path))
                print('FileName : {filename} is of size {size} and was modified on{mdt}'.format(filename=file_path,size=fileSize,mdt=mod_on ))

                diskCompareListDetails.append("\"" + filePathWithoutDriveLetter+"\",\""+str(fileSize) + "\",\"" + mod_on +'"')
                onlyFileNameOnDisk.append("\""+filePathWithoutDriveLetter+"\"")

    except OSError:
        pass
    return "ERROR"

Wrap the problem-causing code in a `try`-`except FileNotFoundError` clause? — , Nov 23 '17 at 21:44
Possible duplicate of [Try/Except in Python: How do you properly ignore Exceptions?](https://stackoverflow.com/questions/730764/try-except-in-python-how-do-you-properly-ignore-exceptions) — zipa, Nov 23 '17 at 21:48

score 1 · Accepted Answer · answered Nov 23 '17 at 22:12

One way is using try-except:

try:
    fileSize = os.path.getsize(file_path)
except OSError as e:
    fileSize = -1
    print('error thrown when handle {0}'.format(file_path)

Alternatively, you can check wether file exists before invoke getsize():

fileSize = -1 if not os.path.exists(file_path) else os.path.getsize(file_path)

Ignore (skip) corrupt files when logging metadata and proceed with the rest

1 Answers1