0

I am trying to upload a large zipped file to Dropbox (about 2-3GB) using Python and the Dropbox API v2. I am using the "chunked method" found here: dropbox API v2 upload large files using python. Here is the code again for convenience:

f = open(file_path)
file_size = os.path.getsize(file_path)

CHUNK_SIZE = 4 * 1024 * 1024

if file_size <= CHUNK_SIZE:

    print dbx.files_upload(f, dest_path)

else:

    upload_session_start_result = dbx.files_upload_session_start(f.read(CHUNK_SIZE))
    cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
                                               offset=f.tell())
    commit = dropbox.files.CommitInfo(path=dest_path)

    while f.tell() < file_size:
        if ((file_size - f.tell()) <= CHUNK_SIZE):
            print dbx.files_upload_session_finish(f.read(CHUNK_SIZE),
                                            cursor,
                                            commit)
        else:
            dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                            cursor.session_id,
                                            cursor.offset)
            cursor.offset = f.tell()

However, when I run it I am getting a

dropbox.exceptions.APIError

as well as

UploadSessoionLookupError

and a

UploadSessionOffsetError

I think the error might be occurring at this line specifically: 

 dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                       cursor.session_id,
                                       cursor.offset)

I've tried swapping that for 

dbx.files_upload_session_append_v2(
                f.read(self.CHUNK_SIZE), cursor)

but that didn't work either. Any suggestions?

Jess
  • 73
  • 2
  • 12
  • `APIError`, `UploadSessionLookupError`, and `UploadSessionOffsetError` are just the error types. They should contain more information, so make sure you check the full error for more information on the issue. – Greg Oct 25 '17 at 18:36
  • The `UploadSessionLookupError` gave me "incorrect_offset" while `UploadSessionOffesetError` gave me "correct_offset = 10000" I still don't know what to do with that information though. – Jess Oct 25 '17 at 18:43
  • Did you also post on the Dropbox forum? I just replied there with more information on that: https://www.dropboxforum.com/t5/API-support/Uploading-large-files-using-chunks/m-p/249182#M14309 – Greg Oct 25 '17 at 19:26

1 Answers1

0

On windows, be sure to open using binary mode

f = open(file_path, 'rb')

f.read(chunk_size) and f.tell() were off.

from python docs

On Windows, tell() can return illegal values (after an fgets()) when reading files with Unix-style line-endings. Use binary mode (‘rb’) to circumvent this problem.

taras
  • 6,566
  • 10
  • 39
  • 50
Antonio Urquidi
  • 1