16

I have tried downloading small files from google Colaboratory. They are easily downloaded but whenever I try to download files which have a large sizes it shows an error? What is the way to download large files?

shivin saluja
  • 199
  • 1
  • 2
  • 5

4 Answers4

17

If you have created a large zip file, say my_archive.zip, then you can download it as following:

  1. Mount your Google drive from your Google colab Notebook. You will be asked to enter a authentication code. 
from google.colab import drive
drive.mount('/content/gdrive',force_remount=True)
  1. Copy the zip file to any of your google drive folder (e.g. downloads folder). You may also copy the file on 'My Drive' which is a root folder.
!cp my_archive.zip '/content/gdrive/My Drive/downloads/'
!ls -lt '/content/gdrive/My Drive/downloads/'

Finally, you can download the zip file from your Google drive to your local machine.

codefun
  • 316
  • 2
  • 6
11

This is how I handle this issue:

from google.colab import auth
from googleapiclient.http import MediaFileUpload
from googleapiclient.discovery import build

auth.authenticate_user()

Then click on the link, authorize Google Drive and paste the code in the notebook.

drive_service = build('drive', 'v3')

def save_file_to_drive(name, path):
    file_metadata = {
      'name': name,
      'mimeType': 'application/octet-stream'
     }

     media = MediaFileUpload(path, 
                    mimetype='application/octet-stream',
                    resumable=True)

     created = drive_service.files().create(body=file_metadata,
                                   media_body=media,
                                   fields='id').execute()

     print('File ID: {}'.format(created.get('id')))

     return created

Then:

save_file_to_drive(destination_name, path_to_file)

This will save whatever files to your Google Drive, where you can download or sync them or whatever.

Eric Antoine Scuccimarra
  • 375
  • 2
  • 7
1

I tried many different solutions. The only way that was effective and quick is to zip the file/folder and then download it directly:

 !zip -r model.zip model.pkl

And to download:

enter image description here

Minions
  • 5,104
  • 5
  • 50
  • 91
0

Google colab doesn't allow you to download large files using files.download(). But you can use one of the following methods to access it:

  1. The easiest one is to use github to commit and push your files and then clone it to your local machine.
  2. You can mount google-drive to your colab instance and write the files there.
Ankur Ankan
  • 2,953
  • 2
  • 23
  • 38
  • 1
    when I run the command - !git push -u origin master It asks for my username and password. How ca I enter my username and password in the current running cell? Can not see a space where I can enter them. – shivin saluja Mar 30 '18 at 13:54
  • @shivinsaluja You can't enter username and password. You will need to setup access to github using ssh: https://help.github.com/articles/connecting-to-github-with-ssh/ – Ankur Ankan Mar 30 '18 at 18:45
  • you cant do it also because sshj-keygen wants keyboard interaction – Stepan Yakovenko Aug 18 '18 at 16:37