file download from google drive to colaboratory

Question

I was trying to download file from my google drive to colaboratory.

file_id = '1uBtlaggVyWshwcyP6kEI-y_W3P8D26sz'

import io
from googleapiclient.http import MediaIoBaseDownload

request = drive_service.files().get_media(fileId=file_id)
downloaded = io.BytesIO()
downloader = MediaIoBaseDownload(downloaded, request)
done = False
while done is False:
  # _ is a placeholder for a progress object that we ignore.
  # (Our file is small, so we skip reporting progress.)
  _, done = downloader.next_chunk()

downloaded.seek(0)
print('Downloaded file contents are: {}'.format(downloaded.read()))

doing so am getting this error:

NameError: name 'drive_service' is not defined

How to remove this error?

score 88 · Answer 1 · edited Jul 09 '22 at 07:23

88

No installing/importing any library. Just put your file id at the end.

!gdown yourFileIdHere

Note: at the time of writing gdown library is preinstalled on Colab.

edited Jul 09 '22 at 07:23

Farid

48
1
6

answered Jun 25 '20 at 05:47

hoper

1,043
7
15

2

Recently gdown has stopped functioning. – AVISHEK GARAIN Mar 06 '22 at 13:07
@AVISHEKGARAIN I've checked. It still works properly! – hoper Mar 09 '22 at 05:57
1

"Access denied with the following error: Cannot retrieve the public link of the file. You may need to change the permission to 'Anyone with the link', or have had many accesses. You may still be able to access the file from the browser:" This is the error. The file is already publicly accessible. Tried with many files. – AVISHEK GARAIN Mar 11 '22 at 08:09
@AVISHEKGARAIN Apparently it's not a public problem. Open an issue in their github page if there's not already one addressing this error. – hoper Mar 14 '22 at 16:41
5

@AVISHEKGARAIN, I was having a similar problem because my file was too big and Google wanted to scan for viruses. I got around it by using this url and the id of the file: `!wget file.csv 'https://docs.google.com/uc?export=download&id=XXX&confirm=t'` I think the `confirm=T` bypasses the virus. – RossHochwert Mar 14 '22 at 21:07
Thanks @RossHochwert I wold definitely try it out. – AVISHEK GARAIN Mar 16 '22 at 16:53
1

Here `id` is what follows after /d/ in the link path. For instance in the following link drive.google.com/file/d/1q_wwOynOUCBmn51RJ3fxinLtuPF64ACg/view?usp=sharing , id is `1q_wwOynOUCBmn51RJ3fxinLtuPF64ACg` – Muhammad Yasir Apr 22 '22 at 16:21

Agile Bean · Answer 2 · 2021-06-23T14:06:21.487

the easiest method to download a file from google drive to colabo notebook is via the colabo api:

from google.colab import drive
drive.mount('/content/gdrive')

!cp '/content/gdrive/My Drive/<file_path_on_google_drive>' <filename_in_colabo>

Remarks:

By the drive.mount(), you can access any file on your google drive.
'My Drive' is equivalent to 'Google Drive' on your local file system.
The file_path is surrounded with single quotes as the standard directory below the mount point ('My Drive') has a space, and you might also have spaces in your path elsewhere anyway.
Very useful to locate your file and get the file path is the file browser (activated by click on left arrow). It lets you click through your mounted folder structure and copy the file path, see image below.

score 8 · Answer 3 · answered Feb 08 '19 at 09:30

Here's an easy way to get by. You may either use wget command or requests module in Python to get the job done.

# find the share link of the file/folder on Google Drive
file_share_link = "https://drive.google.com/open?id=0B_URf9ZWjAW7SC11Xzc4R2d0N2c"

# extract the ID of the file
file_id = file_share_link[file_share_link.find("=") + 1:]

# append the id to this REST command
file_download_link = "https://docs.google.com/uc?export=download&id=" + file_id

The string in file_download_link can be pasted in the browser address bar to get the download dialog box directly.

If you use the wget command:

!wget -O ebook.pdf --no-check-certificate "$file_download_link"

Shaurya Uppal · Answer 4 · 2018-07-22T14:29:37.190

Step 1

!pip install -U -q PyDrive

Step 2

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
# Authenticate and create the PyDrive client.
# This only needs to be done once per notebook.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

Step3

file_id = '17Cp4ZxCYGzWNypZo1WPiIz20x06xgPAt' # URL id. 
downloaded = drive.CreateFile({'id': file_id})
downloaded.GetContentFile('shaurya.txt')

Step4

!ls #to verify content

OR

import os
print(os.listdir())

Especially for limited sharing files, this works very well. – Mai Hai Jan 04 '22 at 03:00 — Mai Hai, Jan 04 '22 at 03:00

score 2 · Answer 5 · answered Feb 11 '18 at 20:05

You need to define a drive API service client to interact with the Google drive API, for instance:

from googleapiclient.discovery import build
drive_service = build('drive', 'v3')

(see the notebook External data: Drive, Sheets, and Cloud Storage/Drive REST API)

score 2 · Answer 6 · answered Feb 12 '18 at 02:49

I recommend you use Pydrive to download your file from google drive. I download 500MB dataset for 5s. 1. install Pydrive

!pip install PyDrive

2. OAouth

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LocalWebserverAuth()

drive = GoogleDrive(gauth)

3. code for download file from google drive

fileId = drive.CreateFile({'id': 'DRIVE_FILE_ID'}) #DRIVE_FILE_ID is file id example: 1iytA1n2z4go3uVCwE_vIKouTKyIDjEq
print fileId['title']  # UMNIST.zip
fileId.GetContentFile('UMNIST.zip')  # Save Drive file as a local file

Cheer Mo Jihad

score 1 · Answer 7 · answered May 12 '22 at 16:53

1

The --id argument has been deprecated so now you simply have to run:

! gdown 1uBtlaggVyWshwcyP6kEI-y_W3P8D26sz

If your file is stored in a variable you can run:

! gdown $my_file_id

answered May 12 '22 at 16:53

rudolfovic

3,163
2
14
38

score 1 · Answer 8 · answered Mar 18 '23 at 14:41

1

For Files: !gdown FileDrivePath

For Folders: !gdown --folder FolderDrivePath

answered Mar 18 '23 at 14:41

jrreda

763
7
7

score 0 · Answer 9 · answered Nov 19 '18 at 05:06

You can also use my implementations on google.colab and PyDrive at https://github.com/ruelj2/Google_drive which makes it a lot easier.

!pip install - U - q PyDrive  
import os  
os.chdir('/content/')  
!git clone https://github.com/ruelj2/Google_drive.git  

from Google_drive.handle import Google_drive  
Gd = Google_drive()  
Gd.load_file(local_dir, file_ID)

score 0 · Answer 10 · answered Feb 08 '23 at 20:38

You can simply copy all your google drive files and folders inside google colab and use it directly using this command

# import drive
from google.colab import drive
drive.mount('/content/drive')

This will ask you for permission after accepting you will have all your google drive inside your colab If you want to use any file just copy the path

score 0 · Answer 11 · answered Jun 05 '23 at 05:54

0

For me, this works: !gdown --fuzzy link_to_file -0 file.zip

answered Jun 05 '23 at 05:54

Max Ph

1
1

file download from google drive to colaboratory

11 Answers11

Linked