13

Wanted to try out python, and google colaboratory seemed the easiest option.I have some files in my google drive, and wanted to upload them into google colaboratory. so here is the code that i am using:

!pip install -U -q PyDrive

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# 1. Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

# 2. Create & upload a file text file.
uploaded = drive.CreateFile({'xyz.csv': 'C:/Users/abc/Google Drive/def/xyz.csv'})
uploaded.Upload()
print('Uploaded file with title {}'.format(uploaded.get('title')))

import pandas as pd
xyz = pd.read_csv('Untitled.csv')

Basically, for user "abc", i wanted to upload the file xyz.csv from the folder "def". I can upload the file, but when i ask for the title it says the title is "Untitled". when i ask for the Id of the file that was uploaded, it changes everytime, so i can not use the Id.

How do i read the file??? and set a proper file name???

xyz = pd.read_csv('Untitled.csv') doesnt work
xyz = pd.read_csv('Untitled') doesnt work
xyz = pd.read_csv('xyz.csv') doesnt work

Here are some other links that i found..

How to import and read a shelve or Numpy file in Google Colaboratory?

Load local data files to Colaboratory

Akhil
  • 165
  • 1
  • 1
  • 8

5 Answers5

16

To read a csv file from my google drive into colaboratory, I needed to do the following steps:

1) I first needed to authorize colaboratory to access my google drive with PyDrive. I used their code example for that. (pasted below)

2) I also needed to log into my drive.google.com to find the target id of the file i wanted to download. I found this by right clicking on the file and copying the shared link for the ID. The id looks something like this: '1BH-rffqv_1auzO7tdubfaOwXzf278vJK'

3) Then I ran downloaded.GetContentFile('myName.csv') - putting in the name i wanted (in your case it is xyz.csv)

This seems to work for me!

I used the code they provided in their example:

# Code to read csv file into colaboratory:
!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# 1. Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

#2. Get the file
downloaded = drive.CreateFile({'id':'1BH-rffqv_1auzO7tdubfaOwXzf278vJK'}) # replace the id with id of file you want to access
downloaded.GetContentFile('xyz.csv')  

#3. Read file as panda dataframe
import pandas as pd
xyz = pd.read_csv('xyz.csv')
Cendria
  • 161
  • 2
3

Okay I'm pretty sure I'm quite late, but I'd like to put this out there, just in case. I think the easiest way you could do this is by

from google.colab import drive
drive.mount("/content/drive")

This will generate a link, click on it and sign in using Google OAuth, paste the key in the colab cell and you're connected!

check out the list of available files in the side bar on the left side and copy the path of the file you want to access. Read it as you would, with any other file.

gavin
  • 793
  • 1
  • 7
  • 19
1

File create takes a file body i its first parameter. If you check the documentation for file create there are a number of fields you can fill out. In the example below you would add them to file_metadata comma separated.

file_metadata = {'name': 'photo.jpg'}
media = MediaFileUpload('files/photo.jpg',
                        mimetype='image/jpeg')
file = drive_service.files().create(body=file_metadata,
                                    media_body=media,
                                    fields='id').execute()

I suggest you read the file upload section of the documentation to get a better idea how upload works and which files can actually be read from within google drive. I am not sure that this is going to give you access to Google colaborate

Possible fix for your code.

I am not a python dev but my guess would be you can set your title by doing this.

uploaded = drive.CreateFile({'xyz.csv': 'C:/Users/abc/Google Drive/def/xyz.csv',
                             'name': 'xyz.csv'})
Linda Lawton - DaImTo
  • 106,405
  • 32
  • 180
  • 449
  • i did read the content related to the error when i tried to execute my code above. Are you saying that i can not upload csv files to be used in python code? can you help out and point out where i have made a mistake in my code above? The code above is just to upload a csv file (from google drive) and assign it to a variable (in google colaboratory) – Akhil Jan 25 '18 at 07:41
  • This was the way a text file was supposed to be uploaded..i have modified it to upload a csv file from folder. what further changes are required, please help! – Akhil Jan 25 '18 at 07:45
  • The file is getting uploaded..but the title is not being set..why is that? – Akhil Jan 25 '18 at 08:13
  • Set Name in the meta data as shown above. You can upload any file you want to google drive but Google drive can only show some types of files and it can only edit some types of files. And to my knowledge it has nothing to do with Google Collaborate. – Linda Lawton - DaImTo Jan 25 '18 at 08:27
  • i think that i am not being able to get my point across....i have no problem with google drive..i have been using it for quite some time as a way to share files..i already have csv files in google drive and for use in R. I wanted to try out python, without installing any software, and i thought google collaborate could help me do that. before implementing any python code, i have to assign the files that i would be using to appropriate variables..these files are in google drive, and i am able to upload to google collaborate, but unable to name them properly!! let me try ur way and see. thanks! – Akhil Jan 25 '18 at 08:42
  • Did you try just adding name in your createfile method? 'name': 'xyz.csv' – Linda Lawton - DaImTo Jan 25 '18 at 08:43
  • it has been an exceptionally convoluted process (trying to import a file from one google app to another). leaving it for now to try another day. Thanks for your help! – Akhil Jan 25 '18 at 10:58
0

I think it's that simple with this command

# Mount Google Drive
import os
from google.colab import drive

drive.mount('/content/drive')
!pwd
!ls

import pandas as pd
df = pd.read_csv('Untitled.csv')

It will require authorization with your Google OAuth, and create authorization key. put the key into the colab cell.

Please aware !, sometimes the file within google colab directory are not update or similar with google drive if you delete or add files in your Google Drive.

YGautomo
  • 619
  • 8
  • 12
0

My Way (without authorization)

Find file ID in your csv file link (file public) -> "https://drive.google.com/file/d/ID/view?usp=sharing"

For example -> "https://drive.google.com/file/d/1MeA00way9lA3_lllI-lsCefPxxxXXXXX/view? usp=sharing"

Put your id -> url = "https://drive.google.com/uc?export=download&id=ID"

For example url = "https://drive.google.com/uc?export=download&id=1MeA00way0lA3_lllI-lsCefPxxxXXXXX"

Final code

url = "https://drive.google.com/uc?export=download&id=1MeA00way0lA3_lllI-lsCefPxxxXXXXX"
df = pd.read_csv(url)
Finley
  • 1
  • 1