2

recently colab removed the ability to connect to google drive from different accounts other than the one you were logged into in google drive. There was a workaround someone posted with the following code which worked great, until now...

!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
%cd /content
!mkdir gdrive
%cd gdrive
!mkdir "My Drive"
%cd ..
%cd ..
!google-drive-ocamlfuse "/content/gdrive/My Drive"

the auth.authenticate_user() line now gives a popup that resembles the recently updated normal authentication process giving this popupgoogle auth popup enter image description here I go through the process and log into my other account and I am met with this message. is there any workaround to this?

the reason this matters is that I have unlimited storage on my edu account for free but I couldn't get my edu account to work the paid version of colab due to security restrictions on my universities system, hence I use a payed colab on my personal account and store my data on the edu account

Phillip Maire
  • 323
  • 2
  • 10

1 Answers1

9

edit: do it all in one cell without printing unneeded information edit2 april 28th 2022: changed !sudo apt install google-drive-ocamlfuse >/dev/null 2>&1 to a new line because sudo apt update was failing on colab and preventing it from running the install line.

!sudo echo -ne '\n' | sudo add-apt-repository ppa:alessandro-strada/ppa >/dev/null 2>&1 # note: >/dev/null 2>&1 is used to supress printing
!sudo apt update >/dev/null 2>&1
!sudo apt install google-drive-ocamlfuse >/dev/null 2>&1
!google-drive-ocamlfuse
!sudo apt-get install w3m >/dev/null 2>&1 # to act as web browser 
!xdg-settings set default-web-browser w3m.desktop >/dev/null 2>&1 # to set default browser 
%cd /content
!mkdir gdrive
%cd gdrive
!mkdir "My Drive"
!google-drive-ocamlfuse "/content/gdrive/My Drive"

then just click the link Failure("Error opening URL:https://accounts.google.com/o/oauth2/auth?client_id=56492... and authorize your account

I found one solution I am not sure how fast it it in terms of connection to grive etc but it mounts at least. I figured this out thanks to link1, link2 first run this, you'll be promted to (click in the box) and then click enter

!sudo add-apt-repository ppa:alessandro-strada/ppa
!sudo apt update
!sudo apt install google-drive-ocamlfuse
!google-drive-ocamlfuse

you'll see this output pictured below, just click the link and authorize your account.

next for some reason you'll need to install a browser, even though you already authorized your account, so run this

!sudo apt-get install w3m # to act as web browser 
!xdg-settings set default-web-browser w3m.desktop # to set default browser

finally mount it

%cd /content
!mkdir gdrive
%cd gdrive
!mkdir "My Drive"
!google-drive-ocamlfuse "/content/gdrive/My Drive"

you should see enter image description here

Phillip Maire
  • 323
  • 2
  • 10
  • I tried this and it worked for gdrive. However, since "from google.colab import auth; auth.authenticate_user()" still failed and your solution didn't seem to fix this, I am unable to use GCP service such as cloud bucket, bigquery, etc and they are also under a different account (but the same one as the gdrive). – kawingkelvin Apr 06 '22 at 23:51
  • it was only supposed to fix the connection to gdrive, however this may help you solve your problem. you can no longer go through googles `auth` for different accounts hence why I used `google-drive-ocamlfuse` (for Gdrive) which is not an official google package. Instead you will need gcsfuse for connecting to GCP, [here](https://www.libhunt.com/compare-gcsfuse-vs-google-drive-ocamlfuse) is the difference between them. [how to mount GCP in colab](https://stackoverflow.com/a/60450255/13944456) also [see this](https://stackoverflow.com/questions/61600439/how-to-mount-gcp-bucket-in-google-colab) – Phillip Maire Apr 07 '22 at 20:51
  • This works in terms of connecting google drive, but I'm getting a huge overhead when performing file operations. It literally takes me five minutes to get a batch (size 64) worth of images from my dataset that's hosted on this drive storage. Is there any faster alternative available? – AurumTechie Aug 11 '22 at 05:39
  • I copy files to colab locally and then use them. My data transfer is relatively fast, I think about 1GB/min or so. I know in the past I ran into issues with colab (no matter what interface I used to access drive data) where I would get massively slowed down and then locked out of drive for 1-2 days because I read/write directly to drive. e.g. trained a model and it would read my full data on each iteration. if this is sudden I suggest waiting 1-2 day and try copying to local. all is cloud based so it shouldn't even depend on your internet connection. – Phillip Maire Aug 16 '22 at 15:50
  • you could also try a friends account, using BOTH their account to access colab but ALSO use their free drive. put your data there, try copying data directly to local and test how long that takes. if there is still an issue with the copy process then I am not sure what is goin on. lmk if this works out! – Phillip Maire Aug 16 '22 at 15:52