2

I am new to google colab and i am figuring out if google colab is able to access files on my computer's cdrive directly. 

import os
path = 'C:\\Users\\guest\\Desktop\\'

for file in os.listdir(path):
    print(file)

The error message that come out is [Errno 2] No such file or directory: 'C:\Users\zhuan.lim\Desktop\script tools\Python Scripts\'

I searched online and some examples said to upload the files first using:

from google.colab import files
uploaded = files.upload()

However, is there another way for google colab to directly read from my drives?

Thanks in advance.

LZa
  • 31
  • 1
  • 1
  • 8

2 Answers2

3

Solution

You can make Google Colab access the files on your computer essentially in three ways:

  1. Upload files to Google Colab.
from google.colab import files
uploaded = files.upload()
  1. Upload your files to your Google Drive account and then mount Google Drive on Colab. In my experience, this has been the most convenient method. Also, note that this allows you to both read and write to Google Drive (as if that is a local drive).
from google.colab import drive
drive.mount('/content/gdrive')
!ls ./content/gdrive

Once loaded, click on Files on the left pane to access the file-structure, as shown in the following screenshot.

enter image description here

Note: Alternatively, click on Files >> Mount Drive and this will insert the code-snippet to mount Google Drive into your Colab Notebook. Once you run that cell, you will see GDrive getting mounted.

  1. Initiate a local runtime and then access it. In this case colab uses your local resources and the local files are accessible to it as well. Please do read the security concerns/warning before initiating this option. I have not personally tried it and your are on your own there.

I will explain option#3 below.

Connecting Colab to Local Runtime

Colab offers you to connect to a local runtime. If you have installed jupyter_http_over_ws as explained here you should be able to just provide the port you used to start the local runtime and connect to it from colab.

Step-1

Click on Reconnect and then select "Connect to local runtime". (Top right corner in colab). enter image description here

Step-2

Click on hyperlink: these instructions, in the pop-up as shown below (in step-3), to install jupyter_http_over_ws, if not already installed.

  1. Install and enable the jupyter_http_over_ws jupyter extension (one-time).
pip install jupyter_http_over_ws
jupyter serverextension enable --py jupyter_http_over_ws
  1. Start server and authenticate.

New notebook servers are started normally, though you will need to set a flag to explicitly trust WebSocket connections from the Colaboratory frontend.

jupyter notebook \
  --NotebookApp.allow_origin='https://colab.research.google.com' \
  --port=8888 \
  --NotebookApp.port_retries=0

For more details, I encourage you to see these instructions.

Step-3

Provide the correct port number (e.g. 8888) that was used to start the local runtime (jupyter notebook on your local machine).

enter image description here

CypherX
  • 7,019
  • 3
  • 25
  • 37
  • Thank you for the detailed explanation for each of the solutions, they are very helpful. I have another question. This works if i have the files/data i require on my local computer. If i have a large quantity of files/data that is stored on a mapped network drive (company server) on my computer, are there any ways to access that directly rather than copying the files? Copying the files might take a long time. – LZa Jan 31 '20 at 03:13
  • If copying files is a concern, you can then use the method "Connecting Colab to Local Runtime", because then you have colab's interface connected to your local jupyter runtime and you can also access the local data (network mapped as well). – CypherX Jan 31 '20 at 23:32
  • Noted on that, i shall try that method out. Thank you so much! – LZa Feb 04 '20 at 03:26
  • What do you mean by "New notebook servers are started normally,", I have to set the flag every time I start jupyter – Jotarata May 14 '22 at 00:43
  • 1
    @Jotarata It means that if you do not pass the optional keyword arguments the notebook server will be started as a regular notebook server (using resources from your computer and looking forward to writing the code in a locally opened Jupyter notebook). By stating the origin as the Google Colab notebook, you are telling the notebook server to expect input from the Colab notebook and not any local one. If you want to avoid having to set the flags every time, either create a Makefile with the commands pasted in, or define a command alias (if on linux or macos or on windows WSL). – CypherX May 14 '22 at 11:28
0

No, there's no other way then files.upload(), because that is the way. But I think your looking for a more user friendly way of getting your files in. You could drag-on-drop your files into Google Drive, and then mount it in your Google Colab session by inserting following lines in a cell and executing it: 

from google.colab import drive
drive.mount('/content/gdrive')

It will prompt you to go to a URL to authenticate yourself. After you've clicked the URL and allowed Google Colab Access to your Google Drive files, you can access your Google Drive files. More elaborate explanation here : Import data into Google Colaboratory

Frederik Bode
  • 2,632
  • 1
  • 10
  • 17