4

I have several different Notebooks in Google Colab that I use to automate some tasks. I want to create a single Notebook that runs all this tasks, so I can open a single tab with one single Notebook, run it and it will run all other tasks inside these different Notebooks.

I have two questions regarding this problem:

  1. The solution I found is not working (I will describe it below). How do I make it work?
  2. Is there a better solution than the one I found?

About the first question:

Image I have Notebook_1 and Notebook_2 each one with a bunch of functions that automate my tasks. What I am doing is, downloading them as Notebook_1.py and Notebook_2.py, saving these files in a Google Drive folder. Then in my Notebook_main, which is the notebook that should house all notebooks, I run:

# Mounts Google Drive
from google.colab import drive
drive.mount('/content/drive')

# Copies .py files into Google Colab
!cp /content/drive/MyDrive/my_modules/Notebook_1.py /content
!cp /content/drive/MyDrive/my_modules/Notebook_2.py /content

# Import Modules

import Notebook_1
import Notebook_2

If I want to run a simple function inside these modules I just do:

Notebook_1.run_simple_function()

and this works. My problem happens when the function I am trying to run from the Notebook_1, for example, uses another module. Then I get the following error:

name 'os' is not defined

I imagine it happens because inside Notebook_1.py I call:

import os
...
os.remove(os.path.join(dir_download, item))
...

And I also think this will happen with all the modules that I call inside Notebook_1.py.

I have tried importing theses modules in Notebook_main, but it did not work. I do not know how to fix this. I need help.

Another issue is that I use a lot of Selenium, which needs to be installed in Google Colab before being imported. So I need to install and import it in Notebook_main and when I run a function with Notebook_1.run_function_that_uses_selenium() it should use the import from Notebook_main.

The second question is simpler. I just want to know if there is a better way to achieve the same result, i.e. run different Notebooks in Google Colab from a single notebook.

My constriction is that I can only use Google Colab and other Google related Platforms, I can not run anything locally.

Ivanhercaz
  • 731
  • 1
  • 11
  • 31
user3347814
  • 1,138
  • 9
  • 28
  • 50
  • 1
    Have you tried the approach described in [this](https://stackoverflow.com/a/59169452/9987623) answer? – AlexK May 29 '22 at 22:29
  • No. The only approach I have tried is the one I have described. I will try your suggestion, though. I still would like to know what I am doing wrong and why my solution is not working. – user3347814 May 30 '22 at 03:38
  • 1
    You may need to provide an [MRE](https://stackoverflow.com/help/minimal-reproducible-example) for the `NameError`. I can easily call a function that uses something out of the standard library and that is saved in a module from a notebook, as you are trying to do. – AlexK Jun 02 '22 at 04:05
  • 1
    For using something like Selenium, I would recommend keeping everything in notebook format, installing and importing packages in notebooks that use those packages (in your case, installing and importing Selenium in Notebook_1.ipynb), and using `%run /content/drive/MyDrive/my_notebooks/Notebook_1.ipynb` in your main notebook, which both runs the installation command and imports the functions into main notebook so you can call them there. – AlexK Jun 02 '22 at 05:48
  • I know this is is a stupid question, but my directory and file name has spaces. What should I do? root:File `'/content/drive/MyDrive/Colab Notebooks/Baixar relatórios.py'` not found. – user3347814 Jun 04 '22 at 19:01
  • 1
    You need to escape those spaces by placing a backslash before the space in the path, e.g., `/content/drive/MyDrive/Colab\ Notebooks/Baixar\ relatórios.py` – AlexK Jun 04 '22 at 19:11
  • %run /content/drive/MyDrive/Colab\ Notebooks/Baixar\ relatórios\ FBads gives me this error: NameError Traceback (most recent call last) /content/drive/MyDrive/Colab Notebooks/Baixar relatórios FBads in () ----> 1 {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Baixar relatórios FBads","provenance":[{"file_id":"1aIoHejsWIurmz5ox__uJIOJu1WS7SBkN","timestamp":1652973400332},{"file_id":"1-9lR1Na16D9hM-dponQi44NBSZJaP5F5","timestamp":1637328344750}],"collapsed_sections":[]},"kernelspec":{"name":"pyth... NameError: name 'null' is not defined – user3347814 Jun 04 '22 at 19:12
  • 1
    See https://stackoverflow.com/questions/55426505/how-can-i-resolve-a-nameerror-name-null-is-not-defined-error-while-trying-t#comment105140585_55426505. At this point, I don't know if you are trying to run a notebook or a .py file, but if it's a .py file you are creating from a notebook, the best thing to do is to create it by going to File > Download > Download .py in Google Colab. – AlexK Jun 04 '22 at 19:30
  • I simply clicked on "copy path" on the file and pasted after %run. I comes without a .extension. I have tried bouth .py and .ipynb. It does not work either way. – user3347814 Jun 04 '22 at 19:40
  • 1
    There is something off about how you created your file. A file should have a file extension. I can't add anything more beyond what I said above. – AlexK Jun 04 '22 at 19:45

1 Answers1

2

Simple.

from google.colab import drive
drive.mount('/content/gdrive')

then

libdir = "/content/gdrive/My\ Drive/Colab\ Notebooks/lib/"
%run {libdir}MyNotebook.ipynb
Lex Podgorny
  • 2,598
  • 1
  • 23
  • 40