1

I have a similar issue as this Can't get Python to import from a different folder The solution there doesn't solve my issue.

I'm working with Airflow lib. The lib updated one of the operators and since I can not at this time upgrade the my Airflow version I want to manually download the operator .py file and use it in my code manualy. 

airflow
    -dags
        --mydag.py
    -AddedOperators
        --sagemaker_tuning_operator.py
        --__init__.py   (empty file)

The sagemaker_tuning_operator.py is this file: https://github.com/apache/airflow/blob/master/airflow/contrib/operators/sagemaker_tuning_operator.py

It contains the class SageMakerTuningOperator

In my mydag.py I do:

from AddedOperators.sagemaker_tuning_operator import SageMakerTuningOperator

When Airflow try to parse mydag.py I get:

No module named AddedOperators.sagemaker_tuning_operator

Luis
  • 1,305
  • 5
  • 19
  • 46

1 Answers1

2

Check if your project directory is in your system path or not. You can can check it as follows:

import sys

print(sys.path)

Note that when running a Python script, sys.path doesn’t care what your current “working directory” is. It only cares about the path to the script. For example, if my shell is currently at the Airflow/ folder and I run python ./dags/mydag.py, then sys.path includes Airflow/dags/ but NOT Airflow/

If you project directory is not in sys path, you can do following:

  • Include it dynamically.

    import sys
sys.path.insert(0, 'path/to/your/')

# import your module now

  • Import all required modules in your project root folder in some file like app.py and then call from here.

ydrall
  • 338
  • 1
  • 3
  • 11
  • It wasn't in sys.path. I did: sys.path.insert(0, '/home/ubuntu/airflow') still doesn't work. same error – Luis Mar 03 '19 at 09:49
  • Adding path dynamically is working fine for me. Can you show what you have done? – ydrall Mar 03 '19 at 09:58