7

I have written a python script to automatize some gsutil operations. gsutil works fine if I run it by command line. But if I translate the same command with subprocess in python I get an error:

BUCKET_NAME = 'datastore-backup'
FOLDER_NAME = 'my_folder'

gcs_path = os.path.join('gs://', BUCKET_NAME, FOLDER_NAME)

files = subprocess.check_output(['gsutil', 'ls', gcs_path], stderr=sys.stdout)
print(files)

I get this error

Traceback (most recent call last):
  File "/Users/dario/Downloads/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 13, in <module>
    import bootstrapping
  File "/Users/dario/Downloads/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 32, in <module>
    import setup  # pylint:disable=g-import-not-at-top
  File "/Users/dario/Downloads/google-cloud-sdk/bin/bootstrapping/setup.py", line 55, in <module>
    from googlecloudsdk.core import properties
  File "/Users/dario/Downloads/google-cloud-sdk/lib/googlecloudsdk/core/properties.py", line 34, in <module>
    from googlecloudsdk.core.util import times
  File "/Users/dario/Downloads/google-cloud-sdk/lib/googlecloudsdk/core/util/times.py", line 55, in <module>
    from dateutil.tz import _common as tz_common
ImportError: cannot import name _common

any help

ZUKINI
  • 195
  • 2
  • 15
DarioB
  • 1,349
  • 2
  • 21
  • 44
  • 2
    I tried to run your code in the Google Cloud Shell but couldn't replicate the error. It seems that it has something to do with your Python environment and not with Google Cloud Platform or `gsutil` command. I've seen this error appear when running the code in PyCharm. Could you confirm if you are indeed trying to run it in PyCharm? If so, check [this](https://intellij-support.jetbrains.com/hc/en-us/community/posts/360000380760-ImportError-when-running-dev-appserver-py-from-pycharm), it might be helpful. – Joan Grau Noël May 17 '19 at 11:48
  • yes i am running this in pycharm..... boxed were already ticked as per solution proposed, no luck... – DarioB May 17 '19 at 11:52
  • I would recommend to change the tags of your question as this is not a Google Cloud Platform issue as long as the gsutil command runs without errors from your command line. This way, your question will reach people more suited to answer it. You could also search/post in the PyCharm community [here](https://intellij-support.jetbrains.com/hc/en-us/community/topics/200379535-PyCharm) – Joan Grau Noël May 17 '19 at 13:20
  • Possible duplicate of [ImportError when running dev\_appserver.py from pycharm](https://stackoverflow.com/questions/51568417/importerror-when-running-dev-appserver-py-from-pycharm) – ZUKINI May 21 '19 at 18:52

1 Answers1

0

I have solved this problem for good. The source of the issue is when you call either subprocess.check or os.system(, the gsutil cli would use the wrong python.

In my case gsutil uses a system python binary that is 2.7. I uncovered this issue by inserting a print(tz) statement in /Users/dario/Downloads/google-cloud-sdk/lib/googlecloudsdk/core/util/times.py. And it turns out that it is loading a Python2.7/site-packages version of dateutil.

So to fix this, you can specify the cloudsdk python binary via the following

export CLOUDSDK_PYTHON=<what-ever-python-distro-you-want>

pyCharm is a wonderful tool second to non. Hope this makes it easy for some to use the built-in runner/debugger of this powerful IDE.

episodeyang
  • 642
  • 8
  • 15