Google app engine deployment fails- Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')

Question

We are using command prompt c:\gcloud app deploy app.yaml, but get the following error:

Running "python3 -m pip install --requirement requirements.txt --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile (PIP_CACHE_DIR=/layers/google.python.pip/pipcache PIP_DISABLE_PIP_VERSION_CHECK=1)"
Step #2 - "build": /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
Step #2 - "build": Done "python3 -m pip install --requirement requirements.txt --upgr..." (34.49892ms)
Step #2 - "build": Failure: (ID: 0ea8a540) /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
Step #2 - "build": --------------------------------------------------------------------------------
Step #2 - "build": Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
Step #2 - "build": Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (12.758866ms)
Step #2 - "build": ERROR: failed to build: exit status 1
Finished Step #2 - "build"
ERROR
ERROR: build step 2 "us.gcr.io/gae-runtimes/buildpacks/python37/builder:python37_20211201_3_7_12_RC00" failed: step exited with non-zero status: 145

Our Requirements.txt is as below. We are currently on Python 3.7 standard app engine

firebase_admin==3.0.0
sendgrid==6.9.3
google-auth==1.35.0
google-auth-httplib2==0.1.0
jinja2==3.0.3
MarkupSafe==2.0.1
pytz==2021.3
Flask==2.0.2
twilio==6.46.0
httplib2==0.20.2
requests==2.24.0
requests_toolbelt==0.9.1
google-cloud-tasks==2.7.1
google-cloud-logging==1.15.1
googleapis-common-protos==1.54.0

Please help.The above code was working well before updating the requirements.txt file. We tried to remove gunicorn to allow the system pickup the latest according to documentation here.

We have a subdirectory structure that stores all the .py files in controllers and db definitions in models. Our main.py has the following -

sys.path.append(os.path.join(os.path.dirname(__file__), '../controllers'))
sys.path.append(os.path.join(os.path.dirname(__file__), '../models'))

Does anyone know how to debug this error - Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__'). What does this mean?

Answer 1

The release 60.3.0 of setuptools has a bug causing the above AttributeError, see : https://github.com/pypa/setuptools/issues/3002

It has been yanked by Pypi since then:

Yanked files are always ignored, unless they are the only file that matches a version specifier that "pins" to an exact version using either == (without any modifiers that make it a range, such as .*) or ===. Matching this version specifier should otherwise be done as per PEP 440 for things like local versions, zero padding, etc.

Source: https://www.python.org/dev/peps/pep-0592/

What you can do as of now is pass the version of the setuptools package explicitly, to use for example version 60.2.0 or 60.3.1.

Answer 2

By installing the previous version 60.2.0 of setuptools before installing pip-tools helped me get it running.

pip install setuptools==60.2.0

Answer 3

I had the same issue when deploying a Google Cloud Function. The error

cloud function Error while finding module specification for 'pip' (AttributeError: module 'main' has no attribute 'file'); Error ID: c84b3231

appeared after commenting out some packages in the requirements.txt, but that was nothing important and likely did not cause it. I guess that it is more a problem of an instability in Google Storage, since that same Cloud Function I was working on had lost its archive already some time before, all of a sudden, out of nowhere, showing:

Archive not found in the storage location cloud function

and I did not delete or change anything that might explain this, as Archive not found in the storage location: Google Function would suggest. Though that answer has one very interesting guess that might explain at least the very first time the "Archive not found" error came up and thus made the CF instable: I might have changed the timezone city of the bucket during browsing the Google Storage. It is too long ago, but I know I browsed the GS, therefore, I cannot exclude this. Quote: "It [the Archive not found error] may occurr too if GCS bucket's region is not matched to your Cloud function region."

After this "Archive not found" crash, I manually added main.py and requirements.txt and filled them again with code from the backup. This worked for some time, but there seems to be some general instability in the Google Storage. Therefore, always keep backups of your deployed scripts.

Then, after getting this pip error of the question in that already instable Cloud Function, waiting for a day or two, Google Function again showed

Archive not found in the storage location cloud function

If you run into this pip error in a Cloud Function, you might consider updating pip in the "requirements.txt" but if you are in such an unstable Cloud Function the better workaround seems to be to create a new Cloud Function and copy everything in there.

The pip error probably just shows that the source script, in this case the requirements.txt, cannot be run since the source code is not fully embedded anymore or has lost some embedding in the Google Storage.

Or you give that Cloud Function a second chance and edit, go to Source tab, click on Dropdown Source code to choose Inline Editor and add main.py and requirements.txt manually (Runtime: Python).

Answer 4

Your setuptools version is likely to be yanked:

https://pypi.org/project/setuptools/60.3.0/

Not sure how to fix that without a working pip though.

Answer 5

Setuptools 60.3.1 is out now. GitHub Link here.