2

Title is self explanatory. I simply downloaded the flexible-hello-world app and deployed it without (almost) a single modification - I deployed it to a service called some-service using this app.yaml

# Copyright 2017, Google, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START gae_flex_quickstart_yaml]
runtime: nodejs
env: flex
service: some-service

# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/nodejs/configuring-your-app-with-app-yaml
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

# [END gae_flex_quickstart_yaml]

I have a billing account enabled for my project.

It hangs at this line:

...
7104cb4c0c814fa53787009 size: 2385
Finished Step #1
PUSH
DONE
--------------------------------------------------------------------------------------------------------------------------------------------------------------

Updating service [some-service] (this may take several minutes)...⠹    

When I go to the app engine console I see it has deployed and can access the URL here: https://some-service-dot-ashored-cloud-dv.uk.r.appspot.com/

But the 502 never goes away.

Help!

EDIT:

Some more information, while it is still hung on the deploy command, I run this in another terminal:

gcloud app instances list --service some-service

and I get:

SERVICE       VERSION          ID                                      VM_STATUS   DEBUG_MODE
some-service  20201119t184828  aef-some--service-20201119t184828-fr7n  TERMINATED

EDIT 2:

When I try to ssh to it I get more weirdness:

gcloud app instances ssh aef-some--service-20201119t184828-fr7n --service some-service --version 20201119t184828

WARNING: This instance is serving live application traffic.  Any changes made could
result in downtime or unintended consequences.
Do you want to continue (Y/n)?  

Sending public key to instance [apps/ashored-cloud-dv/services/some-service/versions/20201119t184828/instances/aef-some--service-20201119t184828-fr7n].
Waiting for operation [apps/ashored-cloud-dv/operations/9de7b298-f4e9-47a7-8a8e-11411e649d50] to complete...done.                                            
ERROR: gcloud crashed (TypeError): can only concatenate str (not "NoneType") to str

EDIT 2:

glcloud --version output:

Google Cloud SDK 319.0.0
bq 2.0.62
cloud-build-local 
core 2020.11.13
gsutil 4.55
Adam Jenkins
  • 51,445
  • 11
  • 72
  • 100
  • By hang, do you mean that the symbol at the end of updating service on CLI stops rolling? Usually App Engine Flex apps [take time to deploy](https://stackoverflow.com/questions/51533528/how-can-i-decrease-deployment-time-of-node-app-on-google-app-engine) (I tried the quickstart you provided and it took me ~8 mins). Could you provide your SDK version? Checking your logs could also help – Donnald Cucharo Nov 20 '20 at 01:25
  • @DonnaldCucharo updated answer with version output, this is what keeps coming up with --verbosity debug: `Updating service [some-service] (this may take several minutes)...⠧DEBUG: Operation [apps/ashored-cloud-dv/operations/a2022048-9e4e-44d8-a9c0-a1e2cd8fcd56] not complete. Waiting to retry.` – Adam Jenkins Nov 20 '20 at 01:34
  • @DonnaldCucharo - I'm completely lost. Now, whenever I hit the google-provided appspot URL in the app engine dashboard for my new service, it ends up directing that traffic to my default service. It's completely unclear to me why. – Adam Jenkins Nov 20 '20 at 02:05
  • If you're still encountering that issue, I would suggest that you reach out to Google Cloud Support so they can check out what's going on with your project. – Donnald Cucharo Nov 20 '20 at 07:34
  • @DonnaldCucharo - I wonder if this has anything to do with the size of the directory holding my containers. I can deploy this app (it takes a while, but it works) in other projects that haven't been as "loved" as this project. Do I need to clean out a storage bucket or something? I've been reviewing the logs and, in the problematic project, there is no log output after the create version entry. – Adam Jenkins Nov 20 '20 at 13:42
  • Each of these issues points to the Virtual Machine being terminated. Is it possible that that JavaScript code has an error that is causing it to stop running? – Gerb Nov 23 '20 at 21:58
  • @Gerb. No, the machine never even tries to come up, according to the logs. It’s with gcloud support, still haven’t gotten an answer. – Adam Jenkins Nov 23 '20 at 22:35
  • 1
    Ah, OK. FWIW, the gcloud "TypeError" was a bug that is fixed in gcloud 320.0.0. It happens when the start operation times out and no hostname is returned to gcloud for it to SSH to. – Gerb Nov 24 '20 at 20:25
  • @Adam There are some things worth checking to see if that helps: GCE CPU quotas as Flex uses GCE VMs to serve your app, IAM permissions for the default [GAE Service Account](https://cloud.google.com/appengine/docs/troubleshooting#healthy-time-out), allocate more resources to see if that's causing the issue and also adding the flag `--verbosity=debug` upon deployment may help. Hope this helps! – bhito Nov 28 '20 at 06:24
  • @bhito the permissions thing is the only one that makes any sense here. The treason is the last log message is the service has been created, with zero log messages that follow. In other projects, there’s tons of logs about the service initializing but not this project. In any case, if it were a permissions issue, then you’d also think I’d get an error message somewhere along the way. I’ve also compared the permissions to a working cloud project and they’re the same. Been with cloud support for over a week, they’re still working on it. – Adam Jenkins Nov 28 '20 at 11:52
  • Could you add the IAM roles that your App Engine's default Service Account has? – bhito Nov 29 '20 at 06:00
  • @bhito - you found it, kind of. I added the Editor permission to the default service account (which I know is overpermissioning, I've got to figure out the exact right ones, but I have a feeling it has to do with storage) and the service came up! – Adam Jenkins Nov 30 '20 at 01:23
  • that's nice! good to hear you've managed to make it work again – bhito Nov 30 '20 at 08:01

1 Answers1

2

tl;dr; permissions

I removed the editor permission from my default app engine service account as recommended in the IAM dashboard.

Nowhere in the docs (that I could find) does it tell you what permissions are needed to deploy a app engine flexible service.

Turns out, you need:

Logs Writer
Storage Object Viewer

Without Storage Object Viewer you'll get an error on deployment telling you the exact issue. Without Logs Writer you will not get an error, but the service will never come up.

What a long 10 days...

EDIT: I was wrong, it says here in the docs what permissions you need.

I asked Google Support to file an internal bug that the correct error message is not being returned if you do not have Logs Writer

Adam Jenkins
  • 51,445
  • 11
  • 72
  • 100