1

I'm trying to run a docker container with an application that uses some GCP services.

I have created and downloaded a service account json key, and I can run the application outside docker when I run 

export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/project-credentials-123.json"

before running the application.

However, when I run the application inside docker, the GCP client libraries can't find the default access keys.

The docker run command I'm using looks like:

docker run -p 8501:8501 -e GOOGLE_APPLICATION_CREDENTIALS=/tmp/keys/project-credentials-123.json -v $GOOGLE_APPLICATION_CREDENTIALS:/tmp/keys/project-credentials-123.json:ro 571a38aaaf15 streamlit run src/app/app.py

where 571a38aaaf15 is the docker image id.

Note that I copied my project-credentials-123.json to /tmp/keys/ to better follow https://cloud.google.com/run/docs/testing/local

Note: similar to Set GOOGLE_APPLICATION_CREDENTIALS in Docker, but I can't comment there due to < 50 reputation.

Myccha
  • 961
  • 1
  • 11
  • 20
  • 1
    If you start your container and then run a shall inside it, can you run your command manually from a shell prompt insider the container? – Kolban Dec 09 '19 at 04:02
  • What are the symptoms / messages from the actual error? – Kolban Dec 09 '19 at 04:03
  • I'm running a python web app, so the only info is that the GOOGLE_APPLICATION_CREDENTIALS could not be detected. The main thing is that I want to share a run command with colleagues, so I don't want them to enter the container interactively to set the path within it. – Myccha Dec 09 '19 at 04:37
  • Howdy ... for debugging, can you setup a running instance of your container and then get to a shell insider the container for testing purposes. Then, manually run the command that is failing and report back the detailed description of the error reported? – Kolban Dec 09 '19 at 04:41
  • Thank you @Kolban, this inspired the answer I shared below (it seems to be working). Do you see any issues? – Myccha Dec 09 '19 at 04:47

3 Answers3

0

I think I figured it out.

The correct format was:

docker run -v /tmp/keys/:/tmp/keys -p 8501:8501 571a38aaaf15 export GOOGLE_APPLICATION_CREDENTIALS="/tmp/keys/project-credentials-123.json" | streamlit run src/app/app.py

My undestanding of what is happening is that

  1. I have a file /tmp/keys/project-credentials-123.json on my laptop, and the -v flag is saying "mount the /tmp/keys/ directory of my machine on the container's /tmp/keys/ directory in the container".
  2. The first command I'm running in the container is exporting this file as my credentials.

P.S. Please let me know of any better way of achieving this.

Myccha
  • 961
  • 1
  • 11
  • 20
  • Where your container will be run? On GCP or elsewhere? – guillaume blaquiere Dec 09 '19 at 08:18
  • The plan is to deploy it in GCP (and most of this will be irrelevant), but most of my team is developing locally, so I had to solve this issue to get them to buy into using docker. – Myccha Dec 09 '19 at 11:19
  • I have follow your guide and I got error like this "java.lang.NullPointerException", can you have me please? – Cobe9696 Apr 01 '22 at 07:02
0

If your container are hosted on GCP, you won't need to mount a volume and set the key like this because you could use the identity of the underlying platform.

When I have to use container locally and I have to inject the keyfile in it, I inject directly the content of the keyfile, without mounting a volume. Like this, i keep my container stateless and compliant with Cloud Run and Knative.

FROM....
....
ENV GOOGLE_APPLICATION_CREDENTIALS /tmp/key.json

# Run the web service on container startup.
CMD if [ -z $KEY ] ; then export  GOOGLE_APPLICATION_CREDENTIALS=; else echo $KEY > /tmp/key.json;fi  && ....your usual command...

You to set to NULL the GOOGLE_APPLICATION_CREDENTIALS else Google library will look if exists before looking into the underlying platform identity.

And then, I run my command like this

Docker run -e KEY="$(cat key.json)" <TAG>
guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • Hi @guillaume, thanks for this idea. I'm wondering though, won't the image still contain a reference to the key even if you've cleared out the environment variable at runtime? Wouldn't this be a security issue? I've read that best practices generally advocate for not building images with state in them and to instead do it at runtime. What do you think? – jamesmortensen Feb 18 '23 at 05:21
0

Streamlit allows for the use of google application credentials as outlined within its documentation. Instead of passing a location of the credentials, you can simply add these credentials to a secrets.toml file when deploying. See below for eg.

    [gcp_service_account]
    type= "xxx"
    project_id= "xxx"
    private_key_id= "xxx"
    private_key= """xxx"""
    client_email= "xxx"
    client_id= "xxx"
    auth_uri= "xxx"
    token_uri= "xxx"
    auth_provider_x509_cert_url= "xxx"
    client_x509_cert_url= "xxx"
Doracahl
  • 336
  • 2
  • 14