Azure Pipeline maven task not picking up GCP credentials

Question

I have created an Azure pipeline that should auth with a GCP service account and do the maven test step. In the test I have a BigQuery client directly connecting to the BigQuery datasets, querying it and doing some assertions, that's why I need the credentials.

It is all working fine on my local machine, where I have pointe GOOGLE_APPLICATION_CREDENTIALS to the .json containing the service account key.

Even in the pipeline, everything works fine with the key (I've checked it on some terraform steps and they are picking up the credentials provided in such a way), except the maven command.

Below is the code I'm using:

Azure pipeline:

steps:
  - task: Bash@3
    displayName: Copy GCP Service Account Key
    inputs:
      workingDirectory: ${{parameters.working_drectory}}
      targetType: 'inline'
      script: 'echo ${{parameters.credentials}} | base64 -d > svc.json'

  - task: laurensknoll.google-cloud-sdk-tasks.gcloud-runner.GcloudRunner@0
    displayName: 'gcloud auth activate-service-account'
    inputs:
      command: 'auth activate-service-account'
      arguments: '--key-file svc.json'
      workingDirectory: ${{parameters.working_drectory}}

  - task: laurensknoll.google-cloud-sdk-tasks.gcloud-runner.GcloudRunner@0
    displayName: 'gcloud config set project'
    inputs:
      command: 'config set project'
      arguments: ${{parameters.project}}
      workingDirectory: ${{parameters.working_drectory}}

  - task: MavenAuthenticate@0
    # This task will authenticate your maven feed for input deps and output deps
    inputs:
      artifactsFeeds: $(incomingFeedName)

  - task: Maven@3
    # The version in the POM has to be set to the 'correct value' which is defined by line 2 of this file
    # Note that this changes the pom, so the cache key in the step called 'Cache Maven' will change, and we will need to change this back later
    inputs:
      mavenPomFile: 'samples/testproject/pom.xml'
      goals: 'test'
      mavenAuthenticateFeed: true
    displayName: Build test and set the version, package

relavant POM part:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven-surefire-plugin.version}</version>
        <configuration>
            <parallel>all</parallel>
            <threadCount>4</threadCount>
            <redirectTestOutputToFile>true</redirectTestOutputToFile>
            <systemPropertyVariables>
                <WSNSHELL_HOME>GOOGLE_APPLICATION_CREDENTIALS</WSNSHELL_HOME>
            </systemPropertyVariables>
        </configuration>
        <dependencies>
            <dependency>
                <groupId>org.apache.maven.surefire</groupId>
                <artifactId>surefire-junit47</artifactId>
                <version>${maven-surefire-plugin.version}</version>
            </dependency>
        </dependencies>
    </plugin>

I have tried as well as passing the GOOGLE_APPLICATION_CREDENTIALS as an option to maven with -DGOOGLE_APPLICATION_CREDENTIALS=svc.json but that hasn't worked either

Just to recap, So you have tested your setup on your machine and it worked. You perform a terraform plan and everything went fine. You execute the plan and its not working on the target machine, right?. You will always need your credentials to perform GCP operations. Also, have you try authenticated trough code? (see [link](https://cloud.google.com/docs/authentication/production#windows)) If so, the environment reference may not hitting your code. (about java setup see this [link](https://cloud.google.com/java/docs/setup)) — Betjens, Mar 08 '22 at 08:11
Not exactly, the terraform plan runs successfully on the target machine (GCP env). All is fine with terraform and everything works. The issue is when I try to execute `mvn test` step after the terraforms steps. This steps is the one that is failing. I've tried passing the credentials to maven directly but it didnt work. I've made it work by moving the service account json to the codebase and auth through java code by using that service account, but I don't want to have credentials in the codebase. I want to use the `GOOGLE_APPLICATION_CREDENTIALS` env variable for the maven step as well. — ivanm, Mar 08 '22 at 08:34
Ohh so you already try using it trough code and worked. Then it means that the file path is working and there should be something on your maven configuration that is causing the issue. Have you try printing the environment variable you are passing? or maybe using it on a small code test? as mention on this [link](https://stackoverflow.com/questions/5510690/environment-variable-with-maven) — Betjens, Mar 08 '22 at 08:47
Yes, I've checked this link. If you take a look at the second part of my question (the pom) you'll notice it's following the answer from the link you have provided. I'll try printing the env variables, but that shouldn't be an issue. — ivanm, Mar 08 '22 at 12:26
Well the env variables should be actually point to your `credentials.json` file if its not properly setup your code will give your errors. By the way are you getting error messages from your `maven test` run? — Betjens, Mar 08 '22 at 12:34
Yes, it's saying `Authorization failed`. So I have this variable, and I point to my `svc.json` which contains my service account. It works locally, but it doesnt work in the pipeline .. — ivanm, Mar 08 '22 at 13:59
Ok, then that means that is not reaching your `svc.json` when its in the pipeline. You have to make sure that the file you are making reference in the pipeline instance exists and its being loaded. I don't see anything else failing. As you already test with code credentials and its working, locally with your `svc.json` is working too so that's the only option left. — Betjens, Mar 08 '22 at 16:07
Hi Betjens, the issue still persists. If you take a look at the azure pipeline I have shared the first step generates the json file with the credentials. The issue is it seems it's not loaded into maven somehow, and thus it fails — ivanm, Mar 10 '22 at 09:28

score 0 · Accepted Answer · answered Mar 11 '22 at 09:01

The issue was that the credentials json file wasn't present in the maven working directory, meaning the maven working directory and other steps working directory wasn't the same.

The issue was fixed by copying the credentials json file to the maven working directory.

Azure Pipeline maven task not picking up GCP credentials

1 Answers1