Google Cloud Speech to text API call fails due to invalid authentication credentials on Android

Question

I'm trying to use the google cloud speech to text API in my android client. I have enabled the API in the Google Cloud and in the console I have generate a new key like this: "Create credentials" -> "Service account" (added details) -> "Create and Continue". This has generated a JSON file that contains this params:

{
  "type": "service_account",
  "project_id": "",
  "private_key_id": "",
  "private_key": "",
  "client_email": "",
  "client_id": "",
  "auth_uri": "",
  "token_uri": "",
  "auth_provider_x509_cert_url": "",
  "client_x509_cert_url": "",
  "universe_domain": ""
}

I have added this JSON file in the raw dir in my android project (this is just a POC so not to worried about security at this point)

Next is the android code:

val req = RecognizeRequest.newBuilder()
               .setConfig(RecognitionConfig.newBuilder()
                        .setEncoding(RecognitionConfig.AudioEncoding.AMR_WB)
                        .setLanguageCode("en-US")
                        .setSampleRateHertz(16000)
                        .build())
                    .setAudio(RecognitionAudio.newBuilder()
                        .setContent(fileByteString))
                        .build()


            val speechClient = SpeechClient.create(SpeechSettings.newBuilder()              .setCredentialsProvider{GoogleCredentials.fromStream(this.resources.openRawResource(R.raw.credentials)) }


            val response = speechClient.recognize(req)

Log.d(TAG, "we have a count of ${response.resultsCount} hits on the  audio file")

            val results = response.resultsList

When the speechClient.recognize(req) fun is called, the API sends back this error message:

com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
        at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:98)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
        at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
        at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127)
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286)
        at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055)
        at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807)
        at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:574)
        at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:544)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
        at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
        at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
        at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567)
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
        at java.lang.Thread.run(Thread.java:764)
     Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
        at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
        at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
        at com.google.cloud.speech.v1.SpeechClient.recognize(SpeechClient.java:252)

Does anyone know how to auth correctly when calling this API please?

I have also tried to generate OAuth & Service Key type of credentials in the Google Cloud Console but I was getting a different error saying something like "Credentials don't contain field `type`"

jccampanero · Answer 1 · 2023-07-15T22:08:03.377

Please, consider review this related SO question, it is defined for TextToSpeechSettings but I think that the solution provided could be of application here as well.

As suggested, basically you need to create a valid GoogleCredentials object from the content of your service account json file:

// R.raw.credential points to the downloaded credential.json file
InputStream stream = getResources().openRawResource(R.raw.credential); 
GoogleCredentials credentials = GoogleCredentials.fromStream(stream);

And then, use the obtained credentials to initialize your SpeechClient.

For instance, adapting the code from my original answer:

import java.io.IOException;
import java.io.InputStream;
import java.util.List;

import com.google.api.gax.core.CredentialsProvider;
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.speech.v1.RecognitionAudio;
import com.google.cloud.speech.v1.RecognitionConfig;
import com.google.cloud.speech.v1.RecognizeRequest;
import com.google.cloud.speech.v1.RecognizeResponse;
import com.google.cloud.speech.v1.SpeechClient;
import com.google.cloud.speech.v1.SpeechRecognitionResult;
import com.google.cloud.speech.v1.SpeechSettings;
import com.google.protobuf.ByteString;

public class RecognitionActivity extends Activity {

  // Your activity definition

  private void performRecognition(ByteString fileByteString) throws IOException {
    // R.raw.credential points to the downloaded credential.json file
    // I tested my code locally in a laptop using
    // GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream("/path/to/credentials.json"));
    InputStream stream = getResources().openRawResource(R.raw.credential);
    GoogleCredentials credentials = GoogleCredentials.fromStream(stream);
    // You can use ServiceAccountCredentials as well instead of the line above:
    // ServiceAccountCredentials credentials = ServiceAccountCredentials.fromStream(stream);
    CredentialsProvider credentialsProvider = FixedCredentialsProvider.create(credentials);
    SpeechSettings speechSettings = SpeechSettings.newBuilder()
        .setCredentialsProvider(credentialsProvider)
        .build();

    SpeechClient speechClient = SpeechClient.create(speechSettings);

    // The rest of your code

    RecognizeRequest req = RecognizeRequest.newBuilder()
        .setConfig(RecognitionConfig.newBuilder()
            .setEncoding(RecognitionConfig.AudioEncoding.AMR_WB)
            .setLanguageCode("en-US")
            .setSampleRateHertz(16000)
            .build())
        .setAudio(RecognitionAudio.newBuilder()
            .setContent(fileByteString))
        .build();

    RecognizeResponse response = speechClient.recognize(req);

    Log.d(TAG, "we have a count of " + response.getResultsCount() + " hits on the  audio file")

    List<SpeechRecognitionResult> results = response.getResultsList();
    
    // Handle results
  }
}

In kotlin (please, forgive me for any inaccuracy) it would look like this:

private fun performRecognition(fileByteString: ByteString) {
    // R.raw.credential points to the downloaded credential.json file
    val stream: InputStream = getResources().openRawResource(R.raw.credential)
    val credentials = GoogleCredentials.fromStream(stream);
    // val credentials = ServiceAccountCredentials.fromStream(stream)
    val credentialsProvider = FixedCredentialsProvider.create(credentials)
    val speechSettings = SpeechSettings.newBuilder()
            .setCredentialsProvider(credentialsProvider)
            .build()

    val speechClient = SpeechClient.create(speechSettings)

    // The rest of your code
    val req = RecognizeRequest.newBuilder()
            .setConfig(RecognitionConfig.newBuilder()
                    .setEncoding(RecognitionConfig.AudioEncoding.AMR_WB)
                    .setLanguageCode("en-US")
                    .setSampleRateHertz(16000)
                    .build())
            .setAudio(RecognitionAudio.newBuilder()
                    .setContent(fileByteString))
            .build()

    val response = speechClient.recognize(req)

    Log.d(TAG, "we have a count of ${response.resultsCount} hits on the  audio file")

    val results = response.resultsList

    // Handle results
}

Please, be sure that you followed the necessary steps for setting up the Speech API, especially those related to configuring the service account.

The service account must be able to interact with the Speech API, assigning a convenient role to it such as Cloud Speech Client:

As described in the aforementioned documentation you can do it from the GCP Console or using the command line.

Hi @jccampanero! Thank you for your response! I have tried your proposed fix but unfortunately the setCredentialsProvider does not like the FixedCredentialsProvider type and is expecting a type Credentials. — android enthusiast, Jul 12 '23 at 09:52
Thank you for the feedback @androidenthusiast. It is very strange, I think it should work properly, `setCredentialsProvider` requires a `CredentialsProvider`. I updated my code to make the code snippet more complete (sorry, I implemented it in pure Java): please, could you double check the proposed solution and see if it works? Thank you. — jccampanero, Jul 12 '23 at 19:12
@androidenthusiast Sorry for disturbing you. Did your program compile in the end? Are you still facing the issue? Can I help you in any way? — jccampanero, Jul 16 '23 at 17:48

Google Cloud Speech to text API call fails due to invalid authentication credentials on Android

1 Answers1