Specify signature name on Vertex AI Predict

Question

I've deployed a tensorflow model in vertex AI platform using TFX Pipelines. The model have custom serving signatures but I'm strugling to specify the signature when I'm predicting.

I've the exact same model deployed in GCP AI Platform and I'm able to specify it.

According to the vertex documentation, we must pass a dictionary containing the Instances (List) and the Parameters (Dict) values.

I've submitted these arguments to this function:

instances: [{"argument_n": "value"}]

parameters: {"signature_name": "name_of_signature"}

Doesn't work, it still get the default signature of the model.

In GCP AI Platform, I've been able to predict directly specifying in the body of the request the signature name:

response = service.projects().predict(
        name=name,
        body={"instances": instances,
        "signature_name": "name_of_signature"},
    ).execute()

@EDIT I've discovered that with the rawPredict method from gcloud it works.

Here is an example:

!gcloud ai endpoints raw-predict {endpoint} --region=us-central1 \
--request='{"signature_name":"name_of_the_signature", \
"instances": [{"instance_0": ["value_0"], "instance_1": ["value_1"]}]}'

Unfortunately, looking at google api models code it only have the predict method, not the raw_predict. So I don't know if it's available through python sdk right now.

Since you are using a custom trained model using TensorFlow, could you share the model as it will help to replicate the scenario. — Sandeep Mohanty, Nov 06 '21 at 12:41
Hello @SandeepMohanty, unfortunately my company doesn't allow me to share the model. I can share the signatures if it help. I've edited the post with some new information. — Artur Lunardi Di Fante, Nov 06 '21 at 12:52
@ArturLunardiDiFante Did you try same signature with different model because that particular signature might work for other model due to **Invalid reduction dimensions** and are you passing values as tensors in model? — , Nov 07 '21 at 01:58
@TensorflowSupport I did. Actually that is nothing wrong with the signature, it is working perfectly, the problem is that I can't specify the signature in the predict method of vertex AI the way I did on AI Platform. I've found a workaround, that is to define my custom signature as the default serving, but the problem it is that doesn't allow me to use the Evaluator Component, since the examples generated from ExampleGen are parsed tf.Examples, and my signature expect named tensors within a dict. — Artur Lunardi Di Fante, Nov 07 '21 at 22:06

Sandeep Mohanty · Accepted Answer · 2021-11-09T10:18:28.543

Vertex AI is a newer platform with limitations that will be improved over time. “signature_name” can be added to HTTP JSON Payload in RawPredictRequest or from gcloud as you have done but right now this is not available in regular predict requests.

Using HTTP JSON payload :

Example:

input.json :

{
   "instances": [
     ["male", 29.8811345124283, 26.0, 1, "S", "New York, NY", 0, 0],
     ["female", 48.0, 39.6, 1, "C", "London / Paris", 0, 1]],
 
     "signature_name": <string>
}

curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/endpoints/${ENDPOINT_ID}:rawPredict \
-d "@input.json"

Specify signature name on Vertex AI Predict

1 Answers1

Linked