How to create model_config file for TFX Serving?

Question

I have a bunch of custom defined model instances inheriting from tf.keras.layers.Layer that I save. I would like to serve them with TFX Serving, which requires me to have a model_config file.

I am wondering how to create this according to the book. Right now I have the following code which I believe is more about my own bricolage than what I am supposed to do...

model_server_config = model_server_config_pb2.ModelServerConfig()

    #Create a config to add to the list of served models
    config_list = model_server_config_pb2.ModelConfigList()       
        
    for i in range(0,len(trainable_unit_name)): # add models one by one to the model config.    
        model_name  = name[i]
        base_path = "/models/{}".format(name[i])
        
        one_config = config_list.config.add()
        
        one_config.name           = model_name
        one_config.base_path      = base_path
        one_config.model_platform ="tensorflow"
        
    model_server_config.model_config_list.MergeFrom(config_list)
    with open(C.CONF_FILEPATH, 'w+') as f:        
        f.write("model_config_list {" + config_list.__str__() + "}") #manually wrap it around "model_config_list { .." because this is the required format by TFX Serving.

score 1 · Answer 1 · answered Oct 23 '20 at 11:54

Complete information regarding Tensorflow Serving Configuration is present in the link, https://www.tensorflow.org/tfx/serving/serving_config.

The answer to your question, how to create model_config_file for Serving, can be found in this TF Serving Github Repository.

For more information about PB Files, refer this Stack Overflow Question.

Providing the Github Code below, just in case the above Github Link doesn't work:

syntax = "proto3";

package tensorflow.serving;
option cc_enable_arenas = true;

import "google/protobuf/any.proto";
import "tensorflow_serving/config/logging_config.proto";
import "tensorflow_serving/sources/storage_path/file_system_storage_path_source.proto";

// The type of model.    
enum ModelType {
  MODEL_TYPE_UNSPECIFIED = 0 [deprecated = true];
  TENSORFLOW = 1 [deprecated = true];
  OTHER = 2 [deprecated = true];
};

// Common configuration for loading a model being served.
message ModelConfig {
  // Name of the model.
  string name = 1;
      
  repeated string alias = 9;

  string base_path = 2;

  // Type of model.
  // TODO(b/31336131): DEPRECATED. Please use 'model_platform' instead.
  ModelType model_type = 3 [deprecated = true];

  // Type of model (e.g. "tensorflow").
  //
  // (This cannot be changed once a model is in serving.)
  string model_platform = 4;

  reserved 5;

  // Version policy for the model indicating which version(s) of the model to
  // load and make available for serving simultaneously.
  // The default option is to serve only the latest version of the model.
  //
  // (This can be changed once a model is in serving.)
  FileSystemStoragePathSourceConfig.ServableVersionPolicy model_version_policy =
      7;

  // String labels to associate with versions of the model, allowing inference
  // queries to refer to versions by label instead of number. Multiple labels
  // can map to the same version, but not vice-versa.
  map<string, int64> version_labels = 8;

  // Configures logging requests and responses, to the model.
  //
  // (This can be changed once a model is in serving.)
  LoggingConfig logging_config = 6;
}

// Static list of models to be loaded for serving.
message ModelConfigList {
  repeated ModelConfig config = 1;
}

// ModelServer config.
message ModelServerConfig {
  // ModelServer takes either a static file-based model config list or an Any
  // proto representing custom model config that is fetched dynamically at
  // runtime (through network RPC, custom service, etc.).
  oneof config {
    ModelConfigList model_config_list = 1;
    google.protobuf.Any custom_model_config = 2;
  }
}

How to create model_config file for TFX Serving?

1 Answers1