How to save the model for text-classification in tensorflow?

Question

Reading tensorflow documentation for text-classification, I have put up a script below that I used to train a model for text classification (positive/negative). I am not sure on one thing. How could I save the model to reuse it later? Also, how can I test for the input test-set I have?

import tensorflow as tf
import tensorflow_hub as hub
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import re
import seaborn as sns



# Load all files from a directory in a DataFrame.
def load_directory_data(directory):
  data = {}
  data["sentence"] = []
  data["sentiment"] = []
  for file_path in os.listdir(directory):
    with tf.gfile.GFile(os.path.join(directory, file_path), "r") as f:
      data["sentence"].append(f.read())
      data["sentiment"].append(re.match("\d+_(\d+)\.txt", file_path).group(1))
  return pd.DataFrame.from_dict(data)

# Merge positive and negative examples, add a polarity column and shuffle.
def load_dataset(directory):
  pos_df = load_directory_data(os.path.join(directory, "pos"))
  neg_df = load_directory_data(os.path.join(directory, "neg"))
  pos_df["polarity"] = 1
  neg_df["polarity"] = 0
  return pd.concat([pos_df, neg_df]).sample(frac=1).reset_index(drop=True)

# Download and process the dataset files.
def download_and_load_datasets(force_download=False):
  dataset = tf.keras.utils.get_file(
      fname="aclImdb.tar.gz", 
      origin="http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz", 
      extract=True)

  train_df = load_dataset(os.path.join(os.path.dirname(dataset), 
                                       "aclImdb", "train"))
  test_df = load_dataset(os.path.join(os.path.dirname(dataset), 
                                      "aclImdb", "test"))

  return train_df, test_df

# Reduce logging output.
tf.logging.set_verbosity(tf.logging.ERROR)

train_df, test_df = download_and_load_datasets()
train_df.head()


# Training input on the whole training set with no limit on training epochs.
train_input_fn = tf.estimator.inputs.pandas_input_fn(
    train_df, train_df["polarity"], num_epochs=None, shuffle=True)

# Prediction on the whole training set.
predict_train_input_fn = tf.estimator.inputs.pandas_input_fn(
    train_df, train_df["polarity"], shuffle=False)
# Prediction on the test set.
predict_test_input_fn = tf.estimator.inputs.pandas_input_fn(
    test_df, test_df["polarity"], shuffle=False)


embedded_text_feature_column = hub.text_embedding_column(
    key="sentence", 
    module_spec="https://tfhub.dev/google/nnlm-en-dim128/1")


estimator = tf.estimator.DNNClassifier(
    hidden_units=[500, 100],
    feature_columns=[embedded_text_feature_column],
    n_classes=2,
    optimizer=tf.train.AdagradOptimizer(learning_rate=0.003))

# Training for 1,000 steps means 128,000 training examples with the default
# batch size. This is roughly equivalent to 5 epochs since the training dataset
# contains 25,000 examples.
estimator.train(input_fn=train_input_fn, steps=1000);

train_eval_result = estimator.evaluate(input_fn=predict_train_input_fn)
test_eval_result = estimator.evaluate(input_fn=predict_test_input_fn)

print "Training set accuracy: {accuracy}".format(**train_eval_result)
print "Test set accuracy: {accuracy}".format(**test_eval_result)

Currently, if I run the above script it retrains the complete model. I want to reuse the model and have it output for some sample texts that I have. How could I do this?

I have tried the following to save:

sess = tf.Session()
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver()
saver.save(sess, 'test-model')

but this throws an error, saying Value Error: No variables to save

I had tried https://www.tensorflow.org/api_docs/python/tf/train/Saver and done `saver.save(estimator, 'my-model', global_step=0)` but it gives an error saying `no variables to save` — John Aristo, May 22 '18 at 19:42
@rhaertel80 Saw your answer at [this SO post](https://stackoverflow.com/questions/46098863/how-to-import-an-saved-tensorflow-model-train-using-tf-estimator-and-predict-on?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa) As I try to use the method you have specified, I get an error saying `Feature sentence is not in features dictionary`" — John Aristo, May 22 '18 at 20:18

score 0 · Answer 1 · answered Dec 05 '18 at 20:00

You can train and predict on a saved/loaded Estimator model simply by passing the model_dir parameter to both the Estimator instance and a tf.estimator.RunConfig instance that is passed to the config parameter of pre-made estimators (since about Tensorflow 1.4--still works on Tensorflow 1.12):

        model_path = '/path/to/model'
        run_config = tf.estimator.RunConfig(model_dir=model_path,
                                            tf_random_seed=72,  #Default=None
                                            save_summary_steps=100,
                                            # save_checkpoints_steps=_USE_DEFAULT,  #Default=1000
                                            # save_checkpoints_secs=_USE_DEFAULT,  #Default=60
                                            session_config=None,
                                            keep_checkpoint_max=12,  #Default=5
                                            keep_checkpoint_every_n_hours=10000,
                                            log_step_count_steps=100,
                                            train_distribute=None,
                                            device_fn=None,
                                            protocol=None,
                                            eval_distribute=None,
                                            experimental_distribute=None)
        classifier = tf.estimator.DNNLinearCombinedClassifier(
            config=run_config,
            model_dir=model_path,
            ...
        )

You'll then be able to call classifier.train() and classifier.predict(), re-run the script skipping the classifier.train() call, and receive the same results after again calling classifier.predict().

This works using a hub.text_embedding_column feature column, and when using categorical_column_with_identity and embedding_column feature columns with a manually saved/restored VocabularyProcessor dictionary.

How to save the model for text-classification in tensorflow?

1 Answers1