9

I'm using TF 1.4. My question is about tf.estimator.Estimator.

I'd like to control the frequency of the "loss and step" Info messages, like:

INFO:tensorflow:loss = 0.00896569, step = 14901 (14.937 sec)

I'm passing a tf.estimator.RunConfig to the Estimator's constructor. But I don't think there is a parameter to control the "loss and step" messages.

I think the parameter is hard-coded in estimator.py, in the _train_model method:

      worker_hooks.extend([
      training.NanTensorHook(estimator_spec.loss),
      training.LoggingTensorHook(
          {
              'loss': estimator_spec.loss,
              'step': global_step_tensor
          },
          every_n_iter=100)
  ])
Olivier Moindrot
  • 27,908
  • 11
  • 92
  • 91
Michael Rubinstein
  • 91
  • 1
  • 4

2 Answers2

6

log_step_count_steps is supported in tensorflow v1.8: https://www.tensorflow.org/api_docs/python/tf/estimator/RunConfig

Lucien Wang
  • 61
  • 1
  • 5
  • 3
    To clarify, you have to pass in a custom `RunConfig` with this property set to the Estimator constructor, `tf.estimator.Estimator(...config=tf.estimator.RunConfig(log_step_count_steps=500))`. – kennysong Jul 07 '18 at 12:09
0

try returning the logging_hook as training_hook param in returned estimator_spec for mode == 'train' Printing extra training metrics with Tensorflow Estimator

https://github.com/tensorflow/tensorflow/pull/619/commits/48603b7faed85753ab905f177cbf4e0c8d1dcb64

https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository

src: https://stackoverflow.com/a/38097276/2218905

Jason
  • 1,974
  • 24
  • 19
  • 2
    I tried your suggestion. It adds a new log to the existing log hooks. Unfortunately, it doesn't allow me to replace or adjust the frequency of the default log hook that is added, unconditionally, in estimator.py. – Michael Rubinstein Dec 24 '17 at 15:09
  • well the logging is in the C++, so it requires rebuilding from source. so just disable logging (set to 'WARN' or ascending severity) and log another way – Jason Mar 22 '18 at 03:55
  • I appreciate your help. However, it doesn't accomplish my goal. – Michael Rubinstein Mar 23 '18 at 16:28
  • see updated with c++ fix when building from source, i havent tried this – Jason Mar 28 '18 at 08:11
  • I don't think this is related to the c++ implementation. As you can see from the original question, the LoggingTensorHook is added in estimator.py. My complaint is that the user of Estimators has no control over the presence or frequency of the logging. – Michael Rubinstein Mar 30 '18 at 17:27