13

I´m using Mac OS el capitán and I am trying to follow the quick start tutorial for OpenNMT pytorch version. In the training step I get the following warning message:

OpenNMT-py/onmt/modules/GlobalAttention.py:177: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument. 

align_vectors = self.sm(align.view(batch*targetL, sourceL))
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/container.py:67: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)

Step 1: Preprocess data (works as expected)

python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo

Step 2: Train model (produces the warning message)

python train.py -data data/demo -save_model demo-model

Has anyone come across this warning or have any pointers to solve it?

prosti
  • 42,291
  • 14
  • 186
  • 151
secuaz
  • 459
  • 3
  • 6
  • 15
  • Maybe [this](https://github.com/pytorch/pytorch/issues/3238) applies here. – Wiktor Stribiżew Feb 27 '18 at 10:48
  • I think the warning doesn´t stop execution, it just takes a long time for each epoch: – secuaz Feb 27 '18 at 11:13
  • I got this output recently, right after the warning: Epoch 1, 50/ 157; acc: 4.21; ppl: 167145.54; 37 src tok/s; 36 tgt tok/s; 2049 s elapsed Epoch 1, 100/ 157; acc: 5.52; ppl: 8718.19; 37 src tok/s; 37 tgt tok/s; 3901 s elapsed – secuaz Feb 27 '18 at 11:14

2 Answers2

16

It is almost always you will need the last dimension when you compute the cross-entropy so your line may look like:

torch.nn.functional.log_softmax(x, -1)
prosti
  • 42,291
  • 14
  • 186
  • 151
9

From the warning it's pretty clear that you have to explicitly mention the dimension since implicit dimension choice for softmax has been deprecated.

In my case, I'm using log_softmaxand I've changed below line of code to include dimension.

torch.nn.functional.log_softmax(x) # This throws warning.

is changed to 

torch.nn.functional.log_softmax(x, dim = 1) # This doesn't throw warning.
Harish Mashetty
  • 433
  • 3
  • 13