Stack Overflow
Stack Overflow

Questions tagged [sampled-softmax]

6 questions
3
votes
1 answer

Keras: Using weights for NCE loss

So here is the model with the standard loss function. target = Input(shape=(1, ), dtype='int32') w_inputs = Input(shape=(1, ), dtype='int32') w_emb = Embedding(V, dim, embeddings_initializer='glorot_uniform',name='word_emb')(w_inputs) w_flat=…
pythOnometrist
  • 6,531
  • 6
  • 30
  • 50
3
votes
1 answer

Sampled Softmax in Keras Model

Some approaches I have considered: Inheriting from Model class Sampled softmax in tensorflow keras Inheriting from Layers class How can I use TensorFlow's sampled softmax loss function in a Keras model? Of the two approaches the Model approach is…
pythOnometrist
  • 6,531
  • 6
  • 30
  • 50
2
votes
0 answers

Why do loss valued increase after some epochs in sampled_softmax_loss

I'm using Tensorflow to train a word2vec skip gram model. The computation graph is in the code below: # training data self.dataset = tf.data.experimental.make_csv_dataset(file_name, batch_size=self.batch_size, column_names=['input', 'output'],…
Thuc Hung
  • 130
  • 1
  • 1
  • 7
1
vote
0 answers

What is the best way to sample one word from tens of million words at the last layer of NNs?

I have a huge NNs with tens of million Softmax probabilities at the last layer. For each given x, I need to randomly sample just one of the output nodes according to the probabilities. I use tf.random.multinomial, but its complexity is O(n), where n…
Shan
  • 53
  • 4
0
votes
1 answer

Loss function in tf.nn.sampled_softmax_loss

I have a question regarding Tensorflow: Which loss function is used in tf.nn.sampled_softmax_loss? I believe it's cross-entropy, but it is not written on the official website. Can anyone confirm my guess?
theabc50111
  • 421
  • 7
  • 16
0
votes
1 answer

To pick the most confident data samples(top 3) per class based on the softmax output

So I have trained a neural network on a particular dataset of 50 classes. During testing, I want to pick the best three samples(most confident ones, i.e: top 3 or top 5 samples) from each class in the testing dataset, How can I do that? Example: If…
Naren
  • 33
  • 5