30

I have just started using Word2vec and I was wondering how can we find the closest word to a vector suppose. I have this vector which is the average vector for a set of vectors:

array([-0.00449447, -0.00310097, 0.02421786, ...], dtype=float32)

Is there a straight forward way to find the most similar word in my training data to this vector?

Or the only solution is to calculate the cosine similarity between this vector and the vectors of each word in my training data, then select the closest one?

Thanks.

sel
  • 942
  • 1
  • 12
  • 25

3 Answers3

48

For gensim implementation of word2vec there is most_similar() function that lets you find words semantically close to a given word:

>>> model.most_similar(positive=['woman', 'king'], negative=['man'])
[('queen', 0.50882536), ...]

or to it's vector representation:

>>> your_word_vector = array([-0.00449447, -0.00310097, 0.02421786, ...], dtype=float32)
>>> model.most_similar(positive=[your_word_vector], topn=1))

where topn defines the desired number of returned results.

However, my gut feeling is that function does exactly the same that you proposed, i.e. calculates cosine similarity for the given vector and each other vector in the dictionary (which is quite inefficient...)

Nicolas Ivanov
  • 798
  • 9
  • 15
  • Can you please explain what is `positive` and `negative` parameter for `most_similar()` method? I just started word2vec. – Om Prakash Mar 14 '18 at 12:56
  • 3
    `positive` param indicates those words that should be similar to the word you are looking for. `negative` - the words that should be different from your word. `most_similar()` function will return the word that best satisfies these criteria. According to [the docs](https://code.google.com/archive/p/word2vec/) the authors of word2vec have shown that the word vectors capture some linguistic regularities, for example vector operations `vector('king') - vector('man') + vector('woman')` is close to `vector('queen')`. That's the motivation for the example above. – Nicolas Ivanov Mar 14 '18 at 14:48
  • Does this just perform a linear scan over the vectors in the model? If so, why not just iterate yourself? – duhaime Apr 09 '19 at 15:44
9

Don't forget to add empty array with negative words in most_similar function:

import numpy as np
model_word_vector = np.array( my_vector, dtype='f')
topn = 20;
most_similar_words = model.most_similar( [ model_word_vector ], [], topn)
Andrew Krizhanovsky
  • 610
  • 7
  • 11
9

Alternatively, model.wv.similar_by_vector(vector, topn=10, restrict_vocab=None) is also available in the gensim package.

Find the top-N most similar words by vector.

Parameters:

  • vector (numpy.array) – Vector from which similarities are to be computed.

  • topn ({int, False}, optional) – Number of top-N similar words to return. If topn is False, similar_by_vector returns the vector of similarity scores.

  • restrict_vocab (int, optional) – Optional integer which limits the range of vectors which are searched for most-similar values. For example, restrict_vocab=10000 would only check the first 10000 word vectors in the vocabulary order. (This may be meaningful if you’ve sorted the vocabulary by descending frequency.)

Returns: Sequence of (word, similarity).

Return type: list of (str, float)

Moobie
  • 1,445
  • 14
  • 21