How to use word2vec to calculate the similarity distance by giving 2 words?

Question

Word2vec is a open source tool to calculate the words distance provided by Google. It can be used by inputting a word and output the ranked word lists according to the similarity. E.g.

Input:

france

Output:

            Word       Cosine distance

            spain              0.678515
          belgium              0.665923
      netherlands              0.652428
            italy              0.633130
      switzerland              0.622323
       luxembourg              0.610033
         portugal              0.577154
           russia              0.571507
          germany              0.563291
        catalonia              0.534176

However, what I need to do is to calculate the similarity distance by giving 2 words. If I give the 'france' and 'spain', how can I get the score 0.678515 without reading the whole words list by giving just 'france'.

score 60 · Accepted Answer · edited Jun 28 '21 at 08:24

60

gensim has a Python implementation of Word2Vec which provides an in-built utility for finding similarity between two words given as input by the user. You can refer to the following:

Intro: http://radimrehurek.com/gensim/models/word2vec.html
Tutorial: http://radimrehurek.com/2014/02/word2vec-tutorial/

UPDATED: Gensim 4.0.0 and above

The syntax in Python for finding similarity between two words goes like this:

>> from gensim.models import Word2Vec
>> model = Word2Vec.load(path/to/your/model)
>> model.wv.similarity('france', 'spain')

edited Jun 28 '21 at 08:24

Jinhua Wang

1,679
1
17
44

answered Aug 21 '14 at 06:04

Satarupa Guha

1,267
13
20

If helpful in this context, see [here](http://radimrehurek.com/gensim/models/word2vec.html) for using vector files created with the original word2vec c language implementation. – matanster Oct 30 '16 at 16:28
2

@Satarupa Guha: what is this similarity fonction? is it cosine? – Chedi Bechikh Jan 20 '17 at 21:23
Yes, it is indeed cosine similarity function. – Satarupa Guha Oct 06 '17 at 02:38

score 39 · Answer 2 · edited Oct 30 '16 at 17:13

39

As you know word2vec can represent a word as a mathematical vector. So once you train the model, you can obtain the vectors of the words spain and france and compute the cosine distance (dot product).

An easy way to do this is to use this Python wrapper of word2vec. You can obtain the vector using this:

>>> model['computer'] # raw numpy vector of a word
array([-0.00449447, -0.00310097, 0.02421786, ...], dtype=float32)

To compute the distances between two words, you can do the following:

>>> import numpy    
>>> cosine_similarity = numpy.dot(model['spain'], model['france'])/(numpy.linalg.norm(model['spain'])* numpy.linalg.norm(model['france']))

edited Oct 30 '16 at 17:13

matanster

15,072
19
88
167

answered Feb 26 '14 at 09:42

phyrox

2,423
15
23

Word2vec gets the cosine distance result,and does the numpy.dot just multiple 2 vectors other than cosine distance? – zhfkt Feb 27 '14 at 15:46
2

similarity function can get the same result in simple way http://radimrehurek.com/gensim/models/word2vec.html – zhfkt Feb 28 '14 at 13:03

score 5 · Answer 3 · answered Mar 18 '16 at 16:52

I just stumbled on this while looking for how to do this by modifying the original distance.c version, not by using another library like gensim.

I didn't find an answer so I did some research, and am sharing it here for others who also want to know how to do it in the original implementation.

After looking through the C source, you will find that 'bi' is an array of indexes. If you provide two words, the index for word1 will be in bi[0] and the index of word2 will be in bi[1].

The model 'M' is an array of vectors. Each word is represented as a vector with dimension 'size'.

Using these two indexes and the model of vectors, look them up and calculate the cosine distance (which is the same as the dot product) like this:

dist = 0;
for (a = 0; a < size; a++) {
    dist += M[a + bi[0] * size] * M[a + bi[1] * size];
}

after this completes, the value 'dist' is the cosine similarity between the two words.

score 0 · Answer 4 · answered Aug 19 '15 at 14:18

0

I have developed a code to help with calculating cosine similarity for 2 sentences / SKUs using gensim. The code can be found here https://github.com/aviralmathur/Word2Vec

The code is using data for Kaggle competition on Crowdflower

It has been developed using Code for Kaggle Tutorial on Word2Vec available here https://www.kaggle.com/c/word2vec-nlp-tutorial

I hope this helps

answered Aug 19 '15 at 14:18

Aviral Mathur

27
3

12

Welcome to StackOverflow, and thanks for answering this question. We don't want answers to only be links, because if the link ever changes or the host goes down, your answer becomes useless. Please edit this answer and include the main points from the code you've linked in this answer. – Aaron D Aug 19 '15 at 14:24
1

@AaronD Too bad Stack Exchange doesn't use https://web.archive.org to mirror links. – Franck Dernoncourt Nov 24 '18 at 22:08

score 0 · Answer 5 · answered Sep 10 '21 at 05:34

If you look at the source code of the Gensim's native method to calculate word similarities, you will find that it calculates word similarities using the following method:

import numpy as np
from gensim import matutils  # utility fnc for pickling, common scipy operations etc

def similarity_cosine(vec1, vec2):
    cosine_similarity = np.dot(matutils.unitvec(vec1), matutils.unitvec(vec2))
    return cosine_similarity

similarity_cosine(model.wv['space'], model.wv['france'])

How to use word2vec to calculate the similarity distance by giving 2 words?

5 Answers5

Linked