0

I have read various research papers that one can retrofitting a fasttext model to improve its accuracy (https://github.com/mfaruqui/retrofitting). However I am having trouble on how to implement it.

The github link above, will take one vector file and retrofitting it, output another vector file. I can load it using gensim library. However, since it is a vector file, it is no longer a model and it will not predict OOV (out-of-vocabulary) words. This makes it pointless. Is there a way to retrain the model somehow so it has better accuracy?

tonywang
  • 181
  • 2
  • 13

1 Answers1

1

As far as I understand by reading the paper and browsing the repository, the proposed methodology only allows to improve the quality of the vectors (.vec) given in input.

As you can read here, fastText's ability to represent out-of-vocabulary words is inherent in the .bin model (which contains the vectors for all the n-grams).

As you too may have understood, there is no out-of-the-box way to retrofit a fastText model, using the proposed methodology.

Stefano Fiorucci - anakin87
  • 3,143
  • 7
  • 26
  • Indeed, the method only works on a fixed set of vectors - not a complete FastText model, with all of its subword info. But a thought for the asker: if there are some fixed set of OOV words of most interest to you – perhaps those named in the lexicon – you could conceivably retrieve *all* words of interest (both in-vocab & OOV) from the FT model, put those in a static `.vec` file, and try the technique on that. You'd not have a full FastText model - but you would have all your new retrofit vectors for the fixed list of words-of-interest. – gojomo Jun 26 '21 at 11:00