FastText quantize documentation incorrect?

Question

I'm unable to run FastText quantization as shown in the documentation. Specifically, as shown at the bottom of the cheat sheet page:

https://fasttext.cc/docs/en/cheatsheet.html

When I attempt to run quantization on my trained model "model.bin":

./fasttext quantize -output model

the following error is printed to the shell:

Empty input or output path.

I've reproduced this problem with builds from the latest code (September 14 2018) and older code (June 21 2018). Since the documented command syntax isn't working, I tried adding an input argument:

./fasttext quantize -input [file] -output model

where [file] is either my training data or trained model. Unfortunately both tries resulted in a segmentation fault with no error message from FastText.

What is the correct command syntax to quantize a FastText model? Also, is it possible to both train and quantize a model in a single run of FastText?

If your trained model is "model.bin", so pass it: `./fasttext quantize -output model.bin` — Nick, Nov 22 '18 at 05:33
Hi did you get a solution to this problem? I tried to quantize a pre-trained word embedding following the command and I get the same error... — Ziqi, Jul 07 '19 at 19:55
Yes, I figured it out after banging on it for quite a while. The documentation was incorrect. Here's an example with some optional arguments: `./fasttext quantize -input training_examples -output model_without_extension -cutoff 2000000 -dsub 8 -retrain` The critical issue is the model filename (model_without_extension). This must be the model filename without an extension. Otherwise it will load the wrong file and segfault. — J-Dinerstein, Jul 09 '19 at 16:14

score 2 · Answer 1 · answered Mar 21 '19 at 13:00

2

Solution in Python:

# Quantize the model with retraining
model.quantize(input=train_data, qnorm=True, retrain=True, cutoff=200000)

# Save quantized model
model.save_model("model_quantized.bin")

answered Mar 21 '19 at 13:00

Flavio

759
1
11
24

score 0 · Answer 2 · answered Jul 11 '19 at 06:04

0

I tried this one worked:

 ./fasttext quantize -input <training set> -output <model name (no suffix) -[options]

answered Jul 11 '19 at 06:04

jayden.ycui

1

score -1 · Answer 3 · answered Mar 21 '19 at 12:35

-1

This is the example that is included in the quantization-example.sh

./fasttext quantize -output "${RESULTDIR}/dbpedia" -input "${DATADIR}/dbpedia.train" -qnorm -retrain -epoch 1 -cuto$

answered Mar 21 '19 at 12:35

Flavio

759
1
11
24

FastText quantize documentation incorrect?

3 Answers3