Stack Overflow
Stack Overflow

Questions tagged [quantization]

Use this tag for questions related to quantization of any kind, such as vector quantization.

Quantization, in mathematics and digital signal processing, is the process of mapping a large set of input values to a (countable) smaller set.

For more, please read the Wikipedia article for more.

444 questions
20
votes
3 answers

Quantize a Keras neural network model

Recently, I've started creating neural networks with Tensorflow + Keras and I would like to try the quantization feature available in Tensorflow. So far, experimenting with examples from TF tutorials worked just fine and I have this basic working…
sikr_
  • 369
  • 1
  • 7
  • 19
19
votes
2 answers

Generate color palette from image with ImageMagick

I need to generate a color palette of the top 5 dominant colors in an image. I'd like to replicate the results of Embedly's extract tool or Color Thief. The following command gives me slightly different results: convert testimage.jpg -format %c…
Ben Foster
  • 34,340
  • 40
  • 176
  • 285
13
votes
2 answers

Color banding only on Android 4.0+

On emulators running Android 4.0 or 4.0.3, I am seeing horrible colour banding which I can't seem to get rid of. On every other Android version I have tested, gradients look smooth. I have a SurfaceView which is configured as RGBX_8888, and the…
threeshinyapples
  • 181
  • 1
  • 5
11
votes
2 answers

Convert/Quantize Float Range to Integer Range

Say I have a float in the range of [0, 1] and I want to quantize and store it in an unsigned byte. Sounds like a no-brainer, but in fact it's quite complicated: The obvious solution looks like this: unsigned char QuantizeFloat(float a) { return…
Nils Pipenbrinck
  • 83,631
  • 31
  • 151
  • 221
11
votes
3 answers

How to calculate audio file size?

You have 30 seconds audio file sampled at a rate of 44.1 KHz and quantized using 8 bits ; calculate the bit rate and the size of mono and stereo versions of this file ؟؟
Osama Al-far
  • 427
  • 3
  • 10
  • 24
10
votes
2 answers

tflite quantized inference very slow

I am trying to convert a trained model from checkpoint file to tflite. I am using tf.lite.LiteConverter. The float conversion went fine with reasonable inference speed. But the inference speed of the INT8 conversion is very slow. I tried to debug by…
wuhy08
  • 341
  • 3
  • 7
10
votes
1 answer

What are the Tensorflow qint8, quint8, qint32, qint16, and quint16 datatypes?

I'm looking at the Tensorflow tf.nn.quantized_conv2d function and I'm wondering what exactly the qint8, etc. dataypes are, particularly if they are the datatypes used for the "fake quantization nodes" in tf.contrib.quantize or are actually stored…
albertNod
  • 103
  • 1
  • 6
10
votes
2 answers

Understanding tf.contrib.lite.TFLiteConverter quantization parameters

I'm trying to use UINT8 quantization while converting tensorflow model to tflite model: If use post_training_quantize = True, model size is x4 lower then original fp32 model, so I assume that model weights are uint8, but when I load model and get…
mrgloom
  • 20,061
  • 36
  • 171
  • 301
10
votes
0 answers

Is it possible to use tf.contrib.quantize.create_training_graph with Keras model?

Is it possible to use tf.contrib.quantize.create_training_graph for model quantiation with already trained Keras model? As I understand I can import tf.Graph from Keras model, but can I finetune it after modification with…
mrgloom
  • 20,061
  • 36
  • 171
  • 301
9
votes
4 answers

Edge TPU Compiler: ERROR: quantized_dimension must be in range [0, 1). Was 3

I'm trying to get a Mobilenetv2 model (retrained last layers to my data) to run on the Google edge TPU Coral. I've followed this tuturial https://www.tensorflow.org/lite/performance/post_training_quantization?hl=en to do the post-training…
Paulo Ribeiro
  • 153
  • 9
9
votes
1 answer

Generate the Dominant Colors for an RGB image with XMLHttpRequest

A Note For Readers: This is a long question, but it needs a background to understand the question asked. The color quantization technique is commonly used to get the dominant colors of an image. One of the well-known libraries that do color…
loretoparisi
  • 15,724
  • 11
  • 102
  • 146
8
votes
0 answers

How can I quantize facenet's Inception-ResNet-v1 model in Tensorflow?

What I want to do I'm trying to create a quantized version of the Inception-ResNet-v1 model used in facenet - with not only quantized weights, but quantized nodes as well, according to Tensorflow's graph_transform guide. What I have tried Using a…
hannabe
  • 156
  • 5
7
votes
3 answers

How do I extract the weights of my quantized model for use on hardware?

EDIT: attaching some code to help generate similar results (appended at end) I have a really small model with architecture [2, 3, 6] where the hidden layer uses ReLU and it's a softmax activation for multiclass classification. Trained offline and…
Chrispresso
  • 3,660
  • 2
  • 19
  • 31
7
votes
1 answer

You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,784] for MNIST dataset

Here is the example I am testing on MNIST dataset for quantization. I am testing my model using below code: import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data from tensorflow.python.framework import graph_util from…
Shringa Bais
  • 191
  • 2
  • 11
7
votes
0 answers

Why my Tensorflow network becomes slower after applying the Quantization tools on GeForce GTX 1080?

I tried the quantization tools in Tensorflow with a toy model. It surely reduced the model to about 25% percent, however, increase the executing time by many times. GPU is fully used when both model runs. So I am wondering what is wrong? I guess…
samuel
  • 346
  • 2
  • 8
1
2 3
…
29 30