Post request tensorflow serving: too many values to unpack (expected 2)

Question

I set up a tensorflow running service with my model, but when I try to do a post request it returns me the following error (get request work):

[nltk_data] Downloading package punkt to /home/viktor/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.

(3, 20)
Traceback (most recent call last):
  File "bert_api.py", line 99, in <module>
    res , res2= requests.post('http://3.143.108.46:8501/v1/models/colbert:predict', 
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 516, in request
    prep = self.prepare_request(req)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 449, in prepare_request
    p.prepare(
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 317, in prepare
    self.prepare_body(data, files, json)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 508, in prepare_body
    body = self._encode_params(data)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 97, in _encode_params
    for k, vs in to_key_val_list(data):
ValueError: too many values to unpack (expected 2)

This is my code:

res, res2 = requests.post('http://url:port/v1/models/colbert:predict', 
    data=sentence_input)
print(res.status_code, res.reason)

my data_sentence is an array of arrays:

 [array([[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
       dtype=int32),
 array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=int32]

`res , res2` do you have two variables? In `requests.post ` the payoad should be a dict not an array — Ajay, Apr 04 '21 at 18:19

Marian · Accepted Answer · 2021-04-07T07:39:26.383

There are two separate issues in your code. One pertains to the payload, the other to the way you are using requests.post.

Requests usage

requests.post, just as requests.request and other similar functions, returns a single instance of Response class (source). For this reason, to fix your error you need to change from

res, res2 = requests.post(...

to

res = requests.post(

Now, typically if you try to take one thing and unpack it into two variables, you would get a more clear error saying that there are not enough values to unpack:

>>> a,b = [1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not enough values to unpack (expected 2, got 1)

The error you are getting actually says the opposite - that you have too many values to unpack. And here's why.

The Response class returned by requests.post defines an __iter__(self) method (source), which will allow you to iterate over the response in chunks instead of reading it all at once. So when you do res, res2 = response, Python will try to unpack the response for you by using the __iter__ implementation. If you have more than 2 chunks in your response, you won't have enough variables to handle the other chunks, so you get the error "too many values to unpack".

A simple demonstration of this behaviour:

>>> class X():
...     def __iter__(self):
...             for i in range(5):
...                     yield i
...
>>> x = X()
>>> a,b = x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
>>> list(x)
[0, 1, 2, 3, 4]

Payload

When you pass in the data keyword argument to requests.post, it assumes that you want to form-encode it (docs). You should either pass a dictionary or a list of tuples, but what you have is a list of arrays, which isn't a form at all, so you don't want to form-encode it.

So when you are passing (data=list_of_arrays), the library expects the first element to be a tuple with two elements. Instead you have an array with much more elements, which again results in the same error: too many values to unpack.

What you want to do instead is to send it as JSON or just as a string. Typically, you would have to follow the docs and do

import json

payload = [np.random.randint(2, size=30).reshape(3,10), np.random.randint(2, size=30).reshape(3,10)]

res = requests.post('/', data=json.dumps(payload)) # pass it as a string

res = requests.post('/', json=payload) # or let the library serialize it

Unfortunately, you will then run into the issue where np.array isn't serializable, so you have to prepare it for serialization first. As per this answer, you can just call .tolist() on np.array which will serialize it to a list of integers for you.

Thus the final solution

import json

payload = [np.random.randint(2, size=30).reshape(3,10), np.random.randint(2, size=30).reshape(3,10)]

# convert both np.arrays to list
payload = [payload[0].tolist(), payload[1].tolist()]
# or this way
payload = [arr.tolist() for arr in data]

# now the payload is serializable
res = requests.post('/', data=json.dumps(payload)) # pass it as a string

res = requests.post('/', json=payload) # or let the library serialize it

score 0 · Answer 2 · answered Apr 06 '21 at 17:10

Looking at the error message i see that you are assigning two variables res, res2 to requests.post response that has more than 2 returning values

#for example 
var1, var2 = [1, 2, 3]
#this will raise the same error as your's

To solve this problem you need to index your response like this below

res, res2 = requests.post('http://url:port/v1/models/colbert:predict', 
    data=sentence_input)[:2]
print(res.status_code, res.reason)

If response to your post request return a list, indexing will solve your problem

Post request tensorflow serving: too many values to unpack (expected 2)

2 Answers2

Requests usage

Payload