Elasticsearch update document in python

Question

I am trying to update document in elasticsearch python,

from elasticsearch import Elasticsearch
from datetime import datetime
from scapy.all import *
import json
# es = Elasticsearch()
es = Elasticsearch(['http://localhost:9200'])

doc = getDoc("1")
    print doc
    if doc != None:
      doc['_op_type'] = 'update'
      doc['_source']['macList'].append('new')
      helpers.bulk(es, doc, stats_only=False)

But this is not working. Can someone please tell me what I am doing wrong ? thanks

This is the error:

Traceback (most recent call last):
  File "./req.py", line 48, in <module>
    helpers.bulk(es, doc, stats_only=False)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 188, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 160, in streaming_bulk
    for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 89, in _process_bulk_chunk
    raise e
elasticsearch.exceptions.RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: index is missing;2: type is missing;3: index is missing;4: type is missing;5: index is missing;6: type is missing;7: index is missing;8: type is missing;9: index is missing;10: type is missing;11: index is missing;12: type is missing;13: index is missing;14: type is missing;')

Update I have tryed this : es.update(index='macs', doc_type='users', id="1", body=doc) and I am having this error: Traceback (most recent call last):

  File "./req.py", line 50, in <module>
    es.update(index='macs', doc_type='users', id="1", body=doc)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 460, in update
    doc_type, id, '_update'), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 329, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 109, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 108, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: script or doc is missing;').

It is so easy with postman but I really don't understand why it is so had with Elasticsearch (python) to update doc. Someone has an other idea to help ?

Do you get some errors? – Nicolas Heimann Jul 06 '16 at 14:41 — Nicolas Heimann, Jul 06 '16 at 14:41

score 2 · Accepted Answer · answered Jul 06 '16 at 15:22

2

Since the second argument in helpers.bulk() method is should be an iterable, try enclosing it in a list.

Also looking at your code, it seems you are expecting only one document from the getDoc() function. So it would be better if you use simple es.update() method instead of using helpers.bulk()

answered Jul 06 '16 at 15:22

rajat

864
1
9
23

thank for your response: I have tryed this :`es.update(index='macs', doc_type='users', id="1", body=doc)`, I got example from http://stackoverflow.com/questions/30598152/how-to-update-a-document-using-elasticsearch-py be style not working – theCode Jul 06 '16 at 16:10
check if your id is correct. Elasticsearch assigns a random id to all the documents, so if you aren't manually setting it to "1", 1 cannot be a id. Further you may want to try 'doc_as_upsert'=True setting in es.update() section. Also can you post what your doc looks like or possibly the mapping – rajat Jul 06 '16 at 17:13
The problem is that elasticsearch update is not accepting unicode dict. Is there any workaround ? – theCode Jul 06 '16 at 17:50
See if you have made the same mistake as in this question [link] (http://stackoverflow.com/questions/30598152/how-to-update-a-document-using-elasticsearch-py) – rajat Jul 06 '16 at 17:55
when I enter something like this `body={"stanford": 1, "parsed_sents": parsed }` it works fine, but when I make es.get(...), the doc is not working anymore (it is unicode dict). I thing the problem is about unicode – theCode Jul 06 '16 at 18:04
that's not how you write, look at the answer !!!!!!!!!!!. It should be body body={"doc": {"stanford": 1, "parsed_sents": parsed }}.Unicode works with elasticsearch. I dont get what you mean by the doc is not working anymore – rajat Jul 06 '16 at 18:05

score 2 · Answer 2 · edited May 23 '17 at 12:01

2

This solution did work for me.

doc= ess.get(...) 
# modify my doc and ...   
coll = Elasticsearch()
coll.update(index='stories-test',doc_type='news',id=hit.meta.id,
                    body= {"doc": doc['_source']} )

edited May 23 '17 at 12:01

Community

1
1

answered Jul 06 '16 at 18:40

theCode

1,997
4
18
28

score 0 · Answer 3 · answered Jul 14 '17 at 14:26

(In case someone lands up here with a similar problem,) I was getting the first error mentioned above:

elasticsearch.exceptions.RequestError: TransportError(400, 
'action_request_validation_exception', 'Validation Failed: 
1: index is missing;2: type is missing;

and the following fixed it for me:

obj.to_dict(include_meta=True)

When converting the obj to dict, I added include_meta=True and it worked for me.

Elasticsearch update document in python

3 Answers3