1

Introduction

I'm using Elastic Search (v5.x) and trying to delete documents, by query.

My index called "data". The documents are stored in hierarchic structure. Documents URL built in this pattern:

https://server.ip/data/{userid}/{document-id}

So, let's say the user-id '1' have two documents stored ('1', '2'). So, their direct URL will be:

https://server.ip/data/1/1
https://server.ip/data/1/2

Target

Now, what I'm trying to do is to delete the user from the system (the user and his stored documents).

The only way that worked for me is to send HTTP DELETE request for each document URL. Like this:

DELETE https://server.ip/data/1/1
DELETE https://server.ip/data/1/2

This is working. But, in this solution I have to call delete multiple times. I want to delete all the documents in one call. So, this solution is rejected.

My first try was to send HTTP DELETE request to

https://server.ip/data/1

Unfortently, it's not working (error code 400).

My second try was to use the _delete_by_query function. Each document that I'm stored is containing the UserId field, which contain the UserId. So, I tried to make a delete query for removing all the documents, in 'data' index, that containing the field with the value 1 ('UserId'==1)

POST https://server.ip/data/_delete_by_query
{
   "query":{
      "match":{
         "UserId":"1"
      }
   }
}

This also not working. The response was HTTP Error Code 400 with this body:

{
   "error":{
      "root_cause":[
         {
            "type":"invalid_type_name_exception",
            "reason":"Document mapping type name can't start with '_'"
         }
      ],
      "type":"invalid_type_name_exception",
      "reason":"Document mapping type name can't start with '_'"
   },
   "status":400
}

Do you know how to solve those problems? Maybe do you have alternative solution?

Thank you!

No1Lives4Ever
  • 6,430
  • 19
  • 77
  • 140

1 Answers1

0

I assume you've got your document_type defined in your logstash conf something like this within your output>elasticsearch:

output {        
        elasticsearch {         
            index => "1"
            document_type => "1type"
            hosts => "localhost"            
        }

        stdout {
             codec => rubydebug
        }
    }

Hence you could simply delete all the documents which has the same type:

curl -XDELETE https://server.ip/data/1/1type

OR try something like this if you're willing to use delete by query:

POST https://server.ip/data/_delete_by_query?UserId=1
{
  "query": {
    "match_all": {}
  }
}

This could be an absolute gem of a source. Hope it helps!

Community
  • 1
  • 1
Kulasangar
  • 9,046
  • 5
  • 51
  • 82
  • _delete_by_query not working for me. The response is 400. Reason: "reason=Document mapping type name can't start with '_'" – No1Lives4Ever Feb 02 '17 at 12:54
  • Try this: `curl -XDELETE http://localhost:9200/index/type` Make sure to rename the index and the type. – Kulasangar Feb 02 '17 at 12:58
  • Not working. Error code 400. Reason= No handler found for uri [/data/1] and method [DELETE] – No1Lives4Ever Feb 02 '17 at 13:00
  • Are you sure that you have the **document_type** being set as *1* in your `logstash`? – Kulasangar Feb 02 '17 at 13:01
  • Pretty sure. GET /data/1/2 is returning the document. This mean that the document_type is "1". I'm wrong? I'm also checked it with GET /data/_mapping and found the "1" in the returning types. – No1Lives4Ever Feb 02 '17 at 13:07