Delete all documents from index/type without deleting type

Question

I know one can delete all documents from a certain type via deleteByQuery.

Example:

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}'

But i have NO term and simply want to delete all documents from that type, no matter what term. What is best practice to achieve this? Empty term does not work.

Link to deleteByQuery

score 196 · Accepted Answer · edited Dec 08 '21 at 07:16

196

I believe if you combine the delete by query with a match all it should do what you are looking for, something like this (using your example):

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
    "query" : { 
        "match_all" : {}
    }
}'

Or you could just delete the type:

curl -XDELETE http://localhost:9200/twitter/tweet

Note: XDELETE is deprecated for later versions of ElasticSearch

edited Dec 08 '21 at 07:16

Anupam

14,950
19
67
94

answered May 28 '14 at 18:08

John Petrone

26,943
6
63
68

10

If you are having custom mappings; do note that the second option will delete the type and its mappings. So don't forget to remap the index type after deleting. Or else you will be messed up. – Finny Abraham Sep 15 '15 at 06:14
25

Ftr: in Elasticsearch 2.0 the [delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/docs-delete-by-query.html) has been removed from the core and now [lives in a plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/plugins-delete-by-query.html). – dtk Nov 16 '15 at 14:41
2

It is not recommended to delete records by this way.Here statement from the docs: " it is problematic since it silently forces a refresh which can quickly cause OutOfMemoryError during concurrent indexing" https://www.elastic.co/guide/en/elasticsearch/reference/1.7/docs-delete-by-query.html – usef_ksa Jan 14 '16 at 09:51
3

Ftr: The delete by query plugin will be [back into ES core](https://github.com/elastic/elasticsearch/pull/18516) as of version 5. – Val Jun 15 '16 at 06:00
1

whoever doesn't know the mapping type (`twitter` in this question), it can be retrieved by running `curl -XGET 'http://localhost:9200/_all/_mapping'`, then, in the returned json, look for `.mappings.` – yair Sep 08 '16 at 08:32
11

If you get "No handler found for uri ... " error, use curl -XPOST 'localhost:9200/twitter/tweet/_delete_by_query?conflicts=proceed&pretty' -d' { "query": { "match_all": {} } }' – Iqbal Nov 24 '16 at 09:28
1

`curl -XDELETE http://localhost:9200/twitter` if you want to delete the whole index. – tedi Nov 03 '17 at 12:50
2

_query DELETE api does not working in ElasticSearch 6+. Use _delete_by_query instead. – Shailesh Pratapwar Feb 05 '19 at 08:25

Iqbal · Answer 2 · 2021-01-11T15:49:29.660

106

The Delete-By-Query plugin has been removed in favor of a new Delete By Query API implementation in core. Read here

curl -XPOST 'localhost:9200/twitter/tweet/_delete_by_query?conflicts=proceed&pretty' -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}'

edited Jan 11 '21 at 15:49

answered Nov 24 '16 at 09:25

Iqbal

2,094
1
20
28

1

Worked for me on es 5.4 – jlunavtgrad May 10 '17 at 20:23
2

Worked for me on ES 6.1.1 – Sebastian Jul 05 '18 at 09:52
8

For ES 6+ you also need `-H 'Content-Type: application/json'` – OMRY VOLK Nov 29 '18 at 15:40

Jay Shah · Answer 3 · 2017-12-19T22:08:15.970

77

From ElasticSearch 5.x, delete_by_query API is there by default

POST: http://localhost:9200/index/type/_delete_by_query

{
    "query": { 
        "match_all": {}
    }
}

edited Dec 19 '17 at 22:08

answered May 17 '17 at 14:17

Jay Shah

3,553
1
27
26

This is nice because it works for child nodes (some of the other answers fail in that case due to "routing_missing_exception") – dnault Jan 25 '18 at 20:08

score 27 · Answer 4 · answered Jan 06 '18 at 09:56

27

You can delete documents from type with following query:

POST /index/type/_delete_by_query
{
    "query" : { 
        "match_all" : {}
    }
}

I tested this query in Kibana and Elastic 5.5.2

answered Jan 06 '18 at 09:56

Luka Lopusina

2,557
3
27
32

score 16 · Answer 5 · answered Apr 29 '15 at 07:15

16

Torsten Engelbrecht's comment in John Petrones answer expanded:

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d 
  '{
      "query": 
      {
          "match_all": {}
      }
   }'

(I did not want to edit John's reply, since it got upvotes and is set as answer, and I might have introduced an error)

answered Apr 29 '15 at 07:15

Brimstedt

3,020
22
32

1

@ChristopheRoussy No without extra plugin, see comments on John Petrone answer – rsilva4 Jul 28 '16 at 11:05

score 13 · Answer 6 · answered Apr 09 '16 at 17:09

13

Starting from Elasticsearch 2.x delete is not anymore allowed, since documents remain in the index causing index corruption.

answered Apr 09 '16 at 17:09

Fabio Fumarola

430
3
9

1

So what is the solution ? – Christophe Roussy Jun 13 '16 at 15:44
1

I use a solution based on alias for index. The main idea is to create new index each time like `news1, news2 and so on` and setup an alias for the current active index to the `news` path. Of course the name of the index is only as example. Here you can find a complete example for [ index alias](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html) and [article](https://www.elastic.co/guide/en/elasticsearch/guide/current/index-aliases.html) which explains a case study. – Fabio Fumarola Jun 18 '16 at 10:51

user1927829 · Answer 7 · 2022-10-06T13:56:54.393

13

Since ElasticSearch 7.x, delete-by-query plugin was removed in favor of new Delete By Query API.

The curl option:

curl -X POST "localhost:9200/my-index/_delete_by_query" -H 'Content-Type: application/json' -d' { "query": { "match_all":{} } } '

Or in Kibana

POST /my-index/_delete_by_query
{
  "query": {
    "match_all":{}
  }
}

edited Oct 06 '22 at 13:56

answered Jul 06 '21 at 17:42

user1927829

302
3
9

score 11 · Answer 8 · answered Mar 20 '18 at 12:00

11

The above answers no longer work with ES 6.2.2 because of Strict Content-Type Checking for Elasticsearch REST Requests. The curl command which I ended up using is this:

curl -H'Content-Type: application/json' -XPOST 'localhost:9200/yourindex/_doc/_delete_by_query?conflicts=proceed' -d' { "query": { "match_all": {} }}'

answered Mar 20 '18 at 12:00

mindas

26,463
15
97
154

1

According to the offical docs I had to remove the _doc part of the URL. – Mark Schäfer Jan 23 '19 at 09:45

score 8 · Answer 9 · answered Dec 15 '17 at 05:34

8

In Kibana Console:

POST calls-xin-test-2/_delete_by_query
{
  "query": { 
    "match_all": {}
  }
}

answered Dec 15 '17 at 05:34

Xin

33,823
14
84
85

score 5 · Answer 10 · answered Jun 11 '15 at 15:31

5

(Reputation not high enough to comment) The second part of John Petrone's answer works - no query needed. It will delete the type and all documents contained in that type, but that can just be re-created whenever you index a new document to that type.

Just to clarify: $ curl -XDELETE 'http://localhost:9200/twitter/tweet'

Note: this does delete the mapping! But as mentioned before, it can be easily re-mapped by creating a new document.

answered Jun 11 '15 at 15:31

Emmy R

124
3
10

2

But you delete all mappings configuration whats you have, it's not recommended when you have specific configuration for any mapping, because the dynamic mappings only create basics fields like string, long, etc... – Carlos Rodriguez Jan 06 '16 at 19:42
2

@CarlosRodriguez but any fancy mapping you have should surely be in source control, and very easy to automatically re-apply, as part of the same script that's doing the delete. – Jonathan Hartley May 23 '16 at 20:35
This answer directly contradicts the question: "Delete all documents ... WITHOUT deleting type". Please do not make assumptions how easy it is to recreate mapping based on YOUR project. Other projects may have more complex procedures for mapping versioning/migration/etc. – VeganHunter Sep 19 '18 at 04:55

score 5 · Answer 11 · edited Jan 17 '17 at 09:10

5

Note for ES2+

Starting with ES 1.5.3 the delete-by-query API is deprecated, and is completely removed since ES 2.0

Instead of the API, the Delete By Query is now a plugin.

In order to use the Delete By Query plugin you must install the plugin on all nodes of the cluster:

sudo bin/plugin install delete-by-query

All of the nodes must be restarted after the installation.

The usage of the plugin is the same as the old API. You don't need to change anything in your queries - this plugin will just make them work.

*For complete information regarding WHY the API was removed you can read more here.

edited Jan 17 '17 at 09:10

Anuragh27crony

2,957
1
19
29

answered Jul 19 '16 at 21:08

Dekel

60,707
10
101
129

From my experience, DeleteByQuery plugin performs very poorly with a large amount of documents. Tested with ES 2.3.2. – ibai Aug 26 '16 at 11:55
1

@ibai, I used it with ES 2.2.0 on an index contain several million documents and it didn't take long (about the same time with the original delete by query API that was in 1.7). Anyway - I guess there is no much choice here, as the API is no longer valid. – Dekel Aug 28 '16 at 01:08

score 5 · Answer 12 · edited May 08 '18 at 03:22

You have these alternatives:

1) Delete a whole index:

curl -XDELETE 'http://localhost:9200/indexName'

example:

curl -XDELETE 'http://localhost:9200/mentorz'

For more details you can find here -https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html

2) Delete by Query to those that match:

curl -XDELETE 'http://localhost:9200/mentorz/users/_query' -d                
    '{
        "query":
            {
                "match_all": {}
            }
    }'

*Here mentorz is an index name and users is a type

score 5 · Answer 13 · answered Dec 12 '19 at 12:27

I'm using elasticsearch 7.5 and when I use

curl -XPOST 'localhost:9200/materials/_delete_by_query?conflicts=proceed&pretty' -d'
{
    "query": {
        "match_all": {}
    }
}'

which will throw below error.

{
  "error" : "Content-Type header [application/x-www-form-urlencoded] is not supported",
  "status" : 406
}

I also need to add extra -H 'Content-Type: application/json' header in the request to make it works.

curl -XPOST 'localhost:9200/materials/_delete_by_query?conflicts=proceed&pretty'  -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}'
{
  "took" : 465,
  "timed_out" : false,
  "total" : 2275,
  "deleted" : 2275,
  "batches" : 3,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [ ]
}

score 3 · Answer 14 · answered Jul 15 '16 at 10:48

3

Just to add couple cents to this.

The "delete_by_query" mentioned at the top is still available as a plugin in elasticsearch 2.x.

Although in the latest upcoming version 5.x it will be replaced by "delete by query api"

answered Jul 15 '16 at 10:48

Walaitki

165
2
11

score 0 · Answer 15 · answered Apr 26 '16 at 08:48

0

Elasticsearch 2.3 the option

    action.destructive_requires_name: true

in elasticsearch.yml do the trip

    curl -XDELETE http://localhost:9200/twitter/tweet

answered Apr 26 '16 at 08:48

JJANSSEN

171
1
6

score 0 · Answer 16 · answered Sep 16 '21 at 14:52

For future readers:

in Elasticsearch 7.x there's effectively one type per index - types are hidden
you can delete by query, but if you want remove everything you'll be much better off removing and re-creating the index. That's because deletes are only soft deletes under the hood, until the trigger Lucene segment merges*, which can be expensive if the index is large. Meanwhile, removing an index is almost instant: remove some files on disk and a reference in the cluster state.

* The video/slides are about Solr, but things work exactly the same in Elasticsearch, this is Lucene-level functionality.

score -1 · Answer 17 · answered Mar 07 '18 at 08:33

If you want to delete document according to a date. You can use kibana console (v.6.1.2)

POST index_name/_delete_by_query
{
      "query" : {
              "range" : {
                 "sendDate" : {
                     "lte" : "2018-03-06"
                              }
                        }
                  }
}

Delete all documents from index/type without deleting type

17 Answers17

Linked