0

I am writing a python program to get the data from the elastic search indices. I want to get the data based on the match query which i indicated is upto 25. I want the first 25 data's. The data's in my index is 10842. But it retrieves all the data from the index of the elastic search. I checked solution from here matchall query for es but it doesn't helped me. Help me with some solutions

Here's the code:

from elasticsearch import Elasticsearch
import elasticsearch.helpers

count = 0
host = 'localhost'
ind = 'apps'
doc_typ = "change_apps"
limit_count = 25

def elasticsearch_import(host,ind,doc_typ,count,limit_count,port=9200,query={},single_line=False,single_line_label="message"):
    data_count=count+limit_count
    print("Data to be get from Elastic Search: ",data_count)
    es = Elasticsearch()

    results = elasticsearch.helpers.scan(es,
    index=ind,
    doc_type=doc_typ,
    preserve_order=True,
    query={"from":count,"size":data_count,"query": {"bool": {"must": [{"match_all": {}}],"must_not": [],"should": [] }},})
    res=[]
    for i in results:
        res.append(i)
    #print("res",res)
    print("Data got from Elastic Search",len(res))

elasticsearch_import(host,ind,doc_typ,count,limit_count)

Output i got:

Data to be get from Elastic Search:  25
Data got from Elastic Search 10842

Required Output:

Data to be get from Elastic Search:  25
Data got from Elastic Search 25
Smack Alpha
  • 1,828
  • 1
  • 17
  • 37

1 Answers1

2

This is what the scan method does... it uses scroll method under the hood, if you look into the api documentation, the size is actually means batch size.

size – size (per shard) of the batch send at each iteration.

If you simply want to get a result with a size, search is good enough, in this case, the size is the result size, and the default value is 10.

ch33hau
  • 2,811
  • 1
  • 15
  • 15
  • the data limit in search i got was 9990. How to increase the limit. the limit goes maximum of 9990 only. I can fetch upto 9990 data only at a single shoot. – Smack Alpha Jun 06 '19 at 12:48
  • Are you getting the error of `from` + `size` cannot be large than `10000`? This is by design for `search` api, if you need something more than 10000, then you should use `scroll` or `scan`. For your 9990's problem, it is probably you have `from` = 10, so with `from`=0, and `size`=10000 you will get all 10000 results. – ch33hau Jun 06 '19 at 14:44