Solr: To get all records

Question

I am trying to upgrade my Solr 4.x version to 5.2.1 Solrcloud implementation. I had written following code to get all the results from Sorl query which works well in Solr single instance mode.

SolrQuery query = new SolrQuery();
query.setQuery("*:*");
query.addSort("agent_status", ORDER.desc);
query.addFilterQuery("account_id:\"" + accountId + "\"");
query.set("rows", Integer.MAX_VALUE);

But code will not work well in SolrCloud implemenation.It throws following exception.

2015-08-14 16:44:45,648 ERROR [solr.core.SolrCore] - [http-8080-8] : java.lang.NegativeArraySizeException
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:58)
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:39)
at org.apache.solr.handler.component.ShardFieldSortedHitQueue.<init>(ShardDoc.java:113)
at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:972)
at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:750)
at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:729)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)

I found that it is failing because of query.set("rows", Integer.MAX_VALUE) statement.People suggested me to use pagination. But, I can not afford doing pagination as there will be too many changes at UI side.

There is one more way where I can first query with some small number & get total number of documents using response.getResults().getNumFound() method & try setting that value to setRows method.But this approach will increase one more call to server.

Is there any other way I can solve this problem?

http://stackoverflow.com/questions/10039778/how-to-get-all-results-from-solr-query http://stackoverflow.com/questions/6385168/get-all-the-results-from-solr-without-10-as-limit http://stackoverflow.com/questions/4662060/in-solr-is-there-an-way-to-fetch-all-results-from-a-query-something-like-start-0 — Mauricio Scheffer, Aug 17 '15 at 22:40

score 1 · Answer 1 · edited May 23 '17 at 12:22

You can always set your rows to be a large value that would encompass your results. Integer.MAX_VALUE will not work due to the size limits of Java Arrays (see here) and the Lucene Priority Queue (see lines 42 - 58).

Solr-534 requested to have essentially what your asking for; there is some good conversation about why and why-not such a feature would be good.

A better question might be how many documents can the UI hold without becoming unusable? However many documents that is, would be a good value for your query to return.

Solr: To get all records

1 Answers1