nextCursorMark missing in deep paging using Solr with composite unique key

Question

I am using DataStax Cassandra and Solr

Cassandra 2.0.10.71 | DSE 4.6.0 | CQL spec 3.1.1 | Thrift protocol 19.39.0

I have a Cassandra column family as:

    CREATE TABLE kSpace.colfam1 (
      id text,
      date timestamp,
      desc text,
      origin set<text>,
      PRIMARY KEY ((id), date)
    ) WITH CLUSTERING ORDER BY (date DESC);

My Solr's schema.xml looks like:

    <schema name="kSpace.colfam1" version="1.5">
      <types>
        <fieldType name="string" class="solr.StrField"/>
        <fieldType name="StringCollectionField" class="solr.StrField" multiValued="true"/>
        <fieldType name="tdate" class="solr.TrieDateField" precisionStep="0" positionIncrementGap="0"/>

        <fieldType name="text" class="solr.TextField">
            <analyzer>
              <tokenizer class="solr.StandardTokenizerFactory"/>
                  <filter class="solr.LowerCaseFilterFactory"/>
            </analyzer>
          </fieldType>
      </types>

        <fields>
          <field name="id" type="string" indexed="true" stored="true" required="true"/>
          <field name="date"  type="tdate" indexed="true" />
          <field name="desc" type="text" indexed="true" />
          <field name="origin" type="StringCollectionField" indexed="true" />
        </fields>

        <defaultSearchField>desc</defaultSearchField>
        <uniqueKey>(id,date)</uniqueKey>
    </schema>

When I query using Solr using following query:

    http://localhost:8983/solr/kSpace.colfam1/select?
      q=*%3A*
      &sort=id+asc%2C+date+desc
      &rows=2
      &wt=json
      &indent=true
      &cursorMark=*

the response comes back, but it does not have any nextCursorMark:

    {
      "responseHeader": {
        "status": 0,
        "QTime": 1,
        "params": {
          "sort": "id asc, date desc",
          "indent": "true",
          "q": "*:*",
          "_": "1429004324135",
          "cursorMark": "*",
          "wt": "json",
          "rows": "2"
        }
      },
      "response": {
        "numFound": 284901,
        "start": 0,
        "docs": [
          {
            "_uniqueKey": "[\"000047bc-921d-4487-b5f3-c70520e0a7bf\",\"1428601411276\"]",
            "id": "000047bc-921d-4487-b5f3-c70520e0a7bf",
            "date": "2015-04-09T17:43:31.276Z",
            "desc": "description1 description2"
          },
          {
            "_uniqueKey": "[\"0000531e-efee-42b4-9c52-136e9a106827\",\"1428601409625\"]",
            "id": "0000531e-efee-42b4-9c52-136e9a106827",
            "date": "2015-04-09T17:43:29.625Z",
            "desc": "description3 description4"
          }
        ]
      }
    }

I am not sure that your definition of the `uniqueKey` is valid, have a read here http://stackoverflow.com/questions/24454755/how-to-set-multiple-fields-as-uniquekey-in-solr that would explain why the cursorMark remains absent, since you do not have a valid `uniqueKey` — cheffe, Apr 14 '15 at 14:07

score 1 · Answer 1 · answered Apr 14 '15 at 14:09

1

Deep paging is available as of Solr 4.7.

DSE 4.6.x has Solr 4.6. You will have to wait to a future release (likely 4.7) to have access to this functionality.

answered Apr 14 '15 at 14:09

phact

7,305
23
27

mbeacom · Answer 2 · 2015-04-24T15:08:13.593

Rahul,

As phact stated, the DataStax Enterprise major release 4.6.x will not include Solr 4.7 (The earliest release of Apache Solr to support deep paging, which is when 'nextCursorMark' was introduced.

You can obtain a copy of DSE 4.7 Early Access Program release via http://eap.datastax.com/ (which I wouldn't suggest for Production use, but it does have Solr 4.7).

In order to allow paging in Solr in earlier versions (Apache Solr < 4.7), you'll have to hack around paging. Requesting a grouped result with a limit on 'row' and a set 'start' index could be a viable alternative.

Check out the official example

Using: rows and start

Example:

?q=yourquery&rows=10&start=0 Then ?q=yourquery&rows=10&start=10

It's not pretty, but until DSE 4.7 is released for public consumption, this is a workable alternative.

nextCursorMark missing in deep paging using Solr with composite unique key

2 Answers2