0

Just after adding an item in my database, I do a request to add and commit this item in SOLR.
It works fine but I wanted to stress test my code.
So I simulate 10 persons which, at the same time, add/update 10 items in 1 second. When I see the result, some of this documents are not indexed correctly. Some of them are not updated (SOLR display an old value).
I think if a request comes when an other request is not finished, SOLR simply ignore the request.

Is there a way to queue the requests ?

FYI I have 2M+ documents with 100+ properties and the request to add and commit one document takes less than 50ms.

The request url I made :

http://localhost:8983/solr/mycore/dataimport?command=full-import&clean=false&commit=true&where=WHERE Id='{my item id}'

My SQL data config :

<document> 
    <entity name="asset" 
        query="         
        SELECT 
            Id,
            Name,
            Code,
            ShortDescription,
            Description
        FROM table
        ${dataimporter.request.where}
        "
    >
        <field column="Name" name="name" />
        <field column="Code" name="code" />
        <field column="ShortDescription" name="shortDescription" />
        <field column="Description" name="description" />
    </entity>
</document>
KoosMos
  • 41
  • 3
  • Check the [accepted answer here](https://stackoverflow.com/questions/43600860/how-does-concurrentupdatesolrclient-handle-update-request). You seem to have some reading to do (pay particular attention to how commit/rollbacks work and [ConcurrentUpdateSolrClient](https://solr.apache.org/docs/8_9_0/solr-solrj/index.html?org/apache/solr/client/solrj/impl/ConcurrentUpdateSolrClient.html) if you're planning for multi-threaded update operations). Hope this helps, happy reading! – nitrin0 Sep 22 '21 at 16:41
  • I'm not sure if you're able to run multiple instances of the same DIH task at the same time. The best solution would be to drop DIH and instead index the documents through the regular http indexing interface (`/update`). – MatsLindh Sep 23 '21 at 07:59
  • Thank you for your responses. I am going to try with the update http interface instead of DIH. – KoosMos Sep 23 '21 at 09:28

0 Answers0