How to update multiple documents in Solr with JSON?

Question

How to update multiple documents in Solr 4.5.1 with JSON? I tried this but it does not work:

POST /solr/mycore/update/json:

{
  "commit": {},
  "add": {
    "overwrite": true,
    "doc": [{
        "thumbnail": "/images/404.png",
        "url": "/404.html?1",
        "id": "demo:/404.html?1",
        "channel": "demo",
        "display_name": "One entry",
        "description": "One entry is not enough."
      }, {
        "thumbnail": "/images/404.png",
        "url": "/404.html?2",
        "id": "demo:/404.html?2",
        "channel": "demo",
        "display_name": "Another entry",
        "description": "Another entry is required."
      }
    ]
  }
}

score 9 · Accepted Answer · answered Nov 27 '13 at 15:47

9

Solr expects one "add"-key in the JSON-structure for each document (which might seem weird, if you think about the original meaning of the key in the object), since it maps directly to the XML format when doing the indexing - and this way you can have metadata for each document by itself.

{
    "commit": {},
    "add": {
        "doc": {
            "id": "321321",
            "name": "barfoo"
        }
    },
    "add": {
        "doc": {
            "id": "123123",
            "name": "Foobar"        
        }
    }
}

.. works. I think allowing an array as the element referenced by "add" would make more sense, but I haven't dug further into the source or know the reasoning behind this.

answered Nov 27 '13 at 15:47

MatsLindh

49,529
4
53
84

Thanks. This is an invalid JSON format, right? There is no chance to produce this output but doing it hand coded. – burnersk Nov 28 '13 at 06:10
It passes some online validators, but http://www.freeformatter.com/json-validator.html explains that "The JSON input is NOT valid according to RFC 4627 (JSON specification). Unexpected duplicate key:add at position 136." – CodeManX Nov 26 '14 at 22:06
1

I get an error: "Error parsing JSON field value. Unexpected OBJECT_START at [50], field=add" – JustAC0der Jan 17 '19 at 08:44
1

@JustAC0der Please add a new question for any new issues - that way it'll be far more visible instead of just popping up in my comment feed. – MatsLindh Jan 17 '19 at 10:11

score 5 · Answer 2 · answered Jan 31 '14 at 15:29

I understand that (at least) from versions 4.0 and older of solr, this has been fixed. Look at http://wiki.apache.org/solr/UpdateJSON.

In ./exampledocs/books.json there is an example of a json file with multiple documents.

[
{
"id" : "978-0641723445",
"cat" : ["book","hardcover"],
"name" : "The Lightning Thief",
"author" : "Rick Riordan",
"series_t" : "Percy Jackson and the Olympians",
"sequence_i" : 1,
"genre_s" : "fantasy",
"inStock" : true,
"price" : 12.50,
"pages_i" : 384
}
,
{
"id" : "978-1423103349",
"cat" : ["book","paperback"],
"name" : "The Sea of Monsters",
"author" : "Rick Riordan",
"series_t" : "Percy Jackson and the Olympians",
"sequence_i" : 2,
"genre_s" : "fantasy",
"inStock" : true,
"price" : 6.49,
"pages_i" : 304
}, 
...
]

While @fiskfisk answer is still a valid JSON, it is not easy to be serializable from a data structure. This one is.

score 3 · Answer 3 · answered Apr 26 '16 at 20:28

elachell is correct that the array format will work if you are just adding documents with the default settings. Unfortunately, that won't work if, for instance, you need to add a custom boost to some of the documents or change the overwrite setting. You then have to use the full object structure with an "add" key for each of them, which as they pointed out, makes this frustratingly annoying to try to serialize from most languages which don't allow the same key more than once in an object:

{
"commit": {},
"add": {
    "doc": {
        "id": "321321",
        "name": "barfoo"
    },
    "boost": 2.0
},
"add": {
    "doc": {
        "id": "123123",
        "name": "Foobar"        
    },
    "boost": 1.5,
    "overwrite": false
  }

}

score 1 · Answer 4 · answered Mar 05 '21 at 16:28

1

Update for SOLR 8.8 (and maybe lower).

The following JSON works for /update/json:

{
  'add': [
    {'id': '123', 'field1': 'foo'},
    {'id': '124', 'field1': 'foo'}
  ],
  'delete': ['111', '106']
}

answered Mar 05 '21 at 16:28

Risadinha

16,058
2
88
91

score 0 · Answer 5 · answered Apr 26 '16 at 20:51

Another option if you are on Solr 4.10 or later is to use a custom JSON structure and tell Solr how to index it (not sure how to add boosts with this method either, but it's a nice option if you already have a data struct in JSON and don't want to convert it over to Solr's format). Here's the Solr documentation on this option:

https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-TransformingandIndexingCustomJSON

How to update multiple documents in Solr with JSON?

5 Answers5