SOLR/Lucene Facet Query - ORs and NOTs

Question

I have a SOLR search that was behaving unexpectedly. When I go into the SOLR administration and run the following searches, I get a weird behavior (Note that myField__s only has 2 possible values, MyValue1 and MyValue2).

// 13,461 found - Good
fq:

// 9,168 found - Good
fq: myField__s:"MyValue1"

// 4,293 found - Good
fq: NOT myField__s:"MyValue1"

// 0 found - Bad, expected 13,461
fq: myField__s:"MyValue1" OR NOT myField__s:"MyValue1"

// 9,168 found - Good
fq: myField__s:"MyValue1" OR NOT myField__s:"MyValue2"

// 9,168 found - Bad, expected 13,461
fq: myField__s:"MyValue1" OR (NOT myField__s:"MyValue1")

I feel like there is some huge thing I'm missing about the fq syntax. Can anyone offer any guidance?

You're last result is a bit surprising to me. I had thought Solr handled that situation. Still, [this answer](http://stackoverflow.com/questions/17221736/whats-wrong-with-this-solr-range-filter-query/17225534#17225534) may point you in the right direction, on why NOT queries in Solr can be a bit hard to predict. — femtoRgon, Nov 30 '15 at 22:13
I think I understand, but `(myField__s:"MyValue1") OR (NOT (myField__s:"MyValue1"))` translates to `myField__s:MyValue1 (-myField__s:MyValue1)` by SOLR. Reading up on things and the answer you posted makes me think this should work. The 2 "main" components don't have + or - so they should be a SHOULD. And the inner part should be taking the whole set and removing anything that matches "MyValue1". So then the 2 SHOULDs should come together as the whole set. No? — Snowy Coder Girl, Nov 30 '15 at 23:17
Yes, that's what seemed surprising to me as well. You might have some luck with it if you add the match all manually though. Something like: `myField__s:MyValue1 (*:* -myField__s:MyValue1)` — femtoRgon, Nov 30 '15 at 23:22
`myField__s:"MyValue1" OR (*:* AND NOT myField__s:"MyValue1")` did indeed return all of the results, as expected. I guess I'll have to manually add that to any NOTs in the query. Thank you. If you add that as an answer I'll mark it. — Snowy Coder Girl, Nov 30 '15 at 23:49

score 1 · Answer 1 · answered Nov 30 '15 at 18:16

1

Solr has free options for boolean search MUST, SHOULD and MUST_NOT. If you include NOT prefix before some term this mean that you add it to boolean query with MUST_NOT option, so document with this term will be exclude from search result. (Prefix AND turns to MUST option and prefix OR turns to SHOULD OPTION). In other words query string

myField__s:"MyValue1" OR NOT myField__s:"MyValue1"

will be parsed to

myField__s:"MyValue1" -myField__s:"MyValue1".

You can see a transformed query in fq field of solr response

{
  "responseHeader":{
    "status":0,
    "QTime":11,
    "params":{
      "debugQuery":"true",
      "indent":"true",
      "q":"some query",
      "wt":"json",
      "fq":"myField__s:MyValue1 -myField__s:MyValue1"],
......

answered Nov 30 '15 at 18:16

Alexander Kuznetsov

3,062
2
25
29

Thanks. For some reason, our SOLR admin doesn't show `params`. Do you know the correct way to implement what I am trying? – Snowy Coder Girl Nov 30 '15 at 18:31
Try to add &debugQuery=true to query request, this show the parser query at the end of response. But it works only for q, not for fq. – Alexander Kuznetsov Nov 30 '15 at 18:49
1

This is helping, but still trying to figure it out - http://lucidworks.com/blog/2011/12/28/why-not-and-or-and-not/ – Snowy Coder Girl Nov 30 '15 at 18:55
Found a checkbox in the solr admin. Can see `filter_queries` and `parsed_filter_queries` now. Looks like `myField__s:"MyValue1" OR (NOT (myField__s:"MyValue1"))` gets parsed to `myField__s:MyValue1 (-myField__s:MyValue1)`. I think that should be correct, but it's still not returning the right number of results. – Snowy Coder Girl Nov 30 '15 at 18:59
Query `myField__s:MyValue1 (-myField__s:MyValue1)` should return zero results, because it should consists of documents with and without term `myField__s:MyValue1`. In any case, if you have `NOT` before term in your query, the query result will not contain documents with this term. – Alexander Kuznetsov Nov 30 '15 at 20:21
But I thought if there was no + or - before a term, it should be a SHOULD and essentially an OR. And I thought the parentheses would solve this issue. Like join the 2 conditions with or (SHOULDs) and then then the inner query of the second condition is a MUST NOT. Is that not correct? – Snowy Coder Girl Nov 30 '15 at 20:47
If SHOULD applaies to query with MUST NOT the result will be query with MUST NOT. So parentheses will not solve your issue. – Alexander Kuznetsov Nov 30 '15 at 21:43
Only rewrite qurey. In case TERM OR NOT TERM query the rewriten query will be *:*. – Alexander Kuznetsov Nov 30 '15 at 22:51
Sorry, didn't follow your last comment. Rewrite it how? We essentially have a front end user interface that allows users to combine things with AND/OR/NOT. – Snowy Coder Girl Nov 30 '15 at 23:09
Combination OR NOT not will work in Solr query language. You should do some manipulation with your users requests to overcome this issue. E.g. made a some reasoning trying to simplify your query. – Alexander Kuznetsov Dec 01 '15 at 14:32

SOLR/Lucene Facet Query - ORs and NOTs

1 Answers1