Solr highlight : InvalidTokenOffsetsException

Question

I use solr 3.4. When I search a word, I have an InvalidTokenOffsetsException. My field type look like :

    <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
                    <filter class="solr.ASCIIFoldingFilterFactory"/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="15" />
        <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
                    <filter class="solr.ASCIIFoldingFilterFactory"/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="15" />
        <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

When I remove "ASCIIFoldingFilterFactory", it works. It's the same problem with "ISOLatin1AccentFilterFactory".

Anyone have a solution ?

thanks

The solution is to put ASCIIFoldingFilterFactory just after tokenizer. — bobosh, Sep 21 '11 at 14:43
UPDATE: Actually it's fixed most of them but I can still get it to fail... but I think I can work around that — Paul Carroll, Dec 14 '11 at 05:53

score 0 · Answer 1 · answered Dec 27 '11 at 11:01

I had the same problem, and reported a bug https://issues.apache.org/jira/browse/LUCENE-3642 – its fixed in trunk right now.

I applied the patch manually and compiled solr my self, worked for both Solr 3.4 and Solr 3.5 although the patch did not apply cleanly and I had to do some manual fixing.

Solr highlight : InvalidTokenOffsetsException

1 Answers1