1

I have a text file that looks like this: 

shooting-stars    "are cool"

I have a lexical analyzer that uses FileInputStream to read the characters one at a time, passing those characters to a switch statement that returns the corresponding lexeme.

In this case, represents assignment so this case passes: 

case 'ð' :
            return new Lexeme("ASSIGN");

For some reason, the file reader stops at that point, returning a null pointer even though it has yet to process the string (or whatever is after the ). Any time it reads in an emoticon it does this. If there were no emoticons, it gets to the end of file. Any ideas?

Wanda
  • 25
  • 3

1 Answers1

0

I suspect the problem is that the character (Unicode code point U+1F4AB) is outside the range of characters that Java represents internally as single char values. Instead, Java represents characters above U+FFFF as two characters known as surrogate pairs, in this case U+D83D followed by U+DCAB. (See this thread for more info and some links.)

It's hard to know exactly what's going on with the little bit of code that you presented, but my guess is that you are not handling this situation correctly. You will need to adjust your processing logic to deal with your emoticons arriving in two pieces.

Community
  • 1
  • 1
Ted Hopp
  • 232,168
  • 48
  • 399
  • 521
  • Probably not the best solution, but using f.skip(3) before returning the new lexeme in the code above, skipped over the remaining bits and allowed for the file reader to consume the next character. Originally I tried to use the surrogate helper methods, but I am not sure how to use them in my particular situation. – Wanda Nov 01 '16 at 05:22
  • Hm yes, what I did is not a good solution. I'll need to explore surrogate pairs again. – Wanda Nov 01 '16 at 05:47