Tokenizing a String in Java

Question

I am currently working on a project to parse strings into individual tokens. This is all and well, but I am stuck (obviously). I currently would like to test if a string matches a keyword (i.e. true, false, class, inherits, etc) and return that string as a token (please note I am not using the StringTokenizer that the Java API provides, IIRC).

Here is my code that pertains to the keywords:

if (isAlpha(ch)){
        System.out.println("Current character:" + ch);
        letters += ch;
        charIndex--;
        while(isDigit(nextChar()) || isAlpha(nextChar())){
            Character c = nextChar();
            System.out.println("C:" + c);
            letters += c;
            charIndex++;
            System.out.println("After another character:" + letters);
        }
        if(letters == "class"){
            token = new Token(token.CLASS, letters);
        }
        if(letters == "inherits"){
            token = new Token(token.INHERIT, letters);
        }
        if(letters == "if"){
            token = new Token(token.IF, letters);
        } //etc etc

I have isolated the problem in my testing to strings that end in an alphabetic character (i.e. "funny"; this token/string would be an identifier). It keeps looping on the "y" and prints "Current character: y." It works with strings like "funny6" though. I may be missing the obvious (having to do with whitespace, perhaps), but any advice would be appreciated.

Thank you!

_winces at strings being compared with `==`_ See http://stackoverflow.com/questions/513832/how-do-i-compare-strings-in-java?rq=1 — The Guy with The Hat, Apr 21 '14 at 19:42
Have you looked into using regular expressions? Also use `stringA.equals(stringB)` instead of `==`. — SamYonnou, Apr 21 '14 at 19:44
Why not to look at the StringTokenizer code and do something similar? — Alexander Kulyakhtin, Apr 21 '14 at 19:47
You are aware that `inherits` is not a keyword in Java, right? — user1803551, Apr 21 '14 at 21:19
^Yes, I am. This is dealing with a fictional language. On another note, there seems to be more answers dealing with string equality than the actual loop, which is really the problem here. — user48326, Apr 21 '14 at 23:07
@user48326 If you make your code compile for us we could help you. — user1803551, Apr 23 '14 at 23:42

Tokenizing a String in Java

0 Answers0