How do you determine token type in Java?

Question

I'm trying to use StringTokenizer and populate 2 stacks with the elements inside the String input. I'm trying to use two if loops embedded in a while loop to populate the right type of token into their respective stacks. I am struggling with how to actually set up a conditional to determine the type. Here's what I have so far.

 Stack numbers = new Stack();
 Stack operators = new Stack();
 StringTokenizer token = new StringTokenizer(expr, delimiters, true);

 while(token.hasMoreTokens()){
   if(token.nextElement() == ){

   }
   if(token.nextElement() == ){

   }
 }

1. Don't call token.nextElement() more than once in the while loop body. 2. Read http://stackoverflow.com/questions/513832/how-do-i-compare-strings-in-java — JB Nizet, Mar 05 '17 at 18:41
don't use `==` to compare strings, use `.equals(otherObject)` instead — Erich Kitzmueller, Mar 05 '17 at 18:43
Since it looks like you're new to this site, I'll point you to [this help page](http://stackoverflow.com/help/someone-answers). You basically have to click the green checkmark next to the bast answer to mark your question as resolved an to reward the answerer. — Todd Sewell, Mar 05 '17 at 21:28

score 0 · Answer 1 · answered Mar 05 '17 at 18:46

First of all, you shouldn't use == with strings. Please use .equals() instead.

The next problem is that the token you compare in the first condition is another than the one in the 2 condition. token.nextToken() gives returns the first token and also deletes it from the Tokenizer.

You could make a temporary variable which stores the first element. You can then use this one to compare it to something.

Stack numbers = new Stack();
Stack operators = new Stack();
StringTokenizer token = new StringTokenizer(expr, delimiters, true);

 while(token.hasMoreTokens()) {
   String tmp = token.nextElement()
   if(tmp.equals(...) ){

   }
   if(tmp.equals(...){

   }
}

okay i thought that the tokenizer just made the type whatever the compiler thinks that it is (ie. if they see 3 they assume int). thank you for your answer! — Philip Kwak, Mar 05 '17 at 21:21

Todd Sewell · Answer 2 · 2017-03-05T21:24:57.313

The answer

The returned tokens are actually Strings, so you can do the usual stuff with them. I don't know what your exact requirements are but something like this should work:

List<String> operators = Arrays.asList("+", "-", "*", "/");
StringTokenizer token = ...;

while(token.hasMoreTokens()){
    String next = token.nextElement();
    if(operators.contains(next)){
        //operator
    }
    else if(next.matches("\\d+")){
        //number
    }
    else {
        //error?
    }
 }

First of all you should use a use a variable for the next token, nextElement() also deletes the token from the Tokenizer so you have to keep it somewhere. Then we can determine the type by comparing String, here I check if it's part of a list of predetermined operators and whether it matches the regex \d+, ie. whether it is a number.

Deprecation

The StringTokenizer JavaDoc says it has been deprecated in favor of String.split:

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

So I would recommend against using this class.

this is part of an algorithm to solve mathematical expressions that are inputted from left to right (ignoring usual order of operations). thank you for your reply — Philip Kwak, Mar 05 '17 at 21:20

How do you determine token type in Java?

2 Answers2

The answer

Deprecation