0

I'm using Cayenne to parse SQL conditions, through org.apache.cayenne.exp.parser.ExpressionParser, which produces a series of org.apache.cayenne.exp.parser.Tokens, and I want to determine the type of each Token (like identifier, equal sign, number, string etc.).

The token type is definitely identified by the ExpressionParser, and it seems to me that it is stored in the int field Token.kind. The values that this field shows in my parsing tests are definitely consistent (for ex. = is always 5, literal strings are always 42, and operators are always 2 etc.).

My problem is just that I cannot find the Java class containing the constants to compare Token.kind values with.

The Javadoc for field Token.kind says:

An integer that describes the kind of this token. This numbering system is determined by JavaCCParser, and a table of these numbers is stored in the file ...Constants.java.

It does not specify the full name of the file, so I downloaded JavaCCParser and I checked several *Constants.* files found in javacc-5.0src.zip, javacc-6.0.zip, the two javacc.jar contained in those two zip, and cayenne-3.0.2-src.tar.gz.

None of the classes I found there seems to me to have constants that consistently match the values I see in my tests. The closest I was able to get to that was with class org.apache.cayenne.exp.parser.ExpressionParserConstants which for ex. contains int PROPERTY_PATH = 34 and int SINGLE_QUOTED_STRING = 42 which definitely match the actual tokens of my test expressions, but other tokens have no corresponding constant in that class, for ex. the = sign (kind = 5) and the and operator (kind = 2).

So my question is if anyone knows in which Java class are those constants defined.

SantiBailors
  • 1,596
  • 3
  • 21
  • 44

1 Answers1

1

First I should mention that ExpressionParser is designed to parse very specific format of Cayenne expressions. It certainly can not be used to parse SQL. So you might be looking in the wrong direction.

Parser itself is generated by JavaCC based on this grammar file. Tokens for the parser are formally defined in the bottom of this file, and are very specific to the task at hand.

andrus_a
  • 2,528
  • 1
  • 16
  • 10
  • Whoops, OK then, thanks for the heads-up about ExpressionParser not being suited for SQL parsing. What led me to believe it was OK to do that is http://cayenne.apache.org/docs/3.0/in-memory-evaluation.html which I found from the post http://stackoverflow.com/questions/898405/how-to-implement-a-sql-like-like-operator-in-java but I might have taken my conclusions a bit too far in thinking that I could parse whole SQL expressions. – SantiBailors Nov 05 '13 at 13:41
  • This is what I do: `String where = "Name='Al' and (City='Ai' or Country='Ai')";` `Expression expression = Expression.fromString(where);` That `expression` instance definitely comes to contain just the tokens I expect. Now, that `Expression.fromString(String)` Cayenne method uses the `ExpressionParser` in the same way I do: `return new ExpressionParser(reader).expression();` So should I just conclude that while my `where` string in my intention is in SQL syntax, it's actually Cayenne syntax which in this case happens to match SQL syntax but I shouldn't rely on this being the case for other SQL? – SantiBailors Nov 05 '13 at 14:10
  • Yes, this particular WHERE is semantically similar with Cayenne expressions. In general this may not be the case. So if you are writing your own parser (and couldn't find a decent open source SQL parser out in the wild, which you should be able to do), you can take ExpressionParser.jjt from Cayenne and extend it for your needs, and the compile it with javaCC. But that the hard way. – andrus_a Nov 05 '13 at 16:06
  • Thanks for the clarification, now I have a better picture. I'm using Cayenne because I need the in-memory evaluation to filter lists through a WHERE, plus I need to store the WHERE tokens, and I need the two things to strictly match (that is, putting the tokens back together into a WHERE and filtering against it should produce exactly the same filtering as the original WHERE as a whole does) so I preferred to use the same library to do both things and because of this I discarded other libraries. But if I can't reliably use Cayenne for SQL, I better go back to looking for another library. – SantiBailors Nov 05 '13 at 16:25
  • I decided to stick to Cayenne for SQL expressions because my SQL is always simple and it definitely complies with Cayenne syntax, which I have the feeling was made to some extent SQL compatible on purpose (see for ex. its LIKE operator). But if I need real SQL support I will try to find another library that can do both SQL parsing *and* in-memory evaluation. The grammar file linked in the answer doesn't contain the constants for `Token.kind` I was looking for but I can figure out the ones I need by testing. – SantiBailors Nov 06 '13 at 13:02