token parsing in java

Question

I have created a file with printStream object as shown below.

PrintStream outPutOffice = new PrintStream(
   new BufferedOutputStream(new FileOutputStream(inDir+"/Office.txt")));
outPutOffice.print(fValue + (findx < agtOffColCount ? "|" : ""));

Now I have to read it content and separate it tokens with "|" as I have written token with "|" separated. I have write code as shown below it will read line correctly but not separate token with "|" character.

BufferedReader inPutAgent = new BufferedReader(
   new InputStreamReader(new FileInputStream(inDir+"/Office.txt")));

String column=inPutAgent.readLine();
String []columnDetail = column.split("|");

columndetail array contains single character in each index instead i want single token in each index.

What is the problem?

You also might want to learn, what the difference between a Writer and a OutputStream is. Take a look at the FileWriter API http://java.sun.com/j2se/1.5.0/docs/api/java/io/FileWriter.html — Tim Büthe, Oct 27 '09 at 11:17

score 5 · Accepted Answer · answered Oct 27 '09 at 11:11

5

the split method works with regular expressions and since the pipe symbol (|) has a special meaning and is reserved, you need to escape it like this:

split("\\|");

You should read about regex here or here

answered Oct 27 '09 at 11:11

Tim Büthe

62,884
17
92
129

Filip Ekberg · Answer 2 · 2009-10-27T11:19:12.927

3

You should look into the StringTokenizer, it's a very handy tool for this types of work.

Example

 StringTokenizer st = new StringTokenizer("this is a test");
 while (st.hasMoreTokens()) {
     System.out.println(st.nextToken());
 }

This will putput

 this
 is
 a
 test

You use the Second Constructor for StringTokenizer to set the delimiter:

StringTokenizer(String str, String delim)

You could also use a Scanner as one of the commentators said this could look somewhat like this

Example

 String input = "1 fish 2 fish red fish blue fish";

 Scanner s = new Scanner(input).useDelimiter("\\s*fish\\s*");

 System.out.println(s.nextInt());
 System.out.println(s.nextInt());
 System.out.println(s.next());
 System.out.println(s.next());

 s.close();

The output would be

 1
 2
 red
 blue

Meaning that it will cut out the word "fish" and give you the rest, using "fish" as the delimiter.

examples taken from the Java API

edited Oct 27 '09 at 11:19

answered Oct 27 '09 at 11:10

Filip Ekberg

36,033
20
126
183

2

StringTokenizer(String str, String "|") This will tokenize the string with | as the separator. – Collin Price Oct 27 '09 at 11:14
1

Check out the Scanner, it's a lot better then the StringTokenizer http://java.sun.com/j2se/1.5.0/docs/api/java/util/Scanner.html – Tim Büthe Oct 27 '09 at 11:16
They both work very well and I would suggest any of them over .split() ! – Filip Ekberg Oct 27 '09 at 11:17
2

@Filip Ekberg: I think split is fine and we should show @lakhaman what's wrong with his code, not just work around it... – Tim Büthe Oct 27 '09 at 11:19
4

From the 1.6 documentation for StringTokenizer ... "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead." ... split or regex are really the preferred methods of tokenizing. – tschaible Oct 27 '09 at 11:22
Split works fine, but StringTokenizer works better for the majority of the tokenizing I've had to do. No idea why they'd discourage it's use, but then again, it's not the first time I've found their "discouragements" and deprecations to be questionable (at best). – Brian Knoblauch Oct 27 '09 at 11:29
@Tim Büther, yes we should provide him a solution to the problem and teach him how to do it correct. Which might not be the same as showing him what was wrong with His code. You already provided the regex-solution, it would be redundant for me to say that aswell. I wanted to contribute with an alternative solution which in my opnion makes code more readable and easier to work with. – Filip Ekberg Oct 27 '09 at 12:12
@tschaible, I actually did not know that. But in earlier versions of Java, it was better to use the tokenizer. afaik. – Filip Ekberg Oct 27 '09 at 12:13

score 2 · Answer 3 · answered Oct 27 '09 at 11:12

2

The argument to split is a regex. You should do:

String []columnDetail = column.split("\\|");

or:

String []columnDetail = column.split("[|]");

answered Oct 27 '09 at 11:12

Maurice Perry

32,610
9
70
97

token parsing in java

3 Answers3

Linked