Using tokenizer to count number of words in a file?

Question

I am making a program that makes an user choose a file then the program reads from the file. Now I've been told to make the program using bufferedreader and string tokenizer. So far I got program opening the file and counting the number of lines. But the number of words is not so easy.

This is my code so far:

int getWords() throws IOException
{
   int count = 0;
   BufferedReader BF = new BufferedReader(new FileReader(f));
   try  {
      StringTokenizer words = new StringTokenizer(BF.readLine()); 
      while(words.hasMoreTokens())
      { 
         count++;
         words.nextToken(); 
      }
      BF.close();
   }  catch(FileNotFoundException e)  {
   }    
   return count;
}

Buffered reader can only read a line at a time but I don't know how to make it read more lines.

while (BF.readLine() != null) You mean that? I tried, did not work. — user3236502, Mar 02 '14 at 19:22
Please do not swallow your exception: http://stackoverflow.com/questions/921471/why-java-people-frequently-consume-exception-silently — aliteralmind, Mar 02 '14 at 19:22

Bitman · Answer 1 · 2014-03-02T19:49:34.627

1

to count words you can use countTokens() instead of loop

to read all lines use

String line = null;
while(null != (line = BF.readLine())) {
StringTokenizer words = new StringTokenizer(line); 
   words.countTokens();//use this value as number of words in line
}

edited Mar 02 '14 at 19:49

answered Mar 02 '14 at 19:25

Bitman

1,996
1
15
18

BasicFile.java:117: error: unexpected type while(null != line = BF.readLine()) ^ required: variable found: value – user3236502 Mar 02 '14 at 19:30
sorry lost braces there. fixed now – Bitman Mar 02 '14 at 19:49

score 0 · Accepted Answer · answered Mar 02 '14 at 19:27

As you said, buffered reader will read one line at a time. So you have to read lines until there are no more lines. readLine() returns null when the end of file is reached.

So do something like this

int getWords() throws IOException {
  int count = 0;
  BufferedReader BF = new BufferedReader(new FileReader(f));
  String line;
  try {
    while ((line = BF.readLine()) != null) {
      StringTokenizer words = new StringTokenizer(line); 
      while(words.hasMoreTokens()) { 
        count++;
        words.nextToken(); 
      }    
    }
    return count;
  } catch(FileNotFoundException e)  {
  } finally {
    BF.close();
  }
  // Either rethrow the exception or return an error code like -1.
}

You are a godsend !! Thanks. Also we haven't really learned to use exceptions in class. — user3236502, Mar 02 '14 at 19:40

Using tokenizer to count number of words in a file?

2 Answers2