Counting the size of a text file in java while writing to the file

Question

This question is a follow up to the accepted answer in this question. I am trying to implement the approach suggested by Aaron : Wrapping the FileOutputStream to include the logic to keep a count of the no.of bytes written so far. However the approach quite doesn't seem to work as expected. The OutputStreamWriter seems to be using a StreamEncoder which is buffering the data before delegating call to the FileOutputStream.write() method.

Here is a small Demo :

package Utils;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;

public class MyFileOutputStream extends FileOutputStream{

private int byteCount;
public int getByteCount() {
    return byteCount;
}

public void setByteCount(int byteCount) {
    this.byteCount = byteCount;
}

public MyFileOutputStream(String arg0) throws FileNotFoundException {
    super(arg0);
    byteCount = 0;
}

@Override
public void write(byte[] b) throws IOException{
    byteCount += b.length;
    super.write(b);
}

@Override
public void write(byte[] b , int off , int len) throws IOException{
    byteCount += len;
    super.write(b, off, len);
}
}

And Driver Class :

  package main;
  import java.io.BufferedWriter;
  import java.io.FileNotFoundException;
  import java.io.FileOutputStream;
  import java.io.IOException;
  import java.io.OutputStreamWriter;
  import java.io.Writer;

  import Utils.MyFileOutputStream;

  public class Driver {

/**
 * @param args
 * @throws IOException 
 */
public static void main(String[] args) throws IOException {
    MyFileOutputStream fos = new MyFileOutputStream("testFile");
    BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fos));
    for(int i=0;i<1000;i++){
        bw.write("Writing this string\n");
        System.out.println("Bytes Written : "+fos.getByteCount());
    }
    bw.close();
    System.out.println(fos.getByteCount());
}

}

Output :-

Bytes Written : 0
Bytes Written : 0
...
Bytes Written : 8192
Bytes Written : 8192
...

As shown by the output the StreamEncoder buffers up to 8192 bytes before delegating call to write() method of FileOutputStream. Is there any work around for this to get the no.of bytes written to a file at any instant of time ?

That's kind of the point isn't it. The `BufferedWriter` buffers the output until it's internal buffer is full, `flush` or `closed` is called at which time it outputs the buffer to the underlying writer stream. — MadProgrammer, May 04 '14 at 04:54
The design also looks slightly wrong (IMHO), I would create a pass through `OutputStream` which took a source `OutputStream`. The "counter" stream would override `write(int)` and record the number of bytes that has passed through it, calling the parent stream's `write` method. This way, you could use any type of stream you like... — MadProgrammer, May 04 '14 at 04:56

score 1 · Answer 1 · answered May 04 '14 at 04:43

1

Is there any work around for this to get the no.of bytes written to a file at any instant of time ?

That's what you're getting. The number of bytes that have been written to the file.

What you really seem to be asking for is the number of chars that have been written to the BufferedWriter. In which case what you need to do is wrap/extend the BufferedWriter.

answered May 04 '14 at 04:43

user207421

305,947
44
307
483

Nope , i need the no.of bytes written to the file but i don't want to get answer in blocks of 8K(beacause of the buffering). I realize that there is no other way but to count the no.of bytes the string has even before writing to the buffered writer. – Kumaran May 04 '14 at 06:24
1

You're not getting it. Data *is written* in 8k chunks *to the file.* Single bytes are not written. There is buffering in the `StreamEncoder.` Nothing you can do about that. It's written in smaller pieces to the `StreamEncoder`, but you can only count that in chars, not bytes. – user207421 May 04 '14 at 07:22
Yes i get that. But , i am dealing with Japanese Kanji characters which are written in to the file with UTF-8 encoding.So counting the characters will not give the exact file size.Hence, i want the byte count. – Kumaran May 04 '14 at 07:53
1

And you're getting it. 8192 bytes at a time. Because that's what the FileOutputStream is receiving. There isn't another answer. – user207421 May 04 '14 at 09:32

score 1 · Answer 2 · answered May 09 '14 at 14:24

You can flush() the outermost writer/stream. That forces BufferedWriter to convert all the characters in its buffer to bytes and send them to the underlying FileOutputStream.

Note that this is a somewhat expensive operation: It will not only convert the bytes but actually write them to the disk. So if you call flush() too often, that will be bad for overall performance.

Another option would be to reduce the buffer size to, say, 128. That would cause 64 times the IO but give a more fine grained picture.

If performance is a problem, then you need to move the buffering further down. Directly write to the OutputStreamWriter and wrap the FileOutputStream in a class which extends BufferedOutputStream.

That way, the characters will converted to bytes immediately and added to the buffer in BufferedOutputStream. Now you just need to ask your BufferedOutputStream how many bytes it has written to the FileOutputStream + this.count.

score 1 · Answer 3 · answered Nov 13 '20 at 12:19

1

How about CountingOutputStream? It will solve your problem easily.

answered Nov 13 '20 at 12:19

Faizan Mubasher

4,427
11
45
81

Counting the size of a text file in java while writing to the file

3 Answers3