19

Java is not my main programming language so I might be asking the obvious.

But is there a simple file-handling library in Java, like in python?

For example I just want to say:

File f = Open('file.txt', 'w')
for(String line:f){
      //do something with the line from file
}

Thanks!

UPDATE: Well, the stackoverflow auto-accepted a weird answer. It has to do with bounty that I placed - so if you want to see other answers, just scroll down!

Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
  • I remember your elaborated answer why did you delete it? – stacker May 18 '10 at 16:58
  • I didn't, I just moved my code to the pastebin, as it was getting quite lengthy. – Andriy Drozdyuk May 18 '10 at 18:33
  • If ever you feel adventurous check scala out. It allows you do complete these types of tasks. The language is written ontop of the java jvm meaning you can incorporate your java and vice versa. – Steve Aug 12 '10 at 00:30

10 Answers10

19

I was thinking something more along the lines of:

File f = File.open("C:/Users/File.txt");

for(String s : f){
   System.out.println(s);
}

Here is my source code for it:

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.util.Iterator;

public abstract class File implements Iterable<String>{
    public final static String READ = "r";
    public final static String WRITE = "w";

    public static File open(String filepath) throws IOException{
        return open(filepath, READ);
    }   

    public static File open(String filepath, String mode) throws IOException{
    if(mode == READ){
        return new ReadableFile(filepath);
    }else if(mode == WRITE){
        return new WritableFile(filepath);
    }
    throw new IllegalArgumentException("Invalid File Write mode '" + mode + "'");
    }

    //common methods
    public abstract void close() throws IOException;

    // writer specific
    public abstract void write(String s) throws IOException;

}

class WritableFile extends File{
    String filepath;
    Writer writer;

    public WritableFile(String filepath){
        this.filepath = filepath;
    }

    private Writer writer() throws IOException{
        if(this.writer == null){
            writer = new BufferedWriter(new FileWriter(this.filepath));
        }
        return writer;
    }

    public void write(String chars) throws IOException{
        writer().write(chars);
    }

    public void close() throws IOException{
        writer().close();
    }

    @Override
    public Iterator<String> iterator() {        
        return null;
    }
}

class ReadableFile extends File implements Iterator<String>{
    private BufferedReader reader;
    private String line;    
    private String read_ahead;

    public ReadableFile(String filepath) throws IOException{        
        this.reader = new BufferedReader(new FileReader(filepath)); 
        this.read_ahead = this.reader.readLine();
    }

    private Reader reader() throws IOException{
         if(reader == null){
               reader = new BufferedReader(new FileReader(filepath));   
         }
         return reader;
    }

    @Override
    public Iterator<String> iterator() {
        return this;
    }

    @Override
    public void close() throws IOException {
        reader().close();
    }

    @Override
    public void write(String s) throws IOException {
        throw new IOException("Cannot write to a read-only file.");
    }

    @Override
    public boolean hasNext() {      
        return this.read_ahead != null;
    }

    @Override
    public String next() {
        if(read_ahead == null)
            line = null;
        else
            line = new String(this.read_ahead);

        try {
            read_ahead = this.reader.readLine();
        } catch (IOException e) {
            read_ahead = null;
            reader.close()
        }
        return line;
    }

    @Override
    public void remove() {
        // do nothing       
    }
}

and here is the unit-test for it:

import java.io.IOException;
import org.junit.Test;

public class FileTest {
    @Test
    public void testFile(){
        File f;
        try {
            f = File.open("File.java");
            for(String s : f){
                System.out.println(s);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Test
    public void testReadAndWriteFile(){
        File from;
        File to;
        try {
            from = File.open("File.java");
            to = File.open("Out.txt", "w");
            for(String s : from){           
                to.write(s + System.getProperty("line.separator"));
            }
            to.close();
        } catch (IOException e1) {
            e1.printStackTrace();
        }   
    }
}
Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
  • That's interesting, you can make it look almost like in Python. I would call the class `File` something else, to avoid confusion with Java's own `java.io.File`. Make the `remove()` method in your `FileIterator` throw `UnsupportedOperationException` instead of leaving it empty. – Jesper May 10 '10 at 20:20
  • Thanks! I was not sure what to do with remove there ;-) I think I WILL keep the file name however, as with namespaces I don't see any problems arise. I made more changes to this - and I would really like someone with java background to take a look at it. I will post my version on pastebin tomorrow. – Andriy Drozdyuk May 10 '10 at 23:53
  • 4
    Drozzy, I think it's actually preferable to leave your code here regardless of the length. I always assume when answering that every other website on the planet will disappear (yes, even MSDN, Wikipedia and the like). Answers here on SO should be able to stand on their own merits. I have no problems with linking to other sites for more detail but the "meat" should be here. Let's just think of what happens to this answer if pastebin goes under (or worse, starts yo monetise their investment). This answer then becomes absolutely useless since it's basically a quote from the question. – paxdiablo May 19 '10 at 05:22
  • I'll even +2 you (Q & A) to try and convince you to leave the code here :-) – paxdiablo May 19 '10 at 05:23
  • +1 for cleverness, but your iterator needs to make sure that the file is closed if an IOException occurs in next() – Justin May 19 '10 at 17:31
  • Thanks! I've added the close. Yours - is precisely the types of comments that I wanted to get, because I am not sure I am handling the resources correctly. – Andriy Drozdyuk May 19 '10 at 19:14
  • Your `next` method should not create a new `String`; just use `line = read_ahead`. Also, it should check for `read_ahead == null` and throw a `NoSuchElementException` (see the JavaDoc for `Iterator`). – Kevin Brock Aug 12 '10 at 02:17
  • Also, the `close()` call in `next()` probably is a compile error since it is declared to throw `IOException`. – Kevin Brock Aug 12 '10 at 02:22
  • Actually, I would recommend throwing some unchecked exception with the IOException as a cause so that the iterator doesn't silently fail from the read exception (in `next()`). – Kevin Brock Aug 12 '10 at 02:23
  • There is no need for `ReadableFile` to have the `reader()` private method. Just make `reader` variable final. Your constructor assigns this. – Kevin Brock Aug 12 '10 at 02:27
  • `WriteableFile` should probably follow the same design as `ReadableFile` in the constructor, then eliminate the `writer()` private method. It's best to open the file during construction in this case so that open failures (e.g. no such file) would occur when the caller opens the file (i.e. at `File.open(...)` instead of the first call to `write(...)`). – Kevin Brock Aug 12 '10 at 02:30
  • Thanks for comments. Any chance you can post up an answer with the changes you are suggesting? – Andriy Drozdyuk Aug 19 '10 at 03:02
  • That's so disgusting. 700 lines of Java to read a file line by line, and that's only a simple use case -- and by no means covers all exceptions. – fijiaaron Mar 30 '12 at 17:37
  • You do realize, that what we are trying to achieve here is a "library" of sorts, that one will simply `import` into their program, and use, right? Also, you are more then welcome to present your version - after all this is what this question is all about. – Andriy Drozdyuk Mar 30 '12 at 17:55
11

Reading a file line by line in Java:

BufferedReader in = new BufferedReader(new FileReader("myfile.txt"));

String line;
while ((line = in.readLine()) != null) {
    // Do something with this line
    System.out.println(line);
}

in.close();

Most of the classes for I/O are in the package java.io. See the API documentation for that package. Have a look at Sun's Java I/O tutorial for more detailed information.

addition: The example above will use the default character encoding of your system to read the text file. If you want to explicitly specify the character encoding, for example UTF-8, change the first line to this:

BufferedReader in = new BufferedReader(
    new InputStreamReader(new FileInputStream("myfile.txt"), "UTF-8"));
Jesper
  • 202,709
  • 46
  • 318
  • 350
  • 1
    Hm.. that still looks clunky :-( – Andriy Drozdyuk May 10 '10 at 13:52
  • 1
    Yes, but that's how it works with Java's standard API, Java isn't the most concise language... – Jesper May 10 '10 at 20:18
  • 1
    So, outside of the toy example of reading a file line by line, what other things do you want to do? It isn't that clunky IMHO. – Justin May 19 '10 at 17:27
  • Well, basically same thing that python's File object does. It also has binary mode. I never had problems with it. http://docs.python.org/py3k/library/functions.html#open – Andriy Drozdyuk May 19 '10 at 20:06
  • It would be cool to have a code-template for this in eclipse, http://stackoverflow.com/questions/1028858/useful-eclipse-java-code-templates/1051926#1051926 – questzen May 23 '10 at 11:30
  • @Chris: added an example to show how you can specify the character encoding. – Jesper Jun 07 '10 at 20:57
  • How could anyone actually like to use code templates? Doesn't it just feel wrong and dirty? I mean if you are really banging out SO many of these things that you need to have IDE help you type it out - there is something fundamentally wrong in the "programming" approach. – Andriy Drozdyuk Sep 17 '10 at 15:59
4

If you already have dependencies to Apache commons lang and commons io this could be an alternative:

String[] lines = StringUtils.split(FileUtils.readFileToString(new File("myfile.txt")), '\n');
for(String line: lines){
      //do something with the line from file
}

(I would prefer Jesper's answer)

stacker
  • 68,052
  • 28
  • 140
  • 210
  • 1
    That reads the entire file into a `String`, and then splits the whole thing in memory... that could use a lot of memory if the file is large. – Jesper May 10 '10 at 20:16
  • I second @Jesper's consideration. It's just too many String's in memory and can literally kill your application (OutOfMemoryError) – gvaish May 19 '10 at 09:47
  • Just added an alternative before the bounty was added, and I also recommended Jespers answer. In cases of smaller files it works fine. – stacker May 19 '10 at 13:30
  • FileUtils has a method readLines, which returns a list of strings. – Krab May 25 '10 at 12:34
  • I try to stay away from anything in Apache commons, as it usually means including all 100 Zillion other dependencies in apache commons. – Andriy Drozdyuk Sep 17 '10 at 16:00
4

If you want to iterate through a file by strings, a class you might find useful is the Scanner class.

import java.io.*;
import java.util.Scanner;

    public class ScanXan {
        public static void main(String[] args) throws IOException {
            Scanner s = null;
            try {
                s = new Scanner(new BufferedReader(new FileReader("myFile.txt")));

                while (s.hasNextLine()) {
                    System.out.println(s.nextLine());
                }
            } finally {
                if (s != null) {
                    s.close();
                }
            }
        }
    }

The API is pretty useful: http://java.sun.com/javase/7/docs/api/java/util/Scanner.html You can also parse the file using regular expressions.

Grantismo
  • 3,819
  • 1
  • 17
  • 16
  • Thanks, but I really don't want to have anything to do with BufferedReaders or anything of that sort. Just give it a filepath, and done! Hm... but if I think about it more - i might wrap this Scanner in my own wrapper that will allow for some convenience. – Andriy Drozdyuk May 19 '10 at 20:08
3

I never get tired of pimping Google's guava-libraries, which takes a lot of the pain out of... well, most things in Java.

How about:

for (String line : Files.readLines(new File("file.txt"), Charsets.UTF_8)) {
   // Do something
}

In the case where you have a large file, and want a line-by-line callback (rather than reading the whole thing into memory) you can use a LineProcessor, which adds a bit of boilerplate (due to the lack of closures... sigh) but still shields you from dealing with the reading itself, and all associated Exceptions:

int matching = Files.readLines(new File("file.txt"), Charsets.UTF_8, new LineProcessor<Integer>(){
  int count;

  Integer getResult() {return count;}

  boolean processLine(String line) {
     if (line.equals("foo")
         count++;
     return true;
  }
});

If you don't actually want a result back out of the processor, and you never abort early (the reason for the boolean return from processLine) you could then do something like:

class SimpleLineCallback extends LineProcessor<Void> {
    Void getResult{ return null; }

    boolean processLine(String line) {
       doProcess(line);
       return true;
    }

    abstract void doProcess(String line);
}

and then your code might be:

Files.readLines(new File("file.txt"), Charsets.UTF_8, new SimpleLineProcessor(){
  void doProcess(String line) {
     if (line.equals("foo");
         throw new FooException("File shouldn't contain 'foo'!");
  }
});

which is correspondingly cleaner.

Cowan
  • 37,227
  • 11
  • 66
  • 65
  • If you post the code to use the callback in a nice way I'll accept :-) – Andriy Drozdyuk May 19 '10 at 14:57
  • @drozzy: done, along with a little tip I'd use to reduce the boilerplate back again. – Cowan May 19 '10 at 21:03
  • Sorry now that I look at it - it is kind of cumbersome. It is not as simple as (for line in file): do stuff. I hope you know what I mean. But I did upvote your answer. Maybe I can use your piece of code as the guts of my implementation - which should take care of a lot of exception handling and proper file/buffer closing. – Andriy Drozdyuk May 20 '10 at 13:35
  • It's hard. Google could have a version of the simple readLines which returns an Iterable, not a List, and doesn't keep in memory, but that's awkward -- fact is, all IO operations in Java can throw IOException, and it's not easy to have a clean API which completely hides that. What should the iterator do when it gets such an exception? Pretend there's no next item? Rethrow an unchecked exception? etc. Checked exceptions unfortunately make it hard to do in a 'least surprise' way. – Cowan May 20 '10 at 13:49
  • I'd say throw an unchecked exception. Silently failing is not really an option. Another option would be to log the failure at that point (but then your `Iterator` would probably just be a private API). – Kevin Brock Aug 12 '10 at 02:36
2
  public static void main(String[] args) throws FileNotFoundException {
    Scanner scanner = new Scanner(new File("scan.txt"));
    try {
      while (scanner.hasNextLine()) {
        System.out.println(scanner.nextLine());
      }
    } finally {
      scanner.close();
    }
  }

Some caveats:

  • That uses the default system encoding, but you should specify the file encoding
  • Scanner swallows I/O exceptions, so you may want to check ioException() at the end for proper error handling
McDowell
  • 107,573
  • 31
  • 204
  • 267
1

Simple example using Files.readLines() from guava-io with a LineProcessor callback:

import java.io.File;
import java.io.IOException;

import com.google.common.base.Charsets;
import com.google.common.io.Files;
import com.google.common.io.LineProcessor;

public class GuavaIoDemo {

    public static void main(String[] args) throws Exception {
        int result = Files.readLines(new File("/home/pascal/.vimrc"), //
            Charsets.UTF_8, // 
            new LineProcessor<Integer>() {
                int counter;

                public Integer getResult() {
                    return counter;
                }

                public boolean processLine(String line) throws IOException {
                    counter++;
                    System.out.println(line);
                    return true;
                }
            });
    }
}
Pascal Thivent
  • 562,542
  • 136
  • 1,062
  • 1,124
0

You could use jython which lets you run Python syntax in Java.

Nathan Voxland
  • 15,453
  • 2
  • 48
  • 64
0

Nice example here: Line by line iteration

Goibniu
  • 2,212
  • 3
  • 34
  • 38
  • This is indeed what I was thinking. Except the code is not reviewed and tested by anyone. It would be great if there was like a community of utilities project. Also - the problem is that code eats up the newline character. (Which I must admit my answer does too - but it should leave the character in). – Andriy Drozdyuk May 19 '10 at 12:40
0

Try looking at groovy!

Its a superset of Java that runs in hte JVM. Most valid Java code is also valid Groovy so you have access any of the million java APIs directly.

In addition it has many of the higher level contructs familiar to Pythonists, plus a number of extensions to take the pain out of Maps, Lists, sql, xml and you guessed it -- file IO.

James Anderson
  • 27,109
  • 7
  • 50
  • 78
  • Yeah, but I really don't like the fact that Groovy can contain java source code. I mean it's like taking really cluttered language syntax and duct-taping some "nice" syntax on top of it. – Andriy Drozdyuk Jul 01 '10 at 00:08