1

I am writing a scraper that downloads a list of mp3s from a certain site (1,700+ mp3s of about 15 MB each). My method is given a List of URLs and a destination folder.

I would like to program a JProgressBar dialog around this that has:

  1. The current transfer rate.
  2. The currently-being-downloaded file information (how much is done / how much is left).

I am using a ReadableByteChannel that transfers into a FileOutputStream (or, more accurately, a FOS that transfers from the RBC).

Here's the code:

private static void download(List<String> mp3_urls, String directory) {
    try {
        for(String mp3_url : mp3_urls) {
            URL url = new URL(mp3_url);
            try (ReadableByteChannel rbc = Channels.newChannel(url.openStream());) {
                Pattern pattern = Pattern.compile("[A-Za-z0-9_]*\\.mp3");
                Matcher matcher = pattern.matcher(mp3_url);
                matcher.find();

                System.out.println("Reading: " + mp3_url);
                try (FileOutputStream fos = new FileOutputStream(directory + File.separator + matcher.group())) {
                    fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
                }
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
}

My questions are:

  1. Does the FileOutputStream::transferFrom method throw any properties that could be caught with a PropertyChangeListener that would give this information??
  2. Am I using methods capable of giving this information?
  3. How would you get this information?
ryvantage
  • 13,064
  • 15
  • 63
  • 112

1 Answers1

1

You're missing the point. transferFrom() ideally happens in the kernel without consulting the application at all. That's the point of it. Either you want progress indications, which you can get with a ProgressMonitorInputStream around a socket or URLConnection input stream, or you want a fast kernel-mode copy. You can't have both.

NB:

  1. The method is FileChannel.transferFrom(), not FileOutputStream::transferFrom().

  2. A single call isn't sufficient. You have to loop. It isn't guaranteed to transfer the entire amount in a single call.

user207421
  • 305,947
  • 44
  • 307
  • 483
  • Well that would be the answer to my #2 question, `Am I using methods capable of giving this information?` So, the answer is no. I got this method from a SO question: http://stackoverflow.com/questions/921262/how-to-download-and-save-a-file-from-internet-using-java. So, I'll have to figure out how to do it without `transferFrom()`. – ryvantage Jul 23 '14 at 02:21
  • I've *told* you how to do it without `transferFrom().` – user207421 Jul 23 '14 at 02:37
  • 1
    Glad to see someone calling out that `transferFrom` must be called in loop. – Brett Okken Jul 23 '14 at 03:13
  • Well, I was having problems with incomplete transfers until I added code to close the `FileOutputStream` (the `try` block). Now every file transfers completely, no problem. Are you sure it must be looped? – ryvantage Jul 23 '14 at 12:49