Processing huge amount of large jpgs

Question

I am writing a program partly just for fun, partly to help me deal with a heap of digital pictures that I want to divide into categories for printing. The main idea is that it should display the pictures in a single column and have a set of checkboxes, with category names, next to each picture. I check the needed checkboxes, press the "go!" button, and the pictures get copied into subfolders, depending on the selected checkboxes.

Now, everything is almost finished - except for one thing. The pictures in question are large jpgs, each about 7-8MB, and there's approximately 700 of them. If I try to load them all at once, naturally, it takes a huge amount of memory and time to load them all. So, is there a good solution to the problem? My two thoughts were as follows.

1) To load the pics 10 at a time and have next/previous buttons somewhere. I don't like this idea, as it adds unneeded elements. 2) To make the application load new pictures as you scroll to them and unload those you scrolled past. I really like this idea.

Can someone point me in the right direction, as to how I can implement the latter idea? I have only found one relevant link on Google, but I cannot say that it helped me, i got a bit confused by some parts of the code.

*"large jpgs, each about 7-8MB"* Depending on compression, that could led to a large variety of final pixel sizes. What sort of WxH are the images? — Andrew Thompson, Nov 25 '12 at 13:38

score 2 · Answer 1 · edited May 23 '17 at 12:18

2

If thumbnails are sufficient, this answer includes a simple approach to resampling and cites some trade-offs. If not, this answer outlines a general approach to displaying and caching recent images.

In either case, the default Boolean renderer/editor of JTable is a JCheckBox. CheckOne is an example.

edited May 23 '17 at 12:18

Community

1
1

answered Nov 25 '12 at 00:10

trashgod

203,806
29
246
1,045

Thanks for the links, I found the second one most useful, I am new to Swing (and actually pretty much a newbie in Java), so I haven't even thought of JTable. – Anton Zujev Nov 26 '12 at 09:10
If you go with `JTable`, the default renderer also recognizes `Icon` & `ImageIcon` for your thumbnails. – trashgod Nov 26 '12 at 10:43

score 1 · Answer 2 · answered Nov 25 '12 at 01:11

1

You have to create thumnails for all pictures, you may keep the thumnails in memory. That may need a lot of time.

Then you are either ready with the pronblem. or the thumbnail will not fit all in memory. If that is the case: you load 30-40 of them, and during scrolling you detct the scroll direction, and load the next bunch in a separate thread.

If the loading is slower than the user scrolls, then you dispaly a default place holder image for such a "not yet loaded pic"

answered Nov 25 '12 at 01:11

AlexWien

28,470
6
53
83

+1 I'd suggest caching the thumbnails, but that's another problem altogether ;) – MadProgrammer Nov 25 '12 at 03:24
A typical UI pending image icon is shown [here](http://stackoverflow.com/a/11092120/230513). – trashgod Nov 25 '12 at 10:21
@AlexWien yes, some kind of loading while scrolling is the thing I'm trying to achieve. – Anton Zujev Nov 26 '12 at 09:44

score 0 · Answer 3 · answered Nov 25 '12 at 01:35

You can shrink down all your images to use less RAM. e.g. this code reduces all images to 200x200. That way you can fit 1000 images in 100MB.

import javax.swing.*;
import java.awt.*;
import java.awt.image.*;
import javax.imageio.ImageIO;
import java.io.File;

public class Scroll extends JPanel {
    public static void main(String[] args) throws Exception {
        JFrame frame = new JFrame();
        JPanel panel = new Scroll();

        panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));

        for(int i = 0; i < 10; i++) {
            JPanel buttonPanel = new JPanel();
            JRadioButton b1 = new JRadioButton("button 1");
            JRadioButton b2 = new JRadioButton("button 2");
            JRadioButton b3 = new JRadioButton("button 3");
            ButtonGroup group = new ButtonGroup();
            group.add(b1);
            group.add(b2);
            group.add(b3);
            buttonPanel.add(b1);
            buttonPanel.add(b2);
            buttonPanel.add(b3);

            BufferedImage buffer = new BufferedImage(200,200,BufferedImage.TYPE_INT_RGB);
            Graphics2D g = buffer.createGraphics();

            BufferedImage image = ImageIO.read(new File("image.jpg"));
            g.scale(buffer.getWidth()*1.0/image.getWidth(),
                    buffer.getHeight()*1.0/image.getHeight());
            g.drawImage(image, 0, 0, null);
            g.dispose();
            JLabel imageLabel = new JLabel(new ImageIcon(buffer));
            JSplitPane splitPane = new JSplitPane();
            splitPane.add(imageLabel, JSplitPane.LEFT);
            splitPane.add(buttonPanel, JSplitPane.RIGHT);
            panel.add(splitPane);
        }
        JScrollPane spane = new JScrollPane(panel);
        frame.add(spane);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(500,600);
        frame.setVisible(true);
    }
}

If you want to dynamically load images as they become visible you have to use empty JPanels for each image instead of ImageIcons and then you override the paint method of those JPanels. The paint method will only be called if the JPanel is visible. So the simplest solution would be to always load the image from disk in the paint method and then draw it to the screen.

import javax.swing.*;
import java.awt.*;
import java.awt.image.*;
import javax.imageio.ImageIO;
import java.io.File;

public class Scroll extends JPanel {
    public static void main(String[] args) throws Exception {
        JFrame frame = new JFrame();
        JPanel panel = new Scroll();

        panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));

        for(int i = 0; i < 10; i++) {
            JPanel buttonPanel = new JPanel();
            JRadioButton b1 = new JRadioButton("button 1");
            JRadioButton b2 = new JRadioButton("button 2");
            JRadioButton b3 = new JRadioButton("button 3");
            ButtonGroup group = new ButtonGroup();
            group.add(b1);
            group.add(b2);
            group.add(b3);
            buttonPanel.add(b1);
            buttonPanel.add(b2);
            buttonPanel.add(b3);

            JPanel imagePanel = new JPanel() {
                {
                    BufferedImage image = ImageIO.read(new File("image.jpg"));
                    setPreferredSize(new Dimension(image.getWidth(), image.getHeight()));
                    image.flush();
                }
                @Override
                public void paint(Graphics g) {
                    try {
                        BufferedImage image = ImageIO.read(new File("image.jpg"));
                        g.drawImage(image, 0, 0, null);
                        image.flush();
                    } catch(Exception e) {
                    }
                }
            };

            JSplitPane splitPane = new JSplitPane();
            splitPane.add(imagePanel, JSplitPane.LEFT);
            splitPane.add(buttonPanel, JSplitPane.RIGHT);
            panel.add(splitPane);
        }
        JScrollPane spane = new JScrollPane(panel);
        frame.add(spane);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(500,600);
        frame.setVisible(true);
    }
}

thanks for the code snippets. I'm trying to go with the second one, the problem is that, with the structure I have at the moment, when the 're-load images' button is pressed, a listener rescans the directory it is in and adds lots of JLabels with the images as ImageIcons. The JLabels are created in a separate class. I assumed I would override paint() in that class and extended JPanel with it, but paint() still isn't getting called. Where would be the most appropriate place to override it? — Anton Zujev, Nov 26 '12 at 09:42

score 0 · Answer 4 · answered Nov 25 '12 at 07:11

If this is a "use once and throw away" kind-of program I wouldn't bother trying to solve this problem. I would use the command line to generate thumbnails for each of the pictures, and use those generated thumbnails in my program. Keeping 700 thumbnails in memory should be doable.

If that is not an option, I would just start with a JTable. You can create a TableModel without having to load all the images into memory. You just need to know the image count. The JTable will only want to render the images which are visible at that moment. Caveat might be that loading the images at the moment the renderer asks for them might be too late. I can imagine that the JTable will not run smoothly when you have to load images with a size of a few MB's. But this can probably be solved by using a cache which gets populated with a worker thread. So if the renderer request the 5th image, you also load the 5 previous images and the 5 next images into the cache.

Also note that if you are moving the pictures iso copying them, this might affect your TableModel as the number of images in your directory will change. Make sure to consider that !

I'm just trying to learn something with this program, so even though it's a one-use thing I want to try and build it the way it was meant to be. Thanks for the tip about JTable, I haven't even thought of that. — Anton Zujev, Nov 26 '12 at 09:37

Processing huge amount of large jpgs

4 Answers4