Large JScrollPane with multiple JTables—how to paginate

Question

I'vve been trying to google this, but most people ask questions related to a large single table, my problem is a bit different.

Sorry I cant provide working code, my program is a bit complex, but if someone can point me in a direction ill try to find a solution.

In short, I have a very large text file that i read and create objects for each line, eg HeaderRecord or ClaimDetail etc. the current file ends up with about 60,000 objects. I then create JTables for each record, iterate through them and then add them to my scrollpane.

It works great, my problem is that loading this data into memory takes alot! (obviously). But even ignoring that, after the data is loading and the screen pops up its still stuck for like a minute, probably trying to draw all of this is the pane.

The only solution i can think of is to somehow paginate the display. only pulling in a certain number of JTables at a time. I have no idea how to go about this! If someone can please help me out with a suggestion of what to look at i would really appreciate it

Thanks

Update: I implemented a swingworker like this:

private class TableRun extends SwingWorker<Void, JTable> {
        private ArrayDeque<FileRecord> fileRecords;
        private final GridBagConstraints gc;
        private final JPanel parentPanel;
        int counter = 1;
        TableRun(ArrayDeque<FileRecord> fileRecords, GridBagConstraints gc, JPanel parentPanel) {
            this.fileRecords = fileRecords;
            this.gc = gc;
            this.parentPanel = parentPanel;
        }
        @Override
        protected Void doInBackground() {
            Iterator<FileRecord> iterator = fileRecords.iterator();
            while (iterator.hasNext()) {
                publish(getTabel(iterator.next()));
                Thread.yield();
            }
            return null;
        }
        @Override
        protected void process(final List<JTable> tables) {
                    Iterator<JTable> iterator = tables.iterator();
                    while(iterator.hasNext()) {
                        JTable table = iterator.next();
                        gc.fill = 1;
                        parentPanel.add(table.getTableHeader(), gc);
                        gc.gridy++;
                        parentPanel.add(table,gc);
                        gc.gridy++;
                        System.out.println("Sequence Nr : " + table.getModel().getValueAt(0,1) + " - Counter :" + counter++);
                    }
        }
    }

It seems to work... mostly. my problem is that in my JFrame constructor i set

this.setExtendedState(JFrame.MAXIMIZED_BOTH);

so the frame opens up on the entire screen. What i want is as the swingworker completes adding the JTable to my panel in the process method it should add those tables to the panel and display them. so it should display as it gets them all the way up until the entire file is read. This doesnt happen. In fact the display just stays blank until i resize the frame, then it actually displays the tables.

I tried to call Outerclass.this.repaint() from within the process method in order to rapaint my Jframe but that didnt work...

Any advice?

Thanks

Answer 1

A complete design in beyond the scope of SO, but here's an outline:

• Use SwingWorker to manage latency while reading the source data; parse the data in your implementation of doInBackground() and publish() records as they arrive; the first few records will be visible immediately, even though the rest may take some time.

• Create a master JTable whose model contains a row for each Record; JTable rendering provides efficient scrolling even for thousands of rows; this table only needs to display enough information for the user to select the desired rows(s).

• Add a ListSelectionListener to the master table; in the listener, update an adjacent JTable with details from the master row; use setModel(), illustrated here, to update the detail table.

Answer 2

This is a real life issue and I am going in a buffer direction over all JTables. In my scenario I have over 50k JTables and the single instantiation is taking over 40minutes of CPU time. Swing messed it up.

My approach as I said will be to create a buffer with a certain number of JTables calculated upon the current parent container currentSize() in pixels. I suggest you create something like 4 or 5 sets of JTables up and down to have a good performance. In case the user does a really loooong scrolling agressively he will get a unavoidable latency that will be as bad as heavy is your components inside each JTable.

Now comes the bad part of it ... you will have to synchronize the scrolling with the buffer size. As you scroll up or down you will be managing the buffer size creating and destroying JTables and hopefully garbagged in some efficient way the collector.

It sounds a little complex but is the only robust solution I can imagine.

good luck and have fun !