2

I'm writing an app in JavaFX that needs to occasionally load large CSV files around 1,000,000 lines long (or possibly more).

When a user clicks a button to start loading the file, a Service is started to load the contents, with a progress/cancel dialog showing in the meantime. The call() method in the Service is basically a while loop that loads another line from the CSV file on each iteration.

The problem is that when I start the service, the progress bar (indeterminate style) becomes jerky. Dragging the dialog is also jerky and laggy.

I wasn't having good luck searching on the web for a solution, but the closest I found was to put a Thread.sleep() in the loop, giving other things like GC a chance to catch up.

This solution seemed to reduce/remove the stuttering, but it would add a lot of time to loading the data. I am also guessing that the exact time to sleep would vary between different processors.

Is there any way to dynamically figure out how long/often to sleep for? Or call some method that would block for just long enough to keep the GUI responsive?

The code for my service:

public class CSVLoadingService extends Service<List<ObservableList<DoubleProperty>>> {
private ObjectProperty<File> srcFile = new SimpleObjectProperty<>();
private IntegerProperty startIndex = new SimpleIntegerProperty(0);
private ObjectProperty<Character> csvDelimeter = new SimpleObjectProperty(CSVParser.DEFAULT_SEPARATOR);
private DoubleProperty invalidCSVReplacement = new SimpleDoubleProperty(0);
private ObjectProperty<Dialog> dialog = new SimpleObjectProperty<>(null);

@Override
protected Task<List<ObservableList<DoubleProperty>>> createTask() {
    return new Task<List<ObservableList<DoubleProperty>>>() {
        final ObjectProperty<File> _srcFile = srcFile;
        final IntegerProperty _startIndex = startIndex;
        final ObjectProperty<Character> _csvDelimeter = csvDelimeter;
        final DoubleProperty _invalidCSVReplacement = invalidCSVReplacement;

        @Override
        protected ObservableList<ObservableList<DoubleProperty>> call() throws Exception {
            if (_startIndex.getValue() < 0)
                throw new IllegalArgumentException("Start index can't be negative.");
            if (_srcFile.getValue() == null)
                throw new IllegalStateException("File can't be null.");

            final ObservableList<ObservableList<DoubleProperty>> result = FXCollections.observableArrayList();

            // Read the data from the CSV file.
            try (final CSVReader reader = new CSVReader(new BufferedReader(new FileReader(_srcFile.getValue())),
                    _csvDelimeter.getValue(),
                    CSVParser.DEFAULT_QUOTE_CHARACTER,
                    _startIndex.getValue()))
            {
                // Read first line.
                String[] csvLine = reader.readNext();

                // If there is actually data, then read the rest of it.
                if (csvLine == null || csvLine.length == 0) {
                    result.clear();
                } else {
                    // Create columns.
                    for (String value : csvLine) {
                        result.add(FXCollections.observableArrayList());
                    }

                    // Parse the CSV reads and add them to the columns.
                    int iteration = 0;
                    do {
                        int i = 0;
                        for (String value : csvLine) {
                            // Convert the string to a number and add it to the column.
                            try {
                                result.get(i).add(new SimpleDoubleProperty(Double.parseDouble(value)));
                            } catch (NumberFormatException|NullPointerException e) {
                                result.get(i).add(_invalidCSVReplacement);
                            }
                        }

                        iteration++;
                    } while (!isCancelled() && null != (csvLine = reader.readNext()));
                }
            }

            return result;
        }
    };
}

@Override
protected void succeeded() {
    super.succeeded();

    if (dialog.getValue() != null) {
        dialog.getValue().close();
    }
}

@Override
protected void failed() {
    super.failed();

    if (dialog.getValue() != null) {
        dialog.getValue().close();
    }
}
  • 2
    Consider providing a [runnable example](https://stackoverflow.com/help/mcve) which demonstrates your problem. This is not a code dump, but an example of what you are doing which highlights the problem you are having. This will result in less confusion and better responses – MadProgrammer Sep 14 '15 at 00:18
  • 2
    This shouldn't happen if you have things set up correctly. If you're using a `javafx.concurrent.Service` that runs the `Task` you create in a background thread by default, which should prevent any GUI lag. Can you post the code from your `Service`? – James_D Sep 14 '15 at 00:19
  • @James_D I added the code. I used a test CSV file I made that is about 800,000 rows long, 90 MB. –  Sep 14 '15 at 00:34
  • Strange loop. Try it the easy way: `while ((csvLine = reader.readNext()) != null)` instead of the initial read and the do/while. – user207421 Sep 14 '15 at 00:56
  • @EJP The initial read was to get the first line of the CSV file so that I could get the number of columns. –  Sep 14 '15 at 01:02
  • Presumably you intend to increment `i` in the inner loop, though that's sort of off topic. Your memory consumption shouldn't be more than a small factor bigger than the file size (if even that), so assuming you are running on a system with a reasonable amount of memory I don't think memory is what's causing the problem. That said, I don't see much wrong with this; I was kind of guessing that you were flooding the UI thread but it seems you are not doing any continual UI updates. – James_D Sep 14 '15 at 01:07
  • It's difficult with these kinds of issues, but you probably need to create a [MCVE] to get any really useful help. You could either mimic the file loading if you can create the same effect that way, or you could provide code to generate an appropriate file. Make the example as simple as possible to reproduce the issue. – James_D Sep 14 '15 at 16:59

1 Answers1

-1

This is a typical scenario for using thread priorities. You want your GUI thread to have a higher priority than your background thread.

mipa
  • 10,369
  • 2
  • 16
  • 35
  • Do you have any example or anywhere to point me on how I would achieve that? (btw I wasn't the one to vote down.) –  Sep 15 '15 at 18:13
  • Here is an old thread which covers the topic. http://stackoverflow.com/questions/1617963/setting-priority-to-javas-threads – mipa Sep 16 '15 at 09:07