Trying to view thousands of thumbnails in JavaFX and it's way too slow

Question

I have a need to view thousands of thumbnails very quickly in a cross-platform application (labeling/verifying images for machine learning). I wrote a thumbnail manager that takes care of creating 200-pixel-high thumbnails (for example) as needed. I wrote a JavaFX app that creates a ScrollPane with a TilePane with 2000 children, each with an ImageView that contains one of these 200x200 images read from disk into an ImageBuffer and converted to a JavaFX Image. I load, convert and add the images to the TilePane in the background (using Platform.runLater), and that all seems to work well.

With 2000 thumbnails at 200x200, the TilePane scrolls really fast, just like I was hoping. But at 400x400, or when I go to 16000 thumbnails (even at 100x100), the display slows to a crawl, with a "spinning lollipop" for several seconds between each screen update.

I'm running with 6GB allocated to the JVM. I told each ImageView to setCache(true) and setCacheHint(CacheHint.SPEED). Everything is loaded into memory and already rendered, and it's still really slow.

Is JavaFX doing a bunch of scaling of images or something on the fly? I'm just wondering what I can do to make this a lot faster.

Below is a sample of what I'm doing, except that this example generates images from scratch instead of reading thumbnails (and generating if needed). But it reproduces the problem:

• With 200 panes, it runs nice and fast (on my laptop).
• With 2000 panes, it is annoyingly slow.
• With 16000 panes, it spins for several seconds between updates, which is unusable.

import java.awt.*;
import java.awt.image.BufferedImage;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.CacheHint;
import javafx.scene.Scene;
import javafx.scene.control.ScrollPane;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.TilePane;
import javafx.stage.Stage;

public class ThumbnailBrowser extends Application {
  public static void main(String[] args) {
    launch(args);
  }

  @Override
  public void start(Stage primaryStage) {
    // Create a Scene with a ScrollPane that contains a TilePane.
    TilePane tilePane = new TilePane();
    tilePane.getStyleClass().add("pane");
    tilePane.setCache(true);
    tilePane.setCacheHint(CacheHint.SPEED);

    ScrollPane scrollPane = new ScrollPane();
    scrollPane.setFitToWidth(true);
    scrollPane.setContent(tilePane);

    Scene scene = new Scene(scrollPane, 1000, 600);
    primaryStage.setScene(scene);

    // Start showing the UI before taking time to load any images
    primaryStage.show();

    // Load images in the background so the UI stays responsive.
    ExecutorService executor = Executors.newFixedThreadPool(20);
    executor.submit(() -> {
      addImagesToGrid(tilePane);
    });
  }

  private void addImagesToGrid(TilePane tilePane) {
    int size = 200;
    int numCells = 2000;
    for (int i = 0; i < numCells; i++) {
      // (In the real application, get a list of image filenames, read each image's thumbnail, generating it if needed.
      // (In this minimal reproducible code, we'll just create a new dummy image for each ImageView)
      ImageView imageView = new ImageView(createFakeImage(i, size));
      imageView.setPreserveRatio(true);
      imageView.setFitHeight(size);
      imageView.setFitWidth(size);
      imageView.setCache(true);
      imageView.setCacheHint(CacheHint.SPEED);
      Platform.runLater(() -> tilePane.getChildren().add(imageView));
    }
  }

  // Create an image with a bunch of rectangles in it just to have something to display.
  private Image createFakeImage(int imageIndex, int size) {
    BufferedImage image = new BufferedImage(size, size, BufferedImage.TYPE_INT_RGB);
    Graphics g = image.getGraphics();
    for (int i = 1; i < size; i ++) {
      g.setColor(new Color(i * imageIndex % 256, i * 2 * (imageIndex + 40) % 256, i * 3 * (imageIndex + 60) % 256));
      g.drawRect(i, i, size - i * 2, size - i * 2);
    }
    return SwingFXUtils.toFXImage(image, null);
  }
}

Update: It turns out that if I replace "TilePane" with "ListView" in the above code, then it scrolls nice and fast, even with 16,000 tiles. But then the problem is that it's in a single vertical list instead of a grid of thumbnails. Perhaps I should ask this as a new topic, but this leads me to the question of how I can extend a ListView to display its elements in a (fixed-size) 2-D grid instead of a 1-D list.

Answer 1

I found an open-source GridView control that seeks to mimic what ListView does, but in a grid, which is what I was looking for. It seems to work great. It doesn't seem to have multi-select built in like ListView does, but I can look at adding support for that (and ideally submit that back to the open source project).

Here is code that demonstrates its use. I had to do the following Maven include:

<dependency>
  <groupId>org.controlsfx</groupId>
  <artifactId>controlsfx</artifactId>
  <version>8.0.6_20</version>
</dependency>

And then here is the Java code. I was having trouble with all of the "Platform.runLater()" calls saturating the JavaFX UI thread, making the UI unresponsive. So now the background thread puts all the images on a concurrent queue (as a "producer"), and yet another thread (a "consumer") reads up to 1000 images off of the queue and adds them to a temporary list, and then does a single call via "Platform.runLater()" to add those to the UI in one action. It then blocks and waits for a semaphore to be released by the runLater() call before gathering another batch of images to send to the next call to runLater(). That way, the UI can respond while images are being added to the grid.

import java.awt.*;
import java.awt.image.BufferedImage;
import java.util.ArrayList;
import java.util.List;
import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.CacheHint;
import javafx.scene.Scene;
import javafx.scene.control.ScrollPane;
import javafx.scene.image.Image;
import javafx.stage.Stage;
import org.controlsfx.control.GridView;
import org.controlsfx.control.cell.ImageGridCell;

// Demo class to illustrate the slowdown problem without worrying about thumbnail generation or fetching.
public class ThumbnailGridViewBrowser extends Application {
  private static final int CELL_SIZE = 200;
  private final ExecutorService executor = Executors.newFixedThreadPool(10);

  public static void main(String[] args) {
    launch(args);
  }

  @Override
  public void start(Stage primaryStage) {
    // Create a Scene with a ScrollPane that contains a TilePane.
    GridView<Image> gridView = new GridView<>();
    gridView.setCellFactory(gridView1 -> new ImageGridCell());
    gridView.getStyleClass().add("pane");
    gridView.setCache(true);
    gridView.setCacheHint(CacheHint.SPEED);
    gridView.setCellWidth(CELL_SIZE);
    gridView.setCellHeight(CELL_SIZE);
    gridView.setHorizontalCellSpacing(10);
    gridView.setVerticalCellSpacing(10);

    ScrollPane scrollPane = new ScrollPane();
    scrollPane.setFitToWidth(true);
    scrollPane.setFitToHeight(true);
    scrollPane.setContent(gridView);

    primaryStage.setScene(new Scene(scrollPane, 1000, 600));

    // Start showing the UI before taking time to load any images
    primaryStage.show();

    // Load images in the background so the UI stays responsive.
    executor.submit(() -> addImagesToGrid(gridView));

    // Quit the application when the window is closed.
    primaryStage.setOnCloseRequest(x -> {
      executor.shutdown();
      Platform.exit();
      System.exit(0);
    });
  }

  private static final Image POISON_PILL = createFakeImage(1, 1);

  private void addImagesToGrid(GridView<Image> gridView) {
    int numCells = 16000;
    final Queue<Image> imageQueue = new ConcurrentLinkedQueue<>();
    executor.submit(() -> deliverImagesToGrid(gridView, imageQueue));
    for (int i = 0; i < numCells; i++) {
      // (In the real application, get a list of image filenames, read each image's thumbnail, generating it if needed.
      // (In this minimal reproducible code, we'll just create a new dummy image for each ImageView)
      imageQueue.add(createFakeImage(i, CELL_SIZE));
    }
    // Add poison image to signal the end of the queue.
    imageQueue.add(POISON_PILL);
  }

  private void deliverImagesToGrid(GridView<Image> gridView, Queue<Image> imageQueue) {
    try {
      Semaphore semaphore = new Semaphore(1);
      semaphore.acquire(); // Get the one and only permit
      boolean done = false;
      while (!done) {
        List<Image> imagesToAdd = new ArrayList<>();
        for (int i = 0; i < 1000; i++) {
          final Image image = imageQueue.poll();
          if (image == null) {
            break; // Queue is now empty, so quit adding any to the list
          }
          else if (image == POISON_PILL) {
            done = true;
          }
          else {
            imagesToAdd.add(image);
          }
        }

        if (imagesToAdd.size() > 0) {
          Platform.runLater(() -> 
          {
            try {
              gridView.getItems().addAll(imagesToAdd);
            }
            finally {
              semaphore.release();
            }
          });
          // Block until the items queued up via Platform.runLater() have been processed by the UI thread and release() has been called.
          semaphore.acquire();
        }
      }
    }
    catch (InterruptedException e) {
      Thread.currentThread().interrupt();
    }
  }

  // Create an image with a bunch of rectangles in it just to have something to display.
  private static Image createFakeImage(int imageIndex, int size) {
    BufferedImage image = new BufferedImage(size, size, BufferedImage.TYPE_INT_RGB);
    Graphics g = image.getGraphics();
    for (int i = 1; i < size; i ++) {
      g.setColor(new Color(i * imageIndex % 256, i * 2 * (imageIndex + 40) % 256, i * 3 * (imageIndex + 60) % 256));
      g.drawRect(i, i, size - i * 2, size - i * 2);
    }
    return SwingFXUtils.toFXImage(image, null);
  }
}

This solution does display 16,000 images with no slowdown, and stays responsive as the images are added. So I think that serves as a good starting point.