Java Observable list causing immense slow down after first use

Question

Introduction

Hello,

I'm working on a java application that makes use of several bidirectional hash-maps and observable lists. My code works but I've come across a bizarre quirk that I can't seem to get rid of. I can't find any information on this online so I thought I'd share it and see if anyone else has ever seen this before.

When the code first opens, there is a method that makes a query to a MySQL database and pulls the names of a list of authors (the program is a library database application) whose names are broken up into separate parts; First Name, Middle Name and Last Name. This method then goes through the cached row set and combines them together then adding them to the bidirectional hash-map and the observable list I mentioned before. These are then used throughout the code for a variety of purposes.

The issue occurs anytime the lists are refreshed after the initial creation. From time to time while the program is running, new queries are made to the database and the list is refreshed to ensure that the most up to date information is available. The first time the method is run, it does all of these steps I mentioned and takes around 8 milliseconds to complete. But when the refreshes are made, that time jumps to around 1,400 milliseconds. Notably, the time gets longer the more refreshes are made. Obviously this is a major spike in time but there's nothing in the code that I can see to indicate what is causing this. I'll explain how I determined that it was the observable list below...

I've linked a picture below to show an example of what I'm talking about. Notice that the number of authors being processed are the same. The other printouts were additonal information that I was testing for at the same time and isn't relevant to this post. enter image description here

This is the code for the method.

public void createAuthorHashMap() throws Exception { String authorFirstName, authorMiddleName, authorLastName, authorFullName; int ID;

    // Get the Cached Row Set for all Authors in the database
    CachedRowSet authorList = connectionCommands.readDatabase(sqlCommands.selectAllAuthor);
        
    // Reset the contents of the hashmap and observable list
    bookAttributes.bidiMapAuthors.clear();
    bookAttributes.listAuthors.clear();
    choiceBoxAuthor.setItems(null);
        
    // Combine the sperate parts of the names from the cached rowset
    while (authorList.next()) {
        ID = authorList.getInt(1);
        authorFirstName = authorList.getString(2);
        authorMiddleName = authorList.getString(3);
        authorLastName = authorList.getString(4);

        if (authorMiddleName == null) {
            authorFullName = (authorFirstName + " " + authorLastName);
        } else {
            authorFullName = (authorFirstName + " " + authorMiddleName + " " + authorLastName);
        }

        // Add the authors to the hashmap and list
        bookAttributes.bidiMapAuthors.put(ID, authorFullName);
        bookAttributes.authors.add(authorFullName);
    }

    // Add the list to the choiceBox and auto-complete textfield
    choiceBoxAuthor.setItems(bookAttributes.authors);
    TextFields.bindAutoCompletion(textFieldAuthor, bookAttributes.listAuthors);
}

And this is the code for the hashmaps and Observable lists.

// Lists
public static ObservableList<String> FictionGenres = FXCollections.observableArrayList();
public static ObservableList<String> NonFictionGenres = FXCollections.observableArrayList();
public static ObservableList<String> blankList=FXCollections.observableArrayList();
public static ObservableList<String> authors = FXCollections.observableArrayList();
public static ObservableList<String> publishers = FXCollections.observableArrayList();
public static ObservableList<String> languages = FXCollections.observableArrayList();
public static ObservableList<String> series = FXCollections.observableArrayList();

// Hashmaps
public static BidiMap<Integer, String> bidiMapAuthors = new TreeBidiMap<>();
public static BidiMap<Integer, String> bidiMapPublishers = new TreeBidiMap<>();
public static BidiMap<Integer, String> bidiMapFictionGenres = new TreeBidiMap<>();
public static BidiMap<Integer, String> bidiMapNonFictionGenres = new TreeBidiMap<>();
public static BidiMap<Integer, String> bidiMapLanguages = new TreeBidiMap<>();
public static BidiMap<Integer, String> bidiMapSeries = new TreeBidiMap<>();

An important note is that the hashmaps and lists are created and stored in a separate class so that they can be accessed across multiple controllers during the program's runtime.

The code itself isn't complicated and is reasonably easy to follow but the part I'd like to draw your attention to are the two lines following the while loop. These are the two lines that add the author's name to the hash-map and list.

bookAttributes.bidiMapAuthors.put(ID, authorFullName);
bookAttributes.authors.add(authorFullName);

Testing

My testing went as such. I put timers around separate sections of the method and ran it multiple times and narrowed the cause of the slow-down to the two lines of code I mentioned before. It should be noted that the timings I got for every other part of the method were exactly what I expected them to be.

I then commented out the line that adds the names to the hash-map and the issue persisted but once I reversed it and commented out the line that adds the names to the list then the issue instantly disappeared and never came back regardless of the number of times I ran the method.

bookAttributes.authors.add(authorFullName);

Would there be any cause for what is going on here or is there an obvious mistake that I overlooked and I'm just an idiot? I'm very curious and I appreciate any information!

I would suggest to create a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example). You can use randomly generated data instead of SQL. Most probably, while creating it, you would be able to narrow down your problem enough to find the solution yourself, if not, it will help other people to quickly reproduce it, profile/benchmark it and give you the answer. — Aivean, Nov 07 '21 at 22:21
The issue might be that you are creating new autocomplete bindings each time rather than updating the existing one with new values (by clearing and adding new suggestions to the binding). See the accepted answer to https://stackoverflow.com/questions/45778462/update-autocomplete-javafx — sprinter, Nov 07 '21 at 23:27
The obvious (and possibly wrong) answer is that the time is taken by whatever the listeners do when the data changes. Try putting the data into a temporary list and then using `ObservableList.setAll`. — tgdavies, Nov 07 '21 at 23:28
@tgdavies I just tried this and it made a slight improvement but by only around 100 ms. Much like before, the first time this method runs, it does so nearly instantly. I think it might do it faster with the setAll approach but the issue I'm having still persists. The documentation isn't providing me with any kind of useful information either but I'll keep experimenting. — TeaDrinker, Nov 08 '21 at 02:24
@tgdavies I found a solution. I'll post a detailed solution tomorrow. — TeaDrinker, Nov 10 '21 at 00:20

score 0 · Accepted Answer · edited Nov 11 '21 at 12:02

Disclaimer I will admit right now that I'm not 100% certain what exactly is the underlying cause of the problem I had so my solution may not work for you. I've tested this until I nearly went blind and I can only offer a 80-90% certainty.

Cause

The initial issue seems to have been that the while loop was writing to a static observable list in another class. Since static loops are in the global variable pool, this meant that each write to the list took a bit longer than normal. Not a performance serious issue until you do this write a couple hundred times at once.

Side Note This is why I'm not entirely certain if this was in fact the cause of the issue because the hash-maps are also static and in the same external class but they didn't cause the slow down. At no point what soever did they cause any problems. My only guess is that the way Observable lists make changes operates in a different way from hash-maps though I'm not nearly knowledgeable about how the lists work to state that definitively.

Solution

Thanks to the advice of @tgdavies, I solved the problem by creating instances of these lists inside the same class as the method, had the method write to these classes and then once all of the similar methods were done, I copied the contents of the temporary lists into the more permanent lists using the 'addAll' method. This change reduced the runtime immensely.

   ObservableList<String> tempAuthorList=FXCollections.observableArrayList();
   ObservableList<String> tempPublisherList=FXCollections.observableArrayList();
   ObservableList<String> tempFictionGenreList=FXCollections.observableArrayList();
   ObservableList<String> tempNonFictionGenreList=FXCollections.observableArrayList();
   ObservableList<String> tempSeriesList=FXCollections.observableArrayList();
   ObservableList<String> tempLanguageList=FXCollections.observableArrayList();
   ObservableList<String> blankList=FXCollections.observableArrayList();

        bookAttributes.obvListAuthors.addAll(tempAuthorList);
        bookAttributes.obvListPublishers.addAll(tempPublisherList);
        bookAttributes.obvListFictionGenres.addAll(tempFictionGenreList);
        bookAttributes.obvListNonFictionGenres.addAll(tempNonFictionGenreList); 
        bookAttributes.obvListLanguages.addAll(tempLanguageList);     
        bookAttributes.obvListSeries.addAll(tempSeriesList);

Another thing I did was also implement a better process for cleaning the choicebox contents. It seemed like this was causing a minor slowdown as well. I got the idea from another commenter named @sprinter and once I did this, any lingering performance problems disappeared.

choiceBoxAuthor.setItems(null); -> choiceBoxAuthor.setItems(blankList);

These fixes caused a few glitches elsewhere that needed to be fixed and I've tested this with a bunch of new values in a variety of ways and the code seems to be running exactly as it should every time. Anything from this point on should simple bug fixes!