I have a HashMap<String, CompletableFuture<HashMap<String, String>>> mapping an item to its attributes, e.g. { "pizza" -> { "calories" -> "120", "fat" -> "12" } }, where the attributes are retrieved from different data sources.
For example, we get the "fat" attribute from the database, while we get the "calories" attribute from Solr.
When I initially retrieve "fat" from the DB, I use supplyAsync in order to not block the main thread, something like:
public CompletableFuture<HashMap<String, String>> getFat(String name, boolean) {
return CompletableFuture.supplyAsync(new Supplier<HashMap<String, String>>() {
@Override
public HashMap<String, String> get() {
HashMap<String, String> attributes = new HashMap<>();
... do work ...
attributes.put(name, attributes);
return attributes;
}
})
}
I then chain this with an asynchronous call to Solr so that I will eventually have an asynchronous Hashmap mapping items to their attributes, namely HashMap<String, CompletableFuture<HashMap<String, String>>> itemsToAttributesMapping; (so I loop through the keys of the hashmap and update the value with the new attributes, accessing the old ones using thenApply).
I finish by mapping the data to a csv, which is where the problem arises:
File file = new File(home + "/Downloads/rmsSkuValidationResults.csv");
try{
FileWriter outputfile = new FileWriter(file);
CSVWriter writer = new CSVWriter(outputfile);
for(String itemKey : itemsToAttributesMapping.keySet()) {
itemsToAttributesMapping.get(itemKey).thenAccept(attributes -> {
String[] row = { attributes.get("calories"),
attributes.get("fat")
... more attributes ...
};
writer.writeNext(row);
});
}
writer.close();
}
catch(Exception e){
e.printStackTrace();
}
As is, the printing to the CSV file works fine for some 800-1100 items, but stops writing after that and the program terminates.
I have tried variations to the above, including using get instead of thenAccept, or adding join after the thenAccept leading to the program hanging (async computations are quick, shouldn't hang).
I have also tried storing the results of the thenAccepts I run, and then calling allOf on them, but this leads to strange behavior where the attributes from Solr stop showing up after a few hundred items. The attributes from the database do show up for every item, which makes me think the first supplyAsync always works, but the subsequent thenApplys that add attributes to the original HashMap<String, CompletableFuture<HashMap<String, String>>> itemsToAttributesMapping; provided by the supplyAsnc stop working.
Any insight on what might be the problem would be much appreciated. Perhaps my understanding of CompletableFuture is incorrect, especially on chaining and resolving futures? Maybe this is a timeout issue, or the threads are being lost? The last approach I expanded upon suggests perhaps the issue is the thenApplys?
