Platform.runLater, too slow for processing large amount of tasks

Question

public class ReadFromQueue
 {
     order = orderFromQueue;

     if(!Repository.ordersIdMap.containsKey(order.orderID))
     {    

         Platform.runLater(new Runnable() 
         {
            @Override public void run() 
            {                                                                    
                Repository.ordersCollection.add(order);                        
            }
         } 
      });
      Repository.ordersIdMap.put(order.orderID, order);
  }

So hi again, I made another question, cause the previous one.. was bad (sorry) Let me put you in this scenario. You have a reader that goes to a queue and gets an order (lets say its a order object, ready to use), this thing work so quick that gets a lot of orders per second.. (To be exact i get 40.000 orders in less than a minute).. I have a Repository that is a singleton class, in this class i have a ordersIdMap (ConcurrentHashMap of key String and value Order) and a ordersCollection that is a ObservableList (the source of my tableView). I CAN'T add the order if already exist in the collection so in my map i save the orderId (string) as the key, so that way if the same order cames again i must update it (the else code is not here but is not importan now). The thing is that calling the Platform.runLater for drawing to the UI is giving me problems.. why? cause if i go to get another order and the Platform.runLater hasn't finished.. the order doesn't get created on the map, so to my "reader" the order its new and create it again (in the case that the order has the same orderId), so i get the same order over and over again.. I must say that some times its quick enough and the order gets the update .. but most of the times is too slow and the order its created again. I also tried to put the "Repository.ordersIdMap.put(order.orderID, order);" right next to the if condition.. so that way the map will have the key no matter what..BUT still doesn't work (why..? dunno).. Also if i dont use the platform.runlater ..its works but i get a lot NullPointersException.. cause im trying to update the UI to quick..i think.. any answer is usefull!!.. THANKS! and sorry for my english.

EDIT: This is the entire code.. the execRpt is a like a "small order" and from that i can create a bigger order.. i receive many execRpt and then create the orders.. if the order exist update..if not add it, but it fails on updating (some times) and just add the order like if it was a new one.

package com.larrainvial.trading.trademonitor.listeners;

import com.larrainvial.trading.emp.Controller;
import com.larrainvial.trading.trademonitor.Repository;
import com.larrainvial.trading.emp.Event;
import com.larrainvial.trading.emp.Listener;
import com.larrainvial.trading.fix44.events.ReceivedExecutionReportEvent;
import com.larrainvial.trading.trademonitor.events.CalculatePositionsEvent;
import com.larrainvial.trading.trademonitor.vo.ExecRptVo;
import com.larrainvial.trading.trademonitor.vo.OrderVo;
import javafx.application.Platform;
import javafx.concurrent.Task;
import quickfix.FieldNotFound;
import quickfix.fix44.ExecutionReport;

public class ReceivedExecutionReportToMonitorListener implements Listener 
{
    private OrderVo orderVo;        
    private String ordStatus = "";
    private String transactTime = "";
    private String text = "";
    private int qty = 0;
    private int cumQty = 0;
    private int lastQty = 0;
    private int leavesQty = 0;
    private double price = 0;
    private double avgPx = 0;
    private double lastPx = 0;

    @Override
    public void eventOccurred(Event event) 
    {
        ExecutionReport executionReport = ((ReceivedExecutionReportEvent)event).message;
        try 
        {               
            String settlType = "";
            String orderID = executionReport.isSetOrderID()? String.valueOf(executionReport.getOrderID().getValue()) : "";
            String execID = executionReport.isSetExecID()? String.valueOf(executionReport.getExecID().getValue()) : "";
            String execType = executionReport.isSetExecType()? String.valueOf(executionReport.getExecType().getValue()) : "";            
            String clOrdID = executionReport.isSetClOrdID()? String.valueOf(executionReport.getClOrdID().getValue()) : "";            
            String clOrdLinkID = executionReport.isSetClOrdLinkID()? String.valueOf(executionReport.getClOrdLinkID().getValue()) : "";            
            transactTime = executionReport.isSetTransactTime() ? String.valueOf(executionReport.getTransactTime().getValue()) : "";
            text = executionReport.isSetText() ? executionReport.getText().getValue().toString() : ""; 
            String tif = executionReport.isSetTimeInForce() ? String.valueOf(executionReport.getTimeInForce().getValue()) : ""; 
            String handlInst = executionReport.isSetHandlInst() ? String.valueOf(executionReport.getHandlInst().getValue()) : "";     
            String securityExchange = executionReport.isSetSecurityExchange()? String.valueOf(executionReport.getSecurityExchange().getValue()) : "";     
            String orderType = executionReport.isSetOrdType()? String.valueOf(executionReport.getOrdType().getValue()) : "";                 
            String account = executionReport.isSetAccount() ? String.valueOf(executionReport.getAccount().getValue()) : "None";
            ordStatus = String.valueOf(executionReport.getOrdStatus().getValue());            
            lastPx = executionReport.isSetLastPx()? executionReport.getLastPx().getValue() : 0;
            price = executionReport.isSetPrice()? executionReport.getPrice().getValue() : 0;
            avgPx = executionReport.isSetAvgPx()? executionReport.getAvgPx().getValue() : 0;            
            lastQty = executionReport.isSetLastQty()? (int)executionReport.getLastQty().getValue() : 0;
            leavesQty = executionReport.isSetLeavesQty()? (int)executionReport.getLeavesQty().getValue() : 0;
            cumQty = executionReport.isSetCumQty()? (int)executionReport.getCumQty().getValue() : 0;
            qty = executionReport.isSetOrderQty()? (int)executionReport.getOrderQty().getValue() : 0;            

            ExecRptVo execRpt =  new ExecRptVo(orderID,
                                          execID,
                                          execType, 
                                          ordStatus, 
                                          clOrdID, 
                                          clOrdLinkID, 
                                          securityExchange,
                                          String.valueOf(executionReport.getSide().getValue()), 
                                          qty, 
                                          lastQty, 
                                          leavesQty, 
                                          cumQty, 
                                          executionReport.getSymbol().getValue().toString(), 
                                          orderType, 
                                          price, 
                                          lastPx, 
                                          avgPx, 
                                          tif, 
                                          "",
                                          handlInst, 
                                          securityExchange, 
                                          settlType, 
                                          account, 
                                          text, 
                                          transactTime);

            orderVo = new OrderVo(execRpt);
            OrderVo orderExist = Repository.ordersIdMap.putIfAbsent(orderID, orderVo);

            if(orderExist == null)
            {                                                                                             
                Platform.runLater(new Runnable() 
                {
                    @Override public void run() 
                    {
                        Repository.ordersCollection.add(orderVo);
                    }                                         
                });
            }        
            else
            {              
                Repository.ordersIdMap.get(orderID).price.set(price);
                Repository.ordersIdMap.get(orderID).qty.set(qty);
                Repository.ordersIdMap.get(orderID).ordStatus.set(ordStatus);
                Repository.ordersIdMap.get(orderID).transactTime.set(transactTime);
                Repository.ordersIdMap.get(orderID).text.set(text);

                if(avgPx > 0)
                    Repository.ordersIdMap.get(orderID).avgPx.set(avgPx);
                if(cumQty > 0)
                    Repository.ordersIdMap.get(orderID).cumQty.set(cumQty);
                if(lastQty > 0)
                    Repository.ordersIdMap.get(orderID).lastQty.set(lastQty);
                if(lastPx > 0)
                    Repository.ordersIdMap.get(orderID).lastPx.set(lastPx);
                if(leavesQty > 0)
                    Repository.ordersIdMap.get(orderID).leavesQty.set(leavesQty);
                if(ordStatus.equals("8"))
                    Repository.ordersIdMap.get(orderID).rejected.set("1");

                Repository.ordersIdMap.get(orderID).execRpts.add(execRpt);                               
            }            
            if(execType.equals("1") || execType.equals("2") || execType.equals("F"))
            {                            
                CalculatePositionsEvent calculatePositionsEvent = new CalculatePositionsEvent(execRpt);
                Controller.dispatchEvent(calculatePositionsEvent);                
            }
        } 
        catch (FieldNotFound ex) 
        {
            ex.printStackTrace();
        } 
        catch (Exception ex) 
        {
            ex.printStackTrace();
        }
    }
}

Questions like this make me wish for the too specific close classification. — Gray, Sep 05 '13 at 21:05
Your English seems fine, but I still can't understand what you're asking. — pamphlet, Sep 05 '13 at 21:07
Its hard to explain.. but the main problem about this.. is that i cant repeat orders on the ordersCollection.. but the if condition its not working always.. some times the orderId key doesnt exist (when it should exist) in the hashMap and it's weird ..cause the code i think is ok. — Gabriel Figox Vargas Fierro, Sep 05 '13 at 21:32

score 2 · Answer 1 · answered Sep 05 '13 at 23:22

2

One problem in your code is that you have a check-then-act pattern which is not atomic. One workaround would be something like:

order = orderFromQueue;
Order alreadyInMap = Repository.ordersIdMap.putIfAbsent(order.orderID, order);

if(alreadyInMap == null) { //it really is a new order
    Platform.runLater(new Runnable() {
        @Override public void run() {
            Repository.ordersCollection.add(order);
        }
    });
}

Also note that "cause if i go to get another order and the Platform.runLater hasn't finished.. the order doesn't get created on the map" does not make sense: when you call Platform.runLater(), the runnable is put in a queue but is executed asynchronously (unless your method already runs on the FX Thread).

answered Sep 05 '13 at 23:22

assylias

321,522
82
660
783

1

I like assylias' example, there is an executable sample based on this pattern, in the [ItemProcessor class here](http://codetidy.com/6574) that was created for a somewhat similar StackOverflow question on [complex JavaFX concurrency patterns](http://stackoverflow.com/questions/18530493/complex-concurrency-in-javafx-using-observablelists-and-properties-from-multipl). – jewelsea Sep 06 '13 at 00:58
Hello to both of you and thank you for the answer. I tried this, but still doesn't work, why?, i don't know how it happens but some times the "alreadyInmap" returns null when the order already exists.. how can this happen? i really don't get it.. cause i already added the order in the "putIfAbsent" method.. i don't have an explanation for this, i just don't get it, cause it seems imposible to fail, but it does, i wanna cry.. – Gabriel Figox Vargas Fierro Sep 06 '13 at 13:33
`alreadyInMap` will always be null if the order already exists. Can you clarify what "doesn't work"? – assylias Sep 06 '13 at 13:51
I edited my question and put the entire code. What "doesn't work?" the "else" part.. cause instead of updating, it creates a new one, i just cant understand how can this posible happend.. – Gabriel Figox Vargas Fierro Sep 06 '13 at 14:01
@GabrielFigoxVargasFierro One issue is that your list of `Repository.ordersIdMap.get(orderID).xxx.set()` in the else block is not atomic - so if you receive two orders with the same id at the same time, they will set things in random order and `Repository.ordersIdMap.get(orderID)` will be a mix of two (or more) orders. Also, I don't know how `OrderVo` is implemented, but unless the setters (as in `avg.set()`) are not properly synchronized, you have no guarantee that the changes will be visible by other threads. Overall there seem to be quite a few concurrency issues in your code. – assylias Sep 06 '13 at 14:11
But once an order with a given orderId is inserted in your map, no other order with the same id will replace it. – assylias Sep 06 '13 at 14:12
@assylias: Thank you for all your help!. The "else" doesn't matter to me right now, i will check that anyway. I just made another test with exact code, but not using the Platform.runLater, and it works, but throws me an error in the output console, doesnt really matter to me right now cause it works.. so i can see that the problem is the Platform.runLater, i think that if i want to use that i will have to us the synchronized method in some way.. to fix this problem. Im gonna start investigating about it – Gabriel Figox Vargas Fierro Sep 06 '13 at 14:57
Still can't fix this..The only thing i know is that the Platform.runLater its giving me problems.. cause if i dont use it..it works perfect! but throws me exceptions.. – Gabriel Figox Vargas Fierro Sep 06 '13 at 16:44
@GabrielFigoxVargasFierro Not sure if it is related, but if the Runnable you submit to Plaftorm.runLater throws an exception, the exception is "swallowed" (otherwise it would stop the whole JavaFX Thread). See also: http://stackoverflow.com/a/12798390/829571. – assylias Sep 06 '13 at 17:00
I MADEEEEEEEEEEEEE IT!!!!! using synchronized, a object and a boolean.. if u want that code i can answer to myself on my own question..lol, Thank you! i can go out and celebrate now! – Gabriel Figox Vargas Fierro Sep 06 '13 at 17:02

Platform.runLater, too slow for processing large amount of tasks

1 Answers1