4

I'm doing some self-learning and experimentation with algorithmic trading and the IB API. I decided to use Java but I'm open to switching to C++. I went through an online tutorial that walks you through the code shown below but was wondering about extending it past just one stock. I want to go through all SP500 stocks and check ticker data to make decisions based on that.

The code below will create a contract for and get data for Microsoft but I'd like to get data for all 500 stocks. All of the other methods defined in the EWrapper interface were left out of the post for more ease of readability.

I'm thinking that I need to store the ticker symbols in a file, parse this, and add each contract one by one to a vector. However, I'm not sure about how to monitor the data after that. It would be nice if I could just sequentially loop through each ticker and make a request for data but I believe the stream is processed on an asynchronous thread (correct me if wrong.)

So how do I go through all 500 stocks and check their ticker data?

Code snippets and explanations would be appreciated. Thanks!

// Import Java utilities and Interactive Brokers API                                            
import java.util.Vector;
import com.ib.client.Contract;
import com.ib.client.ContractDetails;
import com.ib.client.EClientSocket;
import com.ib.client.EWrapper;
import com.ib.client.Execution;
import com.ib.client.Order;
import com.ib.client.OrderState;
import com.ib.client.TagValue;
import com.ib.client.CommissionReport;
import com.ib.client.UnderComp;

// RealTimeBars Class is an implementation of the                                               
// IB API EWrapper class                                                                        
public class RealTimeBars implements EWrapper
{
    // Keep track of the next ID                                                                
    private int nextOrderID = 0;
    // The IB API Client Socket object                                                          
    private EClientSocket client = null;

    public RealTimeBars ()
    {
        // Create a new EClientSocket object                                                    
        client = new EClientSocket (this);
        // Connect to the TWS or IB Gateway application                                         
        // Leave null for localhost                                                             
    // Port Number (should match TWS/IB Gateway configuration                               
        client.eConnect (null, 7496, 0);

        // Pause here for connection to complete                                                
    try
            {
                // Thread.sleep (1000);                                                         
                while (! (client.isConnected()));
            } catch (Exception e) {
            e.printStackTrace ();

        };
        // Create a new contract                                                                
        Contract contract = new Contract ();
        contract.m_symbol = "MSFT";
        contract.m_exchange = "SMART";
        contract.m_secType = "STK";
    contract.m_primaryExch = "NASDAQ";
        contract.m_currency = "USD";
        // Create a TagValue list                                                               
        Vector<TagValue> realTimeBarsOptions = new Vector<TagValue>();
        // Make a call to start off data retrieval                                              
        client.reqRealTimeBars(0, contract,
                               5,            // Bar Size 5 seconds                              
                               "TRADES",     // whatToShow                                      
                               false,         // useRTH                                         
                               realTimeBarsOptions);
        // At this point our call is done and any market data events                            
        // will be returned via the realtimeBar method                                          

    } 

public static void main (String args[])
{
    try
        {
            // Create an instance                                                           
            // At this time a connection will be made                                       
    // and the request for market data will happen                                  
            RealTimeBars myData = new RealTimeBars();
        }
    catch (Exception e)
        {
            e.printStackTrace ();
        }
}    

}

brian
  • 10,619
  • 4
  • 21
  • 79
jonnyd42
  • 490
  • 1
  • 9
  • 23
  • What is your question? – JB Nizet Jul 01 '16 at 06:17
  • @JBNizet not sure how to get data for all 500 and loop through it, comparing it against a condition at each iteration. – jonnyd42 Jul 01 '16 at 06:28
  • You can't do this with realTimeBars unles you have paid for 500 lines of data subscription, the default is 100. You can only do it with reqMarketData() and set snapshot the true. As snapshot implies, it's a one time thing, not a subscription to streaming data. IB isn't really a great data provider for this situation but you could try with snapshot first. – brian Jul 07 '16 at 15:27
  • The realTimeBars is if you want 5 second bars that hopefully include all the highs and lows. The tick by tick data can possibly miss some as it's just sampled a few times per second. Also, I didn't see the question because it wasn't tagged very well, I added tws. – brian Jul 07 '16 at 17:19

1 Answers1

3

I don't know how this will work for all 500, but you can try. The data is from https://raw.githubusercontent.com/datasets/s-and-p-500-companies/master/data/constituents.csv SP

package sp;

import com.ib.client.Contract;
import com.ib.client.EClientSocket;
import com.ib.client.EWrapper;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class SP {
    //just a sample, like this so you can just use Files.lines instead.
    private static List<String> lines = Arrays.asList(new String[]{
        "Symbol,Name,Sector",
        "MMM,3M Company,Industrials",
        "ABT,Abbott Laboratories,Health Care",
        "ABBV,AbbVie,Health Care",
        "ACN,Accenture plc,Information Technology",
        "ATVI,Activision Blizzard,Information Technology",
        "AYI,Acuity Brands Inc,Industrials",
        "ADBE,Adobe Systems Inc,Information Technology",
        "AAP,Advance Auto Parts,Consumer Discretionary",
        "AES,AES Corp,Utilities",
        "AET,Aetna Inc,Health Care",
        "AMG,Affiliated Managers Group Inc,Financials",
        "AFL,AFLAC Inc,Financials",
        "A,Agilent Technologies Inc,Health Care",
        "APD,Air Products & Chemicals Inc,Materials",
        "AKAM,Akamai Technologies Inc,Information Technology",
    });


    public static void main(String[] args) throws InterruptedException{
        EWrapper wrapper = new  Wrapper();
        EClientSocket socket = new EClientSocket(wrapper);
        socket.eConnect("", 4001, 123);
        //supposedly gives frozen last recorded value, not working!
        socket.reqMarketDataType(2);

        AtomicInteger tickerId = new AtomicInteger(0);
        lines.stream().skip(1).forEach(line -> {
            //new cont for every request
            Contract cont = new Contract();
            cont.m_currency = "usd";
            cont.m_exchange = "smart";
            cont.m_secType = "stk";
            cont.m_symbol = line.split(",")[0];
            Data data = new Data(cont, socket);
        });

        //need you own logic for when to end program
        //Thread.sleep(5000);//this thread, Socket starts a reader thread
        //socket.eDisconnect();
    }
}

Wrapper

package sp;

import com.ib.client.CommissionReport;
import com.ib.client.Contract;
import com.ib.client.ContractDetails;
import com.ib.client.EWrapper;
import com.ib.client.Execution;
import com.ib.client.Order;
import com.ib.client.OrderState;
import com.ib.client.TickType;
import com.ib.client.UnderComp;
import java.util.HashMap;
import java.util.Map;

public class Wrapper  implements EWrapper{
    public Map<Integer, Data> dataMap = new HashMap<>();
    public Map<Integer, Strat> orderMap = new HashMap<>();

    //reqMktData snapshots are received here
    @Override
    public void tickPrice(int tickerId, int field, double price, int canAutoExecute) {
        if (field == TickType.LAST) {
            //if you just want the last price
            dataMap.get(tickerId).dataRecd(price);
        }
    } 

    @Override
    public void execDetails(int reqId, Contract contract, Execution execution) {
        orderMap.get(execution.m_orderId).exec(execution);
    }
//snip
}

Data

package sp;

import com.ib.client.Contract;
import com.ib.client.EClientSocket;
import java.util.ArrayList;
import java.util.List;
import java.util.Timer;
import java.util.TimerTask;

public class Data {
            final Contract cont;
    private final EClientSocket socket;
    private final Strat strat;

    private static int nextId = 1; //auto increment for each request
    private final int myId;

    List<Double> prices = new ArrayList<>();
    double lastPrice = -1;

    public Data(Contract cont, EClientSocket socket) {
        this.cont = cont;
        this.socket = socket;
        strat = new Strat(this, socket);
        myId = nextId++;
        ((Wrapper) socket.wrapper()).dataMap.put(myId, this);
        reqData();
//        //call every 10 min
//        Timer timer = new Timer();
//        timer.schedule(new TimerTask() {
//            @Override
//            public void run() {
//                reqData();
//            }
//        }, 10 * 60 * 1000);
    }

    private void reqData(){
        socket.reqMktData(myId, cont, "", false /* true */, null);
    }

    public void dataRecd(double last){
        lastPrice = last;
        prices.add(last);
        strat.check();
    }
}

Strat

package sp;

import com.ib.client.EClientSocket;
import com.ib.client.Execution;

public class Strat {
    public static final int NULL=0, LOOK=1<<0, LONG=1<<1, SHORT=1<<2, WAIT_FILL=1<<3, WAIT_CANCEL=1<<4;
    public int sysState = NULL;
    private final Data data;
    private final EClientSocket socket;

    private static int nextOrderId = 1;

    Strat(Data data, EClientSocket socket) {
        this.data = data;
        this.socket = socket;
        sysState = LOOK;
    }

    void check() {
        System.out.println("should I buy? "+ data.cont.m_symbol + " @ " + data.lastPrice);
        /*if (false && sysState & LOOK == LOOK) {
            ((Wrapper) socket.wrapper()).orderMap.put(nextOrderId, this);
            socket.placeOrder(nextOrderId++, data.cont, new Order());
            sysState = WAIT_FILL;
            nextOrderId++;
        }*/
    }

    public void exec(Execution exec){
        //will be called by wrapper after an exec.
        //sysState = LONG; //or whatever
    }
}
brian
  • 10,619
  • 4
  • 21
  • 79
  • What market data subscriptions do you have? I'm getting an error that I don't have access to the data because I'm not subscribed. I just funded my account. – jonnyd42 Jul 15 '16 at 04:51
  • I have a lot of subscriptions but you need one of the simple bundles that's $10. This data is all level 1, so no need for depth (level 2). – brian Jul 15 '16 at 04:58
  • Will the Network A (NYSE/CTA) be enough? This one is $1.50. Which bundle are you looking at? – jonnyd42 Jul 15 '16 at 05:06
  • It's called a securities and futures bundle. It has all the US stocks and futures. It's $10, but if you have $30 in commissions, it's free. – brian Jul 15 '16 at 13:44
  • Got the data and tested the code you provided. Works great! Since I'm new to Java (coming from C++) I'm running into some issues with overall design. I see that you used a lot of static methods, etc. What is considered good design if I were to redo this code (you said obviously make real classes and not static ones)? I tried to separate stuff but ran into an issue because the SP class creates a new instance of itself and passes that into the EClientSocket. This doesn't necessarily seem like a situation where I need to strictly follow an OOP model. – jonnyd42 Jul 20 '16 at 06:22
  • The more complicated the program, the better it is to design it well. I just meant don't keep everything in one file and use static accessors. There is a C++ API as well if you'd prefer. The SP class is the wrapper as well, see how it implements the EWrapper interface. I would make a separate MyWrapper class and pass that to the socket. The MyWrapper will just have code to pass off the received data to the appropriate Data class. In my programs, my wrapper impl have a bunch of maps from IDs used in requesting data, to the class instance that wants to receive it. – brian Jul 20 '16 at 15:43
  • So if I were to maintain the SP class as an implementation of the EWrapper and add a "MyWrapper" class, the it would be the MyWrapper object passed into the EClientSocket? So you're suggesting that the MyWrapper call methods defined in the SP class, act as another layer of abstraction, and then pass all the results to actual data processing classes? One thing that's confusing me is where I first instantiate an object. Do I just instantiate an instance of "MyWrapper" in the main method and then call all the setup functions I create, etc? Thanks for the patience and detailed answers. – jonnyd42 Jul 24 '16 at 20:22
  • I appreciate this so much. You're really helping me learn here. So in C++ the header file is where you put the class declaration and the implementation goes in the cpp file. You end up with a lot of files but it helps organization in the end. With Java, should each of these (Data, Strategy, MyWrapper, and Main) be in their own file? Apart from organization, is there any other benefit to this? – jonnyd42 Jul 25 '16 at 01:10
  • They'll all be in their own file, Data is in Data.java which gets compiled to Data.class etc.. There's no includes but you have to import what you use unless it's in the same package (java's name for folder). The benefits are the encapsulation every language promises. There are cpp APIs if you like, I've never used them as I think a mistake prone language (for me) is no place for my money! – brian Jul 25 '16 at 01:26
  • @jonnyd42 I refactored it to separate classes. It's simple to do, just harder to download and run for others. The testing isn't working tonight so... – brian Jul 25 '16 at 02:19
  • Thanks! Nothing is working for me right now. I have older versions of code with basic functionality and I can't even get my data to stream in. This is annoying because I work during trading hours and can't work on this side project easily :(. Does IB usually have data streaming issues when trading hours are over? – jonnyd42 Jul 26 '16 at 06:13
  • Stocks don't usually trade at night, try futures. eg. sym = 'es', expiry = '201609' exch = 'globex, type = 'fut' – brian Jul 26 '16 at 14:53
  • Right, but shouldn't we still be able to see the ticker data? – jonnyd42 Jul 26 '16 at 15:37
  • That's what reqMarketDataType(2) is supposed to do but snapshots seem a bit flaky. You said 'stream' so I figured you meant continuously updating. – brian Jul 26 '16 at 15:45
  • For non programming questions, I suggest https://groups.yahoo.com/neo/groups/TWSAPI/info There you can ask about all the quirks of trading with IB. – brian Jul 26 '16 at 15:48
  • I'd like to debug an issue where it's only displaying AKAM as the ticker. It will also help me get acquainted with how java's object system works. I tried jdb but it doesn't seem to provide a very clear insight into what's going on. Do you use an IDE? If so, what should I use? – jonnyd42 Jul 27 '16 at 06:05
  • I use netbeans. Wait till after 9:30. My guess is it's the only quote. – brian Jul 27 '16 at 13:17
  • I added aapl and I got 2 quotes for aapl. There's a bug where I keep using the same contract and use it for the symbol to print out. I'll fix the code. – brian Jul 27 '16 at 13:23
  • I understand that a new data object is made for each contract (ticker), and its constructor makes a strat object which has access to lastPrice to make a decision via the check function. I also understand that tickPrice is a callback and dataRecd gets called after reqMktData does the callback. However, after 1 iteration of all of these gets called, how would I go about re-calling continuously without making new data objects? I can access all the contracts by iterating over dataMap, using getKey to get the Data object, then the contracts. Where is the best place to make the reqMktData call? – jonnyd42 Aug 02 '16 at 05:13
  • @jonnyd42 I just tried by putting a timer in Data but it doesn't work very well. The snapshots arrive unreliably. If you want continuous data, it's better to just get streaming data and limit yourself to 100 or buy more. – brian Aug 02 '16 at 15:42
  • I changed the call to reqMktData to be a stream by changing snapshot bool to false: socket.reqMktData(nextId, cont, "", false, null). The incoming data is asynchronous, right? Wouldn't it be best to make purchasing decisions with atomic instructions? It'd be nice to have orders block so that I don't simultaneously identify two buy points and attempt to over-buy. What method have you used in the past to deal with the asynchronous data? – jonnyd42 Aug 02 '16 at 15:54
  • How do you access the bid, ask, open, high, close, etc? I tried removing the conditional in tickPrice and just calling dataRecd. In dataRecd I created a switch statement to switch on the TickTypes and output "last". Output shows that each case is being triggered but the "last" variable ends up just holding the same number. Thanks for walking me through this; the API documentation hasn't been too helpful and I feel lost. – jonnyd42 Aug 05 '16 at 15:40
  • Also, if I block the thread that SP runs in, will data still stream in and update lastPrice from within the data class? – jonnyd42 Aug 10 '16 at 06:49