How do I get my accounts' positions at Interactive Brokers using Python API?

Question

EDITED: I found a solution regarding the error messages - it was a bug on IB's API. The code I show as an answer below should be useful for those looking for a clean solution to read positions, and also NAVs, from their accounts at IB.

The original question [SEE SOLUTION BELOW; LEAVING ORIGINAL QUESTION HERE FOR CONTEXT]: I'm trying to get all my accounts positions at Interactive Brokers [IB] using the firm's API. Their documentation, although extensive, is extremely confusing. Sample codes are full of unnecessary commands - I want something very streamlined.

I need help with:

1. How to get the information "per account" [SOLVED]
2. How to bring the variables to a DataFrame [SOLVED]
3. How to avoid IB's API from printing a series of error messages [SOLVED]

The code so far:

from ibapi.client import EClient 
from ibapi.wrapper import EWrapper
from ibapi.common import * #for TickerId type
import pandas as pd

class ib_class(EWrapper, EClient): 
    def __init__(self): 
        EClient.__init__(self, self)
        self.all_positions = pd.DataFrame([], columns = ['Account','Symbol', 'Quantity', 'Average Cost'])

    def position(self, account, contract, pos, avgCost):
        index = str(account)+str(contract.symbol)
        self.all_positions.loc[index]=account,contract.symbol,pos,avgCost

    def error(self, reqId:TickerId, errorCode:int, errorString:str):
        if reqId > -1:
            print("Error. Id: " , reqId, " Code: " , errorCode , " Msg: " , errorString)

    def positionEnd(self):
        self.disconnect()

ib_api = ib_class() 
ib_api.connect("127.0.0.1", 7496, 0) 
ib_api.reqPositions()
current_positions = ib_api.all_positions
ib_api.run()

When I run the code above I get a series of (i) account numbers, (ii) the contract symbol, (iii) position and (iv) average cost. This, therefore, answers question #1. You might want to "print" these values to see how IB's API send you the information.

I was also able to define a DataFrame variable all_positions that receives the information I was looking for. See the result below. Note that I had to create an "index" variable that is a unique identifier for each row of the DataFrame (as a combination of the account number and symbol). I didn't find a way to 'append' information to the DataFrame without this 'index' (any idea on how to do it would be welcome):

As for the last issue (the error messages):

Brian's suggestion of the "error" function (see below) got rid of the "-1" errors. But I still get the following:

unhandled exception in EReader thread Traceback (most recent call last): File "C:\Users\danil\Anaconda3\lib\site-packages\ibapi-9.76.1-py3.7.egg\ibapi\reader.py", line 34, in run data = self.conn.recvMsg() File "C:\Users\danil\Anaconda3\lib\site-packages\ibapi-9.76.1-py3.7.egg\ibapi\connection.py", line 99, in recvMsg buf = self._recvAllMsg() File "C:\Users\danil\Anaconda3\lib\site-packages\ibapi-9.76.1-py3.7.egg\ibapi\connection.py", line 119, in _recvAllMsg buf = self.socket.recv(4096) OSError: [WinError 10038] An operation was attempted on something that is not a socket

Answer 1

After a lot of experimentation, here is the final code I wrote, which I placed on a ".py" file named AB_API (so I can "import" it as a library):

from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.common import TickerId
from threading import Thread

import pandas as pd
import time

class ib_class(EWrapper, EClient):

    def __init__(self):
        EClient.__init__(self, self)

        self.all_positions = pd.DataFrame([], columns = ['Account','Symbol', 'Quantity', 'Average Cost', 'Sec Type'])
        self.all_accounts = pd.DataFrame([], columns = ['reqId','Account', 'Tag', 'Value' , 'Currency'])

    def error(self, reqId:TickerId, errorCode:int, errorString:str, advancedOrderRejectJson = ""):
        if reqId > -1:
            print("Error. Id: " , reqId, " Code: " , errorCode , " Msg: " , errorString)

    def position(self, account, contract, pos, avgCost):
        index = str(account)+str(contract.symbol)
        self.all_positions.loc[index]= account, contract.symbol, pos, avgCost, contract.secType
        
    def accountSummary(self, reqId, account, tag, value, currency):
        index = str(account)
        self.all_accounts.loc[index]=reqId, account, tag, value, currency

def connect_to_server(app):

    def run_loop():
        app.run()
    app.connect('127.0.0.1', 7496, 0)
    #Start the socket in a thread
    api_thread = Thread(target=run_loop, daemon=True)
    api_thread.start()
    time.sleep(0.5) #Sleep interval to allow time for connection to server


def read_positions(): #read all accounts positions and return DataFrame with information

    app = ib_class()    
    connect_to_server(app)

    app.reqPositions() # associated callback: position
    print("Waiting for IB's API response for accounts positions requests...\n")
    time.sleep(3)
    current_positions = app.all_positions # associated callback: position
    current_positions.set_index('Account',inplace=True,drop=True) #set all_positions DataFrame index to "Account"
    
    app.disconnect()

    return(current_positions)


def read_navs(): #read all accounts NAVs
    
    app = ib_class()  
    connect_to_server(app)

    app.reqAccountSummary(0,"All","NetLiquidation")  # associated callback: accountSummary / Can use "All" up to 50 accounts; after that might need to use specific group name(s) created on TWS workstation
    print("Waiting for IB's API response for NAVs requests...\n")
    time.sleep(3)
    current_nav = app.all_accounts
    
    app.disconnect()

    return(current_nav)

I can now read all accounts positions and NAVs with two single-line functions:

import IB_API

print("Testing IB's API as an imported library:")

all_positions = IB_API.read_positions()
all_navs = IB_API.read_navs()

If you want to run a very quick test (without importing/calling a function), just below the read_positions() and read_navs(), do the following:

print("Testing IB's API as an imported library:")

all_positions = read_positions()
print(all_positions, '\n')
all_navs = read_navs()
print(all_navs)

I realized some users were getting confused with the concept of importing functions, hence the quick suggestion above.

I wish Interactive Brokers had some simple examples available for clients that would make it easier for someone to test a few ideas and decide if they want to use their API or not.

Answer 2

The error I was getting was generated by a bug on IB's API (you can see another question I asked about the specific error). I'm adding this answer to provide more clear steps on how to downgrade to the older version:

1) Download the older version from here: http://interactivebrokers.github.io/downloads/TWS%20API%20Install%20974.01.msi

IB doesn't show the links for older versions but they keep the file on the Github (note the version name at the end of the link address)

2) Uninstall the current version. From IB's own website, do the following:

• Uninstall the API from the Add/Remove Tool in the Windows Control Panel as usual
• Delete the C:\TWS API\ folder if any files are still remaining to prevent a version mismatch.
• Locate the file C:\Windows\SysWOW64\TwsSocketClient.dll. Delete this file. [didn't find this file]
• Restart the computer before installing a different API version.

3) Install the API using the .msi file

4) Run the Anaconda prompt and navigate to the dir C:\TWS API\source\pythonclient

5) Run: python setup.py install

6) After python setup.py install, restart Spyder (if it was opened; I closed mine before step #5 above)

After I installed this older version, the error disappeared! Now the output on my console is clean again!

Answer 3

Your code is fine. If you want to ignore the errors just override the error callback.

from ibapi.common import * #for TickerId type

def error(self, reqId:TickerId, errorCode:int, errorString:str):
    if reqId > -1:
        print("Error. Id: " , reqId, " Code: " , errorCode , " Msg: " , errorString)

The errors > -1 are for actions that need an ID.

I would still keep track of the connection state if you're getting data or placing orders.

You initialize the ddf twice. You also call super which just logs the data I think.

The socket errors on disconnect are a bug or just the way they do it at IB. It's supposed to have been fixed but it may be a while for it to be in a release. Maybe try updating, I see the pull request to fix it on Apr. 26th.

Answer 4

I guess I don't have enough reputation yet to add a comment to the above code, but the second highest rated code here is badly structured - your imports should be at the top of the file (outside of functions and classes), you also shouldn't define a class nested inside a function, I've actually never seen someone do that in 10 years of coding python. You'll run into scope access issues, and might never even define the variables, probably lose access to "self" etc. Classes should be a grouping of (hopefully) related functions. so the structure should look like:

imports 
class:
   func1()
   func2()
   func3()

Then if you need to call the function you'd do like: class.func1() for example...

To provide an actual example of how you could do this, something like:

from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
import threading

class MyWrapper(EWrapper):
    def __init__(self):
        self.positions = {}

    def position(self, account, contract, position, avgCost):
        self.positions[contract.symbol] = {
            "symbol": contract.symbol,
            "position": float(position),
            "average_cost": avgCost
        }

class IBClient(EClient):
    def __init__(self, wrapper):
        EClient.__init__(self, wrapper)

def request_positions():
    wrapper = MyWrapper()
    client = IBClient(wrapper)
    client.connect("127.0.0.1", 7497, clientId=0)  # Connect to TWS/Gateway
    
    # Request positions
    client.reqPositions()

    # Wait for positions data to be received
    time.sleep(5)  # You might need to adjust the time as needed
    
    # Access the positions data
    positions_data = wrapper.positions
    
    # Filter or process positions as needed
    # e.g., filter by futures or stocks
    
    client.disconnect()

if __name__ == "__main__":
    request_positions()

Then if you wanted to get fancier you could look into threading - re: set and wait (events). Which would allow you to not do "time.sleep" and continue whenever the api responds.