2

Is there a way to use the BQL-formula in Python in the BLPAPI or XBBG API's instead of looping through a bunch of tickers to retrieve data on i.e. all of the stocks of the S&P500 using a BDP or BDS formula? (This will quickly reach the data limit for the day, I suspect, since I want to check a bunch of different indicies).

I found a post from 2019, where BQNT was suggested, but I would prefere to avoid using BQNT, link here: How to implement BQL Bloomberg excel formula to python API (blpapi)?.

Thanks in advance!

Jørgen
  • 37
  • 2
  • 6
  • Bloomberg still haven’t exposed an API for BQL, so its features cannot be accessed from Python (except via their proprietary BQNT). One method might be to drive Excel from Python: get the data via the Excel BQL functions and extract to Python. I haven’t tried this. But getting stock info ticker-by-ticker will perhaps not exceed any limit: it is a common thing for a customer to want to monitor several equity markets at once (even before BQL). I’ve been working with the API for 20y, and haven’t hit any limits (in FI at least). – DS_London Aug 20 '21 at 08:21
  • Hi DS_London, thank you very much for the suggestion! – Jørgen Aug 20 '21 at 09:28
  • 1
    I actually just used BQNT because I want to avoid Excel. I figured out that you can run your BQNT app and then pickle the object to your local BQNT project folder. Then in a different app you can unpickle it. As far as limits go, I accidentally hit a limit when I used BQNT to download the 20K+ constituents of a fixed income index. However, beyond that I have never hit a limit. I just downloaded 15K+ cusips with 10 FLDS. – Eric M Jun 30 '22 at 20:02

3 Answers3

7

Further to the comments, I played around with a proof-of-concept for driving Excel from Python. This quick'n'dirty script opens Excel in the background, puts a BQL formula in a cell, polls for a return value, and fills a DataFrame:

import pandas as pd
import time

import win32com.client as wc

#Get a dispatch interface for the Excel app
_xl = wc.Dispatch("Excel.Application")
#Ensure the Bloomberg addin is loaded
_xl.Workbooks.Open('c:\\blp\\API\\Office Tools\\BloombergUI.xla')

#Create a new workbook
wb = _xl.Workbooks.Add()
ws = wb.Sheets(1)
cl = ws.Cells(1,1) #Cell A1 on Sheet 1

#Define BQL query, and set cell formula
qry ='=@BQL.Query("get(YIELD) for(filter(bonds([\'IBM US Equity\']),CPN_TYP==Fixed and CRNCY==USD))")' 
cl.Formula=qry
_xl.Calculate()

#Check the cell's value: it will likely be #N/A ...
res = cl.Value
nLoop = 0
nTimeout = 100 #ie 10 seconds

#Loop until either get a non-# return or timeout
while res[0]=='#' and nLoop<=nTimeout:
    time.sleep(0.1) #100 ms
    res = cl.Value
    nLoop += 1

if res[0] == '#':
    print('Timed out')
    return

print('Results after {0:} secs'.format(nLoop/10.0))

#The Bloomberg addin will have changed the original BQL formula
#and added a 'cols=x,rows=y' parameter at the end
#This tells us the size of the data
#as BQL doesn't seem to have the option to return a dynamic array
f = cl.Formula
rc = f.split(',')[-1].split(';')
cols  = int(rc[0].split('=')[1])
s = rc[1].split('=')[1]
rows = int(s[0:len(s)-2])

#Retrieve the values from this new range
data = ws.Range(cl,ws.Cells(rows,cols)).Value

#Convert to DataFrame
df=pd.DataFrame(data[1:],columns=data[0])
print(df)

#Tidy up
_xl.DisplayAlerts = False
wb.Close()  
_xl.Quit()

Output:

Results after 1.4 secs
               ID     YIELD
0   DD103619 Corp  1.012017
1   BJ226366 Corp  1.921489
2   DD103620 Corp  3.695580
3   ZS542668 Corp  2.945504
4   BJ226369 Corp  2.899166
5   ZS542664 Corp  1.109456
6   BJ226365 Corp  1.350594
7   ZS542666 Corp  2.732168
8   ZS542661 Corp  0.147570
9   ZS542663 Corp  0.621825
10  EJ772545 Corp  0.391708
11  EJ222340 Corp  2.846866
12  ZS542665 Corp  1.842695
13  EJ299219 Corp  0.224708
14  DD108917 Corp  3.733077
15  AM269440 Corp  0.189621
16  QJ633474 Corp  0.295588
17  BJ226367 Corp  2.727445
18  EC767655 Corp  2.241108
19  EI062653 Corp  2.728811
20  JK138051 Corp  1.077776
21  DD115180 Corp  1.604258
22  DD112334 Corp  1.527195
23  EK063561 Corp  0.570778
24  AM269866 Corp  1.329918
25  JK138053 Corp  2.915085
26  EH589075 Corp  3.110513

If I were to do this in production, I'd wrap the whole thing in a class to avoid stopping and starting Excel each time I wanted to perform a Query. Also, I haven't tested what happens in the user is already running Excel for something else!

DS_London
  • 3,644
  • 1
  • 7
  • 24
  • Nice answer, but it would only works if formula is less than 255 char. Then it throw an error, and I don't know if it is possible to concatenate the formula before adding it into the bql query. – TourEiffel Sep 12 '22 at 14:42
3

Run BQNT on your Bloomberg terminal to ensure the BQL environment is installed.

Follow the steps exactly as followed.

Open file explorer

  • Navigate to C:\blp\bqnt\environments\bqnt-3\Lib\site-packages and copy these folders:

  • bqapi

  • bqbreg

  • bql

  • bqlmetadata

  • bqrequest

  • bqutil

  • ciso8601

  1. Paste them to your python installation folder %userprofile%\Anaconda3\envs{mypythonenv}\lib\site-packages\

Then you can test this code in your code editor. I use Vscode.

import seaborn as sb
import pandas as pd
import matplotlib.pyplot as plt
import bql

bq = bql.Service()
query = """get(px_last)for('AAPL US EQUITY')with(dates=range(-1y,0d),fill='prev')"""
data = bql.combined_df(bq.execute(query)).reset_index()

fig = plt.figure(figsize=(12,8))
sb.lineplot(data=data, x='DATE',y='px_last')
plt.show()

Output

DJB
  • 71
  • 2
  • getting `AttributeError: partially initialized module 'bql' has no attribute 'Service' (most likely due to a circular import)` edit: i'm an idiot and named my script "bql.py" – David Yang Sep 07 '22 at 20:02
  • Working, Nicely ! – TourEiffel Sep 12 '22 at 13:48
  • This will work only in instances where your current env and bqnt env are in sync, which is pretty hard to do with constant updates. My preferred approach is setting up a server instead as shown by @jedi below – jedi May 23 '23 at 20:02
1

Copying BQNT specific modules does not work for someone that has many different environments and many different versions of python.

Another way around it is to create a small local server. A sample working version with JSON serializer is below. BQNT env does not have flask, pyarrow and pickle does not work across different python versions.

This solution may appear a bit involving at first, but I think it is way simpler longer term. You never have to worry about these packages and consistency across them when setting up a new environment.

While the below works and it is similar the way Excel addin works, ensure that you are permitted to run it by checking with your representative.

A sample BQL call:

bql("""get(px_last)for('AAPL US EQUITY')with(dates=range(-1w,0d),fill='prev')""")

returns:

               ID       DATE CURRENCY  px_last
0  AAPL US Equity 2023-05-15      USD  xxxx

...

# notice that serializer kept datatypes correctly
> aapl.dtypes

Out[5]: 
ID                  object
DATE        datetime64[ns]
CURRENCY            object
px_last            float64
dtype: object

# a client version
def bql(query):
    '''
    >>> bql("""get(px_last)for('AAPL US EQUITY')with(dates=range(-1w,0d),fill='prev')""")
    :param query:
    :return:
    '''
    response = requests.post('http://localhost:8000', data={'query': query})

    if response.status_code != 200:
        raise Exception('Query failed with status code {}'.format(response.status_code))

    data = pd.read_json(response.text, orient='split')
    return data

# a version of a server
# the server can be started after setting path to BQNT env
# to get their path: open bqnt and run 
# os.getenv('PATH')
# in a shell script set the path to point to BQNT env and start the server

class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):

    def do_POST(self):
        length = int(self.headers.get('content-length'))
        message = self.rfile.read(length).decode()
        query = parse_qs(message).get('query')[0]
        print(f'Received query: {query}')
        try:
            bq = bql.Service()
            data = bql.combined_df(bq.execute(query)).reset_index()
            result = data.to_json(orient="split")

            self.send_response(200)
            self.end_headers()
            self.wfile.write(result.encode())
        except Exception as e:
            self.send_response(400)
            self.end_headers()
            self.wfile.write(str(e).encode())

if __name__ == "__main__":
    host = 'localhost'
    port = 8000
    print(f'Server BQL starting on {host}:{port}')  # Print host and port
    httpd = HTTPServer((host, port), SimpleHTTPRequestHandler)
    httpd.serve_forever()
jedi
  • 525
  • 4
  • 11