3

Unfortunately I am led on the forums to believe (but without 100% certainty) that the Bloomberg Desktop API does not allow more than one IntradayBarRequest or IntradayTickRequest at a time, which is different from HistoricalDataRequest or Subscriptions, where multiple simultaneous requests are allowed.

This question is thus probably moot, unless somebody tells me that the above is not true

If true, then the only way to handle the question below, is to send each new request only after the previous one has been handled.


I am using the Python Bloomberg Desktop API to access subscription (live updating) and historical daily data for financial securities. In both of those cases I can send multiple simultaneous requests, and when the responses come (not necessarily in the order the requests were sent), I can find out which security the response is associated with, using msg.getElement("securityData").getElementAsString("security") in the case of historical data, or in the case of subscription data, by querying the correlationId that I had set beforehand (at subscription request time) using msg.correlationIds()[0].value().

However I don't know how to do this for IntradayBarResponse requests (and WAPI docs are not helpful). These do not seem to have a setable correlationId, nor do they have the above "securityData" field. If I send multiple intradayBarRequests, how can I find out for which security the responses are?

Here is my code (adapted from the API Python examples).

import blpapi  # interface to bloomberg
import time    # will need this for time parsing
from optparse import OptionParser
import pdb     # debugger, when necessary
import csv     # for csv reading
import string  # for string parsing
from pymongo import MongoClient
import inspect
from datetime import datetime
from bson.son import SON


def parseCmdLine():
    parser = OptionParser(description="Retrieve realtime data.")
    parser.add_option("-a",
                      "--ip",
                      dest="host",
                      help="server name or IP (default: %default)",
                      metavar="ipAddress",
                      default="localhost")
    parser.add_option("-p",
                      dest="port",
                      type="int",
                      help="server port (default: %default)",
                      metavar="tcpPort",
                      default=8194)
    parser.add_option("--me",
                      dest="maxEvents",
                      type="int",
                      help="stop after this many events (default: %default)",
                      metavar="maxEvents",
                      default=100000000000)
    parser.add_option("--mongohost",
                      dest="mongohost",
                      default="192.168.1.30")
    parser.add_option("--mongoport",
                      dest="mongoport",
                      type="int",
                      default=27017)

    (options, args) = parser.parse_args()

    return options


def main():
    options = parseCmdLine()

    # connect to MongoDB MONGO MONGO MONGO MONGO ----------------
    print "Connecting to MongoDB"
    print options.mongohost
    print options.mongoport
    client = MongoClient(options.mongohost, options.mongoport) # connect to MongoDB
    db = client.bb # connect to the DB database
    bbsecs = db.bbsecs
    bbticks = db.bbticks
    # now get the securities list


    # Fill SessionOptions
    sessionOptions = blpapi.SessionOptions()
    sessionOptions.setServerHost(options.host)
    sessionOptions.setServerPort(options.port)

    print "connecting to Bloomberg"
    print "Connecting to %s:%d" % (options.host, options.port)

    # Create a Session
    session = blpapi.Session(sessionOptions)

    # Start a Session
    if not session.start():
        print "Failed to start session."
        return

    # open the market data subscription service
    if not session.openService("//blp/mktbar"):
        print "Failed to open //blp/mktbar"
        return
    if not session.openService("//blp/refdata"):
        print "Failed to open //blp/refdata"
        return



    # now startup the subscription list
    # Now open the secs.dat file and read it, append each to subscription list
    maxtimes = bbticks.aggregate([{'$group': {'_id':'$ticker', 'maxtime':{'$max': '$time'}}}]) # get the last updates by ticker
    refDataService = session.getService("//blp/refdata") # start the ref
    for i in maxtimes["result"]:
        ticker = i["_id"]
        tstamp = i["maxtime"]
        request = refDataService.createRequest("IntradayBarRequest")
        request.set("security", ticker)
        request.set("eventType", "TRADE")
        request.set("interval", 1)
        request.set("startDateTime", tstamp)
        request.set("endDateTime", datetime.now())
        print "Sending Request:", ticker
        session.sendRequest(request)

    subscriptions = blpapi.SubscriptionList()
    secdic = dict() # a new dictionary
    for post in bbsecs.find():
        print(post["ticker"])
        # subscribe tick
        #subscriptions.add(str(post["ticker"]), "LAST_PRICE", [], blpapi.CorrelationId("TICK:" + str(post["ticker"])))
        #subscribe 1 minute bars
        subscriptions.add("//blp/mktbar/ticker/"+str(post["ticker"]), 
                          "LAST_PRICE", 
                          "interval=1.0",
                          blpapi.CorrelationId(str(post["ticker"])))
        # setup the dictionary
        secdic[post["bbsecnum"]] = post["ticker"]
    if not session.openService("//blp/refdata"):
        print "Failed to open //blp/refdata"
        return
    # now subscribe
    session.subscribe(subscriptions)


    # HISTORICALHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
    # Obtain previously opened service
    #refDataService = session.getService("//blp/refdata")
    # Create and fill the request for the historical data
    #request = refDataService.createRequest("HistoricalDataRequest")
    #for post in bbsecs.find():  
    #    request.getElement("securities").appendValue(str(post["ticker"]))
    #request.getElement("fields").appendValue("LAST_PRICE")
    #request.set("periodicityAdjustment", "ACTUAL")
    #request.set("periodicitySelection", "DAILY")
    #request.set("startDate", "20100101")
    #request.set("endDate", "20121231")
    #request.set("maxDataPoints", 2000)
    #print "Sending Request:", request
    # Send the request
    #session.sendRequest(request)
    #hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh


    try:
        # Process received events
        eventCount = 0
        while(True):
            # We provide timeout to give the chance to Ctrl+C handling:
            event = session.nextEvent(500)
            for msg in event:
                if event.eventType() == blpapi.Event.SUBSCRIPTION_STATUS:
                    #print "%s - %s" % (msg.correlationIds()[0].value(), msg)
                    print "subscription status"
                elif event.eventType() == blpapi.Event.SUBSCRIPTION_DATA:
                    key = msg.correlationIds()[0].value()
                    if msg.messageType() == "MarketBarStart":
                        open = msg.getElementAsFloat("OPEN")
                        high = msg.getElementAsFloat("HIGH")
                        low = msg.getElementAsFloat("LOW")
                        close = msg.getElementAsFloat("CLOSE")
                        btstamp = msg.getElementAsDatetime("TIME")
                        tstamp = datetime.now()
                        print "bar", key, close, tstamp
                        bbticks.insert({"type": "BAR", "ticker": key, "value": close, \
                                "open": open, "high": high, "low": low, "close": close, \
                                "time": tstamp})
                    elif msg.messageType() == "MarketBarUpdate":
                        close = msg.getElementAsFloat("CLOSE")
                        #print "tick", close,
                        #bbticks.insert({"type": "TICK", "ticker": key, "value": close, "time": tstamp})

                    #if etype == "TRADE":
                    #    if msg.hasElement("LAST_TRADE"):
                    #        key = msg.correlationIds()[0].value(),
                    #        keytype = key[:(key.index(":"))]
                    #        key = key[(key.index(":") + 1):]
                    #        value = msg.getElementAsString("LAST_TRADE")
                    #        timestamp = msg.getElementAsDatetime("TRADE_UPDATE_STAMP_RT")
                    #        print key, value, 
                    #        bbticks.insert({"ticker": key, "value": value, "timestamp": timestamp})

                else:
                    if msg.messageType() == "HistoricalDataResponse":
                        securityData = msg.getElement("securityData")
                        security = securityData.getElementAsString("security")
                        fieldDataArray = securityData.getElement("fieldData")
                        for j in range(0, fieldDataArray.numValues()):
                            fieldData = fieldDataArray.getValueAsElement(j)
                            field = fieldData.getElement(0)
                            tstamp = field.getValueAsDatetime()
                            tstamp = datetime(tstamp.year, tstamp.month, tstamp.day)
                            field = fieldData.getElement(1)
                            close = field.getValueAsFloat()
                            #print "history", security, close, 
                            #bbticks.insert({"type": "DAILY", "ticker": security, "value": close, "close": close, \
                            #        "time": tstamp})
                    elif msg.messageType() == "IntradayBarResponse":
                        print "IntradayBarResponse"
                        data = msg.getElement("barData").getElement("barTickData")
                        numvals = data.numValues()
                        print numvals
                        if numvals > 0:
                            print data.getValueAsElement(1).getElement(1).getValueAsFloat()



            if event.eventType() == blpapi.Event.SUBSCRIPTION_DATA:
                eventCount += 1
                if eventCount >= options.maxEvents:
                    break
    finally:
        # Stop the session
        session.stop()

if __name__ == "__main__":
    try:
        main()
    except KeyboardInterrupt:
        print "Ctrl+C pressed. Stopping..."

I have looked at the eidData and that is empty, even if I ask for it to be returned. I am looking at currencies, rather than equities, so there is no exchange entitlement required.

> (Pdb) print eid._Element__dataHolder IntradayBarResponse = {
>     barData = {
>         eidData[] = {
>         }
>         barTickData[] = {
>             barTickData = {
>                 time = 2013-08-02T18:36:00.000
>                 open = 4.233100
>                 high = 4.233600
>                 low = 4.233100
>                 close = 4.233400
>                 volume = 0
>                 numEvents = 119
>                 value = 0.000000
>             }
>             barTickData = {
>                 time = 2013-08-02T18:37:00.000
>                 open = 4.233400
>                 high = 4.233700
>                 low = 4.233100
>                 close = 4.233500
>                 volume = 0
>                 numEvents = 96
>                 value = 0.000000
>             }
>             barTickData = {
>                 time = 2013-08-02T18:38:00.000
>                 open = 4.233500
>                 high = 4.233600
>                 low = 4.233300
>                 close = 4.233500
>                 volume = 0
>                 numEvents = 135
>                 value = 0.000000
>             }
>             barTickData = {
>                 time = 2013-08-02T18:39:00.000

I am still looking for a way to associate requests with responses, without having to do the inefficient request....wait for response.... request etc. I am not sure if Bloomberg provides this functionality, however. This limitation appears to exist for historical tick data as well.

Thomas Browne
  • 23,824
  • 32
  • 78
  • 121
  • Given that this service costs thousands of dollars per month, I suggest you ask the provider. – John Zwinck Aug 03 '13 at 09:42
  • 3
    Believe it or not, that has already occurred to me. However if you were to investigate further, you would know that the SERVER api costs many more thousands per month, and that is the only API for which "the provider" provides official API support. With the desktop API, which indeed costs thousands but much less than server, there is no suppport (dubious Bloomberg business strategy to get people to pay even more). Hence my question. Thank you. – Thomas Browne Aug 03 '13 at 09:45
  • 1
    I'm not using the Python API, I'm using the .Net one, but I know that there you can have several IntradayBarRequests "in the air" at once. I use the CorrelID and it works fine. – ytoledano Aug 03 '13 at 21:04

3 Answers3

2

Thomas, you should always use correlationId of a response to find which request it is associated with. IIRC, reference data and historical data requests support multiple securities, but market data and intraday bars may have only one security per request.

That is why there is additional "securityData" field in refdata and historical responses. For market data and intraday bars correlationId is enough to identify the request and hence the security.

Hope it helps.

  • historical intraday bars ("IntradayBarRequest" in the API)do not support a settable correlationID, as far as I can see. That's the problem. The responses *do* have a correlationID, yes, but it is not user settable as far as I can see, and thus is useless. One is forced to request, process until finished, then request again. Unlike historical daily data (securityData) or live subscriptions (settable correlationId) where one can do bulk processing thanks to ability to link responses to requests. – Thomas Browne Aug 03 '13 at 15:25
2

I don't know the python API but I do use the Java API.

I am still looking for a way to associate requests with responses, without having to do the inefficient request....wait for response.... request etc.

As you discovered, you can't send a query for multiple securities for IntradayBarRequests and the feed does not contain security ids (e.g. tickers) to easily map back to your queries.

The easiest solution is to use a correlation ID. When you submit a request to a session, you can provide your own correlation ID. The example below is in Java but I would think that the python API is similar:

Session session = ...; //Bloomberg Session
CorrelationId cId = new CorrelationId(); //Unique ID
session.sendRequest(bbRequest, cId); //provide your own ID

Then, if you use an asynchronous session, you receive asynchronous messages which have a link to the original CorrelationId:

public void processEvent(Event event, Session session) {
    for (Message msg : event) {
        CorrelationID cId = msg.correlationID();
        //here you can link the result back to your original query
    }
}
assylias
  • 321,522
  • 82
  • 660
  • 783
  • Thanks but I tried that, and I got a duplicate correlation ID error in the Python API. I am going to call Bloomberg support tomorrow to try to fix this. – Thomas Browne Aug 04 '13 at 10:16
  • Each correlation should be unique. You can also provide your own ID. In Java: `new CorrelationID(1);` for example for an Id of 1. – assylias Aug 04 '13 at 11:13
0

I don't believe you can subscript to tick or bar data. Here's one reason why... For pricing data, you get a response with each change to the value. Isn't tick data merely a history of those prices? For bar data, isn't that a history of prices with some detail removed by grouping by a period (ie. 10 minutes, 1 hour). So if you could subscribe to either, when should you receive each new response from Bloomberg? Essentially, you could create your own listener that does the grouping, and then sends out a response from the listener with a frequency of your own choosing.

BTW: I use CLR to talk to the .net blp object from python. My solution predates any support Bloomberg for Python, essentially doing what 'can't be done'.

Alfista
  • 1
  • 1
  • Section 4.8 Core Developer Guide (2016): "The Market Bar Service (“//blp/mktbar”) provides streaming (real-time and delayed) intraday bars." You receive a response at every tick... – sdittmar Feb 29 '20 at 06:48
  • @sdittmar: do you know how to get the delayed stream that you mentioned? – HAL9000 Nov 09 '22 at 15:01
  • https://docslib.org/doc/6560522/bloomberg-api-version-3-x-developer-s-guide – sdittmar Jan 07 '23 at 21:35