2

I have coded a Python charting application that takes as input an http feed (e.g. EurUsd ticks) from my broker and then uses Matplotlib to draw price candles and indicators. The live tick data is used to update several numpy arrays which serve as data containers for price candles and indicators at timeframes of 5min, 15min, 60min, 240min and 1day. Good.

The reason why I am using numpy arrays is because this is the expected input format for TA-Lib which I use to calculate ongoing indicator and oscillator values (which are, in turn, passed to Matplotlib for chart rendering). These include RSI, MACD, Stochastics, etc as well as some Bollinger Bands derived from the RSI and MACD arrays. Including signal lines and so forth, each timeframe has 21 arrays. I track 5 timeframes (5min, 15min, 60min, etc) with the result that the application requires 105 numpy arrays to do its job. The price candle array for each timefrime is a 2D array comprised of open, high, low, close, volume and date.

The longest set of arrays are those in the 5min timeframe where I initialize 20 indicator arrays of length 288 (one day's worth of 5min candles) + 288 (an additional day's worth of space leading up to reinitialization) = 576. The candle array has shape 6 to hold OHLCV and Date, so it has size 576 x 6 = 3456.

At the 240min and 1day timeframes, the array sets are equal in number (21) but shorter since they pack less data. Updates for each array's current price candle happens per tick, ranging anything from 10 times per second to once in a minute. Each tick update also cascades to all of the indicators so that's an update to 105 arrays per tick.

My question is whether there are resource considerations I should be aware of with this number of arrays and the frequency of updates to each of the 105 arrays. I get the impression (from this question How much memory in numpy array? Is RAM a limiting factor?) that the memory consumption of these arrays is not an issue since according to the nbytes method the 20 x 576-length arrays consume 4608 bytes each = 90kb in total.

Is this correct and is my design logic sensible?

Community
  • 1
  • 1
venzen
  • 1,119
  • 10
  • 13
  • If all the data is 64 bits, the total size in bytes of all your 105 arrays should be `sum([24*60/j*2*(20+6)*8 for j in (5, 15, 60, 240, 24*60)])` which comes out to `172640`, well under 0.2 Mb of memory. So no, you shouldn't have memory issues, and unless you are doing very weird stuff with the data, you shouldn't have trouble processing it many times per second. – Jaime Jul 28 '13 at 14:39
  • 1
    also: http://c2.com/cgi/wiki?PrematureOptimization Build it and only worry about optimization if it really is a problem. – tacaswell Jul 28 '13 at 15:33
  • @Jaime, thanks for writing my question and its answer in Python. I am giving too much thought to PrematureOptimization as tcaswell correctly diagnosed. With your comments I can proceed boldly and just implement the blessed code. – venzen Jul 28 '13 at 17:11
  • @tcaswell thanks for the link - you hit the nail on the head. – venzen Jul 28 '13 at 17:14
  • @Jaime, you could post your comment as an answer since it seems to respond the OP's question... – Saullo G. P. Castro Aug 02 '13 at 18:28

0 Answers0