0

I have a for loop which deals with more than 9 million combinations (for this, I've used itertools library), must perform the code below faster, it's taking too long to loop over all combinations. Appreciate any suggestions

wb = xw.books('FX VEGA BT.xlsm')
sht = wb.sheets['Sheet1']
#retrieving data from excel
df = pd.DataFrame(sht.range('PY_PNL').value, columns=['10','20','25','40','50','60','70','75','80','90'])

#df has shape of 3115 rows × 10 columns

def sharpe(x):
    s = round(np.average(x)/np.std(x)*np.sqrt(252),2)
    return s

shrps = []
outlist = []

mult = (-1,-2.5,0,1,2.5)

perm = itertools.product(mult,repeat = 10)
for p in perm:
    c = df*p
    c = c.sum(axis='columns')
    outlist.append(p)
    shrps.append(sharpe(c))
Alberto Coppola
  • 1
  • 1
  • 2
    `perform...better`? - Better how? What is `df`? - Please read [mre], amd [ask] and the other links found on that page. – wwii Sep 03 '20 at 14:55
  • Thanks for your advice, wwii. Made some changes, think it's clearer – Alberto Coppola Sep 03 '20 at 15:00
  • 1
    I don't know where you got the idea that `list.append` is the thing you need to optimize, but it's not. It takes a negligible fraction of the runtime. – user2357112 Sep 03 '20 at 15:03
  • Again, without a *minimal* example of the data (`df`) how can we help? Your mre's should not include references to data we cannot access or on some offsite resource. We should be able to copy and paste from your question to our editor. - [How to make good reproducible pandas examples](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples) – wwii Sep 03 '20 at 15:05
  • for just raw append speed a deque is faster than a list (see [here](https://stackoverflow.com/questions/23487307/python-deque-vs-list-performance-comparison)) – ccluff Sep 03 '20 at 15:15
  • state the use case properly, seems like dynamic programming problem – Suryaveer Singh Sep 03 '20 at 18:52

1 Answers1

0

You can use a list comprehension and it'll be a bit more faster:

shrps = [sharpe((df*p).sum(axis='columns')) for p in perms]

If you really need a copy of perm named as outlist, you can use deepcopy from copy package:

import copy

outlist = copy.deepcopy(perm)

To speed up the process more, you can change something (I don't know how it looks like) in sharpe() function.

maciejwww
  • 1,067
  • 1
  • 13
  • 26
  • Actually I need to find to which combination every sharpe is related, I just didn't find a better way to do that than appending the combination to the same dataframe everytime it's looped – Alberto Coppola Sep 03 '20 at 15:21
  • So, I can only propose the list comprehension to speed up it a bit. Good luck! – maciejwww Sep 03 '20 at 15:28