0

I have an inherited some code which was built in a different environment to mine.

I however know that I have all of the scripts required for this to run properly. I'm assuming there is an issue with my environment set up.

It's a very long, multi-module, proprietary project so I can't post any large sections of the project here. My issue is that when I run the code I am getting the error:

[user@localhost mktdata.out]$ /usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
ModuleNotFoundError: No module named 'merlin.tradelogic'
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
/usr/local/lib/python3.6/site-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.
  "module. Expect this to be very slow.", ImportWarning)
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'
Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'

Unless it's found that the hypervolume/pyhv.py:33: ImportWarning: is a related issue, the part of the error message I'm looking at first is:

Traceback (most recent call last):
  File "strats/merlin.py", line 13, in <module>
  File "strats/merlin/tradeopt.py", line 11, in <module>
ModuleNotFoundError: No module named 'merlin.tradelogic'

The start of merlin.py looks like:

#!/usr/bin/env python3
import argparse
import logging
import sys
from datetime import datetime
from merlin.cmdopt import (
        add_basic_opts,add_roll_opts,add_inst_opts,add_pair_opts,
        add_job_opts,add_list_of_opts
        )
from merlin.mktdata import mktdataMain
from merlin.instrument import instrumentMain
from merlin.pair import pairMain
from merlin.tradeopt import tradeoptMain
from merlin.wrtconfig import wrtconfigMain
from merlin.portfolio import portfolioMain
from merlin.overview import overviewMain
from merlin.var import *
from dao.utils.functools import apply_func_seq

the start of tradeopt which is called from merlin.py looks like:

import logging
import os.path
import lzma
import pandas as pd
import numpy as np
import array
import copy
from deap import base,creator,tools
from merlin.var import *
from merlin.stratconfig import MerlinConfig
from merlin.tradelogic import MerlinLogicPy
from merlin.utils import (
        merge_paths,jobs_todo,sort_dict_by_roll,joblist_to_dict
        )
from merlin.tradega import (
        trade_pars,SimSettings,is_valid_pars,evaluate_c,pnl_field_num,
        idx_total_pnl_pos_cost,cx_pars,mu_pars,idx_s1_trade
        )
from dao.iotools.datamodel import *
from dao.fin.stats import sharpe
from dao.fin.algorithms import eaMuPlusLambda
from dao.pair.pairid import PAIR_ID,split_pair2insts
from dao.pair.spread import __pairSpread__
from dao.utils.rw import (
        setup_dir,pickle_dump,pickle_load,find_folders,find_files
        )
from dao.utils.roll import ROLL_ID

logger=logging.getLogger(MERLIN_MAIN_LOGGER)

def tradeoptMain(args):

I think it's suggesting that that merlin.tradelogic is missing.

However when I look at the location /home/user/merlin/bin/strats/merlin I can see several different tradelogic modules:

They are:

tradelogic.cpp
tradelogic.cpython-35m-darwin.so  
tradelogic.pxd
tradelogic.pyx

The lines of code references in the error are as follows: line 13 in /home/user/merlin/bin/strats/merlin.py looks like:

from merlin.tradeopt import tradeoptMain

line 11 in /home/user/merlin/bin/strats/merlin/tradeopt.py looks like:

from merlin.tradelogic import MerlinLogicPy

The only tradelogic script with a MerlinLogicPy is in tradelogic.pyx which is below

cimport cython
from cython.view cimport array as cvarray
import numpy as np
cimport numpy as np

cdef class MerlinLogicPy:
    def __cinit__(self,double entry,double exit,double stop_entry,
            double stop_loss,double norm_pos,unsigned int time_loss,
            bint ignore_reporting_corp)
        self.thisptr=new MerlinLogic(entry,exit,stop_entry,stop_loss,
                norm_pos,time_loss,ignore_reporting_corp)

My $PATH looks like:

[user@localhost ~]$ echo $PATH
/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/user/scoleman/bin:/home/scoleman/bin:/home/user/.local/bin:/home/user/bin:/home/user/condor/bin:/home/user/scoleman/bin:/home/scoleman/bin:/home/user/condor/bin:/home/user/merlin/bin

which has the bin directory in merlin(/home/user/merlin/bin) included which holds all of the code.

I also have Cython version 0.29.5 installed. Do I need to recompile tradelogic.pyx? How can I do that?

I also see that I'm running Python3 (#!/usr/bin/env python3 is at the top of the code). It maybe nothing but the tradelogic.cpython-35m-darwin.so file looks to have been complied using version 3.5, could this be the/an issue? How can I compile a .so file?

halfer
  • 19,824
  • 17
  • 99
  • 186
Stacey
  • 4,825
  • 17
  • 58
  • 99
  • "I also see that I'm running Python 3"--the general term "Python 3" refers to any "Python 3.x" version, but you are correct that for extension modules (compiled modules) they are usually Python X.Y-version specific. You can check your actual Python 3 version by running `python3 --version` at the command line, or `import sys; print(sys.version)` in Python. I think they probably isn't your actual problem but I just thought I'd clarify that. – Iguananaut Jul 01 '19 at 23:13
  • Additionally, your problem might have less to do with your shell's `$PATH`, and more to do with your "python path" aka PYTHONPATH (see `python3 -c 'import sys; print(sys.path)`. Incidentally I just gave this talk to finance scientists on how Python module imports work, which might (partially) help point the way to working out your problem: https://github.com/embray/agentfin/blob/master/notebooks/gaunersdorfer_hommes.ipynb – Iguananaut Jul 01 '19 at 23:17

1 Answers1

0

It looks like your project root is /home/user/merlin/bin/strats/

Try to add it to your PYTHONPATH.

If project root is not part of the PYTHONPATH and not part of the Python libraries, Python won't know where to look for modules that are related to that project.

Here is a good explanation on how it works: How to use PYTHONPATH

Artur Spirin
  • 244
  • 3
  • 5