2

I am working on a project where parts have been easier to write in Python and other parts easier to write in Julia. I just now finally put everything together, and the good news is that it works! But it's terribly slow due to Julia reloading packages every time it's called (and it's called often). I know I should have thought of this beforehand but here we are.

Is there a way I can "initialize" Julia within the Python script and load the modules I want to use once, instead of paying the price every single time I want to use it? Right now the Julia script is just called using subprocess.call(['juliaScript.jl']).

Starting to regret this approach but I really don't want to go through and re-write all the Julia code in Python (or vice versa). The Julia script is called in a double nested for loop in Python so the loading time costs are very significant and annoying.

I have heard of Precompile.jl but it seems very complicated and I think I would rather translate all the code between languages than learn that... I'm hoping someone has a wonderful hack job fix before I start re-writing everything. :)

If it helps, the packages I am loading in Julia are DataFrames, CSV, FFTW, and Statistics.

kirklong
  • 203
  • 1
  • 2
  • 9

1 Answers1

3

Instead of invoking Julia all the time, use a persistent server, something like Joseki.jl (the bin example), then on the Python side use requests to ask it to do things (see Post JSON using Python Requests).

Amadan
  • 191,408
  • 23
  • 240
  • 301
  • Another variant of this that might be a little easier to implement is to create *one* subprocess that runs Julia and communicate with this subprocess through pipes. – Daniel Junglas Mar 10 '20 at 06:21
  • @DanielJunglas In my experience, all modern languages these days have very nifty web libraries, with well-defined protocols; pipes are considerably more work, as more often than not you have to define your communication protocol yourself. (You can totally run a webserver as a subprocess as well though.) – Amadan Mar 10 '20 at 06:22
  • 2
    I guess this is a matter of taste. I have been using pipes all my life, so they feel very natural to me. They also feel they are a bit more lightweight. On the other hand, I agree that depending on what you want to do exactly, they may be a bad choice. Like I said, they are just an alternative. Not necessarily better, not necessarily worse. – Daniel Junglas Mar 10 '20 at 06:38
  • Thanks for these -- I thought about the pipe approach before but didn't know about Joseki.jl. Both seem like valid solutions but both I think were ultimately more work than just converting the code (which I did today). I was really hoping to just be super lazy -- thank you for your thoughtful replies anyways! – kirklong Mar 11 '20 at 20:23