4

I've wrote a script to plot daily data in Julia, but I found it run slow. Seems no ideal method to speed it up.

for example foo.jl

#!/bin/bash
#=
exec julia -O0 --compile=min "${BASH_SOURCE[0]}" "$@"
=#

using UnicodePlots, CSV, DataFrames, Chain, Dates
...

I wish if first time I run 'foo.jl a.csv b.csv", it may takes time to compile/load (10 seconds). And the 2nd time I run it, it can skip the compile process.(It should be around 3 seconds).

Is it possible now? I'm use version 1.8.3. Or, can I setup those packages which I used as default required compiled every time I launch julia to speed it up?

Daniel YC Lin
  • 15,050
  • 18
  • 63
  • 96
  • Does this answer your question? [Julia seems to be very slow](https://stackoverflow.com/questions/73599900/julia-seems-to-be-very-slow) – Andrew Savinykh Dec 09 '22 at 20:47

2 Answers2

5

You need to create a system image and give this a parameter each time when starting Julia:

using PackageCompiler
create_sysimage(["UnicodePlots, CSV, DataFrames, Chain"], sysimage_path="sys_foo.so", precompile_execution_file="script_with_your_typical_workflow.jl")

After this is done you will need to run your code as:

julia --sysimage sys_foo.so foo.jl

For more information see this thread: Julia seems to be very slow

Przemyslaw Szufel
  • 40,002
  • 3
  • 32
  • 62
  • I change to `["UnicodePlots", "CSV", "DataFrames", "Chain"]` The 'Dates' require to removed, maybe already in system image. And it generated. – Daniel YC Lin Dec 12 '22 at 02:50
  • BTW, this method will stick the packages' version according to the document. So, required re-compile when you updated the packages' version. – Daniel YC Lin Dec 12 '22 at 06:55
  • yes - `Dates` should be ommited - my fault :) and unfortunately you will need to recompile when anything changes – Przemyslaw Szufel Dec 12 '22 at 08:42
4

While I believe Przemyslaw's answer is the best advise at this point, I'd like to add two points:

  • Another option is to use the DaemonMode.jl package, which implements a client/server model to keep compiled code in memory.

  • Also, the "time to first X" latency problem has been a high priority for the core language devs for a long time, and it appears at the time of writing that we might start to see material improvements in Julia versions 1.9 and 1.10 towards native code caching. A variety of other avenues for addressing the issues are being pursued more or less actively in parallel, so if you are a reader from 2023 or later building sysimages might not be required any longer.

Nils Gudat
  • 13,222
  • 3
  • 39
  • 60
  • 1
    I saw a graph somewhere with 1.9 compiler improvements - they look Huge indeed :) The Deamon mode looks like something quite useful - have you been testing this in more kind of a production environment (eg. when this needs to run for a week or so between restarts)? – Przemyslaw Szufel Dec 09 '22 at 18:26
  • 1
    I've never really used it other than to see how it works, but I've seen a few positive testimonials on the Julia Community so it seemed worth mentioning. – Nils Gudat Dec 09 '22 at 21:30