2

I have a Julia application that runs fine, but I want to make a compiled and more easily distributable version of it. As far as I understood, this is the job of PackageCompiler.

I'm able to build the executable, and it runs fine when just asking it for its --help, but when I want it to do some real work (processing data), after some initial info output, it crashes with the following error messages:

fatal: error thrown and no exception handler available.
MethodError(f=typeof(Base.convert)(), args=(Int32, nothing), world=0x00000000000065d9)
rec_backtrace at /home/bli/src/julia/src/stackwalk.c:94
record_backtrace at /home/bli/src/julia/src/task.c:219 [inlined]
jl_throw at /home/bli/src/julia/src/task.c:429
jl_method_error_bare at /home/bli/src/julia/src/gf.c:1606
jl_method_error at /home/bli/src/julia/src/gf.c:1624
jl_apply_generic at /home/bli/src/julia/src/gf.c:2161
julia_main at /home/bli/src/qaf_demux/Julia/QafDemux/bin/qaf_demux_to_compile.jl:12
julia_main at /home/bli/src/qaf_demux/Julia/QafDemux/deps/builddir/qaf_demux.so (unknown line)
main at ./deps/builddir/qaf_demux (unknown line)
__libc_start_main at /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
_start at ./deps/builddir/qaf_demux (unknown line)

What does this MethodError mean? What useful information can I get from the backtrace?

Here is the deps/build.jl script used to compile it:

import Pkg
Pkg.add("ArgParse")
Pkg.add("IterTools")
Pkg.add("FASTX")
Pkg.add("CodecZlib")
Pkg.add("REPL")
Pkg.add("PackageCompiler")
using PackageCompiler
build_executable(joinpath(@__DIR__, "../bin/qaf_demux_to_compile.jl"), "qaf_demux", snoopfile=joinpath(@__DIR__, "../bin/snoop.jl"))

snoop.jl calls the main function of my QafDemux package with some test command-line (which corresponds to the one used to test the compiled executable):

#!/usr/bin/env julia

push!(LOAD_PATH, abspath(joinpath(@__DIR__, "../src/")))
import QafDemux
const qd = QafDemux

test_args = "-i ../../test_data/TCR_ampli_R1_50k.fastq.gz -o test_run -b GCAGAGATAAGC GCAGAGATGCAC GCAGAGACTCAG GCAGAGAGGAAT GCAGAGACGAGG GCAGAGAAGGAG GCAGAGATGTTG GCAGAGACAACT GCAGAGAGGCTA GCAGAGAGAATG GCAGAGACCAAC GCAGAGAGAGAC -s 3 -m 2 -p 0.1"
qd.main(split(test_args))

qaf_demux_to_compile.jl, as per the PackageCompiler instructions, is as follows:

#!/usr/bin/env julia

push!(LOAD_PATH, abspath(joinpath(@__DIR__, "../src/")))
import QafDemux
const qd = QafDemux

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    qd.main()
end

The non-compiled version, that works fine, is almost identical:

#!/usr/bin/env julia

push!(LOAD_PATH, abspath(joinpath(@__DIR__, "../src/")))
import QafDemux
const qd = QafDemux

qd.main()

(Cross-posted at a slightly earlier stage of my investigation at https://discourse.julialang.org/t/understanding-runtime-errors-with-packagecompiler-built-executables/29120/2)

Edit: Working solution

Based on the suggestion made by @Anshul Singvi, I modified the main function of my QafDemux module so that it now returns an integer, and use is so that this is also the return value of the julia_main function in the code passed to PackageCompiler.build_executable.

bin/qaf_demux_to_compile.jl:

#!/usr/bin/env julia

push!(LOAD_PATH, abspath(joinpath(@__DIR__, "../src/")))
import QafDemux
const qd = QafDemux

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    return qd.main()
end

(I also had to change the main function so that it accepts an optional command-line, because the snoopfile was actually wrong otherwise).

sophros
  • 14,672
  • 11
  • 46
  • 75
bli
  • 7,549
  • 7
  • 48
  • 94
  • You may also want to check out https://github.com/NHDaly/ApplicationBuilder.jl depending on your use case. – logankilpatrick Sep 25 '19 at 20:51
  • 2
    It could be that you need to return an integer? Try `return 0` at the end of your `julia_main` function. – Anshul Singhvi Sep 25 '19 at 22:23
  • @logankilpatrick I tried to use `ApplicationBulider` too, and the resulting executable had the same problem. That said, I'm not sure to understand the differences between `PackageCompiler` and `ApplicationBuilder`, except the place where the built files are put. It seems to me that in both cases, there still is no single-file standalone executable: both provide shared objects that need be shipped along with the executable. – bli Sep 26 '19 at 08:44
  • @AnshulSinghvi That was (one of) the problems. Thanks. The other was that I hadn't noticed that the compilation failed due to the way I tried to simulate a call with command-line arguments in the snoopfile. The error I reported must have actually been generated using a previously compiled version, without a snoopfile. – bli Sep 26 '19 at 09:47
  • @bli, did you get it to work? – Anshul Singhvi Sep 26 '19 at 13:17
  • 1
    @AnshulSinghvi Yes, thanks. Don't hesitate to post your comment as an answer, maybe with a word about what made you suspect this was the problem. – bli Sep 26 '19 at 16:55

1 Answers1

1

Looking at the stacktrace, you can see that Julia is trying to convert nothing into Int32, in the line: 

MethodError(f=typeof(Base.convert)(), args=(Int32, nothing), world=0x00000000000065d9)

This is because your julia_main function doesn't explicitly return anything, and qd.main also seems not to return anything. In the absence of an explicit return value, Julia defaults to returning nothing.

However, in your method's type contract, it's specified that julia_main must return a Cint, which on your system seems to be an Int32. Therefore, Julia is trying to convert what julia_main (implicitly) returns to an Int32 - which is impossible!

To fix this, you just need to make sure that you return an integer. A nicer thing to do would be:

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    try
        qd.main()
    catch e
        print(stderr, e) # print the error to standard error
        return 1 # in command line tools, a return value of 1 means error
    finally
        return 0 # similarly, a return value of 0 means that the program ran properly
    end
end
Anshul Singhvi
  • 1,692
  • 8
  • 20