erlang rebar escriptize & nifs

Question

I can use nif's if I write the escript myself, however when I use rebar escriptize the nif functions cannot be found. I think it is because *.so objects are not getting packed like beam files. Here is an simple example;

rebar.config:

{deps, [
   {'jiffy', "", {git, "https://github.com/davisp/jiffy.git", {branch, master}}}
]}.
{escript_incl_apps, [jiffy]}.
%% I tried this to see what happens if the so got in there but didn't help
{escript_incl_extra, [{"deps/jiffy/priv/jiffy.so", "/path/to/my/proj"}]}.

test.erl:

-module(test).

-export([main/1]).

main(_Args) ->
    jiffy:decode(<<"1">>),
    ok.

rebar get-deps compile escriptize
./test

and the result is

escript: exception error: undefined function jiffy:decode/1
  in function  test:main/1 (src/test.erl, line 7)
  in call from escript:run/2 (escript.erl, line 741)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

Is there a way to overcome this ?

From the error it looks like it is `jiffy.beam` which cannot be found, not `*.so`. Maybe `escriptize` takes into account only beams from `ebin` ignoring dependencies' `ebin`s? — Ed'ka, Mar 26 '13 at 04:35
@Ed'ka, nope if you add a dependency which is not a nif, it works fine. — cashmere, Mar 26 '13 at 04:44
But if you try to call `jiffy:decode/1` with `jiffy.so` removed from `priv` you should get `Failed to load NIF library` error, not `undefined function` — Ed'ka, Mar 26 '13 at 04:51

score 3 · Accepted Answer · edited Dec 06 '17 at 19:04

3

The problem is that the erlang:load_nif/1 function does not implicitly use any search path nor do anything smart in trying to find the .so file. It just tries to load the file literally as given by the file name argument. If it is not an absolute file name then it will try to load the file relative to the current working directory. It loads exactly what you tell it to load.

So if you call erlang:load_nif("jiffy.so") then it will try to load "jiffy.so" from your current working directory. A simple work around that I have used is to do something like this which uses the NIF_DIR environment variable:

load_nifs() ->
    case os:getenv("NIF_DIR") of
        false -> Path = ".";
        Path -> Path
    end,
    ok = erlang:load_nif(Path ++ "/gpio_nifs", 0).

This can easily be extended to loop down a search path to find the file. Note that NIF_DIR is not a special name, just one I have "invented".

edited Dec 06 '17 at 19:04

Community

1
1

answered Mar 28 '13 at 15:34

rvirding

20,848
2
37
56

I edited jiffy code to do this but didn't help. This is similar to what @lastcanal suggested. Instead of copying to .so to correct place, you look for it recursively. I think this problem is not relavent to loading .so. As Ed'ka pointed out it says undefined function jiffy:decode/1. – cashmere Apr 02 '13 at 15:53
Yes, but it was never stated where `jiffy:decode/1` was supposed to be defined. If it was in the .so file then a reason could have been the file was never loaded. Or not defined at all. – rvirding Apr 02 '13 at 16:11
it is defined [here](https://github.com/davisp/jiffy/blob/master/src/jiffy.erl#L10). And it makes a call to actual nif function, if it was defined properly defined and in the code path it should give an erlang:nif_error right? – cashmere Apr 02 '13 at 16:35
Then cannot load the `jiffy` module and the `undef` error refers to it not being able to find the `jiffy` module and not that the `decode/1` function is undefined. – rvirding Apr 03 '13 at 12:31
1

Yes, this is right. I hard coded the priv dir in jiffy code and it worked. If I bundle the so with the archive it doesn't work. It give me an {error,{load_failed,"Failed to load NIF library: 'dlopen(/Users/ali.yakamercan/src/nifes/test/jiffy/priv/jiffy.so, 2): no suitable image found. Did find:\n\t/Users/xxx/src/nifes/test/jiffy/priv/jiffy.so: stat() failed with errno=20'"}} because 'test' is the escript created by rebar and not a file. And since you cannot load nifs with non-matching names there if no way to load them in my escript. – cashmere Apr 03 '13 at 14:15
I guess the only way to is to modify the code of the dep to look at 'right' place(s) which is less then elegant. I wish there was a better way to this. – cashmere Apr 03 '13 at 14:16

score 1 · Answer 2 · answered Mar 27 '13 at 19:36

It does not seem possible to load a nif from an escript because erlang:load_nif does not look into archives. This is because most operating systems require a physical copy of the *.so that can be mapped to memory.

The best way to overcome this is by copying the *.so files into the output directory of the escript.

  {ok, _Bytes} = file:copy("deps/jiffy/priv/jiffy.so", "bin/jiffy.so"),

Take a look at the escript builder for edis. You will see this is how they load the eleveldb's nif for execution from an escript.

edis code is very similar to [rebar code](https://github.com/rebar/rebar/blob/master/src/rebar_escripter.erl#L69). I tried copying `jiffy.so` to the same directory with the escript but didn't help. — cashmere, Mar 27 '13 at 23:37

erlang rebar escriptize & nifs

2 Answers2

Linked