4

I intend to manipulate binaries using NIFs for an app which I'm planning to code in Erlang. The gist links to the cpp file and erl file for the NIF are given below.

[Erl Gist Link] https://gist.github.com/abhijitiitr/3a5bc97184d6dd32f97b

[C++ Gist Link] https://gist.github.com/abhijitiitr/24d2b780f2cdacebfb07

Basically I'm trying to do a simple test. Share binaries across NIF calls and successfully manipulate them with successive NIF calls.

If you test the code in erlang REPL by

c(binary_test).
Ref=binary_test:open(<<1>>).
binary_test:increment(Ref,<<3>>).

The binaries stored changes in between the NIF calls. The REPL output for the third command is

1
 3
  60
    60
      <<"?">>

I passed <<1>> during the initialize phase. Why did it change to <<60>>? I'm unable to figure out whats happening here. Can somebody point out the error?

C++ compile instructions

clang++ -std=c++11 -stdlib=libc++ -undefined dynamic_lookup -O3 -dynamiclib binary_test.cpp -o binary_test.so -I /usr/local/Cellar/erlang/17.0/lib/erlang/erts-6.0/include/

on my Mac.

Also I wanted to ask about concurrent processes manipulating a shared resource in NIF. Is that possible or there is a rule that NIFs have to be accessed in a single Erlang process.

Hynek -Pichi- Vychodil
  • 26,174
  • 5
  • 52
  • 73
abips
  • 209
  • 3
  • 9
  • What is "a binary"? Do you just mean "an integer"? – unwind Oct 22 '14 at 11:09
  • Binary is a native Erlang type which can contain any type.e.g. <<1>>,<<"lorem ipsum">>,<<[1,2,3]>>. Essentially you can think of it as a ByteString kind of thing. – abips Oct 22 '14 at 11:40
  • 1
    NIFs and resources may be used by many processes simultaneously, but the onus on thread safety is on you. – goertzenator Oct 22 '14 at 13:31

2 Answers2

5

You're running into problems because you're illegally accessing memory. In your BinaryStore constructor you're attempting to save a binary from the argument list passed to binary_test:open/1, but this doesn't work because those arguments are freed once the NIF call finishes with them. You need to save a copy of the argument to use it later. To do this, first add a new member to your BinaryStore class:

    ErlNifEnv* term_env;

Next, modify your constructor to allocate term_env and then use it to copy the incoming term:

    BinaryStore(ERL_NIF_TERM binary)
    {
        term_env = enif_alloc_env();
        binary_term = enif_make_copy(term_env, binary);
    }

This allocates binary_term in the term_env environment and then copies the incoming term into it. You'll also need a destructor to free term_env:

    ~BinaryStore()
    {
        enif_free_env(term_env);
    }

And finally, you need to pass term_env instead of env when inspecting binary_term in the increment_binary function:

    nifpp::get_throws(term_env, binary_term, ibin);

With these modifications in place, I get the following results from running the code:

1> Ref=binary_test:open(<<1>>).
Reading symbols for shared libraries . done
<<>>
2> binary_test:increment(Ref,<<3>>).
1
 3
  1
   1
    <<4>>

(By the way, you should use "\r\n" line endings rather than just "\n" when printing from inside the Erlang emulator so that newlines always return to the leftmost column.)

You still have one remaining problem, which is that you leak the memory allocated for new_bin2.

My advice for learning the details of NIFs is to avoid using packages like nifpp at first, so you can learn the NIF API and all the details regarding memory ownership, resource allocation and deallocation, and argument conversions. Once you understand them, using packages like nifpp becomes much easier and more fruitful.

Steve Vinoski
  • 19,847
  • 3
  • 31
  • 46
  • Thanks a lot for the lucid explanation. The reason for using `nifpp` was basically to write concise code and `nifpp` I think deallocates the memory allocated to `new_bin2` once there aren't any references left pointing to the resource. – abips Oct 22 '14 at 16:50
  • In line 446 [niffpp.h](https://github.com/goertzenator/nifpp/blob/master/nifpp.h), I think the resource is destroyed after an allocation as it should have if invoked through `C` Api. Please do correct me if I'm wrong. – abips Oct 22 '14 at 17:23
  • 1
    But for `new_bin2`, you're simply allocating memory for it via `enif_alloc_binary`, then copying memory into it with `memcpy`. There are no `nifpp` calls involving it. You therefore need to call `enif_release_binary` on it before exiting the `increment_binary` function. Alternatively, get rid of `new_bin2` altogether and just use `ibin` in its place, since there's no need to copy `ibin` just to use its value. – Steve Vinoski Oct 22 '14 at 18:33
  • Thanks, I get your point. Regarding `ibin`, I think I can only access the value `ibin` and not modify it. – abips Oct 22 '14 at 18:38
  • 1
    Correct, you can't change `ibin`. But to use it to replace `new_bin2` is perfectly fine since 1) `new_bin2` is just a copy of `ibin` and 2) you use `new_bin2` only to read from, not to write to. – Steve Vinoski Oct 22 '14 at 19:16
2

ERL_NIF_TERMs must be associated with an ErlNifEnv, and the env that is passed to your nif functions is only valid for the duration of that function call. You are breaking this rule when you store a term into a BinaryStore object and then later use it from another nif call. Your options:

  1. Create a new ErlNifEnv for your binary store and copy terms from the nif call into this new env.

  2. Use C++ data structures (such as std::vector<unsigned char>) to store your binary data. I think this will be simpler for your situation.

goertzenator
  • 1,960
  • 18
  • 28