2

Im writing an Erlang C NIF that will only be used by one Erlang process. I want to create a struct that will hold an array of pointers. I need this to exist in between the process' calls to the NIF.

What I need insight to is the proper way to do this approach from the Erlang NIF side of things. Im thinking of writing a struct outside of all the functions so its accessible to all. When I create it in one call to the NIF, and then come back and use it with another call to the NIF, it seems to work just fine.

Im worried that this could be because the process is staying local to the scheduling thread and therefore does not have to move the struct and underlying array in memory.

Should I be using erlang:memalloc from within a function and avoiding globals all together or, staying as is with global structs?

Possibly return a pointer to a single array containing all my data?

Hynek -Pichi- Vychodil
  • 26,174
  • 5
  • 52
  • 73
BAR
  • 15,909
  • 27
  • 97
  • 185

1 Answers1

4

You could certainly return a pointer to a single array containing your data; to do that, look at ErlNifResourceType. You would pass this back to the calling erlang process, and it in turn would pass it back to you on subsequent NIF calls. This would ensure that only one thread was operating on your data at a time (assuming only one process had a copy of the resource; it's not something you want to share, especially if it contains pointers).

You could also encode it as an erlang list, but that would probably be very inefficient.

That being said, you can use shared memory from a NIF. For example, here's an ets-like database implemented as a NIF using shared data.

You just have to keep in mind that you're accessing shared resources. The NIF API provides thread creation, thread specific data, mutexes, conditions, and read/write locks. You can even send a message to an erlang process from a NIF-created thread (in the event of a long-running NIF call, this is actually how you'd want to implement it to prevent scheduling problems).

Given your requirements, you're probably better off using the ErlNifResource type rather than messing with multithreading and shared resource controls. Technically if you're only using one erlang process you could leave it as a global variable (read: shared resource) without any harmful side effects. That being said, things change, and you don't want to be the cause of someone's headache down the road when they try to use your code from multiple processes. Whichever method you wind up using, make sure it's thread safe.

Soup d'Campbells
  • 2,333
  • 15
  • 14
  • Could you elaborate as to where the shared resources are located in the erlang memory during runtime? – BAR Jan 23 '13 at 11:37
  • NIFs are dynamically linked libraries, so they use the calling process' memory (OS may do some management to prevent multiple applications from loading the same library). Give that, it's just like C. All threads share a common heap, and this is where any dynamically allocated objects go. As for global variables, it's system-specific: some systems will allocate them on the heap, some will allocate them on the stack, and some will allocate them somewhere else altogether. There's a good breakdown here: http://stackoverflow.com/questions/1169858/global-memory-management-in-c-in-stack-or-heap – Soup d'Campbells Jan 23 '13 at 15:34
  • 1
    Short answer: you don't have to worry about calling into your NIF from different scheduling threads; if the variable is a static member variable or a global variable it will be accessible from all scheduler threads. – Soup d'Campbells Jan 23 '13 at 15:35
  • So is the purpose of enif_alloc_resource simply to be able to return pointer's among others to Erlang allowing for GC? Making enif_alloc useful only when returning to Erlang is not necessary, yet an ongoing share of memory space is? – BAR Jan 24 '13 at 11:43
  • The purpose of enif_alloc_resource is for you to be able to declare a native type that you can pass back to erlang, and define its constructor/destructor. If your resource contains pointers you'll still have to destroy the objects they point to explicitly *from the destructor*; the only real benefit of ErlNifResources is that you can pass them back to processes in the VM. enif_alloc is different from enif_alloc_resource; enif_alloc is basically a drop-in replacement for malloc. Useful for NIFs that you use shared memory that doesn't need to be passed back to the VM. – Soup d'Campbells Jan 24 '13 at 15:45