4

So I have been using ETS - works great. However, I use it as a cache of route data - which I load when the module loads, and save when a change is made (it is read far more than written).

I was thinking that DETS would make things much cleaner - I don't have to worry about managing the persistence of the table. Would this be a good use of DETS? (size isn't an issue, mostly concerned about major increase in read performance - all the data can easily fit in memory).

miku
  • 181,842
  • 47
  • 306
  • 310
Michael Neale
  • 19,248
  • 19
  • 77
  • 109

3 Answers3

4

DETS is much much slower than ETS in most cases but I guess that if your data size is small then most if it will be cached by the disk cache and will be faster to retrieve the second time you read it, so the best thing is the try it with your type of use

But have you considered using Mnesia instead of ETS directly? With the Mnesia table type disc_copies you will get the speed of ETS and still persistency

ministaten
  • 56
  • 1
  • Thank you, I will consider it. I am trying to avoid dependencies until I really need them, I don't really need much of mnesia, but I probably should trust it more than I trust the alternative I would produce. – Michael Neale Jan 26 '11 at 23:25
2

Straight from the Dets man page:

"Since all operations performed by Dets are disk operations, it is important to realize that a single look-up operation involves a series of disk seek and read operations. For this reason, the Dets functions are much slower than the corresponding Ets functions, although Dets exports a similar interface."

That is, for a read-mostly storage, Dets is not an optimal choice.

(I must admit I find this design decision weird - A better implementation should cache recent lookups. However, since that Ets and Dets are basic facilities in Erlang, I guess that the implementors left that optimization for the users.)

Little Bobby Tables
  • 5,261
  • 2
  • 39
  • 49
  • 1
    I think the thing to remember is that neither ETS nor DETS are databases, they are datastores. To get a more complex behaviour you need to use them within a proper database application like Mnesia. – rvirding Jan 25 '11 at 11:04
  • I never wrote that Ets or Dets are databases. – Little Bobby Tables Jan 25 '11 at 11:40
  • 1
    What @rvirding is at is that DETS is a low-level system where the caching is stripped away. If you combine ETS and DETS, it is fairly easy to write such a caching layer yourself - but what is a good strategy for cache reclamation? It is usually better to leave such choices up to the user. – I GIVE CRAP ANSWERS Jan 25 '11 at 14:52
0

For write rarely, read many data that you're okay keeping in memory, check out a 'Mochiglobal'. Mochiweb has a neat module that abuses Erlang's shared heap for module constants via the code management system to provide super-fast access to terms by generating modules with given values as constants on the fly.

Riak uses a Mochiglobal for the ring state, IIRC.

Hers's the source:

https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl

There's absolutely no persistence here, of course, but if you're looking to heavily optimize reads, you can't get much better than this.

jnewland
  • 124
  • 1
  • 2