Stack Overflow
Stack Overflow

Questions tagged [disk-based]

5 questions
45
votes
10 answers

Python Disk-Based Dictionary

I was running some dynamic programming code (trying to brute-force disprove the Collatz conjecture =P) and I was using a dict to store the lengths of the chains I had already computed. Obviously, it ran out of memory at some point. Is there any easy…
Claudiu
  • 224,032
  • 165
  • 485
  • 680
11
votes
10 answers

Disk backed dictionary/cache for c#

I'm looking for a drop in solution for caching large-ish amounts of data. related questions but for different languages: Python Disk-Based Dictionary Disk-backed STL container classes? Close question in different terms: Looking for a simple…
BCS
  • 75,627
  • 68
  • 187
  • 294
10
votes
4 answers

Java disk-based key-value storage

Is there an efficient Java implementation of a filesystem-based key-value storage with the following features: Store, overwrite, and retrieve byte arrays by a unique ID (may be assigned by the storage) No memory caching (read means read from file…
Andrey Logvinov
  • 119
  • 1
  • 6
10
votes
3 answers

Java: fast disk-based hash set

I need to store a big hash set, able to contain up to approx 200 millions 40 bit values. Storing it as 200 millions 64 bit value would be acceptable (despite the 200 millions * 16 bits loss). The requirements are: tiny memory footprint (disk space…
cocotwo
  • 1,285
  • 2
  • 13
  • 19
1
vote
2 answers

Has anybody used the WB B-tree library?

I stumbled across the WB on-disk B-tree library: http://people.csail.mit.edu/jaffer/WB It seems like it could be useful for my purposes (swapping data to disk during very large statistical calculations that do not fit in memory), but I was wondering…
Chris B
  • 9,149
  • 4
  • 32
  • 38