Stack Overflow
Stack Overflow

Questions tagged [stxxl]

STXXL is an open source out-of-core implementation of the STL, that is available for all major desktop operating systems under the Boost Software License 1.0.

STXXL is an open source out-of-core implementation of the containers, that are part of the C++ STL. It is released under the Boost Software License 1.0 and available for Linux, Mac OS X, Free BSD and Windows (2000 upwards).

52 questions
9
votes
1 answer

Compiling STXXL under Android NDK r8b

I'm trying to compile STXXL under Android NDK r8b (I have the same problem under the newer r8c btw). I'm compiling using gnustl_static. I need C++11 support so initially I tried setting LOCAL_CPPFLAGS := -std=c++11 but this threw up an error about…
Goz
  • 61,365
  • 24
  • 124
  • 204
3
votes
1 answer

stxxl sorting of very large file (ubuntu)

I am trying to sort a large file with about billion records (each containing four integers). The size of the file would shoot up beyond 50GB. I am testing my code with 400 million records (about 6 GB file). My disk configuration looks like this:…
Chirag Jain
  • 143
  • 2
  • 8
3
votes
2 answers

High memory usage with STXXL

I'm working on a project using STXXL, which I understand to be an out of core version of the C++ STL library. Currently, my program is running fine with it, but the problem I'm facing now is that when the program is running, it uses close to 2GB of…
Andrewziac
  • 155
  • 2
  • 16
3
votes
1 answer

Using STXXL in Qt project

How do I get qmake to include the stxxl.mk file into the generated makefiles? I have a Qt project, which deals with large files (>RAM) and therefore want to employ STXXL. The STXXL documentation says: Building an Application After compiling the…
Nobody moving away from SE
  • 9,347
  • 3
  • 45
  • 92
3
votes
2 answers

STXXL equivalent in Java

I'm searching a collection framework designed for huge datasets in Java that behaves transparently, like STXXL does for C++. It should transparently swap to disk, but in a much more efficient manner than plain OS-based VM swapping. A…
Steve Schnepp
  • 4,620
  • 5
  • 39
  • 54
3
votes
1 answer

`pointer being freed was not allocated` when using stxxl queue

My code seems to work(I haven't tried it with large datasets because of the above error). Code: #include #include #include int main() { //queue q; //this works stxxl::queue q; //does not work for…
Lostsoul
  • 25,013
  • 48
  • 144
  • 239
2
votes
0 answers

How to use a C++ 14 library in a C++ 23 project (MSVC)?

I'm writing a program that makes heavy use of C++ 20 and newer features. The problem is that in the project I need the stxxl library, which is compiled exclusively on C++ 14 . If I try to include it as described in this guide (via .h files and…
Chase
  • 43
  • 4
2
votes
0 answers

STXXL underperforms when using very fast SSDs

I'm currently writing a tool which uses the STXXL to find similarities between a large file on the hard drive and vectors residing in RAM. I wrote an example(for Windows) below to show what I mean. The referenced file is about 38GB large and was…
Kasastran
  • 23
  • 3
2
votes
2 answers

C++: Which Data Type in STXXL is suitable to create External Memory Binary Search Tree?

I want to create an External Memory Binary Search Tree Data Structure whose data sits in the external memory using stxxl as a library. For this purpose, which Data Type in STXXL is suitable to use as nodes in the Tree. If we use stxxl:Vector as the…
Akif Khan
  • 53
  • 8
2
votes
3 answers

How to append a large number of elements to stxxl vector efficiently?

I need to append a large number of elements to a stxxl vector. What is the most efficient way of adding elements to a stxxl vector? Right now, I'm using push_back of the stxxl vector, but it doesn't seem very efficient. It's far from saturating the…
Da Zheng
  • 111
  • 2
  • 8
2
votes
2 answers

STXXL multidimensional vector makes program freeze

Trying to deal with huge data sets, I have been using the convenient STXXL library.Though I have run into a slight problem when trying to use/generate a multidimensional vector. The following program freezes the system for a while and then gets…
Morgan Bengtsson
  • 1,341
  • 1
  • 14
  • 23
2
votes
3 answers

Why cannot structs storing plain pointers to internal memory be stored in stxxl containers?

In the stxxl FAQ, I found this: Parameterizing STXXL Containers STXXL container types like stxxl::vector can be parameterized only with a value type that is a POD (i.e. no virtual functions, no user-defined copy assignment/destructor, etc.) and…
gd1
  • 11,300
  • 7
  • 49
  • 88
2
votes
1 answer

STXXL: limited parallelism during sorting?

I populate a very large array using a stxxl::VECTOR_GENERATOR::result::bufwriter_type (something like 100M entries) which I need to sort in parallel. I use the stxxl::sort(vector->begin(), vector->end(), cmp(), memoryAmount) method, which in…
jrbn
  • 303
  • 2
  • 6
2
votes
0 answers

Does STXXL save the state after crash?

I'm writing a webapp in C++ and would like to use a fast database on SSD. Seems like STXXL would be faster than other choices (slite,mongodb,...) Do STXXL's containers keep the data after accidental shutdown (server crash)? Can the program that was…
kirill_igum
  • 3,953
  • 5
  • 47
  • 73
1
vote
1 answer

external memory vector

I am developing an external memory data structure, and I need to put the data into a vector structure that automatically make swapping (maybe using LRU strategy) in order to keep a fixed RAM memory. I have tried the stxxl vector structure, but the…
darkloz
  • 11
  • 4
1
2 3 4