making non-shared copies of boost::interprocess shared memory objects

Question

I have implemented various classes that are designed to be used in boost::interprocess shared memory segments. All their constructors employ allocator<void,segment_manager> references—some explicitly in the definitions I have written (like the Foo constructor below) and some simply because that's what the boost container definition requires, in boost library code that I should not be changing (like the IndexVector below).

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>

typedef boost::interprocess::managed_shared_memory                   Segment;
typedef boost::interprocess::managed_shared_memory::segment_manager  SegmentManager;
typedef boost::interprocess::allocator< void, SegmentManager >       Allocator;

typedef size_t                                                       Index;
typedef boost::interprocess::allocator< Index, SegmentManager >      IndexAllocator;
typedef boost::interprocess::vector<    Index, IndexAllocator >      IndexVector;

class Foo
{
    public:
        Foo( const Allocator & alloc ) : mData( alloc ) {}
       ~Foo() {}

    private:
        IndexVector mData;

};

Mostly, these objects sit in shared memory. But I sometimes want to create copies of them in non-shared memory. My question is this: do I have to define a whole different class (e.g. Foo_Nonshared) containing different member types (std::vector<Index> instead of my shared IndexVector type) and provide copy/conversion functions between them? That will be a lot of work and a lot of stupid duplication. I could reduce duplication by providing an alternative constructor to the existing Foo class, but then I wouldn't know how to initialize the IndexVector member without an allocator.

Or is there some nice shortcut? I'm imagining some sort of particular allocator instance that I can pass to Foo(), and which will hence be passed on to the IndexVector constructor, which will be recognized by both as meaning "allocate in non-shared memory". Does such a thing exist? Is there a "dummy segment manager" for managing vanilla non-shared memory? Or are there other ways around this problem?

I'm hoping for C++03-compatible answers even though I'm also interested to learn the C++11+ ways of doing things.

Update following question being marked as duplicate: I have read these previous similar questions:

• boost::interprocess Containers of containers NOT in shared memory
• boost::interprocess Containers of containers NOT in shared memory copy

and have tried to generalize what I see there, with some successes and some failures (see listing below). There are a few compiler errors that I haven't been able to resolve, marked ERROR—in particular I cannot figure out how to instantiate methods that iterate over the members of these highly "meta" containers. But with or without those errors, I cannot yet see how to make templates-of-templates into a maintainable solution (my objects, in reality, contain containers of other complex objects, which contain further containers, which AFAICS complicates the syntax beyond sanity... see the part marked "hmm").

I guess, in the end, I might have to re-design to avoid having the same objects in shared and heap memory.

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>

namespace bip = boost::interprocess; // warning: C++11 alias declaration

template <typename T, template<typename...> class Allocator>  // warning: C++11 variadic template
    using Vector = bip::vector< T, Allocator<T>>;             // warning: C++11 alias declaration
// this seems to work to get some of the nested <>ness under control.
// But I can't figure out how to create an iterator to this kind of type (see errors below)

// what once were classes are now class templates

template <template<typename...> class Allocator>              // warning: C++11 variadic template
    class Bar
    {
        public:
             Bar( const Allocator<void> & alloc ) : mInts( alloc ) {}
            ~Bar() {}

            void Report( void );

        private:
            Vector< int, Allocator > mInts;
    };

template <template<typename...> class Allocator>              // warning: C++11 variadic template
    class Foo
    {
        public:
             Foo( const Allocator<void> & alloc ) : mBars( alloc ) {}
            ~Foo() {}

            void Report( void );


        private:
            Vector<  Bar<Allocator>, Allocator >  mBars; // hmm, with more complex structures this is going 
                                                         // to get unmanageably< nested< very< quickly > > > ...

    };


// Define allocator templates

template <typename T>
    using HeapAllocator  = std::allocator<T>; // warning: C++11 alias declaration

template <typename T> 
    using ShmemAllocator = bip::allocator<T, bip::managed_shared_memory::segment_manager>; // warning: C++11 alias declaration

// Define two class variants: one for use on the heap and one for use in shared memory

using HeapFoo  = Foo< HeapAllocator  >; // warning: C++11 alias declaration
using ShmemFoo = Foo< ShmemAllocator >; // warning: C++11 alias declaration

// Try to define methods (unsuccessful so far because of the iterators,
// but they compile OK if the function bodies are left empty):

template <template<typename...> class Allocator>              // warning: C++11 variadic template
    void
    Bar< Allocator >::Report( void )
    {
        std::cout << "[";
        Vector< int, Allocator >::iterator it;
// ERROR:     ^~~~~ expected ';' after expression
        for( it = mInts.begin(); it += mInts.end(); it++ )
            std::cout << ( it == mInts.begin() ? "" : ", " ) << *it;
        std::cout << "]\n";
    }

template <template<typename...> class Allocator>              // warning: C++11 variadic template
    void
    Foo< Allocator >::Report( void )
    {
        Vector< Bar< Allocator >, Allocator >::iterator it;
// ERROR:     ^~~~~ expected ';' after expression
        for( it = mBars.begin(); it += mBars.end(); it++ )
            it->Report();
        std::cout << "\n";
    }

int main( void )
{
    struct shm_remove
    {
         shm_remove() { bip::shared_memory_object::remove( "MySharedMemory" ); }
        ~shm_remove() { bip::shared_memory_object::remove( "MySharedMemory" ); }
    } remover;
    bip::managed_shared_memory   seg( bip::create_only, "MySharedMemory", 65536 );

    ShmemAllocator< void > shalloc( seg.get_segment_manager() );
    HeapAllocator<  void > halloc;

    HeapFoo  foo1( halloc  );
    ShmemFoo foo2( shalloc );
    foo1.Report();
    foo2.Report();  
}

Answer 1

Ok, you've run into the Frequently Annoying Edgecase that template-template arguments aren't first class citizens in C++ (you cannot pass them around/typedef them):

• How to transmit a template?

What shall we do?

1. allocator::rebind<T>

   Allocators have a rebind mechanism, I daresay precisely because of this. So you can pass a alloc<void> as if it is the open template, because you can always get from there to a sibling allocator type by doing Alloc::rebind<T>::other.

2. Add to this the fact that allocators usually have conversion constructors that do this rebinding, you don't need to be overly specific in many places taking allocators

3. in c++11, scoped_allocators have been introduced to avoid having to manually pass allocator instances in a number of places that will do internal construction of elements (e.g. emplace_back).

   There's library magic in place that will automatically add the allocator instance from the container's scoped_allocator as the last constructor argument (by default). Boost Container library has backported the scoped_allocator_adaptor concept to c++03 so you can use it.

Here's a full sample that shows you how to solve the issues you had, and also, how you can mix the heap-based Bar instances with the shared-memory Foo instance:

foo2.add(bar1); // this works because of ... MAGIC!

Which works due to the scoped_allocator mentioned above.

Live On Coliru

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/container/scoped_allocator.hpp>

namespace bip = boost::interprocess;

namespace generic { 

    template <typename T, typename Alloc/* = std::allocator<T>*/ >
        using vector = bip::vector<T, typename Alloc::template rebind<T>::other >;

    template <typename Alloc> struct Bar {
        typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator

        // only require allocator if not default-constructible
        Bar(Alloc alloc = Alloc()) : mInts(alloc) {}

        // conversion constructor so we can convert between allocators 
        template <typename OtherAlloc>
            Bar(Bar<OtherAlloc> const& rhs, Alloc alloc = Alloc())
                : mInts(rhs.mInts.begin(), rhs.mInts.end(), alloc) 
            {
            }

        void Report() const;

        void add(int i) { mInts.emplace_back(i); }

      private:
        template<typename OtherAlloc> friend struct Bar; // we can see each other's mInts
        typedef vector<int, Alloc> ints_t;
        ints_t mInts;
    };

    template <typename Alloc> struct Foo {
        typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator

        Foo(Alloc alloc = Alloc()) : mBars(alloc) {}
        void Report() const;

        template <typename Bar>
        void add(Bar const& bar) { mBars.emplace_back(bar); }

      private:
        typedef vector<Bar<Alloc>, Alloc> mbars_t;
        mbars_t mBars;
    };
}

namespace heap {
    using VAlloc = std::allocator<void>;

    using Bar = generic::Bar<VAlloc>;
    using Foo = generic::Foo<VAlloc>;
}

namespace shared {
    using VAlloc = boost::container::scoped_allocator_adaptor<bip::allocator<void, bip::managed_shared_memory::segment_manager> >;

    using Bar = generic::Bar<VAlloc>;
    using Foo = generic::Foo<VAlloc>;
}

template <typename Alloc> void generic::Bar<Alloc>::Report() const {
    std::cout << "[";
    for (typename ints_t::const_iterator it = mInts.begin(); it != mInts.end(); it++)
        std::cout << (it == mInts.begin() ? "" : ", ") << *it;
    std::cout << "]\n";
}

template <typename Alloc>
void generic::Foo<Alloc>::Report() const {
    for (typename mbars_t::const_iterator it = mBars.begin(); it != mBars.end(); it++)
        it->Report();
    std::cout << "\n";
}

int main(void) {
    struct shm_remove {
        shm_remove()  { bip::shared_memory_object::remove("MySharedMemory"); }
        ~shm_remove() { bip::shared_memory_object::remove("MySharedMemory"); }
    } remover;

    ///////////////////////////////////
    // heap based:
    std::cout << "Heap based storage: \n";

    heap::Foo foo1;
    heap::Bar bar1;

    bar1.add(42);
    bar1.add(2);
    bar1.add(-99);

    foo1.add(bar1);
    foo1.Report();

    /////////////////////////////////
    std::cout << "Shared memory storage: \n";
    bip::managed_shared_memory seg(bip::create_only, "MySharedMemory", 65536);
    shared::VAlloc shalloc(seg.get_segment_manager());

    shared::Foo foo2(shalloc);
    shared::Bar bar2(shalloc);

    bar2.add(43);
    bar2.add(3);
    bar2.add(-98);

    foo2.add(bar2); // of course this works
    foo2.add(bar1); // this works because of ... MAGIC!
    foo2.Report();
}

Prints:

Heap based storage: 
[42, 2, -99]

Shared memory storage: 
[43, 3, -98]
[42, 2, -99]