-1

I'm trying to solve this problem from several days.

In this program there is an unordered_map which from string key to structure value. The idea works, but when I put the unordered_map in a class and then access it more easily I get an error:

Eliminare.exe!std::_Hash<std::_Umap_traits<std::string,Data,std::_Uhash_compare<std::string,std::hash<std::string>,std::equal_to<std::string>>,boost::interprocess::allocator<std::pair<std::string,Data>,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index>>,0>>::_Find_last<std::string>(const std::string & _Keyval, const unsigned __int64 _Hashval) Row 1510    C++
    Eliminare.exe!std::unordered_map<std::string,Data,std::hash<std::string>,std::equal_to<std::string>,boost::interprocess::allocator<std::pair<std::string,Data>,boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,__int64,unsigned __int64,0>,0>,boost::interprocess::iset_index>>>::at(const std::string & _Keyval) Riga 387    C++
Eliminare.exe!MapCreator::getValue(std::string Key) Row 37  C++
Eliminare.exe!main() Row 7  C++

This is the main:

#include "MapCreator.h"

int main() {
    MapCreator mappaClass;

    auto values = mappaClass.getValue("value");
}

While this is in MapCreator.h:

#pragma once
#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <unordered_map>
#include<iostream>
typedef struct Data
{
    int a;
    int b;
}dataClass;

namespace bost = boost::interprocess;
typedef std::string    KeyType;
typedef dataClass MappedType;
typedef std::pair<std::string, dataClass> ValueType;
typedef bost::allocator<ValueType, bost::managed_shared_memory::segment_manager> ShmemAllocator;
typedef std::unordered_map<KeyType, MappedType, std::hash<KeyType>, std::equal_to<KeyType>, ShmemAllocator> MySHMMap;

class MapCreator
{
public:
    MySHMMap* mappa = nullptr;
    MapCreator() {
        bost::shared_memory_object::remove(nameMemory);
        bost::managed_shared_memory segment(bost::create_only, nameMemory, sizeDeclared);
        ShmemAllocator alloc_inst(segment.get_segment_manager());

        mappa = segment.construct<MySHMMap>("MySHMMapName")(alloc_inst);
        dataClass value;
        value.a = 12;
        value.b = 1111;
        mappa->insert(std::pair<std::string, dataClass>("value", value));
        std::cout << mappa->at("value").a;

    }
    dataClass getValue(std::string Key) {
        return mappa->at(Key);
    }
private:
    const int sizeDeclared = 65536;
    const char nameMemory[20] = "SharedMemoryName";
};

The "cout" in the MapCreator constructor correctly prints "12", but using the getValue function, it prints the error mentioned above.

Thanks for your help, if you have any questions about the code, just ask.

INFO:

  • Compiler: Visual C++ with Visual Studio 2022
  • the code without the class works well
  • the version of boost class is: 1.78.0
sehe
  • 374,641
  • 47
  • 450
  • 633
  • 2
    "*in this program there is an unordered_map that goes into a shared memory to access it in another process*" - that will never work. You can't share containers across process boundaries, you can only share fundamental types. Containers typically contain pointers to data, and those pointers are only valid in the process that allocates the memory they point at. Processes don't share memory address spaces. *Maybe* if you used a custom allocator that allocated shared memory... but even that is risky and uncertain to work. So, just don't do it. – Remy Lebeau Mar 09 '22 at 23:22
  • Pointers cannot be shared across processes. If you're picking up any STL structure that uses pointers or allocates memory, it won't work. – cup Mar 09 '22 at 23:25
  • But i already tried the code and it works really well, but when i tried to create the class and make the code cleaner, it seems to lose the pointer. the question therefore is NOT the sharing between the processes, but why that error comes out in this code – Daniele Carriere Mar 09 '22 at 23:35
  • @RemyLebeau I thought it was possible but you had to use custom allocators. – Ben Mar 10 '22 at 11:43
  • @RemyLebeau yes, look the code, i use it. but the error is another. the error is not between processes but in the main process that generate the map – Daniele Carriere Mar 10 '22 at 11:50
  • @RemyLebeau I have a zillion examples of much more complicated containers - including unordered_map - in shared memory. It *can* work with the right allocators. – sehe Mar 10 '22 at 11:53
  • @sehe so do you know what is my error? just copy all code in visual studio And it reproduces the problem – Daniele Carriere Mar 10 '22 at 11:54
  • I do. Was already writing the answer. – sehe Mar 10 '22 at 12:02
  • It's taking a while, but then you'll have something :) – sehe Mar 10 '22 at 12:23

1 Answers1

1

There are multiple issues.

The first one is that segment is a local variable in your constructor, meaning that mappa is a dangling pointer after the constructor returns.

Second, bigger, problem is that your map uses a shared memory allocator correctly, but the strings do NOT. String are also dynamically allocating, so your solution cannot work unless you make the string also use shared memory.

Starting the fix:

namespace bip = boost::interprocess;
namespace bc  = boost::container;
using Segment = bip::managed_shared_memory;
using Mgr     = Segment::segment_manager;

template <typename T> using Alloc = bip::allocator<T, Mgr>;
using MyString   = bc::basic_string<char, std::char_traits<char>, Alloc<char>>;
using KeyType    = MyString;
using MappedType = dataClass;
using ValueType  = std::pair<KeyType const, MappedType>;
using MySHMMap   = std::unordered_map<KeyType, MappedType, std::hash<MyString>,
                                    std::equal_to<MyString>, Alloc<ValueType>>;

Note that on my compiler it's also required to use KeyType const in the ValueType.

In the execution there's additional complexity: segment must be a field, but it can only be constructed from the initializer list (or using NSMI). That means that we have to make the remove also occur (before that) in the initializer list. Here's my initial attempt:

template <typename T> using Alloc = bip::allocator<T, Mgr>;
using MyString   = bc::basic_string<char, std::char_traits<char>, Alloc<char>>;
using KeyType    = MyString;
using MappedType = dataClass;
using ValueType  = std::pair<KeyType const, MappedType>;
using MySHMMap = boost::unordered_map<KeyType, MappedType, boost::hash<MyString>,
                         std::equal_to<MyString>, Alloc<ValueType>>;

class MapCreator {
  public:
    MapCreator()
        : mappa(segment.find_or_construct<MySHMMap>("MySHMMapName")(
              segment.get_segment_manager()))
    {
        mappa->emplace(MyString("value", segment.get_segment_manager()),
                       dataClass{12, 1111});
    }

    dataClass getValue(std::string key) const {
        return mappa->at(MyString(key.c_str(), segment.get_segment_manager()));
    }

  private:
    static constexpr int sizeDeclared = 65536;

    // note: declaration order defines initialization order!
    std::string_view nameMemory = "SharedMemoryName";

    struct pre_remover {
        pre_remover(const char* name)
        {
            bip::shared_memory_object::remove(name);
        }
    } pre_remove{nameMemory.data()};

    Segment segment{bip::create_only, nameMemory.data(), sizeDeclared};
    MySHMMap* mappa = nullptr;
};

Notes

  • there was also no reason at all to allocate a fixed size buffer of 20 chars for the name.
  • The order in which members are declared is the order in which they're initalized. That's important because mappa needs to come after segment, e.g.

I found out that std::hash isn't actually specialized for the MyString instance, and because I know down the line I'd want to change that anyways I'm not wasting time to create a Hash function object. Instead, I'm switching to boost::unordered_map just so boost::hash<> is used

This is already working Live On Coliru

Prints:

12, 1111

Improvements

  1. Scoped Allocators

    Things are pretty clunky around the allocator/get_segment_manager(). Using scoped-allocators you can alleviate that:

    template <typename T>
    using Alloc      = bc::scoped_allocator_adaptor<bip::allocator<T, Mgr>>;
    using MyString   = bc::basic_string<char, std::char_traits<char>, Alloc<char>>;
    using KeyType    = MyString;
    
    template <typename K, typename V, typename H = boost::hash<K>,
              typename Eq = std::equal_to<K>>
    using SHMap = boost::unordered_map<K, V, H, Eq, Alloc<std::pair<K const, V>>>;
    
    using MySHMMap = SHMap<MyString, Data>;
    

    Now you can "simply" write:

    mappa->emplace("value", Data{12, 1111});
    

    Instead of the complicated version before. Live On Coliru

  2. Heterogenous Key Lookup

    The other place where we are constructing MyString (just to throw it away after the at() call...), we can use boost::unordered_map's compatible-key lookup:

    Data getValue(std::string_view key) const {
        auto it = mappa->find(key, mappa->hash_function(), mappa->key_eq());
        return it != mappa->end() ? it->second
                                  : throw std::range_error("getValue");
    }
    

    This requires the hash and equality to be compatible:

    using MySHMMap = SHMap<MyString, Data, std::hash<std::string_view>,
                           std::equal_to<std::string_view>>;
    

    See it Live On Coliru

  3. Using C++20 standard unordered_map instead (see https://en.cppreference.com/w/cpp/container/unordered_map/find):

    struct Hash : std::hash<std::string_view> {
        using is_transparent = std::true_type;
    };
    struct Eq : std::equal_to<std::string_view> {
        using is_transparent = std::true_type;
    };
    using MySHMMap = SHMap<MyString, Data, Hash, Eq>;
    

    And then:

        Data getValue(std::string_view key) const {
    #ifdef __cpp_lib_generic_unordered_lookup
            auto it = mappa->find(key);
    #else
            auto it = mappa->find(
                MyString(key.data(), key.size(), segment.get_segment_manager()));
    
    #endif
            return it != mappa->end() ? it->second
                                      : throw std::range_error("getValue");
        }
    

    See it Live On Coliru as well

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Posting improvements in a minute – sehe Mar 10 '22 at 12:41
  • Posted the improvements I had in mind. – sehe Mar 10 '22 at 13:06
  • Just for completeness the first version running on MSVC: https://rextester.com/UTSE85033 – sehe Mar 10 '22 at 16:44
  • Using scoped_allocators is a lot harder on MSVC - maybe in part due the older versions but here is a start https://rextester.com/KVLT7093 – sehe Mar 10 '22 at 17:04
  • 1
    omg... you are the best... Not only for having solved the problem, but also for the optimization tips you gave me. You've been really professional and kind, now works a wonder :) What I don't understand, however, is, I realized that strings need a pointer, but why could I read the data correctly in the second process? Perhaps because he trifed him in hash and therefore he no longer needed an allocator? – Daniele Carriere Mar 10 '22 at 17:34
  • Some implementations do [SSO](https://stackoverflow.com/questions/10315041/meaning-of-acronym-sso-in-the-context-of-stdstring):Small String Optimization. That means that Small Enough Strings™ won't require dynamic allocations. Your "it was hashed" hypothesis can't explain it because hash-tables *always* have a two-step lookup: first the fast O(1) hash lookup, then a (typically linear) check using the key-equality comparer, which requires access to the full key data. – sehe Mar 10 '22 at 18:17
  • perfect, thx for your help again ;) have a nice day – Daniele Carriere Mar 10 '22 at 18:32
  • oh no! T.T I discovered that actually managed_shared_memory uses a file and not the ram ... do you think it is possible to take it to RAM without distorting the code? – Daniele Carriere Mar 10 '22 at 19:07
  • Managed shared memory uses memory. – sehe Mar 10 '22 at 19:50
  • 1
    Ah is fine, because every time you start the program and create shared memory, the use of my disk rises, and looking on the net I discovered that he uses the disk. But then I noticed that in reading the data does not use the disc so everything ok. – Daniele Carriere Mar 11 '22 at 13:02