I want to write a std::map to a file and read it back. I'm looking for a rather simple and minimalist way to do it, without boost. I found that it is doable with vector like here Reading and writing a std::vector into a file correctly with iterators
I found this question as it relates to what I want to do, except I'm looking for the binary alternative.
For types with no dynamic memory (actually, pointers) involved
template<size_t N>
struct Serial
{
char bin[N];
friend ostream& operator<<(ostream& os, const Serial& s)
{
for(auto c : bin)
os << c;
return os;
}
friend istream& operator>>(istream& is, Serial& s)
{
for(auto& c : bin)
is >> c;
return is;
}
};
struct Key
{
static constexpr size_t size = sizeof(Key);
Key(const Serial<size>& s) { memcpy(this, s.bin, size); }
Serial<size>& serialize() { return *reinterpret_cast<Serial<size>*>(this); }
};
struct Value
{
static constexpr size_t size = sizeof(Value);
Key(const Serial<size>& s) { memcpy(this, s.bin, size); }
Serial<size>& serialize() { return *reinterpret_cast<Serial<size>*>(this); }
};
void write(ostream& os, const std::map<Key, Value>& m)
{
for(const auto& p : m)
os << p.first.serialize() << p.second.serialize();
}
void read(istream& is, std::map<Key, Value>& m)
{
Serial<Key::size> k;
Serial<Value::size> v;
while(is >> k >> v)
m[k] = v;
}
For types with dynamic memory (pointers) involved, the solution will be then entirely dependent on how they work, no magical solution can be provided.
Have you considered JSON?
Welcome to the messy, confusing, inconsistent world of serialization. Hope you enjoy the ride!!
This is an age-old problem: how to write a modestly complex data structure to some text or binary format, and then be able to later read it back. There are a couple of different ways to do this. However, you said you wanted to serialize to a binary format, so I would recommend using MessagePack.
There's a C++11 library for working with the MessagePack format called msgpack11 that's also rather lightweight, which would seem to fit your requirements. Here's an example:
std::map<A, B> my_map;
// To save my_map:
msgpack11::MsgPack msgpack{my_map};
std::string binary_data = msgpack.dump();
// Now you can save binary_data to a file.
// To get the map back:
string error_string;
auto msgpack = msgpack11::MsgPack::parse(binary_data, error_string);
std::map<A, B> my_map;
// Now you need to manually read back the data.
For binary writing you should use write method of ostream
ostream& ostream::write (const char* s, streamsize n);
See documentation here: http://en.cppreference.com/w/cpp/io/basic_ostream
You can't write map to file driectly, you should write it's represntation , developed by you. You would need to write each key/value pair individually or buffer them is a data block and write it into file.This really isn't much more complicated than a for loop, though. If map contains classes that aren't trivially constructed and destroyed, you should implement a method that allows to serialize class' binary data.
Binary implementations will necessarily be non-portable (for the resultant file). If that is not a concern then consider defining a custom allocator that uses a memory mapped file. You would then declare your std:map using that allocator as one of the template arguments. You could use that map directly, or using range insertion to save an existing map to a file. If the key or value require allocators (e.g. strings) you would have to do declare versions of those types using the memory mapped allocator in the template declaration and define assignment operators from the key/value type to the new types. You can find some allocator implementations and further discussion by searching for "memory mapped file stl allocator". Also see: Memory mapped file storage in stl vector
void BinSerialize(ostream &out, int32_t x);
void BinSerialize(ostream &out, int16_t x);
void BinSerialize(ostream &out, int8_t x);
void BinSerialize(ostream &out, const string &s)
{
BinSerialize(out, (int32_t)s.size());
out.write(size.c_str(), s.size()));
}
temmplate<class KeyT, ValueT>
void BinSerialize(ostream &out, const std::map<KeyT, ValueT> &m)
{
BinSerialize(out, (int32_t)m.size());
for (auto& item : m)
{
BinSerialize(out, item.first);
BinSerialize(out, item.second);
}
}
void BinDeserialize(istream &input, int32& x);
void BinDeserialize(istream &input, int16& x);
void BinDeserialize(istream &input, int8& x);
void BinDeserialize(istream &input, string &s)
{
int32_t size;
BinDerialize(out, size);
s.resize(size);
out.read(size.c_str(), size);
}
temmplate<class KeyT, class ValueT>
void BinDeserialize(istream &input, std::map<KeyT, ValueT> &m)
{
int32_t size;
m.clear();
BinDeserialize(out, size);
for (int32_t i=0; i<size; ++i)
{
std::pair<KeyT, ValueT> item;
BinDeserialize(out, item.first);
BinDeserialize(out, item.second);
m.insert(item);
}
}
This is quickly written. It is possible to improve it with templates, to cover all basic types and all STL containers.
Also it would be nice to keep in mind about the endian.
It is better avoid use of overloaded operators in this case. But if you if to do it it is best to define class which will wrapp STL stream and will have own set of overloaded >> << operators. Take a look on Qt QDataStream.
