3

I am using Google protobuf in an embedded application with limited heap memory. I am currently trying to make the application to use dynamic allocation only as a last resort. For this, I have some shared buffer on which I create all the proto messages or most of them. Everything seems to be Ok, except for the cases where the proto message expects a ::std::string parameter.

The way I understand the ::std::string constructors description, is that it will create a copy of the data I supply. For example this constructor:

s = new(sharedBufferAddress) ::std::string(mApn, mApnSize);

where 

char                        mApn[APN_MAX_SIZE];
int8_t                      mApnSize;

will create an object located at the sharedBufferAddress, but the data inside will be copied on a buffer assigned on the heap.

The question is whether there is some way to have the pointer that is returned by the c_str() function to some specified address.

norok2
  • 25,683
  • 4
  • 73
  • 99
Moraru Gabriel
  • 71
  • 4
  • Related: https://stackoverflow.com/questions/30057919/boost-unordered-map-in-shared-memory-using-stdstring-key and https://stackoverflow.com/questions/32581057/shared-memory-of-stdstring-give-segmentation-fault-linux – Galik Oct 01 '18 at 12:46
  • This may provide a solution: https://stackoverflow.com/questions/5312714/need-to-write-shared-memory-allocator-for-c-stdvector – Galik Oct 01 '18 at 12:50
  • String will take a custom buffer allocator as an argument, so this may be your way forward. `explicit basic_string( const Allocator& alloc );` – Gem Taylor Oct 01 '18 at 13:37
  • 1
    Excuse my ignorance in embedded systems specifics, but isn't it possible to create your own allocator that allocates memory wherever you want (stack or heap)? If I understand how allocators work, you can create an allocator that can only allocate on the stack and throws an exception if it exceeds the compile-time allowed size. That allocator should be the manager of your shared memory. – The Quantum Physicist Oct 01 '18 at 14:13
  • the `std::string` is alias of `std::basic_string, std::allocator`. To write custom specialization of `std::allocator>` is undefined behavior. So it does not look like hinted at ways here are actually legal. – Öö Tiib Oct 01 '18 at 14:19
  • 1
    @ÖöTiib What is it exactly that's undefined? I can't understand what you're saying. Please rephrase/elaborate. – The Quantum Physicist Oct 01 '18 at 14:31
  • @TheQuantumPhysicist you are not allowed to specialize `std::allocator`. If you do that then the resulting program behaves in undefined manner. You may specialize `std::allocator` or write templates like `MyAllocator` but these are not what `std::string` uses as allocators. – Öö Tiib Oct 01 '18 at 14:37
  • Another option to consider is to use a custom implementation around string_view instead of string? – Gem Taylor Oct 01 '18 at 18:50
  • @ÖöTiib You don't need to specialize `std::allocator` you can just write a completely new allocator. – Galik Oct 01 '18 at 19:09
  • @Galik your completely new allocator can't be provided to `std::string` since allocator that can be passed to `std::string` has to be of type `std::allocator`. – Öö Tiib Oct 01 '18 at 19:54
  • @ÖöTiib I never heard about that. Can you provide a reference? – Galik Oct 01 '18 at 19:56
  • @Galik read https://en.cppreference.com/w/cpp/string/basic_string what is std::string? – Öö Tiib Oct 01 '18 at 20:00
  • @ÖöTiib I'm not sure what you're getting at. We would not be using a `std::string` in this scenario. We would use something like `std::basic_string, MySpecialAllocator>` instead. – Galik Oct 01 '18 at 20:07
  • @Galik and how you pass such string of different type to google protobuf library functions that expect ::std::string parameter (like OP asked)? – Öö Tiib Oct 01 '18 at 20:18

1 Answers1

0

You can not do much to hack the implementation of std::string. All attempts will do something illegal and potentially result with undefined behaviors.

One option is to evolve custom implementation of Protobuf or custom modifications of stock Protobuf.

Other option is to use existing custom implementations. For example Finnish programmer Petteri Aimonen has implemented Protobuf implementation Nanopb in C for embedded development and released it under zlib license. That can be likely more useful than custom modified stock Protobuf in your embedded project since C and C++ work passably well together.

Öö Tiib
  • 10,809
  • 25
  • 44
  • Thank you for you answer, but changing proto messages is overcomplicated as involves other company as well. – Moraru Gabriel Oct 01 '18 at 15:35
  • @MoraruGabriel You did not describe what types of what sizes you need in messages. You can use other similar implementations that implement support to more types/sizes like that https://github.com/protobuf-c/protobuf-c or you can extend those yourself and if you propose your changes as pull request to them, then you will likely get free code-review to your changes. ;) If it involves other company then certainly avoid hacking standard library or classes like std::string in it. – Öö Tiib Oct 01 '18 at 15:57