2

I have the following struct which will be used to hold plugin information. I am very sure this will change (added to most probably) over time. Is there anything better to do here than what I have done assuming that this file is going to be fixed?

struct PluginInfo
{
    public:
        std::string s_Author;
        std::string s_Process;
        std::string s_ReleaseDate;
        //And so on...

        struct PluginVersion
        {
            public:
                std::string s_MajorVersion;
                std::string s_MinorVersion;
                //And so on...
        };
        PluginVersion o_Version;

        //For things we aren't prepared for yet.
        void* p_Future;
};

Further, is there any precautions I should take when building shared objects for this system. My hunch is I'll run into lots of library incompatibilities. Please help. Thanks

Oded
  • 489,969
  • 99
  • 883
  • 1,009
nakiya
  • 14,063
  • 21
  • 79
  • 118
  • 1
    What makes you think that the things you aren't prepared for will be a single `void *` ? – CB Bailey Oct 14 '10 at 07:00
  • @Mitch Wheat: I am pretty sure it is YAGNI just that it's 'You **Are** Gonna Need It' :) – nakiya Oct 14 '10 at 07:01
  • I definitely wouldn't use void* p_Future. When you find some data you put in there, you have to change the name to p_Now, and minutes later to p_Past. Time is not constant :-) – Philipp Oct 14 '10 at 07:01
  • Now seriously: How are you using PluginInfo? Why is it a problem if it changes? Please add some info about this. – Philipp Oct 14 '10 at 07:02
  • @Charles Bailey: Whatever they are, void* will let me convert back and forth at least. – nakiya Oct 14 '10 at 07:02
  • @Philipp: Because `PluginInfo` struct will be visible to both run-time loaded shared objects and an application. Old shared objects should work with newer versions of application. – nakiya Oct 14 '10 at 07:03
  • A `void` pointer will spell troubles for the future maintenance engineers because all `STL` clases follow RAII (meaning that every object clean up their resources as they are destroyed), but there is no way for a `void` pointer to be cleaned up. – rwong Oct 14 '10 at 07:06
  • Actually, each shared object will supply the application with one dynamically allocated `PluginInfo` object. This will persist throughout the lifetime of the application. Cleanup is the least of my worries. – nakiya Oct 14 '10 at 07:09
  • @Mitch: It's not YAGNI. Forward compatibility, backward compatibility (they're different, see wikipedia), versioning and componentization has to be planned well ahead because of the mode of software distribution. You can't tell your customers to do a complete reinstall every month. At least not until we move everything into the clouds ... – rwong Oct 14 '10 at 07:11
  • @rwong : I'm aware of versioning. How many structs have you seen in your career with unused fields? – Mitch Wheat Oct 14 '10 at 07:13
  • @Mitch: sorry for nit-picking your use of YAGNI. – rwong Oct 14 '10 at 07:17
  • Only a single pointer value will be convertible to and from `void *`. What if you want new text data or multiple numbers? – CB Bailey Oct 14 '10 at 07:18
  • 1
    @nakiya: Please explain how your struct will be used by other source code. If any source code that uses your struct is always compiled together with your struct definition, then you can simply add new fields *when the need arises*, and source code which were written to use the older fields will still compile (simply ignoring the new fields), as long as you do not remove any existing fields. – rwong Oct 14 '10 at 07:23
  • @rwong: That's it. This has to work without recompiling already shipped shared objects. Which is what all the fuss is about. – nakiya Oct 14 '10 at 07:29
  • @nakiya so if the next update to your compiler fixes a bug in std::string, you can't move to it, because your plugins are compiled with the old incompatible version? – Pete Kirkham Oct 14 '10 at 07:37
  • @Pete Kirkham: Obviously, We will have to recompile all shared objects then. But the two scenarios are different right? – nakiya Oct 14 '10 at 07:41
  • @nakiya: it is considered dangerous to pass STL objects between different builds, because when shared objects from incompatible builds are used together at runtime, there is no run-time safety guard that can catch the situation (other than your checking of the version information and diligently incrementing the version number of each build). It will usually cause the program to crash. – rwong Oct 14 '10 at 07:49
  • Either the plugins have a stable binary interface, or you expect them to be recompiled when something in the interface changes. In the latter case, you might want to try and avoid a recompile if the the change is not to shared objects other than the PluginInfo struct. In that case, I'd tend to add to the request interface rather than changing the result of the request (so if you decide you also want the plugin's colour scheme, that is not an extension to PluginInfo, but the result of looking up a different function in the plugin's library) – Pete Kirkham Oct 14 '10 at 07:55

6 Answers6

6

What about this, or am I thinking too simple?

struct PluginInfo2: public PluginInfo
{
    public:
        std::string s_License;
};

In your application you are probably passing around only pointers to PluginInfos, so version 2 is compatible to version 1. When you need access to the version 2 members, you can test the version with either dynamic_cast<PluginInfo2 *> or with an explicit pluginAPIVersion member.

Roland Illig
  • 40,703
  • 10
  • 88
  • 121
  • This is a good idea (a simple one that works and is in the spirit of C++). Care should be taken so that the user's source code always pass *by reference* in order to avoid the [object slicing problem](http://stackoverflow.com/questions/274626/what-is-the-slicing-problem-in-c), [also here](http://stackoverflow.com/questions/274626#274636). – rwong Oct 14 '10 at 07:39
6

Either your plugin is compiled with the same version of C++ compiler and std library source (or its std::string implementation may not be compatible, and all your string fields will break), in which case you have to recompile the plugins anyway, and adding fields to the struct won't matter

Or you want binary compatibility with previous plugins, in which case stick to plain data and fixed size char arrays ( or provide an API to allocate the memory for the strings based on size or passing in a const char* ), in which case it's not unheard of to have a few unused fields in the struct, and then change these to be usefully named items when the need arises. In such cases, it's also common to have a field in the struct to say which version it represents.

But it's very rare to expect binary compatibility and make use of std::string. You'll never be able to upgrade or change your compiler.

Pete Kirkham
  • 48,893
  • 5
  • 92
  • 171
2

One hideous idea:

A std::map<std::string, std::string> m_otherKeyValuePairs; would be enough for the next 500 years.

Edit:

On the other hand, this suggestion is so prone to misuse that it may qualify for a TDWTF.

Another equally hideous idea:
a std::string m_everythingInAnXmlBlob;, as seen in real software.

(hideous == not recommended)

Edit 3:

  • Advantage:
    The std::map member is not subject to object slicing. When older source code copies an PluginInfo object that contains new keys in the property bag, the entire property bag is copied.
  • Disadvantage:
    many programmers will start adding unrelated things to the property bag, and even starts writing code that processes the values in the property bag, leading to maintenance nightmare.
rwong
  • 6,062
  • 1
  • 23
  • 51
  • of course any general property bag comes with a drawback: it's not strongly typed...(not my downvote btw) – Mitch Wheat Oct 14 '10 at 07:14
  • @Mitch: I edited to make it obvious that it's not recommended. It's as widely used as [big ball of mud](http://www.infoq.com/news/2010/09/big-ball-of-mud), though. – rwong Oct 14 '10 at 07:26
2

what rwong suggest (std::map<std::string, std::string>) is a good direction. This is makes it possible to add deliberate string fields. If you want to have more flexibility you might declare an abstract base class

class AbstractPluginInfoElement { public: virtual std::string toString() = 0;};

and

class StringPluginInfoElement : public AbstractPluginInfoElement 
{ 
  std::string m_value;
  public: 
   StringPluginInfoElement (std::string value) { m_value = value; }
   virtual std::string toString() { return m_value;}
};

You might then derive more complex classes like PluginVersion etc. and store a map<std::string, AbstractPluginInfoElement*>.

Philipp
  • 11,549
  • 8
  • 66
  • 126
2

As was said by someone else, for binary compatibility you will most likely restrict yourself to a C API.

The Windows API in many places maintains binary compatibility by putting a size member into the struct:

struct PluginInfo
{
    std::size_t size; // should be sizeof(PluginInfo)

    const char* s_Author;
    const char* s_Process;
    const char* s_ReleaseDate;
    //And so on...

    struct PluginVersion
    {
        const char* s_MajorVersion;
        const char* s_MinorVersion;
        //And so on...
    };
    PluginVersion o_Version;
};

When you create such a beast, you need to set the size member accordingly:

PluginInfo pluginInfo;
pluginInfo.size = sizeof(pluginInfo);
// set other members

When you compile your code against a newer version of the API, where the struct has additional members, its size changes, and that is noted in its size member. The API functions, when being passed such a struct presumably will first read its size member and branch into different ways to handle the struct, depending on its size.

Of course, this assumes that evolution is linear and new data is always only added at the end of the struct. That is, you will never have different versions of such a type that have the same size.


However, using such a beast is a nice way of ensuring that user introduce errors into their code. When they re-compile their code against a new API, sizeof(pluginInfo) will automatically adapt, but the additional members won't be set automatically. A reasonably safety would be gained by "initializing" the struct the C way:

PluginInfo pluginInfo;
std::memset( &pluginInfo, 0, sizeof(pluginInfo) );
pluginInfo.size = sizeof(pluginInfo);

However, even putting aside the fact that, technically, zeroing memory might not put a reasonable value into each member (for example, there could be architectures where all bits set to zero is not a valid value for floating point types), this is annoying and error-prone because it requires three-step construction.

A way out would be to design a small and inlined C++ wrapper around that C API. Something like:

class CPPPluginInfo : PluginInfo {
public:
  CPPPluginInfo()
   : PluginInfo() // initializes all values to 0
  {
    size = sizeof(PluginInfo);
  }

  CPPPluginInfo(const char* author /* other data */)
   : PluginInfo() // initializes all values to 0
  {
    size = sizeof(PluginInfo);
    s_Author = author;
    // set other data
  }
};

The class could even take care of storing the strings pointed to by the C struct's members in a buffer, so that users of the class wouldn't even have to worry about that.


Edit: Since it seems this isn't as clear-cut as I thought it is, here's an example.
Suppose that very same struct will in a later version of the API get some additional member:

struct PluginInfo
{
    std::size_t size; // should be sizeof(PluginInfo)

    const char* s_Author;
    const char* s_Process;
    const char* s_ReleaseDate;
    //And so on...

    struct PluginVersion
    {
        const char* s_MajorVersion;
        const char* s_MinorVersion;
        //And so on...
    };
    PluginVersion o_Version;

    int fancy_API_version2_member;
};

When a plugin linked to the old version of the API now initializes its struct like this

PluginInfo pluginInfo;
pluginInfo.size = sizeof(pluginInfo);
// set other members

its struct will be the old version, missing the new and shiny data member from version 2 of the API. If it now calls a function of the second API accepting a pointer to PluginInfo, it will pass the address of an old PluginInfo, short one data member, to the new API's function. However, for the version 2 API function, pluginInfo->size will be smaller than sizeof(PluginInfo), so it will be able catch that, and treat the pointer as pointing to an object that doesn't have the fancy_API_version2_member. (Presumably, internal of the host app's API, PluginInfo is the new and shiny one with the fancy_API_version2_member, and PluginInfoVersion1 is the new name of the old type. So all the new API needs to do is to cast the PluginInfo* it got handed be the plugin into a PluginInfoVersion1* and branch off to code that can deal with that dusty old thing.)

The other way around would be a plugin compiled against the new version of the API, where PluginInfo contains the fancy_API_version2_member, plugged into an older version of the host app that knows nothing about it. Again, the host app's API functions can catch that by checking whether pluginInfo->size is greater than the sizeof their own PluginInfo. If so, the plugin presumably was compiled against a newer version of the API than the host app knows about. (Or the plugin write failed to properly initialize the size member. See below for how to simplify dealing with this somewhat brittle scheme.)
There's two ways to deal with that: The simplest is to just refuse to load the plugin. Or, if possible, the host app could work with this anyhow, simply ignoring the binary stuff at the end of the PluginInfo object it was passed which it doesn't know how to interpret.
However, the latter is tricky, since you need to decide this when you implement the old API, without knowing exactly what the new API will look like.

sbi
  • 219,715
  • 46
  • 258
  • 445
  • Does this mean that the compiler guarantees that the variables will be placed in memory in the same layout as the code? – nakiya Oct 14 '10 at 10:04
  • And I am pretty sure that evolution is linear. – nakiya Oct 14 '10 at 10:05
  • Thanks for the lengthy reply. But I think you may have misinterpreted my requirements. Suppose in the first version we have header `H1.h`. This will be included in the application `A1` and shared object `D1`. The header will declare `PluginInfo1` struct with basic info. In the second version, `A` will have headers `H1.h` and `H2.h`. `H2.h` will declare `PluginInfo2` which adds another member. `A` will load `D2` which uses `PluginInfo2` from `H2.h`. And, `A` will also have to load `D1`. There will be a single function to retrieve `PluginInfo` from shared object. Which means... – nakiya Oct 14 '10 at 10:17
  • ... what matters is memory layout. If `PluginInfo2` only adds a member to the end of `PluginInfo1` in memory, `A` can work with it properly I think. I maybe wrong though. – nakiya Oct 14 '10 at 10:19
  • Found what I was looking for : http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html says : `adding optional items to the ends of structures is okay, as long as those structures are only allocated within the library`. – nakiya Oct 14 '10 at 11:22
  • @nakiya: This is designed for a scenario, where the `struct`'s name never changes (no `PluginInfo2`), but newer versions of the API simply append data to it. The functions taking this data first look at its `size` member from which they find out the data's version and then treat it according to that. Older plugins can load in newer host apps because the host app supplies reasonable defaults for the newer `struct` members that the old plugins didn't know about. (If you want, newer plugins can even load in older host apps, if the host app just ignores the additional data it doesn't know about.) – sbi Oct 14 '10 at 12:27
  • This system allows such a `struct` to be allocated by the user. (Win32 indeed does that.) – sbi Oct 14 '10 at 12:29
  • @sbi: Can you please explain a little more? Perhaps with an example? thanks. – nakiya Oct 15 '10 at 12:01
  • @sbi:`adding optional items to the ends of structures is okay, as long as those structures are only allocated within the library` I thought what this means is, if a shared lib has `PluginInfo2`, and a newer version of the app knows `PluginInfo2` too, the app can load the lib correctly in addition to the fact that it can load previous libraries - care has to be taken not to access fields which weren't in previous libs - and load even future libraries because it will only be using a subset of the fields in `PluginInfo3`. Am I wrong? – nakiya Oct 15 '10 at 12:05
  • @nakiya: I've tried to add a more thorough explanation. To be honest, I'm not sure what's so hard about this. You might want to explain this, so I can address it properly. – sbi Oct 16 '10 at 21:38
  • Or is it that you're used to dealing with languages that feature reflection? C++ has next to nothing of that. When you pass a pointer to some data to another function, that function interprets the binary data found at the address using the code the compiler has generated from its source. If the caller and callee were compiled using different versions of the data, then neither of them will detect that. Officially this invokes undefined behavior, although I suppose you can get away with a scheme like this on maybe every platform. – sbi Oct 16 '10 at 21:39
  • I think your answer is right. I'm going in circles :). Thanks for the lengthy explanation. – nakiya Oct 17 '10 at 18:22
  • @sbi: I totally read it just now. Yesterday I was just skimming. I don't need inter-plugin calls. Plugins only work with the app so only the app has to know how to handle a `PuginInfo` which makes my life easier. Your answer even considers plugin-plugin connections which is nice!!! – nakiya Oct 18 '10 at 05:06
1

Here's an idea, not sure whether it works with classes, it for sure works with structs: You can make the struct "reserve" some space to be used in the future like this:

struct Foo
{
  // Instance variables here.
  int bar;

  char _reserved[128]; // Make the class 128 bytes bigger.
}

An initializer would zero out whole struct before filling it, so newer versions of the class which would access fields that would now be within the "reserved" area are of sane default values.

If you only add fields in front of _reserved, reducing its size accordingly, and not modify/rearrange other fields you should be OK. No need for any magic. Older software will not touch the new fields as they don't know about them, and the memory footprint will remain the same.

DarkDust
  • 90,870
  • 19
  • 190
  • 224
  • One drawback I can think of is having to know the `sizeof()` for each data type used. But still this is good I think. – nakiya Oct 14 '10 at 08:30