1

i am implmenting an event-driven message processing logic for a speed-sensitive application. I have various business logics which wrapped into a lot of Reactor classes:

class TwitterSentimentReactor{
    on_new_post(PostEvent&);
    on_new_comment(CommentEvent&);
};

class FacebookSentimentReactor{
    on_new_post(PostEvent&);
    on_new_comment(CommentEvent&);
};

class YoutubeSentimentReactor{
    on_new_post(PostEvent&);
    on_new_comment(CommentEvent&);
    on_new_plus_one(PlusOneEvent&);
};

let's say, there are 8 such event types, each Reactor respond to a subset of them.

the core program has 8 'entry point' for the message, which hooked up with some low-level socket processing library, for instance

on_new_post(PostEvent& pe){
    youtube_sentiment_reactor_instance->on_new_post(pe);
    twitter_sentiment_reactor_instance->on_new_post(pe);
    youtube_sentiment_reactor_instance->on_new_post(pe);
}

I am thinking about using std::function and std::bind, to build a std::vector<std::function<>>, then I loop through the vector to call each call-back function.

However, when I tried it,std::function proved to not be fast enough. Is there a fast yet simple solution here? As i mentioned earlier, this is VERY speed sensitive, so i want to avoid using virtual function and inheritance, to cut the v-table look up

comments are welcomed. thanks

R. Martinho Fernandes
  • 228,013
  • 71
  • 433
  • 510
James Bond
  • 7,533
  • 19
  • 50
  • 64
  • 1
    Maybe [this](https://testbit.eu/2013/cpp11-signal-system-performance/) blog post regarding the performance comparison of different C++ signaling implementations is of help here. – Sebastian Dec 29 '13 at 17:36
  • What platform? How portable need it be? Do the callbacks have to manage lifetime? How many millions of times per second do these callbacks get called? – Yakk - Adam Nevraumont Dec 29 '13 at 17:51
  • Since this is so speed sensitive I suppose you made a series of typos and wrote "I read on the Internet" when you meant "I tried it and it was not good enough". – R. Martinho Fernandes Dec 29 '13 at 18:09
  • Consider accepting an answer now, also note that since you are using sockets (ways slower than anythinig) the callback overhead is neglictible (unless you are using Qt slots :P). Again I found after testing that you don't really need anything faster than std::function (in that case my framework is convenient because of C# like syntax) or Offirmo. The reason is instruction cache. The cpu will wait cycles that the code for the actual callback is fetched from cache. The greatest optimization you could do is actually compiling the callbacks in the order in wich are called(or the reverse under GCC) – CoffeDeveloper Aug 06 '14 at 10:29

6 Answers6

2

I think that in your case it is easier to do an interface, as you know are going to call simple member functions that match exactly the expected parameters:

struct IReactor {
    virtual void on_new_post(PostEvent&) =0;
    virtual void on_new_comment(CommentEvent&) =0;
    virtual void on_new_plus_one(PlusOneEvent&) =0;
};

And then make each of your classes inherit and implement this interface.

You can have a simple std::vector<IReactor*> to manage the callbacks.

And remember that in C++, interfaces are just ordinary classes, so you can even write default implementations for some or all of the functions:

struct IReactor {
    virtual void on_new_post(PostEvent&) {}
    virtual void on_new_comment(CommentEvent&) {}
    virtual void on_new_plus_one(PlusOneEvent&) {}
};
rodrigo
  • 94,151
  • 12
  • 143
  • 190
  • thanks, but to improve the spead, i try to avoid using inheritance, to cut the v-table look up :-( – James Bond Dec 29 '13 at 17:43
  • 1
    @ccfenix: I think that you are over-optimizing there. You cannot have polimorphism without _at lease_ one level of indirection, and that's what virtual functions need. The myth of virtual funcions being slow only makes sense in tight loops of numerical computation, and that doesn't seem the case. – rodrigo Dec 29 '13 at 17:47
  • You can't get away without inheritance in this case, essentially. – Puppy Dec 29 '13 at 19:10
2

std::function main performance issue is that whenever you need to store some context (such as bound arguments, or the state of a lambda) then memory is required which often translates into a memory allocation. Also, the current library implementations that exist may not have been optimized to avoid this memory allocation.

That being said:

  • is it too slow ? you will have to measure it for yourself, in your context
  • are there alternatives ? yes, plenty!

As an example, what don't you use a base class Reactor which has all the required callbacks defined (doing nothing by default), and then derive from it to implement the required behavior ? You could then easily have a std::vector<std::unique_ptr<Reactor>> to iterate over!

Also, depending on whether the reactors need state (or not) you may gain a lot by avoiding allocating objects from then and use just functions instead.

It really, really, depends on the specific constraints of your projects.

Matthieu M.
  • 287,565
  • 48
  • 449
  • 722
  • Every `std::function` implementation I have poked at does pImpl based type erasure that routes the calls through a `virtual` interface, so it has (at least slightly) more overhead than a `virtual` interface (possibly less than some, because the table is simple). This is *independent* of if the `std::function` has *context* or not. – Yakk - Adam Nevraumont Dec 29 '13 at 19:08
1

If you need fast delegates and event system take a look to Offirmo: It is as fast as the "Fastest possible delegates", but it has 2 major advantages:

1) it is ready and well tested library (don't need to write your own library from an article)

2) Does not relies on compiler hacks (fully compliant to C++ standard)

https://github.com/Offirmo/impossibly-fast-delegates

If you need a managed signal/slot system I have developed my own(c++11 only).

It is not fast as Offirmo, but is fast enough for any real scenario, most important is order of magnitude faster than Qt or Boost signals and is simple to use.

  • Signal is responsible for firing events.
  • Slots are responsible for holding callbacks.
  • Connect how many Slots as you wish to a Signal.
  • Don't warry about lifetime (everything autodisconnect)

Performance considerations:

The overhead for a std::function is quite low (and improving with every compiler release). Actually is just a bit slower than a regular function call. My own signal/slot library, is capable of 250 millions(I measured the pure overhead) callbacks/second on a 2Ghz processor and is using std::function.

Since your code has to do with network stuff you should mind that your main bottleneck will be the sockets.

The second bottleneck is latency of instruction cache. It does not matter if you use Offirmo (few assembly instructions), or std::function. Most of the time is spent by fetchin instructions from L1 cache. The best optimization is to keep all callbacks code compiled in the same translation unit (same .cpp file) and possibly in the same order in wich callbacks are called (or mostly the same order), after you do that you'll see only a very tiny improvement using Offirmo (seriously, you CAN'T BE faster than Offirmo) over std::function.

Keep in mind that any function doing something really usefull would be at least few dozens instructions (especially if dealing with sockets: you'll have to wait completion of system calls and processor context switch..) so the overhead of the callback system will be neglictible.

CoffeDeveloper
  • 7,961
  • 3
  • 35
  • 69
0

I can't comment on the actual speed of the method that you are using, other than to say:

  1. Premature optimization does not usually give you what you expect.
  2. You should measure the performance contribution before you start slicing and dicing. If you know it won't work before hand, then you can search now for something better or go "suboptimal" for now but encapsulate it so it can be replaced.

If you are looking for a general event system that does not use std::function (but does use virtual methods), you can try this one:

Notifier.h

/* 
 The Notifier is a singleton implementation of the Subject/Observer design
 pattern.  Any class/instance which wishes to participate as an observer
 of an event can derive from the Notified base class and register itself
 with the Notiifer for enumerated events.

 Notifier derived classes implement variants of the Notify function:

 bool Notify(const NOTIFIED_EVENT_TYPE_T& event, variants ....)

 There are many variants possible.  Register for the message 
 and create the interface to receive the data you expect from 
 it (for type safety).  

 All the variants return true if they process the event, and false
 if they do not.  Returning false will be considered an exception/
 assertion condition in debug builds.

 Classes derived from Notified do not need to deregister (though it may 
 be a good idea to do so) as the base class destrctor will attempt to
 remove itself from the Notifier system automatically.

 The event type is an enumeration and not a string as it is in many 
 "generic" notification systems.  In practical use, this is for a closed
 application where the messages will be known at compile time.  This allows
 us to increase the speed of the delivery by NOT having a 
 dictionary keyed lookup mechanism.  Some loss of generality is implied 
 by this.

 This class/system is NOT thread safe, but could be made so with some
 mutex wrappers.  It is safe to call Attach/Detach as a consequence 
 of calling Notify(...).  

 */


/* This is the base class for anything that can receive notifications.
 */

typedef enum
{
   NE_MIN = 0,
   NE_SETTINGS_CHANGED,
   NE_UPDATE_COUNTDOWN,
   NE_UDPATE_MESSAGE,
   NE_RESTORE_FROM_BACKGROUND,
   NE_MAX,
} NOTIFIED_EVENT_TYPE_T;

class Notified
{
public:
   virtual bool Notify(NOTIFIED_EVENT_TYPE_T eventType, const uint32& value)
   { return false; };
   virtual bool Notify(NOTIFIED_EVENT_TYPE_T eventType, const bool& value)
   { return false; };
   virtual bool Notify(NOTIFIED_EVENT_TYPE_T eventType, const string& value)
   { return false; };
   virtual bool Notify(NOTIFIED_EVENT_TYPE_T eventType, const double& value)
   { return false; };
   virtual ~Notified();   
};

class Notifier : public SingletonDynamic<Notifier>
{
public:

private:
   typedef vector<NOTIFIED_EVENT_TYPE_T> NOTIFIED_EVENT_TYPE_VECTOR_T;

   typedef map<Notified*,NOTIFIED_EVENT_TYPE_VECTOR_T> NOTIFIED_MAP_T;
   typedef map<Notified*,NOTIFIED_EVENT_TYPE_VECTOR_T>::iterator NOTIFIED_MAP_ITER_T;

   typedef vector<Notified*> NOTIFIED_VECTOR_T;
   typedef vector<NOTIFIED_VECTOR_T> NOTIFIED_VECTOR_VECTOR_T;

   NOTIFIED_MAP_T _notifiedMap;
   NOTIFIED_VECTOR_VECTOR_T _notifiedVector;
   NOTIFIED_MAP_ITER_T _mapIter;


   // This vector keeps a temporary list of observers that have completely
   // detached since the current "Notify(...)" operation began.  This is
   // to handle the problem where a Notified instance has called Detach(...)
   // because of a Notify(...) call.  The removed instance could be a dead
   // pointer, so don't try to talk to it.
   vector<Notified*> _detached;
   int32 _notifyDepth;

   void RemoveEvent(NOTIFIED_EVENT_TYPE_VECTOR_T& orgEventTypes, NOTIFIED_EVENT_TYPE_T eventType);
   void RemoveNotified(NOTIFIED_VECTOR_T& orgNotified, Notified* observer);

public:

   virtual void Reset();
   virtual bool Init() { Reset(); return true; }
   virtual void Shutdown() { Reset(); }

   void Attach(Notified* observer, NOTIFIED_EVENT_TYPE_T eventType);
   // Detach for a specific event
   void Detach(Notified* observer, NOTIFIED_EVENT_TYPE_T eventType);
   // Detach for ALL events
   void Detach(Notified* observer);

   // This template function (defined in the header file) allows you to
   // add interfaces to Notified easily and call them as needed.  Variants
   // will be generated at compile time by this template.
   template <typename T>
   bool Notify(NOTIFIED_EVENT_TYPE_T eventType, const T& value)
   {
      if(eventType < NE_MIN || eventType >= NE_MAX)
      {
         throw std::out_of_range("eventType out of range");
      }

      // Keep a copy of the list.  If it changes while iterating over it because of a
      // deletion, we may miss an object to update.  Instead, we keep track of Detach(...)
      // calls during the Notify(...) cycle and ignore anything detached because it may
      // have been deleted.
      NOTIFIED_VECTOR_T notified = _notifiedVector[eventType];

      // If a call to Notify leads to a call to Notify, we need to keep track of
      // the depth so that we can clear the detached list when we get to the end
      // of the chain of Notify calls.
      _notifyDepth++;

      // Loop over all the observers for this event.
      // NOTE that the the size of the notified vector may change if
      // a call to Notify(...) adds/removes observers.  This should not be a
      // problem because the list is a simple vector.
      bool result = true;
      for(int idx = 0; idx < notified.size(); idx++)
      {
         Notified* observer = notified[idx];
         if(_detached.size() > 0)
         {  // Instead of doing the search for all cases, let's try to speed it up a little
            // by only doing the search if more than one observer dropped off during the call.
            // This may be overkill or unnecessary optimization.
            switch(_detached.size())
            {
               case 0:
                  break;
               case 1:
                  if(_detached[0] == observer)
                     continue;
                  break;
               default:
                  if(std::find(_detached.begin(), _detached.end(), observer) != _detached.end())
                     continue;
                  break;
            }
         }
         result = result && observer->Notify(eventType,value);
         assert(result == true);
      }
      // Decrement this each time we exit.
      _notifyDepth--;
      if(_notifyDepth == 0 && _detached.size() > 0)
      {  // We reached the end of the Notify call chain.  Remove the temporary list
         // of anything that detached while we were Notifying.
         _detached.clear();
      }
      assert(_notifyDepth >= 0);
      return result;
   }
   /* Used for CPPUnit.  Could create a Mock...maybe...but this seems
    * like it will get the job done with minimal fuss.  For now.
    */
   // Return all events that this object is registered for.
   vector<NOTIFIED_EVENT_TYPE_T> GetEvents(Notified* observer);
   // Return all objects registered for this event.
   vector<Notified*> GetNotified(NOTIFIED_EVENT_TYPE_T event);
};

Notifier.cpp

#include "Notifier.h"

void Notifier::Reset()
{
   _notifiedMap.clear();
   _notifiedVector.clear();
   _notifiedVector.resize(NE_MAX);
   _detached.clear();
   _notifyDepth = 0;
}

void Notifier::Attach(Notified* observer, NOTIFIED_EVENT_TYPE_T eventType)
{
   if(observer == NULL)
   {
      throw std::out_of_range("observer == NULL");
   }
   if(eventType < NE_MIN || eventType >= NE_MAX)
   {
      throw std::out_of_range("eventType out of range");
   }

   _mapIter = _notifiedMap.find(observer);
   if(_mapIter == _notifiedMap.end())
   {  // Registering for the first time.
      NOTIFIED_EVENT_TYPE_VECTOR_T eventTypes;
      eventTypes.push_back(eventType);
      // Register it with this observer.
      _notifiedMap[observer] = eventTypes;
      // Register the observer for this type of event.
      _notifiedVector[eventType].push_back(observer);
   }
   else
   {
      NOTIFIED_EVENT_TYPE_VECTOR_T& events = _mapIter->second;
      bool found = false;
      for(int idx = 0; idx < events.size() && !found; idx++)
      {
         if(events[idx] == eventType)
         {
            found = true;
            break;
         }
      }
      if(!found)
      {
         events.push_back(eventType);
         _notifiedVector[eventType].push_back(observer);
      }
   }
}

void Notifier::RemoveEvent(NOTIFIED_EVENT_TYPE_VECTOR_T& eventTypes, NOTIFIED_EVENT_TYPE_T eventType)
{
   int foundAt = -1;

   for(int idx = 0; idx < eventTypes.size(); idx++)
   {
      if(eventTypes[idx] == eventType)
      {
         foundAt = idx;
         break;
      }
   }
   if(foundAt >= 0)
   {
      eventTypes.erase(eventTypes.begin()+foundAt);
   }
}

void Notifier::RemoveNotified(NOTIFIED_VECTOR_T& notified, Notified* observer)
{
   int foundAt = -1;

   for(int idx = 0; idx < notified.size(); idx++)
   {
      if(notified[idx] == observer)
      {
         foundAt = idx;
         break;
      }
   }
   if(foundAt >= 0)
   {
      notified.erase(notified.begin()+foundAt);
   }
}


void Notifier::Detach(Notified* observer, NOTIFIED_EVENT_TYPE_T eventType)
{
   if(observer == NULL)
   {
      throw std::out_of_range("observer == NULL");
   }
   if(eventType < NE_MIN || eventType >= NE_MAX)
   {
      throw std::out_of_range("eventType out of range");
   }

   _mapIter = _notifiedMap.find(observer);
   if(_mapIter != _notifiedMap.end())
   {  // Was registered
      // Remove it from the map.
      RemoveEvent(_mapIter->second, eventType);
      // Remove it from the vector
      RemoveNotified(_notifiedVector[eventType], observer);
      // If there are no events left, remove this observer completely.
      if(_mapIter->second.size() == 0)
      {
         _notifiedMap.erase(_mapIter);
         // If this observer was being removed during a chain of operations,
         // cache them temporarily so we know the pointer is "dead".
         _detached.push_back(observer);
      }
   }
}

void Notifier::Detach(Notified* observer)
{
   if(observer == NULL)
   {
      throw std::out_of_range("observer == NULL");
   }

   _mapIter = _notifiedMap.find(observer);
   if(_mapIter != _notifiedMap.end())
   {
      // These are all the event types this observer was registered for.
      NOTIFIED_EVENT_TYPE_VECTOR_T& eventTypes = _mapIter->second;
      for(int idx = 0; idx < eventTypes.size();idx++)
      {
         NOTIFIED_EVENT_TYPE_T eventType = eventTypes[idx];
         // Remove this observer from the Notified list for this event type.
         RemoveNotified(_notifiedVector[eventType], observer);
      }
      _notifiedMap.erase(_mapIter);
   }
   // If this observer was being removed during a chain of operations,
   // cache them temporarily so we know the pointer is "dead".
   _detached.push_back(observer);
}


Notified::~Notified()
{
   Notifier::Instance().Detach(this);
}

// Return all events that this object is registered for.
vector<NOTIFIED_EVENT_TYPE_T> Notifier::GetEvents(Notified* observer)
{
   vector<NOTIFIED_EVENT_TYPE_T> result;

   _mapIter = _notifiedMap.find(observer);
   if(_mapIter != _notifiedMap.end())
   {
      // These are all the event types this observer was registered for.
      result = _mapIter->second;
   }

   return result;
}

// Return all objects registered for this event.
vector<Notified*> Notifier::GetNotified(NOTIFIED_EVENT_TYPE_T event)
{
   return _notifiedVector[event];
}

NOTES:

  1. You must call init() on the class before using it.
  2. You don't have to use it as a singleton, or use the singleton template I used here. That is just to get a reference/init/shutdown mechanism in place.
  3. This is from a larger code base. You can find some other examples on github here.
Community
  • 1
  • 1
FuzzyBunnySlippers
  • 3,387
  • 2
  • 18
  • 28
0

There was a topic on SO, where virtually all mechanisms available in C++ was enumerated, but can't find it.

It had a list something like this:

Fast delegates and boost::function performance comparison article: link

Oh, by the way, premature optimization..., profile first then optimize, 80/20-rule, blah-blah, blah-blah, you know ;)

Happy coding!

Ivan Aksamentov - Drop
  • 12,860
  • 3
  • 34
  • 61
0

Unless you can parameterize your handlers statically and get the inlined, std::function<...> is your best option. When type exact type needs to be erased or you need to call run-time specified function you'll have an indirection and, hence, an actual function call without the ability to get things inlined. std::function<...> does exactly this and you won't get better.

Dietmar Kühl
  • 150,225
  • 13
  • 225
  • 380