6

I have code similar to the following code

boost::thread myThread
unsigned char readbuffer[bignumber];
unsigned char writebuffer[bignumber];

for(int i=0; i<bignumber; ++i){
  functiondostuff();
  for(int j=0; j<2; ++j){
    functiondomorestuff();
    myThread = boost::thread(&myClass::myFunction, this, j, i);
  }     
}

myFunction reads from a buffer and writes to another. It will never write to the same location in the write buffer. Am I doing something fundamentally wrong with threads here? Is it bad to loop over a thread creation with the same thread name? It runs smooth for a while and then I get the following exception.

terminate called after throwing an instance of 'boost::exception_detail::clone_impl >' what(): boost::thread_resource_error: Resource temporarily unavailable Aborted

What does this exception mean? Any ideas would be helpful.

deanresin
  • 1,466
  • 2
  • 16
  • 31

2 Answers2

10

There's a limit on the number of threads you can create per process.

On linux, for example,

cat /proc/sys/kernel/threads-max

tells you the current maximum. The default is the number of memory pages/4, so on my system it's 513785, but it may be much much lower on another box. E.g. on my mail server box (512mb RAM) it's only 7295.

You could the limit. But in fact that will be useless because the OS can't schedule them effectively. So, instead, try using a thread pool.

Oh. PS. detach()-ing he threads will help (a lot) with conserving resources. pthreads might be blocking thread creation well before the OS limit is reached because it needs to allocate overhead tracking the active threads. detach frees those up (and removes the error of not joining all threads before program exit).

UPDATE Crazy friday bonus: a thread pool that auto-scales to the number of cores your system has:

#include <boost/thread.hpp>
#include <boost/phoenix.hpp>
#include <boost/optional.hpp>

using namespace boost;
using namespace boost::phoenix::arg_names;

boost::atomic_size_t counter(0ul);

class thread_pool
{
  private:
      mutex mx;
      condition_variable cv;

      typedef function<void()> job_t;
      std::deque<job_t> _queue;

      thread_group pool;

      boost::atomic_bool shutdown;
      static void worker_thread(thread_pool& q)
      {
          while (auto job = q.dequeue())
              (*job)();
      }

  public:
      thread_pool() : shutdown(false) {
          for (unsigned i = 0; i < boost::thread::hardware_concurrency(); ++i)
              pool.create_thread(bind(worker_thread, ref(*this)));
      }

      void enqueue(job_t job) 
      {
          lock_guard<mutex> lk(mx);
          _queue.push_back(std::move(job));

          cv.notify_one();
      }

      optional<job_t> dequeue() 
      {
          unique_lock<mutex> lk(mx);
          namespace phx = boost::phoenix;

          cv.wait(lk, phx::ref(shutdown) || !phx::empty(phx::ref(_queue)));

          if (_queue.empty())
              return none;

          auto job = std::move(_queue.front());
          _queue.pop_front();

          return std::move(job);
      }

      ~thread_pool()
      {
          shutdown = true;
          {
              lock_guard<mutex> lk(mx);
              cv.notify_all();
          }

          pool.join_all();
      }
};

static constexpr size_t bignumber = 1 << 20;

class myClass 
{
    //unsigned char readbuffer[bignumber];
    //unsigned char writebuffer[bignumber];
    void functiondostuff() { }
    void functiondomorestuff() { }

    thread_pool pool; // uses 1 thread per core

  public:
    void wreak_havoc()
    {
        std::cout << "enqueuing jobs... " << std::flush;
        for(size_t i=0; i<bignumber; ++i)
        {
            functiondostuff();
            for(int j=0; j<2; ++j) {
                functiondomorestuff();
                pool.enqueue(bind(&myClass::myFunction, this, j, i));
            }     
        }
        std::cout << "done\n";
    }

  private:
    void myFunction(int i, int j)
    {
        boost::this_thread::sleep_for(boost::chrono::milliseconds(1));
        counter += 1;
    }
};

int main()
{
    myClass instance;
    instance.wreak_havoc();

    size_t last = 0;
    while (counter < (2*bignumber))
    {
        boost::this_thread::sleep_for(boost::chrono::milliseconds(100));
        if ((counter >> 4u) > last)
        {
            std::cout << "Progress: " << counter << "/" << (bignumber*2) << "\n";
            last = counter >> 4u;
        }
    }
}
sehe
  • 374,641
  • 47
  • 450
  • 633
  • Is the limit: total threads created at one time? Because I never have more than 10 threads open at one time and it is usually just 1-3. Also, I don't get the exception if I put a myThread.join() right after my thread creation. – deanresin Mar 21 '14 at 22:34
  • 1
    @deanresin in your sample, clearly you are creating `bignumer*2` threads **at the same time** ("if you don't `join` them, they beat you" :)). The fact that you assign them to the same variable (and then forget about that) doesn't make it different. Anyways, I'm adding a `thread_pool` approach, because it's crazy Friday :) – sehe Mar 21 '14 at 22:43
  • But when the thread code ends doesn't it disappear? My threads are very short pieces of code. There life is short. Tx for your code. I will use it for reference later. – deanresin Mar 21 '14 at 22:48
  • 1
    @deanresin Why don't you /read/ the answer? No they don't completely disappear. You need to `join` or `detach` at the very least, or they /will/ keep consuming resources. Regardless, just creating threads willy-nilly is not going to result in good performance. If your application is IO-bound, you might do 10k connections/s on a single thread (look at Boost Asio). For these situations, learn that asynchrony != concurrency. – sehe Mar 21 '14 at 22:50
  • So even though the thread code ends it still consumes resources? I have to explicitly tell c++ that I'm done with the thread? If I have created (in the inner loop) two threads with the same name then how can I reference each one individually to join them? Is that part of my problem? tx. – deanresin Mar 21 '14 at 22:55
  • 1
    You **did not** "create two threads with the same name". Threads don't have names (yes, some libraries add extensions for that). Threads are just that. Threads. Did you learn about pointers? It's a bit the same. Your thread variable just "points" to a thread resource (actually, it abstracts the platform specific handle type for it). – sehe Mar 21 '14 at 22:57
  • Detach worked. Thanks so much. That was my problem. I create the thread and immediately detach it. – deanresin Mar 21 '14 at 22:57
  • @deanresin and. Please. Start ***reading*** instead of forcing ahead on a dead end. The 90% biggest part of your problem is _thread creation running wild_. It's not productive and you didn't even realize you're doing it. Don't "fix it" by "somewhat unbreaking the implementation". Rethink the approach. – sehe Mar 21 '14 at 22:58
  • One last question. If I create two threads "myThread" how do I join them to main? Will myThread.join() join both of them? – deanresin Mar 21 '14 at 22:59
  • I'm giving up. You don't read. So you don't learn. **No!** `myThread.join()` will (obviously) not join more than 1 thread. When you do `int x = 3; x = 5;` how many integers do you multiply when you write `x*2`? Right. One. Just the last assigned value of course. Pick up a good book, slow down. And keep learning. Thank you. – sehe Mar 21 '14 at 23:02
  • @deanresin If you have two kids "Johnnie" and you kill "Johnnie" will both of them die? – StackedCrooked Mar 21 '14 at 23:06
  • 2
    @StackedCrooked Well. The analogy is broken, because the kids would ***actually*** have the same name. The threads have no such common feature. They just happen to have been assigned to the same `thread` variable once (but never at the same time). So, this is more like _"If you kill a boy (that you met in a particular house), will all living boys that have ever been in the same house as that boy also be killed"_ ? – sehe Mar 21 '14 at 23:08
  • I read the documentation as you suggested and better solved my problem. I didn't know how to retain unique handles to the threads created in the inner loop. My program was crashing because I would lose control of my threads because I didn't have unique handles for them. I added them into a boost::group and used a myGroup.join_all(). I suppose I also could have used a vector to push the handles onto and then popped them off to join them all. Thanks again so much for your help. – deanresin Mar 22 '14 at 06:05
  • @StackedCrooked you are working off the assumption that I knew there were two kids named Johnnie. I knew one of them was called Johnnie. I didn't know how to kill the other one because I didn't know its' name. – deanresin Mar 22 '14 at 06:31
  • 1
    Exactly. I'd say you've got it. Don't replace thread_group by vector though, since you **will** get exception safety wrong. Also, RAII and `join_all` should each be enough reason in itself to use the group. Conceptually, I agree though: is just a container of threads. – sehe Mar 22 '14 at 08:15
-1

First, I wasn't explicitly killing off my threads with thread.join() or thread.detach() so the total number of threads would grow out of control and throw an exception.

Also, in my inner loop I was creating two threads but keeping only one handle. So I lost control of 1 thread each iteration of the outer loop causing the total number of threads to grow past the prescribed limit of threads causing the exception.

To retain unique handles to the threads created in the inner loop I added them into a boost::group and used a myGroup.join_all(). I suppose I also could have used a vector to push the handles onto and then popped them off to join them all.

boost::thread_group myThreadGroup
unsigned char readbuffer[bignumber];
unsigned char writebuffer[bignumber];

for(int i=0; i<bignumber; ++i){
  functiondostuff();
  for(int j=0; j<2; ++j){
    functiondomorestuff();
    myThreadGroup.create_thread(boost::bind(&myClass::myFunction, this, i, j));
  }    
  functiondostuffagain();
  myThreadGroup.join_all(); 
}
deanresin
  • 1,466
  • 2
  • 16
  • 31
  • 2
    You're still creating threads willy nilly. On my box this still throws `what(): boost::thread_resource_error: Resource temporarily unavailable` (that's the box with 32GiB of RAM and 513785 thread limit!). And that's with empty worker threads. Demonstrating the problem with smaller `big_number` on Coliru: http://coliru.stacked-crooked.com/a/b24d82692781e505 – sehe Mar 22 '14 at 09:04
  • Adding a `->detach()` "helps" somewhat, but threads still run wild. E.g. running 'bignumber=1048576` on my heavy machine takes 43.8s, and runs an average of ~80 threads, peaking at 170 threads at one time. This is just horrifically bad. And it's waiting until the system slows for a few milliseconds, and you'll run head-on into that same "resource unavailable" brick wall. E.g. **[Coliru is a fraction slower](http://coliru.stacked-crooked.com/a/d6fd11158cec1de2)**... that's ***with*** `->detach()` and `.join_all(), see? – sehe Mar 22 '14 at 09:10
  • 1
    Maybe now you can see why my answer avoids creating unbounded threads in the first place (and it is has progress reporting! You can abort the work! Yould make different priorities. You could add more threads on the fly!). With empty jobs (`myFunction`) it turns out much slower, I'm guessing it's in the locking on the queue. Lemme fix that too: raising the size of the thread pool to `8*hardware_concurrency()` mad it **faster** already (35s). Now, obviously, removing locking altogether is superior: **[lockfree on Coliru](http://coliru.stacked-crooked.com/a/7aa77f1c662de90a)** – sehe Mar 22 '14 at 10:25
  • Now, you'll see that I use more threads than there are hardware cores there. This **only** works because the jobs do sleeps. If they don't, having more threads actually hurts performance (a lot): [**>5x slower** _(5s)_ with more threads](http://coliru.stacked-crooked.com/a/f63b998598374b37) than [when using **one thread per core** _(0.9s)_](http://coliru.stacked-crooked.com/a/74fd04ce4c7a76ee) – sehe Mar 22 '14 at 10:31
  • I fixed the offending code. I put the join_all() in the wrong place. – deanresin Mar 22 '14 at 17:34
  • Much better. Or should I say, less careless. But now your code is pretty much pointless, as you said ["My threads are very short pieces of code. There _(sic)_ life is short." (21h ago)](http://stackoverflow.com/questions/22569805/boost-thread-throwing-exception-thread-resource-error-resource-temporarily-una/22574823?noredirect=1#comment34357021_22570554). If that's the case then pretty surely the overhead of creating a new thread every single time (2*bignumber times, actually) + synchronization is going to far outweigh the potential gain of parallelizing a section of code :( – sehe Mar 22 '14 at 20:07
  • Are you doing actual IO in that section? In that case, use an ASIO `io_service` to multiplex it on a fixed (number of) thread(s). As long as you create threads in numbers correlated to `bignumber` you're as good as toast. I mean, [scaling parallellism is an acute art.](http://stackoverflow.com/questions/22569805/shorter/22574823?noredirect=1#comment34366446_22574823) Did you actually measure throughputs? (My bet is **a** either you didn't or **b** you did and your claim that the threads are /very shortlived/ is not very accurate.) – sehe Mar 22 '14 at 20:10
  • This is just example code. I have built a JPG decoder (for fun and to learn) and now I'm playing around with threads to learn. I have quickly learned that threads create a huge amount of overhead as my program is now 45 times slower. However, I'm forcing the threads on pre-existing code just to play around. Using threads, I have successfully got my decoder working so right now I'm really happy with that. Onwards and upwards. – deanresin Mar 22 '14 at 20:26
  • Thanks for putting stuff in perspective. I really hope you do give the thread pool based solutions I showed due attention, because frankly it would be a shame to just conclude "threads are hard and scaling is likely not to happen". As you can see going from locking to lockfree job queuing increased speed (of my synthetic benchmark) by ~40x. It's small tweaks that make the difference. You have to have played with these in order to be able to see which approach is right in a real life challenge. Keep learning :) Cheers – sehe Mar 22 '14 at 20:31
  • yeah thanks again for your code. I'll be checking it out soon. :) – deanresin Mar 22 '14 at 23:41