1

I have a little simple program to test wether I can visualize a point cloud from a different thread and continue working in the main thread until typing 'q' in the terminal.

In Ubuntu 10.04, the code works, letting me visualize the cloud as new points are added to it in each iteration. However, in Windows 7 this dosn't work (I'm compiling it with QtCreator). The cloud is shown and new points are computed in each turn, but this never exits. When typing 'q', the loop stops but the visualization thread keeps running. The only way to stop execution is to explicitly use CTRL+C.

More things:

  • If I don't uncomment the addPointCloud line before the !viewer->wasStopped() loop in the Visualize function, the point cloud is never shown. It doesn't matter that later in the loop I explicitly add it. It has to be done before the loop (now that line is commented to demonstrate that behaviour).
  • I also tried to use boost::mutex instead of *tbb::queuing_mutex*, but again, the program won't exit.

Do you have any idea why the thread is never joining?. Also, constructive critics about my thread usage are always welcomed, I want to keep improving.

Here's the code:

#include <boost/thread/thread.hpp>
#include <iostream>
#include <pcl/point_types.h>
#include <pcl/visualization/pcl_visualizer.h>
#include "tbb/queuing_mutex.h"

typedef pcl::PointXYZ PointType;
typedef pcl::PointCloud<PointType> PointCloudType;
typedef tbb::queuing_mutex MutexType;
//typedef boost::mutex MutexType;
MutexType safe_update;

const unsigned int HEIGHT = 100;
const unsigned int WIDTH = 100;
bool has_to_update(true);

void Visualize(PointCloudType::Ptr cloud) {
  pcl::visualization::PCLVisualizer* viewer = new pcl::visualization::PCLVisualizer("Vis in thread",true);
  viewer->setBackgroundColor(1.0,0.0,0.0);
//  viewer->addPointCloud<PointType>(cloud, "sample cloud");
  viewer->addCoordinateSystem(1.0);
  viewer->initCameraParameters();
  viewer->resetCamera();
  while(!viewer->wasStopped()) {
    viewer->spinOnce(100);
    {
//      boost::lock_guard<MutexType> lock(safe_update);
        MutexType::scoped_lock lock(safe_update);
      if(has_to_update) {
        if(!viewer->updatePointCloud<PointType>(cloud, "sample cloud")) {
          viewer->addPointCloud<PointType>(cloud, "sample cloud");
          viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 1, "sample cloud");
          viewer->resetCamera();
        }
        has_to_update = false;
      }
    } // end scoped_lock
  }
  delete viewer;
};

int main(int argc, char** argv) {
  PointCloudType::Ptr c(new PointCloudType);
  c->height=HEIGHT;
  c->width=WIDTH;
  const unsigned int size( c->height*c->width);
  c->points.resize(size);
  for(unsigned int i(0);i<size;++i){
    c->points[i].x = 1024 * rand () / (RAND_MAX + 1.0f);
    c->points[i].y = 1024 * rand () / (RAND_MAX + 1.0f);
    c->points[i].z = 1024 * rand () / (RAND_MAX + 1.0f);
  }
  std::cout << "Filled cloud height: " << c->height << " ** widht = "
            << c->width << " ** size: " << c->points.size()
            << "\n"
  ;
  boost::thread vis_thread( boost::bind( &Visualize, boost::ref(c) ) );
  char exit;
  std::vector<PointType> new_points;
  new_points.resize(10);
  PointType new_point;

  while(exit!='q') {
    for(unsigned int i(0);i<10;++i) {
      new_point.x = 2000 * rand () / (RAND_MAX + 1.0f);
      new_point.y = 2000 * rand () / (RAND_MAX + 1.0f);
      new_point.z = 2000 * rand () / (RAND_MAX + 1.0f);
      std::cout << "New point " << i << " with x = " << new_point.x
                << " ; y = " << new_point.y << " ; z = "
                << new_point.z << "\n"
      ;
      new_points.push_back(new_point);
    }
    {
//      boost::lock_guard<MutexType> lock(safe_update);
      MutexType::scoped_lock lock(safe_update);
      c->insert( c->points.end(), new_points.begin(), new_points.end() );
      has_to_update = true;
    } // end scoped_lock
    std::cout << "Exit?: ";
    std::cin>>exit;
  }
  vis_thread.join();
  return 0;
}

Thanks for your time!.

EDIT: Since I can't use a debugger due to Windows not recognizing the executable format(?) I've put some qDebug() lines over the Visualize function (also, instead of directly calling viewer->wasStopped() now I'm using a volatile intermediate var, stopped):

void Visualize(PointCloudType::Ptr cloud) {
  pcl::visualization::PCLVisualizer* viewer = new pcl::visualization::PCLVisualizer("Vis in thread",true);
  viewer->setBackgroundColor(1.0,0.0,0.0);
  viewer->addPointCloud<PointType>(cloud, "sample cloud");
  viewer->addCoordinateSystem(1.0);
  viewer->initCameraParameters();
  viewer->resetCamera();

  volatile bool stopped( false );
  int iterations( -1 );

  while(!stopped) {
    ++iterations;
    qDebug() << "Before spinOnce - it: << iteration << "\n";
    viewer->spinOnce(100);
    {
//      boost::lock_guard<MutexType> lock(safe_update);
        MutexType::scoped_lock lock(safe_update);
      if(has_to_update) {
        if(!viewer->updatePointCloud<PointType>(cloud, "sample cloud")) {
          viewer->addPointCloud<PointType>(cloud, "sample cloud");
          viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 1, "sample cloud");
          viewer->resetCamera();
        }
        has_to_update = false;
      }
    } // end scoped_lock
    stopped = viewer->wasStopped();
    qDebug() << "Before a new loop - it:" << iteration << "\n";
  }
  delete viewer;
};

Well, Before spinOnce is only displayed once, with iteration=0. The Before a new loop line is never printed.

On the other hand, the main thread keeps calculating and printing those points to the standard output until 'q' is inputted.

It seems that the visualization thread frozens in the viewer->spinOnce(100) call. If instead of spinOnce(100) I use the other visualization method, spin(), nothing changes.

Maybe there's a data race in my code, but for much I keep checking it, I can't find the race myself.

NOTE: According to the PCL library doc, spinOnce(int time) calls the interactor and updates the screen once, whereas spin() calls the interactor and runs an internal loop.

EDIT #2: Today I tried to execute the code again in Ubuntu and resulted in a deadlock with the PCL visualizer. I added some volatile keywords and a new loop check. Now it seems it goes well (at least it worked as expected, no wrong turns...). Here's the new version:

Global vars:

volatile bool has_to_update(true); // as suggested by @daramarak
volatile bool quit(false);         // new while loop control var

Visualize method:

void Visualize(PointCloudType::Ptr cloud) {
  pcl::visualization::PCLVisualizer* viewer = new pcl::visualization::PCLVisualizer("Vis in thread",true);
  viewer->setBackgroundColor(1.0,0.0,0.0);
  viewer->addPointCloud<PointType>(cloud, "sample cloud");
  viewer->addCoordinateSystem(1.0);
  viewer->initCameraParameters();
  viewer->resetCamera();

  while(!viewer->wasStopped() && !quit ) {
    viewer->spinOnce(100);
    {
      MutexType::scoped_lock lock(safe_update);
      if(has_to_update) {
        if(!viewer->updatePointCloud<PointType>(cloud, "sample cloud")) {
          viewer->addPointCloud<PointType>(cloud, "sample cloud");
          viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 1, "sample cloud");
          viewer->resetCamera();
        }
        has_to_update = false;
      }
    } // end scoped_lock
  }
  delete viewer;
};

Main function:

// everything the same until...
std::cin>>exit;
quit = (exit=='q');
// no more changes

I dont' like, however, the new control loop var hack. Isn't there a better way to know when to exit?. Right now, I can't realize any other way...

Adri C.S.
  • 2,909
  • 5
  • 36
  • 63
  • Looks to me like the PCL library have problems with rendering in another thread on Windows. By the way, `has_to_update` should be volatile also. – daramarak Dec 18 '12 at 06:45
  • Yeah, there are problems with multithread rendering. There's a note in the docs: `This class can NOT be used across multiple threads. Only call functions of objects of this class from the same thread that they were created in! Some methods, e.g. addPointCloud, will crash if called from other threads.`. However, in this post the author made it work: [PCLVisualizer class](http://stackoverflow.com/questions/9003239/stream-of-cloud-point-visualization-using-pcl). And in Ubuntu it worked. I'll try to bypass PCL visualization lib and work directly with VTK. I'll keep you informed on the progress :) – Adri C.S. Dec 18 '12 at 08:58
  • @daramarak. I have added a new "Edit" to the question. It seems that in Ubuntu the `spinOnce` also deadlocks sometimes, only that it isn't as often as in Windows. Now it seems that I have managed to avoid that. That said, the new code deadlocks in Windows as if no changes were made. – Adri C.S. Dec 18 '12 at 12:23

1 Answers1

0

I believe that the wasStopped() function is a const member function thereby not changing the state of the object, so there might be an optimization in play here (It might cache the wasStopped() value as the compiler assumes the answer won't change. I suggest you try to wrap the viewer in another object with a function bool wasStopped() volatile, that might prevent such optimizations.

daramarak
  • 6,115
  • 1
  • 31
  • 50
  • Thanks for you reply. I'll try your suggestion. Just a question, why would that optimization 'block' the thread in Windows and not in Ubuntu?. – Adri C.S. Dec 17 '12 at 12:28
  • 1
    It would only occur if you use a different compiler/version on those platforms I guess. You could of course also try with a debugger and try to confirm my suspicion that way. If the compile time is long it might be an good alternative. – daramarak Dec 17 '12 at 12:37
  • Unfortunately the debugger option is not available here. I tried several times to compile the app with debug symbols but QtCreator seems to ignore me. The same goes for *g++.exe* in Windows terminal. – Adri C.S. Dec 17 '12 at 12:41
  • Yeah. I think the problem is Windows itself. Whenever I try to run the debugger this message pop-ups: *not in executable format: File format not recognized*. Strange, since both the debug symbols and a debugger are installed. I hate Windows. – Adri C.S. Dec 17 '12 at 12:51
  • Hi. I wrapped the viewer in a class and redefined `wasStopped` as: `bool wasStopped() volatile { return v->wasStopped(); }`. It didn't work. Then, instead of having `pcl::visualization::PCLViewer* viewer`, I used it as `PCLViewer viewer`. Then, a new error arouse: `error: passing 'volatile pcl::visualization::PCLVisualizer' as 'this' argument of 'bool pcl::visualization::PCLVisualizer::wasStopped() const' discards qualifiers` – Adri C.S. Dec 17 '12 at 15:35
  • @AdriC.S. Hmm, perhaps `volatile bool stopped = v->wasStopped(); return stopped` ??? (clutching straws here :) Otherwise, I am stumped. It might be that it is a deadlock instead. Some clever logging might tell you if the while loop is running after you shut down or if it is stuck at the mutex. – daramarak Dec 17 '12 at 16:13
  • I tried: `bool wasStopped() { volatile bool stopped = v->wasStopped(); return stopped; }` and the thread keep running. :( Thanks for your help, though!. – Adri C.S. Dec 17 '12 at 16:30
  • Then my answer is obviously a dead end. If you could find out if the thread is running or blocking and update your question, I will have another look at it. – daramarak Dec 17 '12 at 16:38
  • I added some more info to the question. The visualization thread frozens in the `spinOnce` call. See the added info for more details. :) – Adri C.S. Dec 17 '12 at 17:13