VTK and MPI: plot distributed data

Question

After some researches, I can't figure out what is the best way to use VTK with MPI in order to be able to visualize distributed data.

The data I want to plot is a set of moving points within a cube (we know the boundaries if this can help). This set of points will be updated at each iteration, and it can also be huge. For the moment we don't really know how our implementation will scale, but the goal is to be able to handle as much points as possible.

Here are the 2 ways I found in my researches for plotting in parallel:

Using vtkMultiProcessController. This approach seems to be rather complex, and the only "documentation" I found are the class documentation and an example. This will probably be enough to use this class, but this will take me too much time in my opinion.
Using VTK parallelized formats like in this link. This option seems easier to use, not really intrusive, and maybe (?) faster than the previous one.

My questions are:

Is there any other methods that can be used to plot a distributed set of points efficiently?
What method is the most appropriate for plotting my dataset?
(Side question) What are the best format supported by VTK (for creating the file) and by Paraview (to view the animation) to plot such an animation?

Thanks a lot for your time!

score 1 · Answer 1 · answered May 04 '18 at 15:08

I think the first way is more reliable, because, I spend a lot of time to work with the second method and I was not able to get a correct result out of it. As you said, VTK classes do not have a good documentation, especially the parallel classes like vtkXMLPStructuredGridWriter.

By the way, I suggest you to use the first method. However, instead of using vtkMultiProcessController, use vtkMPIController which is derived from vtkMultiProcessController class. Recently, I wrote an example for Structured Grid that by doing some changes, you can use it for your own application:

// MPI Library
#include <mpi.h>

//VTK Library
#include <vtkXMLPStructuredGridWriter.h>
#include <vtkStructuredGrid.h>
#include <vtkSmartPointer.h>
#include <vtkFloatArray.h>
#include <vtkCellData.h>
#include <vtkMPIController.h>
#include <vtkProgrammableFilter.h>
#include <vtkInformation.h>

struct Args {
  vtkProgrammableFilter* pf;
  int local_extent[6];
};

// function to operate on the point attribute data
void execute (void* arg) {
  Args* args = reinterpret_cast<Args*>(arg);
  auto info = args->pf->GetOutputInformation(0);
  auto output_tmp = args->pf->GetOutput();  // this is a vtkDataObject*
  auto input_tmp  = args->pf->GetInput();   // this is a vtkDataObject*
  vtkStructuredGrid* output = dynamic_cast<vtkStructuredGrid*>(output_tmp);
  vtkStructuredGrid* input  = dynamic_cast<vtkStructuredGrid*>(input_tmp);
  output->ShallowCopy(input);
  output->SetExtent(args->local_extent);
}

int main (int argc, char *argv[]) {
  MPI_Init (&argc, &argv);
  int myrank;
  MPI_Comm_rank (MPI_COMM_WORLD, &myrank);

  int lx {10}, ly{10}, lz{10};        //local dimensions of the process's grid
  int dims[3] = {lx+1, ly+1, lz+1};
  int global_extent[6] = {0, 2*lx, 0, ly, 0, lz};
  int local_extent[6] = {myrank*lx, (myrank+1)*lx, 0, ly, 0, lz};

  // Create and Initialize vtkMPIController
  auto contr = vtkSmartPointer<vtkMPIController>::New();
  contr->Initialize(&argc, &argv, 1);
  int nranks = contr->GetNumberOfProcesses();
  int rank   = contr->GetLocalProcessId();

  // Create grid points, allocate memory and Insert them
  auto points = vtkSmartPointer<vtkPoints>::New();
  points->Allocate(dims[0]*dims[1]*dims[2]);
  for (int k=0; k<dims[2]; ++k)
    for (int j=0; j<dims[1]; ++j)
      for (int i=0; i<dims[0]; ++i)
        points->InsertPoint(i + j*dims[0] + k*dims[0]*dims[1],
                            i+rank*(dims[0]-1), j, k);

  // Create a density field. Note that the number of cells is always less than
  // number of grid points by an amount of one so we use dims[i]-1
  auto density = vtkSmartPointer<vtkFloatArray>::New();
  density->SetNumberOfComponents(1);
  density->SetNumberOfTuples((dims[0]-1)*(dims[1]-1)*(dims[2]-1));
  density->SetName ("density");
  int index;
  for (int k=0; k<lz; ++k)
    for (int j=0; j<ly; ++j)
      for (int i=0; i<lx; ++i) {
        index = i + j*(dims[0]-1) + k*(dims[0]-1)*(dims[1]-1);
        density->SetValue(index, i+j+k);
      }

  // Create a vtkProgrammableFilter
  auto pf = vtkSmartPointer<vtkProgrammableFilter>::New();

  // Initialize an instance of Args
  Args args;
  args.pf = pf;
  for(int i=0; i<6; ++i) args.local_extent[i] = local_extent[i];

  pf->SetExecuteMethod(execute, &args);

  // Create a structured grid and assign point data and cell data to it
  auto structuredGrid = vtkSmartPointer<vtkStructuredGrid>::New();
  structuredGrid->SetExtent(global_extent);
  pf->SetInputData(structuredGrid);
  structuredGrid->SetPoints(points);
  structuredGrid->GetCellData()->AddArray(density);

  // Create the parallel writer and call some functions
  auto parallel_writer = vtkSmartPointer<vtkXMLPStructuredGridWriter>::New();
  parallel_writer->SetInputConnection(pf->GetOutputPort());
  parallel_writer->SetController(contr);
  parallel_writer->SetFileName("data/test.pvts");
  parallel_writer->SetNumberOfPieces(nranks);
  parallel_writer->SetStartPiece(rank);
  parallel_writer->SetEndPiece(rank);
  parallel_writer->SetDataModeToBinary();
  parallel_writer->Update();
  parallel_writer->Write();

  contr->Finalize();

  // WARNING: it seems that MPI_Finalize is not necessary since we are using
  // Finalize() function from vtkMPIController class. Uncomment the following
  // line and see what happens.
//   MPI_Finalize ();
  return 0;
}

Regarding your last question, If you use VTK library, you can choose between Ascii and Binary formats. The binary format is usually smaller in terms of size. You can set this binary format easily by calling SetDataModeToBinary() function. I used this function in the above code.

Also read this question which is relevant to what you are looking for.

VTK and MPI: plot distributed data

1 Answers1