How to show and work with a 3D image from the Bosphorus database (or a similar database) - Starting from a Nx(3+2) matrix from a range sensor

Question

Best

I've the problem that I don't know how to use the Bosphorus data (or a similar one, which uses the same kind of structure). First of all, my final goal is, to create a 3D face recognition system from the Bosphorus database. But unfortunately I don't know how to do this. I don't ask you how to solve the 3d face recognition but I only really like to know how to handle this data so that I can create this application. I've tried a lot of things but non of them worked. And almost all the papers start from a given 3D face.

This Bosphorus database provides me the following data

• images of a human faces (.png)
• landmarks of a human faces [Person X - Outer left eyebrow : -91.623 7.905 -83.9 ... ]
• an Nx5 matrix

And that its basicly everything.

But before I go further, here is the description of the Nx5 Matrix :

  % Date:   2008
  % Outputs:
  %   zmin      : minimum depth value denoting the background        | =  -1000000           
  %   nrows     : subsampled number of rows                          | =   229
  %   ncols     : subsampled number of columns                       | =   188
  %   imfile    : image file name
  %   data      : Nx5 matrix where columns are 3D coordinates and 2D |  N = 43052 
  %   normalized image coordinates respectively. 2D coordinates are
  %   normalized to the range [0,1]. N = nrows*ncols. In this matrix, values
  %   that are equal to zmin denotes the background.

  -1000000000  -1000000000  -1000000000  0,995567  0,003639
  -1000000000  -1000000000  -1000000000  0,990248  0,003639
  -1000000000  -1000000000  -1000000000  0,984929  0,003639
  ...          ...          ...          ...       ...

And now the real question: How do I handle this data ?

First of all, how do I create an 3D image like in the following image by this data?

Secondly, the Bosphorus data is normalized. But the landmarks are not. How can I plot, use, anker, refer the landmark points on that 3D image?

Third, How can I calculate the geodesic distance between 2 points from the 3D image? E.g. the geodesic distance between two landmarks (shortest path, Dijkstra). Thus how do I hop from the one to another.

And that is basicly it. I've tried a lot of papers but all of them starts with the given 3D image even though they use the Bosphorus database or a similar one.

I hope that someone can help me, everything is welcome

Kind regards

Dieter

-- For this project, I'm using Matlab. But I think that if you've a general idea, that the environment would be independent to solve it --

Answer 1

Question 1 is related to the way you use matlab to process structured data. So you need to know the structure and the matlab operations necessary to deal with them.

The structure is documented at the top of your datafile: it appears to be an Nx5 space-separated matrix so you can use the load command here (type doc load to see how it works, or see this question for a simple example).

facedata = load('yourfile');

Then you need to pick out the 3D data coordinates and plot them. Read about matlab's indexing methods here, and bear in mind that matlab indexes column-first, which throws a lot of people.

faceX = facedata(:,1);
faceY = facedata(:,2);
faceZ = facedata(:,3);

To draw the actual plot you can use scatter3 although it will only draw a point cloud. You may have to shuffle the data around a bit depending on which coordinate comes first in the bosphorus data (they probably say in the docs that come with it) but a safe guess would be XYZ with X to the right and Z vertical so you could plot it as a point cloude using catter3 (http://uk.mathworks.com/help/matlab/ref/scatter3.html).

scatter3(faceX,faceY,faceZ);

(EDIT: I originally suggested surf but it requires matrix inputs as noted by the OP in the comments. I replaced it with scatter3 because it accepts column input, although the OP's solution of using delaunay triangulation and trimesh gives the desired wireframe).

Question 2: the answer is in the text you posted:

data : Nx5 matrix where columns are 3D coordinates and 2D normalized image coordinates respectively.

So just use the first three columns to get the unnormalised data.

Question 3: again you already have the answer: get Dijkstra's shortest path algorithm and implement it. Bear in mind that it may not be the most accurate - there is a good description of some of the issues here.

If you get stuck with that then I would recommend to post a new question specifically about it. Do include details of what you've tried, how it fails and what you expected to see.

In general I would also advise that research papers are going to universally assume that you know the basics of whatever environment they are using - they haven't got room in the journal for much and it would only be repeated anyway. Sometimes for the basic info you have to think laterally: instead of 'how to use bosphorus data in matlab' you can break it up into pieces and then generalise, so you could ask instead 'how to load text data into matlab', 'how to plot 3d data in matlab', 'how to implement dijkstra's shortest path algorithm in matlab' etc.

Answer 2

Best myself

Like said before, the Bosphorus data contains normalized data. Only the first 3 column vectors are here important because they refer to the 3D data.

Thus if xyz, is our matrix with our normalized data

  xyz = 3normalizedColumnsXYZ;

  %% Then, you will get the mesh via the following piece of code

  x_plot = linspace(min( xyz(:,1)),max( xyz(:,1)),n);
  y_plot = linspace(min( xyz(:,2)),max( xyz(:,2)),n);
  [XI, YI] = meshgrid( x_plot, y_plot);
  ZI = griddata( xyz(:,1), xyz(:,2), xyz(:,3), XI, YI);


 %%plot the mesh - if you want to
     figure(1)
     mesh( XI, YI, ZI) 

Kind regards to myself