4

I am trying to see the feasibility of using TensorFlow to identify features in my image data. I have 50x50px grayscale images of nuclei that I would like to have segmented- the desired output would be either a 0 or 1 for each pixel. 0 for the background, 1 as the nucleus.

Example input: raw input data

Example label (what the "label"/real answer would be): output data (label)

Is it even possible to use TensorFlow to perform this type of machine learning on my dataset? I could potentially have thousands of images for the training set.

A lot of the examples have a label correspond to a single category, for example, a 10 number array [0,0,0,0,0,0,0,0,0,0,0] for the handwritten digit data set, but I haven't seen many examples that would output a larger array. I would assume I the label would be a 50x50 array?

Also, any ideas on the processing CPU time for this time of analysis?

Chris Martin
  • 30,334
  • 10
  • 78
  • 137
J_K
  • 43
  • 1
  • 1
  • 6
  • was a solution ever found? I can't find a loss function that will work with pixel-wise segmentation. Instead of classifying each pixel, it just picks one of the (x,y) output tensors to classify. Please see http://stackoverflow.com/questions/37898795/tensorflow-accuracy-at-99-but-predictions-awful – Kendall Weihe Jun 18 '16 at 18:47
  • See https://github.com/Russell91/TensorBox – Martin Thoma Oct 20 '16 at 10:48

3 Answers3

1

Yes, this is possible with TensorFlow. In fact, there are many ways to approach it. Here's a very simple one:

Consider this to be a binary classification task. Each pixel needs to be classified as foreground or background. Choose a set of features by which each pixel will be classified. These features could be local features (such as a patch around the pixel in question) or global features (such as the pixel's location in the image). Or a combination of the two.

Then train a model of your choosing (such as a NN) on this dataset. Of course your results will be highly dependant upon your choice of features.

You could also take a graph-cut approach if you can represent that computation as a computational graph using the primitives that TensorFlow provides. You could then either not make use of TensorFlow's optimization functions such as backprop or if there are some differentiable variables in your computation you could use TF's optimization functions to optimize those variables.

rafaelcosman
  • 2,569
  • 7
  • 23
  • 39
  • 2
    OP asked for Image Segmentation with TF, I assume Deep learning here. We already known DNN is suitable for segmentation task. Most of the literature use deconv or regression to produce densed prediction. However, Tensorflow doesn't seems to have a good method to calculate the loss value. `SoftmaxWithLoss()` only accept `[batch_size class_num]` input which is obviously not helpful in OP case. Since OP accepted your answer, I hope he could explain how he is approaching his problem. – 24hours May 05 '16 at 15:10
1

SoftmaxWithLoss() works for your image segmentation problem, if you reshape the predicted label and true label map from [batch, height, width, channel] to [N, channel].

In your case, your final predicted map will be channel = 2, and after reshaping, N = batchheightwidth, then you can use SoftmaxWithLoss() or similar loss function in tensorflow to run the optimization.

See this question that may help.

Community
  • 1
  • 1
Wei Liu
  • 1,004
  • 1
  • 10
  • 17
  • If the label image being loaded in is (H, W, 3) [RGB Color], how do you reshape those 3 dimensions representing the label to 1 dimension? For example Pascal VOC's segmented images are color coded, but there are only 21 classes I believe. – AJ Venturella Jul 19 '17 at 17:50
0

Try using a convolutional filters for the model. A stacking of convolution and downsampling layers. The input should be the normalized pixel image and output should be the mask. The last layer should be a softmaxWithLoss. HTH.

user1533097
  • 1