3

I'm looking for a way that, given an input image and a neural network, it will output a labeled class for each pixel in the image (sky, grass, mountain, person, car etc).

I've set up Caffe (the future-branch) and successfully run the FCN-32s Fully Convolutional Semantic Segmentation on PASCAL-Context model. However, I'm unable to produce clear labeled images with it.

Images that visualizes my problem:
Input image

ground truth

And my result:

This might be some resolution issue. Any idea of where I'm going wrong?

Shai
  • 111,146
  • 38
  • 238
  • 371
irri
  • 43
  • 3
  • 6

1 Answers1

5

It seems like the 32s model is making large strides and thus working at a coarse resolution. Can you try the 8s model that seems to perform less resolution reduction.
Looking at J Long, E Shelhamer, T Darrell Fully Convolutional Networks for Semantic Segmentation, CVPR 2015 (especially at figure 4) it seems like the 32s model is not designed for capturing fine details of the segmentation.

Shai
  • 111,146
  • 38
  • 238
  • 371
  • 1
    Thank you! That seemed to do the trick. I'm getting good results now. – irri Sep 24 '15 at 05:11
  • @irri does the 8 stride model trained by shelhamer work with you ? https://gist.github.com/shelhamer/91eece041c19ff8968ee – Ahmed Abobakr Oct 23 '15 at 05:58
  • @Shai: Could you probably answer [What is “semantic segmentation” compared to “segmentation” and “scene labeling”?](http://stackoverflow.com/q/33947823/562769)? – Martin Thoma Nov 26 '15 at 22:28
  • @Martin Thoma regular segmentation splits images into useful blobs. Semantic segmentation splits images into useful blobs, and labels exactly what those blobs are. In other words the idea is regular segmentation may outline a chair and a dog separately, but you won't know what each is. With semantic segmentation each will be separated and labeled appropriately as "chair" and "dog". – Andrew Hundt Feb 28 '16 at 01:52
  • @AndrewHundt - this comment belongs to the other thread dealing with segmentation vs. semantic segmentation. – Shai Feb 28 '16 at 06:16