1

I have some questions about how to actually interact with a pre-trained Caffe model. In my case I'm using a model for scene recognition.

In the caffe git repository, there are some code examples in Python and C++ on the implementations of Image Classifiers. However, those do not apply to my use case (since they only classify the input image as ONE class).

My goal is an application that takes an input image (jpg) and outputs the highest predicted class label for each pixel in the input image (e.i., indices for sky, beach, road, car).

Could anyone give me some pointers on how to proceed?

There already seem to exist implementations for this. This demo (http://places.csail.mit.edu/demo.html) is kind of what I what.

Thank you!

Shai
  • 111,146
  • 38
  • 238
  • 371
irri
  • 43
  • 3
  • 6
  • possible duplicate of [Can Caffe classify pixels of an image directly?](http://stackoverflow.com/questions/30198926/can-caffe-classify-pixels-of-an-image-directly) – Shai Sep 08 '15 at 10:01
  • I can recommend the work from the Torr Vision group: https://github.com/torrvision/crfasrnn. Using the included installation guide I got it working quite easily. Note that if you already installed Caffe you might have to be careful with the included Caffe version as the mentioned work uses a custom Caffe which is included. – David Stutz Apr 07 '16 at 09:17

1 Answers1

1

What you are looking for is not image classification, but rather semantic segmentation.

A recent work, by Jonathan Long, Evan Shelhamer and Trevor Darrell is based on Caffe, and can be found here. It uses fully convolutional network, that is, a network with no "InnerProduct" layers only convolutional layers, thus capable of producing outputs with different sizes for different sizes of inputs.

Shai
  • 111,146
  • 38
  • 238
  • 371
  • Thanks for pointing me in the right direction! I've set up Caffe (the future-branch) and successfully run the [FCN-32s Fully Convolutional Semantic Segmentation on PASCAL-Context](https://gist.github.com/shelhamer/80667189b218ad570e82/) model. However, I'm unable to produce clear labeled images with it. Results: [Input image](http://www.cs.stanford.edu/~roozbeh/pascal-context/2008_001997.jpg), [good output result, but other model](http://www.cs.stanford.edu/~roozbeh/pascal-context/2008_001997L.png), [My result output](http://i.imgur.com/YWspltp.png). Any idea of where I'm going wrong? – irri Sep 08 '15 at 02:39
  • 1
    @irri seems like a resolution issue. you might post this as a new question to draw more attention to it. – Shai Sep 08 '15 at 05:14