3

I have a group of images and some separate heatmap data which (imperfectly) explains where subject of the image is. The heatmap data is in a numpy array with shape (224,224,3). I would like to generate bounding box data from this heatmap data.

The heatmaps are not always perfect, So I guess I'm wondering if anyone can think of an intelligent way to do this.

Here are some examples of what happens when I apply the heatmap data to the image:

Image of a cat with a heatmap illuminating the subject of the image

enter image description here

I found a solution to this in matlab, but I have no idea how to read this code! I am a python programmer, unfortunately. https://github.com/metalbubble/CAM/tree/master/bboxgenerator

Anyone have any ideas about how to approach something like this?

zazou
  • 197
  • 3
  • 10

3 Answers3

0

I am not quite sure how the heatmap data of your project exactly looks like, but it seems to me that you can use something like Selective Search. You can also have a look on this interesting paper. Maybe you can use this approach on your dataset.

ITiger
  • 1,056
  • 3
  • 11
  • 24
0

I'm attempting a similar method for automating the creation of bounding boxes (since, lets face it: creating boxes manually takes along time)

this other stackpost covers a similar idea: EDIT: (i originally had put a link to the current stack post - but here is the stack post i was referring to) Generating bounding boxes from heatmap data

the problem at hand that i recognize is that heatmaps can be fragmented and a bit arbitrary. the solution that comes to mind initially is setting a threshold of the heat map. So in the case of the example heat map images - when applying a bounding box cover all regions that are yellow/orange/red than say green/blue.

Shaheen Alemi
  • 1
  • 1
  • 1
    Hi @Shaheen Alemi and welcome ! Could you please add further explanations such as what tools you use or pieces of your code so that people can help you at best ? Also please consider adding tags to your question, It will improve your probability to get answers and also help people in the future that have the same problem as you have today. – Uj Corb Feb 14 '20 at 09:29
0

It depends on how many bounding boxes you need. You can set a threshold and have multiple bounding boxes for each of the highly activated regions, or try connecting the regions (by a morphological operation maybe) and calculate a single bounding box for connected activated pixels.

Ghazal
  • 1
  • 1