1

I have a binary which has some text on different parts of the image like at the bottom, top, center, right middle center, etc.

Original Image

enter image description here

The areas I would like to focus on are the manually drawn regions shown in red.

enter image description here

I calculated the horizontal and vertical summations of the image and plotted them:

plot(sum(edgedImage1,1))

enter image description here

plot(sum(edgedImage1,2))

enter image description here

Can somebody give me explanation of what these plots are telling me about the original image with regards to the structure of which I explained above? Moreover, how could these plots help me extracting those regions I just manually drew in red?

rayryeng
  • 102,964
  • 22
  • 184
  • 193
Naseer
  • 4,041
  • 9
  • 36
  • 72

3 Answers3

2

There's nothing sophisticated about the sum operation. Simply put, sum(edgedImage1,1) computes the sum of all rows for each column in the image and that is what you are plotting. Effectively, you are computing the sum of all non-zero values (i.e. white pixels) over all rows for each column. The horizontal axis in the plot denotes what row's sum you are observing. Similarly, sum(edgedImage,2) computes the sum of all columns for each row of the image and that is what you are plotting.

Because your text is displayed in a horizontal fashion, sum(edgeImage,1) won't be particularly useful. What is very useful is the sum(edgedImage,2) operation. For lines in your image that are blank, the horizontal sum of columns for each row of your image should be a very small value whereas for lines in your image that contain text or strokes, the sum should be quite large. A good example of what I'm talking about is seen towards the bottom of your image. If you consult between rows 600 and 700, you see a huge spike in your plot as there is a lot of text that is surrounded between those rows.

Using this result, a crude way to determine what areas in your image that contain text or strokes in your case would be to find all rows that surpass some threshold. Combined with finding modes or peaks from the sum operation that was just performed, you can very easily localize and separate out each region of text.

You may want to smooth the curve provided by sum(edgedImage,2) if you decide to determine how many blobs of text there are. Once you smooth out this signal, you will clearly see that there are 5 modes corresponding to 5 lines of text.

rayryeng
  • 102,964
  • 22
  • 184
  • 193
  • your explanation is fantastic and really helpful.It would be kind and nicer of you if you could refer me some code links from where I could replicate this or if you have some time please help me bit more on this regarding coding? – Naseer Aug 25 '16 at 19:52
  • 1
    @NaseerAhmed If you provide the original image (without the red rectangles you have drawn), I can show you some sample results with some code I can write. – rayryeng Aug 25 '16 at 19:52
  • I have uploaded original Image.Actually original Image is sobel filter I applied on the input image. – Naseer Aug 25 '16 at 19:55
  • 1
    @NaseerAhmed I'll get back to you soon. – rayryeng Aug 25 '16 at 20:01
  • Ok No problem when You have time. – Naseer Aug 26 '16 at 07:14
  • 1
    @NaseerAhmed Apologies I'll get to your thing soon. It has been a busy day. – rayryeng Aug 26 '16 at 07:15
  • Its ok sir No problem – Naseer Aug 26 '16 at 07:35
  • I have also used some morphological operations and they are helping in detecting those areas but I think I would get good results If somehow I could use horizontal projection information like you suggested. – Naseer Aug 26 '16 at 12:50
  • 1
    @NaseerAhmed Certainly. Let me get into work and I'll post some results. I haven't had the time unfortunately but I haven't forgotten about you. – rayryeng Aug 26 '16 at 12:55
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/121969/discussion-between-naseer-ahmed-and-rayryeng). – Naseer Aug 26 '16 at 20:23
1

The second plot that shows the sum of each row. This can tell you in which rows you have a lot of information and in which you have none.

You can use this plot to find the rectangles by looking for a sharp incline in the value for a start of a rectangle and sharp decline in the value for the end of the rectangle. Before you do it i would low pass filter the data and then look at the derivative of this and look for a big derivative.

You can do the same the first plot but it is more sensitive.

Amitay Nachmani
  • 3,259
  • 1
  • 18
  • 21
1

The minimums in your last plot are the gaps between lines of text ...

You just take the graph and align its y axis to y axis of image and then Threshold areas with too small amount of pixels per column. These areas (Red) are the gaps between lines of Text or whatever you got on the image:

align

Now you should de-skew the image if needed. If the skew is too big you need to apply the de-skew even before the y axis summation.

After this you make the x axis summation plot for each non red region separately and in the same manner detect gaps between characters/words to get the area of each character for OCR. This time you align to x axis

These plots can be also used to OCR if taken on character region instead see

If you do a statistical analysis of the gap/non gap sizes then the most recurrent one is usually the Font spacing/size for regular text.

Community
  • 1
  • 1
Spektre
  • 49,595
  • 11
  • 110
  • 380
  • I am not getting How would I do it in matlab? – Naseer Sep 16 '16 at 16:11
  • @NaseerAhmed I do not code in Matlab at all (as I hate that environment), but you can store the sum plots as 1D vectors and handle them as such. If you do not know how to code basic stuff like tresholding which is in 1D just `for` cycle with single `if` statement then may be you should start with easier task then this. Without coding it yourself you need to go through the gazilion of in-build functions of matlab and look for those that can be exploited for this instead and then assemble them together like a puzzle until you got desirable result. – Spektre Sep 17 '16 at 06:31