11

I have been working on an application that involves font recognition based on a users free hand drawing characters in Android Canvas.

In this application the user is asked to enter some predefined characters in a predefined order (A,a,B,c). Based on this, is there any way to show the very similar font which matches the user's hand writing.

I have researched on this topic found some papers & articles but most of them are recognizing font from a captured image. In that case they are having a lot of problems by segmenting paragraphs, individual letters and so on. But in my scenario I know what letter the user is drawing.

I have some knowledge in OpenCV and Machine Learning. Need help on how to proceed with this problem.

Kevin Katzke
  • 3,581
  • 3
  • 38
  • 47
Dinesh Kannan
  • 1,255
  • 13
  • 32
  • 4
    So, basically, you are asking how to write an app which can tell if my handwriting looks like `Arial` or `Tahoma` or `Century Gothic` or one of the millions (and counting) existing fonts?! – Phantômaxx Aug 27 '16 at 10:27

3 Answers3

2

It is not exactly clear to me what you want to accomplish with your application but I assume that you are trying to output a font from a database of fonts that matches a users handwriting the most.

In Machine Learning this would be a classification problem. The number of classes will by equal to the number of different fonts in your database.

You could solve this with the help of a Convolutional neural network which are widely used for image and video recognition related tasks. If you've never implemented a CNN before I would suggest that you look up this resources to learn about Torch which is a easy-to-start-with toolkit to implement CNN's. (Of course there are more Frameworks such as: Tensor Flow, Caffe, Lasagne, ...)

The main obstacle you will face is that Neural Networks need thousands of images (>100.000) to properly train them and to achieve satisfying results. Furthermore you do not only need the images but also a correct label for each image. Will say, you would need a training image such as a handwritten character and the corresponding font it matches the most out of your database as its label.

I would suggest that you read about so called transfer learning which can give you an initial boost as you do not need to set up a CNN model completely by yourself. In addition people have pre-trained such a model for a related task so that you safe extra time as you would not need to train it for many hours on a GPU. (see CUDA)

A great resource to start with is the paper: How transferable are features in deep neural networks?, which could be helpful for the stated reasons.

To get tons of training and testing data you can look up the following open datasets that provide all types of characters that can be helpful for your task:

For access to a lot of fonts and maybe even the possibility to create further datasets on your own you can have a look at Google Fonts.

Kevin Katzke
  • 3,581
  • 3
  • 38
  • 47
0

You might find this article very interesting : https://erikbern.com/2016/01/21/analyzing-50k-fonts-using-deep-neural-networks/

Seems like a pretty straightforward deep learning supervised learning problem.

Generate a ton of randomly deformed samples for letters of each target font type, and train a convnet on that set?

The ideal would be to have a huge set of labeled, handwriting to font data, but that feels unlikely.

You could also use the generated, progressive to font code to take a bunch of handwritten samples, and transform them to look more like the font of your choice, as a dataset.

This is good place to start : https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py Digit letter recognition with convnets.

This is quite a bit of work though if you haven't worked with that stuff before.

Jules G.M.
  • 3,624
  • 1
  • 21
  • 35
-1

I would suggest using OCR library tesseract. Very well developed and mature. It also has support for training with other languages which you can use to train over a set of font.

Approach

Training:-

  1. Take all 26(per alphabet) images for n fonts. Train tessaract over 26 A's, then 26 B's and soon.

Testing:-

  1. Take a sentence and separate all characters.
  2. For each character, find certainty score(supported in library) from Tesseract. Note, for character 'a, use the trained model on all 'a''s from different fonts.
  3. For all characters, find best font using some metric (average, median, etc). For example: You can sum certainty score each font received for all characters and use the font which got max result.
saurabheights
  • 3,967
  • 2
  • 31
  • 50
  • A downvote means: "this answer is not useful". Not my DV, but strongly agree that this is not useful. Tesseract isn't meant to be used with handwritten characters – Miki Aug 31 '16 at 16:25
  • @Miki: Thanks for your reply. Please see this: http://stackoverflow.com/questions/12310287/tesseract-handwriting-with-dictionary-training . Also, for OCR, Online Handwriting Recognition is great (http://stackoverflow.com/questions/12393358/free-hand-character-recognition-in-android), but here we already know that user has written which character. If online Handwriting Recognition is able to record the subtle difference in character shapes when written, that would be much better, otherwise I think Tesseract is the way. I am open to any thoughts you may have. Thank you. – saurabheights Sep 01 '16 at 01:41
  • Typo: If online Handwriting Recognition is able to **analyze** the subtle difference in character shapes when written, that would be much better, otherwise I think Tesseract is the way. Subtle differences would require checking the character structure, which Tesseract is good at. – saurabheights Sep 01 '16 at 02:59