Classify faces from VNFaceObservation

Question

I'm working with Vision framework to detect faces and objects on multiple images and works fantastic.

But I have a question that I can't find on documentation. The Photos app on iOS classify faces and you can click on face and show all the images with this face.

How can I classify faces like Photos app? Is there any unique identifier or similar to do this?

Thanks!

score 2 · Answer 1 · answered Oct 15 '18 at 05:13

In order to uniquely recognise faces, firstly you need to detect a face, then run it through a CoreML model (or another image classification model type, such as a Tensorflow model) in order to classify the image and tell you the likeliness that the face you captured matches one of the faces trained into your model.

Apple Photos uses machine learning (as mentioned in their iPhone reveal keynote this year) to train the device to recognise faces in your photos. The training would be performed locally on the device, however, Apple does not offer any public APIs (yet) to allow us to do this.

You could send photo data (crops of faces using the tool mentioned above by Paras) to your server and have it train a model (using CoreML's trainer or something like Nvidia DIGITS on AWS or your own server), convert it to CoreML, compile the model then download it to your device and sideload the model. This is as close as you're going to get to "magic" face recognition used by Photos, for now, as the device can only read compiled models.

score 1 · Answer 2 · answered Sep 12 '18 at 21:23

I don't think their is a way to uniquely identify faces returned to you from the vision framework. I checked the UUID property a VNFaceObservation and it is a different identifier every-time.

You might have to make your own CoreML model or just wait/find a good 3rd party one.

I hope someone proves me wrong because I want to know also.

score 0 · Accepted Answer · answered Aug 10 '17 at 07:19

0

You might want to check out this repo https://github.com/KimDarren/FaceCropper

I tested this and works very well, you can even customise as per your need.

answered Aug 10 '17 at 07:19

Paras Gorasiya

1,295
2
13
33

But not identify unique faces and classify like Apple Photos app. Thanks! – mhergon Aug 10 '17 at 07:43
This does identify unique faces just like photos app. – Paras Gorasiya Aug 10 '17 at 07:44
I just provided you an example with that link, you should find something useful. That demo does the same thing you want to do. – Paras Gorasiya Aug 13 '17 at 14:52
This example does not uniquely identify faces. It just gets all the faces in an image, crops them, and then displays them in a `UICollectionView`. – Jon Vogel Sep 12 '18 at 21:21

Classify faces from VNFaceObservation

3 Answers3