Create ML Text Classifier probabilities

Question

I am creating model with Create ML. I am using a JSON file.

let data = try MLDataTable(contentsOf: URL(fileURLWithPath: "poems.json"))
let (trainingData , testingData) = data.randomSplit(by: 0.8, seed: 0)

let classifier = try MLRegressor(trainingData: data, targetColumn: "author", featureColumns: ["text"])
let metadata = MLModelMetadata(author: "ffffff", shortDescription: "sdhkgfjhsgdjfhgs", license: nil, version: "1.0", additional: nil)

try classifier.write(to: URL(fileURLWithPath: "poems.mlmodel"), metadata: metadata)

The JSON file looks like this

{"title":"When You Are Old",
 "author":"William Butler Yeats",
 "text":"When you are old and grey and full of sleep,\nAnd nodding by the   fire, take down this book,\nAnd slowly read, and dream of the soft look\nYour eyes had once, and of their shadows deep;\nHow many loved your moments of glad grace,\nAnd loved your beauty with love false or true,\nBut one man loved the pilgrim Soul in you,\nAnd loved the sorrows of your changing face;\nAnd bending down beside the glowing bars,\nMurmur, a little sadly, how Love fled\nAnd paced upon the mountains overhead\nAnd hid his face amid a crowd of stars."}

Following a tutorial I am trying to do a text detection on the "text" and return the possible "author"

I can make that but I would like to have also the probability.

Creating the model with Create ML, as a Text Classifier I get only output a label: Author. Is there a way with Create ML to have also probability in Text Classification?

Thanks

I also meet the same problem, have you find a solution? – chanjianyi Oct 26 '18 at 07:16 — chanjianyi, Oct 26 '18 at 07:16

score 3 · Answer 1 · answered Sep 08 '18 at 11:47

It doesn't appear to be possible with the current MLTextClassifier API.

If you open your mlmodel file in Netron, https://github.com/lutzroeder/Netron/, it will show you the outputs produced by the model. My guess is that it just gives the class, not the probability.

score 1 · Answer 2 · answered Oct 11 '18 at 01:09

1

I suppose it's not yet supported by the MLTextClassifier API. However you can get probabilities from your model by using the more generic (and overall less accurate) MLClassifier while creating your model.

answered Oct 11 '18 at 01:09

cescofry

3,706
3
26
22

score 0 · Answer 3 · answered Dec 13 '18 at 23:14

The MLModel created in the XCode playground is a text classifier that only returns a label (String). Apple's example on Github (https://apple.github.io/turicreate/docs/userguide/text_classifier/) which uses the turicreate python library to create the model is a Pipeline classifier and it returns a label and its probability (String -> Double).

score 0 · Answer 4 · answered Jan 30 '23 at 16:22

Yes, use NatrualLanguage framework to convert the MLModel into a NLModel and then get confidence.

import CoreML
import NaturalLanguage

let model = try my_model(configuration: MLModelConfiguration()).model
let text = "Hello, World"
let nlmodel = try NLModel(mlModel: model)
let prediction = nlmodel.predictedLabel(for: text)
let predictionSet = nlmodel.predictedLabelHypotheses(for: text, maximumCount: 1)
let confidence = predictionSet["1"] ?? 0.0
print(text + "," + "\(confidence)")

Create ML Text Classifier probabilities

4 Answers4