Scala to Java 8 MLeap Translation

Question

I would like to use MLeap to deploy Spark ML machine learning models and use them to predict in real-time.

The creators put out a Scala tutorial, but I need to support a Java 8 codebase.

How would I implement the following code in Java 8:

val pipeline = SparkUtil.createPipelineModel(uid = "pipeline", Array(featureModel, rfModel))

val sbc = SparkBundleContext()
for(bf <- managed(BundleFile("jar:file:/tmp/mnist.model.rf.zip"))) {
        pipeline.writeBundle.save(bf)(sbc).get
      }

val bundle = (for(bundleFile <- managed(BundleFile("jar:file:/tmp/simple-spark-pipeline.zip"))) yield {
  bundleFile.loadMleapBundle().get
}).opt.get

Answer 1

If you are using plain Spark ML transformers only, you can use the SimpleSparkSerializer to easily save and load your models.

Saving:

new SimpleSparkSerializer().serializeToBundle(model, "jar:file:/tmp/model.zip", trainData);

Loading:

Transformer model = new SimpleSparkSerializer().deserializeFromBundle("jar:file:/tmp/model.zip");

Answer 2

You can skip loading spark and it's bulky classes and load via runtime directly.

private static Transformer kMeansModel;
private static MleapContext mleapContext;
private static BundleBuilder bundleBuilder;

public MLeapLocalService() throws IOException {
    mleapContext = new ContextBuilder().createMleapContext();
    bundleBuilder = new BundleBuilder();
    Resource res = resourceLoader.getResource("classpath:aihello.com/aimodels/kmeans-model.zip");
    kMeansModel = bundleBuilder.load(res.getFile(), mleapContext).root();
}

Then you can make predictions via:

    LeapFrameBuilder builder = new LeapFrameBuilder();
    List<StructField> fields = new ArrayList<StructField>();
    fields.add(builder.createField("docs", builder.createString()));
    StructType schema = builder.createSchema(fields);
    List<Row> rows = new ArrayList<Row>();
    rows.add(builder.createRow(docs));
    DefaultLeapFrame frame = builder.createFrame(schema, rows);
    DefaultLeapFrame returnFrame = kMeansModel.transform(frame).get();