Stack Overflow
Stack Overflow

Questions tagged [mleap]

Deploy Spark Pipelines to Production. MLeap allows data scientists and engineers to deploy machine learning pipelines from Spark and Scikit-learn to a portable format and execution engine. Export a model with Scikit-learn or Spark and execute it using the MLeap Runtime (without dependencies on the Spark Context, or sklearn/numpy/pandas/etc)

Website: https://github.com/combust/mleap

Documentation: http://mleap-docs.combust.ml/

35 questions
5
votes
0 answers

Getting 'com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'ml.combust.mleap.spark'` error

I am trying to run the sample MLeap code example provided here: http://mleap-docs.combust.ml/spark/ All of the code works on my Spark cluster. However, whenever I to execute the following code snippet (again from the above link): for(bundle <-…
user3243499
  • 2,953
  • 6
  • 33
  • 75
3
votes
0 answers

MLeap serializeToBundle error for Pyspark custom Transformer

I have a Pyspark custom Transformer that I am trying to serialize to an mLeap bundle object for later model scoring but I’m getting the following error: ---> 42 self._java_obj.serializeToBundle(transformer._to_java(), path, dataset._jdf) …
femibyte
  • 3,317
  • 7
  • 34
  • 59
3
votes
2 answers

Export spark feature transformation pipeline to a file

PMML, Mleap, PFA currently only support row based transformations. None of them support frame based transformations like aggregates or groupby or join. What is the recommended way to export a spark pipeline consisting of these operations.
Gowrav
  • 627
  • 7
  • 22
3
votes
2 answers

'JavaPackage' object is not callable - MLeap

When I try to serialize a model using MLeap using the following code: import mleap.pyspark from mleap.pyspark.spark_support import SimpleSparkSerializer # Import standard PySpark Transformers and packages from pyspark.ml.feature import…
Tshilidzi Mudau
  • 7,373
  • 6
  • 36
  • 49
3
votes
2 answers

Scala to Java 8 MLeap Translation

I would like to use MLeap to deploy Spark ML machine learning models and use them to predict in real-time. The creators put out a Scala tutorial, but I need to support a Java 8 codebase. How would I implement the following code in Java 8: val…
MiketheViking90
  • 39
  • 5
2
votes
0 answers

How to solve self._java_obj = _jvm().ml.combust.mleap.spark.SimpleSparkSerializer() TypeError: 'JavaPackage' object is not callable

I try to use mleap on EMR cluster, but when i try to use it i receive the following error: self._java_obj = _jvm().ml.combust.mleap.spark.SimpleSparkSerializer() TypeError: 'JavaPackage' object is not callable I load the jar files from the…
Luciano Salemme
  • 387
  • 2
  • 15
2
votes
0 answers

MLeap and Spark ML SQLTransformer

I have a question. I am trying to serialize a PySpark ML model to mleap. However, the model makes use of the SQLTransformer to do some column-based transformations e.g. adding log-scaled versions of some columns. As we all know, Mleap doesn't…
femibyte
  • 3,317
  • 7
  • 34
  • 59
2
votes
0 answers

Using Custom spark transformers in pyspark

How can I use a custom transformer written in scala in a pyspark pipeline. class UpperTransformer(override val uid: String) extends UnaryTransformer[String, String, UpperTransformer] { def this() = this(Identifiable.randomUID("upper")) …
pratyush
  • 36
  • 2
2
votes
0 answers

Generating mleap bundle from my TensorFlow program

I have a TensorFlow model which I am trying to serialise into a MLeap bundle, so that I can use MLeap based predictor to score. But I could not find any API to do so from my TensorFlow code. The MLeap documentation talks about freezing the TF graph…
Gladiator
  • 51
  • 5
1
vote
0 answers

MLeap problem: Impossible to deserialize a bundle written with Scikit-Learn with Pyspark. Error: No bundle.json found

I serialize a model with Scikit-Learn: #Generate data import pandas as pd import numpy as np df = pd.DataFrame(np.random.randn(100, 5), columns=['a', 'b', 'c', 'd', 'e']) df["y"] = (df['a'] > 0.5).astype(int) df.head() from…
Nastasia
  • 557
  • 3
  • 22
1
vote
1 answer

I don't succeed to save (serialize) a zip file with Scikit-Learn with MLeap in Python

I tried that: #Generate data import pandas as pd import numpy as np df = pd.DataFrame(np.random.randn(100, 5), columns=['a', 'b', 'c', 'd', 'e']) df["y"] = (df['a'] > 0.5).astype(int) df.head() from mleap.sklearn.ensemble.forest import…
Nastasia
  • 557
  • 3
  • 22
1
vote
0 answers

Mleap Calculating Confidence and feature score

I was able to successfully serialize a pyspark generated RandomForestModel as Mleap bundle and run it using mleap-runtime. I want to get the confidence score and score for each feature. Is there a way to achieve that without using any spark…
Manish Gupta
  • 11
  • 1
1
vote
1 answer

ML model deployment CI/CD

I am training models using MLFlow on DataBricks and outputing the final models onto S3. Than, using Seldon-Core to to package AND deploy the models to AWS EKS. I am looking for the tool that bridges the gap by taking the model from S3, packages it…
Assaf Moldavsky
  • 1,681
  • 1
  • 19
  • 30
1
vote
0 answers

Is it possible to have more inputs for MLeap Transformer

I have a mleap Transformer which has to have around 14 inputs for transformation.I tried to create core model however i see the error when i increase the number of input variable else its does not create error. package…
Ashwin Padhy
  • 111
  • 1
  • 9
1
vote
1 answer

Unable to run transform in Mleap runtime from Spark model

I'm currently testing the Mleap solution in order to perform prediction on Spark model. In order to do that, I've first implemented the Spark example for linear regression as described here:…
Stéphane Davy
  • 21
  • 5
1
2 3