7

I have a data set for which I use Sklearn Decision Tree regression machine learning package to build a model for prediction purposes. Subsequently, I am trying to utilize scipy.optimize package to solve for the minimized solution based on a given constraint. However, I am not sure if I can take the decision tree model as the objective function for the optimization problem. What should be the approach in a situation like this? I have tried linear regression models such as LarsCV in the past and they worked just fine. But in a linear regression model, you can essentially extract the coefficients and interception point from the model.

Arad Haselirad
  • 314
  • 2
  • 13
  • did you find any solution? mind sharing your experience? – nikn8 Jul 11 '20 at 15:16
  • This questions doesnt has a direct coding question and belongs here: https://stats.stackexchange.com/ – PV8 Jul 13 '20 at 05:52
  • @Neel What I found was that I could use the ML model as the objective function but the only optimization method which worked well with this strategy was "COBYLA" in scipy.optimize.minimize. At least this was the case for my specific problem. – Arad Haselirad Jul 13 '20 at 14:00

1 Answers1

2

Yes; a linear regression model is a straightforward linear function of coefficients (one of which is the "intercept" or "bias").

The problem you have now is that a more complex model isn't quite so simple. You need to load the model into an appropriate engine. To "call" the model, you feed that engine the input vector (the cognate of a list of arguments), and wait for the model to return the prediction.

You need to wrap this process in a function call, perhaps one that issues the model load and processing as external system / shell commands, and returns the results to your main program. Some applications are large enough that it makes sense to implement a full-bore data stream with listener and reporter to handle the throughput.

Does that get you moving?

Prune
  • 76,765
  • 14
  • 60
  • 81
  • Oh I see, so for instance the model is called "Model" and then I go like: def objective(x): return Model.predict(x) – Arad Haselirad Feb 22 '19 at 19:30
  • 1
    @Prune Would you please elaborate a bit?? An small example on "You need to load the model into an appropriate engine. To "call" the model, you feed that engine the input vector (the cognate of a list of arguments), and wait for the model to return the prediction." would definitely help? – nikn8 Jul 11 '20 at 15:14
  • Examples are available in the respective tutorial for your chosen model and system. Stack Overflow is not intended to replace those tutorials. If you have a specific problem with your code, please post a new question. – Prune Jul 12 '20 at 00:59
  • @Prune Thank you for your reply. I do understand SO terms n conditions. Upon *extensive* search on the topic, i just landed to [white paper](https://1drv.ms/b/s!AnkvlPMxJ0p5iyLu24Up0h71GWRR?e=gWnHhj) which has theoretical aspects of implementation. If you can share me an **example link/ tutorial link** having practical implementation, that should be more than enough. Also, I will be posting the question with specific problem. – nikn8 Jul 12 '20 at 09:13
  • What model and system have you chosen? Those will drive a search for appropriate examples. Asking for links to off-site resources is specifically off-topic. However, I'll keep an eye out for your focused question. – Prune Jul 12 '20 at 17:09