0

I am able to dynamically train and create my regression model just fine from a string[] of column names. However, when I try to pass in a dynamic object with the same Parameter names as Dictionary Key Pair properties it throw the error:

System.ArgumentOutOfRangeException: 'Could not find input column '<MyColumn>'' Where <MyColumn> is the first parameter that the model is looking for. 

    private static void TestSinglePrediction(MLContext mlContext, dynamic ratingDataSample, int actual)
    {
        ITransformer loadedModel;

        using (var stream = new FileStream(_modelPath, FileMode.Open, FileAccess.Read, FileShare.Read))
        {
            loadedModel = mlContext.Model.Load(stream);
        }

        var predictionFunction = loadedModel.MakePredictionFunction<dynamic, RatingPrediction>(mlContext);

        var prediction = predictionFunction.Predict(ratingDataSample);

        Console.WriteLine($"**********************************************************************");
        Console.WriteLine($"Predicted rating: {prediction.Rating:0.####}, actual rating: {actual}");
        Console.WriteLine($"**********************************************************************");
    }

I suspect this is because the dynamic object doesn't contain the [Column] attributes that the standard class object I normally would pass in has.

However, I will eventually have hundreds of columns that are auto generated from transposing SQL queries so manually typing each column isn't a feasible approach for the future.

Is there any way I could perhaps apply the attribute at run time? Or any other way I can generically approach this situation? Thanks!

Reed
  • 1,515
  • 1
  • 21
  • 38
  • I filed an issue on ML.NET to allow this runtime-typed prediction: https://github.com/dotnet/machinelearning/issues/1907 . I don't think we can make it work via `dynamic` though: it'll be a different mechanism, slightly harder to implement – Zruty Dec 19 '18 at 17:16
  • This can be done if you use Roslyn to create the class at runtime. See my answer to another question for a solution to this issue: https://stackoverflow.com/questions/66893993/ml-net-create-prediction-engine-using-dynamic-class/66913705#66913705 – Michael Silver Apr 02 '21 at 02:23

1 Answers1

1

This is a great question. The dynamic objects don't work at runtime because ML.NET needs something called a SchemaDefinition for the objects that you pass in so that it knowns where to get the columns it expects.

The simplest way to solve your problem would be to define an object holding only the columns you need at scoring-time, annotated with Column attributes, and manually cast your dynamic object at runtime. This has the main advantage that since you do the casting to the scoring object yourself, you can handle missing data cases yourself without the ML.NET runtime throwing. While your SQL query may give you a large assortment of columns, you won't need the majority of these columns for scoring your model, and therefor don't need to account for them in the scoring object; you only have to account for the columns the model expects.

See this sample from the ML.NET Cookbook for an example of how to score a single row. Behind the scenes, ML.NET is taking the class you defined, and using attributes like Column to construct the SchemaDefinition.

Rogan
  • 21
  • 2