Using ML.net with an ONNX model and GPU

Question

How do you get Microsof.tML to run on an NVDIA GPU using C#? I followed the instructions

https://github.com/dotnet/machinelearning/blob/master/docs/api-reference/tensorflow-usage.md

And my pipeline looks like the below. All versions dependencies (onnxruntime.gpu,Microsoft.ML etc) are 1.5.2 so this should be supported but I get the exception DllNotFoundException: Unable to load DLL 'onnxruntime' or one of its dependencies: The specified module could not be found - and yes onnxruntime does appear in the list of installed nuget packages.

Has anyone gotten this to work? The documentation is still pretty sparse and there don't seem to be any working examples online.

var pipeline = context.Transforms.ResizeImages(
    resizing: ImageResizingEstimator.ResizingKind.Fill,
    outputColumnName: "image", imageWidth: ImageSettings.imageWidth,
    imageHeight: ImageSettings.imageHeight,
    inputColumnName: nameof(ModelInput.Image)
)
.Append(context.Transforms.ExtractPixels(outputColumnName: "image"))
.Append(context.Transforms.ApplyOnnxModel(
    modelFile: "./ObjectDetection/tinyyolov2-8.onnx",
    outputColumnName: "grid", 
    inputColumnName: "image", 
    gpuDeviceId:0)
);

score 2 · Answer 1 · answered Oct 19 '20 at 18:14

2

OK - bad form to answer my own question but the link above saying you must use Cuda 10 seems inaccurate. If you are using the ML.net 1.5 nuget releases you need Cuda 10.2.

11 won't work unless you build from source (and who has the time for that).

Hopefully this solves it but if you were going to comment please do as well!

answered Oct 19 '20 at 18:14

hamishkeith

186
2
5

1

Thanks for posting this! StackOverflow actually encourages people to answer their own questions if you find the answer. Do you remember which version of cuDNN you used? – Ian Newson Jun 16 '21 at 14:20
1

cudnn_ops_infer64_8.dll and cudnn_cnn_infer64_8.dll – hamishkeith Jun 21 '21 at 01:49

score 0 · Answer 2 · answered May 02 '22 at 18:08

Here is the compatibility chart. Newer versions have different compatibilities.

The CUDA Execution Provider enables hardware accelerated computation on Nvidia CUDA-enabled GPUs:

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html

Using ML.net with an ONNX model and GPU

2 Answers2