2

For a project we use WinML to do inference using a fully convolutional network. We query all adapters on the platform and explicitly pass an d3d12 device to the Learning session. For performance reasons we converted the weights of to half float format, and we found out that some GPU doesn't support it. There are actually 2 cases when creating the learning session :

  • For some GPU (Intel HD 4600) WinML is throwing an exception saying that the device doesn't support half float format. For me this isn't clear what is required, as far as I understand WinML doesn't enforce usage of the special ALU found in Nvidia Pascal and later and AMD Polaris and later, so I suspect this support only concerns texture or buffer format. And as far as I know r16f texture format is supported on every dx11 gpu, so this is not clear what is the exact requirement here, and if there is the way to query it beforhand. Currently we try to recreate session with another available device and fallback to cpu if there is none but this isn't really a clean way to proceed.
  • For some other GPU (Nvidia Kepler generation) there is an "unknow exception" that occurs. It doesn't explictly say that half float format is not supported, and I have no idea what's happening.

Any help/insight would be appreciated. Regards, Vincent

Valentin Briukhanov
  • 1,263
  • 9
  • 13
Vincent L
  • 21
  • 2

1 Answers1

1

The Windows Machine Learning implementation uses a variety of checks to determine if a particular GPU supports Float16.

Take a look at the implementation here: https://github.com/microsoft/onnxruntime/blob/bfa996b5fa1ce87d2501abd8371ec5612d288cac/winml/lib/Common/CommonDeviceHelpers.cpp#L140

It contains a combination of a block list:

bool CheckAdapterFP16Blocked(bool isMcdmAdapter, uint32_t vendorId, uint32_t majorVersion, uint32_t minorVersion) {
  switch (vendorId) {
    case c_intelVendorId: {
      if (isMcdmAdapter) {
        return false;
      }

      // Check Intel GPU driver version
      return (majorVersion < 25) || (majorVersion == 25 && minorVersion < 6574) || (majorVersion == 26 && minorVersion < 6572);
    }
  }
  return false;
}

As well as a query to the DML device to indicate whether the underlying hardware supports float16.

 winrt::com_ptr<IDMLDevice> dmlDevice;
  winrt::check_hresult(DMLCreateDevice(
      device,
      DML_CREATE_DEVICE_FLAG_NONE,
      IID_PPV_ARGS(dmlDevice.put())));

  DML_FEATURE_QUERY_TENSOR_DATA_TYPE_SUPPORT float16Query = {DML_TENSOR_DATA_TYPE_FLOAT16};
  DML_FEATURE_DATA_TENSOR_DATA_TYPE_SUPPORT float16Data = {};

  winrt::check_hresult(dmlDevice->CheckFeatureSupport(
      DML_FEATURE_TENSOR_DATA_TYPE_SUPPORT,
      sizeof(float16Query),
      &float16Query,
      sizeof(float16Data),
      &float16Data));
  return float16Data.IsSupported;
Kookei
  • 168
  • 7